Compare commits

..

164 Commits

Author SHA1 Message Date
CJACK.
31e64ff31d Merge pull request #272 from CJackHwang/dev
Remove outdated architecture documentation and improve citation parsing
2026-04-20 18:38:26 +08:00
CJACK.
5984802df4 Merge pull request #273 from CJackHwang/codex/fix-citation-index-normalization-issue
Fix zero-based citation index normalization
2026-04-20 18:35:46 +08:00
CJACK.
e0ed4ba238 Handle one-based and zero-based citation indices safely 2026-04-20 18:29:58 +08:00
CJACK.
ae37654893 Fix zero-based citation index normalization 2026-04-20 18:18:00 +08:00
CJACK.
aa7f821151 Bump version from 3.5.0 to 3.5.1 2026-04-20 17:32:05 +08:00
CJACK.
f7426f9f04 Remove detailed backend architecture explanations
Removed detailed descriptions of routing, execution, adapter layers, tool calling, configuration, streaming capabilities, and observability enhancements.
2026-04-20 17:18:37 +08:00
CJACK.
787e034174 Merge pull request #271 from livesRan/citation注释解析
/v1/chat/completions 接口返回报文中出现了[citation:1][citation:2]等未解析的标签,本次改动将返…
2026-04-20 13:06:38 +08:00
songguoliang
d73f7b8b73 /v1/chat/completions 接口返回报文中出现了[citation:1][citation:2]等未解析的标签,本次改动将返回报文中的标签做了解析 2026-04-20 11:22:31 +08:00
CJACK
b8d844e2f6 docs: remove outdated 3.X architecture documentation from README files 2026-04-20 01:44:58 +08:00
CJACK.
2ba8b143d0 Merge pull request #268 from CJackHwang/dev
chore: bump version to 3.5.0
2026-04-20 01:26:09 +08:00
CJACK
70603a5a90 chore: bump version to 3.5.0 2026-04-20 01:24:31 +08:00
CJACK.
fa51aafdc5 Merge pull request #265 from CJackHwang/dev
refactor: enforce mandatory CDATA wrapping for all string parameters in tool call XML output

## XML工具调用解析与代码围栏感知
- **`f313d00`** – 在工具筛选中增加代码围栏感知能力,防止代码块内的XML工具调用被误判,并优化了提示词指令。
- **`69eb711`** – 扩展工具调用解析器,支持变长Markdown围栏(如 ```` ``` ````)。
- **`5b7cdaa`** – 修复了被Markdown围栏包裹的XML工具调用解析问题。

## 系统提示与思考模式
- **`10d681f`** – 开启思考模式时,向系统提示词中注入对话连贯性与推理指令。

## API与文档对齐
- **`08f32c4`** – 使API文档与当前已实现的路由保持一致。
- **`0e7f5cd`** – 同步工具调用语义文档与当前实现。
- **`2c08375`** – 将模型别名示例刷新为当前默认值。

## 代码质量与强制规范
- **`69b7bc0`** – 强制要求工具调用XML输出中所有字符串参数必须使用CDATA包裹,提升鲁棒性。

## 合并请求
- **`12256ce`** – 合并PR #266:文档准确性更新。
- **`fa38934`** – 合并PR #267:XML解析修复。
2026-04-20 01:20:11 +08:00
CJACK
10d681ffe7 feat: inject conversation continuity and reasoning instructions into system prompt when thinking is enabled 2026-04-20 00:47:05 +08:00
CJACK
f313d0068f feat: implement code fence awareness in tool sieve to prevent false-positive XML tool detection inside code blocks and refine prompt instructions. 2026-04-20 00:13:14 +08:00
CJACK.
12256ceb24 Merge pull request #266 from CJackHwang/codex/update-documentation-for-accuracy
docs: align API docs with implemented routes and limits
2026-04-19 23:43:28 +08:00
CJACK.
2c08375b49 docs: refresh model alias examples to current defaults 2026-04-19 23:42:34 +08:00
CJACK.
fa38934114 Merge pull request #267 from CJackHwang/codex/fix-xml-parsing-for-tool-calls
Strip fenced code blocks before XML tool-call parsing to avoid executing examples
2026-04-19 23:40:26 +08:00
CJACK.
69eb71159d Handle variable-length markdown fences in toolcall parser 2026-04-19 23:37:31 +08:00
CJACK.
0e7f5cdc86 docs: sync tool-calling semantics with current implementation 2026-04-19 23:12:13 +08:00
CJACK.
5b7cdaa729 Fix XML tool-call parsing for fenced markdown examples 2026-04-19 23:11:24 +08:00
CJACK.
08f32c4c40 docs: align API docs with implemented routes 2026-04-19 21:04:06 +08:00
CJACK
69b7bc0c1a refactor: enforce mandatory CDATA wrapping for all string parameters in tool call XML output 2026-04-19 20:11:53 +08:00
CJACK
0f2b5fee23 refactor: enhance XML tool call parsing to support nested structures, CDATA, and repeated tags 2026-04-19 19:58:45 +08:00
CJACK
26d195f2a6 refactor: update tool call format to prefer XML-style parameters with CDATA support for robust content handling 2026-04-19 18:51:25 +08:00
CJACK
790a8ca980 refactor: implement robust think tag stripping and CDATA handling for SSE stream parsing 2026-04-19 18:35:56 +08:00
CJACK
a1ce954ad5 refactor: implement auto-transition from thinking to text content upon detecting </think> tags and remove unused helper functions 2026-04-19 18:05:38 +08:00
CJACK
6688e0ba35 refactor: remove unnecessary whitespace and end-of-sentence markers to align with official DeepSeek chat template encoding 2026-04-19 17:47:45 +08:00
CJACK
c945f49fc4 refactor: remove JSON-based tool call parsing from sieve and delete associated compatibility tests 2026-04-19 13:39:47 +08:00
CJACK
0c644d1f4d refactor: remove legacy function call support and simplify tool sieve logic 2026-04-19 04:38:48 +08:00
CJACK.
146d59e7bf Merge pull request #263 from utafrali/fix/issue-261-bug
fix: Increase account page size limit to 5000
2026-04-18 12:49:08 +08:00
ugurtafrali
daf3307b88 fix: Increase account page size limit to 5000 2026-04-18 05:16:57 +03:00
CJACK.
67501cf4d2 Merge pull request #256 from CJackHwang/dev
全模型全渠道附件上传deepseek功能
全接口兼容性待测试
2026-04-13 04:00:49 +08:00
CJACK
25234af301 feat: enforce request body size limits and restrict inline file count to prevent resource exhaustion 2026-04-13 03:55:14 +08:00
CJACK
2aee80d0d3 fix: update URL decoding method and refine file ID extraction logic to exclude text-based inputs 2026-04-13 03:49:06 +08:00
CJACK
ab9f3cc417 refactor: remove unused leakedDanglingThinkOpenPattern regex from output sanitizer 2026-04-13 03:40:20 +08:00
CJACK
c92ed8d3c3 refactor: rename apiTester testSuccess key to requestSuccess and update localization files 2026-04-13 03:24:39 +08:00
CJACK
d78789a66e feat: implement error handling for empty upstream responses in chat streams and update UI to display stream-level errors 2026-04-13 03:22:38 +08:00
CJACK
acb110865f feat: implement cross-account validation and improved error handling for file attachments in API tester 2026-04-13 03:15:12 +08:00
CJACK
ffca8be597 feat: implement file readiness polling and add IsImage field to upload results 2026-04-13 02:55:45 +08:00
CJACK
7ef6a7d11f feat: update to v3.4.0 and redesign model selection UI with a dropdown and descriptive panel 2026-04-13 02:27:12 +08:00
CJACK
d53a2ea7d2 refactor: remove unused purpose parameter from upload and upstream empty output handlers 2026-04-13 01:59:51 +08:00
CJACK
daa636e040 refactor: handle upstream thinking-only responses as errors and sanitize dangling think tags in output 2026-04-13 01:55:14 +08:00
CJACK
aa41bae044 feat: add file attachment support to chat interface and API requests 2026-04-13 00:04:38 +08:00
CJACK
2027c7cd77 fix: add JSON headers to DeepSeek requests and prevent string content from being parsed as file IDs in OpenAI adapter 2026-04-12 23:49:56 +08:00
CJACK
0591128601 refactor: fix file handling error suppression, optimize hash calculation, and update API documentation with additional models 2026-04-12 23:35:57 +08:00
CJACK
caafdedb00 feat: implement OpenAI-compatible file upload and reference handling for DeepSeek API 2026-04-12 23:30:22 +08:00
CJACK
0a23c77ff7 feat: add sanitization for think tags and BOS markers in leaked output and update golang.org/x/net dependency 2026-04-12 17:43:57 +08:00
CJACK.
d759804c33 Merge pull request #255 from CJackHwang/codex/refactor-prompt-concatenation-using-tokenizer
feat(prompt): tokenizer-style prompt stitching with thinking-prefix support
2026-04-12 17:14:48 +08:00
CJACK.
433a3a877d feat(prompt): align DeepSeek prompt assembly with tokenizer-style turns 2026-04-12 13:59:42 +08:00
CJACK.
792e295512 Merge pull request #254 from CJackHwang/main
Update VERSION
2026-04-08 20:24:03 +08:00
CJACK.
d053d9ad04 Update VERSION 2026-04-08 20:22:55 +08:00
CJACK.
04e025c5e1 Update README.MD 2026-04-08 18:21:09 +08:00
CJACK.
184cbed3cb Merge pull request #252 from CJackHwang/dev
Merge pull request #249 from shuaihaoV/feat/deepseek-model-type-families

Add default, expert, and vision DeepSeek model families
2026-04-08 18:06:07 +08:00
CJACK.
378f99be4a Merge pull request #249 from shuaihaoV/feat/deepseek-model-type-families
Add default, expert, and vision DeepSeek model families
2026-04-08 17:53:02 +08:00
Shuaihao
ba76a2163b Add default, expert, and vision DeepSeek model families 2026-04-08 14:37:22 +08:00
CJACK.
af9c51f3a7 Merge pull request #245 from CJackHwang/dev
Merge pull request #244 from CJackHwang/codex/temporarily-switch-to-internal-usage-count

Temporarily ignore DeepSeek upstream usage fields and prefer internal token estimation
2026-04-07 21:27:32 +08:00
CJACK.
92bb25265e Merge pull request #246 from CJackHwang/codex/fix-review-comments-before-merging
Fix proxy-bound fallback behavior and redact proxy password responses
2026-04-07 21:26:13 +08:00
CJACK.
84050d87e4 fix proxy fallback binding and redact proxy password responses 2026-04-07 21:22:28 +08:00
CJACK.
c6a6f1cf4e Merge pull request #244 from CJackHwang/codex/temporarily-switch-to-internal-usage-count
Temporarily ignore DeepSeek upstream usage fields and prefer internal token estimation
2026-04-07 20:39:36 +08:00
CJACK.
f4ed10d38d disable token-mismatch gate by default in raw stream simulator 2026-04-07 20:38:29 +08:00
CJACK.
d9e65c9710 remove upstream token-usage plumbing and always estimate from content 2026-04-07 20:12:18 +08:00
CJACK.
a14e5b0847 temporarily ignore upstream token usage fields globally 2026-04-07 19:40:47 +08:00
CJACK.
b59e991ad5 Merge pull request #241 from tanaer/feat/proxy-ip-management-dev
feat: 增加 SOCKS5/SOCKS5H 代理管理与账号代理路由
2026-04-07 17:14:48 +08:00
Jason.li
c84347b625 docs: align agent rules with quality gate lint 2026-04-07 14:19:40 +08:00
Jason.li
8ae2ea10c8 feat(proxy): add proxy IP management and account routing
Add admin CRUD and connectivity checks for SOCKS5/SOCKS5H proxy nodes.

Allow accounts to bind to a proxy, route DeepSeek requests through the selected node, and expose proxy management in the admin UI.
2026-04-07 14:16:13 +08:00
CJACK.
d32765bc84 Merge pull request #240 from CJackHwang/dev
Merge pull request #239 from CJackHwang/codex/fix-escaping-issues-and-token-counting

Fix HTML-escaped tool-call args and preserve upstream token usage (stream & non-stream)
2026-04-07 13:16:49 +08:00
CJACK.
08b1344f81 Merge pull request #242 from CJackHwang/codex/fix-issues-in-pull-request-#240
fix: avoid double-decoding XML entity text in markup tool-call parsing
2026-04-07 13:16:01 +08:00
CJACK.
8b0da7b6f8 fix: avoid double XML entity decoding in toolcall parser 2026-04-07 13:14:30 +08:00
CJACK.
1c95942e5d Merge pull request #239 from CJackHwang/codex/fix-escaping-issues-and-token-counting
Fix HTML-escaped tool-call args and preserve upstream token usage (stream & non-stream)
2026-04-07 12:56:02 +08:00
CJACK.
da7c46b278 Limit HTML unescape to markup tool-call parsing 2026-04-07 12:55:06 +08:00
CJACK.
cfcca69396 Update VERSION 2026-04-07 12:46:15 +08:00
CJACK.
4475bfe92f Merge pull request #238 from CJackHwang/codex/remove-project-structure-section-from-main-document
docs: remove duplicated project structure sections from READMEs
2026-04-07 12:36:30 +08:00
CJACK.
77a401fb19 Fix tool-call HTML escaping and stabilize usage token mapping 2026-04-07 12:35:50 +08:00
CJACK.
a935f61f74 docs: remove duplicated project structure sections from READMEs 2026-04-07 12:32:52 +08:00
CJACK.
80b88b37ff Merge pull request #236 from CJackHwang/codex/review-and-reorganize-all-md-documents
docs: add architecture docs and centralize documentation index; update READMEs and API links
2026-04-07 11:55:11 +08:00
CJACK.
475c9086d2 docs: 为展开目录树补充文件夹作用注释 2026-04-07 11:51:14 +08:00
CJACK.
8cfba9c650 Merge pull request #232 from CJackHwang/dev
refactor: improve XML tool parsing robustness, update system prompt constraints, and simplify tool filtering logic
2026-04-07 11:13:44 +08:00
CJACK.
98131881ed Merge pull request #234 from CJackHwang/codex/fix-documentation-and-accumulated_token_usage
Propagate DeepSeek SSE token usage to /v1/responses and remove stale POW env docs
2026-04-07 11:02:44 +08:00
CJACK.
86ecbc89bd Preserve SSE frame delimiters when injecting Gemini usage 2026-04-07 10:59:27 +08:00
CJACK.
668b9c26bd Unify token usage pass-through on OpenAI translate pipeline 2026-04-07 10:16:23 +08:00
CJACK.
5bcea3d727 Propagate upstream token usage across Gemini usage metadata 2026-04-07 10:16:00 +08:00
CJACK.
96b8587c5b Fix token usage propagation and remove stale env docs 2026-04-07 08:27:03 +08:00
CJACK.
d09260d06f Merge pull request #233 from CJackHwang/main
依赖升级
2026-04-07 07:12:40 +08:00
CJACK.
554b95d232 Merge pull request #231 from CJackHwang/dependabot/npm_and_yarn/webui/npm_and_yarn-7c6ac41456
chore(deps-dev): bump vite from 8.0.3 to 8.0.5 in /webui in the npm_and_yarn group across 1 directory
2026-04-07 07:02:53 +08:00
dependabot[bot]
b54ee05d12 chore(deps-dev): bump vite
Bumps the npm_and_yarn group with 1 update in the /webui directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 8.0.3 to 8.0.5
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.0.5
  dependency-type: direct:development
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-06 18:44:30 +00:00
CJACK
9968221633 refactor: improve XML tool parsing robustness, update system prompt constraints, and simplify tool filtering logic 2026-04-07 02:10:45 +08:00
CJACK
b79a13efd5 feat: support explicit prompt token tracking in SSE parsing and stream handlers 2026-04-07 01:39:27 +08:00
CJACK
da778a18fb refactor: replace WASM-based PoW with a high-performance native Go implementation and add context support for cancellation. 2026-04-07 01:20:01 +08:00
CJACK.
10921e0f84 Merge pull request #229 from CJackHwang/dev
refactor: replace WASM-based PoW solver with a native Go implementation in the pow package
2026-04-07 00:57:33 +08:00
CJACK
e7d561694a refactor: replace WASM-based PoW solver with a native Go implementation in the pow package 2026-04-07 00:10:36 +08:00
CJACK.
13687ce787 Merge pull request #227 from CJackHwang/codex/change-empty-responses-to-429
fix(openai): return 429 for empty upstream output
2026-04-06 17:11:07 +08:00
CJACK.
26aa02d4b5 fix(openai): return 429 for empty upstream output 2026-04-06 16:56:17 +08:00
CJACK.
89eaf048c3 Merge pull request #221 from CJackHwang/dev
Dev
2026-04-06 16:50:00 +08:00
CJACK.
904211469a Merge pull request #222 from CJackHwang/codex/resolve-pull-request-issues-and-complete-tests
Add golangci-lint bootstrap and CI lint gate; update docs and .gitignore
2026-04-06 13:56:05 +08:00
CJACK.
530872ff2f Merge pull request #224 from CJackHwang/codex/fix-four-issues-from-pull-requests-99aozg
Fix lint bootstrap behavior and harden SSE token replay parsing
2026-04-06 13:55:30 +08:00
CJACK.
fbe1e25c7b Merge pull request #225 from CJackHwang/codex/fix-golangci-lint-bootstrap-compatibility
Treat missing golangci-lint as bootstrap-compatible
2026-04-06 13:54:50 +08:00
CJACK.
cd7e03d936 Merge pull request #226 from CJackHwang/codex/fix-issue-with-passing-thresholds
Handle deferred Close errors, normalize error messages, and add nolint annotations
2026-04-06 13:54:14 +08:00
CJACK.
37fb758191 Make full quality gates pass across repository 2026-04-06 13:41:58 +08:00
CJACK.
fb6be8a8ee fix lint bootstrap on missing golangci-lint 2026-04-06 13:38:17 +08:00
CJACK.
57114a36f5 fix: address codex review issues for lint bootstrap and token replay 2026-04-06 13:12:36 +08:00
CJACK.
a671d82759 chore: auto-bootstrap golangci-lint for Go 1.26 compatibility 2026-04-06 12:52:56 +08:00
CJACK.
da75ed6966 Merge pull request #220 from CJackHwang/codex/fix-pull-request-review-comments
Migrate and reorganize .golangci.yml to v2 with updated linters and exclusions
2026-04-06 12:33:51 +08:00
CJACK.
3b99d2edbe docs: add full-sample token replay command and report fields 2026-04-06 12:32:31 +08:00
CJACK.
f6c09ebd63 fix: keep node error-branch token semantics and add grep fallback 2026-04-06 12:32:26 +08:00
CJACK.
36af2e00f6 Merge pull request #219 from CJackHwang/dev
Dev
2026-04-06 11:17:39 +08:00
CJACK.
9e0fd83a76 test: validate raw stream token replay and enforce gofmt in lint script 2026-04-06 11:15:08 +08:00
CJACK.
a8c160b05d fix: parse DeepSeek accumulated_token_usage robustly and stabilize lint 2026-04-06 11:14:48 +08:00
CJACK.
89ca57122c fix: migrate golangci config to v2 schema 2026-04-06 09:29:22 +08:00
CJACK
6b6ce3eea8 refactor: move toolcall utility files to internal/toolcall directory 2026-04-06 03:56:42 +08:00
CJACK
870144de17 ci: remove golangci-lint step from quality gates workflow 2026-04-06 03:53:03 +08:00
CJACK
1530246e4f refactor: move tool call parsing and formatting logic to a dedicated internal/toolcall package 2026-04-06 03:19:18 +08:00
CJACK.
d6ecdad6de Merge pull request #218 from CJackHwang/dev
fix: reverse snapshot order to preserve capture sequence during stabl…
2026-04-06 02:55:59 +08:00
CJACK
2857a171cc fix: reverse snapshot order to preserve capture sequence during stable sort 2026-04-06 02:51:06 +08:00
CJACK.
eb8b45e667 Merge pull request #217 from CJackHwang/dev
Dev
2026-04-06 02:47:44 +08:00
CJACK
1664349a29 docs: update documentation for raw stream test samples 2026-04-06 02:44:20 +08:00
CJACK
b105d54c00 feat: add admin endpoints for capturing, querying, and persisting raw upstream samples and increase default capture limits 2026-04-06 02:38:15 +08:00
CJACK
039d7d3db1 feat: implement raw sample capture querying and persistence, and add environment-based configuration for dev capture store. 2026-04-06 02:33:02 +08:00
CJACK
49012a227c feat: implement trimContinuationOverlap utility to remove redundant stream prefixes and add associated tests. 2026-04-06 02:23:28 +08:00
CJACK
4d36afea4c 修复接续流的增量bug 2026-04-06 02:01:41 +08:00
CJACK.
3b29cf3da4 Merge pull request #215 from CJackHwang/dev
fix: ignore INCOMPLETE status messages in SSE stream parsing to prevent stream interruption
2026-04-06 00:31:18 +08:00
CJACK
a608a4bd95 chore(webui): migrate vite to v8 toolchain 2026-04-06 00:28:36 +08:00
CJACK
d9eee5fd2d docs: clarify server binding address and LAN accessibility in documentation and startup logs 2026-04-06 00:13:22 +08:00
CJACK
bac7345309 chore: remove unused file and associated references 2026-04-06 00:00:30 +08:00
CJACK
b699077b43 fix: make FINISHED status check case-insensitive in chat stream parsing 2026-04-05 23:56:19 +08:00
CJACK
49430123d8 fix: ignore INCOMPLETE status messages in SSE stream parsing to prevent stream interruption 2026-04-05 23:38:47 +08:00
CJACK.
6b5af06c19 Merge pull request #214 from CJackHwang/dev
Dev
2026-04-05 23:06:05 +08:00
CJACK
84813eca80 feat: support multi-round upstream captures in raw sample generation 2026-04-05 22:48:41 +08:00
CJACK
2a6b787f38 feat: implement mandatory DeepSeek turn termination markers for system, user, assistant, and tool roles 2026-04-05 21:48:18 +08:00
CJACK
a28c9fb67f feat: implement comprehensive configuration validation and integrate into store loading and server initialization. 2026-04-05 21:18:51 +08:00
CJACK
585d35e592 refactor: standardize prompt markers and remove legacy EOS and system instructions tags 2026-04-05 20:50:12 +08:00
CJACK
dc912419c4 Revert "refactor: rename tool XML wrapper from tool_calls to tool_batch and add schema attention blocks to tool prompts"
This reverts commit ade648033d.
2026-04-05 20:08:04 +08:00
CJACK
ade648033d refactor: rename tool XML wrapper from tool_calls to tool_batch and add schema attention blocks to tool prompts 2026-04-05 19:22:43 +08:00
CJACK
b8e9ca2028 refactor: stop stripping _raw and _xml fields from tool call inputs to preserve raw parameter data 2026-04-05 18:33:15 +08:00
CJACK
f2ad888de4 refactor: clean up config schema by removing legacy toolcall fields, standardizing auto_delete mode, and updating admin API documentation. 2026-04-05 18:16:31 +08:00
CJACK
e37ed38dc8 refactor: remove instruction regarding text-before-tool output in tool prompt documentation 2026-04-05 17:51:26 +08:00
CJACK
5d59775051 refactor: enforce strict XML-only output for tool calls and remove mixed-content instructions 2026-04-05 17:25:52 +08:00
CJACK
22efd8178b refactor: clarify tool call XML placement and add strict rules against result fabrication and internal narration 2026-04-05 17:13:59 +08:00
CJACK
088a750338 feat: add strict schema enforcement and strip hallucinated fields from tool call parameters 2026-04-05 16:59:39 +08:00
CJACK
298a6f27cc refactor: extract SSE parsing and Vercel stream logic into dedicated implementation modules 2026-04-05 16:32:13 +08:00
CJACK
1d80f644d4 refactor: remove legacy environment variable aliases for configuration and concurrency settings 2026-04-05 16:27:58 +08:00
CJACK
0b0cf60982 feat: propagate Proof-of-Work header to auto-continue requests and ensure remote session deletion ignores parent context cancellation 2026-04-05 14:33:09 +08:00
CJACK
32b9cbb61f feat: implement automatic completion continuation for incomplete DeepSeek responses 2026-04-05 13:59:39 +08:00
CJACK
eff979d9ef feat: enhance content filtering, token usage tracking, and stream error handling in chat-stream modules 2026-04-05 13:41:38 +08:00
CJACK
47dc121690 refactor: improve prompt construction by enforcing explicit newline boundaries between role markers and message content 2026-04-05 04:44:46 +08:00
CJACK
b94a16eca9 style: update AutoDeleteSection layout spacing and input styling 2026-04-05 04:37:16 +08:00
CJACK
97e72fb174 feat: add configurable auto-delete modes (none, single, all) for remote chat sessions 2026-04-05 04:18:34 +08:00
CJACK
f7261bec0d docs: remove obsolete raw stream samples and update testing documentation 2026-04-05 03:07:10 +08:00
CJACK
0bebb4b28d refactor: replace processed output comparison with baseline-based validation in SSE simulator 2026-04-05 01:34:06 +08:00
CJACK
93879c9808 feat: implement rawsample package for automated capture persistence and add admin handlers for sample management 2026-04-05 01:12:31 +08:00
CJACK
c9201174f6 feat: add compatibility setting to strip reference markers from model output and update stream handlers accordingly 2026-04-05 00:50:30 +08:00
CJACK
a6836455dc feat: add support for stripping inline comments in .env files and make Docker host port configurable via DS2API_HOST_PORT 2026-04-04 22:30:57 +08:00
CJACK
eab197f4d9 chore: upgrade Go to 1.26.x and update project dependencies 2026-04-04 22:00:18 +08:00
CJACK.
c8b6dfc290 Merge pull request #213 from Kazakiri220/main
Load .env and config.json for local go runs
2026-04-04 21:44:36 +08:00
Kazakiri220
541bd67c7c Load .env and config.json for local go runs 2026-04-04 20:21:17 +08:00
CJACK.
068f4b0df6 Merge pull request #211 from CJackHwang/dev
Merge pull request #209 from CJackHwang/codex/investigate-command-output-loss-issue

Preserve trailing newlines when stripping leaked CONTENT_FILTER and strengthen XML tool-call parsing
2026-04-04 02:20:39 +08:00
CJACK.
5a51045ba4 Merge pull request #212 from CJackHwang/codex/address-handling-of-upstream-vacuum-returns
fix: classify empty upstream output correctly and harden XML tool-name parsing
2026-04-04 02:20:01 +08:00
CJACK.
3497d5d019 fix: classify empty upstream and tighten xml tool-name parsing 2026-04-04 02:14:39 +08:00
CJACK.
95a9d16843 Merge pull request #209 from CJackHwang/codex/investigate-command-output-loss-issue
Preserve trailing newlines when stripping leaked CONTENT_FILTER and strengthen XML tool-call parsing
2026-04-04 01:53:39 +08:00
CJACK.
0847091864 fix: avoid taking generic <name> as xml tool name 2026-04-04 01:52:57 +08:00
CJACK.
c6340354ec fix: keep execute_command args from xml parameters blocks 2026-04-04 01:42:31 +08:00
CJACK.
6bf08e00cd Update VERSION 2026-04-03 23:38:34 +08:00
CJACK.
35221002d5 Merge pull request #207 from CJackHwang/codex/svg
Fix XML passthrough in tool sieve and return 429 on empty upstream output
2026-04-03 23:37:27 +08:00
CJACK.
4b1f1ea550 Preserve suffix after non-tool XML passthrough 2026-04-03 23:36:28 +08:00
CJACK.
0258f83d10 Fix XML passthrough and empty-upstream handling 2026-04-03 22:31:15 +08:00
321 changed files with 88658 additions and 9033 deletions

View File

@@ -1,5 +1,8 @@
# DS2API runtime
# Runtime listen port inside the app/container
PORT=5001
# Docker Compose host port (compose only; container still listens on PORT)
DS2API_HOST_PORT=6011
LOG_LEVEL=INFO
# Admin authentication

View File

@@ -19,7 +19,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: "1.24.x"
go-version: "1.26.x"
- name: Setup Node
uses: actions/setup-node@v4
@@ -28,6 +28,16 @@ jobs:
cache: "npm"
cache-dependency-path: webui/package-lock.json
- name: Setup golangci-lint
uses: golangci/golangci-lint-action@v8
with:
version: v2.11.4
install-mode: binary
verify: true
- name: Go Format & Lint Gates
run: ./scripts/lint.sh
- name: Refactor Line Gate
run: ./tests/scripts/check-refactor-line-gate.sh

View File

@@ -27,7 +27,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: "1.24.x"
go-version: "1.26.x"
- name: Setup Node
uses: actions/setup-node@v4
@@ -79,7 +79,7 @@ jobs:
CGO_ENABLED=0 GOOS="${GOOS}" GOARCH="${GOARCH}" \
go build -trimpath -ldflags="-s -w -X ds2api/internal/version.BuildVersion=${BUILD_VERSION}" -o "${STAGE}/${BIN}" ./cmd/ds2api
cp config.example.json .env.example internal/deepseek/assets/sha3_wasm_bg.7b9ca65ddd.wasm LICENSE README.MD README.en.md "${STAGE}/"
cp config.example.json .env.example LICENSE README.MD README.en.md "${STAGE}/"
cp -R static/admin "${STAGE}/static/admin"
if [ "${GOOS}" = "windows" ]; then

4
.gitignore vendored
View File

@@ -9,6 +9,7 @@ config.json
*.swo
*~
.DS_Store
opencode.json
# Logs
*.log
@@ -58,3 +59,6 @@ Thumbs.db
# Claude Code
.claude/
CLAUDE.local.md
# Local tool bootstrap cache
.tmp/

73
.golangci.yml Normal file
View File

@@ -0,0 +1,73 @@
version: "2"
run:
tests: true
linters:
default: standard
enable:
- errcheck
- govet
- ineffassign
- staticcheck
- unused
settings:
dupl:
threshold: 100
goconst:
min-len: 2
min-occurrences: 2
gocritic:
enabled-tags:
- diagnostic
- experimental
- opinionated
- performance
- style
disabled-checks:
- wrapperFunc
- rangeValCopy
- hugeParam
gocyclo:
min-complexity: 15
lll:
line-length: 140
misspell:
locale: US
nakedret:
max-func-lines: 30
prealloc:
simple: true
range-loops: true
for-loops: false
exclusions:
generated: lax
rules:
- path: (.+)\.go$
text: "ST1000: at least one file in a package should have a package comment"
paths:
- third_party$
- builtin$
- examples$
- vendor$
- webui/node_modules$
issues:
max-issues-per-linter: 0
max-same-issues: 0
formatters:
enable:
- gofmt
settings:
goimports:
local-prefixes:
- ds2api
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
- vendor$
- webui/node_modules$

23
AGENTS.md Normal file
View File

@@ -0,0 +1,23 @@
# AGENTS.md
These rules apply to all agent-made changes in this repository.
## PR Gate
- Before opening or updating a PR, run the same local gates as `.github/workflows/quality-gates.yml`.
- Required commands:
- `./scripts/lint.sh`
- `./tests/scripts/check-refactor-line-gate.sh`
- `./tests/scripts/run-unit-all.sh`
- `npm run build --prefix webui`
## Go Lint Rules
- Run `gofmt -w` on every changed Go file before commit or push.
- Do not ignore error returns from I/O-style cleanup calls such as `Close`, `Flush`, `Sync`, or similar methods.
- If a cleanup error cannot be returned, log it explicitly.
## Change Scope
- Keep changes additive and tightly scoped to the requested feature or bugfix.
- Do not mix unrelated refactors into feature PRs unless they are required to make the change pass gates.

191
API.en.md
View File

@@ -4,6 +4,8 @@ Language: [中文](API.md) | [English](API.en.md)
This document describes the actual behavior of the current Go codebase.
Docs: [Overview](README.en.md) / [Architecture](docs/ARCHITECTURE.en.md) / [Deployment](docs/DEPLOY.en.md) / [Testing](docs/TESTING.md)
---
## Table of Contents
@@ -35,7 +37,7 @@ This document describes the actual behavior of the current Go codebase.
- OpenAI / Claude / Gemini protocols are now mounted on one shared `chi` router tree assembled in `internal/server/router.go`.
- Adapter responsibilities are streamlined to: **request normalization → DeepSeek invocation → protocol-shaped rendering**, reducing legacy split-logic paths.
- Tool-calling semantics are aligned between Go and Node runtime: structured parsing first (JSON/XML/invoke/markup), plus stream-time anti-leak filtering.
- Tool-calling semantics are aligned between Go and Node runtime: parsing is now centered on XML/Markup-family tool syntax (`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml variants), plus stream-time anti-leak filtering.
- `Admin API` separates static config from runtime policy: `/admin/config*` for configuration state, `/admin/settings*` for runtime behavior.
---
@@ -52,8 +54,7 @@ cp config.example.json config.json
Use it per deployment mode:
- Local run: read `config.json` directly
- Docker / Vercel: generate Base64 from `config.json`, then set `DS2API_CONFIG_JSON`
- Compatibility note: `DS2API_CONFIG_JSON` may also contain raw JSON directly; `CONFIG_JSON` is the legacy fallback variable
- Docker / Vercel: generate Base64 from `config.json`, then set `DS2API_CONFIG_JSON`, or paste raw JSON directly
```bash
DS2API_CONFIG_JSON="$(base64 < config.json | tr -d '\n')"
@@ -107,6 +108,7 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
| POST | `/v1/responses` | Business | OpenAI Responses API (stream/non-stream) |
| GET | `/v1/responses/{response_id}` | Business | Query stored response (in-memory TTL) |
| POST | `/v1/embeddings` | Business | OpenAI Embeddings API |
| POST | `/v1/files` | Business | OpenAI Files upload (multipart/form-data) |
| GET | `/anthropic/v1/models` | None | Claude model list |
| POST | `/anthropic/v1/messages` | Business | Claude messages |
| POST | `/anthropic/v1/messages/count_tokens` | Business | Claude token counting |
@@ -130,15 +132,24 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
| GET | `/admin/config/export` | Admin | Export full config (`config`/`json`/`base64`) |
| POST | `/admin/keys` | Admin | Add API key |
| DELETE | `/admin/keys/{key}` | Admin | Delete API key |
| GET | `/admin/proxies` | Admin | List proxies |
| POST | `/admin/proxies` | Admin | Add proxy |
| PUT | `/admin/proxies/{proxyID}` | Admin | Update proxy (empty password keeps old secret) |
| DELETE | `/admin/proxies/{proxyID}` | Admin | Delete proxy (auto-unbind referenced accounts) |
| POST | `/admin/proxies/test` | Admin | Test proxy connectivity |
| GET | `/admin/accounts` | Admin | Paginated account list |
| POST | `/admin/accounts` | Admin | Add account |
| DELETE | `/admin/accounts/{identifier}` | Admin | Delete account |
| PUT | `/admin/accounts/{identifier}/proxy` | Admin | Bind/unbind proxy for an account |
| GET | `/admin/queue/status` | Admin | Account queue status |
| POST | `/admin/accounts/test` | Admin | Test one account |
| POST | `/admin/accounts/test-all` | Admin | Test all accounts |
| POST | `/admin/accounts/sessions/delete-all` | Admin | Delete all sessions for one account |
| POST | `/admin/import` | Admin | Batch import keys/accounts |
| POST | `/admin/test` | Admin | Test API through service |
| POST | `/admin/dev/raw-samples/capture` | Admin | Fire one request and persist it as a raw sample |
| GET | `/admin/dev/raw-samples/query` | Admin | Search current in-memory capture chains by prompt keyword |
| POST | `/admin/dev/raw-samples/save` | Admin | Persist a selected in-memory capture chain as a raw sample |
| POST | `/admin/vercel/sync` | Admin | Sync config to Vercel |
| GET | `/admin/vercel/status` | Admin | Vercel sync status |
| POST | `/admin/vercel/status` | Admin | Vercel sync status / draft compare |
@@ -169,7 +180,7 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
### `GET /v1/models`
No auth required. Returns supported models.
No auth required. Returns the currently supported DeepSeek native model list.
**Response**:
@@ -180,11 +191,21 @@ No auth required. Returns supported models.
{"id": "deepseek-chat", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-reasoner", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-chat-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
{"id": "deepseek-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-chat", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-reasoner", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-chat-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-chat", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-reasoner", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-chat-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
]
}
```
> Note: `/v1/models` returns normalized DeepSeek native model IDs. Common aliases are accepted only as request input and are not expanded as separate items in this endpoint.
### Model Alias Resolution
For `chat` / `responses` / `embeddings`, DS2API follows a wide-input/strict-output policy:
@@ -194,6 +215,13 @@ For `chat` / `responses` / `embeddings`, DS2API follows a wide-input/strict-outp
3. If still unmatched, fall back by known family heuristics (`o*`, `gpt-*`, `claude-*`, etc.).
4. If still unmatched, return `invalid_request_error`.
Current built-in default aliases (excerpt):
- OpenAI: `gpt-4o`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-5`, `gpt-5-mini`, `gpt-5-codex`
- OpenAI reasoning: `o1`, `o1-mini`, `o3`, `o3-mini`
- Claude: `claude-sonnet-4-5`, `claude-haiku-4-5`, `claude-opus-4-6` (plus compatibility aliases `claude-3-5-sonnet` / `claude-3-5-haiku` / `claude-3-opus`)
- Gemini: `gemini-2.5-pro`, `gemini-2.5-flash`
### `POST /v1/chat/completions`
**Headers**:
@@ -207,7 +235,7 @@ Content-Type: application/json
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `model` | string | ✅ | DeepSeek native models + common aliases (`gpt-4o`, `gpt-5-codex`, `o3`, `claude-sonnet-4-5`, etc.) |
| `model` | string | ✅ | DeepSeek native models + common aliases (`gpt-5`, `gpt-5-mini`, `gpt-5-codex`, `o3`, `claude-opus-4-6`, `gemini-2.5-pro`, `gemini-2.5-flash`, etc.) |
| `messages` | array | ✅ | OpenAI-style messages |
| `stream` | boolean | ❌ | Default `false` |
| `tools` | array | ❌ | Function calling schema |
@@ -265,6 +293,7 @@ data: [DONE]
- `deepseek-reasoner` / `deepseek-reasoner-search` models emit `delta.reasoning_content`
- Text emits `delta.content`
- Last chunk includes `finish_reason` and `usage`
- Token counting prefers pass-through from upstream DeepSeek SSE (`accumulated_token_usage` / `token_usage`), and only falls back to local estimation when upstream usage is absent
#### Tool Calls
@@ -297,7 +326,12 @@ When `tools` is present, DS2API performs anti-leak handling:
}
```
**Stream**: Once high-confidence toolcall features are matched, DS2API emits `delta.tool_calls` immediately (without waiting for full JSON closure), then keeps sending argument deltas; confirmed raw tool JSON is never forwarded as `delta.content`.
**Stream**: Once high-confidence toolcall features are matched, DS2API emits `delta.tool_calls` immediately (without waiting for full argument closure), then keeps sending argument deltas; confirmed tool-call fragments are not forwarded as `delta.content`.
Additional notes:
- The parser currently follows XML/Markup-family tool payloads (`<tool_call>`, `<function_call>`, `<invoke>`, `tool_use`, antml variants). Standalone JSON `tool_calls` payloads are not treated as executable tool calls by default.
- `tool_calls` shown inside fenced markdown code blocks (for example, ```json ... ```) are treated as examples, not executable calls.
---
@@ -356,7 +390,8 @@ data: [DONE]
```
If `tool_choice=required` is violated in stream mode, DS2API emits `response.failed` then `[DONE]` (no `response.completed`).
Unknown tool names (outside declared `tools`) are rejected and will not be emitted as valid tool calls.
> Current behavior: the parser tries to extract structured tool calls and does not enforce a hard allow-list reject; your tool executor should still validate against a whitelist before executing.
### `GET /v1/responses/{response_id}`
@@ -375,11 +410,27 @@ Business auth required. Returns OpenAI-compatible embeddings shape.
> Requires `embeddings.provider`. Current supported values: `mock` / `deterministic` / `builtin`. If missing/unsupported, returns standard error shape with HTTP 501.
### `POST /v1/files`
Business auth required. OpenAI Files-compatible upload endpoint; currently only `multipart/form-data` is supported.
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `file` | file | ✅ | Binary payload |
| `purpose` | string | ❌ | Forwarded purpose field |
Constraints and behavior:
- `Content-Type` must be `multipart/form-data` (otherwise `400`).
- Total request size limit is `100 MiB` (over-limit returns `413`).
- Success returns an OpenAI `file` object (`id/object/bytes/filename/purpose/status`, etc.) and includes `account_id` for source-account tracing.
---
## Claude-Compatible API
Besides `/anthropic/v1/*`, DS2API also supports shortcut paths: `/v1/messages`, `/messages`, `/v1/messages/count_tokens`, `/messages/count_tokens`.
Implementation-wise this path is unified on the OpenAI Chat Completions parse-and-translate pipeline to avoid maintaining divergent parsing chains.
### `GET /anthropic/v1/models`
@@ -401,7 +452,7 @@ No auth required.
}
```
> Note: the example is partial; the real response includes historical Claude 1.x/2.x/3.x/4.x IDs and common aliases.
> Note: the example is partial; besides the current primary aliases, the real response also includes Claude 4.x snapshots plus historical 3.x / 2.x / 1.x IDs and common aliases.
### `POST /anthropic/v1/messages`
@@ -514,6 +565,7 @@ Supported paths:
- `/v1/models/{model}:streamGenerateContent` (compat path)
Authentication is the same as other business routes (`Authorization: Bearer <token>` or `x-api-key`).
Implementation-wise this path is unified on the OpenAI Chat Completions parse-and-translate pipeline to avoid maintaining divergent parsing chains.
### `POST /v1beta/models/{model}:generateContent`
@@ -532,6 +584,7 @@ Returns SSE (`text/event-stream`), each chunk as `data: <json>`:
- regular text: incremental text chunks
- `tools` mode: buffered and emitted as `functionCall` at finalize phase
- final chunk: includes `finishReason: "STOP"` and `usageMetadata`
- Token counting prefers pass-through from upstream DeepSeek SSE (`accumulated_token_usage` / `token_usage`), and only falls back to local estimation when upstream usage is absent
---
@@ -642,8 +695,9 @@ Reads runtime settings and status, including:
- `success`
- `admin` (`has_password_hash`, `jwt_expire_hours`, `jwt_valid_after_unix`, `default_password_warning`)
- `runtime` (`account_max_inflight`, `account_max_queue`, `global_max_inflight`, `token_refresh_interval_hours`)
- `compat` (`wide_input_strict_output`, `strip_reference_markers`)
- `responses` / `embeddings`
- `auto_delete` (`sessions`)
- `auto_delete` (`mode`: `none` / `single` / `all`; legacy `sessions=true` is still treated as `all`)
- `claude_mapping` / `model_aliases`
- `env_backed`, `needs_vercel_sync`
- `toolcall` policy is fixed to `feature_match + high` and is no longer returned or editable via settings
@@ -654,9 +708,10 @@ Hot-updates runtime settings. Supported fields:
- `admin.jwt_expire_hours`
- `runtime.account_max_inflight` / `runtime.account_max_queue` / `runtime.global_max_inflight` / `runtime.token_refresh_interval_hours`
- `compat.wide_input_strict_output` / `compat.strip_reference_markers`
- `responses.store_ttl_seconds`
- `embeddings.provider`
- `auto_delete.sessions`
- `auto_delete.mode`
- `claude_mapping`
- `model_aliases`
- `toolcall` policy is fixed and is no longer writable through settings
@@ -684,6 +739,8 @@ The request can send config directly, or wrapped as `{"config": {...}, "mode":"m
Query params `?mode=merge` / `?mode=replace` are also supported.
Import accepts `keys`, `accounts`, `claude_mapping` / `claude_model_mapping`, `model_aliases`, `admin`, `runtime`, `responses`, `embeddings`, and `auto_delete`; legacy `toolcall` fields are ignored.
> `compat` fields are managed via `/admin/settings` or the config file; this import endpoint does not update `compat`.
### `GET /admin/config/export`
Exports full config in three forms: `config`, `json`, and `base64`.
@@ -700,6 +757,26 @@ Exports full config in three forms: `config`, `json`, and `base64`.
**Response**: `{"success": true, "total_keys": 2}`
### `GET /admin/proxies`
Lists proxy configs (password is never returned; use `has_password` as a marker).
### `POST /admin/proxies`
Adds a proxy. Request accepts `id` (optional; auto-generated when omitted), `name`, `type` (`http` / `socks5`), `host`, `port`, `username`, `password`.
### `PUT /admin/proxies/{proxyID}`
Updates a proxy. If `password` is an empty string, the existing secret is preserved.
### `DELETE /admin/proxies/{proxyID}`
Deletes a proxy and automatically clears `proxy_id` on all accounts that reference it.
### `POST /admin/proxies/test`
Tests proxy connectivity: provide `proxy_id` to test a saved proxy; omit it to run a one-off test using proxy fields in the request body.
### `GET /admin/accounts`
**Query params**:
@@ -707,7 +784,7 @@ Exports full config in three forms: `config`, `json`, and `base64`.
| Param | Default | Range |
| --- | --- | --- |
| `page` | `1` | ≥ 1 |
| `page_size` | `10` | 1100 |
| `page_size` | `10` | 15000 |
| `q` | empty | Filter by identifier / email / mobile |
**Response**:
@@ -748,6 +825,14 @@ Returned items also include `test_status`, usually `ok` or `failed`.
**Response**: `{"success": true, "total_accounts": 5}`
### `PUT /admin/accounts/{identifier}/proxy`
Updates proxy binding for a specific account.
- Request body: `{"proxy_id":"..."}`.
- Use empty `proxy_id` to unbind proxy.
- `identifier` supports email / mobile / token-only synthetic id.
### `GET /admin/queue/status`
```json
@@ -758,17 +843,25 @@ Returned items also include `test_status`, usually `ok` or `failed`.
"available_accounts": ["a@example.com"],
"in_use_accounts": ["b@example.com"],
"max_inflight_per_account": 2,
"recommended_concurrency": 8
"global_max_inflight": 8,
"recommended_concurrency": 8,
"waiting": 0,
"max_queue_size": 8
}
```
| Field | Description |
| --- | --- |
| `available` | Currently available accounts |
| `in_use` | Currently in-use accounts |
| `available` | Accounts that still have spare inflight capacity |
| `in_use` | Number of occupied in-flight slots |
| `total` | Total accounts |
| `available_accounts` | List of account IDs with remaining inflight capacity |
| `in_use_accounts` | List of account IDs currently in use |
| `max_inflight_per_account` | Per-account inflight limit |
| `global_max_inflight` | Global inflight limit |
| `recommended_concurrency` | Suggested concurrency (`total × max_inflight_per_account`) |
| `waiting` | Number of queued requests currently waiting |
| `max_queue_size` | Waiting queue limit |
### `POST /admin/accounts/test`
@@ -871,6 +964,74 @@ Test API availability through the service itself.
}
```
### `POST /admin/dev/raw-samples/capture`
Internally issues one `/v1/chat/completions` request through the service, then persists the request metadata and raw upstream SSE into `tests/raw_stream_samples/<sample-id>/`.
Common request fields:
| Field | Required | Default | Notes |
| --- | --- | --- | --- |
| `message` | No | `你好` | Convenience single-turn user message |
| `messages` | No | Auto-derived from `message` | OpenAI-style message array |
| `model` | No | `deepseek-chat` | Target model |
| `stream` | No | `true` | Recommended to keep streaming enabled so raw SSE is recorded |
| `api_key` | No | First configured key | Business API key to use |
| `sample_id` | No | Auto-generated | Sample directory name |
On success, the response headers include:
- `X-Ds2-Sample-Id`
- `X-Ds2-Sample-Dir`
- `X-Ds2-Sample-Meta`
- `X-Ds2-Sample-Upstream`
If the request itself succeeds but the process did not record a new upstream capture, the endpoint returns:
```json
{"detail":"no upstream capture was recorded"}
```
### `GET /admin/dev/raw-samples/query`
Searches the current process's in-memory capture entries and groups `completion + continue` rounds by `chat_session_id`.
**Query parameters**:
| Param | Default | Notes |
| --- | --- | --- |
| `q` | empty | Fuzzy match against request/response text |
| `limit` | `20` | Max number of chains returned |
**Response fields** include:
- `items[].chain_key`
- `items[].capture_ids`
- `items[].round_count`
- `items[].initial_label`
- `items[].request_preview`
- `items[].response_preview`
### `POST /admin/dev/raw-samples/save`
Persists one selected in-memory capture chain into `tests/raw_stream_samples/<sample-id>/`.
Any one of these selectors is accepted:
```json
{"chain_key":"session:xxxx","sample_id":"tmp-from-memory"}
```
```json
{"capture_id":"cap_xxx","sample_id":"tmp-from-memory"}
```
```json
{"query":"Guangzhou weather","sample_id":"tmp-from-memory"}
```
The success payload includes `sample_id`, `dir`, `meta_path`, and `upstream_path`.
### `POST /admin/vercel/sync`
| Field | Required | Notes |

187
API.md
View File

@@ -4,6 +4,8 @@
本文档描述当前 Go 代码库的实际 API 行为。
文档导航:[总览](README.MD) / [架构说明](docs/ARCHITECTURE.md) / [部署指南](docs/DEPLOY.md) / [测试指南](docs/TESTING.md)
---
## 目录
@@ -35,7 +37,7 @@
- OpenAI / Claude / Gemini 三套协议已统一挂在同一 `chi` 路由树上,由 `internal/server/router.go` 负责装配。
- 适配器层职责收敛为:**请求归一化 → DeepSeek 调用 → 协议形态渲染**,减少历史版本中“同能力多处实现”的分叉。
- Tool Calling 的解析策略在 Go 与 Node Runtime 间保持一致:优先结构化解析JSON/XML/invoke/markup),并在流式场景执行防泄漏筛分。
- Tool Calling 的解析策略在 Go 与 Node Runtime 间保持一致:当前以 XML/Markup 家族解析为主(含 `<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml 变体),并在流式场景执行防泄漏筛分。
- `Admin API` 将配置与运行时策略分开:`/admin/config*` 管静态配置,`/admin/settings*` 管运行时行为。
---
@@ -52,8 +54,7 @@ cp config.example.json config.json
按部署方式使用:
- 本地运行:直接读取 `config.json`
- Docker / Vercel`config.json` 生成 Base64填入 `DS2API_CONFIG_JSON`
- 兼容写法:`DS2API_CONFIG_JSON` 也可直接填原始 JSON`CONFIG_JSON` 是旧版兼容回退变量
- Docker / Vercel`config.json` 生成 Base64填入 `DS2API_CONFIG_JSON`,也可以直接填原始 JSON
```bash
DS2API_CONFIG_JSON="$(base64 < config.json | tr -d '\n')"
@@ -107,6 +108,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
| POST | `/v1/responses` | 业务 | OpenAI Responses 接口(流式/非流式) |
| GET | `/v1/responses/{response_id}` | 业务 | 查询已生成 response内存 TTL |
| POST | `/v1/embeddings` | 业务 | OpenAI Embeddings 接口 |
| POST | `/v1/files` | 业务 | OpenAI Files 上传multipart/form-data |
| GET | `/anthropic/v1/models` | 无 | Claude 模型列表 |
| POST | `/anthropic/v1/messages` | 业务 | Claude 消息接口 |
| POST | `/anthropic/v1/messages/count_tokens` | 业务 | Claude token 计数 |
@@ -130,15 +132,24 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
| GET | `/admin/config/export` | Admin | 导出完整配置(含 `config`/`json`/`base64` |
| POST | `/admin/keys` | Admin | 添加 API key |
| DELETE | `/admin/keys/{key}` | Admin | 删除 API key |
| GET | `/admin/proxies` | Admin | 代理列表 |
| POST | `/admin/proxies` | Admin | 添加代理 |
| PUT | `/admin/proxies/{proxyID}` | Admin | 更新代理(留空 password 表示保留原密码) |
| DELETE | `/admin/proxies/{proxyID}` | Admin | 删除代理(自动解绑引用该代理的账号) |
| POST | `/admin/proxies/test` | Admin | 测试代理连通性 |
| GET | `/admin/accounts` | Admin | 分页账号列表 |
| POST | `/admin/accounts` | Admin | 添加账号 |
| DELETE | `/admin/accounts/{identifier}` | Admin | 删除账号 |
| PUT | `/admin/accounts/{identifier}/proxy` | Admin | 为账号绑定/解绑代理 |
| GET | `/admin/queue/status` | Admin | 账号队列状态 |
| POST | `/admin/accounts/test` | Admin | 测试单个账号 |
| POST | `/admin/accounts/test-all` | Admin | 测试全部账号 |
| POST | `/admin/accounts/sessions/delete-all` | Admin | 删除某账号的全部会话 |
| POST | `/admin/import` | Admin | 批量导入 keys/accounts |
| POST | `/admin/test` | Admin | 测试当前 API 可用性 |
| POST | `/admin/dev/raw-samples/capture` | Admin | 直接发起一次请求并保存为 raw sample |
| GET | `/admin/dev/raw-samples/query` | Admin | 按问题关键词查询当前内存抓包链 |
| POST | `/admin/dev/raw-samples/save` | Admin | 把命中的内存抓包链保存为 raw sample |
| POST | `/admin/vercel/sync` | Admin | 同步配置到 Vercel |
| GET | `/admin/vercel/status` | Admin | Vercel 同步状态 |
| POST | `/admin/vercel/status` | Admin | Vercel 同步状态 / 草稿对比 |
@@ -169,7 +180,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
### `GET /v1/models`
无需鉴权。返回当前支持的模型列表。
无需鉴权。返回当前支持的 DeepSeek 原生模型列表。
**响应示例**
@@ -180,11 +191,21 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
{"id": "deepseek-chat", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-reasoner", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-chat-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
{"id": "deepseek-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-chat", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-reasoner", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-chat-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-expert-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-chat", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-reasoner", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-chat-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-vision-reasoner-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
]
}
```
> 说明:`/v1/models` 返回的是规范化后的 DeepSeek 原生模型 ID常见 alias 仅用于请求入参解析,不会在该接口中单独展开返回。
### 模型 alias 解析策略
`chat` / `responses` / `embeddings``model` 字段采用“宽进严出”:
@@ -194,6 +215,13 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
3. 未命中时按模型家族规则回退(如 `o*``gpt-*``claude-*`)。
4. 仍未命中则返回 `invalid_request_error`
当前内置默认 alias节选
- OpenAI`gpt-4o``gpt-4.1``gpt-4.1-mini``gpt-4.1-nano``gpt-5``gpt-5-mini``gpt-5-codex`
- OpenAI Reasoning`o1``o1-mini``o3``o3-mini`
- Claude`claude-sonnet-4-5``claude-haiku-4-5``claude-opus-4-6`(及 `claude-3-5-sonnet` / `claude-3-5-haiku` / `claude-3-opus` 兼容别名)
- Gemini`gemini-2.5-pro``gemini-2.5-flash`
### `POST /v1/chat/completions`
**请求头**
@@ -207,7 +235,7 @@ Content-Type: application/json
| 字段 | 类型 | 必填 | 说明 |
| --- | --- | --- | --- |
| `model` | string | ✅ | 支持 DeepSeek 原生模型 + 常见 alias`gpt-4o``gpt-5-codex``o3``claude-sonnet-4-5` |
| `model` | string | ✅ | 支持 DeepSeek 原生模型 + 常见 alias`gpt-5``gpt-5-mini``gpt-5-codex``o3``claude-opus-4-6``gemini-2.5-pro``gemini-2.5-flash` |
| `messages` | array | ✅ | OpenAI 风格消息数组 |
| `stream` | boolean | ❌ | 默认 `false` |
| `tools` | array | ❌ | Function Calling 定义 |
@@ -265,6 +293,7 @@ data: [DONE]
- `deepseek-reasoner` / `deepseek-reasoner-search` 模型输出 `delta.reasoning_content`
- 普通文本输出 `delta.content`
- 最后一段包含 `finish_reason``usage`
- token 计数优先透传上游 DeepSeek SSE`accumulated_token_usage` / `token_usage`);仅在上游缺失时回退本地估算
#### Tool Calls
@@ -297,12 +326,12 @@ data: [DONE]
}
```
**流式**:命中高置信特征后立即输出 `delta.tool_calls`(不等待完整 JSON 闭合),并持续发送 arguments 增量;已确认的 toolcall 原始 JSON 不会回流到 `delta.content`
**流式**:命中高置信特征后立即输出 `delta.tool_calls`(不等待完整工具参数闭合),并持续发送 arguments 增量;已确认的工具调用片段不会回流到 `delta.content`
补充说明:
- **非代码块上下文**下,工具负载即使与普通文本混合,也会按特征识别并产出可执行 tool call前后普通文本仍可透传
- 解析器 XML/Markup 为最高优先级,并兼容 JSON、ANTML、text-kv 等格式输入;最终按客户端协议转译为对应 tool call 结构OpenAI/Claude/Gemini
- 解析器当前走 XML/Markup 家族(包含 `<tool_call>``<function_call>``<invoke>``tool_use`、antml 风格);纯 JSON `tool_calls` 片段默认不会直接作为可执行调用解析
- Markdown fenced code block例如 ```json ... ```)中的 `tool_calls` 仅视为示例文本,不会被执行。
---
@@ -382,11 +411,27 @@ data: [DONE]
> 需配置 `embeddings.provider`。当前支持:`mock` / `deterministic` / `builtin`。未配置或不支持时返回标准错误结构HTTP 501
### `POST /v1/files`
需要业务鉴权。兼容 OpenAI Files 上传接口,当前仅支持 `multipart/form-data`
| 字段 | 类型 | 必填 | 说明 |
| --- | --- | --- | --- |
| `file` | file | ✅ | 上传文件二进制 |
| `purpose` | string | ❌ | 透传到上游用途字段 |
约束与行为:
- 请求必须为 `multipart/form-data`,否则返回 `400`
- 请求体总大小上限 `100 MiB`(超限返回 `413`)。
- 成功返回 OpenAI `file` 对象(`id/object/bytes/filename/purpose/status` 等字段),并附带 `account_id` 便于定位来源账号。
---
## Claude 兼容接口
除标准路径 `/anthropic/v1/*` 外,还支持快捷路径 `/v1/messages``/messages``/v1/messages/count_tokens``/messages/count_tokens`
实现上统一走 OpenAI Chat Completions 解析与回译链路,避免多套解析逻辑分叉维护。
### `GET /anthropic/v1/models`
@@ -408,7 +453,7 @@ data: [DONE]
}
```
> 说明:示例仅展示部分模型;实际返回包含 Claude 1.x/2.x/3.x/4.x 历史模型 ID 与常见别名。
> 说明:示例仅展示部分模型;实际返回除当前主别名外,还包含 Claude 4.x snapshots以及 3.x / 2.x / 1.x 历史模型 ID 与常见别名。
### `POST /anthropic/v1/messages`
@@ -509,8 +554,6 @@ data: {"type":"message_stop"}
}
```
返回项还会包含 `test_status`,当前值通常为 `ok``failed`
---
## Gemini 兼容接口
@@ -523,6 +566,7 @@ data: {"type":"message_stop"}
- `/v1/models/{model}:streamGenerateContent`(兼容路径)
鉴权方式同业务接口(`Authorization: Bearer <token>``x-api-key`)。
实现上统一走 OpenAI Chat Completions 解析与回译链路,避免多套解析逻辑分叉维护。
### `POST /v1beta/models/{model}:generateContent`
@@ -541,6 +585,7 @@ data: {"type":"message_stop"}
- 常规文本:持续返回增量文本 chunk
- `tools` 场景:会缓冲并在结束时输出 `functionCall` 结构
- 结束 chunk包含 `finishReason: "STOP"``usageMetadata`
- token 计数优先透传上游 DeepSeek SSE`accumulated_token_usage` / `token_usage`);仅在上游缺失时回退本地估算
---
@@ -651,8 +696,9 @@ data: {"type":"message_stop"}
- `success`
- `admin``has_password_hash``jwt_expire_hours``jwt_valid_after_unix``default_password_warning`
- `runtime``account_max_inflight``account_max_queue``global_max_inflight``token_refresh_interval_hours`
- `compat``wide_input_strict_output``strip_reference_markers`
- `responses` / `embeddings`
- `auto_delete``sessions`
- `auto_delete``mode``none` / `single` / `all`;旧配置 `sessions=true` 仍按 `all` 处理
- `claude_mapping` / `model_aliases`
- `env_backed``needs_vercel_sync`
- `toolcall` 策略已固定为 `feature_match + high`,不再通过 settings 返回或修改
@@ -663,9 +709,10 @@ data: {"type":"message_stop"}
- `admin.jwt_expire_hours`
- `runtime.account_max_inflight` / `runtime.account_max_queue` / `runtime.global_max_inflight` / `runtime.token_refresh_interval_hours`
- `compat.wide_input_strict_output` / `compat.strip_reference_markers`
- `responses.store_ttl_seconds`
- `embeddings.provider`
- `auto_delete.sessions`
- `auto_delete.mode`
- `claude_mapping`
- `model_aliases`
- `toolcall` 策略已固定,不再作为可写入字段
@@ -693,6 +740,8 @@ data: {"type":"message_stop"}
也支持在查询参数里传 `?mode=merge` / `?mode=replace`
导入时会接受 `keys``accounts``claude_mapping` / `claude_model_mapping``model_aliases``admin``runtime``responses``embeddings``auto_delete` 等字段;`toolcall` 相关字段会被忽略。
> `compat` 相关字段请通过 `/admin/settings` 或配置文件管理;该导入接口不会更新 `compat`。
### `GET /admin/config/export`
导出完整配置,返回 `config``json``base64` 三种格式。
@@ -709,6 +758,26 @@ data: {"type":"message_stop"}
**响应**`{"success": true, "total_keys": 2}`
### `GET /admin/proxies`
列出代理配置(密码不回传,仅返回 `has_password` 标记)。
### `POST /admin/proxies`
新增代理。请求体支持 `id`(可选,未传则自动生成)、`name``type``http` / `socks5`)、`host``port``username``password`
### `PUT /admin/proxies/{proxyID}`
更新指定代理。若请求中 `password` 为空字符串,则保留原密码。
### `DELETE /admin/proxies/{proxyID}`
删除代理,并自动清空所有引用该代理账号的 `proxy_id`
### `POST /admin/proxies/test`
测试代理连通性:传 `proxy_id` 时测试已保存代理;不传时按请求体代理字段做临时连通性测试。
### `GET /admin/accounts`
**查询参数**
@@ -716,7 +785,7 @@ data: {"type":"message_stop"}
| 参数 | 默认 | 范围 |
| --- | --- | --- |
| `page` | `1` | ≥ 1 |
| `page_size` | `10` | 1100 |
| `page_size` | `10` | 15000 |
| `q` | 空 | 按 identifier / email / mobile 过滤 |
**响应**
@@ -755,6 +824,14 @@ data: {"type":"message_stop"}
**响应**`{"success": true, "total_accounts": 5}`
### `PUT /admin/accounts/{identifier}/proxy`
更新指定账号绑定代理。
- 请求体:`{"proxy_id":"..."}`
- `proxy_id` 传空字符串时表示解绑代理;
- `identifier` 支持 email / mobile / token-only 合成标识。
### `GET /admin/queue/status`
```json
@@ -765,17 +842,25 @@ data: {"type":"message_stop"}
"available_accounts": ["a@example.com"],
"in_use_accounts": ["b@example.com"],
"max_inflight_per_account": 2,
"recommended_concurrency": 8
"global_max_inflight": 8,
"recommended_concurrency": 8,
"waiting": 0,
"max_queue_size": 8
}
```
| 字段 | 说明 |
| --- | --- |
| `available` | 当前可用账号数 |
| `in_use` | 当前使用中的账号数 |
| `available` | 仍有剩余并发槽位的账号数 |
| `in_use` | 当前已占用的 in-flight 槽位数 |
| `total` | 总账号数 |
| `available_accounts` | 仍有剩余并发槽位的账号 ID 列表 |
| `in_use_accounts` | 当前处于使用中的账号 ID 列表 |
| `max_inflight_per_account` | 每账号并发上限 |
| `global_max_inflight` | 全局并发上限 |
| `recommended_concurrency` | 建议并发值(`total × max_inflight_per_account` |
| `waiting` | 当前等待中的请求数 |
| `max_queue_size` | 等待队列上限 |
### `POST /admin/accounts/test`
@@ -877,6 +962,74 @@ data: {"type":"message_stop"}
}
```
### `POST /admin/dev/raw-samples/capture`
直接通过服务自身发起一次 `/v1/chat/completions` 请求,并把请求元信息和上游原始 SSE 保存到 `tests/raw_stream_samples/<sample-id>/`
常用请求字段:
| 字段 | 必填 | 默认值 | 说明 |
| --- | --- | --- | --- |
| `message` | 否 | `你好` | 便捷单轮用户消息 |
| `messages` | 否 | 自动由 `message` 生成 | OpenAI 风格消息数组 |
| `model` | 否 | `deepseek-chat` | 目标模型 |
| `stream` | 否 | `true` | 建议保留流式,以记录原始 SSE |
| `api_key` | 否 | 配置中第一个 key | 调用业务接口使用的 key |
| `sample_id` | 否 | 自动生成 | 样本目录名 |
成功时会在响应头里附带:
- `X-Ds2-Sample-Id`
- `X-Ds2-Sample-Dir`
- `X-Ds2-Sample-Meta`
- `X-Ds2-Sample-Upstream`
如果请求本身成功,但当前进程没有记录到新的上游抓包,会返回:
```json
{"detail":"no upstream capture was recorded"}
```
### `GET /admin/dev/raw-samples/query`
按关键词查询当前进程内存里的抓包记录,并按 `chat_session_id` 归并 `completion + continue` 链。
**查询参数**
| 参数 | 默认值 | 说明 |
| --- | --- | --- |
| `q` | 空 | 按请求体/响应体关键词模糊匹配 |
| `limit` | `20` | 返回链条数上限 |
**响应字段**包含:
- `items[].chain_key`
- `items[].capture_ids`
- `items[].round_count`
- `items[].initial_label`
- `items[].request_preview`
- `items[].response_preview`
### `POST /admin/dev/raw-samples/save`
把当前内存中的某条抓包链落盘为 `tests/raw_stream_samples/<sample-id>/`
支持以下任一种选中方式:
```json
{"chain_key":"session:xxxx","sample_id":"tmp-from-memory"}
```
```json
{"capture_id":"cap_xxx","sample_id":"tmp-from-memory"}
```
```json
{"query":"广州天气","sample_id":"tmp-from-memory"}
```
成功响应会返回 `sample_id``dir``meta_path``upstream_path`
### `POST /admin/vercel/sync`
| 字段 | 必填 | 说明 |

View File

@@ -34,7 +34,7 @@ CMD ["/usr/local/bin/ds2api"]
FROM runtime-base AS runtime-from-source
COPY --from=go-builder /out/ds2api /usr/local/bin/ds2api
COPY --from=go-builder /app/internal/deepseek/assets/sha3_wasm_bg.7b9ca65ddd.wasm /app/sha3_wasm_bg.7b9ca65ddd.wasm
COPY --from=go-builder /app/config.example.json /app/config.example.json
COPY --from=webui-builder /app/static/admin /app/static/admin
@@ -53,13 +53,13 @@ RUN set -eux; \
test -n "${PKG_DIR}"; \
mkdir -p /out/static; \
cp "${PKG_DIR}/ds2api" /out/ds2api; \
cp "${PKG_DIR}/sha3_wasm_bg.7b9ca65ddd.wasm" /out/sha3_wasm_bg.7b9ca65ddd.wasm; \
cp "${PKG_DIR}/config.example.json" /out/config.example.json; \
cp -R "${PKG_DIR}/static/admin" /out/static/admin
FROM runtime-base AS runtime-from-dist
COPY --from=dist-extract /out/ds2api /usr/local/bin/ds2api
COPY --from=dist-extract /out/sha3_wasm_bg.7b9ca65ddd.wasm /app/sha3_wasm_bg.7b9ca65ddd.wasm
COPY --from=dist-extract /out/config.example.json /app/config.example.json
COPY --from=dist-extract /out/static/admin /app/static/admin

236
README.MD
View File

@@ -16,6 +16,10 @@
将 DeepSeek Web 对话能力转换为 OpenAI、Claude 与 Gemini 兼容 API。后端为 **Go 全量实现**,前端为 React WebUI 管理台(源码在 `webui/`,部署时自动构建到 `static/admin`)。
文档入口:[文档导航](docs/README.md) / [架构说明](docs/ARCHITECTURE.md) / [接口文档](API.md)
【感谢Linux.do社区及GitHub社区各位开发者对项目的支持与贡献】
> **重要免责声明**
>
> 本仓库仅供学习、研究、个人实验和内部验证使用,不提供任何形式的商业授权、适用性保证或结果保证。
@@ -24,7 +28,7 @@
>
> 请勿将本项目用于违反服务条款、协议、法律法规或平台规则的场景。商业使用前请自行确认 `LICENSE`、相关协议以及你是否获得了作者的书面许可。
## 架构概览
## 架构概览(摘要)
```mermaid
flowchart LR
@@ -48,7 +52,7 @@ flowchart LR
Auth["Auth Resolver\n(API key / bearer / x-goog-api-key)"]
Pool["Account Pool + Queue\n(并发槽位 + 等待队列)"]
DSClient["DeepSeek Client\n(Session / Auth / HTTP)"]
Pow["PoW WASM\n(wazero 预加载)"]
Pow["PoW 实现\n(纯 Go 毫秒级)"]
Tool["Tool Sieve\n(Go/Node 语义对齐)"]
end
end
@@ -72,32 +76,24 @@ flowchart LR
Bridge --> Client
```
详细架构拆分与目录职责见 [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)。
- **后端**Go`cmd/ds2api/`、`api/`、`internal/`),不依赖 Python 运行时
- **前端**React 管理台(`webui/`),运行时托管静态构建产物
- **部署**本地运行、Docker、Vercel Serverless、Linux systemd
### 3.0 底层架构调整(相较旧版本)
- **统一路由内核**:所有协议入口统一汇聚到 `internal/server/router.go`,并在同一路由树中注册 OpenAI / Claude / Gemini / Admin / WebUI 路由,避免多入口行为漂移。
- **统一执行链路**Claude / Gemini 入口先经 `internal/translatorcliproxy` 做协议转换,再进入 `openai.ChatCompletions` 统一处理工具调用与流式语义,最后再转换回原协议响应。
- **适配器分层更清晰**`internal/adapter/{claude,gemini}` 负责入口/出口协议封装,`internal/adapter/openai` 负责核心执行DeepSeek 侧调用只保留在 OpenAI 内核中。
- **Tool Calling 双运行时对齐**Go 侧(`internal/util`)与 Vercel Node 侧(`internal/js/helpers/stream-tool-sieve`)保持一致的解析/防泄漏语义,覆盖 JSON / XML / invoke / text-kv 多风格输入。
- **配置与运行时设置解耦**:静态配置(`config`)与运行时策略(`settings`)通过 Admin API 分离管理,支持热更新和密码轮换失效旧 JWT。
- **流式能力升级**`/v1/responses` 与 `/v1/chat/completions` 共享更一致的工具调用增量输出策略,降低不同 SDK 下的行为差异。
- **可观测与可运维增强**`/healthz`、`/readyz`、`/admin/version`、`/admin/dev/captures` 形成排障闭环,便于发布后验证。
## 核心能力
| 能力 | 说明 |
| --- | --- |
| OpenAI 兼容 | `GET /v1/models`、`GET /v1/models/{id}`、`POST /v1/chat/completions`、`POST /v1/responses`、`GET /v1/responses/{response_id}`、`POST /v1/embeddings` |
| OpenAI 兼容 | `GET /v1/models`、`GET /v1/models/{id}`、`POST /v1/chat/completions`、`POST /v1/responses`、`GET /v1/responses/{response_id}`、`POST /v1/embeddings`、`POST /v1/files` |
| Claude 兼容 | `GET /anthropic/v1/models`、`POST /anthropic/v1/messages`、`POST /anthropic/v1/messages/count_tokens`(及快捷路径 `/v1/messages`、`/messages` |
| Gemini 兼容 | `POST /v1beta/models/{model}:generateContent`、`POST /v1beta/models/{model}:streamGenerateContent`(及 `/v1/models/{model}:*` 路径) |
| 多账号轮询 | 自动 token 刷新、邮箱/手机号双登录方式 |
| 并发队列控制 | 每账号 in-flight 上限 + 等待队列,动态计算建议并发值 |
| DeepSeek PoW | WASM 计算(`wazero`),无需外部 Node.js 依赖 |
| DeepSeek PoW | 纯 Go 高性能实现DeepSeekHashV1毫秒级响应 |
| Tool Calling | 防泄漏处理:非代码块高置信特征识别、`delta.tool_calls` 早发、结构化增量输出 |
| Admin API | 配置管理、运行时设置热更新、账号测试 / 批量测试、会话清理、导入导出、Vercel 同步、版本检查 |
| Admin API | 配置管理、运行时设置热更新、代理管理、账号测试 / 批量测试、会话清理、导入导出、Vercel 同步、版本检查 |
| WebUI 管理台 | `/admin` 单页应用(中英文双语、深色模式) |
| 运维探针 | `GET /healthz`(存活)、`GET /readyz`(就绪) |
@@ -111,37 +107,45 @@ flowchart LR
| P0 | Anthropic SDKmessages | ✅ |
| P0 | Google Gemini SDKgenerateContent | ✅ |
| P1 | LangChain / LlamaIndex / OpenWebUIOpenAI 兼容接入) | ✅ |
| P2 | MCP 独立桥接层 | 规划中 |
## 模型支持
### OpenAI 接口
### OpenAI 接口`GET /v1/models`
| 模型 | thinking | search |
| --- | --- | --- |
| `deepseek-chat` | ❌ | ❌ |
| `deepseek-reasoner` | ✅ | ❌ |
| `deepseek-chat-search` | ❌ | ✅ |
| `deepseek-reasoner-search` | ✅ | ✅ |
| 模型类型 | 模型 ID | thinking | search |
| --- | --- | --- | --- |
| default | `deepseek-chat` | ❌ | ❌ |
| default | `deepseek-reasoner` | ✅ | ❌ |
| default | `deepseek-chat-search` | ❌ | ✅ |
| default | `deepseek-reasoner-search` | ✅ | ✅ |
| expert | `deepseek-expert-chat` | ❌ | ❌ |
| expert | `deepseek-expert-reasoner` | ✅ | ❌ |
| expert | `deepseek-expert-chat-search` | ❌ | ✅ |
| expert | `deepseek-expert-reasoner-search` | ✅ | ✅ |
| vision | `deepseek-vision-chat` | ❌ | ❌ |
| vision | `deepseek-vision-reasoner` | ✅ | ❌ |
| vision | `deepseek-vision-chat-search` | ❌ | ✅ |
| vision | `deepseek-vision-reasoner-search` | ✅ | ✅ |
### Claude 接口
除原生模型外,也支持常见 alias 输入(如 `gpt-5`、`gpt-5-mini`、`gpt-5-codex`、`gpt-4.1`、`o3`、`claude-opus-4-6`、`claude-sonnet-4-5`、`gemini-2.5-pro`、`gemini-2.5-flash` 等),但 `/v1/models` 返回的是规范化后的 DeepSeek 原生模型 ID。
| 模型 | 默认映射 |
### Claude 接口(`GET /anthropic/v1/models`
| 当前常用模型 | 默认映射 |
| --- | --- |
| `claude-sonnet-4-5` | `deepseek-chat` |
| `claude-haiku-4-5`(兼容 `claude-3-5-haiku-latest` | `deepseek-chat` |
| `claude-opus-4-6` | `deepseek-reasoner` |
可通过配置中的 `claude_mapping` 或 `claude_model_mapping` 覆盖映射关系。
另外,`/anthropic/v1/models` 现已包含 Claude 1.x/2.x/3.x/4.x 历史模型 ID 与常见别名,便于旧客户端直接兼容。
`/anthropic/v1/models` 除上述当前主别名外,还会返回 Claude 4.x snapshots以及 3.x / 2.x / 1.x 历史模型 ID 与常见 alias,便于旧客户端直接兼容。
#### Claude Code 接入避坑(实测)
- `ANTHROPIC_BASE_URL` 推荐直接指向 DS2API 根地址(例如 `http://127.0.0.1:5001`Claude Code 会请求 `/v1/messages?beta=true`。
- `ANTHROPIC_API_KEY` 需要与 `config.json` 中 `keys` 一致;建议同时保留常规 key 与 `sk-ant-*` 形态 key兼容不同客户端校验习惯。
- 若系统设置了代理,建议对 DS2API 地址配置 `NO_PROXY=127.0.0.1,localhost,<你的主机IP>`,避免本地回环请求被代理拦截。
- 如遇“工具调用输出成文本、未执行”问题,请升级到包含 Claude 工具调用多格式解析JSON/XML/ANTML/invoke的版本
- 如遇“工具调用输出成文本、未执行”问题,请优先检查模型输出是否为受支持的 XML/Markup 工具块(例如 `<tool_call>` / `<function_call>` / `<invoke>` / `tool_use`),而不是纯 JSON `tool_calls` 片段
### Gemini 接口
@@ -149,6 +153,15 @@ Gemini 适配器将模型名通过 `model_aliases` 或内置规则映射到 Deep
## 快速开始
### 部署方式优先级建议
推荐按以下顺序选择部署方式:
1. **下载 Release 构建包运行**:最省事,产物已编译完成,最适合大多数用户。
2. **Docker / GHCR 镜像部署**:适合需要容器化、编排或云环境部署。
3. **Vercel 部署**:适合已有 Vercel 环境且接受其平台约束的场景。
4. **本地源码运行 / 自行编译**:适合开发、调试或需要自行修改代码的场景。
### 通用第一步(所有部署方式)
把 `config.json` 作为唯一配置源(推荐做法):
@@ -160,30 +173,21 @@ cp config.example.json config.json
后续部署建议:
- 本地运行:直接读取 `config.json`
- Docker / Vercel由 `config.json` 生成 `DS2API_CONFIG_JSON`Base64注入环境变量
- 兼容写法:`DS2API_CONFIG_JSON` 也可以直接写原始 JSON`CONFIG_JSON` 是旧版回退变量
- Docker / Vercel由 `config.json` 生成 `DS2API_CONFIG_JSON`Base64注入环境变量,也可以直接写原始 JSON
### 方式一:本地运行
### 方式一:下载 Release 构建包
**前置要求**Go 1.26+Node.js 20+(仅在需要构建 WebUI 时)
每次发布 Release 时GitHub Actions 会自动构建多平台二进制包:
```bash
# 1. 克隆仓库
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 2. 配置
# 下载对应平台的压缩包后
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
cp config.example.json config.json
# 编辑 config.json,填入你的 DeepSeek 账号信息和 API key
# 3. 启动
go run ./cmd/ds2api
# 编辑 config.json
./ds2api
```
默认监听地址:`http://localhost:5001`
> **WebUI 自动构建**:本地首次启动时,若 `static/admin` 不存在,会自动尝试执行 `npm ci`(仅在缺少依赖时)和 `npm run build -- --outDir static/admin --emptyOutDir`(需要本机有 Node.js。你也可以手动构建`./scripts/build-webui.sh`
### 方式二Docker 运行
```bash
@@ -191,7 +195,7 @@ go run ./cmd/ds2api
cp .env.example .env
cp config.example.json config.json
# 2. 编辑 .env至少设置 DS2API_ADMIN_KEY
# 2. 编辑 .env至少设置 DS2API_ADMIN_KEY;如需修改宿主机端口,可额外设置 DS2API_HOST_PORT
# DS2API_ADMIN_KEY=请替换为强密码
# 3. 启动
@@ -201,7 +205,7 @@ docker-compose up -d
docker-compose logs -f
```
默认 `docker-compose.yml` 会把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请调整 `ports` 配置。
默认 `docker-compose.yml` 会把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请设置 `DS2API_HOST_PORT=5001`(或者手动调整 `ports` 配置
更新镜像:`docker-compose up -d --build`
@@ -237,35 +241,28 @@ base64 < config.json | tr -d '\n'
详细部署说明请参阅 [部署指南](docs/DEPLOY.md)。
### 方式四:下载 Release 构建包
### 方式四:本地源码运行
每次发布 Release 时GitHub Actions 会自动构建多平台二进制包:
**前置要求**Go 1.26+Node.js `20.19+` 或 `22.12+`(仅在需要构建 WebUI 时)
```bash
# 下载对应平台的压缩包后
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
# 1. 克隆仓库
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 2. 配置
cp config.example.json config.json
# 编辑 config.json
./ds2api
# 编辑 config.json,填入你的 DeepSeek 账号信息和 API key
# 3. 启动
go run ./cmd/ds2api
```
### 方式五OpenCode CLI 接入
默认本地访问地址:`http://127.0.0.1:5001`
1. 复制示例配置:
服务实际绑定:`0.0.0.0:5001`,因此同一局域网设备通常也可以通过你的内网 IP 访问。
```bash
cp opencode.json.example opencode.json
```
2. 编辑 `opencode.json`
- 将 `baseURL` 改为你的 DS2API 地址(例如 `https://your-domain.com/v1`
- 将 `apiKey` 改为你的 DS2API key对应 `config.keys`
3. 在项目目录启动 OpenCode CLI按你的安装方式运行 `opencode`)。
> 建议优先使用 OpenAI 兼容路径(`/v1/*`),即示例里的 `@ai-sdk/openai-compatible` provider。
> 若客户端支持 `wire_api`,可分别测试 `responses` 与 `chat`DS2API 两条链路都兼容。
> **WebUI 自动构建**:本地首次启动时,若 `static/admin` 不存在,会自动尝试执行 `npm ci`(仅在缺少依赖时)和 `npm run build -- --outDir static/admin --emptyOutDir`(需要本机有 Node.js。你也可以手动构建`./scripts/build-webui.sh`
## 配置说明
@@ -286,11 +283,16 @@ cp opencode.json.example opencode.json
],
"model_aliases": {
"gpt-4o": "deepseek-chat",
"gpt-5": "deepseek-chat",
"gpt-5-mini": "deepseek-chat",
"gpt-5-codex": "deepseek-reasoner",
"o3": "deepseek-reasoner"
"o3": "deepseek-reasoner",
"claude-opus-4-6": "deepseek-reasoner",
"gemini-2.5-flash": "deepseek-chat"
},
"compat": {
"wide_input_strict_output": true
"wide_input_strict_output": true,
"strip_reference_markers": true
},
"responses": {
"store_ttl_seconds": 900
@@ -312,7 +314,7 @@ cp opencode.json.example opencode.json
"token_refresh_interval_hours": 6
},
"auto_delete": {
"sessions": false
"mode": "none"
}
}
```
@@ -322,13 +324,14 @@ cp opencode.json.example opencode.json
- `token`:配置文件中即使填写也会在加载时被清空(不会从 `config.json` 读取 token实际 token 仅在运行时内存中维护并自动刷新
- `model_aliases`:常见模型名(如 GPT/Codex/Claude到 DeepSeek 模型的映射
- `compat.wide_input_strict_output`:建议保持 `true`(当前实现默认宽进严出)
- `toolcall`:策略已固定为特征匹配 + 高置信早发,不再作为可配置项
- `compat.strip_reference_markers`:建议保持 `true`,用于清理可见输出中的引用/标记
- `toolcall`:旧字段,当前实现已固定为特征匹配 + 高置信早发;即使保留在配置里也会被忽略
- `responses.store_ttl_seconds``/v1/responses/{id}` 的内存缓存 TTL
- `embeddings.provider`embedding 提供方(当前内置 `deterministic/mock/builtin`
- `claude_mapping`:字典中 `fast`/`slow` 后缀映射到对应 DeepSeek 模型(兼容读取 `claude_model_mapping`
- `admin`管理后台设置JWT 过期时间、密码哈希等),可通过 Admin Settings API 热更新
- `runtime`:运行时参数(并发限制、队列大小、托管账号 token 刷新间隔),可通过 Admin Settings API 热更新;`account_max_queue=0`/`global_max_inflight=0` 表示按推荐值自动计算,`token_refresh_interval_hours=6` 为默认强制重登间隔
- `auto_delete.sessions`:是否在请求结束后自动清理 DeepSeek 会话(默认 `false`,可在 Settings 热更新)
- `auto_delete.mode`请求结束后如何清理 DeepSeek 远端聊天记录,支持 `none`(默认,不删除)、`single`(仅删除当前会话)、`all`(清空全部会话);旧配置里的 `auto_delete.sessions=true` 仍会被视为 `all`
### 环境变量
@@ -341,31 +344,23 @@ cp opencode.json.example opencode.json
| `DS2API_JWT_EXPIRE_HOURS` | Admin JWT 过期小时数 | `24` |
| `DS2API_CONFIG_PATH` | 配置文件路径 | `config.json` |
| `DS2API_CONFIG_JSON` | 直接注入配置JSON 或 Base64 | — |
| `CONFIG_JSON` | 旧版兼容配置注入 | — |
| `DS2API_ENV_WRITEBACK` | 环境变量模式下自动写回配置文件并切换文件模式(`1/true/yes/on` | 关闭 |
| `DS2API_WASM_PATH` | PoW WASM 文件路径 | 自动查找 |
| `DS2API_STATIC_ADMIN_DIR` | 管理台静态文件目录 | `static/admin` |
| `DS2API_AUTO_BUILD_WEBUI` | 启动时自动构建 WebUI | 本地开启Vercel 关闭 |
| `DS2API_DEV_PACKET_CAPTURE` | 本地开发抓包开关(记录最近会话请求/响应体) | 本地非 Vercel 默认开启 |
| `DS2API_DEV_PACKET_CAPTURE_LIMIT` | 本地抓包保留条数(超出自动淘汰) | `5` |
| `DS2API_DEV_PACKET_CAPTURE_MAX_BODY_BYTES` | 单条响应体最大记录字节数 | `2097152` |
| `DS2API_DEV_PACKET_CAPTURE_LIMIT` | 本地抓包保留条数(超出自动淘汰) | `20` |
| `DS2API_DEV_PACKET_CAPTURE_MAX_BODY_BYTES` | 单条响应体最大记录字节数 | `5242880` |
| `DS2API_ACCOUNT_MAX_INFLIGHT` | 每账号最大并发 in-flight 请求数 | `2` |
| `DS2API_ACCOUNT_CONCURRENCY` | 同上(兼容旧名) | — |
| `DS2API_ACCOUNT_MAX_QUEUE` | 等待队列上限 | `recommended_concurrency` |
| `DS2API_ACCOUNT_QUEUE_SIZE` | 同上(兼容旧名) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | 全局最大 in-flight 请求数 | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | 同上(兼容旧名) | — |
| `DS2API_VERCEL_INTERNAL_SECRET` | Vercel 混合流式内部鉴权密钥 | 回退用 `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | 流式 lease 过期秒数 | `900` |
| `DS2API_DEV_PACKET_CAPTURE` | 本地开发抓包开关(记录最近会话请求/响应体) | 本地非 Vercel 默认开启 |
| `DS2API_DEV_PACKET_CAPTURE_LIMIT` | 本地抓包保留条数(超出自动淘汰) | `5` |
| `DS2API_DEV_PACKET_CAPTURE_MAX_BODY_BYTES` | 单条响应体最大记录字节数 | `2097152` |
| `VERCEL_TOKEN` | Vercel 同步 token | — |
| `VERCEL_PROJECT_ID` | Vercel 项目 ID | — |
| `VERCEL_TEAM_ID` | Vercel 团队 ID | — |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel 部署保护绕过密钥(内部 Node→Go 调用) | — |
> 提示:当检测到 `DS2API_CONFIG_JSON/CONFIG_JSON` 时,管理台会显示当前模式风险与自动持久化状态(含 `DS2API_CONFIG_PATH` 路径与模式切换说明)。
> 提示:当检测到 `DS2API_CONFIG_JSON` 时,管理台会显示当前模式风险与自动持久化状态(含 `DS2API_CONFIG_PATH` 路径与模式切换说明)。
## 鉴权模式
@@ -397,7 +392,7 @@ Gemini 路由还可以使用 `x-goog-api-key`,或在没有认证头时使用 `
当请求中带 `tools` 时DS2API 会做防泄漏处理与结构化转译:
1. 只在**非代码块上下文**启用执行型 toolcall 识别(代码块示例默认不触发)
2. 解析层以 XML/Markup 为最高优先级,同时兼容 JSON / ANTML / invoke / text-kv并统一归一到内部工具调用结构
2. 解析层当前以 XML/Markup 家族为准(`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml 变体);纯 JSON `tool_calls` 片段默认不作为可执行调用解析
3. `responses` 流式严格使用官方 item 生命周期事件(`response.output_item.*`、`response.content_part.*`、`response.function_call_arguments.*`
4. `responses` 支持并执行 `tool_choice``auto`/`none`/`required`/强制函数);`required` 违规时非流式返回 `422`,流式返回 `response.failed`
5. 客户端请求哪种协议就按该协议返回工具调用OpenAI/Claude/Gemini 各自原生结构);模型侧优先约束输出规范 XML再由兼容层转译
@@ -408,13 +403,13 @@ Gemini 路由还可以使用 `x-goog-api-key`,或在没有认证头时使用 `
## 本地开发抓包工具
用于定位「responses 思考流/工具调用」等问题。开启后会自动记录最近 N 条 DeepSeek 对话上游请求体与响应体(默认 5 条,超出自动淘汰)。
用于定位「responses 思考流/工具调用」等问题。开启后会自动记录最近 N 条 DeepSeek 对话上游请求体与响应体(默认 20 条,超出自动淘汰;单条响应体默认最多记录 5 MB)。
启用示例:
```bash
DS2API_DEV_PACKET_CAPTURE=true \
DS2API_DEV_PACKET_CAPTURE_LIMIT=5 \
DS2API_DEV_PACKET_CAPTURE_LIMIT=20 \
go run ./cmd/ds2api
```
@@ -422,6 +417,8 @@ go run ./cmd/ds2api
- `GET /admin/dev/captures`:查看抓包列表(最新在前)
- `DELETE /admin/dev/captures`:清空抓包
- `GET /admin/dev/raw-samples/query?q=关键词&limit=20`:按问题关键词查询当前内存抓包,并按 `chat_session_id` 归并 `completion + continue` 链
- `POST /admin/dev/raw-samples/save`:把命中的某条抓包链保存为 `tests/raw_stream_samples/<sample-id>/` 回放样本
返回字段包含:
@@ -429,67 +426,10 @@ go run ./cmd/ds2api
- `response_body`:上游返回的原始流式内容拼接文本
- `response_truncated`:是否触发单条大小截断
## 项目结构
保存接口支持用 `query`、`chain_key` 或 `capture_id` 选中目标。例如:
```text
ds2api/
├── app/ # 统一 HTTP Handler 组装层(供本地与 Serverless 复用)
├── cmd/
│ ├── ds2api/ # 本地 / 容器启动入口
│ └── ds2api-tests/ # 端到端测试集入口
├── api/
│ ├── index.go # Vercel Serverless Go 入口
│ ├── chat-stream.js # Vercel Node.js 流式转发
│ └── (rewrite targets in vercel.json)
├── internal/
│ ├── account/ # 账号池与并发队列
│ ├── adapter/
│ │ ├── openai/ # OpenAI 兼容适配器(含 Tool Call 解析、Vercel 流式 prepare/release
│ │ ├── claude/ # Claude 兼容适配器
│ │ └── gemini/ # Gemini 兼容适配器generateContent / streamGenerateContent
│ ├── admin/ # Admin API handlers含 Settings 热更新)
│ ├── auth/ # 鉴权与 JWT
│ ├── claudeconv/ # Claude 消息格式转换
│ ├── compat/ # Go 版本兼容与回归测试辅助
│ ├── config/ # 配置加载、校验与热更新
│ ├── deepseek/ # DeepSeek API 客户端、PoW WASM
│ ├── js/ # Node 运行时流式处理与兼容逻辑
│ ├── devcapture/ # 开发抓包模块
│ ├── format/ # 输出格式化
│ ├── prompt/ # Prompt 构建
│ ├── server/ # HTTP 路由与中间件chi router
│ ├── sse/ # SSE 解析工具
│ ├── stream/ # 统一流式消费引擎
│ ├── testsuite/ # 端到端测试框架与用例编排
│ ├── translatorcliproxy/ # CLIProxy 桥接与流写入组件
│ ├── util/ # 通用工具函数
│ ├── version/ # 版本解析 / 比较与 tag 规范化
│ └── webui/ # WebUI 静态文件托管与自动构建
├── webui/ # React WebUI 源码Vite + Tailwind
│ └── src/
│ ├── app/ # 路由、鉴权、配置状态管理
│ ├── features/ # 业务功能模块account/settings/vercel/apiTester
│ ├── components/ # 登录/落地页等通用组件
│ └── locales/ # 中英文语言包zh.json / en.json
├── scripts/
│ └── build-webui.sh # WebUI 手动构建脚本
├── tests/
│ ├── compat/ # 兼容性测试夹具与期望输出
│ ├── node/ # Node 侧单元测试chat-stream / tool-sieve
│ └── scripts/ # 统一测试脚本入口unit/e2e
├── docs/ # 部署 / 贡献 / 测试等辅助文档
├── static/admin/ # WebUI 构建产物(不提交到 Git
├── .github/
│ ├── workflows/ # GitHub Actions质量门禁 + Release 自动构建)
│ ├── ISSUE_TEMPLATE/ # Issue 模板
│ └── PULL_REQUEST_TEMPLATE.md
├── config.example.json # 配置文件示例
├── .env.example # 环境变量示例
├── Dockerfile # 多阶段构建WebUI + Go
├── docker-compose.yml # 生产环境 Docker Compose
├── docker-compose.dev.yml # 开发环境 Docker Compose
├── vercel.json # Vercel 路由与构建配置
└── go.mod / go.sum # Go 模块依赖
```json
{"query":"广州天气","sample_id":"gz-weather-from-memory"}
```
## 文档索引
@@ -538,7 +478,7 @@ npm ci --prefix webui && npm run build --prefix webui
go test ./...
# 运行 tool calls 相关测试(调试工具调用问题)
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/util/
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/toolcall/
# 运行端到端测试
./tests/scripts/run-live.sh

View File

@@ -16,6 +16,8 @@ Language: [中文](README.MD) | [English](README.en.md)
DS2API converts DeepSeek Web chat capability into OpenAI-compatible, Claude-compatible, and Gemini-compatible APIs. The backend is a **pure Go implementation**, with a React WebUI admin panel (source in `webui/`, build output auto-generated to `static/admin` during deployment).
Documentation entry: [Docs Index](docs/README.md) / [Architecture](docs/ARCHITECTURE.en.md) / [API Reference](API.en.md)
> **Important Disclaimer**
>
> This repository is provided for learning, research, personal experimentation, and internal validation only. It does not grant any commercial authorization and comes with no warranty of fitness, stability, or results.
@@ -24,7 +26,7 @@ DS2API converts DeepSeek Web chat capability into OpenAI-compatible, Claude-comp
>
> Do not use this project in ways that violate service terms, agreements, laws, or platform rules. Before any commercial use, review the `LICENSE`, the relevant terms, and confirm that you have the author's written permission.
## Architecture Overview
## Architecture Overview (Summary)
```mermaid
flowchart LR
@@ -48,7 +50,7 @@ flowchart LR
Auth["Auth Resolver\n(API key / bearer / x-goog-api-key)"]
Pool["Account Pool + Queue\n(in-flight slots + wait queue)"]
DSClient["DeepSeek Client\n(session / auth / HTTP)"]
Pow["PoW WASM\n(wazero preload)"]
Pow["PoW Solver\n(Pure Go ms-level)"]
Tool["Tool Sieve\n(Go/Node semantic parity)"]
end
end
@@ -72,32 +74,24 @@ flowchart LR
Bridge --> Client
```
For the full module-by-module architecture and directory responsibilities, see [docs/ARCHITECTURE.en.md](docs/ARCHITECTURE.en.md).
- **Backend**: Go (`cmd/ds2api/`, `api/`, `internal/`), no Python runtime
- **Frontend**: React admin panel (`webui/`), served as static build at runtime
- **Deployment**: local run, Docker, Vercel serverless, Linux systemd
### 3.0 Architecture Changes (vs older releases)
- **Unified routing core**: all protocol entries are now centralized through `internal/server/router.go`, with OpenAI / Claude / Gemini / Admin / WebUI routes registered in one tree to avoid multi-entry drift.
- **Unified execution chain**: Claude/Gemini entries are translated by `internal/translatorcliproxy`, then executed through `openai.ChatCompletions` for shared tool-calling and stream semantics, then translated back to the client protocol.
- **Cleaner adapter boundaries**: `internal/adapter/{claude,gemini}` handles protocol wrappers, while `internal/adapter/openai` remains the execution core; upstream DeepSeek calls are retained only in the OpenAI core.
- **Tool-calling parity across runtimes**: Go (`internal/util`) and Vercel Node (`internal/js/helpers/stream-tool-sieve`) follow aligned parsing/anti-leak semantics across JSON / XML / invoke / text-kv inputs.
- **Config/runtime separation**: static config (`config`) and runtime policy (`settings`) are managed independently via Admin APIs, enabling hot updates and password rotation with JWT invalidation.
- **Streaming behavior upgrade**: `/v1/responses` and `/v1/chat/completions` now share a more consistent incremental tool-call emission strategy across SDK ecosystems.
- **Improved operability**: `/healthz`, `/readyz`, `/admin/version`, and `/admin/dev/captures` form a tighter post-deploy diagnostics loop.
## Key Capabilities
| Capability | Details |
| --- | --- |
| OpenAI compatible | `GET /v1/models`, `GET /v1/models/{id}`, `POST /v1/chat/completions`, `POST /v1/responses`, `GET /v1/responses/{response_id}`, `POST /v1/embeddings` |
| OpenAI compatible | `GET /v1/models`, `GET /v1/models/{id}`, `POST /v1/chat/completions`, `POST /v1/responses`, `GET /v1/responses/{response_id}`, `POST /v1/embeddings`, `POST /v1/files` |
| Claude compatible | `GET /anthropic/v1/models`, `POST /anthropic/v1/messages`, `POST /anthropic/v1/messages/count_tokens` (plus shortcut paths `/v1/messages`, `/messages`) |
| Gemini compatible | `POST /v1beta/models/{model}:generateContent`, `POST /v1beta/models/{model}:streamGenerateContent` (plus `/v1/models/{model}:*` paths) |
| Multi-account rotation | Auto token refresh, email/mobile dual login |
| Concurrency control | Per-account in-flight limit + waiting queue, dynamic recommended concurrency |
| DeepSeek PoW | WASM solving via `wazero`, no external Node.js dependency |
| DeepSeek PoW | Pure Go high-performance solver (DeepSeekHashV1), ms-level response |
| Tool Calling | Anti-leak handling: non-code-block feature match, early `delta.tool_calls`, structured incremental output |
| Admin API | Config management, runtime settings hot-reload, account testing/batch test, session cleanup, import/export, Vercel sync, version check |
| Admin API | Config management, runtime settings hot-reload, proxy management, account testing/batch test, session cleanup, import/export, Vercel sync, version check |
| WebUI Admin Panel | SPA at `/admin` (bilingual Chinese/English, dark mode) |
| Health Probes | `GET /healthz` (liveness), `GET /readyz` (readiness) |
@@ -111,37 +105,45 @@ flowchart LR
| P0 | Anthropic SDK (messages) | ✅ |
| P0 | Google Gemini SDK (generateContent) | ✅ |
| P1 | LangChain / LlamaIndex / OpenWebUI (OpenAI-compatible integration) | ✅ |
| P2 | MCP standalone bridge | Planned |
## Model Support
### OpenAI Endpoint
### OpenAI Endpoint (`GET /v1/models`)
| Model | thinking | search |
| --- | --- | --- |
| `deepseek-chat` | ❌ | ❌ |
| `deepseek-reasoner` | ✅ | ❌ |
| `deepseek-chat-search` | ❌ | ✅ |
| `deepseek-reasoner-search` | ✅ | ✅ |
| Family | Model ID | thinking | search |
| --- | --- | --- | --- |
| default | `deepseek-chat` | ❌ | ❌ |
| default | `deepseek-reasoner` | ✅ | ❌ |
| default | `deepseek-chat-search` | ❌ | ✅ |
| default | `deepseek-reasoner-search` | ✅ | ✅ |
| expert | `deepseek-expert-chat` | ❌ | ❌ |
| expert | `deepseek-expert-reasoner` | ✅ | ❌ |
| expert | `deepseek-expert-chat-search` | ❌ | ✅ |
| expert | `deepseek-expert-reasoner-search` | ✅ | ✅ |
| vision | `deepseek-vision-chat` | ❌ | ❌ |
| vision | `deepseek-vision-reasoner` | ✅ | ❌ |
| vision | `deepseek-vision-chat-search` | ❌ | ✅ |
| vision | `deepseek-vision-reasoner-search` | ✅ | ✅ |
### Claude Endpoint
Besides native IDs, DS2API also accepts common aliases as input (for example `gpt-5`, `gpt-5-mini`, `gpt-5-codex`, `gpt-4.1`, `o3`, `claude-opus-4-6`, `claude-sonnet-4-5`, `gemini-2.5-pro`, `gemini-2.5-flash`), but `/v1/models` returns normalized DeepSeek native model IDs.
| Model | Default Mapping |
### Claude Endpoint (`GET /anthropic/v1/models`)
| Current common model | Default Mapping |
| --- | --- |
| `claude-sonnet-4-5` | `deepseek-chat` |
| `claude-haiku-4-5` (compatible with `claude-3-5-haiku-latest`) | `deepseek-chat` |
| `claude-opus-4-6` | `deepseek-reasoner` |
Override mapping via `claude_mapping` or `claude_model_mapping` in config.
In addition, `/anthropic/v1/models` now includes historical Claude 1.x/2.x/3.x/4.x IDs and common aliases for legacy client compatibility.
Besides the current primary aliases above, `/anthropic/v1/models` also returns Claude 4.x snapshots plus historical 3.x / 2.x / 1.x IDs and common aliases for legacy client compatibility.
#### Claude Code integration pitfalls (validated)
- Set `ANTHROPIC_BASE_URL` to the DS2API root URL (for example `http://127.0.0.1:5001`). Claude Code sends requests to `/v1/messages?beta=true`.
- `ANTHROPIC_API_KEY` must match an entry in `keys` from `config.json`. Keeping both a regular key and an `sk-ant-*` style key improves client compatibility.
- If your environment has proxy variables, set `NO_PROXY=127.0.0.1,localhost,<your_host_ip>` for DS2API to avoid proxy interception of local traffic.
- If tool calls are rendered as plain text and not executed, upgrade to a build that includes multi-format Claude tool-call parsing (JSON/XML/ANTML/invoke).
- If tool calls are rendered as plain text and not executed, first verify the model output uses supported XML/Markup tool blocks (`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use`) rather than standalone JSON `tool_calls`.
### Gemini Endpoint
@@ -149,6 +151,15 @@ The Gemini adapter maps model names to DeepSeek native models via `model_aliases
## Quick Start
### Recommended deployment priority
Recommended order when choosing a deployment method:
1. **Download and run release binaries**: the easiest path for most users because the artifacts are already built.
2. **Docker / GHCR image deployment**: suitable for containerized, orchestrated, or cloud environments.
3. **Vercel deployment**: suitable if you already use Vercel and accept its platform constraints.
4. **Run from source / build locally**: suitable for development, debugging, or when you need to modify the code yourself.
### Universal First Step (all deployment modes)
Use `config.json` as the single source of truth (recommended):
@@ -160,48 +171,39 @@ cp config.example.json config.json
Recommended per deployment mode:
- Local run: read `config.json` directly
- Docker / Vercel: generate Base64 from `config.json` and inject as `DS2API_CONFIG_JSON`
- Compatibility note: `DS2API_CONFIG_JSON` may also contain raw JSON directly; `CONFIG_JSON` is the legacy fallback variable
- Docker / Vercel: generate Base64 from `config.json` and inject as `DS2API_CONFIG_JSON`, or paste raw JSON directly
### Option 1: Local Run
### Option 1: Download Release Binaries
**Prerequisites**: Go 1.26+, Node.js 20+ (only if building WebUI locally)
GitHub Actions automatically builds multi-platform archives on each Release:
```bash
# 1. Clone
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 2. Configure
# After downloading the archive for your platform
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
cp config.example.json config.json
# Edit config.json with your DeepSeek account info and API keys
# 3. Start
go run ./cmd/ds2api
# Edit config.json
./ds2api
```
Default URL: `http://localhost:5001`
> **WebUI auto-build**: On first local startup, if `static/admin` is missing, DS2API will auto-run `npm ci` (only when dependencies are missing) and `npm run build -- --outDir static/admin --emptyOutDir` (requires Node.js). You can also build manually: `./scripts/build-webui.sh`
### Option 2: Docker
### Option 2: Docker / GHCR
```bash
# 1. Prepare env file and config file
# Pull prebuilt image
docker pull ghcr.io/cjackhwang/ds2api:latest
# Or run a pinned version
# docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
# Prepare env file and config file
cp .env.example .env
cp config.example.json config.json
# 2. Edit .env (at least set DS2API_ADMIN_KEY)
# DS2API_ADMIN_KEY=replace-with-a-strong-secret
# 3. Start
# Start with compose
docker-compose up -d
# 4. View logs
docker-compose logs -f
```
The default `docker-compose.yml` maps host port `6011` to container port `5001`. If you want `5001` exposed directly, adjust the `ports` mapping.
The default `docker-compose.yml` uses `ghcr.io/cjackhwang/ds2api:latest` and maps host port `6011` to container port `5001`. If you want `5001` exposed directly, set `DS2API_HOST_PORT=5001` (or adjust the `ports` mapping).
Rebuild after updates: `docker-compose up -d --build`
@@ -237,35 +239,28 @@ base64 < config.json | tr -d '\n'
For detailed deployment instructions, see the [Deployment Guide](docs/DEPLOY.en.md).
### Option 4: Download Release Binaries
### Option 4: Local Run
GitHub Actions automatically builds multi-platform archives on each Release:
**Prerequisites**: Go 1.26+, Node.js `20.19+` or `22.12+` (only if building WebUI locally)
```bash
# After downloading the archive for your platform
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
# 1. Clone
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 2. Configure
cp config.example.json config.json
# Edit config.json
./ds2api
# Edit config.json with your DeepSeek account info and API keys
# 3. Start
go run ./cmd/ds2api
```
### Option 5: OpenCode CLI
Default local URL: `http://127.0.0.1:5001`
1. Copy the example config:
The server actually binds to `0.0.0.0:5001`, so devices on the same LAN can usually reach it through your private IP as well.
```bash
cp opencode.json.example opencode.json
```
2. Edit `opencode.json`:
- Set `baseURL` to your DS2API endpoint (for example, `https://your-domain.com/v1`)
- Set `apiKey` to your DS2API key (from `config.keys`)
3. Start OpenCode CLI in the project directory (run `opencode` using your installed method).
> Recommended: use the OpenAI-compatible path (`/v1/*`) via `@ai-sdk/openai-compatible` as shown in the example.
> If your client supports `wire_api`, test both `responses` and `chat`; DS2API supports both paths.
> **WebUI auto-build**: On first local startup, if `static/admin` is missing, DS2API will auto-run `npm ci` (only when dependencies are missing) and `npm run build -- --outDir static/admin --emptyOutDir` (requires Node.js). You can also build manually: `./scripts/build-webui.sh`
## Configuration
@@ -286,11 +281,16 @@ cp opencode.json.example opencode.json
],
"model_aliases": {
"gpt-4o": "deepseek-chat",
"gpt-5": "deepseek-chat",
"gpt-5-mini": "deepseek-chat",
"gpt-5-codex": "deepseek-reasoner",
"o3": "deepseek-reasoner"
"o3": "deepseek-reasoner",
"claude-opus-4-6": "deepseek-reasoner",
"gemini-2.5-flash": "deepseek-chat"
},
"compat": {
"wide_input_strict_output": true
"wide_input_strict_output": true,
"strip_reference_markers": true
},
"responses": {
"store_ttl_seconds": 900
@@ -312,7 +312,7 @@ cp opencode.json.example opencode.json
"token_refresh_interval_hours": 6
},
"auto_delete": {
"sessions": false
"mode": "none"
}
}
```
@@ -322,13 +322,14 @@ cp opencode.json.example opencode.json
- `token`: Even if set in `config.json`, it is cleared during load (DS2API does not read persisted tokens from config); runtime tokens are maintained/refreshed in memory only
- `model_aliases`: Map common model names (GPT/Codex/Claude) to DeepSeek models
- `compat.wide_input_strict_output`: Keep `true` (current default policy)
- `toolcall`: Fixed to feature matching + high-confidence early emit, no longer configurable
- `compat.strip_reference_markers`: Keep `true`; it strips reference markers from visible output
- `toolcall`: Legacy field; the current behavior is fixed to feature matching + high-confidence early emit, and any config value is ignored
- `responses.store_ttl_seconds`: In-memory TTL for `/v1/responses/{id}`
- `embeddings.provider`: Embeddings provider (`deterministic/mock/builtin` built-in)
- `claude_mapping`: Maps `fast`/`slow` suffixes to corresponding DeepSeek models (still compatible with `claude_model_mapping`)
- `admin`: Admin panel settings (JWT expiry, password hash, etc.), hot-reloadable via Admin Settings API
- `runtime`: Runtime parameters (concurrency limits, queue sizes, managed token refresh interval), hot-reloadable via Admin Settings API; `account_max_queue=0`/`global_max_inflight=0` means auto-calculate from recommended values, `token_refresh_interval_hours=6` is the default forced re-login interval
- `auto_delete.sessions`: Whether to auto-delete DeepSeek sessions after request completion (default `false`, hot-reloadable via Settings)
- `auto_delete.mode`: How to clean up DeepSeek remote chat records after each request completes. Supported values: `none` (default, no deletion), `single` (delete only the current session), `all` (delete all sessions); legacy `auto_delete.sessions=true` is still treated as `all`
### Environment Variables
@@ -341,28 +342,23 @@ cp opencode.json.example opencode.json
| `DS2API_JWT_EXPIRE_HOURS` | Admin JWT TTL in hours | `24` |
| `DS2API_CONFIG_PATH` | Config file path | `config.json` |
| `DS2API_CONFIG_JSON` | Inline config (JSON or Base64) | — |
| `CONFIG_JSON` | Legacy compatibility config input | — |
| `DS2API_ENV_WRITEBACK` | Auto-write env-backed config to file and transition to file mode (`1/true/yes/on`) | Disabled |
| `DS2API_WASM_PATH` | PoW WASM file path | Auto-detect |
| `DS2API_STATIC_ADMIN_DIR` | Admin static assets dir | `static/admin` |
| `DS2API_AUTO_BUILD_WEBUI` | Auto-build WebUI on startup | Enabled locally, disabled on Vercel |
| `DS2API_ACCOUNT_MAX_INFLIGHT` | Max in-flight requests per account | `2` |
| `DS2API_ACCOUNT_CONCURRENCY` | Alias (legacy compat) | — |
| `DS2API_ACCOUNT_MAX_QUEUE` | Waiting queue limit | `recommended_concurrency` |
| `DS2API_ACCOUNT_QUEUE_SIZE` | Alias (legacy compat) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | Global max in-flight requests | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | Alias (legacy compat) | — |
| `DS2API_VERCEL_INTERNAL_SECRET` | Vercel hybrid streaming internal auth | Falls back to `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | Stream lease TTL seconds | `900` |
| `DS2API_DEV_PACKET_CAPTURE` | Local dev packet capture switch (record recent request/response bodies) | Enabled by default on non-Vercel local runtime |
| `DS2API_DEV_PACKET_CAPTURE_LIMIT` | Number of captured sessions to retain (auto-evict overflow) | `5` |
| `DS2API_DEV_PACKET_CAPTURE_MAX_BODY_BYTES` | Max recorded bytes per captured response body | `2097152` |
| `DS2API_DEV_PACKET_CAPTURE_LIMIT` | Number of captured sessions to retain (auto-evict overflow) | `20` |
| `DS2API_DEV_PACKET_CAPTURE_MAX_BODY_BYTES` | Max recorded bytes per captured response body | `5242880` |
| `VERCEL_TOKEN` | Vercel sync token | — |
| `VERCEL_PROJECT_ID` | Vercel project ID | — |
| `VERCEL_TEAM_ID` | Vercel team ID | — |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel deployment protection bypass for internal Node→Go calls | — |
> Note: when `DS2API_CONFIG_JSON/CONFIG_JSON` is detected, the Admin UI shows mode risk and auto-persistence status (including `DS2API_CONFIG_PATH` and mode-transition hints).
> Note: when `DS2API_CONFIG_JSON` is detected, the Admin UI shows mode risk and auto-persistence status (including `DS2API_CONFIG_PATH` and mode-transition hints).
## Authentication Modes
@@ -394,21 +390,22 @@ Queue limit = DS2API_ACCOUNT_MAX_QUEUE (default = recommended concurrency)
When `tools` is present in the request, DS2API performs anti-leak handling:
1. Toolcall feature matching is enabled only in **non-code-block context** (fenced examples are ignored)
- In non-code-block context, tool JSON may still be recognized even when mixed with normal prose; surrounding prose can remain as text output.
2. `responses` streaming strictly uses official item lifecycle events (`response.output_item.*`, `response.content_part.*`, `response.function_call_arguments.*`)
3. Tool names not declared in the `tools` schema are strictly rejected and will not be emitted as valid tool calls
2. The parser currently targets XML/Markup-family tool syntax (`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml variants); standalone JSON `tool_calls` payloads are not treated as executable calls by default
3. `responses` streaming strictly uses official item lifecycle events (`response.output_item.*`, `response.content_part.*`, `response.function_call_arguments.*`)
4. `responses` supports and enforces `tool_choice` (`auto`/`none`/`required`/forced function); `required` violations return `422` for non-stream and `response.failed` for stream
5. Valid tool call events are only emitted after passing policy validation, preventing invalid tool names from entering the client execution chain
5. The output protocol follows the client request (OpenAI / Claude / Gemini native shapes); model-side prompting can prefer XML, and the compatibility layer handles the protocol-specific translation
> Note: the current parser still prioritizes “parse successfully whenever possible”; hard allow-list rejection for undeclared tool names is not enabled yet.
## Local Dev Packet Capture
This is for debugging issues such as Responses reasoning streaming and tool-call handoff. When enabled, DS2API stores the latest N DeepSeek conversation payload pairs (request body + upstream response body), defaulting to 5 entries with auto-eviction.
This is for debugging issues such as Responses reasoning streaming and tool-call handoff. When enabled, DS2API stores the latest N DeepSeek conversation payload pairs (request body + upstream response body), defaulting to 20 entries with auto-eviction; each response body is capped at 5 MB by default.
Enable example:
```bash
DS2API_DEV_PACKET_CAPTURE=true \
DS2API_DEV_PACKET_CAPTURE_LIMIT=5 \
DS2API_DEV_PACKET_CAPTURE_LIMIT=20 \
go run ./cmd/ds2api
```
@@ -416,6 +413,8 @@ Inspect/clear (Admin JWT required):
- `GET /admin/dev/captures`: list captured items (newest first)
- `DELETE /admin/dev/captures`: clear captured items
- `GET /admin/dev/raw-samples/query?q=keyword&limit=20`: search current in-memory captures by prompt keyword and group `completion + continue` by `chat_session_id`
- `POST /admin/dev/raw-samples/save`: persist a selected capture chain as `tests/raw_stream_samples/<sample-id>/`
Response fields include:
@@ -423,67 +422,10 @@ Response fields include:
- `response_body`: concatenated raw upstream stream body text
- `response_truncated`: whether body-size truncation happened
## Project Structure
The save endpoint can target a chain by `query`, `chain_key`, or `capture_id`. Example:
```text
ds2api/
├── app/ # Unified HTTP handler assembly (shared by local + serverless)
├── cmd/
│ ├── ds2api/ # Local / container entrypoint
│ └── ds2api-tests/ # End-to-end testsuite entrypoint
├── api/
│ ├── index.go # Vercel Serverless Go entry
│ ├── chat-stream.js # Vercel Node.js stream relay
│ └── (rewrite targets in vercel.json)
├── internal/
│ ├── account/ # Account pool and concurrency queue
│ ├── adapter/
│ │ ├── openai/ # OpenAI adapter (incl. tool call parsing, Vercel stream prepare/release)
│ │ ├── claude/ # Claude adapter
│ │ └── gemini/ # Gemini adapter (generateContent / streamGenerateContent)
│ ├── admin/ # Admin API handlers (incl. Settings hot-reload)
│ ├── auth/ # Auth and JWT
│ ├── claudeconv/ # Claude message format conversion
│ ├── compat/ # Go-version compatibility and regression helpers
│ ├── config/ # Config loading, validation, and hot-reload
│ ├── deepseek/ # DeepSeek API client, PoW WASM
│ ├── js/ # Node runtime stream/compat logic
│ ├── devcapture/ # Dev packet capture module
│ ├── format/ # Output formatting
│ ├── prompt/ # Prompt construction
│ ├── server/ # HTTP routing and middleware (chi router)
│ ├── sse/ # SSE parsing utilities
│ ├── stream/ # Unified stream consumption engine
│ ├── testsuite/ # End-to-end testsuite framework and case orchestration
│ ├── translatorcliproxy/ # CLIProxy bridge and stream writer components
│ ├── util/ # Common utilities
│ ├── version/ # Version parsing/comparison and tag normalization
│ └── webui/ # WebUI static file serving and auto-build
├── webui/ # React WebUI source (Vite + Tailwind)
│ └── src/
│ ├── app/ # Routing, auth, config state
│ ├── features/ # Feature modules (account/settings/vercel/apiTester)
│ ├── components/ # Shared UI pieces (login/landing, etc.)
│ └── locales/ # Language packs (zh.json / en.json)
├── scripts/
│ └── build-webui.sh # Manual WebUI build script
├── tests/
│ ├── compat/ # Compatibility fixtures and expected outputs
│ ├── node/ # Node-side unit tests (chat-stream / tool-sieve)
│ └── scripts/ # Unified test script entrypoints (unit/e2e)
├── docs/ # Deployment / contributing / testing docs
├── static/admin/ # WebUI build output (not committed to Git)
├── .github/
│ ├── workflows/ # GitHub Actions (quality gates + release automation)
│ ├── ISSUE_TEMPLATE/ # Issue templates
│ └── PULL_REQUEST_TEMPLATE.md
├── config.example.json # Config file template
├── .env.example # Environment variable template
├── Dockerfile # Multi-stage build (WebUI + Go)
├── docker-compose.yml # Production Docker Compose
├── docker-compose.dev.yml # Development Docker Compose
├── vercel.json # Vercel routing and build config
└── go.mod / go.sum # Go module dependencies
```json
{"query":"Guangzhou weather","sample_id":"gz-weather-from-memory"}
```
## Documentation Index

View File

@@ -1 +1 @@
3.0.0
3.5.1

View File

@@ -3,9 +3,17 @@ package app
import (
"net/http"
"ds2api/internal/config"
"ds2api/internal/server"
)
func NewHandler() http.Handler {
return server.NewApp().Router
app, err := server.NewApp()
if err != nil {
config.Logger.Error("[app] init failed", "error", err)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
server.WriteUnhandledError(w, err)
})
}
return app.Router
}

View File

@@ -30,8 +30,8 @@ func main() {
opts.Timeout = time.Duration(timeoutSeconds) * time.Second
if err := testsuite.Run(context.Background(), opts); err != nil {
fmt.Fprintln(os.Stderr, err.Error())
_, _ = fmt.Fprintln(os.Stderr, err.Error())
os.Exit(1)
}
fmt.Fprintln(os.Stdout, "testsuite completed successfully")
_, _ = fmt.Fprintln(os.Stdout, "testsuite completed successfully")
}

View File

@@ -18,9 +18,17 @@ import (
)
func main() {
if err := config.LoadDotEnv(); err != nil {
config.Logger.Warn("[dotenv] load failed", "error", err)
}
config.RefreshLogger()
webui.EnsureBuiltOnStartup()
_ = auth.AdminKey()
app := server.NewApp()
app, err := server.NewApp()
if err != nil {
config.Logger.Error("server initialization failed", "error", err)
os.Exit(1)
}
port := strings.TrimSpace(os.Getenv("PORT"))
if port == "" {
port = "5001"

View File

@@ -28,11 +28,8 @@
"o3": "deepseek-reasoner"
},
"compat": {
"wide_input_strict_output": true
},
"toolcall": {
"mode": "feature_match",
"early_emit_confidence": "high"
"wide_input_strict_output": true,
"strip_reference_markers": true
},
"responses": {
"store_ttl_seconds": 900
@@ -50,9 +47,10 @@
"runtime": {
"account_max_inflight": 2,
"account_max_queue": 0,
"global_max_inflight": 0
"global_max_inflight": 0,
"token_refresh_interval_hours": 6
},
"auto_delete": {
"sessions": false
"mode": "none"
}
}

View File

@@ -16,7 +16,8 @@ services:
container_name: ds2api-dev
command: ["go", "run", "./cmd/ds2api"]
ports:
- "${PORT:-5001}:${PORT:-5001}"
# Host port is configurable via DS2API_HOST_PORT; container port stays fixed at 5001.
- "${DS2API_HOST_PORT:-6011}:5001"
env_file:
- .env
environment:

View File

@@ -1,14 +1,16 @@
services:
ds2api:
image: ghcr.io/cjackhwang/ds2api:latest
container_name: ds2api
restart: always
ports:
- "6011:5001"
volumes:
- ./config.json:/app/config.json # 配置文件
- ./.env:/app/.env # 环境变量
environment:
- TZ=Asia/Shanghai
- LOG_LEVEL=INFO
- DS2API_ADMIN_KEY=${DS2API_ADMIN_KEY:-ds2api}
container_name: ds2api
restart: always
env_file:
- .env
ports:
# Host port is configurable via DS2API_HOST_PORT; container port stays fixed at 5001.
- "${DS2API_HOST_PORT:-6011}:5001"
volumes:
- ./config.json:/app/config.json # 配置文件
environment:
- TZ=Asia/Shanghai
- LOG_LEVEL=INFO
- DS2API_ADMIN_KEY=${DS2API_ADMIN_KEY:-ds2api}

136
docs/ARCHITECTURE.en.md Normal file
View File

@@ -0,0 +1,136 @@
# DS2API Architecture & Project Layout
Language: [中文](ARCHITECTURE.md) | [English](ARCHITECTURE.en.md)
> This file is the single architecture source for directory layout, module boundaries, and execution flow.
## 1. Top-level Layout (expanded)
> Notes: this is the **fully expanded** project directory list (excluding metadata/dependency dirs such as `.git/` and `webui/node_modules/`), with each folder annotated by purpose.
```text
ds2api/
├── .github/ # GitHub collaboration and CI config
│ ├── ISSUE_TEMPLATE/ # Issue templates
│ └── workflows/ # GitHub Actions workflows
├── api/ # Serverless entrypoints (Vercel Go/Node)
├── app/ # Application-level handler assembly
├── cmd/ # Executable entrypoints
│ ├── ds2api/ # Main service bootstrap
│ └── ds2api-tests/ # E2E testsuite CLI bootstrap
├── docs/ # Project documentation
├── internal/ # Core implementation (non-public packages)
│ ├── account/ # Account pool, inflight slots, waiting queue
│ ├── adapter/ # Multi-protocol adapters
│ │ ├── claude/ # Claude protocol adapter
│ │ ├── gemini/ # Gemini protocol adapter
│ │ └── openai/ # OpenAI adapter and shared execution core
│ ├── admin/ # Admin API (config/accounts/ops)
│ ├── auth/ # Auth/JWT/credential resolution
│ ├── claudeconv/ # Claude message conversion helpers
│ ├── compat/ # Compatibility and regression helpers
│ ├── config/ # Config loading/validation/hot reload
│ ├── deepseek/ # DeepSeek upstream client capabilities
│ │ └── transport/ # DeepSeek transport details
│ ├── devcapture/ # Dev capture and troubleshooting
│ ├── format/ # Response formatting layer
│ │ ├── claude/ # Claude output formatting
│ │ └── openai/ # OpenAI output formatting
│ ├── js/ # Node runtime related logic
│ │ ├── chat-stream/ # Node streaming bridge
│ │ ├── helpers/ # JS helper modules
│ │ │ └── stream-tool-sieve/ # JS implementation of tool sieve
│ │ └── shared/ # Shared semantics between Go/Node
│ ├── prompt/ # Prompt composition
│ ├── rawsample/ # Raw sample read/write and management
│ ├── server/ # Router and middleware assembly
│ ├── sse/ # SSE parsing utilities
│ ├── stream/ # Unified stream consumption engine
│ ├── testsuite/ # Testsuite execution framework
│ ├── textclean/ # Text cleanup
│ ├── toolcall/ # Tool-call parsing and repair
│ ├── translatorcliproxy/ # Cross-protocol translation bridge
│ ├── util/ # Shared utility helpers
│ ├── version/ # Version query/compare
│ └── webui/ # WebUI static hosting logic
├── plans/ # Stage plans and manual QA records
├── pow/ # PoW standalone implementation + benchmarks
├── scripts/ # Build/release helper scripts
├── tests/ # Test assets and scripts
│ ├── compat/ # Compatibility fixtures + expected outputs
│ │ ├── expected/ # Expected output samples
│ │ └── fixtures/ # Fixture inputs
│ │ ├── sse_chunks/ # SSE chunk fixtures
│ │ └── toolcalls/ # Tool-call fixtures
│ ├── node/ # Node unit tests
│ ├── raw_stream_samples/ # Upstream raw SSE samples
│ │ ├── content-filter-trigger-20260405-jwt3/ # Content-filter terminal sample
│ │ ├── continue-thinking-snapshot-replay-20260405/ # Continue-thinking sample
│ │ ├── guangzhou-weather-reasoner-search-20260404/ # Search/reference sample
│ │ ├── markdown-format-example-20260405/ # Markdown sample
│ │ └── markdown-format-example-20260405-spacefix/ # Space-fix sample
│ ├── scripts/ # Test entry scripts
│ └── tools/ # Testing helper tools
└── webui/ # React admin console source
├── public/ # Static assets
└── src/ # Frontend source code
├── app/ # Routing/state scaffolding
├── components/ # Shared UI components
├── features/ # Feature modules
│ ├── account/ # Account management page
│ ├── apiTester/ # API tester page
│ ├── settings/ # Settings page
│ └── vercel/ # Vercel sync page
├── layout/ # Layout components
├── locales/ # i18n strings
└── utils/ # Frontend utilities
```
## 2. Primary Request Flow
```mermaid
flowchart LR
C[Client/SDK] --> R[internal/server/router.go]
R --> OA[OpenAI Adapter]
R --> CA[Claude Adapter]
R --> GA[Gemini Adapter]
R --> AD[Admin API]
CA --> BR[translatorcliproxy]
GA --> BR
BR --> CORE[internal/adapter/openai ChatCompletions]
OA --> CORE
CORE --> AUTH[internal/auth + config key/account resolver]
CORE --> POOL[internal/account queue + concurrency]
CORE --> TOOL[internal/toolcall parser + sieve]
CORE --> DS[internal/deepseek client]
DS --> U[DeepSeek upstream]
```
## 3. Responsibilities in `internal/`
- `internal/server`: router tree + middlewares (health, protocol routes, Admin/WebUI).
- `internal/adapter/openai`: shared execution core (chat/responses/embeddings + tool semantics).
- `internal/adapter/{claude,gemini}`: protocol wrappers only (no duplicated upstream execution).
- `internal/translatorcliproxy`: structure translation between Claude/Gemini and OpenAI.
- `internal/deepseek`: upstream request/session/PoW/SSE handling.
- `internal/stream` + `internal/sse`: stream parsing and incremental assembly.
- `internal/toolcall`: XML/Markup-family tool-call parsing + anti-leak sieve (`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml variants).
- `internal/admin`: config/accounts/vercel sync/version/dev-capture endpoints.
- `internal/config`: config loading/validation + runtime settings hot-reload.
- `internal/account`: managed account pool, inflight slots, waiting queue.
## 4. WebUI Runtime Relation
- `webui/` stores frontend source (Vite + React).
- Runtime serves static output from `static/admin`.
- On first local startup, if `static/admin` is missing, DS2API may auto-build it (Node.js required).
## 5. Documentation Split Strategy
- Onboarding & quick start: `README.MD` / `README.en.md`
- Architecture & layout: `docs/ARCHITECTURE*.md` (this file)
- API contracts: `API.md` / `API.en.md`
- Deployment/testing/contributing: `docs/DEPLOY*`, `docs/TESTING.md`, `docs/CONTRIBUTING*`
- Deep topics: `docs/toolcall-semantics.md`, `docs/DeepSeekSSE行为结构说明-2026-04-05.md`

136
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,136 @@
# DS2API 架构与项目结构说明
语言 / Language: [中文](ARCHITECTURE.md) | [English](ARCHITECTURE.en.md)
> 本文档用于集中维护“代码目录结构 + 模块边界 + 主链路调用关系”。
## 1. 顶层目录结构(展开)
> 说明:以下为仓库内业务相关目录的**完整展开**(排除 `.git/` 与 `webui/node_modules/` 这类依赖/元数据目录),并标注每个文件夹作用。
```text
ds2api/
├── .github/ # GitHub 协作与 CI 配置
│ ├── ISSUE_TEMPLATE/ # Issue 模板
│ └── workflows/ # GitHub Actions 工作流
├── api/ # Serverless 入口Vercel Go/Node
├── app/ # 应用级 handler 装配层
├── cmd/ # 可执行程序入口
│ ├── ds2api/ # 主服务启动入口
│ └── ds2api-tests/ # E2E 测试集 CLI 入口
├── docs/ # 项目文档目录
├── internal/ # 核心业务实现(不对外暴露)
│ ├── account/ # 账号池、并发槽位、等待队列
│ ├── adapter/ # 多协议适配层
│ │ ├── claude/ # Claude 协议适配
│ │ ├── gemini/ # Gemini 协议适配
│ │ └── openai/ # OpenAI 协议与统一执行核心
│ ├── admin/ # Admin API配置/账号/运维)
│ ├── auth/ # 鉴权/JWT/凭证解析
│ ├── claudeconv/ # Claude 消息格式转换工具
│ ├── compat/ # 兼容性辅助与回归支持
│ ├── config/ # 配置加载、校验、热更新
│ ├── deepseek/ # DeepSeek 上游客户端能力
│ │ └── transport/ # DeepSeek 传输层细节
│ ├── devcapture/ # 开发抓包与调试采集
│ ├── format/ # 响应格式化层
│ │ ├── claude/ # Claude 输出格式化
│ │ └── openai/ # OpenAI 输出格式化
│ ├── js/ # Node Runtime 相关逻辑
│ │ ├── chat-stream/ # Node 流式输出桥接
│ │ ├── helpers/ # JS 辅助函数
│ │ │ └── stream-tool-sieve/ # Tool sieve JS 实现
│ │ └── shared/ # Go/Node 共用语义片段
│ ├── prompt/ # Prompt 组装
│ ├── rawsample/ # raw sample 读写与管理
│ ├── server/ # 路由与中间件装配
│ ├── sse/ # SSE 解析工具
│ ├── stream/ # 统一流式消费引擎
│ ├── testsuite/ # 测试集执行框架
│ ├── textclean/ # 文本清洗
│ ├── toolcall/ # 工具调用解析与修复
│ ├── translatorcliproxy/ # 多协议互转桥
│ ├── util/ # 通用工具函数
│ ├── version/ # 版本查询/比较
│ └── webui/ # WebUI 静态托管相关逻辑
├── plans/ # 阶段计划与人工验收记录
├── pow/ # PoW 独立实现与基准
├── scripts/ # 构建/发布/辅助脚本
├── tests/ # 测试资源与脚本
│ ├── compat/ # 兼容性夹具与期望输出
│ │ ├── expected/ # 预期结果样本
│ │ └── fixtures/ # 测试输入夹具
│ │ ├── sse_chunks/ # SSE chunk 夹具
│ │ └── toolcalls/ # toolcall 夹具
│ ├── node/ # Node 单元测试
│ ├── raw_stream_samples/ # 上游原始 SSE 样本
│ │ ├── content-filter-trigger-20260405-jwt3/ # 风控终态样本
│ │ ├── continue-thinking-snapshot-replay-20260405/ # continue 样本
│ │ ├── guangzhou-weather-reasoner-search-20260404/ # 搜索+引用样本
│ │ ├── markdown-format-example-20260405/ # Markdown 样本
│ │ └── markdown-format-example-20260405-spacefix/ # 空格修复样本
│ ├── scripts/ # 测试脚本入口
│ └── tools/ # 测试辅助工具
└── webui/ # React 管理台源码
├── public/ # 静态资源
└── src/ # 前端源码
├── app/ # 路由/状态框架
├── components/ # 共享组件
├── features/ # 功能模块
│ ├── account/ # 账号管理页面
│ ├── apiTester/ # API 测试页面
│ ├── settings/ # 设置页面
│ └── vercel/ # Vercel 同步页面
├── layout/ # 布局组件
├── locales/ # 国际化文案
└── utils/ # 前端工具函数
```
## 2. 请求主链路
```mermaid
flowchart LR
C[Client/SDK] --> R[internal/server/router.go]
R --> OA[OpenAI Adapter]
R --> CA[Claude Adapter]
R --> GA[Gemini Adapter]
R --> AD[Admin API]
CA --> BR[translatorcliproxy]
GA --> BR
BR --> CORE[internal/adapter/openai ChatCompletions]
OA --> CORE
CORE --> AUTH[internal/auth + config key/account resolver]
CORE --> POOL[internal/account queue + concurrency]
CORE --> TOOL[internal/toolcall parser + sieve]
CORE --> DS[internal/deepseek client]
DS --> U[DeepSeek upstream]
```
## 3. internal/ 子模块职责
- `internal/server`路由树和中间件挂载健康检查、协议入口、Admin/WebUI
- `internal/adapter/openai`统一执行内核chat/responses/embeddings 与 tool calling 语义)。
- `internal/adapter/{claude,gemini}`:协议输入输出适配,不重复实现上游调用逻辑。
- `internal/translatorcliproxy`Claude/Gemini 与 OpenAI 结构互转。
- `internal/deepseek`上游请求、会话、PoW、SSE 消费。
- `internal/stream` + `internal/sse`:流式解析与增量处理。
- `internal/toolcall`:以 XML/Markup 家族为核心的工具调用解析与防泄漏筛分(`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml 变体)。
- `internal/admin`配置管理、账号管理、Vercel 同步、版本检查、开发抓包。
- `internal/config`:配置加载、校验、运行时 settings 热更新。
- `internal/account`:托管账号池、并发槽位、等待队列。
## 4. WebUI 与运行时关系
- `webui/` 是前端源码Vite + React
- 运行时托管目录是 `static/admin`(构建产物)。
- 本地首次启动若 `static/admin` 缺失,会尝试自动构建(依赖 Node.js
## 5. 文档拆分策略
- 总览与快速开始:`README.MD` / `README.en.md`
- 架构与目录:`docs/ARCHITECTURE*.md`(本文件)
- 接口协议:`API.md` / `API.en.md`
- 部署、测试、贡献:`docs/DEPLOY*``docs/TESTING.md``docs/CONTRIBUTING*`
- 专题:`docs/toolcall-semantics.md``docs/DeepSeekSSE行为结构说明-2026-04-05.md`

View File

@@ -9,7 +9,7 @@ Thanks for your interest in contributing to DS2API!
### Prerequisites
- Go 1.26+
- Node.js 20+ (for WebUI development)
- Node.js `20.19+` or `22.12+` (for WebUI development)
- npm (bundled with Node.js)
### Backend Development
@@ -25,7 +25,8 @@ cp config.example.json config.json
# 3. Run backend
go run ./cmd/ds2api
# Default: http://localhost:5001
# Local access: http://127.0.0.1:5001
# Actual bind: 0.0.0.0:5001, so LAN access is available via your private IP
```
### Frontend Development (WebUI)
@@ -40,6 +41,7 @@ npm install
# 3. Start dev server (hot reload)
npm run dev
# Default: http://localhost:5173, auto-proxies API to backend
# host: 0.0.0.0 is not configured, so LAN access is not enabled by default
```
WebUI tech stack:
@@ -57,7 +59,7 @@ docker-compose -f docker-compose.dev.yml up
| Language | Standards |
| --- | --- |
| **Go** | Run `gofmt` and ensure `go test ./...` passes before committing |
| **Go** | Run `./scripts/lint.sh` (gofmt + golangci-lint) and ensure `go test ./...` passes before committing |
| **JavaScript/React** | Follow existing project style (functional components) |
| **Commit messages** | Use semantic prefixes: `feat:`, `fix:`, `docs:`, `refactor:`, `style:`, `perf:`, `chore:` |
@@ -92,58 +94,12 @@ Manually build WebUI to `static/admin/`:
## Project Structure
```text
ds2api/
├── app/ # Shared HTTP handler assembly (local + serverless)
├── cmd/
│ ├── ds2api/ # Local/container entrypoint
│ └── ds2api-tests/ # End-to-end testsuite entrypoint
├── api/
│ ├── index.go # Vercel Serverless Go entry
│ ├── chat-stream.js # Vercel Node.js stream relay
│ └── (rewrite targets in vercel.json)
├── internal/
│ ├── account/ # Account pool and concurrency queue
│ ├── adapter/
│ │ ├── openai/ # OpenAI adapter
│ │ ├── claude/ # Claude adapter
│ │ └── gemini/ # Gemini adapter
│ ├── admin/ # Admin API handlers
│ ├── auth/ # Auth and JWT
│ ├── claudeconv/ # Claude message conversion
│ ├── compat/ # Go-version compatibility and regression helpers
│ ├── config/ # Config loading, validation, and hot-reload
│ ├── deepseek/ # DeepSeek client, PoW WASM
│ ├── js/ # Node runtime stream/compat logic
│ ├── devcapture/ # Dev packet capture
│ ├── format/ # Output formatting
│ ├── prompt/ # Prompt building
│ ├── server/ # HTTP routing (chi router)
│ ├── sse/ # SSE parsing utilities
│ ├── stream/ # Unified stream consumption engine
│ ├── testsuite/ # Testsuite framework and scenario orchestration
│ ├── translatorcliproxy/ # CLIProxy bridge and stream writer
│ ├── util/ # Common utilities
│ ├── version/ # Version parsing and comparison
│ └── webui/ # WebUI static hosting
├── webui/ # React WebUI source
│ └── src/
│ ├── app/ # Routing, auth, config state
│ ├── features/ # Feature modules
│ ├── components/ # Shared components
│ └── locales/ # Language packs
├── scripts/ # Build and test scripts
├── tests/
│ ├── compat/ # Compatibility fixtures and expected outputs
│ ├── node/ # Node-side unit tests
│ └── scripts/ # Test script entrypoints (unit/e2e)
├── plans/ # Plans, gates, and manual smoke-test records
├── static/admin/ # WebUI build output (not committed)
├── Dockerfile # Multi-stage build
├── docker-compose.yml # Production
├── docker-compose.dev.yml # Development
└── vercel.json # Vercel config
```
To avoid documentation drift, directory layout and module responsibilities were moved to:
- [docs/ARCHITECTURE.en.md](./ARCHITECTURE.en.md)
- [docs/README.md](./README.md)
Before contributing, review the architecture doc sections for request flow and `internal/` module boundaries.
## Reporting Issues

View File

@@ -9,7 +9,7 @@
### 前置要求
- Go 1.26+
- Node.js 20+WebUI 开发时)
- Node.js `20.19+``22.12+`WebUI 开发时)
- npm随 Node.js 提供)
### 后端开发
@@ -25,7 +25,8 @@ cp config.example.json config.json
# 3. 启动后端
go run ./cmd/ds2api
# 默认监听 http://localhost:5001
# 本地访问 http://127.0.0.1:5001
# 实际绑定 0.0.0.0:5001可通过局域网 IP 访问
```
### 前端开发WebUI
@@ -40,6 +41,7 @@ npm install
# 3. 启动开发服务器(热更新)
npm run dev
# 默认监听 http://localhost:5173自动代理 API 到后端
# 当前未配置 host: 0.0.0.0,因此默认不对局域网开放
```
WebUI 技术栈:
@@ -57,7 +59,7 @@ docker-compose -f docker-compose.dev.yml up
| 语言 | 规范 |
| --- | --- |
| **Go** | 提交前运行 `gofmt`确保 `go test ./...` 通过 |
| **Go** | 提交前运行 `./scripts/lint.sh`(包含 gofmt+golangci-lint确保 `go test ./...` 通过 |
| **JavaScript/React** | 保持现有代码风格(函数组件) |
| **提交信息** | 使用语义化前缀:`feat:``fix:``docs:``refactor:``style:``perf:``chore:` |
@@ -92,58 +94,12 @@ docker-compose -f docker-compose.dev.yml up
## 项目结构
```text
ds2api/
├── app/ # 统一 HTTP Handler 装配(本地 + Serverless
├── cmd/
│ ├── ds2api/ # 本地/容器启动入口
│ └── ds2api-tests/ # 端到端测试集入口
├── api/
│ ├── index.go # Vercel Serverless Go 入口
│ ├── chat-stream.js # Vercel Node.js 流式转发
│ └── (rewrite targets in vercel.json)
├── internal/
│ ├── account/ # 账号池与并发队列
│ ├── adapter/
│ │ ├── openai/ # OpenAI 兼容适配器
│ │ ├── claude/ # Claude 兼容适配器
│ │ └── gemini/ # Gemini 兼容适配器
│ ├── admin/ # Admin API handlers
│ ├── auth/ # 鉴权与 JWT
│ ├── claudeconv/ # Claude 消息格式转换
│ ├── compat/ # Go 版本兼容与回归测试辅助
│ ├── config/ # 配置加载、校验与热更新
│ ├── deepseek/ # DeepSeek 客户端、PoW WASM
│ ├── js/ # Node 运行时流式/兼容逻辑
│ ├── devcapture/ # 开发抓包
│ ├── format/ # 输出格式化
│ ├── prompt/ # Prompt 构建
│ ├── server/ # HTTP 路由chi router
│ ├── sse/ # SSE 解析工具
│ ├── stream/ # 统一流式消费引擎
│ ├── testsuite/ # 测试集框架与场景编排
│ ├── translatorcliproxy/ # CLIProxy 桥接与流式写入
│ ├── util/ # 通用工具
│ ├── version/ # 版本解析与比较
│ └── webui/ # WebUI 静态托管
├── webui/ # React WebUI 源码
│ └── src/
│ ├── app/ # 路由、鉴权、配置状态
│ ├── features/ # 业务功能模块
│ ├── components/ # 通用组件
│ └── locales/ # 语言包
├── scripts/ # 构建与测试脚本
├── tests/
│ ├── compat/ # 兼容夹具与期望输出
│ ├── node/ # Node 侧单元测试
│ └── scripts/ # 测试脚本入口unit/e2e
├── plans/ # 计划、门禁和手工烟测记录
├── static/admin/ # WebUI 构建产物(不提交)
├── Dockerfile # 多阶段构建
├── docker-compose.yml # 生产环境
├── docker-compose.dev.yml # 开发环境
└── vercel.json # Vercel 配置
```
为避免与其他文档重复维护,目录结构与模块职责已迁移到:
- [docs/ARCHITECTURE.md](./ARCHITECTURE.md)
- [docs/README.md](./README.md)
贡献前建议先阅读架构文档中的“请求主链路”和 `internal/` 模块职责,再定位改动范围。
## 问题反馈

View File

@@ -4,15 +4,18 @@ Language: [中文](DEPLOY.md) | [English](DEPLOY.en.md)
This guide covers all deployment methods for the current Go-based codebase.
Doc map: [Index](./README.md) | [Architecture](./ARCHITECTURE.en.md) | [API](../API.en.md) | [Testing](./TESTING.md)
---
## Table of Contents
- [Recommended deployment priority](#recommended-deployment-priority)
- [Prerequisites](#0-prerequisites)
- [1. Local Run](#1-local-run)
- [2. Docker Deployment](#2-docker-deployment)
- [1. Download Release Binaries](#1-download-release-binaries)
- [2. Docker / GHCR Deployment](#2-docker--ghcr-deployment)
- [3. Vercel Deployment](#3-vercel-deployment)
- [4. Download Release Binaries](#4-download-release-binaries)
- [4. Local Run from Source](#4-local-run-from-source)
- [5. Reverse Proxy (Nginx)](#5-reverse-proxy-nginx)
- [6. Linux systemd Service](#6-linux-systemd-service)
- [7. Post-Deploy Checks](#7-post-deploy-checks)
@@ -20,19 +23,29 @@ This guide covers all deployment methods for the current Go-based codebase.
---
## Recommended deployment priority
Recommended order when choosing a deployment method:
1. **Download and run release binaries**: the easiest path for most users because the artifacts are already built.
2. **Docker / GHCR image deployment**: suitable for containerized, orchestrated, or cloud environments.
3. **Vercel deployment**: suitable if you already use Vercel and accept its platform constraints.
4. **Run from source / build locally**: suitable for development, debugging, or when you need to modify the code yourself.
---
## 0. Prerequisites
| Dependency | Minimum Version | Notes |
| --- | --- | --- |
| Go | 1.26+ | Build backend |
| Node.js | 20+ | Only needed to build WebUI locally |
| Node.js | `20.19+` or `22.12+` | Only needed to build WebUI locally |
| npm | Bundled with Node.js | Install WebUI dependencies |
Config source (choose one):
- **File**: `config.json` (recommended for local/Docker)
- **Environment variable**: `DS2API_CONFIG_JSON` (recommended for Vercel; supports raw JSON or Base64)
- Compatibility note: `CONFIG_JSON` is the legacy fallback variable; `DS2API_CONFIG_JSON` may also contain raw JSON directly
Unified recommendation (best practice):
@@ -47,76 +60,67 @@ Use `config.json` as the single source of truth:
---
## 1. Local Run
## 1. Download Release Binaries
### 1.1 Basic Steps
Built-in GitHub Actions workflow: `.github/workflows/release-artifacts.yml`
- **Trigger**: only on Release `published` (no build on normal push)
- **Outputs**: multi-platform binary archives + `sha256sums.txt`
- **Container publishing**: GHCR only (`ghcr.io/cjackhwang/ds2api`)
| Platform | Architecture | Format |
| --- | --- | --- |
| Linux | amd64, arm64 | `.tar.gz` |
| macOS | amd64, arm64 | `.tar.gz` |
| Windows | amd64 | `.zip` |
Each archive includes:
- `ds2api` executable (`ds2api.exe` on Windows)
- `static/admin/` (built WebUI assets)
- `config.example.json`, `.env.example`
- `README.MD`, `README.en.md`, `LICENSE`
### Usage
```bash
# Clone
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 1. Download the archive for your platform
# 2. Extract
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
# Copy and edit config
# 3. Configure
cp config.example.json config.json
# Open config.json and fill in:
# - keys: your API access keys
# - accounts: DeepSeek accounts (email or mobile + password)
# Edit config.json
# Start
go run ./cmd/ds2api
```
Default address: `http://0.0.0.0:5001` (override with `PORT`).
### 1.2 WebUI Build
On first local startup, if `static/admin/` is missing, DS2API will automatically attempt to build the WebUI (requires Node.js/npm; when dependencies are missing it runs `npm ci` first, then `npm run build -- --outDir static/admin --emptyOutDir`).
Manual build:
```bash
./scripts/build-webui.sh
```
Or step by step:
```bash
cd webui
npm install
npm run build
# Output goes to static/admin/
```
Control auto-build via environment variable:
```bash
# Disable auto-build
DS2API_AUTO_BUILD_WEBUI=false go run ./cmd/ds2api
# Force enable auto-build
DS2API_AUTO_BUILD_WEBUI=true go run ./cmd/ds2api
```
### 1.3 Compile to Binary
```bash
go build -o ds2api ./cmd/ds2api
# 4. Start
./ds2api
```
### Maintainer Release Flow
1. Create and publish a GitHub Release (with tag, for example `vX.Y.Z`)
2. Wait for the `Release Artifacts` workflow to complete
3. Download the matching archive from Release Assets
---
## 2. Docker Deployment
## 2. Docker / GHCR Deployment
### 2.1 Basic Steps
```bash
# Pull prebuilt image
docker pull ghcr.io/cjackhwang/ds2api:latest
# Copy env template and config file
cp .env.example .env
cp config.example.json config.json
# Edit .env and set at least:
# DS2API_ADMIN_KEY=your-admin-key
# Optionally set the host port:
# DS2API_HOST_PORT=6011
# Start
docker-compose up -d
@@ -125,7 +129,13 @@ docker-compose up -d
docker-compose logs -f
```
The default `docker-compose.yml` maps host port `6011` to container port `5001`. If you want `5001` exposed directly, adjust the `ports` mapping.
The default `docker-compose.yml` directly uses `ghcr.io/cjackhwang/ds2api:latest` and maps host port `6011` to container port `5001`. If you want `5001` exposed directly, set `DS2API_HOST_PORT=5001` (or adjust the `ports` mapping).
If you want a pinned version instead of `latest`, you can also pull a specific tag directly:
```bash
docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
```
### 2.2 Update
@@ -138,7 +148,7 @@ docker-compose up -d --build
The `Dockerfile` now provides two image paths:
1. **Default local/dev path (`runtime-from-source`)**: a three-stage build (WebUI build + Go build + runtime).
2. **Release path (`runtime-from-dist`)**: CI first creates `dist/ds2api_<tag>_linux_<arch>.tar.gz`, then Docker directly reuses the binary and `static/admin` assets from those release archives, without running `npm build`/`go build` again.
2. **Release path (`runtime-from-dist`)**: the release workflow first creates tag-named release archives, then copies the Linux bundles to `dist/docker-input/linux_amd64.tar.gz` / `linux_arm64.tar.gz`; Docker consumes those prepared inputs directly, without rerunning `npm build`/`go build`.
The release path keeps Docker images aligned with release archives and reduces duplicate build work.
@@ -198,10 +208,10 @@ Notes:
2. **Import** the project on Vercel
3. **Set environment variables** (minimum required: one variable):
| Variable | Description |
| --- | --- |
| `DS2API_ADMIN_KEY` | Admin key (required) |
| `DS2API_CONFIG_JSON` | Config content, raw JSON or Base64 (optional, recommended) |
| Variable | Description |
| --- | --- |
| `DS2API_ADMIN_KEY` | Admin key (required) |
| `DS2API_CONFIG_JSON` | Config content, raw JSON or Base64 (optional, recommended) |
4. **Deploy**
@@ -244,11 +254,8 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # optional for personal accounts
| Variable | Description | Default |
| --- | --- | --- |
| `DS2API_ACCOUNT_MAX_INFLIGHT` | Per-account inflight limit | `2` |
| `DS2API_ACCOUNT_CONCURRENCY` | Alias (legacy compat) | — |
| `DS2API_ACCOUNT_MAX_QUEUE` | Waiting queue limit | `recommended_concurrency` |
| `DS2API_ACCOUNT_QUEUE_SIZE` | Alias (legacy compat) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | Global inflight limit | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | Alias (legacy compat) | — |
| `DS2API_ENV_WRITEBACK` | When `DS2API_CONFIG_JSON` is present, auto-write to `DS2API_CONFIG_PATH` and switch to file-backed mode after success (`1/true/yes/on`) | Disabled |
| `DS2API_VERCEL_INTERNAL_SECRET` | Hybrid streaming internal auth | Falls back to `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | Stream lease TTL | `900` |
@@ -314,7 +321,7 @@ Error: Command failed: go build -ldflags -s -w -o .../bootstrap ...
1. Open Vercel Project Settings → Build and Development Settings
2. **Clear** custom Go Build Flags / Build Command (recommended)
3. If ldflags must be used, set `-ldflags="-s -w"` (ensure it's one argument)
4. Verify `go.mod` uses a supported version (currently `go 1.24`)
4. Verify `go.mod` uses a supported version (currently `go 1.26.0`)
5. Redeploy (recommended: clear cache)
#### Internal Package Import Error
@@ -350,58 +357,61 @@ If API responses return Vercel HTML `Authentication Required`:
---
## 4. Download Release Binaries
## 4. Local Run from Source
Built-in GitHub Actions workflow: `.github/workflows/release-artifacts.yml`
- **Trigger**: only on Release `published` (no build on normal push)
- **Outputs**: multi-platform binary archives + `sha256sums.txt`
- **Container publishing**: GHCR only (`ghcr.io/cjackhwang/ds2api`)
| Platform | Architecture | Format |
| --- | --- | --- |
| Linux | amd64, arm64 | `.tar.gz` |
| macOS | amd64, arm64 | `.tar.gz` |
| Windows | amd64 | `.zip` |
Each archive includes:
- `ds2api` executable (`ds2api.exe` on Windows)
- `static/admin/` (built WebUI assets)
- `sha3_wasm_bg.7b9ca65ddd.wasm` (optional; binary has embedded fallback)
- `config.example.json`, `.env.example`
- `README.MD`, `README.en.md`, `LICENSE`
### Usage
### 4.1 Basic Steps
```bash
# 1. Download the archive for your platform
# 2. Extract
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
# Clone
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 3. Configure
# Copy and edit config
cp config.example.json config.json
# Edit config.json
# Open config.json and fill in:
# - keys: your API access keys
# - accounts: DeepSeek accounts (email or mobile + password)
# 4. Start
./ds2api
# Start
go run ./cmd/ds2api
```
### Maintainer Release Flow
Default local access URL: `http://127.0.0.1:5001`; the server actually binds to `0.0.0.0:5001` (override with `PORT`).
1. Create and publish a GitHub Release (with tag, for example `vX.Y.Z`)
2. Wait for the `Release Artifacts` workflow to complete
3. Download the matching archive from Release Assets
### 4.2 WebUI Build
### Pull from GHCR (Optional)
On first local startup, if `static/admin/` is missing, DS2API will automatically attempt to build the WebUI (requires Node.js/npm; when dependencies are missing it runs `npm ci` first, then `npm run build -- --outDir static/admin --emptyOutDir`).
Manual build:
```bash
# latest
docker pull ghcr.io/cjackhwang/ds2api:latest
./scripts/build-webui.sh
```
# specific version (example)
docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
Or step by step:
```bash
cd webui
npm install
npm run build
# Output goes to static/admin/
```
Control auto-build via environment variable:
```bash
# Disable auto-build
DS2API_AUTO_BUILD_WEBUI=false go run ./cmd/ds2api
# Force enable auto-build
DS2API_AUTO_BUILD_WEBUI=true go run ./cmd/ds2api
```
### 4.3 Compile to Binary
```bash
go build -o ds2api ./cmd/ds2api
./ds2api
```
---
@@ -458,8 +468,6 @@ server {
# Copy compiled binary and related files to target directory
sudo mkdir -p /opt/ds2api
sudo cp ds2api config.json /opt/ds2api/
# Optional: if you want to use an external WASM file (override the embedded one, from a release package or build output)
# sudo cp /path/to/sha3_wasm_bg.7b9ca65ddd.wasm /opt/ds2api/
sudo cp -r static/admin /opt/ds2api/static/admin
```

View File

@@ -4,15 +4,18 @@
本指南基于当前 Go 代码库,详细说明各种部署方式。
本页导航:[文档总索引](./README.md)[架构说明](./ARCHITECTURE.md)[接口文档](../API.md)[测试指南](./TESTING.md)
---
## 目录
- [部署方式优先级建议](#部署方式优先级建议)
- [前置要求](#0-前置要求)
- [一、本地运行](#一本地运行)
- [二、Docker 部署](#二docker-部署)
- [一、下载 Release 构建包](#一下载-release-构建包)
- [二、Docker / GHCR 部署](#二docker--ghcr-部署)
- [三、Vercel 部署](#三vercel-部署)
- [四、下载 Release 构建包](#四下载-release-构建包)
- [四、本地源码运行](#四本地源码运行)
- [五、反向代理Nginx](#五反向代理nginx)
- [六、Linux systemd 服务化](#六linux-systemd-服务化)
- [七、部署后检查](#七部署后检查)
@@ -20,19 +23,29 @@
---
## 部署方式优先级建议
推荐按以下顺序选择部署方式:
1. **下载 Release 构建包运行**:最省事,产物已编译完成,最适合大多数用户。
2. **Docker / GHCR 镜像部署**:适合需要容器化、编排或云环境部署。
3. **Vercel 部署**:适合已有 Vercel 环境且接受其平台约束的场景。
4. **本地源码运行 / 自行编译**:适合开发、调试或需要自行修改代码的场景。
---
## 0. 前置要求
| 依赖 | 最低版本 | 说明 |
| --- | --- | --- |
| Go | 1.26+ | 编译后端 |
| Node.js | 20+ | 仅在需要本地构建 WebUI 时 |
| Node.js | `20.19+``22.12+` | 仅在需要本地构建 WebUI 时 |
| npm | 随 Node.js 提供 | 安装 WebUI 依赖 |
配置来源(任选其一):
- **文件方式**`config.json`(推荐本地/Docker 使用)
- **环境变量方式**`DS2API_CONFIG_JSON`(推荐 Vercel 使用,支持 JSON 字符串或 Base64 编码)
- 兼容写法:`CONFIG_JSON` 是旧版回退变量;`DS2API_CONFIG_JSON` 也可以直接写原始 JSON
- **环境变量方式**`DS2API_CONFIG_JSON`(推荐 Vercel 使用,支持 JSON 字符串或 Base64 编码,也可以直接写原始 JSON
统一建议(最优实践):
@@ -47,76 +60,67 @@ cp config.example.json config.json
---
## 一、本地运行
## 一、下载 Release 构建包
### 1.1 基本步骤
仓库内置 GitHub Actions 工作流:`.github/workflows/release-artifacts.yml`
- **触发条件**:仅在 Release `published` 时触发(普通 push 不会构建)
- **构建产物**:多平台二进制压缩包 + `sha256sums.txt`
- **容器镜像发布**:仅发布到 GHCR`ghcr.io/cjackhwang/ds2api`
| 平台 | 架构 | 文件格式 |
| --- | --- | --- |
| Linux | amd64, arm64 | `.tar.gz` |
| macOS | amd64, arm64 | `.tar.gz` |
| Windows | amd64 | `.zip` |
每个压缩包包含:
- `ds2api` 可执行文件Windows 为 `ds2api.exe`
- `static/admin/`WebUI 构建产物)
- `config.example.json``.env.example`
- `README.MD``README.en.md``LICENSE`
### 使用步骤
```bash
# 克隆仓库
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 1. 下载对应平台的压缩包
# 2. 解压
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
# 复制并编辑配置
# 3. 配置
cp config.example.json config.json
# 使用你喜欢的编辑器打开 config.json,填入:
# - keys: 你的 API 访问密钥
# - accounts: DeepSeek 账号email 或 mobile + password
# 编辑 config.json
# 启动服务
go run ./cmd/ds2api
```
默认监听 `http://0.0.0.0:5001`,可通过 `PORT` 环境变量覆盖。
### 1.2 WebUI 构建
本地首次启动时,若 `static/admin/` 不存在,服务会自动尝试构建 WebUI需要 Node.js/npm缺依赖时会先执行 `npm ci`,再执行 `npm run build -- --outDir static/admin --emptyOutDir`)。
你也可以手动构建:
```bash
./scripts/build-webui.sh
```
或手动执行:
```bash
cd webui
npm install
npm run build
# 产物输出到 static/admin/
```
通过环境变量控制自动构建行为:
```bash
# 强制关闭自动构建
DS2API_AUTO_BUILD_WEBUI=false go run ./cmd/ds2api
# 强制开启自动构建
DS2API_AUTO_BUILD_WEBUI=true go run ./cmd/ds2api
```
### 1.3 编译为二进制文件
```bash
go build -o ds2api ./cmd/ds2api
# 4. 启动
./ds2api
```
### 维护者发布步骤
1. 在 GitHub 创建并发布 Release带 tag`vX.Y.Z`
2. 等待 Actions 工作流 `Release Artifacts` 完成
3. 在 Release 的 Assets 下载对应平台压缩包
---
## 二、Docker 部署
## 二、Docker / GHCR 部署
### 2.1 基本步骤
```bash
# 拉取预编译镜像
docker pull ghcr.io/cjackhwang/ds2api:latest
# 复制环境变量模板和配置文件
cp .env.example .env
cp config.example.json config.json
# 编辑 .env请改成你的强密码至少设置
# DS2API_ADMIN_KEY=your-admin-key
# 如需修改宿主机端口,可额外设置:
# DS2API_HOST_PORT=6011
# 启动
docker-compose up -d
@@ -125,7 +129,13 @@ docker-compose up -d
docker-compose logs -f
```
默认 `docker-compose.yml` 把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请调整 `ports` 配置。
默认 `docker-compose.yml` 直接使用 `ghcr.io/cjackhwang/ds2api:latest`,并把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请设置 `DS2API_HOST_PORT=5001`(或者手动调整 `ports` 配置
如需固定版本,也可以直接拉取指定 tag
```bash
docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
```
### 2.2 更新
@@ -138,7 +148,7 @@ docker-compose up -d --build
`Dockerfile` 提供两条构建路径:
1. **本地/开发默认路径(`runtime-from-source`**三阶段构建WebUI 构建 + Go 构建 + 运行阶段)。
2. **Release 路径(`runtime-from-dist`**CI 先生成 `dist/ds2api_<tag>_linux_<arch>.tar.gz`,再由 Docker 直接复用该发布包内的二进制和 `static/admin` 产物组装运行镜像,不再重复执行 `npm build`/`go build`
2. **Release 路径(`runtime-from-dist`**发布工作流先生成 tag 命名的 Release 压缩包,再把 Linux 产物复制成 `dist/docker-input/linux_amd64.tar.gz` / `linux_arm64.tar.gz`Docker 构建阶段直接消费这些输入,不再重复执行 `npm build`/`go build`
Release 路径可确保 Docker 镜像与 release 压缩包使用同一套产物,减少重复构建带来的差异。
@@ -198,10 +208,10 @@ healthcheck:
2. **在 Vercel 上导入项目**
3. **配置环境变量**(最少只需设置以下一项):
| 变量 | 说明 |
| --- | --- |
| `DS2API_ADMIN_KEY` | 管理密钥(必填) |
| `DS2API_CONFIG_JSON` | 配置内容JSON 字符串或 Base64 编码(可选,建议) |
| 变量 | 说明 |
| --- | --- |
| `DS2API_ADMIN_KEY` | 管理密钥(必填) |
| `DS2API_CONFIG_JSON` | 配置内容JSON 字符串或 Base64 编码(可选,建议) |
4. **部署**
@@ -244,11 +254,8 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # 个人账号可留空
| 变量 | 说明 | 默认值 |
| --- | --- | --- |
| `DS2API_ACCOUNT_MAX_INFLIGHT` | 每账号并发上限 | `2` |
| `DS2API_ACCOUNT_CONCURRENCY` | 同上(兼容别名) | — |
| `DS2API_ACCOUNT_MAX_QUEUE` | 等待队列上限 | `recommended_concurrency` |
| `DS2API_ACCOUNT_QUEUE_SIZE` | 同上(兼容别名) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | 全局并发上限 | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | 同上(兼容别名) | — |
| `DS2API_ENV_WRITEBACK` | 检测到 `DS2API_CONFIG_JSON` 时自动写入 `DS2API_CONFIG_PATH`,并在成功后转为文件模式(`1/true/yes/on` | 关闭 |
| `DS2API_VERCEL_INTERNAL_SECRET` | 混合流式内部鉴权 | 回退用 `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | 流式 lease TTL | `900` |
@@ -314,7 +321,7 @@ Error: Command failed: go build -ldflags -s -w -o .../bootstrap ...
1. 进入 Vercel Project Settings → Build and Development Settings
2. **清空**自定义 Go Build Flags / Build Command推荐
3. 若必须设置 ldflags使用 `-ldflags="-s -w"`(保证它是一个参数)
4. 确认仓库 `go.mod` 为受支持版本(当前为 `go 1.24`
4. 确认仓库 `go.mod` 为受支持版本(当前为 `go 1.26.0`
5. 重新部署(建议清缓存后 Redeploy
#### Internal 包导入错误
@@ -350,58 +357,61 @@ No Output Directory named "public" found after the Build completed.
---
## 四、下载 Release 构建包
## 四、本地源码运行
仓库内置 GitHub Actions 工作流:`.github/workflows/release-artifacts.yml`
- **触发条件**:仅在 Release `published` 时触发(普通 push 不会构建)
- **构建产物**:多平台二进制压缩包 + `sha256sums.txt`
- **容器镜像发布**:仅发布到 GHCR`ghcr.io/cjackhwang/ds2api`
| 平台 | 架构 | 文件格式 |
| --- | --- | --- |
| Linux | amd64, arm64 | `.tar.gz` |
| macOS | amd64, arm64 | `.tar.gz` |
| Windows | amd64 | `.zip` |
每个压缩包包含:
- `ds2api` 可执行文件Windows 为 `ds2api.exe`
- `static/admin/`WebUI 构建产物)
- `sha3_wasm_bg.7b9ca65ddd.wasm`(可选;程序内置 embed fallback
- `config.example.json``.env.example`
- `README.MD``README.en.md``LICENSE`
### 使用步骤
### 4.1 基本步骤
```bash
# 1. 下载对应平台的压缩包
# 2. 解压
tar -xzf ds2api_<tag>_linux_amd64.tar.gz
cd ds2api_<tag>_linux_amd64
# 克隆仓库
git clone https://github.com/CJackHwang/ds2api.git
cd ds2api
# 3. 配置
# 复制并编辑配置
cp config.example.json config.json
# 编辑 config.json
# 使用你喜欢的编辑器打开 config.json,填入:
# - keys: 你的 API 访问密钥
# - accounts: DeepSeek 账号email 或 mobile + password
# 4. 启动
./ds2api
# 启动服务
go run ./cmd/ds2api
```
### 维护者发布步骤
默认本地访问地址是 `http://127.0.0.1:5001`;服务实际绑定 `0.0.0.0:5001`,可通过 `PORT` 环境变量覆盖。
1. 在 GitHub 创建并发布 Release带 tag`vX.Y.Z`
2. 等待 Actions 工作流 `Release Artifacts` 完成
3. 在 Release 的 Assets 下载对应平台压缩包
### 4.2 WebUI 构建
### 拉取 GHCR 镜像(可选)
本地首次启动时,若 `static/admin/` 不存在,服务会自动尝试构建 WebUI需要 Node.js/npm缺依赖时会先执行 `npm ci`,再执行 `npm run build -- --outDir static/admin --emptyOutDir`)。
你也可以手动构建:
```bash
# latest
docker pull ghcr.io/cjackhwang/ds2api:latest
./scripts/build-webui.sh
```
# 指定版本(示例)
docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
或手动执行:
```bash
cd webui
npm install
npm run build
# 产物输出到 static/admin/
```
通过环境变量控制自动构建行为:
```bash
# 强制关闭自动构建
DS2API_AUTO_BUILD_WEBUI=false go run ./cmd/ds2api
# 强制开启自动构建
DS2API_AUTO_BUILD_WEBUI=true go run ./cmd/ds2api
```
### 4.3 编译为二进制文件
```bash
go build -o ds2api ./cmd/ds2api
./ds2api
```
---
@@ -458,8 +468,6 @@ server {
# 将编译好的二进制文件和相关文件复制到目标目录
sudo mkdir -p /opt/ds2api
sudo cp ds2api config.json /opt/ds2api/
# 可选:若你希望使用外置 WASM 文件(覆盖内置版本,来自 release 包或构建产物)
# sudo cp /path/to/sha3_wasm_bg.7b9ca65ddd.wasm /opt/ds2api/
sudo cp -r static/admin /opt/ds2api/static/admin
```

View File

@@ -1,82 +0,0 @@
# DeepSeek SSE 流格式字段分析2026-04-03
> 日期2026-04-03UTC
>
> 样本:`tests/raw_stream_samples/guangzhou-weather-reasoner-search-20260403/upstream.stream.sse`
>
> 模型:`deepseek-reasoner-search`(搜索 + 思考)
## 1. SSE 事件层结构
原始流由标准 SSE 帧组成,常见形态:
```text
event: <type>
data: <json or text>
```
样本中主要 `event` 类型:
- `ready`:流建立后返回请求/响应消息 ID。
- `update_session`:会话时间戳更新。
- `finish`:流式阶段结束。
- (无 `event` 时)默认为 message 事件,`data:` 中承载主要增量数据。
## 2. `data` JSON 常见字段
上游增量主体多为 JSON Patch 风格对象:
- `p`path字段路径`response/fragments/-1/content`
- `o`op可选操作类型常见 `SET` / `APPEND` / `BATCH`
- `v`value字符串、布尔、对象、数组都可能
示例(语义):
- `{"p":"response/fragments/-1/content","o":"APPEND","v":"..."}`
- `{"p":"response/fragments/-16/status","v":"FINISHED"}`
- `{"p":"response/status","o":"SET","v":"FINISHED"}`
## 3. 搜索+思考场景关键路径
### 3.1 文本内容
- `response/fragments/<idx>/content`
- `response/content`
- `response/thinking_content`
- `response/fragments``APPEND` + fragment 数组)
### 3.2 搜索相关
- `response/fragments/<idx>/results`(检索结果数组)
- `response/search_status`(检索状态,建议跳过展示)
### 3.3 状态相关(重点)
- `response/status = FINISHED`**最终结束信号**(需要保留用于结束判定)
- `response/fragments/<idx>/status = FINISHED`**分片级状态**(高频,建议跳过输出)
- `response/quasi_status`:过程状态(建议跳过输出)
## 4. 泄露问题根因FINISHED 重复)
在搜索 + 思考模型中,`response/fragments/<idx>/status` 会出现大量不同 `<idx>`(例如 `-1/-2/-3/-16...`)的 `FINISHED`
若只过滤固定少量索引(例如仅 `-1/-2/-3`),其他索引的状态会当普通文本透传,导致前端出现:
- `FINISHEDFINISHEDFINISHED...`
## 5. 适配建议(已落地)
1. 跳过所有 `response/fragments/-?\d+/status`
2. 继续保留 `response/status=FINISHED` 作为真正结束判定。
3. 通过独立仿真工具持续回放全部样本,作为回归门禁:
```bash
./tests/scripts/run-raw-stream-sim.sh
```
## 6. 后续扩展建议
- 增加不同模型(`deepseek-chat-search` / 非 search / 非 thinking样本。
- 增加异常样本限流、中断、content_filter、空结果
- 为仿真报告加入字段覆盖率统计(路径频次、事件频次、终止路径命中率)。

View File

@@ -0,0 +1,315 @@
# DeepSeek SSE 行为结构说明(第三方逆向版)
> 说明:本文基于当前仓库 `tests/raw_stream_samples/` 下全部 `upstream.stream.sse` 原始流样本整理而成,属于第三方逆向观察文档,不是官方协议。
> 当前 corpus 由 4 份原始流组成,覆盖搜索+引用、风控终态、Markdown 输出和空格敏感输出等行为。
> 补充:文末还会注明少量“当前实现已确认、但 corpus 尚未完整覆盖”的行为,例如长思考场景下的自动续写状态。
文档导航:[文档总索引](./README.md) / [测试指南](./TESTING.md) / [样本目录说明](../tests/raw_stream_samples/README.md)
## 1. 样本覆盖
下列样本共同构成了本文的观察基础:
| 样本 | 观察重点 |
| --- | --- |
| [guangzhou-weather-reasoner-search-20260404](../tests/raw_stream_samples/guangzhou-weather-reasoner-search-20260404/upstream.stream.sse) | 搜索+思考流程,包含 `reference:N` 引用标记与工具片段 |
| [content-filter-trigger-20260405-jwt3](../tests/raw_stream_samples/content-filter-trigger-20260405-jwt3/upstream.stream.sse) | `CONTENT_FILTER` 终态分支,包含拒答模板与 `ban_regenerate` |
| [markdown-format-example-20260405](../tests/raw_stream_samples/markdown-format-example-20260405/upstream.stream.sse) | Markdown 输出的早期样本,用于观察 token 级输出形态 |
| [markdown-format-example-20260405-spacefix](../tests/raw_stream_samples/markdown-format-example-20260405-spacefix/upstream.stream.sse) | Markdown 输出修正样本,用于验证空格 chunk 必须保留 |
当前 corpus 的整体特征是 `message` 帧占绝对多数,控制事件只占很小一部分,但它们决定了流的生命周期和最终状态。
## 2. 总体结构
DeepSeek 的这类输出可以分成两层看:
1. SSE 事件层。
2. JSON 载荷层。
事件层负责传输边界,载荷层负责业务状态。实现时不要把 HTTP chunk、SSE block 和业务 JSON 混为一体。
最常见的时序可以概括为:
```text
ready
update_session
message(初始化 envelope)
message(正文 / 片段 / 状态增量)
message(状态收口)
finish
update_session
title
close
```
`finish` 表示生成流结束,但不是唯一的终止信号;真正的语义终态通常还要结合 `response/status``quasi_status``close` 一起判断。
## 3. SSE 事件层
当前 corpus 中观察到的事件类型如下:
| 事件 | 作用 | 处理建议 |
| --- | --- | --- |
| `ready` | 传输层就绪,通常携带 `request_message_id``response_message_id``model_type` | 记录元数据即可,不参与正文拼接 |
| `update_session` | 会话时间戳或心跳更新 | 当作会话状态帧处理 |
| `message` | 主体载荷,绝大多数业务信息都在这里 | 必须按顺序解析并保序累积 |
| `finish` | 生成阶段结束 | 作为流结束标记之一 |
| `title` | 会话标题生成结果 | 元数据帧,不参与正文拼接 |
| `close` | 连接关闭信息 | 仅用于收尾与审计 |
说明:
- `message` 是默认事件名SSE 中没有显式 `event:` 时也应按 `message` 处理。
- 目前样本里大量 `message` 帧没有独立的业务前缀,不能靠事件名区分正文和控制帧。
- 可能出现空 payload 的 `message` 帧;它们应被视为 no-op但不能打乱事件顺序。
## 4. 载荷层形态
`message``data:` 部分不是单一 schema而是多种结构混合。当前 corpus 里主要见到以下几种形态:
| 形态 | 典型结构 | 作用 |
| --- | --- | --- |
| 初始化 envelope | `{"v":{"response":{...}}}` | 给出会话初始状态、模型状态和片段容器 |
| 纯文本 token | `{"v":"..."}` | 直接输出可见文本 token |
| 路径补丁 | `{"p":"...","o":"APPEND|SET|BATCH","v":...}` | 对某个状态路径做增量更新 |
| 终态 batch | `{"v":[{"p":"status","v":"CONTENT_FILTER"}, ...]}` | 尾部状态收口,常见于风控终态 |
一个简化后的典型样式如下:
```json
{"v":"输出"}
{"p":"response/fragments/-1/content","o":"APPEND","v":"..."}
{"p":"response/fragments","o":"APPEND","v":[...]}
{"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":211},{"p":"quasi_status","v":"FINISHED"}]}
{"p":"response/status","o":"SET","v":"FINISHED"}
```
注意:
- `v` 可能是字符串、对象、数组、布尔值或数字。
- `o` 当前样本里主要见到 `APPEND``SET``BATCH`
- `v` 为数组时,通常表示一个批量 patch 集合,而不是正文数组。
## 5. 初始化 envelope
每条流开头,常会先出现一个 `message` 帧,内容是完整的 `response` 初始状态。当前 corpus 中,这个 envelope 常见字段包括:
- `message_id`
- `parent_id`
- `model`
- `role`
- `thinking_enabled`
- `ban_edit`
- `ban_regenerate`
- `status`
- `incomplete_message`
- `accumulated_token_usage`
- `files`
- `feedback`
- `inserted_at`
- `search_enabled`
- `fragments`
- `conversation_mode`
- `has_pending_fragment`
- `auto_continue`
- `search_triggered`
这些字段更像会话状态和策略开关,不是正文内容。第三方实现应把它们保留在内部状态树里,而不是直接拼接到最终答案。
## 6. 路径结构
当前 corpus 里观察到的 `p` 路径可以归成几组:
### 6.1 片段级路径
- `response/fragments/-N/content`
- `response/fragments/-N/status`
- `response/fragments/-N/results`
- `response/fragments/-N/elapsed_secs`
这类路径表示某个片段对象的增量更新。`-N` 只是样本中的索引风格,不应被写死成固定数量。
### 6.2 片段容器路径
- `response/fragments`
- `fragments`
这两类路径通常承载 fragment 数组。前者更像响应树中的分支,后者更像终态批处理里的片段集合。
### 6.3 语义状态路径
- `response/status`
- `response/has_pending_fragment`
- `quasi_status`
- `status`
- `ban_regenerate`
这类路径决定流是否结束、是否被风控、是否还有待处理片段。它们不应作为正文输出。
尤其是 `response/status` / `status` 这类路径上的字符串值,应被视为控制信号而不是文本 token。当前已确认需要特殊对待的值包括
- `FINISHED`:正常完成终态,应触发收口。
- `CONTENT_FILTER`:风控终态,应走拒答/模板分支。
- `WIP` / `INCOMPLETE` / `AUTO_CONTINUE`:未完成但可继续生成的中间状态,不应直接输出给客户端。
### 6.4 统计与进度路径
- `accumulated_token_usage`
这类路径用于使用量或进度统计,属于元数据。
### 6.5 非命名空间字段
在片段对象内部,还会看到 `content``references``result``queries``stage_id` 等字段。它们不一定带 `response/...` 前缀,但仍然是协议语义的一部分。
## 7. fragment 类型
当前 corpus 里已经观察到的 fragment 类型如下:
| 类型 | 作用 | 是否应直接渲染 |
| --- | --- | --- |
| `RESPONSE` | 正常回答片段 | 是,属于正文 |
| `THINK` | 推理或阶段提示 | 通常否,按产品策略决定是否展示 |
| `TOOL_SEARCH` | 搜索工具调用元数据 | 否 |
| `TOOL_OPEN` | 打开 / 抽取结果的工具元数据 | 否 |
| `TIP` | 提示 / 警告类片段,常带 `style: WARNING` | 视产品策略决定,通常作为附注 |
| `TEMPLATE_RESPONSE` | 风控拒答模板 | 是,但它属于终态 fallback不是普通正文 |
观察到的典型片段字段:
- `id`
- `type`
- `content`
- `references`
- `stage_id`
- `status`
- `queries`
- `results`
- `result`
- `elapsed_secs`
- `style`
- `hide_on_wip`
第三方实现不要把 `fragment.type``p` 路径混为一谈。`type` 是语义分类,`p` 是状态树位置。
## 8. 终态行为
当前 corpus 里有两条很重要的终态分支。
### 8.1 正常完成
正常回答通常会出现如下收口顺序:
1. `response``BATCH` 更新 `accumulated_token_usage`
2. `response``BATCH` 或单独 patch 更新 `quasi_status: FINISHED`
3. `response/status` 置为 `FINISHED`
4. `finish` 事件到来。
5. 之后可能还有 `update_session``title``close`
### 8.2 风控终态
`content-filter-trigger-20260405-jwt3` 展示了另一种终态路径:
1. 先继续输出一段正常正文。
2. 出现提示类 fragment例如 `TIP`
3. 可能先把 `quasi_status` 提前收口到 `FINISHED`
4. 之后出现一个终态 batch`ban_regenerate` 设为 `true`,把 `status` 置为 `CONTENT_FILTER`,并附带 `TEMPLATE_RESPONSE`
5. 最后再出现 `finish`,然后是收尾事件。
这个分支说明:
- `finish` 不等于正常结束。
- `CONTENT_FILTER` 是一个独立终态,不是普通异常。
- `TEMPLATE_RESPONSE` 不应被当作常规回答流的中间片段,它是终态 fallback。
一个简化的风控尾部可以写成:
```json
{"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":1269},{"p":"quasi_status","v":"FINISHED"}]}
{"v":[{"p":"ban_regenerate","v":true},{"p":"status","v":"CONTENT_FILTER"},{"p":"fragments","v":[{"id":38,"type":"TEMPLATE_RESPONSE","content":"..."}]},{"p":"quasi_status","v":"CONTENT_FILTER"}]}
{"event":"finish"}
```
### 8.3 自动续写中间态(实现补充)
这部分不是当前 corpus 的直接覆盖项,而是 2026-04-05 在长思考实测中观察到、且已在当前实现中兼容的行为:
1. 上游可能先把 `response/status` 或 envelope 内的 `response.status` 置为 `WIP` / `INCOMPLETE`
2. 有时还会伴随 `auto_continue: true`
3. 这表示当前轮输出尚未真正结束,客户端或代理层可以继续调用 continue 接口续写同一条回答。
4. 续写后的内容会承接之前的思考与正文,不应把前一轮状态值泄露成可见文本。
对第三方实现,建议把这一类状态统一当作“可继续的控制信号”:
- 可以据此决定是否继续拉取后续流。
- 不能把 `INCOMPLETE``WIP``AUTO_CONTINUE` 直接拼接到最终文本。
- `finish` 事件本身也不能单独说明回答已完全结束,仍要结合状态字段判断。
## 9. 文本重建规则
如果你的目标是把流重建成最终可见文本,必须遵守下面这些规则:
- 按接收顺序逐个追加 token。
- 不要对每个 `v``trim``TrimSpace`
- 不要丢弃只包含空格的 chunk。
- 不要合并连续空格、换行或 Markdown 符号附近的空白。
- 不要把 `[reference:N]` 视为协议元数据,它在当前 corpus 里就是正文的一部分。
- 如果你要屏蔽引用标记,应当把它做成可配置的后处理,而不是在解析阶段硬删。
- `response/status` / `status` 路径上的状态字符串不应进入正文,即使它们不是终态。
这点对 Markdown、代码块、引用、表格都很关键。样本里已经证明`#``-``>``|` 这类符号后面的空格必须原样保留,否则渲染结果会变形。
## 10. 推荐实现方式
对第三方开发者,建议把实现拆成三条线:
1. 原始事件线:保留 SSE block 顺序、事件名和完整 JSON 载荷。
2. 状态树线:维护 `response``fragments``status``quasi_status` 等结构。
3. 可见文本线:只从明确应渲染的 token / fragment 中拼接最终文本。
一个简单的处理顺序可以是:
```text
parse SSE block
-> 识别 event
-> 解析 JSON payload
-> 更新状态树
-> 识别 status / quasi_status / auto_continue 等控制信号
-> 判定是否有可见文本
-> 追加到输出缓冲
-> 遇到 WIP / INCOMPLETE / AUTO_CONTINUE 时决定是否续写
-> 遇到 FINISHED / CONTENT_FILTER / finish 时收口
```
实现时的兼容原则:
- 未知路径保留,不要报错中断。
- 未知 fragment.type 保留在日志里。
- 不要假设所有模型都一定输出 `thinking_content`,当前 corpus 的推理更多是通过 fragment 类型表达。
- 不要假设 `title` 一定存在,它只是后置元数据。
## 11. 本 corpus 证明了什么
当前样本足以证明以下行为:
- 搜索类模型会把工具调用、结果、引用和正文混在同一条 SSE 流里。
- 风控不会简单地“没有输出”,而是会在正常生成后切换到 `CONTENT_FILTER` 终态。
- Markdown 和代码输出对空格非常敏感,空格 chunk 不能吞。
- `message` 是主体承载层,`ready` / `update_session` / `finish` / `title` / `close` 是控制层。
- `fragment.type` 是可视化和工具链分层的关键,不应只靠 `p` 路径判断。
结合 2026-04-05 的长思考实测,还可以补充一条当前实现层面的结论:
- 长思考场景下,上游可能先给出 `INCOMPLETE` / `WIP` / `AUTO_CONTINUE` 状态,再通过 continue 链路续写;这些状态值本身不应作为正文输出。
## 12. 适用边界
本文是基于当前 corpus 的逆向说明,不是恒定协议。
- 新模型可能增加新的 `p` 路径。
- 新版本可能增加新的 fragment.type。
- `CONTENT_FILTER` 的终态模板内容可能变化。
- 自动续写相关状态(如 `INCOMPLETE` / `AUTO_CONTINUE`)当前主要来自实测与实现兼容逻辑,后续字段形态仍可能变化。
- 解析器应当对未知字段、未知路径、未知事件保持容忍。
如果你要把这份说明用于实际开发,建议同时保留原始流样本、回放脚本和回归测试,不要只依赖本文。

53
docs/README.md Normal file
View File

@@ -0,0 +1,53 @@
# DS2API 文档导航 | Documentation Index
语言 / Language: [中文](README.md) | [English](README.md#english)
## 中文
为减少重复维护,本仓库文档按“入口文档 + 专题文档”拆分。建议从下列顺序阅读:
1. [项目总览README](../README.MD)
2. [架构与目录说明](./ARCHITECTURE.md)
3. [接口文档API](../API.md)
4. [部署指南](./DEPLOY.md)
5. [测试指南](./TESTING.md)
6. [贡献指南](./CONTRIBUTING.md)
### 专题文档
- [Tool Calling 统一语义](./toolcall-semantics.md)
- [DeepSeek SSE 行为结构说明(逆向观察)](./DeepSeekSSE行为结构说明-2026-04-05.md)
### 文档维护约定
- `README.MD` / `README.en.md`:面向首次接触用户,保留“是什么 + 怎么快速跑起来”。
- `docs/ARCHITECTURE*.md`:面向开发者,集中维护项目结构、模块职责与调用链。
- `API*.md`:面向客户端接入者,聚焦接口行为、鉴权和示例。
- 其他 `docs/*.md`:主题化说明,避免在多个文档重复粘贴同一段内容。
---
## English
To reduce maintenance drift, docs are split into an “entry doc + topical docs” layout.
Recommended reading order:
1. [Project overview (README)](../README.en.md)
2. [Architecture and project layout](./ARCHITECTURE.en.md)
3. [API reference](../API.en.md)
4. [Deployment guide](./DEPLOY.en.md)
5. [Testing guide](./TESTING.md)
6. [Contributing guide](./CONTRIBUTING.en.md)
### Topical docs
- [Tool-calling unified semantics](./toolcall-semantics.md)
- [DeepSeek SSE behavior notes (reverse-engineered)](./DeepSeekSSE行为结构说明-2026-04-05.md)
### Maintenance conventions
- `README.MD` / `README.en.md`: onboarding-oriented (“what + quick start”).
- `docs/ARCHITECTURE*.md`: developer-oriented source of truth for module boundaries and execution flow.
- `API*.md`: integration-oriented behavior/contracts.
- Other `docs/*.md`: focused topics, avoid copy-pasting the same section into multiple files.

View File

@@ -2,6 +2,8 @@
语言 / Language: 中文 + English同页
文档导航: [总览](../README.MD) / [架构说明](./ARCHITECTURE.md) / [部署指南](./DEPLOY.md) / [接口文档](../API.md)
## 概述 | Overview
DS2API 提供两个层级的测试:
@@ -180,10 +182,10 @@ go test ./...
```bash
# 运行 tool calls 相关测试(推荐用于调试 tool call 解析问题)
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/util/
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/toolcall/
# 运行单个测试用例
go test -v -run TestParseToolCallsWithDeepSeekHallucination ./internal/util/
go test -v -run TestParseToolCallsWithDeepSeekHallucination ./internal/toolcall/
# 运行 format 相关测试
go test -v ./internal/format/...
@@ -198,13 +200,13 @@ go test -v ./internal/adapter/openai/...
```bash
# 1. 运行 tool calls 相关的所有测试
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/util/
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/toolcall/
# 2. 查看测试输出中的详细调试信息
go test -v -run TestParseToolCallsWithDeepSeekHallucination ./internal/util/ 2>&1
go test -v -run TestParseToolCallsWithDeepSeekHallucination ./internal/toolcall/ 2>&1
# 3. 检查具体测试用例的修复效果
# 测试用例位于 internal/util/toolcalls_test.go包含
# 测试用例位于 internal/toolcall/toolcalls_test.go包含
# - TestParseToolCallsWithDeepSeekHallucination: DeepSeek 典型幻觉输出
# - TestRepairLooseJSONWithNestedObjects: 嵌套对象的方括号修复
# - TestParseToolCallsWithMixedWindowsPaths: Windows 路径处理
@@ -233,9 +235,48 @@ go run ./cmd/ds2api-tests --no-preflight
```
说明:
- 该工具重放 `tests/raw_stream_samples` 下全部样本,按上游 SSE 顺序做 1:1 仿真解析。
- 该工具默认重放 `tests/raw_stream_samples/manifest.json` 声明的 canonical 样本,按上游 SSE 顺序做 1:1 仿真解析。
- 默认校验不出现 `FINISHED` 文本泄露,并要求存在结束信号。
- 结果会写入 `artifacts/raw-stream-sim/*.json`,可供其他测试脚本或排障流程复用
- 默认**不**把 `raw accumulated_token_usage` 与本地解析 token 做强一致校验(当前实现以内容估算为准);如需强校验可显式加 `--fail-on-token-mismatch`
- 每次运行都会把本地派生结果写入 `artifacts/raw-stream-sim/<run-id>/<sample-id>/replay.output.txt`,并输出结构化报告。
- 如果你有历史基线目录,可以通过 `--baseline-root` 让工具直接做文本对比。
- 更完整的协议级行为结构说明见 [DeepSeekSSE行为结构说明-2026-04-05.md](./DeepSeekSSE行为结构说明-2026-04-05.md)。
### 对单个样本做回放比对
```bash
./tests/scripts/compare-raw-stream-sample.sh markdown-format-example-20260405-spacefix
```
说明:
- 该脚本会从 raw-only 样本目录读取 `upstream.stream.sse`
- 回放结果会写入 `artifacts/raw-stream-sim/<run-id>/<sample-id>/`,便于直接查阅。
- 如果传入历史基线目录,脚本会自动对比当前回放输出和基线文本。
### 采集永久样本
本地启动服务后,可以直接打:
```bash
POST /admin/dev/raw-samples/capture
```
这个接口会把请求元信息和上游原始流写入 `tests/raw_stream_samples/<sample-id>/`,以后可以直接拿来做回放和字段分析。派生输出会在本地回放时再生成,不再落在样本目录里。
### 从内存抓包查询并保存样本
如果问题刚刚在本地复现过,也可以先查当前进程内存里的抓包,再选择性落盘:
```bash
GET /admin/dev/raw-samples/query?q=广州&limit=10
POST /admin/dev/raw-samples/save
{"chain_key":"session:xxxx","sample_id":"tmp-from-memory"}
```
说明:
- `query` 会按 `chat_session_id``completion + continue` 归并成一条链,适合定位接续思考问题。
- `save` 支持用 `query``chain_key``capture_id` 选中目标。
- 生成的样本目录仍然是 `tests/raw_stream_samples/<sample-id>/`,可以直接喂给回放脚本。
### 指定输出目录和超时

View File

@@ -1,72 +1,74 @@
# Tool call parsing semanticsGo/Node 统一语义)
本文档描述当前代码中 `ParseToolCallsDetailed` / `parseToolCallsDetailed` 的**实际行为**,用于对齐 Go 与 Node Runtime
本文档描述当前代码中工具调用解析链路的**实际行为**(以 `internal/toolcall` `internal/js/helpers/stream-tool-sieve` 为准)
## 1) 输出结构(当前实现)
文档导航:[总览](../README.MD) / [架构说明](./ARCHITECTURE.md) / [测试指南](./TESTING.md)
- `calls`:解析得到的工具调用列表(`name` + `input`)。
- `sawToolCallSyntax`:检测到工具调用语法特征时为 `true`(例如 `tool_calls``<tool_call>``<function_call>``<invoke>``function.name:`)。
- `rejectedByPolicy`:当前实现固定为 `false`(预留字段,尚未启用 allow-list 拒绝)。
## 1) 当前输出结构
`ParseToolCallsDetailed` / `parseToolCallsDetailed` 返回:
- `calls`:解析出的工具调用列表(`name` + `input`)。
- `sawToolCallSyntax`:检测到工具调用语法特征时为 `true`
- `rejectedByPolicy`:当前实现固定为 `false`(预留字段)。
- `rejectedToolNames`:当前实现固定为空数组(预留字段)。
> 说明:`filterToolCallsDetailed` 当前仅做结构清洗,不做工具名策略拒绝。
> 当前 `filterToolCallsDetailed` 仅做结构清洗,不做 allow-list 工具名拒绝。
## 2) 解析管线
## 2) 解析范围(重点)
1. **示例保护**:若判定为 fenced code block 示例上下文,则跳过执行型解析。
2. **候选片段构建**:从完整文本中构建候选(原文、围绕 `tool_calls` 的 JSON 片段、首尾大括号切片等)。
3. **按序尝试解析(命中即停)**
- 对“明显 JSON 工具载荷候选”(以 `{`/`[` 开头且包含 `tool_calls`/`\"function\"`)先走 JSON 解析,避免 JSON 字符串内偶发 XML 片段误命中;
- 其余候选优先 XML 解析(`<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / `antml:function_call` 等);
- JSON 解析(`{"tool_calls": [...]}`、列表、单对象);
- Markup 解析;
- Text-KV 回退(如 `function.name:` + `function.arguments:`)。
4. **兜底**:候选全部失败后,再对全文做 XML / Text-KV 回退。
当前版本的可执行解析以 **XML/Markup 家族**为主:
## 3) XML 能力边界(当前)
- `<tool_call>...</tool_call>`
- `<function_call>...</function_call>`
- `<invoke ...>...</invoke>`(含自闭合)
- `<tool_use>...</tool_use>`
- antml 变体(如 `antml:function_call` / `antml:argument`
当前已支持输入端的“多 XML/标记风格”解析,包括但不限于
并支持在这些标记块内部解析
- `<tool_call><tool_name>...</tool_name><parameters>...</parameters></tool_call>`
- `<function_call>tool</function_call><function parameter name="x">...</function parameter>`
- `<invoke name="tool"><parameter name="x">...</parameter></invoke>`
- `antml:function_call` / `antml:argument` / `antml:parameters`
- `tool_use` 家族标签
- JSON 参数字符串
- 标签参数(`<parameter name="...">...`
- key/value 风格子标签
但**输出端仍统一转换为 OpenAI 兼容 JSON 事件/对象**`message.tool_calls``delta.tool_calls``response.function_call_arguments.*`)。
## 3) 不应再假设的行为
## 4) 关于“是否可以封装成 XML 再喂给模型”
以下说法在当前实现中已不成立:
结论:**可以做,而且当前解析器已经能兼容 XML 作为输入格式之一**,但代码里并没有 `toolcall.prefer_xml_output` 这个开关。现有可调配置只有:
1. “纯 JSON `tool_calls` 片段会被直接当作可执行工具调用解析”。
2. “存在 `toolcall.mode` / `toolcall.early_emit_confidence` 等可配置开关可以改变解析策略”。
- `toolcall.mode``feature_match` / `off`
- `toolcall.early_emit_confidence``high` / `low` / `off`
当前策略在代码中固定为:
推荐思路仍然是“输入兼容层 + 输出按客户端协议渲染”:
- 特征匹配开启feature-match on
- 高置信度早发开启early emit on
- policy 拒绝字段保留但未启用
1. **Prompt 约束层**:如果你要尝试 XML-first可以在系统提示词里约束模型输出规范 XML tool block例如 `<tool_calls><tool_call>...</tool_call></tool_calls>`)。
2. **解析兼容层**:继续在 parser 中同时接受 JSON / XML / ANTML / invoke / text-kv。
3. **协议归一层**:无论模型输出什么格式,统一落到内部 `ParsedToolCall`
4. **对外渲染层**根据客户端请求协议渲染OpenAI / Claude / Gemini 各自格式)。
## 4) 流式与防泄漏语义
这样可以同时获得
在流式链路中OpenAI / Claude / Gemini 统一内核)
- 减少模型端 JSON 转义/引号错误
- 不破坏现有 SDK / 客户端生态
- 逐步灰度(按模型、按租户、按请求开关)
- 工具调用片段会被优先提取为结构化增量输出
- 已识别的工具调用原始片段不会作为普通文本再次回流
- fenced code block 中的示例内容按文本处理,不作为可执行工具调用
## 5) 落地建议(低风险迭代
## 5) 落地建议(按当前实现
- 继续使用现有的 `toolcall.mode=feature_match``toolcall.early_emit_confidence=high` 作为默认策略
- 如果要试 XML-first把它放在 prompt 层或上游模板层,不要假设代码里已有专门的 XML 输出开关
- 增加观测指标:
- `toolcall_parse_source`json/xml/markup/textkv
- `toolcall_parse_success_rate`
- `toolcall_malformed_rate`
- `toolcall_repair_rate`
- 先在 `responses` 链路灰度,再扩展 `chat.completions`
1. Prompt 里优先约束模型输出 XML/Markup 工具块
2. 执行器侧继续做工具名白名单与参数 schema 校验(不要依赖 parser 代替安全策略)
3. 需要兼容历史“纯 JSON tool_calls”模型输出时请在上游模板层把输出规范化为 XML/Markup 风格再进入 DS2API。
## 6) 兼容性提醒
## 6) 回归验证建议
- 上游模型若输出混合文本 + XML仍可能出现“半结构化”噪声需要依赖现有 sieve 增量消费策略。
- XML 不等于安全:仍需做 tool 名、参数 schema、执行权限的服务端校验。
可直接运行:
```bash
go test -v -run 'TestParseToolCalls|TestRepair' ./internal/toolcall/
node --test tests/node/stream-tool-sieve.test.js
```
重点覆盖:
- `<tool_call>` / `<function_call>` / `<invoke>` / `tool_use` / antml 变体
- 参数 JSON 修复与解析
- 流式增量下的工具调用提取与文本防泄漏

21
go.mod
View File

@@ -3,23 +3,22 @@ module ds2api
go 1.26.0
require (
github.com/andybalholm/brotli v1.0.6
github.com/go-chi/chi/v5 v5.2.3
github.com/andybalholm/brotli v1.2.1
github.com/go-chi/chi/v5 v5.2.5
github.com/google/uuid v1.6.0
github.com/refraction-networking/utls v1.8.2
github.com/tetratelabs/wazero v1.9.0
github.com/router-for-me/CLIProxyAPI/v6 v6.9.14
)
require (
github.com/klauspost/compress v1.17.4 // indirect
github.com/router-for-me/CLIProxyAPI/v6 v6.9.8 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/crypto v0.49.0 // indirect
golang.org/x/net v0.52.0
golang.org/x/sys v0.42.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

58
go.sum
View File

@@ -1,47 +1,43 @@
github.com/andybalholm/brotli v1.0.6 h1:Yf9fFpf49Zrxb9NlQaluyE92/+X7UVHlhMNJN2sxfOI=
github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-chi/chi/v5 v5.2.3 h1:WQIt9uxdsAbgIYgid+BpYc+liqQZGMHRaUwp0JUcvdE=
github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/klauspost/compress v1.17.4 h1:Ej5ixsIri7BrIjBkRZLTo6ghwrEtHFk7ijlczPW4fZ4=
github.com/klauspost/compress v1.17.4/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/refraction-networking/utls v1.8.1 h1:yNY1kapmQU8JeM1sSw2H2asfTIwWxIkrMJI0pRUOCAo=
github.com/refraction-networking/utls v1.8.1/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/refraction-networking/utls v1.8.2 h1:j4Q1gJj0xngdeH+Ox/qND11aEfhpgoEvV+S9iJ2IdQo=
github.com/refraction-networking/utls v1.8.2/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/router-for-me/CLIProxyAPI/v6 v6.9.8 h1:O65R38THenp8E1IK0paQlOfop3Y6UYlfqSdLlepidSY=
github.com/router-for-me/CLIProxyAPI/v6 v6.9.8/go.mod h1:P1jsIPFXorYGuS2N/3BlZYkpRKi/z7+oR3+1tdG0u4k=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I=
github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM=
github.com/router-for-me/CLIProxyAPI/v6 v6.9.14 h1:XItUHrPGE9E5xTeZIPjKGmKqfEs1AZbxl1RPfO5xtrc=
github.com/router-for-me/CLIProxyAPI/v6 v6.9.14/go.mod h1:P1jsIPFXorYGuS2N/3BlZYkpRKi/z7+oR3+1tdG0u4k=
github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w=
github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
github.com/tidwall/match v1.2.0 h1:0pt8FlkOwjN2fPt4bIl4BoNxb98gGHN2ObFEDkrfZnM=
github.com/tidwall/match v1.2.0/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=
golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -13,9 +13,7 @@ import (
func TestPoolEmptyNoAccounts(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "2")
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "")
t.Setenv("DS2API_CONFIG_JSON", `{"keys":["k1"],"accounts":[]}`)
pool := NewPool(config.LoadStore())
if _, ok := pool.Acquire("", nil); ok {
@@ -165,9 +163,7 @@ func TestPoolAcquireWaitTargetAccount(t *testing.T) {
func TestPoolMaxQueueSizeOverride(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "1")
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "5")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "")
t.Setenv("DS2API_CONFIG_JSON", `{"keys":["k1"],"accounts":[{"email":"acc1@example.com","token":"t1"}]}`)
pool := NewPool(config.LoadStore())
status := pool.Status()
@@ -176,19 +172,6 @@ func TestPoolMaxQueueSizeOverride(t *testing.T) {
}
}
func TestPoolQueueSizeAliasEnv(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "1")
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "7")
t.Setenv("DS2API_CONFIG_JSON", `{"keys":["k1"],"accounts":[{"email":"acc1@example.com","token":"t1"}]}`)
pool := NewPool(config.LoadStore())
status := pool.Status()
if got, ok := status["max_queue_size"].(int); !ok || got != 7 {
t.Fatalf("expected max_queue_size=7, got %#v", status["max_queue_size"])
}
}
func TestPoolMultipleAcquireReleaseCycles(t *testing.T) {
pool := newSingleAccountPoolForTest(t, "1")
for i := 0; i < 10; i++ {

View File

@@ -29,13 +29,8 @@ func (p *Pool) ApplyRuntimeLimits(maxInflightPerAccount, maxQueueSize, globalMax
}
func maxInflightFromEnv() int {
for _, key := range []string{"DS2API_ACCOUNT_MAX_INFLIGHT", "DS2API_ACCOUNT_CONCURRENCY"} {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
continue
}
n, err := strconv.Atoi(raw)
if err == nil && n > 0 {
if raw := strings.TrimSpace(os.Getenv("DS2API_ACCOUNT_MAX_INFLIGHT")); raw != "" {
if n, err := strconv.Atoi(raw); err == nil && n > 0 {
return n
}
}
@@ -53,13 +48,8 @@ func defaultRecommendedConcurrency(accountCount, maxInflightPerAccount int) int
}
func maxQueueFromEnv(defaultSize int) int {
for _, key := range []string{"DS2API_ACCOUNT_MAX_QUEUE", "DS2API_ACCOUNT_QUEUE_SIZE"} {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
continue
}
n, err := strconv.Atoi(raw)
if err == nil && n >= 0 {
if raw := strings.TrimSpace(os.Getenv("DS2API_ACCOUNT_MAX_QUEUE")); raw != "" {
if n, err := strconv.Atoi(raw); err == nil && n >= 0 {
return n
}
}

View File

@@ -12,9 +12,7 @@ import (
func newPoolForTest(t *testing.T, maxInflight string) *Pool {
t.Helper()
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", maxInflight)
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "")
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[
@@ -29,9 +27,7 @@ func newPoolForTest(t *testing.T, maxInflight string) *Pool {
func newSingleAccountPoolForTest(t *testing.T, maxInflight string) *Pool {
t.Helper()
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", maxInflight)
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "")
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[{"email":"acc1@example.com","token":"token1"}]
@@ -170,9 +166,9 @@ func TestPoolStatusRecommendedConcurrencyRespectsOverride(t *testing.T) {
}
}
func TestPoolAccountConcurrencyAliasEnv(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "")
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "4")
func TestPoolGlobalMaxInflightEnv(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "1")
t.Setenv("DS2API_GLOBAL_MAX_INFLIGHT", "4")
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[
@@ -183,15 +179,15 @@ func TestPoolAccountConcurrencyAliasEnv(t *testing.T) {
pool := NewPool(config.LoadStore())
status := pool.Status()
if got, ok := status["max_inflight_per_account"].(int); !ok || got != 4 {
if got, ok := status["global_max_inflight"].(int); !ok || got != 4 {
t.Fatalf("unexpected global_max_inflight: %#v", status["global_max_inflight"])
}
if got, ok := status["max_inflight_per_account"].(int); !ok || got != 1 {
t.Fatalf("unexpected max_inflight_per_account: %#v", status["max_inflight_per_account"])
}
if got, ok := status["recommended_concurrency"].(int); !ok || got != 8 {
if got, ok := status["recommended_concurrency"].(int); !ok || got != 2 {
t.Fatalf("unexpected recommended_concurrency: %#v", status["recommended_concurrency"])
}
if got, ok := status["max_queue_size"].(int); !ok || got != 8 {
t.Fatalf("unexpected max_queue_size: %#v", status["max_queue_size"])
}
}
func TestPoolDropsLegacyTokenOnlyAccountOnLoad(t *testing.T) {
@@ -217,9 +213,7 @@ func TestPoolDropsLegacyTokenOnlyAccountOnLoad(t *testing.T) {
func TestPoolAcquireRotatesIntoTokenlessAccounts(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "1")
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "")
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[

View File

@@ -22,6 +22,7 @@ type DeepSeekCaller interface {
type ConfigReader interface {
ClaudeMapping() map[string]string
CompatStripReferenceMarkers() bool
}
type OpenAIChatRunner interface {

View File

@@ -7,6 +7,7 @@ type mockClaudeConfig struct {
}
func (m mockClaudeConfig) ClaudeMapping() map[string]string { return m.m }
func (mockClaudeConfig) CompatStripReferenceMarkers() bool { return true }
func TestNormalizeClaudeRequestUsesConfigInterfaceMapping(t *testing.T) {
req := map[string]any{

View File

@@ -64,7 +64,7 @@ func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, store C
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
defer func() { _ = res.Body.Close() }()
body, _ := io.ReadAll(res.Body)
for k, vv := range res.Header {
for _, v := range vv {
@@ -94,7 +94,7 @@ func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, store C
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
defer func() { _ = res.Body.Close() }()
body, _ := io.ReadAll(res.Body)
if res.StatusCode < 200 || res.StatusCode >= 300 {
for k, vv := range res.Header {
@@ -124,7 +124,7 @@ func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, store C
}
func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Request, resp *http.Response, model string, messages []any, thinkingEnabled, searchEnabled bool, toolNames []string) {
defer resp.Body.Close()
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeClaudeError(w, http.StatusInternalServerError, string(body))
@@ -149,6 +149,7 @@ func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Requ
messages,
thinkingEnabled,
searchEnabled,
h.compatStripReferenceMarkers(),
toolNames,
)
streamRuntime.sendMessageStart()

View File

@@ -21,6 +21,13 @@ type Handler struct {
OpenAI OpenAIChatRunner
}
func (h *Handler) compatStripReferenceMarkers() bool {
if h == nil || h.Store == nil {
return true
}
return h.Store.CompatStripReferenceMarkers()
}
var (
claudeStreamPingInterval = time.Duration(deepseek.KeepAliveTimeout) * time.Second
claudeStreamIdleTimeout = time.Duration(deepseek.StreamIdleTimeout) * time.Second

View File

@@ -138,77 +138,6 @@ func TestHandleClaudeStreamRealtimeThinkingDelta(t *testing.T) {
}
}
func TestHandleClaudeStreamRealtimeToolSafety(t *testing.T) {
h := &Handler{}
resp := makeClaudeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\""}`,
`data: {"p":"response/content","v":",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, false, false, []string{"search"})
frames := parseClaudeFrames(t, rec.Body.String())
for _, f := range findClaudeFrames(frames, "content_block_delta") {
delta, _ := f.Payload["delta"].(map[string]any)
if delta["type"] == "text_delta" && strings.Contains(asString(delta["text"]), `"tool_calls"`) {
t.Fatalf("raw tool_calls JSON leaked in text delta: body=%s", rec.Body.String())
}
}
foundToolUse := false
for _, f := range findClaudeFrames(frames, "content_block_start") {
contentBlock, _ := f.Payload["content_block"].(map[string]any)
if contentBlock["type"] == "tool_use" {
foundToolUse = true
break
}
}
if !foundToolUse {
t.Fatalf("expected tool_use block in stream, body=%s", rec.Body.String())
}
foundToolUseStop := false
for _, f := range findClaudeFrames(frames, "message_delta") {
delta, _ := f.Payload["delta"].(map[string]any)
if delta["stop_reason"] == "tool_use" {
foundToolUseStop = true
break
}
}
if !foundToolUseStop {
t.Fatalf("expected stop_reason=tool_use, body=%s", rec.Body.String())
}
}
func TestHandleClaudeStreamRealtimeToolDetectionFromThinkingFallback(t *testing.T) {
h := &Handler{}
resp := makeClaudeSSEHTTPResponse(
`data: {"p":"response/thinking_content","v":"{\"tool_calls\":[{\"name\":\"search\""}`,
`data: {"p":"response/thinking_content","v":",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, true, false, []string{"search"})
frames := parseClaudeFrames(t, rec.Body.String())
foundToolUse := false
for _, f := range findClaudeFrames(frames, "content_block_start") {
contentBlock, _ := f.Payload["content_block"].(map[string]any)
if contentBlock["type"] == "tool_use" && contentBlock["name"] == "search" {
foundToolUse = true
break
}
}
if !foundToolUse {
t.Fatalf("expected tool_use block from thinking fallback, body=%s", rec.Body.String())
}
}
func TestHandleClaudeStreamRealtimeSkipsThinkingFallbackWhenFinalTextExists(t *testing.T) {
h := &Handler{}
resp := makeClaudeSSEHTTPResponse(

View File

@@ -96,7 +96,7 @@ func TestNormalizeClaudeMessagesToolUseToAssistantToolCalls(t *testing.T) {
if !containsStr(content, "<tool_calls>") || !containsStr(content, "<tool_name>search_web</tool_name>") {
t.Fatalf("expected assistant content to include XML tool call history, got %q", content)
}
if !containsStr(content, `<parameters>{"query":"latest"}</parameters>`) {
if !containsStr(content, "<parameters>\n <query><![CDATA[latest]]></query>\n </parameters>") {
t.Fatalf("expected assistant content to include serialized parameters, got %q", content)
}
}

View File

@@ -1,12 +1,12 @@
package claude
import (
"ds2api/internal/toolcall"
"encoding/json"
"fmt"
"strings"
"ds2api/internal/prompt"
"ds2api/internal/util"
)
func normalizeClaudeMessages(messages []any) []any {
@@ -98,9 +98,10 @@ func buildClaudeToolPrompt(tools []any) string {
}
return "You have access to these tools:\n\n" +
strings.Join(toolSchemas, "\n\n") + "\n\n" +
util.BuildToolCallInstructions(names)
toolcall.BuildToolCallInstructions(names)
}
//nolint:unused // retained for compatibility with pending Claude tool-result prompt flow.
func formatClaudeToolResultForPrompt(block map[string]any) string {
if block == nil {
return ""

View File

@@ -96,6 +96,7 @@ func looksLikeBase64Payload(v string) bool {
return true
}
//nolint:unused // helper kept for compatibility with upcoming sanitize pipeline.
func marshalCompactJSON(v any) string {
b, err := json.Marshal(v)
if err != nil {

View File

@@ -0,0 +1,13 @@
package claude
import textclean "ds2api/internal/textclean"
func cleanVisibleOutput(text string, stripReferenceMarkers bool) string {
if text == "" {
return text
}
if stripReferenceMarkers {
text = textclean.StripReferenceMarkers(text)
}
return text
}

View File

@@ -16,6 +16,8 @@ func (s claudeProxyStoreStub) ClaudeMapping() map[string]string {
return s.mapping
}
func (claudeProxyStoreStub) CompatStripReferenceMarkers() bool { return true }
type openAIProxyStub struct {
status int
body string
@@ -32,11 +34,13 @@ func (s openAIProxyStub) ChatCompletions(w http.ResponseWriter, _ *http.Request)
type openAIProxyCaptureStub struct {
seenModel string
seenReq map[string]any
}
func (s *openAIProxyCaptureStub) ChatCompletions(w http.ResponseWriter, r *http.Request) {
var req map[string]any
_ = json.NewDecoder(r.Body).Decode(&req)
s.seenReq = req
if m, ok := req["model"].(string); ok {
s.seenModel = m
}
@@ -82,3 +86,33 @@ func TestClaudeProxyViaOpenAIPreservesClaudeMapping(t *testing.T) {
t.Fatalf("expected mapped proxy model deepseek-reasoner, got %q", got)
}
}
func TestClaudeProxyTranslatesInlineImageToOpenAIDataURL(t *testing.T) {
openAI := &openAIProxyCaptureStub{}
h := &Handler{OpenAI: openAI}
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", strings.NewReader(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":[{"type":"text","text":"hello"},{"type":"image","source":{"type":"base64","media_type":"image/png","data":"QUJDRA=="}}]}],"stream":false}`))
rec := httptest.NewRecorder()
h.Messages(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d body=%s", rec.Code, rec.Body.String())
}
messages, _ := openAI.seenReq["messages"].([]any)
if len(messages) != 1 {
t.Fatalf("expected one translated message, got %#v", openAI.seenReq)
}
msg, _ := messages[0].(map[string]any)
content, _ := msg["content"].([]any)
if len(content) != 2 {
t.Fatalf("expected translated content blocks, got %#v", msg)
}
imageBlock, _ := content[1].(map[string]any)
if strings.TrimSpace(asString(imageBlock["type"])) != "image_url" {
t.Fatalf("expected image_url block, got %#v", imageBlock)
}
imageURL, _ := imageBlock["image_url"].(map[string]any)
if !strings.HasPrefix(strings.TrimSpace(asString(imageURL["url"])), "data:image/png;base64,") {
t.Fatalf("expected translated data url, got %#v", imageBlock)
}
}

View File

@@ -18,7 +18,7 @@ func normalizeClaudeRequest(store ConfigReader, req map[string]any) (claudeNorma
model, _ := req["model"].(string)
messagesRaw, _ := req["messages"].([]any)
if strings.TrimSpace(model) == "" || len(messagesRaw) == 0 {
return claudeNormalizedRequest{}, fmt.Errorf("Request must include 'model' and 'messages'.")
return claudeNormalizedRequest{}, fmt.Errorf("request must include 'model' and 'messages'")
}
if _, ok := req["max_tokens"]; !ok {
req["max_tokens"] = 8192
@@ -36,7 +36,7 @@ func normalizeClaudeRequest(store ConfigReader, req map[string]any) (claudeNorma
thinkingEnabled = false
searchEnabled = false
}
finalPrompt := deepseek.MessagesPrepare(toMessageMaps(dsPayload["messages"]))
finalPrompt := deepseek.MessagesPrepareWithThinking(toMessageMaps(dsPayload["messages"]), thinkingEnabled)
toolNames := extractClaudeToolNames(toolsRequested)
if len(toolNames) == 0 && len(toolsRequested) > 0 {
toolNames = []string{"__any_tool__"}

View File

@@ -19,14 +19,14 @@ type claudeStreamRuntime struct {
toolNames []string
messages []any
thinkingEnabled bool
searchEnabled bool
bufferToolContent bool
thinkingEnabled bool
searchEnabled bool
bufferToolContent bool
stripReferenceMarkers bool
messageID string
thinking strings.Builder
text strings.Builder
outputTokens int
nextBlockIndex int
thinkingBlockOpen bool
@@ -45,21 +45,23 @@ func newClaudeStreamRuntime(
messages []any,
thinkingEnabled bool,
searchEnabled bool,
stripReferenceMarkers bool,
toolNames []string,
) *claudeStreamRuntime {
return &claudeStreamRuntime{
w: w,
rc: rc,
canFlush: canFlush,
model: model,
messages: messages,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
bufferToolContent: len(toolNames) > 0,
toolNames: toolNames,
messageID: fmt.Sprintf("msg_%d", time.Now().UnixNano()),
thinkingBlockIndex: -1,
textBlockIndex: -1,
w: w,
rc: rc,
canFlush: canFlush,
model: model,
messages: messages,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
bufferToolContent: len(toolNames) > 0,
stripReferenceMarkers: stripReferenceMarkers,
toolNames: toolNames,
messageID: fmt.Sprintf("msg_%d", time.Now().UnixNano()),
thinkingBlockIndex: -1,
textBlockIndex: -1,
}
}
@@ -67,9 +69,6 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ErrorMessage != "" {
s.upstreamErr = parsed.ErrorMessage
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReason("upstream_error")}
@@ -80,10 +79,11 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
contentSeen := false
for _, p := range parsed.Parts {
if p.Text == "" {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(p.Text) {
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
contentSeen = true
@@ -92,7 +92,11 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
if !s.thinkingEnabled {
continue
}
s.thinking.WriteString(p.Text)
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
}
s.thinking.WriteString(trimmed)
s.closeTextBlock()
if !s.thinkingBlockOpen {
s.thinkingBlockIndex = s.nextBlockIndex
@@ -112,13 +116,17 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
"index": s.thinkingBlockIndex,
"delta": map[string]any{
"type": "thinking_delta",
"thinking": p.Text,
"thinking": trimmed,
},
})
continue
}
s.text.WriteString(p.Text)
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
continue
}
s.text.WriteString(trimmed)
if s.bufferToolContent {
if hasUnclosedCodeFence(s.text.String()) {
continue
@@ -144,7 +152,7 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
"index": s.textBlockIndex,
"delta": map[string]any{
"type": "text_delta",
"text": p.Text,
"text": trimmed,
},
})
}

View File

@@ -1,6 +1,7 @@
package claude
import (
"ds2api/internal/toolcall"
"encoding/json"
"fmt"
"time"
@@ -43,12 +44,12 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
s.closeTextBlock()
finalThinking := s.thinking.String()
finalText := s.text.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
if s.bufferToolContent {
detected := util.ParseStandaloneToolCalls(finalText, s.toolNames)
detected := toolcall.ParseStandaloneToolCalls(finalText, s.toolNames)
if len(detected) == 0 && finalText == "" && finalThinking != "" {
detected = util.ParseStandaloneToolCalls(finalThinking, s.toolNames)
detected = toolcall.ParseStandaloneToolCalls(finalThinking, s.toolNames)
}
if len(detected) > 0 {
stopReason = "tool_use"
@@ -64,7 +65,7 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
"input": map[string]any{},
},
})
inputBytes, _ := json.Marshal(tc.Input)
s.send("content_block_delta", map[string]any{
"type": "content_block_delta",
@@ -108,9 +109,6 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
}
outputTokens := util.EstimateTokens(finalThinking) + util.EstimateTokens(finalText)
if s.outputTokens > 0 {
outputTokens = s.outputTokens
}
s.send("message_delta", map[string]any{
"type": "message_delta",
"delta": map[string]any{

View File

@@ -28,6 +28,8 @@ func (streamStatusClaudeStoreStub) ClaudeMapping() map[string]string {
}
}
func (streamStatusClaudeStoreStub) CompatStripReferenceMarkers() bool { return true }
func captureClaudeStatusMiddleware(statuses *[]int) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

View File

@@ -5,6 +5,7 @@ import (
"strings"
)
//nolint:unused // compatibility hook for native Gemini request normalization path.
func collectGeminiPassThrough(req map[string]any) map[string]any {
cfg, _ := req["generationConfig"].(map[string]any)
if len(cfg) == 0 {

View File

@@ -9,6 +9,7 @@ import (
"ds2api/internal/util"
)
//nolint:unused // kept for native Gemini adapter route compatibility.
func normalizeGeminiRequest(store ConfigReader, routeModel string, req map[string]any, stream bool) (util.StandardRequest, error) {
requestedModel := strings.TrimSpace(routeModel)
if requestedModel == "" {
@@ -17,17 +18,17 @@ func normalizeGeminiRequest(store ConfigReader, routeModel string, req map[strin
resolvedModel, ok := config.ResolveModel(store, requestedModel)
if !ok {
return util.StandardRequest{}, fmt.Errorf("Model '%s' is not available.", requestedModel)
return util.StandardRequest{}, fmt.Errorf("model %q is not available", requestedModel)
}
thinkingEnabled, searchEnabled, _ := config.GetModelConfig(resolvedModel)
messagesRaw := geminiMessagesFromRequest(req)
if len(messagesRaw) == 0 {
return util.StandardRequest{}, fmt.Errorf("Request must include non-empty contents.")
return util.StandardRequest{}, fmt.Errorf("request must include non-empty contents")
}
toolsRaw := convertGeminiTools(req["tools"])
finalPrompt, toolNames := openai.BuildPromptForAdapter(messagesRaw, toolsRaw, "")
finalPrompt, toolNames := openai.BuildPromptForAdapter(messagesRaw, toolsRaw, "", thinkingEnabled)
passThrough := collectGeminiPassThrough(req)
return util.StandardRequest{

View File

@@ -2,6 +2,7 @@ package gemini
import "strings"
//nolint:unused // kept for native Gemini adapter route compatibility.
func convertGeminiTools(raw any) []any {
tools, _ := raw.([]any)
if len(tools) == 0 {

View File

@@ -22,6 +22,7 @@ type DeepSeekCaller interface {
type ConfigReader interface {
ModelAliases() map[string]string
CompatStripReferenceMarkers() bool
}
type OpenAIChatRunner interface {

View File

@@ -2,6 +2,7 @@ package gemini
import (
"bytes"
"ds2api/internal/toolcall"
"encoding/json"
"io"
"net/http"
@@ -57,7 +58,7 @@ func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, stream
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
defer func() { _ = res.Body.Close() }()
body, _ := io.ReadAll(res.Body)
for k, vv := range res.Header {
for _, v := range vv {
@@ -87,7 +88,7 @@ func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, stream
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
defer func() { _ = res.Body.Close() }()
body, _ := io.ReadAll(res.Body)
if res.StatusCode < 200 || res.StatusCode >= 300 {
for k, vv := range res.Header {
@@ -131,8 +132,9 @@ func writeGeminiErrorFromOpenAI(w http.ResponseWriter, status int, raw []byte) {
writeGeminiError(w, status, message)
}
//nolint:unused // retained for native Gemini non-stream handling path.
func (h *Handler) handleNonStreamGenerateContent(w http.ResponseWriter, resp *http.Response, model, finalPrompt string, thinkingEnabled bool, toolNames []string) {
defer resp.Body.Close()
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeGeminiError(w, resp.StatusCode, strings.TrimSpace(string(body)))
@@ -140,12 +142,20 @@ func (h *Handler) handleNonStreamGenerateContent(w http.ResponseWriter, resp *ht
}
result := sse.CollectStream(resp, thinkingEnabled, true)
writeJSON(w, http.StatusOK, buildGeminiGenerateContentResponse(model, finalPrompt, result.Thinking, result.Text, toolNames, result.OutputTokens))
stripReferenceMarkers := h.compatStripReferenceMarkers()
writeJSON(w, http.StatusOK, buildGeminiGenerateContentResponse(
model,
finalPrompt,
cleanVisibleOutput(result.Thinking, stripReferenceMarkers),
cleanVisibleOutput(result.Text, stripReferenceMarkers),
toolNames,
))
}
func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, finalText string, toolNames []string, outputTokens int) map[string]any {
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, finalText string, toolNames []string) map[string]any {
parts := buildGeminiPartsFromFinal(finalText, finalThinking, toolNames)
usage := buildGeminiUsage(finalPrompt, finalThinking, finalText, outputTokens)
usage := buildGeminiUsage(finalPrompt, finalThinking, finalText)
return map[string]any{
"candidates": []map[string]any{
{
@@ -162,14 +172,11 @@ func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, final
}
}
func buildGeminiUsage(finalPrompt, finalThinking, finalText string, outputTokens int) map[string]any {
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
if outputTokens > 0 {
completionTokens = outputTokens
reasoningTokens = 0
}
return map[string]any{
"promptTokenCount": promptTokens,
"candidatesTokenCount": reasoningTokens + completionTokens,
@@ -177,10 +184,11 @@ func buildGeminiUsage(finalPrompt, finalThinking, finalText string, outputTokens
}
}
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiPartsFromFinal(finalText, finalThinking string, toolNames []string) []map[string]any {
detected := util.ParseToolCalls(finalText, toolNames)
if len(detected) == 0 && strings.TrimSpace(finalThinking) != "" {
detected = util.ParseToolCalls(finalThinking, toolNames)
detected := toolcall.ParseToolCalls(finalText, toolNames)
if len(detected) == 0 && finalThinking != "" {
detected = toolcall.ParseToolCalls(finalThinking, toolNames)
}
if len(detected) > 0 {
parts := make([]map[string]any, 0, len(detected))
@@ -196,7 +204,7 @@ func buildGeminiPartsFromFinal(finalText, finalThinking string, toolNames []stri
}
text := finalText
if strings.TrimSpace(text) == "" {
if text == "" {
text = finalThinking
}
return []map[string]any{{"text": text}}

View File

@@ -17,6 +17,14 @@ type Handler struct {
OpenAI OpenAIChatRunner
}
//nolint:unused // used by native Gemini stream/non-stream runtime helpers.
func (h *Handler) compatStripReferenceMarkers() bool {
if h == nil || h.Store == nil {
return true
}
return h.Store.CompatStripReferenceMarkers()
}
func RegisterRoutes(r chi.Router, h *Handler) {
r.Post("/v1beta/models/{model}:generateContent", h.GenerateContent)
r.Post("/v1beta/models/{model}:streamGenerateContent", h.StreamGenerateContent)

View File

@@ -12,8 +12,9 @@ import (
streamengine "ds2api/internal/stream"
)
//nolint:unused // retained for native Gemini stream handling path.
func (h *Handler) handleStreamGenerateContent(w http.ResponseWriter, r *http.Request, resp *http.Response, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string) {
defer resp.Body.Close()
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeGeminiError(w, resp.StatusCode, strings.TrimSpace(string(body)))
@@ -27,7 +28,7 @@ func (h *Handler) handleStreamGenerateContent(w http.ResponseWriter, r *http.Req
rc := http.NewResponseController(w)
_, canFlush := w.(http.Flusher)
runtime := newGeminiStreamRuntime(w, rc, canFlush, model, finalPrompt, thinkingEnabled, searchEnabled, toolNames)
runtime := newGeminiStreamRuntime(w, rc, canFlush, model, finalPrompt, thinkingEnabled, searchEnabled, h.compatStripReferenceMarkers(), toolNames)
initialType := "text"
if thinkingEnabled {
@@ -49,6 +50,7 @@ func (h *Handler) handleStreamGenerateContent(w http.ResponseWriter, r *http.Req
})
}
//nolint:unused // retained for native Gemini stream handling path.
type geminiStreamRuntime struct {
w http.ResponseWriter
rc *http.ResponseController
@@ -57,16 +59,17 @@ type geminiStreamRuntime struct {
model string
finalPrompt string
thinkingEnabled bool
searchEnabled bool
bufferContent bool
toolNames []string
thinkingEnabled bool
searchEnabled bool
bufferContent bool
stripReferenceMarkers bool
toolNames []string
thinking strings.Builder
text strings.Builder
outputTokens int
}
//nolint:unused // retained for native Gemini stream handling path.
func newGeminiStreamRuntime(
w http.ResponseWriter,
rc *http.ResponseController,
@@ -75,21 +78,24 @@ func newGeminiStreamRuntime(
finalPrompt string,
thinkingEnabled bool,
searchEnabled bool,
stripReferenceMarkers bool,
toolNames []string,
) *geminiStreamRuntime {
return &geminiStreamRuntime{
w: w,
rc: rc,
canFlush: canFlush,
model: model,
finalPrompt: finalPrompt,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
bufferContent: len(toolNames) > 0,
toolNames: toolNames,
w: w,
rc: rc,
canFlush: canFlush,
model: model,
finalPrompt: finalPrompt,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
bufferContent: len(toolNames) > 0,
stripReferenceMarkers: stripReferenceMarkers,
toolNames: toolNames,
}
}
//nolint:unused // retained for native Gemini stream handling path.
func (s *geminiStreamRuntime) sendChunk(payload map[string]any) {
b, _ := json.Marshal(payload)
_, _ = s.w.Write([]byte("data: "))
@@ -100,33 +106,40 @@ func (s *geminiStreamRuntime) sendChunk(payload map[string]any) {
}
}
//nolint:unused // retained for native Gemini stream handling path.
func (s *geminiStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedDecision {
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ContentFilter || parsed.ErrorMessage != "" || parsed.Stop {
return streamengine.ParsedDecision{Stop: true}
}
contentSeen := false
for _, p := range parsed.Parts {
if p.Text == "" {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(p.Text) {
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
contentSeen = true
if p.Type == "thinking" {
if s.thinkingEnabled {
s.thinking.WriteString(p.Text)
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
}
s.thinking.WriteString(trimmed)
}
continue
}
s.text.WriteString(p.Text)
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
continue
}
s.text.WriteString(trimmed)
if s.bufferContent {
continue
}
@@ -136,7 +149,7 @@ func (s *geminiStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
"index": 0,
"content": map[string]any{
"role": "model",
"parts": []map[string]any{{"text": p.Text}},
"parts": []map[string]any{{"text": trimmed}},
},
},
},
@@ -146,9 +159,10 @@ func (s *geminiStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
return streamengine.ParsedDecision{ContentSeen: contentSeen}
}
//nolint:unused // retained for native Gemini stream handling path.
func (s *geminiStreamRuntime) finalize() {
finalThinking := s.thinking.String()
finalText := s.text.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
if s.bufferContent {
parts := buildGeminiPartsFromFinal(finalText, finalThinking, s.toolNames)
@@ -180,6 +194,6 @@ func (s *geminiStreamRuntime) finalize() {
},
},
"modelVersion": s.model,
"usageMetadata": buildGeminiUsage(s.finalPrompt, finalThinking, finalText, s.outputTokens),
"usageMetadata": buildGeminiUsage(s.finalPrompt, finalThinking, finalText),
})
}

View File

@@ -17,7 +17,8 @@ import (
type testGeminiConfig struct{}
func (testGeminiConfig) ModelAliases() map[string]string { return nil }
func (testGeminiConfig) ModelAliases() map[string]string { return nil }
func (testGeminiConfig) CompatStripReferenceMarkers() bool { return true }
type testGeminiAuth struct {
a *auth.RequestAuth
@@ -41,19 +42,23 @@ func (m testGeminiAuth) Determine(_ *http.Request) (*auth.RequestAuth, error) {
func (testGeminiAuth) Release(_ *auth.RequestAuth) {}
//nolint:unused // reserved test double for native Gemini DS-call path coverage.
type testGeminiDS struct {
resp *http.Response
err error
}
//nolint:unused // reserved test double for native Gemini DS-call path coverage.
func (m testGeminiDS) CreateSession(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "session-id", nil
}
//nolint:unused // reserved test double for native Gemini DS-call path coverage.
func (m testGeminiDS) GetPow(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "pow", nil
}
//nolint:unused // reserved test double for native Gemini DS-call path coverage.
func (m testGeminiDS) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ map[string]any, _ string, _ int) (*http.Response, error) {
if m.err != nil {
return nil, m.err
@@ -62,8 +67,8 @@ func (m testGeminiDS) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ m
}
type geminiOpenAIErrorStub struct {
status int
body string
status int
body string
headers map[string]string
}
@@ -77,11 +82,17 @@ func (s geminiOpenAIErrorStub) ChatCompletions(w http.ResponseWriter, _ *http.Re
}
type geminiOpenAISuccessStub struct {
stream bool
body string
stream bool
body string
seenReq map[string]any
}
func (s geminiOpenAISuccessStub) ChatCompletions(w http.ResponseWriter, _ *http.Request) {
func (s *geminiOpenAISuccessStub) ChatCompletions(w http.ResponseWriter, r *http.Request) {
if r != nil {
var req map[string]any
_ = json.NewDecoder(r.Body).Decode(&req)
s.seenReq = req
}
if s.stream {
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(http.StatusOK)
@@ -99,6 +110,7 @@ func (s geminiOpenAISuccessStub) ChatCompletions(w http.ResponseWriter, _ *http.
_, _ = w.Write([]byte(out))
}
//nolint:unused // helper retained for native Gemini stream fixture tests.
func makeGeminiUpstreamResponse(lines ...string) *http.Response {
body := strings.Join(lines, "\n")
if !strings.HasSuffix(body, "\n") {
@@ -138,7 +150,7 @@ func TestGeminiRoutesRegistered(t *testing.T) {
func TestGenerateContentReturnsFunctionCallParts(t *testing.T) {
h := &Handler{
Store: testGeminiConfig{},
OpenAI: geminiOpenAISuccessStub{
OpenAI: &geminiOpenAISuccessStub{
body: `{"id":"chatcmpl-1","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","tool_calls":[{"id":"call_1","type":"function","function":{"name":"eval_javascript","arguments":"{\"code\":\"1+1\"}"}}]},"finish_reason":"tool_calls"}]}`,
},
}
@@ -178,7 +190,7 @@ func TestGenerateContentReturnsFunctionCallParts(t *testing.T) {
}
func TestGenerateContentMixedToolSnippetAlsoTriggersFunctionCall(t *testing.T) {
h := &Handler{Store: testGeminiConfig{}, OpenAI: geminiOpenAISuccessStub{}}
h := &Handler{Store: testGeminiConfig{}, OpenAI: &geminiOpenAISuccessStub{}}
r := chi.NewRouter()
RegisterRoutes(r, h)
@@ -211,7 +223,7 @@ func TestGenerateContentMixedToolSnippetAlsoTriggersFunctionCall(t *testing.T) {
func TestStreamGenerateContentEmitsSSE(t *testing.T) {
h := &Handler{
Store: testGeminiConfig{},
OpenAI: geminiOpenAISuccessStub{stream: true},
OpenAI: &geminiOpenAISuccessStub{stream: true},
}
r := chi.NewRouter()
RegisterRoutes(r, h)
@@ -245,9 +257,42 @@ func TestStreamGenerateContentEmitsSSE(t *testing.T) {
}
}
func TestGeminiProxyTranslatesInlineImageToOpenAIDataURL(t *testing.T) {
openAI := &geminiOpenAISuccessStub{}
h := &Handler{Store: testGeminiConfig{}, OpenAI: openAI}
r := chi.NewRouter()
RegisterRoutes(r, h)
body := `{"contents":[{"role":"user","parts":[{"text":"hello"},{"inlineData":{"mimeType":"image/png","data":"QUJDRA=="}}]}]}`
req := httptest.NewRequest(http.MethodPost, "/v1beta/models/gemini-2.5-pro:generateContent", strings.NewReader(body))
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
messages, _ := openAI.seenReq["messages"].([]any)
if len(messages) != 1 {
t.Fatalf("expected one translated message, got %#v", openAI.seenReq)
}
msg, _ := messages[0].(map[string]any)
content, _ := msg["content"].([]any)
if len(content) != 2 {
t.Fatalf("expected translated content blocks, got %#v", msg)
}
imageBlock, _ := content[1].(map[string]any)
if strings.TrimSpace(asString(imageBlock["type"])) != "image_url" {
t.Fatalf("expected image_url block, got %#v", imageBlock)
}
imageURL, _ := imageBlock["image_url"].(map[string]any)
if !strings.HasPrefix(strings.TrimSpace(asString(imageURL["url"])), "data:image/png;base64,") {
t.Fatalf("expected translated data url, got %#v", imageBlock)
}
}
func TestGenerateContentOpenAIProxyErrorUsesGeminiEnvelope(t *testing.T) {
h := &Handler{
Store: testGeminiConfig{},
Store: testGeminiConfig{},
OpenAI: geminiOpenAIErrorStub{
status: http.StatusUnauthorized,
body: `{"error":{"message":"invalid api key"}}`,

View File

@@ -0,0 +1,14 @@
package gemini
import textclean "ds2api/internal/textclean"
//nolint:unused // retained for native Gemini output post-processing path.
func cleanVisibleOutput(text string, stripReferenceMarkers bool) string {
if text == "" {
return text
}
if stripReferenceMarkers {
text = textclean.StripReferenceMarkers(text)
}
return text
}

View File

@@ -1,6 +1,7 @@
package openai
import (
"ds2api/internal/toolcall"
"encoding/json"
"net/http"
"strings"
@@ -8,7 +9,6 @@ import (
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
"ds2api/internal/util"
)
type chatStreamRuntime struct {
@@ -22,8 +22,9 @@ type chatStreamRuntime struct {
finalPrompt string
toolNames []string
thinkingEnabled bool
searchEnabled bool
thinkingEnabled bool
searchEnabled bool
stripReferenceMarkers bool
firstChunkSent bool
bufferToolContent bool
@@ -36,7 +37,6 @@ type chatStreamRuntime struct {
streamToolNames map[int]string
thinking strings.Builder
text strings.Builder
outputTokens int
}
func newChatStreamRuntime(
@@ -49,25 +49,27 @@ func newChatStreamRuntime(
finalPrompt string,
thinkingEnabled bool,
searchEnabled bool,
stripReferenceMarkers bool,
toolNames []string,
bufferToolContent bool,
emitEarlyToolDeltas bool,
) *chatStreamRuntime {
return &chatStreamRuntime{
w: w,
rc: rc,
canFlush: canFlush,
completionID: completionID,
created: created,
model: model,
finalPrompt: finalPrompt,
toolNames: toolNames,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
bufferToolContent: bufferToolContent,
emitEarlyToolDeltas: emitEarlyToolDeltas,
streamToolCallIDs: map[int]string{},
streamToolNames: map[int]string{},
w: w,
rc: rc,
canFlush: canFlush,
completionID: completionID,
created: created,
model: model,
finalPrompt: finalPrompt,
toolNames: toolNames,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
stripReferenceMarkers: stripReferenceMarkers,
bufferToolContent: bufferToolContent,
emitEarlyToolDeltas: emitEarlyToolDeltas,
streamToolCallIDs: map[int]string{},
streamToolNames: map[int]string{},
}
}
@@ -96,10 +98,23 @@ func (s *chatStreamRuntime) sendDone() {
}
}
func (s *chatStreamRuntime) sendFailedChunk(status int, message, code string) {
s.sendChunk(map[string]any{
"status_code": status,
"error": map[string]any{
"message": message,
"type": openAIErrorType(status),
"code": code,
"param": nil,
},
})
s.sendDone()
}
func (s *chatStreamRuntime) finalize(finishReason string) {
finalThinking := s.thinking.String()
finalText := sanitizeLeakedOutput(s.text.String())
detected := util.ParseStandaloneToolCallsDetailed(finalText, s.toolNames)
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
detected := toolcall.ParseStandaloneToolCallsDetailed(finalText, s.toolNames)
if len(detected.Calls) > 0 && !s.toolCallsDoneEmitted {
finishReason = "tool_calls"
delta := map[string]any{
@@ -142,7 +157,7 @@ func (s *chatStreamRuntime) finalize(finishReason string) {
if evt.Content == "" {
continue
}
cleaned := sanitizeLeakedOutput(evt.Content)
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" {
continue
}
@@ -166,13 +181,22 @@ func (s *chatStreamRuntime) finalize(finishReason string) {
if len(detected.Calls) > 0 || s.toolCallsEmitted {
finishReason = "tool_calls"
}
usage := openaifmt.BuildChatUsage(s.finalPrompt, finalThinking, finalText)
if s.outputTokens > 0 {
usage["completion_tokens"] = s.outputTokens
if prompt, ok := usage["prompt_tokens"].(int); ok {
usage["total_tokens"] = prompt + s.outputTokens
if len(detected.Calls) == 0 && !s.toolCallsEmitted && strings.TrimSpace(finalText) == "" {
status := http.StatusTooManyRequests
message := "Upstream model returned empty output."
code := "upstream_empty_output"
if strings.TrimSpace(finalThinking) != "" {
message = "Upstream model returned reasoning without visible output."
}
if finishReason == "content_filter" {
status = http.StatusBadRequest
message = "Upstream content filtered the response and returned no output."
code = "content_filter"
}
s.sendFailedChunk(status, message, code)
return
}
usage := openaifmt.BuildChatUsage(s.finalPrompt, finalThinking, finalText)
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
@@ -187,10 +211,10 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ContentFilter {
if strings.TrimSpace(s.text.String()) == "" {
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReason("content_filter")}
}
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReasonHandlerRequested}
}
if parsed.ErrorMessage != "" {
@@ -203,10 +227,11 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
newChoices := make([]map[string]any, 0, len(parsed.Parts))
contentSeen := false
for _, p := range parsed.Parts {
if s.searchEnabled && sse.IsCitation(p.Text) {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
if p.Text == "" {
if cleanedText == "" {
continue
}
contentSeen = true
@@ -217,21 +242,29 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
}
if p.Type == "thinking" {
if s.thinkingEnabled {
s.thinking.WriteString(p.Text)
delta["reasoning_content"] = p.Text
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
}
s.thinking.WriteString(trimmed)
delta["reasoning_content"] = trimmed
}
} else {
s.text.WriteString(p.Text)
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
continue
}
s.text.WriteString(trimmed)
if !s.bufferToolContent {
delta["content"] = p.Text
delta["content"] = trimmed
} else {
events := processToolSieveChunk(&s.toolSieve, p.Text, s.toolNames)
events := processToolSieveChunk(&s.toolSieve, trimmed, s.toolNames)
for _, evt := range events {
if len(evt.ToolCallDeltas) > 0 {
if !s.emitEarlyToolDeltas {
continue
}
filtered := filterIncrementalToolCallDeltasByAllowed(evt.ToolCallDeltas, s.toolNames, s.streamToolNames)
filtered := filterIncrementalToolCallDeltasByAllowed(evt.ToolCallDeltas, s.streamToolNames)
if len(filtered) == 0 {
continue
}
@@ -264,7 +297,7 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
continue
}
if evt.Content != "" {
cleaned := sanitizeLeakedOutput(evt.Content)
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" {
continue
}

View File

@@ -0,0 +1,31 @@
package openai
import (
"fmt"
"regexp"
"strconv"
"strings"
)
var citationMarkerPattern = regexp.MustCompile(`(?i)\[citation:\s*(\d+)\]`)
func replaceCitationMarkersWithLinks(text string, links map[int]string) string {
if strings.TrimSpace(text) == "" || len(links) == 0 {
return text
}
return citationMarkerPattern.ReplaceAllStringFunc(text, func(match string) string {
sub := citationMarkerPattern.FindStringSubmatch(match)
if len(sub) < 2 {
return match
}
idx, err := strconv.Atoi(strings.TrimSpace(sub[1]))
if err != nil || idx <= 0 {
return match
}
url := strings.TrimSpace(links[idx])
if url == "" {
return match
}
return fmt.Sprintf("[%d](%s)", idx, url)
})
}

View File

@@ -0,0 +1,28 @@
package openai
import "testing"
func TestReplaceCitationMarkersWithLinks(t *testing.T) {
raw := "这是一条更新[citation:1],更多信息见[citation:2]。"
links := map[int]string{
1: "https://example.com/news-1",
2: "https://example.com/news-2",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "这是一条更新[1](https://example.com/news-1),更多信息见[2](https://example.com/news-2)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksKeepsUnknownIndex(t *testing.T) {
raw := "只有一个来源[citation:1],未知来源[citation:3]。"
links := map[int]string{1: "https://example.com/a"}
got := replaceCitationMarkersWithLinks(raw, links)
want := "只有一个来源[1](https://example.com/a),未知来源[citation:3]。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}

View File

@@ -18,17 +18,21 @@ type AuthResolver interface {
type DeepSeekCaller interface {
CreateSession(ctx context.Context, a *auth.RequestAuth, maxAttempts int) (string, error)
GetPow(ctx context.Context, a *auth.RequestAuth, maxAttempts int) (string, error)
UploadFile(ctx context.Context, a *auth.RequestAuth, req deepseek.UploadFileRequest, maxAttempts int) (*deepseek.UploadFileResult, error)
CallCompletion(ctx context.Context, a *auth.RequestAuth, payload map[string]any, powResp string, maxAttempts int) (*http.Response, error)
DeleteSessionForToken(ctx context.Context, token string, sessionID string) (*deepseek.DeleteSessionResult, error)
DeleteAllSessionsForToken(ctx context.Context, token string) error
}
type ConfigReader interface {
ModelAliases() map[string]string
CompatWideInputStrictOutput() bool
CompatStripReferenceMarkers() bool
ToolcallMode() string
ToolcallEarlyEmitConfidence() string
ResponsesStoreTTLSeconds() int
EmbeddingsProvider() string
AutoDeleteMode() string
AutoDeleteSessions() bool
}

View File

@@ -3,23 +3,31 @@ package openai
import "testing"
type mockOpenAIConfig struct {
aliases map[string]string
wideInput bool
toolMode string
earlyEmit string
responsesTTL int
embedProv string
aliases map[string]string
wideInput bool
autoDeleteMode string
toolMode string
earlyEmit string
responsesTTL int
embedProv string
}
func (m mockOpenAIConfig) ModelAliases() map[string]string { return m.aliases }
func (m mockOpenAIConfig) CompatWideInputStrictOutput() bool {
return m.wideInput
}
func (m mockOpenAIConfig) CompatStripReferenceMarkers() bool { return true }
func (m mockOpenAIConfig) ToolcallMode() string { return m.toolMode }
func (m mockOpenAIConfig) ToolcallEarlyEmitConfidence() string { return m.earlyEmit }
func (m mockOpenAIConfig) ResponsesStoreTTLSeconds() int { return m.responsesTTL }
func (m mockOpenAIConfig) EmbeddingsProvider() string { return m.embedProv }
func (m mockOpenAIConfig) AutoDeleteSessions() bool { return false }
func (m mockOpenAIConfig) AutoDeleteMode() string {
if m.autoDeleteMode == "" {
return "none"
}
return m.autoDeleteMode
}
func (m mockOpenAIConfig) AutoDeleteSessions() bool { return false }
func TestNormalizeOpenAIChatRequestWithConfigInterface(t *testing.T) {
cfg := mockOpenAIConfig{

View File

@@ -26,8 +26,13 @@ func (h *Handler) Embeddings(w http.ResponseWriter, r *http.Request) {
}
defer h.Auth.Release(a)
r.Body = http.MaxBytesReader(w, r.Body, openAIGeneralMaxSize)
var req map[string]any
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if strings.Contains(strings.ToLower(err.Error()), "too large") {
writeOpenAIError(w, http.StatusRequestEntityTooLarge, "request body too large")
return
}
writeOpenAIError(w, http.StatusBadRequest, "invalid json")
return
}

View File

@@ -0,0 +1,382 @@
package openai
import (
"context"
"crypto/sha256"
"encoding/base64"
"fmt"
"mime"
"net/http"
"net/url"
"path/filepath"
"strings"
"ds2api/internal/auth"
"ds2api/internal/deepseek"
)
const maxInlineFilesPerRequest = 50
type inlineFileUploadError struct {
status int
message string
err error
}
func (e *inlineFileUploadError) Error() string {
if e == nil {
return ""
}
if strings.TrimSpace(e.message) != "" {
return e.message
}
if e.err != nil {
return e.err.Error()
}
return "inline file processing failed"
}
type inlineUploadState struct {
ctx context.Context
handler *Handler
auth *auth.RequestAuth
uploadedByID map[string]string
uploadCount int
}
type inlineDecodedFile struct {
Data []byte
ContentType string
Filename string
ReplacementType string
}
func (h *Handler) preprocessInlineFileInputs(ctx context.Context, a *auth.RequestAuth, req map[string]any) error {
if h == nil || h.DS == nil || len(req) == 0 {
return nil
}
state := &inlineUploadState{
ctx: ctx,
handler: h,
auth: a,
uploadedByID: map[string]string{},
}
for _, key := range []string{"messages", "input", "attachments"} {
if raw, ok := req[key]; ok {
updated, err := state.walk(raw)
if err != nil {
return err
}
req[key] = updated
}
}
if refIDs := collectOpenAIRefFileIDs(req); len(refIDs) > 0 {
req["ref_file_ids"] = stringsToAnySlice(refIDs)
}
return nil
}
func writeOpenAIInlineFileError(w http.ResponseWriter, err error) {
inlineErr, ok := err.(*inlineFileUploadError)
if !ok || inlineErr == nil {
writeOpenAIError(w, http.StatusInternalServerError, "Failed to process file input.")
return
}
status := inlineErr.status
if status == 0 {
status = http.StatusInternalServerError
}
message := strings.TrimSpace(inlineErr.message)
if message == "" {
message = "Failed to process file input."
}
writeOpenAIError(w, status, message)
}
func (s *inlineUploadState) walk(raw any) (any, error) {
switch x := raw.(type) {
case []any:
out := make([]any, len(x))
for i, item := range x {
updated, err := s.walk(item)
if err != nil {
return nil, err
}
out[i] = updated
}
return out, nil
case map[string]any:
if replacement, replaced, err := s.tryUploadBlock(x); replaced || err != nil {
return replacement, err
}
for _, key := range []string{"messages", "input", "attachments", "content", "files", "items", "data", "source", "file", "image_url"} {
if nested, ok := x[key]; ok {
updated, err := s.walk(nested)
if err != nil {
return nil, err
}
x[key] = updated
}
}
return x, nil
default:
return raw, nil
}
}
func (s *inlineUploadState) tryUploadBlock(block map[string]any) (map[string]any, bool, error) {
decoded, ok, err := decodeOpenAIInlineFileBlock(block)
if err != nil {
return nil, true, &inlineFileUploadError{status: http.StatusBadRequest, message: err.Error(), err: err}
}
if !ok {
return nil, false, nil
}
if s.uploadCount >= maxInlineFilesPerRequest {
return nil, true, fmt.Errorf("exceeded maximum of %d inline files per request", maxInlineFilesPerRequest)
}
fileID, err := s.uploadInlineFile(decoded)
if err != nil {
return nil, true, &inlineFileUploadError{status: http.StatusInternalServerError, message: "Failed to upload inline file.", err: err}
}
s.uploadCount++
replacement := map[string]any{
"type": decoded.ReplacementType,
"file_id": fileID,
}
if decoded.Filename != "" {
replacement["filename"] = decoded.Filename
}
if decoded.ContentType != "" {
replacement["mime_type"] = decoded.ContentType
}
return replacement, true, nil
}
func (s *inlineUploadState) uploadInlineFile(file inlineDecodedFile) (string, error) {
sum := sha256.Sum256(append([]byte(file.ContentType+"\x00"+file.Filename+"\x00"), file.Data...))
cacheKey := fmt.Sprintf("%x", sum[:])
if fileID, ok := s.uploadedByID[cacheKey]; ok && strings.TrimSpace(fileID) != "" {
return fileID, nil
}
contentType := strings.TrimSpace(file.ContentType)
if contentType == "" {
contentType = http.DetectContentType(file.Data)
}
result, err := s.handler.DS.UploadFile(s.ctx, s.auth, deepseek.UploadFileRequest{
Filename: file.Filename,
ContentType: contentType,
Data: file.Data,
}, 3)
if err != nil {
return "", err
}
fileID := strings.TrimSpace(result.ID)
if fileID == "" {
return "", fmt.Errorf("upload succeeded without file id")
}
s.uploadedByID[cacheKey] = fileID
return fileID, nil
}
func decodeOpenAIInlineFileBlock(block map[string]any) (inlineDecodedFile, bool, error) {
if block == nil {
return inlineDecodedFile{}, false, nil
}
if strings.TrimSpace(asString(block["file_id"])) != "" {
return inlineDecodedFile{}, false, nil
}
if nested, ok := block["file"].(map[string]any); ok {
decoded, matched, err := decodeOpenAIInlineFileBlock(nested)
if err != nil || !matched {
return decoded, matched, err
}
if decoded.Filename == "" {
decoded.Filename = pickInlineFilename(block, decoded.ContentType, defaultInlinePrefix(decoded.ReplacementType))
}
return decoded, true, nil
}
blockType := strings.ToLower(strings.TrimSpace(asString(block["type"])))
if raw, matched := extractInlineImageDataURL(block); matched {
data, contentType, err := decodeInlinePayload(raw, contentTypeFromMap(block))
if err != nil {
return inlineDecodedFile{}, true, fmt.Errorf("invalid image input")
}
return inlineDecodedFile{
Data: data,
ContentType: contentType,
Filename: pickInlineFilename(block, contentType, "image"),
ReplacementType: "input_image",
}, true, nil
}
if raw, matched := extractInlineFilePayload(block, blockType); matched {
data, contentType, err := decodeInlinePayload(raw, contentTypeFromMap(block))
if err != nil {
return inlineDecodedFile{}, true, fmt.Errorf("invalid file input")
}
return inlineDecodedFile{
Data: data,
ContentType: contentType,
Filename: pickInlineFilename(block, contentType, defaultInlinePrefix(blockType)),
ReplacementType: "input_file",
}, true, nil
}
return inlineDecodedFile{}, false, nil
}
func extractInlineImageDataURL(block map[string]any) (string, bool) {
imageURL := block["image_url"]
switch x := imageURL.(type) {
case string:
if isDataURL(x) {
return strings.TrimSpace(x), true
}
case map[string]any:
if raw := strings.TrimSpace(asString(x["url"])); isDataURL(raw) {
return raw, true
}
}
if raw := strings.TrimSpace(asString(block["url"])); isDataURL(raw) {
return raw, true
}
return "", false
}
func extractInlineFilePayload(block map[string]any, blockType string) (string, bool) {
for _, value := range []any{block["file_data"], block["base64"], block["data"]} {
if raw := strings.TrimSpace(asString(value)); raw != "" {
if strings.Contains(blockType, "file") || block["file_data"] != nil || block["filename"] != nil || block["file_name"] != nil || block["name"] != nil {
return raw, true
}
}
}
return "", false
}
func decodeInlinePayload(raw string, explicitContentType string) ([]byte, string, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil, "", fmt.Errorf("empty payload")
}
if isDataURL(raw) {
return decodeDataURL(raw, explicitContentType)
}
decoded, err := decodeBase64Flexible(raw)
if err != nil {
return nil, "", err
}
contentType := strings.TrimSpace(explicitContentType)
if contentType == "" && len(decoded) > 0 {
contentType = http.DetectContentType(decoded)
}
return decoded, contentType, nil
}
func decodeDataURL(raw string, explicitContentType string) ([]byte, string, error) {
raw = strings.TrimSpace(raw)
if !isDataURL(raw) {
return nil, "", fmt.Errorf("unsupported data url")
}
header, payload, ok := strings.Cut(raw, ",")
if !ok {
return nil, "", fmt.Errorf("invalid data url")
}
meta := strings.TrimSpace(strings.TrimPrefix(header, "data:"))
contentType := strings.TrimSpace(explicitContentType)
if contentType == "" {
contentType = "application/octet-stream"
if meta != "" {
parts := strings.Split(meta, ";")
if len(parts) > 0 && strings.TrimSpace(parts[0]) != "" {
contentType = strings.TrimSpace(parts[0])
}
}
}
if strings.Contains(strings.ToLower(meta), ";base64") {
decoded, err := decodeBase64Flexible(payload)
if err != nil {
return nil, "", err
}
return decoded, contentType, nil
}
decoded, err := url.PathUnescape(payload)
if err != nil {
return nil, "", err
}
return []byte(decoded), contentType, nil
}
func decodeBase64Flexible(raw string) ([]byte, error) {
raw = strings.TrimSpace(raw)
for _, enc := range []*base64.Encoding{base64.StdEncoding, base64.RawStdEncoding, base64.URLEncoding, base64.RawURLEncoding} {
decoded, err := enc.DecodeString(raw)
if err == nil {
return decoded, nil
}
}
return nil, fmt.Errorf("invalid base64 payload")
}
func contentTypeFromMap(block map[string]any) string {
for _, value := range []any{block["mime_type"], block["mimeType"], block["content_type"], block["contentType"], block["media_type"], block["mediaType"]} {
if contentType := strings.TrimSpace(asString(value)); contentType != "" {
return contentType
}
}
if imageURL, ok := block["image_url"].(map[string]any); ok {
for _, value := range []any{imageURL["mime_type"], imageURL["mimeType"], imageURL["content_type"], imageURL["contentType"]} {
if contentType := strings.TrimSpace(asString(value)); contentType != "" {
return contentType
}
}
}
return ""
}
func pickInlineFilename(block map[string]any, contentType string, prefix string) string {
for _, value := range []any{block["filename"], block["file_name"], block["name"]} {
if name := strings.TrimSpace(asString(value)); name != "" {
return filepath.Base(name)
}
}
if prefix == "" {
prefix = "upload"
}
ext := ".bin"
if parsedType := strings.TrimSpace(contentType); parsedType != "" {
if comma := strings.Index(parsedType, ";"); comma >= 0 {
parsedType = strings.TrimSpace(parsedType[:comma])
}
if exts, err := mime.ExtensionsByType(parsedType); err == nil && len(exts) > 0 && strings.TrimSpace(exts[0]) != "" {
ext = exts[0]
}
}
return prefix + ext
}
func defaultInlinePrefix(blockType string) string {
blockType = strings.ToLower(strings.TrimSpace(blockType))
if strings.Contains(blockType, "image") {
return "image"
}
return "upload"
}
func isDataURL(raw string) bool {
return strings.HasPrefix(strings.ToLower(strings.TrimSpace(raw)), "data:")
}
func stringsToAnySlice(items []string) []any {
out := make([]any, 0, len(items))
for _, item := range items {
trimmed := strings.TrimSpace(item)
if trimmed == "" {
continue
}
out = append(out, trimmed)
}
if len(out) == 0 {
return nil
}
return out
}

View File

@@ -0,0 +1,274 @@
package openai
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"ds2api/internal/auth"
"ds2api/internal/deepseek"
)
type inlineUploadDSStub struct {
uploadCalls []deepseek.UploadFileRequest
lastCtx context.Context
completionReq map[string]any
createSession string
uploadErr error
completionResp *http.Response
}
func (m *inlineUploadDSStub) CreateSession(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
if strings.TrimSpace(m.createSession) == "" {
return "session-id", nil
}
return m.createSession, nil
}
func (m *inlineUploadDSStub) GetPow(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "pow", nil
}
func (m *inlineUploadDSStub) UploadFile(ctx context.Context, _ *auth.RequestAuth, req deepseek.UploadFileRequest, _ int) (*deepseek.UploadFileResult, error) {
m.lastCtx = ctx
m.uploadCalls = append(m.uploadCalls, req)
if m.uploadErr != nil {
return nil, m.uploadErr
}
return &deepseek.UploadFileResult{
ID: "file-inline-1",
Filename: req.Filename,
Bytes: int64(len(req.Data)),
Status: "uploaded",
Purpose: req.Purpose,
}, nil
}
func (m *inlineUploadDSStub) CallCompletion(_ context.Context, _ *auth.RequestAuth, payload map[string]any, _ string, _ int) (*http.Response, error) {
m.completionReq = payload
if m.completionResp != nil {
return m.completionResp, nil
}
return makeOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"ok"}`,
`data: [DONE]`,
), nil
}
func (m *inlineUploadDSStub) DeleteSessionForToken(_ context.Context, _ string, _ string) (*deepseek.DeleteSessionResult, error) {
return &deepseek.DeleteSessionResult{Success: true}, nil
}
func (m *inlineUploadDSStub) DeleteAllSessionsForToken(_ context.Context, _ string) error {
return nil
}
func TestPreprocessInlineFileInputsReplacesDataURLAndCollectsRefFileIDs(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &Handler{DS: ds}
req := map[string]any{
"messages": []any{
map[string]any{
"role": "user",
"content": []any{
map[string]any{
"type": "image_url",
"image_url": map[string]any{"url": "data:image/png;base64,QUJDRA=="},
},
},
},
},
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if err := h.preprocessInlineFileInputs(ctx, &auth.RequestAuth{DeepSeekToken: "token"}, req); err != nil {
t.Fatalf("preprocess failed: %v", err)
}
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload, got %d", len(ds.uploadCalls))
}
if ds.lastCtx != ctx {
t.Fatalf("expected upload to use request context")
}
if ds.uploadCalls[0].ContentType != "image/png" {
t.Fatalf("expected image/png, got %q", ds.uploadCalls[0].ContentType)
}
if ds.uploadCalls[0].Filename != "image.png" {
t.Fatalf("expected inferred filename image.png, got %q", ds.uploadCalls[0].Filename)
}
messages, _ := req["messages"].([]any)
first, _ := messages[0].(map[string]any)
content, _ := first["content"].([]any)
block, _ := content[0].(map[string]any)
if block["type"] != "input_image" {
t.Fatalf("expected input_image replacement, got %#v", block)
}
if block["file_id"] != "file-inline-1" {
t.Fatalf("expected file-inline-1 replacement id, got %#v", block)
}
refIDs, _ := req["ref_file_ids"].([]any)
if len(refIDs) != 1 || refIDs[0] != "file-inline-1" {
t.Fatalf("unexpected ref_file_ids: %#v", req["ref_file_ids"])
}
}
func TestPreprocessInlineFileInputsDeduplicatesIdenticalPayloads(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &Handler{DS: ds}
req := map[string]any{
"messages": []any{
map[string]any{
"role": "user",
"content": []any{
map[string]any{"type": "image_url", "image_url": map[string]any{"url": "data:image/png;base64,QUJDRA=="}},
map[string]any{"type": "image_url", "image_url": map[string]any{"url": "data:image/png;base64,QUJDRA=="}},
},
},
},
}
if err := h.preprocessInlineFileInputs(context.Background(), &auth.RequestAuth{DeepSeekToken: "token"}, req); err != nil {
t.Fatalf("preprocess failed: %v", err)
}
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected deduplicated single upload, got %d", len(ds.uploadCalls))
}
refIDs, _ := req["ref_file_ids"].([]any)
if len(refIDs) != 1 || refIDs[0] != "file-inline-1" {
t.Fatalf("unexpected ref_file_ids after dedupe: %#v", req["ref_file_ids"])
}
}
func TestChatCompletionsUploadsInlineFilesBeforeCompletion(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
reqBody := `{"model":"deepseek-chat","messages":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"image_url","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.ChatCompletions(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
if ds.completionReq == nil {
t.Fatal("expected completion payload to be captured")
}
refIDs, _ := ds.completionReq["ref_file_ids"].([]any)
if len(refIDs) != 1 || refIDs[0] != "file-inline-1" {
t.Fatalf("unexpected completion ref_file_ids: %#v", ds.completionReq["ref_file_ids"])
}
}
func TestResponsesUploadsInlineFilesBeforeCompletion(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
r := chi.NewRouter()
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","input":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"input_image","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
refIDs, _ := ds.completionReq["ref_file_ids"].([]any)
if len(refIDs) != 1 || refIDs[0] != "file-inline-1" {
t.Fatalf("unexpected completion ref_file_ids: %#v", ds.completionReq["ref_file_ids"])
}
}
func TestChatCompletionsInlineUploadFailureReturnsBadRequest(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
reqBody := `{"model":"deepseek-chat","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/png;base64,%%%"}}]}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.ChatCompletions(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", rec.Code, rec.Body.String())
}
if ds.completionReq != nil {
t.Fatalf("did not expect completion call on upload decode error")
}
}
func TestResponsesInlineUploadFailureReturnsInternalServerError(t *testing.T) {
ds := &inlineUploadDSStub{uploadErr: errors.New("boom")}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
r := chi.NewRouter()
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","input":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusInternalServerError {
t.Fatalf("expected 500, got %d body=%s", rec.Code, rec.Body.String())
}
if ds.completionReq != nil {
t.Fatalf("did not expect completion call after upload failure")
}
}
func TestVercelPrepareUploadsInlineFilesBeforeLeasePayload(t *testing.T) {
t.Setenv("VERCEL", "1")
t.Setenv("DS2API_VERCEL_INTERNAL_SECRET", "stream-secret")
ds := &inlineUploadDSStub{}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
r := chi.NewRouter()
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","messages":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"image_url","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":true}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions?__stream_prepare=1", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("X-Ds2-Internal-Token", "stream-secret")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("decode response failed: %v body=%s", err, rec.Body.String())
}
payload, _ := out["payload"].(map[string]any)
if payload == nil {
t.Fatalf("expected payload in prepare response, got %#v", out)
}
refIDs, _ := payload["ref_file_ids"].([]any)
if len(refIDs) != 1 || refIDs[0] != "file-inline-1" {
t.Fatalf("unexpected payload ref_file_ids: %#v", payload["ref_file_ids"])
}
}

View File

@@ -0,0 +1,94 @@
package openai
import "strings"
func collectOpenAIRefFileIDs(req map[string]any) []string {
if len(req) == 0 {
return nil
}
out := make([]string, 0, 4)
seen := map[string]struct{}{}
for _, key := range []string{
"ref_file_ids",
"file_ids",
"attachments",
"messages",
"input",
} {
raw := req[key]
if raw == nil {
continue
}
// Skip top-level strings for 'messages' and 'input' as they are likely plain text content,
// not file IDs. String file IDs are expected in 'ref_file_ids' or 'file_ids'.
if key == "messages" || key == "input" {
if _, ok := raw.(string); ok {
continue
}
}
appendOpenAIRefFileIDs(&out, seen, raw)
}
if len(out) == 0 {
return nil
}
return out
}
func appendOpenAIRefFileIDs(out *[]string, seen map[string]struct{}, raw any) {
switch x := raw.(type) {
case string:
addOpenAIRefFileID(out, seen, x)
case []string:
for _, item := range x {
addOpenAIRefFileID(out, seen, item)
}
case []any:
for _, item := range x {
appendOpenAIRefFileIDs(out, seen, item)
}
case map[string]any:
if fileID := strings.TrimSpace(asString(x["file_id"])); fileID != "" {
addOpenAIRefFileID(out, seen, fileID)
}
if strings.Contains(strings.ToLower(strings.TrimSpace(asString(x["type"]))), "file") {
if fileID := strings.TrimSpace(asString(x["id"])); fileID != "" {
addOpenAIRefFileID(out, seen, fileID)
}
}
if fileMap, ok := x["file"].(map[string]any); ok {
if fileID := strings.TrimSpace(asString(fileMap["file_id"])); fileID != "" {
addOpenAIRefFileID(out, seen, fileID)
}
if fileID := strings.TrimSpace(asString(fileMap["id"])); fileID != "" {
addOpenAIRefFileID(out, seen, fileID)
}
}
// Recurse into potential containers. Note: we do NOT recurse into 'content' or 'input'
// if they are plain strings (handled by the top-level switch), but they are usually
// nested inside the map branch anyway.
// To be safe, we only recurse into these known container keys.
for _, key := range []string{"ref_file_ids", "file_ids", "attachments", "messages", "input", "content", "files", "items", "data", "source"} {
if nested, ok := x[key]; ok {
// If it's a message content that is a string, we must NOT treat it as an ID.
if key == "content" || key == "input" {
if _, ok := nested.(string); ok {
continue
}
}
appendOpenAIRefFileIDs(out, seen, nested)
}
}
}
}
func addOpenAIRefFileID(out *[]string, seen map[string]struct{}, fileID string) {
fileID = strings.TrimSpace(fileID)
if fileID == "" {
return
}
if _, ok := seen[fileID]; ok {
return
}
seen[fileID] = struct{}{}
*out = append(*out, fileID)
}

View File

@@ -0,0 +1,202 @@
package openai
import (
"bytes"
"context"
"encoding/json"
"errors"
"mime/multipart"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"ds2api/internal/auth"
"ds2api/internal/deepseek"
)
type managedFilesAuthStub struct{}
func (managedFilesAuthStub) Determine(_ *http.Request) (*auth.RequestAuth, error) {
return &auth.RequestAuth{
UseConfigToken: true,
DeepSeekToken: "managed-token",
CallerID: "caller:test",
AccountID: "acct-123",
TriedAccounts: map[string]bool{},
}, nil
}
func (managedFilesAuthStub) DetermineCaller(_ *http.Request) (*auth.RequestAuth, error) {
return &auth.RequestAuth{
UseConfigToken: true,
DeepSeekToken: "managed-token",
CallerID: "caller:test",
AccountID: "acct-123",
TriedAccounts: map[string]bool{},
}, nil
}
func (managedFilesAuthStub) Release(_ *auth.RequestAuth) {}
type filesRouteDSStub struct {
lastReq deepseek.UploadFileRequest
upload *deepseek.UploadFileResult
err error
}
func (m *filesRouteDSStub) CreateSession(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "", nil
}
func (m *filesRouteDSStub) GetPow(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "", nil
}
func (m *filesRouteDSStub) UploadFile(_ context.Context, _ *auth.RequestAuth, req deepseek.UploadFileRequest, _ int) (*deepseek.UploadFileResult, error) {
m.lastReq = req
if m.err != nil {
return nil, m.err
}
if m.upload != nil {
return m.upload, nil
}
return &deepseek.UploadFileResult{ID: "file-123", Filename: req.Filename, Bytes: int64(len(req.Data)), Purpose: req.Purpose, Status: "uploaded"}, nil
}
func (m *filesRouteDSStub) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ map[string]any, _ string, _ int) (*http.Response, error) {
return nil, errors.New("not implemented")
}
func (m *filesRouteDSStub) DeleteSessionForToken(_ context.Context, _ string, _ string) (*deepseek.DeleteSessionResult, error) {
return &deepseek.DeleteSessionResult{Success: true}, nil
}
func (m *filesRouteDSStub) DeleteAllSessionsForToken(_ context.Context, _ string) error {
return nil
}
func newMultipartUploadRequest(t *testing.T, purpose string, filename string, data []byte) *http.Request {
t.Helper()
var body bytes.Buffer
writer := multipart.NewWriter(&body)
if purpose != "" {
if err := writer.WriteField("purpose", purpose); err != nil {
t.Fatalf("write purpose failed: %v", err)
}
}
part, err := writer.CreateFormFile("file", filename)
if err != nil {
t.Fatalf("create form file failed: %v", err)
}
if _, err := part.Write(data); err != nil {
t.Fatalf("write file failed: %v", err)
}
if err := writer.Close(); err != nil {
t.Fatalf("close writer failed: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/v1/files", &body)
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", writer.FormDataContentType())
return req
}
func TestFilesRouteUploadSuccess(t *testing.T) {
ds := &filesRouteDSStub{}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
r := chi.NewRouter()
RegisterRoutes(r, h)
req := newMultipartUploadRequest(t, "assistants", "notes.txt", []byte("hello world"))
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if ds.lastReq.Filename != "notes.txt" {
t.Fatalf("expected filename notes.txt, got %q", ds.lastReq.Filename)
}
if ds.lastReq.Purpose != "assistants" {
t.Fatalf("expected purpose assistants, got %q", ds.lastReq.Purpose)
}
if string(ds.lastReq.Data) != "hello world" {
t.Fatalf("unexpected uploaded data: %q", string(ds.lastReq.Data))
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("decode response failed: %v body=%s", err, rec.Body.String())
}
if out["object"] != "file" {
t.Fatalf("expected file object, got %#v", out)
}
if out["id"] != "file-123" {
t.Fatalf("expected file id file-123, got %#v", out["id"])
}
if out["filename"] != "notes.txt" {
t.Fatalf("expected filename notes.txt, got %#v", out["filename"])
}
}
func TestFilesRouteUploadIncludesAccountIDForManagedAccount(t *testing.T) {
ds := &filesRouteDSStub{}
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: managedFilesAuthStub{}, DS: ds}
r := chi.NewRouter()
RegisterRoutes(r, h)
req := newMultipartUploadRequest(t, "assistants", "notes.txt", []byte("hello world"))
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("decode response failed: %v body=%s", err, rec.Body.String())
}
if out["account_id"] != "acct-123" {
t.Fatalf("expected account_id acct-123, got %#v", out["account_id"])
}
}
func TestFilesRouteRejectsNonMultipart(t *testing.T) {
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: &filesRouteDSStub{}}
r := chi.NewRouter()
RegisterRoutes(r, h)
req := httptest.NewRequest(http.MethodPost, "/v1/files", bytes.NewBufferString(`{"purpose":"assistants"}`))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", rec.Code, rec.Body.String())
}
}
func TestFilesRouteRequiresFileField(t *testing.T) {
h := &Handler{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: &filesRouteDSStub{}}
r := chi.NewRouter()
RegisterRoutes(r, h)
var body bytes.Buffer
writer := multipart.NewWriter(&body)
if err := writer.WriteField("purpose", "assistants"); err != nil {
t.Fatalf("write field failed: %v", err)
}
if err := writer.Close(); err != nil {
t.Fatalf("close writer failed: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/v1/files", &body)
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", writer.FormDataContentType())
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", rec.Code, rec.Body.String())
}
}

View File

@@ -5,6 +5,7 @@ import (
"encoding/json"
"io"
"net/http"
"strings"
"time"
"ds2api/internal/auth"
@@ -35,39 +36,35 @@ func (h *Handler) ChatCompletions(w http.ResponseWriter, r *http.Request) {
writeOpenAIError(w, status, detail)
return
}
var sessionID string
defer func() {
// 自动删除会话(同步)
// 必须在 Release 之前同步删除,否则:
// 1. 异步删除时账号已被 Release
// 2. 新请求可能获取到同一账号并开始使用
// 3. 异步删除仍在进行,会截断新请求正在使用的会话
if h.Store.AutoDeleteSessions() && a.DeepSeekToken != "" {
deleteCtx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
err := h.DS.DeleteAllSessionsForToken(deleteCtx, a.DeepSeekToken)
if err != nil {
config.Logger.Warn("[auto_delete_sessions] failed", "account", a.AccountID, "error", err)
} else {
config.Logger.Debug("[auto_delete_sessions] success", "account", a.AccountID)
}
}
h.autoDeleteRemoteSession(r.Context(), a, sessionID)
h.Auth.Release(a)
}()
r = r.WithContext(auth.WithAuth(r.Context(), a))
r.Body = http.MaxBytesReader(w, r.Body, openAIGeneralMaxSize)
var req map[string]any
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if strings.Contains(strings.ToLower(err.Error()), "too large") {
writeOpenAIError(w, http.StatusRequestEntityTooLarge, "request body too large")
return
}
writeOpenAIError(w, http.StatusBadRequest, "invalid json")
return
}
if err := h.preprocessInlineFileInputs(r.Context(), a, req); err != nil {
writeOpenAIInlineFileError(w, err)
return
}
stdReq, err := normalizeOpenAIChatRequest(h.Store, req, requestTraceID(r))
if err != nil {
writeOpenAIError(w, http.StatusBadRequest, err.Error())
return
}
sessionID, err := h.DS.CreateSession(r.Context(), a, 3)
sessionID, err = h.DS.CreateSession(r.Context(), a, 3)
if err != nil {
if a.UseConfigToken {
writeOpenAIError(w, http.StatusUnauthorized, "Account token is invalid. Please re-login the account in admin.")
@@ -91,12 +88,45 @@ func (h *Handler) ChatCompletions(w http.ResponseWriter, r *http.Request) {
h.handleStream(w, r, resp, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames)
return
}
h.handleNonStream(w, r.Context(), resp, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.ToolNames)
h.handleNonStream(w, r.Context(), resp, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames)
}
func (h *Handler) handleNonStream(w http.ResponseWriter, ctx context.Context, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled bool, toolNames []string) {
func (h *Handler) autoDeleteRemoteSession(ctx context.Context, a *auth.RequestAuth, sessionID string) {
mode := h.Store.AutoDeleteMode()
if mode == "none" || a.DeepSeekToken == "" {
return
}
deleteBaseCtx := context.WithoutCancel(ctx)
deleteCtx, cancel := context.WithTimeout(deleteBaseCtx, 10*time.Second)
defer cancel()
switch mode {
case "single":
if sessionID == "" {
config.Logger.Warn("[auto_delete_sessions] skipped single-session delete because session_id is empty", "account", a.AccountID)
return
}
_, err := h.DS.DeleteSessionForToken(deleteCtx, a.DeepSeekToken, sessionID)
if err != nil {
config.Logger.Warn("[auto_delete_sessions] failed", "account", a.AccountID, "mode", mode, "session_id", sessionID, "error", err)
return
}
config.Logger.Debug("[auto_delete_sessions] success", "account", a.AccountID, "mode", mode, "session_id", sessionID)
case "all":
if err := h.DS.DeleteAllSessionsForToken(deleteCtx, a.DeepSeekToken); err != nil {
config.Logger.Warn("[auto_delete_sessions] failed", "account", a.AccountID, "mode", mode, "error", err)
return
}
config.Logger.Debug("[auto_delete_sessions] success", "account", a.AccountID, "mode", mode)
default:
config.Logger.Warn("[auto_delete_sessions] unknown mode", "account", a.AccountID, "mode", mode)
}
}
func (h *Handler) handleNonStream(w http.ResponseWriter, ctx context.Context, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string) {
if resp.StatusCode != http.StatusOK {
defer resp.Body.Close()
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
writeOpenAIError(w, resp.StatusCode, string(body))
return
@@ -104,22 +134,21 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, ctx context.Context, re
_ = ctx
result := sse.CollectStream(resp, thinkingEnabled, true)
finalThinking := result.Thinking
finalText := sanitizeLeakedOutput(result.Text)
respBody := openaifmt.BuildChatCompletion(completionID, model, finalPrompt, finalThinking, finalText, toolNames)
if result.OutputTokens > 0 {
if usage, ok := respBody["usage"].(map[string]any); ok {
usage["completion_tokens"] = result.OutputTokens
if prompt, ok := usage["prompt_tokens"].(int); ok {
usage["total_tokens"] = prompt + result.OutputTokens
}
}
stripReferenceMarkers := h.compatStripReferenceMarkers()
finalThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
finalText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
finalText = replaceCitationMarkersWithLinks(finalText, result.CitationLinks)
}
if writeUpstreamEmptyOutputError(w, finalText, result.ContentFilter) {
return
}
respBody := openaifmt.BuildChatCompletion(completionID, model, finalPrompt, finalThinking, finalText, toolNames)
writeJSON(w, http.StatusOK, respBody)
}
func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string) {
defer resp.Body.Close()
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeOpenAIError(w, resp.StatusCode, string(body))
@@ -138,6 +167,7 @@ func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *htt
created := time.Now().Unix()
bufferToolContent := len(toolNames) > 0
emitEarlyToolDeltas := h.toolcallFeatureMatchEnabled() && h.toolcallEarlyEmitHighConfidence()
stripReferenceMarkers := h.compatStripReferenceMarkers()
initialType := "text"
if thinkingEnabled {
initialType = "thinking"
@@ -153,6 +183,7 @@ func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *htt
finalPrompt,
thinkingEnabled,
searchEnabled,
stripReferenceMarkers,
toolNames,
bufferToolContent,
emitEarlyToolDeltas,

View File

@@ -0,0 +1,143 @@
package openai
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
"ds2api/internal/auth"
"ds2api/internal/deepseek"
)
type autoDeleteModeDSStub struct {
resp *http.Response
singleCalls int
allCalls int
lastSessionID string
lastCtxErr error
}
func (m *autoDeleteModeDSStub) CreateSession(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "session-id", nil
}
func (m *autoDeleteModeDSStub) GetPow(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "pow", nil
}
func (m *autoDeleteModeDSStub) UploadFile(_ context.Context, _ *auth.RequestAuth, _ deepseek.UploadFileRequest, _ int) (*deepseek.UploadFileResult, error) {
return &deepseek.UploadFileResult{ID: "file-id", Filename: "file.txt", Bytes: 1, Status: "uploaded"}, nil
}
func (m *autoDeleteModeDSStub) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ map[string]any, _ string, _ int) (*http.Response, error) {
return m.resp, nil
}
func (m *autoDeleteModeDSStub) DeleteSessionForToken(_ context.Context, _ string, sessionID string) (*deepseek.DeleteSessionResult, error) {
m.singleCalls++
m.lastSessionID = sessionID
return &deepseek.DeleteSessionResult{SessionID: sessionID, Success: true}, nil
}
func (m *autoDeleteModeDSStub) DeleteAllSessionsForToken(_ context.Context, _ string) error {
m.allCalls++
return nil
}
func (m *autoDeleteModeDSStub) DeleteSessionForTokenCtx(ctx context.Context, _ string, sessionID string) (*deepseek.DeleteSessionResult, error) {
m.singleCalls++
m.lastSessionID = sessionID
m.lastCtxErr = ctx.Err()
return &deepseek.DeleteSessionResult{SessionID: sessionID, Success: true}, nil
}
func TestChatCompletionsAutoDeleteModes(t *testing.T) {
tests := []struct {
name string
mode string
wantSingle int
wantAll int
}{
{name: "none", mode: "none"},
{name: "single", mode: "single", wantSingle: 1},
{name: "all", mode: "all", wantAll: 1},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
ds := &autoDeleteModeDSStub{
resp: makeOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"hello"}`,
"data: [DONE]",
),
}
h := &Handler{
Store: mockOpenAIConfig{
wideInput: true,
autoDeleteMode: tc.mode,
},
Auth: streamStatusAuthStub{},
DS: ds,
}
reqBody := `{"model":"deepseek-chat","messages":[{"role":"user","content":"hi"}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.ChatCompletions(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
if ds.singleCalls != tc.wantSingle {
t.Fatalf("single delete calls=%d want=%d", ds.singleCalls, tc.wantSingle)
}
if ds.allCalls != tc.wantAll {
t.Fatalf("all delete calls=%d want=%d", ds.allCalls, tc.wantAll)
}
if tc.wantSingle > 0 && ds.lastSessionID != "session-id" {
t.Fatalf("expected single delete for session-id, got %q", ds.lastSessionID)
}
})
}
}
type autoDeleteCtxDSStub struct {
autoDeleteModeDSStub
}
func (m *autoDeleteCtxDSStub) DeleteSessionForToken(ctx context.Context, token string, sessionID string) (*deepseek.DeleteSessionResult, error) {
return m.DeleteSessionForTokenCtx(ctx, token, sessionID)
}
func (m *autoDeleteCtxDSStub) DeleteAllSessionsForToken(_ context.Context, _ string) error {
m.allCalls++
return nil
}
func TestAutoDeleteRemoteSessionIgnoresCanceledParentContext(t *testing.T) {
ds := &autoDeleteCtxDSStub{}
h := &Handler{
Store: mockOpenAIConfig{
wideInput: true,
autoDeleteMode: "single",
},
DS: ds,
}
a := &auth.RequestAuth{DeepSeekToken: "token", AccountID: "acct"}
ctx, cancel := context.WithCancel(context.Background())
cancel()
h.autoDeleteRemoteSession(ctx, a, "session-id")
if ds.singleCalls != 1 {
t.Fatalf("single delete calls=%d want=1", ds.singleCalls)
}
if ds.lastCtxErr != nil {
t.Fatalf("delete ctx should not inherit cancellation, got %v", ds.lastCtxErr)
}
}

View File

@@ -0,0 +1,104 @@
package openai
import (
"io"
"net/http"
"strings"
"time"
"ds2api/internal/auth"
"ds2api/internal/deepseek"
)
const openAIUploadMaxMemory = 32 << 20
func (h *Handler) UploadFile(w http.ResponseWriter, r *http.Request) {
a, err := h.Auth.Determine(r)
if err != nil {
status := http.StatusUnauthorized
detail := err.Error()
if err == auth.ErrNoAccount {
status = http.StatusTooManyRequests
}
writeOpenAIError(w, status, detail)
return
}
defer h.Auth.Release(a)
if !strings.HasPrefix(strings.ToLower(strings.TrimSpace(r.Header.Get("Content-Type"))), "multipart/form-data") {
writeOpenAIError(w, http.StatusBadRequest, "content-type must be multipart/form-data")
return
}
// Enforce a hard cap on the total request body size to prevent OOM
r.Body = http.MaxBytesReader(w, r.Body, openAIUploadMaxSize)
if err := r.ParseMultipartForm(openAIUploadMaxMemory); err != nil {
if strings.Contains(strings.ToLower(err.Error()), "too large") {
writeOpenAIError(w, http.StatusRequestEntityTooLarge, "file size exceeds limit")
return
}
writeOpenAIError(w, http.StatusBadRequest, "invalid multipart form")
return
}
if r.MultipartForm != nil {
defer func() { _ = r.MultipartForm.RemoveAll() }()
}
r = r.WithContext(auth.WithAuth(r.Context(), a))
file, header, err := r.FormFile("file")
if err != nil {
writeOpenAIError(w, http.StatusBadRequest, "file is required")
return
}
defer func() { _ = file.Close() }()
data, err := io.ReadAll(file)
if err != nil {
writeOpenAIError(w, http.StatusBadRequest, "failed to read uploaded file")
return
}
contentType := strings.TrimSpace(header.Header.Get("Content-Type"))
if contentType == "" && len(data) > 0 {
contentType = http.DetectContentType(data)
}
result, err := h.DS.UploadFile(r.Context(), a, deepseek.UploadFileRequest{
Filename: header.Filename,
ContentType: contentType,
Purpose: strings.TrimSpace(r.FormValue("purpose")),
Data: data,
}, 3)
if err != nil {
writeOpenAIError(w, http.StatusInternalServerError, "Failed to upload file.")
return
}
if result != nil && result.AccountID == "" {
result.AccountID = a.AccountID
}
writeJSON(w, http.StatusOK, buildOpenAIFileObject(result))
}
func buildOpenAIFileObject(result *deepseek.UploadFileResult) map[string]any {
if result == nil {
obj := map[string]any{
"id": "",
"object": "file",
"bytes": 0,
"created_at": time.Now().Unix(),
"filename": "",
"purpose": "",
"status": "uploaded",
"status_details": nil,
}
return obj
}
obj := map[string]any{
"id": result.ID,
"object": "file",
"bytes": result.Bytes,
"created_at": time.Now().Unix(),
"filename": result.Filename,
"purpose": result.Purpose,
"status": result.Status,
"status_details": nil,
}
if result.AccountID != "" {
obj["account_id"] = result.AccountID
}
return obj
}

View File

@@ -13,6 +13,13 @@ import (
"ds2api/internal/util"
)
const (
// openAIUploadMaxSize limits total multipart request body size (100 MiB).
openAIUploadMaxSize = 100 << 20
// openAIGeneralMaxSize limits total JSON request body size (100 MiB).
openAIGeneralMaxSize = 100 << 20
)
// writeJSON is a package-internal alias kept to avoid mass-renaming across
// every call-site in this package.
var writeJSON = util.WriteJSON
@@ -28,6 +35,13 @@ type Handler struct {
responses *responseStore
}
func (h *Handler) compatStripReferenceMarkers() bool {
if h == nil || h.Store == nil {
return true
}
return h.Store.CompatStripReferenceMarkers()
}
type streamLease struct {
Auth *auth.RequestAuth
ExpiresAt time.Time
@@ -39,6 +53,7 @@ func RegisterRoutes(r chi.Router, h *Handler) {
r.Post("/v1/chat/completions", h.ChatCompletions)
r.Post("/v1/responses", h.Responses)
r.Get("/v1/responses/{response_id}", h.GetResponseByID)
r.Post("/v1/files", h.UploadFile)
r.Post("/v1/embeddings", h.Embeddings)
}

View File

@@ -1,6 +1,7 @@
package openai
import (
"ds2api/internal/toolcall"
"encoding/json"
"fmt"
"strings"
@@ -75,7 +76,7 @@ func injectToolPrompt(messages []map[string]any, tools []any, policy util.ToolCh
// buildToolCallInstructions delegates to the shared util implementation.
func buildToolCallInstructions(toolNames []string) string {
return util.BuildToolCallInstructions(toolNames)
return toolcall.BuildToolCallInstructions(toolNames)
}
func formatIncrementalStreamToolCallDeltas(deltas []toolCallDelta, ids map[int]string) []map[string]any {
@@ -112,7 +113,7 @@ func formatIncrementalStreamToolCallDeltas(deltas []toolCallDelta, ids map[int]s
return out
}
func filterIncrementalToolCallDeltasByAllowed(deltas []toolCallDelta, allowedNames []string, seenNames map[int]string) []toolCallDelta {
func filterIncrementalToolCallDeltasByAllowed(deltas []toolCallDelta, seenNames map[int]string) []toolCallDelta {
if len(deltas) == 0 {
return nil
}
@@ -138,7 +139,7 @@ func filterIncrementalToolCallDeltasByAllowed(deltas []toolCallDelta, allowedNam
return out
}
func formatFinalStreamToolCallsWithStableIDs(calls []util.ParsedToolCall, ids map[int]string) []map[string]any {
func formatFinalStreamToolCallsWithStableIDs(calls []toolcall.ParsedToolCall, ids map[int]string) []map[string]any {
if len(calls) == 0 {
return nil
}

View File

@@ -3,7 +3,6 @@ package openai
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
@@ -59,21 +58,6 @@ func parseSSEDataFrames(t *testing.T, body string) ([]map[string]any, bool) {
return frames, done
}
func streamHasRawToolJSONContent(frames []map[string]any) bool {
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
content, _ := delta["content"].(string)
if strings.Contains(content, `"tool_calls"`) {
return true
}
}
}
return false
}
func streamHasToolCallsDelta(frames []map[string]any) bool {
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
@@ -101,364 +85,61 @@ func streamFinishReason(frames []map[string]any) string {
return ""
}
func streamToolCallArgumentChunks(frames []map[string]any) []string {
out := make([]string, 0, 4)
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
toolCalls, _ := delta["tool_calls"].([]any)
for _, tc := range toolCalls {
tcm, _ := tc.(map[string]any)
fn, _ := tcm["function"].(map[string]any)
if args, ok := fn["arguments"].(string); ok && args != "" {
out = append(out, args)
}
}
}
}
return out
}
func TestHandleNonStreamToolCallInterceptsChatModel(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, context.Background(), resp, "cid1", "deepseek-chat", "prompt", false, []string{"search"})
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d", rec.Code)
}
out := decodeJSONBody(t, rec.Body.String())
choices, _ := out["choices"].([]any)
if len(choices) != 1 {
t.Fatalf("unexpected choices: %#v", out["choices"])
}
choice, _ := choices[0].(map[string]any)
if choice["finish_reason"] != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, got %#v", choice["finish_reason"])
}
msg, _ := choice["message"].(map[string]any)
if msg["content"] != nil {
t.Fatalf("expected content nil, got %#v", msg["content"])
}
toolCalls, _ := msg["tool_calls"].([]any)
if len(toolCalls) != 1 {
t.Fatalf("expected 1 tool call, got %#v", msg["tool_calls"])
}
}
func TestHandleNonStreamToolCallInterceptsReasonerModel(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/thinking_content","v":"先想一下"}`,
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, context.Background(), resp, "cid2", "deepseek-reasoner", "prompt", true, []string{"search"})
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d", rec.Code)
}
out := decodeJSONBody(t, rec.Body.String())
choices, _ := out["choices"].([]any)
choice, _ := choices[0].(map[string]any)
msg, _ := choice["message"].(map[string]any)
if msg["reasoning_content"] != "先想一下" {
t.Fatalf("expected reasoning_content, got %#v", msg["reasoning_content"])
}
if msg["content"] != nil {
t.Fatalf("expected content nil, got %#v", msg["content"])
}
if choice["finish_reason"] != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, got %#v", choice["finish_reason"])
}
}
func TestHandleNonStreamUnknownToolIntercepted(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"not_in_schema\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, context.Background(), resp, "cid2b", "deepseek-chat", "prompt", false, []string{"search"})
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d", rec.Code)
}
out := decodeJSONBody(t, rec.Body.String())
choices, _ := out["choices"].([]any)
choice, _ := choices[0].(map[string]any)
if choice["finish_reason"] != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, got %#v", choice["finish_reason"])
}
msg, _ := choice["message"].(map[string]any)
toolCalls, _ := msg["tool_calls"].([]any)
if len(toolCalls) != 1 {
t.Fatalf("expected tool_calls for unknown schema name, got %#v", msg["tool_calls"])
}
}
func TestHandleNonStreamEmbeddedToolCallExamplePromotesToolCall(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"下面是示例:"}`,
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: {"p":"response/content","v":"请勿执行。"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, context.Background(), resp, "cid2c", "deepseek-chat", "prompt", false, []string{"search"})
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d", rec.Code)
}
out := decodeJSONBody(t, rec.Body.String())
choices, _ := out["choices"].([]any)
choice, _ := choices[0].(map[string]any)
if choice["finish_reason"] != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, got %#v", choice["finish_reason"])
}
msg, _ := choice["message"].(map[string]any)
toolCalls, _ := msg["tool_calls"].([]any)
if len(toolCalls) != 1 {
t.Fatalf("expected one tool_call field for embedded example: %#v", msg["tool_calls"])
}
content, _ := msg["content"].(string)
if strings.Contains(content, `"tool_calls"`) {
t.Fatalf("expected raw tool_calls json stripped from content, got %#v", content)
}
}
func TestHandleNonStreamFencedToolCallExampleDoesNotPromoteToolCall(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
"data: {\"p\":\"response/content\",\"v\":\"```json\\n{\\\"tool_calls\\\":[{\\\"name\\\":\\\"search\\\",\\\"input\\\":{\\\"q\\\":\\\"go\\\"}}]}\\n```\"}",
`data: [DONE]`,
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, context.Background(), resp, "cid2d", "deepseek-chat", "prompt", false, []string{"search"})
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d", rec.Code)
}
out := decodeJSONBody(t, rec.Body.String())
choices, _ := out["choices"].([]any)
choice, _ := choices[0].(map[string]any)
if choice["finish_reason"] == "tool_calls" {
t.Fatalf("expected fenced example to remain content-only, got finish_reason=%#v", choice["finish_reason"])
}
msg, _ := choice["message"].(map[string]any)
toolCalls, _ := msg["tool_calls"].([]any)
if len(toolCalls) != 0 {
t.Fatalf("expected no tool_call field for fenced example: %#v", msg["tool_calls"])
}
content, _ := msg["content"].(string)
if !strings.Contains(content, `"tool_calls"`) {
t.Fatalf("expected fenced example content preserved, got %q", content)
}
}
// Backward-compatible alias for historical test name used in CI logs.
func TestHandleNonStreamFencedToolCallExamplePromotesToolCall(t *testing.T) {
TestHandleNonStreamFencedToolCallExampleDoesNotPromoteToolCall(t)
}
func TestHandleStreamToolCallInterceptsWithoutRawContentLeak(t *testing.T) {
func TestHandleNonStreamReturns429WhenUpstreamOutputEmpty(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\""}`,
`data: {"p":"response/content","v":",\"input\":{\"q\":\"go\"}}]}"}`,
`data: {"p":"response/content","v":""}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid3", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
h.handleNonStream(rec, context.Background(), resp, "cid-empty", "deepseek-chat", "prompt", false, false, nil)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected status 429 for empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
foundToolIndex := false
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
toolCalls, _ := delta["tool_calls"].([]any)
for _, tc := range toolCalls {
tcm, _ := tc.(map[string]any)
if _, ok := tcm["index"].(float64); ok {
foundToolIndex = true
}
}
}
}
if !foundToolIndex {
t.Fatalf("expected stream tool_calls item with index, body=%s", rec.Body.String())
}
if streamHasRawToolJSONContent(frames) {
t.Fatalf("raw tool_calls JSON leaked in content delta: %s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", out)
}
}
func TestHandleStreamToolCallLargeArgumentsStillIntercepted(t *testing.T) {
func TestHandleNonStreamReturnsContentFilterErrorWhenUpstreamFilteredWithoutOutput(t *testing.T) {
h := &Handler{}
large := strings.Repeat("a", 9000)
payload := fmt.Sprintf(`{"tool_calls":[{"name":"search","input":{"q":"%s"}}]}`, large)
splitAt := len(payload) / 2
resp := makeSSEHTTPResponse(
fmt.Sprintf(`data: {"p":"response/content","v":%q}`, payload[:splitAt]),
fmt.Sprintf(`data: {"p":"response/content","v":%q}`, payload[splitAt:]),
`data: {"code":"content_filter"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid3-large", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
h.handleNonStream(rec, context.Background(), resp, "cid-empty-filtered", "deepseek-chat", "prompt", false, false, nil)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected status 400 for filtered upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
if streamHasRawToolJSONContent(frames) {
t.Fatalf("raw tool_calls JSON leaked in content delta: %s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "content_filter" {
t.Fatalf("expected code=content_filter, got %#v", out)
}
}
func TestHandleStreamReasonerToolCallInterceptsWithoutRawContentLeak(t *testing.T) {
func TestHandleNonStreamReturns429WhenUpstreamHasOnlyThinking(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/thinking_content","v":"思考中"}`,
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: {"p":"response/thinking_content","v":"Only thinking"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid4", "deepseek-reasoner", "prompt", true, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
h.handleNonStream(rec, context.Background(), resp, "cid-thinking-only", "deepseek-reasoner", "prompt", true, false, nil)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected status 429 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
foundToolIndex := false
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
toolCalls, _ := delta["tool_calls"].([]any)
for _, tc := range toolCalls {
tcm, _ := tc.(map[string]any)
if _, ok := tcm["index"].(float64); ok {
foundToolIndex = true
}
}
}
}
if !foundToolIndex {
t.Fatalf("expected stream tool_calls item with index, body=%s", rec.Body.String())
}
if streamHasRawToolJSONContent(frames) {
t.Fatalf("raw tool_calls JSON leaked in content delta: %s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
hasThinkingDelta := false
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if _, ok := delta["reasoning_content"]; ok {
hasThinkingDelta = true
}
}
}
if !hasThinkingDelta {
t.Fatalf("expected reasoning_content delta in reasoner stream: %s", rec.Body.String())
}
}
func TestHandleStreamUnknownToolEmitsToolCall(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"not_in_schema\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid5", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta for unknown schema name, body=%s", rec.Body.String())
}
if streamHasRawToolJSONContent(frames) {
t.Fatalf("did not expect raw tool_calls json leak for unknown schema name: %s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamUnknownToolNoArgsEmitsToolCall(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"not_in_schema\"}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid5b", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta for unknown schema name (no args), body=%s", rec.Body.String())
}
if streamHasRawToolJSONContent(frames) {
t.Fatalf("did not expect raw tool_calls json leak for unknown schema name (no args): %s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", out)
}
}
@@ -500,287 +181,6 @@ func TestHandleStreamToolsPlainTextStreamsBeforeFinish(t *testing.T) {
}
}
func TestHandleStreamToolCallMixedWithPlainTextSegments(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"下面是示例:"}`,
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: {"p":"response/content","v":"请勿执行。"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid7", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta in mixed prose stream, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if !strings.Contains(got, "下面是示例:") || !strings.Contains(got, "请勿执行。") {
t.Fatalf("expected pre/post plain text to pass sieve, got=%q", got)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls for mixed prose, body=%s", rec.Body.String())
}
}
func TestHandleStreamToolCallAfterLeadingTextRemainsText(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"我将调用工具。"}`,
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid7b", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if !strings.Contains(got, "我将调用工具。") {
t.Fatalf("expected leading text to keep streaming, got=%q", got)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamToolCallWithSameChunkTrailingTextRemainsText(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}接下来我会继续说明。"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid7c", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if !strings.Contains(got, "接下来我会继续说明。") {
t.Fatalf("expected trailing plain text to be preserved, got=%q", got)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamFencedToolCallSnippetPromotesToolCall(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
fmt.Sprintf(`data: {"p":"response/content","v":%q}`, "下面是调用示例:\n```json\n"),
fmt.Sprintf(`data: {"p":"response/content","v":%q}`, "{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}\n```\n仅示例不要执行。"),
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid7f", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta for fenced snippet, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if strings.Contains(strings.ToLower(got), "tool_calls") {
t.Fatalf("expected raw fenced tool_calls snippet stripped from content, got=%q", got)
}
if strings.Contains(strings.ToLower(got), "```json") || strings.Contains(got, "\n```\n") {
t.Fatalf("expected consumed fenced tool payload to not leave empty code fence, got=%q", got)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamStandaloneToolCallAfterClosedFenceKeepsFence(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
fmt.Sprintf(`data: {"p":"response/content","v":%q}`, "先给一个代码示例:\n```text\nhello\n```\n"),
fmt.Sprintf(`data: {"p":"response/content","v":%q}`, "{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"),
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid7g", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta for standalone payload, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if !strings.Contains(got, "```") {
t.Fatalf("expected closed fence before standalone tool json to be preserved, got=%q", got)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamToolCallKeyAppearsLateRemainsText(t *testing.T) {
h := &Handler{}
spaces := strings.Repeat(" ", 200)
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{`+spaces+`"}`,
`data: {"p":"response/content","v":"\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go\"}}]}"}`,
`data: {"p":"response/content","v":"后置正文C。"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid8", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if !strings.Contains(got, "后置正文C。") {
t.Fatalf("expected stream to continue after tool json convergence, got=%q", got)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamInvalidToolJSONDoesNotLeakRawObject(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"前置正文D。"}`,
`data: {"p":"response/content","v":"{'tool_calls':[{'name':'search','input':{'q':'go'}}]}"}`,
`data: {"p":"response/content","v":"后置正文E。"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid9", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if streamHasToolCallsDelta(frames) {
t.Fatalf("did not expect tool_calls delta for invalid json, body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
got := content.String()
if !strings.Contains(got, "前置正文D。") || !strings.Contains(got, "后置正文E。") {
t.Fatalf("expected pre/post plain text to remain, got=%q", content.String())
}
if !strings.Contains(strings.ToLower(got), "tool_calls") {
t.Fatalf("expected invalid embedded tool-like json to pass through as text, got=%q", got)
}
}
func TestHandleStreamIncompleteCapturedToolJSONFlushesAsTextOnFinalize(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
@@ -814,108 +214,3 @@ func TestHandleStreamIncompleteCapturedToolJSONFlushesAsTextOnFinalize(t *testin
t.Fatalf("expected incomplete capture to flush as plain text instead of stalling, got=%q", content.String())
}
}
func TestHandleStreamToolCallArgumentsEmitAsSingleCompletedChunk(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search\",\"input\":{\"q\":\"go"}`,
`data: {"p":"response/content","v":"lang\",\"page\":1}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid11", "deepseek-chat", "prompt", false, false, []string{"search"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
if streamHasRawToolJSONContent(frames) {
t.Fatalf("raw tool_calls JSON leaked in content delta: %s", rec.Body.String())
}
argChunks := streamToolCallArgumentChunks(frames)
if len(argChunks) == 0 {
t.Fatalf("expected tool call arguments chunk, got=%v body=%s", argChunks, rec.Body.String())
}
joined := strings.Join(argChunks, "")
if !strings.Contains(joined, `"q":"golang"`) || !strings.Contains(joined, `"page":1`) {
t.Fatalf("unexpected merged arguments stream: %q", joined)
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}
func TestHandleStreamMultiToolCallDoesNotMergeNamesOrArguments(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"search_web\",\"input\":{\"query\":\"latest ai news\"}},{"}`,
`data: {"p":"response/content","v":"\"name\":\"eval_javascript\",\"input\":{\"code\":\"1+1\"}}]}"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid12", "deepseek-chat", "prompt", false, false, []string{"search_web", "eval_javascript"})
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta, body=%s", rec.Body.String())
}
foundSearch := false
foundEval := false
foundIndex1 := false
toolCallsDeltaLens := make([]int, 0, 2)
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
rawToolCalls, hasToolCalls := delta["tool_calls"]
if !hasToolCalls {
continue
}
toolCalls, _ := rawToolCalls.([]any)
toolCallsDeltaLens = append(toolCallsDeltaLens, len(toolCalls))
for _, tc := range toolCalls {
tcm, _ := tc.(map[string]any)
if idx, ok := tcm["index"].(float64); ok && int(idx) == 1 {
foundIndex1 = true
}
fn, _ := tcm["function"].(map[string]any)
name, _ := fn["name"].(string)
switch name {
case "search_web":
foundSearch = true
case "eval_javascript":
foundEval = true
case "search_webeval_javascript":
t.Fatalf("unexpected merged tool name: %s, body=%s", name, rec.Body.String())
}
if args, ok := fn["arguments"].(string); ok && strings.Contains(args, `}{"`) {
t.Fatalf("unexpected concatenated tool arguments: %q, body=%s", args, rec.Body.String())
}
}
}
}
if !foundSearch || !foundEval {
t.Fatalf("expected both tool names in stream deltas, foundSearch=%v foundEval=%v body=%s", foundSearch, foundEval, rec.Body.String())
}
if len(toolCallsDeltaLens) != 1 || toolCallsDeltaLens[0] != 2 {
t.Fatalf("expected exactly one tool_calls delta with two calls, got lens=%v body=%s", toolCallsDeltaLens, rec.Body.String())
}
if !foundIndex1 {
t.Fatalf("expected second tool call index in stream deltas, body=%s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
}

View File

@@ -2,16 +2,24 @@ package openai
import (
"regexp"
"strings"
)
var emptyJSONFencePattern = regexp.MustCompile("(?is)```json\\s*```")
var leakedToolCallArrayPattern = regexp.MustCompile(`(?is)\[\{\s*"function"\s*:\s*\{[\s\S]*?\}\s*,\s*"id"\s*:\s*"call[^"]*"\s*,\s*"type"\s*:\s*"function"\s*}\]`)
var leakedToolResultBlobPattern = regexp.MustCompile(`(?is)<\s*\|\s*tool\s*\|\s*>\s*\{[\s\S]*?"tool_call_id"\s*:\s*"call[^"]*"\s*}`)
// leakedMetaMarkerPattern matches DeepSeek special tokens in BOTH forms:
// - ASCII underscore: <end_of_sentence>
// - U+2581 variant: <end▁of▁sentence> (used in some DeepSeek outputs)
var leakedMetaMarkerPattern = regexp.MustCompile(`(?i)<[\|]\s*(?:assistant|tool|end[_▁]of[_▁]sentence|end[_▁]of[_▁]thinking)\s*[\|]>`)
var leakedThinkTagPattern = regexp.MustCompile(`(?is)</?\s*think\s*>`)
// leakedBOSMarkerPattern matches DeepSeek BOS markers in BOTH forms:
// - ASCII underscore: <begin_of_sentence>
// - U+2581 variant: <begin▁of▁sentence>
var leakedBOSMarkerPattern = regexp.MustCompile(`(?i)<[\|]\s*begin[_▁]of[_▁]sentence\s*[\|]>`)
// leakedMetaMarkerPattern matches the remaining DeepSeek special tokens in BOTH forms:
// - ASCII underscore: <end_of_sentence>, <end_of_toolresults>, <end_of_instructions>
// - U+2581 variant: <end▁of▁sentence>, <end▁of▁toolresults>, <end▁of▁instructions>
var leakedMetaMarkerPattern = regexp.MustCompile(`(?i)<[\|]\s*(?:assistant|tool|end[_▁]of[_▁]sentence|end[_▁]of[_▁]thinking|end[_▁]of[_▁]toolresults|end[_▁]of[_▁]instructions)\s*[\|]>`)
// leakedAgentXMLBlockPatterns catch agent-style XML blocks that leak through
// when the sieve fails to capture them. These are applied only to complete
@@ -35,11 +43,48 @@ func sanitizeLeakedOutput(text string) string {
out := emptyJSONFencePattern.ReplaceAllString(text, "")
out = leakedToolCallArrayPattern.ReplaceAllString(out, "")
out = leakedToolResultBlobPattern.ReplaceAllString(out, "")
out = stripDanglingThinkSuffix(out)
out = leakedThinkTagPattern.ReplaceAllString(out, "")
out = leakedBOSMarkerPattern.ReplaceAllString(out, "")
out = leakedMetaMarkerPattern.ReplaceAllString(out, "")
out = sanitizeLeakedAgentXMLBlocks(out)
return out
}
func stripDanglingThinkSuffix(text string) string {
matches := leakedThinkTagPattern.FindAllStringIndex(text, -1)
if len(matches) == 0 {
return text
}
depth := 0
lastOpen := -1
for _, loc := range matches {
tag := strings.ToLower(text[loc[0]:loc[1]])
compact := strings.ReplaceAll(strings.ReplaceAll(strings.TrimSpace(tag), " ", ""), "\t", "")
if strings.HasPrefix(compact, "</") {
if depth > 0 {
depth--
if depth == 0 {
lastOpen = -1
}
}
continue
}
if depth == 0 {
lastOpen = loc[0]
}
depth++
}
if depth == 0 || lastOpen < 0 {
return text
}
prefix := text[:lastOpen]
if strings.TrimSpace(prefix) == "" {
return ""
}
return prefix
}
func sanitizeLeakedAgentXMLBlocks(text string) string {
out := text
for _, pattern := range leakedAgentXMLBlockPatterns {

View File

@@ -19,13 +19,29 @@ func TestSanitizeLeakedOutputRemovesLeakedWireToolCallAndResult(t *testing.T) {
}
func TestSanitizeLeakedOutputRemovesStandaloneMetaMarkers(t *testing.T) {
raw := "A<| end_of_sentence |><| Assistant |>B<| end_of_thinking |>C<end▁of▁thinking>D<end▁of▁sentence>E"
raw := "A<| end_of_sentence |><| Assistant |>B<| end_of_thinking |>C<end▁of▁thinking>D<end▁of▁sentence>E<| end_of_toolresults |>F<end▁of▁instructions>G"
got := sanitizeLeakedOutput(raw)
if got != "ABCDE" {
if got != "ABCDEFG" {
t.Fatalf("unexpected sanitize result for meta markers: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesThinkAndBosMarkers(t *testing.T) {
raw := "A<think>B</think>C<begin▁of▁sentence>D<| begin_of_sentence |>E<begin_of_sentence>F"
got := sanitizeLeakedOutput(raw)
if got != "ABCDEF" {
t.Fatalf("unexpected sanitize result for think/BOS markers: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesDanglingThinkBlock(t *testing.T) {
raw := "Answer prefix<think>internal reasoning that never closes"
got := sanitizeLeakedOutput(raw)
if got != "Answer prefix" {
t.Fatalf("unexpected sanitize result for dangling think block: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesAgentXMLLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer</result></attempt_completion>"
got := sanitizeLeakedOutput(raw)

View File

@@ -22,6 +22,24 @@ func TestGetModelRouteDirectAndAlias(t *testing.T) {
}
})
t.Run("direct_expert", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/v1/models/deepseek-expert-chat", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
})
t.Run("direct_vision", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/v1/models/deepseek-vision-chat", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
})
t.Run("alias", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/v1/models/gpt-4.1", nil)
rec := httptest.NewRecorder()

View File

@@ -0,0 +1,13 @@
package openai
import textclean "ds2api/internal/textclean"
func cleanVisibleOutput(text string, stripReferenceMarkers bool) string {
if text == "" {
return text
}
if stripReferenceMarkers {
text = textclean.StripReferenceMarkers(text)
}
return sanitizeLeakedOutput(text)
}

View File

@@ -5,22 +5,22 @@ import (
"ds2api/internal/util"
)
func buildOpenAIFinalPrompt(messagesRaw []any, toolsRaw any, traceID string) (string, []string) {
return buildOpenAIFinalPromptWithPolicy(messagesRaw, toolsRaw, traceID, util.DefaultToolChoicePolicy())
func buildOpenAIFinalPrompt(messagesRaw []any, toolsRaw any, traceID string, thinkingEnabled bool) (string, []string) {
return buildOpenAIFinalPromptWithPolicy(messagesRaw, toolsRaw, traceID, util.DefaultToolChoicePolicy(), thinkingEnabled)
}
func buildOpenAIFinalPromptWithPolicy(messagesRaw []any, toolsRaw any, traceID string, toolPolicy util.ToolChoicePolicy) (string, []string) {
func buildOpenAIFinalPromptWithPolicy(messagesRaw []any, toolsRaw any, traceID string, toolPolicy util.ToolChoicePolicy, thinkingEnabled bool) (string, []string) {
messages := normalizeOpenAIMessagesForPrompt(messagesRaw, traceID)
toolNames := []string{}
if tools, ok := toolsRaw.([]any); ok && len(tools) > 0 {
messages, toolNames = injectToolPrompt(messages, tools, toolPolicy)
}
return deepseek.MessagesPrepare(messages), toolNames
return deepseek.MessagesPrepareWithThinking(messages, thinkingEnabled), toolNames
}
// BuildPromptForAdapter exposes the OpenAI-compatible prompt building flow so
// other protocol adapters (for example Gemini) can reuse the same tool/history
// normalization logic and remain behavior-compatible with chat/completions.
func BuildPromptForAdapter(messagesRaw []any, toolsRaw any, traceID string) (string, []string) {
return buildOpenAIFinalPrompt(messagesRaw, toolsRaw, traceID)
func BuildPromptForAdapter(messagesRaw []any, toolsRaw any, traceID string, thinkingEnabled bool) (string, []string) {
return buildOpenAIFinalPrompt(messagesRaw, toolsRaw, traceID, thinkingEnabled)
}

View File

@@ -40,7 +40,7 @@ func TestBuildOpenAIFinalPrompt_HandlerPathIncludesToolRoundtripSemantics(t *tes
},
}
finalPrompt, toolNames := buildOpenAIFinalPrompt(messages, tools, "")
finalPrompt, toolNames := buildOpenAIFinalPrompt(messages, tools, "", false)
if len(toolNames) != 1 || toolNames[0] != "get_weather" {
t.Fatalf("unexpected tool names: %#v", toolNames)
}
@@ -73,20 +73,31 @@ func TestBuildOpenAIFinalPrompt_VercelPreparePathKeepsFinalAnswerInstruction(t *
},
}
finalPrompt, _ := buildOpenAIFinalPrompt(messages, tools, "")
if !strings.Contains(finalPrompt, "After receiving a tool result, use it directly.") {
t.Fatalf("vercel prepare finalPrompt missing final-answer instruction: %q", finalPrompt)
}
if !strings.Contains(finalPrompt, "Only call another tool if the result is insufficient.") {
t.Fatalf("vercel prepare finalPrompt missing retry guard instruction: %q", finalPrompt)
finalPrompt, _ := buildOpenAIFinalPrompt(messages, tools, "", false)
if !strings.Contains(finalPrompt, "Remember: The ONLY valid way to use tools is the <tool_calls> XML block at the end of your response.") {
t.Fatalf("vercel prepare finalPrompt missing final tool-call anchor instruction: %q", finalPrompt)
}
if !strings.Contains(finalPrompt, "TOOL CALL FORMAT") {
t.Fatalf("vercel prepare finalPrompt missing xml format instruction: %q", finalPrompt)
}
if !strings.Contains(finalPrompt, "Do NOT wrap the XML in markdown code fences") {
if !strings.Contains(finalPrompt, "Do NOT wrap XML in markdown fences") {
t.Fatalf("vercel prepare finalPrompt missing no-fence xml instruction: %q", finalPrompt)
}
if strings.Contains(finalPrompt, "```json") {
t.Fatalf("vercel prepare finalPrompt should not require fenced tool calls: %q", finalPrompt)
}
}
func TestBuildOpenAIFinalPromptWithThinkingAddsContinuationContract(t *testing.T) {
messages := []any{
map[string]any{"role": "user", "content": "继续回答上一个问题"},
}
finalPrompt, _ := buildOpenAIFinalPrompt(messages, nil, "", true)
if !strings.Contains(finalPrompt, "Continue the conversation from the full prior context") {
t.Fatalf("expected continuation contract in thinking prompt, got=%q", finalPrompt)
}
if !strings.Contains(finalPrompt, "final user-facing answer only in reasoning") {
t.Fatalf("expected visible-answer contract in thinking prompt, got=%q", finalPrompt)
}
}

View File

@@ -156,6 +156,33 @@ func TestNormalizeResponsesInputAsMessagesFunctionCallItemPreservesConcatenatedA
}
}
func TestCollectOpenAIRefFileIDs(t *testing.T) {
got := collectOpenAIRefFileIDs(map[string]any{
"ref_file_ids": []any{"file-top", "file-dup"},
"attachments": []any{
map[string]any{"file_id": "file-attachment"},
},
"input": []any{
map[string]any{
"type": "message",
"content": []any{
map[string]any{"type": "input_file", "file_id": "file-input"},
map[string]any{"type": "input_file", "id": "file-dup"},
},
},
},
})
want := []string{"file-top", "file-dup", "file-attachment", "file-input"}
if len(got) != len(want) {
t.Fatalf("expected %d file ids, got %#v", len(want), got)
}
for i, id := range want {
if got[i] != id {
t.Fatalf("unexpected file ids at %d: got=%#v want=%#v", i, got, want)
}
}
}
func TestExtractEmbeddingInputs(t *testing.T) {
got := extractEmbeddingInputs([]any{"a", "b"})
if len(got) != 2 || got[0] != "a" || got[1] != "b" {

View File

@@ -1,6 +1,7 @@
package openai
import (
"ds2api/internal/toolcall"
"encoding/json"
"io"
"net/http"
@@ -64,11 +65,20 @@ func (h *Handler) Responses(w http.ResponseWriter, r *http.Request) {
return
}
r.Body = http.MaxBytesReader(w, r.Body, openAIGeneralMaxSize)
var req map[string]any
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if strings.Contains(strings.ToLower(err.Error()), "too large") {
writeOpenAIError(w, http.StatusRequestEntityTooLarge, "request body too large")
return
}
writeOpenAIError(w, http.StatusBadRequest, "invalid json")
return
}
if err := h.preprocessInlineFileInputs(r.Context(), a, req); err != nil {
writeOpenAIInlineFileError(w, err)
return
}
traceID := requestTraceID(r)
stdReq, err := normalizeOpenAIResponsesRequest(h.Store, req, traceID)
if err != nil {
@@ -102,19 +112,27 @@ func (h *Handler) Responses(w http.ResponseWriter, r *http.Request) {
h.handleResponsesStream(w, r, resp, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolChoice, traceID)
return
}
h.handleResponsesNonStream(w, resp, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.ToolNames, stdReq.ToolChoice, traceID)
h.handleResponsesNonStream(w, resp, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolChoice, traceID)
}
func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled bool, toolNames []string, toolChoice util.ToolChoicePolicy, traceID string) {
defer resp.Body.Close()
func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolChoice util.ToolChoicePolicy, traceID string) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeOpenAIError(w, resp.StatusCode, strings.TrimSpace(string(body)))
return
}
result := sse.CollectStream(resp, thinkingEnabled, true)
sanitizedText := sanitizeLeakedOutput(result.Text)
textParsed := util.ParseStandaloneToolCallsDetailed(sanitizedText, toolNames)
stripReferenceMarkers := h.compatStripReferenceMarkers()
sanitizedThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
sanitizedText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
sanitizedText = replaceCitationMarkersWithLinks(sanitizedText, result.CitationLinks)
}
if writeUpstreamEmptyOutputError(w, sanitizedText, result.ContentFilter) {
return
}
textParsed := toolcall.ParseStandaloneToolCallsDetailed(sanitizedText, toolNames)
logResponsesToolPolicyRejection(traceID, toolChoice, textParsed, "text")
callCount := len(textParsed.Calls)
@@ -123,21 +141,13 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
return
}
responseObj := openaifmt.BuildResponseObject(responseID, model, finalPrompt, result.Thinking, sanitizedText, toolNames)
if result.OutputTokens > 0 {
if usage, ok := responseObj["usage"].(map[string]any); ok {
usage["output_tokens"] = result.OutputTokens
if input, ok := usage["input_tokens"].(int); ok {
usage["total_tokens"] = input + result.OutputTokens
}
}
}
responseObj := openaifmt.BuildResponseObject(responseID, model, finalPrompt, sanitizedThinking, sanitizedText, toolNames)
h.getResponseStore().put(owner, responseID, responseObj)
writeJSON(w, http.StatusOK, responseObj)
}
func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolChoice util.ToolChoicePolicy, traceID string) {
defer resp.Body.Close()
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeOpenAIError(w, resp.StatusCode, strings.TrimSpace(string(body)))
@@ -156,6 +166,7 @@ func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request,
}
bufferToolContent := len(toolNames) > 0
emitEarlyToolDeltas := h.toolcallFeatureMatchEnabled() && h.toolcallEarlyEmitHighConfidence()
stripReferenceMarkers := h.compatStripReferenceMarkers()
streamRuntime := newResponsesStreamRuntime(
w,
@@ -166,6 +177,7 @@ func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request,
finalPrompt,
thinkingEnabled,
searchEnabled,
stripReferenceMarkers,
toolNames,
bufferToolContent,
emitEarlyToolDeltas,
@@ -193,7 +205,7 @@ func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request,
})
}
func logResponsesToolPolicyRejection(traceID string, policy util.ToolChoicePolicy, parsed util.ToolCallParseResult, channel string) {
func logResponsesToolPolicyRejection(traceID string, policy util.ToolChoicePolicy, parsed toolcall.ToolCallParseResult, channel string) {
rejected := filteredRejectedToolNamesForLog(parsed.RejectedToolNames)
if !parsed.RejectedByPolicy || len(rejected) == 0 {
return

View File

@@ -1,6 +1,7 @@
package openai
import (
"ds2api/internal/toolcall"
"net/http"
"strings"
@@ -23,8 +24,9 @@ type responsesStreamRuntime struct {
traceID string
toolChoice util.ToolChoicePolicy
thinkingEnabled bool
searchEnabled bool
thinkingEnabled bool
searchEnabled bool
stripReferenceMarkers bool
bufferToolContent bool
emitEarlyToolDeltas bool
@@ -49,7 +51,6 @@ type responsesStreamRuntime struct {
messagePartAdded bool
sequence int
failed bool
outputTokens int
persistResponse func(obj map[string]any)
}
@@ -63,6 +64,7 @@ func newResponsesStreamRuntime(
finalPrompt string,
thinkingEnabled bool,
searchEnabled bool,
stripReferenceMarkers bool,
toolNames []string,
bufferToolContent bool,
emitEarlyToolDeltas bool,
@@ -71,40 +73,65 @@ func newResponsesStreamRuntime(
persistResponse func(obj map[string]any),
) *responsesStreamRuntime {
return &responsesStreamRuntime{
w: w,
rc: rc,
canFlush: canFlush,
responseID: responseID,
model: model,
finalPrompt: finalPrompt,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
toolNames: toolNames,
bufferToolContent: bufferToolContent,
emitEarlyToolDeltas: emitEarlyToolDeltas,
streamToolCallIDs: map[int]string{},
functionItemIDs: map[int]string{},
functionOutputIDs: map[int]int{},
functionArgs: map[int]string{},
functionDone: map[int]bool{},
functionAdded: map[int]bool{},
functionNames: map[int]string{},
messageOutputID: -1,
toolChoice: toolChoice,
traceID: traceID,
persistResponse: persistResponse,
w: w,
rc: rc,
canFlush: canFlush,
responseID: responseID,
model: model,
finalPrompt: finalPrompt,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
stripReferenceMarkers: stripReferenceMarkers,
toolNames: toolNames,
bufferToolContent: bufferToolContent,
emitEarlyToolDeltas: emitEarlyToolDeltas,
streamToolCallIDs: map[int]string{},
functionItemIDs: map[int]string{},
functionOutputIDs: map[int]int{},
functionArgs: map[int]string{},
functionDone: map[int]bool{},
functionAdded: map[int]bool{},
functionNames: map[int]string{},
messageOutputID: -1,
toolChoice: toolChoice,
traceID: traceID,
persistResponse: persistResponse,
}
}
func (s *responsesStreamRuntime) failResponse(message, code string) {
s.failed = true
failedResp := map[string]any{
"id": s.responseID,
"type": "response",
"object": "response",
"model": s.model,
"status": "failed",
"output": []any{},
"output_text": "",
"error": map[string]any{
"message": message,
"type": "invalid_request_error",
"code": code,
"param": nil,
},
}
if s.persistResponse != nil {
s.persistResponse(failedResp)
}
s.sendEvent("response.failed", openaifmt.BuildResponsesFailedPayload(s.responseID, s.model, message, code))
s.sendDone()
}
func (s *responsesStreamRuntime) finalize() {
finalThinking := s.thinking.String()
finalText := sanitizeLeakedOutput(s.text.String())
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
if s.bufferToolContent {
s.processToolStreamEvents(flushToolSieve(&s.sieve, s.toolNames), true)
}
textParsed := util.ParseStandaloneToolCallsDetailed(finalText, s.toolNames)
textParsed := toolcall.ParseStandaloneToolCallsDetailed(finalText, s.toolNames)
detected := textParsed.Calls
s.logToolPolicyRejections(textParsed)
@@ -118,41 +145,21 @@ func (s *responsesStreamRuntime) finalize() {
s.closeMessageItem()
if s.toolChoice.IsRequired() && len(detected) == 0 {
s.failed = true
message := "tool_choice requires at least one valid tool call."
failedResp := map[string]any{
"id": s.responseID,
"type": "response",
"object": "response",
"model": s.model,
"status": "failed",
"output": []any{},
"output_text": "",
"error": map[string]any{
"message": message,
"type": "invalid_request_error",
"code": "tool_choice_violation",
"param": nil,
},
s.failResponse("tool_choice requires at least one valid tool call.", "tool_choice_violation")
return
}
if len(detected) == 0 && strings.TrimSpace(finalText) == "" {
code := "upstream_empty_output"
message := "Upstream model returned empty output."
if finalThinking != "" {
message = "Upstream model returned reasoning without visible output."
}
if s.persistResponse != nil {
s.persistResponse(failedResp)
}
s.sendEvent("response.failed", openaifmt.BuildResponsesFailedPayload(s.responseID, s.model, message, "tool_choice_violation"))
s.sendDone()
s.failResponse(message, code)
return
}
s.closeIncompleteFunctionItems()
obj := s.buildCompletedResponseObject(finalThinking, finalText, detected)
if s.outputTokens > 0 {
if usage, ok := obj["usage"].(map[string]any); ok {
usage["output_tokens"] = s.outputTokens
if input, ok := usage["input_tokens"].(int); ok {
usage["total_tokens"] = input + s.outputTokens
}
}
}
if s.persistResponse != nil {
s.persistResponse(obj)
}
@@ -160,8 +167,8 @@ func (s *responsesStreamRuntime) finalize() {
s.sendDone()
}
func (s *responsesStreamRuntime) logToolPolicyRejections(textParsed util.ToolCallParseResult) {
logRejected := func(parsed util.ToolCallParseResult, channel string) {
func (s *responsesStreamRuntime) logToolPolicyRejections(textParsed toolcall.ToolCallParseResult) {
logRejected := func(parsed toolcall.ToolCallParseResult, channel string) {
rejected := filteredRejectedToolNamesForLog(parsed.RejectedToolNames)
if !parsed.RejectedByPolicy || len(rejected) == 0 {
return
@@ -181,19 +188,17 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ContentFilter || parsed.ErrorMessage != "" || parsed.Stop {
return streamengine.ParsedDecision{Stop: true}
}
contentSeen := false
for _, p := range parsed.Parts {
if p.Text == "" {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(p.Text) {
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
contentSeen = true
@@ -201,21 +206,25 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
if !s.thinkingEnabled {
continue
}
s.thinking.WriteString(p.Text)
s.sendEvent("response.reasoning.delta", openaifmt.BuildResponsesReasoningDeltaPayload(s.responseID, p.Text))
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
}
s.thinking.WriteString(trimmed)
s.sendEvent("response.reasoning.delta", openaifmt.BuildResponsesReasoningDeltaPayload(s.responseID, trimmed))
continue
}
cleanedText := sanitizeLeakedOutput(p.Text)
if cleanedText == "" {
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
continue
}
s.text.WriteString(cleanedText)
s.text.WriteString(trimmed)
if !s.bufferToolContent {
s.emitTextDelta(cleanedText)
s.emitTextDelta(trimmed)
continue
}
s.processToolStreamEvents(processToolSieveChunk(&s.sieve, cleanedText, s.toolNames), true)
s.processToolStreamEvents(processToolSieveChunk(&s.sieve, trimmed, s.toolNames), true)
}
return streamengine.ParsedDecision{ContentSeen: contentSeen}

View File

@@ -48,7 +48,7 @@ func (s *responsesStreamRuntime) processToolStreamEvents(events []toolStreamEven
if !s.emitEarlyToolDeltas {
continue
}
filtered := filterIncrementalToolCallDeltasByAllowed(evt.ToolCallDeltas, s.toolNames, s.functionNames)
filtered := filterIncrementalToolCallDeltasByAllowed(evt.ToolCallDeltas, s.functionNames)
if len(filtered) == 0 {
continue
}

View File

@@ -1,11 +1,11 @@
package openai
import (
"ds2api/internal/toolcall"
"encoding/json"
"strings"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/util"
"github.com/google/uuid"
)
@@ -69,7 +69,7 @@ func (s *responsesStreamRuntime) ensureMessageContentPartAdded() {
}
func (s *responsesStreamRuntime) emitTextDelta(content string) {
if strings.TrimSpace(content) == "" {
if content == "" {
return
}
s.ensureMessageContentPartAdded()
@@ -208,7 +208,7 @@ func (s *responsesStreamRuntime) emitFunctionCallDeltaEvents(deltas []toolCallDe
}
}
func (s *responsesStreamRuntime) emitFunctionCallDoneEvents(calls []util.ParsedToolCall) {
func (s *responsesStreamRuntime) emitFunctionCallDoneEvents(calls []toolcall.ParsedToolCall) {
for idx, tc := range calls {
if strings.TrimSpace(tc.Name) == "" {
continue

View File

@@ -1,12 +1,12 @@
package openai
import (
"ds2api/internal/toolcall"
"encoding/json"
"sort"
"strings"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/util"
)
func (s *responsesStreamRuntime) closeIncompleteFunctionItems() {
@@ -57,7 +57,7 @@ func (s *responsesStreamRuntime) closeIncompleteFunctionItems() {
}
}
func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, finalText string, calls []util.ParsedToolCall) map[string]any {
func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, finalText string, calls []toolcall.ParsedToolCall) map[string]any {
type indexedItem struct {
index int
item map[string]any
@@ -83,13 +83,13 @@ func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, fin
})
} else if len(calls) == 0 {
content := make([]map[string]any, 0, 2)
if strings.TrimSpace(finalThinking) != "" {
if finalThinking != "" {
content = append(content, map[string]any{
"type": "reasoning",
"text": finalThinking,
})
}
if strings.TrimSpace(finalText) != "" {
if finalText != "" {
content = append(content, map[string]any{
"type": "output_text",
"text": finalText,
@@ -136,10 +136,10 @@ func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, fin
}
outputText := s.visibleText.String()
if strings.TrimSpace(outputText) == "" && len(calls) == 0 {
if strings.TrimSpace(finalText) != "" {
if outputText == "" && len(calls) == 0 {
if finalText != "" {
outputText = finalText
} else if strings.TrimSpace(finalThinking) != "" {
} else if finalThinking != "" {
outputText = finalThinking
}
}

View File

@@ -12,149 +12,6 @@ import (
"ds2api/internal/util"
)
func TestHandleResponsesStreamToolCallsHideRawOutputTextInCompleted(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
}
rawToolJSON := `{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"}}]}`
streamBody := sseLine(rawToolJSON) + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"read_file"}, util.DefaultToolChoicePolicy(), "")
completed, ok := extractSSEEventPayload(rec.Body.String(), "response.completed")
if !ok {
t.Fatalf("expected response.completed event, body=%s", rec.Body.String())
}
responseObj, _ := completed["response"].(map[string]any)
outputText, _ := responseObj["output_text"].(string)
if outputText != "" {
t.Fatalf("expected empty output_text for tool_calls response, got output_text=%q", outputText)
}
output, _ := responseObj["output"].([]any)
if len(output) == 0 {
t.Fatalf("expected structured output entries, got %#v", responseObj["output"])
}
hasFunctionCall := false
hasLegacyWrapper := false
for _, item := range output {
m, _ := item.(map[string]any)
if m == nil {
continue
}
if m["type"] == "function_call" {
hasFunctionCall = true
}
if m["type"] == "tool_calls" {
hasLegacyWrapper = true
}
}
if !hasFunctionCall {
t.Fatalf("expected function_call item, got %#v", responseObj["output"])
}
if hasLegacyWrapper {
t.Fatalf("did not expect legacy tool_calls wrapper, got %#v", responseObj["output"])
}
if strings.Contains(outputText, `"tool_calls"`) {
t.Fatalf("raw tool_calls JSON leaked in output_text: %q", outputText)
}
}
func TestHandleResponsesStreamUsesOfficialOutputItemEvents(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
}
streamBody := sseLine(`{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"}}]}`) + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"read_file"}, util.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.output_item.added") {
t.Fatalf("expected response.output_item.added event, body=%s", body)
}
if !strings.Contains(body, "event: response.output_item.done") {
t.Fatalf("expected response.output_item.done event, body=%s", body)
}
if !strings.Contains(body, "event: response.function_call_arguments.done") {
t.Fatalf("expected response.function_call_arguments.done event, body=%s", body)
}
if strings.Contains(body, "event: response.output_tool_call.delta") || strings.Contains(body, "event: response.output_tool_call.done") {
t.Fatalf("legacy response.output_tool_call.* event must not appear, body=%s", body)
}
addedPayloads := extractAllSSEEventPayloads(body, "response.output_item.added")
hasFunctionCallAdded := false
for _, payload := range addedPayloads {
item, _ := payload["item"].(map[string]any)
if item == nil || asString(item["type"]) != "function_call" {
continue
}
hasFunctionCallAdded = true
if asString(item["arguments"]) != "" {
t.Fatalf("expected in-progress function_call.arguments to start empty string, got %#v", item["arguments"])
}
}
if !hasFunctionCallAdded {
t.Fatalf("expected function_call output_item.added payload, body=%s", body)
}
donePayload, ok := extractSSEEventPayload(body, "response.function_call_arguments.done")
if !ok {
t.Fatalf("expected to parse response.function_call_arguments.done payload, body=%s", body)
}
doneCallID := strings.TrimSpace(asString(donePayload["call_id"]))
if doneCallID == "" {
t.Fatalf("expected non-empty call_id in done payload, payload=%#v", donePayload)
}
completed, ok := extractSSEEventPayload(body, "response.completed")
if !ok {
t.Fatalf("expected response.completed payload, body=%s", body)
}
responseObj, _ := completed["response"].(map[string]any)
output, _ := responseObj["output"].([]any)
var completedCallID string
for _, item := range output {
m, _ := item.(map[string]any)
if m == nil || m["type"] != "function_call" {
continue
}
completedCallID = strings.TrimSpace(asString(m["call_id"]))
if completedCallID != "" {
break
}
}
if completedCallID == "" {
t.Fatalf("expected function_call.call_id in completed output, output=%#v", output)
}
if completedCallID != doneCallID {
t.Fatalf("expected completed call_id to match stream done call_id, done=%q completed=%q", doneCallID, completedCallID)
}
}
func TestHandleResponsesStreamDoesNotEmitReasoningTextCompatEvents(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
@@ -181,51 +38,6 @@ func TestHandleResponsesStreamDoesNotEmitReasoningTextCompatEvents(t *testing.T)
}
}
func TestHandleResponsesStreamMultiToolCallKeepsNameAndCallIDAligned(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
}
streamBody := sseLine(`{"tool_calls":[{"name":"search_web","input":{"query":"latest ai news"}},`) +
sseLine(`{"name":"eval_javascript","input":{"code":"1+1"}}]}`) +
"data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"search_web", "eval_javascript"}, util.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
donePayloads := extractAllSSEEventPayloads(body, "response.function_call_arguments.done")
if len(donePayloads) != 2 {
t.Fatalf("expected two response.function_call_arguments.done events, got %d body=%s", len(donePayloads), body)
}
seenNames := map[string]string{}
for _, payload := range donePayloads {
name := strings.TrimSpace(asString(payload["name"]))
callID := strings.TrimSpace(asString(payload["call_id"]))
if name != "search_web" && name != "eval_javascript" {
t.Fatalf("unexpected tool name in done payload: %#v", payload)
}
if callID == "" {
t.Fatalf("expected non-empty call_id in done payload: %#v", payload)
}
seenNames[name] = callID
}
if seenNames["search_web"] == seenNames["eval_javascript"] {
t.Fatalf("expected distinct call_id per tool, got %#v", seenNames)
}
}
func TestHandleResponsesStreamEmitsOutputTextDoneBeforeContentPartDone(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
@@ -297,123 +109,6 @@ func TestHandleResponsesStreamOutputTextDeltaCarriesItemIndexes(t *testing.T) {
}
}
func TestHandleResponsesStreamThinkingAndMixedToolExampleEmitsFunctionCall(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(path, value string) string {
b, _ := json.Marshal(map[string]any{
"p": path,
"v": value,
})
return "data: " + string(b) + "\n"
}
streamBody := sseLine("response/thinking_content", "thinking...") +
sseLine("response/content", "先读取文件。") +
sseLine("response/content", `{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"}}]}`) +
"data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-reasoner", "prompt", true, false, []string{"read_file"}, util.DefaultToolChoicePolicy(), "")
addedPayloads := extractAllSSEEventPayloads(rec.Body.String(), "response.output_item.added")
if len(addedPayloads) < 1 {
t.Fatalf("expected at least one output_item.added event, got %d body=%s", len(addedPayloads), rec.Body.String())
}
completedPayload, ok := extractSSEEventPayload(rec.Body.String(), "response.completed")
if !ok {
t.Fatalf("expected response.completed payload, body=%s", rec.Body.String())
}
responseObj, _ := completedPayload["response"].(map[string]any)
output, _ := responseObj["output"].([]any)
hasMessage := false
hasFunctionCall := false
for _, item := range output {
m, _ := item.(map[string]any)
if m == nil {
continue
}
if asString(m["type"]) == "message" {
hasMessage = true
}
if asString(m["type"]) == "function_call" {
hasFunctionCall = true
}
}
if !hasMessage {
t.Fatalf("expected message output for mixed prose tool example, output=%#v", output)
}
if !hasFunctionCall {
t.Fatalf("expected function_call output for mixed prose tool example, output=%#v", output)
}
}
func TestHandleResponsesStreamToolChoiceNoneStillAllowsFunctionCall(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
}
streamBody := sseLine(`{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"}}]}`) + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
policy := util.ToolChoicePolicy{Mode: util.ToolChoiceNone}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, nil, policy, "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.function_call_arguments.done") {
t.Fatalf("expected function_call events for tool_choice=none, body=%s", body)
}
}
func TestHandleResponsesStreamMalformedToolJSONFallsBackToText(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
}
// invalid JSON (NaN) should remain plain text in strict mode.
streamBody := sseLine(`{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"},"x":NaN}]}`) + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"read_file"}, util.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if strings.Contains(body, "event: response.function_call_arguments.delta") || strings.Contains(body, "event: response.function_call_arguments.done") {
t.Fatalf("did not expect function_call events for malformed payload in strict mode, body=%s", body)
}
if !strings.Contains(body, "event: response.output_text.delta") {
t.Fatalf("expected response.output_text.delta for malformed payload, body=%s", body)
}
if !strings.Contains(body, "event: response.completed") {
t.Fatalf("expected response.completed event, body=%s", body)
}
}
func TestHandleResponsesStreamRequiredToolChoiceFailure(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
@@ -448,7 +143,7 @@ func TestHandleResponsesStreamRequiredToolChoiceFailure(t *testing.T) {
}
}
func TestHandleResponsesStreamRequiredToolChoiceIgnoresThinkingToolPayload(t *testing.T) {
func TestHandleResponsesStreamFailsWhenUpstreamHasOnlyThinking(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
@@ -461,53 +156,13 @@ func TestHandleResponsesStreamRequiredToolChoiceIgnoresThinkingToolPayload(t *te
return "data: " + string(b) + "\n"
}
streamBody := sseLine("response/thinking_content", `{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"}}]}`) +
sseLine("response/content", "plain text only") +
"data: [DONE]\n"
streamBody := sseLine("response/thinking_content", "Only thinking") + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
policy := util.ToolChoicePolicy{
Mode: util.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", true, false, []string{"read_file"}, policy, "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.failed") {
t.Fatalf("expected response.failed event for required tool_choice violation, body=%s", body)
}
if strings.Contains(body, "event: response.completed") {
t.Fatalf("did not expect response.completed after failure, body=%s", body)
}
}
func TestHandleResponsesStreamRequiredMalformedToolPayloadFails(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
}
streamBody := sseLine(`{"tool_calls":[{"name":"read_file","input":{"path":"README.MD"},"x":NaN}]}`) + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
policy := util.ToolChoicePolicy{
Mode: util.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"read_file"}, policy, "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-reasoner", "prompt", true, false, nil, util.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.failed") {
@@ -516,31 +171,13 @@ func TestHandleResponsesStreamRequiredMalformedToolPayloadFails(t *testing.T) {
if strings.Contains(body, "event: response.completed") {
t.Fatalf("did not expect response.completed, body=%s", body)
}
}
func TestHandleResponsesStreamAllowsUnknownToolName(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
sseLine := func(v string) string {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": v,
})
return "data: " + string(b) + "\n"
payload, ok := extractSSEEventPayload(body, "response.failed")
if !ok {
t.Fatalf("expected response.failed payload, body=%s", body)
}
streamBody := sseLine(`{"tool_calls":[{"name":"not_in_schema","input":{"q":"go"}}]}`) + "data: [DONE]\n"
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"read_file"}, util.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.function_call_arguments.done") {
t.Fatalf("expected function_call events for unknown tool, body=%s", body)
errObj, _ := payload["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", payload)
}
}
@@ -559,7 +196,7 @@ func TestHandleResponsesNonStreamRequiredToolChoiceViolation(t *testing.T) {
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, []string{"read_file"}, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, []string{"read_file"}, policy, "")
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("expected 422 for required tool_choice violation, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -586,7 +223,7 @@ func TestHandleResponsesNonStreamRequiredToolChoiceIgnoresThinkingToolPayload(t
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", true, []string{"read_file"}, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", true, false, []string{"read_file"}, policy, "")
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("expected 422 for required tool_choice violation, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -597,33 +234,69 @@ func TestHandleResponsesNonStreamRequiredToolChoiceIgnoresThinkingToolPayload(t
}
}
func TestHandleResponsesNonStreamToolChoiceNoneStillAllowsFunctionCall(t *testing.T) {
func TestHandleResponsesNonStreamReturns429WhenUpstreamOutputEmpty(t *testing.T) {
h := &Handler{}
rec := httptest.NewRecorder()
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"read_file\",\"input\":{\"path\":\"README.MD\"}}]}"}` + "\n" +
`data: {"p":"response/content","v":""}` + "\n" +
`data: [DONE]` + "\n",
)),
}
policy := util.ToolChoicePolicy{Mode: util.ToolChoiceNone}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, nil, policy, "")
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for tool_choice=none handling, got %d body=%s", rec.Code, rec.Body.String())
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, nil, util.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected 429 for empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
out := decodeJSONBody(t, rec.Body.String())
output, _ := out["output"].([]any)
foundFunctionCall := false
for _, item := range output {
m, _ := item.(map[string]any)
if m != nil && m["type"] == "function_call" {
foundFunctionCall = true
}
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", out)
}
if !foundFunctionCall {
t.Fatalf("expected function_call output item for tool_choice=none, got %#v", output)
}
func TestHandleResponsesNonStreamReturnsContentFilterErrorWhenUpstreamFilteredWithoutOutput(t *testing.T) {
h := &Handler{}
rec := httptest.NewRecorder()
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(
`data: {"code":"content_filter"}` + "\n" +
`data: [DONE]` + "\n",
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-chat", "prompt", false, false, nil, util.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400 for filtered empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "content_filter" {
t.Fatalf("expected code=content_filter, got %#v", out)
}
}
func TestHandleResponsesNonStreamReturns429WhenUpstreamHasOnlyThinking(t *testing.T) {
h := &Handler{}
rec := httptest.NewRecorder()
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(
`data: {"p":"response/thinking_content","v":"Only thinking"}` + "\n" +
`data: [DONE]` + "\n",
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-reasoner", "prompt", true, false, nil, util.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected 429 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", out)
}
}
@@ -652,30 +325,3 @@ func extractSSEEventPayload(body, targetEvent string) (map[string]any, bool) {
}
return nil, false
}
func extractAllSSEEventPayloads(body, targetEvent string) []map[string]any {
scanner := bufio.NewScanner(strings.NewReader(body))
matched := false
out := make([]map[string]any, 0, 2)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if strings.HasPrefix(line, "event: ") {
evt := strings.TrimSpace(strings.TrimPrefix(line, "event: "))
matched = evt == targetEvent
continue
}
if !matched || !strings.HasPrefix(line, "data: ") {
continue
}
raw := strings.TrimSpace(strings.TrimPrefix(line, "data: "))
if raw == "" || raw == "[DONE]" {
continue
}
var payload map[string]any
if err := json.Unmarshal([]byte(raw), &payload); err != nil {
continue
}
out = append(out, payload)
}
return out
}

View File

@@ -12,11 +12,11 @@ func normalizeOpenAIChatRequest(store ConfigReader, req map[string]any, traceID
model, _ := req["model"].(string)
messagesRaw, _ := req["messages"].([]any)
if strings.TrimSpace(model) == "" || len(messagesRaw) == 0 {
return util.StandardRequest{}, fmt.Errorf("Request must include 'model' and 'messages'.")
return util.StandardRequest{}, fmt.Errorf("request must include 'model' and 'messages'")
}
resolvedModel, ok := config.ResolveModel(store, model)
if !ok {
return util.StandardRequest{}, fmt.Errorf("Model '%s' is not available.", model)
return util.StandardRequest{}, fmt.Errorf("model %q is not available", model)
}
thinkingEnabled, searchEnabled, _ := config.GetModelConfig(resolvedModel)
responseModel := strings.TrimSpace(model)
@@ -24,9 +24,10 @@ func normalizeOpenAIChatRequest(store ConfigReader, req map[string]any, traceID
responseModel = resolvedModel
}
toolPolicy := util.DefaultToolChoicePolicy()
finalPrompt, toolNames := buildOpenAIFinalPromptWithPolicy(messagesRaw, req["tools"], traceID, toolPolicy)
finalPrompt, toolNames := buildOpenAIFinalPromptWithPolicy(messagesRaw, req["tools"], traceID, toolPolicy, thinkingEnabled)
toolNames = ensureToolDetectionEnabled(toolNames, req["tools"])
passThrough := collectOpenAIChatPassThrough(req)
refFileIDs := collectOpenAIRefFileIDs(req)
return util.StandardRequest{
Surface: "openai_chat",
@@ -40,6 +41,7 @@ func normalizeOpenAIChatRequest(store ConfigReader, req map[string]any, traceID
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
RefFileIDs: refFileIDs,
PassThrough: passThrough,
}, nil
}
@@ -48,11 +50,11 @@ func normalizeOpenAIResponsesRequest(store ConfigReader, req map[string]any, tra
model, _ := req["model"].(string)
model = strings.TrimSpace(model)
if model == "" {
return util.StandardRequest{}, fmt.Errorf("Request must include 'model'.")
return util.StandardRequest{}, fmt.Errorf("request must include 'model'")
}
resolvedModel, ok := config.ResolveModel(store, model)
if !ok {
return util.StandardRequest{}, fmt.Errorf("Model '%s' is not available.", model)
return util.StandardRequest{}, fmt.Errorf("model %q is not available", model)
}
thinkingEnabled, searchEnabled, _ := config.GetModelConfig(resolvedModel)
@@ -68,18 +70,19 @@ func normalizeOpenAIResponsesRequest(store ConfigReader, req map[string]any, tra
messagesRaw = msgs
}
if len(messagesRaw) == 0 {
return util.StandardRequest{}, fmt.Errorf("Request must include 'input' or 'messages'.")
return util.StandardRequest{}, fmt.Errorf("request must include 'input' or 'messages'")
}
toolPolicy, err := parseToolChoicePolicy(req["tool_choice"], req["tools"])
if err != nil {
return util.StandardRequest{}, err
}
finalPrompt, toolNames := buildOpenAIFinalPromptWithPolicy(messagesRaw, req["tools"], traceID, toolPolicy)
finalPrompt, toolNames := buildOpenAIFinalPromptWithPolicy(messagesRaw, req["tools"], traceID, toolPolicy, thinkingEnabled)
toolNames = ensureToolDetectionEnabled(toolNames, req["tools"])
if !toolPolicy.IsNone() {
toolPolicy.Allowed = namesToSet(toolNames)
}
passThrough := collectOpenAIChatPassThrough(req)
refFileIDs := collectOpenAIRefFileIDs(req)
return util.StandardRequest{
Surface: "openai_responses",
@@ -93,6 +96,7 @@ func normalizeOpenAIResponsesRequest(store ConfigReader, req map[string]any, tra
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
RefFileIDs: refFileIDs,
PassThrough: passThrough,
}, nil
}
@@ -152,7 +156,7 @@ func parseToolChoicePolicy(toolChoiceRaw any, toolsRaw any) (util.ToolChoicePoli
case "required":
policy.Mode = util.ToolChoiceRequired
default:
return util.ToolChoicePolicy{}, fmt.Errorf("Unsupported tool_choice: %q", v)
return util.ToolChoicePolicy{}, fmt.Errorf("unsupported tool_choice: %q", v)
}
case map[string]any:
allowedOverride, hasAllowedOverride, err := parseAllowedToolNames(v["allowed_tools"])
@@ -198,7 +202,7 @@ func parseToolChoicePolicy(toolChoiceRaw any, toolsRaw any) (util.ToolChoicePoli
policy.ForcedName = name
policy.Allowed = namesToSet([]string{name})
default:
return util.ToolChoicePolicy{}, fmt.Errorf("Unsupported tool_choice.type: %q", typ)
return util.ToolChoicePolicy{}, fmt.Errorf("unsupported tool_choice.type: %q", typ)
}
default:
return util.ToolChoicePolicy{}, fmt.Errorf("tool_choice must be a string or object")
@@ -206,7 +210,7 @@ func parseToolChoicePolicy(toolChoiceRaw any, toolsRaw any) (util.ToolChoicePoli
if policy.Mode == util.ToolChoiceRequired || policy.Mode == util.ToolChoiceForced {
if len(declaredNames) == 0 {
return util.ToolChoicePolicy{}, fmt.Errorf("tool_choice=%s requires non-empty tools.", policy.Mode)
return util.ToolChoicePolicy{}, fmt.Errorf("tool_choice=%s requires non-empty tools", policy.Mode)
}
}
if policy.Mode == util.ToolChoiceForced {

View File

@@ -41,6 +41,36 @@ func TestNormalizeOpenAIChatRequest(t *testing.T) {
}
}
func TestNormalizeOpenAIChatRequestCollectsRefFileIDs(t *testing.T) {
store := newEmptyStoreForNormalizeTest(t)
req := map[string]any{
"model": "gpt-5-codex",
"messages": []any{
map[string]any{
"role": "user",
"content": []any{
map[string]any{"type": "input_text", "text": "hello"},
map[string]any{"type": "input_file", "file_id": "file-msg"},
},
},
},
"attachments": []any{
map[string]any{"file_id": "file-attachment"},
},
"ref_file_ids": []any{"file-top", "file-attachment"},
}
n, err := normalizeOpenAIChatRequest(store, req, "")
if err != nil {
t.Fatalf("normalize failed: %v", err)
}
if len(n.RefFileIDs) != 3 {
t.Fatalf("expected 3 distinct file ids, got %#v", n.RefFileIDs)
}
if n.RefFileIDs[0] != "file-top" || n.RefFileIDs[1] != "file-attachment" || n.RefFileIDs[2] != "file-msg" {
t.Fatalf("unexpected file ids: %#v", n.RefFileIDs)
}
}
func TestNormalizeOpenAIResponsesRequestInput(t *testing.T) {
store := newEmptyStoreForNormalizeTest(t)
req := map[string]any{

View File

@@ -13,6 +13,7 @@ import (
chimw "github.com/go-chi/chi/v5/middleware"
"ds2api/internal/auth"
"ds2api/internal/deepseek"
)
type streamStatusAuthStub struct{}
@@ -49,10 +50,18 @@ func (m streamStatusDSStub) GetPow(_ context.Context, _ *auth.RequestAuth, _ int
return "pow", nil
}
func (m streamStatusDSStub) UploadFile(_ context.Context, _ *auth.RequestAuth, _ deepseek.UploadFileRequest, _ int) (*deepseek.UploadFileResult, error) {
return &deepseek.UploadFileResult{ID: "file-id", Filename: "file.txt", Bytes: 1, Status: "uploaded"}, nil
}
func (m streamStatusDSStub) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ map[string]any, _ string, _ int) (*http.Response, error) {
return m.resp, nil
}
func (m streamStatusDSStub) DeleteSessionForToken(_ context.Context, _ string, _ string) (*deepseek.DeleteSessionResult, error) {
return &deepseek.DeleteSessionResult{Success: true}, nil
}
func (m streamStatusDSStub) DeleteAllSessionsForToken(_ context.Context, _ string) error {
return nil
}
@@ -137,53 +146,6 @@ func TestResponsesStreamStatusCapturedAs200(t *testing.T) {
}
}
func TestResponsesNonStreamMixedProseToolPayloadHandlerPath(t *testing.T) {
statuses := make([]int, 0, 1)
content, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": "我来调用工具\n{\"tool_calls\":[{\"name\":\"read_file\",\"input\":{\"path\":\"README.MD\"}}]}",
})
h := &Handler{
Store: mockOpenAIConfig{wideInput: true},
Auth: streamStatusAuthStub{},
DS: streamStatusDSStub{resp: makeOpenAISSEHTTPResponse("data: "+string(content), "data: [DONE]")},
}
r := chi.NewRouter()
r.Use(captureStatusMiddleware(&statuses))
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","input":"请调用工具","tools":[{"type":"function","function":{"name":"read_file","description":"read","parameters":{"type":"object","properties":{"path":{"type":"string"}}}}}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(statuses) != 1 || statuses[0] != http.StatusOK {
t.Fatalf("expected captured status 200, got %#v", statuses)
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("decode response failed: %v body=%s", err, rec.Body.String())
}
outputText, _ := out["output_text"].(string)
if outputText != "" {
t.Fatalf("expected output_text hidden for mixed prose tool payload, got %q", outputText)
}
output, _ := out["output"].([]any)
if len(output) != 1 {
t.Fatalf("expected one output item, got %#v", output)
}
first, _ := output[0].(map[string]any)
if first["type"] != "function_call" {
t.Fatalf("expected function_call output item, got %#v", output)
}
}
func TestChatCompletionsStreamContentFilterStopsNormallyWithoutLeak(t *testing.T) {
statuses := make([]int, 0, 1)
h := &Handler{
@@ -233,3 +195,140 @@ func TestChatCompletionsStreamContentFilterStopsNormallyWithoutLeak(t *testing.T
t.Fatalf("expected finish_reason=stop for content-filter upstream stop, got %#v", choice["finish_reason"])
}
}
func TestChatCompletionsStreamEmitsFailureFrameWhenUpstreamOutputEmpty(t *testing.T) {
statuses := make([]int, 0, 1)
h := &Handler{
Store: mockOpenAIConfig{wideInput: true},
Auth: streamStatusAuthStub{},
DS: streamStatusDSStub{resp: makeOpenAISSEHTTPResponse("data: [DONE]")},
}
r := chi.NewRouter()
r.Use(captureStatusMiddleware(&statuses))
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","messages":[{"role":"user","content":"hi"}],"stream":true}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(statuses) != 1 || statuses[0] != http.StatusOK {
t.Fatalf("expected captured status 200, got %#v", statuses)
}
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if len(frames) != 1 {
t.Fatalf("expected one failure frame, got %#v body=%s", frames, rec.Body.String())
}
last := frames[0]
statusCode, ok := last["status_code"].(float64)
if !ok || int(statusCode) != http.StatusTooManyRequests {
t.Fatalf("expected status_code=429, got %#v body=%s", last["status_code"], rec.Body.String())
}
errObj, _ := last["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", last)
}
}
func TestResponsesStreamUsageIgnoresBatchAccumulatedTokenUsage(t *testing.T) {
statuses := make([]int, 0, 1)
h := &Handler{
Store: mockOpenAIConfig{wideInput: true},
Auth: streamStatusAuthStub{},
DS: streamStatusDSStub{resp: makeOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"hello"}`,
`data: {"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":190},{"p":"quasi_status","v":"FINISHED"}]}`,
)},
}
r := chi.NewRouter()
r.Use(captureStatusMiddleware(&statuses))
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","input":"hi","stream":true}`
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(statuses) != 1 || statuses[0] != http.StatusOK {
t.Fatalf("expected captured status 200, got %#v", statuses)
}
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if len(frames) == 0 {
t.Fatalf("expected at least one json frame, body=%s", rec.Body.String())
}
last := frames[len(frames)-1]
resp, _ := last["response"].(map[string]any)
if resp == nil {
t.Fatalf("expected response payload in final frame, got %#v", last)
}
usage, _ := resp["usage"].(map[string]any)
if usage == nil {
t.Fatalf("expected usage in response payload, got %#v", resp)
}
if got, _ := usage["output_tokens"].(float64); int(got) == 190 {
t.Fatalf("expected upstream accumulated token usage to be ignored, got %#v", usage["output_tokens"])
}
}
func TestResponsesNonStreamUsageIgnoresPromptAndOutputTokenUsage(t *testing.T) {
statuses := make([]int, 0, 1)
h := &Handler{
Store: mockOpenAIConfig{wideInput: true},
Auth: streamStatusAuthStub{},
DS: streamStatusDSStub{resp: makeOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"ok"}`,
`data: {"p":"response","o":"BATCH","v":[{"p":"token_usage","v":{"prompt_tokens":11,"completion_tokens":29}},{"p":"quasi_status","v":"FINISHED"}]}`,
)},
}
r := chi.NewRouter()
r.Use(captureStatusMiddleware(&statuses))
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","input":"hi","stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(statuses) != 1 || statuses[0] != http.StatusOK {
t.Fatalf("expected captured status 200, got %#v", statuses)
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("decode response failed: %v body=%s", err, rec.Body.String())
}
usage, _ := out["usage"].(map[string]any)
if usage == nil {
t.Fatalf("expected usage object, got %#v", out)
}
input, _ := usage["input_tokens"].(float64)
output, _ := usage["output_tokens"].(float64)
total, _ := usage["total_tokens"].(float64)
if int(output) == 29 {
t.Fatalf("expected upstream completion token usage to be ignored, got %#v", usage["output_tokens"])
}
if int(total) != int(input)+int(output) {
t.Fatalf("expected total_tokens=input_tokens+output_tokens, usage=%#v", usage)
}
}

View File

@@ -3,7 +3,7 @@ package openai
import (
"strings"
"ds2api/internal/util"
"ds2api/internal/toolcall"
)
func processToolSieveChunk(state *toolStreamSieveState, chunk string, toolNames []string) []toolStreamEvent {
@@ -60,7 +60,7 @@ func processToolSieveChunk(state *toolStreamSieveState, chunk string, toolNames
if pending == "" {
break
}
start := findToolSegmentStart(pending)
start := findToolSegmentStart(state, pending)
if start >= 0 {
prefix := pending[:start]
if prefix != "" {
@@ -74,7 +74,7 @@ func processToolSieveChunk(state *toolStreamSieveState, chunk string, toolNames
continue
}
safe, hold := splitSafeContentForToolDetection(pending)
safe, hold := splitSafeContentForToolDetection(state, pending)
if safe == "" {
break
}
@@ -114,14 +114,10 @@ func flushToolSieve(state *toolStreamSieveState, toolNames []string) []toolStrea
} else {
content := state.capture.String()
if content != "" {
// If the captured text looks like an incomplete XML tool call block,
// swallow it to prevent leaking raw XML tags to the client.
if hasOpenXMLToolTag(content) {
// Drop it silently — incomplete tool call.
} else {
state.noteText(content)
events = append(events, toolStreamEvent{Content: content})
}
// If capture never resolved into a real tool call, release the
// buffered text instead of swallowing it.
state.noteText(content)
events = append(events, toolStreamEvent{Content: content})
}
}
state.capture.Reset()
@@ -130,109 +126,66 @@ func flushToolSieve(state *toolStreamSieveState, toolNames []string) []toolStrea
}
if state.pending.Len() > 0 {
content := state.pending.String()
// Safety: if pending contains XML tool tag fragments (e.g. "tool_calls>"
// from a split closing tag), swallow them instead of leaking.
if hasOpenXMLToolTag(content) || looksLikeXMLToolTagFragment(content) {
// Drop it — likely an incomplete tool call fragment.
} else {
state.noteText(content)
events = append(events, toolStreamEvent{Content: content})
}
// If pending never resolved into a real tool call, release it as text.
state.noteText(content)
events = append(events, toolStreamEvent{Content: content})
state.pending.Reset()
}
return events
}
func splitSafeContentForToolDetection(s string) (safe, hold string) {
func splitSafeContentForToolDetection(state *toolStreamSieveState, s string) (safe, hold string) {
if s == "" {
return "", ""
}
suspiciousStart := findSuspiciousPrefixStart(s)
if suspiciousStart < 0 {
return s, ""
}
if suspiciousStart > 0 {
return s[:suspiciousStart], s[suspiciousStart:]
}
// If suspicious content starts at position 0, keep holding until we can
// parse a complete tool JSON block or reach stream flush.
return "", s
}
func findSuspiciousPrefixStart(s string) int {
start := -1
indices := []int{
strings.LastIndex(s, "{"),
strings.LastIndex(s, "["),
strings.LastIndex(s, "```"),
}
for _, idx := range indices {
if idx > start {
start = idx
if xmlIdx := findPartialXMLToolTagStart(s); xmlIdx >= 0 {
if insideCodeFenceWithState(state, s[:xmlIdx]) {
return s, ""
}
if xmlIdx > 0 {
return s[:xmlIdx], s[xmlIdx:]
}
return "", s
}
// Also check for partial XML tool tag at end of string.
if xmlIdx := findPartialXMLToolTagStart(s); xmlIdx >= 0 && xmlIdx > start {
start = xmlIdx
}
return start
return s, ""
}
func findToolSegmentStart(s string) int {
func findToolSegmentStart(state *toolStreamSieveState, s string) int {
if s == "" {
return -1
}
lower := strings.ToLower(s)
keywords := []string{"tool_calls", "\"function\"", "function.name:", "\"tool_use\""}
bestKeyIdx := -1
for _, kw := range keywords {
idx := strings.Index(lower, kw)
if idx >= 0 && (bestKeyIdx < 0 || idx < bestKeyIdx) {
bestKeyIdx = idx
offset := 0
for {
bestKeyIdx := -1
matchedTag := ""
for _, tag := range xmlToolTagsToDetect {
idx := strings.Index(lower[offset:], tag)
if idx >= 0 {
idx += offset
if bestKeyIdx < 0 || idx < bestKeyIdx {
bestKeyIdx = idx
matchedTag = tag
}
}
}
}
if fnKeyIdx := findQuotedFunctionCallKeyStart(s); fnKeyIdx >= 0 && (bestKeyIdx < 0 || fnKeyIdx < bestKeyIdx) {
bestKeyIdx = fnKeyIdx
}
// Also detect XML tool call tags.
for _, tag := range xmlToolTagsToDetect {
idx := strings.Index(lower, tag)
if idx >= 0 && (bestKeyIdx < 0 || idx < bestKeyIdx) {
bestKeyIdx = idx
if bestKeyIdx < 0 {
return -1
}
}
if bestKeyIdx < 0 {
return -1
}
// For XML tags, the '<' is itself the segment start.
if bestKeyIdx < len(s) && s[bestKeyIdx] == '<' {
if fenceStart, ok := openFenceStartBefore(s, bestKeyIdx); ok {
return fenceStart
if !insideCodeFenceWithState(state, s[:bestKeyIdx]) {
return bestKeyIdx
}
return bestKeyIdx
offset = bestKeyIdx + len(matchedTag)
}
start := strings.LastIndex(s[:bestKeyIdx], "{")
if start < 0 {
start = bestKeyIdx
}
// If the keyword matched inside an XML tag (e.g. "tool_calls" in "<tool_calls>"),
// back up past the '<' to capture the full tag.
if start > 0 && s[start-1] == '<' {
start--
}
if fenceStart, ok := openFenceStartBefore(s, start); ok {
return fenceStart
}
return start
}
func consumeToolCapture(state *toolStreamSieveState, toolNames []string) (prefix string, calls []util.ParsedToolCall, suffix string, ready bool) {
func consumeToolCapture(state *toolStreamSieveState, toolNames []string) (prefix string, calls []toolcall.ParsedToolCall, suffix string, ready bool) {
captured := state.capture.String()
if captured == "" {
return "", nil, "", false
}
// Try XML tool call extraction first.
// XML tool call extraction only.
if xmlPrefix, xmlCalls, xmlSuffix, xmlReady := consumeXMLToolCapture(captured, toolNames); xmlReady {
return xmlPrefix, xmlCalls, xmlSuffix, true
}
@@ -240,45 +193,5 @@ func consumeToolCapture(state *toolStreamSieveState, toolNames []string) (prefix
if hasOpenXMLToolTag(captured) {
return "", nil, "", false
}
lower := strings.ToLower(captured)
keyIdx := -1
keywords := []string{"tool_calls", "\"function\"", "function.name:", "\"tool_use\""}
for _, kw := range keywords {
idx := strings.Index(lower, kw)
if idx >= 0 && (keyIdx < 0 || idx < keyIdx) {
keyIdx = idx
}
}
if fnKeyIdx := findQuotedFunctionCallKeyStart(captured); fnKeyIdx >= 0 && (keyIdx < 0 || fnKeyIdx < keyIdx) {
keyIdx = fnKeyIdx
}
if keyIdx < 0 {
return "", nil, "", false
}
start := strings.LastIndex(captured[:keyIdx], "{")
if start < 0 {
start = keyIdx
}
obj, end, ok := extractJSONObjectFrom(captured, start)
if !ok {
return "", nil, "", false
}
prefixPart := captured[:start]
suffixPart := captured[end:]
parsed := util.ParseStandaloneToolCallsDetailed(obj, toolNames)
if len(parsed.Calls) == 0 {
if parsed.SawToolCallSyntax && parsed.RejectedByPolicy {
// Parsed as tool-call payload but rejected by schema/policy:
// consume it to avoid leaking raw tool_calls JSON to user content.
return prefixPart, nil, suffixPart, true
}
// If it has obvious keywords but failed to parse even after loose repair,
// we still might want to intercept it if it looks like an attempt at tool call.
// For now, keep the original logic but rely on loose JSON repair.
return captured, nil, "", true
}
prefixPart, suffixPart = trimWrappingJSONFence(prefixPart, suffixPart)
return prefixPart, parsed.Calls, suffixPart, true
return "", nil, "", false
}

View File

@@ -1,100 +0,0 @@
package openai
import "strings"
func findQuotedFunctionCallKeyStart(s string) int {
lower := strings.ToLower(s)
quotedIdx := findFunctionCallKeyStart(lower, `"functioncall"`)
bareIdx := findFunctionCallKeyStart(lower, "functioncall")
// Prefer the quoted JSON key whenever we have a structural match.
// Bare-key detection is only for loose payloads where the quoted form
// is absent.
if quotedIdx >= 0 {
return quotedIdx
}
return bareIdx
}
func findFunctionCallKeyStart(lower, key string) int {
for from := 0; from < len(lower); {
rel := strings.Index(lower[from:], key)
if rel < 0 {
return -1
}
idx := from + rel
if isInsideJSONString(lower, idx) {
from = idx + 1
continue
}
if !hasJSONObjectContextPrefix(lower[:idx]) {
from = idx + 1
continue
}
if !hasJSONKeyBoundary(lower, idx, len(key)) {
from = idx + 1
continue
}
j := idx + len(key)
for j < len(lower) && (lower[j] == ' ' || lower[j] == '\t' || lower[j] == '\r' || lower[j] == '\n') {
j++
}
if j < len(lower) && lower[j] == ':' {
k := j + 1
for k < len(lower) && (lower[k] == ' ' || lower[k] == '\t' || lower[k] == '\r' || lower[k] == '\n') {
k++
}
if k < len(lower) && lower[k] != '{' {
from = idx + 1
continue
}
return idx
}
from = idx + 1
}
return -1
}
func isInsideJSONString(s string, idx int) bool {
inString := false
escaped := false
for i := 0; i < idx; i++ {
c := s[i]
if escaped {
escaped = false
continue
}
if c == '\\' && inString {
escaped = true
continue
}
if c == '"' {
inString = !inString
}
}
return inString
}
func hasJSONObjectContextPrefix(prefix string) bool {
return strings.LastIndex(prefix, "{") >= 0
}
func hasJSONKeyBoundary(s string, idx, keyLen int) bool {
if idx > 0 {
prev := s[idx-1]
if isLowerAlphaNumeric(prev) {
return false
}
}
if end := idx + keyLen; end < len(s) {
next := s[end]
if isLowerAlphaNumeric(next) {
return false
}
}
return true
}
func isLowerAlphaNumeric(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b == '_'
}

View File

@@ -1,23 +0,0 @@
package openai
import "testing"
func TestFindQuotedFunctionCallKeyStart_PrefersEarlierBareKey(t *testing.T) {
input := `{functionCall:{"name":"a","arguments":"{}"},"message":"literal text: \"functionCall\": not a key"}`
got := findQuotedFunctionCallKeyStart(input)
want := 1
if got != want {
t.Fatalf("findQuotedFunctionCallKeyStart() = %d, want %d", got, want)
}
}
func TestFindQuotedFunctionCallKeyStart_PrefersEarlierQuotedKey(t *testing.T) {
input := `{"functionCall":{"name":"a","arguments":"{}"},"note":"functionCall appears in prose"}`
got := findQuotedFunctionCallKeyStart(input)
want := 1
if got != want {
t.Fatalf("findQuotedFunctionCallKeyStart() = %d, want %d", got, want)
}
}

View File

@@ -2,48 +2,6 @@ package openai
import "strings"
func extractJSONObjectFrom(text string, start int) (string, int, bool) {
if start < 0 || start >= len(text) || text[start] != '{' {
return "", 0, false
}
depth := 0
quote := byte(0)
escaped := false
for i := start; i < len(text); i++ {
ch := text[i]
if quote != 0 {
if escaped {
escaped = false
continue
}
if ch == '\\' {
escaped = true
continue
}
if ch == quote {
quote = 0
}
continue
}
if ch == '"' || ch == '\'' {
quote = ch
continue
}
if ch == '{' {
depth++
continue
}
if ch == '}' {
depth--
if depth == 0 {
end := i + 1
return text[start:end], end, true
}
}
}
return "", 0, false
}
func trimWrappingJSONFence(prefix, suffix string) (string, string) {
trimmedPrefix := strings.TrimRight(prefix, " \t\r\n")
fenceIdx := strings.LastIndex(trimmedPrefix, "```")
@@ -67,18 +25,3 @@ func trimWrappingJSONFence(prefix, suffix string) (string, string) {
consumedLeading := len(suffix) - len(trimmedSuffix)
return trimmedPrefix[:fenceIdx], suffix[consumedLeading+3:]
}
func openFenceStartBefore(s string, pos int) (int, bool) {
if pos <= 0 || pos > len(s) {
return -1, false
}
segment := s[:pos]
lastFence := strings.LastIndex(segment, "```")
if lastFence < 0 {
return -1, false
}
if strings.Count(segment, "```")%2 == 1 {
return lastFence, true
}
return -1, false
}

View File

@@ -1,30 +1,32 @@
package openai
import (
"ds2api/internal/toolcall"
"strings"
"ds2api/internal/util"
)
type toolStreamSieveState struct {
pending strings.Builder
capture strings.Builder
capturing bool
recentTextTail string
pendingToolRaw string
pendingToolCalls []util.ParsedToolCall
disableDeltas bool
toolNameSent bool
toolName string
toolArgsStart int
toolArgsSent int
toolArgsString bool
toolArgsDone bool
pending strings.Builder
capture strings.Builder
capturing bool
codeFenceStack []int
codeFencePendingTicks int
codeFenceLineStart bool
recentTextTail string
pendingToolRaw string
pendingToolCalls []toolcall.ParsedToolCall
disableDeltas bool
toolNameSent bool
toolName string
toolArgsStart int
toolArgsSent int
toolArgsString bool
toolArgsDone bool
}
type toolStreamEvent struct {
Content string
ToolCalls []util.ParsedToolCall
ToolCalls []toolcall.ParsedToolCall
ToolCallDeltas []toolCallDelta
}
@@ -48,9 +50,10 @@ func (s *toolStreamSieveState) resetIncrementalToolState() {
}
func (s *toolStreamSieveState) noteText(content string) {
if strings.TrimSpace(content) == "" {
if !hasMeaningfulText(content) {
return
}
updateCodeFenceState(s, content)
s.recentTextTail = appendTail(s.recentTextTail, content, toolSieveContextTailLimit)
}
@@ -64,3 +67,107 @@ func appendTail(prev, next string, max int) string {
}
return combined[len(combined)-max:]
}
func hasMeaningfulText(text string) bool {
return strings.TrimSpace(text) != ""
}
func insideCodeFenceWithState(state *toolStreamSieveState, text string) bool {
if state == nil {
return insideCodeFence(text)
}
simulated := simulateCodeFenceState(
state.codeFenceStack,
state.codeFencePendingTicks,
state.codeFenceLineStart,
text,
)
return len(simulated.stack) > 0
}
func insideCodeFence(text string) bool {
if text == "" {
return false
}
return len(simulateCodeFenceState(nil, 0, true, text).stack) > 0
}
func updateCodeFenceState(state *toolStreamSieveState, text string) {
if state == nil || !hasMeaningfulText(text) {
return
}
next := simulateCodeFenceState(
state.codeFenceStack,
state.codeFencePendingTicks,
state.codeFenceLineStart,
text,
)
state.codeFenceStack = next.stack
state.codeFencePendingTicks = next.pendingTicks
state.codeFenceLineStart = next.lineStart
}
type codeFenceSimulation struct {
stack []int
pendingTicks int
lineStart bool
}
func simulateCodeFenceState(stack []int, pendingTicks int, lineStart bool, text string) codeFenceSimulation {
chunk := text
nextStack := append([]int(nil), stack...)
ticks := pendingTicks
atLineStart := lineStart
flushTicks := func() {
if ticks > 0 {
if atLineStart && ticks >= 3 {
applyFenceMarker(&nextStack, ticks)
}
atLineStart = false
ticks = 0
}
}
for i := 0; i < len(chunk); i++ {
ch := chunk[i]
if ch == '`' {
ticks++
continue
}
flushTicks()
switch ch {
case '\n', '\r':
atLineStart = true
case ' ', '\t':
if atLineStart {
continue
}
atLineStart = false
default:
atLineStart = false
}
}
return codeFenceSimulation{
stack: nextStack,
pendingTicks: ticks,
lineStart: atLineStart,
}
}
func applyFenceMarker(stack *[]int, ticks int) {
if stack == nil || ticks <= 0 {
return
}
if len(*stack) == 0 {
*stack = append(*stack, ticks)
return
}
top := (*stack)[len(*stack)-1]
if ticks >= top {
*stack = (*stack)[:len(*stack)-1]
return
}
*stack = append(*stack, ticks)
}

View File

@@ -1,14 +1,14 @@
package openai
import (
"ds2api/internal/toolcall"
"regexp"
"strings"
"ds2api/internal/util"
)
// --- XML tool call support for the streaming sieve ---
//nolint:unused // kept as explicit tag inventory for future XML sieve refinements.
var xmlToolCallClosingTags = []string{"</tool_calls>", "</tool_call>", "</invoke>", "</function_call>", "</function_calls>", "</tool_use>",
// Agent-style XML tags (Roo Code, Cline, etc.)
"</attempt_completion>", "</ask_followup_question>", "</new_task>", "</result>"}
@@ -26,14 +26,16 @@ var xmlToolCallTagPairs = []struct{ open, close string }{
{"<invoke", "</invoke>"},
{"<tool_use", "</tool_use>"},
// Agent-style: these are XML "tool call" patterns from coding agents.
// They get captured → parsed. If parsing fails, the block is consumed
// (swallowed) to prevent raw XML from leaking to the client.
// They get captured → parsed. If parsing fails, the raw XML is preserved
// so the caller can still see the original text.
{"<attempt_completion", "</attempt_completion>"},
{"<ask_followup_question", "</ask_followup_question>"},
{"<new_task", "</new_task>"},
}
// xmlToolCallBlockPattern matches a complete XML tool call block (wrapper or standalone).
//
//nolint:unused // reserved for future fast-path XML block detection.
var xmlToolCallBlockPattern = regexp.MustCompile(`(?is)(<tool_calls>\s*(?:.*?)\s*</tool_calls>|<tool_call>\s*(?:.*?)\s*</tool_call>|<invoke\b[^>]*>(?:.*?)</invoke>|<function_calls?\b[^>]*>(?:.*?)</function_calls?>|<tool_use>(?:.*?)</tool_use>|<attempt_completion>(?:.*?)</attempt_completion>|<ask_followup_question>(?:.*?)</ask_followup_question>|<new_task>(?:.*?)</new_task>)`)
// xmlToolTagsToDetect is the set of XML tag prefixes used by findToolSegmentStart.
@@ -43,7 +45,7 @@ var xmlToolTagsToDetect = []string{"<tool_calls>", "<tool_calls\n", "<tool_call>
"<attempt_completion>", "<ask_followup_question>", "<new_task>"}
// consumeXMLToolCapture tries to extract complete XML tool call blocks from captured text.
func consumeXMLToolCapture(captured string, toolNames []string) (prefix string, calls []util.ParsedToolCall, suffix string, ready bool) {
func consumeXMLToolCapture(captured string, toolNames []string) (prefix string, calls []toolcall.ParsedToolCall, suffix string, ready bool) {
lower := strings.ToLower(captured)
// Find the FIRST matching open/close pair, preferring wrapper tags.
// Tag pairs are ordered longest-first (e.g. <tool_calls before <tool_call)
@@ -66,13 +68,13 @@ func consumeXMLToolCapture(captured string, toolNames []string) (prefix string,
xmlBlock := captured[openIdx:closeEnd]
prefixPart := captured[:openIdx]
suffixPart := captured[closeEnd:]
parsed := util.ParseToolCalls(xmlBlock, toolNames)
parsed := toolcall.ParseToolCalls(xmlBlock, toolNames)
if len(parsed) > 0 {
prefixPart, suffixPart = trimWrappingJSONFence(prefixPart, suffixPart)
return prefixPart, parsed, suffixPart, true
}
// Looks like XML tool syntax but failed to parse — consume it to avoid leak.
return prefixPart, nil, suffixPart, true
// If this block failed to become a tool call, pass it through as text.
return prefixPart + xmlBlock, nil, suffixPart, true
}
return "", nil, "", false
}
@@ -116,32 +118,3 @@ func findPartialXMLToolTagStart(s string) int {
}
return -1
}
// looksLikeXMLToolTagFragment returns true if s looks like a fragment from a
// split XML tool call tag — for example "tool_calls>" or "/tool_call>\n".
// These fragments arise when '<' was consumed separately and the tail remains.
func looksLikeXMLToolTagFragment(s string) bool {
trimmed := strings.TrimSpace(s)
if trimmed == "" {
return false
}
lower := strings.ToLower(trimmed)
// Check for closing tag tails like "tool_calls>" or "/tool_calls>"
fragments := []string{
"tool_calls>", "tool_call>", "/tool_calls>", "/tool_call>",
"function_calls>", "function_call>", "/function_calls>", "/function_call>",
"invoke>", "/invoke>", "tool_use>", "/tool_use>",
"tool_name>", "/tool_name>", "parameters>", "/parameters>",
// Agent-style tag fragments
"attempt_completion>", "/attempt_completion>",
"ask_followup_question>", "/ask_followup_question>",
"new_task>", "/new_task>",
"result>", "/result>",
}
for _, f := range fragments {
if strings.Contains(lower, f) {
return true
}
}
return false
}

View File

@@ -78,6 +78,148 @@ func TestProcessToolSieveXMLWithLeadingText(t *testing.T) {
}
}
func TestProcessToolSievePassesThroughNonToolXMLBlock(t *testing.T) {
var state toolStreamSieveState
chunk := `<tool_call><title>示例 XML</title><body>plain text xml payload</body></tool_call>`
events := processToolSieveChunk(&state, chunk, []string{"read_file"})
events = append(events, flushToolSieve(&state, []string{"read_file"})...)
var textContent strings.Builder
toolCalls := 0
for _, evt := range events {
textContent.WriteString(evt.Content)
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 0 {
t.Fatalf("expected no tool calls for plain XML payload, got %d events=%#v", toolCalls, events)
}
if textContent.String() != chunk {
t.Fatalf("expected XML payload to pass through unchanged, got %q", textContent.String())
}
}
func TestProcessToolSieveNonToolXMLKeepsSuffixForToolParsing(t *testing.T) {
var state toolStreamSieveState
chunk := `<tool_call><title>plain xml</title></tool_call><invoke name="read_file"><parameters>{"path":"README.MD"}</parameters></invoke>`
events := processToolSieveChunk(&state, chunk, []string{"read_file"})
events = append(events, flushToolSieve(&state, []string{"read_file"})...)
var textContent strings.Builder
toolCalls := 0
for _, evt := range events {
textContent.WriteString(evt.Content)
toolCalls += len(evt.ToolCalls)
}
if !strings.Contains(textContent.String(), `<tool_call><title>plain xml</title></tool_call>`) {
t.Fatalf("expected leading non-tool XML to be preserved, got %q", textContent.String())
}
if strings.Contains(textContent.String(), `<invoke name="read_file">`) {
t.Fatalf("expected invoke tool XML to be intercepted, got %q", textContent.String())
}
if toolCalls != 1 {
t.Fatalf("expected exactly one parsed tool call from suffix, got %d events=%#v", toolCalls, events)
}
}
func TestProcessToolSievePassesThroughMalformedExecutableXMLBlock(t *testing.T) {
var state toolStreamSieveState
chunk := `<tool_call><parameters>{"path":"README.md"}</parameters></tool_call>`
events := processToolSieveChunk(&state, chunk, []string{"read_file"})
events = append(events, flushToolSieve(&state, []string{"read_file"})...)
var textContent strings.Builder
toolCalls := 0
for _, evt := range events {
textContent.WriteString(evt.Content)
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 0 {
t.Fatalf("expected malformed executable-looking XML to stay text, got %d events=%#v", toolCalls, events)
}
if textContent.String() != chunk {
t.Fatalf("expected malformed executable-looking XML to pass through unchanged, got %q", textContent.String())
}
}
func TestProcessToolSievePassesThroughFencedXMLToolCallExamples(t *testing.T) {
var state toolStreamSieveState
input := strings.Join([]string{
"Before first example.\n```",
"xml\n<tool_call><tool_name>read_file</tool_name><parameters>{\"path\":\"README.md\"}</parameters></tool_call>\n```\n",
"Between examples.\n```xml\n",
"<tool_call><tool_name>search</tool_name><parameters>{\"q\":\"golang\"}</parameters></tool_call>\n",
"```\nAfter examples.",
}, "")
chunks := []string{
"Before first example.\n```",
"xml\n<tool_call><tool_name>read_file</tool_name><parameters>{\"path\":\"README.md\"}</parameters></tool_call>\n```\n",
"Between examples.\n```xml\n",
"<tool_call><tool_name>search</tool_name><parameters>{\"q\":\"golang\"}</parameters></tool_call>\n",
"```\nAfter examples.",
}
var events []toolStreamEvent
for _, c := range chunks {
events = append(events, processToolSieveChunk(&state, c, []string{"read_file", "search"})...)
}
events = append(events, flushToolSieve(&state, []string{"read_file", "search"})...)
var textContent strings.Builder
toolCalls := 0
for _, evt := range events {
if evt.Content != "" {
textContent.WriteString(evt.Content)
}
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 0 {
t.Fatalf("expected fenced XML examples to stay text, got %d tool calls events=%#v", toolCalls, events)
}
if textContent.String() != input {
t.Fatalf("expected fenced XML examples to pass through unchanged, got %q", textContent.String())
}
}
func TestProcessToolSieveKeepsPartialXMLTagInsideFencedExample(t *testing.T) {
var state toolStreamSieveState
input := strings.Join([]string{
"Example:\n```xml\n<tool_ca",
"ll><tool_name>read_file</tool_name><parameters>{\"path\":\"README.md\"}</parameters></tool_call>\n```\n",
"Done.",
}, "")
chunks := []string{
"Example:\n```xml\n<tool_ca",
"ll><tool_name>read_file</tool_name><parameters>{\"path\":\"README.md\"}</parameters></tool_call>\n```\n",
"Done.",
}
var events []toolStreamEvent
for _, c := range chunks {
events = append(events, processToolSieveChunk(&state, c, []string{"read_file"})...)
}
events = append(events, flushToolSieve(&state, []string{"read_file"})...)
var textContent strings.Builder
toolCalls := 0
for _, evt := range events {
if evt.Content != "" {
textContent.WriteString(evt.Content)
}
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 0 {
t.Fatalf("expected partial fenced XML to stay text, got %d tool calls events=%#v", toolCalls, events)
}
if textContent.String() != input {
t.Fatalf("expected partial fenced XML to pass through unchanged, got %q", textContent.String())
}
}
func TestProcessToolSievePartialXMLTagHeldBack(t *testing.T) {
var state toolStreamSieveState
// Chunk ends with a partial XML tool tag.
@@ -104,15 +246,16 @@ func TestFindToolSegmentStartDetectsXMLToolCalls(t *testing.T) {
want int
}{
{"tool_calls_tag", "some text <tool_calls>\n", 10},
{"gemini_function_call_json", `some text {"functionCall":{"name":"search","args":{"q":"latest"}}}`, 10},
{"tool_call_tag", "prefix <tool_call>\n", 7},
{"invoke_tag", "text <invoke name=\"foo\">body</invoke>", 5},
{"xml_inside_code_fence", "```xml\n<tool_call><tool_name>read_file</tool_name></tool_call>\n```", -1},
{"function_call_tag", "<function_call name=\"foo\">body</function_call>", 0},
{"no_xml", "just plain text", -1},
{"gemini_json_no_detect", `some text {"functionCall":{"name":"search"}}`, -1},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := findToolSegmentStart(tc.input)
got := findToolSegmentStart(nil, tc.input)
if got != tc.want {
t.Fatalf("findToolSegmentStart(%q) = %d, want %d", tc.input, got, tc.want)
}
@@ -120,81 +263,6 @@ func TestFindToolSegmentStartDetectsXMLToolCalls(t *testing.T) {
}
}
func TestFindToolSegmentStartIgnoresFunctionCallProse(t *testing.T) {
input := "Please explain the functionCall API field and how clients should parse it."
if got := findToolSegmentStart(input); got != -1 {
t.Fatalf("expected no tool segment start for prose, got %d", got)
}
}
func TestFindToolSegmentStartDetectsQuotedFunctionCallKey(t *testing.T) {
input := `prefix {"functionCall": {"name":"search_web","args":{"query":"x"}}}`
want := strings.Index(input, "{")
if got := findToolSegmentStart(input); got != want {
t.Fatalf("expected JSON object start %d, got %d", want, got)
}
}
func TestFindToolSegmentStartDetectsLooseFunctionCallKey(t *testing.T) {
input := `prefix {functionCall: {"name":"search_web","args":{"query":"x"}}}`
want := strings.Index(input, "{")
if got := findToolSegmentStart(input); got != want {
t.Fatalf("expected JSON object start %d, got %d", want, got)
}
}
func TestFindToolSegmentStartPrefersQuotedFunctionCallOverEarlierBareProse(t *testing.T) {
input := `prefix {note} functionCall: docs hint {"functionCall":{"name":"search_web","args":{"query":"x"}}}`
want := strings.Index(input, `{"functionCall"`)
if got := findToolSegmentStart(input); got != want {
t.Fatalf("expected quoted functionCall JSON start %d, got %d", want, got)
}
}
func TestFindToolSegmentStartIgnoresLooseFunctionCallProse(t *testing.T) {
input := "Please explain why functionCall: is used in documentation examples."
if got := findToolSegmentStart(input); got != -1 {
t.Fatalf("expected no tool segment start for prose, got %d", got)
}
}
func TestProcessToolSieveDoesNotBufferFunctionCallProse(t *testing.T) {
var state toolStreamSieveState
chunk := "Please explain the functionCall API field and keep streaming this sentence."
events := processToolSieveChunk(&state, chunk, []string{"search_web"})
var text string
for _, evt := range events {
text += evt.Content
if len(evt.ToolCalls) > 0 {
t.Fatalf("expected no tool calls for prose, got %#v", evt.ToolCalls)
}
}
if text != chunk {
t.Fatalf("expected prose to pass through immediately, got %q", text)
}
}
func TestProcessToolSieveDetectsGeminiFunctionCallPayload(t *testing.T) {
var state toolStreamSieveState
events := processToolSieveChunk(&state, `{"functionCall":{"name":"search_web","args":{"query":"latest"}}}`, []string{"search_web"})
events = append(events, flushToolSieve(&state, []string{"search_web"})...)
var textContent string
var toolCalls int
for _, evt := range events {
if evt.Content != "" {
textContent += evt.Content
}
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 1 {
t.Fatalf("expected one tool call from functionCall payload, got events=%#v", events)
}
if strings.Contains(strings.ToLower(textContent), "functioncall") {
t.Fatalf("functionCall json leaked into text content: %q", textContent)
}
}
func TestFindPartialXMLToolTagStart(t *testing.T) {
cases := []struct {
name string
@@ -301,8 +369,8 @@ func TestProcessToolSieveTokenByTokenXMLNoLeak(t *testing.T) {
}
}
// Test that flushToolSieve on incomplete XML does NOT leak the raw XML content.
func TestFlushToolSieveIncompleteXMLDoesNotLeak(t *testing.T) {
// Test that flushToolSieve on incomplete XML falls back to raw text.
func TestFlushToolSieveIncompleteXMLFallsBackToText(t *testing.T) {
var state toolStreamSieveState
// XML block starts but stream ends before completion.
chunks := []string{
@@ -324,8 +392,8 @@ func TestFlushToolSieveIncompleteXMLDoesNotLeak(t *testing.T) {
}
}
if strings.Contains(textContent, "<tool_call") {
t.Fatalf("incomplete XML leaked on flush: %q", textContent)
if textContent != strings.Join(chunks, "") {
t.Fatalf("expected incomplete XML to fall back to raw text, got %q", textContent)
}
}
@@ -362,10 +430,10 @@ func TestOpeningXMLTagNotLeakedAsContent(t *testing.T) {
}
}
func TestProcessToolSieveInterceptsAttemptCompletionLeak(t *testing.T) {
func TestProcessToolSieveFallsBackToRawAttemptCompletion(t *testing.T) {
var state toolStreamSieveState
// Simulate an agent outputting attempt_completion XML tag
// which shouldn't leak to text output, even if it fails to parse as a valid tool.
// Simulate an agent outputting attempt_completion XML tag.
// If it does not parse as a tool call, it should fall back to raw text.
chunks := []string{
"Done with task.\n",
"<attempt_completion>\n",
@@ -389,7 +457,7 @@ func TestProcessToolSieveInterceptsAttemptCompletionLeak(t *testing.T) {
t.Fatalf("expected leading text to be emitted, got %q", textContent)
}
if strings.Contains(textContent, "<attempt_completion>") || strings.Contains(textContent, "result>") {
t.Fatalf("agent XML tag content leaked to text: %q", textContent)
if textContent != strings.Join(chunks, "") {
t.Fatalf("expected agent XML to fall back to raw text, got %q", textContent)
}
}

View File

@@ -0,0 +1,15 @@
package openai
import "net/http"
func writeUpstreamEmptyOutputError(w http.ResponseWriter, text string, contentFilter bool) bool {
if text != "" {
return false
}
if contentFilter {
writeOpenAIErrorWithCode(w, http.StatusBadRequest, "Upstream content filtered the response and returned no output.", "content_filter")
return true
}
writeOpenAIErrorWithCode(w, http.StatusTooManyRequests, "Upstream model returned empty output.", "upstream_empty_output")
return true
}

View File

@@ -52,6 +52,10 @@ func (h *Handler) handleVercelStreamPrepare(w http.ResponseWriter, r *http.Reque
writeOpenAIError(w, http.StatusBadRequest, "invalid json")
return
}
if err := h.preprocessInlineFileInputs(r.Context(), a, req); err != nil {
writeOpenAIInlineFileError(w, err)
return
}
if !util.ToBool(req["stream"]) {
writeOpenAIError(w, http.StatusBadRequest, "stream must be true")
return
@@ -99,10 +103,13 @@ func (h *Handler) handleVercelStreamPrepare(w http.ResponseWriter, r *http.Reque
"final_prompt": stdReq.FinalPrompt,
"thinking_enabled": stdReq.Thinking,
"search_enabled": stdReq.Search,
"tool_names": stdReq.ToolNames,
"deepseek_token": a.DeepSeekToken,
"pow_header": powHeader,
"payload": payload,
"compat": map[string]any{
"strip_reference_markers": h.compatStripReferenceMarkers(),
},
"tool_names": stdReq.ToolNames,
"deepseek_token": a.DeepSeekToken,
"pow_header": powHeader,
"payload": payload,
})
}

View File

@@ -32,6 +32,8 @@ type ConfigStore interface {
RuntimeAccountMaxQueue(defaultSize int) int
RuntimeGlobalMaxInflight(defaultSize int) int
RuntimeTokenRefreshIntervalHours() int
AutoDeleteMode() string
CompatStripReferenceMarkers() bool
AutoDeleteSessions() bool
}
@@ -41,6 +43,10 @@ type PoolController interface {
ApplyRuntimeLimits(maxInflightPerAccount, maxQueueSize, globalMaxInflight int)
}
type OpenAIChatCaller interface {
ChatCompletions(w http.ResponseWriter, r *http.Request)
}
type DeepSeekCaller interface {
Login(ctx context.Context, acc config.Account) (string, error)
CreateSession(ctx context.Context, a *auth.RequestAuth, maxAttempts int) (string, error)

View File

@@ -5,9 +5,10 @@ import (
)
type Handler struct {
Store ConfigStore
Pool PoolController
DS DeepSeekCaller
Store ConfigStore
Pool PoolController
DS DeepSeekCaller
OpenAI OpenAIChatCaller
}
func RegisterRoutes(r chi.Router, h *Handler) {
@@ -25,15 +26,24 @@ func RegisterRoutes(r chi.Router, h *Handler) {
pr.Get("/config/export", h.configExport)
pr.Post("/keys", h.addKey)
pr.Delete("/keys/{key}", h.deleteKey)
pr.Get("/proxies", h.listProxies)
pr.Post("/proxies", h.addProxy)
pr.Put("/proxies/{proxyID}", h.updateProxy)
pr.Delete("/proxies/{proxyID}", h.deleteProxy)
pr.Post("/proxies/test", h.testProxy)
pr.Get("/accounts", h.listAccounts)
pr.Post("/accounts", h.addAccount)
pr.Delete("/accounts/{identifier}", h.deleteAccount)
pr.Put("/accounts/{identifier}/proxy", h.updateAccountProxy)
pr.Get("/queue/status", h.queueStatus)
pr.Post("/accounts/test", h.testSingleAccount)
pr.Post("/accounts/test-all", h.testAllAccounts)
pr.Post("/accounts/sessions/delete-all", h.deleteAllSessions)
pr.Post("/import", h.batchImport)
pr.Post("/test", h.testAPI)
pr.Post("/dev/raw-samples/capture", h.captureRawSample)
pr.Get("/dev/raw-samples/query", h.queryRawSampleCaptures)
pr.Post("/dev/raw-samples/save", h.saveRawSampleFromCaptures)
pr.Post("/vercel/sync", h.syncVercel)
pr.Get("/vercel/status", h.vercelStatus)
pr.Post("/vercel/status", h.vercelStatus)

Some files were not shown because too many files have changed in this diff Show More