Compare commits

...

62 Commits

Author SHA1 Message Date
CJACK.
4438d03c5c Merge pull request #377 from CJackHwang/codex/run-all-tests-and-fix-failures
Fix failing current-input token accounting test
2026-04-30 02:41:08 +08:00
CJACK.
95b7665643 Merge branch 'dev' into codex/run-all-tests-and-fix-failures 2026-04-30 02:39:18 +08:00
CJACK.
9896b1fc33 Merge pull request #378 from NgoQuocViet2001/ai/openai-root-route-aliases
feat(openai): add root route aliases
2026-04-30 02:33:38 +08:00
CJACK.
966f21211d Fix nil-session guard in chat history test 2026-04-30 02:31:06 +08:00
NgoQuocViet2001
7dc3af40b2 feat(openai): add root route aliases 2026-04-30 01:24:53 +07:00
CJACK.
2f6b5ffda0 Fix current-input token text test expectation 2026-04-30 02:22:17 +08:00
CJACK.
85e256ad4d Merge pull request #375 from CJackHwang/codex/investigate-data-loss-issue-in-pr-369
sse/parser: treat object-shaped `v` as visible content, preserve INCOMPLETE across omitted status; add tests and samples
2026-04-30 02:14:26 +08:00
CJACK.
7c3ff6ee7e Merge pull request #374 from shern-point/feat/full-context-file-token-accounting
Feat/full context file token accounting
2026-04-30 02:12:55 +08:00
CJACK.
63e62fd1b0 Merge pull request #372 from shern-point/feat/accurate-context-token-length
Feat/accurate context token length
2026-04-30 02:11:32 +08:00
CJACK.
483d7af3d2 Merge pull request #373 from NgoQuocViet2001/ai/ds2api-small-regression-fix
fix(openai): return 400 for inline file limit
2026-04-30 02:08:22 +08:00
CJACK.
0f89823526 chore(sse): bump client version and refresh longtext stream fixtures 2026-04-30 02:05:45 +08:00
shern-point
6a778e0d35 feat: include inline-uploaded file tokens in context token accounting
Track byte sizes of inline-uploaded files during PreprocessInlineFileInputs and convert them to conservative token estimates (bytes/3). RefFileTokens is threaded through StandardRequest into all OpenAI chat/responses usage builders so returned prompt_tokens/input_tokens reflect the full upstream context cost including attached files.
2026-04-30 01:42:51 +08:00
NgoQuocViet2001
9035c350a7 fix(openai): return 400 for inline file limit 2026-04-30 00:35:59 +07:00
shern-point
ba80052a26 fix: count uploaded file content in context token accounting
PromptTokenText now reflects the actual downstream context cost: the uploaded IGNORE.txt file content plus the neutral live prompt, instead of only the pre-split prompt text.
2026-04-30 01:12:35 +08:00
shern-point
78fdd63470 feat: add full-context token regression coverage and docs
Lock in the current_input_file regression with API-level tests and document that returned context token counts now track full prompt semantics with conservative sizing.
2026-04-30 00:46:06 +08:00
shern-point
4b4f097006 feat: use model-aware prompt counting in Gemini paths
Preserve Gemini prompt token text during normalization and remove the hardcoded DeepSeek model from native Gemini usage helpers.
2026-04-30 00:46:05 +08:00
shern-point
d3018c281b feat: use tokenizer-based counting in Claude token paths
Unify Claude count_tokens, legacy stream accounting, and legacy render usage with preserved prompt text so Claude stops falling back to lossy message formatting.
2026-04-30 00:46:04 +08:00
shern-point
415a2359ad feat: route OpenAI responses usage through preserved prompt text
Use the stored full-context prompt text for responses accounting so neutral placeholder prompts do not underreport returned input token counts.
2026-04-30 00:45:31 +08:00
shern-point
f702d45a24 feat: route OpenAI chat usage through preserved prompt text
Use the stored full-context prompt text for chat non-stream, stream, and retry accounting so current_input_file no longer shrinks returned prompt token counts.
2026-04-30 00:45:30 +08:00
shern-point
90817cb9e2 feat: apply tokenizer-based counting in OpenAI usage builders
Move OpenAI chat and responses usage accounting onto the shared tokenizer-aware counters so prompt and output usage stay model-aware and conservatively sized.
2026-04-30 00:45:29 +08:00
shern-point
b96f736bd2 feat: preserve full prompt text across current_input_file rewrites
Keep token accounting tied to the original prompt even after the live prompt is replaced with a neutral placeholder and hidden context file.
2026-04-30 00:45:01 +08:00
shern-point
8ab028c52a feat: seed PromptTokenText during request normalization
Capture the fully built prompt at normalization time for OpenAI and Gemini-compatible requests so usage paths can reuse the original context text.
2026-04-30 00:44:59 +08:00
shern-point
78366afec5 feat: add PromptTokenText to StandardRequest
Track a dedicated prompt string for token accounting so later prompt rewrites can keep returning full-context counts.
2026-04-30 00:44:57 +08:00
shern-point
bd41c8a90c feat: add tokenizer-based token counting utilities
Use go-tiktoken with embedded vocabularies for accurate BPE token counting. CountPromptTokens applies conservative padding so returned context token counts stay slightly above the real value instead of undercounting.
2026-04-30 00:44:11 +08:00
CJACK.
bc2a78ae29 Merge pull request #370 from CJackHwang/codex/align-vercel-behavior-with-go
fix(vercel): align JS stream parser with Go object-shaped content
2026-04-30 00:12:32 +08:00
CJACK.
192cdf8562 fix(vercel): align JS stream parser with Go object-shaped content 2026-04-29 23:56:16 +08:00
CJACK.
273c18ba0f fix: fallback to /app config when /data is unavailable 2026-04-29 23:40:07 +08:00
CJACK.
ae28e33184 fix: preserve continue state when chunk status is missing 2026-04-29 23:25:18 +08:00
CJACK.
0438ce9a12 Merge pull request #368 from CJackHwang/codex/fix-review-issues-for-pr-#364
Restore thinking fallback for tool-call detection and drop history.txt wrapper tags
2026-04-29 23:07:36 +08:00
CJACK.
af4a067dab Merge pull request #362 from CJackHwang/codex/fix-issue-based-on-feedback
fix(sse): batch tiny stream chunks before emitting
2026-04-29 23:07:03 +08:00
CJACK.
33f6fef015 Fix tool-call fallback on sanitized empty text and remove history wrapper tags 2026-04-29 23:04:45 +08:00
CJACK.
6d3979a1d6 fix(sse): stop scanner sender when stream context cancels 2026-04-29 22:59:22 +08:00
CJACK.
c8922c7a88 Merge pull request #364 from adnxx1wsx/dev
Fix stream compatibility and vision model exposure
2026-04-29 22:02:19 +08:00
MiY
241334c658 Fix stream compatibility and vision model exposure 2026-04-29 20:23:13 +08:00
CJACK.
d7e071b24a Bump version from 4.1.3 to 4.2.0 2026-04-29 19:08:57 +08:00
CJACK.
89225c778e fix(sse): batch tiny stream chunks before emitting 2026-04-29 18:58:54 +08:00
CJACK.
22160de2c4 Merge pull request #359 from NgoQuocViet2001/ai/ds2api-small-fix
fix(openai): keep citation indexes one-based with zero-based references
2026-04-29 18:27:15 +08:00
NgoQuocViet2001
0cbc2c875d fix(openai): keep citation indexes one-based 2026-04-29 15:43:09 +07:00
CJACK.
a0984ef682 Merge pull request #358 from CJackHwang/revert-356-codex/check-version-update-in-automation-scripts
Revert "Verify GHCR latest tag matches release and show version source/latest in dashboard"
2026-04-29 14:49:41 +08:00
CJACK.
babfa973d6 Revert "Verify GHCR latest tag matches release and show version source/latest in dashboard" 2026-04-29 14:47:53 +08:00
CJACK.
ba4071d8b5 Merge pull request #357 from CJackHwang/codex/update-documentation-for-config.json-permissions
Return config persistence warning when config path is read-only; default container config to /data/config.json and update docs
2026-04-29 14:18:25 +08:00
CJACK.
e1f8e493d2 fix: add legacy /app/config.json fallback for container upgrades 2026-04-29 14:12:20 +08:00
CJACK.
907104a735 Merge pull request #356 from CJackHwang/codex/check-version-update-in-automation-scripts
Verify GHCR latest tag matches release and show version source/latest in dashboard
2026-04-29 13:53:42 +08:00
CJACK.
2c8409dcbb fix docker defaults to writable /data config path and align docs 2026-04-29 13:46:22 +08:00
CJACK.
5c23261932 webui: show version source and latest release tag in sidebar 2026-04-29 13:45:33 +08:00
CJACK.
d7125ea106 Bump version from 4.1.2 to 4.1.3 2026-04-29 07:55:48 +08:00
CJACK.
929d9a8ef7 Merge pull request #352 from shern-point/fix/tool-string-schema-protection
Fix/tool type schema protection
2026-04-29 07:51:21 +08:00
CJACK.
c03f733b83 Merge pull request #353 from Gingiris/docs/add-toc
docs: add Table of Contents to README.MD and README.en.md
2026-04-29 07:50:54 +08:00
Gingiris
047fc9bee2 docs: add Table of Contents to README.MD and README.en.md
Both READMEs are 400+ lines with 14 top-level sections and multiple
subsections but have no navigation aid. Add a Table of Contents at the
top of each file to help readers quickly find relevant sections.

Changes:
- README.MD: add 目录 section with links to all h2/h3 headings
- README.en.md: add Table of Contents with matching structure
2026-04-28 12:18:37 -07:00
shern-point
52558838ef docs: document request-scoped tool schema authority 2026-04-29 02:00:20 +08:00
shern-point
f1926a6ced fix: normalize Vercel stream tool arguments by schema 2026-04-29 02:00:01 +08:00
shern-point
6e21714e23 test: cover Claude schema-aware tool normalization 2026-04-29 01:59:42 +08:00
shern-point
48c4f0df9f fix: preserve runtime tool schemas in Claude tool output 2026-04-29 01:59:24 +08:00
shern-point
a550de30af fix: expand shared tool schema extraction 2026-04-29 01:59:05 +08:00
CJACK.
23422e4a8e Merge pull request #350 from ouqiting/fix_chat_histroy
feat: parse split context files in list view
2026-04-29 01:34:10 +08:00
CJACK.
9c33bed403 Merge pull request #349 from RinZ27/fix-docker-non-root
build: improve Docker robustness and fix potential security issues
2026-04-29 01:34:00 +08:00
ouqiting
c81294f1b7 fix(chat-history): support tool turns in parsed HISTORY list view 2026-04-29 01:27:14 +08:00
ouqiting
28d2b0410f feat: parse split context files in list view 2026-04-29 01:15:29 +08:00
RinZ27
0c782407f5 build: improve Docker robustness and fix potential security issues 2026-04-28 23:49:54 +07:00
CJACK.
27eb73d48b Merge pull request #346 from CJackHwang/dev
Normalize string tool inputs and enhance schema protection
2026-04-28 22:06:41 +08:00
CJACK.
685b5011e4 Merge pull request #343 from livesRan/fix-429Resend-pr
支持 reference 引用标签转链接,并兼容 0 基序号映射
2026-04-28 21:47:15 +08:00
songguoliang
15e9eb3639 支持 reference 引用标签转链接,并兼容 0 基序号映射 2026-04-28 16:42:37 +08:00
112 changed files with 12242 additions and 4199 deletions

View File

@@ -165,6 +165,8 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
| PUT | `/admin/chat-history/settings` | Admin | Update conversation history retention limit |
| GET | `/admin/version` | Admin | Check current version and latest Release |
OpenAI `/v1/*` paths are canonical. For clients configured with the bare DS2API service URL, the same OpenAI handlers are also exposed through root shortcuts: `/models`, `/models/{id}`, `/chat/completions`, `/responses`, `/responses/{response_id}`, `/embeddings`, and `/files`.
---
## Health Endpoints
@@ -199,8 +201,7 @@ No auth required. Returns the currently supported DeepSeek native model list.
{"id": "deepseek-v4-pro", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-flash-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
{"id": "deepseek-v4-vision", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
]
}
```
@@ -224,6 +225,8 @@ Built-in aliases come from `internal/config/models.go`; `config.model_aliases` c
- Gemini: `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-pro-vision`
- Other compatibility families: `llama-*`, `qwen-*`, `mistral-*`, and `command-*` fall back through family heuristics
Current vision support resolves only to `deepseek-v4-vision` and does not expose a separate `vision-search` variant.
Retired historical families such as `claude-1.*`, `claude-2.*`, `claude-instant-*`, and `gpt-3.5*` are explicitly rejected.
### `POST /v1/chat/completions`
@@ -917,12 +920,15 @@ Updates proxy binding for a specific account.
"message": "API test successful (session creation only)",
"model": "deepseek-v4-flash",
"session_count": 0,
"config_writable": true
"config_writable": true,
"config_warning": ""
}
```
If a `message` is provided, `thinking` may also be included when the upstream response carries reasoning text.
When the configured file path is not writable (for example, read-only `/app/config.json` inside some containers), login/session testing still proceeds; `config_warning` is returned to indicate token persistence failed and the token is memory-only until restart.
### `POST /admin/accounts/test-all`
Optional request field: `model`.

12
API.md
View File

@@ -165,6 +165,8 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
| PUT | `/admin/chat-history/settings` | Admin | 更新对话记录保留条数 |
| GET | `/admin/version` | Admin | 查询当前版本与最新 Release |
OpenAI `/v1/*` 仍是规范路径。对于只配置 DS2API 根地址的客户端,同一套 OpenAI handler 也通过根路径快捷路由暴露:`/models``/models/{id}``/chat/completions``/responses``/responses/{response_id}``/embeddings``/files`
---
## 健康检查
@@ -204,9 +206,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
{"id": "deepseek-v4-pro-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro-search-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-search-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
{"id": "deepseek-v4-vision-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
]
}
```
@@ -232,6 +232,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
- 其他兼容族:`llama-*``qwen-*``mistral-*``command-*` 会按家族启发式回退
上述 alias 若在请求名后追加 `-nothinking` 后缀,也会映射到对应的强制关闭 thinking 版本。
当前视觉能力仅对应 `deepseek-v4-vision` / `deepseek-v4-vision-nothinking`,不会解析出独立的 `vision-search` 变体。
退役历史模型(如 `claude-1.*``claude-2.*``claude-instant-*``gpt-3.5*`)会被显式拒绝。
@@ -934,12 +935,15 @@ data: {"type":"message_stop"}
"message": "API 测试成功(仅会话创建)",
"model": "deepseek-v4-flash",
"session_count": 0,
"config_writable": true
"config_writable": true,
"config_warning": ""
}
```
如果传入 `message`,还会附带 `thinking`(当上游返回思考内容时)。
当部署环境配置文件路径不可写(例如容器内默认 `/app/config.json` 只读)时,登录与会话测试仍可继续;此时会返回 `config_warning` 提示 token 仅保存在内存、重启后丢失。
### `POST /admin/accounts/test-all`
可选请求字段:`model`

View File

@@ -28,6 +28,8 @@ FROM debian:bookworm-slim AS runtime-base
WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates \
&& groupadd -r ds2api && useradd -r -g ds2api -d /app -s /sbin/nologin ds2api \
&& mkdir -p /app/data && chown -R ds2api:ds2api /app \
&& rm -rf /var/lib/apt/lists/*
COPY --from=busybox-tools /bin/busybox /usr/local/bin/busybox
EXPOSE 5001
@@ -36,8 +38,9 @@ CMD ["/usr/local/bin/ds2api"]
FROM runtime-base AS runtime-from-source
COPY --from=go-builder /out/ds2api /usr/local/bin/ds2api
COPY --from=go-builder /app/config.example.json /app/config.example.json
COPY --from=webui-builder /app/static/admin /app/static/admin
COPY --from=go-builder --chown=ds2api:ds2api /app/config.example.json /app/config.example.json
COPY --from=webui-builder --chown=ds2api:ds2api /app/static/admin /app/static/admin
USER ds2api
FROM busybox-tools AS dist-extract
ARG TARGETARCH
@@ -60,7 +63,8 @@ RUN set -eux; \
FROM runtime-base AS runtime-from-dist
COPY --from=dist-extract /out/ds2api /usr/local/bin/ds2api
COPY --from=dist-extract /out/config.example.json /app/config.example.json
COPY --from=dist-extract /out/static/admin /app/static/admin
COPY --from=dist-extract --chown=ds2api:ds2api /out/config.example.json /app/config.example.json
COPY --from=dist-extract --chown=ds2api:ds2api /out/static/admin /app/static/admin
USER ds2api
FROM runtime-from-source AS final

View File

@@ -31,6 +31,30 @@
>
> 请勿将本项目用于违反服务条款、协议、法律法规或平台规则的场景。商业使用前请自行确认 `LICENSE`、相关协议以及你是否获得了作者的书面许可。
## 目录
- [架构概览(摘要)](#架构概览摘要)
- [核心能力](#核心能力)
- [平台兼容矩阵](#平台兼容矩阵)
- [模型支持](#模型支持)
- [OpenAI 接口](#openai-接口get-v1models)
- [Claude 接口](#claude-接口get-anthropicv1models)
- [Gemini 接口](#gemini-接口)
- [快速开始](#快速开始)
- [方式一:下载 Release 构建包](#方式一下载-release-构建包)
- [方式二Docker 运行](#方式二docker-运行)
- [方式三Vercel 部署](#方式三vercel-部署)
- [方式四:本地源码运行](#方式四本地源码运行)
- [配置说明](#配置说明)
- [鉴权模式](#鉴权模式)
- [并发模型](#并发模型)
- [Tool Call 适配](#tool-call-适配)
- [本地开发抓包工具](#本地开发抓包工具)
- [文档索引](#文档索引)
- [测试](#测试)
- [Release 自动构建GitHub Actions](#release-自动构建github-actions)
- [免责声明](#免责声明)
## 架构概览(摘要)
```mermaid
@@ -107,6 +131,8 @@ flowchart LR
| WebUI 管理台 | `/admin` 单页应用(中英文双语、深色模式,支持查看服务器端对话记录) |
| 运维探针 | `GET /healthz`(存活)、`GET /readyz`(就绪) |
OpenAI `/v1/*` 仍是推荐的规范路径;同时支持 `/models`、`/chat/completions`、`/responses`、`/embeddings`、`/files` 等根路径快捷路由,方便只配置 DS2API 根地址的第三方客户端。
## 平台兼容矩阵
| 级别 | 平台 | 当前状态 |
@@ -134,10 +160,9 @@ flowchart LR
| expert | `deepseek-v4-pro-search-nothinking` | 永久关闭,不受请求参数影响 | ✅ |
| vision | `deepseek-v4-vision` | 默认开启,可由请求参数控制 | ❌ |
| vision | `deepseek-v4-vision-nothinking` | 永久关闭,不受请求参数影响 | ❌ |
| vision | `deepseek-v4-vision-search` | 默认开启,可由请求参数控制 | ✅ |
| vision | `deepseek-v4-vision-search-nothinking` | 永久关闭,不受请求参数影响 | ✅ |
除原生模型外,也支持常见 alias 输入(如 `gpt-4.1`、`gpt-5`、`gpt-5-codex`、`o3`、`claude-*`、`gemini-*` 等),但 `/v1/models` 返回的是规范化后的 DeepSeek 原生模型 ID。若 alias 名本身追加 `-nothinking` 后缀,也会映射到对应的强制关思考模型。完整 alias 行为以 [API.md](API.md#模型-alias-解析策略) 和 `config.example.json` 为准。
当前上游视觉模型只暴露 `vision` 通道,不提供独立的联网搜索视觉变体。
### Claude 接口(`GET /anthropic/v1/models`
@@ -221,6 +246,7 @@ docker-compose logs -f
```
默认 `docker-compose.yml` 会把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请设置 `DS2API_HOST_PORT=5001`(或者手动调整 `ports` 配置)。
同时默认把 `./config.json` 挂载到容器 `/data/config.json`,并设置 `DS2API_CONFIG_PATH=/data/config.json`,用于避免 `/app` 只读导致运行时 token 持久化失败。
更新镜像:`docker-compose up -d --build`
@@ -291,7 +317,7 @@ go run ./cmd/ds2api
- `runtime`:账号并发、队列与 token 刷新策略,可通过 Admin Settings 热更新。
- `auto_delete.mode`:请求结束后的远端会话清理策略,支持 `none` / `single` / `all`。
- `history_split`:旧轮次拆分字段,已废弃并忽略,仅保留兼容旧配置。
- `current_input_file`:唯一生效的独立拆分策略;默认开启且阈值为 `0`,触发时将完整上下文合并上传为隐藏上下文文件。
- `current_input_file`:唯一生效的独立拆分策略;默认开启且阈值为 `0`,触发时将完整上下文合并上传为 `history.txt` 上下文文件。
- 如果关闭 `current_input_file`,请求会直接透传,不上传拆分上下文文件。
- `thinking_injection`:默认开启;在最新 user 消息末尾追加思考增强提示词,提高高强度推理与工具调用前的思考稳定性;`prompt` 留空时使用内置默认提示词。

View File

@@ -28,6 +28,30 @@ Documentation entry: [Docs Index](docs/README.md) / [Architecture](docs/ARCHITEC
>
> Do not use this project in ways that violate service terms, agreements, laws, or platform rules. Before any commercial use, review the `LICENSE`, the relevant terms, and confirm that you have the author's written permission.
## Table of Contents
- [Architecture Overview (Summary)](#architecture-overview-summary)
- [Key Capabilities](#key-capabilities)
- [Platform Compatibility Matrix](#platform-compatibility-matrix)
- [Model Support](#model-support)
- [OpenAI Endpoint](#openai-endpoint-get-v1models)
- [Claude Endpoint](#claude-endpoint-get-anthropicv1models)
- [Gemini Endpoint](#gemini-endpoint)
- [Quick Start](#quick-start)
- [Option 1: Download Release Binaries](#option-1-download-release-binaries)
- [Option 2: Docker / GHCR](#option-2-docker--ghcr)
- [Option 3: Vercel](#option-3-vercel)
- [Option 4: Local Run](#option-4-local-run)
- [Configuration](#configuration)
- [Authentication Modes](#authentication-modes)
- [Concurrency Model](#concurrency-model)
- [Tool Call Adaptation](#tool-call-adaptation)
- [Local Dev Packet Capture](#local-dev-packet-capture)
- [Documentation Index](#documentation-index)
- [Testing](#testing)
- [Release Artifact Automation (GitHub Actions)](#release-artifact-automation-github-actions)
- [Disclaimer](#disclaimer)
## Architecture Overview (Summary)
```mermaid
@@ -104,6 +128,8 @@ For the full module-by-module architecture and directory responsibilities, see [
| WebUI Admin Panel | SPA at `/admin` (bilingual Chinese/English, dark mode, with server-side conversation history) |
| Health Probes | `GET /healthz` (liveness), `GET /readyz` (readiness) |
OpenAI `/v1/*` routes remain canonical, and DS2API also accepts root shortcuts such as `/models`, `/chat/completions`, `/responses`, `/embeddings`, and `/files` for clients configured with the bare service URL.
## Platform Compatibility Matrix
| Tier | Platform | Status |
@@ -126,9 +152,9 @@ For the full module-by-module architecture and directory responsibilities, see [
| default | `deepseek-v4-flash-search` | enabled by default, request-controlled | ✅ |
| expert | `deepseek-v4-pro-search` | enabled by default, request-controlled | ✅ |
| vision | `deepseek-v4-vision` | enabled by default, request-controlled | ❌ |
| vision | `deepseek-v4-vision-search` | enabled by default, request-controlled | ✅ |
Besides native IDs, DS2API also accepts common aliases as input (for example `gpt-4.1`, `gpt-5`, `gpt-5-codex`, `o3`, `claude-*`, `gemini-*`), but `/v1/models` returns normalized DeepSeek native model IDs. The complete alias behavior is documented in [API.en.md](API.en.md#model-alias-resolution) and `config.example.json`.
Current upstream vision support exposes only the `vision` lane and does not provide a separate search-enabled vision variant.
### Claude Endpoint (`GET /anthropic/v1/models`)
@@ -209,6 +235,7 @@ docker-compose up -d
```
The default `docker-compose.yml` uses `ghcr.io/cjackhwang/ds2api:latest` and maps host port `6011` to container port `5001`. If you want `5001` exposed directly, set `DS2API_HOST_PORT=5001` (or adjust the `ports` mapping).
It also mounts `./config.json` to `/data/config.json` and sets `DS2API_CONFIG_PATH=/data/config.json` by default, which avoids runtime token persistence failures caused by read-only `/app`.
Rebuild after updates: `docker-compose up -d --build`
@@ -279,7 +306,7 @@ Common fields:
- `runtime`: account concurrency, queueing, and token refresh behavior, hot-reloadable via Admin Settings.
- `auto_delete.mode`: remote session cleanup after each request, supporting `none` / `single` / `all`.
- `history_split`: legacy multi-turn history split field, now ignored and kept only for backward-compatible config loading.
- `current_input_file`: the only active split mode; it is enabled by default and uploads the full context as a hidden context file once the character threshold is reached.
- `current_input_file`: the only active split mode; it is enabled by default and uploads the full context as a `history.txt` context file once the character threshold is reached.
- If you turn off `current_input_file`, requests pass through directly without uploading any split context file.
For the full environment variable list, see [docs/DEPLOY.en.md](docs/DEPLOY.en.md). For auth behavior, see [API.en.md](API.en.md#authentication).

View File

@@ -1 +1 @@
4.1.2
4.2.0

View File

@@ -35,8 +35,9 @@ func main() {
}
srv := &http.Server{
Addr: "0.0.0.0:" + port,
Handler: app.Router,
Addr: "0.0.0.0:" + port,
Handler: app.Router,
ReadHeaderTimeout: 5 * time.Second,
}
localURL := fmt.Sprintf("http://127.0.0.1:%s", port)
lanIP := detectLANIPv4()

View File

@@ -9,8 +9,9 @@ services:
# Host port is configurable via DS2API_HOST_PORT; container port stays fixed at 5001.
- "${DS2API_HOST_PORT:-6011}:5001"
volumes:
- ./config.json:/app/config.json # 配置文件
- ./config.json:/data/config.json # 配置文件(持久化推荐路径)
environment:
- TZ=Asia/Shanghai
- LOG_LEVEL=INFO
- DS2API_ADMIN_KEY=${DS2API_ADMIN_KEY:-ds2api}
- DS2API_CONFIG_PATH=/data/config.json

View File

@@ -130,6 +130,8 @@ docker-compose logs -f
```
The default `docker-compose.yml` directly uses `ghcr.io/cjackhwang/ds2api:latest` and maps host port `6011` to container port `5001`. If you want `5001` exposed directly, set `DS2API_HOST_PORT=5001` (or adjust the `ports` mapping).
The compose template also defaults to `DS2API_CONFIG_PATH=/data/config.json` with `./config.json:/data/config.json` mounted, so deployments avoid read-only `/app` persistence issues by default.
Compatibility note: when `DS2API_CONFIG_PATH` is unset and runtime base dir is `/app`, newer versions prefer `/data/config.json`; if that file is missing but legacy `/app/config.json` exists, DS2API automatically falls back to the legacy path to avoid post-upgrade config loss.
If you want a pinned version instead of `latest`, you can also pull a specific tag directly:
@@ -195,6 +197,11 @@ Notes:
- **Port**: DS2API listens on `5001` by default; the template sets `PORT=5001`.
- **Persistent config**: the template mounts `/data` and sets `DS2API_CONFIG_PATH=/data/config.json`. After importing config in Admin UI, it will be written and persisted to this path.
- **`open /app/config.json: permission denied`**: this means the instance is trying to persist runtime tokens to a read-only path (commonly `/app` inside the image).
Recommended handling:
1. Set a writable path explicitly: `DS2API_CONFIG_PATH=/data/config.json` (and mount a persistent volume at `/data`);
2. If you bootstrap with `DS2API_CONFIG_JSON` and do not need runtime writeback, keep env-backed mode (`DS2API_ENV_WRITEBACK` disabled);
3. In current versions, login/session tests continue even if persistence fails; Admin API returns a warning that token persistence failed and token is memory-only until restart.
- **Build version**: Zeabur / regular `docker build` does not require `BUILD_VERSION` by default. The image prefers that build arg when provided, and automatically falls back to the repo-root `VERSION` file when it is absent.
- **First login**: after deployment, open `/admin` and login with `DS2API_ADMIN_KEY` shown in Zeabur env/template instructions (recommended: rotate to a strong secret after first login).

View File

@@ -130,6 +130,8 @@ docker-compose logs -f
```
默认 `docker-compose.yml` 直接使用 `ghcr.io/cjackhwang/ds2api:latest`,并把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请设置 `DS2API_HOST_PORT=5001`(或者手动调整 `ports` 配置)。
Compose 模板还会默认设置 `DS2API_CONFIG_PATH=/data/config.json` 并挂载 `./config.json:/data/config.json`,优先避免 `/app` 只读带来的配置持久化问题。
兼容说明:若未设置 `DS2API_CONFIG_PATH` 且运行目录是 `/app`,新版本会优先使用 `/data/config.json`;当该文件不存在但检测到历史 `/app/config.json` 时,会自动回退读取旧路径,避免升级后“配置丢失”。
如需固定版本,也可以直接拉取指定 tag
@@ -195,6 +197,11 @@ healthcheck:
- **端口**:服务默认监听 `5001`,模板会固定设置 `PORT=5001`
- **配置持久化**:模板挂载卷 `/data`,并设置 `DS2API_CONFIG_PATH=/data/config.json`;在管理台导入配置后,会写入并持久化到该路径。
- **`open /app/config.json: permission denied`**:说明当前实例在尝试把运行时 token 持久化到只读路径(常见于镜像内 `/app`)。
处理建议:
1. 显式设置可写路径:`DS2API_CONFIG_PATH=/data/config.json`(并挂载持久卷到 `/data`
2. 若你使用 `DS2API_CONFIG_JSON` 启动且不需要运行时落盘,可保持环境变量模式(`DS2API_ENV_WRITEBACK` 关闭);
3. 最新版本中,即使持久化失败,登录/会话测试仍会继续仅提示“token 未持久化(重启后丢失)”。
- **构建版本号**Zeabur / 普通 `docker build` 默认不需要传 `BUILD_VERSION`;镜像会优先使用该构建参数,未提供时自动回退到仓库根目录的 `VERSION` 文件。
- **首次登录**:部署完成后访问 `/admin`,使用 Zeabur 环境变量/模板指引中的 `DS2API_ADMIN_KEY` 登录(建议首次登录后自行更换为强密码)。

View File

@@ -309,7 +309,18 @@ parse SSE block
- 新模型可能增加新的 `p` 路径。
- 新版本可能增加新的 fragment.type。
- `CONTENT_FILTER` 的终态模板内容可能变化。
- 自动续写相关状态(如 `INCOMPLETE` / `AUTO_CONTINUE`)当前主要来自实测与实现兼容逻辑,后续字段形态仍可能变化。
- 自动续写相关状态(如 `INCOMPLETE` / `AUTO_CONTINUE`)当前主要来自实测与实现兼容逻辑,后续字段形态仍可能变化。当前实现不会仅因早期 `WIP` 状态就自动继续;只有显式 `INCOMPLETE``auto_continue` 信号才会触发 continue。
- 解析器应当对未知字段、未知路径、未知事件保持容忍。
如果你要把这份说明用于实际开发,建议同时保留原始流样本、回放脚本和回归测试,不要只依赖本文。
## 2026-04-29 最近线上样本增量观察
基于 `longtext-deepseek-v4-flash-20260429``longtext-deepseek-v4-pro-20260429` 两个真实账号长文本样本,近期格式变化要点如下:
1. `data:` 事件中仍大量出现 `{"v":"..."}` 的无路径增量(`p` 缺失),解析器必须把空路径视为可见正文候选,而不能只依赖 `response/content`
2. 对象形态 `v`(如 `{"text":"..."}` / `{"content":"..."}`)仍会出现,且可能与无路径 chunk 混用;仅按字符串处理会导致正文丢块。
3. 多轮 continuation 场景下,后续 chunk 可能不再重复显式 `status`,状态机需要保留上一轮 `INCOMPLETE` 语义直到出现终态。
4. 2026-04-29 起客户端头部版本基线上调到 `x-client-version: 2.0.3`,否则部分账号会出现上游行为不一致(包括空输出与补轮异常)。
建议:新增样本默认回放应优先覆盖「长文本 + 多轮 + 无路径 chunk」组合避免只用短样本导致回归漏检。

View File

@@ -98,13 +98,16 @@ DS2API 当前的核心思路,不是把客户端传来的 `messages`、`tools`
- `prompt` 才是对话上下文主载体。
- `ref_file_ids` 只承载文件引用,不承载普通文本消息。
- `tools` 不会作为“原生工具 schema”直接下发给下游而是被改写进 `prompt`
- 对外返回给客户端的 `prompt_tokens` / `input_tokens` / `promptTokenCount` 不再按“最后一条消息”或字符粗估近似返回,而是基于**完整上下文 prompt**做 tokenizer 计数;为了避免上下文实际超限但客户端误以为还能塞下,请求侧上下文 token 会额外保守上浮一点,宁可略大也不低估。
- 当前 `/v1/chat/completions` 业务路径仍是“每次请求新建一个远端 `chat_session_id`,并默认发送 `parent_message_id: null`”;因此 DS2API 对外默认表现为“新会话 + prompt 拼历史”,而不是复用 DeepSeek 原生会话树。
- 但 DeepSeek 远端本身支持同一 `chat_session_id` 的跨轮次持续对话。2026-04-27 已用项目内现有 DeepSeek client 做过一次不改业务代码的双轮实测:同一 `chat_session_id` 下,第 1 轮返回 `request_message_id=1` / `response_message_id=2` / 文本 `SESSION_TEST_ONE`;第 2 轮重新获取一次 PoW并发送 `parent_message_id=2` 后,成功返回 `request_message_id=3` / `response_message_id=4` / 文本 `SESSION_TEST_TWO`。这说明“同远端会话持续聊天”能力存在,且每轮需要携带正确的 parent/message 链接信息,同时重新获取对应轮次可用的 PoW。
- OpenAI Chat / Responses 原生走统一 OpenAI 标准化与 DeepSeek payload 组装Claude / Gemini 会尽量复用 OpenAI prompt/tool 语义,其中 Gemini 直接复用 `promptcompat.BuildOpenAIPromptForAdapter`Claude 消息接口在可代理场景会转换为 OpenAI chat 形态再执行。
- 客户端传入的 thinking / reasoning 开关会被归一到下游 `thinking_enabled`。Gemini `generationConfig.thinkingConfig.thinkingBudget` 会翻译成同一套 thinking 开关;关闭时即使上游返回 `response/thinking_content`,兼容层也不会把它当作可见正文输出。若最终解析出的模型名带 `-nothinking` 后缀,则会无条件强制关闭 thinking优先级高于请求体中的 `thinking` / `reasoning` / `reasoning_effort`。Claude surface 在流式请求且未显式声明 `thinking` 时,仍按 Anthropic 语义默认关闭;但在非流式代理场景,兼容层会内部开启一次下游 thinking用于捕获“正文为空、工具调用落在 thinking 里”的情况,随后在回包前剥离用户不可见的 thinking block。
- 对 OpenAI Chat / Responses 的非流式收尾,如果最终可见正文为空,兼容层会优先尝试把思维链中的独立 DSML / XML 工具块当作真实工具调用解析出来。流式链路也会在收尾阶段做同样的 fallback 检测,但不会因为思维链内容去中途拦截或改写流式输出;thinking / reasoning 增量仍按原样先发,只有在结束收尾时才可能补发最终工具调用结果。补发结果会作为本轮 assistant 的结构化 `tool_calls` / `function_call` 输出返回,而不是塞进 `content` 文本;如果客户端没有开启 thinking / reasoning思维链只用于检测不会作为 `reasoning_content` 或可见正文暴露。只有正文为空且思维链里也没有可执行工具调用时,才继续按空回复错误处理。
- 对 OpenAI Chat / Responses 的非流式收尾,如果最终可见正文为空,兼容层会优先尝试把思维链中的独立 DSML / XML 工具块当作真实工具调用解析出来。流式链路也会在收尾阶段做同样的 fallback 检测,但不会因为思维链内容去中途拦截或改写流式输出;真正的工具识别始终基于原始上游文本,而不是基于“已经做过可见输出清洗”的版本,因此即使最终可见层会剥离完整 leaked DSML / XML `tool_calls` wrapper、并抑制全空参数或无效 wrapper 块,也不会影响真实工具调用转成结构化 `tool_calls` / `function_call`。补发结果会作为本轮 assistant 的结构化 `tool_calls` / `function_call` 输出返回,而不是塞进 `content` 文本;如果客户端没有开启 thinking / reasoning思维链只用于检测不会作为 `reasoning_content` 或可见正文暴露。只有正文为空且思维链里也没有可执行工具调用时,才继续按空回复错误处理。
- OpenAI Chat / Responses 的空回复错误处理之前会默认做一次内部补偿重试:第一次上游完整结束后,如果最终可见正文为空、没有解析到工具调用、也没有已经向客户端流式发出工具调用,并且终止原因不是 `content_filter`,兼容层会复用同一个 `chat_session_id`、账号、token 与工具策略,把原始 completion `prompt` 追加固定后缀 `Previous reply had no visible output. Please regenerate the visible final answer or tool call now.` 后重新提交一次。重试遵循 DeepSeek 多轮对话协议:从第一次上游 SSE 流中提取 `response_message_id`,并在重试 payload 中设置 `parent_message_id` 为该值,使重试成为同一会话的后续轮次而非断裂的根消息;同时重新获取一次 PoW若 PoW 获取失败则回退到原始 PoW。该重试不会重新标准化消息、不会新建 session、不会切换账号也不会向流式客户端插入重试标记第二次 thinking / reasoning 会按正常增量直接接到第一次之后,并继续使用 overlap trim 去重。若第二次仍为空,终端错误码仍保持现有 `upstream_empty_output`;若任一尝试触发空 `content_filter`,不做补偿重试并保持 `content_filter` 错误。JS Vercel 运行时同样设置 `parent_message_id`,但因无法直接调用 PoW API 而复用原始 PoW。
- OpenAI Chat / Responses 在最终可见正文渲染阶段,会把 DeepSeek 搜索返回中的 `[citation:N]` / `[reference:N]` 标记替换成对应 Markdown 链接。`citation` 标记按一基序号解析;`reference` 标记只有在同一段正文中出现 `[reference:0]`(允许冒号后有空格)时才按零基序号映射,并且不会影响同段正文里的 `citation` 标记。
## 5. prompt 是怎么拼出来的
OpenAI Chat / Responses 在标准化后、current input file 之前,会默认执行 `thinking_injection` 增强。它参考 DeepSeek V4 “把控制指令放在 user 消息末尾更稳定”的用法,在最新 user message 后追加思考增强提示词。当前内置默认提示词以 `Reasoning Effort: Absolute maximum with no shortcuts permitted.` 开头,并继续要求模型充分分解问题、覆盖潜在路径与边界条件、把完整推演过程显式写出。该开关默认启用,可通过 `thinking_injection.enabled=false` 关闭;也可以通过 `thinking_injection.prompt` 自定义提示词,留空时使用内置默认提示词。
@@ -154,6 +157,7 @@ OpenAI Chat / Responses 在标准化后、current input file 之前,会默认
兼容层仍接受旧式纯 `<tool_calls>` wrapper但提示词会优先要求模型输出官方 DSML 标签,并强调不能只输出 closing wrapper 而漏掉 opening tag。需要注意这是“兼容 DSML 外壳,内部仍以 XML 解析语义为准”,不是原生 DSML 全链路实现DSML 标签会在解析入口归一化回现有 XML 标签后继续走同一套 parser。
数组参数使用 `<item>...</item>` 子节点表示;当某个参数体只包含 item 子节点时Go / Node 解析器会把它还原成数组,避免 `questions` / `options` 这类 schema 中要求 array 的参数被误解析成 `{ "item": ... }` 对象。若模型把完整结构化 XML fragment 误包进 CDATA兼容层会在保护 `content` / `command` 等原文字段的前提下,尝试把非原文字段中的 CDATA XML fragment 还原成 object / array。不过如果 CDATA 只是单个平面的 XML/HTML 标签,例如 `<b>urgent</b>` 这种行内标记,兼容层会保留原始字符串,不会强行升成 object / array只有明显表示结构的 CDATA 片段,例如多兄弟节点、嵌套子节点或 `item` 列表,才会触发结构化恢复。
在 assistant 最终回包阶段,如果某个 tool 参数在声明 schema 中明确是 `string`,兼容层会在把解析后的 `tool_calls` / `function_call` 重新序列化成 OpenAI / Responses / Claude 可见参数前,递归把该路径上的 number / bool / object / array 统一转成字符串;其中 object / array 会压成紧凑 JSON 字符串。这个保护只对 schema 明确声明为 string 的路径生效,不会改写本来就是 `number` / `boolean` / `object` / `array` 的参数。这样可以兼容 DeepSeek 输出了结构化片段、但上游客户端工具 schema 又严格要求字符串参数的场景(例如 `content``prompt``path``taskId` 等)。
工具 schema 的权威来源始终是**当前请求实际携带的 schema**,而不是同名工具在其他 runtimeClaude Code / OpenCode / Codex 等)里的默认印象。兼容层现在会同时兼容 OpenAI 风格 `function.parameters`、直接工具对象上的 `parameters` / `input_schema`、以及 camelCase 的 `inputSchema` / `schema`,并在最终输出阶段按这份请求内 schema 决定是保留 array/object还是仅对明确声明为 `string` 的路径做字符串化。该规则同样适用于 Claude 的流式收尾和 Vercel Node 流式 tool-call formatter避免不同 runtime 因 schema shape 差异而出现同名工具参数类型漂移。
正例中的工具名只会来自当前请求实际声明的工具;如果当前请求没有足够的已知工具形态,就省略对应的单工具、多工具或嵌套示例,避免把不可用工具名写进 prompt。
对执行类工具,脚本内容必须进入执行参数本身:`Bash` / `execute_command` 使用 `command``exec_command` 使用 `cmd`;不要把脚本示范成 `path` / `content` 文件写入参数。
@@ -243,9 +247,10 @@ OpenAI 文件相关实现:
兼容层现在只保留 `current_input_file` 这一种拆分方式;旧的 `history_split` 已废弃,只保留为兼容旧配置的字段,不再参与请求处理。
- `current_input_file` 默认开启;它用于把“完整上下文”合并进隐藏上下文文件。当最新 user turn 的纯文本长度达到 `current_input_file.min_chars`(默认 `0`)时,兼容层会上传一个文件名为 `IGNORE.txt` 的上下文文件,并在 live prompt 中只保留一个中性的 user 消息要求模型直接回答最新请求,不再暴露文件名或要求模型读取本地文件。
- `current_input_file` 默认开启;它用于把“完整上下文”合并进 `history.txt` 上下文文件。当最新 user turn 的纯文本长度达到 `current_input_file.min_chars`(默认 `0`)时,兼容层会上传一个文件名为 `history.txt` 的上下文文件,并在 live prompt 中只保留一个中性的 user 消息要求模型直接回答最新请求,不再暴露文件名或要求模型读取本地文件。
- 如果 `current_input_file.enabled=false`,请求会直接透传,不上传任何拆分上下文文件。
- 旧的 `history_split.enabled` / `history_split.trigger_after_turns` 会被读取进配置对象以保持兼容,但不会触发拆分上传,也不会影响 `current_input_file` 的默认开启。
- 即使触发 `current_input_file` 后 live prompt 被缩短,对客户端回包里的上下文 token 统计,仍会沿用**拆分前的完整 prompt 语义**做计数,而不是按缩短后的占位 prompt 计算;否则会把真实上下文显著算小。
相关实现:
@@ -256,16 +261,11 @@ OpenAI 文件相关实现:
- 旧历史拆分兼容壳:
[internal/httpapi/openai/history/history_split.go](../internal/httpapi/openai/history/history_split.go)
当前输入转文件启用并触发时,上传文件的真实文件名是 `IGNORE.txt`,文件内容是完整 `messages` 上下文;它仍会先用 OpenAI 消息标准化和 DeepSeek 角色标记序列化,再包进 `IGNORE` 文件边界
当前输入转文件启用并触发时,上传文件的真实文件名是 `history.txt`,文件内容是完整 `messages` 上下文;它仍会先用 OpenAI 消息标准化和 DeepSeek 角色标记序列化,并直接作为 `history.txt` 的纯文本内容上传(不再注入文件边界标签)
```text
[uploaded filename]: IGNORE.txt
[file content end]
[uploaded filename]: history.txt
<begin▁of▁sentence><System>...<User>...<Assistant>...<Tool>...<User>...
[file name]: IGNORE
[file content begin]
```
开启后,请求的 live prompt 不再直接内联完整上下文,而是保留一个 user role 的短提示,提示模型基于已提供上下文直接回答最新请求;上传后的 `file_id` 会进入 `ref_file_ids`
@@ -332,7 +332,7 @@ OpenAI 文件相关实现:
- 大部分结构化语义被压进 `prompt`
- 文件保持文件
- 需要时把完整上下文拆进隐藏上下文文件
- 需要时把完整上下文拆进 `history.txt` 上下文文件
## 12. 修改时必须同步本文档的场景
@@ -345,7 +345,7 @@ OpenAI 文件相关实现:
- tool result 注入方式变更
- tool prompt 模板或 tool_choice 约束变更
- inline 文件上传 / 文件引用收集规则变更
- current input file 触发条件、上传格式、`IGNORE` 包装格式变更
- current input file 触发条件、上传格式、`history.txt` 包装格式变更
-`history_split` 兼容逻辑的读取、忽略或退化行为变更
- completion payload 字段语义变更
- Claude / Gemini 对这套统一语义的复用关系变更

5
go.mod
View File

@@ -10,6 +10,11 @@ require (
github.com/router-for-me/CLIProxyAPI/v6 v6.9.14
)
require (
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/hupe1980/go-tiktoken v0.0.10 // indirect
)
require (
github.com/klauspost/compress v1.18.5 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect

4
go.sum
View File

@@ -2,10 +2,14 @@ github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eT
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hupe1980/go-tiktoken v0.0.10 h1:m6phOJaGyctqWdGIgwn9X8AfJvaG74tnQoDL+ntOUEQ=
github.com/hupe1980/go-tiktoken v0.0.10/go.mod h1:NME6d8hrE+Jo+kLUZHhXShYV8e40hYkm4BbSLQKtvAo=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=

View File

@@ -14,6 +14,7 @@ import (
"github.com/google/uuid"
"ds2api/internal/config"
"ds2api/internal/util"
)
const (
@@ -610,8 +611,8 @@ func buildPreview(item Entry) string {
if candidate == "" {
candidate = strings.TrimSpace(item.UserInput)
}
if len(candidate) > defaultPreviewAt {
return candidate[:defaultPreviewAt] + "..."
if truncated, ok := util.TruncateRunes(candidate, defaultPreviewAt); ok {
return truncated + "..."
}
return candidate
}

View File

@@ -8,6 +8,7 @@ import (
"strings"
"sync"
"testing"
"unicode/utf8"
)
func blockDetailDir(t *testing.T, detailDir string) func() {
@@ -105,6 +106,17 @@ func TestStoreCreatesAndPersistsEntries(t *testing.T) {
}
}
func TestBuildPreviewPreservesUTF8MB4Characters(t *testing.T) {
long := strings.Repeat("😀", defaultPreviewAt+1)
preview := buildPreview(Entry{Content: long})
if !utf8.ValidString(preview) {
t.Fatalf("expected valid utf-8 preview, got %q", preview)
}
if preview != strings.Repeat("😀", defaultPreviewAt)+"..." {
t.Fatalf("unexpected preview: %q", preview)
}
}
func TestStoreTrimsToConfiguredLimit(t *testing.T) {
path := filepath.Join(t.TempDir(), "chat_history.json")
store := New(path)

View File

@@ -79,13 +79,20 @@ func TestGetModelConfigDeepSeekExpertReasonerSearch(t *testing.T) {
}
}
func TestGetModelConfigDeepSeekVisionReasonerSearch(t *testing.T) {
thinking, search, ok := GetModelConfig("deepseek-v4-vision-search")
func TestGetModelConfigDeepSeekVision(t *testing.T) {
thinking, search, ok := GetModelConfig("deepseek-v4-vision")
if !ok {
t.Fatal("expected ok for deepseek-v4-vision-search")
t.Fatal("expected ok for deepseek-v4-vision")
}
if !thinking || !search {
t.Fatalf("expected both true, got thinking=%v search=%v", thinking, search)
if !thinking || search {
t.Fatalf("expected thinking=true search=false, got thinking=%v search=%v", thinking, search)
}
}
func TestGetModelConfigDeepSeekVisionSearchUnsupported(t *testing.T) {
_, _, ok := GetModelConfig("deepseek-v4-vision-search")
if ok {
t.Fatal("expected deepseek-v4-vision-search to be unsupported")
}
}
@@ -748,18 +755,16 @@ func TestOpenAIModelsResponse(t *testing.T) {
t.Fatal("expected non-empty models list")
}
expected := map[string]bool{
"deepseek-v4-flash": false,
"deepseek-v4-flash-nothinking": false,
"deepseek-v4-pro": false,
"deepseek-v4-pro-nothinking": false,
"deepseek-v4-flash-search": false,
"deepseek-v4-flash-search-nothinking": false,
"deepseek-v4-pro-search": false,
"deepseek-v4-pro-search-nothinking": false,
"deepseek-v4-vision": false,
"deepseek-v4-vision-nothinking": false,
"deepseek-v4-vision-search": false,
"deepseek-v4-vision-search-nothinking": false,
"deepseek-v4-flash": false,
"deepseek-v4-flash-nothinking": false,
"deepseek-v4-pro": false,
"deepseek-v4-pro-nothinking": false,
"deepseek-v4-flash-search": false,
"deepseek-v4-flash-search-nothinking": false,
"deepseek-v4-pro-search": false,
"deepseek-v4-pro-search-nothinking": false,
"deepseek-v4-vision": false,
"deepseek-v4-vision-nothinking": false,
}
for _, model := range data {
if _, ok := expected[model.ID]; ok {

View File

@@ -144,10 +144,17 @@ func TestResolveModelCustomAliasToExpert(t *testing.T) {
func TestResolveModelCustomAliasToVision(t *testing.T) {
got, ok := ResolveModel(mockModelAliasReader{
"my-vision-model": "deepseek-v4-vision-search",
"my-vision-model": "deepseek-v4-vision",
}, "my-vision-model")
if !ok || got != "deepseek-v4-vision-search" {
t.Fatalf("expected alias -> deepseek-v4-vision-search, got ok=%v model=%q", ok, got)
if !ok || got != "deepseek-v4-vision" {
t.Fatalf("expected alias -> deepseek-v4-vision, got ok=%v model=%q", ok, got)
}
}
func TestResolveModelHeuristicVisionIgnoresSearchSuffix(t *testing.T) {
got, ok := ResolveModel(nil, "gemini-vision-search")
if !ok || got != "deepseek-v4-vision" {
t.Fatalf("expected heuristic vision alias to resolve without search variant, got ok=%v model=%q", ok, got)
}
}

View File

@@ -22,7 +22,6 @@ var deepSeekBaseModels = []ModelInfo{
{ID: "deepseek-v4-flash-search", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
{ID: "deepseek-v4-pro-search", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
{ID: "deepseek-v4-vision", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
{ID: "deepseek-v4-vision-search", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
}
var DeepSeekModels = appendNoThinkingVariants(deepSeekBaseModels)
@@ -67,7 +66,7 @@ func GetModelConfig(model string) (thinking bool, search bool, ok bool) {
switch baseModel {
case "deepseek-v4-flash", "deepseek-v4-pro", "deepseek-v4-vision":
return !noThinking, false, true
case "deepseek-v4-flash-search", "deepseek-v4-pro-search", "deepseek-v4-vision-search":
case "deepseek-v4-flash-search", "deepseek-v4-pro-search":
return !noThinking, true, true
default:
return false, false, false
@@ -81,7 +80,7 @@ func GetModelType(model string) (modelType string, ok bool) {
return "default", true
case "deepseek-v4-pro", "deepseek-v4-pro-search":
return "expert", true
case "deepseek-v4-vision", "deepseek-v4-vision-search":
case "deepseek-v4-vision":
return "vision", true
default:
return "", false
@@ -359,8 +358,6 @@ func resolveCanonicalModel(aliases map[string]string, model string) (string, boo
useSearch := strings.Contains(model, "search")
switch {
case useVision && useSearch:
return "deepseek-v4-vision-search", true
case useVision:
return "deepseek-v4-vision", true
case useReasoner && useSearch:

View File

@@ -30,9 +30,29 @@ func ResolvePath(envKey, defaultRel string) string {
}
func ConfigPath() string {
if strings.TrimSpace(os.Getenv("DS2API_CONFIG_PATH")) == "" && BaseDir() == "/app" {
return containerDefaultConfigPath()
}
return ResolvePath("DS2API_CONFIG_PATH", "config.json")
}
func containerDefaultConfigPath() string {
// Container images run as non-root by default. Only use /data when mounted/provisioned.
// Otherwise keep /app/config.json so admin-side save does not fail on MkdirAll("/data").
if st, err := os.Stat("/data"); err == nil && st.IsDir() {
return "/data/config.json"
}
return "/app/config.json"
}
func legacyContainerConfigPath() string {
return "/app/config.json"
}
func shouldTryLegacyContainerConfigPath() bool {
return strings.TrimSpace(os.Getenv("DS2API_CONFIG_PATH")) == "" && BaseDir() == "/app"
}
func RawStreamSampleRoot() string {
return ResolvePath("DS2API_RAW_STREAM_SAMPLE_ROOT", "tests/raw_stream_samples")
}

View File

@@ -0,0 +1,28 @@
package config
import (
"os"
"testing"
)
func TestContainerDefaultConfigPath(t *testing.T) {
t.Run("fallback to /app when /data is missing", func(t *testing.T) {
// This test environment does not guarantee a writable/mounted /data.
// If /data is absent we must keep /app fallback to avoid persistence failures.
if _, err := os.Stat("/data"); err == nil {
t.Skip("/data exists in this environment; cannot validate missing-/data fallback")
}
if got := containerDefaultConfigPath(); got != "/app/config.json" {
t.Fatalf("containerDefaultConfigPath() = %q, want %q", got, "/app/config.json")
}
})
t.Run("prefer /data when /data directory exists", func(t *testing.T) {
if _, err := os.Stat("/data"); err != nil {
t.Skip("/data does not exist in this environment")
}
if got := containerDefaultConfigPath(); got != "/data/config.json" {
t.Fatalf("containerDefaultConfigPath() = %q, want %q", got, "/data/config.json")
}
})
}

View File

@@ -87,12 +87,17 @@ func loadConfig() (Config, bool, error) {
}
return cfg, true, err
}
cfg, err := loadConfigFromFile(ConfigPath())
if err != nil {
if shouldTryLegacyContainerConfigPath() {
legacyPath := legacyContainerConfigPath()
if legacyCfg, legacyErr := loadConfigFromFile(legacyPath); legacyErr == nil {
Logger.Info("[config] loaded legacy container config path", "path", legacyPath)
return legacyCfg, false, nil
}
}
if IsVercel() {
// Vercel one-click deploy may start without a writable/present config file.
// Keep an in-memory config so users can bootstrap via WebUI then sync env.
// Vercel may start without writable/present config; keep in-memory bootstrap config.
return Config{}, true, nil
}
return Config{}, false, err

View File

@@ -7,6 +7,7 @@ import (
dsprotocol "ds2api/internal/deepseek/protocol"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
@@ -27,7 +28,7 @@ type continueState struct {
}
// wrapCompletionWithAutoContinue wraps the completion response body so that
// if the upstream indicates the response is incomplete (WIP / INCOMPLETE /
// if the upstream indicates the response is incomplete (INCOMPLETE /
// AUTO_CONTINUE), ds2api will automatically call the DeepSeek continue
// endpoint and splice the continuation SSE stream onto the original.
// The caller sees a single, seamless SSE stream.
@@ -176,12 +177,12 @@ func (s *continueState) observe(data string) {
}
// Path-based status: {"p": "response/status", "v": "FINISHED"}
if p, _ := chunk["p"].(string); p == "response/status" {
if status, _ := chunk["v"].(string); status != "" {
s.lastStatus = strings.TrimSpace(status)
if strings.EqualFold(s.lastStatus, "FINISHED") {
s.finished = true
}
}
s.setStatus(asString(chunk["v"]))
}
if p, _ := chunk["p"].(string); p == "response" {
s.observeBatchPatches("response", chunk["v"])
} else {
s.observeBatchPatches("", chunk["v"])
}
// Nested v.response
v, _ := chunk["v"].(map[string]any)
@@ -189,12 +190,7 @@ func (s *continueState) observe(data string) {
if id := intFrom(response["message_id"]); id > 0 {
s.responseMessageID = id
}
if status, _ := response["status"].(string); status != "" {
s.lastStatus = strings.TrimSpace(status)
if strings.EqualFold(s.lastStatus, "FINISHED") {
s.finished = true
}
}
s.setStatus(asString(response["status"]))
if autoContinue, ok := response["auto_continue"].(bool); ok && autoContinue {
s.lastStatus = "AUTO_CONTINUE"
}
@@ -205,18 +201,56 @@ func (s *continueState) observe(data string) {
if id := intFrom(response["message_id"]); id > 0 {
s.responseMessageID = id
}
if status, _ := response["status"].(string); status != "" {
s.lastStatus = strings.TrimSpace(status)
if strings.EqualFold(s.lastStatus, "FINISHED") {
s.finished = true
}
}
s.setStatus(asString(response["status"]))
}
}
}
// shouldContinue returns true when the upstream indicates the response is
// not yet finished and we have enough information to issue a continue request.
func (s *continueState) observeBatchPatches(parentPath string, raw any) {
if s == nil {
return
}
patches, ok := raw.([]any)
if !ok {
return
}
for _, patch := range patches {
m, ok := patch.(map[string]any)
if !ok {
continue
}
path := strings.TrimSpace(asString(m["p"]))
if path == "" {
continue
}
fullPath := path
if parent := strings.Trim(strings.TrimSpace(parentPath), "/"); parent != "" && !strings.Contains(path, "/") {
fullPath = parent + "/" + path
}
switch strings.Trim(strings.TrimSpace(fullPath), "/") {
case "response/status", "status", "response/quasi_status", "quasi_status":
s.setStatus(asString(m["v"]))
}
}
}
func (s *continueState) setStatus(status string) {
if s == nil {
return
}
normalized := strings.TrimSpace(status)
if normalized == "" {
return
}
s.lastStatus = normalized
if strings.EqualFold(normalized, "FINISHED") || strings.EqualFold(normalized, "CONTENT_FILTER") {
s.finished = true
}
}
// shouldContinue returns true when the upstream explicitly indicates the
// response is incomplete and we have enough information to issue a continue
// request. Plain WIP is not sufficient because normal streams begin in WIP.
func (s *continueState) shouldContinue() bool {
if s == nil {
return false
@@ -225,7 +259,7 @@ func (s *continueState) shouldContinue() bool {
return false
}
switch strings.ToUpper(strings.TrimSpace(s.lastStatus)) {
case "WIP", "INCOMPLETE", "AUTO_CONTINUE":
case "INCOMPLETE", "AUTO_CONTINUE":
return true
default:
return false
@@ -241,3 +275,19 @@ func (s *continueState) prepareForNextRound() {
s.finished = false
s.lastStatus = ""
}
func asString(v any) string {
if v == nil {
return ""
}
switch x := v.(type) {
case string:
return x
default:
s := strings.TrimSpace(strings.ReplaceAll(strings.TrimSpace(fmt.Sprint(v)), "\u0000", ""))
if s == "<nil>" {
return ""
}
return s
}
}

View File

@@ -8,6 +8,7 @@ import (
"io"
"net/http"
"strings"
"sync/atomic"
"testing"
"ds2api/internal/auth"
@@ -124,6 +125,90 @@ func TestCallCompletionAutoContinueThreadsPowHeader(t *testing.T) {
}
}
func TestAutoContinueDoesNotTriggerOnPlainWIPWithoutExplicitContinuationSignal(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"WIP","auto_continue":false}}}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return nil, errors.New("continue should not have been called")
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 0 {
t.Fatalf("expected no continue calls, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte(`"status":"WIP"`)) || !bytes.Contains(out, []byte(`data: [DONE]`)) {
t.Fatalf("expected original body to pass through unchanged, got=%s", string(out))
}
}
func TestAutoContinueTriggersOnResponseBatchQuasiStatusIncomplete(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"WIP","auto_continue":false}}}`,
`data: {"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":2413},{"p":"quasi_status","v":"INCOMPLETE"}]}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`data: {"response_message_id":322,"p":"response/status","v":"FINISHED"}` + "\n" +
`data: [DONE]` + "\n",
)),
}, nil
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 1 {
t.Fatalf("expected exactly one continue call, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte(`"quasi_status","v":"INCOMPLETE"`)) || !bytes.Contains(out, []byte(`"v":"FINISHED"`)) {
t.Fatalf("expected continued output to include initial and final rounds, got=%s", string(out))
}
}
func TestAutoContinueDoesNotTriggerWhenResponseBatchQuasiStatusFinished(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"WIP","auto_continue":false}}}`,
`data: {"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":2413},{"p":"quasi_status","v":"FINISHED"}]}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return nil, errors.New("continue should not have been called")
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 0 {
t.Fatalf("expected no continue calls, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte(`"quasi_status","v":"FINISHED"`)) || !bytes.Contains(out, []byte(`data: [DONE]`)) {
t.Fatalf("expected original finished body to pass through unchanged, got=%s", string(out))
}
}
type failingOrCompletionDoer struct {
completionResp *http.Response
}
@@ -134,3 +219,33 @@ func (d failingOrCompletionDoer) Do(req *http.Request) (*http.Response, error) {
}
return nil, errors.New("forced stream failure")
}
func TestAutoContinuePreservesIncompleteStateWhenNextChunkOmitsStatus(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"INCOMPLETE"}}}`,
`data: {"p":"response/content","v":{"text":"continued"}}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`data: {"response_message_id":322,"p":"response/status","v":"FINISHED"}` + "\n" +
`data: [DONE]` + "\n",
)),
}, nil
})
defer func() { _ = body.Close() }()
_, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 1 {
t.Fatalf("expected exactly one continue call, got %d", continueCalls.Load())
}
}

View File

@@ -2,7 +2,7 @@
"client": {
"name": "DeepSeek",
"platform": "android",
"version": "2.0.1",
"version": "2.0.3",
"android_api_level": "35",
"locale": "zh_CN"
},

View File

@@ -10,6 +10,8 @@ import (
"sync"
"time"
"ds2api/internal/util"
"github.com/google/uuid"
)
@@ -194,7 +196,8 @@ func (c *captureBody) append(chunk string) {
}
remain := maxLen - current
if len(chunk) > remain {
c.buf.WriteString(chunk[:remain])
truncated, _ := util.TruncateUTF8Bytes(chunk, remain)
c.buf.WriteString(truncated)
c.truncated = true
return
}

View File

@@ -4,6 +4,7 @@ import (
"io"
"strings"
"testing"
"unicode/utf8"
)
func TestNewFromEnvDefaults(t *testing.T) {
@@ -82,3 +83,28 @@ func TestWrapBodyTruncatesByLimit(t *testing.T) {
t.Fatalf("expected account id, got %q", items[0].AccountID)
}
}
func TestWrapBodyTruncatesUTF8WithoutBreakingRune(t *testing.T) {
s := &Store{enabled: true, limit: 5, maxBodyBytes: 5}
session := s.Start("test", "http://x", "acc1", map[string]any{"x": 1})
if session == nil {
t.Fatal("expected session")
}
rc := session.WrapBody(io.NopCloser(strings.NewReader("😀xy")), 200)
_, _ = io.ReadAll(rc)
_ = rc.Close()
items := s.Snapshot()
if len(items) != 1 {
t.Fatalf("expected 1 item, got %d", len(items))
}
if !utf8.ValidString(items[0].ResponseBody) {
t.Fatalf("expected valid utf-8 response body, got %q", items[0].ResponseBody)
}
if items[0].ResponseBody != "😀x" {
t.Fatalf("expected rune-safe truncation, got %q", items[0].ResponseBody)
}
if !items[0].ResponseTruncated {
t.Fatal("expected truncated flag true")
}
}

View File

@@ -5,6 +5,7 @@ import (
"fmt"
"time"
"ds2api/internal/prompt"
"ds2api/internal/util"
)
@@ -43,8 +44,23 @@ func BuildMessageResponse(messageID, model string, normalizedMessages []any, fin
"stop_reason": stopReason,
"stop_sequence": nil,
"usage": map[string]any{
"input_tokens": util.EstimateTokens(fmt.Sprintf("%v", normalizedMessages)),
"output_tokens": util.EstimateTokens(finalThinking) + util.EstimateTokens(finalText),
"input_tokens": util.CountPromptTokens(prompt.MessagesPrepareWithThinking(claudeMessageMaps(normalizedMessages), false), model),
"output_tokens": util.CountOutputTokens(finalThinking, model) + util.CountOutputTokens(finalText, model),
},
}
}
func claudeMessageMaps(messages []any) []map[string]any {
if len(messages) == 0 {
return nil
}
out := make([]map[string]any, 0, len(messages))
for _, item := range messages {
msg, ok := item.(map[string]any)
if !ok {
continue
}
out = append(out, msg)
}
return out
}

View File

@@ -29,7 +29,7 @@ func BuildChatCompletionWithToolCalls(completionID, model, finalPrompt, finalThi
"created": time.Now().Unix(),
"model": model,
"choices": []map[string]any{{"index": 0, "message": messageObj, "finish_reason": finishReason}},
"usage": BuildChatUsage(finalPrompt, finalThinking, finalText),
"usage": BuildChatUsageForModel(model, finalPrompt, finalThinking, finalText, 0),
}
}

View File

@@ -70,7 +70,7 @@ func BuildResponseObjectFromItems(responseID, model, finalPrompt, finalThinking,
"model": model,
"output": output,
"output_text": outputText,
"usage": BuildResponsesUsage(finalPrompt, finalThinking, finalText),
"usage": BuildResponsesUsageForModel(model, finalPrompt, finalThinking, finalText, 0),
}
}

View File

@@ -6,6 +6,7 @@ import (
"testing"
"ds2api/internal/toolcall"
"ds2api/internal/util"
)
func TestBuildResponseObjectKeepsFencedToolPayloadAsText(t *testing.T) {
@@ -177,3 +178,17 @@ func TestBuildResponseObjectWithToolCallsCoercesSchemaDeclaredStringArguments(t
t.Fatalf("expected response content stringified by schema, got %#v", args["content"])
}
}
func TestBuildChatUsageForModelUsesConservativePromptCount(t *testing.T) {
prompt := strings.Repeat("上下文token ", 40)
usage := BuildChatUsageForModel("deepseek-v4-flash", prompt, "", "ok", 0)
promptTokens, _ := usage["prompt_tokens"].(int)
if promptTokens <= util.EstimateTokens(prompt) {
t.Fatalf("expected conservative prompt token count > rough estimate, got=%d estimate=%d", promptTokens, util.EstimateTokens(prompt))
}
totalTokens, _ := usage["total_tokens"].(int)
completionTokens, _ := usage["completion_tokens"].(int)
if totalTokens != promptTokens+completionTokens {
t.Fatalf("expected total tokens to add up, got usage=%#v", usage)
}
}

View File

@@ -2,10 +2,10 @@ package openai
import "ds2api/internal/util"
func BuildChatUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
func BuildChatUsageForModel(model, finalPrompt, finalThinking, finalText string, refFileTokens int) map[string]any {
promptTokens := util.CountPromptTokens(finalPrompt, model) + refFileTokens
reasoningTokens := util.CountOutputTokens(finalThinking, model)
completionTokens := util.CountOutputTokens(finalText, model)
return map[string]any{
"prompt_tokens": promptTokens,
"completion_tokens": reasoningTokens + completionTokens,
@@ -16,13 +16,21 @@ func BuildChatUsage(finalPrompt, finalThinking, finalText string) map[string]any
}
}
func BuildResponsesUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
func BuildChatUsage(finalPrompt, finalThinking, finalText string) map[string]any {
return BuildChatUsageForModel("", finalPrompt, finalThinking, finalText, 0)
}
func BuildResponsesUsageForModel(model, finalPrompt, finalThinking, finalText string, refFileTokens int) map[string]any {
promptTokens := util.CountPromptTokens(finalPrompt, model) + refFileTokens
reasoningTokens := util.CountOutputTokens(finalThinking, model)
completionTokens := util.CountOutputTokens(finalText, model)
return map[string]any{
"input_tokens": promptTokens,
"output_tokens": reasoningTokens + completionTokens,
"total_tokens": promptTokens + reasoningTokens + completionTokens,
}
}
func BuildResponsesUsage(finalPrompt, finalThinking, finalText string) map[string]any {
return BuildResponsesUsageForModel("", finalPrompt, finalThinking, finalText, 0)
}

View File

@@ -107,6 +107,7 @@ func (h *Handler) testAccount(ctx context.Context, acc config.Account, model, me
"model": model,
"session_count": 0,
"config_writable": !h.Store.IsEnvBacked(),
"config_warning": "",
}
defer func() {
status := "failed"
@@ -121,8 +122,7 @@ func (h *Handler) testAccount(ctx context.Context, acc config.Account, model, me
return result
}
if err := h.Store.UpdateAccountToken(acc.Identifier(), token); err != nil {
result["message"] = "登录成功但写入运行时 token 失败: " + err.Error()
return result
result["config_warning"] = "登录成功但 token 持久化失败(仅保存在内存,重启后会丢失): " + err.Error()
}
authCtx := &authn.RequestAuth{UseConfigToken: false, DeepSeekToken: token, AccountID: identifier, Account: acc}
proxyCtx := authn.WithAuth(ctx, authCtx)
@@ -136,8 +136,7 @@ func (h *Handler) testAccount(ctx context.Context, acc config.Account, model, me
token = newToken
authCtx.DeepSeekToken = token
if err := h.Store.UpdateAccountToken(acc.Identifier(), token); err != nil {
result["message"] = "刷新 token 成功但写入运行时 token 失败: " + err.Error()
return result
result["config_warning"] = "刷新 token 成功但 token 持久化失败(仅保存在内存,重启后会丢失): " + err.Error()
}
sessionID, err = h.DS.CreateSession(proxyCtx, authCtx, 1)
if err != nil {
@@ -155,6 +154,9 @@ func (h *Handler) testAccount(ctx context.Context, acc config.Account, model, me
if strings.TrimSpace(message) == "" {
result["success"] = true
result["message"] = "Token 刷新成功(登录与会话创建成功)"
if warning, _ := result["config_warning"].(string); strings.TrimSpace(warning) != "" {
result["message"] = result["message"].(string) + "" + warning
}
result["response_time"] = int(time.Since(start).Milliseconds())
return result
}

View File

@@ -15,6 +15,7 @@ import (
"ds2api/internal/devcapture"
adminshared "ds2api/internal/httpapi/admin/shared"
"ds2api/internal/rawsample"
"ds2api/internal/util"
)
type captureChain struct {
@@ -479,10 +480,13 @@ func previewCaptureChainResponse(chain captureChain) string {
func previewText(text string, limit int) string {
text = strings.TrimSpace(text)
if limit <= 0 || len(text) <= limit {
if limit <= 0 {
return text
}
return text[:limit] + "..."
if truncated, ok := util.TruncateRunes(text, limit); ok {
return truncated + "..."
}
return text
}
func captureChainHasTruncatedResponse(chain captureChain) bool {

View File

@@ -10,6 +10,7 @@ import (
"path/filepath"
"strings"
"testing"
"unicode/utf8"
"ds2api/internal/devcapture"
)
@@ -231,6 +232,16 @@ func TestCombineCaptureBodiesPreservesOrderAndSeparators(t *testing.T) {
}
}
func TestPreviewTextPreservesUTF8MB4Characters(t *testing.T) {
preview := previewText(strings.Repeat("😀", 281), 280)
if !utf8.ValidString(preview) {
t.Fatalf("expected valid utf-8 preview, got %q", preview)
}
if preview != strings.Repeat("😀", 280)+"..." {
t.Fatalf("unexpected preview: %q", preview)
}
}
func TestQueryRawSampleCapturesGroupsBySessionAndMatchesQuestion(t *testing.T) {
devcapture.Global().Clear()
defer devcapture.Global().Clear()

View File

@@ -1,6 +1,7 @@
package claude
import (
"ds2api/internal/toolcall"
"fmt"
"strings"
)
@@ -31,30 +32,9 @@ func extractClaudeToolNames(tools []any) []string {
}
func extractClaudeToolMeta(m map[string]any) (string, string, any) {
name, _ := m["name"].(string)
desc, _ := m["description"].(string)
schemaObj := m["input_schema"]
if schemaObj == nil {
schemaObj = m["parameters"]
}
if fn, ok := m["function"].(map[string]any); ok {
if strings.TrimSpace(name) == "" {
name, _ = fn["name"].(string)
}
if strings.TrimSpace(desc) == "" {
desc, _ = fn["description"].(string)
}
if schemaObj == nil {
if v, ok := fn["input_schema"]; ok {
schemaObj = v
}
}
if schemaObj == nil {
if v, ok := fn["parameters"]; ok {
schemaObj = v
}
}
name, desc, schemaObj := toolcall.ExtractToolMeta(m)
if strings.TrimSpace(desc) == "" {
desc = "No description available"
}
return strings.TrimSpace(name), strings.TrimSpace(desc), schemaObj
}

View File

@@ -177,7 +177,7 @@ func stripClaudeThinkingBlocks(raw []byte) []byte {
return out
}
func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Request, resp *http.Response, model string, messages []any, thinkingEnabled, searchEnabled bool, toolNames []string) {
func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Request, resp *http.Response, model string, messages []any, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -205,6 +205,8 @@ func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Requ
searchEnabled,
h.compatStripReferenceMarkers(),
toolNames,
toolsRaw,
buildClaudePromptTokenText(messages, thinkingEnabled),
)
streamRuntime.sendMessageStart()

View File

@@ -81,7 +81,7 @@ func TestHandleClaudeStreamRealtimeTextIncrementsWithEventHeaders(t *testing.T)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil, nil)
body := rec.Body.String()
if !strings.Contains(body, "event: message_start") {
@@ -122,7 +122,7 @@ func TestHandleClaudeStreamRealtimeThinkingDelta(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, true, false, nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, true, false, nil, nil)
frames := parseClaudeFrames(t, rec.Body.String())
foundThinkingDelta := false
@@ -149,7 +149,7 @@ func TestHandleClaudeStreamRealtimeSkipsThinkingFallbackWhenFinalTextExists(t *t
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, true, false, []string{"search"})
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, true, false, []string{"search"}, nil)
frames := parseClaudeFrames(t, rec.Body.String())
for _, f := range findClaudeFrames(frames, "content_block_start") {
@@ -180,7 +180,7 @@ func TestHandleClaudeStreamRealtimeUpstreamErrorEvent(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil, nil)
frames := parseClaudeFrames(t, rec.Body.String())
errFrames := findClaudeFrames(frames, "error")
@@ -217,7 +217,7 @@ func TestHandleClaudeStreamRealtimePingEvent(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil, nil)
frames := parseClaudeFrames(t, rec.Body.String())
if len(findClaudeFrames(frames, "ping")) == 0 {
@@ -271,7 +271,7 @@ func TestHandleClaudeStreamRealtimeToolSafetyAcrossStructuredFormats(t *testing.
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, false, false, []string{"Bash"})
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, false, false, []string{"Bash"}, nil)
frames := parseClaudeFrames(t, rec.Body.String())
foundToolUse := false
@@ -299,7 +299,7 @@ func TestHandleClaudeStreamRealtimeDetectsToolUseWithLeadingProse(t *testing.T)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, false, false, []string{"write_file"})
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, false, false, []string{"write_file"}, nil)
frames := parseClaudeFrames(t, rec.Body.String())
foundToolUse := false
@@ -333,7 +333,7 @@ func TestHandleClaudeStreamRealtimeIgnoresUnclosedFencedToolExample(t *testing.T
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "show example only"}}, false, false, []string{"Bash"})
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "show example only"}}, false, false, []string{"Bash"}, nil)
frames := parseClaudeFrames(t, rec.Body.String())
foundToolUse := false
@@ -365,3 +365,48 @@ func TestHandleClaudeStreamRealtimeIgnoresUnclosedFencedToolExample(t *testing.T
func TestHandleClaudeStreamRealtimePromotesUnclosedFencedToolExample(t *testing.T) {
TestHandleClaudeStreamRealtimeIgnoresUnclosedFencedToolExample(t)
}
func TestHandleClaudeStreamRealtimeNormalizesToolInputBySchema(t *testing.T) {
h := &Handler{}
resp := makeClaudeSSEHTTPResponse(
`data: {"p":"response/content","v":"<tool_calls><invoke name=\"Write\">{\"input\":{\"content\":{\"message\":\"hi\"},\"taskId\":1}}</invoke></tool_calls>"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
toolsRaw := []any{
map[string]any{
"name": "Write",
"inputSchema": map[string]any{
"type": "object",
"properties": map[string]any{
"content": map[string]any{"type": "string"},
"taskId": map[string]any{"type": "string"},
},
},
},
}
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "write"}}, false, false, []string{"Write"}, toolsRaw)
frames := parseClaudeFrames(t, rec.Body.String())
for _, f := range findClaudeFrames(frames, "content_block_delta") {
delta, _ := f.Payload["delta"].(map[string]any)
if delta["type"] != "input_json_delta" {
continue
}
partial := asString(delta["partial_json"])
var args map[string]any
if err := json.Unmarshal([]byte(partial), &args); err != nil {
t.Fatalf("decode partial_json failed: %v payload=%s", err, partial)
}
if args["content"] != `{"message":"hi"}` {
t.Fatalf("expected content normalized to string, got %#v", args["content"])
}
if args["taskId"] != "1" {
t.Fatalf("expected taskId normalized to string, got %#v", args["taskId"])
}
return
}
t.Fatalf("expected input_json_delta frame, body=%s", rec.Body.String())
}

View File

@@ -3,8 +3,6 @@ package claude
import (
"encoding/json"
"net/http"
"ds2api/internal/util"
)
func (h *Handler) CountTokens(w http.ResponseWriter, r *http.Request) {
@@ -26,26 +24,11 @@ func (h *Handler) CountTokens(w http.ResponseWriter, r *http.Request) {
writeClaudeError(w, http.StatusBadRequest, "Request must include 'model' and 'messages'.")
return
}
inputTokens := 0
if sys, ok := req["system"].(string); ok {
inputTokens += util.EstimateTokens(sys)
}
for _, item := range messages {
msg, ok := item.(map[string]any)
if !ok {
continue
}
inputTokens += 2
inputTokens += util.EstimateTokens(extractMessageContent(msg["content"]))
}
if tools, ok := req["tools"].([]any); ok {
for _, t := range tools {
b, _ := json.Marshal(t)
inputTokens += util.EstimateTokens(string(b))
}
}
if inputTokens < 1 {
inputTokens = 1
normalized, err := normalizeClaudeRequest(h.Store, req)
if err != nil {
writeClaudeError(w, http.StatusBadRequest, err.Error())
return
}
inputTokens := countClaudeInputTokens(normalized.Standard)
writeJSON(w, http.StatusOK, map[string]any{"input_tokens": inputTokens})
}

View File

@@ -0,0 +1,7 @@
package claude
import "ds2api/internal/prompt"
func buildClaudePromptTokenText(messages []any, thinkingEnabled bool) string {
return prompt.MessagesPrepareWithThinking(toMessageMaps(messages), thinkingEnabled)
}

View File

@@ -48,16 +48,18 @@ func normalizeClaudeRequest(store ConfigReader, req map[string]any) (claudeNorma
return claudeNormalizedRequest{
Standard: promptcompat.StandardRequest{
Surface: "anthropic_messages",
RequestedModel: strings.TrimSpace(model),
ResolvedModel: dsModel,
ResponseModel: strings.TrimSpace(model),
Messages: payload["messages"].([]any),
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
Surface: "anthropic_messages",
RequestedModel: strings.TrimSpace(model),
ResolvedModel: dsModel,
ResponseModel: strings.TrimSpace(model),
Messages: payload["messages"].([]any),
PromptTokenText: finalPrompt,
ToolsRaw: toolsRequested,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
},
NormalizedMessages: normalizedMessages,
}, nil

View File

@@ -32,11 +32,39 @@ func TestNormalizeClaudeRequest(t *testing.T) {
if len(norm.Standard.ToolNames) == 0 {
t.Fatalf("expected tool names")
}
if norm.Standard.ToolsRaw == nil {
t.Fatalf("expected ToolsRaw preserved for downstream normalization")
}
if norm.Standard.FinalPrompt == "" {
t.Fatalf("expected non-empty final prompt")
}
}
func TestNormalizeClaudeRequestSupportsCamelCaseInputSchemaPromptInjection(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{}`)
store := config.LoadStore()
req := map[string]any{
"model": "claude-sonnet-4-5",
"messages": []any{
map[string]any{"role": "user", "content": "hello"},
},
"tools": []any{
map[string]any{
"name": "todowrite",
"description": "Write todos",
"inputSchema": map[string]any{"type": "object", "properties": map[string]any{"todos": map[string]any{"type": "array"}}},
},
},
}
norm, err := normalizeClaudeRequest(store, req)
if err != nil {
t.Fatalf("normalize failed: %v", err)
}
if !containsStr(norm.Standard.FinalPrompt, `"type":"array"`) {
t.Fatalf("expected inputSchema to be injected into prompt, got=%q", norm.Standard.FinalPrompt)
}
}
func TestNormalizeClaudeRequestInjectsToolsIntoExistingSystemMessage(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{}`)
store := config.LoadStore()

View File

@@ -15,9 +15,11 @@ type claudeStreamRuntime struct {
rc *http.ResponseController
canFlush bool
model string
toolNames []string
messages []any
model string
toolNames []string
messages []any
toolsRaw any
promptTokenText string
thinkingEnabled bool
searchEnabled bool
@@ -47,6 +49,8 @@ func newClaudeStreamRuntime(
searchEnabled bool,
stripReferenceMarkers bool,
toolNames []string,
toolsRaw any,
promptTokenText string,
) *claudeStreamRuntime {
return &claudeStreamRuntime{
w: w,
@@ -59,6 +63,8 @@ func newClaudeStreamRuntime(
bufferToolContent: len(toolNames) > 0,
stripReferenceMarkers: stripReferenceMarkers,
toolNames: toolNames,
toolsRaw: toolsRaw,
promptTokenText: promptTokenText,
messageID: fmt.Sprintf("msg_%d", time.Now().UnixNano()),
thinkingBlockIndex: -1,
textBlockIndex: -1,

View File

@@ -42,7 +42,10 @@ func (s *claudeStreamRuntime) sendPing() {
}
func (s *claudeStreamRuntime) sendMessageStart() {
inputTokens := util.EstimateTokens(fmt.Sprintf("%v", s.messages))
inputTokens := countClaudeInputTokensFromText(s.promptTokenText, s.model)
if inputTokens == 0 {
inputTokens = util.CountPromptTokens(fmt.Sprintf("%v", s.messages), s.model)
}
s.send("message_start", map[string]any{
"type": "message_start",
"message": map[string]any{

View File

@@ -52,6 +52,7 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
detected = toolcall.ParseStandaloneToolCalls(finalThinking, s.toolNames)
}
if len(detected) > 0 {
detected = toolcall.NormalizeParsedToolCallsForSchemas(detected, s.toolsRaw)
stopReason = "tool_use"
for i, tc := range detected {
idx := s.nextBlockIndex + i
@@ -108,7 +109,7 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
}
}
outputTokens := util.EstimateTokens(finalThinking) + util.EstimateTokens(finalText)
outputTokens := util.CountOutputTokens(finalThinking, s.model) + util.CountOutputTokens(finalText, s.model)
s.send("message_delta", map[string]any{
"type": "message_delta",
"delta": map[string]any{

View File

@@ -0,0 +1,20 @@
package claude
import (
"strings"
"ds2api/internal/promptcompat"
"ds2api/internal/util"
)
func countClaudeInputTokens(stdReq promptcompat.StandardRequest) int {
promptText := stdReq.PromptTokenText
if strings.TrimSpace(promptText) == "" {
promptText = stdReq.FinalPrompt
}
return countClaudeInputTokensFromText(promptText, stdReq.ResolvedModel)
}
func countClaudeInputTokensFromText(promptText, model string) int {
return util.CountPromptTokens(promptText, model)
}

View File

@@ -36,16 +36,17 @@ func normalizeGeminiRequest(store ConfigReader, routeModel string, req map[strin
passThrough := collectGeminiPassThrough(req)
return promptcompat.StandardRequest{
Surface: "google_gemini",
RequestedModel: requestedModel,
ResolvedModel: resolvedModel,
ResponseModel: requestedModel,
Messages: messagesRaw,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: stream,
Thinking: thinkingEnabled,
Search: searchEnabled,
PassThrough: passThrough,
Surface: "google_gemini",
RequestedModel: requestedModel,
ResolvedModel: resolvedModel,
ResponseModel: requestedModel,
Messages: messagesRaw,
PromptTokenText: finalPrompt,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: stream,
Thinking: thinkingEnabled,
Search: searchEnabled,
PassThrough: passThrough,
}, nil
}

View File

@@ -227,7 +227,7 @@ func (h *Handler) handleNonStreamGenerateContent(w http.ResponseWriter, resp *ht
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, finalText string, toolNames []string) map[string]any {
parts := buildGeminiPartsFromFinal(finalText, finalThinking, toolNames)
usage := buildGeminiUsage(finalPrompt, finalThinking, finalText)
usage := buildGeminiUsage(model, finalPrompt, finalThinking, finalText)
return map[string]any{
"candidates": []map[string]any{
{
@@ -245,10 +245,10 @@ func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, final
}
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
func buildGeminiUsage(model, finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.CountPromptTokens(finalPrompt, model)
reasoningTokens := util.CountOutputTokens(finalThinking, model)
completionTokens := util.CountOutputTokens(finalText, model)
return map[string]any{
"promptTokenCount": promptTokens,
"candidatesTokenCount": reasoningTokens + completionTokens,

View File

@@ -194,6 +194,6 @@ func (s *geminiStreamRuntime) finalize() {
},
},
"modelVersion": s.model,
"usageMetadata": buildGeminiUsage(s.finalPrompt, finalThinking, finalText),
"usageMetadata": buildGeminiUsage(s.model, s.finalPrompt, finalThinking, finalText),
})
}

View File

@@ -126,6 +126,7 @@ func TestStartChatHistoryRecoversFromTransientWriteFailure(t *testing.T) {
session := startChatHistory(historyStore, req, a, stdReq)
if session == nil {
t.Fatalf("expected session even when initial persistence fails")
return
}
if session.disabled {
t.Fatalf("expected session to remain active after transient start failure")
@@ -194,7 +195,7 @@ func TestHandleStreamContextCancelledMarksHistoryStopped(t *testing.T) {
rec := httptest.NewRecorder()
resp := makeOpenAISSEHTTPResponse(`data: {"p":"response/content","v":"hello"}`, `data: [DONE]`)
h.handleStream(rec, req, resp, "cid-stop", "deepseek-v4-flash", "prompt", false, false, nil, nil, session)
h.handleStream(rec, req, resp, "cid-stop", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, session)
snapshot, err := historyStore.Snapshot()
if err != nil {
@@ -307,14 +308,14 @@ func TestChatCompletionsCurrentInputFilePersistsNeutralPrompt(t *testing.T) {
if err != nil {
t.Fatalf("expected detail item, got %v", err)
}
if full.HistoryText != "" {
t.Fatalf("expected current input file flow to leave history text empty, got %q", full.HistoryText)
}
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected current input upload to happen, got %d", len(ds.uploadCalls))
}
if ds.uploadCalls[0].Filename != "IGNORE.txt" {
t.Fatalf("expected IGNORE.txt upload, got %q", ds.uploadCalls[0].Filename)
if ds.uploadCalls[0].Filename != "history.txt" {
t.Fatalf("expected history.txt upload, got %q", ds.uploadCalls[0].Filename)
}
if full.HistoryText != string(ds.uploadCalls[0].Data) {
t.Fatalf("expected uploaded current input file to be persisted in history text")
}
if len(full.Messages) != 1 {
t.Fatalf("expected neutral prompt to be the only persisted message, got %#v", full.Messages)

View File

@@ -16,12 +16,13 @@ type chatStreamRuntime struct {
rc *http.ResponseController
canFlush bool
completionID string
created int64
model string
finalPrompt string
toolNames []string
toolsRaw any
completionID string
created int64
model string
finalPrompt string
refFileTokens int
toolNames []string
toolsRaw any
thinkingEnabled bool
searchEnabled bool
@@ -36,8 +37,10 @@ type chatStreamRuntime struct {
toolSieve toolstream.State
streamToolCallIDs map[int]string
streamToolNames map[int]string
rawThinking strings.Builder
thinking strings.Builder
toolDetectionThinking strings.Builder
rawText strings.Builder
text strings.Builder
responseMessageID int
@@ -141,7 +144,7 @@ func (s *chatStreamRuntime) finalize(finishReason string, deferEmptyOutput bool)
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
s.finalThinking = finalThinking
s.finalText = finalText
detected := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, s.toolNames)
detected := detectAssistantToolCalls(s.rawText.String(), finalText, s.rawThinking.String(), finalToolDetectionThinking, s.toolNames)
if len(detected.Calls) > 0 && !s.toolCallsDoneEmitted {
finishReason = "tool_calls"
delta := map[string]any{
@@ -186,7 +189,7 @@ func (s *chatStreamRuntime) finalize(finishReason string, deferEmptyOutput bool)
continue
}
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" {
if cleaned == "" || (s.searchEnabled && sse.IsCitation(cleaned)) {
continue
}
delta := map[string]any{
@@ -220,7 +223,7 @@ func (s *chatStreamRuntime) finalize(finishReason string, deferEmptyOutput bool)
s.sendFailedChunk(status, message, code)
return true
}
usage := openaifmt.BuildChatUsage(s.finalPrompt, finalThinking, finalText)
usage := openaifmt.BuildChatUsageForModel(s.model, s.finalPrompt, finalThinking, finalText, s.refFileTokens)
s.finalFinishReason = finishReason
s.finalUsage = usage
s.sendChunk(openaifmt.BuildChatStreamChunk(
@@ -263,21 +266,22 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
}
}
for _, p := range parsed.Parts {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
if cleanedText == "" {
continue
}
contentSeen = true
delta := map[string]any{}
if !s.firstChunkSent {
delta["role"] = "assistant"
s.firstChunkSent = true
}
if p.Type == "thinking" {
rawTrimmed := sse.TrimContinuationOverlap(s.rawThinking.String(), p.Text)
if rawTrimmed != "" {
s.rawThinking.WriteString(rawTrimmed)
contentSeen = true
}
if s.thinkingEnabled {
cleanedText := cleanVisibleOutput(rawTrimmed, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
@@ -286,15 +290,27 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
delta["reasoning_content"] = trimmed
}
} else {
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
rawTrimmed := sse.TrimContinuationOverlap(s.rawText.String(), p.Text)
if rawTrimmed == "" {
continue
}
s.text.WriteString(trimmed)
s.rawText.WriteString(rawTrimmed)
contentSeen = true
cleanedText := cleanVisibleOutput(rawTrimmed, s.stripReferenceMarkers)
if s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed != "" {
s.text.WriteString(trimmed)
}
if !s.bufferToolContent {
if trimmed == "" {
continue
}
delta["content"] = trimmed
} else {
events := toolstream.ProcessChunk(&s.toolSieve, trimmed, s.toolNames)
events := toolstream.ProcessChunk(&s.toolSieve, rawTrimmed, s.toolNames)
for _, evt := range events {
if len(evt.ToolCallDeltas) > 0 {
if !s.emitEarlyToolDeltas {
@@ -335,7 +351,7 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
}
if evt.Content != "" {
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" {
if cleaned == "" || (s.searchEnabled && sse.IsCitation(cleaned)) {
continue
}
contentDelta := map[string]any{

View File

@@ -16,6 +16,8 @@ import (
)
type chatNonStreamResult struct {
rawThinking string
rawText string
thinking string
toolDetectionThinking string
text string
@@ -26,11 +28,12 @@ type chatNonStreamResult struct {
responseMessageID int
}
func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
attempts := 0
currentResp := resp
usagePrompt := finalPrompt
accumulatedThinking := ""
accumulatedRawThinking := ""
accumulatedToolDetectionThinking := ""
for {
result, ok := h.collectChatNonStreamAttempt(w, currentResp, completionID, model, usagePrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw)
@@ -38,15 +41,17 @@ func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Co
return
}
accumulatedThinking += sse.TrimContinuationOverlap(accumulatedThinking, result.thinking)
accumulatedRawThinking += sse.TrimContinuationOverlap(accumulatedRawThinking, result.rawThinking)
accumulatedToolDetectionThinking += sse.TrimContinuationOverlap(accumulatedToolDetectionThinking, result.toolDetectionThinking)
result.thinking = accumulatedThinking
result.rawThinking = accumulatedRawThinking
result.toolDetectionThinking = accumulatedToolDetectionThinking
detected := detectAssistantToolCalls(result.text, result.thinking, result.toolDetectionThinking, toolNames)
detected := detectAssistantToolCalls(result.rawText, result.text, result.rawThinking, result.toolDetectionThinking, toolNames)
result.detectedCalls = len(detected.Calls)
result.body = openaifmt.BuildChatCompletionWithToolCalls(completionID, model, usagePrompt, result.thinking, result.text, detected.Calls, toolsRaw)
result.finishReason = chatFinishReason(result.body)
if !shouldRetryChatNonStream(result, attempts) {
h.finishChatNonStreamResult(w, result, attempts, usagePrompt, historySession)
h.finishChatNonStreamResult(w, result, attempts, usagePrompt, refFileTokens, historySession)
return
}
@@ -67,7 +72,7 @@ func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Co
config.Logger.Warn("[openai_empty_retry] retry request failed", "surface", "chat.completions", "stream", false, "retry_attempt", attempts, "error", err)
return
}
usagePrompt = usagePromptWithEmptyOutputRetry(finalPrompt, attempts)
usagePrompt = usagePromptWithEmptyOutputRetry(usagePrompt, attempts)
currentResp = nextResp
}
}
@@ -82,16 +87,17 @@ func (h *Handler) collectChatNonStreamAttempt(w http.ResponseWriter, resp *http.
result := sse.CollectStream(resp, thinkingEnabled, true)
stripReferenceMarkers := h.compatStripReferenceMarkers()
finalThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
finalToolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
finalText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
finalText = replaceCitationMarkersWithLinks(finalText, result.CitationLinks)
}
detected := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, toolNames)
detected := detectAssistantToolCalls(result.Text, finalText, result.Thinking, result.ToolDetectionThinking, toolNames)
respBody := openaifmt.BuildChatCompletionWithToolCalls(completionID, model, usagePrompt, finalThinking, finalText, detected.Calls, toolsRaw)
return chatNonStreamResult{
rawThinking: result.Thinking,
rawText: result.Text,
thinking: finalThinking,
toolDetectionThinking: finalToolDetectionThinking,
toolDetectionThinking: result.ToolDetectionThinking,
text: finalText,
contentFilter: result.ContentFilter,
detectedCalls: len(detected.Calls),
@@ -101,7 +107,7 @@ func (h *Handler) collectChatNonStreamAttempt(w http.ResponseWriter, resp *http.
}, true
}
func (h *Handler) finishChatNonStreamResult(w http.ResponseWriter, result chatNonStreamResult, attempts int, usagePrompt string, historySession *chatHistorySession) {
func (h *Handler) finishChatNonStreamResult(w http.ResponseWriter, result chatNonStreamResult, attempts int, usagePrompt string, refFileTokens int, historySession *chatHistorySession) {
if result.detectedCalls == 0 && shouldWriteUpstreamEmptyOutputError(result.text) {
status, message, code := upstreamEmptyOutputDetail(result.contentFilter, result.text, result.thinking)
if historySession != nil {
@@ -112,7 +118,7 @@ func (h *Handler) finishChatNonStreamResult(w http.ResponseWriter, result chatNo
return
}
if historySession != nil {
historySession.success(http.StatusOK, result.thinking, result.text, result.finishReason, openaifmt.BuildChatUsage(usagePrompt, result.thinking, result.text))
historySession.success(http.StatusOK, result.thinking, result.text, result.finishReason, openaifmt.BuildChatUsageForModel("", usagePrompt, result.thinking, result.text, refFileTokens))
}
writeJSON(w, http.StatusOK, result.body)
source := "first_attempt"
@@ -139,8 +145,8 @@ func shouldRetryChatNonStream(result chatNonStreamResult, attempts int) bool {
strings.TrimSpace(result.text) == ""
}
func (h *Handler) handleStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
streamRuntime, initialType, ok := h.prepareChatStreamRuntime(w, resp, completionID, model, finalPrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw, historySession)
func (h *Handler) handleStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
streamRuntime, initialType, ok := h.prepareChatStreamRuntime(w, resp, completionID, model, finalPrompt, refFileTokens, thinkingEnabled, searchEnabled, toolNames, toolsRaw, historySession)
if !ok {
return
}
@@ -182,7 +188,7 @@ func (h *Handler) handleStreamWithRetry(w http.ResponseWriter, r *http.Request,
}
}
func (h *Handler) prepareChatStreamRuntime(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) (*chatStreamRuntime, string, bool) {
func (h *Handler) prepareChatStreamRuntime(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) (*chatStreamRuntime, string, bool) {
if resp.StatusCode != http.StatusOK {
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
@@ -210,6 +216,7 @@ func (h *Handler) prepareChatStreamRuntime(w http.ResponseWriter, resp *http.Res
thinkingEnabled, searchEnabled, h.compatStripReferenceMarkers(), toolNames, toolsRaw,
len(toolNames) > 0, h.toolcallFeatureMatchEnabled() && h.toolcallEarlyEmitHighConfidence(),
)
streamRuntime.refFileTokens = refFileTokens
return streamRuntime, initialType, true
}

View File

@@ -148,6 +148,6 @@ func formatFinalStreamToolCallsWithStableIDs(calls []toolcall.ParsedToolCall, id
return shared.FormatFinalStreamToolCallsWithStableIDs(calls, ids, toolsRaw)
}
func detectAssistantToolCalls(text, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(text, exposedThinking, detectionThinking, toolNames)
func detectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking, toolNames)
}

View File

@@ -108,11 +108,12 @@ func (h *Handler) ChatCompletions(w http.ResponseWriter, r *http.Request) {
writeOpenAIError(w, http.StatusInternalServerError, "Failed to get completion.")
return
}
refFileTokens := stdReq.RefFileTokens
if stdReq.Stream {
h.handleStreamWithRetry(w, r, a, resp, payload, pow, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, historySession)
h.handleStreamWithRetry(w, r, a, resp, payload, pow, sessionID, stdReq.ResponseModel, stdReq.PromptTokenText, refFileTokens, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, historySession)
return
}
h.handleNonStreamWithRetry(w, r.Context(), a, resp, payload, pow, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, historySession)
h.handleNonStreamWithRetry(w, r.Context(), a, resp, payload, pow, sessionID, stdReq.ResponseModel, stdReq.PromptTokenText, refFileTokens, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, historySession)
}
func (h *Handler) autoDeleteRemoteSession(ctx context.Context, a *auth.RequestAuth, sessionID string) {
@@ -148,7 +149,7 @@ func (h *Handler) autoDeleteRemoteSession(ctx context.Context, a *auth.RequestAu
}
}
func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
if resp.StatusCode != http.StatusOK {
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
@@ -162,12 +163,11 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, co
stripReferenceMarkers := h.compatStripReferenceMarkers()
finalThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
finalToolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
finalText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
finalText = replaceCitationMarkersWithLinks(finalText, result.CitationLinks)
}
detected := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, toolNames)
detected := detectAssistantToolCalls(result.Text, finalText, result.Thinking, result.ToolDetectionThinking, toolNames)
if shouldWriteUpstreamEmptyOutputError(finalText) && len(detected.Calls) == 0 {
status, message, code := upstreamEmptyOutputDetail(result.ContentFilter, finalText, finalThinking)
if historySession != nil {
@@ -177,6 +177,9 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, co
return
}
respBody := openaifmt.BuildChatCompletionWithToolCalls(completionID, model, finalPrompt, finalThinking, finalText, detected.Calls, toolsRaw)
if refFileTokens > 0 {
addRefFileTokensToUsage(respBody, refFileTokens)
}
finishReason := "stop"
if choices, ok := respBody["choices"].([]map[string]any); ok && len(choices) > 0 {
if fr, _ := choices[0]["finish_reason"].(string); strings.TrimSpace(fr) != "" {
@@ -184,12 +187,12 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, co
}
}
if historySession != nil {
historySession.success(http.StatusOK, finalThinking, finalText, finishReason, openaifmt.BuildChatUsage(finalPrompt, finalThinking, finalText))
historySession.success(http.StatusOK, finalThinking, finalText, finishReason, openaifmt.BuildChatUsageForModel(model, finalPrompt, finalThinking, finalText, refFileTokens))
}
writeJSON(w, http.StatusOK, respBody)
}
func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *http.Response, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -234,6 +237,7 @@ func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *htt
bufferToolContent,
emitEarlyToolDeltas,
)
streamRuntime.refFileTokens = refFileTokens
streamengine.ConsumeSSE(streamengine.ConsumeConfig{
Context: r.Context(),

View File

@@ -93,7 +93,7 @@ func TestHandleNonStreamReturns429WhenUpstreamOutputEmpty(t *testing.T) {
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-empty", "deepseek-v4-flash", "prompt", false, false, nil, nil, nil)
h.handleNonStream(rec, resp, "cid-empty", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, nil)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected status 429 for empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -112,7 +112,7 @@ func TestHandleNonStreamReturnsContentFilterErrorWhenUpstreamFilteredWithoutOutp
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-empty-filtered", "deepseek-v4-flash", "prompt", false, false, nil, nil, nil)
h.handleNonStream(rec, resp, "cid-empty-filtered", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, nil)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected status 400 for filtered upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -131,7 +131,7 @@ func TestHandleNonStreamReturns429WhenUpstreamHasOnlyThinking(t *testing.T) {
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-thinking-only", "deepseek-v4-pro", "prompt", true, false, nil, nil, nil)
h.handleNonStream(rec, resp, "cid-thinking-only", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, nil)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected status 429 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -150,7 +150,7 @@ func TestHandleNonStreamPromotesThinkingToolCallsWhenTextEmpty(t *testing.T) {
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-thinking-tool", "deepseek-v4-pro", "prompt", true, false, []string{"search"}, nil, nil)
h.handleNonStream(rec, resp, "cid-thinking-tool", "deepseek-v4-pro", "prompt", 0, true, false, []string{"search"}, nil, nil)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -181,7 +181,7 @@ func TestHandleNonStreamPromotesHiddenThinkingDSMLToolCallsWhenTextEmpty(t *test
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-hidden-thinking-tool", "deepseek-v4-pro", "prompt", false, false, []string{"search"}, nil, nil)
h.handleNonStream(rec, resp, "cid-hidden-thinking-tool", "deepseek-v4-pro", "prompt", 0, false, false, []string{"search"}, nil, nil)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for hidden thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -211,7 +211,7 @@ func TestHandleStreamToolsPlainTextStreamsBeforeFinish(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid6", "deepseek-v4-flash", "prompt", false, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid6", "deepseek-v4-flash", "prompt", 0, false, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -248,7 +248,7 @@ func TestHandleStreamIncompleteCapturedToolJSONFlushesAsTextOnFinalize(t *testin
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid10", "deepseek-v4-flash", "prompt", false, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid10", "deepseek-v4-flash", "prompt", 0, false, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -282,7 +282,7 @@ func TestHandleStreamPromotesThinkingToolCallsOnFinalizeWithoutMidstreamIntercep
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-thinking-stream", "deepseek-v4-pro", "prompt", true, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid-thinking-stream", "deepseek-v4-pro", "prompt", 0, true, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -291,20 +291,16 @@ func TestHandleStreamPromotesThinkingToolCallsOnFinalizeWithoutMidstreamIntercep
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta from finalize fallback, body=%s", rec.Body.String())
}
reasoningSeen := false
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if asString(delta["reasoning_content"]) != "" {
reasoningSeen = true
t.Fatalf("did not expect leaked reasoning_content markup, body=%s", rec.Body.String())
}
}
}
if !reasoningSeen {
t.Fatalf("expected reasoning_content to stream before finalize fallback, body=%s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
@@ -319,7 +315,7 @@ func TestHandleStreamPromotesHiddenThinkingDSMLToolCallsOnFinalize(t *testing.T)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-hidden-thinking-stream", "deepseek-v4-pro", "prompt", false, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid-hidden-thinking-stream", "deepseek-v4-pro", "prompt", 0, false, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -353,7 +349,7 @@ func TestHandleStreamEmitsDistinctToolCallIDsAcrossSeparateToolBlocks(t *testing
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-multi", "deepseek-v4-flash", "prompt", false, false, []string{"read_file", "search"}, nil, nil)
h.handleStream(rec, req, resp, "cid-multi", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file", "search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -419,7 +415,7 @@ func TestHandleStreamCoercesSchemaDeclaredStringArgumentsOnFinalize(t *testing.T
},
}
h.handleStream(rec, req, resp, "cid-string-protect", "deepseek-v4-flash", "prompt", false, false, []string{"Write"}, toolsRaw, nil)
h.handleStream(rec, req, resp, "cid-string-protect", "deepseek-v4-flash", "prompt", 0, false, false, []string{"Write"}, toolsRaw, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {

View File

@@ -0,0 +1,26 @@
package chat
// addRefFileTokensToUsage adds inline-uploaded file token estimates to an existing
// usage map inside a response object. This keeps the token accounting aware of file
// content that the upstream model processes but that is not part of the prompt text.
func addRefFileTokensToUsage(obj map[string]any, refFileTokens int) {
if refFileTokens <= 0 || obj == nil {
return
}
usage, ok := obj["usage"].(map[string]any)
if !ok || usage == nil {
return
}
for _, key := range []string{"input_tokens", "prompt_tokens"} {
if v, ok := usage[key]; ok {
if n, ok := v.(int); ok {
usage[key] = n + refFileTokens
}
}
}
if v, ok := usage["total_tokens"]; ok {
if n, ok := v.(int); ok {
usage["total_tokens"] = n + refFileTokens
}
}
}

View File

@@ -26,3 +26,59 @@ func TestReplaceCitationMarkersWithLinksKeepsUnknownIndex(t *testing.T) {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksSupportsReferenceMarker(t *testing.T) {
raw := "新闻摘要[reference:1],详情[reference:2]。"
links := map[int]string{
1: "https://example.com/r1",
2: "https://example.com/r2",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "新闻摘要[1](https://example.com/r1),详情[2](https://example.com/r2)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksSupportsReferenceZeroBased(t *testing.T) {
raw := "来源[reference:0] 与 [reference:1]。"
links := map[int]string{
1: "https://example.com/first",
2: "https://example.com/second",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "来源[0](https://example.com/first) 与 [1](https://example.com/second)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksKeepsCitationOneBasedWithZeroBasedReference(t *testing.T) {
raw := "引用[citation:1],来源[reference:0],后续[reference:1]。"
links := map[int]string{
1: "https://example.com/first",
2: "https://example.com/second",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "引用[1](https://example.com/first),来源[0](https://example.com/first),后续[1](https://example.com/second)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksDetectsSpacedReferenceZeroBased(t *testing.T) {
raw := "来源[reference: 0] 与 [reference: 1]。"
links := map[int]string{
1: "https://example.com/first",
2: "https://example.com/second",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "来源[0](https://example.com/first) 与 [1](https://example.com/second)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}

View File

@@ -216,6 +216,45 @@ func TestChatCompletionsInlineUploadFailureReturnsBadRequest(t *testing.T) {
}
}
func TestChatCompletionsInlineUploadLimitReturnsBadRequest(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
content := []any{map[string]any{"type": "input_text", "text": "hi"}}
for i := 0; i < 51; i++ {
content = append(content, map[string]any{
"type": "image_url",
"image_url": map[string]any{"url": "data:image/png;base64,QUJDRA=="},
})
}
body, err := json.Marshal(map[string]any{
"model": "deepseek-v4-flash",
"messages": []any{map[string]any{
"role": "user",
"content": content,
}},
"stream": false,
})
if err != nil {
t.Fatalf("marshal request: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(string(body)))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.ChatCompletions(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "exceeded maximum of 50 inline files per request") {
t.Fatalf("expected inline file limit error, got body=%s", rec.Body.String())
}
if ds.completionReq != nil {
t.Fatalf("did not expect completion call after inline file limit error")
}
}
func TestResponsesInlineUploadFailureReturnsInternalServerError(t *testing.T) {
ds := &inlineUploadDSStub{uploadErr: errors.New("boom")}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}

View File

@@ -39,11 +39,12 @@ func (e *inlineFileUploadError) Error() string {
}
type inlineUploadState struct {
ctx context.Context
handler *Handler
auth *auth.RequestAuth
uploadedByID map[string]string
uploadCount int
ctx context.Context
handler *Handler
auth *auth.RequestAuth
uploadedByID map[string]string
uploadCount int
inlineFileBytes int
}
type inlineDecodedFile struct {
@@ -75,6 +76,9 @@ func (h *Handler) PreprocessInlineFileInputs(ctx context.Context, a *auth.Reques
if refIDs := promptcompat.CollectOpenAIRefFileIDs(req); len(refIDs) > 0 {
req["ref_file_ids"] = stringsToAnySlice(refIDs)
}
if state.inlineFileBytes > 0 {
req["_inline_file_bytes"] = state.inlineFileBytes
}
return nil
}
@@ -135,13 +139,15 @@ func (s *inlineUploadState) tryUploadBlock(block map[string]any) (map[string]any
return nil, false, nil
}
if s.uploadCount >= maxInlineFilesPerRequest {
return nil, true, fmt.Errorf("exceeded maximum of %d inline files per request", maxInlineFilesPerRequest)
err := fmt.Errorf("exceeded maximum of %d inline files per request", maxInlineFilesPerRequest)
return nil, true, &inlineFileUploadError{status: http.StatusBadRequest, message: err.Error(), err: err}
}
fileID, err := s.uploadInlineFile(decoded)
if err != nil {
return nil, true, &inlineFileUploadError{status: http.StatusInternalServerError, message: "Failed to upload inline file.", err: err}
}
s.uploadCount++
s.inlineFileBytes += len(decoded.Data)
replacement := map[string]any{
"type": decoded.ReplacementType,
"file_id": fileID,

View File

@@ -13,7 +13,7 @@ import (
)
const (
currentInputFilename = "IGNORE.txt"
currentInputFilename = promptcompat.CurrentInputContextFilename
currentInputContentType = "text/plain; charset=utf-8"
currentInputPurpose = "assistants"
)
@@ -35,7 +35,6 @@ func (s Service) ApplyCurrentInputFile(ctx context.Context, a *auth.RequestAuth,
if strings.TrimSpace(fileText) == "" {
return stdReq, errors.New("current user input file produced empty transcript")
}
result, err := s.DS.UploadFile(ctx, a, dsclient.UploadFileRequest{
Filename: currentInputFilename,
ContentType: currentInputContentType,
@@ -58,9 +57,13 @@ func (s Service) ApplyCurrentInputFile(ctx context.Context, a *auth.RequestAuth,
}
stdReq.Messages = messages
stdReq.HistoryText = fileText
stdReq.CurrentInputFileApplied = true
stdReq.RefFileIDs = prependUniqueRefFileID(stdReq.RefFileIDs, fileID)
stdReq.FinalPrompt, stdReq.ToolNames = promptcompat.BuildOpenAIPrompt(messages, stdReq.ToolsRaw, "", stdReq.ToolChoice, stdReq.Thinking)
// Token accounting must reflect the actual downstream context:
// the uploaded history.txt file content + the neutral live prompt.
stdReq.PromptTokenText = fileText + "\n" + stdReq.FinalPrompt
return stdReq, nil
}

View File

@@ -14,6 +14,7 @@ import (
"ds2api/internal/auth"
dsclient "ds2api/internal/deepseek/client"
"ds2api/internal/promptcompat"
"ds2api/internal/util"
)
func historySplitTestMessages() []any {
@@ -64,8 +65,8 @@ func TestBuildOpenAICurrentInputContextTranscriptUsesInjectedFileWrapper(t *test
_, historyMessages := splitOpenAIHistoryMessages(historySplitTestMessages(), 1)
transcript := buildOpenAICurrentInputContextTranscript(historyMessages)
if !strings.HasPrefix(transcript, "[file content end]\n\n") {
t.Fatalf("expected injected file wrapper prefix, got %q", transcript)
if strings.Contains(transcript, "[file content end]") || strings.Contains(transcript, "[file content begin]") || strings.Contains(transcript, "[file name]:") {
t.Fatalf("expected plain transcript without file wrapper tags, got %q", transcript)
}
if !strings.Contains(transcript, "<begin▁of▁sentence>") {
t.Fatalf("expected serialized conversation markers, got %q", transcript)
@@ -79,9 +80,7 @@ func TestBuildOpenAICurrentInputContextTranscriptUsesInjectedFileWrapper(t *test
if !strings.Contains(transcript, "<|DSML|tool_calls>") {
t.Fatalf("expected tool calls preserved, got %q", transcript)
}
if !strings.HasSuffix(transcript, "\n[file name]: IGNORE\n[file content begin]\n") {
t.Fatalf("expected injected file wrapper suffix, got %q", transcript)
}
}
func TestSplitOpenAIHistoryMessagesUsesLatestUserTurn(t *testing.T) {
@@ -274,12 +273,12 @@ func TestApplyCurrentInputFileUploadsFirstTurnWithInjectedWrapper(t *testing.T)
t.Fatalf("expected 1 current input upload, got %d", len(ds.uploadCalls))
}
upload := ds.uploadCalls[0]
if upload.Filename != "IGNORE.txt" {
if upload.Filename != "history.txt" {
t.Fatalf("unexpected upload filename: %q", upload.Filename)
}
uploadedText := string(upload.Data)
if !strings.HasPrefix(uploadedText, "[file content end]\n\n") {
t.Fatalf("expected injected file wrapper prefix, got %q", uploadedText)
if strings.Contains(uploadedText, "[file content end]") || strings.Contains(uploadedText, "[file content begin]") || strings.Contains(uploadedText, "[file name]:") {
t.Fatalf("expected uploaded transcript without file wrapper tags, got %q", uploadedText)
}
if !strings.Contains(uploadedText, "<begin▁of▁sentence><User>first turn content that is long enough") {
t.Fatalf("expected serialized current user turn markers, got %q", uploadedText)
@@ -287,13 +286,11 @@ func TestApplyCurrentInputFileUploadsFirstTurnWithInjectedWrapper(t *testing.T)
if !strings.Contains(uploadedText, promptcompat.ThinkingInjectionMarker) {
t.Fatalf("expected thinking injection in current input file, got %q", uploadedText)
}
if !strings.HasSuffix(uploadedText, "\n[file name]: IGNORE\n[file content begin]\n") {
t.Fatalf("expected injected file wrapper suffix, got %q", uploadedText)
}
if strings.Contains(out.FinalPrompt, "first turn content that is long enough") {
t.Fatalf("expected current input text to be replaced in live prompt, got %s", out.FinalPrompt)
}
if strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "IGNORE.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
if strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "history.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
t.Fatalf("expected live prompt not to instruct file reads, got %s", out.FinalPrompt)
}
if !strings.Contains(out.FinalPrompt, "Answer the latest user request directly.") {
@@ -302,6 +299,52 @@ func TestApplyCurrentInputFileUploadsFirstTurnWithInjectedWrapper(t *testing.T)
if len(out.RefFileIDs) != 1 || out.RefFileIDs[0] != "file-inline-1" {
t.Fatalf("expected current input file id in ref_file_ids, got %#v", out.RefFileIDs)
}
if !strings.Contains(out.PromptTokenText, "first turn content that is long enough") {
t.Fatalf("expected prompt token text to preserve original full context, got %q", out.PromptTokenText)
}
}
func TestApplyCurrentInputFilePreservesFullContextPromptForTokenCounting(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{
Store: mockOpenAIConfig{
wideInput: true,
currentInputEnabled: true,
currentInputMin: 0,
thinkingInjection: boolPtr(true),
},
DS: ds,
}
req := map[string]any{
"model": "deepseek-v4-flash",
"messages": historySplitTestMessages(),
}
stdReq, err := promptcompat.NormalizeOpenAIChatRequest(h.Store, req, "")
if err != nil {
t.Fatalf("normalize failed: %v", err)
}
out, err := h.applyCurrentInputFile(context.Background(), &auth.RequestAuth{DeepSeekToken: "token"}, stdReq)
if err != nil {
t.Fatalf("apply current input file failed: %v", err)
}
if out.FinalPrompt == stdReq.FinalPrompt {
t.Fatalf("expected live prompt to be rewritten after current input file")
}
// PromptTokenText must include the uploaded file content (which contains the full context)
// plus the neutral live prompt — reflecting the actual downstream token cost.
if !strings.Contains(out.PromptTokenText, "first user turn") || !strings.Contains(out.PromptTokenText, "latest user turn") {
t.Fatalf("expected prompt token text to contain file context with full conversation, got %q", out.PromptTokenText)
}
if strings.Contains(out.PromptTokenText, "[file content end]") || strings.Contains(out.PromptTokenText, "[file name]:") {
t.Fatalf("expected prompt token text to use raw transcript without wrapper tags, got %q", out.PromptTokenText)
}
if !strings.Contains(out.PromptTokenText, "Answer the latest user request directly.") {
t.Fatalf("expected prompt token text to also include neutral live prompt, got %q", out.PromptTokenText)
}
if strings.Contains(out.FinalPrompt, "first user turn") || strings.Contains(out.FinalPrompt, "latest user turn") {
t.Fatalf("expected live prompt to hide original turns, got %q", out.FinalPrompt)
}
}
func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
@@ -335,8 +378,8 @@ func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
t.Fatalf("expected one current input upload, got %d", len(ds.uploadCalls))
}
upload := ds.uploadCalls[0]
if upload.Filename != "IGNORE.txt" {
t.Fatalf("expected IGNORE.txt upload, got %q", upload.Filename)
if upload.Filename != "history.txt" {
t.Fatalf("expected history.txt upload, got %q", upload.Filename)
}
uploadedText := string(upload.Data)
for _, want := range []string{"system instructions", "first user turn", "hidden reasoning", "tool result", "latest user turn", promptcompat.ThinkingInjectionMarker} {
@@ -344,7 +387,7 @@ func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
t.Fatalf("expected full context file to contain %q, got %q", want, uploadedText)
}
}
if strings.Contains(out.FinalPrompt, "first user turn") || strings.Contains(out.FinalPrompt, "latest user turn") || strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "IGNORE.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
if strings.Contains(out.FinalPrompt, "first user turn") || strings.Contains(out.FinalPrompt, "latest user turn") || strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "history.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
t.Fatalf("expected live prompt to use only a neutral continuation instruction, got %s", out.FinalPrompt)
}
if !strings.Contains(out.FinalPrompt, "Answer the latest user request directly.") {
@@ -352,7 +395,7 @@ func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
}
}
func TestApplyCurrentInputFileLeavesHistoryTextEmpty(t *testing.T) {
func TestApplyCurrentInputFileCarriesHistoryText(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{
Store: mockOpenAIConfig{
@@ -377,8 +420,8 @@ func TestApplyCurrentInputFileLeavesHistoryTextEmpty(t *testing.T) {
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
if out.HistoryText != "" {
t.Fatalf("expected current input file flow to leave history text empty, got %q", out.HistoryText)
if out.HistoryText != string(ds.uploadCalls[0].Data) {
t.Fatalf("expected current input file flow to preserve uploaded text in history, got %q", out.HistoryText)
}
}
@@ -411,15 +454,15 @@ func TestChatCompletionsCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *t
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
upload := ds.uploadCalls[0]
if upload.Filename != "IGNORE.txt" {
if upload.Filename != "history.txt" {
t.Fatalf("unexpected upload filename: %q", upload.Filename)
}
if upload.Purpose != "assistants" {
t.Fatalf("unexpected purpose: %q", upload.Purpose)
}
historyText := string(upload.Data)
if !strings.Contains(historyText, "[file content end]") || !strings.Contains(historyText, "[file name]: IGNORE") {
t.Fatalf("expected injected IGNORE wrapper, got %s", historyText)
if strings.Contains(historyText, "[file content end]") || strings.Contains(historyText, "[file content begin]") || strings.Contains(historyText, "[file name]:") {
t.Fatalf("expected plain history transcript without wrapper tags, got %s", historyText)
}
if !strings.Contains(historyText, "latest user turn") {
t.Fatalf("expected full context to include latest turn, got %s", historyText)
@@ -438,6 +481,16 @@ func TestChatCompletionsCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *t
if len(refIDs) == 0 || refIDs[0] != "file-inline-1" {
t.Fatalf("expected uploaded current input file to be first ref_file_id, got %#v", ds.completionReq["ref_file_ids"])
}
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response failed: %v", err)
}
usage, _ := body["usage"].(map[string]any)
promptTokens := int(usage["prompt_tokens"].(float64))
neutralCount := util.CountPromptTokens(promptText, "deepseek-v4-flash")
if promptTokens <= neutralCount {
t.Fatalf("expected prompt_tokens to exceed neutral live prompt count (includes file context), got=%d neutral=%d", promptTokens, neutralCount)
}
}
func TestResponsesCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *testing.T) {
@@ -480,6 +533,16 @@ func TestResponsesCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *testing
if strings.Contains(promptText, "first user turn") || strings.Contains(promptText, "latest user turn") {
t.Fatalf("expected prompt to hide original turns, got %s", promptText)
}
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response failed: %v", err)
}
usage, _ := body["usage"].(map[string]any)
inputTokens := int(usage["input_tokens"].(float64))
neutralCount := util.CountPromptTokens(promptText, "deepseek-v4-flash")
if inputTokens <= neutralCount {
t.Fatalf("expected input_tokens to exceed neutral live prompt count (includes file context), got=%d neutral=%d", inputTokens, neutralCount)
}
}
func TestChatCompletionsCurrentInputFileMapsManagedAuthFailureTo401(t *testing.T) {

View File

@@ -42,6 +42,14 @@ func TestSanitizeLeakedOutputRemovesDanglingThinkBlock(t *testing.T) {
}
}
func TestSanitizeLeakedOutputRemovesCompleteDSMLToolCallWrapper(t *testing.T) {
raw := "前置文本\n<DSMLtool_calls>\n<DSMLinvoke name=\"Bash\">\n<DSMLparameter name=\"command\"></DSMLparameter>\n</DSMLinvoke>\n</DSMLtool_calls>\n后置文本"
got := sanitizeLeakedOutput(raw)
if got != "前置文本\n\n后置文本" {
t.Fatalf("unexpected sanitize result for leaked dsml wrapper: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesAgentXMLLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer</result></attempt_completion>"
got := sanitizeLeakedOutput(raw)

View File

@@ -18,6 +18,8 @@ import (
)
type responsesNonStreamResult struct {
rawThinking string
rawText string
thinking string
toolDetectionThinking string
text string
@@ -27,11 +29,12 @@ type responsesNonStreamResult struct {
responseMessageID int
}
func (h *Handler) handleResponsesNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
func (h *Handler) handleResponsesNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
attempts := 0
currentResp := resp
usagePrompt := finalPrompt
accumulatedThinking := ""
accumulatedRawThinking := ""
accumulatedToolDetectionThinking := ""
for {
result, ok := h.collectResponsesNonStreamAttempt(w, currentResp, responseID, model, usagePrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw)
@@ -39,11 +42,16 @@ func (h *Handler) handleResponsesNonStreamWithRetry(w http.ResponseWriter, ctx c
return
}
accumulatedThinking += sse.TrimContinuationOverlap(accumulatedThinking, result.thinking)
accumulatedRawThinking += sse.TrimContinuationOverlap(accumulatedRawThinking, result.rawThinking)
accumulatedToolDetectionThinking += sse.TrimContinuationOverlap(accumulatedToolDetectionThinking, result.toolDetectionThinking)
result.thinking = accumulatedThinking
result.rawThinking = accumulatedRawThinking
result.toolDetectionThinking = accumulatedToolDetectionThinking
result.parsed = detectAssistantToolCalls(result.text, result.thinking, result.toolDetectionThinking, toolNames)
result.parsed = detectAssistantToolCalls(result.rawText, result.text, result.rawThinking, result.toolDetectionThinking, toolNames)
result.body = openaifmt.BuildResponseObjectWithToolCalls(responseID, model, usagePrompt, result.thinking, result.text, result.parsed.Calls, toolsRaw)
if refFileTokens > 0 {
addRefFileTokensToUsage(result.body, refFileTokens)
}
if !shouldRetryResponsesNonStream(result, attempts) {
h.finishResponsesNonStreamResult(w, result, attempts, owner, responseID, toolChoice, traceID)
@@ -63,7 +71,7 @@ func (h *Handler) handleResponsesNonStreamWithRetry(w http.ResponseWriter, ctx c
config.Logger.Warn("[openai_empty_retry] retry request failed", "surface", "responses", "stream", false, "retry_attempt", attempts, "error", err)
return
}
usagePrompt = usagePromptWithEmptyOutputRetry(finalPrompt, attempts)
usagePrompt = usagePromptWithEmptyOutputRetry(usagePrompt, attempts)
currentResp = nextResp
}
}
@@ -78,16 +86,17 @@ func (h *Handler) collectResponsesNonStreamAttempt(w http.ResponseWriter, resp *
result := sse.CollectStream(resp, thinkingEnabled, false)
stripReferenceMarkers := h.compatStripReferenceMarkers()
sanitizedThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
toolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
sanitizedText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
sanitizedText = replaceCitationMarkersWithLinks(sanitizedText, result.CitationLinks)
}
textParsed := detectAssistantToolCalls(sanitizedText, sanitizedThinking, toolDetectionThinking, toolNames)
textParsed := detectAssistantToolCalls(result.Text, sanitizedText, result.Thinking, result.ToolDetectionThinking, toolNames)
responseObj := openaifmt.BuildResponseObjectWithToolCalls(responseID, model, usagePrompt, sanitizedThinking, sanitizedText, textParsed.Calls, toolsRaw)
return responsesNonStreamResult{
rawThinking: result.Thinking,
rawText: result.Text,
thinking: sanitizedThinking,
toolDetectionThinking: toolDetectionThinking,
toolDetectionThinking: result.ToolDetectionThinking,
text: sanitizedText,
contentFilter: result.ContentFilter,
parsed: textParsed,
@@ -123,8 +132,8 @@ func shouldRetryResponsesNonStream(result responsesNonStreamResult, attempts int
strings.TrimSpace(result.text) == ""
}
func (h *Handler) handleResponsesStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
streamRuntime, initialType, ok := h.prepareResponsesStreamRuntime(w, resp, owner, responseID, model, finalPrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw, toolChoice, traceID)
func (h *Handler) handleResponsesStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
streamRuntime, initialType, ok := h.prepareResponsesStreamRuntime(w, resp, owner, responseID, model, finalPrompt, refFileTokens, thinkingEnabled, searchEnabled, toolNames, toolsRaw, toolChoice, traceID)
if !ok {
return
}
@@ -165,7 +174,7 @@ func (h *Handler) handleResponsesStreamWithRetry(w http.ResponseWriter, r *http.
}
}
func (h *Handler) prepareResponsesStreamRuntime(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) (*responsesStreamRuntime, string, bool) {
func (h *Handler) prepareResponsesStreamRuntime(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) (*responsesStreamRuntime, string, bool) {
if resp.StatusCode != http.StatusOK {
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
@@ -190,6 +199,7 @@ func (h *Handler) prepareResponsesStreamRuntime(w http.ResponseWriter, resp *htt
h.getResponseStore().put(owner, responseID, obj)
},
)
streamRuntime.refFileTokens = refFileTokens
streamRuntime.sendCreated()
return streamRuntime, initialType, true
}

View File

@@ -130,6 +130,6 @@ func filterIncrementalToolCallDeltasByAllowed(deltas []toolstream.ToolCallDelta,
return shared.FilterIncrementalToolCallDeltasByAllowed(deltas, seenNames)
}
func detectAssistantToolCalls(text, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(text, exposedThinking, detectionThinking, toolNames)
func detectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking, toolNames)
}

View File

@@ -0,0 +1,26 @@
package responses
// addRefFileTokensToUsage adds inline-uploaded file token estimates to an existing
// usage map inside a response object. This keeps the token accounting aware of file
// content that the upstream model processes but that is not part of the prompt text.
func addRefFileTokensToUsage(obj map[string]any, refFileTokens int) {
if refFileTokens <= 0 || obj == nil {
return
}
usage, ok := obj["usage"].(map[string]any)
if !ok || usage == nil {
return
}
for _, key := range []string{"input_tokens", "prompt_tokens"} {
if v, ok := usage[key]; ok {
if n, ok := v.(int); ok {
usage[key] = n + refFileTokens
}
}
}
if v, ok := usage["total_tokens"]; ok {
if n, ok := v.(int); ok {
usage["total_tokens"] = n + refFileTokens
}
}
}

View File

@@ -114,14 +114,15 @@ func (h *Handler) Responses(w http.ResponseWriter, r *http.Request) {
}
responseID := "resp_" + strings.ReplaceAll(uuid.NewString(), "-", "")
refFileTokens := stdReq.RefFileTokens
if stdReq.Stream {
h.handleResponsesStreamWithRetry(w, r, a, resp, payload, pow, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
h.handleResponsesStreamWithRetry(w, r, a, resp, payload, pow, owner, responseID, stdReq.ResponseModel, stdReq.PromptTokenText, refFileTokens, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
return
}
h.handleResponsesNonStreamWithRetry(w, r.Context(), a, resp, payload, pow, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
h.handleResponsesNonStreamWithRetry(w, r.Context(), a, resp, payload, pow, owner, responseID, stdReq.ResponseModel, stdReq.PromptTokenText, refFileTokens, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
}
func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -131,12 +132,11 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
result := sse.CollectStream(resp, thinkingEnabled, true)
stripReferenceMarkers := h.compatStripReferenceMarkers()
sanitizedThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
toolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
sanitizedText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
sanitizedText = replaceCitationMarkersWithLinks(sanitizedText, result.CitationLinks)
}
textParsed := detectAssistantToolCalls(sanitizedText, sanitizedThinking, toolDetectionThinking, toolNames)
textParsed := detectAssistantToolCalls(result.Text, sanitizedText, result.Thinking, result.ToolDetectionThinking, toolNames)
if len(textParsed.Calls) == 0 && writeUpstreamEmptyOutputError(w, sanitizedText, sanitizedThinking, result.ContentFilter) {
return
}
@@ -149,11 +149,14 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
}
responseObj := openaifmt.BuildResponseObjectWithToolCalls(responseID, model, finalPrompt, sanitizedThinking, sanitizedText, textParsed.Calls, toolsRaw)
if refFileTokens > 0 {
addRefFileTokensToUsage(responseObj, refFileTokens)
}
h.getResponseStore().put(owner, responseID, responseObj)
writeJSON(w, http.StatusOK, responseObj)
}
func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request, resp *http.Response, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -195,6 +198,7 @@ func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request,
h.getResponseStore().put(owner, responseID, obj)
},
)
streamRuntime.refFileTokens = refFileTokens
streamRuntime.sendCreated()
streamengine.ConsumeSSE(streamengine.ConsumeConfig{

View File

@@ -18,13 +18,14 @@ type responsesStreamRuntime struct {
rc *http.ResponseController
canFlush bool
responseID string
model string
finalPrompt string
toolNames []string
toolsRaw any
traceID string
toolChoice promptcompat.ToolChoicePolicy
responseID string
model string
finalPrompt string
refFileTokens int
toolNames []string
toolsRaw any
traceID string
toolChoice promptcompat.ToolChoicePolicy
thinkingEnabled bool
searchEnabled bool
@@ -36,8 +37,10 @@ type responsesStreamRuntime struct {
toolCallsDoneEmitted bool
sieve toolstream.State
rawThinking strings.Builder
thinking strings.Builder
toolDetectionThinking strings.Builder
rawText strings.Builder
text strings.Builder
visibleText strings.Builder
responseMessageID int
@@ -141,15 +144,14 @@ func (s *responsesStreamRuntime) finalize(finishReason string, deferEmptyOutput
s.finalErrorStatus = 0
s.finalErrorMessage = ""
s.finalErrorCode = ""
finalThinking := s.thinking.String()
finalToolDetectionThinking := s.toolDetectionThinking.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
if s.bufferToolContent {
s.processToolStreamEvents(toolstream.Flush(&s.sieve, s.toolNames), true, true)
}
textParsed := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, s.toolNames)
finalThinking := s.thinking.String()
finalToolDetectionThinking := s.toolDetectionThinking.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
textParsed := detectAssistantToolCalls(s.rawText.String(), finalText, s.rawThinking.String(), finalToolDetectionThinking, s.toolNames)
detected := textParsed.Calls
s.logToolPolicyRejections(textParsed)
@@ -227,18 +229,19 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
}
}
for _, p := range parsed.Parts {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
contentSeen = true
if p.Type == "thinking" {
rawTrimmed := sse.TrimContinuationOverlap(s.rawThinking.String(), p.Text)
if rawTrimmed != "" {
s.rawThinking.WriteString(rawTrimmed)
contentSeen = true
}
if !s.thinkingEnabled {
continue
}
cleanedText := cleanVisibleOutput(rawTrimmed, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
@@ -248,16 +251,28 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
continue
}
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
rawTrimmed := sse.TrimContinuationOverlap(s.rawText.String(), p.Text)
if rawTrimmed == "" {
continue
}
s.text.WriteString(trimmed)
s.rawText.WriteString(rawTrimmed)
contentSeen = true
cleanedText := cleanVisibleOutput(rawTrimmed, s.stripReferenceMarkers)
if s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed != "" {
s.text.WriteString(trimmed)
}
if !s.bufferToolContent {
if trimmed == "" {
continue
}
s.emitTextDelta(trimmed)
continue
}
s.processToolStreamEvents(toolstream.ProcessChunk(&s.sieve, trimmed, s.toolNames), true, true)
s.processToolStreamEvents(toolstream.ProcessChunk(&s.sieve, rawTrimmed, s.toolNames), true, true)
}
return streamengine.ParsedDecision{ContentSeen: contentSeen}

View File

@@ -4,6 +4,7 @@ import (
"encoding/json"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/sse"
"ds2api/internal/toolstream"
)
@@ -43,7 +44,10 @@ func (s *responsesStreamRuntime) sendDone() {
func (s *responsesStreamRuntime) processToolStreamEvents(events []toolstream.Event, emitContent bool, resetAfterToolCalls bool) {
for _, evt := range events {
if emitContent && evt.Content != "" {
s.emitTextDelta(evt.Content)
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned != "" && (!s.searchEnabled || !sse.IsCitation(cleaned)) {
s.emitTextDelta(cleaned)
}
}
if len(evt.ToolCallDeltas) > 0 {
if !s.emitEarlyToolDeltas {

View File

@@ -145,7 +145,7 @@ func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, fin
}
}
return openaifmt.BuildResponseObjectFromItems(
obj := openaifmt.BuildResponseObjectFromItems(
s.responseID,
s.model,
s.finalPrompt,
@@ -154,4 +154,8 @@ func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, fin
output,
outputText,
)
if s.refFileTokens > 0 {
addRefFileTokensToUsage(obj, s.refFileTokens)
}
return obj
}

View File

@@ -27,7 +27,7 @@ func TestHandleResponsesStreamDoesNotEmitReasoningTextCompatEvents(t *testing.T)
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.reasoning.delta") {
@@ -57,7 +57,7 @@ func TestHandleResponsesStreamEmitsOutputTextDoneBeforeContentPartDone(t *testin
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.output_text.done") {
t.Fatalf("expected response.output_text.done payload, body=%s", body)
@@ -91,7 +91,7 @@ func TestHandleResponsesStreamOutputTextDeltaCarriesItemIndexes(t *testing.T) {
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
deltaPayload, ok := extractSSEEventPayload(body, "response.output_text.delta")
@@ -130,7 +130,7 @@ func TestHandleResponsesStreamEmitsDistinctToolCallIDsAcrossSeparateToolBlocks(t
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, []string{"read_file", "search"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file", "search"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
doneEvents := extractSSEEventPayloads(body, "response.function_call_arguments.done")
@@ -183,7 +183,7 @@ func TestHandleResponsesStreamRequiredToolChoiceFailure(t *testing.T) {
Mode: promptcompat.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.failed") {
@@ -213,7 +213,7 @@ func TestHandleResponsesStreamFailsWhenUpstreamHasOnlyThinking(t *testing.T) {
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.failed") {
@@ -251,11 +251,11 @@ func TestHandleResponsesStreamPromotesThinkingToolCallsOnFinalizeWithoutMidstrea
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.reasoning.delta") {
t.Fatalf("expected reasoning delta in stream body, got %s", body)
if strings.Contains(body, "event: response.reasoning.delta") {
t.Fatalf("did not expect leaked reasoning delta in stream body, got %s", body)
}
if !strings.Contains(body, "event: response.function_call_arguments.done") {
t.Fatalf("expected finalize fallback function call event, got %s", body)
@@ -288,7 +288,7 @@ func TestHandleResponsesStreamPromotesHiddenThinkingDSMLToolCallsOnFinalize(t *t
Mode: promptcompat.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
body := rec.Body.String()
if strings.Contains(body, "event: response.reasoning.delta") {
@@ -317,7 +317,7 @@ func TestHandleResponsesNonStreamRequiredToolChoiceViolation(t *testing.T) {
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("expected 422 for required tool_choice violation, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -344,7 +344,7 @@ func TestHandleResponsesNonStreamRequiredToolChoiceIgnoresThinkingToolPayloadWhe
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", true, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, true, false, []string{"read_file"}, nil, policy, "")
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("expected 422 for required tool_choice violation, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -366,7 +366,7 @@ func TestHandleResponsesNonStreamReturns429WhenUpstreamOutputEmpty(t *testing.T)
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected 429 for empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -388,7 +388,7 @@ func TestHandleResponsesNonStreamReturnsContentFilterErrorWhenUpstreamFilteredWi
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400 for filtered empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -410,7 +410,7 @@ func TestHandleResponsesNonStreamReturns429WhenUpstreamHasOnlyThinking(t *testin
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected 429 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -432,7 +432,7 @@ func TestHandleResponsesNonStreamPromotesThinkingToolCallsWhenTextEmpty(t *testi
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -462,7 +462,7 @@ func TestHandleResponsesNonStreamPromotesHiddenThinkingDSMLToolCallsWhenTextEmpt
Mode: promptcompat.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for hidden thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -509,7 +509,7 @@ func TestHandleResponsesStreamCoercesSchemaDeclaredStringArguments(t *testing.T)
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_string_protect", "deepseek-v4-flash", "prompt", false, false, []string{"Write"}, toolsRaw, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_string_protect", "deepseek-v4-flash", "prompt", 0, false, false, []string{"Write"}, toolsRaw, promptcompat.DefaultToolChoicePolicy(), "")
payload, ok := extractSSEEventPayload(rec.Body.String(), "response.function_call_arguments.done")
if !ok {

View File

@@ -6,12 +6,12 @@ import (
"ds2api/internal/toolcall"
)
func DetectAssistantToolCalls(text, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
textParsed := toolcall.ParseStandaloneToolCallsDetailed(text, toolNames)
func DetectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
textParsed := toolcall.ParseStandaloneToolCallsDetailed(rawText, toolNames)
if len(textParsed.Calls) > 0 {
return textParsed
}
if strings.TrimSpace(text) != "" {
if strings.TrimSpace(visibleText) != "" {
return textParsed
}
thinking := detectionThinking

View File

@@ -7,25 +7,43 @@ import (
"strings"
)
var citationMarkerPattern = regexp.MustCompile(`(?i)\[citation:\s*(\d+)\]`)
var citationMarkerPattern = regexp.MustCompile(`(?i)\[(citation|reference):\s*(\d+)\]`)
func ReplaceCitationMarkersWithLinks(text string, links map[int]string) string {
if strings.TrimSpace(text) == "" || len(links) == 0 {
return text
}
zeroBasedReference := hasZeroBasedReferenceMarker(text)
return citationMarkerPattern.ReplaceAllStringFunc(text, func(match string) string {
sub := citationMarkerPattern.FindStringSubmatch(match)
if len(sub) < 2 {
if len(sub) < 3 {
return match
}
idx, err := strconv.Atoi(strings.TrimSpace(sub[1]))
if err != nil || idx <= 0 {
idx, err := strconv.Atoi(strings.TrimSpace(sub[2]))
if err != nil || idx < 0 {
return match
}
url := strings.TrimSpace(links[idx])
lookupIdx := idx
if strings.EqualFold(sub[1], "reference") && zeroBasedReference {
lookupIdx = idx + 1
}
url := strings.TrimSpace(links[lookupIdx])
if url == "" {
return match
}
return fmt.Sprintf("[%d](%s)", idx, url)
})
}
func hasZeroBasedReferenceMarker(text string) bool {
for _, sub := range citationMarkerPattern.FindAllStringSubmatch(text, -1) {
if len(sub) < 3 || !strings.EqualFold(sub[1], "reference") {
continue
}
idx, err := strconv.Atoi(strings.TrimSpace(sub[2]))
if err == nil && idx == 0 {
return true
}
}
return false
}

View File

@@ -3,6 +3,8 @@ package shared
import (
"regexp"
"strings"
"ds2api/internal/toolcall"
)
var emptyJSONFencePattern = regexp.MustCompile("(?is)```json\\s*```")
@@ -47,10 +49,42 @@ func sanitizeLeakedOutput(text string) string {
out = leakedThinkTagPattern.ReplaceAllString(out, "")
out = leakedBOSMarkerPattern.ReplaceAllString(out, "")
out = leakedMetaMarkerPattern.ReplaceAllString(out, "")
out = stripLeakedToolCallWrapperBlocks(out)
out = sanitizeLeakedAgentXMLBlocks(out)
return out
}
func stripLeakedToolCallWrapperBlocks(text string) string {
if text == "" {
return text
}
var b strings.Builder
pos := 0
for pos < len(text) {
tag, ok := toolcall.FindToolMarkupTagOutsideIgnored(text, pos)
if !ok {
b.WriteString(text[pos:])
break
}
if tag.Start > pos {
b.WriteString(text[pos:tag.Start])
}
if tag.Closing || tag.Name != "tool_calls" {
b.WriteString(text[tag.Start : tag.End+1])
pos = tag.End + 1
continue
}
closeTag, ok := toolcall.FindMatchingToolMarkupClose(text, tag)
if !ok {
b.WriteString(text[tag.Start : tag.End+1])
pos = tag.End + 1
continue
}
pos = closeTag.End + 1
}
return b.String()
}
func stripDanglingThinkSuffix(text string) string {
matches := leakedThinkTagPattern.FindAllStringIndex(text, -1)
if len(matches) == 0 {

View File

@@ -308,6 +308,10 @@ function parseChunkForContent(chunk, thinkingEnabled, currentType, stripReferenc
}
if (val && typeof val === 'object') {
const directContent = asContentString(val, stripReferenceMarkers);
if (directContent) {
parts.push({ text: directContent, type: partType });
}
const resp = val.response && typeof val.response === 'object' ? val.response : val;
if (Array.isArray(resp.fragments)) {
for (const frag of resp.fragments) {
@@ -593,6 +597,12 @@ function asContentString(v, stripReferenceMarkers = true) {
if (Object.prototype.hasOwnProperty.call(v, 'v')) {
return asContentString(v.v, stripReferenceMarkers);
}
if (Object.prototype.hasOwnProperty.call(v, 'text')) {
return asContentString(v.text, stripReferenceMarkers);
}
if (Object.prototype.hasOwnProperty.call(v, 'value')) {
return asContentString(v.value, stripReferenceMarkers);
}
return '';
}
if (v == null) {

View File

@@ -205,14 +205,14 @@ async function handleVercelStream(req, res, rawBody, payload) {
if (detected.length > 0 && !toolCallsDoneEmitted) {
toolCallsEmitted = true;
toolCallsDoneEmitted = true;
sendDeltaFrame({ tool_calls: formatOpenAIStreamToolCalls(detected, streamToolCallIDs) });
sendDeltaFrame({ tool_calls: formatOpenAIStreamToolCalls(detected, streamToolCallIDs, payload.tools) });
} else if (toolSieveEnabled) {
const tailEvents = flushToolSieve(toolSieveState, toolNames);
for (const evt of tailEvents) {
if (evt.type === 'tool_calls' && Array.isArray(evt.calls) && evt.calls.length > 0) {
toolCallsEmitted = true;
toolCallsDoneEmitted = true;
sendDeltaFrame({ tool_calls: formatOpenAIStreamToolCalls(evt.calls, streamToolCallIDs) });
sendDeltaFrame({ tool_calls: formatOpenAIStreamToolCalls(evt.calls, streamToolCallIDs, payload.tools) });
resetStreamToolCallState(streamToolCallIDs, streamToolNames);
continue;
}
@@ -352,14 +352,14 @@ async function handleVercelStream(req, res, rawBody, payload) {
const formatted = formatIncrementalToolCallDeltas(filtered, streamToolCallIDs);
if (formatted.length > 0) {
toolCallsEmitted = true;
sendDeltaFrame({ tool_calls: formatted });
sendDeltaFrame({ tool_calls: formatted });
}
continue;
}
if (evt.type === 'tool_calls') {
toolCallsEmitted = true;
toolCallsDoneEmitted = true;
sendDeltaFrame({ tool_calls: formatOpenAIStreamToolCalls(evt.calls, streamToolCallIDs) });
sendDeltaFrame({ tool_calls: formatOpenAIStreamToolCalls(evt.calls, streamToolCallIDs, payload.tools) });
resetStreamToolCallState(streamToolCallIDs, streamToolNames);
continue;
}
@@ -513,6 +513,11 @@ function observeContinueState(state, chunk) {
if (chunk.p === 'response/status') {
setContinueStatus(state, asString(chunk.v));
}
if (chunk.p === 'response') {
observeContinueBatchPatches(state, 'response', chunk.v);
} else {
observeContinueBatchPatches(state, '', chunk.v);
}
const response = chunk.v && typeof chunk.v === 'object' ? chunk.v.response : null;
if (response && typeof response === 'object') {
const id = numberValue(response.message_id);
@@ -534,13 +539,43 @@ function observeContinueState(state, chunk) {
}
}
function observeContinueBatchPatches(state, parentPath, raw) {
if (!state || !Array.isArray(raw)) {
return;
}
for (const patch of raw) {
if (!patch || typeof patch !== 'object') {
continue;
}
const path = asString(patch.p).trim();
if (!path) {
continue;
}
let fullPath = path;
const parent = asString(parentPath).trim().replace(/^\/+|\/+$/g, '');
if (parent && !path.includes('/')) {
fullPath = `${parent}/${path}`;
}
switch (fullPath.replace(/^\/+|\/+$/g, '')) {
case 'response/status':
case 'status':
case 'response/quasi_status':
case 'quasi_status':
setContinueStatus(state, asString(patch.v));
break;
default:
break;
}
}
}
function setContinueStatus(state, status) {
const normalized = asString(status).trim();
if (!normalized) {
return;
}
state.lastStatus = normalized;
if (normalized.toUpperCase() === 'FINISHED') {
if (['FINISHED', 'CONTENT_FILTER'].includes(normalized.toUpperCase())) {
state.finished = true;
}
}
@@ -549,7 +584,7 @@ function shouldAutoContinue(state) {
if (!state || state.finished || !state.sessionID || state.responseMessageID <= 0) {
return false;
}
return ['WIP', 'INCOMPLETE', 'AUTO_CONTINUE'].includes(asString(state.lastStatus).trim().toUpperCase());
return ['INCOMPLETE', 'AUTO_CONTINUE'].includes(asString(state.lastStatus).trim().toUpperCase());
}
function numberValue(v) {

View File

@@ -2,11 +2,12 @@
const crypto = require('crypto');
function formatOpenAIStreamToolCalls(calls, idStore) {
function formatOpenAIStreamToolCalls(calls, idStore, toolsRaw) {
if (!Array.isArray(calls) || calls.length === 0) {
return [];
}
return calls.map((c, idx) => ({
const normalized = normalizeParsedToolCallsForSchemas(calls, toolsRaw);
return normalized.map((c, idx) => ({
index: idx,
id: ensureStreamToolCallID(idStore, idx),
type: 'function',
@@ -17,6 +18,194 @@ function formatOpenAIStreamToolCalls(calls, idStore) {
}));
}
function normalizeParsedToolCallsForSchemas(calls, toolsRaw) {
if (!Array.isArray(calls) || calls.length === 0) {
return calls;
}
const schemas = buildToolSchemaIndex(toolsRaw);
if (!schemas) {
return calls;
}
let changedAny = false;
const out = calls.map((call) => {
const name = String(call && call.name || '').trim().toLowerCase();
const schema = schemas[name];
if (!schema || !call || !call.input || typeof call.input !== 'object' || Array.isArray(call.input)) {
return call;
}
const [normalized, changed] = normalizeToolValueWithSchema(call.input, schema);
if (!changed || !normalized || typeof normalized !== 'object' || Array.isArray(normalized)) {
return call;
}
changedAny = true;
return { ...call, input: normalized };
});
return changedAny ? out : calls;
}
function buildToolSchemaIndex(toolsRaw) {
if (!Array.isArray(toolsRaw) || toolsRaw.length === 0) {
return null;
}
const out = {};
for (const item of toolsRaw) {
if (!item || typeof item !== 'object' || Array.isArray(item)) {
continue;
}
const [name, schema] = extractToolNameAndSchema(item);
if (!name || !schema || typeof schema !== 'object' || Array.isArray(schema)) {
continue;
}
out[name.toLowerCase()] = schema;
}
return Object.keys(out).length > 0 ? out : null;
}
function extractToolNameAndSchema(tool) {
const fn = tool && typeof tool.function === 'object' && !Array.isArray(tool.function) ? tool.function : null;
const name = firstNonEmptyString(tool.name, fn && fn.name);
const schema = firstNonNil(
tool.parameters,
tool.input_schema,
tool.inputSchema,
tool.schema,
fn && fn.parameters,
fn && fn.input_schema,
fn && fn.inputSchema,
fn && fn.schema,
);
return [name, schema];
}
function normalizeToolValueWithSchema(value, schema) {
if (value == null || !schema || typeof schema !== 'object' || Array.isArray(schema)) {
return [value, false];
}
if (shouldCoerceSchemaToString(schema)) {
return stringifySchemaValue(value);
}
if (looksLikeObjectSchema(schema)) {
if (!value || typeof value !== 'object' || Array.isArray(value)) {
return [value, false];
}
const properties = schema.properties && typeof schema.properties === 'object' && !Array.isArray(schema.properties) ? schema.properties : null;
const additional = schema.additionalProperties;
let changed = false;
const out = {};
for (const [key, current] of Object.entries(value)) {
let next = current;
let fieldChanged = false;
if (properties && Object.prototype.hasOwnProperty.call(properties, key)) {
[next, fieldChanged] = normalizeToolValueWithSchema(current, properties[key]);
} else if (additional != null) {
[next, fieldChanged] = normalizeToolValueWithSchema(current, additional);
}
out[key] = next;
changed = changed || fieldChanged;
}
return changed ? [out, true] : [value, false];
}
if (looksLikeArraySchema(schema)) {
if (!Array.isArray(value) || value.length === 0 || schema.items == null) {
return [value, false];
}
let changed = false;
const out = value.map((item, idx) => {
const itemSchema = Array.isArray(schema.items) ? schema.items[idx] : schema.items;
if (itemSchema == null) {
return item;
}
const [next, itemChanged] = normalizeToolValueWithSchema(item, itemSchema);
changed = changed || itemChanged;
return next;
});
return changed ? [out, true] : [value, false];
}
return [value, false];
}
function shouldCoerceSchemaToString(schema) {
if (!schema || typeof schema !== 'object' || Array.isArray(schema)) {
return false;
}
if (typeof schema.const === 'string') {
return true;
}
if (Array.isArray(schema.enum) && schema.enum.length > 0 && schema.enum.every((item) => typeof item === 'string')) {
return true;
}
if (typeof schema.type === 'string') {
return schema.type.trim().toLowerCase() === 'string';
}
if (Array.isArray(schema.type) && schema.type.length > 0) {
let hasString = false;
for (const item of schema.type) {
if (typeof item !== 'string') {
return false;
}
const typ = item.trim().toLowerCase();
if (typ === 'string') {
hasString = true;
} else if (typ !== 'null') {
return false;
}
}
return hasString;
}
return false;
}
function looksLikeObjectSchema(schema) {
return !!schema && typeof schema === 'object' && !Array.isArray(schema) && (
(typeof schema.type === 'string' && schema.type.trim().toLowerCase() === 'object') ||
(schema.properties && typeof schema.properties === 'object' && !Array.isArray(schema.properties)) ||
schema.additionalProperties != null
);
}
function looksLikeArraySchema(schema) {
return !!schema && typeof schema === 'object' && !Array.isArray(schema) && (
(typeof schema.type === 'string' && schema.type.trim().toLowerCase() === 'array') ||
schema.items != null
);
}
function stringifySchemaValue(value) {
if (value == null) {
return [value, false];
}
if (typeof value === 'string') {
return [value, false];
}
try {
return [JSON.stringify(value), true];
} catch {
return [value, false];
}
}
function firstNonNil(...values) {
for (const value of values) {
if (value != null) {
return value;
}
}
return null;
}
function firstNonEmptyString(...values) {
for (const value of values) {
if (typeof value !== 'string') {
continue;
}
const trimmed = value.trim();
if (trimmed) {
return trimmed;
}
}
return '';
}
function ensureStreamToolCallID(idStore, index) {
if (!(idStore instanceof Map)) {
return `call_${newCallID()}`;

View File

@@ -1,13 +1,12 @@
package promptcompat
import (
"fmt"
"strings"
"ds2api/internal/prompt"
)
const historySplitInjectedFilename = "IGNORE"
const CurrentInputContextFilename = "history.txt"
func BuildOpenAIHistoryTranscript(messages []any) string {
return buildOpenAIInjectedFileTranscript(messages)
@@ -32,5 +31,5 @@ func buildOpenAIInjectedFileTranscript(messages []any) string {
if transcript == "" {
return ""
}
return fmt.Sprintf("[file content end]\n\n%s\n\n[file name]: %s\n[file content begin]\n", transcript, historySplitInjectedFilename)
return transcript
}

View File

@@ -39,20 +39,22 @@ func NormalizeOpenAIChatRequest(store ConfigReader, req map[string]any, traceID
refFileIDs := CollectOpenAIRefFileIDs(req)
return StandardRequest{
Surface: "openai_chat",
RequestedModel: strings.TrimSpace(model),
ResolvedModel: resolvedModel,
ResponseModel: responseModel,
Messages: messagesRaw,
ToolsRaw: req["tools"],
FinalPrompt: finalPrompt,
ToolNames: toolNames,
ToolChoice: toolPolicy,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
RefFileIDs: refFileIDs,
PassThrough: passThrough,
Surface: "openai_chat",
RequestedModel: strings.TrimSpace(model),
ResolvedModel: resolvedModel,
ResponseModel: responseModel,
Messages: messagesRaw,
PromptTokenText: finalPrompt,
ToolsRaw: req["tools"],
FinalPrompt: finalPrompt,
ToolNames: toolNames,
ToolChoice: toolPolicy,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
RefFileIDs: refFileIDs,
RefFileTokens: estimateInlineFileTokens(req),
PassThrough: passThrough,
}, nil
}
@@ -99,20 +101,22 @@ func NormalizeOpenAIResponsesRequest(store ConfigReader, req map[string]any, tra
refFileIDs := CollectOpenAIRefFileIDs(req)
return StandardRequest{
Surface: "openai_responses",
RequestedModel: model,
ResolvedModel: resolvedModel,
ResponseModel: model,
Messages: messagesRaw,
ToolsRaw: req["tools"],
FinalPrompt: finalPrompt,
ToolNames: toolNames,
ToolChoice: toolPolicy,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
RefFileIDs: refFileIDs,
PassThrough: passThrough,
Surface: "openai_responses",
RequestedModel: model,
ResolvedModel: resolvedModel,
ResponseModel: model,
Messages: messagesRaw,
PromptTokenText: finalPrompt,
ToolsRaw: req["tools"],
FinalPrompt: finalPrompt,
ToolNames: toolNames,
ToolChoice: toolPolicy,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
RefFileIDs: refFileIDs,
RefFileTokens: estimateInlineFileTokens(req),
PassThrough: passThrough,
}, nil
}
@@ -356,3 +360,30 @@ func namesToSet(names []string) map[string]struct{} {
}
return out
}
// estimateInlineFileTokens extracts the byte count stashed by PreprocessInlineFileInputs
// and converts it to a conservative token estimate. Inline files are typically images or
// documents that the upstream model will process; we use bytes/3 (rather than bytes/4)
// as a slightly pessimistic approximation so the returned context token count stays
// safely above the real value.
func estimateInlineFileTokens(req map[string]any) int {
raw, ok := req["_inline_file_bytes"]
if !ok {
return 0
}
var bytes int
switch v := raw.(type) {
case int:
bytes = v
case int64:
bytes = int(v)
case float64:
bytes = int(v)
default:
return 0
}
if bytes <= 0 {
return 0
}
return bytes / 3
}

View File

@@ -9,6 +9,7 @@ type StandardRequest struct {
ResponseModel string
Messages []any
HistoryText string
PromptTokenText string
CurrentInputFileApplied bool
ToolsRaw any
FinalPrompt string
@@ -18,6 +19,7 @@ type StandardRequest struct {
Thinking bool
Search bool
RefFileIDs []string
RefFileTokens int
PassThrough map[string]any
}

View File

@@ -13,7 +13,7 @@ func TestStandardRequestCompletionPayloadSetsModelTypeFromResolvedModel(t *testi
{name: "default", model: "deepseek-v4-flash", thinking: false, search: false, modelType: "default"},
{name: "default_nothinking", model: "deepseek-v4-flash-nothinking", thinking: false, search: false, modelType: "default"},
{name: "expert", model: "deepseek-v4-pro", thinking: true, search: false, modelType: "expert"},
{name: "vision", model: "deepseek-v4-vision-search", thinking: false, search: true, modelType: "vision"},
{name: "vision", model: "deepseek-v4-vision", thinking: true, search: false, modelType: "vision"},
}
for _, tc := range tests {

View File

@@ -30,13 +30,7 @@ func injectToolPrompt(messages []map[string]any, tools []any, policy ToolChoiceP
if !ok {
continue
}
fn, _ := tool["function"].(map[string]any)
if len(fn) == 0 {
fn = tool
}
name, _ := fn["name"].(string)
desc, _ := fn["description"].(string)
schema, _ := fn["parameters"].(map[string]any)
name, desc, schema := toolcall.ExtractToolMeta(tool)
name = strings.TrimSpace(name)
if !isAllowed(name) {
continue

View File

@@ -98,6 +98,14 @@ func NewApp() (*App, error) {
r.Get("/v1/responses/{response_id}", responsesHandler.GetResponseByID)
r.Post("/v1/files", filesHandler.UploadFile)
r.Post("/v1/embeddings", embeddingsHandler.Embeddings)
// Root OpenAI aliases support clients configured with the bare DS2API service URL.
r.Get("/models", modelsHandler.ListModels)
r.Get("/models/{model_id}", modelsHandler.GetModel)
r.Post("/chat/completions", chatHandler.ChatCompletions)
r.Post("/responses", responsesHandler.Responses)
r.Get("/responses/{response_id}", responsesHandler.GetResponseByID)
r.Post("/files", filesHandler.UploadFile)
r.Post("/embeddings", embeddingsHandler.Embeddings)
claude.RegisterRoutes(r, claudeHandler)
gemini.RegisterRoutes(r, geminiHandler)
r.Route("/admin", func(ar chi.Router) {

View File

@@ -37,6 +37,13 @@ func TestAPIRoutesRemainRegistered(t *testing.T) {
"GET /v1/responses/{response_id}",
"POST /v1/files",
"POST /v1/embeddings",
"GET /models",
"GET /models/{model_id}",
"POST /chat/completions",
"POST /responses",
"GET /responses/{response_id}",
"POST /files",
"POST /embeddings",
"GET /anthropic/v1/models",
"POST /anthropic/v1/messages",
"POST /anthropic/v1/messages/count_tokens",

View File

@@ -244,11 +244,29 @@ func appendChunkValueContent(v any, partType string, newType *string, parts *[]C
}
*parts = append(*parts, pp...)
case map[string]any:
if appendObjectContentByPath(path, val, partType, parts) {
return false
}
appendWrappedFragments(val, partType, newType, parts)
}
return false
}
func appendObjectContentByPath(path string, val map[string]any, partType string, parts *[]ContentPart) bool {
if path != "response/content" && path != "response/thinking_content" && path != "" {
return false
}
text, _ := val["text"].(string)
if text == "" {
text, _ = val["content"].(string)
}
if text == "" {
return false
}
appendContentPart(parts, text, partType)
return true
}
func appendWrappedFragments(val map[string]any, partType string, newType *string, parts *[]ContentPart) {
resp := val
if wrapped, ok := val["response"].(map[string]any); ok {

View File

@@ -163,3 +163,44 @@ func TestParseSSEChunkForContentStripsLeakedThinkTagsFromText(t *testing.T) {
t.Fatalf("expected leaked think tag to be stripped, got %#v", parts[0])
}
}
func TestParseSSEChunkForContentResponseContentObjectShape(t *testing.T) {
chunk := map[string]any{
"p": "response/content",
"v": map[string]any{"text": "对象内容"},
}
parts, finished, _ := ParseSSEChunkForContent(chunk, false, "text")
if finished {
t.Fatal("expected unfinished")
}
if len(parts) != 1 || parts[0].Text != "对象内容" || parts[0].Type != "text" {
t.Fatalf("unexpected parts: %#v", parts)
}
}
func TestParseSSEChunkForThinkingContentObjectShape(t *testing.T) {
chunk := map[string]any{
"p": "response/thinking_content",
"v": map[string]any{"content": "对象思考"},
}
parts, finished, _ := ParseSSEChunkForContent(chunk, true, "thinking")
if finished {
t.Fatal("expected unfinished")
}
if len(parts) != 1 || parts[0].Text != "对象思考" || parts[0].Type != "thinking" {
t.Fatalf("unexpected parts: %#v", parts)
}
}
func TestParseSSEChunkForContentObjectShapeWithoutPath(t *testing.T) {
chunk := map[string]any{
"v": map[string]any{"text": "无路径对象内容"},
}
parts, finished, _ := ParseSSEChunkForContent(chunk, false, "text")
if finished {
t.Fatal("expected unfinished")
}
if len(parts) != 1 || parts[0].Text != "无路径对象内容" || parts[0].Type != "text" {
t.Fatalf("unexpected parts: %#v", parts)
}
}

View File

@@ -4,12 +4,15 @@ import (
"bufio"
"context"
"io"
"time"
)
const (
parsedLineBufferSize = 128
scannerBufferSize = 64 * 1024
maxScannerLineSize = 2 * 1024 * 1024
minFlushChars = 160
maxFlushWait = 80 * time.Millisecond
)
// StartParsedLinePump scans an upstream DeepSeek SSE body and emits normalized
@@ -20,21 +23,123 @@ func StartParsedLinePump(ctx context.Context, body io.Reader, thinkingEnabled bo
done := make(chan error, 1)
go func() {
defer close(out)
scanner := bufio.NewScanner(body)
scanner.Buffer(make([]byte, 0, scannerBufferSize), maxScannerLineSize)
type scanItem struct {
line []byte
err error
eof bool
}
lineCh := make(chan scanItem, 1)
stopScanner := make(chan struct{})
defer close(stopScanner)
go func() {
sendScanItem := func(item scanItem) bool {
select {
case lineCh <- item:
return true
case <-ctx.Done():
return false
case <-stopScanner:
return false
}
}
defer close(lineCh)
scanner := bufio.NewScanner(body)
scanner.Buffer(make([]byte, 0, scannerBufferSize), maxScannerLineSize)
for scanner.Scan() {
line := append([]byte{}, scanner.Bytes()...)
if !sendScanItem(scanItem{line: line}) {
return
}
}
_ = sendScanItem(scanItem{err: scanner.Err(), eof: true})
}()
ticker := time.NewTicker(maxFlushWait)
defer ticker.Stop()
currentType := initialType
for scanner.Scan() {
line := append([]byte{}, scanner.Bytes()...)
result := ParseDeepSeekContentLine(line, thinkingEnabled, currentType)
currentType = result.NextType
var pending *LineResult
pendingChars := 0
sendResult := func(r LineResult) bool {
select {
case out <- r:
return true
case <-ctx.Done():
done <- ctx.Err()
return false
}
}
flushPending := func() bool {
if pending == nil {
return true
}
if !sendResult(*pending) {
return false
}
pending = nil
pendingChars = 0
return true
}
for {
select {
case out <- result:
case <-ctx.Done():
done <- ctx.Err()
return
case <-ticker.C:
if !flushPending() {
return
}
case item, ok := <-lineCh:
if !ok || item.eof {
if !flushPending() {
return
}
done <- item.err
return
}
line := item.line
result := ParseDeepSeekContentLine(line, thinkingEnabled, currentType)
currentType = result.NextType
canAccumulate := result.Parsed && !result.Stop && result.ErrorMessage == "" && !result.ContentFilter && result.ResponseMessageID == 0
if canAccumulate {
lineChars := 0
for _, p := range result.Parts {
lineChars += len(p.Text)
}
for _, p := range result.ToolDetectionThinkingParts {
lineChars += len(p.Text)
}
if lineChars > 0 {
if pending == nil {
cp := result
pending = &cp
} else {
pending.Parts = append(pending.Parts, result.Parts...)
pending.ToolDetectionThinkingParts = append(pending.ToolDetectionThinkingParts, result.ToolDetectionThinkingParts...)
pending.NextType = result.NextType
}
pendingChars += lineChars
if pendingChars < minFlushChars {
continue
}
if !flushPending() {
return
}
continue
}
}
if !flushPending() {
return
}
if !sendResult(result) {
return
}
}
}
done <- scanner.Err()
}()
return out, done
}

View File

@@ -38,8 +38,8 @@ func TestStartParsedLinePumpMultipleLines(t *testing.T) {
if err := <-done; err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(collected) < 3 {
t.Fatalf("expected at least 3 results, got %d", len(collected))
if len(collected) < 2 {
t.Fatalf("expected at least 2 results, got %d", len(collected))
}
// First should be thinking
if collected[0].Parts[0].Type != "thinking" {
@@ -175,3 +175,31 @@ func TestStartParsedLinePumpThinkingDisabled(t *testing.T) {
t.Fatalf("expected at least 1 part, got %d", len(parts))
}
}
func TestStartParsedLinePumpAccumulatesSmallChunks(t *testing.T) {
body := strings.NewReader(
"data: {\"p\":\"response/content\",\"v\":\"h\"}\n" +
"data: {\"p\":\"response/content\",\"v\":\"i\"}\n" +
"data: [DONE]\n",
)
results, done := StartParsedLinePump(context.Background(), body, false, "text")
collected := make([]LineResult, 0)
for r := range results {
collected = append(collected, r)
}
if err := <-done; err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(collected) != 2 {
t.Fatalf("expected 2 results (accumulated content + done), got %d", len(collected))
}
if len(collected[0].Parts) != 2 {
t.Fatalf("expected 2 accumulated parts, got %d", len(collected[0].Parts))
}
if !collected[1].Stop {
t.Fatal("expected second result to stop")
}
}

View File

@@ -92,11 +92,45 @@ func filterToolCallsDetailed(parsed []ParsedToolCall) ([]ParsedToolCall, []strin
if tc.Input == nil {
tc.Input = map[string]any{}
}
if len(tc.Input) > 0 && !toolCallInputHasMeaningfulValue(tc.Input) {
continue
}
out = append(out, tc)
}
return out, nil
}
func toolCallInputHasMeaningfulValue(v any) bool {
switch x := v.(type) {
case nil:
return false
case string:
return strings.TrimSpace(x) != ""
case map[string]any:
if len(x) == 0 {
return false
}
for _, child := range x {
if toolCallInputHasMeaningfulValue(child) {
return true
}
}
return false
case []any:
if len(x) == 0 {
return false
}
for _, child := range x {
if toolCallInputHasMeaningfulValue(child) {
return true
}
}
return false
default:
return true
}
}
func looksLikeToolCallSyntax(text string) bool {
hasDSML, hasCanonical := ContainsToolCallWrapperSyntaxOutsideIgnored(text)
return hasDSML || hasCanonical

View File

@@ -48,7 +48,7 @@ func buildToolSchemaIndex(toolsRaw any) map[string]any {
if !ok {
continue
}
name, schema := extractToolNameAndSchema(tool)
name, _, schema := ExtractToolMeta(tool)
if name == "" || schema == nil {
continue
}
@@ -60,24 +60,31 @@ func buildToolSchemaIndex(toolsRaw any) map[string]any {
return out
}
func extractToolNameAndSchema(tool map[string]any) (string, any) {
func ExtractToolMeta(tool map[string]any) (string, string, any) {
name := strings.TrimSpace(asStringValue(tool["name"]))
schema := tool["parameters"]
if schema == nil {
schema = tool["input_schema"]
}
desc := strings.TrimSpace(asStringValue(tool["description"]))
schema := firstNonNil(
tool["parameters"],
tool["input_schema"],
tool["inputSchema"],
tool["schema"],
)
if fn, ok := tool["function"].(map[string]any); ok {
if name == "" {
name = strings.TrimSpace(asStringValue(fn["name"]))
}
if schema == nil {
schema = fn["parameters"]
}
if schema == nil {
schema = fn["input_schema"]
if desc == "" {
desc = strings.TrimSpace(asStringValue(fn["description"]))
}
schema = firstNonNil(
schema,
fn["parameters"],
fn["input_schema"],
fn["inputSchema"],
fn["schema"],
)
}
return name, schema
return name, desc, schema
}
func normalizeToolValueWithSchema(value any, schema any) (any, bool) {
@@ -264,3 +271,12 @@ func asStringValue(v any) string {
}
return ""
}
func firstNonNil(values ...any) any {
for _, value := range values {
if value != nil {
return value
}
}
return nil
}

View File

@@ -110,3 +110,52 @@ func TestNormalizeParsedToolCallsForSchemasLeavesAmbiguousUnionUnchanged(t *test
t.Fatalf("expected ambiguous union to stay unchanged, got %#v", got[0].Input["taskId"])
}
}
func TestNormalizeParsedToolCallsForSchemasSupportsCamelCaseInputSchema(t *testing.T) {
toolsRaw := []any{
map[string]any{
"name": "Write",
"inputSchema": map[string]any{
"type": "object",
"properties": map[string]any{
"content": map[string]any{"type": "string"},
},
},
},
}
calls := []ParsedToolCall{{Name: "Write", Input: map[string]any{"content": map[string]any{"message": "hi"}}}}
got := NormalizeParsedToolCallsForSchemas(calls, toolsRaw)
if got[0].Input["content"] != `{"message":"hi"}` {
t.Fatalf("expected camelCase inputSchema content coercion, got %#v", got[0].Input["content"])
}
}
func TestNormalizeParsedToolCallsForSchemasPreservesArrayWhenSchemaSaysArray(t *testing.T) {
toolsRaw := []any{
map[string]any{
"name": "todowrite",
"inputSchema": map[string]any{
"type": "object",
"properties": map[string]any{
"todos": map[string]any{
"type": "array",
"items": map[string]any{
"type": "object",
"properties": map[string]any{
"content": map[string]any{"type": "string"},
"status": map[string]any{"type": "string"},
"priority": map[string]any{"type": "string"},
},
},
},
},
},
},
}
todos := []any{map[string]any{"content": "x", "status": "pending", "priority": "high"}}
calls := []ParsedToolCall{{Name: "todowrite", Input: map[string]any{"todos": todos}}}
got := NormalizeParsedToolCallsForSchemas(calls, toolsRaw)
if !reflect.DeepEqual(got[0].Input["todos"], todos) {
t.Fatalf("expected todos array preserved, got %#v want %#v", got[0].Input["todos"], todos)
}
}

View File

@@ -323,6 +323,28 @@ func TestParseToolCallsDetailedMarksToolCallsSyntax(t *testing.T) {
}
}
func TestParseToolCallsRejectsAllEmptyParameterPayload(t *testing.T) {
text := `<tool_calls><invoke name="Bash"><parameter name="command"></parameter><parameter name="description"> </parameter><parameter name="timeout"></parameter></invoke></tool_calls>`
res := ParseToolCallsDetailed(text, []string{"Bash"})
if !res.SawToolCallSyntax {
t.Fatalf("expected tool syntax to be detected, got %#v", res)
}
if len(res.Calls) != 0 {
t.Fatalf("expected all-empty payload to be rejected, got %#v", res.Calls)
}
}
func TestParseToolCallsPreservesExplicitZeroArgToolCall(t *testing.T) {
text := `<tool_calls><invoke name="noop"></invoke></tool_calls>`
res := ParseToolCallsDetailed(text, []string{"noop"})
if len(res.Calls) != 1 {
t.Fatalf("expected zero-arg tool call to remain valid, got %#v", res.Calls)
}
if len(res.Calls[0].Input) != 0 {
t.Fatalf("expected empty input map for zero-arg tool call, got %#v", res.Calls[0].Input)
}
}
func TestParseToolCallsSupportsInlineJSONToolObject(t *testing.T) {
text := `<tool_calls><invoke name="Bash">{"input":{"command":"pwd","description":"show cwd"}}</invoke></tool_calls>`
calls := ParseToolCalls(text, []string{"bash"})

View File

@@ -44,14 +44,21 @@ func consumeXMLToolCapture(captured string, toolNames []string) (prefix string,
xmlBlock := captured[tag.Start : closeTag.End+1]
prefixPart := captured[:tag.Start]
suffixPart := captured[closeTag.End+1:]
parsed := toolcall.ParseToolCalls(xmlBlock, toolNames)
if len(parsed) > 0 {
parsed := toolcall.ParseStandaloneToolCallsDetailed(xmlBlock, toolNames)
if len(parsed.Calls) > 0 {
prefixPart, suffixPart = trimWrappingJSONFence(prefixPart, suffixPart)
if best == nil || tag.Start < best.start {
best = &candidate{start: tag.Start, prefix: prefixPart, calls: parsed, suffix: suffixPart}
best = &candidate{start: tag.Start, prefix: prefixPart, calls: parsed.Calls, suffix: suffixPart}
}
break
}
if parsed.SawToolCallSyntax {
if rejected == nil || tag.Start < rejected.start {
rejected = &rejectedBlock{start: tag.Start, prefix: prefixPart, suffix: suffixPart}
}
searchFrom = tag.End + 1
continue
}
if rejected == nil || tag.Start < rejected.start {
rejected = &rejectedBlock{start: tag.Start, prefix: prefixPart + xmlBlock, suffix: suffixPart}
}
@@ -75,10 +82,13 @@ func consumeXMLToolCapture(captured string, toolNames []string) (prefix string,
xmlBlock := "<tool_calls>" + captured[invokeTag.Start:closeTag.End+1]
prefixPart := captured[:invokeTag.Start]
suffixPart := captured[closeTag.End+1:]
parsed := toolcall.ParseToolCalls(xmlBlock, toolNames)
if len(parsed) > 0 {
parsed := toolcall.ParseStandaloneToolCallsDetailed(xmlBlock, toolNames)
if len(parsed.Calls) > 0 {
prefixPart, suffixPart = trimWrappingJSONFence(prefixPart, suffixPart)
return prefixPart, parsed, suffixPart, true
return prefixPart, parsed.Calls, suffixPart, true
}
if parsed.SawToolCallSyntax {
return prefixPart, nil, suffixPart, true
}
return prefixPart + captured[invokeTag.Start:closeTag.End+1], nil, suffixPart, true
}

View File

@@ -288,7 +288,7 @@ func TestProcessToolSieveNonToolXMLKeepsSuffixForToolParsing(t *testing.T) {
}
}
func TestProcessToolSievePassesThroughMalformedExecutableXMLBlock(t *testing.T) {
func TestProcessToolSieveSuppressesMalformedExecutableXMLBlock(t *testing.T) {
var state State
chunk := `<tool_calls><invoke name="read_file"><param>{"path":"README.md"}</param></invoke></tool_calls>`
events := ProcessChunk(&state, chunk, []string{"read_file"})
@@ -302,10 +302,39 @@ func TestProcessToolSievePassesThroughMalformedExecutableXMLBlock(t *testing.T)
}
if toolCalls != 0 {
t.Fatalf("expected malformed executable-looking XML to stay text, got %d events=%#v", toolCalls, events)
t.Fatalf("expected malformed executable-looking XML not to become a tool call, got %d events=%#v", toolCalls, events)
}
if textContent.String() != chunk {
t.Fatalf("expected malformed executable-looking XML to pass through unchanged, got %q", textContent.String())
if textContent.Len() != 0 {
t.Fatalf("expected malformed executable-looking XML to be suppressed, got %q", textContent.String())
}
}
func TestProcessToolSieveSuppressesAllEmptyDSMLToolBlock(t *testing.T) {
var state State
chunk := strings.Join([]string{
`<|DSML|tool_calls>`,
`<|DSML|invoke name="Bash">`,
`<|DSML|parameter name="command"></|DSML|parameter>`,
`<|DSML|parameter name="description"> </|DSML|parameter>`,
`<|DSML|parameter name="timeout"></|DSML|parameter>`,
`</|DSML|invoke>`,
`</|DSML|tool_calls>`,
}, "\n")
events := ProcessChunk(&state, chunk, []string{"Bash"})
events = append(events, Flush(&state, []string{"Bash"})...)
var textContent strings.Builder
toolCalls := 0
for _, evt := range events {
textContent.WriteString(evt.Content)
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 0 {
t.Fatalf("expected all-empty DSML block not to produce tool calls, got %d events=%#v", toolCalls, events)
}
if textContent.Len() != 0 {
t.Fatalf("expected all-empty DSML block not to leak as text, got %q", textContent.String())
}
}

View File

@@ -23,9 +23,9 @@ func BuildOpenAIChatCompletion(completionID, model, finalPrompt, finalThinking,
messageObj["tool_calls"] = toolcall.FormatOpenAIToolCalls(detected, nil)
messageObj["content"] = nil
}
promptTokens := EstimateTokens(finalPrompt)
reasoningTokens := EstimateTokens(finalThinking)
completionTokens := EstimateTokens(finalText)
promptTokens := CountPromptTokens(finalPrompt, model)
reasoningTokens := CountOutputTokens(finalThinking, model)
completionTokens := CountOutputTokens(finalText, model)
return map[string]any{
"id": completionID,
@@ -86,9 +86,9 @@ func BuildOpenAIResponseObject(responseID, model, finalPrompt, finalThinking, fi
"content": content,
})
}
promptTokens := EstimateTokens(finalPrompt)
reasoningTokens := EstimateTokens(finalThinking)
completionTokens := EstimateTokens(finalText)
promptTokens := CountPromptTokens(finalPrompt, model)
reasoningTokens := CountOutputTokens(finalThinking, model)
completionTokens := CountOutputTokens(finalText, model)
return map[string]any{
"id": responseID,
"type": "response",
@@ -140,8 +140,8 @@ func BuildClaudeMessageResponse(messageID, model string, normalizedMessages []an
"stop_reason": stopReason,
"stop_sequence": nil,
"usage": map[string]any{
"input_tokens": EstimateTokens(fmt.Sprintf("%v", normalizedMessages)),
"output_tokens": EstimateTokens(finalThinking) + EstimateTokens(finalText),
"input_tokens": CountPromptTokens(fmt.Sprintf("%v", normalizedMessages), model),
"output_tokens": CountOutputTokens(finalThinking, model) + CountOutputTokens(finalText, model),
},
}
}

46
internal/util/text.go Normal file
View File

@@ -0,0 +1,46 @@
package util
import "unicode/utf8"
// TruncateRunes trims a string to at most limit Unicode code points.
func TruncateRunes(text string, limit int) (string, bool) {
if limit < 0 {
return text, false
}
if limit == 0 {
return "", text != ""
}
count := 0
for i := range text {
if count == limit {
return text[:i], true
}
count++
}
return text, false
}
// TruncateUTF8Bytes trims a string to fit within limit bytes without cutting
// through a UTF-8 code point boundary.
func TruncateUTF8Bytes(text string, limit int) (string, bool) {
if limit < 0 {
return text, false
}
if len(text) <= limit {
return text, false
}
if limit == 0 {
return "", true
}
raw := []byte(text)
cut := limit
if cut > len(raw) {
cut = len(raw)
}
for cut > 0 && cut < len(raw) && !utf8.RuneStart(raw[cut]) {
cut--
}
return string(raw[:cut]), true
}

View File

@@ -0,0 +1,87 @@
package util
import (
"strings"
tiktoken "github.com/hupe1980/go-tiktoken"
)
const (
defaultTokenizerModel = "gpt-4o"
claudeTokenizerModel = "claude"
)
func CountPromptTokens(text, model string) int {
base := maxTokenCount(
EstimateTokens(text),
countWithTokenizer(text, model),
)
if base <= 0 {
return 0
}
return base + conservativePromptPadding(base)
}
func CountOutputTokens(text, model string) int {
base := maxTokenCount(
EstimateTokens(text),
countWithTokenizer(text, model),
)
if base <= 0 {
return 0
}
return base
}
func countWithTokenizer(text, model string) int {
text = strings.TrimSpace(text)
if text == "" {
return 0
}
encoding, err := tiktoken.NewEncodingForModel(tokenizerModelForCount(model))
if err != nil {
return 0
}
ids, _, err := encoding.Encode(text, nil, nil)
if err != nil {
return 0
}
return len(ids)
}
func tokenizerModelForCount(model string) string {
model = strings.ToLower(strings.TrimSpace(model))
if model == "" {
return defaultTokenizerModel
}
switch {
case strings.HasPrefix(model, "claude"):
return claudeTokenizerModel
case strings.HasPrefix(model, "gpt-4"), strings.HasPrefix(model, "gpt-5"), strings.HasPrefix(model, "o1"), strings.HasPrefix(model, "o3"), strings.HasPrefix(model, "o4"):
return defaultTokenizerModel
case strings.HasPrefix(model, "deepseek-v4"):
return defaultTokenizerModel
case strings.HasPrefix(model, "deepseek"):
return defaultTokenizerModel
default:
return defaultTokenizerModel
}
}
func conservativePromptPadding(base int) int {
padding := base / 50
if padding < 4 {
padding = 4
}
return padding
}
func maxTokenCount(values ...int) int {
best := 0
for _, v := range values {
if v > best {
best = v
}
}
return best
}

View File

@@ -126,9 +126,12 @@ function binaryExists() {
// 查找占用端口的进程 PID
function findPidByPort(port) {
const numericPort = parseInt(port, 10);
if (isNaN(numericPort)) return [];
try {
if (isWindows) {
const output = execSync(`netstat -ano | findstr :${port} | findstr LISTENING`, {
const output = execSync(`netstat -ano | findstr :${numericPort} | findstr LISTENING`, {
encoding: 'utf-8',
shell: true,
stdio: ['pipe', 'pipe', 'ignore'],
@@ -141,7 +144,7 @@ function findPidByPort(port) {
}
return [...pids];
} else {
const output = execSync(`lsof -ti :${port}`, {
const output = execSync(`lsof -ti :${numericPort}`, {
encoding: 'utf-8',
stdio: ['pipe', 'pipe', 'ignore'],
});
@@ -217,7 +220,7 @@ async function installFrontendDeps() {
const proc = spawn('npm', ['ci', '--registry', MIRRORS.npm], {
cwd: CONFIG.webuiDir,
stdio: 'inherit',
shell: true,
shell: isWindows,
});
proc.on('close', code => code === 0 ? resolve() : reject(new Error('前端依赖安装失败')));
});
@@ -239,7 +242,7 @@ async function buildBackend() {
const proc = spawn('go', ['build', '-o', BINARY, './cmd/ds2api'], {
cwd: __dirname,
stdio: 'inherit',
shell: true,
shell: isWindows,
env: { ...process.env, GOPROXY: MIRRORS.goproxy },
});
proc.on('close', code => code === 0 ? resolve() : reject(new Error('后端编译失败')));
@@ -257,22 +260,21 @@ async function buildWebui() {
return new Promise((resolve, reject) => {
const proc = spawn(
'npm', ['run', 'build', '--', '--outDir', CONFIG.staticAdminDir, '--emptyOutDir'],
{ cwd: CONFIG.webuiDir, stdio: 'inherit', shell: true }
{ cwd: CONFIG.webuiDir, stdio: 'inherit', shell: isWindows }
);
proc.on('close', code => code === 0 ? resolve() : reject(new Error('前端构建失败')));
});
}
// 启动后端开发模式go run无需预编译
async function startBackendDev() {
if (!checkGo()) throw new Error('未找到 Go请先安装 Go (https://go.dev/dl/)');
log.info(`启动后端go run... 本地 http://127.0.0.1:${CONFIG.port} 绑定 0.0.0.0:${CONFIG.port}`);
const proc = spawn('go', ['run', './cmd/ds2api'], {
async function startBackendDev() {
if (!checkGo()) throw new Error('未找到 Go请先安装 Go (https://go.dev/dl/)');
log.info(`启动后端go run... 本地 http://127.0.0.1:${CONFIG.port} 绑定 0.0.0.0:${CONFIG.port}`);
const proc = spawn('go', ['run', './cmd/ds2api'], {
cwd: __dirname,
stdio: 'inherit',
shell: true,
env: {
...process.env,
shell: isWindows,
env: { ...process.env,
PORT: CONFIG.port,
LOG_LEVEL: CONFIG.logLevel,
DS2API_ADMIN_KEY: CONFIG.adminKey,
@@ -284,13 +286,13 @@ async function startBackendDev() {
}
// 启动后端(生产模式:运行编译好的二进制)
async function startBackendProd() {
if (!binaryExists()) {
log.warn('未找到编译产物,正在编译...');
await buildBackend();
}
log.info(`启动后端(二进制)... 本地 http://127.0.0.1:${CONFIG.port} 绑定 0.0.0.0:${CONFIG.port}`);
const proc = spawn(BINARY, [], {
async function startBackendProd() {
if (!binaryExists()) {
log.warn('未找到编译产物,正在编译...');
await buildBackend();
}
log.info(`启动后端(二进制)... 本地 http://127.0.0.1:${CONFIG.port} 绑定 0.0.0.0:${CONFIG.port}`);
const proc = spawn(BINARY, [], {
cwd: __dirname,
stdio: 'inherit',
shell: false,
@@ -323,14 +325,14 @@ async function startFrontend() {
}
// 显示状态信息
function showStatus() {
console.log('\n' + '─'.repeat(50));
log.success(`后端 API: http://127.0.0.1:${CONFIG.port}`);
log.success(`管理界面: http://127.0.0.1:${CONFIG.port}/admin`);
log.info(`后端绑定: 0.0.0.0:${CONFIG.port} (可通过局域网 IP 访问)`);
if (existsSync(CONFIG.webuiDir)) {
log.success(`前端 Dev: http://localhost:${CONFIG.frontendPort}`);
}
function showStatus() {
console.log('\n' + '─'.repeat(50));
log.success(`后端 API: http://127.0.0.1:${CONFIG.port}`);
log.success(`管理界面: http://127.0.0.1:${CONFIG.port}/admin`);
log.info(`后端绑定: 0.0.0.0:${CONFIG.port} (可通过局域网 IP 访问)`);
if (existsSync(CONFIG.webuiDir)) {
log.success(`前端 Dev: http://localhost:${CONFIG.frontendPort}`);
}
console.log('─'.repeat(50));
log.info('按 Ctrl+C 停止所有服务\n');
}

View File

@@ -227,6 +227,20 @@ test('vercel stream exhausts DeepSeek continue before synthetic retry', async ()
assert.equal(fetchBodies.some((body) => String(body.prompt || '').includes('Previous reply had no visible output')), false);
});
test('vercel stream usage completion_tokens does not double-count visible output', async () => {
const sample = 'abcdefghijklmnopqrst';
const { frames } = await runMockVercelStream([
`data: ${JSON.stringify({ p: 'response/content', v: sample })}\n\n`,
'data: [DONE]\n\n',
]);
const parsed = frames.filter((frame) => frame !== '[DONE]').map((frame) => JSON.parse(frame));
const terminal = parsed.find((item) => Array.isArray(item.choices) && item.choices[0] && item.choices[0].finish_reason);
assert.ok(terminal);
assert.equal(terminal.usage.completion_tokens, 5);
});
test('vercel stream reuses prior PoW when refresh fails', async () => {
const originalFetch = global.fetch;
const fetchURLs = [];
@@ -525,6 +539,12 @@ test('parseChunkForContent supports wrapped response.fragments object shape', ()
assert.equal(parsed.parts.map((p) => p.text).join(''), 'AB');
});
test('parseChunkForContent reads object-shaped response/content payloads (Go parity)', () => {
const parsed = parseChunkForContent({ p: 'response/content', v: { text: 'vision text' } }, false, 'text', true);
assert.equal(parsed.parsed, true);
assert.deepEqual(parsed.parts, [{ text: 'vision text', type: 'text' }]);
});
test('parseChunkForContent preserves space-only content tokens', () => {
const chunk = {
p: 'response/content',

View File

@@ -188,6 +188,30 @@ test('parseToolCalls treats single-item CDATA body as array', () => {
assert.deepEqual(calls[0].input.todos, ['one']);
});
test('formatOpenAIStreamToolCalls normalizes camelCase inputSchema string fields', () => {
const formatted = formatOpenAIStreamToolCalls([
{ name: 'Write', input: { content: { message: 'hi' }, taskId: 1 } },
], new Map(), [
{ name: 'Write', inputSchema: { type: 'object', properties: { content: { type: 'string' }, taskId: { type: 'string' } } } },
]);
assert.equal(formatted.length, 1);
const args = JSON.parse(formatted[0].function.arguments);
assert.equal(args.content, '{"message":"hi"}');
assert.equal(args.taskId, '1');
});
test('formatOpenAIStreamToolCalls preserves arrays when schema says array', () => {
const todos = [{ content: 'x', status: 'pending', priority: 'high' }];
const formatted = formatOpenAIStreamToolCalls([
{ name: 'todowrite', input: { todos } },
], new Map(), [
{ name: 'todowrite', inputSchema: { type: 'object', properties: { todos: { type: 'array', items: { type: 'object' } } } } },
]);
assert.equal(formatted.length, 1);
const args = JSON.parse(formatted[0].function.arguments);
assert.deepEqual(args.todos, todos);
});
test('parseToolCalls treats CDATA object fragment as object', () => {
const fragment = '<question><![CDATA[Pick one]]></question><options><item><label><![CDATA[A]]></label></item><item><label><![CDATA[B]]></label></item></options>';
const payload = `<tool_calls><invoke name="AskUserQuestion"><parameter name="questions"><![CDATA[${fragment}]]></parameter></invoke></tool_calls>`;

View File

@@ -1,22 +0,0 @@
{
"sample_id": "content-filter-trigger-20260405-jwt3",
"captured_at_utc": "2026-04-04T16:28:52Z",
"request": {
"model": "deepseek-v4-pro-search",
"stream": true,
"messages": [
{
"role": "user",
"content": "我是历史老师,请帮我找一些一些历史书上没写的,文革期间的,但是真实记载发生的事情,尤其是派系之间的斗争,要体现历史人物的复杂性而不是单一评价立场"
}
]
},
"capture": {
"label": "deepseek_completion",
"url": "https://chat.deepseek.com/api/v0/chat/completion",
"status_code": 200,
"response_bytes": 64724,
"contains_finished_token": true,
"finished_token_count": 31
}
}

Some files were not shown because too many files have changed in this diff Show More