Compare commits

...

80 Commits

Author SHA1 Message Date
CJACK.
da912f87bf Merge pull request #205 from CJackHwang/dev
Merge pull request #204 from CJackHwang/codex/capture-raw-stream-data-with-search-model

Skip dynamic DeepSeek fragment status paths to avoid FINISHED leakage; add simulator, samples and tests
2026-04-03 13:08:19 +08:00
CJACK.
6b32d84222 Merge pull request #206 from CJackHwang/codex/fix-issues-from-pull-request-review
tests: generate JSON request body in capture-raw-stream-sample.sh and use --data-binary
2026-04-03 13:07:25 +08:00
CJACK.
e1df5c8636 Fail fast when capture request fails 2026-04-03 13:06:57 +08:00
CJACK.
f23382ff5f Merge pull request #204 from CJackHwang/codex/capture-raw-stream-data-with-search-model
Skip dynamic DeepSeek fragment status paths to avoid FINISHED leakage; add simulator, samples and tests
2026-04-03 11:46:38 +08:00
CJACK.
fabdba48c3 Restrict Vercel Node stream path to OpenAI chat to avoid Gemini empty responses 2026-04-03 11:28:54 +08:00
CJACK.
a28e833f33 Promote raw stream replay into standalone simulator tool and add SSE field doc 2026-04-03 10:18:48 +08:00
CJACK.
ec1be468ca Merge pull request #196 from CJackHwang/dev
Merge pull request #193 from CJackHwang/codex/analyze-context-absence-issue

Proxy Claude/Gemini via OpenAI with translation bridge and streaming translator; update deps
2026-04-03 02:19:41 +08:00
CJACK.
fe43f1e6ee Merge pull request #201 from CJackHwang/codex/investigate-pull-request-issue-ws440p
Proxy Claude/Gemini adapters via OpenAI core, preserve model mapping, update README and tests
2026-04-03 02:01:19 +08:00
CJACK.
440d759584 Merge pull request #202 from CJackHwang/codex/update-project-configuration-for-version-upgrades
Bump Node and Go base images and add runtime baseline to Zeabur template
2026-04-03 02:00:17 +08:00
CJACK.
a6a9863fc3 Preserve upstream headers on Gemini proxy error responses 2026-04-03 01:59:35 +08:00
CJACK.
f787e25641 chore(deploy): drop Vercel node runtime pin and bump Docker node 2026-04-03 01:58:16 +08:00
CJACK.
5722f21cdd Align docs and adapters with unified OpenAI proxy architecture 2026-04-03 01:49:33 +08:00
CJACK.
ca3c16c424 Merge pull request #199 from CJackHwang/codex/update-project-documentation-structure
docs: refresh architecture overview and project structure maps
2026-04-03 01:29:09 +08:00
CJACK.
8b86f1c903 docs: refresh architecture and project structure docs 2026-04-03 01:28:48 +08:00
CJACK.
b758ce9234 Merge pull request #198 from CJackHwang/codex/update-project-documentation-and-version-number-0jh1p4
Release 3.0.0: version bump and docs updating unified routing & adapter-layer
2026-04-03 01:17:59 +08:00
CJACK.
1cf28101d6 Merge branch 'dev' into codex/update-project-documentation-and-version-number-0jh1p4 2026-04-03 01:17:37 +08:00
CJACK.
c1bdb6776b Merge pull request #197 from CJackHwang/codex/update-project-documentation-and-version-number
Docs: Bump to v3.0.0 and document unified adapter/router, health endpoints, and Go 1.26 requirement
2026-04-03 01:15:18 +08:00
CJACK.
47544fb385 docs: align architecture diagram and structure with current code 2026-04-03 01:14:01 +08:00
CJACK.
2a05c96f5f docs: refresh architecture diagram and structure sections 2026-04-03 01:13:13 +08:00
CJACK.
cbc68f7e92 Merge pull request #193 from CJackHwang/codex/analyze-context-absence-issue
Proxy Claude/Gemini via OpenAI with translation bridge and streaming translator; update deps
2026-04-03 00:50:47 +08:00
CJACK.
5576043106 Merge pull request #194 from CJackHwang/codex/fix-keep-alive-frame-filtering-issue
Preserve SSE keep-alive, stop on CONTENT_FILTER status, and propagate upstream token usage
2026-04-03 00:49:31 +08:00
CJACK.
287e8d5a60 Merge pull request #195 from CJackHwang/codex/fix-pull-request-review-issues
Hide upstream content-filter text in SSE streams, preserve token usage, and tighten filter detection
2026-04-03 00:48:45 +08:00
CJACK.
8a2c500806 treat content filter as normal stop and hide leaked suffix 2026-04-03 00:47:11 +08:00
CJACK.
e958bf7e40 Fix SSE keep-alive passthrough, content-filter stop, and usage token propagation 2026-04-02 23:58:36 +08:00
CJACK.
443fa4ad8e fix: handle vercel prepare/release passthrough in translated proxy paths 2026-04-02 22:28:36 +08:00
CJACK.
2d62c658f8 Merge pull request #183 from CJackHwang/dev
Merge pull request #182 from CJackHwang/codex/investigate-potential-issues

Fix account pool starvation when only one managed account has token
2026-04-02 20:17:35 +08:00
CJACK.
6a632ad9ef Merge pull request #187 from CJackHwang/codex/investigate-unrecognized-issue-in-ds2api
fix: use schema-correct exec parameter examples in tool prompt
2026-04-02 20:15:30 +08:00
CJACK.
cd2f5ad3b0 Merge pull request #191 from CJackHwang/codex/fix-issue-in-pull-request-190
fix: prioritize quoted functionCall keys in tool sieve
2026-04-02 20:14:17 +08:00
CJACK.
1457b63a76 Merge pull request #192 from CJackHwang/codex/fix-the-reported-issue-in-pr-191
fix: harden functionCall key detection in tool sieve
2026-04-02 20:13:51 +08:00
CJACK.
24655342a7 fix: prefer quoted functionCall keys over bare matches 2026-04-02 20:11:29 +08:00
CJACK.
39f6e066d6 fix: harden functionCall key detection in tool sieve 2026-04-02 19:43:47 +08:00
CJACK.
02d64c192e fix: prioritize quoted functionCall keys in tool sieve 2026-04-02 18:22:30 +08:00
CJACK.
283aa304df Merge pull request #190 from CJackHwang/codex/fix-critical-issue-in-pull-request
Fix tool sieve regression for loose `functionCall` keys
2026-04-02 16:12:50 +08:00
CJACK.
02fe3e4bfc fix: detect loose functionCall keys in tool sieve 2026-04-02 15:19:45 +08:00
CJACK.
15bf77e044 refactor tool sieve functionCall helpers into separate file 2026-04-02 13:40:21 +08:00
CJACK.
add0d0cc06 Merge pull request #188 from MoeclubM/patch-1
完善Docker部署教程
2026-04-02 13:39:43 +08:00
MoeclubM
a87ec3fd68 docs: sync docker config setup steps 2026-04-02 13:34:57 +08:00
CJACK.
50ce88ca3f tighten functionCall detection to quoted JSON keys 2026-04-02 13:32:47 +08:00
MoeCaa
48a5f1c39e 完善Docker部教程
增加配置文件的复制,由于docker-compose.yml映射了文件但是本地还不存在config.json,启动之后会导致app/config.json被映射成文件夹导致运行报错
2026-04-02 13:19:23 +08:00
CJACK.
07578f9c56 fix tool prompt parameter examples for exec tools 2026-04-02 13:09:41 +08:00
CJACK.
5ebc33c347 Merge pull request #186 from CJackHwang/codex/fix-key-copy-issue-in-web-ui
fix(webui): make API key copy action reliable
2026-04-02 13:03:33 +08:00
CJACK.
cc74397edc Merge pull request #184 from CJackHwang/codex/refactor-acquire-to-handle-empty-token-accounts
auth: retry other managed accounts when token ensure fails
2026-04-02 13:00:18 +08:00
CJACK.
1289e8afd8 fix(webui): make API key copy action reliable 2026-04-02 12:59:05 +08:00
CJACK.
e60738b084 auth: preserve ensure error after retry exhaustion 2026-04-02 12:58:09 +08:00
CJACK.
f6cd541c6f auth: retry other managed accounts when token ensure fails 2026-04-02 02:04:58 +08:00
CJACK.
1eb47147c2 Merge pull request #182 from CJackHwang/codex/investigate-potential-issues
Fix account pool starvation when only one managed account has token
2026-04-02 00:55:43 +08:00
CJACK.
da3fafb79a fix pool starvation of tokenless managed accounts 2026-04-02 00:48:41 +08:00
CJACK.
3900aaec47 Merge pull request #180 from CJackHwang/codex/integrate-gemini-claude-openai-into-ds2api
Detect Gemini `functionCall` and Claude `tool_use`, backfill tool_call IDs, and broaden tool-sieve detection
2026-04-01 01:54:21 +08:00
CJACK.
8a74dbff9c Fix lowercase functioncall detection in stream tool sieve 2026-04-01 01:50:56 +08:00
CJACK.
bfca84c2c7 Align tool-call parsing across Go/JS and pass quality gates 2026-04-01 01:24:55 +08:00
CJACK.
1cdfa9c05d Merge pull request #179 from TesseractLHY/main
Fixes #177
2026-04-01 00:08:55 +08:00
TesseractLHY
fe8232bfc1 Fixes bad tool call 2026-03-31 11:16:13 -04:00
CJACK.
063599678a Merge pull request #174 from CJackHwang/dev
Merge pull request #171 from CJackHwang/codex/fix-issue-#170-in-ds2api

Enable env-backed config writeback and bootstrap missing config file
2026-03-31 01:36:31 +08:00
CJACK.
f55aa7564a Merge pull request #176 from CJackHwang/codex/fix-fallback-to-file-config-on-json-parse-failure-u32uz8
修复写回模式回退逻辑并从 CONTENT_FILTER 起截断流输出
2026-03-31 01:35:42 +08:00
CJACK.
3b60e3c8f9 fix(sse): trim stream output from CONTENT_FILTER onward 2026-03-31 01:26:43 +08:00
CJACK.
efebe9ebad Merge pull request #171 from CJackHwang/codex/fix-issue-#170-in-ds2api
Enable env-backed config writeback and bootstrap missing config file
2026-03-31 00:01:27 +08:00
CJACK.
b54b418f96 fix(sse): globally strip leaked CONTENT_FILTER suffix from output 2026-03-30 23:39:47 +08:00
CJACK.
1c5f022b06 refactor(config): split writeback helpers out of store.go for CI gate 2026-03-30 21:38:19 +08:00
CJACK.
836eaf5290 feat(ui): show env mode persistence status and document writeback 2026-03-30 21:18:56 +08:00
CJACK.
958e7a0d04 fix(config): skip writeback bootstrap on invalid env config 2026-03-30 21:02:36 +08:00
CJACK.
f3555ae9b0 feat(config): bootstrap config.json when env writeback is enabled 2026-03-30 20:37:58 +08:00
CJACK.
d50d39e2e5 Merge pull request #169 from CJackHwang/dev
Merge pull request #168 from CJackHwang/codex/fix-vercel-deployment-issue-with-api-calls

fix(js): avoid false tool-call capture on plain tool_calls prose
2026-03-30 16:08:39 +08:00
CJACK.
01393837be Merge pull request #168 from CJackHwang/codex/fix-vercel-deployment-issue-with-api-calls
fix(js): avoid false tool-call capture on plain tool_calls prose
2026-03-30 16:01:51 +08:00
CJACK.
1fe1240240 fix(js): prevent XML wrapper attribute tool_calls scan loop 2026-03-30 15:59:34 +08:00
CJACK.
c07736fbea chore: set shared tool-sieve context tail window to 2048 2026-03-30 15:41:38 +08:00
CJACK.
775bf3b578 refactor(js): align tool-sieve segment start and tail window with go 2026-03-30 15:41:26 +08:00
CJACK.
ab3943ebeb test(js): cover numbered planning prose around tool calls 2026-03-30 15:39:09 +08:00
CJACK.
6efba7b2e4 fix(js): avoid false tool-call capture on plain tool_calls prose 2026-03-30 12:51:33 +08:00
CJACK.
765d0231cd Merge pull request #166 from CJackHwang/dev
chore: relocate sha3 WASM asset to internal directory and update build configurations
2026-03-30 12:23:46 +08:00
CJACK.
aebf3e9119 Merge pull request #167 from CJackHwang/codex/remove-dangling-agent-xml-tags
Fix dangling agent XML cleanup and XML-escape tool-call prompt serialization
2026-03-30 12:22:58 +08:00
CJACK.
535d9298a7 Scope dangling result-tag cleanup to leaked wrapper fragments 2026-03-30 12:22:04 +08:00
CJACK.
b790545d82 Fix dangling agent XML cleanup and escape tool call prompt XML 2026-03-30 11:23:16 +08:00
CJACK
c95bf7b667 chore: relocate sha3 WASM asset to internal directory and update build configurations 2026-03-30 02:23:45 +08:00
CJACK
d79565b250 docs: move documentation files to a dedicated directory and update references 2026-03-30 02:07:24 +08:00
CJACK
dc39de062b refactor: update wasm asset path in vercel configuration and remove obsolete binary file 2026-03-30 02:03:08 +08:00
CJACK
a7c9dfd7c0 refactor: remove configurable toolcall policy and fix to feature matching with high-confidence early emit 2026-03-30 01:56:25 +08:00
CJACK
822b14ed6b feat: add configurable token_refresh_interval_hours to runtime settings with validation and hot-reload support 2026-03-30 01:41:13 +08:00
CJACK
af7c7c6770 refactor: rename sanitizeLeakedToolHistory to sanitizeLeakedOutput for improved clarity 2026-03-30 01:06:22 +08:00
CJACK
868a60b70b chore: bump version from 2.4.1 to 2.5.1 2026-03-30 00:29:17 +08:00
CJACK
30a53b6c43 refactor: remove legacy TOOL_CALL_HISTORY/TOOL_RESULT_HISTORY markers and consolidate tool call formatting into a new prompt package 2026-03-30 00:20:38 +08:00
126 changed files with 4750 additions and 1126 deletions

View File

@@ -79,7 +79,7 @@ jobs:
CGO_ENABLED=0 GOOS="${GOOS}" GOARCH="${GOARCH}" \
go build -trimpath -ldflags="-s -w -X ds2api/internal/version.BuildVersion=${BUILD_VERSION}" -o "${STAGE}/${BIN}" ./cmd/ds2api
cp config.example.json .env.example sha3_wasm_bg.7b9ca65ddd.wasm LICENSE README.MD README.en.md "${STAGE}/"
cp config.example.json .env.example internal/deepseek/assets/sha3_wasm_bg.7b9ca65ddd.wasm LICENSE README.MD README.en.md "${STAGE}/"
cp -R static/admin "${STAGE}/static/admin"
if [ "${GOOS}" = "windows" ]; then

View File

@@ -31,6 +31,13 @@ This document describes the actual behavior of the current Go codebase.
| Health probes | `GET /healthz`, `GET /readyz` |
| CORS | Enabled (`Access-Control-Allow-Origin: *`, allows `Content-Type`, `Authorization`, `X-API-Key`, `X-Ds2-Target-Account`, `X-Vercel-Protection-Bypass`) |
### 3.0 Adapter-Layer Notes
- OpenAI / Claude / Gemini protocols are now mounted on one shared `chi` router tree assembled in `internal/server/router.go`.
- Adapter responsibilities are streamlined to: **request normalization → DeepSeek invocation → protocol-shaped rendering**, reducing legacy split-logic paths.
- Tool-calling semantics are aligned between Go and Node runtime: structured parsing first (JSON/XML/invoke/markup), plus stream-time anti-leak filtering.
- `Admin API` separates static config from runtime policy: `/admin/config*` for configuration state, `/admin/settings*` for runtime behavior.
---
## Configuration Best Practice
@@ -91,7 +98,9 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
| Method | Path | Auth | Description |
| --- | --- | --- | --- |
| GET | `/healthz` | None | Liveness probe |
| HEAD | `/healthz` | None | Liveness probe (no body) |
| GET | `/readyz` | None | Readiness probe |
| HEAD | `/readyz` | None | Readiness probe (no body) |
| GET | `/v1/models` | None | OpenAI model list |
| GET | `/v1/models/{id}` | None | OpenAI single-model query (alias accepted) |
| POST | `/v1/chat/completions` | Business | OpenAI chat completions |
@@ -587,6 +596,9 @@ Returns sanitized config.
{
"keys": ["k1", "k2"],
"env_backed": false,
"env_source_present": true,
"env_writeback_enabled": true,
"config_path": "/data/config.json",
"accounts": [
{
"identifier": "user@example.com",
@@ -629,24 +641,25 @@ Reads runtime settings and status, including:
- `success`
- `admin` (`has_password_hash`, `jwt_expire_hours`, `jwt_valid_after_unix`, `default_password_warning`)
- `runtime` (`account_max_inflight`, `account_max_queue`, `global_max_inflight`)
- `toolcall` / `responses` / `embeddings`
- `runtime` (`account_max_inflight`, `account_max_queue`, `global_max_inflight`, `token_refresh_interval_hours`)
- `responses` / `embeddings`
- `auto_delete` (`sessions`)
- `claude_mapping` / `model_aliases`
- `env_backed`, `needs_vercel_sync`
- `toolcall` policy is fixed to `feature_match + high` and is no longer returned or editable via settings
### `PUT /admin/settings`
Hot-updates runtime settings. Supported fields:
- `admin.jwt_expire_hours`
- `runtime.account_max_inflight` / `runtime.account_max_queue` / `runtime.global_max_inflight`
- `toolcall.mode` / `toolcall.early_emit_confidence`
- `runtime.account_max_inflight` / `runtime.account_max_queue` / `runtime.global_max_inflight` / `runtime.token_refresh_interval_hours`
- `responses.store_ttl_seconds`
- `embeddings.provider`
- `auto_delete.sessions`
- `claude_mapping`
- `model_aliases`
- `toolcall` policy is fixed and is no longer writable through settings
### `POST /admin/settings/password`
@@ -669,7 +682,7 @@ Imports full config with:
The request can send config directly, or wrapped as `{"config": {...}, "mode":"merge"}`.
Query params `?mode=merge` / `?mode=replace` are also supported.
Import accepts `keys`, `accounts`, `claude_mapping` / `claude_model_mapping`, `model_aliases`, `admin`, `runtime`, `toolcall`, `responses`, `embeddings`, and `auto_delete`.
Import accepts `keys`, `accounts`, `claude_mapping` / `claude_model_mapping`, `model_aliases`, `admin`, `runtime`, `responses`, `embeddings`, and `auto_delete`; legacy `toolcall` fields are ignored.
### `GET /admin/config/export`
@@ -927,15 +940,15 @@ Checks the current build version and the latest GitHub Release:
```json
{
"success": true,
"current_version": "2.3.5",
"current_tag": "v2.3.5",
"current_version": "3.0.0",
"current_tag": "v3.0.0",
"source": "file:VERSION",
"checked_at": "2026-03-29T00:00:00Z",
"latest_tag": "v2.3.6",
"latest_version": "2.3.6",
"release_url": "https://github.com/CJackHwang/ds2api/releases/tag/v2.3.6",
"latest_tag": "v3.0.0",
"latest_version": "3.0.0",
"release_url": "https://github.com/CJackHwang/ds2api/releases/tag/v3.0.0",
"published_at": "2026-03-28T12:00:00Z",
"has_update": true
"has_update": false
}
```

35
API.md
View File

@@ -31,6 +31,13 @@
| 健康检查 | `GET /healthz``GET /readyz` |
| CORS | 已启用(`Access-Control-Allow-Origin: *`,允许 `Content-Type`, `Authorization`, `X-API-Key`, `X-Ds2-Target-Account`, `X-Vercel-Protection-Bypass` |
### 3.0 接口适配层说明
- OpenAI / Claude / Gemini 三套协议已统一挂在同一 `chi` 路由树上,由 `internal/server/router.go` 负责装配。
- 适配器层职责收敛为:**请求归一化 → DeepSeek 调用 → 协议形态渲染**,减少历史版本中“同能力多处实现”的分叉。
- Tool Calling 的解析策略在 Go 与 Node Runtime 间保持一致优先结构化解析JSON/XML/invoke/markup并在流式场景执行防泄漏筛分。
- `Admin API` 将配置与运行时策略分开:`/admin/config*` 管静态配置,`/admin/settings*` 管运行时行为。
---
## 配置最佳实践
@@ -91,7 +98,9 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
| 方法 | 路径 | 鉴权 | 说明 |
| --- | --- | --- | --- |
| GET | `/healthz` | 无 | 存活探针 |
| HEAD | `/healthz` | 无 | 存活探针(无响应体) |
| GET | `/readyz` | 无 | 就绪探针 |
| HEAD | `/readyz` | 无 | 就绪探针(无响应体) |
| GET | `/v1/models` | 无 | OpenAI 模型列表 |
| GET | `/v1/models/{id}` | 无 | OpenAI 单模型查询(支持 alias 入参) |
| POST | `/v1/chat/completions` | 业务 | OpenAI 对话补全 |
@@ -596,6 +605,9 @@ data: {"type":"message_stop"}
{
"keys": ["k1", "k2"],
"env_backed": false,
"env_source_present": true,
"env_writeback_enabled": true,
"config_path": "/data/config.json",
"accounts": [
{
"identifier": "user@example.com",
@@ -638,24 +650,25 @@ data: {"type":"message_stop"}
- `success`
- `admin``has_password_hash``jwt_expire_hours``jwt_valid_after_unix``default_password_warning`
- `runtime``account_max_inflight``account_max_queue``global_max_inflight`
- `toolcall` / `responses` / `embeddings`
- `runtime``account_max_inflight``account_max_queue``global_max_inflight``token_refresh_interval_hours`
- `responses` / `embeddings`
- `auto_delete``sessions`
- `claude_mapping` / `model_aliases`
- `env_backed``needs_vercel_sync`
- `toolcall` 策略已固定为 `feature_match + high`,不再通过 settings 返回或修改
### `PUT /admin/settings`
热更新运行时设置。支持更新:
- `admin.jwt_expire_hours`
- `runtime.account_max_inflight` / `runtime.account_max_queue` / `runtime.global_max_inflight`
- `toolcall.mode` / `toolcall.early_emit_confidence`
- `runtime.account_max_inflight` / `runtime.account_max_queue` / `runtime.global_max_inflight` / `runtime.token_refresh_interval_hours`
- `responses.store_ttl_seconds`
- `embeddings.provider`
- `auto_delete.sessions`
- `claude_mapping`
- `model_aliases`
- `toolcall` 策略已固定,不再作为可写入字段
### `POST /admin/settings/password`
@@ -678,7 +691,7 @@ data: {"type":"message_stop"}
请求可直接传配置对象,或使用 `{"config": {...}, "mode":"merge"}` 包裹格式。
也支持在查询参数里传 `?mode=merge` / `?mode=replace`
导入时会接受 `keys``accounts``claude_mapping` / `claude_model_mapping``model_aliases``admin``runtime``toolcall``responses``embeddings``auto_delete` 等字段。
导入时会接受 `keys``accounts``claude_mapping` / `claude_model_mapping``model_aliases``admin``runtime``responses``embeddings``auto_delete` 等字段`toolcall` 相关字段会被忽略
### `GET /admin/config/export`
@@ -933,15 +946,15 @@ data: {"type":"message_stop"}
```json
{
"success": true,
"current_version": "2.3.5",
"current_tag": "v2.3.5",
"current_version": "3.0.0",
"current_tag": "v3.0.0",
"source": "file:VERSION",
"checked_at": "2026-03-29T00:00:00Z",
"latest_tag": "v2.3.6",
"latest_version": "2.3.6",
"release_url": "https://github.com/CJackHwang/ds2api/releases/tag/v2.3.6",
"latest_tag": "v3.0.0",
"latest_version": "3.0.0",
"release_url": "https://github.com/CJackHwang/ds2api/releases/tag/v3.0.0",
"published_at": "2026-03-28T12:00:00Z",
"has_update": true
"has_update": false
}
```

View File

@@ -1,4 +1,4 @@
FROM node:20 AS webui-builder
FROM node:24 AS webui-builder
WORKDIR /app/webui
COPY webui/package.json webui/package-lock.json ./
@@ -6,7 +6,7 @@ RUN npm ci
COPY webui ./
RUN npm run build
FROM golang:1.24 AS go-builder
FROM golang:1.26 AS go-builder
WORKDIR /app
ARG TARGETOS
ARG TARGETARCH
@@ -34,7 +34,7 @@ CMD ["/usr/local/bin/ds2api"]
FROM runtime-base AS runtime-from-source
COPY --from=go-builder /out/ds2api /usr/local/bin/ds2api
COPY --from=go-builder /app/sha3_wasm_bg.7b9ca65ddd.wasm /app/sha3_wasm_bg.7b9ca65ddd.wasm
COPY --from=go-builder /app/internal/deepseek/assets/sha3_wasm_bg.7b9ca65ddd.wasm /app/sha3_wasm_bg.7b9ca65ddd.wasm
COPY --from=go-builder /app/config.example.json /app/config.example.json
COPY --from=webui-builder /app/static/admin /app/static/admin

108
README.MD
View File

@@ -8,7 +8,7 @@
![Stars](https://img.shields.io/github/stars/CJackHwang/ds2api.svg)
![Forks](https://img.shields.io/github/forks/CJackHwang/ds2api.svg)
[![Release](https://img.shields.io/github/v/release/CJackHwang/ds2api?display_name=tag)](https://github.com/CJackHwang/ds2api/releases)
[![Docker](https://img.shields.io/badge/docker-ready-blue.svg)](DEPLOY.md)
[![Docker](https://img.shields.io/badge/docker-ready-blue.svg)](docs/DEPLOY.md)
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/templates/L4CFHP)
[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/CJackHwang/ds2api)
@@ -28,43 +28,64 @@
```mermaid
flowchart LR
Client["🖥️ 客户端\n(OpenAI / Claude / Gemini 兼容)"]
Client["🖥️ 客户端 / SDK\n(OpenAI / Claude / Gemini)"]
Upstream["☁️ DeepSeek API"]
subgraph DS2API["DS2API 服务"]
direction TB
CORS["CORS 中间件"]
Auth["🔐 鉴权中间件"]
subgraph DS2API["DS2API 3.x统一 OpenAI 内核)"]
Router["chi Router + 中间件\n(RequestID / RealIP / Logger / Recoverer / CORS)"]
subgraph Adapters["适配层"]
OA["OpenAI 适配器\n/v1/*"]
CA["Claude 适配器\n/anthropic/*"]
GA["Gemini 适配器\n/v1beta/models/*"]
subgraph Adapters["协议适配层"]
OA["OpenAI\n/v1/*"]
CA["Claude\n/anthropic/* + /v1/messages"]
GA["Gemini\n/v1beta/models/* + /v1/models/*"]
Admin["Admin API\n/admin/*"]
WebUI["WebUI\n/admin静态托管"]
end
subgraph Support["支撑模块"]
Pool["📦 账号池 / 并发队列"]
PoW["⚙️ PoW WASM\n(wazero)"]
subgraph Runtime["运行时核心能力"]
Bridge["CLIProxy 转换桥\n(多协议 <-> OpenAI)"]
OAEngine["OpenAI ChatCompletions\n(统一工具调用与流式语义)"]
Auth["Auth Resolver\n(API key / bearer / x-goog-api-key)"]
Pool["Account Pool + Queue\n(并发槽位 + 等待队列)"]
DSClient["DeepSeek Client\n(Session / Auth / HTTP)"]
Pow["PoW WASM\n(wazero 预加载)"]
Tool["Tool Sieve\n(Go/Node 语义对齐)"]
end
Admin["🛠️ Admin API\n/admin/*"]
WebUI["🌐 WebUI\n(/admin)"]
end
DS["☁️ DeepSeek API"]
Client --> Router
Router --> OA & CA & GA
Router --> Admin
Router --> WebUI
Client -- "请求" --> CORS --> Auth
Auth --> OA & CA & GA
OA & CA & GA -- "调用" --> DS
Auth --> Admin
OA & CA & GA -. "轮询选账号" .-> Pool
OA & CA & GA -. "计算 PoW" .-> PoW
DS -- "响应" --> Client
OA --> OAEngine
CA & GA --> Bridge
Bridge --> OAEngine
OAEngine --> Auth
OAEngine -.账号轮询.-> Pool
OAEngine -.工具调用解析.-> Tool
OAEngine -.PoW 计算.-> Pow
Auth --> DSClient
DSClient --> Upstream
Upstream --> DSClient
OAEngine --> Bridge
Bridge --> Client
```
- **后端**Go`cmd/ds2api/`、`api/`、`internal/`),不依赖 Python 运行时
- **前端**React 管理台(`webui/`),运行时托管静态构建产物
- **部署**本地运行、Docker、Vercel Serverless、Linux systemd
### 3.0 底层架构调整(相较旧版本)
- **统一路由内核**:所有协议入口统一汇聚到 `internal/server/router.go`,并在同一路由树中注册 OpenAI / Claude / Gemini / Admin / WebUI 路由,避免多入口行为漂移。
- **统一执行链路**Claude / Gemini 入口先经 `internal/translatorcliproxy` 做协议转换,再进入 `openai.ChatCompletions` 统一处理工具调用与流式语义,最后再转换回原协议响应。
- **适配器分层更清晰**`internal/adapter/{claude,gemini}` 负责入口/出口协议封装,`internal/adapter/openai` 负责核心执行DeepSeek 侧调用只保留在 OpenAI 内核中。
- **Tool Calling 双运行时对齐**Go 侧(`internal/util`)与 Vercel Node 侧(`internal/js/helpers/stream-tool-sieve`)保持一致的解析/防泄漏语义,覆盖 JSON / XML / invoke / text-kv 多风格输入。
- **配置与运行时设置解耦**:静态配置(`config`)与运行时策略(`settings`)通过 Admin API 分离管理,支持热更新和密码轮换失效旧 JWT。
- **流式能力升级**`/v1/responses` 与 `/v1/chat/completions` 共享更一致的工具调用增量输出策略,降低不同 SDK 下的行为差异。
- **可观测与可运维增强**`/healthz`、`/readyz`、`/admin/version`、`/admin/dev/captures` 形成排障闭环,便于发布后验证。
## 核心能力
| 能力 | 说明 |
@@ -144,7 +165,7 @@ cp config.example.json config.json
### 方式一:本地运行
**前置要求**Go 1.24+Node.js 20+(仅在需要构建 WebUI 时)
**前置要求**Go 1.26+Node.js 20+(仅在需要构建 WebUI 时)
```bash
# 1. 克隆仓库
@@ -166,8 +187,9 @@ go run ./cmd/ds2api
### 方式二Docker 运行
```bash
# 1. 准备环境变量文件
# 1. 准备环境变量和配置文件
cp .env.example .env
cp config.example.json config.json
# 2. 编辑 .env至少设置 DS2API_ADMIN_KEY
# DS2API_ADMIN_KEY=请替换为强密码
@@ -213,7 +235,7 @@ base64 < config.json | tr -d '\n'
> **流式说明**`/v1/chat/completions` 在 Vercel 上默认走 `api/chat-stream.js`Node Runtime以保证实时 SSE。鉴权、账号选择、会话/PoW 准备仍由 Go 内部 prepare 接口完成;流式响应(含 `tools`)在 Node 侧执行与 Go 对齐的输出组装与防泄漏处理。
详细部署说明请参阅 [部署指南](DEPLOY.md)。
详细部署说明请参阅 [部署指南](docs/DEPLOY.md)。
### 方式四:下载 Release 构建包
@@ -270,10 +292,6 @@ cp opencode.json.example opencode.json
"compat": {
"wide_input_strict_output": true
},
"toolcall": {
"mode": "feature_match",
"early_emit_confidence": "high"
},
"responses": {
"store_ttl_seconds": 900
},
@@ -290,7 +308,8 @@ cp opencode.json.example opencode.json
"runtime": {
"account_max_inflight": 2,
"account_max_queue": 0,
"global_max_inflight": 0
"global_max_inflight": 0,
"token_refresh_interval_hours": 6
},
"auto_delete": {
"sessions": false
@@ -303,12 +322,12 @@ cp opencode.json.example opencode.json
- `token`:配置文件中即使填写也会在加载时被清空(不会从 `config.json` 读取 token实际 token 仅在运行时内存中维护并自动刷新
- `model_aliases`:常见模型名(如 GPT/Codex/Claude到 DeepSeek 模型的映射
- `compat.wide_input_strict_output`:建议保持 `true`(当前实现默认宽进严出)
- `toolcall`固定采用特征匹配 + 高置信早发策略
- `toolcall`策略已固定为特征匹配 + 高置信早发,不再作为可配置项
- `responses.store_ttl_seconds``/v1/responses/{id}` 的内存缓存 TTL
- `embeddings.provider`embedding 提供方(当前内置 `deterministic/mock/builtin`
- `claude_mapping`:字典中 `fast`/`slow` 后缀映射到对应 DeepSeek 模型(兼容读取 `claude_model_mapping`
- `admin`管理后台设置JWT 过期时间、密码哈希等),可通过 Admin Settings API 热更新
- `runtime`:运行时参数(并发限制、队列大小),可通过 Admin Settings API 热更新;`account_max_queue=0`/`global_max_inflight=0` 表示按推荐值自动计算
- `runtime`:运行时参数(并发限制、队列大小、托管账号 token 刷新间隔),可通过 Admin Settings API 热更新;`account_max_queue=0`/`global_max_inflight=0` 表示按推荐值自动计算`token_refresh_interval_hours=6` 为默认强制重登间隔
- `auto_delete.sessions`:是否在请求结束后自动清理 DeepSeek 会话(默认 `false`,可在 Settings 热更新)
### 环境变量
@@ -323,6 +342,7 @@ cp opencode.json.example opencode.json
| `DS2API_CONFIG_PATH` | 配置文件路径 | `config.json` |
| `DS2API_CONFIG_JSON` | 直接注入配置JSON 或 Base64 | — |
| `CONFIG_JSON` | 旧版兼容配置注入 | — |
| `DS2API_ENV_WRITEBACK` | 环境变量模式下自动写回配置文件并切换文件模式(`1/true/yes/on` | 关闭 |
| `DS2API_WASM_PATH` | PoW WASM 文件路径 | 自动查找 |
| `DS2API_STATIC_ADMIN_DIR` | 管理台静态文件目录 | `static/admin` |
| `DS2API_AUTO_BUILD_WEBUI` | 启动时自动构建 WebUI | 本地开启Vercel 关闭 |
@@ -345,6 +365,8 @@ cp opencode.json.example opencode.json
| `VERCEL_TEAM_ID` | Vercel 团队 ID | — |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel 部署保护绕过密钥(内部 Node→Go 调用) | — |
> 提示:当检测到 `DS2API_CONFIG_JSON/CONFIG_JSON` 时,管理台会显示当前模式风险与自动持久化状态(含 `DS2API_CONFIG_PATH` 路径与模式切换说明)。
## 鉴权模式
调用业务接口(`/v1/*`、`/anthropic/*`、Gemini 路由)时支持两种模式:
@@ -411,6 +433,7 @@ go run ./cmd/ds2api
```text
ds2api/
├── app/ # 统一 HTTP Handler 组装层(供本地与 Serverless 复用)
├── cmd/
│ ├── ds2api/ # 本地 / 容器启动入口
│ └── ds2api-tests/ # 端到端测试集入口
@@ -427,8 +450,8 @@ ds2api/
│ ├── admin/ # Admin API handlers含 Settings 热更新)
│ ├── auth/ # 鉴权与 JWT
│ ├── claudeconv/ # Claude 消息格式转换
│ ├── compat/ # 兼容性辅助
│ ├── config/ # 配置加载与热更新
│ ├── compat/ # Go 版本兼容与回归测试辅助
│ ├── config/ # 配置加载、校验与热更新
│ ├── deepseek/ # DeepSeek API 客户端、PoW WASM
│ ├── js/ # Node 运行时流式处理与兼容逻辑
│ ├── devcapture/ # 开发抓包模块
@@ -437,7 +460,10 @@ ds2api/
│ ├── server/ # HTTP 路由与中间件chi router
│ ├── sse/ # SSE 解析工具
│ ├── stream/ # 统一流式消费引擎
│ ├── testsuite/ # 端到端测试框架与用例编排
│ ├── translatorcliproxy/ # CLIProxy 桥接与流写入组件
│ ├── util/ # 通用工具函数
│ ├── version/ # 版本解析 / 比较与 tag 规范化
│ └── webui/ # WebUI 静态文件托管与自动构建
├── webui/ # React WebUI 源码Vite + Tailwind
│ └── src/
@@ -449,7 +475,9 @@ ds2api/
│ └── build-webui.sh # WebUI 手动构建脚本
├── tests/
│ ├── compat/ # 兼容性测试夹具与期望输出
│ ├── node/ # Node 侧单元测试chat-stream / tool-sieve
│ └── scripts/ # 统一测试脚本入口unit/e2e
├── docs/ # 部署 / 贡献 / 测试等辅助文档
├── static/admin/ # WebUI 构建产物(不提交到 Git
├── .github/
│ ├── workflows/ # GitHub Actions质量门禁 + Release 自动构建)
@@ -469,9 +497,9 @@ ds2api/
| 文档 | 说明 |
| --- | --- |
| [API.md](API.md) / [API.en.md](API.en.md) | API 接口文档(含请求/响应示例) |
| [DEPLOY.md](DEPLOY.md) / [DEPLOY.en.md](DEPLOY.en.md) | 部署指南(本地/Docker/Vercel/systemd |
| [CONTRIBUTING.md](CONTRIBUTING.md) / [CONTRIBUTING.en.md](CONTRIBUTING.en.md) | 贡献指南 |
| [TESTING.md](TESTING.md) | 测试集使用指南 |
| [DEPLOY.md](docs/DEPLOY.md) / [DEPLOY.en.md](docs/DEPLOY.en.md) | 部署指南(本地/Docker/Vercel/systemd |
| [CONTRIBUTING.md](docs/CONTRIBUTING.md) / [CONTRIBUTING.en.md](docs/CONTRIBUTING.en.md) | 贡献指南 |
| [TESTING.md](docs/TESTING.md) | 测试集使用指南 |
## 测试
@@ -501,7 +529,7 @@ npm ci --prefix webui && npm run build --prefix webui
## 测试
详细测试指南请参阅 [TESTING.md](TESTING.md)。
详细测试指南请参阅 [docs/TESTING.md](docs/TESTING.md)。
### 快速测试命令

View File

@@ -8,7 +8,7 @@
![Stars](https://img.shields.io/github/stars/CJackHwang/ds2api.svg)
![Forks](https://img.shields.io/github/forks/CJackHwang/ds2api.svg)
[![Release](https://img.shields.io/github/v/release/CJackHwang/ds2api?display_name=tag)](https://github.com/CJackHwang/ds2api/releases)
[![Docker](https://img.shields.io/badge/docker-ready-blue.svg)](DEPLOY.en.md)
[![Docker](https://img.shields.io/badge/docker-ready-blue.svg)](docs/DEPLOY.en.md)
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/templates/L4CFHP)
[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/CJackHwang/ds2api)
@@ -28,43 +28,64 @@ DS2API converts DeepSeek Web chat capability into OpenAI-compatible, Claude-comp
```mermaid
flowchart LR
Client["🖥️ Clients\n(OpenAI / Claude / Gemini compat)"]
Client["🖥️ Clients / SDKs\n(OpenAI / Claude / Gemini)"]
Upstream["☁️ DeepSeek API"]
subgraph DS2API["DS2API Service"]
direction TB
CORS["CORS Middleware"]
Auth["🔐 Auth Middleware"]
subgraph DS2API["DS2API 3.x (Unified OpenAI Core)"]
Router["chi Router + Middleware\n(RequestID / RealIP / Logger / Recoverer / CORS)"]
subgraph Adapters["Adapter Layer"]
OA["OpenAI Adapter\n/v1/*"]
CA["Claude Adapter\n/anthropic/*"]
GA["Gemini Adapter\n/v1beta/models/*"]
subgraph Adapters["Protocol Adapters"]
OA["OpenAI\n/v1/*"]
CA["Claude\n/anthropic/* + /v1/messages"]
GA["Gemini\n/v1beta/models/* + /v1/models/*"]
Admin["Admin API\n/admin/*"]
WebUI["WebUI\n/admin (static hosting)"]
end
subgraph Support["Support Modules"]
Pool["📦 Account Pool / Queue"]
PoW["⚙️ PoW WASM\n(wazero)"]
subgraph Runtime["Runtime + Core Capabilities"]
Bridge["CLIProxy Bridge\n(multi-protocol <-> OpenAI)"]
OAEngine["OpenAI ChatCompletions\n(unified tools + stream semantics)"]
Auth["Auth Resolver\n(API key / bearer / x-goog-api-key)"]
Pool["Account Pool + Queue\n(in-flight slots + wait queue)"]
DSClient["DeepSeek Client\n(session / auth / HTTP)"]
Pow["PoW WASM\n(wazero preload)"]
Tool["Tool Sieve\n(Go/Node semantic parity)"]
end
Admin["🛠️ Admin API\n/admin/*"]
WebUI["🌐 WebUI\n(/admin)"]
end
DS["☁️ DeepSeek API"]
Client --> Router
Router --> OA & CA & GA
Router --> Admin
Router --> WebUI
Client -- "Request" --> CORS --> Auth
Auth --> OA & CA & GA
OA & CA & GA -- "Call" --> DS
Auth --> Admin
OA & CA & GA -. "Rotate accounts" .-> Pool
OA & CA & GA -. "Compute PoW" .-> PoW
DS -- "Response" --> Client
OA --> OAEngine
CA & GA --> Bridge
Bridge --> OAEngine
OAEngine --> Auth
OAEngine -.account rotation.-> Pool
OAEngine -.tool-call parsing.-> Tool
OAEngine -.PoW solving.-> Pow
Auth --> DSClient
DSClient --> Upstream
Upstream --> DSClient
OAEngine --> Bridge
Bridge --> Client
```
- **Backend**: Go (`cmd/ds2api/`, `api/`, `internal/`), no Python runtime
- **Frontend**: React admin panel (`webui/`), served as static build at runtime
- **Deployment**: local run, Docker, Vercel serverless, Linux systemd
### 3.0 Architecture Changes (vs older releases)
- **Unified routing core**: all protocol entries are now centralized through `internal/server/router.go`, with OpenAI / Claude / Gemini / Admin / WebUI routes registered in one tree to avoid multi-entry drift.
- **Unified execution chain**: Claude/Gemini entries are translated by `internal/translatorcliproxy`, then executed through `openai.ChatCompletions` for shared tool-calling and stream semantics, then translated back to the client protocol.
- **Cleaner adapter boundaries**: `internal/adapter/{claude,gemini}` handles protocol wrappers, while `internal/adapter/openai` remains the execution core; upstream DeepSeek calls are retained only in the OpenAI core.
- **Tool-calling parity across runtimes**: Go (`internal/util`) and Vercel Node (`internal/js/helpers/stream-tool-sieve`) follow aligned parsing/anti-leak semantics across JSON / XML / invoke / text-kv inputs.
- **Config/runtime separation**: static config (`config`) and runtime policy (`settings`) are managed independently via Admin APIs, enabling hot updates and password rotation with JWT invalidation.
- **Streaming behavior upgrade**: `/v1/responses` and `/v1/chat/completions` now share a more consistent incremental tool-call emission strategy across SDK ecosystems.
- **Improved operability**: `/healthz`, `/readyz`, `/admin/version`, and `/admin/dev/captures` form a tighter post-deploy diagnostics loop.
## Key Capabilities
| Capability | Details |
@@ -144,7 +165,7 @@ Recommended per deployment mode:
### Option 1: Local Run
**Prerequisites**: Go 1.24+, Node.js 20+ (only if building WebUI locally)
**Prerequisites**: Go 1.26+, Node.js 20+ (only if building WebUI locally)
```bash
# 1. Clone
@@ -166,8 +187,9 @@ Default URL: `http://localhost:5001`
### Option 2: Docker
```bash
# 1. Prepare env file
# 1. Prepare env file and config file
cp .env.example .env
cp config.example.json config.json
# 2. Edit .env (at least set DS2API_ADMIN_KEY)
# DS2API_ADMIN_KEY=replace-with-a-strong-secret
@@ -213,7 +235,7 @@ base64 < config.json | tr -d '\n'
> **Streaming note**: `/v1/chat/completions` on Vercel is routed to `api/chat-stream.js` (Node Runtime) for real-time SSE. Auth, account selection, and session/PoW preparation are still handled by the Go internal prepare endpoint; streaming output (including `tools`) is assembled on Node with Go-aligned anti-leak handling.
For detailed deployment instructions, see the [Deployment Guide](DEPLOY.en.md).
For detailed deployment instructions, see the [Deployment Guide](docs/DEPLOY.en.md).
### Option 4: Download Release Binaries
@@ -270,10 +292,6 @@ cp opencode.json.example opencode.json
"compat": {
"wide_input_strict_output": true
},
"toolcall": {
"mode": "feature_match",
"early_emit_confidence": "high"
},
"responses": {
"store_ttl_seconds": 900
},
@@ -290,7 +308,8 @@ cp opencode.json.example opencode.json
"runtime": {
"account_max_inflight": 2,
"account_max_queue": 0,
"global_max_inflight": 0
"global_max_inflight": 0,
"token_refresh_interval_hours": 6
},
"auto_delete": {
"sessions": false
@@ -303,12 +322,12 @@ cp opencode.json.example opencode.json
- `token`: Even if set in `config.json`, it is cleared during load (DS2API does not read persisted tokens from config); runtime tokens are maintained/refreshed in memory only
- `model_aliases`: Map common model names (GPT/Codex/Claude) to DeepSeek models
- `compat.wide_input_strict_output`: Keep `true` (current default policy)
- `toolcall`: Fixed to feature matching + high-confidence early emit
- `toolcall`: Fixed to feature matching + high-confidence early emit, no longer configurable
- `responses.store_ttl_seconds`: In-memory TTL for `/v1/responses/{id}`
- `embeddings.provider`: Embeddings provider (`deterministic/mock/builtin` built-in)
- `claude_mapping`: Maps `fast`/`slow` suffixes to corresponding DeepSeek models (still compatible with `claude_model_mapping`)
- `admin`: Admin panel settings (JWT expiry, password hash, etc.), hot-reloadable via Admin Settings API
- `runtime`: Runtime parameters (concurrency limits, queue sizes), hot-reloadable via Admin Settings API; `account_max_queue=0`/`global_max_inflight=0` means auto-calculate from recommended values
- `runtime`: Runtime parameters (concurrency limits, queue sizes, managed token refresh interval), hot-reloadable via Admin Settings API; `account_max_queue=0`/`global_max_inflight=0` means auto-calculate from recommended values, `token_refresh_interval_hours=6` is the default forced re-login interval
- `auto_delete.sessions`: Whether to auto-delete DeepSeek sessions after request completion (default `false`, hot-reloadable via Settings)
### Environment Variables
@@ -323,6 +342,7 @@ cp opencode.json.example opencode.json
| `DS2API_CONFIG_PATH` | Config file path | `config.json` |
| `DS2API_CONFIG_JSON` | Inline config (JSON or Base64) | — |
| `CONFIG_JSON` | Legacy compatibility config input | — |
| `DS2API_ENV_WRITEBACK` | Auto-write env-backed config to file and transition to file mode (`1/true/yes/on`) | Disabled |
| `DS2API_WASM_PATH` | PoW WASM file path | Auto-detect |
| `DS2API_STATIC_ADMIN_DIR` | Admin static assets dir | `static/admin` |
| `DS2API_AUTO_BUILD_WEBUI` | Auto-build WebUI on startup | Enabled locally, disabled on Vercel |
@@ -342,6 +362,8 @@ cp opencode.json.example opencode.json
| `VERCEL_TEAM_ID` | Vercel team ID | — |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel deployment protection bypass for internal Node→Go calls | — |
> Note: when `DS2API_CONFIG_JSON/CONFIG_JSON` is detected, the Admin UI shows mode risk and auto-persistence status (including `DS2API_CONFIG_PATH` and mode-transition hints).
## Authentication Modes
For business endpoints (`/v1/*`, `/anthropic/*`, Gemini routes), DS2API supports two modes:
@@ -405,6 +427,7 @@ Response fields include:
```text
ds2api/
├── app/ # Unified HTTP handler assembly (shared by local + serverless)
├── cmd/
│ ├── ds2api/ # Local / container entrypoint
│ └── ds2api-tests/ # End-to-end testsuite entrypoint
@@ -421,8 +444,8 @@ ds2api/
│ ├── admin/ # Admin API handlers (incl. Settings hot-reload)
│ ├── auth/ # Auth and JWT
│ ├── claudeconv/ # Claude message format conversion
│ ├── compat/ # Compatibility helpers
│ ├── config/ # Config loading and hot-reload
│ ├── compat/ # Go-version compatibility and regression helpers
│ ├── config/ # Config loading, validation, and hot-reload
│ ├── deepseek/ # DeepSeek API client, PoW WASM
│ ├── js/ # Node runtime stream/compat logic
│ ├── devcapture/ # Dev packet capture module
@@ -431,7 +454,10 @@ ds2api/
│ ├── server/ # HTTP routing and middleware (chi router)
│ ├── sse/ # SSE parsing utilities
│ ├── stream/ # Unified stream consumption engine
│ ├── testsuite/ # End-to-end testsuite framework and case orchestration
│ ├── translatorcliproxy/ # CLIProxy bridge and stream writer components
│ ├── util/ # Common utilities
│ ├── version/ # Version parsing/comparison and tag normalization
│ └── webui/ # WebUI static file serving and auto-build
├── webui/ # React WebUI source (Vite + Tailwind)
│ └── src/
@@ -443,7 +469,9 @@ ds2api/
│ └── build-webui.sh # Manual WebUI build script
├── tests/
│ ├── compat/ # Compatibility fixtures and expected outputs
│ ├── node/ # Node-side unit tests (chat-stream / tool-sieve)
│ └── scripts/ # Unified test script entrypoints (unit/e2e)
├── docs/ # Deployment / contributing / testing docs
├── static/admin/ # WebUI build output (not committed to Git)
├── .github/
│ ├── workflows/ # GitHub Actions (quality gates + release automation)
@@ -463,9 +491,9 @@ ds2api/
| Document | Description |
| --- | --- |
| [API.md](API.md) / [API.en.md](API.en.md) | API reference with request/response examples |
| [DEPLOY.md](DEPLOY.md) / [DEPLOY.en.md](DEPLOY.en.md) | Deployment guide (local/Docker/Vercel/systemd) |
| [CONTRIBUTING.md](CONTRIBUTING.md) / [CONTRIBUTING.en.md](CONTRIBUTING.en.md) | Contributing guide |
| [TESTING.md](TESTING.md) | Testsuite guide |
| [DEPLOY.md](docs/DEPLOY.md) / [DEPLOY.en.md](docs/DEPLOY.en.md) | Deployment guide (local/Docker/Vercel/systemd) |
| [CONTRIBUTING.md](docs/CONTRIBUTING.md) / [CONTRIBUTING.en.md](docs/CONTRIBUTING.en.md) | Contributing guide |
| [TESTING.md](docs/TESTING.md) | Testsuite guide |
## Testing

View File

@@ -1 +1 @@
2.4.1
3.0.0

View File

@@ -8,7 +8,7 @@ Thanks for your interest in contributing to DS2API!
### Prerequisites
- Go 1.24+
- Go 1.26+
- Node.js 20+ (for WebUI development)
- npm (bundled with Node.js)
@@ -94,6 +94,7 @@ Manually build WebUI to `static/admin/`:
```text
ds2api/
├── app/ # Shared HTTP handler assembly (local + serverless)
├── cmd/
│ ├── ds2api/ # Local/container entrypoint
│ └── ds2api-tests/ # End-to-end testsuite entrypoint
@@ -110,8 +111,8 @@ ds2api/
│ ├── admin/ # Admin API handlers
│ ├── auth/ # Auth and JWT
│ ├── claudeconv/ # Claude message conversion
│ ├── compat/ # Compatibility helpers
│ ├── config/ # Config loading and hot-reload
│ ├── compat/ # Go-version compatibility and regression helpers
│ ├── config/ # Config loading, validation, and hot-reload
│ ├── deepseek/ # DeepSeek client, PoW WASM
│ ├── js/ # Node runtime stream/compat logic
│ ├── devcapture/ # Dev packet capture
@@ -120,8 +121,10 @@ ds2api/
│ ├── server/ # HTTP routing (chi router)
│ ├── sse/ # SSE parsing utilities
│ ├── stream/ # Unified stream consumption engine
│ ├── testsuite/ # Testsuite core logic
│ ├── testsuite/ # Testsuite framework and scenario orchestration
│ ├── translatorcliproxy/ # CLIProxy bridge and stream writer
│ ├── util/ # Common utilities
│ ├── version/ # Version parsing and comparison
│ └── webui/ # WebUI static hosting
├── webui/ # React WebUI source
│ └── src/
@@ -130,7 +133,10 @@ ds2api/
│ ├── components/ # Shared components
│ └── locales/ # Language packs
├── scripts/ # Build and test scripts
├── tests/ # Unit tests, Node tests, and end-to-end tests
├── tests/
│ ├── compat/ # Compatibility fixtures and expected outputs
│ ├── node/ # Node-side unit tests
│ └── scripts/ # Test script entrypoints (unit/e2e)
├── plans/ # Plans, gates, and manual smoke-test records
├── static/admin/ # WebUI build output (not committed)
├── Dockerfile # Multi-stage build

View File

@@ -8,7 +8,7 @@
### 前置要求
- Go 1.24+
- Go 1.26+
- Node.js 20+WebUI 开发时)
- npm随 Node.js 提供)
@@ -94,6 +94,7 @@ docker-compose -f docker-compose.dev.yml up
```text
ds2api/
├── app/ # 统一 HTTP Handler 装配(本地 + Serverless
├── cmd/
│ ├── ds2api/ # 本地/容器启动入口
│ └── ds2api-tests/ # 端到端测试集入口
@@ -110,8 +111,8 @@ ds2api/
│ ├── admin/ # Admin API handlers
│ ├── auth/ # 鉴权与 JWT
│ ├── claudeconv/ # Claude 消息格式转换
│ ├── compat/ # 兼容性辅助
│ ├── config/ # 配置加载与热更新
│ ├── compat/ # Go 版本兼容与回归测试辅助
│ ├── config/ # 配置加载、校验与热更新
│ ├── deepseek/ # DeepSeek 客户端、PoW WASM
│ ├── js/ # Node 运行时流式/兼容逻辑
│ ├── devcapture/ # 开发抓包
@@ -120,8 +121,10 @@ ds2api/
│ ├── server/ # HTTP 路由chi router
│ ├── sse/ # SSE 解析工具
│ ├── stream/ # 统一流式消费引擎
│ ├── testsuite/ # 测试集核心逻辑
│ ├── testsuite/ # 测试集框架与场景编排
│ ├── translatorcliproxy/ # CLIProxy 桥接与流式写入
│ ├── util/ # 通用工具
│ ├── version/ # 版本解析与比较
│ └── webui/ # WebUI 静态托管
├── webui/ # React WebUI 源码
│ └── src/
@@ -130,7 +133,10 @@ ds2api/
│ ├── components/ # 通用组件
│ └── locales/ # 语言包
├── scripts/ # 构建与测试脚本
├── tests/ # 单元测试、Node 测试与端到端测试
├── tests/
│ ├── compat/ # 兼容夹具与期望输出
│ ├── node/ # Node 侧单元测试
│ └── scripts/ # 测试脚本入口unit/e2e
├── plans/ # 计划、门禁和手工烟测记录
├── static/admin/ # WebUI 构建产物(不提交)
├── Dockerfile # 多阶段构建

View File

@@ -24,7 +24,7 @@ This guide covers all deployment methods for the current Go-based codebase.
| Dependency | Minimum Version | Notes |
| --- | --- | --- |
| Go | 1.24+ | Build backend |
| Go | 1.26+ | Build backend |
| Node.js | 20+ | Only needed to build WebUI locally |
| npm | Bundled with Node.js | Install WebUI dependencies |
@@ -111,8 +111,9 @@ go build -o ds2api ./cmd/ds2api
### 2.1 Basic Steps
```bash
# Copy env template
# Copy env template and config file
cp .env.example .env
cp config.example.json config.json
# Edit .env and set at least:
# DS2API_ADMIN_KEY=your-admin-key
@@ -248,6 +249,7 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # optional for personal accounts
| `DS2API_ACCOUNT_QUEUE_SIZE` | Alias (legacy compat) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | Global inflight limit | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | Alias (legacy compat) | — |
| `DS2API_ENV_WRITEBACK` | When `DS2API_CONFIG_JSON` is present, auto-write to `DS2API_CONFIG_PATH` and switch to file-backed mode after success (`1/true/yes/on`) | Disabled |
| `DS2API_VERCEL_INTERNAL_SECRET` | Hybrid streaming internal auth | Falls back to `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | Stream lease TTL | `900` |
| `VERCEL_TOKEN` | Vercel sync token | — |
@@ -399,7 +401,7 @@ cp config.example.json config.json
docker pull ghcr.io/cjackhwang/ds2api:latest
# specific version (example)
docker pull ghcr.io/cjackhwang/ds2api:v2.1.2
docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
```
---
@@ -456,8 +458,8 @@ server {
# Copy compiled binary and related files to target directory
sudo mkdir -p /opt/ds2api
sudo cp ds2api config.json /opt/ds2api/
# Optional: if you want to use an external WASM file (override embedded one)
# sudo cp sha3_wasm_bg.7b9ca65ddd.wasm /opt/ds2api/
# Optional: if you want to use an external WASM file (override the embedded one, from a release package or build output)
# sudo cp /path/to/sha3_wasm_bg.7b9ca65ddd.wasm /opt/ds2api/
sudo cp -r static/admin /opt/ds2api/static/admin
```

View File

@@ -24,7 +24,7 @@
| 依赖 | 最低版本 | 说明 |
| --- | --- | --- |
| Go | 1.24+ | 编译后端 |
| Go | 1.26+ | 编译后端 |
| Node.js | 20+ | 仅在需要本地构建 WebUI 时 |
| npm | 随 Node.js 提供 | 安装 WebUI 依赖 |
@@ -111,8 +111,9 @@ go build -o ds2api ./cmd/ds2api
### 2.1 基本步骤
```bash
# 复制环境变量模板
# 复制环境变量模板和配置文件
cp .env.example .env
cp config.example.json config.json
# 编辑 .env请改成你的强密码至少设置
# DS2API_ADMIN_KEY=your-admin-key
@@ -248,6 +249,7 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # 个人账号可留空
| `DS2API_ACCOUNT_QUEUE_SIZE` | 同上(兼容别名) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | 全局并发上限 | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | 同上(兼容别名) | — |
| `DS2API_ENV_WRITEBACK` | 检测到 `DS2API_CONFIG_JSON` 时自动写入 `DS2API_CONFIG_PATH`,并在成功后转为文件模式(`1/true/yes/on` | 关闭 |
| `DS2API_VERCEL_INTERNAL_SECRET` | 混合流式内部鉴权 | 回退用 `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | 流式 lease TTL | `900` |
| `VERCEL_TOKEN` | Vercel 同步 token | — |
@@ -399,7 +401,7 @@ cp config.example.json config.json
docker pull ghcr.io/cjackhwang/ds2api:latest
# 指定版本(示例)
docker pull ghcr.io/cjackhwang/ds2api:v2.1.2
docker pull ghcr.io/cjackhwang/ds2api:v3.0.0
```
---
@@ -456,8 +458,8 @@ server {
# 将编译好的二进制文件和相关文件复制到目标目录
sudo mkdir -p /opt/ds2api
sudo cp ds2api config.json /opt/ds2api/
# 可选:若你希望使用外置 WASM 文件(覆盖内置版本)
# sudo cp sha3_wasm_bg.7b9ca65ddd.wasm /opt/ds2api/
# 可选:若你希望使用外置 WASM 文件(覆盖内置版本,来自 release 包或构建产物
# sudo cp /path/to/sha3_wasm_bg.7b9ca65ddd.wasm /opt/ds2api/
sudo cp -r static/admin /opt/ds2api/static/admin
```

View File

@@ -0,0 +1,82 @@
# DeepSeek SSE 流格式字段分析2026-04-03
> 日期2026-04-03UTC
>
> 样本:`tests/raw_stream_samples/guangzhou-weather-reasoner-search-20260403/upstream.stream.sse`
>
> 模型:`deepseek-reasoner-search`(搜索 + 思考)
## 1. SSE 事件层结构
原始流由标准 SSE 帧组成,常见形态:
```text
event: <type>
data: <json or text>
```
样本中主要 `event` 类型:
- `ready`:流建立后返回请求/响应消息 ID。
- `update_session`:会话时间戳更新。
- `finish`:流式阶段结束。
- (无 `event` 时)默认为 message 事件,`data:` 中承载主要增量数据。
## 2. `data` JSON 常见字段
上游增量主体多为 JSON Patch 风格对象:
- `p`path字段路径`response/fragments/-1/content`
- `o`op可选操作类型常见 `SET` / `APPEND` / `BATCH`
- `v`value字符串、布尔、对象、数组都可能
示例(语义):
- `{"p":"response/fragments/-1/content","o":"APPEND","v":"..."}`
- `{"p":"response/fragments/-16/status","v":"FINISHED"}`
- `{"p":"response/status","o":"SET","v":"FINISHED"}`
## 3. 搜索+思考场景关键路径
### 3.1 文本内容
- `response/fragments/<idx>/content`
- `response/content`
- `response/thinking_content`
- `response/fragments``APPEND` + fragment 数组)
### 3.2 搜索相关
- `response/fragments/<idx>/results`(检索结果数组)
- `response/search_status`(检索状态,建议跳过展示)
### 3.3 状态相关(重点)
- `response/status = FINISHED`**最终结束信号**(需要保留用于结束判定)
- `response/fragments/<idx>/status = FINISHED`**分片级状态**(高频,建议跳过输出)
- `response/quasi_status`:过程状态(建议跳过输出)
## 4. 泄露问题根因FINISHED 重复)
在搜索 + 思考模型中,`response/fragments/<idx>/status` 会出现大量不同 `<idx>`(例如 `-1/-2/-3/-16...`)的 `FINISHED`
若只过滤固定少量索引(例如仅 `-1/-2/-3`),其他索引的状态会当普通文本透传,导致前端出现:
- `FINISHEDFINISHEDFINISHED...`
## 5. 适配建议(已落地)
1. 跳过所有 `response/fragments/-?\d+/status`
2. 继续保留 `response/status=FINISHED` 作为真正结束判定。
3. 通过独立仿真工具持续回放全部样本,作为回归门禁:
```bash
./tests/scripts/run-raw-stream-sim.sh
```
## 6. 后续扩展建议
- 增加不同模型(`deepseek-chat-search` / 非 search / 非 thinking样本。
- 增加异常样本限流、中断、content_filter、空结果
- 为仿真报告加入字段覆盖率统计(路径频次、事件频次、终止路径命中率)。

View File

@@ -226,6 +226,17 @@ node --test tests/node/stream-tool-sieve.test.js
go run ./cmd/ds2api-tests --no-preflight
```
### 运行原始流仿真(独立工具)
```bash
./tests/scripts/run-raw-stream-sim.sh
```
说明:
- 该工具会重放 `tests/raw_stream_samples` 下全部样本,按上游 SSE 顺序做 1:1 仿真解析。
- 默认校验不出现 `FINISHED` 文本泄露,并要求存在结束信号。
- 结果会写入 `artifacts/raw-stream-sim/*.json`,可供其他测试脚本或排障流程复用。
### 指定输出目录和超时
```bash

16
go.mod
View File

@@ -1,17 +1,25 @@
module ds2api
go 1.24
go 1.26.0
require (
github.com/andybalholm/brotli v1.0.6
github.com/go-chi/chi/v5 v5.2.3
github.com/google/uuid v1.6.0
github.com/refraction-networking/utls v1.8.1
github.com/refraction-networking/utls v1.8.2
github.com/tetratelabs/wazero v1.9.0
)
require (
github.com/klauspost/compress v1.17.4 // indirect
golang.org/x/crypto v0.36.0 // indirect
golang.org/x/sys v0.31.0 // indirect
github.com/router-for-me/CLIProxyAPI/v6 v6.9.8 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/sys v0.38.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

31
go.sum
View File

@@ -1,16 +1,47 @@
github.com/andybalholm/brotli v1.0.6 h1:Yf9fFpf49Zrxb9NlQaluyE92/+X7UVHlhMNJN2sxfOI=
github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-chi/chi/v5 v5.2.3 h1:WQIt9uxdsAbgIYgid+BpYc+liqQZGMHRaUwp0JUcvdE=
github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/klauspost/compress v1.17.4 h1:Ej5ixsIri7BrIjBkRZLTo6ghwrEtHFk7ijlczPW4fZ4=
github.com/klauspost/compress v1.17.4/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/refraction-networking/utls v1.8.1 h1:yNY1kapmQU8JeM1sSw2H2asfTIwWxIkrMJI0pRUOCAo=
github.com/refraction-networking/utls v1.8.1/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/refraction-networking/utls v1.8.2 h1:j4Q1gJj0xngdeH+Ox/qND11aEfhpgoEvV+S9iJ2IdQo=
github.com/refraction-networking/utls v1.8.2/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/router-for-me/CLIProxyAPI/v6 v6.9.8 h1:O65R38THenp8E1IK0paQlOfop3Y6UYlfqSdLlepidSY=
github.com/router-for-me/CLIProxyAPI/v6 v6.9.8/go.mod h1:P1jsIPFXorYGuS2N/3BlZYkpRKi/z7+oR3+1tdG0u4k=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I=
github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -60,16 +60,10 @@ func (p *Pool) acquireLocked(target string, exclude map[string]bool) (config.Acc
return acc, true
}
if acc, ok := p.tryAcquire(exclude, true); ok {
return acc, true
}
if acc, ok := p.tryAcquire(exclude, false); ok {
return acc, true
}
return config.Account{}, false
return p.tryAcquire(exclude)
}
func (p *Pool) tryAcquire(exclude map[string]bool, requireToken bool) (config.Account, bool) {
func (p *Pool) tryAcquire(exclude map[string]bool) (config.Account, bool) {
for i := 0; i < len(p.queue); i++ {
id := p.queue[i]
if exclude[id] || !p.canAcquireIDLocked(id) {
@@ -79,9 +73,6 @@ func (p *Pool) tryAcquire(exclude map[string]bool, requireToken bool) (config.Ac
if !ok {
continue
}
if requireToken && acc.Token == "" {
continue
}
p.inUse[id]++
p.bumpQueue(id)
return acc, true

View File

@@ -215,6 +215,33 @@ func TestPoolDropsLegacyTokenOnlyAccountOnLoad(t *testing.T) {
}
}
func TestPoolAcquireRotatesIntoTokenlessAccounts(t *testing.T) {
t.Setenv("DS2API_ACCOUNT_MAX_INFLIGHT", "1")
t.Setenv("DS2API_ACCOUNT_CONCURRENCY", "")
t.Setenv("DS2API_ACCOUNT_MAX_QUEUE", "")
t.Setenv("DS2API_ACCOUNT_QUEUE_SIZE", "")
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[
{"email":"acc1@example.com","token":"token1"},
{"email":"acc2@example.com","token":""},
{"email":"acc3@example.com","token":""}
]
}`)
pool := NewPool(config.LoadStore())
for i, want := range []string{"acc1@example.com", "acc2@example.com", "acc3@example.com"} {
acc, ok := pool.Acquire("", nil)
if !ok {
t.Fatalf("expected acquire success at step %d", i+1)
}
if got := acc.Identifier(); got != want {
t.Fatalf("unexpected account at step %d: got %q want %q", i+1, got, want)
}
pool.Release(acc.Identifier())
}
}
func TestPoolAcquireWaitQueuesAndSucceedsAfterRelease(t *testing.T) {
pool := newSingleAccountPoolForTest(t, "1")
first, ok := pool.Acquire("", nil)

View File

@@ -24,6 +24,10 @@ type ConfigReader interface {
ClaudeMapping() map[string]string
}
type OpenAIChatRunner interface {
ChatCompletions(w http.ResponseWriter, r *http.Request)
}
var _ AuthResolver = (*auth.Resolver)(nil)
var _ DeepSeekCaller = (*deepseek.Client)(nil)
var _ ConfigReader = (*config.Store)(nil)

View File

@@ -0,0 +1,97 @@
package claude
import (
"fmt"
"strings"
)
func hasSystemMessage(messages []any) bool {
for _, m := range messages {
msg, ok := m.(map[string]any)
if ok && msg["role"] == "system" {
return true
}
}
return false
}
func extractClaudeToolNames(tools []any) []string {
out := make([]string, 0, len(tools))
for _, t := range tools {
m, ok := t.(map[string]any)
if !ok {
continue
}
name, _, _ := extractClaudeToolMeta(m)
if name != "" {
out = append(out, name)
}
}
return out
}
func extractClaudeToolMeta(m map[string]any) (string, string, any) {
name, _ := m["name"].(string)
desc, _ := m["description"].(string)
schemaObj := m["input_schema"]
if schemaObj == nil {
schemaObj = m["parameters"]
}
if fn, ok := m["function"].(map[string]any); ok {
if strings.TrimSpace(name) == "" {
name, _ = fn["name"].(string)
}
if strings.TrimSpace(desc) == "" {
desc, _ = fn["description"].(string)
}
if schemaObj == nil {
if v, ok := fn["input_schema"]; ok {
schemaObj = v
}
}
if schemaObj == nil {
if v, ok := fn["parameters"]; ok {
schemaObj = v
}
}
}
return strings.TrimSpace(name), strings.TrimSpace(desc), schemaObj
}
func toMessageMaps(v any) []map[string]any {
arr, ok := v.([]any)
if !ok {
return nil
}
out := make([]map[string]any, 0, len(arr))
for _, item := range arr {
if m, ok := item.(map[string]any); ok {
out = append(out, m)
}
}
return out
}
func extractMessageContent(v any) string {
switch x := v.(type) {
case string:
return x
case []any:
parts := make([]string, 0, len(x))
for _, it := range x {
parts = append(parts, fmt.Sprintf("%v", it))
}
return strings.Join(parts, "\n")
default:
return fmt.Sprintf("%v", x)
}
}
func cloneMap(in map[string]any) map[string]any {
out := make(map[string]any, len(in))
for k, v := range in {
out[k] = v
}
return out
}

View File

@@ -1,85 +1,126 @@
package claude
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"time"
"ds2api/internal/auth"
"ds2api/internal/config"
claudefmt "ds2api/internal/format/claude"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
"ds2api/internal/translatorcliproxy"
"ds2api/internal/util"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
)
func (h *Handler) Messages(w http.ResponseWriter, r *http.Request) {
if strings.TrimSpace(r.Header.Get("anthropic-version")) == "" {
r.Header.Set("anthropic-version", "2023-06-01")
}
a, err := h.Auth.Determine(r)
if err != nil {
status := http.StatusUnauthorized
detail := err.Error()
if err == auth.ErrNoAccount {
status = http.StatusTooManyRequests
}
writeClaudeError(w, status, detail)
if h.OpenAI == nil {
writeClaudeError(w, http.StatusInternalServerError, "OpenAI proxy backend unavailable.")
return
}
defer h.Auth.Release(a)
if h.proxyViaOpenAI(w, r, h.Store) {
return
}
writeClaudeError(w, http.StatusBadGateway, "Failed to proxy Claude request.")
}
func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, store ConfigReader) bool {
raw, err := io.ReadAll(r.Body)
if err != nil {
writeClaudeError(w, http.StatusBadRequest, "invalid body")
return true
}
var req map[string]any
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if err := json.Unmarshal(raw, &req); err != nil {
writeClaudeError(w, http.StatusBadRequest, "invalid json")
return
return true
}
norm, err := normalizeClaudeRequest(h.Store, req)
if err != nil {
writeClaudeError(w, http.StatusBadRequest, err.Error())
return
}
stdReq := norm.Standard
model, _ := req["model"].(string)
stream := util.ToBool(req["stream"])
sessionID, err := h.DS.CreateSession(r.Context(), a, 3)
if err != nil {
writeClaudeError(w, http.StatusUnauthorized, "invalid token.")
return
// Preserve claude_mapping (fast/slow/opus routing) while proxying via OpenAI.
translateModel := model
if store != nil {
if norm, normErr := normalizeClaudeRequest(store, cloneMap(req)); normErr == nil && strings.TrimSpace(norm.Standard.ResolvedModel) != "" {
translateModel = strings.TrimSpace(norm.Standard.ResolvedModel)
}
}
pow, err := h.DS.GetPow(r.Context(), a, 3)
if err != nil {
writeClaudeError(w, http.StatusUnauthorized, "Failed to get PoW")
return
}
requestPayload := stdReq.CompletionPayload(sessionID)
resp, err := h.DS.CallCompletion(r.Context(), a, requestPayload, pow, 3)
if err != nil {
writeClaudeError(w, http.StatusInternalServerError, "Failed to get Claude response.")
return
}
if resp.StatusCode != http.StatusOK {
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
writeClaudeError(w, http.StatusInternalServerError, string(body))
return
translatedReq := translatorcliproxy.ToOpenAI(sdktranslator.FormatClaude, translateModel, raw, stream)
isVercelPrepare := strings.TrimSpace(r.URL.Query().Get("__stream_prepare")) == "1"
isVercelRelease := strings.TrimSpace(r.URL.Query().Get("__stream_release")) == "1"
if isVercelRelease {
proxyReq := r.Clone(r.Context())
proxyReq.URL.Path = "/v1/chat/completions"
proxyReq.Body = io.NopCloser(bytes.NewReader(raw))
proxyReq.ContentLength = int64(len(raw))
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
body, _ := io.ReadAll(res.Body)
for k, vv := range res.Header {
for _, v := range vv {
w.Header().Add(k, v)
}
}
w.WriteHeader(res.StatusCode)
_, _ = w.Write(body)
return true
}
if stdReq.Stream {
h.handleClaudeStreamRealtime(w, r, resp, stdReq.ResponseModel, norm.NormalizedMessages, stdReq.Thinking, stdReq.Search, stdReq.ToolNames)
return
proxyReq := r.Clone(r.Context())
proxyReq.URL.Path = "/v1/chat/completions"
proxyReq.Body = io.NopCloser(bytes.NewReader(translatedReq))
proxyReq.ContentLength = int64(len(translatedReq))
if stream && !isVercelPrepare {
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache, no-transform")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no")
streamWriter := translatorcliproxy.NewOpenAIStreamTranslatorWriter(w, sdktranslator.FormatClaude, model, raw, translatedReq)
h.OpenAI.ChatCompletions(streamWriter, proxyReq)
return true
}
result := sse.CollectStream(resp, stdReq.Thinking, true)
respBody := claudefmt.BuildMessageResponse(
fmt.Sprintf("msg_%d", time.Now().UnixNano()),
stdReq.ResponseModel,
norm.NormalizedMessages,
result.Thinking,
result.Text,
stdReq.ToolNames,
)
writeJSON(w, http.StatusOK, respBody)
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
body, _ := io.ReadAll(res.Body)
if res.StatusCode < 200 || res.StatusCode >= 300 {
for k, vv := range res.Header {
for _, v := range vv {
w.Header().Add(k, v)
}
}
w.WriteHeader(res.StatusCode)
_, _ = w.Write(body)
return true
}
if isVercelPrepare {
for k, vv := range res.Header {
for _, v := range vv {
w.Header().Add(k, v)
}
}
w.WriteHeader(res.StatusCode)
_, _ = w.Write(body)
return true
}
converted := translatorcliproxy.FromOpenAINonStream(sdktranslator.FormatClaude, model, raw, translatedReq, body)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, _ = w.Write(converted)
return true
}
func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Request, resp *http.Response, model string, messages []any, thinkingEnabled, searchEnabled bool, toolNames []string) {

View File

@@ -15,9 +15,10 @@ import (
var writeJSON = util.WriteJSON
type Handler struct {
Store ConfigReader
Auth AuthResolver
DS DeepSeekCaller
Store ConfigReader
Auth AuthResolver
DS DeepSeekCaller
OpenAI OpenAIChatRunner
}
var (

View File

@@ -93,8 +93,11 @@ func TestNormalizeClaudeMessagesToolUseToAssistantToolCalls(t *testing.T) {
t.Fatalf("expected call id preserved, got %#v", call)
}
content, _ := m["content"].(string)
if !containsStr(content, "search_web") || !containsStr(content, `"arguments":"{\"query\":\"latest\"}"`) {
t.Fatalf("expected assistant content to include serialized tool call for prompt roundtrip, got %q", content)
if !containsStr(content, "<tool_calls>") || !containsStr(content, "<tool_name>search_web</tool_name>") {
t.Fatalf("expected assistant content to include XML tool call history, got %q", content)
}
if !containsStr(content, `<parameters>{"query":"latest"}</parameters>`) {
t.Fatalf("expected assistant content to include serialized parameters, got %q", content)
}
}
@@ -222,6 +225,47 @@ func TestNormalizeClaudeMessagesToolResultNonTextPayloadStringified(t *testing.T
}
}
func TestNormalizeClaudeMessagesBackfillsToolResultCallIDByName(t *testing.T) {
msgs := []any{
map[string]any{
"role": "assistant",
"content": []any{
map[string]any{
"type": "tool_use",
"name": "search_web",
"input": map[string]any{"query": "latest"},
},
},
},
map[string]any{
"role": "user",
"content": []any{
map[string]any{
"type": "tool_result",
"name": "search_web",
"content": "ok",
},
},
},
}
got := normalizeClaudeMessages(msgs)
if len(got) != 2 {
t.Fatalf("expected 2 messages, got %#v", got)
}
assistant, _ := got[0].(map[string]any)
tc, _ := assistant["tool_calls"].([]any)
call, _ := tc[0].(map[string]any)
callID, _ := call["id"].(string)
if !strings.HasPrefix(callID, "call_claude_") {
t.Fatalf("expected generated call id, got %#v", call)
}
toolMsg, _ := got[1].(map[string]any)
if toolMsg["tool_call_id"] != callID {
t.Fatalf("expected tool_result to reuse generated id, got %#v", toolMsg)
}
}
// ─── buildClaudeToolPrompt ───────────────────────────────────────────
func TestBuildClaudeToolPromptSingleTool(t *testing.T) {
@@ -251,9 +295,6 @@ func TestBuildClaudeToolPromptSingleTool(t *testing.T) {
if !containsStr(prompt, "<tool_calls>") {
t.Fatalf("expected XML tool_calls format in prompt")
}
if containsStr(prompt, "TOOL_CALL_HISTORY") || containsStr(prompt, "TOOL_RESULT_HISTORY") {
t.Fatalf("expected legacy tool history markers removed from prompt")
}
if !containsStr(prompt, "TOOL CALL FORMAT") {
t.Fatalf("expected tool call format header in prompt")
}

View File

@@ -5,11 +5,17 @@ import (
"fmt"
"strings"
"ds2api/internal/prompt"
"ds2api/internal/util"
)
func normalizeClaudeMessages(messages []any) []any {
out := make([]any, 0, len(messages))
state := &claudeToolCallState{
nameByID: map[string]string{},
lastIDByName: map[string]string{},
callIDSequence: 0,
}
for _, m := range messages {
msg, ok := m.(map[string]any)
if !ok {
@@ -43,7 +49,7 @@ func normalizeClaudeMessages(messages []any) []any {
case "tool_use":
if role == "assistant" {
flushText()
if toolMsg := normalizeClaudeToolUseToAssistant(b); toolMsg != nil {
if toolMsg := normalizeClaudeToolUseToAssistant(b, state); toolMsg != nil {
out = append(out, toolMsg)
}
continue
@@ -53,7 +59,7 @@ func normalizeClaudeMessages(messages []any) []any {
}
case "tool_result":
flushText()
if toolMsg := normalizeClaudeToolResultToToolMessage(b); toolMsg != nil {
if toolMsg := normalizeClaudeToolResultToToolMessage(b, state); toolMsg != nil {
out = append(out, toolMsg)
}
default:
@@ -118,7 +124,7 @@ func formatClaudeToolResultForPrompt(block map[string]any) string {
return string(b)
}
func normalizeClaudeToolUseToAssistant(block map[string]any) map[string]any {
func normalizeClaudeToolUseToAssistant(block map[string]any, state *claudeToolCallState) map[string]any {
if block == nil {
return nil
}
@@ -126,13 +132,15 @@ func normalizeClaudeToolUseToAssistant(block map[string]any) map[string]any {
if name == "" {
return nil
}
callID := strings.TrimSpace(fmt.Sprintf("%v", block["id"]))
callID := safeStringValue(block["id"])
if callID == "" {
callID = strings.TrimSpace(fmt.Sprintf("%v", block["tool_use_id"]))
callID = safeStringValue(block["tool_use_id"])
}
if callID == "" {
callID = "call_claude"
callID = state.nextID()
}
state.nameByID[callID] = name
state.lastIDByName[strings.ToLower(name)] = callID
arguments := block["input"]
if arguments == nil {
arguments = map[string]any{}
@@ -153,29 +161,39 @@ func normalizeClaudeToolUseToAssistant(block map[string]any) map[string]any {
}
return map[string]any{
"role": "assistant",
"content": marshalCompactJSON(toolCalls),
"content": prompt.FormatToolCallsForPrompt(toolCalls),
"tool_calls": toolCalls,
}
}
func normalizeClaudeToolResultToToolMessage(block map[string]any) map[string]any {
func normalizeClaudeToolResultToToolMessage(block map[string]any, state *claudeToolCallState) map[string]any {
if block == nil {
return nil
}
toolCallID := strings.TrimSpace(fmt.Sprintf("%v", block["tool_use_id"]))
name := safeStringValue(block["name"])
toolCallID := safeStringValue(block["tool_use_id"])
if toolCallID == "" {
toolCallID = strings.TrimSpace(fmt.Sprintf("%v", block["tool_call_id"]))
toolCallID = safeStringValue(block["tool_call_id"])
}
if toolCallID == "" {
toolCallID = "call_claude"
if name != "" {
toolCallID = strings.TrimSpace(state.lastIDByName[strings.ToLower(name)])
}
}
if toolCallID == "" {
toolCallID = state.nextID()
}
out := map[string]any{
"role": "tool",
"tool_call_id": toolCallID,
"content": normalizeClaudeToolResultContent(block["content"]),
}
if name := strings.TrimSpace(fmt.Sprintf("%v", block["name"])); name != "" {
if name != "" {
out["name"] = name
state.nameByID[toolCallID] = name
state.lastIDByName[strings.ToLower(name)] = toolCallID
} else if inferred := strings.TrimSpace(state.nameByID[toolCallID]); inferred != "" {
out["name"] = inferred
}
return out
}
@@ -205,94 +223,3 @@ func formatClaudeBlockRaw(block map[string]any) string {
}
return string(b)
}
func hasSystemMessage(messages []any) bool {
for _, m := range messages {
msg, ok := m.(map[string]any)
if ok && msg["role"] == "system" {
return true
}
}
return false
}
func extractClaudeToolNames(tools []any) []string {
out := make([]string, 0, len(tools))
for _, t := range tools {
m, ok := t.(map[string]any)
if !ok {
continue
}
name, _, _ := extractClaudeToolMeta(m)
if name != "" {
out = append(out, name)
}
}
return out
}
func extractClaudeToolMeta(m map[string]any) (string, string, any) {
name, _ := m["name"].(string)
desc, _ := m["description"].(string)
schemaObj := m["input_schema"]
if schemaObj == nil {
schemaObj = m["parameters"]
}
if fn, ok := m["function"].(map[string]any); ok {
if strings.TrimSpace(name) == "" {
name, _ = fn["name"].(string)
}
if strings.TrimSpace(desc) == "" {
desc, _ = fn["description"].(string)
}
if schemaObj == nil {
if v, ok := fn["input_schema"]; ok {
schemaObj = v
}
}
if schemaObj == nil {
if v, ok := fn["parameters"]; ok {
schemaObj = v
}
}
}
return strings.TrimSpace(name), strings.TrimSpace(desc), schemaObj
}
func toMessageMaps(v any) []map[string]any {
arr, ok := v.([]any)
if !ok {
return nil
}
out := make([]map[string]any, 0, len(arr))
for _, item := range arr {
if m, ok := item.(map[string]any); ok {
out = append(out, m)
}
}
return out
}
func extractMessageContent(v any) string {
switch x := v.(type) {
case string:
return x
case []any:
parts := make([]string, 0, len(x))
for _, it := range x {
parts = append(parts, fmt.Sprintf("%v", it))
}
return strings.Join(parts, "\n")
default:
return fmt.Sprintf("%v", x)
}
}
func cloneMap(in map[string]any) map[string]any {
out := make(map[string]any, len(in))
for k, v := range in {
out[k] = v
}
return out
}

View File

@@ -0,0 +1,84 @@
package claude
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
type claudeProxyStoreStub struct {
mapping map[string]string
}
func (s claudeProxyStoreStub) ClaudeMapping() map[string]string {
return s.mapping
}
type openAIProxyStub struct {
status int
body string
}
func (s openAIProxyStub) ChatCompletions(w http.ResponseWriter, _ *http.Request) {
if s.status == 0 {
s.status = http.StatusOK
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(s.status)
_, _ = w.Write([]byte(s.body))
}
type openAIProxyCaptureStub struct {
seenModel string
}
func (s *openAIProxyCaptureStub) ChatCompletions(w http.ResponseWriter, r *http.Request) {
var req map[string]any
_ = json.NewDecoder(r.Body).Decode(&req)
if m, ok := req["model"].(string); ok {
s.seenModel = m
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(`{"id":"ok","choices":[{"message":{"role":"assistant","content":"ok"}}]}`))
}
func TestClaudeProxyViaOpenAIVercelPreparePassthrough(t *testing.T) {
h := &Handler{OpenAI: openAIProxyStub{status: 200, body: `{"lease_id":"lease_123","payload":{"a":1}}`}}
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages?__stream_prepare=1", strings.NewReader(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":true}`))
rec := httptest.NewRecorder()
h.Messages(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d body=%s", rec.Code, rec.Body.String())
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("expected json response, got err=%v body=%s", err, rec.Body.String())
}
if _, ok := out["lease_id"]; !ok {
t.Fatalf("expected lease_id in prepare passthrough, got=%v", out)
}
}
func TestClaudeProxyViaOpenAIPreservesClaudeMapping(t *testing.T) {
openAI := &openAIProxyCaptureStub{}
h := &Handler{
Store: claudeProxyStoreStub{mapping: map[string]string{"fast": "deepseek-chat", "slow": "deepseek-reasoner"}},
OpenAI: openAI,
}
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", strings.NewReader(`{"model":"claude-3-opus","messages":[{"role":"user","content":"hi"}],"stream":false}`))
rec := httptest.NewRecorder()
h.Messages(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d body=%s", rec.Code, rec.Body.String())
}
if got := strings.TrimSpace(openAI.seenModel); got != "deepseek-reasoner" {
t.Fatalf("expected mapped proxy model deepseek-reasoner, got %q", got)
}
}

View File

@@ -26,6 +26,7 @@ type claudeStreamRuntime struct {
messageID string
thinking strings.Builder
text strings.Builder
outputTokens int
nextBlockIndex int
thinkingBlockOpen bool
@@ -66,6 +67,9 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ErrorMessage != "" {
s.upstreamErr = parsed.ErrorMessage
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReason("upstream_error")}

View File

@@ -108,6 +108,9 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
}
outputTokens := util.EstimateTokens(finalThinking) + util.EstimateTokens(finalText)
if s.outputTokens > 0 {
outputTokens = s.outputTokens
}
s.send("message_delta", map[string]any{
"type": "message_delta",
"delta": map[string]any{

View File

@@ -1,7 +1,6 @@
package claude
import (
"context"
"net/http"
"net/http/httptest"
"strings"
@@ -9,48 +8,17 @@ import (
"github.com/go-chi/chi/v5"
chimw "github.com/go-chi/chi/v5/middleware"
"ds2api/internal/auth"
)
type streamStatusClaudeAuthStub struct{}
type streamStatusClaudeOpenAIStub struct{}
func (streamStatusClaudeAuthStub) Determine(_ *http.Request) (*auth.RequestAuth, error) {
return &auth.RequestAuth{
UseConfigToken: false,
DeepSeekToken: "direct-token",
CallerID: "caller:test",
TriedAccounts: map[string]bool{},
}, nil
func (streamStatusClaudeOpenAIStub) ChatCompletions(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("data: {\"id\":\"chatcmpl-1\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hello\"},\"finish_reason\":null}]}\n\n"))
_, _ = w.Write([]byte("data: [DONE]\n\n"))
}
func (streamStatusClaudeAuthStub) Release(_ *auth.RequestAuth) {}
type streamStatusClaudeDSStub struct{}
func (streamStatusClaudeDSStub) CreateSession(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "session-id", nil
}
func (streamStatusClaudeDSStub) GetPow(_ context.Context, _ *auth.RequestAuth, _ int) (string, error) {
return "pow", nil
}
func (streamStatusClaudeDSStub) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ map[string]any, _ string, _ int) (*http.Response, error) {
body := "data: {\"p\":\"response/content\",\"v\":\"hello\"}\n" + "data: [DONE]\n"
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: ioNopCloser{strings.NewReader(body)},
}, nil
}
type ioNopCloser struct {
*strings.Reader
}
func (ioNopCloser) Close() error { return nil }
type streamStatusClaudeStoreStub struct{}
func (streamStatusClaudeStoreStub) ClaudeMapping() map[string]string {
@@ -73,9 +41,8 @@ func captureClaudeStatusMiddleware(statuses *[]int) func(http.Handler) http.Hand
func TestClaudeMessagesStreamStatusCapturedAs200(t *testing.T) {
statuses := make([]int, 0, 1)
h := &Handler{
Store: streamStatusClaudeStoreStub{},
Auth: streamStatusClaudeAuthStub{},
DS: streamStatusClaudeDSStub{},
Store: streamStatusClaudeStoreStub{},
OpenAI: streamStatusClaudeOpenAIStub{},
}
r := chi.NewRouter()
r.Use(captureClaudeStatusMiddleware(&statuses))
@@ -83,7 +50,6 @@ func TestClaudeMessagesStreamStatusCapturedAs200(t *testing.T) {
reqBody := `{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":true}`
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)

View File

@@ -0,0 +1,25 @@
package claude
import (
"fmt"
"strings"
)
type claudeToolCallState struct {
nameByID map[string]string
lastIDByName map[string]string
callIDSequence int
}
func (s *claudeToolCallState) nextID() string {
s.callIDSequence++
return fmt.Sprintf("call_claude_%d", s.callIDSequence)
}
func safeStringValue(v any) string {
s, ok := v.(string)
if !ok {
return ""
}
return strings.TrimSpace(s)
}

View File

@@ -1,11 +1,20 @@
package gemini
import "strings"
import (
"fmt"
"strings"
)
const maxGeminiRawPromptChars = 1024
func geminiMessagesFromRequest(req map[string]any) []any {
out := make([]any, 0, 8)
toolCallCounter := 0
nextToolCallID := func() string {
toolCallCounter++
return fmt.Sprintf("call_gemini_%d", toolCallCounter)
}
lastToolCallIDByName := map[string]string{}
if sys := normalizeGeminiSystemInstruction(req["systemInstruction"]); strings.TrimSpace(sys) != "" {
out = append(out, map[string]any{
"role": "system",
@@ -61,8 +70,11 @@ func geminiMessagesFromRequest(req map[string]any) []any {
if name := strings.TrimSpace(asString(fnCall["name"])); name != "" {
callID := strings.TrimSpace(asString(fnCall["id"]))
if callID == "" {
callID = "call_gemini"
if callID = strings.TrimSpace(asString(fnCall["call_id"])); callID == "" {
callID = nextToolCallID()
}
}
lastToolCallIDByName[strings.ToLower(name)] = callID
out = append(out, map[string]any{
"role": "assistant",
"tool_calls": []any{
@@ -91,7 +103,10 @@ func geminiMessagesFromRequest(req map[string]any) []any {
callID = strings.TrimSpace(asString(fnResp["tool_call_id"]))
}
if callID == "" {
callID = "call_gemini"
callID = strings.TrimSpace(lastToolCallIDByName[strings.ToLower(name)])
}
if callID == "" {
callID = nextToolCallID()
}
content := fnResp["response"]
if content == nil {

View File

@@ -82,3 +82,48 @@ func TestGeminiMessagesFromRequestPreservesUnknownPartAsRawJSONText(t *testing.T
t.Fatalf("expected raw base64 payload not to be embedded, got %q", content)
}
}
func TestGeminiMessagesFromRequestBackfillsFunctionResponseCallIDByName(t *testing.T) {
req := map[string]any{
"contents": []any{
map[string]any{
"role": "model",
"parts": []any{
map[string]any{
"functionCall": map[string]any{
"name": "search_web",
"args": map[string]any{"query": "docs"},
},
},
},
},
map[string]any{
"role": "user",
"parts": []any{
map[string]any{
"functionResponse": map[string]any{
"name": "search_web",
"response": map[string]any{"ok": true},
},
},
},
},
},
}
got := geminiMessagesFromRequest(req)
if len(got) != 2 {
t.Fatalf("expected two normalized messages, got %#v", got)
}
assistant, _ := got[0].(map[string]any)
tc, _ := assistant["tool_calls"].([]any)
call, _ := tc[0].(map[string]any)
callID, _ := call["id"].(string)
if !strings.HasPrefix(callID, "call_gemini_") {
t.Fatalf("expected generated call id prefix, got %#v", call)
}
toolMsg, _ := got[1].(map[string]any)
if toolMsg["tool_call_id"] != callID {
t.Fatalf("expected tool response to inherit generated call id, tool=%#v call=%#v", toolMsg, call)
}
}

View File

@@ -24,6 +24,10 @@ type ConfigReader interface {
ModelAliases() map[string]string
}
type OpenAIChatRunner interface {
ChatCompletions(w http.ResponseWriter, r *http.Request)
}
var _ AuthResolver = (*auth.Resolver)(nil)
var _ DeepSeekCaller = (*deepseek.Client)(nil)
var _ ConfigReader = (*config.Store)(nil)

View File

@@ -1,70 +1,134 @@
package gemini
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"github.com/go-chi/chi/v5"
"ds2api/internal/auth"
"ds2api/internal/sse"
"ds2api/internal/translatorcliproxy"
"ds2api/internal/util"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
)
func (h *Handler) handleGenerateContent(w http.ResponseWriter, r *http.Request, stream bool) {
a, err := h.Auth.Determine(r)
if h.OpenAI == nil {
writeGeminiError(w, http.StatusInternalServerError, "OpenAI proxy backend unavailable.")
return
}
if h.proxyViaOpenAI(w, r, stream) {
return
}
writeGeminiError(w, http.StatusBadGateway, "Failed to proxy Gemini request.")
}
func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, stream bool) bool {
raw, err := io.ReadAll(r.Body)
if err != nil {
status := http.StatusUnauthorized
detail := err.Error()
if err == auth.ErrNoAccount {
status = http.StatusTooManyRequests
}
writeGeminiError(w, status, detail)
return
writeGeminiError(w, http.StatusBadRequest, "invalid body")
return true
}
defer h.Auth.Release(a)
var req map[string]any
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeGeminiError(w, http.StatusBadRequest, "invalid json")
return
}
routeModel := strings.TrimSpace(chi.URLParam(r, "model"))
stdReq, err := normalizeGeminiRequest(h.Store, routeModel, req, stream)
if err != nil {
writeGeminiError(w, http.StatusBadRequest, err.Error())
return
}
sessionID, err := h.DS.CreateSession(r.Context(), a, 3)
if err != nil {
if a.UseConfigToken {
writeGeminiError(w, http.StatusUnauthorized, "Account token is invalid. Please re-login the account in admin.")
} else {
writeGeminiError(w, http.StatusUnauthorized, "Invalid token.")
translatedReq := translatorcliproxy.ToOpenAI(sdktranslator.FormatGemini, routeModel, raw, stream)
if !strings.Contains(string(translatedReq), `"stream"`) {
var reqMap map[string]any
if json.Unmarshal(translatedReq, &reqMap) == nil {
reqMap["stream"] = stream
if b, e := json.Marshal(reqMap); e == nil {
translatedReq = b
}
}
return
}
pow, err := h.DS.GetPow(r.Context(), a, 3)
if err != nil {
writeGeminiError(w, http.StatusUnauthorized, "Failed to get PoW (invalid token or unknown error).")
return
}
payload := stdReq.CompletionPayload(sessionID)
resp, err := h.DS.CallCompletion(r.Context(), a, payload, pow, 3)
if err != nil {
writeGeminiError(w, http.StatusInternalServerError, "Failed to get completion.")
return
}
if stream {
h.handleStreamGenerateContent(w, r, resp, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames)
return
isVercelPrepare := strings.TrimSpace(r.URL.Query().Get("__stream_prepare")) == "1"
isVercelRelease := strings.TrimSpace(r.URL.Query().Get("__stream_release")) == "1"
if isVercelRelease {
proxyReq := r.Clone(r.Context())
proxyReq.URL.Path = "/v1/chat/completions"
proxyReq.Body = io.NopCloser(bytes.NewReader(raw))
proxyReq.ContentLength = int64(len(raw))
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
body, _ := io.ReadAll(res.Body)
for k, vv := range res.Header {
for _, v := range vv {
w.Header().Add(k, v)
}
}
w.WriteHeader(res.StatusCode)
_, _ = w.Write(body)
return true
}
h.handleNonStreamGenerateContent(w, resp, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.ToolNames)
proxyReq := r.Clone(r.Context())
proxyReq.URL.Path = "/v1/chat/completions"
proxyReq.Body = io.NopCloser(bytes.NewReader(translatedReq))
proxyReq.ContentLength = int64(len(translatedReq))
if stream && !isVercelPrepare {
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache, no-transform")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no")
streamWriter := translatorcliproxy.NewOpenAIStreamTranslatorWriter(w, sdktranslator.FormatGemini, routeModel, raw, translatedReq)
h.OpenAI.ChatCompletions(streamWriter, proxyReq)
return true
}
rec := httptest.NewRecorder()
h.OpenAI.ChatCompletions(rec, proxyReq)
res := rec.Result()
defer res.Body.Close()
body, _ := io.ReadAll(res.Body)
if res.StatusCode < 200 || res.StatusCode >= 300 {
for k, vv := range res.Header {
for _, v := range vv {
w.Header().Add(k, v)
}
}
writeGeminiErrorFromOpenAI(w, res.StatusCode, body)
return true
}
if isVercelPrepare {
for k, vv := range res.Header {
for _, v := range vv {
w.Header().Add(k, v)
}
}
w.WriteHeader(res.StatusCode)
_, _ = w.Write(body)
return true
}
converted := translatorcliproxy.FromOpenAINonStream(sdktranslator.FormatGemini, routeModel, raw, translatedReq, body)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, _ = w.Write(converted)
return true
}
func writeGeminiErrorFromOpenAI(w http.ResponseWriter, status int, raw []byte) {
message := strings.TrimSpace(string(raw))
var parsed map[string]any
if err := json.Unmarshal(raw, &parsed); err == nil {
if errObj, ok := parsed["error"].(map[string]any); ok {
if msg, ok := errObj["message"].(string); ok && strings.TrimSpace(msg) != "" {
message = strings.TrimSpace(msg)
}
}
}
if message == "" {
message = http.StatusText(status)
}
writeGeminiError(w, status, message)
}
func (h *Handler) handleNonStreamGenerateContent(w http.ResponseWriter, resp *http.Response, model, finalPrompt string, thinkingEnabled bool, toolNames []string) {
@@ -76,12 +140,12 @@ func (h *Handler) handleNonStreamGenerateContent(w http.ResponseWriter, resp *ht
}
result := sse.CollectStream(resp, thinkingEnabled, true)
writeJSON(w, http.StatusOK, buildGeminiGenerateContentResponse(model, finalPrompt, result.Thinking, result.Text, toolNames))
writeJSON(w, http.StatusOK, buildGeminiGenerateContentResponse(model, finalPrompt, result.Thinking, result.Text, toolNames, result.OutputTokens))
}
func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, finalText string, toolNames []string) map[string]any {
func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, finalText string, toolNames []string, outputTokens int) map[string]any {
parts := buildGeminiPartsFromFinal(finalText, finalThinking, toolNames)
usage := buildGeminiUsage(finalPrompt, finalThinking, finalText)
usage := buildGeminiUsage(finalPrompt, finalThinking, finalText, outputTokens)
return map[string]any{
"candidates": []map[string]any{
{
@@ -98,10 +162,14 @@ func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, final
}
}
func buildGeminiUsage(finalPrompt, finalThinking, finalText string) map[string]any {
func buildGeminiUsage(finalPrompt, finalThinking, finalText string, outputTokens int) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
if outputTokens > 0 {
completionTokens = outputTokens
reasoningTokens = 0
}
return map[string]any{
"promptTokenCount": promptTokens,
"candidatesTokenCount": reasoningTokens + completionTokens,

View File

@@ -11,9 +11,10 @@ import (
var writeJSON = util.WriteJSON
type Handler struct {
Store ConfigReader
Auth AuthResolver
DS DeepSeekCaller
Store ConfigReader
Auth AuthResolver
DS DeepSeekCaller
OpenAI OpenAIChatRunner
}
func RegisterRoutes(r chi.Router, h *Handler) {

View File

@@ -64,6 +64,7 @@ type geminiStreamRuntime struct {
thinking strings.Builder
text strings.Builder
outputTokens int
}
func newGeminiStreamRuntime(
@@ -103,6 +104,9 @@ func (s *geminiStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ContentFilter || parsed.ErrorMessage != "" || parsed.Stop {
return streamengine.ParsedDecision{Stop: true}
}
@@ -176,6 +180,6 @@ func (s *geminiStreamRuntime) finalize() {
},
},
"modelVersion": s.model,
"usageMetadata": buildGeminiUsage(s.finalPrompt, finalThinking, finalText),
"usageMetadata": buildGeminiUsage(s.finalPrompt, finalThinking, finalText, s.outputTokens),
})
}

View File

@@ -61,6 +61,44 @@ func (m testGeminiDS) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ m
return m.resp, nil
}
type geminiOpenAIErrorStub struct {
status int
body string
headers map[string]string
}
func (s geminiOpenAIErrorStub) ChatCompletions(w http.ResponseWriter, _ *http.Request) {
for k, v := range s.headers {
w.Header().Set(k, v)
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(s.status)
_, _ = w.Write([]byte(s.body))
}
type geminiOpenAISuccessStub struct {
stream bool
body string
}
func (s geminiOpenAISuccessStub) ChatCompletions(w http.ResponseWriter, _ *http.Request) {
if s.stream {
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("data: {\"id\":\"chatcmpl-1\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hello \"},\"finish_reason\":null}]}\n\n"))
_, _ = w.Write([]byte("data: {\"id\":\"chatcmpl-1\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"world\"},\"finish_reason\":\"stop\"}]}\n\n"))
_, _ = w.Write([]byte("data: [DONE]\n\n"))
return
}
out := s.body
if strings.TrimSpace(out) == "" {
out = `{"id":"chatcmpl-1","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","tool_calls":[{"id":"call_1","type":"function","function":{"name":"eval_javascript","arguments":"{\"code\":\"1+1\"}"}}]},"finish_reason":"tool_calls"}]}`
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(out))
}
func makeGeminiUpstreamResponse(lines ...string) *http.Response {
body := strings.Join(lines, "\n")
if !strings.HasSuffix(body, "\n") {
@@ -98,14 +136,11 @@ func TestGeminiRoutesRegistered(t *testing.T) {
}
func TestGenerateContentReturnsFunctionCallParts(t *testing.T) {
upstream := makeGeminiUpstreamResponse(
`data: {"p":"response/content","v":"{\"tool_calls\":[{\"name\":\"eval_javascript\",\"input\":{\"code\":\"1+1\"}}]}"}`,
`data: [DONE]`,
)
h := &Handler{
Store: testGeminiConfig{},
Auth: testGeminiAuth{},
DS: testGeminiDS{resp: upstream},
OpenAI: geminiOpenAISuccessStub{
body: `{"id":"chatcmpl-1","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","tool_calls":[{"id":"call_1","type":"function","function":{"name":"eval_javascript","arguments":"{\"code\":\"1+1\"}"}}]},"finish_reason":"tool_calls"}]}`,
},
}
r := chi.NewRouter()
RegisterRoutes(r, h)
@@ -115,7 +150,6 @@ func TestGenerateContentReturnsFunctionCallParts(t *testing.T) {
"tools":[{"functionDeclarations":[{"name":"eval_javascript","description":"eval","parameters":{"type":"object","properties":{"code":{"type":"string"}}}}]}]
}`
req := httptest.NewRequest(http.MethodPost, "/v1beta/models/gemini-2.5-pro:generateContent", strings.NewReader(body))
req.Header.Set("Authorization", "Bearer direct-token")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
@@ -144,11 +178,7 @@ func TestGenerateContentReturnsFunctionCallParts(t *testing.T) {
}
func TestGenerateContentMixedToolSnippetAlsoTriggersFunctionCall(t *testing.T) {
upstream := makeGeminiUpstreamResponse(
`data: {"p":"response/content","v":"我来调用工具\n{\"tool_calls\":[{\"name\":\"eval_javascript\",\"input\":{\"code\":\"1+1\"}}]}"}`,
`data: [DONE]`,
)
h := &Handler{Store: testGeminiConfig{}, Auth: testGeminiAuth{}, DS: testGeminiDS{resp: upstream}}
h := &Handler{Store: testGeminiConfig{}, OpenAI: geminiOpenAISuccessStub{}}
r := chi.NewRouter()
RegisterRoutes(r, h)
@@ -157,7 +187,6 @@ func TestGenerateContentMixedToolSnippetAlsoTriggersFunctionCall(t *testing.T) {
"tools":[{"functionDeclarations":[{"name":"eval_javascript","description":"eval","parameters":{"type":"object","properties":{"code":{"type":"string"}}}}]}]
}`
req := httptest.NewRequest(http.MethodPost, "/v1beta/models/gemini-2.5-pro:generateContent", strings.NewReader(body))
req.Header.Set("Authorization", "Bearer direct-token")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
@@ -180,38 +209,25 @@ func TestGenerateContentMixedToolSnippetAlsoTriggersFunctionCall(t *testing.T) {
}
func TestStreamGenerateContentEmitsSSE(t *testing.T) {
upstream := makeGeminiUpstreamResponse(
`data: {"p":"response/content","v":"hello "}`,
`data: {"p":"response/content","v":"world"}`,
`data: [DONE]`,
)
h := &Handler{
Store: testGeminiConfig{},
Auth: testGeminiAuth{},
DS: testGeminiDS{resp: upstream},
Store: testGeminiConfig{},
OpenAI: geminiOpenAISuccessStub{stream: true},
}
r := chi.NewRouter()
RegisterRoutes(r, h)
body := `{"contents":[{"role":"user","parts":[{"text":"hello"}]}]}`
req := httptest.NewRequest(http.MethodPost, "/v1/models/gemini-2.5-pro:streamGenerateContent?alt=sse", strings.NewReader(body))
req.Header.Set("Authorization", "Bearer direct-token")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "data: ") {
t.Fatalf("expected SSE data frames, got body=%s", rec.Body.String())
}
if !strings.Contains(rec.Body.String(), `"finishReason":"STOP"`) {
t.Fatalf("expected stream finish frame, got body=%s", rec.Body.String())
}
frames := extractGeminiSSEFrames(t, rec.Body.String())
if len(frames) == 0 {
t.Fatalf("expected non-empty sse frames, body=%s", rec.Body.String())
t.Fatalf("expected non-empty stream frames, body=%s", rec.Body.String())
}
last := frames[len(frames)-1]
candidates, _ := last["candidates"].([]any)
@@ -229,16 +245,61 @@ func TestStreamGenerateContentEmitsSSE(t *testing.T) {
}
}
func TestGenerateContentOpenAIProxyErrorUsesGeminiEnvelope(t *testing.T) {
h := &Handler{
Store: testGeminiConfig{},
OpenAI: geminiOpenAIErrorStub{
status: http.StatusUnauthorized,
body: `{"error":{"message":"invalid api key"}}`,
headers: map[string]string{
"WWW-Authenticate": `Bearer realm="example"`,
"Retry-After": "30",
"X-RateLimit-Remaining": "0",
},
},
}
r := chi.NewRouter()
RegisterRoutes(r, h)
req := httptest.NewRequest(http.MethodPost, "/v1/models/gemini-2.5-pro:generateContent", strings.NewReader(`{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`))
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("expected 401, got %d body=%s", rec.Code, rec.Body.String())
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("expected json body: %v", err)
}
errObj, _ := out["error"].(map[string]any)
if errObj["status"] != "UNAUTHENTICATED" {
t.Fatalf("expected Gemini status UNAUTHENTICATED, got=%v", errObj["status"])
}
if errObj["message"] != "invalid api key" {
t.Fatalf("expected parsed error message, got=%v", errObj["message"])
}
if got := rec.Header().Get("WWW-Authenticate"); got == "" {
t.Fatalf("expected WWW-Authenticate header to be preserved")
}
if got := rec.Header().Get("Retry-After"); got != "30" {
t.Fatalf("expected Retry-After header 30, got=%q", got)
}
if got := rec.Header().Get("X-RateLimit-Remaining"); got != "0" {
t.Fatalf("expected X-RateLimit-Remaining header 0, got=%q", got)
}
}
func extractGeminiSSEFrames(t *testing.T, body string) []map[string]any {
t.Helper()
scanner := bufio.NewScanner(strings.NewReader(body))
out := make([]map[string]any, 0, 4)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if !strings.HasPrefix(line, "data: ") {
continue
raw := line
if strings.HasPrefix(line, "data: ") {
raw = strings.TrimSpace(strings.TrimPrefix(line, "data: "))
}
raw := strings.TrimSpace(strings.TrimPrefix(line, "data: "))
if raw == "" {
continue
}

View File

@@ -0,0 +1,42 @@
package gemini
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
type openAIProxyStub struct {
status int
body string
}
func (s openAIProxyStub) ChatCompletions(w http.ResponseWriter, _ *http.Request) {
if s.status == 0 {
s.status = http.StatusOK
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(s.status)
_, _ = w.Write([]byte(s.body))
}
func TestGeminiProxyViaOpenAIVercelReleasePassthrough(t *testing.T) {
h := &Handler{OpenAI: openAIProxyStub{status: 200, body: `{"success":true}`}}
req := httptest.NewRequest(http.MethodPost, "/v1beta/models/gemini-2.5-pro:streamGenerateContent?__stream_release=1", strings.NewReader(`{"lease_id":"lease_123"}`))
rec := httptest.NewRecorder()
h.StreamGenerateContent(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("unexpected status: %d body=%s", rec.Code, rec.Body.String())
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("expected json response, got err=%v body=%s", err, rec.Body.String())
}
if v, ok := out["success"].(bool); !ok || !v {
t.Fatalf("expected success=true passthrough, got=%v", out)
}
}

View File

@@ -36,6 +36,7 @@ type chatStreamRuntime struct {
streamToolNames map[int]string
thinking strings.Builder
text strings.Builder
outputTokens int
}
func newChatStreamRuntime(
@@ -97,7 +98,7 @@ func (s *chatStreamRuntime) sendDone() {
func (s *chatStreamRuntime) finalize(finishReason string) {
finalThinking := s.thinking.String()
finalText := sanitizeLeakedToolHistory(s.text.String())
finalText := sanitizeLeakedOutput(s.text.String())
detected := util.ParseStandaloneToolCallsDetailed(finalText, s.toolNames)
if len(detected.Calls) > 0 && !s.toolCallsDoneEmitted {
finishReason = "tool_calls"
@@ -141,7 +142,7 @@ func (s *chatStreamRuntime) finalize(finishReason string) {
if evt.Content == "" {
continue
}
cleaned := sanitizeLeakedToolHistory(evt.Content)
cleaned := sanitizeLeakedOutput(evt.Content)
if cleaned == "" {
continue
}
@@ -165,12 +166,19 @@ func (s *chatStreamRuntime) finalize(finishReason string) {
if len(detected.Calls) > 0 || s.toolCallsEmitted {
finishReason = "tool_calls"
}
usage := openaifmt.BuildChatUsage(s.finalPrompt, finalThinking, finalText)
if s.outputTokens > 0 {
usage["completion_tokens"] = s.outputTokens
if prompt, ok := usage["prompt_tokens"].(int); ok {
usage["total_tokens"] = prompt + s.outputTokens
}
}
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
s.model,
[]map[string]any{openaifmt.BuildChatStreamFinishChoice(0, finishReason)},
openaifmt.BuildChatUsage(s.finalPrompt, finalThinking, finalText),
usage,
))
s.sendDone()
}
@@ -179,7 +187,13 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.ContentFilter || parsed.ErrorMessage != "" {
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ContentFilter {
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReasonHandlerRequested}
}
if parsed.ErrorMessage != "" {
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReason("content_filter")}
}
if parsed.Stop {
@@ -250,7 +264,7 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
continue
}
if evt.Content != "" {
cleaned := sanitizeLeakedToolHistory(evt.Content)
cleaned := sanitizeLeakedOutput(evt.Content)
if cleaned == "" {
continue
}

View File

@@ -105,8 +105,16 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, ctx context.Context, re
result := sse.CollectStream(resp, thinkingEnabled, true)
finalThinking := result.Thinking
finalText := sanitizeLeakedToolHistory(result.Text)
finalText := sanitizeLeakedOutput(result.Text)
respBody := openaifmt.BuildChatCompletion(completionID, model, finalPrompt, finalThinking, finalText, toolNames)
if result.OutputTokens > 0 {
if usage, ok := respBody["usage"].(map[string]any); ok {
usage["completion_tokens"] = result.OutputTokens
if prompt, ok := usage["prompt_tokens"].(int); ok {
usage["total_tokens"] = prompt + result.OutputTokens
}
}
}
writeJSON(w, http.StatusOK, respBody)
}

View File

@@ -1,19 +1,9 @@
package openai
import "strings"
func (h *Handler) toolcallFeatureMatchEnabled() bool {
if h == nil || h.Store == nil {
return true
}
mode := strings.TrimSpace(strings.ToLower(h.Store.ToolcallMode()))
return mode == "" || mode == "feature_match"
return true
}
func (h *Handler) toolcallEarlyEmitHighConfidence() bool {
if h == nil || h.Store == nil {
return true
}
level := strings.TrimSpace(strings.ToLower(h.Store.ToolcallEarlyEmitConfidence()))
return level == "" || level == "high"
return true
}

View File

@@ -0,0 +1,70 @@
package openai
import (
"regexp"
)
var emptyJSONFencePattern = regexp.MustCompile("(?is)```json\\s*```")
var leakedToolCallArrayPattern = regexp.MustCompile(`(?is)\[\{\s*"function"\s*:\s*\{[\s\S]*?\}\s*,\s*"id"\s*:\s*"call[^"]*"\s*,\s*"type"\s*:\s*"function"\s*}\]`)
var leakedToolResultBlobPattern = regexp.MustCompile(`(?is)<\s*\|\s*tool\s*\|\s*>\s*\{[\s\S]*?"tool_call_id"\s*:\s*"call[^"]*"\s*}`)
// leakedMetaMarkerPattern matches DeepSeek special tokens in BOTH forms:
// - ASCII underscore: <end_of_sentence>
// - U+2581 variant: <end▁of▁sentence> (used in some DeepSeek outputs)
var leakedMetaMarkerPattern = regexp.MustCompile(`(?i)<[\|]\s*(?:assistant|tool|end[_▁]of[_▁]sentence|end[_▁]of[_▁]thinking)\s*[\|]>`)
// leakedAgentXMLBlockPatterns catch agent-style XML blocks that leak through
// when the sieve fails to capture them. These are applied only to complete
// wrapper blocks so standalone "<result>" examples in normal output remain
// untouched.
var leakedAgentXMLBlockPatterns = []*regexp.Regexp{
regexp.MustCompile(`(?is)<attempt_completion\b[^>]*>(.*?)</attempt_completion>`),
regexp.MustCompile(`(?is)<ask_followup_question\b[^>]*>(.*?)</ask_followup_question>`),
regexp.MustCompile(`(?is)<new_task\b[^>]*>(.*?)</new_task>`),
}
var leakedAgentWrapperTagPattern = regexp.MustCompile(`(?is)</?(?:attempt_completion|ask_followup_question|new_task)\b[^>]*>`)
var leakedAgentWrapperPlusResultOpenPattern = regexp.MustCompile(`(?is)<(?:attempt_completion|ask_followup_question|new_task)\b[^>]*>\s*<result>`)
var leakedAgentResultPlusWrapperClosePattern = regexp.MustCompile(`(?is)</result>\s*</(?:attempt_completion|ask_followup_question|new_task)\b[^>]*>`)
var leakedAgentResultTagPattern = regexp.MustCompile(`(?is)</?result>`)
func sanitizeLeakedOutput(text string) string {
if text == "" {
return text
}
out := emptyJSONFencePattern.ReplaceAllString(text, "")
out = leakedToolCallArrayPattern.ReplaceAllString(out, "")
out = leakedToolResultBlobPattern.ReplaceAllString(out, "")
out = leakedMetaMarkerPattern.ReplaceAllString(out, "")
out = sanitizeLeakedAgentXMLBlocks(out)
return out
}
func sanitizeLeakedAgentXMLBlocks(text string) string {
out := text
for _, pattern := range leakedAgentXMLBlockPatterns {
out = pattern.ReplaceAllStringFunc(out, func(match string) string {
submatches := pattern.FindStringSubmatch(match)
if len(submatches) < 2 {
return match
}
// Preserve the inner text so leaked agent instructions do not erase
// the actual answer, but strip the wrapper/result markup itself.
return leakedAgentResultTagPattern.ReplaceAllString(submatches[1], "")
})
}
// Fallback for truncated output streams: strip any dangling wrapper tags
// that were not part of a complete block replacement. If we detect leaked
// wrapper tags, strip only adjacent <result> tags to avoid exposing agent
// markup without altering unrelated user-visible <result> examples.
if leakedAgentWrapperTagPattern.MatchString(out) {
out = leakedAgentWrapperPlusResultOpenPattern.ReplaceAllStringFunc(out, func(match string) string {
return leakedAgentResultTagPattern.ReplaceAllString(match, "")
})
out = leakedAgentResultPlusWrapperClosePattern.ReplaceAllStringFunc(out, func(match string) string {
return leakedAgentResultTagPattern.ReplaceAllString(match, "")
})
out = leakedAgentWrapperTagPattern.ReplaceAllString(out, "")
}
return out
}

View File

@@ -0,0 +1,68 @@
package openai
import "testing"
func TestSanitizeLeakedOutputRemovesEmptyJSONFence(t *testing.T) {
raw := "before\n```json\n```\nafter"
got := sanitizeLeakedOutput(raw)
if got != "before\n\nafter" {
t.Fatalf("unexpected sanitized empty json fence: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesLeakedWireToolCallAndResult(t *testing.T) {
raw := "开始\n[{\"function\":{\"arguments\":\"{\\\"command\\\":\\\"java -version\\\"}\",\"name\":\"exec\"},\"id\":\"callb9a321\",\"type\":\"function\"}]< | Tool | >{\"content\":\"openjdk version 21\",\"tool_call_id\":\"callb9a321\"}\n结束"
got := sanitizeLeakedOutput(raw)
if got != "开始\n\n结束" {
t.Fatalf("unexpected sanitize result for leaked wire format: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesStandaloneMetaMarkers(t *testing.T) {
raw := "A<| end_of_sentence |><| Assistant |>B<| end_of_thinking |>C<end▁of▁thinking>D<end▁of▁sentence>E"
got := sanitizeLeakedOutput(raw)
if got != "ABCDE" {
t.Fatalf("unexpected sanitize result for meta markers: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesAgentXMLLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer</result></attempt_completion>"
got := sanitizeLeakedOutput(raw)
if got != "Done.Some final answer" {
t.Fatalf("unexpected sanitize result for agent XML leak: %q", got)
}
}
func TestSanitizeLeakedOutputPreservesStandaloneResultTags(t *testing.T) {
raw := "Example XML: <result>value</result>"
got := sanitizeLeakedOutput(raw)
if got != raw {
t.Fatalf("unexpected sanitize result for standalone result tag: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesDanglingAgentXMLOpeningTags(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer"
got := sanitizeLeakedOutput(raw)
if got != "Done.Some final answer" {
t.Fatalf("unexpected sanitize result for dangling opening tags: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesDanglingAgentXMLClosingTags(t *testing.T) {
raw := "Done.Some final answer</result></attempt_completion>"
got := sanitizeLeakedOutput(raw)
if got != "Done.Some final answer" {
t.Fatalf("unexpected sanitize result for dangling closing tags: %q", got)
}
}
func TestSanitizeLeakedOutputPreservesUnrelatedResultTagsWhenWrapperLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer\nExample XML: <result>value</result>"
got := sanitizeLeakedOutput(raw)
want := "Done.Some final answer\nExample XML: <result>value</result>"
if got != want {
t.Fatalf("unexpected sanitize result for mixed leaked wrapper + xml example: %q", got)
}
}

View File

@@ -1,7 +1,6 @@
package openai
import (
"encoding/json"
"strings"
"ds2api/internal/prompt"
@@ -55,7 +54,18 @@ func normalizeOpenAIMessagesForPrompt(raw []any, traceID string) []map[string]an
}
func buildAssistantContentForPrompt(msg map[string]any) string {
return strings.TrimSpace(normalizeOpenAIContentForPrompt(msg["content"]))
content := strings.TrimSpace(normalizeOpenAIContentForPrompt(msg["content"]))
toolHistory := prompt.FormatToolCallsForPrompt(msg["tool_calls"])
switch {
case content == "" && toolHistory == "":
return ""
case content == "":
return toolHistory
case toolHistory == "":
return content
default:
return content + "\n\n" + toolHistory
}
}
func buildToolContentForPrompt(msg map[string]any) string {
@@ -70,18 +80,6 @@ func normalizeOpenAIContentForPrompt(v any) string {
return prompt.NormalizeContent(v)
}
func normalizeToolArgumentString(raw string) string {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return ""
}
if looksLikeConcatenatedJSON(trimmed) {
// Keep original payload to avoid silent argument rewrites.
return raw
}
return trimmed
}
func normalizeOpenAIRoleForPrompt(role string) string {
role = strings.ToLower(strings.TrimSpace(role))
if role == "developer" {
@@ -96,20 +94,3 @@ func asString(v any) string {
}
return ""
}
func looksLikeConcatenatedJSON(raw string) bool {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return false
}
if strings.Contains(trimmed, "}{") || strings.Contains(trimmed, "][") {
return true
}
dec := json.NewDecoder(strings.NewReader(trimmed))
var first any
if err := dec.Decode(&first); err != nil {
return false
}
var second any
return dec.Decode(&second) == nil
}

View File

@@ -34,20 +34,23 @@ func TestNormalizeOpenAIMessagesForPrompt_AssistantToolCallsAndToolResult(t *tes
}
normalized := normalizeOpenAIMessagesForPrompt(raw, "")
if len(normalized) != 3 {
t.Fatalf("expected 3 normalized messages with tool-call-only assistant turn omitted, got %d", len(normalized))
if len(normalized) != 4 {
t.Fatalf("expected 4 normalized messages with assistant tool history preserved, got %d", len(normalized))
}
toolContent, _ := normalized[2]["content"].(string)
if !strings.Contains(toolContent, `"temp":18`) {
t.Fatalf("tool result should be transparently forwarded, got %q", toolContent)
assistantContent, _ := normalized[2]["content"].(string)
if !strings.Contains(assistantContent, "<tool_calls>") {
t.Fatalf("assistant tool history should be preserved in XML form, got %q", assistantContent)
}
if strings.Contains(toolContent, "[TOOL_RESULT_HISTORY]") {
t.Fatalf("tool history marker should not be injected: %q", toolContent)
if !strings.Contains(assistantContent, "<tool_name>get_weather</tool_name>") {
t.Fatalf("expected tool name in preserved history, got %q", assistantContent)
}
if !strings.Contains(normalized[3]["content"].(string), `"temp":18`) {
t.Fatalf("tool result should be transparently forwarded, got %#v", normalized[3]["content"])
}
prompt := util.MessagesPrepare(normalized)
if strings.Contains(prompt, "[TOOL_CALL_HISTORY]") || strings.Contains(prompt, "[TOOL_RESULT_HISTORY]") {
t.Fatalf("expected no synthetic history markers in prompt: %q", prompt)
if !strings.Contains(prompt, "<tool_calls>") {
t.Fatalf("expected preserved assistant tool history in prompt: %q", prompt)
}
}
@@ -170,8 +173,15 @@ func TestNormalizeOpenAIMessagesForPrompt_AssistantMultipleToolCallsRemainSepara
}
normalized := normalizeOpenAIMessagesForPrompt(raw, "")
if len(normalized) != 0 {
t.Fatalf("expected assistant tool_call-only message omitted, got %#v", normalized)
if len(normalized) != 1 {
t.Fatalf("expected assistant tool_call-only message preserved, got %#v", normalized)
}
content, _ := normalized[0]["content"].(string)
if strings.Count(content, "<tool_call>") != 2 {
t.Fatalf("expected two preserved tool call blocks, got %q", content)
}
if !strings.Contains(content, "<tool_name>search_web</tool_name>") || !strings.Contains(content, "<tool_name>eval_javascript</tool_name>") {
t.Fatalf("expected both tool names in preserved history, got %q", content)
}
}
@@ -192,8 +202,12 @@ func TestNormalizeOpenAIMessagesForPrompt_PreservesConcatenatedToolArguments(t *
}
normalized := normalizeOpenAIMessagesForPrompt(raw, "")
if len(normalized) != 0 {
t.Fatalf("expected assistant tool_call-only content omitted, got %#v", normalized)
if len(normalized) != 1 {
t.Fatalf("expected assistant tool_call-only content preserved, got %#v", normalized)
}
content, _ := normalized[0]["content"].(string)
if !strings.Contains(content, `{}{"query":"测试工具调用"}`) {
t.Fatalf("expected concatenated tool arguments preserved, got %q", content)
}
}
@@ -215,7 +229,7 @@ func TestNormalizeOpenAIMessagesForPrompt_AssistantToolCallsMissingNameAreDroppe
normalized := normalizeOpenAIMessagesForPrompt(raw, "")
if len(normalized) != 0 {
t.Fatalf("expected assistant tool_calls without text omitted, got %#v", normalized)
t.Fatalf("expected assistant tool_calls without text to be dropped when name is missing, got %#v", normalized)
}
}
@@ -237,8 +251,15 @@ func TestNormalizeOpenAIMessagesForPrompt_AssistantNilContentDoesNotInjectNullLi
}
normalized := normalizeOpenAIMessagesForPrompt(raw, "")
if len(normalized) != 0 {
t.Fatalf("expected nil-content assistant tool_call-only message omitted, got %#v", normalized)
if len(normalized) != 1 {
t.Fatalf("expected nil-content assistant tool_call-only message preserved, got %#v", normalized)
}
content, _ := normalized[0]["content"].(string)
if strings.Contains(content, "null") {
t.Fatalf("expected no null literal injection, got %q", content)
}
if !strings.Contains(content, "<tool_calls>") {
t.Fatalf("expected assistant tool history in normalized content, got %q", content)
}
}

View File

@@ -47,8 +47,11 @@ func TestBuildOpenAIFinalPrompt_HandlerPathIncludesToolRoundtripSemantics(t *tes
if !strings.Contains(finalPrompt, `"condition":"sunny"`) {
t.Fatalf("handler finalPrompt should preserve tool output content: %q", finalPrompt)
}
if strings.Contains(finalPrompt, "[TOOL_CALL_HISTORY]") || strings.Contains(finalPrompt, "[TOOL_RESULT_HISTORY]") {
t.Fatalf("handler finalPrompt should not include synthetic history markers: %q", finalPrompt)
if !strings.Contains(finalPrompt, "<tool_calls>") {
t.Fatalf("handler finalPrompt should preserve assistant tool history: %q", finalPrompt)
}
if !strings.Contains(finalPrompt, "<tool_name>get_weather</tool_name>") {
t.Fatalf("handler finalPrompt should include tool name history: %q", finalPrompt)
}
}

View File

@@ -113,7 +113,7 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
return
}
result := sse.CollectStream(resp, thinkingEnabled, true)
sanitizedText := sanitizeLeakedToolHistory(result.Text)
sanitizedText := sanitizeLeakedOutput(result.Text)
textParsed := util.ParseStandaloneToolCallsDetailed(sanitizedText, toolNames)
logResponsesToolPolicyRejection(traceID, toolChoice, textParsed, "text")
@@ -124,6 +124,14 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
}
responseObj := openaifmt.BuildResponseObject(responseID, model, finalPrompt, result.Thinking, sanitizedText, toolNames)
if result.OutputTokens > 0 {
if usage, ok := responseObj["usage"].(map[string]any); ok {
usage["output_tokens"] = result.OutputTokens
if input, ok := usage["input_tokens"].(int); ok {
usage["total_tokens"] = input + result.OutputTokens
}
}
}
h.getResponseStore().put(owner, responseID, responseObj)
writeJSON(w, http.StatusOK, responseObj)
}

View File

@@ -1,11 +1,11 @@
package openai
import (
"encoding/json"
"fmt"
"strings"
"ds2api/internal/config"
"ds2api/internal/prompt"
)
func normalizeResponsesInputItem(m map[string]any) map[string]any {
@@ -148,7 +148,7 @@ func normalizeResponsesInputItemWithState(m map[string]any, callNameByID map[str
functionPayload := map[string]any{
"name": name,
"arguments": stringifyToolCallArguments(argsRaw),
"arguments": prompt.StringifyToolCallArguments(argsRaw),
}
call := map[string]any{
"type": "function",
@@ -211,26 +211,3 @@ func normalizeResponsesFallbackPart(m map[string]any) string {
}
return strings.TrimSpace(fmt.Sprintf("%v", m))
}
func stringifyToolCallArguments(v any) string {
switch x := v.(type) {
case nil:
return "{}"
case string:
s := strings.TrimSpace(x)
if s == "" {
return "{}"
}
s = normalizeToolArgumentString(s)
if s == "" {
return "{}"
}
return s
default:
b, err := json.Marshal(x)
if err != nil || len(b) == 0 {
return "{}"
}
return string(b)
}
}

View File

@@ -49,6 +49,7 @@ type responsesStreamRuntime struct {
messagePartAdded bool
sequence int
failed bool
outputTokens int
persistResponse func(obj map[string]any)
}
@@ -97,7 +98,7 @@ func newResponsesStreamRuntime(
func (s *responsesStreamRuntime) finalize() {
finalThinking := s.thinking.String()
finalText := sanitizeLeakedToolHistory(s.text.String())
finalText := sanitizeLeakedOutput(s.text.String())
if s.bufferToolContent {
s.processToolStreamEvents(flushToolSieve(&s.sieve, s.toolNames), true)
@@ -144,6 +145,14 @@ func (s *responsesStreamRuntime) finalize() {
s.closeIncompleteFunctionItems()
obj := s.buildCompletedResponseObject(finalThinking, finalText, detected)
if s.outputTokens > 0 {
if usage, ok := obj["usage"].(map[string]any); ok {
usage["output_tokens"] = s.outputTokens
if input, ok := usage["input_tokens"].(int); ok {
usage["total_tokens"] = input + s.outputTokens
}
}
}
if s.persistResponse != nil {
s.persistResponse(obj)
}
@@ -172,6 +181,9 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.OutputTokens > 0 {
s.outputTokens = parsed.OutputTokens
}
if parsed.ContentFilter || parsed.ErrorMessage != "" || parsed.Stop {
return streamengine.ParsedDecision{Stop: true}
}
@@ -194,7 +206,7 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
continue
}
cleanedText := sanitizeLeakedToolHistory(p.Text)
cleanedText := sanitizeLeakedOutput(p.Text)
if cleanedText == "" {
continue
}

View File

@@ -183,3 +183,53 @@ func TestResponsesNonStreamMixedProseToolPayloadHandlerPath(t *testing.T) {
t.Fatalf("expected function_call output item, got %#v", output)
}
}
func TestChatCompletionsStreamContentFilterStopsNormallyWithoutLeak(t *testing.T) {
statuses := make([]int, 0, 1)
h := &Handler{
Store: mockOpenAIConfig{wideInput: true},
Auth: streamStatusAuthStub{},
DS: streamStatusDSStub{resp: makeOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"合法前缀"}`,
`data: {"p":"response/status","v":"CONTENT_FILTER","accumulated_token_usage":77}`,
`data: {"p":"response/content","v":"CONTENT_FILTER你好这个问题我暂时无法回答让我们换个话题再聊聊吧。"}`,
)},
}
r := chi.NewRouter()
r.Use(captureStatusMiddleware(&statuses))
RegisterRoutes(r, h)
reqBody := `{"model":"deepseek-chat","messages":[{"role":"user","content":"hi"}],"stream":true}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
if len(statuses) != 1 || statuses[0] != http.StatusOK {
t.Fatalf("expected captured status 200, got %#v", statuses)
}
if strings.Contains(rec.Body.String(), "这个问题我暂时无法回答") {
t.Fatalf("expected leaked content-filter suffix to be hidden, body=%s", rec.Body.String())
}
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
if len(frames) == 0 {
t.Fatalf("expected at least one json frame, body=%s", rec.Body.String())
}
last := frames[len(frames)-1]
choices, _ := last["choices"].([]any)
if len(choices) != 1 {
t.Fatalf("expected one choice in final frame, got %#v", last)
}
choice, _ := choices[0].(map[string]any)
if choice["finish_reason"] != "stop" {
t.Fatalf("expected finish_reason=stop for content-filter upstream stop, got %#v", choice["finish_reason"])
}
}

View File

@@ -1,32 +0,0 @@
package openai
import (
"regexp"
)
var leakedToolHistoryPattern = regexp.MustCompile(`(?is)\[TOOL_CALL_HISTORY\][\s\S]*?\[/TOOL_CALL_HISTORY\]|\[TOOL_RESULT_HISTORY\][\s\S]*?\[/TOOL_RESULT_HISTORY\]`)
var emptyJSONFencePattern = regexp.MustCompile("(?is)```json\\s*```")
var leakedToolCallArrayPattern = regexp.MustCompile(`(?is)\[\{\s*"function"\s*:\s*\{[\s\S]*?\}\s*,\s*"id"\s*:\s*"call[^"]*"\s*,\s*"type"\s*:\s*"function"\s*}\]`)
var leakedToolResultBlobPattern = regexp.MustCompile(`(?is)<\s*\|\s*tool\s*\|\s*>\s*\{[\s\S]*?"tool_call_id"\s*:\s*"call[^"]*"\s*}`)
// leakedMetaMarkerPattern matches DeepSeek special tokens in BOTH forms:
// - ASCII underscore: <end_of_sentence>
// - U+2581 variant: <end▁of▁sentence> (used in some DeepSeek outputs)
var leakedMetaMarkerPattern = regexp.MustCompile(`(?i)<[\|]\s*(?:assistant|tool|end[_▁]of[_▁]sentence|end[_▁]of[_▁]thinking)\s*[\|]>`)
// leakedAgentXMLPattern catches agent-style XML tags that leak through when
// the sieve fails to capture them (e.g. incomplete blocks at stream end).
var leakedAgentXMLPattern = regexp.MustCompile(`(?is)</?(?:attempt_completion|ask_followup_question|new_task|result)>`)
func sanitizeLeakedToolHistory(text string) string {
if text == "" {
return text
}
out := leakedToolHistoryPattern.ReplaceAllString(text, "")
out = emptyJSONFencePattern.ReplaceAllString(out, "")
out = leakedToolCallArrayPattern.ReplaceAllString(out, "")
out = leakedToolResultBlobPattern.ReplaceAllString(out, "")
out = leakedMetaMarkerPattern.ReplaceAllString(out, "")
out = leakedAgentXMLPattern.ReplaceAllString(out, "")
return out
}

View File

@@ -1,130 +0,0 @@
package openai
import "testing"
func TestSanitizeLeakedToolHistoryRemovesMarkerBlocks(t *testing.T) {
raw := "前缀\n[TOOL_CALL_HISTORY]\nfunction.name: exec\nfunction.arguments: {}\n[/TOOL_CALL_HISTORY]\n后缀"
got := sanitizeLeakedToolHistory(raw)
if got != "前缀\n\n后缀" {
t.Fatalf("unexpected sanitized content: %q", got)
}
}
func TestSanitizeLeakedToolHistoryPreservesChunkWhitespace(t *testing.T) {
cases := []struct {
name string
raw string
want string
}{
{
name: "trailing space kept",
raw: "Hello ",
want: "Hello ",
},
{
name: "leading newline kept",
raw: "\nworld",
want: "\nworld",
},
{
name: "surrounding whitespace around marker is preserved",
raw: "A \n[TOOL_RESULT_HISTORY]\nfunction.name: exec\nfunction.arguments: {}\n[/TOOL_RESULT_HISTORY]\n B",
want: "A \n\n B",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := sanitizeLeakedToolHistory(tc.raw)
if got != tc.want {
t.Fatalf("unexpected sanitize result, want %q got %q", tc.want, got)
}
})
}
}
func TestSanitizeLeakedToolHistoryRemovesEmptyJSONFence(t *testing.T) {
raw := "before\n```json\n```\nafter"
got := sanitizeLeakedToolHistory(raw)
if got != "before\n\nafter" {
t.Fatalf("unexpected sanitized empty json fence: %q", got)
}
}
func TestFlushToolSieveDropsToolHistoryLeak(t *testing.T) {
var state toolStreamSieveState
chunk := "[TOOL_CALL_HISTORY]\nstatus: already_called\nfunction.name: exec\nfunction.arguments: {}\n[/TOOL_CALL_HISTORY]"
evts := processToolSieveChunk(&state, chunk, []string{"exec"})
if len(evts) != 0 {
t.Fatalf("expected no immediate output before history block is complete, got %+v", evts)
}
flushed := flushToolSieve(&state, []string{"exec"})
if len(flushed) != 0 {
t.Fatalf("expected history block to be swallowed, got %+v", flushed)
}
}
func TestFlushToolSieveDropsToolResultHistoryLeak(t *testing.T) {
var state toolStreamSieveState
chunk := "[TOOL_RESULT_HISTORY]\nstatus: already_called\nfunction.name: exec\nfunction.arguments: {}\n[/TOOL_RESULT_HISTORY]"
evts := processToolSieveChunk(&state, chunk, []string{"exec"})
if len(evts) != 0 {
t.Fatalf("expected no immediate output before result history block is complete, got %+v", evts)
}
flushed := flushToolSieve(&state, []string{"exec"})
if len(flushed) != 0 {
t.Fatalf("expected result history block to be swallowed, got %+v", flushed)
}
}
func TestSanitizeLeakedToolHistoryRemovesLeakedWireToolCallAndResult(t *testing.T) {
raw := "开始\n[{\"function\":{\"arguments\":\"{\\\"command\\\":\\\"java -version\\\"}\",\"name\":\"exec\"},\"id\":\"callb9a321\",\"type\":\"function\"}]< | Tool | >{\"content\":\"openjdk version 21\",\"tool_call_id\":\"callb9a321\"}\n结束"
got := sanitizeLeakedToolHistory(raw)
if got != "开始\n\n结束" {
t.Fatalf("unexpected sanitize result for leaked wire format: %q", got)
}
}
func TestSanitizeLeakedToolHistoryRemovesStandaloneMetaMarkers(t *testing.T) {
raw := "A<| end_of_sentence |><| Assistant |>B<| end_of_thinking |>C<end▁of▁thinking>D<end▁of▁sentence>E"
got := sanitizeLeakedToolHistory(raw)
if got != "ABCDE" {
t.Fatalf("unexpected sanitize result for meta markers: %q", got)
}
}
func TestSanitizeLeakedToolHistoryRemovesAgentXMLLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer</result></attempt_completion>"
got := sanitizeLeakedToolHistory(raw)
if got != "Done.Some final answer" {
t.Fatalf("unexpected sanitize result for agent XML leak: %q", got)
}
}
func TestProcessToolSieveChunkSplitsResultHistoryBoundary(t *testing.T) {
var state toolStreamSieveState
parts := []string{
"Hello ",
"[TOOL_RESULT_HISTORY]\nstatus: already_called\n",
"function.name: exec\nfunction.arguments: {}\n[/TOOL_RESULT_HISTORY]",
"world",
}
var events []toolStreamEvent
for _, p := range parts {
events = append(events, processToolSieveChunk(&state, p, []string{"exec"})...)
}
events = append(events, flushToolSieve(&state, []string{"exec"})...)
var text string
for _, evt := range events {
if evt.Content != "" {
text += evt.Content
}
if len(evt.ToolCalls) > 0 {
t.Fatalf("did not expect parsed tool calls from history leak: %+v", evt.ToolCalls)
}
}
if text != "Hello world" {
t.Fatalf("expected clean text output preserving boundary spaces, got %q", text)
}
}

View File

@@ -183,7 +183,7 @@ func findToolSegmentStart(s string) int {
return -1
}
lower := strings.ToLower(s)
keywords := []string{"tool_calls", "\"function\"", "function.name:", "[tool_call_history]", "[tool_result_history]"}
keywords := []string{"tool_calls", "\"function\"", "function.name:", "\"tool_use\""}
bestKeyIdx := -1
for _, kw := range keywords {
idx := strings.Index(lower, kw)
@@ -191,6 +191,9 @@ func findToolSegmentStart(s string) int {
bestKeyIdx = idx
}
}
if fnKeyIdx := findQuotedFunctionCallKeyStart(s); fnKeyIdx >= 0 && (bestKeyIdx < 0 || fnKeyIdx < bestKeyIdx) {
bestKeyIdx = fnKeyIdx
}
// Also detect XML tool call tags.
for _, tag := range xmlToolTagsToDetect {
idx := strings.Index(lower, tag)
@@ -240,22 +243,22 @@ func consumeToolCapture(state *toolStreamSieveState, toolNames []string) (prefix
lower := strings.ToLower(captured)
keyIdx := -1
keywords := []string{"tool_calls", "\"function\"", "function.name:", "[tool_call_history]", "[tool_result_history]"}
keywords := []string{"tool_calls", "\"function\"", "function.name:", "\"tool_use\""}
for _, kw := range keywords {
idx := strings.Index(lower, kw)
if idx >= 0 && (keyIdx < 0 || idx < keyIdx) {
keyIdx = idx
}
}
if fnKeyIdx := findQuotedFunctionCallKeyStart(captured); fnKeyIdx >= 0 && (keyIdx < 0 || fnKeyIdx < keyIdx) {
keyIdx = fnKeyIdx
}
if keyIdx < 0 {
return "", nil, "", false
}
start := strings.LastIndex(captured[:keyIdx], "{")
if start < 0 {
if blockStart, blockEnd, ok := extractToolHistoryBlock(captured, keyIdx); ok {
return captured[:blockStart], nil, captured[blockEnd:], true
}
start = keyIdx
}
obj, end, ok := extractJSONObjectFrom(captured, start)

View File

@@ -0,0 +1,100 @@
package openai
import "strings"
func findQuotedFunctionCallKeyStart(s string) int {
lower := strings.ToLower(s)
quotedIdx := findFunctionCallKeyStart(lower, `"functioncall"`)
bareIdx := findFunctionCallKeyStart(lower, "functioncall")
// Prefer the quoted JSON key whenever we have a structural match.
// Bare-key detection is only for loose payloads where the quoted form
// is absent.
if quotedIdx >= 0 {
return quotedIdx
}
return bareIdx
}
func findFunctionCallKeyStart(lower, key string) int {
for from := 0; from < len(lower); {
rel := strings.Index(lower[from:], key)
if rel < 0 {
return -1
}
idx := from + rel
if isInsideJSONString(lower, idx) {
from = idx + 1
continue
}
if !hasJSONObjectContextPrefix(lower[:idx]) {
from = idx + 1
continue
}
if !hasJSONKeyBoundary(lower, idx, len(key)) {
from = idx + 1
continue
}
j := idx + len(key)
for j < len(lower) && (lower[j] == ' ' || lower[j] == '\t' || lower[j] == '\r' || lower[j] == '\n') {
j++
}
if j < len(lower) && lower[j] == ':' {
k := j + 1
for k < len(lower) && (lower[k] == ' ' || lower[k] == '\t' || lower[k] == '\r' || lower[k] == '\n') {
k++
}
if k < len(lower) && lower[k] != '{' {
from = idx + 1
continue
}
return idx
}
from = idx + 1
}
return -1
}
func isInsideJSONString(s string, idx int) bool {
inString := false
escaped := false
for i := 0; i < idx; i++ {
c := s[i]
if escaped {
escaped = false
continue
}
if c == '\\' && inString {
escaped = true
continue
}
if c == '"' {
inString = !inString
}
}
return inString
}
func hasJSONObjectContextPrefix(prefix string) bool {
return strings.LastIndex(prefix, "{") >= 0
}
func hasJSONKeyBoundary(s string, idx, keyLen int) bool {
if idx > 0 {
prev := s[idx-1]
if isLowerAlphaNumeric(prev) {
return false
}
}
if end := idx + keyLen; end < len(s) {
next := s[end]
if isLowerAlphaNumeric(next) {
return false
}
}
return true
}
func isLowerAlphaNumeric(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b == '_'
}

View File

@@ -0,0 +1,23 @@
package openai
import "testing"
func TestFindQuotedFunctionCallKeyStart_PrefersEarlierBareKey(t *testing.T) {
input := `{functionCall:{"name":"a","arguments":"{}"},"message":"literal text: \"functionCall\": not a key"}`
got := findQuotedFunctionCallKeyStart(input)
want := 1
if got != want {
t.Fatalf("findQuotedFunctionCallKeyStart() = %d, want %d", got, want)
}
}
func TestFindQuotedFunctionCallKeyStart_PrefersEarlierQuotedKey(t *testing.T) {
input := `{"functionCall":{"name":"a","arguments":"{}"},"note":"functionCall appears in prose"}`
got := findQuotedFunctionCallKeyStart(input)
want := 1
if got != want {
t.Fatalf("findQuotedFunctionCallKeyStart() = %d, want %d", got, want)
}
}

View File

@@ -44,31 +44,6 @@ func extractJSONObjectFrom(text string, start int) (string, int, bool) {
return "", 0, false
}
func extractToolHistoryBlock(captured string, keyIdx int) (start int, end int, ok bool) {
if keyIdx < 0 || keyIdx >= len(captured) {
return 0, 0, false
}
rest := strings.ToLower(captured[keyIdx:])
switch {
case strings.HasPrefix(rest, "[tool_call_history]"):
closeTag := "[/tool_call_history]"
closeIdx := strings.Index(rest, closeTag)
if closeIdx < 0 {
return 0, 0, false
}
return keyIdx, keyIdx + closeIdx + len(closeTag), true
case strings.HasPrefix(rest, "[tool_result_history]"):
closeTag := "[/tool_result_history]"
closeIdx := strings.Index(rest, closeTag)
if closeIdx < 0 {
return 0, 0, false
}
return keyIdx, keyIdx + closeIdx + len(closeTag), true
default:
return 0, 0, false
}
}
func trimWrappingJSONFence(prefix, suffix string) (string, string) {
trimmedPrefix := strings.TrimRight(prefix, " \t\r\n")
fenceIdx := strings.LastIndex(trimmedPrefix, "```")

View File

@@ -34,7 +34,8 @@ type toolCallDelta struct {
Arguments string
}
const toolSieveContextTailLimit = 256
// Keep in sync with JS TOOL_SIEVE_CONTEXT_TAIL_LIMIT.
const toolSieveContextTailLimit = 2048
func (s *toolStreamSieveState) resetIncrementalToolState() {
s.disableDeltas = false

View File

@@ -104,6 +104,7 @@ func TestFindToolSegmentStartDetectsXMLToolCalls(t *testing.T) {
want int
}{
{"tool_calls_tag", "some text <tool_calls>\n", 10},
{"gemini_function_call_json", `some text {"functionCall":{"name":"search","args":{"q":"latest"}}}`, 10},
{"tool_call_tag", "prefix <tool_call>\n", 7},
{"invoke_tag", "text <invoke name=\"foo\">body</invoke>", 5},
{"function_call_tag", "<function_call name=\"foo\">body</function_call>", 0},
@@ -119,6 +120,81 @@ func TestFindToolSegmentStartDetectsXMLToolCalls(t *testing.T) {
}
}
func TestFindToolSegmentStartIgnoresFunctionCallProse(t *testing.T) {
input := "Please explain the functionCall API field and how clients should parse it."
if got := findToolSegmentStart(input); got != -1 {
t.Fatalf("expected no tool segment start for prose, got %d", got)
}
}
func TestFindToolSegmentStartDetectsQuotedFunctionCallKey(t *testing.T) {
input := `prefix {"functionCall": {"name":"search_web","args":{"query":"x"}}}`
want := strings.Index(input, "{")
if got := findToolSegmentStart(input); got != want {
t.Fatalf("expected JSON object start %d, got %d", want, got)
}
}
func TestFindToolSegmentStartDetectsLooseFunctionCallKey(t *testing.T) {
input := `prefix {functionCall: {"name":"search_web","args":{"query":"x"}}}`
want := strings.Index(input, "{")
if got := findToolSegmentStart(input); got != want {
t.Fatalf("expected JSON object start %d, got %d", want, got)
}
}
func TestFindToolSegmentStartPrefersQuotedFunctionCallOverEarlierBareProse(t *testing.T) {
input := `prefix {note} functionCall: docs hint {"functionCall":{"name":"search_web","args":{"query":"x"}}}`
want := strings.Index(input, `{"functionCall"`)
if got := findToolSegmentStart(input); got != want {
t.Fatalf("expected quoted functionCall JSON start %d, got %d", want, got)
}
}
func TestFindToolSegmentStartIgnoresLooseFunctionCallProse(t *testing.T) {
input := "Please explain why functionCall: is used in documentation examples."
if got := findToolSegmentStart(input); got != -1 {
t.Fatalf("expected no tool segment start for prose, got %d", got)
}
}
func TestProcessToolSieveDoesNotBufferFunctionCallProse(t *testing.T) {
var state toolStreamSieveState
chunk := "Please explain the functionCall API field and keep streaming this sentence."
events := processToolSieveChunk(&state, chunk, []string{"search_web"})
var text string
for _, evt := range events {
text += evt.Content
if len(evt.ToolCalls) > 0 {
t.Fatalf("expected no tool calls for prose, got %#v", evt.ToolCalls)
}
}
if text != chunk {
t.Fatalf("expected prose to pass through immediately, got %q", text)
}
}
func TestProcessToolSieveDetectsGeminiFunctionCallPayload(t *testing.T) {
var state toolStreamSieveState
events := processToolSieveChunk(&state, `{"functionCall":{"name":"search_web","args":{"query":"latest"}}}`, []string{"search_web"})
events = append(events, flushToolSieve(&state, []string{"search_web"})...)
var textContent string
var toolCalls int
for _, evt := range events {
if evt.Content != "" {
textContent += evt.Content
}
toolCalls += len(evt.ToolCalls)
}
if toolCalls != 1 {
t.Fatalf("expected one tool call from functionCall payload, got events=%#v", events)
}
if strings.Contains(strings.ToLower(textContent), "functioncall") {
t.Fatalf("functionCall json leaked into text content: %q", textContent)
}
}
func TestFindPartialXMLToolTagStart(t *testing.T) {
cases := []struct {
name string

View File

@@ -93,18 +93,16 @@ func (h *Handler) handleVercelStreamPrepare(w http.ResponseWriter, r *http.Reque
}
leased = true
writeJSON(w, http.StatusOK, map[string]any{
"session_id": sessionID,
"lease_id": leaseID,
"model": stdReq.ResponseModel,
"final_prompt": stdReq.FinalPrompt,
"thinking_enabled": stdReq.Thinking,
"search_enabled": stdReq.Search,
"tool_names": stdReq.ToolNames,
"toolcall_feature_match": h.toolcallFeatureMatchEnabled(),
"toolcall_early_emit_high": h.toolcallEarlyEmitHighConfidence(),
"deepseek_token": a.DeepSeekToken,
"pow_header": powHeader,
"payload": payload,
"session_id": sessionID,
"lease_id": leaseID,
"model": stdReq.ResponseModel,
"final_prompt": stdReq.FinalPrompt,
"thinking_enabled": stdReq.Thinking,
"search_enabled": stdReq.Search,
"tool_names": stdReq.ToolNames,
"deepseek_token": a.DeepSeekToken,
"pow_header": powHeader,
"payload": payload,
})
}

View File

@@ -21,6 +21,9 @@ type ConfigStore interface {
Update(mutator func(*config.Config) error) error
ExportJSONAndBase64() (string, string, error)
IsEnvBacked() bool
IsEnvWritebackEnabled() bool
HasEnvConfigSource() bool
ConfigPath() string
SetVercelSync(hash string, ts int64) error
AdminPasswordHash() string
AdminJWTExpireHours() int
@@ -28,6 +31,7 @@ type ConfigStore interface {
RuntimeAccountMaxInflight() int
RuntimeAccountMaxQueue(defaultSize int) int
RuntimeGlobalMaxInflight(defaultSize int) int
RuntimeTokenRefreshIntervalHours() int
AutoDeleteSessions() bool
}

View File

@@ -120,12 +120,6 @@ func (h *Handler) configImport(w http.ResponseWriter, r *http.Request) {
next.ModelAliases[k] = v
}
}
if strings.TrimSpace(incoming.Toolcall.Mode) != "" {
next.Toolcall.Mode = incoming.Toolcall.Mode
}
if strings.TrimSpace(incoming.Toolcall.EarlyEmitConfidence) != "" {
next.Toolcall.EarlyEmitConfidence = incoming.Toolcall.EarlyEmitConfidence
}
if incoming.Responses.StoreTTLSeconds > 0 {
next.Responses.StoreTTLSeconds = incoming.Responses.StoreTTLSeconds
}
@@ -150,6 +144,9 @@ func (h *Handler) configImport(w http.ResponseWriter, r *http.Request) {
if incoming.Runtime.GlobalMaxInflight > 0 {
next.Runtime.GlobalMaxInflight = incoming.Runtime.GlobalMaxInflight
}
if incoming.Runtime.TokenRefreshIntervalHours > 0 {
next.Runtime.TokenRefreshIntervalHours = incoming.Runtime.TokenRefreshIntervalHours
}
}
normalizeSettingsConfig(&next)

View File

@@ -8,9 +8,12 @@ import (
func (h *Handler) getConfig(w http.ResponseWriter, _ *http.Request) {
snap := h.Store.Snapshot()
safe := map[string]any{
"keys": snap.Keys,
"accounts": []map[string]any{},
"env_backed": h.Store.IsEnvBacked(),
"keys": snap.Keys,
"accounts": []map[string]any{},
"env_backed": h.Store.IsEnvBacked(),
"env_source_present": h.Store.HasEnvConfigSource(),
"env_writeback_enabled": h.Store.IsEnvWritebackEnabled(),
"config_path": h.Store.ConfigPath(),
"claude_mapping": func() map[string]string {
if len(snap.ClaudeMapping) > 0 {
return snap.ClaudeMapping

View File

@@ -21,16 +21,15 @@ func boolFrom(v any) bool {
}
}
func parseSettingsUpdateRequest(req map[string]any) (*config.AdminConfig, *config.RuntimeConfig, *config.ToolcallConfig, *config.ResponsesConfig, *config.EmbeddingsConfig, *config.AutoDeleteConfig, map[string]string, map[string]string, error) {
func parseSettingsUpdateRequest(req map[string]any) (*config.AdminConfig, *config.RuntimeConfig, *config.ResponsesConfig, *config.EmbeddingsConfig, *config.AutoDeleteConfig, map[string]string, map[string]string, error) {
var (
adminCfg *config.AdminConfig
runtimeCfg *config.RuntimeConfig
toolcallCfg *config.ToolcallConfig
respCfg *config.ResponsesConfig
embCfg *config.EmbeddingsConfig
autoDeleteCfg *config.AutoDeleteConfig
claudeMap map[string]string
aliasMap map[string]string
adminCfg *config.AdminConfig
runtimeCfg *config.RuntimeConfig
respCfg *config.ResponsesConfig
embCfg *config.EmbeddingsConfig
autoDeleteCfg *config.AutoDeleteConfig
claudeMap map[string]string
aliasMap map[string]string
)
if raw, ok := req["admin"].(map[string]any); ok {
@@ -38,7 +37,7 @@ func parseSettingsUpdateRequest(req map[string]any) (*config.AdminConfig, *confi
if v, exists := raw["jwt_expire_hours"]; exists {
n := intFrom(v)
if n < 1 || n > 720 {
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("admin.jwt_expire_hours must be between 1 and 720")
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("admin.jwt_expire_hours must be between 1 and 720")
}
cfg.JWTExpireHours = n
}
@@ -50,59 +49,43 @@ func parseSettingsUpdateRequest(req map[string]any) (*config.AdminConfig, *confi
if v, exists := raw["account_max_inflight"]; exists {
n := intFrom(v)
if n < 1 || n > 256 {
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.account_max_inflight must be between 1 and 256")
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.account_max_inflight must be between 1 and 256")
}
cfg.AccountMaxInflight = n
}
if v, exists := raw["account_max_queue"]; exists {
n := intFrom(v)
if n < 1 || n > 200000 {
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.account_max_queue must be between 1 and 200000")
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.account_max_queue must be between 1 and 200000")
}
cfg.AccountMaxQueue = n
}
if v, exists := raw["global_max_inflight"]; exists {
n := intFrom(v)
if n < 1 || n > 200000 {
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.global_max_inflight must be between 1 and 200000")
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.global_max_inflight must be between 1 and 200000")
}
cfg.GlobalMaxInflight = n
}
if v, exists := raw["token_refresh_interval_hours"]; exists {
n := intFrom(v)
if n < 1 || n > 720 {
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.token_refresh_interval_hours must be between 1 and 720")
}
cfg.TokenRefreshIntervalHours = n
}
if cfg.AccountMaxInflight > 0 && cfg.GlobalMaxInflight > 0 && cfg.GlobalMaxInflight < cfg.AccountMaxInflight {
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.global_max_inflight must be >= runtime.account_max_inflight")
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("runtime.global_max_inflight must be >= runtime.account_max_inflight")
}
runtimeCfg = cfg
}
if raw, ok := req["toolcall"].(map[string]any); ok {
cfg := &config.ToolcallConfig{}
if v, exists := raw["mode"]; exists {
mode := strings.ToLower(strings.TrimSpace(fmt.Sprintf("%v", v)))
switch mode {
case "feature_match", "off":
cfg.Mode = mode
default:
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("toolcall.mode must be feature_match or off")
}
}
if v, exists := raw["early_emit_confidence"]; exists {
level := strings.ToLower(strings.TrimSpace(fmt.Sprintf("%v", v)))
switch level {
case "high", "low", "off":
cfg.EarlyEmitConfidence = level
default:
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("toolcall.early_emit_confidence must be high, low or off")
}
}
toolcallCfg = cfg
}
if raw, ok := req["responses"].(map[string]any); ok {
cfg := &config.ResponsesConfig{}
if v, exists := raw["store_ttl_seconds"]; exists {
n := intFrom(v)
if n < 30 || n > 86400 {
return nil, nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("responses.store_ttl_seconds must be between 30 and 86400")
return nil, nil, nil, nil, nil, nil, nil, fmt.Errorf("responses.store_ttl_seconds must be between 30 and 86400")
}
cfg.StoreTTLSeconds = n
}
@@ -150,5 +133,5 @@ func parseSettingsUpdateRequest(req map[string]any) (*config.AdminConfig, *confi
autoDeleteCfg = cfg
}
return adminCfg, runtimeCfg, toolcallCfg, respCfg, embCfg, autoDeleteCfg, claudeMap, aliasMap, nil
return adminCfg, runtimeCfg, respCfg, embCfg, autoDeleteCfg, claudeMap, aliasMap, nil
}

View File

@@ -21,11 +21,11 @@ func (h *Handler) getSettings(w http.ResponseWriter, _ *http.Request) {
"default_password_warning": authn.UsingDefaultAdminKey(h.Store),
},
"runtime": map[string]any{
"account_max_inflight": h.Store.RuntimeAccountMaxInflight(),
"account_max_queue": h.Store.RuntimeAccountMaxQueue(recommended),
"global_max_inflight": h.Store.RuntimeGlobalMaxInflight(recommended),
"account_max_inflight": h.Store.RuntimeAccountMaxInflight(),
"account_max_queue": h.Store.RuntimeAccountMaxQueue(recommended),
"global_max_inflight": h.Store.RuntimeGlobalMaxInflight(recommended),
"token_refresh_interval_hours": h.Store.RuntimeTokenRefreshIntervalHours(),
},
"toolcall": snap.Toolcall,
"responses": snap.Responses,
"embeddings": snap.Embeddings,
"auto_delete": snap.AutoDelete,

View File

@@ -14,6 +14,9 @@ func validateMergedRuntimeSettings(current config.RuntimeConfig, incoming *confi
if incoming.GlobalMaxInflight > 0 {
merged.GlobalMaxInflight = incoming.GlobalMaxInflight
}
if incoming.TokenRefreshIntervalHours > 0 {
merged.TokenRefreshIntervalHours = incoming.TokenRefreshIntervalHours
}
}
return validateRuntimeSettings(merged)
}

View File

@@ -28,6 +28,25 @@ func TestGetSettingsDefaultPasswordWarning(t *testing.T) {
}
}
func TestGetSettingsIncludesTokenRefreshInterval(t *testing.T) {
h := newAdminTestHandler(t, `{
"keys":["k1"],
"runtime":{"token_refresh_interval_hours":9}
}`)
req := httptest.NewRequest(http.MethodGet, "/admin/settings", nil)
rec := httptest.NewRecorder()
h.getSettings(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
var body map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &body)
runtime, _ := body["runtime"].(map[string]any)
if got := intFrom(runtime["token_refresh_interval_hours"]); got != 9 {
t.Fatalf("expected token_refresh_interval_hours=9, got %d body=%v", got, body)
}
}
func TestUpdateSettingsValidation(t *testing.T) {
h := newAdminTestHandler(t, `{"keys":["k1"]}`)
payload := map[string]any{
@@ -44,6 +63,25 @@ func TestUpdateSettingsValidation(t *testing.T) {
}
}
func TestUpdateSettingsValidationRejectsTokenRefreshInterval(t *testing.T) {
h := newAdminTestHandler(t, `{"keys":["k1"]}`)
payload := map[string]any{
"runtime": map[string]any{
"token_refresh_interval_hours": 0,
},
}
b, _ := json.Marshal(payload)
req := httptest.NewRequest(http.MethodPut, "/admin/settings", bytes.NewReader(b))
rec := httptest.NewRecorder()
h.updateSettings(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", rec.Code, rec.Body.String())
}
if !bytes.Contains(rec.Body.Bytes(), []byte("runtime.token_refresh_interval_hours")) {
t.Fatalf("expected token refresh validation detail, got %s", rec.Body.String())
}
}
func TestUpdateSettingsValidationWithMergedRuntimeSnapshot(t *testing.T) {
h := newAdminTestHandler(t, `{
"keys":["k1"],
@@ -126,6 +164,29 @@ func TestUpdateSettingsHotReloadRuntime(t *testing.T) {
}
}
func TestUpdateSettingsHotReloadTokenRefreshInterval(t *testing.T) {
h := newAdminTestHandler(t, `{
"keys":["k1"],
"runtime":{"token_refresh_interval_hours":6}
}`)
payload := map[string]any{
"runtime": map[string]any{
"token_refresh_interval_hours": 12,
},
}
b, _ := json.Marshal(payload)
req := httptest.NewRequest(http.MethodPut, "/admin/settings", bytes.NewReader(b))
rec := httptest.NewRecorder()
h.updateSettings(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
if got := h.Store.RuntimeTokenRefreshIntervalHours(); got != 12 {
t.Fatalf("token_refresh_interval_hours=%d want=12", got)
}
}
func TestUpdateSettingsPasswordInvalidatesOldJWT(t *testing.T) {
hash := authn.HashAdminPassword("old-password")
h := newAdminTestHandler(t, `{"admin":{"password_hash":"`+hash+`"}}`)
@@ -207,6 +268,30 @@ func TestConfigImportMergeAndReplace(t *testing.T) {
}
}
func TestConfigImportAppliesTokenRefreshInterval(t *testing.T) {
h := newAdminTestHandler(t, `{"keys":["k1"]}`)
replace := map[string]any{
"mode": "replace",
"config": map[string]any{
"keys": []any{"k9"},
"runtime": map[string]any{
"token_refresh_interval_hours": 11,
},
},
}
replaceBytes, _ := json.Marshal(replace)
replaceReq := httptest.NewRequest(http.MethodPost, "/admin/config/import?mode=replace", bytes.NewReader(replaceBytes))
replaceRec := httptest.NewRecorder()
h.configImport(replaceRec, replaceReq)
if replaceRec.Code != http.StatusOK {
t.Fatalf("replace status=%d body=%s", replaceRec.Code, replaceRec.Body.String())
}
if got := h.Store.RuntimeTokenRefreshIntervalHours(); got != 11 {
t.Fatalf("token_refresh_interval_hours=%d want=11", got)
}
}
func TestConfigImportRejectsInvalidRuntimeBounds(t *testing.T) {
h := newAdminTestHandler(t, `{"keys":["k1"]}`)
payload := map[string]any{

View File

@@ -17,7 +17,7 @@ func (h *Handler) updateSettings(w http.ResponseWriter, r *http.Request) {
return
}
adminCfg, runtimeCfg, toolcallCfg, responsesCfg, embeddingsCfg, autoDeleteCfg, claudeMap, aliasMap, err := parseSettingsUpdateRequest(req)
adminCfg, runtimeCfg, responsesCfg, embeddingsCfg, autoDeleteCfg, claudeMap, aliasMap, err := parseSettingsUpdateRequest(req)
if err != nil {
writeJSON(w, http.StatusBadRequest, map[string]any{"detail": err.Error()})
return
@@ -45,13 +45,8 @@ func (h *Handler) updateSettings(w http.ResponseWriter, r *http.Request) {
if runtimeCfg.GlobalMaxInflight > 0 {
c.Runtime.GlobalMaxInflight = runtimeCfg.GlobalMaxInflight
}
}
if toolcallCfg != nil {
if strings.TrimSpace(toolcallCfg.Mode) != "" {
c.Toolcall.Mode = strings.TrimSpace(toolcallCfg.Mode)
}
if strings.TrimSpace(toolcallCfg.EarlyEmitConfidence) != "" {
c.Toolcall.EarlyEmitConfidence = strings.TrimSpace(toolcallCfg.EarlyEmitConfidence)
if runtimeCfg.TokenRefreshIntervalHours > 0 {
c.Runtime.TokenRefreshIntervalHours = runtimeCfg.TokenRefreshIntervalHours
}
}
if responsesCfg != nil && responsesCfg.StoreTTLSeconds > 0 {

View File

@@ -12,8 +12,6 @@ func normalizeSettingsConfig(c *config.Config) {
return
}
c.Admin.PasswordHash = strings.TrimSpace(c.Admin.PasswordHash)
c.Toolcall.Mode = strings.ToLower(strings.TrimSpace(c.Toolcall.Mode))
c.Toolcall.EarlyEmitConfidence = strings.ToLower(strings.TrimSpace(c.Toolcall.EarlyEmitConfidence))
c.Embeddings.Provider = strings.TrimSpace(c.Embeddings.Provider)
}
@@ -27,20 +25,6 @@ func validateSettingsConfig(c config.Config) error {
if c.Responses.StoreTTLSeconds != 0 && (c.Responses.StoreTTLSeconds < 30 || c.Responses.StoreTTLSeconds > 86400) {
return fmt.Errorf("responses.store_ttl_seconds must be between 30 and 86400")
}
if mode := strings.TrimSpace(c.Toolcall.Mode); mode != "" {
switch mode {
case "feature_match", "off":
default:
return fmt.Errorf("toolcall.mode must be feature_match or off")
}
}
if level := strings.TrimSpace(c.Toolcall.EarlyEmitConfidence); level != "" {
switch level {
case "high", "low", "off":
default:
return fmt.Errorf("toolcall.early_emit_confidence must be high, low or off")
}
}
if c.Embeddings.Provider != "" && strings.TrimSpace(c.Embeddings.Provider) == "" {
return fmt.Errorf("embeddings.provider cannot be empty")
}
@@ -57,6 +41,9 @@ func validateRuntimeSettings(runtime config.RuntimeConfig) error {
if runtime.GlobalMaxInflight != 0 && (runtime.GlobalMaxInflight < 1 || runtime.GlobalMaxInflight > 200000) {
return fmt.Errorf("runtime.global_max_inflight must be between 1 and 200000")
}
if runtime.TokenRefreshIntervalHours != 0 && (runtime.TokenRefreshIntervalHours < 1 || runtime.TokenRefreshIntervalHours > 720) {
return fmt.Errorf("runtime.token_refresh_interval_hours must be between 1 and 720")
}
if runtime.AccountMaxInflight > 0 && runtime.GlobalMaxInflight > 0 && runtime.GlobalMaxInflight < runtime.AccountMaxInflight {
return fmt.Errorf("runtime.global_max_inflight must be >= runtime.account_max_inflight")
}

View File

@@ -204,6 +204,45 @@ func TestSwitchAccountNilTriedAccounts(t *testing.T) {
r.Release(a)
}
func TestSwitchAccountSkipsLoginFailureAndContinues(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["managed-key"],
"accounts":[
{"email":"acc1@test.com","password":"pwd","token":"t1"},
{"email":"acc2@test.com","password":"pwd"},
{"email":"acc3@test.com","password":"pwd","token":"t3"}
]
}`)
store := config.LoadStore()
pool := account.NewPool(store)
r := NewResolver(store, pool, func(_ context.Context, acc config.Account) (string, error) {
if acc.Email == "acc2@test.com" {
return "", errors.New("login failed")
}
return "new-token", nil
})
req, _ := http.NewRequest("POST", "/", nil)
req.Header.Set("Authorization", "Bearer managed-key")
a, err := r.Determine(req)
if err != nil {
t.Fatalf("determine failed: %v", err)
}
defer r.Release(a)
if a.AccountID != "acc1@test.com" {
t.Fatalf("expected first account, got %q", a.AccountID)
}
if !r.SwitchAccount(context.Background(), a) {
t.Fatal("expected switch to succeed after skipping failed account")
}
if a.AccountID != "acc3@test.com" {
t.Fatalf("expected fallback to third account, got %q", a.AccountID)
}
if !a.TriedAccounts["acc2@test.com"] {
t.Fatalf("expected failed account to be marked as tried")
}
}
// ─── Release edge cases ─────────────────────────────────────────────
func TestReleaseNilAuth(t *testing.T) {

View File

@@ -40,18 +40,16 @@ type Resolver struct {
Pool *account.Pool
Login LoginFunc
mu sync.Mutex
tokenRefreshedAt map[string]time.Time
tokenRefreshInterval time.Duration
mu sync.Mutex
tokenRefreshedAt map[string]time.Time
}
func NewResolver(store *config.Store, pool *account.Pool, login LoginFunc) *Resolver {
return &Resolver{
Store: store,
Pool: pool,
Login: login,
tokenRefreshedAt: map[string]time.Time{},
tokenRefreshInterval: 6 * time.Hour,
Store: store,
Pool: pool,
Login: login,
tokenRefreshedAt: map[string]time.Time{},
}
}
@@ -72,25 +70,53 @@ func (r *Resolver) Determine(req *http.Request) (*RequestAuth, error) {
}, nil
}
target := strings.TrimSpace(req.Header.Get("X-Ds2-Target-Account"))
acc, ok := r.Pool.AcquireWait(ctx, target, nil)
if !ok {
return nil, ErrNoAccount
}
a := &RequestAuth{
UseConfigToken: true,
CallerID: callerID,
AccountID: acc.Identifier(),
Account: acc,
TriedAccounts: map[string]bool{},
resolver: r,
}
if err := r.ensureManagedToken(ctx, a); err != nil {
r.Pool.Release(a.AccountID)
a, err := r.acquireManagedRequestAuth(ctx, callerID, target)
if err != nil {
return nil, err
}
return a, nil
}
func (r *Resolver) acquireManagedRequestAuth(ctx context.Context, callerID, target string) (*RequestAuth, error) {
tried := map[string]bool{}
var lastEnsureErr error
for {
if target == "" && len(tried) >= len(r.Store.Accounts()) {
if lastEnsureErr != nil {
return nil, lastEnsureErr
}
return nil, ErrNoAccount
}
acc, ok := r.Pool.AcquireWait(ctx, target, tried)
if !ok {
if lastEnsureErr != nil {
return nil, lastEnsureErr
}
return nil, ErrNoAccount
}
a := &RequestAuth{
UseConfigToken: true,
CallerID: callerID,
AccountID: acc.Identifier(),
Account: acc,
TriedAccounts: tried,
resolver: r,
}
if err := r.ensureManagedToken(ctx, a); err != nil {
lastEnsureErr = err
tried[a.AccountID] = true
r.Pool.Release(a.AccountID)
if target != "" {
return nil, err
}
continue
}
return a, nil
}
}
// DetermineCaller resolves caller identity without acquiring any pooled account.
// Use this for local-cache lookup routes that only need tenant isolation.
func (r *Resolver) DetermineCaller(req *http.Request) (*RequestAuth, error) {
@@ -166,16 +192,20 @@ func (r *Resolver) SwitchAccount(ctx context.Context, a *RequestAuth) bool {
a.TriedAccounts[a.AccountID] = true
r.Pool.Release(a.AccountID)
}
acc, ok := r.Pool.Acquire("", a.TriedAccounts)
if !ok {
return false
for {
acc, ok := r.Pool.Acquire("", a.TriedAccounts)
if !ok {
return false
}
a.Account = acc
a.AccountID = acc.Identifier()
if err := r.ensureManagedToken(ctx, a); err != nil {
a.TriedAccounts[a.AccountID] = true
r.Pool.Release(a.AccountID)
continue
}
return true
}
a.Account = acc
a.AccountID = acc.Identifier()
if err := r.ensureManagedToken(ctx, a); err != nil {
return false
}
return true
}
func (r *Resolver) Release(a *RequestAuth) {
@@ -232,10 +262,14 @@ func (r *Resolver) ensureManagedToken(ctx context.Context, a *RequestAuth) error
}
func (r *Resolver) shouldForceRefresh(accountID string) bool {
if r == nil || r.Store == nil {
return false
}
if strings.TrimSpace(accountID) == "" {
return false
}
if r.tokenRefreshInterval <= 0 {
intervalHours := r.Store.RuntimeTokenRefreshIntervalHours()
if intervalHours <= 0 {
return false
}
now := time.Now()
@@ -246,7 +280,7 @@ func (r *Resolver) shouldForceRefresh(accountID string) bool {
r.tokenRefreshedAt[accountID] = now
return false
}
return now.Sub(last) >= r.tokenRefreshInterval
return now.Sub(last) >= time.Duration(intervalHours)*time.Hour
}
func (r *Resolver) markTokenRefreshedNow(accountID string) {

View File

@@ -2,6 +2,7 @@ package auth
import (
"context"
"errors"
"net/http"
"sync/atomic"
"testing"
@@ -244,3 +245,153 @@ func TestDetermineManagedAccountForcesRefreshEverySixHours(t *testing.T) {
t.Fatalf("expected exactly one forced refresh login, got %d", got)
}
}
func TestDetermineManagedAccountUsesUpdatedRefreshInterval(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["managed-key"],
"accounts":[{"email":"acc@example.com","password":"pwd","token":"seed-token"}],
"runtime":{"token_refresh_interval_hours":6}
}`)
store := config.LoadStore()
if err := store.UpdateAccountToken("acc@example.com", "seed-token"); err != nil {
t.Fatalf("update token failed: %v", err)
}
pool := account.NewPool(store)
var loginCount int32
resolver := NewResolver(store, pool, func(_ context.Context, _ config.Account) (string, error) {
n := atomic.AddInt32(&loginCount, 1)
return "fresh-token-" + string(rune('0'+n)), nil
})
req, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
req.Header.Set("x-api-key", "managed-key")
a1, err := resolver.Determine(req)
if err != nil {
t.Fatalf("determine failed: %v", err)
}
if a1.DeepSeekToken != "seed-token" {
t.Fatalf("expected initial token without forced refresh, got %q", a1.DeepSeekToken)
}
resolver.Release(a1)
if got := atomic.LoadInt32(&loginCount); got != 0 {
t.Fatalf("expected no login before runtime update, got %d", got)
}
if err := store.Update(func(c *config.Config) error {
c.Runtime.TokenRefreshIntervalHours = 1
return nil
}); err != nil {
t.Fatalf("update runtime failed: %v", err)
}
resolver.mu.Lock()
resolver.tokenRefreshedAt["acc@example.com"] = time.Now().Add(-2 * time.Hour)
resolver.mu.Unlock()
a2, err := resolver.Determine(req)
if err != nil {
t.Fatalf("determine after runtime update failed: %v", err)
}
defer resolver.Release(a2)
if a2.DeepSeekToken != "fresh-token-1" {
t.Fatalf("expected refreshed token after runtime update, got %q", a2.DeepSeekToken)
}
if got := atomic.LoadInt32(&loginCount); got != 1 {
t.Fatalf("expected exactly one login after runtime update, got %d", got)
}
}
func TestDetermineManagedAccountRetriesOtherAccountOnLoginFailure(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["managed-key"],
"accounts":[
{"email":"bad@example.com","password":"pwd"},
{"email":"good@example.com","password":"pwd","token":"good-token"}
]
}`)
store := config.LoadStore()
pool := account.NewPool(store)
resolver := NewResolver(store, pool, func(_ context.Context, acc config.Account) (string, error) {
if acc.Email == "bad@example.com" {
return "", errors.New("stale account")
}
return "fresh-good-token", nil
})
req, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
req.Header.Set("x-api-key", "managed-key")
a, err := resolver.Determine(req)
if err != nil {
t.Fatalf("determine failed: %v", err)
}
defer resolver.Release(a)
if a.AccountID != "good@example.com" {
t.Fatalf("expected fallback to good account, got %q", a.AccountID)
}
if a.DeepSeekToken == "" {
t.Fatal("expected non-empty token from fallback account")
}
if !a.TriedAccounts["bad@example.com"] {
t.Fatalf("expected bad account to be tracked as tried")
}
}
func TestDetermineTargetAccountDoesNotFallbackOnLoginFailure(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["managed-key"],
"accounts":[
{"email":"bad@example.com","password":"pwd"},
{"email":"good@example.com","password":"pwd","token":"good-token"}
]
}`)
store := config.LoadStore()
pool := account.NewPool(store)
resolver := NewResolver(store, pool, func(_ context.Context, acc config.Account) (string, error) {
if acc.Email == "bad@example.com" {
return "", errors.New("stale account")
}
return "fresh-good-token", nil
})
req, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
req.Header.Set("x-api-key", "managed-key")
req.Header.Set("X-Ds2-Target-Account", "bad@example.com")
_, err := resolver.Determine(req)
if err == nil {
t.Fatal("expected determine to fail for broken target account")
}
}
func TestDetermineManagedAccountReturnsLastEnsureErrorWhenAllFail(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["managed-key"],
"accounts":[
{"email":"bad1@example.com","password":"pwd"},
{"email":"bad2@example.com","password":"pwd"}
]
}`)
store := config.LoadStore()
pool := account.NewPool(store)
ensureErr := errors.New("all credentials stale")
resolver := NewResolver(store, pool, func(_ context.Context, _ config.Account) (string, error) {
return "", ensureErr
})
req, _ := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
req.Header.Set("x-api-key", "managed-key")
_, err := resolver.Determine(req)
if err == nil {
t.Fatal("expected determine to fail")
}
if !errors.Is(err, ensureErr) {
t.Fatalf("expected ensure error, got %v", err)
}
if errors.Is(err, ErrNoAccount) {
t.Fatalf("expected auth-style ensure error, got ErrNoAccount")
}
}

View File

@@ -32,15 +32,12 @@ func (c Config) MarshalJSON() ([]byte, error) {
if strings.TrimSpace(c.Admin.PasswordHash) != "" || c.Admin.JWTExpireHours > 0 || c.Admin.JWTValidAfterUnix > 0 {
m["admin"] = c.Admin
}
if c.Runtime.AccountMaxInflight > 0 || c.Runtime.AccountMaxQueue > 0 || c.Runtime.GlobalMaxInflight > 0 {
if c.Runtime.AccountMaxInflight > 0 || c.Runtime.AccountMaxQueue > 0 || c.Runtime.GlobalMaxInflight > 0 || c.Runtime.TokenRefreshIntervalHours > 0 {
m["runtime"] = c.Runtime
}
if c.Compat.WideInputStrictOutput != nil {
m["compat"] = c.Compat
}
if strings.TrimSpace(c.Toolcall.Mode) != "" || strings.TrimSpace(c.Toolcall.EarlyEmitConfidence) != "" {
m["toolcall"] = c.Toolcall
}
if c.Responses.StoreTTLSeconds > 0 {
m["responses"] = c.Responses
}
@@ -98,9 +95,7 @@ func (c *Config) UnmarshalJSON(b []byte) error {
return fmt.Errorf("invalid field %q: %w", k, err)
}
case "toolcall":
if err := json.Unmarshal(v, &c.Toolcall); err != nil {
return fmt.Errorf("invalid field %q: %w", k, err)
}
// Legacy field ignored. Toolcall policy is fixed and no longer configurable.
case "responses":
if err := json.Unmarshal(v, &c.Responses); err != nil {
return fmt.Errorf("invalid field %q: %w", k, err)
@@ -143,7 +138,6 @@ func (c Config) Clone() Config {
Compat: CompatConfig{
WideInputStrictOutput: cloneBoolPtr(c.Compat.WideInputStrictOutput),
},
Toolcall: c.Toolcall,
Responses: c.Responses,
Embeddings: c.Embeddings,
AutoDelete: c.AutoDelete,

View File

@@ -9,7 +9,6 @@ type Config struct {
Admin AdminConfig `json:"admin,omitempty"`
Runtime RuntimeConfig `json:"runtime,omitempty"`
Compat CompatConfig `json:"compat,omitempty"`
Toolcall ToolcallConfig `json:"toolcall,omitempty"`
Responses ResponsesConfig `json:"responses,omitempty"`
Embeddings EmbeddingsConfig `json:"embeddings,omitempty"`
AutoDelete AutoDeleteConfig `json:"auto_delete"`
@@ -62,14 +61,10 @@ type AdminConfig struct {
}
type RuntimeConfig struct {
AccountMaxInflight int `json:"account_max_inflight,omitempty"`
AccountMaxQueue int `json:"account_max_queue,omitempty"`
GlobalMaxInflight int `json:"global_max_inflight,omitempty"`
}
type ToolcallConfig struct {
Mode string `json:"mode,omitempty"`
EarlyEmitConfidence string `json:"early_emit_confidence,omitempty"`
AccountMaxInflight int `json:"account_max_inflight,omitempty"`
AccountMaxQueue int `json:"account_max_queue,omitempty"`
GlobalMaxInflight int `json:"global_max_inflight,omitempty"`
TokenRefreshIntervalHours int `json:"token_refresh_interval_hours,omitempty"`
}
type ResponsesConfig struct {

View File

@@ -104,6 +104,9 @@ func TestConfigJSONRoundtrip(t *testing.T) {
"fast": "deepseek-chat",
"slow": "deepseek-reasoner",
},
Runtime: RuntimeConfig{
TokenRefreshIntervalHours: 12,
},
VercelSyncHash: "hash123",
VercelSyncTime: 1234567890,
AdditionalFields: map[string]any{
@@ -130,6 +133,9 @@ func TestConfigJSONRoundtrip(t *testing.T) {
if decoded.ClaudeMapping["fast"] != "deepseek-chat" {
t.Fatalf("unexpected claude mapping: %#v", decoded.ClaudeMapping)
}
if decoded.Runtime.TokenRefreshIntervalHours != 12 {
t.Fatalf("unexpected runtime refresh interval: %#v", decoded.Runtime.TokenRefreshIntervalHours)
}
if decoded.VercelSyncHash != "hash123" {
t.Fatalf("unexpected vercel sync hash: %q", decoded.VercelSyncHash)
}

View File

@@ -2,6 +2,7 @@ package config
import (
"encoding/base64"
"errors"
"os"
"strings"
"testing"
@@ -79,6 +80,136 @@ func TestLoadStorePreservesFileBackedTokensForRuntime(t *testing.T) {
}
}
func TestEnvBackedStoreWritebackBootstrapsMissingConfigFile(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {
t.Fatalf("create temp config: %v", err)
}
path := tmp.Name()
_ = tmp.Close()
_ = os.Remove(path)
t.Setenv("DS2API_CONFIG_JSON", `{"keys":["k1"],"accounts":[{"email":"seed@example.com","password":"p"}]}`)
t.Setenv("CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
t.Setenv("DS2API_ENV_WRITEBACK", "1")
store := LoadStore()
if store.IsEnvBacked() {
t.Fatalf("expected writeback bootstrap to become file-backed immediately")
}
if err := store.Update(func(c *Config) error {
c.Accounts = append(c.Accounts, Account{Email: "new@example.com", Password: "p2"})
return nil
}); err != nil {
t.Fatalf("update failed: %v", err)
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read written config: %v", err)
}
if !strings.Contains(string(content), "seed@example.com") {
t.Fatalf("expected bootstrapped config to contain seed account, got: %s", content)
}
if !strings.Contains(string(content), "new@example.com") {
t.Fatalf("expected persisted config to contain added account, got: %s", content)
}
reloaded := LoadStore()
if reloaded.IsEnvBacked() {
t.Fatalf("expected reloaded store to prefer persisted config file")
}
accounts := reloaded.Accounts()
if len(accounts) != 2 {
t.Fatalf("expected 2 accounts after reload, got %d", len(accounts))
}
}
func TestEnvBackedStoreWritebackDoesNotBootstrapOnInvalidEnvJSON(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {
t.Fatalf("create temp config: %v", err)
}
path := tmp.Name()
_ = tmp.Close()
_ = os.Remove(path)
t.Setenv("DS2API_CONFIG_JSON", "{invalid-json")
t.Setenv("CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
t.Setenv("DS2API_ENV_WRITEBACK", "1")
cfg, fromEnv, loadErr := loadConfig()
if loadErr == nil {
t.Fatalf("expected loadConfig error for invalid env json")
}
if !fromEnv {
t.Fatalf("expected fromEnv=true when parsing env config fails")
}
if len(cfg.Keys) != 0 || len(cfg.Accounts) != 0 {
t.Fatalf("expected empty config on parse failure, got keys=%d accounts=%d", len(cfg.Keys), len(cfg.Accounts))
}
if _, statErr := os.Stat(path); !errors.Is(statErr, os.ErrNotExist) {
t.Fatalf("expected no bootstrapped config file, stat err=%v", statErr)
}
}
func TestEnvBackedStoreWritebackFallsBackToPersistedFileOnInvalidEnvJSON(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {
t.Fatalf("create temp config: %v", err)
}
path := tmp.Name()
if _, err := tmp.WriteString(`{"keys":["file-key"],"accounts":[{"email":"persisted@example.com","password":"p"}]}`); err != nil {
t.Fatalf("write temp config: %v", err)
}
_ = tmp.Close()
t.Setenv("DS2API_CONFIG_JSON", "{invalid-json")
t.Setenv("CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
t.Setenv("DS2API_ENV_WRITEBACK", "1")
cfg, fromEnv, loadErr := loadConfig()
if loadErr != nil {
t.Fatalf("expected fallback to persisted file, got error: %v", loadErr)
}
if fromEnv {
t.Fatalf("expected fallback to file-backed mode")
}
if len(cfg.Keys) != 1 || cfg.Keys[0] != "file-key" {
t.Fatalf("unexpected keys after fallback: %#v", cfg.Keys)
}
if len(cfg.Accounts) != 1 || cfg.Accounts[0].Email != "persisted@example.com" {
t.Fatalf("unexpected accounts after fallback: %#v", cfg.Accounts)
}
}
func TestRuntimeTokenRefreshIntervalHoursDefaultsToSix(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[{"email":"u@example.com","password":"p"}]
}`)
store := LoadStore()
if got := store.RuntimeTokenRefreshIntervalHours(); got != 6 {
t.Fatalf("expected default refresh interval 6, got %d", got)
}
}
func TestRuntimeTokenRefreshIntervalHoursUsesConfigValue(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"],
"accounts":[{"email":"u@example.com","password":"p"}],
"runtime":{"token_refresh_interval_hours":9}
}`)
store := LoadStore()
if got := store.RuntimeTokenRefreshIntervalHours(); got != 9 {
t.Fatalf("expected configured refresh interval 9, got %d", got)
}
}
func TestStoreUpdateAccountTokenKeepsIdentifierResolvable(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{
"accounts":[{"email":"user@example.com","password":"p"}]

View File

@@ -40,12 +40,38 @@ func loadConfig() (Config, bool, error) {
}
if rawCfg != "" {
cfg, err := parseConfigString(rawCfg)
if err != nil {
if !IsVercel() && envWritebackEnabled() {
if fileCfg, fileErr := loadConfigFromFile(ConfigPath()); fileErr == nil {
return fileCfg, false, nil
}
}
return cfg, true, err
}
cfg.ClearAccountTokens()
cfg.DropInvalidAccounts()
if IsVercel() || !envWritebackEnabled() {
return cfg, true, err
}
content, fileErr := os.ReadFile(ConfigPath())
if fileErr == nil {
var fileCfg Config
if unmarshalErr := json.Unmarshal(content, &fileCfg); unmarshalErr == nil {
fileCfg.DropInvalidAccounts()
return fileCfg, false, err
}
}
if errors.Is(fileErr, os.ErrNotExist) {
if writeErr := writeConfigFile(ConfigPath(), cfg.Clone()); writeErr == nil {
return cfg, false, err
} else {
Logger.Warn("[config] env writeback bootstrap failed", "error", writeErr)
}
}
return cfg, true, err
}
content, err := os.ReadFile(ConfigPath())
cfg, err := loadConfigFromFile(ConfigPath())
if err != nil {
if IsVercel() {
// Vercel one-click deploy may start without a writable/present config file.
@@ -54,16 +80,6 @@ func loadConfig() (Config, bool, error) {
}
return Config{}, false, err
}
var cfg Config
if err := json.Unmarshal(content, &cfg); err != nil {
return Config{}, false, err
}
cfg.DropInvalidAccounts()
if strings.Contains(string(content), `"test_status"`) && !IsVercel() {
if b, err := json.MarshalIndent(cfg, "", " "); err == nil {
_ = os.WriteFile(ConfigPath(), b, 0o644)
}
}
if IsVercel() {
// Vercel filesystem is ephemeral/read-only for runtime writes; avoid save errors.
return cfg, true, nil
@@ -71,6 +87,24 @@ func loadConfig() (Config, bool, error) {
return cfg, false, nil
}
func loadConfigFromFile(path string) (Config, error) {
content, err := os.ReadFile(path)
if err != nil {
return Config{}, err
}
var cfg Config
if err := json.Unmarshal(content, &cfg); err != nil {
return Config{}, err
}
cfg.DropInvalidAccounts()
if strings.Contains(string(content), `"test_status"`) && !IsVercel() {
if b, err := json.MarshalIndent(cfg, "", " "); err == nil {
_ = os.WriteFile(path, b, 0o644)
}
}
return cfg, nil
}
func (s *Store) Snapshot() Config {
s.mu.RLock()
defer s.mu.RUnlock()
@@ -177,7 +211,7 @@ func (s *Store) Update(mutator func(*Config) error) error {
func (s *Store) Save() error {
s.mu.Lock()
defer s.mu.Unlock()
if s.fromEnv {
if s.fromEnv && (IsVercel() || !envWritebackEnabled()) {
Logger.Info("[save_config] source from env, skip write")
return nil
}
@@ -187,11 +221,15 @@ func (s *Store) Save() error {
if err != nil {
return err
}
return os.WriteFile(s.path, b, 0o644)
if err := writeConfigBytes(s.path, b); err != nil {
return err
}
s.fromEnv = false
return nil
}
func (s *Store) saveLocked() error {
if s.fromEnv {
if s.fromEnv && (IsVercel() || !envWritebackEnabled()) {
Logger.Info("[save_config] source from env, skip write")
return nil
}
@@ -201,7 +239,11 @@ func (s *Store) saveLocked() error {
if err != nil {
return err
}
return os.WriteFile(s.path, b, 0o644)
if err := writeConfigBytes(s.path, b); err != nil {
return err
}
s.fromEnv = false
return nil
}
func (s *Store) IsEnvBacked() bool {

View File

@@ -43,23 +43,11 @@ func (s *Store) CompatWideInputStrictOutput() bool {
}
func (s *Store) ToolcallMode() string {
s.mu.RLock()
defer s.mu.RUnlock()
mode := strings.TrimSpace(strings.ToLower(s.cfg.Toolcall.Mode))
if mode == "" {
return "feature_match"
}
return mode
return "feature_match"
}
func (s *Store) ToolcallEarlyEmitConfidence() string {
s.mu.RLock()
defer s.mu.RUnlock()
level := strings.TrimSpace(strings.ToLower(s.cfg.Toolcall.EarlyEmitConfidence))
if level == "" {
return "high"
}
return level
return "high"
}
func (s *Store) ResponsesStoreTTLSeconds() int {
@@ -166,6 +154,15 @@ func (s *Store) RuntimeGlobalMaxInflight(defaultSize int) int {
return defaultSize
}
func (s *Store) RuntimeTokenRefreshIntervalHours() int {
s.mu.RLock()
defer s.mu.RUnlock()
if s.cfg.Runtime.TokenRefreshIntervalHours > 0 {
return s.cfg.Runtime.TokenRefreshIntervalHours
}
return 6
}
func (s *Store) AutoDeleteSessions() bool {
s.mu.RLock()
defer s.mu.RUnlock()

View File

@@ -0,0 +1,51 @@
package config
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
)
func envWritebackEnabled() bool {
v := strings.ToLower(strings.TrimSpace(os.Getenv("DS2API_ENV_WRITEBACK")))
return v == "1" || v == "true" || v == "yes" || v == "on"
}
func (s *Store) IsEnvWritebackEnabled() bool {
return envWritebackEnabled()
}
func (s *Store) HasEnvConfigSource() bool {
rawCfg := strings.TrimSpace(os.Getenv("DS2API_CONFIG_JSON"))
if rawCfg == "" {
rawCfg = strings.TrimSpace(os.Getenv("CONFIG_JSON"))
}
return rawCfg != ""
}
func (s *Store) ConfigPath() string {
return s.path
}
func writeConfigFile(path string, cfg Config) error {
persistCfg := cfg.Clone()
persistCfg.ClearAccountTokens()
b, err := json.MarshalIndent(persistCfg, "", " ")
if err != nil {
return err
}
return writeConfigBytes(path, b)
}
func writeConfigBytes(path string, b []byte) error {
dir := filepath.Dir(path)
if dir == "." || dir == "" {
return os.WriteFile(path, b, 0o644)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
return fmt.Errorf("mkdir config dir: %w", err)
}
return os.WriteFile(path, b, 0o644)
}

View File

@@ -59,8 +59,9 @@ async function handler(req, res) {
return;
}
// Keep all non-stream behavior on Go side to avoid compatibility regressions.
if (!toBool(payload.stream)) {
// Keep all non-stream behavior and non-OpenAI-chat paths on Go side to avoid
// protocol-shape regressions (e.g. Gemini/Claude clients expecting their own formats).
if (!toBool(payload.stream) || !isNodeStreamSupportedPath(req.url || '')) {
await proxyToGo(req, res, rawBody);
return;
}
@@ -76,6 +77,23 @@ function isVercelRuntime() {
return asString(process.env.VERCEL) !== '' || asString(process.env.NOW_REGION) !== '';
}
function isNodeStreamSupportedPath(rawURL) {
const path = extractPathname(rawURL);
return path === '/v1/chat/completions';
}
function extractPathname(rawURL) {
const text = asString(rawURL);
if (!text) {
return '';
}
const q = text.indexOf('?');
if (q >= 0) {
return text.slice(0, q);
}
return text;
}
module.exports = handler;
module.exports.__test = {
@@ -89,4 +107,6 @@ module.exports.__test = {
boolDefaultTrue,
filterIncrementalToolCallDeltasByAllowed,
estimateTokens,
isNodeStreamSupportedPath,
extractPathname,
};

View File

@@ -193,6 +193,9 @@ function extractContentRecursive(items, defaultType) {
}
function shouldSkipPath(pathValue) {
if (isFragmentStatusPath(pathValue)) {
return true;
}
if (SKIP_EXACT_PATHS.has(pathValue)) {
return true;
}
@@ -204,6 +207,13 @@ function shouldSkipPath(pathValue) {
return false;
}
function isFragmentStatusPath(pathValue) {
if (!pathValue || pathValue === 'response/status') {
return false;
}
return /^response\/fragments\/-?\d+\/status$/i.test(pathValue);
}
function isCitation(text) {
return asString(text).trim().startsWith('[citation:');
}
@@ -225,5 +235,6 @@ module.exports = {
parseChunkForContent,
extractContentRecursive,
shouldSkipPath,
isFragmentStatusPath,
isCitation,
};

View File

@@ -12,12 +12,10 @@ function resolveToolcallPolicy(prepBody, payloadTools) {
if (toolNames.length === 0 && Array.isArray(payloadTools) && payloadTools.length > 0) {
toolNames = ['__any_tool__'];
}
const featureMatchEnabled = boolDefaultTrue(prepBody && prepBody.toolcall_feature_match);
const emitEarlyToolDeltas = featureMatchEnabled && boolDefaultTrue(prepBody && prepBody.toolcall_early_emit_high);
return {
toolNames,
toolSieveEnabled: toolNames.length > 0,
emitEarlyToolDeltas,
emitEarlyToolDeltas: true,
};
}

View File

@@ -140,30 +140,6 @@ function extractJSONObjectFrom(text, start) {
return { ok: false, end: 0 };
}
function extractToolHistoryBlock(captured, keyIdx) {
if (typeof captured !== 'string' || keyIdx < 0 || keyIdx >= captured.length) {
return { ok: false, start: 0, end: 0 };
}
const rest = captured.slice(keyIdx).toLowerCase();
if (rest.startsWith('[tool_call_history]')) {
const closeTag = '[/tool_call_history]';
const closeIdx = rest.indexOf(closeTag);
if (closeIdx < 0) {
return { ok: false, start: 0, end: 0 };
}
return { ok: true, start: keyIdx, end: keyIdx + closeIdx + closeTag.length };
}
if (rest.startsWith('[tool_result_history]')) {
const closeTag = '[/tool_result_history]';
const closeIdx = rest.indexOf(closeTag);
if (closeIdx < 0) {
return { ok: false, start: 0, end: 0 };
}
return { ok: true, start: keyIdx, end: keyIdx + closeIdx + closeTag.length };
}
return { ok: false, start: 0, end: 0 };
}
function trimWrappingJSONFence(prefix, suffix) {
const rightTrimmedPrefix = (prefix || '').replace(/[ \t\r\n]+$/g, '');
const fenceIdx = rightTrimmedPrefix.lastIndexOf('```');
@@ -192,6 +168,5 @@ module.exports = {
parseJSONStringLiteral,
skipSpaces,
extractJSONObjectFrom,
extractToolHistoryBlock,
trimWrappingJSONFence,
};

View File

@@ -237,7 +237,10 @@ function isLikelyJSONToolPayloadCandidate(text) {
return false;
}
const lower = trimmed.toLowerCase();
return lower.includes('tool_calls') || lower.includes('"function"');
return lower.includes('tool_calls')
|| lower.includes('"function"')
|| lower.includes('functioncall')
|| lower.includes('"tool_use"');
}
module.exports = {

View File

@@ -85,6 +85,8 @@ function extractToolCallObjects(text) {
while (true) {
const idxToolCalls = lower.indexOf('tool_calls', offset);
const idxFunction = lower.indexOf('"function"', offset);
const idxFunctionCall = lower.indexOf('functioncall', offset);
const idxToolUse = lower.indexOf('"tool_use"', offset);
let idx = -1;
let matched = '';
if (idxToolCalls >= 0 && (idxFunction < 0 || idxToolCalls <= idxFunction)) {
@@ -94,6 +96,14 @@ function extractToolCallObjects(text) {
idx = idxFunction;
matched = '"function"';
}
if (idxFunctionCall >= 0 && (idx < 0 || idxFunctionCall < idx)) {
idx = idxFunctionCall;
matched = 'functioncall';
}
if (idxToolUse >= 0 && (idx < 0 || idxToolUse < idx)) {
idx = idxToolUse;
matched = '"tool_use"';
}
if (idx < 0) {
break;
}
@@ -102,7 +112,10 @@ function extractToolCallObjects(text) {
const obj = extractJSONObjectFrom(raw, start);
if (obj.ok) {
out.push(raw.slice(start, obj.end).trim());
offset = obj.end;
// Ensure forward progress even when the matched keyword is outside
// the extracted JSON object (e.g. closing XML wrapper tags containing
// "tool_calls" after an earlier JSON arguments object).
offset = Math.max(obj.end, idx + matched.length);
idx = -1;
break;
}
@@ -324,6 +337,20 @@ function parseToolCallItem(m) {
let name = toStringSafe(m.name);
let inputRaw = m.input;
let hasInput = Object.prototype.hasOwnProperty.call(m, 'input');
const fnCall = m.functionCall && typeof m.functionCall === 'object' ? m.functionCall : null;
if (fnCall) {
if (!name) {
name = toStringSafe(fnCall.name);
}
if (!hasInput && Object.prototype.hasOwnProperty.call(fnCall, 'args')) {
inputRaw = fnCall.args;
hasInput = true;
}
if (!hasInput && Object.prototype.hasOwnProperty.call(fnCall, 'arguments')) {
inputRaw = fnCall.arguments;
hasInput = true;
}
}
const fn = m.function && typeof m.function === 'object' ? m.function : null;
if (fn) {

View File

@@ -5,7 +5,7 @@ const {
insideCodeFenceWithState,
} = require('./state');
const { parseStandaloneToolCallsDetailed } = require('./parse');
const { extractJSONObjectFrom, extractToolHistoryBlock, trimWrappingJSONFence } = require('./jsonscan');
const { extractJSONObjectFrom, trimWrappingJSONFence } = require('./jsonscan');
const {
TOOL_SEGMENT_KEYWORDS,
XML_TOOL_SEGMENT_TAGS,
@@ -233,17 +233,6 @@ function consumeToolCapture(state, toolNames) {
}
const start = captured.slice(0, keyIdx).lastIndexOf('{');
const actualStart = start >= 0 ? start : keyIdx;
if (start < 0) {
const history = extractToolHistoryBlock(captured, keyIdx);
if (history.ok) {
return {
ready: true,
prefix: captured.slice(0, history.start),
calls: [],
suffix: captured.slice(history.end),
};
}
}
const obj = extractJSONObjectFrom(captured, actualStart);
if (!obj.ok) {
return { ready: false, prefix: '', calls: [], suffix: '' };

View File

@@ -1,6 +1,7 @@
'use strict';
const TOOL_SIEVE_CONTEXT_TAIL_LIMIT = 4096;
// Keep in sync with Go toolSieveContextTailLimit.
const TOOL_SIEVE_CONTEXT_TAIL_LIMIT = 2048;
function createToolSieveState() {
return {

View File

@@ -4,8 +4,8 @@ const TOOL_SEGMENT_KEYWORDS = [
'tool_calls',
'"function"',
'function.name:',
'[tool_call_history]',
'[tool_result_history]',
'functioncall',
'"tool_use"',
];
const XML_TOOL_SEGMENT_TAGS = [

View File

@@ -0,0 +1,137 @@
package prompt
import (
"encoding/json"
"strings"
)
var promptXMLTextEscaper = strings.NewReplacer(
"&", "&amp;",
"<", "&lt;",
">", "&gt;",
)
// FormatToolCallsForPrompt renders a tool_calls slice into the canonical
// prompt-visible history block used across adapters.
func FormatToolCallsForPrompt(raw any) string {
calls, ok := raw.([]any)
if !ok || len(calls) == 0 {
return ""
}
blocks := make([]string, 0, len(calls))
for _, item := range calls {
call, ok := item.(map[string]any)
if !ok {
continue
}
block := formatToolCallForPrompt(call)
if block != "" {
blocks = append(blocks, block)
}
}
if len(blocks) == 0 {
return ""
}
return "<tool_calls>\n" + strings.Join(blocks, "\n") + "\n</tool_calls>"
}
// StringifyToolCallArguments normalizes tool arguments into a compact string
// while preserving raw concatenated payloads when they already look like model
// output rather than a single JSON object.
func StringifyToolCallArguments(v any) string {
switch x := v.(type) {
case nil:
return "{}"
case string:
s := strings.TrimSpace(x)
if s == "" {
return "{}"
}
s = normalizeToolArgumentString(s)
if s == "" {
return "{}"
}
return s
default:
b, err := json.Marshal(x)
if err != nil || len(b) == 0 {
return "{}"
}
return string(b)
}
}
func formatToolCallForPrompt(call map[string]any) string {
if call == nil {
return ""
}
name := strings.TrimSpace(asString(call["name"]))
fn, _ := call["function"].(map[string]any)
if name == "" && fn != nil {
name = strings.TrimSpace(asString(fn["name"]))
}
if name == "" {
return ""
}
argsRaw := call["arguments"]
if argsRaw == nil {
argsRaw = call["input"]
}
if argsRaw == nil && fn != nil {
argsRaw = fn["arguments"]
if argsRaw == nil {
argsRaw = fn["input"]
}
}
return " <tool_call>\n" +
" <tool_name>" + escapeXMLText(name) + "</tool_name>\n" +
" <parameters>" + escapeXMLText(StringifyToolCallArguments(argsRaw)) + "</parameters>\n" +
" </tool_call>"
}
func normalizeToolArgumentString(raw string) string {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return ""
}
if looksLikeConcatenatedJSON(trimmed) {
// Keep the original payload to avoid silently rewriting model output.
return raw
}
return trimmed
}
func looksLikeConcatenatedJSON(raw string) bool {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return false
}
if strings.Contains(trimmed, "}{") || strings.Contains(trimmed, "][") {
return true
}
dec := json.NewDecoder(strings.NewReader(trimmed))
var first any
if err := dec.Decode(&first); err != nil {
return false
}
var second any
return dec.Decode(&second) == nil
}
func asString(v any) string {
if s, ok := v.(string); ok {
return s
}
return ""
}
func escapeXMLText(v string) string {
if v == "" {
return ""
}
return promptXMLTextEscaper.Replace(v)
}

View File

@@ -0,0 +1,41 @@
package prompt
import "testing"
func TestStringifyToolCallArgumentsPreservesConcatenatedJSON(t *testing.T) {
got := StringifyToolCallArguments(`{}{"query":"测试工具调用"}`)
if got != `{}{"query":"测试工具调用"}` {
t.Fatalf("expected raw concatenated JSON to be preserved, got %q", got)
}
}
func TestFormatToolCallsForPromptXML(t *testing.T) {
got := FormatToolCallsForPrompt([]any{
map[string]any{
"id": "call_1",
"function": map[string]any{
"name": "search_web",
"arguments": map[string]any{"query": "latest"},
},
},
})
if got == "" {
t.Fatal("expected non-empty formatted tool calls")
}
if got != "<tool_calls>\n <tool_call>\n <tool_name>search_web</tool_name>\n <parameters>{\"query\":\"latest\"}</parameters>\n </tool_call>\n</tool_calls>" {
t.Fatalf("unexpected formatted tool call XML: %q", got)
}
}
func TestFormatToolCallsForPromptEscapesXMLEntities(t *testing.T) {
got := FormatToolCallsForPrompt([]any{
map[string]any{
"name": "search<&>",
"arguments": `{"q":"a < b && c > d"}`,
},
})
want := "<tool_calls>\n <tool_call>\n <tool_name>search&lt;&amp;&gt;</tool_name>\n <parameters>{\"q\":\"a &lt; b &amp;&amp; c &gt; d\"}</parameters>\n </tool_call>\n</tool_calls>"
if got != want {
t.Fatalf("unexpected escaped tool call XML: %q", got)
}
}

View File

@@ -44,8 +44,8 @@ func NewApp() *App {
}
openaiHandler := &openai.Handler{Store: store, Auth: resolver, DS: dsClient}
claudeHandler := &claude.Handler{Store: store, Auth: resolver, DS: dsClient}
geminiHandler := &gemini.Handler{Store: store, Auth: resolver, DS: dsClient}
claudeHandler := &claude.Handler{Store: store, Auth: resolver, DS: dsClient, OpenAI: openaiHandler}
geminiHandler := &gemini.Handler{Store: store, Auth: resolver, DS: dsClient, OpenAI: openaiHandler}
adminHandler := &admin.Handler{Store: store, Pool: pool, DS: dsClient}
webuiHandler := webui.NewHandler()

View File

@@ -10,8 +10,9 @@ import (
// CollectResult holds the aggregated text and thinking content from a
// DeepSeek SSE stream, consumed to completion (non-streaming use case).
type CollectResult struct {
Text string
Thinking string
Text string
Thinking string
OutputTokens int
}
// CollectStream fully consumes a DeepSeek SSE response and separates
@@ -26,6 +27,7 @@ func CollectStream(resp *http.Response, thinkingEnabled bool, closeBody bool) Co
}
text := strings.Builder{}
thinking := strings.Builder{}
outputTokens := 0
currentType := "text"
if thinkingEnabled {
currentType = "thinking"
@@ -37,8 +39,14 @@ func CollectStream(resp *http.Response, thinkingEnabled bool, closeBody bool) Co
return true
}
if result.Stop {
if result.OutputTokens > 0 {
outputTokens = result.OutputTokens
}
return false
}
if result.OutputTokens > 0 {
outputTokens = result.OutputTokens
}
for _, p := range result.Parts {
if p.Type == "thinking" {
thinking.WriteString(p.Text)
@@ -48,5 +56,5 @@ func CollectStream(resp *http.Response, thinkingEnabled bool, closeBody bool) Co
}
return true
})
return CollectResult{Text: text.String(), Thinking: thinking.String()}
return CollectResult{Text: text.String(), Thinking: thinking.String(), OutputTokens: outputTokens}
}

View File

@@ -138,3 +138,15 @@ func TestCollectStreamStatusFinished(t *testing.T) {
t.Fatalf("expected 'Hello', got %q", result.Text)
}
}
func TestCollectStreamStopsOnContentFilterStatus(t *testing.T) {
resp := makeHTTPResponse(
"data: {\"p\":\"response/content\",\"v\":\"safe\"}\n" +
"data: {\"p\":\"response/status\",\"v\":\"CONTENT_FILTER\"}\n" +
"data: {\"p\":\"response/content\",\"v\":\"blocked\"}\n",
)
result := CollectStream(resp, false, false)
if result.Text != "safe" {
t.Fatalf("expected stream to stop before blocked tail, got %q", result.Text)
}
}

View File

@@ -0,0 +1,31 @@
package sse
import "strings"
func filterLeakedContentFilterParts(parts []ContentPart) []ContentPart {
if len(parts) == 0 {
return parts
}
out := make([]ContentPart, 0, len(parts))
for _, p := range parts {
cleaned := stripLeakedContentFilterSuffix(p.Text)
if strings.TrimSpace(cleaned) == "" {
continue
}
p.Text = cleaned
out = append(out, p)
}
return out
}
func stripLeakedContentFilterSuffix(text string) string {
if text == "" {
return text
}
upperText := strings.ToUpper(text)
idx := strings.Index(upperText, "CONTENT_FILTER")
if idx < 0 {
return text
}
return strings.TrimRight(text[:idx], " \t\r\n")
}

View File

@@ -10,6 +10,7 @@ type LineResult struct {
ErrorMessage string
Parts []ContentPart
NextType string
OutputTokens int
}
// ParseDeepSeekContentLine centralizes one-line DeepSeek SSE parsing for both
@@ -35,15 +36,26 @@ func ParseDeepSeekContentLine(raw []byte, thinkingEnabled bool, currentType stri
Parsed: true,
Stop: true,
ContentFilter: true,
ErrorMessage: "content filtered by upstream",
NextType: currentType,
OutputTokens: extractAccumulatedTokenUsage(chunk),
}
}
if hasContentFilterStatus(chunk) {
return LineResult{
Parsed: true,
Stop: true,
ContentFilter: true,
NextType: currentType,
OutputTokens: extractAccumulatedTokenUsage(chunk),
}
}
parts, finished, nextType := ParseSSEChunkForContent(chunk, thinkingEnabled, currentType)
parts = filterLeakedContentFilterParts(parts)
return LineResult{
Parsed: true,
Stop: finished,
Parts: parts,
NextType: nextType,
OutputTokens: extractAccumulatedTokenUsage(chunk),
}
}

View File

@@ -40,8 +40,8 @@ func TestParseDeepSeekContentLineContentFilterMessage(t *testing.T) {
if !res.ContentFilter {
t.Fatal("expected content filter flag")
}
if res.ErrorMessage == "" {
t.Fatal("expected error message on content filter")
if res.ErrorMessage != "" {
t.Fatalf("expected empty error message on content filter, got %q", res.ErrorMessage)
}
}

View File

@@ -26,6 +26,33 @@ func TestParseDeepSeekContentLineContentFilter(t *testing.T) {
}
}
func TestParseDeepSeekContentLineContentFilterCodeIncludesOutputTokens(t *testing.T) {
res := ParseDeepSeekContentLine(
[]byte(`data: {"code":"content_filter","accumulated_token_usage":99}`),
false, "text",
)
if !res.Parsed || !res.Stop || !res.ContentFilter {
t.Fatalf("expected content-filter stop result: %#v", res)
}
if res.OutputTokens != 99 {
t.Fatalf("expected output token usage 99, got %d", res.OutputTokens)
}
}
func TestParseDeepSeekContentLineContentFilterStatus(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/status","v":"CONTENT_FILTER"}`), false, "text")
if !res.Parsed || !res.Stop || !res.ContentFilter {
t.Fatalf("expected status-based content-filter stop result: %#v", res)
}
}
func TestParseDeepSeekContentLineCapturesAccumulatedTokenUsage(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":1383},{"p":"quasi_status","v":"FINISHED"}]}`), false, "text")
if res.OutputTokens != 1383 {
t.Fatalf("expected output token usage 1383, got %d", res.OutputTokens)
}
}
func TestParseDeepSeekContentLineContent(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"hi"}`), false, "text")
if !res.Parsed || res.Stop {
@@ -35,3 +62,43 @@ func TestParseDeepSeekContentLineContent(t *testing.T) {
t.Fatalf("unexpected parts: %#v", res.Parts)
}
}
func TestParseDeepSeekContentLineStripsLeakedContentFilterSuffix(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"正常输出CONTENT_FILTER你好这个问题我暂时无法回答"}`), false, "text")
if !res.Parsed || res.Stop {
t.Fatalf("expected parsed non-stop result: %#v", res)
}
if len(res.Parts) != 1 || res.Parts[0].Text != "正常输出" {
t.Fatalf("unexpected parts after filter: %#v", res.Parts)
}
}
func TestParseDeepSeekContentLineDropsPureLeakedContentFilterChunk(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"CONTENT_FILTER你好这个问题我暂时无法回答"}`), false, "text")
if !res.Parsed || res.Stop {
t.Fatalf("expected parsed non-stop result: %#v", res)
}
if len(res.Parts) != 0 {
t.Fatalf("expected empty parts, got %#v", res.Parts)
}
}
func TestParseDeepSeekContentLineTrimsFromContentFilterKeyword(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"模型会在命中 CONTENT_FILTER 时返回拒绝原因。"}`), false, "text")
if !res.Parsed || res.Stop {
t.Fatalf("expected parsed non-stop result: %#v", res)
}
if len(res.Parts) != 1 || res.Parts[0].Text != "模型会在命中" {
t.Fatalf("unexpected parts after filter: %#v", res.Parts)
}
}
func TestParseDeepSeekContentLineContentTextEqualContentFilterDoesNotStop(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"content_filter"}`), false, "text")
if !res.Parsed {
t.Fatalf("expected parsed result: %#v", res)
}
if res.Stop || res.ContentFilter {
t.Fatalf("did not expect content-filter stop for content text: %#v", res)
}
}

View File

@@ -3,6 +3,7 @@ package sse
import (
"bytes"
"encoding/json"
"math"
"strings"
"ds2api/internal/deepseek"
@@ -30,6 +31,9 @@ func ParseDeepSeekSSELine(raw []byte) (map[string]any, bool, bool) {
}
func shouldSkipPath(path string) bool {
if isFragmentStatusPath(path) {
return true
}
if _, ok := deepseek.SkipExactPathSet[path]; ok {
return true
}
@@ -41,6 +45,31 @@ func shouldSkipPath(path string) bool {
return false
}
func isFragmentStatusPath(path string) bool {
if path == "" || path == "response/status" {
return false
}
if !strings.HasPrefix(path, "response/fragments/") || !strings.HasSuffix(path, "/status") {
return false
}
mid := strings.TrimSuffix(strings.TrimPrefix(path, "response/fragments/"), "/status")
if mid == "" {
return false
}
if strings.HasPrefix(mid, "-") {
mid = mid[1:]
}
if mid == "" {
return false
}
for _, r := range mid {
if r < '0' || r > '9' {
return false
}
}
return true
}
func ParseSSEChunkForContent(chunk map[string]any, thinkingEnabled bool, currentFragmentType string) ([]ContentPart, bool, string) {
v, ok := chunk["v"]
if !ok {
@@ -287,3 +316,90 @@ func extractContentRecursive(items []any, defaultType string) ([]ContentPart, bo
func IsCitation(text string) bool {
return bytes.HasPrefix([]byte(strings.TrimSpace(text)), []byte("[citation:"))
}
func hasContentFilterStatus(chunk map[string]any) bool {
if code, _ := chunk["code"].(string); strings.EqualFold(strings.TrimSpace(code), "content_filter") {
return true
}
return hasContentFilterStatusValue(chunk)
}
func hasContentFilterStatusValue(v any) bool {
switch x := v.(type) {
case []any:
for _, item := range x {
if hasContentFilterStatusValue(item) {
return true
}
}
case map[string]any:
if p, _ := x["p"].(string); strings.Contains(strings.ToLower(p), "status") {
if s, _ := x["v"].(string); strings.EqualFold(strings.TrimSpace(s), "content_filter") {
return true
}
}
if code, _ := x["code"].(string); strings.EqualFold(strings.TrimSpace(code), "content_filter") {
return true
}
for _, vv := range x {
if hasContentFilterStatusValue(vv) {
return true
}
}
}
return false
}
func extractAccumulatedTokenUsage(chunk map[string]any) int {
return findAccumulatedTokenUsage(chunk)
}
func findAccumulatedTokenUsage(v any) int {
switch x := v.(type) {
case map[string]any:
if p, _ := x["p"].(string); strings.Contains(strings.ToLower(p), "accumulated_token_usage") {
if n, ok := toInt(x["v"]); ok && n > 0 {
return n
}
}
if n, ok := toInt(x["accumulated_token_usage"]); ok && n > 0 {
return n
}
for _, vv := range x {
if n := findAccumulatedTokenUsage(vv); n > 0 {
return n
}
}
case []any:
for _, item := range x {
if n := findAccumulatedTokenUsage(item); n > 0 {
return n
}
}
}
return 0
}
func toInt(v any) (int, bool) {
switch x := v.(type) {
case int:
return x, true
case int32:
return int(x), true
case int64:
return int(x), true
case float64:
if math.IsNaN(x) || math.IsInf(x, 0) {
return 0, false
}
return int(x), true
case json.Number:
i, err := x.Int64()
if err != nil {
return 0, false
}
return int(i), true
default:
return 0, false
}
}

View File

@@ -90,6 +90,15 @@ func TestShouldSkipPathFragmentStatus(t *testing.T) {
if !shouldSkipPath("response/fragments/-3/status") {
t.Fatal("expected skip for fragment -3 status")
}
if !shouldSkipPath("response/fragments/-16/status") {
t.Fatal("expected skip for fragment -16 status")
}
if !shouldSkipPath("response/fragments/7/status") {
t.Fatal("expected skip for fragment 7 status")
}
if shouldSkipPath("response/status") {
t.Fatal("expected response/status to be handled by finish logic, not skipped")
}
}
func TestShouldSkipPathRegularContent(t *testing.T) {

View File

@@ -0,0 +1,67 @@
package translatorcliproxy
import (
"bytes"
"context"
"strings"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
_ "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator/builtin"
)
func ToOpenAI(from sdktranslator.Format, model string, raw []byte, stream bool) []byte {
return sdktranslator.TranslateRequest(from, sdktranslator.FormatOpenAI, model, raw, stream)
}
func FromOpenAINonStream(to sdktranslator.Format, model string, originalReq, translatedReq, raw []byte) []byte {
var param any
return sdktranslator.TranslateNonStream(context.Background(), sdktranslator.FormatOpenAI, to, model, originalReq, translatedReq, raw, &param)
}
func FromOpenAIStream(to sdktranslator.Format, model string, originalReq, translatedReq, streamBody []byte) []byte {
var out bytes.Buffer
var param any
for _, line := range bytes.Split(streamBody, []byte("\n")) {
trimmed := strings.TrimSpace(string(line))
if trimmed == "" {
continue
}
payload := append([]byte(nil), line...)
if !bytes.HasPrefix(payload, []byte("data:")) {
continue
}
chunks := sdktranslator.TranslateStream(context.Background(), sdktranslator.FormatOpenAI, to, model, originalReq, translatedReq, payload, &param)
for i := range chunks {
out.Write(chunks[i])
if !bytes.HasSuffix(chunks[i], []byte("\n")) {
out.WriteByte('\n')
}
}
}
return out.Bytes()
}
func ParseFormat(name string) sdktranslator.Format {
switch strings.ToLower(strings.TrimSpace(name)) {
case "openai", "openai-chat", "chat", "chat-completions":
return sdktranslator.FormatOpenAI
case "openai-response", "responses", "openai-responses":
return sdktranslator.FormatOpenAIResponse
case "claude", "anthropic":
return sdktranslator.FormatClaude
case "gemini", "google":
return sdktranslator.FormatGemini
case "gemini-cli", "geminicli":
return sdktranslator.FormatGeminiCLI
case "codex", "openai-codex":
return sdktranslator.FormatCodex
case "antigravity":
return sdktranslator.FormatAntigravity
default:
return sdktranslator.FromString(name)
}
}
func ToOpenAIByName(formatName, model string, raw []byte, stream bool) []byte {
return ToOpenAI(ParseFormat(formatName), model, raw, stream)
}

View File

@@ -0,0 +1,72 @@
package translatorcliproxy
import (
"strings"
"testing"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
)
func TestToOpenAIClaude(t *testing.T) {
raw := []byte(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":false}`)
got := ToOpenAI(sdktranslator.FormatClaude, "claude-sonnet-4-5", raw, false)
s := string(got)
if !strings.Contains(s, `"messages"`) || !strings.Contains(s, `"model"`) {
t.Fatalf("unexpected translated request: %s", s)
}
}
func TestFromOpenAINonStreamClaude(t *testing.T) {
original := []byte(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":false}`)
translatedReq := []byte(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":false}`)
openaibody := []byte(`{"id":"chatcmpl_1","object":"chat.completion","created":1,"model":"claude-sonnet-4-5","choices":[{"index":0,"message":{"role":"assistant","content":"hello"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}`)
got := FromOpenAINonStream(sdktranslator.FormatClaude, "claude-sonnet-4-5", original, translatedReq, openaibody)
if !strings.Contains(string(got), `"type":"message"`) {
t.Fatalf("expected claude response format, got: %s", string(got))
}
}
func TestParseFormatAliases(t *testing.T) {
cases := map[string]sdktranslator.Format{
"responses": sdktranslator.FormatOpenAIResponse,
"anthropic": sdktranslator.FormatClaude,
"geminicli": sdktranslator.FormatGeminiCLI,
"openai-codex": sdktranslator.FormatCodex,
"antigravity": sdktranslator.FormatAntigravity,
"chat-completions": sdktranslator.FormatOpenAI,
}
for in, want := range cases {
if got := ParseFormat(in); got != want {
t.Fatalf("ParseFormat(%q)=%q want %q", in, got, want)
}
}
}
func TestToOpenAIByNameAllSupportedFormats(t *testing.T) {
tests := []struct {
name string
format string
model string
body string
}{
{name: "openai", format: "openai", model: "gpt-4.1", body: `{"model":"gpt-4.1","messages":[{"role":"user","content":"hi"}],"stream":false}`},
{name: "responses", format: "responses", model: "gpt-4.1", body: `{"model":"gpt-4.1","input":"hello","stream":false}`},
{name: "claude", format: "claude", model: "claude-sonnet-4-5", body: `{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hello"}],"stream":false}`},
{name: "gemini", format: "gemini", model: "gemini-2.5-pro", body: `{"contents":[{"role":"user","parts":[{"text":"hello"}]}]}`},
{name: "gemini-cli", format: "gemini-cli", model: "gemini-2.5-pro", body: `{"model":"gemini-2.5-pro","messages":[{"role":"user","content":"hello"}],"stream":false}`},
{name: "codex", format: "codex", model: "gpt-5-codex", body: `{"model":"gpt-5-codex","messages":[{"role":"user","content":"hello"}],"stream":false}`},
{name: "antigravity", format: "antigravity", model: "gpt-4.1", body: `{"model":"gpt-4.1","messages":[{"role":"user","content":"hello"}],"stream":false}`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := ToOpenAIByName(tc.format, tc.model, []byte(tc.body), false)
if len(got) == 0 {
t.Fatalf("expected non-empty conversion result for format=%s", tc.format)
}
if !strings.Contains(string(got), `"model"`) {
t.Fatalf("expected model field in converted payload, got=%s", string(got))
}
})
}
}

View File

@@ -0,0 +1,120 @@
package translatorcliproxy
import (
"bytes"
"context"
"net/http"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
)
// OpenAIStreamTranslatorWriter translates OpenAI SSE output to another client format in real-time.
type OpenAIStreamTranslatorWriter struct {
dst http.ResponseWriter
target sdktranslator.Format
model string
originalReq []byte
translatedReq []byte
param any
statusCode int
headersSent bool
lineBuf bytes.Buffer
}
func NewOpenAIStreamTranslatorWriter(dst http.ResponseWriter, target sdktranslator.Format, model string, originalReq, translatedReq []byte) *OpenAIStreamTranslatorWriter {
return &OpenAIStreamTranslatorWriter{
dst: dst,
target: target,
model: model,
originalReq: originalReq,
translatedReq: translatedReq,
statusCode: http.StatusOK,
}
}
func (w *OpenAIStreamTranslatorWriter) Header() http.Header {
return w.dst.Header()
}
func (w *OpenAIStreamTranslatorWriter) WriteHeader(statusCode int) {
if w.headersSent {
return
}
w.statusCode = statusCode
w.headersSent = true
w.dst.WriteHeader(statusCode)
}
func (w *OpenAIStreamTranslatorWriter) Write(p []byte) (int, error) {
if !w.headersSent {
w.WriteHeader(http.StatusOK)
}
if w.statusCode < 200 || w.statusCode >= 300 {
return w.dst.Write(p)
}
w.lineBuf.Write(p)
for {
line, ok := w.readOneLine()
if !ok {
break
}
trimmed := bytes.TrimSpace(line)
if len(trimmed) == 0 {
continue
}
if bytes.HasPrefix(trimmed, []byte(":")) {
if _, err := w.dst.Write(trimmed); err != nil {
return len(p), err
}
if _, err := w.dst.Write([]byte("\n\n")); err != nil {
return len(p), err
}
if f, ok := w.dst.(http.Flusher); ok {
f.Flush()
}
continue
}
if !bytes.HasPrefix(trimmed, []byte("data:")) {
continue
}
chunks := sdktranslator.TranslateStream(context.Background(), sdktranslator.FormatOpenAI, w.target, w.model, w.originalReq, w.translatedReq, trimmed, &w.param)
for i := range chunks {
if len(chunks[i]) == 0 {
continue
}
if _, err := w.dst.Write(chunks[i]); err != nil {
return len(p), err
}
if !bytes.HasSuffix(chunks[i], []byte("\n")) {
if _, err := w.dst.Write([]byte("\n")); err != nil {
return len(p), err
}
}
}
if f, ok := w.dst.(http.Flusher); ok {
f.Flush()
}
}
return len(p), nil
}
func (w *OpenAIStreamTranslatorWriter) Flush() {
if f, ok := w.dst.(http.Flusher); ok {
f.Flush()
}
}
func (w *OpenAIStreamTranslatorWriter) Unwrap() http.ResponseWriter {
return w.dst
}
func (w *OpenAIStreamTranslatorWriter) readOneLine() ([]byte, bool) {
b := w.lineBuf.Bytes()
idx := bytes.IndexByte(b, '\n')
if idx < 0 {
return nil, false
}
line := append([]byte(nil), b[:idx]...)
w.lineBuf.Next(idx + 1)
return line, true
}

View File

@@ -0,0 +1,57 @@
package translatorcliproxy
import (
"net/http/httptest"
"strings"
"testing"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
)
func TestOpenAIStreamTranslatorWriterClaude(t *testing.T) {
original := []byte(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":true}`)
translated := []byte(`{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"hi"}],"stream":true}`)
rec := httptest.NewRecorder()
w := NewOpenAIStreamTranslatorWriter(rec, sdktranslator.FormatClaude, "claude-sonnet-4-5", original, translated)
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(200)
_, _ = w.Write([]byte("data: {\"id\":\"chatcmpl_1\",\"object\":\"chat.completion.chunk\",\"created\":1,\"model\":\"claude-sonnet-4-5\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\"},\"finish_reason\":null}]}\n\n"))
_, _ = w.Write([]byte("data: {\"id\":\"chatcmpl_1\",\"object\":\"chat.completion.chunk\",\"created\":1,\"model\":\"claude-sonnet-4-5\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"},\"finish_reason\":null}]}\n\n"))
_, _ = w.Write([]byte("data: [DONE]\n\n"))
body := rec.Body.String()
if !strings.Contains(body, "event: message_start") {
t.Fatalf("expected claude message_start event, got: %s", body)
}
}
func TestOpenAIStreamTranslatorWriterGemini(t *testing.T) {
original := []byte(`{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}`)
translated := []byte(`{"model":"gemini-2.5-pro","messages":[{"role":"user","content":"hi"}],"stream":true}`)
rec := httptest.NewRecorder()
w := NewOpenAIStreamTranslatorWriter(rec, sdktranslator.FormatGemini, "gemini-2.5-pro", original, translated)
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(200)
_, _ = w.Write([]byte("data: {\"id\":\"chatcmpl_1\",\"object\":\"chat.completion.chunk\",\"created\":1,\"model\":\"gemini-2.5-pro\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"hi\"},\"finish_reason\":null}]}\n\n"))
_, _ = w.Write([]byte("data: [DONE]\n\n"))
body := rec.Body.String()
if !strings.Contains(body, "candidates") {
t.Fatalf("expected gemini stream payload, got: %s", body)
}
}
func TestOpenAIStreamTranslatorWriterPreservesKeepAliveComment(t *testing.T) {
rec := httptest.NewRecorder()
w := NewOpenAIStreamTranslatorWriter(rec, sdktranslator.FormatGemini, "gemini-2.5-pro", []byte(`{}`), []byte(`{}`))
w.Header().Set("Content-Type", "text/event-stream")
w.WriteHeader(200)
_, _ = w.Write([]byte(": keep-alive\n\n"))
body := rec.Body.String()
if !strings.Contains(body, ": keep-alive\n\n") {
t.Fatalf("expected keep-alive comment passthrough, got %q", body)
}
}

Some files were not shown because too many files have changed in this diff Show More