Compare commits

..

81 Commits

Author SHA1 Message Date
CJACK
a901250de7 refactor: replace bufio.Scanner with bufio.Reader for SSE stream parsing and track emitted text to prevent redundant output blocks 2026-05-02 23:50:35 +08:00
CJACK
dc5bffdf89 refactor: centralize assistant turn semantics and stream accumulation into new assistantturn and completionruntime packages 2026-05-02 23:28:43 +08:00
CJACK
eccd8c957b fix: prevent continuation replay overlap by trimming redundant text from thinking and response streams 2026-05-02 21:34:36 +08:00
CJACK.
b1d0ee07c0 Merge pull request #406 from wyv202011y/perf/streaming-speed-optimization
perf(streaming): optimize TTFT and reduce buffering latency
2026-05-02 21:19:04 +08:00
CJACK
0156f6b45b Merge origin/dev into PR 406 2026-05-02 21:17:02 +08:00
CJACK
a9f46f5b25 chore: bump version to 4.3.1 2026-05-02 21:04:12 +08:00
CJACK
e7d6807c7c feat: emit empty completion chunk along with keep-alive heartbeat in chat stream 2026-05-02 20:54:10 +08:00
d407ccb773 perf(streaming): optimize TTFT and reduce buffering latency
Core changes:
- stream.go: New accumulation buffer architecture with scanner goroutine
  + select loop, MinChars=16, MaxWait=10ms, first-flush-immediate
- dedupe.go: Add TrimContinuationOverlapFromBuilder to avoid string copies
- claude/stream_runtime_core.go: Integrate toolstream for incremental text
- claude/stream_runtime_finalize.go: toolstream flush support
- stream_emitter.js: Reduce DeltaCoalescer thresholds (160->16 chars, 80->20ms)
- empty_retry: Add thinking-aware empty output detection
- Fix reasoning_content leak and finish_reason=null in edge cases
- Fix tail content truncation when max_tokens exceeded

Tests: sync test expectations with upstream for thinking content
2026-05-02 20:28:30 +08:00
CJACK
c8f7b6b371 refactor streaming accumulation and chat history UI 2026-05-02 20:15:38 +08:00
CJACK.
20d71f528a Merge pull request #404 from NgoQuocViet2001/ai/openai-file-retrieve
feat(openai): retrieve uploaded file metadata
2026-05-02 15:42:40 +08:00
NgoQuocViet2001
36d0239dc6 feat(openai): retrieve uploaded file metadata 2026-05-02 14:33:42 +07:00
CJACK.
e620752e2b Merge pull request #403 from VanceHud/main
修复了使用Zeabur部署会失败的问题
2026-05-02 14:10:18 +08:00
VanceHud
44cb27872c Merge branch 'CJackHwang:main' into main 2026-05-02 12:19:09 +08:00
CJACK
3d52040b3b merge: sync release smoke fix from dev 2026-05-02 04:23:02 +08:00
CJACK
049e40e5f1 fix: drop obsolete release smoke check 2026-05-02 04:19:23 +08:00
CJACK.
217e0e9f01 Merge pull request #398 from CJackHwang/dev
feat: enhanced input validation, tool call resilience & performance
2026-05-02 04:04:44 +08:00
CJACK
b3c54fcf3d chore: bump version to 4.3.0
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 03:55:36 +08:00
CJACK
1c38709d32 feat: add support for parsing loose JSON lists into arrays in tool call parameters 2026-05-02 03:26:43 +08:00
CJACK
28e800c670 chore: remove obsolete planning and gate target files 2026-05-02 02:40:57 +08:00
CJACK
4389e02b29 feat: implement sync.Pool for tiktoken encoding instances to optimize token counting performance 2026-05-02 02:31:24 +08:00
CJACK
e2756f800d feat: introduce JSON UTF-8 validation middleware and prepend output integrity guard system prompt to messages 2026-05-02 02:22:34 +08:00
CJACK
55abf64717 feat: add model type support for file uploads with automatic resolution and header propagation 2026-05-02 00:55:17 +08:00
CJACK
76ee2faa12 chore: bump version to 4.2.2 and update documentation to reflect improved release workflows, CI dependencies, and project structure 2026-05-01 23:44:07 +08:00
CJACK.
9c3696194c Merge pull request #393 from CJackHwang/dev
Merge pull request #391 from BigUncle/fix/vercel-admin-history-rewrite

Fix: add missing Vercel rewrite rules for admin API routes
2026-05-01 23:28:03 +08:00
CJACK
0bca6e2cee feat: implement context cancellation handling for chat and response stream runtimes to ensure clean termination without retries 2026-05-01 23:20:46 +08:00
CJACK.
934b40e572 Merge pull request #392 from wyv202011y/fix/timeout-and-context-cancel
fix: increase stream timeout constants for large-context models; guar…
2026-05-01 23:17:31 +08:00
CJACK
dd5a0c5213 refactor: update and standardize current input file continuation prompt instructions 2026-05-01 22:27:59 +08:00
CJACK
43402e7a26 refactor: rename history file constant from HISTORY.txt to DS2API_HISTORY.txt across codebase and tests 2026-05-01 22:05:45 +08:00
CJACK.
6373c001f5 Merge pull request #391 from BigUncle/fix/vercel-admin-history-rewrite
Fix: add missing Vercel rewrite rules for admin API routes
2026-05-01 21:45:14 +08:00
BigUncle
3430322e81 docs: add Vercel chat history read-only filesystem troubleshooting 2026-05-01 21:17:52 +08:00
CJACK
df1cfac9bc refactor: replace history transcript format with numbered sections and rename upload file to HISTORY.txt 2026-05-01 21:15:17 +08:00
706e68de23 fix: increase stream timeout constants for large-context models; guard against context-cancelled double-recording
- Increase StreamIdleTimeout from 90s to 300s and MaxKeepaliveCount from 10 to 40
  to prevent premature stream termination with DeepSeek V4 Pro (~50K token contexts)
- Add r.Context().Err() check after ConsumeSSE in empty_retry_runtime (chat + responses)
  to prevent historySession.error() from overwriting historySession.stopped()
  when the request context is cancelled

References:
- MaxKeepaliveCount=10 creates a 50s no-content timeout that kills the stream
  before DeepSeek V4 Pro can produce its first token with large contexts
- Hermes Agent reports 'No response from provider for 180s' because the
  underlying SSE connection was already terminated by ds2api at 50s
- Context cancellation path: OnContextDone -> stopped(), then finalize()
  with empty output -> retry -> error() overwrites stopped()
2026-05-01 21:11:36 +08:00
BigUncle
83b4c7bcad fix: add missing Vercel rewrite rules for admin API routes
/admin/chat-history, /admin/proxies, /admin/dev/raw-samples,
and /admin/dev/captures were falling through to the SPA fallback
(/admin/index.html), causing "Unexpected token '<'" JSON parse
errors on the frontend.
2026-05-01 20:50:12 +08:00
VanceHud
603801c542 Merge branch 'CJackHwang:main' into main 2026-05-01 17:18:00 +08:00
CJACK.
445c95a4f2 Merge pull request #379 from CJackHwang/dev
Merge pull request #377 from CJackHwang/codex/run-all-tests-and-fix-failures

Fix failing current-input token accounting test
2026-05-01 16:12:17 +08:00
CJACK
0a6ef8e3f2 fix: remove bufio.Scanner 2MiB line limit for SSE; support quasi_status direct patch
Replace bufio.Scanner with bufio.NewReaderSize + ReadBytes('\n') across all
SSE read paths to preserve long single-line data (e.g. write_file content).
Add quasi_status and auto_continue handling as direct path-based patches in
both Go continue observer and Node vercel_stream_impl, mirroring existing
batch-patch logic. Add 2MiB+ line throughput tests at every SSE layer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 15:45:17 +08:00
CJACK
fd0ec29991 refactor: generalize DSML tag parsing to tolerate model noise; split tiktoken by build tags
Replace hardcoded DSML typo variant lists in Go/Node tool call parsers with
generalized prefix consumption that tolerates repeated leading <, repeated DSML
prefix noise, and trailing pipe terminators. Split tiktoken-dependent token
counting into a build-tagged file for non-cgo platform compatibility. Add /data
directory to Dockerfile for bind-mount permissions.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 15:17:11 +08:00
VanceHud
febd3ec83a Document Zeabur manual deployment 2026-05-01 14:29:49 +08:00
CJACK
2671298439 fix: coalesce small stream deltas to prevent character swallowing; add read-tool cache guard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 13:53:27 +08:00
CJACK
92e321fe2c 修复吞字问题 2026-05-01 01:31:48 +08:00
CJACK.
fca8c01397 Merge pull request #385 from ouqiting/fix_chat_history
fix: content being overwritten and left empty
2026-05-01 00:21:33 +08:00
ouqiting
667e1e3710 fix: content being overwritten and left empty 2026-04-30 21:36:47 +08:00
CJACK.
4438d03c5c Merge pull request #377 from CJackHwang/codex/run-all-tests-and-fix-failures
Fix failing current-input token accounting test
2026-04-30 02:41:08 +08:00
CJACK.
95b7665643 Merge branch 'dev' into codex/run-all-tests-and-fix-failures 2026-04-30 02:39:18 +08:00
CJACK.
9896b1fc33 Merge pull request #378 from NgoQuocViet2001/ai/openai-root-route-aliases
feat(openai): add root route aliases
2026-04-30 02:33:38 +08:00
CJACK.
966f21211d Fix nil-session guard in chat history test 2026-04-30 02:31:06 +08:00
NgoQuocViet2001
7dc3af40b2 feat(openai): add root route aliases 2026-04-30 01:24:53 +07:00
CJACK.
2f6b5ffda0 Fix current-input token text test expectation 2026-04-30 02:22:17 +08:00
CJACK.
85e256ad4d Merge pull request #375 from CJackHwang/codex/investigate-data-loss-issue-in-pr-369
sse/parser: treat object-shaped `v` as visible content, preserve INCOMPLETE across omitted status; add tests and samples
2026-04-30 02:14:26 +08:00
CJACK.
7c3ff6ee7e Merge pull request #374 from shern-point/feat/full-context-file-token-accounting
Feat/full context file token accounting
2026-04-30 02:12:55 +08:00
CJACK.
63e62fd1b0 Merge pull request #372 from shern-point/feat/accurate-context-token-length
Feat/accurate context token length
2026-04-30 02:11:32 +08:00
CJACK.
483d7af3d2 Merge pull request #373 from NgoQuocViet2001/ai/ds2api-small-regression-fix
fix(openai): return 400 for inline file limit
2026-04-30 02:08:22 +08:00
CJACK.
0f89823526 chore(sse): bump client version and refresh longtext stream fixtures 2026-04-30 02:05:45 +08:00
shern-point
6a778e0d35 feat: include inline-uploaded file tokens in context token accounting
Track byte sizes of inline-uploaded files during PreprocessInlineFileInputs and convert them to conservative token estimates (bytes/3). RefFileTokens is threaded through StandardRequest into all OpenAI chat/responses usage builders so returned prompt_tokens/input_tokens reflect the full upstream context cost including attached files.
2026-04-30 01:42:51 +08:00
NgoQuocViet2001
9035c350a7 fix(openai): return 400 for inline file limit 2026-04-30 00:35:59 +07:00
shern-point
ba80052a26 fix: count uploaded file content in context token accounting
PromptTokenText now reflects the actual downstream context cost: the uploaded IGNORE.txt file content plus the neutral live prompt, instead of only the pre-split prompt text.
2026-04-30 01:12:35 +08:00
shern-point
78fdd63470 feat: add full-context token regression coverage and docs
Lock in the current_input_file regression with API-level tests and document that returned context token counts now track full prompt semantics with conservative sizing.
2026-04-30 00:46:06 +08:00
shern-point
4b4f097006 feat: use model-aware prompt counting in Gemini paths
Preserve Gemini prompt token text during normalization and remove the hardcoded DeepSeek model from native Gemini usage helpers.
2026-04-30 00:46:05 +08:00
shern-point
d3018c281b feat: use tokenizer-based counting in Claude token paths
Unify Claude count_tokens, legacy stream accounting, and legacy render usage with preserved prompt text so Claude stops falling back to lossy message formatting.
2026-04-30 00:46:04 +08:00
shern-point
415a2359ad feat: route OpenAI responses usage through preserved prompt text
Use the stored full-context prompt text for responses accounting so neutral placeholder prompts do not underreport returned input token counts.
2026-04-30 00:45:31 +08:00
shern-point
f702d45a24 feat: route OpenAI chat usage through preserved prompt text
Use the stored full-context prompt text for chat non-stream, stream, and retry accounting so current_input_file no longer shrinks returned prompt token counts.
2026-04-30 00:45:30 +08:00
shern-point
90817cb9e2 feat: apply tokenizer-based counting in OpenAI usage builders
Move OpenAI chat and responses usage accounting onto the shared tokenizer-aware counters so prompt and output usage stay model-aware and conservatively sized.
2026-04-30 00:45:29 +08:00
shern-point
b96f736bd2 feat: preserve full prompt text across current_input_file rewrites
Keep token accounting tied to the original prompt even after the live prompt is replaced with a neutral placeholder and hidden context file.
2026-04-30 00:45:01 +08:00
shern-point
8ab028c52a feat: seed PromptTokenText during request normalization
Capture the fully built prompt at normalization time for OpenAI and Gemini-compatible requests so usage paths can reuse the original context text.
2026-04-30 00:44:59 +08:00
shern-point
78366afec5 feat: add PromptTokenText to StandardRequest
Track a dedicated prompt string for token accounting so later prompt rewrites can keep returning full-context counts.
2026-04-30 00:44:57 +08:00
shern-point
bd41c8a90c feat: add tokenizer-based token counting utilities
Use go-tiktoken with embedded vocabularies for accurate BPE token counting. CountPromptTokens applies conservative padding so returned context token counts stay slightly above the real value instead of undercounting.
2026-04-30 00:44:11 +08:00
CJACK.
bc2a78ae29 Merge pull request #370 from CJackHwang/codex/align-vercel-behavior-with-go
fix(vercel): align JS stream parser with Go object-shaped content
2026-04-30 00:12:32 +08:00
CJACK.
192cdf8562 fix(vercel): align JS stream parser with Go object-shaped content 2026-04-29 23:56:16 +08:00
CJACK.
94c1acace5 Merge pull request #369 from CJackHwang/dev
Merge pull request #368 from CJackHwang/codex/fix-review-issues-for-pr-#364

Restore thinking fallback for tool-call detection and drop history.txt wrapper tags
2026-04-29 23:42:50 +08:00
CJACK.
273c18ba0f fix: fallback to /app config when /data is unavailable 2026-04-29 23:40:07 +08:00
CJACK.
ae28e33184 fix: preserve continue state when chunk status is missing 2026-04-29 23:25:18 +08:00
CJACK.
0438ce9a12 Merge pull request #368 from CJackHwang/codex/fix-review-issues-for-pr-#364
Restore thinking fallback for tool-call detection and drop history.txt wrapper tags
2026-04-29 23:07:36 +08:00
CJACK.
af4a067dab Merge pull request #362 from CJackHwang/codex/fix-issue-based-on-feedback
fix(sse): batch tiny stream chunks before emitting
2026-04-29 23:07:03 +08:00
CJACK.
33f6fef015 Fix tool-call fallback on sanitized empty text and remove history wrapper tags 2026-04-29 23:04:45 +08:00
CJACK.
6d3979a1d6 fix(sse): stop scanner sender when stream context cancels 2026-04-29 22:59:22 +08:00
CJACK.
c8922c7a88 Merge pull request #364 from adnxx1wsx/dev
Fix stream compatibility and vision model exposure
2026-04-29 22:02:19 +08:00
MiY
241334c658 Fix stream compatibility and vision model exposure 2026-04-29 20:23:13 +08:00
CJACK.
d7e071b24a Bump version from 4.1.3 to 4.2.0 2026-04-29 19:08:57 +08:00
CJACK.
89225c778e fix(sse): batch tiny stream chunks before emitting 2026-04-29 18:58:54 +08:00
CJACK.
22160de2c4 Merge pull request #359 from NgoQuocViet2001/ai/ds2api-small-fix
fix(openai): keep citation indexes one-based with zero-based references
2026-04-29 18:27:15 +08:00
NgoQuocViet2001
0cbc2c875d fix(openai): keep citation indexes one-based 2026-04-29 15:43:09 +07:00
173 changed files with 17850 additions and 6049 deletions

View File

@@ -47,7 +47,6 @@ jobs:
- name: Release Blocking Gates
run: |
./tests/scripts/check-stage6-manual-smoke.sh
./tests/scripts/check-refactor-line-gate.sh
./tests/scripts/run-unit-all.sh

1
.gitignore vendored
View File

@@ -29,6 +29,7 @@ yarn.lock
pnpm-lock.yaml
# Build artifacts
dist/
*.tsbuildinfo
.cache/
.parcel-cache/

View File

@@ -33,6 +33,8 @@ Docs: [Overview](README.en.md) / [Architecture](docs/ARCHITECTURE.en.md) / [Depl
| Health probes | `GET /healthz`, `GET /readyz` |
| CORS | Enabled (uniformly covers `/v1/*`, `/anthropic/*`, `/v1beta/models/*`, and `/admin/*`; echoes the browser `Origin` when present, otherwise `*`; default allow-list includes `Content-Type`, `Authorization`, `X-API-Key`, `X-Ds2-Target-Account`, `X-Ds2-Source`, `X-Vercel-Protection-Bypass`, `X-Goog-Api-Key`, `Anthropic-Version`, `Anthropic-Beta`, and also accepts third-party preflight-requested headers such as `x-stainless-*`; `/v1/chat/completions` on Vercel Node Runtime matches the same behavior; internal-only `X-Ds2-Internal-Token` remains blocked) |
- All JSON request bodies must be valid UTF-8; malformed byte sequences are rejected on ingress with `400 invalid json`.
### 3.0 Adapter-Layer Notes
- OpenAI / Claude / Gemini protocols are now mounted on one shared `chi` router tree assembled in `internal/server/router.go`.
@@ -81,7 +83,7 @@ Two header formats accepted:
- Token is in `config.keys`**Managed account mode**: DS2API auto-selects an account via rotation
- Token is not in `config.keys`**Direct token mode**: treated as a DeepSeek token directly
**Optional header**: `X-Ds2-Target-Account: <email_or_mobile>` — Pin a specific managed account.
**Optional header**: `X-Ds2-Target-Account: <email_or_mobile>` — Pin a specific managed account; if the target account does not exist or the managed-account queue is exhausted, the request returns `429`, and current responses do not include `Retry-After`. If the account exists but login/refresh fails, the request returns the underlying `401` or upstream error.
Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=` as the caller credential source.
### Admin Endpoints (`/admin/*`)
@@ -109,6 +111,7 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
| GET | `/v1/responses/{response_id}` | Business | Query stored response (in-memory TTL) |
| POST | `/v1/embeddings` | Business | OpenAI Embeddings API |
| POST | `/v1/files` | Business | OpenAI Files upload (multipart/form-data) |
| GET | `/v1/files/{file_id}` | Business | Retrieve uploaded file status |
| GET | `/anthropic/v1/models` | None | Claude model list |
| POST | `/anthropic/v1/messages` | Business | Claude messages |
| POST | `/anthropic/v1/messages/count_tokens` | Business | Claude token counting |
@@ -165,6 +168,8 @@ Gemini-compatible clients can also send `x-goog-api-key`, `?key=`, or `?api_key=
| PUT | `/admin/chat-history/settings` | Admin | Update conversation history retention limit |
| GET | `/admin/version` | Admin | Check current version and latest Release |
OpenAI `/v1/*` paths are canonical. For clients configured with the bare DS2API service URL, the same OpenAI handlers are also exposed through root shortcuts: `/models`, `/models/{id}`, `/chat/completions`, `/responses`, `/responses/{response_id}`, `/embeddings`, `/files`, and `/files/{file_id}`.
---
## Health Endpoints
@@ -196,11 +201,15 @@ No auth required. Returns the currently supported DeepSeek native model list.
"object": "list",
"data": [
{"id": "deepseek-v4-flash", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-flash-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-flash-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-flash-search-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro-search-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
{"id": "deepseek-v4-vision-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
]
}
```
@@ -224,6 +233,8 @@ Built-in aliases come from `internal/config/models.go`; `config.model_aliases` c
- Gemini: `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-pro-vision`
- Other compatibility families: `llama-*`, `qwen-*`, `mistral-*`, and `command-*` fall back through family heuristics
Current vision support resolves only to `deepseek-v4-vision` and does not expose a separate `vision-search` variant.
Retired historical families such as `claude-1.*`, `claude-2.*`, `claude-instant-*`, and `gpt-3.5*` are explicitly rejected.
### `POST /v1/chat/completions`
@@ -297,7 +308,7 @@ data: [DONE]
- When thinking is enabled, the stream may emit `delta.reasoning_content`
- Text emits `delta.content`
- Last chunk includes `finish_reason` and `usage`
- Token counting prefers pass-through from upstream DeepSeek SSE (`accumulated_token_usage` / `token_usage`), and only falls back to local estimation when upstream usage is absent
- Token counting prefers pass-through from upstream DeepSeek SSE (`accumulated_token_usage` / `token_usage`), and only falls back to local estimation when upstream usage is absent. Failed/interrupted endings (for example `response.failed`) may not include `usage`
#### Tool Calls
@@ -413,7 +424,7 @@ Business auth required. Returns OpenAI-compatible embeddings shape.
| `model` | string | ✅ | Supports native models + alias mapping |
| `input` | string/array | ✅ | Supports string, string array, token array |
> Requires `embeddings.provider`. Current supported values: `mock` / `deterministic` / `builtin`. If missing/unsupported, returns standard error shape with HTTP 501.
> Requires `embeddings.provider`. Current supported values: `mock` / `deterministic` / `builtin` (all three use the same local deterministic implementation). If missing/unsupported, returns standard error shape with HTTP 501.
### `POST /v1/files`
@@ -427,9 +438,13 @@ Business auth required. OpenAI Files-compatible upload endpoint; currently only
Constraints and behavior:
- `Content-Type` must be `multipart/form-data` (otherwise `400`).
- Total request size limit is `100 MiB` (over-limit returns `413`).
- Total request size limit is **100 MiB** (over-limit returns `413`).
- Success returns an OpenAI `file` object (`id/object/bytes/filename/purpose/status`, etc.) and includes `account_id` for source-account tracing.
### `GET /v1/files/{file_id}`
Business auth required. Retrieves the current DeepSeek upload status for a file and returns an OpenAI `file` object. Returns `404` when no matching file is found.
---
## Claude-Compatible API
@@ -481,6 +496,13 @@ anthropic-version: 2023-06-01
| `stream` | boolean | ❌ | Default `false` |
| `system` | string | ❌ | Optional system prompt |
| `tools` | array | ❌ | Claude tool schema |
| `thinking` | object | ❌ | Anthropic thinking config; translated into downstream reasoning control, and ignored by `-nothinking` models |
| `temperature` | number | ❌ | Passed through to the downstream bridge; if `temperature` and `top_p` are both present, `temperature` wins |
| `top_p` | number | ❌ | Passed through when `temperature` is absent |
| `stop_sequences` | array | ❌ | Passed through as downstream stop sequences |
| `tool_choice` | string/object | ❌ | Supports `auto` / `none` / `required` / `{"type":"function","name":"..."}` and is translated to downstream tool choice |
> Note: `thinking`, `temperature`, `top_p`, `stop_sequences`, and `tool_choice` are translated through the compatibility bridge. Final behavior still depends on the selected model and upstream support. When both `temperature` and `top_p` are present, `temperature` takes precedence.
#### Non-Stream Response
@@ -1209,7 +1231,7 @@ Clients should handle HTTP status code plus `error` / `detail` fields.
| Code | Meaning |
| --- | --- |
| `401` | Authentication failed (invalid key/token, or expired admin JWT) |
| `429` | Too many requests (exceeded inflight + queue capacity) |
| `429` | Too many requests (exceeded inflight + queue capacity; current responses do not include `Retry-After`) |
| `503` | Model unavailable or upstream error |
---

31
API.md
View File

@@ -33,6 +33,8 @@
| 健康检查 | `GET /healthz``GET /readyz` |
| CORS | 已启用(统一覆盖 `/v1/*``/anthropic/*``/v1beta/models/*``/admin/*`;浏览器有 `Origin` 时回显该 Origin否则为 `*`;默认允许 `Content-Type`, `Authorization`, `X-API-Key`, `X-Ds2-Target-Account`, `X-Ds2-Source`, `X-Vercel-Protection-Bypass`, `X-Goog-Api-Key`, `Anthropic-Version`, `Anthropic-Beta`,并会放行预检里声明的第三方请求头,如 `x-stainless-*`Vercel 上 `/v1/chat/completions` 的 Node Runtime 也对齐相同行为;内部专用头 `X-Ds2-Internal-Token` 仍被拦截) |
- 所有 JSON 请求体都必须是合法 UTF-8非法字节序列会在入站阶段被拒绝为 `400 invalid json`
### 3.0 接口适配层说明
- OpenAI / Claude / Gemini 三套协议已统一挂在同一 `chi` 路由树上,由 `internal/server/router.go` 负责装配。
@@ -81,7 +83,7 @@ Vercel 一键部署可先只填 `DS2API_ADMIN_KEY`,部署后在 `/admin` 导
- token 在 `config.keys` 中 → **托管账号模式**,自动轮询选择账号
- token 不在 `config.keys` 中 → **直通 token 模式**,直接作为 DeepSeek token 使用
**可选请求头**`X-Ds2-Target-Account: <email_or_mobile>` — 指定使用某个托管账号。
**可选请求头**`X-Ds2-Target-Account: <email_or_mobile>` — 指定使用某个托管账号;如果目标账号不存在,或管理账号队列已耗尽,相关业务请求会返回 `429`,当前不会附带 `Retry-After` 头。若账号存在但登录/刷新失败,则返回对应的 `401` 或上游错误
Gemini 兼容客户端还可以使用 `x-goog-api-key``?key=``?api_key=` 作为凭据来源。
### Admin 接口(`/admin/*`
@@ -109,6 +111,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
| GET | `/v1/responses/{response_id}` | 业务 | 查询已生成 response内存 TTL |
| POST | `/v1/embeddings` | 业务 | OpenAI Embeddings 接口 |
| POST | `/v1/files` | 业务 | OpenAI Files 上传multipart/form-data |
| GET | `/v1/files/{file_id}` | 业务 | 查询已上传文件状态 |
| GET | `/anthropic/v1/models` | 无 | Claude 模型列表 |
| POST | `/anthropic/v1/messages` | 业务 | Claude 消息接口 |
| POST | `/anthropic/v1/messages/count_tokens` | 业务 | Claude token 计数 |
@@ -165,6 +168,8 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
| PUT | `/admin/chat-history/settings` | Admin | 更新对话记录保留条数 |
| GET | `/admin/version` | Admin | 查询当前版本与最新 Release |
OpenAI `/v1/*` 仍是规范路径。对于只配置 DS2API 根地址的客户端,同一套 OpenAI handler 也通过根路径快捷路由暴露:`/models``/models/{id}``/chat/completions``/responses``/responses/{response_id}``/embeddings``/files``/files/{file_id}`
---
## 健康检查
@@ -204,9 +209,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
{"id": "deepseek-v4-pro-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-pro-search-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-search", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []},
{"id": "deepseek-v4-vision-search-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
{"id": "deepseek-v4-vision-nothinking", "object": "model", "created": 1677610602, "owned_by": "deepseek", "permission": []}
]
}
```
@@ -232,6 +235,7 @@ Gemini 兼容客户端还可以使用 `x-goog-api-key`、`?key=` 或 `?api_key=`
- 其他兼容族:`llama-*``qwen-*``mistral-*``command-*` 会按家族启发式回退
上述 alias 若在请求名后追加 `-nothinking` 后缀,也会映射到对应的强制关闭 thinking 版本。
当前视觉能力仅对应 `deepseek-v4-vision` / `deepseek-v4-vision-nothinking`,不会解析出独立的 `vision-search` 变体。
退役历史模型(如 `claude-1.*``claude-2.*``claude-instant-*``gpt-3.5*`)会被显式拒绝。
@@ -306,7 +310,7 @@ data: [DONE]
- 开启 thinking 时会输出 `delta.reasoning_content`
- 普通文本输出 `delta.content`
- 最后一段包含 `finish_reason``usage`
- token 计数优先透传上游 DeepSeek SSE`accumulated_token_usage` / `token_usage`);仅在上游缺失时回退本地估算
- token 计数优先透传上游 DeepSeek SSE`accumulated_token_usage` / `token_usage`);仅在上游缺失时回退本地估算。失败/中断型结束(例如 `response.failed`)可能不会携带 `usage`
#### Tool Calls
@@ -423,7 +427,7 @@ data: [DONE]
| `model` | string | ✅ | 支持原生模型 + alias 自动映射 |
| `input` | string/array | ✅ | 支持字符串、字符串数组、token 数组 |
> 需配置 `embeddings.provider`。当前支持:`mock` / `deterministic` / `builtin`。未配置或不支持时返回标准错误结构HTTP 501
> 需配置 `embeddings.provider`。当前支持:`mock` / `deterministic` / `builtin`(三者都走同一套本地确定性实现)。未配置或不支持时返回标准错误结构HTTP 501
### `POST /v1/files`
@@ -437,9 +441,13 @@ data: [DONE]
约束与行为:
- 请求必须为 `multipart/form-data`,否则返回 `400`
- 请求体总大小上限 `100 MiB`(超限返回 `413`)。
- 请求体总大小上限 **100 MiB**(超限返回 `413`)。
- 成功返回 OpenAI `file` 对象(`id/object/bytes/filename/purpose/status` 等字段),并附带 `account_id` 便于定位来源账号。
### `GET /v1/files/{file_id}`
需要业务鉴权。查询 DeepSeek 上传文件的当前状态,并返回 OpenAI `file` 对象;未找到匹配文件时返回 `404`
---
## Claude 兼容接口
@@ -494,6 +502,13 @@ anthropic-version: 2023-06-01
| `stream` | boolean | ❌ | 默认 `false` |
| `system` | string | ❌ | 可选系统提示 |
| `tools` | array | ❌ | Claude tool 定义 |
| `thinking` | object | ❌ | Anthropic thinking 配置;会转译为下游 reasoning 控制,`-nothinking` 模型会忽略 |
| `temperature` | number | ❌ | 透传到下游;若同时提供 `top_p`,以 `temperature` 为准 |
| `top_p` | number | ❌ | 当未提供 `temperature` 时透传到下游 |
| `stop_sequences` | array | ❌ | 透传到下游停用序列 |
| `tool_choice` | string/object | ❌ | 支持 `auto` / `none` / `required` / `{"type":"function","name":"..."}`,并会转译为下游工具选择 |
> 说明:上述 `thinking`、`temperature`、`top_p`、`stop_sequences`、`tool_choice` 都会走兼容层转译;最终是否生效仍取决于当前模型和上游能力。`temperature` 与 `top_p` 同时存在时,`temperature` 优先。
#### 非流式响应
@@ -1225,7 +1240,7 @@ Gemini 路由使用 Google 风格错误结构:
| 状态码 | 说明 |
| --- | --- |
| `401` | 鉴权失败key/token 无效,或 Admin JWT 过期) |
| `429` | 请求过多(超出并发上限 + 等待队列) |
| `429` | 请求过多(超出并发上限 + 等待队列;当前不附带 `Retry-After` |
| `503` | 模型不可用或上游服务异常 |
---

View File

@@ -29,7 +29,7 @@ WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates \
&& groupadd -r ds2api && useradd -r -g ds2api -d /app -s /sbin/nologin ds2api \
&& mkdir -p /app/data && chown -R ds2api:ds2api /app \
&& mkdir -p /app/data /data && chown -R ds2api:ds2api /app /data \
&& rm -rf /var/lib/apt/lists/*
COPY --from=busybox-tools /bin/busybox /usr/local/bin/busybox
EXPOSE 5001

View File

@@ -17,7 +17,7 @@
语言 / Language: [中文](README.MD) | [English](README.en.md)
将 DeepSeek Web 对话能力转换为 OpenAI、Claude 与 Gemini 兼容 API。后端 **Go 全量实现**,前端为 React WebUI 管理台(源码在 `webui/`,部署时自动构建到 `static/admin`)。
将 DeepSeek Web 对话能力转换为 OpenAI、Claude 与 Gemini 兼容 API。核心后端 **Go** 实现Vercel 流式桥接额外使用少量 Node Runtime,前端为 React WebUI 管理台(源码在 `webui/`,部署时自动构建到 `static/admin`)。
文档入口:[文档导航](docs/README.md) / [架构说明](docs/ARCHITECTURE.md) / [接口文档](API.md)
@@ -119,7 +119,7 @@ flowchart LR
| 能力 | 说明 |
| --- | --- |
| OpenAI 兼容 | `GET /v1/models`、`GET /v1/models/{id}`、`POST /v1/chat/completions`、`POST /v1/responses`、`GET /v1/responses/{response_id}`、`POST /v1/embeddings`、`POST /v1/files` |
| OpenAI 兼容 | `GET /v1/models`、`GET /v1/models/{id}`、`POST /v1/chat/completions`、`POST /v1/responses`、`GET /v1/responses/{response_id}`、`POST /v1/embeddings`、`POST /v1/files`、`GET /v1/files/{file_id}` |
| Claude 兼容 | `GET /anthropic/v1/models`、`POST /anthropic/v1/messages`、`POST /anthropic/v1/messages/count_tokens`(及快捷路径 `/v1/messages`、`/messages` |
| Gemini 兼容 | `POST /v1beta/models/{model}:generateContent`、`POST /v1beta/models/{model}:streamGenerateContent`(及 `/v1/models/{model}:*` 路径) |
| 统一 CORS 兼容 | `/v1/*`、`/anthropic/*`、`/v1beta/models/*`、`/admin/*` 统一走同一套 CORS 策略Vercel 上 `/v1/chat/completions` 的 Node Runtime 也对齐相同放行规则,尽量减少第三方预检请求头限制 |
@@ -131,6 +131,8 @@ flowchart LR
| WebUI 管理台 | `/admin` 单页应用(中英文双语、深色模式,支持查看服务器端对话记录) |
| 运维探针 | `GET /healthz`(存活)、`GET /readyz`(就绪) |
OpenAI `/v1/*` 仍是推荐的规范路径;同时支持 `/models`、`/chat/completions`、`/responses`、`/embeddings`、`/files`、`/files/{file_id}` 等根路径快捷路由,方便只配置 DS2API 根地址的第三方客户端。
## 平台兼容矩阵
| 级别 | 平台 | 当前状态 |
@@ -158,10 +160,9 @@ flowchart LR
| expert | `deepseek-v4-pro-search-nothinking` | 永久关闭,不受请求参数影响 | ✅ |
| vision | `deepseek-v4-vision` | 默认开启,可由请求参数控制 | ❌ |
| vision | `deepseek-v4-vision-nothinking` | 永久关闭,不受请求参数影响 | ❌ |
| vision | `deepseek-v4-vision-search` | 默认开启,可由请求参数控制 | ✅ |
| vision | `deepseek-v4-vision-search-nothinking` | 永久关闭,不受请求参数影响 | ✅ |
除原生模型外,也支持常见 alias 输入(如 `gpt-4.1`、`gpt-5`、`gpt-5-codex`、`o3`、`claude-*`、`gemini-*` 等),但 `/v1/models` 返回的是规范化后的 DeepSeek 原生模型 ID。若 alias 名本身追加 `-nothinking` 后缀,也会映射到对应的强制关思考模型。完整 alias 行为以 [API.md](API.md#模型-alias-解析策略) 和 `config.example.json` 为准。
当前上游视觉模型只暴露 `vision` 通道,不提供独立的联网搜索视觉变体。
### Claude 接口(`GET /anthropic/v1/models`
@@ -246,6 +247,7 @@ docker-compose logs -f
默认 `docker-compose.yml` 会把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请设置 `DS2API_HOST_PORT=5001`(或者手动调整 `ports` 配置)。
同时默认把 `./config.json` 挂载到容器 `/data/config.json`,并设置 `DS2API_CONFIG_PATH=/data/config.json`,用于避免 `/app` 只读导致运行时 token 持久化失败。
镜像会预创建 `/data` 并授权给非 root 的 `ds2api` 用户;如果使用单文件 bind mount请确保宿主机 `config.json` 对容器用户可读写,例如 `chmod 644 config.json`。
更新镜像:`docker-compose up -d --build`
@@ -255,6 +257,10 @@ docker-compose logs -f
2. 部署完成后访问 `/admin`,使用 Zeabur 环境变量/模板指引中的 `DS2API_ADMIN_KEY` 登录。
3. 在管理台导入/编辑配置(会写入并持久化到 `/data/config.json`)。
Zeabur 首次空卷启动时可以没有 `/data/config.json`DS2API 会先使用空的文件模式配置启动,并在管理台首次保存时创建该文件。
不依赖模板手动部署时,在 Zeabur 中选择 GitHub 仓库服务Root Directory 保持 `/`,使用仓库根目录 `Dockerfile` 构建;添加持久卷 `/data`,设置 `PORT=5001`、`DS2API_ADMIN_KEY=你的强密钥`、`DS2API_CONFIG_PATH=/data/config.json`,然后暴露 HTTP 端口 `5001`。更完整步骤见 [docs/DEPLOY.md](docs/DEPLOY.md#不使用模板手动部署)。
说明Zeabur 使用仓库内 `Dockerfile` 直接构建时,不需要额外传入 `BUILD_VERSION`;镜像会优先读取该构建参数,未提供时自动回退到仓库根目录的 `VERSION` 文件。
### 方式三Vercel 部署
@@ -283,7 +289,7 @@ base64 < config.json | tr -d '\n'
### 方式四:本地源码运行
**前置要求**Go 1.26+Node.js `20.19+` 或 `22.12+`(仅在需要构建 WebUI 时)
**前置要求**Go 1.26+Node.js `20.19+` 或 `22.12+`(仅在需要构建 WebUI 时);同时确保 `npm` 可用,建议 `npm 10+`
```bash
# 1. 克隆仓库
@@ -302,7 +308,7 @@ go run ./cmd/ds2api
服务实际绑定:`0.0.0.0:5001`,因此同一局域网设备通常也可以通过你的内网 IP 访问。
> **WebUI 自动构建**:本地首次启动时,若 `static/admin` 不存在,会自动尝试执行 `npm ci`(仅在缺少依赖时)和 `npm run build -- --outDir static/admin --emptyOutDir`(需要本机有 Node.js。你也可以手动构建`./scripts/build-webui.sh`
> **WebUI 自动构建**:本地首次启动时,若 `static/admin` 不存在,会自动尝试执行 `npm ci`(仅在缺少依赖时)和 `npm run build -- --outDir static/admin --emptyOutDir`(需要本机有 Node.js 和 npm)。你也可以手动构建:`./scripts/build-webui.sh`
## 配置说明
@@ -316,7 +322,7 @@ go run ./cmd/ds2api
- `runtime`:账号并发、队列与 token 刷新策略,可通过 Admin Settings 热更新。
- `auto_delete.mode`:请求结束后的远端会话清理策略,支持 `none` / `single` / `all`。
- `history_split`:旧轮次拆分字段,已废弃并忽略,仅保留兼容旧配置。
- `current_input_file`:唯一生效的独立拆分策略;默认开启且阈值为 `0`,触发时将完整上下文合并上传为隐藏上下文文件。
- `current_input_file`:唯一生效的独立拆分策略;默认开启且阈值为 `0`,触发时将完整上下文合并上传为 `DS2API_HISTORY.txt` 上下文文件。
- 如果关闭 `current_input_file`,请求会直接透传,不上传拆分上下文文件。
- `thinking_injection`:默认开启;在最新 user 消息末尾追加思考增强提示词,提高高强度推理与工具调用前的思考稳定性;`prompt` 留空时使用内置默认提示词。
@@ -332,6 +338,7 @@ go run ./cmd/ds2api
| **直通 token 模式** | 传入 token 不在 `config.keys` 中时,直接作为 DeepSeek token 使用 |
可选请求头 `X-Ds2-Target-Account`:指定使用某个托管账号(值为 email 或 mobile
如果指定账号不存在,或者当前管理账号队列已满,请求会返回 `429`;当前 `429` 不附带 `Retry-After` 头。若账号存在但登录/刷新失败,则返回对应的鉴权错误。
Gemini 路由还可以使用 `x-goog-api-key`,或在没有认证头时使用 `?key=` / `?api_key=` 作为调用方凭据。
## 并发模型
@@ -344,7 +351,7 @@ Gemini 路由还可以使用 `x-goog-api-key`,或在没有认证头时使用 `
```
- 当 in-flight 槽位满时,请求进入等待队列,**不会立即 429**
- 超出总承载上限后才返回 `429 Too Many Requests`
- 超出总承载上限后才返回 `429 Too Many Requests`,当前响应不附带 `Retry-After`
- `GET /admin/queue/status` 返回实时并发状态
## Tool Call 适配
@@ -422,10 +429,10 @@ npm run build --prefix webui
工作流文件:`.github/workflows/release-artifacts.yml`
- **触发条件**:仅在 GitHub Release `published` 时触发(普通 push 不会触发)
- **构建产物**:多平台二进制包(`linux/amd64`、`linux/arm64`、`linux/armv7`、`darwin/amd64`、`darwin/arm64`、`windows/amd64`、`windows/arm64`+ `sha256sums.txt`
- **触发条件**默认仅在 GitHub Release `published` 时自动触发;也支持在 Actions 页面手动 `workflow_dispatch`,并填写 `release_tag` 复跑/补发
- **构建产物**:多平台二进制包(`linux/amd64`、`linux/arm64`、`linux/armv7`、`darwin/amd64`、`darwin/arm64`、`windows/amd64`、`windows/arm64`、Linux Docker 镜像导出包 + `sha256sums.txt`
- **容器镜像发布**:仅推送到 GHCR`ghcr.io/cjackhwang/ds2api`
- **每个压缩包包含**`ds2api` 可执行文件、`static/admin`、WASM 文件(同时支持内置 fallback、`config.example.json` 配置示例、README、LICENSE
- **每个二进制压缩包包含**`ds2api` 可执行文件、`static/admin`、`config.example.json`、`.env.example`、`README.MD`、`README.en.md`、`LICENSE`
## 免责声明

View File

@@ -16,7 +16,7 @@
Language: [中文](README.MD) | [English](README.en.md)
DS2API converts DeepSeek Web chat capability into OpenAI-compatible, Claude-compatible, and Gemini-compatible APIs. The backend is a **pure Go implementation**, with a React WebUI admin panel (source in `webui/`, build output auto-generated to `static/admin` during deployment).
DS2API converts DeepSeek Web chat capability into OpenAI-compatible, Claude-compatible, and Gemini-compatible APIs. The core backend is Go-based, with a small Node Runtime bridge used for Vercel streaming, and the React WebUI admin panel lives in `webui/` (build output auto-generated to `static/admin` during deployment).
Documentation entry: [Docs Index](docs/README.md) / [Architecture](docs/ARCHITECTURE.en.md) / [API Reference](API.en.md)
@@ -116,7 +116,7 @@ For the full module-by-module architecture and directory responsibilities, see [
| Capability | Details |
| --- | --- |
| OpenAI compatible | `GET /v1/models`, `GET /v1/models/{id}`, `POST /v1/chat/completions`, `POST /v1/responses`, `GET /v1/responses/{response_id}`, `POST /v1/embeddings`, `POST /v1/files` |
| OpenAI compatible | `GET /v1/models`, `GET /v1/models/{id}`, `POST /v1/chat/completions`, `POST /v1/responses`, `GET /v1/responses/{response_id}`, `POST /v1/embeddings`, `POST /v1/files`, `GET /v1/files/{file_id}` |
| Claude compatible | `GET /anthropic/v1/models`, `POST /anthropic/v1/messages`, `POST /anthropic/v1/messages/count_tokens` (plus shortcut paths `/v1/messages`, `/messages`) |
| Gemini compatible | `POST /v1beta/models/{model}:generateContent`, `POST /v1beta/models/{model}:streamGenerateContent` (plus `/v1/models/{model}:*` paths) |
| Unified CORS compatibility | `/v1/*`, `/anthropic/*`, `/v1beta/models/*`, and `/admin/*` share one CORS policy; on Vercel, the Node Runtime for `/v1/chat/completions` mirrors the same relaxed preflight behavior for third-party clients |
@@ -128,6 +128,8 @@ For the full module-by-module architecture and directory responsibilities, see [
| WebUI Admin Panel | SPA at `/admin` (bilingual Chinese/English, dark mode, with server-side conversation history) |
| Health Probes | `GET /healthz` (liveness), `GET /readyz` (readiness) |
OpenAI `/v1/*` routes remain canonical, and DS2API also accepts root shortcuts such as `/models`, `/chat/completions`, `/responses`, `/embeddings`, `/files`, and `/files/{file_id}` for clients configured with the bare service URL.
## Platform Compatibility Matrix
| Tier | Platform | Status |
@@ -150,9 +152,9 @@ For the full module-by-module architecture and directory responsibilities, see [
| default | `deepseek-v4-flash-search` | enabled by default, request-controlled | ✅ |
| expert | `deepseek-v4-pro-search` | enabled by default, request-controlled | ✅ |
| vision | `deepseek-v4-vision` | enabled by default, request-controlled | ❌ |
| vision | `deepseek-v4-vision-search` | enabled by default, request-controlled | ✅ |
Besides native IDs, DS2API also accepts common aliases as input (for example `gpt-4.1`, `gpt-5`, `gpt-5-codex`, `o3`, `claude-*`, `gemini-*`), but `/v1/models` returns normalized DeepSeek native model IDs. The complete alias behavior is documented in [API.en.md](API.en.md#model-alias-resolution) and `config.example.json`.
Current upstream vision support exposes only the `vision` lane and does not provide a separate search-enabled vision variant.
### Claude Endpoint (`GET /anthropic/v1/models`)
@@ -243,6 +245,10 @@ Rebuild after updates: `docker-compose up -d --build`
2. After deployment, open `/admin` and login with `DS2API_ADMIN_KEY` shown in Zeabur env/template instructions.
3. Import / edit config in Admin UI (it will be written and persisted to `/data/config.json`).
Fresh Zeabur volumes can start without `/data/config.json`; DS2API will boot with an empty file-backed config and create the file on the first Admin UI save.
For manual deployment without the template, create a Zeabur GitHub service, keep Root Directory as `/`, build with the repo-root `Dockerfile`, mount a persistent volume at `/data`, set `PORT=5001`, `DS2API_ADMIN_KEY=your-strong-secret`, and `DS2API_CONFIG_PATH=/data/config.json`, then expose HTTP port `5001`. See [docs/DEPLOY.en.md](docs/DEPLOY.en.md#manual-deployment-without-the-template) for the full guide.
Note: when Zeabur builds directly from the repo `Dockerfile`, you do not need to pass `BUILD_VERSION`. The image prefers that build arg when provided, and automatically falls back to the repo-root `VERSION` file when it is absent.
### Option 3: Vercel
@@ -304,7 +310,7 @@ Common fields:
- `runtime`: account concurrency, queueing, and token refresh behavior, hot-reloadable via Admin Settings.
- `auto_delete.mode`: remote session cleanup after each request, supporting `none` / `single` / `all`.
- `history_split`: legacy multi-turn history split field, now ignored and kept only for backward-compatible config loading.
- `current_input_file`: the only active split mode; it is enabled by default and uploads the full context as a hidden context file once the character threshold is reached.
- `current_input_file`: the only active split mode; it is enabled by default and uploads the full context as a `DS2API_HISTORY.txt` context file once the character threshold is reached.
- If you turn off `current_input_file`, requests pass through directly without uploading any split context file.
For the full environment variable list, see [docs/DEPLOY.en.md](docs/DEPLOY.en.md). For auth behavior, see [API.en.md](API.en.md#authentication).
@@ -407,10 +413,10 @@ npm run build --prefix webui
Workflow: `.github/workflows/release-artifacts.yml`
- **Trigger**: only on GitHub Release `published` (normal pushes do not trigger builds)
- **Outputs**: multi-platform archives (`linux/amd64`, `linux/arm64`, `linux/armv7`, `darwin/amd64`, `darwin/arm64`, `windows/amd64`, `windows/arm64`) + `sha256sums.txt`
- **Trigger**: by default only on GitHub Release `published`; you can also run it manually via `workflow_dispatch` and pass `release_tag` to rerun / backfill
- **Outputs**: multi-platform binary archives (`linux/amd64`, `linux/arm64`, `linux/armv7`, `darwin/amd64`, `darwin/arm64`, `windows/amd64`, `windows/arm64`), Linux Docker image export tarballs, and `sha256sums.txt`
- **Container publishing**: GHCR only (`ghcr.io/cjackhwang/ds2api`)
- **Each archive includes**: `ds2api` executable, `static/admin`, WASM file (with embedded fallback support), `config.example.json`-based config template, README, LICENSE
- **Each binary archive includes**: the `ds2api` executable, `static/admin`, `config.example.json`, `.env.example`, `README.MD`, `README.en.md`, and `LICENSE`
## Disclaimer

View File

@@ -1 +1 @@
4.1.3
4.4.0

View File

@@ -25,6 +25,8 @@ ds2api/
│ ├── chathistory/ # Server-side conversation history storage/query
│ ├── claudeconv/ # Claude message conversion helpers
│ ├── compat/ # Compatibility and regression helpers
│ ├── assistantturn/ # Upstream output to canonical assistant turn / stream event semantics
│ ├── completionruntime/ # Shared Go DeepSeek completion startup, non-stream collection, and retry
│ ├── config/ # Config loading/validation/hot reload
│ ├── deepseek/ # DeepSeek upstream client/protocol/transport
│ │ ├── client/ # Login/session/completion/upload/delete calls
@@ -171,6 +173,8 @@ flowchart LR
- `internal/httpapi/openai/*`: OpenAI HTTP surface split into chat, responses, files, embeddings, history, and shared packages; chat/responses share the promptcompat, stream, and toolcall semantics.
- `internal/httpapi/{claude,gemini}`: protocol wrappers that normalize into the same prompt compatibility semantics without duplicating upstream execution.
- `internal/promptcompat`: compatibility core for turning OpenAI/Claude/Gemini requests into DeepSeek web-chat plain-text context.
- `internal/assistantturn`: Go output-side canonical semantics, converting DeepSeek SSE collection results and stream finalization state into assistant turns and centralizing thinking, tool call, citation, usage, stop/error behavior.
- `internal/completionruntime`: shared Go completion execution helpers for DeepSeek session/PoW/call startup, non-stream collection, and empty-output retry; streaming paths use it to start upstream requests, continue to use `internal/stream` for real-time consumption, and use `assistantturn` during finalization.
- `internal/translatorcliproxy`: structure translation between Claude/Gemini and OpenAI.
- `internal/deepseek/{client,protocol,transport}`: upstream requests, sessions, PoW adaptation, protocol constants, and transport details.
- `internal/js/chat-stream` + `api/chat-stream.js`: Vercel Node streaming bridge; Go prepare/release owns auth, account lease, and completion payload assembly, while Node relays real-time SSE with Go-aligned finalization and tool sieve semantics.

View File

@@ -25,6 +25,8 @@ ds2api/
│ ├── chathistory/ # 服务器端对话记录存储与查询
│ ├── claudeconv/ # Claude 消息格式转换工具
│ ├── compat/ # 兼容性辅助与回归支持
│ ├── assistantturn/ # 上游输出到统一 assistant turn / stream event 的语义层
│ ├── completionruntime/ # Go 主路径共享 DeepSeek completion 启动、非流式收集与 retry
│ ├── config/ # 配置加载、校验、热更新
│ ├── deepseek/ # DeepSeek 上游 client/protocol/transport
│ │ ├── client/ # 登录、会话、completion、上传/删除等上游调用
@@ -171,6 +173,8 @@ flowchart LR
- `internal/httpapi/openai/*`OpenAI HTTP surface按 chat、responses、files、embeddings、history、shared 拆分chat/responses 共享 promptcompat、stream、toolcall 等核心语义。
- `internal/httpapi/{claude,gemini}`:协议输入输出适配,归一到同一套 prompt compatibility 语义,不重复实现上游调用逻辑。
- `internal/promptcompat`OpenAI/Claude/Gemini 请求到 DeepSeek 网页纯文本上下文的兼容内核。
- `internal/assistantturn`Go 输出侧统一语义层,把 DeepSeek SSE 收集结果和流式收尾状态归一成 assistant turn集中处理 thinking、tool call、citation、usage、stop/error 语义。
- `internal/completionruntime`Go surface 共享的 completion 执行辅助,负责 DeepSeek session/PoW/call 启动、非流式 collect 和 empty-output retry流式路径复用它启动上游请求继续用 `internal/stream` 做实时消费,并在最终收尾阶段接入 `assistantturn`
- `internal/translatorcliproxy`Claude/Gemini 与 OpenAI 结构互转。
- `internal/deepseek/{client,protocol,transport}`上游请求、会话、PoW 适配、协议常量与传输层。
- `internal/js/chat-stream` + `api/chat-stream.js`Vercel Node 流式桥Go prepare/release 管理鉴权、账号租约和 completion payloadNode 侧负责实时 SSE 转发并保持 Go 对齐的终结态和 tool sieve 语义。

View File

@@ -36,7 +36,7 @@ go run ./cmd/ds2api
cd webui
# 2. Install dependencies
npm install
npm ci
# 3. Start dev server (hot reload)
npm run dev

View File

@@ -36,7 +36,7 @@ go run ./cmd/ds2api
cd webui
# 2. 安装依赖
npm install
npm ci
# 3. 启动开发服务器(热更新)
npm run dev

View File

@@ -64,8 +64,8 @@ Use `config.json` as the single source of truth:
Built-in GitHub Actions workflow: `.github/workflows/release-artifacts.yml`
- **Trigger**: only on Release `published` (no build on normal push)
- **Outputs**: multi-platform binary archives + `sha256sums.txt`
- **Trigger**: by default only on Release `published`; you can also run it manually via `workflow_dispatch` and pass `release_tag` to rerun / backfill
- **Outputs**: multi-platform binary archives, Linux Docker image export tarballs, and `sha256sums.txt`
- **Container publishing**: GHCR only (`ghcr.io/cjackhwang/ds2api`)
| Platform | Architecture | Format |
@@ -131,6 +131,7 @@ docker-compose logs -f
The default `docker-compose.yml` directly uses `ghcr.io/cjackhwang/ds2api:latest` and maps host port `6011` to container port `5001`. If you want `5001` exposed directly, set `DS2API_HOST_PORT=5001` (or adjust the `ports` mapping).
The compose template also defaults to `DS2API_CONFIG_PATH=/data/config.json` with `./config.json:/data/config.json` mounted, so deployments avoid read-only `/app` persistence issues by default.
The image pre-creates `/data` and grants it to the non-root `ds2api` user. If you bind-mount a single host file, make sure `config.json` is readable/writable by the container user, for example with `chmod 644 config.json`; otherwise Linux UID/GID mismatches can still cause `open /data/config.json: permission denied`.
Compatibility note: when `DS2API_CONFIG_PATH` is unset and runtime base dir is `/app`, newer versions prefer `/data/config.json`; if that file is missing but legacy `/app/config.json` exists, DS2API automatically falls back to the legacy path to avoid post-upgrade config loss.
If you want a pinned version instead of `latest`, you can also pull a specific tag directly:
@@ -196,7 +197,7 @@ This repo includes a `zeabur.yaml` template for one-click deployment on Zeabur:
Notes:
- **Port**: DS2API listens on `5001` by default; the template sets `PORT=5001`.
- **Persistent config**: the template mounts `/data` and sets `DS2API_CONFIG_PATH=/data/config.json`. After importing config in Admin UI, it will be written and persisted to this path.
- **Persistent config**: the template mounts `/data` and sets `DS2API_CONFIG_PATH=/data/config.json`. On a fresh volume, DS2API starts with an empty file-backed config; after importing config in Admin UI, it will be written and persisted to this path.
- **`open /app/config.json: permission denied`**: this means the instance is trying to persist runtime tokens to a read-only path (commonly `/app` inside the image).
Recommended handling:
1. Set a writable path explicitly: `DS2API_CONFIG_PATH=/data/config.json` (and mount a persistent volume at `/data`);
@@ -205,6 +206,37 @@ Notes:
- **Build version**: Zeabur / regular `docker build` does not require `BUILD_VERSION` by default. The image prefers that build arg when provided, and automatically falls back to the repo-root `VERSION` file when it is absent.
- **First login**: after deployment, open `/admin` and login with `DS2API_ADMIN_KEY` shown in Zeabur env/template instructions (recommended: rotate to a strong secret after first login).
#### Manual Deployment Without The Template
If you do not want to use the `zeabur.yaml` one-click template, deploy directly from the repo root with Zeabur's GitHub integration:
1. Fork this repo, or push the code to your own GitHub repository.
2. In Zeabur Dashboard, create a Project, add a Service, then choose a GitHub/Git repository source.
3. Select the repository and branch. Keep Root Directory as `/`.
4. Use the Dockerfile build path. Zeabur auto-detects the repo-root `Dockerfile`; do not set `ZBPACK_IGNORE_DOCKERFILE=true`. If the UI asks for a Dockerfile name, enter `Dockerfile`.
5. Add a persistent volume in the Service settings and mount it at `/data`.
6. Configure environment variables:
| Variable | Recommended value | Description |
| --- | --- | --- |
| `PORT` | `5001` | Service listen port; keep it aligned with the exposed Zeabur HTTP port. |
| `DS2API_ADMIN_KEY` | Strong random string | Required admin login key. |
| `DS2API_CONFIG_PATH` | `/data/config.json` | Recommended persistent config path. |
| `LOG_LEVEL` | `INFO` | Optional log level. |
| `DS2API_CONFIG_JSON` | Raw JSON or Base64 JSON | Optional config bootstrap from env. |
| `DS2API_ENV_WRITEBACK` | `1` | Optional; enable only when using `DS2API_CONFIG_JSON` and you want the initial config written to `/data/config.json`. |
7. Expose HTTP port `5001`. The health check path can be `/healthz`.
8. After deployment, open `/admin`, login with `DS2API_ADMIN_KEY`, then import or edit config in Admin UI. A fresh volume does not need `/data/config.json` up front; the service boots first and creates the file on the first save.
Troubleshooting:
- **Startup log says `open /data/config.json: no such file or directory`**: make sure you deployed a version that includes the fresh-volume bootstrap fix, then redeploy the latest code.
- **`open /app/config.json: permission denied`**: the config path still points at the read-only image directory; mount `/data` and set `DS2API_CONFIG_PATH=/data/config.json`.
- **Config disappears after restart**: check that the `/data` persistent volume is mounted on this service. If you use `DS2API_CONFIG_JSON` but want Admin UI saves persisted, enable `DS2API_ENV_WRITEBACK=1`.
References: Zeabur's official [GitHub/Git integration](https://zeabur.com/docs/en-US/deploy/github), [Dockerfile deployment](https://zeabur.com/docs/en-US/deploy/dockerfile), and [Volumes](https://zeabur.com/docs/data-management/volumes) docs.
---
## 3. Vercel Deployment
@@ -270,6 +302,7 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # optional for personal accounts
| `VERCEL_TOKEN` | Vercel sync token | — |
| `VERCEL_PROJECT_ID` | Vercel project ID | — |
| `VERCEL_TEAM_ID` | Vercel team ID | — |
| `DS2API_CHAT_HISTORY_PATH` | Chat history storage path (must be set to `/tmp/chat_history.json` on Vercel, otherwise unavailable due to read-only filesystem) | `data/chat_history.json` |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Deployment protection bypass for internal Node→Go calls | — |
### 3.4 Vercel Architecture
@@ -359,6 +392,22 @@ If API responses return Vercel HTML `Authentication Required`:
- **Option B**: Add `x-vercel-protection-bypass` header to requests
- **Option C**: Set `VERCEL_AUTOMATION_BYPASS_SECRET` (or `DS2API_VERCEL_PROTECTION_BYPASS`) for internal Node→Go calls
#### Chat History Unavailable (read-only file system)
```text
create chat history dir: mkdir /var/task/data: read-only file system
```
**Cause**: Vercel Serverless functions have a read-only filesystem (`/var/task`). Chat history fails because it cannot create directories there.
**Fix**: Add the following in Vercel Project Settings → Environment Variables:
```text
DS2API_CHAT_HISTORY_PATH=/tmp/chat_history.json
```
`/tmp` is the only writable directory in Vercel Serverless. Data is ephemeral (not persisted across cold starts), but the feature works within a single instance lifetime.
### 3.6 Build Artifacts Not Committed
- `static/admin` directory is not in Git
@@ -401,7 +450,7 @@ Or step by step:
```bash
cd webui
npm install
npm ci
npm run build
# Output goes to static/admin/
```

View File

@@ -64,8 +64,8 @@ cp config.example.json config.json
仓库内置 GitHub Actions 工作流:`.github/workflows/release-artifacts.yml`
- **触发条件**:仅在 Release `published`触发(普通 push 不会构建)
- **构建产物**:多平台二进制压缩包 + `sha256sums.txt`
- **触发条件**默认仅在 Release `published`自动触发;也支持在 Actions 页面手动 `workflow_dispatch`,并填写 `release_tag` 复跑/补发
- **构建产物**:多平台二进制压缩包、Linux Docker 镜像导出包 + `sha256sums.txt`
- **容器镜像发布**:仅发布到 GHCR`ghcr.io/cjackhwang/ds2api`
| 平台 | 架构 | 文件格式 |
@@ -131,6 +131,7 @@ docker-compose logs -f
默认 `docker-compose.yml` 直接使用 `ghcr.io/cjackhwang/ds2api:latest`,并把宿主机 `6011` 映射到容器内的 `5001`。如果你希望直接对外暴露 `5001`,请设置 `DS2API_HOST_PORT=5001`(或者手动调整 `ports` 配置)。
Compose 模板还会默认设置 `DS2API_CONFIG_PATH=/data/config.json` 并挂载 `./config.json:/data/config.json`,优先避免 `/app` 只读带来的配置持久化问题。
镜像内会预创建 `/data` 并授权给非 root 的 `ds2api` 用户;如果你使用 bind mount 单文件,请确保宿主机 `config.json` 至少可被容器用户读取/写入,例如 `chmod 644 config.json`,否则 Linux UID/GID 不一致时仍可能出现 `open /data/config.json: permission denied`
兼容说明:若未设置 `DS2API_CONFIG_PATH` 且运行目录是 `/app`,新版本会优先使用 `/data/config.json`;当该文件不存在但检测到历史 `/app/config.json` 时,会自动回退读取旧路径,避免升级后“配置丢失”。
如需固定版本,也可以直接拉取指定 tag
@@ -196,7 +197,7 @@ healthcheck:
部署要点:
- **端口**:服务默认监听 `5001`,模板会固定设置 `PORT=5001`
- **配置持久化**:模板挂载卷 `/data`,并设置 `DS2API_CONFIG_PATH=/data/config.json`;在管理台导入配置后,会写入并持久化到该路径。
- **配置持久化**:模板挂载卷 `/data`,并设置 `DS2API_CONFIG_PATH=/data/config.json`首次空卷启动时会先使用空的文件模式配置,在管理台导入配置后,会写入并持久化到该路径。
- **`open /app/config.json: permission denied`**:说明当前实例在尝试把运行时 token 持久化到只读路径(常见于镜像内 `/app`)。
处理建议:
1. 显式设置可写路径:`DS2API_CONFIG_PATH=/data/config.json`(并挂载持久卷到 `/data`
@@ -205,6 +206,37 @@ healthcheck:
- **构建版本号**Zeabur / 普通 `docker build` 默认不需要传 `BUILD_VERSION`;镜像会优先使用该构建参数,未提供时自动回退到仓库根目录的 `VERSION` 文件。
- **首次登录**:部署完成后访问 `/admin`,使用 Zeabur 环境变量/模板指引中的 `DS2API_ADMIN_KEY` 登录(建议首次登录后自行更换为强密码)。
#### 不使用模板手动部署
如果你不想使用 `zeabur.yaml` 一键模板,可以直接用 Zeabur 的 GitHub 集成从仓库根目录构建:
1. Fork 本仓库,或把代码推送到你自己的 GitHub 仓库。
2. 在 Zeabur Dashboard 中创建 Project然后添加 Service选择 GitHub/Git 仓库来源。
3. 选择仓库与分支Root Directory 保持 `/`
4. 构建方式使用 Dockerfile。Zeabur 会自动检测仓库根目录的 `Dockerfile`;不要设置 `ZBPACK_IGNORE_DOCKERFILE=true`。如果界面要求填写 Dockerfile 名称,填写 `Dockerfile`
5. 在 Service 配置中添加持久卷,挂载目录填写 `/data`
6. 配置环境变量:
| 变量 | 推荐值 | 说明 |
| --- | --- | --- |
| `PORT` | `5001` | 服务监听端口,需要和 Zeabur 暴露的 HTTP 端口一致。 |
| `DS2API_ADMIN_KEY` | 强随机字符串 | 管理台登录密钥,必填。 |
| `DS2API_CONFIG_PATH` | `/data/config.json` | 配置持久化路径,建议必填。 |
| `LOG_LEVEL` | `INFO` | 可选,日志级别。 |
| `DS2API_CONFIG_JSON` | 原始 JSON 或 Base64 JSON | 可选,用于用环境变量初始化配置。 |
| `DS2API_ENV_WRITEBACK` | `1` | 可选;当设置了 `DS2API_CONFIG_JSON` 且希望首次启动后写入 `/data/config.json` 时再启用。 |
7. 暴露 HTTP 端口 `5001`,健康检查路径可填 `/healthz`
8. 部署完成后访问 `/admin`,用 `DS2API_ADMIN_KEY` 登录,然后在管理台导入或编辑配置。首次空卷可以没有 `/data/config.json`,服务会先启动,第一次保存时自动创建该文件。
常见问题:
- **启动日志出现 `open /data/config.json: no such file or directory`**:请确认已经部署包含“首次空卷启动”修复的版本,并重新部署最新代码。
- **出现 `open /app/config.json: permission denied`**:说明配置路径仍指向镜像内只读目录;设置持久卷 `/data`,并确认 `DS2API_CONFIG_PATH=/data/config.json`
- **管理台保存后重启配置丢失**:检查 `/data` 持久卷是否已挂载到当前服务;如果使用了 `DS2API_CONFIG_JSON`,但想让管理台保存落盘,请启用 `DS2API_ENV_WRITEBACK=1`
参考Zeabur 官方文档的 [GitHub/Git 集成](https://zeabur.com/docs/en-US/deploy/github)、[Dockerfile 部署](https://zeabur.com/docs/zh-CN/deploy/dockerfile) 与 [Volumes](https://zeabur.com/docs/data-management/volumes)。
---
## 三、Vercel 部署
@@ -270,6 +302,7 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # 个人账号可留空
| `VERCEL_TOKEN` | Vercel 同步 token | — |
| `VERCEL_PROJECT_ID` | Vercel 项目 ID | — |
| `VERCEL_TEAM_ID` | Vercel 团队 ID | — |
| `DS2API_CHAT_HISTORY_PATH` | Chat history 存储路径Vercel 上必须设为 `/tmp/chat_history.json`,否则因文件系统只读而不可用) | `data/chat_history.json` |
| `DS2API_VERCEL_PROTECTION_BYPASS` | 部署保护绕过密钥(内部 Node→Go 调用) | — |
### 3.3 运行时行为配置(通过 Admin API 设置)
@@ -369,6 +402,22 @@ No Output Directory named "public" found after the Build completed.
- **方案 B**:请求中添加 `x-vercel-protection-bypass`
- **方案 C**:设置 `VERCEL_AUTOMATION_BYPASS_SECRET`(或 `DS2API_VERCEL_PROTECTION_BYPASS`),仅影响内部 Node→Go 调用
#### Chat History 不可用read-only file system
```text
create chat history dir: mkdir /var/task/data: read-only file system
```
**原因**Vercel Serverless 函数的文件系统(`/var/task`为只读chat history 尝试在该路径下创建目录失败。
**解决**:在 Vercel Project Settings → Environment Variables 中添加:
```text
DS2API_CHAT_HISTORY_PATH=/tmp/chat_history.json
```
`/tmp` 是 Vercel Serverless 环境中唯一可写的目录。数据在函数冷启动之间不会持久化ephemeral但在单个实例生命周期内功能正常。
### 3.6 仓库不提交构建产物
- `static/admin` 目录不在 Git 中
@@ -411,7 +460,7 @@ go run ./cmd/ds2api
```bash
cd webui
npm install
npm ci
npm run build
# 产物输出到 static/admin/
```

View File

@@ -309,7 +309,18 @@ parse SSE block
- 新模型可能增加新的 `p` 路径。
- 新版本可能增加新的 fragment.type。
- `CONTENT_FILTER` 的终态模板内容可能变化。
- 自动续写相关状态(如 `INCOMPLETE` / `AUTO_CONTINUE`)当前主要来自实测与实现兼容逻辑,后续字段形态仍可能变化。
- 自动续写相关状态(如 `INCOMPLETE` / `AUTO_CONTINUE`)当前主要来自实测与实现兼容逻辑,后续字段形态仍可能变化。当前实现不会仅因早期 `WIP` 状态就自动继续;只有显式 `INCOMPLETE``auto_continue` 信号才会触发 continue。
- 解析器应当对未知字段、未知路径、未知事件保持容忍。
如果你要把这份说明用于实际开发,建议同时保留原始流样本、回放脚本和回归测试,不要只依赖本文。
## 2026-04-29 最近线上样本增量观察
基于 `longtext-deepseek-v4-flash-20260429``longtext-deepseek-v4-pro-20260429` 两个真实账号长文本样本,近期格式变化要点如下:
1. `data:` 事件中仍大量出现 `{"v":"..."}` 的无路径增量(`p` 缺失),解析器必须把空路径视为可见正文候选,而不能只依赖 `response/content`
2. 对象形态 `v`(如 `{"text":"..."}` / `{"content":"..."}`)仍会出现,且可能与无路径 chunk 混用;仅按字符串处理会导致正文丢块。
3. 多轮 continuation 场景下,后续 chunk 可能不再重复显式 `status`,状态机需要保留上一轮 `INCOMPLETE` 语义直到出现终态。
4. 2026-04-29 起客户端头部版本基线上调到 `x-client-version: 2.0.3`,否则部分账号会出现上游行为不一致(包括空输出与补轮异常)。
建议:新增样本默认回放应优先覆盖「长文本 + 多轮 + 无路径 chunk」组合避免只用短样本导致回归漏检。

View File

@@ -22,7 +22,7 @@
### 文档维护约定
- 文档更新必须以实际代码实现为依据:总路由装配看 `internal/server/router.go`,协议/resource 路由看 `internal/httpapi/*/**/routes.go``internal/httpapi/admin/handler.go`,配置默认值看 `internal/config/*`,模型/alias 看 `internal/config/models.go`prompt 兼容链路看 `docs/prompt-compatibility.md` 列出的代码入口。
- 文档更新必须以实际代码实现为依据:总路由装配看 `internal/server/router.go`,协议/resource 路由看 `internal/httpapi/**/handler*.go``internal/httpapi/admin/handler.go`,配置默认值看 `internal/config/*`,模型/alias 看 `internal/config/models.go`prompt 兼容链路看 `docs/prompt-compatibility.md` 列出的代码入口。
- `README.MD` / `README.en.md`:面向首次接触用户,保留“是什么 + 怎么快速跑起来”。
- `docs/ARCHITECTURE*.md`:面向开发者,集中维护项目结构、模块职责与调用链。
- `API*.md`:面向客户端接入者,聚焦接口行为、鉴权和示例。
@@ -53,7 +53,7 @@ Recommended reading order:
### Maintenance conventions
- Documentation updates must be grounded in the actual implementation: root routing lives in `internal/server/router.go`, protocol/resource routes live in `internal/httpapi/*/**/routes.go` and `internal/httpapi/admin/handler.go`, config defaults in `internal/config/*`, models/aliases in `internal/config/models.go`, and the prompt compatibility pipeline in the code entrypoints listed by `docs/prompt-compatibility.md`.
- Documentation updates must be grounded in the actual implementation: root routing lives in `internal/server/router.go`, protocol/resource routes live in `internal/httpapi/**/handler*.go` and `internal/httpapi/admin/handler.go`, config defaults in `internal/config/*`, models/aliases in `internal/config/models.go`, and the prompt compatibility pipeline in the code entrypoints listed by `docs/prompt-compatibility.md`.
- `README.MD` / `README.en.md`: onboarding-oriented (“what + quick start”).
- `docs/ARCHITECTURE*.md`: developer-oriented source of truth for module boundaries and execution flow.
- `API*.md`: integration-oriented behavior/contracts.

View File

@@ -60,11 +60,10 @@ npm run build --prefix webui
./tests/scripts/check-refactor-line-gate.sh
./tests/scripts/check-node-split-syntax.sh
./tests/scripts/check-cross-build.sh
# 历史阶段门禁:阶段 6 手工烟测签字检查(默认读取 plans/stage6-manual-smoke.md
./tests/scripts/check-stage6-manual-smoke.sh
```
说明:`plans/stage6-manual-smoke.md` 已移除,阶段 6 手工烟测不再作为当前 CI 或发布门禁。
### 端到端测试 | End-to-End Tests
```bash

View File

@@ -48,6 +48,8 @@ DS2API 当前的核心思路,不是把客户端传来的 `messages`、`tools`
-> 文件收集 / inline 上传 / current input fileOpenAI 链路)
-> completion payload
-> 下游网页对话接口
-> assistantturn 输出语义归一Go 非流式 + 流式收尾)
-> 各协议 rendererOpenAI / Responses / Claude / Gemini
```
对应的关键代码入口:
@@ -72,6 +74,10 @@ DS2API 当前的核心思路,不是把客户端传来的 `messages`、`tools`
[internal/promptcompat/thinking_injection.go](../internal/promptcompat/thinking_injection.go)
- completion payload
[internal/promptcompat/standard_request.go](../internal/promptcompat/standard_request.go)
- Go 输出侧 assistant turn
[internal/assistantturn/turn.go](../internal/assistantturn/turn.go)
- Go completion runtime
[internal/completionruntime/nonstream.go](../internal/completionruntime/nonstream.go)
## 4. 下游真正收到的东西
@@ -98,13 +104,17 @@ DS2API 当前的核心思路,不是把客户端传来的 `messages`、`tools`
- `prompt` 才是对话上下文主载体。
- `ref_file_ids` 只承载文件引用,不承载普通文本消息。
- `tools` 不会作为“原生工具 schema”直接下发给下游而是被改写进 `prompt`
- 对外返回给客户端的 `prompt_tokens` / `input_tokens` / `promptTokenCount` 不再按“最后一条消息”或字符粗估近似返回,而是基于**完整上下文 prompt**做 tokenizer 计数;为了避免上下文实际超限但客户端误以为还能塞下,请求侧上下文 token 会额外保守上浮一点,宁可略大也不低估。
- 当前 `/v1/chat/completions` 业务路径仍是“每次请求新建一个远端 `chat_session_id`,并默认发送 `parent_message_id: null`”;因此 DS2API 对外默认表现为“新会话 + prompt 拼历史”,而不是复用 DeepSeek 原生会话树。
- 但 DeepSeek 远端本身支持同一 `chat_session_id` 的跨轮次持续对话。2026-04-27 已用项目内现有 DeepSeek client 做过一次不改业务代码的双轮实测:同一 `chat_session_id` 下,第 1 轮返回 `request_message_id=1` / `response_message_id=2` / 文本 `SESSION_TEST_ONE`;第 2 轮重新获取一次 PoW并发送 `parent_message_id=2` 后,成功返回 `request_message_id=3` / `response_message_id=4` / 文本 `SESSION_TEST_TWO`。这说明“同远端会话持续聊天”能力存在,且每轮需要携带正确的 parent/message 链接信息,同时重新获取对应轮次可用的 PoW。
- OpenAI Chat / Responses 原生走统一 OpenAI 标准化与 DeepSeek payload 组装Claude / Gemini 会尽量复用 OpenAI prompt/tool 语义,其中 Gemini 直接复用 `promptcompat.BuildOpenAIPromptForAdapter`Claude 消息接口在可代理场景会转换为 OpenAI chat 形态再执行
- OpenAI Chat / Responses 原生走统一 OpenAI 标准化与 DeepSeek payload 组装Claude / Gemini 会尽量复用 OpenAI prompt/tool 语义,其中 Gemini 直接复用 `promptcompat.BuildOpenAIPromptForAdapter`。Go 主服务新增 `completionruntime` 启动层,统一执行 DeepSeek session/PoW/call输出侧新增 `assistantturn` 语义层:非流式 OpenAI Chat / Responses / Claude / Gemini 会把 DeepSeek SSE 收集结果先归一成同一份 assistant turn再分别渲染成各协议原生外形流式 OpenAI Chat / Responses / Claude / Gemini 继续保持各协议实时 SSE framing但最终收尾的 tool fallback、schema 归一、usage、empty-output / content-filter 错误语义同样由 `assistantturn` 判定。Claude / Gemini 的常规 Go 主路径不再依赖内部 `httptest` 转发到 OpenAI handler`translatorcliproxy` 仍保留用于 Vercel bridge、兼容工具和回归测试
- Vercel Node 流式路径本轮不迁移,仍使用现有 Node bridge / stream-tool-sieve 实现;后续若变更 Node 流式语义,需要按 `assistantturn` 的 Go canonical 输出语义同步对齐。
- 客户端传入的 thinking / reasoning 开关会被归一到下游 `thinking_enabled`。Gemini `generationConfig.thinkingConfig.thinkingBudget` 会翻译成同一套 thinking 开关;关闭时即使上游返回 `response/thinking_content`,兼容层也不会把它当作可见正文输出。若最终解析出的模型名带 `-nothinking` 后缀,则会无条件强制关闭 thinking优先级高于请求体中的 `thinking` / `reasoning` / `reasoning_effort`。Claude surface 在流式请求且未显式声明 `thinking` 时,仍按 Anthropic 语义默认关闭;但在非流式代理场景,兼容层会内部开启一次下游 thinking用于捕获“正文为空、工具调用落在 thinking 里”的情况,随后在回包前剥离用户不可见的 thinking block。
- 对 OpenAI Chat / Responses 的非流式收尾,如果最终可见正文为空,兼容层会优先尝试把思维链中的独立 DSML / XML 工具块当作真实工具调用解析出来。流式链路也会在收尾阶段做同样的 fallback 检测,但不会因为思维链内容去中途拦截或改写流式输出;thinking / reasoning 增量仍按原样先发,只有在结束收尾时才可能补发最终工具调用结果。补发结果会作为本轮 assistant 的结构化 `tool_calls` / `function_call` 输出返回,而不是塞进 `content` 文本;如果客户端没有开启 thinking / reasoning思维链只用于检测不会作为 `reasoning_content` 或可见正文暴露。只有正文为空且思维链里也没有可执行工具调用时,才继续按空回复错误处理。
- 对 OpenAI Chat / Responses 的非流式收尾,如果最终可见正文为空,兼容层会优先尝试把思维链中的独立 DSML / XML 工具块当作真实工具调用解析出来。流式链路也会在收尾阶段做同样的 fallback 检测,但不会因为思维链内容去中途拦截或改写流式输出;真正的工具识别始终基于原始上游文本,而不是基于“已经做过可见输出清洗”的版本,因此即使最终可见层会剥离完整 leaked DSML / XML `tool_calls` wrapper、并抑制全空参数或无效 wrapper 块,也不会影响真实工具调用转成结构化 `tool_calls` / `function_call`。补发结果会作为本轮 assistant 的结构化 `tool_calls` / `function_call` 输出返回,而不是塞进 `content` 文本;如果客户端没有开启 thinking / reasoning思维链只用于检测不会作为 `reasoning_content` 或可见正文暴露。只有正文为空且思维链里也没有可执行工具调用时,才继续按空回复错误处理。
- OpenAI Chat / Responses 的空回复错误处理之前会默认做一次内部补偿重试:第一次上游完整结束后,如果最终可见正文为空、没有解析到工具调用、也没有已经向客户端流式发出工具调用,并且终止原因不是 `content_filter`,兼容层会复用同一个 `chat_session_id`、账号、token 与工具策略,把原始 completion `prompt` 追加固定后缀 `Previous reply had no visible output. Please regenerate the visible final answer or tool call now.` 后重新提交一次。重试遵循 DeepSeek 多轮对话协议:从第一次上游 SSE 流中提取 `response_message_id`,并在重试 payload 中设置 `parent_message_id` 为该值,使重试成为同一会话的后续轮次而非断裂的根消息;同时重新获取一次 PoW若 PoW 获取失败则回退到原始 PoW。该重试不会重新标准化消息、不会新建 session、不会切换账号也不会向流式客户端插入重试标记第二次 thinking / reasoning 会按正常增量直接接到第一次之后,并继续使用 overlap trim 去重。若第二次仍为空,终端错误码仍保持现有 `upstream_empty_output`;若任一尝试触发空 `content_filter`,不做补偿重试并保持 `content_filter` 错误。JS Vercel 运行时同样设置 `parent_message_id`,但因无法直接调用 PoW API 而复用原始 PoW。
- OpenAI Chat / Responses 在最终可见正文渲染阶段,会把 DeepSeek 搜索返回中的 `[citation:N]` / `[reference:N]` 标记替换成对应 Markdown 链接。`citation` 标记按一基序号解析;`reference` 标记只有在同一段正文中出现 `[reference:0]`(允许冒号后有空格)时才按零基序号映射,并且不会影响同段正文里的 `citation` 标记。
## 5. prompt 是怎么拼出来的
OpenAI Chat / Responses 在标准化后、current input file 之前,会默认执行 `thinking_injection` 增强。它参考 DeepSeek V4 “把控制指令放在 user 消息末尾更稳定”的用法,在最新 user message 后追加思考增强提示词。当前内置默认提示词以 `Reasoning Effort: Absolute maximum with no shortcuts permitted.` 开头,并继续要求模型充分分解问题、覆盖潜在路径与边界条件、把完整推演过程显式写出。该开关默认启用,可通过 `thinking_injection.enabled=false` 关闭;也可以通过 `thinking_injection.prompt` 自定义提示词,留空时使用内置默认提示词。
@@ -114,6 +124,11 @@ OpenAI Chat / Responses 在标准化后、current input file 之前,会默认
- 普通请求会直接出现在最终 `prompt` 的最新 user block 末尾。
- 如果触发 current input file它会进入完整上下文文件中。
另外,`MessagesPrepareWithThinking` 还会在最终 prompt 的最前面预置一段固定的 system 级“输出完整性约束Output integrity guard
- 如果上游上下文、工具输出或解析后的文本出现乱码、损坏、部分解析、重复或其他畸形片段,不要模仿、不要回显,只输出给用户的正确内容。
- 这段约束位于普通 system / tool prompt 之前,因此是当前最终 prompt 里的最高优先级前置指令。
### 5.1 角色标记
最终 prompt 使用 DeepSeek 风格角色标记:
@@ -152,11 +167,13 @@ OpenAI Chat / Responses 在标准化后、current input file 之前,会默认
工具调用正例现在优先示范官方 DSML 风格:`<|DSML|tool_calls>``<|DSML|invoke name="...">``<|DSML|parameter name="...">`
兼容层仍接受旧式纯 `<tool_calls>` wrapper但提示词会优先要求模型输出官方 DSML 标签,并强调不能只输出 closing wrapper 而漏掉 opening tag。需要注意这是“兼容 DSML 外壳,内部仍以 XML 解析语义为准”,不是原生 DSML 全链路实现DSML 标签会在解析入口归一化回现有 XML 标签后继续走同一套 parser。
数组参数使用 `<item>...</item>` 子节点表示;当某个参数体只包含 item 子节点时Go / Node 解析器会把它还原成数组,避免 `questions` / `options` 这类 schema 中要求 array 的参数被误解析成 `{ "item": ... }` 对象。若模型把完整结构化 XML fragment 误包进 CDATA兼容层会在保护 `content` / `command` 等原文字段的前提下,尝试把非原文字段中的 CDATA XML fragment 还原成 object / array。不过如果 CDATA 只是单个平面的 XML/HTML 标签,例如 `<b>urgent</b>` 这种行内标记,兼容层会保留原始字符串,不会强行升成 object / array只有明显表示结构的 CDATA 片段,例如多兄弟节点、嵌套子节点或 `item` 列表,才会触发结构化恢复。
数组参数使用 `<item>...</item>` 子节点表示;当某个参数体只包含 item 子节点时Go / Node 解析器会把它还原成数组,避免 `questions` / `options` 这类 schema 中要求 array 的参数被误解析成 `{ "item": ... }` 对象。除此之外,解析器还会回收一些更松散的列表写法,例如 JSON array 字面量或逗号分隔的 JSON 项序列,只要它们足够明确;但 `<item>` 仍然是首选形态。若模型把完整结构化 XML fragment 误包进 CDATA兼容层会在保护 `content` / `command` 等原文字段的前提下,尝试把非原文字段中的 CDATA XML fragment 还原成 object / array。不过如果 CDATA 只是单个平面的 XML/HTML 标签,例如 `<b>urgent</b>` 这种行内标记,兼容层会保留原始字符串,不会强行升成 object / array只有明显表示结构的 CDATA 片段,例如多兄弟节点、嵌套子节点或 `item` 列表,才会触发结构化恢复。
Go 侧读取 DeepSeek SSE 时不再依赖 `bufio.Scanner` 的固定 2MiB 单行上限;当写文件类工具把很长的 `content` 放在单个 `data:` 行里返回时,非流式收集、流式解析和 auto-continue 透传都会保留完整行,再进入同一套工具解析与序列化流程。
在 assistant 最终回包阶段,如果某个 tool 参数在声明 schema 中明确是 `string`,兼容层会在把解析后的 `tool_calls` / `function_call` 重新序列化成 OpenAI / Responses / Claude 可见参数前,递归把该路径上的 number / bool / object / array 统一转成字符串;其中 object / array 会压成紧凑 JSON 字符串。这个保护只对 schema 明确声明为 string 的路径生效,不会改写本来就是 `number` / `boolean` / `object` / `array` 的参数。这样可以兼容 DeepSeek 输出了结构化片段、但上游客户端工具 schema 又严格要求字符串参数的场景(例如 `content``prompt``path``taskId` 等)。
工具 schema 的权威来源始终是**当前请求实际携带的 schema**,而不是同名工具在其他 runtimeClaude Code / OpenCode / Codex 等)里的默认印象。兼容层现在会同时兼容 OpenAI 风格 `function.parameters`、直接工具对象上的 `parameters` / `input_schema`、以及 camelCase 的 `inputSchema` / `schema`,并在最终输出阶段按这份请求内 schema 决定是保留 array/object还是仅对明确声明为 `string` 的路径做字符串化。该规则同样适用于 Claude 的流式收尾和 Vercel Node 流式 tool-call formatter避免不同 runtime 因 schema shape 差异而出现同名工具参数类型漂移。
正例中的工具名只会来自当前请求实际声明的工具;如果当前请求没有足够的已知工具形态,就省略对应的单工具、多工具或嵌套示例,避免把不可用工具名写进 prompt。
对执行类工具,脚本内容必须进入执行参数本身:`Bash` / `execute_command` 使用 `command``exec_command` 使用 `cmd`;不要把脚本示范成 `path` / `content` 文件写入参数。
如果当前请求声明了 `Read` / `read_file` 这类读取工具,兼容层会额外注入一条 read-tool cache guard当读取结果只表示“文件未变更 / 已在历史中 / 请引用先前上下文 / 没有正文内容”时模型必须把它视为内容不可用不能反复调用同一个无正文读取应改为请求完整正文读取能力或向用户说明需要重新提供文件内容。这个约束只缓解客户端缓存返回空内容导致的死循环DS2API 不会也无法凭空恢复客户端本地文件正文。
OpenAI 路径实现:
[internal/promptcompat/tool_prompt.go](../internal/promptcompat/tool_prompt.go)
@@ -233,6 +250,14 @@ OpenAI 文件相关实现:
- 文件 ID 收集:
[internal/promptcompat/file_refs.go](../internal/promptcompat/file_refs.go)
OpenAI 的文件上传现在不再是“只传文件本体”的通用路径,而是会先根据请求里的 `model` 解析出 DeepSeek 的上传类型,并把它透传到上传接口的 `x-model-type`。当前可见的上传类型就是 `default` / `expert` / `vision`,其中 vision 请求上传图片时必须带上 `vision`,否则下游容易退回到仅文本或 OCR 语义。这个模型类型会同时用于:
- `/v1/files` 这类独立文件上传入口
- Chat / Responses 的 inline 图片、附件上传
- current input file 触发时生成的 `DS2API_HISTORY.txt` 上下文文件
也就是说,文件上传和完成请求的 `model_type` 现在是一致的:完成 payload 里仍然是 `model_type`,上传文件则会在 DeepSeek 上传阶段携带同样的模型类型信息。
结论:
- “systemprompt 文字”在 prompt 里
@@ -244,9 +269,10 @@ OpenAI 文件相关实现:
兼容层现在只保留 `current_input_file` 这一种拆分方式;旧的 `history_split` 已废弃,只保留为兼容旧配置的字段,不再参与请求处理。
- `current_input_file` 默认开启;它用于把“完整上下文”合并进隐藏上下文文件。当最新 user turn 的纯文本长度达到 `current_input_file.min_chars`(默认 `0`)时,兼容层会上传一个文件名为 `IGNORE.txt` 的上下文文件,并在 live prompt 中只保留一个中性的 user 消息要求模型直接回答最新请求,不再暴露文件名或要求模型读取本地文件
- `current_input_file` 默认开启;它用于把“完整上下文”合并进 `DS2API_HISTORY.txt` 上下文文件。当最新 user turn 的纯文本长度达到 `current_input_file.min_chars`(默认 `0`)时,兼容层会上传一个文件名为 `DS2API_HISTORY.txt` 的上下文文件。文件内容会先做 OpenAI 消息标准化,再序列化成按轮次编号的 `DS2API_HISTORY.txt` 风格 transcript带有 `# DS2API_HISTORY.txt` 标题和 `=== N. ROLE ===` 分段live prompt 中则会给出一个 continuation 语气的 user 消息,引导模型从 `DS2API_HISTORY.txt` 的最新状态继续推进,并直接回答最新请求,避免把任务拉回起点
- 如果 `current_input_file.enabled=false`,请求会直接透传,不上传任何拆分上下文文件。
- 旧的 `history_split.enabled` / `history_split.trigger_after_turns` 会被读取进配置对象以保持兼容,但不会触发拆分上传,也不会影响 `current_input_file` 的默认开启。
- 即使触发 `current_input_file` 后 live prompt 被缩短,对客户端回包里的上下文 token 统计,仍会沿用**拆分前的完整 prompt 语义**做计数,而不是按缩短后的占位 prompt 计算;否则会把真实上下文显著算小。
相关实现:
@@ -257,16 +283,24 @@ OpenAI 文件相关实现:
- 旧历史拆分兼容壳:
[internal/httpapi/openai/history/history_split.go](../internal/httpapi/openai/history/history_split.go)
当前输入转文件启用并触发时,上传文件的真实文件名是 `IGNORE.txt`,文件内容是完整 `messages` 上下文;它仍会先用 OpenAI 消息标准化和 DeepSeek 角色标记序列化,再包进 `IGNORE` 文件边界
当前输入转文件启用并触发时,上传文件的真实文件名是 `DS2API_HISTORY.txt`,文件内容是完整 `messages` 上下文;它仍会先用 OpenAI 消息标准化和 DeepSeek 角色标记序列化,再按轮次编号成 `DS2API_HISTORY.txt` 风格的 transcript不再注入文件边界标签)
```text
[uploaded filename]: IGNORE.txt
[file content end]
[uploaded filename]: DS2API_HISTORY.txt
# DS2API_HISTORY.txt
Prior conversation history and tool progress.
<begin▁of▁sentence><System>...<User>...<Assistant>...<Tool>...<User>...
=== 1. SYSTEM ===
...
[file name]: IGNORE
[file content begin]
=== 2. USER ===
...
=== 3. ASSISTANT ===
...
=== 4. TOOL ===
...
```
开启后,请求的 live prompt 不再直接内联完整上下文,而是保留一个 user role 的短提示,提示模型基于已提供上下文直接回答最新请求;上传后的 `file_id` 会进入 `ref_file_ids`
@@ -318,7 +352,7 @@ OpenAI 文件相关实现:
```json
{
"prompt": "<begin▁of▁sentence><System>原 system / developer\n\nYou have access to these tools: ...<end▁of▁instructions><User>The current request and prior conversation context have already been provided. Answer the latest user request directly.<Assistant>",
"prompt": "<begin▁of▁sentence><System>原 system / developer\n\nYou have access to these tools: ...<end▁of▁instructions><User>Continue from the latest state in the attached DS2API_HISTORY.txt context. Treat it as the current working state and answer the latest user request directly.<Assistant>",
"ref_file_ids": [
"file-current-input-ignore",
"file-systemprompt",
@@ -333,7 +367,7 @@ OpenAI 文件相关实现:
- 大部分结构化语义被压进 `prompt`
- 文件保持文件
- 需要时把完整上下文拆进隐藏上下文文件
- 需要时把完整上下文拆进 `DS2API_HISTORY.txt` 上下文文件,并按轮次编号成 transcript
## 12. 修改时必须同步本文档的场景
@@ -346,7 +380,7 @@ OpenAI 文件相关实现:
- tool result 注入方式变更
- tool prompt 模板或 tool_choice 约束变更
- inline 文件上传 / 文件引用收集规则变更
- current input file 触发条件、上传格式、`IGNORE` 包装格式变更
- current input file 触发条件、上传格式、`DS2API_HISTORY.txt` transcript 结构变更
-`history_split` 兼容逻辑的读取、忽略或退化行为变更
- completion payload 字段语义变更
- Claude / Gemini 对这套统一语义的复用关系变更

View File

@@ -26,7 +26,7 @@
</tool_calls>
```
这不是原生 DSML 全链路实现。DSML 只作为 prompt 外壳和解析入口别名;进入 parser 前会归一化成 `<tool_calls>` / `<invoke>` / `<parameter>`,内部仍以现有 XML 解析语义为准。
这不是原生 DSML 全链路实现。DSML 主要用于让模型有意识地输出协议标识,隔离普通 XML 语义;进入 parser 前会按固定本地标签名归一化成 `<tool_calls>` / `<invoke>` / `<parameter>`,内部仍以现有 XML 解析语义为准。
约束:
@@ -39,7 +39,8 @@
兼容修复:
- 如果模型漏掉 opening wrapper但后面仍输出了一个或多个 invoke 并以 closing wrapper 收尾Go 解析链路会在解析前补回缺失的 opening wrapper。
- 如果模型把 DSML 标签里的分隔符 `|` 写漏成空格(例如 `<|DSML tool_calls>` / `<|DSML invoke>` / `<|DSML parameter>`,或无 leading pipe 的 `<DSML tool_calls>` 形态),或把 `DSML` 与工具标签名直接黏连(例如 `<DSMLtool_calls>` / `<DSMLinvoke>` / `<DSMLparameter>`),或把最前面的 pipe 误写成全宽竖线(例如 `<DSML|tool_calls>` / `<DSML|invoke>` / `<DSML|parameter>`Go / Node 会在固定工具标签名范围内归一化;相似但非工具标签名(如 `tool_calls_extra`)仍按普通文本处理。
- Go / Node 解析层不再枚举每一种 DSML typo。它会把工具标签名前的 `DSML`、管道符 `|` / ``、空白、重复 leading `<` 视为可容忍的协议噪声,然后只匹配固定本地标签名 `tool_calls` / `invoke` / `parameter`。例如 `<DSML|tool_calls>``<<|DSML|tool_calls>``<|DSML tool_calls>``<DSMLtool_calls>``<<DSML|DSML|tool_calls>` 都会归一化;相似但非固定标签名(如 `tool_calls_extra`)仍按普通文本处理。
- 如果模型在固定工具标签名后多输出一个尾部管道符,例如 `<|DSML|tool_calls|` / `<|DSML|invoke|` / `<|DSML|parameter|`,兼容层会把这个尾部 `|` 当作异常标签终止符并补齐缺失的 `>`;如果后面已经有 `>`,也会消费这个多余 `|` 后再归一化。
- 这是一个针对常见模型失误的窄修复不改变推荐输出格式prompt 仍要求模型直接输出完整 DSML 外壳。
-`<invoke ...>` / `<parameter ...>` 不会被当成“已支持的工具语法”;只有 `tool_calls` wrapper 或可修复的缺失 opening wrapper 才会进入工具调用路径。
@@ -53,7 +54,7 @@
在流式链路中Go / Node 一致):
- DSML `<|DSML|tool_calls>` wrapper、兼容变体(`<dsml|tool_calls>``<tool_calls>``<|tool_calls>``<DSML|tool_calls>`)、窄容错空格分隔形态(如 `<|DSML tool_calls>`)、黏连形态(如 `<DSMLtool_calls>`)和 canonical `<tool_calls>` wrapper 都会进入结构化捕获
- DSML `<|DSML|tool_calls>` wrapper、基于固定本地标签名的 DSML 噪声容错形态、尾部管道符形态(如 `<|DSML|tool_calls|`)和 canonical `<tool_calls>` wrapper 都会进入结构化捕获
- 如果流里直接从 invoke 开始,但后面补上了 closing wrapperGo 流式筛分也会按缺失 opening wrapper 的修复路径尝试恢复
- 已识别成功的工具调用不会再次回流到普通文本
- 不符合新格式的块不会执行,并继续按原样文本透传
@@ -61,6 +62,7 @@
- 支持嵌套围栏(如 4 反引号嵌套 3 反引号)和 CDATA 内围栏保护
- 如果模型把 `<![CDATA[` 打开后却没有闭合,流式扫描阶段仍会保守地继续缓冲,不会误把 CDATA 里的示例 XML 当成真实工具调用;在最终 parse / flush 恢复阶段,会对这类 loose CDATA 做窄修复,尽量保住外层已完整包裹的真实工具调用
- 当文本中 mention 了某种标签名(如 `<dsml|tool_calls>` 或 Markdown inline code 里的 `<|DSML|tool_calls>`而后面紧跟真正工具调用时sieve 会跳过不可解析的 mention 候选并继续匹配后续真实工具块,不会因 mention 导致工具调用丢失,也不会截断 mention 后的正文
- Go 侧 SSE 读取不再使用 `bufio.Scanner` 的固定 token 上限;单个 `data:` 行中包含很长的写文件参数时,非流式收集、流式解析与 auto-continue 透传都应保留完整行,再交给 tool parser 处理
另外,`<parameter>` 的值如果本身是合法 JSON 字面量,也会按结构化值解析,而不是一律保留为字符串。例如 `123``true``null``[1,2]``{"a":1}` 都会还原成对应的 number / boolean / null / array / object。
结构化 XML 参数也会还原为 JSON 结构:如果参数体只包含一个或多个 `<item>...</item>` 子节点,会输出数组;嵌套对象里的 item-only 字段也同样按数组处理。例如 `<parameter name="questions"><item><question>...</question></item></parameter>` 会输出 `{"questions":[{"question":"..."}]}`,而不是 `{"questions":{"item":...}}`
@@ -94,7 +96,7 @@ node --test tests/node/stream-tool-sieve.test.js
- DSML `<|DSML|tool_calls>` wrapper 正常解析
- legacy canonical `<tool_calls>` wrapper 正常解析
- 别名变体(`<dsml|tool_calls>``<tool_calls>``<|tool_calls>`、DSML 空格分隔 typo`<|DSML tool_calls>`)和黏连 typo`<DSMLtool_calls>`)正常解析
- 固定本地标签名的 DSML 噪声容错形态(如 `<DSML|tool_calls>``<<|DSML|tool_calls>``<|DSML tool_calls>``<DSMLtool_calls>``<<DSML|DSML|tool_calls>`)正常解析
- 混搭标签DSML wrapper + canonical inner归一化后正常解析
- 波浪线围栏 `~~~` 内的示例不执行
- 嵌套围栏4 反引号嵌套 3 反引号)内的示例不执行

3
go.mod
View File

@@ -6,10 +6,13 @@ require (
github.com/andybalholm/brotli v1.2.1
github.com/go-chi/chi/v5 v5.2.5
github.com/google/uuid v1.6.0
github.com/hupe1980/go-tiktoken v0.0.10
github.com/refraction-networking/utls v1.8.2
github.com/router-for-me/CLIProxyAPI/v6 v6.9.14
)
require github.com/dlclark/regexp2 v1.11.5 // indirect
require (
github.com/klauspost/compress v1.18.5 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect

6
go.sum
View File

@@ -2,10 +2,14 @@ github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eT
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hupe1980/go-tiktoken v0.0.10 h1:m6phOJaGyctqWdGIgwn9X8AfJvaG74tnQoDL+ntOUEQ=
github.com/hupe1980/go-tiktoken v0.0.10/go.mod h1:NME6d8hrE+Jo+kLUZHhXShYV8e40hYkm4BbSLQKtvAo=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -37,6 +41,8 @@ golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

View File

@@ -0,0 +1,64 @@
package assistantturn
import (
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/sse"
)
type StreamEventType string
const (
StreamEventTextDelta StreamEventType = "text_delta"
StreamEventThinkingDelta StreamEventType = "thinking_delta"
StreamEventToolCall StreamEventType = "tool_call"
StreamEventDone StreamEventType = "done"
StreamEventError StreamEventType = "error"
StreamEventPing StreamEventType = "ping"
)
type StreamEvent struct {
Type StreamEventType
Text string
Thinking string
ToolCall any
Error *OutputError
Usage *Usage
}
type Accumulator struct {
inner shared.StreamAccumulator
}
type AccumulatorOptions struct {
ThinkingEnabled bool
SearchEnabled bool
StripReferenceMarkers bool
}
func NewAccumulator(opts AccumulatorOptions) *Accumulator {
return &Accumulator{
inner: shared.StreamAccumulator{
ThinkingEnabled: opts.ThinkingEnabled,
SearchEnabled: opts.SearchEnabled,
StripReferenceMarkers: opts.StripReferenceMarkers,
},
}
}
func (a *Accumulator) Apply(parsed sse.LineResult) shared.StreamAccumulatorResult {
if a == nil {
return shared.StreamAccumulatorResult{}
}
return a.inner.Apply(parsed)
}
func (a *Accumulator) Snapshot() (rawText, text, rawThinking, thinking, detectionThinking string) {
if a == nil {
return "", "", "", "", ""
}
return a.inner.RawText.String(),
a.inner.Text.String(),
a.inner.RawThinking.String(),
a.inner.Thinking.String(),
a.inner.ToolDetectionThinking.String()
}

View File

@@ -0,0 +1,227 @@
package assistantturn
import (
"net/http"
"strings"
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
"ds2api/internal/toolcall"
"ds2api/internal/util"
)
type StopReason string
const (
StopReasonStop StopReason = "stop"
StopReasonToolCalls StopReason = "tool_calls"
StopReasonContentFilter StopReason = "content_filter"
StopReasonError StopReason = "error"
)
type Usage struct {
InputTokens int
OutputTokens int
ReasoningTokens int
TotalTokens int
}
type OutputError struct {
Status int
Message string
Code string
}
type Turn struct {
Model string
Prompt string
RawText string
RawThinking string
DetectionThinking string
Text string
Thinking string
ToolCalls []toolcall.ParsedToolCall
ParsedToolCalls toolcall.ToolCallParseResult
CitationLinks map[int]string
ContentFilter bool
ResponseMessageID int
StopReason StopReason
Usage Usage
Error *OutputError
}
type BuildOptions struct {
Model string
Prompt string
RefFileTokens int
SearchEnabled bool
StripReferenceMarkers bool
ToolNames []string
ToolsRaw any
ToolChoice promptcompat.ToolChoicePolicy
}
type StreamSnapshot struct {
RawText string
VisibleText string
RawThinking string
VisibleThinking string
DetectionThinking string
ContentFilter bool
CitationLinks map[int]string
ResponseMessageID int
AlreadyEmittedCalls bool
AdditionalToolCalls []toolcall.ParsedToolCall
AlreadyEmittedToolRaw bool
}
func BuildTurnFromCollected(result sse.CollectResult, opts BuildOptions) Turn {
thinking := shared.CleanVisibleOutput(result.Thinking, opts.StripReferenceMarkers)
text := shared.CleanVisibleOutput(result.Text, opts.StripReferenceMarkers)
if opts.SearchEnabled {
text = shared.ReplaceCitationMarkersWithLinks(text, result.CitationLinks)
}
parsed := shared.DetectAssistantToolCalls(result.Text, text, result.Thinking, result.ToolDetectionThinking, opts.ToolNames)
calls := toolcall.NormalizeParsedToolCallsForSchemas(parsed.Calls, opts.ToolsRaw)
parsed.Calls = calls
stopReason := StopReasonStop
if result.ContentFilter {
stopReason = StopReasonContentFilter
}
if len(calls) > 0 {
stopReason = StopReasonToolCalls
}
turn := Turn{
Model: opts.Model,
Prompt: opts.Prompt,
RawText: result.Text,
RawThinking: result.Thinking,
DetectionThinking: result.ToolDetectionThinking,
Text: text,
Thinking: thinking,
ToolCalls: calls,
ParsedToolCalls: parsed,
CitationLinks: result.CitationLinks,
ContentFilter: result.ContentFilter,
ResponseMessageID: result.ResponseMessageID,
StopReason: stopReason,
}
turn.Usage = BuildUsage(opts.Model, opts.Prompt, thinking, text, opts.RefFileTokens)
turn.Error = ValidateTurn(turn, opts.ToolChoice)
if turn.Error != nil {
turn.StopReason = StopReasonError
}
return turn
}
func BuildTurnFromStreamSnapshot(snapshot StreamSnapshot, opts BuildOptions) Turn {
thinking := shared.CleanVisibleOutput(snapshot.VisibleThinking, opts.StripReferenceMarkers)
text := shared.CleanVisibleOutput(snapshot.VisibleText, opts.StripReferenceMarkers)
if opts.SearchEnabled {
text = shared.ReplaceCitationMarkersWithLinks(text, snapshot.CitationLinks)
}
parsed := shared.DetectAssistantToolCalls(snapshot.RawText, text, snapshot.RawThinking, snapshot.DetectionThinking, opts.ToolNames)
calls := parsed.Calls
if len(calls) == 0 && len(snapshot.AdditionalToolCalls) > 0 {
calls = snapshot.AdditionalToolCalls
}
calls = toolcall.NormalizeParsedToolCallsForSchemas(calls, opts.ToolsRaw)
parsed.Calls = calls
stopReason := StopReasonStop
if snapshot.ContentFilter {
stopReason = StopReasonContentFilter
}
if len(calls) > 0 || snapshot.AlreadyEmittedCalls || snapshot.AlreadyEmittedToolRaw {
stopReason = StopReasonToolCalls
}
turn := Turn{
Model: opts.Model,
Prompt: opts.Prompt,
RawText: snapshot.RawText,
RawThinking: snapshot.RawThinking,
DetectionThinking: snapshot.DetectionThinking,
Text: text,
Thinking: thinking,
ToolCalls: calls,
ParsedToolCalls: parsed,
CitationLinks: snapshot.CitationLinks,
ContentFilter: snapshot.ContentFilter,
ResponseMessageID: snapshot.ResponseMessageID,
StopReason: stopReason,
}
turn.Usage = BuildUsage(opts.Model, opts.Prompt, thinking, text, opts.RefFileTokens)
if !snapshot.AlreadyEmittedCalls && !snapshot.AlreadyEmittedToolRaw {
turn.Error = ValidateTurn(turn, opts.ToolChoice)
}
if turn.Error != nil && len(calls) == 0 {
turn.StopReason = StopReasonError
}
return turn
}
func BuildUsage(model, prompt, thinking, text string, refFileTokens int) Usage {
inputTokens := util.CountPromptTokens(prompt, model) + refFileTokens
reasoningTokens := util.CountOutputTokens(thinking, model)
outputTokens := reasoningTokens + util.CountOutputTokens(text, model)
return Usage{
InputTokens: inputTokens,
OutputTokens: outputTokens,
ReasoningTokens: reasoningTokens,
TotalTokens: inputTokens + outputTokens,
}
}
func ValidateTurn(turn Turn, policy promptcompat.ToolChoicePolicy) *OutputError {
if policy.IsRequired() && len(turn.ToolCalls) == 0 {
return &OutputError{
Status: http.StatusUnprocessableEntity,
Message: "tool_choice requires at least one valid tool call.",
Code: "tool_choice_violation",
}
}
if len(turn.ToolCalls) > 0 {
return nil
}
if strings.TrimSpace(turn.Text) != "" {
return nil
}
status, message, code := UpstreamEmptyOutputDetail(turn.ContentFilter, turn.Text, turn.Thinking)
return &OutputError{Status: status, Message: message, Code: code}
}
func UpstreamEmptyOutputDetail(contentFilter bool, text, thinking string) (int, string, string) {
_ = text
if contentFilter {
return http.StatusBadRequest, "Upstream content filtered the response and returned no output.", "content_filter"
}
if strings.TrimSpace(thinking) != "" {
return http.StatusTooManyRequests, "Upstream account hit a rate limit and returned reasoning without visible output.", "upstream_empty_output"
}
return http.StatusTooManyRequests, "Upstream account hit a rate limit and returned empty output.", "upstream_empty_output"
}
func ShouldRetryEmptyOutput(turn Turn, attempts, maxAttempts int) bool {
return attempts < maxAttempts &&
!turn.ContentFilter &&
len(turn.ToolCalls) == 0 &&
strings.TrimSpace(turn.Text) == "" &&
strings.TrimSpace(turn.Thinking) == ""
}
func FinishReason(turn Turn) string {
switch turn.StopReason {
case StopReasonToolCalls:
return "tool_calls"
case StopReasonContentFilter:
return "content_filter"
default:
return "stop"
}
}

View File

@@ -0,0 +1,100 @@
package assistantturn
import (
"testing"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
)
func TestBuildTurnFromCollectedTextCitation(t *testing.T) {
turn := BuildTurnFromCollected(sse.CollectResult{
Text: "See [citation:1]",
CitationLinks: map[int]string{1: "https://example.com"},
}, BuildOptions{Model: "deepseek-v4-flash", Prompt: "prompt", SearchEnabled: true, StripReferenceMarkers: true})
if turn.Text != "See [1](https://example.com)" {
t.Fatalf("text mismatch: %q", turn.Text)
}
if turn.StopReason != StopReasonStop {
t.Fatalf("stop reason mismatch: %q", turn.StopReason)
}
if turn.Error != nil {
t.Fatalf("unexpected error: %#v", turn.Error)
}
}
func TestBuildTurnFromCollectedToolCall(t *testing.T) {
turn := BuildTurnFromCollected(sse.CollectResult{
Text: `<tool_calls><invoke name="Write"><parameter name="content">{"x":1}</parameter></invoke></tool_calls>`,
}, BuildOptions{
ToolNames: []string{"Write"},
ToolsRaw: []any{map[string]any{
"name": "Write",
"input_schema": map[string]any{
"type": "object",
"properties": map[string]any{
"content": map[string]any{"type": "string"},
},
},
}},
})
if len(turn.ToolCalls) != 1 {
t.Fatalf("expected one tool call, got %d", len(turn.ToolCalls))
}
if turn.StopReason != StopReasonToolCalls {
t.Fatalf("stop reason mismatch: %q", turn.StopReason)
}
if _, ok := turn.ToolCalls[0].Input["content"].(string); !ok {
t.Fatalf("expected content coerced to string, got %#v", turn.ToolCalls[0].Input["content"])
}
}
func TestBuildTurnFromCollectedThinkingOnlyIsEmptyOutput(t *testing.T) {
turn := BuildTurnFromCollected(sse.CollectResult{Thinking: "hidden"}, BuildOptions{})
if turn.Error == nil || turn.Error.Code != "upstream_empty_output" {
t.Fatalf("expected empty output error, got %#v", turn.Error)
}
}
func TestBuildTurnFromCollectedToolChoiceRequired(t *testing.T) {
turn := BuildTurnFromCollected(sse.CollectResult{Text: "hello"}, BuildOptions{
ToolChoice: promptcompat.ToolChoicePolicy{Mode: promptcompat.ToolChoiceRequired},
})
if turn.Error == nil || turn.Error.Code != "tool_choice_violation" {
t.Fatalf("expected tool choice violation, got %#v", turn.Error)
}
}
func TestBuildTurnFromStreamSnapshotUsesVisibleTextAndRawToolDetection(t *testing.T) {
turn := BuildTurnFromStreamSnapshot(StreamSnapshot{
RawText: `<tool_calls><invoke name="Write"><parameter name="content">{"x":1}</parameter></invoke></tool_calls>`,
VisibleText: "",
}, BuildOptions{
ToolNames: []string{"Write"},
ToolsRaw: []any{map[string]any{
"name": "Write",
"schema": map[string]any{
"type": "object",
"properties": map[string]any{
"content": map[string]any{"type": "string"},
},
},
}},
})
if len(turn.ToolCalls) != 1 {
t.Fatalf("expected stream snapshot tool call, got %d", len(turn.ToolCalls))
}
if _, ok := turn.ToolCalls[0].Input["content"].(string); !ok {
t.Fatalf("expected stream snapshot schema coercion, got %#v", turn.ToolCalls[0].Input["content"])
}
}
func TestBuildTurnFromStreamSnapshotAlreadyEmittedToolAvoidsEmptyError(t *testing.T) {
turn := BuildTurnFromStreamSnapshot(StreamSnapshot{AlreadyEmittedCalls: true}, BuildOptions{})
if turn.Error != nil {
t.Fatalf("unexpected empty-output error after emitted tool call: %#v", turn.Error)
}
if turn.StopReason != StopReasonToolCalls {
t.Fatalf("stop reason mismatch: %q", turn.StopReason)
}
}

View File

@@ -14,6 +14,7 @@ import (
"github.com/google/uuid"
"ds2api/internal/config"
"ds2api/internal/util"
)
const (
@@ -309,8 +310,12 @@ func (s *Store) Update(id string, params UpdateParams) (Entry, error) {
if params.Status != "" {
item.Status = params.Status
}
item.ReasoningContent = params.ReasoningContent
item.Content = params.Content
if params.ReasoningContent != "" || item.ReasoningContent == "" {
item.ReasoningContent = params.ReasoningContent
}
if params.Content != "" || item.Content == "" {
item.Content = params.Content
}
item.Error = strings.TrimSpace(params.Error)
item.StatusCode = params.StatusCode
item.ElapsedMs = params.ElapsedMs
@@ -610,8 +615,8 @@ func buildPreview(item Entry) string {
if candidate == "" {
candidate = strings.TrimSpace(item.UserInput)
}
if len(candidate) > defaultPreviewAt {
return candidate[:defaultPreviewAt] + "..."
if truncated, ok := util.TruncateRunes(candidate, defaultPreviewAt); ok {
return truncated + "..."
}
return candidate
}

View File

@@ -8,6 +8,7 @@ import (
"strings"
"sync"
"testing"
"unicode/utf8"
)
func blockDetailDir(t *testing.T, detailDir string) func() {
@@ -105,6 +106,17 @@ func TestStoreCreatesAndPersistsEntries(t *testing.T) {
}
}
func TestBuildPreviewPreservesUTF8MB4Characters(t *testing.T) {
long := strings.Repeat("😀", defaultPreviewAt+1)
preview := buildPreview(Entry{Content: long})
if !utf8.ValidString(preview) {
t.Fatalf("expected valid utf-8 preview, got %q", preview)
}
if preview != strings.Repeat("😀", defaultPreviewAt)+"..." {
t.Fatalf("unexpected preview: %q", preview)
}
}
func TestStoreTrimsToConfiguredLimit(t *testing.T) {
path := filepath.Join(t.TempDir(), "chat_history.json")
store := New(path)
@@ -481,3 +493,112 @@ func TestStoreWritesOnlyChangedDetailFiles(t *testing.T) {
t.Fatalf("expected untouched detail file to remain byte-identical")
}
}
func TestUpdatePreservesContentWhenNewContentIsEmpty(t *testing.T) {
path := filepath.Join(t.TempDir(), "chat_history.json")
store := New(path)
started, err := store.Start(StartParams{
CallerID: "caller:abc",
Model: "deepseek-v4-flash",
Stream: true,
UserInput: "hello",
})
if err != nil {
t.Fatalf("start entry failed: %v", err)
}
if _, err := store.Update(started.ID, UpdateParams{
Status: "streaming",
ReasoningContent: "let me think",
Content: "I'll help you with that.",
}); err != nil {
t.Fatalf("progress update failed: %v", err)
}
updated, err := store.Update(started.ID, UpdateParams{
Status: "success",
Content: "",
Completed: true,
})
if err != nil {
t.Fatalf("success update failed: %v", err)
}
if updated.Content != "I'll help you with that." {
t.Fatalf("expected content to be preserved, got %q", updated.Content)
}
if updated.ReasoningContent != "let me think" {
t.Fatalf("expected reasoning content to be preserved, got %q", updated.ReasoningContent)
}
full, err := store.Get(started.ID)
if err != nil {
t.Fatalf("get entry failed: %v", err)
}
if full.Content != "I'll help you with that." {
t.Fatalf("expected persisted content to be preserved, got %q", full.Content)
}
if full.ReasoningContent != "let me think" {
t.Fatalf("expected persisted reasoning content to be preserved, got %q", full.ReasoningContent)
}
}
func TestUpdateAllowsSettingContentFromEmpty(t *testing.T) {
path := filepath.Join(t.TempDir(), "chat_history.json")
store := New(path)
started, err := store.Start(StartParams{
CallerID: "caller:abc",
Model: "deepseek-v4-flash",
Stream: true,
UserInput: "hello",
})
if err != nil {
t.Fatalf("start entry failed: %v", err)
}
updated, err := store.Update(started.ID, UpdateParams{
Status: "success",
Content: "final answer",
})
if err != nil {
t.Fatalf("update failed: %v", err)
}
if updated.Content != "final answer" {
t.Fatalf("expected content to be set, got %q", updated.Content)
}
}
func TestUpdateAllowsOverwritingContentWithNewValue(t *testing.T) {
path := filepath.Join(t.TempDir(), "chat_history.json")
store := New(path)
started, err := store.Start(StartParams{
CallerID: "caller:abc",
Model: "deepseek-v4-flash",
Stream: true,
UserInput: "hello",
})
if err != nil {
t.Fatalf("start entry failed: %v", err)
}
if _, err := store.Update(started.ID, UpdateParams{
Status: "streaming",
Content: "partial",
}); err != nil {
t.Fatalf("first update failed: %v", err)
}
updated, err := store.Update(started.ID, UpdateParams{
Status: "success",
Content: "final answer",
})
if err != nil {
t.Fatalf("second update failed: %v", err)
}
if updated.Content != "final answer" {
t.Fatalf("expected content to be overwritten, got %q", updated.Content)
}
}

View File

@@ -0,0 +1,170 @@
package completionruntime
import (
"context"
"fmt"
"io"
"net/http"
"strings"
"ds2api/internal/assistantturn"
"ds2api/internal/auth"
"ds2api/internal/config"
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
)
type DeepSeekCaller interface {
CreateSession(ctx context.Context, a *auth.RequestAuth, maxAttempts int) (string, error)
GetPow(ctx context.Context, a *auth.RequestAuth, maxAttempts int) (string, error)
CallCompletion(ctx context.Context, a *auth.RequestAuth, payload map[string]any, powResp string, maxAttempts int) (*http.Response, error)
}
type Options struct {
StripReferenceMarkers bool
MaxAttempts int
RetryEnabled bool
RetryMaxAttempts int
}
type NonStreamResult struct {
SessionID string
Payload map[string]any
Turn assistantturn.Turn
Attempts int
}
type StartResult struct {
SessionID string
Payload map[string]any
Pow string
Response *http.Response
}
func StartCompletion(ctx context.Context, ds DeepSeekCaller, a *auth.RequestAuth, stdReq promptcompat.StandardRequest, opts Options) (StartResult, *assistantturn.OutputError) {
maxAttempts := opts.MaxAttempts
if maxAttempts <= 0 {
maxAttempts = 3
}
sessionID, err := ds.CreateSession(ctx, a, maxAttempts)
if err != nil {
return StartResult{}, authOutputError(a)
}
pow, err := ds.GetPow(ctx, a, maxAttempts)
if err != nil {
return StartResult{SessionID: sessionID}, &assistantturn.OutputError{Status: http.StatusUnauthorized, Message: "Failed to get PoW (invalid token or unknown error).", Code: "error"}
}
payload := stdReq.CompletionPayload(sessionID)
resp, err := ds.CallCompletion(ctx, a, payload, pow, maxAttempts)
if err != nil {
return StartResult{SessionID: sessionID, Payload: payload, Pow: pow}, &assistantturn.OutputError{Status: http.StatusInternalServerError, Message: "Failed to get completion.", Code: "error"}
}
return StartResult{SessionID: sessionID, Payload: payload, Pow: pow, Response: resp}, nil
}
func ExecuteNonStreamWithRetry(ctx context.Context, ds DeepSeekCaller, a *auth.RequestAuth, stdReq promptcompat.StandardRequest, opts Options) (NonStreamResult, *assistantturn.OutputError) {
start, startErr := StartCompletion(ctx, ds, a, stdReq, opts)
if startErr != nil {
return NonStreamResult{SessionID: start.SessionID, Payload: start.Payload}, startErr
}
maxAttempts := opts.MaxAttempts
if maxAttempts <= 0 {
maxAttempts = 3
}
sessionID := start.SessionID
payload := start.Payload
pow := start.Pow
attempts := 0
currentResp := start.Response
usagePrompt := stdReq.PromptTokenText
accumulatedThinking := ""
accumulatedRawThinking := ""
accumulatedToolDetectionThinking := ""
for {
turn, outErr := collectAttempt(currentResp, stdReq, usagePrompt, opts)
if outErr != nil {
return NonStreamResult{SessionID: sessionID, Payload: payload, Attempts: attempts}, outErr
}
accumulatedThinking += sse.TrimContinuationOverlap(accumulatedThinking, turn.Thinking)
accumulatedRawThinking += sse.TrimContinuationOverlap(accumulatedRawThinking, turn.RawThinking)
accumulatedToolDetectionThinking += sse.TrimContinuationOverlap(accumulatedToolDetectionThinking, turn.DetectionThinking)
turn.Thinking = accumulatedThinking
turn.RawThinking = accumulatedRawThinking
turn.DetectionThinking = accumulatedToolDetectionThinking
turn = assistantturn.BuildTurnFromCollected(sse.CollectResult{
Text: turn.RawText,
Thinking: turn.RawThinking,
ToolDetectionThinking: turn.DetectionThinking,
ContentFilter: turn.ContentFilter,
CitationLinks: turn.CitationLinks,
ResponseMessageID: turn.ResponseMessageID,
}, buildOptions(stdReq, usagePrompt, opts))
retryMax := opts.RetryMaxAttempts
if retryMax <= 0 {
retryMax = shared.EmptyOutputRetryMaxAttempts()
}
if !opts.RetryEnabled || !assistantturn.ShouldRetryEmptyOutput(turn, attempts, retryMax) {
return NonStreamResult{SessionID: sessionID, Payload: payload, Turn: turn, Attempts: attempts}, turn.Error
}
attempts++
config.Logger.Info("[completion_runtime_empty_retry] attempting synthetic retry", "surface", stdReq.Surface, "stream", false, "retry_attempt", attempts, "parent_message_id", turn.ResponseMessageID)
retryPow, powErr := ds.GetPow(ctx, a, maxAttempts)
if powErr != nil {
config.Logger.Warn("[completion_runtime_empty_retry] retry PoW fetch failed, falling back to original PoW", "surface", stdReq.Surface, "retry_attempt", attempts, "error", powErr)
retryPow = pow
}
retryPayload := shared.ClonePayloadForEmptyOutputRetry(payload, turn.ResponseMessageID)
nextResp, err := ds.CallCompletion(ctx, a, retryPayload, retryPow, maxAttempts)
if err != nil {
return NonStreamResult{SessionID: sessionID, Payload: payload, Turn: turn, Attempts: attempts}, &assistantturn.OutputError{Status: http.StatusInternalServerError, Message: "Failed to get completion.", Code: "error"}
}
usagePrompt = shared.UsagePromptWithEmptyOutputRetry(usagePrompt, attempts)
currentResp = nextResp
}
}
func collectAttempt(resp *http.Response, stdReq promptcompat.StandardRequest, usagePrompt string, opts Options) (assistantturn.Turn, *assistantturn.OutputError) {
defer func() {
if err := resp.Body.Close(); err != nil {
config.Logger.Warn("[completion_runtime] response body close failed", "surface", stdReq.Surface, "error", err)
}
}()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
message := strings.TrimSpace(string(body))
if message == "" {
message = http.StatusText(resp.StatusCode)
}
return assistantturn.Turn{}, &assistantturn.OutputError{Status: resp.StatusCode, Message: message, Code: "error"}
}
result := sse.CollectStream(resp, stdReq.Thinking, false)
return assistantturn.BuildTurnFromCollected(result, buildOptions(stdReq, usagePrompt, opts)), nil
}
func buildOptions(stdReq promptcompat.StandardRequest, prompt string, opts Options) assistantturn.BuildOptions {
return assistantturn.BuildOptions{
Model: stdReq.ResponseModel,
Prompt: prompt,
RefFileTokens: stdReq.RefFileTokens,
SearchEnabled: stdReq.Search,
StripReferenceMarkers: opts.StripReferenceMarkers,
ToolNames: stdReq.ToolNames,
ToolsRaw: stdReq.ToolsRaw,
ToolChoice: stdReq.ToolChoice,
}
}
func authOutputError(a *auth.RequestAuth) *assistantturn.OutputError {
if a != nil && a.UseConfigToken {
return &assistantturn.OutputError{Status: http.StatusUnauthorized, Message: "Account token is invalid. Please re-login the account in admin.", Code: "error"}
}
return &assistantturn.OutputError{Status: http.StatusUnauthorized, Message: "Invalid token. If this should be a DS2API key, add it to config.keys first.", Code: "error"}
}
func Errorf(status int, format string, args ...any) *assistantturn.OutputError {
return &assistantturn.OutputError{Status: status, Message: fmt.Sprintf(format, args...), Code: "error"}
}

View File

@@ -0,0 +1,120 @@
package completionruntime
import (
"context"
"io"
"net/http"
"strings"
"testing"
"ds2api/internal/auth"
"ds2api/internal/promptcompat"
)
type fakeDeepSeekCaller struct {
responses []*http.Response
payloads []map[string]any
}
func (f *fakeDeepSeekCaller) CreateSession(context.Context, *auth.RequestAuth, int) (string, error) {
return "session-1", nil
}
func (f *fakeDeepSeekCaller) GetPow(context.Context, *auth.RequestAuth, int) (string, error) {
return "pow", nil
}
func (f *fakeDeepSeekCaller) CallCompletion(_ context.Context, _ *auth.RequestAuth, payload map[string]any, _ string, _ int) (*http.Response, error) {
f.payloads = append(f.payloads, payload)
if len(f.responses) == 0 {
return sseHTTPResponse(http.StatusOK, `data: {"p":"response/content","v":"fallback"}`), nil
}
resp := f.responses[0]
f.responses = f.responses[1:]
return resp, nil
}
func TestExecuteNonStreamWithRetryBuildsCanonicalTurn(t *testing.T) {
ds := &fakeDeepSeekCaller{responses: []*http.Response{sseHTTPResponse(
http.StatusOK,
`data: {"response_message_id":42,"p":"response/content","v":"<tool_calls><invoke name=\"Write\"><parameter name=\"content\">{\"x\":1}</parameter></invoke></tool_calls>"}`,
)}}
stdReq := promptcompat.StandardRequest{
Surface: "test",
ResponseModel: "deepseek-v4-flash",
PromptTokenText: "prompt",
FinalPrompt: "final prompt",
ToolNames: []string{"Write"},
ToolsRaw: []any{map[string]any{
"name": "Write",
"input_schema": map[string]any{
"type": "object",
"properties": map[string]any{
"content": map[string]any{"type": "string"},
},
},
}},
}
result, outErr := ExecuteNonStreamWithRetry(context.Background(), ds, &auth.RequestAuth{}, stdReq, Options{})
if outErr != nil {
t.Fatalf("unexpected output error: %#v", outErr)
}
if result.SessionID != "session-1" {
t.Fatalf("session mismatch: %q", result.SessionID)
}
if got := result.Turn.ResponseMessageID; got != 42 {
t.Fatalf("response message id mismatch: %d", got)
}
if len(result.Turn.ToolCalls) != 1 {
t.Fatalf("expected one tool call, got %d", len(result.Turn.ToolCalls))
}
if _, ok := result.Turn.ToolCalls[0].Input["content"].(string); !ok {
t.Fatalf("expected schema-normalized string argument, got %#v", result.Turn.ToolCalls[0].Input["content"])
}
if result.Turn.Usage.InputTokens == 0 || result.Turn.Usage.TotalTokens == 0 {
t.Fatalf("expected usage to be populated, got %#v", result.Turn.Usage)
}
}
func TestExecuteNonStreamWithRetryUsesParentMessageForEmptyRetry(t *testing.T) {
ds := &fakeDeepSeekCaller{responses: []*http.Response{
sseHTTPResponse(http.StatusOK, `data: {"response_message_id":77,"p":"response/status","v":"FINISHED"}`),
sseHTTPResponse(http.StatusOK, `data: {"response_message_id":78,"p":"response/content","v":"ok"}`),
}}
stdReq := promptcompat.StandardRequest{
Surface: "test",
ResponseModel: "deepseek-v4-flash",
PromptTokenText: "prompt",
FinalPrompt: "final prompt",
}
result, outErr := ExecuteNonStreamWithRetry(context.Background(), ds, &auth.RequestAuth{}, stdReq, Options{RetryEnabled: true})
if outErr != nil {
t.Fatalf("unexpected output error: %#v", outErr)
}
if result.Attempts != 1 {
t.Fatalf("expected one retry, got %d", result.Attempts)
}
if len(ds.payloads) != 2 {
t.Fatalf("expected two completion calls, got %d", len(ds.payloads))
}
if got := ds.payloads[1]["parent_message_id"]; got != 77 {
t.Fatalf("retry parent_message_id mismatch: %#v", got)
}
if result.Turn.Text != "ok" {
t.Fatalf("retry text mismatch: %q", result.Turn.Text)
}
}
func sseHTTPResponse(status int, lines ...string) *http.Response {
body := strings.Join(lines, "\n")
if !strings.HasSuffix(body, "\n") {
body += "\n"
}
return &http.Response{
StatusCode: status,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(body)),
}
}

View File

@@ -79,13 +79,20 @@ func TestGetModelConfigDeepSeekExpertReasonerSearch(t *testing.T) {
}
}
func TestGetModelConfigDeepSeekVisionReasonerSearch(t *testing.T) {
thinking, search, ok := GetModelConfig("deepseek-v4-vision-search")
func TestGetModelConfigDeepSeekVision(t *testing.T) {
thinking, search, ok := GetModelConfig("deepseek-v4-vision")
if !ok {
t.Fatal("expected ok for deepseek-v4-vision-search")
t.Fatal("expected ok for deepseek-v4-vision")
}
if !thinking || !search {
t.Fatalf("expected both true, got thinking=%v search=%v", thinking, search)
if !thinking || search {
t.Fatalf("expected thinking=true search=false, got thinking=%v search=%v", thinking, search)
}
}
func TestGetModelConfigDeepSeekVisionSearchUnsupported(t *testing.T) {
_, _, ok := GetModelConfig("deepseek-v4-vision-search")
if ok {
t.Fatal("expected deepseek-v4-vision-search to be unsupported")
}
}
@@ -748,18 +755,16 @@ func TestOpenAIModelsResponse(t *testing.T) {
t.Fatal("expected non-empty models list")
}
expected := map[string]bool{
"deepseek-v4-flash": false,
"deepseek-v4-flash-nothinking": false,
"deepseek-v4-pro": false,
"deepseek-v4-pro-nothinking": false,
"deepseek-v4-flash-search": false,
"deepseek-v4-flash-search-nothinking": false,
"deepseek-v4-pro-search": false,
"deepseek-v4-pro-search-nothinking": false,
"deepseek-v4-vision": false,
"deepseek-v4-vision-nothinking": false,
"deepseek-v4-vision-search": false,
"deepseek-v4-vision-search-nothinking": false,
"deepseek-v4-flash": false,
"deepseek-v4-flash-nothinking": false,
"deepseek-v4-pro": false,
"deepseek-v4-pro-nothinking": false,
"deepseek-v4-flash-search": false,
"deepseek-v4-flash-search-nothinking": false,
"deepseek-v4-pro-search": false,
"deepseek-v4-pro-search-nothinking": false,
"deepseek-v4-vision": false,
"deepseek-v4-vision-nothinking": false,
}
for _, model := range data {
if _, ok := expected[model.ID]; ok {

View File

@@ -144,6 +144,44 @@ func TestLoadStoreIgnoresLegacyConfigJSONEnv(t *testing.T) {
}
}
func TestExplicitMissingConfigPathBootstrapsEmptyFileBackedStore(t *testing.T) {
path := t.TempDir() + "/config.json"
t.Setenv("DS2API_CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
store, err := LoadStoreWithError()
if err != nil {
t.Fatalf("expected missing explicit config path to bootstrap, got: %v", err)
}
if store.IsEnvBacked() {
t.Fatal("expected bootstrap store to be file-backed")
}
if store.ConfigPath() != path {
t.Fatalf("ConfigPath() = %q, want %q", store.ConfigPath(), path)
}
if len(store.Keys()) != 0 || len(store.Accounts()) != 0 {
t.Fatalf("expected empty bootstrap config, got keys=%d accounts=%d", len(store.Keys()), len(store.Accounts()))
}
if _, statErr := os.Stat(path); !errors.Is(statErr, os.ErrNotExist) {
t.Fatalf("expected bootstrap not to create config until first save, stat err=%v", statErr)
}
if err := store.Update(func(c *Config) error {
c.Keys = []string{"first-key"}
return nil
}); err != nil {
t.Fatalf("update should persist bootstrap config: %v", err)
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("expected first update to write config: %v", err)
}
if !strings.Contains(string(content), "first-key") {
t.Fatalf("expected saved config to contain first key, got: %s", content)
}
}
func TestEnvBackedStoreWritebackBootstrapsMissingConfigFile(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {

View File

@@ -144,10 +144,17 @@ func TestResolveModelCustomAliasToExpert(t *testing.T) {
func TestResolveModelCustomAliasToVision(t *testing.T) {
got, ok := ResolveModel(mockModelAliasReader{
"my-vision-model": "deepseek-v4-vision-search",
"my-vision-model": "deepseek-v4-vision",
}, "my-vision-model")
if !ok || got != "deepseek-v4-vision-search" {
t.Fatalf("expected alias -> deepseek-v4-vision-search, got ok=%v model=%q", ok, got)
if !ok || got != "deepseek-v4-vision" {
t.Fatalf("expected alias -> deepseek-v4-vision, got ok=%v model=%q", ok, got)
}
}
func TestResolveModelHeuristicVisionIgnoresSearchSuffix(t *testing.T) {
got, ok := ResolveModel(nil, "gemini-vision-search")
if !ok || got != "deepseek-v4-vision" {
t.Fatalf("expected heuristic vision alias to resolve without search variant, got ok=%v model=%q", ok, got)
}
}

View File

@@ -22,7 +22,6 @@ var deepSeekBaseModels = []ModelInfo{
{ID: "deepseek-v4-flash-search", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
{ID: "deepseek-v4-pro-search", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
{ID: "deepseek-v4-vision", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
{ID: "deepseek-v4-vision-search", Object: "model", Created: 1677610602, OwnedBy: "deepseek", Permission: []any{}},
}
var DeepSeekModels = appendNoThinkingVariants(deepSeekBaseModels)
@@ -67,7 +66,7 @@ func GetModelConfig(model string) (thinking bool, search bool, ok bool) {
switch baseModel {
case "deepseek-v4-flash", "deepseek-v4-pro", "deepseek-v4-vision":
return !noThinking, false, true
case "deepseek-v4-flash-search", "deepseek-v4-pro-search", "deepseek-v4-vision-search":
case "deepseek-v4-flash-search", "deepseek-v4-pro-search":
return !noThinking, true, true
default:
return false, false, false
@@ -81,7 +80,7 @@ func GetModelType(model string) (modelType string, ok bool) {
return "default", true
case "deepseek-v4-pro", "deepseek-v4-pro-search":
return "expert", true
case "deepseek-v4-vision", "deepseek-v4-vision-search":
case "deepseek-v4-vision":
return "vision", true
default:
return "", false
@@ -359,8 +358,6 @@ func resolveCanonicalModel(aliases map[string]string, model string) (string, boo
useSearch := strings.Contains(model, "search")
switch {
case useVision && useSearch:
return "deepseek-v4-vision-search", true
case useVision:
return "deepseek-v4-vision", true
case useReasoner && useSearch:

View File

@@ -31,13 +31,20 @@ func ResolvePath(envKey, defaultRel string) string {
func ConfigPath() string {
if strings.TrimSpace(os.Getenv("DS2API_CONFIG_PATH")) == "" && BaseDir() == "/app" {
// Official container images commonly run from /app where filesystem may be read-only.
// Prefer /data default so deployments can persist config/token state by mounting a volume.
return "/data/config.json"
return containerDefaultConfigPath()
}
return ResolvePath("DS2API_CONFIG_PATH", "config.json")
}
func containerDefaultConfigPath() string {
// Container images run as non-root by default. Only use /data when mounted/provisioned.
// Otherwise keep /app/config.json so admin-side save does not fail on MkdirAll("/data").
if st, err := os.Stat("/data"); err == nil && st.IsDir() {
return "/data/config.json"
}
return "/app/config.json"
}
func legacyContainerConfigPath() string {
return "/app/config.json"
}

View File

@@ -0,0 +1,28 @@
package config
import (
"os"
"testing"
)
func TestContainerDefaultConfigPath(t *testing.T) {
t.Run("fallback to /app when /data is missing", func(t *testing.T) {
// This test environment does not guarantee a writable/mounted /data.
// If /data is absent we must keep /app fallback to avoid persistence failures.
if _, err := os.Stat("/data"); err == nil {
t.Skip("/data exists in this environment; cannot validate missing-/data fallback")
}
if got := containerDefaultConfigPath(); got != "/app/config.json" {
t.Fatalf("containerDefaultConfigPath() = %q, want %q", got, "/app/config.json")
}
})
t.Run("prefer /data when /data directory exists", func(t *testing.T) {
if _, err := os.Stat("/data"); err != nil {
t.Skip("/data does not exist in this environment")
}
if got := containerDefaultConfigPath(); got != "/data/config.json" {
t.Fatalf("containerDefaultConfigPath() = %q, want %q", got, "/data/config.json")
}
})
}

View File

@@ -52,11 +52,12 @@ func loadStore() (*Store, error) {
func loadConfig() (Config, bool, error) {
rawCfg := strings.TrimSpace(os.Getenv("DS2API_CONFIG_JSON"))
path := ConfigPath()
if rawCfg != "" {
cfg, err := parseConfigString(rawCfg)
if err != nil {
if !IsVercel() && envWritebackEnabled() {
if fileCfg, fileErr := loadConfigFromFile(ConfigPath()); fileErr == nil {
if fileCfg, fileErr := loadConfigFromFile(path); fileErr == nil {
return fileCfg, false, nil
}
}
@@ -67,7 +68,7 @@ func loadConfig() (Config, bool, error) {
if IsVercel() || !envWritebackEnabled() {
return cfg, true, err
}
content, fileErr := os.ReadFile(ConfigPath())
content, fileErr := os.ReadFile(path)
if fileErr == nil {
var fileCfg Config
if unmarshalErr := json.Unmarshal(content, &fileCfg); unmarshalErr == nil {
@@ -79,7 +80,7 @@ func loadConfig() (Config, bool, error) {
if validateErr := ValidateConfig(cfg); validateErr != nil {
return cfg, true, validateErr
}
if writeErr := writeConfigFile(ConfigPath(), cfg.Clone()); writeErr == nil {
if writeErr := writeConfigFile(path, cfg.Clone()); writeErr == nil {
return cfg, false, err
} else {
Logger.Warn("[config] env writeback bootstrap failed", "error", writeErr)
@@ -87,7 +88,7 @@ func loadConfig() (Config, bool, error) {
}
return cfg, true, err
}
cfg, err := loadConfigFromFile(ConfigPath())
cfg, err := loadConfigFromFile(path)
if err != nil {
if shouldTryLegacyContainerConfigPath() {
legacyPath := legacyContainerConfigPath()
@@ -100,6 +101,10 @@ func loadConfig() (Config, bool, error) {
// Vercel may start without writable/present config; keep in-memory bootstrap config.
return Config{}, true, nil
}
if shouldBootstrapMissingConfigFile(err) {
Logger.Warn("[config] config file missing; starting with empty file-backed config", "path", path)
return Config{}, false, nil
}
return Config{}, false, err
}
if IsVercel() {
@@ -109,6 +114,10 @@ func loadConfig() (Config, bool, error) {
return cfg, false, nil
}
func shouldBootstrapMissingConfigFile(err error) bool {
return errors.Is(err, os.ErrNotExist) && strings.TrimSpace(os.Getenv("DS2API_CONFIG_PATH")) != ""
}
func loadConfigFromFile(path string) (Config, error) {
content, err := os.ReadFile(path)
if err != nil {

View File

@@ -7,6 +7,7 @@ import (
dsprotocol "ds2api/internal/deepseek/protocol"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
@@ -27,7 +28,7 @@ type continueState struct {
}
// wrapCompletionWithAutoContinue wraps the completion response body so that
// if the upstream indicates the response is incomplete (WIP / INCOMPLETE /
// if the upstream indicates the response is incomplete (INCOMPLETE /
// AUTO_CONTINUE), ds2api will automatically call the DeepSeek continue
// endpoint and splice the continuation SSE stream onto the original.
// The caller sees a single, seamless SSE stream.
@@ -132,33 +133,51 @@ func pumpAutoContinue(ctx context.Context, pw *io.PipeWriter, initial io.ReadClo
// sentinels are consumed (not forwarded) so that the downstream only sees
// one final [DONE] at the very end.
func streamBodyWithContinueState(ctx context.Context, pw *io.PipeWriter, body io.Reader, state *continueState) (bool, error) {
scanner := bufio.NewScanner(body)
scanner.Buffer(make([]byte, 0, 64*1024), 2*1024*1024)
reader := bufio.NewReaderSize(body, 64*1024)
hadDone := false
for scanner.Scan() {
for {
select {
case <-ctx.Done():
return hadDone, ctx.Err()
default:
}
line := append([]byte{}, scanner.Bytes()...)
trimmed := strings.TrimSpace(string(line))
if trimmed == "" {
continue
}
if strings.HasPrefix(trimmed, "data:") {
data := strings.TrimSpace(strings.TrimPrefix(trimmed, "data:"))
if data == "[DONE]" {
hadDone = true
continue
line, err := reader.ReadBytes('\n')
if len(line) == 0 && err != nil {
if err == io.EOF {
return hadDone, nil
}
state.observe(data)
return hadDone, err
}
if _, err := io.Copy(pw, bytes.NewReader(append(line, '\n'))); err != nil {
trimmed := strings.TrimSpace(string(line))
if trimmed != "" {
if strings.HasPrefix(trimmed, "data:") {
data := strings.TrimSpace(strings.TrimPrefix(trimmed, "data:"))
if data == "[DONE]" {
hadDone = true
if err != nil && err != io.EOF {
return hadDone, err
}
if err == io.EOF {
return hadDone, nil
}
continue
}
state.observe(data)
}
if !strings.HasSuffix(string(line), "\n") {
line = append(line, '\n')
}
if _, copyErr := io.Copy(pw, bytes.NewReader(line)); copyErr != nil {
return hadDone, copyErr
}
}
if err != nil {
if err == io.EOF {
return hadDone, nil
}
return hadDone, err
}
}
return hadDone, scanner.Err()
}
// observe extracts continue-relevant signals from an SSE JSON chunk.
@@ -174,49 +193,100 @@ func (s *continueState) observe(data string) {
if id := intFrom(chunk["response_message_id"]); id > 0 {
s.responseMessageID = id
}
// Path-based status: {"p": "response/status", "v": "FINISHED"}
if p, _ := chunk["p"].(string); p == "response/status" {
if status, _ := chunk["v"].(string); status != "" {
s.lastStatus = strings.TrimSpace(status)
if strings.EqualFold(s.lastStatus, "FINISHED") {
s.finished = true
}
}
s.observeDirectPatch(asString(chunk["p"]), chunk["v"])
if p, _ := chunk["p"].(string); p == "response" {
s.observeBatchPatches("response", chunk["v"])
} else {
s.observeBatchPatches("", chunk["v"])
}
// Nested v.response
v, _ := chunk["v"].(map[string]any)
if response, _ := v["response"].(map[string]any); response != nil {
if id := intFrom(response["message_id"]); id > 0 {
s.responseMessageID = id
}
if status, _ := response["status"].(string); status != "" {
s.lastStatus = strings.TrimSpace(status)
if strings.EqualFold(s.lastStatus, "FINISHED") {
s.finished = true
}
}
if autoContinue, ok := response["auto_continue"].(bool); ok && autoContinue {
if v, _ := chunk["v"].(map[string]any); v != nil {
s.observeResponseObject(v["response"])
}
if message, _ := chunk["message"].(map[string]any); message != nil {
s.observeResponseObject(message["response"])
}
}
func (s *continueState) observeDirectPatch(path string, value any) {
if s == nil {
return
}
switch strings.Trim(strings.TrimSpace(path), "/") {
case "response/status", "status", "response/quasi_status", "quasi_status":
s.setStatus(asString(value))
case "response/auto_continue", "auto_continue":
if v, ok := value.(bool); ok && v {
s.lastStatus = "AUTO_CONTINUE"
}
}
// Nested message.response
if message, _ := chunk["message"].(map[string]any); message != nil {
if response, _ := message["response"].(map[string]any); response != nil {
if id := intFrom(response["message_id"]); id > 0 {
s.responseMessageID = id
}
if status, _ := response["status"].(string); status != "" {
s.lastStatus = strings.TrimSpace(status)
if strings.EqualFold(s.lastStatus, "FINISHED") {
s.finished = true
}
}
func (s *continueState) observeResponseObject(raw any) {
if s == nil {
return
}
response, _ := raw.(map[string]any)
if response == nil {
return
}
if id := intFrom(response["message_id"]); id > 0 {
s.responseMessageID = id
}
s.setStatus(asString(response["status"]))
if autoContinue, ok := response["auto_continue"].(bool); ok && autoContinue {
s.lastStatus = "AUTO_CONTINUE"
}
}
func (s *continueState) observeBatchPatches(parentPath string, raw any) {
if s == nil {
return
}
patches, ok := raw.([]any)
if !ok {
return
}
for _, patch := range patches {
m, ok := patch.(map[string]any)
if !ok {
continue
}
path := strings.TrimSpace(asString(m["p"]))
if path == "" {
continue
}
fullPath := path
if parent := strings.Trim(strings.TrimSpace(parentPath), "/"); parent != "" && !strings.Contains(path, "/") {
fullPath = parent + "/" + path
}
switch strings.Trim(strings.TrimSpace(fullPath), "/") {
case "response/status", "status", "response/quasi_status", "quasi_status":
s.setStatus(asString(m["v"]))
case "response/auto_continue", "auto_continue":
if v, ok := m["v"].(bool); ok && v {
s.lastStatus = "AUTO_CONTINUE"
}
}
}
}
// shouldContinue returns true when the upstream indicates the response is
// not yet finished and we have enough information to issue a continue request.
func (s *continueState) setStatus(status string) {
if s == nil {
return
}
normalized := strings.TrimSpace(status)
if normalized == "" {
return
}
s.lastStatus = normalized
if strings.EqualFold(normalized, "FINISHED") || strings.EqualFold(normalized, "CONTENT_FILTER") {
s.finished = true
}
}
// shouldContinue returns true when the upstream explicitly indicates the
// response is incomplete and we have enough information to issue a continue
// request. Plain WIP is not sufficient because normal streams begin in WIP.
func (s *continueState) shouldContinue() bool {
if s == nil {
return false
@@ -225,7 +295,7 @@ func (s *continueState) shouldContinue() bool {
return false
}
switch strings.ToUpper(strings.TrimSpace(s.lastStatus)) {
case "WIP", "INCOMPLETE", "AUTO_CONTINUE":
case "INCOMPLETE", "AUTO_CONTINUE":
return true
default:
return false
@@ -241,3 +311,19 @@ func (s *continueState) prepareForNextRound() {
s.finished = false
s.lastStatus = ""
}
func asString(v any) string {
if v == nil {
return ""
}
switch x := v.(type) {
case string:
return x
default:
s := strings.TrimSpace(strings.ReplaceAll(strings.TrimSpace(fmt.Sprint(v)), "\u0000", ""))
if s == "<nil>" {
return ""
}
return s
}
}

View File

@@ -8,6 +8,7 @@ import (
"io"
"net/http"
"strings"
"sync/atomic"
"testing"
"ds2api/internal/auth"
@@ -124,6 +125,146 @@ func TestCallCompletionAutoContinueThreadsPowHeader(t *testing.T) {
}
}
func TestAutoContinueDoesNotTriggerOnPlainWIPWithoutExplicitContinuationSignal(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"WIP","auto_continue":false}}}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return nil, errors.New("continue should not have been called")
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 0 {
t.Fatalf("expected no continue calls, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte(`"status":"WIP"`)) || !bytes.Contains(out, []byte(`data: [DONE]`)) {
t.Fatalf("expected original body to pass through unchanged, got=%s", string(out))
}
}
func TestAutoContinuePassesThroughLongSingleSSELine(t *testing.T) {
payload := strings.Repeat("x", 2*1024*1024+4096)
initialBody := `data: {"p":"response/content","v":"` + payload + `"}` + "\n" +
`data: [DONE]` + "\n"
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
return nil, errors.New("continue should not have been called")
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if !bytes.Contains(out, []byte(payload)) {
t.Fatalf("expected long SSE payload to pass through, got len=%d want payload len=%d", len(out), len(payload))
}
if !bytes.Contains(out, []byte(`data: [DONE]`)) {
t.Fatalf("expected final DONE sentinel in body, got len=%d", len(out))
}
}
func TestAutoContinueTriggersOnDirectQuasiStatusIncomplete(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"p":"response/content","v":"<tool_calls><invoke name=\"write_file\"><parameter name=\"content\"><![CDATA[part-one"}`,
`data: {"p":"response/quasi_status","v":"INCOMPLETE"}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`data: {"response_message_id":322,"p":"response/content","v":"-part-two]]></parameter></invoke></tool_calls>"}` + "\n" +
`data: {"p":"response/status","v":"FINISHED"}` + "\n" +
`data: [DONE]` + "\n",
)),
}, nil
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 1 {
t.Fatalf("expected exactly one continue call, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte("part-one")) || !bytes.Contains(out, []byte("-part-two")) {
t.Fatalf("expected continued tool content in body, got=%s", string(out))
}
}
func TestAutoContinueTriggersOnResponseBatchQuasiStatusIncomplete(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"WIP","auto_continue":false}}}`,
`data: {"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":2413},{"p":"quasi_status","v":"INCOMPLETE"}]}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`data: {"response_message_id":322,"p":"response/status","v":"FINISHED"}` + "\n" +
`data: [DONE]` + "\n",
)),
}, nil
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 1 {
t.Fatalf("expected exactly one continue call, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte(`"quasi_status","v":"INCOMPLETE"`)) || !bytes.Contains(out, []byte(`"v":"FINISHED"`)) {
t.Fatalf("expected continued output to include initial and final rounds, got=%s", string(out))
}
}
func TestAutoContinueDoesNotTriggerWhenResponseBatchQuasiStatusFinished(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"WIP","auto_continue":false}}}`,
`data: {"p":"response","o":"BATCH","v":[{"p":"accumulated_token_usage","v":2413},{"p":"quasi_status","v":"FINISHED"}]}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return nil, errors.New("continue should not have been called")
})
defer func() { _ = body.Close() }()
out, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 0 {
t.Fatalf("expected no continue calls, got %d", continueCalls.Load())
}
if !bytes.Contains(out, []byte(`"quasi_status","v":"FINISHED"`)) || !bytes.Contains(out, []byte(`data: [DONE]`)) {
t.Fatalf("expected original finished body to pass through unchanged, got=%s", string(out))
}
}
type failingOrCompletionDoer struct {
completionResp *http.Response
}
@@ -134,3 +275,33 @@ func (d failingOrCompletionDoer) Do(req *http.Request) (*http.Response, error) {
}
return nil, errors.New("forced stream failure")
}
func TestAutoContinuePreservesIncompleteStateWhenNextChunkOmitsStatus(t *testing.T) {
initialBody := strings.Join([]string{
`data: {"response_message_id":321,"v":{"response":{"message_id":321,"status":"INCOMPLETE"}}}`,
`data: {"p":"response/content","v":{"text":"continued"}}`,
`data: [DONE]`,
}, "\n") + "\n"
var continueCalls atomic.Int32
body := newAutoContinueBody(context.Background(), io.NopCloser(strings.NewReader(initialBody)), "session-123", 8, func(context.Context, string, int) (*http.Response, error) {
continueCalls.Add(1)
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(
`data: {"response_message_id":322,"p":"response/status","v":"FINISHED"}` + "\n" +
`data: [DONE]` + "\n",
)),
}, nil
})
defer func() { _ = body.Close() }()
_, err := io.ReadAll(body)
if err != nil {
t.Fatalf("read body failed: %v", err)
}
if continueCalls.Load() != 1 {
t.Fatalf("expected exactly one continue call, got %d", continueCalls.Load())
}
}

View File

@@ -22,6 +22,9 @@ const (
var fileReadySleep = time.Sleep
// ErrUploadFileNotFound indicates that DeepSeek returned no matching uploaded file.
var ErrUploadFileNotFound = errors.New("uploaded file not found")
func (c *Client) waitForUploadedFile(ctx context.Context, a *auth.RequestAuth, result *UploadFileResult) error {
if result == nil || strings.TrimSpace(result.ID) == "" {
return nil
@@ -42,7 +45,7 @@ func (c *Client) waitForUploadedFile(ctx context.Context, a *auth.RequestAuth, r
return fmt.Errorf("waiting for file %s to become ready: %w", result.ID, err)
}
fetched, err := c.fetchUploadedFile(pollCtx, a, result.ID)
fetched, err := c.FetchUploadedFile(pollCtx, a, result.ID)
if err == nil && fetched != nil {
mergeUploadFileResults(result, fetched)
if isReadyUploadFileStatus(result.Status) {
@@ -65,7 +68,8 @@ func (c *Client) waitForUploadedFile(ctx context.Context, a *auth.RequestAuth, r
return fmt.Errorf("file %s did not become ready: %w", result.ID, lastErr)
}
func (c *Client) fetchUploadedFile(ctx context.Context, a *auth.RequestAuth, fileID string) (*UploadFileResult, error) {
// FetchUploadedFile returns metadata for an uploaded DeepSeek file by ID.
func (c *Client) FetchUploadedFile(ctx context.Context, a *auth.RequestAuth, fileID string) (*UploadFileResult, error) {
fileID = strings.TrimSpace(fileID)
if fileID == "" {
return nil, errors.New("file id is required")
@@ -92,7 +96,7 @@ func (c *Client) fetchUploadedFile(ctx context.Context, a *auth.RequestAuth, fil
result := extractFetchedUploadFileResult(resp, fileID)
if result == nil || strings.TrimSpace(result.ID) == "" {
return nil, errors.New("fetch files succeeded without matching file data")
return nil, ErrUploadFileNotFound
}
result.Raw = resp
return result, nil

View File

@@ -23,6 +23,7 @@ type UploadFileRequest struct {
Filename string
ContentType string
Purpose string
ModelType string
Data []byte
}
@@ -54,6 +55,7 @@ func (c *Client) UploadFile(ctx context.Context, a *auth.RequestAuth, req Upload
contentType = "application/octet-stream"
}
purpose := strings.TrimSpace(req.Purpose)
modelType := strings.ToLower(strings.TrimSpace(req.ModelType))
body, contentTypeHeader, err := buildUploadMultipartBody(filename, contentType, req.Data)
if err != nil {
return nil, err
@@ -64,6 +66,9 @@ func (c *Client) UploadFile(ctx context.Context, a *auth.RequestAuth, req Upload
"purpose": purpose,
"bytes": len(req.Data),
}
if modelType != "" {
capturePayload["model_type"] = modelType
}
captureSession := c.capture.Start("deepseek_upload_file", dsprotocol.DeepSeekUploadFileURL, a.AccountID, capturePayload)
attempts := 0
refreshed := false
@@ -81,6 +86,9 @@ func (c *Client) UploadFile(ctx context.Context, a *auth.RequestAuth, req Upload
}
headers := c.authHeaders(a.DeepSeekToken)
headers["Content-Type"] = contentTypeHeader
if modelType != "" {
headers["x-model-type"] = modelType
}
headers["x-ds-pow-response"] = powHeader
headers["x-file-size"] = strconv.Itoa(len(req.Data))
headers["x-thinking-enabled"] = "1"

View File

@@ -82,6 +82,7 @@ func TestUploadFileUsesUploadTargetPowAndMultipartHeaders(t *testing.T) {
var seenTargetPath string
var seenContentType string
var seenFileSize string
var seenModelType string
var seenBody string
call := 0
client := &Client{
@@ -96,6 +97,7 @@ func TestUploadFileUsesUploadTargetPowAndMultipartHeaders(t *testing.T) {
seenPow = req.Header.Get("x-ds-pow-response")
seenContentType = req.Header.Get("Content-Type")
seenFileSize = req.Header.Get("x-file-size")
seenModelType = req.Header.Get("x-model-type")
seenBody = string(bodyBytes)
return &http.Response{StatusCode: http.StatusOK, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(uploadResponse)), Request: req}, nil
default:
@@ -112,6 +114,7 @@ func TestUploadFileUsesUploadTargetPowAndMultipartHeaders(t *testing.T) {
Filename: "demo.txt",
ContentType: "text/plain",
Purpose: "assistants",
ModelType: "vision",
Data: []byte("hello"),
}, 1)
if err != nil {
@@ -140,6 +143,9 @@ func TestUploadFileUsesUploadTargetPowAndMultipartHeaders(t *testing.T) {
if seenFileSize != "5" {
t.Fatalf("expected x-file-size=5, got %q", seenFileSize)
}
if seenModelType != "vision" {
t.Fatalf("expected x-model-type=vision, got %q", seenModelType)
}
if !strings.HasPrefix(seenContentType, "multipart/form-data; boundary=") {
t.Fatalf("expected multipart content type, got %q", seenContentType)
}

View File

@@ -159,6 +159,6 @@ func toStringSet(in []string) map[string]struct{} {
const (
KeepAliveTimeout = 5
StreamIdleTimeout = 90
MaxKeepaliveCount = 10
StreamIdleTimeout = 300
MaxKeepaliveCount = 40
)

View File

@@ -2,7 +2,7 @@
"client": {
"name": "DeepSeek",
"platform": "android",
"version": "2.0.1",
"version": "2.0.4",
"android_api_level": "35",
"locale": "zh_CN"
},
@@ -24,4 +24,4 @@
"skip_exact_paths": [
"response/search_status"
]
}
}

View File

@@ -2,20 +2,24 @@ package protocol
import (
"bufio"
"io"
"net/http"
)
func ScanSSELines(resp *http.Response, onLine func([]byte) bool) error {
scanner := bufio.NewScanner(resp.Body)
buf := make([]byte, 0, 64*1024)
scanner.Buffer(buf, 2*1024*1024)
for scanner.Scan() {
if !onLine(scanner.Bytes()) {
break
reader := bufio.NewReaderSize(resp.Body, 64*1024)
for {
line, err := reader.ReadBytes('\n')
if len(line) > 0 {
if !onLine(line) {
return nil
}
}
if err != nil {
if err == io.EOF {
return nil
}
return err
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}

View File

@@ -0,0 +1,26 @@
package protocol
import (
"io"
"net/http"
"strings"
"testing"
)
func TestScanSSELinesHandlesLongSingleLine(t *testing.T) {
payload := strings.Repeat("x", 2*1024*1024+4096)
body := "data: {\"p\":\"response/content\",\"v\":\"" + payload + "\"}\n"
resp := &http.Response{Body: io.NopCloser(strings.NewReader(body))}
var got string
err := ScanSSELines(resp, func(line []byte) bool {
got = string(line)
return true
})
if err != nil {
t.Fatalf("ScanSSELines returned error: %v", err)
}
if !strings.Contains(got, payload) {
t.Fatalf("long SSE line was not preserved: got len=%d want payload len=%d", len(got), len(payload))
}
}

View File

@@ -10,6 +10,8 @@ import (
"sync"
"time"
"ds2api/internal/util"
"github.com/google/uuid"
)
@@ -194,7 +196,8 @@ func (c *captureBody) append(chunk string) {
}
remain := maxLen - current
if len(chunk) > remain {
c.buf.WriteString(chunk[:remain])
truncated, _ := util.TruncateUTF8Bytes(chunk, remain)
c.buf.WriteString(truncated)
c.truncated = true
return
}

View File

@@ -4,6 +4,7 @@ import (
"io"
"strings"
"testing"
"unicode/utf8"
)
func TestNewFromEnvDefaults(t *testing.T) {
@@ -82,3 +83,28 @@ func TestWrapBodyTruncatesByLimit(t *testing.T) {
t.Fatalf("expected account id, got %q", items[0].AccountID)
}
}
func TestWrapBodyTruncatesUTF8WithoutBreakingRune(t *testing.T) {
s := &Store{enabled: true, limit: 5, maxBodyBytes: 5}
session := s.Start("test", "http://x", "acc1", map[string]any{"x": 1})
if session == nil {
t.Fatal("expected session")
}
rc := session.WrapBody(io.NopCloser(strings.NewReader("😀xy")), 200)
_, _ = io.ReadAll(rc)
_ = rc.Close()
items := s.Snapshot()
if len(items) != 1 {
t.Fatalf("expected 1 item, got %d", len(items))
}
if !utf8.ValidString(items[0].ResponseBody) {
t.Fatalf("expected valid utf-8 response body, got %q", items[0].ResponseBody)
}
if items[0].ResponseBody != "😀x" {
t.Fatalf("expected rune-safe truncation, got %q", items[0].ResponseBody)
}
if !items[0].ResponseTruncated {
t.Fatal("expected truncated flag true")
}
}

View File

@@ -1,13 +1,56 @@
package claude
import (
"ds2api/internal/assistantturn"
"ds2api/internal/toolcall"
"fmt"
"time"
"ds2api/internal/prompt"
"ds2api/internal/util"
)
func BuildMessageResponseFromTurn(messageID, model string, turn assistantturn.Turn, exposeThinking bool) map[string]any {
content := make([]map[string]any, 0, 4)
if exposeThinking && turn.Thinking != "" {
content = append(content, map[string]any{"type": "thinking", "thinking": turn.Thinking})
}
stopReason := "end_turn"
if len(turn.ToolCalls) > 0 {
stopReason = "tool_use"
for i, tc := range turn.ToolCalls {
content = append(content, map[string]any{
"type": "tool_use",
"id": fmt.Sprintf("toolu_%d_%d", time.Now().Unix(), i),
"name": tc.Name,
"input": tc.Input,
})
}
} else {
text := turn.Text
if text == "" && exposeThinking {
text = turn.Thinking
}
if text == "" {
text = "抱歉,没有生成有效的响应内容。"
}
content = append(content, map[string]any{"type": "text", "text": text})
}
return map[string]any{
"id": messageID,
"type": "message",
"role": "assistant",
"model": model,
"content": content,
"stop_reason": stopReason,
"stop_sequence": nil,
"usage": map[string]any{
"input_tokens": turn.Usage.InputTokens,
"output_tokens": turn.Usage.OutputTokens,
},
}
}
func BuildMessageResponse(messageID, model string, normalizedMessages []any, finalThinking, finalText string, toolNames []string) map[string]any {
detected := toolcall.ParseToolCalls(finalText, toolNames)
if len(detected) == 0 && finalText == "" && finalThinking != "" {
@@ -43,8 +86,23 @@ func BuildMessageResponse(messageID, model string, normalizedMessages []any, fin
"stop_reason": stopReason,
"stop_sequence": nil,
"usage": map[string]any{
"input_tokens": util.EstimateTokens(fmt.Sprintf("%v", normalizedMessages)),
"output_tokens": util.EstimateTokens(finalThinking) + util.EstimateTokens(finalText),
"input_tokens": util.CountPromptTokens(prompt.MessagesPrepareWithThinking(claudeMessageMaps(normalizedMessages), false), model),
"output_tokens": util.CountOutputTokens(finalThinking, model) + util.CountOutputTokens(finalText, model),
},
}
}
func claudeMessageMaps(messages []any) []map[string]any {
if len(messages) == 0 {
return nil
}
out := make([]map[string]any, 0, len(messages))
for _, item := range messages {
msg, ok := item.(map[string]any)
if !ok {
continue
}
out = append(out, msg)
}
return out
}

View File

@@ -29,7 +29,7 @@ func BuildChatCompletionWithToolCalls(completionID, model, finalPrompt, finalThi
"created": time.Now().Unix(),
"model": model,
"choices": []map[string]any{{"index": 0, "message": messageObj, "finish_reason": finishReason}},
"usage": BuildChatUsage(finalPrompt, finalThinking, finalText),
"usage": BuildChatUsageForModel(model, finalPrompt, finalThinking, finalText, 0),
}
}

View File

@@ -70,7 +70,7 @@ func BuildResponseObjectFromItems(responseID, model, finalPrompt, finalThinking,
"model": model,
"output": output,
"output_text": outputText,
"usage": BuildResponsesUsage(finalPrompt, finalThinking, finalText),
"usage": BuildResponsesUsageForModel(model, finalPrompt, finalThinking, finalText, 0),
}
}

View File

@@ -6,6 +6,7 @@ import (
"testing"
"ds2api/internal/toolcall"
"ds2api/internal/util"
)
func TestBuildResponseObjectKeepsFencedToolPayloadAsText(t *testing.T) {
@@ -177,3 +178,17 @@ func TestBuildResponseObjectWithToolCallsCoercesSchemaDeclaredStringArguments(t
t.Fatalf("expected response content stringified by schema, got %#v", args["content"])
}
}
func TestBuildChatUsageForModelUsesConservativePromptCount(t *testing.T) {
prompt := strings.Repeat("上下文token ", 40)
usage := BuildChatUsageForModel("deepseek-v4-flash", prompt, "", "ok", 0)
promptTokens, _ := usage["prompt_tokens"].(int)
if promptTokens <= util.EstimateTokens(prompt) {
t.Fatalf("expected conservative prompt token count > rough estimate, got=%d estimate=%d", promptTokens, util.EstimateTokens(prompt))
}
totalTokens, _ := usage["total_tokens"].(int)
completionTokens, _ := usage["completion_tokens"].(int)
if totalTokens != promptTokens+completionTokens {
t.Fatalf("expected total tokens to add up, got usage=%#v", usage)
}
}

View File

@@ -2,10 +2,10 @@ package openai
import "ds2api/internal/util"
func BuildChatUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
func BuildChatUsageForModel(model, finalPrompt, finalThinking, finalText string, refFileTokens int) map[string]any {
promptTokens := util.CountPromptTokens(finalPrompt, model) + refFileTokens
reasoningTokens := util.CountOutputTokens(finalThinking, model)
completionTokens := util.CountOutputTokens(finalText, model)
return map[string]any{
"prompt_tokens": promptTokens,
"completion_tokens": reasoningTokens + completionTokens,
@@ -16,13 +16,21 @@ func BuildChatUsage(finalPrompt, finalThinking, finalText string) map[string]any
}
}
func BuildResponsesUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
func BuildChatUsage(finalPrompt, finalThinking, finalText string) map[string]any {
return BuildChatUsageForModel("", finalPrompt, finalThinking, finalText, 0)
}
func BuildResponsesUsageForModel(model, finalPrompt, finalThinking, finalText string, refFileTokens int) map[string]any {
promptTokens := util.CountPromptTokens(finalPrompt, model) + refFileTokens
reasoningTokens := util.CountOutputTokens(finalThinking, model)
completionTokens := util.CountOutputTokens(finalText, model)
return map[string]any{
"input_tokens": promptTokens,
"output_tokens": reasoningTokens + completionTokens,
"total_tokens": promptTokens + reasoningTokens + completionTokens,
}
}
func BuildResponsesUsage(finalPrompt, finalThinking, finalText string) map[string]any {
return BuildResponsesUsageForModel("", finalPrompt, finalThinking, finalText, 0)
}

View File

@@ -15,6 +15,7 @@ import (
"ds2api/internal/devcapture"
adminshared "ds2api/internal/httpapi/admin/shared"
"ds2api/internal/rawsample"
"ds2api/internal/util"
)
type captureChain struct {
@@ -479,10 +480,13 @@ func previewCaptureChainResponse(chain captureChain) string {
func previewText(text string, limit int) string {
text = strings.TrimSpace(text)
if limit <= 0 || len(text) <= limit {
if limit <= 0 {
return text
}
return text[:limit] + "..."
if truncated, ok := util.TruncateRunes(text, limit); ok {
return truncated + "..."
}
return text
}
func captureChainHasTruncatedResponse(chain captureChain) bool {

View File

@@ -10,6 +10,7 @@ import (
"path/filepath"
"strings"
"testing"
"unicode/utf8"
"ds2api/internal/devcapture"
)
@@ -231,6 +232,16 @@ func TestCombineCaptureBodiesPreservesOrderAndSeparators(t *testing.T) {
}
}
func TestPreviewTextPreservesUTF8MB4Characters(t *testing.T) {
preview := previewText(strings.Repeat("😀", 281), 280)
if !utf8.ValidString(preview) {
t.Fatalf("expected valid utf-8 preview, got %q", preview)
}
if preview != strings.Repeat("😀", 280)+"..." {
t.Fatalf("unexpected preview: %q", preview)
}
}
func TestQueryRawSampleCapturesGroupsBySessionAndMatchesQuestion(t *testing.T) {
devcapture.Global().Clear()
defer devcapture.Global().Clear()

View File

@@ -3,12 +3,20 @@ package claude
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"time"
"ds2api/internal/auth"
"ds2api/internal/completionruntime"
"ds2api/internal/config"
claudefmt "ds2api/internal/format/claude"
"ds2api/internal/httpapi/requestbody"
"ds2api/internal/promptcompat"
streamengine "ds2api/internal/stream"
"ds2api/internal/translatorcliproxy"
"ds2api/internal/util"
@@ -20,20 +28,100 @@ func (h *Handler) Messages(w http.ResponseWriter, r *http.Request) {
if strings.TrimSpace(r.Header.Get("anthropic-version")) == "" {
r.Header.Set("anthropic-version", "2023-06-01")
}
if h.OpenAI == nil {
writeClaudeError(w, http.StatusInternalServerError, "OpenAI proxy backend unavailable.")
if isClaudeVercelProxyRequest(r) && h.proxyViaOpenAI(w, r, h.Store) {
return
}
if h.proxyViaOpenAI(w, r, h.Store) {
if h.Auth == nil || h.DS == nil {
if h.OpenAI != nil && h.proxyViaOpenAI(w, r, h.Store) {
return
}
writeClaudeError(w, http.StatusInternalServerError, "Claude runtime backend unavailable.")
return
}
writeClaudeError(w, http.StatusBadGateway, "Failed to proxy Claude request.")
if h.handleClaudeDirect(w, r) {
return
}
writeClaudeError(w, http.StatusBadGateway, "Failed to handle Claude request.")
}
func isClaudeVercelProxyRequest(r *http.Request) bool {
if r == nil || r.URL == nil {
return false
}
return strings.TrimSpace(r.URL.Query().Get("__stream_prepare")) == "1" ||
strings.TrimSpace(r.URL.Query().Get("__stream_release")) == "1"
}
func (h *Handler) handleClaudeDirect(w http.ResponseWriter, r *http.Request) bool {
raw, err := io.ReadAll(r.Body)
if err != nil {
if errors.Is(err, requestbody.ErrInvalidUTF8Body) {
writeClaudeError(w, http.StatusBadRequest, "invalid json")
} else {
writeClaudeError(w, http.StatusBadRequest, "invalid body")
}
return true
}
var req map[string]any
if err := json.Unmarshal(raw, &req); err != nil {
writeClaudeError(w, http.StatusBadRequest, "invalid json")
return true
}
exposeThinking := false
if enabled, ok := util.ResolveThinkingOverride(req); ok && enabled {
exposeThinking = true
} else if _, ok := util.ResolveThinkingOverride(req); !ok && !util.ToBool(req["stream"]) {
req["thinking"] = map[string]any{"type": "enabled"}
}
norm, err := normalizeClaudeRequest(h.Store, req)
if err != nil {
writeClaudeError(w, http.StatusBadRequest, err.Error())
return true
}
a, err := h.Auth.Determine(r)
if err != nil {
writeClaudeError(w, http.StatusUnauthorized, err.Error())
return true
}
defer h.Auth.Release(a)
if norm.Standard.Stream {
h.handleClaudeDirectStream(w, r, a, norm.Standard)
return true
}
result, outErr := completionruntime.ExecuteNonStreamWithRetry(r.Context(), h.DS, a, norm.Standard, completionruntime.Options{
StripReferenceMarkers: h.compatStripReferenceMarkers(),
RetryEnabled: true,
})
if outErr != nil {
writeClaudeError(w, outErr.Status, outErr.Message)
return true
}
writeJSON(w, http.StatusOK, claudefmt.BuildMessageResponseFromTurn(
fmt.Sprintf("msg_%d", time.Now().UnixNano()),
norm.Standard.ResponseModel,
result.Turn,
exposeThinking,
))
return true
}
func (h *Handler) handleClaudeDirectStream(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, stdReq promptcompat.StandardRequest) {
start, outErr := completionruntime.StartCompletion(r.Context(), h.DS, a, stdReq, completionruntime.Options{})
if outErr != nil {
writeClaudeError(w, outErr.Status, outErr.Message)
return
}
h.handleClaudeStreamRealtime(w, r, start.Response, stdReq.ResponseModel, stdReq.Messages, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw)
}
func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, store ConfigReader) bool {
raw, err := io.ReadAll(r.Body)
if err != nil {
writeClaudeError(w, http.StatusBadRequest, "invalid body")
if errors.Is(err, requestbody.ErrInvalidUTF8Body) {
writeClaudeError(w, http.StatusBadRequest, "invalid json")
} else {
writeClaudeError(w, http.StatusBadRequest, "invalid body")
}
return true
}
var req map[string]any
@@ -206,6 +294,7 @@ func (h *Handler) handleClaudeStreamRealtime(w http.ResponseWriter, r *http.Requ
h.compatStripReferenceMarkers(),
toolNames,
toolsRaw,
buildClaudePromptTokenText(messages, thinkingEnabled),
)
streamRuntime.sendMessageStart()

View File

@@ -28,6 +28,18 @@ func makeClaudeSSEHTTPResponse(lines ...string) *http.Response {
}
}
func makeClaudeContentLine(t *testing.T, text string) string {
t.Helper()
line, err := json.Marshal(map[string]any{
"p": "response/content",
"v": text,
})
if err != nil {
t.Fatalf("marshal content line failed: %v", err)
}
return "data: " + string(line)
}
func parseClaudeFrames(t *testing.T, body string) []claudeFrame {
t.Helper()
chunks := strings.Split(body, "\n\n")
@@ -71,6 +83,17 @@ func findClaudeFrames(frames []claudeFrame, event string) []claudeFrame {
return out
}
func collectClaudeTextDeltas(frames []claudeFrame) string {
var combined strings.Builder
for _, f := range findClaudeFrames(frames, "content_block_delta") {
delta, _ := f.Payload["delta"].(map[string]any)
if delta["type"] == "text_delta" {
combined.WriteString(asString(delta["text"]))
}
}
return combined.String()
}
func TestHandleClaudeStreamRealtimeTextIncrementsWithEventHeaders(t *testing.T) {
h := &Handler{}
resp := makeClaudeSSEHTTPResponse(
@@ -96,8 +119,8 @@ func TestHandleClaudeStreamRealtimeTextIncrementsWithEventHeaders(t *testing.T)
frames := parseClaudeFrames(t, body)
deltas := findClaudeFrames(frames, "content_block_delta")
if len(deltas) < 2 {
t.Fatalf("expected at least 2 text deltas, got=%d body=%s", len(deltas), body)
if len(deltas) < 1 {
t.Fatalf("expected at least 1 text delta, got=%d body=%s", len(deltas), body)
}
combined := strings.Builder{}
for _, f := range deltas {
@@ -111,6 +134,52 @@ func TestHandleClaudeStreamRealtimeTextIncrementsWithEventHeaders(t *testing.T)
}
}
func TestHandleClaudeStreamRealtimeToolBufferedPlainTextDoesNotRepeatFinalText(t *testing.T) {
h := &Handler{}
want := "明白\n\nBash\nIN\npwd\nOUT\nok"
resp := makeClaudeSSEHTTPResponse(
makeClaudeContentLine(t, "明"),
makeClaudeContentLine(t, "白\n\nBash\nIN\npwd\n"),
makeClaudeContentLine(t, "OUT\nok"),
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "use tool"}}, false, false, []string{"Bash"}, nil)
frames := parseClaudeFrames(t, rec.Body.String())
if got := collectClaudeTextDeltas(frames); got != want {
t.Fatalf("unexpected combined text: got %q want %q body=%s", got, want, rec.Body.String())
}
}
func TestHandleClaudeStreamRealtimeTrimsContinuationReplay(t *testing.T) {
h := &Handler{}
prefix := strings.Repeat("A", 40)
resp := makeClaudeSSEHTTPResponse(
`data: {"p":"response/content","v":"`+prefix+`"}`,
`data: {"p":"response/content","v":"`+prefix+` tail"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/anthropic/v1/messages", nil)
h.handleClaudeStreamRealtime(rec, req, resp, "claude-sonnet-4-5", []any{map[string]any{"role": "user", "content": "hi"}}, false, false, nil, nil)
frames := parseClaudeFrames(t, rec.Body.String())
combined := strings.Builder{}
for _, f := range findClaudeFrames(frames, "content_block_delta") {
delta, _ := f.Payload["delta"].(map[string]any)
if delta["type"] == "text_delta" {
combined.WriteString(asString(delta["text"]))
}
}
if got, want := combined.String(), prefix+" tail"; got != want {
t.Fatalf("unexpected combined text: got %q want %q body=%s", got, want, rec.Body.String())
}
}
func TestHandleClaudeStreamRealtimeThinkingDelta(t *testing.T) {
h := &Handler{}
resp := makeClaudeSSEHTTPResponse(

View File

@@ -3,8 +3,6 @@ package claude
import (
"encoding/json"
"net/http"
"ds2api/internal/util"
)
func (h *Handler) CountTokens(w http.ResponseWriter, r *http.Request) {
@@ -26,26 +24,11 @@ func (h *Handler) CountTokens(w http.ResponseWriter, r *http.Request) {
writeClaudeError(w, http.StatusBadRequest, "Request must include 'model' and 'messages'.")
return
}
inputTokens := 0
if sys, ok := req["system"].(string); ok {
inputTokens += util.EstimateTokens(sys)
}
for _, item := range messages {
msg, ok := item.(map[string]any)
if !ok {
continue
}
inputTokens += 2
inputTokens += util.EstimateTokens(extractMessageContent(msg["content"]))
}
if tools, ok := req["tools"].([]any); ok {
for _, t := range tools {
b, _ := json.Marshal(t)
inputTokens += util.EstimateTokens(string(b))
}
}
if inputTokens < 1 {
inputTokens = 1
normalized, err := normalizeClaudeRequest(h.Store, req)
if err != nil {
writeClaudeError(w, http.StatusBadRequest, err.Error())
return
}
inputTokens := countClaudeInputTokens(normalized.Standard)
writeJSON(w, http.StatusOK, map[string]any{"input_tokens": inputTokens})
}

View File

@@ -0,0 +1,7 @@
package claude
import "ds2api/internal/prompt"
func buildClaudePromptTokenText(messages []any, thinkingEnabled bool) string {
return prompt.MessagesPrepareWithThinking(toMessageMaps(messages), thinkingEnabled)
}

View File

@@ -48,17 +48,18 @@ func normalizeClaudeRequest(store ConfigReader, req map[string]any) (claudeNorma
return claudeNormalizedRequest{
Standard: promptcompat.StandardRequest{
Surface: "anthropic_messages",
RequestedModel: strings.TrimSpace(model),
ResolvedModel: dsModel,
ResponseModel: strings.TrimSpace(model),
Messages: payload["messages"].([]any),
ToolsRaw: toolsRequested,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
Surface: "anthropic_messages",
RequestedModel: strings.TrimSpace(model),
ResolvedModel: dsModel,
ResponseModel: strings.TrimSpace(model),
Messages: payload["messages"].([]any),
PromptTokenText: finalPrompt,
ToolsRaw: toolsRequested,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: util.ToBool(req["stream"]),
Thinking: thinkingEnabled,
Search: searchEnabled,
},
NormalizedMessages: normalizedMessages,
}, nil

View File

@@ -8,6 +8,8 @@ import (
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
"ds2api/internal/toolcall"
"ds2api/internal/toolstream"
)
type claudeStreamRuntime struct {
@@ -15,10 +17,11 @@ type claudeStreamRuntime struct {
rc *http.ResponseController
canFlush bool
model string
toolNames []string
messages []any
toolsRaw any
model string
toolNames []string
messages []any
toolsRaw any
promptTokenText string
thinkingEnabled bool
searchEnabled bool
@@ -29,11 +32,18 @@ type claudeStreamRuntime struct {
thinking strings.Builder
text strings.Builder
sieve toolstream.State
rawText strings.Builder
rawThinking strings.Builder
toolDetectionThinking strings.Builder
toolCallsDetected bool
nextBlockIndex int
thinkingBlockOpen bool
thinkingBlockIndex int
textBlockOpen bool
textBlockIndex int
textEmitted bool
ended bool
upstreamErr string
}
@@ -49,6 +59,7 @@ func newClaudeStreamRuntime(
stripReferenceMarkers bool,
toolNames []string,
toolsRaw any,
promptTokenText string,
) *claudeStreamRuntime {
return &claudeStreamRuntime{
w: w,
@@ -62,6 +73,7 @@ func newClaudeStreamRuntime(
stripReferenceMarkers: stripReferenceMarkers,
toolNames: toolNames,
toolsRaw: toolsRaw,
promptTokenText: promptTokenText,
messageID: fmt.Sprintf("msg_%d", time.Now().UnixNano()),
thinkingBlockIndex: -1,
textBlockIndex: -1,
@@ -81,8 +93,28 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
}
contentSeen := false
for _, p := range parsed.ToolDetectionThinkingParts {
trimmed := sse.TrimContinuationOverlapFromBuilder(&s.toolDetectionThinking, p.Text)
if trimmed != "" {
s.toolDetectionThinking.WriteString(trimmed)
}
}
for _, p := range parsed.Parts {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
var rawTrimmed string
if p.Type == "thinking" {
rawTrimmed = sse.TrimContinuationOverlapFromBuilder(&s.rawThinking, p.Text)
} else {
rawTrimmed = sse.TrimContinuationOverlapFromBuilder(&s.rawText, p.Text)
}
if rawTrimmed == "" {
continue
}
if p.Type == "thinking" {
s.rawThinking.WriteString(rawTrimmed)
} else {
s.rawText.WriteString(rawTrimmed)
}
cleanedText := cleanVisibleOutput(rawTrimmed, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
@@ -95,7 +127,7 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
if !s.thinkingEnabled {
continue
}
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
trimmed := sse.TrimContinuationOverlapFromBuilder(&s.thinking, cleanedText)
if trimmed == "" {
continue
}
@@ -125,44 +157,80 @@ func (s *claudeStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
continue
}
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
continue
}
s.text.WriteString(trimmed)
if s.bufferToolContent {
if hasUnclosedCodeFence(s.text.String()) {
continue
s.text.WriteString(cleanedText)
if !s.bufferToolContent {
s.closeThinkingBlock()
if !s.textBlockOpen {
s.textBlockIndex = s.nextBlockIndex
s.nextBlockIndex++
s.send("content_block_start", map[string]any{
"type": "content_block_start",
"index": s.textBlockIndex,
"content_block": map[string]any{
"type": "text",
"text": "",
},
})
s.textBlockOpen = true
}
continue
}
s.closeThinkingBlock()
if !s.textBlockOpen {
s.textBlockIndex = s.nextBlockIndex
s.nextBlockIndex++
s.send("content_block_start", map[string]any{
"type": "content_block_start",
s.send("content_block_delta", map[string]any{
"type": "content_block_delta",
"index": s.textBlockIndex,
"content_block": map[string]any{
"type": "text",
"text": "",
"delta": map[string]any{
"type": "text_delta",
"text": cleanedText,
},
})
s.textBlockOpen = true
s.textEmitted = true
continue
}
events := toolstream.ProcessChunk(&s.sieve, rawTrimmed, s.toolNames)
for _, evt := range events {
if len(evt.ToolCalls) > 0 {
s.closeTextBlock()
s.toolCallsDetected = true
normalized := toolcall.NormalizeParsedToolCallsForSchemas(evt.ToolCalls, s.toolsRaw)
for _, tc := range normalized {
idx := s.nextBlockIndex
s.nextBlockIndex++
s.sendToolUseBlock(idx, tc)
}
continue
}
if evt.Content == "" {
continue
}
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" || (s.searchEnabled && sse.IsCitation(cleaned)) {
continue
}
s.closeThinkingBlock()
if !s.textBlockOpen {
s.textBlockIndex = s.nextBlockIndex
s.nextBlockIndex++
s.send("content_block_start", map[string]any{
"type": "content_block_start",
"index": s.textBlockIndex,
"content_block": map[string]any{
"type": "text",
"text": "",
},
})
s.textBlockOpen = true
}
s.send("content_block_delta", map[string]any{
"type": "content_block_delta",
"index": s.textBlockIndex,
"delta": map[string]any{
"type": "text_delta",
"text": cleaned,
},
})
s.textEmitted = true
}
s.send("content_block_delta", map[string]any{
"type": "content_block_delta",
"index": s.textBlockIndex,
"delta": map[string]any{
"type": "text_delta",
"text": trimmed,
},
})
}
return streamengine.ParsedDecision{ContentSeen: contentSeen}
}
func hasUnclosedCodeFence(text string) bool {
return strings.Count(text, "```")%2 == 1
}

View File

@@ -42,7 +42,10 @@ func (s *claudeStreamRuntime) sendPing() {
}
func (s *claudeStreamRuntime) sendMessageStart() {
inputTokens := util.EstimateTokens(fmt.Sprintf("%v", s.messages))
inputTokens := countClaudeInputTokensFromText(s.promptTokenText, s.model)
if inputTokens == 0 {
inputTokens = util.CountPromptTokens(fmt.Sprintf("%v", s.messages), s.model)
}
s.send("message_start", map[string]any{
"type": "message_start",
"message": map[string]any{

View File

@@ -1,13 +1,15 @@
package claude
import (
"ds2api/internal/assistantturn"
"ds2api/internal/sse"
"ds2api/internal/toolcall"
"ds2api/internal/toolstream"
"encoding/json"
"fmt"
"time"
streamengine "ds2api/internal/stream"
"ds2api/internal/util"
)
func (s *claudeStreamRuntime) closeThinkingBlock() {
@@ -34,6 +36,32 @@ func (s *claudeStreamRuntime) closeTextBlock() {
s.textBlockIndex = -1
}
func (s *claudeStreamRuntime) sendToolUseBlock(idx int, tc toolcall.ParsedToolCall) {
s.send("content_block_start", map[string]any{
"type": "content_block_start",
"index": idx,
"content_block": map[string]any{
"type": "tool_use",
"id": fmt.Sprintf("toolu_%d_%d", time.Now().Unix(), idx),
"name": tc.Name,
"input": map[string]any{},
},
})
inputBytes, _ := json.Marshal(tc.Input)
s.send("content_block_delta", map[string]any{
"type": "content_block_delta",
"index": idx,
"delta": map[string]any{
"type": "input_json_delta",
"partial_json": string(inputBytes),
},
})
s.send("content_block_stop", map[string]any{
"type": "content_block_stop",
"index": idx,
})
}
func (s *claudeStreamRuntime) finalize(stopReason string) {
if s.ended {
return
@@ -41,49 +69,80 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
s.ended = true
s.closeThinkingBlock()
s.closeTextBlock()
finalThinking := s.thinking.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
if s.bufferToolContent {
detected := toolcall.ParseStandaloneToolCalls(finalText, s.toolNames)
if len(detected) == 0 && finalText == "" && finalThinking != "" {
detected = toolcall.ParseStandaloneToolCalls(finalThinking, s.toolNames)
}
if len(detected) > 0 {
detected = toolcall.NormalizeParsedToolCallsForSchemas(detected, s.toolsRaw)
stopReason = "tool_use"
for i, tc := range detected {
idx := s.nextBlockIndex + i
s.send("content_block_start", map[string]any{
"type": "content_block_start",
"index": idx,
"content_block": map[string]any{
"type": "tool_use",
"id": fmt.Sprintf("toolu_%d_%d", time.Now().Unix(), idx),
"name": tc.Name,
"input": map[string]any{},
},
})
inputBytes, _ := json.Marshal(tc.Input)
for _, evt := range toolstream.Flush(&s.sieve, s.toolNames) {
if len(evt.ToolCalls) > 0 {
s.closeTextBlock()
s.toolCallsDetected = true
normalized := toolcall.NormalizeParsedToolCallsForSchemas(evt.ToolCalls, s.toolsRaw)
for _, tc := range normalized {
idx := s.nextBlockIndex
s.nextBlockIndex++
s.sendToolUseBlock(idx, tc)
}
continue
}
if evt.Content != "" {
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" || (s.searchEnabled && sse.IsCitation(cleaned)) {
continue
}
if !s.textBlockOpen {
s.textBlockIndex = s.nextBlockIndex
s.nextBlockIndex++
s.send("content_block_start", map[string]any{
"type": "content_block_start",
"index": s.textBlockIndex,
"content_block": map[string]any{
"type": "text",
"text": "",
},
})
s.textBlockOpen = true
}
s.send("content_block_delta", map[string]any{
"type": "content_block_delta",
"index": idx,
"index": s.textBlockIndex,
"delta": map[string]any{
"type": "input_json_delta",
"partial_json": string(inputBytes),
"type": "text_delta",
"text": cleaned,
},
})
s.send("content_block_stop", map[string]any{
"type": "content_block_stop",
"index": idx,
})
s.textEmitted = true
}
s.nextBlockIndex += len(detected)
} else if finalText != "" {
}
}
s.closeTextBlock()
turn := assistantturn.BuildTurnFromStreamSnapshot(assistantturn.StreamSnapshot{
RawText: s.rawText.String(),
VisibleText: s.text.String(),
RawThinking: s.rawThinking.String(),
VisibleThinking: s.thinking.String(),
DetectionThinking: s.toolDetectionThinking.String(),
AlreadyEmittedCalls: s.toolCallsDetected,
AlreadyEmittedToolRaw: s.toolCallsDetected,
}, assistantturn.BuildOptions{
Model: s.model,
Prompt: s.promptTokenText,
SearchEnabled: s.searchEnabled,
StripReferenceMarkers: s.stripReferenceMarkers,
ToolNames: s.toolNames,
ToolsRaw: s.toolsRaw,
})
finalText := turn.Text
if s.bufferToolContent && !s.toolCallsDetected {
if len(turn.ToolCalls) > 0 {
stopReason = "tool_use"
for _, tc := range turn.ToolCalls {
idx := s.nextBlockIndex
s.nextBlockIndex++
s.sendToolUseBlock(idx, tc)
}
} else if finalText != "" && !s.textEmitted {
idx := s.nextBlockIndex
s.nextBlockIndex++
s.send("content_block_start", map[string]any{
@@ -102,6 +161,7 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
"text": finalText,
},
})
s.textEmitted = true
s.send("content_block_stop", map[string]any{
"type": "content_block_stop",
"index": idx,
@@ -109,7 +169,10 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
}
}
outputTokens := util.EstimateTokens(finalThinking) + util.EstimateTokens(finalText)
if s.toolCallsDetected {
stopReason = "tool_use"
}
s.send("message_delta", map[string]any{
"type": "message_delta",
"delta": map[string]any{
@@ -117,7 +180,7 @@ func (s *claudeStreamRuntime) finalize(stopReason string) {
"stop_sequence": nil,
},
"usage": map[string]any{
"output_tokens": outputTokens,
"output_tokens": turn.Usage.OutputTokens,
},
})
s.send("message_stop", map[string]any{"type": "message_stop"})

View File

@@ -0,0 +1,20 @@
package claude
import (
"strings"
"ds2api/internal/promptcompat"
"ds2api/internal/util"
)
func countClaudeInputTokens(stdReq promptcompat.StandardRequest) int {
promptText := stdReq.PromptTokenText
if strings.TrimSpace(promptText) == "" {
promptText = stdReq.FinalPrompt
}
return countClaudeInputTokensFromText(promptText, stdReq.ResolvedModel)
}
func countClaudeInputTokensFromText(promptText, model string) int {
return util.CountPromptTokens(promptText, model)
}

View File

@@ -33,19 +33,24 @@ func normalizeGeminiRequest(store ConfigReader, routeModel string, req map[strin
toolsRaw := convertGeminiTools(req["tools"])
finalPrompt, toolNames := promptcompat.BuildOpenAIPromptForAdapter(messagesRaw, toolsRaw, "", thinkingEnabled)
if len(toolNames) == 0 && len(toolsRaw) > 0 {
toolNames = []string{"__any_tool__"}
}
passThrough := collectGeminiPassThrough(req)
return promptcompat.StandardRequest{
Surface: "google_gemini",
RequestedModel: requestedModel,
ResolvedModel: resolvedModel,
ResponseModel: requestedModel,
Messages: messagesRaw,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: stream,
Thinking: thinkingEnabled,
Search: searchEnabled,
PassThrough: passThrough,
Surface: "google_gemini",
RequestedModel: requestedModel,
ResolvedModel: resolvedModel,
ResponseModel: requestedModel,
Messages: messagesRaw,
PromptTokenText: finalPrompt,
ToolsRaw: toolsRaw,
FinalPrompt: finalPrompt,
ToolNames: toolNames,
Stream: stream,
Thinking: thinkingEnabled,
Search: searchEnabled,
PassThrough: passThrough,
}, nil
}

View File

@@ -2,8 +2,8 @@ package gemini
import (
"bytes"
"ds2api/internal/toolcall"
"encoding/json"
"errors"
"io"
"net/http"
"net/http/httptest"
@@ -11,7 +11,13 @@ import (
"github.com/go-chi/chi/v5"
"ds2api/internal/assistantturn"
"ds2api/internal/auth"
"ds2api/internal/completionruntime"
"ds2api/internal/httpapi/requestbody"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
"ds2api/internal/toolcall"
"ds2api/internal/translatorcliproxy"
"ds2api/internal/util"
@@ -19,20 +25,90 @@ import (
)
func (h *Handler) handleGenerateContent(w http.ResponseWriter, r *http.Request, stream bool) {
if h.OpenAI == nil {
writeGeminiError(w, http.StatusInternalServerError, "OpenAI proxy backend unavailable.")
if isGeminiVercelProxyRequest(r) && h.proxyViaOpenAI(w, r, stream) {
return
}
if h.proxyViaOpenAI(w, r, stream) {
if h.Auth == nil || h.DS == nil {
if h.OpenAI != nil && h.proxyViaOpenAI(w, r, stream) {
return
}
writeGeminiError(w, http.StatusInternalServerError, "Gemini runtime backend unavailable.")
return
}
writeGeminiError(w, http.StatusBadGateway, "Failed to proxy Gemini request.")
if h.handleGeminiDirect(w, r, stream) {
return
}
writeGeminiError(w, http.StatusBadGateway, "Failed to handle Gemini request.")
}
func isGeminiVercelProxyRequest(r *http.Request) bool {
if r == nil || r.URL == nil {
return false
}
return strings.TrimSpace(r.URL.Query().Get("__stream_prepare")) == "1" ||
strings.TrimSpace(r.URL.Query().Get("__stream_release")) == "1"
}
func (h *Handler) handleGeminiDirect(w http.ResponseWriter, r *http.Request, stream bool) bool {
raw, err := io.ReadAll(r.Body)
if err != nil {
if errors.Is(err, requestbody.ErrInvalidUTF8Body) {
writeGeminiError(w, http.StatusBadRequest, "invalid json")
} else {
writeGeminiError(w, http.StatusBadRequest, "invalid body")
}
return true
}
routeModel := strings.TrimSpace(chi.URLParam(r, "model"))
var req map[string]any
if err := json.Unmarshal(raw, &req); err != nil {
writeGeminiError(w, http.StatusBadRequest, "invalid json")
return true
}
stdReq, err := normalizeGeminiRequest(h.Store, routeModel, req, stream)
if err != nil {
writeGeminiError(w, http.StatusBadRequest, err.Error())
return true
}
a, err := h.Auth.Determine(r)
if err != nil {
writeGeminiError(w, http.StatusUnauthorized, err.Error())
return true
}
defer h.Auth.Release(a)
if stream {
h.handleGeminiDirectStream(w, r, a, stdReq)
return true
}
result, outErr := completionruntime.ExecuteNonStreamWithRetry(r.Context(), h.DS, a, stdReq, completionruntime.Options{
StripReferenceMarkers: h.compatStripReferenceMarkers(),
RetryEnabled: true,
})
if outErr != nil {
writeGeminiError(w, outErr.Status, outErr.Message)
return true
}
writeJSON(w, http.StatusOK, buildGeminiGenerateContentResponseFromTurn(result.Turn))
return true
}
func (h *Handler) handleGeminiDirectStream(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, stdReq promptcompat.StandardRequest) {
start, outErr := completionruntime.StartCompletion(r.Context(), h.DS, a, stdReq, completionruntime.Options{})
if outErr != nil {
writeGeminiError(w, outErr.Status, outErr.Message)
return
}
h.handleStreamGenerateContent(w, r, start.Response, stdReq.ResponseModel, stdReq.PromptTokenText, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw)
}
func (h *Handler) proxyViaOpenAI(w http.ResponseWriter, r *http.Request, stream bool) bool {
raw, err := io.ReadAll(r.Body)
if err != nil {
writeGeminiError(w, http.StatusBadRequest, "invalid body")
if errors.Is(err, requestbody.ErrInvalidUTF8Body) {
writeGeminiError(w, http.StatusBadRequest, "invalid json")
} else {
writeGeminiError(w, http.StatusBadRequest, "invalid body")
}
return true
}
routeModel := strings.TrimSpace(chi.URLParam(r, "model"))
@@ -227,7 +303,7 @@ func (h *Handler) handleNonStreamGenerateContent(w http.ResponseWriter, resp *ht
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, finalText string, toolNames []string) map[string]any {
parts := buildGeminiPartsFromFinal(finalText, finalThinking, toolNames)
usage := buildGeminiUsage(finalPrompt, finalThinking, finalText)
usage := buildGeminiUsage(model, finalPrompt, finalThinking, finalText)
return map[string]any{
"candidates": []map[string]any{
{
@@ -244,11 +320,53 @@ func buildGeminiGenerateContentResponse(model, finalPrompt, finalThinking, final
}
}
func buildGeminiGenerateContentResponseFromTurn(turn assistantturn.Turn) map[string]any {
parts := buildGeminiPartsFromTurn(turn)
return map[string]any{
"candidates": []map[string]any{
{
"index": 0,
"content": map[string]any{
"role": "model",
"parts": parts,
},
"finishReason": "STOP",
},
},
"modelVersion": turn.Model,
"usageMetadata": map[string]any{
"promptTokenCount": turn.Usage.InputTokens,
"candidatesTokenCount": turn.Usage.OutputTokens,
"totalTokenCount": turn.Usage.TotalTokens,
},
}
}
func buildGeminiPartsFromTurn(turn assistantturn.Turn) []map[string]any {
if len(turn.ToolCalls) > 0 {
parts := make([]map[string]any, 0, len(turn.ToolCalls))
for _, tc := range turn.ToolCalls {
parts = append(parts, map[string]any{
"functionCall": map[string]any{
"name": tc.Name,
"args": tc.Input,
},
})
}
return parts
}
text := turn.Text
if text == "" {
text = turn.Thinking
}
return []map[string]any{{"text": text}}
}
//nolint:unused // retained for native Gemini non-stream handling path.
func buildGeminiUsage(finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.EstimateTokens(finalPrompt)
reasoningTokens := util.EstimateTokens(finalThinking)
completionTokens := util.EstimateTokens(finalText)
func buildGeminiUsage(model, finalPrompt, finalThinking, finalText string) map[string]any {
promptTokens := util.CountPromptTokens(finalPrompt, model)
reasoningTokens := util.CountOutputTokens(finalThinking, model)
completionTokens := util.CountOutputTokens(finalText, model)
return map[string]any{
"promptTokenCount": promptTokens,
"candidatesTokenCount": reasoningTokens + completionTokens,

View File

@@ -7,13 +7,14 @@ import (
"strings"
"time"
"ds2api/internal/assistantturn"
dsprotocol "ds2api/internal/deepseek/protocol"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
)
//nolint:unused // retained for native Gemini stream handling path.
func (h *Handler) handleStreamGenerateContent(w http.ResponseWriter, r *http.Request, resp *http.Response, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string) {
func (h *Handler) handleStreamGenerateContent(w http.ResponseWriter, r *http.Request, resp *http.Response, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -28,7 +29,7 @@ func (h *Handler) handleStreamGenerateContent(w http.ResponseWriter, r *http.Req
rc := http.NewResponseController(w)
_, canFlush := w.(http.Flusher)
runtime := newGeminiStreamRuntime(w, rc, canFlush, model, finalPrompt, thinkingEnabled, searchEnabled, h.compatStripReferenceMarkers(), toolNames)
runtime := newGeminiStreamRuntime(w, rc, canFlush, model, finalPrompt, thinkingEnabled, searchEnabled, h.compatStripReferenceMarkers(), toolNames, toolsRaw)
initialType := "text"
if thinkingEnabled {
@@ -64,9 +65,11 @@ type geminiStreamRuntime struct {
bufferContent bool
stripReferenceMarkers bool
toolNames []string
toolsRaw any
thinking strings.Builder
text strings.Builder
accumulator *assistantturn.Accumulator
contentFilter bool
responseMessageID int
}
//nolint:unused // retained for native Gemini stream handling path.
@@ -80,6 +83,7 @@ func newGeminiStreamRuntime(
searchEnabled bool,
stripReferenceMarkers bool,
toolNames []string,
toolsRaw any,
) *geminiStreamRuntime {
return &geminiStreamRuntime{
w: w,
@@ -92,6 +96,12 @@ func newGeminiStreamRuntime(
bufferContent: len(toolNames) > 0,
stripReferenceMarkers: stripReferenceMarkers,
toolNames: toolNames,
toolsRaw: toolsRaw,
accumulator: assistantturn.NewAccumulator(assistantturn.AccumulatorOptions{
ThinkingEnabled: thinkingEnabled,
SearchEnabled: searchEnabled,
StripReferenceMarkers: stripReferenceMarkers,
}),
}
}
@@ -111,35 +121,24 @@ func (s *geminiStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
if !parsed.Parsed {
return streamengine.ParsedDecision{}
}
if parsed.ResponseMessageID > 0 {
s.responseMessageID = parsed.ResponseMessageID
}
if parsed.ContentFilter || parsed.ErrorMessage != "" || parsed.Stop {
if parsed.ContentFilter {
s.contentFilter = true
}
return streamengine.ParsedDecision{Stop: true}
}
contentSeen := false
for _, p := range parsed.Parts {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
contentSeen = true
accumulated := s.accumulator.Apply(parsed)
for _, p := range accumulated.Parts {
if p.Type == "thinking" {
if s.thinkingEnabled {
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
}
s.thinking.WriteString(trimmed)
}
continue
}
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
if p.RawText == "" || p.CitationOnly || p.VisibleText == "" {
continue
}
s.text.WriteString(trimmed)
if s.bufferContent {
continue
}
@@ -149,23 +148,38 @@ func (s *geminiStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Parse
"index": 0,
"content": map[string]any{
"role": "model",
"parts": []map[string]any{{"text": trimmed}},
"parts": []map[string]any{{"text": p.VisibleText}},
},
},
},
"modelVersion": s.model,
})
}
return streamengine.ParsedDecision{ContentSeen: contentSeen}
return streamengine.ParsedDecision{ContentSeen: accumulated.ContentSeen}
}
//nolint:unused // retained for native Gemini stream handling path.
func (s *geminiStreamRuntime) finalize() {
finalThinking := s.thinking.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
rawText, text, rawThinking, thinking, detectionThinking := s.accumulator.Snapshot()
turn := assistantturn.BuildTurnFromStreamSnapshot(assistantturn.StreamSnapshot{
RawText: rawText,
VisibleText: text,
RawThinking: rawThinking,
VisibleThinking: thinking,
DetectionThinking: detectionThinking,
ContentFilter: s.contentFilter,
ResponseMessageID: s.responseMessageID,
}, assistantturn.BuildOptions{
Model: s.model,
Prompt: s.finalPrompt,
SearchEnabled: s.searchEnabled,
StripReferenceMarkers: s.stripReferenceMarkers,
ToolNames: s.toolNames,
ToolsRaw: s.toolsRaw,
})
if s.bufferContent {
parts := buildGeminiPartsFromFinal(finalText, finalThinking, s.toolNames)
parts := buildGeminiPartsFromTurn(turn)
s.sendChunk(map[string]any{
"candidates": []map[string]any{
{
@@ -193,7 +207,11 @@ func (s *geminiStreamRuntime) finalize() {
"finishReason": "STOP",
},
},
"modelVersion": s.model,
"usageMetadata": buildGeminiUsage(s.finalPrompt, finalThinking, finalText),
"modelVersion": s.model,
"usageMetadata": map[string]any{
"promptTokenCount": turn.Usage.InputTokens,
"candidatesTokenCount": turn.Usage.OutputTokens,
"totalTokenCount": turn.Usage.TotalTokens,
},
})
}

View File

@@ -126,6 +126,7 @@ func TestStartChatHistoryRecoversFromTransientWriteFailure(t *testing.T) {
session := startChatHistory(historyStore, req, a, stdReq)
if session == nil {
t.Fatalf("expected session even when initial persistence fails")
return
}
if session.disabled {
t.Fatalf("expected session to remain active after transient start failure")
@@ -194,7 +195,7 @@ func TestHandleStreamContextCancelledMarksHistoryStopped(t *testing.T) {
rec := httptest.NewRecorder()
resp := makeOpenAISSEHTTPResponse(`data: {"p":"response/content","v":"hello"}`, `data: [DONE]`)
h.handleStream(rec, req, resp, "cid-stop", "deepseek-v4-flash", "prompt", false, false, nil, nil, session)
h.handleStream(rec, req, resp, "cid-stop", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, session)
snapshot, err := historyStore.Snapshot()
if err != nil {
@@ -310,16 +311,16 @@ func TestChatCompletionsCurrentInputFilePersistsNeutralPrompt(t *testing.T) {
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected current input upload to happen, got %d", len(ds.uploadCalls))
}
if ds.uploadCalls[0].Filename != "IGNORE.txt" {
t.Fatalf("expected IGNORE.txt upload, got %q", ds.uploadCalls[0].Filename)
if ds.uploadCalls[0].Filename != "DS2API_HISTORY.txt" {
t.Fatalf("expected DS2API_HISTORY.txt upload, got %q", ds.uploadCalls[0].Filename)
}
if full.HistoryText != string(ds.uploadCalls[0].Data) {
t.Fatalf("expected uploaded current input file to be persisted in history text")
}
if len(full.Messages) != 1 {
t.Fatalf("expected neutral prompt to be the only persisted message, got %#v", full.Messages)
t.Fatalf("expected continuation prompt to be the only persisted message, got %#v", full.Messages)
}
if !strings.Contains(full.Messages[0].Content, "Answer the latest user request directly.") {
t.Fatalf("expected neutral prompt to be persisted, got %#v", full.Messages[0])
if !strings.Contains(full.Messages[0].Content, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected continuation prompt to be persisted, got %#v", full.Messages[0])
}
}

View File

@@ -5,7 +5,10 @@ import (
"net/http"
"strings"
"ds2api/internal/assistantturn"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
"ds2api/internal/toolstream"
@@ -16,12 +19,14 @@ type chatStreamRuntime struct {
rc *http.ResponseController
canFlush bool
completionID string
created int64
model string
finalPrompt string
toolNames []string
toolsRaw any
completionID string
created int64
model string
finalPrompt string
refFileTokens int
toolNames []string
toolsRaw any
toolChoice promptcompat.ToolChoicePolicy
thinkingEnabled bool
searchEnabled bool
@@ -33,13 +38,11 @@ type chatStreamRuntime struct {
toolCallsEmitted bool
toolCallsDoneEmitted bool
toolSieve toolstream.State
streamToolCallIDs map[int]string
streamToolNames map[int]string
thinking strings.Builder
toolDetectionThinking strings.Builder
text strings.Builder
responseMessageID int
toolSieve toolstream.State
streamToolCallIDs map[int]string
streamToolNames map[int]string
accumulator shared.StreamAccumulator
responseMessageID int
finalThinking string
finalText string
@@ -50,6 +53,32 @@ type chatStreamRuntime struct {
finalErrorCode string
}
type chatDeltaBatch struct {
runtime *chatStreamRuntime
field string
text strings.Builder
}
func (b *chatDeltaBatch) append(field, text string) {
if text == "" {
return
}
if b.field != "" && b.field != field {
b.flush()
}
b.field = field
b.text.WriteString(text)
}
func (b *chatDeltaBatch) flush() {
if b.field == "" || b.text.Len() == 0 {
return
}
b.runtime.sendDelta(map[string]any{b.field: b.text.String()})
b.field = ""
b.text.Reset()
}
func newChatStreamRuntime(
w http.ResponseWriter,
rc *http.ResponseController,
@@ -63,6 +92,7 @@ func newChatStreamRuntime(
stripReferenceMarkers bool,
toolNames []string,
toolsRaw any,
toolChoice promptcompat.ToolChoicePolicy,
bufferToolContent bool,
emitEarlyToolDeltas bool,
) *chatStreamRuntime {
@@ -76,6 +106,7 @@ func newChatStreamRuntime(
finalPrompt: finalPrompt,
toolNames: toolNames,
toolsRaw: toolsRaw,
toolChoice: toolChoice,
thinkingEnabled: thinkingEnabled,
searchEnabled: searchEnabled,
stripReferenceMarkers: stripReferenceMarkers,
@@ -83,6 +114,11 @@ func newChatStreamRuntime(
emitEarlyToolDeltas: emitEarlyToolDeltas,
streamToolCallIDs: map[int]string{},
streamToolNames: map[int]string{},
accumulator: shared.StreamAccumulator{
ThinkingEnabled: thinkingEnabled,
SearchEnabled: searchEnabled,
StripReferenceMarkers: stripReferenceMarkers,
},
}
}
@@ -91,7 +127,13 @@ func (s *chatStreamRuntime) sendKeepAlive() {
return
}
_, _ = s.w.Write([]byte(": keep-alive\n\n"))
_ = s.rc.Flush()
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
s.model,
[]map[string]any{},
nil,
))
}
func (s *chatStreamRuntime) sendChunk(v any) {
@@ -104,6 +146,23 @@ func (s *chatStreamRuntime) sendChunk(v any) {
}
}
func (s *chatStreamRuntime) sendDelta(delta map[string]any) {
if len(delta) == 0 {
return
}
if !s.firstChunkSent {
delta["role"] = "assistant"
s.firstChunkSent = true
}
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
s.model,
[]map[string]any{openaifmt.BuildChatStreamDeltaChoice(0, delta)},
nil,
))
}
func (s *chatStreamRuntime) sendDone() {
_, _ = s.w.Write([]byte("data: [DONE]\n\n"))
if s.canFlush {
@@ -127,6 +186,15 @@ func (s *chatStreamRuntime) sendFailedChunk(status int, message, code string) {
s.sendDone()
}
func (s *chatStreamRuntime) markContextCancelled() {
s.finalErrorStatus = 499
s.finalErrorMessage = "request context cancelled"
s.finalErrorCode = string(streamengine.StopReasonContextCancelled)
s.finalThinking = s.accumulator.Thinking.String()
s.finalText = cleanVisibleOutput(s.accumulator.Text.String(), s.stripReferenceMarkers)
s.finalFinishReason = string(streamengine.StopReasonContextCancelled)
}
func (s *chatStreamRuntime) resetStreamToolCallState() {
s.streamToolCallIDs = map[int]string{}
s.streamToolNames = map[int]string{}
@@ -136,81 +204,71 @@ func (s *chatStreamRuntime) finalize(finishReason string, deferEmptyOutput bool)
s.finalErrorStatus = 0
s.finalErrorMessage = ""
s.finalErrorCode = ""
finalThinking := s.thinking.String()
finalToolDetectionThinking := s.toolDetectionThinking.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
s.finalThinking = finalThinking
s.finalText = finalText
detected := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, s.toolNames)
if len(detected.Calls) > 0 && !s.toolCallsDoneEmitted {
finalThinking := s.accumulator.Thinking.String()
finalToolDetectionThinking := s.accumulator.ToolDetectionThinking.String()
finalText := s.accumulator.Text.String()
turn := assistantturn.BuildTurnFromStreamSnapshot(assistantturn.StreamSnapshot{
RawText: s.accumulator.RawText.String(),
VisibleText: finalText,
RawThinking: s.accumulator.RawThinking.String(),
VisibleThinking: finalThinking,
DetectionThinking: finalToolDetectionThinking,
ContentFilter: finishReason == "content_filter",
ResponseMessageID: s.responseMessageID,
AlreadyEmittedCalls: s.toolCallsEmitted,
AlreadyEmittedToolRaw: s.toolCallsDoneEmitted,
}, assistantturn.BuildOptions{
Model: s.model,
Prompt: s.finalPrompt,
RefFileTokens: s.refFileTokens,
SearchEnabled: s.searchEnabled,
StripReferenceMarkers: s.stripReferenceMarkers,
ToolNames: s.toolNames,
ToolsRaw: s.toolsRaw,
ToolChoice: s.toolChoice,
})
s.finalThinking = turn.Thinking
s.finalText = turn.Text
if len(turn.ToolCalls) > 0 && !s.toolCallsDoneEmitted {
finishReason = "tool_calls"
delta := map[string]any{
"tool_calls": formatFinalStreamToolCallsWithStableIDs(detected.Calls, s.streamToolCallIDs, s.toolsRaw),
}
if !s.firstChunkSent {
delta["role"] = "assistant"
s.firstChunkSent = true
}
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
s.model,
[]map[string]any{openaifmt.BuildChatStreamDeltaChoice(0, delta)},
nil,
))
s.sendDelta(map[string]any{
"tool_calls": formatFinalStreamToolCallsWithStableIDs(turn.ToolCalls, s.streamToolCallIDs, s.toolsRaw),
})
s.toolCallsEmitted = true
s.toolCallsDoneEmitted = true
} else if s.bufferToolContent {
batch := chatDeltaBatch{runtime: s}
for _, evt := range toolstream.Flush(&s.toolSieve, s.toolNames) {
if len(evt.ToolCalls) > 0 {
batch.flush()
finishReason = "tool_calls"
s.toolCallsEmitted = true
s.toolCallsDoneEmitted = true
tcDelta := map[string]any{
s.sendDelta(map[string]any{
"tool_calls": formatFinalStreamToolCallsWithStableIDs(evt.ToolCalls, s.streamToolCallIDs, s.toolsRaw),
}
if !s.firstChunkSent {
tcDelta["role"] = "assistant"
s.firstChunkSent = true
}
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
s.model,
[]map[string]any{openaifmt.BuildChatStreamDeltaChoice(0, tcDelta)},
nil,
))
})
s.resetStreamToolCallState()
}
if evt.Content == "" {
continue
}
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" {
if cleaned == "" || (s.searchEnabled && sse.IsCitation(cleaned)) {
continue
}
delta := map[string]any{
"content": cleaned,
}
if !s.firstChunkSent {
delta["role"] = "assistant"
s.firstChunkSent = true
}
s.sendChunk(openaifmt.BuildChatStreamChunk(
s.completionID,
s.created,
s.model,
[]map[string]any{openaifmt.BuildChatStreamDeltaChoice(0, delta)},
nil,
))
batch.append("content", cleaned)
}
batch.flush()
}
if len(detected.Calls) > 0 || s.toolCallsEmitted {
if len(turn.ToolCalls) > 0 || s.toolCallsEmitted {
finishReason = "tool_calls"
}
if len(detected.Calls) == 0 && !s.toolCallsEmitted && strings.TrimSpace(finalText) == "" {
status, message, code := upstreamEmptyOutputDetail(finishReason == "content_filter", finalText, finalThinking)
if len(turn.ToolCalls) == 0 && !s.toolCallsEmitted && strings.TrimSpace(turn.Text) == "" {
status, message, code := upstreamEmptyOutputDetail(finishReason == "content_filter", turn.Text, turn.Thinking)
if turn.Error != nil {
status, message, code = turn.Error.Status, turn.Error.Message, turn.Error.Code
}
if deferEmptyOutput {
s.finalErrorStatus = status
s.finalErrorMessage = message
@@ -220,7 +278,7 @@ func (s *chatStreamRuntime) finalize(finishReason string, deferEmptyOutput bool)
s.sendFailedChunk(status, message, code)
return true
}
usage := openaifmt.BuildChatUsage(s.finalPrompt, finalThinking, finalText)
usage := chatUsageFromTurn(turn)
s.finalFinishReason = finishReason
s.finalUsage = usage
s.sendChunk(openaifmt.BuildChatStreamChunk(
@@ -234,6 +292,17 @@ func (s *chatStreamRuntime) finalize(finishReason string, deferEmptyOutput bool)
return true
}
func chatUsageFromTurn(turn assistantturn.Turn) map[string]any {
return map[string]any{
"prompt_tokens": turn.Usage.InputTokens,
"completion_tokens": turn.Usage.OutputTokens,
"total_tokens": turn.Usage.TotalTokens,
"completion_tokens_details": map[string]any{
"reasoning_tokens": turn.Usage.ReasoningTokens,
},
}
}
func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedDecision {
if !parsed.Parsed {
return streamengine.ParsedDecision{}
@@ -242,7 +311,7 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
s.responseMessageID = parsed.ResponseMessageID
}
if parsed.ContentFilter {
if strings.TrimSpace(s.text.String()) == "" {
if strings.TrimSpace(s.accumulator.Text.String()) == "" {
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReason("content_filter")}
}
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReasonHandlerRequested}
@@ -254,109 +323,65 @@ func (s *chatStreamRuntime) onParsed(parsed sse.LineResult) streamengine.ParsedD
return streamengine.ParsedDecision{Stop: true, StopReason: streamengine.StopReasonHandlerRequested}
}
newChoices := make([]map[string]any, 0, len(parsed.Parts))
contentSeen := false
for _, p := range parsed.ToolDetectionThinkingParts {
trimmed := sse.TrimContinuationOverlap(s.toolDetectionThinking.String(), p.Text)
if trimmed != "" {
s.toolDetectionThinking.WriteString(trimmed)
}
}
for _, p := range parsed.Parts {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
if cleanedText == "" {
continue
}
contentSeen = true
delta := map[string]any{}
if !s.firstChunkSent {
delta["role"] = "assistant"
s.firstChunkSent = true
}
batch := chatDeltaBatch{runtime: s}
accumulated := s.accumulator.Apply(parsed)
for _, p := range accumulated.Parts {
if p.Type == "thinking" {
if s.thinkingEnabled {
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
batch.append("reasoning_content", p.VisibleText)
continue
}
if p.RawText == "" {
continue
}
if p.CitationOnly {
continue
}
if !s.bufferToolContent {
batch.append("content", p.VisibleText)
} else {
events := toolstream.ProcessChunk(&s.toolSieve, p.RawText, s.toolNames)
for _, evt := range events {
if len(evt.ToolCallDeltas) > 0 {
if !s.emitEarlyToolDeltas {
continue
}
filtered := filterIncrementalToolCallDeltasByAllowed(evt.ToolCallDeltas, s.streamToolNames)
if len(filtered) == 0 {
continue
}
formatted := formatIncrementalStreamToolCallDeltas(filtered, s.streamToolCallIDs)
if len(formatted) == 0 {
continue
}
batch.flush()
tcDelta := map[string]any{
"tool_calls": formatted,
}
s.toolCallsEmitted = true
s.sendDelta(tcDelta)
continue
}
s.thinking.WriteString(trimmed)
delta["reasoning_content"] = trimmed
}
} else {
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
continue
}
s.text.WriteString(trimmed)
if !s.bufferToolContent {
delta["content"] = trimmed
} else {
events := toolstream.ProcessChunk(&s.toolSieve, trimmed, s.toolNames)
for _, evt := range events {
if len(evt.ToolCallDeltas) > 0 {
if !s.emitEarlyToolDeltas {
continue
}
filtered := filterIncrementalToolCallDeltasByAllowed(evt.ToolCallDeltas, s.streamToolNames)
if len(filtered) == 0 {
continue
}
formatted := formatIncrementalStreamToolCallDeltas(filtered, s.streamToolCallIDs)
if len(formatted) == 0 {
continue
}
tcDelta := map[string]any{
"tool_calls": formatted,
}
s.toolCallsEmitted = true
if !s.firstChunkSent {
tcDelta["role"] = "assistant"
s.firstChunkSent = true
}
newChoices = append(newChoices, openaifmt.BuildChatStreamDeltaChoice(0, tcDelta))
if len(evt.ToolCalls) > 0 {
batch.flush()
s.toolCallsEmitted = true
s.toolCallsDoneEmitted = true
tcDelta := map[string]any{
"tool_calls": formatFinalStreamToolCallsWithStableIDs(evt.ToolCalls, s.streamToolCallIDs, s.toolsRaw),
}
s.sendDelta(tcDelta)
s.resetStreamToolCallState()
continue
}
if evt.Content != "" {
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" || (s.searchEnabled && sse.IsCitation(cleaned)) {
continue
}
if len(evt.ToolCalls) > 0 {
s.toolCallsEmitted = true
s.toolCallsDoneEmitted = true
tcDelta := map[string]any{
"tool_calls": formatFinalStreamToolCallsWithStableIDs(evt.ToolCalls, s.streamToolCallIDs, s.toolsRaw),
}
if !s.firstChunkSent {
tcDelta["role"] = "assistant"
s.firstChunkSent = true
}
newChoices = append(newChoices, openaifmt.BuildChatStreamDeltaChoice(0, tcDelta))
s.resetStreamToolCallState()
continue
}
if evt.Content != "" {
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned == "" {
continue
}
contentDelta := map[string]any{
"content": cleaned,
}
if !s.firstChunkSent {
contentDelta["role"] = "assistant"
s.firstChunkSent = true
}
newChoices = append(newChoices, openaifmt.BuildChatStreamDeltaChoice(0, contentDelta))
}
batch.append("content", cleaned)
}
}
}
if len(delta) > 0 {
newChoices = append(newChoices, openaifmt.BuildChatStreamDeltaChoice(0, delta))
}
}
if len(newChoices) > 0 {
s.sendChunk(openaifmt.BuildChatStreamChunk(s.completionID, s.created, s.model, newChoices, nil))
}
return streamengine.ParsedDecision{ContentSeen: contentSeen}
batch.flush()
return streamengine.ParsedDecision{ContentSeen: accumulated.ContentSeen}
}

View File

@@ -0,0 +1,87 @@
package chat
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"ds2api/internal/promptcompat"
)
func TestChatStreamKeepAliveEmitsEmptyChoiceDataFrame(t *testing.T) {
rec := httptest.NewRecorder()
runtime := newChatStreamRuntime(
rec,
http.NewResponseController(rec),
true,
"chatcmpl-test",
time.Now().Unix(),
"deepseek-v4-flash",
"prompt",
false,
false,
true,
nil,
nil,
promptcompat.DefaultToolChoicePolicy(),
false,
false,
)
runtime.sendKeepAlive()
body := rec.Body.String()
if !strings.Contains(body, ": keep-alive\n\n") {
t.Fatalf("expected keep-alive comment, got %q", body)
}
frames, done := parseSSEDataFrames(t, body)
if done {
t.Fatalf("keep-alive must not emit [DONE], body=%q", body)
}
if len(frames) != 1 {
t.Fatalf("expected one data frame, got %d body=%q", len(frames), body)
}
if got := asString(frames[0]["id"]); got != "chatcmpl-test" {
t.Fatalf("expected completion id to be preserved, got %q", got)
}
if got := asString(frames[0]["object"]); got != "chat.completion.chunk" {
t.Fatalf("expected chat chunk object, got %q", got)
}
choices, _ := frames[0]["choices"].([]any)
if len(choices) != 0 {
t.Fatalf("expected empty choices heartbeat, got %#v", choices)
}
}
func TestChatStreamFinalizeEnforcesRequiredToolChoice(t *testing.T) {
rec := httptest.NewRecorder()
runtime := newChatStreamRuntime(
rec,
http.NewResponseController(rec),
true,
"chatcmpl-test",
time.Now().Unix(),
"deepseek-v4-flash",
"prompt",
false,
false,
true,
[]string{"Write"},
nil,
promptcompat.ToolChoicePolicy{Mode: promptcompat.ToolChoiceRequired},
true,
false,
)
if !runtime.finalize("stop", false) {
t.Fatalf("expected terminal error to be written")
}
if runtime.finalErrorCode != "tool_choice_violation" {
t.Fatalf("expected tool_choice_violation, got %q body=%s", runtime.finalErrorCode, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "tool_choice requires") {
t.Fatalf("expected tool choice error in stream body, got %s", rec.Body.String())
}
}

View File

@@ -7,15 +7,19 @@ import (
"strings"
"time"
"ds2api/internal/assistantturn"
"ds2api/internal/auth"
"ds2api/internal/config"
dsprotocol "ds2api/internal/deepseek/protocol"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
)
type chatNonStreamResult struct {
rawThinking string
rawText string
thinking string
toolDetectionThinking string
text string
@@ -24,13 +28,15 @@ type chatNonStreamResult struct {
body map[string]any
finishReason string
responseMessageID int
outputError *assistantturn.OutputError
}
func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
attempts := 0
currentResp := resp
usagePrompt := finalPrompt
accumulatedThinking := ""
accumulatedRawThinking := ""
accumulatedToolDetectionThinking := ""
for {
result, ok := h.collectChatNonStreamAttempt(w, currentResp, completionID, model, usagePrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw)
@@ -38,15 +44,18 @@ func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Co
return
}
accumulatedThinking += sse.TrimContinuationOverlap(accumulatedThinking, result.thinking)
accumulatedRawThinking += sse.TrimContinuationOverlap(accumulatedRawThinking, result.rawThinking)
accumulatedToolDetectionThinking += sse.TrimContinuationOverlap(accumulatedToolDetectionThinking, result.toolDetectionThinking)
result.thinking = accumulatedThinking
result.rawThinking = accumulatedRawThinking
result.toolDetectionThinking = accumulatedToolDetectionThinking
detected := detectAssistantToolCalls(result.text, result.thinking, result.toolDetectionThinking, toolNames)
detected := detectAssistantToolCalls(result.rawText, result.text, result.rawThinking, result.toolDetectionThinking, toolNames)
result.detectedCalls = len(detected.Calls)
result.body = openaifmt.BuildChatCompletionWithToolCalls(completionID, model, usagePrompt, result.thinking, result.text, detected.Calls, toolsRaw)
addRefFileTokensToUsage(result.body, refFileTokens)
result.finishReason = chatFinishReason(result.body)
if !shouldRetryChatNonStream(result, attempts) {
h.finishChatNonStreamResult(w, result, attempts, usagePrompt, historySession)
h.finishChatNonStreamResult(w, result, attempts, usagePrompt, refFileTokens, historySession)
return
}
@@ -67,7 +76,7 @@ func (h *Handler) handleNonStreamWithRetry(w http.ResponseWriter, ctx context.Co
config.Logger.Warn("[openai_empty_retry] retry request failed", "surface", "chat.completions", "stream", false, "retry_attempt", attempts, "error", err)
return
}
usagePrompt = usagePromptWithEmptyOutputRetry(finalPrompt, attempts)
usagePrompt = usagePromptWithEmptyOutputRetry(usagePrompt, attempts)
currentResp = nextResp
}
}
@@ -80,39 +89,45 @@ func (h *Handler) collectChatNonStreamAttempt(w http.ResponseWriter, resp *http.
return chatNonStreamResult{}, false
}
result := sse.CollectStream(resp, thinkingEnabled, true)
stripReferenceMarkers := h.compatStripReferenceMarkers()
finalThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
finalToolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
finalText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
finalText = replaceCitationMarkersWithLinks(finalText, result.CitationLinks)
}
detected := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, toolNames)
respBody := openaifmt.BuildChatCompletionWithToolCalls(completionID, model, usagePrompt, finalThinking, finalText, detected.Calls, toolsRaw)
turn := assistantturn.BuildTurnFromCollected(result, assistantturn.BuildOptions{
Model: model,
Prompt: usagePrompt,
SearchEnabled: searchEnabled,
StripReferenceMarkers: h.compatStripReferenceMarkers(),
ToolNames: toolNames,
ToolsRaw: toolsRaw,
})
respBody := openaifmt.BuildChatCompletionWithToolCalls(completionID, model, usagePrompt, turn.Thinking, turn.Text, turn.ToolCalls, toolsRaw)
return chatNonStreamResult{
thinking: finalThinking,
toolDetectionThinking: finalToolDetectionThinking,
text: finalText,
rawThinking: result.Thinking,
rawText: result.Text,
thinking: turn.Thinking,
toolDetectionThinking: result.ToolDetectionThinking,
text: turn.Text,
contentFilter: result.ContentFilter,
detectedCalls: len(detected.Calls),
detectedCalls: len(turn.ToolCalls),
body: respBody,
finishReason: chatFinishReason(respBody),
responseMessageID: result.ResponseMessageID,
outputError: turn.Error,
}, true
}
func (h *Handler) finishChatNonStreamResult(w http.ResponseWriter, result chatNonStreamResult, attempts int, usagePrompt string, historySession *chatHistorySession) {
if result.detectedCalls == 0 && shouldWriteUpstreamEmptyOutputError(result.text) {
func (h *Handler) finishChatNonStreamResult(w http.ResponseWriter, result chatNonStreamResult, attempts int, usagePrompt string, refFileTokens int, historySession *chatHistorySession) {
if result.detectedCalls == 0 && strings.TrimSpace(result.text) == "" {
status, message, code := upstreamEmptyOutputDetail(result.contentFilter, result.text, result.thinking)
if result.outputError != nil {
status, message, code = result.outputError.Status, result.outputError.Message, result.outputError.Code
}
if historySession != nil {
historySession.error(status, message, code, result.thinking, result.text)
}
writeUpstreamEmptyOutputError(w, result.text, result.thinking, result.contentFilter)
writeOpenAIErrorWithCode(w, status, message, code)
config.Logger.Info("[openai_empty_retry] terminal empty output", "surface", "chat.completions", "stream", false, "retry_attempts", attempts, "success_source", "none", "content_filter", result.contentFilter)
return
}
if historySession != nil {
historySession.success(http.StatusOK, result.thinking, result.text, result.finishReason, openaifmt.BuildChatUsage(usagePrompt, result.thinking, result.text))
historySession.success(http.StatusOK, result.thinking, result.text, result.finishReason, openaifmt.BuildChatUsageForModel("", usagePrompt, result.thinking, result.text, refFileTokens))
}
writeJSON(w, http.StatusOK, result.body)
source := "first_attempt"
@@ -136,11 +151,12 @@ func shouldRetryChatNonStream(result chatNonStreamResult, attempts int) bool {
attempts < emptyOutputRetryMaxAttempts() &&
!result.contentFilter &&
result.detectedCalls == 0 &&
strings.TrimSpace(result.text) == ""
strings.TrimSpace(result.text) == "" &&
strings.TrimSpace(result.thinking) == ""
}
func (h *Handler) handleStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
streamRuntime, initialType, ok := h.prepareChatStreamRuntime(w, resp, completionID, model, finalPrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw, historySession)
func (h *Handler) handleStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, historySession *chatHistorySession) {
streamRuntime, initialType, ok := h.prepareChatStreamRuntime(w, resp, completionID, model, finalPrompt, refFileTokens, thinkingEnabled, searchEnabled, toolNames, toolsRaw, toolChoice, historySession)
if !ok {
return
}
@@ -182,7 +198,7 @@ func (h *Handler) handleStreamWithRetry(w http.ResponseWriter, r *http.Request,
}
}
func (h *Handler) prepareChatStreamRuntime(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) (*chatStreamRuntime, string, bool) {
func (h *Handler) prepareChatStreamRuntime(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, historySession *chatHistorySession) (*chatStreamRuntime, string, bool) {
if resp.StatusCode != http.StatusOK {
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
@@ -208,8 +224,10 @@ func (h *Handler) prepareChatStreamRuntime(w http.ResponseWriter, resp *http.Res
streamRuntime := newChatStreamRuntime(
w, rc, canFlush, completionID, time.Now().Unix(), model, finalPrompt,
thinkingEnabled, searchEnabled, h.compatStripReferenceMarkers(), toolNames, toolsRaw,
toolChoice,
len(toolNames) > 0, h.toolcallFeatureMatchEnabled() && h.toolcallEarlyEmitHighConfidence(),
)
streamRuntime.refFileTokens = refFileTokens
return streamRuntime, initialType, true
}
@@ -229,7 +247,7 @@ func (h *Handler) consumeChatStreamAttempt(r *http.Request, resp *http.Response,
OnParsed: func(parsed sse.LineResult) streamengine.ParsedDecision {
decision := streamRuntime.onParsed(parsed)
if historySession != nil {
historySession.progress(streamRuntime.thinking.String(), streamRuntime.text.String())
historySession.progress(streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String())
}
return decision
},
@@ -239,11 +257,15 @@ func (h *Handler) consumeChatStreamAttempt(r *http.Request, resp *http.Response,
}
},
OnContextDone: func() {
streamRuntime.markContextCancelled()
if historySession != nil {
historySession.stopped(streamRuntime.thinking.String(), streamRuntime.text.String(), string(streamengine.StopReasonContextCancelled))
historySession.stopped(streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String(), string(streamengine.StopReasonContextCancelled))
}
},
})
if streamRuntime.finalErrorCode == string(streamengine.StopReasonContextCancelled) {
return true, false
}
terminalWritten := streamRuntime.finalize(finalReason, allowDeferEmpty && finalReason != "content_filter")
if terminalWritten {
recordChatStreamHistory(streamRuntime, historySession)
@@ -257,7 +279,7 @@ func recordChatStreamHistory(streamRuntime *chatStreamRuntime, historySession *c
return
}
if streamRuntime.finalErrorMessage != "" {
historySession.error(streamRuntime.finalErrorStatus, streamRuntime.finalErrorMessage, streamRuntime.finalErrorCode, streamRuntime.thinking.String(), streamRuntime.text.String())
historySession.error(streamRuntime.finalErrorStatus, streamRuntime.finalErrorMessage, streamRuntime.finalErrorCode, streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String())
return
}
historySession.success(http.StatusOK, streamRuntime.finalThinking, streamRuntime.finalText, streamRuntime.finalFinishReason, streamRuntime.finalUsage)
@@ -266,7 +288,7 @@ func recordChatStreamHistory(streamRuntime *chatStreamRuntime, historySession *c
func failChatStreamRetry(streamRuntime *chatStreamRuntime, historySession *chatHistorySession, status int, message, code string) {
streamRuntime.sendFailedChunk(status, message, code)
if historySession != nil {
historySession.error(status, message, code, streamRuntime.thinking.String(), streamRuntime.text.String())
historySession.error(status, message, code, streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String())
}
}
@@ -275,6 +297,10 @@ func logChatStreamTerminal(streamRuntime *chatStreamRuntime, attempts int) {
if attempts > 0 {
source = "synthetic_retry"
}
if streamRuntime.finalErrorCode == string(streamengine.StopReasonContextCancelled) {
config.Logger.Info("[openai_empty_retry] terminal cancelled", "surface", "chat.completions", "stream", true, "retry_attempts", attempts, "error_code", streamRuntime.finalErrorCode)
return
}
if streamRuntime.finalErrorMessage != "" {
config.Logger.Info("[openai_empty_retry] terminal empty output", "surface", "chat.completions", "stream", true, "retry_attempts", attempts, "success_source", "none", "error_code", streamRuntime.finalErrorCode)
return

View File

@@ -0,0 +1,87 @@
package chat
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"ds2api/internal/chathistory"
"ds2api/internal/promptcompat"
"ds2api/internal/stream"
)
func TestConsumeChatStreamAttemptMarksContextCancelledState(t *testing.T) {
historyStore := newTestChatHistoryStore(t)
entry, err := historyStore.Start(chathistory.StartParams{
CallerID: "caller:test",
Model: "deepseek-v4-flash",
Stream: true,
UserInput: "hello",
})
if err != nil {
t.Fatalf("start history failed: %v", err)
}
session := &chatHistorySession{
store: historyStore,
entryID: entry.ID,
startedAt: time.Now(),
lastPersist: time.Now(),
finalPrompt: "prompt",
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil).WithContext(ctx)
rec := httptest.NewRecorder()
streamRuntime := newChatStreamRuntime(
rec,
http.NewResponseController(rec),
true,
"cid-cancelled",
time.Now().Unix(),
"deepseek-v4-flash",
"prompt",
false,
false,
true,
nil,
nil,
promptcompat.DefaultToolChoicePolicy(),
false,
false,
)
resp := makeOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"hello"}`,
`data: [DONE]`,
)
h := &Handler{}
terminalWritten, retryable := h.consumeChatStreamAttempt(req, resp, streamRuntime, "text", false, session, true)
if !terminalWritten || retryable {
t.Fatalf("expected cancelled attempt to terminate without retry, got terminalWritten=%v retryable=%v", terminalWritten, retryable)
}
if got, want := streamRuntime.finalErrorCode, string(stream.StopReasonContextCancelled); got != want {
t.Fatalf("expected cancelled final error code %q, got %q", want, got)
}
if streamRuntime.finalErrorMessage == "" {
t.Fatalf("expected cancelled final error message to be preserved")
}
snapshot, err := historyStore.Snapshot()
if err != nil {
t.Fatalf("snapshot failed: %v", err)
}
if len(snapshot.Items) != 1 {
t.Fatalf("expected one history item, got %d", len(snapshot.Items))
}
full, err := historyStore.Get(snapshot.Items[0].ID)
if err != nil {
t.Fatalf("get detail failed: %v", err)
}
if full.Status != "stopped" {
t.Fatalf("expected stopped status, got %#v", full)
}
}

View File

@@ -80,6 +80,10 @@ func writeOpenAIError(w http.ResponseWriter, status int, message string) {
shared.WriteOpenAIError(w, status, message)
}
func writeOpenAIErrorWithCode(w http.ResponseWriter, status int, message, code string) {
shared.WriteOpenAIErrorWithCode(w, status, message, code)
}
func openAIErrorType(status int) string {
return shared.OpenAIErrorType(status)
}
@@ -108,8 +112,8 @@ func replaceCitationMarkersWithLinks(text string, links map[int]string) string {
return shared.ReplaceCitationMarkersWithLinks(text, links)
}
func shouldWriteUpstreamEmptyOutputError(text string) bool {
return shared.ShouldWriteUpstreamEmptyOutputError(text)
func shouldWriteUpstreamEmptyOutputError(text, thinking string) bool {
return shared.ShouldWriteUpstreamEmptyOutputError(text, thinking)
}
func upstreamEmptyOutputDetail(contentFilter bool, text, thinking string) (int, string, string) {
@@ -148,6 +152,6 @@ func formatFinalStreamToolCallsWithStableIDs(calls []toolcall.ParsedToolCall, id
return shared.FormatFinalStreamToolCallsWithStableIDs(calls, ids, toolsRaw)
}
func detectAssistantToolCalls(text, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(text, exposedThinking, detectionThinking, toolNames)
func detectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking, toolNames)
}

View File

@@ -9,6 +9,7 @@ import (
"time"
"ds2api/internal/auth"
"ds2api/internal/completionruntime"
"ds2api/internal/config"
dsprotocol "ds2api/internal/deepseek/protocol"
openaifmt "ds2api/internal/format/openai"
@@ -76,43 +77,40 @@ func (h *Handler) ChatCompletions(w http.ResponseWriter, r *http.Request) {
}
historySession := startChatHistory(h.ChatHistory, r, a, stdReq)
sessionID, err = h.DS.CreateSession(r.Context(), a, 3)
if err != nil {
if a.UseConfigToken {
if !stdReq.Stream {
result, outErr := completionruntime.ExecuteNonStreamWithRetry(r.Context(), h.DS, a, stdReq, completionruntime.Options{
StripReferenceMarkers: h.compatStripReferenceMarkers(),
RetryEnabled: true,
})
sessionID = result.SessionID
if outErr != nil {
if historySession != nil {
historySession.error(http.StatusUnauthorized, "Account token is invalid. Please re-login the account in admin.", "error", "", "")
historySession.error(outErr.Status, outErr.Message, outErr.Code, result.Turn.Thinking, result.Turn.Text)
}
writeOpenAIError(w, http.StatusUnauthorized, "Account token is invalid. Please re-login the account in admin.")
} else {
if historySession != nil {
historySession.error(http.StatusUnauthorized, "Invalid token. If this should be a DS2API key, add it to config.keys first.", "error", "", "")
}
writeOpenAIError(w, http.StatusUnauthorized, "Invalid token. If this should be a DS2API key, add it to config.keys first.")
writeOpenAIErrorWithCode(w, outErr.Status, outErr.Message, outErr.Code)
return
}
return
}
pow, err := h.DS.GetPow(r.Context(), a, 3)
if err != nil {
respBody := openaifmt.BuildChatCompletionWithToolCalls(result.SessionID, stdReq.ResponseModel, result.Turn.Prompt, result.Turn.Thinking, result.Turn.Text, result.Turn.ToolCalls, stdReq.ToolsRaw)
respBody["usage"] = chatUsageFromTurn(result.Turn)
finishReason := chatFinishReason(respBody)
if historySession != nil {
historySession.error(http.StatusUnauthorized, "Failed to get PoW (invalid token or unknown error).", "error", "", "")
historySession.success(http.StatusOK, result.Turn.Thinking, result.Turn.Text, finishReason, chatUsageFromTurn(result.Turn))
}
writeOpenAIError(w, http.StatusUnauthorized, "Failed to get PoW (invalid token or unknown error).")
writeJSON(w, http.StatusOK, respBody)
return
}
payload := stdReq.CompletionPayload(sessionID)
resp, err := h.DS.CallCompletion(r.Context(), a, payload, pow, 3)
if err != nil {
start, outErr := completionruntime.StartCompletion(r.Context(), h.DS, a, stdReq, completionruntime.Options{})
sessionID = start.SessionID
if outErr != nil {
if historySession != nil {
historySession.error(http.StatusInternalServerError, "Failed to get completion.", "error", "", "")
historySession.error(outErr.Status, outErr.Message, outErr.Code, "", "")
}
writeOpenAIError(w, http.StatusInternalServerError, "Failed to get completion.")
writeOpenAIErrorWithCode(w, outErr.Status, outErr.Message, outErr.Code)
return
}
if stdReq.Stream {
h.handleStreamWithRetry(w, r, a, resp, payload, pow, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, historySession)
return
}
h.handleNonStreamWithRetry(w, r.Context(), a, resp, payload, pow, sessionID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, historySession)
refFileTokens := stdReq.RefFileTokens
h.handleStreamWithRetry(w, r, a, start.Response, start.Payload, start.Pow, sessionID, stdReq.ResponseModel, stdReq.PromptTokenText, refFileTokens, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, historySession)
}
func (h *Handler) autoDeleteRemoteSession(ctx context.Context, a *auth.RequestAuth, sessionID string) {
@@ -148,7 +146,7 @@ func (h *Handler) autoDeleteRemoteSession(ctx context.Context, a *auth.RequestAu
}
}
func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
if resp.StatusCode != http.StatusOK {
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
@@ -162,13 +160,12 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, co
stripReferenceMarkers := h.compatStripReferenceMarkers()
finalThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
finalToolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
finalText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
finalText = replaceCitationMarkersWithLinks(finalText, result.CitationLinks)
}
detected := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, toolNames)
if shouldWriteUpstreamEmptyOutputError(finalText) && len(detected.Calls) == 0 {
detected := detectAssistantToolCalls(result.Text, finalText, result.Thinking, result.ToolDetectionThinking, toolNames)
if shouldWriteUpstreamEmptyOutputError(finalText, finalThinking) && len(detected.Calls) == 0 {
status, message, code := upstreamEmptyOutputDetail(result.ContentFilter, finalText, finalThinking)
if historySession != nil {
historySession.error(status, message, code, finalThinking, finalText)
@@ -177,6 +174,9 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, co
return
}
respBody := openaifmt.BuildChatCompletionWithToolCalls(completionID, model, finalPrompt, finalThinking, finalText, detected.Calls, toolsRaw)
if refFileTokens > 0 {
addRefFileTokensToUsage(respBody, refFileTokens)
}
finishReason := "stop"
if choices, ok := respBody["choices"].([]map[string]any); ok && len(choices) > 0 {
if fr, _ := choices[0]["finish_reason"].(string); strings.TrimSpace(fr) != "" {
@@ -184,12 +184,12 @@ func (h *Handler) handleNonStream(w http.ResponseWriter, resp *http.Response, co
}
}
if historySession != nil {
historySession.success(http.StatusOK, finalThinking, finalText, finishReason, openaifmt.BuildChatUsage(finalPrompt, finalThinking, finalText))
historySession.success(http.StatusOK, finalThinking, finalText, finishReason, openaifmt.BuildChatUsageForModel(model, finalPrompt, finalThinking, finalText, refFileTokens))
}
writeJSON(w, http.StatusOK, respBody)
}
func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *http.Response, completionID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *http.Response, completionID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, historySession *chatHistorySession) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -231,9 +231,11 @@ func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *htt
stripReferenceMarkers,
toolNames,
toolsRaw,
promptcompat.DefaultToolChoicePolicy(),
bufferToolContent,
emitEarlyToolDeltas,
)
streamRuntime.refFileTokens = refFileTokens
streamengine.ConsumeSSE(streamengine.ConsumeConfig{
Context: r.Context(),
@@ -250,7 +252,7 @@ func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *htt
OnParsed: func(parsed sse.LineResult) streamengine.ParsedDecision {
decision := streamRuntime.onParsed(parsed)
if historySession != nil {
historySession.progress(streamRuntime.thinking.String(), streamRuntime.text.String())
historySession.progress(streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String())
}
return decision
},
@@ -264,14 +266,14 @@ func (h *Handler) handleStream(w http.ResponseWriter, r *http.Request, resp *htt
return
}
if streamRuntime.finalErrorMessage != "" {
historySession.error(streamRuntime.finalErrorStatus, streamRuntime.finalErrorMessage, streamRuntime.finalErrorCode, streamRuntime.thinking.String(), streamRuntime.text.String())
historySession.error(streamRuntime.finalErrorStatus, streamRuntime.finalErrorMessage, streamRuntime.finalErrorCode, streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String())
return
}
historySession.success(http.StatusOK, streamRuntime.finalThinking, streamRuntime.finalText, streamRuntime.finalFinishReason, streamRuntime.finalUsage)
},
OnContextDone: func() {
if historySession != nil {
historySession.stopped(streamRuntime.thinking.String(), streamRuntime.text.String(), string(streamengine.StopReasonContextCancelled))
historySession.stopped(streamRuntime.accumulator.Thinking.String(), streamRuntime.accumulator.Text.String(), string(streamengine.StopReasonContextCancelled))
}
},
})

View File

@@ -1,6 +1,7 @@
package chat
import (
"context"
"encoding/json"
"io"
"net/http"
@@ -93,7 +94,7 @@ func TestHandleNonStreamReturns429WhenUpstreamOutputEmpty(t *testing.T) {
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-empty", "deepseek-v4-flash", "prompt", false, false, nil, nil, nil)
h.handleNonStream(rec, resp, "cid-empty", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, nil)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected status 429 for empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -112,7 +113,7 @@ func TestHandleNonStreamReturnsContentFilterErrorWhenUpstreamFilteredWithoutOutp
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-empty-filtered", "deepseek-v4-flash", "prompt", false, false, nil, nil, nil)
h.handleNonStream(rec, resp, "cid-empty-filtered", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, nil)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected status 400 for filtered upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -131,14 +132,19 @@ func TestHandleNonStreamReturns429WhenUpstreamHasOnlyThinking(t *testing.T) {
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-thinking-only", "deepseek-v4-pro", "prompt", true, false, nil, nil, nil)
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected status 429 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
h.handleNonStream(rec, resp, "cid-thinking-only", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, nil)
if rec.Code != http.StatusOK {
t.Fatalf("expected status 200 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", out)
choices, _ := out["choices"].([]any)
if len(choices) == 0 {
t.Fatal("expected at least one choice")
}
first, _ := choices[0].(map[string]any)
msg, _ := first["message"].(map[string]any)
if asString(msg["reasoning_content"]) != "Only thinking" {
t.Fatalf("expected reasoning_content='Only thinking', got %#v", msg)
}
}
@@ -150,7 +156,7 @@ func TestHandleNonStreamPromotesThinkingToolCallsWhenTextEmpty(t *testing.T) {
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-thinking-tool", "deepseek-v4-pro", "prompt", true, false, []string{"search"}, nil, nil)
h.handleNonStream(rec, resp, "cid-thinking-tool", "deepseek-v4-pro", "prompt", 0, true, false, []string{"search"}, nil, nil)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -181,7 +187,7 @@ func TestHandleNonStreamPromotesHiddenThinkingDSMLToolCallsWhenTextEmpty(t *test
)
rec := httptest.NewRecorder()
h.handleNonStream(rec, resp, "cid-hidden-thinking-tool", "deepseek-v4-pro", "prompt", false, false, []string{"search"}, nil, nil)
h.handleNonStream(rec, resp, "cid-hidden-thinking-tool", "deepseek-v4-pro", "prompt", 0, false, false, []string{"search"}, nil, nil)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for hidden thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -211,7 +217,7 @@ func TestHandleStreamToolsPlainTextStreamsBeforeFinish(t *testing.T) {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid6", "deepseek-v4-flash", "prompt", false, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid6", "deepseek-v4-flash", "prompt", 0, false, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -239,6 +245,118 @@ func TestHandleStreamToolsPlainTextStreamsBeforeFinish(t *testing.T) {
}
}
func TestHandleStreamThinkingDisabledDoesNotLeakHiddenFragmentContinuations(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/fragments","o":"APPEND","v":[{"type":"THINK","content":"我们"}]}`,
`data: {"p":"response/fragments/-1/content","v":"被"}`,
`data: {"v":"要求"}`,
`data: {"p":"response/fragments","o":"APPEND","v":[{"type":"RESPONSE","content":"答"}]}`,
`data: {"p":"response/fragments/-1/content","v":"案"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-hidden-fragment", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
content := strings.Builder{}
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
content.WriteString(c)
}
}
}
if got := content.String(); got != "答案" {
t.Fatalf("expected only visible response text, got %q body=%s", got, rec.Body.String())
}
}
func TestHandleStreamEmitsSingleChoiceFramesForMultipleParsedParts(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
`data: {"p":"response/fragments","o":"APPEND","v":[{"type":"THINK","content":"我们"},{"type":"THINK","content":"被"},{"type":"THINK","content":"要求"},{"type":"RESPONSE","content":"答"},{"type":"RESPONSE","content":"案"}]}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-multi-parts", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
var reasoning, content strings.Builder
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
if len(choices) != 1 {
t.Fatalf("expected exactly one choice per stream frame, got %d frame=%#v body=%s", len(choices), frame, rec.Body.String())
}
choice, _ := choices[0].(map[string]any)
delta, _ := choice["delta"].(map[string]any)
reasoning.WriteString(asString(delta["reasoning_content"]))
content.WriteString(asString(delta["content"]))
}
if got := reasoning.String(); got != "我们被要求" {
t.Fatalf("first-choice-only client would miss reasoning tokens: got %q body=%s", got, rec.Body.String())
}
if got := content.String(); got != "答案" {
t.Fatalf("first-choice-only client would miss content tokens: got %q body=%s", got, rec.Body.String())
}
}
func TestHandleStreamCoalescesSmallContentDeltas(t *testing.T) {
h := &Handler{}
lines := make([]string, 0, 101)
for i := 0; i < 100; i++ {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": "字",
})
lines = append(lines, "data: "+string(b))
}
lines = append(lines, "data: [DONE]")
resp := makeSSEHTTPResponse(lines...)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-coalesce", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
t.Fatalf("expected [DONE], body=%s", rec.Body.String())
}
var content strings.Builder
contentDeltaFrames := 0
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
if len(choices) != 1 {
t.Fatalf("expected exactly one choice per stream frame, got %d frame=%#v body=%s", len(choices), frame, rec.Body.String())
}
choice, _ := choices[0].(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if c, ok := delta["content"].(string); ok {
contentDeltaFrames++
content.WriteString(c)
}
}
if got, want := content.String(), strings.Repeat("字", 100); got != want {
t.Fatalf("coalesced stream content mismatch: got %q want %q body=%s", got, want, rec.Body.String())
}
if contentDeltaFrames >= 100 {
t.Fatalf("expected coalescing to reduce 100 tiny content frames, got %d body=%s", contentDeltaFrames, rec.Body.String())
}
}
func TestHandleStreamIncompleteCapturedToolJSONFlushesAsTextOnFinalize(t *testing.T) {
h := &Handler{}
resp := makeSSEHTTPResponse(
@@ -248,7 +366,7 @@ func TestHandleStreamIncompleteCapturedToolJSONFlushesAsTextOnFinalize(t *testin
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid10", "deepseek-v4-flash", "prompt", false, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid10", "deepseek-v4-flash", "prompt", 0, false, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -282,7 +400,7 @@ func TestHandleStreamPromotesThinkingToolCallsOnFinalizeWithoutMidstreamIntercep
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-thinking-stream", "deepseek-v4-pro", "prompt", true, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid-thinking-stream", "deepseek-v4-pro", "prompt", 0, true, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -291,20 +409,16 @@ func TestHandleStreamPromotesThinkingToolCallsOnFinalizeWithoutMidstreamIntercep
if !streamHasToolCallsDelta(frames) {
t.Fatalf("expected tool_calls delta from finalize fallback, body=%s", rec.Body.String())
}
reasoningSeen := false
for _, frame := range frames {
choices, _ := frame["choices"].([]any)
for _, item := range choices {
choice, _ := item.(map[string]any)
delta, _ := choice["delta"].(map[string]any)
if asString(delta["reasoning_content"]) != "" {
reasoningSeen = true
t.Fatalf("did not expect leaked reasoning_content markup, body=%s", rec.Body.String())
}
}
}
if !reasoningSeen {
t.Fatalf("expected reasoning_content to stream before finalize fallback, body=%s", rec.Body.String())
}
if streamFinishReason(frames) != "tool_calls" {
t.Fatalf("expected finish_reason=tool_calls, body=%s", rec.Body.String())
}
@@ -319,7 +433,7 @@ func TestHandleStreamPromotesHiddenThinkingDSMLToolCallsOnFinalize(t *testing.T)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-hidden-thinking-stream", "deepseek-v4-pro", "prompt", false, false, []string{"search"}, nil, nil)
h.handleStream(rec, req, resp, "cid-hidden-thinking-stream", "deepseek-v4-pro", "prompt", 0, false, false, []string{"search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -353,7 +467,7 @@ func TestHandleStreamEmitsDistinctToolCallIDsAcrossSeparateToolBlocks(t *testing
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
h.handleStream(rec, req, resp, "cid-multi", "deepseek-v4-flash", "prompt", false, false, []string{"read_file", "search"}, nil, nil)
h.handleStream(rec, req, resp, "cid-multi", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file", "search"}, nil, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -419,7 +533,7 @@ func TestHandleStreamCoercesSchemaDeclaredStringArgumentsOnFinalize(t *testing.T
},
}
h.handleStream(rec, req, resp, "cid-string-protect", "deepseek-v4-flash", "prompt", false, false, []string{"Write"}, toolsRaw, nil)
h.handleStream(rec, req, resp, "cid-string-protect", "deepseek-v4-flash", "prompt", 0, false, false, []string{"Write"}, toolsRaw, nil)
frames, done := parseSSEDataFrames(t, rec.Body.String())
if !done {
@@ -451,3 +565,45 @@ func TestHandleStreamCoercesSchemaDeclaredStringArgumentsOnFinalize(t *testing.T
}
t.Fatalf("expected at least one streamed tool call delta, body=%s", rec.Body.String())
}
func TestHandleNonStreamWithRetryIncludesRefFileTokensInUsage(t *testing.T) {
h := &Handler{}
run := func(refFileTokens int) map[string]any {
resp := makeSSEHTTPResponse(
`data: {"p":"response/content","v":"hello world"}`,
`data: [DONE]`,
)
rec := httptest.NewRecorder()
h.handleNonStreamWithRetry(rec, context.Background(), nil, resp, nil, "", "cid-ref", "deepseek-v4-flash", "prompt", refFileTokens, false, false, nil, nil, nil)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
return decodeJSONBody(t, rec.Body.String())
}
base := run(0)
withRef := run(7)
baseUsage, _ := base["usage"].(map[string]any)
refUsage, _ := withRef["usage"].(map[string]any)
if baseUsage == nil || refUsage == nil {
t.Fatalf("expected usage objects, base=%#v ref=%#v", base["usage"], withRef["usage"])
}
getInt := func(m map[string]any, key string) int {
t.Helper()
v, ok := m[key].(float64)
if !ok {
t.Fatalf("expected numeric %s, got %#v", key, m[key])
}
return int(v)
}
if got := getInt(refUsage, "prompt_tokens") - getInt(baseUsage, "prompt_tokens"); got != 7 {
t.Fatalf("expected prompt_tokens delta 7, got %d", got)
}
if got := getInt(refUsage, "total_tokens") - getInt(baseUsage, "total_tokens"); got != 7 {
t.Fatalf("expected total_tokens delta 7, got %d", got)
}
}

View File

@@ -0,0 +1,26 @@
package chat
// addRefFileTokensToUsage adds inline-uploaded file token estimates to an existing
// usage map inside a response object. This keeps the token accounting aware of file
// content that the upstream model processes but that is not part of the prompt text.
func addRefFileTokensToUsage(obj map[string]any, refFileTokens int) {
if refFileTokens <= 0 || obj == nil {
return
}
usage, ok := obj["usage"].(map[string]any)
if !ok || usage == nil {
return
}
for _, key := range []string{"input_tokens", "prompt_tokens"} {
if v, ok := usage[key]; ok {
if n, ok := v.(int); ok {
usage[key] = n + refFileTokens
}
}
}
if v, ok := usage["total_tokens"]; ok {
if n, ok := v.(int); ok {
usage["total_tokens"] = n + refFileTokens
}
}
}

View File

@@ -130,8 +130,8 @@ func TestHandleVercelStreamPrepareAppliesCurrentInputFile(t *testing.T) {
t.Fatalf("expected payload object, got %#v", body["payload"])
}
promptText, _ := payload["prompt"].(string)
if !strings.Contains(promptText, "Answer the latest user request directly.") {
t.Fatalf("expected neutral prompt, got %s", promptText)
if !strings.Contains(promptText, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected continuation prompt, got %s", promptText)
}
if strings.Contains(promptText, "first user turn") || strings.Contains(promptText, "latest user turn") {
t.Fatalf("expected original turns hidden from prompt, got %s", promptText)

View File

@@ -54,3 +54,31 @@ func TestReplaceCitationMarkersWithLinksSupportsReferenceZeroBased(t *testing.T)
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksKeepsCitationOneBasedWithZeroBasedReference(t *testing.T) {
raw := "引用[citation:1],来源[reference:0],后续[reference:1]。"
links := map[int]string{
1: "https://example.com/first",
2: "https://example.com/second",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "引用[1](https://example.com/first),来源[0](https://example.com/first),后续[1](https://example.com/second)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestReplaceCitationMarkersWithLinksDetectsSpacedReferenceZeroBased(t *testing.T) {
raw := "来源[reference: 0] 与 [reference: 1]。"
links := map[int]string{
1: "https://example.com/first",
2: "https://example.com/second",
}
got := replaceCitationMarkersWithLinks(raw, links)
want := "来源[0](https://example.com/first) 与 [1](https://example.com/second)。"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}

View File

@@ -94,6 +94,9 @@ func TestPreprocessInlineFileInputsReplacesDataURLAndCollectsRefFileIDs(t *testi
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload, got %d", len(ds.uploadCalls))
}
if ds.uploadCalls[0].ModelType != "default" {
t.Fatalf("expected default model type when request omits model, got %q", ds.uploadCalls[0].ModelType)
}
if ds.lastCtx != ctx {
t.Fatalf("expected upload to use request context")
}
@@ -149,7 +152,7 @@ func TestPreprocessInlineFileInputsDeduplicatesIdenticalPayloads(t *testing.T) {
func TestChatCompletionsUploadsInlineFilesBeforeCompletion(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
reqBody := `{"model":"deepseek-v4-flash","messages":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"image_url","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
reqBody := `{"model":"deepseek-v4-vision","messages":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"image_url","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
@@ -163,6 +166,9 @@ func TestChatCompletionsUploadsInlineFilesBeforeCompletion(t *testing.T) {
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
if ds.uploadCalls[0].ModelType != "vision" {
t.Fatalf("expected vision model type for vision request, got %q", ds.uploadCalls[0].ModelType)
}
if ds.completionReq == nil {
t.Fatal("expected completion payload to be captured")
}
@@ -177,7 +183,7 @@ func TestResponsesUploadsInlineFilesBeforeCompletion(t *testing.T) {
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
r := chi.NewRouter()
registerOpenAITestRoutes(r, h)
reqBody := `{"model":"deepseek-v4-flash","input":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"input_image","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
reqBody := `{"model":"deepseek-v4-pro","input":[{"role":"user","content":[{"type":"input_text","text":"hi"},{"type":"input_image","image_url":{"url":"data:image/png;base64,QUJDRA=="}}]}],"stream":false}`
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(reqBody))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
@@ -191,6 +197,9 @@ func TestResponsesUploadsInlineFilesBeforeCompletion(t *testing.T) {
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
if ds.uploadCalls[0].ModelType != "expert" {
t.Fatalf("expected expert model type for pro request, got %q", ds.uploadCalls[0].ModelType)
}
refIDs, _ := ds.completionReq["ref_file_ids"].([]any)
if len(refIDs) != 1 || refIDs[0] != "file-inline-1" {
t.Fatalf("unexpected completion ref_file_ids: %#v", ds.completionReq["ref_file_ids"])
@@ -216,6 +225,45 @@ func TestChatCompletionsInlineUploadFailureReturnsBadRequest(t *testing.T) {
}
}
func TestChatCompletionsInlineUploadLimitReturnsBadRequest(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
content := []any{map[string]any{"type": "input_text", "text": "hi"}}
for i := 0; i < 51; i++ {
content = append(content, map[string]any{
"type": "image_url",
"image_url": map[string]any{"url": "data:image/png;base64,QUJDRA=="},
})
}
body, err := json.Marshal(map[string]any{
"model": "deepseek-v4-flash",
"messages": []any{map[string]any{
"role": "user",
"content": content,
}},
"stream": false,
})
if err != nil {
t.Fatalf("marshal request: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(string(body)))
req.Header.Set("Authorization", "Bearer direct-token")
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
h.ChatCompletions(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "exceeded maximum of 50 inline files per request") {
t.Fatalf("expected inline file limit error, got body=%s", rec.Body.String())
}
if ds.completionReq != nil {
t.Fatalf("did not expect completion call after inline file limit error")
}
}
func TestResponsesInlineUploadFailureReturnsInternalServerError(t *testing.T) {
ds := &inlineUploadDSStub{uploadErr: errors.New("boom")}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}

View File

@@ -12,6 +12,7 @@ import (
"strings"
"ds2api/internal/auth"
"ds2api/internal/config"
dsclient "ds2api/internal/deepseek/client"
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/promptcompat"
@@ -39,11 +40,13 @@ func (e *inlineFileUploadError) Error() string {
}
type inlineUploadState struct {
ctx context.Context
handler *Handler
auth *auth.RequestAuth
uploadedByID map[string]string
uploadCount int
ctx context.Context
handler *Handler
auth *auth.RequestAuth
modelType string
uploadedByID map[string]string
uploadCount int
inlineFileBytes int
}
type inlineDecodedFile struct {
@@ -57,10 +60,19 @@ func (h *Handler) PreprocessInlineFileInputs(ctx context.Context, a *auth.Reques
if h == nil || h.DS == nil || len(req) == 0 {
return nil
}
modelType := "default"
if requestedModel, ok := req["model"].(string); ok {
if resolvedModel, ok := config.ResolveModel(h.Store, requestedModel); ok {
if resolvedType, ok := config.GetModelType(resolvedModel); ok {
modelType = resolvedType
}
}
}
state := &inlineUploadState{
ctx: ctx,
handler: h,
auth: a,
modelType: modelType,
uploadedByID: map[string]string{},
}
for _, key := range []string{"messages", "input", "attachments"} {
@@ -75,6 +87,9 @@ func (h *Handler) PreprocessInlineFileInputs(ctx context.Context, a *auth.Reques
if refIDs := promptcompat.CollectOpenAIRefFileIDs(req); len(refIDs) > 0 {
req["ref_file_ids"] = stringsToAnySlice(refIDs)
}
if state.inlineFileBytes > 0 {
req["_inline_file_bytes"] = state.inlineFileBytes
}
return nil
}
@@ -135,13 +150,15 @@ func (s *inlineUploadState) tryUploadBlock(block map[string]any) (map[string]any
return nil, false, nil
}
if s.uploadCount >= maxInlineFilesPerRequest {
return nil, true, fmt.Errorf("exceeded maximum of %d inline files per request", maxInlineFilesPerRequest)
err := fmt.Errorf("exceeded maximum of %d inline files per request", maxInlineFilesPerRequest)
return nil, true, &inlineFileUploadError{status: http.StatusBadRequest, message: err.Error(), err: err}
}
fileID, err := s.uploadInlineFile(decoded)
if err != nil {
return nil, true, &inlineFileUploadError{status: http.StatusInternalServerError, message: "Failed to upload inline file.", err: err}
}
s.uploadCount++
s.inlineFileBytes += len(decoded.Data)
replacement := map[string]any{
"type": decoded.ReplacementType,
"file_id": fileID,
@@ -168,6 +185,7 @@ func (s *inlineUploadState) uploadInlineFile(file inlineDecodedFile) (string, er
result, err := s.handler.DS.UploadFile(s.ctx, s.auth, dsclient.UploadFileRequest{
Filename: file.Filename,
ContentType: contentType,
ModelType: s.modelType,
Data: file.Data,
}, 3)
if err != nil {

View File

@@ -1,13 +1,18 @@
package files
import (
"context"
"errors"
"io"
"net/http"
"strings"
"time"
"github.com/go-chi/chi/v5"
"ds2api/internal/auth"
"ds2api/internal/chathistory"
"ds2api/internal/config"
dsclient "ds2api/internal/deepseek/client"
"ds2api/internal/httpapi/openai/shared"
)
@@ -21,6 +26,10 @@ type Handler struct {
ChatHistory *chathistory.Store
}
type fileFetcher interface {
FetchUploadedFile(ctx context.Context, a *auth.RequestAuth, fileID string) (*dsclient.UploadFileResult, error)
}
func (h *Handler) UploadFile(w http.ResponseWriter, r *http.Request) {
a, err := h.Auth.Determine(r)
if err != nil {
@@ -66,10 +75,12 @@ func (h *Handler) UploadFile(w http.ResponseWriter, r *http.Request) {
if contentType == "" && len(data) > 0 {
contentType = http.DetectContentType(data)
}
modelType := resolveUploadModelType(h.Store, r)
result, err := h.DS.UploadFile(r.Context(), a, dsclient.UploadFileRequest{
Filename: header.Filename,
ContentType: contentType,
Purpose: strings.TrimSpace(r.FormValue("purpose")),
ModelType: modelType,
Data: data,
}, 3)
if err != nil {
@@ -82,6 +93,70 @@ func (h *Handler) UploadFile(w http.ResponseWriter, r *http.Request) {
shared.WriteJSON(w, http.StatusOK, buildOpenAIFileObject(result))
}
func (h *Handler) RetrieveFile(w http.ResponseWriter, r *http.Request) {
a, err := h.Auth.Determine(r)
if err != nil {
status := http.StatusUnauthorized
detail := err.Error()
if err == auth.ErrNoAccount {
status = http.StatusTooManyRequests
}
shared.WriteOpenAIError(w, status, detail)
return
}
defer h.Auth.Release(a)
fileID := strings.TrimSpace(chi.URLParam(r, "file_id"))
if fileID == "" {
shared.WriteOpenAIError(w, http.StatusBadRequest, "file_id is required")
return
}
fetcher, ok := h.DS.(fileFetcher)
if !ok {
shared.WriteOpenAIError(w, http.StatusNotImplemented, "file retrieval is not available")
return
}
result, err := fetcher.FetchUploadedFile(r.Context(), a, fileID)
if err != nil {
if errors.Is(err, dsclient.ErrUploadFileNotFound) {
shared.WriteOpenAIError(w, http.StatusNotFound, "file not found")
return
}
shared.WriteOpenAIError(w, http.StatusInternalServerError, "Failed to retrieve file.")
return
}
if result != nil && result.AccountID == "" {
result.AccountID = a.AccountID
}
shared.WriteJSON(w, http.StatusOK, buildOpenAIFileObject(result))
}
func resolveUploadModelType(store shared.ConfigReader, r *http.Request) string {
for _, candidate := range []string{r.FormValue("model_type"), r.Header.Get("X-Model-Type")} {
if modelType := normalizeUploadModelType(candidate); modelType != "" {
return modelType
}
}
requestedModel := strings.TrimSpace(r.FormValue("model"))
if requestedModel != "" {
if resolvedModel, ok := config.ResolveModel(store, requestedModel); ok {
if modelType, ok := config.GetModelType(resolvedModel); ok {
return modelType
}
}
}
return "default"
}
func normalizeUploadModelType(raw string) string {
switch strings.ToLower(strings.TrimSpace(raw)) {
case "default", "expert", "vision":
return strings.ToLower(strings.TrimSpace(raw))
default:
return ""
}
}
func buildOpenAIFileObject(result *dsclient.UploadFileResult) map[string]any {
if result == nil {
obj := map[string]any{

View File

@@ -43,6 +43,7 @@ func (managedFilesAuthStub) Release(_ *auth.RequestAuth) {}
type filesRouteDSStub struct {
lastReq dsclient.UploadFileRequest
upload *dsclient.UploadFileResult
fetched *dsclient.UploadFileResult
err error
}
@@ -65,6 +66,16 @@ func (m *filesRouteDSStub) UploadFile(_ context.Context, _ *auth.RequestAuth, re
return &dsclient.UploadFileResult{ID: "file-123", Filename: req.Filename, Bytes: int64(len(req.Data)), Purpose: req.Purpose, Status: "uploaded"}, nil
}
func (m *filesRouteDSStub) FetchUploadedFile(_ context.Context, _ *auth.RequestAuth, fileID string) (*dsclient.UploadFileResult, error) {
if m.err != nil {
return nil, m.err
}
if m.fetched != nil {
return m.fetched, nil
}
return &dsclient.UploadFileResult{ID: fileID, Filename: "notes.txt", Bytes: 11, Purpose: "assistants", Status: "processed"}, nil
}
func (m *filesRouteDSStub) CallCompletion(_ context.Context, _ *auth.RequestAuth, _ map[string]any, _ string, _ int) (*http.Response, error) {
return nil, errors.New("not implemented")
}
@@ -77,7 +88,7 @@ func (m *filesRouteDSStub) DeleteAllSessionsForToken(_ context.Context, _ string
return nil
}
func newMultipartUploadRequest(t *testing.T, purpose string, filename string, data []byte) *http.Request {
func newMultipartUploadRequest(t *testing.T, purpose string, filename string, data []byte, model string) *http.Request {
t.Helper()
var body bytes.Buffer
writer := multipart.NewWriter(&body)
@@ -86,6 +97,11 @@ func newMultipartUploadRequest(t *testing.T, purpose string, filename string, da
t.Fatalf("write purpose failed: %v", err)
}
}
if model != "" {
if err := writer.WriteField("model", model); err != nil {
t.Fatalf("write model failed: %v", err)
}
}
part, err := writer.CreateFormFile("file", filename)
if err != nil {
t.Fatalf("create form file failed: %v", err)
@@ -108,7 +124,7 @@ func TestFilesRouteUploadSuccess(t *testing.T) {
r := chi.NewRouter()
registerOpenAITestRoutes(r, h)
req := newMultipartUploadRequest(t, "assistants", "notes.txt", []byte("hello world"))
req := newMultipartUploadRequest(t, "assistants", "notes.txt", []byte("hello world"), "deepseek-v4-vision")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
@@ -121,6 +137,9 @@ func TestFilesRouteUploadSuccess(t *testing.T) {
if ds.lastReq.Purpose != "assistants" {
t.Fatalf("expected purpose assistants, got %q", ds.lastReq.Purpose)
}
if ds.lastReq.ModelType != "vision" {
t.Fatalf("expected vision model type, got %q", ds.lastReq.ModelType)
}
if string(ds.lastReq.Data) != "hello world" {
t.Fatalf("unexpected uploaded data: %q", string(ds.lastReq.Data))
}
@@ -145,7 +164,7 @@ func TestFilesRouteUploadIncludesAccountIDForManagedAccount(t *testing.T) {
r := chi.NewRouter()
registerOpenAITestRoutes(r, h)
req := newMultipartUploadRequest(t, "assistants", "notes.txt", []byte("hello world"))
req := newMultipartUploadRequest(t, "assistants", "notes.txt", []byte("hello world"), "deepseek-v4-vision")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
@@ -161,6 +180,54 @@ func TestFilesRouteUploadIncludesAccountIDForManagedAccount(t *testing.T) {
}
}
func TestFilesRouteRetrieveSuccess(t *testing.T) {
ds := &filesRouteDSStub{fetched: &dsclient.UploadFileResult{
ID: "file-123",
Filename: "notes.txt",
Bytes: 11,
Purpose: "assistants",
Status: "processed",
}}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: managedFilesAuthStub{}, DS: ds}
r := chi.NewRouter()
registerOpenAITestRoutes(r, h)
req := httptest.NewRequest(http.MethodGet, "/v1/files/file-123", nil)
req.Header.Set("Authorization", "Bearer direct-token")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", rec.Code, rec.Body.String())
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatalf("decode response failed: %v body=%s", err, rec.Body.String())
}
if out["id"] != "file-123" || out["filename"] != "notes.txt" || out["status"] != "processed" {
t.Fatalf("unexpected file object: %#v", out)
}
if out["account_id"] != "acct-123" {
t.Fatalf("expected account_id acct-123, got %#v", out["account_id"])
}
}
func TestFilesRouteRetrieveNotFound(t *testing.T) {
ds := &filesRouteDSStub{err: dsclient.ErrUploadFileNotFound}
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: ds}
r := chi.NewRouter()
registerOpenAITestRoutes(r, h)
req := httptest.NewRequest(http.MethodGet, "/v1/files/missing-file", nil)
req.Header.Set("Authorization", "Bearer direct-token")
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("expected 404, got %d body=%s", rec.Code, rec.Body.String())
}
}
func TestFilesRouteRejectsNonMultipart(t *testing.T) {
h := &openAITestSurface{Store: mockOpenAIConfig{wideInput: true}, Auth: streamStatusAuthStub{}, DS: &filesRouteDSStub{}}
r := chi.NewRouter()

View File

@@ -7,13 +7,14 @@ import (
"strings"
"ds2api/internal/auth"
"ds2api/internal/config"
dsclient "ds2api/internal/deepseek/client"
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/promptcompat"
)
const (
currentInputFilename = "IGNORE.txt"
currentInputFilename = promptcompat.CurrentInputContextFilename
currentInputContentType = "text/plain; charset=utf-8"
currentInputPurpose = "assistants"
)
@@ -35,11 +36,15 @@ func (s Service) ApplyCurrentInputFile(ctx context.Context, a *auth.RequestAuth,
if strings.TrimSpace(fileText) == "" {
return stdReq, errors.New("current user input file produced empty transcript")
}
modelType := "default"
if resolvedType, ok := config.GetModelType(stdReq.ResolvedModel); ok {
modelType = resolvedType
}
result, err := s.DS.UploadFile(ctx, a, dsclient.UploadFileRequest{
Filename: currentInputFilename,
ContentType: currentInputContentType,
Purpose: currentInputPurpose,
ModelType: modelType,
Data: []byte(fileText),
}, 3)
if err != nil {
@@ -62,6 +67,9 @@ func (s Service) ApplyCurrentInputFile(ctx context.Context, a *auth.RequestAuth,
stdReq.CurrentInputFileApplied = true
stdReq.RefFileIDs = prependUniqueRefFileID(stdReq.RefFileIDs, fileID)
stdReq.FinalPrompt, stdReq.ToolNames = promptcompat.BuildOpenAIPrompt(messages, stdReq.ToolsRaw, "", stdReq.ToolChoice, stdReq.Thinking)
// Token accounting must reflect the actual downstream context:
// the uploaded DS2API_HISTORY.txt file content + the continuation live prompt.
stdReq.PromptTokenText = fileText + "\n" + stdReq.FinalPrompt
return stdReq, nil
}
@@ -85,5 +93,5 @@ func latestUserInputForFile(messages []any) (int, string) {
}
func currentInputFilePrompt() string {
return "The current request and prior conversation context have already been provided. Answer the latest user request directly."
return "Continue from the latest state in the attached DS2API_HISTORY.txt context. Treat it as the current working state and answer the latest user request directly."
}

View File

@@ -14,6 +14,7 @@ import (
"ds2api/internal/auth"
dsclient "ds2api/internal/deepseek/client"
"ds2api/internal/promptcompat"
"ds2api/internal/util"
)
func historySplitTestMessages() []any {
@@ -60,27 +61,32 @@ func (streamStatusManagedAuthStub) DetermineCaller(_ *http.Request) (*auth.Reque
func (streamStatusManagedAuthStub) Release(_ *auth.RequestAuth) {}
func TestBuildOpenAICurrentInputContextTranscriptUsesInjectedFileWrapper(t *testing.T) {
func TestBuildOpenAICurrentInputContextTranscriptUsesNumberedHistorySections(t *testing.T) {
_, historyMessages := splitOpenAIHistoryMessages(historySplitTestMessages(), 1)
transcript := buildOpenAICurrentInputContextTranscript(historyMessages)
if !strings.HasPrefix(transcript, "[file content end]\n\n") {
t.Fatalf("expected injected file wrapper prefix, got %q", transcript)
if strings.Contains(transcript, "[file content end]") || strings.Contains(transcript, "[file content begin]") || strings.Contains(transcript, "[file name]:") {
t.Fatalf("expected transcript without file wrapper tags, got %q", transcript)
}
if !strings.Contains(transcript, "<begin▁of▁sentence>") {
t.Fatalf("expected serialized conversation markers, got %q", transcript)
if !strings.Contains(transcript, "# DS2API_HISTORY.txt") {
t.Fatalf("expected history transcript header, got %q", transcript)
}
if !strings.Contains(transcript, "first user turn") || !strings.Contains(transcript, "tool result") {
t.Fatalf("expected historical turns preserved, got %q", transcript)
if !strings.Contains(transcript, "Prior conversation history and tool progress.") {
t.Fatalf("expected history transcript description, got %q", transcript)
}
if !strings.Contains(transcript, "[reasoning_content]") || !strings.Contains(transcript, "hidden reasoning") {
t.Fatalf("expected reasoning block preserved, got %q", transcript)
}
if !strings.Contains(transcript, "<|DSML|tool_calls>") {
t.Fatalf("expected tool calls preserved, got %q", transcript)
}
if !strings.HasSuffix(transcript, "\n[file name]: IGNORE\n[file content begin]\n") {
t.Fatalf("expected injected file wrapper suffix, got %q", transcript)
for _, want := range []string{
"=== 1. USER ===",
"=== 2. ASSISTANT ===",
"=== 3. TOOL ===",
"first user turn",
"tool result",
"[reasoning_content]",
"hidden reasoning",
"<|DSML|tool_calls>",
} {
if !strings.Contains(transcript, want) {
t.Fatalf("expected transcript to contain %q, got %q", want, transcript)
}
}
}
@@ -221,7 +227,7 @@ func TestApplyCurrentInputFileDisabledPassThrough(t *testing.T) {
DS: ds,
}
req := map[string]any{
"model": "deepseek-v4-flash",
"model": "deepseek-v4-vision",
"messages": historySplitTestMessages(),
}
stdReq, err := promptcompat.NormalizeOpenAIChatRequest(h.Store, req, "")
@@ -244,7 +250,7 @@ func TestApplyCurrentInputFileDisabledPassThrough(t *testing.T) {
}
}
func TestApplyCurrentInputFileUploadsFirstTurnWithInjectedWrapper(t *testing.T) {
func TestApplyCurrentInputFileUploadsFirstTurnWithNumberedHistoryTranscript(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{
Store: mockOpenAIConfig{
@@ -274,34 +280,90 @@ func TestApplyCurrentInputFileUploadsFirstTurnWithInjectedWrapper(t *testing.T)
t.Fatalf("expected 1 current input upload, got %d", len(ds.uploadCalls))
}
upload := ds.uploadCalls[0]
if upload.Filename != "IGNORE.txt" {
if upload.Filename != "DS2API_HISTORY.txt" {
t.Fatalf("unexpected upload filename: %q", upload.Filename)
}
uploadedText := string(upload.Data)
if !strings.HasPrefix(uploadedText, "[file content end]\n\n") {
t.Fatalf("expected injected file wrapper prefix, got %q", uploadedText)
if strings.Contains(uploadedText, "[file content end]") || strings.Contains(uploadedText, "[file content begin]") || strings.Contains(uploadedText, "[file name]:") {
t.Fatalf("expected uploaded transcript without file wrapper tags, got %q", uploadedText)
}
if !strings.Contains(uploadedText, "<begin▁of▁sentence><User>first turn content that is long enough") {
t.Fatalf("expected serialized current user turn markers, got %q", uploadedText)
for _, want := range []string{
"# DS2API_HISTORY.txt",
"=== 1. USER ===",
"first turn content that is long enough",
} {
if !strings.Contains(uploadedText, want) {
t.Fatalf("expected uploaded transcript to contain %q, got %q", want, uploadedText)
}
}
if !strings.Contains(uploadedText, promptcompat.ThinkingInjectionMarker) {
t.Fatalf("expected thinking injection in current input file, got %q", uploadedText)
}
if !strings.HasSuffix(uploadedText, "\n[file name]: IGNORE\n[file content begin]\n") {
t.Fatalf("expected injected file wrapper suffix, got %q", uploadedText)
}
if strings.Contains(out.FinalPrompt, "first turn content that is long enough") {
t.Fatalf("expected current input text to be replaced in live prompt, got %s", out.FinalPrompt)
}
if strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "IGNORE.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
if strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
t.Fatalf("expected live prompt not to instruct file reads, got %s", out.FinalPrompt)
}
if !strings.Contains(out.FinalPrompt, "Answer the latest user request directly.") {
t.Fatalf("expected neutral continuation instruction in live prompt, got %s", out.FinalPrompt)
if !strings.Contains(out.FinalPrompt, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected continuation-oriented prompt in live prompt, got %s", out.FinalPrompt)
}
if len(out.RefFileIDs) != 1 || out.RefFileIDs[0] != "file-inline-1" {
t.Fatalf("expected current input file id in ref_file_ids, got %#v", out.RefFileIDs)
}
if !strings.Contains(out.PromptTokenText, "first turn content that is long enough") {
t.Fatalf("expected prompt token text to preserve original full context, got %q", out.PromptTokenText)
}
if !strings.Contains(out.PromptTokenText, "# DS2API_HISTORY.txt") || !strings.Contains(out.PromptTokenText, "=== 1. USER ===") {
t.Fatalf("expected prompt token text to include numbered history transcript, got %q", out.PromptTokenText)
}
}
func TestApplyCurrentInputFilePreservesFullContextPromptForTokenCounting(t *testing.T) {
ds := &inlineUploadDSStub{}
h := &openAITestSurface{
Store: mockOpenAIConfig{
wideInput: true,
currentInputEnabled: true,
currentInputMin: 0,
thinkingInjection: boolPtr(true),
},
DS: ds,
}
req := map[string]any{
"model": "deepseek-v4-vision",
"messages": historySplitTestMessages(),
}
stdReq, err := promptcompat.NormalizeOpenAIChatRequest(h.Store, req, "")
if err != nil {
t.Fatalf("normalize failed: %v", err)
}
out, err := h.applyCurrentInputFile(context.Background(), &auth.RequestAuth{DeepSeekToken: "token"}, stdReq)
if err != nil {
t.Fatalf("apply current input file failed: %v", err)
}
if out.FinalPrompt == stdReq.FinalPrompt {
t.Fatalf("expected live prompt to be rewritten after current input file")
}
// PromptTokenText must include the uploaded file content (which contains the full context)
// plus the neutral live prompt — reflecting the actual downstream token cost.
if !strings.Contains(out.PromptTokenText, "first user turn") || !strings.Contains(out.PromptTokenText, "latest user turn") {
t.Fatalf("expected prompt token text to contain file context with full conversation, got %q", out.PromptTokenText)
}
if strings.Contains(out.PromptTokenText, "[file content end]") || strings.Contains(out.PromptTokenText, "[file name]:") {
t.Fatalf("expected prompt token text to omit file wrapper tags, got %q", out.PromptTokenText)
}
if !strings.Contains(out.PromptTokenText, "# DS2API_HISTORY.txt") || !strings.Contains(out.PromptTokenText, "=== 1. SYSTEM ===") {
t.Fatalf("expected prompt token text to include numbered history transcript, got %q", out.PromptTokenText)
}
if !strings.Contains(out.PromptTokenText, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected prompt token text to also include continuation prompt, got %q", out.PromptTokenText)
}
if strings.Contains(out.FinalPrompt, "first user turn") || strings.Contains(out.FinalPrompt, "latest user turn") {
t.Fatalf("expected live prompt to hide original turns, got %q", out.FinalPrompt)
}
}
func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
@@ -316,7 +378,7 @@ func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
DS: ds,
}
req := map[string]any{
"model": "deepseek-v4-flash",
"model": "deepseek-v4-vision",
"messages": historySplitTestMessages(),
}
stdReq, err := promptcompat.NormalizeOpenAIChatRequest(h.Store, req, "")
@@ -335,20 +397,23 @@ func TestApplyCurrentInputFileUploadsFullContextFile(t *testing.T) {
t.Fatalf("expected one current input upload, got %d", len(ds.uploadCalls))
}
upload := ds.uploadCalls[0]
if upload.Filename != "IGNORE.txt" {
t.Fatalf("expected IGNORE.txt upload, got %q", upload.Filename)
if upload.Filename != "DS2API_HISTORY.txt" {
t.Fatalf("expected DS2API_HISTORY.txt upload, got %q", upload.Filename)
}
if upload.ModelType != "vision" {
t.Fatalf("expected vision model type for vision request, got %q", upload.ModelType)
}
uploadedText := string(upload.Data)
for _, want := range []string{"system instructions", "first user turn", "hidden reasoning", "tool result", "latest user turn", promptcompat.ThinkingInjectionMarker} {
for _, want := range []string{"# DS2API_HISTORY.txt", "=== 1. SYSTEM ===", "=== 2. USER ===", "=== 3. ASSISTANT ===", "=== 4. TOOL ===", "=== 5. USER ===", "system instructions", "first user turn", "hidden reasoning", "tool result", "latest user turn", promptcompat.ThinkingInjectionMarker} {
if !strings.Contains(uploadedText, want) {
t.Fatalf("expected full context file to contain %q, got %q", want, uploadedText)
}
}
if strings.Contains(out.FinalPrompt, "first user turn") || strings.Contains(out.FinalPrompt, "latest user turn") || strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "IGNORE.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
t.Fatalf("expected live prompt to use only a neutral continuation instruction, got %s", out.FinalPrompt)
if strings.Contains(out.FinalPrompt, "first user turn") || strings.Contains(out.FinalPrompt, "latest user turn") || strings.Contains(out.FinalPrompt, "CURRENT_USER_INPUT.txt") || strings.Contains(out.FinalPrompt, "Read that file") {
t.Fatalf("expected live prompt to use only a continuation instruction, got %s", out.FinalPrompt)
}
if !strings.Contains(out.FinalPrompt, "Answer the latest user request directly.") {
t.Fatalf("expected neutral continuation instruction in live prompt, got %s", out.FinalPrompt)
if !strings.Contains(out.FinalPrompt, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected continuation-oriented prompt in live prompt, got %s", out.FinalPrompt)
}
}
@@ -380,6 +445,9 @@ func TestApplyCurrentInputFileCarriesHistoryText(t *testing.T) {
if out.HistoryText != string(ds.uploadCalls[0].Data) {
t.Fatalf("expected current input file flow to preserve uploaded text in history, got %q", out.HistoryText)
}
if !strings.Contains(out.HistoryText, "# DS2API_HISTORY.txt") || !strings.Contains(out.HistoryText, "=== 1. SYSTEM ===") {
t.Fatalf("expected history text to use numbered transcript format, got %q", out.HistoryText)
}
}
func TestChatCompletionsCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *testing.T) {
@@ -411,15 +479,18 @@ func TestChatCompletionsCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *t
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
upload := ds.uploadCalls[0]
if upload.Filename != "IGNORE.txt" {
if upload.Filename != "DS2API_HISTORY.txt" {
t.Fatalf("unexpected upload filename: %q", upload.Filename)
}
if upload.Purpose != "assistants" {
t.Fatalf("unexpected purpose: %q", upload.Purpose)
}
historyText := string(upload.Data)
if !strings.Contains(historyText, "[file content end]") || !strings.Contains(historyText, "[file name]: IGNORE") {
t.Fatalf("expected injected IGNORE wrapper, got %s", historyText)
if strings.Contains(historyText, "[file content end]") || strings.Contains(historyText, "[file content begin]") || strings.Contains(historyText, "[file name]:") {
t.Fatalf("expected history transcript without file wrapper tags, got %s", historyText)
}
if !strings.Contains(historyText, "# DS2API_HISTORY.txt") || !strings.Contains(historyText, "=== 1. SYSTEM ===") {
t.Fatalf("expected history transcript to use numbered sections, got %s", historyText)
}
if !strings.Contains(historyText, "latest user turn") {
t.Fatalf("expected full context to include latest turn, got %s", historyText)
@@ -428,8 +499,8 @@ func TestChatCompletionsCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *t
t.Fatal("expected completion payload to be captured")
}
promptText, _ := ds.completionReq["prompt"].(string)
if !strings.Contains(promptText, "Answer the latest user request directly.") {
t.Fatalf("expected neutral completion prompt, got %s", promptText)
if !strings.Contains(promptText, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected continuation-oriented prompt, got %s", promptText)
}
if strings.Contains(promptText, "first user turn") || strings.Contains(promptText, "latest user turn") {
t.Fatalf("expected prompt to hide original turns, got %s", promptText)
@@ -438,6 +509,16 @@ func TestChatCompletionsCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *t
if len(refIDs) == 0 || refIDs[0] != "file-inline-1" {
t.Fatalf("expected uploaded current input file to be first ref_file_id, got %#v", ds.completionReq["ref_file_ids"])
}
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response failed: %v", err)
}
usage, _ := body["usage"].(map[string]any)
promptTokens := int(usage["prompt_tokens"].(float64))
neutralCount := util.CountPromptTokens(promptText, "deepseek-v4-flash")
if promptTokens <= neutralCount {
t.Fatalf("expected prompt_tokens to exceed neutral live prompt count (includes file context), got=%d neutral=%d", promptTokens, neutralCount)
}
}
func TestResponsesCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *testing.T) {
@@ -470,16 +551,30 @@ func TestResponsesCurrentInputFileUploadsContextAndKeepsNeutralPrompt(t *testing
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected 1 upload call, got %d", len(ds.uploadCalls))
}
historyText := string(ds.uploadCalls[0].Data)
if !strings.Contains(historyText, "# DS2API_HISTORY.txt") || !strings.Contains(historyText, "=== 1. SYSTEM ===") {
t.Fatalf("expected uploaded history text to use numbered transcript format, got %s", historyText)
}
if ds.completionReq == nil {
t.Fatal("expected completion payload to be captured")
}
promptText, _ := ds.completionReq["prompt"].(string)
if !strings.Contains(promptText, "Answer the latest user request directly.") {
t.Fatalf("expected neutral completion prompt, got %s", promptText)
if !strings.Contains(promptText, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") {
t.Fatalf("expected continuation-oriented prompt, got %s", promptText)
}
if strings.Contains(promptText, "first user turn") || strings.Contains(promptText, "latest user turn") {
t.Fatalf("expected prompt to hide original turns, got %s", promptText)
}
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response failed: %v", err)
}
usage, _ := body["usage"].(map[string]any)
inputTokens := int(usage["input_tokens"].(float64))
neutralCount := util.CountPromptTokens(promptText, "deepseek-v4-flash")
if inputTokens <= neutralCount {
t.Fatalf("expected input_tokens to exceed neutral live prompt count (includes file context), got=%d neutral=%d", inputTokens, neutralCount)
}
}
func TestChatCompletionsCurrentInputFileMapsManagedAuthFailureTo401(t *testing.T) {
@@ -606,11 +701,15 @@ func TestCurrentInputFileWorksAcrossAutoDeleteModes(t *testing.T) {
if len(ds.uploadCalls) != 1 {
t.Fatalf("expected current input upload for mode=%s, got %d", mode, len(ds.uploadCalls))
}
historyText := string(ds.uploadCalls[0].Data)
if !strings.Contains(historyText, "# DS2API_HISTORY.txt") || !strings.Contains(historyText, "=== 1. SYSTEM ===") {
t.Fatalf("expected uploaded history text to use numbered transcript format, got %s", historyText)
}
if ds.completionReq == nil {
t.Fatalf("expected completion payload for mode=%s", mode)
}
promptText, _ := ds.completionReq["prompt"].(string)
if !strings.Contains(promptText, "Answer the latest user request directly.") || strings.Contains(promptText, "first user turn") || strings.Contains(promptText, "latest user turn") {
if !strings.Contains(promptText, "Continue from the latest state in the attached DS2API_HISTORY.txt context.") || strings.Contains(promptText, "first user turn") || strings.Contains(promptText, "latest user turn") {
t.Fatalf("unexpected prompt for mode=%s: %s", mode, promptText)
}
})

View File

@@ -42,6 +42,14 @@ func TestSanitizeLeakedOutputRemovesDanglingThinkBlock(t *testing.T) {
}
}
func TestSanitizeLeakedOutputRemovesCompleteDSMLToolCallWrapper(t *testing.T) {
raw := "前置文本\n<DSMLtool_calls>\n<DSMLinvoke name=\"Bash\">\n<DSMLparameter name=\"command\"></DSMLparameter>\n</DSMLinvoke>\n</DSMLtool_calls>\n后置文本"
got := sanitizeLeakedOutput(raw)
if got != "前置文本\n\n后置文本" {
t.Fatalf("unexpected sanitize result for leaked dsml wrapper: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesAgentXMLLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer</result></attempt_completion>"
got := sanitizeLeakedOutput(raw)

View File

@@ -1,7 +1,6 @@
package responses
import (
"context"
"io"
"net/http"
"strings"
@@ -10,121 +9,12 @@ import (
"ds2api/internal/auth"
"ds2api/internal/config"
dsprotocol "ds2api/internal/deepseek/protocol"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
"ds2api/internal/toolcall"
)
type responsesNonStreamResult struct {
thinking string
toolDetectionThinking string
text string
contentFilter bool
parsed toolcall.ToolCallParseResult
body map[string]any
responseMessageID int
}
func (h *Handler) handleResponsesNonStreamWithRetry(w http.ResponseWriter, ctx context.Context, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
attempts := 0
currentResp := resp
usagePrompt := finalPrompt
accumulatedThinking := ""
accumulatedToolDetectionThinking := ""
for {
result, ok := h.collectResponsesNonStreamAttempt(w, currentResp, responseID, model, usagePrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw)
if !ok {
return
}
accumulatedThinking += sse.TrimContinuationOverlap(accumulatedThinking, result.thinking)
accumulatedToolDetectionThinking += sse.TrimContinuationOverlap(accumulatedToolDetectionThinking, result.toolDetectionThinking)
result.thinking = accumulatedThinking
result.toolDetectionThinking = accumulatedToolDetectionThinking
result.parsed = detectAssistantToolCalls(result.text, result.thinking, result.toolDetectionThinking, toolNames)
result.body = openaifmt.BuildResponseObjectWithToolCalls(responseID, model, usagePrompt, result.thinking, result.text, result.parsed.Calls, toolsRaw)
if !shouldRetryResponsesNonStream(result, attempts) {
h.finishResponsesNonStreamResult(w, result, attempts, owner, responseID, toolChoice, traceID)
return
}
attempts++
config.Logger.Info("[openai_empty_retry] attempting synthetic retry", "surface", "responses", "stream", false, "retry_attempt", attempts, "parent_message_id", result.responseMessageID)
retryPow, powErr := h.DS.GetPow(ctx, a, 3)
if powErr != nil {
config.Logger.Warn("[openai_empty_retry] retry PoW fetch failed, falling back to original PoW", "surface", "responses", "stream", false, "retry_attempt", attempts, "error", powErr)
retryPow = pow
}
nextResp, err := h.DS.CallCompletion(ctx, a, clonePayloadForEmptyOutputRetry(payload, result.responseMessageID), retryPow, 3)
if err != nil {
writeOpenAIError(w, http.StatusInternalServerError, "Failed to get completion.")
config.Logger.Warn("[openai_empty_retry] retry request failed", "surface", "responses", "stream", false, "retry_attempt", attempts, "error", err)
return
}
usagePrompt = usagePromptWithEmptyOutputRetry(finalPrompt, attempts)
currentResp = nextResp
}
}
func (h *Handler) collectResponsesNonStreamAttempt(w http.ResponseWriter, resp *http.Response, responseID, model, usagePrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any) (responsesNonStreamResult, bool) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
writeOpenAIError(w, resp.StatusCode, strings.TrimSpace(string(body)))
return responsesNonStreamResult{}, false
}
result := sse.CollectStream(resp, thinkingEnabled, false)
stripReferenceMarkers := h.compatStripReferenceMarkers()
sanitizedThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
toolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
sanitizedText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
sanitizedText = replaceCitationMarkersWithLinks(sanitizedText, result.CitationLinks)
}
textParsed := detectAssistantToolCalls(sanitizedText, sanitizedThinking, toolDetectionThinking, toolNames)
responseObj := openaifmt.BuildResponseObjectWithToolCalls(responseID, model, usagePrompt, sanitizedThinking, sanitizedText, textParsed.Calls, toolsRaw)
return responsesNonStreamResult{
thinking: sanitizedThinking,
toolDetectionThinking: toolDetectionThinking,
text: sanitizedText,
contentFilter: result.ContentFilter,
parsed: textParsed,
body: responseObj,
responseMessageID: result.ResponseMessageID,
}, true
}
func (h *Handler) finishResponsesNonStreamResult(w http.ResponseWriter, result responsesNonStreamResult, attempts int, owner, responseID string, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
if len(result.parsed.Calls) == 0 && writeUpstreamEmptyOutputError(w, result.text, result.thinking, result.contentFilter) {
config.Logger.Info("[openai_empty_retry] terminal empty output", "surface", "responses", "stream", false, "retry_attempts", attempts, "success_source", "none", "content_filter", result.contentFilter)
return
}
logResponsesToolPolicyRejection(traceID, toolChoice, result.parsed, "text")
if toolChoice.IsRequired() && len(result.parsed.Calls) == 0 {
writeOpenAIErrorWithCode(w, http.StatusUnprocessableEntity, "tool_choice requires at least one valid tool call.", "tool_choice_violation")
return
}
h.getResponseStore().put(owner, responseID, result.body)
writeJSON(w, http.StatusOK, result.body)
source := "first_attempt"
if attempts > 0 {
source = "synthetic_retry"
}
config.Logger.Info("[openai_empty_retry] completed", "surface", "responses", "stream", false, "retry_attempts", attempts, "success_source", source)
}
func shouldRetryResponsesNonStream(result responsesNonStreamResult, attempts int) bool {
return emptyOutputRetryEnabled() &&
attempts < emptyOutputRetryMaxAttempts() &&
!result.contentFilter &&
len(result.parsed.Calls) == 0 &&
strings.TrimSpace(result.text) == ""
}
func (h *Handler) handleResponsesStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
streamRuntime, initialType, ok := h.prepareResponsesStreamRuntime(w, resp, owner, responseID, model, finalPrompt, thinkingEnabled, searchEnabled, toolNames, toolsRaw, toolChoice, traceID)
func (h *Handler) handleResponsesStreamWithRetry(w http.ResponseWriter, r *http.Request, a *auth.RequestAuth, resp *http.Response, payload map[string]any, pow, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
streamRuntime, initialType, ok := h.prepareResponsesStreamRuntime(w, resp, owner, responseID, model, finalPrompt, refFileTokens, thinkingEnabled, searchEnabled, toolNames, toolsRaw, toolChoice, traceID)
if !ok {
return
}
@@ -165,7 +55,7 @@ func (h *Handler) handleResponsesStreamWithRetry(w http.ResponseWriter, r *http.
}
}
func (h *Handler) prepareResponsesStreamRuntime(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) (*responsesStreamRuntime, string, bool) {
func (h *Handler) prepareResponsesStreamRuntime(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) (*responsesStreamRuntime, string, bool) {
if resp.StatusCode != http.StatusOK {
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
@@ -190,6 +80,7 @@ func (h *Handler) prepareResponsesStreamRuntime(w http.ResponseWriter, resp *htt
h.getResponseStore().put(owner, responseID, obj)
},
)
streamRuntime.refFileTokens = refFileTokens
streamRuntime.sendCreated()
return streamRuntime, initialType, true
}
@@ -212,7 +103,13 @@ func (h *Handler) consumeResponsesStreamAttempt(r *http.Request, resp *http.Resp
finalReason = "content_filter"
}
},
OnContextDone: func() {
streamRuntime.markContextCancelled()
},
})
if streamRuntime.finalErrorCode == string(streamengine.StopReasonContextCancelled) {
return true, false
}
terminalWritten := streamRuntime.finalize(finalReason, allowDeferEmpty && finalReason != "content_filter")
if terminalWritten {
return true, false
@@ -225,6 +122,10 @@ func logResponsesStreamTerminal(streamRuntime *responsesStreamRuntime, attempts
if attempts > 0 {
source = "synthetic_retry"
}
if streamRuntime.finalErrorCode == string(streamengine.StopReasonContextCancelled) {
config.Logger.Info("[openai_empty_retry] terminal cancelled", "surface", "responses", "stream", true, "retry_attempts", attempts, "error_code", streamRuntime.finalErrorCode)
return
}
if streamRuntime.failed {
config.Logger.Info("[openai_empty_retry] terminal empty output", "surface", "responses", "stream", true, "retry_attempts", attempts, "success_source", "none", "error_code", streamRuntime.finalErrorCode)
return

View File

@@ -0,0 +1,70 @@
package responses
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"ds2api/internal/promptcompat"
"ds2api/internal/stream"
)
func makeResponsesOpenAISSEHTTPResponse(lines ...string) *http.Response {
body := strings.Join(lines, "\n")
if !strings.HasSuffix(body, "\n") {
body += "\n"
}
return &http.Response{
StatusCode: http.StatusOK,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(body)),
}
}
func TestConsumeResponsesStreamAttemptMarksContextCancelledState(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil).WithContext(ctx)
rec := httptest.NewRecorder()
streamRuntime := newResponsesStreamRuntime(
rec,
http.NewResponseController(rec),
true,
"resp-cancelled",
"deepseek-v4-flash",
"prompt",
false,
false,
true,
nil,
nil,
false,
false,
promptcompat.DefaultToolChoicePolicy(),
"",
nil,
)
resp := makeResponsesOpenAISSEHTTPResponse(
`data: {"p":"response/content","v":"hello"}`,
`data: [DONE]`,
)
h := &Handler{}
terminalWritten, retryable := h.consumeResponsesStreamAttempt(req, resp, streamRuntime, "text", false, true)
if !terminalWritten || retryable {
t.Fatalf("expected cancelled attempt to terminate without retry, got terminalWritten=%v retryable=%v", terminalWritten, retryable)
}
if !streamRuntime.failed {
t.Fatalf("expected cancelled response stream to be marked failed")
}
if got, want := streamRuntime.finalErrorCode, string(stream.StopReasonContextCancelled); got != want {
t.Fatalf("expected cancelled final error code %q, got %q", want, got)
}
if streamRuntime.finalErrorMessage == "" {
t.Fatalf("expected cancelled final error message to be preserved")
}
}

View File

@@ -130,6 +130,6 @@ func filterIncrementalToolCallDeltasByAllowed(deltas []toolstream.ToolCallDelta,
return shared.FilterIncrementalToolCallDeltasByAllowed(deltas, seenNames)
}
func detectAssistantToolCalls(text, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(text, exposedThinking, detectionThinking, toolNames)
func detectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
return shared.DetectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking, toolNames)
}

View File

@@ -0,0 +1,26 @@
package responses
// addRefFileTokensToUsage adds inline-uploaded file token estimates to an existing
// usage map inside a response object. This keeps the token accounting aware of file
// content that the upstream model processes but that is not part of the prompt text.
func addRefFileTokensToUsage(obj map[string]any, refFileTokens int) {
if refFileTokens <= 0 || obj == nil {
return
}
usage, ok := obj["usage"].(map[string]any)
if !ok || usage == nil {
return
}
for _, key := range []string{"input_tokens", "prompt_tokens"} {
if v, ok := usage[key]; ok {
if n, ok := v.(int); ok {
usage[key] = n + refFileTokens
}
}
}
if v, ok := usage["total_tokens"]; ok {
if n, ok := v.(int); ok {
usage["total_tokens"] = n + refFileTokens
}
}
}

View File

@@ -12,6 +12,7 @@ import (
"github.com/google/uuid"
"ds2api/internal/auth"
"ds2api/internal/completionruntime"
"ds2api/internal/config"
dsprotocol "ds2api/internal/deepseek/protocol"
openaifmt "ds2api/internal/format/openai"
@@ -92,36 +93,34 @@ func (h *Handler) Responses(w http.ResponseWriter, r *http.Request) {
return
}
sessionID, err := h.DS.CreateSession(r.Context(), a, 3)
if err != nil {
if a.UseConfigToken {
writeOpenAIError(w, http.StatusUnauthorized, "Account token is invalid. Please re-login the account in admin.")
} else {
writeOpenAIError(w, http.StatusUnauthorized, "Invalid token. If this should be a DS2API key, add it to config.keys first.")
responseID := "resp_" + strings.ReplaceAll(uuid.NewString(), "-", "")
if !stdReq.Stream {
result, outErr := completionruntime.ExecuteNonStreamWithRetry(r.Context(), h.DS, a, stdReq, completionruntime.Options{
StripReferenceMarkers: h.compatStripReferenceMarkers(),
RetryEnabled: true,
})
if outErr != nil {
writeOpenAIErrorWithCode(w, outErr.Status, outErr.Message, outErr.Code)
return
}
return
}
pow, err := h.DS.GetPow(r.Context(), a, 3)
if err != nil {
writeOpenAIError(w, http.StatusUnauthorized, "Failed to get PoW (invalid token or unknown error).")
return
}
payload := stdReq.CompletionPayload(sessionID)
resp, err := h.DS.CallCompletion(r.Context(), a, payload, pow, 3)
if err != nil {
writeOpenAIError(w, http.StatusInternalServerError, "Failed to get completion.")
responseObj := openaifmt.BuildResponseObjectWithToolCalls(responseID, stdReq.ResponseModel, result.Turn.Prompt, result.Turn.Thinking, result.Turn.Text, result.Turn.ToolCalls, stdReq.ToolsRaw)
responseObj["usage"] = responsesUsageFromTurn(result.Turn)
h.getResponseStore().put(owner, responseID, responseObj)
writeJSON(w, http.StatusOK, responseObj)
return
}
responseID := "resp_" + strings.ReplaceAll(uuid.NewString(), "-", "")
if stdReq.Stream {
h.handleResponsesStreamWithRetry(w, r, a, resp, payload, pow, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
start, outErr := completionruntime.StartCompletion(r.Context(), h.DS, a, stdReq, completionruntime.Options{})
if outErr != nil {
writeOpenAIErrorWithCode(w, outErr.Status, outErr.Message, outErr.Code)
return
}
h.handleResponsesNonStreamWithRetry(w, r.Context(), a, resp, payload, pow, owner, responseID, stdReq.ResponseModel, stdReq.FinalPrompt, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
refFileTokens := stdReq.RefFileTokens
h.handleResponsesStreamWithRetry(w, r, a, start.Response, start.Payload, start.Pow, owner, responseID, stdReq.ResponseModel, stdReq.PromptTokenText, refFileTokens, stdReq.Thinking, stdReq.Search, stdReq.ToolNames, stdReq.ToolsRaw, stdReq.ToolChoice, traceID)
}
func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Response, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -131,12 +130,11 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
result := sse.CollectStream(resp, thinkingEnabled, true)
stripReferenceMarkers := h.compatStripReferenceMarkers()
sanitizedThinking := cleanVisibleOutput(result.Thinking, stripReferenceMarkers)
toolDetectionThinking := cleanVisibleOutput(result.ToolDetectionThinking, stripReferenceMarkers)
sanitizedText := cleanVisibleOutput(result.Text, stripReferenceMarkers)
if searchEnabled {
sanitizedText = replaceCitationMarkersWithLinks(sanitizedText, result.CitationLinks)
}
textParsed := detectAssistantToolCalls(sanitizedText, sanitizedThinking, toolDetectionThinking, toolNames)
textParsed := detectAssistantToolCalls(result.Text, sanitizedText, result.Thinking, result.ToolDetectionThinking, toolNames)
if len(textParsed.Calls) == 0 && writeUpstreamEmptyOutputError(w, sanitizedText, sanitizedThinking, result.ContentFilter) {
return
}
@@ -149,11 +147,14 @@ func (h *Handler) handleResponsesNonStream(w http.ResponseWriter, resp *http.Res
}
responseObj := openaifmt.BuildResponseObjectWithToolCalls(responseID, model, finalPrompt, sanitizedThinking, sanitizedText, textParsed.Calls, toolsRaw)
if refFileTokens > 0 {
addRefFileTokensToUsage(responseObj, refFileTokens)
}
h.getResponseStore().put(owner, responseID, responseObj)
writeJSON(w, http.StatusOK, responseObj)
}
func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request, resp *http.Response, owner, responseID, model, finalPrompt string, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request, resp *http.Response, owner, responseID, model, finalPrompt string, refFileTokens int, thinkingEnabled, searchEnabled bool, toolNames []string, toolsRaw any, toolChoice promptcompat.ToolChoicePolicy, traceID string) {
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
@@ -195,6 +196,7 @@ func (h *Handler) handleResponsesStream(w http.ResponseWriter, r *http.Request,
h.getResponseStore().put(owner, responseID, obj)
},
)
streamRuntime.refFileTokens = refFileTokens
streamRuntime.sendCreated()
streamengine.ConsumeSSE(streamengine.ConsumeConfig{

View File

@@ -0,0 +1,39 @@
package responses
import (
"strings"
openaifmt "ds2api/internal/format/openai"
)
type responsesDeltaBatch struct {
runtime *responsesStreamRuntime
kind string
text strings.Builder
}
func (b *responsesDeltaBatch) append(kind, text string) {
if text == "" {
return
}
if b.kind != "" && b.kind != kind {
b.flush()
}
b.kind = kind
b.text.WriteString(text)
}
func (b *responsesDeltaBatch) flush() {
if b.kind == "" || b.text.Len() == 0 {
return
}
text := b.text.String()
switch b.kind {
case "reasoning":
b.runtime.sendEvent("response.reasoning.delta", openaifmt.BuildResponsesReasoningDeltaPayload(b.runtime.responseID, text))
case "text":
b.runtime.emitTextDelta(text)
}
b.kind = ""
b.text.Reset()
}

View File

@@ -1,12 +1,14 @@
package responses
import (
"ds2api/internal/assistantturn"
"ds2api/internal/toolcall"
"net/http"
"strings"
"ds2api/internal/config"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/httpapi/openai/shared"
"ds2api/internal/promptcompat"
"ds2api/internal/sse"
streamengine "ds2api/internal/stream"
@@ -18,13 +20,14 @@ type responsesStreamRuntime struct {
rc *http.ResponseController
canFlush bool
responseID string
model string
finalPrompt string
toolNames []string
toolsRaw any
traceID string
toolChoice promptcompat.ToolChoicePolicy
responseID string
model string
finalPrompt string
refFileTokens int
toolNames []string
toolsRaw any
traceID string
toolChoice promptcompat.ToolChoicePolicy
thinkingEnabled bool
searchEnabled bool
@@ -35,29 +38,27 @@ type responsesStreamRuntime struct {
toolCallsEmitted bool
toolCallsDoneEmitted bool
sieve toolstream.State
thinking strings.Builder
toolDetectionThinking strings.Builder
text strings.Builder
visibleText strings.Builder
responseMessageID int
streamToolCallIDs map[int]string
functionItemIDs map[int]string
functionOutputIDs map[int]int
functionArgs map[int]string
functionDone map[int]bool
functionAdded map[int]bool
functionNames map[int]string
messageItemID string
messageOutputID int
nextOutputID int
messageAdded bool
messagePartAdded bool
sequence int
failed bool
finalErrorStatus int
finalErrorMessage string
finalErrorCode string
sieve toolstream.State
accumulator shared.StreamAccumulator
visibleText strings.Builder
responseMessageID int
streamToolCallIDs map[int]string
functionItemIDs map[int]string
functionOutputIDs map[int]int
functionArgs map[int]string
functionDone map[int]bool
functionAdded map[int]bool
functionNames map[int]string
messageItemID string
messageOutputID int
nextOutputID int
messageAdded bool
messagePartAdded bool
sequence int
failed bool
finalErrorStatus int
finalErrorMessage string
finalErrorCode string
persistResponse func(obj map[string]any)
}
@@ -105,6 +106,11 @@ func newResponsesStreamRuntime(
toolChoice: toolChoice,
traceID: traceID,
persistResponse: persistResponse,
accumulator: shared.StreamAccumulator{
ThinkingEnabled: thinkingEnabled,
SearchEnabled: searchEnabled,
StripReferenceMarkers: stripReferenceMarkers,
},
}
}
@@ -136,21 +142,47 @@ func (s *responsesStreamRuntime) failResponse(status int, message, code string)
s.sendDone()
}
func (s *responsesStreamRuntime) markContextCancelled() {
s.failed = true
s.finalErrorStatus = 499
s.finalErrorMessage = "request context cancelled"
s.finalErrorCode = string(streamengine.StopReasonContextCancelled)
}
func (s *responsesStreamRuntime) finalize(finishReason string, deferEmptyOutput bool) bool {
s.failed = false
s.finalErrorStatus = 0
s.finalErrorMessage = ""
s.finalErrorCode = ""
finalThinking := s.thinking.String()
finalToolDetectionThinking := s.toolDetectionThinking.String()
finalText := cleanVisibleOutput(s.text.String(), s.stripReferenceMarkers)
if s.bufferToolContent {
s.processToolStreamEvents(toolstream.Flush(&s.sieve, s.toolNames), true, true)
}
textParsed := detectAssistantToolCalls(finalText, finalThinking, finalToolDetectionThinking, s.toolNames)
detected := textParsed.Calls
finalThinking := s.accumulator.Thinking.String()
finalToolDetectionThinking := s.accumulator.ToolDetectionThinking.String()
finalText := s.accumulator.Text.String()
turn := assistantturn.BuildTurnFromStreamSnapshot(assistantturn.StreamSnapshot{
RawText: s.accumulator.RawText.String(),
VisibleText: finalText,
RawThinking: s.accumulator.RawThinking.String(),
VisibleThinking: finalThinking,
DetectionThinking: finalToolDetectionThinking,
ContentFilter: finishReason == "content_filter",
ResponseMessageID: s.responseMessageID,
AlreadyEmittedCalls: s.toolCallsEmitted,
AlreadyEmittedToolRaw: s.toolCallsDoneEmitted,
}, assistantturn.BuildOptions{
Model: s.model,
Prompt: s.finalPrompt,
RefFileTokens: s.refFileTokens,
SearchEnabled: s.searchEnabled,
StripReferenceMarkers: s.stripReferenceMarkers,
ToolNames: s.toolNames,
ToolsRaw: s.toolsRaw,
ToolChoice: s.toolChoice,
})
textParsed := turn.ParsedToolCalls
detected := turn.ToolCalls
s.logToolPolicyRejections(textParsed)
if len(detected) > 0 {
@@ -162,12 +194,15 @@ func (s *responsesStreamRuntime) finalize(finishReason string, deferEmptyOutput
s.closeMessageItem()
if s.toolChoice.IsRequired() && len(detected) == 0 {
s.failResponse(http.StatusUnprocessableEntity, "tool_choice requires at least one valid tool call.", "tool_choice_violation")
if turn.Error != nil && turn.Error.Code == "tool_choice_violation" {
s.failResponse(turn.Error.Status, turn.Error.Message, turn.Error.Code)
return true
}
if len(detected) == 0 && strings.TrimSpace(finalText) == "" {
status, message, code := upstreamEmptyOutputDetail(finishReason == "content_filter", finalText, finalThinking)
if len(detected) == 0 && strings.TrimSpace(turn.Text) == "" {
status, message, code := upstreamEmptyOutputDetail(finishReason == "content_filter", turn.Text, turn.Thinking)
if turn.Error != nil {
status, message, code = turn.Error.Status, turn.Error.Message, turn.Error.Code
}
if deferEmptyOutput {
s.finalErrorStatus = status
s.finalErrorMessage = message
@@ -179,7 +214,7 @@ func (s *responsesStreamRuntime) finalize(finishReason string, deferEmptyOutput
}
s.closeIncompleteFunctionItems()
obj := s.buildCompletedResponseObject(finalThinking, finalText, detected)
obj := s.buildCompletedResponseObject(turn.Thinking, turn.Text, detected)
if s.persistResponse != nil {
s.persistResponse(obj)
}
@@ -188,6 +223,14 @@ func (s *responsesStreamRuntime) finalize(finishReason string, deferEmptyOutput
return true
}
func responsesUsageFromTurn(turn assistantturn.Turn) map[string]any {
return map[string]any{
"input_tokens": turn.Usage.InputTokens,
"output_tokens": turn.Usage.OutputTokens,
"total_tokens": turn.Usage.TotalTokens,
}
}
func (s *responsesStreamRuntime) logToolPolicyRejections(textParsed toolcall.ToolCallParseResult) {
logRejected := func(parsed toolcall.ToolCallParseResult, channel string) {
rejected := filteredRejectedToolNamesForLog(parsed.RejectedToolNames)
@@ -219,46 +262,27 @@ func (s *responsesStreamRuntime) onParsed(parsed sse.LineResult) streamengine.Pa
return streamengine.ParsedDecision{Stop: true}
}
contentSeen := false
for _, p := range parsed.ToolDetectionThinkingParts {
trimmed := sse.TrimContinuationOverlap(s.toolDetectionThinking.String(), p.Text)
if trimmed != "" {
s.toolDetectionThinking.WriteString(trimmed)
}
}
for _, p := range parsed.Parts {
cleanedText := cleanVisibleOutput(p.Text, s.stripReferenceMarkers)
if cleanedText == "" {
continue
}
if p.Type != "thinking" && s.searchEnabled && sse.IsCitation(cleanedText) {
continue
}
contentSeen = true
batch := responsesDeltaBatch{runtime: s}
accumulated := s.accumulator.Apply(parsed)
for _, p := range accumulated.Parts {
if p.Type == "thinking" {
if !s.thinkingEnabled {
continue
}
trimmed := sse.TrimContinuationOverlap(s.thinking.String(), cleanedText)
if trimmed == "" {
continue
}
s.thinking.WriteString(trimmed)
s.sendEvent("response.reasoning.delta", openaifmt.BuildResponsesReasoningDeltaPayload(s.responseID, trimmed))
batch.append("reasoning", p.VisibleText)
continue
}
trimmed := sse.TrimContinuationOverlap(s.text.String(), cleanedText)
if trimmed == "" {
if p.RawText == "" {
continue
}
if p.CitationOnly {
continue
}
s.text.WriteString(trimmed)
if !s.bufferToolContent {
s.emitTextDelta(trimmed)
batch.append("text", p.VisibleText)
continue
}
s.processToolStreamEvents(toolstream.ProcessChunk(&s.sieve, trimmed, s.toolNames), true, true)
batch.flush()
s.processToolStreamEvents(toolstream.ProcessChunk(&s.sieve, p.RawText, s.toolNames), true, true)
}
return streamengine.ParsedDecision{ContentSeen: contentSeen}
batch.flush()
return streamengine.ParsedDecision{ContentSeen: accumulated.ContentSeen}
}

View File

@@ -4,6 +4,7 @@ import (
"encoding/json"
openaifmt "ds2api/internal/format/openai"
"ds2api/internal/sse"
"ds2api/internal/toolstream"
)
@@ -43,7 +44,10 @@ func (s *responsesStreamRuntime) sendDone() {
func (s *responsesStreamRuntime) processToolStreamEvents(events []toolstream.Event, emitContent bool, resetAfterToolCalls bool) {
for _, evt := range events {
if emitContent && evt.Content != "" {
s.emitTextDelta(evt.Content)
cleaned := cleanVisibleOutput(evt.Content, s.stripReferenceMarkers)
if cleaned != "" && (!s.searchEnabled || !sse.IsCitation(cleaned)) {
s.emitTextDelta(cleaned)
}
}
if len(evt.ToolCallDeltas) > 0 {
if !s.emitEarlyToolDeltas {

View File

@@ -145,7 +145,7 @@ func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, fin
}
}
return openaifmt.BuildResponseObjectFromItems(
obj := openaifmt.BuildResponseObjectFromItems(
s.responseID,
s.model,
s.finalPrompt,
@@ -154,4 +154,8 @@ func (s *responsesStreamRuntime) buildCompletedResponseObject(finalThinking, fin
output,
outputText,
)
if s.refFileTokens > 0 {
addRefFileTokensToUsage(obj, s.refFileTokens)
}
return obj
}

View File

@@ -27,7 +27,7 @@ func TestHandleResponsesStreamDoesNotEmitReasoningTextCompatEvents(t *testing.T)
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.reasoning.delta") {
@@ -57,7 +57,7 @@ func TestHandleResponsesStreamEmitsOutputTextDoneBeforeContentPartDone(t *testin
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.output_text.done") {
t.Fatalf("expected response.output_text.done payload, body=%s", body)
@@ -91,7 +91,7 @@ func TestHandleResponsesStreamOutputTextDeltaCarriesItemIndexes(t *testing.T) {
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
deltaPayload, ok := extractSSEEventPayload(body, "response.output_text.delta")
@@ -109,6 +109,48 @@ func TestHandleResponsesStreamOutputTextDeltaCarriesItemIndexes(t *testing.T) {
}
}
func TestHandleResponsesStreamCoalescesSmallOutputTextDeltas(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
rec := httptest.NewRecorder()
var streamBody strings.Builder
for i := 0; i < 100; i++ {
b, _ := json.Marshal(map[string]any{
"p": "response/content",
"v": "字",
})
streamBody.WriteString("data: ")
streamBody.WriteString(string(b))
streamBody.WriteString("\n")
}
streamBody.WriteString("data: [DONE]\n")
resp := &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(streamBody.String())),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_coalesce", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
payloads := extractSSEEventPayloads(rec.Body.String(), "response.output_text.delta")
if len(payloads) == 0 {
t.Fatalf("expected response.output_text.delta payloads, body=%s", rec.Body.String())
}
var content strings.Builder
for _, payload := range payloads {
content.WriteString(asString(payload["delta"]))
}
if got, want := content.String(), strings.Repeat("字", 100); got != want {
t.Fatalf("coalesced response content mismatch: got %q want %q body=%s", got, want, rec.Body.String())
}
if len(payloads) >= 100 {
t.Fatalf("expected coalescing to reduce 100 tiny text deltas, got %d body=%s", len(payloads), rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "event: response.completed") {
t.Fatalf("expected completed event, body=%s", rec.Body.String())
}
}
func TestHandleResponsesStreamEmitsDistinctToolCallIDsAcrossSeparateToolBlocks(t *testing.T) {
h := &Handler{}
req := httptest.NewRequest(http.MethodPost, "/v1/responses", nil)
@@ -130,7 +172,7 @@ func TestHandleResponsesStreamEmitsDistinctToolCallIDsAcrossSeparateToolBlocks(t
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, []string{"read_file", "search"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file", "search"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
doneEvents := extractSSEEventPayloads(body, "response.function_call_arguments.done")
@@ -183,7 +225,7 @@ func TestHandleResponsesStreamRequiredToolChoiceFailure(t *testing.T) {
Mode: promptcompat.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.failed") {
@@ -213,7 +255,7 @@ func TestHandleResponsesStreamFailsWhenUpstreamHasOnlyThinking(t *testing.T) {
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.failed") {
@@ -251,11 +293,11 @@ func TestHandleResponsesStreamPromotesThinkingToolCallsOnFinalizeWithoutMidstrea
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
body := rec.Body.String()
if !strings.Contains(body, "event: response.reasoning.delta") {
t.Fatalf("expected reasoning delta in stream body, got %s", body)
if strings.Contains(body, "event: response.reasoning.delta") {
t.Fatalf("did not expect leaked reasoning delta in stream body, got %s", body)
}
if !strings.Contains(body, "event: response.function_call_arguments.done") {
t.Fatalf("expected finalize fallback function call event, got %s", body)
@@ -288,7 +330,7 @@ func TestHandleResponsesStreamPromotesHiddenThinkingDSMLToolCallsOnFinalize(t *t
Mode: promptcompat.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
body := rec.Body.String()
if strings.Contains(body, "event: response.reasoning.delta") {
@@ -317,7 +359,7 @@ func TestHandleResponsesNonStreamRequiredToolChoiceViolation(t *testing.T) {
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("expected 422 for required tool_choice violation, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -344,7 +386,7 @@ func TestHandleResponsesNonStreamRequiredToolChoiceIgnoresThinkingToolPayloadWhe
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", true, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, true, false, []string{"read_file"}, nil, policy, "")
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("expected 422 for required tool_choice violation, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -366,7 +408,7 @@ func TestHandleResponsesNonStreamReturns429WhenUpstreamOutputEmpty(t *testing.T)
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected 429 for empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -388,7 +430,7 @@ func TestHandleResponsesNonStreamReturnsContentFilterErrorWhenUpstreamFilteredWi
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-flash", "prompt", 0, false, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400 for filtered empty upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -410,14 +452,26 @@ func TestHandleResponsesNonStreamReturns429WhenUpstreamHasOnlyThinking(t *testin
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("expected 429 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, nil, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for thinking-only upstream output, got %d body=%s", rec.Code, rec.Body.String())
}
out := decodeJSONBody(t, rec.Body.String())
errObj, _ := out["error"].(map[string]any)
if asString(errObj["code"]) != "upstream_empty_output" {
t.Fatalf("expected code=upstream_empty_output, got %#v", out)
output, _ := out["output"].([]any)
if len(output) == 0 {
t.Fatal("expected at least one output item")
}
first, _ := output[0].(map[string]any)
content, _ := first["content"].([]any)
if len(content) == 0 {
t.Fatal("expected at least one content item")
}
firstContent, _ := content[0].(map[string]any)
if asString(firstContent["type"]) != "reasoning" {
t.Fatalf("expected reasoning type, got %v", firstContent["type"])
}
if asString(firstContent["text"]) != "Only thinking" {
t.Fatalf("expected text='Only thinking', got %v", firstContent["text"])
}
}
@@ -432,7 +486,7 @@ func TestHandleResponsesNonStreamPromotesThinkingToolCallsWhenTextEmpty(t *testi
)),
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_test", "deepseek-v4-pro", "prompt", 0, true, false, []string{"read_file"}, nil, promptcompat.DefaultToolChoicePolicy(), "")
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -462,7 +516,7 @@ func TestHandleResponsesNonStreamPromotesHiddenThinkingDSMLToolCallsWhenTextEmpt
Mode: promptcompat.ToolChoiceRequired,
Allowed: map[string]struct{}{"read_file": {}},
}
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", false, false, []string{"read_file"}, nil, policy, "")
h.handleResponsesNonStream(rec, resp, "owner-a", "resp_hidden", "deepseek-v4-pro", "prompt", 0, false, false, []string{"read_file"}, nil, policy, "")
if rec.Code != http.StatusOK {
t.Fatalf("expected 200 for hidden thinking tool calls, got %d body=%s", rec.Code, rec.Body.String())
}
@@ -509,7 +563,7 @@ func TestHandleResponsesStreamCoercesSchemaDeclaredStringArguments(t *testing.T)
Body: io.NopCloser(strings.NewReader(streamBody)),
}
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_string_protect", "deepseek-v4-flash", "prompt", false, false, []string{"Write"}, toolsRaw, promptcompat.DefaultToolChoicePolicy(), "")
h.handleResponsesStream(rec, req, resp, "owner-a", "resp_string_protect", "deepseek-v4-flash", "prompt", 0, false, false, []string{"Write"}, toolsRaw, promptcompat.DefaultToolChoicePolicy(), "")
payload, ok := extractSSEEventPayload(rec.Body.String(), "response.function_call_arguments.done")
if !ok {

View File

@@ -6,12 +6,12 @@ import (
"ds2api/internal/toolcall"
)
func DetectAssistantToolCalls(text, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
textParsed := toolcall.ParseStandaloneToolCallsDetailed(text, toolNames)
func DetectAssistantToolCalls(rawText, visibleText, exposedThinking, detectionThinking string, toolNames []string) toolcall.ToolCallParseResult {
textParsed := toolcall.ParseStandaloneToolCallsDetailed(rawText, toolNames)
if len(textParsed.Calls) > 0 {
return textParsed
}
if strings.TrimSpace(text) != "" {
if strings.TrimSpace(visibleText) != "" {
return textParsed
}
thinking := detectionThinking

View File

@@ -13,7 +13,7 @@ func ReplaceCitationMarkersWithLinks(text string, links map[int]string) string {
if strings.TrimSpace(text) == "" || len(links) == 0 {
return text
}
zeroBased := strings.Contains(strings.ToLower(text), "[reference:0]")
zeroBasedReference := hasZeroBasedReferenceMarker(text)
return citationMarkerPattern.ReplaceAllStringFunc(text, func(match string) string {
sub := citationMarkerPattern.FindStringSubmatch(match)
if len(sub) < 3 {
@@ -24,7 +24,7 @@ func ReplaceCitationMarkersWithLinks(text string, links map[int]string) string {
return match
}
lookupIdx := idx
if zeroBased {
if strings.EqualFold(sub[1], "reference") && zeroBasedReference {
lookupIdx = idx + 1
}
url := strings.TrimSpace(links[lookupIdx])
@@ -34,3 +34,16 @@ func ReplaceCitationMarkersWithLinks(text string, links map[int]string) string {
return fmt.Sprintf("[%d](%s)", idx, url)
})
}
func hasZeroBasedReferenceMarker(text string) bool {
for _, sub := range citationMarkerPattern.FindAllStringSubmatch(text, -1) {
if len(sub) < 3 || !strings.EqualFold(sub[1], "reference") {
continue
}
idx, err := strconv.Atoi(strings.TrimSpace(sub[2]))
if err == nil && idx == 0 {
return true
}
}
return false
}

View File

@@ -3,6 +3,8 @@ package shared
import (
"regexp"
"strings"
"ds2api/internal/toolcall"
)
var emptyJSONFencePattern = regexp.MustCompile("(?is)```json\\s*```")
@@ -47,10 +49,42 @@ func sanitizeLeakedOutput(text string) string {
out = leakedThinkTagPattern.ReplaceAllString(out, "")
out = leakedBOSMarkerPattern.ReplaceAllString(out, "")
out = leakedMetaMarkerPattern.ReplaceAllString(out, "")
out = stripLeakedToolCallWrapperBlocks(out)
out = sanitizeLeakedAgentXMLBlocks(out)
return out
}
func stripLeakedToolCallWrapperBlocks(text string) string {
if text == "" {
return text
}
var b strings.Builder
pos := 0
for pos < len(text) {
tag, ok := toolcall.FindToolMarkupTagOutsideIgnored(text, pos)
if !ok {
b.WriteString(text[pos:])
break
}
if tag.Start > pos {
b.WriteString(text[pos:tag.Start])
}
if tag.Closing || tag.Name != "tool_calls" {
b.WriteString(text[tag.Start : tag.End+1])
pos = tag.End + 1
continue
}
closeTag, ok := toolcall.FindMatchingToolMarkupClose(text, tag)
if !ok {
b.WriteString(text[tag.Start : tag.End+1])
pos = tag.End + 1
continue
}
pos = closeTag.End + 1
}
return b.String()
}
func stripDanglingThinkSuffix(text string) string {
matches := leakedThinkTagPattern.FindAllStringIndex(text, -1)
if len(matches) == 0 {

View File

@@ -0,0 +1,104 @@
package shared
import (
"strings"
"ds2api/internal/sse"
)
type StreamAccumulator struct {
ThinkingEnabled bool
SearchEnabled bool
StripReferenceMarkers bool
RawThinking strings.Builder
Thinking strings.Builder
ToolDetectionThinking strings.Builder
RawText strings.Builder
Text strings.Builder
}
type StreamPartDelta struct {
Type string
RawText string
VisibleText string
CitationOnly bool
}
type StreamAccumulatorResult struct {
ContentSeen bool
Parts []StreamPartDelta
}
func (a *StreamAccumulator) Apply(parsed sse.LineResult) StreamAccumulatorResult {
out := StreamAccumulatorResult{}
for _, p := range parsed.ToolDetectionThinkingParts {
trimmed := sse.TrimContinuationOverlapFromBuilder(&a.ToolDetectionThinking, p.Text)
if trimmed != "" {
a.ToolDetectionThinking.WriteString(trimmed)
}
}
for _, p := range parsed.Parts {
if p.Type == "thinking" {
delta := a.applyThinkingPart(p.Text)
if delta.RawText != "" {
out.ContentSeen = true
}
if delta.RawText != "" || delta.VisibleText != "" {
out.Parts = append(out.Parts, delta)
}
continue
}
delta := a.applyTextPart(p.Text)
if delta.RawText != "" {
out.ContentSeen = true
}
if delta.RawText != "" || delta.VisibleText != "" || delta.CitationOnly {
out.Parts = append(out.Parts, delta)
}
}
return out
}
func (a *StreamAccumulator) applyThinkingPart(text string) StreamPartDelta {
rawTrimmed := sse.TrimContinuationOverlapFromBuilder(&a.RawThinking, text)
if rawTrimmed != "" {
a.RawThinking.WriteString(rawTrimmed)
}
delta := StreamPartDelta{Type: "thinking", RawText: rawTrimmed}
if !a.ThinkingEnabled || rawTrimmed == "" {
return delta
}
cleanedText := CleanVisibleOutput(rawTrimmed, a.StripReferenceMarkers)
if cleanedText == "" {
return delta
}
trimmed := sse.TrimContinuationOverlapFromBuilder(&a.Thinking, cleanedText)
if trimmed == "" {
return delta
}
a.Thinking.WriteString(trimmed)
delta.VisibleText = trimmed
return delta
}
func (a *StreamAccumulator) applyTextPart(text string) StreamPartDelta {
rawTrimmed := sse.TrimContinuationOverlapFromBuilder(&a.RawText, text)
if rawTrimmed == "" {
return StreamPartDelta{Type: "text"}
}
a.RawText.WriteString(rawTrimmed)
delta := StreamPartDelta{Type: "text", RawText: rawTrimmed}
cleanedText := CleanVisibleOutput(rawTrimmed, a.StripReferenceMarkers)
if a.SearchEnabled && sse.IsCitation(cleanedText) {
delta.CitationOnly = true
return delta
}
trimmed := sse.TrimContinuationOverlapFromBuilder(&a.Text, cleanedText)
if trimmed == "" {
return delta
}
a.Text.WriteString(trimmed)
delta.VisibleText = trimmed
return delta
}

View File

@@ -0,0 +1,97 @@
package shared
import (
"testing"
"ds2api/internal/sse"
)
func TestStreamAccumulatorAppliesThinkingAndTextDedupe(t *testing.T) {
acc := StreamAccumulator{ThinkingEnabled: true, StripReferenceMarkers: true}
thinkingPrefix := "this is a long thinking snapshot prefix used by DeepSeek continue replay"
textPrefix := "this is a long visible answer snapshot prefix used by DeepSeek continue replay"
first := acc.Apply(sse.LineResult{
Parsed: true,
Parts: []sse.ContentPart{
{Type: "thinking", Text: thinkingPrefix},
{Type: "text", Text: textPrefix},
},
})
second := acc.Apply(sse.LineResult{
Parsed: true,
Parts: []sse.ContentPart{
{Type: "thinking", Text: thinkingPrefix + " next"},
{Type: "text", Text: textPrefix + " world"},
},
})
if !first.ContentSeen || !second.ContentSeen {
t.Fatalf("expected both chunks to mark content seen")
}
if got := acc.RawThinking.String(); got != thinkingPrefix+" next" {
t.Fatalf("raw thinking = %q", got)
}
if got := acc.Thinking.String(); got != thinkingPrefix+" next" {
t.Fatalf("thinking = %q", got)
}
if got := acc.RawText.String(); got != textPrefix+" world" {
t.Fatalf("raw text = %q", got)
}
if got := acc.Text.String(); got != textPrefix+" world" {
t.Fatalf("text = %q", got)
}
if got := second.Parts[0].VisibleText; got != " next" {
t.Fatalf("thinking delta = %q", got)
}
if got := second.Parts[1].VisibleText; got != " world" {
t.Fatalf("text delta = %q", got)
}
}
func TestStreamAccumulatorKeepsHiddenThinkingForToolDetection(t *testing.T) {
acc := StreamAccumulator{ThinkingEnabled: false, StripReferenceMarkers: true}
result := acc.Apply(sse.LineResult{
Parsed: true,
Parts: []sse.ContentPart{
{Type: "thinking", Text: "<tool_calls></tool_calls>"},
},
ToolDetectionThinkingParts: []sse.ContentPart{
{Type: "thinking", Text: "detect"},
{Type: "thinking", Text: " tools"},
},
})
if !result.ContentSeen {
t.Fatalf("expected hidden thinking to count as upstream content")
}
if got := acc.RawThinking.String(); got != "<tool_calls></tool_calls>" {
t.Fatalf("raw thinking = %q", got)
}
if got := acc.Thinking.String(); got != "" {
t.Fatalf("visible thinking = %q", got)
}
if got := acc.ToolDetectionThinking.String(); got != "detect tools" {
t.Fatalf("tool detection thinking = %q", got)
}
}
func TestStreamAccumulatorSuppressesCitationTextWhenSearchEnabled(t *testing.T) {
acc := StreamAccumulator{SearchEnabled: true, StripReferenceMarkers: true}
result := acc.Apply(sse.LineResult{
Parsed: true,
Parts: []sse.ContentPart{{Type: "text", Text: "[citation:1]"}},
})
if !result.ContentSeen {
t.Fatalf("expected citation chunk to mark upstream content")
}
if len(result.Parts) != 1 || !result.Parts[0].CitationOnly {
t.Fatalf("expected citation-only delta, got %#v", result.Parts)
}
if got := acc.RawText.String(); got != "[citation:1]" {
t.Fatalf("raw text = %q", got)
}
if got := acc.Text.String(); got != "" {
t.Fatalf("visible text = %q", got)
}
}

View File

@@ -1,9 +1,12 @@
package shared
import "net/http"
import (
"net/http"
"strings"
)
func ShouldWriteUpstreamEmptyOutputError(text string) bool {
return text == ""
func ShouldWriteUpstreamEmptyOutputError(text, thinking string) bool {
return strings.TrimSpace(text) == "" && strings.TrimSpace(thinking) == ""
}
func UpstreamEmptyOutputDetail(contentFilter bool, text, thinking string) (int, string, string) {
@@ -18,7 +21,7 @@ func UpstreamEmptyOutputDetail(contentFilter bool, text, thinking string) (int,
}
func WriteUpstreamEmptyOutputError(w http.ResponseWriter, text, thinking string, contentFilter bool) bool {
if !ShouldWriteUpstreamEmptyOutputError(text) {
if !ShouldWriteUpstreamEmptyOutputError(text, thinking) {
return false
}
status, message, code := UpstreamEmptyOutputDetail(contentFilter, text, thinking)

View File

@@ -345,7 +345,7 @@ func TestChatCompletionsStreamRetriesEmptyOutputOnSameSession(t *testing.T) {
func TestChatCompletionsNonStreamRetriesThinkingOnlyOutput(t *testing.T) {
ds := &streamStatusDSSeqStub{resps: []*http.Response{
makeOpenAISSEHTTPResponse(`data: {"response_message_id":99,"p":"response/thinking_content","v":"plan"}`, "data: [DONE]"),
makeOpenAISSEHTTPResponse(`data: {"response_message_id":99}`, "data: [DONE]"),
makeOpenAISSEHTTPResponse(`data: {"p":"response/content","v":"visible"}`, "data: [DONE]"),
}}
h := &openAITestSurface{
@@ -380,9 +380,6 @@ func TestChatCompletionsNonStreamRetriesThinkingOnlyOutput(t *testing.T) {
if asString(message["content"]) != "visible" {
t.Fatalf("expected retry visible content, got %#v", message)
}
if !strings.Contains(asString(message["reasoning_content"]), "plan") {
t.Fatalf("expected first-attempt reasoning to be preserved, got %#v", message)
}
}
func TestChatCompletionsContentFilterDoesNotRetry(t *testing.T) {
@@ -499,7 +496,7 @@ func TestResponsesStreamRetriesThinkingOnlyOutput(t *testing.T) {
func TestResponsesNonStreamRetriesThinkingOnlyOutput(t *testing.T) {
ds := &streamStatusDSSeqStub{resps: []*http.Response{
makeOpenAISSEHTTPResponse(`data: {"response_message_id":88,"p":"response/thinking_content","v":"plan"}`, "data: [DONE]"),
makeOpenAISSEHTTPResponse(`data: {"response_message_id":88}`, "data: [DONE]"),
makeOpenAISSEHTTPResponse(`data: {"p":"response/content","v":"visible"}`, "data: [DONE]"),
}}
h := &openAITestSurface{
@@ -540,9 +537,9 @@ func TestResponsesNonStreamRetriesThinkingOnlyOutput(t *testing.T) {
if len(content) == 0 {
t.Fatalf("expected content entries, got %#v", item)
}
reasoning, _ := content[0].(map[string]any)
if asString(reasoning["type"]) != "reasoning" || !strings.Contains(asString(reasoning["text"]), "plan") {
t.Fatalf("expected preserved reasoning entry, got %#v", content)
textEntry, _ := content[0].(map[string]any)
if asString(textEntry["type"]) != "output_text" || asString(textEntry["text"]) != "visible" {
t.Fatalf("expected visible text entry, got %#v", content)
}
}

Some files were not shown because too many files have changed in this diff Show More