lisone/ds2api
1
0
Fork 0
mirror of https://github.com/CJackHwang/ds2api.git synced 2026-05-01 23:15:27 +08:00
Code Issues Packages Projects Releases Wiki Activity
834 Commits 2 Branches 58 Tags
main
Commit Graph

834 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
CJACK.
445c95a4f2 Merge pull request #379 from CJackHwang/dev 
Merge pull request #377 from CJackHwang/codex/run-all-tests-and-fix-failures

Fix failing current-input token accounting test
v4.2.1 		2026-05-01 16:12:17 +08:00
CJACK
0a6ef8e3f2 fix: remove bufio.Scanner 2MiB line limit for SSE; support quasi_status direct patch 
Replace bufio.Scanner with bufio.NewReaderSize + ReadBytes('\n') across all
SSE read paths to preserve long single-line data (e.g. write_file content).
Add quasi_status and auto_continue handling as direct path-based patches in
both Go continue observer and Node vercel_stream_impl, mirroring existing
batch-patch logic. Add 2MiB+ line throughput tests at every SSE layer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 15:45:17 +08:00
CJACK
fd0ec29991 refactor: generalize DSML tag parsing to tolerate model noise; split tiktoken by build tags 
Replace hardcoded DSML typo variant lists in Go/Node tool call parsers with
generalized prefix consumption that tolerates repeated leading <, repeated DSML
prefix noise, and trailing pipe terminators. Split tiktoken-dependent token
counting into a build-tagged file for non-cgo platform compatibility. Add /data
directory to Dockerfile for bind-mount permissions.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 15:17:11 +08:00
CJACK
2671298439 fix: coalesce small stream deltas to prevent character swallowing; add read-tool cache guard 
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 13:53:27 +08:00
CJACK
92e321fe2c 修复吞字问题 2026-05-01 01:31:48 +08:00
CJACK.
fca8c01397 Merge pull request #385 from ouqiting/fix_chat_history 
fix: content being overwritten and left empty
 2026-05-01 00:21:33 +08:00
ouqiting
667e1e3710 fix: content being overwritten and left empty 2026-04-30 21:36:47 +08:00
CJACK.
4438d03c5c Merge pull request #377 from CJackHwang/codex/run-all-tests-and-fix-failures 
Fix failing current-input token accounting test
 2026-04-30 02:41:08 +08:00
CJACK.
95b7665643 Merge branch 'dev' into codex/run-all-tests-and-fix-failures 2026-04-30 02:39:18 +08:00
CJACK.
9896b1fc33 Merge pull request #378 from NgoQuocViet2001/ai/openai-root-route-aliases 
feat(openai): add root route aliases
 2026-04-30 02:33:38 +08:00
CJACK.
966f21211d Fix nil-session guard in chat history test 2026-04-30 02:31:06 +08:00
NgoQuocViet2001
7dc3af40b2 feat(openai): add root route aliases 2026-04-30 01:24:53 +07:00
CJACK.
2f6b5ffda0 Fix current-input token text test expectation 2026-04-30 02:22:17 +08:00
CJACK.
85e256ad4d Merge pull request #375 from CJackHwang/codex/investigate-data-loss-issue-in-pr-369 
sse/parser: treat object-shaped `v` as visible content, preserve INCOMPLETE across omitted status; add tests and samples
 2026-04-30 02:14:26 +08:00
CJACK.
7c3ff6ee7e Merge pull request #374 from shern-point/feat/full-context-file-token-accounting 
Feat/full context file token accounting
 2026-04-30 02:12:55 +08:00
CJACK.
63e62fd1b0 Merge pull request #372 from shern-point/feat/accurate-context-token-length 
Feat/accurate context token length
 2026-04-30 02:11:32 +08:00
CJACK.
483d7af3d2 Merge pull request #373 from NgoQuocViet2001/ai/ds2api-small-regression-fix 
fix(openai): return 400 for inline file limit
 2026-04-30 02:08:22 +08:00
CJACK.
0f89823526 chore(sse): bump client version and refresh longtext stream fixtures 2026-04-30 02:05:45 +08:00
shern-point
6a778e0d35 feat: include inline-uploaded file tokens in context token accounting 
Track byte sizes of inline-uploaded files during PreprocessInlineFileInputs and convert them to conservative token estimates (bytes/3). RefFileTokens is threaded through StandardRequest into all OpenAI chat/responses usage builders so returned prompt_tokens/input_tokens reflect the full upstream context cost including attached files.
 2026-04-30 01:42:51 +08:00
NgoQuocViet2001
9035c350a7 fix(openai): return 400 for inline file limit 2026-04-30 00:35:59 +07:00
shern-point
ba80052a26 fix: count uploaded file content in context token accounting 
PromptTokenText now reflects the actual downstream context cost: the uploaded IGNORE.txt file content plus the neutral live prompt, instead of only the pre-split prompt text.
 2026-04-30 01:12:35 +08:00
shern-point
78fdd63470 feat: add full-context token regression coverage and docs 
Lock in the current_input_file regression with API-level tests and document that returned context token counts now track full prompt semantics with conservative sizing.
 2026-04-30 00:46:06 +08:00
shern-point
4b4f097006 feat: use model-aware prompt counting in Gemini paths 
Preserve Gemini prompt token text during normalization and remove the hardcoded DeepSeek model from native Gemini usage helpers.
 2026-04-30 00:46:05 +08:00
shern-point
d3018c281b feat: use tokenizer-based counting in Claude token paths 
Unify Claude count_tokens, legacy stream accounting, and legacy render usage with preserved prompt text so Claude stops falling back to lossy message formatting.
 2026-04-30 00:46:04 +08:00
shern-point
415a2359ad feat: route OpenAI responses usage through preserved prompt text 
Use the stored full-context prompt text for responses accounting so neutral placeholder prompts do not underreport returned input token counts.
 2026-04-30 00:45:31 +08:00
shern-point
f702d45a24 feat: route OpenAI chat usage through preserved prompt text 
Use the stored full-context prompt text for chat non-stream, stream, and retry accounting so current_input_file no longer shrinks returned prompt token counts.
 2026-04-30 00:45:30 +08:00
shern-point
90817cb9e2 feat: apply tokenizer-based counting in OpenAI usage builders 
Move OpenAI chat and responses usage accounting onto the shared tokenizer-aware counters so prompt and output usage stay model-aware and conservatively sized.
 2026-04-30 00:45:29 +08:00
shern-point
b96f736bd2 feat: preserve full prompt text across current_input_file rewrites 
Keep token accounting tied to the original prompt even after the live prompt is replaced with a neutral placeholder and hidden context file.
 2026-04-30 00:45:01 +08:00
shern-point
8ab028c52a feat: seed PromptTokenText during request normalization 
Capture the fully built prompt at normalization time for OpenAI and Gemini-compatible requests so usage paths can reuse the original context text.
 2026-04-30 00:44:59 +08:00
shern-point
78366afec5 feat: add PromptTokenText to StandardRequest 
Track a dedicated prompt string for token accounting so later prompt rewrites can keep returning full-context counts.
 2026-04-30 00:44:57 +08:00
shern-point
bd41c8a90c feat: add tokenizer-based token counting utilities 
Use go-tiktoken with embedded vocabularies for accurate BPE token counting. CountPromptTokens applies conservative padding so returned context token counts stay slightly above the real value instead of undercounting.
 2026-04-30 00:44:11 +08:00
CJACK.
bc2a78ae29 Merge pull request #370 from CJackHwang/codex/align-vercel-behavior-with-go 
fix(vercel): align JS stream parser with Go object-shaped content
 2026-04-30 00:12:32 +08:00
CJACK.
192cdf8562 fix(vercel): align JS stream parser with Go object-shaped content 2026-04-29 23:56:16 +08:00
CJACK.
94c1acace5 Merge pull request #369 from CJackHwang/dev 
Merge pull request #368 from CJackHwang/codex/fix-review-issues-for-pr-#364

Restore thinking fallback for tool-call detection and drop history.txt wrapper tags
 2026-04-29 23:42:50 +08:00
CJACK.
273c18ba0f fix: fallback to /app config when /data is unavailable 2026-04-29 23:40:07 +08:00
CJACK.
ae28e33184 fix: preserve continue state when chunk status is missing 2026-04-29 23:25:18 +08:00
CJACK.
0438ce9a12 Merge pull request #368 from CJackHwang/codex/fix-review-issues-for-pr-#364 
Restore thinking fallback for tool-call detection and drop history.txt wrapper tags
 2026-04-29 23:07:36 +08:00
CJACK.
af4a067dab Merge pull request #362 from CJackHwang/codex/fix-issue-based-on-feedback 
fix(sse): batch tiny stream chunks before emitting
 2026-04-29 23:07:03 +08:00
CJACK.
33f6fef015 Fix tool-call fallback on sanitized empty text and remove history wrapper tags 2026-04-29 23:04:45 +08:00
CJACK.
6d3979a1d6 fix(sse): stop scanner sender when stream context cancels 2026-04-29 22:59:22 +08:00
CJACK.
c8922c7a88 Merge pull request #364 from adnxx1wsx/dev 
Fix stream compatibility and vision model exposure
 2026-04-29 22:02:19 +08:00
MiY
241334c658 Fix stream compatibility and vision model exposure 2026-04-29 20:23:13 +08:00
CJACK.
d7e071b24a Bump version from 4.1.3 to 4.2.0 2026-04-29 19:08:57 +08:00
CJACK.
89225c778e fix(sse): batch tiny stream chunks before emitting 2026-04-29 18:58:54 +08:00
CJACK.
22160de2c4 Merge pull request #359 from NgoQuocViet2001/ai/ds2api-small-fix 
fix(openai): keep citation indexes one-based with zero-based references
 2026-04-29 18:27:15 +08:00
NgoQuocViet2001
0cbc2c875d fix(openai): keep citation indexes one-based 2026-04-29 15:43:09 +07:00
CJACK.
a0984ef682 Merge pull request #358 from CJackHwang/revert-356-codex/check-version-update-in-automation-scripts 
Revert "Verify GHCR latest tag matches release and show version source/latest in dashboard"
v4.1.3_beta2 		2026-04-29 14:49:41 +08:00
CJACK.
babfa973d6 Revert "Verify GHCR latest tag matches release and show version source/latest in dashboard" 2026-04-29 14:47:53 +08:00
CJACK.
ba4071d8b5 Merge pull request #357 from CJackHwang/codex/update-documentation-for-config.json-permissions 
Return config persistence warning when config path is read-only; default container config to /data/config.json and update docs
 2026-04-29 14:18:25 +08:00
CJACK.
e1f8e493d2 fix: add legacy /app/config.json fallback for container upgrades 2026-04-29 14:12:20 +08:00