lisone/ds2api
1
0
Fork 0
mirror of https://github.com/CJackHwang/ds2api.git synced 2026-05-20 16:07:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

perf(streaming): optimize TTFT and reduce buffering latency

Browse Source 
Core changes:
- stream.go: New accumulation buffer architecture with scanner goroutine
  + select loop, MinChars=16, MaxWait=10ms, first-flush-immediate
- dedupe.go: Add TrimContinuationOverlapFromBuilder to avoid string copies
- claude/stream_runtime_core.go: Integrate toolstream for incremental text
- claude/stream_runtime_finalize.go: toolstream flush support
- stream_emitter.js: Reduce DeltaCoalescer thresholds (160->16 chars, 80->20ms)
- empty_retry: Add thinking-aware empty output detection
- Fix reasoning_content leak and finish_reason=null in edge cases
- Fix tail content truncation when max_tokens exceeded

Tests: sync test expectations with upstream for thinking content
This commit is contained in:
王
2026-05-02 20:28:30 +08:00
parent 20d71f528a
commit d407ccb773
18 changed files with 667 additions and 291 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
internal/httpapi/openai/chat/handler.go
View File

@@ -108,8 +108,8 @@ func replaceCitationMarkersWithLinks(text string, links map[int]string) string {
return shared.ReplaceCitationMarkersWithLinks(text, links)
}
func shouldWriteUpstreamEmptyOutputError(text string) bool {
return shared.ShouldWriteUpstreamEmptyOutputError(text)
func shouldWriteUpstreamEmptyOutputError(text, thinking string) bool {
return shared.ShouldWriteUpstreamEmptyOutputError(text, thinking)
}
func upstreamEmptyOutputDetail(contentFilter bool, text, thinking string) (int, string, string) {