Compare commits

...

27 Commits

Author SHA1 Message Date
CJACK.
063599678a Merge pull request #174 from CJackHwang/dev
Merge pull request #171 from CJackHwang/codex/fix-issue-#170-in-ds2api

Enable env-backed config writeback and bootstrap missing config file
2026-03-31 01:36:31 +08:00
CJACK.
f55aa7564a Merge pull request #176 from CJackHwang/codex/fix-fallback-to-file-config-on-json-parse-failure-u32uz8
修复写回模式回退逻辑并从 CONTENT_FILTER 起截断流输出
2026-03-31 01:35:42 +08:00
CJACK.
3b60e3c8f9 fix(sse): trim stream output from CONTENT_FILTER onward 2026-03-31 01:26:43 +08:00
CJACK.
efebe9ebad Merge pull request #171 from CJackHwang/codex/fix-issue-#170-in-ds2api
Enable env-backed config writeback and bootstrap missing config file
2026-03-31 00:01:27 +08:00
CJACK.
b54b418f96 fix(sse): globally strip leaked CONTENT_FILTER suffix from output 2026-03-30 23:39:47 +08:00
CJACK.
1c5f022b06 refactor(config): split writeback helpers out of store.go for CI gate 2026-03-30 21:38:19 +08:00
CJACK.
836eaf5290 feat(ui): show env mode persistence status and document writeback 2026-03-30 21:18:56 +08:00
CJACK.
958e7a0d04 fix(config): skip writeback bootstrap on invalid env config 2026-03-30 21:02:36 +08:00
CJACK.
f3555ae9b0 feat(config): bootstrap config.json when env writeback is enabled 2026-03-30 20:37:58 +08:00
CJACK.
d50d39e2e5 Merge pull request #169 from CJackHwang/dev
Merge pull request #168 from CJackHwang/codex/fix-vercel-deployment-issue-with-api-calls

fix(js): avoid false tool-call capture on plain tool_calls prose
2026-03-30 16:08:39 +08:00
CJACK.
01393837be Merge pull request #168 from CJackHwang/codex/fix-vercel-deployment-issue-with-api-calls
fix(js): avoid false tool-call capture on plain tool_calls prose
2026-03-30 16:01:51 +08:00
CJACK.
1fe1240240 fix(js): prevent XML wrapper attribute tool_calls scan loop 2026-03-30 15:59:34 +08:00
CJACK.
c07736fbea chore: set shared tool-sieve context tail window to 2048 2026-03-30 15:41:38 +08:00
CJACK.
775bf3b578 refactor(js): align tool-sieve segment start and tail window with go 2026-03-30 15:41:26 +08:00
CJACK.
ab3943ebeb test(js): cover numbered planning prose around tool calls 2026-03-30 15:39:09 +08:00
CJACK.
6efba7b2e4 fix(js): avoid false tool-call capture on plain tool_calls prose 2026-03-30 12:51:33 +08:00
CJACK.
765d0231cd Merge pull request #166 from CJackHwang/dev
chore: relocate sha3 WASM asset to internal directory and update build configurations
2026-03-30 12:23:46 +08:00
CJACK.
aebf3e9119 Merge pull request #167 from CJackHwang/codex/remove-dangling-agent-xml-tags
Fix dangling agent XML cleanup and XML-escape tool-call prompt serialization
2026-03-30 12:22:58 +08:00
CJACK.
535d9298a7 Scope dangling result-tag cleanup to leaked wrapper fragments 2026-03-30 12:22:04 +08:00
CJACK.
b790545d82 Fix dangling agent XML cleanup and escape tool call prompt XML 2026-03-30 11:23:16 +08:00
CJACK.
034c00f10e Merge pull request #163 from CJackHwang/dev
docs: update API documentation, deployment guides, and README with new admin endpoints, compatibility notes, and build instructions
2026-03-29 19:50:40 +08:00
CJACK.
390f7580e5 Merge pull request #156 from CJackHwang/dev
Merge pull request #153 from CJackHwang/codex/investigate-tool-execution-bugs-in-output-7ocr8f

Relax tool-name allow-listing and improve tool-call detection/parsing across adapters and sieve
2026-03-22 21:40:03 +08:00
CJACK.
586d31e556 Merge pull request #151 from CJackHwang/dev
Merge pull request #149 from CJackHwang/codex/fix-tool-miscall-during-complex-json-test

Ignore tool_call payloads inside fenced code blocks and chat envelopes; stream-aware code-fence tracking
2026-03-22 16:51:17 +08:00
CJACK.
c4a73e871a Merge pull request #148 from CJackHwang/dev
Merge pull request #147 from CJackHwang/codex/fix-tool-call-history-retrieval

Preserve tool call/result roundtrip and raw payloads across Claude, Gemini and OpenAI adapters
2026-03-22 13:43:26 +08:00
CJACK.
25b3292497 Merge pull request #146 from CJackHwang/dev
Merge pull request #145 from CJackHwang/codex/determine-which-pr-fixes-json-leak-issue

Merge pull request #144 from CJackHwang/codex/refactor-codebase-to-remove-redundancy

Refactor tool-sieve and response streaming, remove unused helpers and UI wrappers
2026-03-22 11:05:54 +08:00
CJACK.
11f66db87d Merge pull request #142 from CJackHwang/dev
Merge pull request #141 from CJackHwang/codex/investigate-json-leakage-in-vercel-deployment-rh84s1

Fix raw tool-call JSON leaks when feature_match mode is off
2026-03-22 08:55:29 +08:00
CJACK.
7131b06e26 Merge pull request #138 from CJackHwang/dev
Merge pull request #135 from CJackHwang/codex/add-global-token-refresh-logic

Sanitize leaked tool-history markers, simplify normalization, and add managed token refresh
2026-03-22 01:27:27 +08:00
25 changed files with 467 additions and 25 deletions

View File

@@ -587,6 +587,9 @@ Returns sanitized config.
{ {
"keys": ["k1", "k2"], "keys": ["k1", "k2"],
"env_backed": false, "env_backed": false,
"env_source_present": true,
"env_writeback_enabled": true,
"config_path": "/data/config.json",
"accounts": [ "accounts": [
{ {
"identifier": "user@example.com", "identifier": "user@example.com",

3
API.md
View File

@@ -596,6 +596,9 @@ data: {"type":"message_stop"}
{ {
"keys": ["k1", "k2"], "keys": ["k1", "k2"],
"env_backed": false, "env_backed": false,
"env_source_present": true,
"env_writeback_enabled": true,
"config_path": "/data/config.json",
"accounts": [ "accounts": [
{ {
"identifier": "user@example.com", "identifier": "user@example.com",

View File

@@ -320,6 +320,7 @@ cp opencode.json.example opencode.json
| `DS2API_CONFIG_PATH` | 配置文件路径 | `config.json` | | `DS2API_CONFIG_PATH` | 配置文件路径 | `config.json` |
| `DS2API_CONFIG_JSON` | 直接注入配置JSON 或 Base64 | — | | `DS2API_CONFIG_JSON` | 直接注入配置JSON 或 Base64 | — |
| `CONFIG_JSON` | 旧版兼容配置注入 | — | | `CONFIG_JSON` | 旧版兼容配置注入 | — |
| `DS2API_ENV_WRITEBACK` | 环境变量模式下自动写回配置文件并切换文件模式(`1/true/yes/on` | 关闭 |
| `DS2API_WASM_PATH` | PoW WASM 文件路径 | 自动查找 | | `DS2API_WASM_PATH` | PoW WASM 文件路径 | 自动查找 |
| `DS2API_STATIC_ADMIN_DIR` | 管理台静态文件目录 | `static/admin` | | `DS2API_STATIC_ADMIN_DIR` | 管理台静态文件目录 | `static/admin` |
| `DS2API_AUTO_BUILD_WEBUI` | 启动时自动构建 WebUI | 本地开启Vercel 关闭 | | `DS2API_AUTO_BUILD_WEBUI` | 启动时自动构建 WebUI | 本地开启Vercel 关闭 |
@@ -342,6 +343,8 @@ cp opencode.json.example opencode.json
| `VERCEL_TEAM_ID` | Vercel 团队 ID | — | | `VERCEL_TEAM_ID` | Vercel 团队 ID | — |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel 部署保护绕过密钥(内部 Node→Go 调用) | — | | `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel 部署保护绕过密钥(内部 Node→Go 调用) | — |
> 提示:当检测到 `DS2API_CONFIG_JSON/CONFIG_JSON` 时,管理台会显示当前模式风险与自动持久化状态(含 `DS2API_CONFIG_PATH` 路径与模式切换说明)。
## 鉴权模式 ## 鉴权模式
调用业务接口(`/v1/*`、`/anthropic/*`、Gemini 路由)时支持两种模式: 调用业务接口(`/v1/*`、`/anthropic/*`、Gemini 路由)时支持两种模式:

View File

@@ -320,6 +320,7 @@ cp opencode.json.example opencode.json
| `DS2API_CONFIG_PATH` | Config file path | `config.json` | | `DS2API_CONFIG_PATH` | Config file path | `config.json` |
| `DS2API_CONFIG_JSON` | Inline config (JSON or Base64) | — | | `DS2API_CONFIG_JSON` | Inline config (JSON or Base64) | — |
| `CONFIG_JSON` | Legacy compatibility config input | — | | `CONFIG_JSON` | Legacy compatibility config input | — |
| `DS2API_ENV_WRITEBACK` | Auto-write env-backed config to file and transition to file mode (`1/true/yes/on`) | Disabled |
| `DS2API_WASM_PATH` | PoW WASM file path | Auto-detect | | `DS2API_WASM_PATH` | PoW WASM file path | Auto-detect |
| `DS2API_STATIC_ADMIN_DIR` | Admin static assets dir | `static/admin` | | `DS2API_STATIC_ADMIN_DIR` | Admin static assets dir | `static/admin` |
| `DS2API_AUTO_BUILD_WEBUI` | Auto-build WebUI on startup | Enabled locally, disabled on Vercel | | `DS2API_AUTO_BUILD_WEBUI` | Auto-build WebUI on startup | Enabled locally, disabled on Vercel |
@@ -339,6 +340,8 @@ cp opencode.json.example opencode.json
| `VERCEL_TEAM_ID` | Vercel team ID | — | | `VERCEL_TEAM_ID` | Vercel team ID | — |
| `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel deployment protection bypass for internal Node→Go calls | — | | `DS2API_VERCEL_PROTECTION_BYPASS` | Vercel deployment protection bypass for internal Node→Go calls | — |
> Note: when `DS2API_CONFIG_JSON/CONFIG_JSON` is detected, the Admin UI shows mode risk and auto-persistence status (including `DS2API_CONFIG_PATH` and mode-transition hints).
## Authentication Modes ## Authentication Modes
For business endpoints (`/v1/*`, `/anthropic/*`, Gemini routes), DS2API supports two modes: For business endpoints (`/v1/*`, `/anthropic/*`, Gemini routes), DS2API supports two modes:

View File

@@ -248,6 +248,7 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # optional for personal accounts
| `DS2API_ACCOUNT_QUEUE_SIZE` | Alias (legacy compat) | — | | `DS2API_ACCOUNT_QUEUE_SIZE` | Alias (legacy compat) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | Global inflight limit | `recommended_concurrency` | | `DS2API_GLOBAL_MAX_INFLIGHT` | Global inflight limit | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | Alias (legacy compat) | — | | `DS2API_MAX_INFLIGHT` | Alias (legacy compat) | — |
| `DS2API_ENV_WRITEBACK` | When `DS2API_CONFIG_JSON` is present, auto-write to `DS2API_CONFIG_PATH` and switch to file-backed mode after success (`1/true/yes/on`) | Disabled |
| `DS2API_VERCEL_INTERNAL_SECRET` | Hybrid streaming internal auth | Falls back to `DS2API_ADMIN_KEY` | | `DS2API_VERCEL_INTERNAL_SECRET` | Hybrid streaming internal auth | Falls back to `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | Stream lease TTL | `900` | | `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | Stream lease TTL | `900` |
| `VERCEL_TOKEN` | Vercel sync token | — | | `VERCEL_TOKEN` | Vercel sync token | — |

View File

@@ -248,6 +248,7 @@ VERCEL_TEAM_ID=team_xxxxxxxxxxxx # 个人账号可留空
| `DS2API_ACCOUNT_QUEUE_SIZE` | 同上(兼容别名) | — | | `DS2API_ACCOUNT_QUEUE_SIZE` | 同上(兼容别名) | — |
| `DS2API_GLOBAL_MAX_INFLIGHT` | 全局并发上限 | `recommended_concurrency` | | `DS2API_GLOBAL_MAX_INFLIGHT` | 全局并发上限 | `recommended_concurrency` |
| `DS2API_MAX_INFLIGHT` | 同上(兼容别名) | — | | `DS2API_MAX_INFLIGHT` | 同上(兼容别名) | — |
| `DS2API_ENV_WRITEBACK` | 检测到 `DS2API_CONFIG_JSON` 时自动写入 `DS2API_CONFIG_PATH`,并在成功后转为文件模式(`1/true/yes/on` | 关闭 |
| `DS2API_VERCEL_INTERNAL_SECRET` | 混合流式内部鉴权 | 回退用 `DS2API_ADMIN_KEY` | | `DS2API_VERCEL_INTERNAL_SECRET` | 混合流式内部鉴权 | 回退用 `DS2API_ADMIN_KEY` |
| `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | 流式 lease TTL | `900` | | `DS2API_VERCEL_STREAM_LEASE_TTL_SECONDS` | 流式 lease TTL | `900` |
| `VERCEL_TOKEN` | Vercel 同步 token | — | | `VERCEL_TOKEN` | Vercel 同步 token | — |

View File

@@ -23,6 +23,9 @@ var leakedAgentXMLBlockPatterns = []*regexp.Regexp{
regexp.MustCompile(`(?is)<new_task\b[^>]*>(.*?)</new_task>`), regexp.MustCompile(`(?is)<new_task\b[^>]*>(.*?)</new_task>`),
} }
var leakedAgentWrapperTagPattern = regexp.MustCompile(`(?is)</?(?:attempt_completion|ask_followup_question|new_task)\b[^>]*>`)
var leakedAgentWrapperPlusResultOpenPattern = regexp.MustCompile(`(?is)<(?:attempt_completion|ask_followup_question|new_task)\b[^>]*>\s*<result>`)
var leakedAgentResultPlusWrapperClosePattern = regexp.MustCompile(`(?is)</result>\s*</(?:attempt_completion|ask_followup_question|new_task)\b[^>]*>`)
var leakedAgentResultTagPattern = regexp.MustCompile(`(?is)</?result>`) var leakedAgentResultTagPattern = regexp.MustCompile(`(?is)</?result>`)
func sanitizeLeakedOutput(text string) string { func sanitizeLeakedOutput(text string) string {
@@ -50,5 +53,18 @@ func sanitizeLeakedAgentXMLBlocks(text string) string {
return leakedAgentResultTagPattern.ReplaceAllString(submatches[1], "") return leakedAgentResultTagPattern.ReplaceAllString(submatches[1], "")
}) })
} }
// Fallback for truncated output streams: strip any dangling wrapper tags
// that were not part of a complete block replacement. If we detect leaked
// wrapper tags, strip only adjacent <result> tags to avoid exposing agent
// markup without altering unrelated user-visible <result> examples.
if leakedAgentWrapperTagPattern.MatchString(out) {
out = leakedAgentWrapperPlusResultOpenPattern.ReplaceAllStringFunc(out, func(match string) string {
return leakedAgentResultTagPattern.ReplaceAllString(match, "")
})
out = leakedAgentResultPlusWrapperClosePattern.ReplaceAllStringFunc(out, func(match string) string {
return leakedAgentResultTagPattern.ReplaceAllString(match, "")
})
out = leakedAgentWrapperTagPattern.ReplaceAllString(out, "")
}
return out return out
} }

View File

@@ -41,3 +41,28 @@ func TestSanitizeLeakedOutputPreservesStandaloneResultTags(t *testing.T) {
t.Fatalf("unexpected sanitize result for standalone result tag: %q", got) t.Fatalf("unexpected sanitize result for standalone result tag: %q", got)
} }
} }
func TestSanitizeLeakedOutputRemovesDanglingAgentXMLOpeningTags(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer"
got := sanitizeLeakedOutput(raw)
if got != "Done.Some final answer" {
t.Fatalf("unexpected sanitize result for dangling opening tags: %q", got)
}
}
func TestSanitizeLeakedOutputRemovesDanglingAgentXMLClosingTags(t *testing.T) {
raw := "Done.Some final answer</result></attempt_completion>"
got := sanitizeLeakedOutput(raw)
if got != "Done.Some final answer" {
t.Fatalf("unexpected sanitize result for dangling closing tags: %q", got)
}
}
func TestSanitizeLeakedOutputPreservesUnrelatedResultTagsWhenWrapperLeaks(t *testing.T) {
raw := "Done.<attempt_completion><result>Some final answer\nExample XML: <result>value</result>"
got := sanitizeLeakedOutput(raw)
want := "Done.Some final answer\nExample XML: <result>value</result>"
if got != want {
t.Fatalf("unexpected sanitize result for mixed leaked wrapper + xml example: %q", got)
}
}

View File

@@ -34,7 +34,8 @@ type toolCallDelta struct {
Arguments string Arguments string
} }
const toolSieveContextTailLimit = 256 // Keep in sync with JS TOOL_SIEVE_CONTEXT_TAIL_LIMIT.
const toolSieveContextTailLimit = 2048
func (s *toolStreamSieveState) resetIncrementalToolState() { func (s *toolStreamSieveState) resetIncrementalToolState() {
s.disableDeltas = false s.disableDeltas = false

View File

@@ -21,6 +21,9 @@ type ConfigStore interface {
Update(mutator func(*config.Config) error) error Update(mutator func(*config.Config) error) error
ExportJSONAndBase64() (string, string, error) ExportJSONAndBase64() (string, string, error)
IsEnvBacked() bool IsEnvBacked() bool
IsEnvWritebackEnabled() bool
HasEnvConfigSource() bool
ConfigPath() string
SetVercelSync(hash string, ts int64) error SetVercelSync(hash string, ts int64) error
AdminPasswordHash() string AdminPasswordHash() string
AdminJWTExpireHours() int AdminJWTExpireHours() int

View File

@@ -8,9 +8,12 @@ import (
func (h *Handler) getConfig(w http.ResponseWriter, _ *http.Request) { func (h *Handler) getConfig(w http.ResponseWriter, _ *http.Request) {
snap := h.Store.Snapshot() snap := h.Store.Snapshot()
safe := map[string]any{ safe := map[string]any{
"keys": snap.Keys, "keys": snap.Keys,
"accounts": []map[string]any{}, "accounts": []map[string]any{},
"env_backed": h.Store.IsEnvBacked(), "env_backed": h.Store.IsEnvBacked(),
"env_source_present": h.Store.HasEnvConfigSource(),
"env_writeback_enabled": h.Store.IsEnvWritebackEnabled(),
"config_path": h.Store.ConfigPath(),
"claude_mapping": func() map[string]string { "claude_mapping": func() map[string]string {
if len(snap.ClaudeMapping) > 0 { if len(snap.ClaudeMapping) > 0 {
return snap.ClaudeMapping return snap.ClaudeMapping

View File

@@ -2,6 +2,7 @@ package config
import ( import (
"encoding/base64" "encoding/base64"
"errors"
"os" "os"
"strings" "strings"
"testing" "testing"
@@ -79,6 +80,111 @@ func TestLoadStorePreservesFileBackedTokensForRuntime(t *testing.T) {
} }
} }
func TestEnvBackedStoreWritebackBootstrapsMissingConfigFile(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {
t.Fatalf("create temp config: %v", err)
}
path := tmp.Name()
_ = tmp.Close()
_ = os.Remove(path)
t.Setenv("DS2API_CONFIG_JSON", `{"keys":["k1"],"accounts":[{"email":"seed@example.com","password":"p"}]}`)
t.Setenv("CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
t.Setenv("DS2API_ENV_WRITEBACK", "1")
store := LoadStore()
if store.IsEnvBacked() {
t.Fatalf("expected writeback bootstrap to become file-backed immediately")
}
if err := store.Update(func(c *Config) error {
c.Accounts = append(c.Accounts, Account{Email: "new@example.com", Password: "p2"})
return nil
}); err != nil {
t.Fatalf("update failed: %v", err)
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read written config: %v", err)
}
if !strings.Contains(string(content), "seed@example.com") {
t.Fatalf("expected bootstrapped config to contain seed account, got: %s", content)
}
if !strings.Contains(string(content), "new@example.com") {
t.Fatalf("expected persisted config to contain added account, got: %s", content)
}
reloaded := LoadStore()
if reloaded.IsEnvBacked() {
t.Fatalf("expected reloaded store to prefer persisted config file")
}
accounts := reloaded.Accounts()
if len(accounts) != 2 {
t.Fatalf("expected 2 accounts after reload, got %d", len(accounts))
}
}
func TestEnvBackedStoreWritebackDoesNotBootstrapOnInvalidEnvJSON(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {
t.Fatalf("create temp config: %v", err)
}
path := tmp.Name()
_ = tmp.Close()
_ = os.Remove(path)
t.Setenv("DS2API_CONFIG_JSON", "{invalid-json")
t.Setenv("CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
t.Setenv("DS2API_ENV_WRITEBACK", "1")
cfg, fromEnv, loadErr := loadConfig()
if loadErr == nil {
t.Fatalf("expected loadConfig error for invalid env json")
}
if !fromEnv {
t.Fatalf("expected fromEnv=true when parsing env config fails")
}
if len(cfg.Keys) != 0 || len(cfg.Accounts) != 0 {
t.Fatalf("expected empty config on parse failure, got keys=%d accounts=%d", len(cfg.Keys), len(cfg.Accounts))
}
if _, statErr := os.Stat(path); !errors.Is(statErr, os.ErrNotExist) {
t.Fatalf("expected no bootstrapped config file, stat err=%v", statErr)
}
}
func TestEnvBackedStoreWritebackFallsBackToPersistedFileOnInvalidEnvJSON(t *testing.T) {
tmp, err := os.CreateTemp(t.TempDir(), "config-*.json")
if err != nil {
t.Fatalf("create temp config: %v", err)
}
path := tmp.Name()
if _, err := tmp.WriteString(`{"keys":["file-key"],"accounts":[{"email":"persisted@example.com","password":"p"}]}`); err != nil {
t.Fatalf("write temp config: %v", err)
}
_ = tmp.Close()
t.Setenv("DS2API_CONFIG_JSON", "{invalid-json")
t.Setenv("CONFIG_JSON", "")
t.Setenv("DS2API_CONFIG_PATH", path)
t.Setenv("DS2API_ENV_WRITEBACK", "1")
cfg, fromEnv, loadErr := loadConfig()
if loadErr != nil {
t.Fatalf("expected fallback to persisted file, got error: %v", loadErr)
}
if fromEnv {
t.Fatalf("expected fallback to file-backed mode")
}
if len(cfg.Keys) != 1 || cfg.Keys[0] != "file-key" {
t.Fatalf("unexpected keys after fallback: %#v", cfg.Keys)
}
if len(cfg.Accounts) != 1 || cfg.Accounts[0].Email != "persisted@example.com" {
t.Fatalf("unexpected accounts after fallback: %#v", cfg.Accounts)
}
}
func TestRuntimeTokenRefreshIntervalHoursDefaultsToSix(t *testing.T) { func TestRuntimeTokenRefreshIntervalHoursDefaultsToSix(t *testing.T) {
t.Setenv("DS2API_CONFIG_JSON", `{ t.Setenv("DS2API_CONFIG_JSON", `{
"keys":["k1"], "keys":["k1"],

View File

@@ -40,12 +40,38 @@ func loadConfig() (Config, bool, error) {
} }
if rawCfg != "" { if rawCfg != "" {
cfg, err := parseConfigString(rawCfg) cfg, err := parseConfigString(rawCfg)
if err != nil {
if !IsVercel() && envWritebackEnabled() {
if fileCfg, fileErr := loadConfigFromFile(ConfigPath()); fileErr == nil {
return fileCfg, false, nil
}
}
return cfg, true, err
}
cfg.ClearAccountTokens() cfg.ClearAccountTokens()
cfg.DropInvalidAccounts() cfg.DropInvalidAccounts()
if IsVercel() || !envWritebackEnabled() {
return cfg, true, err
}
content, fileErr := os.ReadFile(ConfigPath())
if fileErr == nil {
var fileCfg Config
if unmarshalErr := json.Unmarshal(content, &fileCfg); unmarshalErr == nil {
fileCfg.DropInvalidAccounts()
return fileCfg, false, err
}
}
if errors.Is(fileErr, os.ErrNotExist) {
if writeErr := writeConfigFile(ConfigPath(), cfg.Clone()); writeErr == nil {
return cfg, false, err
} else {
Logger.Warn("[config] env writeback bootstrap failed", "error", writeErr)
}
}
return cfg, true, err return cfg, true, err
} }
content, err := os.ReadFile(ConfigPath()) cfg, err := loadConfigFromFile(ConfigPath())
if err != nil { if err != nil {
if IsVercel() { if IsVercel() {
// Vercel one-click deploy may start without a writable/present config file. // Vercel one-click deploy may start without a writable/present config file.
@@ -54,16 +80,6 @@ func loadConfig() (Config, bool, error) {
} }
return Config{}, false, err return Config{}, false, err
} }
var cfg Config
if err := json.Unmarshal(content, &cfg); err != nil {
return Config{}, false, err
}
cfg.DropInvalidAccounts()
if strings.Contains(string(content), `"test_status"`) && !IsVercel() {
if b, err := json.MarshalIndent(cfg, "", " "); err == nil {
_ = os.WriteFile(ConfigPath(), b, 0o644)
}
}
if IsVercel() { if IsVercel() {
// Vercel filesystem is ephemeral/read-only for runtime writes; avoid save errors. // Vercel filesystem is ephemeral/read-only for runtime writes; avoid save errors.
return cfg, true, nil return cfg, true, nil
@@ -71,6 +87,24 @@ func loadConfig() (Config, bool, error) {
return cfg, false, nil return cfg, false, nil
} }
func loadConfigFromFile(path string) (Config, error) {
content, err := os.ReadFile(path)
if err != nil {
return Config{}, err
}
var cfg Config
if err := json.Unmarshal(content, &cfg); err != nil {
return Config{}, err
}
cfg.DropInvalidAccounts()
if strings.Contains(string(content), `"test_status"`) && !IsVercel() {
if b, err := json.MarshalIndent(cfg, "", " "); err == nil {
_ = os.WriteFile(path, b, 0o644)
}
}
return cfg, nil
}
func (s *Store) Snapshot() Config { func (s *Store) Snapshot() Config {
s.mu.RLock() s.mu.RLock()
defer s.mu.RUnlock() defer s.mu.RUnlock()
@@ -177,7 +211,7 @@ func (s *Store) Update(mutator func(*Config) error) error {
func (s *Store) Save() error { func (s *Store) Save() error {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
if s.fromEnv { if s.fromEnv && (IsVercel() || !envWritebackEnabled()) {
Logger.Info("[save_config] source from env, skip write") Logger.Info("[save_config] source from env, skip write")
return nil return nil
} }
@@ -187,11 +221,15 @@ func (s *Store) Save() error {
if err != nil { if err != nil {
return err return err
} }
return os.WriteFile(s.path, b, 0o644) if err := writeConfigBytes(s.path, b); err != nil {
return err
}
s.fromEnv = false
return nil
} }
func (s *Store) saveLocked() error { func (s *Store) saveLocked() error {
if s.fromEnv { if s.fromEnv && (IsVercel() || !envWritebackEnabled()) {
Logger.Info("[save_config] source from env, skip write") Logger.Info("[save_config] source from env, skip write")
return nil return nil
} }
@@ -201,7 +239,11 @@ func (s *Store) saveLocked() error {
if err != nil { if err != nil {
return err return err
} }
return os.WriteFile(s.path, b, 0o644) if err := writeConfigBytes(s.path, b); err != nil {
return err
}
s.fromEnv = false
return nil
} }
func (s *Store) IsEnvBacked() bool { func (s *Store) IsEnvBacked() bool {

View File

@@ -0,0 +1,51 @@
package config
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
)
func envWritebackEnabled() bool {
v := strings.ToLower(strings.TrimSpace(os.Getenv("DS2API_ENV_WRITEBACK")))
return v == "1" || v == "true" || v == "yes" || v == "on"
}
func (s *Store) IsEnvWritebackEnabled() bool {
return envWritebackEnabled()
}
func (s *Store) HasEnvConfigSource() bool {
rawCfg := strings.TrimSpace(os.Getenv("DS2API_CONFIG_JSON"))
if rawCfg == "" {
rawCfg = strings.TrimSpace(os.Getenv("CONFIG_JSON"))
}
return rawCfg != ""
}
func (s *Store) ConfigPath() string {
return s.path
}
func writeConfigFile(path string, cfg Config) error {
persistCfg := cfg.Clone()
persistCfg.ClearAccountTokens()
b, err := json.MarshalIndent(persistCfg, "", " ")
if err != nil {
return err
}
return writeConfigBytes(path, b)
}
func writeConfigBytes(path string, b []byte) error {
dir := filepath.Dir(path)
if dir == "." || dir == "" {
return os.WriteFile(path, b, 0o644)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
return fmt.Errorf("mkdir config dir: %w", err)
}
return os.WriteFile(path, b, 0o644)
}

View File

@@ -102,7 +102,10 @@ function extractToolCallObjects(text) {
const obj = extractJSONObjectFrom(raw, start); const obj = extractJSONObjectFrom(raw, start);
if (obj.ok) { if (obj.ok) {
out.push(raw.slice(start, obj.end).trim()); out.push(raw.slice(start, obj.end).trim());
offset = obj.end; // Ensure forward progress even when the matched keyword is outside
// the extracted JSON object (e.g. closing XML wrapper tags containing
// "tool_calls" after an earlier JSON arguments object).
offset = Math.max(obj.end, idx + matched.length);
idx = -1; idx = -1;
break; break;
} }

View File

@@ -1,6 +1,7 @@
'use strict'; 'use strict';
const TOOL_SIEVE_CONTEXT_TAIL_LIMIT = 4096; // Keep in sync with Go toolSieveContextTailLimit.
const TOOL_SIEVE_CONTEXT_TAIL_LIMIT = 2048;
function createToolSieveState() { function createToolSieveState() {
return { return {

View File

@@ -5,6 +5,12 @@ import (
"strings" "strings"
) )
var promptXMLTextEscaper = strings.NewReplacer(
"&", "&amp;",
"<", "&lt;",
">", "&gt;",
)
// FormatToolCallsForPrompt renders a tool_calls slice into the canonical // FormatToolCallsForPrompt renders a tool_calls slice into the canonical
// prompt-visible history block used across adapters. // prompt-visible history block used across adapters.
func FormatToolCallsForPrompt(raw any) string { func FormatToolCallsForPrompt(raw any) string {
@@ -82,8 +88,8 @@ func formatToolCallForPrompt(call map[string]any) string {
} }
return " <tool_call>\n" + return " <tool_call>\n" +
" <tool_name>" + name + "</tool_name>\n" + " <tool_name>" + escapeXMLText(name) + "</tool_name>\n" +
" <parameters>" + StringifyToolCallArguments(argsRaw) + "</parameters>\n" + " <parameters>" + escapeXMLText(StringifyToolCallArguments(argsRaw)) + "</parameters>\n" +
" </tool_call>" " </tool_call>"
} }
@@ -122,3 +128,10 @@ func asString(v any) string {
} }
return "" return ""
} }
func escapeXMLText(v string) string {
if v == "" {
return ""
}
return promptXMLTextEscaper.Replace(v)
}

View File

@@ -26,3 +26,16 @@ func TestFormatToolCallsForPromptXML(t *testing.T) {
t.Fatalf("unexpected formatted tool call XML: %q", got) t.Fatalf("unexpected formatted tool call XML: %q", got)
} }
} }
func TestFormatToolCallsForPromptEscapesXMLEntities(t *testing.T) {
got := FormatToolCallsForPrompt([]any{
map[string]any{
"name": "search<&>",
"arguments": `{"q":"a < b && c > d"}`,
},
})
want := "<tool_calls>\n <tool_call>\n <tool_name>search&lt;&amp;&gt;</tool_name>\n <parameters>{\"q\":\"a &lt; b &amp;&amp; c &gt; d\"}</parameters>\n </tool_call>\n</tool_calls>"
if got != want {
t.Fatalf("unexpected escaped tool call XML: %q", got)
}
}

View File

@@ -0,0 +1,31 @@
package sse
import "strings"
func filterLeakedContentFilterParts(parts []ContentPart) []ContentPart {
if len(parts) == 0 {
return parts
}
out := make([]ContentPart, 0, len(parts))
for _, p := range parts {
cleaned := stripLeakedContentFilterSuffix(p.Text)
if strings.TrimSpace(cleaned) == "" {
continue
}
p.Text = cleaned
out = append(out, p)
}
return out
}
func stripLeakedContentFilterSuffix(text string) string {
if text == "" {
return text
}
upperText := strings.ToUpper(text)
idx := strings.Index(upperText, "CONTENT_FILTER")
if idx < 0 {
return text
}
return strings.TrimRight(text[:idx], " \t\r\n")
}

View File

@@ -40,6 +40,7 @@ func ParseDeepSeekContentLine(raw []byte, thinkingEnabled bool, currentType stri
} }
} }
parts, finished, nextType := ParseSSEChunkForContent(chunk, thinkingEnabled, currentType) parts, finished, nextType := ParseSSEChunkForContent(chunk, thinkingEnabled, currentType)
parts = filterLeakedContentFilterParts(parts)
return LineResult{ return LineResult{
Parsed: true, Parsed: true,
Stop: finished, Stop: finished,

View File

@@ -35,3 +35,33 @@ func TestParseDeepSeekContentLineContent(t *testing.T) {
t.Fatalf("unexpected parts: %#v", res.Parts) t.Fatalf("unexpected parts: %#v", res.Parts)
} }
} }
func TestParseDeepSeekContentLineStripsLeakedContentFilterSuffix(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"正常输出CONTENT_FILTER你好这个问题我暂时无法回答"}`), false, "text")
if !res.Parsed || res.Stop {
t.Fatalf("expected parsed non-stop result: %#v", res)
}
if len(res.Parts) != 1 || res.Parts[0].Text != "正常输出" {
t.Fatalf("unexpected parts after filter: %#v", res.Parts)
}
}
func TestParseDeepSeekContentLineDropsPureLeakedContentFilterChunk(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"CONTENT_FILTER你好这个问题我暂时无法回答"}`), false, "text")
if !res.Parsed || res.Stop {
t.Fatalf("expected parsed non-stop result: %#v", res)
}
if len(res.Parts) != 0 {
t.Fatalf("expected empty parts, got %#v", res.Parts)
}
}
func TestParseDeepSeekContentLineTrimsFromContentFilterKeyword(t *testing.T) {
res := ParseDeepSeekContentLine([]byte(`data: {"p":"response/content","v":"模型会在命中 CONTENT_FILTER 时返回拒绝原因。"}`), false, "text")
if !res.Parsed || res.Stop {
t.Fatalf("expected parsed non-stop result: %#v", res)
}
if len(res.Parts) != 1 || res.Parts[0].Text != "模型会在命中" {
t.Fatalf("unexpected parts after filter: %#v", res.Parts)
}
}

View File

@@ -227,6 +227,24 @@ test('sieve flushes incomplete captured XML tool blocks without leaking raw tags
assert.equal(leakedText.includes('<tool_call'), false); assert.equal(leakedText.includes('<tool_call'), false);
}); });
test('sieve captures XML wrapper tags with attributes without leaking wrapper text', () => {
const events = runSieve(
[
'前置正文H。',
'<tool_calls id="x"><tool_call><tool_name>read_file</tool_name><parameters>{"path":"README.MD"}</parameters></tool_call></tool_calls>',
'后置正文I。',
],
['read_file'],
);
const leakedText = collectText(events);
const hasToolCall = events.some((evt) => evt.type === 'tool_calls' && evt.calls?.length > 0);
assert.equal(hasToolCall, true);
assert.equal(leakedText.includes('前置正文H。'), true);
assert.equal(leakedText.includes('后置正文I。'), true);
assert.equal(leakedText.includes('<tool_calls id=\"x\">'), false);
assert.equal(leakedText.includes('</tool_calls>'), false);
});
test('sieve still intercepts large tool json payloads over previous capture limit', () => { test('sieve still intercepts large tool json payloads over previous capture limit', () => {
const large = 'a'.repeat(9000); const large = 'a'.repeat(9000);
const payload = `{"tool_calls":[{"name":"read_file","input":{"path":"${large}"}}]}`; const payload = `{"tool_calls":[{"name":"read_file","input":{"path":"${large}"}}]}`;
@@ -252,6 +270,46 @@ test('sieve keeps plain text intact in tool mode when no tool call appears', ()
assert.equal(leakedText, '你好,这是普通文本回复。请继续。'); assert.equal(leakedText, '你好,这是普通文本回复。请继续。');
}); });
test('sieve keeps plain "tool_calls" prose as text when no valid payload follows', () => {
const events = runSieve(
['前置。', '这里提到 tool_calls 只是解释,不是调用。', '后置。'],
['read_file'],
);
const leakedText = collectText(events);
const hasToolCall = events.some((evt) => evt.type === 'tool_calls' && evt.calls?.length > 0);
assert.equal(hasToolCall, false);
assert.equal(leakedText.includes('tool_calls'), true);
assert.equal(leakedText, '前置。这里提到 tool_calls 只是解释,不是调用。后置。');
});
test('sieve keeps numbered planning prose before a real tool payload (mobile-chat style)', () => {
const events = runSieve(
[
'好的,我会依次测试每个工具,先把所有工具都调用一遍,然后汇总结果给你看。\n\n1. 获取当前时间\n',
'{"tool_calls":[{"name":"get_current_time","input":{}}]}',
],
['get_current_time'],
);
const leakedText = collectText(events);
const finalCalls = events.filter((evt) => evt.type === 'tool_calls').flatMap((evt) => evt.calls || []);
assert.equal(finalCalls.length, 1);
assert.equal(finalCalls[0].name, 'get_current_time');
assert.equal(leakedText.includes('先把所有工具都调用一遍'), true);
assert.equal(leakedText.includes('1. 获取当前时间'), true);
assert.equal(leakedText.toLowerCase().includes('tool_calls'), false);
});
test('sieve keeps numbered planning prose when no tool payload follows', () => {
const events = runSieve(
['好的,我会依次测试每个工具。\n\n1. 获取当前时间'],
['get_current_time'],
);
const leakedText = collectText(events);
const hasToolCall = events.some((evt) => evt.type === 'tool_calls' && evt.calls?.length > 0);
assert.equal(hasToolCall, false);
assert.equal(leakedText, '好的,我会依次测试每个工具。\n\n1. 获取当前时间');
});
test('sieve emits unknown tool payload (no args) as executable tool call', () => { test('sieve emits unknown tool payload (no args) as executable tool call', () => {
const events = runSieve( const events = runSieve(
['{"tool_calls":[{"name":"not_in_schema"}]}', '后置正文G。'], ['{"tool_calls":[{"name":"not_in_schema"}]}', '后置正文G。'],

View File

@@ -64,6 +64,27 @@ export default function AccountManagerContainer({ config, onRefresh, onMessage,
return ( return (
<div className="space-y-6"> <div className="space-y-6">
{Boolean(config?.env_source_present) && (
<div className={`rounded-xl border px-4 py-3 text-sm ${
config?.env_writeback_enabled
? (config?.env_backed ? 'border-amber-500/30 bg-amber-500/10 text-amber-600' : 'border-emerald-500/30 bg-emerald-500/10 text-emerald-600')
: 'border-amber-500/30 bg-amber-500/10 text-amber-600'
}`}>
<p className="font-medium">
{config?.env_writeback_enabled
? (config?.env_backed
? t('accountManager.envModeWritebackPendingTitle')
: t('accountManager.envModeWritebackActiveTitle'))
: t('accountManager.envModeRiskTitle')}
</p>
<p className="mt-1 text-xs opacity-90">
{config?.env_writeback_enabled
? t('accountManager.envModeWritebackDesc', { path: config?.config_path || 'config.json' })
: t('accountManager.envModeRiskDesc')}
</p>
</div>
)}
<QueueCards queueStatus={queueStatus} t={t} /> <QueueCards queueStatus={queueStatus} t={t} />
<ApiKeysPanel <ApiKeysPanel

View File

@@ -138,7 +138,12 @@
"sessionCount": "Sessions: {count}", "sessionCount": "Sessions: {count}",
"deleteAllSessions": "Delete all sessions", "deleteAllSessions": "Delete all sessions",
"deleteAllSessionsConfirm": "Are you sure you want to delete all sessions for this account? This action cannot be undone.", "deleteAllSessionsConfirm": "Are you sure you want to delete all sessions for this account? This action cannot be undone.",
"deleteAllSessionsSuccess": "Successfully deleted all sessions" "deleteAllSessionsSuccess": "Successfully deleted all sessions",
"envModeRiskTitle": "Environment-variable config mode detected (persistence risk)",
"envModeRiskDesc": "Detected DS2API_CONFIG_JSON/CONFIG_JSON. If DS2API_ENV_WRITEBACK is not enabled, Admin UI edits are in-memory only and may be lost after restart.",
"envModeWritebackPendingTitle": "Env mode + auto-persistence enabled (pending file handoff)",
"envModeWritebackActiveTitle": "Env mode + auto-persistence active",
"envModeWritebackDesc": "The app will auto-create/write the config file and transition to file-backed mode. Current persistence path: {path}"
}, },
"apiTester": { "apiTester": {
"defaultMessage": "Hello, please introduce yourself in one sentence.", "defaultMessage": "Hello, please introduce yourself in one sentence.",

View File

@@ -138,7 +138,12 @@
"sessionCount": "会话: {count}", "sessionCount": "会话: {count}",
"deleteAllSessions": "删除所有会话", "deleteAllSessions": "删除所有会话",
"deleteAllSessionsConfirm": "确定要删除该账号的所有会话吗?此操作不可恢复。", "deleteAllSessionsConfirm": "确定要删除该账号的所有会话吗?此操作不可恢复。",
"deleteAllSessionsSuccess": "删除成功" "deleteAllSessionsSuccess": "删除成功",
"envModeRiskTitle": "当前为环境变量配置模式(有持久化风险)",
"envModeRiskDesc": "检测到 DS2API_CONFIG_JSON/CONFIG_JSON。若未开启 DS2API_ENV_WRITEBACK管理台改动仅在内存生效重启可能丢失。",
"envModeWritebackPendingTitle": "环境变量模式 + 自动持久化已开启(等待落盘)",
"envModeWritebackActiveTitle": "环境变量模式 + 自动持久化已生效",
"envModeWritebackDesc": "程序会自动创建/写入配置文件并在后续切换为文件模式。当前持久化路径:{path}"
}, },
"apiTester": { "apiTester": {
"defaultMessage": "你好,请用一句话介绍你自己。", "defaultMessage": "你好,请用一句话介绍你自己。",