Compare commits

...

18 Commits

Author SHA1 Message Date
CJACK.
dbf2bfb64f Merge pull request #466 from CJackHwang/dev
Merge pull request #465 from CJackHwang/codex/review-and-update-project-documentation

docs: clarify Vercel chat-stream supports root `/chat/completions` alias
2026-05-09 19:46:45 +08:00
CJACK.
9e9a7f1bec Merge pull request #467 from CJackHwang/codex/fix-dsml-delimiter-consistency-in-examples
fix(toolcall): unify DSML delimiter in correct examples
2026-05-09 19:37:40 +08:00
CJACK.
96691aa37a fix(toolcall): unify DSML delimiter in correct examples 2026-05-09 19:29:36 +08:00
CJACK.
a3ce8008af Merge pull request #465 from CJackHwang/codex/review-and-update-project-documentation
docs: clarify Vercel chat-stream supports root `/chat/completions` alias
2026-05-09 19:00:27 +08:00
CJACK.
23a79df687 docs: sync Vercel chat-stream path alias in docs 2026-05-09 18:59:08 +08:00
CJACK.
e251a7ee29 Merge pull request #463 from CJackHwang/codex/align-vercel-js-path-with-go-implementation-tqm8ci
Align Vercel JS CORS Vary-Origin behavior with Go
2026-05-09 18:27:39 +08:00
CJACK.
30cca7cda0 Merge pull request #462 from CJackHwang/codex/align-vercel-js-path-with-go-implementation
fix(vercel): align JS stream path guard with Go /chat/completions alias
2026-05-09 18:26:35 +08:00
CJACK.
ab163edee7 Merge pull request #461 from CJackHwang/codex/refactor-hasasciiprefixfoldat-and-hasdsmlprefix
抽取通用 ASCII 部分前缀匹配以合并重复的 DSML 前缀逻辑
2026-05-09 18:20:31 +08:00
CJACK.
1201c3773f Align Vercel JS CORS Vary-Origin behavior with Go 2026-05-09 18:17:16 +08:00
CJACK.
595ddf52af fix(vercel): align js stream path guard with go chat alias 2026-05-09 18:16:47 +08:00
CJACK.
0adffccd46 refactor tool markup prefix folding helpers 2026-05-09 18:09:12 +08:00
CJACK.
0670d5acb4 Update VERSION 2026-05-09 17:54:26 +08:00
CJACK.
239c4faa97 Merge pull request #460 from waiwaic/main
fix(toolcall): eliminate strings.ToLower panics from Unicode case folding
2026-05-09 16:56:46 +08:00
waiwai
f33789399e fix(toolcall): correct DSML closing tag slash position
The closing tag format was <|/DSML|tag> but must be </|DSML|tag>.
The scanner's closing-tag detection checks text[1] == '/', so the
slash must come immediately after '<', before the first full-width
pipe (U+FF5C). Tags like <|/DSML|tool_calls> would not set
closing=true and would not match any tool markup name.

Files fixed:
- internal/toolcall/tool_prompt.go: all closing tags
- internal/promptcompat/prompt_build_test.go: 1 test expectation
2026-05-09 16:42:22 +08:00
waiwai
1e00e482a6 fix(toolcall): eliminate strings.ToLower panics from Unicode case folding
Replace all strings.ToLower usage with ASCII case-insensitive matching
(hasASCIIPrefixFoldAt, indexASCIIFold, hasDSMLPrefix) to prevent slice
bounds errors when Unicode characters change byte length after case
folding (e.g., Turkish İ U+0130 → i + combining dot: 2 bytes → 3 bytes).

Root cause: code created a strings.ToLower(text) copy, found byte
positions in that copy, then used those positions to slice the
original text — byte offsets that were valid in the lowercased copy
became out-of-bounds in the original when case folding changed byte
lengths.

Files changed:
- toolcalls_scan.go: remove 5 lower usages, add hasDSMLPrefix
- toolcalls_parse_markup.go: remove 3 lower usages, add indexASCIIFold
- toolcalls_markup.go: SanitizeLooseCDATA lower removal
- toolcalls_parse.go: updateCDATAStateForStrip lower removal
- tool_prompt.go: align DSML pipe characters with tool call spec
- tool_prompt_test.go: fix pre-existing test character mismatch
2026-05-09 15:05:51 +08:00
CJACK.
7ab5a0e66d Merge pull request #458 from CJackHwang/dev
Avoid lowercasing ignored XML tails in toolcall
2026-05-08 17:13:00 +08:00
CJACK.
410efbd70b Merge pull request #457 from NgoQuocViet2001/ai/skipxml-lower-hotpath
fix(toolcall): avoid lowercasing ignored XML tails
2026-05-08 17:05:28 +08:00
NgoQuocViet2001
7179b995bb fix(toolcall): avoid lowercasing ignored XML tails 2026-05-08 14:15:32 +07:00
17 changed files with 180 additions and 97 deletions

View File

@@ -243,6 +243,8 @@ Retired historical families such as `claude-1.*`, `claude-2.*`, `claude-instant-
### `POST /v1/chat/completions`
> Path note: besides the canonical `/v1/chat/completions`, DS2API also accepts the root shortcut `/chat/completions`. On Vercel Runtime, `stream=true` on either path is handled by the Node streaming bridge, while non-stream stays on the Go primary path.
**Headers**:
```http

2
API.md
View File

@@ -249,6 +249,8 @@ OpenAI `/v1/*` 仍是规范路径。对于只配置 DS2API 根地址的客户端
### `POST /v1/chat/completions`
> 路径说明:除规范路径 `/v1/chat/completions` 外,也支持根路径快捷别名 `/chat/completions`;在 Vercel Runtime 上,这两个路径的 `stream=true` 请求都会进入 Node 流式桥接逻辑,非流式仍走 Go 主链路。
**请求头**
```http

View File

@@ -295,7 +295,7 @@ cp config.example.json config.json
base64 < config.json | tr -d '\n'
```
> **流式说明**`/v1/chat/completions` 在 Vercel 上默认走 `api/chat-stream.js`Node Runtime以保证实时 SSE。鉴权、账号选择、会话/PoW 准备仍由 Go 内部 prepare 接口完成;流式响应(含 `tools`)在 Node 侧执行与 Go 对齐的输出组装与防泄漏处理。虽然这里只有 OpenAI chat 流式走 Node但 CORS 放行策略仍与 Go 主路由保持一致,统一覆盖第三方客户端预检场景。
> **流式说明**OpenAI Chat 流式在 Vercel 上会由 `api/chat-stream.js`Node Runtime承接,支持规范路径 `/v1/chat/completions` 与根路径快捷别名 `/chat/completions`。鉴权、账号选择、会话/PoW 准备仍由 Go 内部 prepare 接口完成;流式响应(含 `tools`)在 Node 侧执行与 Go 对齐的输出组装与防泄漏处理。虽然这里只有 OpenAI chat 流式走 Node但 CORS 放行策略仍与 Go 主路由保持一致,统一覆盖第三方客户端预检场景。
详细部署说明请参阅 [部署指南](docs/DEPLOY.md)。

View File

@@ -283,7 +283,7 @@ Recommended: convert `config.json` to Base64 locally, then paste into `DS2API_CO
base64 < config.json | tr -d '\n'
```
> **Streaming note**: `/v1/chat/completions` on Vercel is routed to `api/chat-stream.js` (Node Runtime) for real-time SSE. Auth, account selection, and session/PoW preparation are still handled by the Go internal prepare endpoint; streaming output (including `tools`) is assembled on Node with Go-aligned anti-leak handling. This is the only interface family currently routed through Node, and its CORS allow behavior is kept aligned with the Go router so third-party preflight handling stays unified.
> **Streaming note**: OpenAI Chat streaming on Vercel is routed to `api/chat-stream.js` (Node Runtime), with both the canonical `/v1/chat/completions` path and the root shortcut `/chat/completions` supported. Auth, account selection, and session/PoW preparation are still handled by the Go internal prepare endpoint; streaming output (including `tools`) is assembled on Node with Go-aligned anti-leak handling. This is the only interface family currently routed through Node, and its CORS allow behavior is kept aligned with the Go router so third-party preflight handling stays unified.
For detailed deployment instructions, see the [Deployment Guide](docs/DEPLOY.en.md).

View File

@@ -1 +1 @@
4.4.5
4.4.6

View File

@@ -292,7 +292,7 @@ func TestBuildClaudeToolPromptSingleTool(t *testing.T) {
if !containsStr(prompt, "Search the web") {
t.Fatalf("expected description in prompt")
}
if !containsStr(prompt, "<|DSML|tool_calls>") {
if !containsStr(prompt, "<DSMLtool_calls>") {
t.Fatalf("expected DSML tool_calls format in prompt")
}
if !containsStr(prompt, "TOOL CALL FORMAT") {

View File

@@ -19,13 +19,15 @@ const BLOCKED_CORS_REQUEST_HEADERS = new Set([
function setCorsHeaders(res, req) {
const origin = asString(readHeader(req, 'origin'));
res.setHeader('Access-Control-Allow-Origin', origin || '*');
if (origin) {
addVaryHeader(res, 'Origin');
}
res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS, PUT, DELETE');
res.setHeader('Access-Control-Max-Age', '600');
res.setHeader(
'Access-Control-Allow-Headers',
buildCORSAllowHeaders(req),
);
addVaryHeader(res, 'Origin');
addVaryHeader(res, 'Access-Control-Request-Headers');
if (asString(readHeader(req, 'access-control-request-private-network')).toLowerCase() === 'true') {
res.setHeader('Access-Control-Allow-Private-Network', 'true');

View File

@@ -88,7 +88,7 @@ function isVercelRuntime() {
function isNodeStreamSupportedPath(rawURL) {
const path = extractPathname(rawURL);
return path === '/v1/chat/completions';
return path === '/v1/chat/completions' || path === '/chat/completions';
}
function extractPathname(rawURL) {

View File

@@ -74,7 +74,7 @@ func TestBuildOpenAIFinalPrompt_VercelPreparePathKeepsFinalAnswerInstruction(t *
}
finalPrompt, _ := buildOpenAIFinalPrompt(messages, tools, "", false)
if !strings.Contains(finalPrompt, "Remember: The ONLY valid way to use tools is the <|DSML|tool_calls>...</|DSML|tool_calls> block at the end of your response.") {
if !strings.Contains(finalPrompt, "Remember: The ONLY valid way to use tools is the <DSMLtool_calls>...</DSMLtool_calls> block at the end of your response.") {
t.Fatalf("vercel prepare finalPrompt missing final tool-call anchor instruction: %q", finalPrompt)
}
if !strings.Contains(finalPrompt, "TOOL CALL FORMAT") {

View File

@@ -11,46 +11,45 @@ import "strings"
func BuildToolCallInstructions(toolNames []string) string {
return `TOOL CALL FORMAT — FOLLOW EXACTLY:
<|DSML|tool_calls>
<|DSML|invoke name="TOOL_NAME_HERE">
<|DSML|parameter name="PARAMETER_NAME"><![CDATA[PARAMETER_VALUE]]></|DSML|parameter>
</|DSML|invoke>
</|DSML|tool_calls>
<DSMLtool_calls>
<DSMLinvoke name="TOOL_NAME_HERE">
<DSMLparameter name="PARAMETER_NAME"><![CDATA[PARAMETER_VALUE]]></DSMLparameter>
</DSMLinvoke>
</DSMLtool_calls>
RULES:
1) Use the <|DSML|tool_calls> wrapper format.
2) Put one or more <|DSML|invoke> entries under a single <|DSML|tool_calls> root.
3) Put the tool name in the invoke name attribute: <|DSML|invoke name="TOOL_NAME">.
1) Use the <DSMLtool_calls> wrapper format.
2) Put one or more <DSMLinvoke> entries under a single <DSMLtool_calls> root.
3) Put the tool name in the invoke name attribute: <DSMLinvoke name="TOOL_NAME">.
4) All string values must use <![CDATA[...]]>, even short ones. This includes code, scripts, file contents, prompts, paths, names, and queries.
5) Every top-level argument must be a <|DSML|parameter name="ARG_NAME">...</|DSML|parameter> node.
5) Every top-level argument must be a <DSMLparameter name="ARG_NAME">...</DSMLparameter> node.
6) Objects use nested XML elements inside the parameter body. Arrays may repeat <item> children.
7) Numbers, booleans, and null stay plain text.
8) Use only the parameter names in the tool schema. Do not invent fields.
9) Do NOT wrap XML in markdown fences. Do NOT output explanations, role markers, or internal monologue.
10) If you call a tool, the first non-whitespace characters of that tool block must be exactly <|DSML|tool_calls>.
11) Never omit the opening <|DSML|tool_calls> tag, even if you already plan to close with </|DSML|tool_calls>.
10) If you call a tool, the first non-whitespace characters of that tool block must be exactly <DSMLtool_calls>.
11) Never omit the opening <DSMLtool_calls> tag, even if you already plan to close with </DSMLtool_calls>.
12) Compatibility note: the runtime also accepts the legacy XML tags <tool_calls> / <invoke> / <parameter>, but prefer the DSML-prefixed form above.
PARAMETER SHAPES:
- string => <|DSML|parameter name="x"><![CDATA[value]]></|DSML|parameter>
- object => <|DSML|parameter name="x"><field>...</field></|DSML|parameter>
- array => <|DSML|parameter name="x"><item>...</item><item>...</item></|DSML|parameter>
- number/bool/null => <|DSML|parameter name="x">plain_text</|DSML|parameter>
- string => <DSMLparameter name="x"><![CDATA[value]]></DSMLparameter>
- object => <DSMLparameter name="x"><field>...</field></DSMLparameter>
- array => <DSMLparameter name="x"><item>...</item><item>...</item></DSMLparameter>
- number/bool/null => <DSMLparameter name="x">plain_text</DSMLparameter>
【WRONG — Do NOT do these】:
Wrong 1 — mixed text after XML:
<|DSML|tool_calls>...</|DSML|tool_calls> I hope this helps.
<DSMLtool_calls>...</DSMLtool_calls> I hope this helps.
Wrong 2 — Markdown code fences:
` + "```xml" + `
<|DSML|tool_calls>...</|DSML|tool_calls>
<DSMLtool_calls>...</DSMLtool_calls>
` + "```" + `
Wrong 3 — missing opening wrapper:
<|DSML|invoke name="TOOL_NAME">...</|DSML|invoke>
</|DSML|tool_calls>
Remember: The ONLY valid way to use tools is the <|DSML|tool_calls>...</|DSML|tool_calls> block at the end of your response.
<DSMLinvoke name="TOOL_NAME">...</DSMLinvoke>
</DSMLtool_calls>
Remember: The ONLY valid way to use tools is the <DSMLtool_calls>...</DSMLtool_calls> block at the end of your response.
` + buildCorrectToolExamples(toolNames)
}
@@ -141,21 +140,21 @@ func firstScriptExample(names []string) (promptToolExample, bool) {
func renderToolExampleBlock(calls []promptToolExample) string {
var b strings.Builder
b.WriteString("<|DSML|tool_calls>\n")
b.WriteString("<DSMLtool_calls>\n")
for _, call := range calls {
b.WriteString(` <|DSML|invoke name="`)
b.WriteString(` <DSMLinvoke name="`)
b.WriteString(call.name)
b.WriteString(`">` + "\n")
b.WriteString(indentPromptParameters(call.params, " "))
b.WriteString("\n </|DSML|invoke>\n")
b.WriteString("\n </DSMLinvoke>\n")
}
b.WriteString("</|DSML|tool_calls>")
b.WriteString("</DSMLtool_calls>")
return b.String()
}
func indentPromptParameters(body, indent string) string {
if strings.TrimSpace(body) == "" {
return indent + `<|DSML|parameter name="content"></|DSML|parameter>`
return indent + `<DSMLparameter name="content"></DSMLparameter>`
}
lines := strings.Split(body, "\n")
for i, line := range lines {
@@ -169,7 +168,7 @@ func indentPromptParameters(body, indent string) string {
}
func wrapParameter(name, inner string) string {
return `<|DSML|parameter name="` + name + `">` + inner + `</|DSML|parameter>`
return `<DSMLparameter name="` + name + `">` + inner + `</DSMLparameter>`
}
func exampleBasicParams(name string) (string, bool) {
@@ -195,7 +194,7 @@ func exampleBasicParams(name string) (string, bool) {
case "Edit":
return wrapParameter("file_path", promptCDATA("README.md")) + "\n" + wrapParameter("old_string", promptCDATA("foo")) + "\n" + wrapParameter("new_string", promptCDATA("bar")), true
case "MultiEdit":
return wrapParameter("file_path", promptCDATA("README.md")) + "\n" + `<|DSML|parameter name="edits"><item><old_string>` + promptCDATA("foo") + `</old_string><new_string>` + promptCDATA("bar") + `</new_string></item></|DSML|parameter>`, true
return wrapParameter("file_path", promptCDATA("README.md")) + "\n" + `<DSMLparameter name="edits"><item><old_string>` + promptCDATA("foo") + `</old_string><new_string>` + promptCDATA("bar") + `</new_string></item></DSMLparameter>`, true
}
return "", false
}
@@ -203,11 +202,11 @@ func exampleBasicParams(name string) (string, bool) {
func exampleNestedParams(name string) (string, bool) {
switch strings.TrimSpace(name) {
case "MultiEdit":
return wrapParameter("file_path", promptCDATA("README.md")) + "\n" + `<|DSML|parameter name="edits"><item><old_string>` + promptCDATA("foo") + `</old_string><new_string>` + promptCDATA("bar") + `</new_string></item></|DSML|parameter>`, true
return wrapParameter("file_path", promptCDATA("README.md")) + "\n" + `<DSMLparameter name="edits"><item><old_string>` + promptCDATA("foo") + `</old_string><new_string>` + promptCDATA("bar") + `</new_string></item></DSMLparameter>`, true
case "Task":
return wrapParameter("description", promptCDATA("Investigate flaky tests")) + "\n" + wrapParameter("prompt", promptCDATA("Run targeted tests and summarize failures")), true
case "ask_followup_question":
return wrapParameter("question", promptCDATA("Which approach do you prefer?")) + "\n" + `<|DSML|parameter name="follow_up"><item><text>` + promptCDATA("Option A") + `</text></item><item><text>` + promptCDATA("Option B") + `</text></item></|DSML|parameter>`, true
return wrapParameter("question", promptCDATA("Which approach do you prefer?")) + "\n" + `<DSMLparameter name="follow_up"><item><text>` + promptCDATA("Option A") + `</text></item><item><text>` + promptCDATA("Option B") + `</text></item></DSMLparameter>`, true
}
return "", false
}

View File

@@ -7,20 +7,20 @@ import (
func TestBuildToolCallInstructions_ExecCommandUsesCmdExample(t *testing.T) {
out := BuildToolCallInstructions([]string{"exec_command"})
if !strings.Contains(out, `<|DSML|invoke name="exec_command">`) {
if !strings.Contains(out, `<DSMLinvoke name="exec_command">`) {
t.Fatalf("expected exec_command in examples, got: %s", out)
}
if !strings.Contains(out, `<|DSML|parameter name="cmd"><![CDATA[pwd]]></|DSML|parameter>`) {
if !strings.Contains(out, `<DSMLparameter name="cmd"><![CDATA[pwd]]></DSMLparameter>`) {
t.Fatalf("expected cmd parameter example for exec_command, got: %s", out)
}
}
func TestBuildToolCallInstructions_ExecuteCommandUsesCommandExample(t *testing.T) {
out := BuildToolCallInstructions([]string{"execute_command"})
if !strings.Contains(out, `<|DSML|invoke name="execute_command">`) {
if !strings.Contains(out, `<DSMLinvoke name="execute_command">`) {
t.Fatalf("expected execute_command in examples, got: %s", out)
}
if !strings.Contains(out, `<|DSML|parameter name="command"><![CDATA[pwd]]></|DSML|parameter>`) {
if !strings.Contains(out, `<DSMLparameter name="command"><![CDATA[pwd]]></DSMLparameter>`) {
t.Fatalf("expected command parameter example for execute_command, got: %s", out)
}
}
@@ -34,20 +34,20 @@ func TestBuildToolCallInstructions_BashUsesCommandAndDescriptionExamples(t *test
sawDescription := false
for _, block := range blocks {
if !strings.Contains(block, `<|DSML|parameter name="command">`) {
if !strings.Contains(block, `<DSMLparameter name="command">`) {
t.Fatalf("expected every Bash example to use command parameter, got: %s", block)
}
if strings.Contains(block, `<|DSML|parameter name="path">`) || strings.Contains(block, `<|DSML|parameter name="content">`) {
if strings.Contains(block, `<DSMLparameter name="path">`) || strings.Contains(block, `<DSMLparameter name="content">`) {
t.Fatalf("expected Bash examples not to use file write parameters, got: %s", block)
}
if strings.Contains(block, `<|DSML|parameter name="description">`) {
if strings.Contains(block, `<DSMLparameter name="description">`) {
sawDescription = true
}
}
if !sawDescription {
t.Fatalf("expected Bash long-script example to include description, got: %s", out)
}
if strings.Contains(out, `<|DSML|invoke name="Read">`) {
if strings.Contains(out, `<DSMLinvoke name="Read">`) {
t.Fatalf("expected examples to avoid unavailable hard-coded Read tool, got: %s", out)
}
}
@@ -60,10 +60,10 @@ func TestBuildToolCallInstructions_ExecuteCommandLongScriptUsesCommand(t *testin
}
for _, block := range blocks {
if !strings.Contains(block, `<|DSML|parameter name="command">`) {
if !strings.Contains(block, `<DSMLparameter name="command">`) {
t.Fatalf("expected execute_command examples to use command parameter, got: %s", block)
}
if strings.Contains(block, `<|DSML|parameter name="path">`) || strings.Contains(block, `<|DSML|parameter name="content">`) {
if strings.Contains(block, `<DSMLparameter name="path">`) || strings.Contains(block, `<DSMLparameter name="content">`) {
t.Fatalf("expected execute_command examples not to use file write parameters, got: %s", block)
}
}
@@ -80,10 +80,10 @@ func TestBuildToolCallInstructions_ExecCommandLongScriptUsesCmd(t *testing.T) {
}
for _, block := range blocks {
if !strings.Contains(block, `<|DSML|parameter name="cmd">`) {
if !strings.Contains(block, `<DSMLparameter name="cmd">`) {
t.Fatalf("expected exec_command examples to use cmd parameter, got: %s", block)
}
if strings.Contains(block, `<|DSML|parameter name="command">`) || strings.Contains(block, `<|DSML|parameter name="path">`) || strings.Contains(block, `<|DSML|parameter name="content">`) {
if strings.Contains(block, `<DSMLparameter name="command">`) || strings.Contains(block, `<DSMLparameter name="path">`) || strings.Contains(block, `<DSMLparameter name="content">`) {
t.Fatalf("expected exec_command examples not to use command or file write parameters, got: %s", block)
}
}
@@ -100,10 +100,10 @@ func TestBuildToolCallInstructions_WriteUsesFilePathAndContent(t *testing.T) {
}
for _, block := range blocks {
if !strings.Contains(block, `<|DSML|parameter name="file_path">`) || !strings.Contains(block, `<|DSML|parameter name="content">`) {
if !strings.Contains(block, `<DSMLparameter name="file_path">`) || !strings.Contains(block, `<DSMLparameter name="content">`) {
t.Fatalf("expected Write examples to use file_path and content, got: %s", block)
}
if strings.Contains(block, `<|DSML|parameter name="path">`) {
if strings.Contains(block, `<DSMLparameter name="path">`) {
t.Fatalf("expected Write examples not to use path, got: %s", block)
}
}
@@ -111,7 +111,7 @@ func TestBuildToolCallInstructions_WriteUsesFilePathAndContent(t *testing.T) {
func TestBuildToolCallInstructions_AnchorsMissingOpeningWrapperFailureMode(t *testing.T) {
out := BuildToolCallInstructions([]string{"read_file"})
if !strings.Contains(out, "Never omit the opening <|DSML|tool_calls> tag") {
if !strings.Contains(out, "Never omit the opening <DSMLtool_calls> tag") {
t.Fatalf("expected explicit missing-opening-tag warning, got: %s", out)
}
if !strings.Contains(out, "Wrong 3 — missing opening wrapper") {
@@ -120,7 +120,7 @@ func TestBuildToolCallInstructions_AnchorsMissingOpeningWrapperFailureMode(t *te
}
func findInvokeBlocks(text, name string) []string {
open := `<|DSML|invoke name="` + name + `">`
open := `<DSMLinvoke name="` + name + `">`
remaining := text
blocks := []string{}
for {
@@ -129,11 +129,11 @@ func findInvokeBlocks(text, name string) []string {
return blocks
}
remaining = remaining[start:]
end := strings.Index(remaining, `</|DSML|invoke>`)
end := strings.Index(remaining, `</DSMLinvoke>`)
if end < 0 {
return blocks
}
end += len(`</|DSML|invoke>`)
end += len(`</DSMLinvoke>`)
blocks = append(blocks, remaining[:end])
remaining = remaining[end:]
}

View File

@@ -145,7 +145,6 @@ func SanitizeLooseCDATA(text string) string {
return ""
}
lower := strings.ToLower(text)
const openMarker = "<![cdata["
const closeMarker = "]]>"
@@ -154,17 +153,16 @@ func SanitizeLooseCDATA(text string) string {
changed := false
pos := 0
for pos < len(text) {
startRel := strings.Index(lower[pos:], openMarker)
if startRel < 0 {
start := indexASCIIFold(text, pos, openMarker)
if start < 0 {
b.WriteString(text[pos:])
break
}
start := pos + startRel
contentStart := start + len(openMarker)
b.WriteString(text[pos:start])
if endRel := strings.Index(lower[contentStart:], closeMarker); endRel >= 0 {
end := contentStart + endRel + len(closeMarker)
if endRel := indexASCIIFold(text, contentStart, closeMarker); endRel >= 0 {
end := endRel + len(closeMarker)
b.WriteString(text[start:end])
pos = end
continue

View File

@@ -212,17 +212,16 @@ func firstFenceMarkerIndex(line string) int {
}
func updateCDATAStateForStrip(inCDATA bool, cdataFenceMarker, line string) (bool, string) {
lower := strings.ToLower(line)
pos := 0
state := inCDATA
fenceMarker := cdataFenceMarker
lineForFence := line
if !state {
start := strings.Index(lower[pos:], "<![cdata[")
start := indexASCIIFold(line, pos, "<![cdata[")
if start < 0 {
return false, ""
}
pos += start + len("<![cdata[")
pos = start + len("<![cdata[")
state = true
lineForFence = line[pos:]
}
@@ -239,24 +238,23 @@ func updateCDATAStateForStrip(inCDATA bool, cdataFenceMarker, line string) (bool
fenceMarker = ""
}
for pos < len(lower) {
end := strings.Index(lower[pos:], "]]>")
if end < 0 {
for pos < len(line) {
endPos := indexASCIIFold(line, pos, "]]>")
if endPos < 0 {
return true, fenceMarker
}
endPos := pos + end
pos = endPos + len("]]>")
if fenceMarker != "" {
continue
}
if cdataEndLooksStructural(lower, pos) || strings.TrimSpace(lower[pos:]) == "" {
if cdataEndLooksStructural(line, pos) || strings.TrimSpace(line[pos:]) == "" {
state = false
for pos < len(lower) {
start := strings.Index(lower[pos:], "<![cdata[")
for pos < len(line) {
start := indexASCIIFold(line, pos, "<![cdata[")
if start < 0 {
return false, ""
}
pos += start + len("<![cdata[")
pos = start + len("<![cdata[")
state = true
trimmedTail := strings.TrimLeft(line[pos:], " \t")
if marker, ok := parseFenceOpen(trimmedTail); ok {

View File

@@ -141,7 +141,6 @@ func findXMLElementBlocks(text, tag string) []xmlElementBlock {
}
func findXMLStartTagOutsideCDATA(text, tag string, from int) (start, bodyStart int, attrs string, ok bool) {
lower := strings.ToLower(text)
target := "<" + strings.ToLower(tag)
for i := maxInt(from, 0); i < len(text); {
next, advanced, blocked := skipXMLIgnoredSection(text, i)
@@ -152,7 +151,7 @@ func findXMLStartTagOutsideCDATA(text, tag string, from int) (start, bodyStart i
i = next
continue
}
if strings.HasPrefix(lower[i:], target) && hasXMLTagBoundary(text, i+len(target)) {
if hasASCIIPrefixFoldAt(text, i, target) && hasXMLTagBoundary(text, i+len(target)) {
end := findXMLTagEnd(text, i+len(target))
if end < 0 {
return -1, -1, "", false
@@ -165,7 +164,6 @@ func findXMLStartTagOutsideCDATA(text, tag string, from int) (start, bodyStart i
}
func findMatchingXMLEndTagOutsideCDATA(text, tag string, from int) (closeStart, closeEnd int, ok bool) {
lower := strings.ToLower(text)
openTarget := "<" + strings.ToLower(tag)
closeTarget := "</" + strings.ToLower(tag)
depth := 1
@@ -178,7 +176,7 @@ func findMatchingXMLEndTagOutsideCDATA(text, tag string, from int) (closeStart,
i = next
continue
}
if strings.HasPrefix(lower[i:], closeTarget) && hasXMLTagBoundary(text, i+len(closeTarget)) {
if hasASCIIPrefixFoldAt(text, i, closeTarget) && hasXMLTagBoundary(text, i+len(closeTarget)) {
end := findXMLTagEnd(text, i+len(closeTarget))
if end < 0 {
return -1, -1, false
@@ -190,7 +188,7 @@ func findMatchingXMLEndTagOutsideCDATA(text, tag string, from int) (closeStart,
i = end + 1
continue
}
if strings.HasPrefix(lower[i:], openTarget) && hasXMLTagBoundary(text, i+len(openTarget)) {
if hasASCIIPrefixFoldAt(text, i, openTarget) && hasXMLTagBoundary(text, i+len(openTarget)) {
end := findXMLTagEnd(text, i+len(openTarget))
if end < 0 {
return -1, -1, false
@@ -210,16 +208,15 @@ func skipXMLIgnoredSection(text string, i int) (next int, advanced bool, blocked
if i < 0 || i >= len(text) {
return i, false, false
}
tail := strings.ToLower(text[i:])
switch {
case strings.HasPrefix(tail, "<![cdata["):
case hasASCIIPrefixFoldAt(text, i, "<![cdata["):
end := findToolCDATAEnd(text, i+len("<![cdata["))
if end < 0 {
return 0, false, true
}
return end + len("]]>"), true, false
case strings.HasPrefix(tail, "<!--"):
end := strings.Index(tail[len("<!--"):], "-->")
case strings.HasPrefix(text[i:], "<!--"):
end := strings.Index(text[i+len("<!--"):], "-->")
if end < 0 {
return 0, false, true
}
@@ -229,6 +226,42 @@ func skipXMLIgnoredSection(text string, i int) (next int, advanced bool, blocked
}
}
func hasASCIIPrefixFoldAt(text string, start int, prefix string) bool {
if start < 0 || len(text)-start < len(prefix) {
return false
}
for j := 0; j < len(prefix); j++ {
if asciiLower(text[start+j]) != asciiLower(prefix[j]) {
return false
}
}
return true
}
func asciiLower(b byte) byte {
if b >= 'A' && b <= 'Z' {
return b + ('a' - 'A')
}
return b
}
// indexASCIIFold returns the absolute byte position in s where substr (ASCII-only) is
// found case-insensitively, scanning forward from start. Returns -1 if not found.
// Unlike strings.Index on a lowercased copy, this does not allocate or risk byte-length
// mismatch when non-ASCII runes change width under case folding.
func indexASCIIFold(s string, start int, substr string) int {
if start < 0 || len(s)-start < len(substr) {
return -1
}
end := len(s) - len(substr) + 1
for i := start; i < end; i++ {
if hasASCIIPrefixFoldAt(s, i, substr) {
return i
}
}
return -1
}
func findToolCDATAEnd(text string, from int) int {
if from < 0 || from >= len(text) {
return -1

View File

@@ -134,7 +134,6 @@ func scanToolMarkupTagAt(text string, start int) (ToolMarkupTag, bool) {
if start < 0 || start >= len(text) || text[start] != '<' {
return ToolMarkupTag{}, false
}
lower := strings.ToLower(text)
i := start + 1
for i < len(text) && text[i] == '<' {
i++
@@ -144,8 +143,8 @@ func scanToolMarkupTagAt(text string, start int) (ToolMarkupTag, bool) {
closing = true
i++
}
i, dsmlLike := consumeToolMarkupNamePrefix(lower, text, i)
name, nameLen := matchToolMarkupName(lower, i, dsmlLike)
i, dsmlLike := consumeToolMarkupNamePrefix(text, i)
name, nameLen := matchToolMarkupName(text, i, dsmlLike)
if nameLen == 0 {
return ToolMarkupTag{}, false
}
@@ -188,7 +187,6 @@ func IsPartialToolMarkupTagPrefix(text string) bool {
if text == "" || text[0] != '<' || strings.Contains(text, ">") {
return false
}
lower := strings.ToLower(text)
i := 1
for i < len(text) && text[i] == '<' {
i++
@@ -203,13 +201,13 @@ func IsPartialToolMarkupTagPrefix(text string) bool {
if i == len(text) {
return true
}
if hasToolMarkupNamePrefix(lower[i:]) {
if hasToolMarkupNamePrefix(text, i) {
return true
}
if strings.HasPrefix("dsml", lower[i:]) {
if hasASCIIPartialPrefixFoldAt(text, i, "dsml") {
return true
}
next, ok := consumeToolMarkupNamePrefixOnce(lower, text, i)
next, ok := consumeToolMarkupNamePrefixOnce(text, i)
if !ok {
return false
}
@@ -218,10 +216,10 @@ func IsPartialToolMarkupTagPrefix(text string) bool {
return false
}
func consumeToolMarkupNamePrefix(lower, text string, idx int) (int, bool) {
func consumeToolMarkupNamePrefix(text string, idx int) (int, bool) {
dsmlLike := false
for {
next, ok := consumeToolMarkupNamePrefixOnce(lower, text, idx)
next, ok := consumeToolMarkupNamePrefixOnce(text, idx)
if !ok {
return idx, dsmlLike
}
@@ -230,14 +228,14 @@ func consumeToolMarkupNamePrefix(lower, text string, idx int) (int, bool) {
}
}
func consumeToolMarkupNamePrefixOnce(lower, text string, idx int) (int, bool) {
func consumeToolMarkupNamePrefixOnce(text string, idx int) (int, bool) {
if next, ok := consumeToolMarkupPipe(text, idx); ok {
return next, true
}
if idx < len(text) && (text[idx] == ' ' || text[idx] == '\t' || text[idx] == '\r' || text[idx] == '\n') {
return idx + 1, true
}
if strings.HasPrefix(lower[idx:], "dsml") {
if hasASCIIPrefixFoldAt(text, idx, "dsml") {
next := idx + len("dsml")
if next < len(text) && (text[next] == '-' || text[next] == '_') {
next++
@@ -247,21 +245,37 @@ func consumeToolMarkupNamePrefixOnce(lower, text string, idx int) (int, bool) {
return idx, false
}
func hasToolMarkupNamePrefix(lowerTail string) bool {
func hasASCIIPartialPrefixFoldAt(text string, start int, prefix string) bool {
remain := len(text) - start
if remain <= 0 || remain > len(prefix) {
return false
}
for j := 0; j < remain; j++ {
if asciiLower(text[start+j]) != asciiLower(prefix[j]) {
return false
}
}
return true
}
func hasToolMarkupNamePrefix(text string, start int) bool {
for _, name := range toolMarkupNames {
if strings.HasPrefix(lowerTail, name.raw) || strings.HasPrefix(name.raw, lowerTail) {
if hasASCIIPrefixFoldAt(text, start, name.raw) {
return true
}
if hasASCIIPartialPrefixFoldAt(text, start, name.raw) {
return true
}
}
return false
}
func matchToolMarkupName(lower string, start int, dsmlLike bool) (string, int) {
func matchToolMarkupName(text string, start int, dsmlLike bool) (string, int) {
for _, name := range toolMarkupNames {
if name.dsmlOnly && !dsmlLike {
continue
}
if strings.HasPrefix(lower[start:], name.raw) {
if hasASCIIPrefixFoldAt(text, start, name.raw) {
return name.canonical, len(name.raw)
}
}

View File

@@ -949,6 +949,38 @@ func TestSkipXMLIgnoredSectionBoundaryConditions(t *testing.T) {
}
}
func TestSkipXMLIgnoredSectionCommentWithUnicodeKeepsByteOffset(t *testing.T) {
text := "<!-- İ -->x<tool_calls>"
next, adv, blk := skipXMLIgnoredSection(text, 0)
if blk || !adv {
t.Fatalf("skipXMLIgnoredSection() = (%d, %v, %v), want advanced unblocked comment", next, adv, blk)
}
if want := len("<!-- İ -->"); next != want {
t.Fatalf("skipXMLIgnoredSection() next = %d, want %d", next, want)
}
}
func TestSkipXMLIgnoredSectionMatchesCDATAWithoutAllocatingTail(t *testing.T) {
text := "<![cDaTa[<tool_calls>]]><tool_calls>"
next, adv, blk := skipXMLIgnoredSection(text, 0)
if blk || !adv {
t.Fatalf("skipXMLIgnoredSection() = (%d, %v, %v), want advanced unblocked CDATA", next, adv, blk)
}
if want := len("<![cDaTa[<tool_calls>]]>"); next != want {
t.Fatalf("skipXMLIgnoredSection() next = %d, want %d", next, want)
}
tag, ok := FindToolMarkupTagOutsideIgnored(text, 0)
if !ok {
t.Fatal("expected tool tag after skipped CDATA")
}
if tag.Start != next {
t.Fatalf("FindToolMarkupTagOutsideIgnored() start = %d, want %d", tag.Start, next)
}
}
func TestFindToolCDATAEndBoundaryConditions(t *testing.T) {
text := "<![CDATA[hello]]>"

View File

@@ -758,9 +758,11 @@ test('shouldSkipPath skips dynamic response/fragments/*/status paths only', () =
assert.equal(shouldSkipPath('response/status'), false);
});
test('node stream path guard only allows /v1/chat/completions', () => {
test('node stream path guard allows OpenAI v1 and root alias chat completions paths', () => {
assert.equal(isNodeStreamSupportedPath('/v1/chat/completions'), true);
assert.equal(isNodeStreamSupportedPath('/v1/chat/completions?x=1'), true);
assert.equal(isNodeStreamSupportedPath('/chat/completions'), true);
assert.equal(isNodeStreamSupportedPath('/chat/completions?x=1'), true);
assert.equal(isNodeStreamSupportedPath('/v1beta/models/gemini-2.5-flash:streamGenerateContent'), false);
assert.equal(isNodeStreamSupportedPath('/anthropic/v1/messages'), false);
});
@@ -768,6 +770,7 @@ test('node stream path guard only allows /v1/chat/completions', () => {
test('extractPathname strips query only', () => {
assert.equal(extractPathname('/v1/chat/completions?stream=true'), '/v1/chat/completions');
assert.equal(extractPathname('/v1beta/models/gemini-2.5-flash:streamGenerateContent?key=1'), '/v1beta/models/gemini-2.5-flash:streamGenerateContent');
assert.equal(extractPathname('/chat/completions?stream=true'), '/chat/completions');
});
test('setCorsHeaders reflects requested third-party headers and blocks internal-only headers', () => {