Handles cases where a stray backtick opens an inline code span but is never closed.
Previously, any subsequent XML tool tag was treated as inside markdown code and ignored.
Now, tool tags are detected after an unclosed backtick, and the markdown state is reset
when the backtick is confirmed to be literal text at stream boundaries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR #460 introduced fullwidth pipe characters (｜) in DSML tool call formatting
to improve parsing robustness, but models exposed to these fullwidth pipes in
system prompts exhibit significantly higher rates of tool output hallucinations.
Reverting to halfwidth pipes (|) drastically reduces tokenizer/perplexity-driven
hallucinations while retaining the existing confusable-hardening in the parser.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Detect camelCase→PascalCase boundaries between arbitrary prefixes and fixed
local names (tool_calls/invoke/parameter), so that fused forms like
<DSmartToolCalls> are recognized without explicit separator characters.
Also add the underscore-free alias "toolcalls" as a valid DSML local name.
Includes lookalike rejection tests to ensure near-matches like
<DSmartToolCallsExtra> are not falsely accepted.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the hardcoded isToolMarkupPipe (matching |, ｜, ␂, \x02, !) and
isToolCDATAOpenSeparator (exclusion-based) with a single isToolMarkupSeparator
that treats any Unicode punctuation outside structural characters as a valid
DSML separator. This eliminates the need for a per-character allowlist — novel
separators like ※ are automatically supported without code changes. Also
removes the unused cdataPattern regexp and updates docs to use "non-structural
separator" terminology.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add compatibility for <dsml-tool-calls>/<dsml-invoke>/<dsml-parameter>
tag forms alongside the canonical pipe-prefixed DSML shell. Hyphenated
forms only activate when a DSML prefix is detected, preventing false
matches on bare XML lookalikes. Go and Node parsers aligned, with tests
covering here-doc CDATA, streaming sieve, and negative lookalike cases.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace hardcoded DSML typo variant lists in Go/Node tool call parsers with
generalized prefix consumption that tolerates repeated leading <, repeated DSML
prefix noise, and trailing pipe terminators. Split tiktoken-dependent token
counting into a build-tagged file for non-cgo platform compatibility. Add /data
directory to Dockerfile for bind-mount permissions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add shared ToolMarkupTag scanner (toolcalls_scan.go) replacing hardcoded alias tables
- Support DSML collapsed tag names (<DSMLtool_calls>, <DSMLinvoke>, <DSMLparameter>)
- Parse JSON literal values from parameter bodies (123→number, true→bool, null)
- Recover unclosed CDATA in final parse/flush via SanitizeLooseCDATA
- Align Go and Node implementations (scanToolMarkupTagAt, findMatchingToolMarkupClose)
- Reject bare <invoke> as unsupported syntax, only tool_calls wrapper triggers tool path
- Update API.md and toolcall-semantics.md documentation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Guard current_input_file.enabled / thinking_injection.{enabled,prompt} with hasNestedSettingsKey so partial updates don't overwrite omitted fields
- Expand DSML alias support to tolerate space-separated tags (e.g. <|dsml invoke>) alongside pipe-separated forms
- Sync Go sieve, Node sieve, toolcall parser, and tests for all new DSML variants
- Update API.md and toolcall-semantics.md with expanded alias coverage
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add support for DSML wrapper aliases (<dsml|tool_calls>, <|tool_calls>,
<｜tool_calls>) alongside canonical XML. Normalize mixed DSML/canonical
tags instead of rejecting them. Add tilde fence (~~~) support, fix
nested fence and unclosed fence handling, support CDATA-protected fence
content, and skip prose mentions when scanning for real tool blocks.
Mirror all changes between Go and Node.js runtimes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
