---
title: delimiter | CodeWeaver Docs
description: API reference for codeweaver.engine.chunker.delimiter
url: "https://docs.knitli.com/api/engine/chunker/delimiter"
type: static
generatedAt: "2026-04-17T17:21:08.367Z"
---

# delimiter
       [Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F.%20I%20want%20to%20ask%20questions%20about%20it.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F.%20I%20want%20to%20ask%20questions%20about%20it.)[View in Markdown](/codeweaver/api/engine/chunker/delimiter.md)       [Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F)[Share on X](https://x.com/intent/tweet?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F&text=delimiter)[Share on Threads](https://threads.net/intent/post?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F&text=delimiter)[Share on Bluesky](https://bsky.app/intent/compose?text=delimiter%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F)[Share on Reddit](https://reddit.com/submit?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F&title=delimiter)[Share on Hacker News](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F&t=delimiter)[Share on Email](mailto:?subject=delimiter&body=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F)[Share on WhatsApp](https://wa.me/?text=delimiter%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F)[Share on Telegram](https://t.me/share/url?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fchunker%2Fdelimiter%2F&text=delimiter)
# `codeweaver.engine.chunker.delimiter`
[Section titled “codeweaver.engine.chunker.delimiter”](#codeweaverenginechunkerdelimiter)
Delimiter-based code chunking implementation.

Implements pattern-based chunking using delimiter pairs (e.g., braces, parentheses). Uses a three-phase algorithm: match detection, boundary extraction with nesting support, and priority-based overlap resolution.

## Class: `DelimiterChunker`
[Section titled “Class: DelimiterChunker”](#class-delimiterchunker)
Pattern-based chunker using delimiter pairs.

Extracts code chunks based on delimiter patterns (braces, parentheses, etc.) with support for nesting and priority-based overlap resolution.

Algorithm: Phase 1: Match Detection - Find all delimiter occurrences Phase 2: Boundary Extraction - Match starts with ends, handle nesting Phase 3: Priority Resolution - Keep highest-priority non-overlapping boundaries

Attributes: _delimiters: List of delimiter patterns for the target language _language: Programming language being processed

Example: >>> from codeweaver.engine.chunker.base import ChunkGovernor >>> governor = ChunkGovernor(chunk_limit=1000) >>> chunker = DelimiterChunker(governor, language=“python”) >>> chunks = chunker.chunk(“def foo():\n pass”)

### Method: `chunk`
[Section titled “Method: chunk”](#method-chunk)

**

```
chunk()
```

Chunk content using delimiter patterns.

Implements complete delimiter chunking with edge case handling and size constraint enforcement.

Args: content: Source code to chunk file: Optional DiscoveredFile with metadata and source_id context: Optional additional context

Returns: List of CodeChunk objects

Raises: BinaryFileError: If content contains binary data ChunkLimitExceededError: If chunk count exceeds governor limit OversizedChunkError: If individual chunks exceed token limit ParseError: If delimiter matching fails

## Class: `StringParseState`
[Section titled “Class: StringParseState”](#class-stringparsestate)
State for tracking string boundaries during parsing.

Attributes: in_string: Whether currently inside a string literal delimiter: The string delimiter character (’”’, ”’”, or ’`’), or None if not in string