---
title: chunker | CodeWeaver Docs
description: API reference for codeweaver.engine.config.chunker
url: "https://docs.knitli.com/api/engine/config/chunker"
type: static
generatedAt: "2026-04-17T17:21:08.469Z"
---

# chunker
       [Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F.%20I%20want%20to%20ask%20questions%20about%20it.)[Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F.%20I%20want%20to%20ask%20questions%20about%20it.)[View in Markdown](/codeweaver/api/engine/config/chunker.md)       [Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F)[Share on X](https://x.com/intent/tweet?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F&text=chunker)[Share on Threads](https://threads.net/intent/post?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F&text=chunker)[Share on Bluesky](https://bsky.app/intent/compose?text=chunker%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F)[Share on Reddit](https://reddit.com/submit?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F&title=chunker)[Share on Hacker News](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F&t=chunker)[Share on Email](mailto:?subject=chunker&body=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F)[Share on WhatsApp](https://wa.me/?text=chunker%20https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F)[Share on Telegram](https://t.me/share/url?url=https%3A%2F%2Fdocs.knitli.com%2Fcodeweaver%2Fapi%2Fengine%2Fconfig%2Fchunker%2F&text=chunker)
# `codeweaver.engine.config.chunker`
[Section titled “codeweaver.engine.config.chunker”](#codeweaverengineconfigchunker)
Configuration models for user-defined languages and delimiters in CodeWeaver.

## Class: `ChunkerSettings`
[Section titled “Class: ChunkerSettings”](#class-chunkersettings)
Configuration for chunker system.

You can use these settings to customize how CodeWeaver chunks files for indexing, and to add support for custom languages or delimiters.

Note: If you’re adding support for another language, we’d love you to open a pull request to add it to our built-in language definitions! See `codeweaver.core` and `codeweaver.engine` for more details.

### Method: `ensure_models_rebuilt`
[Section titled “Method: ensure_models_rebuilt”](#method-ensure_models_rebuilt)

**

```
ensure_models_rebuilt()
```

Ensure forward references are resolved before first use.

This method must be called at module level after all class definitions to resolve forward references to LanguageFamily and DelimiterPattern types used in CustomLanguage and CustomDelimiter models.

### Method: `model_post_init`
[Section titled “Method: model_post_init”](#method-model_post_init)

**

```
model_post_init()
```

Post-initialization hook.

## Class: `ChunkerSettingsDict`
[Section titled “Class: ChunkerSettingsDict”](#class-chunkersettingsdict)
TypedDict for Chunker settings.

Not intended to be used directly; used for internal type checking and validation.

## Class: `ConcurrencySettings`
[Section titled “Class: ConcurrencySettings”](#class-concurrencysettings)
Concurrency configuration.

## Class: `ConcurrencySettingsDict`
[Section titled “Class: ConcurrencySettingsDict”](#class-concurrencysettingsdict)
TypedDict for concurrency settings.

Not intended to be used directly; used for internal type checking and validation.

## Class: `CustomDelimiter`
[Section titled “Class: CustomDelimiter”](#class-customdelimiter)
A custom delimiter for separating multiple prompts in a single input string. If you only want to define a new language and extensions but not a delimiter, use the `CustomLanguage` class instead.

### Method: `validate_instance`
[Section titled “Method: validate_instance”](#method-validate_instance)

**

```
validate_instance()
```

Validate the instance after initialization.

## Class: `CustomLanguage`
[Section titled “Class: CustomLanguage”](#class-customlanguage)
A custom programming language for language specific parsing.

By default, CodeWeaver only indexes extensions it recognizes. There are a lot (~170 languages and 200+ extensions) but not everything. If you want it to index files with extensions it doesn’t recognize, you can define a custom language here. You only need to do this if you **don’t** want to define a custom delimiter for your language. CodeWeaver will try to detect the best chunking strategy for your language, and will probably do a decent job, but if you want to define custom delimiters, use the `CustomDelimiter` class instead.

## Class: `PerformanceSettings`
[Section titled “Class: PerformanceSettings”](#class-performancesettings)
Performance and resource limit configuration.

## Class: `PerformanceSettingsDict`
[Section titled “Class: PerformanceSettingsDict”](#class-performancesettingsdict)
TypedDict for performance settings.

Not intended to be used directly; used for internal type checking and validation.