DefaultTextChunker class

A default text chunking implementation using native C++ Wasm methods.

Signature:

export declare class DefaultTextChunker implements TextChunker 

Implements: TextChunker

Methods

Method Modifiers Description
chunk(text) Chunks input text according to configured size and overlap.
create(wasmFileset, chunkSize, chunkOverlap, mode) static Initializes the Wasm runtime and creates a new DefaultTextChunker instance.
createFromModule(wasmModule, chunkSize, chunkOverlap, mode) static Creates a DefaultTextChunker from an already initialized Wasm module.

DefaultTextChunker.chunk()

Chunks input text according to configured size and overlap.

Signature:

chunk(text: string): string[];

Parameters

Parameter Type Description
text string

Returns:

string[]

DefaultTextChunker.create()

Initializes the Wasm runtime and creates a new DefaultTextChunker instance.

Signature:

static create(wasmFileset: WasmFileset, chunkSize?: number, chunkOverlap?: number, mode?: ChunkingMode): Promise<DefaultTextChunker>;

Parameters

Parameter Type Description
wasmFileset WasmFileset
chunkSize number
chunkOverlap number
mode ChunkingMode

Returns:

Promise<DefaultTextChunker>

DefaultTextChunker.createFromModule()

Creates a DefaultTextChunker from an already initialized Wasm module.

Signature:

static createFromModule(wasmModule: WasmModule, chunkSize?: number, chunkOverlap?: number, mode?: ChunkingMode): DefaultTextChunker;

Parameters

Parameter Type Description
wasmModule WasmModule
chunkSize number
chunkOverlap number
mode ChunkingMode

Returns:

DefaultTextChunker