Skip to content

Quickstart

Get started with COSMIC in minutes.

Basic Usage

Python API

from cosmic import COSMICChunker, Document

# Initialize chunker with default configuration
chunker = COSMICChunker()

# Create document from text
doc = Document.from_text("""
Your document text here. COSMIC will analyze the structure,
detect semantic boundaries, and create coherent chunks.

This is a new paragraph about a different topic.
The chunker will identify this as a potential boundary.
""")

# Chunk with automatic strategy selection
chunks = chunker.chunk_document(doc, strategy="auto")

# Access chunk data
for chunk in chunks:
    print(f"Domain: {chunk.domain}")
    print(f"Coherence: {chunk.coherence_score:.2f}")
    print(f"Text: {chunk.text[:100]}...")
    print("---")

CLI

# Basic chunking
cosmic chunk document.txt

# Save to JSON
cosmic chunk document.txt --output chunks.json

# Use full pipeline with Ollama
cosmic chunk document.txt --strategy full --ollama auto

Strategies

COSMIC offers multiple chunking strategies:

Strategy Description Best For
auto Automatically selects based on document General use (recommended)
full Complete 6-stage pipeline Well-structured documents
semantic DCS-based boundaries only Documents with clear topics
sliding Similarity-based sliding window Speed-critical applications
fixed Token-based splitting Baseline comparisons 
# Strategy selection
chunks = chunker.chunk_document(doc, strategy="full")
chunks = chunker.chunk_document(doc, strategy="semantic")
chunks = chunker.chunk_document(doc, strategy="sliding")
chunks = chunker.chunk_document(doc, strategy="fixed")

Document Creation

from cosmic import Document

# From plain text
doc = Document.from_text(
    text="Your text content...",
    doc_id="my-document",
    metadata={"source": "user-upload"},
)

# Access properties
print(f"Document ID: {doc.id}")
print(f"Sentences: {doc.num_sentences}")

Configuration

Environment Variables

Create a .env file:

# LLM Provider
COSMIC_LLM_PROVIDER=ollama

# Embedding device
COSMIC_EMBEDDING_DEVICE=cuda  # or cpu, mps

Custom Configuration

from cosmic import COSMICChunker, COSMICConfig
from cosmic.core.config import DCSConfig, ChunkConstraints

config = COSMICConfig(
    dcs=DCSConfig(
        alpha=0.5,   # Topical coherence weight
        beta=0.3,    # Coreference weight
        gamma=0.2,   # Discourse marker weight
    ),
    chunk_constraints=ChunkConstraints(
        min_tokens=50,
        max_tokens=1024,
        target_tokens=512,
    ),
)

chunker = COSMICChunker(config=config)

Load from YAML

config = COSMICConfig.from_yaml("configs/custom.yaml")
chunker = COSMICChunker(config=config)

Output Format

Each chunk contains rich metadata:

chunk.chunk_id          # Unique identifier
chunk.text              # Chunk content
chunk.token_count       # Token count
chunk.domain            # Classified domain
chunk.coherence_score   # Internal coherence (0-1)
chunk.cross_references  # Related chunk IDs
chunk.intent            # Primary intent (define, explain, etc.)

Next Steps