Document Uploads & Processing

Upload proprietary documents directly into your assistant's knowledge base with automated text extraction, chunking, and embedding.

WebChat AI Knowledge Base Document Management — Manage uploaded documents, inspect chunk counts, review processing status, and trigger retries.

File formats & upload constraints

Limits enforced by the backend ingestion pipeline.

Document upload limits
Constraint	Limit	Details
Supported formats	.pdf, .docx, .md, .txt	Validated via magic bytes and MIME types
Max files per upload batch	5 files	Enforced on POST /api/knowledge/.../upload
Max file size	10 MB	Applies per file and per total upload batch
PDF page ceiling	100 pages	Larger PDFs must be split prior to upload
Encrypted / Password PDFs	Rejected	DocumentPasswordProtectedError returned
Extracted characters range	50 to 500,000 chars	Scanned image-only PDFs require OCR first
Chunk token target	500–800 tokens	100-token overlap between adjacent chunks

The ingestion pipeline

How files are transformed into searchable vector representations.

Upload Processing Flow

1. VALIDATION

File & Magic Bytes

Checks file extension and binary magic bytes (%PDF-, PK\x03\x04).

2. EXTRACTION

Parser Extraction

pypdf for PDF, python-docx for DOCX, native utf-8 decoder for TXT/MD.

3. CHUNKING

Semantic Slicing

500–800 tokens per chunk with 100-token overlap along sentence boundaries.

Document lifecycle statuses

Understand the status indicators shown on the Knowledge Base table.

pendingFile has been uploaded and queued for background text extraction.

processingText has been extracted and is actively undergoing chunking and vector embedding generation.

completedAll chunks have been embedded and stored in the tenant vector index. Ready for retrieval.

failedProcessing failed (e.g. corruption, password protection, or rate limit). Manual retry is available.

rate_limitedEmbedding API quota was temporarily reached. Will retry automatically with exponential backoff.

Retrying & deleting documents

How to manage knowledge over time.

Retrying a failed document

If an external embedding provider was temporarily unavailable or hit rate limits, click the Retry button next to the document in the dashboard, or call:

POST /api/knowledge/documents/{documentId}/retry

Deleting a document

Deleting a document permanently removes the document record, all associated vector chunk embeddings from the vector store, and any attached file storage. The assistant immediately stops referencing that material in future chats.