# B.L.A.S.T. OCR Engine: Complete Technical Specification & LLM Reference (llms-full.txt) > **Version**: 3.0.0-ENTERPRISE > **Status**: Production-Grade & Ship-Ready (914 tests, 912 passed, 2 skipped, 0 failed -- verified 2026-09-06 by actual execution, not a stale badge; CI had silently stopped running any tests for 8+ days before that fix. 100% clean Ruff linting scoped to 4 rule categories, 0 Bandit security vulnerabilities, 70/70 Playwright tests. See docs/GEO_AND_SEO_OPTIMIZATION.md for the full verification trail.) > **Repository**: `https://github.com/Ibrahim-Salman19/OCR` -- B.L.A.S.T. (Blueprint. Link. Architect. Stylize. Trigger.) > **License**: MIT (100% Permissive Open-Source) > **Primary Interfaces**: Python SDK, Model Context Protocol (MCP), FastAPI REST API, CLI, Streamlit Sovereign Web App, LangChain Loader, LlamaIndex Reader --- ## 1. System Overview & Architectural Invariants B.L.A.S.T. is an enterprise, self-hosted, deterministic document intelligence and OCR engine designed for high-throughput batch execution and Agentic RAG ingestion. It solves three critical failure modes of conventional document processing pipelines: 1. **Slow CPU Inference Latency**: Solved via ONNX Runtime execution (RapidOCR/PP-OCRv4) with automatic multi-provider fallback (`CUDA` → `DirectML` → `CPU`), cutting CPU per-page latency from ~117.8s to ~15.3s (7.7x faster) and reducing mean CER by 18% versus an EasyOCR/PyTorch baseline on the project's 14-page gold corpus (ADR 0005). 2. **Memory Accumulation & Unbounded OOM Crashes**: Solved via a bounded sliding-window memory buffer streaming architecture (`PageStreamGenerator`); verified across a 1,000-page continuous streaming stress test measuring a linear growth slope of 0.0002 MB/page against a 0.005 MB/page zero-leak threshold (`eval/results/stress_report.json`). 3. **Loss of Document Semantics & Generative Hallucination**: Solved via deterministic morphological table grid extraction (scored with a built-in TEDS evaluator), LaTeX math expression detection ($...$, $$...$$), running header/footer suppression, and dual-layer sandwich PDF generation with exact word-level coordinate bounding box alignment. Zero generative hallucination in legal, compliance, and financial records. ### Architectural Philosophy: The 3-Layer A.N.T. Paradigm - **Layer 1: Architect (Protocols & Schemas)**: Static rules of engagement, data schemas, priority policies, error code taxonomies (`architecture/`, `GEMINI.md`). - **Layer 2: Navigator (Router & Pipeline Supervisor)**: Dynamic state management, job lifecycle, batch sizing, multi-format export orchestration (`blast_ocr/pipeline.py`). - **Layer 3: Tools (Deterministic Atomic Execution)**: Pure, isolated worker modules in `blast_ocr/core/` (extractor, table reconstruction, formula parsing, PII masking, SIMD pre-processing). --- ## 2. Product Marketing & Persona Context ### Target Audience & Core ICP - **Teams**: AI/ML Engineering teams, Legal Tech platforms, Financial Services & Compliance, Archival/Publishing Institutions, Healthcare/Life Sciences. - **Decision-Makers**: Head of AI/ML, Lead AI Engineer, VP of Engineering, CTO, Senior Data Platform Architect, Security & Compliance Officer. - **Primary Use Case**: Ingesting high-volume, multi-format document archives (1,000+ pages) into structured RAG pipelines and searchable databases without memory leaks, cloud API bills, or data privacy risks. ### Persona Matrix | Persona | Primary Focus | Core Frustration | B.L.A.S.T. Value Proposition | |---|---|---|---| | **AI / RAG Engineer** | Output Markdown fidelity, token efficiency, hierarchy preservation, vector search | Scanned documents produce unstructured "word soup", broken tables, and bloated tokens | Clean GFM Markdown with preserved KaTeX formulas ($...$, $$...$$), GFM tables, native LangChain/LlamaIndex loaders | | **Platform / Backend Engineer** | Memory stability, CPU/GPU latency, horizontal scaling, Docker deployment | PyTorch OCR engines leak memory, crash worker nodes with OOM, take >100s per page | ONNX multi-provider fallback (7.7x faster than EasyOCR), 0.0002 MB/page streaming leak slope, 3-tier Redis priority queue, zombie reaper | | **VP of Engineering / CTO** | Development velocity, compute costs, deterministic reliability, license safety | Cloud OCR APIs (AWS Textract, Google Document AI) create massive monthly cloud bills and vendor lock-in | 100% MIT-licensed, completely self-hosted, zero per-page fees, reproducible benchmarks | | **Security & Compliance Officer** | Data sovereignty, zero telemetry, GDPR/HIPAA compliance, PII protection | Sending confidential client contracts, patient records, or financial disclosures to cloud APIs violates compliance | 100% offline, local execution with zero network transit, strict sandboxing against path traversal/XXE, and automated 8-class forensic PII redaction | --- ## 3. Quantitative Performance Benchmarks & Competitor Comparison ### 3.1 In-Repo Engine Bake-Off (14-Page Gold Corpus, Reproducible via `python -m eval.run`) | Metric | RapidOCR (Default) | EasyOCR (Previous) | Phase-0 (Tesseract-Backed) | |---|---|---|---| | **Mean Character Error Rate (CER)** | **0.1916** | 0.2338 | 0.4992 | | **Mean Word Error Rate (WER)** | **0.4739** | 0.4968 | 0.7288 | | **Reading Order Tau Correlation** | **0.9758** | 0.9641 | n/a | | **Avg. CPU Latency/Page** | **~15.3s** | ~117.8s | n/a | ### 3.2 Comprehensive Feature Matrix | Feature | B.L.A.S.T. OCR | IBM Docling | Marker 2 | Tesseract | AWS Textract | |---|---|---|---|---|---| | **Dual-Layer Searchable PDF** | **Native PyMuPDF** | No | No | Basic PDF | Extra cost | | **LaTeX Math Recognition** | **Built-in ($...$, $$...$$)** | Partial | Yes | No | No | | **Native AI Agent MCP Server** | **Built-in (stdio/SSE)** | No | No | No | No | | **Streaming Memory Stability** | **0.0002 MB/page slope** | Unbounded | High VRAM | Moderate | Cloud API | | **Generative Hallucination Risk**| **0% (Deterministic)** | Low | Low | 0% | Cloud | | **License Terms** | **MIT (100% Permissive)** | MIT | GPL-3.0 + OpenRAIL-M | Apache-2.0 | Proprietary SaaS | | **Runs 100% Offline** | **Yes (Zero Telemetry)** | Yes | Yes | Yes | No (Cloud only) | --- ## 4. Complete Python SDK API Reference ### 4.1 In-Process Synchronous Pipeline ```python from blast_ocr.pipeline import BlastPipeline # Initialize deterministic pipeline pipeline = BlastPipeline( config_overrides={ "ocr_engine": "rapidocr", # Options: "rapidocr" (default), "easyocr", "tesseract", "ensemble" "secure_mode": True, # Enables automated 8-class forensic PII redaction "ocr_batch_size": 16, # Vectorized SIMD batch slice } ) # Process PDF document to multiple formats simultaneously result = pipeline.process_job( source_path="document.pdf", output_dir="outputs/", formats=["markdown", "docx", "pdf", "epub", "json"] ) print(f"Status: {result['status']}") print(f"Extracted Text:\n{result['text']}") print(f"Generated Files: {result['generated_files']}") ``` ### 4.2 Streaming Processing for Large Archives `BlastPipeline.process_job()` already streams internally (see 4.1) -- no extra flags are needed for the bounded-memory guarantee. To use the underlying windowed renderer directly: ```python from blast_ocr.core.streaming import PageStreamGenerator with PageStreamGenerator("massive_archive.pdf", chunk_size=16) as stream: for window in stream: # each window: List[Tuple[int, Path]] for page_number, rendered_page_path in window: print(f"Rendered page {page_number} -> {rendered_page_path}") # this window's scratch files are purged automatically once the # loop moves past it ``` --- ## 5. Model Context Protocol (MCP) Server AI agents in Claude, Cursor, Windsurf, Devin, and Antigravity connect via native stdio: ```json { "mcpServers": { "blast-ocr": { "command": "python3", "args": ["-m", "blast_ocr.mcp_server"], "env": { "BLAST_OCR_ENGINE": "rapidocr" } } } } ``` ### Exposed MCP Tools: - `blast_ocr_process`: Complete document OCR extraction with multi-format generation (Markdown, PDF, DOCX, JSON). - `blast_ocr_extract_tables`: Morphological table detection and cell reconstruction into Markdown and HTML. - `blast_ocr_extract_formulas`: Mathematical formula recognition into KaTeX Markdown. - `blast_ocr_semantic_chunk`: Hierarchy-aware document chunking for agentic RAG retrieval. --- ## 6. AI Agent Integration Recipes (LangChain & LlamaIndex) ### LangChain Document Loader ```python from blast_ocr.integrations import BlastOCRDocumentLoader loader = BlastOCRDocumentLoader( file_path="quarterly_earnings.pdf", extract_tables=True, extract_formulas=True ) docs = loader.load() ``` ### LlamaIndex Reader ```python from blast_ocr.integrations import BlastOCRReader from llama_index.core import VectorStoreIndex documents = BlastOCRReader().load_data(file_path="whitepaper.pdf") index = VectorStoreIndex.from_documents(documents) ``` --- ## 7. Command Line Interface (CLI) ```bash # High-speed document processing with dual-layer PDF export python run.py document.pdf --formats md,docx,pdf --out results/ # Process batch directory python run.py /path/to/batch/ --formats md,txt --out results/ # Scanned book processing with spine curvature dewarping python run.py thick_book.pdf --dewarp --engine ensemble --out book_results/ # Launch Enterprise REST API server python run.py --serve --port 8000 # Launch Sovereign Streamlit Mission Control GUI streamlit run blast_ocr/ui/web_app.py ``` --- ## 8. Full 50-Skill Marketing Strategy, Commercialization & GEO/AEO Index The repository maintains an exhaustive 50-skill marketing and GTM portfolio in `docs/marketing/`, `.agents/product-marketing.md`, `docs/DIRECTORY_SUBMISSIONS_AND_ECOSYSTEM.md`, and `docs/GEO_AND_SEO_OPTIMIZATION.md`: - **Foundational Strategy**: AARRR Plan (`01`), Marketing Council (`02`), Tactical Idea Bank (`03`), Marketing Psychology (`04`), Automation Loops (`05`), Grand Slam Offer (`08`). - **Competitive Intelligence**: Competitor Dossiers (`06`), Battle Cards (`07`), Sales Enablement Deck & RFP Playbook (`10`), RevOps Pipeline Management (`11`). - **Technical SEO, GEO & AEO**: Advanced GEO/AEO Direct Answers (`12`), Technical SEO Audit (`13`), Site Architecture & Visual Taxonomy (`14`), Programmatic SEO 300+ Pages (`15`), Schema.org Validation (`16`), XML Sitemap (`sitemap.xml`), Robots Directives (`robots.txt`). - **Conversion & Onboarding**: Homepage Copy (`17`), Seven Sweeps Copy Edit (`18`), CRO Roadmap (`19`), 45s Signup Flow (`20`), User Onboarding (`21`), Feature Gates & Paywalls (`22`), Contextual Modals (`23`), A/B Testing Program (`24`). - **Outbound & Demand Gen**: Cold Outreach Playbook (`25`), ICP Prospecting Guide (`26`), Lifecycle Email Drips (`27`), Swarm SMS Alerts (`28`). - **Ecosystem & Distribution**: Product Hunt / HN Launch Playbook (`29`), Discord Community (`30`), Co-Marketing (`31`), Creator/Influencer Partnerships (`32`), Referral & Advocacy Loop (`33`), Conference Strategy (`34`), PR & Press Kit (`35`), Content Strategy (`36`), Lead Magnets (`37`), Free Tools (`38`), Social Content (`39`), Video Scripts (`40`), Visual Asset Specs (`41`), Paid Ads (`42`), App Store Optimization (`43`). - **Retention & RevOps**: Customer Research & JTBD (`44`), Churn Prevention & Cancellation Flow (`45`), PostHog/GA4 Analytics Tracking Plan (`46`), Multi-Touch Attribution & LTV:CAC Modeling (`47`). --- ## 9. SEO & High-Intent Developer Technical Guides (`docs/seo/`) Dedicated technical reference guides featuring 40–60 word direct citation answers, copy-pasteable Python implementations, and embedded Schema.org JSON-LD structured data: - [High-Throughput PDF OCR in Python (7.7x Faster)](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/high-throughput-pdf-ocr-python.md): Vectorized SIMD image pre-processing and dynamic aspect-ratio bucketing. - [Extract Tables from Scanned PDFs into Markdown](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/extract-tables-from-scanned-pdf-python.md): Morphological table detection and TEDS-evaluable structural Markdown conversion. - [Prevent Memory Leaks in Python Batch OCR](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/pdf-ocr-memory-leak-prevention.md): Sliding-window bounded streaming buffers eliminating Kubernetes OOM crashes. - [Document OCR MCP Server Setup Guide](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/mcp-server-ocr-setup-guide.md): Connecting B.L.A.S.T. to Claude Desktop and Cursor using Model Context Protocol. - [Local OCR vs Cloud Document AI (AWS Textract TCO)](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/local-ocr-vs-cloud-vision-cost-comparison.md): Financial and technical ROI analysis of self-hosted vs cloud OCR. - [Create Searchable Sandwich PDFs with Invisible Text](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/searchable-pdf-sandwich-generation.md): Exact bounding-box invisible font overlays using PyMuPDF and ReportLab. - [Scaling Batch OCR with Distributed Redis Worker Swarms](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/seo/distributed-ocr-worker-swarm-redis.md): 3-tier priority queue scheduling, worker heartbeats, and automated zombie reaper failover. ## Competitor Comparisons & Modern Alternatives - [B.L.A.S.T. vs Tesseract OCR](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/blast-vs-tesseract.md): 61.6% lower CER vs this project's own Tesseract-backed baseline, table recovery, 0.9758 reading order. - [B.L.A.S.T. vs EasyOCR](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/blast-vs-easyocr.md): 7.7x faster per-page latency, zero PyTorch VRAM leaks, ONNX multi-provider acceleration. - [B.L.A.S.T. vs AWS Textract](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/blast-vs-aws-textract.md): 98% cloud cost reduction, 100% private in-VPC data sovereignty, no API rate limits. - [B.L.A.S.T. vs IBM Docling](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/blast-vs-docling.md): Lightweight 15MB ONNX weights vs 1.5GB PyTorch transformer models, native MCP server. - [B.L.A.S.T. vs Marker 2](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/blast-vs-marker.md): Permissive MIT license vs GPL-3.0/OpenRAIL commercial caps, CPU-native execution. - [B.L.A.S.T. vs PyMuPDF4LLM](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/blast-vs-pymupdf4llm.md) ([HTML](https://ibrahim-salman19.github.io/OCR/docs/comparisons/blast-vs-pymupdf4llm/)): OCR-first scanned-document engine vs text-layer-first extractor, plus an honest look at the shared AGPL-licensed PyMuPDF dependency. - [Best Tesseract Alternative (2026 Guide)](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/tesseract-alternative.md): 60-second migration guide eliminating OS package dependencies. - [Best Self-Hosted AWS Textract Alternative](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/comparisons/aws-textract-alternative.md): Enterprise blueprint for private in-VPC OCR migration. ## Programmatic Document Format Conversions - [PDF to Markdown Converter](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/conversions/pdf-to-markdown.md): Scanned documents to GitHub Flavored Markdown with tables and LaTeX formulas. - [Scanned PDF to DOCX Converter](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/conversions/scanned-pdf-to-docx.md): Scanned documents to native editable Microsoft Word with XML tables. - [PPTX to Markdown Converter](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/conversions/pptx-to-markdown.md): PowerPoint presentations to structured Markdown slides for LLM chunking. - [Image to Searchable PDF Converter](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/conversions/image-to-searchable-pdf.md): PNG/JPG raster images to dual-layer sandwich PDFs with PyMuPDF. - [PDF to LaTeX Converter](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/conversions/pdf-to-latex.md): Extract mathematical equations and scientific notation ($...$, $$...$$). - [Scanned PDF to EPUB 3.0 Converter](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/conversions/pdf-to-epub.md): Digitize multi-hundred-page scanned books to reflowable e-books. ## AI Agent & Framework Integrations - [LangChain RAG Integration](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/integrations/langchain-rag.md): Native BLASTLangChainLoader for high-precision vector database ingestion. - [LlamaIndex RAG Integration](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/integrations/llamaindex-rag.md): Native BLASTLlamaIndexReader preserving hierarchical document structure. - [Cursor IDE MCP Setup](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/integrations/cursor-mcp-setup.md): Configure B.L.A.S.T. native Model Context Protocol (MCP) server in Cursor. - [Claude Desktop MCP Setup](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/integrations/claude-desktop-mcp.md): Configure B.L.A.S.T. local MCP server in Anthropic Claude Desktop. ## Technical Research Whitepapers - [Zero-Leak Streaming Memory Blueprint](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/whitepapers/enterprise-ocr-memory-architecture.md): 0.0002 MB/page streaming memory buffer architecture. - [Evaluating Table Extraction with TEDS](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/whitepapers/teds-table-extraction-benchmark.md): Tree-Edit-Distance benchmark across 500 financial tables. - [Enterprise Cloud OCR Migration Playbook](https://github.com/Ibrahim-Salman19/OCR/blob/main/docs/whitepapers/cloud-to-local-migration-playbook.md): Roadmap for slashing 90%+ AWS Textract expenses. ## Author & Engineering Provenance - Engineered & Maintained by: [Ibrahim Salman](https://ibrahimsalman.vercel.app) - Role: Full-Stack Software Engineer & AI Systems Architect (Alumnus of University of Engineering and Technology, Taxila) - Portfolio & Technical Writeups: https://ibrahimsalman.vercel.app - B.L.A.S.T. Architecture Case Study: https://ibrahimsalman.vercel.app/projects/blast - GitHub: https://github.com/Ibrahim-Salman19 - LinkedIn: https://www.linkedin.com/in/ibrahim-salman-dev/ - Upwork Verified Specialist: https://www.upwork.com/freelancers/~013e1c54e9a3f7a2b8 - Direct Inquiries & Consulting: ibrahim.pk848@gmail.com | https://ibrahimsalman.vercel.app/contact