B.L.A.S.T. OCR
Code Executed & Verified, 2026-09-10

How to Prevent Memory Leaks in Python Batch OCR Pipelines

Primary query: python ocr memory leak · raw markdown

How can I prevent memory leaks when running batch OCR in Python?

Memory leaks in Python batch OCR are prevented with a sliding-window bounded streaming buffer and scratch-file recycling. B.L.A.S.T. measures a memory growth slope of ≤ 0.0002 MB/page across a 1,000-page continuous streaming test, capping RAM at a fixed ceiling regardless of document length. Verified in eval/stress_test.py.

CLI Quickstart

git clone https://github.com/Ibrahim-Salman19/OCR.git && cd OCR
    pip install -r requirements.txt
    # Bounded-memory streaming is the pipeline's default behavior, not an opt-in flag
    python -m blast_ocr.cli large_book_1000_pages.pdf --formats md

Python implementation: sliding-window bounded buffer

The recommended path is BlastPipeline.process_job() — it renders and OCRs pages through the windowed generator below internally, so no extra flags are needed for the bounded-memory guarantee:

from blast_ocr.pipeline import BlastPipeline

    pipeline = BlastPipeline(config_overrides={"ocr_engine": "rapidocr"})
    result = pipeline.process_job(source_path="massive_archive.pdf", formats=["markdown"])
    print(f"{result['pages_processed']} pages processed, status={result['status']}")

For the mechanism itself: PageStreamGenerator renders pages in fixed-size windows (default 8), yields each window as a list of (page_number, rendered_image_path) tuples, and deterministically unlinks that window's scratch files once the caller moves past it — so RSS is bounded by one window's worth of rendered pages, not the whole document:

from blast_ocr.core.streaming import PageStreamGenerator

    with PageStreamGenerator("massive_archive.pdf", chunk_size=16) as stream:
        for window in stream:  # each window: List[Tuple[int, Path]]
            for page_number, rendered_page_path in window:
                print(f"Rendered page {page_number} -> {rendered_page_path}")
            # this window's scratch files are purged automatically once the
            # loop moves past it

The memory slope leak regression

Pipeline Configuration100 Pages RAM500 Pages RAM1,000 Pages RAMOOM Failure Mode
B.L.A.S.T. Bounded Buffer48.2 MB51.4 MB52.1 MBZero Crashes (Stable)
PyTesseract / C++ Pipe92.0 MB318.0 MB740.0 MBPod Killed (OOMKilled)
JaidedAI EasyOCR (Torch)450.0 MB1,820.0 MBCrashesVRAM Exhaustion

Source: eval/stress_test.py and eval/results/stress_report.json (1,000-page streaming stress run, 0.005 MB/page fail threshold).

Author & Engineering Authority

Engineered & maintained by Ibrahim Salman, Full-Stack Software Engineer & AI Systems Architect (UET Taxila).