llm-ocr/README.md

OCR

Offline OCR tool based on llama.cpp and its multimodal extension (mmproj). Supports GLM-OCR and other vision‑language models.

Requirements

• C++17 compiler
• CMake ≥ 3.15
• llama.cpp libraries: libllama.so, libmtmd.so, libggml*.so
• llama.cpp headers: llama.h, mtmd.h, mtmd-helper.h, ggml.h
• (Optional) CUDA 12.2 runtime for GPU acceleration

Runtime Dependencies

Feature	Tool
File input	—
Clipboard	wl-paste (Wayland) / xclip (X11)
Screenshot	gnome-screenshot, flameshot, spectacle, maim, or slurp+grim

Build

cmake -S . -B build
cmake --build build

Custom dependency paths

The build expects the llama.cpp libraries and headers under deps/llama.cpp/. To use a different location, edit DEPS_DIR in CMakeLists.txt.

Usage

ocr -m <model.gguf> --mmproj <mmproj.gguf> [options] [<image_path>]

Options

Flag	Description
-m, --model	Model file path (GGUF)
--mmproj	Multimodal projection file path (GGUF)
-p	OCR prompt (default: OCR)
-t	Thread count (default: CPU cores)
--ngl	GPU layers (-1 = all, default: -1)
-c	Context size (default: 8192)
--temp	Sampling temperature (0 = greedy, default: 0)
-s	Random seed
--json	Output in JSON format
--chat-template	Chat template name (default: auto from model)
--screenshot	Interactively select screen region to OCR
-q, --quiet	Suppress info logs, show only OCR result
-h	Show help

Examples

# OCR an image file
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf -p "OCR" screenshot.png

# Read image from clipboard
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf

# Interactive screen region selection
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf --screenshot

# Quiet mode (suppress loading logs)
ocr -m model.gguf --mmproj mmproj.gguf -q image.png

# JSON output
ocr -m model.gguf --mmproj mmproj.gguf --json image.png

Notes

• The --mmproj flag is required (no short form).
• The chat template is auto-detected from the model's GGUF metadata. Use --chat-template to override.
• When add_special=false, BOS is not added because the chat template output already contains the necessary prefix tokens (e.g. [gMASK]<sop> for GLM-OCR models).
• The screenshot temp file is automatically cleaned up after processing.