wangjue/llm-ocr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Run LLM-OCR based on llama.cpp
8 Commits 1 Branch 0 Tags 64 KiB
C++ 93.2%
CMake 6.8%
5d4965f014
Go to file
wangjue 5d4965f014 Add README
2026-05-29 16:25:34 +08:00
.gitignore Add --screenshot flag for interactive screen region selection 2026-05-29 15:38:05 +08:00
CMakeLists.txt feat: implement OCR tool using llama.cpp multimodal API 2026-05-29 14:57:37 +08:00
main.cpp Remove -mm short form, require --mmproj only 2026-05-29 16:22:02 +08:00
README.md Add README 2026-05-29 16:25:34 +08:00

README.md

OCR

Offline OCR tool based on llama.cpp and its multimodal extension (mmproj). Supports GLM-OCR and other vision‑language models.

Requirements

  • C++17 compiler
  • CMake ≥ 3.15
  • llama.cpp libraries: libllama.so, libmtmd.so, libggml*.so
  • llama.cpp headers: llama.h, mtmd.h, mtmd-helper.h, ggml.h
  • (Optional) CUDA 12.2 runtime for GPU acceleration

Runtime Dependencies

Feature Tool
File input —
Clipboard wl-paste (Wayland) / xclip (X11)
Screenshot gnome-screenshot, flameshot, spectacle, maim, or slurp+grim

Build

cmake -S . -B build
cmake --build build

Custom dependency paths

The build expects the llama.cpp libraries and headers under deps/llama.cpp/. To use a different location, edit DEPS_DIR in CMakeLists.txt.

Usage

ocr -m <model.gguf> --mmproj <mmproj.gguf> [options] [<image_path>]

Options

Flag Description
-m, --model Model file path (GGUF)
--mmproj Multimodal projection file path (GGUF)
-p OCR prompt (default: OCR)
-t Thread count (default: CPU cores)
--ngl GPU layers (-1 = all, default: -1)
-c Context size (default: 8192)
--temp Sampling temperature (0 = greedy, default: 0)
-s Random seed
--json Output in JSON format
--chat-template Chat template name (default: auto from model)
--screenshot Interactively select screen region to OCR
-q, --quiet Suppress info logs, show only OCR result
-h Show help

Examples

# OCR an image file
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf -p "OCR" screenshot.png

# Read image from clipboard
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf

# Interactive screen region selection
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf --screenshot

# Quiet mode (suppress loading logs)
ocr -m model.gguf --mmproj mmproj.gguf -q image.png

# JSON output
ocr -m model.gguf --mmproj mmproj.gguf --json image.png

Notes

  • The --mmproj flag is required (no short form).
  • The chat template is auto-detected from the model's GGUF metadata. Use --chat-template to override.
  • When add_special=false, BOS is not added because the chat template output already contains the necessary prefix tokens (e.g. [gMASK]<sop> for GLM-OCR models).
  • The screenshot temp file is automatically cleaned up after processing.