81 lines
3.0 KiB
Markdown
81 lines
3.0 KiB
Markdown
# OCR
|
||
|
||
Offline OCR tool based on [llama.cpp](https://github.com/ggml-org/llama.cpp) and its multimodal extension (mmproj). Supports GLM-OCR and other vision‑language models.
|
||
|
||
## Requirements
|
||
|
||
- **C++17** compiler
|
||
- CMake ≥ 3.15
|
||
- llama.cpp libraries: `libllama.so`, `libmtmd.so`, `libggml*.so`
|
||
- llama.cpp headers: `llama.h`, `mtmd.h`, `mtmd-helper.h`, `ggml.h`
|
||
- (Optional) CUDA 12.2 runtime for GPU acceleration
|
||
|
||
### Runtime Dependencies
|
||
|
||
| Feature | Tool |
|
||
| ------------- | ----------------------------- |
|
||
| File input | — |
|
||
| Clipboard | `wl-paste` (Wayland) / `xclip` (X11) |
|
||
| Screenshot | `gnome-screenshot`, `flameshot`, `spectacle`, `maim`, or `slurp+grim` |
|
||
|
||
## Build
|
||
|
||
```bash
|
||
cmake -S . -B build
|
||
cmake --build build
|
||
```
|
||
|
||
### Custom dependency paths
|
||
|
||
The build expects the llama.cpp libraries and headers under `deps/llama.cpp/`. To use a different location, edit `DEPS_DIR` in `CMakeLists.txt`.
|
||
|
||
## Usage
|
||
|
||
```bash
|
||
ocr -m <model.gguf> --mmproj <mmproj.gguf> [options] [<image_path>]
|
||
```
|
||
|
||
### Options
|
||
|
||
| Flag | Description |
|
||
| ---------------------- | -------------------------------------------- |
|
||
| `-m`, `--model` | Model file path (GGUF) |
|
||
| `--mmproj` | Multimodal projection file path (GGUF) |
|
||
| `-p` | OCR prompt (default: `OCR`) |
|
||
| `-t` | Thread count (default: CPU cores) |
|
||
| `--ngl` | GPU layers (`-1` = all, default: `-1`) |
|
||
| `-c` | Context size (default: `8192`) |
|
||
| `--temp` | Sampling temperature (`0` = greedy, default: `0`) |
|
||
| `-s` | Random seed |
|
||
| `--json` | Output in JSON format |
|
||
| `--chat-template` | Chat template name (default: auto from model) |
|
||
| `--screenshot` | Interactively select screen region to OCR |
|
||
| `-q`, `--quiet` | Suppress info logs, show only OCR result |
|
||
| `-h` | Show help |
|
||
|
||
### Examples
|
||
|
||
```bash
|
||
# OCR an image file
|
||
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf -p "OCR" screenshot.png
|
||
|
||
# Read image from clipboard
|
||
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf
|
||
|
||
# Interactive screen region selection
|
||
ocr -m GLM-OCR-Q8_0.gguf --mmproj mmproj-GLM-OCR-Q8_0.gguf --screenshot
|
||
|
||
# Quiet mode (suppress loading logs)
|
||
ocr -m model.gguf --mmproj mmproj.gguf -q image.png
|
||
|
||
# JSON output
|
||
ocr -m model.gguf --mmproj mmproj.gguf --json image.png
|
||
```
|
||
|
||
## Notes
|
||
|
||
- The `--mmproj` flag is **required** (no short form).
|
||
- The chat template is auto-detected from the model's GGUF metadata. Use `--chat-template` to override.
|
||
- When `add_special=false`, BOS is not added because the chat template output already contains the necessary prefix tokens (e.g. `[gMASK]<sop>` for GLM-OCR models).
|
||
- The screenshot temp file is automatically cleaned up after processing.
|