feat: Add Telegram file/image support with vision, OCR, and document-to-context pipeline

[Copilot]

Telegram only supported text messages. This adds handling for photos and file attachments (PDF, DOCX, images), routing them through a vision/OCR/document extraction pipeline before injecting context into the LLM.

Platform Layer

• IncomingMessage gains attachments: Vec<Attachment> (kind: Image|Pdf|Docx|Other, path, mime_type)
• telegram.rs now handles msg.photo() (highest-res variant) and msg.document(), downloading to per-request temp dirs cleaned up after processing

LLM Message Model (llm.rs)

• ChatMessage.content promoted from Option<String> to MessageContent — either a plain string or Vec<ContentPart> (text/image_url)
• Serializes as plain string when no images present (backward-compatible)

File Processor (src/file_processor/)

• Images: if openrouter.supports_vision = true → base64-encode into multi-modal content parts; otherwise → OCR via ocrs (pure Rust, neural-network-based OCR engine — no system library dependencies) → inject as text
• PDF: text extraction via pdf-extract
• DOCX: text extraction via docx-rs
• Long context (>6000 chars): text is chunked (1000-char chunks with overlap), stored via the existing EmbeddingEngine + sqlite-vec knowledge store, and top-K chunks are RAG-retrieved per user query — no new DB required

OCR

Uses ocrs — a pure Rust neural-network OCR engine backed by the rten inference runtime. Models (text detection + recognition) are downloaded automatically from S3 on first use and cached in ~/.cache/ocrs/. No native system library (e.g. Tesseract) required.

Config

[openrouter]
# supports_vision = false  # Set to true if your model supports image inputs

[ocr]
# model_dir = "~/.cache/ocrs"  # Where OCR model files are cached (downloaded on first use)

New Dependencies

Crate	Purpose
ocrs	OCR fallback for non-vision models (pure Rust, neural-network based)
rten	Neural network inference runtime for ocrs model files
image	Image loading/decoding
pdf-extract	PDF text extraction
docx-rs	DOCX parsing
infer	MIME type detection from bytes
base64	Base64 encoding for vision API image content parts

Original prompt

Use superpower to create a plan to add telegram file and image support, when receive file and image, store in temp and process supported format. For image, when default model is support visual, directly put the image to the model, else, research and find a best way in 2026 rust library ocr the image and put it in context to llm. For docx, pdf, research best method in rush for convert to markdown and also put as context to llm. If pdf or docx has image, extract also the image for answering questions, if the context too long, put it to local vector db (research best for vector in rust or use exists vector db) and pass user questions to vector and find result , also using exists embedded model chosen.

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[chinkan]

@copilot use ocrs https://github.com/robertknight/ocrs as ocr image instead of terract

[Copilot]

@copilot use ocrs https://github.com/robertknight/ocrs as ocr image instead of terract

Switched OCR from tesseract to ocrs (https://github.com/robertknight/ocrs) — pure Rust, neural-network-based, no system library dependencies. Implementation in src/file_processor/mod.rs uses ocrs::OcrEngine with rten::Model via spawn_blocking. Models are auto-downloaded from S3 on first use to ~/.cache/ocrs/. Committed in 12e9337.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• docs.rs
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)