Documentation for Custom Dataset and Custom Indexes

[amitbcp]

Is your feature request related to a problem? Please describe.
Rankify already supports Custom Dataset, and it seems it supports custom indices, but it is super hard to follow how to add a new index, or create a new index, or add a new benchmark dataset for benchmarking

Describe the solution you'd like
We need to update the documentation with standard flows to help users and adopters. Some of the standards flow is Indexing on the data (we need to provide data formats etc), adoption or adding a new benchmark datasets for benchmark curators and on similar lines.

Without it, developers have to jump in the code to figure out what is happening, creating barrier for adoption

[abdoelsayed2016]

Thank you for raising this important issue! You're absolutely right - we need to improve our documentation to make Rankify more accessible and easier to adopt.

Current Documentation

We've recently added some documentation for custom indexing in the README:

• CLI-based indexing: https://github.com/DataScienceUIBK/Rankify?tab=readme-ov-file#-indexing-via-cli

Example:

rankify-index index examples/sample.jsonl \
  --retriever bm25 \
  --output ./indices

We've also included examples/sample.jsonl to show the expected corpus format, and there are examples demonstrating programmatic indexing (without CLI).

What We Need to Improve

However, we acknowledge that more comprehensive documentation is needed:

• Detailed data format specifications for custom datasets
• Step-by-step guide for adding new benchmark datasets
• Clear workflow for creating and using custom indices
• API documentation with more examples
• Tutorial for benchmark curators

Thanks again for the feedback! 🙏