[PR #427] [CLOSED] Add RAG Retrieval Evaluation Harness (diagnostic example) #740

New Issue

GiteaMirror · 2026-04-15T07:01:34-05:00

GiteaMirror commented

2026-04-15 07:01:34 -05:00

📋 Pull Request Information

Original PR: https://github.com/Shubhamsaboo/awesome-llm-apps/pull/427
Author: @Arnav-Ajay
Created: 1/17/2026
Status: ❌ Closed

Base: main ← Head: add-rag-retrieval-eval-harness

📝 Commits (1)

4b05e7d Added RAG retrieval evaluation harness example

📊 Changes

1 file changed (+2 additions, -0 deletions)

View changed files

📝 README.md (+2 -0)

📄 Description

Hi Shubham 👋

This PR adds a small, self-contained example focused on retrieval evaluation in RAG systems.

Most RAG examples demonstrate end-to-end answering.
This repo instead asks an earlier question:

Did retrieval surface the right evidence before generation?

The example provides:

a minimal retrieval-only harness
human-labeled relevant chunks
inspection beyond generation Top-K
explicit visibility into ranking depth and retrieval misses

It’s intentionally diagnostic (not an optimization demo) and makes no
general performance claims.

If this doesn’t fit the direction you want for the list, totally fine —
happy to adjust or remove.

Thanks for maintaining the repo!

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Shubhamsaboo/awesome-llm-apps/pull/427 **Author:** [@Arnav-Ajay](https://github.com/Arnav-Ajay) **Created:** 1/17/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `add-rag-retrieval-eval-harness` --- ### 📝 Commits (1) - [`4b05e7d`](https://github.com/Shubhamsaboo/awesome-llm-apps/commit/4b05e7d766a754a9e58733a516c7683cf5fc05d6) Added RAG retrieval evaluation harness example ### 📊 Changes **1 file changed** (+2 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+2 -0) </details> ### 📄 Description Hi Shubham 👋 This PR adds a small, self-contained example focused on **retrieval evaluation in RAG systems**. Most RAG examples demonstrate end-to-end answering. This repo instead asks an earlier question: > Did retrieval surface the right evidence before generation? The example provides: - a minimal retrieval-only harness - human-labeled relevant chunks - inspection beyond generation Top-K - explicit visibility into ranking depth and retrieval misses It’s intentionally diagnostic (not an optimization demo) and makes no general performance claims. If this doesn’t fit the direction you want for the list, totally fine — happy to adjust or remove. Thanks for maintaining the repo! --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-15 07:01:34 -05:00

GiteaMirror closed this issue