Suites / Gujarati questions, English documents, default English embedder

Gujarati questions, English documents, default English embedder

The same Gujarati questions, but Vector RAG uses the English embedding model most tutorials start with (bge-small-en). Shown to make the point that a vector setup only reaches across languages if its embedder does.

3 runs / 12 files, 0.6 MB / model haiku, thinking off / Vector RAG embedder BAAI/bge-small-en-v1.5. Raw results: 2026-09-30 16:31 2026-09-30 16:36 2026-09-30 16:41

Grape

23 / 23

Off-topic refused: 5 / 5
Input tokens per question: 1,583 (1,573 to 1,594)
LLM calls per question: 1.83 (1.82 to 1.86)
Cost per question: $0.00231 ($0.00228 to $0.00234)
Time per question: 7.7 s (7.4 s to 8.2 s)
Index time: 47 ms

Vector RAG

0 / 23

Off-topic refused: 5 / 5
Input tokens per question: 533
LLM calls per question: 1.00
Cost per question: $0.00059 ($0.00059 to $0.00060)
Time per question: 2.6 s (2.6 s to 2.7 s)
Embedding time: 1.6 min

Questions

right every run some runs never. Left half Grape, right half Vector RAG.