Suites / English

English

28 questions on 12 English Wikipedia articles (tea, octopus, Marie Curie, ...), 630 KB. Four are asked in Hindi, Spanish or French, and five are off-topic.

3 runs / 12 files, 0.6 MB / model haiku, thinking off / Vector RAG embedder BAAI/bge-small-en-v1.5. Raw results: 2026-09-30 16:20 2026-09-30 16:23 2026-09-30 16:26

Grape

23 / 23

Off-topic refused: 5 / 5
Input tokens per question: 1,217 (1,203 to 1,244)
LLM calls per question: 1.12 (1.11 to 1.14)
Cost per question: $0.00148 ($0.00145 to $0.00152)
Time per question: 3.8 s (3.7 s to 3.8 s)
Index time: 47 ms

Vector RAG

19 / 23

Off-topic refused: 5 / 5
Input tokens per question: 1,307
LLM calls per question: 1.00
Cost per question: $0.00155 ($0.00155 to $0.00156)
Time per question: 3.0 s (3.0 s to 3.1 s)
Embedding time: 1.6 min

Questions

right every run some runs never. Left half Grape, right half Vector RAG.