StudioQwen3.8 27B on a 3090 basically ties Opus 5.5 on a Korean RAG benchmark
Original title: qwen3.8 27B on my 3090 basically tied Opus 5.5 on a Korean RAG benchmark, which I did not expect
Running local RAG over Korean PDFs with Allganize's Korean RAG set (300 questions, 64 PDFs), the author found qwen3.8 27B with thinking on scored 77.3% vs Opus 5.5's 76.6%, a statistical tie by McNemar (p=0.82); with Opus as judge it was 80.6% vs 80.3%. Of Opus's 70 misses, only 25 were model errors, 22 were scanned image pages without a text layer and 23 were retrieval misses; switching BM25 from whitespace splitting to a Korean morphological tokenizer lifted hit@5 from 56% to 80%, and gpt-oss 20B went from 57.5% to 69.6% after raising the token limit from 1024 to 8192. KRAFTON's Raon-1 mini preview (30B MoE, ~4B active) landed around 74-75% at 96 tok/s on the 3090 but got stuck in a loop on 16 of 320 questions at temp 0 with thinking on. Scripts and full tables are at https://bric.pe.kr…
The author ran a controlled local RAG comparison with dual judges, reporting error attribution, retrieval tokenization gains, and run-to-run variance that readers can reuse for their own document evaluations.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.