StudioBeeNara v0.4: A 332 MB Local Document Classifier Achieves 91.5% Accuracy on Unseen Folder Trees, with a New 2B Sibling Model
Original title: BeeNara v0.4: my tiny "none fits" document sorter now files on its own only when it's provably safe, plus a 2B sibling for whole folder trees
The author released BeeNara v0.4, a 332 MB CPU-based local document classification model under the Apache-2.0 license. On tasks involving unseen folder trees (1,500 samples), accuracy improved from 70% in v0.2 to 91.5%. Using a Learn-then-Test threshold, it guarantees that automatic archiving errors do not exceed 1% with 90% confidence, achieving an actual auto-archiving rate of 59% with only 0.3% error. Additionally, a new sibling model, BeeNara Big, based on Qwen3.5-2B-Base with LoRA and a decision head, can process entire folder trees (up to ~600 folders) and page images, achieving 92–94% accuracy within 24 folders. However, it requires a GPU (approx. 4.3 GB at 8-bit). The author also noted that label smoothing causes threshold failure.
The author self-tested v0.4's accuracy and auto-archiving threshold performance on unseen folder trees, and shared lessons on how label smoothing invalidates thresholds, which is transferable to similar classification tasks.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.