AI Classification Beats Folder Trees: A Field Study
We ran a controlled experiment: same documents, two tenants, one using deep folders and one using AI classification + tags. Here are the findings.
AI Classification Beats Folder Trees: A Field Study
For decades, document management orthodoxy was: build a deep folder tree, train staff to file consistently, audit compliance. AI-native systems propose an alternative: classify on ingest, tag liberally, search to retrieve. Which actually works better?
We had the chance to compare. Two of our tenants — similar industry (mid-size legal/audit), similar size (60-80 people), similar document volume (~3,000 docs/month) — opted into different strategies for their first 90 days.
- Tenant A: Migrated their old folder tree (8-level deep) and continued to file manually.
- Tenant B: Used AI auto-classification, root folders only by department, freely-applied tags.
Both started fresh from existing shared drives at Week 0.
What we measured
- Time to retrieve a known document
- Documents misfiled (wrong category)
- Search success rate (user marks the result as the one they wanted)
- User satisfaction (NPS-style)
- Compliance findings during a snapshot audit at week 12
Results
Time to retrieve
| Tenant | Median seconds to retrieve a known document |
|---|---|
| A (folders) | 38 seconds |
| B (AI + tags) | 12 seconds |
Tenant B's users were 3x faster. The difference is even larger for newer staff: long-tenure employees know the folder tree; new staff don't and must explore.
Misfile rate
| Tenant | Misfile rate (audit sample) |
|---|---|
| A (folders) | 14% |
| B (AI + tags) | 2.5% |
AI classification was more accurate than humans-filing-into-folders. The reason: AI judges the document content; humans guess the folder based on partial information.
Search success
Both tenants used Papyrus's hybrid search. Search success rate was very close: 88% (Tenant A) vs 91% (Tenant B). Search was the equaliser — even Tenant A's folder-filed documents were findable through search. Folder navigation just stopped being the primary access path.
User satisfaction
Tenant B's NPS was +42. Tenant A's NPS was +18.
Quotes from Tenant A: “I spend too much time deciding what folder this goes in.” / “I can't find things if I didn't file them myself.”
Quotes from Tenant B: “I just upload and forget.” / “Search finds it. I don't need to know where it lives.”
Compliance
This was the surprise. We expected folders to win on compliance — they're the “old world” model auditors expect. They didn't:
| Tenant | Audit findings (out of 30 sampled documents) |
|---|---|
| A (folders) | 5 documents in wrong folder; 2 missing retention tags |
| B (AI + tags) | 1 document with low-confidence AI classification not confirmed |
The audit was more permissive of “AI said X, human didn't confirm” than of “filed by human in wrong place”.
What this means
Folder trees are a 1970s metaphor that survived because nothing better existed. AI classification + tags is what document management was always trying to become. If you're starting fresh — start fresh.
If you're migrating, don't rebuild your folder tree. Bring docs in, let AI classify, add tags, and use folders only for very coarse organisation (by department, by year, but not by detailed type).
The folder tree is a museum exhibit. Visit on Sundays.