Ask an AI about your company and it often gives a confident, wrong answer. Company knowledge is scattered across wikis, shared drives, code comments and the heads of senior engineers, so the AI is missing context. The usual fix has been to find relevant documents and attach them to the prompt: RAG.
In June, Google released a format that says, in effect, "just keep it organized in a folder." It is called OKF (Open Knowledge Format). Videos titled "a folder beat the vector database" and "RAG is dead" got hundreds of thousands of views. So we tested it.
This post covers:
- What RAG and OKF are, and how each works
- What is actually in Google's repository
- Results from the same 18 questions, measured by us
- A trap in Google's example: a "human-verified" SQL query
- When to use which
How to read the numbers. [Official] means material published by Google, Anthropic and others. [Paper] means a public paper. [Measured] means numbers we ran and measured ourselves on our own machine.
1. What are RAG and OKF?
RAG (finding chunks)
RAG (Retrieval-Augmented Generation) means "find, attach, then answer." The name comes from a 2020 Facebook AI Research paper [Paper] (Lewis et al. 2020).
- Split documents into small chunks ahead of time.
- Turn each chunk into a list of numbers. This is called an embedding: when two lists are close, the texts mean similar things.
- When a question comes in, embed it too, pick the few most similar chunks (usually 5), and attach only those to the prompt.
It is fast and cheap. But the AI cannot see anything outside the chunks it picked.
OKF (exploring a folder)
OKF is a knowledge format Google Cloud announced on June 12, 2026 [Official] (Google Cloud blog). Version 0.2 followed on July 24 (spec).
- Markdown files in a folder, one file per concept.
- Each file starts with a short frontmatter block. The only required field is
type. - Files link to each other, and each folder has an
index.mdthat acts as a table of contents. - A folder packaged this way is called a bundle.
The AI reads the table of contents first, follows links, and opens the files it needs. It can also search by keyword. Opening many files costs more time and tokens.
They work at different layers
This matters: OKF decides how knowledge is written down. RAG decides how it is found. Neither Google's announcement nor the spec says OKF "replaces RAG." The real difference is when you organize. Organizing up front makes answers more accurate but needs someone to keep it maintained. Searching on demand is easier to run, but chunks can be cut off or missed.
2. What is in Google's repository?
We opened Google's public repository (GoogleCloudPlatform/open-knowledge-format, commit ad30107).
- It contains a producer agent (Gemini writes an OKF document for each BigQuery table) and a graph viewer.
- The agent's tools are things like listing concepts, reading sources and writing documents.
- Searching the whole codebase for
embedding,vectorandretrievreturns 0 matches. There is no code that answers questions. [Measured]
So to use OKF, you build the reading side yourself, whether that is search or an agent that opens the folder.
3. Testing with the same questions
Setup
- Data: Google's 4 official example bundles (GA4, Stack Overflow, Bitcoin, and Acme Retail, an example company Google made up), 78 Markdown files
- Model: a single model, Claude Haiku 5.5
- Five approaches
- RAG: 181 chunks of 1,200 characters, embedded locally with bge-m3; the top 5 chunks are attached
- OKF: start from the bundle's
index.md, with tools to read, list and search files - Put everything in: all 78 files in the prompt
- Raw RAG: RAG over the 10 public source documents the bundles were built from, as-is
- Hybrid: answer with RAG first; only if it says "can't find," hand off to OKF
- Questions: 10 easy questions (1 run), 8 hard questions (3 runs), 8 raw-source comparison questions (3 runs). The answer key was written by a person reading the bundle sources.
Easy questions: same answers, different cost
Every approach got all 10 easy questions right. The difference was in how much the AI read (tokens). [Measured]
| Approach | Average input tokens per question |
|---|---|
| RAG | ~6,200 |
| OKF | ~40,100 |
| Put everything in | ~68,200 |
For easy questions, RAG is more than 6× cheaper.
A question that counts everything: RAG says 4, OKF says 7
We asked, "How many join documents are there across all bundles?" The answer is 7. RAG pulled the 5 chunks most similar to the question, and only one Stack Overflow join document was among them. So it answered 4.
OKF read the table of contents, searched for join files, and answered 7. Over three runs, RAG went 0/3 and OKF went 3/3. [Measured]
Hard questions: 8 questions × 3 runs
| Question set | RAG | OKF | Put everything in | Hybrid |
|---|---|---|---|---|
| 8 hard questions × 3 runs | 15/24 | 24/24 | 24/24 | 24/24 |
RAG lost in three kinds of questions. [Measured]
- Counting everything (how many join documents?): 0/3
- Comparing across everything (which bundles were verified by a person?): 0/3. Without enough evidence, it could not say "none."
- A chunk cut in half: the SQL it needed to check was split at a chunk boundary, so it could not decide (0/3).
To its credit, RAG did not make things up. When it did not know, it said it could not confirm.
OKF without search
When we removed OKF's search tool and let it only follow links, a single question ("find all deprecated concepts") took up to 80 tool calls and over 200,000 input tokens. With search allowed, the same question took a couple of calls and about 14,000 tokens. [Measured] As a folder grows, "just follow the table of contents" is not enough; you need search.
RAG over the raw sources
The comparison most teams actually face is "RAG over raw documents" vs "documents organized in advance." So we indexed the 10 public sources the bundles cite (the Stack Exchange schema docs, GA4 docs, Bitcoin ETL docs and others) and asked the same questions. [Measured]
| Approach | Correct (8 questions × 3 runs) |
|---|---|
| Raw RAG | 15/24 |
| RAG over the organized bundle | 21/24 |
| OKF | 24/24 |
For "how do you compute the accepted-answer rate?", raw RAG said it could not find it. The needed column (AcceptedAnswerId) is somewhere in the sources, but it never retrieved that chunk. The organized bundle states the formula in one line, so the answer came right away. Metrics newly defined in the bundle do not exist in the sources at all, so raw RAG had no way to know them. This is where organizing in advance paid off the most.
Hybrid: the cheap method first, the expensive one only when needed
Finally, we combined them. RAG answers first; if the answer contains "can't find," only then does OKF take over. Google DeepMind's Self-Route work also cut cost by routing each question this way [Paper] (Li et al. 2024).
- Easy 10/10, hard 24/24, 34/34 correct in total [Measured]
- On easy questions, it used half the tokens of OKF alone (21k vs 40k)
- But 13 times it handed off a question RAG would have gotten right. Sentences like "the answer is X, though part of it can't be confirmed" triggered the hand-off rule.
4. The trap in the "human-verified SQL"
OKF v0.2 added trust features: who verified a document (verified), how long it stays valid (stale_after), and attestation. Attestation checks whether the SQL you just ran is identical to the approved SQL.
In Google's example company, Acme Retail, the gross margin SQL was verified by a person (verified: human:jsmith) and approved for attestation. When we read it, something did not match.
- Company policy: revenue counts only 30 days after delivery
- Approved SQL:
DATE_DIFF(CURRENT_DATE(), DATE(o.order_ts), DAY) >= 30, that is, 30 days from the order date
Attestation still passes, because it is the same SQL as the approved one. The spec itself says verification and attestation are different things (SPEC §10.6). Labels and checks help, but someone still has to check whether the content is right. Also, 3 of the 4 official bundles (61 documents) were written by Gemini and have no record of human verification at all. [Measured]
5. When to use which
| If your knowledge is… | Do this |
|---|---|
| Small (under ~200k tokens) | Put everything in. Anthropic also recommends skipping RAG at this size [Official] (Contextual Retrieval) |
| Company-specific definitions, metrics and policies | Organize it in OKF up front. The raw sources do not contain those definitions |
| Large and constantly changing (tickets, meeting notes, PDFs) | RAG |
| Questions like "how many in total?" or "where is it missing?" | OKF (with search) |
| A mix of the above (most companies) | Hybrid: find with RAG first, hand off to OKF when it can't |
Instead of asking "which is better," ask "what is worth organizing in advance?" The answer follows from there.
Limits of this test
- One model (Claude Haiku 5.5), Google's example bundles, and their 10 source documents.
- Some hard questions were chosen to probe where RAG is weak. Take away which questions each approach is weak at, more than the scores.
- RAG ran with one default setup (1,200-character chunks, bge-m3 embeddings, top 5). Adding keyword search (BM25) or reranking could change the results.
Watch the video
We covered the same experiment, with the real runs on screen, in a Korean video. It goes public on October 31 at 5:00 AM ET.
Sources
- Google Cloud, Introducing the Open Knowledge Format (2026-06-12)
- GoogleCloudPlatform/open-knowledge-format — SPEC.md v0.2, commit
ad30107 - Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
- Li et al., Retrieval Augmented Generation or Long-Context LLMs? (2024)
- Anthropic, Introducing Contextual Retrieval (2024)
