Case study: David's Archive

I built the industry's answer to the reading pile, measured it, and cut it. What survived is the part the industry gets backwards.

Author: David Shatsky · Period covered: April to September 2026 · Status: draft for review, 2026-09-09. The evidence trail and the hypotheses tested along the way are in the decomposition brief; the removal plan that follows from it is here.

Executive summary

The question. Why should a personal reading archive exist when Readwise Reader, NotebookLM, and Curius already do, and what in the one I built actually earns its place?

The answer. The problem is the one the LLM-wiki community names in one line: inputs "pile up faster than I can read." The prevailing answer, from Karpathy's llm-wiki pattern outward, is an inversion of labor: the human sources and asks, "the LLM writes and maintains all of it," and the corpus becomes "a persistent, compounding artifact." My archive takes the other half of the same problem. The model does the cataloging, so a save costs nothing. The shelf is public. And the writing stays human, because writing until I am no longer confused is how I learn, and the wiki pattern's most reported failure mode is prose that was "smoothed" until it "completely lost the original cognitive shape." A design in which the only prose that matters is the curator's cannot fail that way.

What it took to learn that. Between April and July I built the prevailing answer on top of the shelf: a quiz and recall loop, spaced repetition, a daily review queue, retrieval-augmented chat with citations, a Thompson-sampling model router, and a weekly eval harness. The usage data at five months is unambiguous. Capture worked. Everything built to run after capture terminated at or near zero, and not for want of features. What was missing was my own writing; a retention loop over notes I had not written had nothing to retain.

The decision. Retire the machinery in the docs, remove it from the code under review, and reduce the product to four things: save from anywhere with summary and tags generated, notes next to the source in my words, find it again, a public shelf. The router and the eval harness are the most technically interesting code in the repository, and neither serves those four. Keeping them would make the portfolio look better and the product worse.

1. Situation: the reading pile, and the category built to hold it

Knowledge workers consume more than they can process, and the read-it-later category exists to absorb the overflow. It is a mature category with a well-defined feature set. Reader ingests articles, PDFs, YouTube transcripts, newsletters, RSS, EPUBs and X threads; supports highlights with inline notes and tags; runs full-text search; resurfaces highlights through a spaced Daily Review; and ships an on-demand assistant, Ghostreader, for summaries and questions, at $9.99 a month billed annually. Curius sells the public half of the job as "bookmarks for the extremely curious": a shareable bookshelf with highlights and a social layer. Digital gardens and commonplace books are an established niche for "a quietly public way of sharing ideas".

The category is also unstable. Omnivore was acquired by ElevenLabs and shut two weeks later in November 2024, with user data deleted on a fortnight's notice. Pocket, the category's household name, shut down in July 2025 and deleted all saved data by October. None of that was my reason for building; anyone can build their own tool now with agent coding, and doing so is not notable. It is context for why the market gap I first thought I had found does not exist, and why the real reason had to be found elsewhere.

2. Complication: the industry's answer, and the one I built

In April 2026 Karpathy published a one-page pattern that reframed the category. His diagnosis of retrieval-augmented generation is that "the LLM is rediscovering knowledge from scratch on every question. There's no accumulation." His remedy is to have the model "incrementally build and maintain a persistent wiki," so that knowledge "is compiled once and then kept current, not re-derived on every query." The role split is explicit: "You're in charge of sourcing, exploration, and asking the right questions. The LLM does all the grunt work." "You read it; the LLM writes it."

The practitioner reports say the same thing in plainer words. One six-month user: "I tried Notion for a year. I tried plain Obsidian for another. Neither lasted because the maintenance burden always fell on me." And on why the pattern stuck where the others did not: "the LLM maintains the wiki, the human investments are inputs and questions. That single inversion is why this stuck when nothing else did."

My archive was born adjacent to this. The first build, in April, was a catalog: paste a link, the model scrapes it, writes a summary, picks tags, and the post appears on a public shelf. Then, between May and July, it grew the prevailing answer on top:

• a Learn loop (quiz, recall, tutor modes) over the corpus, with pre-generated questions at ingest and a spaced-repetition scheduler driving a daily review queue;
• RAG chat with citations, per-chunk provenance, and admin-curated answer permalinks;
• a Thompson-sampling model router ported from Ramp's published design, routing every generation call across five arms on a cost-weighted, deadline-aware posterior;
• a weekly eval harness scoring retrieval hit rate, citation validity, and LLM-judged answer quality against a versioned golden set, published at /evals failures included;
• a north-star metric of answered questions per week, with an activation event and a retention proxy, rolled up nightly to /stats.

That is 105 commits in May alone. It is also, feature for feature, what a well-read builder in 2026 would ship. The complication is that it was built for a corpus and a user that did not behave the way the design assumed.

3. Resolution, part one: what the numbers said

Pulled live from Firestore on 2026-09-09, with the notes read rather than counted.

Exhibit 1. Adoption funnel, lifetime

Exhibit 2. Hypotheses tested and killed

Two readings of Exhibit 1 are worth separating. The first is that the read flag is broken as a measure: nine of the ten substantive notes sit on posts marked unread. I read or watch in the source and come back to leave a note, which is the default behavior now and distinguishes nothing, but it does make Reader's read-inside-the-app model the odd one out. The second is the one that matters. Capture is healthy. Every stage that was supposed to run after capture terminated, and the reason is on the fourth row. A retention loop over notes that are transcriptions of the source has nothing of mine to retain. The machine was not broken. It was pointed at an input that did not exist.

4. Resolution, part two: the position, and the cut

The finding forced a first-principles restatement of what the product is for, and the restatement is the opposite of where I started.

The wiki pattern's answer to the pile is to let the model write the knowledge. Mine is to let the model do the cataloging and keep the writing. The model reads the source once, writes the summary on the card, and picks tags from a small catalog; that is the whole of its job. The note is mine or it does not exist. The shelf is public because I want it to be, not because that is the job. And ownership is a motive, not a claim: the data and the code are mine, and so is everyone's now.

Stated as a rule, the archive does four things and nothing else:

1. Save from anywhere in one action, phone included, with summary and tags generated so saving costs nothing.
2. Notes next to the source, in my words, kept forever.
3. Find it again, by search and by tag, at 74 posts and at 740.
4. A public shelf anyone can browse cleanly.

Everything else is either the later understanding layer (below) or should not be built. That retires the Learn loop, spaced repetition, RAG chat as a headline, the router, the eval cron, the digest and the drift cron. The removal is review-gated, ships one pull request per surface group with the public shelf up throughout, and every cut leaves a changelog line naming what the feature was for and what its numbers said. The cuts are part of the record, not a cleanup.

The counter-case, at full strength. Almost nobody in the lanes I am hiring into has shipped a bandit router or an eval harness in production. Most hiring managers have not internalized that building got cheap, and they still grade on artifacts. On that view the machinery is the moat and removing it is unilateral disarmament. I think that view is right about hiring managers and wrong about the product, and the two cannot be reconciled by keeping the code. They are reconciled by this document: the judgment of having built the prevailing answer, measured it against a stated north star, and cut it when the north star came back at nine, is a better artifact than the router was. It is also the only one that could not have been vibe-coded in a weekend.

What the cut taught. Three things I would tell a team building in this space.

• A north star can be wrong about the product, not just about the number. "Answered questions per week" measured a chat surface. The product was a shelf. The metric was accurate and irrelevant, and it took five months to notice because the dashboard looked like progress.
• Generated organization without maintenance is noise, not structure. Two hundred tags with fourteen in real use is what the wiki community means by "maintenance grows faster than value," and it happened with the model doing the generating and nobody doing the lint. The pattern's own critics name the same gap: a link "carries exactly one bit of information," and without a maintainer the graph decays (Penfield Labs).
• The expensive input is the one to protect. In this system the scarce resource is not compute, storage, or retrieval quality. It is a curator willing to write. Every feature that made the model write more made that input less likely to appear, because it made the shelf look finished without it.

5. What comes next, and why it is human

The thing the Learn loop was reaching for and could not capture is a deeper understanding of the material, reached by writing about it in my own words as an exercise. That layer exists today as a provider-neutral protocol I run in a chat session, not in the app: recall before any summary, coverage triage, interrogation, the strongest opposing view stated in a form its holder would accept, then a draft with an editor's pass that never writes my sentences. It is not being built into the archive now. When it is, the rule holds: the model tests, pushes back, and shows what I missed. It never drafts the note.

The metrics that replace the north star are small enough to embarrass me. Saves per week, which has always worked. Notes in my own words, currently one of seventy-four, which is the number the whole position rests on. Tag health, currently fourteen of two hundred. Public views, as a side effect I am glad to have. If the second number does not move, the position is a preference, not a product, and the honest thing will be to say so.

The industry has decided that the way to survive the reading pile is to let the model do the knowing. I built that, watched it idle, and concluded that the only part of a reading life worth automating is the part that was never the point.

Sources

• Andrej Karpathy, llm-wiki (gist, April 2026)
• Karpathy's LLM Wiki, Six Months In (practitioner report)
• Penfield Labs, What Karpathy's LLM Wiki Is Missing
• Readwise Reader and Marqly, Readwise Reader Review 2026
• Curius
• Intro to digital gardening
• PCWorld, Mozilla is discontinuing Pocket and Fakespot
• Creativerly, The exit(us) of Omnivore
• This repository: decomposition brief, removal plan, router research report, and the live /stats and /evals pages