VTM Content Library

01The problem

Eight years of marketing output had accumulated as over 2,500 files and roughly fifty gigabytes of decks, leaflets, product screenshots, event photography and brand assets. None of it was searchable. When a colleague needed the current product leaflet or a usable site photograph, the only method was to open folder after folder and guess from filenames like image203.png — and then guess again about which of seven near-identical PDFs was the current one.

The damage was not only lost time. The same product’s module count and capability claims were stated differently across different decks, so whichever file someone happened to open became the version the client heard. A first attempt at fixing this was built and then torn down completely, because it had modelled the problem as a showcase system rather than as an index over the files people actually keep on disk.

02What I built

A file-first index: the files stay where they are and the database holds only the index. A scanner walks the library, opens each file to detect its real format rather than trusting the extension, and writes one row per file into a dedicated schema. Every image then passes through a vision tagging pass that writes a scene description, keywords, an image-type classification and a usability verdict back onto the row, so the search layer can answer a plain-language question instead of only matching filenames. On top sits a password-gated web library where thousands of files collapse into a few hundred cards.

• Format detection by opening the file — page counts and physical dimensions from PDFs, slide counts from presentations, pixel dimensions from images, duration from video — so a one-page A4 PDF is a leaflet and a forty-page landscape PDF is a deck.
• Vision tagging on every image, stored as structured JSON with a search index over it, so the tags are searchable rather than merely recorded.
• A card model instead of a file tree: language and version variants of the same material group into one card behind the newest cover.
• A duplicate policy that distinguishes byte-identical copies, which are removed with an undo manifest, from same-content-different-location copies, which are kept with one marked preferred.
• Private storage throughout — thumbnails and uploaded originals sit in two non-public buckets, the service key never reaches the browser, and every URL the library serves is signed per request.
• A structured content layer, one file per product, that generates pages from source data so a number is corrected once rather than in seven PDFs.

03As a product

Who buys it

Mid-size agencies, studios and product companies with a large, messy, multi-language asset archive and a sales team that has to pull from it weekly. The buyer is the marketing lead who is tired of being the human search index.

Value

It turns an unsearchable shared drive into something a salesperson can query in plain language and trust the answer of. The vision layer is what separates it from a file server with better folders: it indexes what is in the picture, which is the thing filenames never captured. The larger second-order payoff is consistency — once content lives in one structured layer, the company stops telling clients three different versions of the same fact.

Positioning

An enterprise asset manager solves this at enterprise price and requires uploading fifty gigabytes into someone else’s cloud. Drive search reads filenames and document text, not image content, and will not tell you which of seven leaflets is current. This deliberately leaves the bulk on local disk and syncs only an index and small thumbnails, which is why it runs on a free database tier.

Status

Internal tool, built for one company’s marketing and business development teams. The architecture is generic enough to license; no external deployment exists and none is priced.

04How it works

Stack

Python for the scanner, thumbnailer, vision tagger, card builder and page generator — twenty-five scripts, roughly five thousand lines. Postgres on Supabase in a dedicated schema with fourteen migrations, two private storage buckets, and a zero-build web app on Vercel: one serverless function plus a vanilla-JavaScript single page, no framework and no bundler.

Shape

Index, not upload. The scan is read-only, the load is an idempotent upsert on relative path, and every column is classed as either derived-from-scan or earned-elsewhere.

The decision I spent longest on

Making the loader an idempotent upsert on relative path rather than a rebuild. The first version deleted and re-inserted the whole table on each run, which re-issued every primary key — and since the primary key is the thumbnail object name in storage, one rerun orphaned over 2,400 thumbnails and severed more than 250 original-file links, pushing storage from roughly 910 MB to roughly 960 MB against a one-gigabyte ceiling. The rewrite split every column into two classes, derived-from-scan and earned-elsewhere, and made a moved file transfer its storage and tag links instead of dropping them.

05Retrieval architecture

The expensive thinking in this system happens long before anyone searches. A vision pass reads every image once and writes down what is in it, and after that a search is ordinary SQL over text a model already wrote.

That split is the whole design. Putting the model at index time is what makes every result explainable: a hit can always be traced back to the row and the sentence that matched, and it is why the query path has no model in it at all.

ModelAt index time only. The vision pass runs when files are scanned; a search never calls a model, and the browser never sees one.

Corpus

Over 2,500 files, roughly fifty gigabytes of decks, leaflets, module screenshots, event photography and brand marks, on the local disk where they already lived. The database holds an index and nothing else, and only thumbnails plus a selected tier of client-facing originals are ever uploaded, which is the reason a fifty-gigabyte archive is indexed and served out of a free-tier database sitting at about 930 MB against a one-gigabyte ceiling.

Ingestion

On demand, one refresh script, run when the library changes rather than on a clock. The scan is read-only and opens every file to determine what it actually is rather than trusting its extension: page counts and physical millimetre dimensions from PDFs, slide counts from presentations, pixel dimensions from images, duration from video. The load is an idempotent upsert keyed on relative path, and every column is classed as either derived-from-scan or earned-elsewhere. The first version deleted and reinserted the table on each run, which re-issued every primary key, and because that key is the thumbnail’s object name in storage, one rerun orphaned over 2,400 thumbnails.

Index

A row per file, plus two structures built specifically to be queried. The vision output lands in a JSON column under a GIN index, and a generated column concatenates the scene description, the keywords and any text read out of the image into a single string covered by a trigram index. A card-level rollup sits above that, so the searchable unit is the material a person asks for rather than the file it happens to live in.

Query

A short phrase and a handful of filters. The search box takes plain words, and beside it sit array containment on language and year, an image-type selector and a hero-only toggle. Nothing is parsed or rewritten, because the queries are short concrete nouns, a product or a venue or a piece of equipment, and expanding them would only widen a result set already small enough to read.

Selection

The mechanism is three ordinary SQL passes, run in order and merged: card titles first, then the vision text, then filenames. The shaping is deliberate and slightly blunt. A title match is held above the rest, the vision text is rolled up per card so a card matches when anything beneath it does, and duplicates collapse into one row. There is no embedding, no similarity score and no reranking pass, and the honest cost of that shows in a known defect: results order by section and asset count rather than by relevance, so a broadly matching collection can still sit above the exact hit.

Grounding

The model is only ever asked to describe a picture it is looking at, and its answer is constrained to a fixed shape: one scene line, three to six keywords, one classification from a closed six-way list, a people flag, a usability verdict of hero, support or reject, and whatever text is legible in the frame. It is never asked a question, so it has nothing to answer beyond the image. The search returns rows rather than prose, which means a hit can be explained by pointing at the field that matched. Above all of it sits a template layer that holds the facts a deck is allowed to state, one structured file per product, so a corrected number is corrected once instead of in seven PDFs.

Present — this layer exists and runs.Absent by decision — the layer is not there, and the sentence beside it is the reason. Every one of the six is answered on every system in this chapter, so the rows can be read across pages.

06Numbers

Verified — counted from the live index, the repository and measured runs. Archive figures are given as magnitudes because the exact counts are company data. Projected — no before-and-after instrumentation exists and the application collects no usage analytics. Both rows assume the prior method was manual traversal of a fifty-gigabyte tree with non-descriptive filenames, at ten to twenty lookups a week. The direction is well supported by the architecture; the magnitude is not measured.

07Timeline

1. 2026-07A first version is built and running, and a vision pass over the archive produces roughly 3,700 tagged rows.
2. 2026-08That version is torn down completely and rebuilt around four categories: duplicates removed, over 1,500 files reorganised with undo manifests, thumbnails and selected originals moved into two private buckets, and the card interface shipped.
3. 2026-08The delete-and-reinsert loader severs storage links across the library; the loader is rewritten as a natural-key upsert.
4. 2026-08The structured content layer begins — one file per product plus a company-level file, feeding a generator that produces pages from source data.
5. 2026-08Vision tagging reaches full image coverage, the search layer is wired to the tags and shipped, the year gap closes to zero, and the master deck is reviewed down by thirty-eight per cent.

08Looking back

What broke

The loader identity bug is the headline, and it had two relatives: a folder reorganisation silently erased vision tags, and a year backfill done in the database was overwritten on the next scan. All three are the same lesson — any column not derived from the scan has to be explicitly defended against the next scan. Two others cost real days. Signed URLs broke against object keys containing spaces, ampersands or Chinese characters, and the URL the storage API returned was itself unusable because it left spaces unencoded while encoding characters the signature had been computed over. And concurrent editing of the deck review state from a terminal and a browser, with no version field, silently overwrote an entire chapter of decisions before anyone noticed.

What it changed

The common thread is that none of these announced themselves. Nothing crashed; the data was simply wrong afterwards, which is the expensive kind of failure. A presentation export made the same point in miniature by quietly producing nineteen fewer slides than the deck contained, because hidden slides were skipped and the documented option to include them does nothing. What changed in my practice is that I now write down, per column, who is allowed to write it — and I treat a silent success as an unverified one until something independent reads it back.

Where it stands

Live behind a password gate and in use by the marketing and business development teams. Internal, no revenue. Two known items are open: search ranks by section and asset count rather than relevance, so a broad collection can surface above an exact match, and the one-click deck generator is deliberately paused pending a decision about what it should assemble from.