Career Database Pipeline

01The problem

A career history lived in three places that never agreed: the portfolio site, the CV, and the LinkedIn profile. Each was edited by hand at a different moment, so a job title corrected in one place stayed wrong in the other two, and the errors only surfaced when someone external read them side by side. Reconciling the CV against a fresh LinkedIn export exposed three real factual errors, including a role filed under the wrong employer and an entire missing phase of a company I co-founded.

The website half was worse. After the portfolio was rebuilt as a static export, every text and ordering field for 38 projects lived inside a single 417 KB JSON file, so changing one project description meant hand-editing generated JSON. The clock was running too: the hosted original was due to expire, after which this repository is the portfolio’s only home. The gap was not writing. It was having one place where a fact is true, and a mechanical path from there to every surface that repeats it.

02What I built

A Postgres table is the single source of truth for every editable text and ordering field on the site, alongside two sibling tables that serve the CV and the profile. A Node script pulls the table over the REST API and regenerates only the editable slice of the content file, leaving the rest of the repository untouched. Editing happens in a private console rather than in SQL or in JSON. Publishing stays a separate, deliberate step, because the site is a static export and a bad write would otherwise ship itself.

• Nine fields owned by the database: title, nav label, description, meta description, date, listed, in-gallery, nav order and gallery order.
• An explicit slug bridge between two naming namespaces — the CV and archive naming on one side, the live published URL on the other.
• Null means no opinion: a null database field leaves the repository value alone rather than blanking it.
• Pass-through safety: any project with no match on the bridge is left exactly as it is, so nothing can silently disappear.
• Dry run by default. A write flag is required to touch the file, and the field-level diff is printed before that flag is honoured.
• A hard guard that aborts if the credentials point at the wrong database, because reading the wrong one fails silently rather than loudly.

03As a product

Who buys it

Anyone whose public professional identity is spread across a personal site, a CV and a social profile — consultants, agency principals, freelance directors — plus small studios maintaining a project roster on a marketing site.

Value

Edit once, in one place, and let the site, the CV and the profile derive from it. The failure it removes is not slow editing but three documents quietly telling different stories about the same person.

Positioning

A headless CMS solves the website half and knows nothing about a CV. A social profile is a silo with no export discipline. The hosted portfolio builder this replaced has no export and no API at all, and takes the site down with the subscription. This sits below all of them as a plain relational table any of those surfaces can be generated from.

Status

Internal tool, live. It generates the site you are reading. One user, not packaged, not priced.

04How it works

Stack

Next.js static export with TypeScript, Supabase Postgres over PostgREST, a plain Node script for the pull, Vercel for hosting.

Shape

One table, one script, one gate. No CMS runtime, no webhook, no build server logic — the generated file is committed, so every publish is a reviewable change.

The decision I spent longest on

The plan was backwards and only the data revealed it. The intent was that the database overwrites the site — but the database’s year column is human CV prose and its summary field was generated boilerplate, so running the sync as designed would have replaced clean years and real descriptions with worse ones. The fix was to backfill the site into new site-specific columns first, reverse the direction, and keep the CV date and the site date as deliberately separate columns that are never synced onto each other.

05Retrieval architecture

Nothing is retrieved here in the sense the word usually carries, because nothing here is uncertain at the moment it runs. The problem this pipeline actually solves is the older one that retrieval borrows from — record linkage — and it was solved once, by hand, then frozen so that it could never be solved differently a second time.

That freezing is the argument. A match that is recomputed is a match that can change silently, and a published URL is not allowed to change: it is already in the world, inside other people’s links. So the expensive, fuzzy, one-off work happens before anything runs, and what runs is a keyed lookup with no opinion of its own.

ModelNo model anywhere. Nothing in this pipeline calls a model at any point — not to reconcile the two namespaces, not to merge a field, not to write a line of the generated file.

Corpus

Three Postgres tables holding one career history: the project table that feeds this site, with nine editable fields per project, alongside the roles and skills tables that feed the CV and the profile. The repository owns the other half — covers, rollovers, page modules and 1,813 media files — and the two halves are never allowed to own the same field.

Ingestion

By hand, before a deploy, and never on a schedule. Edits are typed into a private console that writes one row and does not publish; a local Node script then pulls the whole table over the REST interface on demand. Its first act is a refusal: if the credentials do not point at the expected database project the run aborts, because two projects held identical copies of these tables after a migration, and reading the stale one produces no error at all.

Index

The bridge column is the index. The site slug is written once per project as an explicit column rather than derived at run time, and each run builds a plain map keyed by it. That map is the only structure the join needs, and it is deliberately dumb: 38 rows in, 38 keys out, no collisions and no second guess.

Query

None. A run takes no query. There is no request, no user and no phrase to interpret — the script pulls every row every time and decides what to do with each one, so nothing ever has to be turned into a search. The closest thing to a parameter is a single write flag, and that governs whether the generated file may be touched at all, not which rows are found.

Selection

Record linkage, run once. Two naming namespaces described the same 38 projects — the CV and archive naming on one side, the live published URL on the other — and were reconciled by three ordered strategies: 10 matched on an exact name, the remaining 28 on token and title similarity, and the last five of those fell out only by forced-unique elimination, one candidate left on each side, therefore the pair. What comes back is then shaped by the merge rule rather than by any score: mapped fields are applied one at a time and nulls are skipped, unmatched projects pass through untouched, and the result is re-sorted by the ordering column with the non-project nav entries left in place. Nothing is ranked, so there is nothing to rerank.

Grounding

There is no model to keep honest, so the gate guards the pipeline against itself. The script defaults to a dry run and prints a field-level diff, and an explicit write flag is required before the generated file can be touched. The acceptance test was that the first generated run had to produce zero field diffs against the file it replaced — and it earned its cost immediately, catching a rebuilt nav that emitted hrefs with a trailing slash the original lacked, and a gallery entry wiped because only project entries were being rebuilt. Underneath all of it sits one rule: a null is an abstention, not a delete.

Present — this layer exists and runs.Absent by decision — the layer is not there, and the sentence beside it is the reason. Every one of the six is answered on every system in this chapter, so the rows can be read across pages.

06Numbers

Verified — counted from the repository, the git log and the live database on 2026-08-12. The two match totals are exhaustive: 10 exact-name plus 28 similarity accounts for all 38, and the five resolved by forced-unique elimination are the tail of that 28, not a third group. Projected — estimated from the documented three-step flow. Nothing here is instrumented — no timing data is recorded and past hand-edits were never counted.

07Timeline

1. 2026-08The hosted portfolio is rescued as a self-owned Next.js static export, one repository, one deploy.
2. 2026-08The career database is reconciled against a fresh profile export. Three factual errors in the work history are corrected.
3. 2026-08The content file becomes generated rather than hand-edited, and the admin console gains a panel for the nine website fields — closing the source-of-truth-with-no-interface gap.
4. 2026-08The site is retitled and a fourth chapter, AI Engineering, is added.
5. 2026-08The pull script is found to be reading the wrong database. A hard assertion on the expected project replaces a comment.

08Looking back

What broke

The quietest failure was the wrong database. Two Postgres projects held identical copies of the career tables after a migration, so reading the stale one produced no error at all — it would simply have republished old content over new edits. The zero-diff acceptance test earned its keep the same week, catching a rebuilt nav emitting hrefs with a trailing slash the original lacked, and the gallery entry being wiped because only project-kind entries were rebuilt. Both would have shipped invisibly. Separately, the host silently rejects pushes whose commit author email is not a recognised account: no build is attempted, nothing looks broken, and the site keeps serving the previous version.

What it changed

The lesson I still carry is about secrets, and it is not the obvious one. A deploy command printed its own next-steps hints, and those hints reproduced a command-line flag with the literal secret value in standard output. The tool’s own helpfulness was the leak, not the input. Secrets go through the environment now, never a flag. More broadly, the ownership split became explicit enough that neither side can destroy the other’s data: the database owns words and ordering, the repository owns media, and a null is an abstention rather than a delete.

Where it stands

Live, and generating the page you are reading. One user, one table, and an editing path that no longer runs through a 417 KB generated file.