WTD Virtual KOL
AI Engineering
An AI short-form video pipeline built by a working director, with the line between what a machine may make and what a human must still do written down before anything is published.
01The problem
A content and video production company has the economics of every small studio: revenue is capped by how many shoots the team can physically run, and the marketing that promotes the studio’s own products is the first thing cut when client work lands. In mid-2026 the obvious answer looked like fully automated AI short-form video — generate clips, publish at volume, collect ad revenue.
Reading the market properly killed that idea twice over. The platform’s inauthentic-content policy makes fully synthetic, no-human-input channels a termination risk, and the ad-revenue arithmetic never worked in the first place: at the going short-form rate a clip needs roughly 599,000 views to produce a thousand US dollars a month. So the real question was not whether a machine can make video. It was which part of a production pipeline a machine may legitimately take over, and what a human still has to do for the result to be publishable at all.
02What I built
A working pipeline that produces short-form video end to end using tools already licensed, plus the part that matters more than the code: a monetisation-safety framework that classifies every production choice by how likely it is to get a channel demonetised, applied as a checklist rather than filed as a document. Stage one is proven end to end. The human-craft, publishing and funnel stages are designed and not yet running.
- A self-contained asynchronous task client for the video service: submit, poll every ten seconds up to a thirty-minute ceiling, download the result, with flags for reference inputs, aspect ratio, duration, resolution, seed and last-frame extraction.
- An audio mastering chain — highpass, two equaliser bands, compression, echo and loudness normalisation to the broadcast-standard target — that turns flat generated audio into something platform-ready.
- A true-physical-loop prompt pattern: the machine returns to its start position while the crushed material stays crushed, engineered for rewatch without lying about physics.
- A persona layer: an AI-generated synthetic host composited over the clip, animated on a rented GPU because the managed APIs refuse synthetic faces outright.
- A typography path that renders Chinese text as transparent image cards and composites them, because the video model garbles rendered Chinese.
- A three-tier monetisation-safety classification applied before production, not after: which shortcuts are safe, which are marginal, and which end a channel.
Drawn, not captured. The rail is the four pipeline stages, the large rectangle is the composite frame with an inset for the synthetic host, and the list below is the render queue with one status mark per row. Nothing here is a video frame or a likeness, and every value is illustrative: one solid mark stands for the single stage proven end to end, the rest are unrun.
03As a product
- Who buys it
- Brands and agencies that need continuous social video volume but cannot afford a shoot per post, and product businesses that want video as a traffic engine rather than as a revenue line. Internally the buyer is the studio itself and an in-house e-commerce brand.
- Value
- The visual generation cost of a short drops to roughly the price of a coffee, while the parts that decide whether the channel survives — sound, edit rhythm, disclosure, curation — stay human. It is AI applied to the expensive but mechanical half of production, by someone who knows which half that is.
- Positioning
- One-click commercial tools charge a monthly floor plus per-clip fees and hand back the same template output everyone else is publishing. This runs on tools already licensed and differentiates on prompt craft, an unsaturated subject niche, and a persona layer the mainstream APIs will not produce at all.
- Status
- Internal tool, prototype. Not sold, not licensed, and not publishing — no channel exists, so there are no views, subscribers or earnings of any kind.
04How it works
- Stack
- A shell task client over the video service API; a commercial text-to-video and image-to-video model with native synced audio; ffmpeg for mastering, compositing and overlay; a Python imaging library for typography cards; a rented consumer GPU running an open-source character animation model for the persona stage. Credentials live in a gitignored store outside the repository and are read at runtime, never passed on the command line.
- Shape
- Single-operator, batch of one. Each clip is a discrete asynchronous task; the GPU stage is rented by the hour and torn down after use. Output is capped at fifteen seconds, and the reference inputs are capped per media type rather than per item.
Width uniform throughout — the reference caps are counts of files allowed in rather than volumes measured flowing through, and nothing else in this pipeline has ever been counted. Every band is therefore drawn at equal width, and the split at the persona level claims no proportion of any kind.
The budget at the top and the refusal in the middle are the same argument. What the model may be shown is decided before the request exists, and the fifteen-second ceiling is spent per media type rather than per file. At the persona level the face-reference channel is switched off permanently, so the pose lane carries on and the face lane stops where it is drawn — motion may be copied from a reference clip, an identifiable human face may not. Nothing leaves without the human pass at the end.
The decision I spent longest on
The video API refuses any reference image it believes contains a real human face — and it refuses AI-generated realistic faces too, which killed the persona format at the API layer. I could abandon the persona, fake it with a static inset, or move that one stage onto a rented GPU running an open model with no content filter. I took the third, and drew the compliance line inside the same decision: the open model’s face-reference toggle, which bleeds a real performer’s features into the output, is switched off permanently. Motion may be copied. A face may not. That constraint was chosen over capability, and it is why the route is defensible rather than merely clever.
05Retrieval architecture
Nothing in this pipeline is looked up. A generative system has the retrieval problem inverted: the model holds no memory between runs and no store to consult, so the only question that carries any weight is what a single request is allowed to put in front of it.
Three of the six layers below are empty for that reason, and the weight of the whole architecture falls on one of the three that remain. The budget is the design — what the model may be shown is capped before a request is built, and what leaves is stopped by a person.
ModelAt query time only. The generation model runs once per clip, at the moment a task is submitted, and nothing of ours is running when a clip is not being made.
- Corpus
- None. There is no body of material to draw on and nothing that accumulates. Reference files sit loose on disk with no database in front of them, and nothing a previous run produced is available to the next one unless a person hands it over. That is deliberate rather than unfinished: the pipeline generates rather than recalls, and a corpus would imply the model can consult something it has seen before, which it cannot. Its entire world is the payload of one request, and the only continuity across runs is a person deciding to carry something forward.
- Ingestion
- By hand, one task at a time. A storyboard prompt with per-second beats is written per clip, and the reference files are pulled out of a folder and tagged with the role each one plays — a look, a motion, a sound — so the model is told what it is being shown rather than left to guess. Two things are refused at the door: a request carrying only audio, because something visual has to anchor it, and any reference image the service believes contains a real human face.
- Index
- None. Nothing here lives long enough to be worth structuring. The whole working set for one clip is a handful of files an operator can see in a single folder, addressed by filename and assembled fresh for each task, then never queried again. An index over that would be a catalogue of six files maintained beside the six files, rebuilt every time a new reference was dropped in — a build step and a staleness problem bought in exchange for a lookup nobody performs. The folder is the index, and one operator can hold it in their head.
- Query
- None. Nothing arrives as a question. The storyboard prompt is language, but it is an instruction rather than a query: it is handed to the model verbatim to be executed, not parsed, expanded, classified or matched, because there is nothing on the other side for it to be matched against. No request is routed and no intent is inferred. The person who wrote the prompt also chose every file attached to it, so there is no gap between what was meant and what was asked, and no layer is needed to close one.
- Selection
- The mechanism is a hard budget, enforced before the request is assembled: at most nine reference images, at most three reference video clips and at most three audio clips, with the fifteen-second ceiling applied to each media type as a whole rather than to each file — so three clips means splitting fifteen seconds three ways, and deciding which three seconds of motion actually matter is the work. What comes back is shaped by pinning rather than by ranking: one persona still is the anchor image on every run, the seed is pinned so a take that worked can be re-rolled with the same look, and the closing frame of one clip is extracted and handed to the next as its opening reference. Nothing is scored, so there is no ordering that could be reranked; the only narrowing that happens after the fact is at the persona stage, which takes an abstract pose representation from its reference clip and nothing else.
- Grounding
- A person, and a rule about faces. Every clip goes through a human sound and edit pass before publish, with no exceptions, and that pass is the reason the route is defensible rather than merely clever. The three-tier monetisation-safety classification is applied as a production checklist before anything is made rather than as a review afterwards. And the face-reference channel on the persona model is switched off permanently, so motion may be copied from a reference clip and an identifiable human face may not. Nothing has been published, so none of this has yet had to hold up in public.
Present — this layer exists and runs.Absent by decision — the layer is not there, and the sentence beside it is the reason. Every one of the six is answered on every system in this chapter, so the rows can be read across pages.
06Numbers
| Measure | Figure | Basis |
|---|---|---|
| Prompt reliability, impossible-materials series | 2 of 3 | Verified |
| Generation cost per clip | CNY 1–2 | Verified |
| Generation latency per clip | 4–8 min | Verified |
| Rented GPU rate, community tier | US$0.34/hr | Verified |
| Delivery loudness after mastering | -14 LUFS | Verified |
| Pipeline stages proven end to end | 1 of 5 | Verified |
| Market benchmark: views per clip for US$1,000 a month of short-form ad revenue | 599,000 | Verified |
| Blended generation cost per publishable short | CNY 4–8 | Projected |
| Persona animation per five-second clip | under US$1 | Projected |
Verified — measured in test runs and read off the mastering chain and the service’s own pricing. The 599,000-view figure is the market benchmark that justified rejecting the ad-revenue model — it is not traction. Nothing has been published, so this project has zero views and zero revenue of its own. Projected — modelled from the observed two-in-three usable rate and the verified hourly rate, counting generation only. No human time, sound recording or editing is included, and failed runs and idle rented time are excluded — both were significant during the proof of concept.
07Timeline
- 2026-06Project opened. The end-to-end pipeline is validated in a single session, the market analysis is completed, the safety framework and the true-physical-loop rule are written, and the persona blocker is found and routed around on paper.
- 2026-06First real proof of concept rendered on a rented GPU: a reference clip drives the synthetic host, with the working parameter recipe and the infrastructure lessons recorded.
- 2026-07The Chinese text overlay path is solved and folded into a reusable video skill.
- 2026-08The image-generation fallback is re-verified after the primary route’s credits lapsed. The project sits at roughly thirty per cent of its designed scope and has not moved since.
08Looking back
What broke
Almost everything on the self-hosted half. The two operating modes of the character model were understood backwards at first, producing the wrong output for several runs. Quality collapsed at low step counts until a step-distillation adapter was added, and the wrong sampler blurred faces regardless. An upscaler ran out of memory at seventy-seven frames and took the whole process down. Rented pods that had been stopped often would not restart because no GPU of that class was free, forcing a delete-and-redeploy against the persisted volume. A content delivery layer in front of the pod proxy blocked job submission entirely, so prompts had to go over an SSH tunnel. Packages installed into the container rather than the volume had to be reinstalled on every new pod.
What it changed
It converted an ambition — AI prints content passively — into an operating model I can defend: AI-assisted, human-curated. The safety framework is the part a client would actually pay for, because it names in advance which shortcuts end a channel: raw generated clips with automated speech, voice clones, reposted files and stock templates are the red tier; non-verbal loops with only synthetic sound and high-frequency auto-posting are amber; AI visuals with human-recorded sound, real edit decisions and correct disclosure are green. The true-physical-loop rule came from the same instinct in a different register. A reversed clip loops perfectly but shows a crushed object un-crushing, which is a lie about physics that viewers feel even if they cannot name it. So the machine returns to its start and the material stays destroyed — a rhythm loop rather than a perfect one, because the perfect one is not honest.
Where it stands
Prototype, paused at the point where the remaining work is human work: recording sound, cutting, and choosing what the funnel should end at. The generation half is proven and reusable, and has already been factored out into a general-purpose video tool used on unrelated projects.