Shipping LLM Prompts as a Build Artifact, Not a Doc

When we decided our Angular component library should be usable by AI coding assistants (Claude Code, Cursor, Windsurf), the first instinct was the obvious one. Write good markdown docs, point the assistant at them, done. That lasted about as long as the next unrelated PR that renamed a component input.

Documentation written for a human reader tolerates staleness in a way documentation for an LLM consumer doesn’t. A person hits a wall, greps the source, moves on. An AI agent takes the doc at face value and generates code against it, confidently and wrong. We watched this happen directly. An agent used TemplateDirective instead of the correct TableTemplateDirective, and not because the docs were silent: a YAML authoring mistake (a gotcha string containing : that got parsed as a mapping instead of plain text) rendered the guidance as [object Object]. The agent did what any reasonable reader does with garbage input. It ignored it and guessed.

That incident, plus a handful like it, forced a shift in how we thought about the problem. If an LLM is going to be a first-class consumer of documentation, that documentation needs the rigor we already give source code: a single source of truth and a gate that fails loud when the compiled output would diverge from reality.

What we built. Every component in the library gets a prompts/<component>.prompt.yml holding its public API surface, known gotchas, accessibility notes, and canonical usage. Cross-cutting concerns get their own YAML files rather than being folded into every component: _tokens.prompt.yml for design tokens, _mixins.prompt.yml for shared SCSS mixins, _utils.prompt.yml for export utilities, _charts.prompt.yml, and _library.prompt.yml for governance rules such as never using native HTML when a component exists.

A generator script, generate-prompts.ts, compiles all of this into two outputs: a bundled components/llms.txt shipped inside the published npm package, so any AI tool gets usable context with zero setup, and a set of per-component prompts/generated/*.prompt.md files consumed by an MCP server for tools like get_component and search_components.

The generation step is explicit:

pnpm run generate:prompts

It runs prompts/scripts/generate-prompts.ts, reads all YAML sources, and emits the generated per-component prompt files plus the bundled components/llms.txt. The generated prompt files are gitignored and rebuilt every time.

Neither output is meant to be hand-edited, ever. They’re gitignored precisely so nobody is tempted to patch a generated file and have it look fine until the next regeneration silently reverts the patch. The sanctioned way to change AI-facing behavior is: edit the YAML, run pnpm run generate:prompts, then run:

pnpm run generate:prompts:check

That verification step diffs the regenerated output against what’s committed. It makes “I edited the YAML” provable in CI, distinct from “I hand-edited the generated file and it happens to look right.”

The generation pipeline treats YAML the way the Angular compiler treats .ts: a compile input with a checked contract, not prose. Anything regeneratable from source shouldn’t be reviewed as a diff, only its source should be. And generate:prompts:check is what actually enforces this. Without it, nothing stops someone from hand-patching the generated .md and having it silently diverge from the YAML the next time CI regenerates.

The gate that actually enforces it. Good intentions don’t stop drift. CI does. A prompt-contract check walks every YAML’s declared module, import_name, and source fields and validates them against the real TypeScript export graph, built by export-graph.ts, which follows export * chains and named re-exports through the actual compiled library. If a YAML claims a component is importable from a path where it no longer lives, the build fails before that stale claim ever reaches a published llms.txt.

The check lives under tools/test-runner/drift-check/. An automated export-coverage + prompt-contract gate, added 2026-06-12, fails CI when a prompts/*.prompt.yml declaration no longer matches the real export surface.

The basic shape of the contract check is:

// tools/test-runner/drift-check/prompt-contract.ts (shape)

for (const yml of promptYamlFiles) {
  const { module, import_name, source, internal } = parsePromptYaml(yml);

  if (internal) {
    assertRendersInternalBanner(yml);
    continue;
  }

  const exportsSurface = buildExportSurface(module); // export-graph.ts

  if (!exportsSurface.has(import_name)) {
    fail(`${yml}: import_name "${import_name}" not found in ${module} export surface`);
  }
}

Components that are legitimately internal, created via createComponent/portal, like column-filter-dropdown, get an explicit internal: true escape hatch. That skips the import check and renders an “Internal component” banner in the generated docs instead of a broken import assertion. A separate, coarser allowlist mechanism at the gate level covers other portal-created components such as badge, tooltip, and toaster.

There are two consumption paths: the shipped llms.txt for zero-setup AI context, and an MCP server, ui-mcp, that serves the same prompt content as live tools such as get_component, list, and search_components.

The rule that mattered most, in practice. When a generated component failed to compile during testing, the fast fix was always sitting right there: open the generated .component.ts, patch the broken line, move on. We banned that outright.

The entire point of the exercise is to prove the served documentation is sufficient for an agent to generate correct code unassisted, and patching the generated output masks the actual gap while invalidating the test.

The enforced loop instead is:

1. Find which YAML fact is missing or wrong.
2. Fix the YAML.
3. Regenerate everything.
4. Delete the broken generated test artifact.
5. Re-dispatch the agent from a clean slate.

Slower per incident, yes. It’s also the only version of the loop that improves the docs instead of laundering around them.

Concrete gaps came out of this process. Agents used TypeScript as casts inside Angular templates:

(valueChanged)="x.set($event.value as string)"

Angular’s template parser doesn’t support TypeScript’s as keyword for narrowing. That wasn’t documented anywhere the agent could see, so it became an explicit governance rule in _library.prompt.yml rather than something assumed as “obvious TS knowledge.”

Agents also reached for native HTML controls by default. Nothing in the prompts explicitly forbade <select> when an equivalent component existed, so the fix was an explicit negative rule, not just documenting the positive API.

And agents passed object items into select without the required [labelKey] and [valueKey] inputs. That became a concrete gotcha in prompts/select.prompt.yml:

# prompts/select.prompt.yml — gotcha entry that had to be added after a real agent failure

gotchas:
  - "Object items REQUIRE [labelKey] and [valueKey]. Without [valueKey], selectionChange emits the full object. Use primitive string arrays when possible."

At least four confirmed documentation gaps were caught by agents actually failing to generate correct code against the served prompt, which is the explicit purpose of the harness. The outcomes were documented from the sessions that built this, but they were not independently benchmarked with a stopwatch, so there is no performance claim attached to them.

One other drift class showed up alongside the prompt-YAML problem. sonar-project.properties sonar.coverage.exclusions and vite.config.mts coverage.exclude have to mirror each other, or files can silently report 0% coverage without any test failing. It has the same underlying shape: derived configuration diverging from its source of truth.

The YAML authoring trap was particularly instructive. A gotcha written as:

TypeScript import: import { ... }

was parsed by the YAML loader as a mapping because of the : sequence. The served prompt then rendered the value as [object Object], which contributed directly to the TemplateDirective versus TableTemplateDirective failure. The fix was simple: quote any YAML list item containing :.

Each of these failures became a permanent, quotable rule in the relevant YAML. Not a one-off fix: a standing fact the next agent, and the next contributor, inherits automatically.

So the rule we ended up with, and the one I’d hand to anyone whose docs have an LLM downstream of them: a wrong output gets fixed at the source, never at the symptom. Treat every “the agent got it wrong” moment as a documentation-gap ticket against the YAML, never a one-off code patch. Gate the YAML-to-export contract in CI so a renamed export or changed module path fails the build instead of shipping a stale llms.txt.

Everything else, the YAML, the generator, the contract gate, and the generated artifacts, exists to make that rule enforceable.

---

References & Sources

• Angular template syntax reference — for the as-cast limitation context
• YAML spec on flow scalars — background on the colon-parsing trap
• Repo-relative: prompts/scripts/generate-prompts.ts, tools/test-runner/drift-check/prompt-contract.ts, tools/test-runner/drift-check/export-graph.ts, CLAUDE.md “Prompt sync” table