From 596355f6a4a3cb56f37614667e0e6dae915f7adf Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 11 Oct 2026 13:06:20 -0400 Subject: [PATCH 1/3] feat(codegen-ts,cli)!: shared-model, docs and api-docs can be ejected; the standalone mermaid-er generator is removed Every stock TypeScript generator can now be ejected except `template`, the Mustache primitive, which has no emit logic of its own to own. ADR-0034 Amendment 7 records the rulings and the reasoning. shared-model: the artifact a consumer's `meta deps sync` verifies stays in the package. `buildSharedModel` (load, select, closure check, serialize, core-vocabulary re-load) and `renderSharedModelFiles` (file names, hash, manifest) are public; the reference template calls both and owns which source files may contribute and where the published version comes from. docs and api-docs: `meta docs` gained a seam. An owned copy is listed in the docs door's own config, `docs: { generators: [...] }`, and carries a `docsSurface` marker saying which surface it takes over. `meta docs` runs it in place of the packaged generator and `meta verify --docs` checks against it. The facts the door used to pass as factory options ride on `ctx.docsDoor`. A docs generator placed in the top-level `generators` is skipped by `meta gen` with a warning that names `docs.generators`; an entry with no marker, or two for one surface, is refused by name. `meta eject docs` prints the docs block wiring and never the array `meta gen` runs. `buildApiModelForRun(ctx)` gathers the run facts the API model reads, so an owned copy picks up a new one on upgrade. mermaid-er: removed, by maintainer ruling. The standalone generator was the older home of a diagram `meta docs` still draws. `mermaidErDiagram()`, its `/generators` export, its registry row and the cross-port manifest row are gone; the diagrams `meta docs` writes are unchanged byte for byte. A config that still names `"mermaid-er"` or imports `mermaidErDiagram` fails with a message saying it was removed and that `meta docs` draws the diagram, whether the import is in the config or in a module the config imports. ADR-0022 Part 2 D1 is amended to match. The signals `meta gen`, `meta gen --list` and `meta verify` give about a doc generator under `generators` cover an owned copy too, and tell its owner to move it to the docs block where a packaged generator is removed: `runGen` reports it in `skippedDocGenerators` and names it in the new `skippedOwnedDocGenerators`, the next step and the `doc-generator-in-gen` advisory say "move", and a copy listed in the docs block marks its catalog row as wired there. The catalog withholds a `generators` import line by the `docsSurface` marker, not by layer. `meta verify` reports the removed name `"mermaid-er"` as refused, with the message `meta gen` fails with. Fixed while proving the drift check: `meta verify --codegen` passed generators neither the project's source files nor a way to read the project, so it failed with "no source files to load" for any project wiring `sharedModelFile()`. `runGen` takes `inputRoot` and generators read the project through `ctx.inputRoot`; `ctx.projectRoot` keeps its meaning. Packaged output is unchanged and gated to be: the byte-identity gate holds each new reference to the packaged generator, and holds the packaged docs generators to the same bytes under the door as from the same facts passed as options. --- AGENTS.md | 2 +- CHANGELOG.md | 99 ++++- .../dependencies-and-shared-models.md | 12 +- .../references/owning-generators.md | 2 +- .../references/typescript-docs.md | 19 +- .../references/typescript.md | 27 +- docs/features/cli.md | 20 +- docs/features/metadata-dependencies.md | 21 +- docs/features/own-your-codegen.md | 112 ++++- .../dependencies-and-shared-models.md | 12 +- .../references/owning-generators.md | 2 +- .../dependencies-and-shared-models.md | 12 +- .../references/owning-generators.md | 2 +- .../dependencies-and-shared-models.md | 12 +- .../references/owning-generators.md | 2 +- .../dependencies-and-shared-models.md | 12 +- .../references/owning-generators.md | 2 +- .../references/typescript-docs.md | 19 +- .../references/typescript.md | 27 +- .../dependencies-and-shared-models.md | 12 +- .../references/owning-generators.md | 2 +- .../references/typescript-docs.md | 19 +- .../references/typescript.md | 27 +- .../generator-registry-conformance/README.md | 2 +- .../registry.json | 7 - .../packages/cli/src/commands/docs.ts | 58 ++- .../packages/cli/src/commands/eject.ts | 123 ++++-- .../packages/cli/src/commands/gen.ts | 1 + .../packages/cli/src/commands/verify.ts | 2 +- .../packages/cli/src/lib/catalog-listing.ts | 24 +- .../packages/cli/src/lib/catalog.ts | 9 +- .../packages/cli/src/lib/codegen-drift.ts | 11 + .../cli/src/lib/doc-generator-advisory.ts | 53 ++- .../cli/src/lib/load-metaobjects-config.ts | 87 +++- .../typescript/packages/cli/src/lib/output.ts | 25 +- .../cli/test/catalog-factory-exports.test.ts | 35 +- .../packages/cli/test/eject-multi.test.ts | 111 ++++- .../cli/test/gen-docs-door-signals.test.ts | 220 ++++++++-- .../packages/cli/test/gen-list.test.ts | 19 +- .../test/integration/eject-docs-tier.test.ts | 371 ++++++++++++++++ .../integration/eject-shared-model.test.ts | 301 +++++++++++++ .../unit/removed-generator-diagnostic.test.ts | 106 +++++ .../packages/codegen-ts/src/docs-door.ts | 73 ++++ .../codegen-ts/src/generator-registry.ts | 36 +- .../packages/codegen-ts/src/generator.ts | 33 ++ .../src/generators/api-docs-file.ts | 40 +- .../codegen-ts/src/generators/api-model.ts | 29 ++ .../codegen-ts/src/generators/docs-file.ts | 25 +- .../codegen-ts/src/generators/index.ts | 6 +- .../codegen-ts/src/generators/mermaid-er.ts | 43 -- .../src/generators/shared-model-file.ts | 239 +--------- .../packages/codegen-ts/src/index.ts | 28 +- .../codegen-ts/src/metaobjects-config.ts | 51 ++- .../src/packaged-docs-generators.ts | 7 +- .../codegen-ts/src/reference-templates.ts | 10 + .../codegen-ts/src/reference/api-docs.ts | 117 +++++ .../packages/codegen-ts/src/reference/docs.ts | 244 +++++++++++ .../codegen-ts/src/reference/shared-model.ts | 121 +++++ .../codegen-ts/src/render-engine/doc-page.ts | 20 + .../packages/codegen-ts/src/runner.ts | 54 ++- .../src/shared-model/build-shared-model.ts | 298 +++++++++++++ .../codegen-ts/src/templates/mermaid-er.ts | 76 +--- .../test/generator-registry.test.ts | 36 +- .../test/generator-spec-resolution.test.ts | 49 ++- .../test/generators/mermaid-er.test.ts | 63 --- .../test/reference-byte-identical.test.ts | 412 +++++++++++++++++- .../test/reference-templates.test.ts | 1 + .../codegen-ts/test/runner-docs-shim.test.ts | 93 +++- .../test/templates/mermaid-er.test.ts | 29 +- ...2-codegen-and-docs-surface-architecture.md | 11 +- .../ADR-0034-codegen-scaffold-and-own.md | 125 +++++- 71 files changed, 3663 insertions(+), 747 deletions(-) create mode 100644 server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts create mode 100644 server/typescript/packages/cli/test/integration/eject-shared-model.test.ts create mode 100644 server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts create mode 100644 server/typescript/packages/codegen-ts/src/docs-door.ts delete mode 100644 server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts create mode 100644 server/typescript/packages/codegen-ts/src/reference/api-docs.ts create mode 100644 server/typescript/packages/codegen-ts/src/reference/docs.ts create mode 100644 server/typescript/packages/codegen-ts/src/reference/shared-model.ts create mode 100644 server/typescript/packages/codegen-ts/src/render-engine/doc-page.ts create mode 100644 server/typescript/packages/codegen-ts/src/shared-model/build-shared-model.ts delete mode 100644 server/typescript/packages/codegen-ts/test/generators/mermaid-er.test.ts diff --git a/AGENTS.md b/AGENTS.md index 03fd7527d..a12005242 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,7 +6,7 @@ MetaObjects is a **cross-language metadata standard** for declaring typed entity The metamodel is the **durable spine**; generated code is the **disposable artifact**. Substrate is local-first: typed metadata lives in your repo, generated code is idiomatic per-language output with **no proprietary runtime**: the generated entity and model code imports nothing from MetaObjects at runtime, and the REST, prompt, client and runtime tiers import ordinary Apache-2.0 packages you can vendor, fork or replace. If `@metaobjectsdev/*` disappears tomorrow, you keep working code. -**Core vs helpers (ADR-0034 Amendment 3, 2026-09-22).** Only the core is a product promise: the metamodel, loader, canonical format and registry; runtime metadata access (the `ObjectManager` and its drivers; the HTTP adapters that mount it on a web framework are helpers, 2026-09-24 ruling); schema migrations (`meta migrate`); the drift gates (`meta verify`); prompt render and the reply parser. Every generator that writes application code into an adopter's repo — routes, controllers, ORM wiring, DTOs, forms, grids, hooks, filter allowlists — is a **reference helper**: it must compile and pass its reference fixtures, and the adopter copies it with `meta eject` and owns the copy. Mechanical test: what the tool guarantees is core; what it writes into your repo is a helper. Every port can eject: `meta eject` (TypeScript), `metaobjects eject` (Python), `mvn metaobjects:eject` (Java/Kotlin), `dotnet meta eject` (C#). Not every generator ships a reference copy: `meta gen --list` marks the ones that do not `package-only` (in TypeScript: `template`, the docs tier and `shared-model`; ADR-0034 Amendment 3 correction, 2026-09-26, narrowed by Amendment 5). When you find a generator defect, fix the reference; do not describe it as a broken core guarantee. +**Core vs helpers (ADR-0034 Amendment 3, 2026-09-22).** Only the core is a product promise: the metamodel, loader, canonical format and registry; runtime metadata access (the `ObjectManager` and its drivers; the HTTP adapters that mount it on a web framework are helpers, 2026-09-24 ruling); schema migrations (`meta migrate`); the drift gates (`meta verify`); prompt render and the reply parser. Every generator that writes application code into an adopter's repo — routes, controllers, ORM wiring, DTOs, forms, grids, hooks, filter allowlists — is a **reference helper**: it must compile and pass its reference fixtures, and the adopter copies it with `meta eject` and owns the copy. Mechanical test: what the tool guarantees is core; what it writes into your repo is a helper. Every port can eject: `meta eject` (TypeScript), `metaobjects eject` (Python), `mvn metaobjects:eject` (Java/Kotlin), `dotnet meta eject` (C#). Not every generator ships a reference copy: `meta gen --list` marks the ones that do not `package-only` (in TypeScript: `template` alone; ADR-0034 Amendment 3 correction, 2026-09-26, narrowed by Amendments 5 and 7). When you find a generator defect, fix the reference; do not describe it as a broken core guarantee. ## Six pillars diff --git a/CHANGELOG.md b/CHANGELOG.md index 5b61aad1c..10f64f0e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -165,10 +165,10 @@ generator catalog and the Atlas evaluation are 1.2 work. own: which objects get a module and where it lands. The module body stays in the package (`renderCallableFile`, `renderTraceHelper`, now exported from `@metaobjectsdev/codegen-ts`), the same split the prompt tier uses. The rule is that every stock generator can be - ejected except the `template` primitive, which has no emit logic of its own to own. Not - yet ejectable in this change: `shared-model` and the docs tier (`docs`, `api-docs`, - `mermaid-er`) in TypeScript. The Java `entity` and `extractor` generators became - ejectable in ADR-0034 Amendment 6. + ejected except the `template` primitive, which has no emit logic of its own to own. + `shared-model`, `docs` and `api-docs` follow in their own entry below (ADR-0034 + Amendment 7), and the Java `entity` and `extractor` generators became ejectable in + ADR-0034 Amendment 6. - **Agent skills: a cost test for owned generators.** The `metaobjects-codegen` skill now says when not to write a generator: only when generator code plus its generated output and added metadata is fewer lines than the hand-written code it @@ -508,6 +508,40 @@ generator catalog and the Atlas evaluation are 1.2 work. docs` wording come from the packages, and each port's existing generators already type a required field as not nullable), so an owned generator needs no resync for this. See [docs/features/reporting.md](docs/features/reporting.md#rows-from-a-dimensions-entity). +- **`shared-model`, `docs` and `api-docs` can be ejected (TypeScript).** With these + three, every stock TypeScript generator can be ejected except `template`, the Mustache + primitive: you already supply its template and it has no emit logic of its own to own, + so it stays `package-only` (the C#, Java and Python ports rule the same way). The + standalone `mermaid-er` generator is not among them: it is removed (see **Removed** + below). Ruling and reasoning: ADR-0034 Amendment 7. Nothing changes for a project that + ejects none of them: packaged output is byte-identical, and gated to be. + - **`meta eject shared-model`** copies a thin generator. The copy owns which of your + source files may contribute and where the published version comes from. Everything a + consumer's `meta deps sync` verifies stays in the package, in two functions the copy + calls and `@metaobjectsdev/codegen-ts` now exports: `buildSharedModel` (load, select, + closure check, serialize, core-vocabulary re-load) and `renderSharedModelFiles` (the + file names, the hash, the manifest). A test publishes with an ejected copy and has a + consumer sync, lock and verify the result. + - **`meta eject docs` and `meta eject api-docs`** copy the generators behind + `meta docs`. List the copy in the docs block of `metaobjects.config.ts`, a new key: + `docs: { generators: [docsFile()] }`. It carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged generator for that + surface. `meta eject` prints exactly that wiring, and its JSON row carries + `door: "docs"` and `wire.into: "docs.generators"`. The copy owns which nodes get a page, + where a page lands, the overview page and which forms of the API reference are written. + A docs generator does not go in the top-level `generators`: `meta gen` skips one there + with a warning that names `docs.generators`. Two entries for one surface, or an entry + with no `docsSurface`, are refused by name. The `requirements` and `agent` pages stay + packaged. + - To change only the wording of a model page you still need no generator: a + `templates/docs/entity-page.md.mustache` in your project wins over the packaged one. + - Steps and what each copy owns: + [own-your-codegen](docs/features/own-your-codegen.md#the-docs-generators-and-shared-model-typescript). +- **`ctx.inputRoot` for a generator that reads the project's own files.** It is the real + project directory. Under `meta gen` it equals `ctx.projectRoot`. Under + `meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is + that tree, as before, because owned `entity` and `routes` copies compute an import path + against it. Read files through `ctx.inputRoot`; `runGen` takes it as `inputRoot`. ### Changed @@ -670,6 +704,27 @@ until you regenerate. One rule is relaxed: an authored `validator.length @min: 0` on a `@required` string now admits the empty string, as the generated schema already did. +### Removed + +- **The standalone `mermaid-er` generator (TypeScript).** `mermaidErDiagram()`, its + `MermaidErOptions` type and the stable name `"mermaid-er"` are gone from + `@metaobjectsdev/codegen-ts/generators`, the generator catalog and the cross-port + generator manifest. It wrote the whole-model ER diagram to a file of its own + (`docs/model.md`) under `meta gen`, and was the older home of a diagram `meta docs` has + drawn since the docs engine took it over: the whole model on the overview page + (`README.md`), and each entity's neighbourhood on that entity's page. Those diagrams are + unchanged, byte for byte. No other port shipped the generator. Ruling: ADR-0034 + Amendment 7; ADR-0022 Part 2 D1 is amended to match. + - **To upgrade**, if your `metaobjects.config.ts` wires it: delete the + `mermaidErDiagram` import and its entry in `generators` (or the `"mermaid-er"` name), + delete the file it wrote (`docs/model.md` under your output directory, unless you moved + it with `outFile`), and run `meta docs`. `meta verify --codegen` reports that file as + no longer generated until it is deleted. + - **A config that still wires it fails with a message that says this**, instead of the + runtime's own words for a missing export. The import is diagnosed in the config and in + a module the config imports, and the name is diagnosed as removed, not as unknown. + - A project that never wired it sees no change. + ### Fixed - **A doc generator skipped by `meta gen` is now reported in every output form.** `meta docs` @@ -692,23 +747,33 @@ until you regenerate. docs` `` with the reason under the row, a wired one reads ``WIRED but skipped by `meta gen` ``, and `--probe` no longer prints `would emit 0` for them. In `--format json` the row carries `note` and `door: "meta docs"`, and `project.wouldEmit` is `null` instead of `0`. - The `mermaid-er` row's note now states the back-compat fact: it is deprecated, a - `mermaidErDiagram()` already wired in `generators` still runs, and `meta docs` draws the - same diagram on its overview page. A neutral name wired as a string (`"docs"`, - `"mermaid-er"`), which `meta gen` refuses, reads ``WIRED by name, which `meta gen` - refuses`` (`project.refused` in `--format json`). + There is no `mermaid-er` row any more: the generator is removed in this release (see + **Removed** above). The neutral name `"docs"` wired as a string, which `meta gen` + refuses, reads ``WIRED by name, which `meta gen` refuses`` (`project.refused` in + `--format json`). - **`meta verify`** said nothing about a config wired this way. It now reports each one as an advisory row (rule `doc-generator-in-gen`, in `antiPatterns.rows[]`) that names `meta verify --docs` as the gate that checks documentation. A neutral name wired as a string is reported under rule `doc-generator-refused-by-gen`, because `meta gen` fails on - it rather than skipping it. Advisory: the exit code is unchanged. + it rather than skipping it. The removed name `"mermaid-er"` is reported under the same + rule, with the message `meta gen` fails with. Advisory: the exit code is unchanged. + - **An owned docs generator under `generators`** (a copy made with `meta eject docs` or + `meta eject api-docs`) is skipped by the same rule and reported in the same places. Its + fix is the opposite one, so each message says to move it to + `docs: { generators: [...] }`, where `meta docs` runs it, and not to remove it: the + runner's warning, the `help[]` next step and the `doc-generator-in-gen` advisory row. A + config that wires only such a copy is no longer told `no entities to generate`. Listed + in the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` + (`project.wiredInDocsDoor` in `--format json`). - **The unknown-generator error** listed `api-docs` among the names `meta gen` accepts. It no longer does, and it says documentation comes from `meta docs`. - `runGen()` returns a new `skippedDocGenerators: string[]` beside `warnings`, and - `@metaobjectsdev/codegen-ts` exports `isPackagedDocsGenerator` (skipped) and - `isNeutralGeneratorName` (refused), so a caller embedding the runner can ask rather than - match a sentence. TypeScript only: in the C#, Java, Kotlin and Python + `runGen()` returns a new `skippedDocGenerators: string[]` beside `warnings`, with + `skippedOwnedDocGenerators` naming the owned copies among them, and + `@metaobjectsdev/codegen-ts` exports `isPackagedDocsGenerator` (skipped), + `isNeutralGeneratorName` (refused) and `removedGeneratorMessage` (refused as removed), so + a caller embedding the runner can ask rather than match a sentence. TypeScript only: in + the C#, Java, Kotlin and Python ports documentation is a separate command (`dotnet meta docs`, `mvn metaobjects:docs`, `metaobjects docs`) and no doc generator is registered for code generation, so there is nothing to wire and nothing to skip. @@ -994,6 +1059,12 @@ until you regenerate. refused.** The preset check read `and` and `or` as field names ("references field 'or' which is not a filterable field"), so a valid composed preset failed to load. It now recurses into each composed clause and checks it like a top-level one, as TypeScript does. +- **`meta verify --codegen` can check a project that wires `sharedModelFile()`.** The gate + regenerates into a temporary tree and passed the generator neither the project's own + source files nor a way to read the project, so it failed with "no source files to load" + for a packaged or an owned `shared-model`, and would have taken the version default from + the temporary tree. It now passes what `meta gen` passes. A publisher's drift gate on its + shared artifact, which the dependency docs describe, works as described. ## [1.0.13] — 2026-10-03 diff --git a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/agent-context/skills/metaobjects-codegen/references/owning-generators.md b/agent-context/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/agent-context/skills/metaobjects-codegen/references/owning-generators.md +++ b/agent-context/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/agent-context/skills/metaobjects-codegen/references/typescript-docs.md b/agent-context/skills/metaobjects-codegen/references/typescript-docs.md index 41dff507a..8001ef660 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript-docs.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript-docs.md @@ -44,11 +44,20 @@ no docs are written by that run. The warning is on stderr, and in a structured r `meta verify` reports the wiring as an advisory row, rule `doc-generator-in-gen`. Remove the entry and run `meta docs`. -**Do not add `mermaidErDiagram()` to a new config.** It is deprecated: `meta docs` already -draws the ER diagram on its overview page with nothing wired. A `mermaidErDiagram()` -already wired in `generators` still runs under `meta gen`. The stable names `"docs"` and -`"mermaid-er"` written as strings in `generators` are refused, so `meta gen` fails on -them; `meta verify` reports that as rule `doc-generator-refused-by-gen`. +**An owned docs generator goes in `docs: { generators: [...] }`, never in `generators`.** +`meta eject docs` (or `api-docs`) copies the generator and prints that wiring; `meta docs` +and `meta verify --docs` then run the copy in place of the packaged one. Under +`generators` it is skipped like the packaged one, and the warning, the `help[]` next step +and the `doc-generator-in-gen` advisory all say to move it, not to remove it. Listed in +the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). + +**There is no `mermaidErDiagram()` generator.** 1.1 removed it: `meta docs` draws the ER +diagram on its overview page and on each entity's page with nothing wired. A config that +still imports it, or names `"mermaid-er"`, fails with a message that says so; delete the +entry and run `meta docs`. The stable name `"docs"` written as a string in `generators` +is refused as well, so `meta gen` fails on it. `meta verify` reports both names as rule +`doc-generator-refused-by-gen`. **Before calling any generated code, read `./docs/api/AGENT-API.md`** — it has the exact imports, signatures, payload field shapes, and runnable examples for this diff --git a/agent-context/skills/metaobjects-codegen/references/typescript.md b/agent-context/skills/metaobjects-codegen/references/typescript.md index c5903cffc..a716f62ad 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript.md @@ -126,10 +126,29 @@ name. Import by default and eject to customize: an owned copy you never change o behind the reference. (`entityFile` / `queriesFile` / `routesFile` / `barrel` were removed from that subpath at 1.0 and restored in 1.1, so on a 1.0.x install an owned copy is the only path for those four.) The engine primitives come from the package main entry, -`@metaobjectsdev/codegen-ts`. Every stock generator can be ejected except the `template` -primitive, which has no emit logic of its own to own. `shared-model` and the docs tier -(`docs`, `api-docs`, `mermaid-er`) are not yet ejectable; `meta gen --list` marks each -entry without a reference copy `package-only`. +`@metaobjectsdev/codegen-ts`. One entry cannot be ejected and is `package-only` in +`meta gen --list`: the `template` primitive, which has no emit logic of its own. + +Three ejectable entries behave differently from the rest: + +- **`docs` and `api-docs` are run by `meta docs`, not `meta gen`.** `meta docs` writes + both with nothing wired; never put a docs generator, packaged or owned, in `generators`. + To change the pages, `meta eject docs` (or `api-docs`) and list the copy in the docs + block, `docs: { generators: [docsFile()] }`: it carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged one. To change only + a page's wording, put your own `templates/docs/entity-page.md.mustache` in the project + instead. +- **The ER diagram comes from `meta docs`** (overview page, and each entity's page). The + standalone `mermaidErDiagram()` / `"mermaid-er"` generator was removed in 1.1; a config + that still wires it fails with a message saying so. Remove it and run `meta docs`. +- **`shared-model` ejects as a thin generator.** The copy owns which source files may + contribute and where the published version comes from. The artifact, its hash and the + manifest a consumer's `meta deps sync` verifies stay in the package + (`buildSharedModel`, `renderSharedModelFiles`); keep calling both. + +In an owned generator, read the project's own files through `ctx.inputRoot`. Under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. **`requirementTests()` — one test stub per requirement the model claims.** A recommended approach, not a contract: a `live` or `partial` requirement gets a stub that fails until diff --git a/docs/features/cli.md b/docs/features/cli.md index 77edfb434..4cc9e37b9 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -280,6 +280,14 @@ none emits every surface the project has something to say with. | requirements | `--requirements` | The declared ledger, as `requirements.md` + `requirements.toon`. | no | | agent | `--agent` | Three pages an agent reads BEFORE touching a tier (below). | yes | +The model and api surfaces each run one generator, and either can be replaced by a copy you +own: `meta eject docs` or `meta eject api-docs`, then list the copy in the `docs` block of +`metaobjects.config.ts`, `docs: { generators: [docsFile()] }`. `meta docs` runs it in place +of the packaged generator for the surface its `docsSurface` marker names, and +`verify --docs` checks the committed pages against it. The requirements and agent surfaces +are not generators and cannot be replaced. See +[own-your-codegen](own-your-codegen.md#docs-and-api-docs-are-run-by-meta-docs). + Two flags are NOT members of that set and do not compose with it: - **`--metamodel`** is a different MODE over a different subject: it documents the BUILT-IN @@ -578,10 +586,16 @@ name: a doc generator (`docsFile()`, `apiDocsFile()`) listed under `generators` (ADR-0025). The result then says so in `warnings[]` and adds a `meta docs` next step to `help[]`; `meta gen --list` marks those rows ``run by `meta docs` `` (`door` in `--format json`), and `meta verify` reports the same wiring as an advisory row with rule -`doc-generator-in-gen`. A neutral stable name written as a string (`"docs"`, -`"mermaid-er"`) is refused rather than skipped, so `meta gen` fails on it: `meta gen --list` +`doc-generator-in-gen`. An owned (ejected) docs generator listed there is skipped the +same way, and every one of those messages tells you to move it to +`docs: { generators: [...] }`, where `meta docs` runs it, instead of removing it; once it +is there, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). The neutral stable name `"docs"` written +as a string is refused rather than skipped, so `meta gen` fails on it: `meta gen --list` marks that row ``WIRED by name, which `meta gen` refuses`` (`project.refused` in -`--format json`), and the advisory row's rule is `doc-generator-refused-by-gen`. +`--format json`), and the advisory row's rule is `doc-generator-refused-by-gen`. The +name `"mermaid-er"` is refused too, as removed: `meta docs` draws the ER diagram, and +the message and the same advisory rule say so. In a structured run every narration line moves to **stderr**, so stdout is one parseable document. `--format` is honored by `gen`, `verify` and `migrate`; any diff --git a/docs/features/metadata-dependencies.md b/docs/features/metadata-dependencies.md index 9a81a171d..67f838da9 100644 --- a/docs/features/metadata-dependencies.md +++ b/docs/features/metadata-dependencies.md @@ -374,13 +374,16 @@ Because it is codegen output, it is **drift-gated**: `meta verify --codegen` in publisher's own build regenerates and diffs it, so a metadata edit that changes the public surface cannot ship unnoticed. -**`shared-model` is registered and discoverable, but deliberately not ejectable.** -It shows up in `meta gen --list` like any other generator, but `meta eject --list` -does not offer it (unlike `entity`/`queries`/`routes`/`barrel`, ADR-0034's four -scaffold-and-own generators). The artifact it emits is a contract whose bytes a -cross-port corpus pins and whose hash consumers verify — a user-owned, editable copy -of the generator would invite an artifact that silently stops matching what -consumers expect. +**`shared-model` can be ejected, and the artifact stays in the package.** +`meta eject shared-model` copies a thin generator you own: which of your source files may +contribute, and where the published version comes from. The artifact is a contract whose +bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer +checks is not in the copy. It stays in two functions the copy calls, `buildSharedModel` +(load, select, closure check, serialize, core-vocabulary re-load) and +`renderSharedModelFiles` (the file names, the hash, the manifest). An unmodified copy +publishes the same bytes as the packaged generator, and a test has a consumer sync and +verify what an ejected copy published. See +[own-your-codegen](own-your-codegen.md#shared-model-you-own-the-inputs-the-package-owns-the-artifact). **Only TypeScript publishes in Phase 1a — and only TypeScript and Python consume.** TypeScript and Python are the only two ports that can *consume* a dependency at all @@ -466,8 +469,8 @@ but **not** built in Phase 1a — do not assume any of the following exists: and plain-directory vendoring - [`abstracts-and-inheritance.md`](abstracts-and-inheritance.md) — `extends:` and `overlay:` within one project; the cross-repository case builds on the same rules -- [`own-your-codegen.md`](own-your-codegen.md) — generator ownership, and why - `shared-model` is registered but not ejectable +- [`own-your-codegen.md`](own-your-codegen.md) — generator ownership, and what an + ejected `shared-model` copy owns - [`cli.md`](cli.md) — the `meta deps` command surface and `verify --deps` - [`CONFORMANCE.md`](../CONFORMANCE.md) — per-port pass status for `dependency-conformance` diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index c6beb4ca9..201d23e36 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -157,6 +157,116 @@ What the diff in step 4 holds: Nothing requires this. An owned copy keeps working, and `meta eject --list` keeps telling you how far it is from the reference. +## The docs generators and `shared-model` (TypeScript) + +Every stock TypeScript generator can be ejected except one. `template` is the Mustache +primitive: you already supply its template, its walk and its name, and it has no emit +logic of its own, so a copy would own nothing. `meta gen --list` marks it `package-only`. +The C#, Java and Python ports rule the same way for theirs (ADR-0034 Amendment 7). + +Three of the ejectable ones differ from the rest in what runs them or in what the copy may +change. + +### `docs` and `api-docs` are run by `meta docs` + +`meta docs` writes the model pages and the API reference with nothing wired. Eject one of +the two generators behind it when you want to change which nodes get a page, where a page +lands, how the overview page is laid out, or which forms of the API reference are written: + +```bash +meta eject docs api-docs +``` + +```ts +import { docsFile } from "./codegen/generators/docs.js"; +import { apiDocsFile } from "./codegen/generators/api-docs.js"; + +export default defineConfig({ + // … + docs: { generators: [docsFile(), apiDocsFile()] }, +}); +``` + +The copies are listed in the `docs` block, which is `meta docs`'s own config, and not in +the top-level `generators`, which is what `meta gen` runs: + +- **`meta docs` runs a listed copy in place of the packaged generator.** Each copy carries + a `docsSurface` marker (`"model"` for `docs`, `"api"` for `api-docs`) saying which surface + it takes over. The summary line of `meta docs` names the generator, so you can see that + your copy ran. +- **`meta verify --docs` checks against it.** That gate calls `meta docs`, so an edit to + the copy shows up as drift in the committed pages until you run `meta docs` again. +- **One generator per surface, and each must say which.** Two entries for one surface, or + an entry with no `docsSurface`, are refused by name. +- **`meta gen` never runs one.** If a copy ends up in `generators`, `meta gen` skips it + with a warning that names `docs.generators`, and `meta docs` keeps running the packaged + generator until you move it. The run's `help[]` next step and the `meta verify` advisory + (rule `doc-generator-in-gen`) say the same: move it, do not remove it. +- **`meta gen --list` shows it.** A copy listed in the docs block marks its row + ``WIRED in `docs.generators` `` (`project.wiredInDocsDoor` in `--format json`). + +You can eject one and not the other. The two page sets link to each other, so `meta docs` +tells both generators the same facts on `ctx.docsDoor`: the api surfaces from your `docs:` +config, the directory of the api surface being written, and whether the model pages are +written. A copy reads them there and takes no options for them. + +| | The copy owns | Stays in the package | +|---|---|---| +| `docs` | which objects and templates get a page, where a page lands, which page template renders, the whole overview page (`renderIndexPage`) | the page payloads (`buildEntityDocData`, `buildTemplateDocData`), the ER diagram (`renderMermaidErBlock`), the page templates | +| `api-docs` | which units get a page, where a page lands, whether the index and the agent form (`AGENT-API.md`) are written | the model of the generated API (`buildApiModelForRun`, which also reads the run: layout, `apiPrefix`, whether Hono routes are wired) and its renderers (`renderEntityApiPage`, `renderApiIndex`, `renderAgentApi`) | + +Two things need no eject. To change the wording of a model page, put a file of the same +name in your project (`templates/docs/entity-page.md.mustache` or +`template-page.md.mustache`); it wins over the packaged one. And the packaged `docsFile()` +and `apiDocsFile()` are never listed anywhere: `meta docs` runs them when the `docs` block +names no generator for their surface. + +The `requirements` and `agent` pages of `meta docs` are not generators in the catalog and +cannot be ejected. + +### The ER diagram comes from `meta docs` + +`meta docs` draws the Mermaid diagrams: the whole model on its overview page (`README.md`) +and each entity's neighbourhood on that entity's page. An owned `docs` copy decides where +the whole-model diagram goes, by where it calls `renderMermaidErBlock`. + +Before 1.1 a standalone generator, `mermaidErDiagram()` (stable name `mermaid-er`), also +wrote the whole-model diagram to `docs/model.md` under `meta gen`. It was the older home of +the same diagram and has been removed. A config that still wires it fails with a message +saying so. To upgrade, delete the import and the `mermaidErDiagram()` entry (or the +`"mermaid-er"` name) from `generators`, delete the `docs/model.md` it wrote, and run +`meta docs`. + +### `shared-model`: you own the inputs, the package owns the artifact + +`sharedModelFile()` publishes part of your model as an artifact and a manifest that another +project's `meta deps sync` copies and verifies +([metadata dependencies](metadata-dependencies.md)). Its output is a contract with that +other project, so the copy is split the way `cube-model`'s is: + +| The copy owns | Stays in the package | +|---|---| +| which of your source files may contribute (`files`, default the run's own) | `buildSharedModel`: the load, the `include` / `exclude` selection, the closure check, the serialized document, the re-load with core vocabulary | +| where the published version comes from (`nearestPackageVersion` reads the nearest `package.json`; replace it with your own source of truth) | `renderSharedModelFiles`: the two file names, the artifact's hash, the manifest | + +```bash +meta eject shared-model +``` + +```ts +import { sharedModelFile } from "./codegen/generators/shared-model.js"; + +generators: [sharedModelFile({ name: "acme-common", include: ["acme::common::**"] })], +``` + +A copy that calls those two functions publishes exactly what the packaged generator +publishes, and a consumer's sync accepts it. Nothing a consumer checks is in the copy, so +you cannot change it there by accident; to change it you would have to replace the calls, +and then you have stopped agreeing with every consumer of the artifact. Read the project's +own files through `ctx.inputRoot`, as the copy does for the version: under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. + ## What `meta eject` hands over, and what stays core (TypeScript) Owning a generator means owning the code it writes **and the code that code calls**, @@ -422,7 +532,7 @@ generator code. The 20-line programmatic shape for each port is in | Port | Invocation | Programmatic — write a `Generator` | Declarative — template + scope | |---|---|---|---| -| **TypeScript** | `meta init` → `meta gen --list --probe` → `meta eject ` → `meta gen` (Bun/Node CLI) | **Yes.** `meta generator new ` writes a working generator of your own into `codegen/generators/` and wires it — the first move for an output nothing ships. To use a stock generator as shipped, import its factory from its package (`meta gen --list` prints the line) and wire it. To change one: `meta init` scaffolds the LAYOUT and an empty selection (ADR-0034 Amendment 2); `meta eject ...` copies each generator you choose into `codegen/generators/*.ts` and prints the import to put in `metaobjects.config.ts`. Edit them freely. The prompt tier (`prompt-render`, `output-parser`, `extractor`, `output-prompt`, `render-helper`) ejects like the rest: you own which templates get a module and where it lands, while the render and extract engines those modules call stay in the package. `requirement-tests` ejects as one file holding the generator and its default stub renderer, so the stub text is yours to change; the requirement walk, each test's identity and the claim digest stay in the package. `cube-model` (the Cube data-model exporter, [cube-export.md](cube-export.md)) ejects as one file as well: which entities get a cube, where the files land and the YAML call are yours, while the mapping (`buildCubeModel`, which raises every refusal) and the YAML writer (`renderCubeYaml`) stay in the package. `callable` and `trace-helper` eject the same way as the prompt tier: the copy decides which objects get a module and where it lands, and the module body comes from `renderCallableFile` / `renderTraceHelper` in the package. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own (the same ruling in every port). Not yet ejectable: the docs tier (`docs`, `api-docs`, `mermaid-er`) and `shared-model` (FR-023's publisher generator). Until each gains a reference copy, `meta gen --list` marks it **package-only** and `meta eject` names it as such. | **Yes** — `templateGenerator({ template, scope, outputPattern })` in the config's `generators: [...]`. No CLI flag: the config already takes generator values. | +| **TypeScript** | `meta init` → `meta gen --list --probe` → `meta eject ` → `meta gen` (Bun/Node CLI) | **Yes.** `meta generator new ` writes a working generator of your own into `codegen/generators/` and wires it — the first move for an output nothing ships. To use a stock generator as shipped, import its factory from its package (`meta gen --list` prints the line) and wire it. To change one: `meta init` scaffolds the LAYOUT and an empty selection (ADR-0034 Amendment 2); `meta eject ...` copies each generator you choose into `codegen/generators/*.ts` and prints the import to put in `metaobjects.config.ts`. Edit them freely. The prompt tier (`prompt-render`, `output-parser`, `extractor`, `output-prompt`, `render-helper`) ejects like the rest: you own which templates get a module and where it lands, while the render and extract engines those modules call stay in the package. `requirement-tests` ejects as one file holding the generator and its default stub renderer, so the stub text is yours to change; the requirement walk, each test's identity and the claim digest stay in the package. `cube-model` (the Cube data-model exporter, [cube-export.md](cube-export.md)) ejects as one file as well: which entities get a cube, where the files land and the YAML call are yours, while the mapping (`buildCubeModel`, which raises every refusal) and the YAML writer (`renderCubeYaml`) stay in the package. `callable` and `trace-helper` eject the same way as the prompt tier: the copy decides which objects get a module and where it lands, and the module body comes from `renderCallableFile` / `renderTraceHelper` in the package. `shared-model` (FR-023's publisher generator) ejects as a thin generator: the copy owns which source files contribute and where the version comes from, and the artifact a consumer verifies comes from `buildSharedModel` / `renderSharedModelFiles` in the package. `docs` and `api-docs` eject too, and `meta docs` runs the copy from `docs: { generators: [...] }`, never from the top-level `generators` ([details](#the-docs-generators-and-shared-model-typescript)). Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own (the same ruling in every port): `meta gen --list` marks it **package-only** and `meta eject` names it as such. | **Yes** — `templateGenerator({ template, scope, outputPattern })` in the config's `generators: [...]`. No CLI flag: the config already takes generator values. | | **Java / Kotlin** | `mvn metaobjects:generate` / `mvn metaobjects:verify` (`metaobjects-maven-plugin`) | **Yes.** Extend `FileEmittingGenerator` and read the model through `ModelWalk` (both in `metaobjects-codegen-base`) for one of your own. Every generator — built-in or your own — is named in `` and loaded from the project classpath: one seam, not two. There is no default suite, so `` is the complete list. Kotlin runs through the same goal. | **Yes** — `TemplateScopeGenerator` wired as an ordinary ``. No CLI flag: `` is already the seam. | | **C#** | `dotnet meta gen` / `dotnet meta verify` (.NET tool) | **Yes.** Implement `IGenerator` in the owned console project `codegen/` and list it in `codegen/Program.cs`; `dotnet meta gen` / `verify --codegen` hand off to that project whenever `codegen/Codegen.csproj` exists. `dotnet meta eject ` scaffolds the project, or write its two files by hand. An owned generator the `--generators` selection does not name still runs. | **Yes** — `dotnet meta gen --template-spec --template-root `. | | **Python** | `metaobjects gen` / `metaobjects verify` (console-script) | **Yes.** Name your generator as `module:symbol` in `--generators` or a target's `generators` in `metaobjects.config.yaml`; the symbol is an instance or a function returning one. Read the model through `metaobjects.codegen.model_walk`. (`--provider module:symbol` registers **metamodel vocabulary**, not a generator.) | **Yes** — `metaobjects gen --template-spec --templates `. | diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md index 41dff507a..8001ef660 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md @@ -44,11 +44,20 @@ no docs are written by that run. The warning is on stderr, and in a structured r `meta verify` reports the wiring as an advisory row, rule `doc-generator-in-gen`. Remove the entry and run `meta docs`. -**Do not add `mermaidErDiagram()` to a new config.** It is deprecated: `meta docs` already -draws the ER diagram on its overview page with nothing wired. A `mermaidErDiagram()` -already wired in `generators` still runs under `meta gen`. The stable names `"docs"` and -`"mermaid-er"` written as strings in `generators` are refused, so `meta gen` fails on -them; `meta verify` reports that as rule `doc-generator-refused-by-gen`. +**An owned docs generator goes in `docs: { generators: [...] }`, never in `generators`.** +`meta eject docs` (or `api-docs`) copies the generator and prints that wiring; `meta docs` +and `meta verify --docs` then run the copy in place of the packaged one. Under +`generators` it is skipped like the packaged one, and the warning, the `help[]` next step +and the `doc-generator-in-gen` advisory all say to move it, not to remove it. Listed in +the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). + +**There is no `mermaidErDiagram()` generator.** 1.1 removed it: `meta docs` draws the ER +diagram on its overview page and on each entity's page with nothing wired. A config that +still imports it, or names `"mermaid-er"`, fails with a message that says so; delete the +entry and run `meta docs`. The stable name `"docs"` written as a string in `generators` +is refused as well, so `meta gen` fails on it. `meta verify` reports both names as rule +`doc-generator-refused-by-gen`. **Before calling any generated code, read `./docs/api/AGENT-API.md`** — it has the exact imports, signatures, payload field shapes, and runnable examples for this diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md index c5903cffc..a716f62ad 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md @@ -126,10 +126,29 @@ name. Import by default and eject to customize: an owned copy you never change o behind the reference. (`entityFile` / `queriesFile` / `routesFile` / `barrel` were removed from that subpath at 1.0 and restored in 1.1, so on a 1.0.x install an owned copy is the only path for those four.) The engine primitives come from the package main entry, -`@metaobjectsdev/codegen-ts`. Every stock generator can be ejected except the `template` -primitive, which has no emit logic of its own to own. `shared-model` and the docs tier -(`docs`, `api-docs`, `mermaid-er`) are not yet ejectable; `meta gen --list` marks each -entry without a reference copy `package-only`. +`@metaobjectsdev/codegen-ts`. One entry cannot be ejected and is `package-only` in +`meta gen --list`: the `template` primitive, which has no emit logic of its own. + +Three ejectable entries behave differently from the rest: + +- **`docs` and `api-docs` are run by `meta docs`, not `meta gen`.** `meta docs` writes + both with nothing wired; never put a docs generator, packaged or owned, in `generators`. + To change the pages, `meta eject docs` (or `api-docs`) and list the copy in the docs + block, `docs: { generators: [docsFile()] }`: it carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged one. To change only + a page's wording, put your own `templates/docs/entity-page.md.mustache` in the project + instead. +- **The ER diagram comes from `meta docs`** (overview page, and each entity's page). The + standalone `mermaidErDiagram()` / `"mermaid-er"` generator was removed in 1.1; a config + that still wires it fails with a message saying so. Remove it and run `meta docs`. +- **`shared-model` ejects as a thin generator.** The copy owns which source files may + contribute and where the published version comes from. The artifact, its hash and the + manifest a consumer's `meta deps sync` verifies stay in the package + (`buildSharedModel`, `renderSharedModelFiles`); keep calling both. + +In an owned generator, read the project's own files through `ctx.inputRoot`. Under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. **`requirementTests()` — one test stub per requirement the model claims.** A recommended approach, not a contract: a `live` or `partial` requirement gets a stub that fails until diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md index 41dff507a..8001ef660 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md @@ -44,11 +44,20 @@ no docs are written by that run. The warning is on stderr, and in a structured r `meta verify` reports the wiring as an advisory row, rule `doc-generator-in-gen`. Remove the entry and run `meta docs`. -**Do not add `mermaidErDiagram()` to a new config.** It is deprecated: `meta docs` already -draws the ER diagram on its overview page with nothing wired. A `mermaidErDiagram()` -already wired in `generators` still runs under `meta gen`. The stable names `"docs"` and -`"mermaid-er"` written as strings in `generators` are refused, so `meta gen` fails on -them; `meta verify` reports that as rule `doc-generator-refused-by-gen`. +**An owned docs generator goes in `docs: { generators: [...] }`, never in `generators`.** +`meta eject docs` (or `api-docs`) copies the generator and prints that wiring; `meta docs` +and `meta verify --docs` then run the copy in place of the packaged one. Under +`generators` it is skipped like the packaged one, and the warning, the `help[]` next step +and the `doc-generator-in-gen` advisory all say to move it, not to remove it. Listed in +the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). + +**There is no `mermaidErDiagram()` generator.** 1.1 removed it: `meta docs` draws the ER +diagram on its overview page and on each entity's page with nothing wired. A config that +still imports it, or names `"mermaid-er"`, fails with a message that says so; delete the +entry and run `meta docs`. The stable name `"docs"` written as a string in `generators` +is refused as well, so `meta gen` fails on it. `meta verify` reports both names as rule +`doc-generator-refused-by-gen`. **Before calling any generated code, read `./docs/api/AGENT-API.md`** — it has the exact imports, signatures, payload field shapes, and runnable examples for this diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md index c5903cffc..a716f62ad 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md @@ -126,10 +126,29 @@ name. Import by default and eject to customize: an owned copy you never change o behind the reference. (`entityFile` / `queriesFile` / `routesFile` / `barrel` were removed from that subpath at 1.0 and restored in 1.1, so on a 1.0.x install an owned copy is the only path for those four.) The engine primitives come from the package main entry, -`@metaobjectsdev/codegen-ts`. Every stock generator can be ejected except the `template` -primitive, which has no emit logic of its own to own. `shared-model` and the docs tier -(`docs`, `api-docs`, `mermaid-er`) are not yet ejectable; `meta gen --list` marks each -entry without a reference copy `package-only`. +`@metaobjectsdev/codegen-ts`. One entry cannot be ejected and is `package-only` in +`meta gen --list`: the `template` primitive, which has no emit logic of its own. + +Three ejectable entries behave differently from the rest: + +- **`docs` and `api-docs` are run by `meta docs`, not `meta gen`.** `meta docs` writes + both with nothing wired; never put a docs generator, packaged or owned, in `generators`. + To change the pages, `meta eject docs` (or `api-docs`) and list the copy in the docs + block, `docs: { generators: [docsFile()] }`: it carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged one. To change only + a page's wording, put your own `templates/docs/entity-page.md.mustache` in the project + instead. +- **The ER diagram comes from `meta docs`** (overview page, and each entity's page). The + standalone `mermaidErDiagram()` / `"mermaid-er"` generator was removed in 1.1; a config + that still wires it fails with a message saying so. Remove it and run `meta docs`. +- **`shared-model` ejects as a thin generator.** The copy owns which source files may + contribute and where the published version comes from. The artifact, its hash and the + manifest a consumer's `meta deps sync` verifies stay in the package + (`buildSharedModel`, `renderSharedModelFiles`); keep calling both. + +In an owned generator, read the project's own files through `ctx.inputRoot`. Under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. **`requirementTests()` — one test stub per requirement the model claims.** A recommended approach, not a contract: a `live` or `partial` requirement gets a stub that fails until diff --git a/fixtures/generator-registry-conformance/README.md b/fixtures/generator-registry-conformance/README.md index b0e41b7cd..f921cc2be 100644 --- a/fixtures/generator-registry-conformance/README.md +++ b/fixtures/generator-registry-conformance/README.md @@ -35,7 +35,7 @@ to make the choice cheap, and `layer` is the axis it is cheap along. | `persistence` | queries, db-context, repository, exposed-table, relations, stored-proc | app shape | | `api` | routes, routes-hono, filter-allowlist, validator, spring-config | app shape | | `client` | form, hooks, grid, grid-hook | app shape | -| `docs` | docs, mermaid-er, api-docs | on by default (`meta docs`) | +| `docs` | docs, api-docs | on by default (`meta docs`) | | `capability` | prompt-render, output-parser, output-prompt, extractor, render-helper, trace-helper, requirement-tests, cube-model, shared-model, template, callable | `meta gen --list --probe` | Six, not ten. An earlier draft split `capability` into `trace` / `requirements` / diff --git a/fixtures/generator-registry-conformance/registry.json b/fixtures/generator-registry-conformance/registry.json index a318ef529..e283f70f2 100644 --- a/fixtures/generator-registry-conformance/registry.json +++ b/fixtures/generator-registry-conformance/registry.json @@ -203,13 +203,6 @@ "layer": "docs", "note": "Owned by `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended native suite.", "ports": ["typescript"] - }, - "mermaid-er": { - "concept": "Neutral Mermaid ER diagram of the entity/relationship model.", - "tier": "neutral", - "layer": "docs", - "note": "Neutral artifact owned by the docs engine (ADR-0020). Deprecated under `meta gen`: a `mermaidErDiagram()` already wired in `generators` still runs; `meta docs` draws the same diagram on its overview page.", - "ports": ["typescript"] } } } diff --git a/server/typescript/packages/cli/src/commands/docs.ts b/server/typescript/packages/cli/src/commands/docs.ts index a9eed1ef9..8cd8ece1f 100644 --- a/server/typescript/packages/cli/src/commands/docs.ts +++ b/server/typescript/packages/cli/src/commands/docs.ts @@ -45,6 +45,8 @@ import { renderCoreMetamodelDocs, } from "@metaobjectsdev/metadata"; import { dbEmittingObjects, DEFAULT_DIALECT, DEFAULT_DOCS_DIR, missingDialectMessage, resolveGenerators, runEmitsHonoRoutes, runEmitsUiTier, runRoutesExpose } from "@metaobjectsdev/codegen-ts"; +import { docsDoorGenerators } from "@metaobjectsdev/codegen-ts"; +import type { DocsDoorFacts } from "@metaobjectsdev/codegen-ts"; import type { MetaDataTypeProvider, MetaRoot } from "@metaobjectsdev/metadata"; import { generateSite, SITE_TEMPLATE_NAMES, SITE_ASSET_NAMES, readSiteFile } from "@metaobjectsdev/docs-site"; // The `agent` schema surface takes the physical schema as an ARGUMENT with its resolvers @@ -611,6 +613,30 @@ export async function docsCommand( warn: (msg) => log.warn(`docs: ${msg}`), }; + // An owned docs generator (ADR-0034 Amendment 7). It is listed in this door's own config, + // `docs: { generators: [...] }`, and says which surface it takes over by its + // `docsSurface` marker; for that surface it runs IN PLACE OF the packaged generator, and + // `verify --docs`, which calls this command, checks the committed pages against it. An + // entry with no marker, or two for one surface, is refused by name. + let ownedModel: Generator | undefined; + let ownedApi: Generator | undefined; + try { + ({ model: ownedModel, api: ownedApi } = docsDoorGenerators(loadedConfig?.docs?.generators ?? [])); + } catch (err) { + log.error(`docs: ${describeError(err)}`); + return 2; + } + // The context a surface's generator runs with. The facts the two page sets must agree + // on (where the api pages are, whether the model pages exist) ride on `ctx.docsDoor` + // rather than on factory options, because a listed generator is already constructed. Its + // own `filter` is applied as the gen runner applies one; the packaged generators carry + // none, so their context is the one above. + const doorCtx = (generator: Generator, facts: DocsDoorFacts): GenContext => ({ + ...ctx, + matches: generator.filter ?? ctx.matches, + docsDoor: facts, + }); + const emit: EmittedFile[] = []; let modelFiles: EmittedFile[] = []; @@ -632,9 +658,11 @@ export async function docsCommand( // polyglot model page references each port's api docs). `labeled` already // carries {label, subDir, baseUrl?} (docsFile ignores the extra `lang`), so // pass it straight through. Otherwise no api refs. - const modelOpts = apiSelected ? { apiSurfaces: labeled } : {}; + const modelGenerator = ownedModel ?? docsFile(); try { - modelFiles = await docsFile(modelOpts).generate(ctx); + modelFiles = await modelGenerator.generate( + doorCtx(modelGenerator, apiSelected ? { apiSurfaces: labeled } : {}), + ); } catch (err) { const msg = describeError(err); // Duplicate output path (silent-overwrite backstop): the generator already @@ -683,12 +711,15 @@ export async function docsCommand( try { // Emit every surface THIS port owns (loop so it generalizes beyond the // single ts surface owned today). + const apiGenerator = ownedApi ?? apiDocsFile(); for (const s of labeled.filter((s) => s.lang === "ts")) { apiFiles.push( - ...(await apiDocsFile({ - subDir: s.subDir, - modelSurface: docsCfg.surfaces.includes("model"), - }).generate(ctx)), + ...(await apiGenerator.generate( + doorCtx(apiGenerator, { + apiSubDir: s.subDir, + modelSurface: docsCfg.surfaces.includes("model"), + }), + )), ); } } catch (err) { @@ -773,11 +804,18 @@ export async function docsCommand( const modelTemplates = modelFiles.length > 0 ? modelFiles.length - entityCount - modelOverview : 0; - const modelSummary = docsCfg.surfaces.includes("model") - ? `${modelOverview} overview + ${entityCount} entity page(s) + ${modelTemplates} template page(s)` - : "model surface skipped"; + // An owned generator decides its own page set, so the overview/entity/template split + // above (which is the packaged generator's) is not claimed for it: its pages are counted + // and it is named, which also tells the reader that their copy is the one that ran. + const ownedBy = (g: Generator | undefined): string => + g === undefined ? "" : ` from the owned generator '${g.name}'`; + const modelSummary = !docsCfg.surfaces.includes("model") + ? "model surface skipped" + : ownedModel !== undefined + ? `${modelFiles.length} model page(s)${ownedBy(ownedModel)}` + : `${modelOverview} overview + ${entityCount} entity page(s) + ${modelTemplates} template page(s)`; const apiSummary = apiFiles.length > 0 - ? `${apiFiles.length} api page(s)` + ? `${apiFiles.length} api page(s)${ownedBy(ownedApi)}` : "no api pages"; // Reported only when it actually wrote something. A project with no ledger emits no // requirements file, and saying "0 requirement pages" would advertise a surface that diff --git a/server/typescript/packages/cli/src/commands/eject.ts b/server/typescript/packages/cli/src/commands/eject.ts index a4f5a092a..4258c2481 100644 --- a/server/typescript/packages/cli/src/commands/eject.ts +++ b/server/typescript/packages/cli/src/commands/eject.ts @@ -32,6 +32,7 @@ import { type RuntimeEjectResult, type RuntimeFileResult, } from "../lib/runtime-eject.js"; import { HTTP_RUNTIME_PACKAGE, OWNED_RUNTIME_DIR } from "@metaobjectsdev/codegen-ts"; +import type { GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; import { emitStructured, type OutputFormat } from "../lib/format.js"; import { ejectLibrary, ejectableLibraryNames, isLibraryName, libraryStaleness, @@ -111,6 +112,20 @@ function extractExportName(importLine: string, name: string): string { return match[1]; } +// The CALL a template's header documents, when it is not the bare `()`. A factory +// that cannot be called without options says so where it says how to import it +// (`// generators: [sharedModelFile({ name, include })]`), and a docs generator shows the +// docs block it is listed in (`// docs: { generators: [docsFile()] }`). Read from the +// header for the reason the import line is: the template is the one thing that knows, and +// the catalog deliberately does not derive a call from an options summary. +const HEADER_ENTRY_RE = /^\/\/\s+(?:docs: \{ )?generators: \[(\w+\(.*\))\](?: \})?,?\s*$/m; + +/** The call to wire: the header's, when it documents one for this template's own export. */ +function extractEntry(templateSource: string, exportName: string): string { + const call = HEADER_ENTRY_RE.exec(templateSource)?.[1]; + return call !== undefined && call.startsWith(`${exportName}(`) ? call : `${exportName}()`; +} + function extractImportLine(templateSource: string, name: string): string { const match = HEADER_IMPORT_RE.exec(templateSource); if (!match?.[1]) { @@ -143,6 +158,9 @@ export interface EjectResult { importLine: string; /** The symbol the template exports — the binding to REPLACE in the config. */ exportName: string; + /** The call to wire, with any options the factory cannot be called without + * (`sharedModelFile({ name, include })`), as the template's header documents it. */ + entry: string; /** The package the generator currently comes from, i.e. the import to remove. */ packageName: string; /** Advisory lines about packages the ejected file imports but the project lacks. */ @@ -243,6 +261,7 @@ export async function ejectGenerator(opts: EjectOptions): Promise { path: rel, importLine, exportName, + entry: extractEntry(templateSource, exportName), packageName: source.packageName, dependencyNotes: notes, templatePackages: requiredPackages(templateSource), @@ -466,13 +485,34 @@ async function listOutput(cwd: string): Promise { // the command // --------------------------------------------------------------------------- +/** + * Where an ejected generator's entry goes, and which command then runs it. + * + * gen the top-level `generators`, run by `meta gen` — every generator but two. + * docs-door `docs: { generators: [...] }`, run by `meta docs` (`docs`, `api-docs`). + */ +interface EjectWiring { + kind: "gen" | "docs-door"; + /** The call, with the options the factory cannot be called without. */ + entry: string; +} + +function wiringFor(entry: GeneratorRegistryEntry | undefined, call: string): EjectWiring { + return { kind: entry?.docsSurface !== undefined ? "docs-door" : "gen", entry: call }; +} + /** One row of the `--format json` payload. */ interface EjectedRow { name: string; path: string; status: EjectResult["status"]; - /** The two edits an adopter makes in `metaobjects.config.ts`. */ - wire: { import: string; entry: string }; + /** The two edits an adopter makes in `metaobjects.config.ts`. `into` is present when the + * entry does NOT go in the top-level `generators`: an owned docs generator is listed in + * the docs door's own block, and `into` names that key. */ + wire: { import: string; entry: string; into?: "docs.generators" }; + /** Present when the copy is run by `meta docs` (and checked by `meta verify --docs`) + * rather than by `meta gen`: the `docs` and `api-docs` generators. */ + door?: "docs"; requires: readonly string[]; /** The adapter source copied beside it (ADR-0034 Amendment 3). Absent when none. */ runtime?: RuntimeFileResult[]; @@ -537,7 +577,7 @@ function unknownNames(names: readonly string[]): string[] { /** Print the per-name text report — every branch below predates this command taking * more than one name, and each was written against a real incident. */ -function reportOne(result: EjectResult, name: string): void { +function reportOne(result: EjectResult, name: string, wiring: EjectWiring): void { if (result.status === "preserved") { // "already exists — left untouched" was the whole message, and it answered the // question nobody has. What an owner needs to know is whether their copy still @@ -594,30 +634,45 @@ function reportOne(result: EjectResult, name: string): void { log.info(`Ejected "${name}" -> ${result.path}. You own it now (ADR-0034 scaffold-and-own).`); } - // REPLACE, never "paste". A generator reaches `generators: [...]` under ONE binding, - // so a reader told to "paste" gets a duplicate identifier at best — and at worst - // deletes nothing, keeps `formFile()` in the array bound to the PACKAGE import, and - // silently runs the packaged generator while editing the ejected file. That failure - // is invisible and is the exact one ejecting exists to prevent. - // - // But eject reads no config, so it cannot know WHICH of the three states this project - // is in, and stating one of them as fact is wrong in the other two. Name the goal, - // then the three branches; the reader knows which one they are looking at. - log.info(`In metaobjects.config.ts, "${result.exportName}" must resolve to this file:`); - log.info(` ${result.importLine}`); - log.info( - ` - If it is imported from "${factoryModuleOf(name) ?? result.packageName}", REPLACE that import with the ` + - "line above. Adding a second one leaves `generators` bound to the PACKAGED " + - "generator, and your edits to this file do nothing.", - ); - log.info( - " - If it is already imported from ./codegen/generators/, it points here already " + - "— nothing to change.", - ); - log.info( - ` - If ${result.exportName}() is not in \`generators\` yet, add the import above ` + - "AND the entry.", - ); + if (wiring.kind === "docs-door") { + // A docs generator is the docs door's, so its wiring is stated in that door's terms + // and nowhere else: the `docs` block of the config, and the command that reads it. + // Sending its owner to the array `meta gen` runs would end in a skipped generator and + // a `meta docs` that still runs the packaged one. + log.info("`meta docs` runs this generator, in place of the packaged one. In metaobjects.config.ts:"); + log.info(` ${result.importLine}`); + log.info(` docs: { generators: [${wiring.entry}] }`); + log.info( + " If the `docs` block already lists docs generators, add this entry to that list. " + + "`meta verify --docs` then checks the committed pages against your copy. " + + "Run `meta docs` to write them.", + ); + } else { + // REPLACE, never "paste". A generator reaches `generators: [...]` under ONE binding, + // so a reader told to "paste" gets a duplicate identifier at best — and at worst + // deletes nothing, keeps `formFile()` in the array bound to the PACKAGE import, and + // silently runs the packaged generator while editing the ejected file. That failure + // is invisible and is the exact one ejecting exists to prevent. + // + // But eject reads no config, so it cannot know WHICH of the three states this project + // is in, and stating one of them as fact is wrong in the other two. Name the goal, + // then the three branches; the reader knows which one they are looking at. + log.info(`In metaobjects.config.ts, "${result.exportName}" must resolve to this file:`); + log.info(` ${result.importLine}`); + log.info( + ` - If it is imported from "${factoryModuleOf(name) ?? result.packageName}", REPLACE that import with the ` + + "line above. Adding a second one leaves `generators` bound to the PACKAGED " + + "generator, and your edits to this file do nothing.", + ); + log.info( + " - If it is already imported from ./codegen/generators/, it points here already " + + "— nothing to change.", + ); + log.info( + ` - If ${wiring.entry} is not in \`generators\` yet, add the import above ` + + "AND the entry.", + ); + } for (const line of result.dependencyNotes) log.info(line); if (result.runtime !== undefined && result.runtime.files.length > 0) reportRuntime(result.runtime.files); } @@ -721,15 +776,23 @@ export async function ejectCommand( runtimeGenerators.add(name); for (const p of result.runtime.packages) runtimePackages.add(p); } + const catalogEntry = catalog[name]; + const wiring = wiringFor(catalogEntry, result.entry); + const docsDoor = wiring.kind === "docs-door"; rows.push({ name, path: result.path, status: result.status, - wire: { import: result.importLine, entry: `${result.exportName}()` }, - requires: catalog[name]?.requires ?? [], + wire: { + import: result.importLine, + entry: wiring.entry, + ...(docsDoor ? { into: "docs.generators" as const } : {}), + }, + ...(docsDoor ? { door: "docs" as const } : {}), + requires: catalogEntry?.requires ?? [], ...(result.runtime !== undefined ? { runtime: result.runtime.files } : {}), }); - if (fmt === "text") reportOne(result, name); + if (fmt === "text") reportOne(result, name, wiring); } } catch (err) { log.error(describeError(err)); diff --git a/server/typescript/packages/cli/src/commands/gen.ts b/server/typescript/packages/cli/src/commands/gen.ts index 2cf6e15ef..46696185e 100644 --- a/server/typescript/packages/cli/src/commands/gen.ts +++ b/server/typescript/packages/cli/src/commands/gen.ts @@ -289,6 +289,7 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat antiPatterns, generatorCount: forgeConfig.generators?.length ?? 0, skippedDocGenerators: result.skippedDocGenerators, + skippedOwnedDocGenerators: result.skippedOwnedDocGenerators, ...(migrateAdvice !== undefined ? { migrateAdvice } : {}), }; const output = diff --git a/server/typescript/packages/cli/src/commands/verify.ts b/server/typescript/packages/cli/src/commands/verify.ts index f6f5fec03..29d22078a 100644 --- a/server/typescript/packages/cli/src/commands/verify.ts +++ b/server/typescript/packages/cli/src/commands/verify.ts @@ -1609,7 +1609,7 @@ export async function verifyCommand( let result; try { - result = await computeCodegenDrift(forgeConfig, codegenRoot, genConfigDir, genCollection.inScope); + result = await computeCodegenDrift(forgeConfig, codegenRoot, genConfigDir, genCollection.inScope, genCollection.ownFiles); } catch (err) { log.error(`verify --codegen: regeneration failed: ${describeError(err)}`); return 1; diff --git a/server/typescript/packages/cli/src/lib/catalog-listing.ts b/server/typescript/packages/cli/src/lib/catalog-listing.ts index ed3a1b343..4da5c7cb7 100644 --- a/server/typescript/packages/cli/src/lib/catalog-listing.ts +++ b/server/typescript/packages/cli/src/lib/catalog-listing.ts @@ -79,8 +79,8 @@ export interface GeneratorCatalogRow { install: { dev: string[]; runtime: string[] }; /** * How to run this generator AS SHIPPED: the import line for its package factory. No - * copy involved; `meta eject` prints the local import for an owned one. Absent on the - * `docs` layer, whose door is `meta docs`. + * copy involved; `meta eject` prints the local import for an owned one. Absent on a row + * with a `door`: `meta docs` runs that generator, and `meta gen` skips it. */ wire?: GeneratorWiring; source: { @@ -102,6 +102,10 @@ export interface GeneratorCatalogRow { /** Present iff it is wired as a stable-name STRING that `meta gen` refuses (a neutral * name): the run fails on it, rather than running or skipping the generator. */ refused?: true; + /** Present iff this project lists an owned generator for this row's docs surface in + * `docs: { generators: [...] }`: `meta docs` runs that copy in place of the packaged + * generator. The right place for it, where `wired` (under `generators`) is not. */ + wiredInDocsDoor?: true; /** This generator's framework is a declared dependency. `null` when neutral or unknown. */ frameworkDetected: boolean | null; /** Files this generator WOULD emit for this model. `null` unless --probe, and @@ -303,6 +307,12 @@ export async function buildCatalogListing(opts: CatalogListingOpts = {}): Promis .filter((spec): spec is string => typeof spec === "string" && isNeutralGeneratorName(spec)), ); + // The docs surfaces this project took over with an owned generator. Keyed by surface, + // as the docs door itself is: an owner may rename the copy. + const ownedDocsSurfaces = new Set( + (opts.project?.config.docs?.generators ?? []).map((g) => g?.docsSurface).filter((x) => x !== undefined), + ); + for (const entry of listCatalog()) { // Asked of the runner's own definition, never restated: which generators `meta gen` // skips is a fact about the runner, and a row that disagreed with it is the defect @@ -381,6 +391,9 @@ export async function buildCatalogListing(opts: CatalogListingOpts = {}): Promis row.project = { wired: project.wiredNames.has(entry.name), ...(refusedNames.has(entry.name) ? { refused: true as const } : {}), + ...(entry.docsSurface !== undefined && ownedDocsSurfaces.has(entry.docsSurface) + ? { wiredInDocsDoor: true as const } + : {}), frameworkDetected, wouldEmit, ...(probeError !== undefined ? { probeError } : {}), @@ -481,8 +494,7 @@ export const REFERENCE_HELPER_NOTE: readonly string[] = [ /** The footer line explaining the `package-only` row mark. */ export const PACKAGE_ONLY_NOTE = "package-only = no reference template ships for it, so `meta eject` cannot copy it. " + - "Run one as shipped by importing its factory — the import sits under its row, except " + - "for the docs layer, whose door is `meta docs`."; + "Run one as shipped by importing its factory — the import sits under its row."; /** * Printed under every row `meta gen` does not run (a row with a `door`). One sentence, @@ -496,7 +508,7 @@ export const SKIPPED_BY_GEN_NOTE = /** * Printed under every neutral row. The other outcome for a `generators` entry: the * resolver refuses a neutral stable name, so the string fails the run where a - * constructed generator is skipped or (the deprecated `mermaidErDiagram()`) still runs. + * constructed generator is skipped. */ export const NAME_REFUSED_BY_GEN_NOTE = "its name as a string under `generators` is refused instead: `meta gen` fails on it."; @@ -528,6 +540,8 @@ export function renderCatalogText(rows: CatalogRow[], probed: boolean): string { : r.door !== undefined ? "WIRED but skipped by `meta gen`" : "WIRED", ); } + // An owned docs generator in the docs block: the place `meta docs` runs it from. + if (r.project?.wiredInDocsDoor) marks.push("WIRED in `docs.generators`"); if (r.source.owned) marks.push("owned"); if (r.source.runtimeCopy !== undefined) { // The adapter source this generator's output imports, compared to the package. diff --git a/server/typescript/packages/cli/src/lib/catalog.ts b/server/typescript/packages/cli/src/lib/catalog.ts index 42e715217..a7bedd07f 100644 --- a/server/typescript/packages/cli/src/lib/catalog.ts +++ b/server/typescript/packages/cli/src/lib/catalog.ts @@ -80,11 +80,14 @@ export interface GeneratorWiring { * other half is `meta eject`, which prints the import for the owned copy — the symbol is * the same in both (gated), so moving between them is one import line. * - * Undefined for the `docs` layer: those generators are the engine behind `meta docs`, - * which is their door, and wiring one into `generators: [...]` is the deprecated route. + * Undefined for an entry `meta docs` runs (one that carries `docsSurface`): that command + * is its door, and `meta gen` skips it in `generators: [...]`, so an import line for that + * array would be an instruction to wire something the run ignores. Asked of the marker + * and not of `layer`: the layer is how the listing is grouped, and a docs-layer generator + * that `meta gen` does run would need its import like any other. */ export function packagedWiring(entry: GeneratorRegistryEntry): GeneratorWiring | undefined { - if (entry.layer === "docs") return undefined; + if (entry.docsSurface !== undefined) return undefined; const from = factoryModuleOf(entry.name); if (from === undefined) return undefined; return { import: `import { ${entry.factoryExport} } from "${from}";` }; diff --git a/server/typescript/packages/cli/src/lib/codegen-drift.ts b/server/typescript/packages/cli/src/lib/codegen-drift.ts index 846ab4990..05312dd36 100644 --- a/server/typescript/packages/cli/src/lib/codegen-drift.ts +++ b/server/typescript/packages/cli/src/lib/codegen-drift.ts @@ -127,12 +127,17 @@ function listFiles(dir: string): string[] { * never committed because the `meta gen` that produced the committed tree * never emitted it either. Undefined ⇒ everything is in scope (byte-identical * to a project with no `scope` declared). + * @param sourceFiles the collection's OWN source files, the same list `meta gen` + * threads onto `ctx.sourceFiles`. `shared-model` loads them standalone to build its + * artifact, so without them the regen asked it to publish nothing and this gate + * failed with "no source files to load" on every project that wires it. */ export async function computeCodegenDrift( config: MetaobjectsGenConfig, metadata: MetaData, projectRoot: string, scope?: (fqn: string) => boolean, + sourceFiles?: readonly string[], ): Promise { const root = isAbsolute(projectRoot) ? projectRoot : resolve(projectRoot); @@ -211,7 +216,13 @@ export async function computeCodegenDrift( genStateDir: join(tempRoot, ".gen-state"), mergeStrategy: "overwrite", baseline: "fresh", + // The real project, for a generator that reads its files (`ctx.inputRoot`): + // `shared-model` takes its default version from the nearest package.json, and an + // owned generator may read anything. `projectRoot` stays the temp tree, which the + // owned route and entity generators compute their adapter import against. + inputRoot: root, ...(scope !== undefined ? { scope } : {}), + ...(sourceFiles !== undefined ? { sourceFiles } : {}), }); // Diff each committed outDir against its temp mirror. diff --git a/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts b/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts index 530057a84..94548f776 100644 --- a/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts +++ b/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts @@ -6,8 +6,15 @@ // this way verified clean and was never told that the generator it listed produces // nothing — nor that the gate it was running does not check documentation at all. // -// A neutral stable NAME (`"docs"`, `"mermaid-er"`) is a different outcome: `meta gen` -// refuses the string and the run fails, so the finding says refused, not skipped. +// A neutral stable NAME (`"docs"`) is a different outcome: `meta gen` refuses the string +// and the run fails, so the finding says refused, not skipped. So is a REMOVED name +// (`"mermaid-er"`, whose diagram `meta docs` draws): the finding carries the resolver's +// own message for it. +// +// An OWNED docs generator (it carries a `docsSurface` marker) is skipped like a packaged +// one, and the fix is the opposite: it is MOVED to `docs: { generators: [...] }`, where +// `meta docs` runs it. Telling its owner to remove it would be telling them to stop +// using the generator they ejected. // // Read off the loaded config, not off a regen: the fact is in `generators`, and asking // the runner's and the resolver's own predicates keeps this from growing a second list @@ -20,6 +27,7 @@ import { generatorRegistry, isNeutralGeneratorName, isPackagedDocsGenerator, + removedGeneratorMessage, DOCS_DOOR_COMMAND, type MetaobjectsGenConfig, } from "@metaobjectsdev/codegen-ts"; @@ -31,8 +39,8 @@ export interface DocGeneratorFinding { file: string; /** The command that produces what the wired generator would have. */ construct: string; - /** True when `meta gen` REFUSES the entry (a neutral stable-name string), so the run - * fails; false when the runner warns about and skips it. */ + /** True when `meta gen` REFUSES the entry (a neutral or removed stable-name string), so + * the run fails; false when the runner warns about and skips it. */ refused: boolean; message: string; } @@ -50,13 +58,34 @@ function finding(wired: string, refused: boolean): DocGeneratorFinding { }; } +/** An owned docs generator listed under `generators`: skipped there, and it goes elsewhere. */ +function ownedFinding(name: string): DocGeneratorFinding { + return { + file: CONFIG_FILE, + construct: DOCS_DOOR_COMMAND, + refused: false, + message: + `${CONFIG_FILE}: the owned docs generator "${name}" is listed under \`generators\`, and ` + + `\`meta gen\` skips it — \`${DOCS_DOOR_COMMAND}\` runs an owned docs generator from ` + + "`docs: { generators: [...] }`, and `meta verify --docs` checks what it writes. " + + "Move it there.", + }; +} + +/** A removed stable name: the resolver refuses it, and its own message says what to do. */ +function removedFinding(message: string): DocGeneratorFinding { + return { file: CONFIG_FILE, construct: DOCS_DOOR_COMMAND, refused: true, message: `${CONFIG_FILE}: ${message}` }; +} + /** * One finding per `generators` entry `meta gen` skips or refuses, in config order. * * A `generators` entry is a constructed generator or a stable-name string (ADR-0021). - * A neutral name is refused by the resolver. Any other string resolves through the - * catalog to the same implementation name the runner tests; a name no entry registers - * is the runner's to refuse, not this scan's. + * A removed or neutral name is refused by the resolver. Any other string resolves + * through the catalog to the same implementation name the runner tests; a name no entry + * registers is the runner's to refuse, not this scan's. A constructed generator is + * tested the way the runner tests it, marker first: an owned copy keeps the packaged + * generator's name unless its owner renamed it. */ export function scanForDocGeneratorsInGen( config: MetaobjectsGenConfig | undefined, @@ -64,7 +93,11 @@ export function scanForDocGeneratorsInGen( const findings: DocGeneratorFinding[] = []; for (const spec of config?.generators ?? []) { if (typeof spec === "string") { - if (isNeutralGeneratorName(spec)) findings.push(finding(`"${spec}"`, true)); + // Every removed name so far is one `meta docs` replaced (gated by a test), which is + // why it is reported here under the docs door. + const removed = removedGeneratorMessage(spec); + if (removed !== undefined) findings.push(removedFinding(removed)); + else if (isNeutralGeneratorName(spec)) findings.push(finding(`"${spec}"`, true)); else if (isPackagedDocsGenerator(generatorRegistry[spec]?.factory().name ?? "")) { findings.push(finding(`"${spec}"`, false)); } @@ -73,7 +106,9 @@ export function scanForDocGeneratorsInGen( // `spec?.name`: the array is typed, but a config is user code, and a hole in it is // the runner's to report rather than a reason for an advisory to throw. const implName = spec?.name; - if (implName !== undefined && isPackagedDocsGenerator(implName)) { + if (spec?.docsSurface !== undefined) { + findings.push(ownedFinding(implName ?? "(unnamed)")); + } else if (implName !== undefined && isPackagedDocsGenerator(implName)) { findings.push(finding(packagedDocsFactoryCall(implName), false)); } } diff --git a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts index 430ae0145..537bb536b 100644 --- a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts +++ b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts @@ -131,6 +131,80 @@ function resolveCliPkg(specifier: string): string { * import of an aliased package falls back to normal module resolution (and to * jiti's alias map on non-Bun runtimes). */ +/** + * Generator factories `@metaobjectsdev/codegen-ts/generators` used to export and no longer + * does, each with what an adopter does instead. + * + * Without this, a removal an adopter WILL hit surfaces as whatever the runtime says about a + * missing export: `(0 , _generators.mermaidErDiagram) is not a function` from a transpiled + * config, or `does not provide an export named 'mermaidErDiagram'` from native ESM. Neither + * names MetaObjects, the removal or the remedy. This is the only place that can: the + * subpath still exists and serves every other factory, so nothing upstream of here can tell + * a correct import from a removed one. + * + * Two arms, because the import does not have to be in the config. `removedGeneratorImportError` + * reads the config SOURCE and can name the file and the import. `removedGeneratorRuntimeError` + * recovers the same diagnostic from the THROW, for a removed export imported by a module the + * config imports (an owned generator under `codegen/generators/`, typically). + * + * `mermaidErDiagram` (removed in 1.1) wrote the ER diagram to a file of its own under + * `meta gen`. It was the older home of a diagram `meta docs` draws. + */ +const REMOVED_GENERATOR_EXPORTS: Readonly> = { + mermaidErDiagram: + "`meta docs` draws the ER diagram, for the whole model on its overview page (README.md) " + + "and around each entity on that entity's page.\n" + + " Fix: delete the import and the `mermaidErDiagram()` entry, then run `meta docs`. " + + "The file it wrote (`docs/model.md` under your output directory, unless you moved it " + + "with `outFile`) is no longer generated; delete that too.", +}; +const GENERATORS_SUBPATH = "@metaobjectsdev/codegen-ts/generators"; + +/** The removed factories a config's own source imports, as one message; undefined when none. */ +export function removedGeneratorImportError(source: string): string | undefined { + const re = new RegExp( + `import\\s*(?:type\\s+)?\\{([^}]*)\\}\\s*from\\s*['"]${GENERATORS_SUBPATH.replace(/[/\\]/g, "\\$&")}['"]`, + "g", + ); + const hits = new Set(); + for (const m of source.matchAll(re)) { + for (const raw of (m[1] ?? "").split(",")) { + const name = raw.trim().replace(/^type\s+/, "").split(/\s+as\s+/)[0]?.trim(); + if (name !== undefined && REMOVED_GENERATOR_EXPORTS[name] !== undefined) hits.add(name); + } + } + if (hits.size === 0) return undefined; + return [...hits] + .sort() + .map( + (name) => + `metaobjects.config.ts imports ${name} from "${GENERATORS_SUBPATH}", and 1.1 removed ` + + `that generator: ${REMOVED_GENERATOR_EXPORTS[name] ?? ""}`, + ) + .join("\n"); +} + +/** + * The same diagnostic, recovered from the throw. It cannot name the file (the error carries + * no reference back to the import that produced it), so it says where to look instead. + * Matching is on the SYMBOL and on the fact of a missing export, not on one message shape: + * the shape is the runtime's, and an adopter's runtime is not ours to choose. + */ +export function removedGeneratorRuntimeError(err: unknown): string | undefined { + const message = err instanceof Error ? err.message : String(err ?? ""); + if (!/is not a function|does not provide an export named|[Ee]xport named .* not found/.test(message)) { + return undefined; + } + const hit = Object.keys(REMOVED_GENERATOR_EXPORTS).find((n) => new RegExp(`\\b${n}\\b`).test(message)); + if (hit === undefined) return undefined; + return ( + `${hit} is no longer exported from "${GENERATORS_SUBPATH}": 1.1 removed that generator. ` + + `Your metaobjects.config.ts does not import it itself, so the import is in a module the ` + + `config imports, most often one of your own under codegen/generators/. ` + + `${REMOVED_GENERATOR_EXPORTS[hit] ?? ""}` + ); +} + function rewriteImportSpecifiers(source: string, aliasMap: Record): string { let result = source; for (const [specifier, resolvedPath] of Object.entries(aliasMap)) { @@ -364,6 +438,8 @@ export async function loadMetaobjectsConfig(projectRoot: string): Promise 0) { help.push( - `\`meta gen\` skipped ${skippedDocs} doc generator(s) — see warnings[]; ${DOCS_DOOR_HELP}`, + `\`meta gen\` skipped ${skippedDocs} doc generator(s) — see warnings[]; ${docsDoorHelp(result)}`, ); } // An absent section is STATED, never omitted: a reader must be able to tell diff --git a/server/typescript/packages/cli/test/catalog-factory-exports.test.ts b/server/typescript/packages/cli/test/catalog-factory-exports.test.ts index b5f7fda34..d6e3eca19 100644 --- a/server/typescript/packages/cli/test/catalog-factory-exports.test.ts +++ b/server/typescript/packages/cli/test/catalog-factory-exports.test.ts @@ -20,7 +20,7 @@ import { join } from "node:path"; import * as coreFactories from "@metaobjectsdev/codegen-ts/generators"; import * as reactFactories from "@metaobjectsdev/codegen-ts-react"; import * as tanstackFactories from "@metaobjectsdev/codegen-ts-tanstack"; -import { factoryModuleOf, listCatalog } from "../src/lib/catalog.js"; +import { catalogEntry, factoryModuleOf, listCatalog, packagedWiring } from "../src/lib/catalog.js"; import { buildCatalogListing, renderCatalogText, type GeneratorCatalogRow } from "../src/lib/catalog-listing.js"; import { ejectGenerator } from "../src/commands/eject.js"; @@ -96,19 +96,46 @@ describe("catalog factory exports", () => { }); describe("meta gen --list — how to wire a generator as shipped", () => { - test("every row outside the docs layer carries its package import, and no derived call", async () => { + test("every row `meta gen` runs carries its package import, and no derived call", async () => { const rows = (await buildCatalogListing()).filter( (r): r is GeneratorCatalogRow => r.kind === "generator", ); for (const row of rows) { - if (row.layer === "docs") { - // `meta docs` is that layer's door; a wiring line would teach the deprecated route. + if (row.door !== undefined) { + // `meta docs` runs it; a line for `generators` would teach wiring `meta gen` skips. expect({ name: row.name, wire: row.wire }).toEqual({ name: row.name, wire: undefined }); continue; } expect(row.wire?.import).toMatch(/^import \{ \w+ \} from "@metaobjectsdev\/codegen-ts[\w/-]*";$/); expect({ name: row.name, keys: Object.keys(row.wire ?? {}) }).toEqual({ name: row.name, keys: ["import"] }); } + // The rows with a door are exactly the two `meta docs` runs. + expect(rows.filter((r) => r.door !== undefined).map((r) => r.name).sort()).toEqual(["api-docs", "docs"]); + }); + + // The import line is withheld because `meta docs` runs the generator (its `docsSurface` + // marker), not because of the layer the listing groups it under. A docs-layer generator + // that `meta gen` does run must print its import like any other. + test("packagedWiring keys on `docsSurface`, not on the docs layer", () => { + const entity = catalogEntry("entity"); + const docs = catalogEntry("docs"); + const apiDocs = catalogEntry("api-docs"); + if (entity === undefined || docs === undefined || apiDocs === undefined) throw new Error("catalog entry missing"); + const entityImport = { import: `import { entityFile } from "@metaobjectsdev/codegen-ts/generators";` }; + + expect(packagedWiring(entity)).toEqual(entityImport); + // Same generator, filed under the docs layer, no marker: it still gets its import. + expect(packagedWiring({ ...entity, layer: "docs" })).toEqual(entityImport); + // Same generator, any layer, carrying the marker: `meta docs` runs it, so no import. + expect(packagedWiring({ ...entity, docsSurface: "model" })).toBeUndefined(); + // The two real ones. + expect(packagedWiring(docs)).toBeUndefined(); + expect(packagedWiring(apiDocs)).toBeUndefined(); + // And with the marker taken off, the docs-layer entry would print its import. + const { docsSurface: _marker, ...unmarked } = docs; + expect(packagedWiring(unmarked)).toEqual({ + import: `import { docsFile } from "@metaobjectsdev/codegen-ts/generators";`, + }); }); test("the engine's factories are served from /generators, the framework packages' from the root", async () => { diff --git a/server/typescript/packages/cli/test/eject-multi.test.ts b/server/typescript/packages/cli/test/eject-multi.test.ts index 102af8627..d05b06c3a 100644 --- a/server/typescript/packages/cli/test/eject-multi.test.ts +++ b/server/typescript/packages/cli/test/eject-multi.test.ts @@ -36,7 +36,8 @@ interface EjectPayload { name: string; path: string; status: string; - wire: { import: string; entry: string }; + wire: { import: string; entry: string; into?: string }; + door?: string; requires: string[]; }>; install: { dev: string[]; runtime: string[]; command: string }; @@ -75,17 +76,115 @@ describe("meta eject takes many names", () => { }); test("a package-only generator is named as such, not as unknown", async () => { - // `shared-model` is a real catalog entry with no reference template. Calling it - // "unknown" sends the reader looking for a typo that is not there. + // `template` is a real catalog entry with no reference template: it is the Mustache + // primitive, the adopter already supplies its template, and there is no emit logic to + // own (ADR-0034 Amendment 7). It is the only one left. Calling it "unknown" sends the + // reader looking for a typo that is not there. const dir = tmp(); try { - expect(await ejectCommand(["shared-model"], dir, "text")).toBe(2); + expect(await ejectCommand(["template"], dir, "text")).toBe(2); const err = erred.join("\n"); expect(err).toContain("package-only"); - expect(err).toContain("shared-model"); - expect(err).not.toContain("unknown name(s): shared-model"); + expect(err).toContain("template"); + expect(err).not.toContain("unknown name(s): template"); expect(err).toContain("@metaobjectsdev/codegen-ts"); expect(err).toContain("Nothing was ejected"); + expect(existsSync(join(dir, "codegen/generators/template.ts"))).toBe(false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("shared-model ejects — a thin generator, and its entry carries the options it needs", async () => { + // It used to answer `package-only`. The owned part is which source files may + // contribute and which version is published; the artifact, its hash and the manifest + // stay in the package (`buildSharedModel`, `renderSharedModelFiles`). + const dir = tmp(); + try { + expect(await ejectCommand(["shared-model"], dir, "json")).toBe(0); + expect(erred.join("\n")).not.toContain("package-only"); + const source = readFileSync(join(dir, "codegen/generators/shared-model.ts"), "utf8"); + expect(source).toContain("export function sharedModelFile("); + expect(source).toContain("buildSharedModel"); + expect(source).toContain("renderSharedModelFiles"); + // The contract is not in the copy: no hashing, no manifest shape, no file names. + expect(source).not.toContain("createHash"); + expect(source).not.toContain("metaobjects.pkg.json\""); + const [row] = payload().ejected; + expect(row?.status).toBe("created"); + // Not `sharedModelFile()`: the factory cannot be called without these two. + expect(row?.wire.entry).toBe("sharedModelFile({ name, include })"); + expect(row?.wire.import).toBe( + 'import { sharedModelFile } from "./codegen/generators/shared-model.js";', + ); + // An ordinary `meta gen` generator: the entry goes in the top-level `generators`. + expect(row?.door).toBeUndefined(); + expect(row?.wire.into).toBeUndefined(); + // The two emitted files import nothing, and the copy imports only the engine. + const { install } = payload(); + expect(install.runtime).toEqual([]); + expect(install.dev).toEqual([`@metaobjectsdev/codegen-ts@^${cliVersion()}`]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("docs and api-docs eject — and say that `meta docs` runs the copy, not `meta gen`", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["docs", "api-docs"], dir, "json")).toBe(0); + expect(erred.join("\n")).not.toContain("package-only"); + const docs = readFileSync(join(dir, "codegen/generators/docs.ts"), "utf8"); + const api = readFileSync(join(dir, "codegen/generators/api-docs.ts"), "utf8"); + // The marker `meta docs` reads off a listed copy is in the file the adopter owns. + expect(docs).toContain('docsSurface: "model"'); + expect(api).toContain('docsSurface: "api"'); + const rows = payload().ejected; + expect(rows.map((r) => r.name)).toEqual(["docs", "api-docs"]); + expect(rows.map((r) => r.wire.entry)).toEqual(["docsFile()", "apiDocsFile()"]); + expect(rows.map((r) => r.wire.import)).toEqual([ + 'import { docsFile } from "./codegen/generators/docs.js";', + 'import { apiDocsFile } from "./codegen/generators/api-docs.js";', + ]); + // The row says which command runs the copy and which config key the entry goes in. + // It is the docs block's, never the array `meta gen` runs. + expect(rows.map((r) => r.door)).toEqual(["docs", "docs"]); + expect(rows.map((r) => r.wire.into)).toEqual(["docs.generators", "docs.generators"]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("the text report says which command runs an ejected docs generator", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["docs"], dir, "text")).toBe(0); + const text = logged.join("\n"); + expect(text).toContain('Ejected "docs" -> codegen/generators/docs.ts'); + expect(text).toContain("`meta docs` runs this generator, in place of the packaged one."); + expect(text).toContain('import { docsFile } from "./codegen/generators/docs.js";'); + expect(text).toContain("docs: { generators: [docsFile()] }"); + expect(text).toContain("`meta verify --docs`"); + // The wiring is the docs door's, start to finish: nothing sends the reader to the + // top-level `generators`, where `meta gen` would skip the copy. + expect(text).not.toContain("in `generators` yet"); + expect(text).not.toContain("leaves `generators` bound"); + expect(text.replaceAll("docs: { generators:", "")).not.toContain("generators:"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("mermaid-er is not a name any more: the generator was removed, and nothing is copied", async () => { + // 1.1 removed the standalone diagram generator (`meta docs` draws the diagram), so + // there is neither a reference template nor a catalog row to answer for it. + const dir = tmp(); + try { + expect(await ejectCommand(["mermaid-er"], dir, "text")).toBe(2); + const err = erred.join("\n"); + expect(err).toContain("unknown name(s): mermaid-er"); + expect(err).not.toContain("package-only"); + expect(existsSync(join(dir, "codegen/generators/mermaid-er.ts"))).toBe(false); } finally { rmSync(dir, { recursive: true, force: true }); } diff --git a/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts b/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts index e58d77bbd..2dbc62774 100644 --- a/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts +++ b/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts @@ -34,8 +34,15 @@ const MODEL = JSON.stringify({ }, }); +interface RepoOpts { + /** Statements placed before the config's `export default`. */ + extra?: string; + /** The config's `docs` block, as an object literal. Absent means no `docs` key. */ + docs?: string; +} + /** `generators` is the config's array literal, verbatim. */ -function setupRepo(generators: string, opts: { extra?: string } = {}): string { +function setupRepo(generators: string, opts: RepoOpts = {}): string { mkdirSync(WORKSPACE_TMP, { recursive: true }); const root = mkdtempSync(join(WORKSPACE_TMP, "gen-docs-door-")); mkdirSync(join(root, "metaobjects"), { recursive: true }); @@ -46,14 +53,14 @@ function setupRepo(generators: string, opts: { extra?: string } = {}): string { join(root, "metaobjects.config.ts"), ` import { defineConfig } from "@metaobjectsdev/codegen-ts"; -import { apiDocsFile, docsFile, mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators"; +import { apiDocsFile, docsFile } from "@metaobjectsdev/codegen-ts/generators"; ${opts.extra ?? ""} export default defineConfig({ outDir: "generated", dialect: "sqlite", dbImport: "../db", extStyle: "none", - generators: ${generators}, + generators: ${generators},${opts.docs !== undefined ? `\n docs: ${opts.docs},` : ""} }); `, ); @@ -81,7 +88,7 @@ afterEach(() => { }); /** Run the CLI against a fresh project and hand back both streams. */ -async function meta(generators: string, args: string[], opts: { extra?: string } = {}): +async function meta(generators: string, args: string[], opts: RepoOpts = {}): Promise<{ code: number; stdout: string; stderr: string; root: string }> { const root = setupRepo(generators, opts); try { @@ -104,11 +111,20 @@ interface ListRow { kind: string; note?: string; door?: string; - project?: { wired: boolean; refused?: true; wouldEmit: number | null; probeError?: string }; + project?: { wired: boolean; refused?: true; wiredInDocsDoor?: true; wouldEmit: number | null; probeError?: string }; } const WIRED = `["entity", apiDocsFile(), docsFile()]`; +/** + * An owned docs generator, as `meta eject docs` leaves one: the packaged generator's + * name, and the `docsSurface` marker that says which surface it takes over. + */ +const OWNED_DOCS = `const ownedDocs = { name: "docs-file", docsSurface: "model" as const, generate: () => [] };`; +const OWNED_MOVE_WARNING = + "[docs-file] is a docs generator: 'meta docs' runs it, from `docs: { generators: [...] }` " + + "in metaobjects.config.ts, and 'meta gen' does not. Move it there. Skipped."; + describe("meta gen — a skipped doc generator is reported in every output form", () => { test("text: the skip is on stderr, and the file listing on stdout is unchanged", async () => { const r = await meta(WIRED, ["gen", "--format", "text"]); @@ -183,23 +199,20 @@ describe("meta gen — a skipped doc generator is reported in every output form" const NEUTRAL_REFUSAL = (name: string): string => `generator "${name}" is neutral (owned by 'meta docs'); not selectable in the gen suite.`; -describe("meta gen — mermaid-er is deprecated: a wired factory runs, the name is refused", () => { - // ADR-0022 Part 2 D1 keeps it deprecated from the gen suite. A config that already - // lists `mermaidErDiagram()` keeps working; the stable name is not selectable. - test("mermaidErDiagram() writes its file and draws no skip warning", async () => { - const r = await meta(`["entity", mermaidErDiagram()]`, ["gen", "--format", "json"]); - expect(r.code).toBe(0); - const payload = JSON.parse(r.stdout) as GenPayload; - expect(payload.gen.map((g) => g.file)).toContain("generated/docs/model.md"); - expect(payload.warnings).toBeUndefined(); - expect(r.stderr).not.toContain("Skipped."); - }); +const MERMAID_ER_REMOVED = 'generator "mermaid-er" was removed in 1.1: `meta docs` draws the ER diagram'; - test("the stable name `mermaid-er` is refused as neutral, and nothing is written", async () => { +describe("meta gen — a doc generator's stable name is refused: `docs` as neutral, `mermaid-er` as removed", () => { + // 1.1 removed the standalone `mermaid-er` generator (ADR-0022 Part 2 D1, as amended): + // `meta docs` draws the diagram. The name is told so, not called neutral or unknown. + test("the stable name `mermaid-er` is refused as removed, and nothing is written", async () => { const root = setupRepo(`["entity", "mermaid-er"]`); try { expect(await run(["gen", "--format", "json", "--cwd", root])).not.toBe(0); - expect(err.join("\n")).toContain(NEUTRAL_REFUSAL("mermaid-er")); + const said = err.join("\n"); + expect(said).toContain(MERMAID_ER_REMOVED); + expect(said).toContain('Remove "mermaid-er" from `generators` and run `meta docs`.'); + expect(said).not.toContain("is neutral"); + expect(said).not.toContain("unknown generator"); expect(existsSync(join(root, "generated"))).toBe(false); } finally { rmSync(root, { recursive: true, force: true }); @@ -289,6 +302,59 @@ describe("meta gen — a config wiring ONLY doc generators is not told to author }); }); +describe("meta gen — an owned docs generator under `generators` is skipped and told where it goes", () => { + // ADR-0034 Amendment 7. The runner skips it by its `docsSurface` marker. Before it was + // also REPORTED as skipped, a config wiring only an owned copy read "no entities to + // generate", and the next step for any skipped doc generator was "remove it". + test("json: wired alone, the summary names the real reason and the next step is to MOVE it", async () => { + const r = await meta(`[ownedDocs]`, ["gen", "--format", "json"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as GenPayload; + expect(payload.gen).toEqual([]); + expect(payload.summary).toContain("doc generator"); + expect(payload.summary).not.toContain("no entities"); + expect(payload.warnings).toEqual([OWNED_MOVE_WARNING]); + expect(payload.help).toEqual([ + "run `meta docs` for documentation, and move the owned docs generators from `generators` " + + "to `docs: { generators: [...] }` in metaobjects.config.ts", + ]); + expect(payload.help.join("\n")).not.toContain("remove"); + expect(payload.help.join("\n")).not.toContain("author entities"); + }); + + test("text: wired alone, stdout names the real reason", async () => { + const r = await meta(`[ownedDocs]`, ["gen", "--format", "text"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + expect(r.stdout).not.toContain("No entities to generate"); + expect(r.stdout).toContain("every wired generator is a doc generator"); + expect(r.stderr).toContain(OWNED_MOVE_WARNING); + }); + + test("json: beside a packaged one, the next step says which to move and which to remove", async () => { + const r = await meta(`["entity", ownedDocs, apiDocsFile()]`, ["gen", "--format", "json"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as GenPayload; + expect(payload.gen.map((g) => g.file)).toEqual(["generated/Talk.ts"]); + expect(payload.warnings).toContain(OWNED_MOVE_WARNING); + expect(payload.warnings?.some((w) => w.includes(API_DOCS_SKIP))).toBe(true); + const step = payload.help.find((h) => h.includes("skipped 2 doc generator(s)")); + expect(step).toContain( + "move the owned docs generators from `generators` to `docs: { generators: [...] }` and remove the packaged ones", + ); + }); + + test("where it belongs, in `docs.generators`, `meta gen` says nothing about it", async () => { + const r = await meta(`["entity"]`, ["gen", "--format", "json"], { + extra: OWNED_DOCS, + docs: `{ generators: [ownedDocs] }`, + }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as GenPayload; + expect(payload.warnings).toBeUndefined(); + expect(payload.help.some((h) => h.includes("meta docs"))).toBe(false); + }); +}); + describe("meta gen --list — a row `meta gen` does not run says so", () => { const rowsOf = (stdout: string): ListRow[] => JSON.parse(stdout) as ListRow[]; const row = (rows: ListRow[], name: string): ListRow => { @@ -319,18 +385,18 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(block(r.stdout, "entity")).not.toContain("meta docs"); }); - // The registry note said "surfaced via `meta docs`", which `meta docs` never did. The - // row states the back-compat fact, and does not present the generator as a choice. - test("text: the mermaid-er row says it is deprecated, and has no door", async () => { + // The standalone `mermaid-er` generator was removed in 1.1, so the catalog has no row + // for it, in either form, and a config that still names it does not break the listing. + test("text: there is no mermaid-er row", async () => { const r = await meta(`["entity"]`, ["gen", "--list", "--format", "text"]); - const mermaid = block(r.stdout, "mermaid-er"); - expect(mermaid).toContain("deprecated"); - expect(mermaid).toContain("do not add it to a new config"); - expect(mermaid).toContain("A `mermaidErDiagram()` already wired in `generators` still runs"); - expect(mermaid).toContain("`meta docs` draws the same diagram on its overview page"); - expect(mermaid).not.toContain("surfaced via"); - expect(mermaid).not.toContain("run by `meta docs`"); - expect(mermaid).not.toContain("not run by `meta gen`"); + expect(r.code).toBe(0); + expect(r.stdout).not.toContain("mermaid-er"); + }); + + test("json: a config that still names it lists cleanly, with no row for it", async () => { + const r = await meta(`["entity", "mermaid-er"]`, ["gen", "--list", "--format", "json"]); + expect(r.code).toBe(0); + expect(rowsOf(r.stdout).some((x) => x.name === "mermaid-er")).toBe(false); }); test("json: the row carries the door and the note", async () => { @@ -342,9 +408,6 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(row(rows, name).door, name).toBe("meta docs"); expect(row(rows, name).note ?? "", name).toContain("meta docs"); } - expect(row(rows, "mermaid-er").door).toBeUndefined(); - expect(row(rows, "mermaid-er").note ?? "").toContain("deprecated"); - expect(row(rows, "mermaid-er").note ?? "").toContain("still runs"); expect(row(rows, "entity").door).toBeUndefined(); expect(row(rows, "entity").note).toBeUndefined(); }); @@ -358,13 +421,13 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { }); // A neutral name written as a string is not skipped: `meta gen` refuses it and exits - // non-zero (asserted above). The mark has to say that, for `docs` and `mermaid-er` both. - const BY_NAME = `["entity", "docs", "mermaid-er", "api-docs"]`; + // non-zero (asserted above). The mark has to say that. + const BY_NAME = `["entity", "docs", "api-docs"]`; test("text: a neutral name wired as a string reads as refused, not skipped or run", async () => { const r = await meta(BY_NAME, ["gen", "--list", "--format", "text"]); expect(r.code).toBe(0); - for (const name of ["docs", "mermaid-er"]) { + for (const name of ["docs"]) { const b = block(r.stdout, name); expect(b, name).toContain("WIRED by name, which `meta gen` refuses"); expect(b, name).not.toContain("WIRED but skipped"); @@ -380,7 +443,7 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { test("json: the refusal is on the row's project facts", async () => { const r = await meta(BY_NAME, ["gen", "--list", "--format", "json"]); const rows = rowsOf(r.stdout); - for (const name of ["docs", "mermaid-er"]) { + for (const name of ["docs"]) { expect(row(rows, name).project?.wired, name).toBe(true); expect(row(rows, name).project?.refused, name).toBe(true); } @@ -389,14 +452,44 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(row(rows, "entity").project?.refused).toBeUndefined(); }); - test("a constructed mermaidErDiagram() reads as WIRED: it still runs", async () => { - const r = await meta(`["entity", mermaidErDiagram(), docsFile()]`, ["gen", "--list", "--format", "json"]); + test("a constructed docsFile() is skipped, not refused: only the name is refused", async () => { + const r = await meta(`["entity", docsFile()]`, ["gen", "--list", "--format", "json"]); const rows = rowsOf(r.stdout); - expect(row(rows, "mermaid-er").project?.wired).toBe(true); - expect(row(rows, "mermaid-er").project?.refused).toBeUndefined(); + expect(row(rows, "docs").project?.wired).toBe(true); expect(row(rows, "docs").project?.refused).toBeUndefined(); }); + // ADR-0034 Amendment 7: an owned docs generator is listed in the docs block, and + // `meta docs` runs it from there. The row has to show that it is in use: read off + // `generators` alone, a correctly wired copy looked like one nobody runs. + const IN_DOCS_DOOR: RepoOpts = { extra: OWNED_DOCS, docs: `{ generators: [ownedDocs] }` }; + + test("json: an owned docs generator in `docs.generators` reads as wired in the docs door", async () => { + const r = await meta(`["entity"]`, ["gen", "--list", "--format", "json"], IN_DOCS_DOOR); + expect(r.code).toBe(0); + const rows = rowsOf(r.stdout); + expect(row(rows, "docs").project?.wiredInDocsDoor).toBe(true); + expect(row(rows, "docs").project?.wired).toBe(false); + // Keyed by surface: the api surface was not taken over. + expect(row(rows, "api-docs").project?.wiredInDocsDoor).toBeUndefined(); + expect(row(rows, "entity").project?.wiredInDocsDoor).toBeUndefined(); + }); + + test("text: the row says where it is wired, and does not call it skipped", async () => { + const r = await meta(`["entity"]`, ["gen", "--list", "--format", "text"], IN_DOCS_DOOR); + expect(r.code).toBe(0); + expect(block(r.stdout, "docs")).toContain("WIRED in `docs.generators`"); + expect(block(r.stdout, "docs")).not.toContain("WIRED but skipped"); + expect(block(r.stdout, "api-docs")).not.toContain("WIRED"); + }); + + test("the same copy under `generators` reads as skipped, and not as wired in the docs door", async () => { + const r = await meta(`["entity", ownedDocs]`, ["gen", "--list", "--format", "json"], { extra: OWNED_DOCS }); + const rows = rowsOf(r.stdout); + expect(row(rows, "docs").project?.wired).toBe(true); + expect(row(rows, "docs").project?.wiredInDocsDoor).toBeUndefined(); + }); + test("--probe: no file count for a generator `meta gen` does not run", async () => { // It printed `would emit 0`, which reads as "nothing in your model needs this" for // a generator whose pages `meta docs` writes for every entity. @@ -407,7 +500,6 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(block(text.stdout, name), name).toContain("run by `meta docs`"); } expect(block(text.stdout, "entity")).toContain("would emit 1"); - expect(block(text.stdout, "mermaid-er")).toContain("would emit 1"); }); test("--probe --format json: wouldEmit is null beside the door, never 0", async () => { @@ -466,15 +558,19 @@ describe("meta verify — a doc generator under `generators` is an advisory", () expect(payload.help.join("\n")).not.toContain("refuses"); }); - // `meta gen` exits non-zero on these two strings (asserted in the mermaid-er block + // `meta gen` exits non-zero on these two strings (asserted in the refused-name block // above), so calling them skipped told the reader the run goes on without them. - test("json: a neutral name wired as a string is reported as refused, not skipped", async () => { + test("json: a neutral or removed name wired as a string is reported as refused, not skipped", async () => { const r = await meta(`["entity", "docs", "mermaid-er"]`, ["verify", "--format", "json"]); const payload = JSON.parse(r.stdout) as VerifyPayload; const rows = payload.antiPatterns.rows.filter((x) => x.rule === "doc-generator-refused-by-gen"); expect(rows).toHaveLength(2); expect(rows[0]?.message).toContain(`"docs" is listed under \`generators\`, and \`meta gen\` refuses it`); - expect(rows[1]?.message).toContain(`"mermaid-er" is listed under \`generators\`, and \`meta gen\` refuses it`); + // The removed name carries the message `meta gen` fails with, word for word. + expect(rows[1]?.message).toBe( + `metaobjects.config.ts: ${MERMAID_ER_REMOVED}, for the whole model on its overview page (README.md) ` + + `and around each entity on that entity's page. Remove "mermaid-er" from \`generators\` and run \`meta docs\`.`, + ); for (const x of rows) { expect(x.message).not.toContain("skips"); expect(x.message).toContain("Remove"); @@ -493,8 +589,40 @@ describe("meta verify — a doc generator under `generators` is an advisory", () expect(r.stderr).not.toContain("`meta gen` skips"); }); - test("a constructed mermaidErDiagram() gets no row: it still runs", async () => { - const r = await meta(`["entity", mermaidErDiagram()]`, ["verify", "--format", "json"]); + // An owned docs generator under `generators` is skipped like a packaged one, and the + // fix is the opposite. "Remove it" would tell its owner to stop using the copy they + // ejected; it goes in the docs block, where `meta docs` runs it. + test("an owned docs generator is told to MOVE to `docs.generators`, not to be removed", async () => { + const r = await meta(`["entity", ownedDocs, apiDocsFile()]`, ["verify", "--format", "json"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as VerifyPayload; + const rows = payload.antiPatterns.rows.filter((x) => x.rule === "doc-generator-in-gen"); + expect(rows).toHaveLength(2); + expect(rows[0]?.message).toBe( + 'metaobjects.config.ts: the owned docs generator "docs-file" is listed under `generators`, and ' + + "`meta gen` skips it — `meta docs` runs an owned docs generator from `docs: { generators: [...] }`, " + + "and `meta verify --docs` checks what it writes. Move it there.", + ); + expect(rows[0]?.message).not.toContain("Remove"); + expect(rows[0]?.construct).toBe("meta docs"); + // The packaged one beside it keeps its own advice. + expect(rows[1]?.message).toContain("Remove apiDocsFile() from `generators`."); + expect(payload.exitCode).toBe(0); + }); + + test("the marker decides, not the name: a renamed owned copy is told the same", async () => { + const extra = `const myPages = { name: "my-pages", docsSurface: "api" as const, generate: () => [] };`; + const r = await meta(`["entity", myPages]`, ["verify", "--format", "text"], { extra }); + expect(r.code).toBe(0); + expect(r.stderr).toContain('the owned docs generator "my-pages" is listed under `generators`'); + expect(r.stderr).toContain("Move it there."); + }); + + test("an owned docs generator where it belongs gets no row", async () => { + const r = await meta(`["entity"]`, ["verify", "--format", "json"], { + extra: OWNED_DOCS, + docs: `{ generators: [ownedDocs] }`, + }); const payload = JSON.parse(r.stdout) as VerifyPayload; expect(payload.antiPatterns.rows.filter((x) => x.rule.startsWith("doc-generator-"))).toEqual([]); }); diff --git a/server/typescript/packages/cli/test/gen-list.test.ts b/server/typescript/packages/cli/test/gen-list.test.ts index 2383f1676..9f045bb97 100644 --- a/server/typescript/packages/cli/test/gen-list.test.ts +++ b/server/typescript/packages/cli/test/gen-list.test.ts @@ -61,15 +61,16 @@ describe("meta gen --list (ADR-0021 D3)", () => { } }); - test("flags docs/mermaid neutral (not part of the recommended native suite)", async () => { + test("flags docs neutral (not part of the recommended native suite)", async () => { const tmp = mkdtempSync(join(tmpdir(), "meta-gen-list-")); try { await genCommand(["--list"], tmp); const text = out(); - // They appear (identity/discoverability) ... + // It appears (identity/discoverability) ... expect(text).toContain("docs"); - expect(text).toContain("mermaid-er"); - // ... but are clearly marked neutral and point at `meta docs`. + // ... but is clearly marked neutral and points at `meta docs`. The standalone + // `mermaid-er` generator that sat beside it was removed in 1.1. + expect(text).not.toContain("mermaid-er"); expect(text.toLowerCase()).toContain("neutral"); expect(text).toContain("meta docs"); } finally { @@ -100,12 +101,18 @@ describe("meta gen --list (ADR-0021 D3)", () => { await genCommand(["--list"], tmp); const text = out(); const line = (name: string) => text.split("\n").find((l) => l.startsWith(` ${name} `)) ?? ""; - expect(line("shared-model")).toContain("package-only"); + // The one entry left: the Mustache primitive, where the adopter already supplies + // the template and there is no emit logic to own (ADR-0034 Amendment 7). expect(line("template")).toContain("package-only"); + expect(text.split("\n").filter((l) => l.includes("[package-only]")).length).toBe(1); // An ejectable entry is never marked. `callable` and `trace-helper` were marked - // until they gained reference templates. + // until they gained reference templates; `shared-model` and the docs tier until + // Amendment 7. expect(line("callable")).not.toContain("package-only"); expect(line("trace-helper")).not.toContain("package-only"); + expect(line("shared-model")).not.toContain("package-only"); + expect(line("docs")).not.toContain("package-only"); + expect(line("api-docs")).not.toContain("package-only"); expect(line("prompt-render")).not.toContain("package-only"); expect(line("entity")).not.toContain("package-only"); expect(text).toContain("package-only = no reference template ships"); diff --git a/server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts b/server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts new file mode 100644 index 000000000..1de13dbd0 --- /dev/null +++ b/server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts @@ -0,0 +1,371 @@ +/** + * The docs tier, owned — end to end, through the BUILT CLI (ADR-0034 Amendment 7). + * + * `docs` and `api-docs` could not be ejected while `meta docs` constructed the packaged + * generators itself: there was nothing to hand an owned copy to. The seam is the docs + * door's own config, `docs: { generators: [...] }`, read by a marker on the generator + * (`docsSurface`). These tests drive it the way an adopter does — `meta eject`, + * `metaobjects.config.ts`, `meta docs`, `meta verify --docs`, `meta gen` — because the + * claim is about which command runs which copy, and no factory-level test can see that. + * + * The standalone `mermaid-er` generator that used to sit in this tier was removed in 1.1: + * `meta docs` draws the diagrams. The last describe pins what a project that still wires + * it is told, by name and by factory. + * + * Each gate is shown to FAIL as well as pass: an edited copy changes the page and the + * drift check convicts the committed tree. A seam that only ever agreed with the packaged + * generator would pass every equality here without running the copy at all. + * + * Subprocess-per-command for the reason `verify-codegen-ejected-generator` gives: the + * config is re-read under a fresh name each load but the modules it imports are cached. + */ +import { describe, test, expect, beforeAll } from "bun:test"; +import { cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { spawnSync } from "node:child_process"; +import { meta, requireFreshDist } from "./support/built-cli.js"; + +const FIXTURES = resolve(import.meta.dirname, "..", "fixtures"); +// Inside the workspace, so an ejected template's `@metaobjectsdev/codegen-ts` import +// resolves by walking up to the workspace root. +const WORKSPACE_TMP = join(FIXTURES, "__tmp__"); +const OUT = "generated"; +// The default `docs.outDir`: where `meta docs` writes and `verify --docs` reads. +const DOCS = join("docs", "generated"); + +/** + * @param docsGenerators what goes in `docs: { generators: [...] }`, the docs door's list. + * @param genGenerators appended to the top-level `generators`, the array `meta gen` runs. + */ +function config(imports: readonly string[], docsGenerators: string, genGenerators = ""): string { + return [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { entityFile, routesFile } from "@metaobjectsdev/codegen-ts/generators";`, + ...imports, + `export default defineConfig({`, + ` outDir: ${JSON.stringify(OUT)},`, + ` dialect: "sqlite",`, + ` dbImport: "~/db",`, + ` extStyle: "none",`, + ` apiPrefix: "/api",`, + ` generators: [entityFile(), routesFile()${genGenerators}],`, + ...(docsGenerators === "" ? [] : [` docs: { generators: [${docsGenerators}] },`]), + `});`, + ].join("\n"); +} + +/** Nothing docs-related listed: `meta docs` runs the packaged generators. */ +const PACKAGED = config([], ""); + +const OWNED_DOCS_IMPORT = `import { docsFile } from "./codegen/generators/docs.js";`; +const OWNED_API_IMPORT = `import { apiDocsFile } from "./codegen/generators/api-docs.js";`; +/** Both owned copies, listed exactly as `meta eject` prints. */ +const OWNED_BOTH = config([OWNED_DOCS_IMPORT, OWNED_API_IMPORT], "docsFile(), apiDocsFile()"); + +function setupRepo(configSource: string): string { + mkdirSync(WORKSPACE_TMP, { recursive: true }); + const root = mkdtempSync(join(WORKSPACE_TMP, "docs-tier-")); + cpSync(join(FIXTURES, "trainer-website-meta"), root, { recursive: true }); + writeFileSync(join(root, "metaobjects.config.ts"), configSource); + // Its own repository. `verify --docs` skips a page the project git-ignores, and this + // directory sits under an ignored path of the workspace it borrows `node_modules` from: + // without a repository of its own every page here would be exempt and the gate would + // have nothing to compare. + expect(spawnSync("git", ["init", "-q"], { cwd: root }).status).toBe(0); + return root; +} + +/** Every file under `/`, path → contents. Empty when the directory is absent. */ +function tree(root: string, dir: string): Map { + const out = new Map(); + const base = join(root, dir); + if (!existsSync(base)) return out; + for (const f of readdirSync(base, { recursive: true, withFileTypes: true })) { + if (!f.isFile()) continue; + const abs = join(f.parentPath, f.name); + out.set(abs.slice(base.length + 1), readFileSync(abs, "utf8")); + } + return out; +} + +function edit(root: string, rel: string, from: string, to: string): void { + const path = join(root, rel); + const before = readFileSync(path, "utf8"); + // An edit that matched nothing would leave the copy unmodified and every "the edited + // copy ran" assertion below would be testing the reference. + expect(before).toContain(from); + writeFileSync(path, before.replace(from, to)); +} + +// Precondition once per FILE, not per test: this gate runs the BUILT CLI. +beforeAll(requireFreshDist); + +describe("docs and api-docs — owned copies run by `meta docs`", () => { + test("unmodified owned copies write exactly the pages `meta docs` writes without them", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const packaged = tree(root, DOCS); + // Vacuity guard: both surfaces are in the tree being compared. + expect([...packaged.keys()]).toEqual(expect.arrayContaining(["README.md", "User.md", "api/README.md", "api/AGENT-API.md", "api/User.md"])); + + // Take ownership of both, changing nothing about them. + expect(await meta(root, "eject", "docs", "api-docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), OWNED_BOTH); + + // The committed pages were written by the PACKAGE's generators. If the owned copies + // would write anything else, this is where it shows. + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + + const owned = await meta(root, "docs"); + expect(owned).toMatchObject({ exit: 0 }); + // The summary names the copy, so "did mine run?" has an answer on the terminal. + expect(owned.output).toContain("from the owned generator 'docs-file'"); + expect(owned.output).toContain("from the owned generator 'api-docs'"); + expect(tree(root, DOCS)).toEqual(packaged); + + // And back: dropping the unmodified copies is equally a no-op. + writeFileSync(join(root, "metaobjects.config.ts"), PACKAGED); + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("an edited model-docs copy is the one that runs, and `verify --docs` convicts the stale pages", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + expect(tree(root, DOCS).get("README.md")).toStartWith("# Data Model\n"); + + expect(await meta(root, "eject", "docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "docsFile()")); + edit(root, "codegen/generators/docs.ts", 'out.push("# Data Model");', 'out.push("# Our Model");'); + + // The committed README still has the packaged title: the gate has to say so. + const stale = await meta(root, "verify", "--docs"); + expect(stale.exit).toBe(1); + expect(stale.output).toContain("README.md"); + + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const pages = tree(root, DOCS); + expect(pages.get("README.md")).toStartWith("# Our Model\n"); + // The api surface was not ejected: the packaged generator still writes it. + expect(pages.has("api/AGENT-API.md")).toBe(true); + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("an edited api-docs copy is the one that runs, with the model surface still packaged", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const before = tree(root, DOCS); + expect(before.has("api/AGENT-API.md")).toBe(true); + + expect(await meta(root, "eject", "api-docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_API_IMPORT], "apiDocsFile()")); + // Own a decision the packaged generator does not offer: no agent form. + edit( + root, + "codegen/generators/api-docs.ts", + " { path: agentFilename, content: renderAgentApi(model, provider) },\n", + "", + ); + + // `meta docs` only writes, so the agent form written before the edit would still be + // on disk. A fresh tree is what shows the copy ran: it no longer contains that file. + rmSync(join(root, DOCS), { recursive: true, force: true }); + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const fresh = tree(root, DOCS); + expect(fresh.has("api/AGENT-API.md")).toBe(false); + // Everything else is untouched, the packaged model surface included. + expect(fresh.get("api/User.md")).toBe(before.get("api/User.md")); + expect(fresh.get("README.md")).toBe(before.get("README.md")); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("`meta gen` is untouched by listed docs generators, and points a misplaced one at the docs door", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs", "api-docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), OWNED_BOTH); + + // Listed where they belong, the copies are none of `meta gen`'s business: it runs + // what is in `generators`, writes no page, and has nothing to say about docs. + const gen = await meta(root, "gen", "--format", "text"); + expect(gen).toMatchObject({ exit: 0 }); + expect(gen.output).not.toContain("docs generator"); + expect(gen.output).not.toContain("docs are produced by"); + const generated = [...tree(root, OUT).keys()]; + expect(generated).toContain("User.ts"); + expect(generated.filter((p) => p.endsWith(".md"))).toEqual([]); + expect(existsSync(join(root, DOCS))).toBe(false); + // The codegen drift gate regenerates through the same runner, so it is unaffected. + expect(await meta(root, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + // The same copy in the array `meta gen` runs is in the wrong place. It is skipped, + // and the warning names where it goes instead of leaving `meta docs` quietly + // running the packaged generator. + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "", ", docsFile()")); + const misplaced = await meta(root, "gen", "--format", "text"); + expect(misplaced).toMatchObject({ exit: 0 }); + expect(misplaced.output).toContain( + "[docs-file] is a docs generator: 'meta docs' runs it, from `docs: { generators: [...] }`", + ); + expect([...tree(root, OUT).keys()].filter((p) => p.endsWith(".md"))).toEqual([]); + // And `meta docs` did not pick it up from there: the packaged generator ran. + const docs = await meta(root, "docs"); + expect(docs).toMatchObject({ exit: 0 }); + expect(docs.output).not.toContain("from the owned generator"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("a `docs.generators` list `meta docs` cannot run is refused by name, by `meta docs` and by `verify --docs`", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs")).toMatchObject({ exit: 0 }); + + // Two for one surface: each would write the same pages. + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "docsFile(), docsFile()")); + const two = await meta(root, "docs"); + expect(two.exit).toBe(2); + expect(two.output).toContain('`docs.generators` lists two generators for the model surface ("docs-file" and "docs-file")'); + expect(existsSync(join(root, DOCS))).toBe(false); + expect((await meta(root, "verify", "--docs")).exit).not.toBe(0); + + // No marker: a generator `meta gen` runs, listed in the docs block by mistake. The + // door has no surface to give it, so it neither runs it nor drops it. + writeFileSync( + join(root, "metaobjects.config.ts"), + config([OWNED_DOCS_IMPORT], "docsFile(), entityFile()"), + ); + const unmarked = await meta(root, "docs"); + expect(unmarked.exit).toBe(2); + expect(unmarked.output).toContain('`docs.generators` lists "entity-file", which carries no `docsSurface`'); + expect(existsSync(join(root, DOCS))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("`meta eject --list` and `meta gen --list` treat the tier like every other ejectable generator", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs", "api-docs")).toMatchObject({ exit: 0 }); + edit(root, "codegen/generators/docs.ts", 'out.push("# Data Model");', 'out.push("# Our Model");'); + + const list = await meta(root, "eject", "--list", "--format", "json"); + expect(list.exit).toBe(0); + const rows = (JSON.parse(list.output) as { generators: Array<{ name: string; status: string }> }).generators; + const status = (name: string) => rows.find((r) => r.name === name)?.status; + expect(status("api-docs")).toBe("identical"); + expect(status("docs")).toBe("differs"); + expect(status("mermaid-er")).toBeUndefined(); + + const catalog = await meta(root, "gen", "--list", "--format", "json"); + expect(catalog.exit).toBe(0); + const catalogRows = JSON.parse(catalog.output) as Array<{ + name: string; + source: { kind: string; ejectable: boolean; owned: boolean | null }; + }>; + for (const name of ["docs", "api-docs"]) { + const row = catalogRows.find((r) => r.name === name); + expect({ name, source: row?.source }).toEqual({ + name, + source: expect.objectContaining({ kind: "reference-template", ejectable: true, owned: true }), + }); + } + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); +}); + +describe("mermaid-er — removed; `meta docs` draws the diagrams", () => { + const REMOVED_FACTORY = [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { entityFile, mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: [entityFile(), mermaidErDiagram()] });`, + ].join("\n"); + const REMOVED_NAME = [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: ["entity", "mermaid-er"] });`, + ].join("\n"); + /** The removed import one module away from the config: the shape an owned generator has. */ + const VIA_OWNED_MODULE = [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { diagram } from "./codegen/generators/diagram.js";`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: [diagram()] });`, + ].join("\n"); + + test("a config that still calls the removed factory is told what replaced it, and nothing is written", async () => { + const root = setupRepo(REMOVED_FACTORY); + try { + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).not.toBe(0); + expect(gen.output).toContain('imports mermaidErDiagram from "@metaobjectsdev/codegen-ts/generators", and 1.1 removed that generator'); + expect(gen.output).toContain("`meta docs` draws the ER diagram"); + expect(gen.output).toContain("delete the import and the `mermaidErDiagram()` entry"); + // Not the runtime's own words for a missing export. + expect(gen.output).not.toContain("is not a function"); + expect(existsSync(join(root, OUT))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("a config that still names \"mermaid-er\" is told the same, not \"unknown generator\"", async () => { + const root = setupRepo(REMOVED_NAME); + try { + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).not.toBe(0); + expect(gen.output).toContain('generator "mermaid-er" was removed in 1.1'); + expect(gen.output).toContain("`meta docs` draws the ER diagram"); + expect(gen.output).not.toContain("unknown generator"); + expect(existsSync(join(root, OUT))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("the removed import in a module the config imports is diagnosed too", async () => { + const root = setupRepo(VIA_OWNED_MODULE); + try { + mkdirSync(join(root, "codegen", "generators"), { recursive: true }); + writeFileSync( + join(root, "codegen", "generators", "diagram.ts"), + [ + `import { mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + `export const diagram = () => mermaidErDiagram({ outFile: "docs/erd.md" });`, + ].join("\n"), + ); + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).not.toBe(0); + expect(gen.output).toContain('mermaidErDiagram is no longer exported from "@metaobjectsdev/codegen-ts/generators"'); + expect(gen.output).toContain("`meta docs` draws the ER diagram"); + expect(gen.output).toContain("codegen/generators/"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("and `meta docs` does draw them: the whole model on the overview page, a neighbourhood on an entity's page", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const pages = tree(root, DOCS); + expect(pages.get("README.md")).toContain("## Diagram\n\n```mermaid\nerDiagram\n"); + expect(pages.get("README.md")).toContain("User ||--o{ Post"); + expect(pages.get("Post.md")).toContain("```mermaid\nflowchart TB\n"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); +}); diff --git a/server/typescript/packages/cli/test/integration/eject-shared-model.test.ts b/server/typescript/packages/cli/test/integration/eject-shared-model.test.ts new file mode 100644 index 000000000..004e856ee --- /dev/null +++ b/server/typescript/packages/cli/test/integration/eject-shared-model.test.ts @@ -0,0 +1,301 @@ +/** + * An OWNED shared-model generator still publishes an artifact a consumer's `meta deps sync` + * verifies — end to end, through the BUILT CLI (ADR-0034 Amendment 7). + * + * `shared-model` was the one generator ruled package-only for a reason of substance: its + * output is not a helper, it is a contract. A consumer hashes the artifact, checks the hash + * against the manifest, and loads the document with its own vocabulary. So it was made + * ejectable the way `cube-model` is split: the copy an adopter owns decides which source + * files may contribute and which version is published, and everything a consumer checks + * stays in the package (`buildSharedModel`, `renderSharedModelFiles`). + * + * The byte-identity gate in codegen-ts proves the copy writes the same two files. This file + * proves the claim that matters to the OTHER project: published by an ejected copy, the + * artifact is the one the dependency corpus pins, and a real consumer syncs it, locks it and + * passes `meta verify --deps` against it. Then the gate is shown to convict: an artifact + * changed after it was published is refused, so the green run above was a real check. + * + * Subprocess-per-command for the reason `verify-codegen-ejected-generator` gives: the + * config is re-read under a fresh name each load but the modules it imports are cached. + */ +import { describe, test, expect, beforeAll } from "bun:test"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { sha256Integrity } from "@metaobjectsdev/sdk"; +import { meta, requireFreshDist } from "./support/built-cli.js"; + +const FIXTURES = resolve(import.meta.dirname, "..", "fixtures"); +// Inside the workspace, so an ejected template's `@metaobjectsdev/codegen-ts` import +// resolves by walking up to the workspace root. +const WORKSPACE_TMP = join(FIXTURES, "__tmp__"); + +const DEP_NAME = "acme-common"; +const ARTIFACT = `${DEP_NAME}.metaobjects.json`; +const MANIFEST = "metaobjects.pkg.json"; +const PUBLISHED = "published"; + +// The artifact every port's dependency corpus pins for this model +// (fixtures/dependency-conformance/README.md), read as raw bytes. +const PINNED_ARTIFACT = readFileSync( + resolve(import.meta.dirname, "../../../../../../fixtures/dependency-conformance/artifacts/acme-common-v1.json"), +); + +/** The publisher's model: the three nodes of the pinned artifact, over two files. */ +const PUBLISHER_BASE = { + "metadata.root": { + package: "acme::common", + children: [ + { + "object.value": { + name: "Address", + children: [{ "field.string": { name: "city" } }, { "field.string": { name: "street" } }], + }, + }, + { + "object.entity": { + name: "Audited", + abstract: true, + children: [{ "field.timestamp": { name: "createdAt" } }], + }, + }, + ], + }, +}; +const PUBLISHER_CUSTOMER = { + "metadata.root": { + package: "acme::common", + children: [ + { + "object.entity": { + name: "Customer", + children: [ + { "source.rdb": { "@table": "customers" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "email", "@maxLength": 120 } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, +}; + +/** The consumer's own model, so the post-sync load has something of its own to combine. */ +const CONSUMER_APP = { + "metadata.root": { + package: "app", + children: [ + { + "object.entity": { + name: "Order", + children: [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, +}; + +function publisherConfig(importFrom: string): string { + return [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { sharedModelFile } from ${JSON.stringify(importFrom)};`, + `export default defineConfig({`, + ` outDir: ${JSON.stringify(PUBLISHED)},`, + ` dialect: "sqlite",`, + ` generators: [sharedModelFile({ name: ${JSON.stringify(DEP_NAME)}, include: ["acme::common::**"], version: "1.0.0" })],`, + `});`, + ].join("\n"); +} + +const PACKAGED = publisherConfig("@metaobjectsdev/codegen-ts/generators"); +const OWNED = publisherConfig("./codegen/generators/shared-model.js"); + +/** A publisher and a consumer, siblings under one root, the consumer depending on what + * the publisher's `meta gen` writes. */ +function setupProjects(): { root: string; publisher: string; consumer: string } { + mkdirSync(WORKSPACE_TMP, { recursive: true }); + const root = mkdtempSync(join(WORKSPACE_TMP, "shared-model-")); + + const publisher = join(root, "publisher"); + mkdirSync(join(publisher, "metaobjects"), { recursive: true }); + writeFileSync(join(publisher, "metaobjects", "meta.common-base.json"), JSON.stringify(PUBLISHER_BASE, null, 2)); + writeFileSync(join(publisher, "metaobjects", "meta.common-customer.json"), JSON.stringify(PUBLISHER_CUSTOMER, null, 2)); + writeFileSync(join(publisher, "metaobjects.config.ts"), PACKAGED); + + const consumer = join(root, "consumer"); + mkdirSync(join(consumer, "metaobjects"), { recursive: true }); + writeFileSync(join(consumer, "metaobjects", "meta.app.json"), JSON.stringify(CONSUMER_APP)); + mkdirSync(join(consumer, ".metaobjects"), { recursive: true }); + writeFileSync( + join(consumer, ".metaobjects", "config.json"), + JSON.stringify({ + schema_version: 1, + sources: [], + dependencies: [{ name: DEP_NAME, path: `../publisher/${PUBLISHED}` }], + }), + ); + return { root, publisher, consumer }; +} + +function published(publisher: string): { artifact: Buffer; manifest: string } { + return { + artifact: readFileSync(join(publisher, PUBLISHED, ARTIFACT)), + manifest: readFileSync(join(publisher, PUBLISHED, MANIFEST), "utf8"), + }; +} + +// Precondition once per FILE, not per test: this gate runs the BUILT CLI. +beforeAll(requireFreshDist); + +describe("shared-model — published by an owned copy, verified by a consumer", () => { + test("an ejected copy publishes the pinned artifact, and `meta deps sync` locks and verifies it", async () => { + const { root, publisher, consumer } = setupProjects(); + try { + // As shipped, first: what the package's generator publishes. + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + const packaged = published(publisher); + expect(packaged.artifact.equals(PINNED_ARTIFACT)).toBe(true); + + // Take ownership, changing nothing, and wire the copy as `meta eject` prints. + const eject = await meta(publisher, "eject", "shared-model", "--format", "text"); + expect(eject).toMatchObject({ exit: 0 }); + expect(eject.output).toContain('import { sharedModelFile } from "./codegen/generators/shared-model.js";'); + // The entry names the options the factory cannot be called without. + expect(eject.output).toContain("If sharedModelFile({ name, include }) is not in `generators` yet"); + writeFileSync(join(publisher, "metaobjects.config.ts"), OWNED); + + // The committed artifact was written by the PACKAGE's generator. The codegen drift + // gate runs the owned copy and finds nothing to change. + expect(await meta(publisher, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + // Published from scratch by the owned copy: the same two files, to the byte. + rmSync(join(publisher, PUBLISHED), { recursive: true, force: true }); + rmSync(join(publisher, ".metaobjects"), { recursive: true, force: true }); + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + const owned = published(publisher); + expect(owned.artifact.equals(PINNED_ARTIFACT)).toBe(true); + expect(owned.manifest).toBe(packaged.manifest); + // The manifest's hash is the hash of the bytes beside it, in the format sdk verifies. + expect((JSON.parse(owned.manifest) as { integrity: string }).integrity).toBe(sha256Integrity(owned.artifact)); + + // The consumer's side of the contract, against what the owned copy published. + const sync = await meta(consumer, "deps", "sync"); + expect(sync).toMatchObject({ exit: 0 }); + const snapshot = join(consumer, ".metaobjects", "deps", DEP_NAME, ARTIFACT); + expect(readFileSync(snapshot).equals(PINNED_ARTIFACT)).toBe(true); + const lock = JSON.parse(readFileSync(join(consumer, ".metaobjects", "deps.lock.json"), "utf8")) as { + dependencies: Record; + }; + expect(lock.dependencies[DEP_NAME]).toMatchObject({ + version: "1.0.0", + integrity: sha256Integrity(PINNED_ARTIFACT), + nodes: ["acme::common::Address", "acme::common::Audited", "acme::common::Customer"], + }); + expect(await meta(consumer, "deps", "check")).toMatchObject({ exit: 0 }); + expect(await meta(consumer, "verify", "--deps")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 240_000); + + test("the consumer's check is real: an artifact changed after it was published is refused", async () => { + const { root, publisher, consumer } = setupProjects(); + try { + expect(await meta(publisher, "eject", "shared-model")).toMatchObject({ exit: 0 }); + writeFileSync(join(publisher, "metaobjects.config.ts"), OWNED); + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + + // What an owned copy that wrote its own bytes after hashing would produce: a document + // that no longer matches the manifest beside it. + const artifact = join(publisher, PUBLISHED, ARTIFACT); + writeFileSync(artifact, readFileSync(artifact, "utf8").replace('"city"', '"town"')); + + const sync = await meta(consumer, "deps", "sync"); + expect(sync.exit).not.toBe(0); + expect(sync.output).toContain("but the manifest records"); + expect(existsSync(join(consumer, ".metaobjects", "deps.lock.json"))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + // `meta verify --codegen` could not run this generator at all, packaged or owned: it + // regenerated without the project's source files ("no source files to load") and read + // the version default from the temp tree it writes into. Both are what `meta gen` gives + // the generator, so the gate now gives them too. + test("the codegen drift gate runs the packaged generator as `meta gen` does: the project's files and its package.json version", async () => { + const { root, publisher } = setupProjects(); + try { + // No `version` option: the default is the nearest package.json, which is the project's. + writeFileSync(join(publisher, "metaobjects.config.ts"), PACKAGED.replace(', version: "1.0.0"', "")); + writeFileSync(join(publisher, "package.json"), JSON.stringify({ name: "publisher", version: "2.0.0" })); + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + expect((JSON.parse(published(publisher).manifest) as { version: string }).version).toBe("2.0.0"); + + expect(await meta(publisher, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + // And it convicts: the model moved, the published artifact did not. + const customer = join(publisher, "metaobjects", "meta.common-customer.json"); + writeFileSync(customer, readFileSync(customer, "utf8").replace('"@maxLength": 120', '"@maxLength": 200')); + const stale = await meta(publisher, "verify", "--codegen"); + expect(stale.exit).toBe(1); + expect(stale.output).toContain(ARTIFACT); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("what the copy owns is honoured: its version source, without touching what a consumer checks", async () => { + const { root, publisher, consumer } = setupProjects(); + try { + expect(await meta(publisher, "eject", "shared-model")).toMatchObject({ exit: 0 }); + // No `version` option, and the copy's own lookup replaced: this project publishes + // the version it keeps in a file of its own rather than in a package.json. + writeFileSync( + join(publisher, "metaobjects.config.ts"), + OWNED.replace(', version: "1.0.0"', ""), + ); + writeFileSync(join(publisher, "VERSION"), "7.8.9\n"); + const copy = join(publisher, "codegen", "generators", "shared-model.ts"); + const source = readFileSync(copy, "utf8"); + const lookup = "const version = opts.version ?? nearestPackageVersion(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd());"; + expect(source).toContain(lookup); + writeFileSync( + copy, + source.replace( + lookup, + 'const version = opts.version ?? readFileSync(join(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd(), "VERSION"), "utf-8").trim();', + ), + ); + + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + const owned = published(publisher); + expect((JSON.parse(owned.manifest) as { version: string }).version).toBe("7.8.9"); + // The artifact is the pinned one still: the version is the manifest's, not the document's. + expect(owned.artifact.equals(PINNED_ARTIFACT)).toBe(true); + + // The drift gate regenerates in a temp tree, and `ctx.inputRoot` is still the PROJECT, + // so the copy's own `VERSION` lookup resolves there too: clean now, and convicted when + // the file moves on. + expect(await meta(publisher, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + expect(await meta(consumer, "deps", "sync")).toMatchObject({ exit: 0 }); + expect(await meta(consumer, "verify", "--deps")).toMatchObject({ exit: 0 }); + const lock = JSON.parse(readFileSync(join(consumer, ".metaobjects", "deps.lock.json"), "utf8")) as { + dependencies: Record; + }; + expect(lock.dependencies[DEP_NAME]?.version).toBe("7.8.9"); + + writeFileSync(join(publisher, "VERSION"), "7.9.0\n"); + const stale = await meta(publisher, "verify", "--codegen"); + expect(stale.exit).toBe(1); + expect(stale.output).toContain(MANIFEST); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); +}); diff --git a/server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts b/server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts new file mode 100644 index 000000000..299dbcd46 --- /dev/null +++ b/server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts @@ -0,0 +1,106 @@ +// A removed generator factory has TWO doors into a project, and both get a diagnostic. +// +// `removedGeneratorImportError` reads the config SOURCE, so it fires when the config itself +// imports the removed factory from "@metaobjectsdev/codegen-ts/generators" and can name the +// file and the import. `removedGeneratorRuntimeError` recovers the same answer from the +// THROW, for a removed export imported by a module the config imports — an owned generator +// under `codegen/generators/` is the usual one — where the source scan sees a clean config. +// +// Without them the removal reaches an adopter as whatever the runtime says about a missing +// export, which names neither MetaObjects, nor the removal, nor what to do. +// +// The one removed factory today is `mermaidErDiagram` (1.1): `meta docs` draws the diagram. + +import { describe, test, expect } from "bun:test"; +import { + removedGeneratorImportError, + removedGeneratorRuntimeError, +} from "../../src/lib/load-metaobjects-config.js"; + +describe("removedGeneratorImportError", () => { + test("names the import, the removal and what replaced it", () => { + const msg = removedGeneratorImportError( + `import { entityFile, mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + ); + expect(msg).toContain('imports mermaidErDiagram from "@metaobjectsdev/codegen-ts/generators"'); + expect(msg).toContain("1.1 removed that generator"); + expect(msg).toContain("`meta docs` draws the ER diagram"); + expect(msg).toContain("delete the import and the `mermaidErDiagram()` entry"); + // The file it used to write is the other thing an upgrade leaves behind. + expect(msg).toContain("docs/model.md"); + }); + + test("sees through an alias, a type import, single quotes and a multi-line list", () => { + for (const source of [ + `import { mermaidErDiagram as er } from "@metaobjectsdev/codegen-ts/generators";`, + `import { mermaidErDiagram } from '@metaobjectsdev/codegen-ts/generators';`, + `import {\n entityFile,\n mermaidErDiagram,\n} from "@metaobjectsdev/codegen-ts/generators";`, + ]) { + expect(removedGeneratorImportError(source)).toContain("mermaidErDiagram"); + } + }); + + test("stays out of the way of every import that still resolves", () => { + for (const source of [ + `import { entityFile, routesFile, docsFile } from "@metaobjectsdev/codegen-ts/generators";`, + // The same symbol from somewhere else is the project's own, not the removed one. + `import { mermaidErDiagram } from "./codegen/generators/my-diagram.js";`, + // The diagram builder `meta docs` uses is public and stays. + `import { renderMermaidErBlock } from "@metaobjectsdev/codegen-ts";`, + "", + ]) { + expect(removedGeneratorImportError(source)).toBeUndefined(); + } + }); +}); + +describe("removedGeneratorRuntimeError", () => { + test("recognises a transpiled config's shape", () => { + const msg = removedGeneratorRuntimeError( + new TypeError("(0 , _generators.mermaidErDiagram) is not a function"), + ); + expect(msg).toContain('mermaidErDiagram is no longer exported from "@metaobjectsdev/codegen-ts/generators"'); + expect(msg).toContain("`meta docs` draws the ER diagram"); + }); + + test("recognises native ESM's two spellings of a missing export", () => { + // Matching is on the SYMBOL and the fact of a missing export, not on one message + // shape: the shape belongs to whichever runtime loaded the module. + for (const message of [ + "The requested module '@metaobjectsdev/codegen-ts/generators' does not provide an export named 'mermaidErDiagram'", + "Export named 'mermaidErDiagram' not found in module '/x/codegen-ts/dist/generators/index.js'.", + "mermaidErDiagram is not a function", + ]) { + expect(removedGeneratorRuntimeError(new SyntaxError(message))).toContain("1.1 removed that generator"); + } + }); + + test("says WHERE to look, since the throw cannot name the file", () => { + const msg = removedGeneratorRuntimeError(new TypeError("mermaidErDiagram is not a function")); + expect(msg).toContain("codegen/generators/"); + expect(msg).toContain("a module the config imports"); + }); + + test("stays out of the way of unrelated errors", () => { + expect(removedGeneratorRuntimeError(new TypeError("foo is not a function"))).toBeUndefined(); + expect(removedGeneratorRuntimeError(new Error("mermaidErDiagram exploded"))).toBeUndefined(); + expect(removedGeneratorRuntimeError(undefined)).toBeUndefined(); + }); + + test("a non-Error throw does not crash the diagnostic", () => { + expect(removedGeneratorRuntimeError("mermaidErDiagram is not a function")).toContain("`meta docs`"); + }); +}); + +describe("the two doors agree", () => { + test("both name the removal and the replacement", () => { + const fromSource = removedGeneratorImportError( + `import { mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + ); + const fromThrow = removedGeneratorRuntimeError(new TypeError("mermaidErDiagram is not a function")); + for (const needle of ["1.1 removed that generator", "`meta docs` draws the ER diagram", "run `meta docs`"]) { + expect(fromSource).toContain(needle); + expect(fromThrow).toContain(needle); + } + }); +}); diff --git a/server/typescript/packages/codegen-ts/src/docs-door.ts b/server/typescript/packages/codegen-ts/src/docs-door.ts new file mode 100644 index 000000000..0187dee56 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/docs-door.ts @@ -0,0 +1,73 @@ +// The docs door's seam for an owned docs generator (ADR-0034 Amendment 7). +// +// `meta docs` is the one door for documentation (ADR-0021 D1, ADR-0025), and it used to +// construct the packaged `docsFile()` / `apiDocsFile()` itself, with options it computed +// from the `docs:` config. There was nothing an owned copy could be handed to, so the two +// generators could not be ejected. The seam has two halves: +// +// - WHICH generator runs. An owned copy is listed in the docs door's own config, +// `docs: { generators: [...] }`, and carries `docsSurface`. It replaces the packaged +// generator for that surface (`docsDoorGenerators`). It is not listed in the top-level +// `generators`: that array is what `meta gen` runs, and `meta gen` does not run docs. +// - WHAT the door tells it. The facts the door used to pass as factory options now ride +// on the context (`ctx.docsDoor`), because a listed generator is already constructed by +// the time the door sees it. They are the facts the two page sets must agree on (where +// the api pages are, whether the model pages exist), so the door states them once and +// both generators read the same answer. +import type { Generator, OwnedDocsSurface } from "./generator.js"; + +/** + * What the docs door tells the generator it runs for a surface. Present on `ctx.docsDoor` + * only under `meta docs` / `meta verify --docs`; absent on a `meta gen` run and on a direct + * `.generate(ctx)` call. + */ +export interface DocsDoorFacts { + /** For the MODEL surface: the api surfaces emitted alongside it, which every model page + * and the index link to. Absent: the model pages link to none. */ + readonly apiSurfaces?: ReadonlyArray<{ label: string; subDir: string; baseUrl?: string }>; + /** For the API surface: the directory under the docs root its pages go in. */ + readonly apiSubDir?: string; + /** For the API surface: whether the model surface is emitted alongside it, so each api + * page can link back to its model page. */ + readonly modelSurface?: boolean; +} + +/** + * The owned generator for each surface out of `docs.generators`, keyed by surface. A + * surface with no entry is run by the packaged generator. + * + * Refused rather than resolved by order or ignored, each naming the entry: + * - an entry with no `docsSurface`. The door runs one generator per surface and has no + * surface to give it, so running it or dropping it would both be a guess. The + * packaged `docsFile()` / `apiDocsFile()` carry no marker and need no listing; a + * generator `meta gen` runs belongs in the top-level `generators`. + * - two entries for one surface. Each would write the same pages, and picking one + * silently is the two-answers defect the single door exists to prevent. + */ +export function docsDoorGenerators( + generators: readonly Generator[], +): Partial> { + const bySurface: Partial> = {}; + for (const generator of generators) { + const surface = generator.docsSurface; + if (surface === undefined) { + throw new Error( + `\`docs.generators\` lists "${generator.name}", which carries no \`docsSurface\`: ` + + "`meta docs` runs one generator per surface (model, api) and cannot tell which one " + + "this replaces. An owned copy from `meta eject docs` or `meta eject api-docs` " + + "carries it. The packaged docs generators need no listing, and a generator " + + "`meta gen` runs belongs in the top-level `generators`.", + ); + } + const first = bySurface[surface]; + if (first !== undefined) { + throw new Error( + `\`docs.generators\` lists two generators for the ${surface} surface ` + + `("${first.name}" and "${generator.name}"): \`meta docs\` runs one per surface. ` + + "Keep one of them.", + ); + } + bySurface[surface] = generator; + } + return bySurface; +} diff --git a/server/typescript/packages/codegen-ts/src/generator-registry.ts b/server/typescript/packages/codegen-ts/src/generator-registry.ts index b6d9b7cf4..d2e0aa01a 100644 --- a/server/typescript/packages/codegen-ts/src/generator-registry.ts +++ b/server/typescript/packages/codegen-ts/src/generator-registry.ts @@ -19,12 +19,14 @@ // // Tiering (ADR-0020 / ADR-0021 D1): // - "native" — the recommended Tier-1 `meta gen` suite (idiomatic emission). -// - "neutral" — Tier-2 artifacts owned by the neutral docs engine. `docs` and -// `mermaid-er` are present here for identity/discoverability but -// are NOT part of the recommended native surface — the canonical -// door for documentation is `meta docs` (D1). +// - "neutral" — Tier-2 artifacts owned by the neutral docs engine. `docs` is present +// here for identity/discoverability but is NOT part of the native +// surface — the door for documentation is `meta docs` (D1). (The +// standalone `mermaid-er` generator was a second neutral entry until +// 1.1 removed it: `meta docs` draws the diagram. See +// metaobjects-config.ts, `removedGeneratorMessage`.) -import type { Generator } from "./generator.js"; +import type { Generator, OwnedDocsSurface } from "./generator.js"; import type { MetaobjectsGenConfig } from "./metaobjects-config.js"; import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; import { @@ -35,7 +37,6 @@ import { callableFile, routesFileHono, namesFile, - mermaidErDiagram, promptRender, outputParser, extractor, @@ -154,6 +155,16 @@ export interface GeneratorRegistryEntry { configKeys?: readonly (keyof MetaobjectsGenConfig)[]; /** True iff this package ships a `src/reference/.ts` for `meta eject`. */ ejectable: boolean; + /** + * The `meta docs` surface an OWNED copy of this generator takes over. Set for the two + * generators `meta docs` runs (`docs`, `api-docs`): their owned copy is listed in + * `docs: { generators: [...] }`, the docs door's own block, and `meta eject` prints that + * wiring for them. Absent for every generator `meta gen` runs. + * + * Resolved, not trusted: a gate requires the reference template's generator to carry + * this same value as its `docsSurface` marker, which is what `meta docs` reads. + */ + docsSurface?: OwnedDocsSurface; /** Optional note — used to point neutral entries at their canonical door. */ note?: string; } @@ -284,6 +295,7 @@ export const generatorRegistry: Record = { factoryExport: "apiDocsFile", options: "filter?, target?", ejectable: ejectable("api-docs"), + docsSurface: "api", }, docs: { name: "docs", @@ -294,19 +306,9 @@ export const generatorRegistry: Record = { factory: () => docsFile(), factoryExport: "docsFile", ejectable: ejectable("docs"), + docsSurface: "model", note: "neutral artifact — use `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended `meta gen` native suite.", }, - "mermaid-er": { - name: "mermaid-er", - kind: "generator", - layer: "docs", - description: "Mermaid ER diagram of the entity/relationship model.", - tier: "neutral", - factory: () => mermaidErDiagram(), - factoryExport: "mermaidErDiagram", - ejectable: ejectable("mermaid-er"), - note: "neutral artifact owned by the docs engine (ADR-0020), deprecated under `meta gen`: do not add it to a new config. A `mermaidErDiagram()` already wired in `generators` still runs; `meta docs` draws the same diagram on its overview page.", - }, // ----- capability ------------------------------------------------------- // Not chosen by browsing this list — chosen because the MODEL already asks for diff --git a/server/typescript/packages/codegen-ts/src/generator.ts b/server/typescript/packages/codegen-ts/src/generator.ts index b91fff542..53416311d 100644 --- a/server/typescript/packages/codegen-ts/src/generator.ts +++ b/server/typescript/packages/codegen-ts/src/generator.ts @@ -2,6 +2,7 @@ import type { ExposeOption } from "./routes-expose.js"; import type { MetaObject, MetaRoot, TypeRegistry } from "@metaobjectsdev/metadata"; import type { LibraryManifest } from "@metaobjectsdev/metadata/library"; import type { RenderContext } from "./render-context.js"; +import type { DocsDoorFacts } from "./docs-door.js"; import type { ResolvedGenConfig } from "./metaobjects-config.js"; import type { OrphanPolicy } from "./reconcile-orphans.js"; import { effectivePackage } from "./docs-paths.js"; @@ -87,9 +88,31 @@ export interface GenContext { * the opposite and is meaningful: the caller looked, and this project opted into none. */ libraries?: readonly LibraryManifest[]; + /** + * The real project directory, for a generator that READS the project's own files: the + * nearest `package.json`, a version file, anything it resolves against the project. Under + * `meta gen` it is `projectRoot`. Under `meta verify --codegen` the run writes into a + * throwaway tree, `projectRoot` is that tree (import paths between an output directory + * and the project are computed against it, so it has to mirror the layout), and this is + * the project the tree mirrors. Read files through this; compute output-relative paths + * against `projectRoot`. Undefined when the caller supplied neither root. + */ + inputRoot?: string; + /** + * What the docs door (`meta docs`, `meta verify --docs`) tells the generator it runs for a + * surface: where the api pages are, whether the model pages are emitted. Absent on a + * `meta gen` run and on a direct `.generate(ctx)` call, where a docs generator's own + * options decide instead. On the context rather than on `config` because it is a fact + * about the run's door, not something a project sets. See docs-door.ts. + */ + docsDoor?: DocsDoorFacts; warn: (msg: string) => void; } +/** The `meta docs` surfaces an owned generator can take over (`Generator.docsSurface`). The + * `requirements` and `agent` surfaces are not generators in the catalog and stay packaged. */ +export type OwnedDocsSurface = "model" | "api"; + export interface Generator { /** kebab-case identifier; surfaces in diagnostics + drift logs. */ name: string; @@ -121,6 +144,16 @@ export interface Generator { * Absent `expose` means every verb. A copy ejected before this marker existed needs * `meta eject routes --force` (or the same one-line addition) to be honoured. */ mountsRoutes?: { readonly expose?: ExposeOption }; + /** Marks a generator that `meta docs` runs for one of its surfaces IN PLACE OF the + * packaged one: `"model"` for the per-object pages and their index, `"api"` for the + * reference of the generated API. It is how an owned (ejected) docs generator is reached: + * list it in `docs: { generators: [...] }` in metaobjects.config.ts, and `meta docs` and + * `meta verify --docs` read it there by this marker (`docsDoorGenerators`). It does not + * go in the top-level `generators`: `meta gen` does not run documentation (ADR-0021 D1, + * ADR-0025), and skips a generator carrying this marker with a warning that says where + * it belongs. The packaged `docsFile()` / `apiDocsFile()` do not carry it: they are what + * `meta docs` runs when nothing is listed, and need no listing. */ + docsSurface?: OwnedDocsSurface; /** §A6 — marks the generator that emits the Names artifact. The runner * aggregates this across the suite into ResolvedGenConfig.includeNames, which the * entity generator reads to decide whether it may reference those constants. diff --git a/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts b/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts index 163e33b69..825408307 100644 --- a/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts @@ -13,7 +13,7 @@ // • `README.md` — the consolidated human index (GitHub treats it as landing), // • `AGENT-API.md` — the token-frugal agent form. // -// Unlike `docs`/`mermaid-er` (Tier-2 neutral, owned by `meta docs`), `api-docs` +// Unlike `docs` (Tier-2 neutral, owned by `meta docs`), `api-docs` // is a NATIVE generator: it is REGISTERED in the generator registry (ADR-0022 // Part 3) — so it appears in `gen --list` and is selectable by its stable name // `api-docs`. It is NOT a `meta docs` mode. @@ -39,7 +39,7 @@ import { type DocPagePlacement, } from "../docs-paths.js"; import { projectProvider } from "../render-engine/framework-provider.js"; -import { buildApiModel } from "./api-model.js"; +import { buildApiModelForRun } from "./api-model.js"; import { renderEntityApiPage, renderApiIndex, @@ -82,31 +82,21 @@ export const apiDocsFile = function apiDocsFile(opts?: ApiDocsFileOpts): Generat const provider = projectProvider(ctx.projectRoot ?? process.cwd()); const layout = ctx.config.outputLayout ?? "flat"; - // Per-call output prefix. Default `docs/api`; `meta docs` passes `api`. - const apiDir = opts?.subDir ?? DEFAULT_API_DIR; + // Per-call output prefix. Default `docs/api`; `meta docs` passes `api`. Under the docs + // door the door's facts decide it and the model cross-link, not this factory's options: + // the model generator is told the same thing, and the two must agree (docs-door.ts). + const door = ctx.docsDoor; + const apiDir = (door !== undefined ? door.apiSubDir : opts?.subDir) ?? DEFAULT_API_DIR; + const modelSurface = door !== undefined ? door.modelSurface : opts?.modelSurface; const indexFilename = `${apiDir}/README.md`; const agentFilename = `${apiDir}/AGENT-API.md`; - // ONE ApiModel feeds every form (Task-1 builder; Task-2 renderers). The - // pkMap is reused from the run's renderContext when present (the real gen - // run always provides it) and derived otherwise. - // Auto-detect: document the OPT-IN Hono CRUD surface iff the Hono routes - // generator is actually in the run. The runner aggregates each generator's - // `emitsHonoRoutes` marker into ctx.config.includeHonoRoutes, so api-docs - // "just works" — it documents Hono exactly when `routesFileHono` is wired, - // and omits it (Fastify-only) otherwise. No explicit opt needed. - const model = buildApiModel(ctx.loadedRoot, { - loadedRoot: ctx.loadedRoot, - outputLayout: layout, - includeHonoRoutes: ctx.config.includeHonoRoutes ?? false, - // The verbs the routes generators actually mount (`expose`), so a page never lists a - // route the project does not serve. - ...(ctx.config.routesExpose !== undefined && { expose: ctx.config.routesExpose }), - // The project's prefix, because every route is mounted inside it. Without - // this the page documents `/authors` for routes served at `/api/authors`. - apiPrefix: ctx.renderContext?.apiPrefix ?? "", - ...(ctx.renderContext?.pkMap !== undefined && { pkMap: ctx.renderContext.pkMap }), - }); + // ONE ApiModel feeds every form (Task-1 builder; Task-2 renderers). The facts of the + // run it reads (layout, whether Hono routes are wired, the verbs the routes generators + // mount, `apiPrefix`, the pk map) are gathered by `buildApiModelForRun`, which the + // reference template calls too, so an owned copy never restates them and cannot fall + // behind on one. + const model = buildApiModelForRun(ctx); // Track (path, fqn) for the SAME hard collision backstop docsFile() uses — // two units that resolve to one path (flat, cross-package short-name clash) @@ -126,7 +116,7 @@ export const apiDocsFile = function apiDocsFile(opts?: ApiDocsFileOpts): Generat // `/`. The href is derived from the SAME // docPageOutputPath placement via surfaceCrossHref so it resolves in both // layouts. ABSENT otherwise. - const modelPageHref = opts?.modelSurface + const modelPageHref = modelSurface ? surfaceCrossHref(path, docPageOutputPath(layout, node)) : undefined; return { path, content: renderEntityApiPage(unit, provider, modelPageHref) }; diff --git a/server/typescript/packages/codegen-ts/src/generators/api-model.ts b/server/typescript/packages/codegen-ts/src/generators/api-model.ts index ce1c54f12..220f28d6b 100644 --- a/server/typescript/packages/codegen-ts/src/generators/api-model.ts +++ b/server/typescript/packages/codegen-ts/src/generators/api-model.ts @@ -144,6 +144,7 @@ import { generatableObjects } from "../source-detect.js"; import { effectivePackage } from "../docs-paths.js"; import { entityOutputPath, type OutputLayout } from "../import-path.js"; import type { RenderContext } from "../render-context.js"; +import type { GenContext } from "../generator.js"; import type { PkInfo } from "../pk-resolver.js"; import { modelFieldShapes, @@ -1258,3 +1259,31 @@ function templateExample(symbolBase: string, symbols: ApiSymbol[]): UnitExample } return { imports: importLines(picks), body }; } + +/** + * The API model for a RUN: {@link buildApiModel} with its context read off the run rather + * than restated by the caller. + * + * Which facts of a run the documented API depends on is this package's knowledge: the page + * layout, whether a Hono routes generator is wired (the runner aggregates each generator's + * `emitsHonoRoutes` marker into `ctx.config.includeHonoRoutes`, so the Hono surface is + * documented exactly when it is emitted), the verbs each routes generator mounts + * (`ctx.config.routesExpose`, from `routesFile({ expose })`), the project's `apiPrefix` + * (every route is mounted inside it, so without it a page documents `/authors` for routes + * served at `/api/authors`), and the run's primary-key map. The `api-docs` generator and its reference template both + * call this, so when the model learns to read another fact of the run, an owned copy reads + * it too on its next package upgrade instead of quietly documenting less than the packaged + * generator does. + */ +export function buildApiModelForRun(ctx: GenContext): ApiModel { + return buildApiModel(ctx.loadedRoot, { + loadedRoot: ctx.loadedRoot, + outputLayout: ctx.config.outputLayout ?? "flat", + includeHonoRoutes: ctx.config.includeHonoRoutes ?? false, + // The verbs the routes generators actually mount (`expose`), so a page never lists a + // route the project does not serve. + ...(ctx.config.routesExpose !== undefined && { expose: ctx.config.routesExpose }), + apiPrefix: ctx.renderContext?.apiPrefix ?? "", + ...(ctx.renderContext?.pkMap !== undefined && { pkMap: ctx.renderContext.pkMap }), + }); +} diff --git a/server/typescript/packages/codegen-ts/src/generators/docs-file.ts b/server/typescript/packages/codegen-ts/src/generators/docs-file.ts index 7336bc5d5..8cd0c08ea 100644 --- a/server/typescript/packages/codegen-ts/src/generators/docs-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/docs-file.ts @@ -20,8 +20,6 @@ import type { MetaObject, MetaRoot } from "@metaobjectsdev/metadata"; import { TYPE_TEMPLATE } from "@metaobjectsdev/metadata"; -import { render } from "@metaobjectsdev/render"; -import type { Provider } from "@metaobjectsdev/render"; import type { Generator, GeneratorFactory, EmittedFile } from "../generator.js"; import { docPageOutputPath, @@ -33,6 +31,7 @@ import { type DocPagePlacement, } from "../docs-paths.js"; import { projectProvider } from "../render-engine/framework-provider.js"; +import { renderDocPage } from "../render-engine/doc-page.js"; import { renderMermaidErBlock } from "../templates/mermaid-er.js"; import { buildEntityDocData } from "./docs-data-builder.js"; import { isReport, servedReport } from "../source-detect.js"; @@ -64,20 +63,6 @@ const TEMPLATE_REF = "docs/entity-page.md"; // sibling artifact, distinct from the entity page (Task 3). const TEMPLATE_PAGE_REF = "docs/template-page.md"; -/** Render one docs page, wrapping any engine error with the page ref + output - * path so a template failure points at the exact page (shared by the entity - * and template.output emission paths — identical error contract). */ -function renderDocPage(ref: string, payload: unknown, provider: Provider, path: string): string { - try { - return render({ ref, payload, provider, format: "markdown" }); - } catch (err) { - const msg = err instanceof Error ? err.message : String(err); - throw new Error(`docs-file: failed rendering '${ref}' for '${path}': ${msg}`, { - cause: err instanceof Error ? err : undefined, - }); - } -} - /** * @deprecated ADR-0021 D1: `meta docs` is the single door for documentation. * `docsFile()` stays as the INTERNAL engine that `meta docs` calls — do NOT add @@ -101,8 +86,14 @@ export const docsFile = function docsFile(opts?: DocsFileOpts): Generator { // baseUrl). The api page for a node lives at `/`, // so the from-path doubles as the page placement. ABSENT/empty surfaces ⇒ // undefined → output byte-identical to historical model-only runs. + // + // Under the docs door the door's facts decide which surfaces those are, not this + // factory's options: the api generator is told the same thing, and the two must agree + // (docs-door.ts). The options decide on a direct call, which has no door. + const door = ctx.docsDoor; + const apiSurfaces = door !== undefined ? door.apiSurfaces : opts?.apiSurfaces; const apiRefsFor = (fromPath: string): Array<{ label: string; href: string }> | undefined => - opts?.apiSurfaces?.map((s) => ({ label: s.label, href: apiSurfaceHref(fromPath, s, fromPath) })); + apiSurfaces?.map((s) => ({ label: s.label, href: apiSurfaceHref(fromPath, s, fromPath) })); // Track every (path, fqn) so we can hard-error on a collision (defense // against silent doc-page overwrite) AFTER all pages are placed. const placements: DocPagePlacement[] = []; diff --git a/server/typescript/packages/codegen-ts/src/generators/index.ts b/server/typescript/packages/codegen-ts/src/generators/index.ts index add750044..8a1f1015b 100644 --- a/server/typescript/packages/codegen-ts/src/generators/index.ts +++ b/server/typescript/packages/codegen-ts/src/generators/index.ts @@ -14,7 +14,10 @@ // // Which entries have a reference copy is declared once, in `REFERENCE_GENERATOR_NAMES` // (reference-templates.ts); `meta gen --list` marks every other entry `package-only` — -// today that is `template`, `shared-model` and the docs tier. +// today that is `template` alone, the Mustache primitive, which has no emit logic of its +// own. `docsFile` and `apiDocsFile` are exported here as the generators `meta docs` runs; +// they need no wiring. An ejected copy of either is listed in `docs: { generators: [...] }`, +// and `meta docs` then runs the copy (Amendment 7). // See spec/decisions/ADR-0034-codegen-scaffold-and-own.md. export { entityFile, type EntityFileOpts } from "./entity-file.js"; @@ -26,7 +29,6 @@ export { callableFile, type CallableFileOpts } from "./callable-file.js"; export { routesFileHono, type RoutesFileHonoOpts } from "./routes-file-hono.js"; export { namesFile } from "./names-file.js"; /** @deprecated ADR-0021 D1 — neutral artifact owned by `meta docs` (ADR-0020); not part of the recommended `meta gen` suite. */ -export { mermaidErDiagram, type MermaidErOptions } from "./mermaid-er.js"; export { promptRender, type PromptRenderOpts } from "./prompt-render-file.js"; export { outputParser, type OutputParserOpts } from "./output-parser-file.js"; export { extractor, type ExtractorOpts } from "./extractor-file.js"; diff --git a/server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts b/server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts deleted file mode 100644 index 098ad7c93..000000000 --- a/server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts +++ /dev/null @@ -1,43 +0,0 @@ -import { oncePerRun, type Generator, type GeneratorFactory } from "../generator.js"; -import { renderMermaidModel } from "../templates/mermaid-er.js"; - -export interface MermaidErOptions { - /** Output path relative to the target's outDir. Defaults to "docs/model.md". */ - outFile?: string; - /** Named output target. */ - target?: string; -} - -/** - * Emit a single Markdown file containing a Mermaid `erDiagram` plus per-entity - * prose sections. The renderer walks the loaded root for all entities; the - * default outFile is "docs/model.md". - * - * TIER NOTE (ADR-0020): the CANONICAL home of the neutral ER diagram is now the - * neutral docs engine — the OVERVIEW/index page (`README.md`) emitted by - * `docsFile()` / `meta docs` embeds the SAME `renderMermaidErBlock()` this - * generator's `renderMermaidModel()` reuses. This standalone Tier-1 generator is - * retained ONLY as a thin back-compat wrapper for adopters that already have - * `mermaidErDiagram` in their `meta gen` config; it adds NO ER logic of its own - * (no duplication) — it delegates to the shared `renderMermaidModel()` builder. - * - * @deprecated ADR-0021 D1: the ER diagram is a Tier-2 neutral artifact owned by - * the docs engine (ADR-0020). Use `meta docs` — it embeds the same ER block. - * This standalone `meta gen` generator is flagged neutral in the registry - * (`--list`) and is no longer part of the recommended native suite. Kept for - * back-compat; do not add to new gen configs. - */ -export const mermaidErDiagram = function mermaidErDiagram( - opts?: MermaidErOptions, -): Generator { - const outFile = opts?.outFile ?? "docs/model.md"; - const generator: Generator = { - name: "mermaid-er-diagram", - generate: oncePerRun((_entities, ctx) => ({ - path: outFile, - content: renderMermaidModel(ctx.loadedRoot), - })), - }; - if (opts?.target) generator.target = opts.target; - return generator; -} as GeneratorFactory; diff --git a/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts b/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts index 1bffd6fdb..e06616b76 100644 --- a/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts @@ -5,75 +5,21 @@ // it as ONE canonical-JSON `metadata.root` document plus a manifest — the artifact a // consumer's `meta deps sync` copies into its own repo (design §2.2, §3.2, §3.5). // -// The algorithm (design §4.3), in order: -// 1. Standalone load of `files` (default: the run's own source files, `ctx.sourceFiles`) -// with `ctx.registry` (the run's composed registry — the SAME vocabulary the -// project's own metadata loaded with), strict. -// 2. Select top-level nodes: never `requirement.*` / `template.*` (a publisher's -// ledger and prompts are not a consumer's shapes), and matching the compiled -// include/exclude scope. -// 3. Closure check: every `extends` target and every REF_BEARING_ATTR_NAMES value -// of a selected node (and of its own descendants) must resolve to a SELECTED -// node, else the selection is not self-contained and the build fails naming the -// (referrer, target) pairs. -// 4. Serialize the selection with `serializeSharedDocument` — raw own-layer form, -// `extends` preserved, sorted by resolution key. -// 5. Re-load the emitted document standalone with CORE providers only, strict — a -// consumer loads it with ITS providers, and a Phase 1 export must not need the -// publisher's non-core vocabulary. -// 6. Emit `.metaobjects.json` + `metaobjects.pkg.json` (design §3.2). -import { createHash } from "node:crypto"; +// This file only wires the run into the two stages that ARE the artifact, both in +// `src/shared-model/build-shared-model.ts`: `buildSharedModel` (load, select, closure, +// serialize, core-vocabulary re-load) and `renderSharedModelFiles` (the artifact, its +// hash and the manifest). What is decided here is what belongs to the publishing project: +// which of its source files may contribute and which version is published. The reference +// template an adopter ejects (`src/reference/shared-model.ts`) is this file against the +// package's public exports, held byte-identical by `test/reference-byte-identical.test.ts`. import { existsSync, readFileSync } from "node:fs"; import { dirname, join } from "node:path"; -import { - type MetaData, - type MetaRoot, - MetaDataLoader, - InMemoryStringSource, - compileScope, - matchesScope, - REF_BEARING_ATTR_NAMES, - resolveObjectRef, - serializeSharedDocument, - packageOfResolutionKey, - PACKAGE_SEPARATOR, - CHILD_REF_SEPARATOR, - TYPE_REQUIREMENT, - TYPE_TEMPLATE, - METAMODEL_VERSION, -} from "@metaobjectsdev/metadata"; -import { FileSource } from "@metaobjectsdev/metadata/core"; import { oncePerRun, type Generator, type GenContext, type EmittedFile } from "../generator.js"; -import { effectivePackage } from "../docs-paths.js"; +import { buildSharedModel, renderSharedModelFiles } from "../shared-model/build-shared-model.js"; -// The FR-023 manifest/artifact tool-file conventions (design §3.2, §4.3, "Hash -// format"). Their single canonical source is `sdk/src/dependencies.ts` -// (`MANIFEST_FILE`, `ARTIFACT_SUFFIX`, `INTEGRITY_PREFIX`) — codegen-ts cannot -// import sdk (the dependency runs the other way: `cli` depends on both, design -// §4.3's closing note), so these mirror those values locally. Keep in sync — -// exported (not merely local) so `cli`, which depends on both packages, can -// assert equality against `sdk`'s exports in a test; see -// `cli/test/shared-model-constants-parity.test.ts`. -export const MANIFEST_FILE = "metaobjects.pkg.json"; -export const ARTIFACT_SUFFIX = ".metaobjects.json"; -export const INTEGRITY_PREFIX = "sha256-"; -/** design §3.2 — the manifest schema's own version, distinct from `metamodelVersion`. */ -const MANIFEST_SCHEMA_VERSION = 1; - -/** - * Separator for `checkClosure`'s dedupe key. A resolution key never contains a NUL, - * so this cannot collide with either half it joins — unlike a printable separator - * (`->`, `→`), which would need a separate argument for why no legal FQN could ever - * contain it. Written as the `\0` ESCAPE, never a raw control byte: a raw 0x00 in - * this source file previously made `git` treat it as binary, which (a) makes - * `git diff`/GitHub show no content for the file at all, and (b) more seriously, - * makes `.githooks/pre-commit`'s public-repo-hygiene grep skip its added lines - * entirely (a binary file's diff is never text-enumerated) — silently defeating that - * guard for any future edit to this file. The escape produces the byte-identical - * runtime string; only the SOURCE FILE's byte content changes (readable text instead - * of an embedded control byte). - */ -const DEDUPE_KEY_SEPARATOR = "\0"; +// Re-exported from here because `generators/index.ts` has always served them from this +// module (as `SHARED_MODEL_*`) for the `cli` parity test against sdk's canonical copies. +export { MANIFEST_FILE, ARTIFACT_SUFFIX, INTEGRITY_PREFIX } from "../shared-model/build-shared-model.js"; export interface SharedModelFileOpts { /** The dependency key a consumer will `meta deps sync` under. Also the artifact's @@ -87,86 +33,12 @@ export interface SharedModelFileOpts { * own source files (`ctx.sourceFiles` — everything `files?` narrows). */ files?: readonly string[]; /** The published version. Default: the nearest `package.json` `version` walking - * up from `ctx.projectRoot`. */ + * up from the project (`ctx.inputRoot`). */ version?: string; /** Named output target, as on every other generator. */ target?: string; } -/** Everything before the first `CHILD_REF_SEPARATOR` that follows the LAST - * `PACKAGE_SEPARATOR` — the object part of a possibly dotted ref (ADR-0029's - * addressing model: package segments never fold onto a non-root child, so a - * `.` before the final `::`-segment can't occur). Mirrors the private - * `splitChildTail` in `naming-refs.ts` (not exported); `extends` and the - * origin `@from`/`@of`/`@via` attrs are the only REF_BEARING_ATTR_NAMES that - * can carry a dotted child tail — every other ref-bearing attr is already a - * bare object ref, and stripping a tail that isn't there is a no-op. */ -function refOwner(ref: string): string { - const lastSep = ref.lastIndexOf(PACKAGE_SEPARATOR); - const segStart = lastSep === -1 ? 0 : lastSep + PACKAGE_SEPARATOR.length; - const dotInSeg = ref.indexOf(CHILD_REF_SEPARATOR, segStart); - return dotInSeg === -1 ? ref : ref.slice(0, dotInSeg); -} - -/** Depth-first walk of a node and its OWN descendants, node itself first. ADR-0039 - * sanctioned own-only case: `serializeSharedDocument` emits each selected node's - * OWN layer only (`extends` preserved, not flattened — design §3.5), so the - * closure check must walk exactly what will be emitted, never what `extends` - * would additionally contribute (that is the base's own layer, checked - * separately if the base is itself selected). */ -function walkOwn(node: MetaData): MetaData[] { - const out: MetaData[] = [node]; - for (const child of node.ownChildren()) out.push(...walkOwn(child)); - return out; -} - -interface ClosurePair { - readonly referrer: string; - readonly target: string; -} - -/** design §4.3 step 3. `standaloneRoot` already loaded successfully (step 1), so - * every ref found here is guaranteed to RESOLVE to some node in it — the loader's - * own validation passes reject a dangling `extends`/object-ref before this ever - * runs. What is NOT guaranteed is that the resolved node is in `selectedKeys`. */ -function checkClosure( - standaloneRoot: MetaRoot, - selected: readonly MetaData[], - selectedKeys: ReadonlySet, -): ClosurePair[] { - const pairs: ClosurePair[] = []; - const seen = new Set(); - for (const top of selected) { - const referrerKey = top.resolutionKey(); - for (const node of walkOwn(top)) { - const referrerPkg = effectivePackage(node) ?? ""; - const candidates: string[] = []; - if (node.superRef !== undefined) candidates.push(node.superRef); - // ADR-0039 sanctioned own-only case (same rationale as walkOwn() above): an - // INHERITED ref-bearing attr belongs to the base node's own layer, not this - // one's, and is checked when the base itself is walked — reading it via the - // resolving `attr()` here would attribute the base's reference to this node. - for (const attrName of REF_BEARING_ATTR_NAMES) { - const value = node.ownAttr(attrName); - if (typeof value === "string") candidates.push(value); - } - for (const raw of candidates) { - const resolved = resolveObjectRef(standaloneRoot, refOwner(raw), referrerPkg).node; - if (resolved === undefined) continue; // already validated at load — defensive only - const targetKey = resolved.resolutionKey(); - if (selectedKeys.has(targetKey)) continue; - const dedupeKey = `${referrerKey}${DEDUPE_KEY_SEPARATOR}${targetKey}`; - if (seen.has(dedupeKey)) continue; - seen.add(dedupeKey); - pairs.push({ referrer: referrerKey, target: targetKey }); - } - } - } - return pairs.sort((a, b) => - a.referrer === b.referrer ? a.target.localeCompare(b.target) : a.referrer.localeCompare(b.referrer), - ); -} - /** design §2.2 — "version defaults to the host package's version". Walks up from * `startDir` for the nearest `package.json` carrying a string `version`. */ function nearestPackageVersion(startDir: string): string | undefined { @@ -200,91 +72,26 @@ export function sharedModelFile(opts: SharedModelFileOpts): Generator { ); } - // Step 1 — standalone load, the run's own composed registry (so the publisher's - // own consumer-supplied providers are honoured here exactly as they were for - // the project's main load), strict. - const loader = new MetaDataLoader({ + const model = await buildSharedModel({ + name: opts.name, + files, + include: opts.include, + ...(opts.exclude !== undefined ? { exclude: opts.exclude } : {}), ...(ctx.registry !== undefined ? { registry: ctx.registry } : {}), - strict: true, }); - const loaded = await loader.load(files.map((p) => new FileSource(p))); - if (loaded.errors.length > 0) { - throw loaded.errors[0]; - } - const standaloneRoot = loaded.root; - // Step 2 — select top-level nodes. ADR-0039 sanctioned own-only case: `MetaRoot` - // has no super chain (it never `extends`), so its own children ARE its effective - // children — the same case `MetaRoot.objects()`/`findObject()` document. - const compiled = compileScope({ include: opts.include, exclude: opts.exclude ?? [] }); - const selected = standaloneRoot - .ownChildren() - .filter((node) => node.type !== TYPE_REQUIREMENT && node.type !== TYPE_TEMPLATE) - .filter((node) => matchesScope(node.resolutionKey(), compiled)); - if (selected.length === 0) { - throw new Error( - `sharedModelFile("${opts.name}"): selected no nodes (include/exclude, always ` + - `excluding requirement.* and template.*) — nothing to publish. Check \`include\`.`, - ); - } - - // Step 3 — closure. - const selectedKeys = new Set(selected.map((n) => n.resolutionKey())); - const closureFailures = checkClosure(standaloneRoot, selected, selectedKeys); - if (closureFailures.length > 0) { - const lines = closureFailures.map((p) => ` ${p.referrer} → ${p.target}`).join("\n"); - throw new Error( - `sharedModelFile("${opts.name}"): the selection is not closed — these references ` + - `point outside it; include it or exclude the referrer:\n${lines}`, - ); - } - - // Step 4 — serialize (already sorted by resolution key; the callee also - // throws if a selected node has no package, per its own contract). - const artifactContent = serializeSharedDocument(selected); - - // Step 5 — re-load standalone with CORE providers only (registry omitted ⇒ - // MetaDataLoader's own default registry), strict. A consumer loads this - // artifact with ITS OWN providers, never the publisher's — so an export - // needing publisher-only vocabulary must fail HERE, not at the consumer. - const artifactPath = `${opts.name}${ARTIFACT_SUFFIX}`; - const coreLoader = new MetaDataLoader({ strict: true }); - const coreLoaded = await coreLoader.load([ - new InMemoryStringSource(artifactContent, { id: artifactPath }), - ]); - if (coreLoaded.errors.length > 0) { - const first = coreLoaded.errors[0]!; - throw new Error( - `sharedModelFile("${opts.name}"): the export needs a provider this toolchain does ` + - `not ship; Phase 1 exports must load with core vocabulary: ${first.message}`, - ); - } - - // Step 6 — emit the artifact + manifest. - const integrity = `${INTEGRITY_PREFIX}${createHash("sha256").update(artifactContent, "utf-8").digest("hex")}`; - const packages = [...new Set(selected.map((n) => packageOfResolutionKey(n.resolutionKey())))].sort(); - const nodes = selected.map((n) => n.resolutionKey()).sort(); - const version = opts.version ?? nearestPackageVersion(ctx.projectRoot ?? process.cwd()); + // Resolved after the selection has passed every check, so a model that cannot be + // published says so before a missing version does. + const version = opts.version ?? nearestPackageVersion(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd()); if (version === undefined) { throw new Error( `sharedModelFile("${opts.name}"): no version available — pass \`version\` explicitly, ` + `or run inside a project with a package.json version at or above the project root.`, ); } - const manifest = { - schema_version: MANIFEST_SCHEMA_VERSION, - name: opts.name, - version, - metamodelVersion: METAMODEL_VERSION, - artifact: artifactPath, - integrity, - packages, - nodes, - }; - return [ - { path: artifactPath, content: artifactContent }, - { path: MANIFEST_FILE, content: JSON.stringify(manifest, null, 2) + "\n" }, - ]; + + const { artifact, manifest } = renderSharedModelFiles(model, version); + return [artifact, manifest]; }), }; if (opts.target !== undefined) generator.target = opts.target; diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index fc229e275..dafb6ee73 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -6,7 +6,7 @@ export { runGen } from "./runner.js"; export type { RunGenOpts, RunGenResult } from "./runner.js"; -export type { Generator, GenContext, EmittedFile, GeneratorFactory } from "./generator.js"; +export type { Generator, GenContext, EmittedFile, GeneratorFactory, OwnedDocsSurface } from "./generator.js"; export { perEntity, perPackage, perModel, oncePerRun } from "./generator.js"; // The one aggregation of the UI-tier markers. Exported because `meta docs` is the // SECOND door onto `agent/ui.md` and must answer "does this run emit a UI tier?" @@ -60,7 +60,7 @@ export { export { isPackagedDocsGenerator, packagedDocsFactoryCall, DOCS_DOOR_COMMAND } from "./packaged-docs-generators.js"; export type { MetaobjectsGenConfig, NormalizedMetaobjectsGenConfig, ResolvedGenConfig, Dialect, ExtStyle, ColumnNamingStrategy, MetaDataTypeProvider, GeneratorSpec, DocsConfig, ResolvedDocsConfig, DocsSurface, ApiSurface, VerifyConfig } from "./metaobjects-config.js"; -export { defineConfig, normalizeConfig, resolveGenerators, isNeutralGeneratorName, resolveDocsConfig } from "./metaobjects-config.js"; +export { defineConfig, normalizeConfig, resolveGenerators, isNeutralGeneratorName, removedGeneratorMessage, resolveDocsConfig } from "./metaobjects-config.js"; // The dialect every generator falls back to when a config declares none. Exported because // `meta docs` must apply the SAME default `normalizeConfig` does — a command that resolved // the dialect differently would document a schema the toolchain does not produce. @@ -132,6 +132,22 @@ export { } from "./docs-paths.js"; export type { DocPageNode, DocPagePlacement } from "./docs-paths.js"; +// The docs tier (ADR-0034 Amendment 7). `docsFile()` and `apiDocsFile()` are served from +// `/generators`; what they compose is public here so an owned (ejected) copy imports only +// this package. The owned part is which nodes get a page, +// where it lands and how the index is laid out; the page payloads (`buildEntityDocData`, +// `buildTemplateDocData`, `buildApiModelForRun`), the renderers and the ER diagram stay in +// the package. `docsDoorGenerators` is how `meta docs` reads the owned copies out of +// `docs: { generators: [...] }`. +export { apiSurfaceHref, surfaceCrossHref } from "./docs-paths.js"; +export { docsDoorGenerators } from "./docs-door.js"; +export type { DocsDoorFacts } from "./docs-door.js"; +export { renderDocPage } from "./render-engine/doc-page.js"; +export { renderMermaidErBlock } from "./templates/mermaid-er.js"; +export { buildApiModel, buildApiModelForRun } from "./generators/api-model.js"; +export type { ApiModel, ApiModelContext, ApiUnitDoc } from "./generators/api-model.js"; +export { renderAgentApi, renderApiIndex, renderEntityApiPage } from "./generators/api-doc-render.js"; + export { isProjection, isWriteThrough } from "./projection/projection-detector.js"; export { isAbstract, emitsInstanceArtifacts, emitsWriteArtifacts } from "./instance-artifacts.js"; // The UI tier asks THESE — "is there an endpoint?" — never the storage predicates. @@ -409,6 +425,14 @@ export type { // Public so an OWNED generator applies it the same way the built-ins do. export { withClientDirective, CLIENT_DIRECTIVE } from "./client-directive.js"; +// FR-023 §4.3 — the shared-model contract. `sharedModelFile()` (served from `/generators`) is +// the reference generator; the two stages it wires are public so an owned (ejected) copy +// imports only this package and keeps only the parts it owns: which source files may +// contribute and which version is published. Everything a consumer's `meta deps sync` +// verifies (the selection, the closure, the bytes, the hash, the manifest) stays here. +export { buildSharedModel, renderSharedModelFiles } from "./shared-model/build-shared-model.js"; +export type { SharedModel, SharedModelFiles, SharedModelInput } from "./shared-model/build-shared-model.js"; + // FR-044 Plan 4 — the Cube exporter. `cubeModel()` is the reference generator; the two pure // stages it wires are public so an owned (ejected) copy imports only this package and keeps // only the parts it owns: which entities get a cube, the file layout and the YAML text. diff --git a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts index 41b75d687..17f3e5574 100644 --- a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts +++ b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts @@ -283,6 +283,20 @@ export interface DocsConfig { baseUrl?: string; surfaces?: DocsSurface[]; apiSurfaces?: ApiSurface[]; + /** + * Docs generators you OWN (`meta eject docs`, `meta eject api-docs`), for `meta docs` to + * run in place of its packaged ones. Each carries a `docsSurface` marker saying which + * surface it takes over, and one per surface is allowed. Leave it out and `meta docs` + * runs the packaged generators, which need no wiring. + * + * It is here, in the docs door's own block, and not in the top-level `generators`, + * because that array is what `meta gen` runs and `meta gen` does not run documentation + * (ADR-0021 D1, ADR-0025; ADR-0034 Amendment 7). + * + * import { docsFile } from "./codegen/generators/docs.js"; + * docs: { generators: [docsFile()] } + */ + generators?: Generator[]; } export interface ResolvedDocsConfig { @@ -379,20 +393,53 @@ export function isNeutralGeneratorName(name: string): boolean { return entry !== undefined && entry.tier !== "native"; } +/** + * Stable generator names that used to resolve and no longer do, each with what replaced + * it. A name that worked in a config yesterday deserves better than "unknown generator": + * that answer sends its reader looking for a typo. + * + * `mermaid-er` (removed in 1.1) wrote the ER diagram to a file of its own under + * `meta gen`. It was the older home of a diagram `meta docs` draws: the whole model on + * the overview page, and each entity's neighbourhood on that entity's page. + * + * Every entry so far is a doc generator `meta docs` replaced, and each message says so. + * `meta verify` relies on that to report a removed name under the docs door, so a test + * holds it: a removed generator of another kind needs its own advisory first. + */ +export const REMOVED_GENERATOR_NAMES: Readonly> = { + "mermaid-er": + 'generator "mermaid-er" was removed in 1.1: `meta docs` draws the ER diagram, for the ' + + "whole model on its overview page (README.md) and around each entity on that entity's " + + 'page. Remove "mermaid-er" from `generators` and run `meta docs`.', +}; + +/** + * What {@link resolveGenerators} says about a stable name it refuses as REMOVED, or + * undefined for any other name. A config that lists the STRING makes `meta gen` fail + * with this message. Public so a caller can ask instead of matching the sentence. + */ +export function removedGeneratorMessage(name: string): string | undefined { + // An own-property test: a name such as "toString" must read as unknown, not as removed. + return Object.hasOwn(REMOVED_GENERATOR_NAMES, name) ? REMOVED_GENERATOR_NAMES[name] : undefined; +} + /** * Materialize the config `generators` array: pass typed generators through * untouched and resolve each stable-name string via the {@link generatorRegistry} * to its factory result (default options). ADR-0021 #1. * * Errors: - * - a NEUTRAL name (`docs`, `mermaid-er`) is owned by `meta docs` (ADR-0021 D1) - * and is not selectable in the gen suite. + * - a REMOVED name ({@link removedGeneratorMessage}) throws saying what replaced it. + * - a NEUTRAL name (`docs`) is owned by `meta docs` (ADR-0021 D1) and is not + * selectable in the gen suite. * - an UNKNOWN name throws listing the available NATIVE names. `api-docs` is left out * of that list: it resolves, and the runner warns about and skips it (ADR-0025). */ export function resolveGenerators(specs: readonly GeneratorSpec[]): Generator[] { return specs.map((spec) => { if (typeof spec !== "string") return spec; + const removed = removedGeneratorMessage(spec); + if (removed !== undefined) throw new Error(removed); const entry = generatorRegistry[spec]; if (entry === undefined) { // A native generator the runner skips is not offered: the list named `api-docs`, diff --git a/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts b/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts index 5e16aff21..0a9c7539b 100644 --- a/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts +++ b/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts @@ -10,9 +10,10 @@ // Keyed by the generator's IMPLEMENTATION name, because that is what a constructed // generator carries. The value is the factory call an adopter wrote, for the message. // -// `mermaid-er-diagram` is not one of them. `mermaidErDiagram()` is deprecated and kept -// running for a config that already lists it, so the runner does not skip it. -// `meta docs` draws the same diagram on its overview page through `docsFile()`. +// An OWNED copy of one of them (docs-door.ts) is skipped too, by its `docsSurface` marker +// and not by this map: an owner may rename the copy, and the fix for it is a different +// one. A packaged generator is removed from `generators`; an owned copy is moved to +// `docs: { generators: [...] }`, where `meta docs` runs it. const PACKAGED_DOCS_FACTORY_CALLS: ReadonlyMap = new Map([ ["docs-file", "docsFile()"], ["api-docs", "apiDocsFile()"], diff --git a/server/typescript/packages/codegen-ts/src/reference-templates.ts b/server/typescript/packages/codegen-ts/src/reference-templates.ts index d4c8d4611..6605405bd 100644 --- a/server/typescript/packages/codegen-ts/src/reference-templates.ts +++ b/server/typescript/packages/codegen-ts/src/reference-templates.ts @@ -29,6 +29,16 @@ export const REFERENCE_GENERATOR_NAMES = [ // generators over `renderCallableFile` / `renderTraceHelper`, so the owned part is which // objects get a module and where it lands. "callable", "trace-helper", + // The publisher's shared-model artifact: a thin generator over `buildSharedModel` and + // `renderSharedModelFiles`. The owned part is which source files may contribute and + // which version is published; the artifact, its hash and the manifest that a + // consumer's `meta deps sync` verifies stay in the package. + "shared-model", + // The docs tier (ADR-0034 Amendment 7). `docs` and `api-docs` are run by `meta docs`, which + // reads an owned copy out of `docs: { generators: [...] }` by its `docsSurface` marker. + // The owned part is which nodes get a page, where it lands and how the index is laid out; + // the page payloads and renderers stay in the package. + "docs", "api-docs", ] as const; export type ReferenceGeneratorName = (typeof REFERENCE_GENERATOR_NAMES)[number]; diff --git a/server/typescript/packages/codegen-ts/src/reference/api-docs.ts b/server/typescript/packages/codegen-ts/src/reference/api-docs.ts new file mode 100644 index 000000000..d09615143 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/reference/api-docs.ts @@ -0,0 +1,117 @@ +// REFERENCE TEMPLATE — copy this into your repo (e.g. codegen/generators/api-docs.ts) and own it. +// Then import it LOCALLY in metaobjects.config.ts: +// import { apiDocsFile } from "./codegen/generators/api-docs.js"; +// +// RUNTIME: this file executes under whatever runs `meta docs`, and the published CLI's +// shebang is `#!/usr/bin/env node` — so it runs under NODE even in a Bun project. Do not +// reach for `Bun.*` globals here; they are undefined and take the whole run down with +// `Bun is not defined`. Use `node:` builtins instead. +// targets: Markdown. The pages describe the code the stock TypeScript generators +// emit (the entity module, queries, the Fastify routes and, when +// `routesFileHono` is wired, the Hono ones, the prompt and output helpers); +// they import nothing. +// use-when: you want to change the API reference `meta docs` writes: which units get +// a page, where the pages land, or which of the three forms are written. +// `meta docs` writes these pages without this file; eject it only to change +// them. If your generators emit a different API than the stock ones, the +// reference is wrong for you until you change what is written here. +// emits: under `docs.outDir`, in the api surface's directory (`api/` by default): +// README.md, the index; AGENT-API.md, the condensed form the agent context +// points at; .md per unit (an entity, a projection, a served report, +// a template). Under `outputLayout: "package"` the unit pages fold under +// their package path. +// customize: which units get a page (`model.units` in `generate`), where a page lands +// (`docPageOutputPath` under `apiDir`) and which forms are written (the +// index and the agent form are the two `unshift`ed files). Stays in the +// package: `buildApiModelForRun`, which reads the model and the run into +// the description of the generated API, and `renderEntityApiPage` / +// `renderApiIndex` / `renderAgentApi`, which write it. The always-on agent context names +// `api/AGENT-API.md`; if you move or drop that file, the pointer is the +// other half. +// composes-with: docs (each api page links back to its model page when `meta docs` writes +// the model surface too) and the routes generators (the Hono routes are +// documented when one is wired). +// +// HOW IT RUNS. `meta docs` runs it, so it is listed in the docs block of +// metaobjects.config.ts, not among the generators `meta gen` runs: +// docs: { generators: [apiDocsFile()] } +// The `docsSurface` marker below says which surface it takes over: with it listed, +// `meta docs` and `meta verify --docs` run THIS generator for the api surface in place of +// the packaged one. Documentation has one door, so `meta gen` never runs it. +// +// Everything below imports ONLY from `@metaobjectsdev/codegen-ts` (the stable engine). + +import { + assertNoDuplicateDocPaths, + buildApiModelForRun, + docPageOutputPath, + projectProvider, + renderAgentApi, + renderApiIndex, + renderEntityApiPage, + surfaceCrossHref, + type DocPageNode, + type DocPagePlacement, + type EmittedFile, + type Generator, +} from "@metaobjectsdev/codegen-ts"; + +// Where the pages go when nothing says otherwise. `meta docs` always says: it passes the +// directory of the api surface it is writing (`docs.apiSurfaces[].subDir`, default `api`). +const DEFAULT_API_DIR = "docs/api"; + +export function apiDocsFile(): Generator { + return { + name: "api-docs", + // The surface this generator takes over when it is listed in `docs.generators`. + docsSurface: "api", + generate(ctx) { + const provider = projectProvider(ctx.projectRoot ?? process.cwd()); + const layout = ctx.config.outputLayout ?? "flat"; + + // What `meta docs` reports about the run: the directory of this surface, and whether + // the model pages are written too. + const apiDir = ctx.docsDoor?.apiSubDir ?? DEFAULT_API_DIR; + const modelSurface = ctx.docsDoor?.modelSurface; + const indexFilename = `${apiDir}/README.md`; + const agentFilename = `${apiDir}/AGENT-API.md`; + + // One model of the generated API feeds every form. `buildApiModelForRun` reads what it + // needs off the run: the layout, whether a Hono routes generator is wired (its routes + // are documented exactly then), the project's `apiPrefix` and the primary keys. + const model = buildApiModelForRun(ctx); + + // Every (path, node) placed, so a collision is refused after all pages are placed. + const placements: DocPagePlacement[] = []; + + // OWNED: which units get a page, and where it lands. + const files: EmittedFile[] = model.units.map((unit) => { + const node: DocPageNode = { name: unit.node, package: unit.package }; + const path = `${apiDir}/${docPageOutputPath(layout, node)}`; + placements.push({ path, fqn: unit.package ? `${unit.package}::${unit.node}` : unit.node }); + // The model page sits at the docs root at the same placement, so the link back is + // derived from it and resolves in both layouts. None when the model surface is off. + const modelPageHref = modelSurface + ? surfaceCrossHref(path, docPageOutputPath(layout, node)) + : undefined; + return { path, content: renderEntityApiPage(unit, provider, modelPageHref) }; + }); + + // OWNED: the index and the agent form, first, and only when a unit page exists. + if (files.length > 0) { + placements.push({ path: indexFilename, fqn: "" }); + placements.push({ path: agentFilename, fqn: "" }); + files.unshift( + { path: indexFilename, content: renderApiIndex(model, layout, provider) }, + { path: agentFilename, content: renderAgentApi(model, provider) }, + ); + } + + // Two units that resolve to one output path are refused, naming both and the path, + // instead of one page silently overwriting the other. + assertNoDuplicateDocPaths(placements); + + return files; + }, + }; +} diff --git a/server/typescript/packages/codegen-ts/src/reference/docs.ts b/server/typescript/packages/codegen-ts/src/reference/docs.ts new file mode 100644 index 000000000..646f2c68a --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/reference/docs.ts @@ -0,0 +1,244 @@ +// REFERENCE TEMPLATE — copy this into your repo (e.g. codegen/generators/docs.ts) and own it. +// Then import it LOCALLY in metaobjects.config.ts: +// import { docsFile } from "./codegen/generators/docs.js"; +// +// RUNTIME: this file executes under whatever runs `meta docs`, and the published CLI's +// shebang is `#!/usr/bin/env node` — so it runs under NODE even in a Bun project. Do not +// reach for `Bun.*` globals here; they are undefined and take the whole run down with +// `Bun is not defined`. Use `node:` builtins instead. +// targets: Markdown, with Mermaid for the diagrams. The pages import nothing. +// use-when: you want to change the model documentation `meta docs` writes: which +// objects and templates get a page, where the pages land, or how the +// overview page is laid out. `meta docs` writes these pages without this +// file; eject it only to change them. To change the WORDING of a page you +// need no generator at all: drop your own +// `templates/docs/entity-page.md.mustache` or +// `templates/docs/template-page.md.mustache` in the project root. +// emits: under `docs.outDir` (default ./docs/generated): README.md, the overview +// with the whole-model ER diagram and an index; .md per object, +// reports included;