diff --git a/AGENTS.md b/AGENTS.md index 03fd7527d..a12005242 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,7 +6,7 @@ MetaObjects is a **cross-language metadata standard** for declaring typed entity The metamodel is the **durable spine**; generated code is the **disposable artifact**. Substrate is local-first: typed metadata lives in your repo, generated code is idiomatic per-language output with **no proprietary runtime**: the generated entity and model code imports nothing from MetaObjects at runtime, and the REST, prompt, client and runtime tiers import ordinary Apache-2.0 packages you can vendor, fork or replace. If `@metaobjectsdev/*` disappears tomorrow, you keep working code. -**Core vs helpers (ADR-0034 Amendment 3, 2026-09-22).** Only the core is a product promise: the metamodel, loader, canonical format and registry; runtime metadata access (the `ObjectManager` and its drivers; the HTTP adapters that mount it on a web framework are helpers, 2026-09-24 ruling); schema migrations (`meta migrate`); the drift gates (`meta verify`); prompt render and the reply parser. Every generator that writes application code into an adopter's repo — routes, controllers, ORM wiring, DTOs, forms, grids, hooks, filter allowlists — is a **reference helper**: it must compile and pass its reference fixtures, and the adopter copies it with `meta eject` and owns the copy. Mechanical test: what the tool guarantees is core; what it writes into your repo is a helper. Every port can eject: `meta eject` (TypeScript), `metaobjects eject` (Python), `mvn metaobjects:eject` (Java/Kotlin), `dotnet meta eject` (C#). Not every generator ships a reference copy: `meta gen --list` marks the ones that do not `package-only` (in TypeScript: `template`, the docs tier and `shared-model`; ADR-0034 Amendment 3 correction, 2026-09-26, narrowed by Amendment 5). When you find a generator defect, fix the reference; do not describe it as a broken core guarantee. +**Core vs helpers (ADR-0034 Amendment 3, 2026-09-22).** Only the core is a product promise: the metamodel, loader, canonical format and registry; runtime metadata access (the `ObjectManager` and its drivers; the HTTP adapters that mount it on a web framework are helpers, 2026-09-24 ruling); schema migrations (`meta migrate`); the drift gates (`meta verify`); prompt render and the reply parser. Every generator that writes application code into an adopter's repo — routes, controllers, ORM wiring, DTOs, forms, grids, hooks, filter allowlists — is a **reference helper**: it must compile and pass its reference fixtures, and the adopter copies it with `meta eject` and owns the copy. Mechanical test: what the tool guarantees is core; what it writes into your repo is a helper. Every port can eject: `meta eject` (TypeScript), `metaobjects eject` (Python), `mvn metaobjects:eject` (Java/Kotlin), `dotnet meta eject` (C#). Not every generator ships a reference copy: `meta gen --list` marks the ones that do not `package-only` (in TypeScript: `template` alone; ADR-0034 Amendment 3 correction, 2026-09-26, narrowed by Amendments 5 and 7). When you find a generator defect, fix the reference; do not describe it as a broken core guarantee. ## Six pillars diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a3d1884b..adb99898f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -178,10 +178,10 @@ generator catalog and the Atlas evaluation are 1.2 work. own: which objects get a module and where it lands. The module body stays in the package (`renderCallableFile`, `renderTraceHelper`, now exported from `@metaobjectsdev/codegen-ts`), the same split the prompt tier uses. The rule is that every stock generator can be - ejected except the `template` primitive, which has no emit logic of its own to own. Not - yet ejectable in this change: `shared-model` and the docs tier (`docs`, `api-docs`, - `mermaid-er`) in TypeScript. The Java `entity` and `extractor` generators became - ejectable in ADR-0034 Amendment 6. + ejected except the `template` primitive, which has no emit logic of its own to own. + `shared-model`, `docs` and `api-docs` follow in their own entry below (ADR-0034 + Amendment 7), and the Java `entity` and `extractor` generators became ejectable in + ADR-0034 Amendment 6. - **Agent skills: a cost test for owned generators.** The `metaobjects-codegen` skill now says when not to write a generator: only when generator code plus its generated output and added metadata is fewer lines than the hand-written code it @@ -521,6 +521,42 @@ generator catalog and the Atlas evaluation are 1.2 work. docs` wording come from the packages, and each port's existing generators already type a required field as not nullable), so an owned generator needs no resync for this. See [docs/features/reporting.md](docs/features/reporting.md#rows-from-a-dimensions-entity). +- **`shared-model`, `docs` and `api-docs` can be ejected (TypeScript).** With these + three, every stock TypeScript generator can be ejected except `template`, the Mustache + primitive: you already supply its template and it has no emit logic of its own to own, + so it stays `package-only` (the C#, Java and Python ports rule the same way). The + standalone `mermaid-er` generator is not among them: it is removed (see **Removed** + below). Ruling and reasoning: ADR-0034 Amendment 7. Nothing changes for a project that + ejects none of them: packaged output is byte-identical, and gated to be. + - **`meta eject shared-model`** copies a thin generator. The copy owns which of your + source files may contribute and where the published version comes from. Everything a + consumer's `meta deps sync` verifies stays in the package, in two functions the copy + calls and `@metaobjectsdev/codegen-ts` now exports: `buildSharedModel` (load, select, + closure check, serialize, core-vocabulary re-load) and `renderSharedModelFiles` (the + file names, the hash, the manifest). A test publishes with an ejected copy and has a + consumer sync, lock and verify the result. + - **`meta eject docs` and `meta eject api-docs`** copy the generators behind + `meta docs`. List the copy in the docs block of `metaobjects.config.ts`, a new key: + `docs: { generators: [docsFile()] }`. It carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged generator for that + surface. `meta eject` prints exactly that wiring, and its JSON row carries + `wire.into: "docs.generators"`. The copy owns which nodes get a page, + where a page lands, the overview page and which forms of the API reference are written. + A docs generator does not go in the top-level `generators`: `meta gen` skips one there + with a warning that names `docs.generators`. Two entries for one surface, or an entry + with no `docsSurface`, are refused by name. The `requirements` and `agent` pages stay + packaged. + - To change only the wording of a model page you still need no generator: a + `templates/docs/entity-page.md.mustache` in your project wins over the packaged one. + - Steps and what each copy owns: + [own-your-codegen](docs/features/own-your-codegen.md#the-docs-generators-and-shared-model-typescript). +- **`ctx.inputRoot` for a generator that reads the project's own files.** It is the real + project directory. Under `meta gen` it equals `ctx.projectRoot`. Under + `meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is + that tree, as before, because owned `entity` and `routes` copies compute an import path + against it. Under `meta docs` and `meta verify --docs` it is the directory that holds + `metaobjects.config.ts`, and `--templates` does not move it. Read files through + `ctx.inputRoot`; `runGen` takes it as `inputRoot`. ### Changed @@ -651,6 +687,16 @@ until you regenerate. - **The config loader no longer rejects an import of the four generators.** 1.0.x answered `import { entityFile } from "@metaobjectsdev/codegen-ts/generators"` with a message naming `meta eject`. That import now loads. +- **`meta docs` stops on a `metaobjects.config.ts` that fails to load.** It prints the load + error, exits 2 and writes nothing, as `meta gen` does. `meta verify --docs` calls + `meta docs`, so it fails on the same config. Before, `meta docs` warned, wrote the model + pages without the config and exited 0. With an owned docs generator listed in + `docs: { generators: [...] }` that would have overwritten your pages with the packaged + generator's output, so the fallback is removed. A project with no `metaobjects.config.ts` + is unaffected: `meta docs` still writes the model pages from metadata alone. + **To upgrade**, if a docs build has been passing with a config-load warning, fix the + config. A config that still imports the removed `mermaidErDiagram` is one such case (see + **Removed** below). - **The agent-context skills teach "import by default, eject to customize".** The codegen skill and the always-on context no longer send an agent to eject a generator it is not going to change. The audit skill reports an owned copy it can prove unmodified as a @@ -698,6 +744,27 @@ until you regenerate. One rule is relaxed: an authored `validator.length @min: 0` on a `@required` string now admits the empty string, as the generated schema already did. +### Removed + +- **The standalone `mermaid-er` generator (TypeScript).** `mermaidErDiagram()`, its + `MermaidErOptions` type and the stable name `"mermaid-er"` are gone from + `@metaobjectsdev/codegen-ts/generators`, the generator catalog and the cross-port + generator manifest. It wrote the whole-model ER diagram to a file of its own + (`docs/model.md`) under `meta gen`, and was the older home of a diagram `meta docs` has + drawn since the docs engine took it over: the whole model on the overview page + (`README.md`), and each entity's neighbourhood on that entity's page. Those diagrams are + unchanged, byte for byte. No other port shipped the generator. Ruling: ADR-0034 + Amendment 7; ADR-0022 Part 2 D1 is amended to match. + - **To upgrade**, if your `metaobjects.config.ts` wires it: delete the + `mermaidErDiagram` import and its entry in `generators` (or the `"mermaid-er"` name), + delete the file it wrote (`docs/model.md` under your output directory, unless you moved + it with `outFile`), and run `meta docs`. `meta verify --codegen` reports that file as + no longer generated until it is deleted. + - **A config that still wires it fails with a message that says this**, instead of the + runtime's own words for a missing export. The import is diagnosed in the config and in + a module the config imports, and the name is diagnosed as removed, not as unknown. + - A project that never wired it sees no change. + ### Fixed - **A doc generator skipped by `meta gen` is now reported in every output form.** `meta docs` @@ -720,23 +787,33 @@ until you regenerate. docs` `` with the reason under the row, a wired one reads ``WIRED but skipped by `meta gen` ``, and `--probe` no longer prints `would emit 0` for them. In `--format json` the row carries `note` and `door: "meta docs"`, and `project.wouldEmit` is `null` instead of `0`. - The `mermaid-er` row's note now states the back-compat fact: it is deprecated, a - `mermaidErDiagram()` already wired in `generators` still runs, and `meta docs` draws the - same diagram on its overview page. A neutral name wired as a string (`"docs"`, - `"mermaid-er"`), which `meta gen` refuses, reads ``WIRED by name, which `meta gen` - refuses`` (`project.refused` in `--format json`). + There is no `mermaid-er` row any more: the generator is removed in this release (see + **Removed** above). The neutral name `"docs"` wired as a string, which `meta gen` + refuses, reads ``WIRED by name, which `meta gen` refuses`` (`project.refused` in + `--format json`). - **`meta verify`** said nothing about a config wired this way. It now reports each one as an advisory row (rule `doc-generator-in-gen`, in `antiPatterns.rows[]`) that names `meta verify --docs` as the gate that checks documentation. A neutral name wired as a string is reported under rule `doc-generator-refused-by-gen`, because `meta gen` fails on - it rather than skipping it. Advisory: the exit code is unchanged. + it rather than skipping it. The removed name `"mermaid-er"` is reported under the same + rule, with the message `meta gen` fails with. Advisory: the exit code is unchanged. + - **An owned docs generator under `generators`** (a copy made with `meta eject docs` or + `meta eject api-docs`) is skipped by the same rule and reported in the same places. Its + fix is the opposite one, so each message says to move it to + `docs: { generators: [...] }`, where `meta docs` runs it, and not to remove it: the + runner's warning, the `help[]` next step and the `doc-generator-in-gen` advisory row. A + config that wires only such a copy is no longer told `no entities to generate`. Listed + in the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` + (`project.wiredInDocsDoor` in `--format json`). - **The unknown-generator error** listed `api-docs` among the names `meta gen` accepts. It no longer does, and it says documentation comes from `meta docs`. - `runGen()` returns a new `skippedDocGenerators: string[]` beside `warnings`, and - `@metaobjectsdev/codegen-ts` exports `isPackagedDocsGenerator` (skipped) and - `isNeutralGeneratorName` (refused), so a caller embedding the runner can ask rather than - match a sentence. TypeScript only: in the C#, Java, Kotlin and Python + `runGen()` returns a new `skippedDocGenerators: string[]` beside `warnings`, with + `skippedOwnedDocGenerators` naming the owned copies among them, and + `@metaobjectsdev/codegen-ts` exports `isPackagedDocsGenerator` (skipped), + `isNeutralGeneratorName` (refused) and `removedGeneratorMessage` (refused as removed), so + a caller embedding the runner can ask rather than match a sentence. TypeScript only: in + the C#, Java, Kotlin and Python ports documentation is a separate command (`dotnet meta docs`, `mvn metaobjects:docs`, `metaobjects docs`) and no doc generator is registered for code generation, so there is nothing to wire and nothing to skip. @@ -1183,6 +1260,12 @@ until you regenerate. `enforce` is on the `MetaReferenceIdentity` that `referenceIdentities()` returns, and that a bare entity name in `@of`, `@via` and `@spine` resolves in the declaring entity's package (the report's, for `@spine`), so another package's entity needs its package name. +- **`meta verify --codegen` can check a project that wires `sharedModelFile()`.** The gate + regenerates into a temporary tree and passed the generator neither the project's own + source files nor a way to read the project, so it failed with "no source files to load" + for a packaged or an owned `shared-model`, and would have taken the version default from + the temporary tree. It now passes what `meta gen` passes. A publisher's drift gate on its + shared artifact, which the dependency docs describe, works as described. ## [1.0.13] — 2026-10-03 diff --git a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/agent-context/skills/metaobjects-codegen/references/owning-generators.md b/agent-context/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/agent-context/skills/metaobjects-codegen/references/owning-generators.md +++ b/agent-context/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/agent-context/skills/metaobjects-codegen/references/typescript-docs.md b/agent-context/skills/metaobjects-codegen/references/typescript-docs.md index 41dff507a..01e2b70ec 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript-docs.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript-docs.md @@ -33,7 +33,8 @@ Other flags: `--layout flat|package`, `--base-url `. Configure defaults in `docs:` block in `metaobjects.config.ts` (`outDir`, `layout`, `baseUrl`, `surfaces`); CLI flags override it. The api surface needs the gen config (it documents what the codegen produced); with no config it is skipped with a note, -and the model surface still emits from metadata alone. +and the model surface still emits from metadata alone. A config that exists but fails to +load stops `meta docs` (the load error, exit 2, nothing written), as it stops `meta gen`. **Never wire `docsFile()` or `apiDocsFile()` into `generators`.** They are the engines behind `meta docs`, so `meta gen` warns about each one it finds in the suite and skips it: @@ -44,11 +45,20 @@ no docs are written by that run. The warning is on stderr, and in a structured r `meta verify` reports the wiring as an advisory row, rule `doc-generator-in-gen`. Remove the entry and run `meta docs`. -**Do not add `mermaidErDiagram()` to a new config.** It is deprecated: `meta docs` already -draws the ER diagram on its overview page with nothing wired. A `mermaidErDiagram()` -already wired in `generators` still runs under `meta gen`. The stable names `"docs"` and -`"mermaid-er"` written as strings in `generators` are refused, so `meta gen` fails on -them; `meta verify` reports that as rule `doc-generator-refused-by-gen`. +**An owned docs generator goes in `docs: { generators: [...] }`, never in `generators`.** +`meta eject docs` (or `api-docs`) copies the generator and prints that wiring; `meta docs` +and `meta verify --docs` then run the copy in place of the packaged one. Under +`generators` it is skipped like the packaged one, and the warning, the `help[]` next step +and the `doc-generator-in-gen` advisory all say to move it, not to remove it. Listed in +the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). + +**There is no `mermaidErDiagram()` generator.** 1.1 removed it: `meta docs` draws the ER +diagram on its overview page and on each entity's page with nothing wired. A config that +still imports it, or names `"mermaid-er"`, fails with a message that says so; delete the +entry and run `meta docs`. The stable name `"docs"` written as a string in `generators` +is refused as well, so `meta gen` fails on it. `meta verify` reports both names as rule +`doc-generator-refused-by-gen`. **Before calling any generated code, read `./docs/api/AGENT-API.md`** — it has the exact imports, signatures, payload field shapes, and runnable examples for this diff --git a/agent-context/skills/metaobjects-codegen/references/typescript.md b/agent-context/skills/metaobjects-codegen/references/typescript.md index c5903cffc..e57030499 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript.md @@ -126,10 +126,29 @@ name. Import by default and eject to customize: an owned copy you never change o behind the reference. (`entityFile` / `queriesFile` / `routesFile` / `barrel` were removed from that subpath at 1.0 and restored in 1.1, so on a 1.0.x install an owned copy is the only path for those four.) The engine primitives come from the package main entry, -`@metaobjectsdev/codegen-ts`. Every stock generator can be ejected except the `template` -primitive, which has no emit logic of its own to own. `shared-model` and the docs tier -(`docs`, `api-docs`, `mermaid-er`) are not yet ejectable; `meta gen --list` marks each -entry without a reference copy `package-only`. +`@metaobjectsdev/codegen-ts`. One entry cannot be ejected and is `package-only` in +`meta gen --list`: the `template` primitive, which has no emit logic of its own. + +Three ejectable entries behave differently from the rest: + +- **`docs` and `api-docs` are run by `meta docs`, not `meta gen`.** `meta docs` writes + both with nothing wired; never put a docs generator, packaged or owned, in `generators`. + To change the pages, `meta eject docs` (or `api-docs`) and list the copy in the docs + block, `docs: { generators: [docsFile()] }`: it carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged one. To change only + a page's wording, put your own `templates/docs/entity-page.md.mustache` in the project + instead. +- **The ER diagram comes from `meta docs`** (overview page, and each entity's page). The + standalone `mermaidErDiagram()` / `"mermaid-er"` generator was removed in 1.1; a config + that still wires it fails with a message saying so. Remove it and run `meta docs`. +- **`shared-model` ejects as a thin generator.** The copy owns which source files may + contribute and where the published version comes from. The artifact, its hash and the + manifest a consumer's `meta deps sync` verifies stay in the package + (`buildSharedModel`, `renderSharedModelFiles`); keep calling both. + +In an owned generator, read the project's own files through `ctx.inputRoot`. Under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. Under `meta docs` it is the directory that holds `metaobjects.config.ts`. **`requirementTests()` — one test stub per requirement the model claims.** A recommended approach, not a contract: a `live` or `partial` requirement gets a stub that fails until diff --git a/docs/features/cli.md b/docs/features/cli.md index 6be6f6059..4c940c412 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -280,6 +280,19 @@ none emits every surface the project has something to say with. | requirements | `--requirements` | The declared ledger, as `requirements.md` + `requirements.toon`. | no | | agent | `--agent` | Three pages an agent reads BEFORE touching a tier (below). | yes | +The model and api surfaces each run one generator, and either can be replaced by a copy you +own: `meta eject docs` or `meta eject api-docs`, then list the copy in the `docs` block of +`metaobjects.config.ts`, `docs: { generators: [docsFile()] }`. `meta docs` runs it in place +of the packaged generator for the surface its `docsSurface` marker names, and +`verify --docs` checks the committed pages against it. The requirements and agent surfaces +are not generators and cannot be replaced. See +[own-your-codegen](own-your-codegen.md#docs-and-api-docs-are-run-by-meta-docs). + +"Needs a gen config? no" means the surface is written when the project has no +`metaobjects.config.ts`. A config that exists but fails to load stops `meta docs`, as it +stops `meta gen`: the load error, exit 2, nothing written. Carrying on without it would run +the packaged generator over the pages an owned copy writes. + Two flags are NOT members of that set and do not compose with it: - **`--metamodel`** is a different MODE over a different subject: it documents the BUILT-IN @@ -578,10 +591,16 @@ name: a doc generator (`docsFile()`, `apiDocsFile()`) listed under `generators` (ADR-0025). The result then says so in `warnings[]` and adds a `meta docs` next step to `help[]`; `meta gen --list` marks those rows ``run by `meta docs` `` (`door` in `--format json`), and `meta verify` reports the same wiring as an advisory row with rule -`doc-generator-in-gen`. A neutral stable name written as a string (`"docs"`, -`"mermaid-er"`) is refused rather than skipped, so `meta gen` fails on it: `meta gen --list` +`doc-generator-in-gen`. An owned (ejected) docs generator listed there is skipped the +same way, and every one of those messages tells you to move it to +`docs: { generators: [...] }`, where `meta docs` runs it, instead of removing it; once it +is there, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). The neutral stable name `"docs"` written +as a string is refused rather than skipped, so `meta gen` fails on it: `meta gen --list` marks that row ``WIRED by name, which `meta gen` refuses`` (`project.refused` in -`--format json`), and the advisory row's rule is `doc-generator-refused-by-gen`. +`--format json`), and the advisory row's rule is `doc-generator-refused-by-gen`. The +name `"mermaid-er"` is refused too, as removed: `meta docs` draws the ER diagram, and +the message and the same advisory rule say so. In a structured run every narration line moves to **stderr**, so stdout is one parseable document. `--format` is honored by `gen`, `verify` and `migrate`; any diff --git a/docs/features/metadata-dependencies.md b/docs/features/metadata-dependencies.md index 9a81a171d..67f838da9 100644 --- a/docs/features/metadata-dependencies.md +++ b/docs/features/metadata-dependencies.md @@ -374,13 +374,16 @@ Because it is codegen output, it is **drift-gated**: `meta verify --codegen` in publisher's own build regenerates and diffs it, so a metadata edit that changes the public surface cannot ship unnoticed. -**`shared-model` is registered and discoverable, but deliberately not ejectable.** -It shows up in `meta gen --list` like any other generator, but `meta eject --list` -does not offer it (unlike `entity`/`queries`/`routes`/`barrel`, ADR-0034's four -scaffold-and-own generators). The artifact it emits is a contract whose bytes a -cross-port corpus pins and whose hash consumers verify — a user-owned, editable copy -of the generator would invite an artifact that silently stops matching what -consumers expect. +**`shared-model` can be ejected, and the artifact stays in the package.** +`meta eject shared-model` copies a thin generator you own: which of your source files may +contribute, and where the published version comes from. The artifact is a contract whose +bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer +checks is not in the copy. It stays in two functions the copy calls, `buildSharedModel` +(load, select, closure check, serialize, core-vocabulary re-load) and +`renderSharedModelFiles` (the file names, the hash, the manifest). An unmodified copy +publishes the same bytes as the packaged generator, and a test has a consumer sync and +verify what an ejected copy published. See +[own-your-codegen](own-your-codegen.md#shared-model-you-own-the-inputs-the-package-owns-the-artifact). **Only TypeScript publishes in Phase 1a — and only TypeScript and Python consume.** TypeScript and Python are the only two ports that can *consume* a dependency at all @@ -466,8 +469,8 @@ but **not** built in Phase 1a — do not assume any of the following exists: and plain-directory vendoring - [`abstracts-and-inheritance.md`](abstracts-and-inheritance.md) — `extends:` and `overlay:` within one project; the cross-repository case builds on the same rules -- [`own-your-codegen.md`](own-your-codegen.md) — generator ownership, and why - `shared-model` is registered but not ejectable +- [`own-your-codegen.md`](own-your-codegen.md) — generator ownership, and what an + ejected `shared-model` copy owns - [`cli.md`](cli.md) — the `meta deps` command surface and `verify --deps` - [`CONFORMANCE.md`](../CONFORMANCE.md) — per-port pass status for `dependency-conformance` diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 09d835167..845e7b5fa 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -157,6 +157,120 @@ What the diff in step 4 holds: Nothing requires this. An owned copy keeps working, and `meta eject --list` keeps telling you how far it is from the reference. +## The docs generators and `shared-model` (TypeScript) + +Every stock TypeScript generator can be ejected except one. `template` is the Mustache +primitive: you already supply its template, its walk and its name, and it has no emit +logic of its own, so a copy would own nothing. `meta gen --list` marks it `package-only`. +The C#, Java and Python ports rule the same way for theirs (ADR-0034 Amendment 7). + +Three of the ejectable ones differ from the rest in what runs them or in what the copy may +change. + +### `docs` and `api-docs` are run by `meta docs` + +`meta docs` writes the model pages and the API reference with nothing wired. Eject one of +the two generators behind it when you want to change which nodes get a page, where a page +lands, how the overview page is laid out, or which forms of the API reference are written: + +```bash +meta eject docs api-docs +``` + +```ts +import { docsFile } from "./codegen/generators/docs.js"; +import { apiDocsFile } from "./codegen/generators/api-docs.js"; + +export default defineConfig({ + // … + docs: { generators: [docsFile(), apiDocsFile()] }, +}); +``` + +The copies are listed in the `docs` block, which is `meta docs`'s own config, and not in +the top-level `generators`, which is what `meta gen` runs: + +- **`meta docs` runs a listed copy in place of the packaged generator.** Each copy carries + a `docsSurface` marker (`"model"` for `docs`, `"api"` for `api-docs`) saying which surface + it takes over. The summary line of `meta docs` names the generator, so you can see that + your copy ran. +- **`meta verify --docs` checks against it.** That gate calls `meta docs`, so an edit to + the copy shows up as drift in the committed pages until you run `meta docs` again. +- **One generator per surface, and each must say which.** Two entries for one surface, or + an entry with no `docsSurface`, are refused by name. +- **A config that fails to load stops `meta docs`.** It prints the load error, exits 2 and + writes nothing, so an error in your copy never ends with the packaged generator's pages + written over yours. +- **`meta gen` never runs one.** If a copy ends up in `generators`, `meta gen` skips it + with a warning that names `docs.generators`, and `meta docs` keeps running the packaged + generator until you move it. The run's `help[]` next step and the `meta verify` advisory + (rule `doc-generator-in-gen`) say the same: move it, do not remove it. +- **`meta gen --list` shows it.** A copy listed in the docs block marks its row + ``WIRED in `docs.generators` `` (`project.wiredInDocsDoor` in `--format json`). + +You can eject one and not the other. The two page sets link to each other, so `meta docs` +tells both generators the same facts on `ctx.docsDoor`: the api surfaces from your `docs:` +config, the directory of the api surface being written, and whether the model pages are +written. A copy reads them there and takes no options for them. + +| | The copy owns | Stays in the package | +|---|---|---| +| `docs` | which objects and templates get a page, where a page lands, which page template renders, the whole overview page (`renderIndexPage`) | the page payloads (`buildEntityDocData`, `buildTemplateDocData`), the ER diagram (`renderMermaidErBlock`), the page templates | +| `api-docs` | which units get a page, where a page lands, whether the index and the agent form (`AGENT-API.md`) are written | the model of the generated API (`buildApiModelForRun`, which also reads the run: layout, `apiPrefix`, whether Hono routes are wired) and its renderers (`renderEntityApiPage`, `renderApiIndex`, `renderAgentApi`) | + +Two things need no eject. To change the wording of a model page, put a file of the same +name in your project (`templates/docs/entity-page.md.mustache` or +`template-page.md.mustache`); it wins over the packaged one. And the packaged `docsFile()` +and `apiDocsFile()` are never listed anywhere: `meta docs` runs them when the `docs` block +names no generator for their surface. + +The `requirements` and `agent` pages of `meta docs` are not generators in the catalog and +cannot be ejected. + +### The ER diagram comes from `meta docs` + +`meta docs` draws the Mermaid diagrams: the whole model on its overview page (`README.md`) +and each entity's neighbourhood on that entity's page. An owned `docs` copy decides where +the whole-model diagram goes, by where it calls `renderMermaidErBlock`. + +Before 1.1 a standalone generator, `mermaidErDiagram()` (stable name `mermaid-er`), also +wrote the whole-model diagram to `docs/model.md` under `meta gen`. It was the older home of +the same diagram and has been removed. A config that still wires it fails with a message +saying so. To upgrade, delete the import and the `mermaidErDiagram()` entry (or the +`"mermaid-er"` name) from `generators`, delete the `docs/model.md` it wrote, and run +`meta docs`. + +### `shared-model`: you own the inputs, the package owns the artifact + +`sharedModelFile()` publishes part of your model as an artifact and a manifest that another +project's `meta deps sync` copies and verifies +([metadata dependencies](metadata-dependencies.md)). Its output is a contract with that +other project, so the copy is split the way `cube-model`'s is: + +| The copy owns | Stays in the package | +|---|---| +| which of your source files may contribute (`files`, default the run's own) | `buildSharedModel`: the load, the `include` / `exclude` selection, the closure check, the serialized document, the re-load with core vocabulary | +| where the published version comes from (`nearestPackageVersion` reads the nearest `package.json`; replace it with your own source of truth) | `renderSharedModelFiles`: the two file names, the artifact's hash, the manifest | + +```bash +meta eject shared-model +``` + +```ts +import { sharedModelFile } from "./codegen/generators/shared-model.js"; + +generators: [sharedModelFile({ name: "acme-common", include: ["acme::common::**"] })], +``` + +A copy that calls those two functions publishes exactly what the packaged generator +publishes, and a consumer's sync accepts it. Nothing a consumer checks is in the copy, so +you cannot change it there by accident; to change it you would have to replace the calls, +and then you have stopped agreeing with every consumer of the artifact. Read the project's +own files through `ctx.inputRoot`, as the copy does for the version: under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. An owned `docs` or `api-docs` copy reads through it too: under `meta docs` it is the +directory that holds `metaobjects.config.ts`. + ## What `meta eject` hands over, and what stays core (TypeScript) Owning a generator means owning the code it writes **and the code that code calls**, @@ -422,7 +536,7 @@ generator code. The 20-line programmatic shape for each port is in | Port | Invocation | Programmatic — write a `Generator` | Declarative — template + scope | |---|---|---|---| -| **TypeScript** | `meta init` → `meta gen --list --probe` → `meta eject ` → `meta gen` (Bun/Node CLI) | **Yes.** `meta generator new ` writes a working generator of your own into `codegen/generators/` and wires it — the first move for an output nothing ships. To use a stock generator as shipped, import its factory from its package (`meta gen --list` prints the line) and wire it. To change one: `meta init` scaffolds the LAYOUT and an empty selection (ADR-0034 Amendment 2); `meta eject ...` copies each generator you choose into `codegen/generators/*.ts` and prints the import to put in `metaobjects.config.ts`. Edit them freely. The prompt tier (`prompt-render`, `output-parser`, `extractor`, `output-prompt`, `render-helper`) ejects like the rest: you own which templates get a module and where it lands, while the render and extract engines those modules call stay in the package. `requirement-tests` ejects as one file holding the generator and its default stub renderer, so the stub text is yours to change; the requirement walk, each test's identity and the claim digest stay in the package. `cube-model` (the Cube data-model exporter, [cube-export.md](cube-export.md)) ejects as one file as well: which entities get a cube, where the files land and the YAML call are yours, while the mapping (`buildCubeModel`, which raises every refusal) and the YAML writer (`renderCubeYaml`) stay in the package. `callable` and `trace-helper` eject the same way as the prompt tier: the copy decides which objects get a module and where it lands, and the module body comes from `renderCallableFile` / `renderTraceHelper` in the package. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own (the same ruling in every port). Not yet ejectable: the docs tier (`docs`, `api-docs`, `mermaid-er`) and `shared-model` (FR-023's publisher generator). Until each gains a reference copy, `meta gen --list` marks it **package-only** and `meta eject` names it as such. | **Yes** — `templateGenerator({ template, scope, outputPattern })` in the config's `generators: [...]`. No CLI flag: the config already takes generator values. | +| **TypeScript** | `meta init` → `meta gen --list --probe` → `meta eject ` → `meta gen` (Bun/Node CLI) | **Yes.** `meta generator new ` writes a working generator of your own into `codegen/generators/` and wires it — the first move for an output nothing ships. To use a stock generator as shipped, import its factory from its package (`meta gen --list` prints the line) and wire it. To change one: `meta init` scaffolds the LAYOUT and an empty selection (ADR-0034 Amendment 2); `meta eject ...` copies each generator you choose into `codegen/generators/*.ts` and prints the import to put in `metaobjects.config.ts`. Edit them freely. The prompt tier (`prompt-render`, `output-parser`, `extractor`, `output-prompt`, `render-helper`) ejects like the rest: you own which templates get a module and where it lands, while the render and extract engines those modules call stay in the package. `requirement-tests` ejects as one file holding the generator and its default stub renderer, so the stub text is yours to change; the requirement walk, each test's identity and the claim digest stay in the package. `cube-model` (the Cube data-model exporter, [cube-export.md](cube-export.md)) ejects as one file as well: which entities get a cube, where the files land and the YAML call are yours, while the mapping (`buildCubeModel`, which raises every refusal) and the YAML writer (`renderCubeYaml`) stay in the package. `callable` and `trace-helper` eject the same way as the prompt tier: the copy decides which objects get a module and where it lands, and the module body comes from `renderCallableFile` / `renderTraceHelper` in the package. `shared-model` (FR-023's publisher generator) ejects as a thin generator: the copy owns which source files contribute and where the version comes from, and the artifact a consumer verifies comes from `buildSharedModel` / `renderSharedModelFiles` in the package. `docs` and `api-docs` eject too, and `meta docs` runs the copy from `docs: { generators: [...] }`, never from the top-level `generators` ([details](#the-docs-generators-and-shared-model-typescript)). Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own (the same ruling in every port): `meta gen --list` marks it **package-only** and `meta eject` names it as such. | **Yes** — `templateGenerator({ template, scope, outputPattern })` in the config's `generators: [...]`. No CLI flag: the config already takes generator values. | | **Java / Kotlin** | `mvn metaobjects:generate` / `mvn metaobjects:verify` (`metaobjects-maven-plugin`) | **Yes.** Extend `FileEmittingGenerator` and read the model through `ModelWalk` (both in `metaobjects-codegen-base`) for one of your own. Every generator — built-in or your own — is named in `` and loaded from the project classpath: one seam, not two. There is no default suite, so `` is the complete list. Kotlin runs through the same goal. | **Yes** — `TemplateScopeGenerator` wired as an ordinary ``. No CLI flag: `` is already the seam. | | **C#** | `dotnet meta gen` / `dotnet meta verify` (.NET tool) | **Yes.** Implement `IGenerator` in the owned console project `codegen/` and list it in `codegen/Program.cs`; `dotnet meta gen` / `verify --codegen` hand off to that project whenever `codegen/Codegen.csproj` exists. `dotnet meta eject ` scaffolds the project, or write its two files by hand. An owned generator the `--generators` selection does not name still runs. | **Yes** — `dotnet meta gen --template-spec --template-root `. | | **Python** | `metaobjects gen` / `metaobjects verify` (console-script) | **Yes.** Name your generator as `module:symbol` in `--generators` or a target's `generators` in `metaobjects.config.yaml`; the symbol is an instance or a function returning one. Read the model through `metaobjects.codegen.model_walk`. (`--provider module:symbol` registers **metamodel vocabulary**, not a generator.) | **Yes** — `metaobjects gen --template-spec --templates `. | diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md index 41dff507a..01e2b70ec 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md @@ -33,7 +33,8 @@ Other flags: `--layout flat|package`, `--base-url `. Configure defaults in `docs:` block in `metaobjects.config.ts` (`outDir`, `layout`, `baseUrl`, `surfaces`); CLI flags override it. The api surface needs the gen config (it documents what the codegen produced); with no config it is skipped with a note, -and the model surface still emits from metadata alone. +and the model surface still emits from metadata alone. A config that exists but fails to +load stops `meta docs` (the load error, exit 2, nothing written), as it stops `meta gen`. **Never wire `docsFile()` or `apiDocsFile()` into `generators`.** They are the engines behind `meta docs`, so `meta gen` warns about each one it finds in the suite and skips it: @@ -44,11 +45,20 @@ no docs are written by that run. The warning is on stderr, and in a structured r `meta verify` reports the wiring as an advisory row, rule `doc-generator-in-gen`. Remove the entry and run `meta docs`. -**Do not add `mermaidErDiagram()` to a new config.** It is deprecated: `meta docs` already -draws the ER diagram on its overview page with nothing wired. A `mermaidErDiagram()` -already wired in `generators` still runs under `meta gen`. The stable names `"docs"` and -`"mermaid-er"` written as strings in `generators` are refused, so `meta gen` fails on -them; `meta verify` reports that as rule `doc-generator-refused-by-gen`. +**An owned docs generator goes in `docs: { generators: [...] }`, never in `generators`.** +`meta eject docs` (or `api-docs`) copies the generator and prints that wiring; `meta docs` +and `meta verify --docs` then run the copy in place of the packaged one. Under +`generators` it is skipped like the packaged one, and the warning, the `help[]` next step +and the `doc-generator-in-gen` advisory all say to move it, not to remove it. Listed in +the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). + +**There is no `mermaidErDiagram()` generator.** 1.1 removed it: `meta docs` draws the ER +diagram on its overview page and on each entity's page with nothing wired. A config that +still imports it, or names `"mermaid-er"`, fails with a message that says so; delete the +entry and run `meta docs`. The stable name `"docs"` written as a string in `generators` +is refused as well, so `meta gen` fails on it. `meta verify` reports both names as rule +`doc-generator-refused-by-gen`. **Before calling any generated code, read `./docs/api/AGENT-API.md`** — it has the exact imports, signatures, payload field shapes, and runnable examples for this diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md index c5903cffc..e57030499 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript.md @@ -126,10 +126,29 @@ name. Import by default and eject to customize: an owned copy you never change o behind the reference. (`entityFile` / `queriesFile` / `routesFile` / `barrel` were removed from that subpath at 1.0 and restored in 1.1, so on a 1.0.x install an owned copy is the only path for those four.) The engine primitives come from the package main entry, -`@metaobjectsdev/codegen-ts`. Every stock generator can be ejected except the `template` -primitive, which has no emit logic of its own to own. `shared-model` and the docs tier -(`docs`, `api-docs`, `mermaid-er`) are not yet ejectable; `meta gen --list` marks each -entry without a reference copy `package-only`. +`@metaobjectsdev/codegen-ts`. One entry cannot be ejected and is `package-only` in +`meta gen --list`: the `template` primitive, which has no emit logic of its own. + +Three ejectable entries behave differently from the rest: + +- **`docs` and `api-docs` are run by `meta docs`, not `meta gen`.** `meta docs` writes + both with nothing wired; never put a docs generator, packaged or owned, in `generators`. + To change the pages, `meta eject docs` (or `api-docs`) and list the copy in the docs + block, `docs: { generators: [docsFile()] }`: it carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged one. To change only + a page's wording, put your own `templates/docs/entity-page.md.mustache` in the project + instead. +- **The ER diagram comes from `meta docs`** (overview page, and each entity's page). The + standalone `mermaidErDiagram()` / `"mermaid-er"` generator was removed in 1.1; a config + that still wires it fails with a message saying so. Remove it and run `meta docs`. +- **`shared-model` ejects as a thin generator.** The copy owns which source files may + contribute and where the published version comes from. The artifact, its hash and the + manifest a consumer's `meta deps sync` verifies stay in the package + (`buildSharedModel`, `renderSharedModelFiles`); keep calling both. + +In an owned generator, read the project's own files through `ctx.inputRoot`. Under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. Under `meta docs` it is the directory that holds `metaobjects.config.ts`. **`requirementTests()` — one test stub per requirement the model claims.** A recommended approach, not a contract: a `live` or `partial` requirement gets a stub that fails until diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index d0d0d2201..1417e31e8 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -32,11 +32,13 @@ selection (every reference from a selected node must resolve to another selected node, or the build fails naming the pair) and re-loads the emitted artifact with core providers only, so a Phase 1a export needing non-core vocabulary fails at publish time. It is registered and shows up in `meta gen --list` like any other -generator — but **it is deliberately not offered by `meta eject --list`** (unlike -the four ADR-0034 scaffold-and-own generators). The artifact is a contract whose -bytes a cross-port corpus pins and whose hash consumers verify; a user-owned, -editable copy would invite an artifact that silently stops matching what -consumers expect. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python +generator, and `meta eject shared-model` copies a thin generator you own: which +source files may contribute, and where the published version comes from. The +artifact is a contract whose bytes a cross-port corpus pins and whose hash +consumers verify, so everything a consumer checks stays in the package, in the two +functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not +replace those calls: an unmodified copy publishes the same bytes as the packaged +generator. Only TypeScript can run it in Phase 1a. **Phase 1a is TypeScript and Python only, full stop** — TypeScript and Python are the only two ports that can *consume* a dependency at all today, and only the TypeScript toolchain can *publish* one; Java, Kotlin, and C# do not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md index c34159b84..839454b75 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/owning-generators.md @@ -10,7 +10,7 @@ it there, in the same change, and keep going: | Port | Where the fix goes | |---|---| -| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. `shared-model` and the docs tier (`docs`, `api-docs`, `mermaid-er`) are not yet ejectable. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | +| **TypeScript** | Your ejected copy under `codegen/generators/` — and, for routes, the HTTP adapter it calls: `meta eject routes` / `routes-hono` / `entity` also copy that adapter's source (mount helpers, filter parser, error envelopes, pagination) into `codegen/runtime/`, and the ejected output imports it from there, so a bug in a route's behaviour is fixed in that copy. If you are running a generator you never ejected (imported from its package), `meta eject ` it first and swap the import for the local one eject prints. Every stock generator can be ejected except the `template` primitive, which has no emit logic of its own to own. An ejected `docs` or `api-docs` goes in `docs: { generators: [...] }`, not in `generators`: `meta docs` runs it. A generator `meta gen --list` marks `package-only` has no reference template: replace it in `generators` with your own `Generator`, starting from its source in the installed package. | | **Java / Kotlin** | Your own generator class. Subclass the reference generator and override the `protected` method that emits the wrong piece; when that piece is `private`, copy the reference generator's source (Apache-2.0) into your own package and edit it. Put the class in a codegen module that the module running `metaobjects:generate` depends on — the plugin loads `` from that module's compile classpath, and `provided` scope keeps it out of your packaged app — then point `` at it. Full steps: the Java and Kotlin references. | | **C# / Python** | Your owned copy (`dotnet meta eject ` / `metaobjects eject `, then edit it), or a generator of your own that replaces the artifact — an `IGenerator` in `codegen/Program.cs` on C#, a `module:symbol` entry on Python — or a template spec (`--template-spec`). | diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md index 41dff507a..01e2b70ec 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-docs.md @@ -33,7 +33,8 @@ Other flags: `--layout flat|package`, `--base-url `. Configure defaults in `docs:` block in `metaobjects.config.ts` (`outDir`, `layout`, `baseUrl`, `surfaces`); CLI flags override it. The api surface needs the gen config (it documents what the codegen produced); with no config it is skipped with a note, -and the model surface still emits from metadata alone. +and the model surface still emits from metadata alone. A config that exists but fails to +load stops `meta docs` (the load error, exit 2, nothing written), as it stops `meta gen`. **Never wire `docsFile()` or `apiDocsFile()` into `generators`.** They are the engines behind `meta docs`, so `meta gen` warns about each one it finds in the suite and skips it: @@ -44,11 +45,20 @@ no docs are written by that run. The warning is on stderr, and in a structured r `meta verify` reports the wiring as an advisory row, rule `doc-generator-in-gen`. Remove the entry and run `meta docs`. -**Do not add `mermaidErDiagram()` to a new config.** It is deprecated: `meta docs` already -draws the ER diagram on its overview page with nothing wired. A `mermaidErDiagram()` -already wired in `generators` still runs under `meta gen`. The stable names `"docs"` and -`"mermaid-er"` written as strings in `generators` are refused, so `meta gen` fails on -them; `meta verify` reports that as rule `doc-generator-refused-by-gen`. +**An owned docs generator goes in `docs: { generators: [...] }`, never in `generators`.** +`meta eject docs` (or `api-docs`) copies the generator and prints that wiring; `meta docs` +and `meta verify --docs` then run the copy in place of the packaged one. Under +`generators` it is skipped like the packaged one, and the warning, the `help[]` next step +and the `doc-generator-in-gen` advisory all say to move it, not to remove it. Listed in +the docs block, its `meta gen --list` row reads ``WIRED in `docs.generators` `` +(`project.wiredInDocsDoor` in `--format json`). + +**There is no `mermaidErDiagram()` generator.** 1.1 removed it: `meta docs` draws the ER +diagram on its overview page and on each entity's page with nothing wired. A config that +still imports it, or names `"mermaid-er"`, fails with a message that says so; delete the +entry and run `meta docs`. The stable name `"docs"` written as a string in `generators` +is refused as well, so `meta gen` fails on it. `meta verify` reports both names as rule +`doc-generator-refused-by-gen`. **Before calling any generated code, read `./docs/api/AGENT-API.md`** — it has the exact imports, signatures, payload field shapes, and runnable examples for this diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md index c5903cffc..e57030499 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript.md @@ -126,10 +126,29 @@ name. Import by default and eject to customize: an owned copy you never change o behind the reference. (`entityFile` / `queriesFile` / `routesFile` / `barrel` were removed from that subpath at 1.0 and restored in 1.1, so on a 1.0.x install an owned copy is the only path for those four.) The engine primitives come from the package main entry, -`@metaobjectsdev/codegen-ts`. Every stock generator can be ejected except the `template` -primitive, which has no emit logic of its own to own. `shared-model` and the docs tier -(`docs`, `api-docs`, `mermaid-er`) are not yet ejectable; `meta gen --list` marks each -entry without a reference copy `package-only`. +`@metaobjectsdev/codegen-ts`. One entry cannot be ejected and is `package-only` in +`meta gen --list`: the `template` primitive, which has no emit logic of its own. + +Three ejectable entries behave differently from the rest: + +- **`docs` and `api-docs` are run by `meta docs`, not `meta gen`.** `meta docs` writes + both with nothing wired; never put a docs generator, packaged or owned, in `generators`. + To change the pages, `meta eject docs` (or `api-docs`) and list the copy in the docs + block, `docs: { generators: [docsFile()] }`: it carries a `docsSurface` marker, and + `meta docs` and `meta verify --docs` run it in place of the packaged one. To change only + a page's wording, put your own `templates/docs/entity-page.md.mustache` in the project + instead. +- **The ER diagram comes from `meta docs`** (overview page, and each entity's page). The + standalone `mermaidErDiagram()` / `"mermaid-er"` generator was removed in 1.1; a config + that still wires it fails with a message saying so. Remove it and run `meta docs`. +- **`shared-model` ejects as a thin generator.** The copy owns which source files may + contribute and where the published version comes from. The artifact, its hash and the + manifest a consumer's `meta deps sync` verifies stay in the package + (`buildSharedModel`, `renderSharedModelFiles`); keep calling both. + +In an owned generator, read the project's own files through `ctx.inputRoot`. Under +`meta verify --codegen` the run writes into a temporary tree and `ctx.projectRoot` is that +tree. Under `meta docs` it is the directory that holds `metaobjects.config.ts`. **`requirementTests()` — one test stub per requirement the model claims.** A recommended approach, not a contract: a `live` or `partial` requirement gets a stub that fails until diff --git a/fixtures/generator-registry-conformance/README.md b/fixtures/generator-registry-conformance/README.md index b0e41b7cd..f921cc2be 100644 --- a/fixtures/generator-registry-conformance/README.md +++ b/fixtures/generator-registry-conformance/README.md @@ -35,7 +35,7 @@ to make the choice cheap, and `layer` is the axis it is cheap along. | `persistence` | queries, db-context, repository, exposed-table, relations, stored-proc | app shape | | `api` | routes, routes-hono, filter-allowlist, validator, spring-config | app shape | | `client` | form, hooks, grid, grid-hook | app shape | -| `docs` | docs, mermaid-er, api-docs | on by default (`meta docs`) | +| `docs` | docs, api-docs | on by default (`meta docs`) | | `capability` | prompt-render, output-parser, output-prompt, extractor, render-helper, trace-helper, requirement-tests, cube-model, shared-model, template, callable | `meta gen --list --probe` | Six, not ten. An earlier draft split `capability` into `trace` / `requirements` / diff --git a/fixtures/generator-registry-conformance/registry.json b/fixtures/generator-registry-conformance/registry.json index a318ef529..e283f70f2 100644 --- a/fixtures/generator-registry-conformance/registry.json +++ b/fixtures/generator-registry-conformance/registry.json @@ -203,13 +203,6 @@ "layer": "docs", "note": "Owned by `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended native suite.", "ports": ["typescript"] - }, - "mermaid-er": { - "concept": "Neutral Mermaid ER diagram of the entity/relationship model.", - "tier": "neutral", - "layer": "docs", - "note": "Neutral artifact owned by the docs engine (ADR-0020). Deprecated under `meta gen`: a `mermaidErDiagram()` already wired in `generators` still runs; `meta docs` draws the same diagram on its overview page.", - "ports": ["typescript"] } } } diff --git a/server/typescript/packages/cli/src/commands/docs.ts b/server/typescript/packages/cli/src/commands/docs.ts index a9eed1ef9..e9410be17 100644 --- a/server/typescript/packages/cli/src/commands/docs.ts +++ b/server/typescript/packages/cli/src/commands/docs.ts @@ -45,6 +45,8 @@ import { renderCoreMetamodelDocs, } from "@metaobjectsdev/metadata"; import { dbEmittingObjects, DEFAULT_DIALECT, DEFAULT_DOCS_DIR, missingDialectMessage, resolveGenerators, runEmitsHonoRoutes, runEmitsUiTier, runRoutesExpose } from "@metaobjectsdev/codegen-ts"; +import { docsDoorGenerators } from "@metaobjectsdev/codegen-ts"; +import type { DocsDoorFacts } from "@metaobjectsdev/codegen-ts"; import type { MetaDataTypeProvider, MetaRoot } from "@metaobjectsdev/metadata"; import { generateSite, SITE_TEMPLATE_NAMES, SITE_ASSET_NAMES, readSiteFile } from "@metaobjectsdev/docs-site"; // The `agent` schema surface takes the physical schema as an ARGUMENT with its resolvers @@ -454,41 +456,30 @@ export async function docsCommand( ? resolvePath(cwd, flags.templates) : genConfigDir; - // Best-effort load of metaobjects.config.ts to pick up consumer-supplied - // providers (e.g. a project's custom field/object subtypes). Unlike `gen`, - // docs does NOT require a config — the Tier-2 "metadata alone" promise must - // hold for config-less projects. If the config is absent or invalid, fall - // back to defaults; the loader still surfaces a stable unknown-subtype error - // if the metadata genuinely uses an unregistered type. + // Load metaobjects.config.ts when the project has one, to pick up consumer-supplied + // providers (e.g. a project's custom field/object subtypes), the `docs` block and any + // owned docs generator it lists. Unlike `gen`, docs does NOT require a config — the + // Tier-2 "metadata alone" promise must hold for config-less projects, so an ABSENT + // config falls back to defaults and stays silent. + // + // A config that EXISTS but fails to load is fatal, with the exit code and message + // `meta gen` gives the same failure. Carrying on without it would run the packaged + // generators over the pages an owned `docs.generators` copy writes, and exit 0. let loadedConfig: Awaited> | undefined; let configProviders: NonNullable>["providers"]> | undefined; // The same config's contributions to the metadata LOAD — providers plus the shipped // `libraries` a project opts into (#333). `configProviders` stays separate because the // site surface has its own loader and takes providers alone. let configLoadOptions: ReturnType = {}; - // hasConfig gates the api surface: api docs describe the GENERATED REST - // surface, which only exists when there is a (loadable) gen config. A config - // that EXISTS but fails to load degrades to model-only with a warning. - const hasConfig = existsSync(join(genConfigDir, "metaobjects.config.ts")); - // Only attempt the load when the file is actually present: absence is the - // expected config-less case (stay silent), but a config that EXISTS yet fails - // to load is surfaced as a warning rather than silently degrading to - // provider-less docs — otherwise a custom-type project would later fail with a - // cryptic unknown-subtype error instead of the real config error. - if (hasConfig) { + if (existsSync(join(genConfigDir, "metaobjects.config.ts"))) { try { loadedConfig = await loadMetaobjectsConfig(genConfigDir); - configProviders = loadedConfig.providers; - configLoadOptions = loadMemoryOptionsFrom(loadedConfig); } catch (err) { - log.warn( - `docs: metaobjects.config.ts failed to load (${describeError(err)}); ` + - `generating docs without its providers`, - ); - loadedConfig = undefined; - configProviders = undefined; - configLoadOptions = {}; + log.error(describeError(err)); + return 2; } + configProviders = loadedConfig.providers; + configLoadOptions = loadMemoryOptionsFrom(loadedConfig); } // Merge the config `docs:` block with CLI overrides over documented defaults. @@ -608,9 +599,34 @@ export async function docsCommand( } as never, renderContext, projectRoot, + inputRoot: genConfigDir, warn: (msg) => log.warn(`docs: ${msg}`), }; + // An owned docs generator (ADR-0034 Amendment 7). It is listed in this door's own config, + // `docs: { generators: [...] }`, and says which surface it takes over by its + // `docsSurface` marker; for that surface it runs IN PLACE OF the packaged generator, and + // `verify --docs`, which calls this command, checks the committed pages against it. An + // entry with no marker, or two for one surface, is refused by name. + let ownedModel: Generator | undefined; + let ownedApi: Generator | undefined; + try { + ({ model: ownedModel, api: ownedApi } = docsDoorGenerators(loadedConfig?.docs?.generators ?? [])); + } catch (err) { + log.error(`docs: ${describeError(err)}`); + return 2; + } + // The context a surface's generator runs with. The facts the two page sets must agree + // on (where the api pages are, whether the model pages exist) ride on `ctx.docsDoor` + // rather than on factory options, because a listed generator is already constructed. Its + // own `filter` is applied as the gen runner applies one; the packaged generators carry + // none, so their context is the one above. + const doorCtx = (generator: Generator, facts: DocsDoorFacts): GenContext => ({ + ...ctx, + matches: generator.filter ?? ctx.matches, + docsDoor: facts, + }); + const emit: EmittedFile[] = []; let modelFiles: EmittedFile[] = []; @@ -632,9 +648,11 @@ export async function docsCommand( // polyglot model page references each port's api docs). `labeled` already // carries {label, subDir, baseUrl?} (docsFile ignores the extra `lang`), so // pass it straight through. Otherwise no api refs. - const modelOpts = apiSelected ? { apiSurfaces: labeled } : {}; + const modelGenerator = ownedModel ?? docsFile(); try { - modelFiles = await docsFile(modelOpts).generate(ctx); + modelFiles = await modelGenerator.generate( + doorCtx(modelGenerator, apiSelected ? { apiSurfaces: labeled } : {}), + ); } catch (err) { const msg = describeError(err); // Duplicate output path (silent-overwrite backstop): the generator already @@ -647,7 +665,7 @@ export async function docsCommand( // The framework templates resolve from disk; inside the compiled `meta` // binary they live on a virtual fs the provider cannot read. Surface that // as an actionable message rather than a cryptic render failure. - if (/entity-page|template-page|failed rendering|ENOENT|not found/i.test(msg)) { + if (ownedModel === undefined && /entity-page|template-page|failed rendering|ENOENT|not found/i.test(msg)) { log.error( `docs: failed to render — templates not found. ` + `Run 'meta docs' from an installed package layout (with on-disk ` + @@ -683,12 +701,15 @@ export async function docsCommand( try { // Emit every surface THIS port owns (loop so it generalizes beyond the // single ts surface owned today). + const apiGenerator = ownedApi ?? apiDocsFile(); for (const s of labeled.filter((s) => s.lang === "ts")) { apiFiles.push( - ...(await apiDocsFile({ - subDir: s.subDir, - modelSurface: docsCfg.surfaces.includes("model"), - }).generate(ctx)), + ...(await apiGenerator.generate( + doorCtx(apiGenerator, { + apiSubDir: s.subDir, + modelSurface: docsCfg.surfaces.includes("model"), + }), + )), ); } } catch (err) { @@ -708,10 +729,6 @@ export async function docsCommand( `meta docs: api surface '${s.lang}' (${s.subDir}) is produced by that port's docs command — run it to populate those pages.`, ); } - } else if (hasConfig) { - // Config present but failed to load — already warned above; don't claim an - // api surface we couldn't build. - info("meta docs: api surface skipped — metaobjects.config.ts failed to load."); } else { info("meta docs: api surface skipped — no metaobjects.config.ts (nothing generated to document)."); } @@ -737,8 +754,6 @@ export async function docsCommand( columnNamingStrategy: strategy, }).generate(ctx)), ); - } else if (hasConfig) { - info("meta docs: agent surface skipped — metaobjects.config.ts failed to load."); } else { info( "meta docs: agent surface skipped — no metaobjects.config.ts (the physical schema " + @@ -773,11 +788,18 @@ export async function docsCommand( const modelTemplates = modelFiles.length > 0 ? modelFiles.length - entityCount - modelOverview : 0; - const modelSummary = docsCfg.surfaces.includes("model") - ? `${modelOverview} overview + ${entityCount} entity page(s) + ${modelTemplates} template page(s)` - : "model surface skipped"; + // An owned generator decides its own page set, so the overview/entity/template split + // above (which is the packaged generator's) is not claimed for it: its pages are counted + // and it is named, which also tells the reader that their copy is the one that ran. + const ownedBy = (g: Generator | undefined): string => + g === undefined ? "" : ` from the owned generator '${g.name}'`; + const modelSummary = !docsCfg.surfaces.includes("model") + ? "model surface skipped" + : ownedModel !== undefined + ? `${modelFiles.length} model page(s)${ownedBy(ownedModel)}` + : `${modelOverview} overview + ${entityCount} entity page(s) + ${modelTemplates} template page(s)`; const apiSummary = apiFiles.length > 0 - ? `${apiFiles.length} api page(s)` + ? `${apiFiles.length} api page(s)${ownedBy(ownedApi)}` : "no api pages"; // Reported only when it actually wrote something. A project with no ledger emits no // requirements file, and saying "0 requirement pages" would advertise a surface that diff --git a/server/typescript/packages/cli/src/commands/eject.ts b/server/typescript/packages/cli/src/commands/eject.ts index a4f5a092a..5e513c834 100644 --- a/server/typescript/packages/cli/src/commands/eject.ts +++ b/server/typescript/packages/cli/src/commands/eject.ts @@ -32,6 +32,7 @@ import { type RuntimeEjectResult, type RuntimeFileResult, } from "../lib/runtime-eject.js"; import { HTTP_RUNTIME_PACKAGE, OWNED_RUNTIME_DIR } from "@metaobjectsdev/codegen-ts"; +import type { GeneratorRegistryEntry } from "@metaobjectsdev/codegen-ts"; import { emitStructured, type OutputFormat } from "../lib/format.js"; import { ejectLibrary, ejectableLibraryNames, isLibraryName, libraryStaleness, @@ -111,6 +112,20 @@ function extractExportName(importLine: string, name: string): string { return match[1]; } +// The CALL a template's header documents, when it is not the bare `()`. A factory +// that cannot be called without options says so where it says how to import it +// (`// generators: [sharedModelFile({ name, include })]`), and a docs generator shows the +// docs block it is listed in (`// docs: { generators: [docsFile()] }`). Read from the +// header for the reason the import line is: the template is the one thing that knows, and +// the catalog deliberately does not derive a call from an options summary. +const HEADER_ENTRY_RE = /^\/\/\s+(?:docs: \{ )?generators: \[(\w+\(.*\))\](?: \})?,?\s*$/m; + +/** The call to wire: the header's, when it documents one for this template's own export. */ +function extractEntry(templateSource: string, exportName: string): string { + const call = HEADER_ENTRY_RE.exec(templateSource)?.[1]; + return call !== undefined && call.startsWith(`${exportName}(`) ? call : `${exportName}()`; +} + function extractImportLine(templateSource: string, name: string): string { const match = HEADER_IMPORT_RE.exec(templateSource); if (!match?.[1]) { @@ -143,6 +158,9 @@ export interface EjectResult { importLine: string; /** The symbol the template exports — the binding to REPLACE in the config. */ exportName: string; + /** The call to wire, with any options the factory cannot be called without + * (`sharedModelFile({ name, include })`), as the template's header documents it. */ + entry: string; /** The package the generator currently comes from, i.e. the import to remove. */ packageName: string; /** Advisory lines about packages the ejected file imports but the project lacks. */ @@ -243,6 +261,7 @@ export async function ejectGenerator(opts: EjectOptions): Promise { path: rel, importLine, exportName, + entry: extractEntry(templateSource, exportName), packageName: source.packageName, dependencyNotes: notes, templatePackages: requiredPackages(templateSource), @@ -466,13 +485,32 @@ async function listOutput(cwd: string): Promise { // the command // --------------------------------------------------------------------------- +/** + * Where an ejected generator's entry goes, and which command then runs it. + * + * gen the top-level `generators`, run by `meta gen` — every generator but two. + * docs-door `docs: { generators: [...] }`, run by `meta docs` (`docs`, `api-docs`). + */ +interface EjectWiring { + kind: "gen" | "docs-door"; + /** The call, with the options the factory cannot be called without. */ + entry: string; +} + +function wiringFor(entry: GeneratorRegistryEntry | undefined, call: string): EjectWiring { + return { kind: entry?.docsSurface !== undefined ? "docs-door" : "gen", entry: call }; +} + /** One row of the `--format json` payload. */ interface EjectedRow { name: string; path: string; status: EjectResult["status"]; - /** The two edits an adopter makes in `metaobjects.config.ts`. */ - wire: { import: string; entry: string }; + /** The two edits an adopter makes in `metaobjects.config.ts`. `into` is present when the + * entry does NOT go in the top-level `generators`: an owned docs generator (`docs`, + * `api-docs`) is listed in the docs door's own block, and `into` names that key. Such a + * copy is run by `meta docs` and checked by `meta verify --docs`, not by `meta gen`. */ + wire: { import: string; entry: string; into?: "docs.generators" }; requires: readonly string[]; /** The adapter source copied beside it (ADR-0034 Amendment 3). Absent when none. */ runtime?: RuntimeFileResult[]; @@ -537,7 +575,7 @@ function unknownNames(names: readonly string[]): string[] { /** Print the per-name text report — every branch below predates this command taking * more than one name, and each was written against a real incident. */ -function reportOne(result: EjectResult, name: string): void { +function reportOne(result: EjectResult, name: string, wiring: EjectWiring): void { if (result.status === "preserved") { // "already exists — left untouched" was the whole message, and it answered the // question nobody has. What an owner needs to know is whether their copy still @@ -594,30 +632,45 @@ function reportOne(result: EjectResult, name: string): void { log.info(`Ejected "${name}" -> ${result.path}. You own it now (ADR-0034 scaffold-and-own).`); } - // REPLACE, never "paste". A generator reaches `generators: [...]` under ONE binding, - // so a reader told to "paste" gets a duplicate identifier at best — and at worst - // deletes nothing, keeps `formFile()` in the array bound to the PACKAGE import, and - // silently runs the packaged generator while editing the ejected file. That failure - // is invisible and is the exact one ejecting exists to prevent. - // - // But eject reads no config, so it cannot know WHICH of the three states this project - // is in, and stating one of them as fact is wrong in the other two. Name the goal, - // then the three branches; the reader knows which one they are looking at. - log.info(`In metaobjects.config.ts, "${result.exportName}" must resolve to this file:`); - log.info(` ${result.importLine}`); - log.info( - ` - If it is imported from "${factoryModuleOf(name) ?? result.packageName}", REPLACE that import with the ` + - "line above. Adding a second one leaves `generators` bound to the PACKAGED " + - "generator, and your edits to this file do nothing.", - ); - log.info( - " - If it is already imported from ./codegen/generators/, it points here already " + - "— nothing to change.", - ); - log.info( - ` - If ${result.exportName}() is not in \`generators\` yet, add the import above ` + - "AND the entry.", - ); + if (wiring.kind === "docs-door") { + // A docs generator is the docs door's, so its wiring is stated in that door's terms + // and nowhere else: the `docs` block of the config, and the command that reads it. + // Sending its owner to the array `meta gen` runs would end in a skipped generator and + // a `meta docs` that still runs the packaged one. + log.info("`meta docs` runs this generator, in place of the packaged one. In metaobjects.config.ts:"); + log.info(` ${result.importLine}`); + log.info(` docs: { generators: [${wiring.entry}] }`); + log.info( + " If the `docs` block already lists docs generators, add this entry to that list. " + + "`meta verify --docs` then checks the committed pages against your copy. " + + "Run `meta docs` to write them.", + ); + } else { + // REPLACE, never "paste". A generator reaches `generators: [...]` under ONE binding, + // so a reader told to "paste" gets a duplicate identifier at best — and at worst + // deletes nothing, keeps `formFile()` in the array bound to the PACKAGE import, and + // silently runs the packaged generator while editing the ejected file. That failure + // is invisible and is the exact one ejecting exists to prevent. + // + // But eject reads no config, so it cannot know WHICH of the three states this project + // is in, and stating one of them as fact is wrong in the other two. Name the goal, + // then the three branches; the reader knows which one they are looking at. + log.info(`In metaobjects.config.ts, "${result.exportName}" must resolve to this file:`); + log.info(` ${result.importLine}`); + log.info( + ` - If it is imported from "${factoryModuleOf(name) ?? result.packageName}", REPLACE that import with the ` + + "line above. Adding a second one leaves `generators` bound to the PACKAGED " + + "generator, and your edits to this file do nothing.", + ); + log.info( + " - If it is already imported from ./codegen/generators/, it points here already " + + "— nothing to change.", + ); + log.info( + ` - If ${wiring.entry} is not in \`generators\` yet, add the import above ` + + "AND the entry.", + ); + } for (const line of result.dependencyNotes) log.info(line); if (result.runtime !== undefined && result.runtime.files.length > 0) reportRuntime(result.runtime.files); } @@ -721,15 +774,22 @@ export async function ejectCommand( runtimeGenerators.add(name); for (const p of result.runtime.packages) runtimePackages.add(p); } + const catalogEntry = catalog[name]; + const wiring = wiringFor(catalogEntry, result.entry); + const docsDoor = wiring.kind === "docs-door"; rows.push({ name, path: result.path, status: result.status, - wire: { import: result.importLine, entry: `${result.exportName}()` }, - requires: catalog[name]?.requires ?? [], + wire: { + import: result.importLine, + entry: wiring.entry, + ...(docsDoor ? { into: "docs.generators" as const } : {}), + }, + requires: catalogEntry?.requires ?? [], ...(result.runtime !== undefined ? { runtime: result.runtime.files } : {}), }); - if (fmt === "text") reportOne(result, name); + if (fmt === "text") reportOne(result, name, wiring); } } catch (err) { log.error(describeError(err)); diff --git a/server/typescript/packages/cli/src/commands/gen.ts b/server/typescript/packages/cli/src/commands/gen.ts index f8757b53c..6a42392aa 100644 --- a/server/typescript/packages/cli/src/commands/gen.ts +++ b/server/typescript/packages/cli/src/commands/gen.ts @@ -290,6 +290,7 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat antiPatterns, generatorCount: forgeConfig.generators?.length ?? 0, skippedDocGenerators: result.skippedDocGenerators, + skippedOwnedDocGenerators: result.skippedOwnedDocGenerators, ...(migrateAdvice !== undefined ? { migrateAdvice } : {}), }; const output = diff --git a/server/typescript/packages/cli/src/commands/verify.ts b/server/typescript/packages/cli/src/commands/verify.ts index 75827c509..ac1dd9bcb 100644 --- a/server/typescript/packages/cli/src/commands/verify.ts +++ b/server/typescript/packages/cli/src/commands/verify.ts @@ -1628,7 +1628,7 @@ export async function verifyCommand( let result; try { - result = await computeCodegenDrift(forgeConfig, codegenRoot, genConfigDir, genCollection.inScope); + result = await computeCodegenDrift(forgeConfig, codegenRoot, genConfigDir, genCollection.inScope, genCollection.ownFiles); } catch (err) { log.error(`verify --codegen: regeneration failed: ${describeError(err)}`); return 1; diff --git a/server/typescript/packages/cli/src/lib/catalog-listing.ts b/server/typescript/packages/cli/src/lib/catalog-listing.ts index ed3a1b343..4da5c7cb7 100644 --- a/server/typescript/packages/cli/src/lib/catalog-listing.ts +++ b/server/typescript/packages/cli/src/lib/catalog-listing.ts @@ -79,8 +79,8 @@ export interface GeneratorCatalogRow { install: { dev: string[]; runtime: string[] }; /** * How to run this generator AS SHIPPED: the import line for its package factory. No - * copy involved; `meta eject` prints the local import for an owned one. Absent on the - * `docs` layer, whose door is `meta docs`. + * copy involved; `meta eject` prints the local import for an owned one. Absent on a row + * with a `door`: `meta docs` runs that generator, and `meta gen` skips it. */ wire?: GeneratorWiring; source: { @@ -102,6 +102,10 @@ export interface GeneratorCatalogRow { /** Present iff it is wired as a stable-name STRING that `meta gen` refuses (a neutral * name): the run fails on it, rather than running or skipping the generator. */ refused?: true; + /** Present iff this project lists an owned generator for this row's docs surface in + * `docs: { generators: [...] }`: `meta docs` runs that copy in place of the packaged + * generator. The right place for it, where `wired` (under `generators`) is not. */ + wiredInDocsDoor?: true; /** This generator's framework is a declared dependency. `null` when neutral or unknown. */ frameworkDetected: boolean | null; /** Files this generator WOULD emit for this model. `null` unless --probe, and @@ -303,6 +307,12 @@ export async function buildCatalogListing(opts: CatalogListingOpts = {}): Promis .filter((spec): spec is string => typeof spec === "string" && isNeutralGeneratorName(spec)), ); + // The docs surfaces this project took over with an owned generator. Keyed by surface, + // as the docs door itself is: an owner may rename the copy. + const ownedDocsSurfaces = new Set( + (opts.project?.config.docs?.generators ?? []).map((g) => g?.docsSurface).filter((x) => x !== undefined), + ); + for (const entry of listCatalog()) { // Asked of the runner's own definition, never restated: which generators `meta gen` // skips is a fact about the runner, and a row that disagreed with it is the defect @@ -381,6 +391,9 @@ export async function buildCatalogListing(opts: CatalogListingOpts = {}): Promis row.project = { wired: project.wiredNames.has(entry.name), ...(refusedNames.has(entry.name) ? { refused: true as const } : {}), + ...(entry.docsSurface !== undefined && ownedDocsSurfaces.has(entry.docsSurface) + ? { wiredInDocsDoor: true as const } + : {}), frameworkDetected, wouldEmit, ...(probeError !== undefined ? { probeError } : {}), @@ -481,8 +494,7 @@ export const REFERENCE_HELPER_NOTE: readonly string[] = [ /** The footer line explaining the `package-only` row mark. */ export const PACKAGE_ONLY_NOTE = "package-only = no reference template ships for it, so `meta eject` cannot copy it. " + - "Run one as shipped by importing its factory — the import sits under its row, except " + - "for the docs layer, whose door is `meta docs`."; + "Run one as shipped by importing its factory — the import sits under its row."; /** * Printed under every row `meta gen` does not run (a row with a `door`). One sentence, @@ -496,7 +508,7 @@ export const SKIPPED_BY_GEN_NOTE = /** * Printed under every neutral row. The other outcome for a `generators` entry: the * resolver refuses a neutral stable name, so the string fails the run where a - * constructed generator is skipped or (the deprecated `mermaidErDiagram()`) still runs. + * constructed generator is skipped. */ export const NAME_REFUSED_BY_GEN_NOTE = "its name as a string under `generators` is refused instead: `meta gen` fails on it."; @@ -528,6 +540,8 @@ export function renderCatalogText(rows: CatalogRow[], probed: boolean): string { : r.door !== undefined ? "WIRED but skipped by `meta gen`" : "WIRED", ); } + // An owned docs generator in the docs block: the place `meta docs` runs it from. + if (r.project?.wiredInDocsDoor) marks.push("WIRED in `docs.generators`"); if (r.source.owned) marks.push("owned"); if (r.source.runtimeCopy !== undefined) { // The adapter source this generator's output imports, compared to the package. diff --git a/server/typescript/packages/cli/src/lib/catalog.ts b/server/typescript/packages/cli/src/lib/catalog.ts index 42e715217..a7bedd07f 100644 --- a/server/typescript/packages/cli/src/lib/catalog.ts +++ b/server/typescript/packages/cli/src/lib/catalog.ts @@ -80,11 +80,14 @@ export interface GeneratorWiring { * other half is `meta eject`, which prints the import for the owned copy — the symbol is * the same in both (gated), so moving between them is one import line. * - * Undefined for the `docs` layer: those generators are the engine behind `meta docs`, - * which is their door, and wiring one into `generators: [...]` is the deprecated route. + * Undefined for an entry `meta docs` runs (one that carries `docsSurface`): that command + * is its door, and `meta gen` skips it in `generators: [...]`, so an import line for that + * array would be an instruction to wire something the run ignores. Asked of the marker + * and not of `layer`: the layer is how the listing is grouped, and a docs-layer generator + * that `meta gen` does run would need its import like any other. */ export function packagedWiring(entry: GeneratorRegistryEntry): GeneratorWiring | undefined { - if (entry.layer === "docs") return undefined; + if (entry.docsSurface !== undefined) return undefined; const from = factoryModuleOf(entry.name); if (from === undefined) return undefined; return { import: `import { ${entry.factoryExport} } from "${from}";` }; diff --git a/server/typescript/packages/cli/src/lib/codegen-drift.ts b/server/typescript/packages/cli/src/lib/codegen-drift.ts index 846ab4990..05312dd36 100644 --- a/server/typescript/packages/cli/src/lib/codegen-drift.ts +++ b/server/typescript/packages/cli/src/lib/codegen-drift.ts @@ -127,12 +127,17 @@ function listFiles(dir: string): string[] { * never committed because the `meta gen` that produced the committed tree * never emitted it either. Undefined ⇒ everything is in scope (byte-identical * to a project with no `scope` declared). + * @param sourceFiles the collection's OWN source files, the same list `meta gen` + * threads onto `ctx.sourceFiles`. `shared-model` loads them standalone to build its + * artifact, so without them the regen asked it to publish nothing and this gate + * failed with "no source files to load" on every project that wires it. */ export async function computeCodegenDrift( config: MetaobjectsGenConfig, metadata: MetaData, projectRoot: string, scope?: (fqn: string) => boolean, + sourceFiles?: readonly string[], ): Promise { const root = isAbsolute(projectRoot) ? projectRoot : resolve(projectRoot); @@ -211,7 +216,13 @@ export async function computeCodegenDrift( genStateDir: join(tempRoot, ".gen-state"), mergeStrategy: "overwrite", baseline: "fresh", + // The real project, for a generator that reads its files (`ctx.inputRoot`): + // `shared-model` takes its default version from the nearest package.json, and an + // owned generator may read anything. `projectRoot` stays the temp tree, which the + // owned route and entity generators compute their adapter import against. + inputRoot: root, ...(scope !== undefined ? { scope } : {}), + ...(sourceFiles !== undefined ? { sourceFiles } : {}), }); // Diff each committed outDir against its temp mirror. diff --git a/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts b/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts index 530057a84..94548f776 100644 --- a/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts +++ b/server/typescript/packages/cli/src/lib/doc-generator-advisory.ts @@ -6,8 +6,15 @@ // this way verified clean and was never told that the generator it listed produces // nothing — nor that the gate it was running does not check documentation at all. // -// A neutral stable NAME (`"docs"`, `"mermaid-er"`) is a different outcome: `meta gen` -// refuses the string and the run fails, so the finding says refused, not skipped. +// A neutral stable NAME (`"docs"`) is a different outcome: `meta gen` refuses the string +// and the run fails, so the finding says refused, not skipped. So is a REMOVED name +// (`"mermaid-er"`, whose diagram `meta docs` draws): the finding carries the resolver's +// own message for it. +// +// An OWNED docs generator (it carries a `docsSurface` marker) is skipped like a packaged +// one, and the fix is the opposite: it is MOVED to `docs: { generators: [...] }`, where +// `meta docs` runs it. Telling its owner to remove it would be telling them to stop +// using the generator they ejected. // // Read off the loaded config, not off a regen: the fact is in `generators`, and asking // the runner's and the resolver's own predicates keeps this from growing a second list @@ -20,6 +27,7 @@ import { generatorRegistry, isNeutralGeneratorName, isPackagedDocsGenerator, + removedGeneratorMessage, DOCS_DOOR_COMMAND, type MetaobjectsGenConfig, } from "@metaobjectsdev/codegen-ts"; @@ -31,8 +39,8 @@ export interface DocGeneratorFinding { file: string; /** The command that produces what the wired generator would have. */ construct: string; - /** True when `meta gen` REFUSES the entry (a neutral stable-name string), so the run - * fails; false when the runner warns about and skips it. */ + /** True when `meta gen` REFUSES the entry (a neutral or removed stable-name string), so + * the run fails; false when the runner warns about and skips it. */ refused: boolean; message: string; } @@ -50,13 +58,34 @@ function finding(wired: string, refused: boolean): DocGeneratorFinding { }; } +/** An owned docs generator listed under `generators`: skipped there, and it goes elsewhere. */ +function ownedFinding(name: string): DocGeneratorFinding { + return { + file: CONFIG_FILE, + construct: DOCS_DOOR_COMMAND, + refused: false, + message: + `${CONFIG_FILE}: the owned docs generator "${name}" is listed under \`generators\`, and ` + + `\`meta gen\` skips it — \`${DOCS_DOOR_COMMAND}\` runs an owned docs generator from ` + + "`docs: { generators: [...] }`, and `meta verify --docs` checks what it writes. " + + "Move it there.", + }; +} + +/** A removed stable name: the resolver refuses it, and its own message says what to do. */ +function removedFinding(message: string): DocGeneratorFinding { + return { file: CONFIG_FILE, construct: DOCS_DOOR_COMMAND, refused: true, message: `${CONFIG_FILE}: ${message}` }; +} + /** * One finding per `generators` entry `meta gen` skips or refuses, in config order. * * A `generators` entry is a constructed generator or a stable-name string (ADR-0021). - * A neutral name is refused by the resolver. Any other string resolves through the - * catalog to the same implementation name the runner tests; a name no entry registers - * is the runner's to refuse, not this scan's. + * A removed or neutral name is refused by the resolver. Any other string resolves + * through the catalog to the same implementation name the runner tests; a name no entry + * registers is the runner's to refuse, not this scan's. A constructed generator is + * tested the way the runner tests it, marker first: an owned copy keeps the packaged + * generator's name unless its owner renamed it. */ export function scanForDocGeneratorsInGen( config: MetaobjectsGenConfig | undefined, @@ -64,7 +93,11 @@ export function scanForDocGeneratorsInGen( const findings: DocGeneratorFinding[] = []; for (const spec of config?.generators ?? []) { if (typeof spec === "string") { - if (isNeutralGeneratorName(spec)) findings.push(finding(`"${spec}"`, true)); + // Every removed name so far is one `meta docs` replaced (gated by a test), which is + // why it is reported here under the docs door. + const removed = removedGeneratorMessage(spec); + if (removed !== undefined) findings.push(removedFinding(removed)); + else if (isNeutralGeneratorName(spec)) findings.push(finding(`"${spec}"`, true)); else if (isPackagedDocsGenerator(generatorRegistry[spec]?.factory().name ?? "")) { findings.push(finding(`"${spec}"`, false)); } @@ -73,7 +106,9 @@ export function scanForDocGeneratorsInGen( // `spec?.name`: the array is typed, but a config is user code, and a hole in it is // the runner's to report rather than a reason for an advisory to throw. const implName = spec?.name; - if (implName !== undefined && isPackagedDocsGenerator(implName)) { + if (spec?.docsSurface !== undefined) { + findings.push(ownedFinding(implName ?? "(unnamed)")); + } else if (implName !== undefined && isPackagedDocsGenerator(implName)) { findings.push(finding(packagedDocsFactoryCall(implName), false)); } } diff --git a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts index 430ae0145..e6c244219 100644 --- a/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts +++ b/server/typescript/packages/cli/src/lib/load-metaobjects-config.ts @@ -131,6 +131,56 @@ function resolveCliPkg(specifier: string): string { * import of an aliased package falls back to normal module resolution (and to * jiti's alias map on non-Bun runtimes). */ +/** + * Generator factories `@metaobjectsdev/codegen-ts/generators` used to export and no longer + * does, each with what an adopter does instead. + * + * Without this, a removal an adopter WILL hit surfaces as whatever the runtime says about a + * missing export: `(0 , _generators.mermaidErDiagram) is not a function` from a transpiled + * config, or `does not provide an export named 'mermaidErDiagram'` from native ESM. Neither + * names MetaObjects, the removal or the remedy. This is the only place that can: the + * subpath still exists and serves every other factory, so nothing upstream of here can tell + * a correct import from a removed one. + * + * The diagnostic is recovered from the THROW, in `removedGeneratorRuntimeError`, so it + * answers for a live import only, wherever that import is: the config itself or a module + * the config imports (an owned generator under `codegen/generators/`, typically). + * + * `mermaidErDiagram` (removed in 1.1) wrote the ER diagram to a file of its own under + * `meta gen`. It was the older home of a diagram `meta docs` draws. + */ +const REMOVED_GENERATOR_EXPORTS: Readonly> = { + mermaidErDiagram: + "`meta docs` draws the ER diagram, for the whole model on its overview page (README.md) " + + "and around each entity on that entity's page.\n" + + " Fix: delete the import and the `mermaidErDiagram()` entry, then run `meta docs`. " + + "The file it wrote (`docs/model.md` under your output directory, unless you moved it " + + "with `outFile`) is no longer generated; delete that too.", +}; +const GENERATORS_SUBPATH = "@metaobjectsdev/codegen-ts/generators"; + +/** + * The diagnostic for a throw that names a removed factory; undefined for any other throw. + * It cannot name the file (the error carries no reference back to the import that produced + * it), so it says where to look instead. Matching is on the SYMBOL and on the fact of a + * missing export, not on one message shape: the shape is the runtime's, and an adopter's + * runtime is not ours to choose. + */ +export function removedGeneratorRuntimeError(err: unknown): string | undefined { + const message = err instanceof Error ? err.message : String(err ?? ""); + if (!/is not a function|does not provide an export named|[Ee]xport named .* not found/.test(message)) { + return undefined; + } + const hit = Object.keys(REMOVED_GENERATOR_EXPORTS).find((n) => new RegExp(`\\b${n}\\b`).test(message)); + if (hit === undefined) return undefined; + return ( + `${hit} is no longer exported from "${GENERATORS_SUBPATH}": 1.1 removed that generator. ` + + `The import is in your metaobjects.config.ts or in a module the config imports, most ` + + `often one of your own under codegen/generators/. ` + + `${REMOVED_GENERATOR_EXPORTS[hit] ?? ""}` + ); +} + function rewriteImportSpecifiers(source: string, aliasMap: Record): string { let result = source; for (const [specifier, resolvedPath] of Object.entries(aliasMap)) { @@ -409,7 +459,14 @@ export async function loadMetaobjectsConfig(projectRoot: string): Promise 0) { help.push( - `\`meta gen\` skipped ${skippedDocs} doc generator(s) — see warnings[]; ${DOCS_DOOR_HELP}`, + `\`meta gen\` skipped ${skippedDocs} doc generator(s) — see warnings[]; ${docsDoorHelp(result)}`, ); } // An absent section is STATED, never omitted: a reader must be able to tell diff --git a/server/typescript/packages/cli/test/catalog-factory-exports.test.ts b/server/typescript/packages/cli/test/catalog-factory-exports.test.ts index b5f7fda34..d6e3eca19 100644 --- a/server/typescript/packages/cli/test/catalog-factory-exports.test.ts +++ b/server/typescript/packages/cli/test/catalog-factory-exports.test.ts @@ -20,7 +20,7 @@ import { join } from "node:path"; import * as coreFactories from "@metaobjectsdev/codegen-ts/generators"; import * as reactFactories from "@metaobjectsdev/codegen-ts-react"; import * as tanstackFactories from "@metaobjectsdev/codegen-ts-tanstack"; -import { factoryModuleOf, listCatalog } from "../src/lib/catalog.js"; +import { catalogEntry, factoryModuleOf, listCatalog, packagedWiring } from "../src/lib/catalog.js"; import { buildCatalogListing, renderCatalogText, type GeneratorCatalogRow } from "../src/lib/catalog-listing.js"; import { ejectGenerator } from "../src/commands/eject.js"; @@ -96,19 +96,46 @@ describe("catalog factory exports", () => { }); describe("meta gen --list — how to wire a generator as shipped", () => { - test("every row outside the docs layer carries its package import, and no derived call", async () => { + test("every row `meta gen` runs carries its package import, and no derived call", async () => { const rows = (await buildCatalogListing()).filter( (r): r is GeneratorCatalogRow => r.kind === "generator", ); for (const row of rows) { - if (row.layer === "docs") { - // `meta docs` is that layer's door; a wiring line would teach the deprecated route. + if (row.door !== undefined) { + // `meta docs` runs it; a line for `generators` would teach wiring `meta gen` skips. expect({ name: row.name, wire: row.wire }).toEqual({ name: row.name, wire: undefined }); continue; } expect(row.wire?.import).toMatch(/^import \{ \w+ \} from "@metaobjectsdev\/codegen-ts[\w/-]*";$/); expect({ name: row.name, keys: Object.keys(row.wire ?? {}) }).toEqual({ name: row.name, keys: ["import"] }); } + // The rows with a door are exactly the two `meta docs` runs. + expect(rows.filter((r) => r.door !== undefined).map((r) => r.name).sort()).toEqual(["api-docs", "docs"]); + }); + + // The import line is withheld because `meta docs` runs the generator (its `docsSurface` + // marker), not because of the layer the listing groups it under. A docs-layer generator + // that `meta gen` does run must print its import like any other. + test("packagedWiring keys on `docsSurface`, not on the docs layer", () => { + const entity = catalogEntry("entity"); + const docs = catalogEntry("docs"); + const apiDocs = catalogEntry("api-docs"); + if (entity === undefined || docs === undefined || apiDocs === undefined) throw new Error("catalog entry missing"); + const entityImport = { import: `import { entityFile } from "@metaobjectsdev/codegen-ts/generators";` }; + + expect(packagedWiring(entity)).toEqual(entityImport); + // Same generator, filed under the docs layer, no marker: it still gets its import. + expect(packagedWiring({ ...entity, layer: "docs" })).toEqual(entityImport); + // Same generator, any layer, carrying the marker: `meta docs` runs it, so no import. + expect(packagedWiring({ ...entity, docsSurface: "model" })).toBeUndefined(); + // The two real ones. + expect(packagedWiring(docs)).toBeUndefined(); + expect(packagedWiring(apiDocs)).toBeUndefined(); + // And with the marker taken off, the docs-layer entry would print its import. + const { docsSurface: _marker, ...unmarked } = docs; + expect(packagedWiring(unmarked)).toEqual({ + import: `import { docsFile } from "@metaobjectsdev/codegen-ts/generators";`, + }); }); test("the engine's factories are served from /generators, the framework packages' from the root", async () => { diff --git a/server/typescript/packages/cli/test/docs-command.test.ts b/server/typescript/packages/cli/test/docs-command.test.ts index 62dc5a1e5..45707b2ed 100644 --- a/server/typescript/packages/cli/test/docs-command.test.ts +++ b/server/typescript/packages/cli/test/docs-command.test.ts @@ -291,10 +291,11 @@ describe("meta docs — standalone neutral metadata docs", () => { expect(files).toContain("Place.md"); }); - test("a broken metaobjects.config.ts is surfaced (not silently swallowed), docs still generate", async () => { - // Basic metadata (no custom types) + a config that THROWS on load. Docs must - // still succeed config-less, but the real config error must be surfaced as a - // warning rather than silently degrading to provider-less docs. + test("a metaobjects.config.ts that fails to load stops docs: the load error, exit 2, nothing written", async () => { + // Basic metadata (no custom types) + a config that THROWS on load. `meta gen` answers + // this with exit 2 and the load error. Docs used to warn, write the model pages without + // the config and exit 0, which runs the packaged generator over the pages an owned + // `docs.generators` copy writes. const root = await project(); await writeFile( join(root, "metaobjects.config.ts"), @@ -315,14 +316,9 @@ describe("meta docs — standalone neutral metadata docs", () => { console.error = origErr; } - expect(code).toBe(0); // config-less generation still works - const files = await readdir(out); - expect(files.length).toBeGreaterThan(0); - // The config failure is SURFACED (warning on stderr), not silently swallowed, - // and carries the loader's real diagnostic so the user can fix it. - const stderr = errLogged.join("\n"); - expect(stderr).toContain("metaobjects.config.ts failed to load"); - expect(stderr).toContain("generating docs without its providers"); + expect(code).toBe(2); + expect(errLogged.join("\n")).toContain("broken config boom"); + expect(existsSync(out)).toBe(false); }); test("exits non-zero with a clear message when metadata is missing", async () => { diff --git a/server/typescript/packages/cli/test/eject-multi.test.ts b/server/typescript/packages/cli/test/eject-multi.test.ts index 102af8627..613f6aa07 100644 --- a/server/typescript/packages/cli/test/eject-multi.test.ts +++ b/server/typescript/packages/cli/test/eject-multi.test.ts @@ -36,7 +36,7 @@ interface EjectPayload { name: string; path: string; status: string; - wire: { import: string; entry: string }; + wire: { import: string; entry: string; into?: string }; requires: string[]; }>; install: { dev: string[]; runtime: string[]; command: string }; @@ -75,17 +75,101 @@ describe("meta eject takes many names", () => { }); test("a package-only generator is named as such, not as unknown", async () => { - // `shared-model` is a real catalog entry with no reference template. Calling it - // "unknown" sends the reader looking for a typo that is not there. + // `template` is a real catalog entry with no reference template: it is the Mustache + // primitive, the adopter already supplies its template, and there is no emit logic to + // own (ADR-0034 Amendment 7). It is the only one left. Calling it "unknown" sends the + // reader looking for a typo that is not there. const dir = tmp(); try { - expect(await ejectCommand(["shared-model"], dir, "text")).toBe(2); + expect(await ejectCommand(["template"], dir, "text")).toBe(2); const err = erred.join("\n"); expect(err).toContain("package-only"); - expect(err).toContain("shared-model"); - expect(err).not.toContain("unknown name(s): shared-model"); + expect(err).toContain("template"); + expect(err).not.toContain("unknown name(s): template"); expect(err).toContain("@metaobjectsdev/codegen-ts"); expect(err).toContain("Nothing was ejected"); + expect(existsSync(join(dir, "codegen/generators/template.ts"))).toBe(false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("shared-model ejects — a thin generator, and its entry carries the options it needs", async () => { + // It used to answer `package-only`. The owned part is which source files may + // contribute and which version is published; the artifact, its hash and the manifest + // stay in the package (`buildSharedModel`, `renderSharedModelFiles`). + const dir = tmp(); + try { + expect(await ejectCommand(["shared-model"], dir, "json")).toBe(0); + expect(erred.join("\n")).not.toContain("package-only"); + const [row] = payload().ejected; + expect(row?.status).toBe("created"); + // Not `sharedModelFile()`: the factory cannot be called without these two. + expect(row?.wire.entry).toBe("sharedModelFile({ name, include })"); + expect(row?.wire.import).toBe( + 'import { sharedModelFile } from "./codegen/generators/shared-model.js";', + ); + // An ordinary `meta gen` generator: the entry goes in the top-level `generators`. + expect(row?.wire.into).toBeUndefined(); + // The two emitted files import nothing, and the copy imports only the engine. + const { install } = payload(); + expect(install.runtime).toEqual([]); + expect(install.dev).toEqual([`@metaobjectsdev/codegen-ts@^${cliVersion()}`]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("docs and api-docs eject — and say that `meta docs` runs the copy, not `meta gen`", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["docs", "api-docs"], dir, "json")).toBe(0); + expect(erred.join("\n")).not.toContain("package-only"); + const rows = payload().ejected; + expect(rows.map((r) => r.name)).toEqual(["docs", "api-docs"]); + expect(rows.map((r) => r.wire.entry)).toEqual(["docsFile()", "apiDocsFile()"]); + expect(rows.map((r) => r.wire.import)).toEqual([ + 'import { docsFile } from "./codegen/generators/docs.js";', + 'import { apiDocsFile } from "./codegen/generators/api-docs.js";', + ]); + // The row says which config key the entry goes in. It is the docs block's, never the + // array `meta gen` runs. + expect(rows.map((r) => r.wire.into)).toEqual(["docs.generators", "docs.generators"]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("the text report says which command runs an ejected docs generator", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["docs"], dir, "text")).toBe(0); + const text = logged.join("\n"); + expect(text).toContain('Ejected "docs" -> codegen/generators/docs.ts'); + expect(text).toContain("`meta docs` runs this generator, in place of the packaged one."); + expect(text).toContain('import { docsFile } from "./codegen/generators/docs.js";'); + expect(text).toContain("docs: { generators: [docsFile()] }"); + expect(text).toContain("`meta verify --docs`"); + // The wiring is the docs door's, start to finish: nothing sends the reader to the + // top-level `generators`, where `meta gen` would skip the copy. + expect(text).not.toContain("in `generators` yet"); + expect(text).not.toContain("leaves `generators` bound"); + expect(text.replaceAll("docs: { generators:", "")).not.toContain("generators:"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("mermaid-er is not a name any more: the generator was removed, and nothing is copied", async () => { + // 1.1 removed the standalone diagram generator (`meta docs` draws the diagram), so + // there is neither a reference template nor a catalog row to answer for it. + const dir = tmp(); + try { + expect(await ejectCommand(["mermaid-er"], dir, "text")).toBe(2); + const err = erred.join("\n"); + expect(err).toContain("unknown name(s): mermaid-er"); + expect(err).not.toContain("package-only"); + expect(existsSync(join(dir, "codegen/generators/mermaid-er.ts"))).toBe(false); } finally { rmSync(dir, { recursive: true, force: true }); } diff --git a/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts b/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts index e58d77bbd..2dbc62774 100644 --- a/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts +++ b/server/typescript/packages/cli/test/gen-docs-door-signals.test.ts @@ -34,8 +34,15 @@ const MODEL = JSON.stringify({ }, }); +interface RepoOpts { + /** Statements placed before the config's `export default`. */ + extra?: string; + /** The config's `docs` block, as an object literal. Absent means no `docs` key. */ + docs?: string; +} + /** `generators` is the config's array literal, verbatim. */ -function setupRepo(generators: string, opts: { extra?: string } = {}): string { +function setupRepo(generators: string, opts: RepoOpts = {}): string { mkdirSync(WORKSPACE_TMP, { recursive: true }); const root = mkdtempSync(join(WORKSPACE_TMP, "gen-docs-door-")); mkdirSync(join(root, "metaobjects"), { recursive: true }); @@ -46,14 +53,14 @@ function setupRepo(generators: string, opts: { extra?: string } = {}): string { join(root, "metaobjects.config.ts"), ` import { defineConfig } from "@metaobjectsdev/codegen-ts"; -import { apiDocsFile, docsFile, mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators"; +import { apiDocsFile, docsFile } from "@metaobjectsdev/codegen-ts/generators"; ${opts.extra ?? ""} export default defineConfig({ outDir: "generated", dialect: "sqlite", dbImport: "../db", extStyle: "none", - generators: ${generators}, + generators: ${generators},${opts.docs !== undefined ? `\n docs: ${opts.docs},` : ""} }); `, ); @@ -81,7 +88,7 @@ afterEach(() => { }); /** Run the CLI against a fresh project and hand back both streams. */ -async function meta(generators: string, args: string[], opts: { extra?: string } = {}): +async function meta(generators: string, args: string[], opts: RepoOpts = {}): Promise<{ code: number; stdout: string; stderr: string; root: string }> { const root = setupRepo(generators, opts); try { @@ -104,11 +111,20 @@ interface ListRow { kind: string; note?: string; door?: string; - project?: { wired: boolean; refused?: true; wouldEmit: number | null; probeError?: string }; + project?: { wired: boolean; refused?: true; wiredInDocsDoor?: true; wouldEmit: number | null; probeError?: string }; } const WIRED = `["entity", apiDocsFile(), docsFile()]`; +/** + * An owned docs generator, as `meta eject docs` leaves one: the packaged generator's + * name, and the `docsSurface` marker that says which surface it takes over. + */ +const OWNED_DOCS = `const ownedDocs = { name: "docs-file", docsSurface: "model" as const, generate: () => [] };`; +const OWNED_MOVE_WARNING = + "[docs-file] is a docs generator: 'meta docs' runs it, from `docs: { generators: [...] }` " + + "in metaobjects.config.ts, and 'meta gen' does not. Move it there. Skipped."; + describe("meta gen — a skipped doc generator is reported in every output form", () => { test("text: the skip is on stderr, and the file listing on stdout is unchanged", async () => { const r = await meta(WIRED, ["gen", "--format", "text"]); @@ -183,23 +199,20 @@ describe("meta gen — a skipped doc generator is reported in every output form" const NEUTRAL_REFUSAL = (name: string): string => `generator "${name}" is neutral (owned by 'meta docs'); not selectable in the gen suite.`; -describe("meta gen — mermaid-er is deprecated: a wired factory runs, the name is refused", () => { - // ADR-0022 Part 2 D1 keeps it deprecated from the gen suite. A config that already - // lists `mermaidErDiagram()` keeps working; the stable name is not selectable. - test("mermaidErDiagram() writes its file and draws no skip warning", async () => { - const r = await meta(`["entity", mermaidErDiagram()]`, ["gen", "--format", "json"]); - expect(r.code).toBe(0); - const payload = JSON.parse(r.stdout) as GenPayload; - expect(payload.gen.map((g) => g.file)).toContain("generated/docs/model.md"); - expect(payload.warnings).toBeUndefined(); - expect(r.stderr).not.toContain("Skipped."); - }); +const MERMAID_ER_REMOVED = 'generator "mermaid-er" was removed in 1.1: `meta docs` draws the ER diagram'; - test("the stable name `mermaid-er` is refused as neutral, and nothing is written", async () => { +describe("meta gen — a doc generator's stable name is refused: `docs` as neutral, `mermaid-er` as removed", () => { + // 1.1 removed the standalone `mermaid-er` generator (ADR-0022 Part 2 D1, as amended): + // `meta docs` draws the diagram. The name is told so, not called neutral or unknown. + test("the stable name `mermaid-er` is refused as removed, and nothing is written", async () => { const root = setupRepo(`["entity", "mermaid-er"]`); try { expect(await run(["gen", "--format", "json", "--cwd", root])).not.toBe(0); - expect(err.join("\n")).toContain(NEUTRAL_REFUSAL("mermaid-er")); + const said = err.join("\n"); + expect(said).toContain(MERMAID_ER_REMOVED); + expect(said).toContain('Remove "mermaid-er" from `generators` and run `meta docs`.'); + expect(said).not.toContain("is neutral"); + expect(said).not.toContain("unknown generator"); expect(existsSync(join(root, "generated"))).toBe(false); } finally { rmSync(root, { recursive: true, force: true }); @@ -289,6 +302,59 @@ describe("meta gen — a config wiring ONLY doc generators is not told to author }); }); +describe("meta gen — an owned docs generator under `generators` is skipped and told where it goes", () => { + // ADR-0034 Amendment 7. The runner skips it by its `docsSurface` marker. Before it was + // also REPORTED as skipped, a config wiring only an owned copy read "no entities to + // generate", and the next step for any skipped doc generator was "remove it". + test("json: wired alone, the summary names the real reason and the next step is to MOVE it", async () => { + const r = await meta(`[ownedDocs]`, ["gen", "--format", "json"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as GenPayload; + expect(payload.gen).toEqual([]); + expect(payload.summary).toContain("doc generator"); + expect(payload.summary).not.toContain("no entities"); + expect(payload.warnings).toEqual([OWNED_MOVE_WARNING]); + expect(payload.help).toEqual([ + "run `meta docs` for documentation, and move the owned docs generators from `generators` " + + "to `docs: { generators: [...] }` in metaobjects.config.ts", + ]); + expect(payload.help.join("\n")).not.toContain("remove"); + expect(payload.help.join("\n")).not.toContain("author entities"); + }); + + test("text: wired alone, stdout names the real reason", async () => { + const r = await meta(`[ownedDocs]`, ["gen", "--format", "text"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + expect(r.stdout).not.toContain("No entities to generate"); + expect(r.stdout).toContain("every wired generator is a doc generator"); + expect(r.stderr).toContain(OWNED_MOVE_WARNING); + }); + + test("json: beside a packaged one, the next step says which to move and which to remove", async () => { + const r = await meta(`["entity", ownedDocs, apiDocsFile()]`, ["gen", "--format", "json"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as GenPayload; + expect(payload.gen.map((g) => g.file)).toEqual(["generated/Talk.ts"]); + expect(payload.warnings).toContain(OWNED_MOVE_WARNING); + expect(payload.warnings?.some((w) => w.includes(API_DOCS_SKIP))).toBe(true); + const step = payload.help.find((h) => h.includes("skipped 2 doc generator(s)")); + expect(step).toContain( + "move the owned docs generators from `generators` to `docs: { generators: [...] }` and remove the packaged ones", + ); + }); + + test("where it belongs, in `docs.generators`, `meta gen` says nothing about it", async () => { + const r = await meta(`["entity"]`, ["gen", "--format", "json"], { + extra: OWNED_DOCS, + docs: `{ generators: [ownedDocs] }`, + }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as GenPayload; + expect(payload.warnings).toBeUndefined(); + expect(payload.help.some((h) => h.includes("meta docs"))).toBe(false); + }); +}); + describe("meta gen --list — a row `meta gen` does not run says so", () => { const rowsOf = (stdout: string): ListRow[] => JSON.parse(stdout) as ListRow[]; const row = (rows: ListRow[], name: string): ListRow => { @@ -319,18 +385,18 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(block(r.stdout, "entity")).not.toContain("meta docs"); }); - // The registry note said "surfaced via `meta docs`", which `meta docs` never did. The - // row states the back-compat fact, and does not present the generator as a choice. - test("text: the mermaid-er row says it is deprecated, and has no door", async () => { + // The standalone `mermaid-er` generator was removed in 1.1, so the catalog has no row + // for it, in either form, and a config that still names it does not break the listing. + test("text: there is no mermaid-er row", async () => { const r = await meta(`["entity"]`, ["gen", "--list", "--format", "text"]); - const mermaid = block(r.stdout, "mermaid-er"); - expect(mermaid).toContain("deprecated"); - expect(mermaid).toContain("do not add it to a new config"); - expect(mermaid).toContain("A `mermaidErDiagram()` already wired in `generators` still runs"); - expect(mermaid).toContain("`meta docs` draws the same diagram on its overview page"); - expect(mermaid).not.toContain("surfaced via"); - expect(mermaid).not.toContain("run by `meta docs`"); - expect(mermaid).not.toContain("not run by `meta gen`"); + expect(r.code).toBe(0); + expect(r.stdout).not.toContain("mermaid-er"); + }); + + test("json: a config that still names it lists cleanly, with no row for it", async () => { + const r = await meta(`["entity", "mermaid-er"]`, ["gen", "--list", "--format", "json"]); + expect(r.code).toBe(0); + expect(rowsOf(r.stdout).some((x) => x.name === "mermaid-er")).toBe(false); }); test("json: the row carries the door and the note", async () => { @@ -342,9 +408,6 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(row(rows, name).door, name).toBe("meta docs"); expect(row(rows, name).note ?? "", name).toContain("meta docs"); } - expect(row(rows, "mermaid-er").door).toBeUndefined(); - expect(row(rows, "mermaid-er").note ?? "").toContain("deprecated"); - expect(row(rows, "mermaid-er").note ?? "").toContain("still runs"); expect(row(rows, "entity").door).toBeUndefined(); expect(row(rows, "entity").note).toBeUndefined(); }); @@ -358,13 +421,13 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { }); // A neutral name written as a string is not skipped: `meta gen` refuses it and exits - // non-zero (asserted above). The mark has to say that, for `docs` and `mermaid-er` both. - const BY_NAME = `["entity", "docs", "mermaid-er", "api-docs"]`; + // non-zero (asserted above). The mark has to say that. + const BY_NAME = `["entity", "docs", "api-docs"]`; test("text: a neutral name wired as a string reads as refused, not skipped or run", async () => { const r = await meta(BY_NAME, ["gen", "--list", "--format", "text"]); expect(r.code).toBe(0); - for (const name of ["docs", "mermaid-er"]) { + for (const name of ["docs"]) { const b = block(r.stdout, name); expect(b, name).toContain("WIRED by name, which `meta gen` refuses"); expect(b, name).not.toContain("WIRED but skipped"); @@ -380,7 +443,7 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { test("json: the refusal is on the row's project facts", async () => { const r = await meta(BY_NAME, ["gen", "--list", "--format", "json"]); const rows = rowsOf(r.stdout); - for (const name of ["docs", "mermaid-er"]) { + for (const name of ["docs"]) { expect(row(rows, name).project?.wired, name).toBe(true); expect(row(rows, name).project?.refused, name).toBe(true); } @@ -389,14 +452,44 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(row(rows, "entity").project?.refused).toBeUndefined(); }); - test("a constructed mermaidErDiagram() reads as WIRED: it still runs", async () => { - const r = await meta(`["entity", mermaidErDiagram(), docsFile()]`, ["gen", "--list", "--format", "json"]); + test("a constructed docsFile() is skipped, not refused: only the name is refused", async () => { + const r = await meta(`["entity", docsFile()]`, ["gen", "--list", "--format", "json"]); const rows = rowsOf(r.stdout); - expect(row(rows, "mermaid-er").project?.wired).toBe(true); - expect(row(rows, "mermaid-er").project?.refused).toBeUndefined(); + expect(row(rows, "docs").project?.wired).toBe(true); expect(row(rows, "docs").project?.refused).toBeUndefined(); }); + // ADR-0034 Amendment 7: an owned docs generator is listed in the docs block, and + // `meta docs` runs it from there. The row has to show that it is in use: read off + // `generators` alone, a correctly wired copy looked like one nobody runs. + const IN_DOCS_DOOR: RepoOpts = { extra: OWNED_DOCS, docs: `{ generators: [ownedDocs] }` }; + + test("json: an owned docs generator in `docs.generators` reads as wired in the docs door", async () => { + const r = await meta(`["entity"]`, ["gen", "--list", "--format", "json"], IN_DOCS_DOOR); + expect(r.code).toBe(0); + const rows = rowsOf(r.stdout); + expect(row(rows, "docs").project?.wiredInDocsDoor).toBe(true); + expect(row(rows, "docs").project?.wired).toBe(false); + // Keyed by surface: the api surface was not taken over. + expect(row(rows, "api-docs").project?.wiredInDocsDoor).toBeUndefined(); + expect(row(rows, "entity").project?.wiredInDocsDoor).toBeUndefined(); + }); + + test("text: the row says where it is wired, and does not call it skipped", async () => { + const r = await meta(`["entity"]`, ["gen", "--list", "--format", "text"], IN_DOCS_DOOR); + expect(r.code).toBe(0); + expect(block(r.stdout, "docs")).toContain("WIRED in `docs.generators`"); + expect(block(r.stdout, "docs")).not.toContain("WIRED but skipped"); + expect(block(r.stdout, "api-docs")).not.toContain("WIRED"); + }); + + test("the same copy under `generators` reads as skipped, and not as wired in the docs door", async () => { + const r = await meta(`["entity", ownedDocs]`, ["gen", "--list", "--format", "json"], { extra: OWNED_DOCS }); + const rows = rowsOf(r.stdout); + expect(row(rows, "docs").project?.wired).toBe(true); + expect(row(rows, "docs").project?.wiredInDocsDoor).toBeUndefined(); + }); + test("--probe: no file count for a generator `meta gen` does not run", async () => { // It printed `would emit 0`, which reads as "nothing in your model needs this" for // a generator whose pages `meta docs` writes for every entity. @@ -407,7 +500,6 @@ describe("meta gen --list — a row `meta gen` does not run says so", () => { expect(block(text.stdout, name), name).toContain("run by `meta docs`"); } expect(block(text.stdout, "entity")).toContain("would emit 1"); - expect(block(text.stdout, "mermaid-er")).toContain("would emit 1"); }); test("--probe --format json: wouldEmit is null beside the door, never 0", async () => { @@ -466,15 +558,19 @@ describe("meta verify — a doc generator under `generators` is an advisory", () expect(payload.help.join("\n")).not.toContain("refuses"); }); - // `meta gen` exits non-zero on these two strings (asserted in the mermaid-er block + // `meta gen` exits non-zero on these two strings (asserted in the refused-name block // above), so calling them skipped told the reader the run goes on without them. - test("json: a neutral name wired as a string is reported as refused, not skipped", async () => { + test("json: a neutral or removed name wired as a string is reported as refused, not skipped", async () => { const r = await meta(`["entity", "docs", "mermaid-er"]`, ["verify", "--format", "json"]); const payload = JSON.parse(r.stdout) as VerifyPayload; const rows = payload.antiPatterns.rows.filter((x) => x.rule === "doc-generator-refused-by-gen"); expect(rows).toHaveLength(2); expect(rows[0]?.message).toContain(`"docs" is listed under \`generators\`, and \`meta gen\` refuses it`); - expect(rows[1]?.message).toContain(`"mermaid-er" is listed under \`generators\`, and \`meta gen\` refuses it`); + // The removed name carries the message `meta gen` fails with, word for word. + expect(rows[1]?.message).toBe( + `metaobjects.config.ts: ${MERMAID_ER_REMOVED}, for the whole model on its overview page (README.md) ` + + `and around each entity on that entity's page. Remove "mermaid-er" from \`generators\` and run \`meta docs\`.`, + ); for (const x of rows) { expect(x.message).not.toContain("skips"); expect(x.message).toContain("Remove"); @@ -493,8 +589,40 @@ describe("meta verify — a doc generator under `generators` is an advisory", () expect(r.stderr).not.toContain("`meta gen` skips"); }); - test("a constructed mermaidErDiagram() gets no row: it still runs", async () => { - const r = await meta(`["entity", mermaidErDiagram()]`, ["verify", "--format", "json"]); + // An owned docs generator under `generators` is skipped like a packaged one, and the + // fix is the opposite. "Remove it" would tell its owner to stop using the copy they + // ejected; it goes in the docs block, where `meta docs` runs it. + test("an owned docs generator is told to MOVE to `docs.generators`, not to be removed", async () => { + const r = await meta(`["entity", ownedDocs, apiDocsFile()]`, ["verify", "--format", "json"], { extra: OWNED_DOCS }); + expect(r.code).toBe(0); + const payload = JSON.parse(r.stdout) as VerifyPayload; + const rows = payload.antiPatterns.rows.filter((x) => x.rule === "doc-generator-in-gen"); + expect(rows).toHaveLength(2); + expect(rows[0]?.message).toBe( + 'metaobjects.config.ts: the owned docs generator "docs-file" is listed under `generators`, and ' + + "`meta gen` skips it — `meta docs` runs an owned docs generator from `docs: { generators: [...] }`, " + + "and `meta verify --docs` checks what it writes. Move it there.", + ); + expect(rows[0]?.message).not.toContain("Remove"); + expect(rows[0]?.construct).toBe("meta docs"); + // The packaged one beside it keeps its own advice. + expect(rows[1]?.message).toContain("Remove apiDocsFile() from `generators`."); + expect(payload.exitCode).toBe(0); + }); + + test("the marker decides, not the name: a renamed owned copy is told the same", async () => { + const extra = `const myPages = { name: "my-pages", docsSurface: "api" as const, generate: () => [] };`; + const r = await meta(`["entity", myPages]`, ["verify", "--format", "text"], { extra }); + expect(r.code).toBe(0); + expect(r.stderr).toContain('the owned docs generator "my-pages" is listed under `generators`'); + expect(r.stderr).toContain("Move it there."); + }); + + test("an owned docs generator where it belongs gets no row", async () => { + const r = await meta(`["entity"]`, ["verify", "--format", "json"], { + extra: OWNED_DOCS, + docs: `{ generators: [ownedDocs] }`, + }); const payload = JSON.parse(r.stdout) as VerifyPayload; expect(payload.antiPatterns.rows.filter((x) => x.rule.startsWith("doc-generator-"))).toEqual([]); }); diff --git a/server/typescript/packages/cli/test/gen-list.test.ts b/server/typescript/packages/cli/test/gen-list.test.ts index 2383f1676..9f045bb97 100644 --- a/server/typescript/packages/cli/test/gen-list.test.ts +++ b/server/typescript/packages/cli/test/gen-list.test.ts @@ -61,15 +61,16 @@ describe("meta gen --list (ADR-0021 D3)", () => { } }); - test("flags docs/mermaid neutral (not part of the recommended native suite)", async () => { + test("flags docs neutral (not part of the recommended native suite)", async () => { const tmp = mkdtempSync(join(tmpdir(), "meta-gen-list-")); try { await genCommand(["--list"], tmp); const text = out(); - // They appear (identity/discoverability) ... + // It appears (identity/discoverability) ... expect(text).toContain("docs"); - expect(text).toContain("mermaid-er"); - // ... but are clearly marked neutral and point at `meta docs`. + // ... but is clearly marked neutral and points at `meta docs`. The standalone + // `mermaid-er` generator that sat beside it was removed in 1.1. + expect(text).not.toContain("mermaid-er"); expect(text.toLowerCase()).toContain("neutral"); expect(text).toContain("meta docs"); } finally { @@ -100,12 +101,18 @@ describe("meta gen --list (ADR-0021 D3)", () => { await genCommand(["--list"], tmp); const text = out(); const line = (name: string) => text.split("\n").find((l) => l.startsWith(` ${name} `)) ?? ""; - expect(line("shared-model")).toContain("package-only"); + // The one entry left: the Mustache primitive, where the adopter already supplies + // the template and there is no emit logic to own (ADR-0034 Amendment 7). expect(line("template")).toContain("package-only"); + expect(text.split("\n").filter((l) => l.includes("[package-only]")).length).toBe(1); // An ejectable entry is never marked. `callable` and `trace-helper` were marked - // until they gained reference templates. + // until they gained reference templates; `shared-model` and the docs tier until + // Amendment 7. expect(line("callable")).not.toContain("package-only"); expect(line("trace-helper")).not.toContain("package-only"); + expect(line("shared-model")).not.toContain("package-only"); + expect(line("docs")).not.toContain("package-only"); + expect(line("api-docs")).not.toContain("package-only"); expect(line("prompt-render")).not.toContain("package-only"); expect(line("entity")).not.toContain("package-only"); expect(text).toContain("package-only = no reference template ships"); diff --git a/server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts b/server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts new file mode 100644 index 000000000..a40e64896 --- /dev/null +++ b/server/typescript/packages/cli/test/integration/eject-docs-tier.test.ts @@ -0,0 +1,479 @@ +/** + * The docs tier, owned — end to end, through the BUILT CLI (ADR-0034 Amendment 7). + * + * `docs` and `api-docs` could not be ejected while `meta docs` constructed the packaged + * generators itself: there was nothing to hand an owned copy to. The seam is the docs + * door's own config, `docs: { generators: [...] }`, read by a marker on the generator + * (`docsSurface`). These tests drive it the way an adopter does — `meta eject`, + * `metaobjects.config.ts`, `meta docs`, `meta verify --docs`, `meta gen` — because the + * claim is about which command runs which copy, and no factory-level test can see that. + * + * The standalone `mermaid-er` generator that used to sit in this tier was removed in 1.1: + * `meta docs` draws the diagrams. The last describe pins what a project that still wires + * it is told, by name and by factory. + * + * Each gate is shown to FAIL as well as pass: an edited copy changes the page and the + * drift check convicts the committed tree. A seam that only ever agreed with the packaged + * generator would pass every equality here without running the copy at all. + * + * Subprocess-per-command for the reason `verify-codegen-ejected-generator` gives: the + * config is re-read under a fresh name each load but the modules it imports are cached. + */ +import { describe, test, expect, beforeAll } from "bun:test"; +import { cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { spawnSync } from "node:child_process"; +import { meta, requireFreshDist } from "./support/built-cli.js"; + +const FIXTURES = resolve(import.meta.dirname, "..", "fixtures"); +// Inside the workspace, so an ejected template's `@metaobjectsdev/codegen-ts` import +// resolves by walking up to the workspace root. +const WORKSPACE_TMP = join(FIXTURES, "__tmp__"); +const OUT = "generated"; +// The default `docs.outDir`: where `meta docs` writes and `verify --docs` reads. +const DOCS = join("docs", "generated"); + +/** + * @param docsGenerators what goes in `docs: { generators: [...] }`, the docs door's list. + * @param genGenerators appended to the top-level `generators`, the array `meta gen` runs. + */ +function config(imports: readonly string[], docsGenerators: string, genGenerators = ""): string { + return [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { entityFile, routesFile } from "@metaobjectsdev/codegen-ts/generators";`, + ...imports, + `export default defineConfig({`, + ` outDir: ${JSON.stringify(OUT)},`, + ` dialect: "sqlite",`, + ` dbImport: "~/db",`, + ` extStyle: "none",`, + ` apiPrefix: "/api",`, + ` generators: [entityFile(), routesFile()${genGenerators}],`, + ...(docsGenerators === "" ? [] : [` docs: { generators: [${docsGenerators}] },`]), + `});`, + ].join("\n"); +} + +/** Nothing docs-related listed: `meta docs` runs the packaged generators. */ +const PACKAGED = config([], ""); + +const OWNED_DOCS_IMPORT = `import { docsFile } from "./codegen/generators/docs.js";`; +const OWNED_API_IMPORT = `import { apiDocsFile } from "./codegen/generators/api-docs.js";`; +/** Both owned copies, listed exactly as `meta eject` prints. */ +const OWNED_BOTH = config([OWNED_DOCS_IMPORT, OWNED_API_IMPORT], "docsFile(), apiDocsFile()"); + +function setupRepo(configSource: string): string { + mkdirSync(WORKSPACE_TMP, { recursive: true }); + const root = mkdtempSync(join(WORKSPACE_TMP, "docs-tier-")); + cpSync(join(FIXTURES, "trainer-website-meta"), root, { recursive: true }); + writeFileSync(join(root, "metaobjects.config.ts"), configSource); + // Its own repository. `verify --docs` skips a page the project git-ignores, and this + // directory sits under an ignored path of the workspace it borrows `node_modules` from: + // without a repository of its own every page here would be exempt and the gate would + // have nothing to compare. + expect(spawnSync("git", ["init", "-q"], { cwd: root }).status).toBe(0); + return root; +} + +/** Every file under `/`, path → contents. Empty when the directory is absent. */ +function tree(root: string, dir: string): Map { + const out = new Map(); + const base = join(root, dir); + if (!existsSync(base)) return out; + for (const f of readdirSync(base, { recursive: true, withFileTypes: true })) { + if (!f.isFile()) continue; + const abs = join(f.parentPath, f.name); + out.set(abs.slice(base.length + 1), readFileSync(abs, "utf8")); + } + return out; +} + +function edit(root: string, rel: string, from: string, to: string): void { + const path = join(root, rel); + const before = readFileSync(path, "utf8"); + // An edit that matched nothing would leave the copy unmodified and every "the edited + // copy ran" assertion below would be testing the reference. + expect(before).toContain(from); + writeFileSync(path, before.replace(from, to)); +} + +// Precondition once per FILE, not per test: this gate runs the BUILT CLI. +beforeAll(requireFreshDist); + +describe("docs and api-docs — owned copies run by `meta docs`", () => { + test("unmodified owned copies write exactly the pages `meta docs` writes without them", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const packaged = tree(root, DOCS); + // Vacuity guard: both surfaces are in the tree being compared. + expect([...packaged.keys()]).toEqual(expect.arrayContaining(["README.md", "User.md", "api/README.md", "api/AGENT-API.md", "api/User.md"])); + + // Take ownership of both, changing nothing about them. + expect(await meta(root, "eject", "docs", "api-docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), OWNED_BOTH); + + // The committed pages were written by the PACKAGE's generators. If the owned copies + // would write anything else, this is where it shows. + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + + const owned = await meta(root, "docs"); + expect(owned).toMatchObject({ exit: 0 }); + // The summary names the copy, so "did mine run?" has an answer on the terminal. + expect(owned.output).toContain("from the owned generator 'docs-file'"); + expect(owned.output).toContain("from the owned generator 'api-docs'"); + expect(tree(root, DOCS)).toEqual(packaged); + + // And back: dropping the unmodified copies is equally a no-op. + writeFileSync(join(root, "metaobjects.config.ts"), PACKAGED); + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("an edited model-docs copy is the one that runs, and `verify --docs` convicts the stale pages", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + expect(tree(root, DOCS).get("README.md")).toStartWith("# Data Model\n"); + + expect(await meta(root, "eject", "docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "docsFile()")); + edit(root, "codegen/generators/docs.ts", 'out.push("# Data Model");', 'out.push("# Our Model");'); + + // The committed README still has the packaged title: the gate has to say so. + const stale = await meta(root, "verify", "--docs"); + expect(stale.exit).toBe(1); + expect(stale.output).toContain("README.md"); + + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const pages = tree(root, DOCS); + expect(pages.get("README.md")).toStartWith("# Our Model\n"); + // The api surface was not ejected: the packaged generator still writes it. + expect(pages.has("api/AGENT-API.md")).toBe(true); + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("an edited api-docs copy is the one that runs, with the model surface still packaged", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const before = tree(root, DOCS); + expect(before.has("api/AGENT-API.md")).toBe(true); + + expect(await meta(root, "eject", "api-docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_API_IMPORT], "apiDocsFile()")); + // Own a decision the packaged generator does not offer: no agent form. + edit( + root, + "codegen/generators/api-docs.ts", + " { path: agentFilename, content: renderAgentApi(model, provider) },\n", + "", + ); + + // `meta docs` only writes, so the agent form written before the edit would still be + // on disk. A fresh tree is what shows the copy ran: it no longer contains that file. + rmSync(join(root, DOCS), { recursive: true, force: true }); + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const fresh = tree(root, DOCS); + expect(fresh.has("api/AGENT-API.md")).toBe(false); + // Everything else is untouched, the packaged model surface included. + expect(fresh.get("api/User.md")).toBe(before.get("api/User.md")); + expect(fresh.get("README.md")).toBe(before.get("README.md")); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("an owned copy that fails to load stops `meta docs`: the pages are not rewritten by the packaged generator", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "docsFile()")); + edit(root, "codegen/generators/docs.ts", 'out.push("# Data Model");', 'out.push("# Our Model");'); + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const owned = tree(root, DOCS); + expect(owned.get("README.md")).toStartWith("# Our Model\n"); + + // An error in the copy: the config imports it, so the config no longer loads. + const copy = join(root, "codegen", "generators", "docs.ts"); + writeFileSync(copy, `${readFileSync(copy, "utf8")}\nthrow new Error("owned docs copy is broken");\n`); + + // `meta gen` is the reference for this failure: exit 2 and the load error. + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).toBe(2); + expect(gen.output).toContain("owned docs copy is broken"); + + const docs = await meta(root, "docs"); + expect(docs.exit).toBe(2); + expect(docs.output).toContain("owned docs copy is broken"); + // The packaged generator would have put "# Data Model" back. + expect(tree(root, DOCS)).toEqual(owned); + expect((await meta(root, "verify", "--docs")).exit).not.toBe(0); + + // And with no pages on disk, none are written. + rmSync(join(root, DOCS), { recursive: true, force: true }); + expect((await meta(root, "docs")).exit).toBe(2); + expect(existsSync(join(root, DOCS))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("a docs generator reads the project's own files through `ctx.inputRoot` under `meta docs`", async () => { + // Hand-written, as the guide has an owned generator read a project file: + // `join(ctx.inputRoot, )`. + const root = setupRepo( + config([`import { titlePage } from "./codegen/generators/title-page.js";`], "titlePage()"), + ); + try { + mkdirSync(join(root, "codegen", "generators"), { recursive: true }); + writeFileSync( + join(root, "codegen", "generators", "title-page.ts"), + [ + `import { readFileSync } from "node:fs";`, + `import { join } from "node:path";`, + `import type { Generator } from "@metaobjectsdev/codegen-ts";`, + `export const titlePage = (): Generator => ({`, + ` name: "title-page",`, + ` docsSurface: "model",`, + ` generate: (ctx) => [`, + ` { path: "README.md", content: "# " + readFileSync(join(ctx.inputRoot as string, "MODEL-TITLE"), "utf8") + "\\n" },`, + ` ],`, + `});`, + ].join("\n"), + ); + + // The file is not there yet. The error is the generator's own, and is not rewritten + // into advice about the packaged page templates. + const missing = await meta(root, "docs"); + expect(missing.exit).toBe(1); + expect(missing.output).toContain("ENOENT"); + expect(missing.output).toContain("MODEL-TITLE"); + expect(missing.output).not.toContain("templates not found"); + expect(existsSync(join(root, DOCS))).toBe(false); + + writeFileSync(join(root, "MODEL-TITLE"), "From The Project"); + + // `--templates` repoints `ctx.projectRoot`. The project's files are not there. + mkdirSync(join(root, "elsewhere"), { recursive: true }); + expect(await meta(root, "docs", "--templates", "elsewhere")).toMatchObject({ exit: 0 }); + expect(tree(root, DOCS).get("README.md")).toBe("# From The Project\n"); + expect(await meta(root, "verify", "--docs")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("`meta gen` is untouched by listed docs generators, and points a misplaced one at the docs door", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs", "api-docs")).toMatchObject({ exit: 0 }); + writeFileSync(join(root, "metaobjects.config.ts"), OWNED_BOTH); + + // Listed where they belong, the copies are none of `meta gen`'s business: it runs + // what is in `generators`, writes no page, and has nothing to say about docs. + const gen = await meta(root, "gen", "--format", "text"); + expect(gen).toMatchObject({ exit: 0 }); + expect(gen.output).not.toContain("docs generator"); + expect(gen.output).not.toContain("docs are produced by"); + const generated = [...tree(root, OUT).keys()]; + expect(generated).toContain("User.ts"); + expect(generated.filter((p) => p.endsWith(".md"))).toEqual([]); + expect(existsSync(join(root, DOCS))).toBe(false); + // The codegen drift gate regenerates through the same runner, so it is unaffected. + expect(await meta(root, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + // The same copy in the array `meta gen` runs is in the wrong place. It is skipped, + // and the warning names where it goes instead of leaving `meta docs` quietly + // running the packaged generator. + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "", ", docsFile()")); + const misplaced = await meta(root, "gen", "--format", "text"); + expect(misplaced).toMatchObject({ exit: 0 }); + expect(misplaced.output).toContain( + "[docs-file] is a docs generator: 'meta docs' runs it, from `docs: { generators: [...] }`", + ); + expect([...tree(root, OUT).keys()].filter((p) => p.endsWith(".md"))).toEqual([]); + // And `meta docs` did not pick it up from there: the packaged generator ran. + const docs = await meta(root, "docs"); + expect(docs).toMatchObject({ exit: 0 }); + expect(docs.output).not.toContain("from the owned generator"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("a `docs.generators` list `meta docs` cannot run is refused by name, by `meta docs` and by `verify --docs`", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs")).toMatchObject({ exit: 0 }); + + // Two for one surface: each would write the same pages. + writeFileSync(join(root, "metaobjects.config.ts"), config([OWNED_DOCS_IMPORT], "docsFile(), docsFile()")); + const two = await meta(root, "docs"); + expect(two.exit).toBe(2); + expect(two.output).toContain('`docs.generators` lists two generators for the model surface ("docs-file" and "docs-file")'); + expect(existsSync(join(root, DOCS))).toBe(false); + expect((await meta(root, "verify", "--docs")).exit).not.toBe(0); + + // No marker: a generator `meta gen` runs, listed in the docs block by mistake. The + // door has no surface to give it, so it neither runs it nor drops it. + writeFileSync( + join(root, "metaobjects.config.ts"), + config([OWNED_DOCS_IMPORT], "docsFile(), entityFile()"), + ); + const unmarked = await meta(root, "docs"); + expect(unmarked.exit).toBe(2); + expect(unmarked.output).toContain('`docs.generators` lists "entity-file", which carries no `docsSurface`'); + expect(existsSync(join(root, DOCS))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("`meta eject --list` and `meta gen --list` treat the tier like every other ejectable generator", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "eject", "docs", "api-docs")).toMatchObject({ exit: 0 }); + edit(root, "codegen/generators/docs.ts", 'out.push("# Data Model");', 'out.push("# Our Model");'); + + const list = await meta(root, "eject", "--list", "--format", "json"); + expect(list.exit).toBe(0); + const rows = (JSON.parse(list.output) as { generators: Array<{ name: string; status: string }> }).generators; + const status = (name: string) => rows.find((r) => r.name === name)?.status; + expect(status("api-docs")).toBe("identical"); + expect(status("docs")).toBe("differs"); + expect(status("mermaid-er")).toBeUndefined(); + + const catalog = await meta(root, "gen", "--list", "--format", "json"); + expect(catalog.exit).toBe(0); + const catalogRows = JSON.parse(catalog.output) as Array<{ + name: string; + source: { kind: string; ejectable: boolean; owned: boolean | null }; + }>; + for (const name of ["docs", "api-docs"]) { + const row = catalogRows.find((r) => r.name === name); + expect({ name, source: row?.source }).toEqual({ + name, + source: expect.objectContaining({ kind: "reference-template", ejectable: true, owned: true }), + }); + } + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); +}); + +describe("mermaid-er — removed; `meta docs` draws the diagrams", () => { + const REMOVED_FACTORY = [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { entityFile, mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: [entityFile(), mermaidErDiagram()] });`, + ].join("\n"); + const REMOVED_NAME = [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: ["entity", "mermaid-er"] });`, + ].join("\n"); + /** The removed import one module away from the config: the shape an owned generator has. */ + const VIA_OWNED_MODULE = [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { diagram } from "./codegen/generators/diagram.js";`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: [diagram()] });`, + ].join("\n"); + + test("a config that still calls the removed factory is told what replaced it, and nothing is written", async () => { + const root = setupRepo(REMOVED_FACTORY); + try { + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).not.toBe(0); + expect(gen.output).toContain('mermaidErDiagram is no longer exported from "@metaobjectsdev/codegen-ts/generators": 1.1 removed that generator'); + // True of this project: the import is in the config itself. + expect(gen.output).toContain("in your metaobjects.config.ts or in a module the config imports"); + expect(gen.output).toContain("`meta docs` draws the ER diagram"); + expect(gen.output).toContain("delete the import and the `mermaidErDiagram()` entry"); + expect(existsSync(join(root, OUT))).toBe(false); + + // `meta docs` reads the same config, so it stops on it too and writes no page. + const docs = await meta(root, "docs"); + expect(docs.exit).toBe(2); + expect(docs.output).toContain("mermaidErDiagram is no longer exported"); + expect(existsSync(join(root, DOCS))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("a commented-out import of the removed factory is not an import: the config loads", async () => { + const root = setupRepo( + [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { entityFile } from "@metaobjectsdev/codegen-ts/generators";`, + `// import { mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + `/* import { mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators"; */`, + `export default defineConfig({ outDir: ${JSON.stringify(OUT)}, dialect: "sqlite", generators: [entityFile()] });`, + ].join("\n"), + ); + try { + const gen = await meta(root, "gen", "--format", "text"); + expect(gen).toMatchObject({ exit: 0 }); + expect(gen.output).not.toContain("mermaidErDiagram"); + expect([...tree(root, OUT).keys()]).toContain("User.ts"); + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + expect(tree(root, DOCS).has("README.md")).toBe(true); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("a config that still names \"mermaid-er\" is told the same, not \"unknown generator\"", async () => { + const root = setupRepo(REMOVED_NAME); + try { + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).not.toBe(0); + expect(gen.output).toContain('generator "mermaid-er" was removed in 1.1'); + expect(gen.output).toContain("`meta docs` draws the ER diagram"); + expect(gen.output).not.toContain("unknown generator"); + expect(existsSync(join(root, OUT))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("the removed import in a module the config imports is diagnosed too", async () => { + const root = setupRepo(VIA_OWNED_MODULE); + try { + mkdirSync(join(root, "codegen", "generators"), { recursive: true }); + writeFileSync( + join(root, "codegen", "generators", "diagram.ts"), + [ + `import { mermaidErDiagram } from "@metaobjectsdev/codegen-ts/generators";`, + `export const diagram = () => mermaidErDiagram({ outFile: "docs/erd.md" });`, + ].join("\n"), + ); + const gen = await meta(root, "gen", "--format", "text"); + expect(gen.exit).not.toBe(0); + expect(gen.output).toContain('mermaidErDiagram is no longer exported from "@metaobjectsdev/codegen-ts/generators"'); + expect(gen.output).toContain("`meta docs` draws the ER diagram"); + expect(gen.output).toContain("codegen/generators/"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); + + test("and `meta docs` does draw them: the whole model on the overview page, a neighbourhood on an entity's page", async () => { + const root = setupRepo(PACKAGED); + try { + expect(await meta(root, "docs")).toMatchObject({ exit: 0 }); + const pages = tree(root, DOCS); + expect(pages.get("README.md")).toContain("## Diagram\n\n```mermaid\nerDiagram\n"); + expect(pages.get("README.md")).toContain("User ||--o{ Post"); + expect(pages.get("Post.md")).toContain("```mermaid\nflowchart TB\n"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 120_000); +}); diff --git a/server/typescript/packages/cli/test/integration/eject-shared-model.test.ts b/server/typescript/packages/cli/test/integration/eject-shared-model.test.ts new file mode 100644 index 000000000..004e856ee --- /dev/null +++ b/server/typescript/packages/cli/test/integration/eject-shared-model.test.ts @@ -0,0 +1,301 @@ +/** + * An OWNED shared-model generator still publishes an artifact a consumer's `meta deps sync` + * verifies — end to end, through the BUILT CLI (ADR-0034 Amendment 7). + * + * `shared-model` was the one generator ruled package-only for a reason of substance: its + * output is not a helper, it is a contract. A consumer hashes the artifact, checks the hash + * against the manifest, and loads the document with its own vocabulary. So it was made + * ejectable the way `cube-model` is split: the copy an adopter owns decides which source + * files may contribute and which version is published, and everything a consumer checks + * stays in the package (`buildSharedModel`, `renderSharedModelFiles`). + * + * The byte-identity gate in codegen-ts proves the copy writes the same two files. This file + * proves the claim that matters to the OTHER project: published by an ejected copy, the + * artifact is the one the dependency corpus pins, and a real consumer syncs it, locks it and + * passes `meta verify --deps` against it. Then the gate is shown to convict: an artifact + * changed after it was published is refused, so the green run above was a real check. + * + * Subprocess-per-command for the reason `verify-codegen-ejected-generator` gives: the + * config is re-read under a fresh name each load but the modules it imports are cached. + */ +import { describe, test, expect, beforeAll } from "bun:test"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { sha256Integrity } from "@metaobjectsdev/sdk"; +import { meta, requireFreshDist } from "./support/built-cli.js"; + +const FIXTURES = resolve(import.meta.dirname, "..", "fixtures"); +// Inside the workspace, so an ejected template's `@metaobjectsdev/codegen-ts` import +// resolves by walking up to the workspace root. +const WORKSPACE_TMP = join(FIXTURES, "__tmp__"); + +const DEP_NAME = "acme-common"; +const ARTIFACT = `${DEP_NAME}.metaobjects.json`; +const MANIFEST = "metaobjects.pkg.json"; +const PUBLISHED = "published"; + +// The artifact every port's dependency corpus pins for this model +// (fixtures/dependency-conformance/README.md), read as raw bytes. +const PINNED_ARTIFACT = readFileSync( + resolve(import.meta.dirname, "../../../../../../fixtures/dependency-conformance/artifacts/acme-common-v1.json"), +); + +/** The publisher's model: the three nodes of the pinned artifact, over two files. */ +const PUBLISHER_BASE = { + "metadata.root": { + package: "acme::common", + children: [ + { + "object.value": { + name: "Address", + children: [{ "field.string": { name: "city" } }, { "field.string": { name: "street" } }], + }, + }, + { + "object.entity": { + name: "Audited", + abstract: true, + children: [{ "field.timestamp": { name: "createdAt" } }], + }, + }, + ], + }, +}; +const PUBLISHER_CUSTOMER = { + "metadata.root": { + package: "acme::common", + children: [ + { + "object.entity": { + name: "Customer", + children: [ + { "source.rdb": { "@table": "customers" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "email", "@maxLength": 120 } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, +}; + +/** The consumer's own model, so the post-sync load has something of its own to combine. */ +const CONSUMER_APP = { + "metadata.root": { + package: "app", + children: [ + { + "object.entity": { + name: "Order", + children: [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, +}; + +function publisherConfig(importFrom: string): string { + return [ + `import { defineConfig } from "@metaobjectsdev/codegen-ts";`, + `import { sharedModelFile } from ${JSON.stringify(importFrom)};`, + `export default defineConfig({`, + ` outDir: ${JSON.stringify(PUBLISHED)},`, + ` dialect: "sqlite",`, + ` generators: [sharedModelFile({ name: ${JSON.stringify(DEP_NAME)}, include: ["acme::common::**"], version: "1.0.0" })],`, + `});`, + ].join("\n"); +} + +const PACKAGED = publisherConfig("@metaobjectsdev/codegen-ts/generators"); +const OWNED = publisherConfig("./codegen/generators/shared-model.js"); + +/** A publisher and a consumer, siblings under one root, the consumer depending on what + * the publisher's `meta gen` writes. */ +function setupProjects(): { root: string; publisher: string; consumer: string } { + mkdirSync(WORKSPACE_TMP, { recursive: true }); + const root = mkdtempSync(join(WORKSPACE_TMP, "shared-model-")); + + const publisher = join(root, "publisher"); + mkdirSync(join(publisher, "metaobjects"), { recursive: true }); + writeFileSync(join(publisher, "metaobjects", "meta.common-base.json"), JSON.stringify(PUBLISHER_BASE, null, 2)); + writeFileSync(join(publisher, "metaobjects", "meta.common-customer.json"), JSON.stringify(PUBLISHER_CUSTOMER, null, 2)); + writeFileSync(join(publisher, "metaobjects.config.ts"), PACKAGED); + + const consumer = join(root, "consumer"); + mkdirSync(join(consumer, "metaobjects"), { recursive: true }); + writeFileSync(join(consumer, "metaobjects", "meta.app.json"), JSON.stringify(CONSUMER_APP)); + mkdirSync(join(consumer, ".metaobjects"), { recursive: true }); + writeFileSync( + join(consumer, ".metaobjects", "config.json"), + JSON.stringify({ + schema_version: 1, + sources: [], + dependencies: [{ name: DEP_NAME, path: `../publisher/${PUBLISHED}` }], + }), + ); + return { root, publisher, consumer }; +} + +function published(publisher: string): { artifact: Buffer; manifest: string } { + return { + artifact: readFileSync(join(publisher, PUBLISHED, ARTIFACT)), + manifest: readFileSync(join(publisher, PUBLISHED, MANIFEST), "utf8"), + }; +} + +// Precondition once per FILE, not per test: this gate runs the BUILT CLI. +beforeAll(requireFreshDist); + +describe("shared-model — published by an owned copy, verified by a consumer", () => { + test("an ejected copy publishes the pinned artifact, and `meta deps sync` locks and verifies it", async () => { + const { root, publisher, consumer } = setupProjects(); + try { + // As shipped, first: what the package's generator publishes. + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + const packaged = published(publisher); + expect(packaged.artifact.equals(PINNED_ARTIFACT)).toBe(true); + + // Take ownership, changing nothing, and wire the copy as `meta eject` prints. + const eject = await meta(publisher, "eject", "shared-model", "--format", "text"); + expect(eject).toMatchObject({ exit: 0 }); + expect(eject.output).toContain('import { sharedModelFile } from "./codegen/generators/shared-model.js";'); + // The entry names the options the factory cannot be called without. + expect(eject.output).toContain("If sharedModelFile({ name, include }) is not in `generators` yet"); + writeFileSync(join(publisher, "metaobjects.config.ts"), OWNED); + + // The committed artifact was written by the PACKAGE's generator. The codegen drift + // gate runs the owned copy and finds nothing to change. + expect(await meta(publisher, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + // Published from scratch by the owned copy: the same two files, to the byte. + rmSync(join(publisher, PUBLISHED), { recursive: true, force: true }); + rmSync(join(publisher, ".metaobjects"), { recursive: true, force: true }); + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + const owned = published(publisher); + expect(owned.artifact.equals(PINNED_ARTIFACT)).toBe(true); + expect(owned.manifest).toBe(packaged.manifest); + // The manifest's hash is the hash of the bytes beside it, in the format sdk verifies. + expect((JSON.parse(owned.manifest) as { integrity: string }).integrity).toBe(sha256Integrity(owned.artifact)); + + // The consumer's side of the contract, against what the owned copy published. + const sync = await meta(consumer, "deps", "sync"); + expect(sync).toMatchObject({ exit: 0 }); + const snapshot = join(consumer, ".metaobjects", "deps", DEP_NAME, ARTIFACT); + expect(readFileSync(snapshot).equals(PINNED_ARTIFACT)).toBe(true); + const lock = JSON.parse(readFileSync(join(consumer, ".metaobjects", "deps.lock.json"), "utf8")) as { + dependencies: Record; + }; + expect(lock.dependencies[DEP_NAME]).toMatchObject({ + version: "1.0.0", + integrity: sha256Integrity(PINNED_ARTIFACT), + nodes: ["acme::common::Address", "acme::common::Audited", "acme::common::Customer"], + }); + expect(await meta(consumer, "deps", "check")).toMatchObject({ exit: 0 }); + expect(await meta(consumer, "verify", "--deps")).toMatchObject({ exit: 0 }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 240_000); + + test("the consumer's check is real: an artifact changed after it was published is refused", async () => { + const { root, publisher, consumer } = setupProjects(); + try { + expect(await meta(publisher, "eject", "shared-model")).toMatchObject({ exit: 0 }); + writeFileSync(join(publisher, "metaobjects.config.ts"), OWNED); + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + + // What an owned copy that wrote its own bytes after hashing would produce: a document + // that no longer matches the manifest beside it. + const artifact = join(publisher, PUBLISHED, ARTIFACT); + writeFileSync(artifact, readFileSync(artifact, "utf8").replace('"city"', '"town"')); + + const sync = await meta(consumer, "deps", "sync"); + expect(sync.exit).not.toBe(0); + expect(sync.output).toContain("but the manifest records"); + expect(existsSync(join(consumer, ".metaobjects", "deps.lock.json"))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + // `meta verify --codegen` could not run this generator at all, packaged or owned: it + // regenerated without the project's source files ("no source files to load") and read + // the version default from the temp tree it writes into. Both are what `meta gen` gives + // the generator, so the gate now gives them too. + test("the codegen drift gate runs the packaged generator as `meta gen` does: the project's files and its package.json version", async () => { + const { root, publisher } = setupProjects(); + try { + // No `version` option: the default is the nearest package.json, which is the project's. + writeFileSync(join(publisher, "metaobjects.config.ts"), PACKAGED.replace(', version: "1.0.0"', "")); + writeFileSync(join(publisher, "package.json"), JSON.stringify({ name: "publisher", version: "2.0.0" })); + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + expect((JSON.parse(published(publisher).manifest) as { version: string }).version).toBe("2.0.0"); + + expect(await meta(publisher, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + // And it convicts: the model moved, the published artifact did not. + const customer = join(publisher, "metaobjects", "meta.common-customer.json"); + writeFileSync(customer, readFileSync(customer, "utf8").replace('"@maxLength": 120', '"@maxLength": 200')); + const stale = await meta(publisher, "verify", "--codegen"); + expect(stale.exit).toBe(1); + expect(stale.output).toContain(ARTIFACT); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); + + test("what the copy owns is honoured: its version source, without touching what a consumer checks", async () => { + const { root, publisher, consumer } = setupProjects(); + try { + expect(await meta(publisher, "eject", "shared-model")).toMatchObject({ exit: 0 }); + // No `version` option, and the copy's own lookup replaced: this project publishes + // the version it keeps in a file of its own rather than in a package.json. + writeFileSync( + join(publisher, "metaobjects.config.ts"), + OWNED.replace(', version: "1.0.0"', ""), + ); + writeFileSync(join(publisher, "VERSION"), "7.8.9\n"); + const copy = join(publisher, "codegen", "generators", "shared-model.ts"); + const source = readFileSync(copy, "utf8"); + const lookup = "const version = opts.version ?? nearestPackageVersion(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd());"; + expect(source).toContain(lookup); + writeFileSync( + copy, + source.replace( + lookup, + 'const version = opts.version ?? readFileSync(join(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd(), "VERSION"), "utf-8").trim();', + ), + ); + + expect(await meta(publisher, "gen")).toMatchObject({ exit: 0 }); + const owned = published(publisher); + expect((JSON.parse(owned.manifest) as { version: string }).version).toBe("7.8.9"); + // The artifact is the pinned one still: the version is the manifest's, not the document's. + expect(owned.artifact.equals(PINNED_ARTIFACT)).toBe(true); + + // The drift gate regenerates in a temp tree, and `ctx.inputRoot` is still the PROJECT, + // so the copy's own `VERSION` lookup resolves there too: clean now, and convicted when + // the file moves on. + expect(await meta(publisher, "verify", "--codegen")).toMatchObject({ exit: 0 }); + + expect(await meta(consumer, "deps", "sync")).toMatchObject({ exit: 0 }); + expect(await meta(consumer, "verify", "--deps")).toMatchObject({ exit: 0 }); + const lock = JSON.parse(readFileSync(join(consumer, ".metaobjects", "deps.lock.json"), "utf8")) as { + dependencies: Record; + }; + expect(lock.dependencies[DEP_NAME]?.version).toBe("7.8.9"); + + writeFileSync(join(publisher, "VERSION"), "7.9.0\n"); + const stale = await meta(publisher, "verify", "--codegen"); + expect(stale.exit).toBe(1); + expect(stale.output).toContain(MANIFEST); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }, 180_000); +}); diff --git a/server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts b/server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts new file mode 100644 index 000000000..9274c1a96 --- /dev/null +++ b/server/typescript/packages/cli/test/unit/removed-generator-diagnostic.test.ts @@ -0,0 +1,56 @@ +// A removed generator factory reaches a project as a failed import, and the config load turns +// that failure into a diagnostic. +// +// `removedGeneratorRuntimeError` recovers the answer from the THROW, so it covers a live +// import wherever it is: in `metaobjects.config.ts` itself, or in a module the config imports +// (an owned generator under `codegen/generators/` is the usual one). +// +// Without it the removal reaches an adopter as whatever the runtime says about a missing +// export, which names neither MetaObjects, nor the removal, nor what to do. +// +// The one removed factory today is `mermaidErDiagram` (1.1): `meta docs` draws the diagram. + +import { describe, test, expect } from "bun:test"; +import { removedGeneratorRuntimeError } from "../../src/lib/load-metaobjects-config.js"; + +describe("removedGeneratorRuntimeError", () => { + test("recognises a transpiled config's shape", () => { + const msg = removedGeneratorRuntimeError( + new TypeError("(0 , _generators.mermaidErDiagram) is not a function"), + ); + expect(msg).toContain('mermaidErDiagram is no longer exported from "@metaobjectsdev/codegen-ts/generators"'); + expect(msg).toContain("`meta docs` draws the ER diagram"); + }); + + test("recognises native ESM's two spellings of a missing export", () => { + // Matching is on the SYMBOL and the fact of a missing export, not on one message + // shape: the shape belongs to whichever runtime loaded the module. + for (const message of [ + "The requested module '@metaobjectsdev/codegen-ts/generators' does not provide an export named 'mermaidErDiagram'", + "Export named 'mermaidErDiagram' not found in module '/x/codegen-ts/dist/generators/index.js'.", + "mermaidErDiagram is not a function", + ]) { + expect(removedGeneratorRuntimeError(new SyntaxError(message))).toContain("1.1 removed that generator"); + } + }); + + test("names the remedy and both places the import can be, since the throw cannot name the file", () => { + const msg = removedGeneratorRuntimeError(new TypeError("mermaidErDiagram is not a function")); + expect(msg).toContain("in your metaobjects.config.ts or in a module the config imports"); + expect(msg).toContain("codegen/generators/"); + expect(msg).toContain("delete the import and the `mermaidErDiagram()` entry"); + expect(msg).toContain("run `meta docs`"); + // The file it used to write is the other thing an upgrade leaves behind. + expect(msg).toContain("docs/model.md"); + }); + + test("stays out of the way of unrelated errors", () => { + expect(removedGeneratorRuntimeError(new TypeError("foo is not a function"))).toBeUndefined(); + expect(removedGeneratorRuntimeError(new Error("mermaidErDiagram exploded"))).toBeUndefined(); + expect(removedGeneratorRuntimeError(undefined)).toBeUndefined(); + }); + + test("a non-Error throw does not crash the diagnostic", () => { + expect(removedGeneratorRuntimeError("mermaidErDiagram is not a function")).toContain("`meta docs`"); + }); +}); diff --git a/server/typescript/packages/codegen-ts/src/docs-door.ts b/server/typescript/packages/codegen-ts/src/docs-door.ts new file mode 100644 index 000000000..0187dee56 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/docs-door.ts @@ -0,0 +1,73 @@ +// The docs door's seam for an owned docs generator (ADR-0034 Amendment 7). +// +// `meta docs` is the one door for documentation (ADR-0021 D1, ADR-0025), and it used to +// construct the packaged `docsFile()` / `apiDocsFile()` itself, with options it computed +// from the `docs:` config. There was nothing an owned copy could be handed to, so the two +// generators could not be ejected. The seam has two halves: +// +// - WHICH generator runs. An owned copy is listed in the docs door's own config, +// `docs: { generators: [...] }`, and carries `docsSurface`. It replaces the packaged +// generator for that surface (`docsDoorGenerators`). It is not listed in the top-level +// `generators`: that array is what `meta gen` runs, and `meta gen` does not run docs. +// - WHAT the door tells it. The facts the door used to pass as factory options now ride +// on the context (`ctx.docsDoor`), because a listed generator is already constructed by +// the time the door sees it. They are the facts the two page sets must agree on (where +// the api pages are, whether the model pages exist), so the door states them once and +// both generators read the same answer. +import type { Generator, OwnedDocsSurface } from "./generator.js"; + +/** + * What the docs door tells the generator it runs for a surface. Present on `ctx.docsDoor` + * only under `meta docs` / `meta verify --docs`; absent on a `meta gen` run and on a direct + * `.generate(ctx)` call. + */ +export interface DocsDoorFacts { + /** For the MODEL surface: the api surfaces emitted alongside it, which every model page + * and the index link to. Absent: the model pages link to none. */ + readonly apiSurfaces?: ReadonlyArray<{ label: string; subDir: string; baseUrl?: string }>; + /** For the API surface: the directory under the docs root its pages go in. */ + readonly apiSubDir?: string; + /** For the API surface: whether the model surface is emitted alongside it, so each api + * page can link back to its model page. */ + readonly modelSurface?: boolean; +} + +/** + * The owned generator for each surface out of `docs.generators`, keyed by surface. A + * surface with no entry is run by the packaged generator. + * + * Refused rather than resolved by order or ignored, each naming the entry: + * - an entry with no `docsSurface`. The door runs one generator per surface and has no + * surface to give it, so running it or dropping it would both be a guess. The + * packaged `docsFile()` / `apiDocsFile()` carry no marker and need no listing; a + * generator `meta gen` runs belongs in the top-level `generators`. + * - two entries for one surface. Each would write the same pages, and picking one + * silently is the two-answers defect the single door exists to prevent. + */ +export function docsDoorGenerators( + generators: readonly Generator[], +): Partial> { + const bySurface: Partial> = {}; + for (const generator of generators) { + const surface = generator.docsSurface; + if (surface === undefined) { + throw new Error( + `\`docs.generators\` lists "${generator.name}", which carries no \`docsSurface\`: ` + + "`meta docs` runs one generator per surface (model, api) and cannot tell which one " + + "this replaces. An owned copy from `meta eject docs` or `meta eject api-docs` " + + "carries it. The packaged docs generators need no listing, and a generator " + + "`meta gen` runs belongs in the top-level `generators`.", + ); + } + const first = bySurface[surface]; + if (first !== undefined) { + throw new Error( + `\`docs.generators\` lists two generators for the ${surface} surface ` + + `("${first.name}" and "${generator.name}"): \`meta docs\` runs one per surface. ` + + "Keep one of them.", + ); + } + bySurface[surface] = generator; + } + return bySurface; +} diff --git a/server/typescript/packages/codegen-ts/src/generator-registry.ts b/server/typescript/packages/codegen-ts/src/generator-registry.ts index b6d9b7cf4..d2e0aa01a 100644 --- a/server/typescript/packages/codegen-ts/src/generator-registry.ts +++ b/server/typescript/packages/codegen-ts/src/generator-registry.ts @@ -19,12 +19,14 @@ // // Tiering (ADR-0020 / ADR-0021 D1): // - "native" — the recommended Tier-1 `meta gen` suite (idiomatic emission). -// - "neutral" — Tier-2 artifacts owned by the neutral docs engine. `docs` and -// `mermaid-er` are present here for identity/discoverability but -// are NOT part of the recommended native surface — the canonical -// door for documentation is `meta docs` (D1). +// - "neutral" — Tier-2 artifacts owned by the neutral docs engine. `docs` is present +// here for identity/discoverability but is NOT part of the native +// surface — the door for documentation is `meta docs` (D1). (The +// standalone `mermaid-er` generator was a second neutral entry until +// 1.1 removed it: `meta docs` draws the diagram. See +// metaobjects-config.ts, `removedGeneratorMessage`.) -import type { Generator } from "./generator.js"; +import type { Generator, OwnedDocsSurface } from "./generator.js"; import type { MetaobjectsGenConfig } from "./metaobjects-config.js"; import { REFERENCE_GENERATOR_NAMES } from "./reference-templates.js"; import { @@ -35,7 +37,6 @@ import { callableFile, routesFileHono, namesFile, - mermaidErDiagram, promptRender, outputParser, extractor, @@ -154,6 +155,16 @@ export interface GeneratorRegistryEntry { configKeys?: readonly (keyof MetaobjectsGenConfig)[]; /** True iff this package ships a `src/reference/.ts` for `meta eject`. */ ejectable: boolean; + /** + * The `meta docs` surface an OWNED copy of this generator takes over. Set for the two + * generators `meta docs` runs (`docs`, `api-docs`): their owned copy is listed in + * `docs: { generators: [...] }`, the docs door's own block, and `meta eject` prints that + * wiring for them. Absent for every generator `meta gen` runs. + * + * Resolved, not trusted: a gate requires the reference template's generator to carry + * this same value as its `docsSurface` marker, which is what `meta docs` reads. + */ + docsSurface?: OwnedDocsSurface; /** Optional note — used to point neutral entries at their canonical door. */ note?: string; } @@ -284,6 +295,7 @@ export const generatorRegistry: Record = { factoryExport: "apiDocsFile", options: "filter?, target?", ejectable: ejectable("api-docs"), + docsSurface: "api", }, docs: { name: "docs", @@ -294,19 +306,9 @@ export const generatorRegistry: Record = { factory: () => docsFile(), factoryExport: "docsFile", ejectable: ejectable("docs"), + docsSurface: "model", note: "neutral artifact — use `meta docs` (the single docs door, ADR-0021 D1); not part of the recommended `meta gen` native suite.", }, - "mermaid-er": { - name: "mermaid-er", - kind: "generator", - layer: "docs", - description: "Mermaid ER diagram of the entity/relationship model.", - tier: "neutral", - factory: () => mermaidErDiagram(), - factoryExport: "mermaidErDiagram", - ejectable: ejectable("mermaid-er"), - note: "neutral artifact owned by the docs engine (ADR-0020), deprecated under `meta gen`: do not add it to a new config. A `mermaidErDiagram()` already wired in `generators` still runs; `meta docs` draws the same diagram on its overview page.", - }, // ----- capability ------------------------------------------------------- // Not chosen by browsing this list — chosen because the MODEL already asks for diff --git a/server/typescript/packages/codegen-ts/src/generator.ts b/server/typescript/packages/codegen-ts/src/generator.ts index b91fff542..b3fc716e8 100644 --- a/server/typescript/packages/codegen-ts/src/generator.ts +++ b/server/typescript/packages/codegen-ts/src/generator.ts @@ -2,6 +2,7 @@ import type { ExposeOption } from "./routes-expose.js"; import type { MetaObject, MetaRoot, TypeRegistry } from "@metaobjectsdev/metadata"; import type { LibraryManifest } from "@metaobjectsdev/metadata/library"; import type { RenderContext } from "./render-context.js"; +import type { DocsDoorFacts } from "./docs-door.js"; import type { ResolvedGenConfig } from "./metaobjects-config.js"; import type { OrphanPolicy } from "./reconcile-orphans.js"; import { effectivePackage } from "./docs-paths.js"; @@ -87,9 +88,33 @@ export interface GenContext { * the opposite and is meaningful: the caller looked, and this project opted into none. */ libraries?: readonly LibraryManifest[]; + /** + * The real project directory, for a generator that READS the project's own files: the + * nearest `package.json`, a version file, anything it resolves against the project. Under + * `meta gen` it is `projectRoot`. Under `meta verify --codegen` the run writes into a + * throwaway tree, `projectRoot` is that tree (import paths between an output directory + * and the project are computed against it, so it has to mirror the layout), and this is + * the project the tree mirrors. Under `meta docs` and `meta verify --docs` it is the + * directory holding `metaobjects.config.ts`, which `--templates` does not move. Read files + * through this; compute output-relative paths against `projectRoot`. Undefined when the + * caller supplied neither root. + */ + inputRoot?: string; + /** + * What the docs door (`meta docs`, `meta verify --docs`) tells the generator it runs for a + * surface: where the api pages are, whether the model pages are emitted. Absent on a + * `meta gen` run and on a direct `.generate(ctx)` call, where a docs generator's own + * options decide instead. On the context rather than on `config` because it is a fact + * about the run's door, not something a project sets. See docs-door.ts. + */ + docsDoor?: DocsDoorFacts; warn: (msg: string) => void; } +/** The `meta docs` surfaces an owned generator can take over (`Generator.docsSurface`). The + * `requirements` and `agent` surfaces are not generators in the catalog and stay packaged. */ +export type OwnedDocsSurface = "model" | "api"; + export interface Generator { /** kebab-case identifier; surfaces in diagnostics + drift logs. */ name: string; @@ -121,6 +146,16 @@ export interface Generator { * Absent `expose` means every verb. A copy ejected before this marker existed needs * `meta eject routes --force` (or the same one-line addition) to be honoured. */ mountsRoutes?: { readonly expose?: ExposeOption }; + /** Marks a generator that `meta docs` runs for one of its surfaces IN PLACE OF the + * packaged one: `"model"` for the per-object pages and their index, `"api"` for the + * reference of the generated API. It is how an owned (ejected) docs generator is reached: + * list it in `docs: { generators: [...] }` in metaobjects.config.ts, and `meta docs` and + * `meta verify --docs` read it there by this marker (`docsDoorGenerators`). It does not + * go in the top-level `generators`: `meta gen` does not run documentation (ADR-0021 D1, + * ADR-0025), and skips a generator carrying this marker with a warning that says where + * it belongs. The packaged `docsFile()` / `apiDocsFile()` do not carry it: they are what + * `meta docs` runs when nothing is listed, and need no listing. */ + docsSurface?: OwnedDocsSurface; /** §A6 — marks the generator that emits the Names artifact. The runner * aggregates this across the suite into ResolvedGenConfig.includeNames, which the * entity generator reads to decide whether it may reference those constants. diff --git a/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts b/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts index 163e33b69..825408307 100644 --- a/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/api-docs-file.ts @@ -13,7 +13,7 @@ // • `README.md` — the consolidated human index (GitHub treats it as landing), // • `AGENT-API.md` — the token-frugal agent form. // -// Unlike `docs`/`mermaid-er` (Tier-2 neutral, owned by `meta docs`), `api-docs` +// Unlike `docs` (Tier-2 neutral, owned by `meta docs`), `api-docs` // is a NATIVE generator: it is REGISTERED in the generator registry (ADR-0022 // Part 3) — so it appears in `gen --list` and is selectable by its stable name // `api-docs`. It is NOT a `meta docs` mode. @@ -39,7 +39,7 @@ import { type DocPagePlacement, } from "../docs-paths.js"; import { projectProvider } from "../render-engine/framework-provider.js"; -import { buildApiModel } from "./api-model.js"; +import { buildApiModelForRun } from "./api-model.js"; import { renderEntityApiPage, renderApiIndex, @@ -82,31 +82,21 @@ export const apiDocsFile = function apiDocsFile(opts?: ApiDocsFileOpts): Generat const provider = projectProvider(ctx.projectRoot ?? process.cwd()); const layout = ctx.config.outputLayout ?? "flat"; - // Per-call output prefix. Default `docs/api`; `meta docs` passes `api`. - const apiDir = opts?.subDir ?? DEFAULT_API_DIR; + // Per-call output prefix. Default `docs/api`; `meta docs` passes `api`. Under the docs + // door the door's facts decide it and the model cross-link, not this factory's options: + // the model generator is told the same thing, and the two must agree (docs-door.ts). + const door = ctx.docsDoor; + const apiDir = (door !== undefined ? door.apiSubDir : opts?.subDir) ?? DEFAULT_API_DIR; + const modelSurface = door !== undefined ? door.modelSurface : opts?.modelSurface; const indexFilename = `${apiDir}/README.md`; const agentFilename = `${apiDir}/AGENT-API.md`; - // ONE ApiModel feeds every form (Task-1 builder; Task-2 renderers). The - // pkMap is reused from the run's renderContext when present (the real gen - // run always provides it) and derived otherwise. - // Auto-detect: document the OPT-IN Hono CRUD surface iff the Hono routes - // generator is actually in the run. The runner aggregates each generator's - // `emitsHonoRoutes` marker into ctx.config.includeHonoRoutes, so api-docs - // "just works" — it documents Hono exactly when `routesFileHono` is wired, - // and omits it (Fastify-only) otherwise. No explicit opt needed. - const model = buildApiModel(ctx.loadedRoot, { - loadedRoot: ctx.loadedRoot, - outputLayout: layout, - includeHonoRoutes: ctx.config.includeHonoRoutes ?? false, - // The verbs the routes generators actually mount (`expose`), so a page never lists a - // route the project does not serve. - ...(ctx.config.routesExpose !== undefined && { expose: ctx.config.routesExpose }), - // The project's prefix, because every route is mounted inside it. Without - // this the page documents `/authors` for routes served at `/api/authors`. - apiPrefix: ctx.renderContext?.apiPrefix ?? "", - ...(ctx.renderContext?.pkMap !== undefined && { pkMap: ctx.renderContext.pkMap }), - }); + // ONE ApiModel feeds every form (Task-1 builder; Task-2 renderers). The facts of the + // run it reads (layout, whether Hono routes are wired, the verbs the routes generators + // mount, `apiPrefix`, the pk map) are gathered by `buildApiModelForRun`, which the + // reference template calls too, so an owned copy never restates them and cannot fall + // behind on one. + const model = buildApiModelForRun(ctx); // Track (path, fqn) for the SAME hard collision backstop docsFile() uses — // two units that resolve to one path (flat, cross-package short-name clash) @@ -126,7 +116,7 @@ export const apiDocsFile = function apiDocsFile(opts?: ApiDocsFileOpts): Generat // `/`. The href is derived from the SAME // docPageOutputPath placement via surfaceCrossHref so it resolves in both // layouts. ABSENT otherwise. - const modelPageHref = opts?.modelSurface + const modelPageHref = modelSurface ? surfaceCrossHref(path, docPageOutputPath(layout, node)) : undefined; return { path, content: renderEntityApiPage(unit, provider, modelPageHref) }; diff --git a/server/typescript/packages/codegen-ts/src/generators/api-model.ts b/server/typescript/packages/codegen-ts/src/generators/api-model.ts index ce1c54f12..220f28d6b 100644 --- a/server/typescript/packages/codegen-ts/src/generators/api-model.ts +++ b/server/typescript/packages/codegen-ts/src/generators/api-model.ts @@ -144,6 +144,7 @@ import { generatableObjects } from "../source-detect.js"; import { effectivePackage } from "../docs-paths.js"; import { entityOutputPath, type OutputLayout } from "../import-path.js"; import type { RenderContext } from "../render-context.js"; +import type { GenContext } from "../generator.js"; import type { PkInfo } from "../pk-resolver.js"; import { modelFieldShapes, @@ -1258,3 +1259,31 @@ function templateExample(symbolBase: string, symbols: ApiSymbol[]): UnitExample } return { imports: importLines(picks), body }; } + +/** + * The API model for a RUN: {@link buildApiModel} with its context read off the run rather + * than restated by the caller. + * + * Which facts of a run the documented API depends on is this package's knowledge: the page + * layout, whether a Hono routes generator is wired (the runner aggregates each generator's + * `emitsHonoRoutes` marker into `ctx.config.includeHonoRoutes`, so the Hono surface is + * documented exactly when it is emitted), the verbs each routes generator mounts + * (`ctx.config.routesExpose`, from `routesFile({ expose })`), the project's `apiPrefix` + * (every route is mounted inside it, so without it a page documents `/authors` for routes + * served at `/api/authors`), and the run's primary-key map. The `api-docs` generator and its reference template both + * call this, so when the model learns to read another fact of the run, an owned copy reads + * it too on its next package upgrade instead of quietly documenting less than the packaged + * generator does. + */ +export function buildApiModelForRun(ctx: GenContext): ApiModel { + return buildApiModel(ctx.loadedRoot, { + loadedRoot: ctx.loadedRoot, + outputLayout: ctx.config.outputLayout ?? "flat", + includeHonoRoutes: ctx.config.includeHonoRoutes ?? false, + // The verbs the routes generators actually mount (`expose`), so a page never lists a + // route the project does not serve. + ...(ctx.config.routesExpose !== undefined && { expose: ctx.config.routesExpose }), + apiPrefix: ctx.renderContext?.apiPrefix ?? "", + ...(ctx.renderContext?.pkMap !== undefined && { pkMap: ctx.renderContext.pkMap }), + }); +} diff --git a/server/typescript/packages/codegen-ts/src/generators/docs-file.ts b/server/typescript/packages/codegen-ts/src/generators/docs-file.ts index 7336bc5d5..8cd0c08ea 100644 --- a/server/typescript/packages/codegen-ts/src/generators/docs-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/docs-file.ts @@ -20,8 +20,6 @@ import type { MetaObject, MetaRoot } from "@metaobjectsdev/metadata"; import { TYPE_TEMPLATE } from "@metaobjectsdev/metadata"; -import { render } from "@metaobjectsdev/render"; -import type { Provider } from "@metaobjectsdev/render"; import type { Generator, GeneratorFactory, EmittedFile } from "../generator.js"; import { docPageOutputPath, @@ -33,6 +31,7 @@ import { type DocPagePlacement, } from "../docs-paths.js"; import { projectProvider } from "../render-engine/framework-provider.js"; +import { renderDocPage } from "../render-engine/doc-page.js"; import { renderMermaidErBlock } from "../templates/mermaid-er.js"; import { buildEntityDocData } from "./docs-data-builder.js"; import { isReport, servedReport } from "../source-detect.js"; @@ -64,20 +63,6 @@ const TEMPLATE_REF = "docs/entity-page.md"; // sibling artifact, distinct from the entity page (Task 3). const TEMPLATE_PAGE_REF = "docs/template-page.md"; -/** Render one docs page, wrapping any engine error with the page ref + output - * path so a template failure points at the exact page (shared by the entity - * and template.output emission paths — identical error contract). */ -function renderDocPage(ref: string, payload: unknown, provider: Provider, path: string): string { - try { - return render({ ref, payload, provider, format: "markdown" }); - } catch (err) { - const msg = err instanceof Error ? err.message : String(err); - throw new Error(`docs-file: failed rendering '${ref}' for '${path}': ${msg}`, { - cause: err instanceof Error ? err : undefined, - }); - } -} - /** * @deprecated ADR-0021 D1: `meta docs` is the single door for documentation. * `docsFile()` stays as the INTERNAL engine that `meta docs` calls — do NOT add @@ -101,8 +86,14 @@ export const docsFile = function docsFile(opts?: DocsFileOpts): Generator { // baseUrl). The api page for a node lives at `/`, // so the from-path doubles as the page placement. ABSENT/empty surfaces ⇒ // undefined → output byte-identical to historical model-only runs. + // + // Under the docs door the door's facts decide which surfaces those are, not this + // factory's options: the api generator is told the same thing, and the two must agree + // (docs-door.ts). The options decide on a direct call, which has no door. + const door = ctx.docsDoor; + const apiSurfaces = door !== undefined ? door.apiSurfaces : opts?.apiSurfaces; const apiRefsFor = (fromPath: string): Array<{ label: string; href: string }> | undefined => - opts?.apiSurfaces?.map((s) => ({ label: s.label, href: apiSurfaceHref(fromPath, s, fromPath) })); + apiSurfaces?.map((s) => ({ label: s.label, href: apiSurfaceHref(fromPath, s, fromPath) })); // Track every (path, fqn) so we can hard-error on a collision (defense // against silent doc-page overwrite) AFTER all pages are placed. const placements: DocPagePlacement[] = []; diff --git a/server/typescript/packages/codegen-ts/src/generators/index.ts b/server/typescript/packages/codegen-ts/src/generators/index.ts index add750044..402e79796 100644 --- a/server/typescript/packages/codegen-ts/src/generators/index.ts +++ b/server/typescript/packages/codegen-ts/src/generators/index.ts @@ -14,7 +14,10 @@ // // Which entries have a reference copy is declared once, in `REFERENCE_GENERATOR_NAMES` // (reference-templates.ts); `meta gen --list` marks every other entry `package-only` — -// today that is `template`, `shared-model` and the docs tier. +// today that is `template` alone, the Mustache primitive, which has no emit logic of its +// own. `docsFile` and `apiDocsFile` are exported here as the generators `meta docs` runs; +// they need no wiring. An ejected copy of either is listed in `docs: { generators: [...] }`, +// and `meta docs` then runs the copy (Amendment 7). // See spec/decisions/ADR-0034-codegen-scaffold-and-own.md. export { entityFile, type EntityFileOpts } from "./entity-file.js"; @@ -25,8 +28,6 @@ export { barrel, type BarrelOpts } from "./barrel.js"; export { callableFile, type CallableFileOpts } from "./callable-file.js"; export { routesFileHono, type RoutesFileHonoOpts } from "./routes-file-hono.js"; export { namesFile } from "./names-file.js"; -/** @deprecated ADR-0021 D1 — neutral artifact owned by `meta docs` (ADR-0020); not part of the recommended `meta gen` suite. */ -export { mermaidErDiagram, type MermaidErOptions } from "./mermaid-er.js"; export { promptRender, type PromptRenderOpts } from "./prompt-render-file.js"; export { outputParser, type OutputParserOpts } from "./output-parser-file.js"; export { extractor, type ExtractorOpts } from "./extractor-file.js"; diff --git a/server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts b/server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts deleted file mode 100644 index 098ad7c93..000000000 --- a/server/typescript/packages/codegen-ts/src/generators/mermaid-er.ts +++ /dev/null @@ -1,43 +0,0 @@ -import { oncePerRun, type Generator, type GeneratorFactory } from "../generator.js"; -import { renderMermaidModel } from "../templates/mermaid-er.js"; - -export interface MermaidErOptions { - /** Output path relative to the target's outDir. Defaults to "docs/model.md". */ - outFile?: string; - /** Named output target. */ - target?: string; -} - -/** - * Emit a single Markdown file containing a Mermaid `erDiagram` plus per-entity - * prose sections. The renderer walks the loaded root for all entities; the - * default outFile is "docs/model.md". - * - * TIER NOTE (ADR-0020): the CANONICAL home of the neutral ER diagram is now the - * neutral docs engine — the OVERVIEW/index page (`README.md`) emitted by - * `docsFile()` / `meta docs` embeds the SAME `renderMermaidErBlock()` this - * generator's `renderMermaidModel()` reuses. This standalone Tier-1 generator is - * retained ONLY as a thin back-compat wrapper for adopters that already have - * `mermaidErDiagram` in their `meta gen` config; it adds NO ER logic of its own - * (no duplication) — it delegates to the shared `renderMermaidModel()` builder. - * - * @deprecated ADR-0021 D1: the ER diagram is a Tier-2 neutral artifact owned by - * the docs engine (ADR-0020). Use `meta docs` — it embeds the same ER block. - * This standalone `meta gen` generator is flagged neutral in the registry - * (`--list`) and is no longer part of the recommended native suite. Kept for - * back-compat; do not add to new gen configs. - */ -export const mermaidErDiagram = function mermaidErDiagram( - opts?: MermaidErOptions, -): Generator { - const outFile = opts?.outFile ?? "docs/model.md"; - const generator: Generator = { - name: "mermaid-er-diagram", - generate: oncePerRun((_entities, ctx) => ({ - path: outFile, - content: renderMermaidModel(ctx.loadedRoot), - })), - }; - if (opts?.target) generator.target = opts.target; - return generator; -} as GeneratorFactory; diff --git a/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts b/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts index 1bffd6fdb..e06616b76 100644 --- a/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts @@ -5,75 +5,21 @@ // it as ONE canonical-JSON `metadata.root` document plus a manifest — the artifact a // consumer's `meta deps sync` copies into its own repo (design §2.2, §3.2, §3.5). // -// The algorithm (design §4.3), in order: -// 1. Standalone load of `files` (default: the run's own source files, `ctx.sourceFiles`) -// with `ctx.registry` (the run's composed registry — the SAME vocabulary the -// project's own metadata loaded with), strict. -// 2. Select top-level nodes: never `requirement.*` / `template.*` (a publisher's -// ledger and prompts are not a consumer's shapes), and matching the compiled -// include/exclude scope. -// 3. Closure check: every `extends` target and every REF_BEARING_ATTR_NAMES value -// of a selected node (and of its own descendants) must resolve to a SELECTED -// node, else the selection is not self-contained and the build fails naming the -// (referrer, target) pairs. -// 4. Serialize the selection with `serializeSharedDocument` — raw own-layer form, -// `extends` preserved, sorted by resolution key. -// 5. Re-load the emitted document standalone with CORE providers only, strict — a -// consumer loads it with ITS providers, and a Phase 1 export must not need the -// publisher's non-core vocabulary. -// 6. Emit `.metaobjects.json` + `metaobjects.pkg.json` (design §3.2). -import { createHash } from "node:crypto"; +// This file only wires the run into the two stages that ARE the artifact, both in +// `src/shared-model/build-shared-model.ts`: `buildSharedModel` (load, select, closure, +// serialize, core-vocabulary re-load) and `renderSharedModelFiles` (the artifact, its +// hash and the manifest). What is decided here is what belongs to the publishing project: +// which of its source files may contribute and which version is published. The reference +// template an adopter ejects (`src/reference/shared-model.ts`) is this file against the +// package's public exports, held byte-identical by `test/reference-byte-identical.test.ts`. import { existsSync, readFileSync } from "node:fs"; import { dirname, join } from "node:path"; -import { - type MetaData, - type MetaRoot, - MetaDataLoader, - InMemoryStringSource, - compileScope, - matchesScope, - REF_BEARING_ATTR_NAMES, - resolveObjectRef, - serializeSharedDocument, - packageOfResolutionKey, - PACKAGE_SEPARATOR, - CHILD_REF_SEPARATOR, - TYPE_REQUIREMENT, - TYPE_TEMPLATE, - METAMODEL_VERSION, -} from "@metaobjectsdev/metadata"; -import { FileSource } from "@metaobjectsdev/metadata/core"; import { oncePerRun, type Generator, type GenContext, type EmittedFile } from "../generator.js"; -import { effectivePackage } from "../docs-paths.js"; +import { buildSharedModel, renderSharedModelFiles } from "../shared-model/build-shared-model.js"; -// The FR-023 manifest/artifact tool-file conventions (design §3.2, §4.3, "Hash -// format"). Their single canonical source is `sdk/src/dependencies.ts` -// (`MANIFEST_FILE`, `ARTIFACT_SUFFIX`, `INTEGRITY_PREFIX`) — codegen-ts cannot -// import sdk (the dependency runs the other way: `cli` depends on both, design -// §4.3's closing note), so these mirror those values locally. Keep in sync — -// exported (not merely local) so `cli`, which depends on both packages, can -// assert equality against `sdk`'s exports in a test; see -// `cli/test/shared-model-constants-parity.test.ts`. -export const MANIFEST_FILE = "metaobjects.pkg.json"; -export const ARTIFACT_SUFFIX = ".metaobjects.json"; -export const INTEGRITY_PREFIX = "sha256-"; -/** design §3.2 — the manifest schema's own version, distinct from `metamodelVersion`. */ -const MANIFEST_SCHEMA_VERSION = 1; - -/** - * Separator for `checkClosure`'s dedupe key. A resolution key never contains a NUL, - * so this cannot collide with either half it joins — unlike a printable separator - * (`->`, `→`), which would need a separate argument for why no legal FQN could ever - * contain it. Written as the `\0` ESCAPE, never a raw control byte: a raw 0x00 in - * this source file previously made `git` treat it as binary, which (a) makes - * `git diff`/GitHub show no content for the file at all, and (b) more seriously, - * makes `.githooks/pre-commit`'s public-repo-hygiene grep skip its added lines - * entirely (a binary file's diff is never text-enumerated) — silently defeating that - * guard for any future edit to this file. The escape produces the byte-identical - * runtime string; only the SOURCE FILE's byte content changes (readable text instead - * of an embedded control byte). - */ -const DEDUPE_KEY_SEPARATOR = "\0"; +// Re-exported from here because `generators/index.ts` has always served them from this +// module (as `SHARED_MODEL_*`) for the `cli` parity test against sdk's canonical copies. +export { MANIFEST_FILE, ARTIFACT_SUFFIX, INTEGRITY_PREFIX } from "../shared-model/build-shared-model.js"; export interface SharedModelFileOpts { /** The dependency key a consumer will `meta deps sync` under. Also the artifact's @@ -87,86 +33,12 @@ export interface SharedModelFileOpts { * own source files (`ctx.sourceFiles` — everything `files?` narrows). */ files?: readonly string[]; /** The published version. Default: the nearest `package.json` `version` walking - * up from `ctx.projectRoot`. */ + * up from the project (`ctx.inputRoot`). */ version?: string; /** Named output target, as on every other generator. */ target?: string; } -/** Everything before the first `CHILD_REF_SEPARATOR` that follows the LAST - * `PACKAGE_SEPARATOR` — the object part of a possibly dotted ref (ADR-0029's - * addressing model: package segments never fold onto a non-root child, so a - * `.` before the final `::`-segment can't occur). Mirrors the private - * `splitChildTail` in `naming-refs.ts` (not exported); `extends` and the - * origin `@from`/`@of`/`@via` attrs are the only REF_BEARING_ATTR_NAMES that - * can carry a dotted child tail — every other ref-bearing attr is already a - * bare object ref, and stripping a tail that isn't there is a no-op. */ -function refOwner(ref: string): string { - const lastSep = ref.lastIndexOf(PACKAGE_SEPARATOR); - const segStart = lastSep === -1 ? 0 : lastSep + PACKAGE_SEPARATOR.length; - const dotInSeg = ref.indexOf(CHILD_REF_SEPARATOR, segStart); - return dotInSeg === -1 ? ref : ref.slice(0, dotInSeg); -} - -/** Depth-first walk of a node and its OWN descendants, node itself first. ADR-0039 - * sanctioned own-only case: `serializeSharedDocument` emits each selected node's - * OWN layer only (`extends` preserved, not flattened — design §3.5), so the - * closure check must walk exactly what will be emitted, never what `extends` - * would additionally contribute (that is the base's own layer, checked - * separately if the base is itself selected). */ -function walkOwn(node: MetaData): MetaData[] { - const out: MetaData[] = [node]; - for (const child of node.ownChildren()) out.push(...walkOwn(child)); - return out; -} - -interface ClosurePair { - readonly referrer: string; - readonly target: string; -} - -/** design §4.3 step 3. `standaloneRoot` already loaded successfully (step 1), so - * every ref found here is guaranteed to RESOLVE to some node in it — the loader's - * own validation passes reject a dangling `extends`/object-ref before this ever - * runs. What is NOT guaranteed is that the resolved node is in `selectedKeys`. */ -function checkClosure( - standaloneRoot: MetaRoot, - selected: readonly MetaData[], - selectedKeys: ReadonlySet, -): ClosurePair[] { - const pairs: ClosurePair[] = []; - const seen = new Set(); - for (const top of selected) { - const referrerKey = top.resolutionKey(); - for (const node of walkOwn(top)) { - const referrerPkg = effectivePackage(node) ?? ""; - const candidates: string[] = []; - if (node.superRef !== undefined) candidates.push(node.superRef); - // ADR-0039 sanctioned own-only case (same rationale as walkOwn() above): an - // INHERITED ref-bearing attr belongs to the base node's own layer, not this - // one's, and is checked when the base itself is walked — reading it via the - // resolving `attr()` here would attribute the base's reference to this node. - for (const attrName of REF_BEARING_ATTR_NAMES) { - const value = node.ownAttr(attrName); - if (typeof value === "string") candidates.push(value); - } - for (const raw of candidates) { - const resolved = resolveObjectRef(standaloneRoot, refOwner(raw), referrerPkg).node; - if (resolved === undefined) continue; // already validated at load — defensive only - const targetKey = resolved.resolutionKey(); - if (selectedKeys.has(targetKey)) continue; - const dedupeKey = `${referrerKey}${DEDUPE_KEY_SEPARATOR}${targetKey}`; - if (seen.has(dedupeKey)) continue; - seen.add(dedupeKey); - pairs.push({ referrer: referrerKey, target: targetKey }); - } - } - } - return pairs.sort((a, b) => - a.referrer === b.referrer ? a.target.localeCompare(b.target) : a.referrer.localeCompare(b.referrer), - ); -} - /** design §2.2 — "version defaults to the host package's version". Walks up from * `startDir` for the nearest `package.json` carrying a string `version`. */ function nearestPackageVersion(startDir: string): string | undefined { @@ -200,91 +72,26 @@ export function sharedModelFile(opts: SharedModelFileOpts): Generator { ); } - // Step 1 — standalone load, the run's own composed registry (so the publisher's - // own consumer-supplied providers are honoured here exactly as they were for - // the project's main load), strict. - const loader = new MetaDataLoader({ + const model = await buildSharedModel({ + name: opts.name, + files, + include: opts.include, + ...(opts.exclude !== undefined ? { exclude: opts.exclude } : {}), ...(ctx.registry !== undefined ? { registry: ctx.registry } : {}), - strict: true, }); - const loaded = await loader.load(files.map((p) => new FileSource(p))); - if (loaded.errors.length > 0) { - throw loaded.errors[0]; - } - const standaloneRoot = loaded.root; - // Step 2 — select top-level nodes. ADR-0039 sanctioned own-only case: `MetaRoot` - // has no super chain (it never `extends`), so its own children ARE its effective - // children — the same case `MetaRoot.objects()`/`findObject()` document. - const compiled = compileScope({ include: opts.include, exclude: opts.exclude ?? [] }); - const selected = standaloneRoot - .ownChildren() - .filter((node) => node.type !== TYPE_REQUIREMENT && node.type !== TYPE_TEMPLATE) - .filter((node) => matchesScope(node.resolutionKey(), compiled)); - if (selected.length === 0) { - throw new Error( - `sharedModelFile("${opts.name}"): selected no nodes (include/exclude, always ` + - `excluding requirement.* and template.*) — nothing to publish. Check \`include\`.`, - ); - } - - // Step 3 — closure. - const selectedKeys = new Set(selected.map((n) => n.resolutionKey())); - const closureFailures = checkClosure(standaloneRoot, selected, selectedKeys); - if (closureFailures.length > 0) { - const lines = closureFailures.map((p) => ` ${p.referrer} → ${p.target}`).join("\n"); - throw new Error( - `sharedModelFile("${opts.name}"): the selection is not closed — these references ` + - `point outside it; include it or exclude the referrer:\n${lines}`, - ); - } - - // Step 4 — serialize (already sorted by resolution key; the callee also - // throws if a selected node has no package, per its own contract). - const artifactContent = serializeSharedDocument(selected); - - // Step 5 — re-load standalone with CORE providers only (registry omitted ⇒ - // MetaDataLoader's own default registry), strict. A consumer loads this - // artifact with ITS OWN providers, never the publisher's — so an export - // needing publisher-only vocabulary must fail HERE, not at the consumer. - const artifactPath = `${opts.name}${ARTIFACT_SUFFIX}`; - const coreLoader = new MetaDataLoader({ strict: true }); - const coreLoaded = await coreLoader.load([ - new InMemoryStringSource(artifactContent, { id: artifactPath }), - ]); - if (coreLoaded.errors.length > 0) { - const first = coreLoaded.errors[0]!; - throw new Error( - `sharedModelFile("${opts.name}"): the export needs a provider this toolchain does ` + - `not ship; Phase 1 exports must load with core vocabulary: ${first.message}`, - ); - } - - // Step 6 — emit the artifact + manifest. - const integrity = `${INTEGRITY_PREFIX}${createHash("sha256").update(artifactContent, "utf-8").digest("hex")}`; - const packages = [...new Set(selected.map((n) => packageOfResolutionKey(n.resolutionKey())))].sort(); - const nodes = selected.map((n) => n.resolutionKey()).sort(); - const version = opts.version ?? nearestPackageVersion(ctx.projectRoot ?? process.cwd()); + // Resolved after the selection has passed every check, so a model that cannot be + // published says so before a missing version does. + const version = opts.version ?? nearestPackageVersion(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd()); if (version === undefined) { throw new Error( `sharedModelFile("${opts.name}"): no version available — pass \`version\` explicitly, ` + `or run inside a project with a package.json version at or above the project root.`, ); } - const manifest = { - schema_version: MANIFEST_SCHEMA_VERSION, - name: opts.name, - version, - metamodelVersion: METAMODEL_VERSION, - artifact: artifactPath, - integrity, - packages, - nodes, - }; - return [ - { path: artifactPath, content: artifactContent }, - { path: MANIFEST_FILE, content: JSON.stringify(manifest, null, 2) + "\n" }, - ]; + + const { artifact, manifest } = renderSharedModelFiles(model, version); + return [artifact, manifest]; }), }; if (opts.target !== undefined) generator.target = opts.target; diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index fc229e275..dafb6ee73 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -6,7 +6,7 @@ export { runGen } from "./runner.js"; export type { RunGenOpts, RunGenResult } from "./runner.js"; -export type { Generator, GenContext, EmittedFile, GeneratorFactory } from "./generator.js"; +export type { Generator, GenContext, EmittedFile, GeneratorFactory, OwnedDocsSurface } from "./generator.js"; export { perEntity, perPackage, perModel, oncePerRun } from "./generator.js"; // The one aggregation of the UI-tier markers. Exported because `meta docs` is the // SECOND door onto `agent/ui.md` and must answer "does this run emit a UI tier?" @@ -60,7 +60,7 @@ export { export { isPackagedDocsGenerator, packagedDocsFactoryCall, DOCS_DOOR_COMMAND } from "./packaged-docs-generators.js"; export type { MetaobjectsGenConfig, NormalizedMetaobjectsGenConfig, ResolvedGenConfig, Dialect, ExtStyle, ColumnNamingStrategy, MetaDataTypeProvider, GeneratorSpec, DocsConfig, ResolvedDocsConfig, DocsSurface, ApiSurface, VerifyConfig } from "./metaobjects-config.js"; -export { defineConfig, normalizeConfig, resolveGenerators, isNeutralGeneratorName, resolveDocsConfig } from "./metaobjects-config.js"; +export { defineConfig, normalizeConfig, resolveGenerators, isNeutralGeneratorName, removedGeneratorMessage, resolveDocsConfig } from "./metaobjects-config.js"; // The dialect every generator falls back to when a config declares none. Exported because // `meta docs` must apply the SAME default `normalizeConfig` does — a command that resolved // the dialect differently would document a schema the toolchain does not produce. @@ -132,6 +132,22 @@ export { } from "./docs-paths.js"; export type { DocPageNode, DocPagePlacement } from "./docs-paths.js"; +// The docs tier (ADR-0034 Amendment 7). `docsFile()` and `apiDocsFile()` are served from +// `/generators`; what they compose is public here so an owned (ejected) copy imports only +// this package. The owned part is which nodes get a page, +// where it lands and how the index is laid out; the page payloads (`buildEntityDocData`, +// `buildTemplateDocData`, `buildApiModelForRun`), the renderers and the ER diagram stay in +// the package. `docsDoorGenerators` is how `meta docs` reads the owned copies out of +// `docs: { generators: [...] }`. +export { apiSurfaceHref, surfaceCrossHref } from "./docs-paths.js"; +export { docsDoorGenerators } from "./docs-door.js"; +export type { DocsDoorFacts } from "./docs-door.js"; +export { renderDocPage } from "./render-engine/doc-page.js"; +export { renderMermaidErBlock } from "./templates/mermaid-er.js"; +export { buildApiModel, buildApiModelForRun } from "./generators/api-model.js"; +export type { ApiModel, ApiModelContext, ApiUnitDoc } from "./generators/api-model.js"; +export { renderAgentApi, renderApiIndex, renderEntityApiPage } from "./generators/api-doc-render.js"; + export { isProjection, isWriteThrough } from "./projection/projection-detector.js"; export { isAbstract, emitsInstanceArtifacts, emitsWriteArtifacts } from "./instance-artifacts.js"; // The UI tier asks THESE — "is there an endpoint?" — never the storage predicates. @@ -409,6 +425,14 @@ export type { // Public so an OWNED generator applies it the same way the built-ins do. export { withClientDirective, CLIENT_DIRECTIVE } from "./client-directive.js"; +// FR-023 §4.3 — the shared-model contract. `sharedModelFile()` (served from `/generators`) is +// the reference generator; the two stages it wires are public so an owned (ejected) copy +// imports only this package and keeps only the parts it owns: which source files may +// contribute and which version is published. Everything a consumer's `meta deps sync` +// verifies (the selection, the closure, the bytes, the hash, the manifest) stays here. +export { buildSharedModel, renderSharedModelFiles } from "./shared-model/build-shared-model.js"; +export type { SharedModel, SharedModelFiles, SharedModelInput } from "./shared-model/build-shared-model.js"; + // FR-044 Plan 4 — the Cube exporter. `cubeModel()` is the reference generator; the two pure // stages it wires are public so an owned (ejected) copy imports only this package and keeps // only the parts it owns: which entities get a cube, the file layout and the YAML text. diff --git a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts index 41b75d687..17f3e5574 100644 --- a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts +++ b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts @@ -283,6 +283,20 @@ export interface DocsConfig { baseUrl?: string; surfaces?: DocsSurface[]; apiSurfaces?: ApiSurface[]; + /** + * Docs generators you OWN (`meta eject docs`, `meta eject api-docs`), for `meta docs` to + * run in place of its packaged ones. Each carries a `docsSurface` marker saying which + * surface it takes over, and one per surface is allowed. Leave it out and `meta docs` + * runs the packaged generators, which need no wiring. + * + * It is here, in the docs door's own block, and not in the top-level `generators`, + * because that array is what `meta gen` runs and `meta gen` does not run documentation + * (ADR-0021 D1, ADR-0025; ADR-0034 Amendment 7). + * + * import { docsFile } from "./codegen/generators/docs.js"; + * docs: { generators: [docsFile()] } + */ + generators?: Generator[]; } export interface ResolvedDocsConfig { @@ -379,20 +393,53 @@ export function isNeutralGeneratorName(name: string): boolean { return entry !== undefined && entry.tier !== "native"; } +/** + * Stable generator names that used to resolve and no longer do, each with what replaced + * it. A name that worked in a config yesterday deserves better than "unknown generator": + * that answer sends its reader looking for a typo. + * + * `mermaid-er` (removed in 1.1) wrote the ER diagram to a file of its own under + * `meta gen`. It was the older home of a diagram `meta docs` draws: the whole model on + * the overview page, and each entity's neighbourhood on that entity's page. + * + * Every entry so far is a doc generator `meta docs` replaced, and each message says so. + * `meta verify` relies on that to report a removed name under the docs door, so a test + * holds it: a removed generator of another kind needs its own advisory first. + */ +export const REMOVED_GENERATOR_NAMES: Readonly> = { + "mermaid-er": + 'generator "mermaid-er" was removed in 1.1: `meta docs` draws the ER diagram, for the ' + + "whole model on its overview page (README.md) and around each entity on that entity's " + + 'page. Remove "mermaid-er" from `generators` and run `meta docs`.', +}; + +/** + * What {@link resolveGenerators} says about a stable name it refuses as REMOVED, or + * undefined for any other name. A config that lists the STRING makes `meta gen` fail + * with this message. Public so a caller can ask instead of matching the sentence. + */ +export function removedGeneratorMessage(name: string): string | undefined { + // An own-property test: a name such as "toString" must read as unknown, not as removed. + return Object.hasOwn(REMOVED_GENERATOR_NAMES, name) ? REMOVED_GENERATOR_NAMES[name] : undefined; +} + /** * Materialize the config `generators` array: pass typed generators through * untouched and resolve each stable-name string via the {@link generatorRegistry} * to its factory result (default options). ADR-0021 #1. * * Errors: - * - a NEUTRAL name (`docs`, `mermaid-er`) is owned by `meta docs` (ADR-0021 D1) - * and is not selectable in the gen suite. + * - a REMOVED name ({@link removedGeneratorMessage}) throws saying what replaced it. + * - a NEUTRAL name (`docs`) is owned by `meta docs` (ADR-0021 D1) and is not + * selectable in the gen suite. * - an UNKNOWN name throws listing the available NATIVE names. `api-docs` is left out * of that list: it resolves, and the runner warns about and skips it (ADR-0025). */ export function resolveGenerators(specs: readonly GeneratorSpec[]): Generator[] { return specs.map((spec) => { if (typeof spec !== "string") return spec; + const removed = removedGeneratorMessage(spec); + if (removed !== undefined) throw new Error(removed); const entry = generatorRegistry[spec]; if (entry === undefined) { // A native generator the runner skips is not offered: the list named `api-docs`, diff --git a/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts b/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts index 5e16aff21..0a9c7539b 100644 --- a/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts +++ b/server/typescript/packages/codegen-ts/src/packaged-docs-generators.ts @@ -10,9 +10,10 @@ // Keyed by the generator's IMPLEMENTATION name, because that is what a constructed // generator carries. The value is the factory call an adopter wrote, for the message. // -// `mermaid-er-diagram` is not one of them. `mermaidErDiagram()` is deprecated and kept -// running for a config that already lists it, so the runner does not skip it. -// `meta docs` draws the same diagram on its overview page through `docsFile()`. +// An OWNED copy of one of them (docs-door.ts) is skipped too, by its `docsSurface` marker +// and not by this map: an owner may rename the copy, and the fix for it is a different +// one. A packaged generator is removed from `generators`; an owned copy is moved to +// `docs: { generators: [...] }`, where `meta docs` runs it. const PACKAGED_DOCS_FACTORY_CALLS: ReadonlyMap = new Map([ ["docs-file", "docsFile()"], ["api-docs", "apiDocsFile()"], diff --git a/server/typescript/packages/codegen-ts/src/reference-templates.ts b/server/typescript/packages/codegen-ts/src/reference-templates.ts index d4c8d4611..6605405bd 100644 --- a/server/typescript/packages/codegen-ts/src/reference-templates.ts +++ b/server/typescript/packages/codegen-ts/src/reference-templates.ts @@ -29,6 +29,16 @@ export const REFERENCE_GENERATOR_NAMES = [ // generators over `renderCallableFile` / `renderTraceHelper`, so the owned part is which // objects get a module and where it lands. "callable", "trace-helper", + // The publisher's shared-model artifact: a thin generator over `buildSharedModel` and + // `renderSharedModelFiles`. The owned part is which source files may contribute and + // which version is published; the artifact, its hash and the manifest that a + // consumer's `meta deps sync` verifies stay in the package. + "shared-model", + // The docs tier (ADR-0034 Amendment 7). `docs` and `api-docs` are run by `meta docs`, which + // reads an owned copy out of `docs: { generators: [...] }` by its `docsSurface` marker. + // The owned part is which nodes get a page, where it lands and how the index is laid out; + // the page payloads and renderers stay in the package. + "docs", "api-docs", ] as const; export type ReferenceGeneratorName = (typeof REFERENCE_GENERATOR_NAMES)[number]; diff --git a/server/typescript/packages/codegen-ts/src/reference/api-docs.ts b/server/typescript/packages/codegen-ts/src/reference/api-docs.ts new file mode 100644 index 000000000..d09615143 --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/reference/api-docs.ts @@ -0,0 +1,117 @@ +// REFERENCE TEMPLATE — copy this into your repo (e.g. codegen/generators/api-docs.ts) and own it. +// Then import it LOCALLY in metaobjects.config.ts: +// import { apiDocsFile } from "./codegen/generators/api-docs.js"; +// +// RUNTIME: this file executes under whatever runs `meta docs`, and the published CLI's +// shebang is `#!/usr/bin/env node` — so it runs under NODE even in a Bun project. Do not +// reach for `Bun.*` globals here; they are undefined and take the whole run down with +// `Bun is not defined`. Use `node:` builtins instead. +// targets: Markdown. The pages describe the code the stock TypeScript generators +// emit (the entity module, queries, the Fastify routes and, when +// `routesFileHono` is wired, the Hono ones, the prompt and output helpers); +// they import nothing. +// use-when: you want to change the API reference `meta docs` writes: which units get +// a page, where the pages land, or which of the three forms are written. +// `meta docs` writes these pages without this file; eject it only to change +// them. If your generators emit a different API than the stock ones, the +// reference is wrong for you until you change what is written here. +// emits: under `docs.outDir`, in the api surface's directory (`api/` by default): +// README.md, the index; AGENT-API.md, the condensed form the agent context +// points at; .md per unit (an entity, a projection, a served report, +// a template). Under `outputLayout: "package"` the unit pages fold under +// their package path. +// customize: which units get a page (`model.units` in `generate`), where a page lands +// (`docPageOutputPath` under `apiDir`) and which forms are written (the +// index and the agent form are the two `unshift`ed files). Stays in the +// package: `buildApiModelForRun`, which reads the model and the run into +// the description of the generated API, and `renderEntityApiPage` / +// `renderApiIndex` / `renderAgentApi`, which write it. The always-on agent context names +// `api/AGENT-API.md`; if you move or drop that file, the pointer is the +// other half. +// composes-with: docs (each api page links back to its model page when `meta docs` writes +// the model surface too) and the routes generators (the Hono routes are +// documented when one is wired). +// +// HOW IT RUNS. `meta docs` runs it, so it is listed in the docs block of +// metaobjects.config.ts, not among the generators `meta gen` runs: +// docs: { generators: [apiDocsFile()] } +// The `docsSurface` marker below says which surface it takes over: with it listed, +// `meta docs` and `meta verify --docs` run THIS generator for the api surface in place of +// the packaged one. Documentation has one door, so `meta gen` never runs it. +// +// Everything below imports ONLY from `@metaobjectsdev/codegen-ts` (the stable engine). + +import { + assertNoDuplicateDocPaths, + buildApiModelForRun, + docPageOutputPath, + projectProvider, + renderAgentApi, + renderApiIndex, + renderEntityApiPage, + surfaceCrossHref, + type DocPageNode, + type DocPagePlacement, + type EmittedFile, + type Generator, +} from "@metaobjectsdev/codegen-ts"; + +// Where the pages go when nothing says otherwise. `meta docs` always says: it passes the +// directory of the api surface it is writing (`docs.apiSurfaces[].subDir`, default `api`). +const DEFAULT_API_DIR = "docs/api"; + +export function apiDocsFile(): Generator { + return { + name: "api-docs", + // The surface this generator takes over when it is listed in `docs.generators`. + docsSurface: "api", + generate(ctx) { + const provider = projectProvider(ctx.projectRoot ?? process.cwd()); + const layout = ctx.config.outputLayout ?? "flat"; + + // What `meta docs` reports about the run: the directory of this surface, and whether + // the model pages are written too. + const apiDir = ctx.docsDoor?.apiSubDir ?? DEFAULT_API_DIR; + const modelSurface = ctx.docsDoor?.modelSurface; + const indexFilename = `${apiDir}/README.md`; + const agentFilename = `${apiDir}/AGENT-API.md`; + + // One model of the generated API feeds every form. `buildApiModelForRun` reads what it + // needs off the run: the layout, whether a Hono routes generator is wired (its routes + // are documented exactly then), the project's `apiPrefix` and the primary keys. + const model = buildApiModelForRun(ctx); + + // Every (path, node) placed, so a collision is refused after all pages are placed. + const placements: DocPagePlacement[] = []; + + // OWNED: which units get a page, and where it lands. + const files: EmittedFile[] = model.units.map((unit) => { + const node: DocPageNode = { name: unit.node, package: unit.package }; + const path = `${apiDir}/${docPageOutputPath(layout, node)}`; + placements.push({ path, fqn: unit.package ? `${unit.package}::${unit.node}` : unit.node }); + // The model page sits at the docs root at the same placement, so the link back is + // derived from it and resolves in both layouts. None when the model surface is off. + const modelPageHref = modelSurface + ? surfaceCrossHref(path, docPageOutputPath(layout, node)) + : undefined; + return { path, content: renderEntityApiPage(unit, provider, modelPageHref) }; + }); + + // OWNED: the index and the agent form, first, and only when a unit page exists. + if (files.length > 0) { + placements.push({ path: indexFilename, fqn: "" }); + placements.push({ path: agentFilename, fqn: "" }); + files.unshift( + { path: indexFilename, content: renderApiIndex(model, layout, provider) }, + { path: agentFilename, content: renderAgentApi(model, provider) }, + ); + } + + // Two units that resolve to one output path are refused, naming both and the path, + // instead of one page silently overwriting the other. + assertNoDuplicateDocPaths(placements); + + return files; + }, + }; +} diff --git a/server/typescript/packages/codegen-ts/src/reference/docs.ts b/server/typescript/packages/codegen-ts/src/reference/docs.ts new file mode 100644 index 000000000..646f2c68a --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/reference/docs.ts @@ -0,0 +1,244 @@ +// REFERENCE TEMPLATE — copy this into your repo (e.g. codegen/generators/docs.ts) and own it. +// Then import it LOCALLY in metaobjects.config.ts: +// import { docsFile } from "./codegen/generators/docs.js"; +// +// RUNTIME: this file executes under whatever runs `meta docs`, and the published CLI's +// shebang is `#!/usr/bin/env node` — so it runs under NODE even in a Bun project. Do not +// reach for `Bun.*` globals here; they are undefined and take the whole run down with +// `Bun is not defined`. Use `node:` builtins instead. +// targets: Markdown, with Mermaid for the diagrams. The pages import nothing. +// use-when: you want to change the model documentation `meta docs` writes: which +// objects and templates get a page, where the pages land, or how the +// overview page is laid out. `meta docs` writes these pages without this +// file; eject it only to change them. To change the WORDING of a page you +// need no generator at all: drop your own +// `templates/docs/entity-page.md.mustache` or +// `templates/docs/template-page.md.mustache` in the project root. +// emits: under `docs.outDir` (default ./docs/generated): README.md, the overview +// with the whole-model ER diagram and an index; .md per object, +// reports included;