Skip to content

feat: parameters shared by several graphs, from parameter files - #21

Merged
psyinf merged 1 commit into
mainfrom
feat/graph-parameters
Sep 21, 2026
Merged

psyinf merged 1 commit into
mainfrom
feat/graph-parameters

Conversation

@psyinf

@psyinf psyinf commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Why

Several pipelines often need the same tuning values, such as a width or a threshold. Today each .fg graph repeats them inline, so keeping them in sync depends on remembering to edit every copy. This PR moves them into a parameter file that graphs name and share.

What

# alerts.fg
params "tuning.json"
in -> MinLength(minLength=$text.minLength) -> long
long -> Truncate(width=$text.width) -> out

# report.fg
params "tuning.json"   # shared with alerts.fg
params "report.json"   # this graph's own parameters
in -> Truncate(width=$text.width) -> out.text
in -> Classify(classes=$report.classes) -> out.size
DslFilterGraph<std::string, std::string> alerts(dsl::loadGraphProgram("graphs/alerts.fg"));
DslFilterGraph<std::string, std::string> narrow(dsl::loadGraphProgram("graphs/alerts.fg", {{"text", {{"width", 6}}}}));
  • $name arguments. Dots walk into nested objects. A parameter can hold any JSON value, so lists and objects can now reach a stage from the DSL, which partly lifts the "flat arguments" limitation.
  • params "file.json" lines. Paths are relative to the graph file. Files apply in order, each overriding the ones before it member by member (JSON merge patch). The overrides argument of loadGraphProgram applies last.
  • API. New header GraphParameters.hpp, included by DslFilterGraph.hpp:
    • dsl::loadGraphProgram(path, overrides)
    • dsl::parseGraphProgram(text, parameters)
    • dsl::bindParameters(program, parameters)
    • dsl::loadParameters(path)
    • validateDslGraph now also takes a parsed GraphProgram.
  • Diagnostics:
    • An unknown name is located and gets a suggestion: 1:22: unknown parameter '$text.widht' — did you mean '$text.width'?
    • A parameter file that is unreadable, invalid JSON or not an object is reported at its params line, with no follow-on "unknown parameter" noise.
    • Unused parameters are deliberately not reported, because a shared file always holds values some graph doesn't use.
  • No file reads at parse time. parseGraphProgram(text) only records references and files. Renderings (toAscii / toMermaid / toDot) show unbound parameters as $name. Building a graph from an unbound program reports each parameter as "not set".
  • Compatibility. Additive. An edge named params keeps working, because a directive needs a string literal right after the keyword. The deprecated hand-written parser does not support the new syntax (noted in its header and the CHANGELOG).

Docs and sample

  • README: new section "Parameters shared by several graphs"; updated feature list and limitations.
  • EXAMPLE.md section 11 and new sample apps/tunedPipeline: two .fg files, tuning.json, report.json.
  • CHANGELOG and TODO (workaround for nested arguments).

Testing

  • tests/FilterGraphTests/ParameterTests.cpp: 16 cases covering binding, nested names, mixing with literals, diagnostics and their locations, syntax errors, rendering, file loading, relative paths, override order and unreadable files.
  • Local MSVC debug (ASan): 140/140 tests pass, and all five samples run.

Out of scope (possible follow-ups)

  • Reloading parameters into a running graph (hot tuning). Parameters are bound when the graph is built.
  • Parameters for the JSON graph format.
  • A whole stage config from one parameter, e.g. Stage($preset).

A stage argument can name a parameter, `Truncate(width=$text.width)`,
whose value comes from a JSON parameter file that several graph files
share. A graph file names its parameter files with `params "file.json"`
lines, relative to the graph file; later files override earlier ones
member by member (JSON merge patch).

- dsl::loadGraphProgram(path, overrides) reads a graph file with its
  parameter files and binds them; parseGraphProgram(text, parameters),
  bindParameters and loadParameters bind values from the caller.
- Dots walk into nested objects, and a parameter can hold any JSON value,
  so lists and objects can now reach a stage from the DSL.
- An unknown name is a located diagnostic with a suggestion; an unreadable
  parameter file is one at its params line; unused parameters are not
  reported. Unbound parameters are reported as not set when a graph is
  built, and renderings show them as `$name`.
- validateDslGraph also takes a parsed GraphProgram.
- New sample apps/tunedPipeline and EXAMPLE.md section 11.
@psyinf
psyinf merged commit 703c3bc into main Sep 21, 2026
3 checks passed
@psyinf
psyinf deleted the feat/graph-parameters branch September 21, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant