From f37ce9defc7ab6b152262f54af83beb59bccc530 Mon Sep 17 00:00:00 2001 From: mogoodrich Date: Fri, 13 Nov 2026 10:10:35 -0500 Subject: [PATCH] ocl: load-test an export on a real OpenMRS database before putting it into pihemr Workflow 6 gets a new step 3: before creating any branch, run pihemr's scripts/ocl-load-test.sh (PIH/pihemr#783) in a throwaway worktree from master. It starts the ZL EMR with the new export on a copy of the nightly ZL seed database and checks the OCL import finished without errors. Asks first (about 10-20 minutes), stops if the export doesn't load, and leaves removing the test instance to the user. Also documents the Action's new "Skip the load test" checkbox, and bumps the plugin to 0.5.0. Co-Authored-By: Claude Opus 5.5 --- plugins/ocl/.claude-plugin/plugin.json | 2 +- plugins/ocl/README.md | 2 ++ plugins/ocl/skills/ocl-search/SKILL.md | 14 ++++++++++---- 3 files changed, 13 insertions(+), 5 deletions(-) diff --git a/plugins/ocl/.claude-plugin/plugin.json b/plugins/ocl/.claude-plugin/plugin.json index 17bce48..0f7322f 100644 --- a/plugins/ocl/.claude-plugin/plugin.json +++ b/plugins/ocl/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "ocl", - "version": "0.4.0", + "version": "0.5.0", "description": "Search and maintain the PIH concept dictionary in Open Concept Lab (OCL): look up concepts, check set membership, clone from CIEL, and create or edit PIH concepts and mappings.", "author": { "name": "Partners In Health" diff --git a/plugins/ocl/README.md b/plugins/ocl/README.md index 7bc7458..1aa207b 100644 --- a/plugins/ocl/README.md +++ b/plugins/ocl/README.md @@ -86,6 +86,8 @@ Each check also watches for OCL fixing the clone bugs it works around (ocl_issue **The same check also runs automatically** when the PIH/pihemr "Update OCL export" GitHub Action puts a new release into the distro. If the release has errors, the Action stops and doesn't open a pull request, so the release never reaches servers. +**Then it test-loads the release on a real OpenMRS database.** The release check only catches problems we already know about, so the Action (and Claude, when you ask it to put a release into the distro) also starts the ZL EMR with the new export on a copy of the nightly ZL seed database and checks that the import finished without errors. It takes about 15–30 minutes in the Action (10–20 when Claude runs it on your machine); tick *Skip the load test* in the Action, or tell Claude to skip it, when you can't wait. The PR says when it was skipped. The test is pihemr's `scripts/ocl-load-test.sh`. + Developers can run the check directly: `python3 -I plugins/ocl/skills/ocl-search/scripts/ocl_release_check.py --unreleased`. Add `--since ` to check against an older release (e.g. `--since 1.8.46` reproduces the 1.8.47 failure), or use `--export --baseline ` to check an export the way the Action does. `--list-checks` lists the checks; the script's header explains how to add or retire one. ## Staying up to date diff --git a/plugins/ocl/skills/ocl-search/SKILL.md b/plugins/ocl/skills/ocl-search/SKILL.md index bf8b1a3..0a81119 100644 --- a/plugins/ocl/skills/ocl-search/SKILL.md +++ b/plugins/ocl/skills/ocl-search/SKILL.md @@ -231,10 +231,16 @@ Changes reach OpenMRS servers through released source versions, not the live sou OpenMRS servers load the PIH dictionary from an export zip committed to the distro, so a release only reaches them once its zip is swapped in. -- **No checkout needed:** `PIH/pihemr` may have an *Update OCL export* GitHub Action (Actions tab → Run workflow) that does all of this and opens a PR. If it exists, point non-developers to it. The steps below are for doing it locally. +- **No checkout needed:** `PIH/pihemr` has an *Update OCL export* GitHub Action (Actions tab → Run workflow) that does all of this and opens a PR. It runs the release check and the load test (step 3) first, unless *Skip the release check* or *Skip the load test* is ticked. Point non-developers to it. The steps below are for doing it locally. 1. **Get the download URL.** `GET …/PIH//export/` with the token returns `302` once the export is built (204 or 208 mean it isn't ready yet; see workflow 5), redirecting to a signed S3 URL such as `https://ocl-production-s3.s3.amazonaws.com/orgs/PIH/PIH_PIH_v1.8.43.2026-10-29_212531.zip?…`. Read the `Location` header (`curl -s -D - -o /dev/null …`), then download that URL in a separate curl *without* the token header. It's already signed, and this keeps the token from going to S3. 2. **Check the zip** before using it. It holds a single `export.json`. Confirm its `id` is the version you wanted, and note how many `concepts` and `mappings` it has, so you can tell the user. -3. **Find the distro folder.** In the pihemr distro (`PIH/pihemr`), it's `content/configuration/backend_configuration/ocl/`. The folder holds exactly one zip, which is the current export. Other PIH config repos (e.g. `config-pihemr/configuration/ocl/`) carry their own, often older, export. Update only the ones the user asks for. -4. **Check master first.** Others commit export bumps often, sometimes several a week. `git fetch`, then compare the branch's zip with `origin/master`'s. If master already has a newer version than the branch, rebase before swapping, so you don't hit a conflict on the binary file. -5. **Swap the zip.** Delete the old one and add the new one, named to match master. Since Oct 2026 that's `orgs_PIH_sources_PIH__.zip`, using the timestamp from the S3 filename. Older commits used the raw S3 name (`PIH_PIH_v..zip`). Then `git grep` for the old filename. No references have been found so far. +3. **Load-test it on a real OpenMRS database** before creating any branch. The release check only catches the problems we know about; this runs the actual import. Ask the user first whether to run it, since it takes about 10–20 minutes. If they skip it, say so in the PR description. + - It uses pihemr's `scripts/ocl-load-test.sh`, the same script the GitHub Action runs. If `origin/master` doesn't have it yet, say the load test isn't available and carry on. + - Don't touch the user's pihemr checkout. Make a throwaway worktree from master: `git -C fetch`, then `git -C worktree add --detach /pihemr-load-test origin/master`. In it, replace the zip in `content/configuration/backend_configuration/ocl/` with the new one, named as in step 6. + - Run `scripts/ocl-load-test.sh --summary-md /load-test.md --openmrs-log /openmrs.log` there, in the background (Java 8, Maven and Docker are needed). It builds pihemr and the ZL EMR distribution with the new export, restores the nightly ZL seed database (`partnersinhealth/zl-emr-seed-haiti`, which already has master's export loaded) into a new `openmrs-docker` instance, starts it, and checks that the OCL import finished with no errors. It clones zl-emr and distro-tools itself; pass `--distro-tools-dir` to use the user's checkout. + - Exit 0: it loaded cleanly; carry on. Exit 1: it didn't; show the user the summary (it lists the items that failed, with the database error for each) and stop. The full log is in `openmrs.log` if they need more. Don't create a branch for an export that doesn't load. Exit 2: the test couldn't run; tell the user why and ask whether to go on without it. + - Afterwards, remove the worktree (`git -C worktree remove --force /pihemr-load-test`). The script stops the instance but keeps it so it can be inspected; tell the user it's there and that `openmrs-docker destroy` removes it (it asks them to confirm, so leave it to them). Also tell them the test installed a pihemr snapshot with this export into `~/.m2`, which their other local builds will use until they rebuild pihemr or Maven pulls a newer snapshot. +4. **Find the distro folder.** In the pihemr distro (`PIH/pihemr`), it's `content/configuration/backend_configuration/ocl/`. The folder holds exactly one zip, which is the current export. Other PIH config repos (e.g. `config-pihemr/configuration/ocl/`) carry their own, often older, export. Update only the ones the user asks for. +5. **Check master first.** Others commit export bumps often, sometimes several a week. `git fetch`, then compare the branch's zip with `origin/master`'s. If master already has a newer version than the branch, rebase before swapping, so you don't hit a conflict on the binary file. +6. **Swap the zip.** Delete the old one and add the new one, named to match master. Since Oct 2026 that's `orgs_PIH_sources_PIH__.zip`, using the timestamp from the S3 filename. Older commits used the raw S3 name (`PIH_PIH_v..zip`). Then `git grep` for the old filename. No references have been found so far.