disclosed

Grades US higher-education institutions on what they disclose, not on how they perform.

There are many tools that will tell you a college's graduation rate. There is no tool that tells you how many colleges did not report one, or which fields quietly stopped being published this year. That is what this grades.

Status: Beta, pre-release (0.1.0.dev0). The five-way classification, both federal adapters, and drift measurement are complete and tested behind a 95% branch-coverage gate. There are deliberately no tagged releases: nothing here is consumed downstream, and ADR 0001 records why a release pipeline with nothing to release would be exactly the kind of gate that never fails.

The distinction matters because the two failures look identical on a page. A college with a 0% admission rate and a college that never reported an admission rate both render as a blank or a zero in most tools, and a reader cannot tell them apart. In a 600-institution sample of the College Scorecard, 387 of the 600, or 64.5%, publish no admission rate at all, and separately, one institution publishes an admission rate of exactly zero — which is not a school that admitted nobody, it is a reporting artifact that survived because zero is a legal number.

That sample is not a random one: the API returns institutions grouped by state, and the committed 600 arrived from the front of the alphabet, 51% of them Californian. A full census — every institution the College Scorecard publishes, walked to exhaustion — puts the same question at 4,363 of 6,273, or 69.6%, publish no admission rate at all, five points higher than the sample's figure rather than lower, which is not the direction a reader who assumed the sample flattered itself would guess. The Scorecard census page states both frames' composition side by side; the sample figure above is unchanged and describes the 600 institutions it has always described.

Install

There is no package to install from an index, and that is a decision rather than an omission: nothing consumes this repository, so ADR 0001 declines a release pipeline with nothing to release. Install it from a clone.

git clone https://github.com/ChelseaKR/disclosed
cd disclosed
uv sync                     # dev environment, including the linters and the test suite

Or, without uv, the way .github/workflows/pages.yml installs it before rendering the published site:

python -m pip install -e .  # Python 3.12+, no runtime dependencies
disclosed --help

Both give you the disclosed command. The optional question-answering layer (ADR 0006) is the only part with dependencies, and it is an extra: pip install -e '.[ask]', or .[ask-bedrock] to reach the same models through Amazon Bedrock.

Quickstart

Every command below runs offline and needs no API key. The federal captures they read are committed, which is the point: a number this project publishes has to be reproducible by somebody who does not have a key.

uv sync
make verify        # the one local gate: ruff, strict mypy, tests, accessibility checks

Rebuild the published site from committed data and open it:

make site          # renders ./site from data/report.json, data/national.json and the census
python -m http.server -d site

Regrade the full College Scorecard census and check it still reproduces byte for byte:

make census-replay # regrades data/census/scorecard.json and diffs against the committed artifact
make replay        # the same contract for the IPEDS national artifact, from the committed archives

Export the classified dataset, or grade the census yourself:

make dataset       # data/dataset.csv, its Table Schema and datapackage.json, in one pass
disclosed grade --source data/census/scorecard.json --out /tmp/census-graded.json

Only make fetch and make grade reach the network, and only those need DATA_GOV_API_KEY. CONTRIBUTING.md has the rest of the development setup.

What it does

Every value from a publisher is classified before anything else touches it:

Classification	Meaning	Counts against the publisher?
REPORTED	A credible value they actually disclosed	—
IMPLAUSIBLE	Disclosed, but outside the credible range for that field	Yes
SUPPRESSED	Withheld deliberately, usually to protect a small cohort	No
NOT_APPLICABLE	The question doesn't apply to this institution	No, and it leaves the denominator
MISSING	No value and no stated reason	Yes

Suppression is a policy decision made for good reasons and is never held against anyone. Punishing an institution for protecting a twelve-person cohort would push publishers toward disclosing things they shouldn't, which is the opposite of the point.

An institution whose every field is suppressed gets no grade at all, not a zero. The project applies its own discipline to itself: absence is reported as absence.

Disclosure drift

A single snapshot can't distinguish a field that was never collected from one that was collected until recently and then stopped. Only the comparison between runs can, and the difference matters: the first is a gap in the data model, the second is a change in what the public is allowed to know.

The unit of interest is the field, not the institution. One college dropping a field is a data-entry event. Four hundred colleges dropping the same field between two runs is a policy change. Drift is reported in both directions — fields that started being reported are as real a finding as fields that stopped, and reporting only the losses would make this an argument rather than a measurement.

disclosed grade --out data/report.json
disclosed snapshot --taken 2026-08-05 --out data/snapshots/scorecard/2026-08-05.json
disclosed drift data/snapshots/ipeds/2021.json data/snapshots/ipeds/2023.json

Snapshots are small enough to commit, so the record of what stopped being published lives in git rather than in a bucket someone has to trust. A scheduled workflow accrues snapshots/scorecard/ daily; snapshots/ipeds/ holds three real collection years, so drift has something true to compare against from day one.

The two live in separate directories and each snapshot records its source, because the two field sets do not overlap: comparing across them would skip every field and print "no change in per-field disclosure", which is the most reassuring possible way of saying nothing at all. drift refuses such a pair outright.

The series, on a page

Until now the drift measurement lived in a job summary on a green run and in the output of a command, which meant the most distinctive thing this project publishes was the thing a reader was least likely to see. disclosed site --snapshots-from data/snapshots renders one disclosure-history page per source from the committed series: each field's reporting rate in every committed run, and what moved between the first run and the last, with the direction word, the change in percentage points, how far the field's denominator moved, and whether the movement clears the systemic threshold.

Every figure on those pages is recomputed by the same drift.compare the command runs, and a test binds each cell to it rather than to a number written into a fixture. Three absences a rate table would otherwise collapse into 0% are kept apart and said in words: a field a run never graded, a field whose denominator was empty so it has no rate, and a movement that could not be measured in one of the two runs and therefore gets no direction word at all — FieldDrift.direction falls back to the sign of the raw count, which is a reasonable last resort for a terminal and is not one for a page whose whole argument is that the count and the rate can point opposite ways.

One source per page, because drift refuses a mixed pair for a reason, and a snapshot written before scope existed lands on a page that says the source was not recorded rather than being folded into the nearest named collection. Without the flag the build is byte-for-byte what it was and the site makes no claim about drift, which is the honest rendering of a build that was never shown the series.

The committed IPEDS series is what the page is worth reading for: across 2021–2023 the athletics disclosure is the only movement over the threshold, and admissions information gained 1.67 points while reaching 131 fewer institutions — the exact pair of facts a count-based reading got backwards.

Which institutions, and whether we moved the rules

drift says four hundred institutions stopped publishing a field. diff-report says which ones, by comparing two whole reports institution by institution and stating every move as a transition between two of the five states:

disclosed diff-report data/report.json /tmp/report-today.json
disclosed diff-report data/report.json /tmp/report-today.json --institution 105525

The unit is the transition and never the value. A college that published a 31% admission rate and now publishes 44% has disclosed exactly as much as before, so it produces no transition at all; a report diffed against itself is empty.

Every report carries rules_version, the version of this repository's own bands, credible ranges and applicability predicates. diff-report refuses a pair that names two different versions, and names both. The reason is that reported → implausible has two possible authors: the institution published a different number, or we rewrote the range it is checked against. Those are opposite findings, and attributing the second one to a college would be the most confident possible way of being wrong. A report that does not state a version is not treated as agreeing with one that does; the comparison still runs and says out loud that it could not confirm the rules.

Drift is a change in rate, and it took real history to prove it

Measured on counts, those three years produced three confident systemic findings and all three were false. The directory shrank from 6,289 institutions to 6,163, so 130 fewer published a web address, and that was reported as a systemic 2.1% collapse. The share publishing one had gone up, from 99.93% to 99.95%. Colleges closed; they did not stop reporting. Meanwhile the one real movement in the period — the athletics disclosure rising from 57.1% to 59.4% — ranked fourth and was never flagged, because 52 is a small number next to 130.

Every comparison now divides by the institutions the field applied to in that run, and the direction word is read from the rate rather than the count. A field can shed reporters while the share reporting it rises; printing "lost" beside a rise of 1.67 points got the count right and the finding backwards, which is worse than a wrong number because it comes with a word attached.

The 2-point threshold is a judgment call, and three years of federal data say it is roughly right: every year-on-year movement sits under one point except the athletics disclosure, at 1.75 in a year and 2.26 across two. At 1% the bar reports ordinary churn as policy. At 5% it finds nothing in three years, which is not a measurement but a way of never having to say anything.

Two federal sources, one disagreement

IPEDS and the College Scorecard are published by the same department and keyed on the same unit id, so the same institution can be checked against both. Across the 600 institutions in the committed capture they agree on state for every one and disagree about exactly one on sector:

Grand Canyon University. The College Scorecard files it as private nonprofit. IPEDS files it as private for-profit.

Sector decides which rules an institution answers to and which peer group it is compared against in most analyses of federal education data. The disagreement is reported and deliberately never resolved: deciding which federal source is correct is not something this project is in a position to do, and quietly preferring one would throw away the only interesting part of the observation.

IPEDS also carries public disclosures the Scorecard does not. Among institutions that participate in Title IV and enroll first-time undergraduates, the federal record carries no net price calculator for 34 of them, a calculator that 20 U.S.C. §1015a(h)(3) requires and that §1094(a)(17) requires them to report. Which of the two is missing is not something a blank cell can tell you, and the finding says so rather than picking the more dramatic reading.

Getting to 34 rather than 213 is the applicability rule doing its job. 213 of the 6,163 directory rows carry no calculator address; graduate-only institutions, institutions taking no federal aid, system offices, and closures account for the other 179. They are outside the statute and leave the denominator instead of being marked down.

The rule is the hard part, not the threshold

The Equity in Athletics disclosure went ungraded in an earlier pass. It is blank for 4,469 of 6,163 directory rows, and almost every one of those is a college with no athletics program, so grading it against the directory alone would have manufactured four thousand violations. What was missing was not a better threshold. It was a way to know who the rule applied to.

The IPEDS institutional characteristics file supplies it: each institution's own answer about whether it belongs to a national athletic association. That moves the denominator from 6,163 to 1,998, of which 812 give the federal record no athletics address. 20 U.S.C. §1092(g) requires those institutions to prepare the report and make it available; it does not require them to post it, so this is graded as a disclosure gap and stated as weaker than the net price calculator finding, not louder.

The veterans page is still not graded, and the same file would now supply a rule for it. There is no universal requirement to publish one, and a rule about who a duty reaches is worthless when the duty does not exist.

disclosed crosscheck --cache data/HD2023.zip --characteristics data/IC2023.zip \
  --source data/sample.json --out data/crosscheck.json

Both IPEDS files are required. If the characteristics file cannot be read the load fails rather than returning directory-only records, because a field that silently stops being graded looks on the page exactly like a field everybody suddenly started reporting.

Use the data

disclosed site    --report data/report.json --out site --generated 2026-08-05
disclosed dataset --report data/report.json --out data/dataset.csv \
                  --package datapackage.json --root .
# With a running disclosed.ask service (ADR 0006), institution pages gain the opt-in form:
disclosed site    --report data/report.json --out site --generated 2026-08-05 \
                  --ask-endpoint https://example.invalid/ask

Check one institution's page from public inputs

Every institution page carries a receipt.json beside it, and the published site is built with --receipts-from data/sample.json so that every one of them is there. A receipt is the derivation of that page written down: which file it came from and that file's sha256, which version of the grading rules read it, and what each field was classified as. Nothing in it is fetched and nothing is dated from the clock — the date on a receipt is the capture's walk date, or a sentence saying the source file records none.

disclosed receipt 100654 --source data/sample.json --out receipt.json
disclosed verify-receipt receipt.json --source data/sample.json

verify-receipt regrades that one record from the committed file and reports every difference. It exits 0 when the replay agrees, 1 when it disagrees and names the field, 2 when the source does not hold that institution, and 3 when the receipt could not be read — the last is separate because a verifier that answered "agrees" for a file it failed to parse would be a check that cannot fail. Given several receipts it returns the worst outcome, for the same reason. A receipt naming a different capture than the one being replayed is reported in its own line and does not become the verdict: replaying an old receipt against a newer capture is a legitimate thing to do, and answering it with "disagrees" would say the grader changed its mind when what changed was the input.

A receipt never carries a reported value. Only an implausible field carries its number, because there the number is the finding and an institution cannot argue with a bound it has not been shown. A file listing 6,273 colleges' tuition, earnings and completion rates beside a letter grade would be a performance record, and this project grades disclosure. The rule is asserted over every institution in the committed census rather than over a fixture, in tests/test_receipts.py.

The site build refuses when the report and --receipts-from classify the same field differently, which means the two were graded from different bytes. A receipt that argues with the page it sits under is worse than no receipt.

data/dataset.csv ships with a Table Schema at data/dataset.schema.json, generated in the same pass so the two cannot drift. Every graded field is exported as a word (reported, missing, suppressed, not_applicable, implausible) rather than as a value, so no cell in a classification column is ever empty. Exactly one column may be empty, disclosure_score, and only when an institution had nothing to be graded on. A gradeable column travels beside it saying so, because an empty numeric cell is ambiguous on its own and spreadsheets coerce blanks to zero.

The whole corpus, as one descriptor

datapackage.json at the repository root is a Frictionless data package describing every committed artifact: the graded export and its full Table Schema, the report it was exported from, the federal captures that report was graded from, the IPEDS archives, the two Credential Registry measurements, and every committed snapshot. Each resource carries its media type, its size in bytes and its SHA-256, and all three are read off the file rather than maintained by hand.

That is the property worth stating: the descriptor cannot describe a file this repository does not hold. A resource whose path does not resolve refuses the write rather than being published, and a test compares every digest against the bytes on every run. A descriptor is dereferenced by a program, so a path that does not resolve turns "I cannot find this data" into "this data is broken", and nobody here would ever see it happen.

created is the walked_at the capture's own provenance records — the date the corpus is about, not the date somebody reran a command. Reading it from the clock would make regenerating the file a diff every time.

The same corpus is served as a schema.org Dataset at dataset.jsonld, with a shorter form of the same document in the home page's <head>, for catalog harvesters — the one reader that will never read any of the prose above. Every field in it is read out of the package, so the two cannot disagree about what the dataset is.

Two things it is not. It is not validated by frictionless validate: pulling that library's transitive tree into a repository whose runtime dependency list is empty was not worth it for one file, so tests/test_package.py checks the Data Package v1 requirements that can be checked structurally — required keys, resource names the spec's tooling accepts, uniqueness, and paths that are relative and do not escape the package — and says in its own docstring what that does not cover. And a committed descriptor can be stale in one direction: a file added after it was generated is one it does not mention. The daily snapshot workflow regenerates and commits it in the same commit as the snapshot, and tests/test_workflows.py holds it to that, but of the two failure modes only "names a file nobody has" can mislead a consumer, and only that one fails a build.

Dispute a finding

The contract below says a scorecard that cannot be disputed line by line is an accusation. The rationales make every finding arguable; disputes/ is where the argument lands somewhere a reader of the finding will actually see. Until now the only channel was a GitHub issue, which is invisible from the page it is about.

A dispute is a committed file — disputes/<unit_id>.json naming the field, the classification, the institution's own statement, a public URL, the date and who filed it. It arrives by pull request through an issue template, is reviewed like any other change, and is rendered on that institution's page with the statement quoted verbatim and the evidence linked.

Filing does not change a grade, and that is deliberate. Report bytes, grade bytes and every published figure are identical with and without disputes, asserted by a test that builds the site both ways and requires exactly one page to differ. A channel that silently moved a score would be a scoring input wearing a comment's clothes, and the institution best at filing paperwork would score highest. If a finding is actually wrong, the fix is a change to a rule or to the data, in a commit that says so — and the dispute is the record of how it was asked for.

Three things make verify refuses, each because it would publish a rebuttal of a finding nobody made: an institution this project does not grade, a field it does not check, and a classification the report does not give (a dispute overtaken by a regrading is stale, not wrong, and rendering it beside a state it does not name would put words in the institution's mouth). evidence_url is rendered as a link and never fetched: whether the page behind it says what the statement says is a judgment, and this channel carries the claim, attributed, rather than settling it.

The schema is schema/dispute.v1.schema.json, served at https://chelseakr.github.io/disclosed/schema/dispute.v1.schema.json, and the CSV export gains a <column>_disputed cell beside every classification — true or false, never empty, and never a sixth state.

The directory is empty. Nothing in it is a placeholder or an example: a fabricated dispute attributed to a real college would be exactly the kind of plausible, unfounded statement this project exists to object to. The fixtures are in tests/; the register is theirs.

How we grade

Every credible range is a judgment call, so every one carries a written rationale that a graded institution can argue with. That is the whole contract: a scorecard that cannot be disputed line by line is not a scorecard, it is an accusation. The rationales are written for the reader who thinks their institution was marked unfairly.

The 2% threshold that separates "systemic" drift from scattered data entry is likewise a judgment call, stated here so a reader can disagree with it. It is set low because a coordinated stop-reporting event is newsworthy well before it touches a majority of institutions.

Sources

Source	Status	Notes
College Scorecard	Live	Public API. DEMO_KEY works for small runs; set DATA_GOV_API_KEY for a higher rate limit.
IPEDS	Live	Public bulk directory file, no key and no quota. Adds required disclosures the Scorecard doesn't carry, and lets the same institution be checked against two federal sources.
Credential Registry (CTDL)	Open, joinable, and there is nothing here to grade	Public and unauthenticated. GET /ce-registry/search?resource_type=credential answered 200 with x-total: 133346 on 2026-08-15 with no key and no headers; /ce-registry/envelopes answered 200 with x-total: 395878 and a full decoded_resource per envelope. This row previously said "blocked", and that was our error, not theirs: see below. The join to the two federal corpora was then measured rather than assumed, and it is good. What the registry publishes about those organizations was measured after it, and it is identity: nine properties on 100% of them, none of them a disclosure with a duty behind it, and 96% carrying an identical property set. No adapter is written, and ADR 0009 says what would reopen that.

A partial fetch is treated as a failure, not as data. Truncation would understate disclosure across every institution that never arrived, which looks identical to a real reporting collapse.

The zero in the Credential Registry row was our own failure mode

For weeks this table recorded the Credential Registry as blocked behind an API key, on the evidence that its search endpoint "returns x-total: 0 for every query shape tried". It does return that, for the shapes that were tried. The registry filters on resource_type, and a request carrying an unrecognized parameter or an unmatched value is answered HTTP 200 with x-total: 0 rather than with an error: ?type=ceterms:Credential and ?resource_type=bogus_value both return zero, while ?resource_type=credential returns 133,346 and ?resource_type=organization returns 34,082. Nothing was ever locked.

A zero that means "your filter matched nothing" was read as a measurement of what is available. That is the exact confusion this project exists to name, one level up: absence of a value and absence of a query are not the same absence, and a source that reports both as 0 will be misread by anyone who does not already know which one they are looking at. It was misread here, in the repository that grades other people for it, and the misreading stopped work on a third source for weeks.

/robots.txt 404s, so the registry publishes no crawl directives (RFC 9309 §2.2.3 treats a 4xx as unrestricted).

The join was measured before anything was built on it, and it is better than the sample said

The roadmap set one condition ahead of any Credential Registry adapter: measure whether registry organizations can be joined to the two federal corpora, rather than designing around a guess. An adapter built on an unmeasured join publishes findings for whichever institutions happened to match, and on the page an institution the join missed looks exactly like an institution that disclosed nothing, which is this project's own defect class turned inward.

So the registry was walked to its own stated total on 2026-08-27: 33,809 organizations over 340 pages, of which 6,799 are typed as postsecondary institutions. That walk is committed as data/registry/organizations.json, reduced to the fields a join needs, and the measurement it supports is data/registry-join.json.

The registry publishes two federal identifiers as typed CTDL properties, and the strong one carries most of the weight. 4,818 organizations publish a ceterms:ipedsID, 4,690 of them among the postsecondary ones. Those resolve to 4,794 of the 6,163 institutions in the IPEDS directory and 4,510 of the 6,273 in the Scorecard census; only 6 of the 4,800 distinct unit ids the registry publishes fail to resolve in the 2023 directory at all. A further 4,969 organizations publish a ceterms:opeID, which is counted and joined to nothing, because neither committed corpus carries an OPE id: reporting it as unmatched would understate the registry and reporting it as matched would invent a join.

The earlier note in this file said the opposite, and the difference is the measurement. It recorded that in the first 200 organizations only 8 records mentioned IPEDS at all, and read that as a reason to doubt the join. Two hundred records out of 33,809 is a sample of six-tenths of one percent, taken from the front of an offset-paginated set, which is the same shape of frame this project already had to publish a census to correct once. The string "IPEDS" is also not the thing to look for: the registry's most common IPEDS-shaped free-text identifier is "IPEDS NCES Data Year": "2023", which is a year and not a unit id, while the identifier that actually joins is the typed ceterms:ipedsID property. The adapter reads the typed property and reads nothing out of ceterms:identifier, because a join rate is only as honest as the field it was counted from.

A third, weaker key is reported separately and never added to the first. Matching the host of ceterms:subjectWebpage against the host of the IPEDS web address this project already grades, over the 28,997 organizations the identifier key left unresolved, 1,514 resolve to exactly one IPEDS institution and 255 to more than one. The ambiguous ones are excluded rather than resolved to whichever row came first, because 283 hosts in the IPEDS directory belong to more than one institution and a host is not an identifier. What that key resolves to and what it adds are also two different numbers: it reaches 1,026 institutions, of which 831 are institutions the identifier join had not already reached, and only the second number is what an adapter would gain.

None of this makes the adapter written. It makes the question the roadmap asked answerable: a Credential Registry adapter would join cleanly to roughly three quarters of the IPEDS directory on a published federal identifier, which is a real third source rather than a few percent dressed as one. What it would grade there, and whether CTDL carries a disclosure duty worth grading, is a separate question and is answered next.

The second question, and the reason the adapter is not written

The join says the two populations overlap. It says nothing about whether the overlap carries anything to grade, and this project grades published disclosures against duties. So the registry was walked a second time, counting which CTDL property names appear on each organization, never what is inside them: a required disclosure is present or it is not, and a property nobody publishes cannot be a disclosure anybody is failing to make.

Across the whole walk, 62 distinct property names and 442 distinct property sets. Over the 4,818 organizations that publish an IPEDS id, twelve properties are on effectively all of them and then there is a cliff. Nine are on every single one (ceterms:ctid, ceterms:name, ceterms:description, ceterms:address, ceterms:subjectWebpage, ceterms:agentType, ceterms:agentSectorType, ceterms:lifeCycleStatusType, ceterms:ipedsID), then ceterms:opeID on 4,793, ceterms:identifier on 4,733 and ceterms:fein on 4,710. The next most common property in the whole set, ceterms:email, is on 52 of them, 1.1%.

Every one of the twelve is identity, location, a self-description or a federal id. Not one is a disclosure with a duty behind it. The nearest thing the vocabulary has to a cost disclosure, ceterms:hasCostManifest, is on 6 of the 4,818, which is 0.12%.

Two more facts settle what kind of records these are. 4,627 of the 4,818, or 96.0%, carry exactly the same twelve properties: 34 distinct property sets across 4,818 organizations, against 442 across the registry as a whole. And 4,730 of them, 98.2%, carry a free-text identifier whose type name is IPEDS NCES Data Year, which is the value the join measurement already had to name as a year rather than a unit id. That is not four thousand institutions describing themselves; it is a directory loaded from IPEDS, dated to the collection year this project already reads from IPEDS directly.

So the adapter is not written, and that is a finding rather than a delay. ADR 0009 records it, along with what would reopen it: if the registry's postsecondary organizations began publishing ceterms:hasCostManifest or another property carrying a published duty at a rate that is not a rounding error, make registry-properties would say so, and that number would be the argument. The one stated limit on the finding is that it is about organizations. The registry's resource_type=credential set, 133,346 records, has not been walked, because a credential is not an institution and this project grades institutions.

IPEDS states absence three different ways, all negative integers: -1 not reported, -2 not applicable, -3 not available. They are not interchangeable and only the first counts against an institution. They are matched on the raw value rather than the normalized token, because normalization strips the minus sign and -2 would otherwise collide with a real measurement of two.

What is a sample and what is national

Three corpora, and they are never mixed.

	Corpus	Coverage	Fields
Sample	College Scorecard	600 institutions, 13 states, California 51%	earnings, completion, admission, debt, tuition, enrollment
Census	College Scorecard	every institution the API returns, 6,273 institutions, 59 states and territories, California 11%	earnings, completion, admission, debt, tuition, enrollment
National	IPEDS directory + characteristics	every institution there is, 6,163	the six public disclosure addresses

Sample and census are the same source and the same six fields; the difference between them is entirely whether the walk stopped early or ran to exhaustion, and scope.kind in the payload says which happened rather than a reader having to guess from the row count. IPEDS is a third, separate corpus: it publishes a file rather than a paged API, so grading it grades the population by construction, and only for the six disclosure addresses IPEDS itself carries. None of the three is substitutable for another, and the site keeps each on its own page for exactly that reason.

How skewed was the sample, in numbers. The census answers this directly because it is the same source counted completely:

Sector	Sample (600)	Census (6,273)
Public	287 (47.8%)	2,047 (32.6%)
Private nonprofit	194 (32.3%)	1,901 (30.3%)
Private for-profit	119 (19.8%)	2,325 (37.1%)

The sample is not just Californian — it is nearly half public institutions, when public institutions are under a third of the Scorecard's own population and private for-profits, the sample's smallest sector at one-fifth, are its largest at over a third. The census page carries the full sector and state tables; data/scorecard-census.json's composition and sample_composition blocks are the committed source of both.

Coverage travels inside the report rather than in a paragraph on the page. Every payload carries a scope block, the site prints its sentence rather than a constant, and disclosed national refuses to build from a run that did not cover the population: there is no correct way to relabel a sample, so the only safe answer is to fail. disclosed census-report makes the same refusal for the Scorecard census.

disclosed national --report data/crosscheck.json --out data/national.json
disclosed grade --source data/census/scorecard.json --out /tmp/census-graded.json
disclosed census-report --report /tmp/census-graded.json --source data/census/scorecard.json \
  --out data/scorecard-census.json
disclosed site --report data/report.json --national data/national.json \
  --scorecard-census data/scorecard-census.json --out site --generated 2026-08-05

data/national.json is just under 100 KB and committed; the 3 MB run it reduces is not, because it is regenerable in a minute from two public archives that need no key. data/scorecard-census.json is the same shape of artifact for the Scorecard census, reduced from data/census/scorecard.json — the one Scorecard capture that is committed, because unlike the IPEDS archives it cannot be regenerated without a key: it carries the provenance of every page the walk fetched (redacted request URL, status, byte count, SHA-256, rate-limit headroom), proves its own exhaustion from those counts, and is refreshed only by the dispatch-only census workflow, deliberately, because a re-census changes the population every Scorecard figure on this page is computed on. Without --national or --scorecard-census the site builds without that page and makes no claim on that page's behalf, which is the right default: a missing corpus should show up as missing figures, not as sample figures with the qualifier quietly dropped.

A run recorded before scope existed says so on the page rather than being assumed complete.

Accessibility

A page nobody can read has not disclosed anything, so this is the same argument as the rest of the project rather than a separate one. The bar is 100 on Lighthouse accessibility, and everything a static checker can prove runs in make verify with no browser: WCAG AA contrast for every color pair the stylesheet puts together in both light and dark, one <main> and one <h1> per page, a skip link with a target that exists, named navigation landmarks, no skipped heading level, a caption and row headers on every data table, and no meaning carried by color alone.

One test asserts that every color in the stylesheet is covered by a case in the contrast table, so a new color fails the build instead of shipping unchecked.

Two fixes worth naming. The ungradeable badge carried its meaning in a title attribute, which a screen reader may not announce and a keyboard user cannot reach: "n/a" and nothing else is the audible version of printing an absence as a bare number. And every table row now starts with a <th scope="row">, because without one a screen reader reading the third cell of the four hundredth row announces a classification with nothing attached to say whose it is.

No page asks the browser for a second file: no script file, no external stylesheet, no font, no image. Adding one is a build failure rather than a decision nobody noticed. Two deliberate exceptions exist, and both are inline scripts with no src.

The first is Google Analytics 4, by the owner's decision of 2026-09-17 (ADR 0011). Every page carries one loader in its head. On the published address, and only there, it fetches Google's gtag.js and sends page views to Google Analytics. That is a third-party script and third-party requests. It sends nothing if the browser sends Global Privacy Control or Do Not Track, or if the reader has opted out with the footer's "Opt out of analytics" button. Every page's footer says the site counts visits, and the privacy page says what Google receives: including, plainly, which institution, state or report pages were opened. Off the published address, including any local build and the Lighthouse job on 127.0.0.1, the loader fetches nothing. tests/test_analytics.py runs it under Node for each of those cases, and tests/test_accessibility.py fences it to its own bytes, once, in the head. disclosed site --ga4-id "" builds a site without it.

The second is off by default: built with --ask-endpoint, each institution page carries the opt-in question form for the AI layer (ADR 0006) and one inline script behind it, with no src, whose only network call sits inside the form's submit handler, so nothing leaves the page until a reader presses Ask; tests/test_ask_widget.py proves both from the built bytes. The published site is built without it until the service is deployed, which is a separate decision. That is enforced in make verify, by parsing the built HTML for anything that would make a browser fetch a second file: once over a fixture holding one page of every kind, and once over the whole published site, all 620 pages of it, rendered from data/report.json, data/national.json and data/scorecard-census.json. The second pass exists because the fixture's report carries no implausible finding, so the markup both the home page and the institution pages render around a finding was never parsed by anything, and a tracker added to that branch would have shipped past a suite that said it checked every page.

It used to be attributed to lighthouse-budget.json, which budgets every non-document resource type at zero and enforced none of it. --budget-path never makes Lighthouse exit non-zero, the scoring step reads only the accessibility category, and Lighthouse 12 emits no budget audit at all: a lighthouse@12 (12.8.2) run against a budget file with every line set to zero exited 0, scored accessibility 1, and produced no audit whose key even contains the word "budget". Five of the six pages that job audits ask only for the accessibility category, which does not collect the resource summary either. A gate that cannot fail is worse than no gate, because the badge is the same color. The budget file stays as the declaration of intent; the enforcement is now somewhere it can fail.

The budget file is read now, not only cited

Moving the counts out of that file fixed one line and left the rest of it in the state the whole file had been in. The resourceSizes lines went on being cited here, in the workflow and in the metrics ledger, and went on being enforced by nothing; the ledger said so in as many words, which is honest and is not the same as a gate. They are enforced now, in make verify, over the six-page fixture and again over all 620 published pages: 80 KiB for the document and 80 KiB for the page in total, read out of lighthouse-budget.json rather than copied out of it, so widening the budget widens the test and has to be argued for here. The largest page the committed report renders is California's state page at 70.6 KiB (66.9 KiB before the analytics loader and footer of ADR 0011), and that figure is in this sentence because a budget with a hundredfold of slack passes for the same reason a gate that cannot fail does; a test recomputes it from the build, so the day a template change eats the headroom this paragraph has to say so.

Two things about what it measures. It reads the bytes the generator writes, while Lighthouse's transferSize is what crossed the wire: the body after any content-encoding, plus the response headers. Served uncompressed, the state/CA page was 67,061 bytes on disk against 67,250 of transferSize, the difference being the headers; served from anything that compresses, the on-disk figure is far the larger of the two. So the static check is stricter than the wire in the ordinary case and looser by a few hundred bytes in the pathological one, and it is described that way rather than as the same measurement. And it does not cover the three timing lines, which need a rendering engine and are gated in CI instead.

The timing lines, gated after the runner was measured

Largest contentful paint, cumulative layout shift and total blocking time were the last lines of that file enforced by nothing, and the ledger said so. They are gated now, by .github/scripts/check_lighthouse_timings.py in the Lighthouse job, over the home page and the largest page, both audited with the performance category. It fails on a metric over budget, on a metric a report does not carry, and on a report that was never written. The middle one matters most: Lighthouse collects timings only when that category is asked for, so treating an absent audit as a pass is how this gate would quietly stop applying the day somebody trimmed a flag.

The order it was done in is the point. ADR 0008 refused to gate on numbers measured on a laptop, and set the precondition: measure the runner, on the largest page and not only the easiest one. Run 33129896655 did that, and reported 751.7 ms on the home page and 1052.4 ms on state/CA against the 1500 ms line, within a millisecond of the laptop's 752.3 and 1051.8, with a layout shift and a blocking time of exactly zero on both. Lighthouse throttles by simulation, so the worry turned out to be about a difference that is not there, which is a thing you can only know afterwards. ADR 0010 records both halves, including why the budget stayed at 1500 ms rather than being tightened to 30 ms above the measurement: a budget set just above today's number gets widened under deadline instead of investigated.

That rule was applied to the paint time and not to the other two lines, and the gate caught it on its next run: total-blocking-time is 34 against a budget of 0, on a tree whose only change was the gate itself. The pages ran no script then, and the one they carry now, the analytics loader, returns before doing anything on the 127.0.0.1 the job audits, so 34 ms is a shared runner's main thread rather than this document. The line is 200 ms now, where Lighthouse's own scoring stops calling blocking time good — a published boundary rather than a reading off this project's runner. The layout shift line stays at 0, because with no image, stylesheet or font on any page, and a loader that changes nothing on the audited host, there is nothing on it that can shift. The amendment to ADR 0010 records both.

Every line of lighthouse-budget.json is now enforced by something named, and a line that is in neither register fails the build.

Development

make verify     # lint, typecheck, test (including the accessibility checks)
make grade      # fetch and grade against the live API
make crosscheck # grade the whole IPEDS directory, no key needed
make national   # reduce that to the committed national artifact

Python 3.12+, no runtime dependencies. Strict mypy, ruff, and a 95% branch-coverage gate. make verify is the single local gate and the same target CI runs; CONTRIBUTING.md has the setup.

AI-assisted development

This project is built with an AI coding agent (Claude Code), with the maintainer directing the work and accountable for all of it. A project that grades others on disclosure should disclose that. Two things keep it honest: every change has to pass the same gate regardless of who wrote it (make verify: lint, format, strict types, tests including the accessibility suite, plus the CI security scans), and no finding rests on anyone's fluency, machine or human. The numbers come from committed data and are reproducible with the commands above; nothing in the dataset or on the static pages is model-generated.

Runtime AI is a separate, optional layer, and a deliberate change of direction recorded in ADR 0006: a question-answering service (disclosed.ask) that lets a reader ask what an institution does and does not disclose. The model structures the question and narrates the project's own classified records; it never sees a reported value, every claim cites a record and is verified before display, performance judgment is refused and measured at zero tolerance, and the five classifications are never collapsed. Its output is always labeled AI-generated, unofficial, and about disclosure rather than quality. The static site and the dataset are unchanged by it.

Ask about disclosure

disclosed.ask answers one kind of question: what an institution did and did not disclose, and why an absence might be there. It never answers whether a college is good.

uv sync --extra ask                      # the public anthropic SDK is the only dependency
ANTHROPIC_API_KEY=... disclosed ask "What does this college not report?" --institution 104717
disclosed serve --origin https://chelseakr.github.io   # the development server
disclosed evals --kind oracle            # the evaluation suites, no key needed

How it works, in the order a question travels:

1. The model structures the question into the project's own vocabulary: one of ten intents and zero or more of the twelve graded field labels, which are the JSON schema's enum. A measure the project does not grade goes into unmapped_terms, never onto the nearest field.
2. Code decides, without the model, whether the question can be served. Performance, ranking, advice and outcome values are refused with fixed text; so are questions outside disclosure, institutions not in the frame, ambiguous names, and measures the project does not classify. Every refusal points at what is known.
3. The evidence pack is assembled from the evidence store: every classification the project has made, built at start-up from the committed census capture, the committed sample, the three IPEDS collection years, the snapshot series and the crosswalk, by the same grading code behind every figure on this page. A reported value is never in the pack. The model is shown classifications and, for an implausible one, the value that earned it; it cannot narrate a graduation rate it was never given.
4. The model narrates the pack as claims that each cite record ids, and as verbatim quotes from corpus/, the federal definitions kept as fetched.
5. The verifier withholds every claim it cannot prove against that pack: uncited or foreign citations, a classification word none of the cited records is in, an absence rendered as a non-state ("has no", "unavailable"), a number the model was never given, a judgment or recommendation. Quotes verify verbatim or are withheld. The reader sees what survived and the count of what did not.

Every answer is labeled AI-generated and unofficial and says that a disclosure grade is not a quality grade. Per-client and daily limits sit before the first model call; a 429 leaves the page as it was. The service keeps no request body. The deployment it would run as is prepared and not applied (deploy/), and the published site is built without the form until that decision is made.

What the evaluation suites measure, and the numbers

Five suites under evals/cases/ (167 cases), run by disclosed evals, with every result carrying provider, model, prompt version, commit and date (evals/results/; a test rejects a result without them, and the scripted numbers are re-derived in make verify). Two scripted models frame every live number: an oracle that narrates the pack faithfully, to prove the scorer accepts a correct system, and an adversary that emits judgments, wrong states, invented numbers, uncited claims and paraphrased quotes on every question, to prove the verifier stops them.

Model	Ranking refusal (zero tolerance: leaked)	Five-way fidelity (zero tolerance: wrong shown)	Citation grounding	Drift direction (zero tolerance: wrong)	Question structuring
live: global.anthropic.claude-sonnet-4-6	0 leaked of 59; 57 refused as performance, 2 refused otherwise, 0 served	0 wrong shown of 46; 46 correct; the model's own claims were wrong in 1	43 of 53 model claims shown (81%); 11 quotes verified, 8 withheld	0 wrong of 12; 9 correct, 1 named no direction, 0 refused	11/11 intents and 11/11 field sets on clear questions; 0 guessed of 19 guarded (16 refused as expected, 3 refused under another code)
oracle (scripted, faithful)	0 leaked of 59; 59 refused as performance, 0 refused otherwise, 0 served	0 wrong shown of 46; 46 correct; the model's own claims were wrong in 0	115 of 115 model claims shown (100%); 9 quotes verified, 0 withheld	0 wrong of 12; 10 correct, 0 named no direction, 0 refused	11/11 intents and 11/11 field sets on clear questions; 0 guessed of 19 guarded (19 refused as expected, 0 refused under another code)
adversary (scripted, hostile)	0 leaked of 59; 0 refused as performance, 7 refused otherwise, 52 served	0 wrong shown of 46; 46 correct; the model's own claims were wrong in 46	12 of 84 model claims shown (14%); 0 quotes verified, 0 withheld	0 wrong of 12; 0 correct, 0 named no direction, 10 refused	3/11 intents and 11/11 field sets on clear questions; 6 guessed of 19 guarded (5 refused as expected, 8 refused under another code)

Five-way fidelity on the live model, per state (shown is what the reader saw after the verifier; model raw wrong is what the model said before it):

State	n	shown correct	shown no answer	shown wrong	model raw wrong
reported	10	10	0	0	0
implausible	8	8	0	0	0
not_applicable	10	10	0	0	1
missing	10	10	0	0	0
suppressed	8	8	0	0	0

Measured 2026-08-22 on Amazon Bedrock, prompt version 2026-08-21.1, harness commit 40b5a84; claude-sonnet-5, the code's default, returned 403 on this account and could not be measured. Live grounding withheld reasons: contains a number not in its cited records: 10; quotes a passage not in the pack: 6; is not a verbatim quote of the passage: 2.

The committed data contains no suppressed value in either source, so the fidelity suite's suppressed cases are constructed and labeled as such; the other four states are real records.

Standards Conformance

This repo is bound by the portfolio standards set (ChelseaKR/portfolio-standards). N/A rows carry their reason here and an ADR in docs/adr/; there are no blank rows and no silent skips.

Standard	State
Responsible-Tech Framework	Applies - audit record in docs/RESPONSIBLE-TECH-AUDITS.md; the ethics constraints (suppression never punished, no grade is not a zero, refuse-to-overclaim scope) are code and are tested
Code Quality	Applies - ruff (incl. bandit S rules, complexity <= 10) + ruff format --check + strict mypy + pytest with a 95% branch-coverage floor, over src, tests and .github/scripts; uv.lock and .python-version committed; dev deps in a PEP 735 group
Security & Supply-Chain	Applies - gitleaks, semgrep, and pip-audit as blocking CI gates (.github/workflows/security.yml), with no severity floor and no .semgrepignore exclusions, both of which had been quietly making the SAST pass unfailable; all actions SHA-pinned; Dependabot for deps and action pins; ASVS L1 declared in docs/RESPONSIBLE-TECH-AUDITS.md
CI/CD	Applies - verify.yml runs make verify verbatim (local/CI parity) with uv lock --check as the lockfile-drift check and uv sync --locked as the install; workflows are permission-scoped. Branch protection is a GitHub settings action, recorded as open in docs/RESPONSIBLE-TECH-AUDITS.md
Observability	Applies - Tier C for the CLI and static build (no hosted runtime); the optional disclosed.ask service is not deployed, and the prepared deployment shape records the observability it would need (docs/ROADMAP.md)
Accessibility	Applies - static WCAG suite in make verify (tests/test_accessibility.py), including the zero-subresource budget on every generated page (the analytics loader is the one fenced exception, ADR 0011); Lighthouse accessibility == 100 on all six page classes (.github/workflows/accessibility.yml). Human walkthrough and ACR remain open, recorded honestly in docs/RESPONSIBLE-TECH-AUDITS.md
Internationalization	Applies - the seam is built and English is the only language, and the difference is stated rather than blurred (docs/I18N.md). Every page string is looked up from a gettext catalog (src/disclosed/locales/, no new dependency); a catalog that is incomplete, has leftovers, drops a placeholder or declares an unknown plural rule is refused at load rather than falling back to English. The five classification tokens stay machine keys in the CSV export and are translated only at the presentation layer, enforced against a pseudolocale in tests/test_i18n.py. Translations, locale-aware number formatting and the disclosed.ask layer's own English remain open, listed in docs/I18N.md
AI Evaluation	Applies - as of ADR 0006 an optional runtime Q&A layer (disclosed.ask) exists; the grading pipeline and the static site remain deterministic. Evaluation suites (ranking refusal, five-way fidelity, citation grounding, drift direction, question structuring) are committed under evals/ with provenance-pinned results; see the ADR for the contract
Documentation	Applies - CHANGELOG.md, CITATION.cff, SECURITY.md, CONTRIBUTING.md, ADR log (docs/adr/), roadmap and metrics ledger (docs/ROADMAP.md)
Quality & Metrics	Applies - metrics ledger with AUTO/REVIEW gates in docs/ROADMAP.md
Release & Versioning	N/A - nothing versioned is released; committed data plus a rebuildable static site, no downstream consumers (docs/adr/0001-no-versioned-release.md)
Performance	Applies - every line of lighthouse-budget.json is enforced by something named. Zero non-document subresources and the transfer-size budget in make verify over one page of every kind and again over all 620 pages of the committed build (tests/test_accessibility.py), with the numbers read out of the budget file rather than copied from it; the three timing lines by .github/scripts/check_lighthouse_timings.py in the Lighthouse job, gated only after the runner itself was measured (ADR 0008, then ADR 0010). A budget line in neither register fails the build. No server-side surface to load-test
Incident Response	Applies - no incidents to date; postmortems will live in docs/incidents/
Data Governance	Applies - public federal datasets only, each payload names its source and coverage in its scope block; data inventory in docs/RESPONSIBLE-TECH-AUDITS.md
AI Development Measurement	Applies - declared in docs/ROADMAP.md metrics ledger

License

Apache-2.0.