[jwrosewell]

For #52.

The problem

The elements PipelineBuilder adds by default, and the cloud engines, accepted every evidence key, so a pipeline accepted every header, If-None-Match included, whatever its engines read. The client-side endpoints' ETag and Vary header (#59) are built from what the pipeline accepts, so Vary named every header, and only #59's list of revalidation headers kept the ETag of a revalidation the same. A cloud pipeline also sent every header and cookie in the evidence to the cloud service, whether or not the service lists it.

• SequenceElementEvidenceKeyFilter and JavaScriptBuilderEvidenceKeyFilter defined filter(), which nothing calls, so the base EvidenceKeyFilter.filter_evidence_key(), which returns true, answered for them (sequenceelement.py, javascriptbuilder.py).
• SetHeaderElement, JSONBundlerElement and CloudEngine do not override get_evidence_key_filter(), so they took FlowElement's, which accepts every key (flowelement.py). Three elements the pipeline builder adds accept every evidence key #52 named the first three. The JSON bundler is the same case, and the builder adds it with the JavaScript builder. CloudEngine is the base of DeviceDetectionCloud, HardwareProfileCloud, IpIntelligenceCloud and LocationCloud, so every cloud pipeline accepted every key through them.

The change

As James decided on 7 November, every element accepts only the evidence that can change its results.

• The sequence element accepts query.session-id and query.sequence. The result changes as the sequence grows, so they belong in the ETag.
• The JavaScript builder accepts query. keys, header.host and header.protocol, tested with key.startswith("query.") where it was "query" in key, which header.x-query also passed.
• The set header element and the JSON bundler read only other elements' results, so each accepts nothing.
• CloudEngine reads only the cloud request engine's results, so it accepts nothing, and so do the four cloud engines built on it. The .NET cloud engines already accept nothing (DeviceDetectionCloudEngine.cs line 74).
• Both filter methods are now filter_evidence_key, so they are called.
• FlowElement.get_evidence_key_filter(), the default for an element that does not give its own filter, accepts no evidence where it accepted every key. specifications#103 says a base Flow Element's default MUST accept no Evidence. .NET and Java already make every element give its own filter, and Go's default accepts none.

The JavaScript builder reads the query values through its own filter (javascriptbuilder.py line 401) and the host and protocol directly (lines 363 and 372), all of which it still accepts, and it used only the query. keys of what that filter returned, so the script it writes is unchanged. The sequence element reads only its two keys.

A cloud pipeline now keeps the evidence the cloud request engine accepts, being the keys the service lists at its evidencekeys endpoint, and the keys the sequence element and the JavaScript builder read. The cloud request engine sends the evidence the pipeline keeps, so it no longer sends the service other headers and cookies. The live service listed 138 keys on 8 November, 25 of them cookies.

Four cloud request engine tests relied on the old behaviour, because the stand-in service lists query.User-Agent alone and they gave other evidence. They now add a test element that accepts every key (tests/classes/accepteverykeyelement.py).

How to verify

Each red run is rc at 0507eef, or this branch before the fix, with only the tests added, and each green run is this branch with the same temporary workflow.

1. The four elements, red (run 42424500636, failure on all four jobs), with only the tests added (0849426). The temporary workflow ran on Ubuntu and Windows with Python 3.12, once on the branch and once with Set the client-side cache headers, with an ETag that is the same in every process and none after errors #59 merged in, so the Vary header itself could be checked. A default pipeline accepted If-None-Match, and with Set the client-side cache headers, with an ETag that is the same in every process and none after errors #59 merged in the Vary header named every header the request carried.

   E           AssertionError: True is not false : header.if-none-match
   E       AssertionError: Lists differ: ['Host', 'User-Agent'] != ['Accept', 'Accept-Language', 'Cookie', 'Ho[40 chars]est']
   E       AssertionError: '"a-tag"' is not None
   tests/test_tmp_vary.py::TemporaryVaryTests::test_vary_names_only_the_headers_the_elements_read Vary: Accept, Accept-Language, Host, User-Agent, X-Test
   FAILED tests/test_default_elements_accept.py::DefaultElementsAcceptTests::test_a_default_pipeline_accepts_only_what_its_elements_read
   FAILED tests/test_default_elements_accept.py::DefaultElementsAcceptTests::test_each_element_accepts_only_what_it_reads
   FAILED tests/test_default_elements_accept.py::DefaultElementsAcceptTests::test_only_the_headers_the_elements_read_are_named_for_vary
   FAILED tests/test_default_elements_accept.py::DefaultElementsAcceptTests::test_processing_sets_aside_the_evidence_no_element_reads
   

2. The four elements, green (run 42424663928, success on all four) at f938233. fiftyone_pipeline_core 289 passed on the branch and 301 with Set the client-side cache headers, with an ETag that is the same in every process and none after errors #59 merged in, with Vary: Host, User-Agent and the revalidation answered 304. fiftyone_pipeline_engines 9, fiftyone_pipeline_engines_fiftyone 34 and fiftyone_pipeline_translation 19 passed, and the examples' offline web tests 11, or 18 with Set the client-side cache headers, with an ETag that is the same in every process and none after errors #59.
3. The cloud request engine tests. The first full build at f938233 failed test_evidence_precedence_2, _4 and _5 and test_reserved_evidence_is_left_out_of_the_body (run 42425200931), for the reason above. With the test element (ab865bb), the new test_evidence_no_element_accepts_is_not_sent failed on rc (run 42426589244) and all 32 passed on the branch (run 42426594361).

   E       AssertionError: 'x-test' unexpectedly found in {'other': 'a cookie', 'x-test': 'a header', 'user-agent': 'iPhone', 'license': 'a-licence-key', 'values': 'device.ismobile,device.istablet'}
   =================== 1 failed, 31 passed, 6 warnings in 1.01s ===================
   

4. The cloud engines. With the tests alone (95ce1e4), both new tests failed (run 42438936386), and with the fix (78dbe3e) all 39 tests in the four test files the workflow ran passed (run 42438936661).

   E           AssertionError: True is not false : query.user-agent
   E       AssertionError: 'x-test' unexpectedly found in {'other': 'a cookie', 'x-test': 'a header', 'user-agent': 'iPhone', 'license': 'a-licence-key', 'values': 'device.ismobile,device.istablet'}
   =================== 2 failed, 37 passed, 7 warnings in 1.36s ===================
   

5. Full build. common-ci's nightly-pull-request.yml, run as a dry run from a temporary branch, built 6cb8928, this branch's head, merged with rc (run 42471441691). All 15 build and test jobs passed, on Ubuntu, Windows and macOS with Python 3.10 to 3.14.

The new tests are these.

• tests/test_default_elements_accept.py in fiftyone_pipeline_core.
  • test_a_default_pipeline_accepts_only_what_its_elements_read, a default pipeline with an element accepting header.user-agent, does not accept header.if-none-match, header.x-test, header.x-query, header.cookie, cookie.query or server.client-ip, and accepts the user agent, the two sequence keys, a query. key, header.host and header.protocol.
  • test_each_element_accepts_only_what_it_reads checks each of the four elements.
  • test_processing_sets_aside_the_evidence_no_element_reads checks that process() leaves header.if-none-match out of the evidence.
  • test_only_the_headers_the_elements_read_are_named_for_vary checks that of a request's headers only Host and User-Agent are ones the pipeline accepts, which is what Set the client-side cache headers, with an ETag that is the same in every process and none after errors #59's Vary names. With Set the client-side cache headers, with an ETag that is the same in every process and none after errors #59 merged in, a temporary test checked the Vary header itself, and that a revalidation got a 304.
• test_evidence_no_element_accepts_is_not_sent in tests/test_credentials.py checks that a header and a cookie the stand-in service does not list are not sent to it.
• tests/test_cloudengine_evidence.py in fiftyone_pipeline_cloudrequestengine.
  • test_a_cloud_engine_accepts_no_evidence checks query, header, cookie, sequence and server keys.
  • test_a_cloud_pipeline_sends_only_the_evidence_the_service_lists checks that with a cloud engine in the pipeline the user agent is sent and the header and cookie the service does not list are not.

An element that declares no evidence

Every element in this repository gives its own filter, apart from test elements and the cloud engines, which take CloudEngine's. No test changed for the new default. With the tests alone (06a5826), the two new tests in test_default_elements_accept.py failed and every other test passed (run 42471232966). With the change (6cb8928), all passed (run 42471233048). Both runs covered fiftyone_pipeline_core (291 tests), fiftyone_pipeline_engines (9), fiftyone_pipeline_engines_fiftyone (34), fiftyone_pipeline_translation (19), the four cloud request engine test files that need no key (39) and fiftyone_ipintelligence_translation (26), on Ubuntu with Python 3.12.

• test_an_element_that_declares_nothing_accepts_nothing checks the default rejects every key.
• test_an_element_that_declares_nothing_widens_nothing checks that adding such an element to a pipeline does not make it accept more.

A customer's own element that relied on the default no longer receives evidence that no other element accepts, and it now has to give its own filter, as the release note says. #59 keeps its list of revalidation headers, as an element of a customer's own can still accept them.

Release note

Each element now accepts only the evidence that can change its results. The sequence element, the JavaScript builder, the JSON bundler, the set header element and the cloud engines no longer accept every evidence key. So a pipeline keeps only the evidence its elements read, the client-side endpoints' Vary header names only those headers, and a cloud pipeline sends the cloud service only the evidence the service lists. An element of your own that does not set its own evidence key filter now accepts no evidence. Override get_evidence_key_filter to name the evidence your element reads.

Other drafts on the same files

None of the open drafts changes these files. #59 and #61 change web.py and evidence.py beside them.

What remains

• specifications#103, which this follows, is a draft stacked on specifications#99.
• fiftyone_pipeline_core and fiftyone_pipeline_cloudrequestengine change, so both are released, and so are the packages that depend on them.