Repository navigation
Make the example profile comments and the Lite file name match the code - #65
Draft
jwrosewell wants to merge 1 commit into
Draft
jwrosewell wants to merge 1 commit into
jwrosewell wants to merge 1 commit into
Conversation
- The IP Intelligence data file update example now names the LowMemory profile for the pipeline that downloads and reloads the Enterprise data file, rather than taking the engine's default. - The Device Detection getting started example's comment said it uses the low memory profile, but the code sets MaxPerformance. The comment now says MaxPerformance, and why it suits the free Lite data file. - The IP Intelligence performance example's header said worker processes share a memory mapped data file under LowMemory, as if that were the default. It now says the default MaxPerformance loads the whole file into each worker, and that -p LowMemory lets the workers share the operating system's file cache. - The IP Intelligence package readme named the Lite data file 51Degrees-LiteIpiV41.ipi, which is CI's name for it. The data repository's scripts save it as 51Degrees-LiteV41.ipi.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Four places in the examples and readmes say something about performance profiles or data files that the code does not do. All line numbers in this section are on rc at 0507eef.
datafileupdate_console.pylines 185 to 203, the pipeline that downloads and reloads the Enterprise file sets noperformance_profile, so it uses the engine's default. That is LowMemory on rc, and #62 changes the default to Balanced, which would hold more of the large Enterprise file in memory. The same example's other pipeline, which reads the free ASN file when there is no licence key, already names LowMemory (line 118).gettingstarted_console.pylines 68 to 74 the comment says "We use the low memory profile" and the code setsperformance_profile = "MaxPerformance".performance_console.pylines 36 to 40 says that with LowMemory the "memory mapped data file is shared between the processes". The example's default is MaxPerformance (runat line 133 and the-poption at line 217), so the sharing only happens when-p LowMemoryis given. The C library also does not memory map the file, as no source undercxx/src/commonorcxx/src/ipicallsmmap,MapViewOfFileorCreateFileMapping.fiftyone_ipintelligence/readme.mdline 28 gives51Degrees-LiteIpiV41.ipi. The data repository's download scripts save the file as51Degrees-LiteV41.ipi, from the linkcxx/data/ip-intelligence/51Degrees-LiteV41.ipi.link, and the data repository's readme lists it under that name. The examples' constantLITE_DATAFILE_NAMEand the on-premise package's readme already use51Degrees-LiteV41.ipi.51Degrees-LiteIpiV41.ipiis the name of CI's copy of the same file.ip-intelligence/ci/fetch-assets.ps1fetches the CI asset under that name and links it intocxx/data/ip-intelligence/51Degrees-LiteV41.ipi, and the on-premise unit tests read the CI copy atassets/51Degrees-LiteIpiV41.ipi(fiftyone_ipintelligence_onpremise/tests/constants.pyline 43). So the line inip-intelligence/readme.mdthat says the unit tests default toassets/51Degrees-LiteIpiV41.ipiis correct and is not changed.Change
datafileupdate_console.pysetsperformance_profile = "LowMemory"on the update pipeline, with a comment saying the Enterprise file is large so LowMemory reads it from disk. The example now keeps LowMemory whatever the engine's default is.gettingstarted_console.pysays it uses MaxPerformance, which loads the whole data file into memory, and that this costs little memory because the free Lite file is about 35 MB, and gives the fastest detections. The profile itself is not changed.performance_console.pysays that with the default MaxPerformance each worker loads the whole data file into its own memory, and that with-p LowMemoryeach worker reads the file from disk, so the operating system's file cache holds one copy for all the workers.fiftyone_ipintelligence/readme.mdnames the Lite file51Degrees-LiteV41.ipi.Lite file examples and every other example keep the profile they name.
Verification
From the repository root, on Windows with Python 3.14.7, I byte-compiled the three changed examples with
python -m py_compile, which passed.I then built the IP Intelligence packages from this checkout into a virtual environment, with the native module built locally by
pip install -e .and the free ASN and Lite data files downloaded bycxx/data/get-data-files.ps1, and ran the example tests that import the two changed IP Intelligence examples. Fromip-intelligence/fiftyone_ipintelligence_examplesgave
3 passed, 17 deselected, beingtest_datafileupdate_console,test_performance_consoleandtest_onpremise_performance_console. The whole oftests/test_examples_without_settings.pygave7 passed, 1 skipped, the skip being the mixed example, which needs Device Detection installed.The update pipeline in
datafileupdate_console.pyonly runs with an Enterprise licence key, which the tests do not have, so the tests cover the example's path without a key and not the changed line. The Device Detection getting started change is a comment, and I did not build Device Detection for it.Remaining