Repository navigation
Read every key under its _51DEGREES_ name, and skip a live cloud test without one - #254
Merged
Merged
Conversation
…common-ci unnamed
Adds ci/key-names.Tests.ps1. It runs each product's run-integration-tests.ps1 and fetch-assets.ps1 against a stand-in common-ci, and checks that: - the resource key comes from _51DEGREES_RESOURCE_KEY_BESPOKE, with no TestResourceKey option - the data license common-ci exports as _51DEGREES_LICENSE_KEY is left as it is - the TAC data file, the CSV and the Enterprise IP Intelligence file are asked for with no key or address option - no CI script reads TestResourceKey, DeviceDetection or IpIntelligence - every tox.ini passes each _51DEGREES_ variable its package reads - the cloud license-key tests read _51DEGREES_LICENSE_KEY_BESPOKE, and the 51Did live test reads _51DEGREES_RESOURCE_KEY_51DID and _51DEGREES_LICENSE_KEY_51DID The CI script tests workflow now also runs when those files change.
common-ci now exports each _51DEGREES_ secret under its own name, fills _51DEGREES_RESOURCE_KEY and _51DEGREES_RESOURCE_KEY_51DID from _51DEGREES_RESOURCE_KEY_BESPOKE, and downloads every data file with _51DEGREES_LICENSE_KEY. No key reaches a script as an option. - run-integration-tests.ps1 in each product reads the resource key from _51DEGREES_RESOURCE_KEY_BESPOKE, not TestResourceKey, and fails naming it when it is not set. Device Detection no longer writes _51DEGREES_LICENSE_KEY from the DeviceDetection option, which cleared the data license common-ci exported. - fetch-assets.ps1 passes no key or address. Device Detection always asks for the TAC data file and the CSV, and IP Intelligence for the Enterprise data file, which common-ci downloads with the data license. - Every tox.ini passes _51DEGREES_*, so the tests get each key they read. - The cloud license-key tests read _51DEGREES_LICENSE_KEY_BESPOKE, a cloud license, and skip naming it when it is not set. _51DEGREES_LICENSE_KEY is the data license, which the cloud does not take on its own. - The 51Did live test reads _51DEGREES_RESOURCE_KEY_51DID and _51DEGREES_LICENSE_KEY_51DID. The Cloud 51Did workflow passes those two and no longer writes _51DEGREES_RESOURCE_KEY or _51DEGREES_LICENSE_KEY. - The CI readmes name the _51DEGREES_ secrets.
Every live cloud test with no key now skips, and its reason names each variable it reads. None fails for want of a key. Before, most failed, one raised at import, one built its example with an empty key, and the location getting started test passed without calling anything. - live_cloud.py (the same in each test folder) says what a test skips with, and no longer says there is no skip. - The Device Detection, IP Intelligence, location, cloud request engine and 51Did live tests call skipTest or pytest.skip where they called fail. - CI still fails a run with no key. Each product's run-integration-tests.ps1 fails naming _51DEGREES_RESOURCE_KEY_BESPOKE, and the Cloud 51Did workflow fails naming _51DEGREES_RESOURCE_KEY_51DID or _51DEGREES_RESOURCE_KEY_BASIC before it runs the tests. - ci/key-names.Tests.ps1 checks that no live test fails for want of a key.
cloud-51did.yml filled _51DEGREES_RESOURCE_KEY_51DID from the Bespoke resource key secret. The 51Did license key unseals only results made with a resource key of its own account, so the licensed redeem came back unreadable. On rc the same key came from the organization variable, and the redeem passed. The workflow now reads the organization secret _51DEGREES_RESOURCE_KEY_51DID, which has to hold the same value as that variable. ci/key-names.Tests.ps1 fails if any workflow fills the 51Did resource key from another secret, and the pipeline CI readme says where the key comes from.
If the _51DEGREES_RESOURCE_KEY_51DID secret is missing while _51DEGREES_LICENSE_KEY_51DID is set, the licensed redeem never runs. The step already fails then, naming the secret, before tox runs. A Pester case now runs the step's script with only the license key set, and fails if the step passes, runs tox, or doesn't name the secret.
The Basic tier step filled _51DEGREES_RESOURCE_KEY_BASIC from the FREE secret, and test_non_marketing_usage_gets_a_51did failed with it. The organization secret _51DEGREES_RESOURCE_KEY_BASIC now exists, so the step reads it by its own name. A Pester test checks every workflow fills it that way, as it already did for the 51DID key.
rc's integration test step now skips the tests marked live, with a warning, when there is no resource key, where this branch failed the step. The merge keeps rc's skip and reads the key from _51DEGREES_RESOURCE_KEY_BESPOKE in place of the TestResourceKey option, in all four product scripts: - The warning names _51DEGREES_RESOURCE_KEY_BESPOKE. - Device Detection runs the Selenium contract against the cloud example only when _51DEGREES_RESOURCE_KEY_BESPOKE is set. - IP Intelligence reports the key's products only when there is a key. - Pipeline runs the 51Did live tests on their own, after the others, only when _51DEGREES_RESOURCE_KEY_BESPOKE is set. The three tox.ini files that conflicted pass _51DEGREES_* and rc's PYTEST_ADDOPTS. ci/run-integration-tests.Tests.ps1 now sets and clears _51DEGREES_RESOURCE_KEY_BESPOKE rather than passing TestResourceKey. The key-names test that expected a throw without the key now expects the step to run and name the variable.
rc now runs the CI script tests when free-threaded-python.yml changes, and says so in the workflow's header comment, where this branch says the tests check each key name. The header now says both, and the paths keep both sides' entries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The workflows now pass their secrets to common-ci with
secrets: inherit, and the CI scripts and tests read every key from the environment under its 51DEGREES name. A live cloud test with no key now skips and names the variable, while CI still fails a run that has no key. This goes with the common-ci draft secrets-by-51degrees-names, in which common-ci exports each key from the organization secret of the same name, downloads every data file from the Distributor with _51DEGREES_LICENSE_KEY, and passes PYPI_TOKEN on under the name the publish asks for.What changed
Workflows
secrets: inheritin place of its mapping. Gone are DEVICE_DETECTION_KEY, IP_INTELLIGENCE_KEY, IPI_DATA_FILE_URL, the TestResourceKey mapping and the explicit PYPI_TOKEN mapping.CI scripts
secrets: inheritthat option is empty, so it would clear the data license common-ci exported.Tests
_51DEGREES_*in place of its own list. Before, several packages didn't pass names their tests read, such as _51DEGREES_RESOURCE_KEY_BESPOKE, _51DEGREES_CLOUD_ENDPOINT and _51DEGREES_DD_PATH, and none passed _51DEGREES_LICENSE_KEY_BESPOKE, _51DEGREES_RESOURCE_KEY_51DID or _51DEGREES_LICENSE_KEY_51DID.Tests
Run locally with Pester 6.2.0 on Windows. On the first commit of this branch, 33 of 104 failed. On the commit before the skip change, 23 of the 28 skip tests failed. On this head, 132 of 132 pass, and the repository's own ci/check-environment-variable-names.ps1 passes. Python's own tests weren't run, as pytest and tox weren't available there. The changed .py files compile.
No GitHub run has been made on this head yet. Before merging, run the CI against common-ci's secrets-by-51degrees-names. Only a run can show that every test gets its key through tox with
_51DEGREES_*(tox 4 takes glob patterns in pass_env), that _51DEGREES_LICENSE_KEY downloads the Enterprise IP Intelligence file under the name the tests expect along with the TAC hash file and the CSV, that the cloud takes _51DEGREES_LICENSE_KEY_BESPOKE on its own, and how the 51Did keys pair up.Before you upgrade
Open questions
How to merge
Merge after common-ci's secrets-by-51degrees-names has merged into main. These workflows call common-ci at main, and before that change its workflows still require a token secret and nothing passes PYPI_TOKEN on under the name the publish asks for. Then merge this into rc.