[miinhho]

What is this PR for?

Keep Linux Python CI dependency locks current and detect upstream compatibility problems without resolving dependencies on every regular CI run.

This PR retains Conda and uses conda-lock to manage Conda and PyFlink dependencies. Scheduled and manual runs resolve fresh dependencies within the supported version constraints, test the resulting locks, and propose changed locks for review only after validation passes. There is no automatic merge.

The existing test jobs are extracted into reusable workflows:

• backend-tests.yml contains the eight backend jobs that previously ran in core.yml.
• selenium-integration.yml contains the Selenium/Spark integration job that previously ran in frontend.yml.

The original workflows remain the entry points for regular CI. The default and Windows builds remain in core.yml; npm audit and Playwright tests remain in frontend.yml.

This separation lets regular CI and dependency refreshes use the same test commands, matrices, and services without maintaining duplicate test definitions. It also limits scheduled validation to the jobs that consume these locks. Regular CI installs committed locks, while refresh validation installs candidate locks without restoring cached Python environments.

The maintenance guide documents dependency updates from Linux, macOS, and Windows, including Docker linux/amd64, validation before submission, and failure reproduction.

Replaces #5584.

What type of PR is it?

Improvement

What is the Jira issue?

• https://issues.apache.org/jira/browse/ZEPPELIN-6739

How should this be tested?

1. Run the core and frontend workflows on this branch. Confirm that the backend and Selenium jobs install the committed locks without re-solving the test dependencies and pass their existing tests.

2. In a test fork where Python dependency refresh exists on the default branch, manually run that workflow against this branch. Confirm that it generates candidate locks, installs them without restoring cached Python environments, and passes both backend and Selenium validation. Fork runs must not publish update PRs.

3. Follow the local procedure in testing/README.md to generate and install locks. On macOS or Windows, use the documented Docker linux/amd64 command. Confirm that generated locks remain in the host checkout, then commit them and run regular CI to test those exact locks.

After merge, confirm that a successful Apache master refresh with changed locks opens or updates a draft PR. Resolution or test failures must leave the committed locks and any existing update PR unchanged.

Questions:

• License files: No.
• Breaking changes for older versions: No.
• Documentation: Yes. Updated testing/README.md.

[miinhho]

For reference, pandas uses scheduled lock-update PRs to keep its CI dependencies current: pandas-dev/pandas#71285

This is an example of the maintenance workflow, not an identical implementation. This PR retains Conda and uses conda-lock, and validates candidate locks with the backend and Selenium tests before creating a draft update PR.

[miinhho]

A note on the diff size: most of the added lines come from the generated conda-lock files replacing the previous Conda and PyFlink locks.

Scope	Files	Lines added	Lines removed
Dependency lockfiles	8	14,779	607
All other changes	22	1,173	726

The non-lock changes include CI workflows and actions, environment definitions, the refresh script, documentation, and a RAT exclusion for generated locks.

These counts do not represent entirely new logic. They also include extracting existing backend and Selenium jobs from core.yml and frontend.yml into reusable workflows. No application runtime source code is changed.