Development and documentation
Install and test a checkout
Use a virtual environment as described in Installation. Install an editable checkout with the test dependencies:
python -m pip install --editable '.[test]'
python -m pytest -q
The optional integration cases require Docker and test real SSH reconnect and PSI/J Slurm execution:
KKRUNNER_RUN_DOCKER_TESTS=1 python -m pytest -q tests/test_ssh_standalone.py tests/test_psij_slurm.py
Managed execution tests need POSIX process groups and Unix sockets (Linux or macOS). Docker SSH/Slurm tests run on Linux.
Build the documentation
The documentation uses Sphinx, the Read the Docs theme, autodoc and autosummary.
MyST renders the Markdown architecture page. These tools are supplied by the
docs extra.
python -m venv .venv
.venv/bin/python -m pip install -e '.[docs]'
make docs PYTHON=.venv/bin/python
make docs-test PYTHON=.venv/bin/python
Use the same Python for installation and make. The docs extra installs
MyST and the theme; installing only the core package does not include them.
Open build/docs/html/index.html. make docs treats Sphinx warnings as
errors; make docs-test executes the guide’s examples, including managed
reconnect and the remote protocol through LocalTransport. Both build against
the installed distribution and do not inject the source directory into imports.
The documentation CI repeats those checks on an isolated wheel installation,
uploads the rendered HTML as an artifact. A separate documentation workflow
deploys to GitHub Pages for v* tags or manual runs.
API pages are generated by autosummary from the actual public types. Edit the
source docstrings and guide pages, rather than generated files in
docs/_generated. Execution boundary and public contract describes ownership and module boundaries;
the test commands above describe how to verify a current checkout. Historical
review and measurement snapshots are kept separately in docs/remarks and
are excluded from the published manual.
Build distributable archives
This step is for sharing or publishing a package. Installing or developing a
checkout only needs the pip install commands above.
python -m pip install build
python -m build
The first command installs the build tool. The second creates a source archive
(sdist) in dist/ and builds an installable wheel from that archive.
The archive includes documentation sources, tests and Docker fixtures. CI
checks package metadata and tests the installed wheel using the unpacked
archive, so an editable checkout cannot hide missing distribution files.
Prereleases and publication
The installed version is 0.1.0b1. It does not promise a stable API or stable persisted development formats. For the first nonpublic evaluation, share the validated wheel and source archive through storage whose access is restricted to the intended testers. GitHub workflow artifacts follow repository access: artifacts in a public repository are not private. A development version alone does not make an upload private.
Ordinary branch pushes and pull requests run checks and retain artifacts.
A v<project.version> tag starts publish-pypi.yml and, after successful
validation, uploads to public PyPI. Do not push such a tag for the nonpublic
evaluation. The version is defined in pyproject.toml; kkrunner.__version__ reads the installed distribution’s
metadata. Commit all candidate files before tagging.
validate.yml builds one wheel and source archive. Its Linux Python 3.9, 3.12
and 3.14 jobs install that same wheel, run the smoke test outside the checkout
and execute the full default suite from the unpacked archive. macOS 15 jobs on
Apple Silicon and Intel use Python 3.12 and the same artifacts for the smoke
test and a selected suite covering imports, CLI, transport, file/run models,
process identity, durable local supervision, reconnect and the direct-process
remote service. These tests do not require an SSH server or a scheduler.
Identity tests simulate clock changes, stale cached creation times, recycled PIDs
and unavailable process information. Linux tests also exercise the real psutil
API against synthetic proc data through its public PROCFS_PATH setting;
no test changes the host’s system clock. Linux CI also exercises the minimum
supported psutil version, 5.8.0, on Python 3.9. A fresh-interpreter integration
test reopens the run store after the submitting client exits.
A separate job builds HTML with warnings as errors and executes the documentation
examples against that wheel. Regular branch/PR checks and tagged candidates call
this shared workflow with both runtime matrices and documentation checks.
The Pages workflow calls it with run-runtime-tests: false: it still builds
and checks the wheel/sdist, installs the wheel with documentation dependencies,
builds HTML with warnings as errors, and executes the same documentation examples.
It skips the Linux/macOS runtime matrices. The documentation job stays in the
shared validation workflow so documentation and packaging errors also block a
PyPI release. Optional Docker SSH/Slurm tests remain available via the command
above.
The publish job waits for the Linux runtime matrix, macOS matrix and documentation job, then downloads the exact artifacts they validated; it does not rebuild them. A failed check blocks publication. There is no need to infer success from separate workflow runs for the same commit.
Before public distribution, configure the GitHub pypi environment and a PyPI
Trusted Publisher for owner ase2sprkkr, repository kkrunner, workflow
publish-pypi.yml and that environment. Protect the environment with reviewers
and v* tag rules. A pending publisher can create the PyPI project on first
use. No API token is needed. Use the actual owner/repository if hosted elsewhere.
GitHub Pages
The documentation site is https://ase2sprkkr.github.io/kkrunner/. In the
repository’s Settings > Pages > Build and deployment, select GitHub
Actions as the source. The github-pages environment must allow v*
release tags and the source branches selected for manual publication. Those
refs must contain the validation and publishing workflows; no generated HTML
or separate gh-pages branch is required.
Pushes and pull requests build HTML and execute the examples. Only a push to
a v* tag or a manual run of pages.yml uploads a Pages artifact and
runs the deployment job, after the package build and documentation checks.
These Pages runs skip the runtime matrices; branch/PR CI and PyPI releases run
all validation jobs. The validation jobs have read-only repository permissions;
the separate deployment job has pages: write and id-token: write.
Deployment concurrency does not cancel an in-progress deployment.
The first publication requires pushing the source to GitHub and completing the Pages settings above. A local Sphinx build alone does not publish the site.