Declare the files a run uses
A file manifest lists the files a run may read, create or modify. KKRunner
builds it from the file fields of ExecutionSpec and uses it
for transfers, output checks, captured file versions and managed-run
reservations. Declare all files the program uses; these declarations do not
restrict the process’s filesystem access.
File paths are relative to ExecutionSpec.directory. Use a plain filename
such as "stdout.txt" for a file in that directory, or an absolute path
for a file elsewhere.
input_fileidentifies the program’s main input file for tracking and transfer. The command must still pass its name to the program if needed; this field does not redirect standard input.stdinsupplies the file whose contents the process reads through standard input. For example,stdin="input.txt"opens that file as the process’s stdin. It may name the same file asinput_file.output_fileanderror_filesave the process’s stdout and stderr.input_filesdeclares additional files read by the program.output_filesdeclares additional files created by the program.cache_filesdeclares files the program may both read and rewrite.
import sys
import tempfile
from pathlib import Path
from kkrunner import ExecutionSpec, TaskFile
from kkrunner.files import FileManifest
workspace = tempfile.TemporaryDirectory()
directory = Path(workspace.name)
(directory / "input.txt").write_text("input")
spec = ExecutionSpec(
[sys.executable, "-c", "import sys; print(sys.stdin.read())"],
directory=directory,
input_file="input.txt",
stdin="input.txt",
output_file="stdout.txt",
output_files=(TaskFile("optional.log", optional=True),),
cache_files=("cache.bin",),
)
manifest = FileManifest.from_spec(spec)
print(len(manifest.inputs), len(manifest.outputs), len(manifest.caches))
print(manifest.outputs[0].optional)
1 1 1
True
TaskFile is immutable. Its type and info strings belong to the
application; KKRunner does not look up file formats. Applications must
resolve pathname templates and callbacks before handing a concrete execution
specification to the runner. An optional declared file may be absent after
successful execution.
Reservations
Reservations prevent a managed run from overwriting files another managed run still needs. They cover the declared read and write paths, including caches, stdout/stderr and transfer destinations; sharing a working directory does not reserve the entire directory.
A runner created by RunManager.create_run reserves its files at submit(),
before uploads, opening outputs or starting the command. The reservation lasts
until the managed run reaches a terminal state, including any transfers and
required result processing. Detaching observation does not release it.
Coordination applies to runs in the same RunStore, even when different
manager instances or projects use that store. ProjectExecutionManager uses
the same mechanism. Raw executor.create_run calls and separate stores do
not coordinate these reservations.
Two active runs may read the same file. A write conflicts with another run’s
read or write. Cache declarations participate in both roles. Local uploads,
download destinations and streamed output paths are checked independently of
the backend. Remote paths are also checked in their target filesystem namespace.
A DuplicateSubmissionError reports an active conflict.
For host aliases or shared mounts, supply the same filesystem_id to
RemoteEndpointExecutor. KKRunner does not infer shared mounts
from different hostnames. Distinct IDs declare distinct target filesystems.
Transfers and captured contents
LaunchSpec provides the local/remote roots and transfer
policies. changed transfers differing contents; always transfers every
existing source; missing fills only missing destinations; never disables
that transfer direction. Missing required inputs remain errors.
Successful durable completion captures file versions and SHA-256 digests.
Historical applications should verify captured files before interpreting them.
HistoricalFileMismatchError indicates that historical content
was replaced. A missing required local output is an ARTIFACTS failure;
a failed remote transfer is a DOWNLOAD failure. Optional missing outputs
are recorded as skipped.