Jobs And DAG Execution Foundations
Summary
Jobs and DAG execution foundations explain how scheduled, triggered, and operator-launched work runs without hiding state inside scripts. Domain pages should link here when they need to explain job catalog metadata, run state, worker leases, DAG task repair, retry, cancellation, artifacts, or escalation to review.
Reader Question
How does LKCI run automated work while keeping it inspectable, retryable, idempotent, and repairable?
Surface Or Workflow
Operators enter through the jobs console and DAG console. Engineers enter through job manifests, thin run.py entrypoints, job/DAG API routes, execution services, scheduler/worker runtime, and tests for schedules, workers, stale recovery, DAG task repair, and job run control.
Lifecycle
A job is registered in the catalog, launched with explicit parameters, records run state, writes artifacts or diagnostics, and either completes, fails, cancels, retries, or escalates to review. DAGs add task dependencies, schedules, task attempts, repair flags, and auto-resume behavior over the same execution posture.
Slow or side-effecting work should not run inline in an API request. API routes schedule, control, or inspect work; services own the behavior; entrypoints stay thin.
Child Threads
- Job Entrypoint Contract: how jobs remain thin and CLI-compatible.
- Job Catalog Metadata: how runnable work is described.
- Jobs Console: how operators inspect and control runs.
- DAG Authoring Console: how DAG work is authored and repaired.
- Idempotency In Operational Jobs: how retries avoid duplicate side effects.
- Run Artifacts And Observability: how evidence is preserved.
Implementation Boundaries
Job and DAG surfaces live under app/web/app/jobs, app/api/jobs, app/web/app/dags, and app/api/dags. Execution runtime lives under services/execution; workflow resumption under services/workflow/resumption; domain job entrypoints under jobs; tests under tests/jobs, tests/dags, tests/execution, and tests/ops.
Tradeoffs
Thin entrypoints and explicit run state create more structure than ad hoc scripts. The benefit is that the same work can be launched by scheduler, operator, CI, or repair flow without changing the core service behavior.
Visual
The current visual is a graph neighborhood. The intended visual is an execution timeline from catalog entry to scheduled run, worker lease, artifact, retry, DAG repair, or review escalation.
Source Evidence
docs/reference/job-layout.mddocs/reference/job-execution-modes.mddocs/domains/dags.mdapp/web/app/jobsapp/api/jobsapp/web/app/dagsapp/api/dagsservices/executionjobstests/jobstests/dagstests/ops