Skip to main content

Resumable execution — runs, attempts & tasks

Persistence & Resume covers the low-level primitive: serialize a stimulation's outstanding frontier so a retry continues where it stopped. This page is the layer on top — the name-based run model CNStra records, and the two independent choices you make per run: do I checkpoint it for resume? and do I keep its history to look at?

The model​

Everything is keyed by name (neuron/collateral names from your registry) — topology is never persisted, code is its source of truth.

ConceptWhat it isId
Stimulationthe stable logical run a human retriesstimulationId (+ optional scopeName)
Attemptone execution of it (a stimulate / activate)attemptNumber
Taskone settled hop within an attemptordinal index

A retry is a new Attempt of the same Stimulation. (stimulationId, attemptNumber) is unique. Roughly: Stimulation ≈ a Temporal Workflow, Attempt ≈ a Run.

scopeName is optional and identifies the CNS/graph a run belongs to — leave it unset and everything lives in one default scope; set it to isolate names and route auto-resume when one process hosts several graphs.

Two independent seams​

On a queue worker (@cnstra/pg-boss / @cnstra/bullmq) these are separate options — use either, both, or neither:

Setup shared by the examples on this page
import { CNS, collateral } from '@cnstra/core';
import type { TCNSNeuron } from '@cnstra/types';
import { CNSInMemoryStimulationRepository } from '@cnstra/persist';
import type { CNSPersistOptionsRegistry, ICNSProgressRepository, ICNSStimulationWriter } from '@cnstra/persist';
import type { CNSPostgresStimulationRepository } from '@cnstra/persist-postgres';
import { createCNSWorker } from '@cnstra/pg-boss';
import type { ICNSPgBossLike } from '@cnstra/pg-boss';

const entry = collateral<unknown>();

declare const neurons: TCNSNeuron[];
declare const boss: ICNSPgBossLike;
declare const registry: CNSPersistOptionsRegistry;
declare const queue: string;
declare const checkpointStore: ICNSProgressRepository;
declare const historyStore: ICNSStimulationWriter;
declare const resume: { repository: ICNSProgressRepository };
declare const observe: ICNSStimulationWriter;
declare const pgStore: CNSPostgresStimulationRepository;
declare const app: {
  post(path: string, handler: (req: { body: unknown }, res: { sendStatus(code: number): void }) => void): void;
};
const cns = new CNS(neurons);

await createCNSWorker({
  boss, cns, registry, queue: 'cns',
  resume:  { repository: checkpointStore }, // frontier checkpoint → restart continues
  observe: historyStore,                    // run/attempt/task history → inspect it
});
  • resume — the frontier (what's left to do), a cheap key→blob checkpoint (ICNSProgressRepository). A retried job continues via cns.activate(...) instead of re-running from the entry. Omit it and a retry just re-runs the whole flow (make neurons idempotent).
  • observe — the history (what happened): the full Stimulation → Attempt → Task model written to an ICNSStimulationRepository. This is exactly what the DevTools render and what a retry admin lists.

They are orthogonal because they answer different questions — "where do I continue?" vs "what happened?" — and often need different storage shapes (a one-blob checkpoint vs a relational history).

Where the history lives​

observe (and the DevTools server) take any ICNSStimulationRepository:

// See it in-process (ephemeral) — @cnstra/persist
import { CNSInMemoryStimulationRepository } from '@cnstra/persist';

// Keep it across restarts — @cnstra/persist-postgres (owns the run/attempt/task tables)
import { CNSPostgresStimulationRepository } from '@cnstra/persist-postgres';
export const history = new CNSPostgresStimulationRepository({ connectionString: process.env.DATABASE_URL });
// tables: `cnstra-persist-postgres migrate`, or auto-created on first use.

Read it back — a ready-made admin waterfall, no schema to write:

import { history } from './history';

const runs     = await history.listStimulations();            // newest-first roster
const attempts = await history.getAttempts(runs[0].stimulationId);
const tasks    = await history.getTasks(attempts[0].stimulationAttemptId);

⚠️ Bound the in-memory store or you will OOM​

CNSInMemoryStimulationRepository retains every run forever by default — fine for tests, fatal for a long-running process. Always pass a bound for observe:

new CNSInMemoryStimulationRepository({
  maxStimulations: 1000,   // evict least-recently-updated beyond the cap (+ its attempts/tasks)
  ttlMs: 60 * 60 * 1000,   // evict runs idle longer than this (lazy sweep on writes)
  deleteOnComplete: true,  // "live only": drop a run the moment it settles (completed OR failed)
});

Use deleteOnComplete when you only watch in-flight runs; use maxStimulations / ttlMs to keep a bounded recent history (e.g. to still inspect or retry failed runs).

The Postgres store persists rows, so bound it with retention on a schedule (cron):

// prune runs not updated for a week, with their attempts and tasks
await pgStore.deleteStale(7 * 24 * 60 * 60 * 1000);

With a pool, the same call keeps the task table in day partitions: it drops the days older than the window and attaches the next seven, so call it at least weekly. Tasks go by their own age, so keep the window above your longest run.

Retry & clone​

Given a run's history you get two admin actions for free (this is what the DevTools Stimulations panel does):

  • Retry — resume the outstanding frontier as a new attempt; only the unfinished branch re-runs, from the input signal its frontier task carries.
  • Clone — a fresh run from the original entry (a clean re-do).

One CNS, two call sites​

Resume is attached per stimulate() call, not baked into the CNS — the CNS is a pure graph. So the same cns can run resumably from a worker and plainly from, say, an HTTP endpoint:

const cns = new CNS(neurons); // one instance for the whole app

// resumable: from the queue worker (persistor + resume + observe wired for you)
createCNSWorker({ boss, cns, registry, queue, resume, observe });

// plain: same cns, no persistence
app.post('/do', (req, res) => { cns.stimulate(entry.createSignal(req.body)); res.sendStatus(202); });

Each stimulate is an independent stimulation (its own context + queue); nothing crosses over. Two caveats about what is shared on the CNS instance: the per-neuron concurrency gate (both call sites contend on it — usually the point) and the dependencies the neurons were built with (the factory's closure or ctx.global, shared by design). Note also that a global cns.addResponseListener (what the DevTools attach) observes all stimulations, whereas a per-call onResponse/observe is selective — so resume and history stay on the worker path only if you attach it per-call, which the worker does.

See also​