Interruptible Recalculation
Spread Rust, Python and JavaScript recalculation over calls with progress, viewport priority and cooperative cancellation.
Rust, Python and JavaScript hosts can yield between recalculation steps while preserving installed results.
Workbook workers
Import the worker client without loading WASM on the calling thread:
import { createWorkbookWorker } from 'formualizer/worker';
const client = await createWorkbookWorker(
new Worker(new URL('formualizer/worker-entry', import.meta.url), { type: 'module' }),
{ autoRecalc: true, autoRecalcBudgetMs: 5 },
);
await client.addSheet('Model');
const viewport = await client.subscribeViewport({
rect: { sheet: 'Model', startRow: 1, startColumn: 1, endRow: 40, endColumn: 12 },
mode: 'values',
onFrame(frame) {
// Keep optimistic edits whose clientSeq is greater than frame.ackClientSeq.
// frame.grid replaces a viewport; frame.patches updates individual cells.
console.log(frame.revision, frame.evaluatedRevision, frame.hasPending);
},
});
await client.setValue('Model', 1, 1, 20);
await client.setFormula('Model', 2, 1, '=A1+22');
await client.recalc({ budgetMs: 5 });
await viewport.setRect({ ...viewport.rect, startRow: 10, endRow: 49 });
await viewport.unsubscribe();
await client.dispose();Worker endpoints accept DOM Worker/MessagePort/worker-global APIs and Node worker_threads Worker/MessagePort/parentPort APIs. The packaged worker-entry targets browser module workers; in Node, use a small host entry:
// workbook-host.mjs
import { parentPort } from 'node:worker_threads';
import * as runtime from 'formualizer';
import { attachWorkbookWorker } from 'formualizer/worker';
await attachWorkbookWorker(parentPort, { runtime });Pass new Worker(new URL('./workbook-host.mjs', import.meta.url)) to createWorkbookWorker. A custom browser host similarly calls attachWorkbookWorker(self, { runtime }). createInProcessWorkbookWorker({ runtime, autoRecalc: true }) uses a MessageChannel for testing or a single-thread fallback. If runtime is omitted on a host, it dynamically imports and initializes formualizer. Your worker bundler/runtime must support the package's bundler-target WASM import; the client-only formualizer/worker entry has no static WASM dependency.
Each mutation resolves with a WorkerStateHeader: revision, evaluatedRevision, ackClientSeq, recalcStatus, and hasActiveJob. Loads reset revision to zero while acknowledging the monotonic client sequence. client.state, client.lastSentClientSeq, client.hasInFlightMutations, and onStatusChange support optimistic views. Ordinary reads return their usual values; getValueWithState and readRangeWithState return { data, ...header }. Cached reads never evaluate or create a missing sheet.
Viewport modes are values (full grids or sparse patches), snapshots (inspection cells), deltas (locations/pending only), and columnar (transferable tags, numbers, string table, pending bitset). All coordinates are 1-based inclusive. Subscriptions prioritize their rectangles unless prioritize: false; setRect retargets them on active and future jobs. Intermediate updates skip unchanged viewports, coalesce dirty rectangles behind a one-frame credit, and honor throttleMs (default 16). Initial, manual, rect-change, and terminal refreshes flush immediately. Call refresh() for a full reread and unsubscribe() or [Symbol.dispose]() when finished. Columnar tags: 0 empty, 1 number, 2 boolean, 3 string, 4 error, 5 temporal/other JSON; numbers holds either the numeric value or the string-table index.
The async RPC surface includes sheet/table/name metadata, scalar/block mutations, editBatch, action boundaries, undo/redo, temporal egress, inspection, dependency tracing, range pages, eval plans, telemetry, one-shot evaluation, parser/tokenizer helpers, and SheetPort sessions. editBatch accepts { op: 'setValue', sheet, row, col, value }, formula/block equivalents, or addSheet; { actionDescription } groups edits into a native undo action when changelog is enabled. An edit batch is ordered, not rollback-atomic on engine failure. createSheetPortSession(manifestOrYaml) returns an async session with manifest/port description, input/output reads, input writes, evaluation and disposal; evaluateSheetPort is the one-shot equivalent. Reload disposes old sessions and cached sheet handles.
autoRecalc folds edits into one background job and yields a real task turn between steps. One-shot evaluateAll, evaluateCell(s), and SheetPort evaluation cancel an implicit auto job first, preserve installed values, then reschedule background work. An explicit paused/driven job retains the engine's RECALC_JOB_ACTIVE refusal. beginRecalc starts/adopts a paused explicit job; stepRecalc, pauseRecalc, prioritize, and cancelRecalc control it. recalc resumes paused work and returns Finished/Paused/Cancelled; returning false from onStep pauses. recalcStream yields detached step reports; breaking defaults to pausing, or cancels with pauseOnReturn: false. Both drivers support AbortSignal, including cancellation while awaiting onProgress; abort rejects with AbortError. Budgets are soft: cancellation cannot preempt a running indivisible WASM unit.
Register custom functions inside the host with attachWorkbookWorker(endpoint, { functions }), load a worker-side ESM module using registerFunctionModule(url, { exportName: 'functions' }), or explicitly opt into allowFunctionFromSource: true for registerFunctionFromSource(name, source, options). Modules export a name-to-callback map (or callback/options entries); the registry persists across reloads unless the host sets clearFunctionsOnLoad. Source registration executes code and is disabled by default. listFunctions and unregisterFunction manage the same registry.
loadFromJson(json, options) forwards workbook options just like the XLSX loader. loadFromXlsxBytes(bytes, options, { transfer: true }) and recalculateXlsxBytes(bytes, limit, options, { transfer: true }) transfer whole ArrayBuffers; typed subviews and Node Buffers are copied first so unrelated slab bytes are never detached. Cache-only XLSX output buffers transfer by default (disable with transfer: false). Error replies preserve domain fields, exact resource strings and recursive causes.
Run npm run test:worker-conformance and npm run example:worker. The examples are examples/worker-reactive-grid.mjs and examples/worker-rpc-service.mjs; the browser fixture also verifies a real Worker. Dispose clients to free jobs, sessions, sheets and the workbook; workers terminate by default, configurable with terminateOnDispose: false.
JavaScript / WASM
The public formualizer facade preserves the same Workbook constructor and JSON/XLSX loaders. Use manual stepping for a viewport loop:
import { Workbook, RecalcJob } from 'formualizer';
const wb = new Workbook();
wb.addSheet('S');
wb.setValue('S', 1, 1, 10);
wb.setFormula('S', 2, 1, '=A1+1');
const viewport = { sheet: 'S', startRow: 1, startColumn: 1, endRow: 20, endColumn: 8 };
const job = wb.beginRecalc({ priority: [viewport] });
console.assert(job instanceof RecalcJob);
while (job.active) {
const report = job.step(5);
// Paint report.changed, or visible cells on report.overflowSheets.
console.log(report.phase, wb.pendingCellsInRect(viewport));
}
console.assert(wb.sheet('S').getValue(2, 1) === 11);Rectangles are finite, 1-based and inclusive, within Excel's row/column bounds. job.prioritize(ranges) replaces queued viewport priorities after a scroll. Edits between steps invalidate schedules as needed; dirty readers are recalculated by the same job. hasPendingInRect and pendingCellsInRect let a host distinguish pending cells from current installed values. collectDeltas defaults to true, and maxDeltaRecords defaults to 1024 per step; overflow conservatively reports sheet names instead of individual rectangles.
For host event-loop responsiveness:
const controller = new AbortController();
const outcome = await wb.evaluateWithYield({
budgetMs: 5,
signal: controller.signal,
async onProgress(report, job) {
console.log(report.progress);
// Await a host update here; inspect/edit/prioritize/cancel outside engine locks.
},
});
if (outcome.status === 'Finished') console.log(outcome.result);The default yield gives a real task turn via setTimeout(0). A custom yieldControl() may be awaited; returning a resolved promise alone does not let timers/UI tasks run. A step executes on the calling thread and may overrun its soft deadline at indivisible graph/schedule builds, families, cycles or synchronous UDFs. budgetMs: null removes the soft deadline; zero still makes progress. Evaluation work/time limits configured on Workbook apply across the whole job, not separately to each step.
recalc({ budgetMs: 5, onStep(report, job) { return false; } }) drives synchronously and pauses an active job on exactly false. Narrow the returned RecalcOutcome: Finished carries result, Paused carries an active job, and Cancelled carries neither. A Paused job releases driver ownership but remains the workbook's active job; resume with job.step() or terminate with job.cancel(). Callback cancellation takes precedence over false. evaluateWithYield ignores callback return values and awaits progress hooks, including terminal hooks.
Only Finished step reports carry a result. Progress includes pass, completedInPass, scheduledInPass, cumulative, totalExact, stagedRemaining and priorityPending. A new pass can change the denominator; false totalExact means the total is not proven fixed. stepComputed, stepElapsedMs and longestUnit describe the current step. Reports and their changed rectangles are detached: retaining or editing one cannot change engine state.
An aborted signal rejects with an error named AbortError, cancels the owned generation and releases its reservation. It interrupts awaited progress/yield hooks promptly; losing promises retain rejection handlers. Abort inside a synchronous UDF only signals cancellation, and cleanup runs after the step returns. wb.cancel() is a cooperative flag; wb.resetCancel() deliberately clears that flag, even during an active job. Reset cannot undo an already aborted signal or terminal job.cancel(). A throwing/rejected hook ends the owned job and rethrows the original value.
A driver reserves its generation through terminal callbacks and cleanup. Competing beginRecalc/driver calls fail with ALREADY_PENDING and boolean explicit; ordinary evaluation, including Sheet and SheetPort evaluation, exposes RECALC_JOB_ACTIVE. Old handles cannot operate on replacements. A job retains the underlying workbook even after the original JS Workbook wrapper is freed; asynchronous abort uses the retained owner too. Do not explicitly free the job handle while a driver owns it. Explicit cancellation controls lifecycle; freeing a handle alone does not terminate a job.
Thrown RecalcJobError values are ordinary Error objects with typed properties, not an exported error constructor. Their code is NO_ACTIVE_JOB, ALREADY_PENDING, ENGINE_POISONED, REENTRANT_CALL, EVALUATION, INVALID_OPTIONS or INVALID_ADDRESS. Narrow code without parsing messages. EVALUATION preserves structured Excel/resource fields and a typed cause. UDF reentry into job APIs fails promptly with REENTRANT_CALL. WASM panic-abort builds do not provide native Rust unwind recovery.
The local assertion-bearing examples and browser preparation commands live in the WASM binding README. They demonstrate viewport scroll/input edits, asynchronous heartbeat/abort/supersede, and pause/inspect/repair/resume. Worker orchestration is a separate host concern.
Python
Use an explicit job when your host owns the loop. All Python rectangles use 1-based, inclusive coordinates. Values are read separately from reports.
import formualizer as fz
wb = fz.Workbook()
wb.add_sheet("S")
wb.set_formula("S", 1, 1, "=1")
wb.set_formula("S", 2, 1, "=A1+1")
with wb.begin_recalc(priority=[("S", 2, 1, 2, 1)]) as job:
for report in job.iter(budget_ms=5):
p = report.progress
print(report.phase, p.completed_in_pass, p.scheduled_in_pass, p.total_exact)
if not wb.has_pending_in_rect("S", 1, 1, 2, 1):
print(wb.get_value("S", 2, 1))job.step(budget_ms=None, *, cancel=None) runs one step. None has no soft deadline, while zero still permits progress at an atomic boundary. job.prioritize(rectangles) replaces the queued priority request; job.cancel() ends the current job. Context exit cancels if it is still active. A job handle owns its workbook and generation: an old handle cannot step or cancel a replacement job.
wb.recalc(5, on_step=callback) drives the loop synchronously. The callback receives every report, including Finished, without an engine lock, so it can read or edit the workbook. Return exactly False to pause. The returned RecalcOutcome has status (Finished, Cancelled or Paused), result on Finished, and job on Paused for later stepping. Callback exceptions preserve the reached synchronous lifecycle state; use manual job ownership when you need explicit exception cleanup.
In an asyncio application:
async def calculate(wb):
result = await wb.evaluate_async(
budget_ms=5,
on_progress=lambda report: print(report.phase, report.progress.cumulative),
priority=[("S", 1, 1, 20, 10)],
)
return resultNative steps run through asyncio.to_thread with the GIL released under the workbook write lock. Progress callbacks execute on the event loop thread, between steps. Other threads may read or edit the workbook; reads mix installed values and old pending values. Task cancellation trips a cooperative token, waits for the running native step, cancels the remaining job, and re-raises asyncio.CancelledError. A long indivisible unit can delay this cleanup. An atomic reservation covers admission and the entire async driver lifetime, including callbacks and cleanup. A second async calculation while a driver or manual job is pending/active raises RecalcAlreadyPendingError promptly, without waiting for a running step's lock. Native admission and cleanup also run on workers; cancellation during admission waits for the worker, cancels any newly admitted job, and re-raises. The driver itself never waits for the engine lock on the event-loop thread. Begin a replacement calculation after the prior driver returns. On Pyodide, the loop runs in-thread and uses await asyncio.sleep(0) between steps; individual units still block that thread. Async progress callback exceptions cancel the remaining job and propagate.
StepReport is frozen. It includes phase/progress, step counts and milliseconds, conservative changed rectangles, overflow_sheets, the longest indivisible unit, and a result only on Finished. Pending queries report raw dirtiness: completion can re-dirty volatile or iterative formulas for the next request. Progress totals can change after edits; total_exact=False means conservative or unknown. Existing evaluation methods raise typed RecalcJobActiveError during an active job. Python UDF exceptions keep their existing error behavior; calls back into the same workbook still raise the reentrancy guard. Workbook.cancel() and CancelToken.cancel() remain safe inside a UDF. job.cancel() takes the engine lock and is guarded against UDF reentrancy. Internal Rust panics retain workbook lock poisoning; discard that workbook.
See the Python API reference for exact signatures, report fields and exception classes. The limits and carve-outs below also apply to Python.
Choose an API
Engine::evaluate_all_until(EvaluationStop) returns EvaluationProgress::Complete(EvalResult) or Incomplete. Repeated calls continue one logical recalculation, with one clock sample, random epoch and diagnostics collection. A different evaluation request safely ends this implicit continuation.
Explicit jobs add a held schedule, formula progress, changed regions and viewport priority. Call begin_recalc, then step_recalc until Finished or Cancelled. A returned step error also ends the job. recalc_with drives the same lifecycle and calls your closure between steps; Break returns Paused and leaves an active job for later manual stepping. Engine and Workbook expose both APIs.
Soft budgets and installed values
An evaluation step stops at an atomic unit boundary after its soft budget. A family run or cycle is never split, and the unit that observes cancellation is not installed. Only committed and flushed results become clean. The phases are BuildingGraph, Scheduling, Evaluating, Finished and Cancelled. Graph construction and scheduling remain indivisible, so a step can exceed its budget.
Jobs currently evaluate single-threaded, including when ordinary parallel evaluation is enabled. Use a native worker thread to keep a UI responsive during long units.
Progress callback
This Workbook example prints pass-local progress. A changed pass invalidates the old denominator; total_exact == false means the total is conservative or unknown. The callback also receives the terminal report.
use formualizer_eval::engine::{EvaluationStop, RecalcOutcome};
use formualizer_workbook::{Workbook, WorkbookConfig};
use std::{ops::ControlFlow, time::Duration};
let mut wb = Workbook::new_with_config(WorkbookConfig::ephemeral());
wb.add_sheet("S")?;
wb.set_formula("S", 1, 1, "=1")?;
wb.set_formula("S", 2, 1, "=A1+1")?;
let outcome = wb.recalc_with(
Default::default(), EvaluationStop::after(Duration::from_millis(5)),
|_, report| {
let p = &report.progress;
let filled = (p.completed_in_pass.saturating_mul(20)
/ p.scheduled_in_pass.max(1)).min(20);
println!("[{}{}] pass {}: {} / {} (exact: {})",
"#".repeat(filled), "-".repeat(20 - filled),
p.pass, p.completed_in_pass, p.scheduled_in_pass, p.total_exact);
ControlFlow::Continue(())
},
)?;
assert!(matches!(outcome, RecalcOutcome::Finished(_)));The callback may read, edit, prioritize or cancel the owner. Finished and Cancelled take precedence over Break on a terminal step. Begin replacement jobs after the driver returns.
Viewport priority
Priority affects order only. The last request wins; dirty precedents and whole touched family runs run first. If expansion exceeds 65,536 formulas, the job falls back to its full schedule without losing work.
use formualizer_eval::engine::{EvaluationStop, EvaluationTarget};
wb.begin_recalc(Default::default())?;
wb.prioritize_recalc(&[EvaluationTarget::Cell {
sheet: "S".into(), row: 2, col: 1, // targets are one-based
}])?;
let sheet = wb.engine().sheet_id("S").unwrap();
while wb.has_active_recalc_job() {
let report = wb.step_recalc(EvaluationStop::after(std::time::Duration::ZERO))?;
if !wb.has_pending_in_rect(sheet, 0, 0, 1, 0) {
// Viewport coordinates are zero-based and inclusive.
println!("{:?}", wb.get_value("S", 2, 1));
}
println!("{:?}", report.changed);
}Reads between steps mix newly installed values and old pending values. Pending queries visit only viewport coordinates and report raw dirtiness; combine them with job phase, since completion can re-dirty volatile or iterative formulas for the next request. pending_cells_in_rect returns pending formula regions, not values.
report.changed.delta conservatively covers changed regions, possibly including unchanged cells in an installed run. With more than max_delta_records coalesced records (default 1,024), records are replaced by overflow_sheets; repaint visible cells on those sheets. The cap bounds returned records, not all temporary collector allocation.
Worker thread and cancellation
This moves the owning engine to a native thread, sends small report summaries over a channel, and cancels from the owning thread. The handshake makes the example deterministic; production hosts can cancel at any time.
use formualizer_eval::engine::{
CancelToken, Engine, EvalConfig, EvaluationStop, RecalcOutcome,
};
use formualizer_workbook::workbook::WBResolver;
use formualizer_parse::parser::parse;
use std::{ops::ControlFlow, sync::mpsc, thread};
let mut engine = Engine::new(WBResolver::default(), EvalConfig::default());
engine.set_cell_formula("S", 1, 1, parse("=1")?)?;
let token = CancelToken::new();
let worker_token = token.clone();
let (tx, rx) = mpsc::channel();
let (go_tx, go_rx) = mpsc::channel();
let worker = thread::spawn(move || {
go_rx.recv().unwrap();
engine.recalc_with(Default::default(), EvaluationStop::cancel(worker_token),
|_, report| {
tx.send((report.phase, report.step_computed_vertices)).unwrap();
ControlFlow::Continue(())
})
});
token.cancel();
go_tx.send(())?;
assert!(matches!(worker.join().unwrap()?, RecalcOutcome::Cancelled));
for (phase, computed) in rx { println!("{phase:?}: {computed}"); }A cross-thread token does not interrupt an indivisible builtin immediately. Builtin polling is tracked as FORM-000321.
Typed errors and lifecycle
Job methods return RecalcJobError directly, including on Workbook:
NoActiveJob: step or priority without a job.AlreadyPending { explicit }: begin or drive while an explicit job or implicit continuation exists.Evaluation(ExcelError): evaluation failed; the job has ended. Inspectsource()or the inner error for resource details, remove the cause, then begin again.
Existing evaluation APIs retain their signatures. During an explicit job they refuse work without changing job state; match the error's ExcelErrorExtra::RecalcJobActive payload, with kind Value. Workbook's existing evaluation methods keep their IoError::Engine wrapping.
Rust function panics are caught at dispatch and become #VALUE! with ExcelErrorExtra::FunctionPanicked { function, message }. String payloads are retained; other payloads have no message. Builtins, direct registrations, Workbook callbacks, reference paths and parallel calls share this policy. IFERROR can replace the cell error, but the executed root still records UnavailableOperation::FunctionPanicked when operation diagnostics are enabled. Source recalculation enables that evidence: coverage is Partial and requireComplete refuses publication. Ordinary returned Excel errors keep their existing completeness behavior. A panicking aggregate kernel discards its buffer and fails every member of that run; no partial values are installed. An engine-internal unwind through a request or job step still propagates. The engine then permanently refuses work: job APIs return RecalcJobError::Poisoned, and existing fallible mutation/evaluation APIs return a Value error with ExcelErrorExtra::EnginePoisoned (editor APIs preserve it in EditorError::Excel). Check Engine::is_poisoned or Workbook::is_poisoned. Infallible mutators panic before mutation with "formualizer engine is poisoned by an earlier internal panic; discard it". cancel_recalc returns None without touching state. Reads and queries remain available but may expose stale or partially installed values. Discard the engine; no recovery is attempted. Raw mutable storage and direct public config fields cannot be revoked, and must never be used to resume a poisoned engine. On wasm32-unknown-unknown, Rust panics trap instead of unwinding: catch_unwind and unwind guards cannot recover or mark an instance after a trap. Discard that workbook/worker or instance. Python exceptions and JS throws already return ordinary function errors; Rust-side internal panics retain Python's workbook-lock poison behavior. Progress callbacks run between steps; their panics retain the already-reached lifecycle state.
Function metadata and resolver callbacks are host code, too. Runtime hooks (caps, reporting/semantic/dependency contracts, schemas/arity, family-kernel selection and format metadata) are caught at their invocation sites. A planning panic declines optimization with conservative capabilities and marks that function instance for the rest of the logical request, including later job steps. Every executed root of that instance receives the typed function-panic error and diagnostic. Untaken calls do not create execution diagnostics. A subsequent request retries the metadata; no function metadata panic poisons the engine. Resolver failures reuse FunctionPanicked with explicit names such as resolver:resolve_source_scalar:ReviewSource and resolver:resolve_source_table:Sales. The source attribution is retained inside ABS(ReviewSource); it is not attributed to ABS. Version/provider callbacks and returned table/range methods have the same narrow containment. The catch finishes inside source-cache guards, and failed values are not cached. Ordinary returned errors retain their existing behavior. A panicking clock provider freezes a failed sample for that logical recalculation. Only roots reading the clock (including NOW/TODAY) receive FunctionPanicked attributed to resolver:clock and the diagnostic; unrelated formulas are unaffected. IFERROR can catch the value. Jobs do not resample a failed clock mid-job; the next request samples again. Deterministic clocks keep their existing behavior. Deferred source replay callbacks are also contained inside their spool guards. Legacy replay failures return a typed request error; engine checkpoint panics still unwind and trigger the internal backstop. CalcObserver is currently a dormant trait without any engine registration or invocation path, so no observer notifications are delivered. Progress callbacks still run between steps; their panics propagate without poisoning or changing the already-reached job state.
Limits and carve-outs
Work and hard-deadline limits accumulate across active steps, excluding idle time between calls. Scratch memory resets per step; owned held schedules count as retained memory unless shared with the engine cache. Soft deadlines do not replace hard resource limits.
Supported edits fold into the job and invalidate its held schedule when needed. Iterative cycles created by edits start from current values, including earlier installed results; they may differ from fresh evaluation. Existing cycles retain their original seeds. Unversioned providers are assumed stable: bump a semantic revision or dirty affected cells if behavior changes.
Raw store mutations and load-time bulk-finish paths are unsupported during a pending recalculation. Live-spill user writes retain the known FORM-000319 limitation. Graph slicing, cost-weighted progress and parallel jobs (FORM-000323) remain future work.
See Dependency Graph and Recalculation and Large Workbook Performance.