Save and resume computations across process boundaries using serializable continuations.
Turmeric's serializable continuations enable suspended computations to be marshalled to bytes, persisted to disk or sent over a network, then resumed in a fresh process. This builds on Phase 18's delimited continuations (shift/reset) which reify the call stack as a heap-allocated closure chain.
Use cases include: - Persistent workflows — Pause and resume multi-step business processes - Distributed task migration — Send half-finished computations to other nodes - Checkpointing — Save state periodically; restart from last checkpoint on crash - Web continuations — Racket-style: serialize "what to do next" as a URL token - Mobile agents — Send code + state to a remote peer for execution - Debugger snapshots — Freeze and replay running program state
A continuation represents "the rest of the computation" — a suspended state that can be resumed later. A serializable continuation can be converted to bytes and restored, even in a different process.
;; Capture a serializable continuation
(serial-reset
(let [x 42]
(serial-shift k
;; k is a serializable continuation expecting an int64
(save-to-disk k "my-continuation.bin")
0)))
;; Later, in another process:
(def k (load-from-disk "my-continuation.bin"))
(serial-resume k 100) ; Resumes computation with x=42, returns 142
Serializable TypeclassNot all values can be serialized. Types must opt-in via the Serializable typeclass:
defclass Serializable [a]
(serialize [x : a] : bytes)
(deserialize [b : bytes] : (Result a cstr))
Primitive types have automatic instances:
(definstance Serializable int64)
(definstance Serializable float64)
(definstance Serializable bool)
(definstance Serializable cstr)
(definstance Serializable bytes)
Container types derive their instances from element types:
(definstance (Serializable (Vec a)) [Serializable a])
(definstance (Serializable (Pair a b)) [Serializable a, Serializable b])
(definstance (Serializable (Option a)) [Serializable a])
(definstance (Serializable (Result a b)) [Serializable a, Serializable b])
Types that do not implement Serializable (file handles, raw pointers, Mutex<T>) cannot be captured in a serializable continuation. The elaborator enforces this at the serial-reset boundary.
System resources like file handles can implement custom marshal/unmarshal logic:
defclass ResourceSerializable [a]
(marshal [x : a] : value)
(unmarshal [v : value] : a)
;; Example: serialize a file handle by its path, reopen on resume
definstance ResourceSerializable FileHandle
(marshal [fh] (file-handle-path fh))
(unmarshal [path] (open-file path ReadOnly))
;; Delimit a serializable region
(serial-reset body)
;; Capture the continuation
(serial-shift [k] body) ; k : serial-continuation<T>
;; Serialize a continuation to bytes
(serial-cont->bytes k) : bytes
;; Deserialize bytes to a continuation
(bytes->serial-cont b) : (Result (serial-continuation<T>) cstr)
;; Resume a continuation with a value
(serial-resume k v) : T
serial-continuation<T> Typedefalias serial-continuation<T>
(struct
[resume : (-> T (serial-continuation<T>))
to-bytes : (-> bytes)
schema-id : cstr]) ; Stable hash of frame chain shape
A multi-step business process that survives crashes:
(defn process-order [order-id : int64] : unit
(serial-reset
;; Step 1: Validate
(def order (load-order order-id))
(unless (valid-order? order)
(throw (validation-error "Invalid order")))
;; Checkpoint after validation
(serial-shift [k]
(save-checkpoint "order-validated" k order)
(continue k order))
;; Step 2: Charge payment
(def charge-result (charge-payment order.payment-info))
(serial-shift [k]
(save-checkpoint "payment-charged" k charge-result)
(continue k charge-result))
;; Step 3: Fulfill
(fulfill order charge-result)))
;; Resume from last checkpoint
defn resume-order [order-id : int64] : unit
(let? [checkpoint (load-latest-checkpoint order-id)
k (bytes->serial-cont checkpoint.bytes)]
(serial-resume k checkpoint.value))
Send a half-finished computation to another node:
;; Node A: Start computation
(def result
(serial-reset
(def task1 (run-task1))
(def task2 (run-task2 task1))
(serial-shift [k]
;; Serialize and send to Node B
(def bytes (serial-cont->bytes k))
(send-to-node-b bytes task2)
(continue k (recv-from-node-b)))))
;; Node B: Resume computation
(defn handle-migration [bytes : bytes, input : any] : bytes
(let? [k (bytes->serial-cont bytes)]
(def result (serial-resume k input))
(serial-cont->bytes (serial-shift [k'] (continue k' result)))))
Serialize "what to do when form is submitted" as a URL token:
;; Generate a form page with continuation token
(defn get-checkout [req : HttpRequest] : HttpResponse
(serial-reset
(def cart (get-cart req.session))
(serial-shift [k]
;; Save continuation, return URL with token
(def token (save-continuation k))
(render-page
(form :action (str "/checkout-submit?token=" token))
(label "Credit Card") (input :type "text" :name "cc")
(submit)))))
;; Handle form submission
(defn post-checkout-submit [req : HttpRequest] : HttpResponse
(let? [token (parse-token req.query.token)
k (load-continuation token)]
(def cc-number (parse-form-data req))
(serial-resume k cc-number))
(render-page (h1 "Thank you for your order!")))
Periodic snapshots for crash recovery:
defn analyze-dataset [data : (Vec Record)] : Report
(defn checkpoint-every [n : int64, items : (Vec Record)] : Report
(let [processed (Vec.new)]
(for-each-with-index items
(fn [i item]
(Vec.push processed (process item))
(when (= (mod (+ i 1) n) 0)
(serial-shift [k]
(save-checkpoint (str "checkpoint-" i) k)
(continue k))))))
(compute-report processed)))
(checkpoint-every 1000 data)
;; On restart: find latest checkpoint and resume
defn recover-analysis [] : Report
(def latest (find-latest-checkpoint))
(let? [k (bytes->serial-cont latest.bytes)]
(serial-resume k))
Continuation frames carry a schema version. If the code changes between serialization and deserialization, an error is returned:
(try-with
(fn []
(def k (bytes->serial-cont bytes))
(serial-resume k value))
(fn [e k]
(match e
(SchemaMismatch old new) ->
(error "Cannot resume: checkpoint uses schema " old
"but current code uses schema " new)
_ -> (raise e))))
If you try to capture an unserializable value, the elaborator produces an error:
error: binding `handle : file-handle` captured inside `serial-reset`
but `file-handle` does not implement `Serializable`
--> src/main.tur:42:5
|
42 | (serial-shift [k] (save-cont! k) 0)
| ^ `handle` captured here
= help: use resource marshalling or move outside serial-reset boundary
To fix: either implement Serializable/ResourceSerializable for the type, or restructure your code to avoid capturing it.
Serialization performs a deep copy of all captured state. Circular reference structures are detected and produce an error:
def circular : (Option (Box circular))
(set-box! circular (Some (Box.new circular)))
;; This will fail with a circular reference error
(serial-reset
(serial-shift [k] (save-cont k) 0))
Serialization always performs a deep clone. Reference-counted values are fully copied into the wire encoding, not shared. On resume, fresh heap allocations are created.
(def r (ref 42))
(serial-reset
(serial-shift [k]
;; Serialization deep-copies r
(def bytes (serial-cont->bytes k))
(continue k)))
;; After deserialization, the resumed continuation has a NEW ref
;; with the same value (42), but it's a different cell in memory
ref<T> and rc<T> Behaviorref<T> may be captured only if T: Serializableref cell is created with that valueref is not preserved — each resumed continuation gets its own independent copystdlib/serial.tur;; File I/O helpers
defn cont-to-file [k : serial-continuation<T>, path : cstr] : (Result unit cstr)
(write-file path (serial-cont->bytes k))
defn cont-from-file [path : cstr] : (Result (serial-continuation<T>) cstr)
(bytes->serial-cont (read-file path))
stdlib/workflow.turA higher-level API for persistent workflows:
;; Define a workflow step that can be suspended and resumed
defworkflow-step process-approval [order-id : int64] : bool
(def approved? (serial-shift [k]
(db-save-continuation order-id k)
false))
(when approved? (fulfill-order! order-id))
approved?
;; Resume a workflow from the database
defn resume-approval [order-id : int64, approved? : bool] : unit
(let? [k (db-load-continuation order-id)]
(serial-resume k approved?))
Only capture what you need. Use identifiers instead of entire objects:
;; Good: capture only the ID
(serial-reset
(def order-id 12345)
(serial-shift [k]
(save-cont k)
(process-order order-id)))
;; Less good: capture the entire order object
(serial-reset
(def order (load-order 12345)) ; Large object
(serial-shift [k]
(save-cont k)
(continue k)))
Deep call stacks increase serialization time and size. Design workflows with shallow stacks where possible.
If storing continuations long-term, consider implementing custom schema evolution logic:
;; Wrap continuation with version info
defn save-versioned [k : serial-continuation<T>, version : int64] : unit
(def bytes (serial-cont->bytes k))
(save-to-disk (struct [version version, data bytes]))
;; On load, verify version compatibility
defn load-versioned [path : cstr] : (Result (serial-continuation<T>) cstr)
(def stored (load-from-disk path))
(when (= stored.version CURRENT_VERSION)
(bytes->serial-cont stored.data))
Deserializing from an untrusted source is analogous to Java deserialization vulnerabilities. Consider:
| Approach | Pros | Cons |
|---|---|---|
| Serializable Continuations | Automatic, composable, captures exact state | Requires Serializable typeclass |
| Hand-Written State Machines | Full control, no overhead | Tedious, error-prone, manual updates |
| Green-Thread Snapshots | Captures full thread state | Not portable, captures OS resources, complex |
| Persistent Processes | Live state, hot reloading | Requires VM support, not suitable for C target |