Generators and Lazy Sequences Guide

(defn integers-from [start : int] : (Generator :int)
  (gen []
    (let [^mut i start]
      (while true
        (yield i)
        (set! i (+ i 1))))))

(defn range-gen [lo : int hi : int] : (Generator :int)
  (gen []
    (let [^mut i lo]
      (while (< i hi)
        (yield i)
        (set! i (+ i 1))))))

(load "stdlib/gen.tur")

(let [g (range-gen 0 5)]
  (while (gen-some? (let [v (gen-next g)]
                      (when (gen-some? v)
                        (println (gen-unwrap v)))
                      v))))

;; gen-for-each macro -- preferred for side effects
(gen-for-each (range-gen 0 5)
              (fn [x] (println x)))
; prints 0 1 2 3 4

(let [arr (gen-collect (range-gen 1 6))]
  (println (gen-arr-len arr))        ; 5
  (println (gen-arr-get arr 2)))     ; 3

gen-nth -- nth yielded value

(gen-nth (range-gen 10 20) 3)   ; => 13 (0-indexed)

yield* -- re-yield from an inner generator

(defn concat-gen [g1 g2] : (Generator :int)
  (gen []
    (yield* g1)
    (yield* g2)))

Supported control flow in gen bodies

Limitations (1.0)

Two yield placements are hard compile errors in 1.0 because the gen/yield state-machine lowering cannot represent them without the post-1.0 CPS pass:

yield / yield* inside a match arm (TUR-E0702)

The state machine needs to save and restore match-arm position across a yield. Without full CPS this is not representable; the compiler rejects it:

;; ERROR: TUR-E0702
(defn broken [flag : int] : (Generator :int)
  (gen []
    (match flag
      0 (yield 42)   ;; 'yield' is not supported inside a 'match' arm
      _ (yield 99))))

Workaround -- yield before or after the match, or use if/cond:

(defn ok [flag : int] : (Generator :int)
  (gen []
    (let [v (if (= flag 0) 42 99)]
      (yield v))))

Recursive generators (TUR-E0703)

A gen body whose enclosing function calls itself is a recursive generator. Suspending across a recursive call requires CPS; the compiler rejects it:

;; ERROR: TUR-E0703
(defn count-down [n : int] : (Generator :int)
  (gen []
    (yield n)
    (count-down (- n 1))))  ;; recursive call inside gen body

Workaround -- unroll the recursion into an explicit loop:

(defn count-down [start : int] : (Generator :int)
  (gen []
    (let [^mut n start]
      (while (>= n 0)
        (yield n)
        (set! n (- n 1))))))

Both limitations are tracked in Phase CF5 of control-flow-completeness-plan.md and require the post-1.0 CPS pass to lift.

Generated C

The compiler emits a struct per gen body and a _next function with a switch on the state tag. Each yield saves state and returns. This means:

• Zero dynamic allocation per next call
• Zero function-pointer indirection
• Inlineable by the C compiler in performance-critical paths

Interpreter (turi) support

The tree-walking interpreter (tur interpret, tur repl, sandbox eval) runs generators too. Instead of the compiled state machine it executes each gen body on its own coroutine stack (the same fiber primitives that back effect handlers): yield suspends the body back to the caller, and gen-next resumes it until the next yield or until the body runs off its end. gen-done? reports exhaustion exactly as the compiled path does -- it flips to true only after a gen-next drives the body past its last yield, so the idiomatic (while (not (gen-done? g)) ...) loop terminates identically under both backends.

Consume generators through the stdlib/gen.tur helpers (gen-some?, gen-unwrap, gen-none, and the gen-for-each / gen-nth / yield* macros): the interpreter provides native implementations of these. Hand-rolled inline-C pointer helpers that dereference the gen-next result directly are user inline-C and remain a compiled-path-only feature (see the eval-api guide); use the stdlib helpers instead when you need a program to run under turi.

Lazy Sequences (Seq)

The Seq type wraps a generator-producing thunk. Combinators chain generators without building intermediate collections.

(import seq/core     :refer [seq-from-vec seq-of empty-seq])
(import seq/builders :refer [seq/range seq/repeat])
(import seq/transform :refer [seq/map seq/filter seq/take seq/drop])
(import seq/combine  :refer [seq/zip seq/concat])
(import seq/consume  :refer [seq/into-vec seq/foldl seq/for-each seq/first])

Creating sequences

;; From a range [lo, hi)
(seq/range 0 10)

;; From a range with step
(seq/range-step 0 20 2)      ; 0 2 4 6 8 10 12 14 16 18

;; Wrap a vec
(seq-from-vec [1 2 3 4 5])

;; Single-element sequence
(seq-of 42)

;; Infinite repetition
(seq/repeat "hello")

;; Infinite thunk calls
(seq/repeatedly (fn [] (rand-int 100)))

;; x, f(x), f(f(x)), ...
(seq/iterate 1 (fn [x] (* x 2)))    ; 1 2 4 8 16 ...

;; Cycle a finite sequence infinitely
(seq/cycle (seq/range 0 3))          ; 0 1 2 0 1 2 0 1 2 ...

;; map
(seq/map (fn [x] (* x x)) (seq/range 1 6))
; lazy: 1 4 9 16 25

;; filter
(seq/filter even? (seq/range 0 10))
; lazy: 0 2 4 6 8

;; take / drop
(seq/take 3 (seq/repeat 7))     ; lazy: 7 7 7
(seq/drop 2 (seq/range 0 5))    ; lazy: 2 3 4

;; take-while / drop-while
(seq/take-while (fn [x] (< x 5)) (seq/range 0 100))
; lazy: 0 1 2 3 4

;; map-indexed: receive (index value) pairs
(seq/map-indexed (fn [i x] (pair i x)) (seq/range 10 13))
; lazy: (0 10) (1 11) (2 12)

;; filter-map: map + filter in one pass (None values dropped)
(seq/filter-map (fn [x] (if (even? x) (some (* x 10)) (none)))
                (seq/range 0 5))
; lazy: 0 20 40

;; flat-map
(seq/flat-map (fn [x] (seq/range 0 x)) (seq/range 1 4))
; lazy: 0   0 1   0 1 2

;; flatten
(seq/flatten (seq-from-vec [(seq/range 0 2) (seq/range 5 7)]))
; lazy: 0 1 5 6

;; concat two sequences
(seq/concat (seq/range 0 3) (seq/range 10 13))
; lazy: 0 1 2 10 11 12

;; zip two sequences (stops at shorter)
(seq/zip (seq/range 0 3) (seq/range 10 13))
; lazy: (0 10) (1 11) (2 12)

;; zip-with a combining function
(seq/zip-with + (seq/range 0 3) (seq/range 10 13))
; lazy: 10 12 14

;; interleave: alternate elements
(seq/interleave (seq/range 0 3) (seq/range 10 13))
; lazy: 0 10 1 11 2 12

Consumers (force evaluation)

;; collect to vec
(seq/into-vec (seq/range 0 5))          ; => [0 1 2 3 4]

;; collect to cons list
(seq/into-list (seq/range 0 3))         ; => (0 1 2)

;; nth element (0-indexed)
(seq/nth 2 (seq/range 10 20))           ; => (some 12)

;; first element
(seq/first (seq/filter even? (seq/range 1 10)))  ; => (some 2)

;; count elements
(seq/count (seq/range 0 100))           ; => 100

;; foldl: left fold with accumulator
(seq/foldl 0 + (seq/range 1 6))        ; => 15

;; reduce: fold with first element as seed
(seq/reduce + (seq/range 1 6))         ; => (some 15)

;; for-each: side effects
(seq/for-each println (seq/range 0 3)) ; prints 0 1 2

;; short-circuit consumers
(seq/any? even? (seq/range 1 10))      ; => true (stops at 2)
(seq/all? even? (seq/range 0 6 2))    ; => true  (checks 0 2 4)
(seq/find even? (seq/range 1 10))     ; => (some 2)
(seq/find-index even? (seq/range 1 10)) ; => (some 1)

Chaining with ->> (pipeline style)

(->> (seq/range 0 1000)
     (seq/filter even?)
     (seq/map (fn [x] (* x x)))
     (seq/take-while (fn [x] (< x 10000)))
     (seq/foldl 0 +))
; => 0 + 4 + 16 + 36 + ... + 9604