Programs and native data

A single expression is enough for each analysis in the word tutorial. A program lets several analyses share named intermediate results. Its bindings can hold constructed values, native selections or records of related native objects, and its statements execute in written order.

Zirql’s read_program owns common declaration collection, lexical names and statement assembly. The host supplies native syntax, admission and output handling, or translates its typed builder into the same program representation. The word executable and CSV example use ConstructedProgram for constructed data with functions, parameters and ordered lets. Native extensions require the corresponding native host interfaces.

Statements and scope

The CSV example accepts ordered let bindings and a final expression:

cargo run --locked --example csv -- '
let selected = [p for p in rows if p.active == "yes"];
{
  "names": [p.name for p in selected],
  "count": length(selected)
}
' \
  examples/csv/people.csv

It returns ["Grace Hopper", "Ada, Lovelace", "Barbara Liskov"] under names, with count 3. let installs an immutable name. Later statements can use it, but earlier statements cannot refer forward to it. Duplicate definitions and shadowing of existing bindings fail before execution. Function names are collected across the whole program, including declarations written after their callers. Functions cannot capture those program bindings, parameters or the current input.

Each statement starts with the original top-level input. A previous statement does not replace that input. Bind its result and refer to the binding explicitly when a later statement should use it.

Native frontends also support explicit output statements. The host defines what report does with the result, including formatting and whether it produces output for a library call. Zirium supports pass-through emitters; Zincite’s output stages are terminal. Their Zirium and Zincite guides document the actual commands. Build the CSV host explains how its constructed-program API reads and prepares the common program. CSV’s final expression renders JSON or Markdown. It accepts typed parameters and named functions, with no final semicolon or report statement. Native hosts accept an optional final semicolon; every binding and parameter requires one.

A Rust embedding lowers parameter declarations to Statement::Parameter and pattern lets to Statement::BindPattern. PreparedProgram checks every statement before execution, carrying kinds forward only after each declaration. Its shared statement driver installs pattern fields atomically and passes the actual current input and shared bindings to entry expressions. Named functions receive explicit arguments. The embedding guide describes the prepare/run sequence.

Required caller parameters

Required parameters let a caller supply typed data without constructing query text:

param prefix: string;
let selected = words | filter(w => starts_with(w, prefix));
report {"words": selected, "count": length(selected)};

Parameter types are int, string and bool. Missing, duplicate, extra or incompatible supplied values fail before domain execution. Defaults are not supported. A parameter becomes available at its written declaration and follows the same no-shadowing rule as other bindings.

Library callers can pass typed values. A host CLI using the shared name=value decoder accepts these forms:

Type	Decoding
string	Preserve all text after the first =, including spaces, quotes and additional = characters
int	Optional sign followed by decimal digits fitting a signed 64-bit integer
bool	Exactly true or false

Values are not substituted into the program’s source. The host documents how to pass them through its command-line interface.

Saved native values

Saving a native selection retains its objects and their source context. A word occurrence can remain attached to the source text, and a document handle can remain attached to its owning document. Saving or reading the binding does not render it.

A record can retain several related native values. Field selection preserves the selected field’s native kind. A later projection decides which properties become scalar report data.

Project a native selection

Zirium can retain MLIR operations and explicitly read their names and body sizes. From a Zirium checkout:

zirium 'filter(op("func.func")) | project(f, {"name": f | attr("sym_name") | one, "n": f | children | count}) | json' examples/cli/function-report.mlir

The result has rows {"name":"double","n":2}, {"name":"mix","n":3} and {"name":"plain","n":1}. Each f is one native function operation. Both fields read that same operation; one requires exactly one name, while count counts its immediate children. The host supplies the MLIR operations and attributes. Zirql evaluates projection fields and assembles their dictionaries. The common reader and Rust builders represent the projection as a core ExpressionStep::Projection. Preparation checks every written field once, including on empty input, and preserves the result schema for later stages. Native filters, matches, ordering, maps, existence, fixed points and paths also use direct core steps. Their bodies keep one active lexical/function frame; hosts supply native subjects, kinds, guard facts, identity, shape and explicit scalar projections. The Rust embedding guide shows the read/build, prepare and evaluate path in the Rust API.

The Zirium language guide explains its native stages. Zincite similarly projects MiniZinc items and declarations; its query guide distinguishes written syntax from facts that need model context.

Construct ordered report rows

The remaining examples assume a richer word model with native occurrences. Suppose its host supplies text and byte_length, each producing one scalar per occurrence. A projection can build a row for each word:

project(w, {
  "word": one(w | text),
  "bytes": one(w | byte_length)
})

On ash apple birch apple, this gives rows ash/3, apple/5, birch/5, apple/5. Both occurrences of apple survive. w is local to the projection; its fields evaluate in written order against the same native subject.

one requires exactly one projected value. Empty or multiple results fail. A missing domain operation or incompatible field type fails preflight even when the input selection is empty. These operations are an illustrative extension of the word model; they are not implemented by the tutorial executable.

Optional projected fields

Value expressions inside a projection use the same optional-data rules as other expressions. If the host supplies metadata as a single record for one word:

project(w, {
  "word": one(w | text),
  "label": lookup(one(w | metadata), "label") default "unlabelled"
})

A missing label gets the fallback. An error obtaining metadata still fails; default handles absence, not operation failures. All branches retain scope, kind and purity checks before any native field executes.

Retain native captures

Native matching enumerates related objects and keeps them together in a record. An occurrence host could supply has_next, next_word and same_text:

match {
  from word in (input | filter(has_next));
  bind following = next_word(word);
  where same_text(word, following);
}

For ash apple apple birch, this finds the adjacent apple occurrences. The capture has separate word and following fields with distinct source positions. The adapter defines those operations; the core defines the clause behavior:

Clause	Meaning
from name in query	Enumerate candidates in native order
bind name = query	Require exactly one native subject for each candidate
where predicate	Continue only when the final predicate answer is known true

Later clauses see earlier names. A name is absent in its own source, and the rightmost from varies fastest. Captures stay local to the match. Duplicate names, shadowing and a binding named input fail.

A false guard rejects a candidate. An unknown final guard or a failed required binding is an error. Empty input does not bypass preflight. Captured handles retain their owner until an explicit projection creates scalar data.

Value matching, written match (value) { ... }, instead selects one result. See functions and control flow.

Explain native connections

A host can expose a named edge for bounded path queries:

paths(next_word, to=prefix("birch"), max_depth=2)

With a native source occurrence ash in ash apple birch, this returns one witness with steps [ash, apple, birch]. The host supplies the next_word edge, the prefix predicate and occurrence identity. max_depth is a required nonnegative integer measured in edges. An edge is one registered identifier, not a pipeline.

The result has source, sink and steps fields. Steps include both endpoints; a source that already matches the target has a zero-edge witness [source]. For each source/sink pair, Zirql chooses one shortest witness within the bound. Sources retain incoming identity order, sinks follow breadth-first discovery, and native neighbour order breaks ties between equally short witnesses. Repeated handles and cycles do not duplicate a pair. Searching continues through matching sinks.

A bound of one finds no witness from ash to birch here. That does not prove global disconnection. An unknown final target predicate fails, and separate host limitations remain attached to the request. Exhaustion is an error rather than a truncated set of witnesses.

Written and implicit output

A host can define terminal output, which ends an expression, or pass-through output, after which stages can continue. It can also choose implicit output for a final expression without a written emitter. The host’s guide specifies that policy and its syntax.

Pure bindings, constructed operands and functions cannot emit or edit. Explicit host effects have the host’s transaction rules; a later failure does not necessarily undo output already emitted. See diagnostics and limits.

Predicate headers can destructure exposed tuples, records and positional lists, for example [first, second, ..] => body. The host must establish those views; Zirql adds no native fields or implicit collection conversions. A mismatch is a located error, and _ keeps the whole subject. Nested predicates bind _ anew while explicit outer names retain their captures. The adapter pattern contract describes the required view and type hooks.

Named declarations, recursion and patterns have their own language chapter. Rust copying, preparation and output contracts are covered by host integration, program construction and execution costs.

Order native selections

Native sorting uses a key projected from each object. In Zirium, this orders functions by their symbol names:

filter(op("func.func")) | sort_by(attr("sym_name")) | head(10)

The selector runs against one operation at a time. Sorting keeps equal-key operations in their incoming order. Single extrema keep the first equal-key item; all-ties extrema keep every tied item in input order. Selectors execute in input order before comparisons begin, including for extrema.

Native counts, text and supported ordinary data use the same structural ordering as constructed keys. Counts and JSON unsigned integers retain their full range; floats retain a separate rank. Native scalar projections must produce exactly one value. An ordinary compound is itself one key. Opaque handles require an explicit supported projection. Missing fields, invalid selector cardinality and inconsistent nominal metadata fail even when the key cannot win.

Empty native extrema retain their input kind, element declaration and owner after ordinary kind, selector and effect preflight.

Constructed collection extrema instead return absence on empty input. See the host language guides for their stages, and ordering adapters for the Rust key representations and lowering APIs.

Correlated native dictionaries

map_by(key_query, value_query) evaluates two independent queries against each native input item. The key must produce exactly one string. Inputs run in their original order, key before value; a duplicate key fails before its value runs. Values retain their native kind and owner, including nested dictionaries and counts. The host checks supported kinds and pure selector policy before execution, even for empty input.

A native report can map function names to body sizes in Zirium:

filter(op("func.func"))
| map_by(attr("sym_name"), children | count)
| sort_by(value) | reverse | head(2) | json

On examples/cli/function-report.mlir, it returns {"mix":3,"double":2}. The dictionary keeps its kind through sorting, reversal and bounds. Equal values keep their current order during sorting; an explicit reversal also reverses ties.

Dictionary values use that same structural contract, including Boolean, absent and compound data. Integer and float kinds remain distinct. Native JSON arrays and objects compare structurally without rendering or narrowing unsigned leaves. The embedding guide documents the callable constructor, map sorting and bounds APIs.

Collection order in native reports

The shared library keeps insertion order for constructed sets and dictionaries. Set duplicates retain their first representative. Constructed sorting returns the same collection kind. Dictionary equality compares associations and set equality compares membership, independently of stored order. Explicit keyed axes retain their shape contract; native fixed-point streams still use ordered equality.

For example, these expressions run with the word example host:

sort({"z": 2, "a": 2, "m": 1})
sort_by({"z": 2, "a": 2, "m": 1}, value)
sort_by({"z": 2, "a": 2, "m": 1}, value, descending=true)

They return dictionaries ordered a, m, z, m, z, a and z, a, m. Equal sort keys retain their incoming order in both directions. Sorting again after reversal keeps the reversed order within ties. Tally entries follow the first occurrence of each key. Empty dictionaries and zero bounds retain the dictionary kind. A dictionary never implicitly becomes a list of entry records.

Host adapters retain this entry order through saved results, copies and nested values. JSON object member order is observable here: a renderer must iterate the retained entries rather than collect them into an automatically sorted map. Native handles and source coordinates survive until explicit rendering.

Zirium retains ordered dictionaries through its library and JSON results and uses operand-order native set algebra. Zincite uses shared typed native ordering, operand-order sets and first-seen tally dictionaries. Its map_by dictionaries support stable value sorting and bounds; string dictionaries render as ordered JSON objects. Both support the shared brace and indexed-array forms. General constructed dictionaries require host adoption and an explicit output encoding; these released hosts pin earlier Zirql versions. Native identity, effects, completeness and data-edit policies stay with each host.

Native existence

Supporting native hosts read where_exists(query) at the start of an expression, in a group or after a pipeline. The pure relation runs against each singleton subject with the current lexical scope. A nonempty native result keeps that original subject, preserving its order and duplicates. A complete empty result rejects it. Unresolved nonemptiness and nested errors fail the query, including a failure after earlier matches. Hosts preflight kinds, selectors and effects even when the incoming stream is empty. Zirium supports ordinary and projected diff relations; Zincite supports pure native report relations. Zincite normalizes a singular node or declaration to its possibly empty selection kind.

Native fixed points

A supporting native host reads fixpoint(query) through the common reader. Each iteration replaces the current selection with the body’s result. Equality is positional and includes duplicates; intermediate selections do not accumulate. The final body execution establishes an unchanged result. Repeating a non-fixed selection fails, using one bounded cycle checkpoint.

Zincite supports this accumulating form:

let seed = declarations | filter(name("C"));
seed | fixpoint(input union referenced_declarations)

The union explicitly retains the current members. Zincite bodies are pure report computation and must return the same admitted native kind. Owners, identity and the active budget survive every iteration. Partial adjacency cannot prove convergence. Zincite uses complete-or-error adjacency and refuses saved partial selections at their responsible locations while retaining other report limitations. Domain closure, paths and specialized reachability remain separate operations.

Zirium entry programs also admit their existing pass-through emission in native fixed-point bodies. Emission occurs on every iteration, including the final unchanged result. Edits, count/map results and effects inside pure function or constructed contexts remain refused. Its query guide defines the entry-program output policy.

Predicate argument subjects

Native predicate bodies can use the host’s permitted expressions with name => body or _. Native filter receives one subject; root tests each candidate operation or ancestor, and path to= tests each candidate target. A native match where body receives the original incoming input, including its collection shape, while earlier captures remain separate lexical names. Diff filters retain their existing change or side-owned operation subject. The host’s projections, domain leaves and effect rules still determine which expressions are meaningful. See scoped predicate adapters for reader, preparation and active-runner obligations.

Collection selection parity

The shared collection operations apply to native selections/maps and ordinary positional arrays, sets and dictionaries. Predicates receive each dictionary value while retained entries keep their exact keys and order. Native subjects retain their host identity, ownership and completeness. Runtime bounds and check messages use prepared arguments with existing native relative-input rules. check returns its input unchanged on success. Explicit keyed arrays support count/length and require conversion before removal/reversal.