Getting started with Zirium

This guide covers source setup and the Rust and Python APIs. For the published Python package, see installation. Both APIs follow the same sequence:

source bytes -> parsed file -> semantic document -> verification/editing -> output

Keep the parsed file when exact source text matters. Lower to a semantic document for resolved SSA values, types, symbols, verification, or structural edits.

Prerequisites

Install Rust 1.88 or newer with Cargo. Python development also needs CPython 3.11 through 3.14 and uv. Maturin uses the Rust toolchain to compile the extension.

CPython 3.14 free-threaded builds are experimental and unsupported. Zirium builds version-specific extensions and does not use abi3.

Check the source tree

Clone the repository, then run the workspace tests:

git clone https://github.com/zayenz/zirium.git
cd zirium
cargo test --workspace

This builds both workspace crates and runs the Rust tests. To generate and open the core API documentation:

cargo doc -p zirium --open

Use Zirium from Rust

For a local consumer next to the Zirium checkout, point Cargo at the core crate:

[dependencies]
zirium = { path = "../zirium/crates/zirium" }

This program reports diagnostics, lowers generic MLIR strictly, validates the document, and prints canonical output:

use zirium::{
    dialect::DialectRegistry,
    parser::ParsedFile,
    printer::PrintLayout,
    semantic::{
        LoweringMode, RetentionProfile,
        lower_with_dialect_registry_and_retention,
    },
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let source = br#""builtin.module"() ({
  %value = "example.make"() : () -> i32
  "example.consume"(%value) : (i32) -> ()
}) : () -> ()
"#;

    let parsed = ParsedFile::parse(source.as_slice())?;

    for diagnostic in parsed.lexer_diagnostics() {
        eprintln!("lexer diagnostic: {diagnostic:?}");
    }
    for diagnostic in parsed.syntax().diagnostics() {
        eprintln!("syntax diagnostic: {diagnostic:?}");
    }

    let lowered = lower_with_dialect_registry_and_retention(
        &parsed,
        LoweringMode::Strict,
        RetentionProfile::SemanticOnly,
        &DialectRegistry::EMPTY,
    );

    for diagnostic in &lowered.diagnostics {
        eprintln!(
            "{} {}: {}",
            diagnostic.code.as_str(),
            diagnostic.range,
            diagnostic.message
        );
    }

    let document = lowered
        .document
        .ok_or("strict semantic lowering failed")?;
    document.validate_structure()?;

    let output = document.canonical_bytes(PrintLayout::Pretty)?;
    print!("{}", String::from_utf8(output)?);
    Ok(())
}

ParsedFile::parse uses the empty registry, which accepts generic quoted operation syntax. Use ParsedFile::parse_with_registry and pass the same registry to lowering and semantic verification when the input contains registered custom syntax.

Build the Python package

Create the locked development environment and install the local extension:

uv sync --locked --python 3.13

uv.lock pins the development tools. Replace 3.13 with another supported CPython version if needed. Check the import:

.venv/bin/python -c 'import zirium; print("zirium imported")'

To check parsing as well as importing:

.venv/bin/python - <<'PY'
import zirium

parsed = zirium.parse_text('"test"() : () -> ()')
assert not parsed.diagnostics
assert parsed.operation_count == 1
assert parsed.operation(0).text() == b'"test"() : () -> ()'
print("parsed one operation")
PY

Format the Python code with:

uv run --locked --no-sync ruff format python

Run the checks with:

uv run --locked --no-sync ruff format --check python
uv run --locked --no-sync ruff check python
uv run --locked --no-sync ty check python
uv run --locked --no-sync pytest

Parse syntax without lowering

Parsing malformed input still returns a File with the original bytes, a recovered CST, and diagnostics:

import zirium

source = b'"broken"(  // unfinished operation\n'
parsed = zirium.parse_bytes(source)

assert parsed.original_bytes() == source

for diagnostic in parsed.diagnostics:
    line, column = parsed.line_column(diagnostic.range[0])
    print(f"{line}:{column}: {diagnostic.kind}: {diagnostic.message}")

for index in range(parsed.operation_count):
    operation = parsed.operation(index)
    print(operation.range, operation.has_error, operation.text())

For bulk syntax inspection, build one immutable packed snapshot and cast its native-endian columns with memoryview:

table = parsed.syntax_table()
node_kinds = memoryview(table.node_kind).cast("H")
node_starts = memoryview(table.node_start).cast("I")
subtree_ends = memoryview(table.node_subtree_end).cast("I")

for index in range(parsed.node_count):
    kind = table.node_kind_name(node_kinds[index])
    start = node_starts[index]
    end = subtree_ends[index]
    print(index, kind, None if start == 2**32 - 1 else start, end)

root = parsed.node(0)
children = memoryview(root.child_indices()).cast("I")
first_child = parsed.node(children[0])

The H and I casts match the native-endian u16 and u32 columns. Codes and indexes are snapshot-local and belong only to the source File; use node_kind_code/node_kind_name and token_kind_code/token_kind_name for checked scalar conversion. File.node() and File.token() create wrappers only when requested.

Use parse_text for Python strings, parse_bytes for arbitrary bytes, and parse_file for a path. parse_text encodes its input as UTF-8. The other two paths preserve raw bytes, including invalid UTF-8. Diagnostic ranges are half-open byte offsets. File.line_column() converts an offset to a one-based line and column; columns count Unicode characters, with invalid UTF-8 replaced as it is when decoding Python-facing diagnostic text.

Lower and verify semantic data

Strict lowering returns no document when semantic resolution is incomplete. Best-effort lowering can return a structurally valid document with diagnostics and placeholder values for unresolved data. You can inspect that document, but verification, editing, and canonical output require it to be complete.

import zirium

parsed = zirium.parse_text("""\
%value = "example.make"() : () -> i32
"example.consume"(%value) : (i32) -> ()
""")

result = parsed.lower_strict("semantic")
if result.document is None:
    for diagnostic in result.diagnostics:
        line, column = parsed.line_column(diagnostic.range[0])
        print(line, column, diagnostic.kind, diagnostic.message)
    raise SystemExit(1)

document = result.document
document.validate_structure()

table = document.operation_table()
for index in range(table.count):
    operation = table.operation(index)  # wrappers are created only on demand
    print(operation.name, operation.source_range)

OperationTable is a frozen snapshot. Its name_code, source_start, and source_end columns are native-endian u32 bytes; root_flags is one byte per row (bit 0 denotes a document root); and name_offsets is a native-endian u32 offset table into name_bytes. 0xffffffff in both source columns denotes an absent range. Dense name codes follow first encounter order. The packed columns remain valid after edits; operation(index) checks the retained generation-checked ID and raises StaleHandleError if that operation was erased.

validate_structure() checks ownership, IDs, parent-child links, and operand targets. verify_semantics() also runs the schemas and verifiers in the registry associated with the parsed file.

For custom syntax, select the required built-ins before parsing. This example uses module, function, and return forms:

import zirium

registry = zirium.DialectRegistry.declarative(
    [
        "builtin.module",
        "func.func",
        "func.return",
    ]
)

parsed = zirium.parse_text(
    "builtin.module { func.func @main() { func.return } }",
    registry=registry,
)
result = parsed.lower_strict()
assert result.document is not None, result.diagnostics
result.document.verify_semantics()
print(result.document.custom_bytes().decode(), end="")

The parsed file and semantic document retain the registry, so you do not need to keep a separate Python reference to it. See custom formats for bundled presets and caller-defined operations.

Choose a retention profile

Lowering accepts three retention profiles:

Python value	Rust value	Retained data	Use it for
`"semantic"`	`SemanticOnly`	Semantic storage	Analysis, canonical output, and lower retained memory.
`"syntax"`	`SyntaxOnly`	Semantic storage, source, and CST, without mappings	Semantic inspection that also needs the parsed syntax.
`"hybrid"`	`Hybrid`	Semantic storage, source, CST, and sparse mappings	Semantic edits followed by source-preserving output.

The default is semantic-only. Hybrid retention uses more memory because the document keeps both representations.

Write output

Original output comes from the parsed file and is byte-for-byte identical to its input:

parsed.write_original("unchanged.mlir")

Canonical output comes entirely from semantic storage:

document.write_canonical("canonical.mlir")

Canonical output normalizes formatting and names, producing the same text for the same semantic structure.

Preserving output requires hybrid retention:

parsed = zirium.parse_text("""\
"template"() {tag = "new"} : () -> ()
"target"() {tag = "old"} : () -> ()
""")
result = parsed.lower_strict("hybrid")
assert result.document is not None, result.diagnostics
document = result.document
table = document.operation_table()
template = table.operation(0)
target = table.operation(1)
replacement = template.attribute_by_name("tag")
assert replacement is not None

with document.edit() as edit:
    edit.set_attribute(target, zirium.AttributeSpecHandle(replacement))

preserved = document.preserving_bytes()
assert preserved.count(b'tag = "new"') == 2

with document.edit() as edit:
    edit.erase(template)

try:
    template.name
except zirium.StaleHandleError:
    pass
else:
    raise AssertionError("an erased operation handle remains usable")

An edit context buffers commands and commits on normal exit. Validation failure or an exception leaves the document unchanged. Surviving operation handles keep their identity; erased handles raise StaleHandleError. Handles from another document raise ForeignHandleError.

Semantic operation, region, block, and value wrappers use immutable, document-scoped identity for equality and hashing. Looking up the same live entity again therefore produces an equal wrapper that deduplicates in sets and works as the same dictionary key. Stale wrappers remain comparable and hashable: comparison does not access the erased entity, and a later occupant of a reused slot has a different identity. Wrappers from different documents are never equal, even when their contents match. Accessing data through a stale wrapper still raises StaleHandleError as above.

SemanticType and SemanticAttribute are immutable snapshot/view wrappers, not entity handles. They intentionally retain normal Python object identity: retrieving the same type or attribute twice returns distinct wrapper objects, even though both expose the same immutable semantic contents. Use their public properties when comparing those contents.

Python edit specifications copy existing semantic values: AttributeSpecHandle wraps an existing attribute, and OperationSpec takes existing types and values from the same document. SemanticOperation.function_type() returns the complete checked function type needed by OperationSpec, so a regionless operation can be inserted using an existing operation as its type template:

registry = zirium.DialectRegistry.baseline()
lowered = zirium.parse_text(
    '%template = "vendor.make"() : () -> i32', registry=registry
).lower_strict("semantic")
assert lowered.document is not None, lowered.diagnostics
document = lowered.document
template = document.operation_table("vendor.make").operation(0)
spec = zirium.OperationSpec(
    "vendor.copy",
    [],
    [template.result_type(0)],
    template.function_type(),
)
with document.edit() as edit:
    edit.insert_root(1, spec)

canonical = document.canonical_bytes()
reparsed = zirium.parse_bytes(canonical, registry=registry).lower_strict("semantic")
assert reparsed.document is not None, reparsed.diagnostics
assert document.structurally_equal(reparsed.document)

All values and types must belong to the edited document; stale handles and handles from another document are rejected. Edit commands remain buffered until the context exits normally. These APIs do not parse fresh type or attribute strings, so deriving a genuinely new signature still requires the Rust API’s TypeSpec. Insertion supports regionless operations and returns no provisional handle; look up the inserted operation after the context commits.

Type changes and related rewrites belong in the same transaction. This avoids publishing an intermediate document and pays cloning and verification costs once for the batch:

lowered = zirium.parse_text(
    '%old = "vendor.make"() : () -> i32\n'
    '%new = "vendor.make"() : () -> i64\n',
    registry=zirium.DialectRegistry.baseline(),
).lower_strict("hybrid")
assert lowered.document is not None, lowered.diagnostics
document = lowered.document
old = document.operation_table().operation(0)
new = document.operation_table().operation(1)

with document.edit() as edit:
    edit.replace_result_types(old, [new.result_type(0)])
    edit.replace_all_uses(old.result(0), new.result(0))

assert old.result_type(0).spelling == "i64"
# Non-structural edits retain source/CST and can still preserve output.
assert b'vendor.make' in document.preserving_bytes()

Commands use pre-transaction handles and run in order on a private working document. If any command or final verification fails, none of the batch becomes visible. Insertion and erasure are structural exceptions: after either commits, use canonical_bytes() or custom_bytes() instead of preserving output.

Inspect types and integer attributes

SemanticType exposes structure without requiring callers to parse spelling. Integer types provide integer_width and integer_signedness; float types provide float_name. Tensor, vector, and memref types provide dimensions, scalable_dimensions, and element_type, while tensors also provide unranked. A dynamic dimension is None in dimensions; scalability is kept separately in scalable_dimensions. Tuple types use element_count and element(index), complex types use element_type, and function types use input_count/input(index) and result_count/result(index).

Properties and indexed accessors return None when they do not apply. Opaque types retain their exact spelling fallback and have no structural fields. Nested types returned by these accessors have canonical spellings.

SemanticAttribute.integer_value returns a Python int for integral decimal or hexadecimal spellings, including signed, unsigned, and wider-than-128-bit values. It returns None for float attributes and non-integral wide-number spellings.

Symbol-reference attributes expose symbol_segments as decoded Unicode path components. This preserves the difference between @library::@"part::name" and @library::@part::@name; symbol_value remains the compatibility string formed by joining segments with ::. Direct-call operations provide the same view through callee_segments, plus the normalized callee string and the retained attribute spelling in callee_spelling. Quoted escapes are decoded as UTF-8. A malformed escape or escaped byte sequence that is not valid UTF-8 produces the normal malformed-attribute semantic diagnostic; best-effort lowering exposes no decoded segments for that invalid value.

Preserving output copies unchanged source and regenerates edited operations or blocks, as in the attribute replacement above. Inserting or erasing an operation invalidates source mappings and makes the document semantic-only, so preserving output is no longer available.

Resource limits

The Python parse functions accept keyword-only limits:

parsed = zirium.parse_file(
    "input.mlir",
    max_file_bytes=64 * 1024 * 1024,
    max_tokens=4_000_000,
    max_delimiter_depth=256,
    max_payload_bytes=16 * 1024 * 1024,
    max_numeric_literal_bytes=4096,
    max_attribute_depth=64,
    max_alias_expansion_depth=64,
)

Exceeding max_file_bytes raises ResourceLimitError before lexing. Other syntax limits return a lossless parsed file with diagnostics where recovery is possible. Rust callers configure the same boundaries with ParseLimits and ParsedFile::parse_with_limits. Alias expansion uses a shared budget across type, attribute, affine, memref, and location aliases. Exceeding the default depth of 64 reports alias expansion depth exceeds limit of 64; a custom limit appears in the message instead.