Skip to content
ZirqlZirqlZirql

Query CSV files with Zirql

The CSV example reads each file as a table of string fields, using its first row as column names. You can filter rows, select fields and combine named files in a report. To build an application like this one, follow Build the CSV host.

These commands run from the root of the Zirql 0.3.0 checkout with Rust 1.88 or later. This is a Cargo example executable; installing the library does not install a CSV command.

Use the tabs to compare a query, its input and its result, or copy a command. The commands pass a multiline query as one quoted shell argument. Results below are indented for reading; the executable writes compact JSON by default. Pass --format json or --format markdown after the query to choose the output format.

The common program reader accepts typed param declarations, named functions, and pattern let bindings followed by one final expression. Lets and parameters require ;; the final expression has no semicolon. Names become visible after their declarations, and duplicate or forward bindings fail. Functions receive tables and thresholds as explicit arguments; they cannot capture ambient data. Use repeated --param name=value arguments to supply required scalar inputs. report statements are unavailable in this example. rows and implicit count select the sole input; with several inputs, choose a table by name. Header declarations remain available even when a CSV contains no data rows.

A typed string function can supply tally keys directly:

Terminal window
cargo run --locked --example csv -- 'fn echo(s:string)->string {s} [echo("ash"),echo("ash")] | tally' examples/csv/people.csv

This writes {"ash":2}, the same JSON object as ["ash","ash"] | tally.

Select rows from one file

Section titled “Select rows from one file”

Select active people and keep their names and notes. Switch to CSV input to see the exact source, including CSV quoting and a cell containing a newline.

let selected = rows | filter(_.active == "yes");
[
  {"name": p.name, "note": p.note}
  for p in selected
]

rows names the sole input table. Filtering retains file order. CSV quoting preserves the comma in Ada’s name, the quote in Grace’s note and the newline in Ada’s note. JSON escapes that newline inside its string. Empty cells stay empty strings. All cells are strings, including id, yes, no and 01; the host does not guess types or trim labels and values.

filter(p => p.active == "yes") names each row; filter(_.active == "yes") uses the unnamed row argument. Both select the same rows as [p for p in rows if p.active == "yes"]. Predicates can capture earlier bindings:

let wanted = "yes";
rows | filter(_.active == wanted)

To count the rows, pipe the table into count:

Terminal window
cargo run --locked --example csv -- \
  'rows | count' \
  examples/csv/people.csv

This prints 4. The shorter count expression uses the same table implicitly. To count only active people, reuse the saved selection:

Terminal window
cargo run --locked --example csv -- '
let selected = rows | filter(_.active == "yes");
selected | count
' \
  examples/csv/people.csv

This prints 3.

Render a Markdown table

Section titled “Render a Markdown table”

Use the same selection with --format markdown to produce a table:

Terminal window
cargo run --locked --example csv -- '
let selected = rows | filter(_.active == "yes");
[{"name": p.name, "note": p.note} for p in selected]
' --format markdown examples/csv/people.csv
| name | note |
| --- | --- |
| Grace Hopper | Said "hello" |
| Ada, Lovelace | First line<br>second line |
| Barbara Liskov |  |

The output includes a blank line before and after the table. Columns follow the first-seen field order, and rows follow the selected input order. Missing and absent cells are blank. Pipes and Markdown punctuation are escaped; angle brackets and ampersands use HTML entities, and newlines become <br>. Nested cells and structural dictionary keys use compact JSON with the same escaping.

A scalar count prints on its own line, scalar collections use a Value column, and records or dictionaries use Key/Value columns. A keyed map of record rows adds Key before the field columns. Empty collections and maps print _No entries._. See Markdown rendering for the complete shape rules and Rust API.

Parsing, evaluation and rendering share one budget. The renderer checks the complete rectangular table size before expansion, so many sparse columns can exceed the limit even with few input rows. A failure leaves stdout empty. --format json selects the default compact JSON output; positional and named file arguments work with either format.

Map people within each team

Section titled “Map people within each team”

Use a checkout containing the mapping stages. This command visits teams in teams.csv order, filters matching active people, and maps each person to a report record. The inner _ denotes a person; t captures the outer team.

Terminal window
cargo run --locked --example csv -- '
teams | flat_map(t =>
  people
  | filter(_.team == t.id and _.active == "yes")
  | map({"team": t.label, "person": _.name})
)
' \
  people=examples/csv/people.csv teams=examples/csv/teams.csv

The exact output is:

[{"team":"Runtime","person":"Barbara Liskov"},{"team":"Compiler","person":"Grace Hopper"},{"team":"Compiler","person":"Ada, Lovelace"}]

flat_map concatenates the per-team positional results once. Operations has no matching people and contributes no records; the two Compiler people retain file order. map alone would retain a nested sequence per team. Both stages are eager, validate written bodies on empty input, and share the request and stream limits. Their inputs and flat_map results must be positional sequences. The capacity example uses the same stages on nested numeric data.

Use your own CSV file

Section titled “Use your own CSV file”

Replace the fixture path with your file. Counting rows needs no column names:

Terminal window
cargo run --locked --example csv -- \
  'rows | count' \
  './my data.csv'

Use rows as the query to inspect the decoded records. Then adapt a selection by changing its field names and string comparisons to match your headers and values. Quote paths containing spaces; for a named input, quote the whole argument, such as 'people=./my data.csv'.

The input must be comma-separated UTF-8 with a header row. Headers must be nonempty and unique, and every data row must have the header’s field count. The example accepts at most 64 KiB across all input files and at most 1,000 data rows per table. These are application limits; the host tutorial explains where to change them.

An identifier header supports access such as p.name. For a header containing spaces, use p["full name"] or lookup(p, "full name"). A column name can also come from a binding:

Terminal window
cargo run --locked --example csv -- '
let column = "name";
[p[column] for p in rows if p.active == "yes"]
' \
  examples/csv/people.csv

This returns ["Grace Hopper", "Ada, Lovelace", "Barbara Liskov"]. Brackets read the computed string key; p.name selects a fixed label. An unknown label fails because the CSV header declares the available columns. Strict field access is checked even when a table has no data rows.

Cells remain strings, so sorting an amount column sorts its text: "10" comes before "2". An empty cell is "", not null. Use explicit scalar conversions for numeric calculations. Handle an empty cell explicitly before parsing it, or let conversion report the malformed value.

Calculate with numeric cells

Section titled “Calculate with numeric cells”

examples/csv/orders.csv keeps quantities and prices as strings:

item,quantity,price
apples,02,1.25e1
pears,10,2.5
plums,1,3

The saved examples/csv/numeric-orders.zirql declares a minimum quantity and a report label. Its named function receives the table, threshold and label explicitly, parses quantity for filtering and summation, and parses price for line totals. F-strings format prices and totals in each line label to two decimal places:

param minimum_quantity: int;
param report_label: string;
fn report_orders(
  orders: sequence<record {item: string, quantity: string, price: string}>,
  minimum: int,
  label: string
) -> record {label: string, quantity: int, lines: sequence<record {item: string, total: float, label: string}>} {
  let selected = orders | filter(parse_int(_.quantity) >= minimum);
  {
    "label": label,
    "quantity": sum([parse_int(row.quantity) for row in selected]),
    "lines": [{
      "item": row.item,
      "total": parse_float(row.quantity) * parse_float(row.price),
      "label": f"{row.item}: {parse_int(row.quantity)} × {parse_float(row.price):.2f} = {parse_float(row.quantity) * parse_float(row.price):.2f}"
    } for row in selected]
  }
}
report_orders(rows, minimum_quantity, report_label)

Run the sole input through rows, with both required parameter values:

Terminal window
cargo run --locked --example csv -- \
  "$(cat examples/csv/numeric-orders.zirql)" \
  --param minimum_quantity=2 --param report_label=orders \
  --format json examples/csv/orders.csv
cargo run --locked --example csv -- \
  "$(cat examples/csv/numeric-orders.zirql)" \
  --param minimum_quantity=2 --param report_label=orders \
  --format markdown examples/csv/orders.csv

To call the same function with a named table, change only the final call’s input:

Terminal window
cargo run --locked --example csv -- \
  "$(sed 's/report_orders(rows,/report_orders(orders,/' examples/csv/numeric-orders.zirql)" \
  --param minimum_quantity=2 --param report_label=orders \
  --format json orders=examples/csv/orders.csv
cargo run --locked --example csv -- \
  "$(sed 's/report_orders(rows,/report_orders(orders,/' examples/csv/numeric-orders.zirql)" \
  --param minimum_quantity=2 --param report_label=orders \
  --format markdown orders=examples/csv/orders.csv

The parameter strings are values, not query fragments. name=FILE arguments still select files; --param name=value supplies parameters. Missing, duplicate, unknown or mistyped parameters fail before execution.

The report has label "orders" and quantity 12, with line totals 25.0 and labels "apples: 2 × 12.50 = 25.00" and "pears: 10 × 2.50 = 25.00". The original "02" and exponent spelling remain in their CSV cells. The scalar reference defines both decimal grammars. Whitespace is rejected without trimming; " 2" fails just like malformed or empty text. These two commands deliberately fail at the conversion and leave stdout empty:

Terminal window
cargo run --locked --example csv -- \
  '[parse_int(row.item) for row in rows]' examples/csv/orders.csv
cargo run --locked --example csv -- \
  '[parse_int(row.note) for row in rows if row.id == "3"]' examples/csv/people.csv

Reuse selections across named files

Section titled “Reuse selections across named files”

Name each input with name=path. Save the active people, build a member list for each team, then use those saved lists for both sorted names and counts.

let selected = people | filter(p => p.active == "yes");
let groups = [
  {
    "team": t.label,
    "members": [p.name for p in selected if p.team == t.id]
  }
  for t in teams
];
[
  {
    "team": g.team,
    "members": sort(g.members),
    "count": length(g.members)
  }
  for g in groups
]

The outer comprehension follows team-file order. Its inner comprehension sees the current team and the saved selected value. sort puts Ada before Grace within the Compiler list, while length counts its original members. Operations has no active members and still appears with an empty list. JSON keeps both array order and the written team, members, count field order.

The member field can also start a pipeline: g.members | sort gives the same result as sort(g.members).

Bindings are immutable and visible after their initializer. A let cannot replace a table name, rows, or an earlier binding. Comprehension names stay inside their expression. The example accepts typed parameters, pure named functions, ordered pattern let bindings and one final value expression through ConstructedProgram. Functions receive table and parameter values explicitly, as in the numeric orders report.

Use exactly one unnamed file, or only named files. Named tables use ASCII identifiers: a letter or underscore followed by letters, digits or underscores. _, rows, input, literals and control/operator keywords are reserved. This external table-name rule uses Zirql’s common keyword classification plus the CSV rows alias. A callable name such as lookup can still name a table. Names must be unique. One named table also supplies rows and the implicit current input. With several tables, use their names explicitly; neither rows nor count chooses a file for you.

Sort rows and reuse the result

Section titled “Sort rows and reuse the result”

Constructed sort_by(rows, "name") takes both a collection and a field name. Its comma belongs to the call; commas or closing delimiters inside a string or comment do not change how the reader classifies it. A saved result accepts the same constructed suffixes:

Terminal window
cargo run --locked --example csv -- '
let ordered = sort_by(rows, "name");
ordered | map(_.name) | count
' examples/csv/people.csv

This prints 4. Hosts with native stages also accept selector forms such as Zirium’s sort_by(names); the shared reader distinguishes those calls before reading their arguments.

Build a dictionary report

Section titled “Build a dictionary report”

When each team needs one count, use its label as a dictionary key. This example uses the same people and teams files as the previous report.

let selected = people | filter(p => p.active == "yes");
sort_by(
  {
    t.label: length([p for p in selected if p.team == t.id])
    for t in teams
  },
  value,
  descending=true
)

The inner comprehension counts selected people in the current team. Every team gets an entry, including Operations with no active members. sort_by orders the counts from largest to smallest and retains the dictionary kind. Equal counts would keep their incoming team order. Duplicate labels fail because a dictionary requires distinct keys.

Use patterns and other expression forms

Section titled “Use patterns and other expression forms”

Patterns can extract fields in a predicate or a let binding. Select the first active person, then extract their name:

Terminal window
cargo run --locked --example csv -- '
let [first, ..] = rows | filter(
  {active: status, ..} => status == "yes"
);
let {name, ..} = first;
name
' \
  examples/csv/people.csv

This returns "Grace Hopper". The record pattern binds the active field to status; .. ignores the other fields. The list pattern requires at least one selected row. If none exists, the let fails at [first, ..] and stdout remains empty.

Both comprehension spellings use the same reader and evaluator:

Terminal window
cargo run --locked --example csv -- \
  '[p.name for p in rows if p.active == "yes"]' \
  examples/csv/people.csv


cargo run --locked --example csv -- \
  '[name | {name, active, ..} in rows where active == "yes"]' \
  examples/csv/people.csv

Both return the three active names in file order. The same patterns and lexical rules apply to named inputs.

Nested filters can refer to an outer argument. To keep only teams with active people:

Terminal window
cargo run --locked --example csv -- '
teams | filter(t =>
  length(people | filter(_.team == t.id and _.active == "yes")) > 0
)
' \
  people=examples/csv/people.csv \
  teams=examples/csv/teams.csv
[
  {"id": "runtime", "label": "Runtime"},
  {"id": "compiler", "label": "Compiler"}
]

The outer filter binds t to a team. Inside the inner filter, _ is a person and t remains available for the comparison. Operations is omitted because its selection is empty.

See values and comprehensions for expression syntax and the language reference for collection stages, conditionals and indexed literals.

Inspect an indexed value

Section titled “Inspect an indexed value”

The reader can also construct values addressed by integer keys. The saved indexed example retains keys 5, 2, 9, with cells at 5 and 2:

Terminal window
cargo run --locked --example csv -- \
  "$(cat examples/csv/indexed-arrays.zirql)" \
  examples/csv/people.csv
{
  "keys": [5, 2, 9],
  "cells": [
    {"key": 5, "value": 50},
    {"key": 2, "value": 20}
  ],
  "second": 20,
  "has_nine": false
}

index_set keeps unused keys; entries visits only present cells. You can also return the keyed value directly:

Terminal window
cargo run --locked --example csv -- \
  '[2: 20, 5: 50] axes([5, 2, 9])' \
  examples/csv/people.csv
{
  "kind": "array",
  "indexing": "explicit",
  "axes": [[
    {"kind": "integer", "value": 5},
    {"kind": "integer", "value": 2},
    {"kind": "integer", "value": 9}
  ]],
  "entries": [
    {"key": [{"kind": "integer", "value": 5}], "value": {"kind": "integer", "value": 50}},
    {"key": [{"kind": "integer", "value": 2}], "value": {"kind": "integer", "value": 20}}
  ]
}

The shared JSON renderer keeps the declared key 9 without adding a cell for it. See keyed arrays for row and column domains, typed empties and sparse cells.

Understand errors

Section titled “Understand errors”

A missing column fails at its original query range:

Terminal window
cargo run --locked --example csv -- \
  'rows | filter(_.missing == "yes")' \
  examples/csv/people.csv

Several named tables make an implicit request ambiguous:

Terminal window
cargo run --locked --example csv -- \
  'count' \
  people=examples/csv/people.csv \
  teams=examples/csv/teams.csv

A finite range can exceed the request’s result allowance before it is built:

Terminal window
cargo run --locked --example csv -- \
  '1..100001' \
  examples/csv/people.csv

Each command exits unsuccessfully with empty stdout. Query diagnostics include the submitted text, byte range and available table paths. CSV failures identify the table and path, with the decoder’s record, line and byte position where available. Malformed query syntax also retains its original byte range. A failed request returns no partial JSON.

How the example evaluates a query

Section titled “How the example evaluates a query”

The example loads at most 64 KiB of CSV source bytes across all input files, using bounded reads. It supplies constructed arrays and records, header types, named-table lookup and a call to Zirql’s JSON renderer. Input loading, decoder admission, copies, parsing, evaluation and output share one budget: 1,000,000 work units, 100,000 produced or copied results, nesting depth 64 and a 1,000-item stream ceiling. Work includes scanned and copied bytes, field metadata and output bytes. The byte cap bounds the CSV decoder’s temporary buffers as well.

ConstructedBinding admits each table against its header-derived declaration. ConstructedProgram reads functions, typed parameters, ordered pattern lets and the final expression, then prepares and evaluates them with the same budget. The example needs no native reader or value-query traits. CSV retains decoding, exact string cells, input-byte limits and the sole/named-table selection policy.

The shared JSON and Markdown renderers format the completed value. run returns a complete buffer; main writes only after success, so a later conversion, evaluation or rendering error leaves stdout empty.

Build the CSV host walks through loading, declaring and binding a table, supplying parameters and returning JSON or Markdown. The minimal embedding starts with values already in memory. Use native integration when your application also needs native handles, source coordinates or domain calls.

Keep distinct rows and count repeated rows

Section titled “Keep distinct rows and count repeated rows”

Rows use structural equality, including each field name and value. unique retains the first row in each group and its incoming position. tally retains the first row as the dictionary key and counts its occurrences. Dictionary results support the usual selection and ordering operations.

Terminal window
cargo run --locked --example csv -- 'rows | unique | count' examples/csv/people.csv
cargo run --locked --example csv -- 'rows | tally | reverse | head(1)' examples/csv/people.csv

The first command returns 4; the second keeps the last distinct row and its count. Row keys remain records rather than becoming rendered strings.