Grammar of Graphics in koryki

koryki turns a data query into a chart with a VISUALISE clause — a declarative Grammar of Graphics (GG) appended to a KQL query and compiled to a Vega-Lite v6 specification. This document explains the Grammar of Graphics and exactly how koryki implements it.

The VISUALISE clause is inspired by ggsql; the grammar, compiler, and semantics here are an independent Java/ANTLR implementation.

1. What is the Grammar of Graphics?

The Grammar of Graphics (Wilkinson, 2005; popularised by ggplot2 and Vega-Lite) says a chart is not a chart type ("bar chart", "pie chart") but a composition of independent parts. Change one part and you get a different chart from the same data:

GG component	Question it answers	Example
Data	what rows?	the query result
Statistical transform	summarise the data first?	bin, count, quartiles, regression
Mark / geom	what shape draws a row?	point, line, bar, area
Aesthetic mapping	which column drives which visual channel?	`revenue → y`, `category → colour`
Scale	how do data values map to pixels/colours?	linear, log, a colour scheme
Coordinate system	what space?	cartesian, polar, a map projection
Facet	split into small multiples?	one panel per category
Guides / labels	axis titles, legend, formats	`"Revenue (€)"`, `%Y-%m`

The power is orthogonality: DRAW bar vs DRAW line swaps the mark without touching the mapping; FACET category adds faceting without touching anything else.

2. How koryki adopts it

A KQL query already defines the data (FIND … FILTER … FETCH …). The VISUALISE clause adds the graphics layer on top of the same query. It is optional — not every result has a meaningful chart.

FIND orders o, o order_details od, od products p, p categories c
FETCH c.category_name category, month(o.order_date) month,
      sum(od.unit_price * od.quantity) revenue
VISUALISE month AS x, revenue AS y, category AS color   -- aesthetic mappings
DRAW line                                                -- mark / geom
LABEL title => 'Revenue per category and month'          -- guides

Each GG component maps to one keyword:

GG component	KQL keyword	Compiles to (Vega-Lite)
aesthetic mapping	`VISUALISE col AS channel`, per-layer `MAPPING`/`REMAPPING`	`encoding`
mark / geom	`DRAW <geom>` (layers stack)	`mark` / `layer[]`
annotation	`PLACE <geom> SETTING …`	own-data annotation layer
statistical transform	`DRAW histogram`/`boxplot`/`smooth`/… or `SETTING aggregate => …`	data computed in the DB
scale	`SCALE [type] channel FROM … TO … VIA … RENAMING …`	`encoding.<ch>.scale`/`axis`/`legend`
coordinate system	`PROJECT [aes] TO <coord> SETTING …`	axis swap / `arc` / `projection`
facet	`FACET vars [BY vars] SETTING …`	`facet` + `resolve`
guides / labels	`LABEL target => 'text'`	`title` / `axis.title` / `legend.title`

Convention (shared with the rest of KQL): keywords are UPPERCASE, everything else — channels, geoms, coordinate names, palettes, transforms — is lowercase ID, validated semantically by the compiler rather than reserved in the grammar.

3. The compilation pipeline

KQL text
  │  ANTLR grammar  (core/src/main/antlr/kql/KQL.g4 — visualiseClause)
  ▼
Parse tree
  │  KQLQueryMapper.toVisualise(...)
  ▼
Visualise AST            (ai.koryki.iql.query.viz.*)
  │                        Visualise · Layer · Mapping · ScaleSpec ·
  │                        FacetSpec · Projection · Label · Rename
  ▼
VegaLite emitter         (ai.koryki.viz.VegaLite)          ── statistical geoms ──►  StatTransform
  │  builds a Jackson ObjectNode tree                          (ai.koryki.viz.stat.*)
  ▼                                                            computes bins / quartiles /
Vega-Lite v6 JSON                                              regression / KDE **as SQL**,
                                                               runs it, inlines the result

4. Aesthetic mappings and channels

VISUALISE country AS x, revenue AS y, category AS color binds output columns (FETCH aliases) to visual channels. Per-layer MAPPING adds channels for one DRAW; REMAPPING overrides a channel (typically onto a statistical-transform output). A mapping to a column not in the query's output is rejected with a clear error.

Channels (KQL name → Vega-Lite channel):

positional: x, y, x2, y2
colour: color (colour), fill
size/shape: size, shape, opacity
text: text (label), tooltip, detail
polar: theta, radius
geographic: longitude (lon), latitude (lat)
choropleth: geometry
styling: strokeWidth (linewidth), strokeDash (linetype), size (fontsize)

A literal instead of a column produces a constant channel value; * (wildcard) auto-maps every output column whose name is a known channel.

5. Marks / geoms

DRAW <geom> chooses the mark; multiple DRAWs stack into a layer[]. KQL geom → Vega-Lite mark:

geom	mark	geom	mark
`point`	point	`text`	text
`line` / `path`	line	`segment` / `rule` / `range`	rule
`bar`	bar	`tile`	rect
`area` / `ribbon`	area	`polygon`	line
`spatial`	geoshape (choropleth)

Statistical geoms (§7) are marks whose data is computed first: histogram, boxplot, smooth, density, violin, heatmap.

PLACE <geom> is an annotation layer — every aesthetic is a literal from SETTING (positional via datum, visual via value), drawn once over its own single-row data.

6. Scales, coordinates, facets, labels

SCALE [type] channel FROM domain TO range VIA transform SETTING … RENAMING …

type: CONTINUOUS · DISCRETE · BINNED · ORDINAL · IDENTITY
FROM [..] → scale.domain; TO [..]/TO palette → scale.range/scale.scheme
VIA transform → scale.type: log/ln, log10, log2, sqrt, square, symlog (and date/datetime/time → temporal)
SETTING breaks => … → axis/legend values/tickCount; reverse => true
RENAMING 'a' => 'b' → axis/legend labelExpr

PROJECT [aes] TO <coord> SETTING … — the coordinate system:

cartesian (default). Listing aesthetics swaps axes (PROJECT y, x TO cartesian).
polar → arc mark; y→theta, x→radius; SETTING inner => … for a donut.
map projections (~24 names: mercator, orthographic, albers, equal_earth, natural, …) → a top-level Vega-Lite projection; x/lon→longitude, y/lat→latitude; SETTING origin => (lon,lat) → rotate, parallel => (a,b) → parallels, sphere/graticule => true → background layers. d3 projects client-side, so point maps work on every dialect with no DB spatial support. Arbitrary CRS (target/source) is rejected with a clear message.

FACET vars [BY vars] SETTING … — small multiples: one field → facet; BY → row/column grid; SETTING ncol => n, free => [x,y] → resolve.scale independent.

LABEL target => 'text' — title sets the chart title; a channel name (x, y, color) sets that guide's title.

7. Statistical layers — computed in the database

koryki does not ship a client-side stats engine. A statistical geom is compiled to SQL, executed on the target database, and the aggregated result is inlined into the spec. This keeps the browser payload small and pushes the heavy lifting to the DB.

ai.koryki.viz.stat.StatTransforms registers:

geom	transform	computes (SQL)
`histogram`	`HistogramStat`	binned counts (`width_bucket` / bin arithmetic)
`boxplot`	`BoxplotStat`	quartiles per group (`percentile_cont`) → rule+bar+tick
`smooth`	`SmoothStat`	OLS trend line (`regr_slope`/`regr_intercept`)
`density`	`DensityStat`	kernel density estimate (KDE grid cross-join)
`violin`	`ViolinStat`	KDE mirrored per group
`heatmap`	`HeatmapStat`	2-D binned counts
(any geom)	`AggregateStat`	opt-in via `SETTING aggregate => 'count'

Engine.executeVegaLite runs the base query once (only when an ordinary layer needs the raw rows), runs each statistical layer's SQL, and passes both to the emitter. Because these queries compute in the DB, exact-value regression across parallel engines can be float-sensitive.