MariaDB DuckDB — native columnar analytics inside MariaDB

The DuckDB Storage Engine brings DuckDB‘s columnar, vectorized analytical engine inside MariaDB Server as a pluggable storage engine (ha_duckdb). It is available in the MariaDB 11.4, 11.8, 12.3 and 13.x branches. You create a table with ENGINE=DuckDB and analytical queries against it run through DuckDB’s columnar execution — no ETL pipelines, no separate analytical cluster, no extra protocols. One server, one SQL interface, the familiar mariadb client.

  • Native ENGINE=DuckDB tables storing data in DuckDB’s columnar format, alongside your ordinary InnoDB tables in the same database.
  • Whole-query pushdown: SELECT (with JOIN, GROUP BY, ORDER BY, window functions, subqueries) is executed by DuckDB’s vectorized, multi-core engine.
  • Cross-engine joins: a single SELECT can combine ENGINE=DuckDB and ENGINE=InnoDB tables with no data copying or ETL.
  • Full MVCC transactions on the DuckDB side, driven through MariaDB’s normal BEGIN/COMMIT/ROLLBACK.

Persistent DuckDB, with a database around it

The engine is DuckDB running as a persistent, in-process instance inside mariadbd — not a CLI you launch per query. That distinction is the whole point:

  • Persistent, always-warm. One shared duckdb::DuckDB instance lives for the lifetime of the server, keeping its buffer pool, catalog, and thread pool hot across queries. You get DuckDB’s columnar/vectorized speed without paying process startup, library load, database-open, and cold-buffer costs on every query.
  • Authentication and authorization. Access to DuckDB-backed objects goes through MariaDB’s user accounts, roles, and GRANT/REVOKE privileges — the same auth and object-level authorization as the rest of your schema. Plain DuckDB has no user/privilege model; here every ENGINE=DuckDB table sits behind MariaDB’s access control, wire protocol, and TLS.
  • Sometimes faster than DuckDB itself. Because the engine keeps a live, warm buffer pool while the standalone DuckDB CLI cold-starts per invocation, the engine can beat native DuckDB on realistic, repeated-query workloads.

Quick start

Create a table with ENGINE=DuckDB, load data, and query it exactly as you would any MariaDB table.

-- 1. A columnar analytical table (no extension needed; the engine is built into the server)
CREATE TABLE sales (
    id         BIGINT PRIMARY KEY,
    region     VARCHAR(64),
    product    VARCHAR(128),
    amount     DECIMAL(12,2),
    sold_at    TIMESTAMP
) ENGINE=DuckDB;

-- 2. Load data with ordinary INSERT, INSERT ... SELECT, or LOAD DATA
INSERT INTO sales VALUES (1, 'EU', 'widget', 19.99, NOW());

-- 3. Run analytics — the whole query is pushed down to DuckDB's vectorized engine
SELECT region,
       product,
       SUM(amount)            AS revenue,
       COUNT(*)               AS orders
FROM sales
WHERE sold_at >= '2025-01-01'
GROUP BY region, product
ORDER BY revenue DESC
LIMIT 10;

When a query touches at least one ENGINE=DuckDB table, MariaDB’s optimizer hands the entire SELECT to the engine’s select_handler, which forwards it to DuckDB. MariaDB-specific syntax (e.g. GROUP BY … WITH ROLLUP, CONVERT(...), LIMIT, OFFSET, COUNT, RLIKE) is rewritten to DuckDB’s dialect on the way down.

What you get

  • A pluggable ENGINE=DuckDB that stores data in DuckDB’s native columnar format; InnoDB and DuckDB tables coexist in one database and one SQL interface.
  • Whole-SELECT pushdown through MariaDB’s select_handler — DuckDB reorders joins, builds hash tables, and runs aggregation/sorting with its vectorized, multi-core engine.
  • Cross-engine joins between DuckDB and non-DuckDB tables (InnoDB, Aria, MyISAM, RocksDB), served on demand through MariaDB’s own optimizer and access paths.
  • Two cross-engine read modes via duckdb_cross_engine_ryow: the default fiber path (committed-data reads, with MariaDB index/range access and Index Condition Pushdown), or Read-Your-Own-Writes (parent transaction’s uncommitted writes visible).
  • MySQL/MariaDB function compatibility handled entirely at runtime (no DuckDB source patches) — hex, oct, bin, locate, mid, regexp_replace, MariaDB week/yearweek/to_days/dayofweek semantics, byte-oriented length, and more.
  • Embedded DuckDB (currently v1.5.5), built from source into the server tree; the engine is loaded via ha_duckdb.so / duckdb.cnf.

Full SQL/behavior reference: the engine’s docs under storage/duckdb/docs/.

How fast is it?

Two things matter for a long-lived analytical server: how fast queries run once the system is warm, and what happens when many connections query at once. The numbers below are from public benchmarks; hardware and versions are listed under each chart.

Warm analytics: at native DuckDB speed

At scale: TPC-H SF1000

TPC-H SF1000 per-query times — engine total 480.8 s vs DuckDB CLI 637.9 s, 17 of 22 queries won

On TPC-H SF1000 — about 8.7 billion rows, 400 GB of Parquet — the engine completes all 22 queries in 480.8 s against 637.9 s for the DuckDB CLI: 1.33x faster overall, winning 17 of 22, with the largest single-query gain at 2.72x (Q15). The comparison is one run per query, so the CLI pays process startup and a cold buffer pool on every invocation; that overhead is exactly what the persistent in-process engine removes. Q21 is a known regression where the CLI wins, and peak memory is similar for both (112 vs 115 GB).

Hetzner, AMD EPYC 9454 (48 cores), 125.5 GB RAM, Micron 7500 PRO NVMe, both limited to 90 GiB, DuckDB v1.5.2.

ClickBench hot-run totals — MariaDB DuckDB engine 24.3 s, DuckDB CLI 26.4 s, ClickHouse 32.5 s

On ClickBench (43 queries, summed), the warm engine completes the suite in 24.3 s, ahead of the DuckDB CLI at 26.4 s and ClickHouse at 32.5 s. The engine keeps DuckDB’s buffer pool and catalog warm inside mariadbd, so repeated analytical queries run at native DuckDB speed and, on this workload, slightly ahead of it. First-touch queries pay a warm-up cost — in the cold run the CLI leads at 117.6 s with the engine at 163.8 s — a cost a long-lived server pays once at startup.

ClickBench, AWS c6a.4xlarge (16 vCPU, 32 GB), DuckDB v1.5.3, ClickHouse 26.6.1.1080. Hot = best of two warm runs per query.

Concurrency: one shared instance, many connections

ClickBench concurrency, 10 threads, AWS c6a.metal — engine 13.985 QPS, ClickHouse 7.883, DuckDB CLI 4.380

With ten concurrent query streams on bare metal, the engine sustains 13.985 QPS against 7.883 for ClickHouse and 4.380 for the DuckDB CLI — roughly 1.8x ClickHouse and 3.2x the CLI, with all three systems completing every query. On a smaller 16-vCPU machine the gap is starker: the engine served every query with a 0.0% error ratio while the per-process CLI setup failed 99.7% of them under the same load. Many MariaDB connections share one always-warm DuckDB instance, so concurrent analytics scales with the hardware.

ClickBench concurrency, 10 threads. AWS c6a.metal (DuckDB CLI v1.5.4) and AWS c6a.4xlarge (16 vCPU, 32 GB, DuckDB CLI v1.5.3).

Full methodology and per-query numbers are in the posts under Content and blogs.

MariaDB DuckDB vs. a separate analytical database

Storing analytical tables inside the same server as your transactional data offers things a bolt-on analytical system cannot:

  • One system. No separate analytical database or data warehouse to deploy, secure, scale, and keep in sync with your primary data.
  • No ETL. No export/import pipelines or nightly batch jobs — the columnar engine runs in-process, on live data.
  • HTAP in one place. InnoDB handles OLTP; DuckDB handles analytics; a single query can join across both.
  • One SQL surface. Same server, same protocol, same mariadb client and drivers — no second query dialect or connection to manage.
  • Access control included. DuckDB objects are governed by MariaDB users, roles, and privileges — unlike a standalone DuckDB file, which has no authentication or authorization model.

How it works

DuckDB is an in-process analytical database. Its performance rests on three pillars: columnar storage (reads only the columns a query needs), vectorized execution (processes data in cache-friendly batches), and parallelism (uses all available cores).

The engine embeds DuckDB as a storage-engine plugin. MariaDB owns metadata (.frm), SQL parsing, optimization, and the client protocol; DuckDB owns data storage, columnar execution, and MVCC transactions. They communicate through MariaDB’s standard handler API and the select_handler pushdown interface:

  • A single duckdb::DuckDB instance (managed by DuckdbManager) opens one duckdb.db file in the MariaDB data directory.
  • Each connection gets a per-thread DuckdbThdContext holding a duckdb::Connection, transaction state, and batched appenders.
  • DDL/DML convertors translate MariaDB TABLE/Field/Alter_info structures into DuckDB SQL, mapping types and requoting identifiers (backticks → double quotes).
  • Writes go through a batched DeltaAppender that accumulates rows via DuckDB’s Appender API and flushes them at commit.

Cross-engine queries

A single SELECT can combine DuckDB tables with tables from other engines:

SELECT d.id, d.amount, i.name
FROM analytics.orders  d      -- ENGINE=DuckDB
JOIN inventory.products i     -- ENGINE=InnoDB
    ON d.product_id = i.id
WHERE d.amount > 1000;

When the planner detects a mix of engines, the select_handler pushes the entire query to DuckDB. DuckDB’s replacement-scan callback redirects references to non-DuckDB tables to the _mdb_scan table function, which lazily spawns a cooperative fiber on a dedicated background THD. That fiber runs a synthetic, projection- and predicate-pushed SELECT ... FROM <table> [WHERE ...] through the full MariaDB pipeline — so the external table is read with MariaDB’s optimizer and access paths (index range/ref, Index Condition Pushdown) — and streams rows back to DuckDB as DataChunks. DuckDB does the join, aggregation, and sorting. No data copying or ETL is required.

Working examples

https://github.com/MariaDB/server/blob/11.8/storage/duckdb/docs/tutorials/owid-co2-emissions.md

https://github.com/MariaDB/server/blob/11.8/storage/duckdb/docs/tutorials/nyc-taxi-trips.md

NYC taxi trips — Parquet load + cross-engine join

The NYC Taxi Trips tutorial puts ~3M trips in an ENGINE=DuckDB table, the taxi-zone lookup in an ENGINE=InnoDB table, and joins them in one SELECT. DuckDB reads the Parquet file in-process, so bulk load is a single statement through run_in_duckdb():

CREATE TABLE trips (
  trip_id          BIGINT NOT NULL,
  pickup_datetime  DATETIME,
  pu_location_id   INT,
  do_location_id   INT,
  tip_amount       DECIMAL(10,2),
  total_amount     DECIMAL(10,2),
  PRIMARY KEY (trip_id)
) ENGINE=DuckDB DEFAULT CHARSET=utf8mb4;
-- DuckDB reads the Parquet file directly, in-process

SELECT run_in_duckdb('INSERT INTO taxi.trips
  SELECT ROW_NUMBER() OVER () AS trip_id, tpep_pickup_datetime,
         "PULocationID", "DOLocationID", tip_amount, total_amount
  FROM read_parquet(''/tmp/yellow_tripdata_2024-01.parquet'')');

The payoff is a plain MariaDB SELECT that joins the DuckDB trips table with the InnoDB zone lookup — no run_in_duckdb(), no data copying, no ETL:

SELECT z.borough,
       COUNT(*)                       AS airport_trips,
       ROUND(AVG(t.total_amount), 2)  AS avg_total
FROM trips t                                  -- ENGINE=DuckDB
JOIN taxi_zones z                             -- ENGINE=InnoDB
  ON t.pu_location_id = z.location_id
WHERE t.do_location_id IN (132, 138)          -- JFK, LaGuardia
GROUP BY z.borough
ORDER BY airport_trips DESC;

OWID CO₂ emissions — CSV load + window functions

The OWID CO₂ Emissions tutorial loads the Our World in Data CO₂ dataset from CSV via DuckDB’s parallel reader, then computes year-over-year change with a LAG() window function pushed down to DuckDB:

SELECT year,
       ROUND(co2, 1) AS co2_mt,
       ROUND((co2 - LAG(co2) OVER (ORDER BY year)) /
             LAG(co2) OVER (ORDER BY year) * 100, 2) AS yoy_growth_pct
FROM co2_emissions
WHERE country = 'China' AND co2 IS NOT NULL
ORDER BY year DESC
LIMIT 5;

Use cases

  • HTAP (Hybrid Transactional/Analytical Processing). InnoDB serves OLTP; DuckDB serves analytics; both live in one database.
  • Ad-hoc analytics. Complex joins, aggregations, subqueries, and window functions over large datasets, without exporting to a separate system.
  • Eliminating ETL complexity. No dedicated analytical cluster or data-movement pipeline — the analytical engine runs in-process.
  • Open-dataset exploration. Load Parquet/CSV and analyze it directly (see the NYC Taxi Trips and OWID CO₂ tutorials under docs/tutorials/).

Roadmap

  • Analytical GIS — geospatial analytics via DuckDB’s Spatial extension.
  • Faster HTAP over InnoDB-only data — run queries against InnoDB-only data through DuckDB’s vectorized execution.
  • Partial query pushdown — a derived handler to push parts of complex queries into DuckDB.
  • Data-lake access — reach data-lake protocols and formats via DuckDB extensions.
  • Parallel cross-engine scan — today each external table is produced by a single fiber-driven query; only the DuckDB side is parallelized. Native support for quack and adbc protocols – MariaDB roles are used to authenticate and describe grants accessing DuckDB objects

Background: what is columnar / vectorized analytics?

Transactional (OLTP) engines like InnoDB store data row by row, which is ideal for reading and updating individual records. Analytical (OLAP) queries instead scan huge numbers of rows but only a few columns, and aggregate them. Columnar storage keeps each column together so a query reads only the columns it needs; vectorized execution processes those columns in batches that fit CPU caches; and the work is spread across all cores. DuckDB combines these techniques in an in-process engine, and this storage engine makes them available inside MariaDB — right next to your transactional data.

Detailed compatibility matrix: docs/mariadb-duckdb-incompatibilities.md.

Content and blogs

  • DuckDB Storage Engine for MariaDB. When the Sea Lion Learns to Quack. — Roman Nozdrin (2026-06). The announcement and design overview: why an in-process columnar engine next to InnoDB, how it compares to MariaDB ColumnStore, cross-engine joins, TPC-H SF10 benchmarks (all 22 queries warm in ~4.3 s, 86.6M rows bulk-loaded in ~33 s via in-engine COPY), and why it ships as a plugin.
  • MariaDB + DuckDB: A New Playground for Analytics – A First Look at the New Storage Engine — Frédéric Descamps (2026-06). A hands-on first look: enabling the plugin with dbdeployer, the new duckdb_* variables and status counters, creating ENGINE=DuckDB tables (and the utf8mb4 charset requirement), mixing InnoDB and DuckDB in one schema, the AirportDB demo, Parquet, and current limitations.
  • One Database, Two Engines: what if InnoDB+DuckDB can be faster than DuckDB alone — Roman Nozdrin (2026-07). A close look at cross-engine performance on TPC-H SF1000: keeping only the two largest fact tables (lineitem, orders) in DuckDB and the rest in InnoDB, versus all tables in DuckDB. Q21 ran within ~7% of the all-DuckDB configuration (and slightly faster in these runs), while the broader six-table Q9 join cost ~44%. The takeaway: a single-copy HTAP model — each table owned by one engine, no analytical replica, no ETL — is practical for many query shapes, and join shape matters as much as the engine boundary.