Continue reading “From a Chocolate Wrapper to Concurrent InnoDB Page Splits”

From a Chocolate Wrapper to Concurrent InnoDB Page Splits appeared first on MariaDB.org

From a Chocolate Wrapper to Concurrent InnoDB Page Splits appeared first on MariaDB.org

We will show, on a real PostgreSQL 16 and a real ClickHouse 26 build, why the same table and the same question cost 1.79 GB of I/O in one engine and 22 MiB in the other, why sixty-four writers queue behind a single row no matter how many cores you buy, why eventually consistent platforms exist at all, and why a vector index is a different kind of object from anything in the relational world. Then we lay out the polyglot persistence decision frame a CTO or an investor should apply before believing any “one database for everything” story, including ours.

What polyglot persistence actually claims

Martin Fowler gave the pattern its name in 2011: polyglot persistence means using different data storage technologies for different data, chosen by how the application reads and writes that data. The idea of polyglot persistence predates the term. Every large internet company arrived at polyglot persistence independently, usually after an outage, and usually after trying very hard not to.

The version of polyglot persistence that matters to a board is narrower. A business that serves consumers on the internet runs four or five workload classes with incompatible physics: transactions that must be exactly right, interactive traffic that must be fast everywhere, analytics that must scan everything, retrieval that must be semantically close, and the elastic cloud substrate underneath all of it. No single engine is optimal for more than one or two of those. Each additional engine is a real cost, so the decision is about which contradictions you can afford to paper over and which you cannot.

Figure 1. The polyglot persistence landscape: five workload classes, five storage contracts.

The OLTP engine is keeping a promise, not hitting a wall

A transaction-processing database sells one thing: ACID. Atomicity, consistency, isolation and durability are not marketing features; they are mechanisms with costs that show up in specific catalog views. Durability is a synchronous write to the write-ahead log before a commit is acknowledged. Isolation is a row-level lock, or in MVCC engines a tuple version chain plus a lock on the row being updated. Atomicity is the ability to roll every one of those back. In PostgreSQL’s MVCC implementation, two sessions that update the same row serialise on that row by design, because that is the only way to keep the second update from clobbering the first.

The cleanest way to see the cost is to measure it. The runs below are on a two-vCPU sandbox with PostgreSQL 16.13, synchronous_commit = off to remove disk latency from the picture, and pgbench driving a single-statement transaction for ten seconds. This is a demonstration of a mechanism, not a benchmark; the absolute numbers are meaningless outside this box, and the shape of the curve is the point.

CREATE TABLE inventory (
    sku_id   INT PRIMARY KEY,
    on_hand  INT NOT NULL
);
INSERT INTO inventory
SELECT g, 1000000000
FROM generate_series(1, 1000) g;

-- hot.sql: every client decrements the same SKU
UPDATE inventory SET on_hand = on_hand - 1 WHERE sku_id = 1;

-- spread.sql: clients decrement a random SKU
set sku random(1, 1000)
UPDATE inventory SET on_hand = on_hand - 1 WHERE sku_id = :sku;

$ for c in 1 8 32 64; do pgbench -n -f hot.sql -c $c -j 2 -T 10; done
clients=1    latency average = 0.065 ms   tps = 15298
clients=8    latency average = 0.260 ms   tps = 30790
clients=32   latency average = 2.020 ms   tps = 15838
clients=64   latency average = 5.871 ms   tps = 10900

$ for c in 1 8 32 64; do pgbench -n -f spread.sql -c $c -j 2 -T 10; done
clients=1    latency average = 0.066 ms   tps = 15238
clients=8    latency average = 0.190 ms   tps = 42089
clients=32   latency average = 0.932 ms   tps = 34339
clients=64   latency average = 2.080 ms   tps = 30774

Both workloads saturate two cores by eight clients, so the interesting comparison is what happens after that. The spread workload holds three quarters of its peak at sixty-four clients. The hot-row workload loses two thirds of its peak and its latency grows ninety-fold. A snapshot of pg_stat_activity during the sixty-four-client hot-row run shows where the time went.

SELECT wait_event_type,
       wait_event,
       COUNT(*)
FROM pg_stat_activity
WHERE backend_type = 'client backend'
  AND query LIKE 'UPDATE inventory%'
GROUP BY 1, 2
ORDER BY 3 DESC;

 wait_event_type |  wait_event   | count
-----------------+---------------+-------
 Lock            | tuple         |    38
 LWLock          | BufferContent |     7
 LWLock          | LockManager   |     6
 Lock            | transactionid |     6
                 |               |     5
 Client          | ClientRead    |     2

Figure 2. Wait events under hot-row contention, the first measurement behind polyglot persistence.

Fifty of sixty-four sessions are queued on the tuple lock or the transaction ID of the session holding it. Nothing here is a bug, a missing index or a tuning gap. The engine is serialising updates to one row because you asked it to guarantee that no decrement is lost. Sharding does not change the arithmetic for a single hot key; it only spreads the keys that are not hot. This is the first fundamental limit that forces polyglot persistence: an ACID row store’s throughput on a contended key is bounded by the serial critical section, and that is exactly what a consumer flash sale, a viral post’s like counter or a global leaderboard produces.

What the consumer internet changed

Enterprise applications of the previous era had a bounded number of users, a business-hours load curve and a tolerance for a few hundred milliseconds. Consumer applications have none of those properties. The traffic is spiky and global, the read-to-write ratio is often a thousand to one, and the product team measures latency at the 99th percentile in a region the database was never deployed in.

Amazon documented the consequences in the Dynamo paper. When a network partition happens, and it will, a system can keep accepting writes or it can keep every replica in agreement, but not both. A shopping cart that refuses to accept an item because a replica is unreachable costs more than a cart that briefly shows a stale item. Cassandra, DynamoDB, Riak and the key-value tier of most large platforms descend from that decision, and polyglot persistence at internet scale starts with it. The write contract becomes “this will converge”, enforced by quorum arithmetic and a conflict rule, rather than “this is true now”, enforced by a lock.

The trade is explicit and tunable. In Cassandra, a read at LOCAL_QUORUM against a replication factor of three touches two of three replicas in the local datacenter and returns the newest timestamped value it sees; the consistency documentation spells out the read-plus-write-greater-than-replication-factor rule that makes that read see the latest acknowledged write. What you give up is any notion of a multi-row transaction, a join or an ad hoc query, which is why the data model is designed backwards from the queries.

CREATE KEYSPACE consumer
WITH replication = {
    'class'      : 'NetworkTopologyStrategy',
    'us-east-1'  : 3,
    'ap-south-1' : 3
};

-- One partition per user, newest activity first, bounded by TTL.
CREATE TABLE consumer.activity_feed_by_user (
    user_id      UUID,
    event_ts     TIMEUUID,
    event_type   TEXT,
    payload      TEXT,
    PRIMARY KEY ((user_id), event_ts)
)
WITH CLUSTERING ORDER BY (event_ts DESC)
 AND default_time_to_live = 2592000
 AND compaction = {
    'class'                : 'TimeWindowCompactionStrategy',
    'compaction_window_unit': 'DAYS',
    'compaction_window_size': 1
 };

-- The application reads at LOCAL_QUORUM and writes at LOCAL_QUORUM:
-- 2 + 2 > 3, so a read observes the latest acknowledged write in-region.
CONSISTENCY LOCAL_QUORUM;
SELECT event_ts, event_type, payload
FROM consumer.activity_feed_by_user
WHERE user_id = 7a3f1c2e-4d5b-4e6f-8a9b-0c1d2e3f4a5b
LIMIT 50;

Notice what is missing. There is no foreign key to a users table, no join to the events catalog, no way to ask “which users had the most events this week” without a full cluster scan or a second table maintained by the application. That is the mirror image of the OLTP limit. An eventually consistent platform buys availability and horizontal write scale by refusing every feature that would require global coordination. Polyglot persistence is what happens when you stop pretending one of these contracts can substitute for the other.

Analytics is columnar because the questions are columnar

An analytical question is a function of a few columns over a very large number of rows. A row store answers it by reading every byte of every row, because the tuple is the unit of storage. A column store keeps each column in its own file, so a scan opens only the columns the query names, and compresses each column with a codec suited to that column’s distribution. This is the second fundamental limit behind polyglot persistence, and it is easy to measure.

The lab table is a 5-million-row, 24-column orders table with realistic width: two free-text addresses, a UUID session ID, a referrer URL, optional notes. It was generated in PostgreSQL 16.13, exported to CSV, and loaded unchanged into ClickHouse 26.7 through chdb, so both engines hold identical data. The question is the one every product dashboard asks first: revenue and order count by region and channel for the last ninety days.

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, TIMING OFF)
SELECT region,
       channel,
       COUNT(*)    AS orders,
       SUM(amount) AS revenue
FROM orders
WHERE order_ts >= now() - interval '90 days'
GROUP BY region, channel
ORDER BY revenue DESC;

 Sort (actual rows=20 loops=1)
   Sort Key: (sum(amount)) DESC
   Buffers: shared hit=123593 read=105740
   I/O Timings: shared read=219.788
   ->  Finalize GroupAggregate (actual rows=20 loops=1)
         ->  Gather Merge (actual rows=60 loops=1)
               Workers Planned: 2
               Workers Launched: 2
               ...
               ->  Partial HashAggregate (actual rows=20 loops=3)
                     ->  Parallel Seq Scan on orders (actual rows=408674 loops=3)
                           Filter: (order_ts >= (now() - '90 days'::interval))
                           Rows Removed by Filter: 1257993
                           Buffers: shared hit=123574 read=105740
 Execution Time: 674.381 ms

-- pg_class.relpages for orders: 229314  (229314 x 8 KB = 1.79 GB heap)

The planner did nothing wrong. There is an index on order_ts, but a quarter of the table matches, so a parallel sequential scan is the right plan, and it is the right plan in every row store. The cost is structural: to sum one NUMERIC column across 1.2 million qualifying rows, the executor pulled all 229,314 heap pages through shared buffers, 105,740 of them from the operating system, because ship_address, session_id and notes live in the same 8 KB pages as amount.

CREATE TABLE lab.orders
(
    order_id        UInt64,
    customer_id     UInt64,
    order_ts        DateTime64(6, 'UTC'),
    region          LowCardinality(String),
    channel         LowCardinality(String),
    status          LowCardinality(String),
    currency        FixedString(3),
    amount          Decimal(12, 2),
    -- ... 16 further columns identical to the PostgreSQL table
    carrier         LowCardinality(String),
    updated_at      DateTime64(6, 'UTC')
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(order_ts)
ORDER BY (region, order_ts, order_id)
SETTINGS index_granularity = 8192;

EXPLAIN indexes = 1
SELECT region,
       channel,
       count()     AS orders,
       sum(amount) AS revenue
FROM lab.orders
WHERE order_ts >= now() - INTERVAL 90 DAY
GROUP BY region, channel
ORDER BY revenue DESC;

Aggregating
   Keys: region, channel
   Aggregates: count(), sum(amount)
   ReadFromMergeTree (lab.orders)
      Parts: 4 | Granules: 166
      Output: region, channel, amount
      Prewhere filter column: order_ts >= '2026-06-07 04:05:17'
      Indexes:
         Partition   Parts: 4/4    Granules: 173/173
         Min-Max     Parts: 4/13   Granules: 173/652
         PrimaryKey  Parts: 4/4    Granules: 166/173

-- 20 rows returned in 0.032 s on the same 2-vCPU sandbox

SELECT column,
       formatReadableSize(sum(column_data_compressed_bytes))   AS compressed,
       formatReadableSize(sum(column_data_uncompressed_bytes)) AS uncompressed
FROM system.parts_columns
WHERE database = 'lab' AND table = 'orders' AND active
  AND column IN ('region','channel','amount','order_ts','ship_address','session_id')
GROUP BY column
ORDER BY sum(column_data_compressed_bytes) DESC;

   ┌─column───────┬─compressed─┬─uncompressed─┐
1. │ ship_address │ 185.17 MiB │ 290.35 MiB   │
2. │ session_id   │ 76.61 MiB  │ 76.29 MiB    │
3. │ amount       │ 19.25 MiB  │ 38.15 MiB    │
4. │ channel      │ 2.73 MiB   │ 4.78 MiB     │
5. │ order_ts     │ 190.29 KiB │ 38.15 MiB    │
6. │ region       │ 26.85 KiB  │ 4.78 MiB     │
   └──────────────┴────────────┴──────────────┘

Figure 3. Row store versus column store for one aggregate, the second measurement behind polyglot persistence.

The four columns the query needs compress to about 22 MiB for the entire table, and the partition, min-max and primary-key indexes cut that to 166 of 652 granules before a byte of amount is decoded. The twenty columns the query does not need, including the 185 MiB of addresses, are never opened. Thirty milliseconds versus seven hundred is not ClickHouse being clever; it is the MergeTree storage layout being shaped like the question. The same layout is why ClickHouse is a poor system of record: a single-row UPDATE is a mutation that rewrites parts, and there is no row lock to serialise two of them.

Cloud-native data platforms changed the unit of scale

The third shift in polyglot persistence is not a data model but an operating model. Aurora, AlloyDB, Spanner, Snowflake, BigQuery and ClickHouse Cloud separate compute from storage, put the storage on a replicated log or an object store, and let capacity change in minutes. For a founder the argument is time to market: a team of four can stand up a multi-AZ PostgreSQL-compatible cluster with automated failover before lunch. For a CFO the argument is that capacity becomes an operating expense that tracks demand.

The trade-offs are just as concrete, and a vendor-neutral practice has to name them. Managed services diverge from the open-source engine they are compatible with: extension allow-lists, superuser removal, version lag behind community releases, and storage layers whose performance characteristics (Aurora’s quorum writes, Spanner’s TrueTime commit wait) differ from the engine’s documentation.

Egress and cross-region replication are priced per byte, which matters precisely when the polyglot persistence topology is moving change streams between stores. And the exit cost is asymmetric: getting a terabyte in is a weekend, getting it out with zero downtime is a project. None of this argues against cloud platforms. It argues for choosing them per workload, which is polyglot persistence applied to the operating model, with the same rigour as the engine itself.

Vector platforms and RAG are a different kind of object

A relational index answers “is this key present” exactly. A B-tree, a hash, a bitmap: the answer is deterministic and complete. A vector index answers “which stored vectors are nearest to this one” approximately, because exact nearest-neighbour search in a thousand dimensions is a full scan. HNSW and IVF indexes trade recall for latency with explicit knobs, and a retrieval-augmented generation pipeline lives or dies on that trade plus the metadata filters applied before or after the approximate search.

That makes vector search a distinct workload class in any polyglot persistence design rather than a feature bolted onto an existing engine, even when it ships inside one. pgvector 0.8 on PostgreSQL 16+ is an excellent choice while the corpus fits in memory, the filter predicates are selective and the embedding refresh rate is modest, because it keeps the vectors transactionally next to the rows they describe. A dedicated platform such as Milvus 2.6 earns its place when index build and query serving need to scale independently, when collections reach hundreds of millions of vectors, or when the workload needs GPU indexes, tiered storage and multi-tenant isolation that a general-purpose engine will not prioritise.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE support_chunks (
    chunk_id    BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    tenant_id   INT          NOT NULL,
    ticket_id   BIGINT       NOT NULL,
    chunk_text  TEXT         NOT NULL,
    embedding   VECTOR(1024) NOT NULL,
    updated_at  TIMESTAMPTZ  NOT NULL DEFAULT now()
);

-- Recall/latency trade-off is explicit: m and ef_construction at build time,
-- hnsw.ef_search at query time. Test on your corpus before fixing these.
CREATE INDEX idx_support_chunks_embedding_hnsw
    ON support_chunks
 USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 128);

SET hnsw.ef_search = 80;
SELECT chunk_id,
       ticket_id,
       1 - (embedding  $1::vector) AS cosine_similarity
FROM support_chunks
WHERE tenant_id = $2
ORDER BY embedding  $1::vector
LIMIT 8;

import psycopg
from openai import OpenAI   # any embedding provider; keep it in-VPC for regulated data

EMBED_MODEL = "text-embedding-3-large"
client = OpenAI()

def embed(text: str) -> list[float]:
    return client.embeddings.create(model=EMBED_MODEL, input=text).data[0].embedding

def retrieve(conn: psycopg.Connection, tenant_id: int, question: str, k: int = 8):
    qvec = embed(question)
    with conn.cursor() as cur:
        cur.execute("SET LOCAL hnsw.ef_search = 80")
        cur.execute(
            """
            SELECT chunk_id, ticket_id, chunk_text,
                   1 - (embedding  %s::vector) AS score
            FROM support_chunks
            WHERE tenant_id = %s
            ORDER BY embedding  %s::vector
            LIMIT %s
            """,
            (qvec, tenant_id, qvec, k),
        )
        return cur.fetchall()

# The ticket text itself is owned by the OLTP schema; this table is rebuilt
# from it when the embedding model changes. Treat it like a materialised view.

Two operational facts follow for any polyglot persistence estate. First, an embedding model change invalidates every vector, so the vector store must be rebuildable from the system of record; it is a derived index, not a database of record. Second, recall is a measured quantity. A RAG pipeline that has never had its retrieval recall measured against a labelled set is not in production, whatever the dashboard says.

The polyglot persistence decision frame

Executives do not need to memorise storage internals. They need a frame that turns “which database” into a question about the workload, so that the answer can be checked against measurement. These are the dimensions that decide it in practice.

Read the rows, not the columns. If a workload needs two cells that live in different columns, that is a second engine, and the honest polyglot persistence question is how the data gets from the first to the second and how stale it is allowed to be on arrival.

What a polyglot persistence topology looks like when it works

Figure 4. Reference polyglot persistence topology with one system of record and derived stores.

The polyglot persistence pattern that survives contact with production has one rule: exactly one store owns each fact, and every other store holds a derived, purpose-shaped copy that can be rebuilt from the owner. The system of record stays an ACID engine, usually PostgreSQL or MySQL with proper HA, because ledgers, orders and inventory are the facts a regulator will ask about. Change data capture through logical decoding and Kafka fans those facts out. The analytics store receives them into a ReplacingMergeTree keyed by the primary key so that CDC updates collapse on merge. The vector store receives the subset that needs embedding. The lakehouse receives everything for retention and model training.

CREATE TABLE analytics.orders_cdc
(
    order_id     UInt64,
    customer_id  UInt64,
    order_ts     DateTime64(6, 'UTC'),
    region       LowCardinality(String),
    channel      LowCardinality(String),
    status       LowCardinality(String),
    amount       Decimal(12, 2),
    _version     UInt64,          -- Debezium source.lsn or ts_ms
    _deleted     UInt8            -- 1 when op = 'd'
)
ENGINE = ReplicatedReplacingMergeTree(
    '/clickhouse/tables/{shard}/analytics/orders_cdc',
    '{replica}',
    _version,
    _deleted
)
PARTITION BY toYYYYMM(order_ts)
ORDER BY (region, order_ts, order_id)
SETTINGS index_granularity = 8192;

-- Dashboards read with FINAL or through a materialised view that
-- pre-aggregates; the raw table is never the source of truth.
SELECT region,
       channel,
       sum(amount) AS revenue
FROM analytics.orders_cdc FINAL
WHERE order_ts >= now() - INTERVAL 90 DAY
  AND _deleted = 0
GROUP BY region, channel
ORDER BY revenue DESC;

The cost of this architecture is real and should be stated to a board plainly. Every arrow is a replication lag to monitor, a schema contract to version, a backfill procedure to rehearse and an on-call surface to staff. Polyglot persistence multiplies the number of things that can be wrong at three in the morning, which is why the number of stores must be the smallest that the workload contradictions force, and never one more.

Questions a CIO, a founder or an investor should ask about polyglot persistence

The technical argument for polyglot persistence above reduces to a handful of questions that a non-specialist can put to any engineering team or any vendor, and that a diligence process should insist on getting answered with evidence.

Which store owns each fact, and can every other store be rebuilt from it without downtime? What is the measured replication lag between the system of record and the analytics store at peak, and who is paged when it exceeds the agreed staleness? What is the hot-key throughput ceiling of the transactional engine on its current hardware, measured the way we measured it above, and how far is peak traffic from that ceiling? What does the cloud bill look like per workload class, including egress between stores, and what is the exit plan for each managed service? What is the recall of the retrieval pipeline against a labelled set, and when was it last measured?

A team that can answer those with catalog views, system.* tables and dated measurements is running polyglot persistence deliberately. A team that answers with a vendor’s architecture slide is running it by accident, and the accident is usually discovered during a growth spike, an audit or an acquisition.

Why a polyglot persistence partner has to be vendor-neutral

Every engine in a polyglot persistence estate is sold by a company whose revenue depends on you choosing it for as much of your estate as possible. The transactional vendor will add columnar indexes and vector types and tell you the second and third engines are unnecessary. The columnar vendor will add row-level updates and tell you it can be the system of record. The cloud vendor will bundle all of them and tell you the topology question is solved. Each claim is partly true, and each is a conflict of interest, because the vendor is paid for the engine and not for the outcome.

MinervaDB is paid for the outcome. We are data platform practitioners rather than a product company: our engineers have run PostgreSQL, MySQL, SQL Server, MongoDB, Cassandra, Redis and Valkey, ClickHouse, Milvus and the managed cloud editions of all of them in production, across more than 900 enterprises, and we have no licence to sell.

That independence is what lets us tell a client that pgvector is enough for their corpus, that their Cassandra cluster should be a PostgreSQL partition, or that the columnar migration they were sold will not fix a hot-row problem. The measurements in this post are the kind of evidence we bring to every architecture review, and the polyglot persistence decision frame above is the one we apply.

If you are designing, funding or acquiring a business that runs on a consumer-facing data platform, the polyglot persistence conversation is the one to have before the growth spike rather than after it. Talk to MinervaDB about an independent architecture review of your data platform, from the system of record to the retrieval layer. As always: test every change on your own workload before it reaches production, and keep a rehearsed disaster-recovery posture for every store in the topology, derived copies included.

Frequently asked questions about polyglot persistence

Is polyglot persistence just over-engineering for a startup? Usually, at first. A single well-run PostgreSQL with a cache in front of it carries most products to meaningful scale. Polyglot persistence becomes necessary when a specific measurement, such as the hot-row ceiling or the analytics scan cost shown above, contradicts a specific product requirement. Add the second store when the measurement says so, not when a slide does.

Can a multi-model database replace polyglot persistence? A multi-model engine reduces the operational surface of polyglot persistence, which is valuable, but it does not change the physics. A row store with a columnar index still reads tuples for transactional work and still serialises on a hot key; a columnar engine with row updates still performs mutations. Evaluate multi-model features by measuring the specific workload against a dedicated engine and pricing the difference.

Does the cloud make the polyglot persistence decision for me? No. Cloud platforms make each engine easier to provision and scale, and they make moving data between engines more expensive. The workload contradictions are the same on-premises and in the cloud; what changes is the operating model and the cost structure of the arrows between stores.

Where does the vector database sit in a polyglot persistence design? As a derived, rebuildable index over facts owned by the system of record. Embedding model changes and recall regressions are routine, so the pipeline that rebuilds the vector store must be as tested as the backup that restores the ledger.

Lab notes for the polyglot persistence measurements: PostgreSQL 16.13 (Ubuntu build), shared_buffers 1 GB, synchronous_commit off, two vCPUs, 7 GB RAM; ClickHouse 26.7.2 embedded via chdb on the same host; 5,000,000 rows generated with generate_series and exported unchanged between engines; pgbench 10-second runs; all outputs are verbatim with trims marked. Figures are mechanism demonstrations from a sandbox and are not vendor benchmarks. Reproduce on your own hardware before drawing capacity conclusions.

Polyglot Persistence: The Proven Case for 5 Data Models appeared first on MariaDB.org

Continue reading “Launching MariaDB’s Database Survey 2026”

Launching MariaDB’s Database Survey 2026 appeared first on MariaDB.org

Launching MariaDB’s Database Survey 2026 appeared first on MariaDB.org

Continue reading “MariaDB Plugins Beyond C++: What the Community Told Us”

MariaDB Plugins Beyond C++: What the Community Told Us appeared first on MariaDB.org

MariaDB Plugins Beyond C++: What the Community Told Us appeared first on MariaDB.org

This post shows how to write that BYOC database security standard so it can be scored, how to collect evidence from the database and the cloud account rather than from a questionnaire, and how to turn the evidence into a percentage and a verdict that behave sensibly. Everything below is runnable. The evaluator output shown later is a real run against a hand-built evidence file, not a mock-up.

What BYOC actually splits, and why BYOC database security has to be scored differently

Bring Your Own Cloud (BYOC) is a specific deployment model, and the security argument hinges on its shape. The vendor’s control plane, meaning the orchestrator, the tenant console, billing and support tooling, runs in the vendor’s account. The data plane, meaning the database nodes, the block storage, the object storage holding data and backups, the network boundary and the KMS keys, runs in your account. ClickHouse describes its BYOC data plane as running entirely in your cloud account with the control plane in ClickHouse’s VPC; Redpanda’s BYOC architecture follows the same split, and so do most of the other vendors offering the model.

The bridge between the two halves is a cross-account IAM role the control-plane agent assumes, plus whatever break-glass path the vendor’s operators use when something needs hands.

That split is what makes BYOC database security scoreable at all. In a fully managed service you can only attest: read the SOC 2 report, accept the shared-responsibility matrix, move on. In BYOC, every object in the data plane is queryable through your own cloud APIs and your own database catalog. You can measure it, and if you can measure it you can score it. The control plane stays opaque, and the standard has to be honest about that boundary rather than pretending a SOC 2 PDF is the same kind of evidence as a security-group rule you read from the API.

Writing a BYOC database security standard that produces a number

Most BYOC database security policies fail the measurability test for a mundane reason: their controls are sentences, not predicates. “Access to production databases must be appropriately restricted” cannot be scored. A control becomes scoreable when it has four properties. It is atomic, testing exactly one thing. It binds to exactly one evidence key, a value some collector can produce without a human interpreting it. It carries an expectation expressed as an operator and a value. And it carries a weight, because a wildcard IAM policy on the control-plane agent and a SOC 2 report that is thirteen months old are not the same size of problem.

The BYOC database security standard below has seven domains and eighteen controls. It is deliberately short. A standard with two hundred controls, half of which nobody can collect evidence for, produces a number nobody trusts. Eighteen controls with real collectors produce a number you can put in a board pack. Add controls only when a collector exists for them.

# MDB-SEC-2026-09-BYOC  --  BYOC database security standard, v1.0
# Every control is atomic, has exactly one evidence key, and carries a weight.
# gate: true means a FAIL caps the whole target at "non-conformant" regardless of score.
standard: MDB-SEC-2026-09-BYOC
version: "1.0"
weights: {critical: 5, high: 3, medium: 1}

domains:
  - id: IAM
    name: Identity and access
    controls:
      - id: IAM-01
        title: No shared superuser / default admin account enabled for application use
        severity: critical
        gate: true
        evidence: db.superuser_logins_last_30d
        expect: {op: eq, value: 0}
      - id: IAM-02
        title: Database authentication is federated (IAM / OIDC / cert), not password-only
        severity: high
        evidence: db.password_only_roles
        expect: {op: eq, value: 0}
      - id: IAM-03
        title: Vendor cross-account role uses an ExternalId and is scoped to the data-plane account only
        severity: critical
        gate: true
        evidence: cloud.vendor_role_external_id_enforced
        expect: {op: eq, value: true}

  - id: NET
    name: Network boundary
    controls:
      - id: NET-01
        title: Database endpoints have no 0.0.0.0/0 ingress on any port
        severity: critical
        gate: true
        evidence: cloud.public_ingress_rules
        expect: {op: eq, value: 0}
      - id: NET-02
        title: Client access only through PrivateLink / Private Service Connect / peering
        severity: high
        evidence: cloud.private_connectivity_only
        expect: {op: eq, value: true}
      - id: NET-03
        title: Egress from the data plane is allow-listed (vendor control-plane CIDRs + object storage only)
        severity: high
        evidence: cloud.egress_allowlisted
        expect: {op: eq, value: true}

  - id: ENC
    name: Encryption
    controls:
      - id: ENC-01
        title: Storage volumes and object-storage buckets encrypted with a customer-managed key
        severity: critical
        gate: true
        evidence: cloud.cmk_encryption
        expect: {op: eq, value: true}
      - id: ENC-02
        title: TLS 1.2+ enforced on every client-facing listener
        severity: high
        evidence: db.tls_min_version
        expect: {op: gte, value: 1.2}
      - id: ENC-03
        title: Backups encrypted with the same or a dedicated CMK, never vendor-default keys
        severity: high
        evidence: cloud.backup_cmk
        expect: {op: eq, value: true}

  - id: CPL
    name: Control-plane access
    controls:
      - id: CPL-01
        title: Vendor operator access to the data plane is break-glass, ticketed, and time-boxed
        severity: high
        evidence: vendor.breakglass_ttl_hours
        expect: {op: lte, value: 8}
      - id: CPL-02
        title: Control-plane agent runs with least-privilege IAM (no iam:*, no kms:ScheduleKeyDeletion)
        severity: critical
        gate: true
        evidence: cloud.agent_policy_wildcards
        expect: {op: eq, value: 0}
      - id: CPL-03
        title: Vendor SOC 2 Type II report reviewed within the last 12 months
        severity: medium
        evidence: vendor.soc2_age_days
        expect: {op: lte, value: 365}

  - id: LOG
    name: Audit and logging
    controls:
      - id: LOG-01
        title: Database audit log (DDL, grants, auth failures) shipped to a SIEM the vendor cannot write to
        severity: high
        evidence: db.audit_log_shipped
        expect: {op: eq, value: true}
      - id: LOG-02
        title: Cloud API audit trail (CloudTrail / Audit Logs) enabled for the data-plane account
        severity: high
        evidence: cloud.api_audit_enabled
        expect: {op: eq, value: true}

  - id: BKP
    name: Backup and recovery
    controls:
      - id: BKP-01
        title: Backups stored in a bucket the vendor control plane cannot delete
        severity: critical
        gate: true
        evidence: cloud.backup_bucket_vendor_delete_denied
        expect: {op: eq, value: true}
      - id: BKP-02
        title: Restore drill executed and evidenced within the last 90 days
        severity: high
        evidence: ops.last_restore_drill_days
        expect: {op: lte, value: 90}

  - id: CFG
    name: Configuration hardening
    controls:
      - id: CFG-01
        title: No database-level access from the public internet (listen / pg_hba / ClickHouse listen_host)
        severity: high
        evidence: db.listens_public
        expect: {op: eq, value: false}
      - id: CFG-02
        title: Version within vendor support window and no unpatched CVE older than 30 days
        severity: medium
        evidence: db.oldest_unpatched_cve_days
        expect: {op: lte, value: 30}

Three design choices in that file matter more than the specific controls, and they are what make BYOC database security measurable rather than merely documented. First, the gate: true flag. Six controls are gates; a failure on any of them makes the target non-conformant regardless of the percentage. Without gates, weighted scoring produces the pathology where a deployment with an internet-exposed listener still reads as “81%, mostly fine”.

Second, every control’s evidence key is a dotted path into a JSON document, so the standard never references a database engine, a cloud provider or a vendor by name. The same file scores a ClickHouse BYOC cluster on AWS and a PostgreSQL BYOC deployment on GCP; only the collectors differ. Third, the file has an ID and a version in the MinervaDB document convention, MDB-SEC-2026-09-BYOC v1.0, because a score is meaningless unless the reader knows which standard it was scored against. Changing a weight changes every historical number, so weights change through a versioned pull request, never in place.

Collecting BYOC database security evidence from the database, not from a questionnaire

The evidence for the db.* keys in a BYOC database security assessment comes from the database’s own catalog. The point of writing BYOC database security collectors as SQL is that the SQL is the audit trail: anyone can rerun the query and get the same answer. Two collectors follow, one for ClickHouse and one for PostgreSQL, each producing the raw values the standard expects. Version pinning matters here; several of these system tables changed shape recently.

ClickHouse BYOC database security collector (24.8 LTS and later)

-- BYOC database security: ClickHouse collector for db.* keys.  Tested syntax on ClickHouse 24.8 LTS and later.
-- Run as a user holding SHOW USERS, SHOW GRANTS and access to system.query_log.

-- IAM-01  logins as the built-in `default` user in the last 30 days
--         (query_log is on by default; session_log is not, so we count from query_log)
SELECT count() AS superuser_logins_last_30d
FROM system.query_log
WHERE user = 'default'
  AND type = 'QueryStart'
  AND event_date >= today() - INTERVAL 30 DAY;

-- IAM-02  roles that can still log in with a password only
--         auth_type became an Array when multiple auth methods per user
--         landed (24.9+); on older builds drop the hasAny() and compare directly
SELECT count() AS password_only_roles
FROM system.users
WHERE hasAny(auth_type,
             ['plaintext_password', 'sha256_password',
              'double_sha1_password', 'bcrypt_password'])
  AND NOT hasAny(auth_type, ['ldap', 'kerberos', 'ssl_certificate', 'jwt', 'http']);

-- CFG-01  is the server listening on anything but private / loopback interfaces?
--         (system.server_settings, 23.x+)
SELECT name, value, changed
FROM system.server_settings
WHERE name IN ('listen_host', 'tcp_port', 'http_port',
               'tcp_port_secure', 'https_port');

-- ENC-02  plaintext listeners left open count as a FAIL for TLS enforcement
SELECT countIf(name IN ('tcp_port', 'http_port') AND value != '') AS plaintext_listeners
FROM system.server_settings;

-- CPL-01 / audit trail  grants issued outside the change window
SELECT event_time, user, query
FROM system.query_log
WHERE type = 'QueryFinish'
  AND query_kind IN ('Grant', 'Revoke', 'Create', 'Drop', 'Alter')
  AND event_date >= today() - INTERVAL 7 DAY
ORDER BY event_time DESC
LIMIT 50;

Two notes on these queries. system.query_log is the right place to count logins by the built-in default user because it is on by default and system.session_log usually is not; if you enable session_log you get a cleaner signal, including failed logins. And the auth_type column on system.users became an array when multiple authentication methods per user landed, so the hasAny() form is the one that survives upgrades. On BYOC deployments you typically cannot edit config.xml directly, but you can read system.server_settings, and that is enough to score the listener posture.

PostgreSQL BYOC database security collector (16 and later)

-- BYOC database security: PostgreSQL collector for db.* keys.  Tested syntax on PostgreSQL 16 and 17.
-- Run as a role holding pg_read_all_settings and pg_read_all_stats.

-- IAM-01  login-capable superusers other than the bootstrap role
SELECT count(*) AS extra_login_superusers
FROM pg_roles
WHERE rolsuper
  AND rolcanlogin
  AND rolname  'postgres';

-- IAM-02  pg_hba entries that authenticate with a password or nothing at all
--         (federated methods: cert, gss, sspi, ldap, radius, oauth in 18+)
SELECT count(*) AS password_only_rules
FROM pg_hba_file_rules
WHERE auth_method IN ('trust', 'password', 'md5', 'scram-sha-256')
  AND error IS NULL;

-- CFG-01 / ENC-02  listener and TLS posture in one pass
SELECT name, setting, unit, context
FROM pg_settings
WHERE name IN ('listen_addresses', 'ssl', 'ssl_min_protocol_version',
               'log_connections', 'password_encryption');

-- LOG-01  is pgaudit loaded and shipping DDL / role changes?
SELECT name, setting
FROM pg_settings
WHERE name IN ('shared_preload_libraries', 'pgaudit.log', 'log_destination');

pg_hba_file_rules is the collector that most often surprises people. It shows the rules PostgreSQL actually loaded, including ones with parse errors, so filter on error IS NULL or you will count rules that are not in effect. On PostgreSQL 18 the oauth method joins the federated list. A superuser that can log in and is not the bootstrap role is the PostgreSQL analogue of the ClickHouse default user problem: it is where application credentials end up when nobody is looking.

Cloud account BYOC database security collector (AWS shown, read-only)

The cloud.* keys are where BYOC database security differs from every other model, because the objects that matter live in your account. The collector below reads security groups, the vendor’s cross-account role trust policy, the agent role’s attached policies, EBS key management and CloudTrail state. It runs under a read-only audit role and writes nothing. The GCP and Azure equivalents are structurally identical: replace the IAM trust-policy check with a Workload Identity binding or a service-principal check, and the EBS/KMS check with CMEK on persistent disks or managed disks.

#!/usr/bin/env python3
"""
collect_aws.py -- BYOC database security: produce the cloud.* evidence keys for one BYOC data-plane account.

Run with a READ-ONLY role in the data-plane account:
    AWS_PROFILE=byoc-audit python3 collect_aws.py 
        --vendor-role ClickHouseBYOCControlPlane 
        --tag Key=byoc-target,Value=analytics-ch-prod > evidence/cloud.json

The script only reads. It writes nothing to AWS.
"""
import argparse
import json

import boto3

WILDCARD_DENYLIST = {"iam:*", "kms:*", "kms:ScheduleKeyDeletion", "s3:*", "ec2:*", "*"}


def public_ingress_rules(ec2, tag_key, tag_val) -> int:
    """NET-01: security-group ingress rules open to the world on data-plane ENIs."""
    sgs = ec2.describe_security_groups(
        Filters=[{"Name": f"tag:{tag_key}", "Values": [tag_val]}])["SecurityGroups"]
    hits = 0
    for sg in sgs:
        for rule in sg["IpPermissions"]:
            hits += sum(1 for r in rule.get("IpRanges", []) if r["CidrIp"] == "0.0.0.0/0")
            hits += sum(1 for r in rule.get("Ipv6Ranges", []) if r["CidrIpv6"] == "::/0")
    return hits


def vendor_role_external_id_enforced(iam, role_name) -> bool:
    """IAM-03: the trust policy must carry a StringEquals on sts:ExternalId."""
    doc = iam.get_role(RoleName=role_name)["Role"]["AssumeRolePolicyDocument"]
    for stmt in doc["Statement"]:
        cond = stmt.get("Condition", {}).get("StringEquals", {})
        if "sts:ExternalId" not in cond:
            return False
    return True


def agent_policy_wildcards(iam, role_name) -> int:
    """CPL-02: count denylisted wildcard actions across inline + attached policies."""
    docs = []
    for name in iam.list_role_policies(RoleName=role_name)["PolicyNames"]:
        docs.append(iam.get_role_policy(RoleName=role_name, PolicyName=name)["PolicyDocument"])
    for att in iam.list_attached_role_policies(RoleName=role_name)["AttachedPolicies"]:
        ver = iam.get_policy(PolicyArn=att["PolicyArn"])["Policy"]["DefaultVersionId"]
        docs.append(iam.get_policy_version(PolicyArn=att["PolicyArn"], VersionId=ver)["PolicyVersion"]["Document"])
    hits = 0
    for doc in docs:
        for stmt in doc["Statement"]:
            if stmt.get("Effect") != "Allow":
                continue
            actions = stmt["Action"] if isinstance(stmt["Action"], list) else [stmt["Action"]]
            hits += sum(1 for a in actions if a in WILDCARD_DENYLIST)
    return hits


def cmk_encryption(ec2, kms, tag_key, tag_val) -> bool:
    """ENC-01: every tagged EBS volume encrypted with a key whose KeyManager is CUSTOMER."""
    vols = ec2.describe_volumes(Filters=[{"Name": f"tag:{tag_key}", "Values": [tag_val]}])["Volumes"]
    if not vols:
        return False
    for v in vols:
        if not v["Encrypted"]:
            return False
        if kms.describe_key(KeyId=v["KmsKeyId"])["KeyMetadata"]["KeyManager"] != "CUSTOMER":
            return False
    return True


def api_audit_enabled(ct) -> bool:
    """LOG-02: at least one multi-region trail that is logging and log-file-validated."""
    for t in ct.describe_trails()["trailList"]:
        if t.get("IsMultiRegionTrail") and t.get("LogFileValidationEnabled"):
            if ct.get_trail_status(Name=t["TrailARN"])["IsLogging"]:
                return True
    return False


if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--vendor-role", required=True)
    ap.add_argument("--tag", required=True, help="Key=,Value= identifying data-plane resources")
    a = ap.parse_args()
    tag_key, tag_val = [kv.split("=", 1)[1] for kv in a.tag.split(",")]

    ec2, iam, kms, ct = (boto3.client(s) for s in ("ec2", "iam", "kms", "cloudtrail"))
    print(json.dumps({"cloud": {
        "public_ingress_rules": public_ingress_rules(ec2, tag_key, tag_val),
        "vendor_role_external_id_enforced": vendor_role_external_id_enforced(iam, a.vendor_role),
        "agent_policy_wildcards": agent_policy_wildcards(iam, a.vendor_role),
        "cmk_encryption": cmk_encryption(ec2, kms, tag_key, tag_val),
        "api_audit_enabled": api_audit_enabled(ct),
    }}, indent=2))

The IAM-03 check deserves emphasis. A vendor cross-account role whose trust policy lacks a sts:ExternalId condition is the textbook confused-deputy setup: any tenant of that vendor who learns your role ARN can ask the vendor’s control plane to act on it. Every serious BYOC vendor issues an ExternalId during onboarding. The control exists because onboarding scripts get copied between accounts, and the condition gets dropped.

The BYOC database security scoring engine

The evaluator that turns evidence into a BYOC database security score is short on purpose. Its rules are the interesting part, and each one is a decision about what the number should mean. The score is the weighted share of applicable controls that pass.

A control whose evidence key is absent is UNKNOWN and scores as a fail; unproven is treated as unsafe, which is the only rule that stops “we did not run the collector” from inflating the number. A control the evidence marks as not applicable leaves the denominator entirely. And any failing gate control caps the verdict at NON-CONFORMANT while still reporting the score, so the percentage remains useful for tracking progress even when the verdict is red.

#!/usr/bin/env python3
"""
byoc_score.py -- BYOC database security: score one target against a control standard.

    python3 byoc_score.py byoc_security_standard.yaml evidence/analytics-ch-prod.json

Rules the number is built on (keep these stable; changing them changes history):
  * score = sum(weight of PASS) / sum(weight of applicable controls)
  * a control with missing evidence is UNKNOWN and scores as FAIL -- unproven is unsafe
  * a control marked not_applicable in evidence is removed from the denominator
  * any gate control that fails caps the verdict at NON-CONFORMANT, whatever the score
"""
import json
import operator
import sys
from collections import defaultdict

import yaml

OPS = {"eq": operator.eq, "ne": operator.ne, "gte": operator.ge, "lte": operator.le}


def lookup(evidence: dict, dotted: str):
    """evidence['db']['tls_min_version'] for 'db.tls_min_version'; None if absent."""
    node = evidence
    for part in dotted.split("."):
        if not isinstance(node, dict) or part not in node:
            return None
        node = node[part]
    return node


def evaluate(standard: dict, evidence: dict) -> dict:
    weights = standard["weights"]
    results, by_domain = [], defaultdict(lambda: {"earned": 0, "possible": 0})
    earned = possible = 0
    gate_failures = []

    for domain in standard["domains"]:
        for ctl in domain["controls"]:
            w = weights[ctl["severity"]]
            value = lookup(evidence, ctl["evidence"])
            if value == "not_applicable":
                status = "N/A"
            elif value is None:
                status = "UNKNOWN"
            else:
                exp = ctl["expect"]
                status = "PASS" if OPS[exp["op"]](value, exp["value"]) else "FAIL"

            if status != "N/A":
                possible += w
                by_domain[domain["id"]]["possible"] += w
                if status == "PASS":
                    earned += w
                    by_domain[domain["id"]]["earned"] += w
                elif ctl.get("gate"):
                    gate_failures.append(ctl["id"])

            results.append({"id": ctl["id"], "domain": domain["id"], "severity": ctl["severity"],
                            "weight": w, "status": status, "observed": value,
                            "expect": ctl["expect"], "title": ctl["title"]})

    score = round(100 * earned / possible, 1) if possible else 0.0
    verdict = "NON-CONFORMANT" if gate_failures else ("CONFORMANT" if score >= 90 else "PARTIAL")
    return {"standard": f'{standard["standard"]} v{standard["version"]}',
            "target": evidence.get("target", "?"),
            "score": score, "verdict": verdict, "gate_failures": gate_failures,
            "domains": {d: round(100 * v["earned"] / v["possible"], 1) if v["possible"] else None
                        for d, v in by_domain.items()},
            "controls": results}


def print_report(rep: dict) -> None:
    print(f'{rep["target"]}  vs  {rep["standard"]}')
    print(f'score {rep["score"]}%  verdict {rep["verdict"]}'
          + (f'  (gates failed: {", ".join(rep["gate_failures"])})' if rep["gate_failures"] else ""))
    print("-" * 78)
    for c in rep["controls"]:
        flag = " GATE" if c["status"] == "FAIL" and c["id"] in rep["gate_failures"] else ""
        print(f'{c["id"]:7} {c["status"]:8} w={c["weight"]}  observed={c["observed"]!r:6}%')


if __name__ == "__main__":
    with open(sys.argv[1]) as f:
        std = yaml.safe_load(f)
    with open(sys.argv[2]) as f:
        ev = json.load(f)
    report = evaluate(std, ev)
    print_report(report)
    with open(sys.argv[2].replace(".json", ".report.json"), "w") as f:
        json.dump(report, f, indent=2)

Here is a hand-built BYOC database security evidence file for a ClickHouse BYOC cluster. The values are illustrative; the file shape is exactly what the collectors above produce once merged. Note the empty ops block: nobody recorded a restore drill, and that absence is going to be scored.

{
  "target": "analytics-ch-prod (ClickHouse BYOC, AWS eu-central-1)",
  "collected_at": "2026-09-03T10:14:00Z",
  "db": {
    "superuser_logins_last_30d": 0,
    "password_only_roles": 2,
    "tls_min_version": 1.2,
    "audit_log_shipped": true,
    "listens_public": false,
    "oldest_unpatched_cve_days": 12
  },
  "cloud": {
    "vendor_role_external_id_enforced": true,
    "public_ingress_rules": 0,
    "private_connectivity_only": true,
    "egress_allowlisted": false,
    "cmk_encryption": true,
    "backup_cmk": true,
    "agent_policy_wildcards": 1,
    "api_audit_enabled": true,
    "backup_bucket_vendor_delete_denied": true
  },
  "vendor": {
    "breakglass_ttl_hours": 4,
    "soc2_age_days": 210
  },
  "ops": {}
}

And the run, unedited:

$ python3 byoc_score.py byoc_security_standard.yaml evidence/analytics-ch-prod.json
analytics-ch-prod (ClickHouse BYOC, AWS eu-central-1)  vs  MDB-SEC-2026-09-BYOC v1.0
score 77.4%  verdict NON-CONFORMANT  (gates failed: CPL-02)
------------------------------------------------------------------------------
IAM-01  PASS     w=5  observed=0            No shared superuser / default admin account 
IAM-02  FAIL     w=3  observed=2            Database authentication is federated (IAM / 
IAM-03  PASS     w=5  observed=True         Vendor cross-account role uses an ExternalId
NET-01  PASS     w=5  observed=0            Database endpoints have no 0.0.0.0/0 ingress
NET-02  PASS     w=3  observed=True         Client access only through PrivateLink / Pri
NET-03  FAIL     w=3  observed=False        Egress from the data plane is allow-listed (
ENC-01  PASS     w=5  observed=True         Storage volumes and object-storage buckets e
ENC-02  PASS     w=3  observed=1.2          TLS 1.2+ enforced on every client-facing lis
ENC-03  PASS     w=3  observed=True         Backups encrypted with the same or a dedicat
CPL-01  PASS     w=3  observed=4            Vendor operator access to the data plane is 
CPL-02  FAIL     w=5  observed=1            Control-plane agent runs with least-privileg GATE
CPL-03  PASS     w=1  observed=210          Vendor SOC 2 Type II report reviewed within 
LOG-01  PASS     w=3  observed=True         Database audit log (DDL, grants, auth failur
LOG-02  PASS     w=3  observed=True         Cloud API audit trail (CloudTrail / Audit Lo
BKP-01  PASS     w=5  observed=True         Backups stored in a bucket the vendor contro
BKP-02  UNKNOWN  w=3  observed=None         Restore drill executed and evidenced within 
CFG-01  PASS     w=3  observed=False        No database-level access from the public int
CFG-02  PASS     w=1  observed=12           Version within vendor support window and no 
------------------------------------------------------------------------------
IAM     76.9%
NET     72.7%
ENC    100.0%
CPL     44.4%
LOG    100.0%
BKP     62.5%
CFG    100.0%

Read the output the way the CISO will. The headline is not 77.4%; it is NON-CONFORMANT with CPL-02 named as the reason, meaning the vendor’s control-plane agent holds a wildcard action in your account. That single line is the conversation to have with the vendor this week.

The 77.4% is the second sentence, and it decomposes: two password-only ClickHouse roles, egress from the data plane not allow-listed, and a restore drill nobody can evidence. The domain breakdown shows CPL at 44.4% and BKP at 62.5%, which is where the next quarter’s work goes. Encryption, logging and configuration hardening are at 100% and can be left alone.

What a BYOC database security percentage means, and what it does not

A BYOC database security conformance percentage is a statement about a standard, not about risk. Two things follow for anyone reporting BYOC database security upward. First, the number is only as good as the weights, and weights are a policy decision, so publish them and version them; a team that quietly changes critical from 5 to 3 to hit a target has broken the metric, not improved security. Second, the number can be gamed by removing controls, which is why the standard’s control count and version travel with every score. “92% against v1.0” and “92% against v1.3” are different claims, and the report.json carries both.

There is also a boundary the standard has to admit. The vendor.* keys are attestations: a SOC 2 report’s age, a break-glass TTL the vendor documented. You are recording that the vendor said something, not measuring it. That is why those controls carry lower weights and no gates. If a vendor offers an API for its break-glass audit log, and some do, promote that control from attestation to measurement and raise its weight in the next version of the standard. The direction of travel for BYOC database security is precisely that: moving controls from the hatched region of Fig. 1 into the measurable one.

Running BYOC database security scoring continuously

A BYOC database security score taken once during onboarding is a photograph. The value of a measurable standard comes from running it on a schedule and treating a drop as an incident. The pipeline in Fig. 2 runs nightly from CI and on two additional triggers: any change to the standard file, and any change in the vendor’s agent version, because agent upgrades are when IAM policies quietly grow. Store each report.json with its timestamp, plot the score per target, and alert when it decreases. A decrease means something in your account changed, or the vendor changed something in your account, and in a BYOC deployment those are the two things you most want to know about.

Open one BYOC database security ticket per FAIL and per UNKNOWN, tagged with the control ID, and close them by re-running the evaluator rather than by hand. UNKNOWN BYOC database security tickets are usually the cheapest to close and the most revealing: they are the controls nobody has ever actually checked.

Where to start with BYOC database security scoring

If you run one BYOC database today, you can have a first BYOC database security score by the end of the week. Take the standard above as v0.1, delete the controls you cannot yet collect evidence for, run the SQL and cloud collectors by hand into a single evidence file, and score it. Then add BYOC database security collectors back one at a time. The first number will be low and the first verdict will probably be NON-CONFORMANT; that is the standard doing its job. A deployment you thought was fine and a deployment you can prove is 77.4% conformant with one named gate failure are different things, and only one of them survives contact with an auditor.

MinervaDB handles BYOC database security reviews for teams running PostgreSQL, ClickHouse, MySQL, MongoDB and Kafka across all three clouds under vendor-neutral 24×7 consultative support and remote DBA engagements, and building a scoreable BYOC database security standard for a specific vendor and estate is a common first deliverable. If you want the collectors extended to your platform, or a review of a standard you have already written, book a working session.

BYOC Database Security: A Measurable Standard in 7 Domains appeared first on MariaDB.org

Continue reading “MariaDB Connector/C Compatibility: An Update on CONC-821”

MariaDB Connector/C Compatibility: An Update on CONC-821 appeared first on MariaDB.org

MariaDB Connector/C Compatibility: An Update on CONC-821 appeared first on MariaDB.org

Combined with what 9.7 LTS already carried across from Enterprise (the hypergraph optimizer, OpenTelemetry, the Group Replication components), the Community versus Enterprise line has moved further in two releases than in the previous ten years.

This post walks through the features that actually change performance, scalability and high availability outcomes, with the configuration and the measurement queries you need to verify each one on your own hardware. It also lists what is still Enterprise-only, because that list is now short enough to reason about honestly. Everything here is version-pinned against the release notes; where Oracle has published performance figures, they are labelled as Oracle’s, not ours.

MySQL 26.7, 9.7 LTS and 8.4 LTS: which one goes to production

Three supported lines shipped in the July 2026 drop, and they are not interchangeable. MySQL 9.7 was the last release to use sequential version numbers; everything after it is YY.M.P, so 26.7.0 is simply the July 2026 Innovation release. Innovation releases are supported only until the next one appears. They have no Extended Support window. MySQL 26.7 is therefore the right choice for a fleet that upgrades quarterly and wants the Change Stream Applier or the Community thread pool today, and the wrong choice for anything that needs to sit still for three years.

The ladder below is the one Oracle supports. In-place upgrades between LTS lines are adjacent-only, so an 8.0 estate cannot jump to 9.7; it goes through 8.4 first. MySQL 8.0 entered Sustaining Support on 21 April 2026, which means no new patches and no new security fixes for on-premises installations (the HeatWave extension to April 2027 does not apply outside OCI).

One practical consequence of calendar versioning that Ronald Bradford flagged in his early-access write-up: any tooling that sorts version strings lexically now believes 26.7 is older than 9.7. Check your inventory scripts, your package pinning and your monitoring dashboards before the first MySQL 26.7 host reports in. Oracle added MYSQL_PREVIOUS_LTS_VERSION and MYSQL_PREVIOUS_LTS_VERSION_ID to mysql_version.h for exactly this reason.

Thread Pool in MySQL 26.7 Community: connection scalability without the Enterprise licence

For fifteen years the Thread Pool plugin was the single most-cited reason to buy MySQL Enterprise Edition. MySQL 26.7.0 ships it in Community. Percona Server has had its own thread pool implementation since 5.5, so the capability is not new to the ecosystem, but this is the first time Oracle’s implementation, with its priority queues, stall detection and the TP_CONNECTION_ADMIN escape hatch, is available without a subscription. On MySQL 9.7 LTS it is still Enterprise-only; the Community transfer is a 26.7 change.

The mechanism is worth understanding before you turn it on, because it changes the shape of your latency curve rather than just shifting it. The default one-thread-per-connection model creates a kernel thread for every session. At two or three thousand active connections the cost is not memory, it is the OS scheduler and the InnoDB mutexes that all those runnable threads contend on. The thread pool replaces that with thread_pool_size thread groups (default 16), each owning a listener thread, a high-priority queue, a low-priority queue and a small set of worker threads. Connections are assigned to groups round-robin.

The pool tries to keep exactly one thread executing per group and only spawns another when a statement exceeds thread_pool_stall_limit (default 60 ms).

Loading it on MySQL 26.7 is a startup option, not a dynamic change, so it needs a restart and a maintenance window:

# /etc/my.cnf  (MySQL 26.7.0 Community; restart required)
[mysqld]
plugin-load-add               = thread_pool.so
thread_handling               = pool-of-threads
thread_pool_size              = 16        # start at physical cores, not vCPUs
thread_pool_stall_limit       = 60        # ms; raise for OLAP-style statements
thread_pool_max_unused_threads = 32       # 26.7 / 9.7.2 default (was 2)
thread_pool_prio_kickup_timer = 1000      # ms before a low-priority stmt is promoted
thread_pool_max_transactions_limit = 0    # 0 = no cap; set only with TP_CONNECTION_ADMIN granted

Two of those defaults changed in this release cycle. thread_pool_max_unused_threads moved from 2 to 32 in both 26.7.0 and 9.7.2, which reduces thread churn on bursty workloads at the cost of a few idle stacks. Verify the plugin loaded and confirm all four components are active (the plugin registers itself plus three Performance Schema tables):

SELECT plugin_name,
       plugin_status,
       plugin_type
  FROM information_schema.plugins
 WHERE plugin_name LIKE 'thread_pool%'
    OR plugin_name LIKE 'TP_%';

Then measure instead of guessing. The statistics table is the one to watch; the ratio of queued to executed statements per group tells you whether thread_pool_size is too low, and a rising stall count with acceptable throughput tells you the stall limit is too aggressive for your statement mix:

SELECT tp_group_id,
       connections_started,
       queries_executed,
       queries_queued,
       stalled_queries_executed,
       threads_started,
       prio_kickups
  FROM performance_schema.tp_thread_group_stats
 ORDER BY tp_group_id;

The honest framing for anyone planning to enable this on MySQL 26.7: the thread pool improves p99 under connection storms and protects InnoDB from thundering-herd contention. It does not raise peak throughput on a workload that was already CPU-bound at moderate concurrency, and on some short-transaction OLTP profiles it costs a few percent of throughput at low connection counts. Run a sysbench sweep from 64 to 4,096 connections with and without it, on your schema, before committing. We covered the plugin’s Community arrival in more detail in MySQL Thread Pool comes to Community in 26.7.

Change Stream Applier: the MySQL 26.7 replication applier built for backlog

This is the largest piece of new replication engineering since the multi-threaded applier gained writeset dependency tracking in 8.0. The Change Stream Applier (CSA) is an opt-in, per-channel alternative to the classic MTA, selected with APPLIER_VERSION = 2. It supports between 1 and 1,024 worker threads per channel, compared to the MTA’s coordinator-bound model, and, more importantly, it separates apply progression from commit progression.

Why that separation matters: in the MTA, a worker that finishes applying a transaction parks until every earlier transaction has committed when replica_preserve_commit_order is ON. Under a write-heavy backlog with long-tail transactions, workers spend more time parked than applying. In the CSA design, later independent transactions keep moving through the scheduler while earlier ones wait on their dependencies; commit order is still preserved, but the avoidable parking is gone. Relay-log reads are parallel and events are released after use, which is what makes the new per-channel memory budget possible.

Enabling it on a MySQL 26.7 replica is a channel-level change and requires the channel to be stopped:

-- MySQL 26.7.0 replica; run per channel, outside peak, with a rollback plan.
STOP REPLICA FOR CHANNEL 'ch_primary';

CHANGE REPLICATION SOURCE TO
    APPLIER_VERSION             = 2,
    APPLIER_WORKER_COUNT        = 64,
    APPLIER_EVENT_MEMORY_LIMIT  = 1073741824,   -- 1 GiB per-channel event cache
    REQUIRE_ROW_FORMAT          = 1,
    GTID_ONLY                   = 1
FOR CHANNEL 'ch_primary';

START REPLICA FOR CHANNEL 'ch_primary';

-- Verify the channel is running with the new applier.
SELECT channel_name,
       applier_version,
       applier_worker_count,
       applier_event_memory_limit
  FROM performance_schema.replication_applier_configuration;

Rollback is symmetric: stop the channel, set APPLIER_VERSION = 1, start it. The existing performance_schema.replication_applier_status_by_worker and replication_applier_status_by_coordinator views keep working, so your lag dashboards do not need to change. APPLIER_EVENT_MEMORY_LIMIT replaces the global replica_pending_jobs_size_max for CSA channels and is silently raised to replica_max_allowed_packet if you set it lower.

The prerequisites are strict and they are where most first attempts fail. The source must run with gtid_mode = ON, the channel must be GTID_ONLY = 1 and REQUIRE_ROW_FORMAT = 1, and statement or mixed binlog formats are rejected outright. File-and-position replication, SOURCE_DELAY, sql_replica_skip_counter, IGNORE_SERVER_IDS and most START REPLICA ... UNTIL forms are unsupported. If you run a delayed replica for operator-error recovery, that channel stays on the MTA.

Oracle’s engineering post reports sysbench oltp_write_only results with 64 workers where the CSA reached roughly 10.5k TPS against 7.2k for the MTA on the default durability profile with ten updates per transaction, and a wider gap with relaxed durability. Those are Oracle-reported numbers on Oracle’s hardware.

The pattern they show is plausible given the design (the gain grows with transaction size and with backlog depth), but the only number that matters for you is your own replica’s Seconds_Behind_Source recovery time after an induced backlog, measured with APPLIER_VERSION = 1 and 2 on the same channel. A clean way to induce one is to STOP REPLICA SQL_THREAD for a fixed interval under production-shaped load, restart it, and record time-to-zero-lag from replication_applier_status_by_worker. Run it at least three times per configuration and report the median.

Group Replication in MySQL 26.7: the MYSQL communication stack becomes the default

MySQL 26.7 changes the default of group_replication_communication_stack from XCOM to MYSQL, deprecates the variable itself along with group_replication_ip_allowlist, and emits deprecation warnings on the XCom-specific options. The MYSQL stack has existed since 8.0.27; what changed is that it is now the path Oracle is committing to, and new clusters bootstrapped on 26.7 will use it unless told otherwise.

Under the MYSQL stack, group communication rides the server’s own listener. group_replication_local_address must be one of the IP:port pairs the server is already bound to, the allowlist is ignored, and access control becomes ordinary MySQL authentication: the replication user needs GROUP_REPLICATION_STREAM and CONNECTION_ADMIN on top of REPLICATION SLAVE and BACKUP_ADMIN. TLS for group traffic is taken from the distributed-recovery settings, and require_secure_transport must agree with group_replication_ssl_mode on every member. Network namespaces work, which they never did under XCom.

The operational point is that this is a topology change, not a parameter change. A cluster cannot run mixed stacks; members on the wrong one simply fail to join, without a helpful error. Moving an existing XCom cluster to the MYSQL stack requires stopping Group Replication on every member and re-bootstrapping, so it is a planned outage or a ClusterSet failover, not something to attempt with SET PERSIST on a live primary. The procedure below is the one we would put in a runbook, with its gates:

-- Phase 0: pre-flight on EVERY member (MySQL 26.7.0). Record the output.
SELECT @@group_replication_communication_stack,
       @@group_replication_local_address,
       @@bind_address,
       @@require_secure_transport,
       @@group_replication_ssl_mode;

SELECT member_host, member_port, member_state, member_role, member_version
  FROM performance_schema.replication_group_members;

-- Phase 1: grant the MYSQL-stack privileges (all members, binlog off).
SET SQL_LOG_BIN = 0;
GRANT GROUP_REPLICATION_STREAM ON *.* TO '${GR_USER}'@'%';
GRANT CONNECTION_ADMIN         ON *.* TO '${GR_USER}'@'%';
SET SQL_LOG_BIN = 1;

-- Phase 2: stop the group, secondaries first, primary last.
STOP GROUP_REPLICATION;

-- Phase 3: switch stack and re-point the local address to a bound port.
SET PERSIST group_replication_communication_stack = 'MYSQL';
SET PERSIST group_replication_local_address       = '10.0.1.11:3306';
SET PERSIST group_replication_group_seeds         =
    '10.0.1.11:3306,10.0.1.12:3306,10.0.1.13:3306';

-- Phase 4: bootstrap ONCE, on the former primary only. CONFIRMATION GATE:
-- a second bootstrap creates a split brain. Verify no member is ONLINE first.
SET GLOBAL group_replication_bootstrap_group = ON;
START GROUP_REPLICATION;
SET GLOBAL group_replication_bootstrap_group = OFF;

-- Phase 5: join the remaining members, then validate.
START GROUP_REPLICATION;
SELECT member_host, member_state, member_role
  FROM performance_schema.replication_group_members;

Firewall rules, load-balancer health checks and MySQL Router bootstrap configuration all encode the port model, so a stack migration on MySQL 26.7 touches more than the database hosts. If you are bootstrapping a new cluster, let the new default stand and skip the migration entirely. If you run an existing XCom cluster and are not moving to 26.7 this quarter, there is no urgency: the deprecation is a warning in 26.7, and 9.7 LTS keeps the XCom default. Plan the switch into the next LTS upgrade rather than as a standalone change.

Two smaller HA items in the same release: the Clone plugin now understands calendar versions and permits cloning from an LTS to the next LTS (but not backwards, and not across non-sequential LTS lines), and the 9.7 LTS transfer already put the Group Replication flow-control statistics, resource manager and primary-election components into Community. For a deeper treatment of write throughput on InnoDB Cluster, see our InnoDB Cluster write-performance analysis.

Optimizer and InnoDB features MySQL 26.7 inherits from 9.7 LTS

Everything in 9.7.0 is in MySQL 26.7, and a few of those features change day-to-day performance work more than anything new to 26.7.

Hypergraph optimizer, now free, still off by default

The hypergraph join optimizer moved from Enterprise to Community in 9.7.0. It is disabled by default and enabled with the optimizer_switch flag, either globally or per statement:

-- Per-statement trial: no global blast radius.
SELECT /*+ SET_VAR(optimizer_switch = 'hypergraph_optimizer=on') */
       o.customer_id,
       SUM(oi.quantity * oi.unit_price) AS revenue
  FROM orders      AS o
  JOIN order_items AS oi ON oi.order_id = o.order_id
  JOIN products    AS p  ON p.product_id = oi.product_id
 WHERE o.created_at >= '2026-08-01'
 GROUP BY o.customer_id
 ORDER BY revenue DESC
 LIMIT 50;

-- Compare plans. Hypergraph only emits EXPLAIN FORMAT=TREE.
EXPLAIN FORMAT=TREE
SELECT /*+ SET_VAR(optimizer_switch = 'hypergraph_optimizer=on') */ ...;

The adoption blocker is the EXPLAIN restriction: hypergraph does not produce TRADITIONAL or JSON output, only TREE. Any plan-diffing tooling built around JSON EXPLAIN stops working the moment you enable it. Note also that MySQL 9.5 already changed the default explain_format to TREE and explain_json_format_version to 2, so scripts that parsed the old tabular output broke one release earlier. Both are dynamic and can be reverted with SET GLOBAL explain_format = 'TRADITIONAL'. Measure before and after with the digest statistics rather than with anecdotes:

SELECT digest,
       LEFT(digest_text, 80)                          AS stmt,
       count_star                                     AS execs,
       ROUND(sum_timer_wait / count_star / 1e9, 3)    AS avg_ms,
       ROUND(quantile_99 / 1e9, 3)                    AS p99_ms,
       sum_rows_examined / count_star                 AS rows_examined_avg
  FROM performance_schema.events_statements_summary_by_digest
 WHERE schema_name = 'app'
 ORDER BY sum_timer_wait DESC
 LIMIT 20;

Foreign keys moved to the SQL layer (9.6)

MySQL 9.6 moved foreign-key enforcement and cascade handling out of InnoDB and into the SQL layer, and MySQL 26.7 inherits that. The stated reason is completeness of the binary log: cascaded deletes and updates now appear as row events. That is good news for CDC pipelines (Debezium and friends finally see cascades) and a surprise for capacity planning, because binlog volume and replication row traffic go up on schemas with deep cascade chains. innodb_native_foreign_keys reverts to the old behaviour. Track the delta with SHOW BINARY LOGS growth per hour before and after the upgrade.

Defaults that changed underneath you

Several InnoDB and replication defaults shifted between 9.3 and 9.5, and every one of them is live on a fresh MySQL 26.7 install. The ones that alter observable behaviour:

Container-aware sizing

Since 9.3 the server reads cgroup limits rather than host totals when auto-sizing innodb_buffer_pool_size, innodb_buffer_pool_instances, innodb_page_cleaners, innodb_purge_threads, the read and parallel-read thread counts and temptable_max_ram. MySQL 26.7 adds the 9.7 refinement for cpuset-cpus. If you run MySQL on Kubernetes, this is the release where a pod with a 16 GiB limit stops trying to allocate a buffer pool sized for the 256 GiB node underneath it. Confirm what the server actually chose after startup with SELECT @@innodb_buffer_pool_size, @@innodb_buffer_pool_instances, @@server_memory; and compare against the pod spec.

The 9.6 undo-truncation work, which in 26.7 persists progress in the tablespace header instead of separate log files, is the other InnoDB item worth noting: long undo truncations now survive a crash cleanly.

Post-quantum TLS in MySQL 26.7

When built against OpenSSL 3.5 or later, MySQL 26.7 negotiates post-quantum key-exchange groups on TLS 1.3 connections. Fourteen new variables control it across five channels: client, admin, asynchronous replication, Group Replication and X Plugin. Each channel has a *_tls_kex list, a *_force_pqc boolean and a *_use_pqc_sign boolean, all defaulting to OFF, so nothing changes until you opt in. The negotiated result is visible per session in Tls_key_exchange_algorithm and Tls_sign_algorithm.

-- Opt replication traffic into a hybrid PQC group first; clients later.
SET PERSIST replication_tls_kex   = 'X25519MLKEM768:X25519';
SET PERSIST replication_force_pqc = ON;

-- Verify on an established replication session.
SHOW SESSION STATUS LIKE 'Tls_key_exchange_algorithm';

The performance angle is real but small: ML-KEM handshakes carry a larger key share, so the cost lands on connection establishment, not on the data path. For a pooled application it is noise. For a workload that opens thousands of short-lived TLS connections per second it is measurable, and it stacks with the doubled caching_sha2_password_digest_rounds. Watch Connections per second against CPU before turning on force_pqc for the client channel. The build dependency matters too: distro packages built against OpenSSL 3.0 silently lack the capability, so check SHOW VARIABLES LIKE 'tls_version' alongside @@version_compile_os and the linked OpenSSL version reported at startup.

What is still Enterprise-only in MySQL 26.7 and 9.7

After the 9.7.0 transfer (hypergraph, telemetry, the three Group Replication components, replication applier metrics, JSON duality-view DML) and the 26.7 thread-pool transfer, the Enterprise value proposition has moved from performance and HA to security, backup and AI. That is worth stating plainly because it changes the renewal conversation. This is the remaining list as of the July 2026 releases:

MySQL Enterprise Monitor is absent from that table because it reached end of life on 1 January 2025. Estates still running it should be on a Percona Monitoring and Management migration path regardless of edition. The renewal audit we run before offering any opinion is short: SHOW PLUGINS, the information_schema.plugins rows with a non-null library, the audit% and keyring% variables, and whether MEB appears in the backup jobs. In most Enterprise estates the answer is MEB plus Audit, and both have substitutes; on 26.7 specifically, the thread pool is no longer a reason to renew.

Upgrade checklist before the first MySQL 26.7 host

Whether the target is 9.7 LTS or MySQL 26.7, the pre-flight is the same, because the landmines arrived in 9.3, 9.5 and 9.6 and 26.7 inherits all of them. Run these on every source and replica and keep the output with the change record:

-- 1. Replication defaults that flip on upgrade (9.5+).
SELECT @@gtid_mode,
       @@enforce_gtid_consistency,
       @@binlog_format;

SELECT channel_name,
       ssl_allowed,
       ssl_verify_server_cert
  FROM performance_schema.replication_connection_configuration;

-- 2. Variables removed in 9.3 that abort startup if left in my.cnf.
SELECT variable_name
  FROM performance_schema.global_variables
 WHERE variable_name IN ('innodb_undo_tablespaces',
                         'innodb_log_file_size',
                         'innodb_log_files_in_group',
                         'replica_parallel_type');

-- 3. Authentication plugins that no longer load by default (8.4+).
SELECT user, host, plugin
  FROM mysql.user
 WHERE plugin IN ('mysql_native_password', 'sha256_password');

-- 4. Functions moved out of core in 9.6 (MD5, SHA1 -> classic_hashing component).
SELECT table_schema, table_name, column_name, generation_expression
  FROM information_schema.columns
 WHERE generation_expression REGEXP 'MD5\(|SHA1\(';

# 5. Oracle's upgrade checker (MySQL Shell). 26.7 adds live progress reporting.
mysqlsh -- util check-for-server-upgrade 
    ${MYSQL_USER}@${MYSQL_HOST}:3306 
    --target-version=26.7.0 
    --output-format=JSON 
    --config-path=/etc/my.cnf > /var/tmp/upgrade-check-$(hostname).json

Then capture the baseline you will compare against after the upgrade: the top-50 digests by total wait from events_statements_summary_by_digest, replication lag distribution over a representative week, binlog growth per hour, and connection and CPU counters. Without the baseline, the post-upgrade argument about whether the hypergraph optimizer or the change-buffer default helped or hurt is opinion. Test on a clone with production-shaped load, keep a downgrade path inside the LTS series, and treat the Innovation track as something you can roll forward from but not back.

Where MySQL 26.7 fits in a production estate

MySQL 26.7 is the most substantial Innovation release since the track was introduced, and for once the interesting parts are not previews. The Community thread pool solves a concurrency problem that used to cost an Enterprise subscription or a switch to Percona Server. The Change Stream Applier is a genuinely new replication architecture, and any estate that has fought replica lag with replica_parallel_workers tuning should benchmark it. The MYSQL communication stack default and post-quantum TLS are both changes you can defer, but not ignore.

The deployment answer for most fleets is still 9.7 LTS for anything that has to stay put, with MySQL 26.7 on replicas and on quarterly-refreshed tiers where the CSA and the thread pool pay for the operational cadence. Whichever way you go, the version ladder is fixed: 8.0 to 8.4 to 9.7, adjacent only, and 8.0 is already out of patches.

If you want a second pair of eyes on an upgrade plan, a Change Stream Applier benchmark or a Group Replication stack migration, our MySQL consulting and MySQL upgrade and migration teams do this work every week across Community, Enterprise and Percona builds. As always: test every change here on a non-production copy first, and keep the DR posture verified before the upgrade window, not after.

References

MySQL 26.7.0 release notes · MySQL 9.7.2 release notes · Oracle: MySQL July 2026 GA releases · Oracle: Introducing the Change Stream Applier · Thread pool operation, MySQL 26.7 reference manual · Group Replication communication stacks · MySQL end-of-life notice · Ronald Bradford: a first look at MySQL 26.7 Early Access

MySQL 26.7 and 9.7 LTS: 7 Powerful Features for Performance, Scalability and High Availability appeared first on MariaDB.org

Source

MariaDB R2DBC Connector 1.4.2 now available appeared first on MariaDB.org

One day, my friend Martín told me about a problem he and his team were dealing with. 

Every time they needed to run a pre-production test, they had to restore a copy of the production database into their test cluster. That process alone takes about three hours because of the size of the database.

If something went wrong during the test and they needed to run it again, they had no choice but to sit through the entire restore cycle again.

Three hours. Twice (or more) just to run the pre-production test.

That’s what motivated me to build mongorewind, a terminal UI tool that watches your MongoDB cluster for changes and lets you instantly undo all of them, inserts, updates, replaces, and deletes. In reverse order, without touching your backup.

How It Works

mongorewind opens a cluster-wide change stream and records every data-modifying operation to a local log file. When you press R to rewind, it applies the inverse of each recorded operation in reverse chronological order:

To undo updates and deletes correctly, mongorewind needs to know what the document looked like before the change. 

It captures this automatically using MongoDB’s changeStreamPreAndPostImages feature, which it enables on every collection it finds — and on any new collection the moment it is created.

Requirements

• Go 1.24+ and MongoDB 6.0+ running as a replica set or sharded cluster (change streams are not available on standalone instances, can be a 1 node replica set)

Installation

git clone https://github.com/zelmario/mongorewind.git
cd mongorewind
go build -o mongorewind .

Or install directly:

go install github.com/zelmario/mongorewind@latest

Running It

Start mongorewind pointing at your cluster before running any tests:

mongorewind --uri "mongodb://host1:27017,host2:27017,host3:27017/?replicaSet=rs0"

While mongorewind is running, you will see a terminal dashboard showing the operations it has recorded:

The status indicator shows ● watching (green) while the change stream is active. When you press R, it switches to ○idle (yellow) while the rewind is in progress.

Once the test run is complete and something went wrong, you can press R and mongorewind undoes every change it recorded — bringing the data back to exactly the state it was in before the test started. No restore, no waiting.

Using It in CI Pipelines

If you are running automated tests in a CI environment, mongorewind also supports a non-interactive mode. You can start the watcher in a terminal or the background and trigger rewinds from your scripts:

bash

# Start the watcher in the background

mongorewind --uri "mongodb://..." &

# Run your test suite

run_tests




# Rewind all changes and run again

mongorewind --rewind

run_tests

mongorewind –rewind exits with code 0 on success and 1 on error, so it integrates naturally into any CI pipeline.

If you use a custom log path, pass the same –log value to both commands so they share the same socket:

mongorewind --log /tmp/mytest.log --uri "mongodb://..." &

mongorewind --log /tmp/mytest.log --rewind

A Few Things to Keep in Mind

Replica set required. Change streams need a replica set or sharded cluster. If you are working locally, you can start a single-node replica set with:

mongod --replSet rs0

And then run rs.initiate() in the mongo shell.

The log file is scoped to a session. The file is truncated on startup and after a successful rewind, so each test session starts clean. System databases (admin, local, config) and system.* collections are ignored automatically.

Pre-images are enabled automatically. mongorewind polls every 2 seconds to catch newly created collections and enables changeStreamPreAndPostImages on them. You don’t need to configure anything manually.

Going Back to Martín’s Problem

With mongorewind, Martín’s team does the restoration once. After that, every time a test fails and they need to start over, they just run mongorewind –rewind. The database goes back to its original state in seconds, and the test runs again. Three hours become just a few seconds.

Contributions are welcome! Since I’m not a developer, your feedback is valuable. If you are a developer and notice any mistakes or want to enhance the script, please feel free to contribute!

The post Mongorewind: Rewind Your MongoDB Test Data Without Restoring a Backup appeared first on Percona.

Mongorewind: Rewind Your MongoDB Test Data Without Restoring a Backup appeared first on MariaDB.org

Yuchen has discovered cases where using ARRAY index for reads causes different query results from the same query not using the index. Looks like a bug (or maybe two). Sometimes one has to choose between being compatible with MySQL and producing correct query results.

On how MySQL implements JSON ARRAY indexing appeared first on MariaDB.org

Oracle offers the same category of functionality, but its server-side plugin is part of the paid MySQL Enterprise Edition. Percona’s implementation is open source and adds three capabilities the Enterprise plugin does not provide: automatic signing-key synchronization from a JWKS endpoint, IdP group-to-role mapping, and proxy-user support. This article explains how the plugin works and why those differences matter in practice.

What OpenID Connect Brings to MySQL Authentication

OpenID Connect is an identity layer built on top of the OAuth 2.0 authorization framework [5]. Whereas OAuth 2.0 governs delegated access to resources, OIDC adds a standardized way for a client to establish who a user is. After a user signs in to an Identity Provider, the IdP issues a signed JSON Web Token (JWT), called an ID token, that carries the user’s identity and attributes in a verifiable, tamper-evident form.

Using that model for MySQL authentication brings several practical advantages over password-based accounts:

• Alignment with single sign-on. Users authenticate once with their IdP and can reuse that session context across OIDC-aware applications, including databases. User lifecycle and password management remain centralized.
• No long-lived secrets on the wire. ID tokens are short-lived and cryptographically signed, so there is no static password to steal, rotate, or accidentally commit to a configuration file.
• Support for hybrid deployments. Organizations that run MySQL on-premises while hosting applications in the cloud can still authenticate through the same identity plane on both sides.
• Broad interoperability. Because OpenID Connect is a widely adopted standard, the plugin can work with any compliant provider, including Keycloak, Okta, Microsoft Entra ID, and Google Identity.

None of that is unique to Percona; Oracle makes a similar value proposition for the Enterprise plugin. The real difference lies in how much operational burden the plugin removes from the administrator, which becomes clear in the next sections.

How OpenID Connect Authentication Works

Once the plugin and its configuration are in place, the authentication path is the same regardless of which IdP issued the token:

1. The user authenticates to the IdP and receives a signed ID token.
2. The token is written to a local file that only the client operating system account can read.
3. The MySQL client uses an option that causes the client-side OIDC plugin to load and read the token from the file. The token is sent to the server as part of the authentication handshake.
4. The server validates the secure channel and decodes the token. It then verifies the token signature using the selected IdP’s public key, checks the expiration time, and validates the configured claims.
5. The server resolves the final identity as either a personal account or a group-based proxy target. The plugin may also return roles mapped from the user’s group membership.

Configuring Trusted Providers and Letting the Plugin Manage the Keys

Identity Providers rotate their signing keys periodically as a basic security measure. If a key is ever compromised, limiting its lifetime reduces the potential impact, and regular rotation also lowers the long-term value of any one key as a target. In practice, rotation is gradual: a new key is published and accepted before it starts signing tokens, and an old key remains valid for a period after it stops signing so that tokens already in flight can still be verified.

Public keys are exposed through the standard JWKS (JSON Web Key Set) endpoint, which applications can use to verify tokens issued by the IdP [6].

The Percona OpenID Connect authentication plugin can download public keys from a configured JWKS endpoint when the plugin is loaded, typically during installation and server startup, and store them in a cache. It also provides a User Defined Function (UDF) that can refresh the cache on demand or periodically through the Event Scheduler.

By contrast, Oracle’s counterpart plugin requires signing keys to be configured statically through the authentication_openid_connect_configuration server variable, supplied either as an inline JSON string or as a path to a JSON file. There is no retrieval or refresh from the JWKS endpoint, so keeping keys current after each rotation remains a manual task for the administrator. In the window just after a rotation, tokens signed with the previous key are still valid but cannot be verified until the configuration is updated. Percona’s plugin supports static key configuration as well, but that mode is better suited to testing or temporary setups than to production.

Example

Using the feature requires two simple steps. First, JWKS endpoint URL must be set in the plugin’s configuration. For example, the below configuration defines IdP named as example-keycloak (pay attention to jwks-url element):

{
  "example-keycloak": {
  "issuer-name": "https://keycloak.example.com/realms/master",
  "jwks-url": "https://keycloak.example.com/realms/master/protocol/openid-connect/certs",
  "audiences": [ "mysql-oidc" ]
  }
}

The second step is ensuring the MySQL event scheduler is running and creating an event updating the keys. For example, to enable updating the keys for example-keycloak every hour run from MySQL client:

CREATE EVENT update_oidc_keys
  ON SCHEDULE EVERY 1 HOUR
  DO SELECT update_jwks("example-keycloak");

Benefits of Using IdP Groups

This is where Percona’s plugin diverges most clearly from the Enterprise implementation.

Groups are managed by the corporate Identity Provider and group membership may be carried by ID tokens. OIDC does not define a standard claim for that, but most IdP implementations allow adding a group claim to the tokens. The Percona’s plugin allows the administrator to configure the group claim name so that it matches the token format used by the chosen IdP.

There are two practical ways to take advantage of this feature:  group-to-role mapping and proxy users.

Group-to-Role Mapping

Membership in a group can automatically translate into MySQL roles and therefore privileges across multiple MySQL servers at the same time. On a single server, the flow looks like this:

1. The administrator creates roles and grants them privileges.
2. The administrator defines the IdP group-to-MySQL role mapping in the plugin configuration file.
3. When the user connects, the plugin returns the roles that match the user’s groups, and the server automatically grants those roles to the user.
4. The user can activate any granted role and exercise the privileges assigned to it.

Please note, that group-to-role mapping still requires an account created for each user, but automates managing user privileges.

Example

To create roles and grant them some privileges one may run:

CREATE ROLE accounting;
GRANT ALL PRIVILEGES ON accounting_database.* TO accounting;
CREATE ROLE sales;
GRANT ALL PRIVILEGES ON sales_database.* TO sales;

Then, to to define the mapping add to IDP configuration:

"group-claim": "groups",
"group-role": [
  { "/accounting": "accounting" },
  { "/marketing": "marketing" }
]

Any user connecting with an ID token containing claim “groups”:[“/accounting”] will be granted with role accounting and effectively obtain access to accounting_database and so on.

Proxy Users

The proxy capability in MySQL allows an authentication plugin to request that the connecting external user be logged in as a different MySQL user. In this model, the external identity is the proxy user and the mapped MySQL account is the proxied user. The purpose is to let multiple users share accounts with the same privilege set, avoiding the need to create a separate personal database account for every individual.

This feature must be supported by the authentication plugin, whose job is to choose the proxied user according to the specifics of the authentication method. In the Percona OIDC plugin, that selection is based on the group claim in the token and works as follows:

1. The administrator creates a proxy user identified by the OIDC plugin. This can be either a single anonymous account (”@”) without a specific group name, referred to as anonymous proxying, or multiple group-related accounts, referred to as named group proxying.
2. The administrator creates proxied users for each group. These accounts should not use a login plugin, so nobody can connect to them directly. The username must match the group name.
3. The administrator grants the PROXY privilege for each proxy user on all related proxied users.
4. When a user connects, in the anonymous proxying case the plugin returns the user’s first group as the proxied username. In the named group proxying case, the plugin checks whether the user belongs to the group and returns that group as the proxied username.
5. The server verifies that the requested proxied account exists and that the proxy user has the required PROXY privilege on it. If both checks succeed, the session runs with the proxied account’s privileges.

Other Features

Supported signing algorithms include RSASSA-PKCS1-v1_5, RSASSA-PSS, and ECDSA with SHA-256, SHA-384, and SHA-512 hashing functions.

The Percona approach uses the client-side OpenID Connect plugin from upstream MySQL, which ensures compatibility with the standard Oracle client.

Both client-side and server-side OpenID Connect plugins ensure that the token is sent via a secure channel. Accepted protocols are TCP protected by TLS, Unix sockets, and shared memory.

What OpenID Connect Authentication Does Not Do

There are some limits worth knowing.

The first comes from MySQL’s authentication design: any authentication plugin is used at connection time only. In the case of OIDC, the token is validated when the user connects, and a session that stays open may outlive the ID token that opened it. There is no out-of-the-box mechanism to force re-authentication after some time (except for idle connection timeout).

A similar situation applies to group-role mapping. The roles tied to the user’s groups in the ID token are granted or revoked at connection time. As a result, if a user is added to or removed from an IdP group, they must reconnect to Percona Server for the change to be reflected in their granted roles.

The proxying mechanism uses group membership claim instead of the token’s subject, so any token signed by a configured IdP that carries the required group is accepted. Group membership is your trust boundary in those modes, so treat it that way.

The current proxying implementation assumes the proxied user’s name matches the group name. This can be a problem when a group name isn’t a valid MySQL username (for example, it’s too long or contains disallowed characters), or when multiple groups need to map to a single account. We plan to add group-to-proxied-account mapping in future releases to address this.

The client-side plugin doesn’t verify the ID token (for example, check whether it has expired) before connecting, and the server doesn’t report the reason for access being denied (for security reasons). A good practice is to obtain a fresh token before connecting.

Conclusion

Functionally, Percona’s OpenID Connect plugin covers the same core ground as the counterpart in MySQL Enterprise Edition: signed ID tokens, claim validation, subject matching, and secure-transport enforcement.

It goes further in several important areas:

• It is open source.
• Keys can stay current automatically through JWKS synchronization.
• Group-to-role mapping allows IdP group membership to drive MySQL role grants for the lifetime of the session.
• Proxy-user support allows many IdP identities to share a smaller set of MySQL accounts.

Our OIDC implementation is suitable for real-world identity operations at scale. It can automatically map identities and groups managed by an IdP to database users and roles, and synchronize cryptographic keys.

References

1. Percona Server for MySQL documentation: OpenID Connect authentication.
2. Percona Server for MySQL documentation: Get started with OpenID Connect authentication.
3. MySQL 9.7 Reference Manual: OpenID Connect Pluggable Authentication.
4. MySQL 9.7 Reference Manual: Proxy Users.
5. OpenID Foundation: How OpenID Connect Works
6. auth0 Docs: JSON Web Key Sets.

Written by Michal Jankowski. Reviewed by Dennis Kittrell and Oleksiy Lukin.
Percona® is a registered trademark of Percona LLC. MySQL® is a registered trademark of Oracle Corporation.

The post OpenID Connect Authentication for MySQL, Now Fully Open Source appeared first on Percona.

OpenID Connect Authentication for MySQL, Now Fully Open Source appeared first on MariaDB.org

The new MyEnv can be downloaded here. How to install MyEnv is described in the MyEnv Installation Guide.

In the inconceivable case that you find a bug in the MyEnv please report it to our public repository on Codeberg.

Any feedback, statements and testimonials are welcome as well! Please send them to us.

Upgrade from 2.x to 3.0

Please check the MyEnv Installation Guide: Upgrading MyEnv.

In MyEnv v3.0.1 tpl/aliases.conf.template has changed. Thus you should replace the MyEnv aliases.conf as follows:

$ cp /etc/myenv/aliases.conf /etc/myenv/aliases.conf.$(date '+%Y-%m-%d')
$ cp myenv/tpl/aliases.conf.template /etc/myenv/aliases.conf

Changes in MyEnv 3.0.1

MyEnv

• Bug in setMyEnv.php fixed, socket was not set correctly.
• Some more aliases added for log tracking.
• Prompt port was always delayed by one switch, fixed.
• Missing port in PS1 prompt fixed again (issues/12).
• up/down for PostgreSQL should work correctly now.
• PostgreSQL should be evaluated correctly now in up.
• up should show PostgreSQL status correctly now (issues/4).
• Environment variables refactored and PostgreSQL specific variables added (issues/3).
• Discrepancies between my.cnf and myenv.conf leads now to abort if relevant configuration variables are affected (issues/5).
• Database seems OK after start which is wrong, timing issue (issues/2).
• hideschema works better now (issues/13).

MyEnv Installer

• Wrong default for socket fixed in installer.
• Nasty error message when installing PostgreSQL suppressed.
• basedir filter implemented.
• Instance type mariadb was added to MyEnv installer (issues/10).
• MySQL 26.7 is recognized correctly now (issues/1).

MyEnv Utilities

• insert_test.sh made more PostgreSQL friendly.
• test table commands made more comfortable for PostgreSQL.

PostgreSQL

• See MyEnv, MyEnv Installer and MyEnv Utilities.

General

• Cosmetic fixes.
• Some bugs and feature requests moved from TODO to …

MariaDB/MySQL Environment MyEnv 3.0.1 has been released appeared first on MariaDB.org

Source

MariaDB 12.3 InnoDB-Accelerated Binary Logging: Delivering 1.50x Higher Throughput appeared first on MariaDB.org

Continue reading “MariaDB Foundation Newsletter – September 2026”

MariaDB Foundation Newsletter – September 2026 appeared first on MariaDB.org

MariaDB Foundation Newsletter – September 2026 appeared first on MariaDB.org

In this post we will discuss performing a same-CA renewal: replacement certificates for server, member, and client leaf are issued by the existing trusted CA, and the X.509 attributes used for cluster membership do not change. In this scenario, the rotateCertificates command reloads TLS material for new connections without restarting mongod or mongos.

Important: Do not apply this hot-reload procedure when replacing the issuing CA, changing a certificate subject DN, or changing cluster-membership attributes. Those are not ordinary renewals.

What rotates, and what does not

Percona Server for MongoDB can reload the files configured through the following TLS options:

net:
  tls:
    mode: requireTLS
    certificateKeyFile: /etc/mongod/tls/server.pem
    CAFile: /etc/mongod/tls/ca.pem
    clusterFile: /etc/mongod/tls/cluster.pem

The certificateKeyFile contains the certificate and private key presented to normal clients. The clusterFile holds the certificate and key that a mongod or mongos process presents when connecting to other cluster members. If clusterFile is not configured, certificateKeyFile is also used for member authentication.

The rotateCertificates command affects new TLS connections. It does not terminate established client sessions or force a replica-set election.

Before the maintenance window

Begin this process well in advance of the certificate expiry, and avoid performing your first attempt in a production environment.

1. Inventory every process and client certificates. Include all mongod members, all mongos routers, application drivers, mongosh hosts, backup jobs, monitoring, and automation tools.
2. Confirm this is a same-CA renewal. The issuer chain trusted by every participant stays the same, and the O, OU, and DC attributes used for default internal X.509 membership matching remain unchanged.
3. Create a new PEM file for every server and client that needs rotation. A PEM file referenced by certificateKeyFile or clusterFile must include both the certificate and its matching private key. The file must strictly contain the key first, followed by the certificate, including their encapsulation boundaries.
4. Verify the new certificate details and validate the cert against the CA before copying to the production TLS directory

Stage a renewed server or member certificate

There are a few limitations for rotating certificates online:

• Each new certificate must have the same filename and same filepath as the certificate it is replacing.
• If the TLS Certificate is password-protected, its password must be the same as the old certificate it is replacing.

If CAFile, a CRL, or another configured TLS input is being renewed as part of the same operation, replace it before invoking the reload command. The command reloads the configured TLS inputs as a set; a missing or invalid input causes the reload to fail.

Luckily, incorrect certificate files will cause the rotation to fail, but will not invalidate the existing configuration or have any other side effects.

Reload one process

Connect directly to the specific mongod or mongos with an administrative user and execute the following command:

db.getSiblingDB("admin").runCommand({rotateCertificates: 1, message: "Renewed TLS certificate"})'

Immediately validate a new TLS connection to that process with a renewed client certificate. Also inspect the log for the successful certificate-rotation message and any TLS errors. Check our documentation for guidelines to perform the procedure on a replica set or sharded cluster.

Final validation and cleanup

After completing the procedure, it is a good idea to reconfirm the expiry date and SANs of the certificate presented by every mongod and mongos. Retain the old certificates only for the approved overlap period, then remove or revoke them. Don’t forget to record the new expiry dates and create alerts with enough lead time before the expiration date of the new certificates.

When the CA or member identity changes

A different procedure is required when any of the following changes:

• The issuing CA or trusted CA chain.
• The subject DN used by a MONGODB-X509 client user.
• The O, OU, or DC values used for default intra-cluster X.509 membership matching.
• net.tls.clusterAuthX509.attributes or net.tls.clusterAuthX509.extensionValue.

This is a topic for another time.

The post Rotating Expiring X.509 Certificates in Percona Server for MongoDB with Minimal Service Interruption appeared first on Percona.

Rotating Expiring X.509 Certificates in Percona Server for MongoDB with Minimal Service Interruption appeared first on MariaDB.org

MariaDB at Percona Live Amsterdam 2026: Ecosystem, Plugins, Security, and Community appeared first on MariaDB.org

MariaDB at Percona Live Amsterdam 2026: Ecosystem, Plugins, Security, and Community appeared first on MariaDB.org

In July our community lead Laura Czajkowski published Introducing Mountaineers: A Way to Say Thank You. That program is why we built the board. Laura wanted contribution to stop disappearing into noise: the bug filed late at night, the forum thread that sat unanswered, the PR, the hour telling engineering what’s broken. Mountaineers tracks that energy and gives something back: recognition, access, swag. Points, Basecamp, and the reward ladder are in her post and on the Mountaineers page. The leaderboard is the public face; my job was to make the numbers trustworthy enough for that program to work.

Why the board exists

Laura’s “what counts” list is what the board has to measure. It isn’t only code. GitHub issues, PRs, and merges count. Forum topics, replies, and accepted solutions count. Blog posts and tutorials count, including work through the Community Writers Program. Direct feedback to engineering counts too. Show up in more than one place in the same month and the climb goes faster. The public leaderboard is how that becomes visible outside the team.

A board only helps if the community trusts it. That’s where the engineering problem starts. The store holds 600+ Percona GitHub repositories; in 2026 staff alone were active in 100+ of them (~150 employee contributors). Staff and community work in the same channels – GitHub and the forum – so without filters you cannot tell whose work you’re looking at. Separating community PRs from staff PRs, and spotting forum questions from community members that still have no answer, isn’t something you do by scrolling notifications. You need a place where that signal is collected, filtered, and easy to inspect.

That’s what the internal dashboards are for. They aren’t the public site. They’re how the community team finds the work worth recognizing and the threads that still need a human.

In 2026 so far community activity is already substantial. On GitHub: ~750 pull requests and issues from 270+ contributors, including ~80 merged PRs. On the forum: ~290 active community users, ~400 active topics, and ~900 community posts. The public board for that year already has 498 people on it (257 GitHub, 240 forum, 5 content). Staff also post heavily in the same places, which is why the dashboards filter staff vs community before anything is scored for Mountaineers. Without that step, employee noise would bury the people the program exists to thank.

To turn that activity into a comparable ranking, we use a simple, fixed set of weights:

Content is still thin on the board. We only recently launched the Community Writers Program, so most climbing today is GitHub and forum. That will change as more posts land.

Those numbers and weights are the why. The rest of the post is the how: private dashboards, a public JSON feed, PSMDB, and a recipe you can reuse.

Private dashboards, public site

The architecture is a bit unusual on purpose. The dashboards live in our private corporate cloud. They talk to APIs, write to PSMDB, and show rich views for staff. The community website is a static Hugo site on GitHub Pages. It has no backend and no path into that cloud.

So we needed a bridge that doesn’t couple the two. Once a day the leaderboard job scores the periods we care about (month, quarter, year), writes community-only JSON, and publishes it to a public GitHub repository: percona/community-leaderboard. The Hugo page embeds a JS widget that fetches those files from raw GitHub URLs. No VPN. No private API. If the internal app is offline for maintenance, the last published feed still works.

We call that lifehack GitHub as a database. It’s a boring, cacheable public artifact. An object store would work the same way. For a static site, a daily JSON dump is often simpler and safer than exposing your analytics database.

The application breaks into a few components. Data sources are polled by indexer jobs and land in PSMDB. The same database feeds internal dashboards (explore, filter staff vs community) and a leaderboard component that builds period reports and publishes JSON to GitHub. The community site only talks to that public feed.

flowchart TB
subgraph sources["Data sources"]
GH["GitHub"]
Forum["Forum"]
Content["Blog / content"]
end
subgraph private["Private app · corporate cloud"]
Idx["Indexer jobs"]
Mongo[("PSMDB")]
Dash["Dashboards"]
LB["Leaderboard component"]
end
subgraph public["Public"]
Feed["JSON on GitHub"]
Site["Community site · Hugo widget"]
end
GH --> Idx
Forum --> Idx
Content --> Idx
Idx -->|"index · upsert"| Mongo
Mongo -->|"read · filter · charts"| Dash
Mongo -->|"score · export"| LB
LB -->|"publish rankings"| Feed
Feed -->|"fetch JSON"| Site
style Mongo fill:#e8f5e9
style Feed fill:#fff8e1
style Site fill:#e1f5ff
style private fill:#f9f7ff

Indexers and the leaderboard component run on a schedule (cron). The diagram above is the data path; the schedule is just when each box wakes up.

On the site, the widget is more than a dumped table. You can switch periods and categories (global, GitHub, forum), open a person and see how they scored, and browse leaders without leaving Hugo. The layout had to feel like part of Community Ascent, not like an admin export pasted into a page. There is also a Global Summit view (top 10 on a mountain) for the people who climb furthest.

Public feed: JSON the widget reads

The Hugo widget does not guess rankings. It loads plain JSON from the public repo. On first paint it fetches meta.json for available periods and categories, then {category}/{period}.json for the table (for example global/2026.json). When you open someone in the modal, it loads users/{period}/{user_key}.json for the breakdown.

The layout is simple:

meta.json
global/{period}.json
github/{period}.json
forum/{period}.json
users/{period}/{user_key}.json

meta.json is the index. The widget uses it to build the period picker:

{
 "generated_at": "2026-08-31T04:11:39+00:00",
 "default_period": "2026",
 "periods": [
 { "key": "2026", "label": "2026", "type": "year" },
 { "key": "2026-Q3", "label": "Q3 2026", "type": "quarter" },
 { "key": "2026-08", "label": "August 2026", "type": "month" }
 ],
 "categories": ["global", "github", "content", "forum"]
}

Each ranking file is one period and one category. The table reads top30:

{
 "period": "2026",
 "period_type": "year",
 "period_label": "2026",
 "category": "global",
 "top30": [
 {
 "rank": 1,
 "user_key": "gh-example",
 "display_name": "Alex Contributor",
 "avatar_url": "https://avatars.githubusercontent.com/u/12345?v=4",
 "github_login": "alex-contrib",
 "forum_username": null,
 "points_total": 595,
 "points_github": 500,
 "points_forum": 95,
 "issues_created": 2,
 "prs_created": 5,
 "prs_merged": 6,
 "topics_created": 1,
 "replies": 4,
 "solutions": 1
 }
 ]
}

The per-user file adds contribution lists for the detail view:

{
 "user_key": "gh-example",
 "display_name": "Alex Contributor",
 "period": "2026",
 "points_total": 595,
 "github_prs": [
 {
 "title": "Fix replication lag in operator",
 "url": "https://github.com/percona/example/pull/42",
 "repo": "percona/example",
 "date": "2026-07-10",
 "merged": true
 }
 ],
 "forum_replies": [
 {
 "title": "Slow queries after upgrade",
 "url": "https://forums.percona.com/t/example/12345",
 "date": "2026-06-15"
 }
 ]
}

That is all the static site needs. Live files are in percona/community-leaderboard; the widget base URL is set in the Hugo layout (window.LB_BASE).

What runs every day

Scheduled indexer jobs poll the APIs we care about, one family of sources each, and upsert documents into PSMDB. Another scheduled job builds the rankings and pushes the public feed. The web UI is how we see the store: date ranges, charts, activity tables, and a leaderboard toggle for community, staff, or all. Staff numbers matter internally. Who answered on the forum this week? Which PRs came from outside? Which community questions are still waiting? The public Mountaineers board stays community-only. Same data, different filter.

One person is often three strings in the data. GitHub login, forum username, and a blog byline under a real name rarely match. Without linking, the same Mountaineer shows up as three rows and their points never add up. Identity mapping is how we stitch that together. You can link accounts by hand in the dashboard. When a display name or handle lines up across sources, the UI also proposes a merge for approval. That is semi-automatic, not a silent auto-join. Only confirmed maps feed the public ranking.

We develop against Percona Server for MongoDB in Docker locally. Same engine shape in production. I run docker compose up, connect from the app, and start indexing. For browsing collections I use MongoDB Compass on localhost:27018 (or whatever port you map). It is the fastest way to check that indexers wrote what you expect before you trust a public export.

services:
 psmdb:
 image: percona/percona-server-mongodb:8.0
 volumes:
 - ./data:/data/db
 environment:
 MONGO_INITDB_ROOT_USERNAME: root
 MONGO_INITDB_ROOT_PASSWORD: changeme
 MONGO_INITDB_DATABASE: dashboard
 ports:
 - "27018:27017"

Pick the image tag for your CPU (8.0-arm64 on Apple Silicon, 8.0 / amd64 elsewhere). No cloud dependency just to try an idea.

MongoDB as the use case

If you like databases, this is the interesting middle of the story. We run Percona Server for MongoDB (PSMDB), but the fit is the document model. Community activity arrives as heterogeneous JSON. Issues, pull requests, forum posts, user profiles, and blog metadata don’t share one neat relational schema. Nested objects are normal, and APIs grow new fields over time. GitHub can suddenly add reactions; the forum can expose a new trust_level or group list. In MongoDB that usually means new keys on the document, not a migration that blocks indexing.

PSMDB fits that. We keep sources in separate collections and store documents close to what the API returned, then add what reporting needs: stable ids and comparable dates for range queries. Upserts by source id make re-runs safe. Staff signals such as forum groups or trust level can stay on the user document while rules evolve. Indexes on the date fields keep month and quarter scans practical. Open one document and you can explain why an event counted or not.

PSMDB is the system of record for events and profiles. Rankings are derived. The public site never connects to it. That split is useful beyond our case: a rich private store for organizers, a dumb public snapshot for the website.

We looked at paid leaderboard products first. They were rigid on sources, weak on staff versus community, or awkward with a static site. Building around PSMDB let us encode our rules instead of bending the program to a SaaS schema.

A few mistakes we deliberately avoided: exposing a public API that talks to the private database; scoring staff and community in one undifferentiated stream for the public board; asking Hugo to compute rankings at build time. The site only fetches JSON. Everything heavy stays behind the corporate network.

Building it without hundreds of hours

I could have written this stack myself. I’ve been in web development for many years and I’ve built similar pipelines before: cron jobs, API clients, admin UIs, JSON exports, frontend widgets. None of it is magic. I know how long the boring parts take when you do them by hand: wiring indexers, shaping documents, iterating on the Hugo widget layout, debugging JavaScript, mapping fields from noisy API payloads. That is often weeks of calendar time that never shows up in a program announcement.

In about a week we had a working path from APIs to PSMDB to rankings to a public widget. An AI assistant did a lot of that scaffolding for me: boilerplate API clients for GitHub and the forum, first-pass mapping of those payloads into MongoDB collections, and early prototypes of the Preact widget (layout, period picker, loading states). I still owned the architecture and the rules; the agent compressed construction I would otherwise have typed line by line. That difference is real, and I notice it because I’m not new to this work. Judgment stayed with the community team and with Laura’s brief: what counts, who is staff, how identities merge, what the public is allowed to see, and how the board should feel on the Ascent pages.

I used a similar approach for semantic search on this site with Postgres and pgvector. Different store, same idea: know what you want, let the assistant handle the repetitive build.

If you want something like this

You don’t need our private cloud. You need the same separation of concerns.

Ask for scheduled indexer jobs that poll your APIs, upsert into Percona Server for MongoDB in Docker, keep documents close to each API’s shape, and add stable ids and dates for range queries.

Ask for a private UI over that database with tables, charts, date filters, and an explicit community / staff / all view, behind basic auth or SSO, so you can verify fairness before anything is public. Use it to find outside PRs and unanswered community questions, not only to draw a ranking.

Ask for a scoring job that builds period rankings, publishes community-only JSON to a public feed (GitHub raw files or object storage), and a small JS widget for your site that only reads that feed.

Collect, understand, publish, render. Honest collection, careful filters, a simple public publish path.

If you try it and get stuck, write to me. I’m happy to talk through what worked and what we threw away. To see the result in production, open the Community Leaderboard. To join the program behind it, start with Laura’s Mountaineers announcement or the program page.

Building Useful Community Software with MongoDB Using AI Agents appeared first on MariaDB.org

We built a vector-bench to stop guessing. You name the engines you want, build them from pinned versions, put each one in the same container on the same cores with the same data, run the same measurements against all of them, and write a report. This post is about how it measures.

If you work with databases but haven’t touched vectors yet, the first half is the part you need.

What’s being indexed

An embedding is a fixed-length array of floats that comes out of a model. The useful property is that semantically similar inputs land close together when you measure the distance between them.

Two distance measures cover almost everything. L2 is an ordinary straight-line distance, the Pythagorean one, extended to however many dimensions you have. Cosine Similarity  measures the angle between two vectors and ignores their length. Which one applies is decided by the model that produced the embeddings. It isn’t a choice you get to make at query time, and getting it wrong is a good way to produce nonsense.

So the query you want is “the 10 rows whose vectors are nearest this one”:

SELECT id FROM documents ORDER BY distance(embedding, ?) LIMIT 10;

That 10 is k.

Now the problem. Answering that exactly means computing the distance from your query vector to every single row, then sorting. No B-tree or hash index helps, because neither one can order a million points by proximity in 1536 dimensions. Exact vector search is a full table scan with a lot of arithmetic bolted on.

A vector index gives up exactness to avoid that. It looks at a few thousand promising candidates instead of every row and returns the best it found. That’s the approximate nearest neighbour search, or ANN. It’s usually right.

“Usually” is doing a lot of work in that sentence, and pinning it down is most of what this benchmark does.

To score that you need to know the right answer in the first place. That’s the ground truth: the true nearest neighbours for every query, computed once by brute force with no index involved. The public ANN datasets ship theirs alongside the vectors, and without it you couldn’t score an approximate index at all.

This is the number that makes everything else meaningful, and it’s the one most vector search claims leave out. That omission is the reason this project exists.

The two kinds of vector index

Almost every database that has added vector search picked one of two designs. They attack the same problem from opposite ends, and which one you have decides what you’re allowed to tune.

HNSW

HNSW stands for Hierarchical Navigable Small World, which is a mouthful for something fairly intuitive. If you’ve ever implemented a skip list, you already have the shape of it.

It’s a graph of vectors built in layers. Every vector is a node, linked to some number of its nearest neighbours. The top layer has few nodes and its links jump long distances across the data. Each layer below has more nodes and shorter links. A search starts at the top and keeps hopping to whichever neighbour is closer to the query. When nothing is closer, it drops a layer and carries on, until it runs out of layers.

Two settings matter:

• M is how many links each node keeps. It’s fixed when the index is built. Higher M means a better-connected graph and better recall, at the cost of a slower build and a bigger index.
• ef_search is how many candidates the search keeps track of while it walks. It’s a session variable, so you can change it per query. Turn it up and the search visits more nodes, gets better recall, and runs slower.

There’s ef_construction too, the same idea applied while the index is being built. Not every engine lets you set it, which turns out to matter when you try to compare them fairly.

IVF

IVF stands for Inverted File. It partitions the data instead of linking it, not unlike list partitioning on a table.

At build time it groups the vectors into nlist clusters, each with a representative vector at its centre. At query time it compares the query against those representatives, picks the closest nprobe clusters, and searches only inside them. It builds much faster than HNSW and uses less memory, but usually gives worse recall at the same speed. It misses when the true neighbour happens to sit just outside the clusters it looked in.

We only test engines running HNSW, which is what most databases shipped. Putting an IVF engine on the same chart would mostly measure the gap between two algorithms rather than how well anybody implemented one, so IVF-only engines get their own bucket.

Why one number is never enough

Recall isn’t a property of an engine. It’s a setting, and ef_search is the dial.

Here’s one HNSW index on one machine, same data, same queries. The only difference is that on the first row the search tracks 10 candidate nodes as it walks the graph, and on the second it tracks 800:

ef_search=10 3,678 queries/sec recall 0.9593
ef_search=800 409 queries/sec recall 0.9987

Keeping 800 candidates instead of 10 finds a better answer and takes nine times as long. Both rows are honest measurements of the same index on the same hardware.

Which is why “our database does 3,678 vector queries a second” tells you nothing. You don’t know how often it was handing back the wrong rows, and the person quoting it may not know either. The reverse is just as empty: recall with no throughput next to it is free, because recall 1.0 is always available if you turn the index off and scan the table.

Every measurement here is a pair. If you take one thing from this post, take that.

What the harness puts on each engine

One table per engine. An id, an integer tag column used only by the filtered tests, the vector, and an HNSW index on it at a configured M.

CREATE TABLE t1 (
id INTEGER PRIMARY KEY,
tag INTEGER NOT NULL,
v VECTOR(1536)
);

Then two queries, plain top-k and the same search restricted to a subset of rows:

SELECT id FROM t1 ORDER BY distance(v, ?) LIMIT 10;
SELECT id FROM t1 WHERE tag < ? ORDER BY distance(v, ?) LIMIT 10;

tag holds values 0 to 99 spread evenly, so tag < 10 passes about 10% of rows and tag < 1 about 1%. That’s how we control selectivity.

Every engine writes all of this differently. Some declare the index inside CREATE TABLE, others want a separate CREATE INDEX, and the distance functions have different names everywhere. Translating that is the driver’s job, and the drivers are the only engine-specific code in the whole harness.

Every engine also has at least one setup detail that will quietly wreck your numbers. PostgreSQL, for instance, stores oversized values out of line in what it calls TOAST, and a 1536-dimension vector counts as oversized. Unless the column is set to STORAGE PLAIN, every single distance comparison pays for an extra fetch. It’s one line of DDL. Miss it and you publish PostgreSQL looking slow for a reason that has nothing to do with its vector search, and you’d never know from the results.

What we measure

Recall against throughput. Iterate ef_search against a fixed index, record recall and QPS at each point, repeat at a few values of M. k=10 throughout. The query vectors come from the dataset’s own held-out query set, never from the rows we loaded, because searching for a vector that’s already in the index is a much easier problem and would flatter everybody equally.

The two settings behave completely differently, and it shapes how long a run takes. ef_search is a session variable, so iterating it reuses the index that’s already built and each extra point costs almost nothing. M is baked into the index, so every value of M means dropping the table and loading the entire dataset again. On a million 1536-dimension vectors that’s hours per value. Hence many ef_search points and very few M values.

Build cost. Wall time, rows per second, index size on disk, peak memory.

This is the easiest place in the whole benchmark to publish a misleading number, because engines don’t build the index the same way. Engines can build indexes either incrementally, bulk, or both. What does that mean? 

Incremental. The graph is updated on every INSERT. Loading is slow, but when the last row lands the index is finished and the table is ready to query.

Bulk. All the rows load first, then the whole graph gets built in one pass. Much faster in total, but the table can’t answer a vector query until the build finishes.

Those are two different operations. One engine in our set does both, and its bulk path loaded 18 times more rows per second than its own incremental path. Same engine, same data, same machine, 18x apart.

So a bulk number from one engine put next to an incremental number from another doesn’t compare engines at all. It compares two ways of building an index, and the ratio looks impressive enough that people quote it anyway. We measure both paths on any engine that has both, and the report says which is which.

Peak memory comes from the server’s container, with the database as the only thing running in it. The harness runs in a separate container and reaches the server over a private network.

That separation matters more than it sounds. The client holds the entire dataset in memory, several GB of Python arrays. If it shared a container with the database, the container’s memory accounting would count those arrays as database memory, and every memory figure we published would be inflated by whatever the client happened to be holding.

Concurrency. QPS and latency percentiles from 1 to 32 clients. Engines cache their graphs in quite different ways and none of that shows up until clients start competing for the same cache. We report how much of the ideal speedup each engine actually got alongside raw QPS, because an engine that stops gaining throughput at 2 clients while its p99 gets 15 times worse is doing something very different from one that keeps scaling, and a throughput column on its own hides that completely.

Filtered search, at several selectivities down to 1% of rows passing. This is the case that’s supposed to justify keeping vectors in your database instead of a dedicated store, so it deserves more attention than it usually gets.

Filtering changes what “correct” means. The true top 10 among rows where tag < 10 is not the true top 10 overall, so for every selectivity we recompute ground truth by brute force over only the rows that pass. Score filtered results against the unfiltered ground truth that shipped with the dataset and every engine gets a recall near zero. We know, because we did exactly that for a while.

Some queries come back with fewer than 10 rows. In one run, 81 out of 200 did. This is not the data running out. At 10% selectivity about 99,000 rows pass the filter, so there are always at least 10 to find. The cause is the order of the operations. HNSW searches by distance first, then applies the WHERE clause. It gathers a few thousand candidates, the filter throws most of them away, and sometimes fewer than 10 are left. (If a filter really did match fewer than 10 rows, the ground truth shrinks too, and the engine still scores 1.0.) Recall already handles this. A row the engine did not return counts as a miss, so six correct rows score 0.6. We report the count because two different problems score the same. “10 rows, four of them wrong” and “six rows, all correct” are both 0.6. The first needs a wider search. The second needs iterative scanning. The count tells you which one you have. It also means the throughput is flattered, since six rows is less work than ten.

Churn. Recall and throughput before and after deleting and reinserting part of the corpus, since deletions leave graph edges pointing at rows that are gone. Whether rebuilding the index recovers what’s lost, we don’t know yet. It’s the obvious next thing to test and we haven’t done it.

Keeping the comparison fair

Everything runs twice.

The normalized pass gives every engine identical CPU, memory and cache budgets, so a difference in the results belongs to the implementation rather than to who was handed more RAM. The tuned pass lets each engine use the settings its own documentation recommends. Tuned is more realistic and less controlled, which is exactly why it doesn’t replace the first one. A result that survives both passes is about the engine. One that flips between them is interesting for a completely different reason.

Cores are pinned explicitly. One logical CPU per physical core, because SMT siblings share execution units and two threads on one core don’t behave like two cores. Never a mix of P-cores and E-cores on hybrid chips either, since migration between core types adds more variance than several of the effects we’re trying to measure. Durability is relaxed the same way everywhere, or we’d be comparing default fsync policies and calling it vector search.

Some differences can’t be equalised at all, so we write them down instead of pretending. A knob only one engine exposes goes unused in the normalized pass, because using it would hand that engine a tuning axis nobody else has. An engine that insists on a particular isolation level gets it set for everyone. And defaults that are obviously placeholders get sized from a shared budget — one family of engines still ships a 16 MiB graph cache, which is nothing, and judging an engine on a value its own vendor expects you to change measures absolutely nothing. All of these land in a “known asymmetries” section above the results.

One hardware note that catches people out. Several of these implementations ship hand-written AVX-512 code for the distance maths, where a single instruction does the arithmetic for 16 floats at once. The same index on a CPU without AVX-512 is effectively a different benchmark, and the slowdown isn’t the same for every engine, so you can’t even scale the numbers to compensate. The CPU model and its feature flags go into every run’s manifest for that reason, along with engine versions and commits, image IDs, and the resource limits as they are actually resolved rather than as we requested them. No manifest, no report.

Reading the results

Read the validity section before you look at a single chart. Our reports go environment, then validity, then known asymmetries, then results, in that order on purpose. A failed phase, an engine returning short result sets, a CPU missing the instruction set the engines wanted — all of it lands in front of you before you’ve formed an opinion.

The thing to watch for is the silent full scan.

Any of these engines will quietly stop using the vector index and scan the table instead. A scan returns exact results, slowly, so in the output it looks like high recall and low throughput. That’s indistinguishable from a conservatively tuned index unless you go and read the query plan.

It happens for thoroughly boring reasons. One engine’s optimizer costs the vector index against a table scan and takes the scan once the LIMIT is above roughly a quarter of the table, and we still haven’t found a setting that moves it. Another falls back with no error and no warning when the query asks for a different distance than the index was built for — build the index for cosine, write the query with the L2 operator, and you get a sequential scan and a sort, with nothing anywhere to tell you.

So every driver runs EXPLAIN for each configuration and checks the index name appears in the plan.

WARNING: vector index NOT used (k=10, filtered=True). Plan: …Seq Scan…

Anything that is scanned goes into validity. This is far and away the easiest way to produce impressive vector benchmark numbers by accident, and if a benchmark doesn’t mention checking for it, we’d want to know why before believing anything in it.

For recall against throughput, the useful presentation is a curve rather than a number. Iterate ef_search, plot recall against QPS, keep the best points: for each level of accuracy, the highest throughput anything reached at it. One engine beats another only where its curve sits above the other’s at the same recall. If the curves cross, then the answer genuinely depends on how accurate you need to be, and saying so is a result rather than a dodge.

Curves do invite comparing shapes instead of heights at one point, so there are bar charts as well, QPS at recall floors of 0.90, 0.95 and 0.99. Pick the accuracy you’d actually accept and read across.

Things that went wrong while we built this

Worth listing, partly because they’re the reason to trust anything else here, and partly because anyone building something similar will walk into them.

Our first ingest numbers were garbage. The load path was doing one INSERT per network round trip with autocommit on, and we measured 88 rows a second. Batching 500 rows per transaction took the same engine to 373. Publishing the first number would have been benchmarking our own client and calling it a database.

Filtered search and churn were scored against full-corpus ground truth even on runs that used a subset of rows. Every engine looked bad and the bug was entirely ours. Ground truth is now keyed on dataset, k, row count and selectivity.

Both resource passes shared one results directory, and the ANN runner skips configurations that already have results. So the tuned pass quietly skipped everything the normalized pass had computed, and our tuned numbers were mostly normalized numbers wearing a different label. That one took an embarrassingly long time to notice.

Readiness probes lie. One engine’s standard “are you accepting connections” check returns success before the database it’s supposed to create actually exists. The probe passed, the first query failed, and we spent a while convinced it was an engine problem.

The most recent one, on a 1536-dimension corpus. The ANN runner holds the whole dataset in memory twice, once in the parent process and again in a forked worker, and the copies aren’t shared. That’s roughly 12 GB for a million embeddings, on top of whatever the server is using, in a container we’d sized for the server alone. The kernel killed the worker. The runner doesn’t check worker exit codes, so it logged “Terminating 1 workers”, exited successfully and wrote no results — which looks exactly like a run that had nothing left to do. Three hours to fail, and it failed silently.

Adding a database

This is the part we cared most about getting right, because the whole point was to avoid rebuilding the apparatus every time somebody ships vector search. Each engine needs:

• a Dockerfile producing a runtime image and a test image from a pinned version
• a config declaring ports, credentials, and which server settings map onto the normalized CPU and memory budget
• a module for the recall and throughput side
• a driver: create index, load, query, filtered query, index size, and the EXPLAIN check

What’s next

Results, published with the manifests and the raw per-configuration records, so you can check them instead of taking our word for it.

Everything is at https://github.com/Percona-Lab/vector-bench harness, drivers, Dockerfiles, docs. If we’re measuring something wrong, or being unfair to an engine you know better than we do, tell us.

The post Benchmarking vector indexes appeared first on Percona.

Benchmarking vector indexes appeared first on MariaDB.org

This performance investigation aims to look into the read/write performance of Percona Server for MySQL 8.4 and how it changed between versions released in 2026:

• 8.4.8-8 released on 12 March 2026
• 8.4.10-10 released on 30 June 2026
• 8.4.11-11 released on 20 August 2026

We want to see if there are improvements in scalability and performance in OLTP read/write operations, where the improvements are most noticeable and how they were achieved. For some readers this material might help with making the decision whether upgrading to a newer version is worth the effort.

An important note is that the new features or security patches will not be taken into consideration.

Measuring Latency (Percentiles) and Resource Utilization (CPU, RAM, I/O) is not in the scope of this post.

2. Configuration and Methodology

The configuration was as follows:

The benchmarks were done across the following dimensions:

One of the points in benchmarking was to create combinations of similar Buffer to Data Ratios, but with the different Database Sizes. This gives us the following possible combinations of innodb_buffer_pool_size and Database Size:

We should be able to see how efficiently the server manages an increasingly larger number of rows while keeping the Buffer to Data Ratio the same.

Execution of the benchmarks was done as follows:

Important Database Configuration options (the actual config files with specific settings for each run can be downloaded from the interactive graphs):

In the high concurrency scenario when all CPU cores are working under maximum load the performance fluctuations might appear out of the ability of a specific CPU crystal to work at a specific sustainable maximum frequency. Intel Xeon Gold 6230 processors installed in the test servers have a base frequency of 2100 MHz and maximum turbo frequency of 3900 MHz. However, such turbo frequency can only be achieved for a short period of time on an isolated core. The load and the heat production of the physical core neighbours limit the frequency of the whole CPU. Some CPU’s were able to hold 2530 MHz on all cores for 20+ hours of intense load, others could only reach 2420 MHz. For consistency of the tests the turbo frequency was capped to 2400 MHz from the beginning on all servers. It helped to eliminate the struggle between turbo mode trying to increase the frequency beyond sustainable levels and the CPU thermal protection bringing the clock down. More stable hardware performance reduced the measurement fluctuations during the benchmark runs regardless if they were done on the same or a different physical server.

3. Results

First, let’s check the I/O bound scenario where the InnoDB Buffer to Data Size is the smallest (1:12).

The graph shows the configurations with innodb_buffer_pool_size=8G and Data Size 96G (or 20M rows per table, 400M rows in total):

[ INTERACTIVE GRAPH ][ TABLE ]

The first thing that catches the eye is the hugely superior performance of the version 8.4.11-11 over 8.4.10-10 and 8.4.8-8 in the high thread numbers. In the situations when the number of physical cores (80) is smaller than the number of threads (128+) the versions 8.4.10-10 and 8.4.8-8 have a steep performance degradation. However, the TPS for 8.4.11-11 keeps growing. This is due to the optimization done to InnoDB LRU pages flushing algorithm. The optimization specifically targeted the scenario when the data size is larger than the available server buffers and the server has many concurrent connections doing random read-write operations. The optimizations in 8.4.11-11 deserve a separate explanation and they will be published in another blog post.

The less noticeable, but important difference can be spotted between the TPS for 8.4.8-8 and 8.4.10-10.

The version 8.4.10-10 shows better performance (especially at the saturation point with 64 threads), which should mostly be attributed to the introduction of Performance Guided Optimization (PGO). 

More information on PGO can be found here:

https://docs.percona.com/percona-server/8.4/pgo.html

With the smaller data and buffer sizes the performance difference gives an almost identical picture:

4G buffer, 48G data [ INTERACTIVE GRAPH ][ TABLE ]

2G buffer, 24G data [ INTERACTIVE GRAPH ][ TABLE ]

Now let’s review what happens with the ratio 1:2.
This time the buffer pool size also plays a more significant role and the performance difference is not characterized by the Buffer / Data size ratio.

With innodb_buffer_pool_size=12G and 24G data size the performance gap between 8.4.11-11 and older versions is still huge as can be seen on the graph:

[ INTERACTIVE GRAPH ][ TABLE ]

However, setting innodb_buffer_pool_size=24G and 48G data size reduces the gap. The superiority of 8.4.11-11 is still visible:

[ INTERACTIVE GRAPH ][ TABLE ]

Moving to innodb_buffer_pool_size=48G and 96G data size shrinks the gap even more:

[ INTERACTIVE GRAPH ][ TABLE ]

In this post we are not going to talk about mechanisms behind shrinking performance gaps in 1:2 Buffer / Data size ratio.

Holding the entire data set in memory is not the most common thing for the database server, but in some cases it happens. Therefore, we are covering such situations as well.

[ INTERACTIVE GRAPH ][ TABLE ]

As the above graph shows, 8.4.10-10 is slightly ahead of 8.4.11-11, but the gap is very small.

This behavior is consistent with other data sizes for fully buffered data:

innodb_buffer_pool_size=64G and 48G Data Size:

[ INTERACTIVE GRAPH ][ TABLE ]

innodb_buffer_pool_size=128G and 96G Data Size:

[ INTERACTIVE GRAPH ][ TABLE ]

Again, we will not go into details about why this happens. Though it is worth noting that both 8.4.10-10 and 8.4.11-11 do better than 8.4.8-8 in all runs and configurations.

The table interpretation of the results is available as well.

4. Comparing with Upstream MySQL 8.4.11.

The performance improvements in Percona Server for MySQL 8.4.11-11 are not a part of the Upstream MySQL 8.4.11. The patch was specifically designed to address the issue of Percona Server being slower than MySQL in I/O bound scenarios.

Also, the patch eliminated the abrupt performance degradation in the higher thread count after reaching the saturation point at 64 threads:

[ INTERACTIVE GRAPH ][ TABLE ]

As the graph shows – Percona Server 8.4.8-8 / 8.4.10-10 was slower than MySQL in lower thread count. Although it was still faster in 128+ threads, the Percona Server was still subject to a substantial slow-down. That is where Percona Server 8.4.11-11 really shines.

However, with the fully buffered data MySQL goes faster than any Percona Server:

[ INTERACTIVE GRAPH ][ TABLE ]

5. Summary

The Performance of the Percona Server 8.4 for MySQL is progressing well from older to newer version offering significant performance improvements especially in the version 8.4.11-11. This version shows very significant improvements in performance on the data sets that require I/O. Also, it outperformed the upstream MySQL 8.4.11.

With fully buffered data sets the version 8.4.10-10 is slightly better than 8.4.11-11. MySQL Server in this case shows the fastest performance.

The PGO had a positive impact demonstrating the version 8.4.10-10 being faster in all tests on all configurations than 8.4.8-8.

The performance depends not only on the ratio between the buffer and the data size, but also on the buffer size.

The post Performance Progression of Percona Server for MySQL 8.4 appeared first on Percona.

Performance Progression of Percona Server for MySQL 8.4 appeared first on MariaDB.org

The posts are also honest about the cost. The work is experimental. The worst bug found was silent data loss on every replicated transaction: the two-phase commit code for mixed-engine transactions dropped prepared DuckDB transactions before they committed, so replication reported success while no rows persisted. The test harness caught it and it was fixed. The row-by-row applier also cannot keep up with bulk writes on the primary.

I want to add a second road to the same place, one that changes nothing inside mysqld. Disclosure first: I build dbtrail, the Apache 2.0 open source tool used below, so read this as one more point in the design space, from someone who is not neutral. Everything here ran on stock Percona Server 8.0 from Docker Hub and stock DuckDB from Homebrew, and the commands at the end reproduce all of it.

Your binlog is already a columnar feed

The storage engine approach puts the column store inside the server, so the server must cooperate: a patched build, a new engine in the commit path, replica tables created by hand with ENGINE=DuckDB. The other road starts from what MySQL already provides. With binlog_format=ROW and binlog_row_image=FULL, the binary log is a complete, ordered feed of every row change, with full before and after images. Any process can consume that feed as a replication client, the way a replica does, and install nothing on the server.

dbtrail is that process. It does two things with the feed:

1. It keeps a short, searchable window of recent events in a plain MySQL table, for investigation and recovery.
2. It moves closed hours out to Parquet files, the open columnar format every analytical engine can read. DuckDB reads it natively.

The pipeline is mysqld -> binlog -> Parquet -> DuckDB. No patched server, no plugin, nothing new in the commit path. And because the feed is history rather than current state, dbtrail can answer a question no replica can: what did this row look like before, and how do I undo what happened to it?

The demo

One container for the source, stock image, with the settings any change-capture consumer needs (plus one line to keep the demo’s own index out of the binlog, since source and index share one server here):

docker run -d --name demo -p 13310:3306 -e MYSQL_ROOT_PASSWORD=demo 
 percona/percona-server:8.0 
 --gtid-mode=ON --enforce-gtid-consistency=ON 
 --binlog-rows-query-log-events=ON --binlog-ignore-db=bintrail_index

binlog_format=ROW and binlog_row_image=FULL are already the 8.0 defaults. The schema is a small shop: customers, orders, order_items. Three short commands point dbtrail at it (init, snapshot, stream; they are in the last section), and from there the work happens in dbtrail’s web console. Then I replayed a five-hour synthetic workload: 3,265,000 row events in all. 2.7 million INSERTs, 510 thousand UPDATEs, 55 thousand DELETEs. (A tip for demo builders: SET TIMESTAMP = in the writing session backdates the binlog event timestamps, so a fast replay spreads across past hours and you can watch retention behave as it would in real life.)

The console’s Status view answers the first question an operator should ask of any change-capture pipeline: did we miss anything?

The green banner is a verdict, not a guess: dbtrail tracks continuity across the captured range, and it says plainly that this is not a liveness check. On my MacBook the capture stream held about 18,000 events per second while it tailed the binlog. That answers the applier problem from the first post: dbtrail does not push rows through the storage engine API one at a time, it batches them into an index, so bulk writes on the primary are absorbed rather than queued.

From hot partitions to Parquet

The index table is range-partitioned by hour. dbtrail archives each closed hour to zstd-compressed Parquet, checks the result, and only then drops the partition. Retention on the expensive tier becomes hours, not months. The demo moved all 3.26 million events out in 14 seconds, and the Status view above already showed the result: 7 archive files, 26 MB. Here is that number next to what the same events cost in InnoDB:

Do not take that 100x as a general truth. Synthetic demo data repeats itself and compresses far too well, and the InnoDB figure includes the primary key and secondary indexes that make the hot tier searchable. On production data expect about one order of magnitude. The direction matches what Evgeniy measured at 500 GB, where the DuckDB engine held TPC-H in 26 percent of the raw CSV size and InnoDB needed 135 percent. Column formats fit this data. Row formats do not.

One mydumper pass adds the state side: a baseline snapshot of the tables themselves, also stored as Parquet. For the demo shop that was 2.6 million rows in 8 MB, done in under four seconds. dbtrail uses baselines to rebuild full tables and single rows at a point in time; here they also give DuckDB something current to query.

Stock DuckDB, no plugins

The archive is just files, so the query engine does not have to be dbtrail. One command, bintrail views (bintrail is the name of dbtrail’s CLI binary), writes a single SQL file of DuckDB view definitions over the layout it knows about: an events view across every archived partition, and one state__

bintrail views --index-dsn "$IDX" --baseline-dir /data/baselines --out views.sql
duckdb
D .read views.sql

That is the whole integration. dbtrail never opens DuckDB and never runs what it prints; the file is plain SQL you can read before you use it. From there, use any DuckDB you like: the CLI, a notebook, a BI tool’s connector. Same laptop, same data, same queries on both engines:

Now the fine print, because a benchmark without it is just marketing. The chart has two groups. The first group queries the event history: those scans hit a 2.8 GB index with the container’s stock 128 MB buffer pool, so InnoDB paid for disk reads, and a bigger pool narrows that gap. The second group runs over current state and is the fair comparison: I grew the pool to 4 GB, both tables sat fully in cache, and the 11x and 20x that remain come from the design of the engine, not from the disk. Both engines returned the same results, down to the last decimal. That is a free integrity check: two independent systems read the same history and agreed.

These are laptop numbers, and I did not run TPC-H. For the ceiling of what columnar execution does at 500 GB, read Evgeniy’s second post. My point is the floor: 2.8 GB of freshly rotated history sat on my laptop as 26 MB of open files, and a stock engine answers questions over them in tens of milliseconds. Nobody patched anything to get here.

The part a replica cannot do

A DuckDB replica holds current state. dbtrail holds what happened. This is the same data the analytics just scanned, now in the console’s Events view, filtered to one order:

Three events tell the row’s whole story: created, cancelled, deleted, each with its GTID and the connection that did it. The expanded UPDATE shows the full before and after images dbtrail keeps for every change.

Here is the detail that surprises people: when I took that screenshot, the MySQL side of the index held zero rows. Rotation had dropped every partition. The console read the answer from the Parquet tier in under 200 milliseconds. The Undo button works from there too:

One click wrote reversal.sql: a script that puts the row back exactly as it was, tagged with the GTID it reverses. Nothing runs on its own; you read the script, then you apply it. The same Restore view takes a whole table and a time window: the demo’s 5,000 deleted orders came back as 5,000 INSERT statements, generated in 0.2 seconds, from files, while the database that held that history no longer existed.

A column store fed by your binlog does not have to be a replica. It can be a time machine.

Trade-offs, honestly

Neither road wins outright.

Where the storage engine is better: its DuckDB copy is always current, and it speaks MySQL protocol. A DuckDB engine on a replica runs seconds behind the primary, and existing BI tools connect to it unchanged. dbtrail’s capture also runs seconds behind, and its console and CLI query that fresh index directly. What waits is the copy DuckDB reads: the events view covers the hours already rotated to Parquet, and the state views show the latest baseline, so that side is as current as your rotation and baseline schedule. If you need a BI tool on MySQL protocol reading a complete, current copy, Evgeniy’s architecture, or a product like HeatWave, aims at exactly that.

Where staying outside the server is better: risk and history. The hardest place to change a database is the commit path, and a storage engine lives there. The 2PC bug shows the kind of failure that layer produces, and it took a dedicated test harness to find it, because it was silent. A replication client cannot corrupt a commit. Its failure modes are lag and gaps, and dbtrail reports gaps as a first-class verdict rather than hoping. History is the other half: audit, point-in-time rebuilds, and row-level undo all come from the same files the analytics read.

Shared constraints. Both roads need ROW format with FULL row images. Both need primary keys to apply or reverse UPDATEs and DELETEs. Both leave the source of truth in InnoDB, untouched. On maturity: the engine posts describe an experiment and say so; dbtrail is released and versioned (v0.65.0 as I write this) but young, and you should question my numbers the same way I question everyone else’s.

Both posts and this one agree on the base facts: row stores are the wrong shape for analytical scans, DuckDB fits MySQL-shaped data well, and the open question is where the column store should live. Evgeniy shows what you gain when the server cooperates. dbtrail shows what you get when you leave the server alone.

Reproduce it

dbtrail binaries are on the releases page; DuckDB comes from your package manager. The web console ships as its own binary, bintrail-console, and can also run capture and console together as one daemon (bintrail-console watch).

# 1. A source with binlogs (ROW + FULL are 8.0 defaults)
docker run -d --name demo -p 13310:3306 -e MYSQL_ROOT_PASSWORD=demo 
 percona/percona-server:8.0 --gtid-mode=ON --enforce-gtid-consistency=ON 
 --binlog-rows-query-log-events=ON --binlog-ignore-db=bintrail_index

# 2. Create your schema, then point dbtrail at it
bintrail init --index-dsn "$IDX"
bintrail snapshot --source-dsn "$SRC" --index-dsn "$IDX" --schemas shop
bintrail stream --source-dsn "$SRC" --index-dsn "$IDX" --server-id 4444 --schemas shop &
bintrail-console serve --index-dsn "$IDX" --baseline-dir /data/baselines &

# 3. Run your workload, then tier the history out and take a baseline
bintrail rotate --index-dsn "$IDX" --retain 7d --archive-dir /data/archives
bintrail dump --source-dsn "$SRC" --output-dir /data/dump --schemas shop
bintrail baseline --input /data/dump --output /data/baselines

# 4. Hand the whole thing to DuckDB
bintrail views --index-dsn "$IDX" --baseline-dir /data/baselines --out views.sql
duckdb -c ".read views.sql" 
 -c "SELECT table_name, event_type, COUNT(*) FROM events GROUP BY 1,2;"

If you try dbtrail, tell me where it fails as much as where it works well. Issues and pull requests are open at github.com/dbtrail/dbtrail, and I am around in the Percona Community Slack.

DuckDB Speed on MySQL with dbtrail, Without a New Storage Engine appeared first on MariaDB.org

This shift highlights the combination of PostgreSQL and ClickHouse as a compelling solution for modern data platforms, with Postgres serving as the authoritative transactional system, and ClickHouse operating as a high-performance analytical platform for processing massive datasets in real time. Together, they function as complementary components connected through continuous Change Data Capture (CDC) synchronization. Before we get into the common architectural patterns, let’s briefly look at why Postgres + ClickHouse.

Why Postgres + ClickHouse?

PostgreSQL excels at Online Transaction Processing (OLTP). Utilizing mature ACID compliance, MVCC, and advanced indexing, it serves as the operational system of record for managing concurrent transactions like accounts, finance, and inventory.

In contrast, analytical workloads like BI dashboards and fraud detection require scanning millions or billions of historical records. Running these massive sequential scans continuously on a production OLTP system increases CPU, memory, and latency, ultimately degrading application performance.

ClickHouse solves this issue as a column-oriented Online Analytical Processing (OLAP) database designed for rapid queries over massive datasets. Instead of replacing PostgreSQL, ClickHouse complements it by offloading complex analytical processing.
Consequently, architectural focus has shifted from choosing between the two platforms to determine how they can work together effectively. This reflects a trend toward polyglot persistence, where specialized databases collaborate to handle distinct transactional and analytical workloads.

Pattern 1: Postgres to ClickHouse Real-Time Analytics

The widely adopted PostgreSQL and ClickHouse architecture uses PostgreSQL as the transactional source of truth while continuously replicating data to ClickHouse for analytical processing. This separates workloads without complex ETL pipelines, allowing applications and BI tools to query ClickHouse directly. Because ClickHouse is optimized for large scans and aggregations, dashboards load significantly faster while keeping production PostgreSQL tables responsive and isolated from heavy reporting risks.

This approach is ideal for real-time systems like SaaS, fintech, and IoT platforms, where fast dashboard updates are a key part of the product experience.

Architecture Overview

In a typical PostgreSQL and ClickHouse deployment, applications run transactional operations on PostgreSQL to ensure data consistency. Committed transactions are then replicated to ClickHouse via a Change Data Capture pipeline, enabling analytical queries to execute independently from the OLTP workload.

By separating the transaction plane from the analytics plane, organizations allow each database to focus on the workload for which it was designed, improving scalability, reducing resource contention, and simplifying performance tuning.

Keeping Analytics in Sync with Change Data Capture (CDC)

This architecture uses Change Data Capture (CDC) to continuously synchronize transactional changes from PostgreSQL to ClickHouse. By capturing inserts, updates, and deletes directly from PostgreSQL’s Write-Ahead Log (WAL) via logical decoding and replication, CDC avoids periodic ETL jobs, minimizing latency for near real-time operational dashboards and analytics.

Enabling this requires configuring the correct PostgreSQL WAL level and exposing individual tables via publications to stream changes.

ALTER SYSTEM SET wal_level = logical;

CREATE PUBLICATION app_events_pub
FOR TABLE
    orders,
    order_events,
    account_events;

CDC services (such as ClickHouse, ClickPipes, or PeerDB) continuously replicate PostgreSQL changes to ClickHouse, reducing data delays and operational overhead compared to traditional batch ETL.

However, monitoring operational factors like replication slots, WAL retention, schema changes, connectivity, backfills, and replication lag is essential, making latency a critical component of real-time dashboard SLOs.

Operational Readiness Checklist

Pre-production deployment of PostgreSQL alongside ClickHouse requires robust operations: identifying authoritative source tables, validating CDC pipelines, and setting replication lag alerts before dashboards rely on the data.

Schema design must transform normalized PostgreSQL tables into denormalized ClickHouse models, defining clear update/delete semantics for append-optimized engines like ReplacingMergeTree. Independently test backfills and run regular reconciliation jobs to detect sync issues early.

Finally, set clear data freshness objectives and monitor metrics like ingestion throughput, WAL growth, slot utilization, and latency, treating CDC as a production service to ensure a reliable platform.

Pattern 2: Hot / Cold Time-Series Data

In this architecture, PostgreSQL stores only recent operational data for transactional processing, while ClickHouse holds the complete historical record. A continuous CDC pipeline synchronizes changes into ClickHouse, enabling PostgreSQL to safely expire older data after verified replication.

Time-Series Data Lifecycle

Combining PostgreSQL and ClickHouse separates operational data from long-term analytical data. Organizations can store only day-to-day operational records in PostgreSQL, while continuously archiving historical data in ClickHouse via a Change Data Capture (CDC) pipeline.

In a typical deployment, PostgreSQL serves as the system of record for transactional workloads, storing recent data, e.g., the last 30 days, to support low-latency OLTP operations. Simultaneously, committed transactions are streamed to ClickHouse via logical replication and a CDC connector such as Debezium, PeerDB, or native decoding. ClickHouse maintains a complete historical archive optimized for analytical queries without affecting operational database performance.

Pattern 3: Federated Query with pg_clickhouse

Instead of forcing applications and BI tools to communicate directly with ClickHouse, PostgreSQL remains the primary SQL endpoint. The pg_clickhouse extension transparently pushes supported analytical queries to ClickHouse while preserving PostgreSQL compatibility.

Federated Query Execution Flow

Upon reaching PostgreSQL, a query is processed by the parser and query planner. The pg_clickhouse extension then automatically determines if it is a transactional (OLTP) or analytical (OLAP) workload. This routing happens before execution and is completely transparent to the application.

OLTP queries, such as inserting orders or retrieving records by primary key, execute locally within PostgreSQL to maintain ACID guarantees, MVCC concurrency control, and low-latency transaction processing.

Analytical queries, like large aggregations or historical reporting scanning millions of rows, are forwarded to ClickHouse, leveraging its columnar storage engine, vectorized execution, and compression for significantly faster performance.

Pattern 4: Embedded Analytics in SaaS Applications

Customer-facing dashboards require analytics to become part of the production application itself. Every customer request may trigger analytical queries while transactional operations continue independently. This makes CDC freshness, query latency, and workload isolation part of the application’s reliability requirements.

Embedded Analytics Operational Components

While PostgreSQL and ClickHouse serve different workloads, the success of a hybrid analytics platform depends on the operational components that ensure data consistency, low latency, and service reliability. The CDC pipeline is only one part of the architecture and operators must also continuously monitor the health of the entire data flow.

End-to-end observability is critical for production deployments. Monitoring should include PostgreSQL replication health, CDC connector status, ClickHouse ingestion throughput, query performance, storage utilization, and dashboard latency. A centralized monitoring platform enables operators to correlate issues across the entire pipeline, reducing mean time to detection (MTTD) and mean time to recovery (MTTR).

Pattern 5: Hybrid

Many production environments combine three specialized database platforms. PostgreSQL serves as the transactional system of record for business-critical operations like user management, orders, accounts, and billing. Its ACID compliance, MVCC model, and mature ecosystem suit Online Transaction Processing (OLTP) workloads requiring strong consistency.

Meanwhile, TimescaleDB handles operational time-series workloads like application metrics, IoT readings, and telemetry. Hypertables, native compression, continuous aggregates, and automated retention enable efficient storage and querying while maintaining full PostgreSQL compatibility.

For analytical processing, ClickHouse offers a column-oriented database optimized for Online Analytical Processing (OLAP). It runs complex queries across billions of rows for dashboards, BI, and trend analysis. Isolating analytical workloads prevents reports and insights from impacting operational application performance.

Postgres vs. TimescaleDB vs. ClickHouse Decision Tree

To optimize performance, scalability, and operational efficiency, a simple decision process helps determine whether PostgreSQL, TimescaleDB, or ClickHouse is the best fit for a particular use case.

Operating Postgres + ClickHouse in Production

Previously, we explored how PostgreSQL and ClickHouse complement each other through real-time operational analytics, hot/cold storage, and federated queries using pg_clickhouse. However, successfully operating them in production requires understanding Change Data Capture (CDC) behavior under failure conditions, monitoring health, preparing runbooks, and establishing clear ownership across all layers. I’ll look at the operational perspective, examining how to keep the architecture healthy, diagnose common problems, and run a reliable production platform.

Native CDC for Operators

Change Data Capture (CDC) is the foundation of PostgreSQL and ClickHouse synchronization, ensuring ClickHouse operates as a near-real-time analytical platform rather than an outdated copy. For operators, understanding its internal mechanics is essential.

In PostgreSQL, CDC utilizes the Write-Ahead Log (WAL), logical decoding, and logical replication instead of scheduled ETL jobs. PostgreSQL continuously records committed transactions in the WAL, which logical replication decodes into inserts, updates, and deletes. The process starts with an initial snapshot of existing data before transitioning into continuous streaming, keeping ClickHouse perfectly synchronized.

While modern services like ClickPipes and PeerDB simplify deployment over traditional Kafka- or Debezium-based architectures, native CDC still requires operational responsibility. Complexity merely shifts to monitoring, validation, and operational governance.

Crucially, operators must monitor logical replication slots. If a CDC consumer stops, PostgreSQL retains unconsumed WAL files, causing uncontrolled disk growth that can exhaust storage and disrupt the primary database.

A useful operational query for monitoring replication slots is shown below:

SELECT
    slot_name,
    plugin,
    active,
    restart_lsn,
    confirmed_flush_lsn
FROM pg_replication_slots;

This information allows operators to verify whether replication slots remain active and downstream consumers acknowledge changes.

Distinguishing between the initial snapshot and continuous replication is critical. Initial snapshots transfer gigabytes or terabytes of data, where replication lag is expected and should not trigger alerts. Afterward, continuous replication must meet defined freshness objectives, limiting delay to seconds or minutes.

ClickHouse managed CDC services require direct PostgreSQL connectivity and do not support proxy layers like PgBouncer, Amazon RDS Proxy, or Supabase Pooler. This must be considered during network, firewall, and infrastructure deployment.

Operational Failure Modes

The primary challenge in PostgreSQL and ClickHouse architectures lies in system interactions, making an effective troubleshooting strategy vital when incidents span multiple layers.

The most frequent operational issue is CDC (Change Data Capture) lag, which leads to stale dashboards and visible delays for users even if PostgreSQL remains healthy. Operators must monitor a range of metrics, including replication lag, slot status, WAL generation rates, ClickHouse throughput, and end-to-end freshness, rather than just basic database health.

Schema evolution also presents difficulties, as application releases regularly modify PostgreSQL tables. Because ClickHouse schemas are denormalized for analysis, schema updates require precise transformations to avoid pipeline interruptions or data gaps.

Furthermore, updates and deletes require deliberate management. While PostgreSQL modifies rows directly, ClickHouse is built for append-heavy workloads. Managing updates demands strategies like ReplacingMergeTree, version columns, or deduplication, while deletes rely on tombstones, soft-delete flags, or scheduled merges.

Query routing errors frequently impact performance; analytical queries on PostgreSQL exhaust resources, whereas point lookups on ClickHouse introduce latency. Federated query setups add further complexity, as performance hinges on whether execution is successfully pushed to ClickHouse or falls back to PostgreSQL.

Finally, security and governance grow more complex because user accounts, roles, authentication, and auditing differ between the platforms. Replicated analytical data often requires distinct access controls, encryption, and logging compared to the source transactional system.

Hybrid Operations with ClusterControl

As organizations adopt specialized databases like PostgreSQL, ClickHouse, TimescaleDB, Redis, Valkey, MongoDB, MySQL, and MariaDB, operational complexity grows. Rather than managing each technology independently, platform teams require unified tooling to support these heterogeneous environments.

ClusterControl fits this strategy by delivering unified lifecycle management, including deployment, monitoring, backup, recovery, and automation, for multiple open-source databases across on-premises and cloud environments. As PostgreSQL and ClickHouse architectures grow more common, its operational management extends well beyond simple server provisioning.

PostgreSQL management involves deployment automation, high availability, backups, PITR, replication monitoring, and upgrade planning. ClickHouse adds backup strategies, merge monitoring, storage capacity planning, and query optimization. The connecting CDC pipeline also requires production-level monitoring, health checks, and incident response.

Support teams must prepare operational runbooks before production deployment. These should document reference architectures, failure modes, CDC troubleshooting, reconciliation workflows, schema migrations, and team escalation paths.

Recommended Reference Architectures

Although PostgreSQL and ClickHouse can be combined in numerous ways, several architectural patterns have consistently emerged across production deployments.

Architecture A: SaaS Operational Analytics

This architecture positions PostgreSQL as the transactional system of record while ClickHouse powers customer-facing dashboards through near-real-time CDC replication. Optionally, pg_clickhouse provides SQL compatibility for existing applications. This pattern is particularly suitable for SaaS platforms, product analytics, billing systems, and customer usage reporting.

Architecture B: Hot / Cold Time-Series Data

Recent operational data remains inside PostgreSQL while historical records migrate into ClickHouse after successful replication. Data expiration is governed by replication watermarks rather than fixed retention schedules, ensuring historical data remains protected.

Architecture C: Federated Analytics

Applications continue connecting to PostgreSQL while pg_clickhouse pushes analytical execution into ClickHouse. This minimizes migration effort while improving analytical performance.

Architecture D: Hybrid Cloud Analytics

Transactional PostgreSQL clusters remain within customer-controlled environments while ClickHouse operates as a managed analytical platform in the cloud. Secure connectivity is established through VPNs, private networking, or dedicated links.

Production Readiness Checklist

Before deploying PostgreSQL and ClickHouse into production, organizations should validate both technical implementation and operational preparedness. Successful production environments depend as much on operational discipline as on architectural design.

Before Deployment

• Identify authoritative source-of-truth tables.
• Define which datasets require CDC.
• Enable logical replication.
• Design ClickHouse analytical schemas.
• Define update and delete handling.
• Establish naming conventions.
• Validate firewall and network connectivity.
• Review proxy limitations.
• Plan initial snapshots and historical backfills.

During Rollout

• Execute the initial snapshot.
• Enable continuous CDC.
• Compare row counts and business metrics.
• Benchmark representative analytical queries.
• Test failover scenarios.
• Validate schema migrations.
• Verify dashboard freshness.
• Ensure retention policies do not remove data prematurely.

After Go-Live

• Monitor replication lag continuously.
• Observe WAL growth.
• Monitor ClickHouse insert throughput.
• Watch merge activity.
• Reconcile PostgreSQL and ClickHouse data regularly.
• Review schema drift after every application release.
• Maintain operational documentation and escalation procedures.

Conclusion

Modern data platforms use specialized architectures where each database handles specific workloads. PostgreSQL serves as a reliable transactional system of record, while ClickHouse enables high-performance real-time analytics without impacting transactional performance.
Operating this architecture requires managing Change Data Capture, replication health, schema evolution, and data reconciliation. In hybrid database environments, platforms like

ClusterControl provide valuable unified management across multiple database technologies.
Ultimately, the future involves combining both technologies into a cohesive platform. Organizations that invest in both the architecture and its supporting operational processes will ensure long-term scalability and production reliability.

Install ClusterControl and try Postgres and ClickHouse free for 30 days

Script Installation Instructions

The installer script is the simplest way to get ClusterControl up and running. Run it on your chosen host, and it will take care of installing all required packages and dependencies.

Offline environments are supported as well. See the Offline Installation guide for more details.

On the ClusterControl server, run the following commands:

wget https://severalnines.com/downloads/cmon/install-cc
chmod +x install-cc
sudo ./install-cc     # omit sudo if you run as root

After the installation is complete, open a web browser, navigate to https:///, and create the first admin user by entering a username (note that “admin” is reserved) and a password on the welcome page. Once you’re in, you can deploy a new database cluster or import an existing one.

The installer script supports a range of environment variables for advanced setup. You can define them using export or by prefixing the install command.

See the list of supported variables and example use cases to tailor your installation.

The post Postgres + ClickHouse Architectural Patterns appeared first on Severalnines.

Postgres + ClickHouse Architectural Patterns appeared first on MariaDB.org

Continue reading “MariaDB Server 12.3, 11.8, 11.4 and 10.11 – Q3 2026 Maintenance Releases, and Goodbye 10.6”

MariaDB Server 12.3, 11.8, 11.4 and 10.11 – Q3 2026 Maintenance Releases, and Goodbye 10.6 appeared first on MariaDB.org

MariaDB Server 12.3, 11.8, 11.4 and 10.11 – Q3 2026 Maintenance Releases, and Goodbye 10.6 appeared first on MariaDB.org

Continue reading “MariaDB Contribution Statistics, January-June 2026”

MariaDB Contribution Statistics, January-June 2026 appeared first on MariaDB.org

MariaDB Contribution Statistics, January-June 2026 appeared first on MariaDB.org

Source

MariaDB Community Server Q3 2026 maintenance releases appeared first on MariaDB.org

Oracle Exadata cost optimization has two honest paths. The first is to keep Exadata and cut what you pay for it: prune unused options, enable fewer cores, and negotiate before the 19c Extended Support uplift lands. The second, and the one this MinervaDB whitepaper engineers in detail, is a staged data transformation onto a fully open source data infrastructure stack — PostgreSQL 18 for OLTP, ClickHouse 26.3 LTS for analytics, Apache Kafka 4.3 with Debezium 3.6 for change data capture, and Valkey 9.1 for caching — that removes the recurring license and support line entirely while holding performance, scalability, availability, and reliability at Exadata-class SLOs.

The finding, stated up front: on a representative Exadata X11M quarter-rack estate, the annual software support bill alone (licenses at 22% of net) exceeds the five-year hardware plus subscription-support cost of the replacement open source stack, and the performance characteristics that justify Exadata — Smart Scan, Storage Indexes, RAC, Hybrid Columnar Compression — each have a measured open source equivalent when the workload is routed to the right engine rather than forced through one. The rest of this paper shows the arithmetic, the architecture, the code, and the failure modes.

1. Where Oracle Exadata spend actually goes

Most Exadata cost optimization conversations start with the rack. That is the wrong place to look. The rack is a one-time capital line; the recurring line is software licensing and its 22% annual support, and on Exadata that line is amplified by three mechanisms: every enabled database-server core is licensed at the Oracle core factor, the storage tier carries its own per-disk software license, and the features that make Exadata fast (RAC, Partitioning, Advanced Compression, In-Memory) are separately licensed options rather than part of Enterprise Edition.

The list prices below are taken from Oracle’s published Technology Global Price List and Engineered Systems Price List (June/August 2026 editions). Real contracts carry discounts, frequently in the 40–70% range; the ratios between line items are what matter for the model, and the support percentage applies to net, not list.

Apply the arithmetic to a quarter rack. Each X11M database server ships with two 96-core AMD EPYC processors; with capacity-on-demand a typical estate enables 64 cores per server, so 128 enabled cores × Oracle’s 0.5 core factor for AMD EPYC = 64 processor licenses (verify against the current Oracle Processor Core Factor Table at engagement time).

A “standard Exadata option set” of EE + RAC + Partitioning + Advanced Compression + Diagnostics/Tuning is $106,000 per processor at list, so 64 processors is $6.78M list and $1.49M per year in support. Add 36 storage-server disk licenses ($360,000 list, $79,200 per year support) and hardware support, and the recurring line on an illustrative quarter rack lands around $1.63M per year at list, roughly $0.65–0.98M per year after a typical 40–60% discount. Over five years that recurring line, not the rack, is the number to optimize.

Two further pressures make 2026–2027 the decision window. Oracle Database 21c reaches end of support on 31 July 2027 with no Extended Support, and 19c Premier Support ends 31 December 2029 with Extended Support carrying a +10% year-one and +20% subsequent-year uplift on the support fee (see the Oracle Database lifecycle tracker; re-verify dates against MOS note 742060.1 before quoting them in a contract). Estates that stay on Exadata will pay more to stand still.

2. Oracle Exadata cost optimization in place (what to do first)

MinervaDB is vendor-neutral, and that cuts both ways: not every Exadata estate should leave. Before any migration business case, run the four measures below. They are reversible, they require no application change, and on several engagements they have cut the recurring line by 25–40% (illustrative range; your figure comes from your own DBA_FEATURE_USAGE_STATISTICS).

2.1 Measure option usage before you pay for it

Licensing follows usage, and Oracle’s own catalog tells you what is used. This is the first query on every MinervaDB Oracle takeover, and it is the evidence for pruning options at renewal. Note the licensing gate: querying DBA_HIST_* or V$ACTIVE_SESSION_HISTORY on an estate without Diagnostics Pack is itself a license event, so check CONTROL_MANAGEMENT_PACK_ACCESS first.

-- 1. Licensing gate: what packs is this instance allowed to use?
SHOW PARAMETER control_management_pack_access;

-- 2. Which separately licensed options are actually in use?
SELECT
    u.name                                   AS feature_name,
    u.detected_usages,
    u.currently_used,
    TO_CHAR(u.first_usage_date, 'YYYY-MM-DD') AS first_used,
    TO_CHAR(u.last_usage_date,  'YYYY-MM-DD') AS last_used
FROM dba_feature_usage_statistics u
WHERE u.name IN (
        'Real Application Clusters (RAC)',
        'Partitioning (user)',
        'Advanced Compression',
        'HeapCompression',
        'Hybrid Columnar Compression',
        'In-Memory Column Store',
        'Active Data Guard - Real-Time Query on Physical Standby',
        'Oracle Multitenant',
        'Automatic Workload Repository',
        'SQL Tuning Advisor',
        'Exadata'
      )
  AND u.dbid = (SELECT dbid FROM v$database)
ORDER BY u.currently_used DESC, u.detected_usages DESC;

-- 3. Enabled cores that drive the processor-license count
SELECT
    cpu_count_current,
    cpu_core_count_current,
    cpu_socket_count_current
FROM v$license;

An option showing currently_used = FALSE and last_used older than the current support term is a renewal negotiation item. In-Memory and Active Data Guard are the most common finds; Tuning Pack is the second.

2.2 Reduce enabled cores with capacity-on-demand

Exadata licenses follow enabled cores, not installed cores. Right-size from DBA_HIST_SYSMETRIC_SUMMARY (Host CPU Utilization (%), p95 over 90 days, only if Diagnostics Pack is licensed — otherwise STATSPACK or OS-level sar data) and reduce enabled cores through OEDA/dbmcli during a maintenance window. Every two cores removed on AMD EPYC saves one processor license of support every year. Blast radius: CPU headroom during peak; rollback: re-enable cores (a reboot of the database server, not a license event, but confirm with your Oracle account team in writing before changing core counts).

2.3 Offload analytics before you offload anything else

The most expensive SQL on most Exadata estates is reporting: long-running aggregations that make Exadata’s Smart Scan look heroic precisely because they are being run on a row-store OLTP engine. Moving those queries to ClickHouse via CDC (Section 5.2) shrinks the Exadata CPU footprint, which in turn reduces enabled cores, which reduces support. This is also the first, lowest-risk phase of the full migration, so nothing done here is wasted if the estate later leaves Exadata entirely.

2.4 Time the negotiation to the lifecycle

Do not renew a multi-year support term that runs across the 19c Extended Support boundary without pricing the uplift in. A credible, funded open source migration plan is the most effective negotiating asset an Oracle customer has; the sections below are that plan.

3. Target open source data infrastructure stack

Exadata is one engine asked to do three jobs: transactional OLTP, analytical reporting, and hot-key lookups that in practice live in the buffer cache and result cache. The open source design principle is the opposite: route each workload class to the engine built for it, connect them with change data capture, and operate all of it with the same observability and SRE discipline. Every component below is 100% open source, community-supported, and runs on commodity x86/ARM hardware or any cloud.

3.1 Workload-to-engine mapping

The mapping is honest about one thing: PostgreSQL alone does not replace Exadata for an estate that mixes heavy analytics with OLTP. The analytics tier is what makes the performance claim hold, and it is why the architecture is a stack rather than a database swap.

4. Options considered and rejected

A migration recommendation is only credible if the alternatives were evaluated on their merits. These were, and each is a legitimate choice for a different constraint set.

Oracle Autonomous Database / Exadata Database Service on OCI or Database@Azure/AWS/Google. Genuine strengths: RAC and Data Guard semantics preserved, patching automated, and the @-cloud variants burn down hyperscaler commitments. Rejected for the cost objective because the license line moves rather than disappears (BYOL or License Included, ECPU-metered), Autonomous removes SYSDBA, RMAN, and OS access, and the exit path from Autonomous is logical-only (Data Pump or GoldenGate). For an estate whose strategic direction is off-Oracle, every Oracle cloud service deepens the moat.

EDB Postgres Advanced Server (EPAS) with Oracle-compatibility mode. Genuine strengths: packages, PL/SQL dialect, OCI-compatible connector — the shortest path when the PL/SQL estate is large and the rewrite budget is the binding constraint. Rejected as the default because it exchanges Oracle lock-in for EDB lock-in and reintroduces a per-core subscription. MinervaDB supports EPAS estates without ideology; we recommend it only against a named PL/SQL-volume requirement and we say so in writing.

PostgreSQL-only with Citus columnar or TimescaleDB for analytics. Genuine strengths: one engine to operate, one skill set, transactional consistency across OLTP and reporting. Rejected for estates with true Exadata-class scan volumes because row-store parallelism and columnar access methods in PostgreSQL do not match a purpose-built vectorised MergeTree engine on the billion-row aggregations that justified Exadata in the first place. It remains the right answer for smaller estates whose “analytics” is a few hundred GB of reporting.

Managed cloud DBaaS (Amazon RDS/Aurora, Azure Database for PostgreSQL, Cloud SQL/AlloyDB) as the landing zone. Genuine strengths: operational automation, PostgreSQL 18 availability within days of community GA. Not rejected — the stack in Figure 1 runs on any of them — but the whitepaper models self-managed on commodity hardware or IaaS because that is the configuration with no vendor line at all.

The trade-offs are documented in MinervaDB’s PostgreSQL cloud guide for AWS, GCP, and Azure.

Staying on Exadata with the Section 2 measures. The right answer for estates with deep RAC-dependent ISV applications under change freeze, or where PL/SQL volume exceeds roughly 500k lines with no rewrite budget. Those estates should optimize in place and revisit at the 19c Extended Support boundary.

5. Performance parity engineering

“Without compromising on performance” is a measurable claim or it is marketing, and it is the test every Oracle Exadata cost optimization plan must pass. The measurement frame we use on every Exadata exit is the same: capture the top-N SQL by elapsed time and I/O from AWR (or STATSPACK on unlicensed estates) before migration, map each statement to its target engine, and compare against pg_stat_statements and system.query_log after migration under production-shaped load. Parity is declared per statement class, never in aggregate.

5.1 OLTP on PostgreSQL 18: partitioning, plans, and I/O

Oracle Partitioning is a paid option; PostgreSQL declarative partitioning is not, and since PostgreSQL 17 it supports partition-wise joins and aggregates, identity columns and exclusion constraints on partitioned tables, and partition pruning at execution time. The pattern below converts a typical Oracle range-partitioned order table, retains the Oracle-side NUMBER precision decisions deliberately, and adds pg_partman for the maintenance that Oracle interval partitioning did implicitly.

CREATE TABLE sales.orders (
    order_id          BIGINT GENERATED ALWAYS AS IDENTITY,
    customer_id       BIGINT        NOT NULL,
    order_ts          TIMESTAMP(0)  NOT NULL,   -- Oracle DATE carries time: map to timestamp(0), never date
    status            VARCHAR(16)   NOT NULL,
    amount            NUMERIC(14,2) NOT NULL,   -- NUMBER(14,2) → numeric(14,2); bare NUMBER hot columns → bigint where domain allows
    region_code       CHAR(3)       NOT NULL,
    CONSTRAINT pk_orders PRIMARY KEY (order_id, order_ts)
) PARTITION BY RANGE (order_ts);

-- pg_partman manages monthly partitions and pre-creates 3 months ahead
SELECT partman.create_parent(
    p_parent_table => 'sales.orders',
    p_control      => 'order_ts',
    p_interval     => '1 month',
    p_premake      => 3
);

CREATE INDEX ix_orders_customer_ts
    ON sales.orders (customer_id, order_ts DESC);

-- Empty string ≠ NULL in PostgreSQL. Oracle code that relied on '' IS NULL must be
-- audited; this CHECK makes the migration assumption explicit and testable.
ALTER TABLE sales.orders
    ADD CONSTRAINT ck_orders_status_not_blank CHECK (status  '');

Plan reasoning, not just DDL: the hot OLTP query “recent orders for a customer” must prune to one or two partitions and walk ix_orders_customer_ts. Verify it with EXPLAIN (ANALYZE, BUFFERS) and look for Partitions removed or a plan that lists only the matching child indexes; a plan that appends every partition is a missing pruning predicate on order_ts, the most common week-one regression on Oracle migrations.

EXPLAIN (ANALYZE, BUFFERS, SETTINGS)
SELECT order_id, order_ts, status, amount
FROM sales.orders
WHERE customer_id = 8812931
  AND order_ts >= now() - INTERVAL '45 days'
ORDER BY order_ts DESC
LIMIT 50;

-- Expected shape (trimmed):
-- Limit  (actual time=0.041..0.118 rows=50 loops=1)
--   ->  Merge Append
--         ->  Index Scan Backward using orders_p2026_08_customer_id_order_ts_idx on orders_p2026_08
--               Index Cond: ((customer_id = 8812931) AND (order_ts >= ...))
--         ->  Index Scan Backward using orders_p2026_07_customer_id_order_ts_idx on orders_p2026_07
--   Buffers: shared hit=14
-- Two partitions touched, 14 buffer hits, no heap fetches beyond the index — this is the target shape.

PostgreSQL 18 changes the I/O story that Exadata customers care about most. The new asynchronous I/O subsystem (io_method = worker by default, io_uring on Linux where enabled at build time) lets sequential scans, bitmap heap scans, and VACUUM issue reads ahead of consumption, and the release notes document the supporting changes: skip-scan on B-tree indexes, parallel GIN builds, and pg_stat_io byte-level accounting (PostgreSQL 18 release notes).

The configuration deltas from default that we apply on an Exadata-replacement OLTP node are listed with their reload/restart requirement, per house convention. Proposed values are for a 64 vCPU / 512 GB / NVMe node.

Test every value in staging under replayed production load before applying to production, and keep a robust DR posture (verified pgBackRest restore) before any restart-class change.

5.2 Analytics on ClickHouse: the Smart Scan replacement

The reporting queries that made Exadata Smart Scan indispensable have a consistent shape: filter by time and a low-cardinality dimension, aggregate measures, group by a handful of columns. On a columnar MergeTree table with the right ORDER BY, ClickHouse reads only the granules and columns the query touches, which is the same I/O-avoidance idea Smart Scan implements in storage cells, applied at the storage format instead of the storage hardware. Engine declarations carry full parameter lists — house rule, and the difference between a table that replicates and one that silently does not.

CREATE TABLE sales.orders_local ON CLUSTER 'analytics'
(
    order_id      UInt64,
    customer_id   UInt64,
    order_ts      DateTime('UTC') CODEC(DoubleDelta, ZSTD(3)),
    status        LowCardinality(String),
    amount        Decimal(14, 2)  CODEC(T64, ZSTD(3)),
    region_code   LowCardinality(FixedString(3)),
    _version      UInt64,                       -- Debezium source.lsn / ts_ms, drives ReplacingMergeTree dedup
    _deleted      UInt8 DEFAULT 0,
    INDEX ix_customer customer_id TYPE bloom_filter(0.01) GRANULARITY 4
)
ENGINE = ReplicatedReplacingMergeTree(
    '/clickhouse/tables/{shard}/sales/orders_local',
    '{replica}',
    _version
)
PARTITION BY toYYYYMM(order_ts)
ORDER BY (region_code, status, order_ts, order_id)
TTL order_ts + INTERVAL 18 MONTH TO VOLUME 'cold_s3'
SETTINGS
    index_granularity            = 8192,
    storage_policy               = 'tiered',
    min_bytes_for_wide_part      = 10485760,
    ttl_only_drop_parts          = 1;

CREATE TABLE sales.orders ON CLUSTER 'analytics'
AS sales.orders_local
ENGINE = Distributed('analytics', 'sales', 'orders_local', cityHash64(customer_id));

-- Kafka engine consumer for the Debezium topic (JSON unwrapped by the ExtractNewRecordState SMT)
CREATE TABLE sales.orders_kafka ON CLUSTER 'analytics'
(
    order_id UInt64, customer_id UInt64, order_ts DateTime('UTC'),
    status String, amount Decimal(14,2), region_code String,
    __lsn UInt64, __deleted String
)
ENGINE = Kafka
SETTINGS
    kafka_broker_list        = 'kafka-1:9092,kafka-2:9092,kafka-3:9092',
    kafka_topic_list         = 'pg.sales.orders',
    kafka_group_name         = 'clickhouse-sales-orders',
    kafka_format             = 'JSONEachRow',
    kafka_num_consumers      = 4,
    kafka_max_block_size     = 65536,
    kafka_handle_error_mode  = 'stream';

CREATE MATERIALIZED VIEW sales.orders_mv ON CLUSTER 'analytics'
TO sales.orders_local AS
SELECT
    order_id, customer_id, order_ts, status, amount,
    toFixedString(region_code, 3) AS region_code,
    __lsn                          AS _version,
    if(__deleted = 'true', 1, 0)   AS _deleted
FROM sales.orders_kafka;

Plan reasoning for the reporting query: with ORDER BY (region_code, status, order_ts, order_id), a query filtering on region and a month range touches only the matching partition and the primary-key granules for that region prefix. EXPLAIN indexes = 1 shows the pruning; system.query_log shows the outcome. On ClickHouse 26.3 LTS the evidence pair is EXPLAIN PIPELINE plus system.trace_log; 26.7 adds EXPLAIN ANALYZE.

EXPLAIN indexes = 1
SELECT
    region_code,
    status,
    count()      AS orders,
    sum(amount)  AS revenue
FROM sales.orders FINAL
WHERE region_code = 'APJ'
  AND order_ts >= toDateTime('2026-07-01 00:00:00', 'UTC')
  AND order_ts = now() - INTERVAL 1 DAY
  AND has(tables, 'sales.orders_local')
GROUP BY normalized_query_hash
ORDER BY p99_ms DESC
LIMIT 20;

Two ClickHouse 26.3 LTS specifics matter for an Exadata replacement. First, async_insert became enabled by default in 26.3; on a CDC-fed estate pin async_insert = 0 in the ingestion profile before cutover and re-enable deliberately under Keeper observation, because the change alters part-count and Keeper load characteristics. Second, use FINAL or argMax patterns on ReplacingMergeTree only where the reporting SLA tolerates it; for dashboards, a scheduled OPTIMIZE ... FINAL on closed partitions or a projection is cheaper. The ClickHouse engineering for this tier is delivered by our sister company, ChistaDATA, and scoped through MinervaDB’s ClickHouse consulting practice.

5.3 Hot keys on Valkey 9.1

Oracle’s result cache and KEEP buffer pool absorb the repeated point reads (session state, entitlement lookups, reference data) that would otherwise be latency outliers. Valkey 9.1 — BSD-3-licensed, Linux Foundation governed, with hash-field expiration and multi-database cluster mode since 9.0 (Valkey 9 release blog) — takes that role with write-through invalidation from the CDC stream, so the cache never serves a value newer data has superseded.

import json, os
from confluent_kafka import Consumer
from valkey.cluster import ValkeyCluster

vk = ValkeyCluster(
    host=os.environ["VALKEY_HOST"], port=6379,
    password=os.environ["VALKEY_PASSWORD"], ssl=True,
)

consumer = Consumer({
    "bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"],
    "group.id": "valkey-entitlement-cache",
    "auto.offset.reset": "earliest",
    "enable.auto.commit": False,
})
consumer.subscribe(["pg.sales.customer_entitlements"])

while True:
    msg = consumer.poll(1.0)
    if msg is None or msg.error():
        continue
    row = json.loads(msg.value())
    key = f"ent:{row['customer_id']}"
    if row.get("__deleted") == "true":
        vk.delete(key)
    else:
        # Hash with per-field TTL (Valkey 9.0+): plan fields expire independently
        vk.hset(key, mapping={"tier": row["tier"], "limits": row["limits_json"]})
        vk.hexpire(key, 86400, "limits")
    consumer.commit(msg)

6. Scalability without RAC

RAC scales one database across nodes over shared storage. The open source stack scales each tier by the mechanism that suits its access pattern, and the honest statement is that write scaling for a single PostgreSQL primary is vertical (PostgreSQL 18 on a 2-socket 192-core node comfortably exceeds the enabled-core footprint most Exadata quarter racks license) while read scaling and analytic scaling are horizontal.

Read scaling for Oracle Exadata cost optimization targets: hot standbys behind PgBouncer with hot_standby_feedback = on and a read-routing pool; replication lag from pg_stat_replication is the SLO. Analytic scaling: add ClickHouse shards, rebalance with Distributed table weights, and keep system.parts per-partition counts inside the merge budget; parallel replicas (enable_parallel_replicas) fan a single heavy query across replicas of one shard.

Write scaling beyond a single primary, if the workload truly demands it after measurement (pg_stat_database.xact_commit rate against saturation evidence in pg_stat_activity wait events), is Citus sharding on a tenant or hash key — a step we scope only after vertical headroom is measured, because it changes the data model.

Connection scaling is solved at the pooler: Oracle’s dedicated-server assumptions meet PostgreSQL’s process-per-connection model, and PgBouncer sizing from measured concurrency is part of every migration design, not an afterthought.

[databases]
sales_rw = host=pg-primary.internal port=5432 dbname=sales pool_size=64
sales_ro = host=pg-replicas.internal port=5432 dbname=sales pool_size=128

[pgbouncer]
listen_addr             = 0.0.0.0
listen_port             = 6432
auth_type               = scram-sha-256
auth_file               = /etc/pgbouncer/userlist.txt
pool_mode               = transaction
max_client_conn         = 10000
default_pool_size       = 64
reserve_pool_size       = 16
server_idle_timeout     = 300
max_prepared_statements = 200      ; 1.24+ enables prepared statements in transaction mode
server_tls_sslmode      = verify-full
server_tls_ca_file      = /etc/ssl/certs/internal-ca.pem
stats_period            = 60

7. Availability and reliability engineering

Availability numbers are engineered, not quoted, and Oracle Exadata cost optimization is meaningless if the replacement stack cannot match the protection tier. Exadata estates typically run RAC for instance failure plus Data Guard for site failure; the open source stack reaches the same protection tiers with Patroni for automated failover, synchronous replication for RPO = 0, pgBackRest for point-in-time recovery, and ReplicatedMergeTree with a dedicated Keeper ensemble for the analytics tier. The arithmetic below is what we put in front of a customer before claiming any nines.

scope: sales-pg18
namespace: /minervadb/
name: pg-az-a

restapi:
  listen: 0.0.0.0:8008
  connect_address: pg-az-a.internal:8008

etcd3:
  hosts: etcd-1.internal:2379,etcd-2.internal:2379,etcd-3.internal:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576        # bytes; async standby is never promoted beyond this
    synchronous_mode: quorum                # Patroni 4.x quorum-based synchronous replication
    synchronous_node_count: 1
    failsafe_mode: true                     # keep primary up if DCS is unreachable but members are
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        wal_level: replica
        max_wal_senders: 16
        max_replication_slots: 16
        hot_standby: "on"
        hot_standby_feedback: "on"
        wal_keep_size: 8GB
        archive_mode: "on"
        archive_command: "pgbackrest --stanza=sales archive-push %p"
        restore_command: "pgbackrest --stanza=sales archive-get %f %p"

postgresql:
  listen: 0.0.0.0:5432
  connect_address: pg-az-a.internal:5432
  data_dir: /pgdata/18/main
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: ${PG_REPL_PASSWORD}
    superuser:
      username: postgres
      password: ${PG_SUPER_PASSWORD}
  create_replica_methods:
    - pgbackrest
    - basebackup
  pgbackrest:
    command: /usr/bin/pgbackrest --stanza=sales --delta restore
    keep_data: true
    no_params: true

tags:
  nofailover: false
  noloadbalance: false
  sync_priority: 100        # prefer this node as the synchronous standby when it is a replica

[global]
repo1-type=s3
repo1-s3-bucket=${PGBACKREST_BUCKET}
repo1-s3-endpoint=s3.ap-south-1.amazonaws.com
repo1-s3-region=ap-south-1
repo1-path=/pgbackrest
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=${PGBACKREST_CIPHER_PASS}
repo1-retention-full=4
repo1-retention-diff=14
repo1-retention-archive-type=full
repo1-bundle=y
repo1-block=y
process-max=8
compress-type=zst
compress-level=3
archive-async=y
spool-path=/var/spool/pgbackrest
log-level-console=info

[sales]
pg1-path=/pgdata/18/main
pg1-port=5432
pg1-user=postgres
# 2.59.0 restricts root execution to `restore` by default — run backups as the postgres user.

Reliability discipline is where Exadata-class expectations are actually met: monthly pgbackrest restore --type=time validation on an isolated host, quarterly Patroni switchover drills with timings recorded, quarterly ClickHouse partition-restore drills via clickhouse-backup, and a written escalation matrix. MinervaDB’s 24×7 emergency DBA coverage operates on S1 15-minute response, and the drills are what make that response meaningful.

8. The migration program: assess, convert, replicate, cut over

8.1 Assessment: measure the estate, never guess it

Oracle Exadata cost optimization by migration starts here: effort is measured from the Oracle catalog. The output is an object-count matrix by complexity class and conversion route (automatic / assisted / manual rewrite), and it is the single artifact that turns a whitepaper into a funded program.

-- Object inventory by type
SELECT owner, object_type, COUNT(*) AS objects
FROM dba_objects
WHERE owner IN (${APP_SCHEMAS})
GROUP BY owner, object_type
ORDER BY objects DESC;

-- PL/SQL volume by unit (lines of code drive conversion effort, not table count)
SELECT owner, name, type, COUNT(*) AS loc
FROM dba_source
WHERE owner IN (${APP_SCHEMAS})
GROUP BY owner, name, type
ORDER BY loc DESC;

-- Constructs with no direct community-PostgreSQL equivalent (each is a line item)
SELECT owner, name, type, COUNT(*) AS hits, 'AUTONOMOUS_TRANSACTION' AS construct
FROM dba_source
WHERE owner IN (${APP_SCHEMAS})
  AND UPPER(text) LIKE '%PRAGMA AUTONOMOUS_TRANSACTION%'
GROUP BY owner, name, type
UNION ALL
SELECT owner, name, type, COUNT(*), 'BULK_COLLECT'
FROM dba_source
WHERE owner IN (${APP_SCHEMAS})
  AND UPPER(text) LIKE '%BULK COLLECT%'
GROUP BY owner, name, type
UNION ALL
SELECT owner, name, type, COUNT(*), 'CONNECT_BY'
FROM dba_source
WHERE owner IN (${APP_SCHEMAS})
  AND UPPER(text) LIKE '%CONNECT BY%'
GROUP BY owner, name, type
ORDER BY hits DESC;

-- Feature dependencies: DB links, AQ, VPD, MV fast refresh
SELECT 'DB_LINK' AS dependency, COUNT(*) AS n FROM dba_db_links
UNION ALL SELECT 'AQ_QUEUE',   COUNT(*) FROM dba_queues WHERE owner IN (${APP_SCHEMAS})
UNION ALL SELECT 'VPD_POLICY', COUNT(*) FROM dba_policies WHERE object_owner IN (${APP_SCHEMAS})
UNION ALL SELECT 'MV_FAST_REFRESH', COUNT(*) FROM dba_mviews WHERE owner IN (${APP_SCHEMAS}) AND refresh_method = 'FAST';

8.2 Conversion with ora2pg and orafce

Ora2Pg produces the assessment report (with its own cost-unit estimate per object) and performs schema, data, and a first-pass PL/SQL conversion; orafce supplies the Oracle-compatible functions (NVL, DECODE, ADD_MONTHS, DBMS_OUTPUT, and others) that keep converted code readable. Packages become schemas plus functions; package state becomes session GUCs or a state table; autonomous transactions become dblink or a background-worker pattern, chosen per call site.

ORACLE_DSN        dbi:Oracle:host=exa-scan.internal;sid=SALESPDB;port=1521
ORACLE_USER       ${ORA_MIG_USER}
ORACLE_PWD        ${ORA_MIG_PASSWORD}
SCHEMA            SALES
PG_VERSION        18
TYPE              TABLE,VIEW,SEQUENCE,TRIGGER,FUNCTION,PROCEDURE,PACKAGE,TYPE,PARTITION,MVIEW
EXPORT_SCHEMA     1
DATA_TYPE         DATE:timestamp(0),LONG:text,LONG RAW:bytea,CLOB:text,NCLOB:text,BLOB:bytea,BFILE:bytea,RAW:bytea,ROWID:oid,FLOAT:double precision,DEC:decimal,DECIMAL:decimal,DOUBLE PRECISION:double precision,INT:numeric,INTEGER:numeric,REAL:real,SMALLINT:smallint,BINARY_FLOAT:double precision,BINARY_DOUBLE:double precision,TIMESTAMP:timestamp,XMLTYPE:xml,BINARY_INTEGER:integer,PLS_INTEGER:integer,TIMESTAMP WITH TIME ZONE:timestamp with time zone,TIMESTAMP WITH LOCAL TIME ZONE:timestamp with time zone
PG_NUMERIC_TYPE   1
PG_INTEGER_TYPE   1
DEFAULT_NUMERIC   bigint
USE_ORAFCE        1
PLSQL_PGSQL       1
NULL_EQUAL_EMPTY  0        ; force the '' vs NULL audit instead of masking it
ESTIMATE_COST     1
COST_UNIT_VALUE   5
PARALLEL_TABLES   8
JOBS              8
ORACLE_COPIES     8
DATA_LIMIT        20000
FILE_PER_TABLE    1
OUTPUT_DIR        /migration/sales

ora2pg -c /migration/ora2pg.conf -t SHOW_REPORT --estimate_cost --dump_as_html > /migration/sales/assessment.html
ora2pg -c /migration/ora2pg.conf -t TABLE     -o schema_tables.sql
ora2pg -c /migration/ora2pg.conf -t PACKAGE   -o packages.sql
ora2pg -c /migration/ora2pg.conf -t FUNCTION  -o functions.sql
# Data: COPY pipeline, 8 parallel jobs, indexes and FKs applied after load
ora2pg -c /migration/ora2pg.conf -t COPY -j 8 -J 8

8.3 CDC with Debezium: parallel run and rehearsable cutover

The cutover is where an Oracle Exadata cost optimization program is won or lost, so it must be rehearsable and reversible. Debezium 3.6’s Oracle connector (documentation) captures from LogMiner without a GoldenGate license, or from OpenLogReplicator for lower source overhead on high-redo estates; XStream requires a GoldenGate license and is not used.

The stream lands in Kafka 4.3 (KRaft-only since 4.0) and is applied to PostgreSQL by the JDBC sink connector during the parallel run. After cutover, the same topology is reversed — Debezium’s PostgreSQL connector on pgoutput feeding a JDBC sink into Oracle — and kept warm as the rollback path until sign-off.

{
  "name": "ora-sales-source",
  "config": {
    "connector.class": "io.debezium.connector.oracle.OracleConnector",
    "tasks.max": "1",
    "database.hostname": "exa-scan.internal",
    "database.port": "1521",
    "database.user": "${ORA_CDC_USER}",
    "database.password": "${ORA_CDC_PASSWORD}",
    "database.dbname": "SALESCDB",
    "database.pdb.name": "SALESPDB",
    "topic.prefix": "ora",
    "table.include.list": "SALES.ORDERS,SALES.ORDER_ITEMS,SALES.CUSTOMER_ENTITLEMENTS",
    "database.connection.adapter": "logminer",
    "log.mining.strategy": "online_catalog",
    "log.mining.batch.size.default": "20000",
    "log.mining.transaction.retention.ms": "3600000",
    "lob.enabled": "false",
    "decimal.handling.mode": "precise",
    "time.precision.mode": "adaptive",
    "snapshot.mode": "initial",
    "schema.history.internal.kafka.bootstrap.servers": "kafka-1:9092,kafka-2:9092,kafka-3:9092",
    "schema.history.internal.kafka.topic": "schema-history.ora.sales",
    "heartbeat.interval.ms": "10000",
    "transforms": "unwrap",
    "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
    "transforms.unwrap.delete.handling.mode": "rewrite",
    "transforms.unwrap.add.fields": "op,source.scn,source.ts_ms",
    "key.converter": "io.apicurio.registry.utils.converter.AvroConverter",
    "value.converter": "io.apicurio.registry.utils.converter.AvroConverter",
    "key.converter.apicurio.registry.url": "http://apicurio.internal:8080/apis/registry/v2",
    "value.converter.apicurio.registry.url": "http://apicurio.internal:8080/apis/registry/v2"
  }
}

Source-side prerequisites are the part Oracle DBAs must own: ARCHIVELOG mode, minimal supplemental logging at database level plus ALL COLUMNS on captured tables, and a CDC user granted the LogMiner privileges the Debezium documentation lists. LogMiner load on the source is measurable in V$SESSION and, on licensed estates, in ASH; on a high-redo estate that overhead is the reason to evaluate OpenLogReplicator.

8.4 Cutover runbook (excerpt) with verification and rollback

The full cutover runbook is a versioned MinervaDB deliverable (MDB-RUN-*) with purpose, scope, prerequisites, roles, stepwise commands, expected output, verification after each phase, rollback, and an escalation matrix. The core sequence, with its verification queries, is reproduced here because it is the part readers ask for.

-- GATE 1 (Oracle side, writes already stopped at the application tier): confirm no in-flight transactions
SELECT COUNT(*) AS active_txns
FROM v$transaction;
-- expected: 0

-- GATE 2 (Kafka Connect): confirm the source connector has no lag (consumer group for the sink)
-- kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --describe --group connect-pg-sales-sink
-- expected: LAG column = 0 on every partition

-- GATE 3 (PostgreSQL side): per-table reconciliation against Oracle counts captured at GATE 1
SELECT
    'orders'                         AS table_name,
    COUNT(*)                         AS row_count,
    md5(string_agg(order_id::text || ':' || amount::text, ',' ORDER BY order_id)) AS content_hash
FROM sales.orders
WHERE order_ts >= DATE '2026-01-01';   -- hot-window hash; full-table hashes run in the parallel-run reports

-- GATE 4: promote PostgreSQL to system of record (reverse CDC connector started, applications re-pointed)
-- Verification after cutover:
SELECT slot_name, active, confirmed_flush_lsn,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)) AS reverse_cdc_lag
FROM pg_replication_slots
WHERE slot_name = 'debezium_reverse';
-- expected: active = t, lag in low MB and falling

-- ROLLBACK (any time before sign-off): stop application writes to PostgreSQL, wait for reverse_cdc_lag = 0 bytes,
-- re-point connection strings to the Oracle service, confirm V$TRANSACTION shows application activity, resume.
-- No object is dropped, truncated, or detached at any point in this runbook without a written confirmation gate.

9. Five-year TCO model (illustrative)

Every figure in this table is illustrative and built from the list prices in Section 1 with a stated discount assumption, so that a reader can substitute their own contract values. It deliberately excludes people cost on both sides — an Exadata estate and an open source stack both need a database engineering function, and the honest difference is skills mix, not headcount. It also excludes the one-time migration program, which is estate-specific and comes out of the Section 8.1 assessment.

The order-of-magnitude gap survives any reasonable discount assumption because the open source column has no line that scales with cores.

What the gap buys is not free: it is spent on the migration program, on the skills to operate three engines instead of one, and on the Data SRE discipline that keeps the SLOs honest. For most estates in the 10–100 TB range, the support line alone repays that investment inside the first support renewal cycle — but that claim is only true for your estate once the Section 8.1 assessment and a load-replay test have been run, which is why MinervaDB does not publish a customer number without them.

10. Risks, honest edges, and where this does not apply

PL/SQL volume is the schedule driver. Data size determines the bulk-load window; procedural code determines the program length. Estates above roughly 500k lines of PL/SQL with packages, autonomous transactions, and REF CURSOR-heavy APIs should evaluate EPAS with eyes open, or phase the exit application by application.

Semantics that silently change. Empty string versus NULL, DATE with time component, NUMBER without precision, sequence CACHE behaviour, and case-sensitive identifiers are the correctness classes that survive a passing conversion and fail in production. Each has a mandatory test in the MinervaDB migration test suite; none is optional.

Optimizer differences are the week-two incident source. There are no hints in community PostgreSQL (pg_hint_plan only under governance), cardinality estimation differs, and Oracle’s adaptive plans have no equivalent. Capture AWR top SQL before migration, compare against pg_stat_statements after, and budget tuning time in the stabilization phase.

ISV applications. Packaged applications certified only on Oracle (many ERP, core banking, and telecom billing platforms) cannot be migrated by the database team; the decision belongs to the application roadmap. The Section 2 measures still apply.

Materialized-view fast refresh ON COMMIT, Advanced Queuing, and fine-grained VPD map to application-level patterns (incremental views, pgmq or Kafka, row-level security) rather than one-to-one features; each is a design decision, not a conversion.

What we did not test for this paper. No benchmark numbers are published here because a benchmark without your workload shape, hardware, and configuration deltas is noise. Where a MinervaDB engagement includes a load-replay comparison, the methodology (hardware, versions, config deltas, dataset, run count, median and spread) is published before the results, per house rule.

Version boundaries. Claims above are pinned to PostgreSQL 18 (asynchronous I/O, skip scan), ClickHouse 26.3 LTS (async_insert default change; 25.8 LTS leaves support on 29 August 2026), Kafka 4.x (KRaft-only), Debezium 3.6, Valkey 9.x, and Oracle 19c/26ai lifecycle dates as of August 2026. Re-verify before relying on any of them in a contract.

11. FAQ: Oracle Exadata cost optimization and open source migration

Is Oracle Exadata cost optimization possible without leaving Oracle?

Yes. Measure option usage in DBA_FEATURE_USAGE_STATISTICS, reduce enabled cores with capacity-on-demand, offload analytics to ClickHouse via CDC, and negotiate ahead of the 19c Extended Support uplift. Those four measures are reversible and typically remove 25–40% of the recurring line (illustrative; your figure comes from your own catalog).

Can PostgreSQL really match Exadata performance?

For OLTP, PostgreSQL 18 on a modern two-socket NVMe node matches or exceeds the enabled-core footprint most quarter-rack estates license, and the plan-level evidence is EXPLAIN (ANALYZE, BUFFERS) against the same statements. For Exadata-class analytics, PostgreSQL alone is not the answer; ClickHouse is, and that is why the target is a stack. Parity is declared per statement class from pg_stat_statements and system.query_log, never in aggregate.

How do we replace RAC?

RAC solves instance-failure RTO and read scaling. Patroni quorum failover with a synchronous standby delivers drill-measured RTO in the tens of seconds at RPO = 0; hot standbys behind PgBouncer deliver read scaling. Genuine write-anywhere requirements are rare and are evaluated separately, with the conflict analysis done before any active-active design is proposed.

Do we need Oracle GoldenGate for the migration?

No. Debezium’s Oracle connector captures from LogMiner without a GoldenGate license; OpenLogReplicator is the lower-overhead alternative on high-redo estates. GoldenGate remains a valid choice where the customer already licenses it.

What is the rollback plan if cutover fails?

Reverse CDC from PostgreSQL to Oracle runs from the moment of cutover until written sign-off. Rollback is: stop application writes, drain reverse lag to zero (measured in pg_replication_slots), re-point connection strings to Oracle. Nothing is dropped, truncated, or decommissioned before sign-off.

How long does an Exadata to open source migration take?

Illustratively 30–42 weeks for a 20 TB estate with moderate PL/SQL, per Figure 2. The schedule scales with procedural code volume rather than data size; the assessment in Section 8.1 replaces that illustration with a measured estimate.

Who supports the open source stack in production?

Community PostgreSQL, ClickHouse, Kafka, Debezium, and Valkey are supported by their projects; enterprise-grade 24×7 support with response SLAs comes from an independent provider. MinervaDB operates on S1 15 minutes / S2 12 hours / S3 24 hours / S4 48 hours, with ClickHouse engineering delivered through ChistaDATA.

Next steps

An Oracle Exadata cost optimization program starts with evidence, not a proposal. MinervaDB’s Exadata assessment delivers, within four weeks, the feature-usage baseline and enabled-core analysis for in-place savings, the object and PL/SQL complexity matrix, the AWR or STATSPACK top-SQL baseline mapped to target engines, and a versioned migration plan with rollback gates — the MDB-MIG deliverable that turns this whitepaper into your program. Talk to the MinervaDB data modernization practice, review our PostgreSQL consulting and PostgreSQL remote DBA services, or book an architecture consultation.

Standing caveat for all guidance in this document: test every change in a staging environment that mirrors production before applying it, and maintain a verified disaster-recovery posture (tested restores, not just backups) throughout any migration.

References

• Oracle Corporation, Oracle Technology Global Price List and Oracle Engineered Systems Price List; Exadata X11M announcement; Exadata documentation.
• PostgreSQL Global Development Group, PostgreSQL 18 released; release notes; logical replication; pg_stat_statements; versioning policy.
• ClickHouse, which version to use in production; Replicated table engines; Kafka table engine; 2026 changelog.
• Debezium, releases and Oracle connector; Apache Kafka, 4.3.1 release.
• Patroni, pgBackRest, PgBouncer, Ora2Pg, orafce, OpenLogReplicator, Valkey 9.
• Oracle Database lifecycle dates: endoflife.date (re-verify against MOS 742060.1).
• Related MinervaDB reading: PostgreSQL migration rollback strategy; PostgreSQL on AWS, GCP, and Azure.

Revision history: v1.0, 24 August 2026 — initial public release. Prepared by MinervaDB Database Architecture Practice; reviewed by MinervaDB PostgreSQL and Oracle practice leads; approved by Shiv Iyer, Founder & CEO, MinervaDB Inc.

Oracle Exadata Cost Optimization: Migrating to an Open Source Data Infrastructure Stack Without Compromising Performance, Scalability, Availability, or Reliability appeared first on MariaDB.org

A BigQuery SOX compliance checklist succeeds or fails on four IT general control (ITGC) domains: access control, change management, operations, and monitoring. Everything an external auditor will test against your BigQuery estate maps to one of those four, and every one of them can be evidenced directly from Google Cloud primitives — IAM policies, Cloud Audit Logs, INFORMATION_SCHEMA, and Cloud KMS. This checklist walks through the twelve controls we implement for financial-reporting datasets in BigQuery, with the exact SQL and configuration each control needs and the evidence query an auditor will accept.

Scope note before we start: SOX (Sarbanes–Oxley Act of 2002, Sections 302 and 404) does not certify databases — it certifies internal control over financial reporting (ICFR). BigQuery lands in scope the moment a dataset feeds a number that appears in a financial statement: revenue marts, billing pipelines, order-to-cash aggregates, close automation. Google’s own infrastructure controls are covered by its SOC 2 Type II reports (issued quarterly, auditable via Compliance Reports Manager); everything above the infrastructure line — who can read the revenue table, who changed the transformation SQL, how long the access trail is retained — is yours, and that is what this checklist covers.

BigQuery SOX Compliance Checklist: The Control Architecture

The twelve controls form three layers: preventive controls on the data itself (IAM, policy tags, row-level security, CMEK, VPC Service Controls), detective controls on activity (audit log pipeline, access monitoring, job history review), and process controls around change (declarative infrastructure, SQL change management, retention and recovery, evidence generation). The diagram below shows how they fit together.

BigQuery SOX Control Architecture (ITGC mapping) VPC SERVICE CONTROLS PERIMETER · ORG POLICIES (CONTROL 6) Preventive — Access 2 · IAM groups, no primitive roles 3 · Policy tags (column-level) 4 · Row access policies 5 · CMEK (Cloud KMS, 90d rotation) roles/bigquery.dataViewer datacatalog.categoryFineGrainedReader cloudkms.cryptoKeyEncrypterDecrypter In-scope datasets (1) finance_prod.revenue_reporting labels: sox_scope=in_scope Financial-reporting marts, close snapshots, billing pipeline Time travel 7d + close snapshots (10) Process — Change mgmt (9) Terraform (datasets, IAM, CMEK) dbt / Dataform SQL in PR review CI deploys via pipeline identity: deploy-sa@finance-prod.iam Humans hold no dataEditor on prod (segregation of duties, control 2) reviewed DDL only audit trail Detective — Monitoring 7 · Data Access logs (on by default) → sink → audit-prod dataset 7-year retention, separate IAM 8 · Access + write monitoring SQL INFORMATION_SCHEMA.JOBS (180d) BigQueryAuditMetadata Alert: any non-pipeline write to an in-scope table 11 · Quarterly IAM diff + attestation from SetIamPolicy log events Evidence pack (12) IAM exports · zero-row write report DDLPR reconciliation · KMS history restore drills · SOC 2 Type II Google Cloud behavior as of August 2026 · BigQuery + Cloud Logging + Cloud KMS + VPC-SC

Layer 1 — Preventive Controls

1. Inventory and label in-scope datasets

You cannot control what you have not scoped. Every SOX engagement starts with a dataset inventory that separates financial-reporting datasets from everything else, because the controls below are expensive to apply estate-wide and auditors only test in-scope objects. Use dataset labels as the scoping mechanism — they are queryable, enforceable in policy, and visible in billing exports.

-- Inventory candidate in-scope datasets and their labels
SELECT
  catalog_name              AS project_id,
  schema_name               AS dataset_id,
  option_value              AS labels
FROM `region-us`.INFORMATION_SCHEMA.SCHEMATA_OPTIONS
WHERE option_name = 'labels';

# Label a dataset as SOX in-scope (reversible, no downtime)
bq update --set_label sox_scope:in_scope 
          --set_label data_owner:finance_engineering 
          finance_prod:revenue_reporting

2. Enforce least-privilege IAM — no primitive roles

The single most common SOX finding we see in BigQuery estates is a primitive role (roles/editor, roles/owner) granted at project level, silently conferring write access to every financial table. The control: all access to in-scope datasets flows through Google Groups mapped to job functions, using predefined BigQuery roles (roles/bigquery.dataViewer, roles/bigquery.dataEditor, roles/bigquery.jobUser) or narrower custom roles — never primitive roles, never individual user grants.

# Evidence query: find primitive-role grants on the project
gcloud projects get-iam-policy finance-prod 
  --flatten="bindings[].members" 
  --filter="bindings.role:(roles/owner OR roles/editor OR roles/viewer)" 
  --format="table(bindings.role, bindings.members)"

Expected output on a passing control is an empty table (or break-glass accounts only, documented and alerted). Anything else is a segregation-of-duties exception the auditor will sample. Pair this with quarterly access reviews: export the IAM policy per dataset, have the data owner attest, retain the attestation. Segregation of duties in BigQuery terms means the humans who write transformation SQL do not hold dataEditor on production financial datasets — deployment happens through a service account owned by the CI pipeline (control 9).

3. Column-level security with policy tags

SOX scoping frequently overlaps PII and payment data. BigQuery’s column-level access control attaches policy tags from a Data Catalog taxonomy to individual columns; at query time, reading a tagged column requires the Fine-Grained Reader role (datacatalog.categoryFineGrainedReader) on that tag. One policy tag per column, taxonomy and table colocated in the same region, and dynamic data masking available on top — masked readers get nulls, hashes, or defaults instead of a permission error, which keeps dashboards alive while protecting the raw value.

-- Verify which columns carry policy tags in an in-scope dataset
SELECT
  table_name,
  column_name,
  policy_tags
FROM `finance_prod.revenue_reporting`.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS
WHERE ARRAY_LENGTH(policy_tags.names) > 0
ORDER BY table_name, column_name;

4. Row-level security for entity and regional segregation

Where a single revenue table serves multiple legal entities, row-level access policies enforce entity segregation inside the table rather than through fragile view sprawl:

CREATE ROW ACCESS POLICY entity_emea_only
ON `finance_prod.revenue_reporting.fct_revenue`
GRANT TO ('group:finance-emea@example.com')
FILTER USING (legal_entity = 'EMEA');

Two operational caveats we state in every engagement: row access policies silently filter rows (users see a subset, not an error), so reconciliation jobs must run as an identity with full-table access; and policies do not apply to time-travel reads by users with bigquery.rowAccessPolicies.overrideTimeTravelRestrictions-adjacent bypass paths — audit who holds table-level admin rights.

5. Customer-managed encryption keys (CMEK)

BigQuery encrypts everything at rest by default, but default encryption gives you no key custody evidence. CMEK puts the key-encryption key in your Cloud KMS keyring: you control rotation, you control revocation, and disabling the key renders the dataset unreadable — a demonstrable termination control. Grant the BigQuery encryption service account (bq-PROJECT_NUMBER@bigquery-encryption.iam.gserviceaccount.com) the roles/cloudkms.cryptoKeyEncrypterDecrypter role, colocate the key with the dataset region (a US multi-region dataset needs a us keyring), and set rotation ≤ 90 days for in-scope data.

# Current vs proposed: rotation period on the SOX keyring
# parameter: rotation-period | current: none (manual) | proposed: 90d | applies: next rotation
gcloud kms keys update sox-bq-key 
  --keyring=sox-keyring --location=us 
  --rotation-period=90d 
  --next-rotation-time=2026-09-01T00:00:00Z

6. Perimeter controls: VPC Service Controls and organization policies

IAM answers “who may read”; it does not answer “where may the data go.” A VPC Service Controls perimeter around the financial projects blocks exfiltration paths IAM cannot see — bq mk --transfer_config into an external project, result extraction to an out-of-perimeter bucket, cross-project table copies. Complement it with organization policies: domain-restricted sharing (constraints/iam.allowedPolicyMemberDomains) so no grant can name an identity outside your Workspace domain, and disable public dataset access. These two org policies alone close the “analyst shares revenue table with personal Gmail” finding that appears in a depressing share of first-year audits.

Layer 2 — Detective Controls

7. The audit log pipeline: export, retain, lock

This is the control auditors spend the most time on, and the one with a hard deadline in it. Per Cloud Audit Logs behavior: Admin Activity, System Event, and Policy Denied logs are always on; Data Access logs are disabled by default across Google Cloud except for BigQuery, where they are enabled by default — every query, every table read, every export lands in BigQueryAuditMetadata (use it, not the legacy AuditData format — see the BigQuery audit logs reference).

The trap is retention. Per Cloud Logging quotas, the _Required bucket holds Admin Activity logs for a non-configurable 400 days, but Data Access logs land in _Default with 30-day retention. SOX audit workpaper retention is seven years. Thirty days of query history does not survive an audit cycle, let alone seven years — so route the logs into BigQuery itself via a log sink, and lock the bucket:

# 1. Sink BigQuery data-access logs into a dedicated audit dataset
gcloud logging sinks create sox-bq-audit-sink 
  bigquery.googleapis.com/projects/audit-prod/datasets/bq_audit_logs 
  --log-filter='resource.type="bigquery_dataset" OR
    protoPayload.metadata."@type"="type.googleapis.com/google.cloud.audit.BigQueryAuditMetadata"' 
  --use-partitioned-tables

# 2. Grant the sink writer identity dataEditor on the audit dataset
#    (printed as writerIdentity by the command above)

# 3. Alternative/parallel: raise _Default retention (1–3650 days, configurable)
#    parameter: retention-days | current: 30 | proposed: 2555 (7y) | applies: immediately
gcloud logging buckets update _Default --location=global --retention-days=2555

The audit dataset lives in a separate project with its own IAM (the people being audited must not hold write access to their own trail — that is the whole point), CMEK-encrypted, with the sink’s partitioned tables giving you cheap seven-year storage on long-term pricing.

Audit-log evidence pipeline (control 7) BigQuery jobs Every query, read, export Data Access logs: ON by default for BigQuery Cloud Logging _Default bucket 30-day retention (configurable 1–3650 days) log sink Audit dataset audit-prod.bq_audit_logs Partitioned · CMEK · 7-year retention · separate IAM Evidence Scheduled queries Quarterly reviews Write alerts SOX workpaper retention is seven years — the 30-day default is the single most common audit-log finding. Auditees hold no write access to the audit project.

8. Access monitoring: the queries that answer “who touched revenue”

With the sink in place, the auditor’s favorite question becomes a query. Who read the revenue tables, when, from where, and did any service account behave anomalously:

-- Who accessed in-scope tables in the last quarter (from the audit sink)
SELECT
  protopayload_auditlog.authenticationInfo.principalEmail AS principal,
  JSON_VALUE(protopayload_auditlog.metadataJson,
             '$.tableDataRead.reason')                    AS read_reason,
  resource.labels.dataset_id                              AS dataset_id,
  COUNT(*)                                                AS access_count,
  MIN(timestamp)                                          AS first_access,
  MAX(timestamp)                                          AS last_access
FROM `audit-prod.bq_audit_logs.cloudaudit_googleapis_com_data_access`
WHERE resource.labels.dataset_id = 'revenue_reporting'
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)
GROUP BY principal, read_reason, dataset_id
ORDER BY access_count DESC;

For interactive investigation inside the retention window, INFORMATION_SCHEMA.JOBS gives you 180 days of job history without any pipeline:

-- Jobs that wrote to in-scope tables outside the deployment service account
SELECT
  user_email,
  job_id,
  statement_type,
  destination_table.dataset_id,
  destination_table.table_id,
  creation_time
FROM `region-us`.INFORMATION_SCHEMA.JOBS_BY_PROJECT
WHERE destination_table.dataset_id = 'revenue_reporting'
  AND statement_type IN ('INSERT', 'UPDATE', 'DELETE', 'MERGE',
                         'CREATE_TABLE_AS_SELECT', 'TRUNCATE_TABLE')
  AND user_email != 'deploy-sa@finance-prod.iam.gserviceaccount.com'
  AND creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)
ORDER BY creation_time DESC;

A passing control returns zero rows: nothing writes to financial tables except the pipeline identity. Wire the same predicate into a log-based alert so an out-of-band write pages someone the day it happens, not the quarter it is sampled.

9. Change management: declarative infrastructure and SQL under version control

Section 404 auditors test change management harder than access control, because manual hotfixes to revenue logic are where restatements come from. The control has three parts. First, schema and infrastructure are declarative — Terraform owns datasets, IAM bindings, CMEK wiring — so every change is a reviewed pull request with an approver who is not the author:

resource "google_bigquery_dataset" "revenue_reporting" {
  dataset_id                      = "revenue_reporting"
  location                        = "US"
  default_partition_expiration_ms = null   # financial data: no silent expiry

  default_encryption_configuration {
    kms_key_name = google_kms_crypto_key.sox_bq_key.id
  }

  labels = {
    sox_scope  = "in_scope"
    data_owner = "finance_engineering"
  }
}

Second, transformation SQL (dbt, Dataform, or plain scheduled queries) lives in the same review gate — no console-edited scheduled queries on in-scope datasets. Third, the audit logs closes the loop: every google.cloud.bigquery.v2.JobService.InsertJob with a DDL statement type against an in-scope dataset should reconcile 1:1 with a merged pull request. That reconciliation, run quarterly, is the change-management evidence.

10. Retention and recoverability: time travel is not a backup

BigQuery’s time travel window is 2–7 days (default 7), with a fixed 7-day fail-safe behind it. That is an operational undo, not a SOX retention control — fourteen days of recoverability does not support a seven-year evidence obligation. The control set: time travel pinned to 7 days on in-scope datasets (state it explicitly, don’t inherit defaults), scheduled table snapshots at period close so every reported number has a frozen source, and no default table expiration on financial datasets (an expiry policy silently deleting revenue history is a control failure you find at the worst possible moment).

-- Snapshot the revenue fact table at quarter close (zero-copy until divergence)
CREATE SNAPSHOT TABLE `finance_prod.period_close.fct_revenue_2026_q2`
CLONE `finance_prod.revenue_reporting.fct_revenue`
FOR SYSTEM_TIME AS OF TIMESTAMP '2026-07-01 00:00:00+00';

Standing caveat: rehearse the restore. A quarterly drill that recovers a snapshot into a scratch dataset and reconciles row counts against the close report is ten minutes of work and the difference between a backup strategy and a backup hope. Test every procedure here in a non-production project before applying it to production, and keep your DR posture current.

Layer 3 — Process and Evidence

11. Quarterly access review with attestation

Auditors sample quarters; the control must therefore fire quarterly without heroics. Export per-dataset IAM (dataset ACLs and project bindings), diff against the previous quarter, route additions to the data owner for attestation, and retain the signed attestation alongside the IAM export in the audit project. The whole loop is scriptable with bq show --format=prettyjson plus a scheduled query over the audit sink for SetIamPolicy events — the grant history is already in your logs:

-- All IAM changes on in-scope datasets this quarter
SELECT
  timestamp,
  protopayload_auditlog.authenticationInfo.principalEmail AS changed_by,
  resource.labels.dataset_id,
  protopayload_auditlog.methodName
FROM `audit-prod.bq_audit_logs.cloudaudit_googleapis_com_activity`
WHERE protopayload_auditlog.methodName LIKE '%SetIamPolicy%'
  AND timestamp >= TIMESTAMP '2026-04-01 00:00:00+00'
ORDER BY timestamp;

12. Evidence pack: make the audit boring

The final control is meta: everything above produces artifacts on a schedule, into one place. Our standard evidence pack per quarter — IAM exports and attestations, the zero-row output of the unauthorized-write query, DDL-to-PR reconciliation, KMS rotation history, restore-drill log, and Google’s SOC 2 Type II report pulled from Compliance Reports Manager for the infrastructure layer. When the evidence generates itself, the audit costs days instead of weeks.

The BigQuery SOX compliance checklist at a glance

Use this table as the working artifact: walk it quarterly, and require that every row can produce its evidence on demand. In our experience the first pass of a BigQuery SOX compliance checklist fails on rows 2, 7, and 9 — primitive roles, 30-day log retention, and console-edited scheduled queries — so start there if audit season is close.

Version boundaries and honest edges

Everything above reflects Google Cloud behavior as of August 2026: BigQuery Data Access logs on by default, _Default bucket at 30 days (configurable 1–3650), _Required fixed at 400 days, time travel 2–7 days with a 7-day fail-safe, INFORMATION_SCHEMA.JOBS at 180 days. Retention defaults and log formats have changed before; re-verify against the linked documentation before you certify a control on them.

This BigQuery SOX compliance checklist covers the BigQuery-native control surface — it does not cover upstream pipeline controls (your Kafka/Datastream/Fivetran layer needs its own change management), application-level controls in your ERP, or the entity-level controls your auditors test outside IT entirely. And SOX applicability is a determination for your auditors and counsel, not your database team: we are engineers, and this is engineering guidance, not legal advice.

Where this fits in a broader governance program

SOX controls on BigQuery rarely stand alone — they usually arrive alongside a broader push to make analytics platforms auditable, the same discipline we describe in our fractional CDO real-time analytics playbook and apply to regulated retail estates in the modern retail data stack. If you are staring down your first SOX cycle on BigQuery — or your auditors just handed you a findings list — MinervaDB’s fractional Chief Data Officer and database governance practice implements exactly this control set, evidence pipeline included. As always: test every control in a staging project before touching production, and keep a rehearsed DR posture behind every retention promise.

BigQuery SOX Compliance Checklist: 12 Proven Controls appeared first on MariaDB.org

For any team running high-connection-count MySQL on Community builds, this is the most consequential edition change since the MySQL 9.7 LTS Community transfers three months earlier.

This MySQL Thread Pool release analysis covers what actually changed, how to turn the plugin on and verify it is working, how to tune and monitor it with the Performance Schema, who should adopt it now versus wait, and where it leaves the Community-versus-Enterprise decision. Every claim is version-pinned; configuration values shown are starting points to be measured against your workload, not universal settings. Standard caveat: test in staging and keep a rollback path before changing thread_handling on a production server.

What MySQL 26.7 Changed, and Why It Matters

MySQL 26.7.0 is the first generally available Innovation release after MySQL 9.7 LTS, and the first MySQL release to use the new YY.M calendar-versioning model (so “26.7” is the July 2026 release, not a 26th major version). Its headline edition change is short to state and large in consequence: the Thread Pool plugin is now included in MySQL Community Edition. It is the same plugin, with the same system variables and the same Performance Schema instrumentation that Enterprise customers have used for years. Oracle confirmed the change in the MySQL July 2026 GA announcement.

This continues a trajectory that began at 9.7. That LTS moved eight capability groups across the Enterprise-to-Community line — the Hypergraph Optimizer, the OpenTelemetry telemetry component, three Group Replication components (flow control, resource manager, primary election), replication applier metrics, and JSON Duality View DML. MySQL 26.7 adds Thread Pool to that list.

The practical significance is that the MySQL Thread Pool was, for fifteen years, the single most-cited Enterprise Edition performance feature. Its move to Community does not invent a capability the open-source world lacked — Percona Server has shipped its own thread pool since 5.5, and MariaDB has long had one — but it makes Oracle’s canonical implementation free on stock MySQL. For the renewal conversation, the effect is decisive: the Enterprise value proposition has shifted away from performance and high availability toward security, backup, AI, and support. Thread Pool is no longer a reason to hold an Enterprise subscription.

The Problem the MySQL Thread Pool Solves: Connection Scaling

To understand the MySQL Thread Pool, start with the default. MySQL’s connection-handling model is thread-per-connection: every client connection is served by one dedicated operating-system thread for its entire lifetime. This is simple and low-latency at moderate concurrency, and it is the right default for most workloads. It degrades predictably when the number of concurrent connections climbs well past the number of CPU cores.

Three mechanisms drive the degradation, and all three are measurable rather than theoretical. Context-switching overhead rises as thousands of runnable threads compete for a handful of cores, so the OS scheduler spends an increasing share of cycles switching rather than executing.

CPU-cache efficiency falls because each thread carries its own stack, and thousands of stacks evict each other’s working set from L2/L3. And transaction parallelism drives contention inside InnoDB — more concurrent transactions means more pressure on shared structures, and past a point additional concurrency reduces throughput instead of increasing it.

The thread pool replaces “one thread per connection” with a bounded set of thread groups. Connections are distributed across thread_pool_size groups (default derived from CPU count), and each group runs a small number of active threads that multiplex many connections. Because the number of actively executing threads is capped near the core count, thread-stack reuse keeps the CPU-cache footprint small and transaction parallelism stays bounded — which is precisely what protects InnoDB from the contention cliff.

The trade is latency fairness: a bounded pool can queue a statement briefly under load, so a latency-critical, moderate-concurrency workload can be worse off on the pool than on the default model.

Enabling the MySQL Thread Pool in 26.7 Community

Because the MySQL Thread Pool is now bundled with Community Edition, enabling it is the same declarative install used on Enterprise, with no commercial package to source first. Confirm the plugin library is present, install the plugin set, and switch the connection-handling model. The plugin is loaded at startup, so the durable configuration lives in my.cnf. The mechanics match the MySQL Thread Pool reference documentation.

-- 1. Confirm you are on a release that ships Thread Pool in Community
SELECT VERSION();   -- expect 26.7.0 or later on a Community build

-- 2. Load the plugin set (INSTALL PLUGIN persists in the mysql.plugin table,
--    but thread_handling must still be set in my.cnf; see step 4)
INSTALL PLUGIN thread_pool SONAME 'thread_pool.so';

-- 3. Verify the plugin and its companion Performance Schema tables are ACTIVE
SELECT PLUGIN_NAME, PLUGIN_STATUS, PLUGIN_TYPE
  FROM information_schema.PLUGINS
 WHERE PLUGIN_NAME LIKE 'thread_pool%'
    OR PLUGIN_NAME LIKE 'tp_%';

The connection-handling model itself is not a runtime-settable variable — it is read at server start. Set it in the configuration file and restart during a maintenance window. State the blast radius before you do: switching thread_handling changes how every connection is scheduled, so validate on a replica first and have the one-line rollback ready.

# my.cnf  — Thread Pool configuration (MySQL 26.7 Community)
# Change: thread_handling  one-connection-per-thread -> pool-of-threads
# Requires a server restart. Rollback: comment these out and restart.
[mysqld]
thread_handling                  = pool-of-threads
thread_pool_size                 = 16      # start at number of physical cores
thread_pool_max_transactions_limit = 512   # ~ cores x 32; caps concurrent txns
thread_pool_stall_limit          = 6       # units of 10ms; 6 = 60ms stall check
thread_pool_algorithm            = 1       # 1 favors high-concurrency OLTP
thread_pool_query_threads_per_group = 2    # worker threads per group to start

-- 4. After restart, confirm the model actually changed
SHOW GLOBAL VARIABLES LIKE 'thread_handling';
-- Expected: thread_handling = pool-of-threads

SHOW GLOBAL VARIABLES LIKE 'thread_pool%';

Tuning the MySQL Thread Pool: The Variables That Matter

Four MySQL Thread Pool variables carry most of the tuning weight. Treat the values below as starting points anchored to hardware, then adjust against measured throughput and latency — never against a rule of thumb alone. The starting points below follow the MySQL Thread Pool tuning guidance.

The thread_pool_stall_limit value is the one most worth understanding. When every thread in a group is busy, the group will not start a new statement until the stall timer expires — this is deliberate, and it is what caps parallelism. Set it too high and a burst of long-running statements can starve short OLTP queries; set it too low and you erode the contention protection you turned the pool on for. This is a workload-shape decision, not a default.

One interaction is worth calling out explicitly, because it trips up first-time tuners. thread_pool_size and thread_pool_max_transactions_limit are not independent knobs. The first sets how many thread groups exist; the second caps how many transactions can execute across all of those groups at once. If you raise thread_pool_size to match a high core count but leave the transaction limit low, you have created groups that are structurally allowed to run work but are throttled from doing so, and throughput plateaus below what the hardware can deliver. Conversely, a generous transaction limit with too few groups concentrates connections and reintroduces the queuing you were trying to avoid.

Tune them as a pair: set thread_pool_size to the physical core count first, then raise or lower thread_pool_max_transactions_limit while watching the stall ratio and InnoDB row-lock waits, and stop at the point where additional concurrency stops improving committed transactions per second. That inflection point is specific to your schema and access pattern, which is exactly why the values in the table above are starting points and not answers.

Monitoring the MySQL Thread Pool with the Performance Schema

The MySQL Thread Pool ships three Performance Schema tables, and they are the measurement source for every tuning decision. Recommendations that are not anchored to these tables are guesses.

-- Live per-group state: how many connections and threads each group holds,
-- and how deep the high/low priority queues are right now
SELECT TP_GROUP_ID,
       CONNECTION_COUNT,
       THREAD_COUNT,
       ACTIVE_THREAD_COUNT,
       QUEUED_QUERIES        -- backlog is the early-warning signal
  FROM performance_schema.tp_thread_group_state
 ORDER BY QUEUED_QUERIES DESC;

-- Stall ratio: the single most important thread pool health metric.
-- A rising ratio means statements are being treated as stalled and
-- spawning extra workers — i.e. the pool is fighting your workload.
SELECT SUM(STALLED_QUERIES_EXECUTED) / NULLIF(SUM(QUERIES_EXECUTED), 0)
         AS stalled_ratio,
       SUM(QUERIES_EXECUTED)  AS total_queries,
       SUM(STALLED_QUERIES_EXECUTED) AS stalled_queries
  FROM performance_schema.tp_thread_group_stats;

/* Illustrative output shape — values will differ on your server
+---------------+---------------+-----------------+
| stalled_ratio | total_queries | stalled_queries |
+---------------+---------------+-----------------+
|        0.0123 |      48211900 |          593006 |
+---------------+---------------+-----------------+
*/

These two tables are the backbone of MySQL Thread Pool monitoring. Read these together. A persistently high QUEUED_QUERIES in tp_thread_group_state combined with a climbing stalled ratio in tp_thread_group_stats tells you the pool is too small for the offered load — raise thread_pool_size or investigate the long statements holding groups busy. A low stalled ratio with acceptable latency means the pool is doing its job. Always baseline both before and after any variable change, and correlate with connection counts the way you would when sizing a client-side pool in our MySQL connection pooling guide.

Who Should Adopt the MySQL Thread Pool in 26.7 — and Who Should Wait

The MySQL Thread Pool is a targeted fix, not a universal upgrade. The clearest MySQL Thread Pool candidates are OLTP servers with thousands of mostly-idle or bursty connections — the classic PHP/short-connection or microservice fan-out pattern — where connection count runs far ahead of core count and throughput visibly collapses past a concurrency threshold.

If you can reproduce a throughput curve that peaks and then falls as concurrency rises, the pool is likely to flatten that fall.

Workloads that should not rush to it: latency-sensitive, moderate-concurrency systems where connection count stays near or below core count. For those, thread-per-connection is already optimal, and the pool’s queuing can only add latency. Analytics/OLAP servers running a small number of large parallel queries gain nothing from connection multiplexing. And if a well-behaved client-side connection pool (ProxySQL, a framework pool) already holds server connections to a sane number, the server-side thread pool addresses a problem you may not have.

Our stance for 26.7 specifically. The feature is genuinely valuable and now free, but 26.7 is an Innovation release, not LTS — it is supported only until the next Innovation release supersedes it, whereas 9.7 LTS carries Premier support to 2034.

For production, the disciplined path is to prove Thread Pool on 26.7 in staging, confirm the throughput win against your own workload, and deploy it in production on the LTS line you actually run — 9.7 — if and when the plugin is available there, rather than putting a short-lived Innovation build under a production estate for a single feature. Treat 26.7 as the release that proves the capability is free, and your LTS as where it lands.

What This Means for the Community-vs-Enterprise Decision

With the MySQL Thread Pool gone to Community, the honest Enterprise Edition value proposition after 26.7 is security, backup, AI, and support — not performance and not high availability, both of which are now fully open source. With the MySQL Thread Pool no longer a differentiator, before renewing an Enterprise subscription run the usage audit: enumerate which Enterprise features are actually in use.

-- The Enterprise renewal audit: what are you actually paying for?
SHOW PLUGINS;

SELECT PLUGIN_NAME, PLUGIN_STATUS, PLUGIN_LIBRARY
  FROM information_schema.PLUGINS
 WHERE PLUGIN_LIBRARY IS NOT NULL;

SHOW GLOBAL VARIABLES LIKE 'audit%';     -- Enterprise Audit in use?
SHOW GLOBAL VARIABLES LIKE 'keyring%';   -- which keyring / KMS integration?

Most Enterprise estates use MySQL Enterprise Backup and Enterprise Audit and little else, and both have credible open-source equivalents (Percona XtraBackup and Percona Audit Log respectively).

Even with the MySQL Thread Pool now free, what legitimately keeps a subscription in place after 26.7 is a narrower list: Enterprise Backup where XtraBackup for your version line is not yet GA, hard Enterprise Firewall or Data Masking compliance requirements with no in-server open-source equivalent, Kerberos/LDAP enterprise authentication, DISA STIG or CIS contractual certification, NDB Cluster, or on-premises vector search and in-database LLMs via MySQL AI.

If none of those apply, Thread Pool’s move removes one of the last performance-shaped reasons to renew. For a deeper look at eliminating server-side contention once the pool is in place, our InnoDB performance optimization guide covers the InnoDB-layer tuning that pairs with it.

MySQL Thread Pool FAQ

Is MySQL 26.7 Thread Pool the same as the Percona or MariaDB thread pool?

Not quite. The MySQL Thread Pool and its cousins solve the same problem with the same core idea — bounded thread groups multiplexing many connections — but they are independent implementations with different variables and internals. What 26.7 changes is that Oracle’s canonical Thread Pool plugin, previously Enterprise-only, is now the free default option on stock MySQL Community, so you no longer need Percona Server or MariaDB to get an Oracle-lineage thread pool.

Do I need to restart MySQL to enable the MySQL Thread Pool?

Yes. thread_handling is read at server startup, so switching from one-connection-per-thread to pool-of-threads requires a restart. Plan it in a maintenance window, validate on a replica first, and keep the rollback (comment out the settings, restart) ready.

Should I run Thread Pool in production on MySQL 26.7?

The MySQL Thread Pool is worth staging, not rushing. Prove it on 26.7 in staging, but for production prefer the LTS line (9.7) that carries long support, since 26.7 is an Innovation release supported only until the next Innovation release. Adopt the pool where connection count greatly exceeds core count and throughput degrades under concurrency; skip it for latency-sensitive, low-concurrency, or OLAP workloads.

Does moving Thread Pool to Community end the case for MySQL Enterprise Edition?

No, but it narrows it. After 26.7, Enterprise’s real value is security (Firewall, Data Masking, enterprise authentication), backup (MEB where XtraBackup is not GA for your line), MySQL AI, and Oracle support/certification — not performance or HA. Run the renewal audit above before deciding.

MySQL Thread Pool: Final Thoughts

MySQL 26.7 moving the Thread Pool plugin to Community edition is a small change to state and a meaningful one to act on for anyone tuning the MySQL Thread Pool. It gives every Community user Oracle’s own connection-scaling model for free, it closes a fifteen-year Enterprise differentiator, and it reshapes the renewal math for anyone paying for Enterprise primarily for performance. The engineering discipline around it is unchanged: enable the MySQL Thread Pool only where the connection-to-core ratio justifies it, tune thread_pool_size and thread_pool_stall_limit against measured throughput, monitor the stall ratio in tp_thread_group_stats, and land it on your LTS line rather than a short-lived Innovation build.

Done that way, the MySQL Thread Pool becomes a measured throughput win rather than a configuration gamble, and the plugin’s new Community status simply removes the licensing friction that used to sit in front of that decision.

If you are weighing a Thread Pool rollout, planning a 9.7 LTS upgrade, or reassessing an Enterprise renewal in light of the 9.7 and 26.7 Community transfers, MinervaDB’s MySQL consulting and 24×7 enterprise-class database support teams do this work daily across 900+ enterprise customers, with staged, reversible changes and a rollback path stated before anything touches production. As always: test before applying to production, and maintain a robust, tested DR posture.

MySQL 26.7 Moves the Thread Pool Plugin to Community Edition appeared first on MariaDB.org

A lot has happened since then. We also talked about the project at the PostgreSQL meetup in Berlin in March, at pgconf.de, and in a lightning talk at pgconf.dev, and a good chunk of what was still on the roadmap back then is live today. We also posted a video introduction to Hackorum, if you prefer to see it in action rather than read about it. This post is a follow up to walk through what changed, what is new, and what we are looking at next.

A quick recap

Hackorum is an Open Source project from the community to be used by the community and hosted on hackorum.dev. The code is on GitHub. If you are new here, the short version that it syncs postgres mailing lists in the background, keeps read status, stars, tags and notes for you, and adds commitfest context, commit and patch history, and contributor info directly next to the discussion.

Everything below is new for users since the last post. I am leaving out internal and admin only changes, since those do not affect how you use the site day to day.

Commits linked back to their discussion

Hackorum now links pushed commits back to the thread and patch discussion that produced them. Contributor profile pages have a new commit history tab, so you can see someone’s message activity and their commits in one place. There is also a community proposal on pgsql-www for public contributor profile pages, with the PostgreSQL Contributors Committee looking to recognize specific contributions beyond code, for example volunteering at a conference. If that lands, it would be a natural fit to surface on this profile too.

Patch CI results, now public

Patches attached to a thread are automatically applied, rebased when master moves, and built and tested with the PostgreSQL test suite. Those results used to be an internal experiment, they are now public for everyone:

• CI status icons in the topic index, and full detail in tabs on the thread itself
• A CI dashboard at hackorum.dev/ci
• A per topic CI history view
• A stats overview page

We reapply the latest version of each patch once a day against current master. If it stops applying, we do not give up right away, we keep retrying for 30 more days, so one bad day does not retire a patch. Only after 30 days of failing do we mark it retired, meaning the base is too old, and stop trying until a new version is posted.

Once something actually gets committed, there is nothing left to test against. There are two ways this shows up. If the committer changed the patch before committing it, the old version no longer applies and Hackorum shows it as no longer matching. If the committer applied it exactly as submitted, the diff against the committed version on GitHub is empty, so we know it went in as-is. Either way, we stop running CI on that thread until a new patch shows up. Some threads get committed in stages, with fixups or follow up patches, so if a new patchset lands afterward, we pick CI back up and start testing it.

One more detail worth calling out is that Hackorum highlights patches that are already committed but still have an open commitfest entry, so those are easy to spot and clean up. That is currently a big issue and might really help to get this easier updated in the future. A future integration with the commitfest might be thinkable.

Support for more mailing lists

Hackorum used to be pg-hackers only. It now also ingests pgsql-bugs, pgsql-docs, pgsql-general and pgsql-patches, with a badge next to each thread so you can see at a glance which list a message came from. We also tried pgsql-committers, it is imported up to around May/June, but we paused it for now. It is mostly terse commit notifications, and without a way to hide a list by default yet, it added more noise than value. We are planning to add more lists, so let us know if you are missing one.

Saved searches

The advanced search from the last post is still there, but you no longer have to retype the same query every time. You can save a search, personally or shared with your team, and pin it to your sidebar. We also ship a few global saved searches out of the box.

Ignore threads you do not care about

A small one, but a popular request was that you can now ignore a thread from the topic list with one click. Ignored threads disappear from your views and search results by default, and you can always list what you have ignored, or bring one back.

Sending email from Hackorum

This was the top item on the “planned” list in February, and it is here now. You can reply to a thread directly from Hackorum, including reply all, with a Thunderbird style selective quote so you only quote the part you are actually replying to. There is a drafts sidebar so you can keep several replies in progress, and a “My emails” page that shows everything you have sent through the site.

This is currently limited to @gmail accounts and being under active testing. If you like early access, just reach out so we can activate it for your user.

Everyday improvements

None of these are headline features on their own, but you will notice them:

• Long threads load in batches, so opening a huge discussion is noticeably faster
• A “jump to latest” button, and a setting to jump straight to your first unread message when you open a thread
• Read messages can collapse automatically if you prefer a shorter view
• Patch diffs now show inline line stats and highlighting
• Sessions now stay signed in for 30 days across devices
• The light/dark theme toggle is available everywhere, even before you sign in

Mobile got a real pass

The mobile experience got a lot of attention: a proper burger menu, a quick shortcut to threads you starred, working swipe back and forward gestures, and layout fixes for tablets and foldables.

What we are working on next

We are looking into AI generated summaries for longer threads, so you can get up to speed on a big discussion without reading every message.

Try it and share feedback

If you want to take a look, just go to https://hackorum.dev/.

The repository, including a simple dev setup, is here: https://github.com/hackorum-dev/hackorum

Want to chat with us, join the hackorum channel on the PostgreSQL Hacking Discord.

Is this useful? What is missing? What would you change? Bug reports, feature requests, and contributions are all welcome: https://github.com/hackorum-dev/hackorum/issues

Thanks for taking a look, and we appreciate any feedback.

Hackorum Update: What’s New Since February appeared first on MariaDB.org

First of all a quick description of our setup. We have a two nodes Patroni cluster on Open Stack virtual machines, PostgreSQL 18, with leader and a read only replica, so we write only on one node and use the second for data extractions. All applications use Kubernetes pods that are also setup in Open Stack and access PostgreSQL via PgBouncer connection pooler. We do have also some Java application making use of their own Hikari connection pooler. All on premises.

The application, being a legacy one, needed to use all the time data types in the timezone of the customer, so I had to setup the timezone of the PostgreSQL cluster different from the one of the VMs, that is in UTC. That was the first error as I set it up using a simple ALTER SYSTEM SET TIMEZONE followed by a SELECT pg_reload_conf(). This changes the timezone on the postgresql.auto.conf configuration file, which is totally good if you have a normal single node or replica PostgreSQL but not with a Patroni Cluster! I will return on this point in a moment.

In itself the migration was fine, but we had to tailor a little bit the resources assigned to the nodes as unfortunately we were not able to test, prior to the migration, some parts of the application. So we had to increase RAM on the VMs and modify shared_buffers parameter, doing a restart of the PostgreSQL cluster one node at a time.

Unfortunately the restart wiped out the timezone configuration that I did, as I should have added this through a patronictl edit-config command as I did for the shared_buffers. At least we recognized quickly the problem as the timezone of the whole Patroni Cluster reverted back to UTC. This time I changed it modifying the Patroni Cluster configuration, using the following patronictl command edit-config:

Fig. 1 – patronictl edit-config command

And adding the correct timezone:

Fig. 2 – Result of edit-config command

Confirmed issuing a show-config:

Fig. 3 – Result of show-config command

After that we resumed normal operations and all seemed fine, problem solved, no big issue or disruptions to our customer.

The day after I got contacted by one developer telling me that PostgreSQL in production is still in UTC. I checked it immediately and it was instead in WAT, confirmed issuing:

Fig. 4 – Screenshot of show timezone in pgAdmin

Then the developers sent me the results of a query done on an application log table showing a clear inconsistency of timestamps, some were correct and some with UTC timezone! That is a very bad scenario, where you do not have a clear indication, all wrong or all correct, but mixed results.

I started suspecting that there was something wrong on the application side as all that I saw on the database was correct, except some of the values that were recorded. So my next move was to ask devops to restart one by one all the application pods (remember that we are using Kubernetes).

This action brought mixed results as at the beginning it seemed that problem was solved and no rows with timestamp with wrong timezone was inserted…but after some time the first rows with UTC started resurfacing again! That was weird so I started digging a little bit more and searched our setup and our structure, as I was sure that the PostgreSQL database in itself was not the culprit.

As you may have guessed at this point there was one Elephant in the room that I had not yet investigated (and no was not PostgreSQL Elephant, AKA Slonik): PgBouncer! In fact I had totally left out of the picture the connection pooler as I thought that it was installed in kubernetes as a sidecar of the application pods, so expecting that a restart of the pods would have affected also PgBouncer.

As that was the architectural schema that was decided some years ago for our applications. Turned out that this was not the case!

Let’s make a quick excursus on why it is extremely important to have a connection pooler in PostgreSQL, the pool mode parameter of PgBouncer and the possible ways of installing and configuring it. These points are important to understand our case.

PostgreSQL connections are costly, mainly in terms of RAM, so much that there is a parameter max_connections to limit the maximum number of sessions, both active and idle. The default value is 100 which obviously is quite low, but rising it means that we need to ensure that we have enough RAM available, as the overhead for each connection is roughly 10 Mb. Bringing this parameter up for example to 500 means having already 5 Gb of RAM used just for idle connections, without even issuing any query!

Here is where connection poolers come to the rescue, as they are capable of recycling the connections, so that applications do not need to open a new connection each time they need to query the database. The pool maintains a fixed set of open connections, all requests borrow a connection, use it, and return it to the pool. The connection itself is never closed between requests, it stays open and ready so that the next request picks it up instantly.

PgBouncer is the de facto standard for connection pooling in PostgreSQL as it is lightweight, easy to configure and maintain. There is a very important setup choice to be made when installing PgBouncer: the pool mode, that decides the behaviour of the pooler. There are 3 possible choices: Session – Transaction – Statement.

Session Pooling: Assigns a server connection to a client as soon as it connects and holds it until the client disconnects. It acts almost like a direct connection to PostgreSQL, supporting all session features, but offers the least connection reuse. This is the most safe way to configure PgBouncer, and since we use some session features like Prepared Statements, this was our choice.

Transaction Pooling: Assigns a server connection only for the duration of a single BEGIN … COMMIT/ROLLBACK transaction. Once the transaction completes, the connection goes back to the pool for another client to use.

Statement Pooling: Assigns a server connection for a single SQL statement, returning it immediately after execution. It allows maximum reuse, but breaks multi-statement transactions (BEGIN … COMMIT) and session features.

One last word on what are the recommended ways to install PgBouncer: first of all it should be on a separate server respect the PostgreSQL cluster, then in a Kubernetes environment, such as ours, it can be installed as a sidecar to Kubernetes pods (which is the recommended way for most cases) or as a separate deployment in its own pod. Turns out that we had installed PgBouncer in this last way instead of having it in the same pod of the application.

Now that we have a complete picture, let’s go back to our case. Since our PgBouncer was installed as a standalone pod and not in sidecar, this become my primary culprit, searching a little bit I found out that there is the possibility in PgBouncer that if a database is created or its timezone is altered via ALTER DATABASE … SET timezone, PgBouncer does not properly invalidate its internal startup-parameter cache.

In our case the timezone was changed through Patroni, not ALTER DATABASE, but PgBouncer still kept a stale cached TimeZone.

So when a connection is recycled:

• PostgreSQL natively sets the timezone to Africa/Lagos.
• PgBouncer looks at its cached baseline state for that database/user profile from when it first booted up (which was reset to UTC when we restarted the Patroni nodes remember!).
• PgBouncer then subtly injects a session-level override back to the client, effectively masking the database’s actual default settings.

It was exactly our case, as the restart of Patroni Cluster nodes returned all the system to UTC, then we modified it again to WAT, but we never restarted the PGBouncer pod, so we still had some sessions in the pool with UTC timezone. That explained also the mix between correct and wrong timezones in the timestamps, it depended from the session with which the row was inserted, if it was one of the old ones we had UTC.

The fix at this point was obvious: clear the cache of PgBouncer. This can be done with a simple restart of that pod or a reload of the configuration. In our case we restarted PgBouncer and surely this action solved all our timezone problems as from that point we had only timestamps with correct timezone, confirming the above scenario.

So this was the curious case of timezone inconsistencies between PostgreSQL and PgBouncer, in the end was easily solved and not disruptive, it was due mainly to a couple of errors and assumptions that I did and that were surely avoidable, that’s why I wanted to bring attention to this subject: prevent others doing my errors and recognize faster the issue.

Summarizing all in a few key points of lessons learned:

• After changing timezone in Patroni, always restart or RECONNECT PgBouncer so the pool doesn’t keep stale session settings.
• Remember to always change timezone in a Patroni cluster using patronictl command edit-config and not ALTER SYSTEM SET TIMEZONE.
• In a Kubernetes environment install PgBouncer as a sidecar to Kubernetes pods instead of a standalone pod.

Hope it will help some of you!

This post is part of the Percona Community Writers Program.

The curios case of timezone inconsistencies between PgBouncer and Patroni Cluster appeared first on MariaDB.org

Summary

Percona has recently been made aware of a security vulnerability affecting PMM. We take the security of our products and the protection of our customers’ data with the utmost seriousness.
This advisory describes the vulnerability, the immediate steps you can take to protect your deployment, and the permanent fix.

Vulnerability details

• CVE identifier number: Pending, this advisory will be updated once assigned 
• CVSS score: 8.7 (High)

PMM’s Grafana instance allows signed-in users, including those with the Viewer role, to call raw data source APIs. If anonymous access has been explicitly enabled (it is off by default), unauthenticated users can also reach these APIs. 

Through the Grafana ClickHouse data source, such a user can submit arbitrary SQL. The data source connects to ClickHouse as the default identity, which holds global DDL, DML, and SOURCES privileges and can make outbound HTTP requests.

Chained together, this allows an attacker to reach AWS IMDSv1, obtain a live EC2 role session, read a Terraform remote-state object from S3, and authenticate as the PMM/Grafana administrator.

Impact

An unauthenticated attacker can cross the public Grafana boundary into internal databases, AWS instance metadata, S3 remote state, and the PMM administrator account. This yields renewable cloud credentials and a remote-state file that can contain many independently reusable secrets and private infrastructure details. 

The severity of impact depends on deployment configuration. The full IMDS-to-credential chain requires anonymous access to be enabled (off by default) and the PMM Server to be running on AWS EC2 with IMDSv1. 

All deployments are affected by the arbitrary SQL execution via the ClickHouse data source.

Remediation

This vulnerability is fixed in PMM 3.9.1, scheduled for release on August 19, 2026. Upgrade as soon as it is available.

Mitigation

If you cannot upgrade immediately, run the script below to close the exploitation path. It creates a least-privilege ClickHouse user for Grafana and points the ClickHouse data source at it, replacing the default superuser and making the exploitation of the vulnerability impossible. 

Before running the script, back up your PMM Server and save the pmm-data volume. 

#!/bin/bash
# Create a least-privilege ClickHouse identity for Grafana and point the
# ClickHouse datasource at it, replacing the default superuser.


set -euo pipefail

CONTAINER=${CONTAINER:-pmm-server}
PMM_HOST=${PMM_HOST:-localhost}
PMM_PORT=${PMM_PORT:-443}
GRAFANA_URL="https://${PMM_HOST}:${PMM_PORT}"
ADMIN_PASS=${ADMIN_PASS:-$(cat /root/pmm-admin-password)}
# Drop-ins are loaded from users.d (users_config defaults to users.xml ->
# users.d), NOT default-users.d.
BOOTSTRAP_XML=/etc/clickhouse-server/users.d/zz-provision-bootstrap.xml

CH_PASS=$(openssl rand -hex 24)
CH_HASH=$(printf '%s' "$CH_PASS" | sha256sum | awk '{print $1}')
BOOT_PASS=$(openssl rand -hex 24)
BOOT_HASH=$(printf '%s' "$BOOT_PASS" | sha256sum | awk '{print $1}')

ch_wait () {
  local user=$1 pass=$2 i
  for i in $(seq 1 45); do
    if docker exec -i "$CONTAINER" clickhouse-client --host 127.0.0.1 
         --user "$user" --password "$pass" -q "SELECT 1" >/dev/null 2>&1; then
      return 0
    fi
    sleep 2
  done
  echo "ERROR: clickhouse did not accept $user within 90s" >&2
  return 1
}

# PMM's ClickHouse default superuser has access_management disabled, so it
# cannot run CREATE USER / GRANT even with its known password. Install a
# short-lived admin to run the DDL instead. Drop-ins must live in users.d;
# and a plaintext  is rejected outright at startup because PMM
# ships allow_plaintext_password=0.
docker exec -u root "$CONTAINER" mkdir -p /etc/clickhouse-server/users.d
docker exec -u root -i "$CONTAINER" bash -c "cat > $BOOTSTRAP_XML" <
    
        
            $BOOT_HASH
            127.0.0.1::1
            default
            default
            1
        
    

XMLEOF
docker exec -u root "$CONTAINER" chown pmm:root "$BOOTSTRAP_XML"
docker exec -u root "$CONTAINER" supervisorctl restart clickhouse
ch_wait provision_admin "$BOOT_PASS"

# grafana_ro holds SELECT and nothing else. Without the SOURCES family it
# cannot call url(), s3(), mongodb(), remote() or file(); readonly=1
# additionally prevents it overriding server settings such as
# max_http_get_redirects. ALTER runs unconditionally so that re-running
# this script rotates the password rather than failing.
docker exec -i "$CONTAINER" clickhouse-client --host 127.0.0.1 
  --user provision_admin --password "$BOOT_PASS" --multiquery <&2
  exit 1
fi

# Transient files hold the CH password; keep them in a private dir and
# always remove them, even if a curl below fails.
umask 077
TMPD=$(mktemp -d)
trap 'rm -rf "$TMPD"' EXIT
curl -sk -u "admin:$ADMIN_PASS" 
  "$GRAFANA_URL/graph/api/datasources/uid/$DS_UID" > "$TMPD/ds-ch.json"
jq --arg p "$CH_PASS" 
  '.jsonData.username = "grafana_ro" | .secureJsonData.password = $p' 
  "$TMPD/ds-ch.json" > "$TMPD/ds-ch.new.json"
curl -sk -u "admin:$ADMIN_PASS" -X PUT -H 'Content-Type: application/json' 
  -d @"$TMPD/ds-ch.new.json" 
  "$GRAFANA_URL/graph/api/datasources/uid/$DS_UID" >/dev/null

# Fail the build rather than come up believing this worked.
if docker exec -i "$CONTAINER" clickhouse-client --host 127.0.0.1 
     --user grafana_ro --password "$CH_PASS" 
     -q "SELECT count() FROM url('http://169.254.169.254/latest/user-data','LineAsString','line String')" 
     >/dev/null 2>&1; then
  echo "FATAL: grafana_ro can still reach url()" >&2
  exit 1
fi
if docker exec -i "$CONTAINER" clickhouse-client --host 127.0.0.1 
     --user grafana_ro --password "$CH_PASS" 
     -q "CREATE TABLE default.zz_provision_check (x String) ENGINE=Memory" 
     >/dev/null 2>&1; then
  echo "FATAL: grafana_ro can still run DDL" >&2
  exit 1
fi

unset CH_PASS BOOT_PASS ADMIN_PASS
echo "ClickHouse datasource now authenticates as grafana_ro."

Run the script as follows: 

CONTAINER=pmm-server PMM_HOST=localhost PMM_PORT=443 ADMIN_PASS=XXXXX bash ./pmm-ch-user.sh

Support & additional resources

If you require further clarification or assistance, we are available 24/7:

• Technical support portal for customers
• Technical support for community

Contact

For questions about this advisory, upgrade planning, or to discuss options for unsupported major versions, open a case via the Percona Customer Portal or contact your Percona Customer Success Manager. 

For other security-related questions, write to security@percona.com.

The post Security Advisory: Privileged ClickHouse access through the Grafana data source in PMM appeared first on Percona.

Security Advisory: Privileged ClickHouse access through the Grafana data source in PMM appeared first on MariaDB.org

Similarly, while dealing with various incidents occurring in Percona XtraDB Cluster or MariaDB Galera Cluster environments, how many times did you stumble upon the fact that the GCache file (galera.cache) is a black box and you can’t inspect it in a meaningful way? 

In some scenarios, having the opportunity to see what exactly ended up in the cache file(s) could help us understand the write workload impact or what happened with transactions.

Why would one need to dig into galera.cache files, though? Let’s think about possible scenarios:

• Debugging replication issues or conflicts (BF aborts, etc).
• Understanding recent workload patterns per table (especially when binary log is not enabled or lost).
• Understanding the IST capacity and why node joining falls back to SST.
• Forensic analysis after incidents.
• Why on-demand gcache.page.X files are created and what transactions are inside.
• What committed writesets are still in “assigned / live” vs “released / reclaimable” state.
• Observe / confirm the impact of binlog_row_image setting on the writesets size.

To address those, I decided to experiment with a tool that would decode the Galera cache files. As a result of these experiments, I recently published gcache-inspector – an open source project available on GitHub. 

Before I introduce how the tool works, let’s quickly review the write set caching process.

What is Galera Cache? 

In short, it is a RingBuffer file storing Write-set Cache, which is also memory-mapped. Every replicated transaction is appended to it. Due to the fixed size, the oldest entries are overwritten to allow new writes. In special circumstances when the cache file is too small to fit a big transaction or old entries are not ready to be removed, additional on-demand cache files are created.

From the operational perspective, the most important role of the Galera cache is to provide quick incremental synchronization (IST) of (re-)joining cluster nodes. Having the cache of enough size, so that it can store enough time’s worth of writes, determines the joining process – whether a restarted node will be able to join quickly via IST or whether it will have to pull a full backup (SST) from the donor.

The diagram below shows the typical transaction lifecycle, role, and structure of the Galera cache.

The IST determination is a bit more complex than you’d expect. The joiner estimates the donor’s capabilities with some safety margin.

It is possible to verify the current potential donor Galera cache coverage from its wsrep_local_cached_downto status variable. Moreover, the cache rotation can be put on hold to extend the donor’s time window coverage via the gcache.freeze_purge_at_seqno provider option.

If the above diagram is difficult to digest, the following blog post should shed light on the process: https://www.percona.com/blog/understanding-ist-donor-selected/

Given all this complexity, you may sometimes just want to check and verify for yourself what on earth is in the Galera cache files, instead of guessing. 

And historically, the cache files were just a mystery – no tools available to actually properly inspect them. This is why I decided to experiment with a utility that would fill that gap.

The gcache-inspector

The tool I ended up with can fully decode the Galera cache files. It makes quick general write patterns statistics, write set nature information, and can decode the actual Row-based events (binary log style).

Gcache-inspector works offline (the examined PXC node can be running or not). You may point it to a galera.cache or gcache.page.X file. Below is an example of the default report without additional options used.

$ gcache-inspector --file node2/data/galera.cache 
=== gcache-inspector 0.2.5 — GCache Summary ===
File:    /data/sandboxes/pxc_msb_pxc8_4_10/node2/data/galera.cache
Size:    128.00 MB
Version: 2   UUID: 62f2ad43-8de5-11f1-9fb8-8bae2a687fc9
Seqno (retained):  2 – 4533  (4532 in cache)
Synced:  yes   Offset: 1704
Encrypted: no
Flavor:  PXC / MySQL 8.x

Write-sets found:  4498  4498 retained, 0 older/overwritten
Decodable seqnos:  3 – 4533  (4498 write-sets; pick one with --seqno)
Time range:        2026-08-01 22:12:58 – 2026-08-05 23:04:02 CEST  (span 96h51m4s, newest 12d ago)
DDL statements:    216
GTID events seen:  0
Rows changed:      513758  (95.20 MB)  [all write-sets]

Top 10 tables by row activity:
  table                                      insert   update   delete    ddl       size
  ───────────────────────────────────────────────────────────────────
  sbtest.sbtest1                               5039     5574       15      2        2.9M
  sbtest.sbtest32                              5053       37       18      2        0.9M
  sbtest.sbtest8                               5050       30       23      2        0.9M
  sbtest.sbtest60                              5049       31       21      2        0.9M
  sbtest.sbtest100                             5045       36       18      2        0.9M
  sbtest.sbtest10                              5054       26       19      2        0.9M
  sbtest.sbtest85                              5055       25       17      2        0.9M
  sbtest.sbtest35                              5042       40       14      2        0.9M
  sbtest.sbtest87                              5045       37       14      2        0.9M
  sbtest.sbtest33                              5048       30       18      2        0.9M

By using the --detail parameter, the tool will show per-individual sequence number details, i.e.:

$ gcache-inspector --file node2/data/galera.cache --detail --no-summary --seqno 100-105

=== Write-sets ===
  seqno 100           453632 B 2026-08-01 22:14:39  RELEASED  sbtest.sbtest16[i:2281 u:0 d:0]
  seqno 101              256 B 2026-08-01 22:14:39  RELEASED  1 DDL: CREATE INDEX k_16 ON sbtest16(k); sbtest.sbtest16[i:0 u:0 d:0]
  seqno 102              432 B 2026-08-01 22:14:39  RELEASED  1 DDL: CREATE TABLE sbtest17(; sbtest.sbtest17[i:0 u:0 d:0]
  seqno 103           540664 B 2026-08-01 22:14:39  RELEASED  sbtest.sbtest17[i:2719 u:0 d:0]
  seqno 104           453632 B 2026-08-01 22:14:39  RELEASED  sbtest.sbtest17[i:2281 u:0 d:0]
  seqno 105              256 B 2026-08-01 22:14:39  RELEASED  1 DDL: CREATE INDEX k_17 ON sbtest17(k); sbtest.sbtest17[i:0 u:0 d:0]

The above example shows that a transaction committed with the sequence number 100 has inserted 2281 rows into the table sbtest16 and did not update or delete any rows.

To see exactly what a given transaction was about, the --decode-rows option prints the whole event details. For example, it’s possible to see what rows were changed under seqno 4252:

$ gcache-inspector --file node2/data/galera.cache --decode-rows --no-summary --seqno 4252

-- seqno 4252 at 2026-08-01 22:15:32 (816 bytes) RELEASED
### DELETE FROM `sbtest`.`sbtest41`
### WHERE
###   @1= 1843
###   @2= 3562
###   @3= '55824051154-00248428540-43829027453-18090470997-77687189613-13487855838-34568671126-01577127301-81564593132-49010886470'
###   @4= '09475435259-72703365718-14065084029-80972334150-38881617733'
### INSERT INTO `sbtest`.`sbtest41`
### SET
###   @1= 1843
###   @2= 4879
###   @3= '37041074202-54426174421-76052854404-43175485519-62755971707-75981734496-81616509419-51624022546-52075561216-00090498892'
###   @4= '46023326729-33104312594-23620888475-28615232417-62781559343'

A DDL investigation example

Handling DDLs in Galera replication may be quite confusing. Even if, for instance, an ALTER query fails on the writer, it still gets replicated, causing surprising errors on the peer members, similar to this:

2026-08-05T21:04:02.137267Z 11 [ERROR] [MY-010584] [Repl] Replica SQL: Error 'Table 'sbtest.foo' doesn't exist' on query. Default database: 'sbtest'. Query: 'alter table foo engine=innodb', Error_code: MY-001146
2026-08-05T21:04:02.137334Z 11 [Warning] [MY-000000] [WSREP] Event 1 Query apply failed: 1, seqno 4533
2026-08-05T21:04:02.138503Z 0 [Note] [MY-000000] [Galera] Member 0(przemek-d1) initiates vote on 62f2ad43-8de5-11f1-9fb8-8bae2a687fc9:4533,aebcd4f61a8a51aa:  Table 'sbtest.foo' doesn't exist, Error_code: 1146;

Although such an event normally produces a GRA file to let us investigate, like in this case: GRA_11_4533_v2.log, now we can also look into the cache file for the same (here the SKIPPED flag confirms it was not applied):

$ gcache-inspector --file node2/data/galera.cache --detail --no-summary --seqno 4533

=== Write-sets ===
  seqno 4533             256 B 2026-08-05 23:04:02  RELEASED|SKIPPED  1 DDL: alter table foo engine=innodb; sbtest.foo[i:0 u:0 d:0]

Encrypted Galera Cache

For strict security compliance cases, Percona XtraDB Cluster allows encrypting the Gcache files. The tool allows inspection of encrypted files as well, if the encryption key or vault credentials are provided. But there is one caveat here. A regular, non-encrypted cache file will contain all replicated transactions immediately. Whilst the encrypted one will not show anything new until the encryption in-memory cache is filled or synced during shutdown. Therefore, new transactions are expected to appear in the encrypted cache file with a delay.

Note: the tool does not support encryption available in MariaDB Galera Cluster Enterprise Edition (no source code access).

An example output against an encrypted file:

$ gcache-inspector --file node1/data/galera.cache --keyring-file /opt/mysql/pxc8.4.10/keyring/component_keyring_file
=== gcache-inspector 0.2.5 — GCache Summary ===
File:   node1/data/galera.cache
Size:    128.00 MB
Version: 2   UUID: 62f2ad43-8de5-11f1-9fb8-8bae2a687fc9
Seqno (retained):  4395 – 4533  (139 in cache)
Synced:  yes   Offset: 1776
Encrypted: yes — decrypted   (enc version 1)
Master key: GaleraKey-d6945297-8f7a-11f1-9533-7a5bf82f508c@62eff734-8de5-11f1-b956-7f3a785ad5e2-1
Key source:/opt/mysql/pxc8.4.10/keyring/component_keyring_file (GaleraKey-d6945297-8f7a-11f1-9533-7a5bf82f508c@62eff734-8de5-11f1-b956-7f3a785ad5e2-1)
Cipher:    AES-256-ctr-file, clear below 0x400, counter from 0x0 [CTR unwrap (zero IV), keyring bytes]
Freshness: on a live node the encrypted file lags the in-memory cache (write-back page cache; flushed on eviction/shutdown)
Flavor:  PXC / MySQL 8.x

Write-sets found:  127  127 retained, 0 older/overwritten
Decodable seqnos:  4395 – 4533  (127 write-sets; pick one with --seqno)
Time range:        2026-08-01 22:15:32 – 2026-08-05 23:04:02 CEST  (span 96h48m30s, newest 12d ago)
DDL statements:    1
GTID events seen:  0
Rows changed:      5782  (2.08 MB)  [all write-sets]

Top 10 tables by row activity:
  table                                      insert   update   delete    ddl       size
  ───────────────────────────────────────────────────────────────────
  sbtest.sbtest1                                  2     5545        1      0        2.0M
  sbtest.sbtest58                                 5        1        2      0        0.0M
  sbtest.sbtest64                                 3        2        2      0        0.0M
  sbtest.sbtest52                                 4        1        2      0        0.0M
  sbtest.sbtest14                                 2        3        1      0        0.0M
  sbtest.sbtest26                                 2        2        2      0        0.0M
  sbtest.sbtest6                                  2        2        2      0        0.0M
  sbtest.sbtest98                                 3        1        2      0        0.0M
  sbtest.sbtest97                                 4        0        2      0        0.0M
  sbtest.sbtest68                                 1        4        0      0        0.0M

Summary

Although in most cases, problems with PXC/Galera replication can be successfully investigated based on error logs, binary logs, and GRA files, there may be more complex cases where you may want to look inside the Galera cache files. Or simply for experimenting or to allow better understanding of how it works. I hope gcache-inspector will help you do this. The tool is available as GPLv3, with Go source code and binary packages ready to play with on GitHub: https://github.com/PrzemekMalkowski/gcache-inspector. Demo recording: https://asciinema.org/a/1263342

If, despite acquiring details, you face undersized gcache or other reasons causing nodes to keep falling back to SST, Percona’s engineers can help you tackle those problems. Talk to us about a cluster health review https://www.percona.com/contact-us/

Additional references about Galera Cache can be found in the following blog posts by other Percona engineers:
https://www.percona.com/blog/all-you-need-to-know-about-gcache-galera-cache/
https://www.percona.com/blog/no-sst-node-rejoins/
https://www.percona.com/blog/understanding-ist-donor-selected/
https://www.percona.com/blog/gcache-and-record-set-cache-encryption-in-percona-xtradb-cluster-part-one/

The article was written by a human

The post Stop guessing at gcache: inspect Galera/PXC write sets with gcache-inspector appeared first on Percona.

Stop guessing at gcache: inspect Galera/PXC write sets with gcache-inspector appeared first on MariaDB.org

Source

What a careful MySQL to MariaDB migration still misses appeared first on MariaDB.org

This is not a changelog read-aloud. The deck is built the way we run client engagements: every claim tied to a measurement source, every architecture drawn with its RPO/RTO consequences, and a clear statement of where each feature does not apply. Fill in the short form at the bottom of this page and the full deck lands in your inbox.

Here is what the MariaDB 12.3 LTS webinar covers, and why we think the timing matters.

Why the 12.3 LTS generation matters right now

MariaDB 10.6 LTS reached end of life in July 2026. If you are still running it, you are now accumulating unpatched CVEs on your primary OLTP tier — that alone justifies an hour with the MariaDB 12.3 LTS webinar material. The practical question for most estates is no longer whether to move, but whether to land on 11.8 LTS or 12.3 LTS.

The support windows are not symmetrical, and this catches people out. MariaDB 12.3 LTS, released Q2 2026, is maintained through 2029 on a three-year window. 11.8 LTS runs to mid-2030 — a longer runway than the newer release. The webinar opens with this decision: which workloads justify the 12.3 feature set, and which are better served by parking on 11.8 until the next LTS cycle. The MariaDB Foundation publishes the current release calendar; the deck maps it onto upgrade sequencing for mixed estates.

MariaDB’s release model has also changed shape: rolling GA releases punctuated by LTS milestones (the .3 releases). If your change-management process still assumes the old cadence, part one of the webinar is the correction.

What the MariaDB 12.3 LTS webinar covers

We built the MariaDB 12.3 LTS webinar for architects and DBAs who own upgrade decisions, not for a general audience.

The MariaDB 12.3 LTS webinar runs five parts across 60 minutes:

• The platform in 2026 — release model evolution, LTS support windows, upgrade planning for estates coming off 10.6.
• Performance — the InnoDB-based binary log, optimizer refinements, and how to validate vendor benchmark claims against your own workload.
• Scalability — parallel replication modes, MaxScale routing, storage engine selection, and analytics offload.
• High availability — Galera Cluster in 12.3, the RPO/RTO decision matrix, MaxScale automated failover, backup and point-in-time recovery.
• AI and vector search — mHNSW indexes, hybrid search with Reciprocal Rank Fusion, and RAG architectures that keep embeddings transactionally consistent with source rows.

Performance: one engine, one recovery protocol, one flush discipline

The headline change in the 12.3 generation is the InnoDB-based binary log. MariaDB has collapsed the old dual-log architecture — binlog and InnoDB redo log, each with its own fsync discipline and a two-phase commit stitching them together — into a single recovery protocol owned by InnoDB.

MariaDB reports roughly 4× write throughput on heavy workloads with this change, with crash safety inherited from InnoDB’s redo protocol. We treat that number the way we treat every vendor benchmark: as a hypothesis to test. The webinar dedicates a section to validation methodology — replay your own production statement digests on the candidate version, hold sync_binlog and innodb_flush_log_at_trx_commit constant across runs, measure P95/P99 latency rather than throughput alone, and run at production concurrency with the thread pool configured the way you actually deploy it.

The optimizer work in 12.3 is quieter but operationally useful: reverse-ordered scans, loose index scan with DESC keys, and improved virtual-column costing. Two new hints — JOIN_FIXED_ORDER and MAX_EXECUTION_TIME — give you per-statement plan control, and ANALYZE FORMAT=JSON now surfaces actual row counts against estimates, which is the fastest way to catch a misestimation before it becomes a 2 a.m. page. Our standing advice, repeated in the deck: capture your top-N statement digests and plans before the upgrade, replay them in staging, and diff the plans before cutover. If you want the monitoring side of that discipline, we published the 20 MariaDB metrics we track in every engagement.

Scalability: the levers most teams never pull

MariaDB GTID is not MySQL GTID — the domain-server-sequence format is incompatible, which matters for anyone running mixed fleets or mid-migration topologies. From that foundation, the webinar works through the two parallel-apply modes: conservative, which parallelizes within group commit boundaries, and optimistic, which applies speculatively and rolls back on conflict.

The change we consider most significant for DR design: since 12.1, parallel replication works into Galera nodes. Async replicas feeding a DR cluster were previously throttled to single-threaded apply, and lag on that link was a standing RPO risk. That constraint is gone, and the deck shows the resulting cluster-to-cluster DR topology.

Beyond replication, part three covers MaxScale — read/write split, causal reads, transaction replay, and an honest note on its Business Source License terms — plus storage engine selection: InnoDB for OLTP, MyRocks where write amplification and compression dominate, Aria for temp tables, S3 for cold archives, ColumnStore for analytics, and Spider for sharding. If you run MariaDB on Kubernetes or cloud VMs, our guide to tuning MariaDB for cloud and containerized environments pairs well with this section.

High availability: pick your RPO before you pick your topology

Part four opens with the decision matrix we use in architecture reviews: asynchronous replication gives you seconds of RPO and seconds-to-minutes of RTO; semi-sync closes RPO to approximately zero at a latency cost; Galera gives you zero RPO within the cluster and seconds of RTO, in exchange for three-node quorum, primary keys on every InnoDB table, and a network you can trust.

One trade-off in 12.3 deserves more attention than it has received: the InnoDB-based binary log and Galera are mutually exclusive. wsrep cannot intercept the atomic commit path, so cluster nodes keep the legacy binlog — meaning the 4× write-path improvement and Galera’s synchronous replication cannot be combined on the same node. The webinar treats this as a first-class design input, not a footnote. We covered the surrounding architecture decisions in MariaDB high availability in 12.3 LTS and in our piece on building fault-tolerant MariaDB infrastructure at internet scale.

The section closes on operations: MaxScale automated failover via mariadbmon, switchover for planned maintenance, hot physical backups with mariabackup (full plus incremental, with the --prepare step), and point-in-time recovery by replaying the binlog to a target GTID. Two cautions from the deck worth repeating here. First: verify that your mariabackup version and any CDC tooling that reads binlog files directly are certified for the new binlog format before you enable it. Second: an untested backup is a hypothesis, not a recovery plan — measure your verified restore time, and rehearse failover quarterly.

The AI-native database: mHNSW, hybrid search, and honest boundaries

MariaDB now ships a native VECTOR column type with an mHNSW index — a modified HNSW graph — and distance functions for both Euclidean and cosine metrics. The 12.3 generation pushes distance computation down into the storage layer and adds extrapolation-based graph pruning; the practical tuning lever is M, the graph connectivity parameter, which trades recall against memory and latency. A retrieval query stays plain SQL:

SELECT doc_id,
       VEC_DISTANCE_COSINE(embedding, @query_vec) AS distance
FROM   knowledge_chunks
ORDER  BY distance
LIMIT  10;

The webinar’s RAG argument is transactional rather than fashionable: embeddings and their source rows commit in one transaction, which eliminates the synchronization drift you accept the moment you bolt a separate vector store onto your OLTP database. Hybrid search fuses keyword and vector results with Reciprocal Rank Fusion, and the deck walks the full architecture. For the index internals, see our earlier post on how vector indexes work in MariaDB and the MariaDB Vector documentation.

And the boundary, stated plainly because vendor decks rarely do: MariaDB Vector is credible up to tens of millions of vectors. At billions-scale, or where GPU-accelerated indexing dominates the workload, a dedicated vector platform is the right call. We say this as a vendor-neutral consultancy — right-size first.

Two reference architectures you can lift directly

The MariaDB 12.3 LTS webinar deck closes its infrastructure arc with two worked topologies. The performance-first design: a 12.3 primary on the InnoDB binlog, two replicas on parallel apply, a delayed replica as a fat-finger safety net, and scheduled backups — seconds of RPO, sub-minute RTO. The availability-first design: a three-node Galera cluster on the legacy binlog with single-writer routing through MaxScale, feeding an async DR cluster that uses the 12.1+ parallel-apply capability. Both diagrams carry version labels and the reasoning for each choice.

Download the MariaDB 12.3 LTS webinar deck (free)

Fill in the form below to get the MariaDB 12.3 LTS webinar PDF — we’ll show the download link immediately and email you a copy. We’ll occasionally send you technical material like this; no spam, and never a sales sequence you didn’t ask for. The MariaDB 12.3 LTS webinar PDF is free, and the download link arrives instantly.

Please enable JavaScript in your browser to complete this form.

First Name *

Last Name *

First Last Name

Email *

URL

Download the Webinar PDF

Planning a MariaDB 12.3 upgrade? Talk to us

MinervaDB runs MariaDB in production for enterprises worldwide — consulting, 24×7 support, and remote DBA across the full lifecycle. If the MariaDB 12.3 LTS webinar leaves you with questions — a 10.6 exit plan, a Galera design, a MariaDB Vector evaluation — and you would benefit from engineers who do this every week, contact us. We’ll tell you honestly if 11.8 is the better landing zone for your workload — and, as always, test everything in staging before it touches production, and keep your DR posture rehearsed.

MariaDB 12.3 LTS Webinar: Performance, Scalability, High Availability & the AI-Native Database appeared first on MariaDB.org

Continue reading “Seravo becomes a Silver Sponsor of MariaDB Foundation”

Seravo becomes a Silver Sponsor of MariaDB Foundation appeared first on MariaDB.org

Seravo becomes a Silver Sponsor of MariaDB Foundation appeared first on MariaDB.org

If you work or study with open source software in Uruguay, this one is for you. It’s a whole day of learning, with the people who build and run these systems every day. No sales pitch, just good technical content and a community that likes to share what it knows.

Let me tell you what to expect, and why it’s worth a spot on your calendar.

What is Percona University?

Percona has run these events for years across the world, including a few stops in South America already. This is the third time it will be held in Uruguay. The idea is simple: bring the open-source database community together to share knowledge for free.

The word “University” is the important part here. This is an educational event, and not a sales one. The talks are technical, and the goal is that you walk out having learned something you can use daily.

You don’t need to be a Percona customer, and you definitely don’t need to be an expert. You just need to be curious about open source databases and want to spend a day getting better at them. Bring your questions, bring a notebook and your laptop, and plan to stay for the conversations between sessions.

Who is it for and why you should come

Come if you work with PostgreSQL, MySQL, MongoDB, or anything in the open source data world. It’s a great place for DBAs, backend developers, SREs, students, and people just getting started to come together and learn from each other. The talks cover real problems: performance tuning, upgrades, monitoring, the things we all fight with in production.

Networking with peers is what I like the most about events like this. You get to meet the people behind the tools, ask engineers the questions you’ve been sitting on for months, in person, and actually talk it through. You meet other folks in Uruguay doing the same work you do, and those hallway conversations are worth as much as the talks themselves.

Open source is built by the community, and it’s nice to see that community in person. We don’t get a full day of this in Montevideo very often at all, so it’s worth showing up!

The details

Here’s what you need to know:

• When: September 23rd, 2026, from 09:00 to 18:00
• Where: Regency Way Montevideo Hotel. Av Gral Rivera 3377
• Cost: FREE!
• Language: talks will be in English and Spanish, with slides you can review afterward

What’s on the agenda

• Opening Session, State of Open Source Database Ecosystem. Peter Zaitsev
• Beyond “Postgres is up”: detecting failures and performance tuning with Coroot. Nikolay Sivko / Agustín Gallego
• bpftrace 301: tracing variables life cycle for advanced bug investigations. Marcos Albe
• TiDB 101. Fernando Ipar
• MySQL 8.4 Upgrade: Best Practices For Zero-Surprise Prod Migration. Fernando Mattera
• From Reactive to Ready: How AI Changed My Work as a Service Delivery Manager. Mariana Bonsignore
• 5-minute lightning talks at the end… and there are still some available slots for talks, so submit yours!

We’re lining up talks from Percona engineers and guest speakers from the region. Expect a mix of topics, with plenty of practical hands-on content. Since it runs all day, there’s room for deeper sessions than a normal meetup allows, plus breaks to grab a coffee and keep talking. The full agenda will live on the Eventbrite page (we’ll keep adding talks as they are confirmed).

How to register

Registration is free on Eventbrite, and it’s quick.

Register here:

https://www.eventbrite.com/e/percona-university-montevideo-uruguay-tickets-1998244427875

Grab your spot early, since seats are limited.

In short, Percona University Montevideo is a full day to learn, meet the community, and get better at the open source databases we all rely on. It’s happening on September 23rd, and everyone is welcome to attend.

Register, tell a coworker, tell a friend, and come say hi. We hope to see you there!

The post Percona University Comes to Uruguay appeared first on Percona.

Percona University Comes to Uruguay appeared first on MariaDB.org

Every MariaDB high availability recommendation here is version-pinned to MariaDB Community Server 12.3 LTS (with 11.8 LTS noted where behavior differs) and anchored to a named metric or system table. Test everything in a staging environment that mirrors production before applying any change, and maintain a verified backup and disaster recovery posture throughout.

What Maximum Availability Means in Measurable Terms

Maximum availability is not a product SKU — it is an engineering outcome defined by two numbers: Recovery Point Objective (RPO), the data you can afford to lose, and Recovery Time Objective (RTO), the downtime you can afford to absorb. A MariaDB high availability architecture is only as good as the RPO/RTO it can demonstrate under a live failover drill.

MariaDB 12.3 LTS lets you engineer three distinct tiers:

• RPO ≈ seconds, RTO ≈ seconds to minutes — GTID-based asynchronous replication with MaxScale automated failover.
• RPO ≈ 0 (per committed transaction), RTO ≈ seconds — semi-synchronous replication with automated failover.
• RPO = 0, RTO ≈ seconds — Galera Cluster synchronous multi-primary replication, extended across datacenters with cluster-to-cluster asynchronous replication.

Treat these tiers as measurable service levels, not marketing labels. An availability SLO of 99.99% allows roughly 52 minutes of downtime per year — a budget that a single unrehearsed failover can consume entirely. That is why every MariaDB high availability design decision below is paired with the status variable, system table, or log line that proves it works, and why the article closes with the drill program that converts configuration into demonstrated RPO/RTO.

The rest of this article builds each tier bottom-up, then combines them into the reference architecture we deploy for production MariaDB high availability engagements.

MariaDB 12.3 LTS: The Availability-Relevant Release Surface

The MariaDB 12.3 changes and improvements notes contain a dense cluster of replication and Galera work relevant to MariaDB high availability. These are the changes that matter operationally:

Carry-over from 11.8 LTS that completes the picture: slave_abort_blocking_timeout (MDEV-34857) automatically aborts long-running transactions that block the replication applier, and asynchronous rollback during crash recovery lets a recovering server accept connections before large rollbacks finish — measure recovery as startup-to-accepting-connections, not total rollback completion.

Tier 1: GTID Replication with Semi-Synchronous Commit

Standard replication remains the foundation of MariaDB high availability. On 12.3 LTS, always run it with GTID (MariaDB format: Domain-ServerID-Sequence) so failover targets can be repointed without file/position arithmetic.

Primary and Replica Configuration

# /etc/my.cnf.d/replication.cnf -- MariaDB 12.3 LTS
# Restart required for server_id and log_bin changes
[mariadb]
server_id                        = 101          # unique per server
log_bin                          = mariadb-bin
binlog_format                    = ROW
log_slave_updates                = ON
gtid_domain_id                   = 1            # distinct per replication domain
gtid_strict_mode                 = ON

# Durability on the primary: RPO depends on these two
innodb_flush_log_at_trx_commit   = 1
sync_binlog                      = 1

# Parallel applier on replicas
slave_parallel_threads           = 8            # size from workload, see below
slave_parallel_mode              = optimistic

Point the replica at the primary using GTID:

CHANGE MASTER TO
    MASTER_HOST     = 'primary.db.internal',
    MASTER_USER     = '${REPL_USER}',
    MASTER_PASSWORD = '${REPL_PASSWORD}',
    MASTER_USE_GTID = slave_pos,
    MASTER_SSL      = 1;

START REPLICA;

Asynchronous replication alone leaves a nonzero RPO: transactions committed on the primary but not yet shipped are lost on failover. Semi-synchronous replication, built into MariaDB Server (no plugin installation required since 10.3), closes that gap by refusing to acknowledge a commit to the client until at least one replica has received the event:

-- Primary (dynamic, no restart)
SET GLOBAL rpl_semi_sync_master_enabled    = ON;
SET GLOBAL rpl_semi_sync_master_timeout    = 2000;        -- ms; then falls back to async
SET GLOBAL rpl_semi_sync_master_wait_point = AFTER_SYNC;  -- lossless wait point

-- Each replica (dynamic, no restart)
SET GLOBAL rpl_semi_sync_slave_enabled = ON;

Figure 3: Semi-synchronous commit path in a MariaDB high availability replication pair — the RPO guarantee and its fallback behavior.

Verify the guarantee is actually active — a silent fallback to asynchronous mode is the classic semi-sync failure mode:

SHOW GLOBAL STATUS
WHERE Variable_name IN
    ('Rpl_semi_sync_master_status',
     'Rpl_semi_sync_master_yes_tx',
     'Rpl_semi_sync_master_no_tx');

Alert when Rpl_semi_sync_master_status reads OFF or Rpl_semi_sync_master_no_tx grows: both mean commits are proceeding without the semi-sync guarantee. For replica lag, do not rely on Seconds_Behind_Master alone — track gtid_slave_pos against gtid_binlog_pos on the primary, and on 12.3 monitor per-worker state in information_schema.SLAVE_WORKER_THREADS when tuning slave_parallel_threads. For replica provisioning at scale, see our note on transferring backups efficiently to a MariaDB replica.

Sizing the Parallel Applier

slave_parallel_threads is not a bigger-is-better knob. In optimistic mode the applier speculatively executes transactions in parallel and rolls back on conflict, so a write pattern with hot rows can spend more time retrying than applying. Size it empirically: start at 4–8 threads, replay production-shaped load, and compare the drain rate of gtid_slave_pos against the primary while watching the retry counters in SHOW GLOBAL STATUS LIKE ‘Slave_retried_transactions’.

On replicas dedicated to failover (rather than read scaling), keep innodb_flush_log_at_trx_commit = 1 and sync_binlog = 1 as well — a failover target with relaxed durability quietly converts your semi-sync RPO ≈ 0 design back into a data-loss scenario the moment it is promoted.

Tier 2: Galera Cluster — Synchronous Multi-Primary

MariaDB Galera Cluster is the core of any zero-data-loss MariaDB high availability design: it delivers RPO = 0 inside a datacenter through certification-based synchronous replication: a transaction commits only after its write set has been replicated to and certified by every node in the cluster. A minimum of three nodes preserves quorum through any single node failure.

Figure 1: MariaDB high availability reference topology — three-node Galera Cluster behind MaxScale automated failover.

Cluster Configuration on 12.3 LTS

Note MDEV-38744: from 12.3, server packages no longer pull the Galera provider — install the galera-4 package explicitly on every node.

# /etc/my.cnf.d/galera.cnf -- MariaDB 12.3 LTS, Galera 4
# Restart required for wsrep_provider changes
[mariadb]
wsrep_on                     = ON
wsrep_provider               = /usr/lib64/galera-4/libgalera_smm.so
wsrep_cluster_name           = prod_cluster_dc1
wsrep_cluster_address        = gcomm://10.0.1.11,10.0.1.12,10.0.1.13
wsrep_node_name              = db-dc1-node1
wsrep_node_address           = 10.0.1.11

# Galera requirements
binlog_format                = ROW
innodb_autoinc_lock_mode     = 2
log_slave_updates            = ON            # required for cluster-to-cluster replication

# Applier parallelism and 12.3 retry behavior
wsrep_slave_threads          = 8
wsrep_applier_retry_count    = 4             # new in 12.3, MDEV-36077

# State transfer
wsrep_sst_method             = mariabackup
wsrep_sst_auth               = ${SST_USER}:${SST_PASSWORD}

# Provider tuning: size gcache so short outages recover via IST, not SST
wsrep_provider_options       = "gcache.size=4G;gcs.fc_limit=256"

Size gcache.size from your write volume: it must hold more write sets than accumulate during your longest tolerated node outage, because a node that finds its missing transactions in the donor gcache rejoins via Incremental State Transfer (IST) instead of a full State Snapshot Transfer (SST).

MariaDB 12.3 makes IST cheaper again by skipping unnecessary foreign key checks during the transfer (MDEV-34822).

Proving Cluster Health

SHOW GLOBAL STATUS
WHERE Variable_name IN
    ('wsrep_cluster_status',          -- must be Primary
     'wsrep_cluster_size',            -- expected node count
     'wsrep_local_state_comment',     -- Synced on a healthy node
     'wsrep_flow_control_paused',     -- fraction of time paused; alert above 0.1
     'wsrep_local_recv_queue_avg',    -- sustained > 0.5 means applier lag
     'wsrep_cert_deps_distance');     -- guides wsrep_slave_threads sizing

wsrep_flow_control_paused is the number that tells you the cluster is throttling writes to protect its slowest node — the mechanism behind most “Galera is slow” incidents. We covered the full diagnostic sequence in Troubleshooting Galera Cluster for Performance Issues. Remember the standing Galera constraints: InnoDB tables with primary keys only, write-set certification conflicts surface as deadlock errors to the application, and large transactions are bounded by wsrep_max_ws_size.

Two design decisions determine whether Galera delivers its RPO = 0 promise in practice. First, quorum arithmetic: always deploy an odd number of nodes (or two nodes plus a Galera arbitrator) so a network partition leaves exactly one primary component; a 50/50 split freezes both halves, which protects consistency at the cost of availability.

Second, write-conflict discipline: Galera is multi-primary, but pointing all writes at a single node through your proxy layer eliminates certification conflicts for most OLTP workloads and makes wsrep_cert_deps_distance far more predictable. Reserve true multi-node writes for workloads you have tested for conflict rate — the counter to watch is wsrep_local_cert_failures.

Multi-Datacenter Design: Parallel Galera-to-Galera Replication in 12.3

Stretching one Galera Cluster across WAN links penalizes every commit with inter-DC round trips. The production-grade pattern for MariaDB high availability across regions is one Galera Cluster per datacenter, connected by GTID-based asynchronous replication — and this is exactly where MariaDB 12.3 LTS delivers its headline improvement: the asynchronous link between two Galera Clusters can now use parallel replication (MDEV-20065).

Before 12.3, the cluster-to-cluster applier was effectively single-threaded, so a write-heavy primary cluster could permanently outrun its DR cluster. Now the replica-side applier fans out through slave_parallel_threads:

Figure 2: Multi-datacenter MariaDB high availability — one Galera Cluster per DC linked by parallel asynchronous replication (new in 12.3).

-- On one node of the DR cluster (DC2)
-- Each cluster keeps a distinct gtid_domain_id (e.g., DC1 = 1, DC2 = 2)
SET GLOBAL slave_parallel_threads = 8;
SET GLOBAL slave_parallel_mode    = optimistic;

CHANGE MASTER TO
    MASTER_HOST     = 'dc1-vip.db.internal',
    MASTER_USER     = '${REPL_USER}',
    MASTER_PASSWORD = '${REPL_PASSWORD}',
    MASTER_USE_GTID = slave_pos,
    MASTER_SSL      = 1;

START REPLICA;

-- Verify parallel apply is active and lag is draining
SHOW REPLICA STATUSG
SELECT @@gtid_slave_pos, @@gtid_binlog_pos;

Every node in each cluster must run log_slave_updates = ON so replicated transactions re-enter the local cluster’s write-set replication, and the replication user should exist on all nodes of the source cluster so the link can be re-pointed after a node failure. Combine this with slave_abort_blocking_timeout (11.8+) on the DR side so a stray analytical query cannot stall the applier indefinitely.

If you do stretch a single Galera Cluster across sites instead — legitimate for metro-distance, low-latency links — declare WAN topology to the provider with gmcast.segment in wsrep_provider_options (a distinct segment ID per site), so Galera relays traffic once per segment instead of once per node and picks IST/SST donors within the local segment. Measure the commit-latency cost before choosing this path: every transaction pays the inter-site round trip at certification time, which is precisely what the cluster-per-DC design above avoids.

Automated Failover with MariaDB MaxScale 25.10

Replication and Galera provide redundancy; MariaDB high availability additionally requires something to detect failure and redirect traffic in seconds. MariaDB MaxScale 25.10 (current release 25.10.3, GA 2026-06-15) pairs a readwritesplit router with automatic failover driven by the MariaDB Monitor:

# /etc/maxscale.cnf -- MaxScale 25.10.3
[Replication-Monitor]
type                     = monitor
module                   = mariadbmon
servers                  = db1,db2,db3
user                     = ${MAXSCALE_USER}
password                 = ${MAXSCALE_PASSWORD}
replication_user         = ${REPL_USER}
replication_password     = ${REPL_PASSWORD}
monitor_interval         = 2s
auto_failover            = true
auto_rejoin              = true
failcount                = 3            # 3 x 2s = failure declared after ~6s

[RW-Split-Service]
type                     = service
router                   = readwritesplit
servers                  = db1,db2,db3
user                     = ${MAXSCALE_USER}
password                 = ${MAXSCALE_PASSWORD}
transaction_replay       = true         # replays in-flight transactions after failover

With auto_failover enabled, MaxScale promotes the most up-to-date replica when the primary fails, repoints the remaining replicas, and auto_rejoin re-subordinates the old primary when it returns — no split brain, no manual CHANGE MASTER. Planned maintenance uses a zero-data-loss switchover instead:

# Verify topology state before acting
maxctrl list servers

# Planned promotion of db2 (waits for replicas to catch up)
maxctrl call command mariadbmon switchover Replication-Monitor db2

# Validate: db2 is Master, others are Slave, Running
maxctrl list servers

Two disclosures belong in every MaxScale design. First, licensing: MaxScale is distributed under the Business Source License (BSL) — not OSI open source — and production use beyond the license grant requires a MariaDB subscription. Second, vendor neutrality: ProxySQL and HAProxy (with an external failover orchestrator) remain capable open-source alternatives; MaxScale earns its place when you need integrated automated failover, transaction replay, and Galera-aware routing in one component. Choose per requirement, not by default.

Do not let the proxy become the new single point of failure. Run at least two MaxScale instances — either behind keepalived with a virtual IP, or using MaxScale cooperative monitoring so only one instance performs failover actions at a time — and monitor them with the same seriousness as the database tier. The availability of a MariaDB high availability stack is the availability of its weakest routing component.

Choosing Your MariaDB High Availability Topology

There is no single correct MariaDB high availability topology — there is a correct topology per RPO/RTO requirement, write pattern, and operational maturity. The matrix below summarizes the trade-offs on MariaDB 12.3 LTS:

RTO figures are illustrative planning values, not benchmarks — validate them with failover drills on your own workload and infrastructure.

Upgrade Risks to Clear Before 12.3 LTS

An upgrade executed carelessly is itself an availability incident, so treat the move to 12.3 LTS as part of the MariaDB high availability program. Three 12.3 changes have direct availability implications:

• innodb_snapshot_isolation now defaults to ON. REPEATABLE READ behaves as true snapshot isolation, and applications written against the previous semantics can see new conflict errors under concurrency. Load-test transaction-heavy paths on a 12.3 clone while watching Innodb_row_lock_% status counters and application error rates by SQL digest. Rollback is dynamic, no restart: SET GLOBAL innodb_snapshot_isolation = OFF;
• Three new reserved words: CONVERSION, ST_COLLECT, TO_DATE. Pre-flight scan information_schema.COLUMNS, TABLES, and ROUTINES for these identifiers before upgrading; rename rather than quote.
• A known replication issue affects master_use_gtid settings during upgrade to 12.3. Capture SHOW REPLICA STATUS (Using_Gtid, Gtid_IO_Pos) and SELECT @@gtid_slave_pos, @@gtid_binlog_pos; on every replica before upgrading, upgrade replicas before the primary, and explicitly re-assert MASTER_USE_GTID after the upgrade instead of trusting persistence.

Also plan the support horizon: under the MariaDB maintenance policy, Community LTS binaries are supported for three years from GA — 12.3 to roughly mid-2029, 11.8 to 2028-06-04 — with two further years of source-only fixes. Estates still on 10.6 passed community binary EOL on 2026-07-06 and need a dated migration plan.

Backups Are Part of the MariaDB High Availability Design

Replication multiplies data; it does not protect it. A DROP TABLE, an application bug, or ransomware replicates to every node and every datacenter in milliseconds, which is why a MariaDB high availability architecture without tested backups is incomplete. Run mariabackup for physical backups from a designated Galera node or replica (the same tooling already serving as your SST method), keep binary logs for point-in-time recovery between backup sets, and store at least one copy outside the failure domain of both datacenters.

The metric that matters is not backup success rate — it is restore time, measured by actually restoring: schedule a quarterly restore drill and record the wall-clock duration against your RTO budget. Backup validation belongs in the same review cadence as failover drills, because the two protect against different failure classes.

Prove It: Drills and Standing Telemetry

A MariaDB high availability stack that has not survived a rehearsed failure is an assumption, not an architecture. Institutionalize quarterly MariaDB high availability drills: kill the primary and measure detection-to-promotion from MaxScale logs; kill a Galera node and measure IST rejoin duration from the error log and wsrep_local_state_comment transitions; fail an entire DC and measure cluster-to-cluster catch-up by watching gtid_slave_pos converge.

Alert continuously on wsrep_cluster_status, wsrep_flow_control_paused, Rpl_semi_sync_master_status, and GTID position drift — these four catch the large majority of MariaDB high availability regressions before customers do.

Conclusion

MariaDB 12.3 LTS is the strongest availability release in the MariaDB Server line to date: parallel cluster-to-cluster replication removes the last structural bottleneck in the dual-DC Galera reference architecture, applier retry and cheaper IST harden day-2 operations, and the surrounding LTS window gives you a platform stable through 2029. Engineered deliberately — GTID everywhere, semi-sync or Galera where RPO demands it, MaxScale or an open-source proxy layer for sub-15-second failover, and drills that prove the numbers — MariaDB high availability on 12.3 LTS supports RPO = 0 designs with single-digit-second in-DC recovery.

MinervaDB builds, audits, and operates these architectures for enterprises worldwide — from topology design and failover automation to 24×7 monitoring with the exact telemetry described above. If you are planning a 12.3 LTS upgrade or a high availability redesign, our MariaDB Remote DBA and Support team can help you get there with measured RPO/RTO outcomes. Contact us at contact@minervadb.com.

MariaDB High Availability: Maximum Availability Solutions in MariaDB 12.3 LTS appeared first on MariaDB.org

Continue reading “Extending MariaDB with Native Aggregate Plugins: Laying the Groundwork for HyperLogLog”

Extending MariaDB with Native Aggregate Plugins: Laying the Groundwork for HyperLogLog appeared first on MariaDB.org

Extending MariaDB with Native Aggregate Plugins: Laying the Groundwork for HyperLogLog appeared first on MariaDB.org

The headline for Redis 8.10 performance is that this is a memory-and-durability release wearing a performance badge. The two changes that will alter how you size and operate a Redis fleet are compact hashes, which store field names once across keys that share a schema, and the BACKUP command family, which finally gives you node-side online backups built on multi-part AOF instead of a hand-rolled BGSAVE-and-copy script. Everything else — the new list, set and stream commands, the JSONPath expansion, the search timeout controls — is genuinely useful, but it will not change your capacity plan.

This post covers the three changes that matter operationally, demonstrates each with runnable commands, states who is affected, and closes with a dated, version-pinned upgrade stance. It is written for engineers running Redis as production infrastructure, not as a changelog retelling.

Step zero: establish exactly what you are running

Before any of this applies, confirm the engine and the version. “Redis” in a real estate is frequently Redis OSS 7.2.4 under BSD, Redis 8.x under the tri-license, Valkey, or a managed service running one of them. Everything downstream — feature availability, licensing posture, upgrade path — depends on the answer.

redis-cli INFO server | grep -E 'redis_version|redis_mode|os|arch_bits|io_threads_active'
redis-cli INFO memory | grep -E 'used_memory_human|maxmemory_policy|mem_allocator'
redis-cli CONFIG GET appendonly appendfsync repl-diskless-sync save

Two facts to record alongside the version:

• Licensing. Redis Open Source 8.0.0 and later ships under a tri-license — RSALv2, SSPLv1, or AGPLv3, user’s choice — and that same tri-license covers the integral modules (RediSearch, RedisJSON, RedisTimeSeries, RedisBloom). Redis 7.2.x and earlier were BSD-3-Clause. This is an architecture input, not legal trivia; route formal compliance questions to counsel.
• Support horizon. As of August 2026, 8.10 is the current release. 8.0’s security support runs to 01 Dec 2026, and 7.2/7.4 to 01 Dec 2029. If you are on 8.0, you have a deadline.

Change 1: Compact hashes and the memory line item

What changed

Redis 8.10 introduces a new hash encoding that stores field names once per template, shared across every key that uses the same field layout. Redis has always had two hash encodings — listpack for small hashes and hashtable for large ones — and in both, every key carried its own copy of every field name. For the single most common Redis modelling pattern in the wild (one hash per entity, identical fields across millions of entities), that repetition was pure overhead.

Consider a million user records stored as user:{id} hashes with fields name, email, age, country, plan, created_at. Before 8.10, those six field-name strings were materialised a million times each. With compact hashes, they are materialised once into a template, and each key stores only its values plus a template reference.

Measuring it

Three new counters expose the encoding directly, so you can verify the saving rather than assume it:

# Distinct templates and how many keys are backed by them
redis-cli INFO stats | grep -E 'hash_templates|hash_template_keys'

# Memory consumed by the templates themselves
redis-cli INFO memory | grep used_memory_hash_templates

# Per-key verification
redis-cli OBJECT ENCODING user:1
redis-cli MEMORY USAGE user:1

The honest way to size the benefit on your data is to load a representative slice into a scratch instance on both versions and compare used_memory. Do not extrapolate from a vendor headline:

# Reproducible A/B: same dataset, two instances, two versions.
# Run on a scratch host. Never point this at production.

for PORT in 6388 6389; do
  redis-cli -p "$PORT" FLUSHALL
done

# Generate 1M schema-identical hashes and pipe them in
python3 - <<'PY' > /tmp/users.redis
for i in range(1, 1_000_001):
    print(f"HSET user:{i} name u{i} email u{i}@example.com age {20 + i % 50} "
          f"country IN plan pro created_at 2026-08-15")
PY

redis-cli -p 6388 --pipe < /tmp/users.redis   # 8.8 instance
redis-cli -p 6389 --pipe < /tmp/users.redis   # 8.10 instance

for PORT in 6388 6389; do
  echo -n "port $PORT: "
  redis-cli -p "$PORT" INFO memory | grep -E '^used_memory:'
done

Report the delta from your own run. That number is the one your capacity plan can spend.

The bulk-load path: HIMPORT

HIMPORT (new in 8.10) is a connection-scoped session that declares field names once and then sends values only. It reduces network bytes and per-command parsing work, and it lands the keys directly in the compact encoding. The fieldset is local to the connection and is discarded when the connection closes or on RESET.

# Declare an ordered fieldset named "u" on this connection
HIMPORT PREPARE u name email age

# Send values only — field names are never repeated on the wire
HIMPORT SET user:1 u alice alice@example.com 30
HIMPORT SET user:2 u bob   bob@example.com   25

# Housekeeping
HIMPORT DISCARD u
HIMPORT DISCARDALL

This is the right tool for migrations, warm-cache rebuilds, and ETL sinks. It is not a replacement for HSET in application code.

Configuration: what to set, and when

Three startup-relevant parameters govern how plain hashes are converted to templates during RDB load. Treat the defaults as correct until measurement says otherwise.

redis-cli CONFIG GET 'hash-rdb-load-*'

Who is affected, and where it does not help

It helps most when you have many keys with an identical field layout — session stores, user/product/device catalogues, feature stores, entity caches.

It helps least when hashes are heterogeneous (every key a different shape), when hashes are few and enormous, or when your memory is dominated by strings, sorted sets or streams rather than hashes. A wide-column model with per-key ad-hoc fields will produce template churn rather than template reuse — watch hash_templates climbing towards hash_template_keys, which is the signature of a schema too variable to benefit.

If your memory pressure is actually eviction policy or TTL hygiene rather than encoding, the fix is elsewhere. Our field guide on mastering Redis TTL covers the expiry-side of the same problem.

Change 2: The BACKUP command family — online backups without the shell scripts

What changed

Until 8.10, “backing up Redis” meant one of a small set of unsatisfying options: trigger BGSAVE and copy dump.rdb, copy the AOF directory and hope you caught a consistent manifest, or take a filesystem/EBS snapshot and accept the fork-timing risk. Redis 8.10 ships BACKUP, a container command that produces a self-contained, restorable artifact set reusing the multi-part AOF (MP-AOF) format, without stopping writes and without you managing rewrites by hand.

A sealed backup is three artefacts:

• appendonly.aof.N.base.rdb — the BASE point-in-time snapshot
• appendonly.aof.N.incr.aof — the INCR file holding writes accumulated after the snapshot
• appendonly.aof.manifest — a standalone manifest describing the set

The workflow

# 1. Open a backup window and produce a fresh BASE.
#    Works whether or not AOF persistence is enabled.
redis-cli BACKUP START

# 2. Ask which immutable files are pinned so far.
#    The data plane can start copying BASE while Redis keeps accumulating INCR.
redis-cli BACKUP LIST

# 3. Freeze the backup: hard-link the INCR, write the manifest.
#    After SEAL, BACKUP LIST also returns the INCR and manifest paths.
redis-cli BACKUP SEAL

# 4. Copy every path reported by BACKUP LIST to your backup target.
#    (rsync/aws s3 cp/azcopy — your data plane, not Redis's job.)

# 5. Release the pinned artefacts once the copy is verified.
redis-cli BACKUP CLEANUP

Two commands for observability at any point in that sequence:

redis-cli BACKUP STATUS   # inspect current backup state
redis-cli BACKUP ABORT    # cancel a backup that has not yet been sealed

Relevant settings:

Restore

Restore is a startup-only setting, preload-file, in the form ::

# Restore from a sealed MP-AOF backup
preload-file aof:/var/backups/redis/appendonly.aof.manifest

# Or from a single RDB
preload-file rdb:/var/backups/redis/dump-2026-08-15.rdb

When preload-file is set, Redis loads only that file or manifest and skips its normal appenddirname and dump.rdb loading. Once preload completes, it resumes the persistence mode you configured.

Why this matters operationally

The genuinely important design detail is that creation is decoupled from finalisation. BACKUP START and BACKUP SEAL are separate calls, so a control plane can stagger START across cluster nodes and avoid every node forking at the same instant — the classic cause of a synchronised RSS spike and a latency cliff across a whole shard group. Each node produces its BASE independently; consistency is established at the seal boundary.

Production safety caveat. A backup you have never restored is not a backup. Before this replaces your existing procedure, run a full restore into an isolated instance, validate key counts and a sample of application-level invariants, and time the restore so you have a real RTO number. Test in staging first; keep your existing backup path running in parallel for at least one full retention cycle.

A backup path is only half of a DR posture. Our Redis performance audit methodology treats restore drills as a first-class deliverable, not an afterthought.

Change 3: Replication and full-sync hardening

The replication path has been getting steady attention across the 8.x line, and 8.10 continues it. Three items are worth your attention.

Replication stream compression. Redis 8.10 adds compression of the replication stream between primaries and replicas, aimed squarely at bandwidth consumption. This matters most for cross-AZ and cross-region topologies, where replication egress is a line item on the cloud bill, and for write-heavy primaries where the output buffer is the constraint. One caveat shipped alongside it: a fix for memory reported for compressed replication clients being lower than actual consumption — evidence that this is a young code path. Watch client_output_buffer behaviour after upgrading.

I/O thread busy-looping for replica clients is fixed. If you run with io-threads enabled and have replicas attached, this was burning CPU for nothing.

Full sync under heavy write load is fixed. Full syncs that coincide with a write burst are exactly the condition under which a replica fails to catch up and re-triggers another full sync — the resync loop that turns a routine replica restart into an incident.

That builds on 8.8, where Redis eliminated RDB checksum computation on diskless transfers on the grounds that the replication link already provides integrity. Redis published a 12 GB full sync on a c8g.2xlarge dropping from 35 seconds to 11 seconds — a 68% reduction — as a result. Redis also reported pipelined SET/HSET/ZADD with an attached replica running 3% to 26% faster after reworking per-write bookkeeping in the replication feed path.

Verify the effect on your own topology rather than trusting the number:

# Full-sync counters: sync_full should stay flat in steady state.
redis-cli INFO stats | grep -E 'sync_full|sync_partial_ok|sync_partial_err'

# Replica-side lag and link health
redis-cli INFO replication

# Backlog sizing — the single most common cause of avoidable full syncs
redis-cli CONFIG GET repl-backlog-size repl-backlog-ttl
redis-cli CONFIG GET client-output-buffer-limit

If sync_partial_err is non-zero and sync_full is climbing, your backlog is too small for your write rate and no amount of release upgrading will fix it.

Redis 8.10 performance in context: the 8.4 → 8.10 curve

Redis 8.10 performance work does not stand alone: the 8.x line has been shipping measurable command-level gains every quarter. The figures below are Redis’s own published measurements, not MinervaDB benchmarks. We reproduce them here with their methodology attached because a number without methodology is marketing.

Redis 8.8 vs 8.6 — tested on AWS m7i.metal-24xl (x86) and m8g (ARM Graviton4), identical builds, official OSS spec, multiple runs:

Redis 8.6 vs 8.4 — single core, m8g.24xlarge (Graviton4):

Redis also reported 3.5M ops/sec on a single node at pipeline depth 16 with 11 I/O threads on m8g.24xlarge, and framed the 8.6 line as 5×+ the caching throughput of Redis 7.2.

How to read these numbers. Treat them as a ceiling on Redis 8.10 performance, not a forecast. They are single-command microbenchmarks at high pipeline depth on large instances. Real application workloads are dominated by round-trip time, key distribution, value size, and the presence of one pathological command in the mix. The 8.x gains are real and they are free on upgrade — but if your p99 is 8 ms because a KEYS call runs every 30 seconds, no release will help you. That class of problem is what our Redis performance troubleshooting field guide exists to localise, and the eBPF-based tracing approach is how we get per-operation visibility when SLOWLOG is not enough.

Redis 8.10 reliability fixes that are upgrade drivers on their own

Read these as security and correctness debt you are carrying if you stay put. Redis 8.10 fixes, among 30+ core items:

• ACL permission bypass in SORT, GEORADIUS, GEORADIUSBYMEMBER, XREAD and XREADGROUP. If you rely on ACLs for multi-tenant isolation, this is a confidentiality issue, not a nuisance.
• Error-reply manipulation via injected rn sequences — a protocol-level response-splitting class of bug.
• AOF load failure when the AOF file had an RDB preamble and active defragmentation was enabled. This is a restore-time failure: the worst possible time to discover it.
• Division-by-zero when active-defragmentation thresholds are misconfigured.
• Clients left permanently blocked on BLPOP, BLMOVE or BLMOVEM after a SORT ... STORE replaced the target key.
• MEMORY USAGE over-reporting, duplicate KeyMeta restoration during AOF rewrite, and RDB load robustness fixes for streams.

Redis 8.8 separately addressed five CVEs (CVE-2026-23479, CVE-2026-25243, CVE-2026-23631, CVE-2026-25588, CVE-2026-25589) covering use-after-free and invalid-memory-access classes. If you are on 8.6 or earlier, you are exposed to all of them.

Verify your ACL surface after upgrading:

redis-cli ACL LIST
redis-cli ACL GETUSER 
redis-cli ACL CAT keyspace

Bounded replies: the quietest Redis 8.10 reliability win

Unbounded replies are one of the more common ways a healthy Redis instance takes an application down — a consumer asks for “everything since last offset” after an outage and receives a multi-gigabyte reply that blows the client output buffer. Redis 8.10 adds explicit caps:

# Cap cumulative entries AND cumulative reply size on stream reads
XREAD MAXCOUNT 5000 MAXSIZE 8388608 STREAMS events $
XREADGROUP GROUP g1 c1 MAXCOUNT 1000 MAXSIZE 1048576 STREAMS events >

Set these in your consumers now. They are the cheapest reliability change in the release.

Related controls in the same theme:

• SUNIONCARD and SDIFFCARD return set-operation cardinalities without materialising the result set — the answer to “how many, roughly” that used to cost a full SUNIONSTORE.
• slowlog-entry-max-argc and slowlog-entry-max-string-len (8.8) cap what a slowlog entry retains, so a pathological command no longer bloats the slowlog itself.
• Redis Search gains a third search-on-timeout mode, RETURN_STRICT, alongside FAIL and the default RETURN — partial results with a strict post-processing timeout, for query paths where an unbounded tail is worse than an incomplete answer.

What Redis 8.10 does not change

Honest edges matter more than feature lists, and three limits bound every Redis 8.10 performance claim above:

• Command execution is still effectively single-threaded. I/O threads parallelise socket work, not command execution. One O(N) command on a large collection still stalls every other client. Data-structure and key-design discipline is unchanged — see our notes on optimizing Redis for mixed read/write workloads.
• Durability is still a configuration decision, not a default. appendfsync everysec still means you can lose roughly a second of writes. Redis’s own guidance remains: run AOF and RDB if you want data-safety comparable to a relational engine. BACKUP is a backup mechanism, not a durability upgrade.
• Cluster semantics are unchanged. Cross-slot operations, resharding mechanics and hash-tag design are what they were.
• Compact hashes are an encoding, not a schema. Redis is still schemaless; the template is an internal optimisation you should measure, not a contract you can rely on.

Redis 8.10 vs Valkey 9.1: how to read the fork

Both engines are shipping serious performance work, and both publish numbers on their own hardware with their own methodology. Valkey 9.1 (19 May 2026) reported 2.1M requests/sec on a single server with 512-byte payloads, 9 I/O threads and pipeline depth 10, a new I/O threading model worth up to 17% across workloads, XRANGE/XREVRANGE up to 30% faster, string memory down up to 20% for values under 128 bytes, and TLS certificate hot-reload for rotation without downtime.

These figures are not comparable to Redis’s. Different instance types, different payload sizes, different pipeline depths, different workload mixes. Anyone presenting a Redis-vs-Valkey throughput ratio derived from the two vendors’ blog posts is doing arithmetic on incompatible inputs.

What is comparable is the decision frame:

• License. Valkey is BSD-3-Clause. Redis 8.x is tri-licensed (RSALv2 / SSPLv1 / AGPLv3). For some enterprises this decides the question before any benchmark runs.
• Feature surface. Redis 8.x ships Search, JSON, TimeSeries and Bloom as integral, same-versioned modules. Valkey addresses this through separate BSD modules (valkey-search, valkey-json, valkey-bloom). If you use in-engine search or vectors, this is the axis that matters.
• Portability. Nothing after 7.2.4 should be assumed portable between the two. Verify per feature, per engine, per version. Compact hashes, HIMPORT and the BACKUP family are Redis-only as of this writing.
• Managed-service reality. Your cloud provider’s roadmap may decide this for you regardless of what you prefer.

We consult on both and recommend against either when it is the wrong fit. Where vector search is the actual requirement at serious scale, compare honestly against pgvector and Milvus before committing to in-engine vectors at all.

Our upgrade stance (dated: 15 August 2026)

If you are on Redis 8.8: upgrade to 8.10 within your normal patch cadence. The ACL bypass fixes and the AOF-with-defrag load failure are the drivers; compact hashes are the bonus. Low risk — 8.8 to 8.10 is an incremental step on the same line.

If you are on Redis 8.6 or 8.4: upgrade with priority. You are carrying five unpatched CVEs from the 8.8 cycle plus the 8.10 ACL fixes, and you are missing the entire 8.8 prefetch and replication-feed performance work.

If you are on Redis 8.0: plan now. Security support ends 01 Dec 2026. Treat this as a scheduled project with a restore drill, not a rolling patch.

If you are on Redis 7.2 or 7.4: the upgrade is a licensing decision as much as a technical one, because it moves you from BSD-3-Clause to the tri-license. Make that call deliberately, with counsel involved, and evaluate Valkey in the same exercise rather than defaulting.

Wait, if: you depend on a module or client library that has not certified against 8.10; you run a managed service where the version is not yours to choose; or you cannot schedule a restore drill against the new BACKUP path within the change window. In the last case, upgrade anyway but do not cut over your backup procedure until the drill is done.

Upgrade checklist

# --- BEFORE ---
# 1. Capture the baseline. You cannot claim an improvement without one.
redis-cli INFO all > /tmp/pre-upgrade-info.txt
redis-cli CONFIG GET '*'  > /tmp/pre-upgrade-config.txt
redis-cli --latency-history -i 5   # run for one full traffic cycle

# 2. Verify a restore works on the CURRENT version before changing anything.
#    Restore into an isolated instance. Never into production.

# 3. Confirm replica health and backlog sizing.
redis-cli INFO replication
redis-cli INFO stats | grep -E 'sync_full|sync_partial_err'

# --- AFTER (per node, replicas first, then failover, then old primary) ---
redis-cli INFO server   | grep redis_version
redis-cli INFO stats    | grep -E 'hash_templates|hash_template_keys'
redis-cli INFO memory   | grep -E 'used_memory:|used_memory_hash_templates'
redis-cli INFO stats    | grep -E 'sync_full|sync_partial_err'
redis-cli ACL LIST

# 4. Compare against the baseline. Then, and only then, exercise BACKUP.
redis-cli BACKUP START && redis-cli BACKUP STATUS

Every step above is read-only or additive. Nothing here drops, truncates or flushes anything — and if you adapt these commands, keep it that way. The FLUSHALL in the compact-hash benchmark earlier is scoped to a scratch host on purpose; it has no business anywhere near a production endpoint.

Frequently asked questions

Does upgrading alone improve Redis 8.10 performance? Partly. Command-level gains from the 8.6 and 8.8 cycles are free on upgrade, and compact hashes reduce memory without application changes. But if your latency comes from an O(N) command, an oversized value, or an undersized replication backlog, no release fixes it — measure first.

Is Redis 8.10 backward compatible with 8.8? Yes for the covered surface. The new commands (HIMPORT, BACKUP, LMOVEM, BLMOVEM, SUNIONCARD, SDIFFCARD) are additive, and compact hashes are an internal encoding change — application code does not change. Validate your client library’s support for the new commands before you use them.

Do compact hashes require me to change my application? No. The encoding is chosen internally. HIMPORT is an optional bulk-load path, useful for migrations and ETL sinks, not a replacement for HSET.

Does the BACKUP command replace RDB and AOF? No. BACKUP produces a restorable artefact set from the MP-AOF format; it does not change your durability configuration. appendfsync and save still govern what you can lose.

How much memory will compact hashes actually save me? It depends entirely on how many of your keys share a field layout. Measure it with the A/B procedure above on your own dataset. Anyone quoting you a percentage without seeing your keyspace is guessing.

Is Redis 8.10 open source? Redis Open Source 8.x is tri-licensed: RSALv2, SSPLv1, or AGPLv3 at the user’s choice. AGPLv3 is OSI-approved; the other two are source-available. We are not lawyers — take licence-compliance questions to counsel.

Should I move to Valkey instead? It depends on your licence constraints, your dependence on in-engine Search/JSON/TimeSeries, and your managed-service provider’s roadmap. Both are credible. Decide on those three axes, not on vendor throughput headlines.

Working with MinervaDB

MinervaDB provides vendor-neutral Redis support and consulting — 24×7 consultative support, remote DBA, performance engineering and HA/DR architecture — as part of our full-stack enterprise database practice across PostgreSQL, MySQL, MariaDB, SQL Server, MongoDB, ClickHouse, Cassandra, Redis, Valkey and cloud DBaaS. If you want Redis 8.10 performance and reliability validated on your own estate — the compact-hash saving measured on your keyspace and the BACKUP path validated with a real restore drill before you trust it, that is a scoped engagement we run regularly.

Further reading from our library: the Redis troubleshooting cheatsheet and the advanced Redis operations cheatsheet.

Standing caveat: every configuration change and command in this post must be tested in a non-production environment before it reaches production, and no upgrade should proceed without a verified, drilled DR posture.

Sources

• Redis 8.10 — What’s new
• Redis Open Source 8.10 release notes (00-RELEASENOTES)
• Redis 8.10.0 release on GitHub
• HIMPORT command reference
• BACKUP command reference
• Redis persistence — online backups with the BACKUP command family
• Redis 8.8 performance improvements
• Redis 8.8 — What’s new
• Announcing Redis 8.6: performance improvements and streams enhancements
• Redis 8.6 — What’s new
• Redis licences (RSALv2 / SSPLv1 / AGPLv3)
• Redis release and end-of-life dates
• Valkey 9.1 release announcement

Redis 8.10 Performance and Reliability: What Actually Changed appeared first on MariaDB.org

Continue reading “MariaDB Foundation Advances TAF with HammerDB 6.0 and xt_reservoir Integration”

MariaDB Foundation Advances TAF with HammerDB 6.0 and xt_reservoir Integration appeared first on MariaDB.org

MariaDB Foundation Advances TAF with HammerDB 6.0 and xt_reservoir Integration appeared first on MariaDB.org

Source

MariaDB Community Server 10.6.28 now available appeared first on MariaDB.org

So we tried it. The first run failed, and it failed in a way that is easy to miss: the replica took every transaction, reported success, and stored nothing. We tracked down why, fixed it, and the whole test suite passes now. This post is what we tested, how we checked it, the bug we found, and where it stands.

It’s still an experiment, not production software. The code and the test harness are on GitHub under GPLv2: https://github.com/Percona-Lab/ducksdb-mysql-engine.

Why replicate into DuckDB

A DuckDB table on one server is already useful. The analytical queries get fast and the application does not change. But almost nobody runs their reports on the primary – they run them on a replica, so the big scans stay out of the way of the OLTP traffic.

So the shape of it is simple. The primary stays InnoDB and takes the writes. The replica has the same tables, only marked ENGINE=DuckDB. Row-based replication ships the changes across, the replica writes them into the column store, and the reports run there. You get an analytics replica out of the replication you already run.

Row events are engine-agnostic on purpose. The primary logs the row changes, not the SQL, and the replica applies them through the storage-engine API. On paper, then, the replica should not care that one side is InnoDB and the other DuckDB. We wanted to see the paper version hold up on a running server.

The setup

Two containers from the same image, one primary and one replica. It’s all in Docker, so it repeats cleanly.

• Primary: InnoDB, binlog_format=ROW, GTID on.
• Replica: same server, GTID on, tables made with ENGINE=DuckDB.
• Replication uses SOURCE_AUTO_POSITION=1.

One thing you have to get right before any data moves. Create the replica tables as ENGINE=DuckDB yourself. A CREATE TABLE … ENGINE=InnoDB on the primary goes into the binlog with the ENGINE word still in it, and the replica runs it exactly as written, so you would end up with an InnoDB table there, not a DuckDB one. There is no automatic mapping. Pre-create the DuckDB tables on the replica, and let the row changes flow into them.

-- primary (InnoDB)
CREATE TABLE t1 (id BIGINT PRIMARY KEY, region INT, amount DECIMAL(12,2)) ENGINE=InnoDB;

-- replica (same columns, DuckDB)
CREATE TABLE t1 (id BIGINT PRIMARY KEY, region INT, amount DECIMAL(12,2)) ENGINE=DuckDB;

The other rule is a primary key on the replica table. UPDATE and DELETE row events find the row by its old image, and the engine needs the key for that. INSERT works without one, but put a key on it anyway.

One script drives all of this: bench/tb/07-replication-spike.sh. It starts both containers, wires up replication, runs every scenario below, and prints PASS or FAIL for each.

What we tested, and how

The part that matters is the checking. Row counts are not enough – the replica can hold the right number of rows and still have the wrong data in them. So after each step the script dumps the whole table on both sides, ordered by primary key, and compares an md5 of the two dumps. One byte off is a FAIL. And rather than sleep between steps, it waits on WAIT_FOR_EXECUTED_GTID_SET(), so the checks do not race the replica.

Here is what went through it.

Basic DML. Insert, update a row, delete a row, compared after each one.

All the column types, in a single wide table: signed and unsigned integers, DECIMAL, DOUBLE, DATE, DATETIME, TIMESTAMP, CHAR, VARCHAR, TEXT, BLOB, a few NULLs, and a unicode string. Insert it, update it, compare byte for byte. Blobs get their own note below.

DDL. ALTER TABLE ADD COLUMN, ALTER TABLE ADD INDEX, and DROP TABLE against a DuckDB replica table. These arrive as statements. We check that the column shows up, the index shows up, the old rows survive, and the drop removes the table.

Transactions. A transaction with two inserts and an update has to land on the replica as one unit. A transaction the primary rolls back has to leave nothing behind. We also open a transaction straight on the replica and both roll it back and commit it, to check the engine’s own commit and rollback.

Bulk load. 5000 rows through LOAD DATA on the primary, has to arrive and match.

Durability. Two cases, and the second is the hard one.

• Clean restart. Stop the replica properly, write on the primary while it is down, start it again, and see it pick up from its GTID position.

• Crash. Apply some rows, then SIGKILL the replica. No clean shutdown, no checkpoint. Bring it back, write more on the primary, and check one exact thing: every row present once. Nothing lost – DuckDB has to replay its write-ahead log when it opens the file – and nothing applied twice, which means the saved position has to line up with the data that actually reached disk.

The bug: multi-engine transactions lost data

The first full run fell down on the wide-table test. Zero rows on the replica, and then everything after it failed too. The applier had stopped with HA_ERR_KEY_NOT_FOUND. It went to UPDATE a row that was not there, because the INSERT before it had returned success and written nothing.

When a scenario fails, the harness saves the applier error, both server logs, and both schemas. The replica log had the line that mattered:

[Warning] Combining the storage engines InnoDB and DuckDB is deprecated, but the
statement or transaction updates both the InnoDB table mysql.slave_worker_info and the
DuckDB table rpl.wide.

That line is the whole thing. A replica does not only write your data. In the same transaction it also writes its own position into InnoDB system tables – mysql.slave_worker_info, the relay-log info, gtid_executed. So every applied transaction touches two engines at once: InnoDB for the position, DuckDB for the data. Two engines means MySQL runs a real two-phase commit: prepare, then commit.

Our prepare was wrong. It took the open DuckDB transaction, moved it into a registry meant for external XA COMMIT, and cleared the per-connection state. Then commit looked at that state, found it empty, and committed nothing. The position went into InnoDB, the GTID advanced, the binlog moved on, and the DuckDB rows were thrown away. No error anywhere. The replica looked healthy while it dropped every write.

We cut it down to the smallest case, with no replication at all. One server, one transaction into a DuckDB table and an InnoDB table:

BEGIN;
INSERT INTO duck VALUES (1,10),(2,20),(3,30);   -- DuckDB
INSERT INTO inno VALUES (1,10),(2,20),(3,30);   -- InnoDB
COMMIT;
-- duck: 0 rows   inno: 3 rows

InnoDB kept its three rows, DuckDB kept none, and COMMIT said it was fine. A DuckDB-only transaction was fine as well, because with one engine MySQL skips the prepare step. It only broke with a second engine in the transaction. And on a replica, that is every transaction.

The fix

Small change, in the engine’s transaction code. prepare now remembers which prepared transaction belongs to the connection, and commit finishes that one instead of an empty state. External XA is untouched. It went out as v0.2.3.

With that in place the reproducer keeps three rows in both tables, and the full run comes back clean, crash test included:

[8]  data integrity: all column types, NULL / unicode / negatives ....... PASS
[9]  DDL replication (ALTER ADD COLUMN / ADD INDEX / DROP) .............. PASS
[10] transactions (atomic commit, rollback, engine commit/rollback) ..... PASS
[11] bulk LOAD DATA on master -> replica ................................ PASS
[12] durability: graceful restart, then SIGKILL crash recovery .......... PASS

VERDICT: PASS=24  FAIL=0

The crash case is the important one. After a SIGKILL in the middle of applying, the replica came back with every committed row exactly once, matching the primary. Committed transactions survive the kill, and the position stays in step with them.

We left two tests behind so this cannot slip back in quietly: an MTR test, txn_mixed_engine, that runs a mixed DuckDB+InnoDB transaction on every build, and scripts/repro-2pc-dataloss.sh, which you can point at any published image to check it.

What works, and what doesn’t yet

Where it stands on v0.2.3, for an InnoDB primary feeding a DuckDB replica:

The things to keep in mind:

• Create the replica tables as ENGINE=DuckDB yourself. A replicated CREATE TABLE keeps the primary’s engine, so it will not turn into DuckDB on its own.

• Replica tables need a primary key for UPDATE and DELETE.

• The applier goes row by row. That is fine for a normal OLTP change stream. It is not fine for keeping up with a primary that bulk-loads at full speed – the replica will fall behind.

• Committed transactions are crash-safe, with one small gap. The engine holds a prepared-but-not-committed transaction in memory only, so a crash in the short window between prepare and commit can lose that single transaction. The applier commits right away, so the window is small, but it is not zero.

• Blobs behave differently over replication than through a direct statement. A plain UPDATE of a BLOB or TEXT column has a known limit in the engine and does not apply. Over replication it does apply, because the row event carries a full before-and-after image instead of the shared buffer the direct path uses.

And the obvious one. This is an experiment. It is a functional result from a test harness on small data, not an HA or failover benchmark. We did not test multi-source replication, filters, or a real write rate.

Where it stands

An InnoDB primary feeding a DuckDB replica works on v0.2.3. Inserts, updates, deletes, every common type, schema changes, transactions, bulk load – they all replicate and match, and it comes back clean from both a graceful restart and a hard kill. The one real bug, silent data loss on every replicated transaction, is found, understood, fixed, and covered by tests.

It is not production-ready, and we do not treat it as such. But the idea holds up. Point normal MySQL replication at a DuckDB replica, and you get an analytics copy that keeps itself in sync.

The post Replicating from InnoDB into a DuckDB storage engine appeared first on Percona.

Replicating from InnoDB into a DuckDB storage engine appeared first on MariaDB.org

Continue reading “MariaDB 13.1 Feature in Focus: JSON Operators and JSON_TABLE Improvements”

MariaDB 13.1 Feature in Focus: JSON Operators and JSON_TABLE Improvements appeared first on MariaDB.org

MariaDB 13.1 Feature in Focus: JSON Operators and JSON_TABLE Improvements appeared first on MariaDB.org

This is an important step for anyone planning infrastructure refreshes around the latest Linux releases, and it’s especially notable for teams evaluating ARM to cut infrastructure spend without sacrificing performance.

What’s new in 8.0.28-12

Starting with this release, Percona Server for MongoDB packages are available for:

• RHEL 10 and derivatives like Oracle Linux 10, Rocky Linux 10, AlmaLinux 10 on x86_64 and ARM (aarch64)
• Debian 13 “Trixie” on x86_64 and ARM64 (aarch64)

Additionally, starting this release, we’ve included Software Bills of Materials (SBOMs) and Vulnerability Exploitability Exchange (VEX) for every release. SBOMs improve software supply chain transparency by documenting the components and dependencies included in a build. They are generated automatically as part of the release pipeline in the industry-standard CycloneDX format. OpenVEX files are published on GitHub Pages and provide the exploitability status of known vulnerabilities. For comprehensive information, refer to our [documentation](../sbom.md).

To learn more, see the full release notes of Percona Server for MongoDB 8.0.28-12.

Ahead of upstream on Debian 13

As of this writing (August 2026), upstream MongoDB Community/Enterprise Server does not yet officially package or support Debian 13. Trixie isn’t in MongoDB’s supported platforms list, and the documented community workaround is to install the Debian 12 “Bookworm” build on Trixie hosts, since a native Trixie server build hasn’t landed yet. PSMDB closes that gap now, with native Debian 13 packages rather than a buggy Bookworm build running out-of-distro.

Why ARM is worth a serious look for MongoDB workloads

We have heard multiple times from you directly, via our forum, or on Reddit about the Interest in ARM for database workloads. Over the last few years, adoption has moved well past the experimental phase to resilient production readiness. Our adoption telemetry data show nearly 3x as many ARM instances over the last 12 months!

I can see a number of benefits and reasons why our community users and customers adopted ARM over AMD or Intel CPU architectures:

• Lower infrastructure costs. Cloud ARM instances, such as AWS Graviton, are commonly cited as running 20–40% cheaper than comparable x86 instances at similar or better performance (sanj.dev, “ARM vs x86 Cloud: Which Architecture is Cheaper in 2025?”), and benchmarking write-ups report up to 60% lower energy consumption for compute-intensive workloads on Graviton compared to x86 equivalents (nOps, “Are you missing out on AWS Graviton Cost Savings?”).
• Competitive, and often better, throughput. RDS PostgreSQL Graviton” benchmark analysis shows Graviton4 delivering up to 40% better performance for OLTP-style workloads versus the previous Graviton3 generation. Apache Doris Delivers 70% Better Value on AWS Graviton. I can see MongoDB database achieving a similar level of performance-to-cost gain. 
• Memory bandwidth is a real differentiator. Graviton3’s memory bandwidth (cited at roughly 115–120 GB/s) is reported to significantly outpace typical Intel Xeon configurations (roughly 60–70 GB/s) and AMD EPYC (roughly 80–90 GB/s) in independent comparisons (byteiota, “ARM vs x86 Cloud: 2025 Performance & Cost Benchmark”), which matters for memory-hungry workloads like MongoDB’s WiredTiger cache and in-memory working sets.

(Note: The above figures come from third-party blogs and vendor case studies rather than peer-reviewed benchmarks. Treat them as directional evidence that ARM is worth evaluating, not a guarantee of results for your specific workload. For the official recommendation based on your workload, reach out to Percona)

Netflix has publicly stated that it saves over $15 million annually after migrating video encoding workloads to Graviton, while also seeing faster processing times, and other large-scale AWS customers have reported double-digit percentage reductions in compute costs after moving meaningful portions of their backend fleets to ARM (byteiota; sanj.dev).

What to watch out for

One RHEL 10 detail to keep in mind when planning a migration: Red Hat raised the CPU baseline for x86_64 to the x86-64-v3 microarchitecture level, meaning the processor needs to support instruction sets such as AVX2 (Red Hat, RHEL 10 architecture documentation; vInfrastructure Blog). On the ARM side, RHEL 10 targets the ARMv8.0-A baseline (Red Hat documentation). This applies equally to Oracle Linux 10, Rocky Linux 10, and AlmaLinux 10, since they build from the same upstream sources. I highly recommend checking your current infrastructure before planning the move, especially for older bare-metal fleets.

In general, but especially on ARM, remember that performance is workload-dependent. Some code paths and workloads leaning on x64-specific instruction extensions may not see the same gains as throughput-oriented, multi-threaded workloads on ARM. Benchmarking your own query patterns and index-heavy operations before a full cutover is essential. General benchmarks are a good signal, not a guarantee.

Percona can help you get there

Migrating a production MongoDB deployment to a different infrastructure is a project with real decision points. There are a number of questions to answer around: Hardware or instance selection, driver and tooling compatibility, benchmarking against your actual workload, and a rollback plan. The Percona Services team helps customers plan and execute exactly this kind of migration. We start from initial architecture assessment and proof-of-concept benchmarking and go through to production cutover and post-migration tuning.

If you’re weighing a move to ARM, or just want to get onto RHEL 10 or Debian 13 without surprises, reach out to Percona to talk through your environment.

The post Percona for MongoDB: RHEL 10, Its Derivatives, and Debian 13 – On Both x86_64 and ARM appeared first on Percona.

Percona for MongoDB: RHEL 10, Its Derivatives, and Debian 13 – On Both x86_64 and ARM appeared first on MariaDB.org

Since I’m not quite up to speed on this topic yet, I like to look up certain terms (parameters, functions, etc.) every now and then to see exactly what they mean or how they work (RTFM!). That’s exactly why links were originally invented—the very thing that first made Gopher and later the Internet/WWW (http) so popular.

Unfortunately, however, these links are often missing from the documentation in question, which disrupts the flow of reading.

Fortunately, though, PostgreSQL is an open-source project, and contributions are highly encouraged! So instead of just grumbling about the documentation, I could add the missing links myself. But how exactly do I go about doing that in an ecosystem that’s new to me and therefore still a bit unfamiliar? An article by Elizabeth Christensen from Crunchy Data titled Contributing to Postgres 101: A Beginner’s Experience helped me get started.

Since I have absolutely no programming experience myself, I see improving the documentation as a great opportunity to actively contribute to the project and help out…

Improving the PostgreSQL Documentation

The PostgreSQL documentation is stored directly in the server repository. So, first, let’s download the Git repository from the PostgreSQL server:

$ git clone http://git.postgresql.org/git/postgresql.git

The next challenge is finding the right file:

$ cd postgresql/doc/src/sgml

The grep command, in all its forms, comes in handy here:

$ grep -r 'Planning for High Availability' *.sgml
high-availability.sgml: <title>Planning for High Availability</title>

The correct document appears to be high-availability.sgml. The PostgreSQL documentation itself is written in SGML, which is similar to HTML and not particularly difficult to learn.

These SGML files can be easily read and edited using your editor of choice with appropriate code highlighting.

Next, we need to check the individual keywords to see if they’ve already been correctly marked up and, if so, add links to them:

Note: Keep in mind that keywords are written with an “_” (underscore) and links with a “-” (hyphen).

While building the documentation, it was also noticed that some link targets (id) hadn’t been set at all, so these had to be adjusted as well:

 <row>
- <entry role="func_table_entry"><para role="func_signature">
+ <entry id="pg-backup-start" role="func_table_entry"><para role="func_signature">
 <indexterm>
 <primary>pg_backup_start</primary>

Quality Assurance

Once all changes have been made, it’s time for quality control. To do this, build the documentation locally:

$ cd postgresql
$ ./configure
$ cd doc
$ make

Exact details on how this works are described here.

If the build finds any errors, they will be displayed and the build will be aborted. If everything runs smoothly, you can now use your browser of choice to check whether everything actually works as intended:

$ firefox src/sgml/html/warm-standby.html

Something else I discovered later:

Building the documentation can take very long. But there is a method to just check the correct syntax of the documentation files, which only takes a few seconds: [ 5 ]

$ make check
make -C ../src/backend generated-headers
make[1]: Entering directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend'
make -C ../include/catalog generated-headers
make[2]: Entering directory '/home/oli/fromdual/postgresql/docu/postgresql/src/include/catalog'
make[2]: Nothing to be done for 'generated-headers'.
make[2]: Leaving directory '/home/oli/fromdual/postgresql/docu/postgresql/src/include/catalog'
make -C nodes generated-header-symlinks
make[2]: Entering directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend/nodes'
make[2]: Nothing to be done for 'generated-header-symlinks'.
make[2]: Leaving directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend/nodes'
make -C utils generated-header-symlinks
make[2]: Entering directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend/utils'
make -C adt jsonpath_gram.h
make[3]: Entering directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend/utils/adt'
make[3]: 'jsonpath_gram.h' is up to date.
make[3]: Leaving directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend/utils/adt'
make[2]: Leaving directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend/utils'
make[1]: Leaving directory '/home/oli/fromdual/postgresql/docu/postgresql/src/backend'
rm -rf '/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install
/usr/bin/mkdir -p '/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install/log
make -C '..' DESTDIR='/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install install >'/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install/log/install.log 2>&1
make -j1 checkprep >>'/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install/log/install.log 2>&1
PATH="/home/oli/fromdual/postgresql/docu/postgresql/tmp_install/usr/local/pgsql/bin:/home/oli/fromdual/postgresql/docu/postgresql/doc:$PATH" LD_LIBRARY_PATH="/home/oli/fromdual/postgresql/docu/postgresql/tmp_install/usr/local/pgsql/lib:$LD_LIBRARY_PATH" INITDB_TEMPLATE='/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install/initdb-template initdb --auth trust --no-sync --no-instructions --lc-messages=C --no-clean '/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install/initdb-template >>'/home/oli/fromdual/postgresql/docu/postgresql'/tmp_install/log/initdb-template.log 2>&1

Submitting the Patch

If everything works as intended and to your satisfaction, you can then proceed to create the patch and submit it:

$ git commit -m 'some references on variables and functions added'
$ git format-patch -1 HEAD

This creates a file containing the commit comment: 0001-some-references-on-variables-and-functions-added.patch.

Apparently, in the PostgreSQL project, you don’t create a merge request to incorporate the patch back into the source code; instead, the patch must be sent to the appropriate mailing list and then merged into the main branch by a developer with merge/commit privileges. I’ve now agreed with “my” committer that we’ll hold the discussion about my patch on the pgsql-docs mailing list.

Let’s see how things go from here and how far I get with my patch…

This page was translated using deepl.com.

Making the PostgreSQL Documentation Even Better appeared first on MariaDB.org

A Fractional Chief Data Officer from MinervaDB gives an enterprise board-level data leadership — strategy, architecture, governance and operations — on a part-time, fixed-fee basis. When that mandate is pointed squarely at real-time analytics, the role stops being an organisational nicety and becomes a profit-and-loss instrument: it determines how quickly your business can see an event, decide on it, and act before the opportunity decays.

This article is deliberately written to be read by five different people at the same table. The CEO will find the commercial thesis. The CTO will find the reference architecture, the latency budget and the engineering standards. The CFO will find unit economics, total cost of ownership and payback. The board and investors will find the governance, risk and diligence position. All five are looking at the same estate; a Fractional Chief Data Officer exists to make sure they are looking at the same numbers.

Why Real-Time Analytics Became a Board-Level Question

For most of the last decade, analytics was a reporting function. Data landed overnight, a warehouse transformed it, and the business read yesterday. That model is no longer competitive in any market where price, inventory, credit, fraud, capacity or customer intent move within the trading day. The shift is not technological fashion. It is a change in where margin is created.

Real-time analytics changes three things a board cares about. It shortens the interval between an event and a decision, which is where most recoverable value sits. It exposes operational truth continuously rather than in a monthly pack, which changes the quality of governance. And it converts analytics from a fixed reporting cost into a variable, attributable one — which is precisely why it needs an owner with executive authority, not a project team.

That owner is the problem. Streaming estates fail commercially far more often than they fail technically. Pipelines get built, dashboards refresh in seconds, and the organisation still argues about which revenue number is correct, still cannot attribute the cloud bill, and still cannot show an auditor where personal data flows. A Fractional Chief Data Officer is the corrective: one accountable executive who holds strategy, architecture, cost and compliance together across the entire real-time path.

What a Fractional Chief Data Officer Owns in a Real-Time Analytics Estate

Real-time analytics is a chain, and a chain is owned end to end or not at all. The diagram below is the path an event travels from the moment it is committed in a system of record to the moment a human or a model acts on it. Every hop in that path has a latency cost, a failure mode, a cost line and a compliance implication. The Fractional Chief Data Officer owns all four dimensions across every hop.

Figure 1 — The real-time analytics path a Fractional Chief Data Officer governs end to end, with the latency budget allocated hop by hop.

Each stage is a genuine engineering discipline. Capture is usually log-based change data capture reading the write-ahead log, following the mechanics described in the PostgreSQL logical replication documentation and implemented with Debezium. Transport is an ordered, replayable log, normally Apache Kafka. Processing is stateful stream computation with checkpointing, typically Apache Flink. Serving is a column store tuned for high-cardinality, low-latency aggregation, such as ClickHouse and its MergeTree family of table engines.

What no vendor supplies is the arbitration between them. Which events are worth streaming at all? Which consumers are entitled to which fields? What is an acceptable staleness for a pricing decision versus a regulatory report? Who pays when a single badly written dashboard query scans forty terabytes? These are executive questions with engineering answers, and they are the daily work of a Fractional Chief Data Officer. MinervaDB backs that judgement with delivery capability through ClickHouse consulting, PostgreSQL consulting and our high-performance data engineering practice.

The Latency Budget: How a Fractional Chief Data Officer Turns Milliseconds Into Margin

Real-time is not a marketing adjective; it is a number with an owner. The single most useful artefact a Fractional Chief Data Officer introduces in the first month is a written latency budget: an explicit allocation of the end-to-end service level objective across every hop, with a named owner and an alert for each allocation. Once the budget exists, arguments about whether the platform is fast enough stop being subjective.

Figure 2 — A published latency budget: every hop has an allocation, an owner and an alert. Unallocated headroom is a deliberate reserve, not luck.

Two disciplines make the budget real. The first is error-budget thinking, borrowed from site reliability engineering and set out in the Google SRE book chapter on service level objectives: an objective without a consequence is a wish. The second is physical design. In a column store, latency and cost are both functions of sort order, partitioning, codecs and pre-aggregation — which is why a Fractional Chief Data Officer signs off storage layout the way a chief financial officer signs off capital expenditure.

ClickHouse · real-time ingest with a governed physical design-- Ordered, replayable ingest. Consumer group, format and parallelism are
-- reviewed artefacts, not defaults inherited from a tutorial.
CREATE TABLE rt.order_event_queue
(
    event_time       DateTime64(3),
    tenant_id        UInt32,
    order_id         String,
    country          LowCardinality(String),
    channel          LowCardinality(String),
    net_amount       Decimal(18, 2),
    is_fraud_flagged UInt8
)
ENGINE = Kafka
SETTINGS kafka_broker_list   = 'kafka-01:9092,kafka-02:9092',
         kafka_topic_list    = 'commerce.orders.v2',
         kafka_group_name    = 'ch_rt_orders',
         kafka_format        = 'JSONEachRow',
         kafka_num_consumers = 4;

-- Serving table. Sort order comes from measured query patterns; retention and
-- tiering are signed off by the Fractional Chief Data Officer as cost policy.
CREATE TABLE rt.order_event
(
    event_date       Date DEFAULT toDate(event_time),
    event_time       DateTime64(3),
    tenant_id        UInt32,
    order_id         String,
    country          LowCardinality(String),
    channel          LowCardinality(String),
    net_amount       Decimal(18, 2),
    is_fraud_flagged UInt8
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(event_date)
ORDER BY (tenant_id, country, event_time)
TTL event_date + INTERVAL 6  MONTH TO VOLUME 'cold',
    event_date + INTERVAL 25 MONTH DELETE
SETTINGS index_granularity = 8192;

-- Pre-aggregate the handful of questions the business asks every minute,
-- so that a dashboard refresh never becomes a full-table scan.
CREATE MATERIALIZED VIEW rt.order_minute_mv TO rt.order_minute AS
SELECT toStartOfMinute(event_time)  AS minute,
       tenant_id,
       country,
       countState()                 AS orders_state,
       sumState(net_amount)         AS revenue_state,
       sumState(is_fraud_flagged)   AS flagged_state
FROM rt.order_event
GROUP BY minute, tenant_id, country;

Decision Decay: The Economic Case a Fractional Chief Data Officer Puts to the CFO

The commercial argument for real-time analytics is not that faster is nicer. It is that the value of a decision decays, often steeply, from the moment the triggering event occurs. A fraud signal acted on in 400 milliseconds prevents a loss; the same signal in the overnight batch documents one. An abandoned basket recovered within the session converts; recovered tomorrow it annoys. A Fractional Chief Data Officer makes that decay curve explicit, attaches revenue to it, and uses it to size investment.

Figure 3 — Decision decay. A Fractional Chief Data Officer prices the area under this curve, then designs the latency budget in Figure 2 to recover it.

Seven Proven Wins a Fractional Chief Data Officer Delivers in Real-Time Analytics

These are the outcomes we contract to. They are stated as measurable changes rather than activities, because a Fractional Chief Data Officer engagement is only credible if the board can verify it.

The Technical Blueprint: What the CTO Gets From a Fractional Chief Data Officer

Engineering teams do not need another strategy deck. They need decisions made, written down and defended. A Fractional Chief Data Officer supplies exactly that: a short set of non-negotiable standards for the streaming estate, each of which removes a recurring class of incident. The three that matter most are the data contract, exactly-once delivery semantics, and measured rather than asserted service levels.

Data contracts for streams, reviewed like an API

Every significant topic publishes a contract: owner, classification, freshness commitment, schema, quality rules and breaking-change policy. Producers cannot silently drop a field. Consumers can depend on a stated service level. Auditors have one artefact to inspect. Quality assertions run in continuous integration in the manner described by the dbt data testing documentation, and a failing contract blocks promotion.

Data contract · versioned in source control, enforced in the pipeline# data-contracts/commerce.orders.v2.yaml
apiVersion: minervadb.com/v1
kind: StreamContract
metadata:
  name: commerce.orders
  version: 2.1.0
  owner: commerce-platform
  steward: fractional-cdo-office
spec:
  classification: restricted
  transport: kafka
  partitions: 24
  keyField: order_id
  ordering: per-key
  deliverySemantics: exactly-once
  freshnessSlo: PT2S          # event time to queryable in ClickHouse
  availabilitySlo: 99.95
  retention: P7D              # log retention; serving store keeps 25 months
  schema:
    - name: order_id
      type: string
      required: true
    - name: customer_email
      type: string
      required: true
      pii: true
      masking: sha256
    - name: net_amount
      type: decimal(18,2)
      required: true
      constraints: [">= 0"]
  quality:
    - rule: uniqueness(order_id)
      threshold: 1.0
      severity: blocker
    - rule: lag_p99_seconds(event_time, ingested_at)
      threshold: 2
      severity: critical
  breakingChangePolicy: majorVersionOnly

Service levels measured, not asserted

The monthly executive report is generated by a query, not written by a person. That single habit removes the most common failure of data leadership, which is a scorecard that quietly reflects opinion. The query below is the shape we deploy on day one of a Fractional Chief Data Officer engagement, and it is the same number the board sees.

ClickHouse · the freshness, quality and cost numbers behind the board pack-- End-to-end freshness and cost, produced by instrumentation rather than opinion.
SELECT
    dataset,
    quantile(0.50)(dateDiff('millisecond', event_time, ingested_at)) AS p50_lag_ms,
    quantile(0.99)(dateDiff('millisecond', event_time, ingested_at)) AS p99_lag_ms,
    countIf(dateDiff('millisecond', event_time, ingested_at) > slo_ms) * 100.0
        / count()                                                    AS slo_breach_pct,
    round(sum(read_bytes) / pow(1024, 4), 3)                         AS tb_scanned,
    round(sum(read_bytes) / pow(1024, 4) * 5.00, 2)                  AS est_cost_usd
FROM rt.pipeline_observability
WHERE ingested_at >= now() - INTERVAL 30 DAY
GROUP BY dataset
HAVING slo_breach_pct > 0.1 OR p99_lag_ms > 2000
ORDER BY slo_breach_pct DESC;

Where this work extends beyond governance into sustained operations, the Fractional Chief Data Officer draws on 24×7 remote DBA support, data analytics and warehousing support and, for retrieval and recommendation workloads, our vector data engineering capability.

Unit Economics: How a Fractional Chief Data Officer Reads the Real-Time Analytics Bill

Real-time platforms are billed by consumption, which means physical design decisions land directly on the invoice. A CFO does not need to understand sort keys; a CFO needs the cost expressed per unit of business activity and trended. A Fractional Chief Data Officer builds that bridge, replacing a single opaque cloud line item with attributable unit costs that a finance function can actually manage.

Figure 4 — Unit economics before and after a Fractional Chief Data Officer takes ownership of the real-time estate.

The mechanism is unglamorous and repeatable. Queries are profiled and the top decile by bytes scanned is redesigned or pre-aggregated. Retention is enforced instead of aspirational. Cold partitions move to cheaper storage on a schedule. Idle clusters are decommissioned rather than tolerated. Chargeback labels are applied so that every terabyte has an owner. MinervaDB runs this discipline continuously through our cloud database optimisation and FinOps practice, and it is what makes the investment case defensible rather than aspirational.

Governance, Risk and Diligence: The Board and Investor View

For a board, real-time analytics raises the stakes on governance rather than lowering them. Data moves faster, reaches more consumers and is embedded in automated decisions, which means an error propagates before anyone notices. Directors are entitled to ask a small number of hard questions, and a Fractional Chief Data Officer exists to answer them with evidence: where does personal data flow, who can read it, how long is it kept, what breaks if a pipeline fails, and what did we know at the moment a decision was automated?

Our governance model follows established practice rather than invention. Data management domains map to the DAMA Data Management Body of Knowledge, and privacy obligations map to the processing principles in Article 5 of the GDPR and their regional equivalents. The value MinervaDB adds is not the framework; it is the engineering rigour with which the framework is made executable in a streaming estate.

Figure 5 — The reference architecture a Fractional Chief Data Officer governs: fit-for-purpose storage under a single governance, security and FinOps plane.

Investors read this differently again. In diligence, data findings rarely kill a deal but frequently move the price. An estate with certified metrics, documented lineage, enforced retention and a measured cost base presents as a managed asset. The same estate without those artefacts presents as a liability with an unknown remediation cost, and it is discounted accordingly. Engaging a Fractional Chief Data Officer eighteen months before a raise or an exit is, in our experience, one of the cheapest forms of valuation protection available. Our perspective on database transformation for CIOs and our global capability centre data leadership programme describe how that capability is sustained at scale.

The First 90 Days With a MinervaDB Fractional Chief Data Officer

Every engagement follows the same evidence-led sequence: assess before advising, architect before building, and prove value on two or three real-time use cases before asking for a larger budget.

Figure 6 — The MinervaDB Fractional Chief Data Officer engagement roadmap, from assessment to a recurring executive scorecard.

The Executive Scorecard a Fractional Chief Data Officer Reports Against

The scorecard is agreed in the first month and reported every month thereafter, generated from instrumentation. Four numbers carry most of the signal for a real-time estate: are we fresh, are we correct, what does it cost, and how fast can we act?

Figure 7 — Indicative twelve-month scorecard. Every figure is produced by a query the client can run independently.

Fractional Chief Data Officer Engagement Models

We offer three engagement shapes. All are fixed monthly fees with a named principal, a defined day commitment and a thirty-day exit. None involves a leverage pyramid or a junior delivery team.

Fixed-scope assessments of two to four weeks are also available, and many clients start there. Where a MinervaDB relationship already exists, the Fractional Chief Data Officer can be layered on top of MinervaDB consultative support without renegotiating the underlying operational contract.

Fractional Chief Data Officer Versus a Full-Time Hire or an Advisory Firm

The honest position is this. At sufficient scale, a permanent Chief Data Officer is the right answer. A Fractional Chief Data Officer is the right answer before you reach that scale, while you are recovering from a stalled real-time programme, or while you are preparing the organisation so that a permanent hire succeeds rather than becomes your second attempt.

Frequently Asked Questions About Fractional Chief Data Officer Services

What is a Fractional Chief Data Officer?

A Fractional Chief Data Officer is a senior data executive engaged part-time on a fixed fee who carries the full mandate of a Chief Data Officer — strategy, architecture, governance, quality, security, cost and value realisation — without the salary, equity and hiring risk of a permanent appointment.

How does a Fractional Chief Data Officer improve real-time analytics specifically?

By owning the whole path rather than a stage of it. That means a published latency budget, data contracts on every significant stream, exactly-once delivery semantics, a serving layer designed for the queries the business actually runs, and a cost model expressed per unit of business activity. Speed becomes a managed commitment instead of a demo.

How much time does the role commit each month?

Typically two to eight days per month. Advisory engagements start at two days, embedded engagements run at four to six, and transformation programmes require eight or more with a MinervaDB delivery pod behind the role.

Which technologies does the mandate cover?

The whole estate: relational systems such as PostgreSQL, MySQL, MariaDB and SQL Server; NoSQL platforms including MongoDB, Cassandra, Redis and DynamoDB; NewSQL engines such as CockroachDB, TiDB and YugabyteDB; streaming infrastructure including Kafka and Flink; column stores including ClickHouse, Druid, Snowflake, BigQuery and Redshift; and cloud native data platforms on AWS, Azure and Google Cloud.

How quickly will we see measurable results?

A prioritised risk register plus a measured latency and cost baseline within thirty days. An approved strategy and investment plan by day sixty. Working real-time data products and a board-readable scorecard by day ninety.

Will a Fractional Chief Data Officer replace our existing data team?

No. The role gives an existing team direction, standards, decision rights and executive cover. In most engagements the team becomes measurably more effective, and one of the deliverables is a hiring and capability plan for strengthening it further.

Can the engagement transition to a permanent Chief Data Officer?

Yes, and we plan for it from the outset. Every artefact — strategy, standards, contracts, runbooks and the scorecard — is written for handover. Many clients use a Fractional Chief Data Officer precisely to prepare the organisation so that a permanent hire succeeds.

How is data confidentiality handled?

Under a mutual non-disclosure agreement, with least-privilege access granted for the duration of the engagement only, and a preference for working inside your perimeter. Where regulation requires it, we work exclusively within your virtual private cloud with no data egress.

Fractional Chief Data Officer: 7 Proven Real-Time Analytics Wins appeared first on MariaDB.org

Continue reading “MariaDB Foundation is pleased to welcome Auree as a Silver Sponsor.”

MariaDB Foundation is pleased to welcome Auree as a Silver Sponsor. appeared first on MariaDB.org

MariaDB Foundation is pleased to welcome Auree as a Silver Sponsor. appeared first on MariaDB.org

Everything below is written from the perspective of database reliability engineering: what changed, why it changes the physics of your write path, what it costs, and how to operationalise it. If you are still on 10.6, 10.11 or 11.4, treat this as an architecture review document rather than a changelog.

Why MariaDB 12.3 Is a Structural Release, Not a Point Upgrade

MariaDB moved to a rolling-plus-LTS cadence several years ago. Rolling releases (12.0, 12.1, 12.2, and now the 13.0 RC line) carry features forward quickly; LTS releases consolidate them into a five-year maintenance window. MariaDB 12.3 is the consolidation point for everything merged since 11.8, which is why the delta looks unusually large.

Three of those changes alter architecture rather than syntax:

1. The binary log can now live inside InnoDB. That removes two-phase commit between the log and the storage engine, collapses the fsync budget of a commit, and makes the binlog crash-safe by inheritance rather than by configuration.
2. The optimizer is now steerable. A comprehensive hint vocabulary lands in the parser, so query plans can be pinned per statement instead of per session or per server.
3. Galera state transfer and conflict handling got cheaper. Incremental State Transfers skip redundant foreign key validation, and write-set application can be retried instead of aborting the applier.

Add the metadata-lock scalability work, the segmented Aria page cache, parallel replication between two Galera clusters, and buffered audit logging, and MariaDB 12.3 becomes the first release in years where the default HA topology should be re-evaluated from first principles.

The Headline Advancement in MariaDB 12.3: InnoDB-Based Binary Log

Historically MariaDB treated the binary log and InnoDB as two independent durable resources. Every commit therefore ran a two-phase commit protocol: prepare in InnoDB, write and sync the binlog, then commit in InnoDB. With sync_binlog=1 and innodb_flush_log_at_trx_commit=1, a durable commit could cost multiple fsync operations and required a recovery-time reconciliation between the two logs after a crash.

From MariaDB 12.3 the binary log can instead be stored in InnoDB-managed, page-structured files that participate in the InnoDB redo log and crash recovery. The files still live on disk as discrete objects with an .ibb extension, but internally they are 16 KB pages with a CRC32 checksum per page, pre-allocated to max_binlog_size (1 GB by default) so that write amplification from file extension disappears.

What actually changes in the commit path

The engine no longer needs to coordinate two logs, so the expensive cross-resource handshake disappears. At innodb_flush_log_at_trx_commit=1 a commit performs a single coordinated fsync instead of the several required by the old protocol. Because the binlog and InnoDB always recover to a mutually consistent state, you can also run with innodb_flush_log_at_trx_commit=0 or 2 and still guarantee that the binary log never diverges from table data after a crash – you are trading durability of the last few commits for throughput, not risking replication divergence.

Two long-standing configuration knobs become obsolete in this mode. sync_binlog is no longer required because crash safety is inherited from InnoDB, and binlog_checksum is unused because every page carries a CRC32 already. If you want integrity on the wire between primary and replica, enable TLS on the replication channel instead.

Positioning becomes GTID-only

The new implementation is GTID-native. There are no .index files, no GTID index files and no .state file. Instead the binary log periodically embeds GTID state records inside itself, by default every 2 MB, and that interval must be a power-of-two multiple of the 16 KB page size. When a replica connects, or when the server restarts, the log is scanned backwards from the most recent state record to recover the correct GTID position.

The practical consequence is that file-and-offset replication coordinates no longer exist on a primary running the InnoDB binlog: SHOW BINLOG EVENTS will generally report offsets of zero and you must navigate by GTID. The status counters binlog_gtid_index_hit and binlog_gtid_index_miss are also retired in this mode.

Backup semantics improve

Because the binlog is now InnoDB data, mariadb-backup includes it in a transactionally consistent way by default. That resolves a long-standing gap where physical backups and binary logs were captured by different mechanisms with slightly different consistency points, which is exactly the seam where point-in-time recovery used to fail during real incidents.

When you should not enable it

The InnoDB-based binary log is not a universal default. Stay on the traditional implementation if any of the following apply to your MariaDB 12.3 deployment:

• You run Galera Cluster. Synchronous multi-master requires the classic binlog implementation.
• Applications or tooling depend on filename/offset replication positions rather than GTIDs.
• Third-party CDC pipelines, such as change-data-capture connectors, parse binlog files directly from disk.
• Your replicas are still below MariaDB 12.3. Upgrade replicas first, then switch the primary.

Note also that the relay log on replicas is unchanged and still uses the traditional format, so the improvement applies to the write path of the primary, not to the apply path of the replica.

Optimizer Advancements in MariaDB 12.3

The second structural change in MariaDB 12.3 is that query plans became controllable at statement granularity. Until now, plan stability on MariaDB was largely a matter of session variables, optimizer_switch bitmasks and index hints. That is a blunt instrument in a multi-tenant fleet where one report can destabilise an OLTP workload.

A complete optimizer hint vocabulary

MariaDB 12.3 ships a MySQL-compatible hint framework covering access methods, join strategy, join order, subquery handling and execution limits:

• Access and algorithm hints: NO_RANGE_OPTIMIZATION, NO_ICP, MRR / NO_MRR, BKA / NO_BKA, BNL / NO_BNL, [NO_]ROWID_FILTER, [NO_]INDEX_MERGE.
• Index hints: [NO_]INDEX, [NO_]JOIN_INDEX, [NO_]GROUP_INDEX, [NO_]ORDER_INDEX.
• Join order hints: JOIN_FIXED_ORDER, JOIN_ORDER, JOIN_PREFIX, JOIN_SUFFIX.
• Subquery hints: SEMIJOIN, SUBQUERY, [NO_]SPLIT_MATERIALIZED, [NO_]DERIVED_CONDITION_PUSHDOWN, [NO_]MERGE.
• Guardrails: MAX_EXECUTION_TIME, plus QB_NAME and implicit query block names so hints can target a specific block of a nested statement.

-- Pin a report query without touching global optimizer_switch
SELECT /*+ QB_NAME(agg) MAX_EXECUTION_TIME(4000) NO_BNL(o) JOIN_PREFIX(c, o) */
       c.region, SUM(o.amount)
FROM   customers c
JOIN   orders o ON o.customer_id = c.id
WHERE  o.created_at >= NOW() - INTERVAL 1 DAY
GROUP  BY c.region;

For internet-scale fleets this is the difference between a plan regression that pages an on-call engineer and one that is contained inside a single statement. Combine it with the extended optimizer trace, which in MariaDB 12.3 can record table and view definitions via optimizer_record_context, and post-incident plan forensics becomes reproducible.

Reverse-ordered scans and DESC key parts

Several optimizations that previously fired only on forward scans now work in reverse order. Rowid filtering and Index Condition Pushdown both apply to reverse-ordered scans, and the loose index scan used for GROUP BY can now exploit indexes with DESC key parts. Descending-order pagination on time-series tables – the classic ORDER BY created_at DESC LIMIT n pattern – is the workload that benefits most. Optimizations for GROUP BY and ORDER BY can also use indexes defined on virtual columns, which finally makes generated-column indexing a first-class tuning technique. For a deeper treatment of join planning, see our analysis of the MariaDB join optimizer.

Engine-level throughput work

Metadata lock scalability was reworked, which matters on servers with tens of thousands of tables and aggressive DDL. The Aria storage engine gained a segmented key cache controlled by aria_pagecache_segments (default 1, maximum 128), reducing mutex contention on internal and temporary tables. Vector distance calculation was accelerated through extrapolation, which is relevant if you are using MariaDB as a vector store; we covered the storage layout in understanding vector indexes in MariaDB.

Replication and Galera Cluster Improvements in MariaDB 12.3

Asynchronous replication between two Galera clusters can now use parallel replication, governed by slave_parallel_threads. This is the topology most large deployments actually run: a synchronous cluster per region, stitched together asynchronously across regions. Until now, the cross-region channel was effectively single-threaded and became the ceiling on write throughput for the whole estate. Removing that ceiling changes regional DR from best-effort to viable.

Other replication-layer changes with operational weight:

• Row events larger than max_packet_size are now fragmented rather than failing the channel – the historical cause of stalled replication on wide BLOB writes.
• Defaults for MASTER_SSL_* are configurable, so TLS on replication channels can be enforced fleet-wide instead of per CHANGE MASTER statement.
• Temporary table behaviour in replication is now predictable and controlled by create_tmp_table_binlog_formats.
• show_slave_auth_info and replicate_same_server_id are proper system variables rather than start-up options only, and the server reports whether it started with skip-slave-start.

On the Galera side, Incremental State Transfers no longer perform needless foreign key checks, which materially shortens the window during which a rejoining node loads the cluster. Write-set application can be retried instead of aborting, controlled by wsrep_applier_retry_count, which reduces spurious node evictions under hot-row contention. Packaging also changed: the Galera dependency has been removed from the server packages, so Galera is now an explicit install rather than an implicit one. If you operate Galera in production, our field notes on troubleshooting writes in Galera Cluster and MariaDB Galera Cluster monitoring pair directly with these changes.

Security, Compatibility and Developer Surface in MariaDB 12.3

Security work in MariaDB 12.3 is focused on key material and identity. Passphrase-protected TLS keys are supported through the new ssl_passphrase system variable, the file key management plugin for transparent data encryption supports SHA-256 on Linux, and SET SESSION AUTHORIZATION allows a privileged session to execute as another user – a cleaner primitive for connection-pool multiplexing and for auditing than credential sharing.

DROP USER now warns when the account still has live sessions, and fails outright in Oracle mode. The audit plugin gained buffered logging via server_audit_file_buffer_size, records the client host and port rather than host alone, and reports the negotiated TLS version. That combination makes audit logging viable on high-QPS nodes where it was previously too expensive; see our guide to MariaDB user activity logging.

The compatibility surface widened significantly. MariaDB 12.3 adds the caching_sha2_password authentication plugin for MySQL clients, Oracle-style TO_DATE(), TO_NUMBER() and TRUNC(), the Oracle (+) outer-join operator in Oracle mode, associative arrays through DECLARE TYPE ... TABLE OF ... INDEX BY, weak SYS_REFCURSOR support with a max_open_cursors ceiling, cursors on prepared statements, the SQL standard SET PATH statement and IS JSON predicate, a basic XML data type, and the ability for UPDATE and DELETE to read from a CTE. Triggers can now fire on multiple events, foreign key constraint names only need to be unique per table, and the 32-level depth limit on JSON functions is gone. Nine new GIS functions improve MySQL 8 parity.

Six CVEs were fixed in the 12.3.2 GA build, the most severe carrying a CVSS v3.1 base score of 8.0. That alone justifies scheduling the upgrade rather than deferring it.

Reference Architecture: Highly Available MariaDB 12.3 at Internet Scale

Availability is not a feature you enable; it is a property that emerges from how you arrange failure domains. The architecture below is the pattern MinervaDB deploys for workloads that must tolerate the loss of a node, a rack, an availability zone or an entire region without data loss and without a maintenance window. It layers synchronous replication inside a region for zero-RPO, asynchronous GTID replication across regions for geographic survivability, and a routing tier that makes failover invisible to the application.

Tier 0 — Enumerate failure domains before choosing technology

Design starts by writing down what can fail and what each failure must cost you. A useful matrix for MariaDB 12.3 deployments looks like this:

Notice that only two rows are solved by replication. The rest are solved by backups, delayed replicas and process discipline. Teams that equate high availability with clustering discover this during their first bad migration.

Tier 1 — Synchronous replication inside the region

Inside a region, use a three-node MariaDB 12.3 Galera cluster with exactly one node per availability zone. Three nodes is the minimum that tolerates the loss of one while retaining a majority; five nodes buys tolerance of two failures at the cost of higher certification latency, because every transaction must be replicated to every node before commit returns.

The parameters that decide whether the cluster is stable under load are not the ones people usually tune. Size gcache.size to cover the longest plausible node outage so a rejoining node can use an Incremental State Transfer rather than a full State Snapshot Transfer; a full SST on a multi-terabyte dataset is a self-inflicted outage. Set wsrep_sst_method=mariabackup so donors remain readable.

Use gmcast.segment so that inter-AZ traffic is not multiplied by the number of nodes. Tune gcs.fc_limit deliberately: flow control is Galera telling you the slowest node cannot keep up, and raising the limit hides the symptom while increasing the amount of data at risk. Finally, keep wsrep_slave_threads aligned with the number of independently writable tables rather than with CPU count.

MariaDB 12.3 improves two of the worst Galera failure modes directly. Incremental State Transfers no longer waste time re-validating foreign keys, so rejoin time drops on schemas with heavy referential integrity. And wsrep_applier_retry_count lets a node retry a conflicting write-set instead of dropping out of the cluster, which is exactly the behaviour you want on hot counters and sequence-like rows.

Tier 2 — Asynchronous replication across regions

Synchronous replication across a wide-area link is an availability anti-pattern: it converts network latency into commit latency and a partition into an outage. Instead, replicate asynchronously from the active regional cluster to a standby cluster using GTIDs.

This is where MariaDB 12.3 changes the calculus. Asynchronous replication between two Galera clusters can now apply in parallel using slave_parallel_threads, so the cross-region channel no longer serialises the write throughput of an entire region. Combine it with slave_parallel_mode=optimistic, GTID-based CHANGE MASTER TO ... master_use_gtid=slave_pos, and enforced TLS defaults on the channel. Add a semi-synchronous acknowledgement with rpl_semi_sync_master_wait_point=AFTER_SYNC only if your RPO requirement is stricter than the replication lag you can actually sustain, and always configure a wait timeout so the primary degrades to asynchronous rather than stalling when the standby is unreachable. Our guide on horizontally scaling MariaDB covers the read-scaling side of this topology.

Tier 3 — The routing tier is what the application actually sees

Applications should never hold a connection to a database node. They connect to MaxScale, which owns topology knowledge and failure detection. Deploy at least two MaxScale instances behind a virtual IP or a Kubernetes service, configure the readwritesplit router, enable transaction replay so in-flight transactions survive a backend failover, and enable causal reads so a read issued immediately after a write is routed to a node that has applied it. Set max_slave_replication_lag to keep stale replicas out of the read pool.

MariaDB 12.3 also carries a connection redirection mechanism in the client/server protocol, which lets a proxy hand a client off to the correct node instead of proxying every packet. For very high connection counts this removes the proxy from the data path once routing is established. ProxySQL remains a valid alternative where query rewriting and fine-grained rule sets matter; see our notes on troubleshooting ProxySQL in high-velocity ingestion.

Tier 4 — Declarative operations on Kubernetes

If you run on Kubernetes, the mariadb-operator expresses this whole topology as custom resources. It supports both asynchronous replication and synchronous Galera topologies, manages MaxScale as a first-class object, performs cluster-aware rolling updates that roll replica pods first and the primary last, and supports blue/green upgrades across two identical clusters so a version change becomes a traffic switch instead of a restart. Physical backups run through mariadb-backup and VolumeSnapshots, and binary log archiving enables point-in-time recovery. Our article on tuning MariaDB for cloud and containerized environments covers the resource-limit and storage-class decisions that make or break this layer.

Tier 5 — Backups are the only true fault tolerance

Replication propagates mistakes at the speed of the network. Backups do not. Run daily full plus hourly incremental mariadb-backup jobs streamed to object storage with versioning and object-lock enabled, replicate the bucket cross-region, and enable InnoDB log archiving so you can replay to a specific log sequence number. Under the InnoDB-based binary log, backups now include binlog files transactionally, which removes the historical seam between a physical backup and the binlog stream.

Then do the part most teams skip: restore automatically on a schedule, into an isolated environment, and measure the restore time against your stated RTO. A backup that has never been restored is a hypothesis. See our reference on transferring backed-up data to a MariaDB replica.

Production Configuration Baselines for MariaDB 12.3

The following is a starting point for a 64-core, 512 GB, NVMe-backed MariaDB 12.3 node participating in a Galera cluster. Treat it as a hypothesis to be validated with your own workload, not a copy-paste answer.

[mariadb]
# --- Durability and memory ---
innodb_buffer_pool_size          = 360G
innodb_buffer_pool_instances     = 16
innodb_log_file_size             = 16G
innodb_flush_log_at_trx_commit   = 1
innodb_flush_method              = O_DIRECT
innodb_io_capacity               = 20000
innodb_io_capacity_max           = 40000
innodb_read_io_threads           = 16
innodb_write_io_threads          = 16
innodb_adaptive_hash_index       = OFF

# --- Replication (GTID everywhere) ---
log_bin
binlog_format                    = ROW
binlog_row_image                 = MINIMAL
gtid_strict_mode                 = ON
slave_parallel_threads           = 16
slave_parallel_mode              = optimistic
expire_logs_days                 = 7

# --- Galera ---
wsrep_on                         = ON
wsrep_provider                   = /usr/lib/galera/libgalera_smm.so
wsrep_sst_method                 = mariabackup
wsrep_slave_threads              = 16
wsrep_applier_retry_count        = 3
wsrep_provider_options           = "gcache.size=64G; gcs.fc_limit=64; gmcast.segment=0; evs.suspect_timeout=PT10S; evs.inactive_timeout=PT30S"

# --- Aria (internal + temporary tables) ---
aria_pagecache_buffer_size       = 8G
aria_pagecache_segments          = 16

# --- Observability and safety ---
server_audit_file_buffer_size    = 8M
max_execution_time               = 30000
optimizer_record_context         = ON

On a non-Galera MariaDB 12.3 primary that uses GTID replication exclusively, you can additionally enable the InnoDB-based binary log and delete sync_binlog and binlog_checksum from your configuration entirely. Verify replica versions first.

Observability: The SLIs That Actually Predict a MariaDB Outage

Most MariaDB dashboards measure the wrong things. Query counts and buffer pool hit ratios are descriptive; they are rarely predictive. The signals below lead incidents by minutes to hours:

• wsrep_flow_control_paused — the fraction of time the cluster was paused by flow control. Anything sustained above 0.02 means the slowest node is governing your write throughput.
• wsrep_local_recv_queue_avg — a rising average means an applier cannot keep up and a certification failure storm is likely.
• wsrep_cert_deps_distance — the achievable parallelism of your workload; it tells you whether raising wsrep_slave_threads would help at all.
• InnoDB history list length — unbounded growth means purge is behind, and purge lag is the silent precursor to a storage emergency.
• Commit latency p99 and fsync rate — the pair that reveals whether the InnoDB binary log change delivered the throughput it promised.
• Replication lag distribution, not average — cross-region RPO is defined by the tail, never by the mean.

Alert on error-budget burn rate rather than static thresholds, and keep a runbook link in every alert. Our field guide to MariaDB performance monitoring metrics expands each of these into queries and thresholds, and troubleshooting MariaDB performance covers the diagnostic path once an alert fires.

Failure Modes and Anti-Patterns to Avoid

• Two-node Galera clusters. A two-node cluster has no majority; losing either node halts the survivor. Use three, or two plus a garbd arbitrator.
• Writing to all Galera nodes without partitioning. Multi-master is a topology, not a write strategy. Route writes for a given hot table to one node to avoid certification conflicts, even in MariaDB 12.3 where retries soften the impact.
• Raising gcs.fc_limit to silence flow control. You are increasing unreplicated data in flight while hiding the node that is actually slow.
• Enabling the InnoDB binary log on a Galera node. It is unsupported in that topology; verify before you roll a configuration change fleet-wide.
• Relying on Seconds_Behind_Master alone. It reports apply lag, not the transport gap. Use GTID position deltas and a heartbeat table.
• DDL without a plan. Schema changes in Galera can block the entire cluster under Total Order Isolation. Use Rolling Schema Upgrade or an online schema change tool with explicit checks.
• Untested failover. If you have not performed a switchover in the last quarter, your RTO is an estimate, not a commitment. Related reading: MariaDB high availability best practices and MariaDB deadlock troubleshooting.

Upgrading from MariaDB 11.8 to MariaDB 12.3

Because MariaDB 12.3 is the first LTS after 11.8, the upgrade crosses four release boundaries of accumulated change. Three categories of breakage deserve a pre-flight check.

New reserved words. CONVERSION, ST_COLLECT and TO_DATE are now reserved. Any identifier using them must be backtick-quoted. Grep your schema and application SQL before the maintenance window, not during it.

Removed system variables. big_tables, large_page_size and storage_engine were removed in the 12.0 line. If they remain in my.cnf, the server will refuse to start.

A GTID setting regression in 12.3.2. When a replica is upgraded from a pre-12.3 release directly to 12.3.2, the master_use_gtid setting from CHANGE MASTER TO is not carried over and resets to DEFAULT. This is fixed in 12.3.3. If you land on 12.3.2, re-apply master_use_gtid immediately after the upgrade and verify with SHOW REPLICA STATUS. Downgrades are unaffected.

The safe rollout order for a Galera plus async topology is: upgrade the DR region first, then read replicas, then the non-primary nodes of the active cluster one at a time, then the primary via a controlled MaxScale switchover. Take a verified mariadb-backup before the first node, run mariadb-upgrade where required, and keep the old binaries installed until you have completed a full business cycle on the new version. Follow the official MariaDB upgrade documentation for platform specifics.

Benchmarking and Capacity Planning for MariaDB 12.3

Do not accept the throughput claims of any release — including this one — without measuring them on your own hardware and workload shape. A defensible benchmark for the MariaDB 12.3 binary log change looks like this:

1. Capture a production workload sample and replay it, rather than running a synthetic uniform-random benchmark that no application resembles.
2. Measure with the traditional binlog at sync_binlog=1, innodb_flush_log_at_trx_commit=1 as your control.
3. Re-measure with the InnoDB-based binlog at innodb_flush_log_at_trx_commit=1. Record commit latency percentiles, not averages, and record device-level fsync counts.
4. Run a third pass at innodb_flush_log_at_trx_commit=2 to quantify what relaxed durability actually buys now that consistency between log and engine is guaranteed regardless.
5. Repeat each pass with a deliberately induced crash to confirm recovery time and GTID continuity.

Capacity planning should then be expressed in headroom rather than utilisation: a cluster running at 70% of measured write capacity has no room to absorb the loss of a node, because the survivors inherit the load and the certification cost of the failure. Size for N-1 in region and N-1 across regions.

Frequently Asked Questions About MariaDB 12.3

Is MariaDB 12.3 a long-term support release?

Yes. MariaDB 12.3 reached Stable/GA on 28 May 2026 and is maintained until June 2029. It is the successor LTS to MariaDB 11.8.

Should I enable the InnoDB-based binary log in MariaDB 12.3?

Enable it on GTID-only replication topologies where you control the tooling and all replicas run MariaDB 12.3 or later. Do not enable it on Galera Cluster nodes, on systems that depend on filename/offset positions, or where third-party tools parse binlog files from disk.

Does MariaDB 12.3 change how Galera Cluster behaves?

Yes, in three ways. Incremental State Transfers skip redundant foreign key checks, write-set application can be retried via wsrep_applier_retry_count, and asynchronous replication between two Galera clusters can now apply in parallel. Galera is also no longer a package dependency of the server, so it must be installed explicitly.

What is the fastest safe upgrade path from MariaDB 10.11 to MariaDB 12.3?

Upgrade in LTS steps — 10.11 to 11.4, 11.4 to 11.8, then 11.8 to MariaDB 12.3 — validating application compatibility at each stop. Skipping LTS boundaries is technically possible but leaves you without a tested rollback point.

How many nodes do I need for a fault-tolerant MariaDB 12.3 cluster?

Three synchronous nodes per region, one per availability zone, plus at least one asynchronous standby cluster in a second region. That combination survives a node failure with zero data loss and a full region failure with sub-second RPO.

Do optimizer hints in MariaDB 12.3 replace query tuning?

No. Hints are a containment tool for plan regressions and a way to protect a fleet from a single pathological statement. Schema design, indexing and statistics remain the durable fix.

Conclusion

MariaDB 12.3 is the most consequential MariaDB release in several years because it changes the cost of durability rather than merely adding syntax. Storing the binary log inside InnoDB collapses the fsync budget of a commit and makes crash safety structural. The optimizer hint framework converts plan stability from a global gamble into a per-statement contract. Galera improvements shorten the rejoin window and reduce spurious evictions. None of that, however, produces availability on its own. Availability comes from arranging those primitives across failure domains: synchronous replication inside a region, asynchronous GTID replication across regions, a routing tier that hides failover from applications, declarative operations, and backups that are restored on a schedule rather than trusted on faith.

If you are planning a MariaDB 12.3 upgrade, a Galera redesign, or a move to multi-region topology, MinervaDB consultative support provides 24×7 engineering for MariaDB, MySQL and PostgreSQL infrastructure at internet scale. You can also review our MariaDB storage and virtualization whitepaper for the infrastructure layer beneath this architecture.

Further reading and authoritative references

• MariaDB 12.3 changes and improvements (official release notes)
• InnoDB-based binary log documentation
• MariaDB Galera Cluster documentation
• MariaDB MaxScale documentation
• Global Transaction ID reference
• Backup and restore, including point-in-time recovery
• Download MariaDB Server from MariaDB Foundation

MariaDB 12.3 LTS: Advances and Building Fault-Tolerant MariaDB Infrastructure at Internet Scale appeared first on MariaDB.org