← Back to Blog

From Big Iron to AMD EPYC: Preparing TAF to Push MariaDB at Scale

Some of the first large database servers I worked with were DEC VAX/VMS systems running Sybase.

They were physically huge machines. Towers of quad 90.91 MHz processors, with more towers beside them holding mirrored RAID5 storage. At the time, this was big iron, taking food stamps in Texas to debit cards.

A lot has changed since then.

But the questions that interest me as a performance engineer have not changed nearly as much.

What happens when we give the database more machine?

How far does performance scale? Where does it flatten out? What part of the database begins limiting throughput? And when more hardware no longer produces more performance, why?

For MariaDB Foundation, those questions are about to get considerably more interesting.

AMD Is Bringing Serious Hardware to MariaDB Foundation

AMD is becoming a sponsor of MariaDB Foundation, and as part of that relationship the Foundation is preparing to benchmark MariaDB on high-end AMD EPYC server hardware.

This is exciting for more than the obvious reason.

Yes, having access to a large modern AMD system gives us considerably more CPU, memory bandwidth, memory capacity, and I/O capability to work with.

But simply putting MariaDB on a very large server and reporting a large benchmark number would leave most of the interesting questions unanswered.

We want to understand how MariaDB gets there.

So before the hardware arrives, we have been preparing TAF, the MariaDB Foundation Test Automation Framework, with two complementary groups of benchmark cases: performance profiles and host-scale tests.

The goal is straightforward:

Measure MariaDB. Find where performance is being spent. Find where scaling changes. Use what we learn to make MariaDB as fast as we can make it.

Profiling MariaDB Across Hardware

TAF Profile Test Cases:

https://github.com/MariaDB/TAF/commit/20cbeaf1c78178bd4c502fee195f12b4b7267347

The first part of the work is the set of MariaDB performance profiles just pushed into TAF.

These profiles are designed to isolate individual MariaDB subsystems and features so their performance cost can be measured rather than guessed.

The profile suite includes cases covering areas such as:

  • binary logging
  • InnoDB-based WAL/binlog behavior
  • semi-synchronous replication settings
  • transaction isolation
  • Adaptive Hash Index
  • InnoDB doublewrite
  • innodb_flush_log_at_trx_commit
  • Performance Schema instrumentation levels
  • thread pool behavior
  • query cache
  • character sets
  • SSL
  • general and slow query logging

The purpose isn’t to declare that one configuration is universally fastest.

It is to change one thing, run the same workload, and measure what that change costs on the hardware underneath it.

That becomes especially interesting when the same profile can be run across very different machines.

A subsystem whose overhead is almost invisible on one system may become important on another. A bottleneck obvious on a smaller system may disappear when given more resources, exposing something deeper inside the server.

TAF gives us a repeatable way to find those differences.

Now We Are Adding Host-Scale Testing

TAF Scale Test Cases:

https://github.com/MariaDB/TAF/commit/96aecbe8c57db9d1988d636d4b78563406f37bc3

The second half of the preparation is new.

We have just pushed a dedicated set of MariaDB scale test cases to TAF.

The initial scale suite covers four workload families:

  • sysbench read workloads
  • sysbench read/write and write workloads
  • HammerDB TPROC-C OLTP workloads
  • HammerDB TPROC-H analytical workloads

Rather than creating test cases useful only on an enormous server, the scale suite is divided into three practical hardware classes.

Small VM

4–8 vCPU

This represents the kind of relatively constrained virtual-machine environment many developers and users actually run.

Medium Host

24–36 vCPU

This begins exercising MariaDB at significantly greater concurrency while still representing a fairly ordinary dedicated database-server class.

Large Host

48–96 vCPU

This is where the upcoming AMD hardware becomes key.

The same TAF methodology can move from a small VM (cloud simulation) all the way to a high-core-count EPYC system (High-end) .

CPU Affinity Gives Us the Comparison

One of the questions I initially had about testing such a powerful machine was surprisingly basic:

What do we compare it to?

A benchmark result from a 96-vCPU server is certainly interesting, but by itself it primarily tells us what happened on that particular configuration.

CPU affinity gives us something much more useful.

TAF now supports restricting the database engine to a selected set of CPUs:

taf.db_cpu_affinity

That means a sufficiently large server can represent multiple controlled CPU allocations.

Conceptually, we can test MariaDB using progressively larger portions of the same host:

4 → 8 → 24 → 36 → 48 → 96 vCPU

The exact CPU lists will be determined from the topology of the hardware itself. Core, SMT, socket, and NUMA layout matter, so CPU groups should be selected from the real topology rather than assumed from logical CPU numbering.

But once those groups are defined, the experimental model becomes very powerful.

We can hold the machine, database version, server configuration, dataset, and workload constant while changing the amount of compute available to MariaDB.

Then we can ask:

How much performance did the additional CPU capacity actually buy us?

Scaling the Workload Too

CPU availability is only one dimension.

The scale cases separately control:

  1. Available CPU
  2. Database size
  3. Workload concurrency

Sysbench can increase thread counts and dataset sizes.

TPROC-C can independently increase virtual users and warehouse population.

TPROC-H can increase analytical database Scale Factor while preserving its own query-stream methodology.

Keeping these controls separate matters.

If CPU count, database size, concurrency, and MariaDB configuration all change at the same time, a bigger result tells us very little about why performance changed.

TAF lets us vary these things deliberately.

That gives us comparisons such as:

Same workload + same database + same MariaDB + more CPU

or:

Same CPU allocation + same database + more concurrent users

Those are far more useful questions than simply asking for the largest benchmark number the machine can produce.

Finding Saturation Is a Result

The new cases deliberately push concurrency beyond the comfortable portion of the curve.

That is intentional.

If throughput continues rising, we learn that MariaDB and the hardware still have room.

If throughput flattens, that is useful.

If additional users reduce throughput, that is useful too.

Then we can reproduce the interesting point with TAF profiling enabled and investigate what changed.

CPU contention, locks, InnoDB behavior, memory access, I/O, scheduling, or another MariaDB subsystem may become the next thing to investigate.

The purpose of a scale test isn’t to avoid saturation.

Finding saturation is part of the test.

In closing

It is amazing to look back at those early transaction-processing systems and see how far both the hardware and database software have come.

From the VAX/VMS and Sybase systems I worked with then to the AMD EPYC hardware and MariaDB the Foundation is preparing to test today, the technology has advanced by leaps and bounds.

The questions we ask of it, however, remain remarkably familiar.

Thank you to AMD for helping the Foundation answer them.

To all, “Happy benchmarking!”

Jonathan (Jeb) Miller has experience in military leadership, Fire/EMS leadership, computer operations leadership, teaching, and more than 27 years of database performance engineering (PervasiveSQL, MySQL, MariaDB).

TAF is the third benchmarking framework he has helped design and build.

#MariaDB #MySQL #PostgreSQL #Database #TAF #DatabasePerformance #Performance

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *