Open Source Performance, Benchmarks and MariaDB: A Conversation with Steve Shaw

Right after the recent MariaDB Meetup in London, I had the pleasure of sitting down with Steve Shaw – founder and CEO of HammerDB, former Principal Engineer at Intel, and board member of the MariaDB Foundation. Steve delivered the keynote talk at the event and has a unique perspective on performance, open source, and the intersection between commercial and community-driven database development.

This blog post captures our conversation – a dialogue between two people passionate about open source and database performance. We talk about Steve’s transition from Intel to HammerDB, his relationship with MariaDB Corporation, the nuances of performance testing, and what lies ahead for the HammerDB and MariaDB communities.

From Intel to HammerDB

Kaj: I’m sitting here with Steve Shaw, Board Member of the MariaDB Foundation, who just gave a superb keynote talking about MariaDB Server performance at our successful MariaDB Meetup last week at the Amazon AWS UK headquarters in Shoreditch in London. Hi Steve!

Steve: Hi Kaj!

Kaj: This is not our first video chat session. Last time I interviewed you in your role as Principal Engineer at Intel. But in London you gave your talk as founder and CEO of HammerDB, the standard tool for database performance testing. Can you tell us a bit about what happened?

Steve: Yes, well Intel had some big organisational changes and from my perspective they were moving away from where my core skill set is – around databases. That gave me a decision to make, and it was in some ways easy because it allowed me to follow my passion for open-source databases. Intel had assigned the copyright of HammerDB to me, so it made sense to set up a company around it. Timing was great too – the TCL language HammerDB is based on had its first new release in 27 years, so I finally had time to create the version of HammerDB I really wanted. That’s now version 5, released two weeks ago.

Collaboration with MariaDB and Performance Focus

Kaj: Your performance presentation was the keynote of the MariaDB London Meetup… So this must mean you’re deeper into MariaDB than just sitting on the MariaDB Foundation board?

Steve: Yes. Having been at Intel for a long time, the term “fellow traveller” comes to mind – focusing on open-source database performance aligns closely with MariaDB’s mission. It’s a good partnership. HammerDB supports many databases – both open source and commercial – so it’s not exclusive, but there’s a natural synergy.

Kaj: Let me ask you about your affiliation with MariaDB Corporation. You said you’re a consultant working for HammerDB, and HammerDB Ltd is a Private Limited Company. But in Jira, MariaDB’s project management and issue tracking tool, I’ve seen the email address steve.shaw@mariadb.com. So are you now also an employee of MariaDB Corporation?

Steve: No. I think of it like when I worked at Intel and had an Oracle badge and email – I was working for Intel. It’s the same here. I’m bringing my HammerDB expertise to MariaDB PLC, focused on performance improvements and automation. HammerDB pays the salary.

Benchmarking and the DeWitt Clause

Kaj: Thanks for that. So your presentation was titled “Comparing Open Source and Proprietary Database Performance”. How can you even do such a comparison, given the “DeWitt Clause” some databases have?

Steve: That’s a great point. The DeWitt Clause, named after Professor David DeWitt, prohibits publishing benchmarks without company approval. Oracle and SQL Server have it. What’s less known is that some cloud-based services built on open-source also include such clauses. It’s important to be aware, because publishing benchmarks without consent can lead to legal consequences. That’s why we focus on open source – with HammerDB, you can benchmark freely and legally.

TPC, Community Standards, and Openness

Kaj: Can you educate us a bit about TPC – the Transaction Processing Performance Council? What should MariaDB users know about TPC-C and TPC-H?

Steve: TPC is a non-profit that brings order to the benchmarking world. TPC-C is for transactional workloads, TPC-H for analytical ones. HammerDB implements workloads based on these specs. In 2019, HammerDB was adopted by TPC, and I’m now involved in their TPC OSS subcommittee, ensuring level playing fields and reliable benchmarks.

How Does MariaDB Compare?

Kaj: So what was the outcome of your comparisons? Are open-source or proprietary databases more performant?

Steve: Commercial databases do perform very well – they’ve had decades and huge investments. But hardware evolves fast. Moore’s Law means a modern open-source database on new hardware can outperform anything on older systems. That’s why benchmarking is important. MariaDB is making huge gains, which is why I’m contributing.

Beyond Benchmarks: HammerDB as a Tool for Testing Configurations

Kaj: Can one use HammerDB for other purposes than picking which database to use?

Steve: Yes, definitely. HammerDB includes step workloads, active session history views, and a web service that graphs performance. It’s not just for choosing a database – it’s for testing different versions, configurations, and hardware setups.

Impressions from the London Meetup

Kaj: Let’s now talk about the meetup. How did your presentation go? Did you get interesting questions?

Steve: Yes, absolutely! There was a lot of interest in DeWitt clauses, in the systems I tested, and in replicating the results. That’s the point of a meetup – sharing knowledge so others can try it out themselves.

Kaj: What about the other presentations? I heard Alex Hanshaw, VP Software Engineering, presented MariaDB Vector Search by comparing an ant and a horse to database vectors containing an alarm clock, a cow, a bicycle and an eagle. That sounded ingenious.

Steve: Absolutely. It was the best presentation I’ve seen on vector databases – clear explanations of what vectors are, how attributes like speed and size can be translated to data. Very educational. Other talks, like on cloud storage and MariaDB HA on RDS, were also excellent – made me think about new things to test myself.

Reflections and Recognition

Kaj: In your role as MariaDB Foundation board member, what’s your feedback to us who work in the Foundation day to day?

Steve: The Foundation is all about community, and initiatives like meetups are fantastic. The more interaction and feedback, the better. You’ve got great things to show – it’s excellent that you’re out there sharing them.

Kaj: Thanks for giving us so much to show! And thanks to AWS for hosting the event. I heard you finally met Tarus Balog, our liaison at AWS. How did that go?

Steve: That was great. AWS awarded me an open-source medal of sorts – recognition of community contributions. They’re deeply engaged, they hosted us, and gave helpful feedback. They’re supporting open source more and more, and there are more AWS-hosted meetups coming.

Concluding Thoughts

Kaj: Indeed thank you AWS! Any concluding words, Steve – on the meetup, on performance, on MariaDB, or perhaps on HammerDB?

Steve: A lot’s happening. We’re seeing 60% performance gains over MariaDB 10.6 and 32% over 11.4 – all transparent and visible in Jira. We’re making HammerDB easier to use, improving services, adding ARM support, and preparing for community-contributed benchmarks on the TPC site. It’s exciting to focus 100% on open-source database performance.

Kaj: Thank you Steve!

Steve: Thank you Kaj!

Final words

The conversation with Steve was a reminder of just how intertwined community and innovation can be. With a background that spans major corporations and open-source leadership, Steve brings valuable insights on how to bridge enterprise-grade performance expectations with the freedom and transparency of open tools.

HammerDB is more than just a benchmark tool – it’s a symbol of what’s possible when expertise and open collaboration meet. As MariaDB continues to evolve, and as initiatives like TPC OSS expand, we’re excited to keep pushing the boundaries of what open-source databases can achieve.

If you’re interested in database performance, open source tooling, or community-driven standards – stay tuned. And join us at our next MariaDB Meetup, in Dublin Thu 5 June 2025.