Extending MariaDB with Native Aggregate Plugins: Laying the Groundwork for HyperLogLog

MariaDB already allows developers to add new Pluggable Data Types and scalar Plugin Functions. One missing piece has been Pluggable Aggregate Functions operating on PDTs. That matters for functionality such as HyperLogLog, where an extension needs to aggregate values into a custom statistical sketch while preserving its native SQL type.

MDEV-40672 closes that architectural gap. Aggregate functions can now participate in MariaDB’s native aggregation infrastructure, including DISTINCT, window execution, prepared statements and Pluggable Data Types.

MariaDB already supports MariaDB_FUNCTION_PLUGIN, a mechanism for registering SQL functions that create regular server Item objects and can behave almost like built-in functions.

Writing User-Defined Functions in Rust

One of the most straightforward ways to add functionality to MariaDB or MySQL server instances is by creating a user-defined function (UDF). These are compiled functions loaded from dynamic binaries that can be much more performant and flexible than what can be written in SQL, providing the same capabilities as builtin functions.

These UDFs are typically written in C or C++, but a library is now available that makes it easy to write them in Rust. This blog discusses some of the reasoning for writing this library, followed by a very basic usage example that doesn’t require any experience with the language.