MariaDB Foundation assisting multiple vendors

I recently helped two of our sponsors simultaneously, DBS Bank and MariaDB Plc, with a recent issue they were experiencing. This actually helped us add support for an extra cloud vendor. But before I tell the story, I first need to give a bit of background.

S3 Engine History

Back in 2019, we added the S3 engine to MariaDB Server. “S3” stands for “Simple Storage Service”, it was developed by Amazon, and it is pretty much the standard for storing objects (typically files) in the cloud. Almost every cloud vendor has their own object storage with an S3-compatible REST API.

Create a link to a remote server and access data using CONNECT SE

In this blog we are going to see how to create a link to a remote server and use it to access multiple tables at once.
In the previous blog we have seen how to establish a remote connection between Docker containers.
The way we did it was to specify the connection string to reference a single table only.
But what if we need more tables, what if need a whole database?

The solution is to link to a remote database with the CREATE SERVER statement.
A link obtained this way can be passed to the CREATE TABLE statement of a storage engine (SE) to make a connection where using the table discovery feature SE will find out about the table fields and create the table.

Using CONNECT SE to access remote JSON data with demo examples in Docker

CONNECT is a storage engine (SE) plugin used to access external, local or remote data. In this blog we are going to show how to install the CONNECT storage engine in a Docker container and how to share JSON data between containers.

Enabling CONNECT SE plugin in Docker

CONNECT SE needs to be installed within the container in order to use it. To see how to do that please check Installing plugins in the MariaDB Docker Library Container.

Create JSON data on remote server

The most important feature of CONNECT SE to MariaDB is the flexibility to create tables from various data sources, like the same database and other DMBS’s tables or files with different formats.

Installing plugins in the MariaDB Docker Library Container

MariaDB plugins are software components that may be added to the core software without the need to rebuild the MariaDB Server. Plugins can be storage engines, additional security requirements, special log information about the server and others. MariaDB has a large number of built-in plugins which are permanently installed (listed in SHOW PLUGINS query). Plugins can be loaded at start-up, during initialization, or loaded dynamically when the server is running.

In this blog we are going to see how to list available plugins in the MariaDB container as well as the methods of installing plugins in a container.