Create a link to a remote server and access data using CONNECT SE

In this blog we are going to see how to create a link to a remote server and use it to access multiple tables at once.
In the previous blog we have seen how to establish a remote connection between Docker containers.
The way we did it was to specify the connection string to reference a single table only.
But what if we need more tables, what if need a whole database?

The solution is to link to a remote database with the CREATE SERVER statement.
A link obtained this way can be passed to the CREATE TABLE statement of a storage engine (SE) to make a connection where using the table discovery feature SE will find out about the table fields and create the table.

Tricky Problems? MariaDB debug container

MariaDB does have bugs. Users see them sometimes. Sometimes developers look for a long time at bug reports and code and still cannot see how the situation occurred. Developers during their analysis ask questions like:

  • I wonder if this was already fixed in {not released version}? But how can I ask a user to test that?
  • Can I get the user to get good stack trace that would help understand this better? But users sometimes find this hard.
  • What exact hardware and kernel configuration is this bug occurring on? And how would I reproduce this?

Using CONNECT SE to access remote JSON data with demo examples in Docker

CONNECT is a storage engine (SE) plugin used to access external, local or remote data. In this blog we are going to show how to install the CONNECT storage engine in a Docker container and how to share JSON data between containers.

Enabling CONNECT SE plugin in Docker

CONNECT SE needs to be installed within the container in order to use it. To see how to do that please check Installing plugins in the MariaDB Docker Library Container.

Create JSON data on remote server

The most important feature of CONNECT SE to MariaDB is the flexibility to create tables from various data sources, like the same database and other DMBS’s tables or files with different formats.

Installing plugins in the MariaDB Docker Library Container

MariaDB plugins are software components that may be added to the core software without the need to rebuild the MariaDB Server. Plugins can be storage engines, additional security requirements, special log information about the server and others. MariaDB has a large number of built-in plugins which are permanently installed (listed in SHOW PLUGINS query). Plugins can be loaded at start-up, during initialization, or loaded dynamically when the server is running.

In this blog we are going to see how to list available plugins in the MariaDB container as well as the methods of installing plugins in a container.

Purge of End of Life versions from Mirrors

OSU Open Source Labs have supported MariaDB for many years in hosting the mirror from which we distribute our releases for which we are most grateful. We were asked if we could reduce the size as our mirror usage had grown to 1TB, rather high compared to other projects.

To accommodate this request we removed all of the end of life releases, 5.5, 10.0 and 10.1 from the mirrors halving the used space.

MariaDB space usage on OSU Open Source Lab mirror

It turns out there were some continuous integration systems still using these repositories.

Developer tip: test MariaDB install/upgrade quickly with Docker

Here is a quick tip for any developer who might want to test if the latest development version of MariaDB installs/upgrades. Traditionally, developers seem to have a bunch of virtual machines lying around which they use to test MariaDB installation and upgrade related things. Snapshotting virtual images, keeping them up-to-date, starting, stopping etc. takes a lot of time and does not feel very convenient.

A much faster option would be to use pristine Docker images for every test. Docker images however normally only run one process and thus do not simulate a complete operating system and lack vital things like systemd interactions. …