Thoughts on MariaDB Server 10.3 from MariaDB Developers Meeting in Amsterdam, part 1
I had the honor of leading a session on Saturday, during the MariaDB Developers Meeting in Amsterdam, brainstorming around MariaDB Server 10.3. It’s definitely time to do that since MariaDB Server 10.2 has entered beta stage. In case you have missed that, I’ve wrapped up what’s included in 10.2 so far in a blog post on my employer’s site. In addition to the features mentioned in that blog post there are a couple of features still coming in 10.2 of which the most notable is that the MyRocks engine will be included. More about this later in another article.
In this article I will go through the first part of the features that were discussed for MariaDB Server 10.3. There will be a second part with more thoughts on other features coming in a couple of days.
During Google Summer of Code 2016 a few nice features were created that are now being polished to make their way into MariaDB Server 10.3. These are:
- Hidden columns
- Long unique constraints
- SQL based CREATE AGGREGATE FUNCTION
New Data Types such as data types for IPv6 and UUID have long been top voted wishes for MariaDB. Now it looks like one step ahead has been made with the implementation of a Pluggable Data Type API. It still exists in a separate branch, but it is now planned for inclusion in MariaDB Server 10.3. On top of the API the idea is to build User Defined Types according to the SQL Standard.
Another very much wished for group of features is to provide better support for CJK (Chinese, Japanese, and Korean) languages. The idea is to include the ngram full-text parser and MeCab full-text parser plugins. Also the GB 18030 standard for Chinese charsets is on the radar for 10.3.
On the storage engine side MyRocks will be included in 10.2, which will open MyRocks to a wider audience. It’s obvious that it will need work during 10.3 timeframe to grow in maturity. Another interesting storage engine discussed during the last few years is Spider, an engine that provides a sharding solution for databases. Although Spider already exists in MariaDB Server, there is a set of patches that needs to be incorporated on the server side to improve functionality and performance. These patches can be found in MDEV-7698.
The most popular storage engine, InnoDB, was also discussed during the meetup, including thoughts on interesting InnoDB features for 10.3. Most features from InnoDB in MySQL 5.7 will be available in MariaDB Server 10.2, but native partitioning will not. Native InnoDB partitioning was discussed, but since partitioning is done at the server level in MariaDB the thinking is that partitioning inside InnoDB will only be included if there is a clear advantage to using it over the server provided partitioning.
In MySQL 8.0 there are a couple of interesting new features for InnoDB that are interesting for MariaDB as well: InnoDB deadlock detect and the new information schema table. The timelines of 10.3 and 8.0 will partially determine whether these will be in 10.3. There are also a couple of very interesting contributions that will be evaluated for inclusion in MariaDB 10.3: “Lock wait policy” and “Persistent autoincrement”.
It seems likely that there will be contributions made from MariaDB Corporation in the area of database compatibility. Including things like missing SQL standard functionality and functionality provided by other database servers. In 10.2 there have been features added in this area like enforced check constraints, improved DEFAULT values and support for multiple triggers per table. Some features that fall into this category, which are targeted for 10.3 are:
- Support for SEQUENCE
- Addition of a PL/SQL parser
- Support for INTERSECT
- Support for EXCEPT
I’ll end this first part of the thoughts from the 10.3 feature session here and come back with the second part in a couple of days.
We’re very interested in hearing your feedback on this! It’s definitely not too late to let us know what you think is important for 10.3. You can comment directly on this blog post or you can share your input on the maria-discuss mailing list. Another way of reaching the developers and others around MariaDB is to reach out on Freenode IRC on channel #maria. If you have input on a specific feature you can also comment on the feature directly in MariaDB’s JIRA.
I would like to see string function improvements. Such as a function to split a string into a table.
Functions supporting return type of table so functions can return usable tables.
And udf’s in python.
And ability for federation engine to call out to any db using odbc.
Have you already tried the CONNECT engine and type ODBC, https://mariadb.com/kb/en/mariadb/connect-table-types-odbc-table-type-accessing-tables-from-other-dbms/ ?
Can’t wait for SEQUENCE objects to land!
Any word on when Galera 4.0 might land?
something like postgres new multi threaded query would be nice to have some day
You can use spider in 10.2 it should support multi threaded query based on partitioning
Multi-threaded by itself is not the real problem when trying to make large requests run faster..
Transaction systems need acid compliance and doing what is necessary for this on large requests
at some point involves disk I/O.
Taking a request and allowing it to run in multiple threads AND then taking each thread and allowing that thread to
to do very small pieces {100’s vs Millions} gets you 10 x – 20 x improvement vs the 2-3 x from just multi-threading.
We have been running a stored procedure/event driven environment that does both on a subset of SQL statements
on INNODB tables for over a year. In 1 example this runs a 4 + hour INSERT/SELECT SQL in 20 minutes.
Is there any tentative timeline when Maria DB 10.3 will be released ?
We are looking forward for password expiration and password history/ reuse features such as https://jira.mariadb.org/browse/MDEV-9245 and https://jira.mariadb.org/browse/MDEV-7597 which are most likely to be available in 10.3 version.
Best Regards,
Dinesh.
In the past, the next release has first come out in alpha a few months after the previous release is stable. 10.2.3 is still beta, so 10.2.5 will be the earliest stable release. Very roughly then, the first 10.3 alpha will probably be later this year, and the first stable next year. See https://jira.mariadb.org/secure/Dashboard.jspa for more accurate estimates when the time is closer.
I spent a lot of time lately looking at and thinking about seconds_behind_master for replication. It occurred to me this past weekend that if in the relay log, you were to include the timestamp of the replica for each event at the time the event is entered into the relay log, we could get a better sense of network latency issues. What I was thinking is that the current seconds_behind_master could be changed to seconds_behind_Io_thread to be more accurate of what it represents. Add a new value in show slave status; called last_network_latency which would look at the difference between the timestamp in the latest entered event in the relay log vs the same events timestamp from the master. This still wouldn’t be a perfect representation of true latency of the master, but it would give a better idea of network latency. This would also open up the possibility for writing utilities to digest the relay logs to determine historical latency issues after the events made it to the replica.
On a secondary note, if possible, it would be nice to change both of these from seconds to milliseconds to get more granular.