It’s been a while since we posted about community organised events. I’m happy to have the chance to talk about my experience attending Percona University in Athens.
For those in a hurry, the main topics are:
1. Databases in the cloud, why vendor-lock is important to consider.
2. How to build Open Source databases, FerretDB and TiDB, two completely different approaches.
3. Upgrading from MySQL 5.7, monitoring and AI to speed up things.
Seeing familiar faces, discussing database experiences
When working within the MariaDB Foundation, it isn’t always easy to get feedback on what works and what doesn’t. It’s easy to get confined into your own SQL ivory tower and not consider how the industry has changed. There are many different approaches to writing applications using databases.
Peter Zaitsev, the founder of Percona has a very pragmatic view on the matter. Rather than proposing a particular technology (be it MySQL, Percona, MariaDB or Postgres), he advocates empowering developers to use what they are more comfortable with, and at the same time help them achieve top performance with the database of their choosing.
Peter Farkas, the CEO of FerretDB, a NoSQL solution was asked: What is better, NoSQL or SQL? The answer: “We shouldn’t even be having this argument! NoSQL and SQL are slowly converging.” Traditional databases are adding NoSQL features, just because it’s obvious that Document Stores are important. NoSQL databases have started to realise the importance of transactions and all the guarantees that a relational database offers. Hence NoSQL solutions are tackling those problems too.
In practice, FerretDB is on the right track there, but more on that later.
The world is moving to the cloud and databases need to allow for that transition
What is clear is that most applications are now written for the cloud. And with a cloud native application, you need a cloud native database. But what is the true cost of having your database provided by a cloud vendor?
Peter Zaitsev tackles this question in his opening session: “Cloud of Serfdom vs Cloud of Freedom: Why Open Source Will Win in the Cloud Age”. The trick is finding which features you should actually use from the cloud. Once you start going very deep into cloud proprietary solutions, you are effectively locked into that provider. Another aspect to be aware of is that the more data you have, the harder it is to move it to any other platform. A nice metaphor used by Peter: “data has gravity, the bigger it gets, the harder it pulls”.
An open source MongoDB alternative
Listening to Peter Farkas’s talk on what FerretDB is was truly eye opening. I won’t go into too many technical details, but what Peter has identified is really important from an Open Source perspective, compatibility and interoperability! Let me explain:
MongoDB is obviously no longer an Open Source product, but the world needs an open source NoSQL solution. And what’s even more important, developers are using MongoDB’s querying API, which is in fact… pretty good. You can’t just start rewriting all applications to use a different NoSQL database, it’s too expensive, and that’s what causes “vendor lock-in”.
So FerretDB solves this lock-in problem by offering an API compatible with MongoDB’s connectors. An application could run queries against FerretDB and think it’s running them against MongoDB. That’s brilliant! As a side note, that’s what makes MariaDB’s Oracle mode so good. Applications written for Oracle work just as well with MariaDB.
Now the other part of the problem that FerretDB has to solve is how do you actually store all that data? Writing a database from scratch takes a lot of resources… That’s not doable for a startup. But then why not leverage an already existing solution? Right! The traditional relational database is pretty good at storing and retrieving data. So FerretDB is effectively an adapter between NoSQL land and SQL land. And to top it all off, the SQL backend is not set in stone! Yes, currently the database backend for FerretDB is Postgres and it can support Sqlite too. But Peter was quite open to supporting a MariaDB compatible contribution should there be some community support with getting it coded. If you’re reading this blog and are interested in picking this up, it would be quite exciting for the Foundation to get MariaDB support in there too!
TiDB, a MariaDB & MySQL compatible distributed database
Mattias Johnson, a very seasoned developer in the MySQL and MariaDB land presented an overview of TiDB. Remember when I said that building a database from scratch takes a lot of effort? Yep, TiDB is just that. A built from the ground up cloud native, distributed database. But even so, they are standing on the shoulders of giants, by leveraging the storage layer of RocksDB.
Note, MariaDB has the same RocksDB storage option available.
Mattias has offered a very pragmatic view on TiDB, showing where it shines – scaling reads,writes and storage to hundreds of terabytes of data across multiple nodes but also what are the limitations – network round trip penalties due to the distributed nature of the system.
One thing to note for TiDB is that it has tools for moving data in and out of TiDB, to many different platforms, be it Apache Kafka, Apache Spark or MariaDB. I’m keen to work with TiDB on making sure that users are able to push data from MariaDB to TiDB, but also be able to get it out and run it in MariaDB when the application’s needs require MariaDB’s specific strengths, such as very low latency queries.
Other noteworthy talks
All of the talks at Percona University were interesting, however these stuck out in my mind:
- MySQL 5.7 Goes EOL: How to Upgrade to MySQL 8
There are a lot of challenges when migrating to MySQL 8, as there are quite some new features added. On top of that, depending on which MySQL 8 minor version you pick, to jump to, there will be differences. Nonetheless if that is what you want to achieve, Ananias Tsalouchidis does a good job of listing the possible gotchas.
As for my position, I stand by MariaDB’s approach of not introducing incompatibilities unless absolutely necessary. In this sense, MariaDB 10.11 LTS is much more compatible with MySQL 5.7 than MySQL 8 is. So if you’re looking for an easier upgrade path, consider switching to MariaDB. From 5.7 we can even do an in-place data-directory migration! - Netdata is a great open source monitoring tool that leverages AI to detect anomalies. A little bit like Zabbix on steroids, combined with Grafana. Pretty cool Open Source project, one that we will at least try out with our mariadb.org infrastructure.
Conclusion
It was great to poke out of our MariaDB bubble for a change. Seeing how different people tackle similar data related challenges was quite refreshing. Lessons learned for me:
- Interoperability is king! Users are much more likely to migrate to your solution if they have an easy way to move out, should the solution not be good enough. In the spirit of Open Source, we need to focus on “open standards” and whenever possible code for compatibility.
- Modular architecture is really important. Having a plugin system encourages external contributions. FerretDB wouldn’t have gotten Sqlite support if the storage layer weren’t easily swappable. MariaDB has storage engines. In the future I would love to see if FerretDB can use MariaDB as a back end.
- At times, it just makes sense to use a well established API. MongoDB obviously did great with its SDKs. That’s why at MariaDB we try to implement all our features as close to the SQL Standard as possible.
And with all this said, remember that MariaDB Foundation is hosting a meetup just like this one in December! Come join us in Germany, for a fully packed, highly technical MariaDB day!