The post The Power Of The Community! appeared first on MariaDB.org.

The post The Power Of The Community! appeared first on MariaDB.org.

Even minutes of downtime can trigger customer loss. 91% of enterprises report costs over $300,000 per hour, with 44% saying it can exceed $1 million.

At this level of operational risk, databases must scale predictably and adapt quickly. Proprietary databases often work against that goal. Escalating licensing fees, usage-based pricing, and long-term vendor lock-in make it harder to tune performance, expand capacity, or respond to incidents without compounding cost and complexity.

This is why open-source databases have become a core part of modernization efforts across banks, fintechs, and payment platforms. They help keep costs predictable as performance, security, and regulatory demands continue to rise.

The Financial Industry Is Under Pressure Like Never Before

Financial institutions are facing unprecedented pressure, putting legacy systems under strain and impacting performance, costs, and compliance.

Regulatory Expectations Keep Rising

Rising regulatory expectations demand continuous auditing and compliance with PCI DSS, GDPR, AML/ATF, and ESG standards. Complete transparency, high security, and verifiable controls are no longer negotiable.

The proprietary databases introduce friction, since closed architectures and licensing controls restrict the disclosure that regulators require.

Customers Expect Real-Time Performance

Even brief transaction delays or outages trigger immediate loss of trust, especially as fintechs set the benchmark for instant, failure-proof experiences. Traditional release cycles and weekend maintenance windows no longer match customer tolerance.

AI and Data Growth Exceed Traditional Limits

AI workloads and high-dimensional data pipelines drive unprecedented data volume, outpacing the capacity of legacy systems. The rising pressure is translating into growing data center spending, which is predicted to reach $1 trillion in 2029, largely driven by these workloads.

Legacy and Proprietary Systems Stall Modernization

Vendor-controlled databases slow deployment pipelines, restrict architectural flexibility, and inflate Total Cost of Ownership (TCO) as estates grow more fragmented. Teams lose velocity as they navigate outdated scaling constraints, rigid licensing, and operational silos.

The Real Constraint: Proprietary Databases and Vendor Lock-In

Even as financial institutions modernize, proprietary databases remain a source of costly, rigid constraints that hinder real progress. Let’s see how these constraints surface in practice.

Licensing Costs that Rise Faster than Value

Proprietary licensing has quietly become a structural tax on modernization.

• Rising license fees without added capability directly slow innovation.
• Compliance-critical features like encryption, auditing, and high availability (HA) are locked behind premium tiers, driving TCO far above infrastructure costs.

When Redis shifted to a more restrictive license, the market reaction was immediate. Nearly 75% of Redis users began exploring alternatives, and over three-quarters were already testing or migrating to new options.

For financial institutions with regulatory, uptime, and AI requirements, this goes beyond cost and starts to limit long-term decisions.

Features Locked Behind Enterprise Tiers

Compliance and security are typically offered as premium add-ons rather than standard features. Multi-layer encryption, detailed auditing, enterprise HA, and reliable backup may require costly upgrades. Maintaining baseline security and regulatory standards can drive unexpected spending.

In contrast, modern open-source databases come with many of these features built in or easily added, giving institutions more control instead of leaving it with the vendor.

Forced Upgrades and Migration Penalties

Closed licensing gives vendors control over upgrade timing and support windows, not the institutions that run the risk. Forced version jumps and commercial changes rarely align with project roadmaps or risk calendars.

In financial services, where planned change is itself a control, this loss of autonomy introduces unnecessary operational and regulatory exposure.

Limited Portability Across Hybrid and Multi-Cloud

Hybrid and multi-cloud have become standard operating models in finance. But proprietary engines often resist movement across regions, providers, or Kubernetes platforms.

That immobility limits options for latency optimization, data sovereignty, and disaster recovery patterns, exactly where many institutions need the most freedom.

DBaaS Convenience That Turns Into Runaway Spend

A managed proprietary DBaaS looks attractive at first. However, at scale, the economics often flip. Percona’s analysis shows that over 74% of DBaaS users cite high and unpredictable costs as their top challenge, a direct result of opaque, usage-based pricing models.

The institutions encounter uncertain costs on IOPS, storage, backups, cross-region traffic, and autoscaling decisions that they have no full control over.

Financial institutions should not rent out their data infrastructure on conditions that may shift unexpectedly to remain competitive. That need for control, portability, and predictability is what’s driving the industry toward open source.

Why Open Source Has Become the Strategic Advantage in Finance

With the global open source database market projected to reach USD 63.48 billion by 2034, more and more financial institutions are recognizing its strategic value. Open source now gives them the control, transparency, and flexibility that proprietary databases can’t match.

Open Source Database Market Size | Source

Let’s look at why open source now leads in every dimension that matters.

Cost Control with Predictable Economics

The shift to open source begins with transparency. Where proprietary vendors raise licensing fees and monetize basic compliance features, open source removes artificial cost ceilings and aligns spend with real usage. This includes:

• No per-core or edition-based licensing
• No penalties for scaling read replicas or adding environments
• No enterprise tier required for auditing, encryption, or HA

Open source restores financial governance and eliminates unexpected cost shocks as data volumes grow. This gives organizations predictable economics and control over their database spending.

Scalability Without Constraint

Open source scales with demand, not with licensing policy. In high-volume trading or high payment flow periods, capacity can grow vertically or horizontally without precipitating premium editions, replica fees, or per-core uplifts.

Cloud-native automation strengthens elasticity across regions, clouds, and Kubernetes environments. It enables scale-out patterns that follow real workload behavior instead of vendor-driven architecture choices.

This saves the IOPS taxes, network surcharges, and hard-and-fast capacity levels that often bloat proprietary DBaaS prices.

Security and Compliance Through Transparency

Financial institutions need to demonstrate every control they implement. Proprietary databases make this challenging because their security mechanisms cannot be inspected or validated.

Open source removes this barrier by giving teams full visibility into how security controls and safeguards function. That clarity sets the foundation for the capabilities that follow:

• Native encryption at rest, in transit, and at granular data levels
• Robust role-based access control (RBAC) and deep auditing frameworks, such as pgAudit
• Configurable policies mapped to PCI DSS, GDPR, AML/ATF, and ESG standards

As regulatory pressure accelerates, this transparency becomes essential. Institutions strengthen compliance, reduce licensing costs, and avoid security features locked behind enterprise editions.

High Availability and Uptime You Can Design, Not Inherit

Financial institutions cannot tolerate unpredictable failover behavior. Open source gives architects full control over HA/DR strategy, rather than relying on opaque vendor-managed mechanisms. This includes:

• Synchronous and asynchronous replication
• Multi-region and multi–data center architectures
• Automated failover without black-box dependencies
• The ability to tune HA for latency, cost, or regulatory constraints

HA is not a feature to buy but an architecture you can design with open source.

Modernization and Innovation at Enterprise Scale

Modern platforms require modular, API driven, cloud native systems. Open source databases integrate naturally into:

• Cloud-native automation and Kubernetes
• CI/CD pipelines to deliver features faster
• Asynchronous architectures
• Telemetry and ML streams

Proprietary databases slow modernization by restricting portability, enforcing rigid architectures, and complicating automation. Open source removes these barriers and gives teams the freedom to experiment and move quickly.

Talent and Ecosystem Alignment

Open source tooling is widely embraced by developers, SREs, architects, and data engineers, as it aligns seamlessly with how modern teams build, test, and scale software. This includes:

• Larger talent pools for PostgreSQL, MySQL, MongoDB, and cloud-native ecosystems
• Faster internal development cycles
• Higher retention and a more collaborative engineering culture

Open source isn’t simply a cheaper alternative, it is structurally aligned with how modern financial institutions operate. It provides the control, visibility, scalability, and freedom needed to compete in a market of data, AI, compliance, and relentless uptime requirements.

Making Open Source Enterprise-Grade: Where Percona Fits In

Open source provides financial institutions with flexibility and cost control. Percona turns that foundation into an operationally mature, secure, high-availability data platform suitable for regulated, always-on financial workloads.

Unified Multi-Database Expertise

Financial systems commonly use multiple data engines, MySQL for transactions, PostgreSQL for analytics, MongoDB for document storage, and more. Percona supports this diversity through a single engineering organization, offering:

• Full production‑grade support for MySQL, PostgreSQL, and MongoDB via Percona’s distributions.
• Cross‑engine expertise in performance, indexing, replication, and schema strategies.
• An integrable observability layer through Percona Monitoring and Management (PMM), with the ability to monitor and control all your supported engines on a single dashboard.

This unified approach gives financial institutions a consistent, reliable way to operate diverse database environments at scale.

Enterprise-Ready Open Source Software

Percona’s database distributions take community editions and enhance them with features and defaults tuned for enterprise production:

• Percona’s PostgreSQL distribution offers enterprise‑ready security and high availability with no licensing fees or hidden add-ons.
• Performance optimizations and configuration tuning beyond stock community defaults.
• High‑availability automation, backup and recovery workflows (including point-in-time recovery), replication/sharding, and automated cluster scaling.

This means financial teams can run open‑source databases with the predictability, operational rigor, and enterprise-grade features often associated with commercial editions.

24×7×365 Support Built for Critical Financial System

Percona keeps your databases online:

• Operators automate backups, scaling, upgrades, and HA.
• PMM offers a single-source monitoring, alerting, and management of MySQL, PostgreSQL, or MongoDB, on-prem, in the cloud, or both.
• Live tracking and assistance during trading periods, settlement hours, or compliance timelines.

Security and Operational Governance

Percona ensures open source works within your compliance framework:

• Traceability (MongoDB, MySQL) through audit logging.
• Data encryption in transit and at rest.
• Fixed configuration defaults that are consistent with the industry best practices.
• Recommendations on how to combine database security with identity access management (IAM) and security information and event management (SIEM) systems.

Modernization and Vendor-Exit Expertise

Percona helps organizations modernize by migrating from proprietary databases to fully open-source solutions with minimal disruption:

• Complete open-source using MySQL, PostgreSQL, and MongoDB.
• On-prem, cloud, hybrid, and multi-cloud elastic deployments.
• Maintain control over cost, compliance, and performance without lock-in to vendors.

Proof in Action: Financial Institutions Winning With Open Source

Modern financial institutions are already succeeding with open-source data infrastructures built and supported by Percona. Their results show what’s possible when open-source databases are engineered, observed, and operated at enterprise scale.

• Payments platforms: Merchant Warrior uses Percona for critical MySQL availability and resilience, supporting millions of transactions across 30,000+ customers. Percona XtraDB Cluster and PMM enable the team to have real-time insights and enterprise-level performance without downtime.
• Fintech and trading platforms: Fiserv implemented hybrid MongoDB, MySQL, and PostgreSQL clusters using Percona with ultra-low latency and performance of microseconds on key loads. This shows the efficiency of open-source software in helping financial platforms scale and maintain rigorous SLAs.
• Credit unions / digital banks: With Percona Server, BBVA moved over 80 applications and 35TB of MongoDB data. This reduced license fees, improved backup performance by 20%, and gave full control over its enterprise database strategy.
• Critical applications and cost savings: Protectall reduced escalating database costs and improved reliability by migrating from MySQL Enterprise to Percona Server. The team then partnered with Percona again to plan a long-term PostgreSQL migration that supports future growth.

What’s Next: Take Control With Open Source

Open source is now the backbone of modern financial infrastructure. When your organization is struggling with increasing costs, modernization pressure, compliance problems, or vendor lock-in, it is time to take charge.

Upgrade your databases with Percona, achieve enterprise-grade support, predictable costs, and operational control with MySQL, PostgreSQL, and MongoDB.

The future of finance is open source. Discover how institutions are reducing costs, ensuring compliance, and driving innovation.

The post Why Modern Finance Runs on Open Source appeared first on Percona.

The post Why Modern Finance Runs on Open Source appeared first on MariaDB.org.

But the actual cost of a proprietary database is not a line item. The predictability dissolves once workloads grow, new services are launched, or regulatory and uptime demands force you to scale aggressively. The bill shows up later as add-ons, markups, and architectural limitations that quietly shape every strategic decision.

Proprietary databases are rarely evaluated on the total cost of ownership across five to ten years. Instead, they are justified as a necessary platform cost, even as their pricing, restrictions, and lock-in reduce your ability to modernize, migrate to the cloud, or optimize for AI and new products. Actually, you don’t just pay for the database. You pay for the way it constrains your options.

This article explains the Total Cost of Ownership (TCO) of proprietary databases, detailing the architectural, operational, and strategic burdens they impose beyond the initial price.

License fees are just the entry price

Most proprietary databases start with a base license model, per-core, per-processor, per-socket, or tied to specific instance sizes and regions in the cloud.

As soon as you scale out (more nodes, more replicas, more environments) or scale up (more powerful instances), your licensing footprint expands, even if business value grows more slowly than technical capacity.

Tiered pricing forces expensive upgrades for critical features, such as Transparent Data Encryption (TDE), granular PCI-DSS auditing, or active-active High Availability (HA).

Cloud database services add another layer by bundling license, compute, storage, and management into opaque service tiers with markups that can exceed 80–100% of the underlying infrastructure cost.

What looked like simple all-in pricing at proof of concept becomes a permanent premium as you scale across regions, add failover zones, and support 24×7 workloads.

Put simply, the license is just the entry ticket; the real spending kicks in as you grow.

Cost #1: Lock-in limits your options

Lock-in is a financial and strategic cost. Proprietary databases encourage deep reliance on vendor-specific extensions, APIs, drivers, and management tooling that make migrations complex and risky.

Over time, these dependencies become embedded in application logic, operational runbooks, and compliance documentation, and raise the exit cost every year.​

This lock-in becomes visible precisely when you need maximum flexibility:

• Cloud strategy shifts (moving from single-cloud to multi-cloud or hybrid) expose how hard it is to re-platform proprietary workloads. Many financial institutions are pursuing multi-cloud or hybrid strategies to meet regulatory requirements (DORA in Europe).
• Mergers and acquisitions create duplicate database estates that cannot be consolidated easily because of incompatible vendors and licensing models.
• Cost-cutting mandates force leadership to accept high database spend because the transition cost is perceived as even higher.

Going open-source becomes a viable alternative. Open source databases keep leverage on your side by using standard interfaces and portable architectures. So you can adjust infrastructure, cloud providers, and operating models without rewriting your core.

Percona’s multi-vendor expertise across MySQL, PostgreSQL, MongoDB, and others is designed to preserve that flexibility while still delivering enterprise-grade support.

Cost #2: Performance tuning becomes a paid feature

Many proprietary platforms offer advanced observability, query analytics, and tuning tools that are available through separate licenses, packs, or cloud add-ons. And you have to pay extra just to monitor what your database is doing in production, right when performance issues, slowdowns, or outages are already affecting your customers. The paradox is that you end up spending more only after the system has already proven to be inadequate.

That model creates a reactive financial posture. Instead of having 24/7 transparency, organizations find themselves “unlocking” features only after a crisis has already occurred. You end up rewarding the vendor with more revenue because their system proved inadequate for your current workload.

On the other hand, open source tooling such as Percona Monitoring and Management (PMM) provides deep query-level visibility, historical performance data, and health checks as part of an open, community-accessible stack.

You are not charged more for insight, you gain observability by default and can invest in expertise rather than feature unlocks.

Cost #3: Scaling means paying twice

When you scale a proprietary database, you pay twice, once for the underlying infrastructure and again for the licenses tied to that infrastructure. Processor-based licensing or per-core licensing means that adding capacity for peak trading days, new AI scoring models, or additional test environments increases software costs.

This occurs even if revenue per transaction does not grow at the same rate. More cores and nodes do not necessarily represent proportional business value, but they do represent proportional license spend.​

In cloud DBaaS models, markups and tier structure amplify this effect. Typical high-availability PostgreSQL workloads can cost more than 100% more on proprietary DBaaS than on open-source deployments on Kubernetes or self-managed infrastructure.

Efficient architectures, read replicas, multi-region setups, and extra capacity for resilience are effectively punished by higher recurring bills rather than rewarded for robustness.

With Percona’s open source approach, organizations scale for performance and resilience without paying the double tax of proprietary licensing and cloud markups.

Cost #4: Support that’s aligned to the vendor, not you

Vendor support is usually positioned as an insurance policy, but its incentives are aligned with renewals rather than with the fastest or most cost-effective solution for your team.

Support SLAs tend to focus on product availability and incident response boundaries rather than on end-to-end business outcomes or architectural improvements. When the “right” answer is to reduce dependence on premium features, simplify architecture, or migrate off the platform entirely, vendor support has little incentive to champion that path.​

But in contrast, support that is database-agnostic and open source–oriented can recommend changes across engines and environments, even when that reduces your spend with any given vendor.

Percona’s support model is built on cross-engine expertise and outcome-focused services such as performance tuning, HA design, and root-cause analysis, rather than upselling proprietary options or new editions. The goal is to optimize your environment, not your license stack.

Cost #5: Operational inflexibility

Proprietary ecosystems often include tightly coupled tooling such as backup utilities, monitoring dashboards, deployment frameworks, and security controls that work with that vendor’s stack.

And over time, this creates tool sprawl and operational silos. It necessitates separate processes for different proprietary databases, such as one for Oracle and another for SQL Server, and distinct runbooks for each cloud-specific DBaaS.

Teams spend more time working around tool boundaries than improving reliability, which prevents standardized operations across environments. Change management, incident response, and compliance reporting have to accommodate multiple incompatible systems, increasing training overhead and the risk of errors.

With open source and vendor-neutral tooling such as PMM and Kubernetes-based operators, organizations can standardize how they deploy, monitor, and secure databases while preserving the freedom to choose engines and clouds. Operations become unified without forcing a single proprietary vendor everywhere.

The hidden risk: Strategic decisions get harder over time

The considerable cost of proprietary databases is that they can leave companies stuck. The longer they remain on proprietary systems, the harder it becomes to make strategic choices.

• Budget uncertainty: Sudden changes in licensing terms, such as shifting from perpetual to subscription or changing virtualization rules, can blow a hole in a digital transformation budget overnight.
• Innovation blockers: If every new microservice triggers a new enterprise database license discussion, your developers will stop proposing new services. Innovation is stifled by procurement friction.
• Long-term planning: When the database is a black box, you cannot accurately predict how it will behave under the next generation of AI workloads, which leads to a conservative, often slower roadmap execution.

What open source changes (when done right)

Open source changes the cost through removing artificial constraints and making pricing a function of infrastructure and expertise instead of vendor permissions.

Organizations pay for compute, storage, and operational skills to run databases like MySQL, PostgreSQL, or MongoDB, not per-core licenses or premium features. It ensures costs are predictable and scale with actual usage, not licensing thresholds.

Open source databases also maximize:

• Freedom of movement: Move across clouds (AWS, GCP, Azure), environments (VMs, Containers), and distributions without asking for permission or renegotiating a contract.
• Full visibility: Open source gives your SREs and architects the ability to “look under the hood,” leading to faster root-cause analysis and more resilient systems.

But open source alone is not enough. Execution, architecture design, performance tuning, HA/DR planning, and day-two operations determine whether open source remains a cost advantage or becomes a different kind of complexity.

Where Percona fits in

Percona bridges the gap between the freedom of open source and the security requirements of modern finance. Its distributions for MySQL, PostgreSQL, and MongoDB upgrade community editions for production with security, HA integrations, backup/restore, and performance tuning built in. Companies gain top-level capabilities without layered licensing fees or edition gates.​

Percona also provides:

• 24×7 support and consulting across multiple engines and environments (on-prem, cloud, Kubernetes). Focused on minimizing downtime and optimizing TCO rather than expanding proprietary footprint.
• Open observability through PMM to give teams a single place to monitor and manage all supported databases with full transparency and no per-node monitoring tax.​

Simply put, Percona reduces long-term database spend by aligning services with customer outcomes, availability, performance, and cost control.

Conclusion: Pay for control, not constraints

Proprietary databases promise certainty but can create dependence. They offer predictable costs in the short term but often lead to long-term limitations and increasing markups. Open source databases, when used and managed properly, shift the focus from restrictions to options by letting you use your budget to buy infrastructure, skills, and flexibility instead of just permission to grow.

The important question is not “How much does this license cost this year?” but rather “What will it cost us to remain flexible over the next five years?”

With open source databases supported by Percona, financial institutions can answer that question with confidence since they are paying for control and options, not restrictions.

The post What You’re Really Paying for With Proprietary Databases appeared first on Percona.

The post What You’re Really Paying for With Proprietary Databases appeared first on MariaDB.org.

Continue reading “MariaDB Hidden Gem: Create Aggregate Function”

The post MariaDB Hidden Gem: Create Aggregate Function appeared first on MariaDB.org.

The post MariaDB Hidden Gem: Create Aggregate Function appeared first on MariaDB.org.

Continue reading “Celebrating the MariaDB Foundation Sea Lions Champions Nominees”

The post Celebrating the MariaDB Foundation Sea Lions Champions Nominees appeared first on MariaDB.org.

The post Celebrating the MariaDB Foundation Sea Lions Champions Nominees appeared first on MariaDB.org.

Continue reading “MariaDB Foundation: Bringing TPC-B Back To Life”

The post MariaDB Foundation: Bringing TPC-B Back To Life appeared first on MariaDB.org.

The post MariaDB Foundation: Bringing TPC-B Back To Life appeared first on MariaDB.org.

Continue reading “MariaDB Community Server Corrective Releases”

The post MariaDB Community Server Corrective Releases appeared first on MariaDB.org.

The post MariaDB Community Server Corrective Releases appeared first on MariaDB.org.

Source

The post MariaDB Enterprise Server Q1 2026 Corrective Releases appeared first on MariaDB.org.

There’s a new place for the people behind the databases to actually talk to each other.

The Percona Community Slack is open. Right now it’s one channel — General — and that’s intentional. It’s a place for DBAs, developers, contributors, and database people of all kinds to meet, swap stories, and get to know who else is out there running open source databases for a living. No silos. No sub-channels for every topic. Just a room.

What it’s for

Come here to talk shop. Share what you’re building, breaking, or fixing. Post about the migration that went sideways, the config that finally clicked, the pager incident you survived. Ask the kind of questions that belong in a conversation rather than a ticket — “how do other people handle X?” is exactly the right energy.

It’s also where we’ll share events we’re attending and, when we have tickets or a spare seat, offer them to the community first. If Percona is heading to a conference near you, this is where you’ll hear about it. And if you’re going somewhere yourself — a meetup, a conference, a local user group — tell us. There might be community members nearby who want to meet up.

That’s the point, really. Less broadcast, more conversation.

What it’s not for

Technical support questions belong on the Percona Community Forums. Forum answers are searchable and don’t disappear into scrollback. Percona engineers and experienced community members watch the forums for questions. Your problem is more likely to get a useful answer there — and it’ll help the person who hits the same issue three months from now.

If you post a support question in Slack, expect to be pointed to the forums. That’s not a brush-off.

A few things that make this work

Introduce yourself. One or two sentences about what you work on and where in the world you are. That’s it. You don’t need a bio.

Share what you’re up to. An event you’re going to, a tool you’ve been testing, a war story from production. The low-key post about a thing you just dealt with is exactly what people come here for.

Lurk freely. You don’t have to post to belong. Read, learn, jump in when you have something to say.

The short version of the rules

Be the person you’d want to share an on-call rotation with.

Treat everyone as a peer. Assume good faith. No harassment. Critique technology on technical merits. Don’t cold-DM people with pitches. Keep private things private. If something needs a moderator’s attention, DM one directly — reports stay confidential.

Come in

If you’re a DBA, a developer, a contributor, or just someone who runs databases and occasionally wants to talk to other people who run databases — you belong here.

Join the Percona Community Slack →

The post The Percona Community Slack is open — come hang out appeared first on MariaDB.org.

Continue reading “A New Pull Request Processing Time Record”

The post A New Pull Request Processing Time Record appeared first on MariaDB.org.

The post A New Pull Request Processing Time Record appeared first on MariaDB.org.

Part 1: why semantic search, what’s already working on the site, the widget, and an overview of the stack.

Architecture

Search runs separately from the website at search.percona.community: FastAPI, a background indexer, and PostgreSQL with pgvector are all in a single Docker Compose file. percona.community remains static on Hugo and GitHub Pages, it doesn’t write directly to the database.

flowchart TB
subgraph users["Users"]
direction TB
User(["User"]) --> Widget["Widget · percona.community"]
Admin(["Admin"]) --> Dash["Admin dashboard · /demo"]
Site["Site · RSS + HTML"]
end
subgraph app["Application"]
direction TB
API["FastAPI · search.percona.community"]
Model["nomic-embed-text-v1"]
Worker["Indexer worker"]
API -->|embed query| Model
Model -->|embedding| API
Worker -->|embed chunks| Model
Model -->|embeddings| Worker
end
subgraph data["Database"]
direction TB
DB[("PostgreSQL + pgvector")]
end
Widget --> API
Dash --> API
Site --> Worker
API -->|read vectors · write queue/history| DB
Worker -->|write vectors · read queue| DB

Search

A visitor enters a query into the widget. The widget sends a POST /search request to FastAPI. The service computes the query embedding with nomic with the prefix search_query: and searches for the nearest vectors in Postgres. The widget knows nothing about pgvector, it only receives JSON with links.

Admin dashboard

On the same FastAPI service I run an admin dashboard at /demo: test queries, search history, a database summary, viewing documents and chunks. The dashboard does not talk to Postgres directly, it only calls the API; the API reads and writes Postgres (search_history, index_queue, search results).

Indexing

To refresh the index, I click Start Indexing in the dashboard, that hits POST /index/start. The same endpoint can be called from outside: a GitHub webhook after a push to the site repo, cron, or curl while debugging. FastAPI enqueues the job in index_queue. A worker in the indexer container picks it up, downloads RSS and HTML from the site, splits text into chunks, computes vectors with nomic (search_document:), and writes to pages, community_nomic, and indexer_runs. The crawl runs in the background and does not block HTTP.

Important limitation: the indexer and the API must use the same embedding model. The query vector and the vectors in the database must be from the same space, otherwise, cosine similarity doesn’t make sense.

pgvector in Postgres

For semantic search, you don’t need an LLM, but an embedding model: a string as input, a vector as output. I chose nomic-embed-text-v1, 768-dimensional, running via sentence-transformers on the CPU, without a paid API.

I’m using Percona Distribution for PostgreSQL 18, pgvector is already included in the distribution; CREATE EXTENSION vector, and you’re done (documentation).

The basic structure is a column with fixed dimensions for the model:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
 id SERIAL PRIMARY KEY,
 chunk_text TEXT,
 embedding vector(768) -- exactly 768 under nomic
);

In the project, the table is named community_nomic: the prefix community_ (site) + the model key nomic. I’m setting up a comparison of embedding models: each model has its own vector table (community_), because the dimensions and embedding spaces are different, so they can’t be mixed in a single table. Currently, there is one model in the project, nomic-embed-text-v1, 768 dimensions; later, I can add a second table community_ and switch the index/API via EMBEDDING_MODEL_KEY.

pgvector compares vectors with several distance operators. I search with cosine distance (the operator in SQL): the smaller the distance, the closer the match. In the widget and API I show similarity, not the raw distance, similarity = 1 - distance, so a higher score means a better hit. The operators:

Simplified search for “nearest chunks”:

SELECT slug, chunk_text,
 1 - (embedding  $query_vector) AS score
FROM community_nomic
ORDER BY embedding  $query_vector
LIMIT 20;

The threshold in the API is min_score (my default is 0.52): anything lower is discarded. On beta I tuned this number for a while, the results changed noticeably depending on this single parameter.

To avoid scanning the entire table as the index grows, I set up an HNSW index (approximate nearest neighbor search):

CREATE INDEX ON community_nomic
 USING hnsw (embedding vector_cosine_ops);

At this scale, a separate vector database wasn’t necessary, a single Postgres instance handles metadata, vectors, and search.

Postgres in Docker: docker-compose

I set up the stack using Docker Compose, Postgres, the API, and the indexer are all in containers, with the same setup locally and in production. Production, EC2 on AWS (search.percona.community), an ARM instance, using the same docker-compose.

In docker-compose.yml, Postgres on Percona looks like this (on Mac ARM):

postgres:
 image: percona/percona-distribution-postgresql:18.1-3-arm64
 environment:
 POSTGRES_USER: postgres
 POSTGRES_PASSWORD: postgres
 POSTGRES_DB: community_search
 ports:
 - "5433:5432"
 volumes:
 - pgdata:/var/lib/postgresql/data
 - ./init:/docker-entrypoint-initdb.d

In init/01-enable-pgvector.sql, include only CREATE EXTENSION IF NOT EXISTS vector. If you’re developing on x86, use amd64 in the image tag instead of arm64, see the options in the Percona documentation. I left arm64 on both Mac and EC2: the configuration is the same.

I view the tables and data in pgAdmin. The pages, community_nomic, and service tables themselves are created when the API and indexer start using ensure_* functions in the code: these are CREATE TABLE IF NOT EXISTS and CREATE INDEX IF NOT EXISTS, not a separate migration directory.

Indexing and Chunking

The site doesn’t write directly to the database, the database is populated by an indexer: a worker fetches RSS and HTML from percona.community, splits the text into chunks, computes embeddings, and writes the rows to community_nomic and pages. The widget and API only read what has already been written during searches.

Why Chunks

At first, I tried a single vector for the entire article. I quickly ran into three problems:

• Long text takes longer to encode and consumes more memory;
• The model has an input length limit;
• a single vector for long text blurs the meaning, a query about a specific paragraph doesn’t map well to the “averaged” embedding of the entire article.

I settled on a 400-word window with a 50-word overlap (chunker.py). Each chunk is a separate line with its own embedding.

The first version of the chunker sliced only the body of the article, without the title, author, date, or tags. For queries like “articles by a certain author,” the results were off: the model saw the text but not the document’s context. I added metadata to each chunk, a Title / Author / Date / Tags / Type block at the beginning of each fragment before calculating the vector.

When searching, the API finds the closest chunks, but the card shows the best chunk for the document (one slug, one card). Without this, a long article would clutter the results with multiple lines.

Database Schema: Two Revisions of the Chunk Tables

I revised the chunk storage schema twice, and separately added utility tables for background indexing and search logs.

Version 1: Everything in a Single Table

The first working schema was a single table for all chunks and document information: each row represented a single article fragment, and it also contained duplicated page metadata (url, title, author, date, tags, content_type) along with chunk_text and embedding.

Pros: one INSERT, one SELECT, no joins.

Cons I encountered:

• one article, dozens of identical copies of title and author;
• when updating a page, it’s easy to get out of sync (one title in chunk #0, another in chunk #3);
• fetching the image and description for the card from chunk_text was unreliable.

Conclusion: A vector layer and a card in the UI serve different purposes.

The code still includes _migrate_chunks_table: when the API and indexer start up (inside ensure_content_table), it drops any extra columns from the chunk table if they are left over from the old prototype.

Version 2: pages + community_nomic

I split the data into two tables:

• pages, one row per document: url, title, type, author, date, tags, images, description.
• community_nomic, only chunks: slug, chunk_index, chunk_text, embedding.

They are linked by slug (stable key from the URL). Search: find the nearest chunks in community_nomic, assemble the card from pages.

In the admin dashboard I can open any indexed document and see what landed in pages (metadata, image, description) and what text was split into chunks.

HNSW on community_nomic:

CREATE INDEX ... ON community_nomic
 USING hnsw (embedding vector_cosine_ops);

That’s why, when switching models, I don’t reuse community_nomic; instead, I create a new table and re-index it. A single search query involves vectors from only one model, both during indexing and in the API.

Indexer: RSS, HTTP, and Queue

The indexer is a separate container that crawls percona.community and populates the database. It starts with RSS, four feeds:

• blog/index.xml
• events/index.xml
• talks/index.xml
• contributors/index.xml

RSS feeds contain a title, link, date, author, tags, and often a short description, but not the full text of the article. For each entry, I perform an HTTP GET on the HTML page and extract the main content (in crawler.py). If the HTML is empty, I fall back to the description from the RSS feed.

The Index and Status tabs in the dashboard, without them, debugging the crawl and embedding would have been a guessing game.

Table Schema

All tables are created when the API and indexer start (ensure_* in code, CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS). There is no separate migrations folder. I don’t use foreign keys between search and utility tables: reindexing deletes and re-inserts rows by slug, and the queue tables are only loosely linked.

Search data

pages and community_nomic are linked by slug (no FK). The indexer writes both; the API reads them on POST /search.

pages

One row per document.

community_nomic

Chunks and vectors (table name = site + model key).

Utility

Three small tables for indexing and debugging.

index_queue

Pending jobs. Written by the API.

indexer_runs

Crawl progress. Written by the indexer worker.

search_history

Search log. Written by the API on each POST /search.

Indexes

Created in the same ensure_* functions as the tables. Besides primary keys and UNIQUE on pages.url and (slug, chunk_index) in community_nomic:

• pages_content_type_idx on content_type, filter by blog / event / talk / contributor in search;
• community_nomic_embedding_idx, HNSW on embedding (vector_cosine_ops); without it, nearest-neighbor search would scan the whole table as chunks grow;
• community_nomic_slug_idx on slug, delete all chunks for one document on reindex;
• search_history_created_at_idx, recent queries first in the dashboard History tab.

index_queue and indexer_runs only have a serial primary key, few rows, a full scan is fine.

How Postgres Responds to a Search Query

The API receives the query text, computes a vector with nomic (search_query: + text), and runs SQL that finds the nearest chunks and joins row metadata from pages.

The First Query Was Naive

At first, I did what the pgvector tutorials suggest, “find the 20 closest vectors”:

SELECT slug, chunk_index, chunk_text,
 1 - (embedding  $query_vector) AS score
FROM community_nomic
ORDER BY embedding  $query_vector
LIMIT 20;

The query worked, but the results were incorrect from a UI perspective. It returns 20 chunks, not 20 documents. A long article with fifteen chunks could take up half the list with a single slug; a short post with one good paragraph didn’t make it to the top. The user sees pages (cards with links), but we search the database by chunks, I close that gap in SQL.

What I do now

1. Join community_nomic + pages by slug.
2. ROW_NUMBER() PARTITION BY slug, I keep one best chunk per document.
3. WHERE score >= min_score (default 0.52).
4. ORDER BY score DESC LIMIT N.

Simplified version of the final query:

WITH ranked AS (
 SELECT
 p.url, p.title, p.content_type, c.slug,
 1 - (c.embedding  $query_vector) AS score,
 ROW_NUMBER() OVER (
 PARTITION BY c.slug
 ORDER BY c.embedding  $query_vector
 ) AS rn
 FROM community_nomic c
 INNER JOIN pages p ON p.slug = c.slug
 -- AND p.content_type = 'blog' -- optional: filter by type
),
best_per_page AS (
 SELECT * FROM ranked WHERE rn = 1
)
SELECT url, title, content_type, slug, score
FROM best_per_page
WHERE score >= 0.52
ORDER BY score DESC
LIMIT 20;

Filtering by content type

The site has blog, event, talk, and contributor, in the widget and on /search/, you can search for all at once or a single type. In the API, this is the content_type field in POST /search; in SQL, AND p.content_type = %s is added when a single type is selected.

Sort order in the widget

In SQL, results are ranked by similarity (ORDER BY score DESC), “what matches the query best?”

On a community site, recent material often matters as much as the top semantic match. An older article might score 0.71 while a newer post on the same topic scores 0.66. I still build the shortlist in SQL (one best chunk per document, min_score threshold), but the API then re-sorts blog, event, and talk by publication date, newest first. Contributors and rows without a date stay at the bottom.

The widget still shows the similarity score on each card so you can see why the page was included:

Summary

Postgres layer: I set this up without a separate vector DB, using pgvector in Percona, two table modifications for chunking, auxiliary tables for background indexing, HNSW, and SQL with “best-fit chunk per document.” The indexer processes RSS and HTML; I manage the database in pgAdmin.

Currently, the search index has about 803 documents and 1,656 vectors, thousands of rows, not billions. This is a community-scale setup: a single Postgres instance on EC2, embedding on the CPU, HNSW on all chunks, the solutions above were chosen with this in mind. When I add videos, GitHub issues, and the forum, the volume will grow, then I’ll re-evaluate the indexing time and hardware.

Note from the author

About six months ago, I already tried to set up something similar to Postgres + vectors using AI agents. Back then, I kept running into the same issues: a clunky startup of the environment, the schema and its modifications, initializing Percona Distribution for PostgreSQL, and pgvector, the agent would either skip a step or suggest incompatible configuration snippets.

This time, with percona.community, went better: the agent set up Compose, ensure_*, search SQL, and the admin dashboard, without that series of failures at startup. More time was spent on the logic (chunks, min_score, result ordering) rather than on “why the database won’t start.”

If you try this setup yourself or notice any inaccuracies, please leave a comment.

The post Building Smart Semantic Search using PostgreSQL and pgvector. Case Study – Part 2 – Postgres Layer appeared first on MariaDB.org.

Continue reading “MariaDB Server 12.3 LTS Released”

The post MariaDB Server 12.3 LTS Released appeared first on MariaDB.org.

The post MariaDB Server 12.3 LTS Released appeared first on MariaDB.org.

I’ll walk through how and why I built the search for our community site: blog, events, talks, and profiles. The post should help if you want to repeat the approach or need a practical case study on simple components. If you’ve done something similar, I’d like to hear your feedback.

Context: Website, Search, and Task

The community team has a website on Hugo, an open source static site generator, hosted for free on GitHub Pages. The site has articles, events, talks, videos, and more.

If you’re thinking of starting your own, I recommend checking out these examples: blog.koehntopp.info, openeverest.io, perconalive.com, oursqlfoundation.org

But a Hugo site is a collection of HTML files without a backend. Search or filters only work via frontend JS or an external service. For a long time our site had no search at all. Then Kai Wagner contributed a JS search for the blog that matched exact words (percona.community/blog).

Recently our community lead Laura Czajkowski asked for smart AI search on the site. We tried several off-the-shelf products; they were either too expensive or a poor fit. We also want search to cover more than the site itself eventually: videos from other platforms, the forum, our GitHub repos, and maybe documentation later.

I suggested building it ourselves. Modern AI assistants are good enough for a prototype like this. Below I’ll explain the stack.

What We’ll Do

The site stays on Hugo and GitHub Pages. The search service runs separately; for this architecture that’s the sensible option. The goal is simple: the user types a query in plain language and gets a list of semantically relevant links.

Kai’s keyword search was a step forward, but it doesn’t catch meaning. Type “postgresql” and you get pages where the word appears. An article about slow queries or replication may be missing if the wording is different. Semantic search works differently: the query and documents become vectors, numeric representations of meaning (embedding). Similar meaning lands nearby in vector space even when the words differ. A query like “how to speed up slow queries in MySQL” can surface tuning and optimization content without those words in the title.

Why not another engine? OpenSearch is a solid open-source option: full-text and vector search, mature ecosystem. I also looked at Manticore Search. Both work, but semantics still need an embedding pipeline (model at index time and on each query). That’s another service to run beside the model.

I wanted my own stack on Postgres with pgvector: a practical experiment, not a hunt for the perfect search product. PostgreSQL with pgvector keeps page metadata, chunks, vectors, and query history in one database. Percona Distribution for PostgreSQL 18 ships pgvector in the distribution; run CREATE EXTENSION vector and you’re set.

The plan has four parts:

1. Widget on the site: search field and results (plain JS; Hugo unchanged).
2. API: takes the query, embeds it with the same model as indexing, searches the DB, returns JSON links.
3. Indexer: background worker that reads RSS/HTML, chunks text, embeds, writes to the DB.
4. PostgreSQL + pgvector: one database for metadata, chunks, vectors, and search history.

Hugo stays static; the smart parts live in a separate service. No separate vector DB, no paid embedding API, no RAG chat, only links.

The diagram shows two flows: search (user query) and indexing (refresh the DB on demand or on a schedule). Top to bottom, from the user:

flowchart TB
User(["👤 User"])
Widget["🔍 JS widget
percona.community · GitHub Pages"]
API["⚡ FastAPI
search.percona.community"]
Model["🧠 Embedding model
shared · API & indexer"]
DB[("🗄️ PostgreSQL + pgvector")]
Content["📰 Content
blog · events · talks"]
Indexer["📥 Indexer worker"]
User -->|"① query"| Widget
Widget -->|"② POST /search"| API
API |embed query| Model
API |"③ vector search"| DB
API -->|"④ results"| Widget
Widget --> User
Content -->|"A. RSS + HTML"| Indexer
Indexer |embed chunks| Model
Indexer -->|"B. chunks + vectors"| DB
style User fill:#e1f5ff
style Widget fill:#fff4e6
style Content fill:#fff9e6
style API fill:#ffe6e6
style Model fill:#fff0f5
style Indexer fill:#f0e6ff
style DB fill:#e6ffe6

The diagram shows the shared embedding model; worth stating explicitly anyway. The indexer and the API must use the same model. Query vectors and stored vectors must share one space or search is meaningless. Don’t mix Nomic at index time with OpenAI at query time, for example. The widget only sends text; it doesn’t know which model runs behind the API.

On paper it looked simple. In practice I changed the database schema three times and tuned ranking so blog posts didn’t crowd out events and talks. The similarity threshold mattered more than I expected: one parameter, large swing in results. Still, within a few days we had a working beta on the live site. Here’s what shipped.

The Result (Spoiler)

It took about three unhurried days and roughly $20 in Cursor tokens to build, debug, and deploy. Try it on percona.community (search icon in the header) or percona.community/search/.

The index currently covers the site: blog, events, talks, member profiles. Video from other platforms, the forum, and GitHub are planned; the design should allow new sources without replacing the stack.

This is beta: the content is public and search isn’t business-critical, but I watch stability and security.

Website Widget

The header has a search icon. Click it to get an input field and a popup with results, similarity score (0 to 1, how close the hit is in meaning), and API latency. The site stays static; the widget calls search.percona.community and renders JSON. “All results” opens /search/.

Try it on percona.community, e.g. slow queries mysql tuning or kubernetes operator database. Comments welcome if something feels off.

Full Results Page

A separate /search/ page with filters by content type, cards, and links.

Example

API

FastAPI at https://search.percona.community: embed the query, search Postgres, return JSON with links, scores, and timings (model vs database).

The service runs on AWS EC2 in Docker Compose: API, indexer, Postgres.

Demo Dashboard

The Cursor AI agent handled a lot of the boilerplate, so I also built a dev dashboard (/demo) to test search, run indexing, inspect history, and browse indexed chunks. Not for production, but it saved debugging time.

Demo Dashboard

Search history: making search better

Indexing status, to see when search data was last updated

Indexed documents with the ability to view data and chunks.

What I Used

Briefly, why this stack (deeper comparison in part two):

• PostgreSQL + pgvector: vectors and metadata in one DB. Cosine similarity plus an HNSW index is enough at community scale. (pgvector in Percona docs)

• Percona Distribution for PostgreSQL 18: PostgreSQL with pgvector and a Docker image. Vanilla Postgres works too if you install the extension; I used Percona to try “their” Postgres + pgvector in a real deploy.

• Python + FastAPI: fast API setup, OpenAPI included, good libraries for crawl/embed/Postgres.

• nomic-embed-text-v1 + sentence-transformers: open model, 768 dims, CPU-friendly, no per-chunk API bill. Index and query must use the same model; Nomic fits. I’ll compare others later.

• Hugo + JavaScript: thin widget on existing static site.

• Docker / Docker Compose: same layout locally and on EC2.

• AWS EC2 + nginx: HTTPS on search.percona.community, CORS for GitHub Pages.

• Cursor: main dev tool; its AI agent helped with boilerplate, wiring API to the demo, and Docker fixes. I still reviewed everything. Without it, the same work would have taken weeks.

How long it took

• ~6 hours with Cursor to a first prototype: crawl, API, Docker, basic demo;
• ~2 more days for schema changes, per-type ranking, embed/page widget, search history, dashboard, indexer fixes, EC2 deploy;
• ~$20 in Cursor tokens total.

Without AI I’d have stretched the same work over weeks. With the agent I mostly wrote tasks, checked output, and fixed edges.

About the code and repository

I’m not publishing the repo yet. The code is tied to percona.community: our RSS feeds, content types, Hugo widget, EC2 layout. It’s an internal prototype, not a reusable library.

If you wanted a drop-in repo: porting someone else’s monolith often takes longer than rebuilding from a clear sketch. Part two will have architecture, schema, and stack notes enough for a Cursor agent (or similar) to rebuild for your feeds and UI.

Interested in a generic open source or search-as-a-service version? Say so in the comments; I’m weighing whether it’s worth a separate project.

What’s Next

Try search on percona.community and comment what you find, especially where semantics beat the old substring search.

Part two will go inside: schema (including those three rewrites), chunking, HNSW, per-type result caps, and a local Docker Compose walkthrough.

The post Building Smart Semantic Search using PostgreSQL and pgvector. Case Study – Part 1 appeared first on MariaDB.org.

Source

The post MariaDB Community Server 12.3 LTS: How It Scales AI Workloads and Delivers 4x Write Performance appeared first on MariaDB.org.

The Percona Operator for PostgreSQL 3.0.0 is here. This is the release that completes the hard fork of the operator from the Crunchy Data PostgreSQL Operator into a fully independent project, with a dedicated upstream.pgv2.percona.com API group for the inherited CRDs, an automatic CRD-rename rollout for existing 2.x installs on upgrade, and a public roadmap that drives what comes next.

This release ships three headline changes that matter for production teams. The CRD renaming under a Percona-owned API group, which finally lets the Crunchy operator and the Percona operator coexist in the same Kubernetes cluster. Proper OLM namespace scoping for OpenShift installations. And the move to the official Percona Distribution image for major PostgreSQL version upgrades, aligning the upgrade path with the same binaries that run in your clusters.

All three land in service of the same goal: making 3.0.0 a clean, durable operational baseline for the operator’s next several years as an independent project. Future releases will be shaped by what the community asks for and contributes back. The public roadmap is the durable signal of that commitment.

In this post, you will learn about:

• The hard fork and how the CRD rename unlocks coexistence with the Crunchy operator
• OLM namespace-scoping improvements for OpenShift installations
• The move to the official Percona Distribution image for major PostgreSQL version upgrades
• Other improvements and the 2.7.0 deprecation
• Supported PostgreSQL versions and platforms

Hard fork: CRDs renamed under upstream.pgv2.percona.com

The Percona Operator for PostgreSQL has, until now, been a soft fork. Custom Resources inherited from Crunchy PGO used the upstream postgres-operator.crunchydata.com API group. The two operators shared CRDs, which meant you could only run one of them in a given Kubernetes cluster. Installing both would lead to overlapping CRDs, conflicting webhooks, and finalizer collisions, so platform teams had to pick a side before they had finished evaluating.

Starting with 3.0.0, every inherited CRD is renamed into a new dedicated upstream.pgv2.percona.com API group (K8SPG-1007). Percona’s own native CRDs (such as PerconaPGCluster under pgv2.percona.com/v2) are unchanged. The change applies to the inherited resources: PostgresCluster, PGUpgrade, PGAdmin, and the rest.

Coexistence: running both operators in the same cluster

The practical effect is that the Crunchy Data PostgreSQL Operator and the Percona Operator for PostgreSQL can now run on the same Kubernetes cluster at the same time, even in the same namespaces, with no CRD or webhook conflict. That unlocks a few real workflows: evaluating both operators on the same staging cluster without spinning up a second cluster, running existing Crunchy-managed clusters in some namespaces while bringing up new Percona-managed clusters in others, or testing a new database version on the Percona side while production stays on Crunchy until you are confident. The choice between the two operators stops being all-or-nothing.

Upgrade behavior for existing 2.x installs

For an existing install, the upgrade to 3.0.0 is mechanically simple. The operator creates the new-API-group CRDs alongside the legacy ones, then runs a one-time migration that updates dependent objects (Secrets, certificates, finalizer references) to point at the new CRD instances. Existing custom resources keep working through the legacy CRDs during the transition, and once migration completes, all reconciliation moves to the new group.

Old PostgresCluster reference:

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata:
  name: cluster1

New (after upgrade to 3.0.0):

apiVersion: upstream.pgv2.percona.com/v1beta1
kind: PostgresCluster
metadata:
  name: cluster1

Day-to-day, your PerconaPGCluster Custom Resource (the one most teams interact with directly) is unchanged. The rename mostly matters in three situations: when a kubectl filter or a GitOps repository hard-codes the old API group, when a CI pipeline references the legacy CRD by name, and when you run the Percona and Crunchy operators side by side and need them not to collide.

Note: During the CRD migration on upgrade, the release notes report brief disruptions to pgBackRest operations (typically 1 to 2 minutes) while Kubernetes propagates certificate changes. Plan the upgrade during a maintenance window if backup continuity is critical, or pause scheduled backups during the upgrade.

Full details on the API-group change are in the Percona PostgreSQL operator documentation.

Improved OLM namespace scoping for OpenShift

OpenShift users install operators through the OpenShift Lifecycle Manager (OLM), and OLM enforces an OperatorGroup to scope which namespaces an operator watches. In practice, 2.x had quirks: teams that selected “Single namespace” mode would sometimes see the operator reconciling CRs in other namespaces, and teams in “All namespaces” mode would sometimes see incomplete coverage when CRs were created in newly-added namespaces.

3.0.0 fixes this by aligning the operator’s namespace watch list with the OperatorGroup that OLM applies. All-namespaces installs watch all namespaces. Single-namespace installs respect the targetNamespaces set on the OperatorGroup.

Why it matters in shared infrastructure

For an OpenShift platform team running shared infrastructure, this distinction matters operationally. A typical setup has the database operator installed once in a platform namespace (such as openshift-operators) but expected to serve PerconaPGCluster resources owned by individual application teams in their own namespaces. If the operator over-reaches into namespaces it should not watch, RBAC noise multiplies. If it under-reaches, application teams file tickets about clusters that never reconcile. The 3.0.0 alignment with OperatorGroup semantics removes both failure modes.

OperatorGroup wiring

For users installing through OLM via the OpenShift web console, the install flow is unchanged. The fix is in how the operator’s reconciler interprets the OLM-supplied namespace scope after install. For users who manage OperatorGroups directly, a single-namespace install looks like this:

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: percona-pg-operator-group
  namespace: postgres-prod
spec:
  targetNamespaces:
    - postgres-prod

And an all-namespaces install:

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: percona-pg-operator-group
  namespace: openshift-operators
spec: {}

The empty spec: {} (or an OperatorGroup with no targetNamespaces) means “watch all namespaces” by OLM convention. The 3.0.0 operator now honors that.

Note: After you upgrade an existing 2.x install to 3.0.0, the operator may begin reconciling PerconaPGCluster resources in namespaces it had previously ignored due to the prior scoping bug. Audit existing CRs across your cluster before upgrading, especially if you have stale test clusters in unintended namespaces. The release notes call this out explicitly.

Note for community vs certified bundle users: Community OLM bundles did not support cluster-wide (all-namespaces) mode in earlier versions, 3.0.0 adds it. Certified bundles already supported cluster-wide mode, but they used a separate stable-cw channel for it with 3.0.0 the channels are unified, so users upgrading from a certified stable-cw install need to switch their subscription channel to stable to receive the upgrade.

For the full install workflow on OpenShift, see the OpenShift installation documentation.

Major PostgreSQL version upgrades now use the official Percona Distribution image

Major-version upgrades (for example, PostgreSQL 17 to 18) require running pg_upgrade, which needs binaries for both the source and target versions in the same environment. The operator has supported major-version upgrades since 2.x, but it shipped its own dedicated upgrade image to do so. That worked, but it meant a Percona-specific image lived in the upgrade path, separate from the same Percona Distribution for PostgreSQL build that runs in your clusters.

Switching to the official Percona Distribution image

In 3.0.0, the operator switches to using the official Percona Distribution for PostgreSQL image for major-version upgrades: percona/percona-distribution-postgresql-upgrade (current tag: 18.4-17.10-16.14-15.18-14.23-1, which encodes the bundled major versions). The benefit is alignment: the binaries that run pg_upgrade are the same binaries that ship in the corresponding percona-distribution-postgresql image you already run in production, built from the same source, signed the same way, and patched on the same schedule. The operator orchestrates the upgrade through the PerconaPGUpgrade Custom Resource that names the source and target versions, the upgrade image, and the target component images (PostgreSQL, pgBouncer, pgBackRest).

Running an upgrade through the PerconaPGUpgrade CR

A PostgreSQL 17 to 18 upgrade looks like this:

apiVersion: pgv2.percona.com/v2
kind: PerconaPGUpgrade
metadata:
  name: cluster1-17-to-18
spec:
  postgresClusterName: cluster1
  image: docker.io/percona/percona-distribution-postgresql-upgrade:18.4-17.10-16.14-15.18-14.23-1
  fromPostgresVersion: 17
  toPostgresVersion: 18
  toPostgresImage: docker.io/percona/percona-distribution-postgresql:18.4-1
  toPgBouncerImage: docker.io/percona/percona-pgbouncer:1.25.2-1
  toPgBackRestImage: docker.io/percona/percona-pgbackrest:2.58.0-2

Apply it with kubectl apply -f upgrade.yaml -n <namespace>. The operator reconciles the upgrade as a controlled, observable process: it brings the cluster down for the upgrade window, runs pg_upgrade from the bundled image, brings the cluster back up on the target version, and updates pgBouncer and pgBackRest images in the same step.

Operationally, this matters for teams running on PostgreSQL’s annual major-version cadence. Every September brings a new major release; staying on a supported version means executing one major upgrade per cluster per year. Pulling the upgrade image from the same percona-distribution-postgresql registry path as the runtime image means image-signature verification, mirror-to-private-registry rules, and CVE-scanning policies you already have in place apply to the upgrade flow without any per-image exception.

Note: The pgaudit extension is not upgraded automatically. After the operator completes the major version upgrade, drop and recreate pgaudit manually in each database that uses it: DROP EXTENSION pgaudit; followed by CREATE EXTENSION pgaudit;. The release notes call this out as a required step (K8SPG-1022). Also worth scanning for collation-dependent indexes after the upgrade and refreshing collation metadata with ALTER DATABASE <name> REFRESH COLLATION VERSION; per the upstream PostgreSQL 18 release notes.

Full procedure, prerequisites, and rollback notes are in the major version upgrade documentation.

Other Improvements

Operational polish landed alongside the headline changes:

• Go 1.26 update (K8SPG-1019): the operator binary is now built with Go 1.26, picking up performance optimizations, tooling improvements, and the security fixes that landed in the Go runtime since the previous release.
• pgaudit upgrade documentation (K8SPG-1022): the major-version upgrade docs now include an explicit pgaudit drop-and-recreate procedure, surfacing the gotcha that previously caught users mid-upgrade.

The release also defaults the cluster-upgrade documentation to PostgreSQL 18 across all examples and tutorials.

Supported software and platforms

The Percona Operator for PostgreSQL 3.0.0 is developed and tested on:

• PostgreSQL: 14.23-1, 15.18-1, 16.14-1, 17.10-1, 18.4-1 
• pgBackRest: 2.58.0-2
• pgBouncer: 1.25.2-1
• Patroni: 4.1.3
• PostGIS: 3.5.6
• PMM Client: 2.44.1-1 and 3.7.1

Supported Kubernetes platforms:

• Google Kubernetes Engine (GKE) 1.33 to 1.35
• Amazon Elastic Kubernetes Service (EKS) 1.33 to 1.35
• OpenShift 4.18 to 4.21
• Azure Kubernetes Service (AKS) 1.33 to 1.35
• Minikube 1.38.1 (Kubernetes v1.35.1) for local development

Deprecation: 2.7.0 support dropped

Support for Custom Resource Definitions from operator version 2.7.0 has been removed. If you are still on 2.7.0, upgrade to 2.8.x or 2.9.x first, then upgrade to 3.0.0. The CRD migration described above only handles 2.8.x and 2.9.x to 3.0.0 transitions cleanly.

Conclusion

3.0.0 is the release where the Percona Operator for PostgreSQL becomes a fully independent project. The CRD rename removes the last upstream coupling that mattered operationally. The OLM scoping fix removes a long-standing OpenShift quirk. The official major-version upgrade image removes one of the more painful operational gaps in earlier versions.

Beyond the technical work, 3.0.0 is also where Percona’s commitment to community-driven development moves from intent to mechanism. The public roadmap is open. The issue tracker is open. The images are freely redistributable. Future releases will be shaped by what the community asks for, files, and contributes back. If there is a feature you want to see in 3.1.0 or 3.2.0, open an issue or a PR, that is where the work happens now.

Try It Out

• Release notes: Percona Operator for PostgreSQL 3.0.0 Release Notes
• Documentation: https://docs.percona.com/percona-operator-for-postgresql/latest/
• GitHub: percona/percona-postgresql-operator
• Public roadmap: https://github.com/orgs/percona/projects/10/views/6
• Issue tracker: https://github.com/percona/percona-postgresql-operator/issues
• Community Forum: forums.percona.com

The post Percona Operator for PostgreSQL 3.0.0: Hard Fork, OLM Scoping, Major Upgrades appeared first on Percona.

The post Percona Operator for PostgreSQL 3.0.0: Hard Fork, OLM Scoping, Major Upgrades appeared first on MariaDB.org.

Continue reading “MariaDB Foundation at Oracle’s MySQL Contributor Summit: Ecosystems, Forks and Constructive Coexistence”

The post MariaDB Foundation at Oracle’s MySQL Contributor Summit: Ecosystems, Forks and Constructive Coexistence appeared first on MariaDB.org.

The post MariaDB Foundation at Oracle’s MySQL Contributor Summit: Ecosystems, Forks and Constructive Coexistence appeared first on MariaDB.org.

Source

The post MariaDB Community Server Q2 2026 corrective releases appeared first on MariaDB.org.

This is part 3 of a 3-part series on running PostgreSQL on Kubernetes with a fully open-source operator. Part 1 walked through the changing open-source landscape and announced the hard fork of the Crunchy Data PostgreSQL Operator into the fully independent Percona PostgreSQL Operator v3.0.0. Part 2 covered the standby cluster method, the safest migration path when downtime budget is tight.

This post covers two simpler paths:

• Backup and restore, the fastest if you can tolerate a short application-downtime window
• Persistent volume reuse, when you want to skip the data copy entirely and keep the existing PGDATA

If you are landing here cold, start with part 1 for the why, then read Part 2 for the standby method. The rest of this post assumes you have already decided to migrate and want a tested playbook.

Different versions may have slight differences in CR fields or behavior. Always consult the official documentation for the operator and PostgreSQL version you are running.

• Application-side connection-string changes beyond updating to the new pgBouncer service
• Schema-changing upgrades, major PostgreSQL version upgrades, or extension migrations
• Crunchy enterprise-only features like TDE or pgBackRest custom encryption
• Operating two operators against the same namespace before the hard fork. Use Percona PostgreSQL Operator v3.0.0 or higher.

This is often the fastest and simplest path, especially when you do not need a live standby. You take a full backup of the Crunchy source cluster, then create a Percona cluster that automatically restores from that backup before its first start.

Data written between the final backup and the application cutover is lost, so the migration window is the time between those two events. For a near-zero-downtime alternative, see part 2: standby cluster method.

export MIGRATION_NS=postgres-migration
kubectl create namespace $MIGRATION_NS

# Generate a self-signed TLS certificate for SeaweedFS S3
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 
  -keyout /tmp/seaweedfs.key 
  -out /tmp/seaweedfs.crt 
  -subj "/CN=seaweedfs-all-in-one"

kubectl -n $MIGRATION_NS create secret tls seaweedfs-s3-tls 
  --cert=/tmp/seaweedfs.crt 
  --key=/tmp/seaweedfs.key

helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm
helm repo update

helm install seaweedfs seaweedfs/seaweedfs 
  --namespace $MIGRATION_NS 
  --version 4.23.0 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-backup-restore/examples/seaweedfs-values.yaml 
  --wait

# Copy and edit the file first to set your credentials.
kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-backup-restore/examples/01-pgbackrest-secrets.yaml

helm install pgo 
  oci://registry.developers.crunchydata.com/crunchydata/pgo 
  -n $MIGRATION_NS 
  --version 5.8.7 
  --set singleNamespace=true 
  --wait

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-backup-restore/examples/02-crunchy-source-cluster.yaml

global:
  repo1-path: /crunchy-to-percona/repo1   # source repo referenced in Percona dataSource
  repo1-s3-uri-style: path                # required for path-style S3 endpoints (SeaweedFS, MinIO)
  repo1-s3-verify-tls: "n"                # skip TLS verification for self-signed cert; remove for AWS S3
repos:
  - name: repo1
    s3:
      bucket: pg-migration
      endpoint: seaweedfs-all-in-one.postgres-migration.svc.cluster.local:8443
      region: us-east-1

kubectl wait pod 
  --selector postgres-operator.crunchydata.com/cluster=crunchy-source,postgres-operator.crunchydata.com/data=postgres 
  -n $MIGRATION_NS 
  --for=condition=Ready 
  --timeout=300s

kubectl wait postgrescluster/crunchy-source 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.pgbackrest.repos[0].stanzaCreated}'=true 
  --timeout=300s

kubectl annotate postgrescluster crunchy-source 
  -n $MIGRATION_NS 
  postgres-operator.crunchydata.com/pgbackrest-backup="$(date +%s)"

kubectl wait job 
  --selector postgres-operator.crunchydata.com/pgbackrest-backup=manual,postgres-operator.crunchydata.com/cluster=crunchy-source 
  -n $MIGRATION_NS 
  --for=condition=Complete 
  --timeout=600s

kubectl apply -n $MIGRATION_NS --server-side 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/tags/v3.0.0/deploy/bundle.yaml

kubectl wait deployment percona-postgresql-operator 
  -n $MIGRATION_NS 
  --for=condition=Available 
  --timeout=120s

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-backup-restore/examples/03-percona-restored-cluster.yaml

dataSource:
  pgbackrest:
    stanza: db
    configuration:
      - secret:
          name: percona-pgbackrest-secret
    global:
      # Must match repo1-path in the Crunchy source cluster exactly.
      repo1-path: /crunchy-to-percona/repo1
      repo1-s3-uri-style: path
      repo1-s3-verify-tls: "n"
    repo:
      name: repo1
      s3:
        bucket: pg-migration
        endpoint: seaweedfs-all-in-one.postgres-migration.svc.cluster.local:8443
        region: us-east-1

backups:
  pgbackrest:
    global:
      repo1-path: /percona-restored/repo1   # different from Crunchy's path

kubectl wait perconapgcluster/percona-restored 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=ready 
  --timeout=600s

PERCONA_PRIMARY=$(kubectl get pod -n $MIGRATION_NS 
  --selector postgres-operator.crunchydata.com/cluster=percona-restored,postgres-operator.crunchydata.com/role=primary 
  -o jsonpath='{.items[0].metadata.name}')

kubectl -n $MIGRATION_NS exec "${PERCONA_PRIMARY}" -c database -- 
  psql -t -c "SELECT pg_is_in_recovery();"

kubectl wait perconapgcluster/percona-restored 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=ready 
  --timeout=600s

kubectl wait perconapgcluster/percona-restored 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.pgbackrest.repos[0].stanzaCreated}'=true 
  --timeout=300s

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-backup-restore/examples/04-post-migration-backup.yaml

kubectl wait perconapgbackup/post-migration-backup 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=Succeeded 
  --timeout=600s

kubectl get service -n $MIGRATION_NS 
  --selector postgres-operator.crunchydata.com/cluster=percona-restored,postgres-operator.crunchydata.com/role=pgbouncer

kubectl delete postgrescluster crunchy-source -n $MIGRATION_NS
helm uninstall pgo -n $MIGRATION_NS

kubectl get postgrescluster crunchy-source -n $MIGRATION_NS 
  -o jsonpath='{.spec.backups.pgbackrest.global.repo1-path}'

kubectl get perconapgcluster percona-restored -n $MIGRATION_NS 
  -o jsonpath='{.spec.dataSource.pgbackrest.global.repo1-path}'

kubectl run -i --rm s3-check 
  --image=perconalab/awscli 
  --restart=Never 
  -n $MIGRATION_NS 
  -- bash -c "
    AWS_ACCESS_KEY_ID=pgmigration 
    AWS_SECRET_ACCESS_KEY=pgmigration123 
    AWS_DEFAULT_REGION=us-east-1 
    aws --endpoint-url https://seaweedfs-all-in-one.${MIGRATION_NS}.svc.cluster.local:8443 
        --no-verify-ssl 
        s3 ls s3://pg-migration
  "

kubectl logs 
  --selector postgres-operator.crunchydata.com/cluster=percona-restored,postgres-operator.crunchydata.com/pgbackrest-restore=percona-restored 
  -n $MIGRATION_NS 
  -c pgbackrest

export MIGRATION_NS=postgres-migration
kubectl create namespace $MIGRATION_NS

helm install pgo 
  oci://registry.developers.crunchydata.com/crunchydata/pgo 
  -n $MIGRATION_NS 
  --version 5.8.7 
  --set singleNamespace=true 
  --wait

kubectl apply -n $MIGRATION_NS --server-side 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/tags/v3.0.0/deploy/bundle.yaml

kubectl wait deployment pgo 
  -n $MIGRATION_NS --for=condition=Available --timeout=120s

kubectl wait deployment percona-postgresql-operator 
  -n $MIGRATION_NS --for=condition=Available --timeout=120s

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-pv/examples/01-crunchy-source-cluster.yaml

kubectl wait pod 
  --selector postgres-operator.crunchydata.com/cluster=crunchy-source,postgres-operator.crunchydata.com/role=master 
  -n $MIGRATION_NS 
  --for=condition=Ready 
  --timeout=300s

PRIMARY=$(kubectl get pod -n $MIGRATION_NS 
  --selector postgres-operator.crunchydata.com/cluster=crunchy-source,postgres-operator.crunchydata.com/role=master 
  -o jsonpath='{.items[0].metadata.name}')

PVC_NAME=$(kubectl get pod -n $MIGRATION_NS "${PRIMARY}" 
  -o jsonpath='{.spec.volumes[?(@.name=="postgres-data")].persistentVolumeClaim.claimName}')

PV_NAME=$(kubectl get pvc -n $MIGRATION_NS "${PVC_NAME}" 
  -o jsonpath='{.spec.volumeName}')

echo "Primary pod: ${PRIMARY}"
echo "PVC:         ${PVC_NAME}"
echo "PV:          ${PV_NAME}"

kubectl patch pv "${PV_NAME}" 
  -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

kubectl get pv -n $MIGRATION_NS

kubectl patch postgrescluster crunchy-source -n $MIGRATION_NS 
  --type=json -p='[{"op":"remove","path":"/metadata/finalizers"}]' 2>/dev/null || true

kubectl delete postgrescluster crunchy-source -n $MIGRATION_NS
helm uninstall pgo -n $MIGRATION_NS

kubectl patch pv "${PV_NAME}" --type=json 
  -p='[{"op":"remove","path":"/spec/claimRef"}]'

kubectl wait pv "${PV_NAME}" 
  --for=jsonpath='{.status.phase}'=Available 
  --timeout=60s

kubectl label pv "${PV_NAME}" percona-pv-migration=migrated

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-pv/examples/02-percona-migrated-cluster.yaml

instances:
  - name: instance1
    replicas: 1
    dataVolumeClaimSpec:
      selector:
        matchLabels:
          percona-pv-migration: migrated

kubectl wait perconapgcluster/percona-migrated 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=ready 
  --timeout=600s

PERCONA_PRIMARY=$(kubectl get pod -n $MIGRATION_NS 
  --selector postgres-operator.crunchydata.com/cluster=percona-migrated,postgres-operator.crunchydata.com/role=primary 
  -o jsonpath='{.items[0].metadata.name}')

kubectl -n $MIGRATION_NS exec "${PERCONA_PRIMARY}" -c database -- 
  psql -t -c "SELECT pg_is_in_recovery();"

kubectl patch perconapgcluster percona-migrated 
  --namespace $MIGRATION_NS 
  --type=json 
  -p='[
    {"op":"remove","path":"/spec/instances/0/dataVolumeClaimSpec/selector"},
    {"op":"replace","path":"/spec/instances/0/replicas","value":3}
  ]'

kubectl wait perconapgcluster/percona-migrated 
  --namespace $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=ready 
  --timeout=300s

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-pv/examples/03-post-migration-backup.yaml

kubectl wait perconapgbackup/post-migration-backup 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=Succeeded 
  --timeout=600s

kubectl get service -n $MIGRATION_NS 
  --selector postgres-operator.crunchydata.com/cluster=percona-migrated,postgres-operator.crunchydata.com/role=pgbouncer

kubectl label pv "${PV_NAME}" percona-pv-migration-

kubectl get pv "${PV_NAME}" --show-labels
kubectl get pv "${PV_NAME}" -o jsonpath='{.status.phase}'

kubectl -n $MIGRATION_NS logs "${PERCONA_PRIMARY}" -c database

The post Migrate from Crunchy Data PostgreSQL Operator to Percona PostgreSQL Operator: Backup-Restore and PV Reuse appeared first on Percona.

The post Migrate from Crunchy Data PostgreSQL Operator to Percona PostgreSQL Operator: Backup-Restore and PV Reuse appeared first on MariaDB.org.

Continue reading “Virtuozzo Renews Sponsorship of MariaDB Foundation”

The post Virtuozzo Renews Sponsorship of MariaDB Foundation appeared first on MariaDB.org.

The post Virtuozzo Renews Sponsorship of MariaDB Foundation appeared first on MariaDB.org.

Source

The post How MariaDB Cloud Optimizes Database Resilience and Cost: A Deep Dive into High Availability appeared first on MariaDB.org.

The financial sector particularly suffers, as proprietary databases hinder system updates for compliance and real-time AI, impose rigid pricing, and shift operational risk to the buyer.

Procurement leaders are starting to see this problem. Over 74% of Database as a Service (DBaaS) users cite high and unpredictable costs as their top challenge due to proprietary pricing structures. Meanwhile, the open source database market is projected to reach $63.48 billion by 2034, signaling a major industry shift.

Switching to open source databases offers procurement teams better financial control, allowing spending to be measured and predicted like any other asset.

This article provides a framework for procurement leaders to realize the ROI of open source databases. It explains how to move beyond license-focused sourcing to a strategy that prioritizes risk reduction, spend predictability, and vendor optionality.

The procurement blind spot: What database TCO really includes

License fees often become the total cost baseline in many sourcing cycles. But that license cost is only a fraction of the true database Total Cost of Ownership (TCO). The massive operational and strategic costs are hidden beneath the surface. The cost drivers show up in six areas:

• Outage and SLA penalties: Downtime incurred due to vendor-managed recovery or architecture constraints.
• Forced over-provisioning: Licensing models that require institutions to buy capacity they may not fully use (because licenses are sold in “blocks” or “cores”). If your workload requires 9 cores, you are often forced to pay for 16.
• Escalating renewal pricing: Per-core or per-instance fees that climb with infrastructure growth, unrelated to feature value.
• Data egress fees and platform taxes: Cloud DBaaS charges for cross-region replication, data exports, backups, and traffic that accumulate unpredictably.
• Staffing and operational overhead: Database administrators and Site Reliability Engineers (SREs) dedicating time to tuning, patching, and managing vendor-specific tooling.
• Migration and switching costs: The financial and technical burden of moving data if vendor changes or licensing terms shift.

Downtime as a financial liability (Not a technical issue)

Procurement teams may not always be the primary owners of downtime risk, but they often influence it through vendor selection, contract terms, and support coverage. Because outages carry measurable business impact, support responsiveness and recovery capability should be evaluated as a financial exposure.

For example, critical-incident response and restoration expectations must be defined and aligned with the organization’s risk tolerance. If not, the institution may be accepting avoidable financial and operational exposure during high-severity events.

The financial model is simple:

(Incident Frequency) × (Incident Duration) × (Cost Per Hour) = Annual Risk Exposure

For a bank with three major outages per year, averaging 4 hours each, with a $500K/hour impact:

3 × 4 × $500,000 = $6 million in annual downtime risk

Open source works best when supported by vendor-agnostic experts like Percona’s. It allows procurement to source support that focuses on restoring service across the entire stack, rather than defending a specific piece of software.

Cost predictability vs. vendor-driven cost escalation

Budget forecasting becomes impossible when database costs are unpredictable. Yet proprietary licensing introduces multiple mechanisms that undermine forecast accuracy and negatively affect business success.

• The scaling penalty: As your customer base grows and you add more hardware, your software costs increase exponentially because of per-core or per-socket licensing.
• Tier creep: You might start on a Standard tier, but as soon as you need a critical security feature like advanced encryption or granular auditing for DORA (Digital Operational Resilience Act) compliance, you are forced into an Enterprise tier that can cost more.

In contrast, open source separates the software cost from growth. If you double your infrastructure to handle peak trading volumes, your software cost remains zero. It lets procurement provide the business with a linear, predictable cost model (you only pay for the infrastructure you use and the expertise required to run it).

Vendor lock-in and contract leverage

The primary objective of a sales team from a proprietary vendor is to make customers more committed to their products. The more vendor-specific features you use, the harder it is for procurement to negotiate when renewing the contract. Over time:

• Switching costs add up: Data migration, changing schemas, and application changes create high barriers to leaving.
• Vendor leverage grows: As integration deepens, alternatives become more costly, reducing competition in renewal negotiations.
• Renewal pricing rises: With fewer alternatives, vendors increase renewal fees, confident that institutions cannot easily leave.

Percona’s research on Redis users shows that nearly 75% have considered or tested alternatives when licensing terms change, but most couldn’t really switch. This is vendor lock-in at its most destructive, as institutions resent the vendor but cannot leave.

Open source gives institutions more options (restores vendor optionality). Procurement can regain power through:

• Multi-vendor support: If one support provider underperforms or raises prices, you can move your support contract to another provider without migrating your data.
• Deployment flexibility: Open source can run on-premise, in any cloud (AWS, Azure, GCP), or in a hybrid model.
• Lower switching costs: Since open source uses standard protocols, it is easier to find talent and tools that work across the stack, and reduce the exit cost of any single relationship.

Operational efficiency as a budget control mechanism

Database operations often increase operating expenses. When teams react to problems rather than prevent them, labor costs rise without notice. Inefficient database management raises labor costs in two ways:

• Specialized labor scarcity: Finding a specialist for a proprietary database is expensive.
• Reactive engineering: When database performance is poor, teams spend more time fixing issues instead of building new products.

Switching to an open-source system with integrated management tools, like Percona Monitoring and Management, can help the organization save valuable engineering time.

For example, if a 10-person engineering team spends 20% of their time on manual database maintenance, that’s like paying two full-time employees just to keep things running. Improving tools and support reduces this work and provides immediate operational ROI.

Data egress fees: The hidden variable cost

In cloud services, it’s usually free to get your data in, but costs can skyrocket when you want to get your data out. Many managed proprietary DBaaS platforms are designed to trap your data. They make it easy to scale up, but charge massive data egress fees if you want to move that data to a third-party analytics tool or a different cloud provider.

Open source databases, particularly when run on Kubernetes or self-managed infrastructure, give you full control over the data path. With that control, procurement and platform stakeholders can design data flows that reduce unnecessary cross-cloud transfers and help minimize egress fees.

Annualized ROIs Summary for procurement

When presenting the move to open source to the executive team, procurement should frame the benefits across the following financial pillars:

Why open source aligns with procurement objectives

Modern procurement is about governance, compliance, and strategic alignment. Open source databases align with these goals better than proprietary ones:

• No licensing premiums for scale or performance. A 10x increase in data does not mean 10x higher license fees.
• Transparent, auditable cost structures. Procurement knows exactly what they are paying for.
• Support can be competitively sourced. Institutions are not locked into a single software vendor.
• Better compliance and security. Open code enables internal security reviews, transparency for auditors, and helps meet regulatory requirements.

Operating open source with procurement-grade assurance

A common objection to open source is that it’s unsupported. Procurement teams need operational assurance, confidence that open source environments meet the same regulatory, availability, and financial standards as proprietary systems.

When evaluating a support partner, procurement should require:

• SLA clarity: Specific, contractually backed response and resolution times.
• Multi-database coverage: One contract that covers MySQL, PostgreSQL, and MongoDB to reduce contract sprawl.
• Regulated-environment experience: A partner who understands PCI-DSS, SOC2, and the high-compliance needs of finance.

Percona meets all of these criteria and offers procurement with a partner that turns technical operations into financial metrics.

Where Percona fits

Percona operationalizes open source databases for regulated, mission-critical environments. It delivers measurable outcomes across four strategic dimensions for procurement teams:

Independent, vendor-neutral support model

Percona provides technology-agnostic support across MySQL, PostgreSQL, MongoDB, MariaDB, and Valkey. It operates independently of cloud providers and database vendors, supporting on premises, cloud, and hybrid environments. This vendor neutrality ensures institutions maintain full control over technology choices without being locked into specific platforms or ecosystems.

Predictable support costs without licensing dependency

Percona’s pricing model decouples support costs from database licensing and creates transparent, forecastable expenses. While proprietary databases force organizations to pay escalating per-core or usage-based fees, Percona’s support subscriptions operate independently of infrastructure growth. For example, organizations like BBVA reduced licensing and support costs while simultaneously improving backup performance by 20% after migrating to Percona Server for MongoDB.

Proven experience supporting regulated financial systems

Percona supports regulated financial systems, including Fortune 500 companies and government agencies, and meets compliance standards such as HIPAA, PCI DSS, GDPR, and DORA EU.

Major financial services implementations include:

• Merchant Warrior: Australia’s payments gateway relies on Percona for critical MySQL availability, supporting millions of transactions across 30,000+ customers.
• MultiPay and Bukalapak: Financial services and e-commerce platforms leveraging Percona’s support to maintain high availability and optimize deployment performance.

Conclusion: Database performance as a spend control strategy

Databases have evolved from technical infrastructure into financial assets. Their uptime, performance, and flexibility influence costs, vendor leverage, and operational resilience. For procurement, buying databases is a strategic investment to control expenses and manage risks.

Organizations gain predictable costs, measurable ROI, vendor optionality, and long-term operational control by choosing open-source databases and partnering with Percona. These advantages compound over time, while proprietary systems often fall short.

Get started with Percona Operators and see how consistency, scale, and freedom come together.

The post How Procurement Leaders Realize ROI from Open Source Databases appeared first on Percona.

The post How Procurement Leaders Realize ROI from Open Source Databases appeared first on MariaDB.org.

Continue reading “ProxySQL joins MariaDB Foundation as Silver Sponsor”

The post ProxySQL joins MariaDB Foundation as Silver Sponsor appeared first on MariaDB.org.

The post ProxySQL joins MariaDB Foundation as Silver Sponsor appeared first on MariaDB.org.

Continue reading “ProxySQL joins MariaDB Foundation as Silver Sponsor”

The post ProxySQL joins MariaDB Foundation as Silver Sponsor appeared first on MariaDB.org.

The post ProxySQL joins MariaDB Foundation as Silver Sponsor appeared first on MariaDB.org.

Continue reading “ProxySQL joins MariaDB Foundation as Silver Sponsor”

The post ProxySQL joins MariaDB Foundation as Silver Sponsor appeared first on MariaDB.org.

The post ProxySQL joins MariaDB Foundation as Silver Sponsor appeared first on MariaDB.org.

Continue reading “Drupal recommends MariaDB”

The post Drupal recommends MariaDB appeared first on MariaDB.org.

The post Drupal recommends MariaDB appeared first on MariaDB.org.

Why we made it

There was already a way to use TidesDB from SQL. It’s TideSQL, which loads the engine into MariaDB as ha_tidesdb, and it works fine. But it doesn’t work with MySQL. So we wanted TidesDB to work with MySQL 9.7.

MariaDB and MySQL share a lot of history, but they are not the same. We couldn’t just recompile the MariaDB plugin against MySQL headers and call it done. The one thing that stayed put through all of it was TidesDB itself, doing exactly what it does anywhere else. Only the server wrapped around was changed. In result we got our implementation, so if you’re on MySQL, you no longer have to switch to MariaDB to give TidesDB a try.

What it actually is

tidesdb-mysql is a loadable plugin, ha_tidesdb.so. The engine gets built on its own and loaded into the server at runtime, the same shape as the MariaDB version. It speaks the MySQL handler API and wires MySQL tables and indexes onto TidesDB column families. After it loads, TidesDB sits right next to InnoDB in SHOW ENGINES and you choose it per table.

Getting started

All you need is Docker. Pull the image and start it:

docker pull perconalab/tidesdb-mysql:0.2.4

docker run -d --name tidesdb 
-e MYSQL_ROOT_PASSWORD=secret 
-p 3306:3306 
perconalab/tidesdb-mysql:0.2.4

The plugin is baked into this image and loaded on boot, so there’s no INSTALL PLUGIN step to remember. Confirm the engine is live:

docker exec tidesdb mysql -uroot -psecret 
-e "SELECT engine, support FROM information_schema.engines WHERE engine='TidesDB';"
# TidesDB | YES

Now make a table and treat it like any other:

CREATE DATABASE shop;
USE shop;

CREATE TABLE products (
id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(64) NOT NULL,
price DECIMAL(10,2) NOT NULL,
KEY idx_price (price)
) ENGINE=TIDESDB;

INSERT INTO products (name, price) VALUES ('Widget', 9.99), ('Gadget', 24.50);
SELECT * FROM products WHERE price < 20;

Transactions, secondary indexes, the usual SQL, it all behaves:

START TRANSACTION;
UPDATE products SET price = price + 1 WHERE name = 'Widget';
COMMIT;

Per-table TidesDB options ride along in MySQL’s ENGINE_ATTRIBUTE JSON field. MySQL doesn’t have MariaDB’s COMPRESSION=… grammar, so the options are identical but you write them differently:

CREATE TABLE events (
id BIGINT PRIMARY KEY AUTO_INCREMENT,
msg TEXT
) ENGINE=TIDESDB
ENGINE_ATTRIBUTE='{"compression":"ZSTD","bloom_filter":true}';

Compression accepts NONE, SNAPPY, LZ4, ZSTD, or LZ4_FAST. Server-wide knobs live in system variables such as tidesdb_default_compression, tidesdb_block_cache_size, tidesdb_compaction_threads, and tidesdb_flush_threads. The full list is in docs/build-and-load.md.

Prove the crash recovery

Write a handful of rows, kill the server with no clean shutdown, bring it back, and count what’s left:

# 1. Write rows inside a transaction and COMMIT.

docker exec -i tidesdb mysql -uroot -psecret <<'SQL'
CREATE DATABASE IF NOT EXISTS t;
CREATE TABLE IF NOT EXISTS t.kv (k INT PRIMARY KEY, v VARCHAR(32)) ENGINE=TIDESDB;
BEGIN;
INSERT INTO t.kv VALUES (1,'a'),(2,'b'),(3,'c'),(4,'d'),(5,'e');
COMMIT;
SELECT COUNT(*) AS before_crash FROM t.kv; -- 5
SQL

# 2. Hard-kill the server (no graceful shutdown) and restart it.

docker kill -s KILL tidesdb
docker start tidesdb
until docker exec tidesdb mysql -uroot -psecret -e 'SELECT 1' >/dev/null 2>&1; do sleep 2; done

# 3. The committed rows are still there.

docker exec tidesdb mysql -uroot -psecret 
-e "SELECT COUNT(*) AS after_crash FROM t.kv;" -- 5

after_crash should come back equal to before_crash.

A few more things to try

Compression is the one people ask about first, so here’s a table that leans on it. We generate a couple thousand rows of repetitive text, which is exactly the shape ZSTD likes:

CREATE TABLE logs (
id BIGINT PRIMARY KEY AUTO_INCREMENT,
level VARCHAR(8) NOT NULL,
body TEXT,
KEY idx_level (level)
) ENGINE=TIDESDB
ENGINE_ATTRIBUTE='{"compression":"ZSTD","bloom_filter":true}';

INSERT INTO logs (level, body)
SELECT IF(RAND() < 0.2, 'warn', 'info'),
REPEAT('the quick brown fox jumps over the lazy dog ', 40)
FROM information_schema.columns
LIMIT 2000;

SELECT level, COUNT(*) AS rows FROM logs GROUP BY level;
SELECT id, LEFT(body, 30) AS preview FROM logs WHERE id = 1000;

The rows go in compressed and come back out as the original text, so queries don’t change at all. If you want to confirm the option actually landed on the table rather than being silently dropped, ask the server what it stored:

SHOW CREATE TABLE logsG
-- ENGINE=TIDESDB ... ENGINE_ATTRIBUTE='{"compression":"ZSTD","bloom_filter":true}'

The bloom filter from that same attribute is what keeps point lookups cheap once the data has compacted down into several on-disk files:

SELECT id, level FROM logs WHERE id = 1500;

A JSON column behaves the way you’d expect, including the ->> extraction operator:

CREATE TABLE kv (k VARCHAR(64) PRIMARY KEY, v JSON) ENGINE=TIDESDB;

INSERT INTO kv VALUES
('en', JSON_OBJECT('lang','English', 'msg','hello')),
('es', JSON_OBJECT('lang','Spanish', 'msg','hola')),
('fr', JSON_OBJECT('lang','French', 'msg','bonjour'));

SELECT k, v->>'$.lang' AS language, v->>'$.msg' AS greeting
FROM kv
ORDER BY k;

And the secondary index on products from earlier is a real index, not decoration. A range query uses it, and EXPLAIN will show idx_price in the key column:

SELECT name, price FROM products WHERE price BETWEEN 5 AND 20 ORDER BY price;

EXPLAIN SELECT name, price FROM products WHERE price BETWEEN 5 AND 20;

What works, and what doesn’t yet

Quite a bit works. The common column types are all there, primary keys single and composite, AUTO_INCREMENT, secondary indexes with index-condition pushdown, COMMIT/ROLLBACK, REPLACE and INSERT … ON DUPLICATE KEY UPDATE, online add/drop index, instant add column, full-text search, spatial indexes, per-row TTL, per-table compression and bloom filters, at-rest encryption, and mixed-engine transactions where a TidesDB table and an InnoDB table share one BEGIN … COMMIT. The functional test suite, which we lifted from TideSQL and then extended, passes 58 of 58 executed tests.

A few things you should know about before you lean on it:

• Native partitioning and the MySQL 9 vector column type aren’t implemented. Those two test cases are skipped deliberately.
• Atomic, crash-safe DDL (the data-dictionary integration) is wired up but we haven’t driven it end-to-end yet. Your data writes are crash-safe; schema changes during a crash are next on the list.
• Replication, foreign keys, and nested savepoints aren’t in scope at the moment.

Treat v0.2.5 as a serious experiment. It’s solid enough that committed data rides through a crash, and it’s not something we’d point production traffic at yet.

Try it, then tell us

docker pull perconalab/tidesdb-mysql:0.2.5

That’s the whole setup. Spin up a table with ENGINE=TIDESDB, run the crash demo, and point your own SQL at it. The source, the build scripts, and the engine patches all live in the tidesdb-mysql repository, and the durability fixes are written up in KNOWN-ISSUES.md. This is a tool made by users for users, so if you give it a spin, we’d genuinely like to hear what held up and what fell over.

The post Running TidesDB as a MySQL 9.7 storage engine appeared first on Percona.

The post Running TidesDB as a MySQL 9.7 storage engine appeared first on MariaDB.org.

A Crunchy to Percona PostgreSQL migration is more straightforward than most cross-operator moves on Kubernetes, because the Percona PostgreSQL Operator is a hard fork of the Crunchy Data PostgreSQL Operator. Same Patroni HA, same pgBackRest backups, same overall CRD shape. This post walks through the safest of the three migration paths: a standby cluster method with near-zero downtime.

This is part 2 of a 3-part series on running PostgreSQL on Kubernetes with a fully open-source operator. Part 1 walked through the changing open-source landscape and announced the hard fork of the Crunchy Data PostgreSQL Operator into the fully independent Percona PostgreSQL Operator v3.0.0.

This post is the first practical playbook of the series. It covers the standby cluster method, the safest migration path when the downtime budget is tight. Part 3 will cover two simpler paths: backup-and-restore and persistent-volume reuse.

If you are landing here without context on why you might want to migrate at all, start with part 1. The rest of this post assumes you have already decided to move and want a tested playbook.

Migration approach in one paragraph

The Percona PostgreSQL Kubernetes Operator is a hard fork of the Crunchy Data PostgreSQL Kubernetes Operator, which simplifies the migration paths considerably: the same underlying tools (Patroni, pgBackRest, PgBouncer) and the same overall design are used in both operators. All three migration paths in this series are reversible: because Percona’s operator is fully open source and remains compatible with the same backup format, the move back to Crunchy is also possible if your team decides to walk it

A note on the storage layer

All examples in this guide use an in-cluster SeaweedFS instance as the pgBackRest S3 repository. SeaweedFS is Apache-2.0 licensed, actively maintained, and a clean drop-in replacement for the role MinIO used to fill in this stack. Any other S3-compatible storage works just as well: AWS S3, Google Cloud Storage (via HMAC keys), Ceph RadosGW, Cloudflare R2, and so on. For non-SeaweedFS endpoints, remove repo1-s3-uri-style: path and repo1-s3-verify-tls: “n” from the pgBackRest configuration and replace the endpoint with your provider’s URL.

What this series does NOT cover

To keep scope honest:

• Application-side connection-string changes beyond updating to the new pgBouncer service. If your app uses connection-pool tuning, custom auth, or a service mesh, that work stays with you.
• Schema-changing upgrades, major PostgreSQL version upgrades, or extension migrations. The PostgreSQL major version must match between the source and the target.
• Crunchy enterprise-only features like TDE, Crunchy Postgres for Kubernetes-specific operators, or pgBackRest custom encryption. If your environment uses these, contact the Percona team for a tailored plan.
• Operating two operators against the same namespace before the PGO hard fork. Use Percona PostgreSQL Operator v3.0.0 or higher.

Tested with

Different versions may differ slightly in CR fields or behavior. Always consult the official documentation for the operator and PostgreSQL version you are running.

Migration using a standby cluster

This is the safest method when the downtime budget is tight. The Percona cluster is brought up as a standby of the Crunchy primary, catches up via pgBackRest plus streaming replication, and is promoted at cutover. The only downtime is the cutover step itself.

You can wire the standby in two ways, and combining both gives you maximum safety:

• pgBackRest repo-based standby seeds the standby from the latest base backup and replays archived WAL
• Streaming replication keeps the standby in sync with the live primary

Overview

Before you begin

Set the target namespace once. Every command in this guide reads from this variable, so you can change it in a single place:

export MIGRATION_NS=postgres-migration
kubectl create namespace $MIGRATION_NS

Deploy SeaweedFS

Skip this step if you already have an S3-compatible repository (AWS S3, GCS, Ceph). Update the endpoint and credentials in the YAML examples accordingly.

SeaweedFS provides an S3-compatible object store that runs inside Kubernetes. Both operators will use it as the shared pgBackRest WAL archive.

TLS is required. pgBackRest always connects to S3 endpoints over HTTPS, even when repo1-s3-verify-tls: “n” is set (that flag skips certificate verification, it does not fall back to HTTP). The steps below generate a self-signed certificate and pass it to SeaweedFS via Helm values.

# Generate a self-signed TLS certificate for SeaweedFS S3
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 
  -keyout /tmp/seaweedfs.key 
  -out /tmp/seaweedfs.crt 
  -subj "/CN=seaweedfs-all-in-one"

kubectl -n $MIGRATION_NS create secret tls seaweedfs-s3-tls 
  --cert=/tmp/seaweedfs.crt 
  --key=/tmp/seaweedfs.key

helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm
helm repo update

helm install seaweedfs seaweedfs/seaweedfs 
  --namespace $MIGRATION_NS 
  --version 4.23.0 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-standby/examples/seaweedfs-values.yaml 
  --wait

The Helm values file in the repo creates the pg-migration bucket on first start, so no separate aws s3 mb step is needed.

Step 0. Create pgBackRest secrets

Both operators need credentials to read and write the shared SeaweedFS bucket. Apply the secrets from examples/01-pgbackrest-secret.yaml after filling in your access key and secret key:

# Copy and edit the file first to set your credentials.

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-standby/examples/01-pgbackrest-secret.yaml

Both secrets contain the same SeaweedFS credentials (pgmigration / pgmigration123). For AWS S3, replace those with your IAM access key ID and secret access key.

Step 1. Start with your existing Crunchy Data cluster

If you already have a running Crunchy cluster, ensure its pgBackRest repo1 points at the shared bucket and path. The repo1-path value must be identical in both cluster specs. Mismatched paths will prevent the Percona standby from finding the WAL archive.

The Helm install below is shown only as a quick way to reproduce this blog post’s example. The migration steps in the rest of this post do not depend on how you deployed the source operator.

Optional: deploy a Crunchy operator to test the migration end to end:

helm install pgo 
  oci://registry.developers.crunchydata.com/crunchydata/pgo 
  -n $MIGRATION_NS 
  --version 5.8.7 
  --set singleNamespace=true 
  --wait

Apply examples/02-crunchy-source-cluster.yaml (or adapt your existing cluster’s pgBackRest config):

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-standby/examples/02-crunchy-source-cluster.yaml

The key pgBackRest settings in the example:

global:
  repo1-path: /crunchy-to-percona/repo1   # shared path, must match Percona side
  repo1-s3-uri-style: path                # required for path-style S3 endpoints (SeaweedFS, MinIO)
  repo1-s3-verify-tls: "n"                # skip TLS verification for self-signed cert; remove for AWS S3
repos:
  - name: repo1
    s3:
      bucket: pg-migration
      endpoint: seaweedfs-all-in-one.postgres-migration.svc.cluster.local:8443
      region: us-east-1

Wait for the cluster to be ready:

kubectl wait pod 
  --selector postgres-operator.crunchydata.com/cluster=crunchy-source,postgres-operator.crunchydata.com/data=postgres 
  --namespace $MIGRATION_NS 
  --for=condition=Ready 
  --timeout=300s

Step 2. Trigger a full backup on the Crunchy cluster

Wait for the pgBackRest stanza to be created:

kubectl wait postgrescluster/crunchy-source 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.pgbackrest.repos[0].stanzaCreated}'=true 
  --timeout=300s

Take a full backup before creating the Percona standby. This gives the standby a recent base to restore from, so it only needs to replay a small amount of WAL to catch up. This matches the realistic production migration pattern.

kubectl annotate postgrescluster crunchy-source 
  --namespace $MIGRATION_NS 
  postgres-operator.crunchydata.com/pgbackrest-backup="$(date +%s)"

Wait for the backup job to complete:

kubectl wait job 
  -l postgres-operator.crunchydata.com/pgbackrest-backup=manual,postgres-operator.crunchydata.com/cluster=crunchy-source 
  -n $MIGRATION_NS 
  --for=condition=Complete 
  --timeout=600s

Step 3. Copy TLS certificates (cross-namespace only)

If the Percona cluster is in a different namespace from the Crunchy cluster, copy the Crunchy TLS secrets to the Percona namespace. These allow mutual TLS authentication during streaming replication:

for secret in crunchy-source-cluster-cert crunchy-source-replication-cert; do
  kubectl get secret "${secret}" -n <CRUNCHY_NS> -o json | 
    yq '{"apiVersion": .apiVersion, "kind": .kind, "data": .data,
         "metadata": {"name": .metadata.name}, "type": .type}' -o yaml | 
    kubectl -n $MIGRATION_NS apply -f -
done

If both clusters are in the same namespace, skip this step. The secrets are already accessible.

Step 4. Deploy the Percona PG Operator

The Crunchy PGO operator can stay in the same or a different namespace.

kubectl apply -n $MIGRATION_NS --server-side 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/tags/v3.0.0/deploy/bundle.yaml

Wait until the operator deployment is ready:

kubectl wait deployment percona-postgresql-operator 
  -n $MIGRATION_NS 
  --for=condition=Available 
  --timeout=120s

Step 5. Create the Percona cluster in standby mode

Note: The kubectl apply below pulls the CR manifest from the migration-from-crunchy-guide branch of the operator repo, which is the source for this guide’s examples. For production deployments, follow the official Percona Operator for PostgreSQL installation documentation and pin to a released version tag rather than a feature branch.

Apply examples/03-percona-standby-cluster.yaml:

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-standby/examples/03-percona-standby-cluster.yaml

The key settings that wire the Percona cluster to the Crunchy source:

standby:
  enabled: true
  repoName: repo1                             # restore initial base backup from this repo
  host: crunchy-source-ha.postgres-migration.svc.cluster.local
  port: 5432

secrets:
  customTLSSecret:
    name: crunchy-source-cluster-cert         # Crunchy CA for mutual TLS
  customReplicationTLSSecret:
    name: crunchy-source-replication-cert     # cert for _crunchyreplication user

The Percona operator will:

1. Restore the base backup from the SeaweedFS bucket.
2. Replay WAL from SeaweedFS until it catches up with the live Crunchy cluster.
3. Switch to streaming replication from crunchy-source-ha.

Wait for the cluster to reach the ready state:

kubectl wait perconapgcluster/percona-standby 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=ready 
  --timeout=600s

Verify that data is replicating to the standby:

STANDBY_POD=$(kubectl get pod -n $MIGRATION_NS 
  -l postgres-operator.crunchydata.com/cluster=percona-standby,postgres-operator.crunchydata.com/data=postgres 
  -o jsonpath='{.items[0].metadata.name}')

kubectl -n $MIGRATION_NS exec "${STANDBY_POD}" -c database -- 
  psql -t -c "SELECT pg_is_in_recovery(), pg_last_wal_replay_lsn();"

Expected output: t (in recovery) and a non-null LSN.

Step 6. Verify replication lag before cutover

Query the Crunchy primary to confirm the Percona standby has caught up:

CRUNCHY_PRIMARY=$(kubectl get pod 
  -l postgres-operator.crunchydata.com/cluster=crunchy-source,postgres-operator.crunchydata.com/role=master 
  -n $MIGRATION_NS 
  -o jsonpath='{.items[0].metadata.name}')

kubectl -n $MIGRATION_NS exec "${CRUNCHY_PRIMARY}" -c database -- 
  psql -c "
    SELECT
        client_addr,
        state,
        pg_wal_lsn_diff(sent_lsn, replay_lsn) AS byte_lag,
        write_lag,
        flush_lag,
        replay_lag
    FROM pg_stat_replication;
  "

Proceed to the next step only when write_lag and replay_lag are NULL or under a few seconds.

Step 7. Cutover the Crunchy cluster

This is the only step that causes downtime. Stop accepting writes on the application side, then patch the Crunchy cluster into standby mode. Patroni steps down and archives the final WAL.

kubectl patch postgrescluster crunchy-source 
  -n $MIGRATION_NS 
  --type=merge 
  -p '{"spec": {"standby": {"enabled": true, "repoName": "repo1"}}}'

Verify demotion (poll until pg_is_in_recovery() returns t):

kubectl -n $MIGRATION_NS exec "${CRUNCHY_PRIMARY}" -c database -- 
  psql -t -c "SELECT pg_is_in_recovery();"

Step 8. (Optional) Shut down the Crunchy cluster

Once the Percona standby has replayed all WAL, shut down the Crunchy cluster to prevent split-brain:

kubectl patch postgrescluster crunchy-source 
  -n $MIGRATION_NS 
  --type=merge 
  -p '{"spec": {"shutdown": true}}'

kubectl wait pod 
  -l postgres-operator.crunchydata.com/cluster=crunchy-source,postgres-operator.crunchydata.com/data=postgres 
  -n $MIGRATION_NS 
  --for=delete 
  --timeout=120s || true

Step 9. Promote the Percona cluster

Confirm that the Percona standby has finished replaying all WAL (the LSN stops advancing):

kubectl -n $MIGRATION_NS exec "${STANDBY_POD}" -c database -- 
  psql -t -c "SELECT pg_last_wal_replay_lsn();"

Run this a few times. When the LSN is stable, replay is complete.

kubectl patch perconapgcluster percona-standby 
  -n $MIGRATION_NS 
  --type=merge 
  -p '{"spec": {"standby": {"enabled": false}}}'

Wait for the cluster to become ready and confirm it is writable:

kubectl wait perconapgcluster/percona-standby 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=ready 
  --timeout=480s

PERCONA_PRIMARY=$(kubectl get pod -n $MIGRATION_NS 
  -l postgres-operator.crunchydata.com/cluster=percona-standby,postgres-operator.crunchydata.com/role=primary 
  -o jsonpath='{.items[0].metadata.name}')

kubectl -n $MIGRATION_NS exec "${PERCONA_PRIMARY}" -c database -- 
  psql -t -c "SELECT pg_is_in_recovery();"

Expected output: f (the cluster is now the primary and accepts writes).

Step 10. Verify stanza creation

kubectl wait perconapgcluster/percona-standby 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.pgbackrest.repos[0].stanzaCreated}'=true 
  --timeout=300s

Step 11. Take a post-migration backup

Apply examples/04-post-migration-backup.yaml:

kubectl apply -n $MIGRATION_NS 
  -f https://raw.githubusercontent.com/percona/percona-postgresql-operator/refs/heads/migration-from-crunchy-guide/e2e-tests/tests/migration-from-crunchy-standby/examples/04-post-migration-backup.yaml

kubectl wait perconapgbackup/post-migration-backup 
  -n $MIGRATION_NS 
  --for=jsonpath='{.status.state}'=Succeeded 
  --timeout=600s

This creates a clean recovery point on the new timeline. All future PITR restores will use this backup as their starting point, independent of the old Crunchy WAL archive.

Reconnecting your application

Update your application’s connection string to point at the Percona cluster’s pgBouncer service:

kubectl get service -n $MIGRATION_NS 
  -l postgres-operator.crunchydata.com/cluster=percona-standby,postgres-operator.crunchydata.com/role=pgbouncer

This migration path works almost entirely out of the box. For users coming from the Crunchy Data PostgreSQL Operator, this method feels familiar because it leverages the same standby/replica mechanisms used for HA and disaster recovery. The key difference is that you can now use this familiar mechanism to migrate safely to the Percona PostgreSQL Operator, a fully open-source alternative running on a fully open-source storage layer.

Rollback

The standby method is the most rollback-friendly of the three. Until you take the post-migration backup, the Crunchy cluster still holds the original timeline. To roll back:

1. Stop writes on the Percona side and patch the Percona cluster back into standby mode (spec.standby.enabled: true).
2. Patch the Crunchy cluster out of standby mode and let Patroni promote it.
3. Verify with pg_is_in_recovery() on both sides.
4. Switch the application connection string back to the Crunchy pgBouncer service.

After Step 11 (post-migration backup), the timelines have diverged. From that point, the rollback story is the same as a fresh restore, and you should treat the Crunchy cluster as a historical reference, not a live target.

Troubleshooting

Percona standby not connecting to the Crunchy primary. Verify the crunchy-source-ha service resolves from within the Percona pod:

kubectl -n $MIGRATION_NS exec "${STANDBY_POD}" -c database -- 
  bash -c "getent hosts crunchy-source-ha.${MIGRATION_NS}.svc.cluster.local"

Replication authentication errors. The Percona standby authenticates as the _crunchyreplication PostgreSQL user using the certificate in crunchy-source-replication-cert. Verify the secret exists and matches what the Crunchy operator generated:

kubectl get secret crunchy-source-replication-cert -n $MIGRATION_NS

pgBackRest restore fails. Confirm both secrets contain identical credentials and that repo1-path is the same in both cluster specs (/crunchy-to-percona/repo1 in this guide). Mismatched paths cause an archive.info missing error. Verify the bucket is reachable:

kubectl run -i --rm s3-check 
  --image=perconalab/awscli 
  --restart=Never 
  -n $MIGRATION_NS 
  -- bash -c "
    AWS_ACCESS_KEY_ID=pgmigration 
    AWS_SECRET_ACCESS_KEY=pgmigration123 
    AWS_DEFAULT_REGION=us-east-1 
    aws --endpoint-url https://seaweedfs-all-in-one.${MIGRATION_NS}.svc.cluster.local:8443 
        --no-verify-ssl 
        s3 ls s3://pg-migration
  "

Timeline history file (00000002.history) missing after promotion. This is a known issue with Crunchy PGO’s async archive mode. After promotion, push the history file synchronously:

kubectl -n $MIGRATION_NS exec "${PERCONA_PRIMARY}" -c database -- 
  bash -c "
    pgbackrest --stanza=db --no-archive-async 
      archive-push "${PGDATA}/pg_wal/00000002.history" || true
  "

What’s next

This was the safest migration path. Part 3 will cover two simpler options:

• Backup and restore. The simplest path. You take a Crunchy pgBackRest backup and the Percona cluster bootstraps from it. Cutover is the time between the final backup and pointing the application at the new cluster.
• Persistent volume reuse. For when you want to skip the data copy entirely. The Percona cluster takes over the existing PGDATA volume, no restore step required.

Pick the method that fits your downtime budget, data size, and storage layout.

This post covers basic deployment patterns and simplified configuration examples. If your environment is more complex, uses custom images, includes Crunchy enterprise features like TDE, or otherwise needs tailored migration steps, contact the Percona team and we will help you plan and execute the move.

Try It Out

• Percona Operator for PostgreSQL docs: https://docs.percona.com/percona-operator-for-postgresql/latest/
• GitHub: https://github.com/percona/percona-postgresql-operator
• Public roadmap: https://github.com/orgs/percona/projects/10/views/6
• Issue tracker: https://github.com/percona/percona-postgresql-operator/issues
• Community Forum: https://forums.percona.com/c/postgresql/percona-kubernetes-operator-for-postgresql/68

The post Migrate from Crunchy Data PostgreSQL Operator to Percona PostgreSQL Operator: Standby Cluster Method appeared first on Percona.

The post Migrate from Crunchy Data PostgreSQL Operator to Percona PostgreSQL Operator: Standby Cluster Method appeared first on MariaDB.org.

Write it up. We’ll publish it, and we’ll pay you.

What we’re doing

The Percona Community Writers Program publishes technical posts from the people actually using these tools — DBAs, developers, contributors, and engineers running real workloads. Posts go up on percona.community/blog under your name, with your bio and links.

For every post we publish, you get:

• $350 paid out after publication
• Community engagement points you can redeem in our swag store for t-shirts, stickers, and other items

The points stack across contributions. The more you write, the more you collect.

A note on payment

Not everyone can accept payment for writing — employment contracts, tax situations, visa rules, and conflict-of-interest policies all get in the way. If that’s you, we’ll donate the same $350 to an open source project or community of your choice on your behalf. Tell us who to send it to when you pitch.

What we want to read

Anything you’ve done with the Percona stack — or alongside it — that another engineer would learn from. Some directions to consider:

• Percona Operators — running databases on Kubernetes, scaling decisions, upgrade paths, what surprised you
• Percona Toolkit — how you use specific tools in your day-to-day, scripts you’ve built around them, edge cases
• Migrations — moving between versions, between database engines, on-premises to cloud, the parts that aren’t in the docs
• Troubleshooting — a real incident, what you saw, what fixed it, what you’d do differently
• Percona Monitoring and Management (PMM) — dashboards you’ve built, alerts that actually catch things, integrations
• Databases themselves — MySQL, PostgreSQL, MongoDB, MariaDB, Valkey, anything in the open source database world you’re hands-on with

We’re not only interested in Percona-product posts. If you’re active in the wider open source database community — contributing to MySQL, PostgreSQL, Valkey, or anywhere else — we want to hear about that work too. Your projects, your perspective, your hard-won opinions.

Standards

Every submission is reviewed by the community team for technical accuracy and grammar before it goes live. We’re not gatekeeping — we’re making sure your name goes on something solid.

One firm rule: no AI-generated content. We run every submission through GPTZero and it has to come back clean. We’re publishing your voice and your experience, not a model’s summary of either. If you used AI to help draft, that’s fine — but the post needs to read as yours and pass the check.

How to start

Pitch us first. A couple of sentences on what you want to write about and why you’re the person to write it is enough. We’ll reply with feedback, a timeline, and any direction that helps you write a stronger post.

You don’t need to be a published writer. You need to have done something and be willing to explain how. A 900-word post about how you debugged a replication lag issue last quarter is more valuable than a 3,000-word survey of the database landscape.

Send pitches and questions to the Percona Community team — by filling in this form.

Open topics: blog, talks, guides

Not sure where to start? Here are some directions we’d love to see covered. Pick one, narrow it down to something you’ve actually done, and pitch us.

Databases

• Automating database setup for production in under a few hours
• Backup and disaster recovery strategies that hold up
• Failure stories — what broke, what you learned

DevOps and reliability

• Database Reliability Engineering (DBRE) in practice
• Site Reliability Engineering (SRE) applied to databases
• Monitoring and SLAs that mean something
• Useful scripts you actually run in production
• Testing and QA for database changes

Distributed computing

• Consensus algorithms and real-world implementations
• Synchronous vs asynchronous replication — trade-offs and where each fits

How-tos

• Moving from a single node to a cluster (any DB engine)
• Batch processing patterns
• Stream processing patterns
• Metrics that actually tell you something

Open source

• Measuring your open source project’s success
• Bug squashing done right
• Licensing — what to know before you pick one
• Vendor lock-in and how to spot it early

We pay engineers to share what they’ve learned. That’s the whole offer. If you’ve got something worth writing, write it.

The post Write for the Percona Community appeared first on MariaDB.org.

Servers Tested:

• MySQL 9.7.0 (PGO-enabled build released by Oracle)
• MySQL 9.7.0 Non-PGO (built without Profile-Guided Optimization — see BUILD.md)

Tier Configurations:

• Tier 2G: 2GB InnoDB buffer pool
• Tier 12G: 12GB InnoDB buffer pool
• Tier 32G: 32GB InnoDB buffer pool

View Results

Interactive Reports

The benchmark reports are available as interactive HTML pages at:

https://percona-lab-results.github.io/2026-pgo/index.html

Performance Graphs

Tier 2G (2GB Buffer Pool):

Tier 12G (12GB Buffer Pool):

Tier 32G (32GB Buffer Pool):

Key Findings

Performance Impact of PGO

MySQL 9.7.0 with Profile-Guided Optimization (PGO) demonstrates measurable performance improvements over the non-PGO build:

Overall Performance Summary:

• Average improvement: 6.5% across all configurations
• Peak improvement: 14.3% (Tier 32G, 1 thread), gradually tapering to 10.3% at 512 threads as concurrency increases
• Performance gains range from 0.5% to 14.3% in most scenarios
• Minor regression (-3.1% at Tier 12G, 128 threads)

Performance by Buffer Pool Size:

• Tier 2G (2GB buffer pool): Average improvement of 3.0%

– Best gains at 4 threads (5.5% improvement)

– Gains range from 0.5% to 5.5% across all thread counts

– Modest improvements with no regressions

• Tier 12G (12GB buffer pool): Average improvement of 4.1%

– Best gains at 4 threads (8.6% improvement)

– Strong gains at low concurrency (1-4 threads: 7.3%-8.6%)

– Minor regression at 128 threads (-3.1%), neutral at 512 threads (-0.0%)

• Tier 32G (32GB buffer pool): Average improvement of 12.2%

– Consistently strong gains across all thread counts (10.3% to 14.3%)

– Peak performance at lowest concurrency (1 thread: 14.3%)

– Maintains 11-12% improvement even at highest concurrency (128-512 threads)

Key Observations:

• PGO provides the most significant benefits with larger buffer pools (32GB tier shows 12.2% average improvement)
• Largest buffer pool configuration benefits from PGO across all concurrency levels with no regressions
• Low to moderate concurrency (1-32 threads) shows best PGO gains across all tiers
• Smaller buffer pools (2GB, 12GB) show more modest improvements and occasional regressions at very high thread counts
• The performance improvements demonstrate PGO’s effectiveness in optimizing hot code paths, particularly when memory resources are abundant

InnoDB Metrics Analysis

Deep analysis of InnoDB metrics reveals the source of PGO’s performance improvements:

Root Cause: CPU-Level Optimizations

• PGO improvements are NOT from I/O optimization, caching, or lock reduction
• Buffer pool hit ratios remain virtually identical between PGO and non-PGO builds
• Lock contention is minimal in both builds
• All I/O metrics scale proportionally with increased throughput

What PGO Actually Optimizes:

• ✓ Better instruction cache utilization
• ✓ Improved branch prediction in hot code paths
• ✓ Optimized function inlining
• ✓ More efficient CPU instruction ordering

The metrics confirm that PGO’s 6.5% average improvement comes entirely from making the CPU more efficient at executing MySQL’s hot code paths, allowing it to process more transactions per second with the same hardware resources.

What is PGO?

Profile-Guided Optimization (PGO) is a compiler optimization technique that uses runtime profiling data to guide code optimization. The compiler first instruments the code, collects execution profiles during typical workload runs, and then recompiles the code with optimizations targeted at the most frequently executed code paths.

Benefits of PGO:

• Improved branch prediction
• Better instruction cache utilization
• Optimized function inlining
• Reduced code bloat
• Better register allocation

Benchmark Methodology

Workload

• Tool: Sysbench OLTP Read/Write benchmark
• Tables: 20 tables
• Table Size: 5,000,000 rows per table
• Thread Counts: 1, 4, 16, 32, 64, 128, 256, 512

Configuration

• Warmup:

– Read-only: 180 seconds

– Read-write: 600 seconds

• Measurement Duration: 900 seconds (15 minutes) per thread count
• Runs: Single run per configuration

System Metrics Collected

• InnoDB storage engine metrics
• MySQL status variables
• MySQL system variables
• System I/O statistics (iostat)
• Virtual memory statistics (vmstat)
• CPU statistics (mpstat)
• System statistics (dstat)

Appendix

For Repository structure, Build steps and Technical details go to https://github.com/Percona-Lab-results/2026-pgo/blob/main/README.md#report-categories

The post MySQL 9.7.0 PGO Benchmark Analysis appeared first on Percona.

The post MySQL 9.7.0 PGO Benchmark Analysis appeared first on MariaDB.org.

Today, we are publishing version currency timelines for Aurora and RDS open source engines. The timelines apply to new major and minor versions going forward and define when you and your teams can expect new versions on AWS. With this predictability, you can plan maintenance windows, upgrade cycles, and Aurora LTS adoption for workloads that prioritize long-term stability.

For the current schedule of upcoming and recently shipped versions, including specific version numbers and target dates, see the release calendar linked from each engine name in the table.

Why timelines differ across engines

The timelines differ by engine because the upstream development and integration models differ. PostgreSQL and MariaDB communities develop in the open, which lets us start validation early. MySQL commits are available closer to public releases. RDS runs the community engine on managed infrastructure, so after a community release passes validation it can ship quickly. Aurora adds a distributed storage layer, Global Database, and serverless capabilities underneath PostgreSQL- and MySQL-compatible engines. Every new version goes through additional validation to verify that those capabilities continue to function correctly. This is why Aurora timelines are longer than RDS timelines for the same engine. Aurora also offers Long-Term Support releases for multi-year stability on a single minor version.

How we choose major version starting points

For PostgreSQL, our first production release of a new major version is typically based on the community <major>.1 release rather than <major>.0. The <major>.1 release generally arrives roughly three months after the initial major release. It incorporates the first round of bug fixes and security patches identified during early production deployments, which provides a more stable starting point for production workloads.

MariaDB follows a similar pattern. The published major version timelines are measured from the community’s first patch release for a new major version rather than the initial <major>.0 release. This gives customers a more mature production baseline to target.

For MySQL, the major version timelines apply to Oracle MySQL LTS major releases and are measured from the corresponding <major>.1 release. This aligns the timelines to the first patch release after the initial LTS major becomes generally available.

What this means for your upgrade planning

Published version currency timelines give you and your teams earlier visibility into release planning and upgrade scheduling. With RDS for PostgreSQL minor versions arriving within 7 days of community release, teams can stay current on security patches and bug fixes with relatively little operational planning. You can enable automatic minor version upgrades to receive patches during maintenance windows. You can also use AWS Organizations upgrade rollout policies to manage deployment sequencing across your development, test, and production environments, or apply upgrades manually based on your own operational processes.

Earlier visibility into major version timelines helps your teams adopt new database capabilities on your own schedule. Knowing when a new version is expected on Aurora or RDS gives teams more time to review release notes, validate application behavior, and prepare rollout plans ahead of adoption. With RDS Database Preview, you get early access to PostgreSQL and MySQL major versions in a non-production environment so you can test application compatibility in advance. With Blue/Green Deployments, you can validate changes before cutover and transition production traffic with minimal downtime.

With Aurora LTS releases, you can prioritize operational stability over rapid feature adoption. Your teams can remain on a stable minor baseline for multiple years while aligning major version upgrades with broader application and infrastructure roadmaps.

For workloads approaching or beyond community end-of-life timelines, Amazon RDS Extended Support gives you additional time to finish upgrades while continuing to receive critical security updates.

Learn more

For detailed upgrade procedures and release guidance, see the Amazon Aurora User Guide and the Amazon RDS User Guide.

About the authors

The post Knowing when new open source database engine versions release on Amazon Aurora and Amazon RDS appeared first on MariaDB.org.

The latest ClusterControl MCP is the major update, providing a more robust implementation with 69 tools and 20 MCP resources / templates for production database operations across ClusterControl-managed environments. CCX MCP is the companion MCP server for CCX, bringing AI-assisted workflows to managed cloud database operations.

Together, they give Severalnines users a practical way to inspect, troubleshoot, and act on database infrastructure from MCP-compatible clients such as Claude Desktop, Claude Code, OpenAI Codex, and other tools that support MCP.

What is new in ClusterControl MCP?

ClusterControl MCP 1.0 moves beyond the earlier MCP concept and provides broader coverage across daily database operations. You can ask questions such as:

• “List all my database clusters and their status.”
• “Show me the topology of cluster 2.”
• “Are there any active alarms across all clusters?”
• “What backup jobs have run on cluster 3?”
• “Show me the top queries by wait time on cluster 1.”
• “Are there any tables without primary keys?”
• “Show me recent transaction deadlocks.”
• “List the log files collected from cluster 1.”
• “Who made changes to cluster 3 in the last hour?”

You can also prepare actions such as:

• “Run a backup on cluster 1 right now.”
• “Create a nightly backup schedule at 02:00.”
• “Put db1.example.com into maintenance from 22:00 to 23:00 UTC.”
• “Create a read-only database user for reporting.”
• “Set max_connections to 500 on db1.example.com.”
• “Restore backup #42 to cluster 1.”

Write operations use a dry-run-first model. The assistant describes what would happen before anything is executed, and high-risk operations include extra warnings.

Example: move from alarm to evidence faster

A common operational flow starts with a broad question:

“Are there any active alarms across all clusters?”

From there, you can drill down:

• “Show me alarms for cluster 3.”
• “Show me the CMON log for cluster 3 from the last hour.”
• “Summarize the warnings by component and hostname.”

That is where the 1.0 implementation becomes useful. It is not just returning a static dashboard view. It can help you move across related operational data: alarms, jobs, CMON controller logs, database server logs, topology, backup history, maintenance windows, and audit events.

Example: inspect and manage backups conversationally

Backups are another area where ClusterControl MCP 1.0 adds practical coverage. You can ask:

• “When was the last successful backup on my MongoDB cluster?”
• “Show me only failed backups on cluster 1.”
• “Does cluster 1 have a backup schedule configured?”

And then prepare a change:

“Create a nightly backup schedule at 02:00 on cluster 1 using xtrabackup.”

The assistant first returns a dry-run preview. Only after confirmation does it execute the change.

Installing ClusterControl MCP

ClusterControl MCP packages are published through the Severalnines repository alongside other ClusterControl components.

Debian / Ubuntu:

apt-get install clustercontrol-mcp

RHEL / Rocky / AlmaLinux:

yum install clustercontrol-mcp

The binary installs to:

/usr/bin/cmon-mcp

The package also installs:

/etc/systemd/system/cmon-mcp.service
/etc/default/cmon-mcp

Setting up ClusterControl MCP in stdio mode

First, we’ll start with stdio mode for when the AI client runs the MCP server locally, such as Claude Desktop or Claude Code.

Claude Desktop configuration:

{
 "mcpServers": {
   "clustercontrol": {
     "command": "cmon-mcp",
     "env": {
       "CMON_ENDPOINT": "https://your-cc-host:9501",
       "CMON_USERNAME": "admin",
       "CMON_PASSWORD": "your-password"
     }
   }
 }
}

Restart Claude Desktop. The hammer icon confirms that the MCP server loaded.

For Claude Code:

claude mcp add clustercontrol -- cmon-mcp 
 -endpoint https://your-cc-host:9501 
 -username admin 
 -password your-password

Setting up ClusterControl MCP in HTTP mode

Use HTTP mode for OpenAI Codex, team access, or multi-client access. Edit:

/etc/default/cmon-mcp

Example:

CMON_ENDPOINT=https://127.0.0.1:9501
CMON_USERNAME=admin
CMON_KEY_FILE=/etc/clustercontrol/id_rsa

MCP_BIND_ADDRESS=0.0.0.0:3000
MCP_BASE_URL=http://your-cc-host:3000
MCP_AUTH_TOKEN=<your-strong-random-token>

Generate a strong token:

openssl rand -hex 32

Restart the service:

systemctl restart cmon-mcp
journalctl -u cmon-mcp -n 20

Connect OpenAI Codex:

codex --mcp-server-uri http://your-cc-host:3000/mcp 
     --mcp-header "Authorization: Bearer <your-token>"

Connect Claude Code over SSE:

claude mcp add clustercontrol --transport sse http://your-cc-host:3000/sse 
 --header "Authorization: Bearer <your-token>"

Connect Claude Desktop over SSE:

{
 "mcpServers": {
   "clustercontrol": {
     "type": "sse",
     "url": "http://your-cc-host:3000/sse",
     "headers": {
       "Authorization": "Bearer <your-token>"
     }
   }
 }
}

CCX MCP: AI-Assisted Workflows for CCX

As noted upfront, CCX MCP brings the MCP-based workflow to Severalnines users running managed cloud databases in CCX. It lets MCP-compatible AI clients interact with CCX datastores, cloud providers, plans, databases, users, firewall rules, backups, parameter groups, and performance data.

Typical prompts include:

• “List my datastores.”
• “Create a PostgreSQL cluster.”
• “Get the connection string for my production database.”
• “Add 10.0.0.0/24 as a trusted source.”
• “Show me the slowest queries.”
• “List available backups for this datastore.”

CCX MCP supports PostgreSQL, MySQL / Percona, MariaDB, Redis, Valkey, and Microsoft SQL Server. It also includes protection behavior for destructive operations, which are blocked by default unless protection is explicitly disabled.

Installing and setting up CCX MCP

CCX MCP can be installed from npm:

npm install @severalnines/ccx-mcp

Or used directly through npx in your MCP client configuration:

{
 "mcpServers": {
   "ccx": {
     "command": "npx",
     "args": ["-y", "@severalnines/ccx-mcp"],
     "env": {
       "CCX_BASE_URL": "https://app.myccx.io",
       "CCX_USERNAME": "your-email@example.com",
       "CCX_PASSWORD": "your-password"
     }
   }
 }
}

OAuth2 is also supported:

{
 "CCX_CLIENT_ID": "your-client-id",
 "CCX_CLIENT_SECRET": "your-client-secret"
}

Claude Code

For Claude Code, register the CCX MCP server in one command. No manual config file editing is required:

claude mcp add ccx -- npx -y @severalnines/ccx-mcp@latest 
 --endpoint https://app.myccx.io 
 --client-id <your-client-id> 
 --client-secret <your-client-secret>

Create OAuth2 credentials in the CCX UI under Account > Security.

Then restart Claude Code, or run /mcp and reconnect. After that, you are ready to start using CCX MCP from your Claude Code session.

Wrapping up

The original ClusterControl MCP work showed how AI assistants could become useful in database operations when connected to the right operational context. The latest version makes that idea much more complete, providing a broader and safer operational interface across clusters, jobs, alarms, backups, logs, performance, users, maintenance, audit, and configuration. Go here for more information on how it works and detailed documentation.

For CCX and service provider users, CCX MCP provides the companion interface for managed cloud database operations. Together, they give database teams a practical way to use AI where it matters: inside real operational workflows, with context, control, and safety.

The post AI-Assisted Production Database Ops with ClusterControl MCP and CCX MCP appeared first on Severalnines.

The post AI-Assisted Production Database Ops with ClusterControl MCP and CCX MCP appeared first on MariaDB.org.

Continue reading “Vibe-coding an Audit Plugin in Under 3 Minutes”

The post Vibe-coding an Audit Plugin in Under 3 Minutes appeared first on MariaDB.org.

The post Vibe-coding an Audit Plugin in Under 3 Minutes appeared first on MariaDB.org.

Source

The post Stop Paying for Air: Most Cloud Database Spend Is Wasted appeared first on MariaDB.org.

Every time a bug like this lands, the same conversation plays out in incident channels across the industry. Are we affected? We don’t even use time-series collections! Heads nod. Everyone moves on.

That’s the mistake.

CVE-2026-8053 is an out-of-bounds memory write in MongoDB’s time-series collection — specifically in the internal mapping between measurement field names and column indexes. Under the right input, the mapping drifts out of sync with the underlying buffer and mongod writes off the end of an allocation. From there, under the right conditions, you can execute arbitrary code as the database process.

Upstream tracking lives at SERVER-126021. CVSS v3.1 puts it at 8.8. CVSS v4.0 puts it at 8.7. The labels say “High.” How that “High” translates into your week depends on a couple of assumptions worth questioning.

Read literally, the prerequisite is “an authenticated user with database write privileges.” Read operationally, that bar is lower than most teams treat it as.

The mitigation you think you have doesn’t exist

Modern stacks have dozens of service accounts, with secrets scattered across config files, pipelines, and laptops you’ve long forgotten about. Others end up in log files on bad days. And every user with write access to your cluster sits one step away from the vulnerable code path. In a world like that, “the attacker would need credentials first” isn’t a speed bump — it’s a shrug.

So the real question was never authenticated vs. unauthenticated. It’s what authentication unlocks. Here, it unlocks Remote Code Execution (RCE), which is exactly what the CVSS score is trying to tell you — even if the industry’s reaction hasn’t quite caught up. Attackers don’t need your time-series collection to already exist – they just need someone’s credentials in the wrong hands, and there are more ways for that to happen than most teams want to admit.

I’m not raising this to be smug. I’m raising it because too many incident channels keep stalling on the wrong question. It isn’t: “Does our app use time-series?” It’s: “What can a user holding our readWrite role actually do this week?”

Until you patch, the answer is more than you think.

What Percona is shipping, and when?

• Percona Server for MongoDB 7.0.34-19 — May 20, 2026
• Percona Server for MongoDB 8.0.23-10 — May 21, 2026
• Percona Server for MongoDB 6.0.28-22 — May 25, 2026

6.0 is on the End-Of-Life (EOL) track. The easy call would be to point at the lifecycle page, note that the upgrade conversation is overdue, and stop there. We’re shipping the fix anyway. Customers running 6.0 in production have real reasons they haven’t migrated yet — frozen application stacks, certification cycles, dependencies that don’t move on quarterly cadences — and none of those reasons are worth exploiting while a migration plan gets approved.

Percona is not building binary packages for the 5.x line. We’re being upfront about that — the calculus on extended support has a limit, and 5.x is past it for us. But the fix itself is already in our public release branch: release-5.0.33-26. If you have a hard requirement on 5.x and the time pressure to meet it, the source is available for building. Percona customers on 5.x can open a ticket, and we’ll work on the case individually.

What to do this week?

Patch! Specifically:

• If you’re on 7.0, upgrade to 7.0.34-19 from May 20 onward.
• If you’re on 8.0, upgrade to 8.0.23-10 from May 21 onward.
• If you’re on 6.0, upgrade to 6.0.28-22 from May 25 onward.
• If you’re on 5.0 and you can’t move, build from release-5.0.33-26. Customers — open a ticket and we’ll help.

As usual, you can download patches from your package manager or Percona Software Downloads page.

If you’re running PSMDB on Kubernetes via the Percona Operator for MongoDB, edit the image tag in your PerconaServerMongoDB custom resource and let the operator roll the cluster. Don’t wait for the June operator release to do it for you. See details in our documentation on how to Upgrade Percona Server for MongoDB.

While you’re in there, audit your custom roles. Anything granting createCollection on a production database is, today, an RCE primitive in waiting. Decide whether the service accounts that hold it actually need it. Decide whether your application users need full readWrite or whether a narrower role would do the same job. Treat the answer as part of your security posture, not as a quarterly cleanup task you’ll get to.

Questions, sharp disagreements, or a 5.x build that won’t compile? Find us on the Percona Forum or, if you’re a customer, in your support portal. If you want to become one and ensure your databases run, check out Percona Services.

The post CVE-2026-8053: “We don’t use time-series” is not a mitigation appeared first on Percona.

The post CVE-2026-8053: “We don’t use time-series” is not a mitigation appeared first on MariaDB.org.

Choosing an open source PostgreSQL operator for Kubernetes used to be a question about features and community size. In 2026, it has become a question about licensing posture, image distribution, and whether the project you pick today will still be operationally open in three years.

This is part 1 of a 3-part series on running PostgreSQL on Kubernetes with a fully open-source operator.

• Part 1 (this post): how the open-source landscape has shifted under your feet, and what to look for in an operator before you commit
• Part 2: migrating from the Crunchy Data PostgreSQL Operator to the Percona PostgreSQL Operator using the standby cluster method (near-zero downtime)
• Part 3: two simpler migration paths: backup-and-restore and persistent-volume reuse

In this post, you will learn about:

• What has changed in the open-source landscape over the last few years, with specific examples
• What licensing and redistribution actually mean for Kubernetes operators in production
• How to evaluate whether a project is “open source in theory” or open source in practice
• Where Percona’s PostgreSQL Operator fits in, and what the practical migration looks like

Open source isn’t what it used to be
The landscape of open source has undergone significant changes in recent years, and selecting the right operator and tooling for PostgreSQL clusters in Kubernetes has never been more important. Three recent shifts illustrate the pattern.

MinIO

MinIO was the default open-source S3-compatible storage backend for Kubernetes workloads for years. The trajectory over the last few years tells the story:

• Switched its license to AGPLv3, with several enterprise features moved into a commercial-only edition
• Entered what amounted to maintenance mode, narrowing community engagement, limiting support to paid subscriptions, and reducing acceptance of community contributions
• On April 25, 2026, the github.com/minio/minio repository was archived by the project owner, ending public development of the open-source version

The code is still cloneable, but the project is no longer maintained as open source. Teams running MinIO in production now need an exit plan.

Bitnami images

Bitnami Docker images have long been a staple for databases (including Postgres), middleware, and developer tooling. In July 2025, Broadcom’s Tanzu Division announced Bitnami Secure Images and signalled the deprecation of the free public catalog. The concrete timeline that followed:

• August 28, 2025: deprecation of non-hardened Debian-based images in the free tier began, and non-latest images started to be removed
• September 29, 2025 (after community pushback): the public docker.io/bitnami catalog was reduced. The remaining free images were limited to a small curated set of latest-version, hardened images intended for development use; older versions of most applications were moved to a “Bitnami Legacy” repository
• The full catalog and the hardened production images now require a paid Bitnami Secure Images subscription, reportedly priced in the tens of thousands of dollars per year per organization

For Kubernetes teams, the practical impact was immediate: any Helm chart that pinned a specific Bitnami image version (a recommended practice) found that image gone or moved, breaking CI pipelines and air-gapped deployments.

Crunchy Data PostgreSQL images

Crunchy Data illustrates the same dynamic in the Postgres operator space. To be clear: the Crunchy Data PostgreSQL Operator is a mature, well-engineered project, and the team behind it has done a lot of valuable work upstream and around pgBackRest and Patroni integrations. The point of this section is not the engineering, it is the redistribution and usage terms that govern the official builds.

Crunchy’s licensing shifts, 2022 to 2024

Between 2022 and 2024, several shifts occurred:

• Redistribution restrictions. While the PostgreSQL code is open source, Crunchy’s official Docker images include branding and enterprise features that are not freely redistributable. The Crunchy Data Developer Program terms describe the software as intended for internal or personal use; production use by larger organizations typically requires an active support subscription.
• Restrictions on consulting and resale. The terms explicitly prohibit using Crunchy’s images to deliver support or consulting services to others without an authorized agreement. The PostgreSQL source code remains open source, but the official images and their packaging are not freely redistributable, which limits practical use in commercial and customer-facing settings.
• Registry move. Most images were moved to registry.developers.crunchydata.com, which requires authentication and acceptance of terms before pulling. That draws a clearer line between open-source code and proprietary builds.

In other words, the project is open source on the code side, but the practical artifacts (images, Helm releases) are gated.

What these restrictions really mean for Kubernetes users

When container images and operators come with redistribution limits, authentication requirements, or “internal-use-only” clauses, the impact on Kubernetes environments is immediate and concrete. Teams can no longer:

• Build air-gapped clusters by mirroring images to a private registry without working through a license review
• Rely on GitOps workflows that assume publicly accessible OCI images
• Fork or customize the operator freely, because official images cannot be redistributed with modifications
• Use the software in commercial or customer-facing products without additional licensing
• Run multi-cluster or multi-tenant Postgres at scale without bumping into usage terms

For a database operator, where almost every operational pattern depends on the container images you can pull and run, these restrictions effectively turn a project into a “source-available but not operationally open” solution. The code is open. The operating story is not.
As a result, many teams are switching to fully open-source alternatives: the Percona Operator for PostgreSQL, CloudNativePG, Zalando Postgres Operator, StackGres, and a few others.

How to evaluate “open source” in 2026

The bigger picture here is that “open source” today often exists more in theory than in practice. It pays to look past the badge and check the operating reality. Three questions to ask before you commit to an operator:

1. Are the container images publicly redistributable?

If you cannot pull the official images without authentication, or you cannot mirror them to your private registry without a license review, your air-gapped and GitOps stories are constrained from day one. This is the question that turned out to be the most consequential one for MinIO, Bitnami, and Crunchy users in 2025.

2. Are core operational features in the open-source build, or behind a paywall?

Backup, monitoring, HA, and security features should be in the build everyone uses, not gated behind an enterprise tier. A “community edition” that omits the feature most teams actually need is a marketing build, not a real open-source build.

3. Is the governance and roadmap public?

A project where you can see the issues, the PRs, and the roadmap is one you can plan around. The Percona PG Operator’s public roadmap is an example of what this looks like in practice. A project run inside a vendor’s private tracker, by contrast, gives you no visibility.
These are not gotchas. They are the questions that decide whether a project will still serve you the same way in three years.

Migrate to freedom

Announcing the hard fork

We strongly believe in fully open-source software and want to increase our investment in the PostgreSQL and Kubernetes ecosystems. To back that up, we have decided to hard fork the Crunchy Data PostgreSQL Kubernetes Operator. Starting from version 3.0.0 (coming soon), the Percona PostgreSQL Kubernetes Operator is a fully independent project, with a public roadmap, public issue tracker, and freely redistributable images.

The hard fork is not a critique of Crunchy’s engineering. It is a commitment that the operator will keep evolving in a fully open-source direction, with no surprises about which features will be available to which audience.

Why migration is straightforward

Because the Percona PostgreSQL Operator is a hard fork of the Crunchy operator, the migration paths are surprisingly straightforward. The same underlying tools (Patroni, pgBackRest, PgBouncer) and the same overall design are used in both, which means migration can be done in multiple ways, sometimes with near-zero downtime, sometimes faster with a small downtime window. The next two posts in this series walk through three concrete options.

What’s next

This was the “why.” The next two posts are the “how”:

• Part 2: Standby cluster migration. Bring up a Percona cluster as a standby of the Crunchy primary, catch it up via pgBackRest plus streaming replication, and promote it at cutover. The only downtime is the cutover itself.
• Part 3: Backup-restore and PV reuse. Two simpler paths: bootstrap a Percona cluster directly from a Crunchy pgBackRest backup, or retain the existing PGDATA persistent volume and have Percona pick up where Crunchy left off.

Reversibility and exit options

All three paths are reversible: because Percona’s operator, images, and tooling are 100 percent open source and remain compatible with the same backup format and the same Patroni HA model, you keep full control. You can migrate back to Crunchy if your team decides to, or out to another open-source operator (CloudNativePG, Zalando, StackGres) using the same patterns. That last journey is a topic for a future article.

This series covers basic deployment patterns and simplified configuration examples. If your environment is more complex, uses custom images, includes Crunchy enterprise features like TDE, or otherwise needs tailored migration steps, contact the Percona team and we will help you plan and execute the move.

Try It Out

• Percona Operator for PostgreSQL docs: https://docs.percona.com/percona-operator-for-postgresql/latest/
• GitHub: https://github.com/percona/percona-postgresql-operator
• Public roadmap: https://github.com/orgs/percona/projects/10/views/6
• Issue tracker: https://github.com/percona/percona-postgresql-operator/issues
• Community Forum: https://forums.percona.com/c/postgresql/percona-kubernetes-operator-for-postgresql/68

The post Not All Open Source Is Equal: Choosing a PostgreSQL Operator for Kubernetes in 2026 appeared first on Percona.

The post Not All Open Source Is Equal: Choosing a PostgreSQL Operator for Kubernetes in 2026 appeared first on MariaDB.org.

For many in the PostgreSQL ecosystem, this landed like a shock. pgBackRest is one of the most widely used backup and recovery tools for PostgreSQL, deeply embedded in production environments across enterprises large and small. Now it was suddenly described as “dead”, “EOL”, or “abandoned”. The trigger was clear: its long-time maintainer, after more than a decade of work, announced he could no longer continue without sustainable funding and would archive the repository.
i
That message spread fast. The interpretation spread even faster.

And it was wrong.

This wasn’t EOL

Open source software doesn’t simply “go end of life” in the way proprietary software does. There is no vendor switch flipped to OFF. No license revoked. No binaries disappearing overnight.

What actually happens is more subtle and more important:

• Maintainers step away
• Funding runs out
• Work stops

That’s not EOL. That’s a sustainability gap.

pgBackRest didn’t die. It hit a problem seen too often in open source world: a critical piece of infrastructure maintained by fewer and fewer people, until it ultimately depended on one person being able to justify working on it full time.

The real problem

The message from the maintainer was not about abandoning the project. It was about reality:

maintaining a widely used tool requires time, and time requires funding

For years, pgBackRest was supported through corporate sponsorship from mainly one vendor. When that disappeared due to the Crunchy Data acquisition, so did the ability to keep investing the same level of effort.

This is the “Nebraska guy problem” in action: software used by a large part of the industry, sustained by a very small number of people.

Yes, anyone can fork the project (and some already did), but:

• trust doesn’t fork
• community doesn’t fork
• sustainability definitely doesn’t fork

A fork without coordination creates fragmentation without adding real value and that weakens the ecosystem. What pgBackRest needed was not a replacement, but continuity.

The danger of bad framing

Calling the project “dead” shifted the conversation in the wrong direction.

Instead of asking:

how do we keep this project healthy?

the discussion drifted at best toward:

what is the strategic solution here?

and more often to:

what do we replace it with?

and

what do we name our fork?

That’s a natural reaction, but it’s not a good one.

Critical infrastructure should not be treated as disposable. Doing so erodes trust in the solutions we rely on and weakens the ecosystem. These foundational pieces should be treated as a shared responsibility so that the entire community becomes stronger.

What happened next

Behind the scenes, things moved quickly, with coordination between David and companies active in the PostgreSQL community.

Conversations started across companies, contributors and the wider ecosystem. The goal wasn’t to “rescue” pgBackRest, but to do something far more valuable: to restore a sustainable model around it.

This is what open source actually requires: not heroics, but coordination.

So what’s with pgBackRest?

It’s all good. Well, better.

The short version:

• Multiple companies coordinated together to ensure continued funding and support around pgBackRest
• Engineering effort is now being shared more broadly to expand the contributor and maintainer base
• Discussions around longer term sustainability and governance in the PostgreSQL ecosystem accelerated significantly
• Percona played an active role in coordinating these efforts, contributing engineering resources, and helping bring organizations together around a sustainable path forward

The project was never closed.

The way (forward) is open

pgBackRest’s situation is not unique. It’s a signal.

The PostgreSQL ecosystem depends on a wide range of tools that don’t have the same visibility, or funding, as the database itself. That gap is becoming harder to ignore.

There’s growing alignment on a few things:

• sustainability needs to be intentional
• funding needs to be easier to organize
• engineering effort needs to be shared

Whether that leads to an umbrella foundation or another model, one thing is clear: the ecosystem needs structures that support both users and maintainers.

The post Backrest’s back, alright! appeared first on MariaDB.org.

Source

The post MariaDB Community Server Q2 2026 maintenance releases appeared first on MariaDB.org.

Continue reading “Introducing Our First MariaDB Server Solution Stack: A Privacy-First Stack with Nextcloud, Passbolt, and MariaDB”

The post Introducing Our First MariaDB Server Solution Stack: A Privacy-First Stack with Nextcloud, Passbolt, and MariaDB appeared first on MariaDB.org.

The post Introducing Our First MariaDB Server Solution Stack: A Privacy-First Stack with Nextcloud, Passbolt, and MariaDB appeared first on MariaDB.org.

Source

The post MariaDB Community Server 10.6 Is Reaching End of Life – Here’s What to Do Next appeared first on MariaDB.org.

Continue reading “Documented: The MariaDB Server (Community) Contribution Process”

The post Documented: The MariaDB Server (Community) Contribution Process appeared first on MariaDB.org.

The post Documented: The MariaDB Server (Community) Contribution Process appeared first on MariaDB.org.

If there was one massive takeaway from the day, it was this: We are officially in the Era of AI and “Agentic” Systems. During the event, the message was clear: we are shifting from basic LLMs (that just answer a prompt and forget it) to autonomous AI Agents that follow a continuous loop of Perception → Planning → Action. MongoDB’s President and CEO, CJ Desai, repeated a powerful phrase:

While AI models change rapidly, the Data Layer is the constant.

Here is a look at our day, what we learned, and the fun we had along the way!

Arriving Early and Chasing Badges

We got a great tip before the event: arrive early to get a head start on the gamified learning! MongoDB had a super nice setup where you could take tests on Credly to earn knowledge badges.

We jumped right in. I got a MongoDB Overview badge, but Keith was on a mission. He completed three different tests (including MongoDB for Developers) and unlocked some cool swag: a really cute, high-quality bag! It was a brilliant way to get attendees engaged right from the morning.

Here are more badges in case you want to get yours!

General Session Highlights

We spent a lot of our time in the main room for the General Session, and the announcements were packed with impressive numbers and tech:

• MongoDB 8.3 is Fast: Osmar Olivo (Senior Director, Database Product Management) shared that the new version brings up to 35% more write throughput, 45% more read throughput, and 15% more for ACID transactions.
• The Scale is Real: We learned that Stripe uses MongoDB to process over $1 trillion in payments volume every year (maintaining 5 nines of availability!). Osmar framed this perfectly: that is 1.5% of the global GDP running through MongoDB.
• LangGraph.js Store Integration: This was a big one for developers. MongoDB is positioning itself as the “memory hard drive” for AI agents. By supporting JavaScript and TypeScript, they are making it super easy for companies to use their existing web developers to build complex AI workflows.
• Hugging Face Partnership: They are scaling with MongoDB Atlas to support over 3 million models, officially tying themselves to the “GitHub for AI.”

Feel free to explore the recorded sessions for more: MongoDB.local London 2026

Guest Speakers

Ulku Rowe (CIO, Commercial Business at Lloyds Banking Group) talked about this being the “Decade of AI.” Lloyds is actively upskilling their current engineers through an internal “AI Academy” built in partnership with Cambridge University! She emphasized that as they build out this infrastructure, partnerships are absolutely critical to their success.

We also heard from Alex Holt from ElevenLabs, a company focused on producing the absolute best, human-sounding voice AI. Their scale is wild: they have 40 million agents running and hit $500 million in Annual Recurring Revenue in just 3 years! Alex mentioned that because many enterprises don’t know how to build agents yet, ElevenLabs uses “forward deployed engineers” to sit directly with customers to build, deploy, and prove the ROI of their voice agents.

The Hands-on Workshop and Our Lunch “Diet”

Later in the day, we attended a hands-on workshop: Designing Memory Systems for AI Agents, hands-on workshop about how AI agents can remember information and use it later to give better responses. We used Python and MongoDB Atlas to build memory into an AI agent and learned how to store, search, update, and manage that memory.
The setup was good, everything was prepared in advance so we could focus on executing the commands and truly understanding the concepts. At the end, we answered some questions and earned another badge!

However, the workshop ran until 1:00 PM. One of our friends had warned us to “go for food fast,” but we were too focused on the workshop! By the time we made it to the lunch area, all the main food was completely sold out.

How did we survive? Chips, candies, and a lot of beverages. Between the sodas, coffee, and tea, we kept our energy, but it was definitely a funny learning experience for next time! I can imagine Keith arriving home for dinner!!

Exploring the Sponsor Hall (And Doing a Podcast!)

We spent our afternoon speaking with sponsors and even got to participate in a quick podcast focusing on AI and how Atlas is being used as a strong platform for these projects!

The person being interviewed was Bikram Das, who is Chief Data Architect at Tata Consulting Services, and we had a great chat with him. TCS and MongoDB have partnered on a super impressive real-time payment and fraud detection platform. They use autonomous AI agents to instantly assess risk, investigate anomalies, and route safe transactions to networks like Visa and SWIFT without any downtime.

We also talked with IBM folks; they showed us their “plug-and-play” enterprise AI foundation. They are focused on letting large companies safely deploy AI agents without having to completely rip out and rebuild their current data infrastructure.

We also stopped by the Accenture booth! They are actively working on integrating AI directly into their platforms so they can offer smarter, more advanced solutions to their customers.

Wrapping Up!

To cap off a great day, MongoDB had one last treat. If you took less than 3 minutes to fill out the end-of-event survey, they handed you a super nice pair of socks. (We love community ideas like this!).

Overall, MongoDB.local London was a great experience. It was a nice space to learn, connect, have hands-on experience, and see exactly how the database world is evolving to meet the Agentic AI era head-on.

See you at the next event!

The post Our Experience at MongoDB.local London 2026: The Era of AI Agents, Badges, and… Surviving on Chips! appeared first on MariaDB.org.

If there was one massive takeaway from the day, it was this: We are officially in the Era of AI and “Agentic” Systems. During the event, the message was clear: we are shifting from basic LLMs (that just answer a prompt and forget it) to autonomous AI Agents that follow a continuous loop of Perception → Planning → Action. MongoDB’s President and CEO, CJ Desai, repeated a powerful phrase:

While AI models change rapidly, the Data Layer is the constant.

Here is a look at our day, what we learned, and the fun we had along the way!

Arriving Early and Chasing Badges

We got a great tip before the event: arrive early to get a head start on the gamified learning! MongoDB had a super nice setup where you could take tests on Credly to earn knowledge badges.

We jumped right in. I got a MongoDB Overview badge, but Keith was on a mission. He completed three different tests (including MongoDB for Developers) and unlocked some cool swag: a really cute, high-quality bag! It was a brilliant way to get attendees engaged right from the morning.

Here are more badges in case you want to get yours!

General Session Highlights

We spent a lot of our time in the main room for the General Session, and the announcements were packed with impressive numbers and tech:

• MongoDB 8.3 is Fast: Osmar Olivo (Senior Director, Database Product Management) shared that the new version brings up to 35% more write throughput, 45% more read throughput, and 15% more for ACID transactions.
• The Scale is Real: We learned that Stripe uses MongoDB to process over $1 trillion in payments volume every year (maintaining 5 nines of availability!). Osmar framed this perfectly: that is 1.5% of the global GDP running through MongoDB.
• LangGraph.js Store Integration: This was a big one for developers. MongoDB is positioning itself as the “memory hard drive” for AI agents. By supporting JavaScript and TypeScript, they are making it super easy for companies to use their existing web developers to build complex AI workflows.
• Hugging Face Partnership: They are scaling with MongoDB Atlas to support over 3 million models, officially tying themselves to the “GitHub for AI.”

Feel free to explore the recorded sessions for more: MongoDB.local London 2026

Guest Speakers

Ulku Rowe (CIO, Commercial Business at Lloyds Banking Group) talked about this being the “Decade of AI.” Lloyds is actively upskilling their current engineers through an internal “AI Academy” built in partnership with Cambridge University! She emphasized that as they build out this infrastructure, partnerships are absolutely critical to their success.

We also heard from Alex Holt from ElevenLabs, a company focused on producing the absolute best, human-sounding voice AI. Their scale is wild: they have 40 million agents running and hit $500 million in Annual Recurring Revenue in just 3 years! Alex mentioned that because many enterprises don’t know how to build agents yet, ElevenLabs uses “forward deployed engineers” to sit directly with customers to build, deploy, and prove the ROI of their voice agents.

The Hands-on Workshop and Our Lunch “Diet”

Later in the day, we attended a hands-on workshop: Designing Memory Systems for AI Agents, hands-on workshop about how AI agents can remember information and use it later to give better responses. We used Python and MongoDB Atlas to build memory into an AI agent and learned how to store, search, update, and manage that memory.
The setup was good, everything was prepared in advance so we could focus on executing the commands and truly understanding the concepts. At the end, we answered some questions and earned another badge!

However, the workshop ran until 1:00 PM. One of our friends had warned us to “go for food fast,” but we were too focused on the workshop! By the time we made it to the lunch area, all the main food was completely sold out.

How did we survive? Chips, candies, and a lot of beverages. Between the sodas, coffee, and tea, we kept our energy, but it was definitely a funny learning experience for next time! I can imagine Keith arriving home for dinner!!

Exploring the Sponsor Hall (And Doing a Podcast!)

We spent our afternoon speaking with sponsors and even got to participate in a quick podcast focusing on AI and how Atlas is being used as a strong platform for these projects!

The person being interviewed was Bikram Das, who is Chief Data Architect at Tata Consulting Services, and we had a great chat with him. TCS and MongoDB have partnered on a super impressive real-time payment and fraud detection platform. They use autonomous AI agents to instantly assess risk, investigate anomalies, and route safe transactions to networks like Visa and SWIFT without any downtime.

We also talked with IBM folks; they showed us their “plug-and-play” enterprise AI foundation. They are focused on letting large companies safely deploy AI agents without having to completely rip out and rebuild their current data infrastructure.

We also stopped by the Accenture booth! They are actively working on integrating AI directly into their platforms so they can offer smarter, more advanced solutions to their customers.

Wrapping Up!

To cap off a great day, MongoDB had one last treat. If you took less than 3 minutes to fill out the end-of-event survey, they handed you a super nice pair of socks. (We love community ideas like this!).

Overall, MongoDB.local London was a great experience. It was a nice space to learn, connect, have hands-on experience, and see exactly how the database world is evolving to meet the Agentic AI era head-on.

See you at the next event!

The post Our Experience at MongoDB.local London 2026: The Era of AI Agents, Badges, and Surviving on Chips! appeared first on MariaDB.org.

We strive to increase adoption by users and across use cases, platforms and means of deployment. …

Continue reading “Unleashing Innovation Through Plugins”

The post Unleashing Innovation Through Plugins appeared first on MariaDB.org.

The post Unleashing Innovation Through Plugins appeared first on MariaDB.org.

Why a new slow log for PostgreSQL?

Motivation and benefits

• Statistically accurate sampling with low overhead. We don’t need to log every single query to draw useful conclusions. PEQL can sample 1 out of every N queries (or 1 out of every N sessions), and doing this for enough time will mean that we can have a sample that represents the overall workload for that time period. The cost on the producer side stays low even on busy servers.
• pt-query-digest compatibility out of the box. The output format mirrors the MySQL/Percona Server slow log, so the same toolchain we already use for MySQL audits works for PostgreSQL with no extra steps.
• Logging to a separate file. All entries go to a dedicated file (default peql-slow.log), not to PostgreSQL’s main error log. That keeps the error log clean for actual errors and lets us point the slow log at a separate mountpoint if we want to isolate its I/O from the rest of the server.
• Rate limiting by both queries and bytes per second. On top of the per-session/per-query 1-in-N sampling, peql.rate_limit_auto_max_queries and peql.rate_limit_auto_max_bytes give us a cluster-wide cap on logged queries per second and on bytes written per second. Useful for guaranteeing that the slow log itself never becomes a performance issue.
• Always-log override for slow outliers. Even when sampling is on, peql.rate_limit_always_log_duration lets us say “but always log anything that takes longer than X ms”. The common queries get randomly sampled; the long-running ones always get logged.
• Extended resource usage metrics. Each entry includes buffer hit/read/dirtied/written counts (shared, local and temp), block I/O timings, WAL records/bytes/full-page images, JIT compilation timings, planning time, optional memory context allocations and an optional wait-event histogram.
• Execution plans embedded in the entry. With peql.log_query_plan = on, the full EXPLAIN ANALYZE output (text or JSON) is appended to each entry, so the plan that produced the metrics is right there next to them when we are reviewing the log later.
• Automatic pause when disk space is low. If the log mountpoint drops below a configurable free-space threshold, PEQL pauses logging on its own (with optional auto-purge of old rotated files) and resumes once there is room again. The database keeps serving traffic; the slow log gets out of the way.

Installing the extension

git clone https://github.com/guriandoro/pg_enhanced_query_logging.git
cd pg_enhanced_query_logging
make USE_PGXS=1
sudo make install USE_PGXS=1

shared_preload_libraries = 'pg_enhanced_query_logging'

CREATE EXTENSION pg_enhanced_query_logging;

./test/deploy_docker_pg18_rhel.sh

A minimal configuration

shared_preload_libraries = 'pg_enhanced_query_logging'
peql.log_min_duration = 0 # log every query
peql.log_verbosity = 'full' # emit all metric lines

log_statement = 'none'
log_min_duration_statement = -1
log_duration = off

What an entry looks like

# Time: 2026-03-11T09:15:32.847291
# User@Host: app_user[app_user] @ 10.0.1.42 []
# Thread_id: 48712 Schema: mydb.public
# Query_id: -6432758210044805760
# Query_time: 1.285034 Lock_time: 0.000000 Rows_sent: 256 Rows_examined: 87500
# Shared_blks_hit: 4096 Shared_blks_read: 312 Shared_blks_dirtied: 0 Shared_blks_written: 0
# Temp_blks_read: 0 Temp_blks_written: 48
# Shared_blk_read_time: 0.024310 Shared_blk_write_time: 0.000000
# WAL_records: 0 WAL_bytes: 0 WAL_fpi: 0
# Plan_time: 0.003210
# Full_scan: Yes Temp_table: No Temp_table_on_disk: Yes Filesort: Yes Filesort_on_disk: No
# JIT_functions: 4 JIT_generation_time: 0.001250 JIT_emission_time: 0.003100
SET timestamp=1741680931;
SELECT o.id, o.total, c.name FROM orders o JOIN customers c ON c.id = o.customer_id
WHERE o.status = 'pending' ORDER BY o.total DESC LIMIT 256;

Feeding it to pt-query-digest

pt-query-digest --type slowlog $(pg_config --logdir)/peql-slow.log

pt-query-digest --type slowlog 
--filter '$event->{Full_scan} eq "Yes"' 
$(pg_config --logdir)/peql-slow.log

curl -LO https://percona.com/get/pt-query-digest
chmod +x pt-query-digest

Example pt-query-digest outputs will look like the following images.

A few useful knobs

• peql.rate_limit: 1-in-N sampling, either per session or per query, with a peql.rate_limit_always_log_duration override so that very slow queries are always captured even when sampling is on.
• peql.log_parameter_values: include actual bind parameter values for prepared statements alongside the placeholder query text.
• peql.log_query_plan: embed the full EXPLAIN ANALYZE output (text or JSON) inside the log entry, so the plan that produced the metrics is right there next to them. This can be expensive in terms of I/O, so use sparingly and only if needed.

Future work

# Full_scan: Yes Temp_table: No Temp_table_on_disk: No Filesort: Yes Filesort_on_disk: No

1. A PEQL-native log format. A more compact, structured format (think key-value pairs with short keys, bitfields for the boolean flags, or a binary framing for the numeric metrics) that drops the bytes-per-query cost without losing any of the information we currently emit. The verbose pt-query-digest-compatible format could still be available for users that want it; the native format would be the recommended option for high-throughput workloads.
2. Tooling for the new format. Once the native format exists, we will either contribute a parser to pt-query-digest so that it can ingest it natively (--type peql or similar), or ship a small companion tool that either post-processes them or produces the same kind of profile reports pt-query-digest does today. Either way, the goal is to keep the analysis workflow we are used to while removing the format-imposed overhead from the producer side.

Conclusion

The post Bringing pt-query-digest-Style Slow Query Analysis to PostgreSQL with pg_enhanced_query_logging appeared first on Percona.

The post Bringing pt-query-digest-Style Slow Query Analysis to PostgreSQL with pg_enhanced_query_logging appeared first on MariaDB.org.

If you’ve been to Percona Live, a Percona.connect, a PGConf, KubeCon, FOSDEM, or pretty much any open source database event in the past few years, you’ve probably already met one of us. We’re the people behind the booth, on stage, organising the speakers, herding the giant Jenga set, or trying to convince you to play a quick game of chess between sessions. Now we’re also the people behind @PerconaCommunity on X and our new Mastodon account on the fediverse.

Each of us will sign our posts with our initials, so you’ll always know who you’re talking to. Here’s who we are.

Laura Czajkowski – Director of Community (LC)

Laura runs the team. She’s been in open source community work since the early 2000s, starting at the University of Limerick’s Skynet computer society and going on to lead community at Canonical (Ubuntu), MongoDB, Couchbase, Vonage, Solace, and Dragonfly before joining Percona. Former Ubuntu LoCo Council and Community Council member. Outside work she’s a Munster and Ireland rugby fan, runs a book club, plays tennis, and books regular trips to Disney World. Find her at laura.community.

Alastair Turner – Postgres Community Advocate (AT)

Alastair has been working with databases since 1995, settling on Postgres around 2002. If you’ve spoken to anyone at Percona about PostgreSQL, Kubernetes, Transparent Data Encryption, or extensions, there’s a good chance it was him. He writes regularly on the Percona Community blog and speaks at PGConf events across Europe and North America. He’s particularly interested in how open source communities work together – and what they can learn from each other.

Daniil Bazhenov – Senior Community Manager (DB)

Daniil organises our conference speakers, runs the Percona Forums, and has been a long-time contributor to the Percona Community blog. If you’ve ever submitted a talk to Percona Live or asked a question on forums.percona.com, you’ve crossed paths with him. He writes hands-on technical content too – GitOps with ArgoCD, PMM monitoring, Percona Everest from source – and hosts the Russian-language Percona Podcast.

Kyle Flanagan – Global Manager, Events (KF)

Kyle is the reason any of our events actually happen. He runs Percona’s global events programme, from Percona Live and Percona.connect to our presence at Open Source Summit, KubeCon, and dozens of regional events each year. Before Percona, he ran executive events at Utah Valley University. If you’ve grabbed a sticker at one of our booths, Kyle probably packed the box it came in.

Edith Puclla – Technology Evangelist (EP)

Originally from Peru, now based in London, Edith is a CNCF Ambassador, Docker Captain, and Data on Kubernetes Ambassador. Her background is in DevOps and infrastructure – Kubernetes, GPUs, Linux, distributed systems – and she contributes to translating Kubernetes documentation into Spanish through SIG-Operators. She’s a regular speaker at FOSDEM, KubeCon, Cloud Native Rejekts, and Percona University events across Latin America.

Why we’re doing this

We spend a lot of our time at events because that’s where the most useful conversations happen – the ones over coffee, at the booth, in the hallway between talks. Being on social gives us a way to keep those conversations going when we’re not in the same room. Expect event updates, contributor shout-outs, things we’ve found useful, and the occasional opinion. If we’ve shared it, we’ve actually read it.

Photo below was taken at our recent team offsite in Antalya – five people who genuinely like working together, in case the smiles don’t give it away.

Find us:

• X: @PerconaBytes
• Mastodon: @PerconaBytes
• Forums: forums.percona.com
• Community blog: percona.community

Come say hi. If we’re at an event near you, the booth is open – and so is the giant Jenga.

The post Meet the Percona Community team appeared first on MariaDB.org.

Continue reading “Adding a New Data Type to MariaDB with Type_handler – Part 5”

The post Adding a New Data Type to MariaDB with Type_handler – Part 5 appeared first on MariaDB.org.

The post Adding a New Data Type to MariaDB with Type_handler – Part 5 appeared first on MariaDB.org.

PSMDB Sandbox is a lightweight web frontend built in Go that ships inside the Percona MongoDB Automation repository. It puts a clean browser interface on top of the full Terraform + Ansible automation stack, so you can spin up, manage, and tear down MongoDB environments without ever touching a config file by hand.

This project was built using vibe coding — the result is a fully functional application developed rapidly without writing every line from scratch. It’s a great example of how AI-assisted development can accelerate tooling projects that would otherwise sit in the backlog forever.

Why a Web UI?

The mongo_terraform_ansible project already automates a lot: it can deploy Percona Server for MongoDB (PSMDB), Percona Backup for MongoDB (PBM), and Percona Monitoring and Management (PMM) across AWS, GCP, Azure, Docker, and Libvirt/KVM. That’s powerful — but the workflow traditionally meant editing .tfvars files, running commands in the right order, and tracking state in your head.

The Go UI changes that. It wraps the same Terraform and Ansible automation in a wizard-style interface, streams live output to your browser, and keeps track of environment state so you always know what’s running, stopped, or in progress.

It’s particularly useful as a testing sandbox for PSMDB features. You can quickly spin up a replica set or sharded cluster, test backup and restore workflows with PBM, explore audit logging, and observe everything through PMM monitoring — all from the browser, and all torn down just as easily when you’re done.

What You Can Configure

Cluster Topology

Define how many clusters and replica sets you want, the number of nodes per replica set, and whether to deploy a sharded cluster or a simple replica set. Each cluster is independently configurable.

PSMDB Version and Packages

Pick the exact Percona Server for MongoDB release you want to test — package identifiers are fetched automatically from the Percona repository listing on startup, so you’re always selecting from what’s genuinely available. For Docker-based environments, image tags are pulled live from Docker Hub and cached for five minutes.

Backup and Restore with PBM

Percona Backup for MongoDB (PBM) can be included in the deployment. PBM is configured with the native storage backend for the supported environments (e.g. an S3 bucket is automatically created for AWS). This makes the sandbox ideal for testing backup policies, point-in-time recovery, and restore scenarios without touching production.

PMM Monitoring

You can include a PMM Server in your environment so every PSMDB node is monitored from the moment it comes up. This makes it straightforward to test alerting rules, explore query analytics, or simply validate that your monitoring setup looks right before applying it elsewhere.

Live Deployment Logs

When you hit Deploy, the UI kicks off terraform init && terraform apply (plus Ansible playbooks for cloud platforms) in a background goroutine and streams the output directly to your browser via Server-Sent Events. No more tailing log files in a separate terminal.

Hosts & Connections Panel

After a successful deployment, the environment detail page shows every host (or container) with:

• Its IP address
• A ready-to-copy connect command (ssh user@host or docker exec -it <name> bash)
• MongoDB connection strings for every replica set and cluster
• Clickable Open buttons for PMM and MinIO Console URLs

Stop, Restart, Reset, and Destroy

Full lifecycle management is available from the UI. For Docker environments, Stop and Restart call docker stop / docker restart filtered by the environment’s prefix. For cloud environments, the corresponding Ansible stop.yml and restart.yml playbooks run. Destroy calls terraform destroy and, on success, automatically cleans up the inventory and redirects you back to the environments list.

Getting Started

git clone https://github.com/percona/mongo_terraform_ansible.git
cd mongo_terraform_ansible/ui-go
go run .

Then open http://127.0.0.1:5001 in your browser.

If you prefer a compiled binary:

go build -o mongodeploy .
./mongodeploy

You can customize the bind address and port with environment variables:

Security note: The UI is designed for local use. It binds to 127.0.0.1 by default. Don’t expose it to the public internet without adding authentication.

Try It and Share Your Feedback

PSMDB Sandbox is a community-contributed tool. If you try it out, run into issues, or have ideas for improvements, open an issue or pull request on GitHub. The project is licensed under Apache 2.0.

Happy deploying!

The post PSMDB Sandbox: A Browser-Based UI for Deploying MongoDB with Terraform and Ansible appeared first on Percona.

The post PSMDB Sandbox: A Browser-Based UI for Deploying MongoDB with Terraform and Ansible appeared first on MariaDB.org.

Continue reading “Adding a New Data Type to MariaDB with Type_handler – Part 4”

The post Adding a New Data Type to MariaDB with Type_handler – Part 4 appeared first on MariaDB.org.

The post Adding a New Data Type to MariaDB with Type_handler – Part 4 appeared first on MariaDB.org.

The autonomy problem

If you work in software development, you have most certainly heard the phrase:

Let’s just use an LLM to solve it!

People tend to forget that it’s a bit more complicated than this:
anybody can easily use LLMs, of course, but using them properly is a different question.
Ideally, we could all just download a simple tool, give it some instructions, and relax:

In all seriousness, every panel in the above image contains details that people tend to ignore, which either results in inefficient workflows or the creation of slop.

We can’t talk about all of them in one go; it would be overly long and complex.
I’ll only focus on panel 2:
what can we do to ensure our uninterrupted gardening normal work?

If you simply download Claude/Codex and start using the CLI tool, VS Code extension, or anything else, you’ll quickly get bored of all the babysitting.

Hey, user, can I execute another slightly different ls command?

Either you decide it isn’t worth the effort because of all the interruptions, or you start blindly hitting Enter: “of course I approve, it should be safe…”

1. Are you really thoroughly reviewing every command it throws at you?
2. Even that 100-line bash script the TUI doesn’t display properly, because it wouldn’t fit on the screen?
3. Have you ever seen an agent circumvent directory permissions by accessing the restricted files through a one-off script instead?

Fine-grained permissions of course exist, and in theory, you could try to configure something like that.
But let’s be honest, most of us won’t take the time, and we likely won’t notice if (3) happens as part of a long script.

Let the AI run free

That’s the point where you might discover the other option:
completely disabling the permission system and letting the AI do whatever it wants.

Nothing can go wrong, it’s only on your machine, right?

Except that:

• it will also have full network access, both for reading and posting
• it can read all your secrets: its own OAuth token, your SSH key, and so on…
• do you load your SSH key into ssh-agent? That’s convenient so you don’t have to enter your password every time, but do you also have a hardware key you have to touch on every use, or can the AI force-push your repository and later say

You are absolutely right! I shouldn’t have done that. If you have backups you can restore them with the following steps: …

Or it might end up in any number of similar situations.
Coding agents aren’t malicious by design, but they can be subject to prompt injection from the web, or simply reach dumb conclusions.
There’s a good reason why Claude, for example, calls this option --dangerously-skip-permissions.

Put them in a cage!

The next obvious choice is to let them run free, but only within a cell:
run the agent inside a container or virtual machine, where it can only access what you let it.

This, however, costs us some convenience, as we face new issues:

• If we completely separate the environment, we can’t access it from our main system.
  Allowing AI tools to push to your repo without confirmation is a bad idea, but maybe you yourself should be able to push somehow?
  Or to verify the changes in a more complex, outside environment?
• Our environment and the AI’s environment are different… which means we have to set up both.
  I hope your project is easy to bootstrap, with proper scripting so you don’t have to do this by hand.
  But is your development environment also easy to bootstrap?

There are some existing, ready-to-use solutions: for example, both Claude Code and OpenAI Codex have support for devcontainers.
If you want an easy setup, these can be an option.

However, I wanted more:
to replicate my main setup exactly – the same compilers, tools, shell and editor settings, and so on.
The AI tools should have the same executables available.
If I have to edit or do something directly in the container, I shouldn’t be surprised by something working differently.

That’s when I remembered: I already have a dotfiles repo. Can I make it even better for this use case?

Automate all the things!

The idea of dotfiles is simple:
a repository where you store your configuration, so when you reinstall your system, or when you have to start using another one, you can quickly replicate your preferred settings.
Editors, shells, git – everything works the same, without spending hours figuring it all out again.

The problem is that it usually only focuses on configuring an already properly installed system.
When you only buy a new PC every few years, or system administrators already set up every server you have to use before your first login, this isn’t a big issue.

But when you want to be able to quickly set up and iterate with throwaway systems?
Then you need better automation!

This is also a solved problem; tools like Ansible and Puppet exist.

The idea is simple:

• instead of manually setting up your system, use an automation tool to install and configure everything
• you can leverage free CI services to make sure that your scripts work when run on a clean system
• while docker/podman traditionally uses its own setup scripting, it is possible to build an image using the same automation tool instead
• the result? Main PC, containers, virtual machines, and quick VPS instances all behaving exactly the same way!

The downside is, of course, that you either have to reinstall your main PC once your new setup is good enough, or accept that it will be slightly different until you do so.
I went with the reinstall; it’s easy once you have things working.

And if you don’t know any of these tools?
That’s the best part – we’re using AI, and AI knows them well.

A side note on architecture

The focus of this blog post is panel 2, not the others.
But I want to at least mention that the architecture and human review, including design review, are as important as with any other AI-driven software project.

If you completely vibe-code it and create an unmaintainable, sloppy dotfiles configuration, you are going to regret it later. This is your everyday work environment.

After the initial idea, when I started to think more about my requirements, I quickly realized that I want something generic.

First, I want to install a different set of packages depending on where I am installing them: containers, WSL instances, or real machines.
My laptop needs slightly different settings compared to my desktop.

Second, I want to be able to do this on multiple distributions.
Previously it was really annoying when I had to debug a distro-specific bug, unless it happened to involve one of my primary Linux distributions.
I am also using a different OS on my work laptop and personal desktop PC because of company requirements.

With a proper Ansible setup, I can make all of these work seamlessly, even autodetecting the environment, and verifying all important configurations on CI for every commit.

Your requirements will most likely be different.
Think about these beforehand and structure your repository accordingly!

Containers or virtual machines?

So far I mentioned both as alternatives, and both have their pros and cons.

For now, I went with containers.
With a few helper scripts I can mount specific directories from the host OS, and I can also specify which docker/podman network the new container should join.
This lets me start up my docker-compose development clusters directly on my main OS, and lets the agent access the development/test database and other containers for its work using the shared network.

dcont run --mount `pwd` --network hackorum_default --context main-dev

I even have a context parameter, which lets me keep multiple independent AI configurations: different system-level CLAUDE.md, plugin set, hooks, and so on.
Underneath, this is just a few specific mounts and symlinks, but the advantage is huge:

• I can quickly experiment without fearing that I’ll break my main workflows
• I have completely separate setups for development and review work, without them conflicting with each other

The upsides are easy integration, lower resource overhead, and quicker spin-up.
I can mount directories directly from the host, and easily interact with docker containers running on the host.

The downside comes from that same integration:
everything is still on the host, and the more access you give to the container, the less secure it becomes.
Tools like GDB and GPU access require extra privileges, and you might have to relax SELinux features for the container.

The privilege problem, and the possibility of giving the container too much access, is a real risk.
Docker, which runs as root on the host, and rootless podman, which maps the container root to the current host user, behave very differently if something is misconfigured – but neither protects the data accessible to the running user.

You can tighten the defaults with flags like --cap-drop=ALL, --security-opt=no-new-privileges, and read-only mounts where possible, but these only narrow the attack surface; they don’t fix what you mount in.
Which means what you mount matters more than which runtime you pick.

Mounts and credentials

.env files, for example, can be challenging:
these can contain API keys, passwords, and other secrets required by the application, which ideally shouldn’t be accessible to the coding agent.
I started using two levels of them – one in the project folder with only generic data, and another one level above containing sensitive login information for external services.
This way, when I mount the project folder, the container can’t access the sensitive .env file.

There are also some special files to watch out for:
mounting /var/run/docker.sock into the container, for example, can break the sandbox completely, as it grants access equivalent to root on the host.

When to pick a VM instead

A container also isn’t a full-fledged desktop.
Personally, I am used to working in terminals; I like tools like tmux or neovim.
But if you prefer desktop applications and IDEs, a full virtual machine might be a better option.

Full virtual machines aren’t more difficult to set up and give you a complete GUI, but they raise a different question:
how do you set everything up without accidental credential leaks?

You either rely on network synchronization between your main OS and the virtual machine – pushing to remotes only from the main OS – or you give the virtual machine a hardware key and store your SSH key on it.

Agents can of course always access and leak their own API keys; we can’t do anything about that with 100% certainty.
But we can aim to reduce their ability to leak anything else, by minimizing what they physically have access to.

An example setup

You can check out my dotfiles for inspiration.
It should be only that:
something you can look at while designing your own version.

It is designed for my workflows, and yours are most likely different.
You also shouldn’t blindly trust a script somebody else’s LLM generated.

The helper script

The repository has a readme; the most interesting part is probably the script I mentioned earlier, which builds and runs the containers.

It is long and complex, and deals with additional details I didn’t even mention here, to keep this introduction from getting too involved.

The basic idea, however, is easy to summarize.
A basic docker command is simple:

docker run -it ubuntu:latest /bin/bash

But it also gets complicated quickly:

• what folders need mounting? (the project, specific directories for tools)
• which networks to join?
• do we have to set up specific hardware, like a GPU?
• do we need specific access permissions for some software?
• and so on

The command quickly becomes longer and longer, and copy-pasting it from notes or shell history isn’t fun.
It is also most likely project-specific.
You want different mounts, different contexts for AIs, different specific permissions.
All this should be configurable, and still simple.

In my case, most of my projects also have .env files set up, which makes it a no-brainer to also support configuration through environment variables.

Most of the time, all I have to do is cd into the project directory and execute dcont without any extra parameters. That starts up a ready-to-use, project-specific setup, and I can immediately start typing instructions to Claude.

The Ansible part

I already mentioned this before, but didn’t go into the details:
you can build docker or podman images with Ansible.

Normally this isn’t that useful:
if the only goal is a container cluster, a Containerfile is much easier to use, and more efficient for rebuilding, since it automatically detects which layers have to be rebuilt.

If the goal, however, is to replicate the same setup on a real host and in a container, the picture is different.
These images are only meant for local use, so layering and image size don’t matter – we’ll never upload them.

Build times are also secondary.
Even if a rebuild is needed once or twice a day, you can continue using the previous version in the meantime and switch later.
And it’s not like we can’t do proper incremental builds with it; Ansible supports that too – it’s just a bit slower than how containers normally do it.

This is included in the same script, and the solution is surprisingly simple:

1. start a container with sleep infinity
2. copy the dotfiles repo into it, since it’s already checked out on the host
3. run the same dotfiles/Ansible script as on other hosts (with proper parameters)
4. set up a proper user
5. commit the image

The same could be done using a Containerfile, but what’s the advantage?
The image isn’t shareable or reusable anyway, and some operations are easier to implement directly in bash.
This process also leaves open the possibility of doing incremental builds, instead of always rerunning the installation script from scratch.

The unsaid part: network access

In all of the sandboxing discussion above, I quietly ignored the question of network access:
if you give unrestricted network access to an LLM agent, you can have a bad time.

Prompt injection exists, even if AI companies try to make it harder and harder.

For most use cases, a complete network ban is also a bad idea for productivity and code quality, which makes this another complex, open-ended question with its own options and tradeoffs – out of scope for this already long blog post.

I hope the information I provided here was useful, and that you can improve your AI setup based on it!

The post How I Stopped Babysitting My Coding Agent (With Dotfiles) appeared first on MariaDB.org.

Continue reading “Long live to dbdeployer!”

The post Long live to dbdeployer! appeared first on MariaDB.org.

The post Long live to dbdeployer! appeared first on MariaDB.org.

Bennie published a post on our Build with AI competition last week, in which he shared that I was lucky enough to land the second place prize. Genuinely flattered, and a real thank you to Peter F, PZ, Vadim, and Bennie for organizing it. The recognition is great. But the part that does not quite come through in the recap is what those six weeks actually felt like from the inside. Forty-plus submissions, 10+ teams, marathon demo sessions that ran out of time twice over, and a constant drumbeat of ideas where every fifth one made me think “wait, we can just… ship that?”

Two submissions that really impressed me (and are worthy of high praise): Kedar Vaijanapurkar shipped a four-tool MySQL stack (Advisor, random data generator, CleanPrompt, and a Query Reviewer), any one of which on its own would have been a strong submission. And Daniil built a leaderboard for Percona ecosystem contributors plus a vector-search prototype running on Percona’s own products, which is exactly the dogfood story we want.

There were a lot more than three projects worth backing, which is part of why a second contest round is being coordinated later this year. A lot of the entries are not waiting for it either – they are already developing into real, operational utilities (some of mine included).

The two submissions of my own that I would point to first are IBEX and percona-dk.

IBEX (Integration Bridge for EXtended systems) is a local MCP multi-tool server that connects either a local model or a Percona-owned LLM to the systems where the most valuable context actually lives. Slack, Notion, Jira, ServiceNow, Salesforce, etc. A solution was needed here since we could not point the standard Claude or ChatGPT connectors at our sensitive internal data, and obviously most of the context that makes LLMs so valuable is precisely that kind of data.

percona-dk is the other one. It started as a way to keep AI honest about our own products by giving the AI tools our teams use (Claude, Cursor, anything that speaks MCP) direct access to Percona’s documentation, so the answer to a question about our products comes from real docs with linked citations instead of stale training data or even scraped web results that can get things wrong. It has evolved a fair bit since the contest. The Percona Community blog and forums are now indexed alongside the docs, Perconians are getting real day-to-day value out of it, and it is starting to look like the kind of thing that could grow into a community utility (perhaps even beyond Percona docs).

Those two were just the start. Once IBEX worked, I needed shared memory across LLMs, so I built that. Once I had three MCP servers running, the boilerplate got annoying, so I built CAIRN, a scaffolding tool that builds on Anthropic’s official MCP builder skill. The official skill walks you through writing a server step by step, but CAIRN spins up a complete, working project in minutes with a streamlined install wizard for non-technical users. It is now in the hands of other Perconians building their own MCP tools, and providing real value of its own. Then I learned about .mcpb files and Desktop Extensions (.dxt), packaged everything that way, and stood up an internal Claude plugin marketplace so any Perconian can install the lot from one place. Each layer opened a door I did not know existed until I was already through it. Some of those doors seemingly materialized from thin air as they magically aligned with new releases from Anthropic.

What started as a competition entry is now a small internal ecosystem. I am still a product person, not a software engineer. I am not going to pretend any of the code is pristine, and a lot of it was vibe-coded with Claude as a partner. But the architecture holds together, it works, and most of it is in daily use by people who are not me. That last part is the bit I am most proud of.

The next batch is pointed squarely at product operations. Making customer signals legible. Making internal telemetry something any teammate can talk to in plain English. The early returns are promising, and what gets me most excited is not the tech itself, it is watching people across Product, Engineering, and Support pull in the same direction with an AI colleague in the room. Turns out the interesting part of AI at work is not the model. It is the connective tissue.

I know Kung Fu

For a product guy who does not code for a living, this era is my “I know kung fu” moment. Not because I suddenly learned to fight. Because the move set I already had – product judgment, systems thinking, customer empathy, the ability to spec a thing precisely – just got a massive upgrade. The gap between “that would be useful” and “that exists now” is short enough to cross in an evening. I do not see it getting longer again.

Thanks for reading this far. If you want more detail or want to try anything not linked here, ping me. I am happy to share more.

The post I Know Kung Fu appeared first on Percona.

The post I Know Kung Fu appeared first on MariaDB.org.

Continue reading “Adding a New Data Type to MariaDB with Type_handler – Part 3”

The post Adding a New Data Type to MariaDB with Type_handler – Part 3 appeared first on MariaDB.org.

The post Adding a New Data Type to MariaDB with Type_handler – Part 3 appeared first on MariaDB.org.

Let’s see how the maintenance goes. It was supposed to be a simple restart. The kind you’ve done a hundred times. You SSH in, run the maintenance, bring the node back up, and go grab a coffee. Except this time, the coffee went cold on the desk… because MySQL refused to start.

The Problem

The error log of Percona XtraDB Cluster (8.0) had the following information:

2025-11-05T05:26:10.982984Z 0 [ERROR] [MY-000059]   [Server] SSL error: Unable to get certificate from '/var/lib/mysql/server-cert.pem'.
2025-11-05T05:26:10.983030Z 0 [Warning] [MY-013595] [Server] Failed to initialize TLS for channel: mysql_main. See below for the description of exact issue.
2025-11-05T05:26:10.983045Z 0 [Warning] [MY-010069] [Server] Failed to set up SSL because of the following SSL library error: Unable to get certificate
2025-11-05T05:26:10.983052Z 0 [Note] [MY-000000] [WSREP] New joining cluster node configured to use specified SSL artifacts
2025-11-05T05:26:10.983083Z 0 [Note] [MY-000000] [Galera] Loading provider /usr/lib64/galera4/libgalera_smm.so initial position: 07c67757-0d18-11ef-b5a9-ee5d87b39aa8:4147053897
2025-11-05T05:26:10.983098Z 0 [Note] [MY-000000] [Galera] wsrep_load(): loading provider library '/usr/lib64/galera4/libgalera_smm.so'
2025-11-05T05:26:10.983742Z 0 [Note] [MY-000000] [Galera] wsrep_load(): Galera 4.22(f6c0465) by Codership Oy <info@codership.com> (modified by Percona <https://percona.com/>) loaded successfully.
2025-11-05T05:26:10.983771Z 0 [Note] [MY-000000] [Galera] Resolved symbol 'wsrep_node_isolation_mode_set_v1'
2025-11-05T05:26:10.983784Z 0 [Note] [MY-000000] [Galera] Resolved symbol 'wsrep_certify_v1'
2025-11-05T05:26:10.983807Z 0 [Note] [MY-000000] [Galera] CRC-32C: using 64-bit x86 acceleration.
2025-11-05T05:26:10.983995Z 0 [Note] [MY-000000] [Galera] not using SSL compression
2025-11-05T05:26:10.984341Z 0 [ERROR] [MY-000000] [Galera] Bad value '/var/lib/mysql/server-cert.pem' for SSL parameter 'socket.ssl_cert': 336245135: 'error:140AB18F:SSL routines:SSL_CTX_use_certificate:ee key too small'
         at /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/rpmbuild/BUILD/Percona-XtraDB-Cluster-8.0.42/percona-xtradb-cluster-galera/galerautils/src/gu_asio.cpp:ssl_prepare_context():471
2025-11-05T05:26:10.984401Z 0 [ERROR] [MY-000000] [Galera] Failed to create a new provider '/usr/lib64/galera4/libgalera_smm.so' with options 'gcache.size=1G;gcache.recover=yes;socket.ssl=yes;socket.ssl_ca=/data00/mysqldata/ca.pem;socket.ssl_cert=/data00/mysqldata/server-cert.pem;socket.ssl_key=/data00/mysqldata/server-key.pem;socket.ssl_key=/var/lib/mysql/server-key.pem;socket.ssl_ca=/var/lib/mysql/ca.pem;socket.ssl_cert=/var/lib/mysql/server-cert.pem': Failed to initialize wsrep provider
2025-11-05T05:26:10.984434Z 0 [ERROR] [MY-000000] [WSREP] Failed to load provider
2025-11-05T05:26:10.984448Z 0 [ERROR] [MY-010119] [Server] Aborting
2025-11-05T05:26:10.984602Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.42-33.1)  Percona XtraDB Cluster (GPL), Release rel33, Revision 6673f8e, WSREP version 26.1.4.3.
2025-11-05T05:26:10.985473Z 0 [ERROR] [MY-010065] [Server] Failed to shutdown components infrastructure.

MySQL was down, and the maintenance clock was running. The certificate file sitting at /var/lib/mysql/server-cert.pem was the same file that had been working perfectly fine before the restart!!
From past history, it was known that the following commands were executed correctly on the same cluster node

SET GLOBAL ssl_ca = '/var/lib/mysql/ca.pem';
  SET GLOBAL ssl_cert = '/var/lib/mysql/server-cert.pem';
  SET GLOBAL ssl_key = '/var/lib/mysql/server-key.pem';
  ALTER INSTANCE RELOAD TLS;

Clients connected over TLS. Galera nodes communicated securely. There were zero complaints from the error log.
In other words, the SSL reload at runtime inherited the process environment that existed when MySQL originally booted. Everything was smooth, but after a restart? MySQL complains and declines to start. So what has changed?

Checking Usual Suspects

File permissions

We checked the PEM files. 

Ownership: mysql:mysql.
Permissions: 644 for the cert, 600 for the key. 

We compared them against the other Galera nodes, and they were identical. This didn’t look like a permissions problem.

Is SELinux to blame here?

SELinux has ruined enough DBA time that it is one of the top spots on such checklists – but it was permissive.

$ getenforce
Permissive

That means it was logging any security issues, but not blocking. And there were no AVC denials related to MySQL or the PEM files in /var/log/audit/audit.log or dmesg!

File corruption

Did the files get corrupted/replaced during or before the MySQL restart?

$ openssl x509 -in /var/lib/mysql/server-cert.pem -noout -text
# Output looked perfectly valid when compared to the output from other nodes

$ openssl rsa -in /var/lib/mysql/server-key.pem -check
RSA key ok

The files were fine. They parsed cleanly. OpenSSL could read them. So why couldn’t MySQL?

More Logs review

We scanned /var/log/messages and journalctl for anything unusual around the time of the restart. No disk errors. No OOM kills. No kernel panics. Nothing that screamed “I am the Dhurandhar that’s destroyed your node.” At this point, most of the usual suspects were guilt-free, staring at us, asking, “Who did it?”

The Clue

It is good to communicate with stakeholders, and we did – “Was there any recent change on your side?” to the client, and then uttered the golden words “Last week the crypto-policy was updated on all of the DB servers to comply with PCI.”

PCI > Crypto-policy – Let’s go and check it !!

$ update-crypto-policies --show
FUTURE

The system was running RHEL’s FUTURE cryptographic policy.

For those unfamiliar (including me at the time), Red Hat Enterprise Linux (and its derivatives, such as Rocky, Alma, and Oracle Linux) ships with a system-wide cryptographic policy framework. It’s a centralized way to enforce minimum standards for TLS versions, cipher suites, key lengths, and signature algorithms across all applications on the system that include OpenSS and yes, anything that links against those libraries… like MySQL.

Here’s a table that shows information about the crypto-policy levels:

So FUTURE demands a 3072-bit RSA key; otherwise, it is blocked. What do we have?

$ openssl rsa -in server-key.pem -text -noout | head -1
RSA Private Key: (2048 bit, 2 primes)

2048 bits! C’mon! And now I recall the error log again… The hint was there:

error:140AB18F:SSL routines:SSL_CTX_use_certificate:ee key too small

Now we have our story straight.
On restart, our PXC cluster node started a new process linked against OpenSSL, which now enforced the FUTURE policy. OpenSSL looked at the 2048-bit RSA certificate and said: “Nope. Too small.”

Fixture

The quick fix here would be to adjust the policy to DEFAULT.

sudo update-crypto-policies --set DEFAULT

This will accept the current SSLs, and the node will join the cluster readily.

Alternatively, to remain compliant and adhere to the security policy strictness, the fixture will be to

• Generate new certificates
• Deploy the keys/certs to all Galera nodes
• Perform a rolling restart

Conclusion

This was a classic case of a problem hiding at the boundary between two domains, database administration and operating system security. The DBA saw valid certificates and correct MySQL configuration. The sysadmin saw a properly hardened system with a strong crypto policy. Neither was wrong. But the intersection of their two correct configurations produced a failure.

This incident reinforces the importance of cross-domain awareness, where resolving database issues sometimes requires understanding and challenging system-level security decisions.

The post Curious case of PXC node that refused to start due to SSL appeared first on Percona.

The post Curious case of PXC node that refused to start due to SSL appeared first on MariaDB.org.

We all know and love the Query Analytics (QAN) dashboard… It’s the first place we look when an incident alert fires or when a developer asks, “Why is the app slow?” or “What was going on during the midnight production outage?”

But sometimes, the standard dashboards just don’t tell the whole story or maybe are not clear enough. QAN is great, but shouldn’t we have more? If you have PMM running, you already have a Ferrari engine under the hood: ClickHouse. Most of us just drive it in first gear using the default UI.

In this post, we are going to take the training wheels off. We will bypass the standard QAN interface and talk directly to the ClickHouse backend to build highly specialised dashboards. We aren’t just looking for “slow” queries anymore; we are hunting for inefficiency, volatility, and the “silent killers” that standard monitoring often misses.

This is the hands-on blog, so grab your coffee and let’s turn that PMM instance into a deep-dive forensic tool.

Create a New Dashboard in PMM

1. Connect to PMM > Dashboards > Create New Dashboard
2. Save it with name “Slow Query Analysis” and Description “Slow Query Analysis from PMM’s QAN database (clickhouse)”
3. Click on add visualisation & select datasource “ClickHouse”
   
   
4. Choose SQL Builder
   
   
5. Paste the following query to get top 10 slow queries from the database
   

   SELECT fingerprint
       FROM pmm.metrics
       WHERE service_type = 'mysql'
         AND $__timeFilter(period_start)
       GROUP BY fingerprint
       ORDER BY sum(m_query_time_sum) DESC
       LIMIT 10

6. Choose “Table View” on the top to view the list
   When you click “Run Query” you will see the top 10 slow queries in the chosen time period.
7. Let’s Save the dashboard after Panel Options updates as follows7.1 Change Panel Name and Description to: “Slow Query Analysis”7.2 Legend Placement to “Bottom”, Values to “min”,”max”, “mean”7.3 Change Axis’ Scale to “Logarithmic”Logarithmic scale on an axis compresses large ranges of data, making it ideal for visualizing metrics with vastly different magnitudes. This provides good visualisation for queries of different execution time frames.7.4 Save DashboardAlright, we’re at our first step. This first result set shows the top 10 slow query fingerprints across all MySQL services tracked by PMM for the selected time range. It provides a quick, environment-wide view of the most expensive query patterns. But this does not provide a clear picture. Let’s refine the dashboard to focus on specific queries, servers and observe their performance over time.Now, let’s introduce a variable to filter the data.
8. Click on Settings on Dashboard’s home page8.1 Choose “Variables” tab and click on “Add Variable”8.2 Add variable configuration and Save Dashboard 
9. Go Back to Dashboard and Edit “Slow Query Analysis” Panel.
   • Now you should see the Query ID filter on the top.
10. Change the query to the following
    

    SELECT
      period_start AS time,
      left(fingerprint, 80) AS query_text,
      sum(m_query_time_sum/m_query_time_cnt) AS query_time
    FROM
      pmm.metrics
    WHERE
      service_type = 'mysql'
      AND $__timeFilter(period_start)
      AND fingerprint IN (
        SELECT fingerprint
        FROM pmm.metrics
        WHERE service_type = 'mysql'
          AND $__timeFilter(period_start)
          AND ($queryid = '' OR queryid = $queryid)
        GROUP BY fingerprint
        ORDER BY sum(m_query_time_sum) DESC
        LIMIT 10
      )
    GROUP BY
      time,
      fingerprint
    ORDER BY
      time,
      query_time DESC

    • Basically the query is fetching start time, query text and average query time for the selected period for the top 10 Queries in that time-frame.
    • There is a filter for the “queryid” variable which you may use if you want to filter on a specific queryid.
    • Choose “Time Series” as “Query Type”
11. Adjust Panel Options11.1 Choose “Standard options” > “Unit” as “Time / Seconds (s)” from drop down.11.2 Choose “Standard options” > “Display name” as “${__field.labels.query_text}”11.3 Click on “Save Dashboard”
12. Your dashboard should be ready

Now, by default this dashboard is plotting top 10 queries. If you have a query fingerprint handy, you may be able to filter the search by that specific query.  That said, this is still plotting queries across all the monitored instances. Let’s move on to add the service_name filter.

Adding service_name filter

1. Add Variable
   1. Create new variable named “service_name”
   2. Use variable type “Query”
   3. Use Data Source as “ClickHouse”
   4. Query:
      

      select distinct service_name from pmm.metrics where service_type = 'mysql';

   5. Unselect all checkboxes in “Selection options”
   6. Save Dashboard
2. Update Query

SELECT
  period_start AS time,
  left(fingerprint, 80) AS query_text,
  sum(m_query_time_sum/m_query_time_cnt) AS query_time
FROM
  pmm.metrics
WHERE
  (service_name = '' OR service_name = '$service_name')
  AND service_type = 'mysql'
  AND $__timeFilter(period_start)
  AND fingerprint IN (
    SELECT fingerprint
    FROM pmm.metrics
    WHERE service_type = 'mysql'
      AND $__timeFilter(period_start)
      AND (service_name = '' OR service_name = '$service_name')
    GROUP BY fingerprint
    ORDER BY sum(m_query_time_sum) DESC
    LIMIT 10
  )
GROUP BY
  time,
  left(fingerprint, 80) 
ORDER BY
  time,
  query_time DESC

I know many of you are naturally curious and enjoy experimenting with PMM and Grafana… So you’ve probably already started thinking about how far this can be taken. Feel free to share your ideas or custom dashboards in the comments.

Sample Dashboards:

The Query Analysis and Insights Dashboard

Okay, for those who are looking to have quick results, I’ve prepared the complete Query Analysis and Insights Dashboard for you to import and use instantly.

By importing the JSON file, you’ll get the full working dashboard with all panels preconfigured, including:

• Slow Query Analysis
• Latency Distribution Heatmap
• Query Volatility (P99 vs Average)
• Lock Wait Ratio Over Time (Top Contended Queries)
• Temporary Table Usage (Disk & Memory)
• Query Efficiency (Rows Examined vs Rows Sent)
• Error Rate vs Throughput
• Workload Distribution by User
• Query Volume by Client Host
• Execution Time vs Lock Wait Time

This allows you to instantly explore PMM Query Analytics data, adjust time ranges and filters, and correlate query performance, contention, and workload behavior without recreating the dashboard from scratch.

Dashboard JSON available here:

• Grafana: https://grafana.com/grafana/dashboards/24896
• GitHub:  https://github.com/Percona-Lab/pmm-dashboards/query_analysis_insights.json

Give it a go and let me know if you have suggestions or requests. Also consider sharing if you create something interesting.

Cheers.

The post Building Query Analysis and Insights Dashboard in PMM appeared first on Percona.

The post Building Query Analysis and Insights Dashboard in PMM appeared first on MariaDB.org.

Continue reading “Adding a New Data Type to MariaDB with Type_handler – Part 2”

The post Adding a New Data Type to MariaDB with Type_handler – Part 2 appeared first on MariaDB.org.

The post Adding a New Data Type to MariaDB with Type_handler – Part 2 appeared first on MariaDB.org.

Many MySQL configurations inherit redo log sizing from defaults, aging blog posts, or configuration folklore.

innodb_redo_log_capacity gets set once… and then quietly fades into the background.

But redo log capacity directly shapes how efficiently MySQL absorbs writes, manages checkpoint pressure, and handles burst-heavy workloads.

Set it too low, and aggressive flushing can throttle throughput.
Set it too high, and crash recovery can become painfully long.

Redo logs are more than crash insurance.

They are part of your write-performance architecture.

Redo logs are the shock absorbers of write-heavy MySQL. Too small, and performance jolts. Too large, and recovery drags.

Why Redo Logs Matter

InnoDB redo logs are often described as crash recovery journals, but that description undersells their real operational value.

Redo logs function as a write buffer between committed transactions and eventual data file writes.

When a transaction commits:

• Changes are written to the redo log first
• Dirty pages remain in memory
• Data pages are flushed later

This write-ahead logging (WAL) design allows MySQL to:

• Absorb bursts of write activity
• Reduce immediate random disk writes
• Smooth checkpoint behavior
• Preserve durability

Redo logs act like pressure regulators in a write-heavy system.

They absorb pressure spikes so the entire system doesn’t thrash every time demand increases.

Without enough redo capacity, MySQL has less room to absorb write bursts before it must flush aggressively.

Checkpoint Age and Flushing Pressure

Redo log sizing becomes most visible when checkpoint pressure builds.

Checkpoint age represents how far current write activity has advanced beyond the last durable checkpoint:

Checkpoint Age = Current LSN - Last Checkpoint LSN

As checkpoint age approaches total redo capacity:

• Adaptive flushing intensifies
• Page cleaners become more aggressive
• Dirty pages flush faster
• Disk I/O spikes
• Latency often becomes unstable

This is where undersized redo logs can trigger flush storms.

MySQL isn’t writing more data. It’s being forced to write sooner and less efficiently.

Useful metrics

• Innodb_checkpoint_age
• Innodb_buffer_pool_pages_dirty
• Innodb_data_fsyncs
• Innodb_log_waits

When redo space shrinks, MySQL doesn’t stop writing. It starts panicking earlier.

Symptoms of Undersized Redo

Small redo logs rarely announce themselves directly.

Instead, they often masquerade as generalized storage or write-performance issues.

Common warning signs

• Periodic write stalls
• Spikes in fsync activity
• Sharp increases in page cleaner workload
• TPS drops during burst traffic
• Dirty page percentage volatility
• Stable CPU, unstable write latency

A common misdiagnosis

Many systems blame disks when the real issue is insufficient redo headroom.

If writes are arriving faster than redo can comfortably buffer them, MySQL is forced into reactive flushing patterns.

The problem may not be disk speed.

It may be timing pressure.

Measuring with Status Counters

Redo log sizing should be based on observed workload, not memory percentages or inherited defaults.

Step 1: Measure redo generation rate

Use:

SHOW ENGINE INNODB STATUS;

Track:

• Log sequence number
• Log flushed up to
• Last checkpoint at

Measure LSN growth over time:

Redo Generation Rate = (LSN delta) / elapsed time

Example

If LSN grows by 4 GB over one hour:

• 1 GB redo capacity = frequent pressure
• 4 GB redo capacity = ~1 hour buffer
• 8 GB redo capacity = larger burst tolerance

Step 2: Watch for log stress

SHOW GLOBAL STATUS LIKE 'Innodb_log_waits';
SHOW GLOBAL STATUS LIKE 'Innodb_os_log%';

Key metric

Innodb_log_waits

If this value increases, transactions are waiting for log free space.

That is one of the clearest signs your redo logs may be too small.

Innodb_log_waits is less a tuning suggestion and more a smoke alarm.

Practical Sizing Strategy

Forget percentage-of-RAM formulas.

Redo logs should be sized around workload intensity.

A practical starting point:

Size redo capacity to hold 30 to 60 minutes of peak redo generation

Example

Peak redo generation = 6 GB/hour

Minimum:

30 minutes = 3 GB

Safer:

60 minutes = 6 GB

Heavy burst environments:

Larger sizing may reduce flush volatility further

Trade-Offs

Smaller Redo Logs

Pros:

• Faster crash recovery
• Lower storage footprint

Cons:

• Increased checkpoint pressure
• More aggressive flushing
• Greater write instability

Larger Redo Logs

Pros:

• Better burst absorption
• Smoother sustained write performance
• Reduced flush storms

Cons:

• Longer crash recovery
• Delayed visibility into pressure buildup

Common Mistakes

1. Treating redo like buffer pool sizing
   Redo capacity is about write throughput buffering, not memory caching.

2. Ignoring Innodb_log_waits
   This can leave obvious pressure invisible until performance suffers.

3. Oversizing without testing recovery
   Large redo logs may improve runtime but worsen restart scenarios.

4. Sizing for average load instead of peak
   Redo logs exist to absorb pressure spikes, not calm periods.

Final Thoughts

The right redo log size isn’t about maximizing a configuration value.

It’s about matching capacity to workload behavior.

• Too small, and MySQL becomes reactive.
• Too large, and crash recovery becomes the hidden tax.

When redo logs are properly sized, they fade into the background.

They quietly absorb bursts, smooth checkpoint behavior, and preserve performance consistency under pressure.

Redo logs work best when they disappear into the background, quietly absorbing pressure instead of creating it.

Stop guessing.

Measure your workload, observe your log pressure, and size with intent.

The post InnoDB Redo Log Sizing: Stop Guessing, Start Measuring appeared first on MariaDB.org.

At 80%, we have enough time to change it to BIGINT.…. Right? Let’s see.

So we used pt-online-schema-change to perform the alter.

It started running at a good pace but slowed over time.

Why?

Well, let’s look at the definition of the table:

mysql> show create table myschema.mytableG
*************************** 1. row ***************************
       Table: mytable
Create Table: CREATE TABLE `mytable` (
  `id` int unsigned NOT NULL AUTO_INCREMENT,
  `long_column` varchar(1000) NOT NULL,
  `state` tinyint unsigned NOT NULL,
  `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `short_column` varchar(30) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `idx_long_column` (`long_column`,`state`),
  KEY `idx_short_column` (`short_column`,`state`),
  KEY `idx_short_col2` (`short_column`)
) ENGINE=InnoDB AUTO_INCREMENT=4009973818 DEFAULT CHARSET=utf8mb3

NOTE1: The index on long_column is for a varchar column with a length of 1000; it may not be required, and an index prefix may be more helpful here.

NOTE2: The index idx_short_col2 is duplicated, as it is covered by the index idx_short_column.

Those changes require testing and are out of scope for this emergency, but they are worth mentioning.

Table size:

+---------------+------------+------------+---------+----------+---------+----------+--------+
| TABLE_SCHEMA  | TABLE_NAME | TABLE_ROWS | DATA_GB | INDEX_GB | FREE_GB | TOTAL_GB | ENGINE |
+---------------+------------+------------+---------+----------+---------+----------+--------+
| myschema      | mytable    | 3906921584 |    1118 |     1790 |       0 |     2907 | InnoDB |
+---------------+------------+------------+---------+----------+---------+----------+--------+

Look at the indexes being way bigger than the data.

mysql> SELECT database_name, table_name, index_name, ROUND(stat_value * @@innodb_page_size / 1024 / 1024, 2) AS size_in_mb FROM mysql.innodb_index_stats WHERE stat_name = 'size' AND index_name != 'PRIMARY' and database_name='myschema' and table_name='mytable' ORDER BY size_in_mb DESC;
+---------------+------------+-------------------+------------+
| database_name | table_name | index_name        | size_in_mb |
+---------------+------------+-------------------+------------+
| myschema      | mytable    | idx_long_column   | 1583538.95 |
| myschema      | mytable    | idx_short_column  |  126432.98 |
| myschema      | mytable    | idx_short_col2    |  122699.95 |
+---------------+------------+-------------------+------------+
3 rows in set (0.01 sec)

While the pt-online-schema-change runs, it copies the data to a new table. As the data is being copied, the secondary indexes must be maintained.

NOTE the huge index for a varchar(1000) that is ~1.5T in size. Maintaining such an index becomes increasingly expensive as the data size increases.

The pt-online-schema-change had been running for ~8 days, and its latest estimate was 53 more days, which we can’t afford, since the maximum value would be exceeded in ~15 days. 

Copying `myschema`.`mytable`:  12% 53+16:48:01 remain
Copying `myschema`.`mytable`:  12% 53+16:48:30 remain
Copying `myschema`.`mytable`:  12% 53+16:48:59 remain
Copying `myschema`.`mytable`:  12% 53+16:49:26 remain
Copying `myschema`.`mytable`:  12% 53+16:49:53 remain
Copying `myschema`.`mytable`:  12% 53+16:50:19 remain
Copying `myschema`.`mytable`:  12% 53+16:50:49 remain
Copying `myschema`.`mytable`:  12% 53+16:51:17 remain
Copying `myschema`.`mytable`:  12% 53+16:51:45 remain

So what do we do now?

We suggested canceling the pt-online-schema-change and creating an Aurora blue-green deployment.

Then perform the direct ALTER on the green cluster. And finally, when ready, do the failover.

Sounds good, doesn’t it?

First, we need to ensure that the new cluster (green) has the replica_type_conversions  parameter in its cluster parameter group to “ALL_NON_LOSSY, ALL_UNSIGNED” in order to be able to replicate from an int unsigned column to a bigint unsigned column.

So we tried that, it started too fast ~0.036% per minute, that’s 2 days. That’s great!

We left the process running over the weekend, but we noticed it started to slow down again… By Monday, it was advancing at ~0.01% every 5 mins, which gives an ETA of 34 days. 

Why? 

Again, using the direct ALTER MySQL copies the data to a temp table, and the bigger the data, the harder it is to maintain the indexes. 

Again, unacceptable.

Note that with the above 2 approaches, we lost ~12 days of precious time, and the deadline for auto_increment exhaustion was approaching.

Then we thought: What if we drop the secondary indexes, do the alter, and then add the indexes back?

In theory, it should be faster, as:

• Dropping the indexes is a metadata-only operation with ONLINE DDL.
• Altering the column datatype from INT to BIGINT is not an ONLINE operation, but the fact that it doesn’t have to update secondary indexes during row copying to a new temporary table prevents the slowdown.
  
• Adding back the secondary indexes is an ONLINE DDL operation:

“Online DDL support for adding secondary indexes means that you can generally speed the overall process of creating and loading a table and associated indexes by creating the table without secondary indexes, then adding secondary indexes after the data is loaded.”

https://dev.mysql.com/doc/refman/8.4/en/innodb-online-ddl-operations.html

So let’s do this:

The deletion of the indexes was really quick, as expected (metadata-only operation):

mysql> ALTER TABLE myschema.mytable DROP INDEX idx_long_column, DROP INDEX idx_short_column, DROP INDEX idx_short_col2;
Query OK, 0 rows affected (49.40 sec)
Records: 0  Duplicates: 0  Warnings: 0

Then the change of the datatype:

mysql> ALTER TABLE myschema.mytable CHANGE COLUMN id id bigint unsigned NOT NULL AUTO_INCREMENT;
Query OK, 4058047205 rows affected (13 hours 9 min 10.62 sec)
Records: 4058047205  Duplicates: 0  Warnings: 0

Looks very promising!!!

The final step, add back the indexes:

mysql> ALTER TABLE myschema.mytable ADD INDEX `idx_long_column` (`long_column`,`state`), ADD INDEX `idx_short_column` (`short_column`,`state`), ADD INDEX `short_col2` (`short_column`);
ERROR 1878 (HY000): Temporary file write failure.

Why?

Well, the INPLACE operation uses the tmp dir to write sort files. In Aurora, there are certain limits for the temporary space based on the instance type. 

In a regular MySQL instance, we can modify the innodb_tmpdir to another location with enough disk space; however, in Aurora, the parameter is not modifiable, which could have made the whole process easier.

Even with a larger instance type, it’s hard to create the 1.5T index without breaking open the piggy bank.

Last resort, add the indexes back with the COPY algorithm:

mysql> ALTER TABLE myschema.mytable ALGORITHM=COPY, ADD INDEX `idx_long_column` (`long_column`,`state`), ADD INDEX `idx_short_column` (`short_column`,`state`), ADD INDEX `idx_short_col2` (`short_column`);
Query OK, 4147498819 rows affected (6 days 1 hour 55 min 57.00 sec)
Records: 4147498819  Duplicates: 0  Warnings: 0

Why does it work? Because ALTER TABLE using the COPY algorithm uses the datadir as the destination for the temporary table, the rows are copied there. It doesn’t have the limitation of the temporary directory mentioned above.

We were able to make it on time about 4 days before the auto_increment exhaustion, preventing downtime.

In retrospective we could have used the following approach to avoid the use of the blue/green deployment:

1. Perform a pt-online-schema-change on the main table, dropping the indexes, and changing the column type to bigint. ( with –no-swap-tables –no-drop-old-table –no-drop-new-table –no-drop-triggers).
2. Add the secondary indexes using the direct alter with the COPY algorithm in the _new table.
3. Once the alter finishes, swap the tables and drop the triggers.

Conclusion:

What initially looked like an easy task with pt-online-schema-change, ended up being more complex. 

You need to check the data definition, the index sizes, the Aurora limits, and how the different algorithms work to make a decision on the best way to proceed with those tasks, specially on situations like these where you have the pressure of the auto_increment being exhausted and there’s risk of downtime if it is not done on time.

And of course, monitor auto_increment exhaustion for your tables, and use a reasonable threshold that gives you enough time to plan and change the table definition. You can use Percona Monitoring and Management for this, specifically on the MySQL > MySQL Table Details dashboard.

The post <H1> Run an ALTER TABLE for a huge table in Aurora appeared first on Percona.

The post Run an ALTER TABLE for a huge table in Aurora appeared first on MariaDB.org.

While the Valkey Operator has not yet released a GA 1.0 version, we wanted to take this opportunity to highlight some recently added features.

Cluster Configuration

Up until recently, there was no native ability to provide configuration parameters to the Valkey server process running inside each deployed pod. This hurdle is now overcome, and you can supply configuration natively within the deployment CR.

apiVersion: valkey.io/v1alpha1
kind: ValkeyCluster
metadata:
  name: my-valkey-cluster1
spec:
  shards: 3
  replicas: 1
  config:
    maxmemory: 500mb
    maxmemory-policy: allkeys-lfu
    maxclients: 5000
    commandlog-execution-slower-than: 10000

For now, these parameters are set on initial cluster deployment. There is already traction underway to allow certain parameters to be dynamically set at runtime. There are a small handful of certain cluster-based parameters that cannot be overridden by the user, otherwise it would break operator functionality.

User Access Control List (ACL)

Managing users is always a tedious task for any database administrator. Creating ACLs for users in Valkey can be a bit confusing coming from a traditional RDBMS using GRANT syntax. To make things just a bit easier, Valkey Operator has added user permissions management to the deployment CR.

Firstly, create your Secret containing usernames, and passwords:

apiVersion: v1
kind: Secret
metadata:
  name: valkey-cluster-sample-users
data:
  alicepw: M21wdHlQQHNzdzByZA==
  davidold: OVYqTHQlYXU4Mk5tdTlyeQ==
  davidnew: VmFsa2V5I1J1bHojMjIzMw==

Next, deploy your cluster with users:

apiVersion: valkey.io/v1alpha1
kind: ValkeyCluster
metadata:
  name: my-cool-valkey-cluster
spec:
  shards: 3
  replicas: 1
  users:
    - name: alice
      enabled: true
      passwordSecret:
        name: valkey-cluster-sample-users
        keys: [alicepw]
      commands:
        allow: ["@read", "@write", "@connection"]
        deny: ["@admin", "@dangerous"]
      keys:
        readWrite: ["app:*", "cache:*"]
        readOnly: ["shared:*", "config:*"]
        writeOnly: ["logs:*", "metrics:*"]
    - name: david
      enabled: true
      passwordSecret:
        name: valkey-cluster-sample-users
        keys:
          - davidold
          - davidnew
      commands:
        allow: ["@admin"]

There’s quite a lot going on here. Let’s break it down by first looking at the user ‘alice’: 

The ‘alice’ user is enabled, with a password found in the referenced Secret and secret key. Next, we can see what commands, or in this case, command groups (Noted with ‘@’) that alice is allowed to execute, and which commands/groups are denied. Lastly, permissions on specific key patterns are identified for maximum security restrictions.

The other user, ‘david’, can access all of the admin-group commands, and cannot read or write to any keys. Note that david’s secret key reference is an array, which means you can provide multiple passwords per user; great for password rotation! Once david confirms the new password, the old password references can be removed from the CR and Secret, and the Valkey Operator will synchronize the ACLs.

Users are dynamic, which means they can be added, removed, and modified without restarting the cluster.

TLS Support

Bring on the encryption! TLS support was also recently added to the Valkey Operator. Create your Secret with the CA, TLS Key, and Cert files, and tell the CR where to find them:

apiVersion: valkey.io/v1alpha1
kind: ValkeyCluster
metadata:
  name: cluster-sample
spec:
  shards: 3
  replicas: 1
  tls:
    certificate:
      secretName: my-valkey-tls-secret

Once deployed, the Valkey operator will mount the referenced secret to each pod, and add all the proper configuration parameters. By doing so, the operator enforces SSL/TLS communication between each Valkey cluster node, securing node-to-node, and replication traffic within your kubernetes network. Additionally, by creating user certificates signed by the same CA, traffic between your clients, and the Valkey clusters nodes is secured. This configuration is BYOC (bring-your-own-certificate), which works well with the popular CertManager, or other certificate authority you may be using.

On The Horizon

As a teaser, here are a couple other features coming soon to Valkey Operator:

• Data Persistence: The ability to enable background snapshots of the in-memory dataset for backup, and recovery. Additionally, supporting the AOF (append-only file) for streaming changes.
• Simple Replication: The operator currently only supports Valkey in cluster mode. Be on the lookout for traditional primary -> N-replica configurations, along with Sentinel monitoring.

Join Us

Want to contribute to the Valkey Operator? Join any of the discussions/issues on our github, or come introduce yourself in the Valkey Slack community.

The post Managing Valkey Cluster in Kubernetes appeared first on Percona.

The post Managing Valkey Cluster in Kubernetes appeared first on MariaDB.org.

Archived, you mean EOL?

No! Open source software rarely has a hard “end of life.” What it does have are maintainership gaps and those can be just as serious.

It’s different when PostgreSQL community announces a major version EOL. This happens because Community chooses to not to support it and move on to focus on newer versions.

Reading the message from David Steele, the long-time primary maintainer of pgBackRest you will not find “end of life” term. The project is marked read-only and no longer actively maintained, but that is not the same as being permanently dead.

pgBackRest is not “end of life.” There is no governing body declaring support ended. What happened is simpler and more common in open source: the maintainer can no longer afford to continue.

So what happened then?

This requires some story telling and I don’t think that I can do it better than Lætitia Avrot already did in her blogpost (though I do not like the title):

Crunchy Data, which had sponsored pgBackRest for most of its life and employed David, was sold. After that, David spent months looking for a position that would let him keep working on the project. He also tried to secure independent sponsorship. Neither worked out. He needs to make a living. The project requires sustained effort which he can no longer provide without being paid for it.

This is the issue. An experienced developer, who wants to work on the project (that a big chunk of enterprises use) finds himself to be the “Nebraska guy” from XKCD comic.

When you look at the situation we’re in this is the classic “Nebraska guy problem”: critical infrastructure maintained by a single person. pgBackRest is widely used in production, yet its sustainability dependson one individual being able to justify working on it. That does not seem fair and David did right to point this out with his move.

Of course, if anyone in the community chose to, they can still maintain the project by forking it. But why, since the problem is elsewhere?

Most people understand that engineers need to be paid for their work. What not everyone realizes is that the free for all software that the open source license provides does not mean free as in beer. Someone still needs to fund it!

Unfortunately “someone” almost certainly is going to be “no-one” unless “anyone” realizes they are going to miss the software if nobody maintains it anymore.

While there’s a claim to be made that:

Companies are as good as they have to and as bad as they are allowed to

And often we see that an entity uses software they do not have to pay license fees for, treating this as cost optimization. There is also a large chunk of organizations that realize this is not a good long term strategy. Actively lowering the operational risk is important.

This is where foundations typically kick in: providing an easy way for organizations to contribute and ensure the longevity and healthiness of the projects. But PostgreSQL does not (yet) have one.

Where are we now?

There’s a lot of backchannel talks happening.

Join the ones on:

• Telegram
• Discord
• Slack
• Reddit (I don’t have enough karma to engage there :/)

or let us know what is your stance (Percona Community Forum thread available) so that we can represent you in the discussions we are having.

A lot of blog posts have been written on this subject, check out Planet PostgreSQL to find some of them! I particularly enjoyed some of them, the one from Stefanie Janine Stölting, I feel I am mostly aligned with. PostgreSQL needs an Ecosystem Umbrella Foundation

The future of open source is on us

Reading that a project is EOL is triggering to me. When long-time maintainer announced plans to step away after more than a decade of work, instead of focusing on what the problem is that caused him to do so and how to solve the issue.Naming it “dead” complicate things even further. Labeling the project as “dead” doesn’t solve the problem. Rather, it accelerates the wrong response. Users start looking for replacements instead of asking how to sustain the project.

This is not the way, young Padawan!

We need a body that helps both users and authors by:

1. Providing governance and a helping hand to the ecosystem. Yes, this is also funding
2. Providing guarantees of healthiness. This means users will have it easier to know the tools are in good shape.

So what’s with pgBackRest

While we talk here in the public, a lot of decisions are being made and Percona among other companies is working towards resolving this situation.

Have patience. Work is already underway behind the scenes, and the situation is evolving. There will be positive news resolving the situation coming soon, as Open Source doesn’t die!

The post Open source doesn’t die. It gets unfunded. appeared first on MariaDB.org.

Archived, you mean EOL?

No! Open source software rarely has a hard “end of life.” What it does have are maintainership gaps and those can be just as serious.

It’s different when PostgreSQL community announces a major version EOL. This happens because Community chooses to not to support it and move on to focus on newer versions.

Reading the message from David Steele, the long-time primary maintainer of pgBackRest you will not find “end of life” term. The project is marked read-only and no longer actively maintained, but that is not the same as being permanently dead.

pgBackRest is not “end of life.” There is no governing body declaring support ended. What happened is simpler and more common in open source: the maintainer can no longer afford to continue.

So what happened then?

This requires some story telling and I don’t think that I can do it better than Lætitia Avrot already did in her blogpost (though I do not like the title):

Crunchy Data, which had sponsored pgBackRest for most of its life and employed David, was sold. After that, David spent months looking for a position that would let him keep working on the project. He also tried to secure independent sponsorship. Neither worked out. He needs to make a living. The project requires sustained effort which he can no longer provide without being paid for it.

This is the issue. An experienced developer, who wants to work on the project (that a big chunk of enterprises use) finds himself to be the “Nebraska guy” from XKCD comic.

When you look at the situation we’re in this is the classic “Nebraska guy problem”: critical infrastructure maintained by a single person. pgBackRest is widely used in production, yet its sustainability dependson one individual being able to justify working on it. That does not seem fair and David did right to point this out with his move.

Of course, if anyone in the community chose to, they can still maintain the project by forking it. But why, since the problem is elsewhere?

Most people understand that engineers need to be paid for their work. What not everyone realizes is that the free for all software that the open source license provides does not mean free as in beer. Someone still needs to fund it!

Unfortunately “someone” almost certainly is going to be “no-one” unless “anyone” realizes they are going to miss the software if nobody maintains it anymore.

While there’s a claim to be made that:

Companies are as good as they have to and as bad as they are allowed to

And often we see that an entity uses software they do not have to pay license fees for, treating this as cost optimization. There is also a large chunk of organizations that realize this is not a good long term strategy. Actively lowering the operational risk is important.

This is where foundations typically kick in: providing an easy way for organizations to contribute and ensure the longevity and healthiness of the projects. But PostgreSQL does not (yet) have one.

Where are we now?

There’s a lot of backchannel talks happening.

Join the ones on:

• Telegram
• Discord
• Slack
• Reddit (I don’t have enough karma to engage there :/)

or let us know what is your stance (Percona Community Forum thread available) so that we can represent you in the discussions we are having.

A lot of blog posts have been written on this subject, check out Planet PostgreSQL to find some of them! I particularly enjoyed some of them, the one from Stefanie Janine Stölting, I feel I am mostly aligned with. PostgreSQL needs an Ecosystem Umbrella Foundation

The future of open source is on us

Reading that a project is EOL is triggering to me. When long-time maintainer announced plans to step away after more than a decade of work, instead of focusing on what the problem is that caused him to do so and how to solve the issue.Naming it “dead” complicate things even further. Labeling the project as “dead” doesn’t solve the problem. Rather, it accelerates the wrong response. Users start looking for replacements instead of asking how to sustain the project.

This is not the way, young Padawan!

We need a body that helps both users and authors by:

1. Providing governance and a helping hand to the ecosystem. Yes, this is also funding
2. Providing guarantees of healthiness. This means users will have it easier to know the tools are in good shape.

So what’s with pgBackRest

While we talk here in the public, a lot of decisions are being made and Percona among other companies is working towards resolving this situation.

Have patience. Work is already underway behind the scenes, and the situation is evolving. There will be positive news resolving the situation coming soon, as Open Source doesn’t die!

The post Open source doesn’t die. It gets unfunded. appeared first on MariaDB.org.

MariaDB has announced that September 30, 2026 will be the end-of-life date for continued maintenance and regular binary releases of MySQL Galera Cluster. We want to be clear about what this means for the organizations that rely on PXC: nothing is changing. Our commitment to PXC and the community that runs it is as strong as ever.

What is ending upstream is precisely what we already have in place. For anyone looking for an alternative path forward, PXC is the natural place to land.

What PXC users can count on

• Our open Galera fork: Percona maintains its own Galera repository, open today and staying that way. We track upstream Galera releases, carry the fixes our customers need, and keep the codebase fully available for the community. PXC is built on this work, on terms we control.
• Regular releases at the current cadence: Binary releases, bug fixes, and security patches continue to ship on the same terms and schedule our users have come to expect. You can review our full release history and release notes on the Percona documentation site.
• Long-term support: PXC remains fully supported under our existing long-term support terms. If your organization is planning three to five years ahead, PXC is a safe foundation for those plans.
• Compatibility and ecosystem integration: Strong binary compatibility with MySQL and Percona Server for MySQL, tight integration with Percona XtraBackup and Percona Monitoring and Management, and continued support across Kubernetes and traditional deployment environments.

What we’re continuing to invest in

Our engineering teams remain committed to making PXC better, focused on the things that make it a trusted choice: performance, stability, security, and a smooth operator experience. That work continues at pace. The PXC you depend on today will keep getting better, and the PXC you are evaluating for tomorrow will be ready when you need it.

Talk to us

If you have specific questions about your PXC deployment, your upgrade path, or your long-term high availability strategy, we’d love to hear from you. Reach out to your Percona contact, post a question in the Percona community forums, or connect with our team directly. High availability is too important to leave to uncertainty, and we are here to make sure you have the clarity and the support you need.

The post Continued Commitment to Percona XtraDB Cluster appeared first on Percona.

The post Continued Commitment to Percona XtraDB Cluster appeared first on MariaDB.org.

Continue reading “From Ecosystem to Architecture: Expanding How We Look at MariaDB”

The post From Ecosystem to Architecture: Expanding How We Look at MariaDB appeared first on MariaDB.org.

The post From Ecosystem to Architecture: Expanding How We Look at MariaDB appeared first on MariaDB.org.

Continue reading “Adding a New Data Type to MariaDB with Type_handler – Part 1”

The post Adding a New Data Type to MariaDB with Type_handler – Part 1 appeared first on MariaDB.org.

The post Adding a New Data Type to MariaDB with Type_handler – Part 1 appeared first on MariaDB.org.

The Case

The case that triggered this blog was an attempt to migrate from one cloud vendor, to a recent version of PostgreSQL on a DBaaS offering of another cloud vendor. They started observing huge replication lag and reported to Percona. As usual, we started with pg_gather data collection.

(At Percona, we use pg_gather for diagnosis. Even though this blog and diagnosis refers to pg_gather report, any good diagnosis tool / scripts which can help to study the wait-event pattern and lag details could be able to help)

We saw upto 4.5 terabyte lag is happening at the transmission side (Publisher) on the customer case. The “Transmission”  lag” is the difference between the latest generated LSN and the LSN which the WAL Sender is able to send (sent_lsn of pg_stat_replication). That’s a first indication that the problem is mainly at the publisher side (WAL Sender) and it is not able to send the information fast enough.

Next step of investigation is to understand what both those WAL Senders might be doing. The wait event information for each WAL Sender could provide a clear clue on where the delay is happening.

Both the WAL Senders are mainly waiting in “WalSenderWriteData” event upto 85% of its time. This is a very unusual level of wait.

Following is the logic behind this.

1. Logical decoding hands a finished record to
   WalSndWriteData()
2. The data is queued in the
   libpq  send buffer with
   pq_putmessage_noblock
3. A non-blocking flush is attempted with
   pq_flush_if_writable() .  But when the kernel send buffer is full, internal_flush_buffer() returns 0 with EAGAIN / EWOULDBLOCK  and data stays buffered —
   pq_is_send_pending()  becomes
   true
4. The fast-path return is skipped, so it enters
   ProcessPendingWrites()
5. There,
   WalSndWait(WL_SOCKET_WRITEABLE | WL_SOCKET_READABLE, sleeptime, WAIT_EVENT_WAL_SENDER_WRITE_DATA)  blocks until the socket is writable again or the subscriber sends a reply — that wait is what shows up as wait-event WalSenderWriteData

Source code reference : src/backend/replication/walsender.c

Which means that this wait event could happen due to the following reasons which I could think about. I would appreciate your comments if you know more reasons.

1. The subscriber’s apply worker not consuming the stream fast enough (apply lock contention, slow I/O, heavy CPU load on the subscriber) — its TCP receive buffer fills up, the TCP window shrinks to zero,  the publisher’s
   send()  returns EAGAIN, and the WAL sender spins in the loop.
2. Saturated network bandwidth between publisher and subscriber. There we have two cases. Traffic originating from Publisher to Subscriber could be slow OR handshake/acknowledgement traffic from Subscriber back to Publisher could be slow. The symptoms may differ.
3. Large decoded transactions produce bursts of WalSndWriteData calls faster than the network can absorb them. This is generally a temporary problem and the cluster might catchup once the overhead of the large transaction is over.

Now the question would be : Now we know all the probable causes, but how to narrow down to the most probable cause ?, so that we can have an action plan.

At Percona, our engineers take time to put all hypotheses for testing, trying to simulate similar conditions and produce observable and reproducible cases. One might argue that we can use low level tracing / OS level tools at this stage to narrow down. Definitely, Yes. That’s the most appropriate thing to do. However many of the users may not have low level access and DBaaS offerings prevent it by design. But the good news is that wait – event patterns can tell us a story in more detail.

Slow Network traffic from Publisher to subscriber

If the network connectivity from Publisher side is slow or suffering with high latency, the send buffer won’t get cleared fast enough. Resulting in repeated attempts to send the data.

When we simulated the situation in the lab, we observed the similar Transmission side lag

Since this is a network connection problem, automatically the acknowledgment coming from the subscriber side will also be delayed and expected to show lags in all Write, Flush and Replay stages, which is visible in the data collection.

Obviously, our next question is what WALSender is doing ?. The wait events reveal that

The WAL sender is struggling to send data to Standby. This matches with what the user was seeing in their database.

Meanwhile, at the subscriber side, what we could see is that the apply worker is majorly sitting idle in the main loop: LogicalApplyMain

Two other major symptoms to be noted in the cases is 1). much smaller “Write lag” and compared to “Transmission lag” and  2.). Both the subscribers are suffering the lag, which is less probable if the problem is on the subscriber side.  Even additional clues like data collection running longer when executed from the publisher side about the subscriber instance is also supplementary evidence.

All the above symptoms helps us to conclude with a reasonable level of confidence that the network traffic from the Publisher is the problem.

Overloaded or slow Subscriber node

The problem could be caused by subscriber not communicating fast enough with Publisher. In such cases the replication lag is expected. I tested that scenario and the following is the observation.

The cause of the lag shifts from the “Transmission lag” to “Replica Write Lag”, which  is the difference between send_lsn and write_lsn.

The WAL Sender / publisher side don’t have any problem in sending

That looks really cool. The major wait event is WalSenderWaitForWal. Which means that the WAL sender is just waiting for the next WAL to be flushed and ready, In other words, sleeping until the next commit.

However, the situation on the Subscriber side is different. Unlike in the previous case, the apply worker could be busy with all sorts of work, no more free time to wait in the main loop.

The wait events and their percentage may vary depending on the performance-bottleneck on the subscriber side.

Slow traffic from Subscriber to Publisher

The impact of the network traffic from the Subscriber side back to Publisher has less effect than that from Publisher side. Because the data flow is from Publisher to Subscriber. The Subscriber needs to send only handshake acknowledgment information back to the publisher, which requires less bandwidth. So an apply worker can be waiting in the main loop (LogicalApplyMain)

But contrary to expectations, there could be cases of significant CPU usage if there are repeated attempts to reach primary, it may be consuming significant CPU cycles.  If we are suspecting network problems from the subscriber side, paying close attention to PostgreSQL logs is important.

There can be timeout captured in PostgreSQL logs at the publisher side

2026-04-23 17:34:07.078 UTC [36057] postgres@postgres LOG:  terminating walsender process due to replication timeout
2026-04-23 17:34:07.078 UTC [36057] postgres@postgres CONTEXT:  slot "sub", output plugin "pgoutput", in the commit callback, associated LSN DA/70450B8
2026-04-23 17:34:07.078 UTC [36057] postgres@postgres STATEMENT:  START_REPLICATION SLOT "sub" LOGICAL D9/DF13B50 (proto_version '4', streaming 'parallel', origin 'any', publication_names '"pub"')

Corresponding errors might be appearing at subscriber side also

2026-04-23 17:36:00.933 UTC [38413] ERROR:  could not receive data from WAL stream: SSL connection has been closed unexpectedly
2026-04-23 17:36:00.940 UTC [38428] LOG:  logical replication apply worker for subscription "sub" has started
2026-04-23 17:36:00.946 UTC [18241] LOG:  background worker "logical replication apply worker" (PID 38413) exited with exit code 1

All these are indications of poor connection.

Summary

PostgreSQL Wait events provide lots of visibility into PostgreSQL and underlying infrastructure problems. Reading it with PostgreSQL statistics information (pg_stat_*) and PostgreSQL log  could provide answers for a lot of questions as follows easily.

1. Where is the problem ?. Which side of replication is lagging ?
2. What are WAL Senders and WAL Receivers doing ? Are they busy doing something ? Is there anything pending or sitting idle ?
3. Are there any hits of underlying infrastructure problems ?
4. Is the problem correlatable with the wait events ?

Collecting and correlating the wait-event data from both sides of the replication, checking the LSN differences,  and information from PostgreSQL logs  gives us a complete picture.

All it takes is just a couple of minutes! With the right tools and methods in hand. I would like to encourage readers to make use of wait event pattern analysis for easy spotting of performance bottlenecks, if you are not doing it already. It can save you from the treachery of all indepth tracing. Observed Replication lag can be treated as a symptom of something more serious underlying.

This blog is written for those who don’t have access to OS level, But if we have access, getting into details is far easier. For example, the send queue (Send-Q) of the Publisher host can be checked like:

postgres@node0:~$ ss -tnp
State            Recv-Q            Send-Q                         Local Address:Port                       Peer Address:Port             Process
ESTAB            0                 18791538                          172.18.0.2:5432                         172.18.0.3:44974             users:(("postgres",pid=48178,fd=11))

The post Troubleshooting logical replication delay made easy appeared first on Percona.

The post Troubleshooting logical replication delay made easy appeared first on MariaDB.org.

In this blog post, we follow the same example, but instead of the success story, we explore how OAuth keeps our PostgreSQL servers secure.

We won’t focus on complex attack vectors, like the examples in the second blog post in the OIDC series.
Instead of social engineering, we’ll look at practical errors, misconfigurations and honest mistakes – understanding error messages and how to fix them.

Improved test setup

Along with the step by step tutorial, previously we also linked a docker/podman compose configuration.
This is still available, and we even improved it for testing the error scenarios.

If you want to update the configuration manually instead, this is what changed: we duplicated everything!

• Instead of a single testuser, we have two: testuser and testuser2, both using the same asdfasdf password
• Instead of one client, we have two: pgtest and pgtest2
• Instead of one scope, we have two: pgscope, pgscope2
• Instead of one realm, we have two – containing exactly the same setup: pgrealm and wrongrealm

A role named pgrole is also defined.
The pgtest2 client and pgscope2 scope both require the pgrole role.
Only testuser2 is assigned this role; testuser does not have it.

The following table summarizes the access matrix:

Success despite a FATAL error?

However, before we start using all these additional items, let’s go back to the end of the Keycloak story, where we succeeded in logging in.
Or did we?
While psql logged us in, if we checked the server error log, we could see the following there:

FATAL: OAuth bearer authentication failed for user "testuser"

But if we are observant enough, this message is logged before we even go to the device authentication website of Keycloak, and enter the authentication code.
This isn’t a real error, it’s just a side effect of how OAuth is implemented internally, and will most likely be fixed in PostgreSQL 19, the FATAL message will no longer show up.

As for PostgreSQL 18, unfortunately, we have to live with this.
This also means that we can’t rely on simply looking for OAuth authentication errors in the server log, because all OAuth authentication failures will result in exactly the same message, no matter if they are logged because of this harmless situation or because of a real authentication issue.

A workaround is to rely on the validators instead: since the server is unaware of the exact error situation anyway – it delegates validation to the validator – these plugins will print out much more detailed log messages.
We’ll see some examples later with pg_oidc_validator, as we explore the error scenarios.

However, keep in mind that the sentence above says log messages, and not errors or fatal errors.
Validators are not allowed to print out ERROR and FATAL messages for authentication failures, so users have to look for WARNING or LOG level messages for the details.
In practice, PostgreSQL will still print the same generic FATAL error message about OAuth bearer authentication failing – but it will appear after the validator-specific WARNING or LOG messages that contain the actual diagnostic information.

Why does it happen?

Earlier we already established that PostgreSQL validates that the client and the server use the same issuer, but didn’t go into more detail than this.

Usually when a service validates user input, it does so on the server.
The main reason for this is that developers can trust the backend, controlled by administrators, while they can’t trust the frontend, potentially used by malicious users.

But are we validating user input in this case?
Why does the user even have to specify the issuer, since the server already knows it, it’s in the HBA configuration?

Because this check isn’t the server validating the user, it’s the opposite:
the user validating the server.

1. The client sends an empty connection request to the server.
2. The server confirms that we are using OAuth, and sends back its issuer URL.
3. The client checks whether the issuer it is planning to use – or has already used, if it already has a valid token – matches the one sent by the server.
   • If it doesn’t match, it aborts the login attempt and prints an error.
   • If it does match, it continues with a real authentication attempt.

sequenceDiagram
participant C as psql (Client)
participant S as PostgreSQL Server
participant K as Keycloak
C->>S: Connection request (no token)
S-->>S: No token, auth fails
Note right of S: FATAL logged here (harmless side effect)
S->>C: OAuth challenge + issuer URL
C-->>C: Compare issuer URL with oauth_issuer
alt Issuer mismatch
C-->>C: Abort with error
else Issuer matches
C->>K: Device authorization flow
K->>C: Access token
C->>S: Connection request (with token)
S->>S: Validator checks token
S->>C: Authentication success
end

The FATAL error in the server log is a side effect of the first empty authentication attempt, that wasn’t fixed in time before the PG18 release.

Why do we need this check?

With the improved configuration, we can test what happens when we specify the wrong issuer:

bin/psql -h 127.0.0.1 'dbname=postgres oauth_issuer=https://keycloak:8443/realms/wrongrealm oauth_client_id=pgtest'
psql: error: connection to server at "127.0.0.1", port 5432 failed: server's discovery document at https://keycloak:8443/realms/pgrealm/.well-known/openid-configuration (issuer "https://keycloak:8443/realms/pgrealm") is incompatible with oauth_issuer (https://keycloak:8443/realms/wrongrealm)

Notice that compared to the correct command, which had the pgrealm, we are using the other Keycloak realm.
And if we check the server log, we can see that there are no additional log messages there – we see a single OAuth FATAL error, which is not a real error, just the side effect we are investigating.
This error is entirely on the client side.

And that brings us back to the question:
why do we need this?

On one hand, it helps us prevent honest mistakes early.
In our example, wrongrealm and pgrealm are exactly the same – they have users with the same name, scopes with the same names, clients with the same names.
If there’s a misconfiguration, and the server and the client use different realms in a similar setup, everything would seem to work – the user trying to log in would be able to log in, get a token, psql would send it to the server…
and then on the server the validator would reject it – assuming that it is a good validator, like pg_oidc_validator.
No harm done – other than disclosing a token to the server that shouldn’t have been sent there –, but figuring out what the problem is could take a while.

On the other hand: what if we aren’t dealing with a malicious user, but a malicious server?

In the previous attack vectors we showcased, the attacker was always a third party:
somebody who wanted to steal access to the database server.

But we don’t necessarily need a different unknown adversary, it could be the server we are using:
do we absolutely know and trust its administrators?
Sometimes yes, sometimes no.

Those administrators might be aware that we are also using OAuth for something else, and might plot to gain access to it.
So instead of sending us the issuer we expect, the server sends us something else – for example a spoofed site, tricking us to complete login into a different service.

Remember the earlier situation where the Fake Photo Gallery Website used Client ID spoofing to gain access to the PostgreSQL Database?
This situation is basically the same – the only difference is that this time PostgreSQL Database is trying to gain access to Photo Gallery.

sequenceDiagram
participant U as User (psql)
participant M as Malicious PostgreSQL Server
participant SSO as Other service
U->>M: Connection request
M->>U: OAuth challenge + spoofed issuer URL (instead of expected issuer)
Note over U: Without issuer check, user proceeds
U->>SSO: Authenticates, thinking it is for PostgreSQL
SSO->>U: Access token (valid for a different service)
U->>M: Sends token to server
Note over M: Malicious server now holds a token valid for the other service

The client-side issuer check prevents this: psql compares the issuer URL from the server against the oauth_issuer it was configured with, and aborts if they don’t match.

While requiring the client to specify the issuer may seem redundant, it’s an important safeguard that prevents tokens from being disclosed to malicious servers.

Can we verify the validator?

If we specify an incorrect issuer, the client rejects it before completing the OAuth flow – that’s great, but this means the validator isn’t part of the picture.
Can we even test that a validator handles this situation correctly, to verify that it properly rejects an attempt with an incorrect issuer?
Security-aware users might want to double check that somebody using a modified psql is also properly rejected.

This is possible: in our next blog post, we’ll see how to implement custom clients outside psql, possibly using other OAuth flows.
In that scenario, we’ll be able to send custom tokens to the server, which has many uses – one of which is internal testing of OAuth validators.

Rest assured, pg_oidc_validator handles this correctly – and we’ll show you how to verify it yourself in the next post.
If you are using a different validator, stay tuned to see how you can verify it!

With pg_oidc_validator, you will see something like this in the server log:

WARNING: OAuth validation failed with exception: claim value does not match expected value
FATAL: OAuth bearer authentication failed for user "testuser"
DETAIL: Connection matched file "/pg_hba.conf" line 119: "host all all 127.0.0.1/32 oauth issuer=https://keycloak:8443/realms/pgrealm,scope="pgscope email",map=kcmap"

Here “claim value does not match expected value” means that a field (claim) in the JWT doesn’t match our expectation.
While this might seem generic, currently pg_oidc_validator only validates exactly one field in this way: the issuer.

On the client side, you can see the following generic error message:

Connection error: connection to server at "127.0.0.1", port 5432 failed: retrying connection with new bearer token
connection to server at "127.0.0.1", port 5432 failed: FATAL: OAuth bearer authentication failed for user "testuser"

Which is generally true for most OAuth errors – validators are expected not to provide detailed information about why they reject a connection back to the client, to limit the information available to potential attackers.

Signature failure

For some it might be surprising that we are getting an error about the issuer, and not about the token.
Why is that?

The reason we use JWTs for access tokens is because they are cryptographically signed tokens.
While they contain the payload in clear text, the token ends with a signature, a proof that it was generated by the issuer we trust.

This means that if the token was generated by a different issuer, it is signed by a different key.

However, the order of operations inside the validator is different:
first we validate the fields in the cleartext data we have strong expectations about – in this case the issuer.
Then, after that’s valid, we also verify that the signature matches the public key of the issuer.