Build an “ask the docs” app with Laravel 13 and MariaDB Vector

Earlier this month we wrote that Laravel 13 now supports MariaDB Vector natively. Mohamed Touhami Rhaima, one of the community contributors who made that happen, has now written a tutorial on how to use it, and recorded a video version.

Why MariaDB Vector in Laravel

With Laravel 13 and MariaDB 11.7 or later, you can store embeddings and search them by meaning in the same database as the rest of your application data, using the framework’s own database features.

This tutorial walks through a small reference application. It reads markdown files, splits them into passages, turns each passage into an embedding, stores it in MariaDB, and answers questions by finding the closest passages and handing them to an AI agent. The finished code is on GitHub.

Laravel 13 supports MariaDB Vector natively, so you need no extra database package and no raw SQL:

• $table->vector() and $table->vectorIndex() create the column and its index in a migration.
• The AsVector cast reads and writes the vector as a PHP array.
• selectVectorDistance() and whereVectorSimilarTo() run the similarity search in the query builder.
• Str::of($text)->toEmbeddings() turns text into an embedding through the Laravel AI SDK.

By the end, you will have an ask command and a small HTTP endpoint that answer questions from your own documents, with the source of each answer.

This tutorial targets Laravel 13. If you are still on Laravel 12, the community package devilsberg/laravel-mariadb-vector by Erik Ros adds vector support for MariaDB. Its method names are similar to the native ones but not identical, so follow its own README on Laravel 12.

Set up

You need PHP 8.3 or later, Composer, Docker, and an OpenAI API key (or another embedding provider supported by the Laravel AI SDK). The steps below were tested on a clean install with PHP 8.4, Composer 2.10 and Docker 29. Clone the repository and install the dependencies:

git clone https://github.com/Rhaima96/laravel-mariadb-vector.git
cd laravel-mariadb-vector
composer install
cp .env.example .env && php artisan key:generate

Start MariaDB with Docker. The compose file runs MariaDB 12.3 and publishes it on port 3308, so a MySQL server you already run on 3306 keeps working:

docker compose up -d

Then open .env and add your key. The database settings already match the compose file:

DB_CONNECTION=mariadb
DB_HOST=127.0.0.1
DB_PORT=3308
DB_DATABASE=vectordemo
DB_USERNAME=vector
DB_PASSWORD=secret

OPENAI_API_KEY=sk-...

Never commit your .env file: it holds your API key.

Use the mariadb driver, not mysql. Laravel can connect to a MariaDB server through either driver, but only the mariadb driver knows MariaDB’s SQL. With the mysql driver against a MariaDB server, vector indexes cannot be created, vector distance queries are disabled, and uuid() columns become char(36) instead of MariaDB’s native uuid. The DB_CONNECTION=mariadb line above is what you want.

The schema

The whole database side is one migration. Each row of the documents table holds one passage of text, the file it came from, and its embedding:

Schema::create('documents', function (Blueprint $table) {
    $table->id();
    $table->string('source');
    $table->text('content');

    // 1536 dimensions: the size of OpenAI's text-embedding-3-small output.
    // MariaDB rejects any vector of a different length at insert time.
    $table->vector('embedding', 1536);

    $table->timestamps();

    // MariaDB: vector index ... M=6 DISTANCE=cosine.
    $table->vectorIndex('embedding');
});

The number in vector('embedding', 1536) must match your embedding model. If you switch to a model with a different output size, change this number and re-run the migration.

vectorIndex() builds an HNSW index with cosine distance. Three rules follow from that, and all three are easy to trip over:

1. The indexed column must be NOT NULL. Do not add ->nullable() to it.
2. A table can have only one vector index.
3. MariaDB uses the index only for a query that sorts by the same distance function as the index, in ascending order, with a LIMIT. Here that is cosine distance, and the search in the next sections is built exactly that way.

Run the migration:

php artisan migrate

Do not call Schema::ensureVectorExtensionExists(). It appears in the migration example of Laravel’s AI SDK documentation, but it is a helper for PostgreSQL’s pgvector extension. On MariaDB it throws Extensions are only supported by Postgres. MariaDB has vector support built in and needs no extension, so leave that line out.

The model

The Document model needs one thing beyond a normal Eloquent model: the AsVector cast on the embedding column.

use Illuminate\Database\Eloquent\Casts\AsVector;
use Illuminate\Database\Eloquent\Model;

class Document extends Model
{
    protected $fillable = ['source', 'content', 'embedding'];

    protected function casts(): array
    {
        // Reads MariaDB's little-endian float32 bytes as an array of floats,
        // writes an array back through vec_fromtext().
        return ['embedding' => AsVector::class];
    }
}

In your code, $document->embedding is a plain PHP array of floats. MariaDB stores it as a compact binary vector, and the cast converts in both directions, so you never write the conversion yourself.

This is enough to store embeddings. The next section fills the table with the passages of your own documents.

Ingest your documents

The content/ folder holds three short markdown files about embeddings, MariaDB Vector and the Laravel mariadb driver. The ingest command reads each file, splits it into passages, embeds every passage and stores it:

php artisan ingest

The core of the command is a single loop. Str::of($chunk)->toEmbeddings() calls your embedding provider through the Laravel AI SDK and returns the vector, and the AsVector cast stores it:

foreach ($this->chunk(file_get_contents($file)) as $chunk) {
    Document::create([
        'source' => basename($file),
        'content' => $chunk,
        'embedding' => Str::of($chunk)->toEmbeddings(),
    ]);
}

Before embedding, the text is split into passages. The command splits on blank lines and packs paragraphs together up to about 600 characters, so a passage rarely cuts a sentence in half. This is simple on purpose: it works well for markdown prose, and you can replace it with a token-aware splitter if your documents need it.

Why split at all? A question is usually answered by one paragraph, not a whole file. Small passages give the search something precise to match and give the agent only the context it needs.

The command empties the table before it starts, so you can edit the files and run it again. Each run calls your embedding provider once per passage, which costs very little for this sample content. When it finishes, it prints how many passages it stored.

Search by meaning

Searching is a method on the model. It takes a question and returns the closest passages, nearest first, each with a distance attribute:

public static function search(string $question, int $limit = 4, float $minSimilarity = 0.25): Collection
{
    $vector = Str::of($question)->toEmbeddings();

    return static::query()
        ->select('id', 'source', 'content')
        ->selectVectorDistance('embedding', $vector, as: 'distance')
        ->whereVectorSimilarTo('embedding', $vector, minSimilarity: $minSimilarity)
        ->limit($limit)
        ->get();
}

Three things happen in this query:

• selectVectorDistance() adds a distance column to each result. On MariaDB it runs VEC_DISTANCE_COSINE, so a smaller number means a closer passage.
• whereVectorSimilarTo() keeps only the passages that are similar enough to the question, and orders them from nearest to farthest. The minSimilarity argument is the cut-off: raise it for stricter matches, lower it to get more results.
• limit() keeps the best few.

One consequence of the limit: MariaDB’s index returns at most 4 rows before the cut-off is applied, so the cut-off filters the 4 nearest passages. The search can return fewer than 4 passages, but never more.

The question is embedded once, and the same vector is reused for the distance column and for the filter. Laravel’s documentation shows that whereVectorSimilarTo also accepts a plain string and embeds it for you, but the distance column needs the vector as well, so that would embed the same question twice.

Try it with the ask command, which prints the matches before it answers:

php artisan ask "why should I not use the mysql driver with MariaDB?"

The command lists the matched files with their distances. For this question, the four closest passages come from laravel-driver.md (three of them) and mariadb-vector.md, with distances between 0.33 and 0.60. The smaller the distance, the closer the passage is to the question.

Answer with an agent

The search gives you passages, not an answer. The last step passes those passages to an AI agent, which writes the answer using only what you found. In the Laravel AI SDK, an agent is a small class:

final class DocsAgent implements Agent
{
    use Promptable;

    public function __construct(private Collection $context) {}

    public function instructions(): string
    {
        return "Answer the question using only the context below. Quote the source file name you relied on. If the context does not contain the answer, say so plainly instead of guessing.\n\n"
            .$this->context
                ->map(fn ($document) => "### {$document->source}\n{$document->content}")
                ->implode("\n\n");
    }
}

The matched passages become part of the agent’s instructions, each under its source file name. The instructions tell the agent to answer only from that context, to name the file it used, and to say so when the context does not contain the answer. This keeps answers tied to your documents and makes them easy to check.

Using the agent takes one line. The ask command does this after it prints the matches:

$answer = (new DocsAgent($matches))->prompt($question)->text;

For the question above, the agent explains that the mysql driver can appear to work with MariaDB but fails on some features, such as vector queries and indexes, and that Laravel provides a dedicated mariadb driver for these differences. It ends with Source: laravel-driver.md. The exact wording changes from run to run.

This pattern is called retrieval-augmented generation: MariaDB finds the relevant text, and the model writes the answer from it. Your documents live in your database, and at question time only the few passages that matched are sent to the chat model. Keep in mind that your embedding provider still receives the text you embed during ingestion.

Expose it as an API

The same search and agent can sit behind an HTTP endpoint. First enable API routes in bootstrap/app.php:

->withRouting(
    web: __DIR__.'/../routes/web.php',
    api: __DIR__.'/../routes/api.php',
    commands: __DIR__.'/../routes/console.php',
    health: '/up',
)

Then add one route in routes/api.php:

Route::post('/ask', AskController::class)->middleware('throttle:10,1');

The controller validates the question, runs the same Document::search() as the command, and returns the answer with its sources:

public function __invoke(Request $request): JsonResponse
{
    $question = $request->validate(['question' => ['required', 'string', 'max:500']])['question'];

    $matches = Document::search($question);

    if ($matches->isEmpty()) {
        return response()->json(['message' => 'Nothing similar enough in the ingested documents.'], 404);
    }

    return response()->json([
        'answer' => (new DocsAgent($matches))->prompt($question)->text,
        'sources' => $matches->map(fn (Document $match) => [
            'source' => $match->source,
            'distance' => round((float) $match->distance, 4),
        ])->values(),
    ]);
}

Start the server and call it:

php artisan serve

curl -s -X POST localhost:8000/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question":"why should I not use the mysql driver with MariaDB?"}'

The response has an answer like the command’s, plus the sources that backed it:

{"answer": "The `mysql` driver can appear to work with MariaDB, but ... Source: `laravel-driver.md`.",
 "sources": [{"source": "laravel-driver.md", "distance": 0.329}, {"source": "laravel-driver.md", "distance": 0.467}, ...]}

A missing question returns a 422 validation error, and a question that matches nothing in your documents returns a 404.

Before you expose this endpoint. It is a demo, so it has no authentication, and every call spends embedding and chat tokens on your account. The throttle of ten requests per minute limits the cost but does not remove it. Add authentication before you put it on a public server, and set APP_DEBUG=false in production: .env.example has it set to true, and a debug error page shows a full stack trace.

Run the embeddings locally with Ollama

You can swap the embedding provider for a local one, so that the text of your documents never leaves your machine during ingestion. This section uses Ollama with the nomic-embed-text model, which produces 768-dimensional embeddings. It was tested on a fresh clone of the repository with MariaDB 12.3.

One limit to keep in mind: only the embeddings become local. The agent that writes the answer still uses the chat provider from config/ai.php, which is OpenAI unless you change it, so at question time the matched passages and the question still go to that provider. Local chat models are outside the scope of this tutorial.

Install Ollama, start it, and download the model. The commands below are for macOS with Homebrew; on other systems, follow the install page on ollama.com:

brew install ollama
brew services start ollama
ollama pull nomic-embed-text

Check that the model answers, and that the vector has the size you expect:

curl -s localhost:11434/api/embed -d '{"model":"nomic-embed-text","input":"hello"}' \
  | python3 -c 'import sys,json; d=json.load(sys.stdin); print(len(d["embeddings"][0]))'

The output should be 768. Now make three changes in the project.

1. Choose Ollama for embeddings. Publish the AI SDK configuration and change the default embeddings provider in config/ai.php:

php artisan vendor:publish --tag=ai-config

'default_for_embeddings' => 'ollama',

The SDK reaches Ollama at http://localhost:11434 and uses nomic-embed-text as its default embeddings model. If Ollama runs on another machine, set OLLAMA_URL in .env.

2. Change the vector size in the migration. The column must match the model, so replace 1536 with 768:

$table->vector('embedding', 768);

Then rebuild the table and ingest again. migrate:fresh drops every table of the database, which is fine for this demo:

php artisan migrate:fresh
php artisan ingest

The rebuild is required, not optional. Embeddings from different models are not comparable, so a table must hold the vectors of one model only. If you switch models later, re-ingest everything.

3. Raise the similarity cut-off. Distances are not on the same scale from one model to another. With OpenAI’s text-embedding-3-small, the relevant passages of the sample question were between 0.33 and 0.60, and the default minSimilarity of 0.25 (maximum distance 0.75) separates them from unrelated text. With nomic-embed-text, the same question matched at 0.23 to 0.38, while an unrelated question (best chocolate cake recipe) still matched at 0.64 to 0.66. Under the default cut-off, those unrelated passages are returned, and the API never answers with a 404.

In Document::search(), raise the default to 0.45, which keeps a maximum distance of 0.55:

public static function search(string $question, int $limit = 4, float $minSimilarity = 0.45): Collection

The ask command takes the same value as an option, so you can try other cut-offs without editing code: php artisan ask "..." --similarity=0.45.

Now repeat the earlier tests. The relevant question returns its four passages and an answer, with distances between 0.23 and 0.38:

curl -s -X POST localhost:8000/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question":"why should I not use the mysql driver with MariaDB?"}'

The unrelated question now returns 404 Not Found:

curl -s -i -X POST localhost:8000/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question":"best chocolate cake recipe"}' | head -n 1

The value 0.45 fits this small sample and this model. Treat it as a starting point: run a few questions you know your documents can and cannot answer, look at the distances the ask command prints, and place the cut-off between the two groups.

Where to go next

You now have the full loop: a vector column and index in MariaDB, embeddings stored through an Eloquent cast, a similarity search in the query builder, and an agent that answers from the results. From here you can ingest your own documentation, change the number of passages sent to the agent, or tune the similarity cut-off to your content. If you switch embedding model, read Erik Ros’s MariaDB Vector in Laravel: insights on choosing an embedding model first: the model decides how good your search results are.

To use this in your own project rather than the cloned repository, install the AI SDK with composer require laravel/ai.

---

Mohamed Touhami Rhaima  is a full-stack Laravel and Vue.js developer in Gabès, Tunisia. He contributed MariaDB support for vector distance queries to Laravel 13, and he publishes open-source Laravel packages on Packagist.*