How to Choose an Embedding Model: In-Database vs Gemini

Dimensions, benchmarks, and prices do not tell you which embedding model finds the right answers in your data. A test does: a set of questions whose right answers you know, run against each candidate model.

This guide builds such a test in Oracle AI Database 26ai and uses it to compare a small model running inside the database, all-MiniLM-L12-v2, with Google's gemini-embedding-001, in English and in four other languages. It ends with practical rules for choosing.

Code for This Guide

The examples are files 08 to 10 in the examples/ch06 folder of the Oracle AI code repository on GitHub, each with its output. They use the sample help desk schema from setup/atlas, whose 24 knowledge base articles already have an embedding from each model.

These examples come from AI Applications with Oracle Database 26ai and APEX 26.1, a book of 237 tested examples of AI in Oracle Database and APEX.

The articles were embedded, and the EMBED function the test calls was created, in how to generate embeddings in SQL with VECTOR_EMBEDDING and how to generate embeddings with Gemini from PL/SQL.

The Two Candidates

Build an Evaluation Set

The evaluation set is a table of questions customers might ask, each with the article that answers it. This one has 28 questions: 20 in English and 8 in Spanish, German, French, and Portuguese.

Write the questions in your users' words, not the articles' words. "The deals board just spins" tests whether a model knows it means "Pipeline page does not load"; copying the article title would only test word matching.

The table also stores each question's embedding from both models, so the comparisons below never call Gemini again.

Example:

create table eval_questions (
  question_id  number         constraint eval_questions_pk primary key,
  lang         varchar2(2)    default 'en' not null,
  question     varchar2(200)  not null,
  article_id   varchar2(10)   not null constraint eval_questions_article_fk
                                references kb_articles,
  minilm       vector(384, float32),
  gemini       vector(3072, float32)
);

insert into eval_questions (question_id, question, article_id) values
  ( 1, 'I typed my password wrong a few times and now I am locked out', 'KB-101'),
  ( 2, 'The e-mail to reset my password never showed up',                'KB-102'),
  ( 3, 'The codes from my authenticator app are always rejected',        'KB-103'),
  ( 4, 'How can our accountant see invoices without seeing the CRM?',    'KB-104'),
  ( 5, 'I got an alert about a login that was not me',                   'KB-105'),
  ( 6, 'Our card was charged two times for one invoice',                 'KB-201'),
  ( 7, 'Where can I get PDF copies of all our bills?',                   'KB-202'),
  ( 8, 'We are exempt from sales tax but still pay it',                  'KB-203'),
  ( 9, 'What do we pay if we move to a bigger plan mid-month?',          'KB-204'),
  (10, 'Can we print our PO number on the invoice?',                     'KB-205'),
  (11, 'The deals board just spins and never shows anything',            'KB-301'),
  (12, 'After importing a spreadsheet we have the same person twice',    'KB-302'),
  (13, 'The phone app closes right after I open it',                     'KB-401'),
  (14, 'I edited a record on the plane and my change was lost',          'KB-402'),
  (15, 'Our weekly e-mailed report stopped coming',                      'KB-501'),
  (16, 'Revenue in the dashboards does not match the invoice list',      'KB-503'),
  (17, 'Our webhook stopped firing',                                     'KB-601'),
  (18, 'The API keeps answering 429 Too Many Requests',                  'KB-602'),
  (19, 'Meetings do not show up in Outlook',                             'KB-603'),
  (20, 'How do we install the desktop app on 300 laptops?',              'KB-702');

-- questions in Spanish, German, French, and Portuguese
insert into eval_questions (question_id, lang, question, article_id) values
  (21, 'es', 'Nos cobraron dos veces la misma factura',                  'KB-201'),
  (22, 'es', 'La aplicación del móvil se cierra nada más abrirla',       'KB-401'),
  (23, 'de', 'Nach mehreren falschen Passwörtern bin ich gesperrt',      'KB-101'),
  (24, 'de', 'Unser wöchentlicher Bericht kommt nicht mehr per E-Mail',  'KB-501'),
  (25, 'fr', 'Où puis-je télécharger toutes nos factures en PDF ?',      'KB-202'),
  (26, 'fr', 'Notre webhook ne se déclenche plus',                       'KB-601'),
  (27, 'pt', 'Recebi um alerta de um acesso que não fui eu',             'KB-105'),
  (28, 'pt', 'Como instalar o aplicativo em 300 computadores?',          'KB-702');

set timing on
update eval_questions
set    minilm = embed(question, 'MINILM'),
       gemini = embed(question, 'GEMINI');
set timing off
commit;

Output:

Table EVAL_QUESTIONS created.

20 rows inserted.

8 rows inserted.

28 rows updated.

Elapsed: 00:00:23.633

Commit complete.

Fifty-six embeddings, 28 of them at Google, in about 24 seconds. Storing them matters more than it seems. A first version of the comparison called EMBED inside the ranking query, and the database called it once per question and article: 192 requests to Google and nearly three minutes, for work the stored column does with none. A function that calls a provider belongs in an UPDATE or a PL/SQL loop, not in a query that runs it for every row.

Compare the Models

For each question and each model, the next query ranks all 24 articles by distance, finds the rank of the right article, and counts per language how often each model put it first.

Example:

-- for each question and model: the rank of the right article among all 24
with ranks as (
  select q.question_id, q.lang, q.article_id, a.article_id as candidate,
         rank() over (partition by q.question_id
                      order by vector_distance(a.embedding, q.minilm, cosine))
           as minilm_rank,
         rank() over (partition by q.question_id
                      order by vector_distance(a.gemini_embedding, q.gemini, cosine))
           as gemini_rank
  from   eval_questions q cross join kb_articles a
)
select lang, count(*) as questions,
       count(case when minilm_rank = 1 then 1 end) as minilm_first,
       count(case when gemini_rank = 1 then 1 end) as gemini_first
from   ranks
where  candidate = article_id
group  by lang
order  by questions desc, lang;

Output:

LANG       QUESTIONS    MINILM_FIRST    GEMINI_FIRST
_______ ____________ _______________ _______________
en                20              19              19
de                 2               0               2
es                 2               0               2
fr                 2               2               2
pt                 2               2               2

In English, the two models tie: 19 of 20 right articles first for each. On short, clear help desk texts, the small model in the database is as good as Gemini's. In German and Spanish, the small model fails every question, while Gemini gets them all.

Look Closer at Other Languages

Example:

with ranks as (
  select q.question_id, q.lang, q.question, q.article_id, a.article_id as candidate,
         rank() over (partition by q.question_id
                      order by vector_distance(a.embedding, q.minilm, cosine))
           as minilm_rank,
         rank() over (partition by q.question_id
                      order by vector_distance(a.gemini_embedding, q.gemini, cosine))
           as gemini_rank
  from   eval_questions q cross join kb_articles a
)
select lang, question, minilm_rank, gemini_rank
from   ranks
where  candidate = article_id
and    lang <> 'en'
order  by question_id;

Output:

LANG    QUESTION                                                      MINILM_RANK    GEMINI_RANK
_______ __________________________________________________________ ______________ ______________
es      Nos cobraron dos veces la misma factura                                14              1
es      La aplicación del móvil se cierra nada más abrirla                      5              1
de      Nach mehreren falschen Passwörtern bin ich gesperrt                    12              1
de      Unser wöchentlicher Bericht kommt nicht mehr per E-Mail                 9              1
fr      Où puis-je télécharger toutes nos factures en PDF ?                     1              1
fr      Notre webhook ne se déclenche plus                                      1              1
pt      Recebi um alerta de um acesso que não fui eu                            1              1
pt      Como instalar o aplicativo em 300 computadores?                         1              1

8 rows selected.

The French and Portuguese questions succeed with the small model only because they share words with the English articles, such as webhook, PDF, and alerta, not because the model understands them. Without a shared word, the right article falls to 5th, 9th, 12th, or 14th place out of 24.

For this data the choice is clear: if customers write in English, the in-database model is enough; if they write in several languages, Gemini is worth its cost. Your data may say otherwise, which is the point of testing.

Rules for Choosing

• Start in the database. A model inside the database has no cost per request, no network, and no data leaving your control, and for short English text it is often as good as a provider's model.
• Use a provider when your text is in several languages, when it is long and cannot be chunked well, or when your own test shows a clear gain.
• Test on your data. A few dozen questions with known answers decide more than any public benchmark.
• Never mix models. Embed the data and the questions with the same model, and re-embed everything when you change models.
• Plan for failure with a provider: batch the requests, embed in the background, and keep in-database embeddings as a fallback.

Keep the evaluation set after the choice is made. The same questions measure the accuracy of vector indexes, compare semantic with keyword search, and check the answers of AI features later.

Conclusion

Choose an embedding model with a test, not a spec sheet: a table of real questions with known right answers, embedded once with each candidate and stored, then ranked with VECTOR_DISTANCE and RANK. On short English help desk text, a 384-dimension model inside Oracle matched Gemini's 3,072-dimension model, while only Gemini handled German and Spanish. Start in the database, move to a provider when your test shows the gain, and never mix models in one search.