aurio9923

Trying NVIDIA Auto Ontology (1): Certifying Definitions on a Five-Compound Example

When you let an AI agent query a database, table and column names alone do not carry assumptions such as “which assay is the current in-house standard” or “how to put values recorded in different units on the same scale.” The SQL can be syntactically correct and still return the wrong aggregate. So I set up Auto Ontology, a semantic layer for databases that NVIDIA publishes on GitHub, in a local Docker environment and ran it with the OpenAI API (gpt-6.1-sol). Using a small drug-discovery assay DB whose answer you can check by eye (3 tables, 5 compounds, 15 measurement rows), this post walks through how Auto Ontology works and where a human has to step in before the numbers come back right.

How Auto Ontology links table definitions to in-house calculation rules

Here, an ontology means a shared dictionary that maps “the words people use in questions” to “the actual tables, columns and formulas.” Auto Ontology scans the contents of a connected relational database and drafts this dictionary automatically. It then serves the definitions that humans have reviewed and approved to AI agents through its web chat or MCP (Model Context Protocol).

It works with four main elements.

  • Terms: business concepts such as Compound and Assay, and the join relationships between tables.
  • Column Attributes: a description of what each column holds, together with sample values actually stored in it (nM, uM and so on).
  • SQL Attributes (named formulas): fixed SQL attached to a term, such as “to get the in-house standard IC50, restrict to a specific protocol and QC-passing rows, convert units to nM and average.”
  • Certification (approval flag): every automatically created item starts as unapproved (Pending Approval), and a reviewer checks it and marks it approved (Certified).
Three steps of Auto Ontology: 1. semantic compilation (LLM draft), 2. human review and certification, 3. answers from certified definitions
Figure 1. The basic flow of Auto Ontology. Drafting from the DB (semantic compilation) runs automatically with an LLM. Adding the project rules that live outside the DB as descriptions and SQL Attributes, and certifying them, lets agents aggregate by the same rules every time.

The example: a mini DB with 3 tables, 5 compounds and 15 rows that you can check by eye

A complex schema makes it hard to follow what is happening, so I prepared a small PostgreSQL database with just three tables: compounds (5 rows), assays (2 rows) and ic50_results (15 rows).

Table 1. Compound table compounds (5 rows)

compound_idcompound_nameseries_codemolecular_weight
CMP-01Lead-101Series-A412.5
CMP-02Lead-102Series-A428.5
CMP-03Lead-103Series-A446.5
CMP-04Backup-201Series-B398.4
CMP-05Backup-202Series-B415.4

The assays table holds two assays (ASSAY_A and ASSAY_B) against the same mutant EGFR. As is common with master tables in practice, nothing in the table says which one is the current in-house standard protocol.

Table 2. Assay table assays (2 rows)

assay_idtarget_genevariant_namepanel_code
ASSAY_AEGFRL858R/T790MKINASE_PANEL_A
ASSAY_BEGFRL858R/T790MKINASE_PANEL_B

Below are all 15 rows of the measurement table ic50_results. The scenario is that the project rules say: “Use ASSAY_B, run at physiological ATP, as the standard; adopt only qc_status = 'PASS'; and multiply uM values from the external contractor (CRO_A) by 1,000 to convert them to nM.”

Table 3. Measurement table ic50_results (15 rows). The “Notes” column on the right is not in the DB; it is the meaning shared among the people who run the assays.

result_idcompound_idassay_idlab_siteraw_valueraw_unitqc_statusNotes (meaning held outside the DB)
1–5CMP-01–05ASSAY_AINTERNAL1.0 / 10 / 8.0 / 12 / 50nMPASSInitial screen (low ATP). CMP-01 looks like #1 at 1.0 nM
6CMP-01ASSAY_BINTERNAL40.0nMPASSStandard ASSAY_B (40.0 nM)
7CMP-01ASSAY_BCRO_A0.040uMPASS40.0 nM after conversion → mean 40.0 nM (#3)
8CMP-02ASSAY_BINTERNAL1.2nMFAILFailed QC because of precipitation (mixing it in lowers the mean to about 13.7 nM)
9CMP-02ASSAY_BINTERNAL20.0nMPASSStandard ASSAY_B (20.0 nM)
10CMP-02ASSAY_BCRO_A0.020uMPASS20.0 nM after conversion → mean 20.0 nM (#2)
11CMP-03ASSAY_BINTERNAL5.0nMPASSStandard ASSAY_B (5.0 nM)
12CMP-03ASSAY_BCRO_A0.005uMPASS5.0 nM after conversion → mean 5.0 nM (true #1)
13CMP-04ASSAY_BCRO_A0.050uMPASSRaw value is 0.050, but after conversion → 50.0 nM (#4)
14CMP-05ASSAY_BINTERNAL200.0nMPASSStandard ASSAY_B (200.0 nM)
15CMP-05ASSAY_BCRO_A0.200uMPASS200.0 nM after conversion → mean 200.0 nM (#5)

The problem I gave the AI is: “Following the project rules, compute the mean IC50 of all five compounds and rank them from most to least potent.” You can read the correct answer off Table 3 by eye. Taking only the passing ASSAY_B data and putting it in nM gives, from most potent (smallest IC50): CMP-03 (5.0 nM) → CMP-02 (20.0 nM) → CMP-01 (40.0 nM) → CMP-04 (50.0 nM) → CMP-05 (200.0 nM). If you pick ASSAY_A instead, CMP-01 (1.0 nM) comes first, and if you forget the uM conversion, CMP-04 (0.050) looks like the smallest value. Keeping the FAIL row does not change the ranking, but it shifts the mean of CMP-02 from 20.0 nM to about 13.7 nM.

Step 1: What semantic compilation (generating an ontology from metadata) can and cannot do

Connecting this DB to Auto Ontology and running semantic compilation (“Semantic Compilation” in the UI) generated three terms (Assay, Compound, IC50 Result), 13 column attributes and the relationships between the tables in about 32 seconds (Figure 2).

Three-node semantic graph generated by Auto Ontology and the detail panel for Assay
Figure 2. The automatically generated three-node semantic graph (left) and the detail panel for the Assay node (right). The Description field on the right shows that, from the column names target_gene and variant_name, the LLM drafted a slightly off description: “A test used to detect specific genetic variants…”

The right panel of Figure 2 shows both what semantic compilation does well and where it falls short. Foreign keys and sample values are linked automatically, but the Assay description was drafted by reading the column names target_gene and variant_name literally, as “a test used to detect specific genetic variants in a target gene.” In reality it is an assay that measures the kinase-inhibitory activity of compounds. Automatically created items start in the unapproved (Pending) state so that a human can correct this kind of subtle mismatch.

In this state, with semantic compilation finished but no project rules taught by a human yet, I asked: “Compute the mean IC50 (nM) of all 5 compounds in the official corporate standard EGFR assay, and list compound ID, compound name and mean IC50 (nM) from most potent (smallest IC50) to least potent.”

  • LLM without the ontology (the LLM writes SQL itself and runs it directly against the DB): it wrote and ran SQL nine times, counting the schema checks (10,643 tokens), and handled the unit conversion and the PASS filter on its own. In the end, though, it said “there is no information indicating which one is the ‘official corporate standard,’ so it cannot be determined uniquely,” laid out tables for both assays, and stopped by asking back which one was the standard.
  • Auto Ontology (semantic compilation only, no certified SQL yet): it prefaced its answer with “No definition identifying the ‘official corporate standard’ was found in the catalog, so the following is a provisional result assuming ASSAY_A is the standard,” and returned the wrong ranking, with CMP-01 (1.0 nM) first.

Since nothing in the DB says “standard = ASSAY_B,” reading the schema automatically cannot settle the correct answer. Filling that gap is the next step.

Step 2: Write the project rules into descriptions and SQL Attributes and certify them

First, I rewrote the Assay description. The draft from semantic compilation described a genetic test guessed from the column names, so I added that it is a kinase inhibition assay and which assay is the current standard (Table 4).

Table 4. Rewriting the Assay description

StageDescription
BeforeA test used to detect specific genetic variants in a target gene.
AfterIn vitro kinase inhibition assay protocol used to measure compound IC50 potency against target EGFR variants (ASSAY_B is the official 1 mM ATP standard; ASSAY_A is the retired low-ATP screen).

Next, I registered one 7-line SQL Attribute (certified_standard_egfr_ic50_nm) that expresses the project rules as they are, and changed its status from Pending Approval to Certified (Figure 3). I also certified the names and descriptions of the three terms.

certified_standard_egfr_ic50_nm in the Pending Approval state certified_standard_egfr_ic50_nm in the Certified state
Figure 3. The registered SQL Attribute before and after approval. Left: Pending Approval right after registration. Right: after review, with the green Certified badge.

The registered SQL is short enough to write by hand.

SELECT c.compound_id, c.compound_name, c.series_code,
       AVG(CASE WHEN r.raw_unit = 'uM' THEN r.raw_value * 1000.0 ELSE r.raw_value END) AS standard_ic50_nm
FROM public.compounds c
JOIN public.ic50_results r ON r.compound_id = c.compound_id
WHERE r.assay_id = 'ASSAY_B'
  AND r.qc_status = 'PASS'
GROUP BY c.compound_id, c.compound_name, c.series_code

Figure 4 shows the same question, asked in English from the Chat screen after certification.

Chat screen after certification: applies ASSAY_B, the PASS filter and the uM conversion, and returns CMP-03 (5.0 nM) first
Figure 4. The Chat response after certification. The generated SQL applies ASSAY_B, PASS and the uM→nM conversion (including the external CRO data), and the ranking from CMP-03 (5.0 nM) to CMP-05 (200.0 nM) matches the answer checked in Table 3.

This time it was no longer “a provisional result based on an assumption.” Following the certified definition, it applied ASSAY_B, qc_status = 'PASS' and the 1,000× uM conversion, and returned the correct ranking, CMP-03 (5.0 nM) → CMP-02 (20.0 nM) → CMP-01 (40.0 nM) → CMP-04 (50.0 nM) → CMP-05 (200.0 nM), from a single question.

To see whether the result holds outside Chat, I also tried a self-built AI agent (the “external agent” below). It has four tools: search terms, get the SQL Attributes attached to a term, check whether a question can be answered, and answer a question. Each matches a feature that Auto Ontology’s MCP server provides, and internally they call Auto Ontology’s API. Given the same question, the external agent also returned the same five-row table (8,202 tokens).

Because I rewrote the description and registered the SQL Attribute at the same time, I have not separated which of the two led to the correct aggregation. That said, the unit conversion in the SQL that Chat built was the same expression as in the SQL Attribute, and the external agent named the SQL Attribute explicitly in its question.

Another useful behavior: when asked about something that is not in the DB, it stopped instead of guessing. I asked, “Among these five compounds, which one has the lowest hERG cardiotoxicity risk and the highest oral bioavailability (F%) in rats?” Chat answered “The available data contain no hERG inhibition measurements or rat oral bioavailability (F%), so the matching compound cannot be identified.” The external agent also confirmed that the only certified definition was EGFR inhibition IC50 and stopped with “These cannot be estimated from the EGFR values to produce a ranking.” It ran no SQL at all and finished after checking terms and attributes (3,824 tokens).

Snags when running with the OpenAI API

The Quickstart in the official README assumes NVIDIA NIM (an nvapi-... key). Running it with the official OpenAI API (https://api.openai.com/v1) and a reasoning model, as here, took a few fixes.

  1. Choosing the model client (auto_ontology/utils/llm_invoke.py): the original code sent any model whose name did not start with a prefix such as openai/ to the NVIDIA ChatNVIDIA class, so I changed it to use ChatOpenAI when the endpoint URL contains api.openai.com. I also enabled the Responses API (use_responses_api=True) so the reasoning model can make tool calls. If the callers’ max_tokens stays small, reasoning uses up the tokens and the output gets cut off, so I also raised the limit to at least 8,192.
  2. Embedding dimensions and vendor-specific parameters (auto_ontology/utils/embedding.py): the PostgreSQL column for vector search (pgvector) is defined as halfvec(2048), so with text-embedding-3-large I had to specify dimensions=2048 and drop the NVIDIA NIM-specific extra_body parameter.
  3. Precedence of CONNECTION_STRINGS in .env: if CONNECTION_STRINGS stays in the .env file, the DB from the environment variable always takes precedence for compilation, even after you add or switch DBs in the web UI (Settings → Connections). When you manage connections from the UI, leave CONNECTION_STRINGS= empty, and the DBs registered in the UI are compiled.

Impressions and what’s next

Running this miniature example with five compounds and 15 rows made Auto Ontology’s role quite clear. Rather than a system that guesses even unwritten in-house rules automatically, it fits better to see it as “a system that lets semantic compilation draft the tedious table structures and relationships, and then makes AI agents connected through Chat or MCP consistently follow calculation rules (SQL Attributes) that humans have certified.” The practical benefit, I felt, is that once a rule is certified, the person asking no longer has to add “use ASSAY_B, multiply uM by 1,000…” to every prompt.

On the other hand, extending this to SAR (structure–activity relationship) data from a real drug discovery project raises other problems that an SQL ontology alone cannot solve, for example classifying core scaffolds from the compounds’ SMILES strings, or running matched molecular pair (MMP) analysis per substituent. Next time, I plan to build a more realistic database and report how Auto Ontology does when combined with an RDKit chemical-structure Skill.

Sources, environment and test data

  • Software: NVIDIA Auto Ontology (NVIDIA/auto-ontology, Apache-2.0 license).
  • Environment: Docker Compose on Windows WSL2 (Ubuntu 22.04) (PostgreSQL 17 + pgvector, FastAPI backend, ingestion service, Next.js frontend).
  • Models: via the OpenAI API, gpt-6.1-sol as the reasoning model, gpt-6.1-terra for auxiliary tasks that do not use reasoning, and text-embedding-3-large (dimensions=2048) for embeddings. The LLM without the ontology and the external agent used for comparison also ran on gpt-6.1-sol.
  • Language: the runs reported in the text were done in Japanese, and the questions and answers quoted here are translations. Figures 3 and 4 were captured again with the SQL Attribute description and the question in English; the Figure 4 run returned the same ranking.
  • Mini test DB (mini_assay_intro): fictional synthetic data made to check the behavior by eye (compounds 5 rows, assays 2 rows, ic50_results 15 rows); these are not real compounds or experimental values. Fractions such as 4.9999999441 in mean_ic50_nm in Figure 4 are rounding errors: the PostgreSQL columns were defined as single-precision floating point (REAL), so values such as 0.005 cannot be represented exactly in binary.