Vector Database

Vector Database for Searching Health Evidence by Meaning

A vector database is a database management system that holds a list of numbers for each stored passage, image or record, and answers a query by returning the items whose lists lie closest to the list made from the query. The lists, called embeddings, come from a separate machine learning model, so the database finds text with a similar meaning even when it shares no words with the query. In health economics this is the retrieval step behind tools that search guidance, trial reports or clinical notes before a language model drafts an answer. Closeness in the vector space measures how alike two items look to the embedding model, not whether a statement is true, so a retrieved passage still has to be checked against its source before a figure from it enters a review or model.

Embeddings, similarity scores and nearest neighbours

Pan and colleagues, surveying vector database management systems, describe documents and images being encoded by an embedding model into vectors of a fixed number of dimensions and then stored. Johnson and colleagues report that such embeddings usually have 50 to more than 1,000 dimensions. The vectors stand in for the real items, and either the user or the database runs the embedding model.

Similarity between two vectors is measured by a score. Common choices include the inner product, cosine similarity and Euclidean distance. Cosine similarity ignores the length of the vectors and compares only their direction.

$$\cos(\mathbf{q}, \mathbf{x}) = \frac{\sum_{i=1}^{D} q_i x_i}{\sqrt{\sum_{i=1}^{D} q_i^2} \sqrt{\sum_{i=1}^{D} x_i^2}}$$

where $\mathbf{q}$ is the query vector, $\mathbf{x}$ is a stored vector, $D$ is the number of dimensions, $q_i$ and $x_i$ are the values in dimension $i$, and the result runs between minus 1 and 1, with larger values meaning more similar. The numerator alone is the inner product, which Pan and colleagues note can rank two long identical vectors as more similar than two short identical ones because it scales with magnitude.

The basic query asks for the $k$ stored vectors nearest the query. An exact $k$-nearest-neighbour search guarantees the true nearest items, while an approximate query accepts results within a set factor of the nearest distance in exchange for speed, and most systems support the approximate form. A range query instead returns every item within a chosen distance. Malkov and Yashunin define the quality of an approximate search, its recall, as the share of the true $k$ nearest neighbours that the search finds.

Indexes that avoid comparing every vector

Comparing the query with all $N$ stored vectors costs time in proportion to $N \times D$, which Pan and colleagues call prohibitive when both are large. An index cuts the comparisons by splitting the collection so that only a small part is scanned. The survey groups indexes by structure into tables of buckets, trees and graphs.

Table indexes hash each vector into a bucket. Locality sensitive hashing uses random functions with error guarantees but can need many tables. Other table indexes learn the buckets, typically with k-means, the centroid method also used in cluster analysis of patients or providers, and the query scans only the buckets nearest to it.

Graph indexes link each vector to near neighbours and search along the links. The hierarchical navigable small world (HNSW) method of Malkov and Yashunin builds layers of proximity graphs. Each new element is given a top layer drawn from an exponentially decaying distribution, so upper layers hold few elements joined by long links. A search starts at the top layer, moves greedily towards the query, then drops a layer and repeats, ending on the ground layer, which holds every element.

$$l = \lfloor -\ln(u) \times m_L \rfloor$$

where $l$ is the highest layer that a new element joins, $u$ is a random number drawn uniformly between 0 and 1, $m_L$ is a normalisation factor, and the authors suggest $m_L = 1/\ln(M)$, with $M$ the number of links made for each new element. Pan and colleagues attribute its support in many commercial systems to its being easy to build, updatable and fast to query.

Quantisation compresses the stored vectors. A product quantiser splits each vector into sub-vectors and replaces each with the nearest of a small set of learned centroids. In the scheme Johnson and colleagues describe, each sub-quantiser has 256 values, so each sub-vector fits in one byte, and the query is compared with the codes without rebuilding the full vectors. The compression is lossy, so distances become estimates.

Filters on ordinary fields

Health evidence carries fields that matter as much as meaning, such as document type, publication date or disease area. Pan and colleagues call a search that must also satisfy a condition on such fields a predicated or hybrid query. The filter can be applied before the vector search, after it, or during it.

Each choice has a cost. Blocking filtered vectors out of a graph index beforehand can disconnect the graph, so some true neighbours become unreachable. Filtering afterwards can leave fewer than $k$ results, because the $k$ nearest items may fail the condition, and systems that do this often retrieve extra candidates first, with no clear rule for how many. A review team asking only for technology appraisals from a given period should therefore check how its system applies the filter.

Native systems, extended systems and libraries

Pan and colleagues divide current systems into native ones built around vectors, such as Milvus and Qdrant, and extended ones that add vector search to an existing database, such as pgvector for PostgreSQL. Libraries such as Faiss supply indexes without the full set of database functions.

The survey reports that extended relational systems offer the most capabilities, possibly with less speed. A team that already keeps unit costs or trial extracts in a relational database can add vector search there instead of running a second system. In pgvector, for example, a query uses an approximate index when one exists and otherwise scans every row exactly, so the same request can be exact or approximate depending on how the table is set up.

Published uses at scale and in health evidence work

The retrieval-augmented generation model of Lewis and colleagues shows the pattern at scale. Each Wikipedia article was split into 100-word chunks, giving 21 million passages, each was embedded, and a single Faiss index with an HNSW approximation served retrieval. The index needed about 100 GB of memory, cut to 36 GB with compression.

Amugongo and colleagues review 70 studies of retrieval-augmented generation in health care. In one included study, guidance documents on liver disease were embedded and searched in a vector database before a model answered clinical questions. In another, chunked patient notes were searched with Faiss to screen patients for a clinical trial, and the review reports that this could improve efficiency and reduce costs. The review also notes that the retrieval data may contain sensitive patient information, which brings data security rules into the design.

A smaller collection may need no dedicated system. The Office for National Statistics Data Science Campus built a tool for the NICE surveillance team to find related clinical guideline recommendations among about twenty thousand. When the team trialled Sentence-BERT, embeddings could be created in advance for each recommendation and stored for later comparison, for example by cosine similarity. The top 200 matches went to reviewers, and the delivered tool loaded its data from static files, with the option of a database later. Johnson and colleagues describe exact brute-force search as useful for small datasets, and Malkov and Yashunin report a brute-force query over one million 128-dimensional image vectors taking 94 ms.

Worked example with four passages and one filter

All numbers here are illustrative, and real embeddings are far longer. A team searching for evidence on cost per QALY holds four passages, each with a document type field.

Step 1. The query embeds as (0.6, 0.8, 0, 0). Every vector has length 1, so cosine similarity equals the inner product.

$$\cos(\mathbf{q}, \mathbf{a}) = 0.6 \times 0.8 + 0.8 \times 0.6 = 0.96$$

where $\mathbf{q}$ is the query vector and $\mathbf{a}$ is the vector of passage A. In the same way passage B scores 0.36, passage C scores 0.80 and passage D scores 0.

Step 2. With $k = 2$ and no filter, the result is A (0.96) and C (0.80).

Step 3. With the filter "type is Appraisal" applied after the search, C is removed and only A remains, one result where two were asked for. Applied before the search, the candidates are A and D, and the result is A (0.96) and D (0), so the second item meets the filter but shares nothing with the query.

Step 4. Storage for one million passages with 768 dimensions held as 4-byte numbers is

$$S = N \times D \times 4 = 1{,}000{,}000 \times 768 \times 4 = 3{,}072{,}000{,}000$$

where $S$ is the storage in bytes, $N$ is the number of passages and $D$ is the number of dimensions, about 3.1 GB. A product quantiser with 64 one-byte codes per vector stores 64,000,000 bytes, a 48-fold reduction. Malkov and Yashunin put the HNSW links at about 60 to 450 bytes per element, so their range adds about 60 MB to 450 MB.

Step 5. With $M = 16$, $m_L = 1/\ln(16) = 0.361$, and an element reaches layer 1 or above with probability one in 16. About 62,500 of the million elements sit in layer 1 and about 3,906, one in 256, in layer 2, so a search makes long jumps among a few thousand elements before working down through layer 1 and the ground layer.

Step 6. If an exact search finds the true 10 nearest passages and the approximate search returns 9 of them,

$$\text{recall} = \frac{k'}{k} = \frac{9}{10} = 0.9$$

where $k'$ is the number of true nearest neighbours returned and $k$ is the number requested. One relevant passage in ten was missed, which matters in a systematic review where every eligible study counts.

Settings that change what is retrieved

Each setting below alters which passages come back. A method that uses retrieval should report them, since a reviewer cannot repeat the search without them.

• Embedding model and score. Pan and colleagues find no principled rule for choosing the score, and it is often chosen from experience.
• Index parameters. Malkov and Yashunin give $M$ a reasonable range of 5 to 48, with larger values better for high recall and using more memory, and set search quality with a candidate list size, $ef$.
• Compression. Quantised codes save memory but estimate distances.
• Updates. Indexes whose partitions are learned from the data can become unbalanced as new documents arrive, which may require a rebuild.
• Chunk size. Lewis and colleagues used 100-word passages.

Vector databases beside graphs and keyword search

A vector database answers the question of which stored items resemble a query. A knowledge graph or graph database answers which items are joined by a named relation, such as a drug and the trial that tested it, and each has its own page in this set. Some systems hold both, and Pan and colleagues list Neo4j, a property graph database, among those with vector search.

Keyword search matches terms, so it can miss a passage that an embedding finds through different wording. Pages in this set on text embeddings, semantic search and retrieval-augmented generation cover how vectors are made and how retrieved text reaches a language model, with wider rules under artificial intelligence in health.

Limitations and common misreadings

A common error is treating a ranked list of similar passages as a finished evidence search. Five recur in health economics work.

1. Reading similarity as relevance or truth. Pan and colleagues stress that vector queries rely on a vague notion of semantic similarity that is hard to capture. A close passage may describe a different population or an outdated price.
2. Treating approximate results as complete. Approximate indexes trade recall for speed, so searches that must find every eligible study need exact search or a measured recall.
3. Ignoring the filter strategy. Post-filtering can return fewer results than requested and any filtered k-nearest query can fill its list with dissimilar items.
4. Assuming retrieval fixes model errors. Amugongo and colleagues cite a study in which a model gave incorrect answers even with retrieval, so passages used for data extraction or a health technology assessment still need checking at source.
5. Overlooking privacy. Pan and colleagues note that security and privacy matter more for managed cloud services and remain open problems, so indexed patient notes fall under the same data governance rules as the notes.

Sources

• Amugongo LM, Mascheroni P, Brooks S, Doering S, Seidel J. Retrieval augmented generation for large language models in healthcare: a systematic review. PLOS Digital Health. 2025;4(6):e0000877. https://doi.org/10.1371/journal.pdig.0000877
• Johnson J, Douze M, Jégou H. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data. 2021;7(3):535-547. https://doi.org/10.1109/TBDATA.2019.2921572 (arXiv:1702.08734 read)
• Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). arXiv:2005.11401. https://arxiv.org/abs/2005.11401
• Malkov YA, Yashunin DA. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020;42(4):824-836. https://doi.org/10.1109/TPAMI.2018.2889473 (arXiv:1603.09320 read)
• Office for National Statistics Data Science Campus. Supporting healthcare guidance recommendations using natural language processing. 31 January 2022. https://datasciencecampus.ons.gov.uk/supporting-healthcare-guidance-recommendations-using-natural-language-processing/
• Pan JJ, Wang J, Li G. Survey of vector database management systems. The VLDB Journal. 2024;33(5):1591-1615. https://doi.org/10.1007/s00778-024-00864-x (arXiv:2310.14021v1 read)