Pg_vectorsearch: Postgresga Yangi ANN Qidiruv Usuli

Pg_vectorsearch — bu PostgreSQL uchun yuqori samarali yaqin qidiruv (ANN) uchun yangi kengaytma. U IVFFlat va HNSW indekslariga qaraganda tezroq ishlaydi. Hozirda alpha versiyada bo'lib, u test qilish uchun mo'ljallangan, ishlab chiqarish uchun emas.

Pg_vectorsearchning asosiy xususiyatlari:

• PRISM indeksi (100 milliondan ortiq vektorlar) — IVF uslubidagi, hierarxik yo'nalish bilan;
• RaBitQ kvantizatsiyasi — teoretik xatolar bilan ikki bosqichli qidiruv;
• SIMD-optimallashtirilgan masofa hisoblash (AVX2, AVX512, NEON);
• Dinamik yangilanishlar — kiritish, yangilanish va o'chirish, post-listlarni qayta taqsimlash orqali;
• pgvector bilan moslashuvchanlik — mavjud vektor va yarim vektor ustunlarini shunchaki shunday indekslash.

PRISM indeksi

PRISM (Partitioned Routing Index for Similarity Matching) — bu PostgreSQL uchun mahalliy indeks kirish usuli. U pgvectorning IVFFlat indeksiga o'xshaydi, lekin zamonaviy kvantizatsiya, yo'nalish va taqsimlash texnikalariga asoslanadi. U yuqori samarali ishlash uchun C tilida yozilgan.

PRISM indeksi grafik asosidagi HNSW indeksiga qaraganda tezroq ishlaydi. U 100 million vektorli to'plamda ekvivalent xotira bilan yuqori xotiradan foydalanish imkoniyatiga ega.

Qidiruv jarayoni

PRISM indeksi quyidagi bosqichlardan iborat:

Query Flow:
1. Traverse centroid tree to find nearest clusters (buffer cache)
2. Scan posting lists for candidates (buffer cache, async I/O)
3. RaBitQ two-stage filtering: estimate → error bound → rerank
4. Full-precision reranking from heap
5. Return k nearest neighbors

• Markaziy daraxtni kuzatish — eng yaqin klasterlarni topish (buffer cache);
• Post-listlarni skanlash — nomzodlarni topish (buffer cache, asinxron I/O);
• RaBitQ ikki bosqichli filtrlash: baholash → xatolar chegarasi → qayta tartiblash;
• To'liq aniqlikdagi qayta tartiblash — heapdan;
• k ta eng yaqin qidiruv natijalari qaytariladi.

O'rnatish va foydalanish

Pg_vectorsearch kengaytmasini o'rnatish uchun quyidagi buyruqlarni bajaring:

CREATE EXTENSION pg_vectorsearch;

-- PLAIN keeps the vector inline. EXTERNAL (the default) toasts a value
-- over ~2 kB, and rerank then pays an extra fetch the planner does not cost.
CREATE TABLE items (
    id serial PRIMARY KEY,
    embedding vec32(3) STORAGE PLAIN
);

-- Load first; the index is sized from the row count at CREATE INDEX.
-- 500 rows is enough for the planner to use the index.
INSERT INTO items (embedding)
SELECT ARRAY[i::real, (i % 7)::real, (i % 5)::real]::vec32
FROM generate_series(1, 1000) i;

-- For inner product or cosine, see Operator classes below.
CREATE INDEX ON items USING prism (embedding vec32_l2_ops);

-- 0 = auto (~0.95 recall). Raise for recall, lower for speed.
SET prism.nprobe = 40;

SELECT * FROM items
ORDER BY embedding <-> '[3,1,2]'
LIMIT 5;

CREATE EXTENSION pg_vectorsearch;

Kengaytmani o'rnatgandan so'ng, ikkita usul bor:

-- The table you already have, included so the example runs on its own.
CREATE EXTENSION vector;

CREATE TABLE documents (
    id        bigserial PRIMARY KEY,
    body      text,
    embedding vector(384)
);
CREATE INDEX documents_embedding_hnsw
    ON documents USING hnsw (embedding vector_l2_ops);

-- Install pg_vectorsearch next to pgvector and build the index on the
-- existing column. The column is not rewritten or copied.
CREATE EXTENSION pg_vectorsearch;

CREATE INDEX documents_embedding_prism
    ON documents USING prism (embedding vec32_l2_ops);

-- Your queries keep working as before. PRISM is normally faster, so
-- the planner should prefer it over the HNSW index while both exist.
-- Verify with an EXPLAIN:
EXPLAIN (COSTS OFF)
SELECT id, body
FROM documents
ORDER BY embedding <-> (SELECT embedding FROM documents WHERE id = 42)
LIMIT 10;
--  Limit
--    InitPlan 1
--      ->  Index Scan using documents_pkey on documents documents_1
--            Index Cond: (id = 42)
--    ->  Index Scan using documents_embedding_prism on documents
--          Order By: (embedding <-> (InitPlan 1).col1)

-- Once you are happy with it, drop the HNSW index.
DROP INDEX documents_embedding_hnsw;

• Mahalliy turlar. Yangidan boshlayapsizmi yoki pgvectordan foydalanmayapsizmi? Vektorlarni pg_vectorsearchning o'z vec32 (float32) yoki vec16 (float16) ustunlarida saqlang. Boshqa kengaytma kerak emas.
• Mavjud pgvector jadvalari. Agar sizda vector yoki halfvec ustunlari bo'lsa, pg_vectorsearchni pgvector bilan birga o'rnatib, mavjud ustunlarga PRISM indeksini qurishingiz mumkin.

Ikkala usul ham bir xil indeksni quradi va bir xil operatorlar (<->, <#>, <=>) va sozlash parametrlaridan foydalanadi.

# Release build
meson setup builddir --buildtype=release

# Build and install to PostgreSQL
meson install -C builddir

Misollar

Jadval yaratish va indeks qurish misoli:

CREATE TABLE items (
id serial PRIMARY KEY,
embedding vec32(3) STORAGE PLAIN
);

INSERT INTO items (embedding)
SELECT ARRAY[i::real, (i % 7)::real, (i % 5)::real]::vec32
FROM generate_series(1, 1000) i;

CREATE INDEX ON items USING prism (embedding vec32_l2_ops);

SET prism.nprobe = 40;

SELECT * FROM items
ORDER BY embedding <-> '[3,1,2]'
LIMIT 5;

Vec16 turini ishlatish orqali vektorlarning jadvalda egallagan joyini yarimga kamaytirish mumkin.

Agar sizda pgvectorning vector yoki halfvec ustunlari bo'lsa, siz PRISM indeksini shu ustunlarga qurishingiz mumkin. Jadval va so'rovlar o'zgarishsiz qoladi.

Pg_vectorsearchning asosiy afzalliklari — bu yuqori samarali ishlash va PostgreSQL bilan moslashuvchanligi. U IVFFlat va HNSW indekslariga qaraganda tezroq ishlaydi va xotiradan samarali foydalanadi.

Asl manba: github.com