Membangun Aplikasi RAG dengan Next.js + Vector Database

Update 2026: Artikel ini sudah diuji di production dengan Next.js 14+ (App Router), PostgreSQL + pgvector, dan OpenAI API. Cocok untuk developer yang sudah paham dasar Next.js dan ingin integrasi AI.

Pendahuluan

Hai developer! Pernah nggak sih kamu mau bikin chatbot yang paham dokumen perusahaan kamu? Atau Q&A system yang jawab berdasarkan knowledge base internal?

Jawabannya: RAG (Retrieval-Augmented Generation).

Di artikel ini, kita bahas cara bangun aplikasi RAG lengkap: 1. Apa itu RAG — Konsep dan kenapa penting 2. Arsitektur RAG — Pipeline lengkap 3. Vector Database — pgvector setup di PostgreSQL 4. Embedding Generation — Convert teks ke vector 5. Retrieval Layer — Semantic search 6. Generation Layer — LLM response 7. Next.js Integration — API routes + streaming UI 8. Deployment — Docker + monitoring 9. Performance & Cost — Optimasi 10. Production Checklist

1. Apa itu RAG?

RAG = Retrieval-Augmented Generation.

Sederhananya:

User Question ↓[Retrieval] Cari konteks relevan dari knowledge base ↓[Generation] LLM jawab berdasarkan konteks + question ↓Answer

Kenapa RAG?

• No hallucination — Jawaban berdasarkan fakta dari dokumen

• Up-to-date — Update knowledge base tanpa retrain model

• Domain-specific — Paham konteks perusahaan

• Cost-effective — Pakai model kecil + context injection

Kapan Pakai RAG?

• Chatbot customer service

• Internal knowledge base Q&A

• Document analysis

• Code assistant dengan context repo

2. Arsitektur RAG

┌─────────────────────────────────────────────────┐
│ Next.js App │
├─────────────────────────────────────────────────┤
│ Frontend (Chat UI) │
│ ↓ │
│ API Route /api/chat │
│ ↓ │
│ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Retrieval │ │ Generation │ │
│ │ - Embed query│ │ - Build prompt │ │
│ │ - Vector │ │ - Call LLM │ │
│ │ search │ │ - Stream response │ │
│ └──────┬───────┘ └──────────┬───────────┘ │
│ ↓ ↑ │
├─────────┼────────────────────────┼────────────────┤
│ PostgreSQL + pgvector │ │ LLM API (OpenAI)│
│ (knowledge base) │ │ │
└──────────────────────────┴────┴────────────────────┘

3. Vector Database Setup (pgvector)

Install pgvector

# Di Docker, gunakan image postgres dengan pgvector# docker-compose.yml
services:
 postgres:
 image: pgvector/pgvector:pg16
 environment:
 POSTGRES_USER: ai_user
 POSTGRES_PASSWORD: ai_pass
 POSTGRES_DB: ai_db
 volumes:
 - pgvector_data:/var/lib/postgresql/data
 ports:
 - "5432:5432"

volumes:
 pgvector_data:

Enable Extension & Create Table

-- Enable pgvector
CREATE EXTENSION IF NOT EXISTS vector;

-- Tabel documents
CREATE TABLE documents ( id SERIAL PRIMARY KEY, content TEXT NOT NULL, metadata JSONB, embedding vector(1536) -- OpenAI text-embedding-3-small = 1536 dims);

-- Index untuk similarity search (IVFFlat atau HNSW)
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

Prisma Schema

model Document { id Int @id @default(autoincrement()) content String metadata Json? embedding Unsupported("vector(1536)")? createdAt DateTime @default(now())}

4. Embedding Generation

Generate Embeddings

// lib/embeddings.ts
import OpenAI from 'openai'

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

export async function generateEmbedding(text: string): Promise<number[]> { const response = await openai.embeddings.create({ model: 'text-embedding-3-small', input: text, }) return response.data[0].embedding}

// Batch embed documents
export async function embedDocuments(docs: string[]): Promise<number[][]> { const response = await openai.embeddings.create({ model: 'text-embedding-3-small', input: docs, }) return response.data.map((d) => d.embedding)}

Store Documents

// lib/rag.ts
import { prisma } from '@/lib/prisma'import { generateEmbedding } from '@/lib/embeddings'

export async function storeDocument(content: string, metadata?: any) { const embedding = await generateEmbedding(content)

 await prisma.document.create({ data: { content, metadata, embedding: embedding as any, // Prisma accepts array for vector }, })}

// Chunk large documents
export function chunkText(text: string, chunkSize = 1000): string[] { const chunks: string[] = [] for (let i = 0; i < text.length; i += chunkSize) { chunks.push(text.slice(i, i + chunkSize)) } return chunks}

5. Retrieval Layer

Semantic Search

// lib/retrieval.ts
import { prisma } from '@/lib/prisma'import { generateEmbedding } from '@/lib/embeddings'

export async function retrieveContext( query: string, limit: number = 5): Promise<{ content: string; score: number }[]> { const queryEmbedding = await generateEmbedding(query)

 // pgvector cosine similarity search const results = await prisma.$queryRaw` SELECT content, 1 - (embedding <=> ${queryEmbedding}::vector) as similarity FROM documents ORDER BY embedding <=> ${queryEmbedding}::vector LIMIT ${limit} `

 return results}

Hybrid Search (Optional)

// Combine vector + keyword search
export async function hybridSearch(query: string, limit = 5) { const vectorResults = await retrieveContext(query, limit * 2)

 // Simple keyword filter const keywordResults = await prisma.document.findMany({ where: { content: { contains: query.split(' ')[0] }, }, take: limit, })

 // Merge & dedupe const merged = [...vectorResults] for (const kr of keywordResults) { if (!merged.find((m) => m.content === kr.content)) { merged.push({ content: kr.content, score: 0.5 }) } }

 return merged.slice(0, limit)}

6. Generation Layer

Build Prompt

// lib/prompt.ts
export function buildRAGPrompt( question: string, context: string[]): string { return `You are a helpful assistant. Answer the question based on the provided context.

Context:${context.map((c, i) => `[${i + 1}] ${c}`).join('\n\n')}

Question: ${question}

Instructions:- Jawab dalam Bahasa Indonesia- Gunakan hanya informasi dari context- Jika tidak tahu, katakan "Maaf, informasi tidak tersedia"- Berikan sumber reference jika ada

Answer:`}

Streaming LLM Response

// app/api/chat/route.ts
import { NextRequest, NextResponse } from 'next/server'import OpenAI from 'openai'import { retrieveContext } from '@/lib/retrieval'import { buildRAGPrompt } from '@/lib/prompt'

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

export async function POST(req: NextRequest) { const { question } = await req.json()

 // 1. Retrieve const context = await retrieveContext(question, 5) const contextTexts = context.map((c) => c.content)

 // 2. Build prompt const prompt = buildRAGPrompt(question, contextTexts)

 // 3. Stream generation const stream = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [{ role: 'user', content: prompt }], stream: true, })

 // 4. Return streaming response const encoder = new TextEncoder() const readable = new ReadableStream({ async start(controller) { for await (const chunk of stream) { const text = chunk.choices[0]?.delta?.content || '' controller.enqueue(encoder.encode(text)) } controller.close() }, })

 return new Response(readable, { headers: { 'Content-Type': 'text/plain; charset=utf-8', 'Cache-Control': 'no-cache', }, })}

7. Next.js Integration

Chat UI Component

// components/ChatBox.tsx'use client'

import { useState } from 'react'

export function ChatBox() { const [messages, setMessages] = useState<{ role: string; content: string }[]>([]) const [input, setInput] = useState('') const [loading, setLoading] = useState(false)

 async function sendMessage() { if (!input.trim()) return

 const userMsg = { role: 'user', content: input } setMessages((prev) => [...prev, userMsg]) setInput('') setLoading(true)

 // Stream response const response = await fetch('/api/chat', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ question: input }), })

 const reader = response.body?.getReader() const decoder = new TextDecoder() let assistantMsg = ''

 setMessages((prev) => [...prev, { role: 'assistant', content: '' }])

 if (reader) { while (true) { const { done, value } = await reader.read() if (done) break assistantMsg += decoder.decode(value) setMessages((prev) => { const newMsgs = [...prev] newMsgs[newMsgs.length - 1] = { role: 'assistant', content: assistantMsg } return newMsgs }) } }

 setLoading(false) }

 return ( <div className="max-w-2xl mx-auto p-4"> <div className="space-y-4 mb-4"> {messages.map((msg, i) => ( <div key={i} className={msg.role === 'user' ? 'text-right' : 'text-left'}> <div className={`inline-block p-3 rounded-lg ${ msg.role === 'user' ? 'bg-blue-500 text-white' : 'bg-gray-100' }`}> {msg.content} </div> </div> ))} {loading && <div className="text-gray-500">Thinking...</div>} </div>

 <div className="flex gap-2"> <input value={input} onChange={(e) => setInput(e.target.value)} onKeyPress={(e) => e.key === 'Enter' && sendMessage()} placeholder="Tanya sesuatu..." className="flex-1 p-2 border rounded-lg" /> <button onClick={sendMessage} className="px-4 py-2 bg-blue-500 text-white rounded-lg"> Send </button> </div> </div> )}

8. Deployment

Docker Setup

# DockerfileFROM node:18-alpine AS builderWORKDIR /appCOPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:18-alpine AS runnerWORKDIR /appENV NODE_ENV=production
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 3000CMD ["node", "server.js"]

Environment Variables

# .env.localOPENAI_API_KEY=sk-...DATABASE_URL=postgresql://ai_user:ai_pass@postgres:5432/ai_dbEMBEDDING_MODEL=text-embedding-3-smallLLM_MODEL=gpt-4o-mini

9. Performance & Cost Optimization

Caching Embeddings

// Cache embeddings untuk query yang sama
const embeddingCache = new Map<string, number[]>()

export async function getCachedEmbedding(text: string) { if (embeddingCache.has(text)) { return embeddingCache.get(text)! } const embedding = await generateEmbedding(text) embeddingCache.set(text, embedding) return embedding}

Batch Processing

// Embed banyak dokumen sekaligus (lebih murah)
const embeddings = await embedDocuments(chunks) // 1 API call untuk N docs

Model Selection

Reduce Token Usage

// Hanya ambil context yang relevan (top-k)
const context = await retrieveContext(query, 3) // bukan 10

// Truncate context jika terlalu panjang
const truncated = contextTexts.join('\n').slice(0, 4000)

10. Production Checklist

✅ Vector DB

• [ ] pgvector extension enabled

• [ ] HNSW index created

• [ ] Embedding dimension match (1536 for OpenAI)

• [ ] Backup strategy

✅ Retrieval

• [ ] Similarity threshold (jangan ambil yang terlalu jauh)

• [ ] Chunk size optimal (500-1000 chars)

• [ ] Hybrid search (optional)

✅ Generation

• [ ] Streaming enabled (UX lebih baik)

• [ ] Prompt template tested

• [ ] Fallback untuk API error

• [ ] Rate limiting

✅ Cost

• [ ] Embedding cache

• [ ] Model yang tepat (mini vs full)

• [ ] Token limit per request

• [ ] Monitor usage

✅ Security

• [ ] API key di environment variable

• [ ] User input sanitized

• [ ] Rate limit per user

• [ ] No PII in logs

11. Troubleshooting

Embedding dimension mismatch

# Error: vector dimension mismatch
# Solusi: pastikan embedding model sama
# OpenAI text-embedding-3-small = 1536
# OpenAI text-embedding-3-large = 3072
# Sesuaikan column type: vector(1536) atau vector(3072)

Slow retrieval

-- Cek apakah index digunakan
EXPLAIN ANALYZE
SELECT * FROM documents
ORDER BY embedding <=> '[...]'::vector
LIMIT 5;

-- Jika seq scan, rebuild index
REINDEX INDEX documents_embedding_idx;

High cost

// Monitor token usage dari response
const completion = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  max_tokens: 500, // ← Batasi output
  messages: [...],
})
console.log(completion.usage.total_tokens)

Hallucination

// Tambahkan instruksi strict di prompt
const prompt = `Jawab HANYA berdasarkan context. Jika jawaban tidak ada di context, katakan "Maaf, informasi tidak tersedia". JANGAN mengarang jawaban.

Context: ${context}`

12. Kesimpulan

Nah itu dia panduan membangun aplikasi RAG dengan Next.js + Vector DB.

Kesimpulan utama: 1. RAG = Retrieval + Generation — jawaban berdasarkan fakta 2. pgvector — vector DB gratis di PostgreSQL 3. Embedding — convert teks ke vector (OpenAI API) 4. Retrieval — semantic search dengan cosine similarity 5. Generation — LLM dengan streaming response 6. Optimization — cache, batch, model selection

Best practices: - Chunk documents dengan size optimal - Gunakan HNSW index untuk speed - Stream response untuk UX lebih baik - Monitor cost & token usage - Always fallback kalau LLM API error

Stack yang direkomendasikan: - Vector DB: pgvector (PostgreSQL), atau Pinecone (managed) - Embedding: OpenAI text-embedding-3-small - LLM: GPT-4o-mini (cost-effective), GPT-4o (complex tasks) - Framework: Next.js 14 App Router - Deployment: Docker + Caddy

FAQ

Q: pgvector vs Pinecone - mana yang better? A: pgvector gratis (sudah punya PostgreSQL), Pinecone managed tapi bayar. Untuk startup, pgvector cukup.

Q: Berapa chunk size yang optimal? A: 500-1000 karakter. Terlalu kecil = context terfragmentasi, terlalu besar = token waste.

Q: GPT-4o-mini cukup nggak? A: Untuk 80% kasus cukup. Complex reasoning baru butuh GPT-4o.

Q: Bagaimana handle dokumen besar (PDF 100 halaman)? A: Split per page/section, chunk, embed semua. Retrieval ambil top-5 relevan.

Q: RAG vs Fine-tuning - mana yang dipilih? A: RAG untuk knowledge base yang sering update. Fine-tuning untuk style/behavior khusus.