
PERMENPAR-AI PROJECT
====================

This project creates a Retrieval-Augmented-Generation (RAG) knowledge base from the uploaded
"Permenpar 6 Tahun 2025.pdf" and exposes:
 - Embedding script (rag/embed.py)
 - Query CLI (rag/query.py)
 - Flask API (api/app.py) with endpoints:
     POST /query  -> returns top-k retrieved passages + (optionally) LLM answer
 - Web GUI (web/) using Flask + Bootstrap
 - Streamlit demo (scripts/streamlit_app.py) (optional)

**Important:** This repository only scaffolds the pipeline and writes scripts. It does not precompute embeddings.
Run the steps below on your machine.

Quickstart
----------
1. Create venv and install dependencies:
   python -m venv venv
   source venv/bin/activate   # Windows: venv\Scripts\activate
   pip install -r requirements.txt

2. Prepare data:
   - The original PDF is at data/raw/Permenpar_6_Tahun_2025.pdf
   - Run:
       python rag/extract_pdf.py
     This will extract text and create data/processed/permenpar_paragraphs.txt

3. Build embeddings & FAISS index:
       python rag/embed.py
   This will produce rag/vector.index and rag/chunks.jsonl

4. Run API:
       cd api
       FLASK_APP=app.py flask run
   or:
       python api/app.py

5. Web GUI:
       python api/app.py
   Open http://127.0.0.1:5000

LLM Integration
---------------
The API supports optional LLM answering:
 - Set environment variable OPENAI_API_KEY to use OpenAI (gpt-3.5/4).
 - Or configure local LLM usage in api/llm_provider.py (instructions in file).

Files of interest
-----------------
 - rag/extract_pdf.py   -> extract PDF into paragraph chunks
 - rag/embed.py         -> compute embeddings (sentence-transformers) and write FAISS index
 - rag/query.py         -> simple CLI examples to query vector DB
 - api/app.py           -> Flask API + web UI routes
 - web/templates/*      -> HTML templates (Bootstrap)
 - scripts/streamlit_app.py -> quick Streamlit demo

If you want, I can also run extraction and build a small sample CSV (50 paragraphs) here and return it — say "buat sample".

