Train a Custom AI Model on Your PDF Documents Using LangChain

Modern applications increasingly need AI that understands your data — not just generic knowledge from pre-trained models. Whether you're building a chatbot for internal documents, an automated report analysis system, or a smart knowledge search engine, customizing an AI model to your own PDF files can be a game-changer.

In this tutorial, you’ll learn how to train a custom AI model on your PDF documents using LangChain, one of the most powerful open-source frameworks for building LLM-powered applications. Instead of actually retraining a base model (which is expensive and unnecessary), you’ll use an industry-standard approach:

Ingest your PDF files → chunk text → embed it → store in a vector database → query it intelligently using LangChain.

By the end of this guide, you will:

• Load PDF documents using LangChain loaders

• Split and chunk large PDF texts efficiently

• Generate embeddings using OpenAI, HuggingFace, or any preferred provider

• Store embeddings in a vector database (FAISS or Pinecone)

• Build a RAG (Retrieval-Augmented Generation) pipeline

• Build a simple chatbot or Q&A interface that answers using only your PDFs

• Learn best practices for improving accuracy, chunking, and retrieval

This tutorial is designed to be 100% practical, with full code samples and a ready-to-run implementation.

Project Setup & Requirements

In this section, we’ll set up everything needed to build your custom PDF-trained AI model using LangChain.

1. What We’re Building

We’ll create a Python project that:

1. Loads and processes PDF files

2. Converts them into clean, searchable text

3. Splits the content into semantic chunks

4. Generates embeddings

5. Stores embeddings in a vector store

6. Builds a Retrieval-Augmented Generation (RAG) pipeline using LangChain

7. Enables querying the system—like your own private ChatGPT trained on PDFs

2. Tools & Libraries

Purpose	Library
AI framework	LangChain
PDF loading	langchain-community loaders (PyPDFLoader)
Embeddings	OpenAI / HuggingFace / Ollama embeddings
Vector database	FAISS (local) or Pinecone
Environment config	python-dotenv
Optional	Streamlit (for UI chatbot)

3. Requirements

Python Version

You need Python 3.10+.

Check your version:

python --version

4. Create the Project

mkdir pdf-qa-langchain
cd pdf-qa-langchain

5. Create a Virtual Environment

python -m venv venv
source venv/bin/activate      # Mac / Linux
venv\Scripts\activate         # Windows

6. Install Dependencies

Option A: Using OpenAI Embeddings

pip install langchain langchain-community openai faiss-cpu python-dotenv pypdf

Option B: Using HuggingFace Embeddings

pip install langchain langchain-community sentence-transformers faiss-cpu python-dotenv pypdf

Option C: Using Ollama (Local Models Like Llama 3)

pip install langchain langchain-community ollama faiss-cpu python-dotenv pypdf

7. Setup Environment Variables

Create a .env file:

OPENAI_API_KEY=your_api_key_here

For HuggingFace:

HUGGINGFACEHUB_API_TOKEN=your_token_here

For Pinecone (if used later):

PINECONE_API_KEY=your_key_here

8. Project Structure

Here’s the clean structure we’ll use:

pdf-qa-langchain/
│
├── data/
│   └── your-pdfs-here.pdf
│
├── embeddings/
│   └── vectorstore.faiss        # Auto-generated
│
├── .env
├── ingest.py                    # Process PDFs + create embeddings
├── query.py                     # Ask questions to your PDF-trained AI
└── requirements.txt

9. Download Your PDFs

Place all your documents in:

data/

The system will automatically load every PDF in this folder.

Loading and Parsing PDF Documents

In this section, we’ll load PDF files and extract clean, searchable text using LangChain’s built-in PDF loaders.

1. Loading PDFs with LangChain

LangChain provides PDF loaders in langchain-community.
We’ll use PyPDFLoader, the most stable and reliable for text extraction.

Create a new file:

ingest.py

Add the following:

import os
from langchain_community.document_loaders import PyPDFLoader

DATA_PATH = "data"


def load_pdfs():
    documents = []
    for file in os.listdir(DATA_PATH):
        if file.endswith(".pdf"):
            loader = PyPDFLoader(os.path.join(DATA_PATH, file))
            docs = loader.load()
            documents.extend(docs)
    return documents


if __name__ == "__main__":
    docs = load_pdfs()
    print(f"Loaded {len(docs)} pages from PDFs.")

2. How PyPDFLoader Works

Each PDF page becomes a Document object with:

• page_content: Text extracted

• metadata: Includes filenames, page numbers, etc.

Example of a loaded document:

print(docs[0].metadata)

Output:

{
  'source': 'data/manual.pdf',
  'page': 1
}

This metadata is critical later for traceability.

3. Cleaning PDF Text (Optional but Recommended)

Many PDFs include:

• Headers

• Footers

• Page numbers

• Repeated sections

We can clean the text by applying a simple filter.

Add this:

def clean_text(text: str) -> str:
    # Basic cleanup—extend as needed
    lines = text.split("\n")
    cleaned = [line.strip() for line in lines if line.strip()]
    return " ".join(cleaned)

Apply cleaning when loading:

for d in docs:
    d.page_content = clean_text(d.page_content)

4. Combine Everything

Updated ingest.py (PDF loading section):

import os
from langchain_community.document_loaders import PyPDFLoader

DATA_PATH = "data"


def clean_text(text: str) -> str:
    lines = text.split("\n")
    cleaned = [line.strip() for line in lines if line.strip()]
    return " ".join(cleaned)


def load_pdfs():
    documents = []
    for file in os.listdir(DATA_PATH):
        if file.endswith(".pdf"):
            loader = PyPDFLoader(os.path.join(DATA_PATH, file))
            docs = loader.load()
            for d in docs:
                d.page_content = clean_text(d.page_content)
            documents.extend(docs)
    return documents


if __name__ == "__main__":
    docs = load_pdfs()
    print(f"Loaded and cleaned {len(docs)} pages.")

5. Test the PDF Loader

Run:

python ingest.py

Expected output:

Loaded and cleaned 42 pages.

You now have clean, ready-to-chunk PDF documents.

Chunking and Splitting PDF Text

Chunking is one of the most important steps in building a high-quality RAG system.
Good chunking = better recall, better answers, lower hallucinations.

In this section, we’ll use LangChain’s RecursiveCharacterTextSplitter to break your PDF text into smart, overlapping chunks.

1. Why Chunking Matters

PDF pages can be long, unstructured, or contain multiple topics.
LLMs perform best when information is split into small, coherent pieces.

Ideal chunk size

• Chunk size: 500–1,000 characters

• Chunk overlap: 50–150 characters

• Prevents loss of meaning between boundary splits

• Helps maintain topic continuity

2. Using RecursiveCharacterTextSplitter

Add this to ingest.py:

from langchain.text_splitter import RecursiveCharacterTextSplitter

Then create the splitter:

def chunk_documents(documents):
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=150, separators=["\n\n", "\n", ".", " ", ""]
    )
    return splitter.split_documents(documents)

3. Why RecursiveCharacterTextSplitter?

This splitter:

1. Tries to break on logical separators (paragraphs → sentences → spaces).

2. Only falls back to character splitting if forced.

3. Creates semantic-friendly chunks that maintain context.

4. Full Chunking Pipeline

Update ingest.py:

def chunk_documents(documents):
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=150, separators=["\n\n", "\n", ".", " ", ""]
    )
    chunks = splitter.split_documents(documents)
    return chunks


if __name__ == "__main__":
    docs = load_pdfs()
    chunks = chunk_documents(docs)
    print(f"Loaded {len(docs)} pages.")
    print(f"Created {len(chunks)} chunks.")

Run:

python ingest.py

Example output:

Loaded and cleaned 1 pages.
Loaded 1 pages.
Created 1 chunks.

5. Best Practices for Chunking PDF Text

✔ Keep chunks between 300–1200 characters

Below 300 = too little info
Above 1200 = weaker retrieval

✔ Use overlap to preserve context

Overlap keeps sentences intact across chunks.

✔ Avoid splitting in the middle of tables or code

If your PDFs contain structured data, reduce chunk_size for safety.

✔ Keep metadata

LangChain preserves metadata automatically.

6. Ready for Embeddings

You now have:

• Clean PDF text

• Split into optimized chunks

• Ready for vectorization

Next, we’ll convert chunks into embeddings.

Creating Embeddings for Your PDF Chunks

Now that your PDF text is cleaned and chunked, the next step is to convert each chunk into embeddings—numeric vector representations of your text. These vectors will later be stored in a vector database and used for fast, semantic search in the RAG pipeline.

1. What Are Embeddings?

Embeddings transform text into high-dimensional vectors.
These vectors capture semantic meaning, so:

• Similar text → close vectors

• Different text → distant vectors

This is how your AI model “understands” your custom PDF.

2. Choose Your Embedding Provider

LangChain supports multiple embedding generators.

You can use:

Option A — OpenAI Embeddings (most accurate)

Model: text-embedding-3-large or 3-small

Option B — HuggingFace Sentence Transformers (free & offline)

Model: all-MiniLM-L6-v2, etc.

Option C — Local Ollama Embeddings (Llama 3, Mistral, etc.)

3. Install Required Libraries (if you haven't)

OpenAI

pip install openai

HuggingFace

pip install sentence-transformers

Ollama

(Requires Ollama installed locally)

4. Add Embeddings Code to ingest.py

Option A: OpenAI Embeddings

from langchain_openai import OpenAIEmbeddings
from dotenv import load_dotenv

load_dotenv()

def create_embeddings():
    return OpenAIEmbeddings(model="text-embedding-3-small")

Option B: HuggingFace Embeddings

from langchain_community.embeddings import HuggingFaceEmbeddings

def create_embeddings():
    return HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

Option C: Ollama Local Embeddings

from langchain_community.embeddings import OllamaEmbeddings

def create_embeddings():
    return OllamaEmbeddings(model="llama3")

5. Generate Embeddings + Store in Vector Database

We’ll use FAISS, a fast local vector store.

Add this:

from langchain_community.vectorstores import FAISS

def store_embeddings(chunks, embeddings):
    vectorstore = FAISS.from_documents(chunks, embeddings)
    vectorstore.save_local("embeddings")
    print("Vector store saved to /embeddings directory")

6. Full Ingestion Pipeline

Update ingest.py:

if __name__ == "__main__":
    # 1. Load
    docs = load_pdfs()
    print(f"Loaded {len(docs)} pages.")

    # 2. Chunk
    chunks = chunk_documents(docs)
    print(f"Created {len(chunks)} chunks.")

    # 3. Embeddings
    embeddings = create_embeddings()

    # 4. Store
    store_embeddings(chunks, embeddings)

    print("Ingestion complete.")

7. Run the Embedding Process

Run:

python ingest.py

Expected output:

Loaded 7 pages.
Created 32 chunks.
Vector store saved to /embeddings
Ingestion complete.

Your local embeddings/ folder now contains:

• index.faiss → Vector index

• index.pkl → Metadata and document mapping

This is now your trained custom knowledge base.

Building the Retrieval-QA (RAG) Pipeline

Now that your PDF chunks are embedded and stored in FAISS, it’s time to build the retrieval pipeline—the heart of your custom AI model.
This is where your application retrieves relevant chunks and uses an LLM to generate answers based on your documents.

1. What Is RAG?

Retrieval-Augmented Generation (RAG) enhances an LLM by giving it access to your data.

Workflow:

1. User asks a question

2. Retriever searches FAISS for relevant chunks

3. Top chunks are passed to the LLM

4. LLM generates an answer grounded in your PDFs

This avoids hallucination and ensures responses are based on your documents.

2. Create a query.py File

Create:

query.py

You'll load your vector store, attach a retriever, and build the RAG chain.

3. Load FAISS Vector Store

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings
from dotenv import load_dotenv

load_dotenv()


def load_vectorstore():
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    vectorstore = FAISS.load_local(
        "embeddings", embeddings, allow_dangerous_deserialization=True
    )
    return vectorstore

allow_dangerous_deserialization=True is required starting LangChain 0.3 for local FAISS.

4. Create a Retriever

Once loaded:

def get_retriever(vectorstore):
    retriever = vectorstore.as_retriever(
        search_kwargs={"k": 4}   # return top 4 chunks
    )
    return retriever

You can experiment with k = 3–8.

5. Choose an LLM (OpenAI / HuggingFace / Ollama)

Option A — OpenAI (Recommended)

from langchain_openai import ChatOpenAI

def get_llm():
    return ChatOpenAI(model="gpt-4o-mini")

Option B — Ollama (Local)

from langchain_community.llms import Ollama

def get_llm():
    return Ollama(model="llama3")

6. Build the RAG Chain

Using LangChain Expression Language (LCEL):

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

def build_rag(retriever):
    llm = ChatOpenAI(model="gpt-4o-mini")

    prompt = ChatPromptTemplate.from_template(
        """
You are an AI assistant that answers questions based ONLY on the provided context.

<context>
{context}
</context>

Question: {question}
"""
    )

    # LCEL pipeline
    return (
        {"context": retriever, "question": "question"}
        | prompt
        | llm
        | StrOutputParser()
    )

7. Create a Function to Ask Questions

def ask(question: str):
    vs = load_vectorstore()
    retriever = get_retriever(vs)
    rag = build_rag(retriever)
    return rag.invoke({"question": question})

8. Add CLI Execution Block

if __name__ == "__main__":
    while True:
        q = input("\nAsk a question (or 'exit'): ")
        if q.lower() == "exit":
            break
        print("\nANSWER:\n", ask(q))

9. Test Your RAG System

Run:

python query.py

Try:

What is LangChain used for?

You’ll get an answer based entirely on your PDF’s content.

10. Optional: Print Sources (Highly Recommended)

Add this inside ask() if you want to see which PDF chunks were used:

sources = result.get("context", [])
for doc in sources:
    print("\nSource page:", doc.metadata.get("page"))
    print(doc.page_content[:200], "...")

🎉 Your Custom AI Model Is Now Functional

You have successfully built:

• PDF ingestion

• Text cleaning

• Chunking

• Embedding generation

• Vector store

• Retriever

• LLM answering

• Full RAG pipeline

This is the core of document-trained AI apps.

Creating a Chatbot UI (Optional, Streamlit)

Now that your RAG pipeline is working in the terminal, let’s create a simple, clean, modern chatbot UI using Streamlit.
This UI allows users to upload questions and interact with your custom-trained PDF AI model in a friendly web interface.

1. Install Streamlit

Run:

pip install streamlit

2. Create app.py

Inside your project root, create:

app.py

3. Streamlit UI + RAG Integration (LangChain 0.3 Compatible)

Here is a fully working chatbot UI using your existing RAG code:

import streamlit as st
from dotenv import load_dotenv
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

# Load vector store
def load_vectorstore():
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    return FAISS.load_local(
        "embeddings",
        embeddings,
        allow_dangerous_deserialization=True
    )

# Build RAG pipeline
def build_rag():
    vectorstore = load_vectorstore()
    retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

    llm = ChatOpenAI(model="gpt-4o-mini")

    prompt = ChatPromptTemplate.from_template("""
You are a helpful assistant. Answer the user's question using ONLY the context below.
If the answer is not in the context, say you cannot find it.

<context>
{context}
</context>

Question: {question}
""")

    rag_chain = (
        {"context": retriever, "question": "question"}
        | prompt
        | llm
        | StrOutputParser()
    )

    return rag_chain

rag_chain = build_rag()

# --- Streamlit UI ---

st.set_page_config(page_title="PDF AI Chatbot", page_icon="📄")

st.title("📄 PDF AI Chatbot")
st.write("Ask questions based on your custom PDF-trained model.")

# Initialize session history
if "history" not in st.session_state:
    st.session_state.history = []

question = st.text_input("Enter your question:")

if st.button("Ask") and question.strip():
    answer = rag_chain.invoke({"question": question})

    st.session_state.history.append((question, answer))

# Chat history UI
for q, a in st.session_state.history:
    st.markdown(f"**🧑‍💻 You:** {q}")
    st.markdown(f"**🤖 AI:** {a}")
    st.markdown("---")

4. Run the Chatbot UI

Start Streamlit:

streamlit run app.py

You’ll see a browser window open automatically.

5. What You Get

✔ Modern chat-style interface
✔ Messages stored in session state
✔ Answers grounded in your PDF content
✔ Uses your FAISS embeddings + RAG pipeline
✔ Fast and lightweight, no backend server required

6. Optional Enhancements

I can help you add:

• File uploader (upload PDFs directly in the UI)

• Chat bubbles with colors

• Model selection (OpenAI / HuggingFace / Ollama)

• Source citations (show PDF page numbers)

• Dark mode

• Big enterprise-style layout

Just tell me if you want any of these.

Optimizing Your Model (Accuracy, Speed, Cost)

Your RAG pipeline is now functional and wrapped in a simple UI.
This section will help you optimize the system for maximum accuracy, fast responses, and minimal costs.

1. Improving Accuracy

a. Tune Chunk Size & Overlap

Chunking impacts retrieval more than any other step.

Recommended settings:

• Chunk size: 800–1,200 characters

• Overlap: 100–200 characters

Why:

Larger chunks → more context
Overlap → smooth sentence continuity

Update in ingest.py:

RecursiveCharacterTextSplitter(
    chunk_size=1200,
    chunk_overlap=180
)

b. Increase Retriever Depth (k)

In your retriever:

retriever = vectorstore.as_retriever(search_kwargs={"k": 6})

More chunks retrieved = better answers (but slower LLM input).

Good values:

Data Type	Recommended k
Narratives	4–6
Technical manuals	6–8
Legal/Policies	8–10

c. Add a Better Prompt Template

Current prompt is basic.
Upgrade to a context-aware grounding prompt:

prompt = ChatPromptTemplate.from_template("""
You are a professional assistant that must answer using ONLY the context provided.

If the answer is not in the context, reply:
"I cannot find information about that in the document."

Context:
{context}

Question: {question}

Answer clearly and concisely:
""")

This eliminates hallucinations.

d. Switch to Higher-Quality Embedding Models

Best embedding models:

• OpenAI text-embedding-3-large

• HuggingFace: all-mpnet-base-v2

• Cohere embed-english-v3.0

• Local: nomic-embed-text

Use:

OpenAIEmbeddings(model="text-embedding-3-large")

High-quality embeddings = better vector search = better answers.

e. Use Re-Ranking (Optional, Powerful)

Add a second step using a local cross-encoder model to re-rank retrieved chunks.

Tools:

• sentence-transformers cross-encoder

• Cohere Rerank

This increases retrieval accuracy by 10–30%.

If you want, we can add a re-ranking step in Section 10.

2. Improving Speed

a. Use a Faster LLM

Swap to:

ChatOpenAI(model="gpt-4o-mini")

or for local:

Ollama(model="llama3")

Mini models = 3× faster and cheaper.

b. Reduce Chunk Size

If latency is more important than accuracy:

chunk_size=600
chunk_overlap=100

Smaller chunks = faster prompt processing.

c. Use FAISS GPU (Optional)

If you have a CUDA-capable GPU:

pip install faiss-gpu

FAISS GPU speeds up retrieval 10–20×.

d. Cache Responses

Add caching to avoid re-running similar queries.

from langchain_core.caches import InMemoryCache
from langchain.globals import set_llm_cache

set_llm_cache(InMemoryCache())

Instant responses for repeat queries.

3. Reducing Costs

a. Use Smaller Embedding Models

Instead of 3-large, use:

OpenAIEmbeddings(model="text-embedding-3-small")

b. Use Local Models

Run the entire RAG pipeline offline with:

Ollama(model="llama3")

or

HuggingFaceEmbeddings()

c. Limit Retrieved Tokens

Adjust:

search_kwargs={"k": 3}

Lower k = less prompt length = cheaper LLM calls.

4. Recommended Profiles

🔥 High Accuracy

• Chunk: 1200 / overlap 180

• Embeddings: OpenAI 3-large

• k = 6–8

• GPT-4o or Llama 3 70B

• Optional: Rerank

⚡ High Speed

• Chunk: 600

• Embeddings: 3-small

• k = 3

• gpt-4o-mini or llama3 8B

💸 Low Cost

• Local embeddings

• Local LLM

• k = 3

• Small chunks

5. Summary

In this section, you learned how to improve:

✔ Model accuracy

Through embeddings, prompt tuning, and retriever optimization

✔ Speed

By adjusting chunk sizes and using faster models

✔ Cost

By choosing the right embedding and generation strategy

Adding Source Citations (Show PDF Pages & Snippets)

Up to now, your AI correctly answers questions using your custom PDF knowledge base — but it doesn’t show where the answer came from.

In this section, you will enhance your RAG pipeline so it returns:

✔ PDF page numbers
✔ Text snippets
✔ Source filenames
✔ Chunk metadata

This dramatically increases trust, auditability, and debuggability.

1. Why Add Citations?

Source citations allow you to:

• Verify the answer is grounded in your PDFs

• Debug incorrect responses

• Build enterprise-grade compliance tools

• Build knowledge bases with traceable provenance

2. Retrieve Documents Along With the Answer

In LangChain 0.3+, the easiest way to attach citations is to separate retrieval from generation.

Instead of passing the retriever directly into LCEL as a pipe component, we manually fetch the documents.

Update query.py:

def ask_with_sources(question: str):
    vectorstore = load_vectorstore()
    retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

    # Step 1: Get relevant chunks
    docs = retriever.invoke(question)

    # Combine all chunks into a single context block
    context = "\n\n".join(d.page_content for d in docs)

    # Step 2: Build prompt
    llm = ChatOpenAI(model="gpt-4o-mini")
    prompt = f"""
You are an AI assistant who answers questions using ONLY the context below.
If the context does not contain the answer, say you cannot find it.

Context:
{context}

Question: {question}

Answer:
"""

    answer = llm.invoke(prompt).content

    return answer, docs

This gives you:

• The answer

• The exact documents used to produce it

3. Print Citations in the Terminal

Add:

answer, sources = ask_with_sources(q)
print("\nANSWER:\n", answer)

print("\nSOURCES:")
for doc in sources:
    print(f"- Page: {doc.metadata.get('page')} | File: {doc.metadata.get('source')}")
    print("  Snippet:", doc.page_content[:200], "...\n")

Example output:

ANSWER:
LangChain enables developers to build LLM applications using retrieval, chaining, and vector stores.

SOURCES:
- Page: 3 | File: data/sample-pdf-for-langchain.pdf
  Snippet: LangChain enables developers to build applications powered by large language models ...

4. Adding Citations in Streamlit UI

Open app.py and modify your answer block.

Below is the answer:

with st.expander("📄 Sources"):
    for doc in sources:
        st.markdown(f"**File:** {doc.metadata.get('source')}")
        st.markdown(f"**Page:** {doc.metadata.get('page')}")
        st.markdown("**Snippet:**")
        st.write(doc.page_content[:300] + "...")
        st.markdown("---")

Now the UI will show clickable, expandable source sections.

5. Improving Citations (Optional Enhancements)

a. Highlight text used

We can highlight text spans with:

• spaCy

• regex keyword matching

• LLM-based re-highlighting

b. Link directly to PDF pages

Using:

file.pdf#page=3

In Streamlit:

st.markdown(f"[Open Page {page}]({url}#page={page})")

c. Sort sources by score

If using a retriever with scoring:

docs = retriever.get_relevant_documents(question)
docs = sorted(docs, key=lambda d: d.metadata["score"])

d. Deduplicate overlapping chunks

Chunk overlap can cause duplicate citations; remove duplicates via:

unique = { (d.metadata['source'], d.metadata['page']): d for d in docs }
docs = list(unique.values())

6. Summary

You now have:

✔ Accurate answer

✔ Sources (pages, filenames)

✔ Snippet preview

✔ Streamlit integration

Your RAG system is now transparent and production-ready.

Optional Advanced Features (Re-ranking, Multi-PDF Chat, Upload UI, Memory, etc.)

Now that your core RAG system is fully functional with citations and a UI, we can extend it with powerful advanced features used in production-grade applications.

This section covers:

1. Re-ranking for higher accuracy

2. Multi-PDF conversation & retrieval

3. Upload PDFs directly in the UI

4. Conversation memory

5. Streaming responses

6. Chat history storage

7. Summaries, document QA, and advanced tools

1. Re-Ranking (Massive Accuracy Boost)

Retrievers retrieve via cosine similarity.
But cosine similarity is not always accurate for semantic relevance.

Enter re-ranking — a second filtering step that scores retrieved chunks using a small cross-encoder.

Best reranker models:

• cross-encoder/ms-marco-MiniLM-L-6-v2

• Cohere rerank-english-v3.0 (API-based)

• Jina AI reranker

✔ Add Re-Ranking with HuggingFace (Free & Local)

Install:

pip install sentence-transformers

Add to query.py:

from sentence_transformers import CrossEncoder

reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

Update retrieval:

def rerank_documents(question, docs):
    pairs = [(question, doc.page_content) for doc in docs]
    scores = reranker.predict(pairs)

    # Attach scores to docs
    for doc, score in zip(docs, scores):
        doc.metadata["rerank_score"] = float(score)

    # Sort highest first
    return sorted(docs, key=lambda d: d.metadata["rerank_score"], reverse=True)

Apply after the retriever:

raw_docs = retriever.invoke(question)
docs = rerank_documents(question, raw_docs)

📈 Benefit:

Accuracy improves 10–40%, especially for complex questions.

2. Multi-PDF Chat (Multiple Documents at Once)

If your FAISS store includes multiple PDFs, you're good — but you can also group results by document.

Add this grouping:

from collections import defaultdict

def group_by_pdf(docs):
    groups = defaultdict(list)
    for d in docs:
        groups[d.metadata["source"]].append(d)
    return groups

This lets you display:

• Which PDF contributed the most

• Which pages were used

• Cross-document answers

Useful for enterprise knowledge bases.

3. Upload PDFs in Streamlit UI

Modify app.py:

uploaded_files = st.file_uploader(
    "Upload PDF files",
    type=["pdf"],
    accept_multiple_files=True
)

After upload:

1. Save PDFs to data/

2. Run the ingestion pipeline again

3. Load updated FAISS into the session

Pseudo-code:

if uploaded_files:
    for pdf in uploaded_files:
        with open(f"data/{pdf.name}", "wb") as f:
            f.write(pdf.getbuffer())

    st.success("PDF uploaded! Rebuilding embeddings...")

    run_ingest()  # Your ingestion pipeline

    st.session_state.vectorstore = load_vectorstore()

This creates a dynamic document-knowledge chatbot.

4. Add Conversation Memory

Memory allows the chatbot to:

• Understand follow-up questions

• Keep context

• Continue discussions naturally

Use LangChain's memory:

from langchain.memory import ConversationBufferMemory

Integrate memory:

memory = ConversationBufferMemory(return_messages=True)

messages = memory.load_memory_variables({})["history"]
messages.append({"role": "user", "content": question})

answer = llm.invoke(messages).content

memory.save_context({"input": question}, {"output": answer})

5. Streaming Responses (Just like ChatGPT)

In Streamlit:

response = ""
for chunk in llm.stream(prompt):
    response += chunk
    st.write(response)

The answer appears token-by-token.

6. Persistent Chat History (Local or DB)

Local storage:

import json

with open("history.json", "a") as f:
    f.write(json.dumps({"q": question, "a": answer}) + "\n")

Or use SQLite:

pip install sqlmodel

7. Summaries, Document QA, and Tools

1. Summaries

summary = llm.invoke(f"Summarize:\n{context}").content

 

2. Extract structured data

json_output = llm.invoke("Extract key facts as JSON:\n" + context).content

 

3. Compare two PDFs

"Compare the following two contexts:\nA:{context1}\nB:{context2}"

 

8. Recommended Setup for Production

Feature	Purpose
Re-ranking	Higher accuracy
Chunking tuned by document type	More natural results
Memory	Natural multi-turn chat
Citations	Trust & compliance
PDF upload	Multi-document chat
Streaming	Better UX
SQLite/Postgres history	Audit trail
Health monitoring	Reliability

You've now transformed your simple PDF RAG chatbot into a feature-rich AI knowledge system used in real-world production apps.

Conclusion

You’ve just built a complete, production-ready AI system trained on your own PDF documents — using a modern LangChain 0.3+ pipeline. This tutorial guided you from raw PDFs to a fully interactive chatbot UI with citations, re-ranking, optimization strategies, and optional advanced features like file uploads and conversation memory.

This is the same architecture used by top companies building internal knowledge assistants, document search engines, and AI-powered helpdesks.

💡 What You Accomplished

Throughout this tutorial, you:

✔ Loaded and cleaned PDF documents

Using PyPDFLoader and custom text cleaning

✔ Split documents into optimized semantic chunks

With RecursiveCharacterTextSplitter

✔ Generated high-quality embeddings

Using OpenAI or local HuggingFace/Ollama models

✔ Stored vectors in FAISS

Building a fast, local vector database

✔ Built a complete RAG pipeline

With LCEL and retrieval → context → LLM → answer

✔ Added citations with page numbers and snippets

For transparency and verifiability

✔ Designed a Streamlit chatbot UI

Offering a modern, interactive user experience

✔ Explored advanced features

Re-ranking, multi-PDF chat, file upload, memory, and more

🚀 Where to Go Next

You now have a solid foundation for:

• Internal AI knowledge bases

• AI-powered PDF assistants

• Legal / finance document analysis tools

• Enterprise RAG systems

• Document QA and reporting systems

To push this further, you might explore:

• Vector stores like Pinecone, Weaviate, or Milvus

• Hybrid search (BM25 + embeddings)

• Fine-tuning lightweight local models

• Caching + prompt optimization

• Deploying via Docker, Railway, or HuggingFace Spaces

📦 Final Deliverables (What You Have Now)

• Complete RAG ingestion pipeline

• PDF-trained AI model

• Query script

• Chatbot UI

• Citations system

• Advanced features and enhancements

• A reusable project template

🎉 Closing Note

You’ve built something powerful — a custom AI system that understands your documents.
This is the future of applied AI: private, domain-specific, and grounded in your organization’s knowledge.

You can find the full source code on our GitHub.

That's just the basics. If you need more deep learning about AI, ML, and LLMs, you can take the following cheap course:

• The AI Engineer Course 2025: Complete AI Engineer Bootcamp
• The Complete AI Guide: Learn ChatGPT, Generative AI & More
• The Complete Agentic AI Engineering Course (2025)
• Generative AI for Beginners
• Complete Data Science,Machine Learning,DL,NLP Bootcamp
• Complete MLOps Bootcamp With 10+ End To End ML Projects
• AI & ML Made Easy: From Basic to Advanced (2025)
• Machine Learning for Absolute Beginners - Level 1
• LLM Engineering: Master AI, Large Language Models & Agents
• A deep understanding of AI large language model mechanisms
• Master LLM Engineering & AI Agents: Build 14 Projects - 2025
• LangChain- Develop AI Agents with LangChain & LangGraph

Thanks!