LangChain is glue. It wires prompts, models, retrievers, parsers, and memory into one pipeline so you stop hand-writing every call. This page teaches it to interview depth, and stays honest about when it earns its keep and when a plain SDK call is clearer.
LangChain is an orchestration framework. It defines a common interface (a Runnable) for every stage, prompt, model, retriever, parser, so you compose them into a chain and call .invoke() once. The data passes through the stages in order, each transforming it.
bedrock.converse(...) or client.messages.create(...) is shorter and clearer, and you can read exactly what goes to the model. LangChain pays off when you have multiple stages, swappable components, and retrieval or memory. If the app is one call, skip it.Now the words: the core abstraction is the Runnable (anything with .invoke, .stream, .batch). Chains are Runnables composed together. The company splits into langchain-core (interfaces), provider packages like langchain-aws / langchain-openai, and langchain-community (integrations). LangGraph is the newer library for stateful, branching, agentic flows.
| Block | One line |
|---|---|
| PromptTemplate | A string with {slots} that fills in variables to build the final prompt. |
| ChatModel | The wrapper around the LLM (ChatBedrock, ChatOpenAI); takes messages, returns an AIMessage. |
| OutputParser | Turns the raw model reply into what you want: a string, JSON, or a typed object. |
| DocumentLoader | Reads a source (PDF, web page, Notion, S3) into Document objects. |
| TextSplitter | Cuts long docs into chunks that fit the context window and embed cleanly. |
| Embeddings | Turns each chunk (and the query) into a vector of numbers representing meaning. |
| VectorStore | Stores those vectors and does similarity search (FAISS, Pinecone, pgvector). |
| Retriever | The query-time interface: give it a question, it returns the most relevant chunks. |
| Memory | Holds chat history across turns so the model remembers the conversation. |
Now the words: a Document is text plus metadata. Embeddings and VectorStore are covered in depth in the companion Embeddings tutorial and Pinecone tutorial in this folder. A Retriever is any object with get_relevant_documents(query); a vector store exposes one via .as_retriever().
cat file | grep err | wc -l reads left to right, each command's output feeds the next. LCEL uses the exact same | idea: prompt | model | parser means fill the prompt, send to the model, parse the reply.LCEL (LangChain Expression Language) overloads the | operator so any Runnable composes with the next. The composed object is itself a Runnable, so it gets .invoke(), .stream(), .batch(), and async for free.
from langchain_core.prompts import ChatPromptTemplate from langchain_core.output_parsers import StrOutputParser from langchain_aws import ChatBedrock prompt = ChatPromptTemplate.from_template("Explain {topic} in one sentence.") model = ChatBedrock(model_id="anthropic.claude-sonnet-4-5-20250929-v1:0") parser = StrOutputParser() chain = prompt | model | parser # the pipe composes one Runnable print(chain.invoke({"topic": "vector databases"})) # -> "A vector database stores embeddings and finds nearest neighbors by similarity."
chain.stream(...) just works. That is the real reason to reach for LangChain over raw calls.Now the words: LCEL = LangChain Expression Language. The | is Python's __or__ overloaded on Runnables. A dict of Runnables (see next section) becomes a RunnableParallel that runs branches concurrently; RunnablePassthrough forwards input unchanged.
Now the words: RAG = Retrieval-Augmented Generation. "Stuffing" means concatenating retrieved chunks into the prompt (the stuff strategy); alternatives for many chunks are map_reduce and refine. The made-up answer is a hallucination: fluent, confident, wrong.
Two phases. Indexing (once, offline): load docs, split, embed, store. Querying (per request): embed the question, retrieve, build the prompt, call the model, parse.
from langchain_community.document_loaders import PyPDFLoader from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_aws import BedrockEmbeddings from langchain_community.vectorstores import FAISS # 1. load -> your rag-docs-chatbot read PDFs by hand; this is that step docs = PyPDFLoader("handbook.pdf").load() # 2. split into overlapping chunks that fit the window chunks = RecursiveCharacterTextSplitter( chunk_size=1000, chunk_overlap=150).split_documents(docs) # 3. embed + store (swap FAISS for Pinecone by changing this one line) emb = BedrockEmbeddings(model_id="amazon.titan-embed-text-v2:0") vectorstore = FAISS.from_documents(chunks, emb) vectorstore.save_local("faiss_index")
from langchain_core.prompts import ChatPromptTemplate from langchain_core.runnables import RunnablePassthrough from langchain_core.output_parsers import StrOutputParser from langchain_aws import ChatBedrock retriever = vectorstore.as_retriever(search_kwargs={"k": 3}) prompt = ChatPromptTemplate.from_template( "Answer using ONLY the context. If it is not there, say you don't know.\n\n" "Context:\n{context}\n\nQuestion: {question}") def format_docs(ds): return "\n\n".join(d.page_content for d in ds) # the dict runs both branches; {question} passes through untouched chain = ( {"context": retriever | format_docs, "question": RunnablePassthrough()} | prompt | ChatBedrock(model_id="anthropic.claude-sonnet-4-5-20250929-v1:0") | StrOutputParser() ) print(chain.invoke("How many PTO days carry over?"))
Prefer the higher-level helper? The library ships create_retrieval_chain + create_stuff_documents_chain, which do the same wiring with less code. They return the answer plus the source documents, handy for showing citations.
from langchain.chains import create_retrieval_chain from langchain.chains.combine_documents import create_stuff_documents_chain combine = create_stuff_documents_chain(llm, prompt) # prompt uses {context} rag = create_retrieval_chain(retriever, combine) out = rag.invoke({"input": "How many PTO days carry over?"}) print(out["answer"], out["context"]) # answer + the source chunks
{context: retriever, question: passthrough} | prompt | model | parser from memory proves you understand the framework, not just its shortcuts.Now the words: chunk_overlap keeps context from being cut mid-sentence at chunk boundaries. RecursiveCharacterTextSplitter splits on paragraphs, then sentences, then words, preserving structure. k is how many chunks to retrieve. The dict literal in LCEL is sugar for RunnableParallel.
Every vector store exposes .as_retriever(), so downstream code never knows which backend it hit. Swapping is one line:
# prototype: in-memory, zero infra from langchain_community.vectorstores import FAISS store = FAISS.from_documents(chunks, emb) # production: managed, scales to millions of vectors from langchain_pinecone import PineconeVectorStore store = PineconeVectorStore.from_documents(chunks, emb, index_name="handbook") # already on Postgres? reuse it from langchain_postgres import PGVector store = PGVector.from_documents(chunks, emb, connection=DB_URL) retriever = store.as_retriever(search_type="mmr", search_kwargs={"k": 4})
| Knob | What it does |
|---|---|
| Similarity (top-k) | Return the k nearest vectors by cosine distance. The default, fast and simple. |
| MMR | Maximal Marginal Relevance. Trades some similarity for diversity so you do not get four near-duplicate chunks. |
| Metadata filter | Restrict search to chunks matching a tag (e.g. {"dept": "HR"}) before ranking. |
| Score threshold | Drop weak matches so an off-topic question returns nothing instead of noise. |
How the vectors get created (the embedding models, dimensionality, cosine vs dot product) is its own topic. See the Embeddings tutorial for what a 1536-dimensional vector means, and the Pinecone tutorial for how a managed vector index actually stores and searches them at scale.
Now the words: ANN (approximate nearest neighbor) is the fast search algorithm vector stores use (HNSW, IVF). MMR re-ranks for diversity. Hybrid search blends vector similarity with keyword (BM25) matching. LangChain wraps all of these behind the same Retriever call.
Memory collects prior turns and injects them into the next prompt. Two strategies, and the choice is a cost tradeoff:
| Type | How | Cost tradeoff |
|---|---|---|
| Buffer | Keeps the full verbatim history and resends it every turn. | Perfect recall, but tokens grow every turn until you blow the window. |
| Windowed buffer | Keeps only the last N turns. | Bounded cost, but forgets anything older than N. |
| Summary | An LLM compresses old turns into a running summary. | Cheap and unbounded history, but lossy and adds a summarization call. |
from langchain_core.runnables.history import RunnableWithMessageHistory from langchain_community.chat_message_histories import ChatMessageHistory store = {} def get_history(session_id): return store.setdefault(session_id, ChatMessageHistory()) chat = RunnableWithMessageHistory( chain, get_history, input_messages_key="question", history_messages_key="history") # history is keyed by session, so users don't cross streams chat.invoke({"question": "What is the PTO policy?"}, config={"configurable": {"session_id": "user-42"}}) chat.invoke({"question": "And for part-timers?"}, # remembers turn 1 config={"configurable": {"session_id": "user-42"}})
ConversationBufferMemory classes are deprecated. Modern LangChain does history with RunnableWithMessageHistory (or LangGraph's persistence). Also: memory + RAG both eat the context window. A long chat history plus four retrieved chunks plus the system prompt can silently overflow the limit. Budget your tokens.Now the words: the context window is the max tokens per call. session_id partitions history per user. Summary memory adds an extra LLM call, so it trades money-per-turn for a smaller prompt. For a chatbot with logins like yours, persist history in a store (Redis, DynamoDB), not process memory.
ReAct = Reason + Act. The model emits a thought, chooses an action (a tool + inputs), reads the observation, and loops. Modern LangChain leans on native tool calling: you bind tools to the model and it returns structured calls you execute.
from langchain_core.tools import tool @tool def get_balance(account_id: str) -> str: """Look up an account balance by id.""" return db.lookup(account_id) llm_with_tools = llm.bind_tools([get_balance]) resp = llm_with_tools.invoke("What's the balance on account A-19?") print(resp.tool_calls) # [{'name':'get_balance','args':{'account_id':'A-19'}}] # you run the tool, feed the result back, model composes the final answer
Now the words: ReAct = Reason+Act prompting pattern. Tool calling (a.k.a. function calling) is the model returning structured tool invocations natively. AgentExecutor is the older loop runner; LangGraph is the current answer for stateful agents. Guard every agent with a max-iterations cap so it cannot loop forever.
| Gotcha | What happens | Fix |
|---|---|---|
| Context bloat | Retrieved chunks + chat history + system prompt silently blow the context window; calls truncate or error. | Cap k, trim history, count tokens before sending. |
| Silent cost | A "chatty" chain fires several LLM calls per request (rephrase, retrieve, summarize, answer). The bill balloons. | Log every call, prefer fewer stages, cache where you can. |
| Over-abstraction | The framework hides the actual prompt sent to the model, so you cannot tell why output is wrong. | Print the rendered prompt; do not debug blind. |
| Version churn | Fast breaking releases; the tutorial you found imports paths that no longer exist. | Pin versions; use langchain-core + provider packages. |
| Opaque chains | A deep chain fails and the stack trace tells you nothing useful. | Use LangSmith tracing or add plain logging between stages. |
| You didn't need it | A one-shot prompt wrapped in three LangChain classes is harder to read than a direct SDK call. | Use the raw SDK for simple calls; reserve LangChain for real pipelines. |
# 1. see exactly what the model receives print(prompt.invoke({"context": ctx, "question": q}).to_string()) # 2. trace the whole chain in LangSmith (set two env vars) import os os.environ["LANGCHAIN_TRACING_V2"] = "true" os.environ["LANGCHAIN_API_KEY"] = "ls-..." # now every .invoke() shows each stage's input/output, latency, and token cost
Now the words: LangSmith is LangChain's tracing and eval product. Token budget = tracking how many tokens each part of the prompt consumes. Pinning to langchain-core plus a provider package (langchain-aws) shields you from most churn in the big meta-package.