Agent Libraries#

There are many frameworks that make agentic systems easier to implement. One interested in agents will be glad to know that the vast majority of the most popular and well-documented AI agent development is happening in an ecosystem centered around open models. Here is a guide to the most prominent frameworks, models, and learning resources:

• LlamaIndex: The leading framework for building powerful Retrieval-Augmented Generation (RAG) agents (agents that can query our own data). While its name is inspired by Meta’s Llama model, it is fully model-agnostic. Its documentation and tutorials heavily feature Llama, OpenAI’s GPT, and Anthropic’s Claude.

• OpenAI Agents SDK

• LangGraph

  TIP

  • LangChain is for building a RAG (Retrieval Augmented Generation) system, a simple chatbot, or a data extraction pipeline where the steps are known in advance
  • LangGraph is for building an Agent that needs to use tools, handle ambiguous instructions, collaborate with other agents, or requires a “human-in-the-loop” workflow.

• (Hugging Face) smolagents

Quick Start Explore in Kaggle#

These frameworks make it easy to get started by simplifying standard low-level tasks like calling LLMs, defining and parsing tools, and chaining calls together. However, they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug. They can also make it tempting to add complexity when a simpler setup would suffice.

OpenML suggests that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code. If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about what’s under the hood are a common source of customer error.

Those who want to pick up of agent basics quickly can check out this Jupyter notebook Explore in Kaggle

The “Inner Speech” Theory (Cognitive Psychology)#

This is the deepest, most foundational layer. The reason the ReAct architecture works so well is that it computationally mimics a core human cognitive function.

1. Vygotsky’s Thought and Language: This is the origin of the “private speech” (thinking aloud) to “inner speech” (internal monologue) theory. It explains why language is a tool for self-regulation and planning.
2. Baddeley’s “Working Memory” Model: This provides the cognitive architecture, explaining the “Phonological Loop” (the “inner voice”) as the specific mechanism for holding and manipulating verbal information (i.e., the plan) in short-term memory.

The ReAct paper essentially created a Vygotskian agent. It forces the LLM to use “private speech” (the Thought trace) to regulate its own behavior, which is a far more robust method than simple stimulus-response.

The “Agent/Policy” Theory (Reinforcement Learning)#

This is the most important piece. ReAct assumes we are fluent in the language of Reinforcement Learning (RL). When it uses words like “policy,” “agent,” “state,” “action,” and “observation,” it’s borrowing the entire formal mathematical framework of Markov Decision Processes (MDPs).

RL is a fascinating intersection of computer science, optimal control, and advanced mathematics. To learn it “deeply” means starting with the mathematical foundations before jumping into the “deep” (neural network) part. Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto is the foundational text for the entire field and formally introduces the core mathematical concepts including:

• Markov Decision Processes (MDPs): The mathematical framework for all RL problems.

• Bellman Equations: The fundamental equations that all RL algorithms try to solve.

• Dynamic Programming: The theoretical (but often impractical) “perfect” solution.

• Monte Carlo Methods & Temporal-Difference (TD) Learning: The breakthroughs that make RL practical (this includes Q-Learning and SARSA).

  TIP

  It is not an exaggeration to say that without the Monte Carlo (MC) method, modern AI simply would not exist. The fundamental reason is that AI is essentially the study of high-dimensional probability distributions.

  To systematically study Monte Carlo method, here is the recommended resource List:

  • Statistical Mechanics: Algorithms and Computations by Werner Krauth

    Understand MC as a method for solving definite integrals (calculating areas/volumes) in high dimensions. If we want to find the area of an irregular shape, we can’t use calculus. Instead, enclose it in a box, throw 10,000 random darts at the box, and count how many land inside the shape. The ratio gives us the approximated area. This is exactly how we calculate “Expected Value” (the core of RL). We don’t solve the integral; we sample rewards and average them.

As we read it, we will have a series of “aha!” moments. We will see that the ReAct loop is essentially an implementation of an RL “policy,” where:

• State = The history of all past thoughts, actions, and observations (the “scratchpad”).
• Action = The choice to either generate a Thought or an Act (like calling a tool).
• Policy = The LLM itself, which has been “prompted” to decide the best action given the current state.

OpenML provides the following hand-made study materials to assist the study of Reinforcement Learning: An Introduction:

• Anki cards for selected concepts and exercises (with solutions) from the book

The “Reasoning” and “Acting” Theory (Other LLM Papers)#

Indicating in its title, ReAct didn’t invent “reasoning” or “acting” in LLMs. It was the first to synergize them. The paper was a direct response to two other lines of research that were popular at the time.

• The “Reasoning” (Chain-of-Thought) Track: The ReAct paper is building directly on the Chain-of-Thought Prompting Elicits Reasoning in Large Language Models paper, which showed that we could get an LLM to “reason” by simply prompting it to “think step-by-step.” The ReAct Thought step is a more structured version of CoT.
• The “Acting” (Tool-Use) Track: It also builds on papers like Toolformer and other research showing that LLMs could be prompted to use external tools (like a calculator or a search API).

The ReAct paper’s core argument is that both of these tracks are flawed on their own. “Reasoning-only” agents hallucinate, and “Acting-only” agents can’t plan.

Building the Core ReAct Agent Loop#

A perfect starting point is a tutorial that implements the ReAct loop directly. These guides show us how to build the “brain” of the agent. We will write Python code to manage the “Thought, Act, Observation” cycle using nothing but a standard LLM API.

A great resource for this is “Building an AI agent from scratch in Python” by Leonie Monigatti. It’s a clean, simple implementation that uses an API (Anthropic’s, but it’s trivial to swap for OpenAI’s) to create the agent class, memory, and tool-use logic from the ground up.

Building a Core Capability: RAG from Scratch#

Most agents need to access knowledge. The most common way to do this is Retrieval-Augmented Generation (RAG). Before an agent can use a RAG tool, it helps to build that tool ourselves. This involves:

• Loading documents (e.g., text files, PDFs).
• Chunking them into small pieces.
• Using an embedding model (like one from sentence-transformers) to turn those chunks into vectors.
• Storing those vectors in a simple vector database (like ChromaDB or even a basic Python-based FAISS index).
• Writing a search function that embeds a user’s query and finds the most similar chunks.

The Hugging Face blog has a fantastic, concise article titled “Code a simple RAG from scratch” that does exactly this. It shows us how to build the retrieval and generation parts using standard Python libraries, giving us a fundamental agent-ready tool.

By starting with these resources, we will build the two most important parts of any agent: the reasoning loop (ReAct) and the knowledge retrieval (RAG), all from first principles.

Mission Critical Projects - Paid Google Gemini CLI#

Google Gemini CLI is one of the best solutions for mission-critical coding projects because

• it is very fast,
• it understands your entire codebase, and
• it offers frictionless user experience

TIP

Google Gemini web app is also a great alternative for great amount of coding scenarios:

• prototyping
• coding up parts of a large project in isolation
• etc.

We can also upload the entire codebase folder in which case the Geminin web can largely perform the same task as Germini CLI does

In such cases, a great diff tool, such as Diffchecker would come handy when we want to compare local and remote code versions

To make sure we are using the most advanced model, we must enable preview features in the CLI configuration. We only need to do this once.

1. Open your terminal and run gemini
2. Type /settings and press Enter
3. Locate Preview Features and toggle it to true.
4. Restart the CLI (type /quit and run gemini again).

Once preview features are enabled, we can pick the Pro 3 model directly from the command line using the --model flag.

1
gemini --model gemini-3-pro-preview

Alternatively, if we are already inside the tool or want to set it permanently, we can use these methods:

• Interactive Switch: Inside the CLI, type /model. We will now see “Auto (Gemini 3)” or “Pro (gemini-3-pro-preview)” as options. Select one of them

• Environment Variable: To make this default without typing the flag every time, set the environment variable in our shell profile (.bashrc or .zshrc):

  1
  export GEMINI_MODEL="gemini-3-pro-preview"
  

Experimental Projects - Mixture of Gemini Free Tier and Local Agent#

The advanced model of Gemini CLI, however, is paid which is total okay; Keeping a local, cost-effective “sandbox” for experimental coding, however, prevents burning API credits on non-critical tasks. This section discusses an experimental approach to explore the free alternatives.

1
gemini chat --model gemini-2.0-flash

Continue#

Continue is a good option if we develop codes in IDE, such as JetBrain, and can be installed as an IDE plugin.

Once the plugin is installed, configure it in the following way:

1. Click the Continue icon in IDE sidebar (right).

2. Pull the following ollama models:

   1
   ollama pull starcoder2:3b
   2
   ollama pull llama3.1
   

3. Click the gear icon (⚙️) to open config.yaml and make it editable by following IDE’s prompt

4. Replace the config.yaml contents with the following

   1
   name: Local Agent Config
   2
   version: 1.0.0
   3
   
   4
   models:
   5
     # ---------------------------------------------------------
   6
     # PRIMARY AGENT (Use Llama 3.1 for stability)
   7
     # ---------------------------------------------------------
   8
     - name: Llama 3.1 (Agent)
   9
       provider: ollama
   10
       model: llama3.1:latest
   11
       roles:
   12
         - chat
   13
         - edit
   14
         - embed
   15
       # Explicitly tell Continue this model handles tools well
   16
       capabilities:
   17
         - tool_use
   18
   
   19
     # ---------------------------------------------------------
   20
     # SECONDARY (gpt-oss)
   21
     # ---------------------------------------------------------
   22
     - name: GPT-OSS
   23
       provider: ollama
   24
       model: gpt-oss:20b
   25
       roles:
   26
         - chat
   27
         - edit
   28
   
   29
     # ---------------------------------------------------------
   30
     # AUTOCOMPLETE (Starcoder2)
   31
     # ---------------------------------------------------------
   32
     - name: Starcoder2
   33
       provider: ollama
   34
       model: starcoder2:3b
   35
       roles:
   36
         - autocomplete
   37
       data:
   38
         debounceDelay: 500
   39
       requestOptions:
   40
         max_tokens: 1024
   41
   
   42
   # ---------------------------------------------------------
   43
   # CONTEXT PROVIDERS
   44
   # ---------------------------------------------------------
   45
   contextProviders:
   46
     - name: codebase
   47
       params:
   48
         nRetrieve: 25
   49
         useReranking: true
   50
     - name: docs
   51
     - name: open
   52
     - name: terminal
   53
     - name: diff
   54
   
   55
   allowAnonymousTelemetry: false
   

Now we can start using it. By default, Continue plugin does not “knows” the project structure; this is a significant difference with Gemini CLI and is good in some cases where we don’t want agent to take full charge of all works but only offers advice and we are still the person who put down final lines of code

(To be continued…)