Function Calling & Agent Memory

📌 Deep Dive: How Function Calling Works

We know that agents use "tools" to interact with the world, but how does the code actually look? The magic behind tool use is a feature native to modern LLMs (like GPT-4 and Gemini) called Function Calling.

An LLM does not execute code. It just generates text. To make it use a tool, you follow a three-step dance:

• Step 1 (The Schema): When you send a prompt to the LLM, you also send a JSON dictionary describing your available tools (e.g., {"name": "getWeather", "parameters": ["city"]}).
• Step 2 (The Generation): If the LLM decides it needs the weather, it stops generating regular text. Instead, it generates a perfectly formatted JSON string: {"tool": "getWeather", "arguments": {"city": "Paris"}}.
• Step 3 (The Execution): Your application code catches this JSON, actually runs the getWeather Python or Node.js function, and appends the result (e.g., "25°C") to the chat history for the LLM to read.

📌 Agent Memory: Short-Term vs Long-Term

If an agent is going to perform a complex task over several minutes (or days), it needs to remember what it has already done. We split agent memory into two categories:

• Short-Term Memory (The Context Window): This is the immediate chat history. Every time the agent takes a step, you append the action and the result to the array of messages and send the whole array back to the LLM. Because context windows are limited (and get expensive), developers often use a "sliding window" to only keep the last 10 messages.
• Long-Term Memory (Vector Databases): If an agent learns a user's preference ("User prefers Python over JavaScript") or finishes a massive sub-task, it can save that fact as an embedding into a Vector Database. Days later, when the agent wakes up for a new task, it can search the database to instantly retrieve those past memories using RAG (Retrieval-Augmented Generation).