Tools (Function Calling)
Some LLMs, in addition to generating text, can also trigger actions.
There is a concept known as “tools,” or “function calling”.
It allows the LLM to call, when necessary, one or more available tools, usually defined by the developer.
A tool can be anything: a web search, a call to an external API, or the execution of a specific piece of code, etc.
LLMs cannot actually call the tool themselves; instead, they express the intent
to call a specific tool in their response (instead of responding in plain text).
We, as developers, should then execute this tool with the provided arguments and report back
the results of the tool execution.
For example, we know that LLMs themselves are not very good at math.
If your use case involves occasional math calculations, you might want to provide the LLM with a “math tool.”
By declaring one or multiple tools in the request to the LLM,
it can then decide to call one of them if it deems it appropriate.
Given a math question along with a set of math tools, the LLM might decide that to properly answer the question,
it should first call one of the provided math tools.
Let’s see how this works in practice (with and without tools):
An example of a message exchange without tools:
Request:
- messages:- UserMessage:- text: What is the square root of 475695037565?Response:
- AiMessage:- text: The square root of 475695037565 is approximately 689710.
Close, but not correct.
An example of a message exchange with the following tools:
@Tool("Sums 2 given numbers")
public double sum(double a, double b) {return a + b;
}@Tool("Returns a square root of a given number")
public double squareRoot(double x) {return Math.sqrt(x);
}
Request 1:
- messages:- UserMessage:- text: What is the square root of 475695037565?
- tools:- sum(double a, double b): Sums 2 given numbers- squareRoot(double x): Returns a square root of a given numberResponse 1:
- AiMessage:- toolExecutionRequests:- squareRoot(475695037565)... here we are executing the squareRoot method with the "475695037565" argument and getting "689706.486532" as a result ...Request 2:
- messages:- UserMessage:- text: What is the square root of 475695037565?- AiMessage:- toolExecutionRequests:- squareRoot(475695037565)- ToolExecutionResultMessage:- text: 689706.486532Response 2:
- AiMessage:- text: The square root of 475695037565 is 689706.486532.
As you can see, when an LLM has access to tools, it can decide to call one of them when appropriate.
This is a very powerful feature.
In this simple example, we gave the LLM primitive math tools,
but imagine if we gave it, for example, googleSearch and sendEmail tools
and a query like “My friend wants to know recent news in the AI field. Send the short summary to friend@email.com,”
then it could find recent news using the googleSearch tool,
then summarize it and send the summary via email using the sendEmail tool.
:::note
To increase the chances of the LLM calling the right tool with the right arguments,
we should provide a clear and unambiguous:
- name of the tool
- description of what the tool does and when it should be used
- description of every tool parameter
A good rule of thumb: if a human can understand the purpose of a tool and how to use it,
chances are that the LLM can too.
:::
LLMs are specifically fine-tuned to detect when to call tools and how to call them.
Some models can even call multiple tools at once, for examp