---
title: Automatic Tool Execution
slug: automatic-tool-execution
docTags: 
createdAt: 2024-01-15T08:43:12.327Z
---

::Image[]{src="https://api.archbee.com/api/optimize/2iwq25Nmq-PWFfoTn8b_b/BdSyxEcefHMuLNkKhFzLE_chatlabexample.png" size="80" width="781" height="456" position="center" showCaption="false"}

# Introduction

Functionary provides a model that can make intelligent decisions regarding which functions/tools to use. However, it does not actually execute the function/tool. To bring this up another level, you can even automatically execute functions/tools once it is decided by Functionary! In this guide, you will learn how to do that with [chatlab](https://github.com/rgbkrk/chatlab). The code used in this tutorial is provided in [this Github repository](https://github.com/MeetKai/functionary_tutorials/tree/main/automatic_tool_execution).

## Prerequisite

- An understanding of how to run Functionary vLLM server and make API requests to the running server
- Basic skills in Python programming and interacting with APIs

## What you'll learn

- How to use chatlab
- How to configure and integrate chatlab directly with Functionary
- How to get a model response grounded in function outputs end-to-end with Functionary and chatlab

## What you'll need

- A machine capable of running inference with a local [Functionary-v1.4 model](https://huggingface.co/meetkai/functionary-7b-v1.4) (at least 24GB GPU VRAM)
- A machine with Functionary's dependencies installed

# Setup and Requirements

## Functionary

Start a Functionary vLLM server with functionary-v1.4 model

```python
python3 server_vllm.py --model meetkai/functionary-7b-v1.4 --max-model-len 4096
```

The functionary-v1.4 model is trained on context-window of 4K so pass in --max-model-len of 4096.

## Chatlab

Install the chatlab python binary package. In this tutorial, we will use version 1.3.0.

```shell
pip3 install chatlab==1.3.0
```

## Note on Requirements

Please note that Chatlab's Chat class currently doesn't support Parallel Function calling. Thus, this tutorial is compatible with Functionary Version 1.4 only and may not work correctly with Functionary Version 2.\* models.

## Define Python Function

Let's assume that you are one of the car dealers at Functionary car dealership. You would like to create a chatbot that can assist you or your customers in quickly getting the prices of certain car models available in the dealership. You will create this Python function:

```python
def get_car_price(car_name: str):
    """this function is used to get the price of the car given the name
    :param car_name: name of the car to get the price
    """
    car_price = {
        "rhino": {"price": "$20000"},
        "elephant": {"price": "$25000"} 
    }
    for key in car_price:
        if key in car_name.lower():
            return {"price": car_price[key]}
    return {"price": "unknown"}
```

This function queries a dictionary mapping car model names to its respective prices. It returns an "unknown" value if the car model name is not found.

# Before you begin

Before you begin, let's imagine packages like chatlab are not around.

## Manually create LLM Function

Now, a customer approaches your car dealer chatbot asking about the price of the car model "Rhino". Firstly, you would need to manually convert the Python function into the tool dictionary required in OpenAI API:

```python
functions = [
    {
        "name": "get_car_price",
        "description": "this function is used to get the price of the car given the name",
        "parameters": {
            "type": "object",
            "properties": {
                "car_name": {
                    "type": "string",
                    "description": "name of the car to get the price"
                }
            },
            "required": ["car_name"]
        }
    }
]
```

Thereafter, you would perform inference on Functionary.

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="functionary")

messages = [
    {
        "role": "user",
        "content": "What is the price of the car named 'Rhino'?"
    }
]

assistant_msg = client.chat.completions.create(
    model="meetkai/functionary-7b-v1.4",
    messages=messages,
    functions=functions,
    temperature=0.0,
).choices[0].message
```

This yields the following response:

```python
ChatCompletionMessage(
    content=None,
    role='assistant',
    function_call=FunctionCall(
        arguments='{"car_name": "Rhino"}',
        name='get_car_price'
    ),
    tool_calls=None,
    tool_call_id=None,
    name=None
)
```

## Manually execute function

As we can see above, Functionary makes the correct decision to call the \`get\_car\_price\` function with \`Rhino\` as input. However, we need to manually execute this function and append its output to the conversation for Functionary to generate appropriate model responses back to the customer.

```python
import json

def execute_function_call(message):
    if message.function_call.name == "get_car_price":
        car_name = json.loads(message.function_call.arguments)["car_name"]
        results = get_car_price(car_name)
    else:
        results = f"Error: function {message.function_call.name} does not exist"
    return results

if assistant_msg.function_call is not None:
    results = execute_function_call(assistant_msg)
    messages.append({"role": "assistant", "name": assistant_msg.function_call.name, "content": assistant_msg.function_call.arguments})
    messages.append({"role": "function", "name": assistant_msg.function_call.name, "content": str(results)})
    
    output_msg = client.chat.completions.create(
        model="meetkai/functionary-7b-v1.4",
        messages=messages,
        functions=functions,
        temperature=0.0,
    ).choices[0].message
    
    print(output_msg)
```

This yields the final response:

```python
ChatCompletionMessage(
    content="The price of the car named 'Rhino' is $20000.",
    role='assistant',
    function_call=None,
    tool_calls=None,
    tool_call_id=None,
    name=None,
)
```

This simple example shows that Functionary can:

- Intelligently decide on the correct function to use given the conversation
- Analyze the function output and generate response grounded in the output

However, as you can see, this requires manually creating the function configuration and executing the functions called until a model response is generated. This is where **automatic tool execution** will be helpful.

# Call real python functions automatically

Now, we show that the Functionary can be further enhanced with automatic execution of Python functions. To call the real Python function, get the result and extract the result to respond, you can use [chatlab](https://github.com/rgbkrk/chatlab). The following example uses chatlab==1.3.0:

```python
import chatlab
import asyncio

chat = chatlab.Chat(model="meetkai/functionary-7b-v1.4", base_url="http://localhost:8000/v1", api_key="functionary")
chat.register(get_car_price)
asyncio.run(chat.submit("What is the price of the car named 'Rhino'?", stream=False))

for message in chat.messages:
    role = message["role"].upper()
    if "function_call" in message:
        func_name = message["function_call"]["name"]
        func_param = message["function_call"]["arguments"]
        print(f"{role}: call function: {func_name}, arguments:{func_param}")
    else:
        content = message["content"]
        print(f"{role}: {content}")
```

The output will look like this:

```text
USER: What is the price of the car named 'Rhino'?
ASSISTANT: call function: get_car_price, arguments:{
    "car_name": "Rhino"
}
FUNCTION: {'price': {'price': '$20000'}}
ASSISTANT: The price of the car named 'Rhino' is $20000.
```

Now, Functionary will be called iteratively and chatlab will automatically execute any function called by Functionary until no more functions are to be called and a model response is generated for the customer. This is all done with the ease of a single command:

```python
chat.submit("What is the price of the car named 'Rhino'?")
```

# Congratulations

Congratulations on completing this tutorial. We hope you have learnt about how to further harness the power of Functionary by combining with automatic tool execution libraries like chatlab. Feel free to try out the example notebooks in the [Github repository](https://github.com/MeetKai/functionary_tutorials/tree/main/automatic_tool_execution) and explore Functionary's function calling capabilities with your own functions.

## Summary

- Performing inference using Functionary
- Experiencing how Functionary calls functions and generates model responses
- Learning how to execute functions called by Functionary automatically with chatlab