Functionary

Dataset	Model Name	Function Calling Accuracy (Name & Arguments)
In-house data	MeetKai-functionary-small-v2.2	0.546
In-house data	MeetKai-functionary-medium-v2.2	0.664
In-house data	OpenAI-gpt-3.5-turbo-1106	0.531
In-house data	OpenAI-gpt-4-1106-preview	0.737

Shell

pip install -r requirements.txt

Shell

python3 server_vllm.py --model "meetkai/functionary-small-v2.2" --host 0.0.0.0

Python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="functionary")

client.chat.completions.create(
    model="meetkai/functionary-small-v2.2",
    messages=[{"role": "user",
            "content": "What is the weather for Istanbul?"}
    ],
    tools=[{
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        }
                    },
                    "required": ["location"]
                }
            }
        }],
    tool_choice="auto"
)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="functionary")

client.chat.completions.create(
    model="meetkai/functionary-small-v2.2",
    messages=[{"role": "user",
            "content": "What is the weather for Istanbul?"}
    ],
    tools=[{
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        }
                    },
                    "required": ["location"]
                }
            }
        }],
    tool_choice="auto"
)

Python

import requests

data = {
    'model': 'meetkai/functionary-small-v2.2', # model name here is the value of argument "--model" in deploying: server_vllm.py or server.py
    'messages': [
        {
            "role": "user",
            "content": "What is the weather for Istanbul?"
        }
    ],
    'tools':[ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ]
}

response = requests.post("http://127.0.0.1:8000/v1/chat/completions", json=data, headers={
    "Content-Type": "application/json",
    "Authorization": "Bearer xxxx"
})

# Print the response text
print(response.text)

import requests

data = {
    'model': 'meetkai/functionary-small-v2.2', # model name here is the value of argument "--model" in deploying: server_vllm.py or server.py
    'messages': [
        {
            "role": "user",
            "content": "What is the weather for Istanbul?"
        }
    ],
    'tools':[ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ]
}

response = requests.post("http://127.0.0.1:8000/v1/chat/completions", json=data, headers={
    "Content-Type": "application/json",
    "Authorization": "Bearer xxxx"
})

# Print the response text
print(response.text)

Shell

sudo docker run --gpus all -it --shm-size=8g --name functionary -v ${PWD}/functionary_workspace:/workspace -p 8000:8000 nvcr.io/nvidia/pytorch:22.12-py3

Feature/Project	Functionary	NexusRaven	Gorilla	Glaive	GPT-4-1106-preview
Single Function Call
Parallel Function Calls
Nested Function Calls
Following up on Missing Function Arguments
Multi-turn
Generate Model Responses Grounded in Tools Execution Results
Chit-chat

Model	Description	Compute Requirements (for FP16 HF model weights)
functionary-small-v2.2 / GGUF	8k context	Any GPU with 24GB VRAM
functionary-medium-v2.2 / GGUF	8k context, better accuracy	2 x A100-80GB or equivalent
functionary-7b-v2.1 / GGUF	8k context	Any GPU with 24GB VRAM
functionary-7b-v2 / GGUF	Parallel function call support.	Any GPU with 24GB VRAM
functionary-7b-v1.4 / GGUF	4k context, better accuracy (deprecated)	Any GPU with 24GB VRAM
functionary-7b-v1.1	4k context (deprecated)	Any GPU with 24GB VRAM
functionary-7b-v0.1	2k context (deprecated) Not recommended, use 2.1 onwards	Any GPU with 24GB VRAM

Key Features

Model Features

Server/Inference Features

Us vs the Industry

Performance

Function Prediction Evaluation

Get Started

Installation

Running the server

Server Options

Running the client

OpenAI Compatible Usage

Raw Usage

Models Available

Compatibility information

How it works?

License