Hotel Booking Chatbot using real APIs

introduction in this tutorial, you will learn how to create your own chatbot using functionary's function calling feature with some real world apis equipping functionary to call real world apis helps you connect functionary to external systems that provide the most up to date and accurate information through this tutorial, you will see how easy it is to set up functionary to call real world apis and how functionary is able to plan between different api endpoints towards answering users' queries all the code provided here can also be found in this github repository which contains information on how to get the chatbot running in your local machines what is an api? an application programming interface (api) is a kind of mechanism that enables communication between two software systems data is sent via requests and responses in a typical setting, the client application sends the request to the server and the server will send the corresponding response containing the data that fulfills the request back to the client for example, the road traffic app in your phone is able to send an api request requesting for the traffic at certain areas to the national transport ministry's road traffic monitoring system the road traffic monitoring system will then send the relevant response back to the mobile app to display to the user apis as effective tools for function calling the request response nature of apis works similarly to how functions work (receiving inputs and providing outputs) furthermore, the above example illustrates that apis allow for retrieving the most up to date and accurate information thus, apis serve perfectly well as effective tools to be used by functionary in real world applications in this tutorial, we will demonstrate this ability with a simple hotel booking chatbot what you'll build in this tutorial, you'll build a simple hotel booking chatbot application in python with functionary and real apis from hotel com provider in rapidapi your chatbot will be able to provide information about the hotels that users are interested in, and help users search for hotels for their upcoming holidays prerequisite an understanding of how to run functionary vllm server and make api requests to the running server basic skills in python programming and interacting with apis what you'll learn how to subscribe to and use apis from rapidapi how to register api endpoints as tools for functionary how to call functionary and get function call responses how to return the function response to functionary and respond to the user how to create a simple chatbot in command line interface that iteratively converses with the user and makes function calls what you'll need a machine capable of running inference with a local functionary small v2 2 or older model (at least 24gb gpu vram) a machine with functionary's dependencies installed setup and requirements functionary start a functionary vllm server with any functionary model python3 server vllm py model meetkai/functionary small v2 2 max model len 8192 rapidapi before you can start using any public apis as tools for functionary, you need to sign up for a rapidapi account and subscribe to the apis that you need sign up for rapidapi account click on the "sign up" button on the top right hand corner of the rapidapi homepage as shown above you will be brought to the signing up page where you can choose either to sign up with google, github, facebook or manually subscribe to hotels com provider apis after logging in, proceed to the rapidapi hub page thereafter, search for the hotels com provider apis as shown below click on the " hotels com provider" option and it will bring you to the api endpoints page in this page, click on "pricing" at the top of page over at the pricing page, you are allowed to choose different plans which over different request and rate limits since hotels com provider is a freemium type of api, we will go ahead and choose the free basic plan it has a request and rate limit of 600/month and 5/second respectively, enough for the purpose of this tutorial using the apis the hotels com provider apis provide several endpoints for this tutorial, we will just use the following endpoints hotels search > searches hotels by region id and preferences hotel details > gets the details of a particular hotel regions search > search for region/location/city/hotel by name and returns the respective id (needed by both hotels search and hotel details) these endpoints provide the necessary tools for a simple hotel booking chatbot where there are search and information gathering capabilities regarding hotels converting apis to llm tools the functionary openai compatible vllm server works with tools defined in openai's format thus, we will need to provide the api endpoints in the same tool definition format for functionary to access each tool and its parameters contains the name, description, and various requirements indicated in json schema format fortunately, rapidapi provides the details for each api endpoint in the platform api for example, lets look at the hotel details endpoint on platform api page the endpoint description can be converted to tool description while each parameter contains all the necessary information to be converted to the json schema format compatible with openai tools after copying the information over, here's the list of tools \[ { "type" "function", "function" { "name" "hotel details", "description" "get hotel details", "parameters" { "type" "object", "properties" { "domain" { "type" "string", "description" "", "default" "us" }, "hotel id" { "type" "number", "description" "hotel id this can be found from hotels search " }, "locale" { "type" "string", "description" "", "default" "en us" } }, "required" \["hotel id"] } } }, { "type" "function", "function" { "name" "regions search", "description" "search region, locations, city or hotel by name returns the region and hotel id needed for other tools ", "parameters" { "type" "object", "properties" { "query" { "type" "string", "description" "query live search" }, "domain" { "type" "string", "description" "", "default" "us" }, "locale" { "type" "string", "description" "", "default" "en us" } }, "required" \["query"] } } }, { "type" "function", "function" { "name" "hotels search", "description" "get hotels by the filter (meta / domains list) indicate the `region id` > use `region search` first, check in and check out date, number of adults and children ", "parameters" { "type" "object", "properties" { "region id" { "type" "number", "description" "region id > use `regions search`" }, "locale" { "type" "string", "description" "", "default" "en us" }, "checkin date" { "type" "string", "description" "checkin date, yyyy mm dd" }, "sort order" { "type" "string", "description" "", "enum" \["review", "recommended", "distance", "price low to high", "property class", "price relevant"] }, "adults number" { "type" "number", "description" "number of adults" }, "domain" { "type" "string", "description" "", "default" "us" }, "checkout date" { "type" "string", "description" "checkout date, yyyy mm dd" } }, "required" \["region id", "checkin date", "sort order", "adults number", "checkout date"] } } } ] note that the "locale" and "domain" parameters have been set to "en us" and "us" respectively by default while the optional parameters are left out for convenience since the regions search tool is required by the other two tools (region id and hotel id respectively), the tool description was modified to reflect this too this json schema definition can also be found in the hotels api specification json file contained in the github repository of this tutorial using the api converted tools to use the api converted tools that we have just created, we can simply load the json file containing the tools as a python list variable and provide the tools for model inference with open("hotels api specification json", "r") as file tools = json load(file) def infer model(messages list\[dict], model str) \# infer the model output = ( llm client chat completions create( model=model, messages=messages, tools=tools, temperature=0 0 ) choices\[0] message ) return output creating your own chatbot now that you have learnt how to subscribe to the hotel planning api endpoints by hotels com provider and how functionary call the apis via generating tool call, we can proceed to the actual task of creating your own hotel planning chatbot we will first begin by introducing the overall pipeline and important points to note, followed by going through each step in the pipeline the code for creating the chatbot can be found in this github repository overall pipeline the main variable in the chatbot application is messages messages is a list variable that tracks the current conversation each turn in messages is a dictionary following the same specification created by openai the following flowchart shows the pipeline design of the hotel planning chatbot we will go in depth into the various stages in the pipeline in the following subtopics begin conversation & user input we first begin the conversation whenever we reset the conversation, messages will be re initialized to contain the introductory messages only the introductory messages consists of a greeting by the user and the intro msg by the assistant this is added to prompt engineer functionary to be able to identify itself as a hotel planning chatbot def reset messages(intro msg str) \# clear the previous conversation from the terminal os system("cls" if os name == "nt" else "clear") \# print the introductory messages print("===================") print("welcome to the hotel planning chatbot program powered by functionary ") print('press ctrl+c anytime to stop enter "n" to start a new chat ') print("===================") return \[ {"role" "user", "content" "hello!"}, {"role" "assistant", "content" intro msg}, ] model inference once the user input is provided, it will be packaged into a user message with the "content" field being the user query messages will then be passed to functionary for inference we will use the openai api to send requests to the functionary vllm server from openai import openai llm client = openai(base url="http //localhost 8000/v1", api key="functionary") def infer model(messages list\[dict], model str) \# infer the model output = ( llm client chat completions create( model=model, messages=messages, tools=tools, temperature=0 0 ) choices\[0] message ) \# postprocess model output if len(output tool calls) > 0 fn calls = \[] for tool call in output tool calls tool call = tool call function model dump() tool call\["arguments"] = json loads(tool call\["arguments"]) fn calls append(tool call) return fn calls if output function call is not none fn calls = \[output function call] return fn calls return output content note that after we receive the output message response, we will perform some postprocessing specifically, if the response contains tool calls, the infer model function will output a list of dictionaries containing the tool name and arguments otherwise, the function will return the model response in string format if the response is a model response execute tools if functionary generates a decision to call tools, we will need to execute them by making api requests to the respective hotels com provider rapidapi api endpoints an overview of the process of calling tools and receiving their responses is illustrated in the figure below when functionary decides to make a tool call, it will generate the name of the tool and its arguments the arguments are also the parameters/inputs for the corresponding api endpoint with all the information provided, an api request can then be sent to the api endpoint once the api endpoint finishes processing the request, it will send the response back to our program we will then include the api responses into messages as part of the conversation the following code snippets shows how to call the hotels com provider apis def call hotels com provider api(api name str, input data dict, rapidapi key str) urls = { "hotel details" "https //hotels com provider p rapidapi com/v2/hotels/details", "hotel info" "https //hotels com provider p rapidapi com/v2/hotels/info", "regions search" "https //hotels com provider p rapidapi com/v2/regions", "hotels search" "https //hotels com provider p rapidapi com/v2/hotels/search", } \# raise error if the api name is not in urls assert api name in urls, f"`api name` is not a valid api name " \# add "domain" and "locale" into input data if they are not present if "domain" not in input data input data\["domain"] = "us" if "locale" not in input data input data\["locale"] = "en us" \# send the request and get the response url = urls\[api name] headers = { "x rapidapi key" rapidapi key, "x rapidapi host" "hotels com provider p rapidapi com", } response = requests get(url, headers=headers, params=input data) if response status code != 200 errors json = json loads(response text)\["detail"] errors text = \[] for error in errors json errors text append(f"{error\['loc']\[ 1]} {error\['msg']}") return {"message" " | " join(errors text)} results = response json() if api name == "hotels search" results = results\["properties"] final res = \[] for result in results \# only append those that are available if result\["availability"]\["available"] is true final res append( { "id" result\["id"], "name" result\["name"], "price per night" f"{result\['price']\['lead']\['currencyinfo']\['code']} {result\['price']\['lead']\['currencyinfo']\['symbol']}{result\['price']\['lead']\['formatted']}", "review score" result\["reviews"]\["score"], "hotel id" result\["id"], } ) \# just take top 5 if len(final res) == 5 break return final res elif api name == "regions search" if results\["data"]\[0]\["@type"] == "gaiaregionresult" return {"region id" results\["data"]\[0]\["gaiaid"]} else return { "hotel id" results\["data"]\[0]\["hotelid"], "city id" results\["data"]\[0]\["cityid"], } else return { "hotel info" results\["propertycontentsectiongroups"]\["aboutthisproperty"]\[ "sections" ]\[0]\["bodysubsections"]\[0]\["elements"]\[0]\["items"]\[0]\["content"]\["text"] } first, we need to check if the tool name provided by functionary is a valid api name this is just an additional check since with grammar sampling enabled, functionary will always generate a valid tool name thereafter, we will make the request to the endpoint via python's requests package rapidapi's platform api page has example codes for various languages, including using python requests package note that a rapidapi api key is needed for each request you can find it in the code snippets section once we have the response, we will do some slight postprocessing before returning the responses to functionary if the api is regions search, we will return either "region id" or "hotel id" and "city id" depending on whether the search query is a region or a hotel if the api is hotels search, we will return top 5 search results with each entry containing "name", "price per night", "review score", "hotel id", etc if the api is hotel details, we will return the information of the hotel in "hotel info" congratulations congratulations on completing this tutorial and successfully creating your first hotel planning chatbot using functionary's function calling/tool use capability! we hope you have learnt about how functionary's function calling feature can be used with real world apis as tools users can ask about hotel recommendations, hotel information, etc and the system will intelligently fetch the latest data from an external api and respond with an answer functionary's function calling feature is intended to help you with setting up your own chatbot applications it can call functions intelligently, extract parameters as deterministically as possible, and performs analysis and content creation you just need to provide it with your desired apis and it will handle the rest towards creating useful and helpful responses to users feel free to try out other apis in your own pipeline and explore the capabilities of functionary