GPT Function Call

A constraint of large language models (LLMs) is that their knowledge is confined to the data they were trained on, and their functionality is limited to predicting the next token in a sequence. To address this, various initiatives have aimed to equip LLMs with tool-use capabilities, such as this paper from Meta AI Toolformer: Language Models Can Teach Themselves to Use Tools.

OpenAI initially introduced their approach to tool use through ChatGPT Plugins, which debuted on March 23, 2023. Later, on June 13, 2023, the release of GPT-3.5 and GPT-4 included Plugin functionalities via a new feature called Function Call. These models have been fine-tuned to both detect when a function needs to be called and to respond with JSON that adheres to the function signature. Function calling allows developers to more reliably get structured data back from the model. Common use cases are as follows:

  • Create chatbots that answer questions by calling external tools (e.g., like ChatGPT Plugins)
  • Convert natural language into API calls or database queries
  • Extract structured data from text

In this discussion, we’ll explore examples of function calls from OpenAI’s official documentation and create chatbots that answer questions by calling external tools.

import json

import openai
from dotenv import load_dotenv


load_dotenv()


%load_ext autoreload
%autoreload 2

Create chatbots that answer questions by calling external tools

As previously mentioned, GPT lacks access to external data and is limited in tasks that would be better handled in a non-autoregressive manner. For instance, GPT can’t answer a question like “What’s the weather like in Boston?” because it doesn’t have access to current weather data. Therefore, if posed with such a question, it will clarify that it can’t provide real-time information.

completion = openai.ChatCompletion.create(
    model="gpt-4-0613",
    messages=[{"role": "user", "content": "What's the weather like in Boston?"}],
)
reply_content = completion.choices[0].message.content
print(reply_content)

Sorry, as an AI, I don’t have real-time capabilities to provide current weather updates. Please check a reliable weather forecast service for this information.

Now, let’s assume we have a function named get_current_weather, which could serve as a wrapper for a weather API or perform searches in a weather database.

Using Function Call, the process unfolds as follows:

  1. Define the functions with names, descriptions, and parameters.
  2. Provide the function to GPT. GPT will determine when it is appropriate to utilize the function.
  3. If GPT chooses to use the function, it will generate the necessary parameters for the function call.
  4. Execute the function.
  5. Attach the function’s output to the message chain and prompt GPT to generate a response incorporating this output.

Define get_current_weather mock

def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    weather_info = {
        "location": location,
        "temperature": "72",
        "unit": unit,
        "forecast": ["sunny", "windy"],
    }
    return json.dumps(weather_info)

Step 1. Define function names, descriptions, and parameters

functions = [
    {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA",
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"]
                },
            },
            "required": ["location"],
        },
    }
]
available_functions = {
    "get_current_weather": get_current_weather,
}  # only one function in this example, but you can have multiple

Step 2. Provide the function to GPT. GPT will determine when it is appropriate to utilize the function.

Once we’ve defined functions and available_functions, we can pose the question “What’s the weather like in Boston?” to GPT. GPT will assess whether it’s appropriate to use the functions we’ve provided. In this case, it deems it suitable and returns a “function_call” field in its response.

Note that function_call itself is a JSON object with "name" denoting the name of the function and "arguments" denoting the arguments we should be using to call the function.

def run_conversation_initial(messages, functions=functions) -> str:
    response = openai.ChatCompletion.create(
        model="gpt-4-0613",
        messages=messages,
        functions=functions,
        function_call="auto",  # auto is default, but we'll be explicit
        temperature=0.0,
    )
    return response["choices"][0]["message"]

If the question isn’t relevant to the function, GPT will answer based on its own knowledge. We can examine the "function_call" field to determine if GPT did not utilize functions and handle the branching accordingly. Specifically, non-function call returns will NOT have the "function_call" field.

response = run_conversation_initial(messages=[{"role": "user", "content": "What's the mayor of Boston?"}])
if not response.get("function_call"):
    print("No Function Call requested!")

print(response)

No Function Call requested!
{
  "role": "assistant",
  "content": "The current mayor of Boston is Michelle Wu."
}

Step 3. If GPT chooses to use the function, it will return 'function_call' field and the necessary parameters.

response = run_conversation_initial(
    messages=[{"role": "user", "content": "What's the weather like in Boston?"}],
    functions=functions,
)
if response.get("function_call"):
    print("GPT requests us to call a function!!\n")
    print(f"{response}\n\n")
    print(f"Name of the function to call:\n{response['function_call']['name']}\n")
    print(f"Arguments for calling the function:\n{response['function_call']['arguments']}")

GPT requests us to call a function!!
{
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "get_current_weather",
    "arguments": "{\n  \"location\": \"Boston\"\n}"
  }
}

Name of the function to call:
get_current_weather

Arguments for calling the function:
{
  "location": "Boston"
}

Step 4. Call the function

Calling the function is relatively straightforward; we retrieve the function from the available_functions dictionary and execute it using the arguments provided by GPT. We can do it using the following function.

Note that the function call will take "role" of "function" with the following schema. As a reminder, users take the "role" of "user" and the LLM takes the "role" of "assistant".

{
    "role": "function",
    "name": function_name,
    "content": function_return_json,
}


def function_call(*, response_message: dict, available_functions: dict):
    function_name = response_message["function_call"]["name"]
    fuction_to_call = available_functions[function_name]
    function_args = json.loads(response_message["function_call"]["arguments"])
    function_response = fuction_to_call(**function_args)

    return {
        "role": "function",
        "name": function_name,
        "content": str(function_response),
    }

function_response = function_call(response_message=response, available_functions=available_functions)
print(function_response)

{‘role’: ‘function’, ‘name’: ‘get_current_weather’, ‘content’: ‘{“location”: “Boston”, “temperature”: “72”, “unit”: “fahrenheit”, “forecast”: [“sunny”, “windy”]}’}

Step 5. Attach the function’s output to the message chain and prompt GPT to generate a response incorporating this output.

Having fulfilled GPT’s request, we now update the chat conversation to include both the function call request and the function call results.

As a reminder, our conversation history is as follows.

Initially we ask a question as "user":

{"role": "user", "content": "What's the weather like in Boston?"}

Then GPT the "assistant" requested for a "Function Call".

{
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "get_current_weather",
    "arguments": "{\n  \"location\": \"Boston, MA\"\n}"
  }
}

Then the "Function Call" returned a response with the role "function".

{
    "role": "function",
    "name": "get_current_weather",
    "content": '{"location": "Boston, MA", "temperature": "72", "unit": "fahrenheit", "forecast": ["sunny", "windy"]'},
}

So, now we can send the whole conversation history to GPT and ask it to generate a response.

conversation: list[dict] = [
    {"role": "user", "content": "What's the weather like in Boston?"},
    {
        "role": "assistant",
        "content": None,
        "function_call": {
            "name": "get_current_weather",
            "arguments": "{\n  \"location\": \"Boston, MA\"\n}"
        }
    },
    {
        "role": "function",
        "name": "get_current_weather",
        "content": '{"location": "Boston, MA", "temperature": "72", "unit": "fahrenheit", "forecast": ["sunny", "windy"]}',
    }
]

second_response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-0613",
    messages=conversation,
    temperature=0.0,
)

print(second_response.choices[0].message.content)

The weather in Boston is currently sunny and windy with a temperature of 72 degrees Fahrenheit.

Putting it all together

Now with all the components ready, we can chain everything together using the function defined below. This conversation chain will attempt to answer questions from the users and, if applicable, utilizes the available_functions and the functions definition from the earler part of the notebook.

def run_conversation_chain(*, user_message: str, functions: list[dict], available_functions: dict) -> str:
    # Step 1: send the conversation and available functions to GPT
    messages = [{"role": "user", "content": user_message}]
    response = openai.ChatCompletion.create(
        model="gpt-4-0613",
        messages=messages,
        functions=functions,
        function_call="auto",  # auto is default, but we'll be explicit
        temperature=0.0,
    )
    response_message = response["choices"][0]["message"]

    # Step 2: check if GPT wanted to call a function, if not, return the response
    if not response_message.get("function_call"):
        return response_message['content']
    
    # Step 3: call the function
    # Note: the JSON response may not always be valid; be sure to handle errors
    function_message = function_call(
        response_message=response_message,
        available_functions=available_functions
    )

    # Step 4: send the info on the function call and function response to GPT
    messages.append(response_message)  # extend conversation with assistant's reply
    messages.append(function_message)  # extend conversation with function response

    second_response = openai.ChatCompletion.create(
        model="gpt-4-0613",
        messages=messages,
        temperature=0.0,
    )  # get a new response from GPT where it can see the function response
    return second_response.choices[0].message.content

response = run_conversation_chain(
    user_message="What is the weather like in Boston?",
    functions=functions,
    available_functions=available_functions,
)
print(response)

The current weather in Boston is sunny and windy with a temperature of 72 degrees Fahrenheit.

response = run_conversation_chain(
    user_message="Who is the mayor of Boston?",
    functions=functions,
    available_functions=available_functions,
)
print(response)

The current mayor of Boston is Michelle Wu.

Conclusion

The Function Call feature fundamentally enhances the utility of Large Language Models like GPT-3.5 and GPT-4. By enabling real-time data retrieval and action execution, it paves the way for more dynamic and context-aware applications. This guide has provided a step-by-step walkthrough to integrate this feature, effectively bridging the gap between static text generation and actionable outputs. Future studies will delve deeper into its potential, exploring API and SQL generation as well as structured data extraction capabilities.