Text generation

Assistive's Chat APIs allow you to interact with highly capable large language models from OpenAI such as GPT-4 and GPT-3.5. They're capable of generating human-like text, writing code, using custom tools, and more. And, on the Assistive API they come with Code Interpreter and Browsing tools built-in.

Chat Messages API

The Chat Messages API generates a response given an array of messages. It has built-in Code Interpreter and Browsing tools which are called on the server-side, requiring no additional action from the application.

This API differs slightly to OpenAI's Chat Completions API due to its ability to respond with multiple messages (e.g. tool calls and the assistant's response)

Example non-streaming response

{
  "messages": [
    {
      "role": "assistant",
      "content": "Hello there, how may I assist you today?",
    }
  ]
}

Example streaming response

data: {"role":"assistant","content":""}

data: {"content":"Hello"}

....

data: {}

data: [DONE]

data: [END]

Chat example

Here's a simple example of using the Chat Messages API to build a simple chat application in NodeJS.

First, define an array of messages. This might start empty, or you can include a system prompt to give the model instructions.

let messages = [];

Now, we'll define a function that accepts an input from the user, appends it to the messages array, and receives a response from the Chat Messages API.

async function chat(input) {
    messages.push({ role: "user", content: input });

    const response = await fetch("https://api.assistive.chat/v1/chat/messages", {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
            "Authorization": "Bearer " + ASSISTIVE_API_KEY
        },
        body: JSON.stringify({ 
            model: "gpt-4-turbo-tools",
            messages: messages
        }),
    });

    const json = await response.json();

    // Append all messages received to the message history
    messages.push(...json.messages);
}

Now, this function can be called to converse with the model. Each time it's called, the user's message and the assistant's response are added to the messages array, and they'll be included as context in future calls.

Chat example with browsing tool

Building on the previous example, you can easily build a simple chat application that can browse the web using the built-in Browsing tool.

All you have to do is include browsing in the tools parameter in the API request.

async function chat(input) {
    messages.push({ role: "user", content: input });

    const response = await fetch("https://api.assistive.chat/v1/chat/messages", {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
            "Authorization": "Bearer " + ASSISTIVE_API_KEY
        },
        body: JSON.stringify({ 
            model: "gpt-4-turbo-tools",
            messages: messages,
            tools: ["browsing"]
        }),
    });

    const json = await response.json();

    // Append all messages received to the message history
    messages.push(...json.messages);
}

Now, you can try using chat() to ask the model to browse the web for you. This is what the messages array will look like after using the function:

[
    { 
        role: "user", 
        content: "Search the web for 'hello world'"
    },
    { 
        role: "assistant", 
        content: null,
        tool_calls: [
            {
                id: "call_NhGNTLCkZm46pKmTzirhrneT",
                type: "function",
                function: {
                    name: "browsing",
                    arguments: "{\"query\":\"hello world\"}"
                }
            }
        ]
    },
    {
        role: "tool",
        tool_call_id: "call_NhGNTLCkZm46pKmTzirhrneT",
        content: "[RESPONSE FROM BROWSING TOOL]"
    },
    {
        role: "assistant",
        content: "[ASSISTANT RESPONSE]"
    }
];

You can then display the model's response to the user.

Chat example with streaming

Streaming is slightly more complex to implement due to the need for handling multiple messages potentially being received in the stream. Streaming responses returns partial message deltas. The end of a message is delimited by [DONE], and the end of the stream is delimited by [END].

Here's the chat() function, modified to support streaming:

async function chat(input) {
    messages.push({ role: "user", content: input });

    const response = await fetch("https://api.assistive.chat/v1/chat/messages", {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
            "Authorization": "Bearer " + ASSISTIVE_API_KEY
        },
        body: JSON.stringify({
            model: "gpt-4-turbo-tools",
            messages: messages,
            stream: true
        }),
    });

    const completion = response.body;
    const reader = completion.getReader();

    // This object will be updated as data is streamed
    let result = { content: "" };

    // A buffer of incoming data
    let buffer = '';

    // Add an empty message to the empty array
    messages.push({})

    while (true) {
        const { value, done } = await reader.read();

        // Add the incoming data to the buffer
        buffer += new TextDecoder().decode(value);

        while (true) {
            // Regex to match the json part of the received data value
            const regex = /data:\s*([\s\S]*?)\n\n/g;
            const match = regex.exec(buffer);

            if (match) {
                // Handle end of stream
                if (match[1].includes("[END]")) {

                    // Remove last message if empty
                    if (!messages[messages.length - 1].role) {
                        messages.pop();
                    }

                    // Remove the processed part from the buffer
                    buffer = buffer.substring(match.index + match[0].length);
                    break;
                }
                // Handle end of message
                else if (match[1].includes("[DONE]")) {

                    messages[messages.length - 1] = result;
                    messages.push({})
                    result = { content: "" };

                    // Remove the processed part from the buffer
                    buffer = buffer.substring(match.index + match[0].length);
                }
                // Handle incoming message
                else {

                    // Extract the JSON part from each match
                    const jsonPart = JSON.parse(match[1])

                    console.log(jsonPart)

                    // Role
                    if (jsonPart.role) {
                        result.role = jsonPart.role;
                    }

                    // Tool calls
                    if (jsonPart.tool_calls) {
                        if (!result.tool_calls) {
                            result.tool_calls = [
                                {
                                    id: "",
                                    type: "function",
                                    function: {
                                        name: "",
                                        arguments: ""
                                    }
                                }
                            ];
                        }

                        if (jsonPart.tool_calls[0].id) {
                            result.tool_calls[0].id = jsonPart.tool_calls[0].id;
                        }

                        if (jsonPart.tool_calls[0].function.name) {
                            result.tool_calls[0].function.name += jsonPart.tool_calls[0].function.name;
                        }

                        if (jsonPart.tool_calls[0].function.arguments) {
                            result.tool_calls[0].function.arguments += jsonPart.tool_calls[0].function.arguments;
                        }

                    }

                    // Tool name
                    if (jsonPart.name) {
                        result.name = jsonPart.name;
                    }

                    // Tool call ID
                    if (jsonPart.tool_call_id) {
                        result.tool_call_id = jsonPart.tool_call_id;
                    }

                    // Message content
                    if (jsonPart.content) {
                        result.content = result.content + jsonPart.content;
                    }

                    // Update messages array
                    messages[messages.length - 1] = result;

                    // Remove the processed part from the buffer
                    buffer = buffer.substring(match.index + match[0].length);
                }
            } else {
                break;
            }
        }

        if (done) {
            console.log(messages);
            break;
        }

    }
}

Using custom tools

Custom tools are a way to extend the capabilities of the model by providing it with a list of tools it may call. This allows the model to call functions or external APIs.

Defining custom tools

To define custom tools, you must provide an array of tools in the custom_tools parameter. You may also want to instruct the model how to use the tool in the system prompt.

For example, this is how you would define a tool that allows the model to search the web (this is similar to the built-in Browsing tool):

{
   "model": "gpt-4-turbo-tools",
   "messages": [
       {
           "role": "system",
           "content": "You are a helpful assistant. Use the `web_search` tool to search the web."
       },
       {
           "role": "user",
           "content": "Search the web for 'hello world'"
       }
   ]
   "custom_tools": [
       {
           "type": "function",
           "function": {
               "name": "web_search",
               "description": "Search the web.",
               "parameters": {
                   "type": "object",
                   "properties": {
                       "query": {
                           "type": "string",
                           "description": "Search query",
                       },
                   },
                   "required": ["query"],
               }
           }
       }
  ]
}

Receiving custom tool calls

When the model calls a tool, it will include it under the tool_calls parameter in the message object. Currently, the model will only call one tool at a time.

For example, this is how the model would perform a search using the web_search tool defined in the previous step:

 {
     "role": "assistant",
     "content": "",
     "tool_calls": [
         {
             "id": "call_NhGNTLCkZm46pKmTzirhrneT",
             "type": "function",
             "function": {
                 "name": "web_search",
                 "arguments": "{\"query\":\"hello world\"}"
             }
         }
     ]
 }

Responding to custom tool calls

Custom tool calls require a response from your application. You may choose to call a function, or send a request to an API. After performing the required action, respond with the tool call's output by appending a message in the following format to the message list:

    {
        "tool_call_id": "call_NhGNTLCkZm46pKmTzirhrneT",
        "role": "tool",
        "content": "[TOOL OUTPUT]"
    }