Advanced: Tool Calling

Tool Calling (Tool Calling / Function Calling) provides powerful and flexible capabilities for large models to access external data and execute operations.

This chapter will detail how to implement custom tools.

Tool Calling Flow:

Tool calling requires multiple rounds of conversation between the application and the model. Includes the following steps:

1. Send a request to the model with tools it can call
2. Receive tool calls from the model
3. Execute code in the application using inputs from the tool call
4. Send execution results as a second request to the model
5. Receive the final response from the model (or more tool calls)

As shown:

Implementation

Define Tool

Open the cmd/llm.go file and add the following code.

First define the weather query function:

llm.go

func GetWeather(location string) string {
    
    weatherData := map[string]interface{}{
        "location":    location,
        "temperature": 30,
        "unit":        "Celsius",
        "forecast":    "Sunny",
        "humidity":    35,
    }

    result, _ := json.Marshal(weatherData)
    return string(result)
}

Describe the tool's name, function, and parameter format:

// Define weather query tool
var weatherDefinition = openai.FunctionDefinition{
	Name:        "get_weather",
	Description: "Get the current weather in a given location",
	Parameters: json.RawMessage(`{
		"type": "object",
		"properties": {
			"location": {
				"type": "string",
				"description": "The city to get the weather for, e.g. 杭州"
			}
		},
		"required": ["location"]
	}`),
}

Field Description:

• Name: Tool name, used to identify which tool to call.
• Description: Tool function description, LLM will decide whether to call this tool based on the description.
• Parameters: Use JSON schema to define parameter names, types, and descriptions.

Define parameter struct for parsing parameters returned by LLM:

llm.go

// WeatherRequest Weather query request parameters
type WeatherRequest struct {
    Location string `json:"location"`
}

Add Tool to Request

Add tool definition to the request sent to LLM:

llm.go

request := openai.ChatCompletionRequest{
    Model:       model,
    Messages:    h.messages,
    Temperature: 0.7,
    Stream:      true,
    Tools: []openai.Tool{
        ... // Other tools
        {
            Type:     openai.ToolTypeFunction,
            Function: &weatherDefinition,
        },
    },
}

Handle Tool Calls

The response from the large model will contain tool call names and parameters. Since we use streaming mode, parameters will be sent in multiple fragments.

Why do we need to collect parameters?

In streaming mode, tool calls may be sent multiple times:

1. First: {"location":
2. Second: "杭州"
3. Third: }

Need to concatenate all fragments into complete JSON: {"location":"杭州"}

Initialize parameter collector:

Before the stream processing loop starts, initialize a Map to collect parameters:

llm.go

// Use Map to collect tool call parameters (supports multiple concurrent tool calls)
toolCallsMap := make(map[int]*openai.ToolCall)

// Process streaming response
for {
    response, err := stream.Recv()
    // ... error handling
}

Identify and collect weather tool calls:

Save tool calls and collect parameters:

llm.go

// Add weather tool parameter collection
if toolCall.Function.Name == "get_weather" {
    // First time receiving this tool call
    toolCallsMap[*toolCall.Index] = &toolCall
}

if toolCall.Function.Name == "" {
    // Parameter fragment, append to corresponding tool call
    toolCallsMap[*toolCall.Index].Function.Arguments += toolCall.Function.Arguments
}

Execute Tool and Request Again

Handle tool calls:

When the stream ends, execute all collected tool calls:

if isFinished {
    // Check if there are tool calls
    if len(toolCallsMap) > 0 {
        // 1. Build Assistant message
        var assistantToolCalls []openai.ToolCall
        for _, toolCall := range toolCallsMap {
            assistantToolCalls = append(assistantToolCalls, *toolCall)
        }

        assistantMsg := openai.ChatCompletionMessage{
            Role:      openai.ChatMessageRoleAssistant,
            Content:   fullResponse,
            ToolCalls: assistantToolCalls,
        }
        h.messages = append(h.messages, assistantMsg)

        // 2. Iterate, there may be multiple tool calls
        for _, toolCall := range toolCallsMap {
            var req WeatherRequest
            
            // Parse parameters
            if err := json.Unmarshal([]byte(toolCall.Function.Arguments), &req); err == nil {
                // Call actual weather query function
                weather := GetWeather(req.Location)
                
                // Add tool result to conversation history
                h.messages = append(h.messages, openai.ChatCompletionMessage{
                    Role:       openai.ChatMessageRoleTool,
                    Content:    weather,
                    ToolCallID: toolCall.ID,
                })
            }
        }

        // 3. Clear processed tool calls
        toolCallsMap = make(map[int]*openai.ToolCall)
        
        // 4. Update request messages (include tool results)
        request.Messages = h.messages

        // 5. Request LLM again (let it generate response based on tool results)
        stream, err = h.client.CreateChatCompletionStream(h.ctx, request)
        if err != nil {
            h.logger.WithError(err).Error("Error creating chat completion stream")
        }
        continue  // Continue processing new stream
    }
    break  // No tool calls, end loop
}

Test Effect

Run Program

Ensure RustPBX is started, then run the updated client:

cd cmd
go run . \
  --endpoint ws://127.0.0.1:8080 \
  --tts aliyun --speaker longyumi_v2 \
  --asr aliyun \
  --openai-key your_dashscope_api_key \
  --model qwen-plus \
  --openai-endpoint https://dashscope.aliyuncs.com/compatible-mode/v1 \
  --greeting "你好，有什么可以帮你的吗"

Test Conversation

Try the following questions:

Question	Expected Behavior
"How's the weather in Hangzhou today?"	✅ Calls get_weather("杭州")
"Beijing weather"	✅ Calls get_weather("北京")
"Hello"	❌ Doesn't call tool, direct reply
"What's 1+1?"	❌ Doesn't call tool, LLM calculates directly

Audio Reply Example:

Complete Code

Complete implementation code can be viewed here: https://gist.github.com/yeoleobun/4b5707f2c23ac587b18d365019147a9a

Summary

Through this chapter, you learned:

1. What tool calling is and why it's needed
2. How to define tool functions and specifications
3. How to collect tool parameters in streaming mode
4. How to execute tools and return results to LLM

Tool calling upgrades your AI Agent from a "chatbot" to a "capable assistant", a key technology for building practical AI applications!

Next Steps

🔗 OpenAI Function Calling

Learn more about Function Calling mechanism

📄️ RustPBX Feature Overview

Learn more about RustPBX features