解决问题而非制造问题！GLM-4-AllTools API革新大模型使用体验

zenRRan

发布于 2024-07-15 14:36:17

2400

发布于 2024-07-15 14:36:17

文章被收录于专栏：深度学习自然语言处理

在日常使用大模型的时候你是否遇到过以下问题：

需要自己字斟句酌提示词（Prompt），进行CoT
需要自己为大模型分解任务
需要在多种大模型，多种工具（绘图模型，集成开发环境IDE，搜索引擎….）之间辗转才能达到最终目标......

这些都是普遍存在，靠目前的单一模型无法解决的问题。

把时间推回到250万年前，旧石器时代见证了人类从简单工具的使用者到复杂技术创造者的转变。我们的祖先南方古猿、直立人等，逐步发展出更高级的工具，如手斧和刮刀，以适应狩猎和生活的需求。大约4万年前，现代人类——智人——的出现标志着技术和文化的飞跃，工具进一步精致化如骨针、鱼钩，艺术创作如洞穴壁画和雕塑也随之兴起。这一时期，火的利用、语言和社会结构的形成，为人类文明的后续发展奠定了坚实的基础。

我们无法保证大模型的“进化”和人类的进化史必定存在强相关性，不过，若以历史作为教科书，解决问题的答案或许已在书中沉淀万年。我们相信，大模型调用工具的能力（或称作智能体能力）是解决前文问题的一种有效方法。

GLM-4-AllTools - 革新大模型API调用体验

我们在 bigmodel.cn[1]对最新的GLM-4-AllTools API进行了评测。

GLM-4-AllTools: Chat API，集成了调用工具的能力，能够自主使用多种工具组合和多轮对话来达成用户任务。和传统LLM API相比，GLM-4-AllTools API集成了五大核心功能，包括智能编程助手，安全代码沙盒，实时联网搜索，绘图设计工具，函数调用能力。其中函数调用功能支持用户自定义函数，为进阶开发者提供了便利。当模型识别到解决用户输入的请求需要搜索引擎时，它会自动激活内置的联网搜索功能，以提供实时且全面的搜索结果。搜索完成后，内置的网页浏览工具能够打开并细致查看网页内容，从而进行更深入的分析。此外，借助卓越的自我规划能力和内置的代码沙盒、绘图设计等工具，API能够仅通过一次互动就高效地处理复杂的任务。

下面请跟随我们的脚步，我们将一步一步组合这些工具，测试GLM-4-AllTools的真正实力：

Case1: 实时查询欧洲杯淘汰赛对阵

Tools: Web Search

代码：

from zhipuai import ZhipuAI
client = ZhipuAI(api_key="Zhipu API Key") # 请填写您自己的APIKey

response = client.chat.completions.create(
    model="glm-4-alltools",  # 填写需要调用的模型名称
    stream=True,
    messages=[
        {
            "role": "user",
            "content":[
                {
                    "type":"text",
                    "text":"帮我查询本届欧洲杯淘汰赛对阵情况。"
                }
            ]
        }
    ],
    tools=[
        {
            "type": "web_browser" #联网搜索
        },
    ]
)

for trunk in response:
    if trunk.choices[0].delta.content is not None:
        print(trunk.choices[0].delta.content, end="")

回复：

2024年欧洲杯淘汰赛的对阵情况如下：

- 6月30日：
  - 瑞士 vs 意大利（00:00）
  - 德国 vs 丹麦（03:00）

- 7月1日：
  - 英格兰 vs 斯洛伐克（00:00）
  - 西班牙 vs 格鲁吉亚（03:00）

- 7月2日：
  - 法国 vs 比利时（00:00）
  - 葡萄牙 vs 斯洛文尼亚（03:00）

- 7月3日：
  - 荷兰 vs 罗马尼亚（00:00）
  - 奥地利 vs 土耳其（03:00）

请注意，以上时间均为北京时间。

Case2: 查询慕尼黑天气，自动绘画天气曲线

Tools: Web Search, Code Interpreter, Sandbox

from zhipuai import ZhipuAI
client = ZhipuAI(api_key="") # 请填写您自己的APIKey

response = client.chat.completions.create(
    model="glm-4-alltools",  # 填写需要调用的模型名称
    stream=True,
    messages=[
        {
            "role": "user",
            "content":[
                {
                    "type":"text",
                    "text":"帮我查询慕尼黑07/05至07/15的日平均气温。并将所有平均气温数值组成数列，绘出折线图显示趋势。"
                }
            ]
        }
    ],
    tools=[
        {
            "type": "web_browser"
        },
        {
            "type": "code_interpreter", #智能编程助手
        },
    ]
)


for trunk in response:
    print(trunk)
    if trunk.choices[0].delta.content is not None:
        print(trunk.choices[0].delta.content, end="")

回复：这次我们进入到GLM-4-AllTools API的工作流中，看看模型为了解决Case2执行了哪些步骤：

#search
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': 'search'})]), finish_reason=None, index=0)], created=1720460600, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': '("'})]), finish_reason=None, index=0)], created=1720460600, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': 'M'})]), finish_reason=None, index=0)], created=1720460600, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': 'un'})]), finish_reason=None, index=0)], created=1720460600, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': 'ich'})]), finish_reason=None, index=0)], created=1720460600, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ' average'})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ' temperature'})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ' July'})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ' '})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': '5'})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ' to'})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ' July'})]), finish_reason=None, index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
...
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_bwN-_DC2XvXuP2meMxKzd', function=None, type='web_browser', web_browser={'input': ''})]), finish_reason='tool_calls', index=0)], created=1720460601, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='tool', tool_calls=[ChoiceDeltaToolCall(index=None, id=None, function=None, type='web_browser', web_browser={'outputs': [{'title': 'Weather in Munich in July 2024', 'link': 'https://world-weather.info/forecast/germany/munich/july-2024/', 'content': 'Detailed ⚡ Munich Weather Forecast for July 2024 – day/night 🌡️ temperatures, precipitations – World-Weather.info'}, {'title': 'Munich, Bavaria, Germany Monthly Weather | AccuWeather', 'link': 'https://www.accuweather.com/en/de/munich/80331/july-weather/178086', 'content': 'Get the monthly weather forecast for Munich, Bavaria, Germany, including daily high/low, historical averages, to help you plan ahead.'}, {'title': 'Munich, Bavaria Weather Forecast July 2024: Daily Highs/Lows & Rain Trends', 'link': 'https://www.weathertab.com/en/c/e/07/federal-republic-of-germany/bavaria/munich/', 'content': 'Explore comprehensive July 2024 weather forecasts for Munich, including daily high and low temperatures, precipitation risks, and monthly temperature trends. Featuring detailed day-by-day forecasts, dynamic graphs of daily rain probabilities, and temperature trends to help you plan ahead.'}, {'title': 'Yearly & Monthly weather - Munich, Germany', 'link': 'https://www.weather-atlas.com/en/germany/munich-climate', 'content': 'Detailed climate information with charts - average monthly weather with temperature, pressure, humidity, precipitation, wind, daylight, sunshine, visibility, and UV index data. Munich sees longer daylight hours from 15.2 hours up to 16 hours from May to July, while the shortest daylight hours ranging from 8.5 hours to 8.9 hours happens between ...'}, {'title': 'Munich climate: weather by month, temperature, rain - Climates to Travel', 'link': 'https://www.climatestotravel.com/climate/germany/munich', 'content': 'At the former Munich-Riem airport, the daily average temperature ranges from -0.4 °C (31.4 °F) in January to 18.6 °C (65.5 °F) in July. Here are the average temperatures. Precipitation is quite abundant, around 950 millimeters (37.5 inches) per year.'}, {'title': 'July weather - Summer 2024 - Munich, Germany', 'link': 'https://www.weather-atlas.com/en/germany/munich-weather-july', 'content': 'Average high temperature in July: 23.1°C; The warmest month (with the highest average high temperature) is July (23.1°C). The month with the lowest average high temperature is January (2.7°C). Average low temperature in July: 12.6°C; The month with the highest average low temperature is July (12.6°C).'}, {'title': 'Munich weather in July 2024 | Sunheron', 'link': 'https://www.sunheron.com/europe/germany/bavaria/munich-weather-july/', 'content': 'The average daily temperature in July in Munich is 25 °C (78 °F). The ideal temperature for enjoying sunbathing by the water and a good temperature for other activities. For some people, it will be too hot in the direct sunlight and the stay may not be so pleasant.'}, {'title': 'Munich in july: average Weather, Temperature and Climate', 'link': 'https://www.whereandwhen.net/when/europe/germany/munich/july/', 'content': 'The weather in july in Munich is wet (with 6.9in of precipitation over 10 days). This is better than the previous month since in june it receives an average of 7.8in of precipitation over 12 days. The climate comfortable in that locality this month. Temperatures are up to 76°F. The seasonal minimum is 63°F.'}, {'title': 'Munich weather in July 2024 | Germany: How hot? - Weather2Travel.com', 'link': 'https://www.weather2travel.com/germany/munich/july/', 'content': 'Munich weather in July 2024. Expect 23°C daytime maximum temperatures in the shade with on average 8 hours of sunshine per day in Munich in July. Check more long-term weather averages for Munich in July before you book your next holiday to Germany in 2024/2025.'}]})]), finish_reason=None, index=0)], created=1720460602, model='glm-4-alltools', usage=None, extra_json=None)

...
#code interpreter, simulate in sandbox
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': '\n'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': 'plt'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': '.'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': 'grid'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': '('})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': 'True'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': ')'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': '\n'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': 'plt'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': '.'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': 'show'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': '()'})]), finish_reason=None, index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_ByzIxpcobsFBHhonh9UgI', function=None, type='code_interpreter', code_interpreter={'input': ''})]), finish_reason='tool_calls', index=0)], created=1720460616, model='glm-4-alltools', usage=None, extra_json=None)
ChatCompletionChunk(id='8822098568301210663', choices=[Choice(delta=ChoiceDelta(content=None, role='tool', tool_calls=[ChoiceDeltaToolCall(index=None, id=None, function=None, type='code_interpreter', code_interpreter={'outputs': [{'type': 'file', 'file': 'http://all-tool-interpreter.cn-wlcb.ufileos.com/3776fa6a-c175-457b-8911-0a697383fb03_fig.png?UCloudPublicKey=TOKEN_69f58d9b-ddb3-4bd0-9b29-5a4fe422c720&Expires=1720467819&Signature=dEJgDv6+/HkQokVWmJidf39WxpU='}]})]), finish_reason=None, index=0)], created=1720460619, model='glm-4-alltools', usage=None, extra_json=None)

最后返图：

Case3: Function Calling

Tools: Code Interpreter, Sandbox

#完整代码
from zhipuai import ZhipuAI
client = ZhipuAI(api_key="") # 请填写您自己的APIKey

response = client.chat.completions.create(
    model="glm-4-alltools",  # 填写需要调用的模型名称
    stream=True,
    messages=[
        {
            "role": "user",
            "content":[
                {
                    "type":"text",
                    "text":"帮我查询慕尼黑07/05至07/15的日平均气温。并用日平均气温数列绘出折线图显示趋势。"
                }
            ]
        }
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_average_temperature_data_by_date",
                "description": "用于汇总某地某段时间内的日平均气温，输入地点(city)，日期范围(from_day,to_day)，返回对应的日平均气温数列。",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "from_day": {
                            "description": "开始日期，格式为mm/dd",
                            "type": "string"
                        },
                        "to_day": {
                            "description": "截止日期，格式为mm/dd",
                            "type": "string"
                        },
                        "avg_temperature": {
                            "description": "平均气温",
                            "type": "string"
                        },
                    "required": ["city","from_day","to_day"]
                    },
                }
            }
        },

        {
            "type": "code_interpreter",
        },
    ]
)


for trunk in response:
    print(trunk)
    if trunk.choices[0].delta.content is not None:
        print(trunk.choices[0].delta.content, end="")

需要调用函数 Function Call，模型会返回 status='requires_action'

ChatCompletionChunk(id='8822109666497191178', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=None, id='call_cPf6NnnlWzSB0Fm7ZpgUF', function=ChoiceDeltaToolCallFunction(arguments='{"city":"慕尼黑"," from_day":"07/05"," to_day":"07/15"}', name='get_average_temperature_data_by_date'), type='function')]), finish_reason='tool_calls', index=0)], created=1720463116, model='glm-4-alltools', usage=CompletionUsage(prompt_tokens=435, completion_tokens=50, total_tokens=485), extra_json=None, status='requires_action')

添加函数调用的执行结果（Tool Message），以及模型生成的函数参数（Assistant Message）。


    messages=[
        {
            "role": "user",
            "content":[
                {
                    "type":"text",
                    "text":"帮我查询慕尼黑07/05至07/15的日平均气温。并用日平均气温数列绘出折线图显示趋势。"
                }
            ]
        },
        {
              "role": "assistant",
              "content":"arguments='{city\":\"慕尼黑\", \"from_day\":\"07/05\",\" to_day\":\"07/15\"}', name='get_average_temperature_data_by_date'"
        },
        {
            "role": "tool",
            "content": "[21.7, 16.6, 29.2, 25.0, 22.2, 25.9, 18.8, 22.2, 26.2, 26.1, 19.6]"
        }
    ],

#执行
ChatCompletionChunk(id='8822109666497168900', choices=[Choice(delta=ChoiceDelta(content=None, role='tool', tool_calls=[ChoiceDeltaToolCall(index=None, id=None, function=None, type='code_interpreter', code_interpreter={'outputs': [{'type': 'file', 'file': 'http://all-tool-interpreter.cn-wlcb.ufileos.com/b9200b5b-73ed-4893-8187-1f7a1ab1c7d1_fig.png?UCloudPublicKey=TOKEN_69f58d9b-ddb3-4bd0-9b29-5a4fe422c720&Expires=1720470579&Signature=oSMDBUI6hDdZboES85CZUxgAZRc='}]})]), finish_reason=None, index=0)], created=1720463379, model='glm-4-alltools', usage=None, extra_json=None)

每次GLM-4-AllTools API调用结束，都会在最后的ChatCompletionChunk返回此次调用的Token使用情况，帮助用户进行及时快速监测。

#计费
ChatCompletionChunk(id='8822109735216674822', choices=[Choice(delta=ChoiceDelta(content=None, role='assistant', tool_calls=None), finish_reason='stop', index=0)], created=1720462433, model='glm-4-alltools', usage=CompletionUsage(prompt_tokens=1518, completion_tokens=313, total_tokens=1831), extra_json=None, status='completed'

总结：一番体验下来我对 GLM-4-AllTools API 是越用越喜欢，它主要有以下几个优点：