OpenAI: Updated function calling to support files, images as tool call outputs (twitter.com)

🤖 AI Summary
OpenAI updated its function-calling feature so tools (plugins, external APIs, or agent actions) can return not just structured JSON/text but actual files and images as outputs. That means a tool call can now deliver attachments — e.g., screenshots, generated images, PDFs, CSVs, or other binary artifacts — which the model can reference and present in subsequent replies. The change streamlines workflows where tools produce visual or downloadable results and removes awkward workarounds like embedding base64 blobs or separately hosting assets. For the AI/ML community this lowers friction for multimodal and agent-driven pipelines: retrieval agents can return documents and charts directly to users, image-processing tools can pass visual results back into the conversation for further reasoning, and model-assisted apps can hand off exportable artifacts (reports, datasets) without extra plumbing. Developers should expect SDK and API updates to surface file/image attachments (likely as file objects or presigned URLs) and will need to handle sizing, scanning, and privacy considerations. Overall, the feature makes tool-augmented LLM systems more practical for real-world, multimodal tasks — improving UX and enabling richer end-to-end automation.
Loading comments...
loading comments...