Kyle YU

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Let's learn more! In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point—MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable—without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. I created it while taking inspiration from Femke Plantinga's post on this topic. If you want to understand this better with code, I have linked my article in the comments. ____ If you want to learn AI/ML engineering, I have put together a free PDF (530+ pages) with 150+ core DS/ML lessons. Get here: https://lnkd.in/gi6xKmDc ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

1,382

48 Comments

🚀 Dijkstra dethroned? Tsinghua University’s new shortest path algorithm just rewrite the rules of graph traversal. For 40+ years, Dijkstra’s algorithm was the gold standard for finding shortest paths in weighted graphs. But now, a team from 🇨🇳 Tsinghua University has introduced a recursive partial ordering method that outperforms Dijkstra—especially on directed graphs. 🔍 What’s different? Instead of sorting all vertices by distance (which adds log-time overhead), this new approach uses a clever recursive structure that breaks the O(m + n log n) barrier. It’s faster, leaner, and already winning awards at STOC 2025 🏆. 📍 Why it matters: Think Google Maps, Uber routing, disaster evacuation planning, circuit design—any system that relies on real-time pathfinding across massive graphs. This breakthrough could mean faster navigation, lower compute costs, and smarter agentic systems. 💡 As someone building agentic workflows and democratizing AI tooling, I see this as a seismic shift in how we think about graph-based applications. Let’s rethink what “shortest path” really means. Sometimes, the fastest route isn’t the one we’ve been taking for decades. Read the full article here: https://lnkd.in/drgxYbN4 #GraphAlgorithms #AIEngineering #DijkstraVsTsinghua #ShortestPath #AgenticAI #GoogleMaps #Pathfinding #TechBreakthrough #LLMWorkflows #Innovation

243

Our team ByteBrain, in collaboration with Tsinghua University, proposed ChatTS, a time-series multimodal LLM, accepted to VLDB 2025. Here is a brief introduction: Understanding time series is crucial for its application in real-world scenarios. Recently, large language models (LLMs) have been increasingly applied to time series tasks, leveraging their strong language capabilities to enhance various applications. However, research on multimodal LLMs (MLLMs) for time series understanding and reasoning remains limited, primarily due to the scarcity of high-quality datasets that align time series with textual information. This paper introduces ChatTS, a novel MLLM designed for time series analysis. ChatTS treats time series as a modality, similar to how vision MLLMs process images, enabling it to perform both understanding and reasoning with time series. To address the scarcity of training data, we propose an attribute-based method for generating synthetic time series with detailed attribute descriptions. We further introduce Time Series Evol-Instruct, a novel approach that generates diverse time series Q&As, enhancing the model’s reasoning capabilities. To the best of our knowledge, ChatTS is the first MLLM that takes multivariate time series as input for understanding and reasoning, which is fine-tuned exclusively on synthetic datasets. We evaluate its performance using benchmark datasets with real-world data, including six alignment tasks and four reasoning tasks. Our results show that ChatTS significantly outperforms existing vision-based MLLMs (e.g., GPT-4o) and text/agent-based LLMs, achieving a 46.0% improvement in alignment tasks and a 25.8% improvement in reasoning tasks. We have open-sourced the source code, model checkpoint and datasets at https://lnkd.in/gYDJ9S4j Paper: https://lnkd.in/gqXW9tPY About ByteBrain Team: ByteBrain is a platform of AI for Infra (AI for System) at ByteDance. The goal is to leverage AI to automatically optimize system performance and reduce labor and resource costs. ByteBrain integrates machine learning, LLM and OR techniques to support both internal infrastructure systems at ByteDance as well as the Volcano Engine Cloud.

154

3 Comments

🚨 New work alert: 𝐆𝐫𝐚𝐩𝐡𝐃𝐚𝐧𝐜𝐞𝐫 🚨 Reasoning over graph-structured knowledge is hard: it requires precise function execution, multi-hop exploration, and adaptive decision making—not just similarity-based node retrieval. We introduce 𝐆𝐫𝐚𝐩𝐡𝐃𝐚𝐧𝐜𝐞𝐫, an RL framework that trains LLMs to adaptively interleave natural-language reasoning with graph function execution while answering graph-related questions. At the core of GraphDancer is a 𝐠𝐫𝐚𝐩𝐡-𝐚𝐰𝐚𝐫𝐞 𝐜𝐮𝐫𝐫𝐢𝐜𝐮𝐥𝐮𝐦 that allows even moderate-sized models (e.g., 3B parameters) to internalize multi-hop reasoning skills and generalize robustly to 𝐮𝐧𝐬𝐞𝐞𝐧 𝐝𝐨𝐦𝐚𝐢𝐧𝐬 and 𝐡𝐞𝐭𝐞𝐫𝐨𝐠𝐞𝐧𝐞𝐨𝐮𝐬 𝐠𝐫𝐚𝐩𝐡 𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞𝐬. 🌟 Key features - 𝐈𝐧𝐭𝐞𝐫𝐥𝐞𝐚𝐯𝐞𝐝 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 & 𝐀𝐜𝐭𝐢𝐨𝐧: Train agents to alternate between <𝘵𝘩𝘪𝘯𝘬> (reasoning) and <𝘨𝘳𝘢𝘱𝘩> (function execution) blocks - 𝐆𝐫𝐚𝐩𝐡-𝐀𝐰𝐚𝐫𝐞 𝐂𝐮𝐫𝐫𝐢𝐜𝐮𝐥𝐮𝐦 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: A novel training scheduler that progresses from simple lookup tasks to complex neighborhood expansion and exploration - 𝐂𝐫𝐨𝐬𝐬-𝐃𝐨𝐦𝐚𝐢𝐧 𝐆𝐞𝐧𝐞𝐫𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 Trained on a single domain (Academic), GraphDancer generalizes to unseen domains and question types spanning E-commerce, Literature, Healthcare, and Legal Despite using only a 3B backbone, GraphDancer outperforms baselines built on 14B models and even GPT-4o-mini! 𝐏𝐚𝐩𝐞𝐫: https://lnkd.in/gb68ese9 𝐏𝐫𝐨𝐣𝐞𝐜𝐭: https://lnkd.in/gRM-Q8cu 𝐂𝐨𝐝𝐞: https://lnkd.in/ggp2PFkN 𝐌𝐨𝐝𝐞𝐥: https://lnkd.in/gHEs9SYg This project is led by my amazing PhD student Yuyang Bai, in collaboration with Zhuofeng Li, Ping Nie, and Jianwen Xie. We also thank Prof. Cheng Zhang for his support. #LLM #ReinforcementLearning #GraphReasoning #AIResearch

143

6 Comments

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,671

190 Comments

I’m starting a new Medium series where I study ML system design in depth and connect it directly to modern LLM systems. The first piece is based on 𝗗𝗲𝘀𝗶𝗴𝗻𝗶𝗻𝗴 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 𝗯𝘆 Chip Huyen, I’m focusing on one question: What do you need to understand before writing a single line of model code? A few ideas that stood out while writing Part 1: - The model is usually the smallest part of the system. - Latency matters more than average accuracy in production. - ML objectives must be explicitly tied to business metrics. - Batch vs. stream data decisions shape everything downstream. LLM systems follow the same principles just with different failure modes. Writing this forced me to think less about algorithms and more about: data pipelines, ownership, observability, and system trade-offs. I’m planning this as a 3-part deep dive: Part 1 — Foundations Part 2 — Building the Core Part 3 — Deployment & Monitoring If you’re studying ML system design (or thinking about LLM systems beyond prompts), I’d love to hear what topics you think are most misunderstood. 🔗 Link: https://lnkd.in/eQbaCp3u #MachineLearning #MLOps #LLM #AIEngineering #SystemsDesign #AI #DataScience

95

NVIDIA Research just published an intersting paper stating that SLMs aren't just "budget alternatives" to LLMs, but they're often superior for real-world applications. Their analysis of popular agents like MetaGPT, Open Operator, and Cradle shows that 40-70% of current LLM calls could be effectively replaced by specialized SLMs. Meanwhile, Andrej Karpathy envisions an even more radical future: a small "cognitive core" model that runs locally by default, with always-on capabilities, multimodal input/output, and the ability to dial up reasoning power when needed while delegating complex tasks to cloud-based oracles. The convergence is clear: we're moving from monolithic, cloud-dependent LLMs toward modular, efficient architectures. SLMs offer 10-30x better cost efficiency, lower latency, and can be fine-tuned faster and at a reasonable cost. For agentic systems performing repetitive and specialized task are better handled with careful context engineering and specialized SLMs. The future isn't about choosing between big and small models, but it's about intelligent orchestration of both. Paper: https://lnkd.in/gZAgH4Y6 Andrej Karpathy's X post: https://lnkd.in/g23777Mc

114

2 Comments

Throughout my Ph.D., I have developed numerous profiling tools to understand and diagnose latency bottlenecks in deep learning models. Most existing commercial profilers are either designed for very specific purposes and are difficult to customize, or they offer complex features with steep learning curves. As a result, building profiling tools often becomes one of the most painful parts of research. Surprisingly, almost every researcher ends up reinventing this wheel to showcase latency improvements in their own work. Encouraged by my advisor, Prof. Diana Marculescu, we decided to consolidate these tools and open-source them to benefit future researchers in this area. We compiled and released them as ELANA: A Simple Energy & Latency Analyzer for LLMs: 📄 Technical report: https://lnkd.in/g8HmscPP 💻 Github repo: https://lnkd.in/gDUA2_Ny ELANA is a lightweight, academic-friendly profiler built on Hugging Face APIs for analyzing LLM latency and energy efficiency. We hope it will serve as a foundation for reproducible benchmarking, enable fair comparisons across models and systems, and accelerate research on resource-efficient LLMs. Huge thanks to Bokun Wang and Prof. Diana Marculescu for their invaluable support on both the tool and the technical report.

50

1 Comment