Function Calling Deep Dive
Enable your chatbot to take real actions—query databases, call APIs, process payments, and interact with external systems. Transform passive AI into an active assistant.
Function calling (also called "tool use") is arguably the most powerful capability of modern LLMs. Instead of just generating text, the model can recognize when a user's request requires external data or actions, then intelligently call your Java functions to fulfill that request.
When a user asks "What's the status of my order ORD-12345?", the LLM doesn't hallucinate an answer. Instead, it recognizes this requires the orderStatus function, extracts the order ID, calls your function, and weaves the real result into a natural language response.
How Function Calling Works
User Request
User asks: 'What's the weather in Tokyo?'
LLM Analysis
Model recognizes this requires the currentWeather function
Function Call
Spring AI invokes your Java function with {location: 'Tokyo'}
Result Processing
Your function returns real weather data from an API
Natural Response
LLM crafts: 'It's currently 68°F and partly cloudy in Tokyo.'
What Can Functions Do?
Any operation your Java application can perform, your AI chatbot can now intelligently trigger.
API Integration
Call external APIs like weather, maps, or payment gateways
Database Queries
Fetch order status, user profiles, or inventory data
Calculations
Compute shipping costs, taxes, or loan payments
Scheduling
Book appointments, check availability, set reminders
Search
Query product catalogs, documentation, or knowledge bases
E-commerce
Process orders, apply discounts, handle returns
Define Your First Function
Functions in Spring AI are just regular Java Function<Request, Response> beans. The @Description annotation tells the LLM when to use this function.
@ConfigurationpublicclassFunctionConfig{@Bean@Description("Get the current weather for a given location")publicFunction<WeatherRequest,WeatherResponse>currentWeather(){return request ->{// In production, call a real weather APIreturnnewWeatherResponse(
request.location(),"72°F","Sunny with light clouds");};}}// Request and Response recordspublicrecordWeatherRequest(String location,String unit){}publicrecordWeatherResponse(String location,String temperature,String conditions){}This example demonstrates the fundamental pattern for creating callable functions in Spring AI. The Function<WeatherRequest, WeatherResponse> interface is a standard Java functional interface, making it instantly familiar to any Java developer. Spring AI introspects this bean, extracts its parameter types, and automatically generates a JSON schema that the LLM can understand.
The @Description annotation serves as documentation for the AI model. When you register this function, Spring AI sends both the function signature and this description to the LLM. The model uses this information to decide when calling your function is appropriate. A vague description like "gets data" will lead to poor function selection, while a specific description like "Get the current weather conditions and temperature for a given city or location" helps the model make accurate decisions.
Notice that WeatherRequest and WeatherResponse are simple Java records. Spring AI uses Jackson to serialize and deserialize these objects. The property names in your records become the parameter names in the function schema—solocation and unitare exactly what the LLM will see and try to populate from the user's message.
The @Description is crucial! It's sent to the LLM as part of the function schema. Write it from the AI's perspective—what does this function do, and when should it be used?
Enable Functions in ChatClient
Register functions with your ChatClient using .defaultFunctions(). The LLM will automatically call them when appropriate based on user input.
@RestController@RequestMapping("/api/assistant")publicclassAssistantController{privatefinalChatClient chatClient;publicAssistantController(ChatClient.Builder builder){this.chatClient = builder
.defaultSystem("""
You are a helpful assistant with access to real-time tools.
When users ask about weather, use the currentWeather function.
When they ask about orders, use the orderStatus function.
Always provide helpful, accurate information based on tool results.
""").defaultFunctions("currentWeather","orderStatus","calculateShipping").build();}@PostMapping("/chat")publicStringchat(@RequestBodyChatRequest request){return chatClient.prompt().user(request.message()).call().content();}}Understanding the Flow
.defaultFunctions(...)Register functions by their bean names. These are available for every request unless overridden.
System PromptGuide the LLM on when to use functions. Be explicit: "When users ask about weather, use the currentWeather function."
The system prompt plays a critical role in function calling success. While the LLM can infer when to use functions based on the function descriptions alone, explicit guidance in the system prompt dramatically improves accuracy. By telling the model exactly which scenarios map to which functions, you reduce the chance of the model trying to answer questions it should delegate to a function.
The .defaultFunctions() method registers functions by their bean names. This means the function name in your configuration class (like currentWeather()) becomes the identifier you use here. Spring AI looks up these beans from the application context, so you have full access to dependency injection—your functions can use services, repositories, API clients, and any other Spring-managed beans.
When a request comes in, the ChatClient sends the user's message along with the function schemas to the LLM. If the model determines a function call is needed, it returns a special response indicating which function to call and with what arguments. Spring AI automatically deserializes these arguments into your request object, invokes the function, and sends the result back to the LLM for final response generation. This entire round-trip happens transparently within the.call() method.
Build a Real E-commerce Assistant
Let's create a practical example with multiple functions that work together. This assistant can check orders, inventory, and calculate shipping—all through natural conversation.
@ConfigurationpublicclassEcommerceFunctions{privatefinalOrderRepository orderRepository;privatefinalInventoryService inventoryService;privatefinalShippingCalculator shippingCalculator;@Bean@Description("Get the status of an order by order ID")publicFunction<OrderStatusRequest,OrderStatusResponse>orderStatus(){return request ->{Order order = orderRepository.findById(request.orderId()).orElseThrow(()->newOrderNotFoundException(request.orderId()));returnnewOrderStatusResponse(
order.getId(),
order.getStatus().name(),
order.getEstimatedDelivery(),
order.getTrackingNumber());};}@Bean@Description("Check if a product is in stock and get available quantity")publicFunction<InventoryRequest,InventoryResponse>checkInventory(){return request ->{InventoryItem item = inventoryService.check(request.productId());returnnewInventoryResponse(
request.productId(),
item.isInStock(),
item.getQuantityAvailable(),
item.getRestockDate());};}@Bean@Description("Calculate shipping cost based on destination and package weight")publicFunction<ShippingRequest,ShippingResponse>calculateShipping(){return request ->{ShippingQuote quote = shippingCalculator.calculate(
request.destinationZip(),
request.weightLbs(),
request.shippingMethod());returnnewShippingResponse(
quote.getCost(),
quote.getEstimatedDays(),
quote.getCarrier());};}}Request/Response Records
// Order StatuspublicrecordOrderStatusRequest(@JsonProperty(required =true)String orderId
){}publicrecordOrderStatusResponse(String orderId,String status,LocalDate estimatedDelivery,String trackingNumber
){}// Inventory CheckpublicrecordInventoryRequest(@JsonProperty(required =true)String productId
){}publicrecordInventoryResponse(String productId,boolean inStock,int quantityAvailable,LocalDate restockDate
){}// Shipping CalculationpublicrecordShippingRequest(@JsonProperty(required =true)String destinationZip,@JsonProperty(required =true)double weightLbs,String shippingMethod // Optional: "standard", "express", "overnight"){}publicrecordShippingResponse(BigDecimal cost,int estimatedDays,String carrier
){}Pro tip: Use @JsonProperty(required = true) on mandatory parameters. The LLM will be told these fields are required in the schema.
This e-commerce example showcases how multiple functions work together to create a complete customer service assistant. Each function handles a specific concern: order tracking, inventory checking, and shipping estimation. The LLM intelligently decides which function to call based on the user's intent. For questions like "Is the blue widget in stock?", it calls checkInventory. For "How much would it cost to ship to 90210?", it uses calculateShipping.
Notice how each function is self-contained and stateless. The orderStatus function takes an order ID and returns status information—it doesn't need to know about previous messages or maintain any state. This design makes functions easy to test, scale, and reuse across different chatbot contexts.
The request and response records define the contract between the LLM and your code. The LLM sees the field names, types, and which fields are required. Well-designed DTOs guide the model to extract the right information from user messages. For example,ShippingRequest has optional shippingMethod—if the user doesn't specify, the function can apply a default. Required fields ensure the LLM asks for missing information before calling the function.
Dynamic Function Selection
Not all users should have access to all functions. Here's how to dynamically enable functions based on user role, subscription level, or context.
@RestController@RequestMapping("/api/agent")publicclassDynamicAgentController{privatefinalChatClient.Builder chatClientBuilder;privatefinalMap<String,Function<?,?>> availableFunctions;@PostMapping("/chat")publicStringchat(@RequestBodyChatRequest request,@RequestHeader("X-User-Role")String userRole){// Dynamically select functions based on user roleList<String> enabledFunctions =switch(userRole){case"admin"->List.of("orderStatus","checkInventory","modifyOrder","issueRefund");case"support"->List.of("orderStatus","checkInventory","createTicket");case"customer"->List.of("orderStatus","calculateShipping");default->List.of("orderStatus");};ChatClient client = chatClientBuilder
.defaultSystem("You are a helpful assistant. Use available tools when relevant.").defaultFunctions(enabledFunctions.toArray(newString[0])).build();return client.prompt().user(request.message()).call().content();}}Security is paramount when giving AI assistants the ability to take actions. Role-based function access ensures that users can only invoke functions appropriate to their permission level. A customer shouldn't be able to issue refunds, and a support agent might not have access to system administration functions.
In this pattern, instead of configuring functions at application startup with .defaultFunctions(), we build a fresh ChatClient for each request with only the permitted functions enabled. The LLM will only see and be able to call the functions you explicitly allow. This is defense-in-depth: even if a user tries to trick the AI into calling a privileged function, that function simply doesn't exist in the context.
You can extend this pattern beyond roles. Consider enabling different functions based on subscription tier (premium users get advanced analytics functions), time of day (scheduling functions only during business hours), orconversation context (order modification functions only after the user has identified an order).
Robust Error Handling
Functions can fail—services go down, validation fails, data doesn't exist. Handle errors gracefully so the LLM can communicate issues naturally to users.
@Bean@Description("Search for products by name or category")publicFunction<ProductSearchRequest,ProductSearchResponse>searchProducts(){return request ->{try{// Validate inputif(request.query()==null|| request.query().isBlank()){returnnewProductSearchResponse(List.of(),"Please provide a search term",false);}List<Product> results = productService.search(
request.query(),
request.category(),
request.maxResults()!=null? request.maxResults():10);returnnewProductSearchResponse(
results.stream().map(p ->newProductSummary(p.getId(), p.getName(), p.getPrice())).toList(),null,true);}catch(ServiceUnavailableException e){
log.error("Product service unavailable", e);returnnewProductSearchResponse(List.of(),"Search is temporarily unavailable. Please try again.",false);}catch(Exception e){
log.error("Unexpected error in product search", e);returnnewProductSearchResponse(List.of(),"An error occurred while searching. Please try again.",false);}};}// Response includes error handling fieldspublicrecordProductSearchResponse(List<ProductSummary> products,String errorMessage,boolean success
){}Real-world functions interact with external services that can fail in countless ways. Database connections time out,third-party APIs return 500 errors, users provide invalid input. Your functions must anticipate these failures and communicate them in a way the LLM can understand and relay to users naturally.
The response structure includes three key elements: the results (empty on failure), an errorMessageexplaining what went wrong, and a success boolean for quick status checking. This pattern allows the LLM to generate contextually appropriate responses. On success, it might say "I found 5 products matching your search." On validation failure: "I need a search term to help you find products." On service error: "I'm sorry, our product search is temporarily unavailable. Please try again in a few minutes."
Notice the comprehensive try-catch structure. Input validation happens first, before any external calls.Known exceptions like ServiceUnavailableException get specific handling with user-friendly messages. The generic catch(Exception) block ensures unexpected errors don't crash the function—they still return a graceful error response that the LLM can work with.
Never throw exceptions from functions! Return error information in your response object. The LLM needs structured data to explain the problem to users.
Streaming with Function Calls
Functions work seamlessly with streaming. The LLM pauses streaming momentarily to execute the function, then continues with the result integrated into the response.
@GetMapping(value ="/chat/stream", produces =MediaType.TEXT_EVENT_STREAM_VALUE)publicFlux<String>streamChat(@RequestParamString message){return chatClient.prompt().user(message).functions("currentWeather","orderStatus").stream().content();}// Note: Function calls happen synchronously during streaming.// The LLM will:// 1. Determine a function needs to be called// 2. Execute the function (blocks briefly)// 3. Continue streaming the response with function resultsStreaming and function calling might seem incompatible—how can you stream a response while waiting for a function to execute? Spring AI handles this elegantly. The model begins generating text, and when it needs to call a function, the stream pauses briefly while your function executes. Once complete, streaming resumes with the function result incorporated into the response.
From the user's perspective using a frontend with Server-Sent Events, they see the AI start typing, perhaps hear "Let me check the weather for you..." appear token-by-token, then after a brief pause for the API call, continue with "It's currently 72°F and sunny in San Francisco." This creates a natural, conversational experience even when external data is being fetched.
Testing Function Calls
Test that the LLM correctly identifies when to call functions and handles the results appropriately.
@SpringBootTestclassFunctionCallingTest{@AutowiredprivateChatClient chatClient;@MockBeanprivateOrderRepository orderRepository;@TestvoidshouldCallOrderStatusFunction(){// GivenOrder mockOrder =newOrder("ORD-123",OrderStatus.SHIPPED,LocalDate.now().plusDays(2),"1Z999AA1012345678");when(orderRepository.findById("ORD-123")).thenReturn(Optional.of(mockOrder));// WhenString response = chatClient.prompt().user("What's the status of order ORD-123?").call().content();// ThenassertThat(response).containsIgnoringCase("shipped");assertThat(response).contains("1Z999AA1012345678");verify(orderRepository).findById("ORD-123");}@TestvoidshouldHandleFunctionError(){// Givenwhen(orderRepository.findById(any())).thenThrow(newOrderNotFoundException("ORD-999"));// WhenString response = chatClient.prompt().user("Where is my order ORD-999?").call().content();// ThenassertThat(response).containsAnyOf("not found","couldn't find","doesn't exist");}}Testing function-calling behavior requires a slightly different approach than traditional unit tests. You're testing the integration between the LLM and your functions—verifying that natural language queries correctly trigger function calls and that results are incorporated into responses.
The tests above use @MockBean to control the backend services your functions call. This isolates the test from external dependencies while still exercising the full function-calling pipeline. We verify that the LLM correctly identifies when to call orderStatus, passes the right order ID, and incorporates the result (tracking number, status) into its response.
The error handling test is equally important. It confirms that when the function returns an error (order not found), the LLM generates an appropriate user-facing message rather than exposing technical details or falling back to hallucination. Consider building a comprehensive test suite covering edge cases: missing parameters, malformed input, partial data, and various error conditions.
Function Calling Best Practices
Write clear @Description annotations
The LLM uses this to decide when to call your function. Be specific about the function's purpose and when it should be used.
Use descriptive parameter names
Parameter names become part of the schema. 'destinationZipCode' is clearer than 'zip' or 'code'.
Return structured error information
Include success/failure flags and error messages in responses so the LLM can explain issues naturally.
Keep functions focused
Each function should do one thing well. If you need complex workflows, compose multiple functions.
Limit available functions
Too many functions can confuse the model. Enable only relevant functions for each context.
Log function calls
Track which functions are called and why. This helps debug unexpected behavior and optimize performance.