Concept for Callbacks for Events within Spring AI Framework

[henningSaulCM]

Hi,

I'm relatively knew to Spring AI (and AI in general). I'm currently looking into re-implementing a demo application (built using some other AI framework) using Spring AI. That other AI framework has some concept and mechanism for "Callbacks" to respond to events within the framework, e.g. calling function/tool, function/tool returned with result, etc. The demo application returns those "events" to the frontend to provide some feedback to the user for the current request, e.g.

8:[{"type":"events","data":{"title":"Using tool: 'local_index_query_engine' with inputs: 'query: How can I create an Article?'"}}]
8:[{"type":"events","data":{"title":"Retrieving context for query: 'How can I create an Article?'"}}]
8:[{"type":"events","data":{"title":"Retrieved 2 sources to use as context for the query"}}]
8:[{"type":"tools","data":{"toolCall":{"id":"call_5M54tvFLzvK0X3t6ArqV9L0c","name":"local_index_query_engine","input":{"query":"How can I create an Article?"}},"toolOutput":{"output":"To create an Article, you can follow these steps....","isError":false}}}]
...

Is there a similar concept in Spring AI (existing or planned) or can you think of alternative approaches to expose the internal steps of the framework to the end user/in a client request?

Your feedback is much appreciated!
--Henning

[tobins]

Hello!

Would Function Calling work for what you are interested in doing?

https://docs.spring.io/spring-ai/reference/api/functions.html

With Spring AI, you can give the AI model functions to invoke to complete a request.

[henningSaulCM]

Thanks for chiming in, @tobins.

My question is to some extent within the context of Function Calling (which we already do/use).

To be concrete, I was looking for an equivalent (or an alterative approach) to https://docs.llamaindex.ai/en/stable/module_guides/observability/callbacks/root/.
This allows one to easily get callbacks/notifications for steps/actions within the Framework.

The example (typescript) app I'm looking at uses the above mechanism to notify the frontend about events, e.g. when a tool is called during the execution.

  callbackManager.on("llm-tool-call", (event) => {
    const { name, input } = event.detail.payload.toolCall;
    const inputString = Object.entries(input)
      .map(([key, value]) => `${key}: ${value}`)
      .join(", ");
    appendEventData(
      stream,
      `Using tool: '${name}' with inputs: '${inputString}'`,
    );
  });

  callbackManager.on("llm-tool-result", (event) => {
    const { toolCall, toolResult } = event.detail.payload;
    appendToolData(stream, toolCall, toolResult);
  });