Log requests and responses to Open AI

[iAMSagar44]

Has anyone managed to find a way to log the requests and responses to/from Open AI using Spring AI.
I have tried various logging settings in the application.properties but no luck.
With Langchain4J, there is a specific property for logging the requests and responses to/from the AI models, but couldn't find anything similar with Spring AI.

[iamroberthanson]

I was looking to do this too, and did a little digging.

RestClient uses a default RestClient.Builder bean, but you can override it with your own. Here I took the default implementation and added a requestInterceptor() to the builder. The code is in Groovy, so you will need to add some semi-colons and stuff.

// Groovy code   

/**
 * This overrides the default factory bean for the REST client, and allows us to customize it.
 * In this case we are adding an interceptor to log the request and response.
 * The rest of this code comes from the default implementation in
 *   org.springframework.boot.autoconfigure.web.client.RestClientAutoConfiguration
 */
@Bean
@Scope("prototype")
RestClient.Builder restClientBuilder (RestClientBuilderConfigurer restClientBuilderConfigurer)
{
    RestClient.Builder builder = RestClient.builder()
        .requestFactory(ClientHttpRequestFactories.get(ClientHttpRequestFactorySettings.DEFAULTS))
        .requestInterceptor(new RestClientInterceptor())
    return restClientBuilderConfigurer.configure(builder);
}

Then your interceptor would look like this. This is just some working test code right now, so it just prints to the console.

import org.apache.commons.io.IOUtils
import org.springframework.http.HttpRequest
import org.springframework.http.client.ClientHttpRequestExecution
import org.springframework.http.client.ClientHttpRequestInterceptor
import org.springframework.http.client.ClientHttpResponse
import org.springframework.util.StreamUtils

import java.nio.charset.Charset

class RestClientInterceptor implements ClientHttpRequestInterceptor
{

    @Override
    ClientHttpResponse intercept (HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException
    {
        // log the request
        logRequest(request, body)
        
        // execute the request
        ClientHttpResponse response = execution.execute(request, body)
        
        // wrap the response that buffers the body, so that we can read from it multiple times
        BufferingClientHttpResponseWrapper responseWrapper = new BufferingClientHttpResponseWrapper(response)

        // log the response
        logResponse(responseWrapper)

        // return the wrapped response
        return responseWrapper
    }

    static void logRequest (HttpRequest request, byte[] bytes)
    {
        println "Request: ${request.method} ${request.URI}"
        println "Headers: ${request.headers}"
        println "Body: ${new String(bytes)}"
    }

    static void logResponse (ClientHttpResponse response)
    {
        println "Response: ${response.statusCode} ${response.statusText}"
        println "Headers: ${response.headers}"
        println "Body: ${IOUtils.toString(response.body, Charset.forName('UTF-8')).length()}"
    }

    // A Groovy-ier version of:
    // https://github.com/spring-projects/spring-framework/blob/main/spring-web/src/main/java/org/springframework/http/client/BufferingClientHttpResponseWrapper.java
    static class BufferingClientHttpResponseWrapper implements ClientHttpResponse
    {
        @Delegate ClientHttpResponse response;
        byte[] body;

        BufferingClientHttpResponseWrapper (ClientHttpResponse response)
        {
            this.response = response;
        }

        @Override
        InputStream getBody() throws IOException
        {
            if (this.body == null) {
                this.body = StreamUtils.copyToByteArray(this.response.getBody())
            }
            return new ByteArrayInputStream(this.body)
        }
    }
}

Because you can only read the body of the response once, you need to wrap the response in a wrapper class that buffers the body. If you are using Java, you can use the link in the comment for the wrapper class to get the Java-ish source.