Spring AI

Part of Java is Coming for Python’s Lunch - The Rise of JVM AI Agents. Slides 25–35.

Spring AI slide 25 ↗

the ChatClient you already know how to wire

One builder call configures the reviewer once slide 26 ↗

private val reviewer: ChatClient = builder.clone()
    .defaultSystem(Prompts.REVIEWER)
    .defaultTools(tools)
    .defaultAdvisors(SimpleLoggerAdvisor(), TimingAdvisor())
    .build()

A tool is a Spring bean method with a description slide 27 ↗

integration

@Component
class SpringAiTravelTools(private val facts: TravelFacts) {

    @Tool(description = "Calculate hotel nights and total costs of a trip")
    fun calculateCosts(
        @ToolParam(description = "The trip id") tripId: Long,
    ): CostBreakdown = facts.costs(tripId)
}

model tool call / result our bean

The request reads like the question and returns a Kotlin type slide 28 ↗

type safety

var draft = reviewer.prompt()
    .user(Prompts.review(tripId))
    .call()
    .entity<ReviewOutcome>()

Every call passes through a chain of advisors before it reaches the model slide 29 ↗

SimpleLoggerAdvisor

TimingAdvisor

ToolCallingAdvisor

before advice after advice

Prompt

Response

ChatModel LLM provider

tool call
result

our application

@Tool beans

TravelFacts

REST API

An advisor wraps every call like a servlet filter slide 30 ↗

observability

class TimingAdvisor : CallAdvisor {
    override fun adviseCall(
        request: ChatClientRequest,
        chain: CallAdvisorChain,
    ): ChatClientResponse {
        val started = System.currentTimeMillis()
        val response = chain.nextCall(request)
        val usage = response.chatResponse()?.metadata?.usage
        log.info("took {} ms, tokens: {}",
            System.currentTimeMillis() - started, usage?.totalTokens)
        return response
    }
}

Every review shows up as a trace without extra code slide 31 ↗

observability

spring.ai.chat.client spring.ai.advisor gen_ai.client.operation spring.ai.tool getTrip spring.ai.tool listPolicies gen_ai.client.operation

Chat memory is one more advisor, the history lives in a table slide 32 ↗

context

val memory = MessageWindowChatMemory.builder()
    .chatMemoryRepository(jdbcChatMemoryRepository)
    .maxMessages(20)
    .build()

chatClient.prompt()
    .advisors(MessageChatMemoryAdvisor.builder(memory).build())
    .advisors { it.param(ChatMemory.CONVERSATION_ID, "trip-$tripId") }
    .user("Why was my trip rejected?")
    .call()
    .content()

Spring AI already ships a judge: the `FactCheckingEvaluator` slide 33 ↗

reliability

val judge: Evaluator = FactCheckingEvaluator.builder()
    .chatClientBuilder(builder)
    .build()

val evaluation = judge.evaluate(
    EvaluationRequest(tripFacts.map(::Document), draft.toString()))

if (!evaluation.isPass) revise(draft)

The judge implements Spring AI's own `Evaluator` contract slide 34 ↗

reliability

@Component
class TripReviewEvaluator(builder: ChatClient.Builder) : Evaluator {
    private val judge = builder.clone()
        .defaultSystem(Prompts.JUDGE)
        .build()

    override fun evaluate(request: EvaluationRequest): EvaluationResponse {
        val judgement = judge.prompt()
            .user(Prompts.judge(request.dataList, request.responseContent))
            .call()
            .entity<Judgement>()
        return EvaluationResponse(
            judgement.accepted, judgement.feedback, emptyMap())
    }
}

The whole agent loop is a Kotlin `repeat` slide 35 ↗

reliability

override fun review(tripId: Long): ReviewOutcome {
    var draft = reviewer.prompt()
        .user(Prompts.review(tripId))
        .call()
        .entity<ReviewOutcome>()

    repeat(MAX_REVISIONS) {
        val evaluation = judge.evaluate(evaluationRequest(tripId, draft))
        if (evaluation.isPass) return draft
        draft = reviewer.prompt()
            .user(Prompts.revise(tripId, draft, evaluation.feedback))
            .call()
            .entity<ReviewOutcome>()
    }
    return draft
}