# Java is Coming for Python’s Lunch - The Rise of JVM AI Agents

Source: /26-j-fall/

## Java is Coming for Python’s Lunch - The Rise of JVM AI Agents

Slide 1: /26-j-fall/#/1

#### Java is Coming for Python’s Lunch

#### The Rise of JVM AI Agents

Frederik Pietzko - JavaCro 2026

### Q&A

Slide 68: /26-j-fall/#/68

Slides & code

> Speaker notes: - Slides + demo code - Spring AI, Embabel, Koog variants

## Agents in production need what Java has delivered for decades

Slide 2: /26-j-fall/#/2

context
  reliability
  type safety
  performance
  observability
  integration

> Speaker notes: - Demo agent: prompt + API key - Production agent needs more - context - data, domain model, business rules - reliability - retries, timeouts, transactions - type safety - output checked by the compiler - performance - concurrent network calls - observability - what the agent did and why - integration - systems we already run - Nothing new - Java's job for decades

### Java has kept the enterprise running for 30 years

Slide 3: /26-j-fall/#/3

1996Java 1.0
  2004Spring 1.0
  2014Java 8 lambdas
  2023virtual threads
  2025JVM agent frameworks

> Speaker notes: - 30 years of resilient enterprise systems - banks, insurers, airlines, logistics - Evolved without breaking users - Spring, Java 8 lambdas, Java 21 virtual threads - [click] Agent frameworks - Spring AI, Koog, Embabel - all within the last year or so - Agents = next chapter, not a different book

### What we'll cover

Slide 4: /26-j-fall/#/4

1. Why python was first and why Java will last
2. The demo: reviewing business trips
3. Spring AI
4. Embabel
5. Koog

## Why Python was first

Slide 5: /26-j-fall/#/5

#### and why Java will last

> Speaker notes: - Not bashing Python - it earned its place - Why Python won round one - Why the JVM fits now - Six tiles from the intro - one tag per slide

### AI started in the lab, and the lab spoke Python

Slide 6: /26-j-fall/#/6

research & quick prototyping

    numpy
    pandas
    scikit-learn

> Speaker notes: - AI = research topic, done by scientists - Python: notebooks, no build, no types, throwaway prototypes - Early network effects - [click] numpy - [click] pandas - [click] scikit-learn - every paper in Python - next paper too - Won on ecosystem, not language - Research code runs once on a laptop - no reliability, type safety, observability needed

### Researchers joined companies and brought Python with them

Slide 7: /26-j-fall/#/7

research lab

  companies

  LangChain
  LangGraph

> Speaker notes: - Researchers hired into industry, kept Python - [click] First agent frameworks in Python - LangChain, LangGraph - tutorials, courses - all Python - "AI means Python" becomes common wisdom

### Python was the glue, the heavy lifting always ran in C and C++

Slide 8: /26-j-fall/#/8

the API you call

    the code that actually does the math

> Speaker notes: - numpy, PyTorch, TensorFlow - C, C++, CUDA underneath - Python = thin convenience layer - Top layer always replaceable

### LLMs moved the heavy lifting into the datacenter

Slide 9: /26-j-fall/#/9

your application
  HTTPS

    thousands of GPUs

> Speaker notes: - LLMs mostly too big to run locally - datacenters, thousands of GPUs, plain APIs - Heavy lifting no longer on our machine - "Python has the math libs" argument gone - Left on our side: classic enterprise software - calling services, validation, failure handling, 24/7 - Java's 30 year track record

### What travels over the wire is JSON and JSON Schema

Slide 10: /26-j-fall/#/10

type safety

```json
{
  "name": "create_invoice",
  "parameters": {
    "type": "object",
    "properties": {
      "customer": { "type": "string" },
      "total":    { "type": "number" },
      "dueDate":  { "type": "string", "format": "date" }
    },
    "required": ["customer", "total", "dueDate"]
  }
}
```

> Speaker notes: - Everything is JSON - requests, responses, tool definitions - Structured output + tool calling = JSON Schema - Typical tool definition = a type definition in JSON - Python: dicts, validation bolted on - Java: typed JSON for 20 years

### An agent is mostly concurrent network IO, and the JVM is built for it

Slide 11: /26-j-fall/#/11

performance

  your agent
  JSON
  LLM API
  JSON
  tools & services

> Speaker notes: - Network IO + JSON handling - call model, parse, call tool, repeat - [click] Tools = more HTTP calls - JVM strengths: HTTP clients, connection pools, virtual threads - Performance = waiting well - thousands of parallel runs blocked on LLM calls - virtual threads, no reactive gymnastics - one JVM vs worker processes and the GIL

### Jackson turns a record into the schema and the answer back into a record

Slide 12: /26-j-fall/#/12

type safety

**Java**

```java
record Invoice(String customer, BigDecimal total, LocalDate dueDate) {}

JsonSchema schema = schemaGenerator.generateSchema(Invoice.class);

Invoice invoice = mapper.readValue(llmResponse, Invoice.class);
```

> Speaker notes: - Same invoice, as a record - Jackson: type -> JSON Schema - Jackson: response -> type - Compiler catches mismatches - Malformed answer fails at the boundary - What you see is what runs

### The context an agent needs already lives in JVM systems

Slide 13: /26-j-fall/#/13

context

    the data, the domain model, the business rules

    banking
    insurance
    logistics
    manufacturing
    healthcare

> Speaker notes: - Most enterprise systems in Java - banking, insurance, logistics, manufacturing, healthcare - Agent only as good as its context - model knows the world, not your customers, contracts, rules - already modelled in these systems - Most automation potential lives here

### Put the agent inside the system, not next to it

Slide 14: /26-j-fall/#/14

integration

    an MCP server and a Python agent per legacy system

    the agent lives in the Java project you already have

> Speaker notes: - Python way - MCP server per legacy system - separate Python agent - two stacks, deployments, teams - [click] Agent inside the Java project - same build, tests, ops, people - Integration for free - tools = plain method calls - same transactions, security, DB connections - no second domain model

### A production agent needs the same safety net as any other service

Slide 15: /26-j-fall/#/15

reliabilityobservability

    your agent

    retries
    security
    OpenTelemetry
    Prometheus
    Grafana

> Speaker notes: - LLM calls: slow, flaky, expensive - unreliable remote service - Reliability - retries, timeouts, circuit breakers, rate limits - Resilience4j, Spring Retry - security + transactions around tool calls - Observability - [click] model + tool calls as trace spans - [click] tokens + latency as metrics - [click] existing ops dashboards - No observability, no production - Spring AI built in, Embabel + Koog add spans

### The big JVM vendors are building best-in-class agent frameworks

Slide 16: /26-j-fall/#/16

Spring AI
  Koog

> Speaker notes: - Not a hobby scene anymore - [click] Spring AI - Spring team, Broadcom - [click] Koog - JetBrains, Kotlin first - [click] Embabel - Rod Johnson, creator of Spring - The three for the rest of the talk

### Most AI projects fail because they automate the wrong thing

Slide 17: /26-j-fall/#/17

replacing human interaction

    boring, repetitive, time-consuming processes

> Speaker notes: - Most AI projects fail - Automating human interaction = usually wrong - chatbots replacing support, sales, advisors - [click] Boring, repetitive, time-consuming processes - Legacy processes already in Java - Need all six tiles from the start

## The demo

Slide 18: /26-j-fall/#/18

#### one boring business process, reviewed by three agents

> Speaker notes: - One demo, three frameworks - Who owns the control flow? - Spring AI - my own loop in Kotlin - Embabel - declared steps, planner finds the path - Koog - my graph, Koog walks it - Domain stays the same, only the reviewer changes - Travel expense approval - everyone has it, nobody likes it

### A trip is a plain JPA aggregate with its bookings

Slide 19: /26-j-fall/#/19

context

```kotlin
@Entity
class Trip(
    var traveler: String,
    var destination: String,
    var startDate: LocalDate,
    var status: TripStatus = TripStatus.DRAFT,
    @OneToMany(cascade = [CascadeType.ALL])
    val hotels: MutableSet<HotelBooking> = linkedSetOf(),
    @OneToMany(cascade = [CascadeType.ALL])
    val flights: MutableSet<FlightBooking> = linkedSetOf(),
    @OneToMany(cascade = [CascadeType.ALL])
    val trains: MutableSet<TrainBooking> = linkedSetOf(),
)
```

> Speaker notes: - Boring Spring Boot 4 service - Spring Data JPA, REST, H2 - Hotels, flights, trains - prices, dates, classes - facts to check - Trimmed: purpose, origin, end date, submit timestamp - Context tile - data already lives here

### The rules are sentences written for people, not for code

Slide 20: /26-j-fall/#/20

context

```kotlin
Policy("Hotel budget",
    "Hotels may cost at most 180 EUR per night. " +
    "In London, Paris and Zurich the limit is 250 EUR per night.")

Policy("Rail first",
    "If the train journey takes 4 hours or less, the train must be used " +
    "instead of a flight. Second class only.")
```

> Speaker notes: - Five policies: hotel budget, flight class, rail first, lead time, weekend stays - Why a model - prose by HR, full of exceptions - if-else for all = boring work - Policies = table rows - POST /api/policies at runtime, e.g. car rental - next review picks it up, no redeploy - Confluence rules checked by hand

### Every agent reads the same facts through one read-only service

Slide 21: /26-j-fall/#/21

integration

```kotlin
@Service
@Transactional(readOnly = true)
class TravelFacts(private val trips: TripRepository, private val policies: PolicyRepository) {
    fun trip(tripId: Long): TripSnapshot
    fun policies(): List<PolicyText>
    fun costs(tripId: Long): CostBreakdown
}
```

> Speaker notes: - Bridge domain - agents - each framework exposes these as tools - logic here, not in framework code - readOnly = true - agent looks, never changes - job writes the result, not the model - Snapshots, not entities - no lazy loading, no Hibernate proxies in prompts - Costs computed in code - models bad at arithmetic - Bodies omitted - plain mapping

### Three agents, one contract

Slide 22: /26-j-fall/#/22

type safety

```kotlin
@Serializable
data class ReviewOutcome(
    val verdict: Verdict,
    val summary: String,
    val violations: List<String> = emptyList(),
)

interface TripReviewer {
    val agent: ReviewAgent
    fun review(tripId: Long): ReviewOutcome
}
```

> Speaker notes: - Same ReviewOutcome for every framework - one implementation each - verdict, summary, violations - a type, not a chat message - Verdict enum: APPROVED, NEEDS_CHANGES, REJECTED - maps to trip status - Red line: how does each framework get to this type?

### Choosing the framework is one request parameter

Slide 23: /26-j-fall/#/23

reliability

```http
POST /api/trips/2/submit?agent=EMBABEL
```

```kotlin
@Async
@TransactionalEventListener
fun on(event: TripSubmitted) {
    val review = reviewers.getValue(event.agent).review(event.tripId)
    reviews.save(TripReview(event.tripId, event.agent, review))
}
```

> Speaker notes: - Submit publishes TripSubmitted, returns 202 - Review after commit, on another thread - Async + TransactionalEventListener - plain Spring - no DB transaction held while waiting on the model - getValue(event.agent) - all TripReviewer beans injected, picked by parameter - SPRING_AI, EMBABEL, KOOG - Simplified: IN_REVIEW, duration, REVIEW_FAILED - Seeded trips - trip 1: Zagreb - Ljubljana by train, compliant - trip 2: Vienna - Munich, business class 1h flight, 520 EUR hotel, 5 days ahead - trip 2 in the demo

### Every reviewer drafts, a judge audits, a rejection goes back

Slide 24: /26-j-fall/#/24

reviewer with tools
  draft
  judge without tools
  accepted
  ReviewOutcome
  feedback

> Speaker notes: - Judge pattern - all three use it - reviewer with tools writes a draft - independent call audits it against the facts - Why - LLMs confident when wrong - fresh context catches many mistakes - cheap insurance - reliability tile - [click] Rejection - feedback back to the reviewer - max two revisions - Same pattern, three different expressions

## Spring AI

Slide 25: /26-j-fall/#/25

#### the ChatClient you already know how to wire

> Speaker notes: - Control flow: I own it - plain Kotlin loop

### One builder call configures the reviewer once

Slide 26: /26-j-fall/#/26

```kotlin
private val reviewer: ChatClient = builder.clone()
    .defaultSystem(Prompts.REVIEWER)
    .defaultTools(tools)
    .defaultAdvisors(SimpleLoggerAdvisor(), TimingAdvisor())
    .build()
```

> Speaker notes: - Auto-configured ChatClient.Builder, injected - clone() - judge builds its own client - Three defaults - system prompt - tools - next slide - advisors - interceptors around every call - Nothing sent yet - config, like RestClient - SpringAiTripReviewer.kt

### A tool is a Spring bean method with a description

Slide 27: /26-j-fall/#/27

integration

```kotlin
@Component
class SpringAiTravelTools(private val facts: TravelFacts) {

    @Tool(description = "Calculate hotel nights and total costs of a trip")
    fun calculateCosts(
        @ToolParam(description = "The trip id") tripId: Long,
    ): CostBreakdown = facts.costs(tripId)
}
```

model
  tool call / result
  our bean

> Speaker notes: - Also getTrip, listPolicies - JSON Schema from the method signature - description + ToolParam -> schema - CostBreakdown -> JSON - [click] Model never runs our code - asks for a tool by name + args - Spring AI calls the method, returns the result - 2.1: loop is an advisor - ToolCallingAdvisor - Integration for free - normal bean - transactions, security, repositories - TravelFacts shared with the REST API

### The request reads like the question and returns a Kotlin type

Slide 28: /26-j-fall/#/28

type safety

```kotlin
var draft = reviewer.prompt()
    .user(Prompts.review(tripId))
    .call()
    .entity<ReviewOutcome>()
```

> Speaker notes: - prompt, user, call - reads like the question - entity<ReviewOutcome> - schema from the class - Jackson back into the data class - Jackson slide, as one call - Malformed answer fails at the boundary - var - judge may replace the draft

### Every call passes through a chain of advisors before it reaches the model

Slide 29: /26-j-fall/#/29

SimpleLoggerAdvisor

  TimingAdvisor

  ToolCallingAdvisor

  before advice
  after advice

  Prompt

  Response

  ChatModel
  LLM provider

    tool call
    result

    our application

    @Tool beans

    TravelFacts

    REST API

> Speaker notes: - Picture behind the builder - Round trip - prompt in - before advice in every advisor - ChatModel outside the chain - remote provider - response out - after advice - nested like servlet filters - Outer two: SimpleLoggerAdvisor, TimingAdvisor - order from getOrder, not builder position - Inner: ToolCallingAdvisor - since 2.1 the tool loop = just an advisor - [click] Tool call into our application - tools = domain logic, not AI stack - @Tool beans -> TravelFacts, same as REST API - loops until final answer - TimingAdvisor measures the whole loop

### An advisor wraps every call like a servlet filter

Slide 30: /26-j-fall/#/30

observability

```kotlin
class TimingAdvisor : CallAdvisor {
    override fun adviseCall(
        request: ChatClientRequest,
        chain: CallAdvisorChain,
    ): ChatClientResponse {
        val started = System.currentTimeMillis()
        val response = chain.nextCall(request)
        val usage = response.chatResponse()?.metadata?.usage
        log.info("took {} ms, tokens: {}",
            System.currentTimeMillis() - started, usage?.totalTokens)
        return response
    }
}
```

> Speaker notes: - Servlet filter, interceptor, aspect - before, chain, after - chain.nextCall - everything downstream incl. the model - Real uses: chat memory, RAG, guardrails, logging, tool loop - SimpleLoggerAdvisor - same shape - Log line shortened

### Every review shows up as a trace without extra code

Slide 31: /26-j-fall/#/31

observability

  spring.ai.chat.client
  spring.ai.advisor
  gen_ai.client.operation
  spring.ai.tool getTrip
  spring.ai.tool listPolicies
  gen_ai.client.operation

> Speaker notes: - Micrometer observations out of the box - ChatClient span, one per advisor - [click] Model call - gen_ai.client.operation - OTel GenAI conventions, model name, tokens - [click] getTrip - [click] listPolicies - our methods, fast and cheap - [click] Second model call - the draft - Time spent waiting on the model - virtual threads - gen_ai.client.token.usage metric -> Prometheus, Grafana - Shape real, timings illustrative

### Chat memory is one more advisor, the history lives in a table

Slide 32: /26-j-fall/#/32

context

```kotlin
val memory = MessageWindowChatMemory.builder()
    .chatMemoryRepository(jdbcChatMemoryRepository)
    .maxMessages(20)
    .build()

chatClient.prompt()
    .advisors(MessageChatMemoryAdvisor.builder(memory).build())
    .advisors { it.param(ChatMemory.CONVERSATION_ID, "trip-$tripId") }
    .user("Why was my trip rejected?")
    .call()
    .content()
```

> Speaker notes: - Not in the demo - first traveler question: why? - Advisor, same before / after shape - load conversation before, append after - CONVERSATION_ID - one per trip - MessageWindowChatMemory - last 20 messages - system messages kept - compression = window or own ChatMemory - Pluggable repository - in memory default, JDBC: Postgres, MySQL, H2 - Cassandra, Neo4j, Mongo starters - Long term: VectorStoreChatMemoryAdvisor

### Spring AI already ships a judge: the `FactCheckingEvaluator`

Slide 33: /26-j-fall/#/33

reliability

```kotlin
val judge: Evaluator = FactCheckingEvaluator.builder()
    .chatClientBuilder(builder)
    .build()

val evaluation = judge.evaluate(
    EvaluationRequest(tripFacts.map(::Document), draft.toString()))

if (!evaluation.isPass) revise(draft)
```

> Speaker notes: - LLM as judge built in - Evaluator interface - testing chapter of the docs, not test only - Context docs + response under test - trip facts as Documents, draft as response - isPass - evaluator decides, code reacts - Two built in - FactCheckingEvaluator - claims backed by docs, hallucination check - RelevancyEvaluator - fits question + context, RAG check - Not enough for us - yes / no only, no feedback - checks facts, not policies - Illustration - demo uses its own evaluator

### The judge implements Spring AI's own `Evaluator` contract

Slide 34: /26-j-fall/#/34

reliability

```kotlin
@Component
class TripReviewEvaluator(builder: ChatClient.Builder) : Evaluator {
    private val judge = builder.clone()
        .defaultSystem(Prompts.JUDGE)
        .build()

    override fun evaluate(request: EvaluationRequest): EvaluationResponse {
        val judgement = judge.prompt()
            .user(Prompts.judge(request.dataList, request.responseContent))
            .call()
            .entity<Judgement>()
        return EvaluationResponse(
            judgement.accepted, judgement.feedback, emptyMap())
    }
}
```

> Speaker notes: - Same interface, my prompt, my types - EvaluationRequest in, EvaluationResponse out - dataList: trip, costs, policies as Documents - responseContent: the draft - response: pass + feedback - Inside: ChatClient without tools, typed Judgement - user line shortened - Why - reviewer depends on Evaluator, not a prompt - swappable judge - reusable in integration tests on recorded reviews - TripReviewEvaluator.kt

### The whole agent loop is a Kotlin `repeat`

Slide 35: /26-j-fall/#/35

reliability

```kotlin
override fun review(tripId: Long): ReviewOutcome {
    var draft = reviewer.prompt()
        .user(Prompts.review(tripId))
        .call()
        .entity<ReviewOutcome>()

    repeat(MAX_REVISIONS) {
        val evaluation = judge.evaluate(evaluationRequest(tripId, draft))
        if (evaluation.isPass) return draft
        draft = reviewer.prompt()
            .user(Prompts.revise(tripId, draft, evaluation.feedback))
            .call()
            .entity<ReviewOutcome>()
    }
    return draft
}
```

> Speaker notes: - No agent abstraction - ordinary Kotlin, readable by everyone - [click] Judge - injected Evaluator - facts as documents, draft as response - isPass - return the draft - [click] Failed - one more reviewer call with feedback - MAX_REVISIONS = 2 - no runaway token burn - [click] What you see is what runs - debugger, unit tests, stack traces - tests fake only the ChatModel - Price: every branch and loop is mine as the flow grows

## Embabel

Slide 36: /26-j-fall/#/36

#### declare the steps, let the planner find the path

> Speaker notes: - Rod Johnson, creator of Spring - Kotlin, works from Java, on top of Spring AI - Spring AI : Embabel = Servlet API : Spring MVC - Control flow: nobody writes it down - typed actions + goals, planner derives the order - Same TravelFacts, ReviewOutcome, judge - Changes how the 2nd and 3rd agent get built

### Embabel plans like a game AI: actions, conditions, goals

Slide 37: /26-j-fall/#/37

@Action methods
  types
  GOAP planner
  plan
  @AchievesGoal
  replan

> Speaker notes: - GOAP - goal oriented action planning - from games - F.E.A.R. enemies - preconditions, effects, search to the goal - Preconditions / effects mostly types - TripDossier action needs a TripDossier first - named conditions where types are not enough - Planner = algorithm, not LLM - deterministic, explainable, logged - [click] Replan after every action - unexpected effect - another route - Cost and value 0 - 1 per action - cheaper route preferred - Newer: Utility, Supervisor planners, @State machines

### A* walks the cheapest chain of actions to the goal and replans when the world changes

Slide 38: /26-j-fall/#/38

= 1 }">
    = 1 ? 'url(#goap-head-on)' : 'url(#goap-head)'" />

    loadDossier · 1

  = 1, 'is-off': $clicks >= 2 }">

    draftWithTools · 8

  = 2 }">
    = 2 ? 'url(#goap-head-on)' : 'url(#goap-head)'" />

    draftReview · 3

  = 3 }">
    = 3 ? 'url(#goap-head-on)' : 'url(#goap-head)'" />

    judgeDraft · 3

  = 4, 'is-off': $clicks >= 5 }">

    acceptDraft · 0
    {{ $clicks >= 5 ? 'pre DRAFT_ACCEPTED = false' : 'pre DRAFT_ACCEPTED' }}

  = 5 }">
    = 5 ? 'url(#goap-head-on)' : 'url(#goap-head)'" />

    reviseDraft · 3
    pre DRAFT_REJECTED

    TripReviewRequest

  = 1 }">

    TripDossier

  = 2 }">

    ReviewOutcome

  = 3 }">

    Judgement

  = 4 }">

    EmbabelReview

  = 4 }" x="1265" y="480" text-anchor="middle">{{ $clicks >= 5 ? 'replanned · cost 10' : 'plan · cost 7' }}

    world state = pre / post

    action · cost

> Speaker notes: - The graph the planner searches - Boxes = world states, types on the blackboard - Arrows = actions, pre -> post - Numbers = cost - draftWithTools - illustrative - Spring AI / Koog way: model fetches via tools - expensive: 8 - [click] Start at the request, two ways out - loadDossier 1 vs draftWithTools 8 - cheap one first - [click] Via dossier to ReviewOutcome: 1 + 3 = 4 - cheaper than 8, tool route dropped - arithmetic, not vibes - [click] judgeDraft -> Judgement - [click] acceptDraft free - plan total 7 - found before any action runs - deterministic, logged - [click] Judge rejects - DRAFT_ACCEPTED false, acceptDraft impossible - replan: reviseDraft, total 10 - No hand-written if - conditions decide the edges

### The caller names an input and a goal type, never the steps

Slide 39: /26-j-fall/#/39

type safety

```kotlin
data class TripReviewRequest(val tripId: Long)
data class TripDossier(val trip: TripSnapshot, val costs: CostBreakdown)
data class EmbabelReview(val outcome: ReviewOutcome, val revised: Boolean)
```

```kotlin
AgentInvocation.create<EmbabelReview>(agentPlatform)
    .invoke(TripReviewRequest(tripId))
    .outcome
```

> Speaker notes: - Three data classes - agent vocabulary - typed blackboard - actions read inputs, write results - Invocation: TripReviewRequest in, EmbabelReview wanted - no step names, no graph, no loop - .outcome unwraps ReviewOutcome for TripReviewer

### An `@Action` can be plain code that never calls a model

Slide 40: /26-j-fall/#/40

```kotlin
@Action
fun loadDossier(request: TripReviewRequest): TripDossier =
    TripDossier(facts.trip(request.tripId), facts.costs(request.tripId))
```

> Speaker notes: - First plan step, no LLM - TripReviewRequest in, TripDossier out - enough for the planner - Loading data: deterministic, fast, free - Spring AI / Koog: model fetches via tool - here: code fetches upfront, model gets the full dossier - Agent inside the system - just call the service

### A model-backed action asks for an object, not a string

Slide 41: /26-j-fall/#/41

type safety

```kotlin
@Action
fun draftReview(dossier: TripDossier, context: OperationContext): ReviewOutcome =
    context.ai()
        .withDefaultLlm()
        .withSystemPrompt(Prompts.REVIEWER)
        .createObject("Review this trip: $dossier")
```

> Speaker notes: - TripDossier in, ReviewOutcome out - runs after loadDossier - createObject - return type defines the output - Prompt runner resembles ChatClient - not a coincidence - Tools line omitted - next slide

### Tools are granted per model call, not per agent

Slide 42: /26-j-fall/#/42

integration

```kotlin
@Component
class EmbabelTravelTools(private val facts: TravelFacts) {
    @LlmTool(description = "List all company travel policies")
    fun listPolicies(): List<PolicyText> = facts.policies()
}
```

```kotlin
context.ai().withDefaultLlm().withToolObject(tools)  // draftReview
context.ai().withDefaultLlm()                        // judgeDraft
```

> Speaker notes: - Tool looks like everywhere else - bean, method, description, TravelFacts - schema from the signature - Per call, not per agent - reviewer gets tools, judge does not - Why - least privilege by default - fewer tools - fewer wrong calls, fewer tokens - Trip + costs in the dossier - only listPolicies needed

### A domain object can bring its own tools

Slide 43: /26-j-fall/#/43

context

```kotlin
data class TripDossier(val trip: TripSnapshot, val costs: CostBreakdown) {

    @LlmTool(description = "Hotels above a nightly limit in EUR")
    fun hotelsAbove(limit: Double): List<HotelLine> =
        trip.hotels.filter { it.nightlyRate > limit }
}
```

```kotlin
context.ai().withDefaultLlm().withToolObject(dossier)
```

> Speaker notes: - Not in the demo - Domain integrated context engineering - behavior on the domain object, callable while in play - no mirror tools class - hotelsAbove - precise question, exact answer - no arithmetic or filtering by the model - Rod's warning - only safe behavior - models get trigger happy - mutations belong in code actions

### A side effect waits for a type only a human can produce

Slide 44: /26-j-fall/#/44

reliability

```kotlin
@Action
fun askTraveler(proposal: RebookingProposal): ApprovedRebooking =
    WaitFor.confirmation(ApprovedRebooking(proposal), "Rebook ${proposal.hotel}?")

@Action
fun rebook(approved: ApprovedRebooking): HotelBooking =
    bookings.rebook(approved.proposal)
```

> Speaker notes: - Not in the demo - the fixing agent - No undo registry - prevent instead of compensate - rebook needs ApprovedRebooking - only producer: human confirmation - type = permission, model cannot skip it - WaitFor parks the process in WAITING - embabel-agent-starter-jdbc: snapshot in the DB - restart or other node resumes, no waiting thread - @Action knobs: readOnly, canRerun

### The memory is the typed blackboard, not a growing chat transcript

Slide 45: /26-j-fall/#/45

context

```kotlin
context.ai().withDefaultLlm().withToolObject(tools)
    .withToolLoopTransformers(
        SlidingWindowTransformer(20),
        ToolResultTruncatingTransformer(4_000),
    )
```

> Speaker notes: - Embabel docs quote - "does not maintain a conversation thread to do its core work" - "context compression is not required for most tasks" - Fresh prompt per action from blackboard objects - judge sees TripDossier + ReviewOutcome, not 40 messages - blackboard = working memory = my data classes - Transformers - for long tool loops - sliding window keeps the system message - truncated tool results - own transformer for summarizing - Chatbots: explicit Conversation, chat store module

### Spring AI traces the model calls, Embabel adds the plan around them

Slide 46: /26-j-fall/#/46

observability

  agent TripReviewAgent
  action loadDossier
  action draftReview
  gen_ai.client.operation
  tool listPolicies
  gen_ai.client.operation
  action judgeDraft
  gen_ai.client.operation
  action acceptDraft

> Speaker notes: - Spring AI metrics carry over - Zero config - every model call = ChatModel observation - model, tokens, latency, token usage metric - [click] embabel-agent-starter-observability - agent span, action spans, tools, replanning - which step burned time and tokens - @Tracked nests own methods - MDC: run id, agent, action - Exporters: Zipkin, OTLP, Langfuse - Labels simplified, timings illustrative

### The judge declares both outcomes, conditions decide which one is true

Slide 47: /26-j-fall/#/47

```kotlin
@Action(post = [DRAFT_ACCEPTED, DRAFT_REJECTED])
fun judgeDraft(dossier: TripDossier, draft: ReviewOutcome, ...): Judgement

@Condition(name = DRAFT_ACCEPTED)
fun draftAccepted(judgement: Judgement): Boolean = judgement.accepted

@Condition(name = DRAFT_REJECTED)
fun draftRejected(judgement: Judgement): Boolean = !judgement.accepted
```

> Speaker notes: - Types cannot express accepted vs rejected - both Judgement - post: one of these may change after judging - every goal condition must be some action's post - learned the hard way building the demo - [click] Plain predicates on Judgement - exclusive by construction - side-effect free, unit tested - Judge body: createObject without tools, omitted

### Two goals finish the review, the true condition picks one

Slide 48: /26-j-fall/#/48

reliability

```kotlin
@AchievesGoal(description = "The trip review was confirmed by the judge")
@Action(pre = [DRAFT_ACCEPTED])
fun acceptDraft(draft: ReviewOutcome, judgement: Judgement): EmbabelReview =
    EmbabelReview(draft, revised = false)

@AchievesGoal(description = "The trip review was corrected after the judge's feedback")
@Action(pre = [DRAFT_REJECTED])
fun reviseDraft(dossier: TripDossier, draft: ReviewOutcome, judgement: Judgement, ...) =
    EmbabelReview(/* one more model call with the feedback */, revised = true)
```

> Speaker notes: - Both produce EmbabelReview - the goal type - Accepted: draft as is, no extra call - [click] Rejected: one more reviewer call with feedback - Difference to the others - revises once, no re-judge - Spring AI, Koog loop up to twice - loop possible via @State or a new draft to judge - Descriptions tell platform and other agents what the goal means

### Nobody wrote this order down, the planner derived it from the types

Slide 49: /26-j-fall/#/49

TripReviewRequest
  loadDossier
  TripDossier
  draftReview
  ReviewOutcome

    judgeDraft

    .outcome

    EmbabelReview
    accept | revise
    Judgement

> Speaker notes: - Plan found for our invocation - arrows = @Action, boxes = blackboard types - sequence never written down - [click] Branch = the conditions - accepted - acceptDraft, rejected - reviseDraft - Why compelling - new action, e.g. exchange rates - used without rewiring - refactor a type - compiler flags misfit actions - Trade-off - flow not in one place - read from the planner log - opposite of the Spring AI repeat

### Shared actions live in a component, not inside one agent

Slide 50: /26-j-fall/#/50

type safety

```kotlin
@EmbabelComponent
class TripActions(val facts: TravelFacts, val rates: ExchangeRates) {

    @Action fun loadDossier(request: TripReviewRequest): TripDossier = ...

    @Action(readOnly = true)
    fun inEuro(dossier: TripDossier): EuroCosts = rates.convert(dossier.costs)
}
```

> Speaker notes: - Not in the demo - how I would grow it - loadDossier moved into a component - New: inEuro for foreign hotel prices - Component is not an agent - contributes actions, goals, conditions - Spring stereotype, normal injection - Only types - any agent needing a TripDossier can plan with it - no calls by name - Utility planner: most valuable next action across the platform - Every id -> rich object method = an action waiting to happen

### A whole agent becomes one step of the next agent

Slide 51: /26-j-fall/#/51

```kotlin
@Agent(description = "Settles expense claims after a trip")
class ExpenseClaimAgent(private val tripReview: TripReviewAgent) {

    @Action
    fun reviewTrip(claim: ExpenseClaim): EmbabelReview =
        RunSubagent.fromAnnotatedInstance(tripReview, EmbabelReview::class.java)

    @AchievesGoal(description = "The expense claim was settled")
    @Action
    fun settle(claim: ExpenseClaim, review: EmbabelReview): Settlement = TODO()
}
```

> Speaker notes: - Not in the demo - next agent for the travel team - Agents are beans - inject one into another - RunSubagent - full trip review = one action - result = just another blackboard type - New agent adds only settling - Scales out: goals via MCP or A2A - Plumbing simplified - claim must provide TripReviewRequest

### Every new agent needs fewer new actions

Slide 52: /26-j-fall/#/52

trip review
    loadDossier
    draftReview
    judgeDraft
    accept | revise

    expense claim
    loadDossier
    trip review
    matchReceipts
    settle

    visa check
    loadDossier
    trip review
    checkVisa

> Speaker notes: - The Embabel takeaway - First agent: four new actions - solid = new, outlined = reused - [click] Second: two of four reused - dossier + trip review as subagent - [click] Third, visa check: one new action - planner wires an unwritten flow - Why Embabel - connected by types, not drawn edges or loops - Spring AI: copy the loop, graph: redraw edges - Agents 2 and 3 illustrative

## Koog

Slide 53: /26-j-fall/#/53

#### the agent is a graph you can read

> Speaker notes: - JetBrains, Kotlin first, Java API - Born in JetBrains products, open source 2025, 1.x stable - JVM, Android, iOS, JS, Wasm - Control flow: I draw the graph, Koog walks it - opposite of Embabel - every edge written down - Same demo, ReviewOutcome, judge - Graph by hand first, then library subgraphs - LangGraph, but typed

### The same review becomes a graph of nodes and edges

Slide 54: /26-j-fall/#/54

tool call

    tool call

    text

    text

    accepted

    rejected

    start

    review

    run-tools

    send-tool-results

    structure-draft

    judge

    finish

> Speaker notes: - Every box = one line of Kotlin - Left to right - review asks the model - tool call - run-tools - send-tool-results - repeats until a text answer - structure-draft -> ReviewOutcome - judge accepts or sends back - Two cycles - [click] tool loop - Spring AI: hidden in an advisor - [click] revision loop - Spring AI: a repeat - here: visible edges - Long form first - shows what subgraphs do later

### Tools live in a `ToolRegistry` the agent is built with

Slide 55: /26-j-fall/#/55

integration

```kotlin
class KoogTravelTools(private val facts: TravelFacts) : ToolSet {
    @Tool
    @LLMDescription("Load a submitted trip with all bookings")
    fun getTrip(@LLMDescription("The trip id") tripId: Long): TripSnapshot =
        facts.trip(tripId)
}
```

```kotlin
val tools = ToolRegistry {
    tools(KoogTravelTools(facts))
}

AIAgent(promptExecutor = executor, strategy = strategy,
    agentConfig = config, toolRegistry = tools)
```

> Speaker notes: - Like @Tool, @LlmTool - methods + descriptions - ToolSet groups them, also listPolicies, calculateCosts - same TravelFacts - Registry = everything the agent may call - Kotlin DSL, same style as strategy + prompt - Koog executes, model only asks - combinable with plus - shared registries - Alternatives - class based tools, typed Args - built-in tools: talk to user, end run

### Every node is a typed value declared up front

Slide 56: /26-j-fall/#/56

type safety

```kotlin
strategy<String, ReviewOutcome>("trip-review") {
    var revisions = 0

    val review by nodeLLMRequest("review")
    val runTools by nodeExecuteTools("run-tools")
    val sendToolResults by nodeLLMSendToolResults("send-tool-results")
    val draft by nodeLLMRequestStructured<ReviewOutcome>("structure-draft")
    // edges on the next slide
}
```

> Speaker notes: - strategy<String, ReviewOutcome> - prompt -> our type - Four library nodes - LLM request, execute tools, send results, structured request - Typed input / output per node - mismatched types do not compile - revisions - local var, captured by the strategy - Judge later, as a subgraph

### Four edges are the whole tool loop

Slide 57: /26-j-fall/#/57

```kotlin
edge(nodeStart forwardTo review)
edge(review forwardTo runTools onToolCalls { true })
edge(runTools forwardTo sendToolResults)
edge(sendToolResults forwardTo runTools onToolCalls { true })

edge(review forwardTo draft onTextMessage { true } transformed { STRUCTURE })
edge(sendToolResults forwardTo draft onTextMessage { true } transformed { STRUCTURE })
```

> Speaker notes: - First four lines = tool loop - onToolCalls - model asked for a tool - results back to the model, may ask again - [click] Done researching - text answer - structure-draft - transformed: "return final review as structured output" - parsed into ReviewOutcome - Control flow as data - trace, checkpoint, visualize - Every agent needs this loop

### Koog ships the whole tool loop as one `subgraphWithTask`

Slide 58: /26-j-fall/#/58

```kotlin
val review by nodeLLMRequest("review")
val runTools by nodeExecuteTools("run-tools")
val sendToolResults by nodeLLMSendToolResults("send-tool-results")
val draft by nodeLLMRequestStructured<ReviewOutcome>("structure-draft")

edge(review forwardTo runTools onToolCalls { true })
edge(runTools forwardTo sendToolResults)
edge(sendToolResults forwardTo runTools onToolCalls { true })
edge(review forwardTo draft onTextMessage { true } transformed { STRUCTURE })
edge(sendToolResults forwardTo draft onTextMessage { true } transformed { STRUCTURE })
```

```kotlin
val review by subgraphWithTask<String, ReviewOutcome>(name = "review") { task ->
    task
}
```

> Speaker notes: - What we just built: four nodes, five edges - [click] What the demo ships - same loop as a library subgraph - String in, ReviewOutcome out - typed node - lambda: input -> task, here input = task - Inside - finish tool finalize_task_result with ReviewOutcome schema - forced tool calls until the finish tool - finish tool call = structured output, no extra request - Principle moved into the library - Gotcha: inner nodes count against maxAgentIterations - raised to 200

### The judge is a `subgraphWithVerification` that never sees the reviewer's chat

Slide 59: /26-j-fall/#/59

type safety

```kotlin
val judge by subgraph<ReviewOutcome, CriticResult<ReviewOutcome>>(
    "judge", freshHistory = true,
) {
    val verify by subgraphWithVerification<ReviewOutcome>(tools = emptyList()) { draft ->
        llm.writeSession {
            clearHistory()
            appendPrompt { system(Prompts.JUDGE) }
        }
        Prompts.judge(facts.trip(tripId), facts.policies(), facts.costs(tripId), draft)
    }
    nodeStart then verify then nodeFinish
}
```

> Speaker notes: - Judge pattern shipped: subgraphWithVerification - draft in, CriticResult out - successful, feedback, draft - same finish tool trick, verdict instead of review - freshHistory = true - default: runs inside the reviewer's chat - fresh: empty chat, reviewer's chat restored afterwards - [click] No tools, judge prompt, facts + draft - independent opinion, no self-agreement - subgraphs nest - still just a node - suspend everywhere - coroutines all the way

### The revision loop stays a plain node and two edges

Slide 60: /26-j-fall/#/60

reliability

```kotlin
val revise by node<CriticResult<ReviewOutcome>, String>("revise") { critique ->
    revisions++
    Prompts.revise(tripId, critique.input, critique.feedback)
}

nodeStart then review then judge
edge(judge forwardTo nodeFinish
    onCondition { it.successful || revisions >= MAX_REVISIONS } transformed { it.input })
edge(judge forwardTo revise onCondition { !it.successful })
revise then review
```

> Speaker notes: - Revision policy is mine, not library - plain node: verdict -> next reviewer prompt - [click] then = unconditional edge - [click] Finish on accept or MAX_REVISIONS (2) - else revise, back into review - reviewer keeps its chat - knows its last draft - Gotcha: at the cap both conditions true - first matching edge wins - order matters - Safety net: maxAgentIterations = 200 - Lines wrapped, KoogTripReviewer.kt

### Two library subgraphs and one node of mine are the whole agent

Slide 61: /26-j-fall/#/61

subgraphWithTask

    subgraphWithVerification

    tool calls

    accepted

    rejected

    start

    review

    tools

    judge

  freshHistory = true

    revise

    finish

    Koog library subgraph

    my node and edges

> Speaker notes: - Same agent as the first map - same behavior, same tests - Tool loop + structuring = one subgraph - Judge = subgraph with fresh history - revise + its edges = my policy - Library subgraphs first, single nodes where control is needed

### Koog runs on the same Spring AI `ChatModel`

Slide 62: /26-j-fall/#/62

integration

```kotlin
@Bean
fun koogPromptExecutor(chatModel: ChatModel): PromptExecutor = MultiLLMPromptExecutor(
    SpringAiLLMClient.builder()
        .chatModel(chatModel)
        .provider(LLMProvider.OpenAI)
        .build()
)
```

AIAgent

  SpringAiLLMClient

  ChatModel

> Speaker notes: - Koog clients: OpenAI, Anthropic, Google, Ollama, ... - Spring shop: no second credentials, HTTP clients - koog-spring-ai wraps Boot's ChatModel - [click] Koog: graph + tools, Spring AI: model calls only - Gotcha: auto-config missed the Spring AI 2.1 Responses API model - one bean wired by hand, normally the starter - No response schema forwarding - JSON format in the prompt

### Spring AI traces the model calls, Koog adds the graph around them

Slide 63: /26-j-fall/#/63

observability

  agent trip-review
  subgraph review
  gen_ai.client.operation
  tool getTrip
  gen_ai.client.operation
  subgraph judge
  gen_ai.client.operation

> Speaker notes: - Same picture as Embabel - on purpose - Free via the bridge - ChatModel instrumented with Micrometer - model, latency, tokens, token usage metric - Spring AI blind spots - Koog runs tools - no Spring AI tool span - no idea which node made the call - [click] Koog OpenTelemetry feature - spans for agent run, subgraphs, nodes, tools - subgraphs still visible in the trace - OTel GenAI semantic conventions - Labels simplified, timings illustrative

### One `install` sends the whole graph to Langfuse

Slide 64: /26-j-fall/#/64

observability

```kotlin
AIAgent(
    promptExecutor = executor,
    strategy = reviewStrategy(tripId),
    agentConfig = agentConfig,
    toolRegistry = tools,
) {
    install(OpenTelemetry) {
        setServiceInfo("trip-review", "1.0.0")
        addLangfuseExporter()
    }
}
```

> Speaker notes: - Features installed like Ktor plugins - OpenTelemetry, tracing, event handlers, persistence, chat memory - addLangfuseExporter - ships with Koog - LANGFUSE env vars: host, keys - prompts, responses, tokens, cost per node - also W&B Weave, Datadog, any OTLP - Prompts masked unless verbose - Both OpenTelemetry - same backend as the submitting REST call - Not in the demo repo - first line before production

### Chat memory is a feature I install, compression is a node I place

Slide 65: /26-j-fall/#/65

context

```kotlin
install(ChatMemory) {
    chatHistoryProvider = JdbcChatHistory(dataSource)
    windowSize(20)
}
```

```kotlin
val compress by nodeLLMCompressHistory<ReceivedToolResults>(
    strategy = HistoryCompressionStrategy.FromLastNMessages(5),
)
```

> Speaker notes: - Two problems, two tools - ChatMemory - across runs - load / store history per session id - agent.run(input, sessionId) - "why was my trip rejected?" - JdbcChatHistory mine - ChatHistoryProvider: store + load - windowSize, filterMessages built in - History compression - within one run - a node - I pick where - after tool bursts, before the context fills - strategies: TLDR, last N, chunked, fact retrieval - LongTermMemory - vector store, RAG recall - Review agent needs none - chat agents do

### Every tool can register its undo, a rollback runs them backwards

Slide 66: /26-j-fall/#/66

reliability

```kotlin
install(Persistence) {
    storage = JVMFilePersistenceStorageProvider(checkpoints)
    enableAutomaticPersistence = true
    rollbackToolRegistry = RollbackToolRegistry {
        registerRollback(bookings::addHotel, bookings::removeHotel)
    }
}
```

```kotlin
context.persistence().rollbackToCheckpoint(checkpointId, context)
```

> Speaker notes: - Next agent fixes the trip - swaps the 520 EUR hotel - side effects, bad run = broken state - Persistence checkpoints - automatic: after every node - history, node, storage - crash - resume from the checkpoint - RollbackToolRegistry - unique to Koog - Spring AI: own undo log, Embabel: human approval first - each side-effect tool names its undo tool - rollback: restore state, undo tools in reverse - Undo is my code - Koog guarantees call + order - bookings, checkpoints illustrative - review agent read only

### The graph is one strategy, Koog lets every agent pick its own

Slide 67: /26-j-fall/#/67

graph strategy
  functional loop
  ReAct and chat
  LLM planner
  GOAP planner
  subgraphs

> Speaker notes: - Map only, all documented - Graph + subgraphs - today, highlighted - [click] Functional strategy - suspend lambda, while loop - Spring AI repeat inside Koog - [click] Predefined: single run, ReAct, chat - [click] LLM planners - model writes and revises the plan, optional critic - flexible, only as good as the model - [click] GOAP planner - preconditions, effects, A* - Embabel idea as a Koog strategy - Subgraphs nest - big agents from small tested pieces - Wrap-up - Spring AI: I write the loop - Embabel: planner finds it - Koog: I draw it, or pick any of the above - all three: inside our Java service, same ChatModel, same traces