[igalshilman]

Summary

A review of what one turn of restate.agents costs in Restate: the commits of the key's database (ctx.db) and the calls to the companion. It makes a turn cheaper in every framework, and fixes the bugs the review confirmed along the way.

Measured on Restate 1.8.1 with Pydantic AI, a warm key, for a turn of one reply with 3 parallel tool calls and then an answer:

	Before	After
Database commits	5	4
Companion calls	9	4
Journal entries of the chat invocation	50	31–33

A plain turn goes from 3 commits and 3 companion calls to 2 and 2 (24 journal entries to 17). A key's first turn makes one schema commit instead of two.

1. A turn's final reply commits with its answer

A turn made 2r + 1 commits (r model requests): its prompt, each reply and each batch of tool results, then the final reply and the answer one after the other, with only memory between them. Each commit is two state writes, a run and a round trip.

• In answer_once's run, a step that no tool waits for (a reply without tool calls, ADK's end-of-agent event) is held in memory. The next write writes it in its own transaction, so the final reply commits with the answer.
• Replies with tool calls still commit before their tools run, and tool results before the model is asked again.
• What is held is written before a publish, before a call record, before the Restate calls of LangGraph's nodes, and when the attempt fails in process.
• A crash between the final reply and the answer now asks the model again once; before, the retry carried on from the stored reply.

Commits per warm turn:

Scenario	Pydantic AI, OpenAI, LangGraph	ADK
Plain answer	3 → 2	4 → 2
One round of tool calls	5 → 4	6 → 4
Two rounds	7 → 6	8 → 6

2. A turn publishes after each step it commits

A followed turn published at its start and at every tool call's start and end: 3 + 2k companion calls (k tool calls). A publish carries nothing; it wakes followers, who read the conversation again.

This fixes a visibility bug. Pydantic AI, OpenAI and ADK store a reply's results together, after the tools end and before the next model request, and no publish followed that commit. A follower saw tool results only when the next tool call started or the turn ended, a model request late. Of the 8 revisions of the turn above, 6 showed a follower nothing new.

• The turn now publishes once after each write of its steps that commits (a reply with tool calls, the tools' results) and waits for it. A follower sees the calls as their tools start, and the results before the model is asked again.
• Steering is unchanged: each integration stores the results, waits for that publish, then takes the messages it returned.
• The companion's start makes the turn's first publish and returns what publish returns (Started.publishes). A companion that returns nothing from start (an earlier deployment, or the TypeScript SDK's) still gets a publish from the turn.
• tool_call_started only forgets the stored call records; tool_call_ended does nothing and is kept for callers of restate.agents.internal.
• The companion no longer clears an empty watching at each publish.
• A loop that commits each tool result on its own publishes once per result.

3. Bugs the review confirmed

• Results lost when a provider reuses a tool call id across turns (Pydantic AI, OpenAI). A result was looked up by its call's id over earlier turns, found there, and never stored. Later turns told the model the call was interrupted (Pydantic AI), or sent a call without output, which a strict provider refuses on every later turn (OpenAI). A result for a call the turn has open is now the turn's own once it comes after the latest call of that id in the run's items.
• A cancellation during a commit left the conversation's positions out of step with the database. The rows were committed but the next position was not advanced, so closing the turn failed on a unique constraint and the turn stayed open. A write and its bookkeeping now finish together; the caller gets the cancellation at once.
• docs/agents/frameworks.md named the wrong Pydantic AI capability as innermost and as the one that stores snapshots.

4. Commits that nothing needed

• A plan read inside a tool call (Pydantic AI) committed a row. Reads are now kept in memory and written with the call's next write; a call that only reads commits nothing.
• The call records' schema was created in a commit of its own on a key's first turn, even when no tool ran. restate.sessions.internal.ensure_schemas applies several schemas in one transaction, and answer_once uses it.

Docs

docs/agents/turns.md, access-patterns.md, learnings.md, conversation.md and frameworks.md say what is true now; porting.md says what the TypeScript port must change (start taking Started.publishes and returning what publish returns).

Tests

• New tests for each change: one commit for the reply and its answer; a crash sweep over the final commit; a held reply stored before a publish and when an attempt fails; what a follower sees right after each publish (Pydantic AI, OpenAI, ADK); publishes per turn; a steered message sent while tools run; a companion whose start returns nothing; reused call ids, with a provider that refuses a call without output; a write cancelled while it commits; a plan read that commits nothing; one schema commit on a key's first turn.
• The full suite passes: 396 in process, 260 against a Restate 1.8.1 server.

Not in this PR

• The TypeScript port.
• Whether the prompt's row should also wait for the turn's first commit: one more commit saved per turn, but a turn killed during its first model request would leave no trace.
• Messages that Pydantic AI capabilities enqueue during a run (Planning, DetectLoops) are still not stored, so a retry and the next turn don't see them. A fix exists, but it changes what the output check and the views see, so it needs a decision first.

🤖 Generated with Claude Code

[github-actions]

Test Results

  8 files  ±0    8 suites  ±0   4m 0s ⏱️ +49s
 71 tests ±0   71 ✅ ±0  0 💤 ±0  0 ❌ ±0 
322 runs  ±0  322 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 7c37cfb. ± Comparison against base commit 48e4133.

♻️ This comment has been updated with latest results.

[pullfrog]

ℹ️ No critical issues. One minor suggestion inline.

Reviewed changes

I reviewed all 29 files of the PR at 5de21bf, covering the held-row ledger, publish-after-commit, the companion's start, shielded writes, the call-id reuse fix, deferred plan reads and ensure_schemas.

• Held steps commit with the next write. In answer_once, steps with no tool call or result keep their positions but stay in memory. The next ledger write, a publish, a call record, a LangGraph node's Restate call or a failing attempt writes them first. The database therefore always holds a prefix of the turn's positions, so a crash can only produce a state the old code could also reach.
• Publishes follow commits. _Turn._stored awaits a publish after each write that commits. tool_call_ended becomes a no-op, and the companion's start publishes and returns Published when the turn passes Started.publishes. Older and TypeScript companions still get a publish from the turn.
• Shielded ledger writes. _insert, answered, close, drop, snapshot, ask, snapshot_between and flush finish once started, and close takes the lock before checking its status. A cancelled write can no longer leave the ledger's positions out of step with the database.
• Reused call ids across turns. new_results treats a result as the turn's own when its id matches an open call and the result comes after the latest call with that id. Pydantic AI now uses it too.
• Deferred plan reads and one schema commit. A plan read inside a tool call is journaled with that call's next write. answer_once applies both schemas in one transaction.

I ran tests/agents_conversation.py, tests/agents_publishes.py and the in-process tests in the Pydantic AI, OpenAI, ADK, stores and LangGraph files locally, and all pass. The Docker-harness tests can't run here. I also checked that id reuse within a single turn isn't a remaining gap: the OpenAI Agents SDK itself rejects it (ModelBehaviorError: Model reused a completed tool call ID).

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[pullfrog]

✅ No new issues found. The batching suggestion from the last review is addressed.

Reviewed changes

I reviewed the one commit pushed since the previous Pullfrog review (5de21bf → 7c37cfb).

• Batched a graph step's tool results. _copied now collects consecutive ToolMessages and stores them with a single turn.append(*results). It writes them before the next AIMessage and again after the loop, so a parallel tool step costs one commit and one publish. The explicit stepped(ctx) call is gone, because Turn._stored already publishes after each write that commits. known is still updated for every copied message.
• Added test_a_steps_tool_messages_are_copied_in_one_write. It checks the write grouping and known, both with and without a final reply after the results. I ran it locally along with the neighbouring checkpoint test, and both pass. ruff is clean.
• Fixed typing for CI. This adds an annotation on open_ids in new_results and moves the ADK imports in tests/agents_publishes.py to their concrete modules. Neither changes behavior.

  ^{｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}