[sohailshk]

Fixes #41194

message_from_string and message_from_conversation popped max_tokens from the inference_args dict they received. RunInference passes the same dict for every batch, so after the first batch max_tokens was gone and later requests used the default of 1024. Both functions now pop from a copy.

New tests call each function twice with the same dict and check that the dict is unchanged and that both requests use the given max_tokens. They fail on master and pass with the fix (Anthropic client mocked); anthropic_inference_test.py passes.

[github-actions]

📋 Review Summary

This PR resolves a bug in anthropic_inference.py where message_from_string and message_from_conversation mutated the inference_args dictionary in-place via .pop('max_tokens', 1024). By shallow-copying inference_args prior to popping max_tokens, subsequent batch requests properly retain the configured max_tokens parameter.

🔍 General Feedback

• The fix is clean, minimal, and accompanied by targeted unit tests verifying both functions preserve inference_args across repeated calls.
• Note (optional follow-up): AnthropicModelHandler.request() also mutates the incoming inference_args dictionary in-place when injecting handler-level system or output_config defaults (inference_args['system'] = ...). While less destructive than .pop(), shallow-copying inference_args inside AnthropicModelHandler.request as well would fully prevent side effects on caller-supplied dictionaries.

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

[github-actions]

Assigning reviewers:

R: @jrmccluskey for label python.
Backup reviewer: tvalentyn (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[codecov]

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.15%. Comparing base (0a963fc) to head (f4c0221).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
...on/apache_beam/ml/inference/anthropic_inference.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #41196      +/-   ##
============================================
- Coverage     59.25%   59.15%   -0.11%     
  Complexity     4894     4894              
============================================
  Files          1461     1461              
  Lines        186046   186048       +2     
  Branches       3648     3648              
============================================
- Hits         110244   110052     -192     
- Misses        72445    72639     +194     
  Partials       3357     3357              

Flag	Coverage Δ
python	80.54% <0.00%> (-0.24%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jrmccluskey]

No reason to do this extra bit of work copying the dict on every invocation.

Suggested change

	inference_args = dict(inference_args)
	max_tokens = inference_args.pop('max_tokens', 1024)
	max_tokens = inference_args.get('max_tokens', 1024)

[sohailshk]

Thanks! I tried the .get version first, but then max_tokens is still in inference_args, so whenever a user sets it, messages.create(max_tokens=..., **inference_args) raises TypeError: got multiple values for keyword argument 'max_tokens'.

I pushed 3408f45, which drops the copy instead: the 1024 default is passed only when inference_args has no max_tokens, and inference_args is passed through as is. The two new tests fail on master and pass with this; anthropic_inference_test.py passes (25 tests).