[florence-bockting]

Description

This PR is connected to stan-dev/posterior#470 where we introduced warning classes.
These can now be extracted and collected through the new .collect_warnings() function in loo.
This collector ensures that the same warning is printed only once (and not multiple times when e.g. running some posterior function in a loop).

This PR is related to Issue #353 (but it does not solve it entirely).

TODOs

• test added
• News item added

The PR was assisted by an LLM.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.71%. Comparing base (d18de1f) to head (7c3f1f6).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #416      +/-   ##
==========================================
+ Coverage   92.65%   92.71%   +0.06%     
==========================================
  Files          31       31              
  Lines        3240     3268      +28     
==========================================
+ Hits         3002     3030      +28     
  Misses        238      238              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 7c3f1f6 is merged into master:

Status	Name	Current	Incoming	Interval
✔️	loo_matrix	1.42s	1.42s	[-0.68%, +0.36%]
✔️	loo_function	1.32s	1.32s	[-0.65%, +0.56%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[jgabry]

This looks good. I made a few small suggestions.

Also one design question: before this PR users would see something like

There were 20 warnings (use warnings() to see them)"

and now with this PR they just get a single warning with no count. For something like ESS capping that's probably good, but for something like whether 1 observation or 900 observations get a Pareto k warning it could matter. Should we add something like "(n times)" to the message? I'm not sure. Maybe @avehtari and/or @VisruthSK have thoughts on this too.

[florence-bockting]

Should we add something like "(n times)" to the message? I'm not sure. Maybe @avehtari and/or @VisruthSK have thoughts on this too.

I added the count. A warning that occurs more than once now ends with (n times), for example The ESS has been capped to avoid unstable estimates. (3 times). Happy to change this again if desired.

[VisruthSK]

Totally missed this, I like how that looks.