Skip to content

Add summarize-failures action to use on PRs - #41208

Draft
derrickaw wants to merge 9 commits into
masterfrom
20261110_summaryFailures
Draft

derrickaw wants to merge 9 commits into
masterfrom
20261110_summaryFailures

Conversation

@derrickaw

Copy link
Copy Markdown
Collaborator

Also, add /gemini review and /summarize-failures actions to documentation.

@github-actions github-actions Bot added build size/m M: 50-249 lines changed labels Nov 11, 2026
PR_NUMBER: ${{ github.event.issue.number || inputs.pr_number }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
script: |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing all this bespoke context gathering, have you tried just prompting Gemini and having it do the work? Something like:

Review the CI failures in PR XYZ. Summarize the failures and print them in format ...

This is similar to how the review bot works, and I empirically found that this worked better than trying to hydrate the correct context. It is more expensive, but the token count just won't be that high regardless in my experience. I am not sure if that holds here where parsing logs can be quite a pain

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, tried to improve it. Thanks

@derrickaw

Copy link
Copy Markdown
Collaborator Author

/gemini review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR adds a new PR Failure Summary GitHub Actions workflow (pr_failure_summary.yml) triggered via /summarize-failures comments or workflow_dispatch to cluster failing CI checks using Gemini CLI and post a sticky root-cause summary comment. It also updates pr-bot and contributor documentation to surface /summarize-failures and /gemini review.

🔍 General Feedback

  • The security model for the issue_comment trigger is well thought out: PR code is not checked out (ref: 'master' with sparse checkout only for setup-gh-cli-linux), Gemini CLI is restricted to an explicit read-only tool allowlist, and untrusted input is passed via files/env vars rather than interpolated into the shell script.
  • Moving concurrency from the workflow level to the job level in pr_failure_summary.yml is recommended so unrelated PR comments do not cancel an in-progress failure summary run.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build size/m M: 50-249 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants