[damccorm]

Summary

Fixes transient org.apache.iceberg.exceptions.ForbiddenException: Exceeded rate limits: too many table update operations for this table failures in BigQueryMetastoreCatalogIT during IcebergIO Integration Tests (e.g., https://github.com/apache/beam/actions/runs/43154192604).

Root Cause

BigQuery enforces a per-table metadata update quota of 5 table operations per 10 seconds (rateLimitExceeded: Exceeded rate limits: too many table update operations for this table, surfaced by BigQueryMetastoreCatalog as org.apache.iceberg.exceptions.ForbiddenException). When a multi-snapshot test such as testStreamingCdcReadMixedDeleteAndOverwriteSnapshots creates a table and commits 4 snapshots (5 table operations) and completes its DirectRunner pipeline within 10 seconds (9.7s in run 43154192604), the 6th table operation (catalog.dropTable(identifier) in catalogCleanup) hits the 10-second rate limit window.

Fix

Override catalogCleanup(List<Namespace>) in BigQueryMetastoreCatalogIT to retry super.catalogCleanup(namespaces) up to 3 times with a 5s backoff when ForbiddenException is thrown, allowing the 10-second per-table rate limit window to clear before dropping the table and namespace.

Verification

• Triggered IcebergIO Integration Tests on this branch: https://github.com/apache/beam/actions/runs/43166035024

[damccorm]

/gemini review

[github-actions]

📋 Review Summary

This PR resolves transient ForbiddenException rate-limit failures during test teardown in BigQueryMetastoreCatalogIT by adding a retry mechanism with linear backoff in catalogCleanup. The implementation is clean, properly handles thread interrupts, and gives BigQuery's 10-second per-table metadata operation quota window sufficient time to reset.

🔍 General Feedback

• Because super.catalogCleanup(namespaces) iterates over catalog.listTables(namespace) and drops each table before dropping the namespace, retrying super.catalogCleanup is safe when a dropTable call hits a rate limit (already-dropped tables will no longer be returned by listTables on subsequent attempts).

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.