Repository navigation
Conversation
|
/gemini review |
There was a problem hiding this comment.
📋 Review Summary
This PR resolves flakiness in Bigtable integration tests (bigtableio_it_test.py) caused by google-cloud-bigtable >= 2.44.0 disabling retries on DirectRow.commit() when set_cell() omits an explicit timestamp. By passing datetime.now(timezone.utc) to setup set_cell() calls and asserting row.commit().code == 0, setup mutations become idempotent (enabling automatic retries on transient warmup errors) and fail fast if a commit still does not succeed.
🔍 General Feedback
- The root cause analysis and fix are clean, targeted, and consistent with existing patterns in
bigtableio_it_test.py. - Adding
self.assertEqual(0, ...commit().code)on all pre-test setup writes will significantly improve debuggability if non-retryable or persistent gRPC errors occur during test setup.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #41231 +/- ##
=========================================
Coverage 57.75% 57.76%
Complexity 2617 2617
=========================================
Files 1202 1202
Lines 171679 171679
Branches 2163 2163
=========================================
+ Hits 99159 99166 +7
+ Misses 69862 69855 -7
Partials 2658 2658
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
In
google-cloud-bigtable >= 2.44.0(upgraded in #40042 / #40960),DirectRow.commit()delegates to_DataApiTarget.mutate_row(row_key, mutations), which only retries transientDeadlineExceededandServiceUnavailableerrors whenall(m.is_idempotent() for m in mutations)isTrue.When
DirectRow.set_cell()is called without an explicittimestamp, it setstimestamp_micros = -1(_SERVER_SIDE_TIMESTAMP), causingSetCell.is_idempotent()to returnFalseand disabling retries completely. If the single 20-secondMutateRowattempt times out while a newly created Bigtable instance/table/column family is warming up (as seen inTestWriteToBigtableXlangIT.test_delete_cells_mutationin run 43118590693),DirectRow.commit()catchesGoogleAPICallError, clearsself._mutationsinfinally, and returns a non-OKStatuswithout raising an exception—leaving the table empty and causingactual_row = self.table.read_row('key-1')to returnNone(AttributeError: 'NoneType' object has no attribute 'find_cells').Changes
datetime.now(timezone.utc)timestamp toset_cell()in pre-test setup writes inbigtableio_it_test.pysoSetCell.is_idempotent()isTrueandDirectRow.commit()retries transientDeadlineExceeded/ServiceUnavailableerrors during Bigtable instance/table warmup.row.commit().code == 0on pre-test setup commits so any setup failure surfaces immediately with its gRPC status code.Verification
PostCommit Python Xlang Gcp Dataflow: https://github.com/apache/beam/actions/runs/43152154383