Add support for batch requests in gcloud.storage

[jgeewax]

Or wait for the new primitive client that simplifies this.

[tseaver]

Is there a server-side API feature that this issue refers to?

[silvolu]

https://cloud.google.com/storage/docs/json_api/v1/how-tos/batch

Not sure if apitools helps with this.

[tseaver]

Cool, I had missed that doc.

[silvolu]

It's kinda easy to get lost in the Cloud Encyclopedia :D

[tseaver]

I think we would end up implementing this by subclassing Connection, overriding api_request such that it stored params for non-idempotent HTTP requests (anything other than GET) (similar to the Transaction's mutation in datastore).

I don't know how it should handle interleaving reads: write-after-read can lead to inconsistent state without some help from the server to isolate other changes.

Another approach would be to use a pattern like #352, where the context manager hooks api_request to defer the writes.

[tseaver]

OK, here's a plan:

• Move datastore.batch._Batches (which is just a thread-local stack) up to gcloud._localstack. Move the 'thread-local stack' implementation up from datastore. #622
• Add storage.batch.Batch, paralleling datastore.batch.Batch, and exposing the same methods as storage.connection.Connection. This class will "store up" the mutations and generate the multipart/mixed payload defined in Sending Batch Requests. As with the datastore variant, this Batch should be usable as a context manager, as well as in "imperative" mode (where it will need to be passed into the buckets as their connection?). #15: Add storage.batch.Batch #654
• Update storage.api to look first for the active batch, falling back to the implied default connection.
• Update Bucket.__init__() to look first for the active batch, but fall back (per Should Blob and Bucket constructors fall back to _implicit_environ defaults? #621).

[dhermes]

SGTM.