Add support for object change notification to storage

[tseaver]

See: https://cloud.google.com/storage/docs/object-change-notification

E.g., we could add a watch() method to Bucket, which would register the notification URL and return an instance of a new Channel class: that instance would have a close() method which would turn off the notifications.

[tseaver]

Supporting OCN is a prerequisite to doing "safe" caching of bucket keys, as suggested in #19.

[tseaver]

@jgeewax can you give us a feel for how important this issue (and #19) are? Should they be blockers for Storage M1?

[dhermes]

FWIW, there are 34 methods in storage and this is one of them. This would target storage.channels.stop and storage.objects.watchAll.

---

Can move this elsewhere, but is a list of the other 32 methods, not all of which we support

Each of the 5 resources

• buckets
• bucketAccessControls
• objects
• objectAccessControls
• defaultObjectAccessControls

have the 6 verbs insert, get, list, update, patch, delete and in addition there is a way to concatenate objects and to copy them

• storage.objects.compose
• storage.objects.copy

[tseaver]

@dhermes, #19 was a request for a way to cache all the blob IDs in a bucket: I have the sense that we cannot "safely" provide that feature without having a way to invalidate the cache, which requires the storage.objects.watchAll API (and therefore storage.channels.stop).

[dhermes]

Yes the "get all keys" / "get all objects" concept is not something that really applies to a mature application (one with more than a few thousand objects). As for the people "just getting started", the two HTTP requests won't be such a big deal. What's more, we shouldn't be encouraging people to get a list of "all objects" locally; they should only use what they need (e.g. for serving a request for a signed-in user).

[jgeewax]

I think this one should probably not be a total blocker for a 1.0 release, but I really like the API that @tseaver suggests. It makes sample code look like:

bucket = storage.get_bucket('...')

# Set up a 'watch' on the bucket (ie, tell it to ping a URL when things happen):
channel = bucket.watch('http://...', channel_id='my-magical-channel')
channel2 = bucket.watch('http://...')

# Stop notifications..
channel.close()
channel2.close()

# Stop all notifications (can we do this?)
# Is this the right name? Maybe bucket.get_notification_channels() ?
channels = bucket.get_channels()
for channel in channels:
  channel.close()

# Get a channel by ID and stop those notifications.
# Maybe bucket.get_notification_channel() ?
channel = bucket.get_channel('my-magical-channel')
channel.close()

[dhermes]

Based on "I think this one should probably not be a total blocker for a 1.0 release" I'm moving Stable -> Future.

[lukesneeringer]

Hello,
One of the challenges of maintaining a large open source project is that sometimes, you can bite off more than you can chew. As the lead maintainer of google-cloud-python, I can definitely say that I have let the issues here pile up.

As part of trying to get things under control (as well as to empower us to provide better customer service in the future), I am declaring a "bankruptcy" of sorts on many of the old issues, especially those likely to have been addressed or made obsolete by more recent updates.

My goal is to close stale issues whose relevance or solution is no longer immediately evident, and which appear to be of lower importance. I believe in good faith that this is one of those issues, but I am scanning quickly and may occasionally be wrong. If this is an issue of high importance, please comment here and we will reconsider. If this is an issue whose solution is trivial, please consider providing a pull request.

Thank you!