PROPOSAL: Storage API consistency fix

[dhermes]

Guiding principles:

• Getters and setters should never make HTTP requests. Lazy loading is OK,
  but only when it involves instance creation / other local (non-network bound) behavior. For example, in Bucket.acl this already happens:

@property
def acl(self):
    """Create our ACL on demand."""
    if self._acl is None:
        self._acl = BucketACL(self)
    return self._acl

• More generally HTTP requests should be limited to explicit method calls. This also rules out constructors loading data.
• Blob, Bucket, and *ACL (the only nouns) instances should have load(), exists(), create(), delete(), and update() methods. This design gives rise to code like

blob = Blob('/remote/path.txt', bucket=bucket, properties=properties)
try:
    blob.load()  # Just metadata
except NotFound:
    blob.upload_from_file(filename)  # Sends metadata from properties

(this maybe screams for get_or_create(), we'll feel it out as we develop). It's unclear if it's worth making a distinction between storage.NOUN.update <--> PUT and storage.NOUN.patch <--> PATCH. (As of right now, we don't implement PUT / update anywhere.)

• exists() should use fields in the requests to minimize the payload.
• A Connection should not be required to be bound to any object (one of the nouns Bucket, ACL, or Blob) but should be an optional argument to methods which actually talk to the API.
• We should strongly consider adding a last_updated field to classes to indicate the last time the values were updated (from the server).
• For list methods: List all buckets top-level, e.g.

storage.get_all_buckets(connection=optional_connection)

and then bucket.get_all_objects(). It's unclear how the other 3 nouns (objectAccessControls, bucketAccessControls and defaultObjectAccessControls) will handle this. Right now they are handled via ObjectACL.reload() and BucketACL.reload() (a superclass of DefaultObjectACL).

• Implicit behavior (default project, default bucket and/or default connection) should be used wherever possible (and documented)

@tseaver Please weigh in. This was inspired by our discussion at the end of #604.

/cc @thobrla I'd like to close #545 and focus on this (possibly broken down into sub-bugs). Does that sound OK?

[thobrla]

Closing #545 and focusing on this+sub-bugs sounds good to me; this captures the concerns I had there.

[dhermes]

Thanks!

[dhermes]

Aside: I made a list of all the 34 API methods and the corresponding code paths.

[dhermes]

@tseaver I'd like to get moving on this soon. WDYT of the proposals?

[tseaver]

@dhermes I'm in "quick hit" mode today, but can review in more depth tomorrow.

[dhermes]

I'm in Portland mostly AFK, so next few days is fine.