Add support for BIGTABLE_EMULATOR_HOST environment variable

[garye]

The Python client should support the Bigtable emulator.

This should work similarly to the python support for PUBSUB_EMULATOR_HOST and DATASTORE_EMULATOR_HOST.

cc @mbrukman

[dhermes]

👍 I just noticed last week that the Bigtable emulator had been added. I've been meaning to sort out some b0rken-ness with the other clients (never checked if they worked after adding gRPC support to the clients).

[dhermes]

@garye I mostly have this finished but it seems like most of system_tests/bigtable.py fails because the backend comes back with UNAVAILABLE. Is there a list of methods actually working in the emulator?

[garye]

Instance-related commands aren't implemented and I suspect that's what you're running into. It's meant for table-level admin/read/write operations, so you can reference any project/instance when you create a table.

[dhermes]

OK. It seems empirically that deleting a table doesn't work. Also, I'm starting to think some of the UNAVAILABLE I'm seeing are masked as other issues, e.g. creating a table that already exists (since a previous delete failed). And possibly my system test has too high a QPS for the server?

[garye]

• UNAVAILABLE was because the server crashed when asked to do a ReadRows with no Rows specified. This should be supported so we'll fix ASAP, but it will be at least a week until a fix can make it into a gcloud release.
• Deleting a table should work fine, it does locally when I ran a patched emulator that handles the ReadRows request.
• The TableAdmin tests are supported and pass locally if enabled.
• Reading with a label is not supported and isn't high on the list since it's seldom used. I'd skip that test in emulator mode.

[dhermes]

Thanks for digging into this! We already skip the IAM tests on the Pub / Sub side when running in the emulator, so not a big deal for labels.

The sugar to allow local connections when the emulator already works before waiting for the gcloud CLI fix to be released. Worth adding that support and then adding a working tox -e bigtable-emulator in a subsequent PR?

[garye]

Yeah that works for me, I'd like to see something out sooner rather than later.

[dhermes]

@garye Now that #2249 has been merged, can you give me a heads up when the fixes are in so I can add tox -e bigtable-emulator?

[garye]

@dhermes The latest gcloud/emulator release should have the fix. Can you give it a shot?

[dhermes]

@garye Thanks for the heads-up. Check out #2322 and LMK if I've disable too many test cases.