Google Cloud Bigtable Python Library (Alpha)

This library supports RPC requests to the Google Cloud Bigtable API over HTTP/2. In order to support this, we'll rely on grpc. Unfortunately, the install story of gRPC is still developing. We are working with the gRPC team to rapidly make the install story more user-friendly.

The Cloud Bigtable API is supported via JSON over HTTP/1.1 (see the gcloud CLI use of the API). However, features of HTTP/2 such as streaming are used by the Bigtable API and are not possible to support via HTTP/1.1.

Make sure you have downloaded homebrew on OS X or linuxbrew on Linux. (On Linux, also be sure to add brew to ${PATH} as instructed.)

First, install the gRPC core (C/C++) library

curl -fsSL https://goo.gl/getgrpc | bash

Since this uses brew to install, this cannot be run as root (via sudo).

Next, install the Python grpcio library via:

BREW_PREFIX=$(brew --prefix)
[sudo] CFLAGS=-I${BREW_PREFIX}/include LDFLAGS=-L${BREW_PREFIX}/lib \
pip install --upgrade grpcio

You may wish to run this as root (via sudo) so it can be included with your machine's Python libraries. If not, you'll need to use a Python virtual environment so that non-privileged (i.e. non-sudo) installs are allowed.

Finally, you can install this library via

[sudo] pip install -e git+https://github.com/dhermes/gcloud-python-bigtable#egg=gcloud-bigtable

Again, you may wish to install as root or in a virtual environment.

Since the gRPC core is installed via brew, the system libraries are not in a place that Python can readily find them.

In order to run code that uses gcloud_bigtable with these libraries, you'll need to set the LD_LIBRARY_PATH environment variable:

BREW_PREFIX=$(brew --prefix)
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${BREW_PREFIX}/lib

You can make requests with your own Google account by using the gcloud CLI tool. You can create an access token via

gcloud login

and then from there, the token created will be picked up automatically when you create an object which requires authentication:

from gcloud_bigtable.cluster import Cluster
cluster = Cluster(project_id, zone, cluster_id)

If instead you'd like to use a service account, you can set an environment variable to the path containing the service account credentials JSON file:

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"

If you are familiar with the oauth2client library, you can create a credentials object directly and pass it to the constructor of an object which requires authentication:

from gcloud_bigtable.cluster import Cluster
cluster = Cluster(project_id, zone, cluster_id,
                  credentials=credentials)

1. Visit Google Cloud Console

2. Either create a new project or visit an existing one

3. In the project, click "APIs & auth > APIs". The URI should be of the form

   https://console.developers.google.com/project/{project-id}/apiui/apis/library
   

4. On this page, search for bigtable, and click both Cloud Bigtable API and Cloud Bigtable Table Admin API.

5. For each API, click "Enable API" (if not already enabled)

1. Visit Google Cloud Console

2. Either create a new project or visit an existing one

3. In the project, click "APIs & auth > Credentials". The URI should be of the form

   https://console.developers.google.com/project/{project-id}/apiui/credential
   

4. On this page, click "Create new Client ID", select "Service account" as your "Application type" and then download the JSON key provided.

After downloading, you can use this file as your GOOGLE_APPLICATION_CREDENTIALS.

1. Visit Google Cloud Console

2. Either create a new project or visit an existing one

3. In the project, click "Storage > Cloud Bigtable". The URI should be of the form

   https://console.developers.google.com/project/{project-id}/bigtable/clusters
   

4. On this page, click Create a cluster and take note of the "Cluster ID" and "Zone" you use when creating it.

Unfortunately, the gRPC Python library does not surface exceptions returned from the API. An issue has been filed with the gRPC team about this problem.

See CONTRIBUTING.md for instructions on development.