Skip to content

Commit 94f6690

Browse files
chore: add readme for google-cloud-bigquery-jdbc (#4071)
* chore: add readme for `google-cloud-bigquery-jdbc` * chore: address gemini review comments * lint: fix typos * chore: address pr comments and refine readme content * refactor: address PR review feedback * chore: update jar name gs location --------- Co-authored-by: Kirill Logachev <kirl@google.com>
1 parent 0c030d0 commit 94f6690

File tree

1 file changed

+291
-0
lines changed

1 file changed

+291
-0
lines changed
Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
# Google BigQuery JDBC Client for Java
2+
3+
Java idiomatic client for [BigQuery JDBC][product-docs].
4+
5+
[![Maven][maven-version-image]][maven-version-link]
6+
![Stability][stability-image]
7+
8+
- [Product Documentation][product-docs]
9+
- [Client Library Documentation][javadocs]
10+
11+
12+
## Quickstart
13+
14+
15+
If you are using Maven, add this to your pom.xml file:
16+
17+
```xml
18+
<dependency>
19+
<groupId>com.google.cloud</groupId>
20+
<artifactId>google-cloud-bigquery-jdbc</artifactId>
21+
<version>LATEST_VERSION</version>
22+
</dependency>
23+
```
24+
25+
If you are using Gradle without BOM, add this to your dependencies:
26+
27+
```Groovy
28+
implementation 'com.google.cloud:google-cloud-bigquery-jdbc:LATEST_VERSION'
29+
```
30+
31+
If you are using SBT, add this to your dependencies:
32+
33+
```Scala
34+
libraryDependencies += "com.google.cloud" % "google-cloud-bigquery-jdbc" % "LATEST_VERSION"
35+
```
36+
37+
## Authentication
38+
39+
See the [Authentication][authentication] section in the base directory's README.
40+
41+
## Authorization
42+
43+
The client application making API calls must be granted [authorization scopes][auth-scopes] required for the desired BigQuery JDBC APIs, and the authenticated principal must have the [IAM role(s)][predefined-iam-roles] required to access GCP resources using the BigQuery JDBC API calls.
44+
45+
## Developer Guide
46+
47+
### Prerequisites
48+
49+
You need to have either Java with Maven installed or Docker. You might want to install [`Make`](https://www.gnu.org/software/make/) to simplify running commands, otherwise please look into Makefile to check for specific configurations.
50+
51+
### Setup
52+
53+
`make install` primarily relies on `mvn install` command. All following commands are primarily applicable for the `google-cloud-bigquery-jdbc` project.
54+
You can also use `make clean` to clean the project and `make lint` to format the code.
55+
56+
### Running tests
57+
58+
#### Unittests
59+
60+
Run all unittests
61+
62+
`make unittest`
63+
64+
Run specific unittests
65+
66+
`make unittest test=<filter>`
67+
68+
Please reference [Maven documentation](https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html) for details about `<filter>`
69+
70+
Example: `make unittest test=BigQueryArrowStructTest`
71+
72+
#### Integration tests
73+
74+
IMPORTANT: Running integration tests will skip unit tests by default. To include unit tests, run `make integration-test skipSurefire=false`. Primary focus of this command is to run specific set of tests without a lot of overhead.
75+
76+
Set the following environment variables to run the integration tests:
77+
78+
```
79+
# Default gcloud auth setup
80+
export GOOGLE_APPLICATION_CREDENTIALS=<path-to-service-account-json-config-file>
81+
82+
# Test specfic envs
83+
export SA_EMAIL=email@email.com
84+
export SA_SECRET=<path-to-service-account-json-config-file>
85+
# Alternatively it can be json content:
86+
export SA_SECRET=`cat <path-to-service-account-json-config-file>`
87+
```
88+
89+
Run all integration tests (currently takes 15+ minutes, so this is discouraged).
90+
91+
`make integration-test`
92+
93+
Run specific integration test
94+
95+
`make integration-test test=<filter>`
96+
97+
Please reference [Maven documentation](https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html) for details about `<filter>`
98+
99+
Example: `make integration-test test=ITBigQueryJDBCTest#testValidServiceAccountAuthenticationOAuthPvtKey`
100+
101+
### Dockerized environment
102+
103+
If you don't have Java or Maven, or if you want to test changes with a different Java version, you can leverage dockerized environment.
104+
105+
One-time run commands are similar to local development make commands:
106+
107+
```
108+
make docker-build
109+
make docker-unittest
110+
make docker-integration-test
111+
```
112+
113+
Please note that running unit or integration tests within docker doesn't leverage maven cache because it is not persisted.
114+
If you want to run multiple commands, you can start a session and treat this shell session as your local environment.
115+
116+
```
117+
make docker-session
118+
```
119+
120+
All Docker commands rely only on `GOOGLE_APPLICATION_CREDENTIALS` env being present, it will create rest of env vars as needed.
121+
122+
### Packaging
123+
124+
There are a few ways to package the Google JDBC Driver. The output of the packaging commands can be found in the `target` directory.
125+
126+
`make package` or `make docker-package` will create both a thin jar and a zip file with all dependencies.
127+
`make package-all-dependencies` or `make docker-package-all-dependencies` will create a single jar with all dependencies included.
128+
`make package-all-dependencies-shaded` or `make docker-package-all-dependencies-shaded` will create a single shaded jar with all dependencies included.
129+
130+
#### Thin jar
131+
132+
The thin jar is created with `make package`. The thin jar is packaged as a zip file with its dependencies listed in a `dependencies.txt` file, compatible with `pom.xml`.
133+
134+
#### All dependencies
135+
136+
The jar with all dependencies is also created. This jar includes all dependencies and can be used as a standalone jar with tools like R-Studio. You can run `make package-all-dependencies` or `make docker-package-all-dependencies` to build only the jar with all dependencies.
137+
138+
#### Shaded Jar
139+
You can also build a shaded jar with all dependencies. This can be done by running `make package-all-dependencies-shaded` or `make docker-package-all-dependencies-shaded`.
140+
141+
### Nightly builds
142+
143+
The nightly build runs the full set of integration tests, including the `ITBigQueryJDBCTest` and `ITNightlyBigQueryTest` test suites. It also includes some long-running tests (takes 20+ minutes to complete).
144+
145+
**Note:** These builds are intended for testing and development purposes only and are not recommended for production use.
146+
147+
Nightly Integration Tests include a step to build a full and thin jars and upload them to Google Storage.
148+
149+
They can be retrieved via following commands:
150+
151+
```
152+
gsutil cp gs://bq_devtools_release_private/drivers/jdbc/google-cloud-bigquery-jdbc-latest-all.jar .
153+
gsutil cp gs://bq_devtools_release_private/drivers/jdbc/google-cloud-bigquery-jdbc-latest.zip .
154+
```
155+
156+
#### Performance tests
157+
158+
Cloud Build Pipeline is uploading latest full jar to the internal location for perf tests once a week.
159+
160+
### Code Coverage
161+
162+
We're using [JaCoCo](https://www.eclemma.org/jacoco/) to track Code Coverage. `Makefile` has 2 separate set of commands for unittests and integration tests reports.
163+
164+
You can run `make unit-test-coverage` to generate a coverage report for unit tests. The output will be in `jacoco-unittests.zip`.
165+
You can run `make full-coverage` to generate a coverage report for both unit and integration tests. The output will be in `jacoco-full.zip`.
166+
167+
You can also run `make docker-coverage` which will produce both results. You can find `jacoco-unittests.zip` and `jacoco-full.zip` files in the root with results.
168+
169+
## Getting Started
170+
171+
### Prerequisites
172+
173+
You will need a [Google Cloud Platform Console][developer-console] project with the BigQuery JDBC [API enabled][enable-api].
174+
175+
[Follow these instructions][create-project] to get your project set up. You will also need to set up the local development environment by
176+
[installing the Google Cloud Command Line Interface][cloud-cli] and running the following commands in command line:
177+
`gcloud auth login` and `gcloud config set project [YOUR PROJECT ID]`.
178+
179+
### Installation and setup
180+
181+
You'll need to obtain the `google-cloud-bigquery-jdbc` library. See the [Quickstart](#quickstart) section
182+
to add `google-cloud-bigquery-jdbc` as a dependency in your code.
183+
184+
## About BigQuery JDBC
185+
186+
187+
[BigQuery JDBC][product-docs]
188+
189+
See the [BigQuery JDBC client library docs][javadocs] to learn how to
190+
use this BigQuery JDBC Client Library.
191+
192+
193+
194+
195+
196+
197+
## Troubleshooting
198+
199+
To get help, follow the instructions in the [shared Troubleshooting document][troubleshooting].
200+
201+
## Supported Java Versions
202+
203+
Java 8 or above is required for using this client.
204+
205+
Google's Java client libraries,
206+
[Google Cloud Client Libraries][cloudlibs]
207+
and
208+
[Google Cloud API Libraries][apilibs],
209+
follow the
210+
[Oracle Java SE support roadmap][oracle]
211+
(see the Oracle Java SE Product Releases section).
212+
213+
### For new development
214+
215+
In general, new feature development occurs with support for the lowest Java
216+
LTS version covered by Oracle's Premier Support (which typically lasts 5 years
217+
from initial General Availability). If the minimum required JVM for a given
218+
library is changed, it is accompanied by a [semver][semver] major release.
219+
220+
Java 11 and (in September 2021) Java 17 are the best choices for new
221+
development.
222+
223+
### Keeping production systems current
224+
225+
Google tests its client libraries with all current LTS versions covered by
226+
Oracle's Extended Support (which typically lasts 8 years from initial
227+
General Availability).
228+
229+
#### Legacy support
230+
231+
Google's client libraries support legacy versions of Java runtimes with long
232+
term stable libraries that don't receive feature updates on a best efforts basis
233+
as it may not be possible to backport all patches.
234+
235+
Google provides updates on a best efforts basis to apps that continue to use
236+
Java 7, though apps might need to upgrade to current versions of the library
237+
that supports their JVM.
238+
239+
#### Where to find specific information
240+
241+
The latest versions and the supported Java versions are identified on
242+
the individual GitHub repository `github.com/GoogleAPIs/java-SERVICENAME`
243+
and on [google-cloud-java][g-c-j].
244+
245+
## Versioning
246+
247+
248+
This library follows [Semantic Versioning](http://semver.org/).
249+
250+
251+
252+
## Contributing
253+
254+
255+
Contributions to this library are always welcome and highly encouraged.
256+
257+
See [CONTRIBUTING][contributing] for more information how to get started.
258+
259+
Please note that this project is released with a Contributor Code of Conduct. By participating in
260+
this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more
261+
information.
262+
263+
264+
## License
265+
266+
Apache 2.0 - See [LICENSE][license] for more information.
267+
268+
Java is a registered trademark of Oracle and/or its affiliates.
269+
270+
[product-docs]: https://cloud.google.com/bigquery
271+
[javadocs]: https://cloud.google.com/java/docs/reference/google-cloud-bigquery/latest/history
272+
[stability-image]: https://img.shields.io/badge/stability-unknown-red
273+
[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigquery-jdbc.svg
274+
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-bigquery-jdbc/0.0.0
275+
[authentication]: https://github.com/googleapis/google-cloud-java#authentication
276+
[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes
277+
[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
278+
[iam-policy]: https://cloud.google.com/iam/docs/overview#cloud-iam-policy
279+
[developer-console]: https://console.developers.google.com/
280+
[create-project]: https://cloud.google.com/resource-manager/docs/creating-managing-projects
281+
[cloud-cli]: https://cloud.google.com/cli
282+
[troubleshooting]: https://github.com/googleapis/google-cloud-java/blob/main/TROUBLESHOOTING.md
283+
[contributing]: https://github.com/googleapis/java-bigquery/blob/main/CONTRIBUTING.md
284+
[code-of-conduct]: https://github.com/googleapis/java-bigquery/blob/main/CODE_OF_CONDUCT.md
285+
[license]: https://github.com/googleapis/java-bigquery/blob/main/LICENSE
286+
287+
[semver]: https://semver.org/
288+
[cloudlibs]: https://cloud.google.com/apis/docs/client-libraries-explained
289+
[apilibs]: https://cloud.google.com/apis/docs/client-libraries-explained#google_api_client_libraries
290+
[oracle]: https://www.oracle.com/java/technologies/java-se-support-roadmap.html
291+
[g-c-j]: https://github.com/googleapis/google-cloud-java

0 commit comments

Comments
 (0)