Skip to content

Commit abd7bb8

Browse files
Neenu1995gemini-code-assist[bot]
Neenu1995
and
gemini-code-assist[bot]
authored
docs(bigquery-jdbc): add developer and maintainer guide (#13879)
b/538176465 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent 74785ef commit abd7bb8

1 file changed

Lines changed: 179 additions & 0 deletions

File tree

‎java-bigquery-jdbc/DEVELOPMENT.md‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# BigQuery JDBC Developer & Contributor Guide
2+
3+
This guide details the architectural design, core abstractions, coding principles, and testing workflows for developers contributing to the `google-cloud-bigquery-jdbc` module.
4+
5+
---
6+
7+
## Table of Contents
8+
1. [Core Architecture & Component Map](#1-core-architecture--component-map)
9+
2. [Developer Guardrails & Rules of Engagement](#2-developer-guardrails--rules-of-engagement)
10+
3. [Build & Test Playbook](#3-build--test-playbook)
11+
- [Local Build Commands](#local-build-commands)
12+
- [Running Unit Tests](#running-unit-tests)
13+
- [Running Integration Tests](#running-integration-tests)
14+
- [Dockerized Execution](#dockerized-execution)
15+
4. [Logging Architecture & Developer Conventions](#4-logging-architecture--developer-conventions)
16+
- [Instantiating Loggers](#instantiating-loggers)
17+
- [Developer Logging Rules & Conventions](#developer-logging-rules--conventions)
18+
5. [Pre-PR Checklist](#5-pre-pr-checklist)
19+
20+
---
21+
22+
## 1. Core Architecture & Component Map
23+
24+
The driver is structured to provide high performance, zero-allocation MDC log tracing, strict JDBC compliance, and seamless execution over the Google Cloud BigQuery REST and Storage APIs.
25+
26+
```mermaid
27+
graph TD
28+
Client[Client Application / BI Tool] -->|DriverManager.getConnection| Driver[BigQueryDriver]
29+
Driver -->|Parses URI & Options| UrlUtil[BigQueryJdbcUrlUtility]
30+
Driver -->|Configures Logging| RootLogger[BigQueryJdbcRootLogger]
31+
Driver -->|Creates| Conn[BigQueryConnection]
32+
Conn -->|Dynamic Context Proxy| Proxy[BigQueryJdbcContextProxy]
33+
Proxy -->|MDC Tracing| Mdc[BigQueryJdbcMdc]
34+
Proxy -->|Delegates Exec| DirectConn[Client Session]
35+
Conn -->|Type Mapping & Coercion| Coercion[BigQueryJdbcTypeMappings & BigQueryCoercion]
36+
Conn -->|REST / Storage API| BQSDK[google-cloud-bigquery]
37+
```
38+
39+
### Key Abstractions
40+
41+
- **`BigQueryDriver`**: JDBC entry point registered with `java.sql.DriverManager`. Intercepts `jdbc:bigquery://` URLs, initializes early logger state, and instantiates `BigQueryConnection`.
42+
- **`BigQueryConnection`**: Represents an active BigQuery session, holding dataset defaults, connection configuration maps, and transaction/session state (`EnableSession=true`, `session_id`).
43+
- **`BigQueryJdbcUrlUtility`**: Parses and validates connection string parameters using a bounded LRU parse cache (`PARSE_CACHE`) to avoid heavy allocations during frequent connection creation.
44+
- **`BigQueryJdbcContextProxy`**: A dynamic proxy layer (`java.lang.reflect.Proxy`) wrapping JDBC statements, connections, and metadata. Intercepts calls to propagate ThreadLocal MDC parameters (`connectionId`) across execution threads and enforce state validation (`checkClosed()`).
45+
- **`BigQueryJdbcTypeMappings` & `BigQueryCoercion`**: Centralized mapping logic handling standard JDBC-to-BigQuery SQL type mappings (`StandardSQLTypeName`) and object coercions (`Date`, `Timestamp`, `BigDecimal`, etc.).
46+
- **`BigQueryArrowResultSet`**: Custom result set implementation accelerating large query result retrieval via the BigQuery Storage Read API gRPC stream.
47+
48+
---
49+
50+
## 2. Developer Guardrails & Rules of Engagement
51+
52+
> [!IMPORTANT]
53+
> **Adhere strictly to the following guardrails when making code changes:**
54+
55+
1. **Visibility Principle**: Always default to the most restrictive access level (`private`, package-private, or `@InternalApi`). Do **NOT** expose classes or methods as `public` unless strictly required by standard JDBC interfaces.
56+
2. **Explicit Class Imports**: Always write explicit `import` statements. Do **NOT** use wildcard star imports or inline fully qualified class names (e.g., use `import java.math.BigDecimal;` instead of `java.math.BigDecimal` inline).
57+
3. **Logger Preference**: Always prefer `BigQueryJdbcCustomLogger` over `java.util.logging.Logger`. Format strings using `String.format(...)` before logging, as `BigQueryJdbcRootLogger` evaluates `record.getMessage()` directly.
58+
4. **Exception Handling**: Always throw exceptions from the `com.google.cloud.bigquery.exception` package (`BigQueryJdbcException`, `BigQueryJdbcSqlSyntaxErrorException`, `BigQueryConversionException`).
59+
5. **No Mocking of Final JDK Classes**: Do **NOT** mock final JDK types (`BigDecimal`, `LocalDate`, `Instant`, `UUID`) with Mockito. Mocking final JDK classes is unstable and can cause JVM crashes under JDK 21+. Always construct real instances in unit tests.
60+
61+
---
62+
63+
## 3. Build & Test Playbook
64+
65+
Builds and test tasks are managed via the module [Makefile](Makefile).
66+
67+
### Local Build Commands
68+
69+
```bash
70+
# Build & install module locally
71+
make install
72+
73+
# Clean project target directory
74+
make clean
75+
76+
# Format code and check linter compliance
77+
make lint
78+
```
79+
80+
### Running Unit Tests
81+
82+
```bash
83+
# Run all unit tests
84+
make unittest
85+
86+
# Run a specific unit test class
87+
make unittest test=BigQueryPreparedStatementTest
88+
89+
# Run a specific unit test method
90+
make unittest test=BigQueryPreparedStatementTest#testSetObjectWithTemporalTypes
91+
```
92+
93+
### Running Integration Tests
94+
95+
> [!WARNING]
96+
> Integration tests connect to real GCP BigQuery resources and require valid GCP credentials.
97+
98+
```bash
99+
# Set GCP service account credentials
100+
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json
101+
102+
# Run a specific integration test
103+
make integration-test test=ITBigQueryJDBCTest#testValidServiceAccountAuthenticationOAuthPvtKey
104+
```
105+
106+
### Dockerized Execution
107+
108+
If local Java/Maven environments are not available, use the dockerized environment:
109+
110+
```bash
111+
# Start an interactive shell session inside Docker container
112+
make docker-session
113+
114+
# Run unit tests inside Docker
115+
make docker-unittest
116+
```
117+
118+
---
119+
120+
## 4. Logging Architecture & Developer Conventions
121+
122+
The driver uses a custom logging subsystem built on top of `java.util.logging`: `BigQueryJdbcCustomLogger` and `BigQueryJdbcRootLogger`.
123+
124+
### Instantiating Loggers
125+
126+
- **For Instance Components** (`BigQueryConnection`, `BigQueryStatement`, `BigQueryDatabaseMetaData`):
127+
Use `this.toString()` to include instance identity in logger output:
128+
```java
129+
private final BigQueryJdbcCustomLogger LOG = new BigQueryJdbcCustomLogger(this.toString());
130+
```
131+
- **For Static / Utility Components** (`BigQueryJdbcUrlUtility`, `BigQueryJdbcTypeMappings`):
132+
Use the class name:
133+
```java
134+
private static final BigQueryJdbcCustomLogger LOG =
135+
new BigQueryJdbcCustomLogger(BigQueryJdbcTypeMappings.class.getName());
136+
```
137+
138+
### Developer Logging Rules & Conventions
139+
140+
1. **Method Entry / Exit Tracing**:
141+
Methods at `FINER` level must log entrance and exit points:
142+
```java
143+
public ResultSet executeQuery(String sql) throws SQLException {
144+
LOG.finer("++enter++");
145+
try {
146+
// ... execution logic ...
147+
return rs;
148+
} finally {
149+
LOG.finer("++exit++");
150+
}
151+
}
152+
```
153+
2. **Format Placeholders (Zero Allocation)**:
154+
Avoid string concatenation in log calls. Use formatting placeholders or `Supplier<String>` lambdas to prevent unneeded string allocation when the log level is disabled:
155+
```java
156+
// Recommended: Use printf-style formatting
157+
LOG.fine("Executing query on dataset: %s, table: %s", datasetId, tableId);
158+
159+
// Recommended: Use supplier lambda for expensive calculations
160+
LOG.fine(() -> "Parsed properties: " + complexObject.toDebugString());
161+
```
162+
3. **Caller Inference & MDC Propagation**:
163+
- `BigQueryJdbcCustomLogger` automatically wraps log records in `BigQueryJdbcLogRecord`, which inspects the stack trace to accurately infer caller class and method names.
164+
- `BigQueryJdbcMdc` maintains `connectionId` in a `ThreadLocal` context. When logging from proxy or worker threads, always ensure MDC context is preserved or propagated via `BigQueryJdbcContextProxy`.
165+
166+
---
167+
168+
## 5. Pre-PR Checklist
169+
170+
Before submitting a Pull Request:
171+
172+
- [ ] All new classes and methods use the narrowest possible visibility scope (`private` or package-private).
173+
- [ ] No inline fully qualified names or wildcard star imports are present.
174+
- [ ] All logger instances use `BigQueryJdbcCustomLogger`.
175+
- [ ] Method entrance/exit logging (`++enter++` / `++exit++`) is included for complex internal routines.
176+
- [ ] All method changes and feature additions are covered by corresponding JUnit 5 tests.
177+
- [ ] Unit tests pass cleanly without Mockito `UnnecessaryStubbingException` warnings.
178+
- [ ] All `Statement`, `ResultSet`, or `DatabaseMetaData` objects returned by public entry points are properly wrapped via `BigQueryJdbcContextProxy.wrap()`.
179+
- [ ] Code formatting and linting pass via `make lint`.

0 commit comments

Comments
 (0)