Skip to content

Commit 876d3e7

Browse files
devin-ai-integration[bot]bot_apk
andauthored
docs(source-postgres): improve documentation accuracy and completeness (#74295)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: bot_apk <apk@cognition.ai>
1 parent acafe26 commit 876d3e7

2 files changed

Lines changed: 24 additions & 22 deletions

File tree

docs/integrations/sources/postgres.md

Lines changed: 22 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Airbyte's certified Postgres connector offers the following features:
99
- All available [sync modes](https://docs.airbyte.com/cloud/core-concepts#connection-sync-modes), providing flexibility in how data is delivered to your destination.
1010
- Reliable replication at any table size with [checkpointing](https://docs.airbyte.com/understanding-airbyte/airbyte-protocol/#state--checkpointing) and chunking of database reads.
1111

12-
The contents below include a 'Quick Start' guide, advanced setup steps, and reference information (data type mapping, and changelogs). See [here](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting) to troubleshooting issues with the Postgres connector.
12+
The contents below include a 'Quick Start' guide, advanced setup steps, and reference information (data type mapping, and changelogs). See [here](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting) to troubleshoot issues with the Postgres connector.
1313

1414
</HideInUI>
1515

@@ -35,13 +35,13 @@ These steps create a dedicated read-only user for replicating data. Alternativel
3535

3636
The following commands will create a new user:
3737

38-
```roomsql
38+
```sql
3939
CREATE USER <user_name> PASSWORD 'your_password_here';
4040
```
4141

4242
Now, provide this user with read-only access to relevant schemas and tables. Re-run this command for each schema you expect to replicate data from:
4343

44-
```roomsql
44+
```sql
4545
GRANT USAGE ON SCHEMA <schema_name> TO <user_name>;
4646
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO <user_name>;
4747
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO <user_name>;
@@ -77,7 +77,7 @@ Now, click `Set up source` in the Airbyte UI. Airbyte will now test connecting t
7777

7878
## Advanced Configuration using CDC
7979

80-
Airbyte uses [logical replication](https://www.postgresql.org/docs/10/logical-replication.html) of the Postgres write-ahead log (WAL) to incrementally capture deletes using a replication plugin:
80+
Airbyte uses [logical replication](https://www.postgresql.org/docs/current/logical-replication.html) of the Postgres write-ahead log (WAL) to incrementally capture deletes using a replication plugin:
8181

8282
- See [here](https://docs.airbyte.com/understanding-airbyte/cdc) to learn more on how Airbyte implements CDC.
8383
- See [here](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#cdc-requirements) to learn more about Postgres CDC requirements and limitations.
@@ -174,10 +174,10 @@ ALTER TABLE tbl1 REPLICA IDENTITY FULL;`. Ensure that TOAST-able tables use non
174174
2. Create the Postgres publication. You should include all tables you want to replicate as part of the publication:
175175

176176
```
177-
CREATE PUBLICATION airbyte_publication FOR TABLE <tbl1, tbl2, tbl3>;`
177+
CREATE PUBLICATION airbyte_publication FOR TABLE <tbl1, tbl2, tbl3>;
178178
```
179179

180-
The publication name is customizable. Refer to the [Postgres docs](https://www.postgresql.org/docs/10/sql-alterpublication.html) if you need to add or remove tables from your publication in the future.
180+
The publication name is customizable. Refer to the [Postgres docs](https://www.postgresql.org/docs/current/sql-alterpublication.html) if you need to add or remove tables from your publication in the future.
181181
</FieldAnchor>
182182

183183
:::note
@@ -196,7 +196,7 @@ The Postgres source currently offers 3 methods of replicating updates to your de
196196

197197
### CDC
198198

199-
Airbyte uses [logical replication](https://www.postgresql.org/docs/10/logical-replication.html) of the Postgres write-ahead log (WAL) to incrementally capture deletes using a replication plugin. To learn more how Airbyte implements CDC, refer to [Change Data Capture (CDC)](https://docs.airbyte.com/understanding-airbyte/cdc/). We recommend configuring your Postgres source with CDC when:
199+
Airbyte uses [logical replication](https://www.postgresql.org/docs/current/logical-replication.html) of the Postgres write-ahead log (WAL) to incrementally capture deletes using a replication plugin. To learn more how Airbyte implements CDC, refer to [Change Data Capture (CDC)](https://docs.airbyte.com/understanding-airbyte/cdc/). We recommend configuring your Postgres source with CDC when:
200200

201201
- You need a record of deletions.
202202
- You have a very large database (500 GB or more).
@@ -243,7 +243,7 @@ If you are using SSH tunneling, as Airbyte Cloud requires encrypted communicatio
243243
For SSH Tunnel Method, select:
244244

245245
- `No Tunnel` for a direct connection to the database
246-
- `SSH Key Authentication` to use an RSA Private as your secret for establishing the SSH tunnel
246+
- `SSH Key Authentication` to use an RSA private key as your secret for establishing the SSH tunnel
247247
- `Password Authentication` to use a password as your secret for establishing the SSH tunnel
248248

249249
#### Connect via SSH Tunnel
@@ -255,7 +255,7 @@ When using an SSH tunnel, you are configuring Airbyte to connect to an intermedi
255255
To connect to a Postgres instance via an SSH tunnel:
256256

257257
1. While [setting up](#step-2-create-a-new-postgres-source-in-airbyte-ui) the Postgres source connector, from the SSH tunnel dropdown, select:
258-
- SSH Key Authentication to use a private as your secret for establishing the SSH tunnel
258+
- SSH Key Authentication to use a private key as your secret for establishing the SSH tunnel
259259
- Password Authentication to use a password as your secret for establishing the SSH Tunnel
260260
2. For **SSH Tunnel Jump Server Host**, enter the hostname or IP address for the intermediate (bastion) server that Airbyte will connect to.
261261
3. For **SSH Connection Port**, enter the port on the bastion server. The default port for SSH connections is 22.
@@ -278,14 +278,16 @@ The command produces the private key in PEM format and the public key remains in
278278

279279
## Configuring Entra authentication
280280

281-
The Airbyte source can be configured to authenticate as a Microsoft Entra service principal.
282-
This allows Airbyte to use short-lived identity tokens to authenticate to an Azure Postgres server.
281+
The Airbyte source can be configured to authenticate as a Microsoft Entra service principal.
282+
This allows Airbyte to use short-lived identity tokens to authenticate to an Azure Postgres server.
283283
Consult the Microsoft [documentation on this topic](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-azure-ad-authentication) for more detail on configuring the server and other Entra resources.
284284

285-
To configure the Airbyte Postgres source with Entra authentication, set the `Username` to the Entra ID, as discussed
286-
in [Microsoft's documentation](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/how-to-configure-sign-in-azure-ad-authentication).
287-
Set the password to a client secret for your Entra service principal.
288-
Expand the Optional Fields and provide the Entra tenant ID and Entra client (or app) ID of the service principal.
285+
To configure the Airbyte Postgres source with Entra authentication:
286+
287+
1. Set the `Username` to the Entra ID, as discussed in [Microsoft's documentation](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/how-to-configure-sign-in-azure-ad-authentication).
288+
2. Set the password to a client secret for your Entra service principal.
289+
3. Enable the **Entra service principal authentication** toggle.
290+
4. Provide the **Entra tenant ID** and **Entra client (or app) ID** of the service principal.
289291

290292
## Limitations & Troubleshooting
291293

@@ -358,11 +360,11 @@ According to Postgres [documentation](https://www.postgresql.org/docs/14/datatyp
358360
| 3.7.2 | 2026-03-04 | [74294](https://github.com/airbytehq/airbyte/pull/74294) | Fix CDC bug where a replication slot can be advanced too far, losing needed WAL segments. Remove CVEs. |
359361
| 3.7.1 | 2026-01-27 | [72396](https://github.com/airbytehq/airbyte/pull/72396) | This version causes issues discovering streams for some users. Do not use. |
360362
| 3.7.0 | 2025-08-12 | [57511](https://github.com/airbytehq/airbyte/pull/57511) | Add configurations for Azure authentication to Azure Postgres servers. |
361-
| 3.6.35 | 2025-06-12 | [61527](https://github.com/airbytehq/airbyte/pull/61527) | Add error handling for connection issues and adopt the latest CDK version. |
362-
| 3.6.34 | 2025-05-11 | [60876](https://github.com/airbytehq/airbyte/pull/60876) | Cache CDC initial state once constructed. |
363-
| 3.6.33 | 2025-05-11 | [60214](https://github.com/airbytehq/airbyte/pull/60214) | Migrate to new Gradle flow. |
364-
| 3.6.32 | 2025-05-8 | [59722](https://github.com/airbytehq/airbyte/pull/59722) | Consolidate gradle set up. |
365-
| 3.6.31 | 2025-04-18 | [58132](https://github.com/airbytehq/airbyte/pull/58132) | Fix vulnerabilities in dependencies. |
363+
| 3.6.35 | 2025-06-18 | [61527](https://github.com/airbytehq/airbyte/pull/61527) | Add error handling for connection issues and adopt the latest CDK version. |
364+
| 3.6.34 | 2025-05-29 | [60876](https://github.com/airbytehq/airbyte/pull/60876) | Cache CDC initial state once constructed. |
365+
| 3.6.33 | 2025-05-13 | [60214](https://github.com/airbytehq/airbyte/pull/60214) | Migrate to new Gradle flow. |
366+
| 3.6.32 | 2025-05-08 | [59722](https://github.com/airbytehq/airbyte/pull/59722) | Consolidate gradle set up. |
367+
| 3.6.31 | 2025-04-24 | [58132](https://github.com/airbytehq/airbyte/pull/58132) | Fix vulnerabilities in dependencies. |
366368
| 3.6.30 | 2025-03-06 | [55234](https://github.com/airbytehq/airbyte/pull/55234) | Update base image version for certified DB source connectors |
367369
| 3.6.29 | 2025-02-13 | [53649](https://github.com/airbytehq/airbyte/pull/53649) | Fix issue that column default value did not get converted |
368370
| 3.6.28 | 2024-12-23 | [50870](https://github.com/airbytehq/airbyte/pull/50870) | Use airbyte/java-connector-base:2.0.0 |

docs/integrations/sources/postgres/postgres-troubleshooting.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ To customize the JDBC connection beyond common options, specify additional suppo
132132

133133
Example: key1=value1&key2=value2&key3=value3
134134

135-
These parameters will be added at the end of the JDBC URL that the AirByte will use to connect to your Postgres database.
135+
These parameters will be added at the end of the JDBC URL that Airbyte will use to connect to your Postgres database.
136136

137137
The connector now supports `connectTimeout` and defaults to 60 seconds. Setting connectTimeout to 0 seconds will set the timeout to the longest time available.
138138

@@ -146,7 +146,7 @@ The Postgres connector may need some time to start processing the data in the CD
146146
- When the connection is set up for the first time and a snapshot is needed
147147
- When the connector has a lot of change logs to process
148148

149-
The connector waits for the default initial wait time of 5 minutes (300 seconds). Setting the parameter to a longer duration will result in slower syncs, while setting it to a shorter duration may cause the connector to not have enough time to create the initial snapshot or read through the change logs. The valid range is 120 seconds to 1200 seconds.
149+
The connector waits for the default initial wait time of 20 minutes (1200 seconds). Setting the parameter to a longer duration will result in slower syncs, while setting it to a shorter duration may cause the connector to not have enough time to create the initial snapshot or read through the change logs. The valid range is 120 seconds to 2400 seconds.
150150

151151
If you know there are database changes to be synced, but the connector cannot read those changes, the root cause may be insufficient waiting time. In that case, you can increase the waiting time (example: set to 600 seconds) to test if it is indeed the root cause. On the other hand, if you know there are no database changes, you can decrease the wait time to speed up the zero record syncs.
152152

0 commit comments

Comments
 (0)