PostgreSQL Source and Databricks Target

The following table identifies the recommended data-type mappings for Database Ingestion and Replication configurations with a PostgreSQL source and a Databricks target:

PostgreSQL Source Data Type	Databricks Target Data Type
bigint	bigint
bit varying(precision), 1 <= p <= 83886080	binary
bit(precision), 1 <= p <= 83886080	binary
boolean	boolean
box	binary
character varying(size), 1 <= size <= 10485760	string
character(size), 1 <= size <= 10485760	string
cidr	string
circle	binary
date	string
daterange	string
double precision	double
inet	string
int4range	string
int8range	string
integer	int
json	string
jsonb	string
line	binary
lseg	binary
macaddr	string
macaddr8	string
money	decimal(19,2)
numeric	string
numeric(p,s), 1 <= p <= 38, 0 <= s <= 38	decimal(p,s), 1 <= p <= 38, 0 <= s <= 38
numeric(p,s), 39 <= p <= 1000, 39 <= s <= 1000	string
numrange	string
path	binary
point	binary
polygon	binary
real	float
smallint	int
time(precision) with time zone, 0 <= p <= 6	string
time(precision) without time zone, 0 <= p <= 6	string
timestamp(precision) with time zone, 0 <= p <= 6	timestamp
timestamp(precision) without time zone, 0 <= p <= 6	timestamp
tsrange	string
tstzrange	string
uuid	string
xml	string

Limitations

•Database ingestion and replication initial load and incremental load jobs can replicate data from PostgreSQL bytea, text, xml, and other large-object columns to Databricks targets if you select the Include LOBs option under Advanced on the Source page of the task wizard. LOB column data is truncated before being written to the target if it is greater in size than a byte limit that depends on the LOB type. For more information, see the "Include LOBs" description in Configure a PostgreSQL source.

Unsupported source data types

For initial load jobs, Database Ingestion and Replication does not support the following PostgreSQL data types:

•ABSTIME

•Array types

•NAME

•NUMERIC with a scale greater than the precision

•Object identifier types

•PG_LSN

•RELTIME

•Text search types:

- TSQUERY
- TSVECTOR

•User-defined types

For incremental load jobs, Database Ingestion and Replication does not support the following PostgreSQL data types, in addition to those not supported for initial load jobs:

•Spatial types

- Box
- Circle
- Line
- LSeg
- Path
- Point
- Polygon

•Unbounded varying types

Database ingestion and replication jobs either fail to deploy or propagate nulls for columns that have these data types.