DBZ-7920 Add support for 'infinity' values for PostgreSQL

[mfvitale]

closes: https://issues.redhat.com/browse/DBZ-7920

Missing task:

• Perform a manual e2e test
• Add support and e2e tests for other destination platforms
  • MySQL
  • Oracle
  • SQLServer
  • Db2

[Naros]

Hi @mfvitale , one question. I see you marked do manual e2e test, what about these values from PG going to other database platforms? Should we introduce an E2E test that only applies to the PG source but can be run against the other sink targets for completeness?

[mfvitale]

what about these values from PG going to other database platforms?

This is a good question. I just tried PostgreSQL to PostgreSQL. I'll add e2e for the others.

[Naros]

So I suspect we likely will want to add some methods on the DatabaseDialect so that various implementations can control what values should be serialized to the target system in the case of +/-INFINITY from PostgreSQL:

It looks like from what I quickly found:

Vendor	-INFINITY	+INFINITY
Oracle/MySQL	1000/01/01 00:00:00	9999/12/31 23:59:59
SQL Server/Db2	0001/01/01 00:00:00	9999/12/31 23:59:59

[mfvitale]

So I suspect we likely will want to add some methods on the DatabaseDialect so that various implementations can control what values should be serialized to the target system in the case of +/-INFINITY from PostgreSQL:

It looks like from what I quickly found:

Vendor -INFINITY +INFINITY
Oracle/MySQL 1000/01/01 00:00:00 9999/12/31 23:59:59
SQL Server/Db2 0001/01/01 00:00:00 9999/12/31 23:59:59

Thanks for it.

This open another question: should we manage also the opposite? These values should be converted to +/-infinity when PostgreSQL is the destination?

[mfvitale]

This is my final founding about the other databases support

Vendor	-INFINITY	+INFINITY
Oracle	-4712-01-01T00:00:00+00:00	9999-12-31T23:59:59+00:00
MySQL	1970-01-01T00:00:01+00:00	2038-01-19T03:14:07+00:00
SQL Server/Db2	0001/01/01 00:00:00	9999/12/31 23:59:59

[mfvitale]

@Naros can you give another look?

[Naros]

This open another question: should we manage also the opposite? These values should be converted to +/-infinity when PostgreSQL is the destination?

That's a good question and I guess it would depend on the complexity.

I haven't looked at the recent changes, but I could imagine that if the calendar limits for another vendor go beyond the limits of PostgreSQL, then it would make logical sense to convert those to +/- INFINITY since we would be incapable of writing the actual date value into PostgreSQL without an error.

[Naros]

LGTM, just a few inline comments & my comment above about translating to PG +/- INFINITY values.

[Naros]

I must admit, I'm not a fan of extending a class with the same name; should we think about prefixing the derived class with a database prefix of sorts?

[mfvitale]

Or we can prefix the base one with "Debezium" as done for "Connect" one. WDYT? I am open for both.

[Naros]

Or we can prefix the base one with "Debezium" as done for "Connect" one.

Ya lets go with that approach, its more consistent with what we have already with prefixes, @mfvitale

[mfvitale]

I haven't looked at the recent changes, but I could imagine that if the calendar limits for another vendor go beyond the limits of PostgreSQL, then it would make logical sense to convert those to +/- INFINITY since we would be incapable of writing the actual date value into PostgreSQL without an error.

The only issue seems to be only for the Oracle -infinity value (-4712-01-01T00:00:00+00:00). In that case maybe it is better to convert all infinity values from the source to the correct destination value. So I think it is better to manage this in a separate issue. WDYT?

[Naros]

The only issue seems to be only for the Oracle -infinity value (-4712-01-01T00:00:00+00:00). In that case maybe it is better to convert all infinity values from the source to the correct destination value. So I think it is better to manage this in a separate issue. WDYT?

Makes sense to me. 👍

[Naros]

Reran the tests for Oracle and they all checked out 👍, so LGTM.

[mfvitale]

The only issue seems to be only for the Oracle -infinity value (-4712-01-01T00:00:00+00:00). In that case maybe it is better to convert all infinity values from the source to the correct destination value. So I think it is better to manage this in a separate issue. WDYT?

Makes sense to me. 👍

Opened https://issues.redhat.com/browse/DBZ-8033