Flink Mongo CDC 2.3.0 remove copy.existing.pipeline config?

[newsbreak-tonglin]

@Jiabao-Sun Hi, Some problems occured when I use Flink Mongo CDC 2.3.0.

Has copy.existing.pipeline config been removed from Flink Mongo CDC 2.3.0? What can we do if we want to use Snapshot Data Filters?

Caused by: org.apache.flink.table.api.ValidationException: Unsupported options found for 'mongodb-cdc'.

Unsupported options:

copy.existing.pipeline

Supported options:

batch.size
chunk-meta.group.size
collection
connection.options
connector
copy.existing
copy.existing.queue.size
database
heartbeat.interval.ms
hosts
password
poll.await.time.ms
poll.max.batch.size
property-version
scan.incremental.snapshot.chunk.size.mb
scan.incremental.snapshot.enabled
username
at org.apache.flink.table.factories.FactoryUtil.validateUnconsumedKeys(FactoryUtil.java:624)
at org.apache.flink.table.factories.FactoryUtil$FactoryHelper.validate(FactoryUtil.java:914)
at org.apache.flink.table.factories.FactoryUtil$TableFactoryHelper.validate(FactoryUtil.java:978)
at com.ververica.cdc.connectors.mongodb.table.MongoDBTableSourceFactory.createDynamicTableSource(MongoDBTableSourceFactory.java:61)
at org.apache.flink.table.factories.FactoryUtil.createDynamicTableSource(FactoryUtil.java:156)
... 40 more

[Jiabao-Sun]

Hi @newsbreak-tonglin.
The copy.existing.pipeline may be cause the incorrect result in incremental read scenario, so we temporarily removed it.

[newsbreak-tonglin]

@Jiabao-Sun Thank you for your reply

[herunkang2018]

@Jiabao-Sun Could you give some details about incorrect result in incremental read scenario? Thanks a lot.