Flink MySQL CDC 2.3 Connection Pool Issue #2440
Unanswered
nathan-szz
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When using flink-sql-connector-mysql-cdc:2.3.0, I encountered an error upon task startup. It appears to be related to the unavailability of connections in the MySQL connection pool.
Adding 'connection.pool.size' ='200' resolves the issue mentioned above, and the task can execute normally.
However, in version 2.2.0, I didn't encounter this error. Without adding the aforementioned parameter, there are no issues, and the task can run normally.
org.apache.flink.util.FlinkRuntimeException: org.apache.flink.util.FlinkRuntimeException: java.sql.SQLTransientConnectionException: connection-pool-mysqltest-primary.mysql-bitnami:3306 - Connection is not available, request timed out after 30002ms.
at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.openJdbcConnection(DebeziumUtils.java:64) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.MySqlValidator.validate(MySqlValidator.java:72) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.MySqlSource.createEnumerator(MySqlSource.java:170) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at org.apache.flink.runtime.source.coordinator.SourceCoordinator.start(SourceCoordinator.java:222) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.applyCall(RecreateOnResetOperatorCoordinator.java:315) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator.start(RecreateOnResetOperatorCoordinator.java:70) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.start(OperatorCoordinatorHolder.java:198) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.startOperatorCoordinators(DefaultOperatorCoordinatorHandler.java:165) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.startAllOperatorCoordinators(DefaultOperatorCoordinatorHandler.java:82) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.scheduler.SchedulerBase.startScheduling(SchedulerBase.java:605) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.jobmaster.JobMaster.startScheduling(JobMaster.java:1046) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.jobmaster.JobMaster.startJobExecution(JobMaster.java:963) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.jobmaster.JobMaster.onStart(JobMaster.java:422) ~[flink-runtime-1.16.0.jar:1.16-SNAPSHOT]
at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStart(RpcEndpoint.java:198) ~[flink-dist-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StoppedState.lambda$start$0(AkkaRpcActor.java:622) ~[flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StoppedState.start(AkkaRpcActor.java:621) ~[flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:190) ~[flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_7e7f5d74-19da-43b1-93a0-b22723578747.jar:1.16.0]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
Caused by: org.apache.flink.util.FlinkRuntimeException: java.sql.SQLTransientConnectionException: connection-pool-mysqltest-primary.mysql-bitnami:3306 - Connection is not available, request timed out after 30002ms.
at com.ververica.cdc.connectors.mysql.source.connection.JdbcConnectionFactory.connect(JdbcConnectionFactory.java:72) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:890) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:885) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at io.debezium.jdbc.JdbcConnection.connect(JdbcConnection.java:418) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.openJdbcConnection(DebeziumUtils.java:61) ~[flink-sql-connector-mysql-cdc-2.3.0.jar:2.3.0]
... 37 more
Beta Was this translation helpful? Give feedback.
All reactions