Open
Description
Related Template(s)
cassandra-to-bigtable
Template Version
N/A
What happened?
It's been failing on both main branch and PR runs.
https://github.com/GoogleCloudPlatform/DataflowTemplates/pull/2287/checks?check_run_id=39722028625.
I checked that there isn't recent change on Template code However
Update: it's not permared but highly flaky. We probably need retry or wait on Cassandra resource manager spin up
Relevant log output
com.datastax.oss.driver.api.core.AllNodesFailedException: Could not reach any contact point, make sure you've provided valid addresses (showing first 1 nodes, use getAllErrors() for more): Node(endPoint=/10.128.0.106:47906, hostId=null, hashCode=1b5c3f95): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|connecting...] Protocol initialization request, step 1 (OPTIONS): failed to send request (io.netty.channel.StacklessClosedChannelException)]
at com.datastax.oss.driver.api.core.AllNodesFailedException.copy(AllNodesFailedException.java:141)
at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
at com.datastax.oss.driver.api.core.session.SessionBuilder.build(SessionBuilder.java:835)
at org.apache.beam.it.cassandra.CassandraResourceManager.<init>(CassandraResourceManager.java:97)
Suppressed: com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|connecting...] Protocol initialization request, step 1 (OPTIONS): failed to send request (io.netty.channel.StacklessClosedChannelException)
at com.datastax.oss.driver.internal.core.channel.ProtocolInitHandler$InitRequest.fail(ProtocolInitHandler.java:356)
at com.datastax.oss.driver.internal.core.channel.ChannelHandlerRequest.writeListener(ChannelHandlerRequest.java:87)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
Suppressed: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /10.128.0.106:47906
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
Caused by: io.netty.channel.StacklessClosedChannelException
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0()(Unknown Source)