键盘上的艺术^ 2021-10-12 14:34 采纳率: 0%
浏览 60

flink运行任务报错

flink运行任务报错,Failed to construct kafka consumer,下面提示 Too many open files ,请问这个是什么原因,具体怎么排查或者解决,麻烦大佬指点一下

2021-10-12 14:19:02
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:765)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:633)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:615)
    at org.apache.flink.streaming.connectors.kafka.internal.Kafka010PartitionDiscoverer.initializeConnections(Kafka010PartitionDiscoverer.java:58)
    at org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:573)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
    at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:309)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$1(StreamTask.java:545)
    at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:503)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:566)
    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:752)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:569)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.KafkaException: java.io.IOException: Too many open files
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:129)
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:156)
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:160)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:701)
    ... 15 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
    at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
    at java.nio.channels.Selector.open(Selector.java:227)
    at org.apache.kafka.common.network.Selector.<init>(Selector.java:127)
    ... 18 more
  • 写回答

2条回答 默认 最新

  • 此木Y 2021-10-12 18:01
    关注

    1.ulimit -aS | grep open 用这个命令看一下容器的句柄数量
    2.看一下配置文件的taskmanager的并行度,降低一下试试

    评论

报告相同问题?

问题事件

  • 创建了问题 10月12日

悬赏问题

  • ¥60 版本过低apk如何修改可以兼容新的安卓系统
  • ¥25 由IPR导致的DRIVER_POWER_STATE_FAILURE蓝屏
  • ¥50 有数据,怎么建立模型求影响全要素生产率的因素
  • ¥50 有数据,怎么用matlab求全要素生产率
  • ¥15 TI的insta-spin例程
  • ¥15 完成下列问题完成下列问题
  • ¥15 C#算法问题, 不知道怎么处理这个数据的转换
  • ¥15 YoloV5 第三方库的版本对照问题
  • ¥15 请完成下列相关问题!
  • ¥15 drone 推送镜像时候 purge: true 推送完毕后没有删除对应的镜像,手动拷贝到服务器执行结果正确在样才能让指令自动执行成功删除对应镜像,如何解决?