在flink-1.13.2中测试执行批处理版本的单词计数程序时,出现"Exception in thread "Thread-5" java.lang.IllegalStateException: T

2022-04-11 08:48:23.0

问题描述

在flink-1.13.2中测试执行批处理版本的单词计数程序时:

$ cd ~/bigdata/flink-1.13.2
$ ./bin/flink run -t yarn-per-job  -p 2 -yjm 1G -ytm 1G ./examples/batch/WordCount.jar

程序正常执行,单词计数结果也正常输出,但是最后出现下面这样的异常信息:

Exception in thread "Thread-5" java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:164)
        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:183)
        at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2780)
        at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3036)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2995)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2968)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2848)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1200)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1812)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1789)

解决办法

在Flink的conf/flink-conf.yaml文件中,添加下面这行内容即可:

classloader.check-leaked-classloader: false

《PySpark原理深入与编程实战》