Spark InputCoercionException异常

2021-11-15 17:11:33.0

在IntellIJ IDEA中开发Spark程序,当调用dataframe.show()方法时,却出现如下错误信息:Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.exc.InputCoercionException。如下所示:

Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/exc/InputCoercionException
	at com.fasterxml.jackson.module.scala.deser.NumberDeserializers$.(ScalaNumberDeserializersModule.scala:48)
	at com.fasterxml.jackson.module.scala.deser.NumberDeserializers$.(ScalaNumberDeserializersModule.scala)
	at com.fasterxml.jackson.module.scala.deser.ScalaNumberDeserializersModule.$init$(ScalaNumberDeserializersModule.scala:60)
	at com.fasterxml.jackson.module.scala.DefaultScalaModule.(DefaultScalaModule.scala:18)
	at com.fasterxml.jackson.module.scala.DefaultScalaModule$.(DefaultScalaModule.scala:36)
	at com.fasterxml.jackson.module.scala.DefaultScalaModule$.(DefaultScalaModule.scala)
	......
	at com.xueai8.elt.MetroELTJob.main(MetroELTJob.scala)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.exc.InputCoercionException
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 33 more

解决办法

因为我这里使用的是Spark 3.1.2,所以在pom.xml中添加如下依赖即可:

<dependency>
    <groupid>com.fasterxml.jackson.core</groupid>
    <artifactid>jackson-cor</artifactid>
    <version>2.10.0</version>
</dependency>

再次执行程序,一切OK!


《PySpark原理深入与编程实战》