Spark异常处理：java.io.InvalidClassException: org.apache.commons.lang3.time.FastDateParser; local class incompatible: stream classdesc serialVersionUID = 2, local class serialVersionUID = 3
相关阅读：
Spark异常处理：java.lang.NoSuchMethodError: io.netty.channel.DefaultFileRegion.(Ljava/io/File;JJ)V
环境介绍：
系统：Centos 7.2
Zeppelin：0.8.0
Spark：apache 2.1.0
问题描述：
Spark on zeppelin执行任务出现异常，异常信息如下：
INFO [2018-07-25 13:55:30,000] ({dag-scheduler-event-loop} Logging.scala[logInfo]:54) - ResultStage 2 (csv at NativeMethodAccessorImpl.java:0) failed in 0.147 s due to Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 8, 192.168.1.45, executor 0): java.io.InvalidClassException: org.apache.commons.lang3.time.FastDateParser; local class incompatible: stream classdesc serialVersionUID = 2, local class serialVersionUID = 3
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:80)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
主要异常为
java.io.InvalidClassException: org.apache.commons.lang3.time.FastDateParser; local class incompatible: stream classdesc serialVersionUID = 2, local class serialVersionUID = 3
异常分析：
这个异常说的是本地编译的class UID和集群上的UID不一致
出现这个异常的原因有多个：
具体原因请根据实际情况判断
1.spark运行的jar包，和集群运行的jar包不一致
2.编译运行的类jdk版本，和spark集群的jdk版本不一致
3.Scala版本不一致
异常处理：
思路：
1.先判断最基本的jdk版本和scala版本是否一致
2.再去判断运行的jar包是否一致
跟着思路走：
1.scala和jdk版本没有异常，然后去判断jar包，其中从HORTONWORKS找到了一篇的文章，https://community.hortonworks.com/content/supportkb/154247/error-message-javaioinvalidclassexception-orgapach.html
可惜提供的解决办法，并不适用于当前的状况
2.查看zeppelin spark解释器日志发现如下一段信息：
INFO [2018-07-25 13:54:58,370] ({pool-2-thread-2} Logging.scala[logInfo]:54) - Running Spark version 2.1.0.2.6.0.3-8
发现了异常点，zeppelin调用的spark版本竟然是hdp的spark，我当前的环境为apache 的spark版本
异常处理：
在zeppelin目录下，为了之前测试zeppelin与hdp spark相关依赖问题，将spark相关依赖包放到了zeppelin/lib下，测试过后，忘记删除，导致于现有apache spark集群jar包冲突，删除即可
注：
这是个低级错误，但是发生如上异常时，跟着思路走即可，原因有可能不同
如果这个问题还没有解决，说明是jar包不一致导致的，解决办法请看这篇文章！！！
亲，看完了点个赞呗！
