spark load 报错
用户名802327 发布于2021-06 浏览:1131 回复:1
0
收藏
快速回复

2021-06-01 09:10:18,690 INFO (Load etl checker|37) [LoadManager.lambda$processEtlStateJobs$7():467] update load job etl status failed. job id: 15488
org.apache.doris.common.DataQualityException: errCode = 2, detailMessage = quality not good enough to cancel
at org.apache.doris.load.loadv2.SparkLoadJob.unprotectedProcessEtlFinish(SparkLoadJob.java:345) ~[palo-fe.jar:3.4.0]
at org.apache.doris.load.loadv2.SparkLoadJob.updateEtlStatus(SparkLoadJob.java:298) ~[palo-fe.jar:3.4.0]
at org.apache.doris.load.loadv2.LoadManager.lambda$processEtlStateJobs$7(LoadManager.java:465) ~[palo-fe.jar:3.4.0]
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) [?:1.8.0_251]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) [?:1.8.0_251]
at java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3566) [?:1.8.0_251]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) [?:1.8.0_251]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) [?:1.8.0_251]
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) [?:1.8.0_251]
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) [?:1.8.0_251]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:1.8.0_251]
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) [?:1.8.0_251]
at org.apache.doris.load.loadv2.LoadManager.processEtlStateJobs(LoadManager.java:463) [palo-fe.jar:3.4.0]
at org.apache.doris.load.loadv2.LoadEtlChecker.runAfterCatalogReady(LoadEtlChecker.java:42) [palo-fe.jar:3.4.0]
at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) [palo-fe.jar:3.4.0]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) [palo-fe.jar:3.4.0]

 

 

spark load 报这个错 在哪里可以看到 错误的具体信息呢

收藏
点赞
0
个赞
共1条回复 最后由Ling缪回复于2021-06
#2Ling缪回复于2021-06

quality not good enough to cancel 这个是报错关键,意思是你的数据质量不合格。

如果你有 load 的label 的话,通过 show load 命令可以看到错误数据样例。

0
快速回复
TOP
切换版块