Flink开启后产生大量不健康的tablet
不愿当小卒啊啊 发布于2021-05 浏览:3075 回复:2
0
收藏

目前transaction的清理周期改成5分钟,只保留最近5分钟的。
flink现在一开,就会产生不健康的tablets,这个是什么情况,是不是跟Transaction清太快有关?

收藏
点赞
0
个赞
共2条回复 最后由13671653088回复于2021-05
#313671653088回复于2021-05

当前不太可能减少任务数量,请问这个副本延迟有没有什么参数可以调节,加快同步速度的?

当前flink执行stream load会有如下这些报错信息:

Streamload Response2:status: 200, resp msg: OK, resp content: {
        "TxnId": 11593972,
        "Label": "audit_20210521_102622_dde2d38a142c44d3a9a443f9886b8862",
        "Status": "Fail",
        "Message": "errCode = 2, detailMessage = Failed to commit txn 11593972. Tablet [2561774] success replica num 1 is less than quorum replica num 2 while error backends 10011",
        "NumberTotalRows": 3,
        "NumberLoadedRows": 3,
        "NumberFilteredRows": 0,
        "NumberUnselectedRows": 0,
        "LoadBytes": 1327,
        "LoadTimeMs": 55,
        "BeginTxnTimeMs": 2,
        "StreamLoadPutTimeMs": 9,
        "ReadDataTimeMs": 0,
        "WriteDataTimeMs": 38,
        "CommitAndPublishTimeMs": 0
}


Streamload Response2:status: 200, resp msg: OK, resp content: {
        "TxnId": 11594008,
        "Label": "audit_20210521_102622_c4c848e087bd43aebce3a32b7c999ffe",
        "Status": "Fail",
        "Message": "already stopped, skip waiting for close. cancelled/!eos: : 1/0",
        "NumberTotalRows": 0,
        "NumberLoadedRows": 0,
        "NumberFilteredRows": 0,
        "NumberUnselectedRows": 0,
        "LoadBytes": 399,
        "LoadTimeMs": 104,
        "BeginTxnTimeMs": 0,
        "StreamLoadPutTimeMs": 0,
        "ReadDataTimeMs": 0,
        "WriteDataTimeMs": 103,
        "CommitAndPublishTimeMs": 0
}
0
#2IamStrangers回复于2021-05

应该和清理transaction的设置无关,这个需要针对某个不健康的 tablet,通过 show tablet 等命令看下tablet相关的副本情况。

有可能是是因为导入的 publish 任务过多,部分副本有延迟,导致部分副本的版本落后。

0
快速回复
TOP
切换版块