膜拜钉宫大人 2019-12-03 17:08 采纳率: 100%
浏览 353
已采纳

请问如何处理这样的binlog日志文件?

1.拿到手的是这样的一个log文件xxx.log:

orderPlan_test
b'{"eventType":"INSERT","schemaName":"qixiao_test","rowAfter":{"update_time":1575291953000,"order_plan_id":63000036509,"create_time":1575291953000,"click_through_json":"{\\"landingPage\\":\\"http://www.baidu.com\\",\\"landingPageFlag\\":0,\\"appName\\":\\"tgnr120205\\",\\"account\\":\\"tgnr120205\\"}"},"tableName":"qx_order_plan_click_through","timestamp":1575291953000}'
creative_test
b'{"rowBefore":{"update_time":1575291971000,"create_time":1575291971000,"submit_time":1575291971000,"name":"\xe6\x88\x91\xe7\x9a\x84\xe5\x88\x9b\xe6\x84\x8f20191202210554","prod_status":0,"landing_page":"http://www.baidu.com","id":64000017479,"prod_fail_type":0,"landing_page_flag":0,"material_json":"{\\"name\\":\\"\xe6\x88\x91\xe7\x9a\x84\xe5\x88\x9b\xe6\x84\x8f20191202210554\\",\\"url\\":1585,\\"title\\":\\"\xe5\xb9\xbf\xe5\x91\x8a\xe6\x96\x87\xe6\xa1\x88123\\",\\"previewData\\":{\\"buttonTitle\\":\\"\xe4\xba\x86\xe8\xa7\xa3\xe8\xaf\xa6\xe6\x83\x85\\"},\\"events\\":{\\"impressionUrl\\":null,\\"clickUrl\\":null},\\"feedId\\":\\"1569467048\\",\\"clickThrough\\":{\\"landingPage\\":\\"http://www.baidu.com\\",\\"landingPageFlag\\":0,\\"appName\\":\\"tgnr120205\\",\\"account\\":\\"tgnr120205\\"}}","status":1},"eventType":"UPDATE","schemaName":"qixiao_test","rowAfter":{"update_time":1575291971000,"create_time":1575291971000,"submit_time":1575291971000,"name":"\xe6\x88\x91\xe7\x9a\x84\xe5\x88\x9b\xe6\x84\x8f20191202210554","prod_status":1,"landing_page":"http://www.baidu.com","id":64000017479,"prod_fail_type":0,"landing_page_flag":0,"material_json":"{\\"name\\":\\"\xe6\x88\x91\xe7\x9a\x84\xe5\x88\x9b\xe6\x84\x8f20191202210554\\",\\"url\\":1585,\\"title\\":\\"\xe5\xb9\xbf\xe5\x91\x8a\xe6\x96\x87\xe6\xa1\x88123\\",\\"previewData\\":{\\"buttonTitle\\":\\"\xe4\xba\x86\xe8\xa7\xa3\xe8\xaf\xa6\xe6\x83\x85\\"},\\"events\\":{\\"impressionUrl\\":null,\\"clickUrl\\":null},\\"feedId\\":\\"1569467048\\",\\"clickThrough\\":{\\"landingPage\\":\\"http://www.baidu.com\\",\\"landingPageFlag\\":0,\\"appName\\":\\"tgnr120205\\",\\"account\\":\\"tgnr120205\\"}}","status":1},"tableName":"qx_creative_staging","timestamp":1575291971000}'
orderPlan_test
b'{"eventType":"INSERT","schemaName":"qixiao_test","rowAfter":{"update_time":1575291953000,"order_plan_id":63000036509,"create_time":1575291953000,"target_json":"[{\\"id\\":32,\\"type\\":1,\\"isAnti\\":false},{\\"id\\":33,\\"type\\":1,\\"isAnti\\":false},{\\"id\\":1,\\"type\\":210,\\"isAnti\\":false},{\\"id\\":3,\\"type\\":210,\\"isAnti\\":false},{\\"id\\":4,\\"type\\":210,\\"isAnti\\":false},{\\"id\\":2,\\"type\\":210,\\"isAnti\\":false}]"},"tableName":"qx_order_plan_target","timestamp":1575291953000}'
………………

2.明显可见每隔一行是个json格式,我现在希望将这个解析成dic。如果手动赋值,显然是可行的。
图片说明
3.但是若果我用with open(filename,'r') 打开binlog文件,每一行结果是str格式,无法decode。请问该如何解决?图片说明
4.如果用with open(filename,'r+b')打开还是不行
图片说明
5.1L回答的方法的问题是rowBefore,rowAfter里的中文编码,以及这两个key里嵌套的json格式不对

  • 写回答

1条回答 默认 最新

  • 澈丹丶 2019-12-03 18:49
    关注
    import ast
    with open('test_log.log', 'r') as f:
        for row in f.readlines():
            if row.startswith("b'{"):
                res = ast.literal_eval(row.lstrip("b'").rstrip().rstrip("'").replace("\\\\", "\\"))
    
                print(res)
    
    
    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 #MATLAB仿真#车辆换道路径规划
  • ¥15 java 操作 elasticsearch 8.1 实现 索引的重建
  • ¥15 数据可视化Python
  • ¥15 要给毕业设计添加扫码登录的功能!!有偿
  • ¥15 kafka 分区副本增加会导致消息丢失或者不可用吗?
  • ¥15 微信公众号自制会员卡没有收款渠道啊
  • ¥100 Jenkins自动化部署—悬赏100元
  • ¥15 关于#python#的问题:求帮写python代码
  • ¥20 MATLAB画图图形出现上下震荡的线条
  • ¥15 关于#windows#的问题:怎么用WIN 11系统的电脑 克隆WIN NT3.51-4.0系统的硬盘