Python中如何爬取动态网页

Question

请教大神，在Python中利用正则爬取用js生成的数据，爬取网易新闻中的评论

Answer 1

你用python 网络包，多次请求，然后再 解析源码
例如:
网易评论页：
http://comment.news.163.com/news2_bbs/C4IUUH5K000189FH.html

加载完页面后，有个再次广告请求(这里你自己仔细看下 那个请求是请求的评论内容)：
https://tns.simba.taobao.com/?name=itemdsp&o=j&spm=a2e01.7920152.0.0.cnNa7l&templateId=23032&url=http%3A%2F%2Fcomment.news.163.com%2Fnews2_bbs%2FC4IUUH5K000189FH.html&iswt=1&pid=tt_16143929_2253494_46970876&refpid=tt_16143929_2253494_46970876&refpos=%2Cn%2Ci&adx_type=0&pvid=0a672a0f00005815e4f10693007aaebc_0&ps_id=ab5f5cad819d724eb5907a17109ab670&fl=7&tanxdspv=http%3A%2F%2Frdstat.tanx.com%2Ftrd%3Ff%3D%26k%3Da09e279ad7f7a12a%26p%3Dmm_16143929_2253494_46970876%26pvid%3D0a672a0f00005815e4f10693007aaebc%26s%3D300x250%26d%3D17534123%26t%3D1477829873&count=28&p4p=jsonp0iuwlr2n2

网易返回
jsonp0iuwlr2n2 =[{"TITLE":"Crocs\u5b98\u65b9\u65d7\u8230\u5e97!\u5e03\u9a86\u683c\u9c7c\u5634\u978b\u5761
\u8ddf\u51c9\u978b","RESOURCEID":"44846141610","GOODSPRICE":49900,"LOCATION":"\u4e0a\u6d77","TBGOODSLINK"
:"https:\/\/asearch.alicdn.com\/bao\/uploaded\/i1\/199250309690686146\/TB2.WiYbmiK.eBjSZFsXXbxZpXa_!
!0-saturn_solar.jpg_sum.jpg","WANGWANGID":"crocs\u5b98\u65b9\u65d7\u8230","EURL":"https:\/\/srd.simba

对应中文
官方旗舰店!布骆格鱼嘴鞋坡