u011066523 2016-03-10 10:15 采纳率: 0%
浏览 3021

java 爬虫 新浪微博 webclient 模拟登陆 “登陆”按钮无效

用WebClient模拟新浪微博登陆,有js加载问题:
异常如下:
2016-03-10 18:06:19,311 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,025 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,059 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [v="0", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [cookie2="1cb93763dd883cf48427d2013efbb15b", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [t="e5f50b797c4e59839d80b1961ae45398", version:0, domain:.taobao.com, path:/, expiry:Wed Jun 08 18:06:21 CST 2016] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,205 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [_tb_token_="6v92BCkEPJskjf", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,249 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,331 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,340 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,358 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,395 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,412 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,451 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,533 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,728 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,809 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'text/javascript'.
2016-03-10 18:06:22,071 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:22,084 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [cna="HjVoD15v7R4CAdz4Edobp0iZ", version:0, domain:.mmstat.com, path:/, expiry:Sun Mar 08 18:06:22 CST 2026] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [sca="37e55bc2", version:0, domain:.mmstat.com, path:/, expiry:null] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [tbsa="a63b6f131cd4ff69ca5bf61d_1457604382_1", version:0, domain:.mmstat.com, path:/, expiry:null] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [atpsida="13075126e2ff6c8b00f358f8_1457604382", version:0, domain:.mmstat.com, path:/, expiry:Sun Mar 08 18:06:22 CST 2026] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:23,071 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:23,081 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,208 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,226 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,302 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'http://wbpctips.mobile.sina.cn/adfront/loginadv.php?posid=pos55dc4af3e7219&psid=PDPS000000025446&cip=220.248.17.218&cursor=15&eData=32.07,1&uid=1000959614999&callback=wbad_14576528474261&rnd=14577027909572', but got 'text/html'.
2016-03-10 18:06:51,313 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,343 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,435 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'http://login.sina.com.cn/sso/prelogin.php?entry=weibo&callback=sinaSSOController.preloginCallBack&su=&rsakt=mod&client=ssologin.js(v1.4.18)&_=1457604411336', but got 'text/html'.
2016-03-10 18:07:21,391 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:07:21,422 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:07:51,352 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] INFO [com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine] - Caught script exception

模拟点击“登陆”的a标签,预计因为js没有加载所以无法提交;a标签点击过后 还是返回的登陆界面!
模拟点击代码:
HtmlAnchor login_a = page.getFirstByXPath("//*[@id=\"pl_login_form\"]/div[2]/div[1]/div/a[2]");
login_a.click();

有谁知道什么解决不?
  • 写回答

2条回答

  • zxq82 2016-08-30 08:04
    关注

    可能是JS异步加载,导致数据还未加载完成,可以将进程休眼5秒钟,再执行后续操作。Thread.sleep(5000)

    评论

报告相同问题?

悬赏问题

  • ¥15 maple软件,用solve求反函数出现rootof,怎么办?
  • ¥65 汇编语言除法溢出问题
  • ¥15 Visual Studio问题
  • ¥15 state显示变量是字符串形式,但是仍然红色,无法引用,并显示类型不匹配
  • ¥20 求一个html代码,有偿
  • ¥100 关于使用MATLAB中copularnd函数的问题
  • ¥20 在虚拟机的pycharm上
  • ¥15 jupyterthemes 设置完毕后没有效果
  • ¥15 matlab图像高斯低通滤波
  • ¥15 针对曲面部件的制孔路径规划,大家有什么思路吗