u011066523 2016-03-10 10:15 采纳率: 0%
浏览 3024

java 爬虫 新浪微博 webclient 模拟登陆 “登陆”按钮无效

用WebClient模拟新浪微博登陆，有js加载问题：
异常如下：
2016-03-10 18:06:19,311 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,025 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,059 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [v="0", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [cookie2="1cb93763dd883cf48427d2013efbb15b", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [t="e5f50b797c4e59839d80b1961ae45398", version:0, domain:.taobao.com, path:/, expiry:Wed Jun 08 18:06:21 CST 2016] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,205 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [_tb_token_="6v92BCkEPJskjf", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,249 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,331 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,340 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,358 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,395 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,412 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,451 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,533 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,728 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,809 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'text/javascript'.
2016-03-10 18:06:22,071 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:22,084 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [cna="HjVoD15v7R4CAdz4Edobp0iZ", version:0, domain:.mmstat.com, path:/, expiry:Sun Mar 08 18:06:22 CST 2026] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [sca="37e55bc2", version:0, domain:.mmstat.com, path:/, expiry:null] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [tbsa="a63b6f131cd4ff69ca5bf61d_1457604382_1", version:0, domain:.mmstat.com, path:/, expiry:null] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [atpsida="13075126e2ff6c8b00f358f8_1457604382", version:0, domain:.mmstat.com, path:/, expiry:Sun Mar 08 18:06:22 CST 2026] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:23,071 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:23,081 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,208 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,226 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,302 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'http://wbpctips.mobile.sina.cn/adfront/loginadv.php?posid=pos55dc4af3e7219&psid=PDPS000000025446&cip=220.248.17.218&cursor=15&eData=32.07,1&uid=1000959614999&callback=wbad_14576528474261&rnd=14577027909572', but got 'text/html'.
2016-03-10 18:06:51,313 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,343 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,435 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'http://login.sina.com.cn/sso/prelogin.php?entry=weibo&callback=sinaSSOController.preloginCallBack&su=&rsakt=mod&client=ssologin.js(v1.4.18)&_=1457604411336', but got 'text/html'.
2016-03-10 18:07:21,391 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:07:21,422 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:07:51,352 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] INFO [com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine] - Caught script exception

模拟点击“登陆”的a标签，预计因为js没有加载所以无法提交；a标签点击过后 还是返回的登陆界面！
模拟点击代码：
HtmlAnchor login_a = page.getFirstByXPath("//*[@id=\"pl_login_form\"]/div[2]/div[1]/div/a[2]");
login_a.click();

有谁知道什么解决不？
  • 写回答

2条回答

  • zxq82 2016-08-30 08:04
    关注

    可能是JS异步加载，导致数据还未加载完成，可以将进程休眼5秒钟，再执行后续操作。Thread.sleep(5000)

    评论

报告相同问题？

悬赏问题

  • ¥15 如何在scanpy上做差异基因和通路富集？
  • ¥20 关于#硬件工程#的问题，请各位专家解答！
  • ¥15 关于#matlab#的问题：期望的系统闭环传递函数为G(s)=wn^2/s^2+2¢wn+wn^2阻尼系数¢=0.707，使系统具有较小的超调量
  • ¥15 FLUENT如何实现在堆积颗粒的上表面加载高斯热源
  • ¥30 截图中的mathematics程序转换成matlab
  • ¥15 动力学代码报错，维度不匹配
  • ¥15 Power query添加列问题
  • ¥50 Kubernetes&Fission&Eleasticsearch
  • ¥15 報錯：Person is not mapped，如何解決？
  • ¥15 c++头文件不能识别CDialog