2 u011066523 u011066523 于 2016.03.10 18:15 提问

java 爬虫 新浪微博 webclient 模拟登陆 “登陆”按钮无效

用WebClient模拟新浪微博登陆,有js加载问题:
异常如下:
2016-03-10 18:06:19,311 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,025 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,059 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [v="0", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [cookie2="1cb93763dd883cf48427d2013efbb15b", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,204 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [t="e5f50b797c4e59839d80b1961ae45398", version:0, domain:.taobao.com, path:/, expiry:Wed Jun 08 18:06:21 CST 2016] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,205 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [_tb_token_="6v92BCkEPJskjf", version:0, domain:.taobao.com, path:/, expiry:null] Illegal domain attribute "taobao.com". Domain of origin: "js1.t.sinajs.cn"
2016-03-10 18:06:21,249 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,331 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,340 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,358 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,395 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,412 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,451 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,533 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,728 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:21,809 [main] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'text/javascript'.
2016-03-10 18:06:22,071 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:22,084 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [cna="HjVoD15v7R4CAdz4Edobp0iZ", version:0, domain:.mmstat.com, path:/, expiry:Sun Mar 08 18:06:22 CST 2026] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [sca="37e55bc2", version:0, domain:.mmstat.com, path:/, expiry:null] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [tbsa="a63b6f131cd4ff69ca5bf61d_1457604382_1", version:0, domain:.mmstat.com, path:/, expiry:null] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:22,143 [main] WARN [org.apache.http.client.protocol.ResponseProcessCookies] - Cookie rejected [atpsida="13075126e2ff6c8b00f358f8_1457604382", version:0, domain:.mmstat.com, path:/, expiry:Sun Mar 08 18:06:22 CST 2026] Illegal domain attribute "mmstat.com". Domain of origin: "g.alicdn.com"
2016-03-10 18:06:23,071 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:23,081 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,208 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,226 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,302 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'http://wbpctips.mobile.sina.cn/adfront/loginadv.php?posid=pos55dc4af3e7219&psid=PDPS000000025446&cip=220.248.17.218&cursor=15&eData=32.07,1&uid=1000959614999&callback=wbad_14576528474261&rnd=14577027909572', but got 'text/html'.
2016-03-10 18:06:51,313 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,343 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:06:51,435 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'http://login.sina.com.cn/sso/prelogin.php?entry=weibo&callback=sinaSSOController.preloginCallBack&su=&rsakt=mod&client=ssologin.js(v1.4.18)&_=1457604411336', but got 'text/html'.
2016-03-10 18:07:21,391 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:07:21,422 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] WARN [com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl] - Obsolete content type encountered: 'application/x-javascript'.
2016-03-10 18:07:51,352 [JS executor for com.gargoylesoftware.htmlunit.WebClient@145a27f] INFO [com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine] - Caught script exception

模拟点击“登陆”的a标签,预计因为js没有加载所以无法提交;a标签点击过后 还是返回的登陆界面!
模拟点击代码:
HtmlAnchor login_a = page.getFirstByXPath("//*[@id=\"pl_login_form\"]/div[2]/div[1]/div/a[2]");
login_a.click();

有谁知道什么解决不?

2个回答

zxq82
zxq82   2016.08.30 16:04

可能是JS异步加载,导致数据还未加载完成,可以将进程休眼5秒钟,再执行后续操作。Thread.sleep(5000)

meistime
meistime   2017.05.11 09:50

是没有获取到登录按钮,试试这样获取 HtmlSubmitInput submitInput = form.getInputByValue("登 录");

Csdn user default icon
上传中...
上传图片
插入图片
准确详细的回答,更有利于被提问者采纳,从而获得C币。复制、灌水、广告等回答会被删除,是时候展现真正的技术了!