java.io.IOException: Server returned HTTP response code: 502 for URL:XXXXXX 5C

我在从系统数据库中拿到20W张图片的url,然后下载到本地的时候部分图片下载出现了这个错误:
java.io.IOException: Server returned HTTP response code: 502 for URL:XXXXX
代码:

public static String downLoadFromUrl(String urlStr, String fileName, String savePath) {
        JSONObject response = new JSONObject();
        JSONArray failds = new JSONArray();
        try {

            URL url = new URL(urlStr);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            // 设置超时间为3秒
            conn.setConnectTimeout(10 * 1000);
            // 防止屏蔽程序抓取而返回403错误
            conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");

            // 得到输入流
            InputStream inputStream = conn.getInputStream();
            // 获取自己数组
            byte[] getData = readInputStream(inputStream);

            // 文件保存位置
            File saveDir = new File(savePath);
            if (!saveDir.exists()) {
                saveDir.mkdirs();
                logger.info("创建文件夹:" + saveDir.exists());
            }
            File file = new File(saveDir + File.separator + fileName);
            FileOutputStream fos = new FileOutputStream(file);
            fos.write(getData);
            if (fos != null) {
                fos.close();
            }
            if (inputStream != null) {
                inputStream.close();
            }
            return saveDir + File.separator + fileName;
        } catch (Exception e) {
            logger.error(">>>>>>image download faild,image url:" + urlStr + ",fileName:"+ fileName);
            failds.add(urlStr);
            e.printStackTrace();
        }
        response.put("faild",failds);
        return response.toString();

    }

    public InputStream getConnectionStream(String fullUrl) throws IOException {
        HttpClient client = new DefaultHttpClient();
        HttpGet request = new HttpGet(fullUrl);
        HttpResponse response = client.execute(request);
        InputStream is = response.getEntity().getContent();

        return is;
    }

    public static byte[] readInputStream(InputStream inputStream) throws IOException {
        byte[] buffer = new byte[1024];
        int len = 0;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        while ((len = inputStream.read(buffer)) != -1) {
            bos.write(buffer, 0, len);
        }
        bos.close();
        return bos.toByteArray();
    }

下载图片就是循环调用这块代码,我应该如何优化才能避免这个错?

1个回答

其实主要原因应该是你20w个请求循环发送,循环的效率很快,其实可以看做发送量多于服务器可以处理的数据量。这样就会造成请求阻塞,部分连接超时未响应。你需要在发送下载请求的时候进行延迟请求。可以考虑手动写一个连接池限定同时发送连接数量。或者如果你不考虑下载时间的问题可以请求响应以后再去请求一个新的,这样效率会下降,但能保证请求能正常响应。

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
java.io.IOException: Server returned HTTP response code: 505 for URL

碰到个很奇怪的问题: 代码如下: URL url = new URL(urlStr); URLConnection hpCon = url.openConnection(); InputStream in = hpCon.getInputStream(); 然后我发送的一个URL是 : http://localhost:8000/account/accountTo.query?param=SR4A V12f 报错:java.io.IOException: Server returned HTTP response code: 505 for URL: http://localhost:8000/account/accountTo.query?param=SR4A V12f 这个URL在任何的浏览器都有数据显示,说明和服务器的通信正常。 这里的参数 param=SR4A V12f 中间有个空格 ,我尝试了如果没有空格,访问正常,能够得到数据。是不是因为多了个空格而出问题了?

Server returned HTTP response code: 403 for URL:

Java代码中访问链接提示java.io.IOException: Server returned HTTP response code: 403 for URL: 错误 ``` URL url = new URL(requestUrl); httpUrlConn = (HttpURLConnection) url.openConnection(); httpUrlConn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)"); ``` 已经设置了User-Agent还是不行 ``` java.io.IOException: Server returned HTTP response code: 403 for URL: http://www.dianping.com/shop/93433662 at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at com.Utils.MyFunction.httpRequest(MyFunction.java:37) at com.net.service.DianPing.getDianPing(DianPing.java:41) at com.net.service.TodayTemperatureService.getDianPing(TodayTemperatureService.java:164) at com.net.service.TodayTemperatureService.main(TodayTemperatureService.java:174) ```

Logstash与tcp,错误:java.io.IOException:连接被对等方重置

<div class="post-text" itemprop="text"> <p>我想在我的GolangApp中使用logstash。</p> <p>/etc/logstash/conf.d/first-pipeline.conf </p> <pre><code>input { tcp { port =&gt; 5959 codec =&gt; json } } #filter {} output { elasticsearch { hosts =&gt; [ "localhost:9200" ] } } </code></pre> <p>和用于运行logstash的命令:</p> <p>/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/first-pipeline.conf --path.settings=/etc/logstash</p> <pre><code> Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2018-12-09T09:11:14,984][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2018-12-09T09:11:14,995][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=&gt;"6.5.1"} [2018-12-09T09:11:16,968][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=&gt;"main", "pipeline.workers"=&gt;4, "pipeline.batch.size"=&gt;125, "pipeline.batch.delay"=&gt;50} [2018-12-09T09:11:17,347][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=&gt;{:removed=&gt;[], :added=&gt;[http://localhost:9200/]}} [2018-12-09T09:11:17,356][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=&gt;http://localhost:9200/, :path=&gt;"/"} [2018-12-09T09:11:17,589][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=&gt;"http://localhost:9200/"} [2018-12-09T09:11:17,655][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=&gt;6} [2018-12-09T09:11:17,660][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=&gt;6} [2018-12-09T09:11:17,692][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=&gt;"LogStash::Outputs::ElasticSearch", :hosts=&gt;["//localhost:9200"]} [2018-12-09T09:11:17,730][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=&gt;nil} [2018-12-09T09:11:17,786][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=&gt;{"template"=&gt;"logstash-*", "version"=&gt;60001, "settings"=&gt;{"index.refresh_interval"=&gt;"5s"}, "mappings"=&gt;{"_default_"=&gt;{"dynamic_templates"=&gt;[{"message_field"=&gt;{"path_match"=&gt;"message", "match_mapping_type"=&gt;"string", "mapping"=&gt;{"type"=&gt;"text", "norms"=&gt;false}}}, {"string_fields"=&gt;{"match"=&gt;"*", "match_mapping_type"=&gt;"string", "mapping"=&gt;{"type"=&gt;"text", "norms"=&gt;false, "fields"=&gt;{"keyword"=&gt;{"type"=&gt;"keyword", "ignore_above"=&gt;256}}}}}], "properties"=&gt;{"@timestamp"=&gt;{"type"=&gt;"date"}, "@version"=&gt;{"type"=&gt;"keyword"}, "geoip"=&gt;{"dynamic"=&gt;true, "properties"=&gt;{"ip"=&gt;{"type"=&gt;"ip"}, "location"=&gt;{"type"=&gt;"geo_point"}, "latitude"=&gt;{"type"=&gt;"half_float"}, "longitude"=&gt;{"type"=&gt;"half_float"}}}}}}}} [2018-12-09T09:11:17,809][INFO ][logstash.inputs.tcp ] Automatically switching from json to json_lines codec {:plugin=&gt;"tcp"} [2018-12-09T09:11:17,862][INFO ][logstash.inputs.tcp ] Starting tcp input listener {:address=&gt;"0.0.0.0:5959", :ssl_enable=&gt;"false"} [2018-12-09T09:11:18,097][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=&gt;"main", :thread=&gt;"#&lt;Thread:0x42d68f8e run&gt;"} [2018-12-09T09:11:18,157][INFO ][logstash.agent ] Pipelines running {:count=&gt;1, :running_pipelines=&gt;[:main], :non_running_pipelines=&gt;[]} [2018-12-09T09:11:18,329][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=&gt;9600} </code></pre> <p>Code in golang:</p> <pre><code>import ( "encoding/json" "fmt" "github.com/heatxsink/go-logstash" "time" ) func main() { l := logstash.New("0.0.0.0", 5959, 5) _, err := l.Connect() if err != nil { fmt.Println(err) } dataMap := map[string]int{"apple": 5, "lettuce": 7} jsonMap, _ := json.Marshal(dataMap) err = l.Writeln(string(jsonMap)) if err != nil { fmt.Println(err) } } </code></pre> <p>当我试图请求登录应用程序时,终端会显示这个错误:</p> <blockquote> <p>[2018-12-09T09:12:41,954][ERROR][logstash.inputs.tcp ] Error in Netty pipeline: java.io.IOException: Connection reset by peer</p> </blockquote> <p>所有的东西都在我的本地系统中。请帮帮我!</p> </div>

Server returned HTTP response code: 400 for URL

代码如下 String s = HttpHelper.doGet(articleVO.getImg()+"?x-oss-process=image/info"); 用阿里云的info获取图片的信息 路径没有问题 可以打开 但是却报错 java.io.IOException: Server returned HTTP response code: 400 for URL: 不知道是什么原因 进行去除空格之后还是出现这种一样报异常 请大佬帮忙看一下是什么原因

cas 单点登录 在没有改 服务器返回用户多个信息之前 还能用 修改完 就报这错

严重: Server returned HTTP response code: 500 for URL: https://sso.wsria.com:8443/cas/serviceValidate?ticket=ST-3-4iBEtP4rEcrO2GOVD6gF-cas01.example.org&service=http%3A%2F%2F127.0.0.1%3A8010%2Fclient%2F%3Bjsessionid%3DED18610650116B80E39AD18DB537DB61 java.io.IOException: Server returned HTTP response code: 500 for URL: https://sso.wsria.com:8443/cas/serviceValidate?ticket=ST-3-4iBEtP4rEcrO2GOVD6gF-cas01.example.org&service=http%3A%2F%2F127.0.0.1%3A8010%2Fclient%2F%3Bjsessionid%3DED18610650116B80E39AD18DB537DB61 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1627) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) at org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:326) at org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:305) at org.jasig.cas.client.validation.AbstractCasProtocolUrlBasedTicketValidator.retrieveResponseFromServer(AbstractCasProtocolUrlBasedTicketValidator.java:50) at org.jasig.cas.client.validation.AbstractUrlBasedTicketValidator.validate(AbstractUrlBasedTicketValidator.java:207) at org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:169) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.jasig.cas.client.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:76) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) 六月 21, 2017 4:03:14 下午 org.apache.catalina.core.StandardWrapperValve invoke 严重: Servlet.service() for servlet [jsp] in context with path [/client] threw exception java.lang.RuntimeException: java.io.IOException: Server returned HTTP response code: 500 for URL: https://sso.wsria.com:8443/cas/serviceValidate?ticket=ST-3-4iBEtP4rEcrO2GOVD6gF-cas01.example.org&service=http%3A%2F%2F127.0.0.1%3A8010%2Fclient%2F%3Bjsessionid%3DED18610650116B80E39AD18DB537DB61 at org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:341) at org.jasig.cas.client.util.CommonUtils.getResponseFromServer(CommonUtils.java:305) at org.jasig.cas.client.validation.AbstractCasProtocolUrlBasedTicketValidator.retrieveResponseFromServer(AbstractCasProtocolUrlBasedTicketValidator.java:50) at org.jasig.cas.client.validation.AbstractUrlBasedTicketValidator.validate(AbstractUrlBasedTicketValidator.java:207) at org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:169) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.jasig.cas.client.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:76) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)

Server returned HTTP response code: 403 for URL:

出现这种问题很头痛,不知道是不是访问不到对端服务,还是有其它问题 2017-3-21 9:05:03 com.sun.xml.ws.server.sei.TieHandler createResponse 严重: 无法访问位于以下位置的 WSDL: http://xxxxxxx?wsdl。该操作失败并显示: Server returned HTTP response code: 403 for URL: http://xxxxxx?wsdl。 javax.xml.ws.WebServiceException: 无法访问位于以下位置的 WSDL: http:xxxxxxxx?wsdl。该操作失败并显示: Server returned HTTP response code: 403 for URL: xxxx

使用HttpURLConnection 返回 HTTP response code: 500

try { // 创建SSLContext对象,并使用我们指定的信任管理器初始化 TrustManager[] tm = { new MyX509TrustManager() }; SSLContext sslContext = SSLContext.getInstance("SSL", "SunJSSE"); sslContext.init(null, tm, new java.security.SecureRandom()); // 从上述SSLContext对象中得到SSLSocketFactory对象 SSLSocketFactory ssf = sslContext.getSocketFactory(); URL url = new URL(requestUrl); HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); conn.setSSLSocketFactory(ssf); conn.setRequestMethod(requestMethod); conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)"); conn.setDoOutput(true); conn.setDoInput(true); conn.setUseCaches(false); // 设置请求方式(GET/POST) // 当outputStr不为null时向输出流写数据 if (null != outputStr) { OutputStream outputStream = conn.getOutputStream(); // 注意编码格式 outputStream.write(outputStr.getBytes("UTF-8")); outputStream.close(); } // 从输入流读取返回内容 InputStream inputStream = conn.getInputStream(); InputStreamReader inputStreamReader = new InputStreamReader(inputStream, "utf-8"); BufferedReader bufferedReader = new BufferedReader(inputStreamReader); String str = null; StringBuffer buffer = new StringBuffer(); while ((str = bufferedReader.readLine()) != null) { buffer.append(str); 程序执行到InputStream inputStream = conn.getInputStream();抛出异常 java.io.IOException: Server returned HTTP response code: 500 我发现conn对象的method属性是get,和这有关系吗

NC启动客户端时候报的错误,急急急急

run in local application. 错误: Exception caught while download server code version, try again... 错误: Exception caught while download server code version, try again... 错误: Exception caught while download server code version, try again... 错误: Exception caught while download server code version, try again... URL:http://127.0.0.1:80 java.lang.RuntimeException: Exception caught while download server code version: at nc.bs.framework.codesync.client.DownloadService.downloadServerVID(DownloadService.java:96) at nc.bs.framework.codesync.client.DownloadService.initVID(DownloadService.java:29) at nc.bs.framework.codesync.client.LocalPackageVIDCache.initPackageVIDCache(LocalPackageVIDCache.java:10) at nc.ui.sm.login.AppletContainer.initCodeCacheParams(AppletContainer.java:552) at nc.ui.sm.login.AppletContainer.init(AppletContainer.java:395) at nc.starter.test.JStarter.StartNC(JStarter.java:128) at nc.starter.test.JStarter.main(JStarter.java:389) Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL: http://127.0.0.1:80/CodeSynServlet at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1152) at nc.bs.framework.codesync.client.DownloadService.post(DownloadService.java:279) at nc.bs.framework.codesync.client.DownloadService.downloadServerVID(DownloadService.java:69) ... 6 more 这是错误代码!

请教Java 网络编程 Server returned HTTP response code: 400

下面是简单的代码: URL url = new URL(uri); URLConnection rulConnection = url.openConnection(); HttpURLConnection httpUrlConnection = (HttpURLConnection) rulConnection; httpUrlConnection.setDoInput(true); httpUrlConnection.setUseCaches(false); httpUrlConnection.setRequestMethod("POST"); httpUrlConnection.connect(); InputStreamReader isr =new InputStreamReader(httpUrlConnection.getInputStream(),"GB2312"); 如果我的uri 是http://search.jd.com/Search?keyword=华为&enc=utf-8&area=2 没问题,能够读取网页内容 可是换成http://item.jd.com/827017.html 就会抛异常 java.io.IOException: Server returned HTTP response code: 400 for URL: http://item.jd.com/827017.html at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) at ReadURL.getBufferedFromURL(ReadURL.java:12) at ReadURL.getItemDetail(ReadURL.java:29) at ReadURL.main(ReadURL.java:83) java.lang.NullPointerException 请问这是为什么同一个网站的不同网址,为什么有的能访问,有的抛异常?

https的图片通过java的URL类下载不了,报451错误

图片地址如下:https://swmedia-4cd6.kxcdn.com/media/catalogue/Aruba/Postage-stamps/A-s.jpg 每当执行下面的语句,到InputStream is = conn.getInputStream();的时候就会报如下错误:java.io.IOException: Server returned HTTP response code: 451 for URL 代码如下: URL url = new URL(src); HttpURLConnection conn = (HttpURLConnection)url.openConnection(); conn.setDoInput(true); conn.setRequestMethod("GET"); InputStream is = conn.getInputStream();

who can help me solve the http 401 error

when use drools guvnor tools by configuration,it auto generate a binary package link url, like "http://localhost:8080/guvnor/rest/packages/knowledgebaseSource/binary " then i use it as a source to call ResourceFactory.newUrlResource(Uri uri) result is http response 401 error Detail infomation is :Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: http://localhost:8080/guvnor/rest/packages/knowledgebaseSource/binary that's say i have no auth.....

使用IDEA运行maven插件没问题,使用mvn命令运行插件报错?

使用swagger2markup插件,在IDEA里可以直接运行插件,生成asciidoc文件。 ![图片说明](https://img-ask.csdn.net/upload/201902/28/1551323694_720508.png) IDEA运行的默认执行的命令: *** mvn io.github.swagger2markup:swagger2markup-maven-plugin:1.2.0:convertSwagger2markup。 因为要编写技术文档,所以我需要mvn命令,但用PowerShell执行mvn的时候就报错了。* ``` java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost:8032/v2/api-docs at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at io.swagger.parser.util.RemoteUrl.urlToString(RemoteUrl.java:130) at io.swagger.parser.Swagger20Parser.read(Swagger20Parser.java:81) 。。。。。。。 ``` 项目是运行启动了的,8032/v2/api/docs能得到json数据

微博粉丝服务平台扩展开发过程中遇到请求微博接口的400和未知源问题!

基于微博粉丝服务平台扩展开发过程中遇到请求微博接口的400和未知源问题! 希望各路大侠前来搭救!多谢! java.io.IOException: Server returned HTTP response code: 400 for URL: https://m.api.weibo.com/2/messages/menu/show.json?access_token=(此处略去) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)

HttpsURLConnection getinputstream ioexception

java.io.IOException: Server returned HTTP response code: 500 for URL: https://202.108.60.54:8443/users/login at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1627) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) at com.unplatform.comm.HttpsPost.post(HttpsPost.java:138) at com.unplatform.vis.util.InsurancePostUtil.logIn(InsurancePostUtil.java:69) at com.unplatform.vis.web.controller.ExechangeDataController.exechangeData(ExechangeDataController.java:57) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:863) at javax.servlet.http.HttpServlet.service(HttpServlet.java:710) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:745)

sqoop1.99.6启动job报错

用start job -jid 1报错 Exception has occurred during processing command Exception: org.apache.sqoop.common.SqoopException Message: CLIENT_0001:Server has returned exception Stack trace: at org.apache.sqoop.client.request.ResourceRequest (ResourceRequest.java:129) at org.apache.sqoop.client.request.ResourceRequest (ResourceRequest.java:179) at org.apache.sqoop.client.request.JobResourceRequest (JobResourceRequest.java:112) at org.apache.sqoop.client.request.SqoopResourceRequests (SqoopResourceRequests.java:157) at org.apache.sqoop.client.SqoopClient (SqoopClient.java:452) at org.apache.sqoop.shell.StartJobFunction (StartJobFunction.java:80) at org.apache.sqoop.shell.SqoopFunction (SqoopFunction.java:51) at org.apache.sqoop.shell.SqoopCommand (SqoopCommand.java:135) at org.apache.sqoop.shell.SqoopCommand (SqoopCommand.java:111) at org.codehaus.groovy.tools.shell.Command$execute (null:-1) at org.codehaus.groovy.runtime.callsite.CallSiteArray (CallSiteArray.java:42) at org.codehaus.groovy.tools.shell.Command$execute (null:-1) at org.codehaus.groovy.tools.shell.Shell (Shell.groovy:101) at org.codehaus.groovy.tools.shell.Groovysh (Groovysh.groovy:-1) at sun.reflect.GeneratedMethodAccessor23 (null:-1) at sun.reflect.DelegatingMethodAccessorImpl (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method (Method.java:498) at org.codehaus.groovy.reflection.CachedMethod (CachedMethod.java:90) at groovy.lang.MetaMethod (MetaMethod.java:233) at groovy.lang.MetaClassImpl (MetaClassImpl.java:1054) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter (ScriptBytecodeAdapter.java:128) at org.codehaus.groovy.tools.shell.Groovysh (Groovysh.groovy:173) at sun.reflect.GeneratedMethodAccessor22 (null:-1) at sun.reflect.DelegatingMethodAccessorImpl (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method (Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce (PogoMetaMethodSite.java:267) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite (PogoMetaMethodSite.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite (AbstractCallSite.java:141) at org.codehaus.groovy.tools.shell.Groovysh (Groovysh.groovy:121) at org.codehaus.groovy.tools.shell.Shell (Shell.groovy:114) at org.codehaus.groovy.tools.shell.Shell$leftShift$0 (null:-1) at org.codehaus.groovy.tools.shell.ShellRunner (ShellRunner.groovy:88) at org.codehaus.groovy.tools.shell.InteractiveShellRunner (InteractiveShellRunner.groovy:-1) at sun.reflect.GeneratedMethodAccessor20 (null:-1) at sun.reflect.DelegatingMethodAccessorImpl (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method (Method.java:498) at org.codehaus.groovy.reflection.CachedMethod (CachedMethod.java:90) at groovy.lang.MetaMethod (MetaMethod.java:233) at groovy.lang.MetaClassImpl (MetaClassImpl.java:1054) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter (ScriptBytecodeAdapter.java:128) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter (ScriptBytecodeAdapter.java:148) at org.codehaus.groovy.tools.shell.InteractiveShellRunner (InteractiveShellRunner.groovy:100) at sun.reflect.GeneratedMethodAccessor19 (null:-1) at sun.reflect.DelegatingMethodAccessorImpl (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method (Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce (PogoMetaMethodSite.java:267) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite (PogoMetaMethodSite.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite (AbstractCallSite.java:137) at org.codehaus.groovy.tools.shell.ShellRunner (ShellRunner.groovy:57) at org.codehaus.groovy.tools.shell.InteractiveShellRunner (InteractiveShellRunner.groovy:-1) at sun.reflect.NativeMethodAccessorImpl (NativeMethodAccessorImpl.java:-2) at sun.reflect.NativeMethodAccessorImpl (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method (Method.java:498) at org.codehaus.groovy.reflection.CachedMethod (CachedMethod.java:90) at groovy.lang.MetaMethod (MetaMethod.java:233) at groovy.lang.MetaClassImpl (MetaClassImpl.java:1054) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter (ScriptBytecodeAdapter.java:128) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter (ScriptBytecodeAdapter.java:148) at org.codehaus.groovy.tools.shell.InteractiveShellRunner (InteractiveShellRunner.groovy:66) at java_lang_Runnable$run (null:-1) at org.codehaus.groovy.runtime.callsite.CallSiteArray (CallSiteArray.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite (AbstractCallSite.java:108) at org.codehaus.groovy.runtime.callsite.AbstractCallSite (AbstractCallSite.java:112) at org.codehaus.groovy.tools.shell.Groovysh (Groovysh.groovy:463) at org.codehaus.groovy.tools.shell.Groovysh (Groovysh.groovy:402) at org.apache.sqoop.shell.SqoopShell (SqoopShell.java:130) Caused by: Exception: org.apache.sqoop.common.SqoopException Message: GENERIC_HDFS_CONNECTOR_0007:Invalid output directory - Unexpected exception Stack trace: at org.apache.sqoop.connector.hdfs.HdfsToInitializer (HdfsToInitializer.java:71) at org.apache.sqoop.connector.hdfs.HdfsToInitializer (HdfsToInitializer.java:35) at org.apache.sqoop.driver.JobManager (JobManager.java:449) at org.apache.sqoop.driver.JobManager (JobManager.java:373) at org.apache.sqoop.driver.JobManager (JobManager.java:276) at org.apache.sqoop.handler.JobRequestHandler (JobRequestHandler.java:380) at org.apache.sqoop.handler.JobRequestHandler (JobRequestHandler.java:116) at org.apache.sqoop.server.v1.JobServlet (JobServlet.java:96) at org.apache.sqoop.server.SqoopProtocolServlet (SqoopProtocolServlet.java:79) at javax.servlet.http.HttpServlet (HttpServlet.java:646) at javax.servlet.http.HttpServlet (HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:206) at org.apache.hadoop.security.authentication.server.AuthenticationFilter (AuthenticationFilter.java:644) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter (DelegationTokenAuthenticationFilter.java:304) at org.apache.hadoop.security.authentication.server.AuthenticationFilter (AuthenticationFilter.java:592) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve (StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve (StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve (StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve (ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve (StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter (CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor (Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler (Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker (JIoEndpoint.java:489) at java.lang.Thread (Thread.java:748) Caused by: Exception: java.io.IOException Message: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length; Host Details : local host is: "node01/192.168.65.100"; destination host is: "node01":9870; Stack trace: at org.apache.hadoop.net.NetUtils (NetUtils.java:818) at org.apache.hadoop.ipc.Client (Client.java:1549) at org.apache.hadoop.ipc.Client (Client.java:1491) at org.apache.hadoop.ipc.Client (Client.java:1388) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker (ProtobufRpcEngine.java:233) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker (ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy19 (null:-1) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB (ClientNamenodeProtocolTranslatorPB.java:907) at sun.reflect.NativeMethodAccessorImpl (NativeMethodAccessorImpl.java:-2) at sun.reflect.NativeMethodAccessorImpl (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method (Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler (RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call (RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call (RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call (RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler (RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy20 (null:-1) at org.apache.hadoop.hdfs.DFSClient (DFSClient.java:1666) at org.apache.hadoop.hdfs.DistributedFileSystem$29 (DistributedFileSystem.java:1576) at org.apache.hadoop.hdfs.DistributedFileSystem$29 (DistributedFileSystem.java:1573) at org.apache.hadoop.fs.FileSystemLinkResolver (FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem (DistributedFileSystem.java:1588) at org.apache.hadoop.fs.FileSystem (FileSystem.java:1683) at org.apache.sqoop.connector.hdfs.HdfsToInitializer (HdfsToInitializer.java:58) at org.apache.sqoop.connector.hdfs.HdfsToInitializer (HdfsToInitializer.java:35) at org.apache.sqoop.driver.JobManager (JobManager.java:449) at org.apache.sqoop.driver.JobManager (JobManager.java:373) at org.apache.sqoop.driver.JobManager (JobManager.java:276) at org.apache.sqoop.handler.JobRequestHandler (JobRequestHandler.java:380) at org.apache.sqoop.handler.JobRequestHandler (JobRequestHandler.java:116) at org.apache.sqoop.server.v1.JobServlet (JobServlet.java:96) at org.apache.sqoop.server.SqoopProtocolServlet (SqoopProtocolServlet.java:79) at javax.servlet.http.HttpServlet (HttpServlet.java:646) at javax.servlet.http.HttpServlet (HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:206) at org.apache.hadoop.security.authentication.server.AuthenticationFilter (AuthenticationFilter.java:644) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter (DelegationTokenAuthenticationFilter.java:304) at org.apache.hadoop.security.authentication.server.AuthenticationFilter (AuthenticationFilter.java:592) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain (ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve (StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve (StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve (StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve (ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve (StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter (CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor (Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler (Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker (JIoEndpoint.java:489) at java.lang.Thread (Thread.java:748) Caused by: Exception: java.lang.Throwable Message: RPC response exceeds maximum data length Stack trace: at org.apache.hadoop.ipc.Client$IpcStreams (Client.java:1864) at org.apache.hadoop.ipc.Client$Connection (Client.java:1183) at org.apache.hadoop.ipc.Client$Connection (Client.java:1079) 哪位大侠帮忙看看:主要应该是这句 Caused by: Exception: java.io.IOException Message: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length; Host Details : local host is: "node01/192.168.65.100"; destination host is: "node01":9870; 但不知道问题出在哪里 我的link配置: From database configuration Schema name: mysql Table name: help_topic Table SQL statement: Table column names: Partition column name: Null value allowed for the partition column: Boundary query: Incremental read Check column: Last value: To HDFS configuration Override null value: Null value: Output format: 0 : TEXT_FILE 1 : SEQUENCE_FILE Choose: 0 Compression format: 0 : NONE 1 : DEFAULT 2 : DEFLATE 3 : GZIP 4 : BZIP2 5 : LZO 6 : LZ4 7 : SNAPPY 8 : CUSTOM Choose: 0 Custom compression format: Output directory: hdfs://node01:9870/sqoop Append mode: Throttling resources

关于cas使用saml协议验证问题

错误代码如下: HTTP Status 500 – Internal Server Error Type Exception Report Message IO error sending HTTP request to /samlValidate Description The server encountered an unexpected condition that prevented it from fulfilling the request. Exception java.lang.RuntimeException: IO error sending HTTP request to /samlValidate org.jasig.cas.client.validation.Saml11TicketValidator.retrieveResponseFromServer(Saml11TicketValidator.java:212) org.jasig.cas.client.validation.AbstractUrlBasedTicketValidator.validate(AbstractUrlBasedTicketValidator.java:193) org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:204) org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:97) Root Cause java.io.IOException: Server returned HTTP response code: 500 for URL: http://localhost:9000/samlValidate?TARGET=http%3A%2F%2Flocalhost%3A8080%2F sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894) sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) org.jasig.cas.client.validation.Saml11TicketValidator.retrieveResponseFromServer(Saml11TicketValidator.java:210) org.jasig.cas.client.validation.AbstractUrlBasedTicketValidator.validate(AbstractUrlBasedTicketValidator.java:193) org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:204) org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:97) Note The full stack trace of the root cause is available in the server logs. cas server版本为4.0.7,关于saml协议的配置按照其官网的配置,不明白错误发生在哪里,求解答

向url使用post传参报错

public void Post(String url,JSONObject obj) { try { URL u = new URL(url); HttpURLConnection connection = (HttpURLConnection) u .openConnection(); connection.setDoOutput(true); connection.setDoInput(true); connection.setRequestMethod("POST"); connection.setUseCaches(false); connection.setInstanceFollowRedirects(true); //connection.setRequestProperty("connection", "Keep-Alive"); //connection.setRequestProperty("Content-Type", "text/plain; charset=utf-8"); connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)"); connection.connect(); //POST请求 DataOutputStream out = new DataOutputStream( connection.getOutputStream()); out.writeBytes(obj.toString()); System.out.println(obj.toString()); out.flush(); out.close(); //读取响应 BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream())); String lines; StringBuffer sb = new StringBuffer(""); while ((lines = reader.readLine()) != null) { lines = new String(lines.getBytes(), "utf-8"); sb.append(lines); } System.out.println(sb); reader.close(); connection.disconnect(); } catch (MalformedURLException e) { e.printStackTrace(); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } 报错信息"contentplain":"测试副本","contentrich":"大幅答复","operator":"测试","projectsid":"rx","pubdate":"2011-12-24","scope":"8","tags":"标记","title":" 测试内容以"} java.io.IOException: Server returned HTTP response code: 421 for URL: http://dev.orangeinvest.cn/svc/action/ann.post.do at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1627) at com.app.DateSend.Post(DateSend.java:40) at com.web.postService.realnotice(postService.java:167) at com.web.postService.doPost(postService.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)

用java写了一个获取页面内容的方法,报错返回521码

我的代码如下: strUrl = "http://www.tlnews.cn/dzb/tlrb/html/2016-04/15/node_164.html"; public static String getUrlStr(String strUrl, String charSet){ String urlStr = ""; try { URL url = new URL(strUrl); URLConnection uc = url.openConnection(); uc.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"); uc.setRequestProperty("Connection", "Keep-Alive"); uc.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)"); uc.connect(); InputStream is = uc.getInputStream(); InputStreamReader isr = new InputStreamReader(is, charSet); BufferedReader br = new BufferedReader(isr); StringBuffer strs = new StringBuffer(); String str; while ((str = br.readLine()) != null) { strs.append(str + "\r\n"); } urlStr = strs.toString(); isr.close(); br.close(); } catch (Exception e) { e.printStackTrace(); } return urlStr; } 一开始是报403错误,后来加上了uc.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");这句,开始报521错误。我要抓取的链接是http://www.tlnews.cn/dzb/tlrb/html/2016-04/15/node_164.html 错误信息: java.io.IOException: Server returned HTTP response code: 521 for URL: http://www.tlnews.cn/dzb/tlrb/html/2016-04/14/node_164.html at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at wy.base.test.TestURL.getUrlStr(TestURL.java:32) at wy.base.test.TestURL.main(TestURL.java:14)

java访问WebService返回xml数据生成为本地文件

部分代码如下: public class DomXMLString { private static String SERVICES_HOST = "www.webxml.com.cn"; private static String NETDATA_URL = "http://183.131.242.70:8089/ship/interface/shipInfoService.asmx/getTask"; private static String LOCAL_PC_SAVEFILE_URL = "E:Myeclipse/sy_Task.xml"; private DomXMLString(){} public static void main(String[] args)throws Exception { Document document = getTask(NETDATA_URL); helloOK(document,LOCAL_PC_SAVEFILE_URL); } private static Document getTask(String netXMLDataURL) { Document document = null; DocumentBuilderFactory documentBF = DocumentBuilderFactory.newInstance(); documentBF.setNamespaceAware(true); try{ DocumentBuilder documentB = documentBF.newDocumentBuilder(); InputStream inputStream = getSoapInputStream(netXMLDataURL); document = documentB.parse(inputStream); inputStream.close(); }catch(DOMException e){ e.printStackTrace(); return null; }catch(ParserConfigurationException e){ e.printStackTrace(); return null; }catch(SAXException e){ e.printStackTrace(); return null; }catch(IOException e){ e.printStackTrace(); return null; } return document; } 运行提示错误为: java.io.IOException: Server returned HTTP response code: 500 for URL: http://183.131.242.70:8089/ship/interface/shipInfoService.asmx/getTask at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626) at org.tempuri.DomXMLString.getSoapInputStream(DomXMLString.java:79) at org.tempuri.DomXMLString.getTask(DomXMLString.java:52) at org.tempuri.DomXMLString.main(DomXMLString.java:39) Exception in thread "main" java.lang.IllegalArgumentException: InputStream cannot be null at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:117) at org.tempuri.DomXMLString.getTask(DomXMLString.java:53) at org.tempuri.DomXMLString.main(DomXMLString.java:39) 之前也查了一些资料,想请问是不是我设置的路径有问题以及解决办法,求大神指点迷津

大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了

大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...

在中国程序员是青春饭吗?

今年,我也32了 ,为了不给大家误导,咨询了猎头、圈内好友,以及年过35岁的几位老程序员……舍了老脸去揭人家伤疤……希望能给大家以帮助,记得帮我点赞哦。 目录: 你以为的人生 一次又一次的伤害 猎头界的真相 如何应对互联网行业的「中年危机」 一、你以为的人生 刚入行时,拿着傲人的工资,想着好好干,以为我们的人生是这样的: 等真到了那一天,你会发现,你的人生很可能是这样的: ...

程序员请照顾好自己,周末病魔差点一套带走我。

程序员在一个周末的时间,得了重病,差点当场去世,还好及时挽救回来了。

技术大佬:我去,你写的 switch 语句也太老土了吧

昨天早上通过远程的方式 review 了两名新来同事的代码,大部分代码都写得很漂亮,严谨的同时注释也很到位,这令我非常满意。但当我看到他们当中有一个人写的 switch 语句时,还是忍不住破口大骂:“我擦,小王,你丫写的 switch 语句也太老土了吧!” 来看看小王写的代码吧,看完不要骂我装逼啊。 private static String createPlayer(PlayerTypes p...

上班一个月,后悔当初着急入职的选择了

最近有个老铁,告诉我说,上班一个月,后悔当初着急入职现在公司了。他之前在美图做手机研发,今年美图那边今年也有一波组织优化调整,他是其中一个,在协商离职后,当时捉急找工作上班,因为有房贷供着,不能没有收入来源。所以匆忙选了一家公司,实际上是一个大型外包公司,主要派遣给其他手机厂商做外包项目。**当时承诺待遇还不错,所以就立马入职去上班了。但是后面入职后,发现薪酬待遇这块并不是HR所说那样,那个HR自...

女程序员,为什么比男程序员少???

昨天看到一档综艺节目,讨论了两个话题:(1)中国学生的数学成绩,平均下来看,会比国外好?为什么?(2)男生的数学成绩,平均下来看,会比女生好?为什么?同时,我又联想到了一个技术圈经常讨...

为什么本科以上学历的人只占中国人口的5%,但感觉遍地都是大学生?

中国大学生占总人口不到5% 2017年,中国整体的本科率仅有5.9%;如果算上研究生,这一比例可以进一步上升到6.5% 为什么在国家统计局推出的这份年鉴中,学历的最高一阶就是到研究生,而没有进一步再统计博士生的数量的。 原因其实并不难理解,相比全国和各省整体人口体量,博士生的占比非常之低,属于绝对意义上的小概率样本。 这一点,我们从上表中的各省研究生占比情况也可以看出端倪。除北京、天津、上海三...

副业收入是我做程序媛的3倍,工作外的B面人生是怎样的?

提到“程序员”,多数人脑海里首先想到的大约是:为人木讷、薪水超高、工作枯燥…… 然而,当离开工作岗位,撕去层层标签,脱下“程序员”这身外套,有的人生动又有趣,马上展现出了完全不同的A/B面人生! 不论是简单的爱好,还是正经的副业,他们都干得同样出色。偶尔,还能和程序员的特质结合,产生奇妙的“化学反应”。 @Charlotte:平日素颜示人,周末美妆博主 大家都以为程序媛也个个不修边幅,但我们也许...

MySQL数据库面试题(2020最新版)

文章目录数据库基础知识为什么要使用数据库什么是SQL?什么是MySQL?数据库三大范式是什么mysql有关权限的表都有哪几个MySQL的binlog有有几种录入格式?分别有什么区别?数据类型mysql有哪些数据类型引擎MySQL存储引擎MyISAM与InnoDB区别MyISAM索引与InnoDB索引的区别?InnoDB引擎的4大特性存储引擎选择索引什么是索引?索引有哪些优缺点?索引使用场景(重点)...

如果你是老板,你会不会踢了这样的员工?

有个好朋友ZS,是技术总监,昨天问我:“有一个老下属,跟了我很多年,做事勤勤恳恳,主动性也很好。但随着公司的发展,他的进步速度,跟不上团队的步伐了,有点...

我入职阿里后,才知道原来简历这么写

私下里,有不少读者问我:“二哥,如何才能写出一份专业的技术简历呢?我总感觉自己写的简历太烂了,所以投了无数份,都石沉大海了。”说实话,我自己好多年没有写过简历了,但我认识的一个同行,他在阿里,给我说了一些他当年写简历的方法论,我感觉太牛逼了,实在是忍不住,就分享了出来,希望能够帮助到你。 01、简历的本质 作为简历的撰写者,你必须要搞清楚一点,简历的本质是什么,它就是为了来销售你的价值主张的。往深...

程序员写出这样的代码,能不挨骂吗?

当你换槽填坑时,面对一个新的环境。能够快速熟练,上手实现业务需求是关键。但是,哪些因素会影响你快速上手呢?是原有代码写的不够好?还是注释写的不够好?昨夜...

外包程序员的幸福生活

今天给你们讲述一个外包程序员的幸福生活。男主是Z哥,不是在外包公司上班的那种,是一名自由职业者,接外包项目自己干。接下来讲的都是真人真事。 先给大家介绍一下男主,Z哥,老程序员,是我十多年前的老同事,技术大牛,当过CTO,也创过业。因为我俩都爱好喝酒、踢球,再加上住的距离不算远,所以一直也断断续续的联系着,我对Z哥的状况也有大概了解。 Z哥几年前创业失败,后来他开始干起了外包,利用自己的技术能...

带了6个月的徒弟当了面试官,而身为高级工程师的我天天修Bug......

即将毕业的应届毕业生一枚,现在只拿到了两家offer,但最近听到一些消息,其中一个offer,我这个组据说客户很少,很有可能整组被裁掉。 想问大家: 如果我刚入职这个组就被裁了怎么办呢? 大家都是什么时候知道自己要被裁了的? 面试软技能指导: BQ/Project/Resume 试听内容: 除了刷题,还有哪些技能是拿到offer不可或缺的要素 如何提升面试软实力:简历, 行为面试,沟通能...

优雅的替换if-else语句

场景 日常开发,if-else语句写的不少吧??当逻辑分支非常多的时候,if-else套了一层又一层,虽然业务功能倒是实现了,但是看起来是真的很不优雅,尤其是对于我这种有强迫症的程序"猿",看到这么多if-else,脑袋瓜子就嗡嗡的,总想着解锁新姿势:干掉过多的if-else!!!本文将介绍三板斧手段: 优先判断条件,条件不满足的,逻辑及时中断返回; 采用策略模式+工厂模式; 结合注解,锦...

离职半年了,老东家又发 offer,回不回?

有小伙伴问松哥这个问题,他在上海某公司,在离职了几个月后,前公司的领导联系到他,希望他能够返聘回去,他很纠结要不要回去? 俗话说好马不吃回头草,但是这个小伙伴既然感到纠结了,我觉得至少说明了两个问题:1.曾经的公司还不错;2.现在的日子也不是很如意。否则应该就不会纠结了。 老实说,松哥之前也有过类似的经历,今天就来和小伙伴们聊聊回头草到底吃不吃。 首先一个基本观点,就是离职了也没必要和老东家弄的苦...

记录下入职中软一个月(外包华为)

我在年前从上一家公司离职,没想到过年期间疫情爆发,我也被困在家里,在家呆着的日子让人很焦躁,于是我疯狂的投简历,看面试题,希望可以进大公司去看看。 我也有幸面试了我觉得还挺大的公司的(虽然不是bat之类的大厂,但是作为一名二本计算机专业刚毕业的大学生bat那些大厂我连投简历的勇气都没有),最后选择了中软,我知道这是一家外包公司,待遇各方面甚至不如我的上一家公司,但是对我而言这可是外包华为,能...

为什么程序员做外包会被瞧不起?

二哥,有个事想询问下您的意见,您觉得应届生值得去外包吗?公司虽然挺大的,中xx,但待遇感觉挺低,马上要报到,挺纠结的。

当HR压你价,说你只值7K,你该怎么回答?

当HR压你价,说你只值7K时,你可以流畅地回答,记住,是流畅,不能犹豫。 礼貌地说:“7K是吗?了解了。嗯~其实我对贵司的面试官印象很好。只不过,现在我的手头上已经有一份11K的offer。来面试,主要也是自己对贵司挺有兴趣的,所以过来看看……”(未完) 这段话主要是陪HR互诈的同时,从公司兴趣,公司职员印象上,都给予对方正面的肯定,既能提升HR的好感度,又能让谈判气氛融洽,为后面的发挥留足空间。...

面试:第十六章:Java中级开发

HashMap底层实现原理,红黑树,B+树,B树的结构原理 Spring的AOP和IOC是什么?它们常见的使用场景有哪些?Spring事务,事务的属性,传播行为,数据库隔离级别 Spring和SpringMVC,MyBatis以及SpringBoot的注解分别有哪些?SpringMVC的工作原理,SpringBoot框架的优点,MyBatis框架的优点 SpringCould组件有哪些,他们...

面试阿里p7,被按在地上摩擦,鬼知道我经历了什么?

面试阿里p7被问到的问题(当时我只知道第一个):@Conditional是做什么的?@Conditional多个条件是什么逻辑关系?条件判断在什么时候执...

Python爬虫,高清美图我全都要(彼岸桌面壁纸)

爬取彼岸桌面网站较为简单,用到了requests、lxml、Beautiful Soup4

无代码时代来临,程序员如何保住饭碗?

编程语言层出不穷,从最初的机器语言到如今2500种以上的高级语言,程序员们大呼“学到头秃”。程序员一边面临编程语言不断推陈出新,一边面临由于许多代码已存在,程序员编写新应用程序时存在重复“搬砖”的现象。 无代码/低代码编程应运而生。无代码/低代码是一种创建应用的方法,它可以让开发者使用最少的编码知识来快速开发应用程序。开发者通过图形界面中,可视化建模来组装和配置应用程序。这样一来,开发者直...

面试了一个 31 岁程序员,让我有所触动,30岁以上的程序员该何去何从?

最近面试了一个31岁8年经验的程序猿,让我有点感慨,大龄程序猿该何去何从。

6年开发经验女程序员,面试京东Java岗要求薪资28K

写在开头: 上周面试了一位女程序员,上午10::30来我们部门面试,2B哥接待了她.来看看她的简历: 个人简历 个人技能: ● 熟悉spring mvc 、spring、mybatis 等框架 ● 熟悉 redis 、rocketmq、dubbo、zookeeper、netty 、nginx、tomcat、mysql。 ● 阅读过juc 中的线程池、锁的源...

大三实习生,字节跳动面经分享,已拿Offer

说实话,自己的算法,我一个不会,太难了吧

程序员垃圾简历长什么样?

已经连续五年参加大厂校招、社招的技术面试工作,简历看的不下于万份 这篇文章会用实例告诉你,什么是差的程序员简历! 疫情快要结束了,各个公司也都开始春招了,作为即将红遍大江南北的新晋UP主,那当然要为小伙伴们做点事(手动狗头)。 就在公众号里公开征简历,义务帮大家看,并一一点评。《启舰:春招在即,义务帮大家看看简历吧》 一石激起千层浪,三天收到两百多封简历。 花光了两个星期的所有空闲时...

Java岗开发3年,公司临时抽查算法,离职后这几题我记一辈子

前几天我们公司做了一件蠢事,非常非常愚蠢的事情。我原以为从学校出来之后,除了找工作有测试外,不会有任何与考试有关的事儿。 但是,天有不测风云,公司技术总监、人事总监两位大佬突然降临到我们事业线,叫上我老大,给我们组织了一场别开生面的“考试”。 那是一个风和日丽的下午,我翘着二郎腿,左手端着一杯卡布奇诺,右手抓着我的罗技鼠标,滚动着轮轴,穿梭在头条热点之间。 “淡黄的长裙~蓬松的头发...

大牛都会用的IDEA调试技巧!!!

导读 前天面试了一个985高校的实习生,问了他平时用什么开发工具,他想也没想的说IDEA,于是我抛砖引玉的问了一下IDEA的调试用过吧,你说说怎么设置断点...

都前后端分离了,咱就别做页面跳转了!统统 JSON 交互

文章目录1. 无状态登录1.1 什么是有状态1.2 什么是无状态1.3 如何实现无状态1.4 各自优缺点2. 登录交互2.1 前后端分离的数据交互2.2 登录成功2.3 登录失败3. 未认证处理方案4. 注销登录 这是本系列的第四篇,有小伙伴找不到之前文章,松哥给大家列一个索引出来: 挖一个大坑,Spring Security 开搞! 松哥手把手带你入门 Spring Security,别再问密...

立即提问
相关内容推荐