Split Windows

Description

The Dotty Software Company makes software that is displayed on inexpensive text based terminals. One application for this system has a main window that can be subdivided into further subwindows. Your task is to take a description of the screen layout after a sequence of window splits and draw the minimum sized window grid that is consistent with the description.
In this problem we will concentrate on the boundaries of windows, so all the characters inside of windows will be left blank. Each window that is not further subdivided has a label. Each label is a distinct uppercase letter. For a text terminal the boundaries of windows must be drawn with characters, chosen as follows: A capital letter label is placed in the upper left-hand corner of each undivided window. Asterisks,'*', appear in corners of windows where there is not a label. Dashes, '-', appear on upper and lower boundaries where there are not corners. Vertical bars, '|', appear on side boundaries where there are not corners.

For example, the sequence of splits below would generate Window 1: Initially there could be an application window labeled M, that is split next into left and right subwindows, adding label R, and the left subwindow is split into top and bottom subwindows, adding the label C.

For each pattern of splits there is a binary tree of characters that can describe it. The window splitting and tree structures are described together, building up from the simplest cases.

  1. A window may be an undivided rectangle. Such a window has a capital letter as label. The tree for the window contains just the label.

  2. A window may either be split into left and right subwindows or into top and bottom subwindows, and the corresponding trees have as root the boundary character for the split: a vertical line '|' or a horizontal dash '-' respectively. The root has left and right subtrees corresponding to the top and bottom or left and right subwindows respectively.

Tree 1, above, and Trees 2-4, below, would be consistent with Windows 1-4. Note that Tree 4 contains Trees 2 and 3.

The trees may be more succinctly expressed via a preorder traversal:

  1. The preorder traversal of a tree with just one node (containing a letter) is that letter.

  2. The preorder traversal of a tree with a left and a right subtree is the character from the root of the tree ('-' or '|') followed by the preorder traversal of the left subtree, and then the preorder traversal of the right subtree.

The preorder traversals for Trees 1 through 4 are

|-MCR -|-ABC-D|E-FG -P-|Q|RST |-|-ABC-D|E-FG-P-|Q|RST

Each undivided window must have space for at least one character inside. Hence each tree of splits will be associated with a minimum window size. Windows 1-4 are minimum sized windows for Trees 1-4. Each window illustrates the fact that even in a minimum sized window, not all undivided windows contain only one character.

Consider Tree 4 and Window 4. The main window is split into a left window with Tree 2 and right window with Tree 3. The left window is like Window 2, but the right window is not just like Window 3. The heights of left and right subwindows must match, so the right window must be stretched.

The stretching rule depends on a definition of the size of windows. For dimension calculations it is easiest to imagine that a window contains its interior and a half character wide boundary on all sides, so the total dimensions of a window are one more than the dimensions of the interior. Hence the minimum dimensions of a window are 2 by 2, since a window must contain one character inside, and we add one for the boundary. This definition also means that the sum of the widths of left and right subwindows is the width of their enclosing window. The sum of the heights of top and bottom subwindows is the height of their enclosing window.

The right window in Window 4 must be stretched to match the height 10 of the left window. The right window is split into a top with tree P having minimum height 2 and a bottom with tree -|Q|RST having minimum height 4. The rule for the dimensions in the stretched window is that the heights of the subwindows expand in proportion to their minimum heights, if possible. Some symbols may help here: Let D = 10 be the height of the combined stretched window. We want to determine D1 and D2, the stretched heights of the top and bottom subwindow. Call the corresponding minimum dimensions d = 6, d1 = 2, and d2 = 4. If the window were expanded from a total height d to D in proportion, we would have D1 = d1*(D/d) = 2*(10/6) = 3.333...and D2 = d2*(D/d) = 6.666.... Since the results are not integers we increase D1 to 4 and decrease D2 to 6.

There is a similar calculation for the bottom window with tree -|Q|RST. It is further subdivided into a top with tree |Q|RS and a bottom with tree T, each having minimum height 2 = d1 = d2. The heights need to add up to D = 6, so they are increased proportionally to D1 = D2 = 2*(6/4) = 3 (exact integers).

The final dimensions of an enclosing window are always determined before the final dimensions of its subwindows. In this example only heights needed to be apportioned. If all horizontal and vertical splits were interchanged in this example, producing a tree -|-|ABC|D-E|FG|P|-Q-RST, then widths would be apportioned correspondingly, as shown in the third part of the sample output below. If the proportion calculations do not work out to integers, it is always the top or left subwindow whose dimension is increased to the next integer.
Input

The first line of input contains one integer, which is the total number of preorder traversals describing window structures. This line is followed by one line for each preorder traversal. Each preorder traversal will contain appropriate dividers '|' and '-' and from 1 to 26 uppercase letters.
Output

For each preorder traversal, print the number of the preorder traversal on one line followed by the minimum sized window grid that the traversal could represent.
Sample Input

3
|-MCR
|-|-ABC-D|E-FG-P-|Q|RST
-|-|ABC|D-E|FG|P|-Q-RST
Sample Output

1
M-R-*
| | |
C-* |
| | |
--*
2
A-C-P-----*
| | | |
B-* | |
| | | |
D-*-Q-R-S-*
| | | | |
E-F-* | | |
| | T-*-*-*
| G-* |
| | | |
--*-----*
3
A-B-D-E---*
| | | | |
C-*-* F-G-*
| | | | |
P---Q-*T*-*
| | | |
| R--* |
| | | |
| S--* |
| | | |
-----*--*

2个回答

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
爬虫(python):为什么验证码总是验证失败(内有描述)?
#这是一个试图模拟12306登陆的程序,只到验证码部分 import urllib.request as U import urllib.parse as P import http.cookiejar as C import ssl import chardet as cd ssl._create_default_https_context = ssl._create_unverified_context #无视证书的有效性 opener = U.build_opener(U.HTTPCookieProcessor(C.CookieJar())) U.install_opener(opener) #创建一个访问者(具有cookie功能) req = U.Request("https://kyfw.12306.cn/passport/captcha/captcha-image64?login_site=E&module=login&rand=sjrand&1581337391968&callback=jQuery19109972447551572461_1581326959299&_=1581326959322") req.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" res = opener.open(req) #申请验证码 url = "data:image/jpg;base64," + res.read().decode("utf-8").split('({"image":"')[1].split('","result_message"')[0] #12306分为申请验证码和生成两部分,这是根据两部分的URL规律,生成的验证码图片的URL req = U.Request(url) req.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" res = opener.open(req) code_img = res.read() with open("D:\\py\\测试_练习综合体\\py练习\\imagecode12306.png","wb") as f: f.write(code_img) #获取验证码 pass_code = input("请输入验证码(坐标):") #根据图片获取验证码坐标 data = {"callback":"jQuery19109972447551572461_1581326959299","answer":pass_code,"rand":"sjrand","login_site":"E","_":"1581326959323"} data = P.urlencode(data).encode("utf-8") req = U.Request("https://kyfw.12306.cn/passport/captcha/captcha-check?callback=jQuery19109972447551572461_1581326959299&answer=188%2C49%2C30%2C39&rand=sjrand&login_site=E&_=1581326959323") req.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" res = opener.open(req,data = data) html = res.read().decode("utf-8") #验证码验证 #疑问:为什么验证码验证总是失败了(通过html获得结果)
html2pdf导出,windows下正常,linux下乱吗
本来直接用的jsoup,换了linux后乱码了,最后发现linux下读取个文件都乱码 linux下网页内容字节流保存本地xml文件正常没有乱码,然后读取文件就乱码了, 各位大神这啥原因啊,代码里编码都对应的,windows下都正常的,换linux就乱码了 public String convert2PDF() { PdfContentByte content = null; BaseFont base = null; Rectangle pageRect = null; String pdfPath = context .getRealPath("/pdfIn/" + (new SimpleDateFormat("yyyyMMddHHmmssSSS") .format(new Date()) + ".pdf")); String outPath = context .getRealPath("/pdfOut/" + (new SimpleDateFormat("yyyyMMddHHmmssSSS") .format(new Date()) + ".pdf")); String fontPath = context.getRealPath("/font/msyh.ttf"); String contextPath = context.getContextPath(); // FileOutputStream fos; InputStream is; try { jsp = jsp == null ? "" : jsp; // URL url = new URL(jsp); byte bytes[] = new byte[1024 * 1000]; String tmpXml = context.getRealPath("/tmpXml/" + (new SimpleDateFormat("yyyyMMddHHmmssSSS") .format(new Date()) + ".html")); File xml = new File(tmpXml); if (!xml.getParentFile().exists()) xml.getParentFile().mkdirs(); if (!xml.exists()) xml.createNewFile(); int index = 0; is = url.openStream(); int count = is.read(bytes, index, 1024 * 100); while (count != -1) { index += count; count = is.read(bytes, index, 1); } fos = new FileOutputStream(xml); System.out.println(index); fos.write(bytes, 0, index); // is.close(); fos.close(); FileInputStream fis = new FileInputStream(xml); InputStreamReader isr = new InputStreamReader(fis, "utf-8"); BufferedReader br = new BufferedReader(isr); StringBuffer sb = new StringBuffer(); String line = ""; while ((line = br.readLine()) != null) { sb.append(line); } br.close(); System.err.println(sb.toString()); //TODO 读取本地文件乱码问题 org.jsoup.nodes.Document doc1 = Jsoup.parse(sb.toString()); // org.jsoup.nodes.Document doc2 = Jsoup.parse(xml, "GBK"); System.out.println(doc1.toString()); // System.out.println(doc2.toString()); File tmp = new File(pdfPath); if (!tmp.getParentFile().exists()) tmp.getParentFile().mkdirs(); // System.out.println("-- created -in===" + tmp.getPath()); Document document = new Document(); PdfWriter writer = PdfWriter.getInstance(document, new FileOutputStream(tmp)); document.open(); // Connection conn = Jsoup.connect(jsp); // conn.header( // "User-Agent", // "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"); // org.jsoup.nodes.Document doc = conn.timeout(5000).get(); // doc1.select("div#getpdf").remove(); InputStream in = new ByteArrayInputStream(doc1.toString().getBytes( "utf-8")); // System.out // .println("-- FileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStream"); XMLWorkerHelper.getInstance().parseXHtml(writer, document, in, Charset.forName("utf-8")); // System.out // .println("-- FileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStream"); document.close(); File out = new File(outPath); if (!out.getParentFile().exists()) out.getParentFile().mkdirs(); if (!out.exists()) out.createNewFile(); System.out.println("-- created -out===" + out.getPath()); PdfReader pdfReader = new PdfReader(tmp.getPath()); PdfStamper pdfStamper = new PdfStamper(pdfReader, new FileOutputStream(out)); // PdfGState gs = new PdfGState(); base = BaseFont.createFont("STSong-Light", "UniGB-UCS2-H", BaseFont.NOT_EMBEDDED); // base = BaseFont.createFont(fontPath, BaseFont.IDENTITY_H, // BaseFont.NOT_EMBEDDED); System.out.println("-- -fontPath===" + fontPath); if (base == null || pdfStamper == null) { msg = "文件生成失败!"; ActionContext.getContext().put("msg", msg); path = "error"; } // 设置透明度为0.4 gs.setFillOpacity(0.4f); gs.setStrokeOpacity(0.4f); int toPage = pdfStamper.getReader().getNumberOfPages(); for (int i = 1; i <= toPage; i++) { pageRect = pdfStamper.getReader().getPageSizeWithRotation(i); // 计算水印X,Y坐标 float x = pageRect.getWidth() / 2; float y = pageRect.getHeight() / 2; // 获得PDF最顶层 content = pdfStamper.getOverContent(i); content.saveState(); // set Transparency content.setGState(gs); content.beginText(); content.setColorFill(BaseColor.GRAY); content.setFontAndSize(base, 60); // 水印文字成45度角倾斜 content.showTextAligned(Element.ALIGN_CENTER, "eeeee", x, y, 45); content.endText(); } // pdfStamper.close(); // tmp.delete(); // path = jsp.split(contextPath)[0] + contextPath+"/"+ // out.getPath().replace("\\", // "/").split(contextPath)[1].split("/")[1]+"/"+out.getPath().replace("\\", // "/").split(contextPath)[1].split("/")[2]; path = out.getPath().replace("\\", "/").split("pdfOut")[0] + "pdfOut/$" + out.getPath().replace("\\", "/").split("pdfOut")[1] .split("/")[1]; System.out.println("-- created -pdf path===" + path); } catch (Exception ex) { ex.printStackTrace(); msg = "文件生成异常!"; ActionContext.getContext().put("msg", msg); path = "error"; } finally { content = null; base = null; pageRect = null; } return SUCCESS; }
ictclas4j分词工具搭建在hadoop上出现了很多windows底下没有出现过的错误
本人将在windows上调试好的分词工具包移到bantu底下的eclipse上,运行时出现了Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.ArrayList.elementData(ArrayList.java:418) at java.util.ArrayList.get(ArrayList.java:431) at org.ictclas4j.bean.Dictionary.findInModifyTable(Dictionary.java:464) at org.ictclas4j.bean.Dictionary.getHandle(Dictionary.java:386) at org.ictclas4j.segment.PosTagger.posTag(PosTagger.java:149) at org.ictclas4j.segment.PosTagger.recognition(PosTagger.java:73) at org.ictclas4j.segment.SegTag.split(SegTag.java:90) at test.Main.main(Main.java:23)
windows环境下,使用java解压tar.gz包出现中文乱码
public void unzipOarFile(String outputDirectory) { FileInputStream fis = null; ArchiveInputStream in = null; BufferedInputStream bufferedInputStream = null; try { fis = new FileInputStream(zipfileName); GZIPInputStream is = new GZIPInputStream(new BufferedInputStream(fis)); in = new ArchiveStreamFactory().createArchiveInputStream("tar", is); bufferedInputStream = new BufferedInputStream(in); TarArchiveEntry entry = (TarArchiveEntry) in.getNextEntry(); while (entry != null) { String name = entry.getName(); String[] names = name.split("/"); String fileName = outputDirectory; for (int i = 0; i < names.length; i++) { String str = names[i]; fileName = fileName + File.separator + str; } if (name.endsWith("/")) { mkFolder(fileName); } else { File file = mkFile(fileName); bufferedOutputStream = new BufferedOutputStream(new FileOutputStream(file)); int b; while ((b = bufferedInputStream.read()) != -1) { bufferedOutputStream.write(b); } bufferedOutputStream.flush(); bufferedOutputStream.close(); } entry = (TarArchiveEntry) in.getNextEntry(); } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } catch (ArchiveException e) { e.printStackTrace(); } finally { try { if (bufferedInputStream != null) { bufferedInputStream.close(); } } catch (IOException e) { e.printStackTrace(); } } }
从windows导入mac的android项目,下载到真机时出现空指针错误
![图片说明](https://img-ask.csdn.net/upload/201706/08/1496905838_954210.jpg) 导入的项目错误F:\Android\AndroidStudioProjects\SCar\app\build\intermediates\split-apk\debug\slices\slice_3.apk 这个文件我有: ![图片说明](https://img-ask.csdn.net/upload/201706/08/1496906349_455771.jpg) 但是不知道这个去哪里设置 求师傅帮忙!谢谢了!
采用scrapy框架爬取二手房数据,显示没有爬取到页面和项目,不清楚问题原因
1.item ``` import scrapy class LianjiaItem(scrapy.Item): # define the fields for your item here like: # 房屋名称 name = scrapy.Field() # 房屋户型 type = scrapy.Field() # 建筑面积 area = scrapy.Field() # 房屋朝向 direction = scrapy.Field() # 装修情况 fitment = scrapy.Field() # 有无电梯 elevator = scrapy.Field() # 房屋总价 total_price = scrapy.Field() # 房屋单价 unit_price = scrapy.Field() # 房屋产权 property = scrapy.Field() ``` 2.settings ``` BOT_NAME = 'lianjia' SPIDER_MODULES = ['lianjia.spiders'] NEWSPIDER_MODULE = 'lianjia.spiders' USER_AGENT = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; AcooBrowser; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" ROBOTSTXT_OBEY = False ITEM_PIPELINES = { 'lianjia.pipelines.FilterPipeline': 100, 'lianjia.pipelines.CSVPipeline': 200, } ``` 3.pipelines ``` import re from scrapy.exceptions import DropItem class FilterPipeline(object): def process_item(self,item,spider): item['area'] = re.findall(r"\d+\.?\d*",item["area"])[0] if item["direction"] == '暂无数据': raise DropItem("房屋朝向无数据,抛弃此项目:%s"%item) return item class CSVPipeline(object): index = 0 file = None def open_spider(self,spider): self.file = open("home.csv","a") def process_item(self, item, spider): if self.index == 0: column_name = "name,type,area,direction,fitment,elevator,total_price,unit_price,property\n" self.file.write(column_name) self.index = 1 home_str = item['name']+","+item['type']+","+item['area']+","+item['direction']+","+item['fitment']+","+item['elevator']+","+item['total_price']+","+item['unit_price']+","+item['property']+"\n" self.file.write(home_str) return item def close_spider(self,spider): self.file.close() ``` 4.lianjia_spider ``` import scrapy from scrapy import Request from lianjia.items import LianjiaItem class LianjiaSpiderSpider(scrapy.Spider): name = 'lianjia_spider' # 获取初始请求 def start_requests(self): # 生成请求对象 url = 'https://bj.lianjia.com/ershoufang/' yield Request(url) # 实现主页面解析函数 def parse(self, response): # 使用xpath定位到二手房信息的div元素,保存到列表中 list_selector = response.xpath("//li/div[@class = 'info clear']") # 依次遍历每个选择器,获取二手房的名称,户型,面积,朝向等信息 for one_selector in list_selector: try: name = one_selector.xpath("div[@class = 'title']/a/text()").extract_first() other = one_selector.xpath("div[@class = 'address']/div[@class = 'houseInfo']/text()").extract_first() other_list = other.split("|") type = other_list[0].strip(" ") area = other_list[1].strip(" ") direction = other_list[2].strip(" ") fitment = other_list[3].strip(" ") total_price = one_selector.xpath("//div[@class = 'totalPrice']/span/text()").extract_first() unit_price = one_selector.xpath("//div[@class = 'unitPrice']/@data-price").extract_first() url = one_selector.xpath("div[@class = 'title']/a/@href").extract_first() yield Request(url,meta={"name":name,"type":type,"area":area,"direction":direction,"fitment":fitment,"total_price":total_price,"unit_price":unit_price},callback=self.otherinformation) except: pass current_page = response.xpath("//div[@class = 'page-box house-lst-page-box']/@page-data").extract_first().split(',')[1].split(':')[1] current_page = current_page.replace("}", "") current_page = int(current_page) if current_page < 100: current_page += 1 next_url = "https://bj.lianjia.com/ershoufang/pg%d/" %(current_page) yield Request(next_url,callback=self.otherinformation) def otherinformation(self,response): elevator = response.xpath("//div[@class = 'base']/div[@class = 'content']/ul/li[12]/text()").extract_first() property = response.xpath("//div[@class = 'transaction']/div[@class = 'content']/ul/li[5]/span[2]/text()").extract_first() item = LianjiaItem() item["name"] = response.meta['name'] item["type"] = response.meta['type'] item["area"] = response.meta['area'] item["direction"] = response.meta['direction'] item["fitment"] = response.meta['fitment'] item["total_price"] = response.meta['total_price'] item["unit_price"] = response.meta['unit_price'] item["property"] = property item["elevator"] = elevator yield item ``` 提示错误: ``` de - interpreting them as being unequal if item["direction"] == '鏆傛棤鏁版嵁': 2019-11-25 10:53:35 [scrapy.core.scraper] ERROR: Error processing {'area': u'75.6', 'direction': u'\u897f\u5357', 'elevator': u'\u6709', 'fitment': u'\u7b80\u88c5', 'name': u'\u6b64\u6237\u578b\u517113\u5957 \u89c6\u91ce\u91c7\u5149\u597d \u65e0\u786c\u4f24 \u4e1a\u4e3b\u8bda\u610f\u51fa\u552e', 'property': u'\u6ee1\u4e94\u5e74', 'total_price': None, 'type': u'2\u5ba41\u5385', 'unit_price': None} Traceback (most recent call last): File "f:\python_3.6\venv\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks current.result = callback(current.result, *args, **kw) File "F:\python_3.6\lianjia\lianjia\pipelines.py", line 25, in process_item home_str = item['name']+","+item['type']+","+item['area']+","+item['direction']+","+item['fitment']+","+item['elevator']+","+item['total_price']+","+item['unit_price']+ ","+item['property']+"\n" TypeError: coercing to Unicode: need string or buffer, NoneType found ```
com.jacob包 在Windows10开发环境运行正常,在Windows server 2008 抛出异常
com.jacob.com.ComFailException: Invoke of: AudioOutputStream Source: Description: at com.jacob.com.Dispatch.invokev(Native Method) at com.jacob.com.Dispatch.invokev(Dispatch.java:625) at com.jacob.com.Dispatch.invoke(Dispatch.java:498) at com.jacob.com.Dispatch.putRef(Dispatch.java:819) at net.bjnblh.dc.textToSpeech.service.MSTTSSpeech.saveToWav(MSTTSSpeech.java:274) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl.makeWavAndReturnUrl(NoteReadingServiceImpl.java:70) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl.docToHtml(NoteReadingServiceImpl.java:53) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl$$FastClassBySpringCGLIB$$9c509776.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at com.alibaba.druid.support.spring.stat.DruidStatInterceptor.invoke(DruidStatInterceptor.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl$$EnhancerBySpringCGLIB$$62d6289f.docToHtml(<generated>) at net.bjnblh.dc.textToSpeech.controller.noteReadingController.getHtmlByWord(noteReadingController.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:832) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:743) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:961) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:895) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:869) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at net.bjnblh.dc.core.filter.CrossFilter.doFilter(CrossFilter.java:37) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1533) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1489) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) ``` public void saveToWav(String text, String filePath) { // 创建输出文件流对象 ax=new ActiveXComponent("Sapi.SpFileStream"); spFileStream=ax.getObject(); // 创建音频流格式对象 if(spAudioFormat==null) { ax=new ActiveXComponent("Sapi.SpAudioFormat"); spAudioFormat=ax.getObject(); } // 设置音频流格式类型 Dispatch.put(spAudioFormat,"Type",new Variant(this.formatType)); // 设置文件输出流的格式 Dispatch.putRef(spFileStream,"Format",spAudioFormat); // 调用输出文件流对象的打开方法,创建一个.wav文件 Dispatch.call(spFileStream,"Open",new Variant(filePath),new Variant(3),new Variant(true)); // 设置声音对象的音频输出流为输出文件流对象 Dispatch.putRef(spVoice,"AudioOutputStream",spFileStream); // 调整音量和读的速度 Dispatch.put(spVoice,"Volume",new Variant(this.volume));// 设置音量 Dispatch.put(spVoice,"Rate",new Variant(this.rate));// 设置速率 // 开始朗读 Dispatch.call(spVoice,"Speak",new Variant(text)); /* 分一句话去读 Long initTime =System.currentTimeMillis(); for (Element element:elements) { String text =element.childNode(0).toString(); String[] strings= text.split("\\p{P}"); String newNodeHtml =""; for (int n=0 ; n<strings.length;n++) { String start=String.valueOf(System.currentTimeMillis() - initTime); System.out.println(start); Dispatch.call(spVoice,"Speak",new Variant(strings[n])); newNodeHtml += "<span id=" +start+ ">" + strings[n] +"</span>"; } listValue.add(newNodeHtml); }*/ // 关闭输出文件流对象,释放资源 Dispatch.call(spFileStream,"Close"); Dispatch.putRef(spVoice,"AudioOutputStream",null);//此处异常 } ```
一个百度拇指医生爬虫,想要先实现爬取某个问题的所有链接,但是爬不出来东西。求各位大神帮忙看一下这是为什么?
#写在前面的话 在这个爬虫里我想实现把百度拇指医生里关于“咳嗽”的链接全部爬取下来,下一步要进行的是把爬取到的每个链接里的items里面的内容爬取下来,但是我在第一步就卡住了,求各位大神帮我看一下吧。之前刚刚发了一篇问答,但是不知道怎么回事儿,现在找不到了,(貌似是被删了...?)救救小白吧!感激不尽! 这个是我的爬虫的结构 ![图片说明](https://img-ask.csdn.net/upload/201911/27/1574787999_274479.png) ##ks: ``` # -*- coding: utf-8 -*- import scrapy from kesou.items import KesouItem from scrapy.selector import Selector from scrapy.spiders import Spider from scrapy.http import Request ,FormRequest import pymongo class KsSpider(scrapy.Spider): name = 'ks' allowed_domains = ['kesou,baidu.com'] start_urls = ['https://www.baidu.com/s?wd=%E5%92%B3%E5%97%BD&pn=0&oq=%E5%92%B3%E5%97%BD&ct=2097152&ie=utf-8&si=muzhi.baidu.com&rsv_pq=980e0c55000e2402&rsv_t=ed3f0i5yeefxTMskgzim00cCUyVujMRnw0Vs4o1%2Bo%2Bohf9rFXJvk%2FSYX%2B1M'] def parse(self, response): item = KesouItem() contents = response.xpath('.//h3[@class="t"]') for content in contents: url = content.xpath('.//a/@href').extract()[0] item['url'] = url yield item if self.offset < 760: self.offset += 10 yield scrapy.Request(url = "https://www.baidu.com/s?wd=%E5%92%B3%E5%97%BD&pn=" + str(self.offset) + "&oq=%E5%92%B3%E5%97%BD&ct=2097152&ie=utf-8&si=muzhi.baidu.com&rsv_pq=980e0c55000e2402&rsv_t=ed3f0i5yeefxTMskgzim00cCUyVujMRnw0Vs4o1%2Bo%2Bohf9rFXJvk%2FSYX%2B1M",callback=self.parse,dont_filter=True) ``` ##items: ``` # -*- coding: utf-8 -*- # Define here the models for your scraped items # # See documentation in: # https://docs.scrapy.org/en/latest/topics/items.html import scrapy class KesouItem(scrapy.Item): # 问题ID question_ID = scrapy.Field() # 问题描述 question = scrapy.Field() # 医生回答发表时间 answer_pubtime = scrapy.Field() # 问题详情 description = scrapy.Field() # 医生姓名 doctor_name = scrapy.Field() # 医生职位 doctor_title = scrapy.Field() # 医生所在医院 hospital = scrapy.Field() ``` ##middlewares: ``` # -*- coding: utf-8 -*- # Define here the models for your spider middleware # # See documentation in: # https://docs.scrapy.org/en/latest/topics/spider-middleware.html from scrapy import signals class KesouSpiderMiddleware(object): # Not all methods need to be defined. If a method is not defined, # scrapy acts as if the spider middleware does not modify the # passed objects. @classmethod def from_crawler(cls, crawler): # This method is used by Scrapy to create your spiders. s = cls() crawler.signals.connect(s.spider_opened, signal=signals.spider_opened) return s def process_spider_input(self, response, spider): # Called for each response that goes through the spider # middleware and into the spider. # Should return None or raise an exception. return None def process_spider_output(self, response, result, spider): # Called with the results returned from the Spider, after # it has processed the response. # Must return an iterable of Request, dict or Item objects. for i in result: yield i def process_spider_exception(self, response, exception, spider): # Called when a spider or process_spider_input() method # (from other spider middleware) raises an exception. # Should return either None or an iterable of Request, dict # or Item objects. pass def process_start_requests(self, start_requests, spider): # Called with the start requests of the spider, and works # similarly to the process_spider_output() method, except # that it doesn’t have a response associated. # Must return only requests (not items). for r in start_requests: yield r def spider_opened(self, spider): spider.logger.info('Spider opened: %s' % spider.name) class KesouDownloaderMiddleware(object): # Not all methods need to be defined. If a method is not defined, # scrapy acts as if the downloader middleware does not modify the # passed objects. @classmethod def from_crawler(cls, crawler): # This method is used by Scrapy to create your spiders. s = cls() crawler.signals.connect(s.spider_opened, signal=signals.spider_opened) return s def process_request(self, request, spider): # Called for each request that goes through the downloader # middleware. # Must either: # - return None: continue processing this request # - or return a Response object # - or return a Request object # - or raise IgnoreRequest: process_exception() methods of # installed downloader middleware will be called return None def process_response(self, request, response, spider): # Called with the response returned from the downloader. # Must either; # - return a Response object # - return a Request object # - or raise IgnoreRequest return response def process_exception(self, request, exception, spider): # Called when a download handler or a process_request() # (from other downloader middleware) raises an exception. # Must either: # - return None: continue processing this exception # - return a Response object: stops process_exception() chain # - return a Request object: stops process_exception() chain pass def spider_opened(self, spider): spider.logger.info('Spider opened: %s' % spider.name) ``` ##piplines: ``` # -*- coding: utf-8 -*- # Define your item pipelines here # # Don't forget to add your pipeline to the ITEM_PIPELINES setting # See: https://docs.scrapy.org/en/latest/topics/item-pipeline.html import pymongo from scrapy.utils.project import get_project_settings settings = get_project_settings() class KesouPipeline(object): def __init__(self): host = settings["MONGODB_HOST"] port = settings["MONGODB_PORT"] dbname = settings["MONGODB_DBNAME"] sheetname= settings["MONGODB_SHEETNAME"] # 创建MONGODB数据库链接 client = pymongo.MongoClient(host = host, port = port) # 指定数据库 mydb = client[dbname] # 存放数据的数据库表名 self.sheet = mydb[sheetname] def process_item(self, item, spider): data = dict(item) self.sheet.insert(data) return item ``` ##settings: ``` # -*- coding: utf-8 -*- # Scrapy settings for kesou project # # For simplicity, this file contains only settings considered important or # commonly used. You can find more settings consulting the documentation: # # https://docs.scrapy.org/en/latest/topics/settings.html # https://docs.scrapy.org/en/latest/topics/downloader-middleware.html # https://docs.scrapy.org/en/latest/topics/spider-middleware.html BOT_NAME = 'kesou' SPIDER_MODULES = ['kesou.spiders'] NEWSPIDER_MODULE = 'kesou.spiders' # Crawl responsibly by identifying yourself (and your website) on the user-agent #USER_AGENT = 'kesou (+http://www.yourdomain.com)' # Obey robots.txt rules ROBOTSTXT_OBEY = False # Configure maximum concurrent requests performed by Scrapy (default: 16) #CONCURRENT_REQUESTS = 32 # Configure a delay for requests for the same website (default: 0) # See https://docs.scrapy.org/en/latest/topics/settings.html#download-delay # See also autothrottle settings and docs #DOWNLOAD_DELAY = 3 # The download delay setting will honor only one of: #CONCURRENT_REQUESTS_PER_DOMAIN = 16 #CONCURRENT_REQUESTS_PER_IP = 16 # Disable cookies (enabled by default) COOKIES_ENABLED = False # Disable Telnet Console (enabled by default) #TELNETCONSOLE_ENABLED = False USER_AGENT="Mozilla/5.0 (Windows NT 10.0; WOW64; rv:67.0) Gecko/20100101 Firefox/67.0" # Override the default request headers: #DEFAULT_REQUEST_HEADERS = { # 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', # 'Accept-Language': 'en', #} # Enable or disable spider middlewares # See https://docs.scrapy.org/en/latest/topics/spider-middleware.html #SPIDER_MIDDLEWARES = { # 'kesou.middlewares.KesouSpiderMiddleware': 543, #} # Enable or disable downloader middlewares # See https://docs.scrapy.org/en/latest/topics/downloader-middleware.html #DOWNLOADER_MIDDLEWARES = { # 'kesou.middlewares.KesouDownloaderMiddleware': 543, #} # Enable or disable extensions # See https://docs.scrapy.org/en/latest/topics/extensions.html #EXTENSIONS = { # 'scrapy.extensions.telnet.TelnetConsole': None, #} # Configure item pipelines # See https://docs.scrapy.org/en/latest/topics/item-pipeline.html ITEM_PIPELINES = { 'kesou.pipelines.KesouPipeline': 300, } # MONGODB 主机名 MONGODB_HOST = "127.0.0.1" # MONGODB 端口号 MONGODB_PORT = 27017 # 数据库名称 MONGODB_DBNAME = "ks" # 存放数据的表名称 MONGODB_SHEETNAME = "ks_urls" # Enable and configure the AutoThrottle extension (disabled by default) # See https://docs.scrapy.org/en/latest/topics/autothrottle.html #AUTOTHROTTLE_ENABLED = True # The initial download delay #AUTOTHROTTLE_START_DELAY = 5 # The maximum download delay to be set in case of high latencies #AUTOTHROTTLE_MAX_DELAY = 60 # The average number of requests Scrapy should be sending in parallel to # each remote server #AUTOTHROTTLE_TARGET_CONCURRENCY = 1.0 # Enable showing throttling stats for every response received: #AUTOTHROTTLE_DEBUG = False # Enable and configure HTTP caching (disabled by default) # See https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#httpcache-middleware-settings #HTTPCACHE_ENABLED = True #HTTPCACHE_EXPIRATION_SECS = 0 #HTTPCACHE_DIR = 'httpcache' #HTTPCACHE_IGNORE_HTTP_CODES = [] #HTTPCACHE_STORAGE = 'scrapy.extensions.httpcache.FilesystemCacheStorage' ``` ##run.py: ``` # -*- coding: utf-8 -*- from scrapy import cmdline cmdline.execute("scrapy crawl ks".split()) ``` ##这个是运行出来的结果: ``` PS D:\scrapy_project\kesou> scrapy crawl ks 2019-11-27 00:14:17 [scrapy.utils.log] INFO: Scrapy 1.7.3 started (bot: kesou) 2019-11-27 00:14:17 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.1.0, parsel 1.5.2, w3lib 1.21.0, Twis.7.0, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryphy 2.6.1, Platform Windows-10-10.0.18362-SP0 2019-11-27 00:14:17 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'kesou', 'COOKIES_ENABLED': False, 'NEWSPIDER_MODULE': 'spiders', 'SPIDER_MODULES': ['kesou.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:67.0) Gecko/20100101 Firefox/67 2019-11-27 00:14:17 [scrapy.extensions.telnet] INFO: Telnet Password: 051629c46f34abdf 2019-11-27 00:14:17 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.logstats.LogStats'] 2019-11-27 00:14:19 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2019-11-27 00:14:19 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2019-11-27 00:14:19 [scrapy.middleware] INFO: Enabled item pipelines: ['kesou.pipelines.KesouPipeline'] 2019-11-27 00:14:19 [scrapy.core.engine] INFO: Spider opened 2019-11-27 00:14:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2019-11-27 00:14:19 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2019-11-27 00:14:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.baidu.com/s?wd=%E5%92%B3%E5%97%BD&pn=0&oq=%E5%92%B3%E5&ct=2097152&ie=utf-8&si=muzhi.baidu.com&rsv_pq=980e0c55000e2402&rsv_t=ed3f0i5yeefxTMskgzim00cCUyVujMRnw0Vs4o1%2Bo%2Bohf9rFXJvk%2FSYX% (referer: None) 2019-11-27 00:14:20 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.baidu.com/s?wd=%E5%92%B3%E5%97%BD&pn=0&oq=%B3%E5%97%BD&ct=2097152&ie=utf-8&si=muzhi.baidu.com&rsv_pq=980e0c55000e2402&rsv_t=ed3f0i5yeefxTMskgzim00cCUyVujMRnw0Vs4o1%2Bo%2Bohf9rFFSYX%2B1M> (referer: None) Traceback (most recent call last): File "d:\anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback yield next(it) File "d:\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 84, in evaluate_iterable for r in iterable: File "d:\anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output for x in result: File "d:\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 84, in evaluate_iterable for r in iterable: File "d:\anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr> return (_set_referer(r) for r in result or ()) File "d:\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 84, in evaluate_iterable for r in iterable: File "d:\anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr> return (r for r in result or () if _filter(r)) File "d:\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 84, in evaluate_iterable for r in iterable: File "d:\anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr> return (r for r in result or () if _filter(r)) File "D:\scrapy_project\kesou\kesou\spiders\ks.py", line 19, in parse item['url'] = url File "d:\anaconda3\lib\site-packages\scrapy\item.py", line 73, in __setitem__ (self.__class__.__name__, key)) KeyError: 'KesouItem does not support field: url' 2019-11-27 00:14:20 [scrapy.core.engine] INFO: Closing spider (finished) 2019-11-27 00:14:20 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 438, 'downloader/request_count': 1, 'downloader/request_method_count/GET': 1, 'downloader/response_bytes': 68368, 'downloader/response_count': 1, 'downloader/response_status_count/200': 1, 'elapsed_time_seconds': 0.992207, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2019, 11, 26, 16, 14, 20, 855804), 'log_count/DEBUG': 1, 2019-11-27 00:14:20 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 438, 'downloader/request_count': 1, 'downloader/request_method_count/GET': 1, 'downloader/response_bytes': 68368, 'downloader/response_count': 1, 'downloader/response_status_count/200': 1, 'elapsed_time_seconds': 0.992207, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2019, 11, 26, 16, 14, 20, 855804), 'log_count/DEBUG': 1, 'log_count/ERROR': 1, 'log_count/INFO': 10, 'response_received_count': 1, 'scheduler/dequeued': 1, 'scheduler/dequeued/memory': 1, 'scheduler/enqueued': 1, 'scheduler/enqueued/memory': 1, 'spider_exceptions/KeyError': 1, 'start_time': datetime.datetime(2019, 11, 26, 16, 14, 19, 863597)} 2019-11-27 00:14:21 [scrapy.core.engine] INFO: Spider closed (finished) ```
模型分别在mac和windows服务器上跑,准确率相差60%多!
这是一个用cnn做文本分类的一个模型,我在自己的mac上跑准确率有90%,但是放到windows服务器上准确率竟然只有25%,不知道是什么原因? from __future__ import print_function import numpy as np from keras.utils import np_utils from keras.preprocessing.text import Tokenizer from keras.preprocessing.sequence import pad_sequences import pandas as pd import os from keras import backend as K print('Loading Dict') embeddings_index = {} f = open(os.path.join( 'glove.6B.100d.txt')) for line in f: values = line.split() word = values[0] coefs = np.asarray(values[1:], dtype='float32') embeddings_index[word] = coefs f.close() print('Loading dataset') tmp=pd.read_csv('train.csv') train_X=np.array(tmp.iloc[:,2]).astype('str') train_y=np.array(tmp.iloc[:,0]).astype('int16') train_y_ohe = np_utils.to_categorical(train_y) del tmp tmp=pd.read_csv('test.csv') test_X=np.array(tmp.iloc[:,2]).astype('str') test_y=np.array(tmp.iloc[:,0]).astype('int16') test_y_ohe = np_utils.to_categorical(test_y) del tmp train_y_ohe=train_y_ohe.astype('float32') test_y_ohe=test_y_ohe.astype('float32') X=np.append(train_X,test_X) print('Tokening') t = Tokenizer() t.fit_on_texts(X) vocab_size = len(t.word_index) + 1 # integer encode the documents encoded_X = t.texts_to_sequences(X) # pad documents to a max length of x words max_length = 50 padded_X = pad_sequences(encoded_X, maxlen=max_length, padding='post') embedding_matrix = np.zeros((vocab_size, 100)).astype('float32') for word, i in t.word_index.items(): embedding_vector = embeddings_index.get(word) if embedding_vector is not None: embedding_matrix[i] = embedding_vector padded_X_train=pad_sequences(encoded_X[0:119999],maxlen=max_length, padding='post') padded_X_test=pad_sequences(encoded_X[119999:127598],maxlen=max_length, padding='post') padded_X_test=padded_X_test.astype('float32') padded_X_train=padded_X_train.astype('float32') print('Estabilish model') from keras.models import Model from keras.layers import Dense,Embedding,Convolution1D,concatenate,Flatten,Input,MaxPooling1D,Dropout,Merge from keras.callbacks import TensorBoard K.clear_session() x=Input(shape=(50,),dtype='float32') embed=Embedding(input_dim=vocab_size,output_dim=100,weights=[embedding_matrix],input_length=max_length)(x) cnn1=Convolution1D(128,9,activation='relu',padding='same',strides=1)(embed) cnn1=MaxPooling1D(5)(cnn1) cnn2=Convolution1D(128,6,activation='relu',padding='same',strides=1)(embed) cnn2=MaxPooling1D(5)(cnn2) cnn3=Convolution1D(128,3,activation='relu',padding='same',strides=1)(embed) cnn3=MaxPooling1D(5)(cnn3) cnn=concatenate([cnn1,cnn2,cnn3]) flat=Flatten()(cnn) drop=Dropout(0.1)(flat) y=Dense(5,activation='softmax')(drop) model=Model(inputs=x,outputs=y) model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) tensorboard=TensorBoard(log_dir='./logs',write_graph=True,write_grads=True,histogram_freq=True) model.fit(padded_X_train, train_y_ohe, epochs=5, batch_size=10000, verbose=1,callbacks=[tensorboard],validation_data=[padded_X_test,test_y_ohe]) '''pred0=model.predict_classes(padded_X,verbose=0) acc_train=np.sum(train_y==pred0,axis=0)/train_X.shape[0]'''
yolo3 darknet.py问题
我用darknetAB https://github.com/AlexeyAB/darknet 编译gpu版本后生成darknet.py文件 然后我也编译了yolo_cpp_dll.sln文件 生成dll文件 然后运行darknet.py文件 不显示图片 异常退出 ![图片说明](https://img-ask.csdn.net/upload/201911/02/1572688446_628910.png) 百度了这个问题 有人说要换python3.5版本 我也尝试了 但是也是不行 不会显示图片。请问各位大佬到底怎么解决??急!!!谢谢!!! ``` #!python3 """ Python 3 wrapper for identifying objects in images Requires DLL compilation Both the GPU and no-GPU version should be compiled; the no-GPU version should be renamed "yolo_cpp_dll_nogpu.dll". On a GPU system, you can force CPU evaluation by any of: - Set global variable DARKNET_FORCE_CPU to True - Set environment variable CUDA_VISIBLE_DEVICES to -1 - Set environment variable "FORCE_CPU" to "true" To use, either run performDetect() after import, or modify the end of this file. See the docstring of performDetect() for parameters. Directly viewing or returning bounding-boxed images requires scikit-image to be installed (`pip install scikit-image`) Original *nix 2.7: https://github.com/pjreddie/darknet/blob/0f110834f4e18b30d5f101bf8f1724c34b7b83db/python/darknet.py Windows Python 2.7 version: https://github.com/AlexeyAB/darknet/blob/fc496d52bf22a0bb257300d3c79be9cd80e722cb/build/darknet/x64/darknet.py @author: Philip Kahn @date: 20180503 """ #pylint: disable=R, W0401, W0614, W0703 from ctypes import * import math import random import os def sample(probs): s = sum(probs) probs = [a/s for a in probs] r = random.uniform(0, 1) for i in range(len(probs)): r = r - probs[i] if r <= 0: return i return len(probs)-1 def c_array(ctype, values): arr = (ctype*len(values))() arr[:] = values return arr class BOX(Structure): _fields_ = [("x", c_float), ("y", c_float), ("w", c_float), ("h", c_float)] class DETECTION(Structure): _fields_ = [("bbox", BOX), ("classes", c_int), ("prob", POINTER(c_float)), ("mask", POINTER(c_float)), ("objectness", c_float), ("sort_class", c_int)] class IMAGE(Structure): _fields_ = [("w", c_int), ("h", c_int), ("c", c_int), ("data", POINTER(c_float))] class METADATA(Structure): _fields_ = [("classes", c_int), ("names", POINTER(c_char_p))] #lib = CDLL("/home/pjreddie/documents/darknet/libdarknet.so", RTLD_GLOBAL) #lib = CDLL("libdarknet.so", RTLD_GLOBAL) hasGPU = True if os.name == "nt": cwd = os.path.dirname(__file__) os.environ['PATH'] = cwd + ';' + os.environ['PATH'] winGPUdll = os.path.join(cwd, "yolo_cpp_dll.dll") winNoGPUdll = os.path.join(cwd, "yolo_cpp_dll_nogpu.dll") envKeys = list() for k, v in os.environ.items(): envKeys.append(k) try: try: tmp = os.environ["FORCE_CPU"].lower() if tmp in ["1", "true", "yes", "on"]: raise ValueError("ForceCPU") else: print("Flag value '"+tmp+"' not forcing CPU mode") except KeyError: # We never set the flag if 'CUDA_VISIBLE_DEVICES' in envKeys: if int(os.environ['CUDA_VISIBLE_DEVICES']) < 0: raise ValueError("ForceCPU") try: global DARKNET_FORCE_CPU if DARKNET_FORCE_CPU: raise ValueError("ForceCPU") except NameError: pass # print(os.environ.keys()) # print("FORCE_CPU flag undefined, proceeding with GPU") if not os.path.exists(winGPUdll): raise ValueError("NoDLL") lib = CDLL(winGPUdll, RTLD_GLOBAL) except (KeyError, ValueError): hasGPU = False if os.path.exists(winNoGPUdll): lib = CDLL(winNoGPUdll, RTLD_GLOBAL) print("Notice: CPU-only mode") else: # Try the other way, in case no_gpu was # compile but not renamed lib = CDLL(winGPUdll, RTLD_GLOBAL) print("Environment variables indicated a CPU run, but we didn't find `"+winNoGPUdll+"`. Trying a GPU run anyway.") else: lib = CDLL("./libdarknet.so", RTLD_GLOBAL) lib.network_width.argtypes = [c_void_p] lib.network_width.restype = c_int lib.network_height.argtypes = [c_void_p] lib.network_height.restype = c_int copy_image_from_bytes = lib.copy_image_from_bytes copy_image_from_bytes.argtypes = [IMAGE,c_char_p] def network_width(net): return lib.network_width(net) def network_height(net): return lib.network_height(net) predict = lib.network_predict_ptr predict.argtypes = [c_void_p, POINTER(c_float)] predict.restype = POINTER(c_float) if hasGPU: set_gpu = lib.cuda_set_device set_gpu.argtypes = [c_int] make_image = lib.make_image make_image.argtypes = [c_int, c_int, c_int] make_image.restype = IMAGE get_network_boxes = lib.get_network_boxes get_network_boxes.argtypes = [c_void_p, c_int, c_int, c_float, c_float, POINTER(c_int), c_int, POINTER(c_int), c_int] get_network_boxes.restype = POINTER(DETECTION) make_network_boxes = lib.make_network_boxes make_network_boxes.argtypes = [c_void_p] make_network_boxes.restype = POINTER(DETECTION) free_detections = lib.free_detections free_detections.argtypes = [POINTER(DETECTION), c_int] free_ptrs = lib.free_ptrs free_ptrs.argtypes = [POINTER(c_void_p), c_int] network_predict = lib.network_predict_ptr network_predict.argtypes = [c_void_p, POINTER(c_float)] reset_rnn = lib.reset_rnn reset_rnn.argtypes = [c_void_p] load_net = lib.load_network load_net.argtypes = [c_char_p, c_char_p, c_int] load_net.restype = c_void_p load_net_custom = lib.load_network_custom load_net_custom.argtypes = [c_char_p, c_char_p, c_int, c_int] load_net_custom.restype = c_void_p do_nms_obj = lib.do_nms_obj do_nms_obj.argtypes = [POINTER(DETECTION), c_int, c_int, c_float] do_nms_sort = lib.do_nms_sort do_nms_sort.argtypes = [POINTER(DETECTION), c_int, c_int, c_float] free_image = lib.free_image free_image.argtypes = [IMAGE] letterbox_image = lib.letterbox_image letterbox_image.argtypes = [IMAGE, c_int, c_int] letterbox_image.restype = IMAGE load_meta = lib.get_metadata lib.get_metadata.argtypes = [c_char_p] lib.get_metadata.restype = METADATA load_image = lib.load_image_color load_image.argtypes = [c_char_p, c_int, c_int] load_image.restype = IMAGE rgbgr_image = lib.rgbgr_image rgbgr_image.argtypes = [IMAGE] predict_image = lib.network_predict_image predict_image.argtypes = [c_void_p, IMAGE] predict_image.restype = POINTER(c_float) predict_image_letterbox = lib.network_predict_image_letterbox predict_image_letterbox.argtypes = [c_void_p, IMAGE] predict_image_letterbox.restype = POINTER(c_float) def array_to_image(arr): import numpy as np # need to return old values to avoid python freeing memory arr = arr.transpose(2,0,1) c = arr.shape[0] h = arr.shape[1] w = arr.shape[2] arr = np.ascontiguousarray(arr.flat, dtype=np.float32) / 255.0 data = arr.ctypes.data_as(POINTER(c_float)) im = IMAGE(w,h,c,data) return im, arr def classify(net, meta, im): out = predict_image(net, im) res = [] for i in range(meta.classes): if altNames is None: nameTag = meta.names[i] else: nameTag = altNames[i] res.append((nameTag, out[i])) res = sorted(res, key=lambda x: -x[1]) return res def detect(net, meta, image, thresh=.5, hier_thresh=.5, nms=.45, debug= False): """ Performs the meat of the detection """ #pylint: disable= C0321 im = load_image(image, 0, 0) if debug: print("Loaded image") ret = detect_image(net, meta, im, thresh, hier_thresh, nms, debug) free_image(im) if debug: print("freed image") return ret def detect_image(net, meta, im, thresh=.5, hier_thresh=.5, nms=.45, debug= False): #import cv2 #custom_image_bgr = cv2.imread(image) # use: detect(,,imagePath,) #custom_image = cv2.cvtColor(custom_image_bgr, cv2.COLOR_BGR2RGB) #custom_image = cv2.resize(custom_image,(lib.network_width(net), lib.network_height(net)), interpolation = cv2.INTER_LINEAR) #import scipy.misc #custom_image = scipy.misc.imread(image) #im, arr = array_to_image(custom_image) # you should comment line below: free_image(im) num = c_int(0) if debug: print("Assigned num") pnum = pointer(num) if debug: print("Assigned pnum") predict_image(net, im) letter_box = 0 #predict_image_letterbox(net, im) #letter_box = 1 if debug: print("did prediction") # dets = get_network_boxes(net, custom_image_bgr.shape[1], custom_image_bgr.shape[0], thresh, hier_thresh, None, 0, pnum, letter_box) # OpenCV dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, None, 0, pnum, letter_box) if debug: print("Got dets") num = pnum[0] if debug: print("got zeroth index of pnum") if nms: do_nms_sort(dets, num, meta.classes, nms) if debug: print("did sort") res = [] if debug: print("about to range") for j in range(num): if debug: print("Ranging on "+str(j)+" of "+str(num)) if debug: print("Classes: "+str(meta), meta.classes, meta.names) for i in range(meta.classes): if debug: print("Class-ranging on "+str(i)+" of "+str(meta.classes)+"= "+str(dets[j].prob[i])) if dets[j].prob[i] > 0: b = dets[j].bbox if altNames is None: nameTag = meta.names[i] else: nameTag = altNames[i] if debug: print("Got bbox", b) print(nameTag) print(dets[j].prob[i]) print((b.x, b.y, b.w, b.h)) res.append((nameTag, dets[j].prob[i], (b.x, b.y, b.w, b.h))) if debug: print("did range") res = sorted(res, key=lambda x: -x[1]) if debug: print("did sort") free_detections(dets, num) if debug: print("freed detections") return res netMain = None metaMain = None altNames = None def performDetect(imagePath="data/dog.jpg", thresh= 0.25, configPath = "./cfg/yolov3.cfg", weightPath = "yolov3.weights", metaPath= "./cfg/coco.data", showImage= True, makeImageOnly = False, initOnly= False): """ Convenience function to handle the detection and returns of objects. Displaying bounding boxes requires libraries scikit-image and numpy Parameters ---------------- imagePath: str Path to the image to evaluate. Raises ValueError if not found thresh: float (default= 0.25) The detection threshold configPath: str Path to the configuration file. Raises ValueError if not found weightPath: str Path to the weights file. Raises ValueError if not found metaPath: str Path to the data file. Raises ValueError if not found showImage: bool (default= True) Compute (and show) bounding boxes. Changes return. makeImageOnly: bool (default= False) If showImage is True, this won't actually *show* the image, but will create the array and return it. initOnly: bool (default= False) Only initialize globals. Don't actually run a prediction. Returns ---------------------- When showImage is False, list of tuples like ('obj_label', confidence, (bounding_box_x_px, bounding_box_y_px, bounding_box_width_px, bounding_box_height_px)) The X and Y coordinates are from the center of the bounding box. Subtract half the width or height to get the lower corner. Otherwise, a dict with { "detections": as above "image": a numpy array representing an image, compatible with scikit-image "caption": an image caption } """ # Import the global variables. This lets us instance Darknet once, then just call performDetect() again without instancing again global metaMain, netMain, altNames #pylint: disable=W0603 assert 0 < thresh < 1, "Threshold should be a float between zero and one (non-inclusive)" if not os.path.exists(configPath): raise ValueError("Invalid config path `"+os.path.abspath(configPath)+"`") if not os.path.exists(weightPath): raise ValueError("Invalid weight path `"+os.path.abspath(weightPath)+"`") if not os.path.exists(metaPath): raise ValueError("Invalid data file path `"+os.path.abspath(metaPath)+"`") if netMain is None: netMain = load_net_custom(configPath.encode("ascii"), weightPath.encode("ascii"), 0, 1) # batch size = 1 if metaMain is None: metaMain = load_meta(metaPath.encode("ascii")) if altNames is None: # In Python 3, the metafile default access craps out on Windows (but not Linux) # Read the names file and create a list to feed to detect try: with open(metaPath) as metaFH: metaContents = metaFH.read() import re match = re.search("names *= *(.*)$", metaContents, re.IGNORECASE | re.MULTILINE) if match: result = match.group(1) else: result = None try: if os.path.exists(result): with open(result) as namesFH: namesList = namesFH.read().strip().split("\n") altNames = [x.strip() for x in namesList] except TypeError: pass except Exception: pass if initOnly: print("Initialized detector") return None if not os.path.exists(imagePath): raise ValueError("Invalid image path `"+os.path.abspath(imagePath)+"`") # Do the detection #detections = detect(netMain, metaMain, imagePath, thresh) # if is used cv2.imread(image) detections = detect(netMain, metaMain, imagePath.encode("ascii"), thresh) if showImage: try: from skimage import io, draw import numpy as np image = io.imread(imagePath) print("*** "+str(len(detections))+" Results, color coded by confidence ***") imcaption = [] for detection in detections: label = detection[0] confidence = detection[1] pstring = label+": "+str(np.rint(100 * confidence))+"%" imcaption.append(pstring) print(pstring) bounds = detection[2] shape = image.shape # x = shape[1] # xExtent = int(x * bounds[2] / 100) # y = shape[0] # yExtent = int(y * bounds[3] / 100) yExtent = int(bounds[3]) xEntent = int(bounds[2]) # Coordinates are around the center xCoord = int(bounds[0] - bounds[2]/2) yCoord = int(bounds[1] - bounds[3]/2) boundingBox = [ [xCoord, yCoord], [xCoord, yCoord + yExtent], [xCoord + xEntent, yCoord + yExtent], [xCoord + xEntent, yCoord] ] # Wiggle it around to make a 3px border rr, cc = draw.polygon_perimeter([x[1] for x in boundingBox], [x[0] for x in boundingBox], shape= shape) rr2, cc2 = draw.polygon_perimeter([x[1] + 1 for x in boundingBox], [x[0] for x in boundingBox], shape= shape) rr3, cc3 = draw.polygon_perimeter([x[1] - 1 for x in boundingBox], [x[0] for x in boundingBox], shape= shape) rr4, cc4 = draw.polygon_perimeter([x[1] for x in boundingBox], [x[0] + 1 for x in boundingBox], shape= shape) rr5, cc5 = draw.polygon_perimeter([x[1] for x in boundingBox], [x[0] - 1 for x in boundingBox], shape= shape) boxColor = (int(255 * (1 - (confidence ** 2))), int(255 * (confidence ** 2)), 0) draw.set_color(image, (rr, cc), boxColor, alpha= 0.8) draw.set_color(image, (rr2, cc2), boxColor, alpha= 0.8) draw.set_color(image, (rr3, cc3), boxColor, alpha= 0.8) draw.set_color(image, (rr4, cc4), boxColor, alpha= 0.8) draw.set_color(image, (rr5, cc5), boxColor, alpha= 0.8) if not makeImageOnly: io.imshow(image) io.show() detections = { "detections": detections, "image": image, "caption": "\n<br/>".join(imcaption) } except Exception as e: print("Unable to show image: "+str(e)) return detections if __name__ == "__main__": print(performDetect()) ```
爬取网页只能读取到“请刷新页面或按键盘F5”
以下是源码 ``` import urllib from urllib import request import re import random url = "http://x77558.net/bbs/thread.php?fid=6" user_agent = [ "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11", "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Maxthon 2.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; TencentTraveler 4.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; The World)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X MetaSr 1.0; SE 2.X MetaSr 1.0; .NET CLR 2.0.50727; SE 2.X MetaSr 1.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Avant Browser)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)", "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5", "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5", "Mozilla/5.0 (iPad; U; CPU OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5", "Mozilla/5.0 (Linux; U; Android 2.3.7; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1", "MQQBrowser/26 Mozilla/5.0 (Linux; U; Android 2.3.7; zh-cn; MB200 Build/GRJ22; CyanogenMod-7) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1", "Opera/9.80 (Android 2.3.4; Linux; Opera Mobi/build-1107180945; U; en-GB) Presto/2.8.149 Version/11.10", "Mozilla/5.0 (Linux; U; Android 3.0; en-us; Xoom Build/HRI39) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13", "Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.337 Mobile Safari/534.1+", "Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.70 Safari/534.6 TouchPad/1.0", "Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124", "Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0; HTC; Titan)", "UCWEB7.0.2.37/28/999", "NOKIA5700/ UCWEB7.0.2.37/28/999", "Openwave/ UCWEB7.0.2.37/28/999", "Mozilla/4.0 (compatible; MSIE 6.0; ) Opera/UCWEB7.0.2.37/28/999", # iPhone 6: "Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25", ] # read the url and return a list named page_data def read_url(url,page_data,headers): req = urllib.request.Request(url, headers=headers) for i in range(3): web_data = urllib.request.urlopen(req).read() web_data = web_data.decode("gbk",errors = 'ignore')# the second parament can solver the problem that in # error decode page_data.append(str(web_data)) return page_data # find taget in the page , used re , an return a list def find_tag(tagstr,idx,data,lists): lists.append(re.findall(tagstr,data[idx])) return lists # read the list to download the photo which type is jpg def download_jpg(lists,path): for lis in lists: for l in lis: print(l) name = l.split("/")[-1] print(name) if ".jpg" or ".png" in l: if "js" in l: continue elif "http" in l: # sometimes met a missing name 403 , the solve is in the another file named download.py urllib.request.urlretrieve(l,path+name) else: continue tagstr = '<a title="开放主题" href="(.*?)"' page_data = [] img_url_list = [] url_lsit = [] img_list = [] while len(page_data)==0 or page_data[-1]=="请刷新页面或按键盘F5": headers = {'User-Agent': random.choice(user_agent)} read_url(url,page_data,headers) print(page_data[-1]) ```
图形块的一个表示计算,要求使用C程序
Problem Description Your current task is to make a ground plan for a residential building located in HZXJHS. So you must determine a way to split the floor building with walls to make apartments in the shape of a rectangle. Each built wall must be paralled to the building's sides. The floor is represented in the ground plan as a large rectangle with dimensions n×m, where each apartment is a smaller rectangle with dimensions a×b located inside. For each apartment, its dimensions can be different from each other. The number a and b must be integers. Additionally, the apartments must completely cover the floor without one 1×1 square located on (x,y). The apartments must not intersect, but they can touch. For this example, this is a sample of n=2,m=3,x=2,y=2. To prevent darkness indoors, the apartments must have windows. Therefore, each apartment must share its at least one side with the edge of the rectangle representing the floor so it is possible to place a window. Your boss XXY wants to minimize the maximum areas of all apartments, now it's your turn to tell him the answer. Input There are at most 10000 testcases. For each testcase, only four space-separated integers, n,m,x,y(1≤n,m≤108,n×m>1,1≤x≤n,1≤y≤m). Output For each testcase, print only one interger, representing the answer. Sample Input 2 3 2 2 3 3 1 1 Sample Output 1 2
python爬虫爬取斗图啦上的图片,打开爬取的图片显示图片错误
![图片说明](https://img-ask.csdn.net/upload/201908/03/1564803739_452406.png) ```![图片说明](https://img-ask.csdn.net/upload/201908/03/1564803394_897302.png) import requests,re,os from bs4 import BeautifulSoup def get_url(url): headers={ 'User_Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36', 'Referrer':url } res = requests.get(url,headers=headers) text = res.text soup = BeautifulSoup(text,'lxml') divs = soup.find('div',class_='page-content text-center') a_s = divs.find_all('a',attrs={'class': 'col-xs-6 col-sm-3'}) for a in a_s: #print(a) herf = a['href'] img = a.find('img') print(img) #获取最内层标签方法如下 if a.img['class']==['gif']: pass else: alt = a.img['alt'] alt = re.sub(r'[,@??!!:。]','',alt) #print(alt) data = a.img['data-original'] print(data) datastr = '.'+data.split('.')[-1] filename = alt + datastr #print(filename) #print(os.getcwd()) if os.path.exists(os.getcwd() + "\斗图啦\\"+filename): print('文件已经存在') else: filename = os.getcwd() + "\斗图啦\\"+filename print(filename) with open(filename,'w') as fp: fp.write(data) def main(): if os.path.exists(os.getcwd()+'\斗图啦\\'): print('文件夹已存在') else: os.mkdir(os.getcwd() + "\斗图啦\\") #for x in range(1,101): # url = 'http://www.doutula.com/photo/list/?page=%d' %x # get_url(url) url = 'http://www.doutula.com/photo/list/?page=1' get_url(url) if __name__ == '__main__': main() ``` ```
theano 运行报错 安装了MingGW依然不行
想用theano运行regularization 安装了MingGW c++complier依然报错 ``` from sklearn.datasets import load_boston import theano.tensor as T import numpy as np import matplotlib.pyplot as plt import theano class Layer(object): def __init__(self,inputs,in_size,out_size,activation_function=None): self.W = theano.shared(np.random.normal(0,1,(in_size,out_size))) self.b = theano.shared(np.zeros((out_size,)) + 0.1) self.Wx_plus_b = T.dot(inputs, self.W) + self.b self.activation_function = activation_function if activation_function is None: self.outputs = self.Wx_plus_b else: self.outputs = self.activation_function(self.Wx_plus_b) def minmax_normalization(data): xs_max = np.max(data, axis=0) xs_min = np.min(data, axis=0) xs = (1-0)*(data - xs_min)/(xs_max - xs_min) + 0 return xs np.random.seed(100) x_dataset = load_boston() x_data = x_dataset.data # minmax normalization, rescale the inputs x_data = minmax_normalization(x_data) y_data = x_dataset.target[:,np.newaxis] #cross validation, train test data split x_train, y_train = x_data[:400], y_data[:400] x_test, y_test = x_data[400:], y_data[400:] x = T.dmatrix('x') y = T.dmatrix('y') l1 = Layer(x, 13, 50, T.tanh) l2 = Layer(l1.outputs, 50, 1, None) #compute cost cost = T.mean(T.square(l2.outputs - y)) #cost = T.mean(T.square(l2.outputs - y)) + 0.1*((l1.W**2).sum() + (l2.W**2).sum()) #l2 regulization #cost = T.mean(T.square(l2.outputs - y)) + 0.1*(abs(l1.W).sum() + abs(l2.W).sum()) #l1 regulization gW1, gb1, gW2, gb2 = T.grad(cost, [l1.W,l1.b,l2.W,l2.b]) #gradient descend learning_rate = 0.01 train = theano.function(inputs=[x,y], updates=[(l1.W,l1.W-learning_rate*gW1), (l1.b,l1.b-learning_rate*gb1), (l2.W,l2.W-learning_rate*gW2), (l2.b,l2.b-learning_rate*gb2)]) compute_cost = theano.function(inputs=[x,y], outputs=cost) #record cost train_err_list = [] test_err_list = [] learning_time = [] for i in range(1000): if 1%10 == 0: #record cost train_err_list.append(compute_cost(x_train,y_train)) test_err_list.append(compute_cost(x_test,y_test)) learning_time.append(i) #plot cost history plt.plot(learning_time, train_err_list, 'r-') plt.plot(learning_time, test_err_list,'b--') plt.show() #作者:morvan 莫凡 https://morvanzhou.github.io ``` 报错如下: You can find the C code in this temporary file: C:\Users\Elena\AppData\Local\Temp\theano_compilation_error_cns9ecbh Traceback (most recent call last): File "c:\Users\Elena\PycharmProjects\theano\regularization.py", line 2, in <module> import theano.tensor as T File "C:\Users\Elena\Anaconda3\lib\site-packages\theano\__init__.py", line 110, in <module> from theano.compile import ( File "C:\Users\Elena\Anaconda3\lib\site-packages\theano\compile\__init__.py", line 12, in <module> from theano.compile.mode import * File "C:\Users\Elena\Anaconda3\lib\site-packages\theano\compile\mode.py", line 11, in <module> import theano.gof.vm File "C:\Users\Elena\Anaconda3\lib\site-packages\theano\gof\vm.py", line 674, in <module> from . import lazylinker_c File "C:\Users\Elena\Anaconda3\lib\site-packages\theano\gof\lazylinker_c.py", line 140, in <module> preargs=args) File "C:\Users\Elena\Anaconda3\lib\site-packages\theano\gof\cmodule.py", line 2396, in compile_str (status, compile_stderr.replace('\n', '. '))) Exception: Compilation failed (return status=1): C:\Users\Elena\AppData\Local\Theano\compiledir_Windows-10-10.0.17134-SP0-Intel64_Family_6_Model_142_Stepping_9_GenuineIntel-3.6.5-64\lazylinker_ext\mod.cpp:1:0: sorry, unimplemented: 64-bit mode not compiled in . #include <Python.h> 终端显示gcc有安装 ![图片说明](https://img-ask.csdn.net/upload/201909/29/1569745503_383115.png)
python3.5错误'module' object is not callable
(1) import urllib.request from cons import headers def getUrlList(): req=urllib.request('https://mm.taobao.com/tstar/search/tstar_model.do?_input_charset=utf-8') req.add_header('user-agent',headers()) # print (headers()) html=urllib.urlopen(req, data='q&viewFlag=A&sortType=default&searchStyle=&searchRegion=city%3A&searchFansNum=&currentPage=1&pageSize=100').read() print (html) getUrlList() (2) import random headerstr='''Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36 Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)''' def headers(): header=headerstr.split('\n') length=len(header) return header[random.randint(0,length-1)] 运行(1) 产生错误如下: D:\programmingtools\anaconda\python.exe D:/programmingtools/pycharmpro/files/201711112013/taobeauty.py Traceback (most recent call last): File "D:/programmingtools/pycharmpro/files/201711112013/taobeauty.py", line 13, in <module> getUrlList() File "D:/programmingtools/pycharmpro/files/201711112013/taobeauty.py", line 6, in getUrlList req=urllib.request('https://mm.taobao.com/tstar/search/tstar_model.do?_input_charset=utf-8') TypeError: 'module' object is not callable Process finished with exit code 1
wpf做如何做可以上下拖动Y轴坐标的折线图 用WPFVisifire.Charts做的不能拖动
一下是我写的代码 但是不能上下拖动Y轴坐标,数据使用txt文件来读取 急用 各位大神帮帮忙 谢谢 using System; using System.Collections; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Text.RegularExpressions; using System.Threading.Tasks; using System.Windows; using System.Windows.Controls; using System.Windows.Data; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Imaging; using System.Windows.Navigation; using System.Windows.Shapes; using Visifire.Charts; namespace zhexian { /// <summary> /// MainWindow.xaml 的交互逻辑 /// </summary> public partial class MainWindow : Window { public MainWindow() { InitializeComponent(); } #region 公共属性 Visifire.Charts.Chart chart = new Visifire.Charts.Chart(); DataSeries dataSeries; string[] subLines = { "" }; Visifire.Charts.DataPoint dataPoint; #endregion /// <summary> /// 创建折线图 /// </summary> /// <param name="path"></param> private void zhexian(string path) { using (Stream resourceStream = new FileStream(path, FileMode.Open)) { using (StreamReader reader = new StreamReader(resourceStream, Encoding.GetEncoding("GB2312"))) { chart.Width = 980; chart.Height = 580; chart.Margin = new Thickness(100, 5, 10, 5); ArrayList mydata; dataPoint = new Visifire.Charts.DataPoint(); #region 创建折线图1 //解析所有行数据 var strings = reader.ReadToEnd().Split(new char[] { '\n' }, StringSplitOptions.RemoveEmptyEntries); mydata = new ArrayList(); ArrayList madata2 = new ArrayList(); for (int i = 0; i < strings.Length; i++) { sj j = new sj(); string[] stringArr = Regex.Split(strings[i], " "); j.number = new double[stringArr.Length]; for (int h = 0; h < stringArr.Length; h++) { j.number[h] = Double.Parse(stringArr[h]); } madata2.Add(j); } //整理后的列数据 string[] strArr = Regex.Split(strings[0], " "); for (int i = 0; i < strArr.Length; i++) { sj sj = new sj(); sj.number = new double[madata2.Count]; int h = 0; foreach (sj item in madata2) { if (i < item.number.Length) { sj.number[h] = item.number[i]; h++; } } mydata.Add(sj); } //开始划线 foreach (sj item in mydata) { double num1 = 1; // 创建一个新的数据线。 dataSeries = new DataSeries(); // 设置数据线的格式。 dataSeries.LegendText = "樱桃"; dataSeries.RenderAs = RenderAs.Line;//折线图 for (int i = 0; i < item.number.Length; i++) { // 创建一个数据点的实例。 dataPoint = new Visifire.Charts.DataPoint(); // 设置X轴点 dataPoint.XValue = num1; //设置Y轴点 string num = item.number[i].ToString(); dataPoint.YValue = Double.Parse(num); dataPoint.MarkerSize = 8; num1++; //添加数据点 dataSeries.DataPoints.Add(dataPoint); } // 添加数据线到数据序列。 chart.Series.Add(dataSeries); } #endregion } } //将数据绑定到Grid面板上 System.Windows.Controls.Grid gr = new System.Windows.Controls.Grid(); gr.Children.Add(chart); Simon.Children.Add(gr); } /// <summary> /// 窗体加载 /// </summary> /// <param name="sender"></param> /// <param name="e"></param> private void Window_Loaded(object sender, RoutedEventArgs e) { string path = @"C:\Users\AllDream\Desktop\新建文件夹 (2)\ZheXian\zhexian\shuju.txt"; zhexian(path); huoqu(); } //创建一个double类型的数组类用于存储重新组合之后的数据 public class sj { public double[] number = null; } public void huoqu() { Point zuobiao = new Point(); object xzou = dataPoint.XValue; double oo = (double)xzou; zuobiao.X = oo; zuobiao.Y = dataPoint.YValue; } } }
C#:未将对象引用设置到对象的实例 (System.NullReferenceException)
![图片说明](https://img-ask.csdn.net/upload/201501/27/1422348462_454037.png) 代码如下: using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; using System.Windows.Forms; namespace ConsoleExamples { static class Program { /// <summary> /// 应用程序的主入口点。 /// </summary> [STAThread] static void Main(string[] args) { Console.Write("请输入x和y(例如12,15),然后按回车键:"); string s = Console.ReadLine(); string[] a = s.Split(','); int x = int.Parse(a[0]); int y = int.Parse(a[1]); int z = x * y; Console.WriteLine("x*y={0}", z); Console.WriteLine("请按任意键结束程序。"); Console.ReadKey(); } } }
C# 为什么ReportViewer做的报表个别电脑出现不显示的情况?
在阜阳现场调试的时候,有一个电脑出现的打开报表时显示找不到方法“Void Microsoft.Reporting.PreviewItemContext.setbpath(System string,System string,Microsoft.Reporting.DefinitionSoure)”。 同一个安装包,只有一个电脑出现这个问题,怀疑是那台电脑缺少什么动态链接库,尴尬的是我不知道缺的什么。 对方电脑系统是Win7 代码编译器VS2005, 代码不是我敲的,前前前辈写的,我维护一下。 ``` using System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using System.Drawing; using System.Text; using System.Windows.Forms; using System.IO; using System.Data.SqlClient; namespace ConcreteApp.report { public partial class MRSGMXBB : Form { private System.Data.SqlClient.SqlConnection con = new System.Data.SqlClient.SqlConnection(); //-----配置数据库专用变量 private string ServerName = ""; private string DBName = ""; private string SqlName = ""; private string SqlPWD = ""; private string UID1; private string sqlstr = "select QDDH 强度代号 from t_QDDH "; public MRSGMXBB() { InitializeComponent(); } private void MRSGMXBB_Load(object sender, EventArgs e) { // TODO: 这行代码将数据加载到表“cDB_TEMPDataSet.t_shengchanrenwu”中。您可以根据需要移动或移除它。 this.t_shengchanrenwuTableAdapter1.Fill(this.cDB_TEMPDataSet.t_shengchanrenwu); // TODO: 这行代码将数据加载到表“cDBDataSetIP.t_shengchanrenwu”中。您可以根据需要移动或移除它。 this.t_shengchanrenwuTableAdapter.Fill(this.cDBDataSetIP.t_shengchanrenwu); this.reportViewer1.RefreshReport(); this.date1ToolStripTextBox.Text = DateTime.Now.ToShortDateString(); this.date2ToolStripTextBox.Text = DateTime.Now.AddDays(1).ToShortDateString(); string[] tmpcodeA; UID1 = ""; try { //读取配置文件 using (StreamReader sr = new StreamReader(Application.StartupPath + "/config.ini")) { try { tmpcodeA = sr.ReadLine().Split('|'); this.ServerName = tmpcodeA[0].ToString(); this.DBName = tmpcodeA[1].ToString(); this.SqlName = tmpcodeA[2].ToString(); this.SqlPWD = tmpcodeA[3].ToString(); } catch (Exception ex) { MessageBox.Show("读取配置文件失败!请检查程序目录下的config.ini文件。" + ex.ToString(), "提示"); } } //验证登陆 con.ConnectionString = ("SERVER=" + ServerName + ";UID=" + SqlName + ";PWD=" + SqlPWD + ";DATABASE=" + DBName + ""); con.Open(); SqlDataAdapter da = new SqlDataAdapter(sqlstr, con); DataSet ds = new DataSet(); da.Fill(ds); con.Close(); for (int i = 0; i < ds.Tables[0].Rows.Count; i++) CBpz.Items.Add(ds.Tables[0].Rows[i][0].ToString()); } catch (Exception ex) { MessageBox.Show("数据库连接失败!请重新配置" + ex.ToString(), "提示", MessageBoxButtons.OK, MessageBoxIcon.Asterisk); } } private void fillBy1ToolStripButton_Click(object sender, EventArgs e) { if (CBpz.Text == "") { try { this.t_shengchanrenwuTableAdapter.FillBy(this.cDBDataSetIP.t_shengchanrenwu, date1ToolStripTextBox.Text, date2ToolStripTextBox.Text); this.reportViewer1.RefreshReport(); } catch (System.Exception ex) { System.Windows.Forms.MessageBox.Show(ex.Message); } } else { try { this.t_shengchanrenwuTableAdapter.FillBy1(this.cDBDataSetIP.t_shengchanrenwu, date1ToolStripTextBox.Text, date2ToolStripTextBox.Text,CBpz.Text); this.reportViewer1.RefreshReport(); } catch (System.Exception ex) { System.Windows.Forms.MessageBox.Show(ex.Message); } } } } } ```
为什么我一用代理就会出现'list' object is not callable错误
import urllib.request import random import re def url_open(url): headers = {'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Referer':'http://www.gamersky.com/ent/201711/975615_2.shtml', 'Upgrade-Insecure-Requests':'1'} data = None req = urllib.request.Request(url, data, headers) req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.221 Safari/537.36 SE 2.X MetaSr 1.0') iplist = ['182.16.175.2:53281','201.144.14.229:53281',' 87.228.29.154:53281'] proxy_support = urllib.request.ProxyHandler({'http':random.choice(iplist)}) opener = urllib.request.build_opener(proxy_support) opener.addheaders('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.221 Safari/537.36 SE 2.X MetaSr 1.0') urllib.request.install_opener(opener) response = urllib.request.urlopen(url) html = response.read() return html def find_imgs(url): html = url_open(url).decode('utf-8') pa = re.compile(r'http.+?/.jpg') imglists = pa.findall(html) return imglists def save_imgs(img_addrs): for each in img_addrs: filename = each.split('/')[-1] with open(filename, 'wb') as f: img = url_open(each) f.write(img) def download_mm(pages=10): url = 'http://www.gamersky.com/' for i in range(pages): if i==1: page_url = url + 'ent/201711/975615' + 'shtml' else: page_url = url + 'ent/201711/975615_' + str(i) + 'shtml' img_addrs = find_imgs(page_url) save_imgs(img_addrs) if __name__ == '__main__': download_mm()
终于明白阿里百度这样的大公司,为什么面试经常拿ThreadLocal考验求职者了
点击上面↑「爱开发」关注我们每晚10点,捕获技术思考和创业资源洞察什么是ThreadLocalThreadLocal是一个本地线程副本变量工具类,各个线程都拥有一份线程私有的数
程序员必须掌握的核心算法有哪些?
由于我之前一直强调数据结构以及算法学习的重要性,所以就有一些读者经常问我,数据结构与算法应该要学习到哪个程度呢?,说实话,这个问题我不知道要怎么回答你,主要取决于你想学习到哪些程度,不过针对这个问题,我稍微总结一下我学过的算法知识点,以及我觉得值得学习的算法。这些算法与数据结构的学习大多数是零散的,并没有一本把他们全部覆盖的书籍。下面是我觉得值得学习的一些算法以及数据结构,当然,我也会整理一些看过
Linux(服务器编程):15---两种高效的事件处理模式(reactor模式、proactor模式)
前言 同步I/O模型通常用于实现Reactor模式 异步I/O模型则用于实现Proactor模式 最后我们会使用同步I/O方式模拟出Proactor模式 一、Reactor模式 Reactor模式特点 它要求主线程(I/O处理单元)只负责监听文件描述符上是否有事件发生,有的话就立即将时间通知工作线程(逻辑单元)。除此之外,主线程不做任何其他实质性的工作 读写数据,接受新的连接,以及处...
阿里面试官问我:如何设计秒杀系统?我的回答让他比起大拇指
你知道的越多,你不知道的越多 点赞再看,养成习惯 GitHub上已经开源 https://github.com/JavaFamily 有一线大厂面试点脑图和个人联系方式,欢迎Star和指教 前言 Redis在互联网技术存储方面使用如此广泛,几乎所有的后端技术面试官都要在Redis的使用和原理方面对小伙伴们进行360°的刁难。 作为一个在互联网公司面一次拿一次Offer的面霸,打败了...
五年程序员记流水账式的自白。
不知觉已中码龄已突破五年,一路走来从起初铁憨憨到现在的十九线程序员,一路成长,虽然不能成为高工,但是也能挡下一面,从15年很火的android开始入坑,走过java、.Net、QT,目前仍处于android和.net交替开发中。 毕业到现在一共就职过两家公司,目前是第二家,公司算是半个创业公司,所以基本上都会身兼多职。比如不光要写代码,还要写软著、软著评测、线上线下客户对接需求收集...
C语言魔塔游戏
很早就很想写这个,今天终于写完了。 游戏截图: 编译环境: VS2017 游戏需要一些图片,如果有想要的或者对游戏有什么看法的可以加我的QQ 2985486630 讨论,如果暂时没有回应,可以在博客下方留言,到时候我会看到。 下面我来介绍一下游戏的主要功能和实现方式 首先是玩家的定义,使用结构体,这个名字是可以自己改变的 struct gamerole { char n
一文详尽系列之模型评估指标
点击上方“Datawhale”,选择“星标”公众号第一时间获取价值内容在机器学习领域通常会根据实际的业务场景拟定相应的不同的业务指标,针对不同机器学习问题如回归、分类、排...
究竟你适不适合买Mac?
我清晰的记得,刚买的macbook pro回到家,开机后第一件事情,就是上了淘宝网,花了500元钱,找了一个上门维修电脑的师傅,上门给我装了一个windows系统。。。。。。 表砍我。。。 当时买mac的初衷,只是想要个固态硬盘的笔记本,用来运行一些复杂的扑克软件。而看了当时所有的SSD笔记本后,最终决定,还是买个好(xiong)看(da)的。 已经有好几个朋友问我mba怎么样了,所以今天尽量客观
程序员一般通过什么途径接私活?
二哥,你好,我想知道一般程序猿都如何接私活,我也想接,能告诉我一些方法吗? 上面是一个读者“烦不烦”问我的一个问题。其实不止是“烦不烦”,还有很多读者问过我类似这样的问题。 我接的私活不算多,挣到的钱也没有多少,加起来不到 20W。说实话,这个数目说出来我是有点心虚的,毕竟太少了,大家轻喷。但我想,恰好配得上“一般程序员”这个称号啊。毕竟苍蝇再小也是肉,我也算是有经验的人了。 唾弃接私活、做外...
压测学习总结(1)——高并发性能指标:QPS、TPS、RT、吞吐量详解
一、QPS,每秒查询 QPS:Queries Per Second意思是“每秒查询率”,是一台服务器每秒能够相应的查询次数,是对一个特定的查询服务器在规定时间内所处理流量多少的衡量标准。互联网中,作为域名系统服务器的机器的性能经常用每秒查询率来衡量。 二、TPS,每秒事务 TPS:是TransactionsPerSecond的缩写,也就是事务数/秒。它是软件测试结果的测量单位。一个事务是指一...
Python爬虫爬取淘宝,京东商品信息
小编是一个理科生,不善长说一些废话。简单介绍下原理然后直接上代码。 使用的工具(Python+pycharm2019.3+selenium+xpath+chromedriver)其中要使用pycharm也可以私聊我selenium是一个框架可以通过pip下载 pip install selenium -i https://pypi.tuna.tsinghua.edu.cn/simple/ 
阿里程序员写了一个新手都写不出的低级bug,被骂惨了。
这种新手都不会范的错,居然被一个工作好几年的小伙子写出来,差点被当场开除了。
Java工作4年来应聘要16K最后没要,细节如下。。。
前奏: 今天2B哥和大家分享一位前几天面试的一位应聘者,工作4年26岁,统招本科。 以下就是他的简历和面试情况。 基本情况: 专业技能: 1、&nbsp;熟悉Sping了解SpringMVC、SpringBoot、Mybatis等框架、了解SpringCloud微服务 2、&nbsp;熟悉常用项目管理工具:SVN、GIT、MAVEN、Jenkins 3、&nbsp;熟悉Nginx、tomca
2020年,冯唐49岁:我给20、30岁IT职场年轻人的建议
点击“技术领导力”关注∆  每天早上8:30推送 作者| Mr.K   编辑| Emma 来源| 技术领导力(ID:jishulingdaoli) 前天的推文《冯唐:职场人35岁以后,方法论比经验重要》,收到了不少读者的反馈,觉得挺受启发。其实,冯唐写了不少关于职场方面的文章,都挺不错的。可惜大家只记住了“春风十里不如你”、“如何避免成为油腻腻的中年人”等不那么正经的文章。 本文整理了冯
程序员该看的几部电影
##1、骇客帝国(1999) 概念:在线/离线,递归,循环,矩阵等 剧情简介: 不久的将来,网络黑客尼奥对这个看似正常的现实世界产生了怀疑。 他结识了黑客崔妮蒂,并见到了黑客组织的首领墨菲斯。 墨菲斯告诉他,现实世界其实是由一个名叫“母体”的计算机人工智能系统控制,人们就像他们饲养的动物,没有自由和思想,而尼奥就是能够拯救人类的救世主。 可是,救赎之路从来都不会一帆风顺,到底哪里才是真实的世界?
Python绘图,圣诞树,花,爱心 | Turtle篇
每周每日,分享Python实战代码,入门资料,进阶资料,基础语法,爬虫,数据分析,web网站,机器学习,深度学习等等。 公众号回复【进群】沟通交流吧,QQ扫码进群学习吧 微信群 QQ群 1.画圣诞树 import turtle screen = turtle.Screen() screen.setup(800,600) circle = turtle.Turtle()...
作为一个程序员,CPU的这些硬核知识你必须会!
CPU对每个程序员来说,是个既熟悉又陌生的东西? 如果你只知道CPU是中央处理器的话,那可能对你并没有什么用,那么作为程序员的我们,必须要搞懂的就是CPU这家伙是如何运行的,尤其要搞懂它里面的寄存器是怎么一回事,因为这将让你从底层明白程序的运行机制。 随我一起,来好好认识下CPU这货吧 把CPU掰开来看 对于CPU来说,我们首先就要搞明白它是怎么回事,也就是它的内部构造,当然,CPU那么牛的一个东
还记得那个提速8倍的IDEA插件吗?VS Code版本也发布啦!!
去年,阿里云发布了本地 IDE 插件 Cloud Toolkit,仅 IntelliJ IDEA 一个平台,就有 15 万以上的开发者进行了下载,体验了一键部署带来的开发便利。时隔一年的今天,阿里云正式发布了 Visual Studio Code 版本,全面覆盖前端开发者,帮助前端实现一键打包部署,让开发提速 8 倍。 VSCode 版本的插件,目前能做到什么? 安装插件之后,开发者可以立即体验...
破14亿,Python分析我国存在哪些人口危机!
2020年1月17日,国家统计局发布了2019年国民经济报告,报告中指出我国人口突破14亿。 猪哥的朋友圈被14亿人口刷屏,但是很多人并没有看到我国复杂的人口问题:老龄化、男女比例失衡、生育率下降、人口红利下降等。 今天我们就来分析一下我们国家的人口数据吧! 一、背景 1.人口突破14亿 2020年1月17日,国家统计局发布了 2019年国民经济报告 ,报告中指出:年末中国大陆总人口(包括31个
2019年除夕夜的有感而发
天气:小雨(加小雪) 温度:3摄氏度 空气:严重污染(399) 风向:北风 风力:微风 现在是除夕夜晚上十点钟,再有两个小时就要新的一年了; 首先要说的是我没患病,至少现在是没有患病;但是心情确像患了病一样沉重; 现在这个时刻应该大部分家庭都在看春晚吧,或许一家人团团圆圆的坐在一起,或许因为某些特殊原因而不能团圆;但不管是身在何处,身处什么境地,我都想对每一个人说一句:新年快乐! 不知道csdn这...
听说想当黑客的都玩过这个Monyer游戏(1~14攻略)
第零关 进入传送门开始第0关(游戏链接) 请点击链接进入第1关: 连接在左边→ ←连接在右边 看不到啊。。。。(只能看到一堆大佬做完的留名,也能看到菜鸡的我,在后面~~) 直接fn+f12吧 &lt;span&gt;连接在左边→&lt;/span&gt; &lt;a href="first.php"&gt;&lt;/a&gt; &lt;span&gt;←连接在右边&lt;/span&gt; o...
在家远程办公效率低?那你一定要收好这个「在家办公」神器!
相信大家都已经收到国务院延长春节假期的消息,接下来,在家远程办公可能将会持续一段时间。 但是问题来了。远程办公不是人在电脑前就当坐班了,相反,对于沟通效率,文件协作,以及信息安全都有着极高的要求。有着非常多的挑战,比如: 1在异地互相不见面的会议上,如何提高沟通效率? 2文件之间的来往反馈如何做到及时性?如何保证信息安全? 3如何规划安排每天工作,以及如何进行成果验收? ......
作为一个程序员,内存和磁盘的这些事情,你不得不知道啊!!!
截止目前,我已经分享了如下几篇文章: 一个程序在计算机中是如何运行的?超级干货!!! 作为一个程序员,CPU的这些硬核知识你必须会! 作为一个程序员,内存的这些硬核知识你必须懂! 这些知识可以说是我们之前都不太重视的基础知识,可能大家在上大学的时候都学习过了,但是嘞,当时由于老师讲解的没那么有趣,又加上这些知识本身就比较枯燥,所以嘞,大家当初几乎等于没学。 再说啦,学习这些,也看不出来有什么用啊!
2020年的1月,我辞掉了我的第一份工作
其实,这篇文章,我应该早点写的,毕竟现在已经2月份了。不过一些其它原因,或者是我的惰性、还有一些迷茫的念头,让自己迟迟没有试着写一点东西,记录下,或者说是总结下自己前3年的工作上的经历、学习的过程。 我自己知道的,在写自己的博客方面,我的文笔很一般,非技术类的文章不想去写;另外我又是一个还比较热衷于技术的人,而平常复杂一点的东西,如果想写文章写的清楚点,是需要足够...
别低估自己的直觉,也别高估自己的智商
所有群全部吵翻天,朋友圈全部沦陷,公众号疯狂转发。这两周没怎么发原创,只发新闻,可能有人注意到了。我不是懒,是文章写了却没发,因为大家的关注力始终在这次的疫情上面,发了也没人看。当然,我...
这个世界上人真的分三六九等,你信吗?
偶然间,在知乎上看到一个问题 一时间,勾起了我深深的回忆。 以前在厂里打过两次工,做过家教,干过辅导班,做过中介。零下几度的晚上,贴过广告,满脸、满手地长冻疮。 再回首那段岁月,虽然苦,但让我学会了坚持和忍耐。让我明白了,在这个世界上,无论环境多么的恶劣,只要心存希望,星星之火,亦可燎原。 下文是原回答,希望能对你能有所启发。 如果我说,这个世界上人真的分三六九等,...
节后首个工作日,企业们集体开晨会让钉钉挂了
By 超神经场景描述:昨天 2 月 3 日,是大部分城市号召远程工作的第一天,全国有接近 2 亿人在家开始远程办公,钉钉上也有超过 1000 万家企业活跃起来。关键词:十一出行 人脸...
Java基础知识点梳理
Java基础知识点梳理 摘要: 虽然已经在实际工作中经常与java打交道,但是一直没系统地对java这门语言进行梳理和总结,掌握的知识也比较零散。恰好利用这段时间重新认识下java,并对一些常见的语法和知识点做个总结与回顾,一方面为了加深印象,方便后面查阅,一方面为了学好java打下基础。 Java简介 java语言于1995年正式推出,最开始被命名为Oak语言,由James Gosling(詹姆
2020年全新Java学习路线图,含配套视频,学完即为中级Java程序员!!
新的一年来临,突如其来的疫情打破了平静的生活! 在家的你是否很无聊,如果无聊就来学习吧! 世上只有一种投资只赚不赔,那就是学习!!! 传智播客于2020年升级了Java学习线路图,硬核升级,免费放送! 学完你就是中级程序员,能更快一步找到工作! 一、Java基础 JavaSE基础是Java中级程序员的起点,是帮助你从小白到懂得编程的必经之路。 在Java基础板块中有6个子模块的学
B 站上有哪些很好的学习资源?
哇说起B站,在小九眼里就是宝藏般的存在,放年假宅在家时一天刷6、7个小时不在话下,更别提今年的跨年晚会,我简直是跪着看完的!! 最早大家聚在在B站是为了追番,再后来我在上面刷欧美新歌和漂亮小姐姐的舞蹈视频,最近两年我和周围的朋友们已经把B站当作学习教室了,而且学习成本还免费,真是个励志的好平台ヽ(.◕ฺˇд ˇ◕ฺ;)ノ 下面我们就来盘点一下B站上优质的学习资源: 综合类 Oeasy: 综合
相关热词 c# 为空 判断 委托 c#记事本颜色 c# 系统默认声音 js中调用c#方法参数 c#引入dll文件报错 c#根据名称实例化 c#从邮件服务器获取邮件 c# 保存文件夹 c#代码打包引用 c# 压缩效率
立即提问