Split Windows

Description

The Dotty Software Company makes software that is displayed on inexpensive text based terminals. One application for this system has a main window that can be subdivided into further subwindows. Your task is to take a description of the screen layout after a sequence of window splits and draw the minimum sized window grid that is consistent with the description.
In this problem we will concentrate on the boundaries of windows, so all the characters inside of windows will be left blank. Each window that is not further subdivided has a label. Each label is a distinct uppercase letter. For a text terminal the boundaries of windows must be drawn with characters, chosen as follows: A capital letter label is placed in the upper left-hand corner of each undivided window. Asterisks,'*', appear in corners of windows where there is not a label. Dashes, '-', appear on upper and lower boundaries where there are not corners. Vertical bars, '|', appear on side boundaries where there are not corners.

For example, the sequence of splits below would generate Window 1: Initially there could be an application window labeled M, that is split next into left and right subwindows, adding label R, and the left subwindow is split into top and bottom subwindows, adding the label C.

For each pattern of splits there is a binary tree of characters that can describe it. The window splitting and tree structures are described together, building up from the simplest cases.

  1. A window may be an undivided rectangle. Such a window has a capital letter as label. The tree for the window contains just the label.

  2. A window may either be split into left and right subwindows or into top and bottom subwindows, and the corresponding trees have as root the boundary character for the split: a vertical line '|' or a horizontal dash '-' respectively. The root has left and right subtrees corresponding to the top and bottom or left and right subwindows respectively.

Tree 1, above, and Trees 2-4, below, would be consistent with Windows 1-4. Note that Tree 4 contains Trees 2 and 3.

The trees may be more succinctly expressed via a preorder traversal:

  1. The preorder traversal of a tree with just one node (containing a letter) is that letter.

  2. The preorder traversal of a tree with a left and a right subtree is the character from the root of the tree ('-' or '|') followed by the preorder traversal of the left subtree, and then the preorder traversal of the right subtree.

The preorder traversals for Trees 1 through 4 are

|-MCR -|-ABC-D|E-FG -P-|Q|RST |-|-ABC-D|E-FG-P-|Q|RST

Each undivided window must have space for at least one character inside. Hence each tree of splits will be associated with a minimum window size. Windows 1-4 are minimum sized windows for Trees 1-4. Each window illustrates the fact that even in a minimum sized window, not all undivided windows contain only one character.

Consider Tree 4 and Window 4. The main window is split into a left window with Tree 2 and right window with Tree 3. The left window is like Window 2, but the right window is not just like Window 3. The heights of left and right subwindows must match, so the right window must be stretched.

The stretching rule depends on a definition of the size of windows. For dimension calculations it is easiest to imagine that a window contains its interior and a half character wide boundary on all sides, so the total dimensions of a window are one more than the dimensions of the interior. Hence the minimum dimensions of a window are 2 by 2, since a window must contain one character inside, and we add one for the boundary. This definition also means that the sum of the widths of left and right subwindows is the width of their enclosing window. The sum of the heights of top and bottom subwindows is the height of their enclosing window.

The right window in Window 4 must be stretched to match the height 10 of the left window. The right window is split into a top with tree P having minimum height 2 and a bottom with tree -|Q|RST having minimum height 4. The rule for the dimensions in the stretched window is that the heights of the subwindows expand in proportion to their minimum heights, if possible. Some symbols may help here: Let D = 10 be the height of the combined stretched window. We want to determine D1 and D2, the stretched heights of the top and bottom subwindow. Call the corresponding minimum dimensions d = 6, d1 = 2, and d2 = 4. If the window were expanded from a total height d to D in proportion, we would have D1 = d1*(D/d) = 2*(10/6) = 3.333...and D2 = d2*(D/d) = 6.666.... Since the results are not integers we increase D1 to 4 and decrease D2 to 6.

There is a similar calculation for the bottom window with tree -|Q|RST. It is further subdivided into a top with tree |Q|RS and a bottom with tree T, each having minimum height 2 = d1 = d2. The heights need to add up to D = 6, so they are increased proportionally to D1 = D2 = 2*(6/4) = 3 (exact integers).

The final dimensions of an enclosing window are always determined before the final dimensions of its subwindows. In this example only heights needed to be apportioned. If all horizontal and vertical splits were interchanged in this example, producing a tree -|-|ABC|D-E|FG|P|-Q-RST, then widths would be apportioned correspondingly, as shown in the third part of the sample output below. If the proportion calculations do not work out to integers, it is always the top or left subwindow whose dimension is increased to the next integer.
Input

The first line of input contains one integer, which is the total number of preorder traversals describing window structures. This line is followed by one line for each preorder traversal. Each preorder traversal will contain appropriate dividers '|' and '-' and from 1 to 26 uppercase letters.
Output

For each preorder traversal, print the number of the preorder traversal on one line followed by the minimum sized window grid that the traversal could represent.
Sample Input

3
|-MCR
|-|-ABC-D|E-FG-P-|Q|RST
-|-|ABC|D-E|FG|P|-Q-RST
Sample Output

1
M-R-*
| | |
C-* |
| | |
--*
2
A-C-P-----*
| | | |
B-* | |
| | | |
D-*-Q-R-S-*
| | | | |
E-F-* | | |
| | T-*-*-*
| G-* |
| | | |
--*-----*
3
A-B-D-E---*
| | | | |
C-*-* F-G-*
| | | | |
P---Q-*T*-*
| | | |
| R--* |
| | | |
| S--* |
| | | |
-----*--*

2个回答

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
爬虫(python):为什么验证码总是验证失败(内有描述)?
#这是一个试图模拟12306登陆的程序,只到验证码部分 import urllib.request as U import urllib.parse as P import http.cookiejar as C import ssl import chardet as cd ssl._create_default_https_context = ssl._create_unverified_context #无视证书的有效性 opener = U.build_opener(U.HTTPCookieProcessor(C.CookieJar())) U.install_opener(opener) #创建一个访问者(具有cookie功能) req = U.Request("https://kyfw.12306.cn/passport/captcha/captcha-image64?login_site=E&module=login&rand=sjrand&1581337391968&callback=jQuery19109972447551572461_1581326959299&_=1581326959322") req.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" res = opener.open(req) #申请验证码 url = "data:image/jpg;base64," + res.read().decode("utf-8").split('({"image":"')[1].split('","result_message"')[0] #12306分为申请验证码和生成两部分,这是根据两部分的URL规律,生成的验证码图片的URL req = U.Request(url) req.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" res = opener.open(req) code_img = res.read() with open("D:\\py\\测试_练习综合体\\py练习\\imagecode12306.png","wb") as f: f.write(code_img) #获取验证码 pass_code = input("请输入验证码(坐标):") #根据图片获取验证码坐标 data = {"callback":"jQuery19109972447551572461_1581326959299","answer":pass_code,"rand":"sjrand","login_site":"E","_":"1581326959323"} data = P.urlencode(data).encode("utf-8") req = U.Request("https://kyfw.12306.cn/passport/captcha/captcha-check?callback=jQuery19109972447551572461_1581326959299&answer=188%2C49%2C30%2C39&rand=sjrand&login_site=E&_=1581326959323") req.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" res = opener.open(req,data = data) html = res.read().decode("utf-8") #验证码验证 #疑问:为什么验证码验证总是失败了(通过html获得结果)
XPath无法准确获取怎么办
参照《从零开始学网络爬虫》案例,爬取豆瓣图书Top250的信息 https://book.douban.com/top250 爬取前需要用XPath获取书名、作者等标签信息,在浏览器中检查网页信息,并右击,copy XPath获取元素的XPath ![图片说明](https://img-ask.csdn.net/upload/202002/15/1581778537_466127.png) 书中原版代码如下 ``` import csv from lxml import etree import requests headers = { 'user-agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36' } urls = ['https://book.douban.com/top250?start={}'.format(str(i)) for i in range(0,250,25)] wenben = open('E:\demo.csv','wt',newline='',encoding='utf-8') writer = csv.writer(wenben) writer.writerow(('name','url','author','publisher','date','price','rate','comment')) for url in urls: html = requests.get(url,headers=headers) selector = etree.HTML(html.text) infos = selector.xpath('//tr[@class="item"]') for info in infos: name = info.xpath('td/div/a/@title')[0] url = info.xpath('td/div/a/@href')[0] book_infos = info.xpath('td/p/text()')[0] author = book_infos.split('/')[0] publisher = book_infos.split('/')[-3] date = book_infos.split('/')[-2] price = book_infos.split('/')[-1] rate = info.xpath('td/div/span[2]/text()')[0] comments = info.xpath('td/div/span[2]/text()')[0] comment = comments[0] if len(comments) != 0 else "空" writer.writerow((name,url,author,publisher,date,price,rate,comment)) print(name) wenben.close() print("输出完成!") ``` 可以发现,以书名为例,原版中获取的XPath如下 ``` 'td/div/a/@title' ``` 但是我通过浏览器检查元素获取到的XPath如下 ``` *[@id="content"]/div/div[1]/div/table[1]/tbody/tr/td[2]/div[1]/a ``` 而且按照自己获取的XPath进行爬取,并不能爬取到网页信息。只有按照原版的XPath才能正确爬取到网页信息。 请问各位大神,为什么从浏览器端获取的XPath与案例并不一致,如何自行获取正确的XPath
c# 调用python突然出现问题,新人求助,可追加悬赏
c# 调用python突然出现问题,我之前用cmd调用python做GNSS,今天准备改改程序,手贱无聊用火绒清理下文件与注册表。然后未动之前成功的代码,运行出现python输出为null,我重新搞了python环境依然没修好(py测试文件能从vs中启动)。如下C#测试代码如下: using System.Diagnostics; using System.Text.RegularExpressions; using System.Windows; using System.Windows.Controls; using System.Windows.Input; namespace test { /// <summary> /// MainWindow.xaml 的交互逻辑 /// </summary> public partial class MainWindow : Window { static double [] intArr; public MainWindow() { InitializeComponent(); } private void CheckBox1_Checked(object sender, RoutedEventArgs e) { } private void CheckBox_Checked(object sender, RoutedEventArgs e) { } private static double Azimuth() { double Ea = 6378.137; // 赤道半径 Km double Eb = 6356.725; var ec = Eb + (Ea - Eb) * (90.0 - 180) / 90.0; var ed = ec * Math.Cos(DegToRad(148)); var dx = ed * DegToRad(30 - 30); var dy = ec * DegToRad(180 -0); double theta = RadToDeg(Math.Atan2(dx, dy)); if (theta < 0) { theta = 360 + theta; } return theta; } private void Button_Click(object sender, RoutedEventArgs e) { if (!String.IsNullOrEmpty(txtName.Text))//字符非空 { if (txtName.Text == "11") { string[] strArr = new string[2];//参数列表 string sArguments = @"main.py";//这里是python的文件名字 strArr[0] = "40"; strArr[1] = "116"; RunPythonScript(sArguments, "-u", strArr); double[] intArrr = intArr; string c = intArr[0].ToString(); MessageBox.Show(c); } } } public static double rad(double d) { return d * Math.PI / 180.0; } private static double DegToRad(double d) { return d * Math.PI / 180d; } public static double GetAzimuth() { double lat1 = rad(148); double lat2 = rad(150); double lon1 = rad(32); double lon2 = rad(30); double azimuth = Math.Sin(lat1) * Math.Sin(lat2) + Math.Cos(lat1) * Math.Cos(lat2) * Math.Cos(lon2 - lon1); azimuth = Math.Sqrt(1 - azimuth * azimuth); azimuth = Math.Cos(lat2) * Math.Sin(lon2 - lon1) / azimuth; azimuth = Math.Asin(azimuth) * 180 / Math.PI; if (Double.IsNaN(azimuth)) { if (lon1 < lon2) { azimuth = 90.0; } else { azimuth = 270.0; } } return azimuth; } public struct PointLonLatAlt { public double Lon; public double Lat; public double Alt; public PointLonLatAlt(double lon, double lat, double alt) { Lon = lon; Lat = lat; Alt = alt; } } //将弧度转化位角度 public static double RadToDeg(double radian) { return radian * 180.0 / Math.PI; } public static void RunPythonScript(string sArgName, string args = "", params string[] teps) { Process p = new Process(); //string path = System.AppDomain.CurrentDomain.SetupInformation.ApplicationBase + sArgName;// 获得python文件的绝对路径(将文件放在c#的debug文件夹中可以这样操作) string path = @"D:\ana\Lib" + sArgName;//(因为我没放debug下,所以直接写的绝对路径,替换掉上面的路径了) p.StartInfo.FileName = @"D:\ana\python.exe";//(注意:用的话需要换成自己的)没有配环境变量的话,可以像我这样写python.exe的绝对路径(用的话需要换成自己的)。如果配了,直接写"python.exe"即可 string sArguments = path; foreach (string sigstr in teps) { sArguments += " " + sigstr;//传递参数 } sArguments += " " + args; p.StartInfo.Arguments = sArguments; p.StartInfo.UseShellExecute = false; p.StartInfo.RedirectStandardOutput = true; p.StartInfo.RedirectStandardInput = true; p.StartInfo.RedirectStandardError = true; p.StartInfo.CreateNoWindow = false; p.Start(); p.BeginOutputReadLine(); p.OutputDataReceived += new DataReceivedEventHandler(p_OutputDataReceived); Console.ReadLine(); p.WaitForExit(); } //输出打印的信息 static void p_OutputDataReceived(object sender, DataReceivedEventArgs e) { if (!string.IsNullOrEmpty(e.Data)) { string b; Console.WriteLine(e.Data + Environment.NewLine); string a = e.Data; b = a; string[] strArr = a.Split( ' '); intArr= new double[strArr.Length]; for (int i = 0; i < strArr.Length; i++) { intArr[i] = Convert.ToDouble(strArr[i]); } a = b; ; } } private void TextBox_TextChanged(object sender, TextChangedEventArgs e) { } private void textname(object sender, TextCompositionEventArgs e) { } } 其中e.date为参数传回C#,一直为null。代码本身在之前测试没有问题,今天 不知为何。我怀疑cmd脚本没有输入python脚本数据。
python程序运行成功,为什么没有爬取到具体的网址??
import requests import re headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) ' 'Chrome/79.0.3941.4 Safari/537.36'} def sou_gou(company): url = 'https://news.sogou.com/news?query=' + company + '&shid=djt1' res = requests.get(url, headers=headers).text p_href = '<h3 class="vrTitle">.*?<a href="(.*?)"' p_title = '<h3 class="vr-Title">.*?<a href=.*?" target="_blank">(.*?)</a>' p_info = '<p class="news-from">(.*?)</p>' href = re.findall(p_href, res, re.S) title = re.findall(p_title, res, re.S) info = re.findall(p_info, res, re.S) file1 = open('E:\\python数据爬取\\搜狗数据挖掘报告.txt', 'a') file1.write(company + '数据挖掘completed!' + '\n' + '\n') source = [] date = [] for i in range(len(title)): title[i] = title[i].strip() title[i] = re.sub('<.*?>', '', title[i]) info[i] = re.sub('<.*?>', '', info[i]) source.append(info[i].split('&nbsp;')[0]) date.append(info[i].split('&nbsp;')[1]) source[i] = source[i].strip() date[i] = date[i].strip() file1.write(str(i + 1) + '.' + title[i] + '(' + date[i] + '-' + source[i] + ')' + '\n') file1.write(href[i] + '\n') file1.write('-------------------' + '\n' + '\n') file1.close() company_name = ['华能信托', '阿里巴巴', '万科集团', '百度', '腾讯', '京东'] for j in company_name: sou_gou(j) print(j + '搜狗新闻爬取成功')
求助,python 报错:AttributeError: module 'log0' has no attribute 'out'怎么办?
python代码: ``` from downloader import Downloader #, cStringIO, cPickle from threading import Thread from time import sleep import log0 as log from os.path import basename import requests as req import pickle from os.path import exists db='E:/tmp/download.data' def append(obj): try: if exists(db): with open(db,'rb') as f: data=pickle.load(f) else: data={} except: data={} data[obj['url']]=obj with open(db,'wb') as f: pickle.dump(data,f) def load(url): if not exists(db): return None try: with open(db,'rb') as f: data=pickle.load(f) return data.get(url) except: return None def out(msg): print(msg) import time from os.path import basename, exists, getsize from queue import Queue from threading import Lock, Thread, current_thread import requests as req import random as rand import conf class Downloader: KB=1024 MB=KB*KB GB=KB*MB range_size=MB max_workers=10 spd_refresh_interval=1 user_agents=[ 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246', 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1', 'Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36' 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36', 'Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0' ] chunk_size=KB max_error=0.1 #单线程允许最大出错率 max_error_one_worker=0.5 #仅剩一个线程时允许的最大出错率 home='E:/tmp/' #下载目录 def __init__(self,c): self.__locks={i:Lock() for i in ('file','worker_info','itr_job','download_info')} self.__config=c self.__alive=False self.__fails=Queue() self.__conf=c c=conf.load(c['url']) if c: self.__conf=c self.__init_from_conf() else: self.__init_task() def __init_from_conf(self): self.__download_offset=self.__conf['offset'] for i in self.__conf['fails']: self.__fails.put(i) def __get_agent(self): return self.user_agents[rand.randint(0,len(self.user_agents)-1)] def __init_task(self): headers={'Range':'bytes=0-0'} headers['User-Agent']=self.__get_agent() print(headers) try: r=req.get(self.__conf['url'],headers=headers,stream=True) self.__conf['name'] = basename(self.__conf['url']) or str(int(round(time.time()*1000))) self.__conf['206'] = r.status_code == 206 or r.headers.get('Accept-Ranges')=='bytes' if self.__conf['206']: self.__conf['len']=int(r.headers['Content-Range'].split('/')[-1]) elif r.status_code!=200: log.out('init task err') return else: self.__conf['len']=int(r.headers['Content-Length']) r.close() self.__download_offset=0 self.__conf['init']=True except Exception as e: log.out(e) def __itr_job(self): if self.__locks['itr_job'].acquire(): if not self.__fails.empty(): ans=self.__fails.get() elif self.__download_offset<self.__conf['len']: o=self.__download_offset ans=(o,min(self.__conf['len']-1,o+self.range_size-1)) self.__download_offset+=self.range_size else: ans=(-1,-1) self.__locks['itr_job'].release() return ans def __has_job(self): if self.__locks['itr_job'].acquire(): ans=self.__download_offset<self.__conf['len'] or not self.__fails.empty() self.__locks['itr_job'].release() return ans def __download_no_206(self): headers={'User-Agent':self.__get_agent()} r=req.get(self.__conf['url'],headers=headers,stream=True) self.__download_offset=0 if r.status_code != 200: r.close() self.__stopped() return try: for con in r.iter_content(chunk_size=self.chunk_size): if self.__kill_signal: break self.__file.write(con) l=len(con) self.__down_bytes+=l self.__download_offset+=l t0=time.time() t=t0-self.__last_time if t>=self.spd_refresh_interval: self.__down_spd=self.__down_bytes/t log.out('downspd: %d KB/s'%(self.__down_spd/self.KB)) self.__last_time=t0 self.__down_bytes=0 except: pass r.close() self.__stopped() def __download_206(self): file_len=self.__conf['len'] total=0 error=0 kill=False with req.session() as sess: while True: s,e=self.__itr_job() if s==-1: log.out('no job stop') break headers={'Range':'bytes=%d-%d'%(s,e)} headers['User-Agent']=self.__get_agent() try: r=sess.get(self.__conf['url'],headers=headers,stream=True) total+=1 if r.status_code!=206: self.__fails.put((s,e)) error+=1 if error>self.max_error*total: if self.__locks['worker_info'].acquire(): num=self.__current_workers self.__locks['worker_info'].release() if error>self.max_error_one_worker*total or num>1: break continue for con in r.iter_content(chunk_size=self.chunk_size): if self.__locks['worker_info'].acquire(): if self.__kill_signal: self.__locks['worker_info'].release() kill=True break self.__locks['worker_info'].release() if self.__locks['file'].acquire(): self.__file.seek(s) self.__file.write(con) l=len(con) s+=l self.__locks['file'].release() if self.__locks['download_info'].acquire(): self.__down_bytes+=l t0=time.time() t=t0-self.__last_time if t>=self.spd_refresh_interval: log.out('downspd: %d KB/s'%(self.__down_spd/self.KB)) self.__down_spd=self.__down_bytes/t self.__down_bytes=0 self.__last_time=t0 self.__locks['download_info'].release() if s<=e and s<file_len: self.__fails.put((s,e)) if kill: break except : self.__fails.put((s,e)) error+=1 if error>self.max_error*total: if self.__locks['worker_info'].acquire(): num=self.__current_workers self.__locks['worker_info'].release() if error>self.max_error_one_worker*total or num>1: break self.__stopped() def __start_worker(self,target): if self.__locks['worker_info'].acquire(): if self.__kill_signal: self.__locks['worker_info'].release() return False if self.__current_workers<self.max_workers: Thread(target=target).start() self.__current_workers+=1 log.out('new worker started,current workers %d'%self.__current_workers) self.__locks['worker_info'].release() return True def __start_workers(self): for _ in range(self.max_workers): if not self.__start_worker(self.__download_206): break time.sleep(0.8) def start(self): if self.__alive: log.out('already started!') return if self.__conf.get('status')=='done': log.out('already done') return self.__alive=True self.__kill_signal=False self.__conf['status']='working' self.__down_bytes=0 self.__down_spd=0 self.__last_time=0 self.__current_workers=0 self.__start_time=time.time() try: path=self.home+self.__conf['name'] self.__file=open(path,(exists(path) and 'rb+') or 'wb' ) if not self.__conf['206']: Thread(target=self.__start_workers).start() else: self.__start_worker(self.__download_no_206) log.out('starting done!') except: log.out('starting failed') def stop(self): if self.__kill_signal: return log.out('stopping') if self.__locks['worker_info'].acquire(): self.__kill_signal=True if self.__conf['status']=='working': self.__conf['status']='stopped' self.__locks['worker_info'].release() def __after_stopped(self): if not self.__kill_signal: self.__kill_signal=True __alive=False self.__file.close() log.out('total time: %.2f'%(time.time()-self.__start_time)) self.__conf['offset']=self.__download_offset if not self.__has_job(): self.__conf['status']='done' elif self.__conf.get('status')!='stopped': self.__conf['status']='error' leak=0 ls=[] while not self.__fails.empty(): i=self.__fails.get() leak+=i[1]-i[0]+1 ls.append(i) self.__conf['fails']=ls leak+=max(self.__conf['len']-self.__download_offset,0) log.out('total leak: %d'%leak) conf.append(self.__conf) def __stopped(self): if self.__locks['worker_info'].acquire(): self.__current_workers-=1 log.out('%s stopped'%current_thread().name) if self.__current_workers==0: self.__after_stopped() self.__locks['worker_info'].release() #!/usr/bin/env python # coding=utf-8 #import importlib,sys #import sys #sys.setdefaultencoding('gbk') '''import sys import imp import sys reload(sys) sys.setdefaultencoding('utf8') ''' ''' import sys sys.setdefaultencoding('utf-8') import jieba import json''' def main(): from bs4 import BeautifulSoup import urllib.request import urllib.parse as parse import ssl import re import os,os.path import codecs import requests def getHtml(url): global html page = urllib.request.urlopen(url) html = page.read() return html def file(url1,file_name,name): print(url1) #file(name,save_path,filename) #url1= +'/' + filename url1=url1.encode() #file = open(name ,'wb+') #file.write(url1 ) #file.close() #print(file_name) headers = {'Host': 'https://files.pythonhosted.org/packages/','User-Agent':'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER','Referer': 'https://pypi.org/', 'Connection': 'keep-alive', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Accept-Language': 'zh-CN,zh;q=0.8'} #req = urllib.urlretrieve(download_url,headers=headers) #urllib.request.urlopen('https://www.lfd.uci.edu/~gohlke/pythonlibs/') #req = urllib.request.Request(url=url,headers=header) #request = urllib.request.urlopen(url1) #response = urllib.request.urlopen(request) import socket import urllib.request #设置超时时间为30s socket.setdefaulttimeout(5) #解决下载不完全问题且避免陷入死循环 '''try: urllib.request.urlretrieve(url1.decode(),name) except socket.timeout:''' count = 1 while count <= 1: import time # 格式化成2016-03-20 11:45:39形式 print(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())) # 格式化成Sat Mar 28 22:24:24 2016形式 print(time.strftime("%a %b %d %H:%M:%S %Y", time.localtime())) # 将格式字符串转换为时间戳 a = "Sat Mar 28 22:24:24 2016" print(time.mktime(time.strptime(a,"%a %b %d %H:%M:%S %Y"))) try: urllib.request.urlretrieve(url1.decode(),name) print('\nchangshi'+str(count)+'over\n') break except socket.timeout: err_info = 'Reloading for %d time'%count if count == 1 else 'Reloading for %d times'%count print(err_info) count += 1 except urllib.error.HTTPError: print('urllib.error.HTTPError') except urllib.error.URLError: print('urllib.error.URLError') except ssl.SSLWantReadError: print('ssl.SSLWantReadError') if count > 1: print("downloading picture fialed!") #urllib.request.urlretrieve(url1.decode(),name) global i i += 1 print(url1.decode()) #file = open(name ,'wt+') #file.write(str(req.content())) #file.close() print(file_name) global x print("Completed : .... %d ..." % x) '''for i in range(len(name_list)): j=0 if name_list[i-24:i+1]=='https://pypi.org/project/': name_list1.append(name_list[i+1:i+60])''' print('\n........'+name+'..........complete\n') '''headers = {'Host': 'download.lfd.uci.edu','User-Agent':'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER','Referer': 'https://www.lfd.uci.edu/~gohlke/pythonlibs/', 'Connection': 'keep-alive', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Accept-Language': 'zh-CN,zh;q=0.8'} #req = urllib.urlretrieve(download_url,headers=headers) #urllib.request.urlopen('https://www.lfd.uci.edu/~gohlke/pythonlibs/') #req = urllib.request.Request(url=url,headers=header) request = requests.get(url=url1,headers=headers) #response = urllib.request.urlopen(request) global i i += 1 file = open(name ,'wb+') file.write(request.content) file.close() print(file_name) print("Completed : .... %d ..." % x)''' save_path = os.getcwd() url = 'https://www.lfd.uci.edu/' html = getHtml(url) html=''' </li> <li><a id="imagecodecs-lite"></a><strong><a href="https://www.lfd.uci.edu/~gohlke/#python">Imagecodecs-lite</a></strong> (deprecated): a subset of <a href="https://www.lfd.uci.edu/~gohlke/pythonlibs/#imagecodecs">imagecodecs</a>. <ul> <li><a href="javascript:;" onclick=" javascript:dl([101,99,106,112,118,103,115,49,47,119,116,45,104,111,95,51,48,108,105,50,53,101,113,109,97,46,110,121,100], &quot;5B1E23C97AFG4D0&lt;KD05=@A9D:B?B?H6H&gt;6:2J&gt;:I&lt;ID:GIJH8;@&quot;); &quot;javascript: dl(&quot;" title="[1 KB] [Feb 17, 2020]">imagecodecs_lite‑2020.1.31‑py3‑none‑any.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,97,51,110,111,49,45,116,106,101,99,113,105,119,50,108,95,115,48,52,100,118,56,53,47,54,112,103,104,46,57,109], &quot;@=7:IDF6G;N0J893C89@?&gt;;685=A4ML4=L159I1E59I1E5&lt;;2?0NCHBL&lt;K&gt;&quot;); &quot;javascript: dl(&quot;" title="[148 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp38‑cp38‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,49,47,119,100,46,48,110,99,115,50,104,45,57,51,111,108,113,97,56,112,95,106,109,103,116,105,101,53,118], &quot;89E@CLKH1IFAGJ7&gt;3J78D?IHJ;950&lt;4094=;7C=B;7C=B;2I6=942:?&quot;); &quot;javascript: dl(&quot;" title="[120 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp38‑cp38‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,99,112,108,105,119,50,109,103,113,110,45,57,55,48,49,115,118,47,100,111,53,51,95,97,104,101,116,54,46,106,52], &quot;?5M81@DJA36G7I0CBI0?F23JI:5=&gt;;L&gt;5LE:01E&lt;:01E&lt;6:439FG6BKNL4H2&quot;); &quot;javascript: dl(&quot;" title="[145 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp37‑cp37m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,57,47,55,112,101,49,105,106,115,95,99,51,116,50,110,113,53,45,48,108,97,46,118,119,109,111,104,100,103], &quot;8=7?3F@&lt;16HDL4:IK4:89C6&lt;4A=B50E5=E;A:3;2A:3;2HAG6&gt;;=EGJC&quot;); &quot;javascript: dl(&quot;" title="[118 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp37‑cp37m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,115,119,50,54,47,111,112,101,48,105,103,110,100,53,109,99,45,46,97,106,51,52,118,104,95,49,116,108,113,57], &quot;02CL6F=J49&gt;B:7?5&lt;7?0HK9J7@28IMAI2AD@?6D3@?6D3&gt;@19;HB&gt;&lt;3EA1GK&quot;); &quot;javascript: dl(&quot;" title="[137 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp36‑cp36m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,50,110,116,51,97,111,104,45,95,57,103,101,99,119,115,105,118,54,108,113,112,100,109,106,49,47,46,53,48], &quot;&gt;0GCD@K2I?F4:;&lt;5E;&lt;&gt;8B?2;70LH9JH0J37&lt;D3A7&lt;D3AF7=?130J=6B&quot;); &quot;javascript: dl(&quot;" title="[112 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp36‑cp36m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,53,49,51,113,100,108,47,52,118,54,46,106,105,109,99,57,112,103,97,101,110,115,48,95,104,119,50,45,116,111], &quot;EJ;3@80L6&gt;@206&lt;=BAC&gt;M4C&gt;EG5&lt;LCKJF1?:1J:2K&gt;@20K&gt;@20=KI&lt;DGB=497:IH5&quot;); &quot;javascript: dl(&quot;" title="[133 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp35‑cp35m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,46,106,118,49,101,51,109,119,108,47,104,116,53,99,113,50,105,45,100,57,111,112,48,115,95,110,97,103], &quot;G?1&gt;E2&lt;;9=E5&lt;9@6JK4=DB4=GH8@;4A?F3C03?05A=E5&lt;A=E5&lt;6A7@I5?07:8&quot;); &quot;javascript: dl(&quot;" title="[110 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp35‑cp35m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,54,118,48,53,115,45,112,116,110,106,51,46,100,108,99,52,109,55,50,105,47,49,104,97,119,113,103,111,101,95,57], &quot;4B9I6137D&gt;6BADC@GJL&gt;K&lt;L&gt;4M=C7L5B2EN;EB;:5&gt;6BA5&gt;6BA@5HC8MG@&lt;0?;HF=&quot;); &quot;javascript: dl(&quot;" title="[145 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp27‑cp27m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,115,51,118,57,104,112,50,110,48,103,109,55,95,49,100,108,116,101,106,45,113,53,46,47,111,99,119,97,105], &quot;06BD52E@GI56;GL:K9AIH&gt;AI0&lt;?L@AC68=3F=6F1CI56;CI56;:CJL716FJ4?&quot;); &quot;javascript: dl(&quot;" title="[120 KB] [Dec 04, 2019]">imagecodecs_lite‑2019.12.3‑cp27‑cp27m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,112,116,57,114,103,106,100,122,97,101,115,46,51,111,47,48,105,99,49,108,113,53,50,109,45,118], &quot;:F5D0IE1&gt;@G849A=69A:HC@19HF?B2;BF;&lt;;183;47&quot;); &quot;javascript: dl(&quot;" title="[1.1 MB] [Dec 04, 2019]">imagecodecs‑lite‑2019.12.3.tar.gz</a></li> ''' print('html done') #html.decode('utf-8') #print(html) '''headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1)AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11'} r = requests.get(url, headers = headers) r.encoding = "utf-8" soup = BeautifulSoup(r.text, "html.parser") #html_mod=re.sub(pattern=".",repl=".",string=html.decode('utf-8')) for link in soup.find_all('a'): #soup.find_all返回的为列表 print(link.get('href')) #name_list+=link ''' name_list = html#soup.find_all('a')#re.findall(r']">*-cp38-win_amd64.whl',html.decode('utf-8')) x=1 files=os.listdir(save_path) print(files) print(type(name_list)) name_list=str(name_list) name_list1=[] #print(name_list) #for name in name_list: k=0 # name[k]=str(name1[k]) for i in range(len(name_list)): j=0 if name_list[i-2:i+1]==']">': name_list1.append(name_list[i+1:i+60]) global m if k<len(name_list1): for l in range(len(name_list1[k])): if l-9>=0: if name_list1[k][l-4:l]=='.whl' or name_list1[k][l-3:l]=='.gz' or name_list1[k][l-4:l]=='.zip': j=1 m=l if j==1: name_list1[k]=name_list1[k][0:m] k+=1 '''if j==0: name_list.remove(name)''' #file_name = os.path.join(save_path ,name) i=0 #print(name) print(name_list1) for name in name_list1: j=0 for l in range(len(name)): if l-9>=0: if name[l-4:l]=='.whl' or name[l-3:l]=='.gz' or name[l-4:l]=='.zip': j=1 m=l if j==1: name=name[0:m] k+=1 if name in files: continue '''if name=='Delny‑0.4.1‑cp27‑none‑win_amd64.whl</a></li>\n<li>' or name==Delny‑0.4.1‑cp27‑none‑win32.whl</a></li> </ul> </: continue ''' print('no:'+str(x)) print('\ndownload '+name) # importlib.reload(sys) #imp.reload(sys) for l in range(len(name)): if l-9>=0: if name[l-4:l]=='.whl' or name[l-3:l]=='.gz' or name[l-4:l]=='.zip': j=1 m=l if j==1: name=name[0:m] k+=1 string='https://download.lfd.uci.edu/pythonlibs/s2jqpv5t/' + name#[0:4+name.find('.whl')]#https://download.lfd.uci.edu/pythonlibs/s2jqpv5t/ print('00'+save_path) count=0 v=0 for p in range(len(string)): if string[p]=='\\': if v==0: string=string[:6]+'//'+string[7:] else: string=string[:p]+'/'+string[p+1:] v+=1 if string[p-3:p]=='win': string=string[:p-4]+'-'+string[p-3:] if p<len(string): if (string[p]=='\u2011')==True: if p+1<len(string): string=string[:p]+'-'+string[p+1:] '''if string[p-2]>='0' and string[p-2]<='9' and string[p-1]>='0' and string[p-1]<='9': if (string[p]>='a'and string[p]<='z') or (string[p]>='A'and string[p]<='Z'): string=string[:p]+string[p+1:]''' if p>=len(string): break '''if name[:9]=='ad3‑2.2.1': print('aaa') continue''' conf={'url':string} d=Downloader(conf) d.start() #file(string,save_path,name) x=x+1 print('09'+name_list) print('finished') if __name__ == '__main__': main() ``` 报错: >>> ======================== RESTART: E:\2345Downloads\44.py ======================= Warning: This project has moved to logzero (see https://github.com/metachris/logzero) html done <class 'str'> ['imagecodecs_lite‑2020.1.31‑py3‑none‑any.whl', 'imagecodecs_lite‑2019.12.3‑cp38‑cp38‑win_amd64.whl', 'imagecodecs_lite‑2019.12.3‑cp38‑cp38‑win32.whl', 'imagecodecs_lite‑2019.12.3‑cp37‑cp37m‑win_amd64.whl', 'imagecodecs_lite‑2019.12.3‑cp37‑cp37m‑win32.whl', 'imagecodecs_lite‑2019.12.3‑cp36‑cp36m‑win_amd64.whl', 'imagecodecs_lite‑2019.12.3‑cp36‑cp36m‑win32.whl', 'imagecodecs_lite‑2019.12.3‑cp35‑cp35m‑win_amd64.whl', 'imagecodecs_lite‑2019.12.3‑cp35‑cp35m‑win32.whl', 'imagecodecs_lite‑2019.12.3‑cp27‑cp27m‑win_amd64.whl', 'imagecodecs_lite‑2019.12.3‑cp27‑cp27m‑win32.whl', 'imagecodecs‑lite‑2019.12.3.tar.gz'] no:1 download imagecodecs_lite‑2020.1.31‑py3‑none‑any.whl 00E:\2345Downloads Warning (from warnings module): File "C:\Users\ASUS\AppData\Local\Programs\Python\Python38\lib\site-packages\conf\reader.py", line 39 warnings.warn('cannot parse files of type "%s"' % suffix) UserWarning: cannot parse files of type ".whl" {'Range': 'bytes=0-0', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'} Traceback (most recent call last): File "E:\2345Downloads\44.py", line 254, in start self.__file=open(path,(exists(path) and 'rb+') or 'wb' ) FileNotFoundError: [Errno 2] No such file or directory: 'E:/tmp/imagecodecs_lite-2020.1.31-py3-none-any.whl' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "E:\2345Downloads\44.py", line 616, in <module> main() File "E:\2345Downloads\44.py", line 606, in main d.start() File "E:\2345Downloads\44.py", line 259, in start except: log.out('starting failed') AttributeError: module 'log0' has no attribute 'out' >>> 求高手解决
刚学Python,爬网站图片的时候代码报“”label empty or too long“”错误
下面是代码,刚开始的时候能爬一点图片,过几秒就报错,不知道是怎么回事,求大佬指点! ``` import requests import re for i in range(0,200): base_url = ("https://www.buxiuse.com/?page={}".format(i)) headers = { 'headers':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3947.100 Safari/537.36' } response = requests.get(base_url,headers=headers) page_urls = re.findall('<div class=".*?"> <span class=".*?"> <a href="(.*?)" class=".*?">',response.text) for urls in page_urls: response = requests.get(urls,headers=headers) jpg_urls = re.findall('<img src="(.*?)" width=".*?" referrerpolicy=".*?">',response.text) for jpg in jpg_urls: jpg_name = jpg.split('/')[-1] response = requests.get(jpg,headers=headers) with open('IMG\\' + jpg_name,'wb') as f: f.write(response.content) print("下载完成=======>",jpg_name) ```
python多线下载为什么下载不了?starting failed
代码: ``` from downloader import Downloader #, cStringIO, cPickle from threading import Thread from time import sleep import log2 as log from os.path import basename import requests as req import pickle from os.path import exists db='E:/tmp/download.data' def append(obj): try: if exists(db): with open(db,'rb') as f: data=pickle.load(f) else: data={} except: data={} data[obj['url']]=obj with open(db,'wb') as f: pickle.dump(data,f) def load(url): if not exists(db): return None try: with open(db,'rb') as f: data=pickle.load(f) return data.get(url) except: return None def out(msg): print(msg) import time from os.path import basename, exists, getsize from queue import Queue from threading import Lock, Thread, current_thread import requests as req import random as rand import conf class Downloader: KB=1024 MB=KB*KB GB=KB*MB range_size=MB max_workers=10 spd_refresh_interval=1 user_agents=[ 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246', 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1', 'Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36' 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36', 'Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0' ] chunk_size=KB max_error=0.1 #单线程允许最大出错率 max_error_one_worker=0.5 #仅剩一个线程时允许的最大出错率 home='E:/tmp/' #下载目录 def __init__(self,c): self.__locks={i:Lock() for i in ('file','worker_info','itr_job','download_info')} self.__config=c self.__alive=False self.__fails=Queue() self.__conf=c c=conf.load(c['url']) if c: self.__conf=c self.__init_from_conf() else: self.__init_task() def __init_from_conf(self): self.__download_offset=self.__conf['offset'] for i in self.__conf['fails']: self.__fails.put(i) def __get_agent(self): return self.user_agents[rand.randint(0,len(self.user_agents)-1)] def __init_task(self): headers={'Range':'bytes=0-0'} headers = {'Host': 'https://files.pythonhosted.org/packages/','User-Agent':'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER','Referer': 'https://pypi.org/', 'Connection': 'keep-alive', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Accept-Language': 'zh-CN,zh;q=0.8'} headers['User-Agent']=self.__get_agent() print(headers) try: r=req.get(self.__conf['url'],headers=headers,stream=True) self.__conf['name'] = basename(self.__conf['url']) or str(int(round(time.time()*1000))) self.__conf['206'] = r.status_code == 206 or r.headers.get('Accept-Ranges')=='bytes' if self.__conf['206']: self.__conf['len']=int(r.headers['Content-Range'].split('/')[-1]) elif r.status_code!=200: out('init task err') return else: self.__conf['len']=int(r.headers['Content-Length']) r.close() self.__download_offset=0 self.__conf['init']=True except Exception as e: out(e) def __itr_job(self): if self.__locks['itr_job'].acquire(): if not self.__fails.empty(): ans=self.__fails.get() elif self.__download_offset<self.__conf['len']: o=self.__download_offset ans=(o,min(self.__conf['len']-1,o+self.range_size-1)) self.__download_offset+=self.range_size else: ans=(-1,-1) self.__locks['itr_job'].release() return ans def __has_job(self): if self.__locks['itr_job'].acquire(): ans=self.__download_offset<self.__conf['len'] or not self.__fails.empty() self.__locks['itr_job'].release() return ans def __download_no_206(self): headers={'User-Agent':self.__get_agent()} r=req.get(self.__conf['url'],headers=headers,stream=True) self.__download_offset=0 if r.status_code != 200: r.close() self.__stopped() return try: for con in r.iter_content(chunk_size=self.chunk_size): if self.__kill_signal: break self.__file.write(con) l=len(con) self.__down_bytes+=l self.__download_offset+=l t0=time.time() t=t0-self.__last_time if t>=self.spd_refresh_interval: self.__down_spd=self.__down_bytes/t out('downspd: %d KB/s'%(self.__down_spd/self.KB)) self.__last_time=t0 self.__down_bytes=0 except: pass r.close() self.__stopped() def __download_206(self): file_len=self.__conf['len'] total=0 error=0 kill=False with req.session() as sess: while True: s,e=self.__itr_job() if s==-1: out('no job stop') break headers={'Range':'bytes=%d-%d'%(s,e)} headers['User-Agent']=self.__get_agent() try: r=sess.get(self.__conf['url'],headers=headers,stream=True) total+=1 if r.status_code!=206: self.__fails.put((s,e)) error+=1 if error>self.max_error*total: if self.__locks['worker_info'].acquire(): num=self.__current_workers self.__locks['worker_info'].release() if error>self.max_error_one_worker*total or num>1: break continue for con in r.iter_content(chunk_size=self.chunk_size): if self.__locks['worker_info'].acquire(): if self.__kill_signal: self.__locks['worker_info'].release() kill=True break self.__locks['worker_info'].release() if self.__locks['file'].acquire(): self.__file.seek(s) self.__file.write(con) l=len(con) s+=l self.__locks['file'].release() if self.__locks['download_info'].acquire(): self.__down_bytes+=l t0=time.time() t=t0-self.__last_time if t>=self.spd_refresh_interval: out('downspd: %d KB/s'%(self.__down_spd/self.KB)) self.__down_spd=self.__down_bytes/t self.__down_bytes=0 self.__last_time=t0 self.__locks['download_info'].release() if s<=e and s<file_len: self.__fails.put((s,e)) if kill: break except : self.__fails.put((s,e)) error+=1 if error>self.max_error*total: if self.__locks['worker_info'].acquire(): num=self.__current_workers self.__locks['worker_info'].release() if error>self.max_error_one_worker*total or num>1: break self.__stopped() def __start_worker(self,target): if self.__locks['worker_info'].acquire(): if self.__kill_signal: self.__locks['worker_info'].release() return False if self.__current_workers<self.max_workers: Thread(target=target).start() self.__current_workers+=1 out('new worker started,current workers %d'%self.__current_workers) self.__locks['worker_info'].release() return True def __start_workers(self): for _ in range(self.max_workers): if not self.__start_worker(self.__download_206): break time.sleep(0.8) def start(self): if self.__alive: out('already started!') return if self.__conf.get('status')=='done': out('already done') return self.__alive=True self.__kill_signal=False self.__conf['status']='working' self.__down_bytes=0 self.__down_spd=0 self.__last_time=0 self.__current_workers=0 self.__start_time=time.time() try: path=self.home+self.__conf['name'] self.__file=open(path,(exists(path) and 'rb+') or 'wb' ) if not self.__conf['206']: Thread(target=self.__start_workers).start() else: self.__start_worker(self.__download_no_206) out('starting done!') except: out('starting failed') def stop(self): if self.__kill_signal: return out('stopping') if self.__locks['worker_info'].acquire(): self.__kill_signal=True if self.__conf['status']=='working': self.__conf['status']='stopped' self.__locks['worker_info'].release() def __after_stopped(self): if not self.__kill_signal: self.__kill_signal=True __alive=False self.__file.close() out('total time: %.2f'%(time.time()-self.__start_time)) self.__conf['offset']=self.__download_offset if not self.__has_job(): self.__conf['status']='done' elif self.__conf.get('status')!='stopped': self.__conf['status']='error' leak=0 ls=[] while not self.__fails.empty(): i=self.__fails.get() leak+=i[1]-i[0]+1 ls.append(i) self.__conf['fails']=ls leak+=max(self.__conf['len']-self.__download_offset,0) out('total leak: %d'%leak) conf.append(self.__conf) def __stopped(self): if self.__locks['worker_info'].acquire(): self.__current_workers-=1 out('%s stopped'%current_thread().name) if self.__current_workers==0: self.__after_stopped() self.__locks['worker_info'].release() #!/usr/bin/env python # coding=utf-8 #import importlib,sys #import sys #sys.setdefaultencoding('gbk') '''import sys import imp import sys reload(sys) sys.setdefaultencoding('utf8') ''' ''' import sys sys.setdefaultencoding('utf-8') import jieba import json''' def main(): from bs4 import BeautifulSoup import urllib.request import urllib.parse as parse import ssl import re import os,os.path import codecs import requests def getHtml(url): global html page = urllib.request.urlopen(url) html = page.read() return html def file(url1,file_name,name): print(url1) #file(name,save_path,filename) #url1= +'/' + filename url1=url1.encode() #file = open(name ,'wb+') #file.write(url1 ) #file.close() #print(file_name) headers = {'Host': 'https://files.pythonhosted.org/packages/','User-Agent':'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER','Referer': 'https://pypi.org/', 'Connection': 'keep-alive', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Accept-Language': 'zh-CN,zh;q=0.8'} #req = urllib.urlretrieve(download_url,headers=headers) #urllib.request.urlopen('https://www.lfd.uci.edu/~gohlke/pythonlibs/') #req = urllib.request.Request(url=url,headers=header) #request = urllib.request.urlopen(url1) #response = urllib.request.urlopen(request) import socket import urllib.request #设置超时时间为30s socket.setdefaulttimeout(5) #解决下载不完全问题且避免陷入死循环 '''try: urllib.request.urlretrieve(url1.decode(),name) except socket.timeout:''' count = 1 while count <= 1: import time # 格式化成2016-03-20 11:45:39形式 print(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())) # 格式化成Sat Mar 28 22:24:24 2016形式 print(time.strftime("%a %b %d %H:%M:%S %Y", time.localtime())) # 将格式字符串转换为时间戳 a = "Sat Mar 28 22:24:24 2016" print(time.mktime(time.strptime(a,"%a %b %d %H:%M:%S %Y"))) try: urllib.request.urlretrieve(url1.decode(),name) print('\nchangshi'+str(count)+'over\n') break except socket.timeout: err_info = 'Reloading for %d time'%count if count == 1 else 'Reloading for %d times'%count print(err_info) count += 1 except urllib.error.HTTPError: print('urllib.error.HTTPError') except urllib.error.URLError: print('urllib.error.URLError') except ssl.SSLWantReadError: print('ssl.SSLWantReadError') if count > 1: print("downloading picture fialed!") #urllib.request.urlretrieve(url1.decode(),name) global i i += 1 print(url1.decode()) #file = open(name ,'wt+') #file.write(str(req.content())) #file.close() print(file_name) global x print("Completed : .... %d ..." % x) '''for i in range(len(name_list)): j=0 if name_list[i-24:i+1]=='https://pypi.org/project/': name_list1.append(name_list[i+1:i+60])''' print('\n........'+name+'..........complete\n') '''headers = {'Host': 'download.lfd.uci.edu','User-Agent':'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER','Referer': 'https://www.lfd.uci.edu/~gohlke/pythonlibs/', 'Connection': 'keep-alive', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Accept-Language': 'zh-CN,zh;q=0.8'} #req = urllib.urlretrieve(download_url,headers=headers) #urllib.request.urlopen('https://www.lfd.uci.edu/~gohlke/pythonlibs/') #req = urllib.request.Request(url=url,headers=header) request = requests.get(url=url1,headers=headers) #response = urllib.request.urlopen(request) global i i += 1 file = open(name ,'wb+') file.write(request.content) file.close() print(file_name) print("Completed : .... %d ..." % x)''' save_path = os.getcwd() url = 'https://www.lfd.uci.edu/' html = getHtml(url) html=''' </li> <li><a id="kwant"></a><strong><a href="http://kwant-project.org/">Kwant</a></strong>: quantum transport simulations made easy.<br> Requires <a href="https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy">numpy+mkl</a> and <a href="https://www.lfd.uci.edu/~gohlke/pythonlibs/#tinyarray">tinyarray</a>. <ul> <li><a href="javascript:;" onclick=" javascript:dl([101,116,54,104,51,56,113,108,46,99,118,106,49,119,109,45,50,110,115,95,112,107,47,105,97,53,52,100], &quot;A?:5C9H0ED&lt;G@0&gt;;7I7;&gt;8C34&gt;8C34&gt;&lt;F@BG=J1I7&lt;26&quot;); &quot;javascript: dl(&quot;" title="[2.5 MB] [Jul 06, 2019]">kwant‑1.4.1‑cp38‑cp38‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,45,107,108,97,47,116,113,110,99,56,49,118,46,104,50,115,105,53,112,106,119,52,51], &quot;?&gt;C6B;A541D3750:&lt;E&lt;:08BF908BF90D@7F&gt;&lt;D=2&quot;); &quot;javascript: dl(&quot;" title="[2.1 MB] [Jul 06, 2019]">kwant‑1.4.1‑cp38‑cp38‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,46,48,105,104,110,51,107,108,99,115,118,109,113,55,100,53,47,54,50,49,119,45,116,112,97,95,52,106], &quot;9BK&lt;G:?F@6DH4FEC0J01E8G5=E8G5=;ED24IH;&gt;AJ0D37&quot;); &quot;javascript: dl(&quot;" title="[2.4 MB] [Feb 28, 2019]">kwant‑1.4.0‑cp37‑cp37m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,52,112,106,45,109,51,99,108,48,104,107,46,53,118,97,105,116,113,119,47,55,50,110,49,115], &quot;HE2A1=&lt;@C:B&gt;F@3G;0;83615D3615D43B?F5E;B97&quot;); &quot;javascript: dl(&quot;" title="[2.1 MB] [Feb 28, 2019]">kwant‑1.4.0‑cp37‑cp37m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,112,99,97,49,105,54,113,115,108,109,53,52,116,51,118,106,107,110,104,50,95,47,48,45,119,100,46], &quot;7C?60&gt;:&lt;E@H2A&lt;G3J;JFG10=5G10=59GH4AD29I5;JHB8&quot;); &quot;javascript: dl(&quot;" title="[2.4 MB] [Feb 28, 2019]">kwant‑1.4.0‑cp36‑cp36m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,47,48,107,53,108,49,119,52,105,110,115,50,104,112,106,116,45,51,113,99,97,46,118,109,54], &quot;:;&gt;B=F3?026D9?@5E7E1@C=AH@C=AHG@689A;E6&lt;4&quot;); &quot;javascript: dl(&quot;" title="[2.1 MB] [Feb 28, 2019]">kwant‑1.4.0‑cp36‑cp36m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,97,51,53,50,107,46,105,54,49,47,104,52,109,100,115,118,119,108,48,112,116,45,99,113,110,106,95], &quot;&gt;3IGC?2D9FC1294@0HDE85;5BEFC12EFC12&lt;E@6HJ0&lt;=7;5@:A&quot;); &quot;javascript: dl(&quot;" title="[2.4 MB] [Feb 28, 2019]">kwant‑1.4.0‑cp35‑cp35m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,46,52,50,104,49,118,119,99,48,107,113,97,115,47,51,45,105,110,112,53,108,106,116,109], &quot;&lt;2E:B5CF=7B&gt;C=96;AF?40108?7B&gt;C?7B&gt;CG?6@A&gt;2063D&quot;); &quot;javascript: dl(&quot;" title="[2.0 MB] [Feb 28, 2019]">kwant‑1.4.0‑cp35‑cp35m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,49,108,109,119,46,113,107,97,95,99,105,110,53,51,104,100,116,112,54,50,52,115,45,118,47,106], &quot;ECI5AG&lt;@H9A=DH637;@F04=4CF9A=DF9A=D2F3:;872?BD43&gt;1&quot;); &quot;javascript: dl(&quot;" title="[2.1 MB] [Jan 06, 2018]">kwant‑1.3.2‑cp34‑cp34m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,106,99,112,47,113,97,116,52,51,49,107,53,109,104,105,110,108,119,45,115,46,50,118], &quot;CE042F;6312873:A5?6B9D8DEB1287B1287&lt;BA&gt;?8EDA=@&quot;); &quot;javascript: dl(&quot;" title="[1.8 MB] [Jan 06, 2018]">kwant‑1.3.2‑cp34‑cp34m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,51,110,97,105,113,45,47,104,108,99,112,46,100,109,55,53,95,119,49,106,50,107,54,118,116,52,115], &quot;JDC4:G?H69:D&gt;6EA21H5B;B;059:D&gt;59:D&gt;=5A31@2=&lt;FI;A78&quot;); &quot;javascript: dl(&quot;" title="[13.5 MB] [May 15, 2017]">kwant‑1.1.3‑cp27‑cp27m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,55,113,108,46,104,107,106,49,115,118,99,50,119,47,45,51,97,116,110,53,109,112,105], &quot;8;61E9CA=:E;0=5&lt;@BA&gt;7373?&gt;:E;0&gt;:E;0D&gt;&lt;FB?;3&lt;42&quot;); &quot;javascript: dl(&quot;" title="[6.7 MB] [May 15, 2017]">kwant‑1.1.3‑cp27‑cp27m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,113,99,45,52,119,95,49,111,100,116,48,46,47,118,55,97,112,108,107,115,54,110,105,50,101,104,53,106,109], &quot;CGK0@=J9&lt;1@G&gt;&lt;B4?E926;:;J21@G&gt;2E7EH24FE5?L8D3;4IA&quot;); &quot;javascript: dl(&quot;" title="[13.4 MB] [Sep 11, 2015]">kwant‑1.0.5‑cp27‑none‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,106,47,105,101,113,111,97,108,50,107,45,112,104,118,48,49,110,119,99,115,53,51,46,55,116], &quot;C804;=DH1B;8G19A6@H:?F&gt;FD:B;8G:@5@3:A2@E8FA&lt;7&quot;); &quot;javascript: dl(&quot;" title="[6.7 MB] [Sep 11, 2015]">kwant‑1.0.5‑cp27‑none‑win32.whl</a></li> </ul> </li> <li><a id="la"></a><strong><a href="https://github.com/kwgoodman/la">La</a></strong>: aka larry, the labeled numpy array. <ul> <li><a href="javascript:;" onclick=" javascript:dl([101,97,109,99,108,48,51,46,110,54,50,105,47,95,53,104,113,55,45,100,112,118,52,101,115,106,116,119], &quot;G9H?CD=I;2C5=;30A46@646BFD4A2C5=A2C5=1AJ:7&lt;01B8E6J&gt;3&quot;); &quot;javascript: dl(&quot;" title="[139 KB] [Apr 11, 2016]">la‑0.7.0.dev0‑cp35‑cp35m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,108,55,110,99,112,48,47,116,106,109,115,119,118,100,53,51,104,45,105,113,46,101,50,97], &quot;:F8C4&lt;&gt;7634?&gt;60GA5D1D5D=E&lt;5A34?&gt;A34?&gt;9A;B2?FD;@0&quot;); &quot;javascript: dl(&quot;" title="[137 KB] [Apr 11, 2016]">la‑0.7.0.dev0‑cp35‑cp35m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,109,55,118,110,115,50,106,104,113,116,108,53,97,100,54,101,47,52,105,112,46,95,51,45,99,119,48], &quot;4568C2;9@HCFA@:&lt;GJD1DJD=?2JGHCFAGHCFA0GIB3E&lt;0=&gt;ADI7:&quot;); &quot;javascript: dl(&quot;" title="[137 KB] [Apr 11, 2016]">la‑0.7.0.dev0‑cp34‑cp34m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,105,48,55,112,101,52,53,99,51,115,50,109,97,47,110,118,113,106,116,108,46,119,104,100,45], &quot;9:A@3?6B=7385=C&lt;H1D2D1DG4?1H7385H7385;HE0&gt;8:DEFC&quot;); &quot;javascript: dl(&quot;" title="[136 KB] [Apr 11, 2016]">la‑0.7.0.dev0‑cp34‑cp34m‑win32.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,45,105,55,100,116,97,99,106,113,101,95,54,50,110,104,47,115,119,118,109,108,52,46,53,48,112], &quot;@&lt;78IBG4?6I&lt;2?D50HF2FHF39BH06I&lt;206I&lt;2C0A1=:5C3;EFA&gt;D&quot;); &quot;javascript: dl(&quot;" title="[137 KB] [Apr 11, 2016]">la‑0.7.0.dev0‑cp27‑cp27m‑win_amd64.whl</a></li> <li><a href="javascript:;" onclick=" javascript:dl([101,119,55,53,48,105,118,112,47,115,45,108,116,97,113,104,99,100,106,51,110,109,46,50,101], &quot;8FA=652;7?6F17:&lt;93E1E3E@G539?6F19?6F1D904CBFE0&gt;:&quot;); &quot;javascript: dl(&quot;" title="[136 KB] [Apr 11, 2016]">la‑0.7.0.dev0‑cp27‑cp27m‑win32.whl</a></li> </ul> </li> ''' print('html done') #html.decode('utf-8') #print(html) '''headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1)AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11'} r = requests.get(url, headers = headers) r.encoding = "utf-8" soup = BeautifulSoup(r.text, "html.parser") #html_mod=re.sub(pattern=".",repl=".",string=html.decode('utf-8')) for link in soup.find_all('a'): #soup.find_all返回的为列表 print(link.get('href')) #name_list+=link ''' name_list = html#soup.find_all('a')#re.findall(r']">*-cp38-win_amd64.whl',html.decode('utf-8')) x=1 files=os.listdir(save_path) print(files) print(type(name_list)) name_list=str(name_list) name_list1=[] #print(name_list) #for name in name_list: k=0 # name[k]=str(name1[k]) for i in range(len(name_list)): j=0 if name_list[i-2:i+1]==']">': name_list1.append(name_list[i+1:i+60]) global m if k<len(name_list1): for l in range(len(name_list1[k])): if l-9>=0: if name_list1[k][l-4:l]=='.whl' or name_list1[k][l-3:l]=='.gz' or name_list1[k][l-4:l]=='.zip': j=1 m=l if j==1: name_list1[k]=name_list1[k][0:m] k+=1 '''if j==0: name_list.remove(name)''' #file_name = os.path.join(save_path ,name) i=0 #print(name) print(name_list1) for name in name_list1: j=0 for l in range(len(name)): if l-9>=0: if name[l-4:l]=='.whl' or name[l-3:l]=='.gz' or name[l-4:l]=='.zip': j=1 m=l if j==1: name=name[0:m] k+=1 if name in files: continue '''if name=='Delny‑0.4.1‑cp27‑none‑win_amd64.whl</a></li>\n<li>' or name==Delny‑0.4.1‑cp27‑none‑win32.whl</a></li> </ul> </: continue ''' print('no:'+str(x)) print('\ndownload '+name) # importlib.reload(sys) #imp.reload(sys) for l in range(len(name)): if l-9>=0: if name[l-4:l]=='.whl' or name[l-3:l]=='.gz' or name[l-4:l]=='.zip': j=1 m=l if j==1: name=name[0:m] k+=1 string='https://download.lfd.uci.edu/pythonlibs/s2jqpv5t/' + name#[0:4+name.find('.whl')]#https://download.lfd.uci.edu/pythonlibs/s2jqpv5t/ print('00'+save_path) count=0 v=0 for p in range(len(string)): if string[p]=='\\': if v==0: string=string[:6]+'//'+string[7:] else: string=string[:p]+'/'+string[p+1:] v+=1 if string[p-3:p]=='win': string=string[:p-4]+'-'+string[p-3:] if p<len(string): if (string[p]=='\u2011')==True: if p+1<len(string): string=string[:p]+'-'+string[p+1:] '''if string[p-2]>='0' and string[p-2]<='9' and string[p-1]>='0' and string[p-1]<='9': if (string[p]>='a'and string[p]<='z') or (string[p]>='A'and string[p]<='Z'): string=string[:p]+string[p+1:]''' if p>=len(string): break '''if name[:9]=='ad3‑2.2.1': print('aaa') continue''' conf={'url':string} d=Downloader(conf) d.start() #file(string,save_path,name) x=x+1 print('09'+name_list) print('finished') if __name__ == '__main__': main() ``` 求高手解决
scrapy存到mysql查询无数据
## 1. 问题描述 尝试使用scrapy框架爬取网站,将爬取的数据存储到mysql数据库,执行完毕之后没有报错,但是我查询数据时,显示没有数据 (代码框架参考使用该博主代码尝试运行: https://www.cnblogs.com/fromlantianwei/p/10607956.html) ## 2. 部分截图 1. scrapy项目: ![图片说明](https://img-ask.csdn.net/upload/202003/04/1583310103_446281.png) 数据库创建: ![图片说明](https://img-ask.csdn.net/upload/202003/04/1583310345_774265.png) ##3. 相关代码 scrapy框架代码: (1)tencent爬虫文件 ``` # -*- coding: utf-8 -*- import scrapy from urllib import parse import re from copy import deepcopy from ScrapyPro3.items import ScrapyPro3Item class tencentSpider(scrapy.Spider): name = 'tencent' allowed_domains = [] start_urls = [ 'http://tieba.baidu.com/mo/q----,sz@320_240-1-3---2/m?kw=%E6%A1%82%E6%9E%97%E7%94%B5%E5%AD%90%E7%A7%91%E6%8A%80%E5%A4%A7%E5%AD%A6%E5%8C%97%E6%B5%B7%E6%A0%A1%E5%8C%BA&pn=26140', ] def parse(self, response): # 总页面 item = ScrapyPro3Item() all_elements = response.xpath(".//div[@class='i']") # print(all_elements) for all_element in all_elements: content = all_element.xpath("./a/text()").extract_first() content = "".join(content.split()) change = re.compile(r'[\d]+.') content = change.sub('', content) item['comment'] = content person = all_element.xpath("./p/text()").extract_first() person = "".join(person.split()) # 去掉点赞数 评论数 change2 = re.compile(r'点[\d]+回[\d]+') person = change2.sub('', person) # 选择日期 change3 = re.compile(r'[\d]?[\d]?-[\d][\d](?=)') date = change3.findall(person) # 如果为今天则选择时间 change4 = re.compile(r'[\d]?[\d]?:[\d][\d](?=)') time = change4.findall(person) person = change3.sub('', person) person = change4.sub('', person) if time == []: item['time'] = date else: item['time'] = time item['name'] = person # 增加密码 活跃 item['is_active'] = '1' item['password'] = '123456' print(item) yield item # 下一页 """next_url = 'http://tieba.baidu.com/mo/q----,sz@320_240-1-3---2/' + parse.unquote( response.xpath(".//div[@class='bc p']/a/@href").extract_first()) print(next_url) yield scrapy.Request( next_url, callback=self.parse, )""" ``` (2)item文件 ``` # -*- coding: utf-8 -*- # Define here the models for your scraped items # # See documentation in: # https://doc.scrapy.org/en/latest/topics/items.html import scrapy class ScrapyPro3Item(scrapy.Item): # define the fields for your item here like: # name = scrapy.Field() comment = scrapy.Field() time = scrapy.Field() name = scrapy.Field() password = scrapy.Field() is_active = scrapy.Field() ``` (3)pipelines文件 # -*- coding: utf-8 -*- # Define your item pipelines here # # Don't forget to add your pipeline to the ITEM_PIPELINES setting # See: https://docs.scrapy.org/en/latest/topics/item-pipeline.html """class Scrapypro3Pipeline(object): def process_item(self, item, spider): return item""" import pymysql from twisted.enterprise import adbapi class Scrapypro3Pipeline(object): def __init__(self, dbpool): self.dbpool = dbpool @classmethod def from_settings(cls, settings): # 函数名固定,会被scrapy调用,直接可用settings的值 """ 数据库建立连接 :param settings: 配置参数 :return: 实例化参数 """ adbparams = dict( host='localhost', db='mu_ke', user='root', password='root', cursorclass=pymysql.cursors.DictCursor # 指定cursor类型 ) # 连接数据池ConnectionPool,使用pymysql或者Mysqldb连接 dbpool = adbapi.ConnectionPool('pymysql', **adbparams) # 返回实例化参数 return cls(dbpool) def process_item(self, item, spider): """ 使用twisted将MySQL插入变成异步执行。通过连接池执行具体的sql操作,返回一个对象 """ query = self.dbpool.runInteraction(self.do_insert, item) # 指定操作方法和操作数据 # 添加异常处理 query.addCallback(self.handle_error) # 处理异常 def do_insert(self, cursor, item): # 对数据库进行插入操作,并不需要commit,twisted会自动commit insert_sql = """ insert into login_person(name,password,is_active,comment,time) VALUES(%s,%s,%s,%s,%s) """ cursor.execute(insert_sql, (item['name'], item['password'], item['is_active'], item['comment'], item['time'])) def handle_error(self, failure): if failure: # 打印错误信息 print(failure)``` ``` (4) settings文件 ``` # -*- coding: utf-8 -*- # Scrapy settings for ScrapyPro3 project # # For simplicity, this file contains only settings considered important or # commonly used. You can find more settings consulting the documentation: # # https://doc.scrapy.org/en/latest/topics/settings.html # https://doc.scrapy.org/en/latest/topics/downloader-middleware.html # https://doc.scrapy.org/en/latest/topics/spider-middleware.html BOT_NAME = 'ScrapyPro3' SPIDER_MODULES = ['ScrapyPro3.spiders'] NEWSPIDER_MODULE = 'ScrapyPro3.spiders' # Crawl responsibly by identifying yourself (and your website) on the user-agent USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36' MYSQL_HOST = 'localhost' MYSQL_DBNAME = 'mu_ke' MYSQL_USER = 'root' MYSQL_PASSWD = 'root' # Obey robots.txt rules ROBOTSTXT_OBEY = False # Configure maximum concurrent requests performed by Scrapy (default: 16) #CONCURRENT_REQUESTS = 32 # Configure a delay for requests for the same website (default: 0) # See https://doc.scrapy.org/en/latest/topics/settings.html#download-delay # See also autothrottle settings and docs #DOWNLOAD_DELAY = 3 # The download delay setting will honor only one of: #CONCURRENT_REQUESTS_PER_DOMAIN = 16 #CONCURRENT_REQUESTS_PER_IP = 16 # Disable cookies (enabled by default) #COOKIES_ENABLED = False # Disable Telnet Console (enabled by default) #TELNETCONSOLE_ENABLED = False # Override the default request headers: #DEFAULT_REQUEST_HEADERS = { # 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', # 'Accept-Language': 'en', #} # Enable or disable spider middlewares # See https://doc.scrapy.org/en/latest/topics/spider-middleware.html #SPIDER_MIDDLEWARES = { # 'ScrapyPro3.middlewares.ScrapyPro3SpiderMiddleware': 543, #} # Enable or disable downloader middlewares # See https://doc.scrapy.org/en/latest/topics/downloader-middleware.html #DOWNLOADER_MIDDLEWARES = { # 'ScrapyPro3.middlewares.ScrapyPro3DownloaderMiddleware': 543, #} # Enable or disable extensions # See https://doc.scrapy.org/en/latest/topics/extensions.html #EXTENSIONS = { # 'scrapy.extensions.telnet.TelnetConsole': None, #} # Configure item pipelines # See https://doc.scrapy.org/en/latest/topics/item-pipeline.html ITEM_PIPELINES = { 'ScrapyPro3.pipelines.Scrapypro3Pipeline':200, } # Enable and configure the AutoThrottle extension (disabled by default) # See https://doc.scrapy.org/en/latest/topics/autothrottle.html #AUTOTHROTTLE_ENABLED = True # The initial download delay #AUTOTHROTTLE_START_DELAY = 5 # The maximum download delay to be set in case of high latencies #AUTOTHROTTLE_MAX_DELAY = 60 # The average number of requests Scrapy should be sending in parallel to # each remote server #AUTOTHROTTLE_TARGET_CONCURRENCY = 1.0 # Enable showing throttling stats for every response received: #AUTOTHROTTLE_DEBUG = False # Enable and configure HTTP caching (disabled by default) # See https://doc.scrapy.org/en/latest/topics/downloader-middleware.html#httpcache-middleware-settings #HTTPCACHE_ENABLED = True #HTTPCACHE_EXPIRATION_SECS = 0 #HTTPCACHE_DIR = 'httpcache' #HTTPCACHE_IGNORE_HTTP_CODES = [] #HTTPCACHE_STORAGE = 'scrapy.extensions.httpcache.FilesystemCacheStorage' ``` (5)start文件——执行爬虫文件 ``` from scrapy import cmdline cmdline.execute(["scrapy","crawl","tencent"]) ``` 数据库创建代码: ``` create database mu_ke; CREATE TABLE `login_person` ( `id` int(10) NOT NULL AUTO_INCREMENT, `name` varchar(100) DEFAULT NULL, `passsword` varchar(100) DEFAULT NULL, `is_active` varchar(100) DEFAULT NULL, `comment` varchar(100) DEFAULT NULL, `time` varchar(100) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=1181 DEFAULT CHARSET=utf8; select count(name) from login_person;#查询结果条数为0 ``` # 运行完代码后查询数据,显示条数为0,这里面有什么问题吗? (1) 执行过程正常 (2)运行 pycharm2019.3 python3.8 mysql8.0(workbench8.0) (3) 数据连接没有
html2pdf导出,windows下正常,linux下乱吗
本来直接用的jsoup,换了linux后乱码了,最后发现linux下读取个文件都乱码 linux下网页内容字节流保存本地xml文件正常没有乱码,然后读取文件就乱码了, 各位大神这啥原因啊,代码里编码都对应的,windows下都正常的,换linux就乱码了 public String convert2PDF() { PdfContentByte content = null; BaseFont base = null; Rectangle pageRect = null; String pdfPath = context .getRealPath("/pdfIn/" + (new SimpleDateFormat("yyyyMMddHHmmssSSS") .format(new Date()) + ".pdf")); String outPath = context .getRealPath("/pdfOut/" + (new SimpleDateFormat("yyyyMMddHHmmssSSS") .format(new Date()) + ".pdf")); String fontPath = context.getRealPath("/font/msyh.ttf"); String contextPath = context.getContextPath(); // FileOutputStream fos; InputStream is; try { jsp = jsp == null ? "" : jsp; // URL url = new URL(jsp); byte bytes[] = new byte[1024 * 1000]; String tmpXml = context.getRealPath("/tmpXml/" + (new SimpleDateFormat("yyyyMMddHHmmssSSS") .format(new Date()) + ".html")); File xml = new File(tmpXml); if (!xml.getParentFile().exists()) xml.getParentFile().mkdirs(); if (!xml.exists()) xml.createNewFile(); int index = 0; is = url.openStream(); int count = is.read(bytes, index, 1024 * 100); while (count != -1) { index += count; count = is.read(bytes, index, 1); } fos = new FileOutputStream(xml); System.out.println(index); fos.write(bytes, 0, index); // is.close(); fos.close(); FileInputStream fis = new FileInputStream(xml); InputStreamReader isr = new InputStreamReader(fis, "utf-8"); BufferedReader br = new BufferedReader(isr); StringBuffer sb = new StringBuffer(); String line = ""; while ((line = br.readLine()) != null) { sb.append(line); } br.close(); System.err.println(sb.toString()); //TODO 读取本地文件乱码问题 org.jsoup.nodes.Document doc1 = Jsoup.parse(sb.toString()); // org.jsoup.nodes.Document doc2 = Jsoup.parse(xml, "GBK"); System.out.println(doc1.toString()); // System.out.println(doc2.toString()); File tmp = new File(pdfPath); if (!tmp.getParentFile().exists()) tmp.getParentFile().mkdirs(); // System.out.println("-- created -in===" + tmp.getPath()); Document document = new Document(); PdfWriter writer = PdfWriter.getInstance(document, new FileOutputStream(tmp)); document.open(); // Connection conn = Jsoup.connect(jsp); // conn.header( // "User-Agent", // "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"); // org.jsoup.nodes.Document doc = conn.timeout(5000).get(); // doc1.select("div#getpdf").remove(); InputStream in = new ByteArrayInputStream(doc1.toString().getBytes( "utf-8")); // System.out // .println("-- FileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStream"); XMLWorkerHelper.getInstance().parseXHtml(writer, document, in, Charset.forName("utf-8")); // System.out // .println("-- FileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStreamFileInputStream"); document.close(); File out = new File(outPath); if (!out.getParentFile().exists()) out.getParentFile().mkdirs(); if (!out.exists()) out.createNewFile(); System.out.println("-- created -out===" + out.getPath()); PdfReader pdfReader = new PdfReader(tmp.getPath()); PdfStamper pdfStamper = new PdfStamper(pdfReader, new FileOutputStream(out)); // PdfGState gs = new PdfGState(); base = BaseFont.createFont("STSong-Light", "UniGB-UCS2-H", BaseFont.NOT_EMBEDDED); // base = BaseFont.createFont(fontPath, BaseFont.IDENTITY_H, // BaseFont.NOT_EMBEDDED); System.out.println("-- -fontPath===" + fontPath); if (base == null || pdfStamper == null) { msg = "文件生成失败!"; ActionContext.getContext().put("msg", msg); path = "error"; } // 设置透明度为0.4 gs.setFillOpacity(0.4f); gs.setStrokeOpacity(0.4f); int toPage = pdfStamper.getReader().getNumberOfPages(); for (int i = 1; i <= toPage; i++) { pageRect = pdfStamper.getReader().getPageSizeWithRotation(i); // 计算水印X,Y坐标 float x = pageRect.getWidth() / 2; float y = pageRect.getHeight() / 2; // 获得PDF最顶层 content = pdfStamper.getOverContent(i); content.saveState(); // set Transparency content.setGState(gs); content.beginText(); content.setColorFill(BaseColor.GRAY); content.setFontAndSize(base, 60); // 水印文字成45度角倾斜 content.showTextAligned(Element.ALIGN_CENTER, "eeeee", x, y, 45); content.endText(); } // pdfStamper.close(); // tmp.delete(); // path = jsp.split(contextPath)[0] + contextPath+"/"+ // out.getPath().replace("\\", // "/").split(contextPath)[1].split("/")[1]+"/"+out.getPath().replace("\\", // "/").split(contextPath)[1].split("/")[2]; path = out.getPath().replace("\\", "/").split("pdfOut")[0] + "pdfOut/$" + out.getPath().replace("\\", "/").split("pdfOut")[1] .split("/")[1]; System.out.println("-- created -pdf path===" + path); } catch (Exception ex) { ex.printStackTrace(); msg = "文件生成异常!"; ActionContext.getContext().put("msg", msg); path = "error"; } finally { content = null; base = null; pageRect = null; } return SUCCESS; }
ictclas4j分词工具搭建在hadoop上出现了很多windows底下没有出现过的错误
本人将在windows上调试好的分词工具包移到bantu底下的eclipse上,运行时出现了Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.ArrayList.elementData(ArrayList.java:418) at java.util.ArrayList.get(ArrayList.java:431) at org.ictclas4j.bean.Dictionary.findInModifyTable(Dictionary.java:464) at org.ictclas4j.bean.Dictionary.getHandle(Dictionary.java:386) at org.ictclas4j.segment.PosTagger.posTag(PosTagger.java:149) at org.ictclas4j.segment.PosTagger.recognition(PosTagger.java:73) at org.ictclas4j.segment.SegTag.split(SegTag.java:90) at test.Main.main(Main.java:23)
windows环境下,使用java解压tar.gz包出现中文乱码
public void unzipOarFile(String outputDirectory) { FileInputStream fis = null; ArchiveInputStream in = null; BufferedInputStream bufferedInputStream = null; try { fis = new FileInputStream(zipfileName); GZIPInputStream is = new GZIPInputStream(new BufferedInputStream(fis)); in = new ArchiveStreamFactory().createArchiveInputStream("tar", is); bufferedInputStream = new BufferedInputStream(in); TarArchiveEntry entry = (TarArchiveEntry) in.getNextEntry(); while (entry != null) { String name = entry.getName(); String[] names = name.split("/"); String fileName = outputDirectory; for (int i = 0; i < names.length; i++) { String str = names[i]; fileName = fileName + File.separator + str; } if (name.endsWith("/")) { mkFolder(fileName); } else { File file = mkFile(fileName); bufferedOutputStream = new BufferedOutputStream(new FileOutputStream(file)); int b; while ((b = bufferedInputStream.read()) != -1) { bufferedOutputStream.write(b); } bufferedOutputStream.flush(); bufferedOutputStream.close(); } entry = (TarArchiveEntry) in.getNextEntry(); } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } catch (ArchiveException e) { e.printStackTrace(); } finally { try { if (bufferedInputStream != null) { bufferedInputStream.close(); } } catch (IOException e) { e.printStackTrace(); } } }
从windows导入mac的android项目,下载到真机时出现空指针错误
![图片说明](https://img-ask.csdn.net/upload/201706/08/1496905838_954210.jpg) 导入的项目错误F:\Android\AndroidStudioProjects\SCar\app\build\intermediates\split-apk\debug\slices\slice_3.apk 这个文件我有: ![图片说明](https://img-ask.csdn.net/upload/201706/08/1496906349_455771.jpg) 但是不知道这个去哪里设置 求师傅帮忙!谢谢了!
模型分别在mac和windows服务器上跑,准确率相差60%多!
这是一个用cnn做文本分类的一个模型,我在自己的mac上跑准确率有90%,但是放到windows服务器上准确率竟然只有25%,不知道是什么原因? from __future__ import print_function import numpy as np from keras.utils import np_utils from keras.preprocessing.text import Tokenizer from keras.preprocessing.sequence import pad_sequences import pandas as pd import os from keras import backend as K print('Loading Dict') embeddings_index = {} f = open(os.path.join( 'glove.6B.100d.txt')) for line in f: values = line.split() word = values[0] coefs = np.asarray(values[1:], dtype='float32') embeddings_index[word] = coefs f.close() print('Loading dataset') tmp=pd.read_csv('train.csv') train_X=np.array(tmp.iloc[:,2]).astype('str') train_y=np.array(tmp.iloc[:,0]).astype('int16') train_y_ohe = np_utils.to_categorical(train_y) del tmp tmp=pd.read_csv('test.csv') test_X=np.array(tmp.iloc[:,2]).astype('str') test_y=np.array(tmp.iloc[:,0]).astype('int16') test_y_ohe = np_utils.to_categorical(test_y) del tmp train_y_ohe=train_y_ohe.astype('float32') test_y_ohe=test_y_ohe.astype('float32') X=np.append(train_X,test_X) print('Tokening') t = Tokenizer() t.fit_on_texts(X) vocab_size = len(t.word_index) + 1 # integer encode the documents encoded_X = t.texts_to_sequences(X) # pad documents to a max length of x words max_length = 50 padded_X = pad_sequences(encoded_X, maxlen=max_length, padding='post') embedding_matrix = np.zeros((vocab_size, 100)).astype('float32') for word, i in t.word_index.items(): embedding_vector = embeddings_index.get(word) if embedding_vector is not None: embedding_matrix[i] = embedding_vector padded_X_train=pad_sequences(encoded_X[0:119999],maxlen=max_length, padding='post') padded_X_test=pad_sequences(encoded_X[119999:127598],maxlen=max_length, padding='post') padded_X_test=padded_X_test.astype('float32') padded_X_train=padded_X_train.astype('float32') print('Estabilish model') from keras.models import Model from keras.layers import Dense,Embedding,Convolution1D,concatenate,Flatten,Input,MaxPooling1D,Dropout,Merge from keras.callbacks import TensorBoard K.clear_session() x=Input(shape=(50,),dtype='float32') embed=Embedding(input_dim=vocab_size,output_dim=100,weights=[embedding_matrix],input_length=max_length)(x) cnn1=Convolution1D(128,9,activation='relu',padding='same',strides=1)(embed) cnn1=MaxPooling1D(5)(cnn1) cnn2=Convolution1D(128,6,activation='relu',padding='same',strides=1)(embed) cnn2=MaxPooling1D(5)(cnn2) cnn3=Convolution1D(128,3,activation='relu',padding='same',strides=1)(embed) cnn3=MaxPooling1D(5)(cnn3) cnn=concatenate([cnn1,cnn2,cnn3]) flat=Flatten()(cnn) drop=Dropout(0.1)(flat) y=Dense(5,activation='softmax')(drop) model=Model(inputs=x,outputs=y) model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) tensorboard=TensorBoard(log_dir='./logs',write_graph=True,write_grads=True,histogram_freq=True) model.fit(padded_X_train, train_y_ohe, epochs=5, batch_size=10000, verbose=1,callbacks=[tensorboard],validation_data=[padded_X_test,test_y_ohe]) '''pred0=model.predict_classes(padded_X,verbose=0) acc_train=np.sum(train_y==pred0,axis=0)/train_X.shape[0]'''
采用scrapy框架爬取二手房数据,显示没有爬取到页面和项目,不清楚问题原因
1.item ``` import scrapy class LianjiaItem(scrapy.Item): # define the fields for your item here like: # 房屋名称 name = scrapy.Field() # 房屋户型 type = scrapy.Field() # 建筑面积 area = scrapy.Field() # 房屋朝向 direction = scrapy.Field() # 装修情况 fitment = scrapy.Field() # 有无电梯 elevator = scrapy.Field() # 房屋总价 total_price = scrapy.Field() # 房屋单价 unit_price = scrapy.Field() # 房屋产权 property = scrapy.Field() ``` 2.settings ``` BOT_NAME = 'lianjia' SPIDER_MODULES = ['lianjia.spiders'] NEWSPIDER_MODULE = 'lianjia.spiders' USER_AGENT = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; AcooBrowser; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" ROBOTSTXT_OBEY = False ITEM_PIPELINES = { 'lianjia.pipelines.FilterPipeline': 100, 'lianjia.pipelines.CSVPipeline': 200, } ``` 3.pipelines ``` import re from scrapy.exceptions import DropItem class FilterPipeline(object): def process_item(self,item,spider): item['area'] = re.findall(r"\d+\.?\d*",item["area"])[0] if item["direction"] == '暂无数据': raise DropItem("房屋朝向无数据,抛弃此项目:%s"%item) return item class CSVPipeline(object): index = 0 file = None def open_spider(self,spider): self.file = open("home.csv","a") def process_item(self, item, spider): if self.index == 0: column_name = "name,type,area,direction,fitment,elevator,total_price,unit_price,property\n" self.file.write(column_name) self.index = 1 home_str = item['name']+","+item['type']+","+item['area']+","+item['direction']+","+item['fitment']+","+item['elevator']+","+item['total_price']+","+item['unit_price']+","+item['property']+"\n" self.file.write(home_str) return item def close_spider(self,spider): self.file.close() ``` 4.lianjia_spider ``` import scrapy from scrapy import Request from lianjia.items import LianjiaItem class LianjiaSpiderSpider(scrapy.Spider): name = 'lianjia_spider' # 获取初始请求 def start_requests(self): # 生成请求对象 url = 'https://bj.lianjia.com/ershoufang/' yield Request(url) # 实现主页面解析函数 def parse(self, response): # 使用xpath定位到二手房信息的div元素,保存到列表中 list_selector = response.xpath("//li/div[@class = 'info clear']") # 依次遍历每个选择器,获取二手房的名称,户型,面积,朝向等信息 for one_selector in list_selector: try: name = one_selector.xpath("div[@class = 'title']/a/text()").extract_first() other = one_selector.xpath("div[@class = 'address']/div[@class = 'houseInfo']/text()").extract_first() other_list = other.split("|") type = other_list[0].strip(" ") area = other_list[1].strip(" ") direction = other_list[2].strip(" ") fitment = other_list[3].strip(" ") total_price = one_selector.xpath("//div[@class = 'totalPrice']/span/text()").extract_first() unit_price = one_selector.xpath("//div[@class = 'unitPrice']/@data-price").extract_first() url = one_selector.xpath("div[@class = 'title']/a/@href").extract_first() yield Request(url,meta={"name":name,"type":type,"area":area,"direction":direction,"fitment":fitment,"total_price":total_price,"unit_price":unit_price},callback=self.otherinformation) except: pass current_page = response.xpath("//div[@class = 'page-box house-lst-page-box']/@page-data").extract_first().split(',')[1].split(':')[1] current_page = current_page.replace("}", "") current_page = int(current_page) if current_page < 100: current_page += 1 next_url = "https://bj.lianjia.com/ershoufang/pg%d/" %(current_page) yield Request(next_url,callback=self.otherinformation) def otherinformation(self,response): elevator = response.xpath("//div[@class = 'base']/div[@class = 'content']/ul/li[12]/text()").extract_first() property = response.xpath("//div[@class = 'transaction']/div[@class = 'content']/ul/li[5]/span[2]/text()").extract_first() item = LianjiaItem() item["name"] = response.meta['name'] item["type"] = response.meta['type'] item["area"] = response.meta['area'] item["direction"] = response.meta['direction'] item["fitment"] = response.meta['fitment'] item["total_price"] = response.meta['total_price'] item["unit_price"] = response.meta['unit_price'] item["property"] = property item["elevator"] = elevator yield item ``` 提示错误: ``` de - interpreting them as being unequal if item["direction"] == '鏆傛棤鏁版嵁': 2019-11-25 10:53:35 [scrapy.core.scraper] ERROR: Error processing {'area': u'75.6', 'direction': u'\u897f\u5357', 'elevator': u'\u6709', 'fitment': u'\u7b80\u88c5', 'name': u'\u6b64\u6237\u578b\u517113\u5957 \u89c6\u91ce\u91c7\u5149\u597d \u65e0\u786c\u4f24 \u4e1a\u4e3b\u8bda\u610f\u51fa\u552e', 'property': u'\u6ee1\u4e94\u5e74', 'total_price': None, 'type': u'2\u5ba41\u5385', 'unit_price': None} Traceback (most recent call last): File "f:\python_3.6\venv\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks current.result = callback(current.result, *args, **kw) File "F:\python_3.6\lianjia\lianjia\pipelines.py", line 25, in process_item home_str = item['name']+","+item['type']+","+item['area']+","+item['direction']+","+item['fitment']+","+item['elevator']+","+item['total_price']+","+item['unit_price']+ ","+item['property']+"\n" TypeError: coercing to Unicode: need string or buffer, NoneType found ```
com.jacob包 在Windows10开发环境运行正常,在Windows server 2008 抛出异常
com.jacob.com.ComFailException: Invoke of: AudioOutputStream Source: Description: at com.jacob.com.Dispatch.invokev(Native Method) at com.jacob.com.Dispatch.invokev(Dispatch.java:625) at com.jacob.com.Dispatch.invoke(Dispatch.java:498) at com.jacob.com.Dispatch.putRef(Dispatch.java:819) at net.bjnblh.dc.textToSpeech.service.MSTTSSpeech.saveToWav(MSTTSSpeech.java:274) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl.makeWavAndReturnUrl(NoteReadingServiceImpl.java:70) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl.docToHtml(NoteReadingServiceImpl.java:53) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl$$FastClassBySpringCGLIB$$9c509776.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at com.alibaba.druid.support.spring.stat.DruidStatInterceptor.invoke(DruidStatInterceptor.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at net.bjnblh.dc.textToSpeech.service.impl.NoteReadingServiceImpl$$EnhancerBySpringCGLIB$$62d6289f.docToHtml(<generated>) at net.bjnblh.dc.textToSpeech.controller.noteReadingController.getHtmlByWord(noteReadingController.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:832) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:743) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:961) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:895) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:869) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at net.bjnblh.dc.core.filter.CrossFilter.doFilter(CrossFilter.java:37) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1533) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1489) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) ``` public void saveToWav(String text, String filePath) { // 创建输出文件流对象 ax=new ActiveXComponent("Sapi.SpFileStream"); spFileStream=ax.getObject(); // 创建音频流格式对象 if(spAudioFormat==null) { ax=new ActiveXComponent("Sapi.SpAudioFormat"); spAudioFormat=ax.getObject(); } // 设置音频流格式类型 Dispatch.put(spAudioFormat,"Type",new Variant(this.formatType)); // 设置文件输出流的格式 Dispatch.putRef(spFileStream,"Format",spAudioFormat); // 调用输出文件流对象的打开方法,创建一个.wav文件 Dispatch.call(spFileStream,"Open",new Variant(filePath),new Variant(3),new Variant(true)); // 设置声音对象的音频输出流为输出文件流对象 Dispatch.putRef(spVoice,"AudioOutputStream",spFileStream); // 调整音量和读的速度 Dispatch.put(spVoice,"Volume",new Variant(this.volume));// 设置音量 Dispatch.put(spVoice,"Rate",new Variant(this.rate));// 设置速率 // 开始朗读 Dispatch.call(spVoice,"Speak",new Variant(text)); /* 分一句话去读 Long initTime =System.currentTimeMillis(); for (Element element:elements) { String text =element.childNode(0).toString(); String[] strings= text.split("\\p{P}"); String newNodeHtml =""; for (int n=0 ; n<strings.length;n++) { String start=String.valueOf(System.currentTimeMillis() - initTime); System.out.println(start); Dispatch.call(spVoice,"Speak",new Variant(strings[n])); newNodeHtml += "<span id=" +start+ ">" + strings[n] +"</span>"; } listValue.add(newNodeHtml); }*/ // 关闭输出文件流对象,释放资源 Dispatch.call(spFileStream,"Close"); Dispatch.putRef(spVoice,"AudioOutputStream",null);//此处异常 } ```
爬取网页只能读取到“请刷新页面或按键盘F5”
以下是源码 ``` import urllib from urllib import request import re import random url = "http://x77558.net/bbs/thread.php?fid=6" user_agent = [ "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11", "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Maxthon 2.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; TencentTraveler 4.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; The World)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X MetaSr 1.0; SE 2.X MetaSr 1.0; .NET CLR 2.0.50727; SE 2.X MetaSr 1.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Avant Browser)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)", "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5", "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5", "Mozilla/5.0 (iPad; U; CPU OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5", "Mozilla/5.0 (Linux; U; Android 2.3.7; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1", "MQQBrowser/26 Mozilla/5.0 (Linux; U; Android 2.3.7; zh-cn; MB200 Build/GRJ22; CyanogenMod-7) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1", "Opera/9.80 (Android 2.3.4; Linux; Opera Mobi/build-1107180945; U; en-GB) Presto/2.8.149 Version/11.10", "Mozilla/5.0 (Linux; U; Android 3.0; en-us; Xoom Build/HRI39) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13", "Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.337 Mobile Safari/534.1+", "Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.70 Safari/534.6 TouchPad/1.0", "Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124", "Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0; HTC; Titan)", "UCWEB7.0.2.37/28/999", "NOKIA5700/ UCWEB7.0.2.37/28/999", "Openwave/ UCWEB7.0.2.37/28/999", "Mozilla/4.0 (compatible; MSIE 6.0; ) Opera/UCWEB7.0.2.37/28/999", # iPhone 6: "Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25", ] # read the url and return a list named page_data def read_url(url,page_data,headers): req = urllib.request.Request(url, headers=headers) for i in range(3): web_data = urllib.request.urlopen(req).read() web_data = web_data.decode("gbk",errors = 'ignore')# the second parament can solver the problem that in # error decode page_data.append(str(web_data)) return page_data # find taget in the page , used re , an return a list def find_tag(tagstr,idx,data,lists): lists.append(re.findall(tagstr,data[idx])) return lists # read the list to download the photo which type is jpg def download_jpg(lists,path): for lis in lists: for l in lis: print(l) name = l.split("/")[-1] print(name) if ".jpg" or ".png" in l: if "js" in l: continue elif "http" in l: # sometimes met a missing name 403 , the solve is in the another file named download.py urllib.request.urlretrieve(l,path+name) else: continue tagstr = '<a title="开放主题" href="(.*?)"' page_data = [] img_url_list = [] url_lsit = [] img_list = [] while len(page_data)==0 or page_data[-1]=="请刷新页面或按键盘F5": headers = {'User-Agent': random.choice(user_agent)} read_url(url,page_data,headers) print(page_data[-1]) ```
yolo3 darknet.py问题
我用darknetAB https://github.com/AlexeyAB/darknet 编译gpu版本后生成darknet.py文件 然后我也编译了yolo_cpp_dll.sln文件 生成dll文件 然后运行darknet.py文件 不显示图片 异常退出 ![图片说明](https://img-ask.csdn.net/upload/201911/02/1572688446_628910.png) 百度了这个问题 有人说要换python3.5版本 我也尝试了 但是也是不行 不会显示图片。请问各位大佬到底怎么解决??急!!!谢谢!!! ``` #!python3 """ Python 3 wrapper for identifying objects in images Requires DLL compilation Both the GPU and no-GPU version should be compiled; the no-GPU version should be renamed "yolo_cpp_dll_nogpu.dll". On a GPU system, you can force CPU evaluation by any of: - Set global variable DARKNET_FORCE_CPU to True - Set environment variable CUDA_VISIBLE_DEVICES to -1 - Set environment variable "FORCE_CPU" to "true" To use, either run performDetect() after import, or modify the end of this file. See the docstring of performDetect() for parameters. Directly viewing or returning bounding-boxed images requires scikit-image to be installed (`pip install scikit-image`) Original *nix 2.7: https://github.com/pjreddie/darknet/blob/0f110834f4e18b30d5f101bf8f1724c34b7b83db/python/darknet.py Windows Python 2.7 version: https://github.com/AlexeyAB/darknet/blob/fc496d52bf22a0bb257300d3c79be9cd80e722cb/build/darknet/x64/darknet.py @author: Philip Kahn @date: 20180503 """ #pylint: disable=R, W0401, W0614, W0703 from ctypes import * import math import random import os def sample(probs): s = sum(probs) probs = [a/s for a in probs] r = random.uniform(0, 1) for i in range(len(probs)): r = r - probs[i] if r <= 0: return i return len(probs)-1 def c_array(ctype, values): arr = (ctype*len(values))() arr[:] = values return arr class BOX(Structure): _fields_ = [("x", c_float), ("y", c_float), ("w", c_float), ("h", c_float)] class DETECTION(Structure): _fields_ = [("bbox", BOX), ("classes", c_int), ("prob", POINTER(c_float)), ("mask", POINTER(c_float)), ("objectness", c_float), ("sort_class", c_int)] class IMAGE(Structure): _fields_ = [("w", c_int), ("h", c_int), ("c", c_int), ("data", POINTER(c_float))] class METADATA(Structure): _fields_ = [("classes", c_int), ("names", POINTER(c_char_p))] #lib = CDLL("/home/pjreddie/documents/darknet/libdarknet.so", RTLD_GLOBAL) #lib = CDLL("libdarknet.so", RTLD_GLOBAL) hasGPU = True if os.name == "nt": cwd = os.path.dirname(__file__) os.environ['PATH'] = cwd + ';' + os.environ['PATH'] winGPUdll = os.path.join(cwd, "yolo_cpp_dll.dll") winNoGPUdll = os.path.join(cwd, "yolo_cpp_dll_nogpu.dll") envKeys = list() for k, v in os.environ.items(): envKeys.append(k) try: try: tmp = os.environ["FORCE_CPU"].lower() if tmp in ["1", "true", "yes", "on"]: raise ValueError("ForceCPU") else: print("Flag value '"+tmp+"' not forcing CPU mode") except KeyError: # We never set the flag if 'CUDA_VISIBLE_DEVICES' in envKeys: if int(os.environ['CUDA_VISIBLE_DEVICES']) < 0: raise ValueError("ForceCPU") try: global DARKNET_FORCE_CPU if DARKNET_FORCE_CPU: raise ValueError("ForceCPU") except NameError: pass # print(os.environ.keys()) # print("FORCE_CPU flag undefined, proceeding with GPU") if not os.path.exists(winGPUdll): raise ValueError("NoDLL") lib = CDLL(winGPUdll, RTLD_GLOBAL) except (KeyError, ValueError): hasGPU = False if os.path.exists(winNoGPUdll): lib = CDLL(winNoGPUdll, RTLD_GLOBAL) print("Notice: CPU-only mode") else: # Try the other way, in case no_gpu was # compile but not renamed lib = CDLL(winGPUdll, RTLD_GLOBAL) print("Environment variables indicated a CPU run, but we didn't find `"+winNoGPUdll+"`. Trying a GPU run anyway.") else: lib = CDLL("./libdarknet.so", RTLD_GLOBAL) lib.network_width.argtypes = [c_void_p] lib.network_width.restype = c_int lib.network_height.argtypes = [c_void_p] lib.network_height.restype = c_int copy_image_from_bytes = lib.copy_image_from_bytes copy_image_from_bytes.argtypes = [IMAGE,c_char_p] def network_width(net): return lib.network_width(net) def network_height(net): return lib.network_height(net) predict = lib.network_predict_ptr predict.argtypes = [c_void_p, POINTER(c_float)] predict.restype = POINTER(c_float) if hasGPU: set_gpu = lib.cuda_set_device set_gpu.argtypes = [c_int] make_image = lib.make_image make_image.argtypes = [c_int, c_int, c_int] make_image.restype = IMAGE get_network_boxes = lib.get_network_boxes get_network_boxes.argtypes = [c_void_p, c_int, c_int, c_float, c_float, POINTER(c_int), c_int, POINTER(c_int), c_int] get_network_boxes.restype = POINTER(DETECTION) make_network_boxes = lib.make_network_boxes make_network_boxes.argtypes = [c_void_p] make_network_boxes.restype = POINTER(DETECTION) free_detections = lib.free_detections free_detections.argtypes = [POINTER(DETECTION), c_int] free_ptrs = lib.free_ptrs free_ptrs.argtypes = [POINTER(c_void_p), c_int] network_predict = lib.network_predict_ptr network_predict.argtypes = [c_void_p, POINTER(c_float)] reset_rnn = lib.reset_rnn reset_rnn.argtypes = [c_void_p] load_net = lib.load_network load_net.argtypes = [c_char_p, c_char_p, c_int] load_net.restype = c_void_p load_net_custom = lib.load_network_custom load_net_custom.argtypes = [c_char_p, c_char_p, c_int, c_int] load_net_custom.restype = c_void_p do_nms_obj = lib.do_nms_obj do_nms_obj.argtypes = [POINTER(DETECTION), c_int, c_int, c_float] do_nms_sort = lib.do_nms_sort do_nms_sort.argtypes = [POINTER(DETECTION), c_int, c_int, c_float] free_image = lib.free_image free_image.argtypes = [IMAGE] letterbox_image = lib.letterbox_image letterbox_image.argtypes = [IMAGE, c_int, c_int] letterbox_image.restype = IMAGE load_meta = lib.get_metadata lib.get_metadata.argtypes = [c_char_p] lib.get_metadata.restype = METADATA load_image = lib.load_image_color load_image.argtypes = [c_char_p, c_int, c_int] load_image.restype = IMAGE rgbgr_image = lib.rgbgr_image rgbgr_image.argtypes = [IMAGE] predict_image = lib.network_predict_image predict_image.argtypes = [c_void_p, IMAGE] predict_image.restype = POINTER(c_float) predict_image_letterbox = lib.network_predict_image_letterbox predict_image_letterbox.argtypes = [c_void_p, IMAGE] predict_image_letterbox.restype = POINTER(c_float) def array_to_image(arr): import numpy as np # need to return old values to avoid python freeing memory arr = arr.transpose(2,0,1) c = arr.shape[0] h = arr.shape[1] w = arr.shape[2] arr = np.ascontiguousarray(arr.flat, dtype=np.float32) / 255.0 data = arr.ctypes.data_as(POINTER(c_float)) im = IMAGE(w,h,c,data) return im, arr def classify(net, meta, im): out = predict_image(net, im) res = [] for i in range(meta.classes): if altNames is None: nameTag = meta.names[i] else: nameTag = altNames[i] res.append((nameTag, out[i])) res = sorted(res, key=lambda x: -x[1]) return res def detect(net, meta, image, thresh=.5, hier_thresh=.5, nms=.45, debug= False): """ Performs the meat of the detection """ #pylint: disable= C0321 im = load_image(image, 0, 0) if debug: print("Loaded image") ret = detect_image(net, meta, im, thresh, hier_thresh, nms, debug) free_image(im) if debug: print("freed image") return ret def detect_image(net, meta, im, thresh=.5, hier_thresh=.5, nms=.45, debug= False): #import cv2 #custom_image_bgr = cv2.imread(image) # use: detect(,,imagePath,) #custom_image = cv2.cvtColor(custom_image_bgr, cv2.COLOR_BGR2RGB) #custom_image = cv2.resize(custom_image,(lib.network_width(net), lib.network_height(net)), interpolation = cv2.INTER_LINEAR) #import scipy.misc #custom_image = scipy.misc.imread(image) #im, arr = array_to_image(custom_image) # you should comment line below: free_image(im) num = c_int(0) if debug: print("Assigned num") pnum = pointer(num) if debug: print("Assigned pnum") predict_image(net, im) letter_box = 0 #predict_image_letterbox(net, im) #letter_box = 1 if debug: print("did prediction") # dets = get_network_boxes(net, custom_image_bgr.shape[1], custom_image_bgr.shape[0], thresh, hier_thresh, None, 0, pnum, letter_box) # OpenCV dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, None, 0, pnum, letter_box) if debug: print("Got dets") num = pnum[0] if debug: print("got zeroth index of pnum") if nms: do_nms_sort(dets, num, meta.classes, nms) if debug: print("did sort") res = [] if debug: print("about to range") for j in range(num): if debug: print("Ranging on "+str(j)+" of "+str(num)) if debug: print("Classes: "+str(meta), meta.classes, meta.names) for i in range(meta.classes): if debug: print("Class-ranging on "+str(i)+" of "+str(meta.classes)+"= "+str(dets[j].prob[i])) if dets[j].prob[i] > 0: b = dets[j].bbox if altNames is None: nameTag = meta.names[i] else: nameTag = altNames[i] if debug: print("Got bbox", b) print(nameTag) print(dets[j].prob[i]) print((b.x, b.y, b.w, b.h)) res.append((nameTag, dets[j].prob[i], (b.x, b.y, b.w, b.h))) if debug: print("did range") res = sorted(res, key=lambda x: -x[1]) if debug: print("did sort") free_detections(dets, num) if debug: print("freed detections") return res netMain = None metaMain = None altNames = None def performDetect(imagePath="data/dog.jpg", thresh= 0.25, configPath = "./cfg/yolov3.cfg", weightPath = "yolov3.weights", metaPath= "./cfg/coco.data", showImage= True, makeImageOnly = False, initOnly= False): """ Convenience function to handle the detection and returns of objects. Displaying bounding boxes requires libraries scikit-image and numpy Parameters ---------------- imagePath: str Path to the image to evaluate. Raises ValueError if not found thresh: float (default= 0.25) The detection threshold configPath: str Path to the configuration file. Raises ValueError if not found weightPath: str Path to the weights file. Raises ValueError if not found metaPath: str Path to the data file. Raises ValueError if not found showImage: bool (default= True) Compute (and show) bounding boxes. Changes return. makeImageOnly: bool (default= False) If showImage is True, this won't actually *show* the image, but will create the array and return it. initOnly: bool (default= False) Only initialize globals. Don't actually run a prediction. Returns ---------------------- When showImage is False, list of tuples like ('obj_label', confidence, (bounding_box_x_px, bounding_box_y_px, bounding_box_width_px, bounding_box_height_px)) The X and Y coordinates are from the center of the bounding box. Subtract half the width or height to get the lower corner. Otherwise, a dict with { "detections": as above "image": a numpy array representing an image, compatible with scikit-image "caption": an image caption } """ # Import the global variables. This lets us instance Darknet once, then just call performDetect() again without instancing again global metaMain, netMain, altNames #pylint: disable=W0603 assert 0 < thresh < 1, "Threshold should be a float between zero and one (non-inclusive)" if not os.path.exists(configPath): raise ValueError("Invalid config path `"+os.path.abspath(configPath)+"`") if not os.path.exists(weightPath): raise ValueError("Invalid weight path `"+os.path.abspath(weightPath)+"`") if not os.path.exists(metaPath): raise ValueError("Invalid data file path `"+os.path.abspath(metaPath)+"`") if netMain is None: netMain = load_net_custom(configPath.encode("ascii"), weightPath.encode("ascii"), 0, 1) # batch size = 1 if metaMain is None: metaMain = load_meta(metaPath.encode("ascii")) if altNames is None: # In Python 3, the metafile default access craps out on Windows (but not Linux) # Read the names file and create a list to feed to detect try: with open(metaPath) as metaFH: metaContents = metaFH.read() import re match = re.search("names *= *(.*)$", metaContents, re.IGNORECASE | re.MULTILINE) if match: result = match.group(1) else: result = None try: if os.path.exists(result): with open(result) as namesFH: namesList = namesFH.read().strip().split("\n") altNames = [x.strip() for x in namesList] except TypeError: pass except Exception: pass if initOnly: print("Initialized detector") return None if not os.path.exists(imagePath): raise ValueError("Invalid image path `"+os.path.abspath(imagePath)+"`") # Do the detection #detections = detect(netMain, metaMain, imagePath, thresh) # if is used cv2.imread(image) detections = detect(netMain, metaMain, imagePath.encode("ascii"), thresh) if showImage: try: from skimage import io, draw import numpy as np image = io.imread(imagePath) print("*** "+str(len(detections))+" Results, color coded by confidence ***") imcaption = [] for detection in detections: label = detection[0] confidence = detection[1] pstring = label+": "+str(np.rint(100 * confidence))+"%" imcaption.append(pstring) print(pstring) bounds = detection[2] shape = image.shape # x = shape[1] # xExtent = int(x * bounds[2] / 100) # y = shape[0] # yExtent = int(y * bounds[3] / 100) yExtent = int(bounds[3]) xEntent = int(bounds[2]) # Coordinates are around the center xCoord = int(bounds[0] - bounds[2]/2) yCoord = int(bounds[1] - bounds[3]/2) boundingBox = [ [xCoord, yCoord], [xCoord, yCoord + yExtent], [xCoord + xEntent, yCoord + yExtent], [xCoord + xEntent, yCoord] ] # Wiggle it around to make a 3px border rr, cc = draw.polygon_perimeter([x[1] for x in boundingBox], [x[0] for x in boundingBox], shape= shape) rr2, cc2 = draw.polygon_perimeter([x[1] + 1 for x in boundingBox], [x[0] for x in boundingBox], shape= shape) rr3, cc3 = draw.polygon_perimeter([x[1] - 1 for x in boundingBox], [x[0] for x in boundingBox], shape= shape) rr4, cc4 = draw.polygon_perimeter([x[1] for x in boundingBox], [x[0] + 1 for x in boundingBox], shape= shape) rr5, cc5 = draw.polygon_perimeter([x[1] for x in boundingBox], [x[0] - 1 for x in boundingBox], shape= shape) boxColor = (int(255 * (1 - (confidence ** 2))), int(255 * (confidence ** 2)), 0) draw.set_color(image, (rr, cc), boxColor, alpha= 0.8) draw.set_color(image, (rr2, cc2), boxColor, alpha= 0.8) draw.set_color(image, (rr3, cc3), boxColor, alpha= 0.8) draw.set_color(image, (rr4, cc4), boxColor, alpha= 0.8) draw.set_color(image, (rr5, cc5), boxColor, alpha= 0.8) if not makeImageOnly: io.imshow(image) io.show() detections = { "detections": detections, "image": image, "caption": "\n<br/>".join(imcaption) } except Exception as e: print("Unable to show image: "+str(e)) return detections if __name__ == "__main__": print(performDetect()) ```
wpf做如何做可以上下拖动Y轴坐标的折线图 用WPFVisifire.Charts做的不能拖动
一下是我写的代码 但是不能上下拖动Y轴坐标,数据使用txt文件来读取 急用 各位大神帮帮忙 谢谢 using System; using System.Collections; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Text.RegularExpressions; using System.Threading.Tasks; using System.Windows; using System.Windows.Controls; using System.Windows.Data; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Imaging; using System.Windows.Navigation; using System.Windows.Shapes; using Visifire.Charts; namespace zhexian { /// <summary> /// MainWindow.xaml 的交互逻辑 /// </summary> public partial class MainWindow : Window { public MainWindow() { InitializeComponent(); } #region 公共属性 Visifire.Charts.Chart chart = new Visifire.Charts.Chart(); DataSeries dataSeries; string[] subLines = { "" }; Visifire.Charts.DataPoint dataPoint; #endregion /// <summary> /// 创建折线图 /// </summary> /// <param name="path"></param> private void zhexian(string path) { using (Stream resourceStream = new FileStream(path, FileMode.Open)) { using (StreamReader reader = new StreamReader(resourceStream, Encoding.GetEncoding("GB2312"))) { chart.Width = 980; chart.Height = 580; chart.Margin = new Thickness(100, 5, 10, 5); ArrayList mydata; dataPoint = new Visifire.Charts.DataPoint(); #region 创建折线图1 //解析所有行数据 var strings = reader.ReadToEnd().Split(new char[] { '\n' }, StringSplitOptions.RemoveEmptyEntries); mydata = new ArrayList(); ArrayList madata2 = new ArrayList(); for (int i = 0; i < strings.Length; i++) { sj j = new sj(); string[] stringArr = Regex.Split(strings[i], " "); j.number = new double[stringArr.Length]; for (int h = 0; h < stringArr.Length; h++) { j.number[h] = Double.Parse(stringArr[h]); } madata2.Add(j); } //整理后的列数据 string[] strArr = Regex.Split(strings[0], " "); for (int i = 0; i < strArr.Length; i++) { sj sj = new sj(); sj.number = new double[madata2.Count]; int h = 0; foreach (sj item in madata2) { if (i < item.number.Length) { sj.number[h] = item.number[i]; h++; } } mydata.Add(sj); } //开始划线 foreach (sj item in mydata) { double num1 = 1; // 创建一个新的数据线。 dataSeries = new DataSeries(); // 设置数据线的格式。 dataSeries.LegendText = "樱桃"; dataSeries.RenderAs = RenderAs.Line;//折线图 for (int i = 0; i < item.number.Length; i++) { // 创建一个数据点的实例。 dataPoint = new Visifire.Charts.DataPoint(); // 设置X轴点 dataPoint.XValue = num1; //设置Y轴点 string num = item.number[i].ToString(); dataPoint.YValue = Double.Parse(num); dataPoint.MarkerSize = 8; num1++; //添加数据点 dataSeries.DataPoints.Add(dataPoint); } // 添加数据线到数据序列。 chart.Series.Add(dataSeries); } #endregion } } //将数据绑定到Grid面板上 System.Windows.Controls.Grid gr = new System.Windows.Controls.Grid(); gr.Children.Add(chart); Simon.Children.Add(gr); } /// <summary> /// 窗体加载 /// </summary> /// <param name="sender"></param> /// <param name="e"></param> private void Window_Loaded(object sender, RoutedEventArgs e) { string path = @"C:\Users\AllDream\Desktop\新建文件夹 (2)\ZheXian\zhexian\shuju.txt"; zhexian(path); huoqu(); } //创建一个double类型的数组类用于存储重新组合之后的数据 public class sj { public double[] number = null; } public void huoqu() { Point zuobiao = new Point(); object xzou = dataPoint.XValue; double oo = (double)xzou; zuobiao.X = oo; zuobiao.Y = dataPoint.YValue; } } }
python3.5错误'module' object is not callable
(1) import urllib.request from cons import headers def getUrlList(): req=urllib.request('https://mm.taobao.com/tstar/search/tstar_model.do?_input_charset=utf-8') req.add_header('user-agent',headers()) # print (headers()) html=urllib.urlopen(req, data='q&viewFlag=A&sortType=default&searchStyle=&searchRegion=city%3A&searchFansNum=&currentPage=1&pageSize=100').read() print (html) getUrlList() (2) import random headerstr='''Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36 Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)''' def headers(): header=headerstr.split('\n') length=len(header) return header[random.randint(0,length-1)] 运行(1) 产生错误如下: D:\programmingtools\anaconda\python.exe D:/programmingtools/pycharmpro/files/201711112013/taobeauty.py Traceback (most recent call last): File "D:/programmingtools/pycharmpro/files/201711112013/taobeauty.py", line 13, in <module> getUrlList() File "D:/programmingtools/pycharmpro/files/201711112013/taobeauty.py", line 6, in getUrlList req=urllib.request('https://mm.taobao.com/tstar/search/tstar_model.do?_input_charset=utf-8') TypeError: 'module' object is not callable Process finished with exit code 1
大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了
大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...
【JSON解析】浅谈JSONObject的使用
简介 在程序开发过程中,在参数传递,函数返回值等方面,越来越多的使用JSON。JSON(JavaScript Object Notation)是一种轻量级的数据交换格式,同时也易于机器解析和生成、易于理解、阅读和撰写,而且Json采用完全独立于语言的文本格式,这使得Json成为理想的数据交换语言。 JSON建构于两种结构: “名称/值”对的集合(A Collection of name/va...
《MySQL 性能优化》之理解 MySQL 体系结构
本文介绍 MySQL 的体系结构,包括物理结构、逻辑结构以及插件式存储引擎。
程序员请照顾好自己,周末病魔差点一套带走我。
程序员在一个周末的时间,得了重病,差点当场去世,还好及时挽救回来了。
卸载 x 雷某度!GitHub 标星 1.5w+,从此我只用这款全能高速下载工具!
作者 | Rocky0429 来源 | Python空间 大家好,我是 Rocky0429,一个喜欢在网上收集各种资源的蒟蒻… 网上资源眼花缭乱,下载的方式也同样千奇百怪,比如 BT 下载,磁力链接,网盘资源等等等等,下个资源可真不容易,不一样的方式要用不同的下载软件,因此某比较有名的 x 雷和某度网盘成了我经常使用的工具。 作为一个没有钱的穷鬼,某度网盘几十 kb 的下载速度让我...
只因接了一个电话,程序员被骗 30 万!
今天想给大家说一个刚刚发生在我身边的一起真实的诈骗经历,我的朋友因此被骗走30万。注:为了保护当事人隐私,部分情节进行了修改。1平安夜突来的电话开始以为就像普通的诈骗一样,想办法让你把钱...
我一个37岁的程序员朋友
周末了,人一旦没有点事情干,心里就瞎想,而且跟几个老男人坐在一起,更容易瞎想,我自己现在也是 30 岁了,也是无时无刻在担心自己的职业生涯,担心丢掉工作没有收入,担心身体机能下降,担心突...
python自动下载图片
近日闲来无事,总有一种无形的力量萦绕在朕身边,让朕精神涣散,昏昏欲睡。 可是,像朕这么有职业操守的社畜怎么能在上班期间睡瞌睡呢,我不禁陷入了沉思。。。。 突然旁边的IOS同事问:‘嘿,兄弟,我发现一个网站的图片很有意思啊,能不能帮我保存下来提升我的开发灵感?’ 作为一个坚强的社畜怎么能说自己不行呢,当时朕就不假思索的答应:‘oh, It’s simple. Wait for me for a ...
一名大专同学的四个问题
【前言】   收到一封来信,赶上各种事情拖了几日,利用今天要放下工作的时机,做个回复。   2020年到了,就以这一封信,作为开年标志吧。 【正文】   您好,我是一名现在有很多困惑的大二学生。有一些问题想要向您请教。   先说一下我的基本情况,高考失利,不想复读,来到广州一所大专读计算机应用技术专业。学校是偏艺术类的,计算机专业没有实验室更不用说工作室了。而且学校的学风也不好。但我很想在计算机领...
复习一周,京东+百度一面,不小心都拿了Offer
京东和百度一面都问了啥,面试官百般刁难,可惜我全会。
Java 14 都快来了,为什么还有这么多人固守Java 8?
从Java 9开始,Java版本的发布就让人眼花缭乱了。每隔6个月,都会冒出一个新版本出来,Java 10 , Java 11, Java 12, Java 13, 到2020年3月份,...
达摩院十大科技趋势发布:2020 非同小可!
【CSDN编者按】1月2日,阿里巴巴发布《达摩院2020十大科技趋势》,十大科技趋势分别是:人工智能从感知智能向认知智能演进;计算存储一体化突破AI算力瓶颈;工业互联网的超融合;机器间大规模协作成为可能;模块化降低芯片设计门槛;规模化生产级区块链应用将走入大众;量子计算进入攻坚期;新材料推动半导体器件革新;保护数据隐私的AI技术将加速落地;云成为IT技术创新的中心 。 新的画卷,正在徐徐展开。...
轻松搭建基于 SpringBoot + Vue 的 Web 商城应用
首先介绍下在本文出现的几个比较重要的概念: 函数计算(Function Compute): 函数计算是一个事件驱动的服务,通过函数计算,用户无需管理服务器等运行情况,只需编写代码并上传。函数计算准备计算资源,并以弹性伸缩的方式运行用户代码,而用户只需根据实际代码运行所消耗的资源进行付费。Fun: Fun 是一个用于支持 Serverless 应用部署的工具,能帮助您便捷地管理函数计算、API ...
Python+OpenCV实时图像处理
目录 1、导入库文件 2、设计GUI 3、调用摄像头 4、实时图像处理 4.1、阈值二值化 4.2、边缘检测 4.3、轮廓检测 4.4、高斯滤波 4.5、色彩转换 4.6、调节对比度 5、退出系统 初学OpenCV图像处理的小伙伴肯定对什么高斯函数、滤波处理、阈值二值化等特性非常头疼,这里给各位分享一个小项目,可通过摄像头实时动态查看各类图像处理的特点,也可对各位调参、测试...
2020年一线城市程序员工资大调查
人才需求 一线城市共发布岗位38115个,招聘120827人。 其中 beijing 22805 guangzhou 25081 shanghai 39614 shenzhen 33327 工资分布 2020年中国一线城市程序员的平均工资为16285元,工资中位数为14583元,其中95%的人的工资位于5000到20000元之间。 和往年数据比较: yea...
为什么猝死的都是程序员,基本上不见产品经理猝死呢?
相信大家时不时听到程序员猝死的消息,但是基本上听不到产品经理猝死的消息,这是为什么呢? 我们先百度搜一下:程序员猝死,出现将近700多万条搜索结果: 搜索一下:产品经理猝死,只有400万条的搜索结果,从搜索结果数量上来看,程序员猝死的搜索结果就比产品经理猝死的搜索结果高了一倍,而且从下图可以看到,首页里面的五条搜索结果,其实只有两条才是符合条件。 所以程序员猝死的概率真的比产品经理大,并不是错...
害怕面试被问HashMap?这一篇就搞定了!
声明:本文以jdk1.8为主! 搞定HashMap 作为一个Java从业者,面试的时候肯定会被问到过HashMap,因为对于HashMap来说,可以说是Java集合中的精髓了,如果你觉得自己对它掌握的还不够好,我想今天这篇文章会非常适合你,至少,看了今天这篇文章,以后不怕面试被问HashMap了 其实在我学习HashMap的过程中,我个人觉得HashMap还是挺复杂的,如果真的想把它搞得明明白...
毕业5年,我问遍了身边的大佬,总结了他们的学习方法
我问了身边10个大佬,总结了他们的学习方法,原来成功都是有迹可循的。
推荐10个堪称神器的学习网站
每天都会收到很多读者的私信,问我:“二哥,有什么推荐的学习网站吗?最近很浮躁,手头的一些网站都看烦了,想看看二哥这里有什么新鲜货。” 今天一早做了个恶梦,梦到被老板辞退了。虽然说在我们公司,只有我辞退老板的份,没有老板辞退我这一说,但是还是被吓得 4 点多都起来了。(主要是因为我掌握着公司所有的核心源码,哈哈哈) 既然 4 点多起来,就得好好利用起来。于是我就挑选了 10 个堪称神器的学习网站,推...
这些软件太强了,Windows必装!尤其程序员!
Windows可谓是大多数人的生产力工具,集娱乐办公于一体,虽然在程序员这个群体中都说苹果是信仰,但是大部分不都是从Windows过来的,而且现在依然有很多的程序员用Windows。 所以,今天我就把我私藏的Windows必装的软件分享给大家,如果有一个你没有用过甚至没有听过,那你就赚了????,这可都是提升你幸福感的高效率生产力工具哦! 走起!???? NO、1 ScreenToGif 屏幕,摄像头和白板...
阿里面试,面试官没想到一个ArrayList,我都能跟他扯半小时
我是真的没想到,面试官会这样问我ArrayList。
曾经优秀的人,怎么就突然不优秀了。
职场上有很多辛酸事,很多合伙人出局的故事,很多技术骨干被裁员的故事。说来模板都类似,曾经是名校毕业,曾经是优秀员工,曾经被领导表扬,曾经业绩突出,然而突然有一天,因为种种原因,被裁员了,...
大学四年因为知道了这32个网站,我成了别人眼中的大神!
依稀记得,毕业那天,我们导员发给我毕业证的时候对我说“你可是咱们系的风云人物啊”,哎呀,别提当时多开心啦????,嗯,我们导员是所有导员中最帅的一个,真的???? 不过,导员说的是实话,很多人都叫我大神的,为啥,因为我知道这32个网站啊,你说强不强????,这次是绝对的干货,看好啦,走起来! PS:每个网站都是学计算机混互联网必须知道的,真的牛杯,我就不过多介绍了,大家自行探索,觉得没用的,尽管留言吐槽吧???? 社...
良心推荐,我珍藏的一些Chrome插件
上次搬家的时候,发了一个朋友圈,附带的照片中不小心暴露了自己的 Chrome 浏览器插件之多,于是就有小伙伴评论说分享一下我觉得还不错的浏览器插件。 我下面就把我日常工作和学习中经常用到的一些 Chrome 浏览器插件分享给大家,随便一个都能提高你的“生活品质”和工作效率。 Markdown Here Markdown Here 可以让你更愉快的写邮件,由于支持 Markdown 直接转电子邮...
看完这篇HTTP,跟面试官扯皮就没问题了
我是一名程序员,我的主要编程语言是 Java,我更是一名 Web 开发人员,所以我必须要了解 HTTP,所以本篇文章就来带你从 HTTP 入门到进阶,看完让你有一种恍然大悟、醍醐灌顶的感觉。 最初在有网络之前,我们的电脑都是单机的,单机系统是孤立的,我还记得 05 年前那会儿家里有个电脑,想打电脑游戏还得两个人在一个电脑上玩儿,及其不方便。我就想为什么家里人不让上网,我的同学 xxx 家里有网,每...
史上最全的IDEA快捷键总结
现在Idea成了主流开发工具,这篇博客对其使用的快捷键做了总结,希望对大家的开发工作有所帮助。
阿里程序员写了一个新手都写不出的低级bug,被骂惨了。
这种新手都不会范的错,居然被一个工作好几年的小伙子写出来,差点被当场开除了。
谁是华为扫地僧?
是的,华为也有扫地僧!2020年2月11-12日,“养在深闺人不知”的华为2012实验室扫地僧们,将在华为开发者大会2020(Cloud)上,和大家见面。到时,你可以和扫地僧们,吃一个洋...
Idea 中最常用的10款插件(提高开发效率),一定要学会使用!
学习使用一些插件,可以提高开发效率。对于我们开发人员很有帮助。这篇博客介绍了开发中使用的插件。
AI 没让人类失业,搞 AI 的人先失业了
最近和几个 AI 领域的大佬闲聊 根据他们讲的消息和段子 改编出下面这个故事 如有雷同 都是巧合 1. 老王创业失败,被限制高消费 “这里写我跑路的消息实在太夸张了。” 王葱葱哼笑一下,把消息分享给群里。 阿杰也看了消息,笑了笑。在座几位也都笑了。 王葱葱是个有名的人物,21岁那年以全额奖学金进入 KMU 攻读人工智能博士,累计发表论文 40 余篇,个人技术博客更是成为深度学习领域内风向标。 ...
立即提问