如何用python获取这个网页的HTML（超文本链接语言）？

Question

https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

<ul>
<li><a href="javascript:;" onclick=" javascript:dl([101,48,53,104,121,106,49,115,51,116,54,105,113,98,110,46,119,45,52,50,95,47,109,97,99,100,112,56,111,118,108], &quot;6B4;IL18DI3;I&lt;K@0&gt;5&gt;B@GI7J@GI7J@?:=CFEH9A&gt;?2M&quot;); &quot;javascript: dl(&quot;" title="[70 KB] [Oct 17, 2019]">pyqpbo‑0.1.2‑cp38‑cp38‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,105,113,119,47,112,98,121,111,51,108,116,110,50,46,48,53,45,115,49,104,118,99,106,56], &quot;A&lt;F14D?:3461457@&gt;=B=&lt;@E48G@E48G@20;8&lt;=2C9&quot;); &quot;javascript: dl(&quot;" title="[57 KB] [Oct 17, 2019]">pyqpbo‑0.1.2‑cp38‑cp38‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,115,106,121,95,112,53,109,49,99,118,100,105,116,52,46,97,48,108,51,45,119,98,47,113,55,54,111,104,110,50], &quot;0M1G495&lt;F42G4EJC@&gt;7&gt;MC84BHC84BH6CD;L3?6:I=&gt;DKA&quot;); &quot;javascript: dl(&quot;" title="[69 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp37‑cp37m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,46,112,55,116,98,108,115,106,48,99,47,109,113,119,50,53,104,111,51,121,45,118,110,105,49], &quot;6&gt;7&lt;1E?3:1C&lt;14AD80H0&gt;D91B2D91B2;D=GFB&gt;0=@5&quot;); &quot;javascript: dl(&quot;" title="[57 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp37‑cp37m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,113,119,53,48,95,47,49,97,116,109,110,46,106,108,121,45,50,104,99,111,112,118,100,98,52,115,105,51,54], &quot;I@&lt;0DE285D&gt;0DGC?3;6;@?BDKL?BDKL9?1J:479FLH;1A=&quot;); &quot;javascript: dl(&quot;" title="[66 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp36‑cp36m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,48,106,49,115,116,111,105,119,99,104,108,113,121,46,53,54,118,50,112,47,110,109,45,98,51], &quot;3A1;B@&gt;4CB&lt;;BG5F0=2=AF8BH?F8BH?EF76DHA=79:&quot;); &quot;javascript: dl(&quot;" title="[54 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp36‑cp36m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,112,95,118,45,110,108,46,99,98,109,52,111,47,97,121,54,115,49,104,51,48,50,100,119,113,105,116,53,106], &quot;@ELH02KJ&lt;70CK&lt;0&gt;H08;3D6A6E370CK370CK93GI41=9F?:6GB5&quot;); &quot;javascript: dl(&quot;" title="[66 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp35‑cp35m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,110,50,46,115,111,47,118,121,109,112,99,53,48,119,49,108,106,51,104,98,116,45,105,113], &quot;31@G96;D5:9A;597G9C4E&lt;2&gt;21E:9A;E:9A;8E=F0A12=B?&quot;); &quot;javascript: dl(&quot;" title="[54 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp35‑cp35m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,46,121,47,116,113,108,98,119,112,115,53,54,99,51,50,118,52,111,109,105,45,95,104,97,100,49,110,48,106], &quot;9&gt;L48?:32&lt;8=@281486ADK0I0&gt;D&lt;8=@D&lt;8=@BD7CJEGBH;@07F5&quot;); &quot;javascript: dl(&quot;" title="[62 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp34‑cp34m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,111,98,110,45,109,46,115,116,47,51,106,119,99,112,49,105,121,118,113,52,53,104,48,50,108], &quot;6G:B=AD78&lt;=9C8=@B=103F5&gt;5G3&lt;=9C3&lt;=9C43;?29G5;EH&quot;); &quot;javascript: dl(&quot;" title="[56 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp34‑cp34m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,49,46,98,47,105,121,112,99,115,119,110,113,50,48,55,106,45,100,95,52,54,97,118,111,109,53,104,116,108], &quot;8&lt;?;6FIK376&lt;&gt;365;62G@=101&lt;@76&lt;&gt;@76&lt;&gt;H@94:BEHADC19JL&quot;); &quot;javascript: dl(&quot;" title="[71 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp27‑cp27m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,50,116,115,46,121,110,53,98,45,51,113,108,106,112,99,111,105,109,48,47,118,49,104,119,55], &quot;20&lt;:=D61C&gt;=0HC=4:=7?8B3E308&gt;=0H8&gt;=0HA8G@5903GF;&quot;); &quot;javascript: dl(&quot;" title="[60 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp27‑cp27m‑win32.whl</a></li>
</ul>
</li><ul>
<li><a href="javascript:;" onclick=" javascript:dl([101,48,53,104,121,106,49,115,51,116,54,105,113,98,110,46,119,45,52,50,95,47,109,97,99,100,112,56,111,118,108], &quot;6B4;IL18DI3;I&lt;K@0&gt;5&gt;B@GI7J@GI7J@?:=CFEH9A&gt;?2M&quot;); &quot;javascript: dl(&quot;" title="[70 KB] [Oct 17, 2019]">pyqpbo‑0.1.2‑cp38‑cp38‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,105,113,119,47,112,98,121,111,51,108,116,110,50,46,48,53,45,115,49,104,118,99,106,56], &quot;A&lt;F14D?:3461457@&gt;=B=&lt;@E48G@E48G@20;8&lt;=2C9&quot;); &quot;javascript: dl(&quot;" title="[57 KB] [Oct 17, 2019]">pyqpbo‑0.1.2‑cp38‑cp38‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,115,106,121,95,112,53,109,49,99,118,100,105,116,52,46,97,48,108,51,45,119,98,47,113,55,54,111,104,110,50], &quot;0M1G495&lt;F42G4EJC@&gt;7&gt;MC84BHC84BH6CD;L3?6:I=&gt;DKA&quot;); &quot;javascript: dl(&quot;" title="[69 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp37‑cp37m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,46,112,55,116,98,108,115,106,48,99,47,109,113,119,50,53,104,111,51,121,45,118,110,105,49], &quot;6&gt;7&lt;1E?3:1C&lt;14AD80H0&gt;D91B2D91B2;D=GFB&gt;0=@5&quot;); &quot;javascript: dl(&quot;" title="[57 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp37‑cp37m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,113,119,53,48,95,47,49,97,116,109,110,46,106,108,121,45,50,104,99,111,112,118,100,98,52,115,105,51,54], &quot;I@&lt;0DE285D&gt;0DGC?3;6;@?BDKL?BDKL9?1J:479FLH;1A=&quot;); &quot;javascript: dl(&quot;" title="[66 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp36‑cp36m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,48,106,49,115,116,111,105,119,99,104,108,113,121,46,53,54,118,50,112,47,110,109,45,98,51], &quot;3A1;B@&gt;4CB&lt;;BG5F0=2=AF8BH?F8BH?EF76DHA=79:&quot;); &quot;javascript: dl(&quot;" title="[54 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp36‑cp36m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,112,95,118,45,110,108,46,99,98,109,52,111,47,97,121,54,115,49,104,51,48,50,100,119,113,105,116,53,106], &quot;@ELH02KJ&lt;70CK&lt;0&gt;H08;3D6A6E370CK370CK93GI41=9F?:6GB5&quot;); &quot;javascript: dl(&quot;" title="[66 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp35‑cp35m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,110,50,46,115,111,47,118,121,109,112,99,53,48,119,49,108,106,51,104,98,116,45,105,113], &quot;31@G96;D5:9A;597G9C4E&lt;2&gt;21E:9A;E:9A;8E=F0A12=B?&quot;); &quot;javascript: dl(&quot;" title="[54 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp35‑cp35m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,46,121,47,116,113,108,98,119,112,115,53,54,99,51,50,118,52,111,109,105,45,95,104,97,100,49,110,48,106], &quot;9&gt;L48?:32&lt;8=@281486ADK0I0&gt;D&lt;8=@D&lt;8=@BD7CJEGBH;@07F5&quot;); &quot;javascript: dl(&quot;" title="[62 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp34‑cp34m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,111,98,110,45,109,46,115,116,47,51,106,119,99,112,49,105,121,118,113,52,53,104,48,50,108], &quot;6G:B=AD78&lt;=9C8=@B=103F5&gt;5G3&lt;=9C3&lt;=9C43;?29G5;EH&quot;); &quot;javascript: dl(&quot;" title="[56 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp34‑cp34m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,49,46,98,47,105,121,112,99,115,119,110,113,50,48,55,106,45,100,95,52,54,97,118,111,109,53,104,116,108], &quot;8&lt;?;6FIK376&lt;&gt;365;62G@=101&lt;@76&lt;&gt;@76&lt;&gt;H@94:BEHADC19JL&quot;); &quot;javascript: dl(&quot;" title="[71 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp27‑cp27m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,50,116,115,46,121,110,53,98,45,51,113,108,106,112,99,111,105,109,48,47,118,49,104,119,55], &quot;20&lt;:=D61C&gt;=0HC=4:=7?8B3E308&gt;=0H8&gt;=0HA8G@5903GF;&quot;); &quot;javascript: dl(&quot;" title="[60 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp27‑cp27m‑win32.whl</a></li>
</ul>
</li><ul>
<li><a href="javascript:;" onclick=" javascript:dl([101,48,53,104,121,106,49,115,51,116,54,105,113,98,110,46,119,45,52,50,95,47,109,97,99,100,112,56,111,118,108], &quot;6B4;IL18DI3;I&lt;K@0&gt;5&gt;B@GI7J@GI7J@?:=CFEH9A&gt;?2M&quot;); &quot;javascript: dl(&quot;" title="[70 KB] [Oct 17, 2019]">pyqpbo‑0.1.2‑cp38‑cp38‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,105,113,119,47,112,98,121,111,51,108,116,110,50,46,48,53,45,115,49,104,118,99,106,56], &quot;A&lt;F14D?:3461457@&gt;=B=&lt;@E48G@E48G@20;8&lt;=2C9&quot;); &quot;javascript: dl(&quot;" title="[57 KB] [Oct 17, 2019]">pyqpbo‑0.1.2‑cp38‑cp38‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,115,106,121,95,112,53,109,49,99,118,100,105,116,52,46,97,48,108,51,45,119,98,47,113,55,54,111,104,110,50], &quot;0M1G495&lt;F42G4EJC@&gt;7&gt;MC84BHC84BH6CD;L3?6:I=&gt;DKA&quot;); &quot;javascript: dl(&quot;" title="[69 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp37‑cp37m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,46,112,55,116,98,108,115,106,48,99,47,109,113,119,50,53,104,111,51,121,45,118,110,105,49], &quot;6&gt;7&lt;1E?3:1C&lt;14AD80H0&gt;D91B2D91B2;D=GFB&gt;0=@5&quot;); &quot;javascript: dl(&quot;" title="[57 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp37‑cp37m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,113,119,53,48,95,47,49,97,116,109,110,46,106,108,121,45,50,104,99,111,112,118,100,98,52,115,105,51,54], &quot;I@&lt;0DE285D&gt;0DGC?3;6;@?BDKL?BDKL9?1J:479FLH;1A=&quot;); &quot;javascript: dl(&quot;" title="[66 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp36‑cp36m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,48,106,49,115,116,111,105,119,99,104,108,113,121,46,53,54,118,50,112,47,110,109,45,98,51], &quot;3A1;B@&gt;4CB&lt;;BG5F0=2=AF8BH?F8BH?EF76DHA=79:&quot;); &quot;javascript: dl(&quot;" title="[54 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp36‑cp36m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,112,95,118,45,110,108,46,99,98,109,52,111,47,97,121,54,115,49,104,51,48,50,100,119,113,105,116,53,106], &quot;@ELH02KJ&lt;70CK&lt;0&gt;H08;3D6A6E370CK370CK93GI41=9F?:6GB5&quot;); &quot;javascript: dl(&quot;" title="[66 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp35‑cp35m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,110,50,46,115,111,47,118,121,109,112,99,53,48,119,49,108,106,51,104,98,116,45,105,113], &quot;31@G96;D5:9A;597G9C4E&lt;2&gt;21E:9A;E:9A;8E=F0A12=B?&quot;); &quot;javascript: dl(&quot;" title="[54 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp35‑cp35m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,46,121,47,116,113,108,98,119,112,115,53,54,99,51,50,118,52,111,109,105,45,95,104,97,100,49,110,48,106], &quot;9&gt;L48?:32&lt;8=@281486ADK0I0&gt;D&lt;8=@D&lt;8=@BD7CJEGBH;@07F5&quot;); &quot;javascript: dl(&quot;" title="[62 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp34‑cp34m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,111,98,110,45,109,46,115,116,47,51,106,119,99,112,49,105,121,118,113,52,53,104,48,50,108], &quot;6G:B=AD78&lt;=9C8=@B=103F5&gt;5G3&lt;=9C3&lt;=9C43;?29G5;EH&quot;); &quot;javascript: dl(&quot;" title="[56 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp34‑cp34m‑win32.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,49,46,98,47,105,121,112,99,115,119,110,113,50,48,55,106,45,100,95,52,54,97,118,111,109,53,104,116,108], &quot;8&lt;?;6FIK376&lt;&gt;365;62G@=101&lt;@76&lt;&gt;@76&lt;&gt;H@94:BEHADC19JL&quot;); &quot;javascript: dl(&quot;" title="[71 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp27‑cp27m‑win_amd64.whl</a></li>
<li><a href="javascript:;" onclick=" javascript:dl([101,50,116,115,46,121,110,53,98,45,51,113,108,106,112,99,111,105,109,48,47,118,49,104,119,55], &quot;20&lt;:=D61C&gt;=0HC=4:=7?8B3E308&gt;=0H8&gt;=0HA8G@5903GF;&quot;); &quot;javascript: dl(&quot;" title="[60 KB] [Feb 25, 2018]">pyqpbo‑0.1.2‑cp27‑cp27m‑win32.whl</a></li>
</ul>
</li>

这是部分下载后的内容

Answer 1

实验了一下，加了个请求头，试过可以获取，我的代码

import requests
import html

headers = {"User-Agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.6) ",
           "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
           "Accept-Language": "en-us",
           "Connection": "keep-alive",
           "Accept-Charset": "GB2312,utf-8;q=0.7,*;q=0.7"}

url = 'https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted'
print("开始获取...")
res = requests.get(url, headers=headers, allow_redirects=False)

# 过滤下转义字符
ac_html = html.unescape(res.text)
print("获取完毕")

f = open('twisted.html', 'w', encoding='utf-8')
f.write(ac_html)
f.close()