老师，视频的操作比语音快好几秒，听起来有些就错过了

from urllib.request import Request,build_opener,HTTPCookieProcessor
from urllib.parse import urlencode
from fake_useragent import UserAgent
from http.cookiejar import MozillaCookieJar
import ssl
def get_cookie():
    url = 'https://rl.mail.qq.com/cgi-bin/getinvestigate?sid=ucPJvMl86d29Q1wC'
    headers={
        'User-Agent':UserAgent().chrome
    }
    data = {
        'user': '523531642',
        'password': '*********'
    }
    context = ssl._create_unverified_context()
    res = Request(url,headers=headers,data=urlencode(data).encode())
    cookiejar = MozillaCookieJar()
    handler = HTTPCookieProcessor(cookiejar)
    opener = build_opener(handler)
    resp = opener.open(res)
    cookiejar.save('cookie.text',ignore_discard=True,ignore_expires=True)
def use_cookie():
    info_url='https://mail.qq.com'
    headers = {
        'User-Agent': UserAgent().chrome
    }
    context = ssl._create_unverified_context()
    res = Request(info_url, headers=headers, data=urlencode(headers).encode())
    cookiejar = MozillaCookieJar()
    cookiejar.load('cookie.text',ignore_expires=True,ignore_discard=True)
    handler = HTTPCookieProcessor(cookiejar)
    opener = build_opener(handler)
    resp = opener.open(res)
    print(resp.read())
if __name__ == '__main__':
    get_cookie()
    #use_cookie()

老师这个qq邮箱是https   我保存的的cookie文件为什么不行

是登录的url有问题吗？老师我的代码有问题吗？

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

.mail.qq.com   TRUE   /  FALSE     qm_sk  
.mail.qq.com   TRUE   /  FALSE     qm_ssum    
.qq.com    TRUE   /  FALSE     qm_sk  
.qq.com    TRUE   /  FALSE     qm_ssum

这里的百度的url地址,加s和不加s为啥爬出来的区别这么大??????????

为什么使用代理出现网页得那种感觉了？

老师：我运行代码就报错，

2020-03-30 15:25:37 [scrapy.utils.log] INFO: Scrapy 2.0.1 started (bot: book)

2020-03-30 15:25:37 [scrapy.utils.log] INFO: Versions: lxml 4.5.0.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.5.2, w3lib 1.21.0, Twisted 20.3.0, Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)], pyOpenSSL 19.1.0 (OpenSSL 1.1.1d  10 Sep 2019), cryptography 2.8, Platform Windows-10-10.0.17763-SP0

2020-03-30 15:25:37 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor

2020-03-30 15:25:37 [scrapy.crawler] INFO: Overridden settings:

{'BOT_NAME': 'book',

 'NEWSPIDER_MODULE': 'book.spiders',

 'SPIDER_MODULES': ['book.spiders'],

 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '

               '(KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'}

2020-03-30 15:25:37 [scrapy.extensions.telnet] INFO: Telnet Password: 55216b70a4289d90

2020-03-30 15:25:37 [scrapy.middleware] INFO: Enabled extensions:

['scrapy.extensions.corestats.CoreStats',

 'scrapy.extensions.telnet.TelnetConsole',

 'scrapy.extensions.logstats.LogStats']

2020-03-30 15:25:37 [scrapy.middleware] INFO: Enabled downloader middlewares:

['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',

 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',

 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',

 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',

 'scrapy.downloadermiddlewares.retry.RetryMiddleware',

 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',

 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',

 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',

 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',

 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',

 'scrapy.downloadermiddlewares.stats.DownloaderStats']

2020-03-30 15:25:37 [scrapy.middleware] INFO: Enabled spider middlewares:

['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',

 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',

 'scrapy.spidermiddlewares.referer.RefererMiddleware',

 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',

 'scrapy.spidermiddlewares.depth.DepthMiddleware']

2020-03-30 15:25:37 [scrapy.middleware] INFO: Enabled item pipelines:

[]

2020-03-30 15:25:37 [scrapy.core.engine] INFO: Spider opened

2020-03-30 15:25:37 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2020-03-30 15:25:37 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023

2020-03-30 15:25:37 [scrapy.core.scraper] ERROR: Error downloading <GET htttp://www.zongheng.com/rank/details.html?rt=1&p=1>

Traceback (most recent call last):

  File "C:\Programs\Python\Python38\lib\site-packages\twisted\internet\defer.py", line 1416, in _inlineCallbacks

    result = result.throwExceptionIntoGenerator(g)

  File "C:\Programs\Python\Python38\lib\site-packages\twisted\python\failure.py", line 512, in throwExceptionIntoGenerator

    return g.throw(self.type, self.value, self.tb)

  File "C:\Programs\Python\Python38\lib\site-packages\scrapy\core\downloader\middleware.py", line 42, in process_request

    defer.returnValue((yield download_func(request=request, spider=spider)))

  File "C:\Programs\Python\Python38\lib\site-packages\scrapy\utils\defer.py", line 55, in mustbe_deferred

    result = f(*args, **kw)

  File "C:\Programs\Python\Python38\lib\site-packages\scrapy\core\downloader\handlers\__init__.py", line 74, in download_request

    raise NotSupported("Unsupported URL scheme '%s': %s" %

scrapy.exceptions.NotSupported: Unsupported URL scheme 'htttp': no handler available for that scheme

2020-03-30 15:25:37 [scrapy.core.engine] INFO: Closing spider (finished)

2020-03-30 15:25:37 [scrapy.statscollectors] INFO: Dumping Scrapy stats:

{'downloader/exception_count': 1,

 'downloader/exception_type_count/scrapy.exceptions.NotSupported': 1,

 'downloader/request_bytes': 323,

 'downloader/request_count': 1,

 'downloader/request_method_count/GET': 1,

 'elapsed_time_seconds': 0.324646,

 'finish_reason': 'finished',

 'finish_time': datetime.datetime(2020, 3, 30, 7, 25, 37, 768198),

 'log_count/ERROR': 1,

 'log_count/INFO': 10,

 'scheduler/dequeued': 1,

 'scheduler/dequeued/memory': 1,

 'scheduler/enqueued': 1,

 'scheduler/enqueued/memory': 1,

 'start_time': datetime.datetime(2020, 3, 30, 7, 25, 37, 443552)}

2020-03-30 15:25:37 [scrapy.core.engine] INFO: Spider closed (finished)

运行不出来

老师，我执行(scrapy runspider xxx.py)时报这个错误，这是什么原因啊。在这里解决了一个问题，
又会有另一个问题出来，现在是在不知道怎么办了；希望老师帮帮忙，尽量在这个环节说的细一点，谢谢老师

猫眼电影网站改了吧，爬不到内容

from urllib.request import urlopen,Request
from urllib.parse import quote
def get_html(url):
    headers={
        "User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"
    }
    request=Request(url,headers=headers)
    response=urlopen(request)
    return response.read().decode()
def save_html(html,filename):
    with open(filename,'w',encoding='utf-8') as f:
        f.write(html)

def main():
    content = input("请输入要获取的贴吧:")
    num = int(input("请输入要获取的页:"))

    for i in range(num):
        url="https://tieba.baidu.com/f?kw=' + quote(content) + '&ie=utf-8&pn={}".format(i*50)
        print(url)
        html=get_html(url)
        filename = '第' + str(i + 1) + '页.html'
        save_html(html,filename)
    # url = "https://tieba.baidu.com/f?kw=%E5%B0%9A%E5%AD%A6%E5%A0%82&ie=utf-8&pn=50"
    # html = get_html(url)
    # save_html(html)
    # url = "https://tieba.baidu.com/f?kw=%E5%B0%9A%E5%AD%A6%E5%A0%82&ie=utf-8&pn=100"
    # html = get_html(url)
    # save_html(html)

if __name__ == '__main__':
    main()
    
    
 老师，请问一下这个运行出来就只有下面那一行成功了，后面就报错是怎么回事儿呀?麻烦老师帮我看一下

老师这几个数据提取那个好用：正则表达式，Beautiful Soup4，pyQuery，XPath，JsonPath

from urllib.request import urlopen,Request
from urllib.parse import quote
#报错
      #UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-17: ordinal not in range(128)
# #url里面的参数不能为中文

args = 'python'
url = f'https://www.baidu.com/s?wd={quote(args)}'
header = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36'}
req = Request(url,headers=header)

resp = urlopen(req)

print(resp.read().decode()[:1500])

老师帮我看下哪里错了

老师，请问一下可以新建一个爬虫的项目，然后创建多个子文件嘛？我看老师是 一次创建一个新的项目.

老师，我用的是火狐浏览器，我已经安装了selenium包，把火狐对应的driver也放到了Python环境中，可是却不能运行代码

把Proxy-SwitchyOmega_v2.3.16文件拖到Google扩展程序的时候显示程序包无效。然后网上找方法又变成扩展程序又变成这样，打不开扩展文件的这个选项页面。