V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
coolloves
V2EX  ›  Python

请教一个 requests 使用代理的问题!

  •  
  •   coolloves · 2016-06-17 18:27:24 +08:00 · 8737 次点击
    这是一个创建于 3086 天前的主题,其中的信息可能已经有所发展或是发生改变。
    #!/usr/bin/python

    import requests
    import random

    def rewrite(ip):
    return "{\'http\'"+":"+"\'http://"+ip+"\'}" #格式化代理

    def req(url,p=None):
    r = None
    headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) chrome/47.0.1453.110 Safari/5
    37.36','Refefer':url}
    s = requests.Session()
    with open("/home/tech/proxyok","r") as f:
    ips=f.read().strip('\n').split()
    ipss = map(rewrite,ips)
    proxy = random.choice(ipss)

    if proxy:
    try:
    r = s.get(url,headers=headers,proxies=proxy,timeout=3)
    except Exception,e:
    print e
    else:
    try:
    r = s.get(url,headers=headers)
    except Exception,e:
    print e
    return r

    if __name__ == "__main__":
    print req('http://baidu.com')

    运行后报'str' object has no attribute 'get'

    我以为是 requests.get 这个 get 的问题,后来把 try 去掉后,追踪到以下错误,但是我不明白是什么原因,新人,请指点一下,谢谢!
    Traceback (most recent call last):
    File "httpreq.py", line 31, in <module>
    print req('http://xueqiu.com')
    File "httpreq.py", line 20, in req
    r = s.get(url,headers=headers,proxies=p,timeout=3)
    File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 310, in get
    return self.request('GET', url, **kwargs)
    File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
    File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
    File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 155, in send
    conn = self.get_connection(request.url, proxies)
    File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 119, in get_connection
    proxy = proxies.get(urlparse(url).scheme)
    AttributeError: 'str' object has no attribute 'get'
    12 条回复    2016-06-19 07:51:36 +08:00
    aitaii
        1
    aitaii  
       2016-06-17 18:38:59 +08:00
    lxy
        2
    lxy  
       2016-06-17 18:49:01 +08:00
    报错信息写得很清楚了,一步步往上看,你的 proxy 数据类型不对。
    mringg
        3
    mringg  
       2016-06-17 19:57:19 +08:00 via Android
    传入的应该是 dict 而不是 str
    Kisesy
        4
    Kisesy  
       2016-06-17 20:51:55 +08:00
    看看你注释的地方
    coolloves
        5
    coolloves  
    OP
       2016-06-17 20:57:18 +08:00
    我传入的是 dict 啊,所以我才比较纳闷
    [root@localhost sh]# python httpreq.py
    {'http':'http://116.226.98.54:8118'} 这个是我打印出来的 proxy
    True
    'str' object has no attribute 'get'
    None
    coolloves
        6
    coolloves  
    OP
       2016-06-17 20:59:27 +08:00
    @Kisesy
    我的代理抓下来是 ip:port 格式保存的
    我那个注释就是把代理改成{'http':'http://116.226.98.54:8118'} 格式
    Kisesy
        7
    Kisesy  
       2016-06-17 21:04:16 +08:00   ❤️ 1
    你确定这是 dict ?
    return "{\'http\'"+":"+"\'http://"+ip+"\'}"

    这样写不好吗
    def rewrite(ip):
    return {'http':'http://' + ip}
    coolloves
        8
    coolloves  
    OP
       2016-06-17 21:06:22 +08:00
    我那个 rewrite 返回的是 str,不是 dict,所以 proxies 传入的类型错误了,我刚看了下 p 的类型是 str,不是 dict,多谢各位.
    coolloves
        9
    coolloves  
    OP
       2016-06-17 21:07:14 +08:00
    @Kisesy 是的,我那样返回的是 str,刚学 python,多谢指点.
    SuperFashi
        10
    SuperFashi  
       2016-06-17 21:12:13 +08:00 via Android
    @coolloves 虽然你是传了个 str 回去,不过 eval 一发即可
    siteshen
        11
    siteshen  
       2016-06-17 23:10:09 +08:00
    @SuperFashi 别带坏初学者!!
    SlipStupig
        12
    SlipStupig  
       2016-06-19 07:51:36 +08:00
    @SuperFashi eval 是方便,但是不安全,能少用就少用
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   1162 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 24ms · UTC 19:00 · PVG 03:00 · LAX 11:00 · JFK 14:00
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.