V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
Tornado Documentation
http://www.v2ex.com/tornado/
Tornado on GitHub
https://github.com/facebook/tornado/
Tornado Gists
http://tornadogists.org/
kidlj
V2EX  ›  Tornado

阅读 Tornado 源码过程中的一个疑惑,求解答

  •  
  •   kidlj · 2016-01-05 19:50:34 +08:00 · 9152 次点击
    这是一个创建于 3247 天前的主题,其中的信息可能已经有所发展或是发生改变。

    下面是 tornado.httpclient.AsyncHTTPClient类的 fetch()方法的源代码。我没有在里面找到任何"fetch"的动作,它是怎么实现 “ Executes a request, asynchronously returning an HTTPResponse”的?

    完整代码在: https://github.com/tornadoweb/tornado/blob/master/tornado/httpclient.py

    def fetch(self, request, callback=None, raise_error=True, **kwargs):
            """Executes a request, asynchronously returning an `HTTPResponse`.
            The request may be either a string URL or an `HTTPRequest` object.
            If it is a string, we construct an `HTTPRequest` using any additional
            kwargs: ``HTTPRequest(request, **kwargs)``
            This method returns a `.Future` whose result is an
            `HTTPResponse`. By default, the ``Future`` will raise an
            `HTTPError` if the request returned a non-200 response code
            (other errors may also be raised if the server could not be
            contacted). Instead, if ``raise_error`` is set to False, the
            response will always be returned regardless of the response
            code.
            If a ``callback`` is given, it will be invoked with the `HTTPResponse`.
            In the callback interface, `HTTPError` is not automatically raised.
            Instead, you must check the response's ``error`` attribute or
            call its `~HTTPResponse.rethrow` method.
            """
            if self._closed:
                raise RuntimeError("fetch() called on closed AsyncHTTPClient")
            if not isinstance(request, HTTPRequest):
                request = HTTPRequest(url=request, **kwargs)
            else:
                if kwargs:
                    raise ValueError("kwargs can't be used if request is an HTTPRequest object")
            # We may modify this (to add Host, Accept-Encoding, etc),
            # so make sure we don't modify the caller's object.  This is also
            # where normal dicts get converted to HTTPHeaders objects.
            request.headers = httputil.HTTPHeaders(request.headers)
            request = _RequestProxy(request, self.defaults)
            future = TracebackFuture()
            if callback is not None:
                callback = stack_context.wrap(callback)
    
                def handle_future(future):
                    exc = future.exception()
                    if isinstance(exc, HTTPError) and exc.response is not None:
                        response = exc.response
                    elif exc is not None:
                        response = HTTPResponse(
                            request, 599, error=exc,
                            request_time=time.time() - request.start_time)
                    else:
                        response = future.result()
                    self.io_loop.add_callback(callback, response)
                future.add_done_callback(handle_future)
    
            def handle_response(response):
                if raise_error and response.error:
                    future.set_exception(response.error)
                else:
                    future.set_result(response)
            self.fetch_impl(request, handle_response)
            return future
    
    def fetch_impl(self, request, callback):
            raise NotImplementedError()
    
    6 条回复    2016-01-06 23:50:35 +08:00
    phithon
        1
    phithon  
       2016-01-05 20:26:06 +08:00   ❤️ 1
    ![]( )
    impl 看名字应该是个接口,等着子类去实现的。
    你可以看 AsyncHTTPClient 的 `__new__` 方法,实际上执行 `new AsyncHTTPClient()` ,其返回的对象是一个 SimpleAsyncHTTPClient 实例, SimpleAsyncHTTPClient 是对 AsyncHTTPClient 的实现。
    所以,最后执行的 fetch_impl 方法,实际上在这里:
    https://github.com/tornadoweb/tornado/blob/6b312f7a73dc78de4b1be2a71a5bbfa51acc725d/tornado/simple_httpclient.py#L122
    kidlj
        2
    kidlj  
    OP
       2016-01-05 20:29:09 +08:00 via Android
    @phithon 非常感谢!
    sujin190
        3
    sujin190  
       2016-01-05 20:45:30 +08:00
    https://github.com/tornadoweb/tornado/blob/master/tornado/httpclient.py#L148
    这设置了生成实例时是哪个实现的实例
    sujin190
        4
    sujin190  
       2016-01-05 20:47:41 +08:00
    不过比较坑的是 SimpleAsyncHTTPClient 本身不支持连接池,每次都是打开新连接,除此之外还有一个实现是 CurlAsyncHTTPClient ,支持连接池,不过在 pypy 下似乎会出现栈溢出错误
    kidlj
        5
    kidlj  
    OP
       2016-01-05 21:33:08 +08:00 via Android
    @sujin190 谢谢。我的疑问得到了解答。
    mqingyn616
        6
    mqingyn616  
       2016-01-06 23:50:35 +08:00
    @sujin190 这个问题有一个比较靠谱的解决方案,可以看 github 上的这个 pr , https://github.com/tornadoweb/tornado/pull/1622

    给 SimpleAsyncHTTPClient 增加连接池,亲测可用。
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   3674 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 26ms · UTC 10:41 · PVG 18:41 · LAX 02:41 · JFK 05:41
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.