V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
V2EX 提问指南
wico77
V2EX  ›  问与答

elasticsearch 搜索错误

  •  
  •   wico77 · 2016-01-20 16:59:46 +08:00 · 3418 次点击
    这是一个创建于 3231 天前的主题,其中的信息可能已经有所发展或是发生改变。

    服务器配置 16G 内存, 500G 硬盘,分了 8G 给 ES 用。网站搭配 flask+nginx+mongodb+elasticsearch,最近 google webmaster 里老出现 500 和 502 错误,见前一个帖子 http://v2ex.com/t/247625#reply9 ,现在找到原因是 ES 全文搜索有错误,我在 ES 日志里看到不少这种搜索错误,请问是什么原因造成的?内存还是小吗?
    [2016-01-19 23:42:38,283][DEBUG][action.search.type ] [Razorback] [exta][2], node[7B-RkEI5Rq2uLL2yqg3CPA], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@5a7adcd4] lastShard [true]
    org.elasticsearch.search.SearchParseException: [exta][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"query_string": {"query": "gymnast torture\"}}}]]
    at org.elasticsearch.search.SearchService.parseSource(SearchService.java:721)
    at org.elasticsearch.search.SearchService.createContext(SearchService.java:557)
    at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:529)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:291)
    at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
    at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
    at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
    Caused by: org.elasticsearch.index.query.QueryParsingException: [exta] Failed to parse query [gymnast torture]
    at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:250)
    at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:302)
    at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:382)
    at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:281)
    at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:276)
    at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
    at org.elasticsearch.search.SearchService.parseSource(SearchService.java:705)
    ... 9 more
    Caused by: org.apache.lucene.queryparser.classic.ParseException: Cannot parse 'gymnast torture\': Lexical error at line 1, column 17. Encountered: <EOF> after : ""
    at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:137)
    at org.apache.lucene.queryparser.classic.MapperQueryParser.parse(MapperQueryParser.java:887)
    at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:233)
    ... 15 more
    Caused by: org.apache.lucene.queryparser.classic.TokenMgrError: Lexical error at line 1, column 17. Encountered: <EOF> after : ""
    at org.apache.lucene.queryparser.classic.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1133)
    at org.apache.lucene.queryparser.classic.QueryParser.jj_scan_token(QueryParser.java:601)
    at org.apache.lucene.queryparser.classic.QueryParser.jj_3R_2(QueryParser.java:484)
    at org.apache.lucene.queryparser.classic.QueryParser.jj_3_1(QueryParser.java:491)
    at org.apache.lucene.queryparser.classic.QueryParser.jj_2_1(QueryParser.java:477)
    at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:228)
    at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:214)
    at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:172)
    at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:127)
    ... 17 more

    10 条回复    2016-01-20 20:24:40 +08:00
    windfarer
        1
    windfarer  
       2016-01-20 17:06:59 +08:00
    错误很明显嘛
    Parse Failure [Failed to parse source [{"query": {"query_string": {"query": "gymnast torture\"}}}]]
    Cannot parse 'gymnast torture\': Lexical error at line 1, column 17. Encountered: <EOF> after : ""

    传过去的查询 DSL 不正确
    Zzzzzzzzz
        2
    Zzzzzzzzz  
       2016-01-20 17:07:05 +08:00
    报错很明显, 你传的 json 有问题啊.....

    org.elasticsearch.search.SearchParseException: [exta][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"query_string": {"query": "gymnast torture\"}}}]]
    wico77
        3
    wico77  
    OP
       2016-01-20 17:11:56 +08:00
    @Zzzzzzzzz 你的意思是 query 查询的词后面有个'\'?
    我使用 pyes ,获取查询词之后的语句是这样的:
    bt = QueryStringQuery(keyword.lower())
    s = Search(bt)
    videos =conn.search(s, size=32, indices="exta",start=(page-1)*32)
    请问传过去的查询如何修正?
    wico77
        4
    wico77  
    OP
       2016-01-20 17:29:30 +08:00
    @windfarer 请指点如何修改
    windfarer
        5
    windfarer  
       2016-01-20 17:52:40 +08:00
    @wico77 pyes 我没用过啊,那几个 python 的库选来选去,我们最后选了官方的 elasticsearch-dsl-py
    Zzzzzzzzz
        6
    Zzzzzzzzz  
       2016-01-20 18:06:53 +08:00
    @wico77 我没复现你的问题, 我猜是不是低版本 simplejson 导致的, 印象里以前 simplejson 确实有过 slash 问题, 先升级一下重启应用试试.
    wico77
        7
    wico77  
    OP
       2016-01-20 18:51:14 +08:00
    @Zzzzzzzzz pip install simplejson
    Collecting simplejson
    /usr/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
    InsecurePlatformWarning
    Downloading simplejson-3.8.1.tar.gz (76kB)
    100% |████████████████████████████████| 77kB 520kB/s
    Building wheels for collected packages: simplejson
    Running setup.py bdist_wheel for simplejson
    Stored in directory: /root/.cache/pip/wheels/4e/25/10/2ff5f7347927f4816794414d599a5383c7f8dd9a9ff729af28
    Successfully built simplejson
    Installing collected packages: simplejson
    Successfully installed simplejson-3.8.1
    我安装了下 simplejson ,难道是默认没安装 simplejson ?
    Zzzzzzzzz
        8
    Zzzzzzzzz  
       2016-01-20 19:02:50 +08:00
    @wico77 从 2.6 开始引入的自带 json 包就是 simplejson, 不过考虑到最新的 simplejson 包会做一些优化和解决一些 bug 一般都优先导入 simplejson 包, 没有再回退到 python 自带的 json 库, 我看 pyes 也是这样做的, 用了最新的 simplejson 还出错么?
    wico77
        9
    wico77  
    OP
       2016-01-20 19:42:44 +08:00
    @Zzzzzzzzz 等明天看 ES 的 log 再回复你
    wico77
        10
    wico77  
    OP
       2016-01-20 20:24:40 +08:00
    @Zzzzzzzzz 用了最新的 simplejson 如果 keyword 末端有\也会出错,我把这些 escape 去掉就好了
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   5873 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 26ms · UTC 01:58 · PVG 09:58 · LAX 17:58 · JFK 20:58
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.