Elasticsearch权威指南（中文版）

下载 27

问此恨、何时是已

发布于

3662

人观看

#信息技术

Elasticsearch是一个实时分布式搜索和分析引擎。它让你以前所未有的速度处理大数据成为可能。它用于全文搜索、结构化搜索、分析以及将这三者混合使用： 1.维基百科使用Elasticsearch提供全文搜索并高亮关键字，以及输入实时搜索（search-as-you-type）和搜索纠错（did-you-mean）等搜索建议功能。 2.美国卫报使用Elasticsearch结合用户日志和社交网络数据提供给他们的编辑以实时的反馈，以便及时了解公众对新发表的文章回应。 3.StackOverflow结合全文搜索与地理位置查询，以及more-like-this功能来找到相关的问题和答案。 4.Github使用Elasticsearch检索1300亿行的代码

展开查看详情

1 .

2 .Table of Contents 1. Introduction 2. 门 i. ii. iii. API iv. v. vi. vii. viii. 结 ix. x. 结语 3. i. ii. iii. iv. 转 v. 扩 vi. 扩 vii. 应对 4. i. ii. iii. 获 iv. v. vi. 创 vii. 删 viii. ix. x. Mget xi. xii. 结语 5. 删查 i. ii. iii. 删 iv. 检 v. vi. 请 vii. 6. i. ii. 类 iii. 页 iv. 查询 7. i. 类 ii. 值对 iii. iv. v.

3 . vi. 类 8. 结构查询 i. 请查询 ii. 结构查询 iii. 查询过滤 iv. 查询 v. 过滤查询 vi. 验证查询 vii. 结语 9. i. ii. iii. iv. 10. i. 查询阶 ii. 阶 iii. 选项 iv. 扫滚 11. i. 创删 ii. 设 iii. iv. 义 v. vi. 对 vii. source viii. all ix. ID x. 动态 xi. 义动态 xii. 认 xiii. xiv. 别 12. i. ii. 动态 iii. 实时 iv. 变 v. 13. 结构 i. 查询值 ii. 组过滤 iii. 查询值 iv. v. 围 vi. 处 Null 值 vii. 缓 viii. 过滤顺 14. i. 查询 ii. 词查询 iii. 组查询 iv. 尔 v. vi.

4 .vii. 联

5 .Elasticsearch 权阅读 Elasticsearch权书 Elasticsearch the definitive guide clinton gormley zachary tong 译 Looly 译 @iridiumcao @cvvnx1 @conan007ai @sailxjx @wxlfight @xieyunzi @xdream86 @conan007ai @williamzhao @dingusxp 谢译伙们~~ 邮 loolly@gmail.com @ 项 https://github.com/looly/elasticsearch-definitive-guide-cn http://git.oschina.net/loolly/elasticsearch-definitive-guide-cn 阅读

6 .http://es.xiaoleilu.com/ 说 Elasticsearch 简单译时统习语较译欢 issue: github git@osc 译键约 index -> type -> 类 token -> filter -> 过滤 analyser -> Pull Request 对Pull Request @numbbbbb The Swift Programming Language 协谢 1. fork 项 2. fork过项项 clone 3. 运 git remote add looly git@github.com:looly/elasticsearch-definitive-guide-cn.git 库为远库 4. 运 git pull looly master 5. 译 6. commit push 库 git push origin master 7. 陆Github 页 pull request 钮击说 1~3 执译须执 4 库这样执 5~7 1. 还译 gitbook 经译节陆续 gitbook 2. 为译译贝

7 . 门 Elasticsearch 实时让处为结构这维 Elasticsearch 键输实时 (search-as-you-type) 纠错(did-you- mean) 议卫报 Elasticsearch结户络给们编辑实时馈时众对发应 StackOverflow结查询 more-like-this 问题 Github Elasticsearch检 1300亿码 Elasticsearch 仅业还让 DataDog Klout这样创业变扩 Elasticsearch 笔记运计务处 PB级别 Elasticsearch 项术创统库这经这术实时应对户门槛长这书说经够对这现库识显们够过时间过滤们够进处义词给吗们结吗们进线对实时处吗这 Elasticsearch Elasticsearch 浏览让烂库为库实难查询 Elasticsearch 认识

8 .为懂 Elasticsearch Apache Lucene(TM) 论还专领 Lucene 认为为进库 Lucene 库须 Java 为发语应 Lucene 杂检识 Elasticsearch Java 发 Lucene 为实现过简单 RESTful API 隐 Lucene 杂让变简单过 Elasticsearch 仅仅 Lucene 们还这样实时储实时扩务处 PB级结构结构这务应过简单 RESTful API 语户 Elasticsearch 许值对隐杂论习产环 Elasticsearch Apache 2 license 许费载对Elasticsearch 问题领 Elasticsearch 级这历 Shay Banon 刚结业发伦习师过为给构谱构 Lucene Lucene 较难 Shay Lucene 码 Java 员应发项 “Compass” Shay 这处环实时 Compass库为务 Elasticsearch 现 2010 2 Elasticsearch 经为Github 欢项码贡过300 营Elasticsearch 们边业边发过Elasticsearch 远对 Shay 她谱 ……

9 . Elasticsearch Elasticsearch 运让们 Elasticsearch Java www.java.com elasticsearch.org/download 载 Elasticsearch curl -L -O http://download.elasticsearch.org/PATH/TO/VERSION.zip <1> unzip elasticsearch-$VERSION.zip cd elasticsearch-$VERSION 1. elasticsearch.org/download 获 URL 产环时还 Debian RPM 这 downloads page Puppet module Chef cookbook Marvel Marvel Elasticsearch 监发环费 Sense 户过浏览 Elasticsearch进 Elasticsearch线码带 View in Sense 链击进 Sense 应实 Marvel 须过 Elasticsearch 运码书动 Marvel Elasticsearch 录运载 ./bin/plugin -i elasticsearch/marvel/latest 监过闭Marvel echo 'marvel.agent.enabled: false' >> ./config/elasticsearch.yml 运 Elasticsearch Elasticsearch 经备绪执启动 ./bin/elasticsearch 护进运 -d 另终进测试 curl 'http://localhost:9200/?pretty'

10 . { "status": 200, "name": "Shrunken Bones", "version": { "number": "1.4.0", "lucene_version": "4.10" }, "tagline": "You Know, for Search" } 这说 ELasticsearch 经启动运们实验节节 (node) 运 Elasticsearch实 (cluster) 组 cluster.name 节们协转扩节组 cluster.name 认值这样启动节络另过 config/ 录 elasticsearch.yml 启ELasticsearch 这 Elasticsearch 运 Ctrl-C 键终调 shutdown API 闭 curl -XPOST 'http://localhost:9200/_shutdown' 查 Marvel Sense Marvel 为监浏览过访问 http://localhost:9200/_plugin/marvel/ Marvel 过击 dashboards 单访问Sense 发访问 http://localhost:9200/_plugin/marvel/sense/

11 . Elasticsearch Elasticsearch Java Java API Elasticsearch为Java 户两户节户 (node client) 节户节 (none data node) 换储够转发请对应节传输户 (Transport client) 这轻传输户够发请远简单转发请给节两 Java 户过9300 Elasticsearch传输协议(Elasticsearch Transport Protocol) 节间过9300 进节组 TIP Java 户 Elasticsearch 须节则们识别 Java API 请查节 Java API HTTP协议 JSON为 RESTful API 语 RESTful API 过9200 Elasticsearch进欢 WEB 户实见过 curl Elasticsearch NOTE Elasticsearch 语户 ——Groovy Javascript .NET PHP Perl Python Ruby ——还户这 Elasticsearch发请组 HTTP请样 curl -X<VERB> '<PROTOCOL>://<HOST>/<PATH>?<QUERY_STRING>' -d '<BODY>' VERB HTTP GET , POST , PUT , HEAD , DELETE PROTOCOL http https协议 Elasticsearch https 时 HOST Elasticsearch 节节 localhost PORT Elasticsearch HTTP 务认为9200 QUERY_STRING 选查询请 ?pretty 请观读 JSON BODY JSON 请请话举说为计们这样 curl -XGET 'http://localhost:9200/_count?pretty' -d ' { "query": { "match_all": {} } }

12 . ' Elasticsearch 类 200 OK HTTP 态码 JSON 响应 HEAD 请请 JSON 响应 { "count" : 0, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 } } 们 HTTP头为们让 curl 显们显 curl -i : curl -i -XGET 'localhost:9200/' 对书们简 curl 请还 curl 请 curl -XGET 'localhost:9200/_count?pretty' -d ' { "query": { "match_all": {} } }' 们简这样 GET /_count { "query": { "match_all": {} } } 实 Sense

13 .应对简单键值时拥杂结构另对组总这对储库这组库现对对应查询时们 Elasticsearch (document oriented) 这储对 (document) 仅仅储还 (index) Elasticsearch 对进过滤这这 Elasticsearch 够执杂 JSON ELasticsearch Javascript对 (JavaScript Object Notation) JSON 为 JSON现经语经为NoSQL领标简洁简单阅读 JSON 户对 { "email": "john@smith.com", "first_name": "John", "last_name": "Smith", "info": { "bio": "Eco-warrior and defender of the weak", "age": 25, "interests": [ "dolphins", "whales" ] }, "join_date": "2014/05/01" } user 对杂结构对义经现 JSON Elasticsearch 对转为 JSON 结构简单 NOTE 语应块结构转换为JSON 语处细节请查 “ serialization ” or “ marshalling ”两处 JSON 块 Elasticsearch 户动为 JSON

14 . 们现进简单绍 (indexing) (search) (aggregations) 过这们让对Elasticsearch 觉们陆续绍术语马们书节讨这风 Elasticsearch 让们员录设们刚 Megacorp 这时资门让们创员录这录进怀实时协够值标签纯检员结构查 30岁员简单杂语(phrase) 结键够图这员们储员员 Elasticsearch 储为 (indexing) 过们应该储哪 Elasticsearch 归类 (type), 这类 (index) 们简单对图类传统库 Relational DB -> Databases -> Tables -> Rows -> Columns Elasticsearch -> Indices -> Types -> Documents -> Fields Elasticsearch (indices) 库类 (types) 类 (documents) (Fields) 义经 (index)这词 Elasticsearch 义 : 词 (index) 传统库库储 index indices indexes 动词储词检查询这 SQL INSERT 键别经传统库为 B-Tree 检 Elasticsearch Lucene (inverted index) 结构达认拥这样们们节详细讨论为创员录们进

15 . 为员 (document) 应员类为 employee employee 类归 megacorp megacorp 储 Elasticsearch 实际这许骤们过执 PUT /megacorp/employee/1 { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests": [ "sports", "music" ] } 们 path: /megacorp/employee/1 说 megacorp employee 类 1 这员 ID 请实 JSON 这员 “John Smith” 25岁欢简单额创义类们够 Elasticsearch 经设让们录员 PUT /megacorp/employee/2 { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests": [ "music" ] } PUT /megacorp/employee/3 { "first_name" : "Douglas", "last_name" : "Fir", "age" : 35, "about": "I like to build cabinets", "interests": [ "forestry" ] }

16 .检现 Elasticsearch 经储们业务够检单员这对 Elasticsearch 说简单们执 HTTP GET请 “ ”—— 类 ID 这们 JSON GET /megacorp/employee/1 响应 John Smith JSON _source { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests": [ "sports", "music" ] } } 们过HTTP GET 检样们 DELETE 删 HEAD 检查们 PUT 简单 GET 请简单—— 轻获让们进尝试东简单们尝试简单员请 GET /megacorp/employee/_search 们 megacorp employee 类们结键 _search ID 响应 hits 组们认 10 结 { "took": 6, "timed_out": false, "_shards": { ... }, "hits": { "total": 3, "max_score": 1, "hits": [ { "_index": "megacorp", "_type": "employee", "_id": "3", "_score": 1, "_source": { "first_name": "Douglas", "last_name": "Fir", "age": 35, "about": "I like to build cabinets", "interests": [ "forestry" ] } },

17 . { "_index": "megacorp", "_type": "employee", "_id": "1", "_score": 1, "_source": { "first_name": "John", "last_name": "Smith", "age": 25, "about": "I love to go rock climbing", "interests": [ "sports", "music" ] } }, { "_index": "megacorp", "_type": "employee", "_id": "2", "_score": 1, "_source": { "first_name": "Jane", "last_name": "Smith", "age": 32, "about": "I like to collect rock albums", "interests": [ "music" ] } } ] } } 响应仅诉们哪这 — 们给户结时让们 “Smith” 员这们轻级这查询 (query string) 为们传递URL 样传递查询语 GET /megacorp/employee/_search?q=last_name:Smith 们请 _search 键查询语传递给 q= 这样为Smith 结 { ... "hits": { "total": 2, "max_score": 0.30685282, "hits": [ { ... "_source": { "first_name": "John", "last_name": "Smith", "age": 25, "about": "I love to go rock climbing", "interests": [ "sports", "music" ] } }, { ... "_source": { "first_name": "Jane", "last_name": "Smith", "age": 32, "about": "I like to collect rock albums", "interests": [ "music" ] } } ] } }

18 . DSL语查询查询过 (ad hoc) 阅简单节 Elasticsearch 查询语 DSL查询(Query DSL), 许构杂查询 DSL(Domain Specific Language 领语 ) JSON请现们这样 “Smith” 查询: GET /megacorp/employee/_search { "query" : { "match" : { "last_name" : "Smith" } } } 这查询结东变们查询 (query string) 为请这请 JSON match 语查询类们杂们让变杂们为“Smith” 员们龄 30岁员们语过滤 (filter), 们执结构 GET /megacorp/employee/_search { "query" : { "filtered" : { "filter" : { "range" : { "age" : { "gt" : 30 } <1> } }, "query" : { "match" : { "last_name" : "smith" <2> } } } } } <1> 这查询间过滤 (range filter), 查龄 30岁 —— gt 为"greater than" 缩 <2> 这查询 match 语 (query) 现语们详细讨论们过滤 (filter) 执间 match 语现们结显 32岁 “Jane Smith” 员 { ... "hits": { "total": 1, "max_score": 0.30685282, "hits": [ { ... "_source": { "first_name": "Jane", "last_name": "Smith", "age": 32, "about": "I like to collect rock albums", "interests": [ "music" ] } } ]

19 . } } 为简单过龄筛选让们尝试级 —— 传统库难实现们欢“rock climbing” 员 GET /megacorp/employee/_search { "query" : { "match" : { "about" : "rock climbing" } } } 们 match 查询 about "rock climbing" 们两 { ... "hits": { "total": 2, "max_score": 0.16273327, "hits": [ { ... "_score": 0.16273327, <1> "_source": { "first_name": "John", "last_name": "Smith", "age": 25, "about": "I love to go rock climbing", "interests": [ "sports", "music" ] } }, { ... "_score": 0.016878016, <2> "_source": { "first_name": "Jane", "last_name": "Smith", "age": 32, "about": "I like to collect rock albums", "interests": [ "music" ] } } ] } } <1><2> 结评认 Elasticsearch 结评对结进谓结评查询显 John Smith about “rock climbing” 为 Jane Smith 现结 “rock” 她 abuot 为 “rock” “climbing” 她 _score John 这释 Elasticsearch 进结 (relevance) Elasticsearch 这传统库为传统库对记录查询

20 . 语们单词这时单词语(phrases) 们查询时 "rock" "climbing" 邻员记录这们 match 查询变为 match_phrase 查询 : GET /megacorp/employee/_search { "query" : { "match_phrase" : { "about" : "rock climbing" } } } 问该查询 John Smith { ... "hits": { "total": 1, "max_score": 0.23013961, "hits": [ { ... "_score": 0.23013961, "_source": { "first_name": "John", "last_name": "Smith", "age": 25, "about": "I love to go rock climbing", "interests": [ "sports", "music" ] } } ] } } 们应欢结 (highlight) 键这样户为这查询 Elasticsearch 让们语 highlight GET /megacorp/employee/_search { "query" : { "match_phrase" : { "about" : "rock climbing" } }, "highlight": { "fields" : { "about" : {} } } } 们运这语时结结 highlight 这 about 标识单词

21 .{ ... "hits": { "total": 1, "max_score": 0.23013961, "hits": [ { ... "_score": 0.23013961, "_source": { "first_name": "John", "last_name": "Smith", "age": 25, "about": "I love to go rock climbing", "interests": [ "sports", "music" ] }, "highlight": { "about": [ "I love to go rock climbing" <1> ] } } ] } } <1> 节阅读

22 . 们还许职员录进 Elasticsearch (aggregations) 许杂统计 SQL GROUP BY 举让们职员兴爱 GET /megacorp/employee/_search { "aggs": { "all_interests": { "terms": { "field": "interests" } } } } 暂时语查询结 { ... "hits": { ... }, "aggregations": { "all_interests": { "buckets": [ { "key": "music", "doc_count": 2 }, { "key": "forestry", "doc_count": 1 }, { "key": "sports", "doc_count": 1 } ] } } } 们两职员对乐兴欢欢运动这预计们实时查询语动态计们 "Smith" 兴爱们语 GET /megacorp/employee/_search { "query": { "match": { "last_name": "smith" } }, "aggs": { "all_interests": { "terms": { "field": "interests" } } } } all_interests 经变查询语 ...

23 . "all_interests": { "buckets": [ { "key": "music", "doc_count": 2 }, { "key": "sports", "doc_count": 1 } ] } 许级汇总让们统计兴职员龄 GET /megacorp/employee/_search { "aggs" : { "all_interests" : { "terms" : { "field" : "interests" }, "aggs" : { "avg_age" : { "avg" : { "field" : "age" } } } } } } 虽这结杂 ... "all_interests": { "buckets": [ { "key": "music", "doc_count": 2, "avg_age": { "value": 28.5 } }, { "key": "forestry", "doc_count": 1, "avg_age": { "value": 35 } }, { "key": "sports", "doc_count": 1, "avg_age": { "value": 25 } } ] } 该结结们兴该兴员现兴额拥 avg_age 显该兴员龄还语觉过这杂处类

24 . 结这简够 Elasticsearch 这为简还 —— 这构级语让觉调问书这问题细节让 Elasticsearch 过

25 . 节们 Elasticsearch 扩务处 PB级们给 Elasticsearch Elasticsearch为设计隐杂 Elasticsearch 统发现运笔记运拥 100 节样 Elasticsearch 隐统杂这层动 (shards) 们节节对负载丢节请应节论节还节缝扩阅读书时 Elasticsearch 补节这节给扩转处储执这节读 —— 懂这 Elasticsearch 这够 Elasticsearch 读们时头阅

26 .现对Elasticsearch Elasticsearch 习轻习 Elasticsearch 检 Elasticsearch 产详细诉Elasticsearch 应输书级专节阐专级别刚这暂时 Elasticsearch 认户预时时顾这节

27 . 补节这 Elasticsearch 环补节这节们释术语 (cluster) 节 (node) (shard) Elasticsearch 扩处这读 —— Elasticsearch 时长时间远转 —— Elasticsearch 过这查阅 Elasticsearch 构扩统扩购买务 (纵扩 (vertical scale or scaling up)) 购买务扩 (horizontal scale or scaling out) Elasticsearch虽获纵扩扩应该过节摊负载对库扩动这设备对说 Elasticsearch 节扩这这这们创 (cluster) 节 (node) (shards) 进扩证时

28 . 们启动单节还这图1 图1 节节 (node) Elasticsearch实 (cluster) 节组们 cluster.name 们协负载节删节时节选举为节 (master), 临时级别变删节节级别变这长时该节为颈节为节们节节为户们够节节节哪节们转发请应节们访问节负责节给户这 Elasticsearch处

29 . Elasticsearch 监统计 (cluster health) 态 green yellow red GET /_cluster/health 运查询这 { "cluster_name": "elasticsearch", "status": "green", <1> "timed_out": false, "number_of_nodes": 1, "number_of_data_nodes": 1, "active_primary_shards": 0, "active_shards": 0, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 0 } <1> status 们兴 status 综标务颜义颜义 green yellow red 节们说 (primary shard) (replica shard) 说这颜态实际环义

5点赞

4收藏

27下载