登录 注册

es 集群以及 es-head 插件的安装

本贴最后更新于 2532 天前，其中的信息可能已经时移世异

elasticserach 安装

@(ELK 日志系统)

一、首先安装 supervisor

supervisor 是 linux 用于管理后台程序的守护进程、方便管理集群

二、安装 es

  • 下载最新版的 es 官网地址
  • 准备好两台服务器(能够互相 ping 通)
    • 10.66.30.221
    • 10.66.2.90

es 配置文件示例

# ======================== Elasticsearch Configuration =========================
  #
  # NOTE: Elasticsearch comes with reasonable defaults for most settings.
  #       Before you set out to tweak and tune the configuration, make sure you
  #       understand what are you trying to accomplish and the consequences.
  #
  # The primary way of configuring a node is via this file. This template lists
  # the most important settings you may want to configure for a production cluster.
  #
  # Please consult the documentation for further information on configuration options:
  # https://www.elastic.co/guide/en/elasticsearch/reference/index.html
  #
  # ---------------------------------- Cluster -----------------------------------
  #
  # Use a descript ive name for your cluster:
  #
  cluster.name: es-cluster
  #
  # ------------------------------------ Node ------------------------------------
  #
  # Use a descriptive name for the node:
  #
  node.name: node-2
  #
  # Add custom attributes to the node:
  #
  #node.attr.rack: r1
  #
  # ----------------------------------- Paths ------------------------------------
  #
  # Path to directory where to store the data (separate multiple locations by comma):
  #
  path.data: /home/es/path/to/data
  #
  # Path to log files:
  #
  path.logs: /home/es/path/to/logs
  bootstrap.memory_lock: true
  #
  # ----------------------------------- Memory -----------------------------------
  #
  # Lock the memory on startup:
  #
  #bootstrap.memory_lock: true
  #
  # Make sure that the heap size is set to about half the memory available
  # on the system and that the owner of the process is allowed to use this
  # limit.
  #
  # Elasticsearch performs poorly when the system is swapping the memory.
  #
  # ---------------------------------- Network -----------------------------------
  #
  network.host: 10.66.2.90
  #
  # Set a custom port for HTTP:
  #
  http.port: 9200
  transport.tcp.port: 9300
  transport.tcp.compress: true
  discovery.zen.ping.unicast.hosts:
          - 10.6.2.90
          - 10.66.30.221
  #
  # For more information, consult the network module documentation.
  #
  # --------------------------------- Discovery ----------------------------------
  #
  # Pass an initial list of hosts to perform discovery when new node is started:
  # The default list of hosts is ["127.0.0.1", "[::1]"]
  #
  #discovery.zen.ping.unicast.hosts: ["host1", "host2"]
  #
  # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
  #
  discovery.zen.minimum_master_nodes: 2
  http.cors.enabled: true
  http.cors.allow-origin: "*"
  #
  # For more information, consult the zen discovery module documentation.
  #
  # ---------------------------------- Gateway -----------------------------------
  #
  # Block initial recovery after a full cluster restart until N nodes are started:
  #
  #gateway.recover_after_nodes: 3
  #
  # For more information, consult the gateway module documentation.
  #
  # ---------------------------------- Various -----------------------------------
  #
  # Require explicit names when deleting indices:
  #
  #action.destructive_requires_name: true

es 启动常见的错误

问题 1：

$ ./elasticsearch
...
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
[2016-10-31T04:55:45,240][INFO ][o.e.n.Node               ] [vJDcSkt] stopping ...
[2016-10-31T04:55:45,249][INFO ][o.e.n.Node               ] [vJDcSkt] stopped
[2016-10-31T04:55:45,249][INFO ][o.e.n.Node               ] [vJDcSkt] closing ...
[2016-10-31T04:55:45,257][INFO ][o.e.n.Node               ] [vJDcSkt] closed

如图所示有两个错误

错误 1 的解决办法
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]

修改/etc/security/limits.conf 文件，添加或修改如下行：

退出当前账号，然后重新登录

*      hard    nofile    65536
*      soft    nofile    65536

错误 2 的解决办法

max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

切换到root用户修改配置sysctl.conf
vi /etc/sysctl.conf
添加下面配置：
vm.max_map_count=262144
并执行命令：
sysctl -p
然后，重新启动elasticsearch，即可启动成功。

问题 2：

max number of threads [1024] for user [root] is too low, increase to at least [2048]

解决办法

切换到 root 用户，进入 limits.d 目录下修改配置文件。
vim /etc/security/limits.d/90-nproc.conf

修改如下内容：

* soft nproc 1024

修改为

* soft nproc 2048

问题 3：

ERROR: bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

原因：
这是在因为 Centos6 不支持 SecComp，而 ES5.2.0 默认 bootstrap.system_call_filter 为 true 进行检测，所以导致检测失败，失败后直接导致 ES 不能启动。
解决：
在 elasticsearch.yml 中配置 bootstrap.system_call_filter 为 false，注意要在 Memory 下面:

bootstrap.memory_lock: false
bootstrap.system_call_filter: false

问题 4：elasticsearch 索引问题

logstash 传入 es 的序列必须全为小写，否则会报 400 的错误

解决：

直接在 filebeat.yml 中设置 index

output.logstash:
# The Logstash hosts
hosts: ["10.66.140.37:5044"]
worker: 1
loadbalance: true
index: "opennbk-filebeat"

####问题 5：elasticsearch 使用 supervisord 启动时需要切换用户到非 root 启动问题

ES 不能用 root 账户启动,否则会抛出异常 但是使用 supervisord -c ..supervisor.conf 启动的时候会出现一下错误

解决:

这个时候我们可以吧 supervisor.conf 文件中的 program 选项中的 user (谁来启动)注释,然后修改 program 选项中的 command(启动命令) 为:su -c "ES 启动脚本的目录" 非 root 用户.

问题 6

ERROR: [1] bootstrap checks failed

[1]: memory locking requested for elasticsearch process but memory is not locked

修改/etc/security/limits.conf 文件，添加或修改如下行：

退出当前账号，然后重新登录

*   soft    memlock unlimited
*   soft    memlock unlimited

三、supervisor 配置 es

[program: es_node2]
command=/home/es/elasticsearch-6.0.0/bin/elasticsearch
numprocs=1
user=es
autostart=true
autorestart=true

es 插件 head 安装

1.首先安装 nodejs

2.全局安装 grunt

npm install -g grunt-cli

3.克隆 head 工程

git clone https://github.com/mobz/elasticsearch-head

4.修改 head 目录下的 Gruntfile.js 文件

添加 hostname

 connect: {
                          server: {
                                  options: {
                                          port: 9100,
                                          hostname: '*',
                                          base: '.',
                                          keepalive: true
                                  }
                          }
                  }

修改 head 目录下的 app.js 文件。
vim/_site/app.js

 app.App = ui.AbstractWidget.extend({
                 defaults: {
                         base_uri: null
                 },
                 init: function(parent) {
                         this._super();
                         this.prefs = services.Preferences.instance();
                         this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://10.66.2.90:9200" || "http://10.66.30.221:9200";
                         if( this.base_uri.charAt( this.base_uri.length - 1 ) !== "/" ) {
                                 // XHR request fails if the URL is not ending with a "/"
                                 this.base_uri += "/";
                         }
                         if( this.config.auth_user ) {

修改 es 目录下的 elasticsearch.yml 文件

http.cors.enabled: true
http.cors.allow-origin: "*"

grunt server 启动

  • Elasticsearch

    Elasticsearch 是一个基于 Lucene 的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于 RESTful 接口。Elasticsearch 是用 Java 开发的，并作为 Apache 许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。

    117 引用 • 99 回帖 • 210 关注
  • es集群
    1 引用
  • es-head插件
    1 引用
昆明 位置
1.9k 11 784 114 148 70 6 43 739

相关帖子

回帖
es 集群以及 es-head 插件的安装

欢迎来到这里！

我们正在构建一个小众社区，大家在这里相互信任，以平等 • 自由 • 奔放的价值观进行分享交流。最终，希望大家能够找到与自己志同道合的伙伴，共同成长。

注册 关于
请输入回帖内容 ...
yjq14114
昆明
回帖
17
 帖子
4
 积分
790