logstash未成功将数据发送到外部弹性搜索

以下是logstash.err的错误:

法拉第::连接失败:文件结束到达/ opt/logst///////////////////////////////////////////////////////////////////////////////////////////vendor/bundle/jruby/1.9/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139 run_request atopt /logstash/vendor/bundle/jruby/1.9/gems/faraday-0.9.0/lib/ faraday/connection.rb:377 perform_request at /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.1/lib/elasticsearch/transport/transport/http/faraday.rb:24在org打电话/jruby/RubyProc.java:271 perform_request at /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.1/lib/elasticsearch/transport/transport/base.rb:187 perform_request at/opt /logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.1/lib/elasticsearch/transport/transport/http/faraday.rb:20 perform_request at /opt/logstash/vendor/bundle/jruby/1.9 /gems/elasticsearch-transport-1.0.1/lib/elasticsearch/transport/client.rb:102 perform_request at/opt/logstash/v endor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.1/lib/elasticsearch/api/namespace/common.rb:21 get_template at /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch- api-1.0.1/lib/elasticsearch/api/actions/indices/get_template.rb:24 template_exists？at /opt/logstash/lib/logstash/outputs/elasticsearch/protocol.rb:132 template_install atopt /logstash/lib/logstash/outputs/elasticsearch/protocol.rb:21在/ opt/logstash/lib/logstash /注册输出/ elasticsearch.rb:259每个在org/jruby/RubyArray.java:1613输出工作者/opt/logstash/lib/logstash/pipeline.rb:220 start_outputs atopt /logstash/lib/logstash/pipeline.rb:152

这是我的输出配置:

output {
        elasticsearch { 
            host => "X.X.X.X"
            port => "9300"
            protocol => "http"
            cluster => "elasticsearch_david"
        }   
    }

没有连接问题,任何想法？

使用tcpdump进一步调查给出:

GET /_template/logstash HTTP/1.1
User-Agent: Faraday v0.9.0
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
Connection: close

John Petrone.. 5

问题是端口和协议不匹配:

output {
        elasticsearch { 
            host => "X.X.X.X"
            port => "9300"
            protocol => "http"
            cluster => "elasticsearch_david"
        }   
    }

您将协议设置为"http",这将需要端口9200(ES用于http请求的默认端口),但端口设置为9300,这是用于群集间通信的端口,通常与"节点"协议一起使用.

遗憾的是,文档与协议的默认值相矛盾:

协议

Value can be any of: "node", "transport", "http"
There is no default value for this setting.

选择用于与Elasticsearch通信的协议.

'node'协议将作为普通的Elasticsearch节点连接到集群(但不会存储数据).这允许您使用多播发现之类的东西.如果使用节点协议,则必须允许端口9300(或您配置的任何端口)上的双向通信.

"传输"协议将连接到您指定的主机,并且不会在Elasticsearch集群中显示为"节点".这在您无法允许从Elasticsearch集群出站到此Logstash服务器的连接的情况下非常有用.

'http'协议将使用Elasticsearch REST/HTTP接口与elasticsearch进行通信.

在与Elasticsearch交谈时,所有协议都将使用批量请求.

java/jruby下的默认协议设置是"node".非java rubies上的默认协议是"http"

您最好的选择是将协议设置设置为"node","http"或"transport"之一,具体取决于您要执行的操作,并让logstash为您设置适当的端口:

output {
    elasticsearch { 
    host => "X.X.X.X"
    protocol => "http"
    cluster => "elasticsearch_david"
} 

请参阅http://logstash.net/docs/1.4.1/outputs/elasticsearch#protocol