我正在尝试使用logstash回填一些过去的Apache访问日志数据,因此我需要将事件@timestamp设置为出现在日志消息中的日期.这是我当前的logstash配置:
input { tcp { type => "access_log" port => 9293 } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } date { # Try to pull the timestamp from the 'timestamp' field (parsed above with # grok). The apache time format looks like: "18/Aug/2011:05:44:34 -0700" locale => "en" timezone => "America/New_York" match => { "timestamp" => "dd/MMM/yyyy:HH:mm:ss Z" } add_tag => [ "tsmatch" ] } } output { stdout { codec => rubydebug } }
但是,日期过滤器似乎不会更新事件@timestamp,即使正确捕获了Apache时间戳并且正则表达式应该匹配它.输出数据如下所示:
{ "message" => "56.116.21.231 - - [20/Nov/2013:22:47:08 -0500] \"GET /xxxx/1.305/xxxx/xxxx.zip HTTP/1.1\" 200 33002333 \"-\" \"xxxxx/3.0.3 CFNetwork/609.1.4 Darwin/13.0.0\"", "@timestamp" => "2013-12-01T12:54:27.920Z", "@version" => "1", "type" => "access_log", "host" => "0:0:0:0:0:0:0:1%0:51045", "clientip" => "56.116.21.231", "ident" => "-", "auth" => "-", "timestamp" => "20/Nov/2013:22:47:08 -0500", "verb" => "GET", "request" => "/xxxx/1.305/xxxx/xxxx.zip", "httpversion" => "1.1", "response" => "200", "bytes" => "33002333", "referrer" => "\"-\"", "agent" => "\"xxxxx/3.0.3 CFNetwork/609.1.4 Darwin/13.0.0\"", "tags" => [ [0] "tsmatch" ] }
关于什么可能出错的任何想法?
我正在使用logstash-1.2.2 flatjar.
好的,我发现了问题,我在匹配操作中使用了错误的语法:
match => ["timestamp","dd/MMM/yyyy:HH:mm:ss Z"]
不
match => {"timestamp"=>"dd/MMM/yyyy:HH:mm:ss Z"}