📅 最后修改于: 2020-11-01 04:22:50 🧑 作者: Mango
Logstash在输入和输出之间的管道中间使用过滤器。 Logstash的过滤器用于度量和创建诸如Apache-Access之类的事件。许多过滤器插件用于管理Logstash中的事件。在这里，以Logstash聚合过滤器为例，我们正在过滤数据库中每个SQL事务的持续时间并计算总时间。
使用Logstash-plugin实用程序安装聚合筛选器插件。 Logstash插件是Logstash中bin文件夹中Windows的批处理文件。
>logstash-plugin install logstash-filter-aggregate
在此配置中，您可以看到三个’if’语句，用于初始化，递增和生成事务的总持续时间，即sql_duration 。聚合插件用于添加在输入日志的每个事件中都存在的sql_duration。
input {
file {
path => "C:/tpwork/logstash/bin/log/input.log"
}
}
filter {
grok {
match => [
"message", "%{LOGLEVEL:loglevel} -
%{NOTSPACE:taskid} - %{NOTSPACE:logger} -
%{WORD:label}( - %{INT:duration:int})?"
]
}
if [logger] == "TRANSACTION_START" {
aggregate {
task_id => "%{taskid}"
code => "map['sql_duration'] = 0"
map_action => "create"
}
}
if [logger] == "SQL" {
aggregate {
task_id => "%{taskid}"
code => "map['sql_duration'] ||= 0 ;
map['sql_duration'] += event.get('duration')"
}
}
if [logger] == "TRANSACTION_END" {
aggregate {
task_id => "%{taskid}"
code => "event.set('sql_duration', map['sql_duration'])"
end_of_task => true
timeout => 120
}
}
}
output {
file {
path => "C:/tpwork/logstash/bin/log/output.log"
}
}
我们可以使用以下命令运行Logstash。
>logstash –f logstash.conf
以下代码块显示了输入日志数据。
INFO - 48566 - TRANSACTION_START - start
INFO - 48566 - SQL - transaction1 - 320
INFO - 48566 - SQL - transaction1 - 200
INFO - 48566 - TRANSACTION_END - end
如配置文件中所指定，记录器所在的最后一个’if’语句-TRANSACTION_END，它打印总事务时间或sql_duration。这已在output.log中以黄色突出显示。
{
"path":"C:/tpwork/logstash/bin/log/input.log","@timestamp": "2016-12-22T19:04:37.214Z",
"loglevel":"INFO","logger":"TRANSACTION_START","@version": "1","host":"wcnlab-PC",
"message":"8566 - TRANSACTION_START - start\r","tags":[]
}
{
"duration":320,"path":"C:/tpwork/logstash/bin/log/input.log",
"@timestamp":"2016-12-22T19:04:38.366Z","loglevel":"INFO","logger":"SQL",
"@version":"1","host":"wcnlab-PC","label":"transaction1",
"message":" INFO - 48566 - SQL - transaction1 - 320\r","taskid":"48566","tags":[]
}
{
"duration":200,"path":"C:/tpwork/logstash/bin/log/input.log",
"@timestamp":"2016-12-22T19:04:38.373Z","loglevel":"INFO","logger":"SQL",
"@version":"1","host":"wcnlab-PC","label":"transaction1",
"message":" INFO - 48566 - SQL - transaction1 - 200\r","taskid":"48566","tags":[]
}
{
"sql_duration":520,"path":"C:/tpwork/logstash/bin/log/input.log",
"@timestamp":"2016-12-22T19:04:38.380Z","loglevel":"INFO","logger":"TRANSACTION_END",
"@version":"1","host":"wcnlab-PC","label":"end",
"message":" INFO - 48566 - TRANSACTION_END - end\r","taskid":"48566","tags":[]
}