详解Prometheus 抓取 nginx 访问日志的指标

gem install fluentd
gem install fluent-plugin-prometheus

<source>
  @type tail
  path /var/log/nginx/access.log
  pos_file /var/log/td-agent/nginx-access.log.pos
  tag nginx.access
  <parse>
    @type nginx
  </parse>
</source>
<filter nginx.access>
  @type grep
  <regexp>
    key status
    pattern ^4|5[0-9][0-9]$
  </regexp>
</filter>
<match nginx.access>
  @type prometheus
  <metric>
    name nginx_http_status_total
    type counter
    desc Nginx HTTP status code counter
    keys status
    <labels>
      status ${status}
    </labels>
  </metric>
  <metric>
    name nginx_http_request_bytes_total
    type counter
    desc Total bytes received by the Nginx server
    key bytes
  </metric>
</match>
<source>
  @type prometheus_monitor
  <labels>
    tag nginx.access
  </labels>
</source>
<source>
  @type prometheus_output_monitor
  <labels>
    tag nginx.access
  </labels>
</source>
<match **>
  @type stdout
</match>

- **Source**：读取 `/var/log/nginx/access.log` 文件，并解析日志。
- **Filter**：通过正则表达式过滤状态码为 4xx 或 5xx 的日志。
- **Match**：将解析后的日志信息转化为 Prometheus 格式的指标。
- **Prometheus 输出**：启动一个 HTTP 端点，供 Prometheus 抓取这些指标。

fluentd -c /path/to/fluent.conf

scrape_configs:
  - job_name: 'nginx_logs'
    static_configs:
      - targets: ['localhost:24231']