PingCAP-Infra-Meetup-101-wangpenghan-log-observability

下载 0

TiDB

发布于

4106

人观看

信息技术

本次分享王鹏翰老师首先阐述了 Log 和 observability 是什么，以及建立可观察性（observability）的三个主要手段：logging, metrics, tracing。然后对 Filebeat 的设计架构，实现细节，以及如何对 filebeat 进行合理的监控进行了介绍。最后分享了在生产环境中 50TB/day 日志传输量下的 filebeat 调优细节。

展开查看详情

1 .Log, Observability and Filebeat Dashbase 王鹏翰

2 .

3 .Log ？

4 . Raft is a con mana sensu ging a s algo replic r ithm ated l for og. o ra Log ？ r Ap pl Au E — ic AS at AB T io DA n E Lo IS TH g OG Binlog E L TH

5 . “A log is perhaps the simplest possible storage abstraction. It is an append-only, totally-ordered sequence of records ordered by time.” – Jay Kreps [1] [1] https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

6 .Log Immutable Order by time Timestamp + Data

7 .Observability ?

8 .long long time ago ……

9 .Log with Grep

10 . Monitor what’s broken, and why?

11 .Monitor StatsD Graphite Count Timing Gauges Sets ——— Samping

12 .

13 .

14 .Monolithic MicroService

15 .Cloud and Cloud Native

16 .

17 .

18 .https://peter.bourgon.org/blog/2017/02/21/metrics-tracing-and-logging.html

19 .LOG

20 .Trace

21 . Gauge Counter Histogram Metric

22 . Spec OpenMetric OpenTracing OpenLog ? Not exists … Elastic Common Schema maybe a little like openLog

23 .TSDB Prometheus InﬂuxDB OpenTSDB

24 . Log Logstash Elasticsearch Beats [Filebeat] Loki Fluent FluentBit

25 . Trace Zipkin Elastic apm agent Jeager Skywalking

26 . Filebeat Lightweight Shipper for Logs

27 .Head First Log Shipper Send log stable and fast to a search engine

28 .

29 .

0 点赞

0 收藏

0下载