Extending and Customizing Percona Monitoring and Management

1 . Extending and Customizing PMM Michael Coburn Percona 

2 .Michael Coburn ● Product Manager for PMM (as well as for Percona Toolkit) ● At Percona for 6 years across multiple MySQL roles ○ Principal Architect, Managing Consultant, Technical Account Manager 2 

3 .What is PMM ● Free, Open Source database troubleshooting and performance optimization platform for MySQL, MongoDB, and PostgreSQL ○ We also support: ■ ProxySQL ■ Amazon RDS MySQL & Aurora MySQL ■ Remote MySQL & PostgreSQL instances ● Runs in your secure environment (this is not a SaaS product!) and on your equipment ● Secured with SSL between client and server 3 

4 .Customizing Dashboards Click to add text 

5 .Grafana in a Nutshell ● Open Source data visualisation tool ● Dashboards are just JSON ● Popular datasources ○ Prometheus ○ CloudWatch ○ Graphite ○ Elasticsearch ● Templated Variables ○ Define your graph metrics, and let the hosts get filled in automatically ○ GREAT for large, dynamic environments where hosts are considered ephemeral 5 

6 .Dashboard Example - MySQL Overview 6 

7 .Editing a Dashboard ● Enter into Edit mode ● Review Metrics formulas & define graph characteristics 7 

8 .Collect Metrics from anywhere Click to add text 

9 .External Exporters ● Prometheus is a very active community, hundreds of exporters exist ● First configure an Exporter to run, then notify PMM to scrape the new Exporter ● pmm-admin add external:service --service-port=9187 postgresql 9 

10 .Adding Dashboards ● Create your own! ● Or use one from grafana.com/dashboards and edit as needed ○ Datasource name (change to Prometheus) ○ Edit instance variables to match PMM variable names (i.e. $host when most grafana.com Dashboards use $instance) 10 

11 .Custom Queries Available for MySQL and PostgreSQL 

12 .Custom Queries ● MySQL and PostgreSQL support the concept of Custom Queries - run SELECT statements in order to create Prometheus metric series that you can then plot on a graph ● Configuration is enabled by default and executed by the Exporter ● Runs every 60 seconds 12 

13 .Users table - MySQL view 13 

14 .Users table - queries-mysqld.yml ● Configuration is stored in /usr/local/pmm-client/ 14 

15 .Users table - Advanced Data Exploration 15 

16 .Customizing Exporter Options 

17 .Exporter Options ● Exporters have multiple collectors - specific code to collect specific metrics from the service being monitored ● Percona balances maximum data collection with minimum impact on the observed system ● This leads to some Exporter collectors being disabled 17 

18 .node_exporter Example ● 48 total collectors: 32 are enabled by default, 16 disabled ● We want to enable monitoring of NTP ○ pmm-admin remove linux:metrics ○ pmm-admin add linux:metrics -- --collectors.enabled=...,ntp ○ ps aux | grep node_exporter 18 

19 .Using Grafana Alerting Click to add text 

20 .Grafana Alerting ● Rules are added to graphs that react based on defined thresholds ● Alerts are sent to pre-defined Notification Channels ○ You can have multiple channels defined 20 

21 .Grafana Alerting 21 

22 .Grafana Notification Channels ● Defines how you will learn about new alerts ● Over a dozen are available: ○ Email ○ Slack ○ PagerDuty ○ AlertManager ○ ... 22 

23 .Using MySQL Data Source Click to add text 

24 .MySQL Data Source ● Grafana feature which PMM inherits from upstream ● Results of a SELECT statement can draw graphs ● Our example is using Percona Production data 24 

25 .PMM Versions over Time 25 

26 .Do's and Don'ts Click to add text 

27 .PMM Best Practices ● Grafana Alerting works great for < 50 instances ○ over that it becomes prohibitive to manage the variable -> string mappings ● Don't edit Percona-provided Dashboards - your changes will be overwritten at next upgrade! ○ Instead copy the dashboard and edit your copy, and it will survive upgrades ● Send us your ideas, and even better, your Dashboards ○ Percona loves Pull Requests! Your changes can make it into a future release ● Participate in the percona.com/forums ! 27 

28 .Rate My Session 28 

29 .29