Prometheus can tell you if your MariaDB is alive, but it can’t tell you why it’s struggling unless you tell it what to look for.
Let’s watch MariaDB get scraped by Prometheus. First, we need the exporter.
# On your MariaDB server
wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.15.0/mysqld_exporter-0.15.0.linux-amd64.tar.gz
tar xvf mysqld_exporter-0.15.0.linux-amd64.tar.gz
cd mysqld_exporter-0.15.0.linux-amd64
Now, set up a user in MariaDB for Prometheus to read from. This user only needs PROCESS and SELECT privileges.
-- In your MariaDB client
CREATE USER 'prometheus'@'localhost' IDENTIFIED BY 'your_strong_password';
GRANT PROCESS, SELECT ON *.* TO 'prometheus'@'localhost';
FLUSH PRIVILEGES;
The exporter needs a .my.cnf file to connect. Create this in the user’s home directory that will run the exporter (e.g., /home/prometheus/.my.cnf).
[client]
user=prometheus
password=your_strong_password
Ensure this file has strict permissions: chmod 600 /home/prometheus/.my.cnf.
Now, start the exporter:
./mysqld_exporter --config.my-cnf=/home/prometheus/.my.cnf --web.listen-address=":9104"
You should see output indicating it’s listening. On your Prometheus server, add a scrape target in prometheus.yml:
scrape_configs:
- job_name: 'mariadb'
static_configs:
- targets: ['your_mariadb_server_ip:9104']
Reload Prometheus. You can now visit http://your_prometheus_server_ip:9090/targets and see your MariaDB exporter as healthy. Querying http://your_mariadb_server_ip:9104/metrics will show you raw data like mysql_global_status_threads_connected.
The real power is in alerting. Let’s say you want to know when MariaDB is overloaded with connections. A common alert is for Threads_connected exceeding a threshold.
# In your Prometheus rules file (e.g., mariadb_rules.yml)
groups:
- name: mariadb.rules
rules:
- alert: HighMariaDBConnections
expr: mysql_global_status_threads_connected > 100
for: 5m
labels:
severity: warning
annotations:
summary: "High number of connections to MariaDB instance {{ $labels.instance }}"
description: "MariaDB instance {{ $labels.instance }} has {{ $value }} active connections, exceeding the threshold of 100."
This rule checks the mysql_global_status_threads_connected metric. If it stays above 100 for 5 minutes, it fires a warning alert. The {{ $labels.instance }} and {{ $value }} are templated to give context in the alert notification.
Another critical alert is for replication lag. You’ll need to enable log_bin and binlog_format = ROW in your MariaDB configuration (my.cnf or mariadb.conf.d/50-server.cnf).
[mysqld]
log_bin = /var/log/mysql/mysql-bin.log
binlog_format = ROW
After restarting MariaDB, you can create an alert for replication lag:
# In your Prometheus rules file
groups:
- name: mariadb.rules
rules:
- alert: MariaDBReplicationLagging
expr: time() - mysql_slave_status_master_log_file_seconds > 60
for: 2m
labels:
severity: critical
annotations:
summary: "MariaDB replication lag on {{ $labels.instance }}"
description: "MariaDB instance {{ $labels.instance }} is lagging behind master by {{ printf \"%.0f\" (time() - mysql_slave_status_master_log_file_seconds) }} seconds."
This rule uses mysql_slave_status_master_log_file_seconds (which represents the timestamp of the last replicated event) and compares it to the current time. If the difference exceeds 60 seconds for 2 minutes, it triggers a critical alert.
The mysql_global_status_uptime metric is also useful for detecting unexpected restarts. You can alert if the uptime is too low, indicating a recent crash.
# In your Prometheus rules file
groups:
- name: mariadb.rules
rules:
- alert: MariaDBLowUptime
expr: mysql_global_status_uptime < 3600 # Less than 1 hour
for: 1m
labels:
severity: critical
annotations:
summary: "MariaDB instance {{ $labels.instance }} has low uptime"
description: "MariaDB instance {{ $labels.instance }} has only been up for {{ printf \"%.0f\" mysql_global_status_uptime }} seconds, indicating a recent restart."
This alert fires if mysql_global_status_uptime drops below 3600 seconds (1 hour).
Many forget that the exporter itself can fail or be unreachable. A basic check for the exporter’s health can be done via Prometheus’s own up metric.
# In your Prometheus rules file
groups:
- name: mariadb.rules
rules:
- alert: MariaDBExporterUnreachable
expr: up{job="mariadb"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "MariaDB exporter unreachable for job {{ $labels.job }} on instance {{ $labels.instance }}"
description: "Prometheus cannot scrape metrics from MariaDB instance {{ $labels.instance }}."
This alert fires if the up metric for the mariadb job is 0, meaning Prometheus couldn’t connect to the exporter’s :9104 port.
The mysql_global_status_aborted_clients metric is a good indicator of clients being disconnected unexpectedly, often due to network issues or server load.
# In your Prometheus rules file
groups:
- name: mariadb.rules
rules:
- alert: HighMariaDBAbortedClients
expr: rate(mysql_global_status_aborted_clients[5m]) > 5
for: 5m
labels:
severity: warning
annotations:
summary: "High rate of aborted clients on MariaDB instance {{ $labels.instance }}"
description: "MariaDB instance {{ $labels.instance }} is experiencing a high rate of aborted clients ({{ $value }} per second)."
This rule uses rate() to look at the increase in aborted_clients over 5 minutes. If the rate exceeds 5 per second, it signals a problem.
Finally, don’t overlook the basic disk space. While not directly from the mysqld_exporter, you’d typically monitor disk usage on the MariaDB host itself using node_exporter. However, you can infer potential issues from metrics like mysql_global_status_innodb_data_reads or mysql_global_status_innodb_data_writes if they spike dramatically without a corresponding increase in query load, suggesting heavy I/O that might be constrained by disk.
The next thing you’ll likely want to monitor is the performance of individual queries, which requires more advanced configuration and often custom dashboards to visualize query latency and throughput.