Monitor TLS Certificate Expiry

Probe TLS endpoints with blackbox_exporter and alert when a certificate expires within 21 days or the chain fails validation.

~30 mintlsmonitoringprometheus

Task

Catch expiring or misconfigured certificates before browsers and API clients reject connections.

When to use this

A wildcard certificate covers six hostnames. Renewal fails silently because the DNS challenge record was removed, and three weeks later mobile clients begin rejecting API calls while the team assumes the certificate is fine.

Prerequisites

Prometheus and Alertmanager
blackbox_exporter reachable from Prometheus
A list of externally visible TLS endpoints

Steps

1. Define an HTTPS probe module that validates the certificate chain and records expiry.

modules:
  https_2xx:
    prober: http
    timeout: 5s
    http:
      preferred_ip_protocol: ip4
      tls_config:
        insecure_skip_verify: false

Expected output: The probe returns probe_success 1 and probe_ssl_earliest_cert_expiry as a Unix timestamp.

2. Configure Prometheus to probe each hostname through the exporter.

- job_name: blackbox_tls
  metrics_path: /probe
  params:
    module: [https_2xx]
  static_configs:
    - targets:
        - https://api.example.com
        - https://www.example.com
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - target_label: __address__
      replacement: blackbox-exporter:9115

Expected output: Each target produces probe_success and probe_ssl_earliest_cert_expiry series labeled with the instance URL.

3. Alert when expiry is within 21 days and when the probe fails for five minutes.

groups:
  - name: tls
    rules:
      - alert: TLSCertExpiringSoon
        expr: probe_ssl_earliest_cert_expiry - time() < 21 * 86400
        for: 1h
        labels:
          severity: warning
      - alert: TLSProbeFailing
        expr: probe_success{job="blackbox_tls"} == 0
        for: 5m
        labels:
          severity: critical

Expected output: Both alerts appear in Alertmanager with the affected hostname as the instance label.

4. Test with a self-signed endpoint and confirm the probe fails chain validation.

curl -s 'http://blackbox-exporter:9115/probe?module=https_2xx&target=https://self-signed.example.com' | grep probe_success

Expected output: probe_success 0 alongside probe_ssl_last_chain_error, proving invalid chains are detected.

Troubleshooting

Problem

probe_success is 0 while the certificate is valid in a browser

Solution

The chain is likely incomplete; check with openssl s_client -showcerts and serve the full intermediate chain from the origin.

Problem

The expiry metric is missing for some targets

Solution

Confirm the module performs a successful TLS handshake; expired or unreachable endpoints do not emit expiry until the handshake succeeds.

Limitations

Only endpoints you list are monitored; internal services need separate targets.
Expiry monitoring does not detect revocation or a compromised private key.

Expiry is predictable, outages are not

Certificate expiry is one of the few failures with a known date. A probe that records probe_ssl_earliest_cert_expiry turns that date into a metric, and a simple expression alerts before users notice. Chain validation matters too: a certificate can be valid but unusable if the intermediate is missing, which returns a handshake failure rather than an expiry warning.

Probe like a client

Set insecure_skip_verify: false. A probe that ignores verification will happily report a healthy endpoint that browsers reject. Point targets at the public hostname, not the origin IP, so DNS, SNI, and the full chain are exercised. If the site sits behind a CDN, the probe measures the edge certificate, which is the one clients see.

probe_ssl_earliest_cert_expiry - time() < 21 * 86400

Alert with enough lead time

Twenty-one days leaves room for a failed renewal, a manual fix, and a change review. Alert at warning severity first, then add a critical alert at seven days. Include the hostname in the instance label and a link to the renewal runbook. Silence the warning only with a ticket that has an owner and an expiry.

Close the loop

Renewal automation should be monitored separately: if the renewal job fails, the expiry alert is the backstop, not the primary signal. Test the probe against a known-bad endpoint after changing modules so you know the failure path works. Review the target list quarterly; new hostnames added without probes are unmonitored certificates.