Backends

Markus comes with several backends. You can also write your own.

Logging metrics

class markus.backends.logging.LoggingMetrics(options)

Metrics backend that publishes to Python logging

To use, add this to your backends list:

{
    'class': 'markus.backends.logging.LoggingMetrics',
    'options': {
        'logger_name': 'markus',
        'leader': 'METRICS',
    }
}

The markus.backends.logging.LoggingMetrics backend generates logging messages in this format:

leader|timestamp|metric_type|stat|value|#tags

For example:

METRICS|2017-03-06 11:30:00|histogram|foo|4321|#key1:val

This will log at the logging.INFO level.

Options:

  • logger_name: the name for the logger

    Defaults to "markus".

  • leader: string at the start of the metrics line

    This makes it easier to parse logs for metrics data–you look for the leader and everything after that is parseable data.

    Defaults to "METRICS".

gauge(stat, value, tags=None)

Set a gauge

histogram(stat, value, tags=None)

Report a histogram

incr(stat, value=1, tags=None)

Increment a counter

timing(stat, value, tags=None)

Report a timing

class markus.backends.logging.LoggingRollupMetrics(options)

EXPERIMENTAL BACKEND FOR ROLLUPS

To use, add this to your backends list:

{
    'class': 'markus.backends.logging.LoggingRollupMetrics',
    'options': {
        'logger_name': 'markus',
        'leader': 'ROLLUP',
        'flush_interval': 10
    }
}

The markus.backends.logging.LoggingRollupMetrics backend generates rollups every flush_interval of stats generated during that period.

For incr stats, it shows count and rate.

For gauge stats, it shows count, current value, min value, and max value for the period.

For timing and histogram stats, it shows count, min, average, median, 95%, and max for the period.

This will log at the logging.INFO level.

Options:

  • logger_name: the name for the logger

    Defaults to "markus".

  • leader: string at the start of the metrics line

    This makes it easier to parse logs for metrics data–you look for the leader and everything after that is parseable data.

    Defaults to "ROLLUP".

  • flush_interval: interval to generate rollup data

    markus.backends.logging.LoggingRollupMetrics will spit out rollup data every flush_interval seconds.

    Defaults to 10 seconds.

Statsd metrics

class markus.backends.statsd.StatsdMetrics(options)

Uses pystatsd client for statsd pings.

This requires the pystatsd module and requirements to be installed. To install those bits, do:

$ pip install markus[statsd]

To use, add this to your backends list:

{
    'class': 'markus.backends.statsd.StatsdMetrics',
    'options': {
        'statsd_host': 'statsd.example.com',
        'statsd_port': 8125,
        'statsd_prefix': None,
        'statsd_maxudpsize': 512,
    }
}

Options:

  • statsd_host: the hostname for the statsd daemon to connect to

    Defaults to 'localhost'.

  • statsd_port: the port for the statsd daemon to connect to

    Defaults to 8125.

  • statsd_prefix: the prefix to use for statsd data

    Defaults to None.

  • statsd_maxudpsize: the maximum data to send per packet

    Defaults to 512.

gauge(stat, value, tags=None)

Set a gauge

histogram(stat, value, tags=None)

Does the same thing as timing

incr(stat, value=1, tags=None)

Increment a counter

timing(stat, value, tags=None)

Measure a timing for statistical distribution

Datadog metrics

class markus.backends.datadog.DatadogMetrics(options)

Uses the Datadog DogStatsd client for statsd pings.

This requires the Datadog backend and requirements be installed. To install those bits, do:

$ pip install markus[datadog]

To use, add this to your backends list:

{
    'class': 'markus.backends.datadog.DatadogMetrics',
    'options': {
        'statsd_host': 'localhost',
        'statsd_port': 8125,
        'statsd_namespace': '',
    }
}

Options:

  • statsd_host: the hostname for the statsd daemon to connect to

    Defaults to "localhost".

  • statsd_port: the port for the statsd daemon to connect to

    Defaults to 8125.

  • statsd_namespace: the namespace to use for statsd data

    Defaults to ''.

gauge(stat, value, tags=None)

Set a gauge

histogram(stat, value, tags=None)

Measure a value for statistical distribution

incr(stat, value=1, tags=None)

Increment a counter

timing(stat, value, tags=None)

Measure a timing for statistical distribution

Cloudwatch metrics

class markus.backends.cloudwatch.CloudwatchMetrics(options)

Publishes metrics to stdout for Cloudwatch

This prints to stdout in this format:

MONITORING|unix_epoch_timestamp|value|metric_type|my.metric.name|#tag1:value,tag2

It lets you generate metrics for reading/consuming in Cloudwatch.

For example, Datadog can consume metrics formatted this way from Cloudwatch allowing you to generate metrics in AWS Lambda functions and have them show up in Datadog.

To use, add this to your backends list:

{
    'class': 'markus.backends.cloudwatch.CloudwatchMetrics',
}

This backend doesn’t take any options.

Note

Datadog doesn’t support metrics other than incr (count) and gauge. This backend will send timing and histogram metrics as gauges.

gauge(stat, value, tags=None)

Set a gauge

histogram(stat, value, tags=None)

Does the same thing as gauge

incr(stat, value=1, tags=None)

Increment a counter

timing(stat, value, tags=None)

Does the same thing as gauge

Writing your own

  1. Subclass markus.backends.BackendBase.
  2. Implement __init__. It takes a single “options” dict with stuff the user configured.
  3. Implement incr, gauge, timing, and histogram and have them do whatever is appropriate in the context of your backend.
class markus.backends.BackendBase(options)

Markus Backend superclass that defines API backends should follow

__init__(options)

Implement this. The options dict is the user-specified options.

gauge(stat, value, tags=None)

Implement this. This is a gauge-type metric.

histogram(stat, value, tags=None)

Implement this. This is a histogram-type metric.

incr(stat, value=1, tags=None)

Implement this. This is a counter-type metric.

timing(stat, value, tags=None)

Implement this. This is a timing-type metric.

For example, here’s a backend that prints metrics to stdout:

>>> import markus
>>> from markus.backends import BackendBase

>>> class StdoutMetrics(BackendBase):
...     def __init__(self, options):
...         self.prefix = options.get('prefix', '')
...
...     def _generate(self, kind, stat, value, tags, **extras):
...         print('%s %s %s %s tags=%s %s' % (kind, self.prefix, stat, value, tags, extras))
...
...     def incr(self, stat, value, tags=None):
...         self._generate('incr', stat, value, tags)
...
...     def gauge(self, stat, value, tags=None):
...         self._generate('gauge', stat, value, tags)
...
...     def timing(self, stat, value, tags=None):
...         self._generate('timing', stat, value, tags)
...
...     def histogram(self, stat, value, tags=None):
...         self._generate('histogram', stat, value, tags)
...
>>> markus.configure([{'class': StdoutMetrics, 'options': {'prefix': 'foo'}}], raise_errors=True)

>>> metrics = markus.get_metrics('test')
>>> metrics.incr('key1', value=1)
incr foo test.key1 1 tags=None {}

This will print to stdout foo incr test.key1 1 None {}.