Usage¶
Markus is used similar to Python’s built-in logging library. To use Markus, you need to do three things:
Configure Markus’ backends using
markus.configure()
.For each Python module or class or however you want to organize it, you use
markus.get_metrics()
to get amarkus.main.MetricsInterface
.Use the various metrics reporting methods on the
markus.main.MetricsInterface
.
markus.configure
¶
- markus.configure(backends, raise_errors=False)¶
Instantiate and configures backends.
- Parameters:
backends (list-of-dicts) –
the backend configuration as a list of dicts where each dict specifies a separate backend.
Each backend dict consists of three things:
class
with a value that is either a Python class or a dotted Python path to one(optional)
options
dict with options for the backend in question to configure it(optional)
filters
list of filters to apply to metrics emitted by this backend
See the documentation for the backends you’re using to know what is configurable in the options dict.
bool (raise_errors) – whether or not to raise an exception if something happens in configuration; if it doesn’t raise an exception, it’ll log the exception
For example, this sets up a default
markus.backends.logging.LoggingMetrics
backend:import markus markus.configure([ { "class": "markus.backends.logging.LoggingMetrics", } ])
This sets up a
markus.backends.logging.LoggingMetrics
backend with options:import markus markus.configure([ { "class": "markus.backends.logging.LoggingMetrics", "options": { "logger_name": "metrics" } } ])
This sets up a
markus.backends.logging.LoggingMetrics
backend that adds a tag to every metric:import markus from markus.filters import AddTagFilter markus.configure([ { "class": "markus.backends.logging.LoggingMetrics", "filters": [AddTagFilter("color:blue")], } ])
You can set up zero or more backends.
Note
During application startup, Markus should get configured before the app starts generating metrics. Any metrics generated before Markus is configured will get dropped.
However, anything can call
markus.get_metrics()
and get amarkus.main.MetricsInterface
before Markus has been configured including at module load time.
markus.get_metrics
¶
- markus.get_metrics(thing='', extra='', filters=None)¶
Return MetricsInterface instance with specified prefix.
The prefix is prepended to all keys emitted with this
markus.main.MetricsInterface
.The
markus.main.MetricsInterface
is not tied to metrics backends. The list of active backends are globally configured. This allows us to createmarkus.main.MetricsInterface
classes without having to worry about bootstrapping order of the app.- Parameters:
thing (class/instance/str) –
The prefix to use for keys generated with this
markus.main.MetricsInterface
.If this is a string, then it uses it as a prefix.
If this is a class, it uses the dotted Python path.
If this is an instance, it uses the dotted Python path plus
str(instance)
.extra (str) – Any extra bits to add to the end of the prefix.
filters (list of MetricsFilters) – any filters to apply to all metrics generated using this MetricsInterface
- Returns:
a
MetricsInterface
instance
Examples:
>>> from markus import get_metrics
Create a MetricsInterface with the prefix “myapp” and generate a count with stat “myapp.thing1” and value 1:
>>> metrics = get_metrics("myapp") >>> metrics.incr("thing1", value=1)
Create a MetricsInterface with the prefix of the Python module it’s being called in:
>>> metrics = get_metrics(__name__)
Create a MetricsInterface with the prefix as the qualname of the class:
>>> class Foo: ... def __init__(self): ... self.metrics = get_metrics(self)
Create a prefix of the class path plus some identifying information:
>>> class Foo: ... def __init__(self, myprefix): ... self.metrics = get_metrics(self, extra=myprefix) ... >>> foo = Foo("jim")
Assume that
Foo
is defined in themyapp
module. Then this will generate the prefixmyapp.Foo.jim
.Create a MetricsFilter and add it to the metrics interface:
>>> from markus.main import MetricsFilter >>> class BlueTagFilter(MetricsFilter): ... def filter(self, record): ... record.tags.append('color:blue') ... return record ... >>> metrics = get_metrics('foo', filters=[BlueTagFilter()])
markus.main.MetricsRecord
¶
- class markus.main.MetricsRecord(stat_type, key, value, tags)¶
Record for a single emitted metric.
- Attribute stat_type:
the type of the stat (“incr”, “gauge”, “timing”, “histogram”)
- Attribute key:
the full key for this record
- Attribute value:
the value for this record
- Attribute tags:
list of tag strings
markus.main.MetricsInterface
¶
- class markus.main.MetricsInterface(prefix, filters=None)¶
Interface to generating metrics.
This is the interface to generating metrics. When you call methods on this instance, it publishes those metrics to the configured backends.
In this way, code can get a
markus.main.MetricsInterface
at any time even before backends have been configured. Further, backends can be switched around without affecting existingmarkus.main.MetricsInterface
instancess.See
markus.get_metrics()
for generatingmarkus.main.MetricsInterface
instances.Create a MetricsInterface.
- Parameters:
prefix (str) –
Use alphanumeric characters and underscore and period. Anything else gets converted to a period. Sequences of periods get collapsed to a single period.
The prefix is prepended to all keys emitted by this metrics interface.
filters (list of MetricsFilter) – list of filters to apply to records being emitted
- extend_prefix(prefix)¶
Returns a duplicate MetricsInterface with prefix extended
- Parameters:
prefix – the prefix to append to the end of the existing prefix
- Returns:
a new MetricsInterface with adjusted prefix
Example:
import markus metrics = markus.get_metrics("key1") metrics.incr("stat1") # key1.stat1 sub_metrics = metrics.with_prefix("key2") sub_metrics.incr("stat1") # key1.key2.stat1
- incr(stat, value=1, tags=None)¶
Incr is used for counting things.
- Parameters:
stat (string) – A period delimited alphanumeric key.
value (int) – A value to increment the count by. Usually this is 1.
tags (list-of-strings) –
Each string in the tag consists of a key and a value separated by a colon. Tags can make it easier to break down metrics for analysis.
For example
["env:stage", "compressed:yes"]
.To pass no tags, either pass an empty list or
None
.
For example:
>>> import markus >>> metrics = markus.get_metrics("foo") >>> def chop_vegetable(kind): ... # chop chop chop ... metrics.incr("vegetable", value=1)
You can also use incr to decrement by passing a negative value.
- gauge(stat, value, tags=None)¶
Gauges are used for measuring things.
- Parameters:
stat (string) – A period delimited alphanumeric key.
value (int) – The measured value of the thing being measured.
tags (list-of-strings) –
Each string in the tag consists of a key and a value separated by a colon. Tags can make it easier to break down metrics for analysis.
For example
["env:stage", "compressed:yes"]
.To pass no tags, either pass an empty list or
None
.
For example:
>>> import markus >>> metrics = markus.get_metrics("foo") >>> def parse_payload(payload): ... metrics.gauge("payload_size", value=len(payload)) ... # parse parse parse
- timing(stat, value, tags=None)¶
Record a timing value.
Record the length of time of something to be added to a set of values from which a statistical distribution is derived.
Depending on the backend, you might end up with count, average, median, 95% and max for a set of timing values.
This is useful for analyzing how long things take to occur. For example, how long it takes for a function to run, to upload files, or for a database query to execute.
- Parameters:
stat (string) – A period delimited alphanumeric key.
value (int) – A timing in milliseconds.
tags (list-of-strings) –
Each string in the tag consists of a key and a value separated by a colon. Tags can make it easier to break down metrics for analysis.
For example
["env:stage", "compressed:yes"]
.To pass no tags, either pass an empty list or
None
.
For example:
>>> import time >>> import markus >>> metrics = markus.get_metrics("foo") >>> def upload_file(payload): ... start_time = time.perf_counter() # this is in seconds ... # upload the file ... timing = (time.perf_counter() - start_time) * 1000.0 # convert to ms ... metrics.timing("upload_file_time", value=timing)
Note
If you’re timing a function or a block of code, it’s probably more convenient to use
markus.main.MetricsInterface.timer()
ormarkus.main.MetricsInterface.timer_decorator()
.
- histogram(stat, value, tags=None)¶
Record a histogram value.
Record a value to be added to a set of values from which a statistical distribution is derived.
Depending on the backend, you might end up with count, average, median, 95% and max for a set of values.
This is useful for analyzing distributions of values. For example, what’s the median and 95% upload file size? What’s the most expensive thing sold?
- Parameters:
stat (string) – A period delimited alphanumeric key.
value (int) – The value of the thing.
tags (list-of-strings) –
Each string in the tag consists of a key and a value separated by a colon. Tags can make it easier to break down metrics for analysis.
For example
["env:stage", "compressed:yes"]
.To pass no tags, either pass an empty list or
None
.
For example:
>>> import time >>> import markus >>> metrics = markus.get_metrics("foo") >>> def finalize_sale(cart): ... for item in cart: ... metrics.histogram("item_cost", value=item.cost) ... # finish finalizing
Note
For metrics backends that don’t have histogram, this will do the same as timing.
- timer(stat, tags=None)¶
Contextmanager for easily computing timings.
- Parameters:
stat (string) – A period delimited alphanumeric key.
tags (list-of-strings) –
Each string in the tag consists of a key and a value separated by a colon. Tags can make it easier to break down metrics for analysis.
For example
["env:stage", "compressed:yes"]
.To pass no tags, either pass an empty list or
None
.
For example:
>>> mymetrics = get_metrics(__name__)
>>> def long_function(): ... with mymetrics.timer('long_function'): ... # perform some thing we want to keep metrics on ... pass
Note
All timings generated with this are in milliseconds.
- timer_decorator(stat, tags=None)¶
Timer decorator for easily computing timings.
- Parameters:
stat (string) – A period delimited alphanumeric key.
tags (list-of-strings) –
Each string in the tag consists of a key and a value separated by a colon. Tags can make it easier to break down metrics for analysis.
For example
["env:stage", "compressed:yes"]
.To pass no tags, either pass an empty list or
None
.
For example:
>>> mymetrics = get_metrics(__name__) >>> @mymetrics.timer_decorator("long_function") ... def long_function(): ... # perform some thing we want to keep metrics on ... pass
Note
All timings generated with this are in milliseconds.
markus.utils
¶
- markus.utils.generate_tag(key, value=None)¶
Generate a tag for use with the tag backends.
The key and value (if there is one) are sanitized according to the following rules:
after the first character, all characters must be alphanumeric, underscore, minus, period, or slash–invalid characters are converted to “_”
lowercase
If a value is provided, the final tag is key:value.
The final tag must start with a letter. If it doesn’t, an “a” is prepended.
The final tag is truncated to 200 characters.
If the final tag is “device”, “host”, or “source”, then a “_” will be appended the end.
- Parameters:
key (str) – the key to use
value (str) – the value (if any)
- Returns:
the final tag
Examples:
>>> from markus.utils import generate_tag >>> generate_tag("yellow") 'yellow' >>> generate_tag("rule", "is_yellow") 'rule:is_yellow'
Some examples of sanitizing:
>>> from markus.utils import generate_tag >>> generate_tag("rule", "THIS$#$%^!@IS[]{$}GROSS!") 'rule:this_______is_____gross_' >>> generate_tag("host") 'host_'
Example using it with
markus.main.MetricsInterface.incr()
:>>> import markus >>> from markus.utils import generate_tag >>> mymetrics = markus.get_metrics(__name__) >>> mymetrics.incr( ... "somekey", ... value=1, ... tags=[generate_tag("rule", "is_yellow")] ... )