At high traffic levels, the locking around sending on channels can cause
a large amount of blocking and CPU usage. These adds an event queue
mechanism so that events are queued for short period of time, and
flushed in batches to the main exporter goroutine periodically.
The default is is to flush every 1000 events, or every 200ms, whichever
happens first.
Signed-off-by: Clayton O'Neill <claytono@github.com>
This adds two tests.
The first test is to validate that two successive counter events will
increment a counter. This is more about ensuring we can look up the
same metric twice in a row than checking increment functionality.
The second test verifies that the hashLabels function returns different
results when labels are changed.
Signed-off-by: Clayton O'Neill <claytono@github.com>
This reworks/rewrites the way that metric registration and tracking is
handled across all of statsd_exporter. The goal here is to reduce
memory and cpu usage, but also to reduce complexity by unifying metric
registration with the ttl tracking for expiration.
Some high level notes:
* Previously metric names and labels were being hashed three times for
every event accepted: in the container code, the save label set code and
again in the prometheus client libraries. This unifies the first two
and caches the results of `GetMetricWith` to avoid the third.
* This optimizes the label hashing to reduce cpu overhead and memory
allocations. The label hashing code previously showed up high on all
profiling done for both CPU and memory allocations
Using the BenchmarkExporterListener benchmark, the improvement looks
like this.
Before:
cpu: 11,341,797 ns/op
memory allocated: 1,731,119 B/op
memory allocations: 58,028 allocs/op
After:
cpu: 7,084,651 ns/op
memory allocated: 906,556 B/op
memory allocations: 42,026 allocs/op
Signed-off-by: Clayton O'Neill <claytono@github.com>
Previously we were sorting labels in every container to build a map key,
but also when generating the hash key by calling the `LabelsToSignature`
method. This moves the sorting to be done early in the event processing
then passes the sorted label array around.
Signed-off-by: Clayton O'Neill <claytono@github.com>
In #198 the signature for setting up the exporter changed slightly, a
change that #212 and #213 didn't have. This doesn't change anything
substantial.
Signed-off-by: Matthias Rampke <mr@soundcloud.com>
While it doesn't report it as a metric, when collecting, the registry
considers the base metric name to below to the histogram that's
registered. This means that we need to prevent other metrics from
registering anything under this name, in addition to checking for the
bucket, sum and count suffixes.
Signed-off-by: Clayton O'Neill <claytono@github.com>
This adds sanity checking before registering a metric to ensure that the
metric isn't already registered under an existing name. This prevents
the exporter from getting into a state where it has accepted and
registered conflicting metrics, and then cannot report the metrics it
has collected.
Signed-off-by: Clayton O'Neill <claytono@github.com>
- use MetricVec family instead of Metric
- dynamic label values instead of ConstLabels
- use dto.Metric to gain histrogram value in exporter_test
- remove hash calculations
Signed-off-by: Ivan Mikheykin <ivan.mikheykin@flant.com>
The escaping changed in 728bdc52ae, before
0.1.0. Add a test for the escape function, and test various cases,
including those mentioned in the README.
This reveals that the README is inaccurate, adjust it to what has been
the reality for many years. Fixes#97.
* Remove option flag
* Update NewExporter call signature to not take suffix boolean
* Update tests to reflect new behavior and signature
* Update documentation to reflect option flag removal
* add TCP StatsD listener support
* add listen-tcp flag to control UDP/TCP mode on same port
* statsdListenUDP/statsdListenTCP as string, and alias listen-address to listen-udp
* add stats for tcp error/line_too_long
* add test for TCP listener
Instead now we will send a warning message to the logs
and continue on with our day.
Also a guard has been added when we detect a channel has been
closed and the `Exporter.Listen` function will return out of
its loop vs always running. When a channel is closed there will
be no more data to consume, so this seems correct. It also allows
for a test to be written for the panic.
