[docs] Add example tracing infrastructure (#1866)

This adds an example on how to get Grafana Tempo up to receive spans as
well as Grafana itself to view them.

I've added this as a separate Tracing doc in the installation guide as
the Advanced one was starting to get rather full.

Fixes: #1791
This commit is contained in:
Daenney 2023-06-09 16:04:23 +02:00 committed by GitHub
parent b401bd1ccb
commit 6f6b8576f0
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 108 additions and 0 deletions

BIN
docs/assets/tracing.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 266 KiB

View file

@ -0,0 +1,44 @@
# Tracing
GoToSocial comes with [OpenTelemetry][otel] based tracing built-in. It's not wired through every function, but our HTTP handlers and database library will create spans. How to configure tracing is explained in the [Observability configuration reference][obs].
In order to receive the traces, you need something to ingest them and then visualise them. There are many options available including self-hosted and commercial options.
We provide an example of how to do this using [Grafana Tempo][tempo] to ingest the spans and [Grafana][grafana] to explore them. Please beware that the configuration we provide is not suitable for a production setup. It can be used safely for local development and can provide a good starting point for setting up your own tracing infrastructure.
You'll need the files in [`example/tracing`][ext]. Once you have those you can run `docker-compose up -d` to get Tempo and Grafana running. With both services running, you can add the following to your GoToSocial configuration and restart your instance:
```yaml
tracing-enabled: true
tracing-transport: "grpc"
tracing-endpoint: "localhost:4317"
tracing-insecure: true
```
[otel]: https://opentelemetry.io/
[obs]: ../../configuration/observability/
[tempo]: https://grafana.com/oss/tempo/
[grafana]: https://grafana.com/oss/grafana/
[ext]: https://github.com/superseriousbusiness/gotosocial/tree/main/example/tracing
## Querying and visualising traces
Once you execute a few queries against your instance, you'll be able to find them in Grafana. You can use the Explore tab and pick Tempo as the datasource. Because our example configuration for Grafana enables [TraceQL][traceql], the Explore tab will have the TraceQL query type selected by default. You can switch to "Search" instead and find all traces emitted by GoToSocial under the "GoToSocial" service name.
Using TraceQL, a simple TraceQL query to find all traces related to requests to `/api/v1/instance` would look like this:
```
{.http.route = "/api/v1/instance"}
```
If you wanted to see all GoToSocial traces, you could instead run:
```
{.service.name = "GoToSocial"}
```
Once you select a trace, a second panel will open up visualising the span. You can drill down from there, by clicking into every sub-span to see what it was doing.
![Grafana showing a trace for the /api/v1/instance endpoint](../assets/tracing.png)
[traceql]: https://grafana.com/docs/tempo/latest/traceql/

View file

@ -0,0 +1,24 @@
version: "3"
services:
tempo:
image: docker.io/grafana/tempo:2.1.1
command: [ "-config.file=/etc/tempo.yaml" ]
volumes:
- ./tempo.yaml:/etc/tempo.yaml
- ./tempo-data:/tmp/tempo
ports:
- "3200:3200" # tempo
- "9095:9095" # tempo grpc
- "4317:4317" # otlp grpc
grafana:
image: docker.io/grafana/grafana:9.5.2
volumes:
- ./grafana-datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
environment:
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
- GF_AUTH_DISABLE_LOGIN_FORM=true
- GF_FEATURE_TOGGLES_ENABLE=traceqlEditor
ports:
- "3000:3000"

View file

@ -0,0 +1,16 @@
apiVersion: 1
datasources:
- name: Tempo
type: tempo
access: proxy
orgId: 1
url: http://tempo:3200
basicAuth: false
isDefault: true
version: 1
editable: false
apiVersion: 1
uid: tempo
jsonData:
httpMethod: GET

View file

@ -0,0 +1,23 @@
server:
http_listen_port: 3200
distributor:
receivers:
otlp:
protocols:
grpc:
ingester:
max_block_duration: 5m
compactor:
compaction:
block_retention: 1h
storage:
trace:
backend: local
wal:
path: /tmp/tempo/wal
local:
path: /tmp/tempo/blocks

View file

@ -63,6 +63,7 @@ nav:
- "installation_guide/third_party.md" - "installation_guide/third_party.md"
- "installation_guide/websocket.md" - "installation_guide/websocket.md"
- "installation_guide/advanced.md" - "installation_guide/advanced.md"
- "installation_guide/tracing.md"
- "Configuration": - "Configuration":
- "configuration/index.md" - "configuration/index.md"
- "configuration/general.md" - "configuration/general.md"