Skip to content

Commit 1f7e8d1

Browse files
bobheadxibobheadxi
authored
log: add log sampling (sourcegraph#38451)
Enables log sampling by default for Sourcegraph components that use the new internal logging library - the first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output. It can be configured for each service using the environment variables SRC_LOG_SAMPLING_INITIAL and SRC_LOG_SAMPLING_THEREAFTER, and if SRC_LOG_SAMPLING_INITIAL is set to 0 or -1 the sampling will be disabled entirely.
1 parent 6b4fa3c commit 1f7e8d1

6 files changed

Lines changed: 20 additions & 3 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,12 @@ All notable changes to Sourcegraph are documented in this file.
2323

2424
### Changed
2525

26-
- Updated minimum required veresion of `git` to 2.35.2 in `gitserver` and `server` Docker image. This addresses a few vulnerabilities disclosed in https://github.blog/2022-04-12-git-security-vulnerability-announced/.
26+
- Updated minimum required veresion of `git` to 2.35.2 in `gitserver` and `server` Docker image. This addresses [a few vulnerabilities announced by GitHub](https://github.blog/2022-04-12-git-security-vulnerability-announced/).
2727
- Search: Pasting a query with line breaks into the main search query input will now replace them with spaces instead of removing them. [#37674](https://github.com/sourcegraph/sourcegraph/pull/37674)
2828
- Rewrite resource estimator using the latest metrics [#37869](https://github.com/sourcegraph/sourcegraph/pull/37869)
2929
- Selecting a line multiple times in the file view will only add a single browser history entry [#38204](https://github.com/sourcegraph/sourcegraph/pull/38204)
3030
- The panels on the homepage (recent searches, etc) are now turned off by default. They can be re-enabled by setting `experimentalFeatures.showEnterpriseHomePanels` to true. [#38431](https://github.com/sourcegraph/sourcegraph/pull/38431)
31+
- Log sampling is now enabled by default for Sourcegraph components that use the [new internal logging library](https://github.com/sourcegraph/log) - the first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output. It can be configured for each service using the environment variables `SRC_LOG_SAMPLING_INITIAL` and `SRC_LOG_SAMPLING_THEREAFTER`, and if `SRC_LOG_SAMPLING_INITIAL` is set to `0` or `-1` the sampling will be disabled entirely. [#38451](https://github.com/sourcegraph/sourcegraph/pull/38451)
3132

3233
### Fixed
3334

‎doc/admin/observability/logs.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,15 @@ We also include the following non-OpenTelemetry fields:
5353
"Function": "string",
5454
}
5555
```
56+
57+
## Log sampling
58+
59+
Sourcegraph services that have migrated to the [new internal logging standard](../../dev/how-to/add_logging.md) have log sampling enabled by default.
60+
The first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output.
61+
62+
This behaviour can be configured for each service using the following environment variables:
63+
64+
* `SRC_LOG_SAMPLING_INITIAL`: the number of entries with identical messages to always output per second
65+
* `SRC_LOG_SAMPLING_THEREAFTER`: the number of entries with identical messages to discard before emitting another one per second, after `SRC_LOG_SAMPLING_INITIAL`.
66+
67+
Setting `SRC_LOG_SAMPLING_INITIAL` to `0` or `-1` will disable log sampling entirely.

‎go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,7 @@ require (
143143
github.com/sourcegraph/go-lsp v0.0.0-20200429204803-219e11d77f5d
144144
github.com/sourcegraph/go-rendezvous v0.0.0-20210910070954-ef39ade5591d
145145
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf
146-
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2
146+
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838
147147
github.com/sourcegraph/run v0.9.0
148148
github.com/sourcegraph/scip v0.1.0
149149
github.com/sourcegraph/sourcegraph/enterprise/dev/ci/images v0.0.0-20220203145655-4d2a39d3038a

‎go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2172,6 +2172,8 @@ github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf h1:oAdWFqhStsWii
21722172
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf/go.mod h1:ppFaPm6kpcHnZGqQTFhUIAQRIEhdQDWP1PCv4/ON354=
21732173
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2 h1:zJfrKH3roMV69m7n06C5y2Xk00HextnxvlG0gnpQolI=
21742174
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
2175+
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838 h1:8wknDSCUVYbaRT63OD+Iyv+stwze5kE2FAzbA1HrNMA=
2176+
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
21752177
github.com/sourcegraph/oauth2 v0.0.0-20210825125341-77c1d99ece3c h1:HGa4iJr6MGKnB5qbU7tI511NdGuHUHnNCqP67G6KmfE=
21762178
github.com/sourcegraph/oauth2 v0.0.0-20210825125341-77c1d99ece3c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
21772179
github.com/sourcegraph/run v0.9.0 h1:mj4pwBqCB+5qEaTp+rhauh5ubYI8n/icOkeiLTCT9Xg=

‎lib/go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ require (
2121
github.com/muesli/termenv v0.9.0
2222
github.com/sourcegraph/go-diff v0.6.1
2323
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf
24-
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2
24+
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838
2525
github.com/stretchr/testify v1.7.2
2626
github.com/xeipuuv/gojsonschema v1.2.0
2727
go.uber.org/atomic v1.9.0

‎lib/go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,8 @@ github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf h1:oAdWFqhStsWii
311311
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf/go.mod h1:ppFaPm6kpcHnZGqQTFhUIAQRIEhdQDWP1PCv4/ON354=
312312
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2 h1:zJfrKH3roMV69m7n06C5y2Xk00HextnxvlG0gnpQolI=
313313
github.com/sourcegraph/log v0.0.0-20220704182018-fbd708c153e2/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
314+
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838 h1:8wknDSCUVYbaRT63OD+Iyv+stwze5kE2FAzbA1HrNMA=
315+
github.com/sourcegraph/log v0.0.0-20220707160925-6a936691c838/go.mod h1:zWEPlKrWBUVpko/tOgDS+qrp7BmzaCcmUrh9+ver1iQ=
314316
github.com/sourcegraph/yaml v1.0.1-0.20200714132230-56936252f152 h1:z/MpntplPaW6QW95pzcAR/72Z5TWDyDnSo0EOcyij9o=
315317
github.com/sourcegraph/yaml v1.0.1-0.20200714132230-56936252f152/go.mod h1:GIjDIg/heH5DOkXY3YJ/wNhfHsQHoXGjl8G8amsYQ1I=
316318
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=

0 commit comments

Comments
 (0)