Skip to content

When running from within a k8s pod, use the k8s API to get the cgroup name #5576

Merged
cakrit merged 12 commits intonetdata:masterfrom
cakrit:k8s-1
Mar 13, 2019
Merged

When running from within a k8s pod, use the k8s API to get the cgroup name #5576
cakrit merged 12 commits intonetdata:masterfrom
cakrit:k8s-1

Conversation

@cakrit
Copy link
Copy Markdown
Contributor

@cakrit cakrit commented Mar 7, 2019

Summary

Fixes #5387 (just the pod/container monitoring part, presentation will be a different issue)
Fixes #3520 (again, just wrt naming, presentation will be a new issue).

k8s don't necessarily have a docker executable available. Use the k8s pod API to get the pod name.
Used for cgroup naming.
Disable cgroups that are not returned by the k8s API (e.g. pause containers).

@cakrit cakrit added the area/collectors Everything related to data collection label Mar 7, 2019
@cakrit cakrit requested review from ktsaou and vlvkobal as code owners March 7, 2019 17:03
@cakrit cakrit requested a review from paulfantom March 7, 2019 17:35
fi
done

if [ -z "${NAME}" ] && [ -n "${KUBERNETES_SERVICE_HOST}" ] && [ -n "${KUBERNETES_PORT_443_TCP_PORT}" ]; then
Copy link
Copy Markdown
Contributor

@paulfantom paulfantom Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KUBERNETES_PORT_443_TCP_PORT -> KUBERNETES_API_PORT

No need to copy-paste from k8s docs when it doesn't improve UX.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen KUBERNETES_API_PORT in any official docs, or even mentioned in issues. How can I trust it to be the https port?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worked on all tests with this, so leaving it as is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worked on all tests with this, so leaving it as is.

Comment thread collectors/cgroups.plugin/cgroup-name.sh.in
@cakrit
Copy link
Copy Markdown
Contributor Author

cakrit commented Mar 8, 2019

This won't be considered complete until #3520 is implemented (at least to the point of correctly showing only the relevant cgroups, with appropriate names).

@netdatabot netdatabot added the area/packaging Packaging and operating systems support label Mar 9, 2019
@cakrit cakrit changed the title [WIP] When running from within a k8s pod, use the k8s API to get the pod name When running from within a k8s pod, use the k8s API to get the cgroup name Mar 12, 2019
@cakrit cakrit requested a review from mfundul as a code owner March 13, 2019 10:47
Copy link
Copy Markdown
Member

@ktsaou ktsaou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably do something about the jq command to make sure it is installed with netdata.

I mean, to be installed by kickstart.sh and netdata.spec.

@cakrit cakrit merged commit b7998ec into netdata:master Mar 13, 2019
@cakrit cakrit deleted the k8s-1 branch March 21, 2019 17:55
@woosley
Copy link
Copy Markdown
Contributor

woosley commented Jun 2, 2019

is this change just for netdata running in a k8s Pod? what about the situation when netdata is running directly on the VM with kubernetes running?

I have this situation so now the cgroup names are not mapped to the human readable docker container name.

@cakrit
Copy link
Copy Markdown
Contributor Author

cakrit commented Jun 3, 2019

jq is only uses in k8s. For the problem you have, please open a new issue with the details and we'll be happy to help. The log contains many entries from cgroup-name.sh, so please include info from the log as well.

jackyhuang85 pushed a commit to jackyhuang85/netdata that referenced this pull request Jan 1, 2020
… name (netdata#5576)

* When running from within a k8s pod, use the k8s API to get the pod name

* Check the last part of an underscore-delimited or slash-delimited container id

* Add a docker image builder that adds a single image to a user-specified registry, for use in k8s

* When running in k8s, disable cgroups that the pod API does not return

* Use longer name for k8s containers

* Add reference to build-test.sh to packaging/docker/README.md

* Anonymous statistics should not break when /proc/1/sched is not available
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/collectors Everything related to data collection area/packaging Packaging and operating systems support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Kubernetes pod/containers monitoring Kubernetes Monitoring Improvement Ideas

6 participants