Skip to content

Fix invalid credentials error.#10

Merged
itsdalmo merged 11 commits into
masterfrom
fix_credentials_error
Aug 1, 2018
Merged

Fix invalid credentials error.#10
itsdalmo merged 11 commits into
masterfrom
fix_credentials_error

Conversation

@itsdalmo

Copy link
Copy Markdown
Contributor

This fixes a problem where listKeys responded with 401 Bad credentials. However, I can't really explain WHY it fixes the problem.

According to the docs:
https://golang.org/pkg/context/#Background

context.Background() "is never canceled, has no values, and has no deadline". However, it seems like it was somehow cancelled since Github stopped responding with 401 after I changed to context.TODO().

According to the oauth2 docs:
https://godoc.org/golang.org/x/oauth2

You should be able to use either context.Background() or context.TODO() if you are not passing an actual context in. Not sure whats up here.

@itsdalmo itsdalmo changed the title Fix invalid credentials error. WIP: Fix invalid credentials error. Jul 31, 2018
@itsdalmo

Copy link
Copy Markdown
Contributor Author

So, I reverted the changes here and no longer had a problem. Very strange.

@itsdalmo

Copy link
Copy Markdown
Contributor Author

This was a learning experience:
https://medium.com/@tjholowaychuk/aws-lambda-lifecycle-and-in-memory-caching-c9cd0844e072

Since I cache the Github clients when looping over repositories in the handler, I believe the errors were caused by expired tokens from getting the same Lambda container for >1h. The fix will be to make sure we start with fresh clients every time the lambda is invoked.

@mikael-lindstrom

Copy link
Copy Markdown

This explains the case for nodejs, it should be similar in go https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/. Basically the function can skip initialization and go directly to executing the function. Which means there could be things cached from previous runs.

@itsdalmo

itsdalmo commented Jul 31, 2018

Copy link
Copy Markdown
Contributor Author

Will test the current code in the wild. There is more work to be done though:

@itsdalmo itsdalmo changed the title WIP: Fix invalid credentials error. Fix invalid credentials error. Aug 1, 2018
@itsdalmo

itsdalmo commented Aug 1, 2018

Copy link
Copy Markdown
Contributor Author

The updated Lambda has been deployed and tested, and refreshing workers in case the token is expired seems to work fine. I think we should merge this as is and roll it out, and then write a test case for rotating clients in the future (probably will require mocking getInstallationClient).

@mikael-lindstrom mikael-lindstrom left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@itsdalmo itsdalmo merged commit b607760 into master Aug 1, 2018
@itsdalmo itsdalmo deleted the fix_credentials_error branch August 1, 2018 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants