Should we use a fixed seed for random in tests?

Checklist

I have looked into the Readme and Examples, and have not found a suitable solution or answer.
I have looked into the API documentation and have not found a suitable solution or answer.
I have searched the issues and have not found a suitable solution or answer.
I have searched the Auth0 Community forums and have not found a suitable solution or answer.
I agree to the terms within the Auth0 Code of Conduct.

Description

I was checking out the code in auth0/rest.py and noticed that _calculate_wait uses randint to add jitter to retry delays

auth0-python/auth0/rest.py

Lines 243 to 251 in 10555cb

    
           def _calculate_wait(self, attempt: int) -> int: 
        
               # Retry the request. Apply a exponential backoff for subsequent attempts, using this formula: 
        
               # max(MIN_REQUEST_RETRY_DELAY, min(MAX_REQUEST_RETRY_DELAY, (100ms * (2 ** attempt - 1)) + random_between(1, MAX_REQUEST_RETRY_JITTER))) 
        
               # Increases base delay by (100ms * (2 ** attempt - 1)) 
        
               wait = 100 * 2 ** (attempt - 1) 
        
               # Introduces jitter to the base delay; increases delay between 1ms to MAX_REQUEST_RETRY_JITTER (100ms) 
        
               wait += randint(1, self.MAX_REQUEST_RETRY_JITTER())

This randomness could make tests flaky, especially if they depend on timing or retry behavior, since the wait time varies between runs.

Fixing a seed (e.g., random.seed(42) in a test fixture) would make test outcomes consistent and easier to debug. Without it, we risk intermittent failures in CI or locally, which could slow down development. (Im not saying this problems is happening, but it can happen). I didn't see any seeds being set into the test code...

What do you think about adding this to the test setup? I’d be happy to submit a PR if you’re open to it!

One quick question: I saw pytest-randomly in poetry.lock under test-randomorder—does that already handle seeds in some way I might have missed?

Thanks for your input!

Reproduction

.

Additional context

No response

auth0-python version

master branch

Python version

3.12.1

@allrob23

@allrob23 Please feel free to open a PR. I can review it and will update you acordingly.

@kishore7snehil

@kishore7snehil I opened the PR, can you please review? #690

I created conftest.py for each test folder

This has been accepted and released in v4.10.0

Navigation Menu

Repository navigation

Should we use a fixed seed for random in tests? #684

Checklist

Description

Reproduction

Additional context

auth0-python version

Python version

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	def _calculate_wait(self, attempt: int) -> int:
	# Retry the request. Apply a exponential backoff for subsequent attempts, using this formula:
	# max(MIN_REQUEST_RETRY_DELAY, min(MAX_REQUEST_RETRY_DELAY, (100ms * (2 ** attempt - 1)) + random_between(1, MAX_REQUEST_RETRY_JITTER)))

	# Increases base delay by (100ms * (2 ** attempt - 1))
	wait = 100 * 2 ** (attempt - 1)

	# Introduces jitter to the base delay; increases delay between 1ms to MAX_REQUEST_RETRY_JITTER (100ms)
	wait += randint(1, self.MAX_REQUEST_RETRY_JITTER())

Should we use a fixed seed for random in tests? #684

Description

Checklist

Description

Reproduction

Additional context

auth0-python version

Python version

Activity

kishore7snehil commented on May 8, 2025

allrob23 commented on May 8, 2025

kishore7snehil commented on Jun 10, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions