Skip to content

test: unify exponential backoff use - #187

Merged
bornast merged 1 commit into
masterfrom
test/unify-backoff-use
Mar 27, 2026
Merged

test: unify exponential backoff use#187
bornast merged 1 commit into
masterfrom
test/unify-backoff-use

Conversation

@droguljic

@droguljic droguljic commented Mar 26, 2026

Copy link
Copy Markdown
Contributor
  • All tests now use util/backOff
  • Max wait time is now ~10 minutes
  • Jitter is dropped to eliminate flaky tests
  • By default NonRetryableError stops execution

@droguljic droguljic added this to the v2 milestone Mar 26, 2026
@droguljic
droguljic requested review from bornast and mandryllo March 26, 2026 16:01
Comment thread tests/database/util.ts Outdated
Comment on lines +66 to +75
await backOff(async () => {
try {
const replicaDBInstanceId = db.replica!.instance
.identifier as unknown as string;
const deleteCommand = new DeleteDBInstanceCommand({
DBInstanceIdentifier: replicaDBInstanceId,
SkipFinalSnapshot: true,
});

await ctx.clients.rds.send(deleteCommand);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this try to delete replica every time?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

Comment thread tests/database/util.ts
return;
}
},
{ numOfAttempts: 10 },

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see an issue with removing this configuration here, in my experience it takes ~10 minutes (or even more) to delete a replica so in fact if max wait time is now ~5 minutes we should configure this backoff with that in mind (I'm ok with removing jitter)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bumped the max wait time to ~10 minutes. A few times I did tests in CI env ~5 minutes was sufficient to delete the replica.

- All tests now use `util/backOff`
- Max wait time is now ~10 minutes
- Jitter is dropped to eliminate flaky tests
- By default `NonRetryableError` stops execution
@droguljic
droguljic force-pushed the test/unify-backoff-use branch from dceba10 to 9b3f449 Compare March 27, 2026 09:53
@bornast
bornast merged commit 3aed680 into master Mar 27, 2026
2 checks passed
@bornast
bornast deleted the test/unify-backoff-use branch March 27, 2026 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants