Repository navigation
feat: add progress tracking for suite benchmarking #1497
Description
Problem
When benchmarking models across OpenML suites (e.g., OpenML-CC18 with multiple tasks), users have no visibility into:
- How many tasks have completed
- How long each task takes
- Estimated time remaining
- Whether the process is still running or has stalled
Proposed Solution
Add a run_suite_with_progress() function that wraps suite benchmarking with a progress bar using tqdm.
Benefits
- Better UX: Users can monitor long-running benchmarks
- Debugging: Easy to identify slow/stuck tasks
- Planning: ETA helps users schedule experiments
- AI-on-Demand Platform: Improves platform usability for researchers
Activity
Aymuos22 commentedon Nov 24, 2025
Hi @anujagrawal699 ,
Can you assign this to me??
feat: add progress tracking for suite benchmarking openml#1497
PGijsbers commentedon Dec 11, 2025
Sorry for the slow response. I think at this time we are not looking to increase the surface area of the API for features which are easily taken care of by a user (such as writing a loop with tqdm progress bar). If we were to do something like this in the future, I think it would be more likely that we would consider expanding run_model_on_task to take a task list as opposed to adding a new separate function.
I am closing this issue for now. For users that want to see a feature like this, please indicate by "upvoting" the original proposal with something like a 👍