Repository navigation
Dataset download progress bar #1333
Description
Description
Downloading large datasets can take an unpredictable amount of time. It would be nice if a progress bar could be shown.
Steps/Code to Reproduce
openml_dataset = openml.datasets.get_dataset(ID)
X, y, _, _ = openml_dataset.get_data()
Expected Results
A progress bar is shown (either in standard out or in a notebook)
Actual Results
Nothing is shown
Versions
All
Activity
PGijsbers commentedon Mar 23, 2024
It seems that there is a
progressparameter to thefget_objectcall when downloading from minio: https://github.com/minio/minio-py/blob/74e8e5200d4dbf48ae9bbd6a8a2f54614e1958b3/minio/api.py#L1066it's not in the main doc pages for whatever reason. I think it would be reasonable to only implement this for downloads from minio as otherwise files should generally be small (especially once we drop arff).
PGijsbers commentedon Apr 2, 2024
This is implemented in #1335, which can hopefully be included in 0.15, whenever that may be.
PGijsbers commentedon Sep 16, 2024
Added in #1335, so it will be in the next release.
Enable it by setting
openml.config.show_progress = True.Might rename it to the more verbose but clearer "show_download_progress" before PyPI release - just something that came into my mind now.