Skip to content

get_dataset pickle protocol problem #898

Closed

Description

@ieCecchetti

Description

Steps/Code to Reproduce

# load as dataframe
curr_ds = openml.datasets.get_dataset(2)
X, y, categorical_indicator, attribute_names = curr_ds.get_data(
    dataset_format='dataframe',
    target=curr_ds.default_target_attribute
)

Expected Results

X and y matrix of anneal DS

Actual Results

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\name\AppData\Local\JetBrains\PyCharm 2019.3.2\plugins\python\helpers\pydev_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "C:\Users\name\AppData\Local\JetBrains\PyCharm 2019.3.2\plugins\python\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/name/PycharmProjects/metaLearning/openMLFeatureExtractor.py", line 31, in
curr_ds = openml.datasets.get_dataset(int(curr_ds_starts[0]))
File "C:\Users\name\AppData\Local\Continuum\anaconda3\envs\metaenv\lib\site-packages\openml\datasets\functions.py", line 530, in get_dataset
description, features, qualities, arff_file
File "C:\Users\name\AppData\Local\Continuum\anaconda3\envs\metaenv\lib\site-packages\openml\datasets\functions.py", line 1023, in _create_dataset_from_description
qualities=qualities,
File "C:\Users\name\AppData\Local\Continuum\anaconda3\envs\metaenv\lib\site-packages\openml\datasets\dataset.py", line 183, in init
self.data_pickle_file = self._create_pickle_in_cache(data_file)
File "C:\Users\name\AppData\Local\Continuum\anaconda3\envs\metaenv\lib\site-packages\openml\datasets\dataset.py", line 406, in _create_pickle_in_cache
data, categorical, attribute_names = pickle.load(fh)
ValueError: unsupported pickle protocol: 5.

Versions

Windows-10-10.0.16299-SP0
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)]
NumPy 1.18.1
SciPy 1.4.1
Scikit-Learn 0.22.1

Activity

  1. Andreimdvn commented on Mar 11, 2020

    @Andreimdvn

    I had this problem when I downloaded a dataset with python 3.8 then i downgraded python to 3.7. The dataset is cached with pickle protocol 5 wich is introduced in 3.8. You can remove the cached datasests from C:\Users\name.openml

  2. mfeurer commented on Jul 6, 2020

    @mfeurer
    Collaborator

    Thanks for the explanation @Andreimdvn

    We added a better error handling with #925 -> OpenML-Python will ignore the pickle file it cannot load in the case of a pickle protocol error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions