Cope with empty steps in scikit-learn

Reproduce with:

import openml

import sklearn.pipeline
import sklearn.preprocessing
import sklearn.feature_selection
import sklearn.svm

#flow = openml.flows.get_flow(7103)

flow = sklearn.pipeline.Pipeline(
    [
        ('imputation', sklearn.preprocessing.Imputer()),
        ('onehot', sklearn.preprocessing.OneHotEncoder(handle_unknown='ignore',
                                                       categorical_features=[])),
        ('variancethreshold', None),
        ('classifier', sklearn.svm.SVC())
    ]
)

task = openml.tasks.get_task(3)
#run = openml.runs.run_flow_on_task(task, flow)
run = openml.runs.run_model_on_task(task, flow)

follow up of issue #479

Current Outcome:

Traceback (most recent call last):
  File "/home/janvanrijn/projects/openml-python-contrib/openmlcontrib/testjan.py", line 24, in <module>
    run = openml.runs.run_model_on_task(task, flow)
  File "/home/janvanrijn/projects/openml-python/openml/runs/functions.py", line 50, in run_model_on_task
    add_local_measures=add_local_measures)
  File "/home/janvanrijn/projects/openml-python/openml/runs/functions.py", line 129, in run_flow_on_task
    run.parameter_settings = OpenMLRun._parse_parameters(flow)
  File "/home/janvanrijn/projects/openml-python/openml/runs/run.py", line 455, in _parse_parameters
    True, flow.flow_id)
  File "/home/janvanrijn/projects/openml-python/openml/runs/run.py", line 402, in extract_parameters
    sorted(expected_parameters| expected_components), sorted(model_parameters)))
ValueError: Parameters of the model do not match the parameters expected by the flow:
expected flow parameters: ['classifier', 'imputation', 'memory', 'onehot', 'steps']
model parameters: ['classifier', 'imputation', 'memory', 'onehot', 'steps', 'variancethreshold']

What would be the expected outcome?

The variancethreshold should be part of the expected flow parameters.

Using sklearn version 0.21.3, using the modified exampled from above as mentioned below, the error doesn't seem to occur.

from sklearn.impute import SimpleImputer 
import sklearn.pipeline
import sklearn.svm

import openml

clf = sklearn.pipeline.Pipeline(
    [
        ('imputation', SimpleImputer()),
        ('variancethreshold', None),
        ('classifier', sklearn.svm.SVC(gamma='auto'))
    ]
)
task = openml.tasks.get_task(3)
run, flow = openml.runs.run_model_on_task(model=clf, task=task, return_flow=True)

Also, flow.parameters does contain variancethreshold as null in the serialized string.

Can this issue be closed or am I missing something?

I don't think you're missing anything.

But as we support the passthrough now, what do you think about actively forbidden the None to make the behavior consistent and to keep that step in the pipeline?

Sure.
I'll see if I can add it to #866.

That would be great!

We have taken the decision to not provide support None as an estimator placeholder, from the OpenML end, and encourage users to use 'drop' or 'passthrough' wherever sklearn supports so.
This has been implemented in #866.

Navigation Menu

Repository navigation

Cope with empty steps in scikit-learn #480

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Cope with empty steps in scikit-learn #480

Description

Activity

janvanrijn commented on Jun 19, 2018

mfeurer commented on Jun 20, 2018

Neeratyoy commented on Nov 12, 2019

mfeurer commented on Nov 12, 2019

Neeratyoy commented on Nov 12, 2019

mfeurer commented on Nov 12, 2019

Neeratyoy commented on Nov 15, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions