Repository navigation
Raise warning if sklearn model already fitted prior to _run_model_on_task/fold() #968
Description
Description
Give a warning if model is already fitted.
This is acceptable in case of custom experimentation, but not desirable if we want to upload to OpenML. For transparent interpretation of OpenML results, the user should be told of what's happening.
Can be checked for in the _run_model_on_fold() function in openml.extensions.sklearn.extension prior to calling of model_copy.fit().
Possible solutions:
Activity
Deleting TODO that will addressed by #968
Neeratyoy commentedon Oct 28, 2020
Based on this line here and the documentation of clone which suggests a new estimator that has not been fitted is being created, this issue is rendered moot right?
I have got a branch locally that fixes this issue, but it appears it is just redundant code, so hence will push only if you feel we should still have an explicit check.
@mfeurer @PGijsbers
PGijsbers commentedon Oct 28, 2020
To me it looks like the intention of this message is that if
_run_model_on_foldis called with a model that is already fit, a warning should be raised because the model that is already fit is ignored and a new one is trained instead. This might not be what the user intended when they provided an already fit model to this function. But @mfeurer wrote this message and the code, so I'm interested to hear if I interpreted this correctly 😉Additional fixes to PR 777 (#967)
mfeurer commentedon Oct 29, 2020
Sorry, could you please fill me in which message you are talking about? We should add a message which warns if the estimator is already fitted. There are scikit-learn helpers for checking that.
PGijsbers commentedon Nov 3, 2020
Closed by #989