Repository navigation
OpenML package fails to import if home directory is not writable #883
Description
Description
It looks like OpenML package is trying to write into an user home directory when it is imported. I think this is really not nice. Just importing a package should not have such a side effect. You should write only if there is a request for that.
This broke D3M build process now that we added the OpenML package as a dependency to d3m core package. :-(
Steps/Code to Reproduce
Create an user on your system which has /nonexitent as its home directory. That directory should not exist. Try import openml when running under such user.
Expected Results
Package loads.
Actual Results
It fails with:
File "/usr/local/lib/python3.6/dist-packages/openml/__init__.py", line 18, in <module>
from . import _api_calls
File "/usr/local/lib/python3.6/dist-packages/openml/_api_calls.py", line 8, in <module>
from . import config
File "/usr/local/lib/python3.6/dist-packages/openml/config.py", line 241, in <module>
_setup()
File "/usr/local/lib/python3.6/dist-packages/openml/config.py", line 133, in _setup
os.mkdir(os.path.expanduser(os.path.join('~', '.openml')))
FileNotFoundError: [Errno 2] No such file or directory: '/nonexistent/.openml'
Versions
>>> import platform; print(platform.platform())
Linux-5.0.0-32-generic-x86_64-with-Ubuntu-18.04-bionic
>>> import sys; print("Python", sys.version)
Python 3.6.8 (default, Oct 7 2019, 12:59:55)
[GCC 8.3.0]
>>> import numpy; print("NumPy", numpy.__version__)
NumPy 1.17.3
>>> import scipy; print("SciPy", scipy.__version__)
SciPy 1.3.1
>>> import sklearn; print("Scikit-Learn", sklearn.__version__)
Scikit-Learn 0.21.3
>>> import openml; print("OpenML", openml.__version__)
OpenML 0.10.1
Metadata
Metadata
Assignees
Labels
No labels
Activity
mitar commentedon Nov 13, 2019
If you have OpenML system wide install, try:
sudo -u nobody python3 -c 'import openml'mitar commentedon Nov 13, 2019
So the problem to me here is that this happens not the first time code needs to store something into the cache, but when you import the package.
mitar commentedon Nov 13, 2019
Even easier:
mfeurer commentedon Nov 18, 2019
Thank you very much for reporting this issue. However, to my understanding this is the standard behavior for Unixoid systems. But as that standard says, we should not rely on this directory being present and do not fail on it not existing.
mitar commentedon Nov 26, 2019
I mean, you should not be writing at just importing the module, without calling anything. This is just not nice.
joaquinvanschoren commentedon Jul 2, 2020
So, should we create the folder when the user first downloads/uploads something? I.e. when we need to start caching?
mitar commentedon Jul 2, 2020
Yes.
Fix #883 #884 #906 #972
Fix #883 #884 #906 #972
Rework local openml directory (#987)