Repository navigation
Freeze when retrieving a task #388
Description
I just installed OpenML on a new machine, running Windows 10 and Python 3.6.3. I installed the master branch as of today (4.1.2018). I can not retrieve a task, as the process simply freezes when I call get_task. MWE:
import openml
t = openml.tasks.get_task(3)
it seems to have to do with the locking mechanism, as interrupting this call gives the following stack trace:
File "\openml\tasks\functions.py", line 239, in get_task
dataset = get_dataset(task.dataset_id)
File "openml\datasets\functions.py", line 284, in get_dataset
lock_path=os.path.join(config.get_cache_directory(), 'locks'),
File "\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py", line 174, in __enter__
self.acquire()
File "\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py", line 156, in acquire
gotten = r(self._try_acquire, blocking, watch)
File "\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\_utils.py", line 91, in __call__
return fn(*args, **kwargs)
File "\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py", line 94, in _try_acquire
self.trylock()
It seems the process gets stuck trying to acquire the lock.
Removing the locking mechanism manually (editing out the code) resolves any lock-related issues.*
I am not sure if this means perhaps the issue is with oslo_concurrency.lockutils, or OpenML's use of it. Any idea?
* It introduces a new issue where an xmlquality is assumed to always have a value entry here, this turns out to be a false assumption, generating a KeyError.
Activity
mfeurer commentedon Jan 4, 2018
This might to be a duplicate of #387. Does this only happen on a windows machine or also on a Linux machine?
PGijsbers commentedon Jan 4, 2018
Currently do not have access to a Linux machine, will try when I get home. I read #387 but understood that he was successfully able to do some tasks. I can not even retrieve the first task, so I thought it might be a different issue.
rquintino commentedon Apr 2, 2018
Same issue here, also on Windows 10
Python 3.6.3 :: Anaconda custom (64-bit)
get task never finishes, noticed that cache dir, locks dir and datasets.functions.get_dataset file are created (file with with zero bytes). Task.xml is also downloaded on tasks\12.
interrupting gives:
PermissionError Traceback (most recent call last)
C:\tools\Anaconda3\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py in _try_acquire(self, blocking, watch)
93 try:
---> 94 self.trylock()
95 except IOError as e:
C:\tools\Anaconda3\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py in trylock(self)
216 def trylock(self):
--> 217 msvcrt.locking(self.lockfile.fileno(), msvcrt.LK_NBLCK, 1)
218
PermissionError: [Errno 13] Permission denied
During handling of the above exception, another exception occurred:
KeyboardInterrupt Traceback (most recent call last)
in ()
1 model = RandomForestClassifier()
----> 2 task = openml.tasks.get_task(12)
C:\tools\Anaconda3\lib\site-packages\openml-0.6.0-py3.6.egg\openml\tasks\functions.py in get_task(task_id)
237 try:
238 task = _get_task_description(task_id)
--> 239 dataset = get_dataset(task.dataset_id)
240 class_labels = dataset.retrieve_class_labels(task.target_name)
241 task.class_labels = class_labels
C:\tools\Anaconda3\lib\site-packages\openml-0.6.0-py3.6.egg\openml\datasets\functions.py in get_dataset(dataset_id)
282 with lockutils.external_lock(
283 name='datasets.functions.get_dataset:%d' % dataset_id,
--> 284 lock_path=os.path.join(config.get_cache_directory(), 'locks'),
285 ):
286 did_cache_dir = _create_dataset_cache_directory(dataset_id)
C:\tools\Anaconda3\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py in enter(self)
172
173 def enter(self):
--> 174 self.acquire()
175 return self
176
C:\tools\Anaconda3\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py in acquire(self, blocking, delay, max_delay, timeout)
154 sleep_func=self.sleep_func, watch=watch)
155 with watch:
--> 156 gotten = r(self._try_acquire, blocking, watch)
157 if not gotten:
158 self.acquired = False
C:\tools\Anaconda3\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners_utils.py in call(self, fn, *args, **kwargs)
89 self.attempts += 1
90 try:
---> 91 return fn(*args, **kwargs)
92 except RetryAgain:
93 maybe_delay = self.attempts * self.delay
C:\tools\Anaconda3\lib\site-packages\fasteners-0.14.1-py3.6.egg\fasteners\process_lock.py in _try_acquire(self, blocking, watch)
92 def _try_acquire(self, blocking, watch):
93 try:
---> 94 self.trylock()
95 except IOError as e:
96 if e.errno in (errno.EACCES, errno.EAGAIN):
KeyboardInterrupt:
mfeurer commentedon Apr 3, 2018
Most likely fixed by #429. Please reopen if this issue still exists.