Repository navigation
Abnormal task retrieval time #385
Description
Hello everyone
I am using pyhon_develop on my windows machine, and when I try to use get_task(taskid) for my experiments, some tasks take an awfully long time to be retrieved, independantly of their size (as an example, the task 3 is taking ages, though it has a reasonable size). I've done some tests, and it appears that it is the same tasks that take long to be retrieved each time. I attached a sample of my code for you. I used the print in order to see if the get_task was working, as I saw that the get_tasks would take what seems to ba an abnormally long time.
dataset_collection.txt
Activity
mfeurer commentedon Dec 7, 2017
Hi @gabartas thanks for reporting this. I do not have a windows machine with python available so I can only check this on Ubuntu. I reduced your example to:
and got the following results:
so no abnormal retrieval times for me. Could you please post the output of the script here?
gabartas commentedon Dec 8, 2017
mfeurer commentedon Dec 11, 2017
Could you please check if there is a directory
3in your task cache directory and remove it if it's there? Also, please check if you have a directory3in your dataset cache directory and remove it. Then, try again. I'm not really sure where the cache directory ends up on Windows, but I assume it is in your user directory in a folder called.openml.gabartas commentedon Dec 12, 2017
mfeurer commentedon Dec 13, 2017
Is it possible for you to profile the code?
gabartas commentedon Jan 23, 2018
Hello, sorry I never saw your answer, here is my complete code (I've edited out my keys), I'm not sure if it's what you mean by "profile the code". The code keeps going wrong when getting the task for the task 3 (the 'task acquired' message never pops up) :
import openml as ml
from sklearn import preprocessing, ensemble
from pprint import pprint
import numpy as np
import pandas as pd
#a functions which find and get the best run on a task
from annexe import get_best_run
#the tasks I intend to collect
tasksid = [3889, 3902, 219, 9914, 9952, 3917, 9957, 9971, 3918, 3913, 9970,
14965, 145976, 145854, 145804, 7295, 145855, 3, 125923, 145972, 37,
31, 146066, 3493, 145862, 145836, 3903, 145878, 9967, 9946, 3899,
146085, 145834, 6566, 3954, 3494, 145853, 34536, 146065, 145833,
3891, 146064, 9889, 146012, 10101, 14951, 3896, 3492, 3485,
145872, 145857, 9976, 9980, 43, 145979]
#the flows concerning the experiment
flowsid = [5636, 5527, 6794, 6766, 6767]
datasets = []
labels = []
for t in tasksid:
print(t)
task = ml.tasks.get_task(t)
print("task acquired")
datasets.append(task.get_dataset().get_data())
print("dataset acquired")
bestscore = 0
bestid = 0
score = 0
tempid = 0
for f in flowsid:
myrun = ml.runs.list_runs(flow = [f], task = [t], size = 1)
tempid = list(myrun)[0]
score = ml.runs.get_run(tempid).evaluations['area_under_roc_curve']
if score>bestscore:
bestid = f
bestscore = score
labels.append(bestid)
np.save("datasets.npy",datasets)
np.save("labels.npy",labels)
mfeurer commentedon Jan 24, 2018
No, I meat that it would be great if you could run your code using cProfile to see what part exactly is slow. You can also check this answer on stackoverflow.
PGijsbers commentedon Apr 4, 2018
Task 3 was one of the tasks affected by a bug which prevented users from downloading said task through the Python API (all tasks which shared task and dataset id had this). It has been fixed in #429.