Repository navigation
String feature support #257
Description
Hi, this is more a question than a bug report. I tried downloading dataset 40748 that includes sting features and I get this error:
File "/usr/local/lib/python2.7/dist-packages/openml-0.3.0-py2.7.egg/openml/datasets/dataset.py", line 197, in get_data
raise PyOpenMLError('Dataset not compatible, PyOpenML cannot handle string features')
openml.exceptions.PyOpenMLError: Dataset not compatible, PyOpenML cannot handle string features
Given that arff module that is used internally already supports string features is it planned to add support for text datasets to openml-python?
Activity
mfeurer commentedon May 18, 2017
Thank you very much for reporting this. Currently, we don't plan to implement this ourselves. If you want this feature in the near future, please feel free to submit a PR to the development branch
joaquinvanschoren commentedon May 18, 2017
rkadlec commentedon May 22, 2017
Do you have an estimate what all has to be changed in order to support string features?
For instance, recently I noticed that when I upload dataset with string features OpenML.org doesn't compute MD5 hash for it, therefore Weka plugin refuses to work with it.
mfeurer commentedon May 22, 2017
That's a good question. Basically, it is more or less a question of how to handle the data after it is loaded. Currently, data is loaded as a numpy array. I know that numpy can support strings, but can't really foresee the consequences of loading data as a numpy array. As long as data is loaded as a numpy array, it should be a matter of changing this line.
Regarding the open issue: did you try this with the python API?
amueller commentedon Oct 10, 2017
do we want to switch to just returning pandas dataframes?
mfeurer commentedon Oct 10, 2017
What's the advantage of returning pandas over numpy? Does pandas allow per-column data types (which I think numpy doesn't)?
amueller commentedon Oct 10, 2017
@mfeurer yes. It is actually possible in numpy, but people rarely use it. Also pandas has efficient storage for strings.
mfeurer commentedon Oct 10, 2017
Then go ahead :) I assumed it's simply a nicer interface, but in the end it doesn't really matter whether numpy or pandas is returned.
mfeurer commentedon Mar 7, 2019
String feature support was added via #548 and should be explained in the development branch documentation.