Uploading datasets

Hey all,
I am working with dataset uploading and I stumbled upon something.

openml-python/openml/datasets/dataset.py

Lines 373 to 397 in f618d81

    
               def publish(self): 
        
                   """Publish the dataset on the OpenML server. 
        
                   Upload the dataset description and dataset content to openml. 
        
                   Returns 
        
                   ------- 
        
                   return_code : int 
        
                       Return code from server 
        
                   return_value : string 
        
                       xml return from server 
        
                   """ 
        
                   file_elements = {'description': self._to_xml()} 
        
                   file_dictionary = {} 
        
                   if self.data_file is not None: 
        
                       file_dictionary['dataset'] = self.data_file 
        
                   return_value = _perform_api_call("/data/", file_dictionary=file_dictionary, 
        
                                                    file_elements=file_elements) 
        
                   self.dataset_id = int(xmltodict.parse(return_value)['oml:upload_data_set']['oml:id']) 
        
                   return self

The function publish() in the OpenMLDataset makes use of the xml description of a dataset and an arff file to upload a dataset at OpenML. However in the way that the class is implemented right now, self.data_file is a string containing the path to the dataset file.

In my opinion we should have a method that takes the description and the arff file as an argument at openml.datasets at the functions module.

Something like:
publish_dataset(description, file)

What is your opinion regarding this?

@amueller

@amueller @janvanrijn @joaquinvanschoren you might be interested in that. When looking at the suggestion, please keep in mind that neither a run nor a flow are currently created by a user but rather by a helper function.

Short version of the answer (from phone) I agree.

Slightly longer: dataset class is a little complicated ATM and contains many things that it shouldn't have imo, e.g. reference to features, qualities, maybe even splits? Ideally this should all be decoupled (might be a bit hard, I never wanted to start this)

I agree that the class is a bit complicated, but I disagree on that it shouldn't contain the features and qualities. This makes it easy for a user to access all kinds of things related to datasets. It does not contain the splits, these are in the task.

I believe this is now implemented in #440.

Navigation Menu

Repository navigation

Uploading datasets #442

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	def publish(self):
	"""Publish the dataset on the OpenML server.

	Upload the dataset description and dataset content to openml.

	Returns
	-------
	return_code : int
	Return code from server

	return_value : string
	xml return from server
	"""

	file_elements = {'description': self._to_xml()}
	file_dictionary = {}

	if self.data_file is not None:
	file_dictionary['dataset'] = self.data_file

	return_value = _perform_api_call("/data/", file_dictionary=file_dictionary,
	file_elements=file_elements)

	self.dataset_id = int(xmltodict.parse(return_value)['oml:upload_data_set']['oml:id'])
	return self

Uploading datasets #442

Description

Activity

mfeurer commented on Apr 18, 2018

janvanrijn commented on Apr 18, 2018

mfeurer commented on Apr 18, 2018

mfeurer commented on Jun 8, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions