More extensive unit tests for run to / from xml #465

Activity

1. janvanrijn commented on May 5, 2018

   

   janvanrijn

   on May 5, 2018

   MemberAuthor

   More actions

   I added two unit tests:

   • test_run_description_deserialize_serialize
   • test_run_description_deserialize_serialize

   the first one is deactivated, as the python api is indeed not ready for this yet (although it's not a bad thing, it doesn't seem to miss any obvious fields)
   the second one runs nicely (had to add something to server for that)
2. mfeurer commented on Jun 19, 2018

   

   mfeurer

   on Jun 19, 2018

   Collaborator

   More actions

   I just wanted to have a look at this issue because getting this right is crucial to the success of OpenML. However, I can't find your tests. Could you please post a link to the tests you added so that I can extend them?
3. janvanrijn commented on Jun 19, 2018

   

   janvanrijn

   on Jun 19, 2018

   MemberAuthor

   More actions

   Great catch.

   They are actually in the PR that will be deleted.

   
       @unittest.skip('Function _create_description_xml does not write server fields yet.')
       def test_run_description_deserialize_serialize(self):
           openml.config.server = self.production_server
           run_id = list(openml.evaluations.list_evaluations(function='predictive_accuracy',
                                                             flow=[7707], size=1).keys())[0]
   
           run_xml_orig = openml._api_calls._perform_api_call('run/%d' %run_id)
           run_obj_orig = openml.runs.functions._create_run_from_xml(run_xml_orig)
           run_xml_prime = run_obj_orig._create_description_xml()
           # TODO: _create_description_xml does not add run id, uploader, etc
           self.assertEqual(run_xml_orig, run_xml_prime)
   
       def test_run_description_deserialize_serialize(self):
           model = DecisionTreeClassifier(max_depth=1)
           task = openml.tasks.get_task(119)
           run_orig = openml.runs.run_model_on_task(task, model)
           run_orig = run_orig.publish()
           run_new = openml.runs.get_run(run_orig.run_id)
   
           # evaluations might not be aligned (original run has some locally generated measures,
           # downloaded run might have or have not obtained server evals)
           run_orig.evaluations = None
           run_new.evaluations = None
           run_orig.fold_evaluations = None
           run_new.fold_evaluations = None
           run_orig.sample_evaluations = None
           run_new.sample_evaluations = None
   
           self.assertEqual(run_orig._create_description_xml(), run_new._create_description_xml())
   

   We should merge these two functions to the development branch.
4. mfeurer

   closed this as completedon Jul 19, 2018
5. mfeurer

   reopened this on Jul 19, 2018
6. mfeurer

   added a commit that references this issue on Sep 20, 2018

   Merge branch 'develop' into add/#465

   0648d58