Skip to content

First run of sklearn classifier crashes constantly #562

Closed

Description

@janvanrijn

see attached error.

My guess: flow exists is called, and it returns null. Flow can be uploaded, but flow id is not fixed.

should be tested with a sentinel

Traceback (most recent call last):
  File "/home/janvanrijn/projects/sklearn-bot/examples/run_on_study.py", line 53, in <module>
    run()
  File "/home/janvanrijn/projects/sklearn-bot/examples/run_on_study.py", line 45, in run
    args.upload_result)
  File "/home/janvanrijn/projects/sklearn-bot/sklearnbot/bot/run.py", line 69, in run_bot_on_task
    run = openml.runs.run_model_on_task(task, classifier)
  File "/home/janvanrijn/projects/openml-python/openml/runs/functions.py", line 48, in run_model_on_task
    add_local_measures=add_local_measures)
  File "/home/janvanrijn/projects/openml-python/openml/runs/functions.py", line 146, in run_flow_on_task
    "same: '%s' vs '%s'" % (str(flow.flow_id), str(flow_id))
ValueError: Result from API call flow_exists and flow.flow_id are not same: '108950' vs 'False'

Activity

  1. mfeurer commented on Oct 5, 2018

    @mfeurer
    Collaborator

    Could you please attach a minimal example to reproduce this?

  2. janvanrijn commented on Oct 5, 2018

    @janvanrijn
    MemberAuthor

    It is kind of hard to create a MWE, because it's hard to unit test. (while writing this, I found a nifty solution)

    The current unit tests use the run_flow_on_task fn, which automatically uploads the flow, whereas the run_model_on_task potentially uploads the flow internally, triggering this problem.

    import hashlib
    import openml
    import sklearn.pipeline
    import sklearn.tree
    import time
    
    openml.config.server = 'https://test.openml.org/api/v1'
    
    md5 = hashlib.md5()
    md5.update(str(time.time()).encode('utf-8'))
    sentinel = md5.hexdigest()[:10]
    sentinel = 'TEST%s' % sentinel
    
    task = openml.tasks.get_task(115)
    clf = sklearn.pipeline.Pipeline(steps=[(sentinel,
                                            sklearn.tree.DecisionTreeClassifier())])
    openml.runs.run_model_on_task(clf, task)
    
  3. changed the title [-]First run of flow crashes constantly[/-] [+]First run of sklearn classifier crashes constantly[/+] on Oct 5, 2018
  4. PGijsbers commented on Apr 15, 2019

    @PGijsbers
    Collaborator

    The MWE does not produce an error for me. I think this got fixed in #516.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions