<html xmlns="http://www.w3.org/1999/xhtml">

<head>

<meta http-equiv="X-UA-Compatible" content="IE=Edge" />

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

<title>Basic Usage &#8212; OpenML 0.8.0dev documentation</title>

<link rel="stylesheet" href="_static/bootstrap-sphinx.css" type="text/css" />

<link rel="stylesheet" href="_static/pygments.css" type="text/css" />

<link rel="stylesheet" type="text/css" href="_static/gallery.css" />

<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>

<script type="text/javascript" src="_static/jquery.js"></script>

<script type="text/javascript" src="_static/underscore.js"></script>

<script type="text/javascript" src="_static/doctools.js"></script>

<script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>

<script type="text/javascript" src="_static/js/jquery-1.11.0.min.js"></script>

<script type="text/javascript" src="_static/js/jquery-fix.js"></script>

<script type="text/javascript" src="_static/bootstrap-3.3.7/js/bootstrap.min.js"></script>

<script type="text/javascript" src="_static/bootstrap-sphinx.js"></script>

<link rel="index" title="Index" href="genindex.html" />

<link rel="search" title="Search" href="search.html" />

<meta charset='utf-8'>

<meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'>

<meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1'>

<meta name="apple-mobile-web-app-capable" content="yes">

</head><body>

<a href="https://github.com/openml/openml-python"

class="visible-desktop hidden-xs"><img

id="gh-banner"

style="position: absolute; top: 50px; right: 0; border: 0;"

src="https://s3.amazonaws.com/github/ribbons/forkme_right_red_aa0000.png"

alt="Fork me on GitHub"></a>

<script>

// Adjust banner height.

$(function () {

var navHeight = $(".navbar .container").css("height");

$("#gh-banner").css("top", navHeight);

});

</script>

<div id="navbar" class="navbar navbar-default navbar-fixed-top">

<div class="container">

<div class="navbar-header">

<!-- .btn-navbar is used as the toggle for collapsed navbar content -->

<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">

<span class="icon-bar"></span>

<span class="icon-bar"></span>

<span class="icon-bar"></span>

</button>

<a class="navbar-brand" href="index.html">

OpenML</a>

<span class="navbar-text navbar-version pull-left"><b>0.8.0dev</b></span>

</div>

<div class="collapse navbar-collapse nav-collapse">

<ul class="nav navbar-nav">

<li><a href="index.html">Start</a></li>

<li><a href="#">User Guide</a></li>

<li><a href="api.html">API</a></li>

<li><a href="progress.html">Changelog</a></li>

<li><a href="contributing.html">Contributing</a></li>

<li><a href="progress.html">Progress</a></li>

<li class="dropdown globaltoc-container">

<a role="button"

id="dLabelGlobalToc"

data-toggle="dropdown"

data-target="#"

href="index.html">Site <b class="caret"></b></a>

<ul class="dropdown-menu globaltoc"

role="menu"

aria-labelledby="dLabelGlobalToc"></ul>

</ul>

<form class="navbar-form navbar-right" action="search.html" method="get">

<div class="form-group">

<input type="text" name="q" class="form-control" placeholder="Search" />

</div>

<input type="hidden" name="check_keywords" value="yes" />

<input type="hidden" name="area" value="default" />

</div>

</div>

</div>

<div class="container">

<div class="row">

<div class="col-md-3">

<div id="sidebar" class="bs-sidenav" role="complementary"><ul>

<li><a class="reference internal" href="#">Basic Usage</a><ul>

<li><a class="reference internal" href="#installation-set-up">Installation &amp; Set up</a></li>

<li><a class="reference internal" href="#key-concepts">Key concepts</a></li>

<li><a class="reference internal" href="#working-with-tasks">Working with tasks</a></li>

<li><a class="reference internal" href="#running-machine-learning-algorithms-and-uploading-results">Running machine learning algorithms and uploading results</a></li>

<li><a class="reference internal" href="#datasets">Datasets</a></li>

<li><a class="reference internal" href="#advanced-topics">Advanced topics</a></li>

</div>

</div>

<div class="col-md-9 content">

<div class="section" id="basic-usage">

<span id="usage"></span><h1>Basic Usage<a class="headerlink" href="#basic-usage" title="Permalink to this headline">¶</a></h1>

<p>This document will guide you through the most important use cases, functions

and classes in the OpenML Python API. Throughout this document, we will use

<a class="reference external" href="http://pandas.pydata.org/">pandas</a> to format and filter tables.</p>

<div class="section" id="installation-set-up">

<h2>Installation &amp; Set up<a class="headerlink" href="#installation-set-up" title="Permalink to this headline">¶</a></h2>

<p>The OpenML Python package is a connector to <a class="reference external" href="https://www.openml.org/">OpenML</a>.

It allows to use and share datasets and tasks, run

machine learning algorithms on them and then share the results online.</p>

<p>The following tutorial gives a short introduction on how to install and set up

the OpenML python connector, followed up by a simple example.</p>

<ul class="simple">

<li><a class="reference external" href="examples/introduction_tutorial.html">Introduction</a></li>

<div class="section" id="key-concepts">

<h2>Key concepts<a class="headerlink" href="#key-concepts" title="Permalink to this headline">¶</a></h2>

<p>OpenML contains several key concepts which it needs to make machine learning

research shareable. A machine learning experiment consists of one or several

<strong>runs</strong>, which describe the performance of an algorithm (called a <strong>flow</strong> in

OpenML), its hyperparameter settings (called a <strong>setup</strong>) on a <strong>task</strong>. A

<strong>Task</strong> is the combination of a <strong>dataset</strong>, a split and an evaluation

metric. In this user guide we will go through listing and exploring existing

<strong>tasks</strong> to actually running machine learning algorithms on them. In a further

user guide we will examine how to search through <strong>datasets</strong> in order to curate

a list of <strong>tasks</strong>.</p>

<p>A further explanation is given in the

<a class="reference external" href="https://openml.github.io/OpenML/#concepts">OpenML user guide</a>.</p>

<div class="section" id="working-with-tasks">

<h2>Working with tasks<a class="headerlink" href="#working-with-tasks" title="Permalink to this headline">¶</a></h2>

<p>You can think of a task as an experimentation protocol, describing how to apply

a machine learning model to a dataset in a way that it is comparable with the

results of others (more on how to do that further down). Tasks are containers,

defining which dataset to use, what kind of task we’re solving (regression,

classification, clustering, etc…) and which column to predict. Furthermore,

it also describes how to split the dataset into a train and test set, whether

to use several disjoint train and test splits (cross-validation) and whether

this should be repeated several times. Also, the task defines a target metric

for which a flow should be optimized.</p>

<p>Below you can find our tutorial regarding tasks and if you want to know more

you can read the <a class="reference external" href="https://docs.openml.org/#tasks">OpenML guide</a>:</p>

<ul class="simple">

<li><a class="reference external" href="examples/tasks_tutorial.html">Tasks</a></li>

<div class="section" id="running-machine-learning-algorithms-and-uploading-results">

<h2>Running machine learning algorithms and uploading results<a class="headerlink" href="#running-machine-learning-algorithms-and-uploading-results" title="Permalink to this headline">¶</a></h2>

<p>In order to upload and share results of running a machine learning algorithm

on a task, we need to create an <a class="reference internal" href="generated/openml.OpenMLRun.html#openml.OpenMLRun" title="openml.OpenMLRun"><code class="xref py py-class docutils literal notranslate"><span class="pre">OpenMLRun</span></code></a>. A run object can

be created by running a <a class="reference internal" href="generated/openml.OpenMLFlow.html#openml.OpenMLFlow" title="openml.OpenMLFlow"><code class="xref py py-class docutils literal notranslate"><span class="pre">OpenMLFlow</span></code></a> or a scikit-learn compatible

model on a task. We will focus on the simpler example of running a

scikit-learn model.</p>

<p>Flows are descriptions of something runable which does the machine learning.

A flow contains all information to set up the necessary machine learning

library and its dependencies as well as all possible parameters.</p>

<p>A run is the outcome of running a flow on a task. It contains all parameter

settings for the flow, a setup string (most likely a command line call) and all

predictions of that run. When a run is uploaded to the server, the server

automatically calculates several metrics which can be used to compare the

performance of different flows to each other.</p>

<p>So far, the OpenML python connector works only with estimator objects following

the <a class="reference external" href="http://scikit-learn.org/dev/developers/contributing.html#apis-of-scikit-learn-objects">scikit-learn estimator API</a>.

Those can be directly run on a task, and a flow will automatically be created or

downloaded from the server if it already exists.</p>

<p>The next tutorial covers how to train different machine learning models,

how to run machine learning models on OpenML data and how to share the results:</p>

<ul class="simple">

<li><a class="reference external" href="examples/flows_and_runs_tutorial.html">Flows and Runs</a></li>

<div class="section" id="datasets">

<h2>Datasets<a class="headerlink" href="#datasets" title="Permalink to this headline">¶</a></h2>

<p>OpenML provides a large collection of datasets and the benchmark

“<a class="reference external" href="https://docs.openml.org/benchmark/">OpenML100</a>” which consists of a curated

list of datasets.</p>

<p>You can find the dataset that best fits your requirements by making use of the

available metadata. The tutorial which follows explains how to get a list of

datasets, how to filter the list to find the dataset that suits your

requirements and how to download a dataset:</p>

<ul class="simple">

<li><a class="reference external" href="examples/datasets_tutorial.html">Filter and explore datasets</a></li>

<p>OpenML is about sharing machine learning results and the datasets they were

obtained on. Learn how to share your datasets in the following tutorial:</p>

<ul class="simple">

<li><a class="reference external" href="examples/create_upload_tutorial.html">Upload a dataset</a></li>

<div class="section" id="advanced-topics">

<h2>Advanced topics<a class="headerlink" href="#advanced-topics" title="Permalink to this headline">¶</a></h2>

<p>We are working on tutorials for the following topics:</p>

<ul class="simple">

<li>Querying datasets (TODO)</li>

<li>Creating tasks (TODO)</li>

<li>Working offline (TODO)</li>

<li>Analyzing large amounts of results (TODO)</li>

</div>

</div>

<footer class="footer">

<div class="container">

<p class="pull-right">

<a href="#">Back to top</a>

</p>

<p>

&copy; Copyright 2014-2018, Matthias Feurer, Andreas Müller, Farzan Majdani, Joaquin Vanschoren, Jan van Rijn, Arlind Kadra and Pieter Gijsbers.<br/>

Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.8.1.<br/>

</p>

</div>

</body>