[ENH] V1 → V2 API Migration

This issue tracks the migration of the openml-python SDK from the legacy OpenML API v1 (PHP-based) in openml/OpenML to the newer API v2 (Python-based) in openml/server-api, along with a structural refactor of the codebase to support both APIs during the transition.

The SDK currently relies on the v1 API, which should be progressively replaced by v2 while keeping backward compatibility and fallback where v2 endpoints are incomplete.

Goals

Separate API interaction from SDK logic by introducing clear abstraction layers
Modularize HTTP handling, API resources, and SDK-facing interfaces
Allow easy switching between API v1 and v2
Provide a transparent fallback to v1 for missing or unimplemented v2 endpoints

Proposed design (high level)

A shared HTTPClient responsible for:
- retries
- proxy configurations
- caching
- exception handling
- thread safety
Abstract API "blueprints" (e.g. DatasetsAPI, TasksAPI) defining SDK-facing methods
Version-specific implementations (*V1, *V2) mapping those methods to concrete endpoints
A fallback proxy that routes calls to v1 if a v2 endpoint is unavailable (unless strict=True)
A central backend/context that controls API version selection and exposes resources internally

A draft implementation sketch is included below to illustrate the architecture (not final API).

# handles HTTP requests, retries, and proxy configuration

class HTTPClient:
    def __init__(self, base_url: str):
        self.base_url = base_url

    def get(self, path, params=None):
        return requests.get(f"{self.base_url}{path}", params=params)

    def post(self, path, params=None, files=None):
        return requests.post(f"{self.base_url}{path}", params=params, files=files)


# abstract blueprints defining SDK-exposed API endpoints and methods

class DatasetsAPI(ABC):
    @abstractmethod
    def get(self, id: int) -> dict: ...

class TasksAPI(ABC):
    @abstractmethod
    def get(self, id: int) -> dict: ...

    @abstractmethod
    def list(self, id: int) -> dict: ...


# version-specific implementations built on top of the API blueprints

class DatasetsV1(DatasetsAPI):
    def __init__(self, http: HTTPClient):
        self._http = http

    def get(self, id: int) -> dict:
        r = self._http.get(f"/api/v1/json/data/{id}")
        d = r.json()["data_set_description"]
        return {
            "id": d["data_set_description"]["id"],
            "name": d["data_set_description"]["name"],
            "version": d["data_set_description"]["version"],
        }

class DatasetsV2(DatasetsAPI):
    def __init__(self, http: HTTPClient):
        self._http = http

    def get(self, id: int) -> dict:
        r = self._http.get(f"/datasets/{id}")
        d = r.json()
        return {
            "id": d["id"],
            "name": d["name"],
            "version": d["version"],
        }

class TasksV1(TasksAPI):
    pass

class TasksV2(TasksAPI):
    pass


# proxy that falls back to v1 when v2 endpoints are missing or unimplemented

class FallbackProxy:
    def __init__(self, primary, fallback, *, strict: bool):
        self._primary = primary
        self._fallback = fallback
        self._strict = strict

    def __getattr__(self, name):
        primary_attr = getattr(self._primary, name)

        if not callable(primary_attr):
            return primary_attr

        def wrapper(*args, **kwargs):
            try:
                return primary_attr(*args, **kwargs)
            except NotImplementedError:
                if self._strict:
                    raise
                return getattr(self._fallback, name)(*args, **kwargs)

        return wrapper


# core backend holding API bindings, exposed internally and to the SDK

class APIBackend:
    def __init__(self, *, datasets, tasks):
        self.datasets = datasets
        self.tasks = tasks


def build_backend(version: str, strict: bool) -> APIBackend:
    v1_http = HTTPClient("https://www.openml.org")
    v2_http = HTTPClient("http://127.0.0.1:8001")

    v1 = APIBackend(
        datasets=DatasetsV1(v1_http),
        tasks=TasksV1(v1_http),
    )

    if version == "v1":
        return v1

    v2 = APIBackend(
        datasets=DatasetsV2(v2_http),
        tasks=TasksV2(v2_http),
    )

	if strict:
		return v2

    return APIBackend(
        datasets=FallbackProxy(v2.datasets, v1.datasets),
        tasks=FallbackProxy(v2.tasks, v1.tasks),
    )


class APIContext:
    def __init__(self):
        self._backend = build_backend("v1", strict=False)

    def set_version(self, version: str, strict: bool = False):
        self._backend = build_backend(version, strict)

    @property
    def backend(self):
        return self._backend


# SDK-facing entry points and version switching helpers

api_context = APIContext()

def set_api_version(version: str, strict=False):
    api_context.set_version(version=version, strict=strict)

openml.set_api_version("v2", strict=True)

api_context.backend.data.get(31)
api_context.backend.tasks.list()

Work Items / Refactor scope

Base Scaffolding

Foundation: Establish the initial folder and file structure along with minimal base implementations for core components (HTTP client, backend, resource interfaces, versioned stubs). This provides a stable scaffold that subsequent refactor and migration work can build on.
HTTPClient features: Extend the shared HTTP client to provide consistent, reusable infrastructure concerns across all API versions and resources.
Fallback policy: Define and implement the fallback behavior that transparently redirects calls from v2 to v1 when endpoints are missing or unimplemented, with strict mode disabling fallback and surfacing errors.

Resource implementations

Implement resource classes that define SDK-facing methods and map them to concrete API endpoints. Each resource should have a clear abstract interface and version-specific implementations (*V1, *V2).

I will be working on Datasets Resource implementation

Currently I am working on the EvaluationMeasures.
Afterwards I would like to work on Flows.

@EmanAbdelhaleem

@EmanAbdelhaleem, please comment on #1593 so you can be assigned.
@JATAYU000, please comment on #1592 so you can be assigned.
@rohansen856, please comment on #1594 so you can be assigned.

I would be workinng on Studies implementation.

@geetu040

Hi @geetu040 !! I would like to work on Flows .
Correct me if something I understood wrong .

I have to implement resource classes for flow that define SDK-facing methods and map them to concrete API endpoints. Each resource should have a clear abstract interface and version-specific implementations (*V1, *V2).
Refactor the flows/functions.py
implement HTTPClient features
Work on fallback policy on flow module

Since evaluations/functions.py contains EvaluationMeasures, EstimationProcedures, Evaluations, I will be working on the three of them (I was actually doing so already, just realized now that they are separated here)

Comment review/changes across all stacked PRs

cache
1. if in sdk a particular function used cache, then this functionality should be kept in the following way
2. all the resource methods used in a function, when they use the http client for requests, should pass the parameter use_cache, if the function gives the option to resert/override the existing cache, use reset_cache
3. use_cache and reset_cache parameters can only be used with http.get since other method don't make sense to have cache
4. remove all cache related functions/lines in the sdk facing code
use_api_key
1. post and delete methods in http client will use the api key automatically, but for get methods that need api_key for authentication should set this param to true when using http object: http.get(..., use_api_key=True)
keep thread-safety
1. relevant for: datasets, flows, runs, tasks
2. keep the @openml.utils.thread_safe_if_oslo_installed decorator on top of functions that had this in the sdk
3. it locks a function based on the id and doesn't involve cache should it should work fine with migration
publish()
1. implement publish method in all resource classes OpenML[Resource].
2. This will override the publish method implemented in the OpenMLBase class.
3. This publish method should collect files and use api_context.backend.[resource].publish which sends requests and parses for id internally.
4. api_context.backend.[resource].publish will lead to ResourceV1.publish in _api/resources/base/versions.py
delete()
1. relevant for: datasets, flows, tasks, runs, studies, users
2. don't implement delete anywhere in any of the resource classes, it's already implemented in ResourceV1 and ResourceV2 and should be used without overloading
3. in [resource]/functions.py, update the method delete_[resource] to use api_context.backend.[resource].delete instead of openml.utils._delete_entity
tag()
1. relevant for: datasets, tasks, flows, setup, runs
2. implement push_tag method in all resource classes OpenML[Resource].
3. This will override the push_tag method implemented in the OpenMLBase class.
4. This push_tag method should use api_context.backend.[resource].tag function instead of _tag_openml_base which will entirely be removed from utils
5. tag method is already implemented in the version classes ResourceV1 and ResourceV2, so no need to implement this in the api resource classes
6. This is to avoid dispatching in the api classes
untag()
1. relevant for: datasets, tasks, flows, setup, runs
2. similar work as for tag()
3. implement remove_tag and use api_context.backend.[resource].untag
use ResourceAPI._not_supported for methods that are not implemented in v2
1. we want to raise same error message (custom exception OpenMLNotSupportedError) for all not implemented methods
2. use self._not_supported(method="...") inside the method which is not implemented in the resource classes
3. use # noqa: ARG002 comment for the params to avoid ruff failures
4. for reference check methods in ResourceV2
inheritance in resources
1. make sure your resource class inherits a version specific and a resource specific class
2. this is to ensure all attributes and helper methods from base, version and resource parents classes are enabled
3. e.g RunsV1 should inherit from ResourceV1 and RunsAPI
4. class RunsV1(ResourceV1, RunsAPI): ...
please replace api_context.backend.[resource]s.[method](...) with openml._backend.[resource].[method](....)
1. e.g api_context.backend.datasets.get_features(self.dataset_id) becomes openml._backend.dataset.get_features(self.dataset_id)
2. remove the imports for api_context and simply import openml at top if to be added
3. don't do from openml import _backend, we want to be consistent throughout the sdk
4. this should also fix the circular import issue, I have tested that but let me know if it still persists.
rename api resoruce classes: DatasetsV1(ResourceV1, DatasetsAPI) to DatasetV1API(ResourceV1API, DatasetAPI)
1. also implement your resource in the respective file in openml/_api/resources
2. this is to use better naming conventions and avoid any major merge conflict in future

@geetu040

@geetu040 If the v2 methods aren’t implemented because the endpoints don’t exist, then I don’t need to implement the combined class, right?

@geetu040 If the v2 methods aren’t implemented because the endpoints don’t exist, then I don’t need to implement the combined class, right?

the combined class checks if the output from both versions match and the fallback works as supposed to, so if v2 endpoints don't exist, you can skip the matching outputs but you would still need the combined class for fallback

@geetu040

@geetu040

publish()
implement publish method in all resource classes OpenML[Resource].
This will override the publish method implemented in the OpenMLBase class.
This publish method should collect files and use api_context.backend.[resource].publish which sends requests and parses for id internally.
api_context.backend.[resource].publish will lead to ResourceV1.publish in _api/resources/base/versions.py

Should I implement the publish function for the evaluations resource?

Should I implement the publish function for the evaluations resource?

No, evaluations can't be published in the current state of sdk

@EmanAbdelhaleem

Added 2 more points (10, 11) in #1575 (comment), please update accordingly
CC: @EmanAbdelhaleem @rohansen856 @JATAYU000 @satvshr @Omswastik-11

No, evaluations can't be published in the current state of sdk

Setups are the same too, right?

@geetu040

An idea was to initialize:

    v1_http_client = HTTPClient(
            server=settings.api.v1.server,
            base_url=settings.api.v1.base_url,
            api_key=settings.api.v1.api_key,
            timeout=settings.api.v1.timeout,
            retries=settings.connection.retries,
            delay_method=settings.connection.delay_method,
            delay_time=settings.connection.delay_time,
        )
        v2_http_client = HTTPClient(
            server=settings.api.v2.server,
            base_url=settings.api.v2.base_url,
            api_key=settings.api.v2.api_key,
            timeout=settings.api.v2.timeout,
            retries=settings.connection.retries,
            delay_method=settings.connection.delay_method,
            delay_time=settings.connection.delay_time,
        )

inside TestAPIBase itself to prevent devs from having to initialize it and prevent code duplication. @geetu040 you mentioned not wanting to do this right now because other people may be using different ports (in settings.api.v2.server), but we should do something like this now, which can be used by all once all PRs tests are passing and are in a mergeable state (to prevent the different settings.api.v2.server paths).

@JATAYU000

Following up on #1616 (comment)

For tests that currently mock delete, please update them as follows:

Patch requests.Session.request instead of delete.
Access call arguments via mock_delete.call_args.kwargs instead of .args.
Add a check that the HTTP method is 'DELETE'.
Print mock_delete.call_args.kwargs to inspect its structure and adjust any other test checks accordingly.

Example diff:

-@mock.patch.object(requests.Session, "delete")
+@mock.patch.object(requests.Session, "request")
 def test_delete_unknown_run(mock_delete, test_files_directory, test_api_key):
    openml.config.start_using_configuration_for_example()
    content_file = test_files_directory / "mock_responses" / "runs" / "run_delete_not_exist.xml"
    mock_delete.return_value = create_request_response(
        status_code=412,
        content_filepath=content_file,
    )

    with pytest.raises(
        OpenMLServerException,
        match="Run does not exist",
    ):
        openml.runs.delete_run(9_999_999)

    run_url = "https://test.openml.org/api/v1/xml/run/9999999"

-    assert run_url == mock_delete.call_args.args[0]
+    assert run_url == mock_delete.call_args.kwargs.get("url")
+    assert 'DELETE' == mock_delete.call_args.kwargs.get("method")
     assert test_api_key == mock_delete.call_args.kwargs.get("params", {}).get("api_key")

Relevant for

[ENH] V1 → V2 API Migration - core structure (#1576)

[ENH] V1 → V2 API Migration - evaluations (#1606)

[ENH] V1 → V2 API Migration - evaluation measures (#1603)

[ENH] V1 → V2 API Migration - setups (#1619)

[ENH] V1 → V2 API Migration - estimation procedures (#1604)

[ENH] V1 → V2 API Migration - studies (#1610)

Navigation Menu

Repository navigation

[ENH] V1 → V2 API Migration #1575

Goals

Proposed design (high level)

Work Items / Refactor scope

Base Scaffolding

Resource implementations

28 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[ENH] V1 → V2 API Migration #1575

Description

Goals

Proposed design (high level)

Work Items / Refactor scope

Base Scaffolding

Resource implementations

Activity

JATAYU000 commented on Jan 2, 2026

EmanAbdelhaleem commented on Jan 2, 2026

fkiraly commented on Jan 2, 2026

rohansen856 commented on Jan 2, 2026

Omswastik-11 commented on Jan 5, 2026

EmanAbdelhaleem commented on Jan 6, 2026

28 remaining items

geetu040 commented on Jan 29, 2026

EmanAbdelhaleem commented on Jan 29, 2026

geetu040 commented on Jan 29, 2026

EmanAbdelhaleem commented on Feb 2, 2026

geetu040 commented on Feb 2, 2026

geetu040 commented on Feb 2, 2026

EmanAbdelhaleem commented on Feb 2, 2026

satvshr commented on Feb 4, 2026

geetu040 commented on Feb 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions