[ENH] V1 → V2 API Migration

This issue tracks the migration of the openml-python SDK from the legacy OpenML API v1 (PHP-based) in openml/OpenML to the newer API v2 (Python-based) in openml/server-api, along with a structural refactor of the codebase to support both APIs during the transition.

The SDK currently relies on the v1 API, which should be progressively replaced by v2 while keeping backward compatibility and fallback where v2 endpoints are incomplete.

Goals

Separate API interaction from SDK logic by introducing clear abstraction layers
Modularize HTTP handling, API resources, and SDK-facing interfaces
Allow easy switching between API v1 and v2
Provide a transparent fallback to v1 for missing or unimplemented v2 endpoints

Proposed design (high level)

A shared HTTPClient responsible for:
- retries
- proxy configurations
- caching
- exception handling
- thread safety
Abstract API "blueprints" (e.g. DatasetsAPI, TasksAPI) defining SDK-facing methods
Version-specific implementations (*V1, *V2) mapping those methods to concrete endpoints
A fallback proxy that routes calls to v1 if a v2 endpoint is unavailable (unless strict=True)
A central backend/context that controls API version selection and exposes resources internally

A draft implementation sketch is included below to illustrate the architecture (not final API).

# handles HTTP requests, retries, and proxy configuration

class HTTPClient:
    def __init__(self, base_url: str):
        self.base_url = base_url

    def get(self, path, params=None):
        return requests.get(f"{self.base_url}{path}", params=params)

    def post(self, path, params=None, files=None):
        return requests.post(f"{self.base_url}{path}", params=params, files=files)


# abstract blueprints defining SDK-exposed API endpoints and methods

class DatasetsAPI(ABC):
    @abstractmethod
    def get(self, id: int) -> dict: ...

class TasksAPI(ABC):
    @abstractmethod
    def get(self, id: int) -> dict: ...

    @abstractmethod
    def list(self, id: int) -> dict: ...


# version-specific implementations built on top of the API blueprints

class DatasetsV1(DatasetsAPI):
    def __init__(self, http: HTTPClient):
        self._http = http

    def get(self, id: int) -> dict:
        r = self._http.get(f"/api/v1/json/data/{id}")
        d = r.json()["data_set_description"]
        return {
            "id": d["data_set_description"]["id"],
            "name": d["data_set_description"]["name"],
            "version": d["data_set_description"]["version"],
        }

class DatasetsV2(DatasetsAPI):
    def __init__(self, http: HTTPClient):
        self._http = http

    def get(self, id: int) -> dict:
        r = self._http.get(f"/datasets/{id}")
        d = r.json()
        return {
            "id": d["id"],
            "name": d["name"],
            "version": d["version"],
        }

class TasksV1(TasksAPI):
    pass

class TasksV2(TasksAPI):
    pass


# proxy that falls back to v1 when v2 endpoints are missing or unimplemented

class FallbackProxy:
    def __init__(self, primary, fallback, *, strict: bool):
        self._primary = primary
        self._fallback = fallback
        self._strict = strict

    def __getattr__(self, name):
        primary_attr = getattr(self._primary, name)

        if not callable(primary_attr):
            return primary_attr

        def wrapper(*args, **kwargs):
            try:
                return primary_attr(*args, **kwargs)
            except NotImplementedError:
                if self._strict:
                    raise
                return getattr(self._fallback, name)(*args, **kwargs)

        return wrapper


# core backend holding API bindings, exposed internally and to the SDK

class APIBackend:
    def __init__(self, *, datasets, tasks):
        self.datasets = datasets
        self.tasks = tasks


def build_backend(version: str, strict: bool) -> APIBackend:
    v1_http = HTTPClient("https://www.openml.org")
    v2_http = HTTPClient("http://127.0.0.1:8001")

    v1 = APIBackend(
        datasets=DatasetsV1(v1_http),
        tasks=TasksV1(v1_http),
    )

    if version == "v1":
        return v1

    v2 = APIBackend(
        datasets=DatasetsV2(v2_http),
        tasks=TasksV2(v2_http),
    )

	if strict:
		return v2

    return APIBackend(
        datasets=FallbackProxy(v2.datasets, v1.datasets),
        tasks=FallbackProxy(v2.tasks, v1.tasks),
    )


class APIContext:
    def __init__(self):
        self._backend = build_backend("v1", strict=False)

    def set_version(self, version: str, strict: bool = False):
        self._backend = build_backend(version, strict)

    @property
    def backend(self):
        return self._backend


# SDK-facing entry points and version switching helpers

api_context = APIContext()

def set_api_version(version: str, strict=False):
    api_context.set_version(version=version, strict=strict)

openml.set_api_version("v2", strict=True)

api_context.backend.data.get(31)
api_context.backend.tasks.list()

Work Items / Refactor scope

Base Scaffolding

Foundation: Establish the initial folder and file structure along with minimal base implementations for core components (HTTP client, backend, resource interfaces, versioned stubs). This provides a stable scaffold that subsequent refactor and migration work can build on.
HTTPClient features: Extend the shared HTTP client to provide consistent, reusable infrastructure concerns across all API versions and resources.
Fallback policy: Define and implement the fallback behavior that transparently redirects calls from v2 to v1 when endpoints are missing or unimplemented, with strict mode disabling fallback and surfacing errors.

Resource implementations

Implement resource classes that define SDK-facing methods and map them to concrete API endpoints. Each resource should have a clear abstract interface and version-specific implementations (*V1, *V2).

I will be working on Datasets Resource implementation

Currently I am working on the EvaluationMeasures.
Afterwards I would like to work on Flows.

@EmanAbdelhaleem

@EmanAbdelhaleem, please comment on #1593 so you can be assigned.
@JATAYU000, please comment on #1592 so you can be assigned.
@rohansen856, please comment on #1594 so you can be assigned.

I would be workinng on Studies implementation.

@geetu040

Hi @geetu040 !! I would like to work on Flows .
Correct me if something I understood wrong .

I have to implement resource classes for flow that define SDK-facing methods and map them to concrete API endpoints. Each resource should have a clear abstract interface and version-specific implementations (*V1, *V2).
Refactor the flows/functions.py
implement HTTPClient features
Work on fallback policy on flow module

Since evaluations/functions.py contains EvaluationMeasures, EstimationProcedures, Evaluations, I will be working on the three of them (I was actually doing so already, just realized now that they are separated here)

@geetu040

@geetu040

publish()
implement publish method in all resource classes OpenML[Resource].
This will override the publish method implemented in the OpenMLBase class.
This publish method should collect files and use api_context.backend.[resource].publish which sends requests and parses for id internally.
api_context.backend.[resource].publish will lead to ResourceV1.publish in _api/resources/base/versions.py

Should I implement the publish function for the evaluations resource?

Should I implement the publish function for the evaluations resource?

No, evaluations can't be published in the current state of sdk

@EmanAbdelhaleem

Added 2 more points (10, 11) in #1575 (comment), please update accordingly
CC: @EmanAbdelhaleem @rohansen856 @JATAYU000 @satvshr @Omswastik-11

No, evaluations can't be published in the current state of sdk

Setups are the same too, right?

@geetu040

An idea was to initialize:

    v1_http_client = HTTPClient(
            server=settings.api.v1.server,
            base_url=settings.api.v1.base_url,
            api_key=settings.api.v1.api_key,
            timeout=settings.api.v1.timeout,
            retries=settings.connection.retries,
            delay_method=settings.connection.delay_method,
            delay_time=settings.connection.delay_time,
        )
        v2_http_client = HTTPClient(
            server=settings.api.v2.server,
            base_url=settings.api.v2.base_url,
            api_key=settings.api.v2.api_key,
            timeout=settings.api.v2.timeout,
            retries=settings.connection.retries,
            delay_method=settings.connection.delay_method,
            delay_time=settings.connection.delay_time,
        )

inside TestAPIBase itself to prevent devs from having to initialize it and prevent code duplication. @geetu040 you mentioned not wanting to do this right now because other people may be using different ports (in settings.api.v2.server), but we should do something like this now, which can be used by all once all PRs tests are passing and are in a mergeable state (to prevent the different settings.api.v2.server paths).

@JATAYU000

Following up on #1616 (comment)

For tests that currently mock delete, please update them as follows:

Patch requests.Session.request instead of delete.
Access call arguments via mock_delete.call_args.kwargs instead of .args.
Add a check that the HTTP method is 'DELETE'.
Print mock_delete.call_args.kwargs to inspect its structure and adjust any other test checks accordingly.

Example diff:

-@mock.patch.object(requests.Session, "delete")
+@mock.patch.object(requests.Session, "request")
 def test_delete_unknown_run(mock_delete, test_files_directory, test_api_key):
    openml.config.start_using_configuration_for_example()
    content_file = test_files_directory / "mock_responses" / "runs" / "run_delete_not_exist.xml"
    mock_delete.return_value = create_request_response(
        status_code=412,
        content_filepath=content_file,
    )

    with pytest.raises(
        OpenMLServerException,
        match="Run does not exist",
    ):
        openml.runs.delete_run(9_999_999)

    run_url = "https://test.openml.org/api/v1/xml/run/9999999"

-    assert run_url == mock_delete.call_args.args[0]
+    assert run_url == mock_delete.call_args.kwargs.get("url")
+    assert 'DELETE' == mock_delete.call_args.kwargs.get("method")
     assert test_api_key == mock_delete.call_args.kwargs.get("params", {}).get("api_key")

Relevant for

[ENH] V1 → V2 API Migration - core structure (#1576)

[ENH] V1 → V2 API Migration - evaluations (#1606)

[ENH] V1 → V2 API Migration - evaluation measures (#1603)

[ENH] V1 → V2 API Migration - setups (#1619)

[ENH] V1 → V2 API Migration - estimation procedures (#1604)

[ENH] V1 → V2 API Migration - studies (#1610)

Finally completed! Thanks everyone for your contributions.

Also on behalf of us at OpenML :) Sorry for the at times choppy collaboration. The scale of it wasn't what I expected at first, and on top of that I unfortunately was regularly pulled away for urgent infrastructure and security problems. But I'm happy we got there in the end, and we can take it from here!

Navigation Menu

Repository navigation

[ENH] V1 → V2 API Migration #1575

Goals

Proposed design (high level)

Work Items / Refactor scope

Base Scaffolding

Resource implementations

31 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[ENH] V1 → V2 API Migration #1575

Description

Goals

Proposed design (high level)

Work Items / Refactor scope

Base Scaffolding

Resource implementations

Activity

JATAYU000 commented on Jan 2, 2026

EmanAbdelhaleem commented on Jan 2, 2026

fkiraly commented on Jan 2, 2026

rohansen856 commented on Jan 2, 2026

Omswastik-11 commented on Jan 5, 2026

EmanAbdelhaleem commented on Jan 6, 2026

31 remaining items

EmanAbdelhaleem commented on Feb 2, 2026

geetu040 commented on Feb 2, 2026

geetu040 commented on Feb 2, 2026

EmanAbdelhaleem commented on Feb 2, 2026

satvshr commented on Feb 4, 2026

geetu040 commented on Feb 6, 2026

geetu040 commented on Nov 9, 2026

PGijsbers commented on Nov 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions