[BUG] Fix Sklearn Models detection by safely registering SklearnExtension

[geetu040]

Running a scikit-learn Pipeline with openml.runs.run_model_on_task fails even when openml-sklearn is installed.

Reproduction:

import openml
from sklearn import impute, tree, pipeline

clf = pipeline.Pipeline(
    steps=[
        ('imputer', impute.SimpleImputer()),
        ('estimator', tree.DecisionTreeClassifier(max_depth=2))
    ]
)

task = openml.tasks.get_task(32)
run = openml.runs.run_model_on_task(clf, task, avoid_duplicate_runs=False)

print(run)

This raises:

ValueError: No extension registered which can handle model: Pipeline(...).
But it looks related to scikit-learn. Please install the OpenML scikit-learn extension (openml-sklearn) and try again.

This happens even if openml-sklearn is already installed.

If openml-sklearn is installed, sklearn models (including Pipeline) should be detected automatically without requiring any manual imports.

SklearnExtension is only registered if it is imported somewhere during execution. In normal usage, this does not happen, so the extension is never added to openml.extensions.extensions.

Possible fixes:

• Safely import and register SklearnExtension when openml-sklearn is available. This can be done by adding a safe import statement or moving the register_extension out of openml_sklearn/__init__.py
• Improve the error message when openml-sklearn is not installed. The current link does not lead to an installation guide.

[EmanAbdelhaleem]

Hi, I think I will pick this to work on.

[EmanAbdelhaleem]

@geetu040 is there some available link to an installation guide for openml-sklearn?

[geetu040]

I don't think there is any, just install it from the source package in editable mode and it should work

git clone git@github.com:openml/openml-sklearn.git
cd openml-sklearn
pip install -e .

[EmanAbdelhaleem]

I wasn't asking for me but for the second possible fix, you mentioned that the current link does not lead to an installation guide, so I was asking if there is a different valid link.

As a solution for the issue, I have added the following code lines for a safe import at the get_extension_by_model in openml/extensions/functions.py:

    # Import openml_sklearn to register sklearnExtension
    if type(model).__module__.startswith("sklearn"):
        if importlib.util.find_spec("openml_sklearn") is None:
            raise ImportError(SKLEARN_HINT)
        import openml_sklearn

Although the issue is solved by this, I cannot commit the changes cuz they don't pass the ruff checks due to:

1. nested if statement: I believe I need it
2. unused import: although the import is not used but we need it for registering SklearnExtension

So, should I just force the commit or work on openml-sklearn and move the register_extension out of openml_sklearn/__init__.py?

[geetu040]

move the register_extension out of openml_sklearn/__init__.py?

I would suggest not to work on the openml-sklearn repo, since we are going to move it's code back into openml-python

Although the issue is solved by this, I cannot commit the changes cuz they don't pass the ruff checks due to:

I don't think it should be done in get_extension_by_model, rather it should be imported on the top of the file, and imported safely only if openml-sklearn is installed.

nested if statement: I believe I need it

yes you need an if statement for safe import, but do you think is there a better utility to check if openml-sklearn is installed

unused import: although the import is not used but we need it for registering SklearnExtension

I see, can this test be skipped? by adding a noqa comment or anything like that?

[EmanAbdelhaleem]

Well,

• I added the safe import at the top of openml/runs/__init__.py file
• Get rid of the nested if
• added # noqa for skipping the unused import test