Is mlflow safe?

GHSA-26p8-2jq9-3vq9

MLflow 3.12.0 introduced registration and deserialization support for third-party scorers in OSS MLflow. During scorer deserialization, the server accepts a request-controlled Python module path beneath one of four allowed package prefixes, imports that module, obtains a request-controlled class, and instantiates it synchronously.

An HTTP caller can combine this behavior with MLflow's built-in experiment, run, and artifact-upload endpoints:

• Create an experiment whose local artifact location points to a writable allowed scorer package.
• Create a run, causing MLflow to allocate <artifact_location>/<run_id>/artifacts/.
• Upload a Python module into that run artifact directory.
• Submit a serialized third-party scorer whose module path resolves to the uploaded file.
• Trigger server-side import and class construction through /ajax-api/3.0/mlflow/scorer/invoke.

On the default no-auth server using the official MLflow v3.14.0 image, this produces unauthenticated arbitrary Python code execution with the privileges of the MLflow server process. The reproduced image runs the process as uid 0 and permits writes to its installed Python package tree.

The attached proof of concept is intentionally harmless. The uploaded constructor performs one action only: it writes a random marker beside its own uploaded run artifact. The marker is then downloaded through MLflow's run-artifact endpoint and checked for an exact match.

At release commit 86cd7f5a1dc2, mlflow/genai/scorers/scorer_utils.py defines these allowed roots:

THIRD_PARTY_SCORER_ALLOWED_MODULES = frozenset({
    "mlflow.genai.scorers.ragas",
    "mlflow.genai.scorers.deepeval",
    "mlflow.genai.scorers.trulens",
    "mlflow.genai.scorers.phoenix",
})

In mlflow/genai/scorers/base.py, the deserializer accepts either an exact root or any descendant:

module_path == m or module_path.startswith(m + ".")

It then performs the following operations using request-controlled values:

module = importlib.import_module(module_path)
scorer_class = getattr(module, class_name, None)
scorer_instance = scorer_class(**init_kwargs)

A package-name prefix is not a provenance boundary. If the server process can write below an allowed package, an attacker-created descendant can satisfy the string-prefix check and become importable Python code.

Relevant source locations at the v3.14.0 release commit:

• mlflow/genai/scorers/scorer_utils.py, allowed third-party scorer modules
• mlflow/genai/scorers/base.py, third-party scorer deserialization and construction

In mlflow/server/handlers.py, _invoke_scorer_handler() parses the supplied serialized scorer and calls:

scorer = Scorer.model_validate_json(serialized_scorer)

Only after deserialization and construction does the handler call:

batches = get_trace_batches_for_scorer(trace_ids, scorer, tracking_store)

The attacker-controlled constructor therefore runs before the server validates or resolves the supplied trace IDs.

The minimal proof class is deliberately not a complete MLflow scorer. After its constructor writes the marker, later trace-batching logic accesses scorer attributes that the proof class does not implement. The resulting HTTP 500 occurs after code execution and does not invalidate the proof.

No background job execution or special job-execution server option is required to reach the constructor.

The experiment creation endpoint accepts an artifact_location. For file-backed experiments, MLflow derives each run artifact directory as:

<artifact_location>/<run_id>/artifacts/

The proof of concept selects this package directory in the official v3.14.0 image:

file:///usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens

After creating a run and uploading audit_<token>.py, the resulting filesystem location is:

/usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens/<run_id>/artifacts/audit_<token>.py

The corresponding import path is:

mlflow.genai.scorers.trulens.<run_id>.artifacts.audit_<token>

The attack does not overwrite an existing package module. It creates a new run-scoped module in the normal file-backed artifact layout. The run ID is returned to the caller.

With authentication disabled, as in the default reproduced server configuration, no credentials are required.

When MLflow basic authentication is enabled, authentication still applies. However, the scorer-invocation AJAX endpoint is assigned to validate_gateway_proxy, which returns True without performing an experiment- or scorer-level resource permission check.

Practical exploitation in an authenticated deployment still requires the caller to create an experiment and run and upload artifacts to them. The key authorization issue is that scorer deserialization itself is not gated by a resource-specific permission check for the supplied experiment before the import occurs.

The attached reproduction is intentionally isolated:

• exact official MLflow v3.14.0 image pinned by manifest digest;
• Docker network disabled;
• MLflow bound to container loopback;
• no shell command or external callback;
• no external request;
• no host filesystem write;
• marker created beside the uploaded run artifact;
• marker retrieved through the run-artifact download endpoint; and
• disposable container removed after completion.

Run:

./run-official-image.sh

Expected output:

MLFLOW_VERSION=3.14.0
PACKAGE_DIR=/usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens
CREATE_EXPERIMENT_STATUS=200
CREATE_RUN_STATUS=200
UPLOAD_STATUS=200
INVOKE_STATUS=500
PROOF_DOWNLOAD_STATUS=200
PROOF_MATCH=1
RESULT=AFFECTED

The security-relevant result is:

PROOF_MATCH=1
RESULT=AFFECTED

This proves that the attacker-controlled constructor executed in the MLflow server process before the later scorer-processing failure.

• Release: MLflow v3.14.0
• Release commit: 86cd7f5a1dc2
• Official image: ghcr.io/mlflow/mlflow:v3.14.0
• Manifest digest: sha256:967de3ca34b6a5751718e380a0a383eb0ce1a6f5b484f363bd0da7559117a5f8
• Python version: 3.10.18
• Image-configured user: empty
• Observed runtime user: uid 0
• Server bind: 127.0.0.1:5000
• Container network: disabled
• Backend store: sqlite:////work/mlflow.db
• Default artifact root: /work/artifacts
• Authentication: disabled
• Special job-execution option: none

Server command inside the disposable container:

mlflow server \
  --host 127.0.0.1 \
  --port 5000 \
  --backend-store-uri sqlite:////work/mlflow.db \
  --default-artifact-root /work/artifacts

The vulnerable third-party scorer deserialization behavior was introduced by commit:

02eea26035b9d8e65af31e6e8f058ca195f62cdd
Enable third-party scorer registration in OSS MLflow (#22634)

The feature is absent from v3.11.1 and first appears in v3.12.0.

Runtime reproduction succeeds on v3.14.0. The relevant mlflow/genai/scorers/base.py content is unchanged at master commit:

20858c9426a9644dfcf9545a92cbb534376779eb

No fixed release was identified as of 2026-07-12.

On the reproduced stock no-auth image, any network caller who can reach the tracking server can execute Python code as the MLflow process without credentials or user interaction.

Successful exploitation provides confidentiality, integrity, and availability impact over all data, credentials, network access, and filesystem locations available to the MLflow server process.

The stock image's runtime uid 0 increases container-level impact. Host impact still depends on the container runtime configuration, mounted paths, granted capabilities, socket exposure, and other isolation controls.