Azure AutoML Batch End point job creation fails with Docker build-user requested bokeh==3.8.2 and azureml-train-automl-runtime 1.61.0 depends on bokeh<3.0.0

Hi @Santos Vighnesh , thanks for posting your issue here.

This is a classic pip dependency conflict during Docker image build. azureml-train-automl-runtime 1.61.0 requires bokeh<3.0.0 but your environment is pinning bokeh==3.8.2, so conda can't resolve it and the whole build fails. microsoft

The fix depends on whether you actually need bokeh in your scoring/inference code:

If you don't need bokeh directly, just remove bokeh==3.8.2 from your conda.yml or requirements.txt and let the AutoML runtime pull in a compatible version automatically.

If you do need bokeh, pin it to something compatible like bokeh==2.4.3 which satisfies the <3.0.0 constraint.

Also worth checking if a transitive dependency like panel, holoviews, or hvplot is silently pulling in bokeh 3.x. Run pip show bokeh in your local env to trace where it's coming from.

After updating the environment definition, trigger a fresh image build and redeploy the batch endpoint. The build should get past the conda env create step cleanly.

Please upvote and accept the answer if it helps!

Hello @Santos Vighnesh ,

Welcome to Microsoft Q&A .Thank you for reaching out to us.

Based on the information provided, the failure is occurring during the Azure Machine Learning environment image build process, specifically while creating the Conda/Pip environment required for the AutoML Batch Endpoint deployment.

The error indicates a Python dependency resolution conflict rather than an issue with the trained model, scoring script, input data, or Batch Endpoint service.

The key conflict identified in the logs is

The user requested bokeh==3.8.2

azureml-train-automl-runtime 1.61.0 depends on bokeh<3.0.0

The deployment environment being built contains a requirement for bokeh==3.8.2

while azureml-train-automl-runtime==1.61.0 requires bokeh<3.0.0

Since these requirements cannot be satisfied together, Pip cannot complete dependency resolution. This causes the following sequence:

Python dependency conflict > Conda environment creation failure > Docker image build failure > Batch Endpoint deployment creation failure

The current evidence indicates that the deployment environment contains an incompatible dependency requirement that needs to be aligned with the AutoML runtime dependency constraints.

Please check if the following steps help-

1. Identifying the source of the Bokeh dependency Please review the original environment configuration used for the Batch Endpoint deployment. Check the following locations:
   • conda.yml
   • environment.yml
   • requirements.txt
   • Deployment YAML definition
   • SDK code used to create the Azure ML environment
   • Additional packages added after AutoML model generation
   Search specifically for bokeh==3.8.2 Also review whether another package introduced a Bokeh dependency indirectly. Packages such as panel , holoviews , hvplot ,jupyter-bokeh may introduce Bokeh as a transitive dependency even when Bokeh is not explicitly listed. The file shown in the logs azureml-environment-setup/mutated_conda_dependencies.yml is generated by Azure Machine Learning during the environment build process. It should not be modified directly because it may be regenerated during deployment. Changes should be made in the original environment specification.
2. Removing the explicit Bokeh version pin If Bokeh is not required by the inference or scoring workload, remove the explicit dependency bokeh==3.8.2. Allowing the AutoML runtime package to resolve its compatible dependencies is the preferred approach. Reducing unnecessary dependencies in inference environments can also help prevent future package conflicts.
3. Using a compatible Bokeh version if required If the scoring workload requires Bokeh functionality, update the dependency to a version compatible with the AutoML runtime. For example bokeh<3.0.0 or bokeh==2.4.3 , then this aligns with azureml-train-automl-runtime==1.61.0 which requires Bokeh versions below 3.0.0.
4. Validating using the original AutoML-generated environment If the AutoML-generated environment was modified after model training:
   • Test deployment using the original AutoML-generated environment without additional dependency changes.
   • Compare the generated environment with the customized deployment environment.
   • Add only packages required for the inference workload.
   This helps confirm whether the conflict was introduced by additional environment customization.
5. Checking for duplicate package definitions Verify that Bokeh is not defined multiple times through different dependency sources. For example, avoid configurations such as: dependencies:   - bokeh=3.8.2 pip:   - bokeh==3.8.2 Duplicate or conflicting package definitions can cause dependency resolution failures during environment creation.
6. Reviewing AutoML package compatibility only if the Bokeh dependency is not directly found If bokeh==3.8.2 is not present in the environment configuration, review the AutoML package versions used during training and deployment. Confirm compatibility between:
   • azureml-train-automl-runtime
   • azureml-automl-runtime
   • azureml-automl-core
   • azureml-core
   This check is mainly applicable when environments are generated automatically or when there has been a change in SDK/runtime versions between training and deployment.
7. Recreating the environment and redeploy After updating the environment definition:
   1. Create a new environment version or update the deployment environment configuration.
   2. Trigger a fresh Azure Machine Learning environment image build.
   3. Redeploy the Batch Endpoint.
   4. Confirm that the build completes successfully beyond the conda env create stage.

The following references might be helpful , please check them out

• About Azure Machine Learning environments - Azure Machine Learning | Microsoft Learn
• Manage Azure Machine Learning environments with the CLI & SDK (v2) - Azure Machine Learning | Microsoft Learn
• Deploy models for scoring in batch endpoints - Azure Machine Learning | Microsoft Learn
• CLI (v2) environment YAML schema - Azure Machine Learning | Microsoft Learn

 Thank you

Please "Accept" the answer with an "Upvote" if the response was helpful. This will be benefitting other community members who face the same issue.

Please note this is occurs during batch end point job creation in Azure Automated ML-docker creates the image here it shows version mismatch bokeh=3.8.2