Announcing Optuna 4.6

Introduction

We are excited to announce the release of Optuna 4.6, the latest version of our black-box optimization framework. This release significantly improves the user experience and optimization performance, with a particular focus on the following highlights:

• Introduction of an interactive interface powered by Large Language Models (LLMs)
• Further speed enhancements for GPSampler
• Full support for multi-objective and constrained optimization in AutoSampler
• Additions of robust Bayesian optimization packages

Beyond these significant improvements, this version also includes several updates to enhance usability, including an OpenTelemetry integration tutorial, LaTeX support in OptunaHub, and structured output support for the Optuna MCP Server. In this article, we will detail these noteworthy updates.

Get started with the new features by installing the latest version with the following command!

pip install -U optuna optuna-dashboard optunahub

Optuna Dashboard LLM Integration

We are introducing an LLM integration feature to the Optuna Dashboard. By enabling this, you can now utilize two key functions: 1) filtering trials using natural language and 2) automatic graph generation.

The natural language trial filtering feature, as shown in Figure 1, allows you to filter the trials displayed in graphs and tables using natural language queries. Previously, the filter conditions in Optuna Dashboard were quite limited, as supporting a wide variety of conditions would complicate the UI and hinder usability. With this new feature, the LLM generates a filter function based on your instructions, enabling flexible filtering through a simple interface.

Figure 1: Natural language trial filtering feature.

Next, let’s take a look at the automatic graph generation feature. Optuna Dashboard supports general-purpose graphs for any optimization task, including visualizing objective value history, Pareto fronts, and contour plots. However, it was challenging to support visualizations tailored to specific optimization tasks that are needed in only a few use cases. Now, as shown in Figure 2, you can describe the plot you want in natural language, and the LLM will generate a visualization function using Plotly.js. This enables task-specific visualizations that were previously unsupported.

Figure 2: Example of the automatic graph generation feature. The LLM used was OpenAI’s GPT-5 Mini. The input prompt was “Plot intermediate values of all trials in a study.”

For instructions on how to use the LLM integration, please refer to the Tutorial: LLM Integration — Optuna Dashboard documentation. Please note that the performance of these features depends on the LLM being used.

Speeding up GPSampler

GPSampler is a Gaussian Process-based Bayesian optimization sampler known for its high sample efficiency. However, when the evaluation time of the objective function is less than a few hours, the sampling time of GPSampler can become a bottleneck.

In Optuna v4.6, we have made several modifications to GPSampler’s internal implementation to accelerate sampling time while maintaining optimization performance. Specifically, we implemented caching for repeated calculations, removed unnecessary operation calls for empty tensors, and parallelized the optimization of the acquisition function. In the experimental setup shown in Figure 3, we achieved a speedup of nearly 8x. The parallelization of the acquisition function optimization is detailed in a separate article (in Japanese). Please note that the actual speedup may vary depending on the computational environment.

Figure 3: Speedup effect of GPSampler compared to v4.5. The horizontal axis represents the number of trials, and the vertical axis shows the cumulative execution time. The benchmark was performed on the 10-dimensional Rastrigin function, for both v4.5 and v4.6, with three different random seeds. The shaded areas represent the standard error from these three seeds.

Full Support for Multi-objective and Constrained Optimization in AutoSampler

AutoSampler is the most popular package in OptunaHub, enabling efficient optimization by automatically selecting the best algorithm. In Optuna v4.6, we have fully implemented sampler selection rules for multi-objective and constrained optimization (Figure 4). The latest AutoSampler now strategically uses samplers like GPSampler for constrained and multi-objective optimization (implemented across Optuna v4.2-v4.5) and NSGAIIISampler for many-objective optimization, leading to significant improvements in benchmark performance.

For more details, please see our blog post, “AutoSampler: Full Support for Multi-Objective & Constrained Optimization.”

Figure 4: The new AutoSampler fully supports multi-objective and constrained optimization.

Additions of Robust Bayesian Optimization Packages

Robust Bayesian optimization is a method for ensuring that optimal solutions obtained through simulations can be stably reproduced in real-world experiments, even when parameter values are noisy. In practice, maintaining experimental parameters (e.g., temperature and pressure) precisely is challenging and can lead to variations in results. To address this issue, research and development are underway to develop a framework for finding optimal solutions in which the objective function’s value does not degrade significantly even when the set values deviate slightly.

In Optuna v4.6, we have made CARBO and Value at Risk (VaR), which are robust Bayesian optimization methods developed in the fields of experimental chemistry and machine learning, respectively, available via OptunaHub. As shown in Figure 5, the introduced methods can perform robust optimization even in the presence of input noise.

Figure 5: Results of constrained robust Bayesian optimization by CARBO (for details on the problem setup, please see the PR). On the contour plot, blue indicates a good objective value, while yellow indicates a poor one. The region where constraints are not satisfied is shaded. The yellow point is the robust parameter discovered by CARBO, and the light blue box represents the range of parameter values that the yellow point can take due to noise. The figure shows that the robust solution almost always satisfies the constraints and achieves a good objective value, even in the presence of noise.

Other Updates

Tutorial for Monitoring Optuna Dashboard with OpenTelemetry

We have released a new tutorial, “Tutorial: Integrating with OpenTelemetry,” for monitoring Optuna Dashboard using OpenTelemetry. OpenTelemetry is a framework that simplifies the monitoring and tracking of performance and resource usage in applications and systems. By integrating Optuna Dashboard with OpenTelemetry (Figure 6), you can:

• Monitor HTTP request metrics (latency, error rate, throughput).
• Track the performance of database queries.
• Export metrics to Prometheus for storage and visualization in Grafana.

If you are interested in monitoring your Optuna Dashboard, we encourage you to try it out.

Figure 6: Diagram of the integration between Optuna Dashboard and OpenTelemetry.

LaTeX Support on OptunaHub Package Detail Pages

We’ve added support for LaTeX mathematical expressions on the package detail pages in OptunaHub. With this update, package developers can now use both inline ($expr$, `$expr$`) and block ($$expr$$) expressions with GitHub-compatible syntax. For users, this dramatically improves documentation readability, as formulas are now beautifully rendered. Figure 7 shows an example of formulas from the CARBOSampler’s package detail page.

Figure 7: Example of LaTeX formulas on a package detail page.

Optuna MCP Server Now Supports Structured Output

Optuna MCP Server now supports Structured Output, a feature introduced in the latest Model Context Protocol specification (2025–06–18). This enhancement improves stability by automatically validating the results returned from the Optuna MCP Server to the MCP client against an output schema generated from type annotations. For more technical details, please refer to the official documentation.

Figure 8: Example of Structured Output in Optuna MCP Server. (Left) In optuna-mcp 0.1.2, the study name was embedded in text, requiring the LLM to recognize and extract it. (Right) In optuna-mcp 0.2.0, the study name is defined in a JSON schema, allowing for easy retrieval and validation.

What’s Ahead

In this article, we have introduced the release highlights of Optuna v4.6. These highlights are directly tied to the three core concepts of the Optuna v5 roadmap, which we unveiled this past May.

Unleashing the Potential of Black-box Optimization with Generative AI

• Introduction of Optuna Dashboard LLM Integration
• Structured Output Support for Optuna MCP Server

Making Optuna Available Anytime, Anywhere

• Tutorial for Monitoring Optuna Dashboard with OpenTelemetry

Continuing the Sustainable Growth of Optuna

• Speeding up GPSampler
• Full Support for Multi-Objective and Constrained Optimization in AutoSampler
• Additions of Robust Bayesian Optimization Packages
• LaTeX Support on OptunaHub Package Detail Pages

We are preparing to change the default sampler, prototyping performance improvements by replacing components with Rust, and validating prompt-optimization techniques. We plan to continue releasing exciting new features as we move toward Optuna v5, so please stay tuned!

Contributors

The v4.6 release includes contributions from the following contributors. Thank you all!

@AddyM, @GabrielRomaoG, @Jongwan93, @Kaichi-Irie, @ParagEkbote, @Zrahay, @c-bata, @contramundum53, @dross20, @euangoodbrand, @fusawa-yugo, @gen740, @jaikumarm, @kAIto47802, @ktns, @nabenabe0928, @nihalsiddiqui7, @not522, @satyarth7srivastava, @sawa3030, @toshihikoyanase, @unKnownNG, @y0z