Optuna for .NET
OptunaSharp is a faithful and complete .NET port of Optuna, an automatic hyperparameter optimization software framework, particularly designed for machine learning.
OptunaSharp is thread-safe and truly parallel.
nJobs runs trials on all your cores at once, with no GIL in the way, and workers on
many machines share one study, Python Optuna workers included.
OptunaSharp supports parallelism powered by:
single-threaded, reused dedicated threads or the .NET ThreadPool.
OptunaSharp is cloud-ready, with supported integrations for:
Artifact storage -
Amazon S3 ·
Azure Blob Storage ·
Google Cloud Storage ·
Cloudflare R2
Study storage -
PostgreSQL ·
MySQL ·
Redis ·
gRPC storage proxy
The compatibility target is Optuna 5.x. When OptunaSharp behavior and Optuna
behavior differ, Optuna is treated as the source of truth and OptunaSharp aligns to it.
OptunaSharp is a complete 100% port of Optuna 5.x with full and backwards compatability with optuna's integrations.
-
Define-by-run API -
Study,TrialandFrozenTrialwith everysuggest_*,Optimize, ask-and-tell, enqueued trials, multi-objective directions, constraints, user attributes and callbacks; float, int and categorical distributions with search-space inference. All of it lives in the pure-managed Core package, built on NumSharp. -
Samplers - Optuna's Random, Grid, TPE, CMA-ES, GP, QMC, BruteForce, PartialFixed, NSGA-II and NSGA-III, plus samplers Optuna does not have: AGE-MOEA-2, SMS-EMOA, MOEA/D, RVEA, LM-CMA, VkD-CMA, differential evolution, PSO, the natural evolution strategies (OpenAI-ES, SNES, xNES), TuRBO, REMBO, SAASBO and JANUS (with its DE and multi-objective variants).
- Integrations - a native scikit-learn-style
OptunaSearchCV,LightGBMTuner, ML.NET, training-loop pruning callbacks, Microsoft.Extensions.DependencyInjection registrations, and theoptunasharpcommand-line tool, a port of theoptunaCLI.
- Integrations - a native scikit-learn-style
-
Python bridges - OptunaHub packages, BoTorch, pycma and pymoo, driven in-process through pythonnet.
-
Pruners - Median, Percentile, SuccessiveHalving, Hyperband, Threshold, Patient, Wilcoxon and Nop.
-
Analysis - parameter importance (fANOVA, mean decrease impurity, PED-ANOVA and SHAP, all pure-managed and byte-exact), terminators, and the data behind all 12
plot_*functions, which the Plotly.NET package draws as interactive figures. -
Storage - in-memory and journal-file storage in Core; SQLite, PostgreSQL, MySQL, Redis journals and a gRPC storage proxy as companion packages. Python Optuna can work on the same study through the journal file, any of the databases, Redis or the gRPC proxy. On the SQLite, PostgreSQL and MySQL storages, heartbeats with
RetryFailedTrialCallbackretry the trials of a worker that died. -
Artifacts - a file-system store in Core; Amazon S3, Google Cloud Storage, Azure Blob Storage and Cloudflare R2 as companion packages.
-
Parallel and thread-safe -
Optimize(nJobs: n)runs n trials at once, with no GIL in the way. Measured on 32 logical CPUs, 64 trials of a ~50 ms CPU-bound objective finished 6.7× faster atnJobs: 8and 9.2× faster atnJobs: 16than atnJobs: 1, while Python Optuna's time stays flat because the GIL runs one Python thread at a time (Performance).- Trials run on OptunaSharp's own dedicated threads by default, or on the .NET ThreadPool.
- Sampling is serial per sampler, as in Optuna, unless
SamplerConcurrency.PerWorkergives each worker its own Random, TPE, CMA-ES, VkD-CMA or Grid sampler. - A
CancellationToken, orStopSignalsfor Ctrl+C and SIGTERM, stops a run gracefully: no new trial starts, and the trials in flight finish and are stored. - Studies, trials, samplers, pruners and storages are safe to share between threads;
docs/THREADING.mdhas the design, its guarantees and its limits.
-
Reproducible and tested - a seeded run is bit-identical to Python wherever the path is deterministic, enforced by a live-Optuna parity harness (pythonnet) that asks real Optuna for each answer in-process and asserts that OptunaSharp matches. Everything builds and tests on net8.0 and net10.0 across Windows, Linux and macOS, behind strong-naming and package gates, and all 20 packages release together from one tag.
OptunaSharp has reimplemented byte-for-byte many cutting-edge papers and state of the art algorithms and will continue to doing so.
Found an incredible paper? You believe an optimizer/sampler/technique has achieved a breakthrough or has been referenced more than 50 times?
AGE-MOEA-2, SMS-EMOA, MOEA/D, RVEA, LM-CMA, VkD-CMA, differential evolution, PSO, the natural evolution strategies (OpenAI-ES, SNES, xNES), TuRBO, REMBO, SAASBO and JANUS (with its DE and multi-objective variants).
And we would love to accept more!
Optuna is an automatic hyperparameter optimization framework with an imperative, define-by-run API. The two core concepts:
- Study - an optimization session over an objective function.
- Trial - a single evaluation of the objective, during which hyperparameters are suggested.
# Optuna (Python) - the behavior OptunaSharp targets
import optuna
def objective(trial):
x = trial.suggest_float("x", -10, 10)
return (x - 2) ** 2
study = optuna.create_study(sampler=optuna.samplers.TPESampler(seed=42))
study.optimize(objective, n_trials=20)
print(study.best_value, study.best_params)The C# quick start above is the same study, written against the same define-by-run shape:
snake_case names become PascalCase (suggest_float → SuggestFloat, n_trials → nTrials),
while parameter order, defaults and error messages are kept as Optuna has them.
The documentation site, scisharp.github.io/OptunaSharp:
- Start here - Introduction · Getting Started · Key Features · FAQ
- Guides - Study and Trials · Distributions · Search Spaces · Samplers · Pruners · Multi-objective Optimization · Storage · Artifacts · Terminators · Parameter Importance · Visualization · Logging · Exceptions
- Recipes - Overview · Saving/Resuming with RDB · Multi-objective Optimization · User Attributes · Command-Line Interface · User-Defined Sampler · User-Defined Pruner · Callback for Study.Optimize · Specify Hyperparameters Manually · Ask-and-Tell Interface · Re-use the Best Trial · Journal-File Storage · Artifacts · Wilcoxon Pruner · Ablation Study with Brute Force
- Integrations - Overview
- Model training - scikit-learn-style search · ML.NET · LightGBM · Training-loop pruning
- Storage - SQLite · EF Core (PostgreSQL / MySQL) · Redis · gRPC
- Python bridges - Python.NET runtime · OptunaHub · BoTorch · PyCma · pymoo
- Cloud artifacts - Amazon S3 · Google Cloud Storage · Azure Blob Storage · Cloudflare R2
- Tools - Plotly.NET figures · Command-line tool
- Methods - Overview: a page per algorithm, each with its paper - the samplers, pruners, importance evaluators and terminators, and the operators and kernels they are built from.
- API reference - OptunaSharp, generated from the XML documentation of the Core package.
Measured on an Intel Core i9-13900K (8 performance and 16 efficiency cores, 32 logical CPUs) under Windows 11: OptunaSharp in .NET 10 Release builds, and Python 3.12.12 with Optuna 5.0.0 on the same machine. Each configuration ran in its own process, and times are medians of three runs unless a table says otherwise. Every benchmark process opted out of Windows power throttling, which otherwise moves a busy console process onto the efficiency cores after a few seconds. Absolute times depend on the hardware, so compare within a row.
64 trials of a CPU-bound objective that takes about 50 ms on one core, with RandomSampler. Each
language's loop was calibrated to ~50 ms on its own; OptunaSharp ran the sweep twice.
nJobs |
OptunaSharp | Speedup | Python Optuna 5.0 |
|---|---|---|---|
| 1 | 3.21-3.24 s | 1.0× | 3.05 s |
| 2 | 1.60-1.61 s | 2.0× | 3.08 s |
| 4 | 0.82 s | 3.9× | 3.09 s |
| 8 | 0.47-0.49 s | 6.6-6.9× | 3.12 s |
| 16 | 0.34-0.36 s | 9.0-9.4× | 3.09 s |
Python's time stays flat because the GIL runs one Python thread at a time. Trials that wait rather than
compute run all at once in both: 640 trials sleeping 100 ms took 1.26 s at nJobs: 64, and 1,280 took
1.27 s at nJobs: 128 (Python: 1.06 s and 1.13 s; ideal 1.0 s).
One trial at a time, the same seeded study in both languages, on a cheap sphere objective so that the sampler's own work dominates. OptunaSharp's times include the .NET JIT warming up.
| Study | OptunaSharp | Python Optuna 5.0 |
|---|---|---|
| TPE, 6 parameters, 500 trials | 0.61 s | 2.52 s |
| TPE multivariate, 6 parameters, 500 trials | 0.66 s | 2.42 s |
| TPE, 100 parameters, 150 trials | 1.6 s | 19.1 s |
| GP, 6 parameters, 100 trials | 3.47 s | 17.55 s |
| CMA-ES, 6 parameters, 3,000 trials | 0.74 s | 5.21 s |
| CMA-ES, 200 parameters, 100 trials | 1.0 s | 3.3 s |
| NSGA-II, 6 parameters, 3,000 trials | 0.31 s | 3.39 s |
| QMC, 6 parameters, 3,000 trials | 0.13 s | 3.49 s |
| Random, 6 parameters, 5,000 trials | 0.15 s | 1.36 s |
| Random, 1,000 parameters, 200 trials | 0.6 s | 8.4 s |
| Random, 4 parameters, 50,000 trials: each block of 10,000 | 0.08-0.25 s | 1.9-2.1 s |
Every OptunaSharp row except GP was measured again on 2026-11-09 after the Rustuna-derived speedups of plan 13 and plan 14, which keep every seeded result: the same probes built before and after plan 14, run back to back. GP moved less than the noise of a machine that was also running other work, so it keeps its earlier number.
40 float parameters in [-10, 10], seed 0, each sampler's defaults (TPE multivariate for one objective, independent for two), the workloads of Rustuna's announcement. Rustuna is the Optuna team's Rust port. OptunaSharp ran on 2026-11-09 after plan 14, on a machine also running other work (about 30 % CPU load), the other two on 2026-11-08 on a quieter one, so OptunaSharp's column errs on the slow side. OptunaSharp's times include the .NET JIT warming up.
| Study | OptunaSharp | Python Optuna 5.0 | Rustuna 0.2.1 |
|---|---|---|---|
| TPE, 1,000 trials | 3.2 s | 35.2 s | 1.36 s |
| TPE, two objectives, 1,000 trials | 10.5 s | 163 s | 6.30 s |
| CMA-ES, 10,000 trials | 3.6 s | 56.9 s | 1.59 s |
| CMA-ES, 100,000 trials | 22.1 s | not run | 13.6 s |
| NSGA-II, 10,000 trials | 1.4 s | 44.2 s | 0.82 s |
| NSGA-II, 100,000 trials | 8.6 s | not run | 19.4 s |
OptunaSharp's seeded trials are Optuna's; Rustuna's differ from the first trial (its own random generator). Where OptunaSharp is still behind Rustuna, part of the gap is the protocol it keeps so Python workers can share a study: CMA-ES pickles its state into the storage every generation, and TPE scores candidates in NumPy's order. The Performance and Compatibility guide lists the opt-in switches that trade that compatibility for speed.
For memory, new InMemoryStorage(compactFinishedTrials: true) keeps finished trials compactly and gives the same
trials: the 100,000-trial CMA-ES study above keeps 257 MB instead of 1,287 MB once finished, and NSGA-II's 88 MB
instead of 671 MB, in the same time. Finished trials then hand out read-only views instead of dictionaries; the
guide lists what that changes.
Trials run on OptunaSharp's own dedicated threads by default, or on the .NET ThreadPool
(TrialExecutorKind.DedicatedThreads or TrialExecutorKind.ThreadPool). Three runs of each:
| Workload | Dedicated threads | ThreadPool |
|---|---|---|
5,000 cheap Random trials, nJobs: 8 |
9.5k-10.2k trials/s | 9.2k-10.0k trials/s |
2,000 TPE trials, nJobs: 8 |
512-570 trials/s | 499-526 trials/s |
200 calls of Optimize(nTrials: 8, nJobs: 8) |
1,035-1,115 calls/s | 973-1,131 calls/s |
128 trials blocking 1.5 s, nJobs: 64 (ideal 3.0 s) |
3.2 s | 6.2 s |
Longest delay before other Task.Run work started, nJobs: 64 / 32 |
2.2 ms / 2.4 ms | 1.6 s / 0.74 s |
The two match on throughput. Once nJobs reaches the pool's ready threads (32 on this machine), the
ThreadPool runs fewer trials at once and delays the rest of the application, so it suits an nJobs well
below the core count.
nJobs: 8, best of three runs: one sampler shared by every worker, taking one call at a time
(SamplerConcurrency.Shared, Optuna's behaviour and the default), against
one sampler per worker (SamplerConcurrency.PerWorker):
| Study | Shared | PerWorker |
|---|---|---|
| TPE multivariate, 10 parameters, 600 trials, 5 ms objective | 1.39 s | 1.15 s |
| CMA-ES, 40 parameters, 2,000 trials, cheap objective | 2.17 s | 0.82 s |
| TPE, 3 parameters, 400 trials, 20 ms objective | 1.54 s | 1.54 s |
A sampler per worker pays where sampling dominates and changes nothing when the objective does. Slow
model-based samplers stay shared: a GPSampler study of 120 trials with a 20 ms objective took 7.53 s at
nJobs: 1 and 9.30 s at nJobs: 8.
dotnet add package OptunaSharpThe package targets .NET 8 and .NET 10. Storages,
cloud artifact stores, Plotly figures, the CLI and
the Python bridges are separate OptunaSharp.* packages (listed in
the package README).
using OptunaSharp;
using OptunaSharp.Samplers;
var study = Optuna.CreateStudy(sampler: new TPESampler(seed: 42));
study.Optimize(trial =>
{
double x = trial.SuggestFloat("x", -10, 10);
return (x - 2) * (x - 2);
}, nTrials: 20);
Console.WriteLine(study.BestValue); // 0.0004973004749973951
Console.WriteLine(study.BestParams["x"]); // 2.0223002348641756Those two numbers are exactly what Python Optuna 5.0.0 prints for the same seeded study.
OptunaSharp.Core depends on NumSharp and its native LAPACK/CBLAS backend
NumSharp.Interop.OpenBLAS, both consumed as NuGet packages from nuget.org (version pinned by
the NumSharpVersion property in Directory.Build.props). Build and CI restore them from nuget.org;
the OpenBLAS package ships the native LAPACK binaries (in runtimes/<rid>/native/), so np.linalg
computes through LAPACK and is bit-identical to NumPy - no extra setup.
The matching NumSharp source is vendored reference-only as a pinned git submodule at refs/NumSharp
(the same release tag), for browsing - it is not part of the build. Restore it, if you want it, with
git submodule update --init refs/NumSharp.
# Build everything
dotnet build SciSharp.OptunaSharp.sln -c Release
# Unit tests (both TFMs), excluding known-failing/heavy/slow categories as per-push CI does
# (drop `&TestCategory!=Slow` for the full run)
dotnet test test/OptunaSharp.Tests/OptunaSharp.Tests.csproj -c Release \
--filter "TestCategory!=OpenBugs&TestCategory!=HighMemory&TestCategory!=RequiresPython&TestCategory!=Slow"
# Live-Optuna interop: needs a discoverable Python with the vendored Optuna installed, which is what
# the parity is measured against (`git submodule update --init refs/optuna`, then
# `pip install -e refs/optuna`)
dotnet test test/OptunaSharp.Tests.Interop/OptunaSharp.Tests.Interop.csproj -c Release
# Pack the OptunaSharp package
dotnet pack src/OptunaSharp.Core/OptunaSharp.Core.csproj -c Release -o packagesTest categories (OpenBugs, Misaligned, RequiresPython, HighMemory, WindowsOnly, Slow) are typed
attributes in test/OptunaSharp.Tests/TestCategory.cs, so CI filter strings stay stable as the
suite grows.
Internal members are exposed to a dotnet run script that overrides its assembly name to a declared
friend (see src/OptunaSharp.Core/Assembly/Properties.cs). A script under the repo root inherits
Directory.Build.props and is signed automatically; one run from elsewhere must pass the signing
properties itself:
#:project <path-to-your-clone>/src/OptunaSharp.Core/OptunaSharp.Core.csproj
#:property AssemblyName=OptunaSharp.DotNetRunScript
#:property PublishAot=false
#:property SignAssembly=true
#:property AssemblyOriginatorKeyFile=<path-to-your-clone>/Open.snkMIT - see LICENSE. OptunaSharp is a derivative of Optuna (© Preferred Networks, Inc.) and retains its copyright notice as the MIT License requires.
- NumSharp - NumPy for .NET (OptunaSharp's numerical foundation)
- The SciSharp STACK - open-source machine learning for .NET