Releases: facebook/Ax
Releases · facebook/Ax
V0.4.3 - Maintenance Release
Key things:
- Update to BoTorch 0.12.0 – which included some major updates to underlying modeling, please see BoTorch release for detailed description
- Fix for a sql storage bug related to an import trying to import a deleted file
Compatibility
- BoTorch 0.12.0 -- which included some major updates to underlying modeling, please see BoTorch release for detailed description
- pin numpy to < 2.0 until tensorboard cuts a release (#2641)
- Make SQLAlchemy==1.4.17 a required dependency (#2690)
New Features
- AnalysisCard refactor (#2589)
- New parallel coordinates plot (#2590)
- Add an optimal_value to mixed-integer problems (#2596)
- Add AutoTransitionAfterGenCriterion to storage (#2614)
- Storage support for RBFKernel & LogNormalPrior (#2616)
- Add storage support for SobolQMCNormalSampler (#2622)
- AuxiliaryExperiment (#2632)
- Add auxiliary_experiments to Experiment (#2634)
- Create Scheduler.generate_candidates() function (#2640)
- AnalysisCard load/save methods (#2645)
- Healthcheck analysis class (#2646)
- add auxiliary experiments to SQAExperiment (#2658)
- Add Scheduler.compute_analyses method (#2660)
- Change time limit for unit tests to 60 seconds and introduce a decorator for long running tests (#2661)
- Add utility methods to Analysis (#2663)
- Introduce ParamBasedTestProblem for benchmarking (#2675)
- Run ModelLauncher with a BenchmarkRunner and BenchmarkMetric (#2681)
- Add init_position argument to UniformGenerator (#2686)
- Run Torchvision problems with Benchmark[Problem/Runner/Metric]; consolidate PyTorchCNN problems (#2688)
- allow more candidate trials than max_trials in Scheduler (#2689)
- create utility for trying to load GS (#2694)
- Implement Data/MapData.clone and use it in Experiment.clone_with (#2703)
- Compute ground-truth optimization trace on BenchmarkProblem (#2704)
- Add evaluate_oracle method to BenchmarkRunner (#2705)
- Merge create_single_objective_problem_from_botorch and create_multi_objective_problem_from_botorch; support constrained MOO (#2722)
- Add an option to override ModelSpec.model_key (#2726)
- Update test_generation_strategy to utilize MBM models (#2734)
- Refactor computation of optimization trace (#2747)
- Update the remaining models to use new default covar & likelihood modules (#2742)
- Add qLogNParEGO to ACQUISITION_FUNCTION_REGISTRY (#2749)
- Transform batch to new sq (#2755)
Bug Fixes
- Disable long test timeouts when using a debugger (#2745)
- Require all problems to have an optimal_value; simplify Problem inheritance (#2601)
- Update the default SingleTaskGP prior (#2610)
- Remove reference_point from multi-objective benchmark problems (#2619)
- More precise type annotations for MultiObjectiveOptimizationConfig (#2620)
- Change blob type to str (#2624)
- Revert D60472381 (#2627)
- Check for Nans and Infs in TensorboardMetric (#2628)
- Add warning to SerializationMixin, to prevent usage that will result in circular deps (#2629)
- Use Standardize by default for SingleTaskGP (#2630)
- Modify get_data to error out on nan/inf (#2633)
- fix bug in UnitX when transforming new search space (#2639)
- Add _maybe_clear_multiplexer method to deal with memory leaks (#2642)
- Fix Surrogate.best_out_of_sample_point (#2652)
- Fix clear_multiplexer (#2656)
- Fix Pyre errors in sqa_classes (#2659)
- Relax type hints on compute() (#2662)
- Fix type error in validate_sphinx script (#2665)
- Make sure random seed persists beyond storage (#2671)
- Log an exception rather than erroring out when data is found for metrics that are not attached to the experiment (#2680)
- fix input args for target-aware model in TLMB (#2685)
- Stop calling float() on single element Series (#2691)
- Serialize data classes based on their fields only (#2697)
- [Website] Use mathjax svg renderer (#2707)
- Take out error about HSS not being supported with batch trials (#2709)
- Make sure is_ax_equal returns a boolean (#2714)
- Remove internal type shed for plotly (#2715)
- Bring back ax/benchmark/init.py (#2719)
- BUG FIX: Only construct surrogate once in SurrogateRunner (#2720)
- Mock out downloading data from internet in torchvision unit test (#2725)
- robustify PL test (#2729)
- Require unique model_key in GenerationNode, clean up model selection errors (#2730)
- Add longtest decorator to pairwise model tests (#2731)
- Ensure new arm names do not match a different name on the experiment (#2732)
- Ignore fixed parameters for search space dimensionality computation (#2765)
- In ‘best_point’ don’t require GeneratorRun to have best_arm_predictions to predict from model (#2767)
- Fix Scheduler test_run_trials_and_yield_results_with_early_stopper that was timing out the CI (#2770)
- Make Acquisition.optimize work with discrete optimizer regardless of whether raw_samples is in optimizer_options (#2772)
Other Changes
- Document setup_import_mocks (#2736)
- Rename maximum_hypervolume to optimal_value in MOO Benchmark Problems (#2598)
- Make all benchmark problems take an observe_noise_stds argument (#2600)
- Move lazy construction of a surrogate from problem to runner (#2603)
- Update BestModelSelector to return a ModelSpec rather than the index (#2609)
- Add STATUSES_EXPECTING_DATA to trials consts (#2621)
- Drop critertion from AutoTransitionAfterGenCriterion name (#2625)
- Expanded testing get_pbo_experiment testing stub (#2631)
- Use reduced state for axolotl (#2637)
- AnalysisCard encoder/decoder refactor (#2643)
- SQAAnalysisCard refactor (#2644)
- codemod pep585_imports (#2648)
- Add Sphinx doc to Healthcheck analysis (#2649)
- Pass botorch_model_class to Surrogate._set_formatted_inputs (#2653)
- [Move model input constructor helper functions off of Surrogate, to standalone functions](htt...
V0.4.2 - Maintenance Release
Key things:
- Update to BoTorch 0.12.0 – which included some major updates to underlying modeling, please see BoTorch release for detailed description
- Fix for a sql storage bug related to an import trying to import a deleted file
Compatibility
- BoTorch 0.12.0 -- which included some major updates to underlying modeling, please see BoTorch release for detailed description
- pin numpy to < 2.0 until tensorboard cuts a release (#2641)
- Make SQLAlchemy==1.4.17 a required dependency (#2690)
New Features
- AnalysisCard refactor (#2589)
- New parallel coordinates plot (#2590)
- Add an optimal_value to mixed-integer problems (#2596)
- Add AutoTransitionAfterGenCriterion to storage (#2614)
- Storage support for RBFKernel & LogNormalPrior (#2616)
- Add storage support for SobolQMCNormalSampler (#2622)
- AuxiliaryExperiment (#2632)
- Add auxiliary_experiments to Experiment (#2634)
- Create Scheduler.generate_candidates() function (#2640)
- AnalysisCard load/save methods (#2645)
- Healthcheck analysis class (#2646)
- add auxiliary experiments to SQAExperiment (#2658)
- Add Scheduler.compute_analyses method (#2660)
- Change time limit for unit tests to 60 seconds and introduce a decorator for long running tests (#2661)
- Add utility methods to Analysis (#2663)
- Introduce ParamBasedTestProblem for benchmarking (#2675)
- Run ModelLauncher with a BenchmarkRunner and BenchmarkMetric (#2681)
- Add init_position argument to UniformGenerator (#2686)
- Run Torchvision problems with Benchmark[Problem/Runner/Metric]; consolidate PyTorchCNN problems (#2688)
- allow more candidate trials than max_trials in Scheduler (#2689)
- create utility for trying to load GS (#2694)
- Implement Data/MapData.clone and use it in Experiment.clone_with (#2703)
- Compute ground-truth optimization trace on BenchmarkProblem (#2704)
- Add evaluate_oracle method to BenchmarkRunner (#2705)
- Merge create_single_objective_problem_from_botorch and create_multi_objective_problem_from_botorch; support constrained MOO (#2722)
- Add an option to override ModelSpec.model_key (#2726)
- Update test_generation_strategy to utilize MBM models (#2734)
- Refactor computation of optimization trace (#2747)
- Update the remaining models to use new default covar & likelihood modules (#2742)
- Add qLogNParEGO to ACQUISITION_FUNCTION_REGISTRY (#2749)
- Transform batch to new sq (#2755)
Bug Fixes
- Disable long test timeouts when using a debugger (#2745)
- Require all problems to have an optimal_value; simplify Problem inheritance (#2601)
- Update the default SingleTaskGP prior (#2610)
- Remove reference_point from multi-objective benchmark problems (#2619)
- More precise type annotations for MultiObjectiveOptimizationConfig (#2620)
- Change blob type to str (#2624)
- Revert D60472381 (#2627)
- Check for Nans and Infs in TensorboardMetric (#2628)
- Add warning to SerializationMixin, to prevent usage that will result in circular deps (#2629)
- Use Standardize by default for SingleTaskGP (#2630)
- Modify get_data to error out on nan/inf (#2633)
- fix bug in UnitX when transforming new search space (#2639)
- Add _maybe_clear_multiplexer method to deal with memory leaks (#2642)
- Fix Surrogate.best_out_of_sample_point (#2652)
- Fix clear_multiplexer (#2656)
- Fix Pyre errors in sqa_classes (#2659)
- Relax type hints on compute() (#2662)
- Fix type error in validate_sphinx script (#2665)
- Make sure random seed persists beyond storage (#2671)
- Log an exception rather than erroring out when data is found for metrics that are not attached to the experiment (#2680)
- fix input args for target-aware model in TLMB (#2685)
- Stop calling float() on single element Series (#2691)
- Serialize data classes based on their fields only (#2697)
- [Website] Use mathjax svg renderer (#2707)
- Take out error about HSS not being supported with batch trials (#2709)
- Make sure is_ax_equal returns a boolean (#2714)
- Remove internal type shed for plotly (#2715)
- Bring back ax/benchmark/init.py (#2719)
- BUG FIX: Only construct surrogate once in SurrogateRunner (#2720)
- Mock out downloading data from internet in torchvision unit test (#2725)
- robustify PL test (#2729)
- Require unique model_key in GenerationNode, clean up model selection errors (#2730)
- Add longtest decorator to pairwise model tests (#2731)
- Ensure new arm names do not match a different name on the experiment (#2732)
- Ignore fixed parameters for search space dimensionality computation (#2765)
- In ‘best_point’ don’t require GeneratorRun to have best_arm_predictions to predict from model (#2767)
- Fix Scheduler test_run_trials_and_yield_results_with_early_stopper that was timing out the CI (#2770)
- Make Acquisition.optimize work with discrete optimizer regardless of whether raw_samples is in optimizer_options (#2772)
Other Changes
- Document setup_import_mocks (#2736)
- Rename maximum_hypervolume to optimal_value in MOO Benchmark Problems (#2598)
- Make all benchmark problems take an observe_noise_stds argument (#2600)
- Move lazy construction of a surrogate from problem to runner (#2603)
- Update BestModelSelector to return a ModelSpec rather than the index (#2609)
- Add STATUSES_EXPECTING_DATA to trials consts (#2621)
- Drop critertion from AutoTransitionAfterGenCriterion name (#2625)
- Expanded testing get_pbo_experiment testing stub (#2631)
- Use reduced state for axolotl (#2637)
- AnalysisCard encoder/decoder refactor (#2643)
- SQAAnalysisCard refactor (#2644)
- codemod pep585_imports (#2648)
- Add Sphinx doc to Healthcheck analysis (#2649)
- Pass botorch_model_class to Surrogate._set_formatted_inputs (#2653)
- [Move model input constructor helper functions off of Surrogate, to standalone functions](h...
V0.4.1 - Maintenance Release
Compatibility
- Require BoTorch version v0.11.3 (#2594)
New Features
- Add TimeAsFeature transform (#2438)
- Add tutorial to mimic parameter constraint functionality for nonlinear constraints (#2500)
- Add option for using posterior predictive in cross-validation (#2517)
- Refactor BestModelSelctor to operate on ModelSpecs (#2557)
- Templates for questions on github (#2575)
Bug Fixes
- Update tests so that data contains metric_names that are on the corresponding experiments (#2422)
- Use name of enum instead of casting to string (#2423)
- Fill missing values with NaN in observation_data_to_array (#2430)
- Prevent unused NaNs from marking constraints as infeasible in pareto_frontier_evaluator (#2431)
- Add copyright comment (#2445)
- Fix various spelling errors in GS (#2448)
- Add warning when metric is skipped because it’s not on the experiment (#2449)
- Stop ignoring unused keyword arguments (#2451)
- Improve error clarity in TensorboardMetric (#2456)
- Save status quo name and feature when multiple status quo present (#2457)
- Remove unused import from MOO tutorial (#2462)
- Update MBM tutorial with modern, non-deprecated functionality and accurate description of defaults (#2466)
- Include CANDIDATE trials in pending_features (#2482)
- Test that we’re not referencing same parameter dict in ObservationFeatures.from_arm (#2483)
- Do not deduplicate against FAILED in GenerationStrategy/Node (#2488)
- Use updated BoTorch HitAndRunPolytopeSampler for Sobol fallback (#2492)
- Fix test_optimize_l0_homotopy test numerics (#2493)
- Remove unused imports from ax/ (#2501)
- Remove unused experiment arg from compute_model_fit_metrics_from_modelbridge (#2504)
- Fix equality check for floats (#2507)
- Fix MOO tutorial (#2509)
- Remove unnecessary pyre-fixme (#2513)
- Disallow modeling metrics not attached to the experiment (#2518)
- Fix pending point bug in ExternalGenerationNode (#2521)
- Fix num_bayesopt_trials logic to support ensembling (#2522)
- Update np.Inf -> np.inf per numpy 2.0 release (#2527)
- Reinforce testenv in Ax test case (#2530)
- Avoid errors in telemetry due to node-based GenerationStrategy (#2554)
- Use fast_botorch_optimize to fix test timeouts (#2556)
- Make MBM acquisition pass the correct argos to each BoTorch optimizer (#2571)
- Remove unnecessary special-casing for qEHVI and qMES from optimizer argparse (#2572)
- Remove link to the Raytune Tutorial (#2578)
- Expand trials_as_df to support multiple GenerationNodes in a single trial (#2577)
- Fixing typo in docstring of get_sobol_botorch_modular_acquisition (#2583)
- Fix incorrect status quo update warning (#2586)
Other Changes
- Update PairwiseModelBridge to support non-preference outcomes (#2424)
- Allow more flexible definition of which trial statuses to fit (#2432)
- Support multiple node generation for single trial (#2428)
- Add TimeAsFeature to transform registery (#2439)
- Handle pending observations in TimeAsFeature (#2440)
- Handle ObservationFeatures without trial_index in Realativize (#2441)
- Pyre update (#2442)
- Creating preference testing stub (#2443)
- Add qExpectedUtilityOfBestOption to test_PairwiseModelBridge (#2444)
- Add transition edges property to GenerationNode (https://github.com/facebook/Ax/pull/24460)
- Update should_transition_to_next_node to handle multiple transition edges (#2447)
- Rename TaskEncode transform to TaskChoiceToIntTaskChoice (#2450)
- Explicit mention that AxService API is recommended throughout docs (#2453)
- Add validation of transition edges to GS (#2454)
- Another pyre update (#2463)
- Always consider choice parameters with 2 values as ordered (#2464)
- Fix forward (#2465 )
- Change flag name from supress_generation_errors to raise_generation_errors (#2468)
- Improve typing in SyntheticFunctions (#2470)
- Clean up unnecessary todos in _get_model_state_from_last_generator_run (#2476)
- Update docstring to signal in-place_modification (#2477)
- Test against python 3.12 (#2478)
- Have MBM only pass acqf input constructor kwargs to BoTorch when they are not None (#2480)
- Enable evaluating Branin test experiments with None status quo (#2489)
- Remove unused suppress_all_errors kwarg (#2491)
- Adding analysis_batch class to ax.analysis (#2498)
- Adding schema for AnalysisBatch and Analysis (#2499)
- Passing current_node_name when gs is node based (#2505)
- Put model fit data in gen_metadata (#2511)
- Save full[er] GeneratorRuns ([https://github....
V0.4.0 - Maintenance Release
V0.4.0 - Maintenance Release
Compatibility
FYI on future deprecations:
- ALEBO AND REMBO will be deprecated and removed in the next release
New Features
- New, simplified TensorboardMetric (#2236)
- Add support for noisy problems to the Ax Benchmarks (#2255)
- Add ExternalGenerationNode (#2266)
- Add ExternalGenerationNode tutorial (#2281)
- Enable batched benchmarks (with BatchTrial) (#2331)
- Entropy of observations metric (#2340)
- Global Sensitivity Analysis for Categorical Features (#2357)
- Enable Sobol sensitivity analysis for SAAS models (#2363)
- Support fixed features in Service API (#2372)
- Support X_observed=None in Acquisition (#2393)
- Added PredictedOutcomesDotPlot to ax.analysis (#2303)
Bug Fixes
- Address misc lint+pyre failures impacting OSS linter (#2244)
- Improve flakey test reliability (#2246)
- Ensure minimize is passed to make_experiment (#2251)
- Fix GenerationNode transition logic (#2253)
- Handle errors in score trace computations (#2263)
- Fix Sphinx build errors (#2267)
- Fix flaky test for sensitivity analysis (#2268)
- Do not re-attach the same data in get_test_map_data_experiment (#2273)
- Fic issue with SQA storage never removing Data objects + upgrade testing for scheduler with Map Data intermediate results (#2276)
- Fix prediction on training data in transformed space when calculating model fit quality metrics (#2279)
- Fix integration test (#2282)
- do not group by time cols when creating observations (#2293)
- Fix test_compare_to_baseline_equal (#2296)
- Fix test_sqa_storage_map_metric_experiment (#2297)
- Ax Trial: Bug fix for error message that references nonexistent function (#2304)
- Cloning over _time_created when cloning Experiment (#2307)
- Fix "cannot collect test class 'TestMetric' because it has a init constructor (from: ax/core/tests/test_experiment.py)" (#2308)
- Address non-determinism in model.metric_names in tests (#2309)
- fix skip_runners_and_metrics for metrics on generator runs with mutable multi-objective optimization config (#2312)
- Properly handle generators in Data.from_multiple (#2318)
- Update some test helpers in core_stubs (#2319)
- Rework TransitionCriterion storage to remove circular dep (#2320)
- Fix generation node tests (#2322)
- Fix docstring in Ax SyntheticFunction._f, Pyre fix (#2329)
- Fix missing DB update on AxClient.stop_trial_early (#2337)
- Fix deprecation warning from pandas.read_json (#2346)
- Remove unused import (#2348)
- Add an error if "use_batch_trials" is passed to AxClient (#2355)
- Add List[str] to TConfig definition (#2360)
- HSS: only check for dependents if the root parameter is present when casting parameterization (#2361)
- Only untransform objective thresholds in TorchModelbridge.gen if they are not None (#2374)
- Make sure TestCase.setUp is executed in tests (#2384)
- Do not transform search space in-place in Modelbridge._get_transformed_gen_args (#2386)
- Fix warning filters in TestCase, introduce AxParameterWarning (#2389)
- Fix broken test from D56359739 (#2400)
- Fix sphinx build (#2419)
Other Changes
- Add output_tasks to MTGP in MBM (#2241)
- Pyre Configurationless migration (#2243, #2261, #2359, #2368)
- Update docstring for GenerationNode.gen & fit (#2245)
- Implement Data.eq (#2247)
- copying "cross_validation_helper" code from ax.plot (#2249)
- validate metrics after setting options (#2250)
- Introduces AxGenerationException to facilitate exception handling (#2254)
- Light cleanup of GenerationStrategyInterface (#2256)
- Add Sobol benchmark method (#2257)
- Light GenStrategy cleanup (#2258)
- Extend docstring of extract_parameter_constraints (#2262)
- Retain original data timestamp in experiment.clone_with (#2269)
- Docstring clarification (#2270)
- Clean up too-verbose logs (#2275)
- Add fit_out_of_design to TorchOptConfig (#2277)
- Update doc strings on TransitionCriterion to improve usability (#2278)
- split common target into native/non-native parts (#2280)
- Unblock node based GS in AxClient.get_optimization_trace (#2283)
- Allow customizing num_init_trials in MBM benchmark method (#2286)
- Update log message in WithDBSettingsBase._load_experiment_and_generation_strategy (#2287)
- Abstract out attribute copy during clone_to (#2288)
- updates trial status during clone (#2290)
- Plot top n features in countours (#2291)
- Expose fit_out_of_design (#2292)
- Clean up too verbose logs pt. 2 (#2298)
- Changed concatenation to f-strings (#2300)
- Silence input normalization warnings in cross validation (#2310)
- Clean up GenNode class 1/3 (#2313)
- Return weights from Acquisition.optimize (#2314)
- Upgrade actions/checkout & actions/setup-python (#2315)
- Rename OrderedChoiceEncode => OrderedChoiceToIntegerRange (#2323)
- Tests for sequential=True as default for models in ModelBridge factory (#2324)
- add MergeRepeatedMeasurements to the transform registry (#2325)
- Typing improvements to RangeParameter (#2327)
- update run_metadata in BaseTrial.run instead of overwriting (#2328)
- Allow for passing model_gen_kwargs in benchmarks (#2336)
- only infer reference point in global stopping if there is data (#2338)
- Allow bulk_fetch_trial_data to return mix of successes/failures (#2339)
- Tests which store and load dataframe and figure (#2344)
- Add e2e tests with HSS (#2345)
- Switch legacy LCE-A to qLogNEI from legacy ei_or_nei (#2352)
- Remove fall-back to list (#2353)
- Rename global stopping tests_strategies -> test_strategies (#2354)
- Update HSS dummy value logic & expose it in Cast ([#2362](https://git...
v0.3.7 - Maintenance Release
- Bump required Botorch version to v0.10.0
- New SLURM (via https://github.com/facebookincubator/submitit) tutorial notebook
- Miscellaneous big fixes and improvements
v0.3.6 - Maintenance Release
Compatibility
New Features
- Allow batch trial to be constructed with a list of
GeneratorRun
s (#1995). - Add
label_dict
to tile plots (#2007). - Augment
exp_to_df
with a "reason" column, improve readability of "feasibility" column (#1973, #2047). - Create output message comparing baseline to optimal result in
report_utils.py
(#1997, #1998, #2016, #2025, #2031, #2042, #2046, #2050). - Allow custom search spaces in
get_experiment_with_observations
(#2027). - Partial support for
GenerationNode
s inGenerationStrategy
(#1985, #1986, #1991, #2002, #2003, #2018, #2019, #2024, #2033, #2034, #2045).
Bug Fixes
- Fix usage of batch shape for warp transform (#1994).
- Use default dtype in
Experiment.clone_with
(375bf47). - Change DerelativizeTransform to not use model predictions when
use_raw_status_quo
isTrue
or when the status quo is infeasible (#2036).
Deprecations
- Rename
Models.BOTORCH
toModels.LEGACY_BOTORCH
(#1981).
Other Changes
- Do not call
dataset.X
in input transform constructors (#1993). - Move pending point utils to core Ax (#2006).
- Load Experiment without runners and metrics in the case where search space and optimization config are immutable (#1656).
- Workaround for tutorial visualizations not working in colab and remote setup (#2030).
- Add
extract_pending_observations
function that auto-deploys to the correct pending points function for the use case (#2039). - Wait to re-poll if all results are
MetricFetchE
(#2055). - Benchmarks:
v0.3.5 Release
- Bump required botorch version to 0.9.4
- Miscellaneous bug fixes and improvements
v0.3.4 Release
- Bump required botorch version to 0.9.2, fixing major bug in single-objective optimization with outcome constraints, see botorch release 0.9.2 for details
v0.3.3 Release
- Remove typeguard usage in trial attaching function and replace with manual runtime type checking (this was causing errors for some users in Google Colab notebooks)
- Miscellaneous bug fixes and improvements
v0.3.2 Release
- Bump required Botorch version to v0.8.5
- Miscellaneous big fixes and improvements