Update #711 checking if OptimizationFunction is used for derivative based optimizers #711 #715

ParasPuneetSingh · 2024-03-05T20:02:05Z

Added checking if OptimizationFunction is used for derivative based optimizers #711

Adding trait for requiring derivatives and throwing error to use OptimizationFunction

Update OptimizationOptimJL.jl

codecov · 2024-03-06T16:24:12Z

Codecov Report

Attention: Patch coverage is 0% with 50 lines in your changes are missing coverage. Please review.

Project coverage is 11.37%. Comparing base (80f2155) to head (e0610cd).
Report is 2 commits behind head on master.

Files	Patch %	Lines
lib/OptimizationNLopt/src/OptimizationNLopt.jl	0.00%	20 Missing ⚠️
lib/OptimizationOptimJL/src/OptimizationOptimJL.jl	0.00%	7 Missing ⚠️
...onStrategy/src/OptimizationCMAEvolutionStrategy.jl	0.00%	4 Missing ⚠️
...zationEvolutionary/src/OptimizationEvolutionary.jl	0.00%	4 Missing ⚠️
lib/OptimizationFlux/src/OptimizationFlux.jl	0.00%	4 Missing ⚠️
lib/OptimizationGCMAES/src/OptimizationGCMAES.jl	0.00%	4 Missing ⚠️
lib/OptimizationMOI/src/OptimizationMOI.jl	0.00%	4 Missing ⚠️
...timizationOptimisers/src/OptimizationOptimisers.jl	0.00%	1 Missing ⚠️
...onPolyalgorithms/src/OptimizationPolyalgorithms.jl	0.00%	1 Missing ⚠️
...zationSpeedMapping/src/OptimizationSpeedMapping.jl	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #715       +/-   ##
===========================================
- Coverage   66.01%   11.37%   -54.64%     
===========================================
  Files          22       22               
  Lines        1477     1512       +35     
===========================================
- Hits          975      172      -803     
- Misses        502     1340      +838

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Vaibhavdixit02

This should be set for all the packages in lib/ (i.e. the solver wrappers)

Updated the trait requiresderivative to requiresgradient.

Added requiresgradient, requireshessian, requiresconshess and requiresconsjac traits.

Added new traits: requiresgradient, requireshessian, requiresconsjac and requiresconshess.

Added the traits requiresgradient, requireshessian, requiresconsjac and requiresconshess.

Added traits requireshessian, requiresgradient, requiresconshess and requiresconsjac.

Added traits requireshessian, requiresgradient, requiresconsjac and requiresconshess.

Vaibhavdixit02 · 2024-03-10T02:59:20Z

lib/OptimizationOptimJL/src/OptimizationOptimJL.jl

@@ -14,6 +14,7 @@ SciMLBase.requiresbounds(opt::Optim.SAMIN) = true
 SciMLBase.supports_opt_cache_interface(opt::Optim.AbstractOptimizer) = true
 SciMLBase.supports_opt_cache_interface(opt::Union{Optim.Fminbox, Optim.SAMIN}) = true
 SciMLBase.supports_opt_cache_interface(opt::Optim.ConstrainedOptimizer) = true
+SciMLBase.requiresgradient(Optim.AbstractOptimizer) = !(opt isa Optim.ZerothOrderOptimizer)


Looks good, but this wrapper will need more of the traits set to true (Newton, NewtonTrustRegion and IPNewton)

And not all of them follow the abstract type, especially check the SAMIN and FminBox algorithms

Vaibhavdixit02 · 2024-03-10T03:01:38Z

lib/OptimizationBBO/src/OptimizationBBO.jl

@@ -9,6 +9,11 @@ abstract type BBO end
 SciMLBase.requiresbounds(::BBO) = true
 SciMLBase.allowsbounds(::BBO) = true
 SciMLBase.supports_opt_cache_interface(opt::BBO) = true
+SciMLBase.requiresgradient(::BBO) = false


Since the default value is false, you can only declare the ones you need to set to true here, so for example you'd not need to define anything here in OptimzationBBO since its all false

Added traits for IPNewton, Newton, and NewtonTrustRegion.

Added requiresgradient trait to NLOPT.jl

Set trait requiresgradient to true.

Vaibhavdixit02 · 2024-03-12T22:53:50Z

lib/OptimizationNLopt/src/OptimizationNLopt.jl

@@ -8,6 +8,7 @@ using Optimization.SciMLBase

 SciMLBase.allowsbounds(opt::Union{NLopt.Algorithm, NLopt.Opt}) = true
 SciMLBase.supports_opt_cache_interface(opt::Union{NLopt.Algorithm, NLopt.Opt}) = true
+SciMLBase.requiresgradient(opt::Union{NLopt.Algorithm, NLopt.Opt}) = true


It would be useful to check if NLopt has a type hierarchy we can use here for doing this better, since it has all kinds of algorithms anything we put here will be wrong - in which case I would want to set the hessian to true as well.

How about if I set the traits requiresgradient and requireshessian true for only the gradient-based algorithms (local and global). Only the derivative-free local optimizers will have these traits as false then.

That's what I am hoping we can do yeah. But you should also be able to set the requireshessian to true only for second order methods by looking at the docs of nlopt. For the derivative-free vs others you plan to use the prefix in the names of the methods?

Vaibhavdixit02 · 2024-03-12T22:55:07Z

lib/OptimizationOptimJL/src/OptimizationOptimJL.jl

@@ -128,7 +136,8 @@ function SciMLBase.__solve(cache::OptimizationCache{
    local x, cur, state

    cur, state = iterate(cache.data)
-
+    (requiresgradient(Optim.AbstractOptimizer))


This should be done in the solve defined in SciMLBase, I'll point you to it later once this PR is done, you can remove this error from here

Set traits requiresgradient and requiresconsjac to true for certain algorithms.

Set trait requiresgradient to true.

Removed redundant traits declared.

Set traits requiresgradient, requireshessian and requiresconsjac to true based on the algorithm.

Updated traits for Newton and NewtonTrustRegion as they are second order, they automatically require the calculation of hessians.

Updated traits.

Set the requiresgradient for Fminbox to true as it is required for the objective function evaluation and constraint handling.

Set traits requiresgradient, requireshessian, requiresconshess and requiresconsjac to true.

Removed redundant check for requiresgradient trait.

lib/OptimizationOptimJL/src/OptimizationOptimJL.jl

Vaibhavdixit02 · 2024-04-01T19:49:24Z

@ParasPuneetSingh there's a few errors here please fix them from the CI run it seems like syntax issues

Corrected syntax errors.

Fixed syntax errors.

ParasPuneetSingh · 2024-04-03T09:03:09Z

I checked the Cl run and corrected the syntax errors in OptimizationNLopt.jl but now it says that "Optim.AbstractOptimizer" is not a valid function argument name, but it has been used for other traits as well, I don't get this syntax error.

Vaibhavdixit02 · 2024-04-03T15:50:41Z

lib/OptimizationPolyalgorithms/src/OptimizationPolyalgorithms.jl

@@ -4,6 +4,8 @@ using Reexport
 @reexport using Optimization
 using Optimization.SciMLBase, OptimizationOptimJL, OptimizationOptimisers

+SciMLBase.requiresgradient(opt::PolyOpt) = true
+
 struct PolyOpt end


PolyOpt is defined after the trait call switch around the order

Interchanged the struct definition and trait checking.

Vaibhavdixit02

Change the compat bound on SciMLBase to 2.30.0 then this should be ready to merge

Updated compat for SciMLBase

Vaibhavdixit02 · 2024-04-04T23:04:45Z

docs/Project.toml

@@ -64,7 +64,7 @@ OptimizationPolyalgorithms = "0.1, 0.2"
 OptimizationSpeedMapping = "0.1, 0.2"
 OrdinaryDiffEq = "6"
 ReverseDiff = ">= 1.9.0"
-SciMLBase = "2"


Not just in docs also the Project.toml in the base directory of the repo

Updated SciMLBase compat to 2.30.0

Project.toml

ParasPuneetSingh added 2 commits March 5, 2024 17:21

Update OptimizationOptimJL.jl

01f4479

Adding trait for requiring derivatives and throwing error to use OptimizationFunction

Merge pull request #1 from ParasPuneetSingh/ParasPuneetSingh-patch-1

404f4f3

Update OptimizationOptimJL.jl

ParasPuneetSingh changed the title ~~Update #711 Added a new trait for checking if OptimizationFunction is used for derivative based optimizers #711~~ Update #711 checking if OptimizationFunction is used for derivative based optimizers #711 Mar 5, 2024

Vaibhavdixit02 reviewed Mar 6, 2024

View reviewed changes

ParasPuneetSingh added 6 commits March 9, 2024 23:31

Update OptimizationOptimJL.jl

b2c044d

Updated the trait requiresderivative to requiresgradient.

Update OptimizationBBO.jl

ef4e4a5

Added requiresgradient, requireshessian, requiresconshess and requiresconsjac traits.

Update OptimizationCMAEvolutionStrategy.jl

ed73535

Added new traits: requiresgradient, requireshessian, requiresconsjac and requiresconshess.

Update OptimizationEvolutionary.jl

134d7fc

Added the traits requiresgradient, requireshessian, requiresconsjac and requiresconshess.

Update OptimizationFlux.jl

702e095

Added traits requireshessian, requiresgradient, requiresconshess and requiresconsjac.

Update OptimizationGCMAES.jl

6e2afe3

Added traits requireshessian, requiresgradient, requiresconsjac and requiresconshess.

Vaibhavdixit02 reviewed Mar 10, 2024

View reviewed changes

ParasPuneetSingh added 3 commits March 12, 2024 23:32

Update OptimizationOptimJL.jl

d1c0111

Added traits for IPNewton, Newton, and NewtonTrustRegion.

Update OptimizationNLopt.jl

1ba53fb

Added requiresgradient trait to NLOPT.jl

Update OptimizationOptimisers.jl

1767142

Set trait requiresgradient to true.

Vaibhavdixit02 reviewed Mar 12, 2024

View reviewed changes

ParasPuneetSingh added 9 commits March 13, 2024 14:08

Update OptimizationPRIMA.jl

da5955b

Set traits requiresgradient and requiresconsjac to true for certain algorithms.

Update OptimizationPolyalgorithms.jl

64e16ec

Set trait requiresgradient to true.

Update OptimizationSpeedMapping.jl

3ae278f

Set trait requiresgradient to true.

Update OptimizationBBO.jl

4887714

Removed redundant traits declared.

Update OptimizationNLopt.jl

a1348e3

Set traits requiresgradient, requireshessian and requiresconsjac to true based on the algorithm.

Update OptimizationOptimJL.jl

0b6f600

Updated traits for Newton and NewtonTrustRegion as they are second order, they automatically require the calculation of hessians.

Update OptimizationOptimJL.jl

6ea74cf

Updated traits.

Update OptimizationOptimJL.jl

5132aea

Set the requiresgradient for Fminbox to true as it is required for the objective function evaluation and constraint handling.

Update OptimizationMOI.jl

785c218

Set traits requiresgradient, requireshessian, requiresconshess and requiresconsjac to true.

Vaibhavdixit02 mentioned this pull request Mar 22, 2024

Don't add a differentiation algorithm to the PRIMA example #720

Closed

ParasPuneetSingh added 2 commits March 26, 2024 16:13

Update OptimizationOptimJL.jl

8f3e381

Removed redundant check for requiresgradient trait.

Update OptimizationOptimJL.jl

a5652d7

Update OptimizationOptimJL.jl

6379379

Vaibhavdixit02 reviewed Apr 1, 2024

View reviewed changes

lib/OptimizationOptimJL/src/OptimizationOptimJL.jl Outdated Show resolved Hide resolved

Update lib/OptimizationOptimJL/src/OptimizationOptimJL.jl

6fb808b

ParasPuneetSingh added 4 commits April 3, 2024 13:50

Update OptimizationNLopt.jl

ea6d522

Corrected syntax errors.

Update OptimizationOptimJL.jl

73cd120

Fixed syntax errors.

Update OptimizationOptimJL.jl

bae6741

Update OptimizationOptimJL.jl

b9854f5

Vaibhavdixit02 reviewed Apr 3, 2024

View reviewed changes

Update OptimizationPolyalgorithms.jl

15a5fa0

Interchanged the struct definition and trait checking.

Vaibhavdixit02 approved these changes Apr 4, 2024

View reviewed changes

Update Project.toml

0857a62

Updated compat for SciMLBase

Vaibhavdixit02 reviewed Apr 4, 2024

View reviewed changes

ParasPuneetSingh and others added 2 commits April 5, 2024 11:41

Update Project.toml

bb82850

Updated SciMLBase compat to 2.30.0

Merge branch 'master' into master

2d6b7c4

Vaibhavdixit02 reviewed Apr 7, 2024

View reviewed changes

Project.toml Outdated Show resolved Hide resolved

Update Project.toml

e0610cd

Vaibhavdixit02 merged commit 8320ff3 into SciML:master Apr 7, 2024
38 of 43 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update #711 checking if OptimizationFunction is used for derivative based optimizers #711 #715

Update #711 checking if OptimizationFunction is used for derivative based optimizers #711 #715

ParasPuneetSingh commented Mar 5, 2024 •

edited

Loading

codecov bot commented Mar 6, 2024 •

edited

Loading

Vaibhavdixit02 left a comment

Vaibhavdixit02 Mar 10, 2024

Vaibhavdixit02 Mar 10, 2024

Vaibhavdixit02 Mar 10, 2024

Vaibhavdixit02 Mar 12, 2024

ParasPuneetSingh Mar 13, 2024

Vaibhavdixit02 Mar 13, 2024

Vaibhavdixit02 Mar 12, 2024

Vaibhavdixit02 commented Apr 1, 2024

ParasPuneetSingh commented Apr 3, 2024

Vaibhavdixit02 Apr 3, 2024

Vaibhavdixit02 left a comment

Vaibhavdixit02 Apr 4, 2024

Update #711 checking if OptimizationFunction is used for derivative based optimizers #711 #715

Update #711 checking if OptimizationFunction is used for derivative based optimizers #711 #715

Conversation

ParasPuneetSingh commented Mar 5, 2024 • edited Loading

codecov bot commented Mar 6, 2024 • edited Loading

Codecov Report

Vaibhavdixit02 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vaibhavdixit02 commented Apr 1, 2024

ParasPuneetSingh commented Apr 3, 2024

Choose a reason for hiding this comment

Vaibhavdixit02 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ParasPuneetSingh commented Mar 5, 2024 •

edited

Loading

codecov bot commented Mar 6, 2024 •

edited

Loading