multipathfinder crashes if PosDefException is thrown #181
Comments
|
I have mixed feelings on this. We do have a resample and restart mechanism, so we could suppress errors. But the source of the error matters. We should detect and gracefully handle all errors coming from failed optimization. Clearly in this case we don't, so I'll try to identify which case we are missing. Errors in model evaluation, however, are another thing. I lean towards we should not suppress these errors, as they do reflect problems that should be resolved, and an error message won't be as useful as a traceback (FWIW Turing won't suppress these either). Instead, the model implementer should detect errors that are acceptable and work around them, if possible. I'm traveling until the end of the month so will look into this then. In the meantime, if this is a blocker for you, the issue is probably a non-finite number appearing in a position, gradient, or log-density. You may be able to detect this in a callback and return true to trigger termination. |
|
Thanks for the fast reply! I agree with you on this. Error from failed optimization should be caught, while errors from the model should be handled appropriately by the modeling package. |
|
As an update to this issue, for another ODE model with a bad startguess I encountered an Optim.jl related error (see below). This error is rare, but can happen for ODE models when having a bad startguess. ERROR: AssertionError: isfinite(phi_c) && isfinite(dphi_c)
Stacktrace:
[1] (::LineSearches.HagerZhang{…})(ϕ::Function, ϕdϕ::LineSearches.var"#ϕdϕ#6"{…}, c::Float64, phi_0::Float64, dphi_0::Float64)
@ LineSearches ~/.julia/packages/LineSearches/G1LRk/src/hagerzhang.jl:299
[2] HagerZhang
@ ~/.julia/packages/LineSearches/G1LRk/src/hagerzhang.jl:101 [inlined]
[3] perform_linesearch!(state::Optim.LBFGSState{…}, method::Optim.LBFGS{…}, d::Optim.ManifoldObjective{…})
@ Optim ~/.julia/packages/Optim/ZhuZN/src/utilities/perform_linesearch.jl:58
[4] update_state!(d::NLSolversBase.TwiceDifferentiable{…}, state::Optim.LBFGSState{…}, method::Optim.LBFGS{…})
@ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/solvers/first_order/l_bfgs.jl:204
[5] optimize(d::NLSolversBase.TwiceDifferentiable{…}, initial_x::Vector{…}, method::Optim.LBFGS{…}, options::Optim.Options{…}, state::Optim.LBFGSState{…})
@ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/optimize/optimize.jl:54
[6] optimize(d::NLSolversBase.TwiceDifferentiable{…}, initial_x::Vector{…}, method::Optim.LBFGS{…}, options::Optim.Options{…})
@ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/optimize/optimize.jl:36
[7] __solve(cache::OptimizationBase.OptimizationCache{…})
@ OptimizationOptimJL ~/.julia/packages/OptimizationOptimJL/fdBis/src/OptimizationOptimJL.jl:214
[8] solve!
@ ~/.julia/packages/SciMLBase/JUp1I/src/solve.jl:188 [inlined]
[9] #solve#625
@ ~/.julia/packages/SciMLBase/JUp1I/src/solve.jl:96 [inlined]
[10] optimize_with_trace(prob::OptimizationProblem{…}, optimizer::Optim.LBFGS{…}; progress_name::String, progress_id::Base.UUID, maxiters::Int64, callback::Nothing, fail_on_nonfinite::Bool, kwargs::@Kwargs{})
@ Pathfinder ~/.julia/packages/Pathfinder/p5Vuk/src/optimize.jl:46
[11] optimize_with_trace
@ ~/.julia/packages/Pathfinder/p5Vuk/src/optimize.jl:20 [inlined]
[12] _pathfinder(rng::Random._GLOBAL_RNG, prob::OptimizationProblem{…}, logp::Pathfinder.var"#logp#23"{…}; history_length::Int64, optimizer::Optim.LBFGS{…}, ndraws_elbo::Int64, executor::Transducers.SequentialEx{…}, kwargs::@Kwargs{…})
@ Pathfinder ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:288
[13] _pathfinder
@ ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:277 [inlined]
[14] _pathfinder_try_until_succeed(rng::Random._GLOBAL_RNG, prob::OptimizationProblem{…}, logp::Pathfinder.var"#logp#23"{…}; ntries::Int64, init_scale::Int64, init_sampler::Pathfinder.UniformSampler{…}, allow_mutating_init::Bool, kwargs::@Kwargs{…})
@ Pathfinder ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:263
[15] _pathfinder_try_until_succeed
@ ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:251 [inlined]
[16] #22
@ ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:180 [inlined]
[17] progress(f::Pathfinder.var"#22#24"{…}; name::String)
@ ProgressLogging ~/.julia/packages/ProgressLogging/6KXlp/src/ProgressLogging.jl:262
[18] progress
@ ~/.julia/packages/ProgressLogging/6KXlp/src/ProgressLogging.jl:258 [inlined]
[19] pathfinder(prob::OptimizationProblem{…}; rng::Random._GLOBAL_RNG, history_length::Int64, optimizer::Optim.LBFGS{…}, ndraws_elbo::Int64, ndraws::Int64, input::Function, kwargs::@Kwargs{…})
@ Pathfinder ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:179
[20] pathfinder
@ ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:165 [inlined]
[21] #pathfinder#20
@ ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:163 [inlined]
[22] pathfinder
@ ~/.julia/packages/Pathfinder/p5Vuk/src/singlepath.jl:142 [inlined]
[23] #29
@ ~/.julia/packages/Pathfinder/p5Vuk/src/multipath.jl:166 [inlined]
[24] next
@ ~/.julia/packages/Transducers/IAWgA/src/library.jl:54 [inlined]
[25] (::Transducers.var"#50#51"{…})(u0::Tuple{…}, iresult::BangBang.SafeCollector{…})
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/library.jl:1302
[26] wrapping
@ ~/.julia/packages/Transducers/IAWgA/src/core.jl:734 [inlined]
[27] next(rf::Transducers.Reduction{…}, result::Transducers.PrivateState{…}, input::Tuple{…})
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/library.jl:1300
[28] macro expansion
@ ~/.julia/packages/Transducers/IAWgA/src/core.jl:181 [inlined]
[29] macro expansion
@ ~/.julia/packages/Transducers/IAWgA/src/processes.jl:239 [inlined]
[30] macro expansion
@ ~/.julia/packages/Transducers/IAWgA/src/simd.jl:41 [inlined]
[31] __foldl__
@ ~/.julia/packages/Transducers/IAWgA/src/processes.jl:238 [inlined]
[32] (::Transducers.var"#257#259"{…})(id::Symbol)
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/progress.jl:95
[33] __progress(f::Transducers.var"#257#259"{…}; name::String)
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/progress.jl:74
[34] __progress
@ ~/.julia/packages/Transducers/IAWgA/src/progress.jl:70 [inlined]
[35] __foldl__
@ ~/.julia/packages/Transducers/IAWgA/src/progress.jl:81 [inlined]
[36] #transduce#141
@ ~/.julia/packages/Transducers/IAWgA/src/processes.jl:519 [inlined]
[37] transduce(rf1::Transducers.Reduction{…}, init::BangBang.SafeCollector{…}, coll::Transducers.ProgressLoggingFoldable{…})
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/processes.jl:508
[38] transduce(xform::Transducers.Composition{…}, f::Transducers.AdHocRF{…}, init::BangBang.SafeCollector{…}, coll::Transducers.ProgressLoggingFoldable{…}; kwargs::@Kwargs{})
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/processes.jl:502
[39] transduce(xform::Transducers.Composition{…}, f::Transducers.AdHocRF{…}, init::BangBang.SafeCollector{…}, coll::Transducers.ProgressLoggingFoldable{…})
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/processes.jl:500
[40] _collect(xf::Transducers.Map{…}, coll::Transducers.ProgressLoggingFoldable{…}, ::Transducers.SizeStable, ::Base.HasShape{…})
@ Transducers ~/.julia/packages/Transducers/IAWgA/src/processes.jl:806
[41] collect
@ ~/.julia/packages/Transducers/IAWgA/src/processes.jl:802 [inlined]
[42] collect(itr::Transducers.Eduction{…}, ex::Transducers.SequentialEx{…})
@ Folds.Implementations ~/.julia/packages/Folds/qbSal/src/collect.jl:10
[43] multipathfinder(optim_fun::OptimizationFunction{…}, ndraws::Int64; init::Vector{…}, input::PEtabLogDensity{…}, nruns::Int64, ndraws_elbo::Int64, ndraws_per_run::Int64, rng::Random._GLOBAL_RNG, history_length::Int64, optimizer::Optim.LBFGS{…}, executor::Transducers.SequentialEx{…}, executor_per_run::Transducers.SequentialEx{…}, importance::Bool, kwargs::@Kwargs{…})
@ Pathfinder ~/.julia/packages/Pathfinder/p5Vuk/src/multipath.jl:188
[44] multipathfinder
@ ~/.julia/packages/Pathfinder/p5Vuk/src/multipath.jl:132 [inlined]
[45] #multipathfinder#27
@ ~/.julia/packages/Pathfinder/p5Vuk/src/multipath.jl:130 [inlined]
[46] _get_starting_points(petab_problem::PEtabODEProblem{…}, dir_save::String; nstartguesses::Int64, van_der_pool::Bool)
@ Main ~/Dropbox/PhD/Projects/Bayesian_benchmark/Benchmarks/Get_startguesses.jl:41
[47] top-level scope
@ ~/Dropbox/PhD/Projects/Bayesian_benchmark/Benchmarks/Test_Boehm.jl:11
Some type information was truncated. Use `show(err)` to see complete types. |
|
I just tried to test this with the latest versions of all packages on Julia v1.10, but it failed to load because ModelingToolkit failed to precompile. Do you happen to have a |
|
Sorry for late reply (missed this in my notifications). Hopefully this attached manifest file makes it run If this does not work I can try to create a smaller MVE (it is hard though to have a smaller MVE as this error mainly seem to occur with a hard to evaluate likelihood). Alternatively, I can try via StableRNG hacking set up a MVE where failure is faster (once I updated PEtab to the new ModelingToolkit version which should resolve the MTK problem) |
Thanks for this package, great that Julia has a pathfinder implementation!
Regarding the issue, when running
multipathfinderfor a hard to integrate ODE model, for one of the initial points aPosDefExceptionis thrown which causesmultipathfinderto fail.I think having an entire potentially long run of
multipathfinderfail due to a single bad startguess is not ideal. So maybe there should be some exception handling inmultipathfinderallowing a bad initial point to be discarded? This exception handling should cover both Hessian inverse computation, as well the optimization run (I have seen that sometimes Optim.LBFGS can also throw an error). To alert the user a warning can be thrown when this happens.MVE
MVE (can take a bit of time to run).
With output;
I ran everything using Julia 1.10 on Ubuntu with the latest possible versions of each package.
The text was updated successfully, but these errors were encountered: