Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgumentError: Union{} does not have elements when trying to run a state space model #2151

Closed
sjwild opened this issue Jan 5, 2024 · 5 comments

Comments

@sjwild
Copy link

sjwild commented Jan 5, 2024

I am trying to run a state space model, and I keep encountering the error in the subject line when using ReverseDiff. I found a previous issue raising it, but it was with Julia 1.4. (EDIT: link)

Here is a minimum reproducible example with Julia 1.10.0 and Turing 0.30.0

#Julia 1.10

using Plots, StatsPlots
using Turing, ReverseDiff, Memoization 
using LinearAlgebra


# build data
t = [1:.05:8;]
σ = .3
y = @. rand(sin(t) + Normal(0, σ))

scatter(t, y)

@model function state_space(y, 
    TT,
    ::Type{T}=Float64) where {T}

    # Priors 
    α ~ Normal(y[1], 0.001)
    τ ~ Exponential(1)
    η ~ filldist(Normal(0, 1), TT-1)
    σ ~ Exponential(1)

    # create latent variable
    x = Vector{T}(undef, TT)

    x[1] = α

    for t in 2:TT
        x[t] = x[t-1] + η[t-1] * τ
    end

    # measurement model
    y ~ MvNormal(x, σ * I)

    return x

end

n_iter = 1500
model = state_space(y, length(t))
chns = sample(model, NUTS(n_iter * 2, 0.8; adtype = AutoReverseDiff(true)), n_iter);

And here is the stacktrace

ERROR: ArgumentError: Union{} does not have elements
Stacktrace:
  [1] eltype(::Type{Union{}})
    @ Base ./abstractarray.jl:237
  [2] eltype(vi::DynamicPPL.ThreadSafeVarInfo{DynamicPPL.TypedVarInfo{…}, Vector{…}}, spl::DynamicPPL.Sampler{NUTS{…}})
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/abstract_varinfo.jl:366
  [3] get_matching_type(spl::DynamicPPL.Sampler{…}, vi::DynamicPPL.ThreadSafeVarInfo{…}, ::Type{…})
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/compiler.jl:722
  [4] matchingvalue(sampler::DynamicPPL.Sampler{…}, vi::DynamicPPL.ThreadSafeVarInfo{…}, value::Type{…})
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/compiler.jl:693
  [5] matchingvalue(context::DynamicPPL.SamplingContext{…}, vi::DynamicPPL.ThreadSafeVarInfo{…}, value::Type)
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/compiler.jl:706
  [6] macro expansion
    @ ~/.julia/packages/DynamicPPL/YWihz/src/model.jl:993 [inlined]
  [7] make_evaluate_args_and_kwargs
    @ ~/.julia/packages/DynamicPPL/YWihz/src/model.jl:971 [inlined]
  [8] _evaluate!!(model::DynamicPPL.Model{…}, varinfo::DynamicPPL.ThreadSafeVarInfo{…}, context::DynamicPPL.SamplingContext{…})
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/model.jl:962
  [9] evaluate_threadsafe!!
    @ ~/.julia/packages/DynamicPPL/YWihz/src/model.jl:952 [inlined]
 [10] evaluate!!
    @ ~/.julia/packages/DynamicPPL/YWihz/src/model.jl:887 [inlined]
 [11] logdensity(f::LogDensityFunction{…}, θ::ReverseDiff.TrackedArray{…})
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/logdensityfunction.jl:94
 [12] Fix1
    @ Base ./operators.jl:1118 [inlined]
 [13] ReverseDiff.GradientTape(f::Base.Fix1{…}, input::Vector{…}, cfg::ReverseDiff.GradientConfig{…})
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/api/tape.jl:199
 [14] ReverseDiff.GradientTape(f::Function, input::Vector{Float64})
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/api/tape.jl:198
 [15] _compiledtape
    @ LogDensityProblemsADReverseDiffExt ~/.julia/packages/LogDensityProblemsAD/OQ0BL/ext/LogDensityProblemsADReverseDiffExt.jl:56 [inlined]
 [16] #ADgradient#1
    @ LogDensityProblemsADReverseDiffExt ~/.julia/packages/LogDensityProblemsAD/OQ0BL/ext/LogDensityProblemsADReverseDiffExt.jl:50 [inlined]
 [17] ADgradient(ad::AutoReverseDiff, ℓ::LogDensityFunction{…})
    @ Turing.Essential ~/.julia/packages/Turing/tdsVz/src/essential/ad.jl:44
 [18] ADgradient(ad::AutoReverseDiff, ℓ::LogDensityFunction)
    @ Turing.Essential ~/.julia/packages/Turing/tdsVz/src/essential/ad.jl:20 [inlined]
 [19] initialstep(rng::Random.TaskLocalRNG, model::DynamicPPL.Model{…}, spl::DynamicPPL.Sampler{…}, vi::DynamicPPL.TypedVarInfo{…}; init_params::Nothing, nadapts::Int64, kwargs::@Kwargs{…})
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/hmc.jl:148
 [20] step(rng::Random.TaskLocalRNG, model::DynamicPPL.Model{…}, spl::DynamicPPL.Sampler{…}; initial_params::Nothing, kwargs::@Kwargs{…})
    @ DynamicPPL ~/.julia/packages/DynamicPPL/YWihz/src/sampler.jl:116
 [21] step
    @ ~/.julia/packages/DynamicPPL/YWihz/src/sampler.jl:99 [inlined]
 [22] macro expansion
    @ ~/.julia/packages/AbstractMCMC/Es490/src/sample.jl:127 [inlined]
 [23] macro expansion
    @ ~/.julia/packages/ProgressLogging/6KXlp/src/ProgressLogging.jl:328 [inlined]
 [24] macro expansion
    @ ~/.julia/packages/AbstractMCMC/Es490/src/logging.jl:9 [inlined]
 [25] mcmcsample(rng::Random.TaskLocalRNG, model::DynamicPPL.Model{…}, sampler::DynamicPPL.Sampler{…}, N::Int64; progress::Bool, progressname::String, callback::Nothing, discard_initial::Int64, thinning::Int64, chain_type::Type, initial_state::Nothing, kwargs::@Kwargs{…})
    @ AbstractMCMC ~/.julia/packages/AbstractMCMC/Es490/src/sample.jl:117
 [26] sample(rng::Random.TaskLocalRNG, model::DynamicPPL.Model{…}, sampler::DynamicPPL.Sampler{…}, N::Int64; chain_type::Type, resume_from::Nothing, initial_state::Nothing, progress::Bool, nadapts::Int64, discard_adapt::Bool, discard_initial::Int64, kwargs::@Kwargs{})
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/hmc.jl:117
 [27] sample
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/hmc.jl:86 [inlined]
 [28] #sample#2
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:190 [inlined]
 [29] sample
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:183 [inlined]
 [30] #sample#1
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:180 [inlined]
 [31] sample(model::DynamicPPL.Model{…}, alg::NUTS{…}, N::Int64)
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:174
 [32] top-level scope
    @ REPL[11]:1
Some type information was truncated. Use `show(err)` to see complete types.
@devmotion
Copy link
Member

As the stacktrace states, the problem is that in Julia 1.10 the call to return_type in https://github.com/TuringLang/DynamicPPL.jl/blob/11a75fffe3737a5ce06407444a81d588c3b4e188/src/abstract_varinfo.jl#L366 returns a Union{} but eltype(Union{}) throws an error.

A few comments:

Regarding the second point, adding print statements such as @show vi[spl] right before the failing code reveals the underlying problem:

ERROR: MethodError: vcat(::ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}, ::ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}, ::ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}, ::ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}) is ambiguous.

Candidates:
  vcat(var"376"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, var"377"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(X1::Union{Number, AbstractVecOrMat{<:Number}}, X::Union{Number, AbstractVecOrMat{<:Number}}...)
    @ SparseArrays ~/.asdf/installs/julia/1.10.0/share/julia/stdlib/v1.10/SparseArrays/src/sparsevector.jl:1235
  vcat(var"376"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, var"377"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"376"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, var"377"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"339"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"348"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, var"349"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"339"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"339"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"348"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, var"349"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"348"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, var"349"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"372"::AbstractVector, var"373"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"372"::AbstractVector, var"373"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"372"::AbstractVector, var"373"::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"337"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"344"::AbstractVector, var"345"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVector...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"337"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"337"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"344"::AbstractVector, var"345"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::AbstractVecOrMat...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52
  vcat(var"344"::AbstractVector, var"345"::AbstractVector, x::ReverseDiff.TrackedArray{V, D, 1} where {V, D}, xs::Union{Number, AbstractVector}...)
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/derivatives/arrays.jl:52

Possible fix, define
  vcat(::ReverseDiff.TrackedArray{…} where {}, ::ReverseDiff.TrackedArray{…} where {}, ::ReverseDiff.TrackedArray{…} where {}, ::Vararg{…})

Stacktrace:
  [1] getindex(vi::TypedVarInfo{@NamedTuple{…}, ReverseDiff.TrackedReal{…}}, spl::Sampler{NUTS{…}})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/varinfo.jl:1447
  [2] macro expansion
    @ ./show.jl:1181 [inlined]
  [3] eltype(vi::TypedVarInfo{@NamedTuple{…}, ReverseDiff.TrackedReal{…}}, spl::Sampler{NUTS{…}})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/abstract_varinfo.jl:368
  [4] get_matching_type(spl::Sampler{NUTS{…}}, vi::TypedVarInfo{@NamedTuple{…}, ReverseDiff.TrackedReal{…}}, ::Type{Float64})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/compiler.jl:722
  [5] matchingvalue(sampler::Sampler{NUTS{…}}, vi::TypedVarInfo{@NamedTuple{…}, ReverseDiff.TrackedReal{…}}, value::Type{Float64})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/compiler.jl:693
  [6] matchingvalue(context::SamplingContext{Sampler{…}, DefaultContext, Random.TaskLocalRNG}, vi::TypedVarInfo{@NamedTuple{…}, ReverseDiff.TrackedReal{…}}, value::Type)
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/compiler.jl:706
  [7] macro expansion
    @ ~/.julia/dev/DynamicPPL/src/model.jl:993 [inlined]
  [8] make_evaluate_args_and_kwargs
    @ ~/.julia/dev/DynamicPPL/src/model.jl:971 [inlined]
  [9] _evaluate!!(model::Model{…}, varinfo::TypedVarInfo{…}, context::SamplingContext{…})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/model.jl:962
 [10] evaluate_threadunsafe!!
    @ ~/.julia/dev/DynamicPPL/src/model.jl:936 [inlined]
 [11] evaluate!!
    @ ~/.julia/dev/DynamicPPL/src/model.jl:889 [inlined]
 [12] logdensity(f::LogDensityFunction{TypedVarInfo{…}, Model{…}, SamplingContext{…}}, θ::ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{…}, Vector{…}})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/logdensityfunction.jl:94
 [13] Fix1
    @ Base ./operators.jl:1118 [inlined]
 [14] ReverseDiff.GradientTape(f::Base.Fix1{…}, input::Vector{…}, cfg::ReverseDiff.GradientConfig{…})
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/api/tape.jl:199
 [15] ReverseDiff.GradientTape(f::Function, input::Vector{Float64})
    @ ReverseDiff ~/.julia/packages/ReverseDiff/UJhiD/src/api/tape.jl:198
 [16] _compiledtape
    @ LogDensityProblemsADReverseDiffExt ~/.julia/packages/LogDensityProblemsAD/OQ0BL/ext/LogDensityProblemsADReverseDiffExt.jl:56 [inlined]
 [17] #ADgradient#1
    @ LogDensityProblemsADReverseDiffExt ~/.julia/packages/LogDensityProblemsAD/OQ0BL/ext/LogDensityProblemsADReverseDiffExt.jl:50 [inlined]
 [18] ADgradient(ad::AutoReverseDiff, ℓ::LogDensityFunction{TypedVarInfo{…}, Model{…}, SamplingContext{…}})
    @ Turing.Essential ~/.julia/packages/Turing/tdsVz/src/essential/ad.jl:44
 [19] ADgradient(ad::AutoReverseDiff, ℓ::LogDensityFunction{TypedVarInfo{…}, Model{…}, SamplingContext{…}})
    @ Turing.Essential ~/.julia/packages/Turing/tdsVz/src/essential/ad.jl:20 [inlined]
 [20] initialstep(rng::Random.TaskLocalRNG, model::Model{…}, spl::Sampler{…}, vi::TypedVarInfo{…}; init_params::Nothing, nadapts::Int64, kwargs::@Kwargs{})
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/hmc.jl:148
 [21] step(rng::Random.TaskLocalRNG, model::Model{…}, spl::Sampler{…}; initial_params::Nothing, kwargs::@Kwargs{})
    @ DynamicPPL ~/.julia/dev/DynamicPPL/src/sampler.jl:116
 [22] step
    @ ~/.julia/dev/DynamicPPL/src/sampler.jl:99 [inlined]
 [23] macro expansion
    @ ~/.julia/packages/AbstractMCMC/Es490/src/sample.jl:127 [inlined]
 [24] macro expansion
    @ ~/.julia/packages/ProgressLogging/6KXlp/src/ProgressLogging.jl:328 [inlined]
 [25] macro expansion
    @ ~/.julia/packages/AbstractMCMC/Es490/src/logging.jl:9 [inlined]
 [26] mcmcsample(rng::Random.TaskLocalRNG, model::Model{…}, sampler::Sampler{…}, N::Int64; progress::Bool, progressname::String, callback::Nothing, discard_initial::Int64, thinning::Int64, chain_type::Type, initial_state::Nothing, kwargs::@Kwargs{})
    @ AbstractMCMC ~/.julia/packages/AbstractMCMC/Es490/src/sample.jl:117
 [27] sample(rng::Random.TaskLocalRNG, model::Model{…}, sampler::Sampler{…}, N::Int64; chain_type::Type, resume_from::Nothing, initial_state::Nothing, progress::Bool, nadapts::Int64, discard_adapt::Bool, discard_initial::Int64, kwargs::@Kwargs{})
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/hmc.jl:117
 [28] sample
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/hmc.jl:86 [inlined]
 [29] #sample#2
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:190 [inlined]
 [30] sample
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:183 [inlined]
 [31] #sample#1
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:180 [inlined]
 [32] sample(model::Model{…}, alg::NUTS{…}, N::Int64)
    @ Turing.Inference ~/.julia/packages/Turing/tdsVz/src/mcmc/Inference.jl:174
 [33] top-level scope
    @ REPL[18]:1
Some type information was truncated. Use `show(err)` to see complete types.

@sjwild
Copy link
Author

sjwild commented Jan 6, 2024

Thanks for the quick response! I'll have a look at the resources you posted.

@ElOceanografo
Copy link
Contributor

+1 on this issue, I just ran into the same thing with my own state space model.

@devmotion
Copy link
Member

The problem should be fixed by TuringLang/DynamicPPL.jl#568 (but tests currently fail due to an issue with LinearAlgebra on Julia 1.10 and a seemingly unrelated problem with Turing 0.30.2).

@devmotion
Copy link
Member

Fixed by TuringLang/DynamicPPL.jl#568 which will be available in the next DynamicPPL release: JuliaRegistries/General#99754

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants