Limitations of FiniteDifferences.to_vec for Tangent()

[theogf]

to_vec is great for working on complex structures but it has the limit of only reconstructing the primal representation.

For example, related to #270 , if I had a to_vec(x::AbstractVector{T,Missing}) where {T} definition such as:

function to_vec(x::AbstractVector{T,Missing}) where {T}
   is = findall(ismissing, x)
   not_is = findall(!ismissing, x)
   xval, from_xval = to_vec(x[is])
   function from_vec(x)
      y = similar(x)
      y[is] = from_xval(x)
      y[not_is] .= missing
      y
   end
   xval, from_vec
end

The primal reconstruction would work fine, but instead of returning some NoTangent for the missing values, it would just return missing and mess up the rest.

One solution would be to specify a dual version which would put the right Tangent types where needed.

[oxinabox]

In the long term we want to move away from to_vec entirely and just have FiniteDifferences work on Tangent objects etc directly.