Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getindex frule behavior #790

Closed
samanklesaria opened this issue Apr 15, 2024 · 1 comment
Closed

getindex frule behavior #790

samanklesaria opened this issue Apr 15, 2024 · 1 comment

Comments

@samanklesaria
Copy link

samanklesaria commented Apr 15, 2024

I was hoping someone might be able to explain the current frule for getindex. Say I want to compute $\nabla_a \text{getindex}(a, i)$ with forward mode AD where $a$ is a 5-element vector. Intuitively, I would expect to get back a five element vector, where the only nonzero element is at index $i$. But when I try frule((NoTangent(), Ones(5), NoTangent()), getindex, ones(5), 1), I get a tangent of 1, not [1,0,0,0,0]. Why?

Specifically, I would have thought that the proper frule would be something like this:

function ChainRulesCore.frule((_, ẋ), ::typeof(getindex), x::AbstractArray, ix::Integer)
	return x[ix], sparsevec([ix], ẋ[ix], length(x))
end
@oxinabox
Copy link
Member

the frule is returning the pushforward result, not the gradient (nor the pullback result).
It is the result of pertubing the input with by a small amount and measuring how much the output changes.
It is thus expected that it is the same "type" as the output, which in this case is a scalar.
(this is incontrast to rrules's pullback which is same "type" as the input)
More details can be found in the docs https://juliadiff.org/ChainRulesCore.jl/dev/maths/propagators.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants