Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add in place version of findall function #54722

Closed
wants to merge 2 commits into from

Conversation

albertomercurio
Copy link
Contributor

Hello,

I added the in place version of the findall function. This is very useful when it is iteratively called inside a function, maybe with some preallocated cache, avoiding to allocate a lot of memory.

@jakobnissen
Copy link
Contributor

See also #43737 - the lazy version might be able to be copied into a vector.

@albertomercurio
Copy link
Contributor Author

So you mean something like findall!(out, A)?

@jakobnissen
Copy link
Contributor

What I mean is that the lazy version of findall is more universal, in the sense that both eager findall (the one which allocates a vector), and the in-place version can be easily written in terms of the lazy one.
As an example, let's take the general definition of lazy findall:

julia> lazy_findall(p, it) = (k for (k, v) in pairs(it) if p(v));

julia> collect(lazy_findall(isodd, [8,3,6,1,3,7,8,9])) |> print # eager version
[2, 4, 5, 6, 8]

julia> copyto!(zeros(Int, 5), lazy_findall(isodd, [8,3,6,1,3,7,8,9])) |> print # in-place version
[2, 4, 5, 6, 8]

Here, the eager version is simply lazy_findall + collect, and the in-place version is simply lazy_findall + copyto!.

@albertomercurio
Copy link
Contributor Author

albertomercurio commented Jun 7, 2024

Has this implementation the same performances of my case? Because in your example this involves Iterators, right?

Copy link
Contributor

@Seelengrab Seelengrab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure this by itself is a good idea. Overwriting the existing data seems very counterintuitive, especially since the original array is not resized & keeps unrelated data around.

Comment on lines +2735 to +2736
n = count(f, A)
I = @view(A[1:n])
Copy link
Contributor

@Seelengrab Seelengrab Jun 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why count first and then loop over the array again? That's double the work, if you move the @view to after the loop, you can just take the view from begin:(begin+count). In the loop you can still safely write into A, there's no need for I up front.

Comment on lines +2737 to +2741
cnt = 1
@inbounds for (i, a) in enumerate(A)
if f(a)
I[cnt] = i
cnt += 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1-based indexing won't work correctly for e.g. OffsetArrays. Not all AbstractArrays are 1-based.

@StefanKarpinski StefanKarpinski added the triage This should be discussed on a triage call label Jun 7, 2024
@albertomercurio
Copy link
Contributor Author

Ok so I think that this implementation is not an optimal one. Let’s close it, and wait for the Lazy implementation discussed above.

@Keno
Copy link
Member

Keno commented Jun 20, 2024

Triage does not like this for a few reasons:

  1. The array needing to be of a type compatible with the index type of the array.
  2. That could be fixed by having input and output be separate arguments that are allowed to alias, but even then, it's not clear what it should do if output is not large enough.
  3. Overall, this feels wrong. It's API with odd corner cases that exists only for performance optimization. This should be done either through the lazy version mentioned or by improving memory optimizations.

@albertomercurio
Copy link
Contributor Author

Ok let’s close this PR

@LilithHafner LilithHafner removed the triage This should be discussed on a triage call label Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants