Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sparse dot product #21

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Sparse dot product #21

wants to merge 3 commits into from

Conversation

epaaso
Copy link

@epaaso epaaso commented Jan 18, 2024

When using big datasets, calculation of the dot product with numpy requires assigning a lot of virtual memory to the dense connectivity matrix. Most computers won't be able to allocate that much RAM.

Luckily, scipy has a sparse matrix dot operation that is both much faster and requires much less RAM.
It is a method the sparse matrix object.

Also, we don't have to check if the matrix is sparse because numpy has an inner method for dense matrices that is called in the same way.

ikarus/classifier.py Outdated Show resolved Hide resolved
@dohmjan
Copy link
Collaborator

dohmjan commented Feb 27, 2024

Hi! Thank you for the hint and your PR. I left a comment regarding the axis in the summation. Apart from that it looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants