Skip to content
This repository has been archived by the owner on Mar 20, 2024. It is now read-only.

Fix spike waveform transpose issue in Python3 #36

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

jonnew
Copy link
Member

@jonnew jonnew commented Oct 26, 2016

  • Implements the same fix given in OpenEphys.py in main directory, which
    I guess is for Python2.7
  • Requesting code review before merge

- Implements the same fix given in OpenEphys.py in main directory, which
  I guess is for Pytho2.7
@jonnew
Copy link
Member Author

jonnew commented Oct 26, 2016

As a side note, why are their different versions of these libraries being maintained for Python 2.7 and 3.5 when there are standard mechanisms in place to detect the interpreter being used?

https://docs.python.org/3/howto/pyporting.html

jonnew added 2 commits October 26, 2016 19:44
- Make use of numpy structured arrays to get rid while loop
- Single call to np.fromfile
- On my machine with a nice SSD, for a 40 MB .spike file, the load time
  went from 15.1 sec to about 7 milliseconds.
- This could probably be applied to other data types (continuous and
  events) but I'm not interested in doing it right now.
- I'm only editing one version of the OpenEphys.py file...
- Make use of numpy structured arrays to get rid while loop
- Single call to np.fromfile
- On my machine with a nice SSD, for a 40 MB .spike file, the load time
  went from 15.1 sec to about 7 milliseconds.
- This could probably be applied to other data types (continuous and
  events) but I'm not interested in doing it right now.
- I'm only editing one version of the OpenEphys.py file...
- In keeping with just adding new functions to preserve backwards
  compatibility (instead of versioning...) I named the routine
  loadSpikeFast
@jonnew
Copy link
Member Author

jonnew commented Oct 26, 2016

I created a loadSpikesFast function that results in many orders of magnitude speed up in load times for spikes using numpy's structured arrays. Requesting test and code review before merge. Also, I stopped editing the OpenEphys.py file in the main directory because it seems pretty stupid to maintain two files.

@jvoigts
Copy link
Member

jvoigts commented Oct 28, 2016

This looks very clean and fairly bug proof, unless someone else would like to test this on some other example files i'd say we can just merge this?
Oh, and of course spikes have a color.

@jonnew
Copy link
Member Author

jonnew commented Oct 29, 2016

OK, cool. One thing to note is that the return type is different than loadSpikes and I have not found a clean way around this. Basically, the actually data fields are nested ones step lower down in the dictionary than for the loadSpikes data:

data = loadSpikesFaster(filename)
print(data['data']['waveform'])

vs.

data = loadSpikes(filename)
print(data['spikes'])

to get equivalent data access. This is why I created the new function instead of just modifying existing. Ways to do this without breaking code would be best, but ideas I've had are not elegant. Of course this is also a result of returning the header with the data. Perhaps the best way around this would be to have two functions, one that returns the header only, for generic file types, and one that returns only the data. Most generic would be to have two functions getData and getHeader. getData should be able to deduce return type based upon header info and delegate to 'private' functions.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants