Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pose reader classes supporting ingestion of SLEAP tracking data #206

Merged
merged 12 commits into from
Aug 30, 2023

Conversation

jkbhagatio
Copy link
Member

aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
@ttngu207
Copy link
Contributor

One potential change to the Sleaper reader class is to remove the requirement of model_path as an argument to instantiate the class.
Instead, this model_path can be specified upload loading the data.

I will review and potentially propose a modification to the current solution.

@ttngu207
Copy link
Contributor

ttngu207 commented Jul 12, 2023

We discussed the possibility of the reader not needing the model path. The model path can be extracted from some meta information somewhere.
This model path information is stored anyway, perhaps better yet that we store a timestamp and the model path as event, for each time the model is changed.

@glopesdev
Copy link
Contributor

glopesdev commented Aug 10, 2023

@ttngu207 @jkbhagatio @lochhh today we were discussing file name conventions for binary SLEAP acquisition files, and we are currently leaning towards simply using the relative model path as an identifier suffix: <hpc_node_used_for_training>_<job_id>_<training_completed_time>_<model_type>

An example file name would be: CameraTop_SLEAP_ENC1_9834659_20230307_TailTattoo_2023-08-10T14-08-00.bin

Even though it feels long, we were comparing it with other alternatives such as assigning a GUID to each trained model and it wouldn't be much different: CameraTop_SLEAP_C9A139B9-E3DE-4FAC-BFFA-2421B2E5F621.bin

The former is arguably much more readable and ammenable to wildcard glob search, e.g. searching for all tracking data of <model_type> and hpc_node would be CameraTop_SLEAP_ENC1_*_TailTattoo_*

@jkbhagatio
Copy link
Member Author

jkbhagatio commented Aug 10, 2023

Given the above, we assume that model metadata config file lives next to the model?

@glopesdev also if we go with the dataframe columns being:

class | class_conf | part | part_conf | x | y

will need to slightly change this reader's read() method to output in this format

@jkbhagatio
Copy link
Member Author

Hi all, there should be a working version of this now, via the Pose reader.

It should work with the example files in '/ceph/aeon/aeon/data/processed/test-enc1...'

We should add a proper test for this (and other readers), and decide where it will finally live (see #226), but this provides a working solution for now.

@jkbhagatio jkbhagatio requested review from glopesdev, lochhh, ttngu207 and JaerongA and removed request for bruno-f-cruz August 11, 2023 18:09
Copy link
Contributor

@glopesdev glopesdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, just have a few comments to look into before merging. I think most of them can be left for when we decide to move this reader into the core io module, but some could be nice to do right now.

aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
Copy link
Contributor

@glopesdev glopesdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more optimization and formatting questions, but otherwise looks good!

aeon/util.py Outdated Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
aeon/schema/social.py Outdated Show resolved Hide resolved
Copy link
Contributor

@glopesdev glopesdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just left some final comments for consideration and I think the refactoring of tests should be left to another PR, since they seem to be unrelated to testing this reader and possibly more related to the #228 PR.

aeon/schema/social.py Outdated Show resolved Hide resolved
aeon/schema/social.py Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this refactoring of the tests outside of this PR? I am happy discussing and refactoring the standard imports as we discussed on #238 but would make it easier to keep track of changes if that was done in a separate PR.

@glopesdev glopesdev changed the title Sleap reader Add pose reader classes supporting ingestion of SLEAP tracking data Aug 30, 2023
@jkbhagatio
Copy link
Member Author

Looks good, just left some final comments for consideration and I think the refactoring of tests should be left to another PR, since they seem to be unrelated to testing this reader and possibly more related to the #228 PR.

@glopesdev Sounds good, read all of your most recent replies/suggestions here and agree with them all - will do them all before merging, and move the tests to #228

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants