Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low recall and precision using v3.6.0 #49

Open
LilyWang07 opened this issue Nov 5, 2024 · 1 comment
Open

Low recall and precision using v3.6.0 #49

LilyWang07 opened this issue Nov 5, 2024 · 1 comment

Comments

@LilyWang07
Copy link

LilyWang07 commented Nov 5, 2024

I ran NanoCaller v3.6.0 on ONT NA24385 data using the default parameters:
NanoCaller --bam ${bam} --ref ${reference} --cpu 15

I also tried specifying the SNP and indel models:
NanoCaller --bam ${bam} --ref ${reference} --seq ont --snp_model ONT-HG002 --indel_model ONT-HG002 --sample HG002

I compared the calling results using hap.py and the hg19 reference. When using both sets of parameters, the evaluated indel recall and precision were ~0.086 and ~.007, respectively. The SNP recall and precision were ~0.44 and ~0.81. Do these low values make sense? If not, please let me know how to fix this. Thanks!

@umahsn
Copy link
Collaborator

umahsn commented Nov 7, 2024

Hi,

These results are very unexpected. Can you please some more information about the sample and the ground truth used? In particular, what is the coverage and average read length of the sample? What is the ground truth for NA24385 that you are using? Are you also using a high confidence interval for your ground truth?

What is the performance if you evaluate using the method shown here [ONT Case Study.md](https://github.com/WGLab/NanoCaller/blob/master/docs/ONT Case Study.md)? You can replace GRCh38 with GRCh37 in the case study since you are using hg19.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants