Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sniff: refactor to just use the csv crate and drop the qsv-sniffer crate #1976

Open
jqnatividad opened this issue Jul 17, 2024 · 1 comment
Labels
dependencies Pull requests that update a dependency file enhancement New feature or request. Once marked with this label, its in the backlog.

Comments

@jqnatividad
Copy link
Owner

Currently, sniff fails to detect valid CSV files that other qsv commands can successfully open.
This is because, sniff is powered by the qsv-sniffer crate, which is a qsv-optimized fork of the unmaintained csv-sniffer crate.
The csv-sniffer crate uses the Viterbi algorithm to sniff and infer CSV metadata and there are fairly common CSV configurations where it fails.

Describe the solution you'd like
Remove the qsv-sniffer crate altogether and just use the csv crate to get CSV metadata.

Describe alternatives you've considered
There was a discussion to create a clone of python's csv-sniffer in #1719 , but that's a non-trivial project by itself

@jqnatividad
Copy link
Owner Author

See #1977

@jqnatividad jqnatividad added enhancement New feature or request. Once marked with this label, its in the backlog. dependencies Pull requests that update a dependency file labels Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file enhancement New feature or request. Once marked with this label, its in the backlog.
Projects
None yet
Development

No branches or pull requests

1 participant