A whole bunch of stuff. #4
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Figure out why Qt craps out after first row of input.
Need to add handling for commas in card names.
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match (does not handle multiple matches). Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). New: Added usage. Todo: Add search by specific edition.
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match (does not handle multiple matches). Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). New: Added usage. Todo: Add search by specific edition.
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match. Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). Depends: PySide New: Added usage, proper file handling. Todo: Add search by specific edition.
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match. Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). Depends: PySide New: Added usage, proper file handling. Todo: Add search by specific edition.
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match. Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). Depends: docopt, PySide New: Added usage, proper file handling. Todo: Add search by specific edition. Add formatting for deckstats.net input and output.
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match. Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). Depends: docopt, PySide New: Added usage, proper file handling. Todo: Add search by specific edition. Add formatting for deckstats.net input and output. Clean up documentation (Python style guide??).
Reads CSV with name and quantity of card. Scrapes for prices based on exact name match. Only returns first result / latest printing of card (see http://magiccards.info/search.html and http://magiccards.info/syntax.html). Depends: docopt, PySide New: Added usage, proper file handling. Todo: Add search by specific edition. Add handling for multiple instances of same card (duplicates, different printing, foil). Improve formatting for deckstats.net input and output. Clean up documentation (Python style guide??).
Minor improvements to results accuracy. Seeing lots of misses due to multiple results. Can't use both exact name "!" and edition "e:" in same query. At ~97% accuracy with personal collection. Known causes of misses: Non-alphanumeric characters (e.g. Æther Burst, R&D's Secret Lair, Legions of Lim-Dûl). Some of these work when the correct character is used; others do not (sometimes depends on whether input CSV has correct character). Multiple results (e.g. Forest 10e also returns Karplusan Forest 10e). Query does not work with both full name and edition? No price returned at all (e.g. Purphoros's Emissary). Possible solutions: Re-try with exact name only on initial miss (still questionable). Retrieve additional info from mtgjson via DeckBrew's API (i.e. number within printing), then try exact URL of card (e.g. http://magiccards.info/cstd/en/29.html).
This reverts commit dff8b3d.
Moved bulk of price fetching functionality into new class GetPrices (mtgs_getprices.py). Moved webpage rendering to new class WebRenderer (mtgs_webrenderer.py). New error classes (mtgs_error.py).
Converted to Python 3. Added MTG JSON. No longer uses numpy. More comprehensive set_defs.csv Much to do.
Found that card entries from MTG JSON are wildly inconsistent in availability of "numbers" and "mciNumbers" fields. Made MTGCard a little easier to work with from the outside. Improved scraping to use exact URL whenever possible (hampered by inconsistent data). Updated write and summary methods. Better sample files.
Quick fix for output path.
Debug output goes to "./debug.log" instead of terminal. Cleaned up debugging. Cleaned up some documentation.
Added sample files SOI.csv and SOI_out.csv and corresponding debug.log. Note that the 7 misses were all due to no price data on the correctly-scraped pages.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Basically an entirely rewrite of getmagiccardprices.py
It now has some configurable elements, can take a collection of cards exported from deckstats.net, and handles prints of the same card from different editions.