CHANGELOG

v0.7.6 (2024-09-23)

Other

🚀 Prepare patch release (838670e)
⚡️ Display banner more quickly and separate commands from main file (9a28915)
Update README.md (185978c)
Update README.md (52b7904)
Update README.md (0f2d4b7)
📝 Explain how to use Syphoon proxies and news commands (6dec24e)
📝 Add new images for readme (e5eac4f)
remove automate patch (ab59abd)

v0.7.5 (2024-09-20)

Other

Update README.md (0ece034)

v0.7.4 (2024-09-19)

Other

⚗️ Avoid creation of useless json file (a3f3bd2)

v0.7.3 (2024-09-19)

Other

🐛 Fix: NoneType has no attributte 'text'. occuring while checking wether supplier is verified or not (87718f9)

v0.7.2 (2024-09-18)

Other

💄 Split CLI Options and Arguments when user asking for --help (5a2ca7f)

v0.7.1 (2024-09-18)

Other

🚀 New release's first patch (6197b0f)
♿️ Allow empty string as api_key value ... a kind of reset (2417622)

v0.7.0 (2024-09-18)

Other

🚀 Prepare minor version (decfd3a)
✨ Auto install playwright (992f6d6)

v0.6.17 (2024-09-18)

Other

Merge pull request #19 from poneoneo/hide-input

Hide input while typing api-key (0da7c0c)

🚀 Prepare to create new patch for the next pr (e189ae2)
✨ Hide api key while typing (970afcd)
📝 Update license footer info (dc9c8f8)
🚀 Block new release and push MIT license file (d20cae3)

v0.6.16 (2024-09-16)

Other

🚀 Trigger patch event (e850c3b)
🔧 Set version source to 'tag' (12b8090)
📄 Remove GNU license (236500c)
📝 Update badges (e0156bf)
🚀 Publish to pypi and upload to gh-release based on release id (24d1f6b)

v0.6.15 (2024-09-15)

Other

🚀 Remobe released check (46cf456)

v0.6.14 (2024-09-15)

Other

🚀 Automate package distributions upload (7bfc5a0)

v0.6.13 (2024-09-15)

Other

🔧 Udate build command (1e84d05)
🔧 Uncomment build_command (3188666)
🚀 Prepare publish step (a1c64e5)

v0.6.12 (2024-09-15)

Other

🥅 Catch no such table product for non initialized sqlite file

\ (5f1341f)

v0.6.11 (2024-09-15)

Other

🚀 Use prebuild semantic-release actions (9f3110b)
🔧 Update project conf file (b111602)

v0.6.9 (2024-09-14)

v0.6.8 (2024-09-15)

v0.6.7 (2024-09-14)

Other

fix: 🔖 Release version v0.6.6 (1fe7278)

v0.6.6 (2024-09-15)

v0.6.5 (2024-09-14)

Other

🚀 Force patch version (29c087e)
🚀 Add build step (b0b8e59)
🚀 Set upload to pypi to true (0e5726c)
➕ Turn back semantic-release to dev-depencies (cbb4df3)
🔧 Change semantic release parameters (506171d)
🚀 Use semantic release only (13c66c6)
🚀 Do not create virtualenv (b4be4fc)
🔧 Update semantic release settings (66aa6e5)
🚀 Run semantic throught poetry (7003b15)
🚀 Add execution permission to activate env (ec45804)
🚀 Provide the wall path to activate virtualenv (2a90e19)
🚀 Activate shell before run publish (8621e64)
🔧 Add python semantic release to project dependencies (3cd5268)
🚀 Create poetry virtualenv (d8a1689)
🔧 Bump new version (5e7577d)
🚀 Update commiter's name and email (d6c4654)
🚀 Update env var name to fit with env in pyproject (a14d3b7)
🚀 Remove githuaction as committer name and committer_email (9d6c377)
🚀 Reset previous deploy command (73a071f)
🚀 Update deploy command (41ec9f9)
🔧 Fix token value is missing (7332f23)
🚀 Fix no such command -D (790d4d8)
🚀 Update release workflow (66cc1fc)
🔧 Prepare release workflow (d20605a)
🔧 Bump new version (901f268)
🔧 Fix tuple_type error (08f23c1)
🔧 Update semanctic release behaviour (07a8ecc)
📝 Add current version number to init file (0424aa7)

v0.6.4 (2024-09-12)

Other

0.6.4

Automatically generated by python-semantic-release (2694c84)

💚 Reset CD workflow (f430bbc)
💚 Update build process order (bb88e44)
💚 Update deploy steps (03f8848)
💚 Remove pipy env (eac3de5)

v0.6.3 (2024-09-11)

Other

🔖 Bump new version (98344dc)
💄 Center banner (cb4fcfd)
💄 Center banner (b6edddc)
🔖 Bump a new version (195a1be)
⚡️ Display banner more quickly

diplay banner only when a specific command is called rather than display it for the whole main.py file (b63d596)

👷 Update deploy process (f4494c7)
fix 💚 Udpate deploy process (c9cbe47)
Update release_to_pypi.yml (7263305)
💚 Update build process (862cdec)
👷 Change publish and build command (e424927)

v0.6.2 (2024-09-11)

Other

0.6.2

Automatically generated by python-semantic-release (1b8fd2a)

💚 Update deployement process (c3d2744)
🚑️ Fix Not such file directory (bd80624)
🔖 Bump new version (ce1185f)
💚 Update actions versions for semantic release (2c970cb)

v0.6.1 (2024-09-10)

Other

🚀 Remove testpypi from worflow (25a8862)

v0.6.0 (2024-09-10)

Other

0.6.0

Automatically generated by python-semantic-release (07d549d)

🔖 Release new version (0c2f6e1)
🔖 Bumpt to new version (dd19ed9)
📝 Add code quality and covearage badges from codacy (129df3d)
✅ Update coverage report file (df4b34a)
💄 Display Banner for all commands (ab0304a)
✅ Only get 3 suppliers instead of all (1356ede)
🥅 Catch 407 error due to http adresse instead of websocket adress from brighdata

inform users they need to use an adress wich starts with wss (b246a53)

🔖 Release a new version (580cdb0)
✨ Add version command (f25e357)
💄 Add author name at the botton of banner (55cdfc2)
✨ Display Banner for syphoon commands (7dc50a4)
✨ Prevent to run scraper without Syphoon or Brighdata api key (3f7f83a)
🐛 Fix playwright has not "__impl" attribute (dc0a07b)
💄 Add a banner for Bright data commands (a674723)
🔖 Release 0.6.0 (0b97870)
🔖 Release 0.6.0 (b887989)
✅ Upgrade code coverage on providers async process (1efe769)
🦺 Change code scraping process for proxies related based on tests (aadfe98)
🔥 Remove tests file cache and faker depency (46f22b7)
🙈 Untrack cache files and secrets (d250ac1)
🙈 Untrack cache files (83fbd1d)
📈 Untrack .env_tests and __pycache file in tests folder (e0ad9d9)
📈 Avoid github to detect html files in tests/ folder as code project (74b8f94)
📝 Add a new markdown file to introduce package on pypi (967bbb2)
🐛 Fix: mport error while import test module (0fde5a5)
🔐 Stop track .env (a05c502)
🔐 Hide secrets and stop to track it (38f0f17)
➕ Add pandas dep and set the project to handle it again (e4b4510)
➖ Chore: Remove Unsed deps (dbd686b)
🔐 Chore: loguru level to production mode (2e392a6)
➖ Docs: remove unsed deps (a6345dc)
Merge pull request #16 from poneoneo/dependabot/pip/cryptography-43.0.1

chore(deps): bump cryptography from 42.0.8 to 43.0.1 (c9b05df)

chore(deps): bump cryptography from 42.0.8 to 43.0.1

Bumps cryptography from 42.0.8 to 43.0.1.

Changelog
Commits

updated-dependencies:

dependency-name: cryptography dependency-type: direct:production ...

Signed-off-by: dependabot[bot] <support@github.com> (ece0e86)

🚀 Chore: add github release to cd pipeline (4c5b697)
📝 Docs: update package description on pypi (315c25a)
chore(*): prepare ci/cd worflow (323e9ec)
docs(*): change help messages for some commands (953a599)
feat(aba-cli-scraper): add syphoon provider support (d815724)
feat(aba-cli-scraper): add command support for syphoon proxy provider (91d8adc)
docs(*): add remove useless link in ko-fi (b48a5c4)
docs(*): add custom link to my ko-fi account (9a0a1e8)
docs(*): add a way to sponsor the project (eefec81)
docs(*): add a way to sponsor the project (64662f1)

v0.5.5 (2024-08-30)

Other

0.5.5

Automatically generated by python-semantic-release (911ff7f)

doc(changelog): remove secret (e0c2799)

v0.5.4 (2024-08-30)

Other

0.5.4

Automatically generated by python-semantic-release (ecf5159)

doc(changelog): remove .md prettier hook (3c782e4)
doc(changelog): format changelog (2e5d555)
test(*): remove all html files (121286f)
test(*): remove 2 unwanted html files from test (eda0ed5)
doc(readme): add coverage bage (9ef72a8)
test(test/*): add more coverage (74c02f5)
test(test/*): add test coverage report (2b73b0c)
test(test/*): add tests for all commands and some subprocess's commands (106aa36)
refactor(*): update readme (dfa6560)
refactor(*): add short descrption msg for each commands. (01c55e5)
refactor(*): add docstring for almost all functions (1c4d5a6)
style(*): setup linting and formating operations for each commit (915eb9d)
style(pyproject.toml): set line-lenth to 100 (b94799d)
style(*): lets try first commit with pre-commit (5f91e97)
style(*): setup linting and formating operations for each commit (09c75cc)
style(pyproject.toml): set indent-style to space (02aa13c)
style(pyproject.toml): set line-lenth to 90 (8fdb590)
style(*): setup linting and formating operations for each commit (2b050f3)
style(*): setup linting and formating operations for each commit (664ef4b)
style(*): setup linting and formating operations for each commit (0899054)
docs(readme and more): setup env to build online docs (fcebe1e)
docs(readme and more): setup env to build online docs (65e139e)
docs(readme and more): setup env to build online docs (aaada9f)
docs(readme and more): setup env to build online docs (aa884b5)
docs(readme and more): setup env to build online docs (57eca86)
docs(readme and more): setup env to build online docs (3570157)
docs(readme and more): setup env to build online docs (e2096e5)
docs(readme and more): setup env to build online docs (69fadcc)
docs(readme and more): setup env to build online docs (54f6c84)
docs(readme and more): setup env to build online docs (9148bb5)
docs(readme and more): setup env to build online docs (7807214)
docs(readme and more): setup env to build online docs (fb1943d)

v0.5.3 (2024-08-25)

Other

0.5.3

Automatically generated by python-semantic-release (cd39be3)

v0.5.2 (2024-08-25)

Other

0.5.2

Automatically generated by python-semantic-release (2b78454)

v0.5.1 (2024-08-25)

Other

0.5.1

Automatically generated by python-semantic-release (9726960)

v0.5.0 (2024-08-25)

Other

0.5.0

Automatically generated by python-semantic-release (510c765)

docs(readme): add informations about how to use ai-agent command (b62b6d0)
docs(readme): add a more deep about text (d34f7bc)
docs(*): add a short description (a180254)
docs(readme): add informations about how to use ai-agent command (ce92625)
docs(readme): add info about how to use ai-agent command (dbe5bc7)
docs(readme): add info above ai-agent video (bb6f205)
docs(readme): add video about how to use ai-gent command (32f64c4)
docs(readme): update tags (393c229)
docs(readme): add tutorial about ai-agent command (ece0214)
fix(*): delete buffer.csv based on sys plateform (79cd27b)
perf(ai-agent): add a better quality to plot generation (7b285d8)
fix(ai-agent command): The file 'dh5.csv' was not found. (c7239cf)
fix(ai-agent command): add a way to select csv file to talk with. (7ee86a8)
Merge pull request #15 from poneoneo/fix-all-issues-

Fix all issues with all changes recommended (90e8c5c)

style(*): apply ruff formating (b6347f1)
style(ruff): apply ruff formating (5a2402d)
refactor(parser): fix refactoring issue #9 (24c778c)
chore(commit): fix commit conflict (b916340)
style(terminal): pretty printing dataframe result after each query to ai-agent (54bcd88)
chore(*): remove unsed code (0c41ac0)
chore(*): add ignore type on chat method (8182fac)
feat(*): add ai-agent to generate plot or made a query based on scraped data saved in csv file (68e2649)

v0.4.4 (2024-08-17)

Other

0.4.4

Automatically generated by python-semantic-release (0836ed5)

v0.4.3 (2024-08-17)

Other

0.4.3

Automatically generated by python-semantic-release (2fe18ec)

v0.4.2 (2024-08-17)

Other

0.4.2

Automatically generated by python-semantic-release (9911ea0)

v0.4.1 (2024-08-17)

Other

0.4.1

Automatically generated by python-semantic-release (4e23b46)

v0.4.0 (2024-08-17)

Other

0.4.0

Automatically generated by python-semantic-release (4ff6bf7)

v0.3.4 (2024-08-17)

Other

0.3.4

Automatically generated by python-semantic-release (5c59430)

chore(pyproject.toml): bump version to 0.4.4 (a13ad61)
docs(readme): add s/o to trogon (4cda9fb)
docs(readme): workflow-demo video + hide my credentials (1d8b2b0)
docs(readme): add informations about new features (afb924c)
docs(readme): text-mode-demo video (bc0b06b)
docs(readme): workflow-demo video (f170d5b)
chore(pyproject.toml): bump version in config file (0896a11)
chore(.env): remove api key (1a23bd1)
fix(*): avoiding empty database when keywords is doesnt match language links (5f0d563)
feat(aba_cli_scrappe, *): add an amazing interactive interface to fill each commands and parameters (93b3dd9)

v0.3.3 (2024-08-14)

Other

0.3.3

Automatically generated by python-semantic-release (2396f27)

Update README.md (bd73f9d)
Update README.md (bf24235)
Update README.md (fee5203)
Update README.md (0e0bcbe)
Update README.md (64d68ad)
docs(README): fix video again (0d1d1ab)
docs(README): fix video in docs (1b7eb51)
docs(readme): add a short video of the whole workflow (8415482)
docs(readme): add a short video of the whole workflow (434c6e8)
0.3.3

Automatically generated by python-semantic-release (dd32c41)

docs(readme): prettify README (a1ea908)
0.3.3

Automatically generated by python-semantic-release (7d39883)

docs(readme): add pretty tags and informations about the project (241bd14)
chore(pyproject): bump version to 0.3.3 (9247d2f)
0.3.3

Automatically generated by python-semantic-release (e4d2081)

build(poetry.lock, pyproject): update compatibility all patch python versions (45f9f7b)
chore(aba_cli_scrapper): force custom utils function for minimum_to_order to return float (b8fda98)
docs(pyproject): bump version to 0.3.2 (640610b)
fix(aba_cli_scrapper): catch FileNotFoundError into pretty panel message while initializing mysql db + change back type of minimum_to_order from int to float (17280a1)
docs(readme): improve readme with new informations about how to use commands and add some images (68bb223)
chore(images): add images for logo and help command (463f575)

v0.3.2 (2024-08-11)

Other

0.3.2

Automatically generated by python-semantic-release (eab979f)

chore(typed_datas): change type of minimum_to_order from int to float (248ca3e)
fix(aba_cli_scrapper): fix ValueError: could not convert string to float: '' while parsing minimum_to_order (c7a9655)

v0.3.1 (2024-08-10)

Other

0.3.1

Automatically generated by python-semantic-release (7b07735)

fix(*): fixing release v0.3.0 with plawright instalation (833bfe3)
fix(poetry.lock): update poetry.lock and pyproject.toml to fix depencies bugs while instalation of playwright (40af7fa)

v0.3.0 (2024-08-09)

Other

0.3.0

Automatically generated by python-semantic-release (e51d796)

chore(*): Merge branch 'release/0.3.0' (854bd82)
0.3.0

Automatically generated by python-semantic-release (4705196)

chore(pyproject): bumping version to 0.3.0 (5fbbb9f)
fix(aba_cli_scrapper_main_): add pymysql as DBAPI while initializing mysql database without auto-fill flag (8ab4437)
chore(package_structure): update requirements for test project (abaa932)
build(aba_cli_scrapper*): change database api engine from myqlclient to pymysql + fixing errors while parsing html process

going through all scripts tags inside div[class='container'] and looking for the right tag switching between to case while parsing suppliers and products

BREAKING CHANGE: database api engine has changed from myqlclient to pymysql (d19ed75)

fix(poetry.lock): update poetry.lock to fix depencies bugs while instalation (442110b)

feat(*): add information for export-as-csv command + add message for async scraping models

  - when export-as-csv is called a verification will be done to check that this operation is performing On
  windowns platform if not an error will be raised with message &#34;this feature is only available on windows platform&#34;

  - when async scraping is performed a verification will be done to check if an api key is set
   if not an error will be raised with message &#34;you need to set your SCRAPING BROWSER API key from BrightData to Enable Async Scraping&#34; ([`f516006`](https://github.com/poneoneo/Alibaba-CLI-Scraper/commit/f51600617527793055fe257f06da033d1847d751))

docs(README): add my bright data afiliate link (9b07711)
refactor(images): add path to export-as-csv.gif into images folder (cf6b58d)
docs(readme): add movie to show how to use export-as-csv command (390386f)
style(main.py): update error message for export-as-csv subcommand (f691533)
docs(readme): add a table of contents to the readme and explain how to set an api key an export data as a csv (5b0a103)
refactor(aba_cli_scrapper): rewrinting Parser methods into a more pythonic way as recommended by some issues (25a457e)
style(scrape_from_disk.py): avoid to filter divs and json dicts twice + add related key words to pyproject.toml (ade96db)
build(pyproject.toml, poetry.lock): update poetry lock file to include modifications due python required versions: ^3.11 (21ea4b4)
test(readme): remove all htmlfiles in tests packages (11b841b)
docs(readme): add images to readme (055fb40)
docs(readme): update readme and remove unwanted htmlfiles... again :) (de93480)
docs(readme): add new help image result (a5b031c)
refactor(package_structure): remove unwanted htmlfiles (dbe2caf)
chore(project_config): bump version to 0.2.0 (75f6c3d)
style(cli_api): print pretty message after setting api key (f31a63a)
chore(project_config): last settings for semantic version (9227b30)
0.2.0

Automatically generated by python-semantic-release (6b2a9b1)

style(commands): add message at the end of export-as-csv command (534073c)
chore(dependencies): add dev as default branch (da585c8)
chore(semantic_config): add dev as default branch for semantic version sys (302672e)
chore(semantic_config): add semantic verions system (3456bc8)
chore(dependencies): remove and reinstall mysqlclient (ca0c5d3)
feat(cli_api): allow to set bright data api key to use async api (286f441)
feat(cli_api): add export from sqlite to csv command + fix base 10 error bug with minimum_to_order (8077cf5)
chore(package_structure): track *.html file (085c95c)
chore(package_structure): add htmlfile that will be used for test (c70899c)
chore(package_structure): prepare html file to be tested (2bb6a33)
chore(package_structure): remove requirements.txt (729e2d0)
docs(readme): add image for readme (30e9d8b)
refactor(package_structure): remove json with countries an related short name (42862fd)
refactor(package_structure): remove .env file (e5f3605)
refactor(package_structure): remove src and txt file (5e32b28)
docs(readme): add video (cb12ed5)
feat(api commands): add scraper command (97ddc71)
style(api commands): update error message (86291d0)
feat(api commands): update message when integrity error occurs (2fe402c)
feat(main.py): exit typer app when mysql upate operation is done (c2146bb)
docs(readme):update readme with new functionality (8cf4d19)
release(v0.1.9): v0.1.9

BREAKING CHANGES: v0.1.9 (985887b)

fix(web_scrapper): unbound local error (6a6dadc)
fix(*): MOdulenot found in lastest realease (2282960)
refactor(project): turn images folder into an external packages (029f57b)
feat(commands api): add short name for parameters of each subcommands (6484ca8)
fix(datatbase): catching errors when datatbase is updated twice + pretty printed message commands ended with sucess (0daf861)
test(tests): switch to test version (c82af59)
feat(main): show success message after db init command (d445671)
fix(*): fixing release v0.1.6 (03ed972)
fix(*): fixing release v0.1.5 (6110de6)
fix(*): fixing release v0.1.4 (c6d7adf)
fix(*): fixing release v0.1.3 (b74d237)
release(v0.1.2): first release aba-cli-scrapper==0.1.2

BREAKING CHANGE: first release aba-cli-scrapper==0.1.2 (8aca981)

style(commands docs): update info after commands with aba-run (ddb5a82)
fix(dependencies security): REDOS vulnerability (e61a26d)
fix(progress bar): progress bar bugs + deactivate logger + tell user when when internet is turned off (8cea768)
fix(secret_file): keyerror with .env file (bbdf600)
docs(readme): how to use sync api (711fc86)
docs(readme): readme correction (6e2b7dd)
docs(readme): add help image to git index (050f959)
fix(depencies security): redos vulnerability (8028dc3)
build(pyproject): update package project file with info about latest testing version (ff2f02f)
style(project licence): add gpl licence on top of main.py (270a4ae)
docs(readme): update readme with info about commands features and limit (d1007ab)
style(type hint): update type hint to enhance lint warinings on typed dict (0baca81)
chrore(api command): change aba to aba-run as a base command to add in environnement variable (642471f)
fix(playwright): Fix all errors when using async playwright + add more details about products and suppliers + add cli commands to env var

-add typed dict for products and suppliers (b3677a9)
refactor(readme): tracking .gitignore file (45bf01f)
fix(setup): conflict depencies with pip + remove history about package on readme file (e2ae78e)
docs(readme): readme bugs (5817adb)
refactor(ali2b-cli-scrapper): remove main from ignore file (60c0c52)
docs(readme): modify instructions to macth package installations and usage (43aedef)
chore(pyproject): add package file config (7c285c9)
chore(*): design whole project as a python package (7e947fb)
docs(README): add images to replace overview image (9768592)
docs(readme): overview image was not appear on github (024ccb4)
docs(README): convert overview image from .webp to .png and replace alt name (aa8f275)
docs(README): convert overview image from .webp to .png (31cf4aa)
docs(README): add image to introduce cli tool and fix conflict (0aac326)
docs(README): add image to introduce cli tool (6e2be37)
refactor(src): linting errors and assign api key value (b91ddd6)
refactor(src): lower country code inside bright data api key (90b360e)
docs(README): add more details about cli commands (5a0f166)
refactor(.gitignore): ignore cache file from src folder (0f98a1a)
feat(src): update dynamicaly database credential without update .json file + dynamicaly update bright api key base user country (e7c2305)
refactor(.gitignore): ignore python cache file (4f99648)
refactor(src): update api key for bright data and info succes message (ed5aa83)
refactor(requirements): update requirements file (c8f3da1)
fix(src): trouble with db-update (218a0e9)
docs(README): false instructions (a6eaecc)
fix(engine_and_database): fix no such table supplier (91c0731)
docs(src): update help message (bf0520e)
style(src): remove unused print statement (4c55f3c)
style(src): udpdate help message at the end of scrapper. (92bad37)
docs(images): add images folder (6710077)
feat(*): shutdown debug logger and add sync-api as a new feature. finally increase async speed (2ce71f9)
docs(readme): add instruction to readme (a4a6ef1)
fix(src/web_scrapper): fix many errors with response body from page and add sync api (cdc97ac)
refactor (ab67368)
refactor(src): update web_scrapper.py (6ae490e)
refactor(src): change the way to import TaskGroup() (4e754ce)
refactor(src):change the way to import TaskGroup() (8f08a37)
docs(README): replace main.py with app.py (63b9da4)
fix(src): UnboundLocalError: local variable 'save_in' referenced before assignment (2795233)
refactor(src/web_scrapper): replace asyncio.TaskGroup with asyncio.gather (fe43886)
fix(src): minimum to order column was return a string instead of an interger (0c369a8)
fix(db-init command): ModuleNotFoundError: No module named 'pymysql' has been fixed

pymysql has been replaced but was used when user initialized his database. (3f1f07e)

docs(readme): update readme (b9f5467)
chore(src): remane main.py to app.py (342dee2)
docs(readme): add an expansive description of how to install the project and use ALIABABA-CLI-SCRAPPER (39789c8)
chore(DBAPIS): remove pymysql as a DBAPIS cause of sql injections vulnearability. replacing with mysqlclient

BREAKING CHANGE: remove pymysql as a DBAPIS cause of sql injections vulnearability. replacing with mysqlclient (810a653)

chore(requirements): update requirements and depedencies (488d0f5)
feat(whole project): cli support with three command allowing to user: scrapping, initialize and update database with new data.

run-scrapper command to scrapping data from Alibaba based on your requirement. ** db-init command to initialize the database support only mysql and sqlite.** ** db-update command to update the database with new data inside your scrapping results folder.** (06c82a8)

refactor(tests): tests become a package (a6e3b3c)
refactor(whole project): src directory become a package (56fa893)
chore(pipfile.lock .gitignore): track pipfile.lock file to ensure crossplateforme running as recommended by Kenneth Reitz (48513cb)
refactor(pays_data): move pays_data.json to src/ (d48a22f)
chore(licence): add GNU General Public License v3.0 (26dacbb)
refactor(project): move models.py to src/ (ceb4e3b)
refactor(project): track project from src/ folder (9c48b46)
refactor(project): remove all modules into src/ directory according to the new structure (d24de4a)
chore(dependencies and packages): update requirements.txt file with new packages and depencies (d2eece8)
refactor(project): Delete Alibaba_Pages_results directory

html pages result should not be tracked (2b25bd9)

refactor(gitignore): untrack scrapping results (db6a500)
refactor(main): gather all functions features in one file

-the purpose of this refactor is to prepare cli features for the project (371973a)

chore(dependencies): replace pony by sqlmodel (7ea52dd)
refactor(html_to_disk): remove useless logger (e14192d)
refactor(db_orm): use sqlmodel instead of pony as orm engine (94c4d3c)
docs(html_parser): add documentation for PageParser class and remove useless loggers (828b6b9)
feat(utils_scrapping): clean products and suppliers attributes before to insert them in dictionnary

-add methode to retrieve products price without their currency -add method to handling potencial errors while we are getting mininal to order number -return royaume uni there not matching with our json file (84ea38a)

feat(web_scrapper): scrapping based on keywords entered by user then save results in directory specified by user (f9797ae)
feat(engine_and_database): create all tables in database and add products and suppliers rows in database finally save all changes (79b0442)
refactor(scrape_from_disk): create a class to handle all the process needed to get suppliers and products (7af6a83)
refactor(pipfile): update pipfile (57e3cb8)
feat(models): create two tables Products and Suppliers and commit any changes to sqlite database (a7f9bbb)
feat(html_to_disk): catch potential errors that could be occured while writing html content to disk (c0d6169)
refactor(pays_data): add informations to convert country abbreviation to all string name (f284165)
refactor(*): create many modules to separate main.py file from scrapping and saving logics (c1b3708)
docs(README): add Title, description and installation flow (81983e0)
refactor(.env): decouple SBR_WS_CDP_LIST variable from web_scrapper.py file (28a6ebe)
refactor(html_to_disk, main): write logs info to stderr instead of logs files (6762a92)
refactor(playwright_links_getter, main): rename playwright_links_getter.py to main.py (71f164d)
refactor(scrapper): delete scrapper.py file (7ff1f4f)
track changes from .env file (3a1f833)
refactor(Alibaba_Pages_results): track changes from folder that contains pages result (e4d2632)
feat(html_to_disk): save pages results into disk (0480185)
refactor(playwright_links_getter):rename file main to Playwright_links_getter (d998ca3)
refactor(gitignore): untrack Alibaba_Scrapper_info (f2a5c42)
refactor(.gitignore): untrack changes in .gitignore (c2fd5b0)
refactor(gitignore): add main to .gitignore file (4e4e9ea)
refactor(requirements, Pipfile): add requirements files (9d67f32)
refactor(scrapper): get html content from all pages results(42) matching with keywords:machines agricoles asynchronously (a29dde2)
chore(models.py): map dataclasses to create an Mysql database (ad32cc6)
refactor(models_blue_print.txt): build blueprint for database model part 1 (f1a276f)
refactor(gitignore): add gitignore file (7ec42fd)
docs(readme): add informations about fields that will be retrieved (b97bfe1)
docs(readme): add informations about fields that will be retrieved (57cab65)
docs(readme): add informations about fields that will be retrieved (59c7f4f)
docs(readme): add sample of csv result and add info about python version required (f11ae0c)
docs(readme): add sample of csv result and add info about python version required (3a0a596)
docs(readme): add sample of csv result and add info about python version required (c1a4b76)
docs(readme): add sample of csv result and add info about python version required (b0bc45c)
chore: remove all htmlfiles in tests packages (9c7c45c)
docs:update image in readme (e3db496)
docs: add images to readme (53c7cf3)
0.2.0

Automatically generated by python-semantic-release (5a80bf3)

fix: Merge tag 'V0.2.2' into dev

v0.2.2 (e0c5437)

chhore: remove unwanted htmlfiles

V0.2.1 remove unwanted htmlfiles (be10451)

chore(commits): Merge tag 'v0.2.0' into dev

v0.2.0 (a9834fc)

0.2.0

Automatically generated by python-semantic-release (6f83e4d)

fix: fix release v0.2.2 to remove unwanted htmlfiles and update readme (51b8f41)
0.2.0

Automatically generated by python-semantic-release (4ee95c1)

docs: update readme and remove unwanted htmlfiles... again :) (107a12b)
docs: add new help image result (c36c5e0)
chore: remove unwanted htmlfiles (36e383c)
feat(cli_api): add export-as-csv and set-api-key commands (070425b)
chore(project_config): bump version to 0.2.0 (99d91ba)
style(cli_api): print pretty message after setting api key (c4ffac7)
chore(project_config): last settings for semantic version (7d0539a)
0.2.0

Automatically generated by python-semantic-release (2829328)

v0.2.0-rc.1 (2024-07-27)

Other

0.2.0-rc.1

Automatically generated by python-semantic-release (98afa25)

style(commands): add message at the end of export-as-csv command (b720067)
chore(dependencies): add dev as default branch (2101305)
chore:add dev as default for semantic version sys (e920324)
chore:add semantic verions system (7374420)
refactor: remove and reinstall mysqlclient (68bbfd2)
feature:set bright data api key before to use async api (76613fc)
feature:add export from sqlite to csv command + fix base 10 error bug with minimum_to_order (582aa31)
track *.html file (6dd754b)
add htmlfile that will be used for test (e2350ff)
refactor:prepare html file to be tested (335da7c)
remove requirements file (1aa0354)
docs:add image for readme (ba83e6b)
remove json with countries an related short name (56bfc13)
refactor: remove .env file (0a72726)
refactor: remove src and txt file (3140d87)
Merge branch 'main' into dev (0d91af5)
add video (016211b)
add scraper command (27c7ae5)
Merge tag 'V0.1.11' into dev

first version that will now follows the semantic versioning and git flow cycle developement (bc8e735)

v0.1.11 (2024-07-24)

Other

Merge branch 'release/0.1.11' (37e27de)
style: update error message (0caf61f)
style:update message when integrity error occurs (5ca4ca3)
fix: exit typer app when mysql upate operation is done (61b471b)
docs:update readme with new functionality (ff4803c)
track main (9b5638b)
realease: v0.1.9 (695fd11)
fix: unbound local error (5570c23)
fix:MOdulenot found in lastest realease (cebc0fe)
turn images folder into an external packages (edf6eea)
Merge pull request #1 from poneoneo/dev

sync dev (f1d6cc1)

patch:follow main branch (6aadd83)
feat:add short name for parameters of each subcommands (3d30f89)
patch+style: catching errors when datatbase is updated twice + pretty printed message commands ended with sucess (1b5cf74)
switch to test version (f3f1e07)
show success message after db init command (93852c4)
fixing release v0.1.6 (b35ddee)
fixing release v0.1.5 (572c6aa)
fixing release v0.1.4 (d3740f5)
fixing release v0.1.3 (3b4592d)
first release aba-cli-scrapper==0.1.2 (7a48dba)
update info after commands with aba-run (cf5c796)
fix REDOS vulnerability (dbec139)
fix progress bar bugs + deactivate logger + tell user when when internet is turned off (d1675d5)
fix keyerror with .env file (8f26121)
how to use sync api (6527ae2)
readme correction (8371181)
add help image to git index (321a3fe)
fix redos vulnerability (2b8df5e)
update package project file with info about latest testing version (f305e16)
add gpl licence on top of main.py (61b3989)
update readme with info about commands features and limit (f08a3db)
update type hint to enhance lint warinings on typed dict (1a3198d)
change aba to aba-run as a base command to add in environnement variable (4a05cc7)
Fix all errors when using async playwright + add more details about products and suppliers + add cli commands to env var

-add typed dict for products and suppliers (84f3fb2)
tracking .gitignore file (25d4502)
fix conflict depencies with pip + remove history about package on readme file (4e5a574)
fix readme bugs (b98515b)
remove main from ignore file (f8d2eb9)
modify instructions to macth package installations and usage (bb99cde)
add package file config (939f11c)
design whole project as a python package (5f823ec)
add images to replace overview image (63681c8)
fix: overview image was not appear on github (fb68387)
convert overview image from .webp to .png and replace alt name (7c980dc)
convert overview image from .webp to .png (1f92fb6)
add image to introduce cli tool and fix conflict (f1f84fd)
add image to introduce cli tool (4d6e61a)
fix linting errors and assign api key value (12e407c)
lower country code inside bright data api key (65f56eb)
add more details about cli commands (e8e1cfc)
ignore cache file from src folder (35dd705)
update dynamicaly database credential without update .json file + dynamicaly update bright api key base user country (4f283bf)
ignore python cache file (579dd1e)
update api key for bright data and info succes message (17c6ff4)
update requirements file (57c7532)
fix: trouble with db-update (94725f8)
fix:false instructions (16b8dcd)
fix: no such table: supplier (bafe5d4)
update help message (bc7514f)
remove unused print statement (a11658c)
udpdate help message at the end of scrapper. (e37997b)
add images folder (d20377e)
shutdown debug logger and add sync-api as a new feature. finally increase async speed (1b1657e)
add instruction to readme (786e44b)
fix: fix many errors with response body from page and add sync api (aab032b)
refactor (f56ee90)
refactor (348ac62)
refactor (2143a5b)
refactor:change the way to import TaskGroup() (c9ba843)
update: replace main.py with app.py (fa59a13)
fix:UnboundLocalError: local variable 'save_in' referenced before assignment (b8b58db)
refactor:replace asyncio.TaskGroup with asyncio.gather (f946f41)
fix:minimum to order column was return a string instead of an interger (7fb5252)
fix[db-init]: ModuleNotFoundError: No module named 'pymysql' has been fixed

pymysql has been replaced but was used when user initialized his database. (2d53cab)

doc: update readme (a1fcab6)
doc:update readme (e3bca3e)
chore: try to run app.py whitout writing python interpreter each time (cda5e8b)
chore: remane main.py to app.py (9691a0c)
docs!: add an expansive description of how to install the project and use ALIABABA-CLI-SCRAPPER (64a96fc)
"chore!: remove pymysql as a DBAPIS cause of sql injections vulnearability. replacing with mysqlclient" (a81adb1)
chore:update requirements and depedencies (592b5b5)
feat[whole project]: cli support with three command allowing to user: scrapping, initialize and update database with new data.

run-scrapper command to scrapping data from Alibaba based on your requirement. ** db-init command to initialize the database support only mysql and sqlite.** ** db-update command to update the database with new data inside your scrapping results folder.** (a4518b7)

refactor:tests become a package (738998e)
refactor: src directory become a package (4ab2dd3)
chore(): track pipfile.lock file to ensure crossplateforme running as recommended by Kenneth Reitz (e0ee195)
move pays_data.json to src/ (e558de1)
chore(licence): add GNU General Public License v3.0 (808aaaf)
move models.py to src/ (5c8852e)
refactor(project): track project from src/ folder (64bf861)
refactor(project): remove all modules into src/ directory according to the new structure (7a80ee7)
chore(dependencies and packages): update requirements.txt file with new packages and depencies (3219193)
Delete Alibaba_Pages_results directory

html pages result should not be tracked (29b58f0)

untrack scrapping results (3663a6b)
refactor: gather all functions features in one file

-the purpose of this refactor is to prepare cli features for the project (fff96e0)

chore(dependencies): replace pony by sqlmodel (d15efb8)
refactor: remove useless logger (2bc1df0)
refactor(db_orm): use sqlmodel instead of pony as orm engine (e9cda24)
docs(html_parser): add documentation for PageParser class and remove useless loggers (83293c1)
feat:clean products and suppliers attributes before to insert them in dictionnary

-add methode to retrieve products price without their currency -add method to handling potencial errors while we are getting mininal to order number -return royaume uni there not matching with our json file (bef9b19)

feat:scrapping based on keywords entered by user then save results in directory specified by user (dc1eec4)
create all tables in database and add products and suppliers rows in database finally save all changes (77df801)
create a class to handle all the process needed to get suppliers and products (50c1d75)
update pipfile (084cad8)
create two tables Products and Suppliers and commit any changes to sqlite database (a9b09cb)
catch potential errors that could be occured while writing html content to disk (0b0567d)
add informations to convert country abbreviation to all string name (c2cd6f9)
create many modules to separate main.py file from scrapping and saving logics (0fb6773)
add Title, description and installation flow (742fea3)
decouple SBR_WS_CDP_LIST variable from web_scrapper.py file (25a4b15)
write logs info to stderr instead of logs files (7335248)
rename playwright_links_getter.py to main.py (cbb3a4b)
delete scrapper.py file (e24186b)
track changes from .env file (ffaa947)
track changes from folder that contains pages result (f5af17a)
save pages results into disk (57c7efd)
rename file main to Playwright_links_getter (56ecdf8)
untrack Alibaba_Scrapper_info (4f87768)
untrack changes in .gitignore (ed7dbcc)
add main to .gitignore file (d3c7796)
add requirements files (a261e77)
get html content from all pages results(42) matching with keywords:machines agricoles asynchronously (8c79bab)
map dataclasses to create an Mysql database (426b6d9)
build blueprint for database model part 1 (0ed3e01)
add gitignore file (15cc865)
Initial commit (439e286)

Files

CHANGELOG.md

Latest commit

History

CHANGELOG.md

File metadata and controls

CHANGELOG

v0.7.6 (2024-09-23)

Other

v0.7.5 (2024-09-20)

Other

v0.7.4 (2024-09-19)

Other

v0.7.3 (2024-09-19)

Other

v0.7.2 (2024-09-18)

Other

v0.7.1 (2024-09-18)

Other

v0.7.0 (2024-09-18)

Other

v0.6.17 (2024-09-18)

Other

v0.6.16 (2024-09-16)

Other

v0.6.15 (2024-09-15)

Other

v0.6.14 (2024-09-15)

Other

v0.6.13 (2024-09-15)

Other

v0.6.12 (2024-09-15)

Other

v0.6.11 (2024-09-15)

Other

v0.6.9 (2024-09-14)

v0.6.8 (2024-09-15)

v0.6.7 (2024-09-14)

Other

v0.6.6 (2024-09-15)

v0.6.5 (2024-09-14)

Other

v0.6.4 (2024-09-12)

Other

v0.6.3 (2024-09-11)

Other

v0.6.2 (2024-09-11)

Other

v0.6.1 (2024-09-10)

Other

v0.6.0 (2024-09-10)

Other

v0.5.5 (2024-08-30)

Other

v0.5.4 (2024-08-30)

Other

v0.5.3 (2024-08-25)

Other

v0.5.2 (2024-08-25)

Other

v0.5.1 (2024-08-25)

Other

v0.5.0 (2024-08-25)

Other

v0.4.4 (2024-08-17)

Other

v0.4.3 (2024-08-17)

Other

v0.4.2 (2024-08-17)

Other

v0.4.1 (2024-08-17)

Other

v0.4.0 (2024-08-17)

Other

v0.3.4 (2024-08-17)

Other

v0.3.3 (2024-08-14)

Other

v0.3.2 (2024-08-11)

Other