Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rewrite to typescript with puppeteer-cluster + additional options and hooks. #554

Open
wants to merge 55 commits into
base: master
Choose a base branch
from
Open

Rewrite to typescript with puppeteer-cluster + additional options and hooks. #554

wants to merge 55 commits into from

Commits on Oct 25, 2021

  1. networkidle0 -> networkidle2

    @jan-rycko
    jan-rycko committed Oct 25, 2021
    Configuration menu
    Copy the full SHA
    933c8d9 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2021

  1. Added exclude option.

    @jan-rycko
    jan-rycko committed Oct 26, 2021
    Configuration menu
    Copy the full SHA
    bc56629 View commit details
    Browse the repository at this point in the history

  2. Unnecessary glob-to-regexp.

    @jan-rycko
    jan-rycko committed Oct 26, 2021
    Configuration menu
    Copy the full SHA
    10b8b80 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2021

  1. Base path in options.

    @jan-rycko
    jan-rycko committed Oct 27, 2021
    Configuration menu
    Copy the full SHA
    d843ff7 View commit details
    Browse the repository at this point in the history

  2. console.log

    @jan-rycko
    jan-rycko committed Oct 27, 2021
    Configuration menu
    Copy the full SHA
    6aa4f68 View commit details
    Browse the repository at this point in the history

  3. Using calculated basePath, not only on localhost

    @jan-rycko
    jan-rycko committed Oct 27, 2021
    Configuration menu
    Copy the full SHA
    558504e View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2022

  1. Fix for null base path.

    @jan-rycko
    jan-rycko committed Mar 7, 2022
    Configuration menu
    Copy the full SHA
    ca63d63 View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2022

  1. Default base path and wait for response option.

    @jan-rycko
    jan-rycko committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    098f362 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2022

  1. Customizable logging with handling in afterFetch. 

    Run snapshots for html/png/jpg at once.
Added custom processHtml/Css/Page functions.
Custom filename for snapshots.
    @jan-rycko
    jan-rycko committed Mar 16, 2022
    Configuration menu
    Copy the full SHA
    51257ba View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2022

  1. More optimizations.

    @jan-rycko
    jan-rycko committed Mar 17, 2022
    Configuration menu
    Copy the full SHA
    ca8d3b3 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2022

  1. Minor style processing improvements.

    @jan-rycko
    jan-rycko committed Mar 18, 2022
    Configuration menu
    Copy the full SHA
    1be88eb View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2022

  1. No-port mode. 

    Passing options to external processors.
    @jan-rycko
    jan-rycko committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    109e5f3 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2022

  1. Fixed tests

    @jan-rycko
    jan-rycko committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    eabd0c8 View commit details
    Browse the repository at this point in the history

  2. Rewrite to typescript, puppeteer upgrade, with ts-jest. 

    Tests are passing.
    @jan-rycko
    jan-rycko committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    38ce0b3 View commit details
    Browse the repository at this point in the history

  3. Rewrite to puppeteer-cluster.

    @jan-rycko
    jan-rycko committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    756a6ce View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2022

  1. Finished rewrite to puppeteer cluster. 

    Fixed tests.
Cluster has 5s idle time on end to ensure closed crawling, timeout needed to be prolonged in tests to handle this.
    @jan-rycko
    jan-rycko committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    d548690 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2022

  1. dist in files

    @jan-rycko
    jan-rycko committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    b60ad49 View commit details
    Browse the repository at this point in the history

  2. More logs for pageerror

    @jan-rycko
    jan-rycko committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    45c9672 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2022

  1. concurrencyType from puppeteer-cluster added to options.

    @jan-rycko
    jan-rycko committed Apr 4, 2022
    Configuration menu
    Copy the full SHA
    771895f View commit details
    Browse the repository at this point in the history

  2. Build

    @jan-rycko
    jan-rycko committed Apr 4, 2022
    Configuration menu
    Copy the full SHA
    60692e5 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2022

  1. Package json for react-snap-typed

    @jan-rycko
    jan-rycko committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    4f641b2 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2022

  1. pageerror message and cancelable wait for idle.

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    fd3b6bd View commit details
    Browse the repository at this point in the history

  2. Build 1.26.0

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    474733b View commit details
    Browse the repository at this point in the history

  3. Fix.

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    e57b353 View commit details
    Browse the repository at this point in the history

  4. 1.26.2

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    1de2258 View commit details
    Browse the repository at this point in the history

  5. 1.26.3

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    a01276b View commit details
    Browse the repository at this point in the history

  6. Rebuild 1.26.4

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    2c9c303 View commit details
    Browse the repository at this point in the history

  7. Ignore canceled promise error.

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    6ae68c3 View commit details
    Browse the repository at this point in the history

  8. Typo 1.26.6

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    01e4753 View commit details
    Browse the repository at this point in the history

  9. 1.26.7

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    ed63af9 View commit details
    Browse the repository at this point in the history

  10. 1.26.8

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    de87ffe View commit details
    Browse the repository at this point in the history

  11. 1.26.9

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    b517b39 View commit details
    Browse the repository at this point in the history

  12. 1.26.10

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    7bd2c84 View commit details
    Browse the repository at this point in the history

  13. page/browser close strategy

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    d402690 View commit details
    Browse the repository at this point in the history

  14. Logs 1.26.12

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    c1d04fc View commit details
    Browse the repository at this point in the history

  15. Logs 1.26.13

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    6542cb2 View commit details
    Browse the repository at this point in the history

  16. Version

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    7d44b74 View commit details
    Browse the repository at this point in the history

  17. Logs 1.26.15

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    e49883e View commit details
    Browse the repository at this point in the history

  18. Logs 1.26.16

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    ea3aa21 View commit details
    Browse the repository at this point in the history

  19. cleanup hook 1.26.17

    @jan-rycko
    jan-rycko committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    ce40e9b View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2022

  1. cleanupBrowser

    @jan-rycko
    jan-rycko committed Jun 6, 2022
    Configuration menu
    Copy the full SHA
    0e6cefd View commit details
    Browse the repository at this point in the history

  2. 1.26.18

    @jan-rycko
    jan-rycko committed Jun 6, 2022
    Configuration menu
    Copy the full SHA
    a7400db View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2022

  1. Disabled websocket connections in puppeteer_utils.

    @jan-rycko
    jan-rycko committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    dfa8e13 View commit details
    Browse the repository at this point in the history

  2. Rollback for now.

    @jan-rycko
    jan-rycko committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    77744bd View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2022

  1. Basic auth.

    @jan-rycko
    jan-rycko committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    3f7401c View commit details
    Browse the repository at this point in the history

  2. Use timeout option.

    @jan-rycko
    jan-rycko committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    3f8e922 View commit details
    Browse the repository at this point in the history

  3. Version

    @jan-rycko
    jan-rycko committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    3cd2709 View commit details
    Browse the repository at this point in the history

  4. Version.

    @jan-rycko
    jan-rycko committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    736c4d8 View commit details
    Browse the repository at this point in the history

  5. Version.

    @jan-rycko
    jan-rycko committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    7bd7be2 View commit details
    Browse the repository at this point in the history

  6. Retries on errors.

    @jan-rycko
    jan-rycko committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    e053573 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2022

  1. Version.

    @jan-rycko
    jan-rycko committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    d7c8b54 View commit details
    Browse the repository at this point in the history

  2. Disable check for unique url when requesting for second snapshot.

    @jan-rycko
    jan-rycko committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    17f8746 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2022

  1. Disabled browser.close()

    @jan-rycko
    jan-rycko committed Nov 9, 2022
    Configuration menu
    Copy the full SHA
    866b132 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2023

  1. 1.26.30: Audit fixes.

    @jan-rycko
    jan-rycko committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    22e32bb View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2023

  1. preinstall -> force-resolutions

    @jan-rycko
    jan-rycko committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    54e82f0 View commit details
    Browse the repository at this point in the history