Skip to content

Commit

Permalink
refactor: add timeout for adaptive crawler's enqueueLinks waitFor call
Browse files Browse the repository at this point in the history
  • Loading branch information
B4nan committed Jun 6, 2024
1 parent 533bd3f commit 3780547
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -404,10 +404,10 @@ export class AdaptivePlaywrightCrawler extends PlaywrightCrawler {

return playwrightContext.parseWithCheerio();
},
async enqueueLinks(options = {}) {
async enqueueLinks(options = {}, timeoutMs = 5_000) {
const selector = options.selector ?? 'a';
const locator = playwrightContext.page.locator(selector).first();
await locator.waitFor();
await locator.waitFor({ timeout: timeoutMs, state: 'attached' });

const urls = await extractUrlsFromPage(
playwrightContext.page,
Expand Down

0 comments on commit 3780547

Please sign in to comment.