Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Puppeteer 等响应、拦请求 #69

Open
felix-cao opened this issue Sep 4, 2018 · 0 comments
Open

Puppeteer 等响应、拦请求 #69

felix-cao opened this issue Sep 4, 2018 · 0 comments

Comments

@felix-cao
Copy link
Owner

felix-cao commented Sep 4, 2018

一、场景

有两个API请求,当拿到API1响应后的数据data,若该数据不是空数组时立刻去请求API2,否则不请求。
假设AP1为:api/tool/${id}, API2为:api/tool/${id}/tab

需要拿到第一次响应后的数据,拦截第二次请求并让第二次请求等待 n 秒后再去触发API请求

二、等待第一次请求响应的数据

const id = 100;
const urlStr = `api/tool/${id}`;
const Response = await newPage.waitForResponse(res => res.url().includes(urlStr));
if(!Response.ok()) {
  throw new Error(JSON.stringify({ErrorAPIURL: Response.url(), status: Response.status()}));
}

const data = JSON.parse(await Response.text());

waitForResponse 文档

三、拦截第二次请求

const id = 100;
const urlStr = `api/tool/${id}/tab`;
await newPage.setRequestInterception(true);
const interceptedRequest = async (request) => {
  if(request.url().includes(`${urlStr}/`)) {
    await newPage.waitForTimeout(500);
    request.continue();
    return;
  }
  request.continue();
}
if(!newPage.listenerCount('request')) {  
  newPage.on('request', interceptedRequest);
}
  • on 方法listenerCount 方法
  • 我的场景是在一个循环中,newPage.listenerCount('request')是为了防止添加过多的page.on('request')监听器

Reference

@felix-cao felix-cao changed the title 蜘蛛爬虫入门之Puppeteer 蜘蛛爬虫入门之 Puppeteer Nov 6, 2018
@felix-cao felix-cao changed the title 蜘蛛爬虫入门之 Puppeteer Puppeteer 等响应、拦请求 Feb 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant