Skip to content

Commit

Permalink
Readme: note about avoid race conditions & memory leaks when closing …
Browse files Browse the repository at this point in the history
…contexts
  • Loading branch information
elacuesta committed Jul 24, 2023
1 parent dac8ee3 commit b25c64c
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 2 deletions.
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ persistent. See also [`BrowserType.launch_persistent_context`](https://playwrigh
Note that persistent contexts are launched independently from the main browser
instance, hence keyword arguments passed in the
[`PLAYWRIGHT_LAUNCH_OPTIONS`](#playwright_launch_options)
setting do not apply.
setting do not apply.

### Creating a context during a crawl

Expand Down Expand Up @@ -592,14 +592,22 @@ def parse(self, response):
async def parse_in_new_context(self, response):
page = response.meta["playwright_page"]
title = await page.title()
await page.close()
await page.context.close()
return {"title": title}

async def close_context_on_error(self, failure):
page = failure.request.meta["playwright_page"]
await page.close()
await page.context.close()
```

### Avoid race conditions & memory leaks when closing contexts
Make sure to close the page before closing the context. See
[this comment](https://github.com/scrapy-plugins/scrapy-playwright/issues/191#issuecomment-1548097114)
in [#191](https://github.com/scrapy-plugins/scrapy-playwright/issues/191)
for more information.

### Maximum concurrent context count

Specify a value for the `PLAYWRIGHT_MAX_CONTEXTS` setting to limit the amount
Expand Down
1 change: 1 addition & 0 deletions examples/contexts.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ async def parse(self, response):
page = response.meta["playwright_page"]
context_name = response.meta["playwright_context"]
storage_state = await page.context.storage_state()
await page.close()
await page.context.close()
return {
"url": response.url,
Expand Down
2 changes: 1 addition & 1 deletion tests/test_browser_contexts.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,8 @@ async def test_contexts_startup(self):

page = resp.meta["playwright_page"]
storage_state = await page.context.storage_state()
await page.context.close()
await page.close()
await page.context.close()
cookie = storage_state["cookies"][0]
assert cookie["name"] == "foo"
assert cookie["value"] == "bar"
Expand Down

0 comments on commit b25c64c

Please sign in to comment.