Skip to content

Commit

Permalink
Address review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
janbuchar committed Sep 10, 2024
1 parent 354e321 commit 52c27f9
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 16 deletions.
15 changes: 5 additions & 10 deletions docs/deployment/apify_platform.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
id: apify-platform
title: Apify Platform
title: Apify platform
description: Apify platform - large-scale and high-performance web scraping
---

Expand Down Expand Up @@ -126,8 +126,8 @@ There are several things worth mentioning here.
### Helper functions for default Key-Value Store and Dataset

To simplify access to the _default_ storages, instead of using the helper functions of respective storage classes, you could use:
- [`Actor.set_value()`](https://docs.apify.com/sdk/python/reference/class/Actor#set_value), [`Actor.get_value()`](https://docs.apify.com/sdk/python/reference/class/Actor#get_value), [`Actor.get_input()`](https://docs.apify.com/sdk/python/reference/class/Actor#get_input) for `Key-Value Store`
- [`Actor.push_data()`](https://docs.apify.com/sdk/python/reference/class/Actor#push_data) for `Dataset`
- [`Actor.set_value()`](https://docs.apify.com/sdk/python/reference/class/Actor#set_value), [`Actor.get_value()`](https://docs.apify.com/sdk/python/reference/class/Actor#get_value), [`Actor.get_input()`](https://docs.apify.com/sdk/python/reference/class/Actor#get_input) for [`Key-Value Store`](https://docs.apify.com/sdk/python/reference/class/KeyValueStore)
- [`Actor.push_data()`](https://docs.apify.com/sdk/python/reference/class/Actor#push_data) for [`Dataset`](https://docs.apify.com/sdk/python/reference/class/Dataset)

### Using platform storage in a local Actor

Expand All @@ -141,7 +141,7 @@ If you don't plan to force usage of the platform storages when running the Actor

### Getting public url of an item in the platform storage

If you need to share a link to some file stored in a Key-Value Store on Apify Platform, you can use [`get_public_url()`](https://docs.apify.com/sdk/python/reference/class/KeyValueStore#get_public_url) method. It accepts only one parameter: `key` - the key of the item you want to share.
If you need to share a link to some file stored in a [Key-Value](https://docs.apify.com/sdk/python/reference/class/KeyValueStore) Store on Apify Platform, you can use [`get_public_url()`](https://docs.apify.com/sdk/python/reference/class/KeyValueStore#get_public_url) method. It accepts only one parameter: `key` - the key of the item you want to share.

<CodeBlock language="python">
{GetPublicUrlSource}
Expand Down Expand Up @@ -238,12 +238,7 @@ in the [proxy dashboard](https://console.apify.com/proxy).

### Apify Proxy vs. Own proxies

The `ProxyConfiguration` class covers both Apify Proxy and custom proxy URLs so that
you can easily switch between proxy providers. However, some features of the class
are available only to Apify Proxy users, mainly because Apify Proxy is what
one would call a super-proxy. It's not a single proxy server, but an API endpoint
that allows connection through millions of different IP addresses. So the class
essentially has two modes: Apify Proxy or Own (third party) proxy.
The [`ProxyConfiguration`](https://docs.apify.com/sdk/python/reference/class/ProxyConfiguration) class covers both Apify Proxy and custom proxy URLs so that you can easily switch between proxy providers. However, some features of the class are available only to Apify Proxy users, mainly because Apify Proxy is what one would call a super-proxy. It's not a single proxy server, but an API endpoint that allows connection through millions of different IP addresses. So the class essentially has two modes: Apify Proxy or Own (third party) proxy.

The difference is easy to remember.
- If you're using your own proxies - you should create a <ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> instance directly.
Expand Down
8 changes: 4 additions & 4 deletions docs/introduction/09_running_in_cloud.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ We started this guide by using the Crawlee CLI to bootstrap the project - it off

The first step will be installing two new dependencies:

- Apify SDK, a toolkit for working with the Apify Platform. This will allow us to wire the storages (e.g. `RequestQueue` and `Dataset`) to the Apify cloud products. This will be a dependency of our project.
- Apify SDK, a toolkit for working with the Apify Platform. This will allow us to wire the storages (e.g. [`RequestQueue`](https://docs.apify.com/sdk/python/reference/class/RequestQueue) and [`Dataset`](https://docs.apify.com/sdk/python/reference/class/Dataset)) to the Apify cloud products. This will be a dependency of our project.

```bash
poetry add apify
Expand Down Expand Up @@ -55,9 +55,9 @@ apify login
## Adjusting the code
Now that you have your account set up, you will need to adjust the code a tiny bit. We will use the [Apify SDK](https://docs.apify.com/sdk/js/), which will help us to wire the Crawlee storages (like the `RequestQueue`) to their Apify Platform counterparts - otherwise Crawlee would keep things only in memory.
Now that you have your account set up, you will need to adjust the code a tiny bit. We will use the [Apify SDK](https://docs.apify.com/sdk/python/), which will help us to wire the Crawlee storages (like the [`RequestQueue`](https://docs.apify.com/sdk/python/reference/class/RequestQueue)) to their Apify Platform counterparts - otherwise Crawlee would keep things only in memory.
Open your `src/main.py` file, and wrap everyting in your `main` function with the `Actor` context manager. Your code should look like this:
Open your `src/main.py` file, and wrap everyting in your `main` function with the [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) context manager. Your code should look like this:
<CodeBlock className="language-python" title="src/main.py">
{MainExample}
Expand All @@ -67,7 +67,7 @@ The context manager will configure Crawlee to use the Apify API instead of its d
:::info Understanding `async with Actor` behavior with environment variables
The `Actor` context manager works conditionally based on the environment variables, namely based on the `APIFY_IS_AT_HOME` env var, which is set to `true` on the Apify Platform. This means that your project will remain working the same locally, but will use the Apify API when deployed to the Apify Platform.
The [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) context manager works conditionally based on the environment variables, namely based on the `APIFY_IS_AT_HOME` env var, which is set to `true` on the Apify Platform. This means that your project will remain working the same locally, but will use the Apify API when deployed to the Apify Platform.
:::
Expand Down
2 changes: 0 additions & 2 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,6 @@ module.exports = {
'introduction/saving-data',
'introduction/refactoring',
'introduction/deployment',
// TODO: add once SDK v2 is released
// 'introduction/running-in-cloud',
],
},
{
Expand Down

0 comments on commit 52c27f9

Please sign in to comment.