docs: add-blog #2539

souravjain540 · 2024-06-14T17:58:17Z

Adding new blog to Crawlee Blog

souravjain540 · 2024-06-17T13:06:54Z

Cc: @janbuchar / @barjin

janbuchar

Nice article. I found several minor issues worth addressing. Also it'd be nice to

link to a repo with the complete source code, ideally it should be hosted on Apify's github
run all code snippets through a code formatter

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

janbuchar · 2024-06-17T13:39:10Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+
+## Prerequisites
+
+To use Crawlee, you need to have Node.js 16 or higher version.


Suggested change

To use Crawlee, you need to have Node.js 16 or higher version.

To use Crawlee, you need to have Node.js 16 or newer.

janbuchar · 2024-06-17T13:40:18Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+
+You can install the latest version of Node.js from the [official website](https://nodejs.org/en/). This great [Node.js installation guide](https://blog.apify.com/how-to-install-nodejs/) gives you tips to avoid issues later on. 
+
+## Creating React app


Suggested change

## Creating React app

## Creating a React app

janbuchar · 2024-06-17T13:40:50Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+npx create-vite@latest
+```
+
+You can check out the [Vite Docs](https://vitejs.dev/guide/) to create a React app.


Suggested change

You can check out the [Vite Docs](https://vitejs.dev/guide/) to create a React app.

You can check out the [Vite Docs](https://vitejs.dev/guide/) for more details on how to create a React app.

janbuchar · 2024-06-17T13:44:25Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+
+Additionally, Crawlee supports headless browser libraries like [Playwright](https://playwright.dev/) and [Puppeteer](https://pptr.dev/) for scraping of websites that are JavaScript-rendered. 
+
+After installing the libraries, it’s time to create the scraper code.


Wait, which ones do I need? Netflix is an SPA, so I'll need Playwright or Puppeteer, right? Which one do we want?

Netflix is an SPA but in this use case, it works good with CheerioCrawler.

Then we should just remove this or reword it so that it's clear that playwright will not be necessary.

janbuchar · 2024-06-17T13:57:11Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+const allShows = [];
+let genreShows = [];
+shows.forEach((show) => {
+    genreShows.push(show);
+    if (genreShows.length === 40) {
+    allShows.push(genreShows);
+    genreShows = [];
+    }
+});
+if (genreShows.length > 0) {
+    allShows.push(genreShows);
+}


Instead of relying on the shows array to be sorted by genre and having exactly 40 items, I'd make a Map with the genre as key and an array of show titles as value.

Or - even better - capture the structure straight from the HTML. You can call $(element) to query the element's subtree.

const out = $('[data-uia="collections-row"]').map((_, el) => { // get a genre row const genre = $(el).find('[data-uia="collections-row-title"]').text(); // pick its (genre) title const items = $(el).find('[data-uia="collections-title"]').map((_, el) => $(el).text()).get(); // pick all items in the genre return { genre, items }; });

janbuchar · 2024-06-17T13:59:17Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+npm start
+```
+
+After running this command, you will see a `storage` folder with the `key_value_stores/default/results.json` file. The scrapped data will be stored in JSON format in this file.


Suggested change

After running this command, you will see a `storage` folder with the `key_value_stores/default/results.json` file. The scrapped data will be stored in JSON format in this file.

After running this command, you will see a `storage` folder with the `key_value_stores/default/results.json` file. The scraped data will be stored in JSON format in this file.

janbuchar · 2024-06-17T14:01:19Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+}
+
+function App() {
+  const [count, setCount] = useState(null);


I think this is not a count, it's an index into an array of shows, isn't it?

barjin · 2024-06-18T08:45:02Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+Add the following code in `'scripts'` object:
+
+```
+'start': 'node src/scraper.js'


Suggested change

'start': 'node src/scraper.js'

"start": "node src/scraper.js"

barjin · 2024-06-18T08:52:50Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+slug: netflix-show-recommender
+title: 'Building a Netflix show recommender using Crawlee and React'
+tags: [community]
+description: 'Create a Netflix show recommendation system using Crawlee to scrape the data, JavaScript to code, and React to build the front end.'


Is this the wording we agreed upon? 😄 If I clicked on an article about recommender systems with scraping (sounds super cool) and got a simple React App, I would be a bit disappointed.

Netflix is a large player in the realm of recommender systems, with the Netflix Prize, their research papers, and stuff... This article is going to have a lot of very strong SEO competition with this name.

Yes, we agreed on it. I got the point, and I agree you are right, but this article is not from us. As said in the beginning, it is from one of the community members, and we made it clear that it is not supposed to be that perfect; it is just an app showing something like this can be created through a little work.

janbuchar · 2024-06-19T07:57:16Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md


 :::tip
-Before we start this tutorial, we recommend you [visit Crawlee's GitHub](https://github.com/apify/crawlee) and check out the codebase and installation guide. If you like Crawlee, do give us a star. 
+If you are liking this blog so far, we request you to [give Crawlee a star on GitHub](https://github.com/apify/crawlee), it helps us to reach and help more developers. 


One paragraph in? That's a bit early to like it.

janbuchar · 2024-06-19T08:11:36Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+      genre: genres,
+      shows: shows,


This looks suspicious - storing genres (plural) under genre (singular)

souravjain540 · 2024-06-19T11:21:57Z

@janbuchar can we please make it live today? I need to give link to marketing for the newsletter :)

janbuchar · 2024-06-19T11:24:08Z

website/blog/2024/06-10-creating-a-netflix-show-recommender-using-crawlee-and-react/index.md

+To use Crawlee, you need to have Node.js 16 or newer.
+
+:::tip
+If you are Crawlee blog so far, we request you to [give Crawlee a star on GitHub](https://github.com/apify/crawlee), it helps us to reach and help more developers. 


Suggested change

If you are Crawlee blog so far, we request you to [give Crawlee a star on GitHub](https://github.com/apify/crawlee), it helps us to reach and help more developers.

If you like the posts on the Crawlee blog so far, please consider [giving Crawlee a star on GitHub](https://github.com/apify/crawlee), it helps us to reach and help more developers.

janbuchar · 2024-06-19T11:28:53Z

@janbuchar can we please make it live today? I need to give link to marketing for the newsletter :)

I believe so, there is just a handful of comments to resolve.

souravjain540 · 2024-06-19T12:38:13Z

@janbuchar done :)

add-blog

2fd87cb

souravjain540 requested review from janbuchar and B4nan June 14, 2024 17:58

souravjain540 self-assigned this Jun 14, 2024

add a note

0abf160

janbuchar reviewed Jun 17, 2024

View reviewed changes

barjin reviewed Jun 18, 2024

View reviewed changes

fixes

5e177aa

souravjain540 requested a review from janbuchar June 19, 2024 07:02

janbuchar reviewed Jun 19, 2024

View reviewed changes

fix

12e843d

souravjain540 requested a review from janbuchar June 19, 2024 09:52

janbuchar reviewed Jun 19, 2024

View reviewed changes

last-fix

065a9f6

souravjain540 requested a review from janbuchar June 19, 2024 12:37

janbuchar approved these changes Jun 19, 2024

View reviewed changes

janbuchar merged commit b5d063b into master Jun 19, 2024
9 checks passed

janbuchar deleted the add-blog branch June 19, 2024 13:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add-blog #2539

docs: add-blog #2539

souravjain540 commented Jun 14, 2024

souravjain540 commented Jun 17, 2024

janbuchar left a comment

janbuchar Jun 17, 2024

janbuchar Jun 17, 2024

janbuchar Jun 17, 2024

janbuchar Jun 17, 2024

souravjain540 Jun 17, 2024

janbuchar Jun 17, 2024

janbuchar Jun 17, 2024

barjin Jun 18, 2024 •

edited

Loading

janbuchar Jun 17, 2024

janbuchar Jun 17, 2024

barjin Jun 18, 2024

barjin Jun 18, 2024

souravjain540 Jun 18, 2024

janbuchar Jun 19, 2024

janbuchar Jun 19, 2024

souravjain540 commented Jun 19, 2024

janbuchar Jun 19, 2024

janbuchar commented Jun 19, 2024

souravjain540 commented Jun 19, 2024


		## Prerequisites

		To use Crawlee, you need to have Node.js 16 or higher version.


		You can install the latest version of Node.js from the [official website](https://nodejs.org/en/). This great [Node.js installation guide](https://blog.apify.com/how-to-install-nodejs/) gives you tips to avoid issues later on.

		## Creating React app

	You can check out the [Vite Docs](https://vitejs.dev/guide/) to create a React app.
	You can check out the [Vite Docs](https://vitejs.dev/guide/) for more details on how to create a React app.


		Additionally, Crawlee supports headless browser libraries like [Playwright](https://playwright.dev/) and [Puppeteer](https://pptr.dev/) for scraping of websites that are JavaScript-rendered.

		After installing the libraries, it’s time to create the scraper code.

	After running this command, you will see a `storage` folder with the `key_value_stores/default/results.json` file. The scrapped data will be stored in JSON format in this file.
	After running this command, you will see a `storage` folder with the `key_value_stores/default/results.json` file. The scraped data will be stored in JSON format in this file.

	'start': 'node src/scraper.js'
	"start": "node src/scraper.js"

	If you are Crawlee blog so far, we request you to [give Crawlee a star on GitHub](https://github.com/apify/crawlee), it helps us to reach and help more developers.
	If you like the posts on the Crawlee blog so far, please consider [giving Crawlee a star on GitHub](https://github.com/apify/crawlee), it helps us to reach and help more developers.

docs: add-blog #2539

docs: add-blog #2539

Conversation

souravjain540 commented Jun 14, 2024

souravjain540 commented Jun 17, 2024

janbuchar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

barjin Jun 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

souravjain540 commented Jun 19, 2024

Choose a reason for hiding this comment

janbuchar commented Jun 19, 2024

souravjain540 commented Jun 19, 2024

barjin Jun 18, 2024 •

edited

Loading