O'Reilly logo

Learning Scrapy by Dimitrios Kouzis-Loukas

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 3. Basic Crawling

This is a very important chapter, which you will probably read several times, and return to often for finding solutions. We are going to start by explaining how to install Scrapy, and then move on to the methodology of developing Scrapy crawlers along with numerous examples and alternative implementations. Before we start, let's take a look at some important notions.

Since we are quickly moving to the fun coding part, it's important to be able to use the code segments you find in this book. When you see the following:

$ echo hello world
hello world

it means you are to type echo hello world on a terminal (skip the dollar sign). The line(s) that follow are the output as seen on your terminal.

Tip

We will use the terms 'terminal', ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required