O'Reilly logo

Learning Scrapy by Dimitrios Kouzis-Loukas

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Further settings

We will now explore some less common aspects of Scrapy and settings related to extending Scrapy, which we will see in more detail in later chapters.

Further settings

Further Scrapy settings

Project-related settings

Under this umbrella, you will find housekeeping settings related to a specific project, such as BOT_NAME, SPIDER_MODULES, and so on. It's good to have a quick look at them in the documentation because they may increase your productivity for specific use cases, but typically, Scrapy's startproject and genspider commands provide sensible defaults and you may be okay without ever explicitly changing them. Mail-related settings, such as MAIL_FROM ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required