Reading the spider web

In the Spider class, the process_page() method is separate from run() because run() is easier to understand when you don't have to look at all the code in process_page() at the same time. This choice exemplifies two reasons behind Python's modular design philosophy.

It's a good idea to separate out chunks of code that do different things:

• Separating tasks into chunks makes it possible to use each chunk on its own.

• Reading a chunk of code that does only one thing is easier than reading a chunk of code that does multiple things.

Was this article helpful?

0 0

Post a comment