The Twisted Framework

The Twisted framework is an alternative way of writing networked applications in Python. While the classes in SocketServer are designed around spawning new threads and processes to handle requests, Twisted uses a loop over the select function (as in the client example above) to timeshare between all pending processes.

Download the Twisted framework libraries from the Twisted web site at http://twistedmatrix. com/projects/core/, and install them.

For simple applications, programming in Twisted can be almost exactly like programming using the SocketServer classes. Below is TwistedMirrorServer.py, a Twisted implementation of the mirror server defined earlier in this chapter. Note that it looks a lot like the implementation that used SocketServer classes, once we account for the fact that Twisted uses different names for the objects provided by both it and the SocketServer framework (for instance, Twisted uses "factory" instead of "server" and "transport" instead of "wfile"):

from twisted.internet import protocol, reactor from twisted.protocols import basic class MirrorProtocol(basic.LineReceiver):

"Handles one request to mirror some text."

def lineReceived(self, line):

"""The client has sent in a line of text. Write out the mirrored version."""

class MirrorFactory(protocol.ServerFactory): protocol = MirrorProtocol if _name_ == '_main_':

print 'Usage: %s [hostname] [port number]' % sys.argv[0] sys.exit(l) hostname = sys.argv[1] port = int(sys.argv[2])

reactor.listenTCP(port, MirrorFactory(), interface=hostname) reactor.run()

This works just the same as the other MirrorServer implementations, but it runs faster because there's no need to spawn new threads.

Deferred Objects

Because Twisted servers run all of their code in a single thread, it's very important that you write your Twisted code so that it never blocks waiting for something to happen. It's bad enough when a single request drags on because the server has to consult a slow database to fulfill it—imagine what it would be like if every request were stopped in its tracks just because one of them caused a database call.

The Twisted team has implemented new, blocking-free ways to do just about anything that might cause a process to block: accessing a database, getting output from a subprocess, and using most of the popular Internet protocols. Behind the scenes, these implementations either feed into the same select loop that drives your main application, or they use threads.

In Twisted, the standard way to do something that might take a while is to obtain a Deferred object that knows how to do it and then register what you want to do next as a callback of the Deferred object.

Suppose you have some users who use your TwistedMirrorServer so much that it's putting a load on your CPU. You decide to change the mirror server so that any given user can only mirror one line of text every five seconds. You might be tempted to implement this feature by calling time.sleep for the appropriate interval if a user tries to use the server too often, like this:

#!/usr/bin/python

#This example is BAD! Do not use it! from twisted.internet import protocol, reactor from twisted.protocols import basic import time class MirrorProtocol(basic.LineReceiver):

"Handles one request to mirror some text."

"""Set the timeout counter to a value that will always let a new user's first request succeed immediately.""" self.lastUsed = 0

def lineReceived(self, line):

"""The client has sent in a line of text. Write out the mirrored version, possibly waiting for a timeout to expire before we do. Note: this is a very bad implementation because we're running this in a Twisted server, but time.sleep() is a blocking call."""

elapsed = time.time() - self.lastUsed print elapsed if elapsed < (self.factory.PER_USER_TIMEOUT * 1000):

time.sleep(self.factory.PER_USER_TIMEOUT-elapsed) self.transport.write(line[::-1]+ '\r\n') self.lastUsed = time.time()

class MirrorFactory(protocol.ServerFactory):

"A server for the Mirror protocol defined above." protocol = MirrorProtocol PER_USER_TIMEOUT = 5

The problem is that time.sleep blocks the thread until it's complete. Since Twisted servers run in a single thread, calling time.sleep will prevent any client from having their text mirrored until that time.sleep call returns.

Fortunately, the Twisted team has implemented a non-blocking equivalent to time.sleep, called callLater. This method returns a Deferred object that will call the given function after a certain amount of time has elapsed. This gives you the equivalent functionality of time.sleep, but it doesn't block, so the ability of the Twisted server to deal with other clients is not impaired:

from twisted.internet import protocol, reactor from twisted.protocols import basic import time class MirrorProtocol(basic.LineReceiver):

"Handles one request to mirror some text."

"""Set the timeout counter to a value that will always let a new user's first request succeed immediately.""" self.lastUsed = 0

def lineReceived(self, line):

"""The client has sent in a line of text. Write out the mirrored version, possibly waiting for a timeout to expire before we do. This is a good implementation because it uses a method that returns a Deferred object (reactor.callLater()) and registers a callback (writeLine) with that object."""

elapsed = time.time() - self.lastUsed if elapsed < self.factory.PER_USER_TIMEOUT:

reactor.callLater(self.factory.PER_USER_TIMEOUT-elapsed, self.writeLine, line)

else:

self.writeLine(line)

def writeLine(self, line):

"Writes the given line and sets the user's timeout." self.transport.write(line[::-1] + '\r\n') self.lastUsed = time.time()

class MirrorFactory(protocol.ServerFactory):

"A server for the Mirror protocol defined above." protocol = MirrorProtocol PER_USER_TIMEOUT = 5

print 'Usage: %s [hostname] [port number]' % sys.argv[0] sys.exit(l) hostname = sys.argv[1] port = int(sys.argv[2])

reactor.listenTCP(port, MirrorFactory(), interface=hostname) reactor.run()

This is not a general example: It only works because callLater has already been implemented as a non-blocking equivalent to the blocking sleep function. If you're going to write Twisted code, you'll need to find or write a non-blocking equivalent to every blocking function you want to call. Using Twisted requires a different way of thinking about programming, but its unique approach offers a higher-performance way to write network clients and servers.

Was this article helpful?

0 0