Few Special Cases

Here's what a continuation line looks like using the open syntactic pairs rule. Delimited constructs, such as lists in square brackets, can span across any number of lines L Good, Bad, Ugly Open pairs may span lines This also works for anything in parentheses (expressions, function arguments, function headers, tuples, and generator expressions), as well as anything in curly braces (dictionaries and, in 3.0, set literals and set and dictionary comprehensions). Some of these are tools we'll study...

First Example

Let's turn to a real example to show how these ideas work in practice. To begin, let's define a class named FirstClass by running a Python class statement interactively > > > class FirstClass Define a class object def setdata(self, value) Define class methods self.data value self is the instance print(self.data) self.data per instance We're working interactively here, but typically, such a statement would be run when the module file it is coded in is imported. Like functions created with...

Second Example Intersecting Sequences

Let's look at a second function example that does something a bit more useful than multiplying arguments and further illustrates function basics. In Chapter 13, we coded a for loop that collected items held in common in two strings. We noted there that the code wasn't as useful as it could be because it was set up to work only on specific variables and could not be rerun later. Of course, we could copy the code and paste it into each place where it needs to be run, but this solution is neither...

Simple Interactive Loop

Suppose you're asked to write a Python program that interacts with a user in a console window. Maybe you're accepting inputs to send to a database, or reading numbers to be used in a calculation. Regardless of the purpose, you need to code a loop that reads one or more inputs from a user typing on a keyboard, and prints back a result for each. In other words, you need to write a classic read evaluate print loop program. In Python, typical boilerplate code for such an interactive loop might look...

Tale of Three Systems

The only time package imports are actually required is to resolve ambiguities that may arise when multiple programs with same-named files are installed on a single machine. This is something of an install issue, but it can also become a concern in general practice. Let's turn to a hypothetical scenario to illustrate. Suppose that a programmer develops a Python program that contains a file called utilities.py for common utility code and a top-level file named main.py that users launch to start...

Third Example

This time, we'll define a subclass of SecondClass that implements three specially named attributes that Python will call automatically _init_is run when a new instance object is created (self is the new ThirdClass _add_is run when a ThirdClass instance appears in a + expression. _str_is run when an object is printed (technically, when it's converted to its print string by the str built-in function or its Python internals equivalent). Our new subclass also defines a...

About the Programs in This Book

This fourth edition of this book, and all the program examples in it, is based on Python version 3.0. In addition, most of its examples run under Python 2.6, as described in the text, and notes for Python 2.6 readers are mixed in along the way. Because this text focuses on the core language, however, you can be fairly sure that most of what it has to say won't change very much in future releases of Python. Most of this book applies to earlier Python versions, too, except when it does not...

Acknowledgments

As I write this fourth edition of this book in 2009, I can't help but be in a sort of mission accomplished state of mind. I have now been using and promoting Python for 17 years, and have been teaching it for 12 years. Despite the passage of time and events, I am still constantly amazed at how successful Python has been over the years. It has grown in ways that most of us could not possibly have imagined in 1992. So, at the risk of sounding like a hopelessly self-absorbed author, you'll have to...

Adding Decorator Arguments

The timer decorator of the prior section works, but it would be nice if it was more configurable providing an output label and turning trace messages on and off, for instance, might be useful in a general-purpose tool like this. Decorator arguments come in handy here when they're coded properly, we can use them to specify configuration options that can vary for each decorated function. A label, for instance, might be added as follows def timer(label '') def decorator(func) def onCall(*args)...

Adding Specific Formatting

Another similarity with expressions is that more specific layouts can be achieved by adding extra syntax in the format string. For the formatting method, we use a colon after the substitution target's identification, followed by a format specifier that can name the field size, justification, and a specific type code. Here's the formal structure of what can appear as a substitution target in a format string fieldname is a number or keyword naming an argument, followed by optional .name attribute...

Adding Tests and Nested Loops filter

List comprehensions are even more general than shown so far. For instance, as we learned in Chapter 14, you can code an if clause after the for to add selection logic. List comprehensions with if clauses can be thought of as analogous to the filter built-in discussed in the prior chapter they skip sequence items for which the if clause is not true. To demonstrate, here are both schemes picking up even numbers from 0 to 4 like the map list comprehension alternative of the prior section, the...

Advanced Module Topics

This chapter concludes this part of the book with a collection of more advanced module-related topics data hiding, the_future_ module, the_name_variable, sys.path changes, listing tools, running modules by name string, transitive reloads, and so on along with the standard set of gotchas and exercises related to what we've covered in this part of the book. Along the way, we'll build some larger and more useful tools than we have so far, that combine functions and modules. Like functions, modules...

Advanced String Formatting Expressions

For more advanced type-specific formatting, you can use any of the conversion type codes listed in Table 7-4 in formatting expressions they appear after the character in substitution targets. C programmers will recognize most of these because Python string formatting supports all the usual C printf format codes (but returns the result, instead of displaying it, like printf). Some of the format codes in the table provide alternative ways to format the same type for instance, e, f, and g provide...

Argument Passing Basics

Earlier in this part of the book, I noted that arguments are passed by assignment. This has a few ramifications that aren't always obvious to beginners, which I'll expand on in this section. Here is a rundown of the key points in passing arguments to functions Arguments are passed by automatically assigning objects to local variable names. Function arguments references to (possibly) shared objects sent by the caller are just another instance of Python assignment at work. Because references are...

Arguments and Shared References

a is assigned to (references) passed object Changes local variable a only > > > b 88 > > > f(b) > > > print(b) a and b both reference same 88 initially > > > def changer(a, b) Arguments assigned references to objects a 2 Changes local name's value only b 0 'spam' Changes shared object in-place > > > L 1, 2 Caller > > > changer(X, L) Pass immutable and mutable objects > > > X, L X is unchanged, L is different In this code, the changer function assigns...

Assignment Statements

We've been using the Python assignment statement for a while to assign objects to names. In its basic form, you write the target of an assignment on the left of an equals sign, and the object to be assigned on the right. The target on the left may be a name or object component, and the object on the right can be an arbitrary expression that computes an object. For the most part, assignments are straightforward, but here are a few properties to keep in mind Assignments create object references....

Assignments Expressions and Prints

Now that we've had a quick introduction to Python statement syntax, this chapter begins our in-depth tour of specific Python statements. We'll begin with the basics assignment statements, expression statements, and print operations. We've already seen all of these in action, but here we'll fill in important details we've skipped so far. Although they're fairly simple, as you'll see, there are optional variations for each of these statement types that will come in handy once you begin writing...

Augmenting Methods The Good

What we really want to do here is somehow augment the original giveRaise, instead of replacing it altogether. The good way to do that in Python is by calling to the original version directly, with augmented arguments, like this def giveRaise(self, percent, bonus .10) Person.giveRaise(self, percent + bonus) Good augment original This code leverages the fact that a class method can always be called either through an instance (the usual way, where Python sends the instance to the self argument...

Automatic stream redirection

This technique of redirecting printed text by assigning sys.stdout is commonly used in practice. One potential problem with the last section's code, though, is that there is no direct way to restore the original output stream should you need to switch back after printing to a file. Because sys.stdout is just a normal file object, you can always save it and restore it if needed C misc> c python30 python > > > import sys > > > temp sys.stdout Save for restoring later > > >...

Avoiding Mutable Argument Changes

This behavior of in-place changes to mutable arguments isn't a bug it's simply the way argument passing works in Python. Arguments are passed to functions by reference (a.k.a. pointer) by default because that is what we normally want. It means we can pass large objects around our programs without making multiple copies along the way, and we can easily update these objects as we go. In fact, as we'll see in Part VI, Python's class model depends upon changing a passed-in self argument in-place,...

Basic Dictionary Operations

In normal operation, you create dictionaries with literals and store and access items by key with indexing python > > > D 'spam' 2, 'ham' 1, 'eggs' 3 Make a dictionary > > > D 'spam' Fetch a value by key > > > D Order is scrambled Here, the dictionary is assigned to the variable D the value of the key 'spam' is the integer 2, and so on. We use the same square bracket syntax to index dictionaries by key as we did to index lists by offset, but here it means access by key, not by...

Basic Usage

The basic format of the with statement looks like this with expression as variable with-block The expression here is assumed to return an object that supports the context management protocol (more on this protocol in a moment). This object may also return a value that will be assigned to the name variable if the optional as clause is present. Note that the variable is not necessarily assigned the result of the expression the result of the expression is the object that supports the context...

Beware of Cyclic Data Structures

We actually encountered this concept in a prior exercise if a collection object contains a reference to itself, it's called a cyclic object. Python prints a whenever it detects a cycle in the object, rather than getting stuck in an infinite loop > > > L 'grail' Append reference to same object > > > L.append(L) Generates cycle in object Besides understanding that the three dots in square brackets represent a cycle in the object, this case is worth knowing about because it can lead to...

Block Delimiters Indentation Rules

Python detects block boundaries automatically, by line indentation that is, the empty space to the left of your code. All statements indented the same distance to the right belong to the same block of code. In other words, the statements within a block line up vertically, as in a column. The block ends when the end of the file or a lesser-indented line is encountered, and more deeply nested blocks are simply indented further to the right than the statements in the enclosing block. For instance,...

Bonus Points

You can get can get bonus points here for changing these functions to compute the maximum, rather than minimum, values. This one's easy the first two versions only require changing < to > , and the third simply requires that we return tmp -l instead of tmp 0 . For an extra point, be sure to set the function name to max as well (though this part is strictly optional). It's also possible to generalize a single function to compute either a minimum or a maximum value, by evaluating comparison...

Bound Methods and Other Callable Objects

As mentioned earlier, bound methods can be processed as generic objects, just like simple functions they can be passed around a program arbitrarily. Moreover, because bound methods combine both a function and an instance in a single package, they can be treated like any other callable object and require no special syntax when invoked. The following, for example, stores four bound method objects in a list and calls them later with normal call expressions > > > x Number(2) Class instance...

Builtin Numeric Tools

Besides the built-in number literals shown in Table 5-1, Python provides a set of tools for processing number objects +, -, *, , > > , **, & , etc. pow, abs, round, int, hex, bin, etc. Utility modules We'll meet all of these as we go along. Although numbers are primarily processed with expressions, built-ins, and modules, they also have a handful of type-specific methods today, which we'll meet in this chapter as well. Floating-point numbers, for example, have an as_integer_ratio method...

Changing Mutable Class Attributes Can Have Side Effects

This gotcha is really an extension of the prior. Because class attributes are shared by all instances, if a class attribute references a mutable object, changing that object in-place from any instance impacts all instances at once > > > class C shared Class attribute def _init_(self) self.perobj Instance attribute > > > x C() Two instances > > > y C() Implicitly share class attrs > > > y.shared, y.perobj ( , ) > > > x.shared.append('spam') Impacts y s view too >...

Chapter Summary

This chapter introduced the basics of coding classes in Python. We studied the syntax of the class statement, and we saw how to use it to build up a class inheritance tree. We also studied how Python automatically fills in the first argument in method functions, how attributes are attached to objects in a class tree by simple assignment, and how specially named operator overloading methods intercept and implement built-in operations for our instances (e.g., expressions and printing). Now that...

Character Encoding Schemes

Most programmers think of strings as series of characters used to represent textual data. The way characters are stored in a computer's memory can vary, though, depending on what sort of character set must be recorded. The ASCII standard was created in the U.S., and it defines many U.S. programmers' notion of text strings. ASCII defines character codes from 0 through 127 and allows each character to be stored in one 8-bit byte (only 7 bits of which are actually used). For example, the ASCII...

Class Blunders II Retaining Multiple Instances

Curiously, the decorator function in this example can almost be coded as a class instead of a function, with the proper operator overloading protocol. The following slightly simplified alternative works similarly because its_init_is triggered when the decorator is applied to the class, and its_call_is triggered when a subject class instance is created. Our objects are really instances of Tracer this time, and we essentially just trade an enclosing scope reference for an instance attribute here...

Classes Can Intercept Python Operators

Let's move on to the third major difference between classes and modules operator overloading. In simple terms, operator overloading lets objects coded with classes intercept and respond to operations that work on built-in types addition, slicing, printing, qualification, and so on. It's mostly just an automatic dispatch mechanism expressions and other built-in operations route control to implementations in classes. Here, too, there is nothing similar in modules modules can implement function...

Classes Versus Dictionaries

Although the simple classes of the prior section are meant to illustrate class model basics, the techniques they employ can also be used for real work. For example, Chapter 8 showed how to use dictionaries to record properties of entities in our programs. It turns out that classes can serve this role, too they package information like dictionaries, but can also bundle processing logic in the form of methods. For reference, here is the example for dictionary-based records we used earlier in the...

Clicking File Icons

On Windows, the Registry makes opening files with icon clicks easy. Python automatically registers itself to be the program that opens Python program files when they are clicked. Because of that, it is possible to launch the Python programs you write by simply clicking (or double-clicking) on their file icons with your mouse cursor. On non-Windows systems, you will probably be able to perform a similar trick, but the icons, file explorer, navigation schemes, and more may differ slightly. On...

Clicking Icons on Windows

To illustrate, let's keep using the script we wrote earlier, script1.py, repeated here to minimize page flipping import sys Load a library module print(2 ** 100) Raise 2 to a power print(x * 8) String repetition As we've seen, you can always run this file from a system command line C misc> c python30 python scriptl.py However, icon clicks allow you to run the file without any typing at all. If you find this file's icon for instance, by selecting Computer (or My Computer in XP) in your Start...

Coding Ascii Text

Let's step through some examples that demonstrate text coding basics. As we've seen, ASCII text is a simple type of Unicode, stored as a sequence of byte values that represent characters > > > ord('X') 'X' has binary value 88 in the default encoding > > > chr(88) 88 stands for character 'X' > > > S 'XYZ' A Unicode string of ASCII text > > > len(S) 3 characters long > > > ord(c) for c in S 3 bytes with integer ordinal values 88, 89, 90 Normal 7-bit ASCII text like...

Coding Exceptions Classes

Let's look at an example to see how class exceptions translate to code. In the following file, classexc.py, we define a superclass called General and two subclasses called Specificl and Specific2. This example illustrates the notion of exception categories General is a category name, and its two subclasses are specific types of exceptions within the category. Handlers that catch General will also catch any subclasses of it, including Specific1 and Specific2 class General(Exception) pass class...

Coding Functions

Although it wasn't made very formal, we've already used some functions in earlier chapters. For instance, to make a file object, we called the built-in open function similarly, we used the len built-in function to ask for the number of items in a collection object. In this chapter, we will explore how to write new functions in Python. Functions we write behave the same way as the built-ins we've already seen they are called in expressions, are passed values, and return results. But writing new...

Coding Mixin Display Classes

As we've seen, Python's default way to print a class instance object isn't incredibly useful def _init_(self) No_repr_or_str_ self.data1 food > > > print(X) Default class, address < _main_.Spam object at 0x008648l8> Displays instance in Python 2.6 As you saw in Chapter 29 when studying operator overloading, you can provide a _str_or_repr_method to implement a custom string representation of your own. But, rather than coding one of these in each and every class you wish to print, why...

Coding Properties with Decorators

Although we're saving additional details until the next chapter, we introduced function decorator basics earlier, in Chapter 31. Recall that the function decorator syntax is automatically translated to this equivalent by Python, to rebind the function name to the result of the decorator callable def func(args) func decorator(func) Because of this mapping, it turns out that the property built-in can serve as a decorator, to define a function that will run automatically when an attribute is...

Coding Unicode Strings

Encoding and decoding become more meaningful when you start dealing with actual non-ASCII Unicode text. To code arbitrary Unicode characters in your strings, some of which you might not even be able to type on your keyboard, Python string literals support both xNN hex byte value escapes and uNNNN and UNNNNNNNN Unicode escapes in string literals. In Unicode escapes, the first form gives four hex digits to encode a 2-byte (16-bit) character code, and the second gives eight hex digits for a 4-byte...

Coding Unicode Strings in Python

Now that I've shown you the basics of Unicode strings in 3.0, I need to explain that you can do much the same in 2.6, though the tools differ. unicode is available in Python 2.6, but it is a distinct data type from str, and it allows free mixing of normal and Unicode strings when they are compatible. In fact, you can essentially pretend 2.6's str is 3.0's bytes when it comes to decoding raw bytes into a Unicode string, as long as it's in the proper form. Here is 2.6 in action (all other...

Comparisons Equality and Truth

All Python objects also respond to comparisons tests for equality, relative magnitude, and so on. Python comparisons always inspect all parts of compound objects until a result can be determined. In fact, when nested objects are present, Python automatically traverses data structures to apply comparisons recursively from left to right, and as deeply as needed. The first difference found along the way determines the comparison result. For instance, a comparison of list objects compares all their...

Compile It Maybe

After finding a source code file that matches an import statement by traversing the module search path, Python next compiles it to byte code, if necessary. (We discussed byte code in Chapter 2.) Python checks the file timestamps and, if the byte code file is older than the source file (i.e., if you've changed the source), automatically regenerates the byte code when the program is run. If, on the other hand, it finds a .pyc byte code file that is not older than the corresponding .py source...

Comprehending List Comprehensions

With such generality, list comprehensions can quickly become, well, incomprehensible, especially when nested. Consequently, my advice is typically to use simple for loops when getting started with Python, and map or comprehensions in isolated cases where they are easy to apply. The keep it simple rule applies here, as always code conciseness is a much less important goal than code readability. However, in this case, there is currently a substantial performance advantage to the extra complexity...

Comprehending Set and Dictionary Comprehensions

In a sense, set and dictionary comprehensions are just syntactic sugar for passing generator expressions to the type names. Because both accept any iterable, a generator works well here > > > x * x for x in range(10) Comprehension 0, 1, 4, 81, 64, 9, 16, 49, 25, 36 > > > set(x * x for x in range(10)) Generator and type name 0, 1, 4, 81, 64, 9, 16, 49, 25, 36 > > > x x * x for x in range(10) 0 0, 1 1, 2 4, 3 9, 4 16, 5 25, 6 36, 7 49, 8 64, 9 81 > > > dict((x, x * x) for...

Converting Encodings

So far, we've been encoding and decoding strings to inspect their structure. More generally, we can always convert a string to a different encoding than the source character set default, but we must provide an explicit encoding name to encode to and decode from > > > S.encode() Default utf-8 encoding > > > T S.encode('cp500') Convert to EBCDIC > > > U T.decode('cp500') Convert back to Unicode > > > U.encode() Default utf-8 encoding again Keep in mind that the special...

Counting Instances with Class Methods

Interestingly, a class method can do similar work here the following has the same behavior as the static method version listed earlier, but it uses a class method that receives the instance's class in its first argument. Rather than hardcoding the class name, the class method uses the automatically passed class object generically class Spam numlnstances 0 Use class method instead of static Spam.numlnstances + 1 def printNumlnstances(cls) print(Number of instances , cls.numlnstances)...

Decorators Versus Manager Functions

Regardless of such subtleties, the Tracer class decorator example ultimately still relies on_getattr__to intercept fetches on a wrapped and embedded instance object. As we saw earlier, all we've really accomplished is moving the instance creation call inside a class, instead of passing the instance into a manager function. With the original non-decorator tracing example, we would simply code instance creation differently class Spam Non-decorator version food Wrapper(Spam()) Special creation...

Defaults and Mutable Objects

Default argument values are evaluated and saved when a def statement is run, not when the resulting function is called. Internally, Python saves one object per default argument attached to the function itself. That's usually what you want because defaults are evaluated at def time, it lets you save values from the enclosing scope, if needed. But because a default retains an object between calls, you have to be careful about changing mutable defaults. For instance, the following function uses an...

Delegation Based Managers Revisited

The object-oriented tutorial of Chapter 27 presented a Manager class that used object embedding and method delegation to customize its superclass, rather than inheritance. Here is the code again for reference, with some irrelevant testing removed def _init_(self, name, job None, pay 0) self.name name self.job job self.pay pay def lastName(self) return self.name.split() -l def giveRaise(self, percent) self.pay int(self.pay * (1 + percent)) def __str__(self) return ' Person s, s ' (self.name,...

Development Tools for Larger Projects

Once you've mastered the basics, you'll find your Python programs becoming substantially larger than the examples you've experimented with so far. For developing larger systems, a set of development tools is available in Python and the public domain. You've seen some of these in action, and I've mentioned a few others. To help you on your way, here is a summary of some of the most commonly used tools in this domain PyDoc's help function and HTML interfaces were introduced in Chapter 15. PyDoc...

Dictionary View Iterators

As we saw briefly in Chapter 8, in Python 3.0 the dictionary keys, values, and items methods return iterable view objects that generate result items one at a time, instead of producing result lists all at once in memory. View items maintain the same physical ordering as that of the dictionary and reflect changes made to the underlying dictionary. Now that we know more about iterators, here's the rest of the story > > > D dict(a 1, b 2, c 3) > > > K D.keys() A view object in 3.0, not...

Encoding and Decoding NonASCII text

Now, if we try to encode a non-ASCII string into raw bytes using as ASCII, we'll get an error. Encoding as Latin-1 works, though, and allocates one byte per character encoding as UTF-8 allocates 2 bytes per character instead. If you write this string to a file, the raw bytes shown here is what is actually stored on the file for the encoding types given UnicodeEncodeError 'ascii' codec can't encode characters in position 0-1 ordinal not in range(l28) > > > S.encode('latin-1') One byte per...

End of line is end of statement

The second and more significant syntax component you won't find in Python code is the semicolon. You don't need to terminate statements with semicolons in Python the way you do in C-like languages x 1 In Python, the general rule is that the end of a line automatically terminates the statement that appears on that line. In other words, you can leave off the semicolons, and it works the same way x 1 There are some ways to work around this rule, as you'll see in a moment. But, in general, you...

Examples

Let's type a few for loops interactively now, so you can see how they are used in practice. Basic usage As mentioned earlier, a for loop can step across any kind of sequence object. In our first example, for instance, we'll assign the name x to each of the three items in a list in turn, from left to right, and the print statement will be executed for each. Inside the print statement (the loop body), the name x refers to the current item in the list > > > for x in spam, eggs, ham print(x,...

Expression Statements

In Python, you can use an expression as a statement, too that is, on a line by itself. But because the result of the expression won't be saved, it usually makes sense to do so only if the expression does something useful as a side effect. Expressions are commonly used as statements in two situations Some functions and methods do lots of work without returning a value. Such functions are sometimes called procedures in other languages. Because they don't return values that you might be interested...

Extended Comprehension Syntax for Sets and Dictionaries

Like list comprehensions and generator expressions, both set and dictionary comprehensions support nested associated if clauses to filter items out of the result the following collect squares of even items (i.e., items having no remainder for division by 2) in a range > > > x * x for x in range(10) if x 2 0 Lists are ordered > > > x * x for x in range(10) if x 2 0 But sets are not > > > x x * x for x in range(10) if x 2 0 Neither are dict keys Nested for loops work as well,...

F

Justification, 769 factoring of code, 649 factory design pattern, 768 factory functions, 420 false and true values, 246 fieldname, 185 file execution, 25 file icon clicks, 47-51 limitations, 50 file input output, Python 3.0, 900 file iterators, 352 file object methods and printing operations, 297 file object type, 97 files, 225, 229-239 advanced file methods, 238 common operations, 230 examples of usage, 232-238 file context managers, 238 packed binary data, storing and parsing in files, 237...

File context managers

You'll also want to watch for Chapter 33's discussion of the file's context manager support, new in Python 3.0 and 2.6. Though more a feature of exception processing than files themselves, it allows us to wrap file-processing code in a logic layer that ensures that the file will be closed automatically on exit, instead of relying on the autoclose on garbage collection with open(r'C misc data.txt') as myfile See Chapter 33 for details for line in myfile use line here The try finally statement...

For More Help

Python's standard manual set today includes valuable pointers for usage on various platforms. The standard manual set is available in your Start button on Windows after Python is installed (option Python Manuals), and online at http www.python.org. Look for the manual set's top-level section titled Using Python for more platform-specific pointers and hints, as well as up-to-date cross-platform environment and command-line details. As always, the Web is your friend, too, especially in a field...

Full Credit

The following file shows three ways to code this operation, at least one of which was suggested by a student in one of my courses The first function fetches the first argument (args is a tuple) and traverses the rest by slicing off the first (there's no point in comparing an object to itself, especially if it might be a large structure). The second version lets Python pick off the first and rest of the arguments automatically, and so avoids an index and slice. The third converts from a tuple to...

Function Annotations in

In Python 3.0 (but not 2.6), it's also possible to attach annotation information arbitrary user-defined data about a function's arguments and result to a function object. Python provides special syntax for specifying annotations, but it doesn't do anything with them itself annotations are completely optional, and when present are simply attached to the function object's_annotations_attribute for use by other We met Python 3.0's keyword-only arguments in the prior chapter annotations generalize...

Function Decorators

Function decorators have been available in Python since version 2.5. As we saw earlier in this book, they are largely just syntactic sugar that runs one function through another at the end of a def statement, and rebinds the original function name to the result. A function decorator is a kind of runtime declaration about the function whose definition follows. The decorator is coded on a line just before the def statement that defines a function or method, and it consists of the symbol followed...

Function Interfaces and Callback Based Code

As an example, the tkinter GUI toolkit (named Tkinter in Python 2.6) allows you to register functions as event handlers (a.k.a. callbacks) when events occur, tkinter calls the registered objects. If you want an event handler to retain state between events, you can register either a class's bound method or an instance that conforms to the expected interface with_call__. In this section's code, both x.comp from the second example and x from the first can pass as function-like objects this way....

Functional Programming Tools filter and reduce

The map function is the simplest representative of a class of Python built-ins used for functional programming tools that apply functions to sequences and other iterables. Its relatives filter out items based on a test function (filter) and apply functions to pairs of items and running results (reduce). Because they return iterables, range and filter both require list calls to display all their results in 3.0. For example, the following filter call picks out items in a sequence that are greater...

Gaming Images Serial Ports XML Robots and More

Python is commonly applied in more domains than can be mentioned here. For example, you can do Game programming and multimedia in Python with the pygame system Serial port communication on Windows, Linux, and more with the PySerial extension Image processing with PIL, PyOpenGL, Blender, Maya, and others Robot control programming with the PyRo toolkit XML parsing with the xml library package, the xmlrpclib module, and third-party extensions Artificial intelligence programming with neural network...

Generalized Set Functions

Because these are tools worth reusing (and they're too big to retype interactively), we'll store the functions in a module file called inter2.py (if you've forgotten how modules and imports work, see the introduction in Chapter 3, or stay tuned for in-depth coverage in Part V). In both functions, the arguments passed in at the call come in as the args tuple. As in the original intersect, both work on any kind of sequence. Here, they are processing strings, mixed types, and more than two...

Generalizing for Keywords and Defaults

The prior version illustrates the basics we need to employ, but it's fairly limited it supports validating arguments passed by position only, and it does not validate keyword arguments (in fact, it assumes that no keywords are passed in a way that makes argument position numbers incorrect). Additionally, it does nothing about arguments with defaults that may be omitted in a given call. That's fine if all your arguments are passed by position and never defaulted, but less than ideal in a general...

Generalizing for Public Declarations

Now that we have a Private implementation, it's straightforward to generalize the code to allow for Public declarations too they are essentially the inverse of Private declarations, so we need only negate the inner test. The example listed in this section allows a class to use decorators to define a set of either Private or Public instance attributes (attributes stored on an instance or inherited from its classes), with the following semantics Private declares attributes of a class's instances...

Handling Errors by Testing Inputs

So far so good, but notice what happens when the input is invalid Enter text xxx error text omitted ValueError invalid literal for int() with base 10 'xxx' The built-in int function raises an exception here in the face of a mistake. If we want our script to be robust, we can check the string's content ahead of time with the string object's isdigit method > > > S '123' > > > T 'xxx' > > > S.isdigit(), T.isdigit() This also gives us an excuse to further nest the statements in...

Handling Errors with try Statements

The preceding solution works, but as you'll see later in the book, the most general way to handle errors in Python is to catch and recover from them completely using the Python try statement. We'll explore this statement in depth in Part VII of this book, but as a preview, using a try here can lead to code that some would claim is simpler than the prior version reply input('Enter text ') if reply 'stop' break try This version works exactly like the previous one, but we've replaced the explicit...

How to Contact Us

Please address comments and questions concerning this book to the publisher 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We will also maintain a web page for this book, where we list errata, examples, and any additional information. You can access this page at To comment or ask technical questions about this book, send email to For more information about our books, conferences, Resource Centers, and the O'Reilly Network, see our website...

How to Set Configuration Options

The way to set Python-related environment variables, and what to set them to, depends on the type of computer you're working on. And again, remember that you won't necessarily have to set these at all right away especially if you're working in IDLE (described in Chapter 3), configuration is not required up front. But suppose, for illustration, that you have generally useful module files in directories called utilities and packagel somewhere on your machine, and you want to be able to import...

IDLE Basics

IDLE is easy to start under Windows it has an entry in the Start button menu for Python (see Figure 2-1, shown previously), and it can also be selected by right-clicking on a Python program icon. On some Unix-like systems, II IDLE is officially a corruption of IDE, but it's really named in honor of Monty Python member Eric Idle. you may need to launch IDLE's top-level script from a command line, or by clicking on the icon for the idle.pyw or idle.py file...

Implementation Details I

This code is a bit complex, and you're probably best off tracing through it on your own to see how it works. To help you study, though, here are a few highlights worth mentioning. The first-cut privacy example shown in Chapter 29 used inheritance to mix in a _setattr_to catch accesses. Inheritance makes this difficult, however, because differentiating between accesses from inside or outside the class is not straightforward (inside access should be allowed to run normally, and outside access...

Implementing Private Attributes

The following class decorator implements a Private declaration for class instance at-tributes that is, attributes stored on an instance, or inherited from one of its classes. It disallows fetch and change access to such attributes from outside the decorated class, but still allows the class itself to access those names freely within its methods. It's not exactly C++ or Java, but it provides similar access control as an option in Python. We saw an incomplete first-cut implementation of instance...

Import and reload Usage Notes

For some reason, once people find out about running files using import and reload, many tend to focus on this alone and forget about other launch options that always run the current version of the code (e.g., icon clicks, IDLE menu options, and system command lines). This approach can quickly lead to confusion, though you need to remember when you've imported to know if you can reload, you need to remember to use parentheses when you call reload (only), and you need to remember to use reload in...

Imports and Attributes

Figure 21-1 sketches the structure of a Python program composed of three files a.py, b.py, and c.py. The file a.py is chosen to be the top-level file it will be a simple text file of statements, which is executed from top to bottom when launched. The files b.py and c.py are modules they are simple text files of statements as well, but they are not usually launched directly. Instead, as explained previously, modules are normally imported by other files that...

Indexing and Slicinggetitemandsetitem

If defined in a class (or inherited by it), the_getitem_method is called automatically for instance-indexing operations. When an instance X appears in an indexing expression like X i , Python calls the_getitem__method inherited by the instance, passing X to the first argument and the index in brackets to the second argument. For example, the following class returns the square of an index value > > > class Indexer > > > X 2 X i calls X._getitem_(i) print(X i , end ' ')...

Indexing Slicing and Matrixes

Becauselists are sequences, indexing and slicing work the same way for lists as they do for strings. However, the result of indexing a list is whatever type of object lives at the offset you specify, while slicing a list always returns a new list > > > L 'spam', 'Spam', 'SPAM ' > > > L 2 Offsets start at zero > > > L -2 Negative count from the right > > > L 1 Slicing fetches sections One note here because you can nest lists and other object types within lists, you will...

Internet Scripting

Python comes with standard Internet modules that allow Python programs to perform a wide variety of networking tasks, in client and server modes. Scripts can communicate over sockets extract form information sent to server-side CGI scripts transfer files by FTP parse, generate, and analyze XML files send, receive, compose, and parse email fetch web pages by URLs parse the HTML and XML of fetched web pages communicate over XML-RPC, SOAP, and Telnet and more. Python's libraries make these tasks...

Introducing the Python Interpreter

So far, I've mostly been talking about Python as a programming language. But, as currently implemented, it's also a software package called an interpreter. An interpreter is a kind of program that executes other programs. When you write a Python program, the Python interpreter reads your program and carries out the instructions it contains. In effect, the interpreter is a layer of software logic between your code and the computer hardware on your machine. When the Python package is installed on...

Is Python a Scripting Language

Python is a general-purpose programming language that is often applied in scripting roles. It is commonly defined as an object-oriented scripting language a definition that blends support for OOP with an overall orientation toward scripting roles. In fact, people often use the word script instead of program to describe a Python code file. In this book, the terms script and program are used interchangeably, with a slight * For a more complete look at the Python philosophy, type the command...

Lambda Basics

The lambda's general form is the keyword lambda, followed by one or more arguments (exactly like the arguments list you enclose in parentheses in a def header), followed by an expression after a colon lambda argumentl, argument2, argumentN expression using arguments Function objects returned by running lambda expressions work exactly the same as those created and assigned by defs, but there are a few differences that make lambdas useful in specialized roles lambda is an expression, not a...

List Iteration and Comprehensions

More generally, lists respond to all the sequence operations we used on strings in the prior chapter, including iteration tools > > > 3 in 1, 2, 3 Membership print(x, end ' ') Iteration We will talk more formally about for iteration and the range built-ins in Chapter 13, because they are related to statement syntax. In short, for loops step through items in any sequence from left to right, executing one or more statements for each item. The last items in Table 8-1, list comprehensions and...

List method calls

Like strings, Python list objects also support type-specific method calls, many of which change the subject list in-place > > > L.append('please') Append method call add item at end > > > L.sort() Sort list items ('S' < 'e') Methods were introduced in Chapter 7. In brief, they are functions (really, attributes that reference functions) that are associated with particular objects. Methods provide type-specific tools the list methods presented here, for instance, are generally...

Literals and Basic Properties

Python 3.0 string objects originate when you call a built-in function such as str or bytes, process a file created by calling open (described in the next section), or code literal syntax in your script. For the latter, a new literal form, b'xxx' (and equivalently, B'xxx') is used to create bytes objects in 3.0, and bytearray objects may be created by calling the bytearray function, with a variety of possible arguments. More formally, in 3.0 all the current string literal forms 'xxx', xxx, and...

Local Names Are Detected Statically

As you know, Python classifies names assigned in a function as locals by default they live in the function's scope and exist only while the function is running. What you may not realize is that Python detects locals statically, when it compiles the def's code, rather than by noticing assignments as they happen at runtime. This leads to one of the most common oddities posted on the Python newsgroup by beginners. Normally, a name that isn't assigned in a function is looked up in the enclosing...

Local Variables

Probably the most interesting part of this example is its names. It turns out that the variable res inside intersect is what in Python is called a local variable a name that is visible only to code inside the function def and that exists only while the function runs. In fact, because all names assigned in any way inside a function are classified as local variables by default, nearly all the names in intersect are local variables res is obviously assigned, so it is a local variable. Arguments...

Loop else

When combined with the loop else clause, the break statement can often eliminate the need for the search status flags used in other languages. For instance, the following piece of code determines whether a positive integer y is prime by searching for factors greater than 1 Rather than setting a flag to be tested when the loop is exited, it inserts a break where a factor is found. This way, the loop else clause can assume that it will be executed only if no factor is found if you don't hit the...

Loop Statements Versus Recursion

Though recursion works for summing in the prior sections' examples, it's probably overkill in this context. In fact, recursion is not used nearly as often in Python as in more esoteric languages like Prolog or Lisp, because Python emphasizes simpler procedural statements like loops, which are usually more natural. The while, for example, often makes things a bit more concrete, and it doesn't require that a function be defined to allow recursive calls > > > L 1, 2, 3, 4, 5 > > > sum...

Management Techniques Compared

To summarize the coding differences in all four attribute management schemes we've seen in this chapter, let's quickly step through a more comprehensive computed-attribute example using each technique. The following version uses properties to intercept and calculate attributes named square and cube. Notice how their base values are stored in names that begin with an underscore, so they don't clash with the names of the properties themselves 2 dynamically computed attributes with properties...

Managing Functions and Classes Directly

Most of our examples in this chapter have been designed to intercept function and instance creation calls. Although this is typical for decorators, they are not limited to this role. Because decorators work by running new functions and classes through decorator code, they can also be used to manage function and class objects themselves, not just later calls made to them. Imagine, for example, that you require methods or classes used by an application to be registered to an API for later...

Mapping Functions over Sequences map

One of the more common things programs do with lists and other sequences is apply an operation to each item and collect the results. For instance, updating all the counters in a list can be done easily with a for loop > > > counters 1, 2, 3, 4 > > > updated > > > for x in counters updated.append(x + 10) Add 10 to each item > > > updated 11, 12, 13, 14 But because this is such a common operation, Python actually provides a built-in that does most of the work for you. The...

Metaclasses Versus Class Decorators Round

Just in case this chapter has not yet managed to make your head explode, keep in mind again that the prior chapter's class decorators often overlap with this chapter's metaclasses in terms of functionality. This derives from the fact that Class decorators rebind class names to the result of a function at the end of a class statement. Metaclasses work by routing class object creation through an object at the end of a class statement. Although these are slightly different models, in practice they...

Methods Are Objects Bound or Unbound

Methods in general, and bound methods in particular, simplify the implementation of many design goals in Python. We met bound methods briefly while studying_call_in Chapter 29. The full story, which we'll flesh out here, turns out to be more general and flexible than you might expect. In Chapter 19, we learned how functions can be processed as normal objects. Methods are a kind of object too, and can be used generically in much the same way as other objects they can be assigned, passed to...

Minimize Cross File Changes

Here's another scope-related issue although we can change variables in another file directly, we usually shouldn't. Module files were introduced in Chapter 3 and are covered in more depth in the next part of this book. To illustrate their relationship to scopes, consider these two module files X 99 This code doesn't know about second.py print(first.X) Okay references a name in another file first.X 88 But changing it can be too subtle and implicit t Multithreading runs function calls in parallel...

Module Design Concepts

Like functions, modules present design tradeoffs you have to think about which functions go in which modules, module communication mechanisms, and so on. All of this will become clearer when you start writing bigger Python systems, but here are a few general ideas to keep in mind You're always in a module in Python. There's no way to write code that doesn't live in some module. In fact, code typed at the interactive prompt really goes in a built-in module called _main_ the only unique things...

Module Imports and Reloads

So far, I've been talking about importing modules without really explaining what this term means. We'll study modules and larger program architecture in depth in Part V, but because imports are also a way to launch programs, this section will introduce enough module basics to get you started. In simple terms, every file of Python source code whose name ends in a .py extension is a module. Other files can access the items a module defines by importing that module import operations essentially...