Simple PyFit Example

The previous section has hopefully given you a good feeling for what FIT does and how it can benefit you. Unfortunately, setting up FIT is more complicated than it needs to be. The documentation is patchy at best, and if you're not using FitNesse, it fails to address many implementation questions, particularly to do with running PyFit from a build. Fortunately, this lack of information conceals a simple process. It should be easy to set up a build on a new machine. Each additional package that...

Simple RSS Reader

The project introduced in Chapter 4 is a simple command-line RSS reader (a.k.a. aggregator). As noted, RSS is a way of distributing content that is frequently updated. Examples include new articles, blog postings, podcasts, build results, and comic strips. A single source is referred to as a feed. An aggregator is a program that pulls down one or more RSS feeds and interleaves them. The one constructed here will be very simple. The two feeds we'll be using are from two of my favorite comic...

About the Author

JEFF YOUNKER is chief engineer of Data-Pipes (www.data-pipes.com ). His educational background carefully avoided computers, but he was drawn in anyway. Most of his misguided adulthood has been spent in large installation systems administration, tool smithing, and release engineering, with a peculiar obsession involving both monitoring and rapid deployment. Over the last several years, he's had the pleasure of working with Python full time. Having escaped Texas nearly a decade ago, he now lives...

Adding a File

You've already learned how to create a new Python module in Chapter 2. Create one now called examples. common. It doesn't matter what's in the module at the moment. Your Pydev Package Explorer should look something like Figure 3-14. r Z Pydev Package Explorer 0 Icrj > agile file usr local svn repostTrunk trunk) Figure 3-14. examples.common.py created and added, but not committed The question mark glyph indicates that Subversion doesn't know about this file yet. You should take a look at the...

Aggregating Two Feeds

In this example, two separate feeds need to be combined, and the items in the output must be sorted by date. As with the previous example, the name of the feed should be included with its title, so the individual feed items need to identify where they come from. A session might look like this rsreader http www.xkcd.com rss.xml http www.pvponline.com rss.xml rsreader http www.xkcd.com rss.xml http www.pvponline.com rss.xml The feeds must be retrieved separately. This is a design constraint from...

Code Coverage

Code coverage is a family of measurements. There are many different kinds of code coverage. Cem Kaner covers 101 of them in his paper Software Negligence and Testing Coverage (www.kaner.com coverage.htm). With 100 percent statement coverage, all statements in a program have been executed. This is not to say that all expressions have been executed. It is also not the same as saying that all branches have been executed. An if-then-else statement has been executed even if only the else block has...

Coding Conventions

There are three primary aspects to coding conventions. The typographical standards dictate the code's appearance. How many spaces is each block indented Do spaces bracket the equal sign in an assignment How about in a keyword assignment Naming conventions determine how names for variables, classes, and methods are chosen. They provide a common grammar and map the system metaphor into the programming language. Structural conventions determine how a project is laid out. They determine where data...

Command[python setuppy test descriptionRunning unit tests descriptionDone Unit tests run

The change is saved, and the Buildbot master is reconfigured buildbot reconfig usr local buildbot master rsreader Reconfiguration appears to have completed successfully. Committing the recent changes causes a build, or, if the changes have already been committed, then a build can be triggered from the command line. The resulting successful build is shown in Figure 6-15. j_i- _,.- - tittp localliost 8010 waterfall RSReader last build current activity Figure 6-15. The test step succeeds when all...

Copying Files

Normally in Eclipse, you use the copy and paste operations to copy files. This is a no-no when using Subversion and many other source control systems. Subversion needs to maintain the history of a copied file. Because copy and paste are separate operations, that information is lost. Subversive gets around this by providing a special copy operation in the Team menu. Copying files from one directory to another is much like moving files. The files to be copied are selected from an explorer, and...

Editing a File

Just open an editor and go to town. The complications come when it is time to submit. Subversive and the Synchronize view make it easy to anticipate conflicts, though. You can see this by making changes to the previously added common. py file. Adding a doc string like A sample edit will suffice. Notice that when you type, there are no changes in the synchronization window. This is because you haven't saved the changes to the filesystem yet. That's indicated by...

Enslaving Buildbot

In grand strokes, creating a basic Buildbot slave is similar to creating a master, but much simpler in the initial details. If running on a separate system, as in this example, then Buildbot must be installed first. Then the build user and Buildbot directories are created, a slave instance is created, the configuration files are updated, and Buildbot is started. In this test environment, the slave runs on slave-lnx01. sudo mkdir usr local buildbot slave rsreader sudo chown build build usr local...

Factory pythonBuilder

This time when you run the build, the second builder shows up in a second column, as in Figure 5-11. RSRcadcr last build cunrcnt activity . J. 7python24 lib python24 sitc- packages' Figure 5-11. Simultaneous Python 2.4 and 2.5 builds The second builder executes in parallel with the first. There are differences that are immediately apparent. The build step takes much longer under 2.4 than under 2.5. If you look at the log, the reason for this should quickly become clear ez_setup.py is not using...

FaddStep ShellCommand command[rm rf sitepkgs faddStep ShellCommand command[mkdir sitepkgs faddStep ShellCommand

F1.addStep(ShellCommand, command mkdir, site_bin ) f1.addStep(Compile, command python, . setup.py, build ) f1.addStep(Compile, command python, . setup.py, install, --install-scripts, site_bin ) Once again, we're saving the changes, reconfiguring Buildbot, and triggering a build. The waterfall display is shown in Figure 5-10. It clearly shows the new step and the last step's output it is clear that the packages have been freshly installed.

Getting Started with Setuptools

Setuptools is driven by the program setup.py. This file is created by hand. There's nothing special about the file name it is chosen by convention, but it's a very strong convention. If you've used Distutils, then you're already familiar with the process. Setuptools just adds a variety of new keywords. The minimal setup.py for this project looks like this from setuptools import setup, find_packages setup( basic package data name RSReader, version 0.1, package structure packages...

Getting Subverted

The first step is installing Subversion. Subversion is available from http subversion. tigris. org . If you're running on Linux and you installed your system with development tools included, then the odds are good that you've already got Subversion installed. If Subversion is not installed, chances are that packaged binaries can be located for your system at and if worse comes to worst, the source code is also available there. Once Subversion is installed, the first step in creating your...

Getting the Revision

The Buildbot 0.7.7 documentation suggests that this information is in the revision build property. BuildStep.getProperty() and BuildStep.setProperty() form the core of the build properties system, but only custom tasks can use them. ShellCommand classes have another access mechanism command strings are wrapped in the WithProperties class, and this class expands them at runtime.2 The documentation suggests that the revision property is set by the SVN build step. Alas, that is not true. You must...

HaltOnFailure True versionversion

Now the build step has access to the revision and version, so you can finally parameterize the URL url return 'revision' self.getProperty('revision'), 'version' self.version self.setUrl(reports, self.url self.expansions()) self.step_status.setText(Reports published) self.finished(SUCCESS) The step now expands the URL. You'll want the step to go yellow while it runs. reports, Unzipping package ) self.setUrl(reports, self.url self.expansions()) self.step_status.setText(Reports published)...

Hooking Up Source Control

The build master and build slave are now on separate hosts. They both need access to the Subversion repository. Until now, Subversion has been accessed directly through the filesystem, but this will no longer work. However, this isn't a simple choice. Subversion can be accessed remotely via a bewildering spectrum of methods. Repositories can be accessed through the Subversion network server, through WebDAV and the Apache web server, or by tunneling over a shell transport such as SSH....

How It Works

If a function begins with the string test, then it is treated as a test. JsUnit is a direct translation of the XUnit framework, as is Python's unittest. There is a one-to-one correspondence between most of the major concepts. This is shown in Table 10-3. Table 10-3. The Correspondence Between unittest and JsUnit Extend unittest.TestCase Include app jsUnitCore.js Import subject code Include subject code setUp() and tearDown() methods setUp() and tearDown()...

Installing Plug Ins

Eclipse plug-ins are published on little web sites known as update sites, and are very easy to install. You give Eclipse the URL for an update site, and it sucks down the plug-in and installs it. Be careful though don't confuse the web site for a plug-in with its update site. The web site for Mylyn is www.eclipse.org mylyn, while the update site is located at http download. Start the installation process by selecting Help > Software Updates > Find and Install. This brings up the Install...

Looking Under the Hood

Many external tools interact with the code stored in Eclipse. In order to work with them, it is necessary to understand how Eclipse lays out its directories. To begin, you'll need to change directories to the root of your Eclipse workspace on my machine, it is Documents workspace. From there, we'll go on a brief tour. (My machine's name is phytoplankton and my username is jeff.) phytoplankton jeff cd Documents workspace phytoplankton Documents workspace jeff ls -aF . The directory .metadata...

Manyto Many Relationships

As with one-to-many relationships, creating the simplest many-to-many relationships is a little more involved than with SQLObject, as the intermediate tables must be explicitly described. The schema described here is pictured in Figure 9-7. course_table Table('course', schema, Column('id', Integer, primary_key True), Column('name', String(64), nullable False), enrolled_assc_table Table('enrolled_assc', schema, Column('student_id', Integer, ForeignKey('student.id')), Column('course_id', Integer,...

Mastering Buildbot

The build master is configured before the slave, as the slave's status is determined through its interactions with the build master. Creating the build user and the directories are the first steps. Tip If you're trying to install Buildbot on Windows systems, it is started with buildbot.bat. This script is installed into Python25 scripts. Unfortunately, it has a hard-coded reference to a nonexistent script in Python23. This reference will need to be changed by hand. sudo mkdir -p usr local...

Oneto Many Relationships

SQLObject specifies joins (specifically inner joins) declaratively. The products of the joins appear as arrays contained in instance variables. To demonstrate, I've expanded the schema to include an e-mail address for each student. Each student may have more than one e-mail address (see Figure 9-2). Figure 9-2. A many-to-one relationship between student and e-mail address Figure 9-2. A many-to-one relationship between student and e-mail address The corresponding SQL for the new table is as...

Overriding Existing Methods Monkeypatching

The code turning a feed object into text has been written. The next step converts URLs into feed objects. This is the magic that FeedParser provides. The test harness doesn't have control over network connections, and the Net at large can't be controlled without some pretty involved network hackery. More important, the tests shouldn't be dependent on external resources unless they're included as part of the build. All of these concerns can be surmounted by hacking FeedParser on the fly. Its...

Paying for More Functionality

At this point, I'd suggest trying out Pydev Extensions. If you end up using Eclipse for Python development (as I hope you do), then it is a worthwhile investment. It gives you a number of features missing from the free version of Pydev, most of them relating to much-improved code analysis. Notably, it includes a much better go to definition of function feature for navigating your code. Pydev uses an external program (Bicycle Repair Man) that has trouble with class methods, but Pydev Extensions...

Refactoring Reimplementing fromurls

Is there any functionality still unimplemented in AggregateFeed For the moment, it doesn't appear so. However, I'm not comfortable with the code as it stands it seems overly complicated. The discomfort comes from the interactions between from_urls(), feeds_from_urls(), and combine_feeds(). The data flow exhibits a Y shape, as shown in Figure 7-6. Figure 7-6. Questionably complicated data flow Figure 7-6. Questionably complicated data flow A collection of URLs is passed down one leg, it is...

Resolving Conflicts

Suppose that someone else has made a change while you were editing standard. py. Once again, you've both changed the comment line for standard.HelloWorld.main(). This time, the other user has committed their change before you have. When you attempt to submit, you see the window shown in Figure 3-20. L error occurred during the operation execution. Tor more information see the text listed below. Some of selected resources were not committed. Figure 3-20. The commit operation failed. Clicking the...

Running a Test

You can run tests stand-alone or distributed. Stand-alone tests are suitable for developing the tests themselves or interactively testing small pieces of code, as they require the user to interact with a web browser. Distributed tests are run from within the build. They use a farm of web browsers that may reside on other machines. To start with, I'll demonstrate stand-alone testing. Once you've gained an understanding of how to use JsUnit, I'll move on to using distributed tests, in order to...

Running Acceptance Tests

Acceptance tests occupy a different place in the build infrastructure than unit tests. The build fails if unit tests fails, but the product fails if acceptance tests fail. The build must always work, but the product doesn't have to work until delivery, so acceptance tests are not expected to pass with every build. However, acceptance tests do yield useful information when run. Their successes and failures suggest how close the product is to completion. This information is interesting to...

Running Tests After Every Change

Eclipse contains a mechanism intended to produce incremental builds. Eclipse activates builders when projects change. Builders take a list of changes since their last invocation, and then perform an update task. This may be an extension to Eclipse or a program run external to the IDE. The builder mechanism is a hook to run the unit tests after every change. Additional builders are defined from the project properties menu. The project properties are accessed through the application menu or the...

Short Iterations

Short iterations serve multiple purposes. They allow you to deliver a working product to your customer at regular intervals. They allow you to see how accurate your estimates are on a regular basis. Finally, they give you an opportunity to regularly reexamine your development processes. You get to plan for the future and look back at the past while everything is still fresh in your mind. Note that shorter iterations are not necessarily better. Time spans in the range of two weeks to one month...

Some Notes About Building Multiple Versions

One of the primary goals of continuous integration is a replicable build. When you build a given version of the software, you should produce the same end product every time the build is performed. And multiple builds will inevitably be performed. Developers will build the product on their local boxes. The continuous integration system will produce test builds on a build farm. A final production packaging system may produce a further build. Each build version is tagged with a unique tag denoting...

Subversion to Buildbot Over

Subversion should send notifications to Buildbot when changes are committed. This is done with hooks. Hooks are programs triggered by events in Subversion. The events are commits, locking, and revision property changes, and there are hooks for several steps in each kind of event. The commit event is the interesting one for our purpose. The commit process has three hooks. The start-commit hook is called before the commit transaction is created. The pre-commit hook is called after the commit...

Supporting Python Builds

This chapter's ultimate goal is supporting both Python 2.4 and Python 2.5, and we're getting very close. Python 2.4 must be installed, a new scheduler added, and a new builder defined. The master configuration must also be refactored along the way. Installing Python 2.4 is precisely analogous to installing Python 2.5. It is placed in a directory named python2.4, directly beneath the slave's root directory. Supplying the new directory to the . configure step is the only change in procedure. curl...

Test Driven Development

As noted previously, a unit-testing effort will fail unless the tests provide more perceived reliability than the combined costs of maintenance and creation. There are two clear ways to ensure this. Perceived utility can be increased, or the costs of maintenance and creation can be decreased. The practices of TDD address both. TDD is a style with unique characteristics. Perhaps most glaringly, tests are written before the tested code. The first time you encounter this, it takes a while to wrap...

The IDE Eclipsing the Command Line

There are two main ways you can work with Python through the command line or through an IDE. Both have their distinct advantages and disadvantages. I'm not going to give short shrift to either, but this chapter is mostly about using the Eclipse IDE to work with Python. To truly take advantage of agile development methodologies, chances are you're going to want to use an IDE. IDEs offer a range of features that are at best poorly implemented in command-line tools. A short list includes a wealth...

The Problems with Not Unit Testing

I make the bald-faced assertion that no programmer completely understands any system of nontrivial complexity. If that programmer existed, then he would produce completely bug-free code. I've yet to see that in practice, but absence of evidence is not evidence of absence, so that person might exist. Instead, I think that programmers understand most of the salient features of their own code, and this is good enough in the real world. What about working with another programmer's code While you...

Unit Testing

The focus in this chapter is on programmer tests. From this point forward, I shall use the terms unit test and programmer test interchangeably. If I need to refer to customer tests, I'll name them explicitly. So why unit testing Simply put, unit testing makes your life easier. You'll spend less time debugging and documenting, and it results in better designs. These are broad claims, so I'll spend some time backing them up. Developers resort to debugging when a bug's location can't be easily...

URI and URL

The URI format identifies documents unambiguously. Once obscure, it can now be seen even on billboards for toilet paper. A URI has four parts, organized as follows scheme hierarchical part query fragment The scheme identifies the kind of resource, and it determines how the other three parts are interpreted. Common schemes include the following https, for encrypted web pages file, for files on the local system mailto, for e-mail addresses The hierarchical part is separated from the scheme by a...

Use nose to run tests testsuitenosecollector

Notice that setup.py already requires Nose, so it is always guaranteed to be there. The dependency could have been specified through the tests_require property. This property specifies packages that will only be installed for testing. Had Nose been installed through those routes, it would not be generally available for development work. Nose isn't required for production, but it doesn't hurt to bundle it along, and it makes it much easier to set up a development environment. writing...

Wed Dec xkcdcom Python Mon Dec xkcdcom Far Away

Items 'date' Wed, 05 Dec 2007 05 00 00 -0000, 'title' Python , 'date' Mon, 03 Dec 2007 05 00 00 -0000, 'title' Far Away feed 'feed' 'title' xkcd.com , 'entries' items computed_items RSReader().feed_listing(feed) assert_equals(printed_items, computed_items) You save the change and run the test, and it should succeed. The line defining printed_items is identical in both acceptance_tests.py and application_tests.py, so the definition can and should be moved to a common location. That module will...

Writing Requirements

With some FIT implementations, the requirement documents can be in many different formats. The test tables can be extracted from Microsoft Word documents, Excel spreadsheets, and HTML documents. Any format capable of representing tables can theoretically be used as a source document as long as a converter is supplied. My favorite document source is a spreadsheet. Spreadsheets are eminently capable of creating, manipulating, and formatting tables, and everyone knows how to use one. In...

The Team Repository View

The team repository view supplants most of the Subversion status operations we looked at earlier in the chapter. It shows how the repository is different from your working copy, and how your working copy is different from the repository. It can combine those views showing all changes, or it can show only files with conflicts. It can show the aggregate changes, or it can break the differences down by revision. Open the repository view by choosing the menu item Window gt Open View gt Other. This...

Importing from Subversion

Once you've installed Subversive and restarted Eclipse, you can import from the repository. Select File gt Import, which will bring up the Import project window, shown in Figure 3-3. Figure 3-3. Importing an existing project Figure 3-3. Importing an existing project Select SVN gt Projects from SVN, and then click the Next button. This will take you to the repository selection screen, shown in Figure 3-4. Checkout Project from SVN repository Select an existing repository location or create a new...