The importance of Code Templates

It's been a while now since I posted anything... It's been a busy couple of months, and I just couldn't spare the time, unfortunately.

But I haven't been completely idle, either. I actually wrote some Python code that was intended for production use at work (which I'm not sure I can share, at least not directly), which was cool and enlightening. I also had several insights into things that have made changes to my preferred coding style, most of which I'm going to share with you here.

First, as I was thrashing through 20+ classes, nominal interfaces and nominal abstract classes, all of which were going to need to be (potentially) maintained by others who weren't as familiar with Python as I am, I determined that I should make some attempt to make the structure as consistent as possible for everyone who might have to look at the code later on. The first step in this effort, for me, at least, was to build out a template-file for Python modules, including both main-code structure and some built-in unit-testing.

This template-file can be downloaded from my public Dropbox folder at dl.dropbox.com/u/1917253/site-packages/Template.py

As of today, here's what that template looks like:

# Python module template
"""Python module template. Provides no functionality."""

__author__  = 'Brian D. Allbee'
__licence__ = 'http://creativecommons.org/licenses/by-sa/3.0/'
__version__ = '0.2'
__copyright__ = "Copyright 2011, Brian D. Allbee"
__credits__ = ["Brian D. Allbee"]
__maintainer__ = "Brian D. Allbee"
__email__ = "brian.allbee@gmail.com"
__status__ = "Development"

#####################################
# Build an "__all__" list to        #
# support                           #
# "from Namespace.blah import *"    #
# syntax                            #
#####################################
__all__ = []

#####################################
# Required imports                  #
#####################################

from DocMeta import *

#####################################
# Functions defined in the module   #
#####################################

#####################################
# Interfaces defined in the module  #
#####################################

#####################################
# Abstract Classes defined in the   #
# module                            #
#####################################

#####################################
# Classes defined in the module     #
#####################################

class ClassName( object ):
    """Class doc-string."""

    ##################################
    # Class Attributes               #
    ##################################

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    ##################################
    # Class Properties               #
    ##################################

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def __init__( self ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
#        if self.__class__ != ClassName:
#            raise NotImplementedError( 'ClassName is (nominally) a final class, and is not intended to be derived from.' )
        # Nominally abstract: Don't allow instantiation of the class
#        if self.__class__ == ClassName:
#            raise NotImplementedError( 'ClassName is (nominally) an interface, and is not intended to be instantiated.' )
#            raise NotImplementedError( 'ClassName is (nominally) an abstract class, and is not intended to be instantiated.' )
        pass

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

__all__ += [ 'ClassName' ]

#####################################
# Package sub-modules and -packages #
#####################################

#####################################
# Unit-test the module on main      #
#####################################

if __name__ == '__main__':
    import os, sys, unittest
    from UnitTestUtilities import *

    testSuite = unittest.TestSuite()
    testResults = unittest.TestResult()

    #################################
    # Unit-test Constants           #
    #################################

    class testConstants( unittest.TestCase ):
        """Unit-tests the constants defined in the module."""
        pass
    
    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testConstants ) )

    #################################
    # Unit-test Functions           #
    #################################

    #################################
    # Unit-test Interfaces          #
    #################################

    #################################
    # Unit-test Abstract Classes    #
    #################################

    #################################
    # Unit-test Classes             #
    #################################

    class ClassNameDerived( ClassName ):
        def __init__( self ):
            pass
    
    class testClassName( unittest.TestCase ):
        """Unit-tests the ClassName class."""
   
        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass
        
#        def testDerived( self ):
#            """Testing abstract nature of the ClassName class."""
#            pass
    
#        def testFinal( self ):
#            """Testing final nature of the ClassName class."""
#            pass
    
        def testConstruction( self ):
            """Testing construction of the ClassName class."""
            pass
    
        def testPropertyCountAndTests( self ):
            """Testing the properties of the ClassName class."""
            items = getMemberNames( ClassName )[0]
            actual = len( items )
            expected = 0
            self.assertEquals( expected, actual, 'ClassName is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the ClassName class."""
            items = getMemberNames( ClassName )[1]
            actual = len( items )
            expected = 0
            self.assertEquals( expected, actual, 'ClassName is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testClassName ) )

    # Run unit-tests and install
    print '#'*79
    print 'Running unit-tests'
    testSuite.run( testResults )
    print '#'*79
    if testResults.errors == [] and testResults.failures == []:
        print 'All unit-tests passed! Ready for build/copy!'
        # Uncomment the following line to allow installation
        # import shutil
        sitePackagesDirs = []
        for path in sys.path:
            if path.split( os.sep )[-1].lower() == 'site-packages':
                sitePackagesDirs.append( path )
        for path in sitePackagesDirs:
            destinationFile = path + os.sep + __file__
            try:
                shutil.copyfile( './%s' % __file__, destinationFile )
                print 'Copied %s to %s' % ( __file__, path )
            except:
                print 'Could not copy file %s to location %s' % ( __file__, path )
    else:
        print 'Unit-tests failed (see items below)'
        print '#'*79
        if testResults.errors:
            print 'Errors'
            print '#' + '-'*77 + '#'
            for error in testResults.errors:
                print error[1]
                print '#' + '-'*77 + '#'
            print
        if testResults.failures:
            if testResults.errors:
                print '#'*79
            print 'Failures'
            print '#' + '-'*77 + '#'
            for failure in testResults.failures:
                print failure[1]
                print '#' + '-'*77 + '#'
            print
    print '#'*79

There's a lot of stuff going on here that may not be immediately apparent:

Line(s)

4-11

Some meta-data about the module, which I found a fairly complete list of at this University of Colorado page. Some of these items (__version__ for example) will appear in standard pydoc output, some will not.

19, 91

Per official Python documentation, __all__ should be used to ensure that module- and package-items can be imported using the from [module] import [item|*] syntax that is common in Python code. An example of this, as it relates to the provided template class ClassName appears on line 91.

25, 71

The DocMeta module is a documentation-meta-data module I built for my own ends, that makes documenting things like function and method arguments, configuration file needs, and so on easier, using a set of decorator functions. An example of it's output (from pydoc browsing) for the ClassName class listed in the template is:

class ClassName(__builtin__.object)
     Class doc-string.
 
   Methods defined here:

__init__(self)
    Object constructor.
     
    ###################
    ## Arguments     ##
    ###################
     + self ................ The object-instance that the method is called against.

The DocMeta module has been shared, though I haven't yet posted about it. It can be found at dl.dropbox.com/u/1917253/site-packages/DocMeta.py

44-91

This is a template class, ClassName, including commented sections for everything I thought I might ever need:

Line(s)

48

Class Attributes (class-defined attributes and their values, where applicable.

52, 56 and 60

Property Getter, Setter and Deleter methods - since I prefer to have the getter methods freestanding and use a property call later to assign them to properties.

64

Property declarations

68-81

A stub object-constructor method, with commented-out common code for making a class nominally final (75-76), nominally abstract (78, 80), or nominally an interface (78-79).

84

An object destructor. I haven't actually found a need for one yet in my Python travels, but that doesn't mean I won't

88

Class methods.

I've found that I prefer to group methods that are being overridden from other classes in their own sections, whenever I can, after the "local" methods have been defined.

91

Inclusion of the class into the __all__ list

94

Package- and module-inclusions, each of which should be added to the __all__ list.

98-177

Unit-test definitions. I have a very specific unit-test structure that I prefer (at least for now), that I've written some utility code for in the UnitTestUtilities module (again, I'll share that soon).

Line(s)

105, 116 and 177

Define a TestSuite to gather up all the unit-tests in, and make sure that each TestCase-derived test-class is added to that suite.

112-116

Unit-test the package/module constants. They should be predictable, and raise errors if breaking changes are caused by altering them.

119

Unit-test any functions defined in the module.

123

Unit-test any nominal interfaces defined in the module. They may have nominally-abstract properties (I typically just do something like PropertyName = property() for abstract properties) and nominally-abstract methods (I usually just raise a NotImplementedError).

127

Unit-test any nominal abstract classes defined in the module. Any nominally-abstract properties and methods defined in one of these follows the pattern for nominal interfaces, above, while concrete properties and methods should be tested "normally."

131-177

Unit-test any classes defined in the module. At the very least, I like to provide template locations for:

Line(s)

134

A class derived from the class being tested, for use in either testing it's final or nominally-abstract nature (147, 151)

141, 144

Standard setUp and tearDown methods

147

A commented-out test method for nominally-final classes (75-76).

151

A commented-out test method for nominally-abstract classes or nominal interfaces (78-80).

155

Unit-tests for object-construction, though typically I find that the abstract/interface/final tests suffice.

159-166

Test for the expected number of class-properties, and that those properties each have an easily-identifiable test-method.

This functionality is dependent on the UnitTestUtilities module mentioned earlier. What it does, in a nutshell, is to use the inspect module's capabilities to find all class-members that are data descriptors (properties), then check each named item thus found to assure that there is a test-method defined for each (by looking for a testPropertyName method in the test-case class if the property name of the class being tested is PropertyName, for example). It also requires the developer to pay attention to the number and names of proerties that are being acquired from a super-class, which I've found very helpful.

168-173

Test for the expected number of class-methods, and that those methods each have an easily-identifiable test-method.

Apart from the fact that this test-method is looking for methods instead of properties, it behaves much like the property-oriented test-method above.

178-218

Unit-test runs and module installation on successful test-runs.

The last several lines, 178-218, run the unit-tests defined, and if there are no failures, attempts to copy the file being tested to any/all of the various site-packages directories that the script can discover.