Slower than I'd hoped, and ...

Well, I've had a few more days to ponder on things, and my work situation, while *very* nice/neat/cool, just isn't going to allow me to continue at anywhere near the pace I'd been maintaining before the change in jobs happened. In addition, I'm actually writing Python code at work, and though I'm doing a lot of really neat things with it, I'm not sure whether I can share any of it at all (I certainly can't without approval from my boss).

I'll try to ask around and see if any of what I'm doing is OK to share here (I hope so, since I've written a couple thousand lines of code so far), but until I know for sure, I'm gonna err on the side of caution, and not post anything unless I *know* it's mine, unencumbered.

Interlude: Enumerations and TypedContainers

So, while I ponder my next step(s), it occurred to me that I have some old code lying around that could stand to be polished and/or brought up to my current standards. One item from that mix is an attempt to implement an Enumerated Type, much like the types provided in Java and C#. Although it isn't, technically, a strongly-typed container like TypedDictionary, TypedList or TypedTuple are, it has enough similar characteristics that dropping the Enumeration class into the TypedContainers module felt reasonable. Consider:

• An enumeration is a collection of values, like dictionaries, lists and tuples;
• Also like dictionaries, lists, and tuples, enumerations are (or at least can be considered as a type);
• Value-membership is also a shared characteristic. That is, all of the established types and enumerations both should allow a developer to determine whether some arbitrary value is a member of the instance.
• Unlike those types, however, an enumeration has a fixed, immutable set of values.

Whether the member-values of an enumeration need to be of the same type or not is an open question, to my mind. If they are, so be it. But, ultimately, the purpose that an enumeration serves (mainly, that I'm aware of) is to collect some fixed set of values, which feels like a type, regardless.

So that (I hope) justifies the decision to put Enumeration into TypedContainers (shared at dl.dropbox.com/u/1917253/site-packages/TypedContainers.py).

By preference, I'd like to be able to use code structures like:

SomeStuff = Enumeration(
    'This is an enumeration of values for some purpose', 
    Good=1,
    Bad=-1,
    Indifferent=0
    )

# Dot-notation should be an option:
goodValue = SomeStuff.Good

# For comparison/membership-detection, "in" should be used:
if( goodValue in SomeStuff ):
    # do something

The creation of an enumeration (lines 1-6) should allow for the generation of an instance doc-string (though, to be truthful, I'm not sure that it'll ever be of much use). At present, this doc-string won't appear anywhere, but I'll eventually get my head together on the documentation side of things to work out a way to retrieve it for API documentation, so it might as well be available now, rather than having to come back and implement it later. The generation of member-names and values should be as simple as providing them suring the construction of the instance, and they should not be limited in number or type.

Use of dot-notation for specific member-names/-values should be allowed (line 9), and detection of whether some arbitrary value is present within the Enumeration should be as simple as possible (I prefer the in keyword, lines 12-13).

With all of these requirements/preferences in mind, here's the (updated) code that I generated to meet them:

class Enumeration( object ):
    """Provides a "formal" enumeration-type, allowing a single class-instance to be
used to provide dot-notation enumeration values."""

    ##################################
    # Class Attributes               #
    ##################################

    __isLocked = False
    _iterableItems = []
    _key = 0

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetMembers( self ):
        """Gets the members of the Enumeration as a Dictionary value."""
        return self.__members

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetMemberNames( self ):
        """Gets the names of the Enumeration's members."""
        return self.__members.keys()

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetMemberValues( self ):
        """Gets the values of the Enumeration's members."""
        return self.__members.values()

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    ##################################
    # Class Properties               #
    ##################################

    Members = property( _GetMembers, None, None, _GetMembers.__doc__ )
    MemberNames = property( _GetMemberNames, None, None, _GetMemberNames.__doc__ )
    MemberValues = property( _GetMemberValues, None, None, _GetMemberValues.__doc__ )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'docString', None, 'The documentation-string for the Enumeration.' )
    @DocumentArgument( 'keyword', 'members', None, 'The members of the Enumeration, in the form "MemberName=MemberValue".' )
    @DocumentException( NotImplementedError, 'used as a base class' )
    @DocumentException( TypeError, 'not supplied with a keyword-argument set' )
    @DocumentException( KeyError, 'an enumeration member-name is specified that is used internally by the class (see it\'s properties)' )
    def __init__( self, docString, **members ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != Enumeration:
            raise NotImplementedError( 'Enumeration is (nominally) a final class, and is not intended to be derived from.' )
        if type( members ) != types.DictType:
            raise TypeError( "Enumeration expected a keyword-list of member names and values." )
        self.__isLocked = False
        self._key = 0
        self.__members = {}
        forbiddenKeys = dir( self )
        for theMember in members:
            if theMember in forbiddenKeys:
                raise KeyError( "An Enumeration cannot override the %s key-name: It's reserved by the object." % ( theMember ) )
            self.__dict__[ theMember ] = members[ theMember ]
            self.__members[ theMember ] = members[ theMember ]
        if docString != None:
            self.__doc__ = docString + (' %s' % ( self.__members.keys() ) ).replace( "'", '' )
        self._iterableItems = self.__members.values()
        self.__isLocked = True

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def __setattr__( self, name, value ):
        """Intercepts attempts to set attribute-values, and prevents them from being set if the instance has been locked."""
        # Allow changes to self._key for iteration purposes:
        if not self.__isLocked or name == '_key':
            return object.__setattr__( self, name, value )
        raise AttributeError( 'Enumerations are immutable after instantiation.' )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def __iter__( self ):
        """Standard iterator handle."""
        self._key = 0
        return copy.copy( self )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentException( StopIteration, 'the end of an iteration against an instance is reached (per specifications for an iterable object)' )
    def next( self ):
        """Standard iteration mechanism."""
        try:
            thisKey = self._key
            self._key += 1
            return self._iterableItems[ thisKey ]
        except IndexError:
            self._key = 0
            raise StopIteration

__all__ += [ 'Enumeration' ]

Commentary

Line(s)

9-11

Provides a default value for the __isLocked attribute of the instance. Without this, the __setattr__ call made implicitly during object-construction to set it's value to False fails, since no attribute exists to check. _key is similarly set, to avoid it being similarly constrained.

17-30, 44-46

My typical property-getter and property-declaration structure, for read-only properties.

52-77

Object constructor:

63-64

Checks that the members argument is a dictionary, which is the default type for a keyword argument list. This may not be strictly necessary, but given that I anticipate the use of constructor-calls along the lines of MyEnumeration = Enumeration( **someDictionary ), it felt safer to leave it in place than to remove it.

65-68

Sets the default values for various properties, then gets every member-name of the instance (with dir()) in order to have a list of forbidden member-names for later checking/comparison.

69-73

Iterates through the provided member names/values dictionary, checks each key-name therein against the forbidden keys (raising a KeyError if there's a collision), and if everything's OK, it attaches the member name/value to the instance's __dict__ to make it available in a dot-notation syntax later. This is also where (line 72) the internal __members dictionary gets populated so that the various Member... properties and getters defined previously have something to work with.

74-75

Sets the __doc__ of the instance, making sure to include the provided member-names.

76

Sets the _iterableItems array up, so that it's available for use by the iteration methods later.

77

Locks the object so that any future use of __setattr__ will know that the object is locked, and should not be modified.

88-93

Leverages the built-in __setattr__ method to prevent the addition of new attributes or modification of existing ones once the instance is locked.

96-99, 101-111

The __iter__ method is Python's standard mechanism for iterating over iterator-types - and since I wanted Enumeration to support the in keyword to determine membership, it needs to be iterable. In this particular implementation, all it does is return a shallow copy of the original instance to be iterated over with next. Similarly, the next method is a standard iteration-mechanism, which returns the next item from the container/collection.

There are some new/unusual/interesting items in the unit-tests, I think, so I'll share those as well.

    class EnumerationDerived( Enumeration ):
        def __init__( self, docString, **members ):
            Enumeration.__init__( self, docString, **members )
    
    class testEnumeration( unittest.TestCase ):
        """Unit-tests the Enumeration class."""
    
        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass

        def getGoodValues( self ):
            return [
                {'name1':'value1', 'name2':'value2'},
                {'name1':True, 'name2':False},
                {'name1':-1, 'name2':0, 'name3':1},
                ]
        
        def testFinal( self ):
            """Testing final nature of the Enumeration class."""
            try:
                testObject = EnumerationDerived( 'docstring', Name='value' )
                self.fail( 'Enumeration is nominally a final class, and should not be extensible.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Attempting to instantiate a class derived from Enumeration should raise NotImplementedError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
    
        def testConstruction( self ):
            """Testing construction of the Enumeration class."""
            testValues = self.getGoodValues()
            for testValue in testValues:
                testObject = Enumeration( 'docstring', **testValue )
                self.assertEquals( testValue.keys(), testObject.MemberNames, 'An Enumeration instance created with %s as members should have %s as member names, but %s was returned instead.' % ( testValue, testValue.keys(), testObject.MemberNames ) )
                self.assertEquals( testValue.values(), testObject.MemberValues, 'An Enumeration instance created with %s as members should have %s as member values, but %s was returned instead.' % ( testValue, testValue.values(), testObject.MemberValues ) )
                self.assertEquals( testValue, testObject.Members, 'An Enumeration instance created with %s as members should have %s as member values, but %s was returned instead.' % ( testValue, testValue, testObject.Members ) )
            # Construction should fail with bad docstrings...
            badDocstrings = [ True, 1, object() ]
            for testValue in badDocstrings:
                try:
                    testObject = Enumeration( testValue, One=1, Two=2 )
                    self.fail( '%s should not be a valid docstring for an Enumeration' % ( testValue ) )
                except TypeError:
                    pass
                except Exception, error:
                    self.fail( 'Passing %s as a docstring should raise TypeError, but %s was raised instead:\n  %s' % ( testValue, error.__class__.__name__, error ) )
    
        def testPropertyCountAndTests( self ):
            """Testing the properties of the Enumeration class."""
            items = getMemberNames( Enumeration )[0]
            actual = len( items )
            expected = 3
            self.assertEquals( expected, actual, 'Enumeration is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the Enumeration class."""
            items = getMemberNames( Enumeration )[1]
            actual = len( items )
            expected = 1
            self.assertEquals( expected, actual, 'Enumeration is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Test properties

        def testMemberNames( self ):
            """Tests the MemberNames property of the Enumeration class."""
            #Tested in construction
            pass

        def testMemberValues( self ):
            """Tests the MemberValues property of the Enumeration class."""
            #Tested in construction
            pass

        def testMembers( self ):
            """Tests the Members property of the Enumeration class."""
            #Tested in construction
            pass

        # Test methods

        def testnext( self ):
            """Tests the next method of the Enumeration class."""
            #Tested in iterability tests
            pass

        # Test iterabilility
        def testIterability( self ):
            """Tests the various iteration-dependent functions of the Enumeration class."""
            testValues = self.getGoodValues()
            for testValue in testValues:
                testObject = Enumeration( 'docstring', **testValue )
                for member in testValue:
                    memberValue = testValue[ member ]
                    self.assertTrue( memberValue in testObject, 'Test of %s being in the Enumeration\'s values should return true.' % ( memberValue ) )
            testObject = Enumeration( 'docstring', Zero=0, One=1, Two=2, Three=3 )
            testValue = 0
            for value in testObject:
                self.assertEquals( testValue, value, 'The iterated value should match' )
                testValue += 1

        # Test immutability after instantiation
        def testImmutability( self ):
            """Tests the immutability of the members/values of an Enumeration after it's been instantiated."""
            testObject = Enumeration( None, Good=1, Bad=-1, Indifferent=0 )
            try:
                testObject.Ugly = 2
                self.fail( 'Once instantiated, the members of an Enumeration should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Once instantiated, modification of the members of an Enumeration should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__ ) )
            try:
                testObject.Good = 2
                self.fail( 'Once instantiated, the members of an Enumeration should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Once instantiated, modification of the members of an Enumeration should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__ ) )

        # Test dot-notation access
        def testDotAccess( self ):
            """Tests dot-notation access to member values by name."""
            testObject = Enumeration( None, Good=1, Bad=-1, Indifferent=0 )
            self.assertTrue( hasattr( testObject, 'Good' ), 'Member names should be accessible as attributes of the Enumeration instance.' )
            self.assertTrue( hasattr( testObject, 'Bad' ), 'Member names should be accessible as attributes of the Enumeration instance.' )
            self.assertTrue( hasattr( testObject, 'Indifferent' ), 'Member names should be accessible as attributes of the Enumeration instance.' )
            self.assertEquals( testObject.Good, 1, 'Member values accessed through dot-notation should match the values supplied at creation.' )
            self.assertEquals( testObject.Bad, -1, 'Member values accessed through dot-notation should match the values supplied at creation.' )
            self.assertEquals( testObject.Indifferent, 0, 'Member values accessed through dot-notation should match the values supplied at creation.' )

        # Test reserved-name creation - all should raise errors
        def testReservedNames( self ):
            """Tests that the reserved names (existing attributes) aren't allowed as member-names."""
            names = [ '_key', 'Members', 'MemberNames', 'MemberValues' ]
            for name in names:
                members = { name:True }
                try:
                    testObject = Enumeration( 'docstring', **members )
                    self.fail( 'Creation of an Enumeration with a member-name of %s should raise KeyError' % ( name ) )
                except KeyError:
                    pass
                except Exception, error:
                    self.fail( 'Creation of an Enumeration with a member-name of %s should raise KeyError but %s was raised instead:\n  %s' % ( name, error.__class__.__name__, error ) )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testEnumeration ) )

Line(s)

14-19

While poking around at some proof-of-concept PHP code at work over the last few weeks, it occurred to me that a test-case class could be extended upon to provide at lease some of the typical functionality I find myself using. One of those typical items is the generation of lists of good and bad property-values for various cases. In this case, rather than repeating the "good" cases at least twice (see lines 33, 99), which could lead to the values being tested for those methods getting out of sync and leading to inconsistent testing, I'm calling one method to return all of the good values.

I may very well refactor the UnitTestUtilities module to make use of this concept (and while I'm there, maybe attach the oft-repeated testMethodCountAndTests and testPropertyCountAndTests methods so that I'll have a unit-test class that just does that automatically.

31-48

Apart from the fact that it makes use of the good-values method noted above, this is pretty typical for construction-testing. Note that the test-method tests both good and bad values.

70-83, 87-90

These test-methods probably look a bit odd, since they are required, but they have no useful implementation. The useful tests all exist elsewhere, so the required test-methods simply note that this is the case as a comment, and pass.

93-105

This marks the first test-method of this set that tests an overall characteristic of the class - in this case, it's iterability. The method tests both whether the in keyword works as expected/desired (line 100) for each member, and the "overall" iterability of instances (102-105).

108-124

This method tests that instances are immutable after creation (another "characteristic"-based test-method).

127-135

This method tests that member-names specified at construction are available as attributes of the instance by name, and that the values behind those attribute-names match what was supplied.

138-149

This test-method was almost missed... I realized just before I was first scheduling this for publication on the blog that I hadn't tested whether using any of the "reserved" names at construction would raise the errors I wanted. I could've gone back and added these tests to the constructor tests instead of making a new method specifically for them, but since it's kinda critical that class members not get destroyed, it seemed apropos to call the tests out in their own method.

And that wraps up the Enumeration...

Where to go from here?

With MySQLConnector (and the supporting development that led up to it) complete, I'm at something of a loss as to where to go from here. My daily routine hasn't settled down yet, and though I'd hoped to be able to get some serious code-blogging time in over the week off from work between Christmas and New Year's, that hasn't happened. One complete computer rebuild, which pointed out a critical flaw in my home-system backup-plan, correcting that and trying like hell to back things up before the main drive on the old machine died completely. The random chaos of the holiday season. The demands of family while I'm home and they're awake. The list goes on and on. I did at least manage to get the data-connectors module (up through MySQLConnector) completed, which I'd promised myself I'd get done.

I have a list of possibles to pick from, some of which are dependent on other items on that list.

But the simple fact of the matter is that I'm kinda stuck here. If any readers have a preference what I should tackle next, please comment and let me know. Otherwise, I'll poke around a bit and see what, if anything, strikes my fancy (though I have to say that I'm leaning towards the MiniAppServer idea at present).

The MySQLConnector module

Happy New Year, readers! The bulk of today's code is shared at dropbox.com/u/1917253/site-packages/MySQLConnector.py.

So the last leg of our journey into Python-database connectivity is (finally) the MySQLConnector class. There's been a lot of what I'd call "support" development to get to this point:

• Three interfaces (IsDataConnector, IsQuery, and IsResultSet);
• One abstract class (BaseDataConnector); and
• Three concrete classes (Query, ResultSet and Record).

These took considerably longer to write about than I'd expected, though I'm glad that they provided enough material for the number of posts that they generated. There's also a fair amount of code that had to be written: With unit-tests, the DataConnectors module is just barely closer to 2,400 lines of code than 2,300, in about a 40:60 split between "real" code and test-code.

One (hopefully significant) advantage of this approach is that the creation of the MySQLConnector class is going to be crazy-simple. As a derivative of both BaseDataConnector and IsConfigurable, there's really not a whole lot of code that's specific to the MySQLConnector to be written, and other similar classes (say, a hypothetical PostGreSQLConnector) would require a similarly small level of effort. Consider that MySQLConnector has the following properties and methods:

• The Configure method, from IsConfigurable, which may need specific implementation, but that was implemented at a basic level in BaseDataConnector.
• The Connect method, required by IsDataConnector, which will need implementation.
• The Database property, from BaseDataConnector, that is complete and tested until or unless we need to make implementation-specific changes.
• The Execute method, required by IsDataConnector, which will need implementation.
• The Host property, from BaseDataConnector, also complete and tested, barring implementation-specific changes.
• The Password property, from BaseDataConnector, also complete and tested, barring implementation-specific changes.
• The User property, from BaseDataConnector, also complete and tested, barring implementation-specific changes.
• And, of course, unit-tests for the new class. Ideally, there should also be some system tests to test/prove out functionality against an actual database.

As I was working on MySQLConnector, I discovered that I'd never finished figuring out how to unit-test the Configure method of it, and that I was missing a Configuration class:

Configuration (from the Configuration module):

class Configuration( ConfigParser.ConfigParser, object ):
    """Represents configuration data derived from a text-file."""

    #################################
    # Class Attributes              #
    #################################

    #################################
    # Class Property-Getter Methods #
    #################################

    def _GetConfigFile( self ):
        """Gets the config-file-path specified at object creation."""
        return self._configFile

    def _GetSections( self ):
        """Gets the available configuration-sections."""
        if self._configSections == None:
            try:
                self.read( self._configFile )
            except Exception, error:
                raise AttributeError( '%s.Sections: Could not read configuration from %s. Original error:\n\t%s' % ( self.__class__.__name__, self._configFile, error ) )
            self._configSections = {}
            for theSection in self.sections():
                self._configSections[ theSection ] = {}
                for theItem in self.items( theSection, 1 ):
                    itemName = theItem[0]
                    itemValue = theItem[1]
                    if itemValue[0] in [ '[', '{', '(' ]:
                        # Evaluate the value instead... Ugly, but viable...
                        itemValue = eval( itemValue )
                    self._configSections[ theSection ][ itemName ] = itemValue
        return self._configSections

    #################################
    # Class Property-Setter Methods #
    #################################

    #################################
    # Class Properties              #
    #################################

    ConfigFile = property( _GetConfigFile, None, None, _GetConfigFile.__doc__ )
    Sections = property( _GetSections, None, None, _GetSections.__doc__ )

    #################################
    # Object Constructor            #
    #################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'filePath', None, '(String) The path to the configuration-file.' )
    def __init__( self, filePath ):
        """Object constructor."""
        ConfigParser.ConfigParser.__init__( self )
        self._configSections = None
        self._configFile = filePath

    #################################
    # Object Destructor             #
    #################################

    #################################
    # Class Methods                 #
    #################################

__all__ += [ 'Configuration' ]

This will likely need to be refactored at some point, since there are other configuration mechanisms than the ConfigParser base here. For now, though, I'll leave it be. Configuration instances are basically just wrappers around ConfigParser instances, that provide ConfigFile and Sections properties. The ConfigFile is probably pretty straightforward - it's a pointer to the configuration-file that the instance uses. Sections might be a little harder to understand, though - I wanted to have a simple access mechanism for the configuration-sections of a given configuration-file, in order to be able to retrieve section-items without having to make a function-call every time.

With that in place, I could finish the unit-tests for BaseDataConnector:

        def testConfigure( self ):
            """Unit-tests the Configure method of the BaseDataConnector abstract class."""
            # First, we need to build out a configuration-file
            hostName = 'hostName'
            databaseName = 'databaseName'
            userName = 'userName'
            userPassword = 'userPassword'
            configString = """[Default Section]
SectionName1: SectionValue1
SectionName2: SectionValue2

[Data Access]
host: %s
database: %s
user: %s
password: %s
""" % ( hostName, databaseName, userName, userPassword )
            fp = open( 'test.cfg', 'w' )
            fp.write( configString )
            fp.close()
            # Now we need to create the configuration object and read in the configuration values
            configParser = Configuration( 'test.cfg' )
            # Now, finally, we can test the configuration
            testObject = BaseDataConnectorDerived()
            testObject.Configure( configParser, 'Data Access' )
            self.assertEquals( testObject.Host, hostName, 'The host name should be retrievable from configuration' )
            self.assertEquals( testObject.Database, databaseName, 'The database name should be retrievable from configuration' )
            self.assertEquals( testObject.User, userName, 'The user name should be retrievable from configuration' )
            self.assertEquals( testObject.Password, userPassword, 'The user password should be retrievable from configuration' )
            os.unlink( 'test.cfg' )

All this is doing is creating a hard-coded (and very simple) configuration-file, with a Data Access section that contains the host, database and user names for the connector, and the password that it should use. Looking at lines 22 and 24-25, creation of a BaseDataConnector-derived instance should always be feasible with this sort of code-structure: Create an instance with no arguments, create a configuration instance, then call the Configure method of the data-connector instance.

It may bear noting that MySQLConnector doesn't have any sort of direct support for cursors - that may be something that I'll add in to BaseDataConnector at some point in the future, if there's support for it across MySQL (yes), PostgreSQL (not sure), and ODBC (also not sure), which are the three data-connection types I expect this to support in the long run.

MySQLConnector (Nominal final class):

#####################################
# Convenience imports               #
#####################################

# See To-do list on MySQLConnector.__init__

#####################################
# Classes defined in the module     #
#####################################

class MySQLConnector( BaseDataConnector, IsConfigurable, object ):
    """Class doc-string."""

    ##################################
    # Class Attributes               #
    ##################################

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetConnection( self ):
        """Gets the connection to the specified MySQL database."""
        if self._connection == None:
            self.Connect()
        return self._connection

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'value', None, 'The name of the database the connection will be made to.' )
    def _SetDatabase( self, value ):
        if self._connection != None:
            raise AttributeError( '%s.Database cannot be reset once a connection is established' % ( self.__class__.__name__ ) )
        BaseDataConnector._SetDatabase( self, value )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'value', None, 'The name or IP address of the host that the database being connected to lives on.' )
    def _SetHost( self, value ):
        if self._connection != None:
            raise AttributeError( '%s.Host cannot be reset once a connection is established' % ( self.__class__.__name__ ) )
        BaseDataConnector._SetHost( self, value )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'value', None, 'The password that will be used to connect to the database.' )
    def _SetPassword( self, value ):
        if self._connection != None:
            raise AttributeError( '%s.Password cannot be reset once a connection is established' % ( self.__class__.__name__ ) )
        BaseDataConnector._SetPassword( self, value )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'value', None, 'The user-name that will be used to connect to the database.' )
    def _SetUser( self, value ):
        if self._connection != None:
            raise AttributeError( '%s.User cannot be reset once a connection is established' % ( self.__class__.__name__ ) )
        BaseDataConnector._SetUser( self, value )

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _DelConnection( self ):
        self._connection = None

    ##################################
    # Class Properties               #
    ##################################

    Connection = property( _GetConnection, None, None, 'Gets the connection to the specified MySQL database.' )
    Database = property( BaseDataConnector._GetDatabase, _SetDatabase, BaseDataConnector._DelDatabase, BaseDataConnector._GetDatabase.__doc__ )
    Host = property( BaseDataConnector._GetHost, _SetHost, BaseDataConnector._DelHost, BaseDataConnector._GetHost.__doc__ )
    Password = property( BaseDataConnector._GetPassword, _SetPassword, BaseDataConnector._DelPassword, BaseDataConnector._GetPassword.__doc__ )
    User = property( BaseDataConnector._GetUser, _SetUser, BaseDataConnector._DelUser, BaseDataConnector._GetUser.__doc__ )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'keyword', 'parameters', 'host', 'The name or IP-address of the host that the database resides upon.' )
    @DocumentArgument( 'keyword', 'parameters', 'database', 'The name of the database that the connection will be made to.' )
    @DocumentArgument( 'keyword', 'parameters', 'user', 'The user-name used to connect to the database.' )
    @DocumentArgument( 'keyword', 'parameters', 'password', 'The password used to connect to the database.' )
    @ToDo( 'Figure out what items from DataConnectors should be imported for convenience in using the MySQLConnector class.' )
    def __init__( self, **parameters ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != MySQLConnector:
            raise NotImplementedError( 'MySQLConnector is (nominally) a final class, and is not intended to be derived from.' )
        self._connection = None
        BaseDataConnector.__init__( self, **parameters )

    ##################################
    # Object Destructor              #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def __del__( self ):
        self._Close()

    ##################################
    # Class Methods                  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _Close( self ):
        if self._connection:
            self._connection.close()

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def Connect( self ):
        """Connects to the database specified by Database, residing on Host, using User and Password to connect."""
        self._connection = MySQLdb.connect( 
            host=self._host, 
            db=self._database, 
            user=self._user, 
            passwd=self._password
            )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'query', None, 'The IsQuery object that will be executed against the object\'s Connection.' )
    def Execute( self, query ):
        """Executes the provided IsQuery object's query against the object's connection, returning one or more results."""
        if not isinstance( query, IsQuery ):
            raise TypeError( '%s.Execute expects an instance of IsQuery as it\'s query argument.' % ( self.__class__.__name__ ) )
        self.Connection.query( query.Sql )
        resultSets = []
        resultSet = []
        results = self.Connection.store_result()
        while results:
            row = list( results.fetch_row( 1, 1 ) )
            while row:
                resultSet += row
                row = results.fetch_row( 1, 1 )
            if self.Connection.next_result() == 0:
                resultSets.append( resultSet )
                resultSet = []
                results = self.Connection.store_result()
            else:
                break
        resultSets.append( resultSet )
        return resultSets

__all__ += [ 'MySQLConnector' ]

Line(s)

5

This is a fine-tuning item that will almost certainly evolve in an obvious way as I start writing code that actually uses MySQLConnector: There will almost certainly be some (small?) set of imports from the base DataConnectors module that will be needed frequently enough to warrant importing them as part of the MySQLConnector import. Off the top of my head, I'd expect at least Query and maybe ResultSet to be needed, but I'll wait to see what actually crops up as I start using the module...

23-27

An almost-typical property getter method, the main difference being that the underlying MySQLdb Connection object isn't instantiated until it's needed.

33-59

Overridden property-setter methods. The only additional functionality that they provide is to prevent the modification of any connection-control property (the host, database- and user-name, and password used for the connection) once the connection has been instantiated. The rationale for this is, basically, that once a connection has been established, changes to that connection should not be allowed. Chances are good that an error of some type would be thrown anyway, but since I'd rather raise errors as close to their "real" source as possible, I'm explicitly causing that to happen here.

73-77

The actual property declarations. Since most of these are a mix of the functionality from BaseDataConnector (for the _Get... methods) and local implementations (_Set... methods), there's a mix of complete external method-references and "normal" local references to accommodate.

101-103, 109-112

I'm not sure that a formal object-destructor's actually needed, since the underlying MySQLdb Connection object probably closes the database connection all on it's own, but to be safe, I'm providing one that calls to the _Close method to make sure that the connection is closed cleanly when the object is destroyed.

The _Close method is necessary because when the destructor fires, it doesn't recognize object properties. I think I mentioned this same pattern somewhere before, but I cannot recall where, exactly.

114-122

A public method to open the database-connection with the instance's parameters.

124-144

The method that actually queries against the database, returning a list of lists of dictionaries, ready to be passed to ResultSet constructors.

128-129

Typical type-checking of the supplied query argument.

130

Run the provided Query's SQL against the database, readying the results for formatting/digestion.

131, 132

Prep the lists of all results, and the first list of rows for use.

133

Get (and store) the first result-set returned.

This uses store_result instead of use_result, in the belief that a well-designed application (and database) should rarely return so many records that storing them wouldn't provide faster throughput than keeping the database-connection open and retrieving result-sets one at a time.

134

For so long as there are results:

135

Get the first row of the current result-set

136-137

For so long as there's a valid row, add the row to the current result-set, and get the next row.

139-142

Once the current result's row-set has been processed, if there is another result-set after the current one, append the current results to the list to be returned, reset the row-list, and get/store the next result-set.

143-144

If there are no other result-sets, then exit the loop.

145

Make sure that the last set of rows gets attached to the results to be returned before returning them.

Unit-tests

Some of the unit-tests (the ones that are very basic system-tests, all in the testExecute method) require some actual tables on the test_database database:

CREATE TABLE `test_database`.`test_table_1` (
  `id` INT UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'The record-id',
  `first_name` VARCHAR(30)  NOT NULL COMMENT 'A first name',
  `last_name` varchar(30)  NOT NULL COMMENT 'A last-name',
  PRIMARY KEY (`id`)
)
ENGINE = MyISAM
COMMENT = 'A test-table';

INSERT INTO test_table_1 ( first_name, last_name )
VALUES ( 'Brian', 'Allbee' ), ( 'Joe', 'Smith' );

CREATE TABLE `test_database`.`test_table_2` (
  `id` int UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'Record-ID',
  `name_id` int  NOT NULL 
      COMMENT 'ID of the record from test_table_1 that relates to this record',
  `email` varchar(60)  NOT NULL COMMENT 'An email address',
  PRIMARY KEY (`id`)
)
ENGINE = MyISAM
COMMENT = 'Another test table';

INSERT INTO test_table_2 ( name_id, email )
VALUES ( 1, 'brian.allbee@somedomain.com' ), ( 2, 'joe.smith@somedomain.com' );

    class MySQLConnectorDerived( MySQLConnector ):
        def __init__( self, **parameters ):
            MySQLConnector.__init__( self, **parameters )
        def __del__( self ):
            try:
                MySQLConnector.__del__( self )
            except:
                pass
    
    class testMySQLConnector( unittest.TestCase ):
        """Unit-tests the MySQLConnector class."""
    
        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass

        def getLiveConnection( self ):
            """Returns a MySQLConnector instance pointing to a common local test-database."""
            return MySQLConnector( host='localhost', database='test_database', user='test_user', password='test_password' )
        
        def testFinal( self ):
            """Testing final nature of the MySQLConnector class."""
            try:
                testObject = MySQLConnectorDerived()
                self.fail( 'MySQLConnector is nominally a final class, and should not be extendable.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Instantiating a derivation of MySQLConnection should raise NotImplementedError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
    
        def testConstruction( self ):
            """Testing construction of the MySQLConnector class."""
            testObject = MySQLConnector()
            self.assertTrue( isinstance( testObject, MySQLConnector ), 'Instances of MySQLConnector should be instances of MySQLConnector.' )
            self.assertEquals( testObject.Host, None, 'With no host supplied at construction, it\'s property value should be None' )
            self.assertEquals( testObject.Database, None, 'With no database supplied at construction, it\'s property value should be None' )
            self.assertEquals( testObject.User, None, 'With no user supplied at construction, it\'s property value should be None' )
            self.assertEquals( testObject.Password, None, 'With no password supplied at construction, it\'s property value should be None' )
            testHost = 'testHost'
            testDatabase = 'database'
            testUser = 'testUser'
            testPassword = 'testPassword'
            testObject = MySQLConnector( host=testHost, database=testDatabase, user=testUser, password=testPassword )
            self.assertEquals( testObject.Host, testHost, 'The host supplied at construction should be present in the object\'s properties' )
            self.assertEquals( testObject.Database, testDatabase, 'The database supplied at construction should be present in the object\'s properties' )
            self.assertEquals( testObject.User, testUser, 'The user supplied at construction should be present in the object\'s properties' )
            self.assertEquals( testObject.Password, testPassword, 'The password supplied at construction should be present in the object\'s properties' )
    
        def testPropertyCountAndTests( self ):
            """Testing the properties of the MySQLConnector class."""
            items = getMemberNames( MySQLConnector )[0]
            actual = len( items )
            expected = 5
            self.assertEquals( expected, actual, 'MySQLConnector is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the MySQLConnector class."""
            items = getMemberNames( MySQLConnector )[1]
            actual = len( items )
            expected = 3
            self.assertEquals( expected, actual, 'MySQLConnector is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Unit-test properties

        def testConnection( self ):
            """Unit-tests the Connection property of the MySQLConnector class."""
            testObject = MySQLConnector( host='localhost', database='no_such_database', user='test_user', password='test_password' )
            try:
                testObject.Connect()
                self.fail( 'Trying to connect to a non-existant database should raise OperationalError.' )
            except MySQLdb.OperationalError:
                pass
            except Exception, error:
                self.fail( 'Trying to connect to a non-existant database should raise OperationalError, but %s was raised instead:\n  %s.' % ( error.__class__.__name__, error ) )

        def testDatabase( self ):
            """Unit-tests the Database property of the MySQLConnector class."""
            testObject = self.getLiveConnection()
            oldDatabase = testObject.Database
            testObject.Database = 'ook'
            self.assertEquals( testObject.Database, 'ook', 'Before a connection is made, the object should allow modification of the Database property.' )
            testObject.Database = oldDatabase
            testObject.Connect()
            try:
                testObject.Database = 'ook'
                self.fail( 'Once a connection is established, the Database property should be immutable' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Modifying the Database property after connection is establilshed should raise AttributeError, bu %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        def testHost( self ):
            """Unit-tests the Host property of the MySQLConnector class."""
            testObject = self.getLiveConnection()
            oldHost = testObject.Host
            testObject.Host = 'ook'
            self.assertEquals( testObject.Host, 'ook', 'Before a connection is made, the object should allow modification of the Host property.' )
            testObject.Host = oldHost
            testObject.Connect()
            try:
                testObject.Host = 'ook'
                self.fail( 'Once a connection is established, the Host property should be immutable' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Modifying the Host property after connection is establilshed should raise AttributeError, bu %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        def testPassword( self ):
            """Unit-tests the Password property of the MySQLConnector class."""
            testObject = self.getLiveConnection()
            oldPassword = testObject.Password
            testObject.Password = 'ook'
            self.assertEquals( testObject.Password, 'ook', 'Before a connection is made, the object should allow modification of the Password property.' )
            testObject.Password = oldPassword
            testObject.Connect()
            try:
                testObject.Password = 'ook'
                self.fail( 'Once a connection is established, the Password property should be immutable' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Modifying the Password property after connection is establilshed should raise AttributeError, bu %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

#        def testQueue( self ):
#            """Unit-tests the Queue property of the MySQLConnector class."""
#            pass

        def testUser( self ):
            """Unit-tests the User property of the MySQLConnector class."""
            testObject = self.getLiveConnection()
            oldUser = testObject.User
            testObject.User = 'ook'
            self.assertEquals( testObject.User, 'ook', 'Before a connection is made, the object should allow modification of the User property.' )
            testObject.User = oldUser
            testObject.Connect()
            try:
                testObject.User = 'ook'
                self.fail( 'Once a connection is established, the User property should be immutable' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Modifying the User property after connection is establilshed should raise AttributeError, bu %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        # Unit-test methods

        def testConfigure( self ):
            """Unit-tests the Configure method of the MySQLConnector class"""
        def testConfigure( self ):
            """Unit-tests the Configure method of the BaseDataConnector abstract class."""
            # First, we need to build out a configuration-file
            hostName = 'localhost'
            databaseName = 'test_database'
            userName = 'test_user'
            userPassword = 'test_password'
            configString = """[Default Section]
SectionName1: SectionValue1
SectionName2: SectionValue2

[Data Access]
host: %s
database: %s
user: %s
password: %s
""" % ( hostName, databaseName, userName, userPassword )
            fp = open( 'test.cfg', 'w' )
            fp.write( configString )
            fp.close()
            # Now we need to create the configuration object and read in the configuration values
            configParser = Configuration( 'test.cfg' )
            # Now, finally, we can test the configuration
            testObject = MySQLConnector()
            testObject.Configure( configParser, 'Data Access' )
            self.assertEquals( testObject.Host, hostName, 'The host name should be retrievable from configuration' )
            self.assertEquals( testObject.Database, databaseName, 'The database name should be retrievable from configuration' )
            self.assertEquals( testObject.User, userName, 'The user name should be retrievable from configuration' )
            self.assertEquals( testObject.Password, userPassword, 'The user password should be retrievable from configuration' )
            os.unlink( 'test.cfg' )

        def testConnect( self ):
            """Unit-tests the Connect method of the MySQLConnector class."""
            pass # Tested in the various property-tests above...

        def testExecute( self ):
            """Unit-tests the Execute method of the MySQLConnector class."""
            testObject = self.getLiveConnection()
            # Single-table, single row, single field
            MyQuery = Query( testObject, "SELECT first_name FROM test_table_1 LIMIT 1;" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 1, '%s should return a result-set' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0] ), 1, '%s should return a single row' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), [ 'first_name' ], '%s should return only the first_name field' % ( MyQuery.Sql ) )
            # Single-table, single row, all fields
            MyQuery = Query( testObject, "SELECT * FROM test_table_1 LIMIT 1;" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 1, '%s should return a result-set' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0] ), 1, '%s should return a single row' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0][0] ), 3, '%s should return three fields' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), [ 'first_name', 'last_name', 'id' ], '%s should return first_name, last_name and id fields' % ( MyQuery.Sql ) )
            # Single-table, all rows, one field
            MyQuery = Query( testObject, "SELECT first_name FROM test_table_1;" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 1, '%s should return a result-set' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0] ), 2 )
            self.assertEquals( testResults[0][0].keys(), [ 'first_name' ] )
            self.assertEquals( testResults[0][0].keys(), testResults[0][1].keys(), 'Field-names in different rows of the result-set should be identical' )
            # Single-table, all rows, all fields
            MyQuery = Query( testObject, "SELECT * FROM test_table_1;" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 1, '%s should return a result-set' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0] ), 2, '%s should return two rows' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0][0] ), 3, '%s should return three fields' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), [ 'first_name', 'last_name', 'id' ], '%s should return first_name, last_name and id fields' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), testResults[0][1].keys(), 'Field-names in different rows of the result-set should be identical' )
            # Multiple tables (joined), all fields
            MyQuery = Query( testObject, "SELECT t1.*, t2.* FROM test_table_1 t1 LEFT JOIN test_table_2 t2 ON t1.id=t2.name_id;" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 1, '%s should return a result-set' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0] ), 2, '%s should return two rows' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), ['first_name', 'last_name', 't2.id', 'email', 'name_id', 'id'] )
            self.assertEquals( testResults[0][0].keys(), testResults[0][1].keys(), 'Field-names in different rows of the result-set should be identical' )
            # Multiple tables, multiple result-sets
            MyQuery = Query( testObject, "SELECT * FROM test_table_1;SELECT * FROM test_table_2;" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 2, '%s should return two result-sets' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[0] ), 2, '%s should return two rows in it\'s first result-set' % ( MyQuery.Sql ) )
            self.assertEquals( len( testResults[1] ), 2, '%s should return two rows in it\'s second result-set' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), [ 'first_name', 'last_name', 'id' ], '%s should return first_name, last_name and id fields' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[0][0].keys(), testResults[0][1].keys(), 'Field-names in different rows of the result-set should be identical' )
            self.assertEquals( testResults[1][0].keys(), ['name_id', 'id', 'email'], '%s should return name_id, id and email fields' % ( MyQuery.Sql ) )
            self.assertEquals( testResults[1][0].keys(), testResults[1][1].keys(), 'Field-names in different rows of the result-set should be identical' )
            # Query that returns no results
            MyQuery = Query( testObject, "SELECT * FROM test_table_1 WHERE first_name='ook';" )
            testResults = testObject.Execute( MyQuery )
            self.assertEquals( len( testResults ), 1 )
            self.assertEquals( len( testResults[0] ), 0 )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testMySQLConnector ) )

Line(s)

1-8

An almost-typical derived-test class. The main difference here is the inclusion of an explicit object-destructor that cannot throw errors.

Unit-testing the object-destructor in any sort of meaningful way has eluded me still - I don't like not having such a test, but I'm recdonciled to having to take it on faith that the destructor is doing what it's supposed to do...

19-21

A convenience method to return a valid MySQLConnector instance pointing to the test database.

23-31

Standard final-class testing.

33-49

Object-construction testing. Note that the properties being passed as arguments to the constructor are being explicitly verified, though they shouldn't have to be as long as MySQLConnector remains true to the interface of BaseDataConnector. These are in place as much to make sure that potential future breaking changes underneath the MySQLConnector class would be caught before introducing bugs to a production system.

With the exception of the testConfigure and testExecute methods, most of the rest of the tests provided here are pretty typical. The testConfigure method is almost identical to the same method in the test-suite for BaseDataConnector, but is provided as a separate test-set in case of future modification to the Configure method of either BaseDataConnector or MySQLConnector.

testExecute, on the other hand, is an entirely different flavor of unicorn. This is really more of a system/integration test than a "real" unit-test, though I think it may straddle the line to some degree. It's necessary because the Execute method of MySQLConnector exists, but at the same time, there's no good way to really test it without making a connection to the database, and running queries against that connection. I'll hammer out some additional (and more detailed) integration tests that will eventually be added to the test-suite for MySQLConnector, but for now, this feels pretty good: Given the two tables of two rows present in the test database, I'm pretty confident that I've hit all of the meaningful variations of number of results, rows and fields that could come up against that data-set.