Slower than I'd hoped, and ...

Well, I've had a few more days to ponder on things, and my work situation, while *very* nice/neat/cool, just isn't going to allow me to continue at anywhere near the pace I'd been maintaining before the change in jobs happened. In addition, I'm actually writing Python code at work, and though I'm doing a lot of really neat things with it, I'm not sure whether I can share any of it at all (I certainly can't without approval from my boss).

I'll try to ask around and see if any of what I'm doing is OK to share here (I hope so, since I've written a couple thousand lines of code so far), but until I know for sure, I'm gonna err on the side of caution, and not post anything unless I *know* it's mine, unencumbered.

Interlude: Enumerations and TypedContainers

So, while I ponder my next step(s), it occurred to me that I have some old code lying around that could stand to be polished and/or brought up to my current standards. One item from that mix is an attempt to implement an Enumerated Type, much like the types provided in Java and C#. Although it isn't, technically, a strongly-typed container like TypedDictionary, TypedList or TypedTuple are, it has enough similar characteristics that dropping the Enumeration class into the TypedContainers module felt reasonable. Consider:

• An enumeration is a collection of values, like dictionaries, lists and tuples;
• Also like dictionaries, lists, and tuples, enumerations are (or at least can be considered as a type);
• Value-membership is also a shared characteristic. That is, all of the established types and enumerations both should allow a developer to determine whether some arbitrary value is a member of the instance.
• Unlike those types, however, an enumeration has a fixed, immutable set of values.

Whether the member-values of an enumeration need to be of the same type or not is an open question, to my mind. If they are, so be it. But, ultimately, the purpose that an enumeration serves (mainly, that I'm aware of) is to collect some fixed set of values, which feels like a type, regardless.

So that (I hope) justifies the decision to put Enumeration into TypedContainers (shared at dl.dropbox.com/u/1917253/site-packages/TypedContainers.py).

By preference, I'd like to be able to use code structures like:

SomeStuff = Enumeration(
    'This is an enumeration of values for some purpose', 
    Good=1,
    Bad=-1,
    Indifferent=0
    )

# Dot-notation should be an option:
goodValue = SomeStuff.Good

# For comparison/membership-detection, "in" should be used:
if( goodValue in SomeStuff ):
    # do something

The creation of an enumeration (lines 1-6) should allow for the generation of an instance doc-string (though, to be truthful, I'm not sure that it'll ever be of much use). At present, this doc-string won't appear anywhere, but I'll eventually get my head together on the documentation side of things to work out a way to retrieve it for API documentation, so it might as well be available now, rather than having to come back and implement it later. The generation of member-names and values should be as simple as providing them suring the construction of the instance, and they should not be limited in number or type.

Use of dot-notation for specific member-names/-values should be allowed (line 9), and detection of whether some arbitrary value is present within the Enumeration should be as simple as possible (I prefer the in keyword, lines 12-13).

With all of these requirements/preferences in mind, here's the (updated) code that I generated to meet them:

class Enumeration( object ):
    """Provides a "formal" enumeration-type, allowing a single class-instance to be
used to provide dot-notation enumeration values."""

    ##################################
    # Class Attributes               #
    ##################################

    __isLocked = False
    _iterableItems = []
    _key = 0

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetMembers( self ):
        """Gets the members of the Enumeration as a Dictionary value."""
        return self.__members

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetMemberNames( self ):
        """Gets the names of the Enumeration's members."""
        return self.__members.keys()

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def _GetMemberValues( self ):
        """Gets the values of the Enumeration's members."""
        return self.__members.values()

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    ##################################
    # Class Properties               #
    ##################################

    Members = property( _GetMembers, None, None, _GetMembers.__doc__ )
    MemberNames = property( _GetMemberNames, None, None, _GetMemberNames.__doc__ )
    MemberValues = property( _GetMemberValues, None, None, _GetMemberValues.__doc__ )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'docString', None, 'The documentation-string for the Enumeration.' )
    @DocumentArgument( 'keyword', 'members', None, 'The members of the Enumeration, in the form "MemberName=MemberValue".' )
    @DocumentException( NotImplementedError, 'used as a base class' )
    @DocumentException( TypeError, 'not supplied with a keyword-argument set' )
    @DocumentException( KeyError, 'an enumeration member-name is specified that is used internally by the class (see it\'s properties)' )
    def __init__( self, docString, **members ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != Enumeration:
            raise NotImplementedError( 'Enumeration is (nominally) a final class, and is not intended to be derived from.' )
        if type( members ) != types.DictType:
            raise TypeError( "Enumeration expected a keyword-list of member names and values." )
        self.__isLocked = False
        self._key = 0
        self.__members = {}
        forbiddenKeys = dir( self )
        for theMember in members:
            if theMember in forbiddenKeys:
                raise KeyError( "An Enumeration cannot override the %s key-name: It's reserved by the object." % ( theMember ) )
            self.__dict__[ theMember ] = members[ theMember ]
            self.__members[ theMember ] = members[ theMember ]
        if docString != None:
            self.__doc__ = docString + (' %s' % ( self.__members.keys() ) ).replace( "'", '' )
        self._iterableItems = self.__members.values()
        self.__isLocked = True

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def __setattr__( self, name, value ):
        """Intercepts attempts to set attribute-values, and prevents them from being set if the instance has been locked."""
        # Allow changes to self._key for iteration purposes:
        if not self.__isLocked or name == '_key':
            return object.__setattr__( self, name, value )
        raise AttributeError( 'Enumerations are immutable after instantiation.' )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    def __iter__( self ):
        """Standard iterator handle."""
        self._key = 0
        return copy.copy( self )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentException( StopIteration, 'the end of an iteration against an instance is reached (per specifications for an iterable object)' )
    def next( self ):
        """Standard iteration mechanism."""
        try:
            thisKey = self._key
            self._key += 1
            return self._iterableItems[ thisKey ]
        except IndexError:
            self._key = 0
            raise StopIteration

__all__ += [ 'Enumeration' ]

Commentary

Line(s)

9-11

Provides a default value for the __isLocked attribute of the instance. Without this, the __setattr__ call made implicitly during object-construction to set it's value to False fails, since no attribute exists to check. _key is similarly set, to avoid it being similarly constrained.

17-30, 44-46

My typical property-getter and property-declaration structure, for read-only properties.

52-77

Object constructor:

63-64

Checks that the members argument is a dictionary, which is the default type for a keyword argument list. This may not be strictly necessary, but given that I anticipate the use of constructor-calls along the lines of MyEnumeration = Enumeration( **someDictionary ), it felt safer to leave it in place than to remove it.

65-68

Sets the default values for various properties, then gets every member-name of the instance (with dir()) in order to have a list of forbidden member-names for later checking/comparison.

69-73

Iterates through the provided member names/values dictionary, checks each key-name therein against the forbidden keys (raising a KeyError if there's a collision), and if everything's OK, it attaches the member name/value to the instance's __dict__ to make it available in a dot-notation syntax later. This is also where (line 72) the internal __members dictionary gets populated so that the various Member... properties and getters defined previously have something to work with.

74-75

Sets the __doc__ of the instance, making sure to include the provided member-names.

76

Sets the _iterableItems array up, so that it's available for use by the iteration methods later.

77

Locks the object so that any future use of __setattr__ will know that the object is locked, and should not be modified.

88-93

Leverages the built-in __setattr__ method to prevent the addition of new attributes or modification of existing ones once the instance is locked.

96-99, 101-111

The __iter__ method is Python's standard mechanism for iterating over iterator-types - and since I wanted Enumeration to support the in keyword to determine membership, it needs to be iterable. In this particular implementation, all it does is return a shallow copy of the original instance to be iterated over with next. Similarly, the next method is a standard iteration-mechanism, which returns the next item from the container/collection.

There are some new/unusual/interesting items in the unit-tests, I think, so I'll share those as well.

    class EnumerationDerived( Enumeration ):
        def __init__( self, docString, **members ):
            Enumeration.__init__( self, docString, **members )
    
    class testEnumeration( unittest.TestCase ):
        """Unit-tests the Enumeration class."""
    
        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass

        def getGoodValues( self ):
            return [
                {'name1':'value1', 'name2':'value2'},
                {'name1':True, 'name2':False},
                {'name1':-1, 'name2':0, 'name3':1},
                ]
        
        def testFinal( self ):
            """Testing final nature of the Enumeration class."""
            try:
                testObject = EnumerationDerived( 'docstring', Name='value' )
                self.fail( 'Enumeration is nominally a final class, and should not be extensible.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Attempting to instantiate a class derived from Enumeration should raise NotImplementedError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
    
        def testConstruction( self ):
            """Testing construction of the Enumeration class."""
            testValues = self.getGoodValues()
            for testValue in testValues:
                testObject = Enumeration( 'docstring', **testValue )
                self.assertEquals( testValue.keys(), testObject.MemberNames, 'An Enumeration instance created with %s as members should have %s as member names, but %s was returned instead.' % ( testValue, testValue.keys(), testObject.MemberNames ) )
                self.assertEquals( testValue.values(), testObject.MemberValues, 'An Enumeration instance created with %s as members should have %s as member values, but %s was returned instead.' % ( testValue, testValue.values(), testObject.MemberValues ) )
                self.assertEquals( testValue, testObject.Members, 'An Enumeration instance created with %s as members should have %s as member values, but %s was returned instead.' % ( testValue, testValue, testObject.Members ) )
            # Construction should fail with bad docstrings...
            badDocstrings = [ True, 1, object() ]
            for testValue in badDocstrings:
                try:
                    testObject = Enumeration( testValue, One=1, Two=2 )
                    self.fail( '%s should not be a valid docstring for an Enumeration' % ( testValue ) )
                except TypeError:
                    pass
                except Exception, error:
                    self.fail( 'Passing %s as a docstring should raise TypeError, but %s was raised instead:\n  %s' % ( testValue, error.__class__.__name__, error ) )
    
        def testPropertyCountAndTests( self ):
            """Testing the properties of the Enumeration class."""
            items = getMemberNames( Enumeration )[0]
            actual = len( items )
            expected = 3
            self.assertEquals( expected, actual, 'Enumeration is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the Enumeration class."""
            items = getMemberNames( Enumeration )[1]
            actual = len( items )
            expected = 1
            self.assertEquals( expected, actual, 'Enumeration is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Test properties

        def testMemberNames( self ):
            """Tests the MemberNames property of the Enumeration class."""
            #Tested in construction
            pass

        def testMemberValues( self ):
            """Tests the MemberValues property of the Enumeration class."""
            #Tested in construction
            pass

        def testMembers( self ):
            """Tests the Members property of the Enumeration class."""
            #Tested in construction
            pass

        # Test methods

        def testnext( self ):
            """Tests the next method of the Enumeration class."""
            #Tested in iterability tests
            pass

        # Test iterabilility
        def testIterability( self ):
            """Tests the various iteration-dependent functions of the Enumeration class."""
            testValues = self.getGoodValues()
            for testValue in testValues:
                testObject = Enumeration( 'docstring', **testValue )
                for member in testValue:
                    memberValue = testValue[ member ]
                    self.assertTrue( memberValue in testObject, 'Test of %s being in the Enumeration\'s values should return true.' % ( memberValue ) )
            testObject = Enumeration( 'docstring', Zero=0, One=1, Two=2, Three=3 )
            testValue = 0
            for value in testObject:
                self.assertEquals( testValue, value, 'The iterated value should match' )
                testValue += 1

        # Test immutability after instantiation
        def testImmutability( self ):
            """Tests the immutability of the members/values of an Enumeration after it's been instantiated."""
            testObject = Enumeration( None, Good=1, Bad=-1, Indifferent=0 )
            try:
                testObject.Ugly = 2
                self.fail( 'Once instantiated, the members of an Enumeration should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Once instantiated, modification of the members of an Enumeration should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__ ) )
            try:
                testObject.Good = 2
                self.fail( 'Once instantiated, the members of an Enumeration should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Once instantiated, modification of the members of an Enumeration should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__ ) )

        # Test dot-notation access
        def testDotAccess( self ):
            """Tests dot-notation access to member values by name."""
            testObject = Enumeration( None, Good=1, Bad=-1, Indifferent=0 )
            self.assertTrue( hasattr( testObject, 'Good' ), 'Member names should be accessible as attributes of the Enumeration instance.' )
            self.assertTrue( hasattr( testObject, 'Bad' ), 'Member names should be accessible as attributes of the Enumeration instance.' )
            self.assertTrue( hasattr( testObject, 'Indifferent' ), 'Member names should be accessible as attributes of the Enumeration instance.' )
            self.assertEquals( testObject.Good, 1, 'Member values accessed through dot-notation should match the values supplied at creation.' )
            self.assertEquals( testObject.Bad, -1, 'Member values accessed through dot-notation should match the values supplied at creation.' )
            self.assertEquals( testObject.Indifferent, 0, 'Member values accessed through dot-notation should match the values supplied at creation.' )

        # Test reserved-name creation - all should raise errors
        def testReservedNames( self ):
            """Tests that the reserved names (existing attributes) aren't allowed as member-names."""
            names = [ '_key', 'Members', 'MemberNames', 'MemberValues' ]
            for name in names:
                members = { name:True }
                try:
                    testObject = Enumeration( 'docstring', **members )
                    self.fail( 'Creation of an Enumeration with a member-name of %s should raise KeyError' % ( name ) )
                except KeyError:
                    pass
                except Exception, error:
                    self.fail( 'Creation of an Enumeration with a member-name of %s should raise KeyError but %s was raised instead:\n  %s' % ( name, error.__class__.__name__, error ) )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testEnumeration ) )

Line(s)

14-19

While poking around at some proof-of-concept PHP code at work over the last few weeks, it occurred to me that a test-case class could be extended upon to provide at lease some of the typical functionality I find myself using. One of those typical items is the generation of lists of good and bad property-values for various cases. In this case, rather than repeating the "good" cases at least twice (see lines 33, 99), which could lead to the values being tested for those methods getting out of sync and leading to inconsistent testing, I'm calling one method to return all of the good values.

I may very well refactor the UnitTestUtilities module to make use of this concept (and while I'm there, maybe attach the oft-repeated testMethodCountAndTests and testPropertyCountAndTests methods so that I'll have a unit-test class that just does that automatically.

31-48

Apart from the fact that it makes use of the good-values method noted above, this is pretty typical for construction-testing. Note that the test-method tests both good and bad values.

70-83, 87-90

These test-methods probably look a bit odd, since they are required, but they have no useful implementation. The useful tests all exist elsewhere, so the required test-methods simply note that this is the case as a comment, and pass.

93-105

This marks the first test-method of this set that tests an overall characteristic of the class - in this case, it's iterability. The method tests both whether the in keyword works as expected/desired (line 100) for each member, and the "overall" iterability of instances (102-105).

108-124

This method tests that instances are immutable after creation (another "characteristic"-based test-method).

127-135

This method tests that member-names specified at construction are available as attributes of the instance by name, and that the values behind those attribute-names match what was supplied.

138-149

This test-method was almost missed... I realized just before I was first scheduling this for publication on the blog that I hadn't tested whether using any of the "reserved" names at construction would raise the errors I wanted. I could've gone back and added these tests to the constructor tests instead of making a new method specifically for them, but since it's kinda critical that class members not get destroyed, it seemed apropos to call the tests out in their own method.

And that wraps up the Enumeration...

Where to go from here?

With MySQLConnector (and the supporting development that led up to it) complete, I'm at something of a loss as to where to go from here. My daily routine hasn't settled down yet, and though I'd hoped to be able to get some serious code-blogging time in over the week off from work between Christmas and New Year's, that hasn't happened. One complete computer rebuild, which pointed out a critical flaw in my home-system backup-plan, correcting that and trying like hell to back things up before the main drive on the old machine died completely. The random chaos of the holiday season. The demands of family while I'm home and they're awake. The list goes on and on. I did at least manage to get the data-connectors module (up through MySQLConnector) completed, which I'd promised myself I'd get done.

I have a list of possibles to pick from, some of which are dependent on other items on that list.

But the simple fact of the matter is that I'm kinda stuck here. If any readers have a preference what I should tackle next, please comment and let me know. Otherwise, I'll poke around a bit and see what, if anything, strikes my fancy (though I have to say that I'm leaning towards the MiniAppServer idea at present).