The DataConnectors module (part 6)

So, the last generic-but-concrete class that I was working on, you may remember, was the Query class. It was stalled for a time because I wasn't sure how I really wanted the process of executing a query against a database to work. My primary concern was which object was going to be responsible for what part of the result-acquisition.

In retrospect, this feels to me now like the entire question is tied to making a decision about where the line should be drawn for the logical encapsulation-points for the relevant classes, and to some extent, it's also partly a question of how tightly coupled the classes themselves are. Actually, there are several principles that I'm trying to find a good balance between for this part of the object-stack, so I'll list them out, and examine different scenarios against them:

Encapsulate what varies

"When designing software, look for the portions most likely to change and prepare them for future expansion by shielding the rest of the program from that change. Hide the potential variation behind an interface. Then, when the implementation changes, software written to the interface doesn't need to change. This is called encapsulating variation." (from blogs.msdn.com/b/steverowe). There may be other methods than hiding things behind an interface, but that's likely the most common (and best?) practice.

Program to an interface, not to an implementation

Any two objects that conform to the same interface should be functionally interchangeable, so that other objects that care only about that interface then don't have to know or care about the implementation underneath it.

The Single Responsibility Principle

Any given object should have a single responsibility, that responsibility should be entirely encapsulated within the class, and it's properties and methods should be narrowly aligned with that responsibility.

I've listed these in the order of priority that I think makes the most sense for this particular problem, my argument being that of all of these, the Single Responsibility Principle is the most flexible and I'm all but certain that encapsulation of what varies the least flexible. Programming to an interface rather than to an implementation will, I think, be implicit with any reasonable design for the problem, but should be kept in mind as a higher priority than the class' responsibility.

Scenario 1: Query handles everything (almost)

The first scenario centers around the concept that the Query is responsible for the storage of the SQL to be executed, the execution of that SQL against the data-source, and the handling and storage of results returned from that execution. In this scenario, the Query must have (or be provided) a data-source object (derived from BaseDataConnector) which must have a method to execute an arbitrary SQL string against the database it represents, and that returns results to the Query. In keeping with the idea that the Query holds as much relevant responsibility as possible, those results would probably not be in ResultSet/Record instances.

Encapsulating what varies

Since the most likely point of variation is the specific mechanism that executes a SQL string against a data-source, and what structure/format those results come back in, encapsulating that in the BaseDataConnector-derived data-source feels pretty good.

Programming to an interface, not to an implementation

This scnario also feels pretty good in this respect, since Query doesn't need to know (or care) what kind of BaseDataConnector it's data-source is, just that it is a BaseDataConnector.

The Single Responsibility Principle

Assuming that the BaseDataConnector returns the simplest possible structure/format of it's results (most likely a list of dictionary-instances), and that Query is responsible for taking that raw result-set and converting them into ResultSet/Record structures, the responsibilities for each class might be stated something like so:

BaseDataConnector

Manages internal and/or low-level connection to and interaction with a back-end data-source.

Query

Provides user/developer-level mechanisms to interact with a back-end database.

Other Considerations

I'm pretty sure that I noted, some few posts ago, that for performance reasons it would be desirable to be able to queue up queries in order to minimize the number of connections made against the back-end database. This approach doesn't prevent that sort of capability, but I suspect that it would be more difficult to implement. It would almost certainly require a change to the responsibility of the BaseDataConnector-derived classes, if only to provide some mechanism to be able to associate an arbitary number of ResultSet instances with a particular Query object, but since creation of a ResultSet and it's associated Record instances to indicate the beginning of a query-association can be done in code, not as part of the "real" data-access, this may not be too onerous.

An example of code using this structure would look something like this:

# Create the data-source, where DataConnector is derived from BaseDataConnector
dataSource = DataConnector( host='host_name', database='database_name', user='user_name', password='password' )
# Create the Query instance, with SQL that returns two result-sets
sql = """SELECT somefield 
    FROM sometable 
    ORDER BY somefield;
SELECT someotherfield
    FROM sometable
    ORDER BY someotherfield;"""
myQuery = Query( dataSource, sql )
someResults, someOtherResults = myQuery.Results

On reflection, the idea of queueing up multiple queries is, perhaps, not as critical as I'd thought. Consider that this example returns multiple result-sets, and is managed in a single Query instance. The performance concerns about multiple database-connections could be mitigated easily enough by taking this sort of approach, provided that the number of result-sets returned by any given chunk of SQL were known and didn't change based on input parameters. This approach is feeling better and better to me the more I think on it.

Scenario 2: BaseDataConnector handles everything (almost)

This scenario's concept is diametrically opposed from the first: Here, the BaseDataConnector-derived objects handle as much of the interaction as possible, while Query becomes little more than a dumb data-object, containing the SQL to be executed, if it exists at all.

Encapsulating what varies

The encapsulation of implementation-variations is not significantly different here from the first scenario - it's still centered around the potential differences between different database-engines, how their connection and query-execution works, and what the results-structure looks like.

Programming to an interface, not to an implementation

Again, not significantly different from the first scenario - the specifics of what the interface (at the BaseDataConnector level) provides change, but we're still keeping the implementation hidden from the rest of the world.

The Single Responsibility Principle

This scenario does make drastic changes to the responsibilities of the objects, though. Fundamentally, there's no real need for any Query objects at all, since the actual SQL could just be passed to a Query method in a data-source object, and the results retrieved directly from there. If we assume that there's no need to have a Query object at all, the responsibilities break down to:

BaseDataConnector

Manages all interaction with a back-end data-aource, including execution of queries against it.

Other Considerations

This scenario, assuming that there is no Query object in the mix at all, pretty much prohibits any sort of queueing of queries. If we assume that a single SQL execution can be unpacked to a number of result-set variables in a manner similar to the example code for the first scenario, maybe this consideration is moot. The only thing we lose in this scenario is the ability to have state-data associated with the SQL of the query.

An example of code using this structure would look something like this:

# Create the data-source, where DataConnector is derived from BaseDataConnector
dataSource = DataConnector( host='host_name', database='database_name', user='user_name', password='password' )
# execute a query against the data-source
sql = """SELECT somefield 
    FROM sometable 
    ORDER BY somefield;
SELECT someotherfield
    FROM sometable
    ORDER BY someotherfield;"""
someResults, someOtherResults = dataSource.Query( sql )

I'm going to stop with these two scenarios, at least for now, because the desire for state-data/SQL association is a point woth more detailed consideration. When I originally conceived the DataConnectors module, it was, admittedly, with a specific eye towards a structure more along the lines of the structure that I'd gotten accustomed to as a CFML programmer. That included some query- or result-set-specific properties being available, including:

1. Data-source type (through a data-source association);
2. Authentication and authorization (also through a data-source association, but easily converted into a host/database/user/password structure);
3. Timeout control (though I don't know that I ever used this functionality);
4. Caching control;
5. Association of results to variables within the code;
6. Execution-time information; and
7. Paging of results;

Of these, the first two can be converted into object-constructor arguments easily enough, by passing the authentication/authorization values as simple values, and the data-source type as a class derived from BaseDataConnector. They don't require any sort of state-data at a Query-object level at all. Timeout control, if it's even something that can be usefully implemented, doesn't feel like it would require any local state-data.

Caching control might, though, depending on how that caching were implemented. If it's implemented outside the database (say, by picking a result-set and writing it to a file), the query would potentially need to be able to keep track of how long the cached data-sets were valid. Since data-source-side caching may or may not be available for any given database-engine, that's a point of variation that should be encapsulated somewhere (though where is still an open question).

So long as we assume that the unpacking of results (as shown in the previous code-examples) will work, the association of results to variables is moot.

Execution-time information would require state-data somewhere, and the Query object seems like the only place that makes sense. It can't be usefully associated as a ResultSet level, since the number of result-sets from a given query will vary - particularly if multiple result-sets are expected with any frequency.

Paging of results is probably something that makes more sense at the ResultSet level, so it can be safely disregarded for the purpose of discussion of BaseDataConnector and Query responsibilities.

With all of this in mind, my decision is to make the Query object as smart as possible, and the BaseDataConnector-derived objects pretty simple (though not completely dumb). I believe that this will afford the best development experience in the long run, and it keeps the possibility of query-state-dependent capabilities open on a longer-term basis, should there be a need for them somewhere down the line. With all of that in mind, then, the IsDataConnector interface loses it's Query method in favor of an Execute method, with changes made accordingly to it's unit-tests:

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'query', None, 'The IsQuery object that will be executed against the object\'s Connection.' )
    def Execute( self, query ):
        """Executes the provided IsQuery object's query against the object's connection, returning one or more results."""
        raise NotImplementedError( '%s.Query is not implemented as required by IsDataConnector.' % ( self.__class__.__name__ ) )

        def testExecute( self ):
            """Unit-tests the Execute method of the IsDataConnector interface."""
            testObject = IsDataConnectorDerived()
            try:
                testObject.Execute( None )
                self.fail( 'Execute should raise a NotImplementedError.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Execute should raise a NotImplementedError, but %s was raised instead:\n  %s' % ( error.__class__.__mame__, error ) )

The BaseDataConnector inherits that abstract Execute method, so it's unit-tests need to be updated accordingly:

        def testExecute( self ):
            """Unit-tests the Execute method of the BaseDataConnector abstract class."""
            self.assertEquals( BaseDataConnector.Execute, IsDataConnector.Execute, 'BaseDataConnector should not define the Execute method.' )

...and now Query can be defined. There are some commented-out properties and methods in this code that I've left in place for later, mostly centering around caching of query results in the future.

Query (Nominal final class):

class Query( IsQuery, object ):
    """Represents a query (and it's results) against a back-end data-source."""

    ##################################
    # Class Attributes               #
    ##################################

    ##################################
    # Class Property-Getter Methods  #
    ##################################

#    def _GetCacheExpiresAt( self ):
#        """Gets the time at which the query's cached results will expire."""
#        return self._cacheExpiresAt

    def _GetDatasource( self ):
        """Gets the Datasource used to execute the query's SQL."""
        return self._datasource

    def _GetResults( self ):
        """Gets the results from the execution of the query's SQL against the data-source."""
        if not self._results:
            if self._sql:
                self.__Execute()
        return self._results

    def _GetSql( self ):
        """Gets the SQL to be executed against the data-source to generate the results."""
        return self._sql

    ##################################
    # Class Property-Setter Methods  #
    ##################################

#    def _SetCacheExpiresAt( self, value ):
#        """Sets the time at which the query's cached results will expire."""
#        self.self._cacheExpiresAt = value

    def _SetDatasource( self, value ):
        """Sets the Datasource used to execute the query's SQL."""
        if not isinstance( value, IsDataConnector ):
            raise TypeError( '%s.Datasource expects an instance of IsDataConnector.' % ( self.__class__.__name__ ) )
        if self._results != None:
            raise AttributeError( '%s.Datasource cannot be reset once results have been retrieved.' % ( self.__class__.__name__ ) )
        self._datasource = value

    def _SetResults( self, value ):
        """Sets the results from the execution of the query's SQL against the data-source."""
        self._results = value

    def _SetSql( self, value ):
        """Sets the SQL to be executed against the data-source to generate the results."""
        if type( value ) != types.StringType:
            raise TypeError( '%s.Sql expects a non-empty string value.' % ( self.__class__.__name__ ) )
        if value == '':
            raise ValueError( '%s.Sql expects a non-empty string value.' % ( self.__class__.__name__ ) )
        if self._results != None:
            raise AttributeError( '%s.Sql cannot be reset once results have been retrieved.' % ( self.__class__.__name__ ) )
        self._sql = value

    ##################################
    # Class Property-Deleter Methods #
    ##################################

#    def _DelCacheExpiresAt( self ):
#        """Deletes the time at which the query's cached results will expire - uncaches the results in the process."""
#        self._cacheExpiresAt = 0

    def _DelDatasource( self ):
        """Deletes the Datasource used to execute the query's SQL."""
        self._datasource = None

    def _DelResults( self ):
        """Deletes the results from the execution of the query's SQL against the data-source."""
        self._results = None

    def _DelSql( self ):
        """Deletes the SQL to be executed against the data-source to generate the results."""
        self._sql = None

    ##################################
    # Class Properties               #
    ##################################

#    CacheExpiresAt = property( _GetCacheExpiresAt, _SetCacheExpiresAt, None, IsQuery.CacheExpiresAt.__doc__ )
    Datasource = property( _GetDatasource, _SetDatasource, None, IsQuery.Datasource.__doc__ )
    Results = property( _GetResults, None, None, IsQuery.Results.__doc__ )
    Sql = property( _GetSql, _SetSql, None, IsQuery.Sql.__doc__ )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'datasource', None, 'The Datasource used to execute the query\'s SQL.' )
    @DocumentArgument( 'argument', 'sql', None, 'The SQL to be executed against the data-source to generate the results.' )
#    @DocumentArgument( 'keyword', 'parameters', 'cacheFor', '(Non-negative integer, optional) The number of seconds that the Query\'s results should be cached for.' )
#    @DocumentArgument( 'keyword', 'parameters', 'cacheUntil', '(Time, optional) A time that the Query\'s results will be cached until.' )
#    def __init__( self, datasource, sql, **parameters ):
    def __init__( self, datasource, sql ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != Query:
            raise NotImplementedError( 'Query is (nominally) a final class, and is not intended to be derived from.' )
        # Set default values:
        self._DelDatasource()
        self._DelResults()
        self._DelSql()
        # Set properties from arguments
        self._SetDatasource( datasource )
        self._SetSql( sql )

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

#    def Cache( self ):
#        """Caches the results of the Query in a manner yet to be determined."""
#        raise NotImplementedError( 'Query.Cache has not been implemented yet.' )

    def __Execute( self ):
        """Executes the query's SQL against it's specified Datasource, retrieves the results, and builds out a list of ResultSet objects in the Query's Results."""
        self._results = TypedList( [ ResultSet ] )
        rawResults = self.Datasource.Execute( self )
        for theResult in rawResults:
            self._results.append( ResultSet( theResult ) )

#    def Uncache( self ):
#        """Uncaches the results of the Query."""
#        raise NotImplementedError( 'Query.Uncache has not been implemented yet.' )

__all__ += [ 'Query' ]

Given my typical patterns, and assuming that you've been following along for any length of time, there's not a whole lot out of the ordinary here. The Datasource and Sql properties are the exception, in that they allow modification until the query's results are retrieved.

Similarly, with the exception of the FixedResultsConnector class, the unit-tests are pretty typical, though again, they are explicitly testing the mutability of the Datasource and Sql properties before and after results are acquired. The Results property tests may feel a bit on the sparse side also, but bear in mind that all we care about at this level is that we get back a TypedList of ResultSet instances - ResultSet is already tested, so that's all we need to know...

    class QueryDerived( Query ):
        def __init__( self, datasource, sql ):
            Query.__init__( self, datasource, sql )
    
    class FixedResultsConnector( BaseDataConnector ):
        def __init__( self, **parameters ):
            BaseDataConnector.__init__( self, **parameters )
        def Execute( self, query ):
            return [
                    # First result-set
                    [
                        {'results1row':1, 'results1field1':'results1row1field1', 'results1field2':'results1row1field2' },
                        {'results1row':2, 'results1field1':'results1row2field1', 'results1field2':'results1row2field2' },
                    ],
                    # Second result-set
                    [
                        {'results2row':1, 'results2field1':'results2row1field1'},
                    ],
                ]
        def ResultsCount( self ):
            return length( self.Execute( self, '' ) )
        def ResultsFields( self ):
            results = self.Execute( self, '' )
            keySets = []
            for theResult in results:
                keysets.append( theResult[0].keys() )
            return keySets
            

    class testQuery( unittest.TestCase ):
        """Unit-tests the Query class."""
    
        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass
        
        def testFinal( self ):
            """Testing final nature of the Query class."""
            testDataSource = FixedResultsConnector()
            testSql = 'SELECT * FROM sometable;'
            try:
                testObject = QueryDerived( testDataSource, testSql )
                self.fail( 'Query is nominally a final class, and should not be instantiable.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Attempting to instantiate an object derived from Query should raise NotImplemetedError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            testObject = Query( testDataSource, testSql )
            self.assertTrue( isinstance( testObject, Query ), 'Creating a Query instance should result in a Query instance.' )
    
        def testConstruction( self ):
            """Testing construction of the Query class."""
            pass
    
        def testPropertyCountAndTests( self ):
            """Testing the properties of the Query class."""
            items = getMemberNames( Query )[0]
            actual = len( items )
            expected = 3
            self.assertEquals( expected, actual, 'Query is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the Query class."""
            items = getMemberNames( Query )[1]
            actual = len( items )
            expected = 0
            self.assertEquals( expected, actual, 'Query is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Test properties

        def testDatasource( self ):
            """Tests the Datasource property of the Query class."""
            testDataSource = FixedResultsConnector()
            testOtherDataSource = FixedResultsConnector()
            testSql = 'SELECT * FROM sometable;'
            testObject = Query( testDataSource, testSql )
            self.assertEquals( testObject.Datasource, testDataSource, 'The Datasource property should be set and gettable after creation.' )
            testObject.Datasource = testOtherDataSource
            self.assertEquals( testObject.Datasource, testOtherDataSource, 'The Datasource property should be settabe and gettable after creation if results haven\'t been retrieved.' )
            testResults = testObject.Results
            try:
                testObject.Datasource = testDataSource
                self.fail( 'A Query\'s Datasource property should not be modifiable after results have been retrieved' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to change a Query\'s Datasource should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        def testResults( self ):
            """Tests the Results property of the Query class."""
            testDataSource = FixedResultsConnector()
            testSql = 'SELECT * FROM sometable;'
            testObject = Query( testDataSource, testSql )
            queryResults1, queryResults2 = testObject.Results
            resultSet1, resultSet2 = testDataSource.Execute( '' )
            self.assertEquals( resultSet1[0].keys(), queryResults1.FieldNames )
            self.assertEquals( len( resultSet1 ), len( queryResults1 ) )
            self.assertEquals( resultSet2[0].keys(), queryResults2.FieldNames )
            self.assertEquals( len( resultSet2 ), len( queryResults2 ) )

        def testSql( self ):
            """Tests the Sql property of the Query class."""
            testDataSource = FixedResultsConnector()
            testSql = 'SELECT * FROM sometable;'
            testOtherSql = 'SELECT * FROM someothertable;'
            testObject = Query( testDataSource, testSql )
            self.assertEquals( testObject.Sql, testSql, 'The Sql property should be set and gettable after creation.' )
            testObject.Sql = testOtherSql
            self.assertEquals( testObject.Sql, testOtherSql, 'The Sql property should be settabe and gettable after creation if results haven\'t been retrieved.' )
            testResults = testObject.Results
            try:
                testObject.Sql = testSql
                self.fail( 'A Query\'s Sql property should not be modifiable after results have been retrieved' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to change a Query\'s Sql should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testQuery ) )

The DataConnectors module (part 5)

Looking at the requirements (both functional and logical) for ResultSet, most of them are already accounted for simply by making ResultSet derive from TypedList, and hard-setting the member-types to Record. Consider:

• A result-set is a sequence of records - Check;
• A result-set can be iterated over, and the fields of any individual row or record can be accessed using a syntax along the lines of result[ i ].FieldName, or within an iteration over results, results.FieldName - Check;
• The individual rows, once populated, are immutable - Check;

But there are other requirements, more based on the logical expectation(s) of a record or row from a query:

• Each row or record of a result-set should have the same structure (e.g., the same field-names as attributes), even if they have a null or None value;
• Once populated, the sequence of rows cannot be altered, including adding new rows, removing existing ones, or changing any row-values, either singly or as part of a slice;

These requirements haven't been dealt with yet, since they reside at the ResultSet level of the stack (and they may carry back up to the IsResultSet nominal interface as well). But given the functionality we know of in TypedList, and the object-locking-pattern that was used in Record, the basic mechanisms aren't difficult to implement. Since there will be overrides of many (all?) of the TypedList methods in order to check for lock-state on the record-set object, as well as to check the field-structure of the records, each of these methods has to be tested at the ResultSet level, just as they were for TypedList, though... And if you recall, there was a substantial amount of testing that had to happen there, for all of the various list-behaviors...

ResultSet (Nominal final class):

class ResultSet( IsResultSet, TypedList, object ):
    """Class doc-string"""

    ##################################
    # Class Attributes               #
    ##################################

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    def _GetFieldNames( self ):
        return self.__fieldNames

    def _GetRowCount( self ):
        return len( self )

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    ##################################
    # Class Properties               #
    ##################################

    FieldNames = property( _GetFieldNames, None, None, IsResultSet.FieldNames.__doc__ )
    RowCount = property( _GetRowCount, None, None, IsResultSet.RowCount.__doc__ )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'results', None, '(iterable collection of dictionaries) The sequence of rows to populate in the result-set.' )
    def __init__( self, results ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != ResultSet:
            raise NotImplementedError( 'ResultSet is (nominally) a final class, and is not intended to be derived from.' )
        TypedList.__init__( self, [ Record ] )
        self.__fieldNames = []
        self.__resultsLocked = False
        for item in results:
            if not isinstance( item, Record ):
                item = Record( **item )
            if self.__fieldNames == []:
                self.__fieldNames = item.FieldNames
            else:
                if self.__fieldNames != item.FieldNames:
                    raise AttributeError( '%s Error: Expected a row with %s fields, but a row with %s fields was present instead' % ( self.__class__.__name__, self.__fieldNames, item.FieldNames ) )
            self.append( item )
        self.__resultsLocked = True

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

    ##################################
    # TypedList Methods overridden   #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to add to the current list.' )
    def __add__( self, items ):
        """Called when adding items to the list with a "+" (e.g., TL + items)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __add__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.Keys != self.__fieldNames:
                raise ValueError( '%s __add__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__add__( self, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'item', None, '(List or derived, required) The list of item to add to the current list.' )
    def __delitem__( self, item ):
        """Called when deleting an item from the list."""
        if self.__resultsLocked:
            raise AttributeError( '%s __delitem__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        TypedList.__delitem__( self, item )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to append to the current list.' )
    def __iadd__( self, items ):
        """Called when adding items to the list with a "+=" (e.g., TL += item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __iadd__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.Keys != self.__fieldNames:
                raise ValueError( '%s __iadd__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__iadd__( self, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'index', None, '(Integer, required) The index-location to set the item in.' )
    @DocumentArgument( 'argument', 'item', None, '(Any, required) The item to set in the specified list-location.' )
    def __setitem__( self, index, item ):
        """Called when setting the value of a specified location in the list (TL[n] = item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __setitem__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        if item.FieldNames != self.__fieldNames:
            raise ValueError( '%s __setitem__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__setitem__( self, index, item )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'start', None, '(Integer, required) The starting position of the slice to set the supplied items in.' )
    @DocumentArgument( 'argument', 'end', None, '(Integer, required) The ending position of the slice to set the supplied items in.' )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to set in the specified slice-location.' )
    def __setslice__( self, start, end, items ):
        """Called when setting the value(s) of a slice-location in the list (TL[1:2] = item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __setslice__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.FieldNames != self.__fieldNames:
                raise ValueError( '%s __setslice__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__setslice__( self, start, end, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'item', None, '(Any, required) The item to append.' )
    def append( self, item ):
        """Called when appending items to the list."""
        if self.__resultsLocked:
            raise AttributeError( '%s error: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        if item.FieldNames != self.__fieldNames:
            raise ValueError( '%s append: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.append( self, item )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to extend to the current list.' )
    def extend( self, items ):
        """Called when extending items to a the end of the list."""
        if self.__resultsLocked:
            raise AttributeError( '%s error: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.FieldNames != self.__fieldNames:
                raise ValueError( '%s error: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.extend( self, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'index', None, '(Integer, required) The index-location to insert the item in.' )
    @DocumentArgument( 'argument', 'item', None, '(Any, required) The item to insert in the specified list-location.' )
    def insert( self, index, item ):
        """Called when inserting items to a specified location in the list (TL[1] = item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s error: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        if item.FieldNames != self.__fieldNames:
            raise ValueError( '%s append: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.insert( self, index, item )

__all__ += [ 'ResultSet' ]

Line(s)

12-16, 30-31

Standard read-only property set-up...

37-56

Object constructor:

42-43

Enforcing a nominally-final class for the object.

44

Making sure that TypedList functionality is available for instances.

45-46

Setting instance-values for the two new attributes of the class.

47-55

Populating the instance with the results provided:

48-49

The class should, ideally, be able to handle results as either an iterable of Record instances, or an iterable of dictionary instances - if the item is a dictionary, it will need to be made into a Record instance.

I feel like I've neglected type-checking the item at this point, making sure that it's a dictionary, but I don't know that it would really make a difference. Instantiation of a Record with anything other than a dictionary or a keyword argument list raises an error on it's own, so I don't think explicitly checking makes enough of a difference to implement that. At least not yet.

50-54

This is where the ResultSet's field-names are set, on the first pass through the loop, and where the structural consistency of subsequent records is checked.

55

If no errors were raised by the time that we reach this point, then the item is appended to the underlying list.

56

Locks the object, preventing futher modification.

70-158

The various built-ins and methods here all follow more or less the same pattern:

• Check to see if the object is locked, and raise AttributeError if it is;
• Check for structural consistency with the established field-names for the item or items supplied;
• If no errors are raised, call the parent method from TypedList to perform the actual operation.

Ultimately, they're all just checking to see if the operation is allowed before performing it - nothing too fancy, though some cases may look a bit more convoluted than is perhaps typical.

82-88

Since the records in a ResultSet should be immutable, deletion has to be prevented as well.

This is something that I overlooked in Record, so I'll need to get that in place there as well.

Unit-tests

class ResultSetDerived( ResultSet ):
        def __init__( self, results ):
            ResultSet.__init__( self, results )
    
    class testResultSet( unittest.TestCase ):
        """Unit-tests the ResultSet class."""

        def getBadRecord( self ):
            return { 'Number1':3, 'Name':'ook eek', 'IsValid':True }

        def getGoodRecord( self ):
            return { 'Number':3, 'Name':'ook eek', 'IsValid':True }

        def getBadValues( self ):
            return [ 1, 'ook', True, None, {} ]

        def getGoodValues( self ):
            return [
                { 'Number':1, 'Name':'ook', 'IsValid':True },
                { 'Number':2, 'Name':'eek', 'IsValid':False },
                { 'Number':None, 'Name':None, 'IsValid':None },
                ]

        def getInconsistentValues( self ):
            return [
                { 'Number':1, 'Name':'ook', 'IsValid':True },
                { 'Number':1, 'IsValid':True },
                { 'Number':2, 'Name':'eek' },
                { 'Name':'eek', 'IsValid':False },
                { 'Number':None },
                { 'IsValid':None },
                ]

        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass
        
        def testFinal( self ):
            """Testing final nature of the ResultSet class."""
            try:
                testObject = ResultSetDerived( self.getGoodValues() )
                self.fail( 'ResultSet is nominally a final class, and should not be extendable.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Instantiating an object derived from ResultSet should raise NotImplementedError, by %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        def testConstruction( self ):
            """Testing construction of the ResultSet class."""
            testObject = ResultSet( self.getGoodValues() )
            # TODO: Test values...
            # Test inconsistent values - all should raise errors
            inconsistentValues = self.getInconsistentValues()
            # Test creation with an iterable
            try:
                testObject = ResultSet( inconsistentValues )
                self.fail( 'Construction with inconsistent record-structures should raise an error.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Construction with inconsistent record-structures should raise AttributeError, but %s was raised instead:\n  %s.' % ( error.__class__.__name__, error ) )

        def testPropertyCountAndTests( self ):
            """Testing the properties of the ResultSet class."""
            items = getMemberNames( ResultSet )[0]
            actual = len( items )
            expected = 3
            self.assertEquals( expected, actual, 'ResultSet is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the ResultSet class."""
            items = getMemberNames( ResultSet )[1]
            actual = len( items )
            expected = 4
            self.assertEquals( expected, actual, 'ResultSet is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Test list behavior
        def testListBehavior( self ):
            """Unit-tests list-like behavior of ResultSet."""
            # Test good values - all should raise errors
            goodValues = self.getGoodValues()
            testObject = ResultSet( goodValues )
            validRecord = Record( **self.getGoodRecord() )
            # Test L = L + i structure
            try:
                testObject + [ validRecord ]
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L = L += i structure
            try:
                testObject += [ validRecord ]
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L.append equivalent
            try:
                testObject.append( validRecord )
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L.extend equivalent
            try:
                testObject.extend( [ validRecord ] )
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L.insert equivalent
            try:
                testObject.insert( 0, validRecord )
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L[index] = value equivalent
            try:
                testObject[ 0 ] = validRecord
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L[slice] = [ i, j ] equivalent
            try:
                testObject[ 0:1 ] = validRecord
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test del L[i]
            try:
                del testObject[ 0 ]
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        # Test Properties

        def testMemberTypes( self ):
            """Unit-tests the MemberTypes property of the ResultSet class."""
            self.assertEquals( TypedList.MemberTypes, BaseTypedCollection.MemberTypes, 'TypedList should inherit the MemberTypes property of BaseTypedCollection.' )

        def testFieldNames( self ):
            """Unit-tests the FieldNames property of the ResultSet class."""
            goodValues = self.getGoodValues()
            testObject = ResultSet( goodValues )
            expected = Record( **goodValues[ 0 ] ).FieldNames
            self.assertEquals( testObject.FieldNames, expected, 'The field-names of a ResultSet should be the same as the field-names of it\'s Record-instances.' )

        def testRowCount( self ):
            """Unit-tests the RowCount property of the ResultSet class."""
            goodValues = self.getGoodValues()
            testObject = ResultSet( goodValues )
            self.assertEquals( testObject.RowCount, len(goodValues), 'The number of rows in RowCount should be the same as the number of items presented at construction.' )

        # Test Methods

        def testappend( self ):
            """Unit-tests the append method of the ResultSet class."""
            pass # This is tested in testListBehavior, above.

        def testextend( self ):
            """Unit-tests the extend method of the ResultSet class."""
            pass

        def testinsert( self ):
            """Unit-tests the insert method of the ResultSet class."""
            pass # This is tested in testListBehavior, above.

        def testIsValidMemberType( self ):
            """Unit-tests the IsValidMemberType method of the ResultSet class."""
            self.assertEquals( TypedList.IsValidMemberType, BaseTypedCollection.IsValidMemberType, 'TypedList should inherit the IsValidMemberType method of BaseTypedCollection.' )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testResultSet ) )

I'm going to skip commenting on the unit-tests this time around - they are all either very straightforward (I believe), or repeats of previous test-structures. The only new thing here, really, is the use of a couple of methods (getBadRecord, getGoodRecord, getGoodValues, and getInconsistentValues) used to generate results for creation of RecordSet instances for the various test-methods.

The unit-tests for RecordSet also revealed a flaw in Record: I'd forgotten to initialize a local __fieldNames attribute, and as a result, the value being set in that property for the first instance was shared across all instances (it was behaving as a static attribute). The changes to Record and it's unit-tests are:

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'keyword', 'properties', 'variable', 'Key/value pairs providing the record\'s field-names and values of those fields.' )
    def __init__( self, **properties ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != Record:
            raise NotImplementedError( 'Record is (nominally) a final class, and is not intended to be derived from.' )
        self.__fieldNames = []
        for theKey in properties:
            self.__setattr__( theKey, properties[ theKey ] )
            if not theKey in self.__fieldNames:
                self.__fieldNames.append( theKey )
        self.__recordIsLocked = True

# break

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'name', None, '(String, required) The name of the attribute to be set.' )
    @DocumentArgument( 'argument', 'value', None, '(Any type, required) The value to be set in the named attribute.' )
    def __setattr__( self, name, value ):
        """Object-attribute setter."""
        if name == '_Record__recordIsLocked' and self.__recordIsLocked != False:
            raise AttributeError( 'The record is locked, and cannot be unlocked' )
        if self.__recordIsLocked:
            raise AttributeError( 'The record has been locked, and the "%s" attribute cannot be set.' % ( name ) )
        if hasattr( self, name ) and not name in [ '_Record__recordIsLocked', '_Record__fieldNames' ]:
            raise AttributeError( 'The "%s" attribute already exists, and cannot be reset.' % ( name ) )
        return object.__setattr__( self, name, value )

The changes here are:

Line(s)

8

self.__fieldNames is being explicitly set to an empty list

26

Explicitly checking the property-name against both _Record__recordIsLocked and _Record__fieldNames, in order to allow either to be set if the object isn't locked yet.

This feels like it may not be complete, so I may very well come back to it later...

        def testFieldNames( self ):
            """Unit-tests the FieldNames property of the Record class."""
            testProperties = { 'Property1':1, 'Property2':'ook', 'Property3':True }
            testProperties2 = { 'Property':1 }
            testObject = Record( **testProperties )
            self.assertEquals( testObject.FieldNames, testProperties.keys() )
            try:
                testObject.Keys = 'ook'
                self.fail( 'A Record\'s FieldNames cannot be altered.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Changes to a record\'s FieldNames should raise AttributeError, bu %s was raised instead:\n  %s ' % ( error.__class__.__name__, error ) )
            testObject = Record( **testProperties2 )
            self.assertEquals( testObject.FieldNames, testProperties2.keys(), 'The field-names for each record should be local to the instance' )

The change here is the addition of the last two lines - since each Record instance should have set it's FieldNames property independently, the second testObject should have the same keys as the testProperties2 dictionary. If the attribute were not explicitly set locally, it would have the keys from testProperties instead.

And that's the last of the "simple" concrete classes that weren't waiting on research. At least until the need for another one rears it's head. Next (finally), I'll get back to defining where query-execution takes place - you may remember that I set that aside to do some research in my previous post. I've done that research, and I can move on to implementation now...

The DataConnectors module (part 4)

So, before I start digging into the code for the concrete classes of DataConnectors, I want to do a quick review of how they are all supposed to fit together:

• An instance of BaseDataConnector (which implements IsDataConnector) will provide the basic connectivity to the back-end data-store, as well as the mechanism(s) for executing queries against that data-store;
• Queries are objects implementing IsQuery, that store the SQL to be executed, as well as the results of that execution once it's run in one or more ResultSet objects;
• The ResultSet instances (which implement IsResultSet, and are TypedLists keep track of the fields returned, row-by-row.

There's a piece missing from the most recent class-diagram: how the records are going to be represented... We need to add a Record class to handle that, like so:

Ideally, what I'd prefer from a code-structure standpoint, is to be able to iterate over a ResultSet instance, and use object-/dot-notation for the fields. For example, something like:

for record in results:
    print record.FirstName, record.LastName

Providing that capability is easy, since Python's objects can have dynamically-assigned attributes and values. The downside to taking that very basic approach, though is that those records, and the fields within them, are mutable in the code, and for a database-record that's read-only, that's not a good thing. There is a way to work around that, though, and I'll demonstrate that a bit later.

Another thing that hasn't really been thrashed out yet is when and where query-execution occurs. I've been contemplating two basic approaches, and trying to weight between their merits and drawbacks:

Execution happens at the Query:

In this model, the Query uses it's Datasource to connect to the back-end database, executes the SQL, and generates the result-set.

Advantages

• It feels to me like query-objects preserve the single responsibility principle better this way - really, at least in theory, all they need is a connection-object, supplied by the data-connector, and away they go.
• I suspect that this will allow for easier implementation of "one-off" queries - I don't know how many times in my career a project has crossed my desk where the work needed essentially boiled down to "run another query, and add the results to the display." I believe that optimizing the data-connector approach for this sort of one-off case wouldn't be difficult, but I'm not completely sure about that at this point.
• It feels like this approach would lend itself well (or at least better) to lazy instantiation of query-results: that is, create as many query-objects as even might be necessary, but don't actually get the results from the database until (or even unless) they are specifically requested. At least for basic fetch-and-display processes, this could be very advantageous.

Drawbacks

• There is a strong potential, when (not if) this module is expanded to include back-end databases other than MySQL, that query-objects may need to be customized to each individual database-engine. I don't have a strong feeling at this point for just how likely this really is, but since the logical point of encapsulation for any database-engine-specific capabilities is at the data-connector level, not at the query level, this feels like it would be less than optimal the first time that this kind of database-engine variation happens. It's not future-proof, and could easily require a lot of new work or rework, and I feel that's a major concern.
• If the assumption that one-off queries would be easier to implement in this model holds true, that could tend to reinforce that sort of design/solution in applications. On the surface, this doesn't sound all that bad, but if each one-off query represents some (hopefully small) amount of database-connection time, the more this sort of design is propagated, the more performance could suffer as a result. I'm not a big fan of making things easier in the shorter term at the expense of longer-term stability or maintainability.
• It doesn't feel like this approach would facilitate building up a list of queries, then executing all of them at once - at least not as easily.

Execution happens at the BaseDataConnector:

In this model, the connector (a derivative of BaseDataConnector) connects to the datasource, then runs the query or queries specified, handing the results back to the Query object(s) that supplied the SQL in the first place so that they can keep track of them.

Advantages

• The major advantage is almost certainly that keeping the actual execution of the query encapsulated in the data-connector (which is the place where it might vary) will protect us from the potential need to have to spin off a Query-variant for each and every data-connector-type that we might eventually want. As I noted above, I don't have a strong feeling for how likely this really is, so more research on that topic is probably in order. It should be noted, however, that even when that research is complete, and even if it proves that the connector-types that we're planning aren't going to be affected, that does not mean that they won't change, or that some new, hot-topic database-engine won't come out that would require distinct Query classes...
• A major advantage to this approach is, I think, that queries could be queued up and deferred until the developer knows that the code will need them.

Drawbacks

• There might be significant design changes needed for BaseDataObject-derived objects in order to take advantage of query-queueing capabilities. Or it may not even be feasible to reconcile those objects' data-needs with a queueable query-stack, in which case a mixed model might be necessary (which is more up-front work).

So, it feels like more research is needed, and possibly some rework on the designs of BaseDataConnector, and maybe some minor changes to Query will be needed as well. In the interests of moving this post along (while I do my research in the background), I'm not going to work on Query just yet. That class, and any changes to BaseDataConnector will be the topic of the next post, and I'll attack the two remaining general-purpose concrete classes left: Record and ResultsSet.

Instances of the Record class represent single records/rows returned from a query executed against the database. They are the core member-type for a ResultSet instance (which in turn represent the collections of records from a query). Record-objects should (for the time being) be immutable once instantiated: we don't want to allow changes to a record's data inside an application without having some way to re-synchronize that data with the original record, which is why the BaseDataObject abstract class exists. For similar reasons, ResultSet instances should also be immutable once instantiated. Both are variations of a "dumb" data-object concept (though ResultSet is not completely dumb, since it has all of the functionality of a TypedList): objects whose sole purpose is to represent some data-structure.

Since I'm expecting a Record instance to be generated from a dictionary data-structure, where the dictionary's keys are the column-names and the values are the field-values, the first challenge to overcome is to find a way to pass a dictionary structure into an object at creation time, without having to go through the typical property2=value2, property2=value2 structure that we might expect given the code I've shown so far. These names/values will vary from one record-structure to another, and having to code that structure into a record's constructor would mean having to generate a Record-derived class for every data-type that needs to be represented. That, obviously, is not sustainable. Fortunately, Python provides a mechanism to pass a dictionary as a keyword-argument list. We've already seen the recieving-end structure (see BaseDataConnector.__init__ for an example), though I don't see that I've provided an example of the "sending" structure yet. To illustrate that, here's a quick little script:

class DictProperties( object ):
    """A test-class, created to see how dictionary arguments can be 
pushed into an object's properties."""
    def __init__( self, **properties ):
        """Object constructor."""
        for theKey in properties:
            self.__setattr__( theKey, properties[ theKey ] )

properties = {
    'Prop1':1,
    'Prop2':'ook',
    'Prop3':True,
    }

testObject = DictProperties( **properties )

print 'testObject.Prop1 ... %s' % ( testObject.Prop1 )
print 'testObject.Prop2 ... %s' % ( testObject.Prop2 )
print 'testObject.Prop3 ... %s' % ( testObject.Prop3 )

When this script is run, it prints out:

testObject.Prop1 ... 1
testObject.Prop2 ... ook
testObject.Prop3 ... True

This is what we want: The elements of the dictionary (properties) are being set as "real" attributes in the object-instance. But how does it work? The secret is in lines 4, 6-7, and 15:

Line(s)

4

The **properties argument is a keyword-argument list. Internally, that is a dictionary that can be iterated over in line 6.

6-7

As the properties keyword/dictionary is being iterated over (line 6), we take advantage of a Python built-in: __setattr__. When present, __setattr__ is called when an attribute assignment is attempted, instead of simply storing the value in the object's instance dictionary. Remember this, because it's going to be a key part of the immutability of a Record later on...

15

This is the part that allows an arbitrary dictionary to be passed into the object's constructor (it would work with any other method as well). Instead of supplying Prop1=1, Prop2='ook', Prop3=True as arguments to the constructor, which would tie each instantiation to a known structure, we pass what is essentially a keyword-argument-reference to a dictionary (**properties) instead. The constructor recognizes it as a keyword-arguments structure (which it's expecting), and handles it just like it would if each keyword/value were supplied directly.

So an arbitrary data-structure can be passed to an object during creation. That's the first hurdle passed.

The next hurdle is to make those values immutable after they've been set. Getting that to work takes a bit of modification that's easier to explain by showing the code, so here's a modified copy of the same script above:

class DictProperties( object ):
    """A test-class, created to see how dictionary arguments can be 
pushed into an object's properties."""
    # Reserved attributes
    __objectIsLocked = False
    # Object constructor
    def __init__( self, **properties ):
        """Object constructor."""
        for theKey in properties:
            self.__setattr__( theKey, properties[ theKey ] )
        self.__objectIsLocked = True
    def __setattr__( self, name, value ):
        """Object-attribute setter."""
        if name == '__objectIsLocked' and self.__objectIsLocked:
            raise AttributeError( 'The object is locked, and cannot be unlocked' )
        if self.__objectIsLocked:
            raise AttributeError( 'The object has been locked, and the "%s" attribute cannot be set.' % ( name ) )
        return object.__setattr__( self, name, value )

properties = {
    'Prop1':1,
    'Prop2':'ook',
    'Prop3':True,
    }

testObject = DictProperties( **properties )

print 'testObject.Prop1 ... %s' % ( testObject.Prop1 )
print 'testObject.Prop2 ... %s' % ( testObject.Prop2 )
print 'testObject.Prop3 ... %s' % ( testObject.Prop3 )

try:
    testObject.Prop3 = False
except AttributeError:
    print 'testObject.Prop3 ... Reset failed as expected'
try:
    testObject.Ook = False
except AttributeError:
    print 'testObject.Ook ..... New-attribute set failed as expected'

When this is run, the results are:

testObject.Prop1 ... 1
testObject.Prop2 ... ook
testObject.Prop3 ... True
testObject.Prop3 ... Reset failed as expected
testObject.Ook ..... New-attribute set failed as expected

The magic that makes this work is in the overridce of __setattr__ (lines 12-18). Since __setattr__ is being overridden, the local method gets called instead of the built-in that exists in object. It checks to see if the private __objectIsLocked attribute is being set, and if it's already been set to True (which would lock the object's attributes, as we'll see shortly), and raises an error if this is the case. If the attribute being set is not __objectIsLocked, control passes to the next if on line 16, where a general check of __objectIsLocked is performed. If __objectIsLocked is True, then the object's attributes are locked, and an error is raised, otherwise control passes on, and the results of the base object's __setattr__, with the same arguments, is returned.

That's pretty much all we need, really, for Record:

Record (Nominal final class):

class Record( object ):
    """Represents an immutable record as retrieved from a query against a database."""

    ##################################
    # Class Attributes               #
    ##################################

    __fieldNames = []
    __recordIsLocked = False

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    def _GetFieldNames( self ):
        """Gets the field-names of the Record object created at instantiation."""
        return self.__fieldNames

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    ##################################
    # Class Properties               #
    ##################################

    FieldNames = property( _GetFieldNames, None, None, 'Gets the field-names of the Record object created at instantiation.' )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'keyword', 'properties', 'variable', 'Key/value pairs providing the record\'s field-names and values of those fields.' )
    def __init__( self, **properties ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != Record:
            raise NotImplementedError( 'Record is (nominally) a final class, and is not intended to be derived from.' )
        for theKey in properties:
            self.__setattr__( theKey, properties[ theKey ] )
            if not theKey in self.__fieldNames:
                self.__fieldNames.append( theKey )
        self.__recordIsLocked = True

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'name', None, '(String, required) The name of the attribute to be set.' )
    @DocumentArgument( 'argument', 'value', None, '(Any type, required) The value to be set in the named attribute.' )
    def __setattr__( self, name, value ):
        """Object-attribute setter."""
        if name == '_Record__recordIsLocked' and self.__recordIsLocked != False:
            raise AttributeError( 'The record is locked, and cannot be unlocked' )
        if self.__recordIsLocked:
            raise AttributeError( 'The record has been locked, and the "%s" attribute cannot be set.' % ( name ) )
        if hasattr( self, name ) and name != '_%s__recordIsLocked' % ( self.__class__.__name__ ):
            raise AttributeError( 'The "%s" attribute already exists, and cannot be reset.' % ( name ) )
        return object.__setattr__( self, name, value )

__all__ += [ 'Record' ]

Line(s)

8-9

Since Record is nominally a final class, and cannot be extended, the __keys and __recordIsLocked can be safely made private. They are also defined as class-level attributes so that they will already exist when __setattr__ is defined, helping to protect those attributes from modification if a Record object is created with "__keys" or "__recordIsLocked" fields.

15-17, 31

A typical, though read-only property for the record, providing the field-names of the record.

37-48

The object constructor:

42-43

My standard check to assure that classes extending Record, so long as they call Record.__init__, will raise an error.

44-47

Reads in the keyword arguments, and calls __setattr__ to set the supplied property names and values in the object.

One caveat about this format is that construction of a Record must use a syntax like aRecord = Record( **fields ). If a call like aRecord = Record( fields ) is attempted, a TypeError will be raised, even if the argument is a dictionary.

48

Locks the record, preventing addition or modification of properties in the object.

58-69

Overrides the default __setattr__ provided by the base object to prevent modification of the object's state-data once the object's been locked:

63-64

Allows the record to be locked, but not unlocked, by specifically checking for the private attribute __recordIsLocked, and allowing it to be changed only if it's False.

This could be changed, if extension of Record was needed, by changing the check to look something like this:

if name == '_%s__recordIsLocked' % ( self.__class__.__name__ ) and self.__recordIsLocked != False:
    # do the rest

65-66

Checks to see whether the object is locked, regardless of the attribute-modification being attempted, and raises an error if it is.

67-68

Checks specifically for attempts to re-set an existing attribute. I don't think this would ever come into play, since trying to duplicate key-names in either a dictionary or a keyword-argument list will raise errors, but since I can't categorically rule the possibility out, I checked for it anyway. I cannot think of a way to test it, though...

Unit-tests

    class testRecord( unittest.TestCase ):
        """Unit-tests the Record class."""
    
        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass
        
        def testFinal( self ):
            """Testing final nature of the Record class."""
            try:
                testObject = RecordDerived()
                self.fail( 'Record is nominally a final class, and should not be extendable.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Instantiating an object derived from Record should raise NotImplementedError, by %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
    
        def testConstruction( self ):
            """Testing construction of the Record class."""
            testObject = Record()
            self.assertTrue( isinstance( testObject, Record ), 'Instances of Record should be instances of Record... Duh!' )
            testProperties = { 'Property1':1, 'Property2':'ook', 'Property3':True }
            testObject = Record( **testProperties )
            for property in testProperties:
                testValue = eval( 'testObject.%s' % property )
                self.assertEquals( testValue, testProperties[ property ], '' )
    
        def testPropertyCountAndTests( self ):
            """Testing the properties of the Record class."""
            items = getMemberNames( Record )[0]
            actual = len( items )
            expected = 1
            self.assertEquals( expected, actual, 'Record is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the Record class."""
            items = getMemberNames( Record )[1]
            actual = len( items )
            expected = 0
            self.assertEquals( expected, actual, 'Record is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Test properties

        def testFieldNames( self ):
            """Unit-tests the FieldNames property of the Record class."""
            testProperties = { 'Property1':1, 'Property2':'ook', 'Property3':True }
            testObject = Record( **testProperties )
            self.assertEquals( testObject.FieldNames, testProperties.keys() )
            try:
                testObject.Keys = 'ook'
                self.fail( 'A Record\'s FieldNames cannot be altered.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Changes to a record\'s FieldNames should raise AttributeError, bu %s was raised instead:\n  %s ' % ( error.__class__.__name__, error ) )
                

        def testImmutability( self ):
            """Unit-tests the immutability of records."""
            testProperties = { 'Property1':1, 'Property2':'ook', 'Property3':True }
            testObject = Record( **testProperties )
            try:
                testObject.NewProperty = 'ook'
                self.fail( 'Once created, a record should not allow the creation of a new attribute/field.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Changes to a record\'s data should raise AttributeError, bu %s was raised instead:\n  %s ' % ( error.__class__.__name__, error ) )
            try:
                testObject.Property1 = 'ook'
                self.fail( 'Once created, a record should not allow the alteration of an existing attribute/field.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Changes to a record\'s data should raise AttributeError, bu %s was raised instead:\n  %s ' % ( error.__class__.__name__, error ) )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testRecord ) )

Most of the unit-tests are pretty typical, so there's not a lot to comment on:

Line(s)

25

Note the object-construction argument: **testPropertiesm not testProperties. THis is an example of the syntax noted above for creation of a Record.

64-81

Tests the immutability of a Record:

68-74

Testing that creation of a new attribute on a Record instance isn't allowed.

75-81

Testing that alteration of an existing attribute is not allowed.

Though this is only one of the two concrete classes I said I was going to hit, this post is pretty long, so I'll pick up next time with ResultSet.