Friday, December 16, 2011

The DataConnectors module (part 5)

Looking at the requirements (both functional and logical) for ResultSet, most of them are already accounted for simply by making ResultSet derive from TypedList, and hard-setting the member-types to Record. Consider:

  • A result-set is a sequence of records - Check;
  • A result-set can be iterated over, and the fields of any individual row or record can be accessed using a syntax along the lines of result[ i ].FieldName, or within an iteration over results, results.FieldName - Check;
  • The individual rows, once populated, are immutable - Check;
But there are other requirements, more based on the logical expectation(s) of a record or row from a query:
  • Each row or record of a result-set should have the same structure (e.g., the same field-names as attributes), even if they have a null or None value;
  • Once populated, the sequence of rows cannot be altered, including adding new rows, removing existing ones, or changing any row-values, either singly or as part of a slice;
These requirements haven't been dealt with yet, since they reside at the ResultSet level of the stack (and they may carry back up to the IsResultSet nominal interface as well). But given the functionality we know of in TypedList, and the object-locking-pattern that was used in Record, the basic mechanisms aren't difficult to implement. Since there will be overrides of many (all?) of the TypedList methods in order to check for lock-state on the record-set object, as well as to check the field-structure of the records, each of these methods has to be tested at the ResultSet level, just as they were for TypedList, though... And if you recall, there was a substantial amount of testing that had to happen there, for all of the various list-behaviors...

ResultSet (Nominal final class):

class ResultSet( IsResultSet, TypedList, object ):
    """Class doc-string"""

    ##################################
    # Class Attributes               #
    ##################################

    ##################################
    # Class Property-Getter Methods  #
    ##################################

    def _GetFieldNames( self ):
        return self.__fieldNames

    def _GetRowCount( self ):
        return len( self )

    ##################################
    # Class Property-Setter Methods  #
    ##################################

    ##################################
    # Class Property-Deleter Methods #
    ##################################

    ##################################
    # Class Properties               #
    ##################################

    FieldNames = property( _GetFieldNames, None, None, IsResultSet.FieldNames.__doc__ )
    RowCount = property( _GetRowCount, None, None, IsResultSet.RowCount.__doc__ )

    ##################################
    # Object Constructor             #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'results', None, '(iterable collection of dictionaries) The sequence of rows to populate in the result-set.' )
    def __init__( self, results ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != ResultSet:
            raise NotImplementedError( 'ResultSet is (nominally) a final class, and is not intended to be derived from.' )
        TypedList.__init__( self, [ Record ] )
        self.__fieldNames = []
        self.__resultsLocked = False
        for item in results:
            if not isinstance( item, Record ):
                item = Record( **item )
            if self.__fieldNames == []:
                self.__fieldNames = item.FieldNames
            else:
                if self.__fieldNames != item.FieldNames:
                    raise AttributeError( '%s Error: Expected a row with %s fields, but a row with %s fields was present instead' % ( self.__class__.__name__, self.__fieldNames, item.FieldNames ) )
            self.append( item )
        self.__resultsLocked = True

    ##################################
    # Object Destructor              #
    ##################################

    ##################################
    # Class Methods                  #
    ##################################

    ##################################
    # TypedList Methods overridden   #
    ##################################

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to add to the current list.' )
    def __add__( self, items ):
        """Called when adding items to the list with a "+" (e.g., TL + items)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __add__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.Keys != self.__fieldNames:
                raise ValueError( '%s __add__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__add__( self, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'item', None, '(List or derived, required) The list of item to add to the current list.' )
    def __delitem__( self, item ):
        """Called when deleting an item from the list."""
        if self.__resultsLocked:
            raise AttributeError( '%s __delitem__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        TypedList.__delitem__( self, item )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to append to the current list.' )
    def __iadd__( self, items ):
        """Called when adding items to the list with a "+=" (e.g., TL += item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __iadd__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.Keys != self.__fieldNames:
                raise ValueError( '%s __iadd__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__iadd__( self, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'index', None, '(Integer, required) The index-location to set the item in.' )
    @DocumentArgument( 'argument', 'item', None, '(Any, required) The item to set in the specified list-location.' )
    def __setitem__( self, index, item ):
        """Called when setting the value of a specified location in the list (TL[n] = item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __setitem__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        if item.FieldNames != self.__fieldNames:
            raise ValueError( '%s __setitem__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__setitem__( self, index, item )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'start', None, '(Integer, required) The starting position of the slice to set the supplied items in.' )
    @DocumentArgument( 'argument', 'end', None, '(Integer, required) The ending position of the slice to set the supplied items in.' )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to set in the specified slice-location.' )
    def __setslice__( self, start, end, items ):
        """Called when setting the value(s) of a slice-location in the list (TL[1:2] = item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s __setslice__: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.FieldNames != self.__fieldNames:
                raise ValueError( '%s __setslice__: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.__setslice__( self, start, end, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'item', None, '(Any, required) The item to append.' )
    def append( self, item ):
        """Called when appending items to the list."""
        if self.__resultsLocked:
            raise AttributeError( '%s error: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        if item.FieldNames != self.__fieldNames:
            raise ValueError( '%s append: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.append( self, item )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'items', None, '(List or derived, required) The list of items to extend to the current list.' )
    def extend( self, items ):
        """Called when extending items to a the end of the list."""
        if self.__resultsLocked:
            raise AttributeError( '%s error: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        # check items
        for item in items:
            if item.FieldNames != self.__fieldNames:
                raise ValueError( '%s error: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.extend( self, items )

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'index', None, '(Integer, required) The index-location to insert the item in.' )
    @DocumentArgument( 'argument', 'item', None, '(Any, required) The item to insert in the specified list-location.' )
    def insert( self, index, item ):
        """Called when inserting items to a specified location in the list (TL[1] = item)."""
        if self.__resultsLocked:
            raise AttributeError( '%s error: the %s is locked and cannot be modified' % ( self.__class__.__name__, self.__class__.__name__ ) )
        if item.FieldNames != self.__fieldNames:
            raise ValueError( '%s append: %s is not a Record instance with fields %s' % ( self.__class__.__name__, item, self.__fieldNames ) )
        TypedList.insert( self, index, item )

__all__ += [ 'ResultSet' ]
Line(s)
12-16, 30-31
Standard read-only property set-up...
37-56
Object constructor:
42-43
Enforcing a nominally-final class for the object.
44
Making sure that TypedList functionality is available for instances.
45-46
Setting instance-values for the two new attributes of the class.
47-55
Populating the instance with the results provided:
48-49
The class should, ideally, be able to handle results as either an iterable of Record instances, or an iterable of dictionary instances - if the item is a dictionary, it will need to be made into a Record instance.
I feel like I've neglected type-checking the item at this point, making sure that it's a dictionary, but I don't know that it would really make a difference. Instantiation of a Record with anything other than a dictionary or a keyword argument list raises an error on it's own, so I don't think explicitly checking makes enough of a difference to implement that. At least not yet.
50-54
This is where the ResultSet's field-names are set, on the first pass through the loop, and where the structural consistency of subsequent records is checked.
55
If no errors were raised by the time that we reach this point, then the item is appended to the underlying list.
56
Locks the object, preventing futher modification.
70-158
The various built-ins and methods here all follow more or less the same pattern:
  • Check to see if the object is locked, and raise AttributeError if it is;
  • Check for structural consistency with the established field-names for the item or items supplied;
  • If no errors are raised, call the parent method from TypedList to perform the actual operation.
Ultimately, they're all just checking to see if the operation is allowed before performing it - nothing too fancy, though some cases may look a bit more convoluted than is perhaps typical.
82-88
Since the records in a ResultSet should be immutable, deletion has to be prevented as well.
This is something that I overlooked in Record, so I'll need to get that in place there as well.

Unit-tests

class ResultSetDerived( ResultSet ):
        def __init__( self, results ):
            ResultSet.__init__( self, results )
    
    class testResultSet( unittest.TestCase ):
        """Unit-tests the ResultSet class."""

        def getBadRecord( self ):
            return { 'Number1':3, 'Name':'ook eek', 'IsValid':True }

        def getGoodRecord( self ):
            return { 'Number':3, 'Name':'ook eek', 'IsValid':True }

        def getBadValues( self ):
            return [ 1, 'ook', True, None, {} ]

        def getGoodValues( self ):
            return [
                { 'Number':1, 'Name':'ook', 'IsValid':True },
                { 'Number':2, 'Name':'eek', 'IsValid':False },
                { 'Number':None, 'Name':None, 'IsValid':None },
                ]

        def getInconsistentValues( self ):
            return [
                { 'Number':1, 'Name':'ook', 'IsValid':True },
                { 'Number':1, 'IsValid':True },
                { 'Number':2, 'Name':'eek' },
                { 'Name':'eek', 'IsValid':False },
                { 'Number':None },
                { 'IsValid':None },
                ]

        def setUp( self ):
            pass
    
        def tearDown( self ):
            pass
        
        def testFinal( self ):
            """Testing final nature of the ResultSet class."""
            try:
                testObject = ResultSetDerived( self.getGoodValues() )
                self.fail( 'ResultSet is nominally a final class, and should not be extendable.' )
            except NotImplementedError:
                pass
            except Exception, error:
                self.fail( 'Instantiating an object derived from ResultSet should raise NotImplementedError, by %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        def testConstruction( self ):
            """Testing construction of the ResultSet class."""
            testObject = ResultSet( self.getGoodValues() )
            # TODO: Test values...
            # Test inconsistent values - all should raise errors
            inconsistentValues = self.getInconsistentValues()
            # Test creation with an iterable
            try:
                testObject = ResultSet( inconsistentValues )
                self.fail( 'Construction with inconsistent record-structures should raise an error.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Construction with inconsistent record-structures should raise AttributeError, but %s was raised instead:\n  %s.' % ( error.__class__.__name__, error ) )

        def testPropertyCountAndTests( self ):
            """Testing the properties of the ResultSet class."""
            items = getMemberNames( ResultSet )[0]
            actual = len( items )
            expected = 3
            self.assertEquals( expected, actual, 'ResultSet is expected to have %d properties to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s property (test%s), but none was identifiable.' % ( item, item ) )

        def testMethodCountAndTests( self ):
            """Testing the methods of the ResultSet class."""
            items = getMemberNames( ResultSet )[1]
            actual = len( items )
            expected = 4
            self.assertEquals( expected, actual, 'ResultSet is expected to have %d methods to test, but %d were dicovered by inspection.' % ( expected, actual ) )
            for item in items:
                self.assertTrue( HasTestFor( self, item ), 'There should be a test for the %s method (test%s), but none was identifiable.' % ( item, item ) )

        # Test list behavior
        def testListBehavior( self ):
            """Unit-tests list-like behavior of ResultSet."""
            # Test good values - all should raise errors
            goodValues = self.getGoodValues()
            testObject = ResultSet( goodValues )
            validRecord = Record( **self.getGoodRecord() )
            # Test L = L + i structure
            try:
                testObject + [ validRecord ]
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L = L += i structure
            try:
                testObject += [ validRecord ]
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L.append equivalent
            try:
                testObject.append( validRecord )
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L.extend equivalent
            try:
                testObject.extend( [ validRecord ] )
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L.insert equivalent
            try:
                testObject.insert( 0, validRecord )
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L[index] = value equivalent
            try:
                testObject[ 0 ] = validRecord
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test L[slice] = [ i, j ] equivalent
            try:
                testObject[ 0:1 ] = validRecord
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )
            # Test del L[i]
            try:
                del testObject[ 0 ]
                self.fail( 'Once a ResultSet has been populated, it should be immutable.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Attempting to modify a ResultSet once it\'s been populated should raise AttributeError, but %s was raised instead:\n  %s' % ( error.__class__.__name__, error ) )

        # Test Properties

        def testMemberTypes( self ):
            """Unit-tests the MemberTypes property of the ResultSet class."""
            self.assertEquals( TypedList.MemberTypes, BaseTypedCollection.MemberTypes, 'TypedList should inherit the MemberTypes property of BaseTypedCollection.' )

        def testFieldNames( self ):
            """Unit-tests the FieldNames property of the ResultSet class."""
            goodValues = self.getGoodValues()
            testObject = ResultSet( goodValues )
            expected = Record( **goodValues[ 0 ] ).FieldNames
            self.assertEquals( testObject.FieldNames, expected, 'The field-names of a ResultSet should be the same as the field-names of it\'s Record-instances.' )

        def testRowCount( self ):
            """Unit-tests the RowCount property of the ResultSet class."""
            goodValues = self.getGoodValues()
            testObject = ResultSet( goodValues )
            self.assertEquals( testObject.RowCount, len(goodValues), 'The number of rows in RowCount should be the same as the number of items presented at construction.' )

        # Test Methods

        def testappend( self ):
            """Unit-tests the append method of the ResultSet class."""
            pass # This is tested in testListBehavior, above.

        def testextend( self ):
            """Unit-tests the extend method of the ResultSet class."""
            pass

        def testinsert( self ):
            """Unit-tests the insert method of the ResultSet class."""
            pass # This is tested in testListBehavior, above.

        def testIsValidMemberType( self ):
            """Unit-tests the IsValidMemberType method of the ResultSet class."""
            self.assertEquals( TypedList.IsValidMemberType, BaseTypedCollection.IsValidMemberType, 'TypedList should inherit the IsValidMemberType method of BaseTypedCollection.' )

    testSuite.addTests( unittest.TestLoader().loadTestsFromTestCase( testResultSet ) )

I'm going to skip commenting on the unit-tests this time around - they are all either very straightforward (I believe), or repeats of previous test-structures. The only new thing here, really, is the use of a couple of methods (getBadRecord, getGoodRecord, getGoodValues, and getInconsistentValues) used to generate results for creation of RecordSet instances for the various test-methods.

The unit-tests for RecordSet also revealed a flaw in Record: I'd forgotten to initialize a local __fieldNames attribute, and as a result, the value being set in that property for the first instance was shared across all instances (it was behaving as a static attribute). The changes to Record and it's unit-tests are:

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'keyword', 'properties', 'variable', 'Key/value pairs providing the record\'s field-names and values of those fields.' )
    def __init__( self, **properties ):
        """Object constructor."""
        # Nominally final: Don't allow any class other than this one
        if self.__class__ != Record:
            raise NotImplementedError( 'Record is (nominally) a final class, and is not intended to be derived from.' )
        self.__fieldNames = []
        for theKey in properties:
            self.__setattr__( theKey, properties[ theKey ] )
            if not theKey in self.__fieldNames:
                self.__fieldNames.append( theKey )
        self.__recordIsLocked = True

# break

    @DocumentArgument( 'argument', 'self', None, SelfDocumentationString )
    @DocumentArgument( 'argument', 'name', None, '(String, required) The name of the attribute to be set.' )
    @DocumentArgument( 'argument', 'value', None, '(Any type, required) The value to be set in the named attribute.' )
    def __setattr__( self, name, value ):
        """Object-attribute setter."""
        if name == '_Record__recordIsLocked' and self.__recordIsLocked != False:
            raise AttributeError( 'The record is locked, and cannot be unlocked' )
        if self.__recordIsLocked:
            raise AttributeError( 'The record has been locked, and the "%s" attribute cannot be set.' % ( name ) )
        if hasattr( self, name ) and not name in [ '_Record__recordIsLocked', '_Record__fieldNames' ]:
            raise AttributeError( 'The "%s" attribute already exists, and cannot be reset.' % ( name ) )
        return object.__setattr__( self, name, value )

The changes here are:

Line(s)
8
self.__fieldNames is being explicitly set to an empty list
26
Explicitly checking the property-name against both _Record__recordIsLocked and _Record__fieldNames, in order to allow either to be set if the object isn't locked yet.
This feels like it may not be complete, so I may very well come back to it later...

        def testFieldNames( self ):
            """Unit-tests the FieldNames property of the Record class."""
            testProperties = { 'Property1':1, 'Property2':'ook', 'Property3':True }
            testProperties2 = { 'Property':1 }
            testObject = Record( **testProperties )
            self.assertEquals( testObject.FieldNames, testProperties.keys() )
            try:
                testObject.Keys = 'ook'
                self.fail( 'A Record\'s FieldNames cannot be altered.' )
            except AttributeError:
                pass
            except Exception, error:
                self.fail( 'Changes to a record\'s FieldNames should raise AttributeError, bu %s was raised instead:\n  %s ' % ( error.__class__.__name__, error ) )
            testObject = Record( **testProperties2 )
            self.assertEquals( testObject.FieldNames, testProperties2.keys(), 'The field-names for each record should be local to the instance' )

The change here is the addition of the last two lines - since each Record instance should have set it's FieldNames property independently, the second testObject should have the same keys as the testProperties2 dictionary. If the attribute were not explicitly set locally, it would have the keys from testProperties instead.

And that's the last of the "simple" concrete classes that weren't waiting on research. At least until the need for another one rears it's head. Next (finally), I'll get back to defining where query-execution takes place - you may remember that I set that aside to do some research in my previous post. I've done that research, and I can move on to implementation now...

No comments:

Post a Comment