At long last (On a couple of levels I have to add)!

First and foremost – after a long absence from blogging due to a life-changing event (i.e. recent addition to the family) I have managed to write up an article that has been on my mind since June last year! Secondly, because I finally managed to get the example to work – after months of on and off attempts I had been blocked by a very silly and obvious issue (more later) and this evening I managed to concentrate and see it through.

So what’s the big deal?

My desire was to write a blog explaining the beauty of using POCO from EF 4.0 all the way up through WCF to a client application. I had naively assumed that it would be a straightforward case of building the EFDM, creating my POCO classes, create the appropriate Context Interface (see my earlier blog article on creating an appropriate Context Interface) and then write the WCF service to use the EFDM and expose the same POCO classes to the client – how naïve!

The issue actually came up when I tried to pass the EF “filled” POCO classes back through the WCF service. Bang! I got stumped with the following exception:

System.ServiceModel.CommunicationException: The underlying connection was closed: The connection was closed unexpectedly. ---> System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Although I had the WCF Tracing enabled; I couldn’t for the life of me figure out what it was. Finally after digging through fresh traces I read carefully what the buried exception was:

There was an error while trying to serialize parameter http://tempuri.org/:GetDiverResult. The InnerException message was 'Object graph for type 'DiveLogger.Base.DataContracts.DiveProfile' contains cycles and cannot be serialized if reference tracking is disabled.'.  Please see InnerException for more details.

Of course! How silly of me! It was the fact that the navigation properties of my entities were actually causing cyclic references. Now there was a couple of ways I could do this.

  1. Scrutinize my navigation properties and make sure that the return reference properties were made private (this way the XML Serialization of WCF would overlook it as they would be POCO (i.e. no DataContract attributes here!) – it does work as I tried it out for a laugh.
  2. Utilize the blogged “Cyclic References Aware Contract Behavior” derived from IContractBehavior example. This would allow me to pass my POCO class through un-changed and not cause an exception (that would shutdown my service connection unexpectedly).

In the attached code example I have used the second option as I want to try and demonstrate how we can go from Data Model through EF context to the client via a WCF Service, without decorating our class with any special attributes or referencing any EF or Serialization classes – i.e. real to goodness POCO!

As my regular readers will recognize by now that my reason for being so excited about this (apart from the ability to inject interfaces and have my separation of concerns) is for the all important Unit Tests – after all what fun would it be?

The idea of this exercise was to show that with .NET 4.0 we can now release our bonds to specific assemblies / classes and have truly lightweight classes. Consider how pre-4.0 would have been done:

POCO through technology stack

Each layer would have involved some form of transformation from one type to another simply to avoid issues with the associated technology stack.

Consider the “new” way with .NET 4.0:

POCO through technology stack

Now we don’t have to do any transformation from one type to another because the same type is used all the way up the technology stack without any issues (except for the one described earlier).

So how was it done?

The following steps will guide you through the process:

1. Define our POCO classes according to our Data Model:

[POCO classes]

POCO Definition


[Data Model]

EF Data Model
2. Remove the Custom Tool from the DataMapping properties
(so that we can use POCO):
 

EFDM Custom Tool
 

Effect of removing EFDM Custom Tool
3. Define the Context Interface that will become the injection point in future references to the DAL:

    public interface IDiveLoggerContext
    {
        IObjectSet AccessTypes { get; }
        IObjectSet CertificationTypes { get; }
        IObjectSet DiveProfiles { get; }
        IObjectSet Divers { get; }
        IObjectSet DiveSites { get; }
        IObjectSet SurfaceTypes { get; }
        IObjectSet WaterBodyTypes { get; }
    }

And implement it
 

Implement Context Interface

4. In a WCF project define the Service Interface (we will not decorate the DataContracts as they are POCO):

namespace DiveLogger.Service
{
    [ServiceContract]
    public interface ILoggerService
    {
        [OperationContract]
        Diver GetDiver(int id);
    }
}
5. Create the CyclicReferencesAwareContractBehavior, CyclicReferencesAwareAttribute and ApplyCyclicDataContractSerializerOperationBehavior classes as described in Chabsters blog (WCF Cyclic references support). Obviously if there are no cyclic references in any of our entities, then chances are you will not need to put in this workaround for WCF 4.0.
 

Cyclic Reference Treatment Classes
 

And use it on the Service contract
 

Use CyclicReferencesAware attribute
 

Now the XmlSerialization can handle those pesky cyclic reference Navigation Properties and our WCF Service will work.
6. Utilize the Context Interface we created earlier to separate the DAL from the Service:

    public class LoggerService : ILoggerService
    {
        private IDiveLoggerContext m_context;

        public LoggerService()
        {
            Initialize(null);
        }

        internal LoggerService(IDiveLoggerContext i_context)
        {
            Initialize(i_context);
        }

        private void Initialize(IDiveLoggerContext i_context)
        {
            m_context = i_context ?? new DiveLoggerContext(UtilityFunctions.BuildAdoConnectionString(null));
        }

        public Diver GetDiver(int id)
        {
            if (id < 1 || id > 999)
                throw new ArgumentOutOfRangeException("Diver ID needs to be between 1 and 999", "id");

            IQueryable query = from diver in m_context.Divers
                                      .Include("CertificationTypes")
                                      .Include("DiveProfiles")
                                      .Include("DiveProfiles.DiveSite")
                                      .Include("DiveProfiles.WaterBodyType")
                                      .Include("DiveProfiles.AccessType")
                                      .Include("DiveProfiles.SurfaceType")
                                      where diver.Id == id
                                      select diver;
            Diver retVal = query.ToList().SingleOrDefault();
            return retVal;
        }
    }
7. In our Unit Test we can mock out the Context Interface instead of generating an actual instance:

    [TestMethod]
    public void TestGetSingleDiver()
    {
        // - ARRANGE -
        // Create the stub instance
        IDiveLoggerContext context = MockRepository.GenerateStub();

        // define/create dummy data
        int id = 500;
        string firstName = "Sherlock";
        string lastName = "Holmes";
        string title = "Mr.";
        IObjectSet divers = TestHelper.CreateDivers(id, firstName, lastName).AsObjectSet();

        // declare instance that we want to "retrieve"
        Diver individual;

        // Explicitly state how the stubs should behave
        context.Stub(stub => stub.Divers).Return(divers);

        // Create a real instance of the Servcie that we want to put under test, injecting the dependency in the constructor
        LoggerService service = new LoggerService(context);

        // - ACT -
        individual = service.GetDiver(id);
        // - ASSERT -
        // Make absoultely sure that the expected excption type was thrown
        Assert.IsNotNull(individual);
        // Make sure that the method was NOT called.
        context.AssertWasCalled(stub => { var temp = stub.Divers; });
    }

This will mean that we can get more realistic code-coverage on our Service:
 

Code Coverage

The attached code contains all of the classes and code mentioned in the steps outlined previously.

  • Share/Bookmark

Many times a custom tool that is created to overcome a particular “issue” ends up morphing into something that is greater than the initial idea. Case in point, I had come across many instances where the sheer bulk of Sprint Backlog Items that had to be entered in to TFS almost made me break out in to a cold sweat – it is an arduous task to do it via the Team Explorer Window when you are doing the entries one by one:

TFS Sprint Backlog Item

So, typically for large volumes of Sprint Backlog Items I would prefer to use the Export to Excel capability of a TFS WorkItem Query:

Export To Excel

I could then enter the bulk of Sprint Backlog Items entering the required data in each row (representing each SBI) and then use the Publish feature of the TFS Excel Team Add-In:

TFS Excel Add-In Publish Change

The only issue with this approach is that I’d have to go back and link each SBI in Team Explorer with the correct PBI!

So I created an Add-In to overcome this. At first the Add-In would only pull back the PBI Id and Title and allow the user to type in a new PBI Id and publish the link information, but then I realized it would be great (for planning purposes) to be able to pull back whatever field from a PBI that I wanted. So I created a mapping between the PBI Field name and a Column name that the user would type in to Excel (the red shows the mapping from the Excel Column text; the purple shows the mapping from the TFS Template definition for a Product Backlog Item:

JasPWarE TFS Excel Add-In Mapping

I can then pull back whatever I like and be able to see how it relates to the list of Sprint Backlog Items I have. I also created an informative video to illustrate how to use the add-in.

You can get the latest version from:

http://www.petrellyn.com/jaspware/ExcelTfsAddIn

  • Share/Bookmark

Recently I had been working on creating Sprint Backlog Items in Excel and then using the Team Explorer add-in for Excel to publish the changes back to the TFS server. It was great to be able to view tasks in Excel and update similar tasks through a spreadsheet manner rather than swapping back and fourth in Team Explorer of Visual Studio. The only issue was the linking to Product Backlog Items; there wasn’t any way to link the published SBI with appropriate Product Backlog Items – it meant I would have to go back to Team Explorer and do it through there. I had already worked on an Excel add-in to calculate work remaining for each team member:

Excel Capacity Worksheet

The functionality behind this feature is based on certain fields being defined in the SCRUM template (Estimated Effort (Scrum), Work Remaining (Scrum)). The Excel Add-In exposes Formulas that can be utilized within a cell:

GetTotalHoursRemaining Excel Formula

GetTotalHoursRemaining() provides the calculation of Total hours remaining in a given Sprint for one or more team members:

GetTotalHoursRemaining Excel Formula

Returning to the problem at hand, I realized that it would be simple enough to add functionality into the existing Add-In to be able to “pull” the PBI links down to the sheet and even “push” new links back up to the server; after all each PBI has an ID that can be represented as text, and the process of adding a link to any TFS WorkItem is accomplished through the following call:

workItem.Links.Add(new RelatedLink(linkId));

With a constructor signature for a RelatedLink instance accepts an int:

RelatedLink(int relatedWorkItemId)

This meant that I could scour the ID column of the Sprint Backlog Item in Excel and can pull off the IDs as integers, then scouring the column that had the Product Backlog Item IDs I could update the Sprint Backlog Item internal Links:

Sprint Backlog Item ID Column

By adding a further column (in the Spreadsheet) that can be identified by the Add-In I could pull or push the links to the Product Backlog Items:

Refresh TFS Item Links

I created a simplistic installer (it makes assumptions that you are running Office 2007 and that you are using a suitable template such as the Conchango SCRUM for TeamSystem):

http://www.petrellyn.com/jaspware/ExcelTfsAddIn/index.php?lang=eng

  • Share/Bookmark

On several occasions I have come across instances where people have mistakenly called the interface that a Service implements, the Service itself. The truth of the matter is that the Service can be formed by a number of interfaces (referred to as Service Contracts), each of which contain various methods (referred to as Operation Contracts) returning or receiving simple or complex data (referred to as Data Contracts). Consider the following example:

Example WeatherService

Example WeatherService

From a “traditional sense” it represents a class that implements two interfaces – now hold that thought.

Consider the same example from a Service point of view through the convention used by Thomas Erl:

Thomas Erl style Service Diagram

Thomas Erl style Service Diagram

Two Service Contracts (ITemperature and IRainfall) are defined as follows:

    /// 
    /// ServiceContract that provides temperature information.
    /// 
    [ServiceContract]
    public interface ITemperature
    {
        /// 
        /// Retrieves the temperature for a specified location given the Latitude and Longitude
        /// 
        /// <param name="location">Location information for retrieving Temperature.</param>
        /// Temperature of location.
        [OperationContract]
        double GetTemperature(Location location);
    }

    /// 
    /// ServiceContract that provides rainfall information.
    /// 
    [ServiceContract]
    public interface IRainfall
    {
        /// 
        /// Retrieves the rainfall for a specified location given the Latitude and Longitude
        /// 
        /// <param name="location">Location for retrieving Rainfall.</param>
        /// Rainfall of the given location.
        [OperationContract]
        double GetRainfall(Location location);
    }

Where Location is the DataContract defined as:


    /// 
    /// DataContract that represents a location specified by its Latitude and Longitude.
    /// 
    [DataContract]
    public class Location
    {
        /// 
        /// Gets / sets the Latitude of the location.
        /// 
        [DataMember(IsRequired = true)]
        public double Latitude { get; set; }

        /// 
        /// Gets / sets the Longitude of the location.
        /// 
        [DataMember(IsRequired = true)]
        public double Longitude { get; set; }
    }

The class that implements them, WeatherService, is defined as:


    /// 
    /// ServiceContract implementation class
    /// 
    public class WeatherService : IRainfall, ITemperature
    {
        /// 
        /// Retrieves the rainfall for a specified location given the Latitude and Longitude
        /// 
        /// <param name="location">Location specific information for retrieving Rainfall.</param>
        /// Rainfall of the given location.
        public double GetRainfall(Location location)
        {
            return _GetRainfall(location.Latitude, location.Longitude);
        }

        /// 
        /// Retrieves the temperature for a specified location given the Latitude and Longitude
        /// 
        /// <param name="location">Location specific information for retrieving Temperature.</param>
        /// Temperature of location.
        public double GetTemperature(Location location)
        {
            return _GetTemperature(location.Latitude, location.Longitude);
        }

        private double _GetTemperature(double latitude, double longitude)
        {
            return new Random().NextDouble();
        }

        private double _GetRainfall(double latitude, double longitude)
        {
            return new Random().NextDouble();
        }
    }

So now when we refer to a Service we are talking about the implementation of one or more ServiceContracts (as in the case of the Weather Service). One developer may be working on a single ServiceContract that the Service implements (such as ITemperature) but it does not necessarily mean that they are working on the whole of the Service.

  • Share/Bookmark

Visual Studio 2010 - Beta 2

Whilst preparing for my Entity Framework 4.0 and Unit Testing presentation at the recent New England Code Camp, I came across an issue with my code that I couldn’t entirely understand. Picture the scenario:

In Visual Studio 2008 utilizing EF 1.0:

  • Create a repository Assembly that contains the model and entity classes (i.e. no POCO) for a group of tables in my DB
  • Create a Manager class to expose public functions / methods to retrieve data from the DB via the EntityContext
    [store the Manager.cs file in a common area and create a Link to it from the project].
  • Create an extension method that will handle the usage of Lambda expressions inside the context of .Include() [Did I say I hate "magic strings"?]
    [store the ObjectQueryExtension.cs file in a common area and create a Link to it from the project]
  • Create a simple Console app to call the methods on the Manager class
    [store the Program.cs file in a common area and create a Link to it from the project].

Visual Studio 2008

In Visual Studio 2010 Beta 2 utilizing EF 4.0:

  • Create a repository Assembly that contains the model and entity classes (i.e. no POCO) for a group of tables in my DB
  • Create a Manager class to expose public functions / methods to retrieve data from the DB via the EntityContext
    [create a Link to the Manager.cs file from the project in a common area].
  • Create an extension method that will handle the usage of Lambda expressions inside the context of .Include()
    [create a Link to the ObjectQueryExtension.cs file in the common area from the project]
  • Create a simple Console app to call the methods on the Manager class
    [create a Link to the Program.cs file in the common area from the project].

Visual Studio 2010 - Beta 2

As you will see, the only difference between the two solutions is the actual EntityFramework context definition; one utilizes EF 1.0 and the other EF 4.0.

Compile and execute both and it works perfectly, same Program.cs code for both (making the Console Application); same Manager.cs and ObjectQueryExtension.cs code for both (making the RepositoryManager assembly).

Now the fun starts. I then worked my way back to using Dependency Injection and created the unit test methods based on the VS 2010 project described above. When the compiler attempts to compile the following section of code:


IQueryable<Person> query = context.PersonSet
                               .Include(p => p.PersonalDetail)
                               .Include("FavoriteBeers.Beer")
                               .Include(p => p.Customer.Include<Customer, CustomerType>(c => c.CustomerType))
                               .Include(p => p.Addresses);

The following compile error was the result of the attempted compilation against the preceding code:


Ef4.0AndEf1.0\PocoInEF4.0\EFWorkshop.Poco.RepositoryManager\Manager.cs(78,53): error CS1660: Cannot convert lambda expression to type 'string' because it is not a delegate type
Ef4.0AndEf1.0\PocoInEF4.0\EFWorkshop.Poco.RepositoryManager\Manager.cs(78,58): error CS0311: The type 'EFWorkshop.Poco.Base.Entities.Customer' cannot be used as type parameter 'TSource' in the generic type or method 'EFWorkshop.Ef.Repository.ObjectQueryExtension.Include<TSource,TPropType>(TSource, System.Linq.Expressions.Expression<System.Func<TSource,TPropType>>)'. There is no implicit reference conversion from 'EFWorkshop.Poco.Base.Entities.Customer' to 'System.Data.Objects.DataClasses.IEntityWithRelationships'.
Ef4.0AndEf1.0\Common\ObjectQueryExtension.cs(118,33): (Related location)
Ef4.0AndEf1.0\PocoInEF4.0\EFWorkshop.Poco.RepositoryManager\Manager.cs(78,60): error CS1061: 'System.Linq.IQueryable<EFWorkshop.Poco.Base.Entities.Person>' does not contain a definition for 'Customer' and no extension method 'Customer' accepting a first argument of type 'System.Linq.IQueryable<EFWorkshop.Poco.Base.Entities.Person>' could be found (are you missing a using directive or an assembly reference?)

However, the following “magic string” laden Includes compile and execute fine:


IQueryable<Person> query = _context.People
    .Include("FavoriteBeers.Beer")
    .Include("PersonalDetail")
    .Include("Customer.CustomerType")
    .Include("Addresses");

Therefore it would seem that the non POCO based classes permit us having the Lambda expression based includes as described earlier; however the moment that POCO is introduced that style of Include is no longer viable – or is it?

  • Share/Bookmark

Wow – June 24th was the last entry! First of all I need to apologize for my bad blogging; I have no other excuse except for the volume of work AT work. Sure I could have re-prioritized and it may have made a difference, but I don’t think my employer would have been very pleased.

So, New England Code Camp, Microsoft Offices, Waltham, MA – full information can be found at http://www.thedevcommunity.org/Events/PresentationList.aspx?id=13. Today I am giving two presentations:

Using Entity Framework’s New POCO Features: Part 2 (Unit Testing)

Level: Intermediate

Starts: Oct 17 2009 2:50 PM

Ends: Sep 17 2009 4:05 PM

Room: MPR A

Speaker: James Phillips

In many cases Unit Testing is considered a chore rather than another development task and often ends up being the last task in a development cycle. More often than not, the sheer work involved in preparing unit tests for existing code can lead to the production of Integration Tests rather than true Unit Tests. Where a unit of code that is under test relies on an external resource, such as a Database or Configuration file, the dependency can lead to testing of the underlying mechanism as well as the unit being tested. This was especially true with Entity Framework 1.0 shipped with .NET Framework 3.5 Service Pack 1. With the advent of .NET Framework 4.0, the Entity Framework has advanced in favor of better Unit Testing with the use of POCO and the ability to create interfaces based on IObjectSet. This presentation will cover the examples that can lead to true Unit Testing as opposed to Integration Testing and provide valuable feedback metrics such as code coverage and automated build time reporting of results.

SCRUM and TFS

Level: Introductory

Starts: Oct 17 2009 4:10 PM

Ends: Sep 17 2009 5:25 PM

Room: Rhode Island

Speaker: James Phillips

SCRUM has grown in popularity and acceptance by many companies over the world with numbers of registered SCRUM Masters reaching 51,955 (11 March 2009 – Jeff Sutherland). Although SCRUM does not stipulate what tools to use to produce the necessary artifacts, Microsoft Team Foundation System provides a number of features via TFS Explorer that facilitate capturing the artifacts of SCRUM and is a useful tool for any SCRUM Master, Team and Product Owner. This presentation will highlight the SCRUM framework and show you practical use of TFS and other tools that facilitate the ceremonies and artifacts of SCRUM.

The slides and code are available for download here:

SCRUM_And_TFS.zip

EF_POCO_And_UnitTesting_slides.zip”

EF_POCO_And_UnitTesting_code.zip

EF_POCO_And_UnitTesting.zip

  • Share/Bookmark

After publishing the blog article on Mocking in EF 2.0, Diego Vega (Microsoft Program Manager Entity Framework and LINQ to SQL Product Teams in Redmond) pointed out that the following code snippet is quite inefficient:


    var customers = (from cus in _context.Customers
                    where cus.CustomerID == i_customerId
                    select cus);
    if (customers.Count() == 1)
    {
        o_customer = customers.Single<Customer>();
    }
    else
    {
        o_customer = null;
    }

It turns out that even though the expression does not change between the call to customers.Count() and customers.Single<Customer>(), they will actually cause two calls to the Database. I had mistakenly assumed that the result would be cached as there was no change to the “query”.

By using the SingleOrDefault<T>() function that is now available in EF 2.0 (through IQueryable<T>) we actually save on the round trip.


    o_customer = (from cus in _context.Customers
                    where cus.CustomerID == i_customerId
                    select cus).SingleOrDefault<Customer>();

Something to bear in mind when working with IQueryable<T>.

a2dbm5vn79

  • Share/Bookmark

Finally! I’ve been itching to get into the latest incarnation of the EF for a while now and finally I had the opportunity to take it around the block (kick the tires and generally rough it up). My main interest in EF 2.0 was the supposed support for Unit Testing and the improvements on using POCO to map to the Data Model.

I have to admit, although I liked EF 1.0 when I first started using it, one of my biggest bug bears was the fact that you had to “disconnect” your entities before you could really work with them outside the context (no pun intended) of the Entity Framework. I was also quite miffed when I discovered there was no easy way to mock the underlying data layer so I ended up with code like this:


    /// <summary>
    /// Loads a customer instance with the relevant information from the database.
    /// </summary>
    /// <param name="i_customerId">The customerId of the customer data to be retrieved.</param>
    /// <param name="o_customer">The customer instance to be created.</param>
    public void Load(string i_customerId, out Customer o_customer)
    {
        if (string.IsNullOrEmpty(i_customerId))
        {
            throw new ArgumentException("Parameter cannot be null.", "i_customerId");
        }
        int numericVal;
        if (!int.TryParse(i_customerId, out numericVal))
        {
            throw new ArgumentException("Parameter cannot be non-numeric.", "i_customerId");
        }
        if (numericVal < 0 || numericVal > 9999)
        {
            throw new ArgumentOutOfRangeException("i_customerId");
        }

        m_customerRepository.Load(i_customerId, out o_customer);

        if (o_customer != null)
        {
            if (!string.IsNullOrEmpty(o_customer.ContactName) && o_customer.ContactName.Contains(" "))
            {
                o_customer.ContactName = o_customer.ContactName.Trim(' ');
                string[] names = o_customer.ContactName.Split(' ');
                if (names.Length > 1)
                {
                    names[names.Length - 1] = names[names.Length - 1].ToUpper();
                }
                o_customer.ContactName = string.Join(" ", names);
            }
        }
    }

In this case, m_customerRepository is the injected ICustomerRepository instance. When we look at the implementation of the actual data layer class (which does not get tested by the Unit Test, we find that inside the Load method we have the following:


    /// <summary>
    /// Loads a customer instance with the relevant information from the database.
    /// </summary>
    /// <param name="i_customerId">The customerId of the customer data to be retrieved.</param>
    /// <param name="o_customer">The customer instance to be created.</param>
    public void Load(string i_customerId, out BaseCustomer o_customer)
    {
        Customer customer = (CustomerSet.Where(cust => !string.IsNullOrEmpty(cust.CustomerID) &&
                                                       cust.CustomerID == i_customerId)).First();
        if (customer != null)
        {
            o_customer = new BaseCustomer()
                             {
                                 Address = customer.Address,
                                 City = customer.City,
                                 CompanyName = customer.CompanyName,
                                 ContactName = customer.ContactName,
                                 ContactTitle = customer.ContactTitle,
                                 Country = customer.Country,
                                 CustomerID = customer.CustomerID,
                                 Fax = customer.Fax,
                                 Phone = customer.Phone,
                                 PostalCode = customer.PostalCode,
                                 Region = customer.Region
                             };
        }
        else
        {
            o_customer = null;
        }
    }

Not the best way of doing things, that is for sure! In fact it is downright ugly (IMHO). So when I heard that there were improvements to the EF for .NET 4.0, especially in the area of Unit Testing I was curious to say the least. As I delved in deeper I started finding more and more things that made it more attractive to my style of development. For example one of the beauties of EF 2.0 is the fact that you can remove the CustomTool that generates the entity classes that are bound to the data model (through the edmx file). When you do this, you effectively get rid of the code generation for the EF instance that you have loaded in your project. There are some excellent examples (and walkthroughs available) from the ADO.NET Team blog:

POCO in Entity Framework : Part 1 – The Experience (excellent walkthrough on removing the CustomTool)

POCO in Entity Framework : Part 2 – Complex Types, Deferred Loading and Explicit Loading

POCO in Entity Framework : Part 3 – Change Tracking with POCO

So now what? Great! So now I can use my POCO to update the Data Model. But I still hadn’t found out how to do true unit testing with DI and mocking? I was quite flummoxed until I realized (with a helpful pointer from a friend at Microsoft – thanks Jason) that the answer was staring me in the face:

“Can’t you create a mock class that derives from IObjectSet instance yourself, or is there a problem doing that?”

Well yes, I did have a problem with that – it meant that I would have to write more code. I was naively hoping to have something like:


List<Customer> cusList = TestHelper.CreateCustomerList();
IObjectSet<Customer> context = cusList.AsObjectSet();

So I was being lazy… I guess that, with each version of .NET, I had become more and more accustomed to so much being done for me that stumbling across something as “simple” as creating an AsObjectSet() function that was not available was bit of a shock. More so when you look at what IS available on a List<entity> method / property list.

At first I contented myself with just doing what was obvious – create a mock class that inherited from IObjectSet<Customer>, before I realized (with another push from Jason) that I could make it more generic and have a MockObjectSet<T> class:


    internal class MockObjectSet<T> : IObjectSet<T>
        where T : class
    {
        public MockObjectSet(List<T> entityList)
        {
            if (entityList == null)
            {
                throw new ArgumentNullException("entityList");
            }
            else
            {
                _repository = entityList.ToList();
            }
        }

        IList<T> _repository;

        #region IObjectSet<T> Members

        public void AddObject(T entity)
        {
            _repository.Add(entity);
        }

        public void Attach(T entity)
        {
            this.AddObject(entity);
        }

        public void DeleteObject(T entity)
        {
            _repository.Remove(entity);
        }

        #endregion

        #region IEnumerable<T> Members

        public IEnumerator<T> GetEnumerator()
        {
            return _repository.GetEnumerator();
        }

        #endregion

        #region IEnumerable Members

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            return _repository.GetEnumerator();
        }

        #endregion

        #region IQueryable Members

        public Type ElementType
        {
            get { return typeof(T); }
        }

        public System.Linq.Expressions.Expression Expression
        {
            get { return _repository.AsQueryable<T>().Expression; }
        }

        public IQueryProvider Provider
        {
            get { return _repository.AsQueryable<T>().Provider; }
        }

        #endregion
    }

It is important to note here that the TestHelper.CreateCustomerList() function has several overrides and returns a List<Customer> filled with dummy data.

After playing around a bit I realized that I could actually just create an Extension Method that would create an instance of the mock CustomerSet and therefore I could call it from within my Unit Test code. The Extension Method looks like this:


    public static class ObjectSetExtension
    {
        public static IObjectSet<T> AsObjectSet<T>(this List<T> entities) where T : class
        {
            return new MockObjectSet<T>(entities);
        }
    }

Now if we revisit the unit test code, we get the following:


        [TestMethod]
        public void TestLoadValidCustomerContactNameWithSurname()
        {
            // Arrange
            // Create the stub instance
            INorthwindContext context = MockRepository.GenerateStub<INorthwindContext>();
            // Create the dummy data
            const string customerId = "555";
            const string contactName = "James Person";
            IObjectSet<Customer> customers = TestHelper.CreateCustomerList(contactName, customerId).AsObjectSet();

            // declare the dummy instance we are going to use
            Customer loadedCustomer;

            // Explicitly state how the stubs should behave
            context.Stub(stub => stub.Customers).Return(customers);

            // Create a real instance of the CustomerManager that we want to put under test
            Managers.CustomerManager manager = new Managers.CustomerManager(context);

            // Act
            manager.Load(customerId, out loadedCustomer);

            // Assert
            context.AssertWasCalled(stub => { var temp = stub.Customers; });
            // Check the expected nature of the dummy intance
            Assert.IsNotNull(loadedCustomer);
            Assert.IsNotNull(loadedCustomer.ContactName);
            Assert.IsTrue(loadedCustomer.ContactName == "James PERSON");
        }

If we compare the two managers again (the manager that I had created in a previous blog posting depended on EF 1.0), we will see that the EF 2.0 instance actually contains lambda expressions to do the queries.

EF 1.0:


    /// <summary>
    /// Loads a customer instance with the relevant information from the database.
    /// </summary>
    /// <param name="i_customerId">The customerId of the customer data to be retrieved.</param>
    /// <param name="o_customer">The customer instance to be created.</param>
    public void Load(string i_customerId, out Customer o_customer)
    {
        if (string.IsNullOrEmpty(i_customerId))
        {
            throw new ArgumentException("Parameter cannot be null.", "i_customerId");
        }
        int numericVal;
        if (!int.TryParse(i_customerId, out numericVal))
        {
            throw new ArgumentException("Parameter cannot be non-numeric.", "i_customerId");
        }
        if (numericVal < 0 || numericVal > 9999)
        {
            throw new ArgumentOutOfRangeException("i_customerId");
        }

        m_customerRepository.Load(i_customerId, out o_customer);

        if (o_customer != null)
        {
            if (!string.IsNullOrEmpty(o_customer.ContactName) && o_customer.ContactName.Contains(" "))
            {
                o_customer.ContactName = o_customer.ContactName.Trim(' ');
                string[] names = o_customer.ContactName.Split(' ');
                if (names.Length > 1)
                {
                    names[names.Length - 1] = names[names.Length - 1].ToUpper();
                }
                o_customer.ContactName = string.Join(" ", names);
            }
        }
    }

EF 2.0:


    /// <summary>
    /// Loads a customer instance with the relevant information from the database.
    /// </summary>
    /// <param name="i_customerId">The customerId of the customer data to be retrieved.</param>
    /// <param name="o_customer">The customer instance to be created.</param>
    public void Load(string i_customerId, out Customer o_customer)
    {
        if (string.IsNullOrEmpty(i_customerId))
        {
            throw new ArgumentException("Parameter cannot be null.", "i_customerId");
        }
        int numericVal;
        if (!int.TryParse(i_customerId, out numericVal))
        {
            throw new ArgumentException("Parameter cannot be non-numeric.", "i_customerId");
        }
        if (numericVal < 0 || numericVal > 9999)
        {
            throw new ArgumentOutOfRangeException("i_customerId");
        }

        var customers = (from cus in _context.Customers
                         where cus.CustomerID == i_customerId
                         select cus);
        if (customers.Count() == 1)
        {
            o_customer = customers.Single<Customer>();
        }
        else
        {
            o_customer = null;
        }

        if (o_customer != null)
        {
            if (!string.IsNullOrEmpty(o_customer.ContactName) && o_customer.ContactName.Contains(" "))
            {
                o_customer.ContactName = o_customer.ContactName.Trim(' ');
                string[] names = o_customer.ContactName.Split(' ');
                if (names.Length > 1)
                {
                    names[names.Length - 1] = names[names.Length - 1].ToUpper();
                }
                o_customer.ContactName = string.Join(" ", names);
            }
        }
    }

In the second code, snippet, because I am calling straight to an instance of IObjectSet<Customer>, it could either be my mocked one or the actual Entity Framework instance, which looks like this (thanks to POCO binding):


    public class NorthwindContext : ObjectContext, INorthwindContext
    {

        public NorthwindContext()
            : base("name=NorthwindEntities", "NorthwindEntities")
        {
        }

        private ObjectSet<Order> _orders;
        private ObjectSet<Employee> _employees;
        private ObjectSet<Customer> _customers;

        #region INorthwindContext Members

        IObjectSet<Employee> INorthwindContext.Employees
        {
            get { return _employees ?? (_employees = CreateObjectSet<Employee>()); }
        }

        IObjectSet<Customer> INorthwindContext.Customers
        {
            get { return _customers ?? (_customers = CreateObjectSet<Customer>()); }
        }

        IObjectSet<Order> INorthwindContext.Orders
        {
            get { return _orders ?? (_orders = CreateObjectSet<Order>()); }
        }

        #endregion
    }

This means that when I run my code coverage for the EF 2.0 version, I will be hitting the true boundary between the entity and the model, thanks to a combination of POCO and the separation of concerns brought about by IObjectSet.

Ok, so there’s no kitchen sink – what would you do with it if there was?

  • Share/Bookmark

Now the title might be a bit mis-leading, but rather than discuss one issue or observation at a time, I wanted to touch on all topics in the same post.

Recently I have been delving deeper and deeper into the issues that I have seen first hand and heard / read about with regards to Scrum and I raise them here as a point of discussion and as a reminder to myself in the future when I wonder just what the heck was I thinking about?

Multi-tasking

First and foremost, something close to home that is the subject of simian (human) multi-tasking. Often when I broach this subject with my peers there is a lot of references to “Jamie hours” and being the inhuman way of doing things – i.e. my long hours of work and investigation. In other words – get a life, turn off the laptop and go to bed! There is a really good article in Time magazine that discusses the human capacity of multi-tasking.

I say this in jest but the topic is quite serious; should a Scrum Master take on tasks in a Scrum team in as much as being in the trenches with the team whilst at the same time as being the facilitator? I asked this question recently on LinkedIn and the few replies that I got are more or less what I had expected; “of course they can” and in some cases, should. And for the most part I do agree. It would appear to make common sense that someone who is on a Scrum Team as Scrum Master would need to take on a few non-critical tasks to at least lessen the burden on the team as well as “feel the pain” that the team goes through. This is made easier when the Scrum Master is a technical lead, a senior developer or SQ lead, someone who is accustomed to the typical tasks of the team.

However, when the role is filled by someone that is traditionally Project Management based, would they be able to jump in and out of the role? I would say that it depends on the Scrum experience of the individual – someone that is new to Scrum who is re-tooling as a Scrum Master from a Project Management position, would need time to settle in to the role and go through several sprints before even entertaining the idea of rolling up their sleeves and getting stuck in with the team. On the other hand it could depend entirely on the environment that the development is taking place. If it is a highly regulatory environment (Petro-chemical industry, aviation, military, healthcare) then the Scrum Master will probably be inundated with the various QA documents that are required.

Multi-project

Leading on from the discussion of a Multi-tasking Scrum Master, the context was set for a one to one correlation of Scrum Master to team. Something completely different is a one to many mapping of Scrum Master to two or more teams. In this scenario I would imagine that the traditional Project Manager role would be better suited to swapping between teams in a given time frame – although I could also foresee there being a decrease in efficiency on the part of the Scrum Master; what would happen if the Sprint Planning for one team collided with another important ceremony of the other team, such as a Sprint Retrospective. I would say that this would probably be more problematic than the first scenario and could lead to the failure of two or more teams, not just one. There again, it does depend on the individual and the teams that they are working with.

If I can do it; so can you.

Dangerous phrase that; many have lost their shirt or worse on such a challenge. However it does raise a valid point and should serve as a glimmer of hope to those that are embarking on the path to agility.

True, it is documented that many projects that have attempted to adopt agile methodologies have fallen by the wayside because of a plethora of reasons; but it only takes one team to pull it off in a company to be able to “inspire” other teams to follow suit. The main issue I can envisage is the fact that all too often people have the belief that if team X did it and it worked for them, then let’s take that model and apply it to team Y. They often overlook some of the dynamics that permitted team X to adopt and fly with agility. A small, close-knit, co-located team, bonding from the beginning across the disciplines; cannot be compared to teams that are dysfunctional from the outset (Ken Schwaber recently gave a great presentation on “Flaccid Scrum” and Martin Fowler blogged on this very issue). Also there are teams that feel it cannot be done because they have not been through the motions of Scrum to realize that it is capable of helping them to improve their process. The important point for those individuals to realize is that Scrum is not the silver bullet – it is the de-fogger of the windscreen, blowing away all the mist and shrouded activity that goes on in other styles of software development. With the tight feedback loop, it will highlight the issues of the development process faster and with more clarity than before – providing a much needed view of the issues well before it is too late.

Many un-initiated see a well oiled fully functional Scrum team as a bunch of hippy-tree-huggers because of their bonding and team work; however, few realize that this is a result of being immersed in a routine that requires each one of them to have faith in their colleagues and to be able to self-organize. In my experience it was not a pre-requisite to joining the team, it was a by-product of agility.

So then we come back to the title statement “If I can do it; so can you”. Although this is not entirely true, there is some smidgen of truth in it. After all, what is a Product Backlog? Isn’t it a list of requirements (stories) or defects (remember – bugs are now first class citizens). What is a Sprint backlog? It’s a set of tasks to complete within a given time. The main difference are the roles of the Product Owner and the Scrum Master and the fact that the team is central to the development process and take part in the estimation and sizing of the stories. For those of us doing Scrum it makes so much sense and can often be a sticking point when talking to our non-Scrum colleagues.

An important fact to bear in mind, covered in books and presentations, is that no two projects are alike; what works for one team might not work for another. For certain types of development a one or two week sprint is more manageable because of the nature of the work being done; or a more traditional three or four week sprint is necessary because of the type of stories being dealt with. With Scrum the Sprint Retrospective is fundamental for providing the necessary feedback to be able to make adjustments and fine tune the process – without it there is no feedback loop and the system is running un-controlled.

  • Share/Bookmark

People often wonder where I get the time and energy to work the way that I do; the truth is it that it is all down to the passion I have for my work as a software engineer and the type of work that I am involved in. I am also very passionate about pushing the boundaries and I find it very difficult to accept that something can’t or won’t be done on the basis of “But, we’ve always done it that way” – it doesn’t necessarily mean that it is the right way, nor does it mean that it is appropriate for today’s world. It might have kept companies ahead of the game 10 or 15 years ago, but the world has moved on since then and now we have to be faster, more efficient, better. For me Agile Software Development lends itself to this change and is all about change.

Becoming Agile... in an imperfect world - Manning Publications

Becoming Agile... in an imperfect world - Manning Publications - Greg Smith and Ahmed Sidky

I will be honest and say that this is the first “work related” book that actually captured my attention to the point that I would sit and read whilst my loved ones were enjoying their sweet slumber (and obviously blog about it).

This book is highly informative; being neither condescending, nor pushy and over-bearing on the way that agile adoption should be approached. Without intending to be a partisan; I would even go as far as saying that it is more than recommended reading, it should be required reading for anyone thinking of migrating to agile; recent adoptees of agile or anyone in between.

From the first chapters that guide us through the principles (I loved the annotated 12 principles of the Manifesto for Agile Software Development – section 1.1.2) and the descriptive text on the “paradigm shift from a plan-driven mentality”; to the readiness assessments and the importance of obtaining executive support; through to the population of the product backlog and on to the first iterations of an example company adopting Agility in their software development process.

There is no substitute for actually reading the book, which is a must; however various chapters and sections drew my attention more than others.

Section 1.1.2 – The agile principles:

Although the manifesto has been analyzed and described on various blogs and books; the descriptive text for each principle was really handy when evangelizing on agile and is a good primer for anyone new to the subject (and possibly even for those of us who have been practicing agile).

Section 1.2 – A paradigm shift from a plan-driven mentality:

This is a must read for those of us who have come from years of waterfall and attempts at changes to “traditional” methodologies or processes. The section highlights the change in mentality that is needed to move from more traditional ways of software development to a leaner, more agile way.

Section 2.4 – What does it look like when a team “becomes agile”:

With comparative plan related diagrams of how the change is made from waterfall to agile and the breakdown of how that transition was made; is comforting for those that have made the change as well as those that are thinking about/planning the change.

Section 3.2 – The different flavors of agile:

This section has a really good breakdown of strengths and weaknesses of the two foremost agile methods; Scrum and XP.

Section 3.3 – Create your own flavor to become agile within your constraints:

When I first started in agile there seemed to be a common mantra that permeated throughout the meetings and blogs that effectively stated if you weren’t doing it exactly by the book, you weren’t being agile. It is completely preposterous.
How can the same agile development methods used by a web advertising company be cookie-cut for a software company writing client server applications in a strong regulatory environment? The answer is – it can’t. There will need to be some adaptation. After all isn’t that one of the principle concepts of agile; what is the point of having Sprint Retrospectives to adjust and tweak a process in a constant attempt to improve? This section is a must read for those that are constantly being belittled or set-upon by “those in the know”.

Chapter 4 – The fitness test: all about readiness assessments:

Although I have not had the opportunity of actually performing this exercise; having read through the chapter it is very informative and goes in to great detail on how a company can be measured for it’s fitness to adopt agile development.

Section 6.3.1 – Tough questions:

I liked this section because of the example of typical questions asked by the team that will be adopting agile development and the possible answers to give about the migration process.

Section 7.1.1 – Attributes of a good coach:

This section contains a very good breakdown of the attributes that should be sought when selecting an agile coach. The following sections go into further detail about training and coaching; as well as details on the interaction with managers and stakeholders.

Section 7.3 – Creating a team with an agile mindset:

This section explains what ingredients are needed to create a sound agile team; from “Culture and roles” through to “Characteristics that influence individual performance”.

Chapter 8 – Injecting agility into your current process:

Just the title alone gives the idea of the maturity of mind that the authors have when confronting a company wanting to adopt agile development methods. From documenting the existing process (not everyone has it written in stone and perhaps not all aspects of the company are actually doing what it says in the process hand-book); to deciding what to keep and what to change.

Chapters 9, 10 and 11

Discuss the selection of a pilot project and the team that will be on the pilot project. This is very informative for those companies that have made up their mind to move to agile and are looking around for the best project to guinea pig.

Section 12.3 – Feature cards compared to…:

As a Scrum practitioner it was interesting to read the comparisons between feature cards, user stories, use cases and functional specifications.

Section 12.5 – Hard-copy vs. electronic cards:

I found this section particularly useful as it highlighted the benefits of both, using SharePoint as an example of the electronic format. Although the idea is to keep it as simple as possible I am an advocate of using technology where possible – more than anything to cut down on the waste generated by so much paper; but also for the fact that you can view the information at any time (this is effective when trying to explain PBI or SBI to persons not located in the same area).

Chapter 13 – Prioritizing the backlog:

This is a subject that I find is the least covered in discussions with peers (both at work and outside) and it is really helpful to see the suggestions posed throughout the chapter. Peppered with examples of a product backlog and how to handle various items; this chapter is a good starting point for broadening the discussion.

Chapter 14 – Estimating at the right level with the right people:

I still remember being asked to estimate for a particular requirement when I still felt that I did not have enough information to do a sound enough job of it. How different it is when you are doing it as a team and everyone can voice their thoughts on the matter. This chapter is particularly useful for those still in “traditional” methods or recent adopters of agile methods. Of particular interest is section 14.1 – Contrasting traditional and agile estimation techniques and section 14.2 – The importance of whole-team estimation; both of which ease the reader into the subject that is quite explanatory with regards to the estimation process in agile.

Section 16.3 – Identifying and estimating tasks:

This section has a nice “call-out” titled “Task assignments aren’t permanent” that describes how the early stages, specialist people will be suited to certain tasks and how with the passage of time and acquired experience in an agile environment team members will become more capable of taking on varied tasks that perhaps were out of their realm in the beginning.

The latter chapters of the book, starting with chapter 20, describe the process of adapting to change and learning as we go. Section 20.1 – Common reasons for adapting (i.e. make those tweaks mentioned earlier); describes common issues that crop up and some ways that teams adapt to the situation described.
Chapter 22 is particularly interesting with a detailed explanation on how to prepare and manage the retrospective.
Chapter 23 is a must read for companies that have successfully started a pilot project and are now looking at how to push across the divide and bring about change at an enterprise level.

As I stated at the beginning of this article, I would seriously make this book required reading for those that are at all interested in adopting agile or have just started – it is clear, concise and has plenty of example scenarios that many individuals and corporations would identify with.

  • Share/Bookmark