Testing with Managed Extensibility Framework (MEF) for Silverlight

In the previous post, we developed test strategies for a three tiers, MVVM based Silverlight application, which uses the Managed Extensibility Framework (MEF) in order to loosen the coupling between the components and to provide better testability and extensibility.

In this post, we’ll see how the Managed Extensibility Framework (MEF) fit in with the overall testing strategy and try to figure out the best way to use it in the application in order to provide the best degree of testability.

To MEF or not to MEF (during tests)?

When coming to test MEF based application, the first question that comes to mind is whether to use the MEF composition container to instantiate dependencies during tests.

In component tests the answer is usually NO, in order to provide good degree of separation of concerns we need to isolate the component as much as possible, following that we should not let MEF interfere with the tests.

However, in some cases, the tested functionality uses MEF in order to provide special features such as downloading and loading xaps on the fly, in such cases we’ll need to put MEF into action and allow it to instantiate the dependencies. Since we’ll still need to replace the other components in the application with test doubles (fakes, mock, dummies, etc) – we’ll need to override the default composition container with a custom container that includes the real component under test and fake every thing else.

Don’t use the static composition container in the application

Since we need to exclude MEF in some (most) tests and to replace its composition container in others – we can’t use the default static composition container that MEF provides by default. Here’s why:

1) In order to replace the default static container we need to call CompositionHost.Initialize(container) that cannot be called more than once, which means that if more than one test will try to replace the container we’ll experience an ‘InvalidOperationException’ (this is a good example for the shared fixture anti-pattern) 

2) Even if we wouldn’t have to include MEF in any of our tests, when using the static container the CompositionInitializer.SatisfyImports (that put MEF into action) can be called from any class in the application, which means that we might not be not be able to exclude MEF in some of the tests 

Create composition container instance and call SatifyImportsOnce on the container itself

So what can we do? In order to retire ourselves from the static container, the application should create CompositionContainer object upon initialization, add the appropriate catalogs and/or the appropriate objects (via ComposePart) to the container, and call SatifyImportsOnce on the composition container itself in order to satisfy the imports of a given instance (usually the root object).

Allow replacing the container instance  with a test double

If internal classes in the application need to satisfy imports on the fly, we need to provide them with a way to access the global composition container such that we can replace the calls to the container with an empty implementation during tests.

In order to accomplish that, we’ll encapsulate the CompositionContainer with CompositionContainerService class that is accessible only though ICompositionService interface. We’ll introduce a new singleton called CompositionServiceProvide that will be shared widely and expose ICompositionService interface to enable access to the CompositionContainerService. The CompositionServiceProvide will allow replacing the CompositionContainerService with a dummy test double through the ICompositionServiceProviderTesting interface (implemented explicitly).

Here’s the code for the CompositionContainerService :

   1: public class CompositionContainerService : ICompositionService

   2: {

   3:     private readonly CompositionContainer m_container;

   4:     private readonly AggregateCatalog m_aggregateCatalog;

   5:  

   6:     public CompositionContainerService()

   7:     {

   8:         m_aggregateCatalog = new AggregateCatalog();

   9:         m_container = new CompositionContainer(m_aggregateCatalog);

  10:         

  11:     }

  12:  

  13:     public void SatisfyImports(object o)

  14:     {

  15:         m_container.SatisfyImportsOnce(o);

  16:     }

  17:  

  18:     public void ComposeParts(params object[] attributedParts)

  19:     {

  20:         m_container.ComposeParts(attributedParts);

  21:     }

  22:  

  23:     public void AddCatalog(ComposablePartCatalog catalog)

  24:     {

  25:         m_aggregateCatalog.Catalogs.Add(catalog);

  26:     }

  27:  

  28:     public CompositionContainer Container

  29:     {

  30:         get { return m_container; }

  31:     }

  32: }

Here’s the code for the singleton CompositionServiceProvider :

   1: public interface ICompositionServiceProviderTesting

   2: {

   3:     void SetCompositionService(ICompositionService compositionService);

   4: }

   5:  

   6: public class CompositionServiceProvider : ICompositionServiceProviderTesting

   7: {

   8:     static readonly CompositionServiceProvider s_instance = new CompositionServiceProvider();

   9:  

  10:     public static CompositionServiceProvider Instance

  11:     {

  12:         get { return s_instance; }

  13:     }

  14:  

  15:     private ICompositionService m_compositionService;

  16:  

  17:     public ICompositionService Service

  18:     {

  19:         get

  20:         {

  21:             if (m_compositionService == null)

  22:             {

  23:                 m_compositionService = new CompositionContainerService();

  24:             }

  25:  

  26:             return m_compositionService;

  27:         }

  28:     }

  29:  

  30:     void ICompositionServiceProviderTesting.SetCompositionService(ICompositionService compositionService)

  31:     {

  32:         m_compositionService = compositionService;

  33:     }

  34: }

 

Using MEF during tests

With MEF for Silverlight, developers can take advantage of the new DeploymentCatalog for downloading only a portion of the code when the user connect, and adding code (downloading extra xap files) on demand as the application makes progress. When a new xap is downloaded to the client browser and loaded into the container, MEF searches the xap for exports and start a process of re-satisfying the imports.

We’ll add the required functionality for downloading (AddXap) and loading (LoadXap) xaps to the container into the CompositionServiceProvider (described above)

   1: public class CompositionServiceProvider : ICompositionServiceProviderTesting

   2: {

   3:     ...

   4:  

   5:     readonly Dictionary<string, DeploymentCatalog> m_catalogs = new Dictionary<string, DeploymentCatalog>();

   6:  

   7:     public void AddXap(string uri, EventHandler<AsyncCompletedEventArgs> downloadCompletedCallback)

   8:     {

   9:         var catalog = new DeploymentCatalog(uri);

  10:         m_catalogs.Add(uri, catalog);

  11:  

  12:         catalog.DownloadCompleted += downloadCompletedCallback;

  13:         catalog.DownloadAsync();

  14:     }

  15:  

  16:     public void LoadXap(string uri)

  17:     {

  18:         DeploymentCatalog catalog = m_catalogs[uri];

  19:         Service.AddCatalog(catalog);

  20:  

  21:     }

  22:  

  23: }

Take the following application for example, when the user connect it’s presented with a basic UI that allows initiating basic activities, while the user is looking at the screen, another xap that includes an extra user control is downloaded in the background. When the user click on the ‘Extension’ button, the second xap (given that it is already available on the client) is loaded into the container, and as a result a dedicated property on the VIew that is configured to import the new user control (according to its special metadata) is automatically populated. Once the property is populated with the new user control, the View takes care of displaying the user control on the gray panel. 

Here’s the application:

Here’s the View code:

   1: [Export]

   2:  public partial class BooksPage : Page, IPartImportsSatisfiedNotification

   3:  {

   4:      [ImportingConstructor]

   5:      public  BooksPage(BooksViewModel viewModel)

   6:      {

   7:          InitializeComponent();

   8:  

   9:          DataContext = viewModel;

  10:      }

  11:      

  12:      [ImportMany(AllowRecomposition=true)]

  13:      public Lazy<UserControl, IWidgetMetadata>[] Widgets { get; set; }

  14:  

  15:      private void extentionButton_Click(object sender, System.Windows.RoutedEventArgs e)

  16:      {

  17:          // Load the xap (which as an effect will also re-satisfy the dependencies)

  18:          CompositionServiceProvider.Instance.LoadXap("PrototypeMVVM.Extentions.xap");

  19:      }

  20:  

  21:      public void OnImportsSatisfied()

  22:      {

  23:          wrapPanel1.Children.Clear();

  24:  

  25:          foreach (var widget in Widgets)

  26:          {

  27:              if (widget.Metadata.Location == WidgetLocation.Bottom)

  28:              {

  29:                  wrapPanel1.Children.Add(widget.Value);

  30:              }

  31:          }

  32:      }

  33:  }

In order to verify that once the ‘Extension’ button is clicked the appropriate xap is loaded and the View is updated appropriately -we need to take the following steps;

1)  Replace the original container with a new container

2)  Create a real View, real ViewModel, fake DAL and add them to the container via ComposePart (notice that we didn’t use static TypeCatalog in order to configure the container, It’s simpler to simply add the objects that are required to satisfy the imports)

3) Download the second xap and wait until the download complete.

4) Simulate click on the ‘Extension’ button

5) Verify that the new user control  is displayed

Here’s the test code:

   1: [TestMethod]

   2: [Asynchronous]

   3: public void CheckCompositionAfterPackageLoad()

   4: {

   5:     // Create the dependencies and add them to the container

   6:     var dalFake = new BooksDataServiceProviderFake();

   7:     var viewModel = new BooksViewModel(dalFake);

   8:     var view = new BooksPage(viewModel);

   9:  

  10:     // *** UsersWindow is a lazy import that is not being used in the test

  11:     // nevertheless, we need to instantiate it (or add it to TypeCatalog) and 

  12:     // its dependencies (and there dependencies) and add them to the container

  13:     var usersWindow = new UsersWindow();

  14:     var usersViewModel = new UsersViewModel(new UsersDataServiceProviderFake());

  15:  

  16:     var container = CompositionServiceProvider.Instance.Service.Container;

  17:     container.ComposeParts(view, usersWindow, usersViewModel);

  18:     container.SatisfyImportsOnce(view);

  19:  

  20:     TestPanel.Children.Add(view);

  21:  

  22:     // download the extension package

  23:     const string extentionsXap = "PrototypeMVVM.Extentions.xap";

  24:     CompositionServiceProvider.Instance.AddXap(extentionsXap, (sender, args) => m_xapDownloaded = true);

  25:  

  26:     EnqueueConditionalTimeoutChecker timeoutChecker = new EnqueueConditionalTimeoutChecker();

  27:  

  28:     // wait for download the complete

  29:     EnqueueConditional(() =>

  30:     {

  31:         timeoutChecker.Check();

  32:  

  33:         return m_xapDownloaded;

  34:     });

  35:  

  36:     // Click on the extention button

  37:     TestApi.EnqueueButtonClick(view.ExtensionButton, this);

  38:  

  39:     // since we only loading the xap and not downloading it, the UI re-composition should 

  40:     // happen synchronously

  41:     EnqueueCallback(() => Assert.AreEqual(1, view.WrapPanel1.Children.Count));

  42:     

  43:     EnqueueTestComplete();

  44: }

Testing Silverlight 4.0 App with MVVM, MEF and WCF Data Services

In the previous post, we reviewed a simple 3 tiers Silverlight application that follows the MVVM design pattern, uses MEF for applying the dependency injection pattern, and uses WCF data service implemented using a custom data provider for CRUDing data in a RESTfull way (via HTTP).

In this post, we’ll put together a testing strategy for the application and demonstrate testing in component level and multi-component level.

In next posts, we’ll see how MEF composition container fit in with the overall strategy.

Lets start by reintroducing the application.

The Application

The front end

Presented above is the front end of a book store app that displays list of books available in a remote storage and enables the user to change books description and save the changes. In addition, Clicking on the ‘Users’ button opens a new child window the displays read only list of users that are authorized for making changes.   

High level design

Here’s the application high level design.

This design presented above is a typical way of implementing a data driven, multi tier app when using WPF/Silverlight tech with MVVM. 

Most frequently, every View is attached to a ViewModel that encapsulates related presentation logic and state. The ViewModel interact with one or more data providers in order to query/manipulate distant data, the data providers interact with remote data service in a RESTfull way (via HTPP) through WCF data services, the data service encapsulates a data model (in the case, the BookStore class hierarchy) that is also exposed to the client tier through auto generated proxies and enable CRUDing though repository abstraction.

Now let’s get in to details and see how we’re going to test this application.

Component tests

Testing the View (UI controls sub-layer)

The industry experience in testing client driven applications led to the conclusion that instead of struggling with UI Automation, it's better to develop ultra thin Views that can be excluded during automatic tests without loosing much test coverage. The idea is to test all presentation concerns though a non visual mediator component (such as the presenter in MVP or the controller in MVC) and to leave UI layout tests for manual testing.

This approach makes much sense for windows based (winforms) apps since the potential for getting the binding between the visual components (Views) and the mediator components wrongs was minimal. It makes less sense for xaml based apps (such as in Silverlight) since the binding between the View and ViewModel (the mediator) is extremely loosed. That being said, UI automation for Silverlight controls has proven to have low ROI (Return of Investment) due to the un-structural nature of xaml controls and lack of support for testing in Silverlight itself. One can try to control the UI programmatically (examples available in the attached code) or using UI Automation frameworks like CodedUI, but both techniques produces code that is hard to write and harder to maintain, especially in the first phases of the development when the UI layout changes frequently.

Testing the ViewModel (presentation logic sub-layer)

The ViewModel handles presentation logic and state, mainly by querying/commanding the data providers and maintaining presentation state, which is reflected on the View surface via binding. In order to test the ViewModel in isolation and provide clear separation of concerns, we’ll exclude the View and use fake DAL.

Here’s the top level workflows:

In one direction, the workflow starts by changing the ViewModel (e.g. change the description of one of the books), simulating an action (e.g. simulate a click on the ‘Save’ button), and verifying that the DAL has change appropriately (e.g. verify that the matching book entity has changed appropriately).

In the opposite direction, the workflow starts by changing the DAL and verifying against the ViewModel.

And here’s the test code for the first workflow:

   1: [TestMethod]

   2: public void ChanegViewModelCheckFakeDal()

   3: {

   4:     var providerFake = new BooksDataServiceProviderFake();

   5:     var booksViewModel = new BooksViewModel(providerFake);

   6:  

   7:     const string description = "aa";

   8:     

   9:     // Change ViewModel state

  10:     booksViewModel.Books[0].Description = description;

  11:  

  12:     booksViewMode.SaveCommand.Execute(null);

  13:  

  14:     // No need to wait for a-sync response, we use fake DAL

  15:     providerFake.GetBooks((sender, args) => m_books = args.Result);

  16:  

  17:     // Verify that fake DAL has changed appropriately

  18:     Assert.AreEqual(description, m_books.ToList()[0].Description);

  19: }

And in the opposite direction

   1: [TestMethod]

   2: public void ChangeFakeDalCheckViewModel()

   3: {

   4:     var providerFake = new BooksDataServiceProviderFake();

   5:     var booksViewModel = new BooksViewModel(providerFake);

   6:  

   7:     // No need to wait for a-sync response, we use fake DAL

   8:     providerFake.GetBooks((sender, args) => m_books = args.Result);

   9:     const string description = "aa";

  10:  

  11:     // Change DAL data 

  12:     m_books.ToList()[0].Description = description;

  13:  

  14:     // Refresh the ViewModel

  15:     booksViewModel.Refresh();

  16:  

  17:     // Verify that the ViewModel has changed appropriately

  18:     Assert.AreEqual(description, booksViewModel.Books[0].Description);

  19: }

 

Multi component tests

Testing ViewModel & Dal

There’re many reasonable workflows for multi component tests. The most common one is to initiate the workflow from the VIewModel, wait for changes to propagate to the underlying DAL and for the response to return to the ViewModel, and then validate against the ViewModel. Another way to go it to change the ViewModel and validate against the DAL, and in the other direction change the DAL and validate against the ViewModel.

We will demonstrate the first workflow:

The workflow starts by changing the state of the ViewModel (change the description of a book), initiating a Command or other method call/s that drive an interesting workflow (execute the Save command), waiting for the DAL to finish executing the command (raise the BookSaved event), and finally verifying that the ViewModel has changed appropriately.

Since the goal of this test is to validate the ViewModel and DAL (both client side components) - we use faked data service, however, In many cases it makes more sense to use the real data service, The decision whether to use fake service or real one usually depends on the complexity of the service and the required test coverage. In some cases the real service is heavy weight or coupled to other heavy weight services that cannot be faked, thus it makes more sense to fake it. Some applications require more test coverage, so faking the service enables more control of its behavior.

Here’s the test code:

   1: [TestClass]

   2: [Tag("Test business and presentation logic")]

   3: public class MultiComponentTests : SilverlightTest

   4: {

   5:     private bool m_booksSaved;

   6:     private BooksViewModel m_booksViewModelChanged;

   7:  

   8:     [TestMethod]

   9:     [Asynchronous]

  10:     public void ChangeViewModelCheckViewModel()

  11:     {

  12:         var dataServiceFakeUri =

  13:             new Uri("http://localhost:21978/BookStoreDataServiceFake.svc");

  14:  

  15:         var configuration =

  16:             new DalConfigurationFake { BooksDataServiceRoot = dataServiceFakeUri };

  17:  

  18:         var provider = new BooksDataServiceProvider(configuration);

  19:  

  20:         BooksViewModel booksViewModel = new BooksViewModel(provider);

  21:   

  22:         // Wait until the viewModel books collection is populated with default data

  23:         EnqueueConditional(() => CheckBooksArrived(booksViewModel));

  24:  

  25:         const string description = "aa";

  26:         

  27:         // Change ViewModel state and save changes

  28:         EnqueueCallback

  29:          (

  30:             () =>

  31:                 {

  32:                     // Change ViewModel state

  33:                     booksViewModel.Books[0].Description = description;

  34:  

  35:                     // Execute the save command

  36:                     booksViewModel.SaveCommand.Execute(null);

  37:  

  38:                 }

  39:          );

  40:  

  41:         booksViewModel.BooksSaved += (sender, args) => m_booksSaved = args.Result;

  42:  

  43:         // Wait until save request returns

  44:         EnqueueConditional

  45:             (

  46:                 () => m_booksSaved

  47:             );

  48:  

  49:         // Create new fresh ViewModel after changes have been saved

  50:         EnqueueCallback

  51:             (

  52:                 () => m_booksViewModelChanged = new BooksViewModel(provider)

  53:             );

  54:  

  55:         // Wait until the viewModel books collection is populated with the new data

  56:         EnqueueConditional(() => CheckBooksArrived(m_booksViewModelChanged));

  57:  

  58:         EnqueueCallback

  59:             (

  60:                  // Verify that the new ViewModel has changed as appropriate

  61:                 () => Assert.AreEqual(description, m_booksViewModelChanged.Books[0].Description)

  62:             );

  63:  

  64:         // Flush

  65:         EnqueueTestComplete();

  66:     }

  67:  

  68:     private static bool CheckBooksArrived(BooksViewModel viewModel)

  69:     {

  70:         return viewModel.Books != null;

  71:     }

  72: }

We created a data provider that target a fake data service, created a ViewModel, waited for it to load, changed the description of the first book, saved the changes and waited for the Save operation to complete.

Now, we need to validate that the changes that were made on the underlying DAL are reflected on the ViewModel. For this, we need to clean the ViewModel and load it again with books from the DAL. Since the ViewModel doesn’t expose some kind of a clean method nor allow changing the Books collection from the outside - we created a new ViewModel, waited for it to load and verified that the book has changed appropriately.

Notice that either the ViewModel or the underlying data provider must support raising a special event to notify that the a-sync save operation has completed, in our case, the ViewModel exposes the SavedEvent (that is used only by test code). This is one of the testability requirements that we must insist on during the design process (it falls under the ‘Observeability’ sub category).

End2End tests

To complete the picture, we need to have tests that run end to end, with real View (optionally), real ViewModel, real DAL and real data service. These tests run in lower frequency and are much harder to write and maintain. If we have good coverage in our component and multi component tests, we can cut down on these tests and still maintain good or even better quality.

Please keep in mind that this test strategy is recommended for medium to large applications that are planed to last no less than 4 years (thus requires good test coverage). Many other applications might be better served with only unit tests and end2end tests.

The source code can be downloaded from here.