In the previous post, we reviewed a simple 3 tiers Silverlight application that follows the MVVM design pattern, uses MEF for applying the dependency injection pattern, and uses WCF data service implemented using a custom data provider for CRUDing data in a RESTfull way (via HTTP).
In this post, we’ll put together a testing strategy for the application and demonstrate testing in component level and multi-component level.
In next posts, we’ll see how MEF composition container fit in with the overall strategy.
Lets start by reintroducing the application.
The Application
The front end
Presented above is the front end of a book store app that displays list of books available in a remote storage and enables the user to change books description and save the changes. In addition, Clicking on the ‘Users’ button opens a new child window the displays read only list of users that are authorized for making changes.
High level design
Here’s the application high level design.
This design presented above is a typical way of implementing a data driven, multi tier app when using WPF/Silverlight tech with MVVM.
Most frequently, every View is attached to a ViewModel that encapsulates related presentation logic and state. The ViewModel interact with one or more data providers in order to query/manipulate distant data, the data providers interact with remote data service in a RESTfull way (via HTPP) through WCF data services, the data service encapsulates a data model (in the case, the BookStore class hierarchy) that is also exposed to the client tier through auto generated proxies and enable CRUDing though repository abstraction.
Now let’s get in to details and see how we’re going to test this application.
Component tests
Testing the View (UI controls sub-layer)
The industry experience in testing client driven applications led to the conclusion that instead of struggling with UI Automation, it's better to develop ultra thin Views that can be excluded during automatic tests without loosing much test coverage. The idea is to test all presentation concerns though a non visual mediator component (such as the presenter in MVP or the controller in MVC) and to leave UI layout tests for manual testing.
This approach makes much sense for windows based (winforms) apps since the potential for getting the binding between the visual components (Views) and the mediator components wrongs was minimal. It makes less sense for xaml based apps (such as in Silverlight) since the binding between the View and ViewModel (the mediator) is extremely loosed. That being said, UI automation for Silverlight controls has proven to have low ROI (Return of Investment) due to the un-structural nature of xaml controls and lack of support for testing in Silverlight itself. One can try to control the UI programmatically (examples available in the attached code) or using UI Automation frameworks like CodedUI, but both techniques produces code that is hard to write and harder to maintain, especially in the first phases of the development when the UI layout changes frequently.
Testing the ViewModel (presentation logic sub-layer)
The ViewModel handles presentation logic and state, mainly by querying/commanding the data providers and maintaining presentation state, which is reflected on the View surface via binding. In order to test the ViewModel in isolation and provide clear separation of concerns, we’ll exclude the View and use fake DAL.
Here’s the top level workflows:
In one direction, the workflow starts by changing the ViewModel (e.g. change the description of one of the books), simulating an action (e.g. simulate a click on the ‘Save’ button), and verifying that the DAL has change appropriately (e.g. verify that the matching book entity has changed appropriately).
In the opposite direction, the workflow starts by changing the DAL and verifying against the ViewModel.
And here’s the test code for the first workflow:
1: [TestMethod]2: public void ChanegViewModelCheckFakeDal()
3: {4: var providerFake = new BooksDataServiceProviderFake();
5: var booksViewModel = new BooksViewModel(providerFake);
6: 7: const string description = "aa";
8: 9: // Change ViewModel state
10: booksViewModel.Books[0].Description = description; 11: 12: booksViewMode.SaveCommand.Execute(null); 13: 14: // No need to wait for a-sync response, we use fake DAL
15: providerFake.GetBooks((sender, args) => m_books = args.Result); 16: 17: // Verify that fake DAL has changed appropriately
18: Assert.AreEqual(description, m_books.ToList()[0].Description); 19: }And in the opposite direction
1: [TestMethod]2: public void ChangeFakeDalCheckViewModel()
3: {4: var providerFake = new BooksDataServiceProviderFake();
5: var booksViewModel = new BooksViewModel(providerFake);
6: 7: // No need to wait for a-sync response, we use fake DAL
8: providerFake.GetBooks((sender, args) => m_books = args.Result);9: const string description = "aa";
10: 11: // Change DAL data
12: m_books.ToList()[0].Description = description; 13: 14: // Refresh the ViewModel
15: booksViewModel.Refresh(); 16: 17: // Verify that the ViewModel has changed appropriately
18: Assert.AreEqual(description, booksViewModel.Books[0].Description); 19: }Multi component tests
Testing ViewModel & Dal
There’re many reasonable workflows for multi component tests. The most common one is to initiate the workflow from the VIewModel, wait for changes to propagate to the underlying DAL and for the response to return to the ViewModel, and then validate against the ViewModel. Another way to go it to change the ViewModel and validate against the DAL, and in the other direction change the DAL and validate against the ViewModel.
We will demonstrate the first workflow:
The workflow starts by changing the state of the ViewModel (change the description of a book), initiating a Command or other method call/s that drive an interesting workflow (execute the Save command), waiting for the DAL to finish executing the command (raise the BookSaved event), and finally verifying that the ViewModel has changed appropriately.
Since the goal of this test is to validate the ViewModel and DAL (both client side components) - we use faked data service, however, In many cases it makes more sense to use the real data service, The decision whether to use fake service or real one usually depends on the complexity of the service and the required test coverage. In some cases the real service is heavy weight or coupled to other heavy weight services that cannot be faked, thus it makes more sense to fake it. Some applications require more test coverage, so faking the service enables more control of its behavior.
Here’s the test code:
1: [TestClass]2: [Tag("Test business and presentation logic")]
3: public class MultiComponentTests : SilverlightTest
4: {5: private bool m_booksSaved;
6: private BooksViewModel m_booksViewModelChanged;
7: 8: [TestMethod] 9: [Asynchronous]10: public void ChangeViewModelCheckViewModel()
11: { 12: var dataServiceFakeUri =13: new Uri("http://localhost:21978/BookStoreDataServiceFake.svc");
14: 15: var configuration =16: new DalConfigurationFake { BooksDataServiceRoot = dataServiceFakeUri };
17: 18: var provider = new BooksDataServiceProvider(configuration);
19: 20: BooksViewModel booksViewModel = new BooksViewModel(provider);
21: 22: // Wait until the viewModel books collection is populated with default data
23: EnqueueConditional(() => CheckBooksArrived(booksViewModel)); 24: 25: const string description = "aa";
26: 27: // Change ViewModel state and save changes
28: EnqueueCallback 29: ( 30: () => 31: {32: // Change ViewModel state
33: booksViewModel.Books[0].Description = description; 34: 35: // Execute the save command
36: booksViewModel.SaveCommand.Execute(null);
37: 38: } 39: ); 40: 41: booksViewModel.BooksSaved += (sender, args) => m_booksSaved = args.Result; 42: 43: // Wait until save request returns
44: EnqueueConditional 45: ( 46: () => m_booksSaved 47: ); 48: 49: // Create new fresh ViewModel after changes have been saved
50: EnqueueCallback 51: (52: () => m_booksViewModelChanged = new BooksViewModel(provider)
53: ); 54: 55: // Wait until the viewModel books collection is populated with the new data
56: EnqueueConditional(() => CheckBooksArrived(m_booksViewModelChanged)); 57: 58: EnqueueCallback 59: (60: // Verify that the new ViewModel has changed as appropriate
61: () => Assert.AreEqual(description, m_booksViewModelChanged.Books[0].Description) 62: ); 63: 64: // Flush
65: EnqueueTestComplete(); 66: } 67: 68: private static bool CheckBooksArrived(BooksViewModel viewModel)
69: {70: return viewModel.Books != null;
71: } 72: }We created a data provider that target a fake data service, created a ViewModel, waited for it to load, changed the description of the first book, saved the changes and waited for the Save operation to complete.
Now, we need to validate that the changes that were made on the underlying DAL are reflected on the ViewModel. For this, we need to clean the ViewModel and load it again with books from the DAL. Since the ViewModel doesn’t expose some kind of a clean method nor allow changing the Books collection from the outside - we created a new ViewModel, waited for it to load and verified that the book has changed appropriately.
Notice that either the ViewModel or the underlying data provider must support raising a special event to notify that the a-sync save operation has completed, in our case, the ViewModel exposes the SavedEvent (that is used only by test code). This is one of the testability requirements that we must insist on during the design process (it falls under the ‘Observeability’ sub category).
End2End tests
To complete the picture, we need to have tests that run end to end, with real View (optionally), real ViewModel, real DAL and real data service. These tests run in lower frequency and are much harder to write and maintain. If we have good coverage in our component and multi component tests, we can cut down on these tests and still maintain good or even better quality.
Please keep in mind that this test strategy is recommended for medium to large applications that are planed to last no less than 4 years (thus requires good test coverage). Many other applications might be better served with only unit tests and end2end tests.
The source code can be downloaded from here.
Great! Thank you very much! So clear description I've never met before and even coudn't imagine
ReplyDelete