Design Codes

Wednesday, October 13, 2010

Developing Silverlight 4.0 Three Tiers App with MVVM and WCF Data Services (OData via ATOM)

In this post we’ll walk through the process of developing a three tiers, rich web application using Silverlight 4.0. As many applications built on top of WPF/Silverlight technologies, we’ll based the design on MVVM design pattern, and we’ll use WCF data services to query/update data exposed by a mid tier application.

In the next posts we’ll add the new Managed Extensibility Framework (MEF) to the application, and put together a testing strategy.

Silverlight 4.0

Silverlight introduces a quick and simple way to develop rich web applications. As oppose to traditional Web applications (developed with ASP.NET for instance), Silverlight is a client-side runtime environment, which means that the entire application is downloaded and run on the client machine, resulting a great user experience. With Silverlight, developers design xaml pages and target a managed API that is based on a subset of the .NET Framework (.NET Framework for Silverlight), traditionally using visual studio with C# and Visual Basic.

When the user connect to a Silverlight application, a .xap file that contains the application code and resources is being downloaded from the remote Silverlight host into the client machine. Since this can take quite a while, developers try to keep the xap file as small as possible by applying different patterns that aim to reduce the application code base.

When using MEF for Silverlight, developers can take advantage of the new DeploymentCatalog (previously PackageCatalog) that enables downloading only a portion of the code when the user connect, and adding code (downloading extra xap files) on demand as the application make progress.

MVVM

MVVM pattern, much like its equivalents MVC/P/X, separate the UI and the underlying presentation and business logic into three separate classes. The view, encapsulates the user interface and user interface logic. The view model encapsulates presentation logic and state, and the model encapsulates the application's business logic and data.

The view interacts with the view model, typically through data binding, commands, and change notification events. The view model, in turn, interacts with the model, acting as an intermediary between the view and the model and performing any necessary model-level data conversion and aggregation.

Silverlight/WPF provide power full binding infrastructure that makes MVVM a preferred choice for almost every application. In most cases, the ViewModel is assigned as the DataContext of its pair View, allowing every control on the View to easily bind any of its properties to a matching property on the ViewModel. In addition,Instead of implementing event handlers (such as button click) in the VIew code behind, the View can bind to Command objects exposed by its ViewModel.

In the other direction, the ViewModel notifies the View of any state changes via change notification events through INotifyPropertyChanged and INotifyCollectionChanged interfaces, and about data validation via the IDataErrorInfo or INotifyDataErrorInfo interfaces.

Often enough, Model objects implement the INotifyPropertyChanged, INotifyCollectionChanged and IDataErrorInfo interfaces in order to notify the View on state change on their side, since the Views have no direct link the the Models, the notifications are being propagated from the Model to the View through the ViewModel .

While implementing the MVVM pattern, we need to figure out whether the View instantiate the VIewModel on the code behind, or whether a third-party component hook the ViewModel to the View. If you’re looking for an answer in MVVM spec, you will not find it there! that’s for you to decide after looking at the overall application architecture. The natural way is to let the View create its ViewModel, however, applications that use Dependency injection technique (to loosen the coupling between the components ) will prefer to inject the ViewModel (perhaps by using dependency injection container) to the View.

WCF RIA Services

Typical Silverlight application runs solely on the client machine, unlike traditional Web applications, there’re no automatic round trips to the server for fetching/submitting data and for updating the display. However, many Silverlight applications need to establish communication with back end server/s, for instance, to initiate remote action/calculation or to query/manipulate some distanced data. To accomplish this, one can make use of the WCF RIA Services technology that provides framework components, tools, and services that make the application logic on the server available to the Silverlight client without having to manually duplicate that programming logic.

WCF Data Services

While the WCF RIA Services was designed specifically for end-to-end Silverlight solutions (where client & server are designed and deployed together), many Silverlight applications require loosely coupled services that can be targeted/reused by other applications.

With WCF Data Services, developers design services that are accessible through a REST-based (HTTP) protocol called OData (www.odata.org) , thus can be consumed by different applications of different platform (Java, PHP, .NEY, etc.).

The WCF Data Services environment provides .NET developers with a server framework for creating REST-based data-centric web services on top of data model with appropriate security, and client libraries for accessing those services.

WCF Data Services can expose data (as OData feeds) that originates from various sources. One can create an OData based service by using an ADO.NET Entity Framework data model, by using the LinqToSql technology, or by using custom provider that uses standard CLR classes. In this walkthrough, we’ll implement a custom provider that exposes a data model that describes a book store, and support update operations.

WCF Data Services v.s. WCF RIA Services

Obviously, there’s an overlap between RIA Services for Silverlight and WCF Data Services support for Silverlight. So basic guideline is to use WCF RIA Services for traditional operation-based services and use WCF Data Services for a more REST-based approach. WCF RIA Services is supposed to eventually also support OData.

WCF Data Services v.s. WCF Services

While WCF Data Services/OData is intended for services that primarily expose data with few (if any) service operations, straight WCF is more applicable for services that primarily provide service operations with data being only a small consideration.

The Application - Client Side

In this walkthrough, we’ll dive through the implementation of the following 3-tiers application:

The application includes a simple View that presents the books that are available in the book store. A mid tier based on WCF data services that allows querying/updating books, and make use of a repository that pull out the books from the data storage and allows updating the metadata associated with the books.

Lets see how the main parts in the application collaborate.

The workflow starts from the ViewModel, which makes queries/performs updates on the DalService, which propagate the call to the DataServiceProvider, which send http GET/UPDATE to the data service, that propagate the request/command to the repository.

Let see how the application main end2end sequence (getting books from the data storage) looks like:

As the sequence describes, the View (BooksPage) prompt the ViewModel to refresh its state, in turn, the ViewModel calls GetBook on the DelService, passing it with a delegate to be invoked when the books arrive. The DalService propagate the request to the DataServiceProvider (through its abstraction), which construct a query and pass it to the web service over HTTP. The web service turn to the repository (through its abstraction) that fetch the data from the storage and return query able collection of books. The result is being transferred back to the client application, and travels all the way back to the ViewModel, which in turn update its internal books collection and notify the BooksPage that its state has changed through the INotifyPropertyChanged interface.

Since the data service is REST based, it is being queried using standard http GET request. In response, it returns the data as an Atom-Pub feed. Lets look at the generated http traffic using our good friend Fiddler.

The Request Header:

   1: GET /BookStoreDataService.svc/BooksGET /BookStoreDataService.svc/Books HTTP/1.1 HTTP/1.1

The Response Body:

   1: HTTP/1.1 200 OK   2: Server: ASP.NET Development Server/10.0.0.0   3: Date: Wed, 13 Oct 2010 11:03:36 GMT   4: X-AspNet-Version: 4.0.30319   5: DataServiceVersion: 1.0;   6: Content-Length: 2023   7: Cache-Control: no-cache   8: Content-Type: application/atom+xml;charset=utf-8   9: Connection: Close  10:    11: <?xml version="1.0" encoding="utf-8" standalone="yes"?>  12: <feed xml:base="http://127.0.0.1:16081/BookStoreDataService.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">  13:   <title type="text">Books</title>  14:   <id>http://127.0.0.1:16081/BookStoreDataService.svc/Books</id>  15:   <updated>2010-10-13T11:03:36Z</updated>  16:   <link rel="self" title="Books" href="Books" />  17:   <entry>  18:     <id>http://127.0.0.1:16081/BookStoreDataService.svc/Books('Book1')</id>  19:     <title type="text"></title>  20:     <updated>2010-10-13T11:03:36Z</updated>  21:     <author>  22:       <name />  23:     </author>  24:     <link rel="edit" title="Book" href="Books('Book1')" />  25:     <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Authors" type="application/atom+xml;type=feed" title="Authors" href="Books('Book1')/Authors" />  26:     <category term="PrototypeMVVM.Web.Core.Models.Book" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />  27:     <content type="application/xml">  28:       <m:properties>  29:         <d:Name>Book1</d:Name>  30:         <d:Description>Aviad</d:Description>  31:       </m:properties>  32:     </content>  33:   </entry>  34:   <entry>  35:     <id>http://127.0.0.1:16081/BookStoreDataService.svc/Books('Book2')</id>  36:     <title type="text"></title>  37:     <updated>2010-10-13T11:03:36Z</updated>  38:     <author>  39:       <name />  40:     </author>  41:     <link rel="edit" title="Book" href="Books('Book2')" />  42:     <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Authors" type="application/atom+xml;type=feed" title="Authors" href="Books('Book2')/Authors" />  43:     <category term="PrototypeMVVM.Web.Core.Models.Book" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />  44:     <content type="application/xml">  45:       <m:properties>  46:         <d:Name>Book2</d:Name>  47:         <d:Description></d:Description>  48:       </m:properties>  49:     </content>  50:   </entry>  51: </feed>

Note: you can’t use ‘localhost’ in the URI if you want to see the http traffic on fiddler (.NET bypass the fiddler proxy for "localhost" addresses). To workaround this, replace the ‘localhost’ with ‘ipv4.fiddler’ (when fiddler sees a request bound for "ipv4.fiddler", it simply changes it to 127.0.0.1).

Now lets see some code!

The View Code Behind

   1: public partial class BooksPage : Page

   2: {

   3:     public BooksPage()

   4:     {

   5:         InitializeComponent();

6:

   7:         DataContext = new BooksViewModel();

   8:     }

   9: }

Since all the presentation logic is placed in the VIewModel, the only thing left to do is to instantiate the ViewModel and to assign it as the DataContext of the View. Notice that the View includes no event handler implementations (the Commands in the ViewModel handle those), no link to the data layer and no assignments of values to named controls.

This greatly promotes testability since it allows mocking the View in most of the tests, which will naturally surround the VIewModel. And yet still, in order to achieve full test coverage, we’ll need to ensure that the View xaml is right, meaning that all the appropriate properties of its controls are bound to the right properties of the ViewModel. To achieve this, we’ll have to write extra tests that include a real View, real ViewModel and fake everything else, only to verify that the link between the View and the ViewModel is correct.

The View axml

   1: <navigation:Page x:Class="PrototypeMVVM.Core.Views.BooksPage"    2:            xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"    3:            xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"    4:            xmlns:d="http://schemas.microsoft.com/expression/blend/2008"   5:            xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"   6:            mc:Ignorable="d"   7:            xmlns:navigation="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Navigation"   8:            d:DesignWidth="248" d:DesignHeight="298"   9:            Title="Page1 Page" xmlns:sdk="http://schemas.microsoft.com/winfx/2006/xaml/presentation/sdk" >  10:     <Grid x:Name="LayoutRoot" Height="490" Width="776" >  11:         <sdk:Label Content="Books" Height="28" HorizontalAlignment="Left" Margin="16,9,0,0"   12:                    VerticalAlignment="Top" Width="75" />  13:         <sdk:DataGrid ItemsSource="{Binding Path=Books}"   14:                       AutoGenerateColumns="False" Height="216" HorizontalAlignment="Left"   15:                       Margin="16,32,0,0" VerticalAlignment="Top" Width="216">  16:             <sdk:DataGrid.Columns>  17:                 <sdk:DataGridTextColumn Binding="{Binding Path=Name}" Header="Name" Width="SizeToHeader" />  18:                 <sdk:DataGridTextColumn Binding="{Binding Path=Description}" Header="Description" Width="SizeToHeader" />  19:             </sdk:DataGrid.Columns>  20:         </sdk:DataGrid>  21:         <Button ToolTipService.ToolTip="{Binding Path=SaveButtonToolTip}"  22:                 Height="23" HorizontalAlignment="Right" Margin="0,254,699,0"   23:                 VerticalAlignment="Top" Width="61" Command="{Binding Path=SaveCommand}" Content="Save" />  24:     </Grid>  25: </navigation:Page>

Since the ViewModel is assigned as the DataContext of the View, in order to apply binding, we only need to specify the name of the ViewModel property. For instance, in order to bind the ‘Save’ button tool tip to the ‘SaveButtonToolTip’ property of the VIewModel - we use the following syntax: {Binding Path=SaveButtonToolTip}.

The VIewModel class (where the bulk of the code reside)

   1: public class BooksViewModel : ViewModelBase   2: {   3:     private IList<BookViewRow> m_books;   4:     5:     public BooksViewModel()   6:     {   7:         SaveButtonToolTip = "Save Changes";   8:         Refresh();   9:    10:         SaveCommand = new DelegateCommand<object>(o => DalService.Instance.Save());  11:     }  12:    13:     public ICommand SaveCommand { get; private set; }  14:    15:     public string SaveButtonToolTip { get; private set; }  16:       17:     public IList<BookViewRow> Books  18:     {  19:         get  20:         {  21:             return m_books;  22:         }  23:         private set   24:         {  25:             m_books = value;  26:             RaisePropertyChanged(() => Books);  27:         }  28:     }  29:    30:     public void Refresh()  31:     {  32:         DalService.Instance.GetBooks(  33:             (sender, args) => LoadBookse(args.Result));  34:     }  35:    36:     private void LoadBookse(IEnumerable<Book> books)  37:     {  38:         List<BookViewRow> bookViewRows = books.Select(book => new BookViewRow(book)).ToList();  39:    40:         Books = bookViewRows;  41:     }  42: }

The ViewModel exposes ‘SaveButtonToolTip’ and ‘Books’ properties that are bound (in the xaml) to the button tooltip provider and to the data grid respectively. The SaveCommand property is bound to the Command property of the ‘Save’ button, such that when the user click on the button, ‘Execute’ is being called on the ICommand object exposed by SaveCommand property.

Additionally, the ViewModel is in charge of getting books from the DalService. As you can see, it initiate an a-sync call to the DalService, and when the response arrive, it updates the Books property which update a private m_books field and notify the View about the change through the INotifyPropertyChanged interface.

The Books property is a List of objects of the type BookViewRow, which defines the visual representation of the data grid by exposing a property for every column on the data grid surface. As you can see in the xaml, every grid column is bound to a different property of the BookViewRow class.

In addition, you may also notice that the setters of each of the properties includes call to DalService.Instance.UpdateObject(m_Book). As a result, whenever the user changes value of a cell on the grid surface, the DalService is being updated. We’ll revisit this little piece of code latter on.

   1: public class BookViewRow : INotifyPropertyChanged   2: {   3:     private readonly Book m_Book;   4:     5:     public BookViewRow(Book Book)   6:     {   7:         m_Book = Book;   8:     }   9:    10:     public string Name  11:     {  12:         get { return m_Book.Name; }  13:         set  14:         {  15:             m_Book.Name = value;  16:             DalService.Instance.UpdateObject(m_Book);  17:         }  18:     }  19:    20:     public string Description  21:     {  22:         get  23:         {  24:             return m_Book.Description;  25:         }  26:         set  27:         {  28:             m_Book.Description = value;  29:             DalService.Instance.UpdateObject(m_Book);  30:         }  31:     }  32:    33:     public event PropertyChangedEventHandler PropertyChanged  34:     {  35:         add  36:         {  37:             m_Book.PropertyChanged += value;  38:         }  39:         remove  40:         {  41:    42:             m_Book.PropertyChanged -= value;  43:         }  44:     }  45: }

As you can see, the VIewModel derive from ViewModelBase class which encapsulate a mechanism for updating the View when ever a property on the ViewModel change.

   1: public class ViewModelBase : INotifyPropertyChanged   2: {   3:     protected void RaisePropertyChanged<TViewModel>(Expression<Func<TViewModel>> property)   4:     {   5:         var expression = property.Body as MemberExpression;   6:         var member = expression.Member;   7:     8:         if (PropertyChanged != null)   9:         {  10:             PropertyChanged(this, new PropertyChangedEventArgs(member.Name));  11:         }  12:     }  13:    14:     public event PropertyChangedEventHandler PropertyChanged;  15: }

In addition, instead of writing a new SaveCommand class and assign it to the SaveCommand property, the latter property in assigned to an instance of DelegateCommand, which is constructed with a delegate to a method in the ViewModel that implement the save operation. This allows us to implement the command logic in the ViewModel without having to create a concrete command class that calls the appropriate method on the ViewModel.

   1: public class DelegateCommand<T> : ICommand   2: {   3:     private readonly Action<T> m_executeAction;   4:     5:     private readonly Func<T, bool> m_canExecuteAction;   6:     7:     public DelegateCommand(Action<T> executeAction)   8:         : this(executeAction, null)   9:     {  10:     }  11:    12:     public DelegateCommand(Action<T> executeAction, Func<T, bool> canExecuteAction)  13:     {  14:         m_executeAction = executeAction;  15:         m_canExecuteAction = canExecuteAction;  16:     }  17:    18:     public event EventHandler CanExecuteChanged;  19:    20:     public bool CanExecute(object parameter)  21:     {  22:         if (m_canExecuteAction != null)  23:         {  24:             return (m_canExecuteAction((T)parameter));  25:         }  26:    27:         return m_executeAction != null;  28:     }  29:    30:     public void Execute(object parameter)  31:     {  32:         if (m_executeAction != null)  33:         {  34:             m_executeAction((T)parameter);  35:         }  36:     }  37: }

The DalService

The DalService is a Singleton that’s accessible to all the parts in the application, its job is to abstract away the communication with the data services. It defines the URI of the data service and encapsulates a replaceable DataServiceProvider that communicate directly to the data service.

   1: public class DalService : IIDalServiceTesting   2: {   3:     IDataServiceProvider m_dataServiceProvider;   4:     private Uri m_datServiceRoot = new Uri("http://localhost:16081/BookStoreDataService.svc");   5:     6:     static readonly DalService s_instance = new DalService();   7:     public static DalService Instance   8:     {   9:         get { return s_instance; }  10:     }  11:    12:     private IDataServiceProvider DataServiceProvider  13:     {  14:         get  15:         {  16:             return m_dataServiceProvider ?? (m_dataServiceProvider = new DataServiceProvider(m_datServiceRoot));  17:         }  18:     }  19:    20:    21:     public void GetBooks(EventHandler<ResponseEventArgs<IEnumerable<Book>>> handler)  22:     {  23:         DataServiceProvider.GetBooks(handler);    24:     }  25:    26:     public void Save()  27:     {  28:         DataServiceProvider.Save();  29:     }  30:    31:     public void UpdateObject(Book book)  32:     {  33:         DataServiceProvider.UpdateObject(book);  34:     }  35:    36:     void IIDalServiceTesting.SetDataServiceRoot(Uri uri)  37:     {  38:         m_datServiceRoot = uri;  39:     }  40:    41:     void IIDalServiceTesting.SetDataServiceProvider(IDataServiceProvider provider)  42:     {  43:         m_dataServiceProvider = provider;  44:     }  45: }

The DalSevice allows 1) fetching books asynchronously through ‘GetBooks’, 2) notifying the ‘WCF data services client runtime’ that a book has changed though ‘UpdateObject’, and 3) saving the changes made on the books through ‘Save’. All the operations that were mentioned are implemented in the underlying IDataServiceProvider object.

The DalService allows test units to replace its underlying DataServiceProvider with an alternative implementation or an empty one through SetDataServiceProvider, which belongs to the IDalServiceTesting interface and implemented explicitly.

Since IDalServiceTesting is implemented explicitly, callers must cast the DalService to IDalServiceTesting in order to call SetDataServiceProvider.

   1: IIDalServiceTesting dalServiceTesting = DalService.Instance;   2: dalServiceTesting.SetDataServiceProvider(new DataServiceProviderNull());

This a good practice since it hides away the DalService test related functionality, and it discourage developers from calling this code from non-test code.

Another thing to notice is that we can change the URI of the data service, also through the IDalServiceTesting interface. This can be used in order to replace the data service host with a test host, which will allow us to plug in alternative implementation and to test the DalService in isolation.

The DataServiceProvider

   1: public class DataServiceProvider : IDataServiceProvider   2: {   3:     class RequestInfo<T>   4:     {   5:         public IQueryable<T> Query { get; private set; }   6:         public EventHandler<ResponseEventArgs<IEnumerable<T>>> Handler { get; private set; }   7:     8:         public RequestInfo(IQueryable<T> query, EventHandler<ResponseEventArgs<IEnumerable<T>>> handler)   9:         {  10:             Query = query;  11:             Handler = handler;  12:         }  13:     }  14:    15:     private readonly BookStore m_bookStore;  16:    17:     public DataServiceProvider(Uri serviceRoot)  18:     {  19:         m_bookStore = new BookStore(serviceRoot);  20:     }  21:    22:    23:     public void GetBooks(EventHandler<ResponseEventArgs<IEnumerable<Book>>> handler)  24:     {  25:         var query = from g in m_bookStore.Books  26:                     select g;  27:    28:         var dataServiceQuery = (DataServiceQuery<Book>)query;  29:         dataServiceQuery.BeginExecute(OnBooksArrived, new RequestInfo<Book>(query, handler));  30:    31:     }  32:    33:     private void OnBooksArrived(IAsyncResult ar)  34:     {  35:         var requestInfo = (RequestInfo<Book>)ar.AsyncState;  36:         EventHandler<ResponseEventArgs<IEnumerable<Book>>> handler = requestInfo.Handler;  37:         var query = (DataServiceQuery<Book>)requestInfo.Query;  38:         IEnumerable<Book> books = query.EndExecute(ar);  39:         if (books == null)  40:         {  41:             handler(this, new ResponseEventArgs<IEnumerable<Book>> ("Response returned blanck"));  42:             return;  43:         }  44:    45:         handler(this, new ResponseEventArgs<IEnumerable<Book>> (books));  46:     }  47:    48:     public void Save()  49:     {  50:         //m_BookStore.AddToBooks(new Book() { DbName = "ssss" + s_counter++});  51:         m_bookStore.BeginSaveChanges(SaveChangesOptions.ReplaceOnUpdate, SaveCallback, null);  52:     }  53:    54:     public void UpdateObject(Book book)  55:     {  56:         m_bookStore.UpdateObject(book);  57:     }  58:    59:     private void SaveCallback(IAsyncResult ar)  60:     {  61:         m_bookStore.EndSaveChanges(ar);  62:     }  63: }

The DataSevicePorvider instantiate a BookStore class injecting it with the URI of the remote service. The BookStore is an auto generated proxy that was created by ‘WCF data services client runtime’ based on the matching server object exposed by the data service (we’ll get into it shortly). The BookStore proxy is the root proxy, and as such it is used to query for books and to save the changes made to the in-memory books collection.

Notice that unlike the full version of .NET Framework, .NET for Silverlight only support asynchronous queries and asynchronous saving.

The Application - Server Side

The server side application is all about providing REST-full data services to the clients.

As mentioned above, we’ll not make use of the entity framework, we’ll create our very own data model and implement a custom update service provider that will allow the clients to update the data, which in turn will be stored in an xml based data storage.

So, lets start. A typical WCF data service provider is hosted by a web project that contains special class that look like this:

   1: public class BookStoreDataService :  DataService<BookStore>, IServiceProvider   2: {   3:     public static void InitializeService(DataServiceConfiguration config)   4:     {   5:         config.SetEntitySetAccessRule("*", EntitySetRights.All);   6:         config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;   7:         config.DataServiceBehavior.AcceptProjectionRequests = true;   8:     }   9:    10:     public object GetService(Type serviceType)  11:     {  12:         if (serviceType == typeof(IDataServiceUpdateProvider))  13:         {  14:             return new BookStoreUpdateProvider(CurrentDataSource);  15:         }  16:         return null;  17:     }  18: }

The service derive form DataSevice<T>, where T defines the root class of the object model that is being exposed. In our case, it will expose the BookStore class.

In the above implementation, the data service is implemented in the web project. In order to enable better testing experience, we’ll do things differently. We’ll keep the web project as thin as possible, and move all the juicy implementation to a separate assembly. This way we can replace the web project with a test web project, which will allow us to host the test framework and the WCF data service from the same web project, and to switch from real data service to mock data service without having to load different projects.

The BookStoreDataService

   1: // BookStoreDataServiceCore resides in a separate assembly   2:  public class BookStoreDataService : BookStoreDataServiceCore       3:  {   4:      public static void InitializeService(DataServiceConfiguration config)   5:      {   6:          config.SetEntitySetAccessRule("*", EntitySetRights.All);   7:          config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;   8:          config.DataServiceBehavior.AcceptProjectionRequests = true;   9:      }  10:  }

The BookStoreDataServiceCore

   1: // We use custom data provider   2:    public class BookStoreDataServiceCore : DataService<BookStore>, IServiceProvider   3:    {   4:        protected override BookStore CreateDataSource()   5:        {   6:            var authors = new List<Author>    7:                {new Author("Mark"), new Author("Sarit"), new Author("John")};   8:     9:            var repository = RepositoryBuilder.Create<Book>("Books.xml");  10:    11:            return new BookStore(repository, authors);  12:        }  13:    14:        public object GetService(Type serviceType)  15:        {  16:            // We provide only update services..  17:            if (serviceType == typeof(IDataServiceUpdateProvider))  18:                return new BookStoreUpdateProvider(CurrentDataSource);  19:    20:            return null;  21:        }  22:    }

Since the BookStoreDataServiceCore derive from DataService<BookStore> – we need to implement the CreateDataSource method in order to instantiate the BookStore object, which is the root of our data model (if will not implement it – the runtime will create a default instance of the BookStore for us). In order to enable testing the data model in isolation, we inject the BookStore with a replaceable repository object that encapsulates the CRUD logic.

In addition, we support updates on the data model through BookStoreUpdateProvider that is retuned by GetService.

The BookStoreUpdateProvider

   1: public class BookStoreUpdateProvider : IDataServiceUpdateProvider   2: {   3:     private readonly BookStore m_bookStore;   4:     private readonly List<Action> m_pendingChanges = new List<Action>();   5:     public BookStoreUpdateProvider(BookStore bookStore)   6:     {   7:         m_bookStore = bookStore;   8:     }   9:    10:     public object CreateResource(string containerName, string fullTypeName)  11:     {  12:         var book = new Book();  13:         // And register pending change to add the resource to the resource set list  14:         m_pendingChanges.Add(() => m_bookStore.Add(book));  15:    16:         return book;  17:     }  18:    19:     public object GetResource(IQueryable query, string fullTypeName)  20:     {  21:         object resource = Enumerable.Cast<object>(query).FirstOrDefault();  22:    23:         return resource;  24:     }  25:    26:     public object ResetResource(object resource)  27:     {  28:         return resource;  29:     }  30:    31:     public void SetValue(object targetResource, string propertyName, object propertyValue)  32:     {  33:         PropertyInfo propertyInfo = targetResource.GetType().GetProperty(propertyName, BindingFlags.Public | BindingFlags.Instance);  34:         m_pendingChanges.Add(() => propertyInfo.SetValue(targetResource, propertyValue, null));  35:    36:     }  37:    38:     public object GetValue(object targetResource, string propertyName)  39:     {  40:         return null;  41:     }  42:    43:     public void SetReference(object targetResource, string propertyName, object propertyValue)  44:     {  45:     }  46:    47:     public void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded)  48:     {  49:     }  50:    51:     public void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved)  52:     {  53:     }  54:    55:     public void DeleteResource(object targetResource)  56:     {  57:     }  58:    59:     public void SaveChanges()  60:     {  61:         if (m_pendingChanges.Count == 0) return;  62:    63:         foreach (var pendingChange in m_pendingChanges)  64:         {  65:             pendingChange();  66:         }  67:    68:         m_pendingChanges.Clear();  69:         m_bookStore.SubmitChanges();  70:     }  71:    72:     public object ResolveResource(object resource)  73:     {  74:         return resource;  75:     }  76:    77:    78:     public void ClearChanges()  79:     {  80:     }  81:    82:     public void SetConcurrencyValues(object resourceCookie, bool? checkForEquality, IEnumerable<KeyValuePair<string, object>> concurrencyValues)  83:     {  84:    85:     }  86: }

Every time that the client make changes to a proxy object and call UpdateObject on the root proxy (using DalService.Instance.UpdateObject, see BookViewRow implementation above) – the SetValue method is called on the BookStoreUpdateProvider, which in turn add invocation delegate to the pending changes list. Once BeginSaveChanges is called on the root proxy (using DalService.Instance.Save) – the SaveChanges method is called , which in turn execute all the pending invocations and calls SubmitChanges on the server side BookStore object, which in turn uses the repository object to persist the changes.

When the SaveChanges method is called, the client sends changes back to the data service. SaveChanges can fail when data changes in the client conflict with changes (made by other clients) in the data service. OData provides some support for optimistic concurrency that enables the data service to detect update conflicts (by using Concurrency tokens, which are included in the eTag header of requests to and responses from the data service, and managed by the WCF Data Services client).

The BookStore

   1: public class BookStore    2:   {   3:       private readonly IRepository<Book> m_repository;   4:       private readonly List<Author> m_authors;   5:     6:       public BookStore(IRepository<Book> repository, List<Author> authors)   7:       {   8:           m_repository = repository;   9:           m_authors = authors;  10:       }  11:    12:       public IQueryable<Book> Books  13:       {  14:           get { return m_repository.Query(); }  15:       }  16:    17:       public IQueryable<Author> Tables  18:       {  19:           get  20:           {  21:               return m_authors.AsQueryable();  22:           }  23:       }  24:    25:    26:       public void Add(Book book)  27:       {  28:           m_repository.Add(book);  29:       }  30:    31:       public void SubmitChanges()  32:       {  33:           m_repository.SubmitChanges();  34:       }  35:   }

Notice that the BookStore exposes properties that implement the IQueryable<T> interface, the BookStore proxy that is generated in the client exposes matching properties of type DataServiceQuery<T>

The BookStore auto generate proxy

   1: public partial class BookStore : global::System.Data.Services.Client.DataServiceContext   2:  {   3:      /// <summary>   4:      /// There are no comments for Books in the schema.   5:      /// </summary>   6:      [System.CodeDom.Compiler.GeneratedCodeAttribute("System.Data.Services.Design", "1.0.0")]   7:      public global::System.Data.Services.Client.DataServiceQuery<Book> Books   8:      {   9:          get  10:          {  11:              if ((this._Books == null))  12:              {  13:                  this._Books = base.CreateQuery<Book>("Books");  14:              }  15:              return this._Books;  16:          }  17:      }

As we saw in the client side DataServicePorvider implementation, client code can make use of the Books property in order to query for books.

Running the Application

We will use the same web project in order to run both Silverlight application and WCF data service. To achieve this, the Silverlight application need to be added to the ‘Silverlight Applications’ list of the web project.

As a result, after compilation, the Silverlight application will be packed into xap file and appear under the ClientBin directory, and a new aspx file will be added in order to enable running the Silverlight application in debug mode.

Additionally, we need to add the BookStoreDataService.svc to the web project, so when we’ll run the web project, it will start the data service.

Consuming the WCF data service via PowerShell Script

Let see how we can query for data from the books store using power shell script.

Open power shell and add the following script:

   1: $webclient = new-object system.net.webclient   2: $webclient.UseDefaultCredentials = $true   3: $uri = "http://localhost:16081/BookStoreDataService.svc/Books"   4: $xml = $webclient.DownloadString($uri)   5: $xml   6:  

And the result:

Testing the Application

As you may notice, every aspect in the design is affected by the need to provide great testing experience. Starting from using MVVM, through abstracting away the data access layer and providing ways to mock the DataServiceProvider and change the data service URI, to moving the WCF data services logic from the web project, and using the repository pattern in the BookStote data model.

In the Testing Silverlight 4.0 App with MVVM, MEF and WCF Data Services post we’ll see how we can take advantage of the those patterns (together with the MEF framework) in order to develop component/multi-component level tests for our application.

I hope you find this article useful, your comments will be appreciated!

The source code can be downloaded from here

Sunday, August 8, 2010

The Pillars of Concurrency

In recent years, microprocessors designers are hitting the walls of memory latency, heat dissipation, and number of transistors for single core processors. As a result, the microprocessors industry consistently introducing more cores on the same chip in order to keep on providing stronger processors and maintain Moore's law. Consequentially, applications' parallelism become more fine-grained, which translates to one application having hundreds or even thousands of threads working concurrently.

While concurrent programming popularity increases, Herb Sutter makes the case that we need to have a consistent mental model for reasoning about concurrency. According to the new model, concurrency requirements and techniques are divided into the 3 categories (pillars).

Pillar 1 – Responsiveness and Isolation

Pillar 1 is about running time-consuming operations (such as downloads and db transactions) on a separate, dedicated background threads in order to keep time sensitive threads (such as the UI thread) responsive, while letting the background threads communicate via asynchronous messages.

For this Pillar, it doesn’t matter how many cores available in the machine, we care about responsiveness even when we target a single core machine. However, since with this Pillar some work is being done in parallel, we might see an increase in performances and make better use of the cores in the machine purely as a side effect.

Background Worker

When working with .NET, the BackgroundWorker abstraction can be used to execute the background work asynchronously.

We simply drag the BackgroundWorker component onto the form, add an event handler for the DoWork event, and call the time-consuming operation in this event handler. We call RunWorkerAsync to start the operation and CancelAsync to cancel the operation. While the operation makes progress, we can call ReportProgress to send progress updates notifications .

To receive progress updates and operation completed notifications, we handle the ProgressChanged and RunWorkerCompleted events. The nice thing about the BackgroundWorker is that the ProgressChanged and the RunWorkerCompleted handlers are being executed from the context of the UI thread.

Pillar 2 – Throughput and Scalability

Pillar 2 is about scaling to increasing number of cores and maintaining high throughout, mainly by introducing more and more parallelism.

Perhaps the biggest challenges associated with scalability is finding the most efficient way to control the concurrency of the application. Practically, we need to figure out how to divide the work and distribute it among the threads in a way that put the maximum amount of cores to work with minimum contentions.

Fortunately, we are not alone in this battle, frameworks such as OpenMP, TPL and Parallel-Studio can help us with producing scalable applications that will execute faster and faster on future machines having more and more cores (re-enable the free lunch).

Thread v.s. ThreadPool

The first problem that we face when executing asynchronous work is that the cost of threads creation can significantly hurt performance. Luckily, the thread-pool solves that by maintaining heuristic "pool" of threads that are just waiting for work items, when they are done executing a work item, they go back to the pool and wait for the next one, etc.

The thread pool employ the use of a ‘thread injection and retirement algorithm’ which optimizes the creation and destruction of threads by looking at the number of running work items and the number of cores available in the machine.

ThreadPool v.s. TPL (Task Parallel Library)

Before the introduction of .NET 4.0, in order to produce scalable programs, we didn’t had much of a choice but to use the thread pool directly, or to create our own threads factory.

The thing with working directly with the thread pool is that as long as there’re available threads in the pool (which is usually most of the time) – it executes work items immediately. With that approach, if we wont control the amount of work items that we pass over to thread pool, we can get pretty fast into a point where there’re to many threads executing work at once resulting in a great amount of context switches, or in the other extreme, to a point where work items execution is delayed for very long periods.

At the time, scalability wasn’t much of a concern so dealing directly with the thread pool worked out just fine. However, with the many core revolution/evolution knocking on our doors, there’s a need for a better solution. Practically, we need another ‘black box’ between the application and thread pool that will control the amount of work items delivered to thread pool at a given time, in most cases according to the machine architecture, workload and CPU utilization.

In.NET 4.0, Microsoft introduced a new library called TPL (Task Parallel Library) that plays the role of the ‘back box’ mentioned above. TPL executes the asynchronous work (tasks) in a way that fit the number of available cores, and provides developers with more control over the way in which that tasks get scheduled and executed.

With TPL, developers separate the work into small pieces (tasks) and let the runtime execute them in a scalable fashion, naturally, via the thread pool. Tasks that submitted for execution from the context of non-TPLs thread (such as the UI thread) are being scheduled to the thread pool global queue. On the other hand, tasks that are submitted from the context of another task (their parent task), are stored in a separate queue owned by the worker thread of their parent task, and executed by the same worker thread in a LIFO fashion. When a worker thread is done executing its tasks (parents and childes), it looks for work in the global queue, if there isn't any, it tries to steal work from the local queues of its fellow threads.

The worker threads local queues are implemented as work stealing queues (WSQ). WSQ has two ends, it allows lock-free pushes and pops from one end (“private”), but requires synchronization from the other end (“public”). A worker thread pushes child tasks to the private end, and pop tasks from the private end (in a LIFO fashion) avoiding all locking and achieving great caching behavior. When the worker thread requires to steal, it grabs tasks from the public end where synchronization is required.

Pillar 3 - Consistency

With the addition of concurrency comes the responsibility for ensuring safety. When running multiple threads that access the same memory, we need to make sure that there's no corruption by applying the appropriate synchronization, and that while synchronizing there's no possibility for a deadlock.

A corruption occurs when one or more threads are writing a piece of data while one or more threads are also reading that piece of data. To avoid corruption, we use different kind of locks and different locking schemes to protect the data from concurrent access. Some locks grant exclusive access to only one thread, others grant shared access to readers and exclusive access to writers, there’s the immutability approach where objects are designed as read-only and cloned on every write, thus no lock is required on read, and the list goes on and on.

Sadly, in this battle we are alone, and there by there's a fundamental set of concepts that we need to learn and become comfortable with in order to be able to avoid the many hazards associated with lock based programming. The most common kind of hazard is that we simply forget to take a lock when it's required, which can lead to data races, resulting the generation of incorrect results. Less common hazard is a Deadlock, which can arise when multiple lock are being acquired in the wrong order. When there are more threads waiting at a lock than can be serviced, a lock convoy can arise, which means that a continuously growing backup of requests will remained unhandled and eventually begin timing out.

In order to reduce the risk of using lock, it's recommended to use 'out of the box' lock free data structure, or take some ideas from functional languages and instead of sharing data, create new clones of the data on every write and allow lock free reads, in F#, the compiler does the cloning for us, it looks like this:

Races, deadlocks and convoys aside, perhaps the most fundamental objection associated with locks is that they do not compose, i.e. abstractions that expose thread safe operations may produce correct results when used independently, but expose invalid intermediate state when combined. A classic example for the composition problem is when we need to implement an activity that transfer money from one account object to another while the intermediate state, when non of the accounts contains the money, must not be visible to other threads. The fact is that we can’t implement this feature without introducing some kind of a global lock that will surround the transfer, and will have to be used when ever threads access the accounts.

Transactional Memory

In an attempt to provide a solution for data sharing that is easy to use and less error prone than fine grained locks – researchers are working on a transactional memory solution (STM). With STM, instead of using locks to synchronize access to shared objects, we simply wrap all the code that access those objects in a transaction and let the runtime execute it atomically and in isolation by doing the appropriate synchronization behind the scenes.

STM generates a single lock illusion and still maintain great scalability by utilizing an optimistic concurrency control technique, by which instead of acquiring a lock pessimistically letting only one thread enter the atomic region at a time, the runtime assumes that there is no contention and allows all the threads to execute atomic regions concurrently, it detects if the assumption is incorrect (transaction conflict) whenever a thread exists an atomic region, and if that’s the case, it rolls back changes that were made in the region and re-execute.

As illustrated in the diagram above, STM copies the objects that are being written into to shadow copies, stores the versions of the objects that are being read, and execute the transaction while instead of writing into the objects master location – it writes into the shadow copies, at the end of the transaction, it compare the versions of the reads with their current version to see if any of them has changed by other transaction, if that’s the case, there’s a conflict thus it re-executes the transaction (no need to rollback since changes were made on the shadow copies), other wise it submits the transaction by copying the shadow copies into their master location and updating the reads versions (under the protection of fine grained locks).