Optimizing XamDataGrid Performance Using External Data Operations

A number of line of business applications use XamDataGrid for its rich feature set, high customizability and performance. If there’s been one constant development in the use of the XamDataGrid across all industries, it’s been the increasing amount of data displayed in the XamDataGrid. Traditionally, grid controls perform data operations such as sorting, filtering and grouping by themselves on the UI thread. However, with increasing data loads, developers are looking for ways to perform these operations on a separate thread, or on a different machine altogether. This is why developers have asked us to have the XamDataGrid pass the arguments of these operations, and without performing this operation, to display the processed result. We’ve added this functionality in the 12.1 release, and it has enabled us to significantly improve performance when large datasets are bound to the XamDataGrid. This blog post describes how to take advantage of this functionality, and describes its implications for performance in terms of memory footprint and time it takes to perform an operation.

Please download the sample project – it illustrates how to activate off-grid processing of data manipulations, and compares the performance of a XamDataGrid bound to the same dataset in the two cases – when internal and external operations are used. The project is built using Visual Studio 2010 and .NET Framework 4. It uses a trial version of the 12.1 WPF product, so you can build and run it without any additional downloads. Fully-functional free 30-day trial of the NetAdvantage for WPF product, which includes the XamDataGrid is available. Here’s a screenshot of the sample project:

What’s in the sample

In the sample application, there are two XamDataGrids side by side - the one on the left uses internal operations, and the one on the right performs these operations through the CollectionView object instead. Both grids are bound to a dataset containing 100,000 records. In order to illustrate the difference in performance due to internal or external operations, we’ll compare the performance of sorting and grouping. However, the exact same results would also apply to filtering and summary computations.

Performance implications of using internal sort/group

By default, the XamDataGrid only creates record objects to represent the data items which are currently visible (as opposed to all the records in the dataset). However, when sort/group operations are performed by the XamDataGrid internally, it needs to create record objects to represent all the items in the data set, so it could sort/group them. Even though UI elements are not created for all these records due to the XamDataGrid UI virtualization logic, the record objects can have a substantial impact on the application’s memory footprint (as we’ll see in the performance comparison below), especially when the datasets bound to the XamDataGrid are large. So how does this impact performance? The first time a user sorts/groups a grid, there is a delay during which the XamDataGrid creates record objects for all the data items, and then performs the operation the user requested. This puts the entire dataset in memory in terms of record objects managed by the XamDataGrid. While this increases the memory footprint, and causes a delay the first time such an operation is executed, subsequent sort/group operations are quite fast, because the entire dataset is already in memory.

Let’s see how this impacts performance. Please run the sample, and sort, or group the XamDataGridon the left - you will see the time and memory footprint change (delta) reported above it. On my machine, when the internal operations grid is sorted for the first time, the operation takes 4.3 seconds and memory consumption increases by 40MB. Subsequent sorts take around 1 second, and memory footprint stays the same (since it’s already at its maximum – all data items are represented as record objects). Once the dataset is already fully loaded, grouping takes about 2.8 seconds, ungrouping 1.2 seconds.

Performance implications of using external sort/group

You can control how sorting, grouping, filtering and summary computations are performed using the SortEvaluationMode, GroupByEvaluationMode, FilterEvaluationMode and SummaryEvaluationMode properties. They give you the ability to to disable the internal sorting, grouping, filtering, and summary calculations, and to have those performed manually, or through the CollectionView object. This means that the grid doesn’t have to initialize record objects to represent the entire data set – instead, it relies on the backend to reorder the dataset. In the case of using the collection view (by setting the above properties to “UseCollectionView”), this requires no implementation of extra logic. However, if you have a backend server, where calculations/reordering are performed, you can use the Manual setting for the above properties. This will cause the XamDataGrid to fire the Sorting/Grouping/RecordFilterChanging events, enabling you to pass these onto the backend, where the calculations are performed. The XamDataGridwill display the reordered dataset once it’s been processed by the backend.

Using the UseCollectionView and Manual settings for the sorting/grouping saves the XamDataGrid the need to instantiate record objects for the entire dataset, thus keeping the memory footprint constantly low. The time component of performance is up to the backend you’re using for the processing of these operations. In this sample, in the XamDataGridon the right, we’re using the CollectionView object for handling these operations instead of the internal logic.

How about performance in this case? Please run the sample, and sort, or group the XamDataGridon the right – the time and memory footprint change will be reported right above. When sorting and grouping is performed by the collectionView (using the UseCollectionView value for SortEvaluationMode and GroupByEvaluationMode properties), the memory footprint stays the same, and the time it takes is as follows – sorting 2.5 seconds, grouping 4.9 seconds, ungrouping 2.5 seconds. There are three points to mention here:

1. The memory footprint stays constant, because no extra record objects are instantiated due to sorting/grouping

2. The time it takes to sort/group is constant, i.e. there’s no significant delay when performing the first sort/group (as is the case when internal operations are used, due to initializing record objects)

3. Time time it takes to sort/group is greater than the time it takes to perform the same operations if internal operations are used. However the time aspect of performance is entirely up to the backend you’d like to use for these operations. Even so, using the collectionView objects gives you low-memory consumption, and good first-time sort/group performance (while in the internal operations case you have an extra delay the first time an operation is performed) and may be useful in some cases.

Summary calculations

The default internal summary calculations require the grid to initialize record objects to represent the entire dataset. Using the SummaryEvaluationMode property, summary calculations can also be performed using Linq, or manually. While using Linq requires no extra implementation on your side, using the manual computation mode requires handling the QuerySummaryResultevent, where you can perform your own evaluation logic, and provide a value for the summary result. Using the manual mode enables you to keep the memory consumption low, and gives you the opportunity to improve the calculation time as much as you can by optimizing the backend.

Summary

In this blogpost, I presented how to set the XamDataGrid to perform sorting/grouping/filtering/summary calculations externally. We described the API to use for this purpose, and looked into the performance implications of using external operations. The sample project can be used to easily demonstrate the difference between the two scenarios and can be readily changed to let you see the improvement in performance you’d see when you bind your own data to it. Using manual mode, you can essentially remove any limitations on sorting/grouping/filtering/summaries imposed so far by the hardware of the client machine running your application – using a powerful calculation backend machines is the best way to process large datasets efficiently, while delivering a fast and responsive UI. You can start by talking to the users of your applications, and finding out about the views where they would benefit from larger amounts of data being displayed to help them make a decision. Once you find out about those, the XamDataGrid will help you meet the increasing hunger of your users for more data to be displayed, while delivering an excellent user experience.