The objective was to get responsiveness metrics of various WebReport services, such as displaying the first page, navigating through pages, export to XML, etc. Although Arnica WebReport is a highly scalable system, which greatly benefits from distributed processing in a load-balanced clustered environment, we deliberately conducted this test on a single commodity grade entry-level server which costs approx. $1,200.
The following are the server and environment specifications:
- Single-CPU, Quad-core Intel Q9550 @ 2.83 GHz
- 8GB DDR2 PC2-6400 memory
- Intel Rapid Storage RAID-1 mirrored disk system with 7,200 RPM disks (WD6400AAKS)
- Windows 2008 Standard SP2, 64bit
- .NET 2.0
A report was created against AdventureWorks database running on SQL Server 2008 R2, 64bit. We used a query to obtain 12,000 records with configuration set to display 20 records per page, i.e. to show 600 pages in the report.
We used the following query:
SELECT TOP 12000 AccountNumber, BillToAddressID, Comment, ContactID, CreditCardApprovalCode, CreditCardID, CurrencyRateID, CustomerID, DueDate, Freight, ModifiedDate, OnlineOrderFlag, OrderDate, PurchaseOrderNumber, RevisionNumber, rowguid, SalesOrderID, SalesOrderNumber, SalesPersonID, ShipDate, ShipMethodID, ShipToAddressID, Status, SubTotal, TaxAmt, TerritoryID, TotalDue FROM Sales.SalesOrderHeader
The query responsiveness was 0.218 seconds, which is an insignifcant contributor to the response latency, but it is not really a reporting issue.
Here are the metrics which we collected from various WebReport services when using this query:
- Time to first page (includes SQL query latency): 1.310 seconds
- Navigation to the next page: 0.390 seconds
- Navigation from the first page to the last page: 0.390 seconds
- Navigation from the first page to the page #300 (in the middle of the page range): 0.421 seconds
- Export all records to Excel with no formatting (generates 1.9MB XLS output): 0.250 seconds
- Export all records to CSV text (generates 1.1MB text output)): 0.234 seconds
- Export all records to XML (generates 3.8MB XML output): 3.921 seconds
- Generate continuous HTML for all pages at once (generates 5MB HTML output): 10.436 seconds
- Generate HTML for all pages at once with page break formatting (generates 9MB HTML output): 74.568 seconds
As we can see from these results, WebReport data processing engine is very fast. HTML processing is the most resource intensive operation especially when conditional formatting is engaged. While building HTML output during page navigation requires almost no tangible time, generating HTML output at once for all records is a more time consuming process; although the later scenario is unlikely in dynamic web applications as browsers might take even more time to parse such large content before it is displayed, still these results might be useful as a benchmark when planning system features, capacities and responsiveness.
The report used in this benchmark is available from Arnica Software demo site, at the following link: http://www.arnicasoftware.com/demo/arnica-webreport-large-report-performance-test/1516/index.aspx
Other demo reports can be accessed here: http://www.arnicasoftware.com/resources/demo.aspx
The report discussed in this article looks like this: