Meaningless Numbers

November 24, 2009

For those of you who enjoy performance metrics and statistics, I bring to you the first semi-serious attempt at producing some for an ESA application.  Personally, I am a realist, who thrives on theory but trusts only empirical data, and isn’t happy until the two meet.  And I’m pretty happy at the moment.  Theory says that a stateful ESA server should be many times more efficient than a stateless server given that there is no state get/set overhead.  So when these first practical tests are performed and the numbers look good, I am happy.

Test PC is an AMD triple core, 4G memory running server 2008 x64.  Server test is simply incrementing a number, but that of course comes through as a discrete method request via a web service as normal.

Test harness is a WPF application which pretends to be some number of distinct users and then runs a preset script indefinitely and short random intervals.  I ran 3 instances of this each pretending to be 100 users, very fast typing users, as they pushed through between 2 and 5 transactions/second.

I took screenshots of task manager at the start, after 300 users logged in, after 100 then 200 then all of them became busy.  Some are memory sorted, the rest are cpu sorted.  A final shot was taken after stopping, but not logging out the 300 users.  The test harnesses also log a time-stamped entry every 200 transactions.

Here they are

Now for the transaction rates – they are all slightly varied as the request intervals have a small random component.   They also ran for different lengths of time as indicated above – 100 went usy, then 200 then 300 then they stopped).
Tester 1 (time=0:20, transactions=5400, rate=16200/minute)
Tester 2 (time=0:39, transactions=13200, rate=20300)
Tester 3 (time=1:06, transactions=22000 rate=20000)

Total transaction rate was 56000 transactions / minute and the heart rate had just started to go up.  And thats just one AppServer!  One day soon I will set up a dedicated machine with a load balancer and maybe 4 appservers and another PC running the test harnesses and see if I can break 1 million transactions / minute on pretty standard hardware.

You know what I’m gonna say.



  1. […] This post was mentioned on Twitter by matt clarke, matt clarke. matt clarke said: Meaningless Numbers: http://wp.me/pH4wX-1c […]

  2. […] blogged previously about the internal performance testing we’ve done and how we use our platform to power our SaaS ERP application but, at the end of the […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: