Expensive toasters

October 10, 2011

While out shopping the other day, I stumbled upon a stack of toasters on display – I don’t think it was a sale or special – just a display stand.  $12.95.  OK, to be honest, I don’t get out much (shopping-wise) but to retail a toaster for $12.95 is pretty amazing.  Having fixed and cleaned various toasters in the past (the mister-i-can-fix-it deep inside us) I know what makes up a toaster.  And I just can not believe the price.  This got me thinking and took me on a journey where I was ultimately able to refine my Christmas wish from yesterday.

I want as assembly line and componentry for software.

My journey began by decomposing the costs of the toaster (all balanced against the final quality) into buckets.  Raw materials, labour, packaging, transport, stocking etc.  What I realised was that the labour component has to be minimal.  $12.95 doesn’t buy a lot of labour.  This is clearly the result of some form of automation/assembly line and componentry where labour costs are minimal.  Lets call this the factory pattern.  The other way of making toasters would be in a workshop, where the input is pretty much raw materials such as sheet metal.  The inputs to the factory are a variety of prefabricated components and designs, which are assembled by hand or by machine.  In a workshop, the design is executed on an individual basis, with components made from the raw materials as required.  If toasters were made this way, you would need to move the decimal in the price a few spaces over; they would cost $12950.00 as they would have a huge labour component.  Now as time went on in a workshop, power tools of increasing quality may be introduced, and some componentry such nuts and bolts, which would reduce the labour and thus costs.  So now our toaster is $1295.00.  Still pretty expensive.   Some clever person then decides to take it to the next level – lets design a bunch of reusable components, make a factory to produce 1000’s of sets of these components, and then some robotics to assemble them as required, into a few different models as sales forecasts predict.   Clearly there is a serious investment here, in the tooling, design and preparation of this toaster factory, but in reality a large number of components (screws, nuts, springs)  are already being produced for another item that they can reuse.  So this is not imagination land – this is real – $12.95 toasters.

[Harsh realisation warning!]

The world of software development  still runs on the workshop pattern.  We have not invented a software factory!

I actually find that quite embarrassing – I always thought IT folk were pretty clever.  How come we haven’t copied the manufacturing world into the factory pattern?

It gets worse too.  I did warn you.  Manufacturing consumes raw materials.  Building software does not!  I can conjure as many 1’s and 0’s in fancy patterns as I need at zero cost.  So we in software development actually have more to gain from the factory pattern than manufacturing does!  Rock bottom coming here (I warned ya!): imagine if you have made a factory to produce 2-slice toasters and then demand grows for a 4-slice toaster.  You will need to redesign and implement some tooling to produce larger face plates and such.  What can we do?  Inherit!  SidePanel4Slice : SidePanel2Slice { this.size = 2 * base.size; }.  You get the picture.  Manufacturing got a huge benefit from the  factory pattern, orders of magnitude, and the world of software development has more to gain from the adoption of a factory pattern than manufacturing.  Its embarrassing that we haven’t done so yet.

Its time to stop demanding bigger and better power tools from the vendors, and demand a software factory.

Redmond are you listening?


What I want for Christmas

October 7, 2011

It is early October so you got start dropping hints now…

This year, I am going to try a different tack.  I have enough undies, socks, beer steins etc.  What I dont have… is time.  And it seems I am not alone… the world over has software developers working crazy hours, and still coming over budget and late.  And buggy.  So I have been thinking – ok dreaming – of a way to fix this.  Its Friday, beers at the 4Pines, but I want to be able to finish the day at 5, not 7:30.  Its thirsty work, I am motivated.

What I want for Christmas is…

[deep breath]

A software development platform that lets me model, draw lots of pictures.  Kinda UML-ish but better.  I want a DSL, a customisable, extensible DSL that the modeller understands.  I want code reuse – I am so tired of lifting stuff from here & there.  I want to write code – good code – once.  Twice is too many.  I really want to separate my presentation layer from my business layer.  I want to draw my screens, not code them.  In any technology – iPhone, XAML, Android, HTML5… I want an ORM that does everything for me – that an ORM should do.  So it should just let me write SQL if I want, or use stored procs and triggers.  If I want.  If not it does it all for me.  No code – just pictures.  And I want it to run anywhere – cloud, my laptop, Azure, datacentre….  With one click.  And I dont want to worry about scalability – just scale it for me.  And I want easy testing and debugging.   And logging.

[/Deep Breath]

Santa Clause, are you listening?  I sure hope the folks at Microsoft, Apple, Oracle and IBM are because they’re the ones who can grant me my wish.


Discovery vs Invention

August 6, 2011

A dear friend of mine, who I spend a lot of time walking, kayaking and of course talking with comes up with up the occasional gem, a real pearl of wisdom.  For example, in reference to modern surfing competitions, he says ‘In my day, the best surfer out there was the one having the most fun’.  Hmmm not sure if everyone will get that or not.  My favourite of his gems is this, in reference to an unruly or disruptive person: ‘That [person|kid|…] needs a good listening to!’  Instead of ‘sit down and shut up and listen – you need a good talking to‘, it goes ‘sit down and start talking – I’m listening – you must have something on your mind’.  I have had the opportunity to use that a number of times, all with tremendous success.   People generally become disruptive when they are bored or oppressed, when no one will listen to them, when they have no way to vent their thoughts.  The internet, and blogging of course, are modern remedies for that.

I needed that introduction, because, as you will have just guessed, I have something on my mind I need to get out.

In a moment.

First, lets make sure we can discern between discovery and invention.  Discovery is the uncovering of an existing truth, something consistent with the known rules and laws of math, science etc at the time.   Invention is using those known rules and either changing them or adding to them to come up with a new ‘truth’.  Reading between those words is the idea that invention is dynamic and constantly open to reinvention.

OK, onto whats on my mind, right after we answer a quick question.  Was object-oriented software development discovered or invented?  Restated: is OO technology a naturally occurring thing or is it man-made?  If you believe it to be discovered, please leave a comment.

So whats on my mind?  The ardent reader by now will have seen it coming – I have re-invented object-oriented software development methods.  With a little help from my friends of course.  Or maybe de-invented, as it is my opinion that when OO techniques first came to the fore, in the early 90’s, there was great promise that it would revolutionise software development, which it clearly has not.  Those early days were focussed on simplicity – an object is simply data + methods, and that approach to be pervasive throughout – so your analysis and design is done in an object oriented way (OOAD) and then you program that way (OOP).  In those days, tools & technology let us down; there were no useful OOAD capable  modelling tools, nor were there any languages that were capable of properly implementing all of this.  Again, those who will claim that c++, smalltalk, eiffel etc were all capable languages in the early 90’s are welcome to state their case.  The closest IMO was a 3 1/2 GL called Gupta or later Centura SQLWindows.  In 1993, it was a WYSIWYG windows designer and full MI OO language.  Sadly, it was interpreted, not compiled, and more sadly, had no dynamic instantiation  – it was GUI based, so you could create windows objects on the fly, but not abstract classes.  As we moved through the 90’s, we got Delphi, Java and eventually .NET.

And dont forget VB.  Wasn’t exactly object oriented (lacked polymorphism) but it was easy to use and had some good stuff.  It was the beginning of the end.  Back in the 80’s, when we were doing Cobol CICS and IMS on IBM mainframes, we were basically programming abstractly – we were treating streams of bytes from 3270 terminals as formatted data and applying business rules and persisting that data.

Then along VB, Delphi and even good ole Gupta, and suddenly we could code directly against the ‘real’ data – on a form – we could trap on_lostfocus and apply formatting and rules to the contents of a textbox on a screen.  Too cool!  Too easy.  Far too easy in fact.  We had stopped programming abstractly, and coding rules directly in screens.  At the cost of all reusability.  Didnt we hear that its best practice to separate the presentation layer from the functional layer?  Coding on_lostfocus rules on a screen does not abide by that.  But luckily some of us hung on to that abstraction.  It was tough for a while, but we rode it out.  The rest of the world went on to MVC, MVVM, n-tier architectures, SOA and so on.  When .NET came along, we finally had a platform that had fantastic OO features (apart from MI – later on that!) and no real business application development platform or paradigm to cause legacy issues (as Java does with j2EE).  I can see those ‘leave a comment’ buttons clicking furiously now…

Sticking with a model of OO simplicity – data + methods – and focusing on abstraction, leaves you with a very simple 2-tier architecture: the business or model layer and the presentation layer.  The presentation layer can change data or invoke methods of the business layer.  All rules, formatting, persistence, everything happens inside the business layer, in a wonderful world of ultimately reusable components.

OK, here it comes – deep breath – I have invented the software factory pattern, as opposed to the workshop pattern, where the gains are greater than the gains of manufacturing assembly lines vs workshops.

As far as I know, nobody else has.  One of the signs of a true reusable architecture is code normalisation – every rule is embodied in a pattern and that pattern is a reusable component and so every discrete rule is only coded once, and reused.  No ctrl-c/ctrl-v.  Reused by declaration, in a model.  MDD to the max.  DSLs are a wonderful thing, but why not make the DSL based on an extensible set of abstract Types, and why not allow the modeller or studio to fully understand the DSL, to bring it to the domain of the analyst rather than just the developer?  Model Driven Development based on an extensible (uninhibited) DSL with no code gen, only declaration.  With client side agnosticism – strap on a Silverlight, Windows Forms, Android front end without any code changes – the presentation layer is simply a subscriber to various data + methods of the business layer.  Sound too good to be true?  It isn’t, cause I’ve been building systems this way for literally decades.

Good to finally get that off my chest.


Reuse in todays world.

February 4, 2011

Imagine a world without factories (ok ok the greenies – like me – might see this in quite a positive light); that is a world without automation, without standards, without reusable components.  A world where everything is hand made.  Again, hand made is often perceived as a very good thing – albeit expensive – but many products in todays world are much better made in a factory – on an assembly line (ideally by tools/robots not people) where people of course (not machines!) designed the whole process.  Cheaper, more consistent and higher quality than hand made.

Well, in the world of software development, we are in the dark ages.  Everything is hand made, but with fantastic power tools.  We have not adopted the factory pattern [yet].  But you know, thinking clearly about this – we have so much more to gain than manufacturing by adopting the assembly line approach: first, we consume no raw materials; I can conjure up as many 1’s & 0′ in fancy patterns as I need for free.  Next, we have polymorphism; imagine we have designed an assembly line for 2 slice toasters and someone needs a 4 slice version.  We have lots of short panels – we retool to make longer panels, make many of those and assemble them on a modified assembly line.   In software, I inherit a 4 slice toaster side panel from a 2-slice toaster side panel in 1 line of code and adjust the size.   Virtually free.  I inherit all the goodness of the 2-slice panel but modify or tweak something.  Note: a lot of careful design has gone into the breakdown of the components – dont immediately get upset because your first effort at reuse in a similar way ended in disaster.

OK, now what?  I have all these components, how do I assemble them ad run them?   First, lets look at the evolution of the components.  They probably look like numbers, strings dates etc and various combinations of these.  But wait!  These are classes – they can have methods as well as data.  So now we have numbers, strings and dates on steroids.  And all polymorphic.  I can inherit an email address Type from string, or a phone number Type.  Maybe a dollar value from number, or a percentage.  Now any time I need one, I just pull it out from the toolkit.

“Pull it out of the toolkit”?  I suppose that means I model something with it.  Imagination time again; those that are not dreamers or dont know the TLAs of DSL and MDA can skip this bit.  Imagine a world where DSLs are mainstream – you may invent a language to suit your needs at any time.  Only proviso – it needs to be built upon the foundation Types from above (which include booleans, blobs and all known Types) and coded – once in a language.  Oh, and it needs to be documented, in a manner that a modelling tool understands to that the modelling process implicitly can speak the DSL.  Have you got the picture?  I am in a new domain, I invent a new language (in object.method notation of course) and I can model in that language.  Wouldnt that be cool!  And finally, having described both the Types/DSL in the modeller, and the application using that DSL, is it a big ask to be able to execute this application (remember above we had to code our DSL as well as document it).

Whats this blog all about?  Reuse.  So now the full circle: remember the core Types – number, string, date etc – they are reused across all of your applications!  Think about it for a while – that’s basically the promise of th OO panacea of the 90’s, albeit a few years late.

Now back to that lovely wild green happy planet with no factories…


Just how lazy can you get!

February 1, 2010

It has been said that necessity is the mother of invention.  I disagree.  It may be the father of invention, but the mother of invention is laziness.  And I am lazy, really really lazy.  I also get bored doing simple repetitive tasks.  Sounds pretty normal eh?  Well, I will claim to out-lazy the lot of ya!   You see, I am so lazy I have invented an assembly line paradigm for software development.  Think what that did for the manufacturing industry – imagine the cost of a toaster if built by hand from raw materials – even with the worlds best power tools.  $1000+ for sure – every hour of labour will cost you, and there’s a lot of labour in a toaster formed from sheet metal.

So some lazy guy decided do a bit of hard yakka – create some standards, do some design, build a factory to prefab components etc and then he just watched it tick along.  churning out high quality consistent yet inexpensive goods.

The amazing thing with software development is that we are abstract – we consume no raw materials!  So we don’t need to actually prefab anything – we just design them and conjure them up at will – and zero cost.

Sounds really good to me.  So, I ask, why is the rest of the world seeking power tools?  Bigger, faster, stronger, quicker power tools.  And thats just what the vendors are giving us.  Stop and think about this – isn’t it simple – why don’t we ask the vendors for true reuse – an assembly line for software development.  Maybe they’re not lazy enough, in the right way.  Or maybe they like doing simple repetitive tasks – starting each project from a clean slate and building the foundations again.  And each screen/class/? from scratch.

I wont say it’s not hard – because it is.  Or rather it was.  Its done now, and we at expanz are living off the avails.

Code is not a bad thing – it needs to be written.  Once.  Anything more than that is wrong.  We all normalise our databases, but love to repeat code.  And test it, and fix, and test it and write some more and then fix that.  If most of the code of a project was inherited from previous work, there’s that much less to test and fix.

So let’s get all you lazy people out there to challenge the vendors to give us something really useful to us – give us an assembly line, not power tools.


There goes the neighbourhood

January 17, 2010

The term ‘multi-tenanted’ is another of those misunderstood and abused terms.  Multi is pretty easy to understand – but ‘tenant’ is where the confusion lies.

I love communities; by locale and/or by sport or anything.  Groups and communities make the world go round.  Lets start with where you live; it’s a community (for city folk anyways), and if you’re in a unit block (and you rent) you are a tenant.  There are other tenants in other units, they come and go, and there are other unit blocks nearby with tenants.  Somewhere there may be a pool or other shared resource, available to the community, or some portion of.

Basing a ‘multi-tenanted’ IT system on that paradigm is what we tried to achieve at expanz.  Ultimately it means different things to different folk, so flexibility is paramount, but you gotta start somewhere.  Remember, we are a platform first and foremost, and our ERP system is simply a sample implementation.  So at the platform level, even the whole notion of multi-tenancy must be optional – not every system needs that.  Most LOB systems certainly dont.  What the ESA gives you is the concept of a tenant id at a session level (technically called a Context Id – more generic) that does not need to be used.  If used, when a user logs in (creates a Session) you can assign a Context Id and easily refer to that from anywhere in your code.

Now thats the most complex way to achieve it.  You could simply create application servers and databases as required, with zero coding effort and complete isolation between tenants.  But that doesnt scale to large numbers, and – in our minds more importantly – breaking that isolation is tough.  Back to the community thing.   In our ERP system, we use Session Context as Host Company Id – a Host Company being a tenant, with unlimited users per tenant.  Being a full featured ERP system, we have the concept of Articles (and services) as an abstract concept with stock being defined as an Article at a Stock Location.  Now we have a handful of ERP installs (each with their own AppServer farm); each of these is a different community.  There is an Event Management community, a Distribution community etc.  Each of these communities has a few shared resources and specialisations unique to that community, and common/shared data as well.  This is paramount.  When two companies trade goods, and they are both tenants in our ERP system, its best to have a common set of articles, rather than forcing a many-many mapping.  Sure you need to have an internal and external part number for trading partners outside the community, but for multiple tenants trading on the same community article abc123 should be universal.  Certainly there needs to be tenant-specific details – descriptions, pricing, stock etc but much of the data should be common.  This applies to a handful of features in the ERP system, and can differ by community.

Another great use for this is in accounting – different divisions of the same company can run as multiple tenants, with their own unique ledgers, but can share articles, suppliers etc and the parent company can then perform group-level or aggregated financial reporting.  This of course breaks the isolation between tenants, but for good – not evil – purposes.

Finally, where different tenants in a community have customisations, those customisations need to live side-by-side with core code and other customisations.  The ESA allows for this purely through metadata (and technically via reflection).  So a company that has a custom Sales Order class simply overrides the default classname with its own in the configuration and when a SalesOrder for that tenant is requested, the factory gives it a custom class.  Of course, the underlying database schema needs to be thought of carefully, and customisations necessarily must use non-mandatory columns for their Fields, but in general most of the customisations are code-based, not data-based.  For a community at least.  The schema of our Event/CRM database is different from the schema of our Strata Management database.

OK, one last point – at the platform level again, that good ole’ Session Context can be used to gather statistics and influence certain aspects of a Site (Load balancer + application server(s)).  Eg statistics by tenant, coercing SLA/performance features etc.  You don’t want one bad tenant hogging resources.  There goes the neighbourhood.


Art vs Science

December 27, 2009

I went on a run this morning and so had time to reflect on the past week or two.  At some point I went through my Christmas checklist:

  • drink too much – check
  • eat too much – check
  • have fervent religious debate – nope!!!

Now as an atheist I am finding it harder and harder to have good religious debates, even at Christmas.  What is one to do, when one loves so much to argue fiercely have a calm open-minded discussion?  Start one yourself!  So here I go: is this thing called Software Development (or Software Production as the marketing guys would like) more Art or Science?

I want your opinion (no I just want an argument) but first you’re gonna get mine.

I’ve come full circle on this: at first I felt it was art, then decided it was probably science, but have since realised it’s a bit of both but more heavily weighted towards art.  When I sit in Visual Studio, or Eclipse, or Blend, or Photoshop – I feel more of an artist with a canvas and a palette than a scientist with a lab and instruments.  It’s creative.  Sure there is some component of known algorithms and simple best practices, but it’s highly creative and inventive.

Whether you are writing device drivers for Linux, a 3D audio rendering engine for WoW, a cost-center P & L accounting report, or an online blogging engine GUI, it is a highly creative exercise.  If it wasnt, we’d get a machine to do it.

Which brings me to the other reason (apart from wanting to argue): this thing – Software Development – is being rebranded as Software Production by the marketing guys because we have invented the assembly line pattern for software as the manufacturing industry did for hardware.  And cars and toasters are not developed on an assembly line, they are produced.  So as a Software Developer Producer I feel that it is taking away my artistic rights and freedoms.  Which as someone who knows that it is an art starts to feel uneasy about it.

Having sat on both sides of the fence – as tools/platforms producer and consumer, I know how important flexibility and openness is to a platform, both pragmatically and emotively to the developers (oh I just can’t call them Producers?) using it.

So to the marketing guys – maybe ‘Production’ needs a second thought, and to the developers – your artistic freedoms are intact and in fact more highly regarded than ever.

Your thoughts?



December 14, 2009

The first time I was exposed to a guaranteed ERP implementation advert, I had to check the calendar.  Nope, not April 1.  Quick look around – nobody watching me quietly grinning, no sign of hidden cameras…  OK, better read it again – I must be missing something obvious.

” We guarantee a successful ERP implementation, following our simple 2-step process.  First we carefully analyse and model your business, then we change your business to make it fit the software”

Today I would put my glasses on and try again, but that was a long time ago and my sight was perfect.  OK, I give up – I dont get it.  Am I supposed to laugh, cry, what?  Yes, it was one of those laugh and cry situations – funny to you but real and so you want to cry.  I could not imagine people actually choosing to do this – I had technology and processes that could easily develop a perfect-fit solution, yet people were willing to pay good money for a generic solution (although guaranteed implementation success). 

ERP had such a bad name that companies had to do that – guarantee success – to get a foot in. 

Now in reality, there could be nothing better than an all-in-one integrated ERP system that covers every aspect of an Enterprises systems.  No duplication or integration hassles.  All reporting data available at one place.  Beautiful.  And in real time.  Almost. 

Whats missing?  Well, the endpoints of your business – the customers and suppliers.  They are outside the Enterprise.  They are part f the larger community that touches your Enterprise.  They need access to your ERP system – so we have rebranded ours CRP – Community Resource Planning – because we love to open portals for customers and suppliers to create and manage orders, accounts etc.

Now thats better.  A single source of truth – all data in one integrated system, that is open to the community and all in real time.  And 100% customised fit for your business.

So you ask how technology and processes can achieve that at reasonable cost?  Easy – nothing different than we’ve been talking about.  Abstract business objects, complete separation of business layer from presentation layer, rich/thin clients and enhanced Business Process Modelling.

I can inherit from a string.  I can inherit from a business object like a person.  I can inherit a parent class and recast its aggregate children to a better class at the same time (see https://expanz.wordpress.com/2009/11/10/multi-context-inheritance-and-other-space-time-phenonema/).  Finally, I can inherit an ERP CRP system.  In a single line, I inherit a full system, and then I customise it.  Inherit, not copy/paste.  With screens, yes I can copy/paste because there is no code there – just layout.  In fact I use that process – screen customisation – to help with the gap analysis, leaving behind tailored functional screens.  Easy.


Even we’re feeling the heat today

November 27, 2009

Normally there is a lovely ocean breeze blowing in Manly- home of expanz – but today its hot and the wind is out of the north west.  So we’ve got the fan on.  Every time it gets hot the global warming debate heats up, and then the IT world talks of ‘green IT’. 

My take on green IT is pretty much my take on the current world of IT as a whole – we’re doing it wrong!  OK, not wrong, but far less than optimal – we’re not being smart about it.  We’re taking the safe shortcuts, not investing in anything but the very short term and certainly not thinking outside the square.

Thats why the ESA is here.  Its different and its long term.  And its very very green (IMO of course).  How?  In many ways… Its an assembly line, so it reduces development effort – less A/C, lighting and PC power for the dev team.  The AppServer, being stateful, is far more efficient than a typical stateless model (for business apps) and so the server room uses less power.

But most importantly, and what most people miss – fail to see – its that it can be used to build systems that are a 100% fit to a business and so optimise thats business’s work flows.  This of course is not solely the domain of the ESA – this is an area of ‘green IT’ that is inevitably overlooked.  The fact that a business’s IT systems can greatly alter their overall efficiency is clear and simple, but a significant portion of that is related to sustainability.

I love our ERP system and the clients we have – lets do a short case study on one of them purely from a ‘green IT’ point of view.  A distributor of organic food & drinks adopted our system and here are a few improvements we have made: first, the sales reps are now online with mobile devices, rather than having to return to the warehouse, cradle & sync the orders.  They leave their last customer for the day and drive home, not to the warehouse then home.   Next of course is realtime visibility and accuracy of the orders, so the warehouse becomes more efficient in its stock planning and movements.  Our forecasting module is so highly tailored for them that their wastage is greatly reduced (they do a lot of perishables that are seasonal) but they almost never short supply either.   We save a lot of paper as well – taking POD electronically – signature capture on the mobile devices, using screen PDF’s and other reports vs hard copy.  Delivery route optimisation is easy these days as well – here’s a list of addresses I need to visit, combined with a good realtime mapping and traffic system and you spend far less time stuck in traffic.

I believe that businesses adopting new IT systems should think more laterally in ways that a new system can improve and optimise their overall day to day activities, rather than taking an inflexible package and keeping the status quo.  Continual improvement.

Green IT should focus more on the real world savings a tailored system can provide rather than simply reducing server CPU cycles.


Meaningless Numbers

November 24, 2009

For those of you who enjoy performance metrics and statistics, I bring to you the first semi-serious attempt at producing some for an ESA application.  Personally, I am a realist, who thrives on theory but trusts only empirical data, and isn’t happy until the two meet.  And I’m pretty happy at the moment.  Theory says that a stateful ESA server should be many times more efficient than a stateless server given that there is no state get/set overhead.  So when these first practical tests are performed and the numbers look good, I am happy.

Test PC is an AMD triple core, 4G memory running server 2008 x64.  Server test is simply incrementing a number, but that of course comes through as a discrete method request via a web service as normal.

Test harness is a WPF application which pretends to be some number of distinct users and then runs a preset script indefinitely and short random intervals.  I ran 3 instances of this each pretending to be 100 users, very fast typing users, as they pushed through between 2 and 5 transactions/second.

I took screenshots of task manager at the start, after 300 users logged in, after 100 then 200 then all of them became busy.  Some are memory sorted, the rest are cpu sorted.  A final shot was taken after stopping, but not logging out the 300 users.  The test harnesses also log a time-stamped entry every 200 transactions.

Here they are

Now for the transaction rates – they are all slightly varied as the request intervals have a small random component.   They also ran for different lengths of time as indicated above – 100 went usy, then 200 then 300 then they stopped).
Tester 1 (time=0:20, transactions=5400, rate=16200/minute)
Tester 2 (time=0:39, transactions=13200, rate=20300)
Tester 3 (time=1:06, transactions=22000 rate=20000)

Total transaction rate was 56000 transactions / minute and the heart rate had just started to go up.  And thats just one AppServer!  One day soon I will set up a dedicated machine with a load balancer and maybe 4 appservers and another PC running the test harnesses and see if I can break 1 million transactions / minute on pretty standard hardware.

You know what I’m gonna say.