Bill Payment Arthritis

I have spent couple of hours today trying to pay a telephone bill via the telephone provider's portal and these are the issues I faced:

- The telephone provider's portal has a prominent login button (with Signup) and signing up here/logging in here does not enable me for all services. For example, I can view the Value added services but not bill payment although 'Bill Payment' is UI-linked very close to the login button.

- Click on actual 'Bill Payment' link and I'm taken to a different site with the standard warning 'You are moving out of the trusted site with secure credentials...etc' and I wonder what has happened since I didn't log out.

- Next, I go to the real bill payment portal and from the URL I can see it is a different hosted site and I register/login once again. The password rules are different here (length of this password has to be 8 characters) while my earlier password to the apparently same portal had different rules with length/alpha numeric constraints. Nothing new here, so I proceed.

- Once I successfully login my Bill Pay portal, the bill is promptly put up (good integration with the database!) and then the next interface takes over. It looks like some kind of web service which displays all the banks that are approved with visa/master/diner card options and I proudly click my NetBanking.

- And now is the finale! I get an exception -" apache error - string out of length-10".
I forgive the site thinking perhaps the session didn't handle it well and again I go all the way back to my first page, login two times with different credentials and wait for the page to come up. With bated breath!

- Now, a different exception comes up - "Fatal error: Allowed memory size of 125829120 bytes exhausted".

- Now my son, aged 12, gives me an idea! Do it really fast - keep clicking all options as fast as you can and beat the time-out error. I laugh at that! What can a user do now other than perhaps dance in front of the computer to keep it in good humour.

I'm not sure if the Bill Payment test engineers tested much, but this is what anyone should test which I christen 'common sense tests'.

1. Estimate the number of users who would hit the site on an average and maximum (bill payment date). If there are 50,000 subscribers who appear in the log for a month, it is safe to assume 70% of the subscribers as maximum. I know that most companies don't have the luxury of affording a commercial testing tool that can simulate thousands of users and using an 'evaluation version' of a load tester does NOT give the actual picture at all. The results of the evaluation versions are not reliable and one has to find ways of simulating the no. of users by other means.

2. No. of users, no. of concurrent users, geographic distributions have to be tested. For example, Asia has about 300,000 million users while it progressively decreases for Europe, North America and becomes miniscule for Africa and Middle East. It is the not the % of Internet users that matter but the actual volume.

Account for latency tests from those demographic regions.

3. While ideally you will get marketing data about user behavior, how long a user's 'eye balls' are stuck to your landing page or how many times the same user might visit a typical category of web site, how many of us can afford the Forrester or Gartner report in terms of money or time? Rely on the good old fashion logs instead. There are open source based OLAP tools where you can create simple dimensions that you want to measure and run the logs through them. You will get a fair indication about user behaviour. If your company has a Tools Team, ask them to analyze for you. Base your tests on the data you get for your portal.

4. More often than not, like my Bill Pay application, each app is integrated (SOA) with another provider either providing a service or consuming or both and there will be standard SLAs for capacity and volume on both sides besides protocols, security measures etc. Check to have tests to see if the SLAs are defined as required by your business. It should have the right capacity planning.

5. Look for those pages or links or queries that are most accessed by users. A 'Search' or 'My Orders' is more critical than 'Customer Case Studies' from a performance point of view. Test and see if all parts of the portal have uniform performance problems or not. In technical papers, an often quoted line is this 'Run a mix of processing patterns and check the limits of infrastructure' :-). How's that for jargonizing!

6. Check for functional reliability - this means whether it is one user or thousand of them they can 'feel' the same for accuracy, security and ease of use and not broken sessions. I saw a hilarious note from a blog ( http://blogs.cio.com/node/228) that specified some rules that could affect functional reliability.

All traffic is encrypted. All fields that display sensitive information are invisible, unless you move the mouse pointer over it, and click (hold the click to see the info). All screen savers are locked on blank screen (no user customizable fancy-dancy screen savers) - and set at 1 Minute, maximum - no user ability to change / reset this. All user systems have USB disabled, no CD-ROM drive and no floppy drive. All passwords must be a minimum of 8 characters long, have at least 2 numerics, 2 symbols, 2 capital letters and 2 lower case letters. Zero repeat characters and no character can be used in the same position more than once in 16 months. Passwords must be reset every 28 days - no exceptions.


7. Last but not the least, examine whatever channels you have access to for 'unintended consequences'.

How a user did use it not to hack the system but to actually use it in some ways. When I got the string length exception in my portal, I wanted to somehow pay my bill. I googled for the error that said it happens with long named attachments. So, I set about looking for clues in the http path to avoid that. I couldn't hack it but what I'm saying is that a tester should observe a hacker/unintended user's behaviour and convert that to a test too.

Now, I sympathize/empathize with my Bill Payment portal. It was very kind to me! Tomorrow I will physically go and pay the bill!