AKH Online, Inc

In this article, I stress the importance of bringing true QA assets to bear on your development enterprise. Here we talk about process.

Two types of "testers"

It's generally agreed that there are 3 types of application errors: syntax errors, run-time errors, and logic errors. To find them, you may need two types of tester roles: QA and Business.

Syntax and run-time errors are visible for anyone following a test plan. You do not have to be a rocket scientist to see that the application breaks when serving a page of equations - or that "pasword" is spelled wrong. The job of the QA tester is to explore every possible input in an attempt to stop the application from performing basic service requirements correctly.

Logic errors are a little trickier. You will need a rocket scientist to see that - while the application does return a page of equations - the resulting formula is actually wrong. Business testers know the rules an application should follow, and are qualified to audit the results.

This little article is about testing for syntax and run-time errors in the role of a QA tester.

Regression test plan

With every new release of the application, a complete "regression test" may be required. This is because the dependencies of one part of a program upon another may not have been fully considered. For example, the addition a preferred text size to the "My Preferences" section can impact the login process (where user preferences are often retrieved). Simply testing the "My Preferences" section will not expose a potential bug in this example.

Because it's so important to explore every nook and cranny of your application, you really need to have a test plan. This checklist will help your tester hit every function of your application, using each supported browser and as every type of user.

Basic concept of a regression test plan

Here are some of the items a QA tester should apply while testing:

Multiple browsers

Check the application with more than one browser. It’s also a good idea to test on more than one device, even from multiple locations.

Display scripting errors

Insure that your browser is set to display scripting errors. In IE this is Options | Advanced | Display a notification about every script error

Entering data

Special characters In form fields that ask for text input, provide values that include any of these characters:

• & ampersand and % percent symbols
• ' “ single AND double quote
• , comma
• < less than sign

See if the values and code survive the storage, reporting and retrieval process. Single quotes affect JavaScript & SQL. Commas can cripple reports.

Numeric fields

Enter text in a field that requests a number. Try 29.9 as an age, or 10.0E as a price.

Losing your place

With a complex web form designed to edit or update lots of stuff, check for a fixed visual indicator of what it is that you are changing. Note any page in which a user who was distracted from the page for some period of time could forget what record was being changed.

Leaving data out: Empty fields

Attempt to submit the form without entering anything. For any page in which a pop-up launches to add additional required settings, attempt to proceed anyway (ignore or close it). Be really dumb...

Data formats

Test any field which suggests that a user is responsible for formatting the data (for example, "Enter the date in this format"). Reject it as non-compliant if it accepts anything but the required format without correction, or breaks.

Dates

In any form field that accepts a date, see if it is possible to enter a date prior to 1752 and check the application's behavior thereafter. If you can’t go that far back, try anything before 1970.

Query strings

If you notice a web page that was loaded by use of a query string in this or similar format :

remove one of the "=value" parts from the query string. Hit Enter to load the page again.

Words vs. numbers

If one of the values is a number, replace it with a neutral word like "one" and hit Enter to load the page again.
If any of the values is a recognizable word, insert the character "&" or “<” into
the middle of the word, and hit Enter to load the page again.

PCI Compliance

Normally required when your site is handling money, consider testing for PCI Compliance even when you're just logging in users for non-financial reasons. All the basic requirements (against cross-site scripting, data injection attacks, etc), are really just darn good ideas in running a tight virtual ship. Consider that someone using your site may have (foolishly) used the same password that they also bank with.

How to report something you found

Explain the nature of the problem in detail. Simple statements like “it didn't work” or “it displayed an error” don't help too much. What did the first line or two of the error say? How did you know that it didn't work? Copy the URL from the top of the browser. Grab a “screen capture” if you know how.

We will need to recreate the issue in order to fix it, so please list the specific steps – from the top – that we would follow to see the same thing as you. If possible, ask a co-worker to try the same steps on their computer and note the results if different.

Too many conditions?

A common problem in defining an issue comes from changing too many things between each attempt to recreate it. For example, you may see an error on your computer and ask a co-worker to reproduce it. But he or she may log in as a different type of user and might try a different brand of browser and complete the form differently than you did. When bug hunting, try to keep as many things exactly the same as you can. Then, you'll be able to find the specific text, setting, thing or things that all conspired to reveal that elusive error!

Is it really breaking?
Is it really working?

Please don't describe every unexpected thing as an error. A popup tells you February 30th "is not a valid date". That's not an error. Nothing is breaking here. It's a message.

If in doubt, check the design specification or ask the developers what you're seeing and why.