Software Testing

Introduction to Testing Principles and Test Process   

It is important that we achieve optimum results while testing the software without deviating from the goal. But we need to determine that we are following the right strategy for testing. For that, we need to stick to some basic testing principles. Mentioned below are the seven common testing principles and best practices in the software industry.

1. Testing shows the presence of bugs:

Without testing, we cannot conclude whether the software is defect-free or not. Once the software is tested only then we can understand if defects are present or not. Without testing we cannot understand if the software is working correctly or not. Also, a point here to note is that if we do not find defects it does not mean that the software is defect-free.

2. Exhaustive testing is impossible:

While testing we cannot test all the possible combinations of input and output for any given system. But that does not mean we do not have to test the system thoroughly. Exhaustive testing simply means that we need to set priorities, understand the risk and impact, and analyse before test execution.

3. Early testing is necessary:

Early testing is important to minimize the cost of removing defects. So, the process to find defects should start as early as possible. If the testers are not included in the requirement phase and if the defects or some human error is getting introduced, then it will cost human effort and money. It can ease the process, if testing is introduced in the early phase of the software life cycle.

4. Defect clustering:

A small number of modules usually contain most of the defects discovered during the pre-release testing or are responsible for most of the operational failures. The Pareto principle of 80-20 rule states that 80% of the defects are found in 20% of the module.

Reasons for failure can consist of:

The system is overly complex and complicated.
The code is volatile.
Interdependency of modules, change in one module depends on the change of the other module.
The inexperience of team.

5. Pesticide paradox:

If the same test cases are executed repeatedly after some time these test cases will no longer find new defects. So, to overcome this, test cases should be reviewed again, and new test cases should be added regularly.

6. Testing is context-dependent:

Testing is context dependent. Testing is done differently for different modules. For example, testing for safety or medical related systems, testing is to ensure accuracy and performance. Such systems are tested differently when compared to the testing for the e-commerce web site, as it is more related to the user experience and usability. So according to the context of testing one can divide what to test and how much to test it.

7. Absence of error fallacy:

Fixing and finding defects does not help if the system built is unstable and does not fulfil the user needs and expectations. A system with no defects does not mean that the system is ready to be delivered. This should not be the acceptance criteria.

Fundamentals of test process

1. Planning and control

Test Planning is the activity of defining the objectives of testing and the specification of test activity to meet the objectives and mission. In test planning activity, activities such as defining scope, approach, resources, and environments are discussed. In the test planning process, we have to identify which test items are to be tested and which are out of scope, roles and responsibilities of the resources in the team and roles of the stakeholders, the relationship between the testers and the developers and the environment requirements for testing the system.

The test control is the ongoing activity of comparing actual progress against the plan and reporting the status including deviations from the plan. This needs to be taken care of throughout the test activity.

2. Analysis and design

Analysis and design are concerned with the fine detail of what to test (i.e., test conditions) and how to consider test data that will be required for the test. Conditions on how to create the test cases and how to create test conditions so that a small number of test cases can cover as many of the test conditions as possible. Analysis and design are the activities during which general test objectives are transformed into tangible test cases and test conditions.

The main points in the analysis and design are as follows:

Reviewing requirements and other documents required for testing.
Each specification document is called test basis.
Analysis test data and conditions by analysing the test basis.
Design test cases.
Details of what the test condition should look like, and which tools should be considered for testing.

3. Implementation and execution

This is the activity where test procedures or scripts are specified by combining the test cases in a particular order and including any other information needed for test execution, the environment is set up and tests are executed. While the tests are executed, log the output, compare the actual and the expected output. If they do not match, look for the cause.

Mentioned below are the activities which are involved in implementation and execution:

Prioritising the test cases, creating the test data, writing the test procedures (where test procedures are the steps to execute the test cases).
Collecting test cases into test suites. In the test suites the test cases will be executed one after the another for efficiency.
The order of the test cases is determined manually or by automation tools.
Keeping a log of the output status of the test cases executed. Whether the test cases passed or failed, number of the cases which failed and to track the defects associated with those issues.
Reporting the defects to the developers and providing them steps to reproduce.
Version of the software used for testing, test data and testing tools used.
After fixing the bug we need to retest it and confirm whether the bug is fixed or not. Also, we need to check if this fix has any impact on other features.

4. Evaluating the exit criteria and reporting them

Evaluating the exit criteria is the activity where test execution is assessed against the defined objective.

Activities in this process include the following mentioned below:

Checking whether the previously defined exit criteria has been met or not.
Determine if more cases are required to achieve the exit criteria.
Writing the result for business sponsors and other stakeholders for reference.

5. Test Closure

This occurs at the project milestones such as when a software system is released, a test project is completed or cancelled, or a maintenance release has been completed. In the test closure process, it concentrates on ensuring that everything is finalised, synchronised and well documented.

In this process the activities included focus on following objectives: