How to do Integration Testing

How to do Integration Testing

Integration Testing is a four step procedure. Below are the steps for creating integration test cases:

1. Identify Unit Interfaces: The developer of each program unit identifies and documents the unit’s interfaces for the following unit operations:- External inquiry (responding to queries from terminals for information)- External input (managing transaction data entered for processing)- External filing (obtaining, updating, or creating transactions on computer files)- Internal filing (passing or receiving information from other logical processing units)- External display (sending messages to terminals)- External output (providing the results of processing to some output device or unit)2. Reconcile Interfaces for Completeness: The information needed for the integration test template is collected for all program units in the software being tested. Whenever one unit interfaces with another, those interfaces are reconciled. For example, if program unit A transmits data to program unit B, program unit B should indicate that it has received that input from program unit A. Interfaces not reconciled are examined before integration tests are executed.3. Create Integration Test Conditions: One or more test conditions are prepared for integrating each program unit. After the condition is created, the number of the test condition is documented in the test template.4. Evaluate the Completeness of Integration Test Conditions: The following list of questions will help guide evaluation of the completeness of integration test conditions recorded on the integration testing template. This list can also help determine whether test conditions created for the integration process are complete.1. Is an integration test developed for each of the following external inquiries:- Record test?- File test?- Search test?- Match/merge test?- Attributes test?- Stress test?- Control test?2. Are all interfaces between modules validated so that the output of one is recorded as input to another?3. If file test transactions are developed, do the modules interface with all those indicated files?4. Is the processing of each unit validated before integration testing?5. Do all unit developers agree that integration test conditions are adequate to test each unit’s interfaces?6. Are all software units included in integration testing?7. Are all files used by the software being tested included in integration testing?8. Are all business transactions associated with the software being tested included in integration testing?9. Are all terminal functions incorporated in the software being tested included in integration testing?

How to do Boundary Value Analysis

How to do Boundary Value Analysis

Boundary value is a software design technique to determine the test cases covering off by one errors.

Boundary value analysis is a methodology for designing test cases that concentrates software testing effort on cases near the limits of valid ranges Boundary value analysis is a method which refines equivalence partitioning. Boundary value analysis generates test cases that highlight errors better than equivalence partitioning.

The purpose of boundary value analysis is to concentrate the testing effort on error prone areas by accurately pinpointing the boundaries of conditions,(e.g., a programmer may specify >, when the requirement states > or =).

To set up boundary value analysis test cases you first have to determine which boundaries you have at the interface of a software component.This has to be done by applying the equivalence partitioning technique. Boundary value analysis and equivalence partitioning are inevitably linked together. For the example of the month in a date you would have the following partitions:

 ... -2 -1  0 1 .............. 12 13  14  15 ..... 
      --------------|-------------------|---------------------
 invalid partition 1     valid partition    invalid partition 2

Applying boundary value analysis you have to select now a test case at each each side of the boundary between two partitions. In the above example this would be 0 and 1 for the lower boundary as well as 12 and 13 for the upper boundary. Each of these pairs consists of a "clean" and a "dirty" test case. A "clean" test case should give you a valid operation result of your program. A "dirty" test case should lead to a correct and specified input error treatment such as the limiting of values, the usage of a substitute value, or in case of a program with a user interface, it has to lead to warning and request to enter correct data. The boundary value analysis can have 6 testcases. n, n-1,n+1 for the upper limit and n, n-1,n+1 for the lower limit.

There are two steps to perform boundary value analysis:
1. Identify the equivalence classes.

2. Design test cases. 

Step 1.

Follow the same rules you used in equivalence partitioning. However, consider the output specifications as well. For example, if the output specifications for the inventory system stated that a report on inventory should indicate a total quantity for all products no greater than 999,999, then you d add the following classes to the ones you found previously:

 The valid class ( 0 < = total quantity on hand < = 999,999 )

 The invalid class (total quantity on hand <0)

 The invalid class (total quantity on hand> 999,999 )

Step 2.

In this step, you derive test cases from the equivalence classes. The process is similar to that of equivalence partitioning but the rules for designing test cases differ. With equivalence partitioning, you may select any test case within a range and any on either side of it with boundary analysis, you focus your attention on cases close to the edges of the range.

Rules for Test Cases

1. If the condition is a range of values, create valid test cases for each end of the range and invalid test cases just beyond each end of the range. For example, if a valid range of quantity on hand is -9,999 through 9,999, write test cases that include:
1.        the valid test case quantity on hand is -9,999
2.       the valid test case quantity on hand is 9,999
3.       the invalid test case quantity on hand is -10,000 and
4.       the invalid test case quantity on hand is 10,000
You may combine valid classes wherever possible, just as you did with equivalencepartitioning, and, once again, you may not combine invalid classes. Don�t forget to consider output conditions as well. In our inventory example the output conditions generate the following test cases:
1.       the valid test case total quantity on hand is 0
2.       the valid test case total quantity on hand is 999,999
3.       the invalid test case total quantity on hand is -1 and
4.       the invalid test case total quantity on hand is 1,000,000
2. A similar rule applies where the, condition states that the number of values must lie within a certain range select two valid test cases, one for each boundary of the range, and two invalid test cases, one just below and one just above the acceptable range.

3. Design tests that highlight the first and last records in an input or output file. 

4.Look for any other extreme input or output conditions, and generate a test for each of them.

Integration Testing

Integration Testing

 Integration testing is designed to test the structure and the architecture of the software and determine whether all software components interface properly. Integration testing does not verify that the system is functionally correct, only that it performs as designed.

It is the process of identifying errors introduced by combining individual program unit tested modules. Integration Testing should not begin until all units are known to perform according to the unit specifications. It can start with testing several logical units or can incorporate all units in a single integration test.
Below are the four steps of creating integration test cases:
Step 1 - Identify Unit Interfaces: The developer of each program unit identifies and documents the unit’s interfaces for the following unit operations:
- Responding to queries from terminals for information- Managing transaction data entered for processing- Obtaining, updating, or creating transactions on computer files- Passing or receiving information from other logical processing units- Sending messages to terminals- Providing the results of processing to some output device or unit
Step 2 - Reconcile Interfaces for Completeness: The information needed for the integration test template is collected for all program units in the software being tested. Whenever one unit interfaces with another, those interfaces are reconciled. For example, if program unit A transmits data to program unit B, program unit B should indicate that it has received that input from program unit A. Interfaces not reconciled are examined before integration tests are executed.
Step 3 - Create Integration Test Conditions: One or more test conditions are prepared for integrating each program unit. After the condition is created, the number of the test condition is documented in the test template.
Step 4 - Evaluate the Completeness of Integration Test Conditions: The following list of questions will help guide evaluation of the completeness of integration test conditions recorded on the integration testing template. This list can also help determine whether test conditions created for the integration process are complete.
Q1. Is an integration test developed for each of the following external inquiries:
- Record test?- File test?- Search test?- Match/merge test?- Attributes test?- Stress test?- Control test?
Q2. Are all interfaces between modules validated so that the output of one is recorded as input to another?
Q3. If file test transactions are developed, do the modules interface with all those indicated files?
Q4. Is the processing of each unit validated before integration testing?
Q5. Do all unit developers agree that integration test conditions are adequate to test each unit’s interfaces?
Q6. Are all software units included in integration testing?
Q7. Are all files used by the software being tested included in integration testing?
Q8. Are all business transactions associated with the software being tested included in integration testing?
Q9. Are all terminal functions incorporated in the software being tested included in integration testing?

Equivalence Partitioning

Equivalence Partitioning

Equivalence partitioning is a test case selection technique for black box testing. In this method, testers identify various classes of input conditions called as equivalence classes. These classes are identified such that each member of the class causes the same kind of processing and output to occur.

Basically, a class is a set of input conditions that are similar in nature for a system. In this test case selection technique, it is assumed that if the system will handle one case in the class erroneously, it would handle all cases erroneously.
This technique drastically cuts down the number of test cases required to test a system reasonably. Using this technique, one can found the most errors with the smallest number of test cases.
To use equivalence partitioning, you will need to:Determining conditions to be testedDefining and designing testsDetermining conditions to be tested:All valid input data for a given condition are likely to go through the same process.Invalid data can go through various processes and need to be evaluated more carefully. For example:Treat the blank entry differently than an incorrect entry.Treat a value differently if it is less than or greater than a range of values.If there are multiple error conditions within a function, one error may override the other, which means that the subordinate error does not get tested unless the other value is valid.Defining and Designing Test Cases:First, include valid tests and include as many valid tests as possible in one test case.For invalid input, include only one test in a test case in order to isolate the error.Example: In a company, first three digits of all employee IDs, the minimum number is 333 and the maximum number is 444. For the fourth and fifth digits, the minimum number is 11 and the maximum number is 99.So, for the first three digits the various test conditions can be
a. = or > 333 and = or <>b. <>c. > 444, (invalid input, above the range)d. Blank, (invalid input, below the range).And for the third and fourth digits the various test conditions can bee. = or > 11 and = or <>f. <>g. > 99, (invalid input, above the range)h. blank, (invalid input, below the range)
Now, while using equivalence partitioning; only one value that represents each of the eight equivalence classes needs to be tested.
Now, after identifying the tests, you will need to create test cases to test each equivalence class. Create one test case for the valid input conditions and identify separate test cases for each invalid input.
As a black box tester, you might not know the manner in which the programmer has coded the error handling. So, you will need to create separate tests for each invalid input, to avoid masking the result in the event one error takes priority over another.
Thus, based on the test conditions, there can be seven test cases:Test case for a and e - (both are valid)Test case for b and e - (only the first one is invalid)Test case for c and e - (only the first one is invalid)Test case for d and e - (only the first one is invalid)Test case for a and f - (only the second one is invalid)Test case for a and g - (only the second one is invalid)Test case for a and h - (only the second one is invalid)

How to decide when software is ready to release

How to decide when software is ready to release

Deciding when software is ready to release is a difficult. You have pressure from all sides to release perfect software, with added features. Below are some common facts:
- The engineers have said that the code was complete months ago, but are still making code changes.
- Sales people promised the software to major accounts months ago and are making commitments for the next release.
- The product manager wants a few more features added and you want to release a zero defect software.
Having no specific release criteria, many organizations wait for the manager to say "Release it", not knowing when or why the decision was made at that time. In other cases, there is a general group consensus to just push the software out the door because the tired and stressed group wants to move out from the seemingly never ending cycles.
You can never take all of the stress out of releasing good software, but by planning ahead and setting up a good defect tracking repository, predetermining acceptable bug counts, properly testing the software and effectively triaging the defects you can always know the current state of the software, but most importantly know when the software is ready to ship.
Defect Tracking Repository:
The most common as well as important tool to determine the state of the software is the defect tracking repository. The repository must be able to provide information required for the triage process. The repository must include the following info:
- The expected results of the testers test- The steps to recreating the defect- The version in which the defect was found- The module in which the defect was found- The severity of the defect- A short descriptive name for the defect- A description of the defect- The actual results if the testers test- Resolution notes
The project team should be able to query the database and gather information such as defect counts at various severity levels, and module levels, descriptions of the defects and steps to reproduce them. The repository becomes the archive for the project and it becomes important that correct and complete information be entered in order to use this information for later projects as a planning tool.
Predetermined Acceptable Bug Counts: The goal of releasing software with no defects cannot be achieved given the limited time and resources so, in the preliminary planning of the project exit criteria must be set up. The exit criteria should then be translated into the test plan outline, which should include acceptable bug count levels in order to ship the software.
The exact acceptable bug count level is not some magic number that will insure a successful product, but more a target goal that the group can use to see that they are moving forward toward a common goal. An acceptable bug count statement may look like this:
- Showstoppers: There may not be any- High: There may not be any- Medium: There may not be any that have a high probability of appearance to the customer- Low: There may be a minimal amount in specified core areas and all other areas may have any amount.
In order for the entire group to know the state of the software at all timesit would be preferable to produce a weekly build grading process. The idea is that each week the build would be treated as if it were the software to be shipped and given a grade. The weekly grade keeps the team current on the software state and the team can see the software progressively improve.
This quantitative method takes a lot of the guesswork out of the equation.
Proper Testing: The key to shipping quality software is finding and fixing all defects. The testers responsibility becomes finding the defects and properly reporting them. In order for the testers to find the defects the lead tester must set up in the test plan outline, the specific stages of testing to ensure that the process is organized and covers all aspects of the software. The test plan may contain the following 7 phases:
- Test Case and Test Suite Planning- Unit Testing- Regression Testing- System Testing- Regression testing- Performance Testing- Release Candidate Testing
The goal is to turn up as many defects as early as possible in order to make the fixes easier on the engineers and to provide ample time for regression testing since every time the code is altered there is a greater risk of creating more defects.
As important as finding the defect, the tester must be able to correctly report it in the repository. This becomes extremely important in large projects where the defect counts may rise in the thousands.
Effective Triage:
The Test Lead and Engineer must sit down regularly to evaluate the reported defects and drive the process forward in the most efficient manner for the engineers as well as the testers. Much of the decision making here is based on queries from the defect repository, which shows the importance of the accuracy and completeness of the repository.
The creation of a "Hot List" which is a list of important bugs is a great tool to create. This can be done in an plain Excel sheet, which identifies the most important defects, their module and a brief description. This list is great to use in triage to identify the defect action items for the upcoming week.
Defects that prevent further testing in a certain area must be given precedence and the engineers can assist in the risk analysis of future modules to assist the testers in their test case development. In the determination of what to fix next generally it is advantageous to group and fix defects relating to the same module at the same time even though there may be other important defects in other modules. Grouping of defects assists the engineers finish a module and move on.
Shipping quality software is not an easy task but by being organized from the start of the project the ship date and defect counts don't have to be a "gut feeling". The project must have a Defect Tracking Repository that must be maintained and updated continually with correct and complete information. The Repository is used as the main tool for an effective triage process and in conjunction with the Test Plan Outline that must be createdat the start of the project describing in detail the exit criteria to ship the software.
Please note the entire process rests on proper testing and the use of proper test planning which is what uncovers the defects. If you do not uncover the defects the users of the software will.
Thus, by following the above described process it is possible to know the state of the software at any time so that the entire team knows the goal, sees the progress and knows when the software is ready to release.

Test Readiness

Test Readiness

Before starting the actual testing, it is important to check whether the system / project / environment is ready for testing. This is called as Test Readiness Review. It is better to do it with a checklist.
Below is a sample Test Readiness Review Checklist:
1. Whether all the tests are conducted according to the Test Plan / Cases ?
2. Are all problems / defects translated into Defect Reports ?
3. Are all the Defect Reports satisfactorily resolved ?
4. Is the log of the tests conducted available?
5. Is unit testing complete in all respects?
6. Is Integration testing complete ?
7. Is all the relevant documentation baselined ?
8. Is all work products products baselined?
9. Is the test plan baselined ?
10. Does the test plan contain the strategy / procedure to conduct the system test ?
11. Are baselined test designs and test cases ready?
12. Is unit/integrated test software ready ?
13. Is the user manual ready?
14. Is the installation procedure documented ?
15. Are all the product requirements implemented? 
16. Is the list of known problems available? Is there any "workaround" procedure for the known bugs ?
17. Are test environment needs met for Hardware, code, procedures, scripts, test tools etc.? 
18. List of exceptions in test software and test procedures and their work around if any?
19. Is the test reporting tool is available?
20. Are the designers educated on Test reporting tool?
21. Is any standard methodology / tool used and is appropriate to the type of the project?
22. Is the criteria for regression testing defined? Has the regression testing been done accordingly?
23. Is the source code available from the client for performing regression testing complete in all respects?
24. Is the source code freezed for testing?

Testing Team Organizing

Testing Team Organizing

The people component includes human resource allocations and the required skill sets. The test team should comprise the highest-caliber personnel possible. They are usually extremely busy because their talents put them in great demand, and it therefore becomes vital to build the best case possible for using these individuals for test purposes. A test team leader and test team need to have the right skills and experience, and be motivated to work on the project. Ideally, they should be professional quality assurance specialists but can represent the executive sponsor, users, technical operations, database administration, computer center, independent parties, etc. The latter is particularly useful during final system and acceptance testing. In any event, they should not represent the development team, for they may not be as unbiased as an outside party. This is not to say that developers shouldn't test. For they should unit and function test their code extensively before handing it over to the test team.

There are two areas of responsibility in testing: 
1. Testing the application, which is the responsibility of the test team2. The overall testing processes, which is handled by the test manager. 
The test manager directs one or more testers, is the interface between quality assurance and the development organization, and manages the overall testing effort. Responsibilities include:
• Setting up the test objectives • Defining test resources • Creating test procedures • Developing and maintaining the test plan • Designing test cases • Designing and executing automated testing tool scripts • Test case development • Providing test status • Writing reports • Defining the roles of the team members • Managing the test resources • Defining standards and procedures • Ensuring quality of the test process • Training the team members • Maintaining test statistics and metrics
The test team must be a set of team players and have the following responsibilities:
• Execute test cases according to the plan • Evaluate the test results • Report errors • Design and execute automated testing tool scripts • Recommend application improvements • Record defects 
The main function of a team member is to test the application and report defects to the development team by documenting them in a defect tracking system. Once the development team corrects the defects, the test team re executes the tests which discovered the original defects. 
It should be pointed out that the roles of the test manager and team members are not mutually exclusive. Some of the team leader’s responsibilities are shared with the team member and visa versa. 
The basis for allocating dedicated testing resources is the scope of the functionality and the development time frame, e.g., a medium development project will require more testing resources than a small one. If project A of medium complexity requires a testing team of 5, project B with twice the scope would require 10 testers (given the same resources).
Another rule of thumb is that the testing costs approach 25% of the total budget. Since the total project cost is known, the testing effort can be calculated and translated to tester headcount.
The best estimate is a combination of the project scope, test team skill levels, and project history. A good measure of required testing resources for a particular project is the histories of multiple projects, i.e., testing resource levels and performance compared to similar projects.