Cookie Testing

Below is a list of major scenarios for cookies testing of a website. Multiple test cases can be generated from
these scenarios by performing various combinations.

1. Check if the application is writing cookies properly or not.
2. Test to make sure that no personal or sensitive data is stored in the cookie. If it is there in cookies, it should be in encrypted format.
3. If the application under test is a public website, there should not be overuse of cookies. It may result in loss of website traffic if browser is prompting for cookies more often.
4. Close all browsers, delete all previously written cookies and disable the cookies from your browser settings. Navigate or use that part of web site which use cookies. It should display appropriate messages like "For smooth functioning of this site please enable cookies on your browser."
5. Set browser options to prompt whenever cookie is being stored / saved in your system. Navigate or use that part of web site which use cookies. It will prompt and ask if you want to accept or reject the cookie. Application under test should display an appropriate message if you reject the cookies. Also, check that if pages are getting crashed or data is getting corrupted.
6. Close all browsers windows and manually delete all cookies. Navigate various web pages and check and see if these web pages show unexpected behavior.
7. Edit few cookies manually in notepad or some other editor. Make modifications like alter the cookie content, name of the cookie, change expiry date etc. Now, test the site functionality. Corrupted cookies should not allow to read the data inside it.
8. Cookies written by one web site should not be accessible by other website.
9. If you are testing an online shopping portal, Check if reaching to your final order summary page deletes the cookie of previous page of shopping cart properly and no invalid action or purchase got executed from same logged in user.
10. Check if the application under test is writing the cookies properly on different browsers as intended and site works properly using these cookies. This test can be done on browsers like different versions of internet explorer, Mozilla Firefox, Netscape, Opera etc.
11. If the application under test is using cookies to maintain the logging state for users. Check if some id is being displayed in the address bar. Now, change the id & press enter. It should display an access denied message and and you should not be able to see other user's account.

Performance Testing vs Load Testing

Performance Testing vs Load Testing

Performance testing - It is performed to evaluate the performance of components of a particular system in a specific situation. It very wide term. It includes: Load Testing, Stress Testing, capacity testing, volume testing, endurance testing, spike testing, scalability testing and reliability testing etc. This type of testing generally does not give pass or fail. It is basically done to set the benchmark & standard of the application against Concurrency / Throughput, Server response time, Latency, Render response time etc. In other words, you can say it is technical & formal evaluation for responsiveness, speed, scalability and stability characteristics.
Load Testing is subset of performance testing. It is done by constantly increasing the load on the application under test till the time it reaches the threshold limit. The main goal of load testing is to identify the upper limit of the system in terms of database, hardware and network etc. The common goal of doing the load testing is to set the SLAs for the application. Example of load testing can be:

Running multiple applications on a computer simultaneously - starting with one application, then start second application, then third and so on....Now see the performance of your computer.

Endurance test is also a part of load testing which used to calculate metrics like Mean Time Between Failure and Mean Time to Failure.
Load Testing helps to determine:

• Throughput
• Peak Production Load
• Adequacy of H/W environment
• Load balancing requirements
• How many users application can handle with optimal performance results
• How many users hardware can handle with optimal performance results

Stress testing - It is done to evaluate the application's behaviour beyond normal or peak load conditions. It is basically testing the functionality of the application under high loads. Normally these are related to synchronization issues, memory leaks or race conditions etc. Some testing experts also call it as fatigue testing. Sometimes, it becomes difficult to set up a controlled environment before running the test. Example of Stress testing is:

A banking application can take a maximum user load of 20000 concurrent users. Increase the load to 21000 and do some transaction like deposit or withdraw. As soon as you did the transaction, banking application server database will sync with ATM database server. Now check with the user load of 21000 does this sync happened successfully. Now repeat the same test with 22000 thousand concurrent users and so on.

Spike test is also a part of stress testing which is performed when application is loaded with heavy loads repeatedly and increase beyond production operations for short duration.
Stress Testing helps to determine:

• Errors in slowness & at peak user loads
• Any security loop holes with over loads
• How the hardware reacts with over loads
• Data corruption issues at over loads

Priortize bugs

Priortize bugs

You must have seen developers & testers fighting with each others on priority of bugs. Testers log the bug as high priority or critical and sometimes, developers surprised to see the priority. Most of the times, both developer & tester will not agree on the priority set by each other. I've seen sometimes, even product manager / project manager / business analyst are not able to prioritize the defects or bugs EFFECTIVELY. Even in many process oriented organizations, it becomes a challenge.
Since, these bugs are risk to the application and project delivery, so, let's apply basic principle of risk management here. i.e. Risk Value = Probability x Impact

In case of bugs, Priority = Severity Value x Priority Value

Now, you can use this priority number to actually prioritize your bugs and fix them in the required order. Lets see the entire process in the below example:
1. Dear Testers - Please do not set priority of the bug. Do it only if you are interacting with the customer and thoroughly understand the business flow implemented in the application. Priority should be set based on the criticality of the business / functional requirement. Also, set severity only after doing a through impact analysis. Best is if we can have bug triage meeting at this stage, so that everyone can provide their inputs to identify the correct severity level.
2. Set four severity levels: High, Medium, Low, Lowest. Set their values as 4, 3, 2, 1. 4 as highest and 1 as lowest.

Severity	Severity Value	Example
High	4	Data loss, missing feature, security violation
Medium	3	Feature is missing but workaround is there, if high profiles users like management will get an impact, then it would be of high severity 4
Low	2	It is impacting minor things, but, user is able to do the work
Lowest	1	A cosmetic error, spelling mistake, typo etc..

3. Set four priority levels: Critical, High, Medium, Low. Set them values as 4, 3, 2, 1. 4 as Critical and 1 as low. Please ask the business analyst to set the priority.

Priority	Priority Value	Example
Critical	4	Application is getting started, Critical business flow got stuck, Data loss, business loss, loss of end user, a security violation, system is very unstable
High	3	Feature is missing which is not critical for only very few users that do not have much impact on business, if high profiles users like management will get an impact, then it would be of high severity 4
Medium	2	Feature is missing but workaround is there. If users from management group are getting impacted, then it can be of priority 3 or 4. It can also be some feature of the functionality is not working
Lowest	1	A cosmetic error, spelling mistake, typo. Here, one important point to note is that even a spelling mistake or typo can also be of high priority or critical. e.g. If there is a spelling mistake of name of a person from senior management, then it may have high priority.

Priority can be calculated as

Bug Id	Bug Description	Severity Value	Priority Value	Priority
Bug_1	Bug Description 1	2	4	8
Bug_2	Bug Description 2	3	2	6
Bug_3	Bug Description 3	1	1	1
Bug_4	Bug Description 4	4	3	12
Bug_5	Bug Description 5	2	4	8

Now, the bug with highest priority should be fixed first. In this case bug “Bug_4” should be fixed first. Then Bug_1 and Bug_5 should be fixed and so on.

This way we will have a number defining the priority of the based on the two logical facts.

You do not need to follow this practice all the time. After development, initial during testing/bug fixing phase, you may skip this process. Eventually, at that time, we have to fix all the bugs.
When you see there is a production release in next 2-4 days or more or less depending upon the size of release and there are lot of open bugs, you may prioritize bugs based on this approach. Best is do this exercise 5-7 days before the final release. Also, I would suggest this kind of exercise for all customer reported bugs during UAT or post production release.
As a pilot, I've applied this technique to one of my project. Let's see how practical is this. Please share your views as well on this approach.

Unit Testing and best practices in Unit testing

Unit Testing and best practices in Unit testing

Unit testing is a software development process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation. The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as expected. Each unit is tested separately before integrating them into modules to test the interfaces between modules.By means of effective Unit testing large percentage of defects are identified. Unit testing is performed by developers.

Each module that is developed by designers need to be tested individually to verify proper operation so that any faulty module can be fixed immediately rather than let it exist and then cause some major issue in the integration phase. Once all of the units in a program have been found to be working efficiently and without any bugs, larger components of the program can be evaluated by means of integration testing. Though Unit testing may be time consuming, tedious and requires thoroughness on the part of the development team, but in the long run it can avoid major pitfalls in the software.

Benefits of unit testing:

• The modular approach during Unit testing eliminates the dependency on other modules during testing. 
• We can test parts of a project with out waiting for the other parts to be available.
• Designers can identify and fix problem immediately, as the modules are best known to them. This helps in fixing multiple problems simultaneously.
• Cost of fixing a defect identified during the early stages is less compared to that during later stage.
• Debugging is simplified. 
• Structural coverage of code is higher.
• Unit testing is more cost effective compared to the other stages of testing

Misconceptions about Unit Testing:

• Integration Tests will Catch all the Bugs Anyway: This is one of the common misconceptions of designers. Complexity of the issue rises while it passes through various testing cycles and then when the bug is raised during the later stages, the resolution time will be high as the scope of the bug widens. It is better to weed off the crop before it poisons the whole farm.
• Programmers Dilemma: Most designers believe Unit testing is not actually required as it is time consuming. They feel they are too good programmers and their software doesn’t need Unit tests. But in the real world, everyone makes mistakes. Real software systems are much more complex. Software behavior various in different environment and with different scenarios. Coding is not a one pass process. Enhancements to the code can be made only when we know the existing module is functioning as expected.
• It Consumes Too Much Time: Developers are often in a hurry to complete their code and integrate it. Unit Testing is most often considered a useless activity, as they feel anyways the code will be tested by QA. There is no point in having a system which works but not exactly as it is supposed to function and is  to be full of bugs. Practically, such an approach to development will often result in software which will not even run. The net result is that a lot of time will be spent tracking down relatively simple bugs which are wholly contained within particular units. Individually, such bugs may be trivial, but collectively they result in an excessive period of time integrating the software to produce a system which is unlikely to be reliable. This can also lead to failure to meet the required deadlines.

Unit Testing Best Practices:

• Ensure each Unit Test case is independent of each other. As the software is prone to changes during the Unit Testing due to enhancements/changes to the requirements. Hence any given behavior should be specified in one and only one test. Otherwise if you later change that behavior, you’ll have to change multiple tests.
• Test only one code at a time. It is always recommended to test each of the modules independently and not while all are chained together. Otherwise you will have lots of overlap between tests and changes to one unit may effect all other modules and cause the software to fail.
• Name your unit tests clearly and consistently. Ensure that your test cases are easily readable so that anyone picking up Unit test cases can execute them without any issues. Ensure the test case nomenclature is consistent throughout.
• Before changing a module interface or implementation, make sure that the module has test cases and that it passes its tests before changing the implementation. This way you can know that your changes didn't break anything.
• Always ensure the bug identified during Unit Testing is fixed before moving it to the next phase.

Unit Testing  Techniques: Structural, Functional & Error based Techniques

Structural Techniques: It is a White box testing technique that uses an internal perspective of the system to design test cases based on internal structure. It requires programming skills to identify all paths through the software. The tester chooses test case inputs to exercise paths through the code and determines the appropriate outputs. Major Structural techniques are:

• Statement Testing: A test strategy in which each statement of a program is executed at least once. 
• Branch Testing: Testing in which all branches in the program source code are tested at least once.
• Path Testing: Testing in which all paths in the program source code are tested at least once.
• Condition Testing: Condition testing allows the programmer to determine the path through a program by selectively executing code based on the comparison of a value  
• Expression Testing: Testing in which the application is tested for different values of Regular Expression.

Functional testing techniques: These are Black box testing techniques which tests the functionality of the application. Some functionality testing techniques are:

• Input domain testing: This testing technique concentrates on size and type of every input object in terms of boundary value analysis and Equivalence class.  
• Boundary Value: Boundary value analysis is a  software testing design technique in which tests are designed to include representatives of boundary values.  
• Syntax checking: This is a technique which is used to check the Syntax of the application. 
• Equivalence Partitioning: This is a software testing technique that divides the input data of a software unit into partition of data from which test cases can be derived

Error based Techniques: The best person to know the defects in his code is the person who has designed it.
Few of the Error based techniques are:

• Fault seeding techniques can be used so that known defects can be put into the code and tested until they are all found.
• Mutation Testing: This is done by mutating certain statements in your source code and checking if your test code is able to find the errors. Mutation testing is very expensive to run, especially on very large applications.
• Historical Test data: This technique calculates the priority of each test case using historical information from the previous executions of the test case. 

Careful approach to Unit testing helps detecting many bugs at a stage of the software development where they can be corrected economically. It is a tedious process when bugs are detected and corrected at later stages of software development as fixing the bugs is difficult, time consuming and costly. Efficiency and quality are best served by testing software as early in the life cycle. Whenever any changes are made to the software we need to ensure regression testing is performed. Testing strategies like thorough unit testing, good management of the testing process, and appropriate use of tools helps in maximizing the effectiveness of testing effort. Effective unit testing is all part of developing a very high quality software product which can benefit the organization on a whole.

Smoke Testing vs Sanity Testing

 Smoke Testing vs Sanity Testing

Smoke Testing: Software Testing done to ensure that whether the build can be accepted for through software testing or not. Basically, it is done to check the stability of the build received for software testing.

Sanity testing: After receiving a build with minor changes in the code or functionality, a subset of regression test cases are executed that to check whether it rectified the software bugs or issues and no other software bug is introduced by the changes. Sometimes, when multiple cycles of regression testing are executed, sanity testing of the software can be done at later cycles after through regression test cycles. If we are moving a build from staging / testing server to production server, sanity testing of the software application can be done to check that whether the build is sane enough to move to further at production server or not.

Difference between Smoke & Sanity Software Testing:

• Smoke testing is a wide approach where all areas of the software application are tested without getting into too deep. However, a sanity software testing is a narrow regression testing with a focus on one or a small set of areas of functionality of the software application.
• The test cases for smoke testing of the software can be either manual or automated. However, a sanity test is generally without test scripts or test cases.
• Smoke testing is done to ensure whether the main functions of the software application are working or not. During smoke testing of the software, we do not go into finer details. However, sanity testing is a cursory software testing type. It is done whenever a quick round of software testing can prove that the software application is functioning according to business / functional requirements.
• Smoke testing of the software application is done to check whether the build can be accepted for through software testing. Sanity testing of the software is to ensure whether the requirements are met or not.

Difference between test case and test scenario

Test Scenario : A set of test cases that ensure that the business process flows are tested from end to end. They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one.

Test Case : Is a document which describes the input, Action/Event and expected response to determine whether the particular functionality of the application is working fine.

Functional Testing vs Non Functional Testing

S. No	Functional Testing	Non-Functional Testing
1	Testing developed application against business requirements. Functional testing is done using the functional specifications provided by the client or by using the design specifications like use cases provided by the design team.	Testing the application based on the clients and performance requirement. Non-Functioning testing is done based on the requirements and test scenarios defined by the client.
2	Functional testing covers · Unit Testing · Smoke testing / Sanity testing · Integration Testing (Top Down, Bottom up Testing) · Interface & Usability Testing · System Testing · Regression Testing · Pre User Acceptance Testing (Alpha & Beta) · User Acceptance Testing · White Box & Black Box Testing· Globalization & Localization Testing	Non-Functional testing covers · Load and Performance Testing · Ergonomics Testing · Stress & Volume Testing · Compatibility & Migration Testing · Data Conversion Testing · Security / Penetration Testing · Operational Readiness Testing · Installation Testing · Security Testing (Application Security, Network Security, System Security)