Why Automation testing?1) You have some new releases and bug fixes in working module. So how will you ensure that the new bug fixes have not introduced any new bug in previous working functionality? You need to test the previous functionality also. So will you test manually all the module functionality every time you have some bug fixes or new functionality addition? Well you might do it manually but then you are not doing testing effectively. Effective in terms of company cost, resources, Time etc. Here comes need of Automation.- So automate your testing procedure when you have lot of regression work.
2) You are testing a web application where there might be thousands of users interacting with your application simultaneously. How will you test such a web application? How will you create those many users manually and simultaneously? Well very difficult task if done manually.- Automate your load testing work for creating virtual users to check load capacity of your application.
3) You are testing application where code is changing frequently. You have almost same GUI but functional changes are more so testing rework is more.- Automate your testing work when your GUI is almost frozen but you have lot of frequently functional changes.
What are the Risks associated in Automation Testing?There are some distinct situations where you can think of automating your testing work. I have covered some risks of automation testing here. If you have taken decision of automation or are going to take sooner then think of following scenarios first.
1) Do you have skilled resources? For automation you need to have persons having some programming knowledge. Think of your resources. Do they have sufficient programming knowledge for automation testing? If not do they have technical capabilities or programming background that they can easily adapt to the new technologies? Are you going to invest money to build a good automation team? If your answer is yes then only think to automate your work.
2) Initial cost for Automation is very high:I agree that manual testing has too much cost associated to hire skilled manual testers. And if you are thinking automation will be the solution for you, Think twice. Automation cost is too high for initial setup i.e. cost associated to automation tool purchase, training and maintenance of test scripts is very high.There are many unsatisfied customers regretting on their decision to automate their work. If you are spending too much and getting merely some good looking testing tools and some basic automation scripts then what is the use of automation?
3) Do not think to automate your UI if it is not fixed:Beware before automating user interface. If user interface is changing extensively, cost associated with script maintenance will be very high. Basic UI automation is sufficient in such cases.
4) Is your application is stable enough to automate further testing work?It would be bad idea to automate testing work in early development cycle (Unless it is agile environment). Script maintenance cost will be very high in such cases.
5) Are you thinking of 100% automation?Please stop dreaming. You cannot 100% automate your testing work. Certainly you have areas like performance testing, regression testing, load/stress testing where you can have chance of reaching near to 100% automation. Areas like User interface, documentation, installation, compatibility and recovery where testing must be done manually.
6) Do not automate tests that run once:Identify application areas and test cases that might be running once and not included in regression. Avoid automating such modules or test cases.
7) Will your automation suite be having long lifetime?Every automation script suite should have enough life time that its building cost should be definitely less than that of manual execution cost. This is bit difficult to analyze the effective cost of each automation script suite. Approximately your automation suite should be used or run at least 15 to 20 times for separate builds (General assumption. depends on specific application complexity) to have good ROI.
Here is the conclusion:
Automation testing is the best way to accomplish most of the testing goals and effective use of resources and time. But you should be cautious before choosing the automation tool. Be sure to have skilled staff before deciding to automate your testing work. Otherwise your tool will remain on the shelf giving you no ROI. Handing over the expensive automation tools to unskilled staff will lead to frustration. Before purchasing the automation tools make sure that tool is a best fit to your requirements. You cannot have the tool that will 100% match with your requirements. So find out the limitations of the tool that is best match with your requirements and then use manual testing techniques to overcome those testing tool limitations. Open source tool is also a good option to start with automation. To know more on choosing automation tools read my previous posts here and here.
Instead of relying 100% on either manual or automation use the best combination of manual and automation testing. This is the best solution (I think) for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.
Monday, March 24, 2008
Reader Tips
James Bach - The consulting software Tester
Michael Bolton - Blog on Software Testing and Quality observations
Matt Heusser - Thoughts on life, Software development and Testing
Shrini Kulkarni - A Thinking Tester
Pradeep Soundarajan - Critical thoughts on Software Testing
Erkan Yilmaz - Good stories worth to read
Jeff Fry - Thoughts on the craft of software testing
Jonathan Kohl - Software Investigator
Cem Kaner - Blog on craft and community of software testing
Elisabeth Hendrickson - Agile Testing Guru
Debasis Pradhan - The bug hunter
Rahul Verma - Testing Perspective
Michael Hunter - The guy making developers cry since 1995
Scott Barber Better Testing, Better results
Michael Bolton - Blog on Software Testing and Quality observations
Matt Heusser - Thoughts on life, Software development and Testing
Shrini Kulkarni - A Thinking Tester
Pradeep Soundarajan - Critical thoughts on Software Testing
Erkan Yilmaz - Good stories worth to read
Jeff Fry - Thoughts on the craft of software testing
Jonathan Kohl - Software Investigator
Cem Kaner - Blog on craft and community of software testing
Elisabeth Hendrickson - Agile Testing Guru
Debasis Pradhan - The bug hunter
Rahul Verma - Testing Perspective
Michael Hunter - The guy making developers cry since 1995
Scott Barber Better Testing, Better results
Web Testing: Complete guide on testing web applications
In my previous post I have outlined points to be considered while testing web applications. Here we will see some more details on web application testing with web testing test cases. Let me tell you one thing that I always like to share practical knowledge, which can be useful to users in their career life. This is a quite long article so sit back and get relaxed to get most out of it.
Let’s have first web testing checklist.1) Functionality Testing2) Usability testing3) Interface testing4) Compatibility testing5) Performance testing6) Security testing
1) Functionality Testing:
Test for - all the links in web pages, database connection, forms used in the web pages for submitting or getting information from user, Cookie testing.
Check all the links:
Test the outgoing links from all the pages from specific domain under test.
Test all internal links.
Test links jumping on the same pages.
Test links used to send the email to admin or other users from web pages.
Test to check if there are any orphan pages.
Lastly in link checking, check for broken links in all above-mentioned links.
Test forms in all pages:Forms are the integral part of any web site. Forms are used to get information from users and to keep interaction with them. So what should be checked on these forms?
First check all the validations on each field.
Check for the default values of fields.
Wrong inputs to the fields in the forms.
Options to create forms if any, form delete, view or modify the forms.
Let’s take example of the search engine project currently I am working on, In this project we have advertiser and affiliate signup steps. Each sign up step is different but dependent on other steps. So sign up flow should get executed correctly. There are different field validations like email Ids, User financial info validations. All these validations should get checked in manual or automated web testing.
Cookies testing:Cookies are small files stored on user machine. These are basically used to maintain the session mainly login sessions. Test the application by enabling or disabling the cookies in your browser options. Test if the cookies are encrypted before writing to user machine. If you are testing the session cookies (i.e. cookies expire after the sessions ends) check for login sessions and user stats after session end. Check effect on application security by deleting the cookies. (I will soon write separate article on cookie testing)
Validate your HTML/CSS:If you are optimizing your site for Search engines then HTML/CSS validation is very important. Mainly validate the site for HTML syntax errors. Check if site is crawlable to different search engines.
Database testing:Data consistency is very important in web application. Check for data integrity and errors while you edit, delete, modify the forms or do any DB related functionality.Check if all the database queries are executing correctly, data is retrieved correctly and also updated correctly. More on database testing could be load on DB, we will address this in web load or performance testing below.
2) Usability Testing:
Test for navigation:Navigation means how the user surfs the web pages, different controls like buttons, boxes or how user using the links on the pages to surf different pages.Usability testing includes:Web site should be easy to use. Instructions should be provided clearly. Check if the provided instructions are correct means whether they satisfy purpose.Main menu should be provided on each page. It should be consistent.
Content checking: Content should be logical and easy to understand. Check for spelling errors. Use of dark colors annoys users and should not be used in site theme. You can follow some standards that are used for web page and content building. These are common accepted standards like as I mentioned above about annoying colors, fonts, frames etc.Content should be meaningful. All the anchor text links should be working properly. Images should be placed properly with proper sizes.These are some basic standards that should be followed in web development. Your task is to validate all for UI testing
Other user information for user help:Like search option, sitemap, help files etc. Sitemap should be present with all the links in web sites with proper tree view of navigation. Check for all links on the sitemap.“Search in the site” option will help users to find content pages they are looking for easily and quickly. These are all optional items and if present should be validated.
3) Interface Testing:The main interfaces are:Web server and application server interfaceApplication server and Database server interface.
Check if all the interactions between these servers are executed properly. Errors are handled properly. If database or web server returns any error message for any query by application server then application server should catch and display these error messages appropriately to users. Check what happens if user interrupts any transaction in-between? Check what happens if connection to web server is reset in between?
4) Compatibility Testing:Compatibility of your web site is very important testing aspect. See which compatibility test to be executed:
Browser compatibility
Operating system compatibility
Mobile browsing
Printing options
Browser compatibility:In my web-testing career I have experienced this as most influencing part on web site testing.Some applications are very dependent on browsers. Different browsers have different configurations and settings that your web page should be compatible with. Your web site coding should be cross browser platform compatible. If you are using java scripts or AJAX calls for UI functionality, performing security checks or validations then give more stress on browser compatibility testing of your web application.Test web application on different browsers like Internet explorer, Firefox, Netscape navigator, AOL, Safari, Opera browsers with different versions.
OS compatibility:Some functionality in your web application is may not be compatible with all operating systems. All new technologies used in web development like graphics designs, interface calls like different API’s may not be available in all Operating Systems.Test your web application on different operating systems like Windows, Unix, MAC, Linux, Solaris with different OS flavors.
Mobile browsing:This is new technology age. So in future Mobile browsing will rock. Test your web pages on mobile browsers. Compatibility issues may be there on mobile.
Printing options:If you are giving page-printing options then make sure fonts, page alignment, page graphics getting printed properly. Pages should be fit to paper size or as per the size mentioned in printing option.
5) Performance testing:Web application should sustain to heavy load. Web performance testing should include:Web Load TestingWeb Stress Testing
Test application performance on different internet connection speed.In web load testing test if many users are accessing or requesting the same page. Can system sustain in peak load times? Site should handle many simultaneous user requests, large input data from users, Simultaneous connection to DB, heavy load on specific pages etc.
Stress testing: Generally stress means stretching the system beyond its specification limits. Web stress testing is performed to break the site by giving stress and checked how system reacts to stress and how system recovers from crashes.Stress is generally given on input fields, login and sign up areas.
In web performance testing web site functionality on different operating systems, different hardware platforms is checked for software, hardware memory leakage errors,
6) Security Testing:
Following are some test cases for web security testing:
Test by pasting internal url directly into browser address bar without login. Internal pages should not open.
If you are logged in using username and password and browsing internal pages then try changing url options directly. I.e. If you are checking some publisher site statistics with publisher site ID= 123. Try directly changing the url site ID parameter to different site ID which is not related to logged in user. Access should denied for this user to view others stats.
Try some invalid inputs in input fields like login username, password, input text boxes. Check the system reaction on all invalid inputs.
Web directories or files should not be accessible directly unless given download option.
Test the CAPTCHA for automates scripts logins.
Test if SSL is used for security measures. If used proper message should get displayed when user switch from non-secure http:// pages to secure https:// pages and vice versa.
All transactions, error messages, security breach attempts should get logged in log files somewhere on web server.
Let’s have first web testing checklist.1) Functionality Testing2) Usability testing3) Interface testing4) Compatibility testing5) Performance testing6) Security testing
1) Functionality Testing:
Test for - all the links in web pages, database connection, forms used in the web pages for submitting or getting information from user, Cookie testing.
Check all the links:
Test the outgoing links from all the pages from specific domain under test.
Test all internal links.
Test links jumping on the same pages.
Test links used to send the email to admin or other users from web pages.
Test to check if there are any orphan pages.
Lastly in link checking, check for broken links in all above-mentioned links.
Test forms in all pages:Forms are the integral part of any web site. Forms are used to get information from users and to keep interaction with them. So what should be checked on these forms?
First check all the validations on each field.
Check for the default values of fields.
Wrong inputs to the fields in the forms.
Options to create forms if any, form delete, view or modify the forms.
Let’s take example of the search engine project currently I am working on, In this project we have advertiser and affiliate signup steps. Each sign up step is different but dependent on other steps. So sign up flow should get executed correctly. There are different field validations like email Ids, User financial info validations. All these validations should get checked in manual or automated web testing.
Cookies testing:Cookies are small files stored on user machine. These are basically used to maintain the session mainly login sessions. Test the application by enabling or disabling the cookies in your browser options. Test if the cookies are encrypted before writing to user machine. If you are testing the session cookies (i.e. cookies expire after the sessions ends) check for login sessions and user stats after session end. Check effect on application security by deleting the cookies. (I will soon write separate article on cookie testing)
Validate your HTML/CSS:If you are optimizing your site for Search engines then HTML/CSS validation is very important. Mainly validate the site for HTML syntax errors. Check if site is crawlable to different search engines.
Database testing:Data consistency is very important in web application. Check for data integrity and errors while you edit, delete, modify the forms or do any DB related functionality.Check if all the database queries are executing correctly, data is retrieved correctly and also updated correctly. More on database testing could be load on DB, we will address this in web load or performance testing below.
2) Usability Testing:
Test for navigation:Navigation means how the user surfs the web pages, different controls like buttons, boxes or how user using the links on the pages to surf different pages.Usability testing includes:Web site should be easy to use. Instructions should be provided clearly. Check if the provided instructions are correct means whether they satisfy purpose.Main menu should be provided on each page. It should be consistent.
Content checking: Content should be logical and easy to understand. Check for spelling errors. Use of dark colors annoys users and should not be used in site theme. You can follow some standards that are used for web page and content building. These are common accepted standards like as I mentioned above about annoying colors, fonts, frames etc.Content should be meaningful. All the anchor text links should be working properly. Images should be placed properly with proper sizes.These are some basic standards that should be followed in web development. Your task is to validate all for UI testing
Other user information for user help:Like search option, sitemap, help files etc. Sitemap should be present with all the links in web sites with proper tree view of navigation. Check for all links on the sitemap.“Search in the site” option will help users to find content pages they are looking for easily and quickly. These are all optional items and if present should be validated.
3) Interface Testing:The main interfaces are:Web server and application server interfaceApplication server and Database server interface.
Check if all the interactions between these servers are executed properly. Errors are handled properly. If database or web server returns any error message for any query by application server then application server should catch and display these error messages appropriately to users. Check what happens if user interrupts any transaction in-between? Check what happens if connection to web server is reset in between?
4) Compatibility Testing:Compatibility of your web site is very important testing aspect. See which compatibility test to be executed:
Browser compatibility
Operating system compatibility
Mobile browsing
Printing options
Browser compatibility:In my web-testing career I have experienced this as most influencing part on web site testing.Some applications are very dependent on browsers. Different browsers have different configurations and settings that your web page should be compatible with. Your web site coding should be cross browser platform compatible. If you are using java scripts or AJAX calls for UI functionality, performing security checks or validations then give more stress on browser compatibility testing of your web application.Test web application on different browsers like Internet explorer, Firefox, Netscape navigator, AOL, Safari, Opera browsers with different versions.
OS compatibility:Some functionality in your web application is may not be compatible with all operating systems. All new technologies used in web development like graphics designs, interface calls like different API’s may not be available in all Operating Systems.Test your web application on different operating systems like Windows, Unix, MAC, Linux, Solaris with different OS flavors.
Mobile browsing:This is new technology age. So in future Mobile browsing will rock. Test your web pages on mobile browsers. Compatibility issues may be there on mobile.
Printing options:If you are giving page-printing options then make sure fonts, page alignment, page graphics getting printed properly. Pages should be fit to paper size or as per the size mentioned in printing option.
5) Performance testing:Web application should sustain to heavy load. Web performance testing should include:Web Load TestingWeb Stress Testing
Test application performance on different internet connection speed.In web load testing test if many users are accessing or requesting the same page. Can system sustain in peak load times? Site should handle many simultaneous user requests, large input data from users, Simultaneous connection to DB, heavy load on specific pages etc.
Stress testing: Generally stress means stretching the system beyond its specification limits. Web stress testing is performed to break the site by giving stress and checked how system reacts to stress and how system recovers from crashes.Stress is generally given on input fields, login and sign up areas.
In web performance testing web site functionality on different operating systems, different hardware platforms is checked for software, hardware memory leakage errors,
6) Security Testing:
Following are some test cases for web security testing:
Test by pasting internal url directly into browser address bar without login. Internal pages should not open.
If you are logged in using username and password and browsing internal pages then try changing url options directly. I.e. If you are checking some publisher site statistics with publisher site ID= 123. Try directly changing the url site ID parameter to different site ID which is not related to logged in user. Access should denied for this user to view others stats.
Try some invalid inputs in input fields like login username, password, input text boxes. Check the system reaction on all invalid inputs.
Web directories or files should not be accessible directly unless given download option.
Test the CAPTCHA for automates scripts logins.
Test if SSL is used for security measures. If used proper message should get displayed when user switch from non-secure http:// pages to secure https:// pages and vice versa.
All transactions, error messages, security breach attempts should get logged in log files somewhere on web server.
Winrunner questions from Interviewer point of view
Explain WinRunner testing process?WinRunner testing process involves six main stagesi. Create GUI Map File so that WinRunner can recognize the GUI objects in the application being testedii. Create test scripts by recording, programming, or a combination of both. While recording tests, insert checkpoints where you want to check the response of the application being tested.iii. Debug Test: run tests in Debug mode to make sure they run smoothlyiv. Run Tests: run tests in Verify mode to test your application.v. View Results: determines the success or failure of the tests.vi. Report Defects: If a test run fails due to a defect in the application being tested, you can report information about the defect directly from the Test Results window.
What is contained in the GUI map?- WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
- There are 2 types of GUI Map files.i. Global GUI Map file: a single GUI Map file for the entire applicationii. GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
How does WinRunner recognize objects on the application?WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.
Have you created test scripts and what is contained in the test scripts?Yes I have created test scripts. It contains the statement in Mercury Interactive’s Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using WinRunner’s visual programming tool, the Function Generator.
How does WinRunner evaluates test results?Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.
Have you performed debugging of the scripts?Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.
How do you run your test scripts?We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results. 1. Where did you learn WinRunner and TestDirector?
If they say it was in a mercury class, ask if they can show you theircertificate of completion. If they say no, let them know you will verify itwith Mercury.
2. Have you ever created a start-up script?
If they answer “yes”, ask themwhat was in it and how they got WR to execute the start-up script when WR isinvoked.
They should answer something like this, “In the start-up script, we loaded allthe gui maps for the application, any library files we needed, and any customgui object mapping we have to do. We also loaded global variables and systemvariables here. The startup script location is added to the WR .ini file,wrun.ini located under the windows or winnt directory as wrun.ini”
3. What is the difference between writing a function and writing a script?
They should mention some of these things:1. A function goes into a “compiled module”, a script does not.2. A function follows strict “c” syntax. For instance, you have to declareall variables created and used in the function. In a script you do not.
4. What is the difference between “set_window” and “win_activate”.?When would you use “set_window” and when would you use “win_activate”?win_activate has the format win_activate(window);. The win_activate functionmakes the specified window the active window by bringing it into focus andraising it to the top of the display. (It is the equivalent to clicking on thewindow banner)
Set_window has the following format: set_window(window,[time]);The set_window function directs input to the correct application window. Thisdirects the GUI map to this window. It also sets the scope for objectidentification in the GUI map.
The most important difference is that set_window has a timing option.WinRunner will wait a maximum of the number used in the function, PLUS thesystem set timeout, to wait for the window to appear. Win_activate assumes thewindow is already on the desktop and has no timing option.
What is contained in the GUI map?- WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
- There are 2 types of GUI Map files.i. Global GUI Map file: a single GUI Map file for the entire applicationii. GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
How does WinRunner recognize objects on the application?WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.
Have you created test scripts and what is contained in the test scripts?Yes I have created test scripts. It contains the statement in Mercury Interactive’s Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using WinRunner’s visual programming tool, the Function Generator.
How does WinRunner evaluates test results?Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.
Have you performed debugging of the scripts?Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.
How do you run your test scripts?We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results. 1. Where did you learn WinRunner and TestDirector?
If they say it was in a mercury class, ask if they can show you theircertificate of completion. If they say no, let them know you will verify itwith Mercury.
2. Have you ever created a start-up script?
If they answer “yes”, ask themwhat was in it and how they got WR to execute the start-up script when WR isinvoked.
They should answer something like this, “In the start-up script, we loaded allthe gui maps for the application, any library files we needed, and any customgui object mapping we have to do. We also loaded global variables and systemvariables here. The startup script location is added to the WR .ini file,wrun.ini located under the windows or winnt directory as wrun.ini”
3. What is the difference between writing a function and writing a script?
They should mention some of these things:1. A function goes into a “compiled module”, a script does not.2. A function follows strict “c” syntax. For instance, you have to declareall variables created and used in the function. In a script you do not.
4. What is the difference between “set_window” and “win_activate”.?When would you use “set_window” and when would you use “win_activate”?win_activate has the format win_activate(window);. The win_activate functionmakes the specified window the active window by bringing it into focus andraising it to the top of the display. (It is the equivalent to clicking on thewindow banner)
Set_window has the following format: set_window(window,[time]);The set_window function directs input to the correct application window. Thisdirects the GUI map to this window. It also sets the scope for objectidentification in the GUI map.
The most important difference is that set_window has a timing option.WinRunner will wait a maximum of the number used in the function, PLUS thesystem set timeout, to wait for the window to appear. Win_activate assumes thewindow is already on the desktop and has no timing option.
Regression Testing with Regression Testing Tools and methods
What is Regression Software Testing?Regression means retesting the unchanged parts of the application. Test cases are re-executed in order to check whether previous functionality of application is working fine and new changes have not introduced any new bugs.
This is the method of verification. Verifying that the bugs are fixed and the newly added feature have not created in problem in previous working version of software.
Why regression Testing?Regression testing is initiated when programmer fix any bug or add new code for new functionality to the system. It is a quality measure to check that new code complies with old code and unmodified code is not getting affected.Most of the time testing team has task to check the last minute changes in the system. In such situation testing only affected application area in necessary to complete the testing process in time with covering all major system aspects.
How much regression testing?This depends on the scope of new added feature. If the scope of the fix or feature is large then the application area getting affected is quite large and testing should be thoroughly including all the application test cases. But this can be effectively decided when tester gets input from developer about the scope, nature and amount of change.
What we do in regression testing?
Rerunning the previously conducted tests
Comparing current results with previously executed test results.
Regression Testing Tools:Automated Regression testing is the testing area where we can automate most of the testing efforts. We run all the previously executed test cases this means we have test case set available and running these test cases manually is time consuming. We know the expected results so automating these test cases is time saving and efficient regression testing method. Extent of automation depends on the number of test cases that are going to remain applicable over the time. If test cases are varying time to time as application scope goes on increasing then automation of regression procedure will be the waste of time.
Most of the regression testing tools are record and playback type. Means you will record the test cases by navigating through the AUT and verify whether expected results are coming or not.Example regression testing tools are:
Winrunner
QTP
AdventNet QEngine
Regression Tester
vTest
Watir
Selenium
actiWate
Rational Functional Tester
SilkTest
Most of the tools are both Functional as well as regression testing tools.
Regression Testing Of GUI application:It is difficult to perform GUI(Graphical User Interface) regression testing when GUI structure is modified. The test cases written on old GUI either becomes obsolete or need to reuse. Reusing the regression testing test cases means GUI test cases are modified according to new GUI. But this task becomes cumbersome if you have large set of GUI test cases.
This is the method of verification. Verifying that the bugs are fixed and the newly added feature have not created in problem in previous working version of software.
Why regression Testing?Regression testing is initiated when programmer fix any bug or add new code for new functionality to the system. It is a quality measure to check that new code complies with old code and unmodified code is not getting affected.Most of the time testing team has task to check the last minute changes in the system. In such situation testing only affected application area in necessary to complete the testing process in time with covering all major system aspects.
How much regression testing?This depends on the scope of new added feature. If the scope of the fix or feature is large then the application area getting affected is quite large and testing should be thoroughly including all the application test cases. But this can be effectively decided when tester gets input from developer about the scope, nature and amount of change.
What we do in regression testing?
Rerunning the previously conducted tests
Comparing current results with previously executed test results.
Regression Testing Tools:Automated Regression testing is the testing area where we can automate most of the testing efforts. We run all the previously executed test cases this means we have test case set available and running these test cases manually is time consuming. We know the expected results so automating these test cases is time saving and efficient regression testing method. Extent of automation depends on the number of test cases that are going to remain applicable over the time. If test cases are varying time to time as application scope goes on increasing then automation of regression procedure will be the waste of time.
Most of the regression testing tools are record and playback type. Means you will record the test cases by navigating through the AUT and verify whether expected results are coming or not.Example regression testing tools are:
Winrunner
QTP
AdventNet QEngine
Regression Tester
vTest
Watir
Selenium
actiWate
Rational Functional Tester
SilkTest
Most of the tools are both Functional as well as regression testing tools.
Regression Testing Of GUI application:It is difficult to perform GUI(Graphical User Interface) regression testing when GUI structure is modified. The test cases written on old GUI either becomes obsolete or need to reuse. Reusing the regression testing test cases means GUI test cases are modified according to new GUI. But this task becomes cumbersome if you have large set of GUI test cases.
How to write effective Test cases, procedures and definitions
Writing effective test cases is a skill and that can be achieved by some experience and in-depth study of the application on which test cases are being written.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
How to write effective Test cases, procedures and definitions
Writing effective test cases is a skill and that can be achieved by some experience and in-depth study of the application on which test cases are being written.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
How to get job in Software Testing quickly?
All these questions are similar and I want to give also similar answer for this. I have written post on choosing software testing as your career where you can analyze your abilities and know which are the most important skills required for software testing.
I will continue mentioning that “know your interest before going into any career field”. Just going to software testing career or any other hot career is wrong and which may result into loss of your job interest as well as your job.
Now you know your abilities, skills, interests right? and you have made decision to go for software testing career as it is your favorite career and you suit most in this career. So here is a guideline for how to get a good job in software testing field.
If you are a fresher and just passed out from your college or passing out in coming months then you need to prepare well for some software testing methodologies. Prepare all the manual testing concepts. If possible have some hands-on on some automation and bug tracking tools like winrunner and test director. It is always a good idea to join any software testing institute or class which will provide you a good start and direction of preparation. You can join any 4 months duration software testing course or can do diploma in software testing which is typically of 6 months to 1 year. Keep the preparation going on in your course duration. This will help you to start giving interviews right after getting over your course.
If you have some sort of previous IT experience and want to switch to software testing then it’s somewhat simple for you. Show your previous IT experience in your resume while applying for software testing jobs. If possible do some crash course to get idea of software testing concepts like I mentioned for freshers above. Keep in mind that you have some kind of IT experience so be prepared for some tough interview questions here.
As companies always prefer some kind of relevant experience for any software job, its better if you have relevant experience in software testing and QA. It may be any kind of software testing tools hands-on or some testing course from reputed institutes.
Please always keep in mind- Do not add fake experience of any kind. This can ruin your career forever. Wait and try for some more days to get the job by your abilities instead of getting into trap of fake experience.
Last important words, Software testing is not ‘anyone can do career!’ Remove this attitude from your mind if someone has told such kind of foolish thing to you. Testing requires in depth knowledge of SDLF, out of box thinking, analytical skill and some programming language skill apart from software testing basics.
I will continue mentioning that “know your interest before going into any career field”. Just going to software testing career or any other hot career is wrong and which may result into loss of your job interest as well as your job.
Now you know your abilities, skills, interests right? and you have made decision to go for software testing career as it is your favorite career and you suit most in this career. So here is a guideline for how to get a good job in software testing field.
If you are a fresher and just passed out from your college or passing out in coming months then you need to prepare well for some software testing methodologies. Prepare all the manual testing concepts. If possible have some hands-on on some automation and bug tracking tools like winrunner and test director. It is always a good idea to join any software testing institute or class which will provide you a good start and direction of preparation. You can join any 4 months duration software testing course or can do diploma in software testing which is typically of 6 months to 1 year. Keep the preparation going on in your course duration. This will help you to start giving interviews right after getting over your course.
If you have some sort of previous IT experience and want to switch to software testing then it’s somewhat simple for you. Show your previous IT experience in your resume while applying for software testing jobs. If possible do some crash course to get idea of software testing concepts like I mentioned for freshers above. Keep in mind that you have some kind of IT experience so be prepared for some tough interview questions here.
As companies always prefer some kind of relevant experience for any software job, its better if you have relevant experience in software testing and QA. It may be any kind of software testing tools hands-on or some testing course from reputed institutes.
Please always keep in mind- Do not add fake experience of any kind. This can ruin your career forever. Wait and try for some more days to get the job by your abilities instead of getting into trap of fake experience.
Last important words, Software testing is not ‘anyone can do career!’ Remove this attitude from your mind if someone has told such kind of foolish thing to you. Testing requires in depth knowledge of SDLF, out of box thinking, analytical skill and some programming language skill apart from software testing basics.
How to find a bug in application? Tips and Tricks
think finding a blocker bug like any system crash is often rewarding! No I don’t think like that. You should try to find out the bugs that are most difficult to find and those always misleads users.
Finding such a subtle bugs is most challenging work and it gives you satisfaction of your work. Also it should be rewarded by seniors. I will share my experience of one such subtle bug that was not only difficult to catch but was difficult to reproduce also.I was testing one module from my search engine project. I do most of the activities of this project manually as it is a bit complex to automate. That module consist of traffic and revenue stats of different affiliates and advertisers. So testing such a reports is always a difficult task. When I tested this report it was showing the data accurately processed for some time but when tried to test again after some time it was showing misleading results. It was strange and confusing to see the results.
There was a cron (cron is a automated script that runs after specified time or condition) to process the log files and update the database. Such multiple crons are running on log files and DB to synchronize the total data. There were two crons running on one table with some time intervals. There was a column in table that was getting overwritten by other cron making some data inconsistency. It took us long time to figure out the problem due to the vast DB processes and different crons.
My point is try to find out the hidden bugs in the system that might occur for special conditions and causes strong impact on the system. You can find such a bugs with some tips and tricks.
So what are those tips:
1) Understand the whole application or module in depth before starting the testing.
2) Prepare good test cases before start to testing. I mean give stress on the functional test cases which includes major risk of the application.
3) Create a sufficient test data before tests, this data set include the test case conditions and also the database records if you are going to test DB related application.
4) Perform repeated tests with different test environment.
5) Try to find out the result pattern and then compare your results with those patterns.
6) When you think that you have completed most of the test conditions and when you think you are tired somewhat then do some monkey testing.
7) Use your previous test data pattern to analyse the current set of tests.
Try some standard test cases for which you found the bugs in some different application. Like if you are testing input text box try inserting some html tags as the inputs and see the output on display page.
9) Last and the best trick is try very hard to find the bug As if you are testing only to break the application!
Finding such a subtle bugs is most challenging work and it gives you satisfaction of your work. Also it should be rewarded by seniors. I will share my experience of one such subtle bug that was not only difficult to catch but was difficult to reproduce also.I was testing one module from my search engine project. I do most of the activities of this project manually as it is a bit complex to automate. That module consist of traffic and revenue stats of different affiliates and advertisers. So testing such a reports is always a difficult task. When I tested this report it was showing the data accurately processed for some time but when tried to test again after some time it was showing misleading results. It was strange and confusing to see the results.
There was a cron (cron is a automated script that runs after specified time or condition) to process the log files and update the database. Such multiple crons are running on log files and DB to synchronize the total data. There were two crons running on one table with some time intervals. There was a column in table that was getting overwritten by other cron making some data inconsistency. It took us long time to figure out the problem due to the vast DB processes and different crons.
My point is try to find out the hidden bugs in the system that might occur for special conditions and causes strong impact on the system. You can find such a bugs with some tips and tricks.
So what are those tips:
1) Understand the whole application or module in depth before starting the testing.
2) Prepare good test cases before start to testing. I mean give stress on the functional test cases which includes major risk of the application.
3) Create a sufficient test data before tests, this data set include the test case conditions and also the database records if you are going to test DB related application.
4) Perform repeated tests with different test environment.
5) Try to find out the result pattern and then compare your results with those patterns.
6) When you think that you have completed most of the test conditions and when you think you are tired somewhat then do some monkey testing.
7) Use your previous test data pattern to analyse the current set of tests.
Try some standard test cases for which you found the bugs in some different application. Like if you are testing input text box try inserting some html tags as the inputs and see the output on display page.
9) Last and the best trick is try very hard to find the bug As if you are testing only to break the application!
Most frequently asked winrunner interview questions with answers
How you used WinRunner in your project?Yes, I have been using WinRunner for creating automated scripts for GUI, functional and regression testing of the AUT.
Explain WinRunner testing process?WinRunner testing process involves six main stagesi. Create GUI Map File so that WinRunner can recognize the GUI objects in the application being testedii. Create test scripts by recording, programming, or a combination of both. While recording tests, insert checkpoints where you want to check the response of the application being tested.iii. Debug Test: run tests in Debug mode to make sure they run smoothlyiv. Run Tests: run tests in Verify mode to test your application.v. View Results: determines the success or failure of the tests.vi. Report Defects: If a test run fails due to a defect in the application being tested, you can report information about the defect directly from the Test Results window.
What is contained in the GUI map?- WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
- There are 2 types of GUI Map files.i. Global GUI Map file: a single GUI Map file for the entire applicationii. GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
How does WinRunner recognize objects on the application?WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.
Have you created test scripts and what is contained in the test scripts?Yes I have created test scripts. It contains the statement in Mercury Interactive’s Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using WinRunner’s visual programming tool, the Function Generator.
How does WinRunner evaluates test results?Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.
Have you performed debugging of the scripts?Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.
How do you run your test scripts?We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results.
Explain WinRunner testing process?WinRunner testing process involves six main stagesi. Create GUI Map File so that WinRunner can recognize the GUI objects in the application being testedii. Create test scripts by recording, programming, or a combination of both. While recording tests, insert checkpoints where you want to check the response of the application being tested.iii. Debug Test: run tests in Debug mode to make sure they run smoothlyiv. Run Tests: run tests in Verify mode to test your application.v. View Results: determines the success or failure of the tests.vi. Report Defects: If a test run fails due to a defect in the application being tested, you can report information about the defect directly from the Test Results window.
What is contained in the GUI map?- WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
- There are 2 types of GUI Map files.i. Global GUI Map file: a single GUI Map file for the entire applicationii. GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
How does WinRunner recognize objects on the application?WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.
Have you created test scripts and what is contained in the test scripts?Yes I have created test scripts. It contains the statement in Mercury Interactive’s Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using WinRunner’s visual programming tool, the Function Generator.
How does WinRunner evaluates test results?Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.
Have you performed debugging of the scripts?Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.
How do you run your test scripts?We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results.
Winrunner questions from Interviewer point of view
Screening questions: Interviewer can Reject the candidate if he /she is not able to answer 2 questions out of following 4 questions:
1. Where did you learn WinRunner and TestDirector?
If they say it was in a mercury class, ask if they can show you theircertificate of completion. If they say no, let them know you will verify itwith Mercury.
2. Have you ever created a start-up script?
If they answer “yes”, ask themwhat was in it and how they got WR to execute the start-up script when WR isinvoked.
They should answer something like this, “In the start-up script, we loaded allthe gui maps for the application, any library files we needed, and any customgui object mapping we have to do. We also loaded global variables and systemvariables here. The startup script location is added to the WR .ini file,wrun.ini located under the windows or winnt directory as wrun.ini”
3. What is the difference between writing a function and writing a script?
They should mention some of these things:1. A function goes into a “compiled module”, a script does not.2. A function follows strict “c” syntax. For instance, you have to declareall variables created and used in the function. In a script you do not.
4. What is the difference between “set_window” and “win_activate”.?When would you use “set_window” and when would you use “win_activate”?win_activate has the format win_activate(window);. The win_activate functionmakes the specified window the active window by bringing it into focus andraising it to the top of the display. (It is the equivalent to clicking on thewindow banner)
Set_window has the following format: set_window(window,[time]);The set_window function directs input to the correct application window. Thisdirects the GUI map to this window. It also sets the scope for objectidentification in the GUI map.
The most important difference is that set_window has a timing option.WinRunner will wait a maximum of the number used in the function, PLUS thesystem set timeout, to wait for the window to appear. Win_activate assumes thewindow is already on the desktop and has no timing option.
1. Where did you learn WinRunner and TestDirector?
If they say it was in a mercury class, ask if they can show you theircertificate of completion. If they say no, let them know you will verify itwith Mercury.
2. Have you ever created a start-up script?
If they answer “yes”, ask themwhat was in it and how they got WR to execute the start-up script when WR isinvoked.
They should answer something like this, “In the start-up script, we loaded allthe gui maps for the application, any library files we needed, and any customgui object mapping we have to do. We also loaded global variables and systemvariables here. The startup script location is added to the WR .ini file,wrun.ini located under the windows or winnt directory as wrun.ini”
3. What is the difference between writing a function and writing a script?
They should mention some of these things:1. A function goes into a “compiled module”, a script does not.2. A function follows strict “c” syntax. For instance, you have to declareall variables created and used in the function. In a script you do not.
4. What is the difference between “set_window” and “win_activate”.?When would you use “set_window” and when would you use “win_activate”?win_activate has the format win_activate(window);. The win_activate functionmakes the specified window the active window by bringing it into focus andraising it to the top of the display. (It is the equivalent to clicking on thewindow banner)
Set_window has the following format: set_window(window,[time]);The set_window function directs input to the correct application window. Thisdirects the GUI map to this window. It also sets the scope for objectidentification in the GUI map.
The most important difference is that set_window has a timing option.WinRunner will wait a maximum of the number used in the function, PLUS thesystem set timeout, to wait for the window to appear. Win_activate assumes thewindow is already on the desktop and has no timing option.
Some tricky question answers
1. Define the following along with examplesa. Boundary Value testingb. Equivalence testingc. Error Guessingd. Desk checkinge. Control Flow analysisAnswer:1-a) Boundary value Analysis: -A process of selecting test cases/data byidentifying the boundaries that separate valid and invalid conditions. Tests areconstructed to test the inside and outside edges of these boundaries, in addition tothe actual boundary points. or A selection technique in which test data are chosen tolie along “boundaries” of the input domain [or output range] classes, data structures,procedure parameters, etc. Choices often include maximum, minimum, and trivialvalues or parameters.E.g. - Input data 1 to 10 (boundary value)Test input data 0, 1, 2 to 9, 10, 11
1-b) Equivalence testing: -The input domain of the system is partitioned into classesof representative values, so that the no of test cases can be limited to one-per-class,which represents the minimum no. of test cases that must be executed.E.g.- valid data range: 1-10Test set:-2; 5; 14
1-c) Error guessing: -Test data selection technique. The selection criterion is to pickvalues that seem likely to cause errors Error guessing is based mostly uponexperience, with some assistance from other techniques such as boundary valueanalysis. Based on experience, the test designer guesses the types of errors thatcould occur in a particular type of software and designs test cases to uncover them.E.g. - For example, if any type of resource is allocated dynamically, a good place tolook for errors is in the de-allocation of resources. Are all resources correctly deallocated,or are some lost as the software executes?
1-d) Desk checking: -Desk checking is conducted by the developer of the system orprogram. The process involves reviewing the complete product to ensure that it isstructurally sound and that the standards and requirements have been met. This isthe most traditional means for analyzing a system or program.
1-e) Control Flow Analysis: -It is based upon graphical representation of theprogram process. In control flow analysis; the program graphs has nodes whichrepresent a statement or segment possibly ending in an unresolved branch. Thegraph illustrates the flow of program control from one segment to another asillustrated through branches .the objective of control flow analysis is to determine
the potential problems in logic branches that might result in a loop condition orimproper processing .
1-b) Equivalence testing: -The input domain of the system is partitioned into classesof representative values, so that the no of test cases can be limited to one-per-class,which represents the minimum no. of test cases that must be executed.E.g.- valid data range: 1-10Test set:-2; 5; 14
1-c) Error guessing: -Test data selection technique. The selection criterion is to pickvalues that seem likely to cause errors Error guessing is based mostly uponexperience, with some assistance from other techniques such as boundary valueanalysis. Based on experience, the test designer guesses the types of errors thatcould occur in a particular type of software and designs test cases to uncover them.E.g. - For example, if any type of resource is allocated dynamically, a good place tolook for errors is in the de-allocation of resources. Are all resources correctly deallocated,or are some lost as the software executes?
1-d) Desk checking: -Desk checking is conducted by the developer of the system orprogram. The process involves reviewing the complete product to ensure that it isstructurally sound and that the standards and requirements have been met. This isthe most traditional means for analyzing a system or program.
1-e) Control Flow Analysis: -It is based upon graphical representation of theprogram process. In control flow analysis; the program graphs has nodes whichrepresent a statement or segment possibly ending in an unresolved branch. Thegraph illustrates the flow of program control from one segment to another asillustrated through branches .the objective of control flow analysis is to determine
the potential problems in logic branches that might result in a loop condition orimproper processing .
Priority and Severity
What is a test strategy?Answer:A test strategy must address the risks and present a process that can reduce thoserisks.The two components of Test strategy are:a) Test Factor: The risk of issue that needs to be addressed as a part of the teststrategy. Factors that are to be addressed in testing a specific applicationsystem will form the test factor.b) Test phase: The phase of the systems development life cycle in which testingwill occur.]
Q. When to stop testing?
Answer:a) When all the requirements are adequately executed successfully through testcasesb) Bug reporting rate reaches a particular limitc) The test environment no more exists for conducting testingd) The scheduled time for testing is overe) The budget allocation for testing is over]
Q. Your company is about to roll out an E-Commerce application. It is notpossible to test the application on all types of browsers on all platforms andoperating systems. What steps would you take in the testing environment toreduce the business risks and commercial risks?Answer:Compatibility testing should be done on all browsers (IE, Netscape, Mozilla etc.)across all the operating systems (win 98/2K/NT/XP/ME/Unix etc.)]
Q. What’s the difference between priority and severity?
Answer:“Priority” is associated with scheduling, and “severity” is associated with standards.“Priority” means something is afforded or deserves prior attention; a precedenceestablished by order of importance (or urgency). “Severity” is the state or quality ofbeing severe; severe implies adherence to rigorous standards or high principles andoften suggests harshness; severe is marked by or requires strict adherence torigorous standards or high principles, e.g. a severe code of behavior. The wordspriority and severity do come up in bug tracking. A variety of commercial, problemtracking/management software tools are available. These tools, with the detailedinput of software test engineers, give the team complete information so developerscan understand the bug, get an idea of its ’severity’, reproduce it and fix it. The fixesare based on project ‘priorities’ and ’severity’ of bugs. The ’severity’ of a problem isdefined in accordance to the customer’s risk assessment and recorded in theirselected tracking tool. A buggy software can ’severely’ affect schedules, which, inturn can lead to a reassessment and renegotiation of ‘priorities’.]
Q. When to stop testing?
Answer:a) When all the requirements are adequately executed successfully through testcasesb) Bug reporting rate reaches a particular limitc) The test environment no more exists for conducting testingd) The scheduled time for testing is overe) The budget allocation for testing is over]
Q. Your company is about to roll out an E-Commerce application. It is notpossible to test the application on all types of browsers on all platforms andoperating systems. What steps would you take in the testing environment toreduce the business risks and commercial risks?Answer:Compatibility testing should be done on all browsers (IE, Netscape, Mozilla etc.)across all the operating systems (win 98/2K/NT/XP/ME/Unix etc.)]
Q. What’s the difference between priority and severity?
Answer:“Priority” is associated with scheduling, and “severity” is associated with standards.“Priority” means something is afforded or deserves prior attention; a precedenceestablished by order of importance (or urgency). “Severity” is the state or quality ofbeing severe; severe implies adherence to rigorous standards or high principles andoften suggests harshness; severe is marked by or requires strict adherence torigorous standards or high principles, e.g. a severe code of behavior. The wordspriority and severity do come up in bug tracking. A variety of commercial, problemtracking/management software tools are available. These tools, with the detailedinput of software test engineers, give the team complete information so developerscan understand the bug, get an idea of its ’severity’, reproduce it and fix it. The fixesare based on project ‘priorities’ and ’severity’ of bugs. The ’severity’ of a problem isdefined in accordance to the customer’s risk assessment and recorded in theirselected tracking tool. A buggy software can ’severely’ affect schedules, which, inturn can lead to a reassessment and renegotiation of ‘priorities’.]
Software testing interview questions Test manager Part I
Q Customer has reported severe defects in Daily balance report. The customeris unhappy that the problem is not fixed even after a week. What action youas a PM will take to restore confidence of customer and ensure that this willnot happen in suture?Answer:Conflict resolution – Get on your customer wavelength. Get the facts and askquestions, get detail info and take notes listen carefully. Establish and initiate anaction program(admit error if it is there, negotiate satisfactory solution, state thesolution and get agreement, take action and follow up with customer). Finallyestablish proper daily problem review process to prevent such problems in future.
Q. It’s observed that the testers in your organization are performing tests on thedeliverable even after significant defects have been found. This has resultedin unnecessary testing of little value because re-testing needs to be doneafter defects have been rectified. You are the test manager and going toupdate the test plan with recommendations on when to stop testing. List therecommendations you are going to make.Answer:Following steps need to be taken .a) Acceptance criteria should tightenb) Test cases should be re-evaluated (preferably peer review)c) If possible more test cases should be added. With boundary value andequivalence class partition cases.d) More test cases with invalid condition should be addede) Stop criteria needs to be modified
Q. You are newly appointed as a test lead in an organization which uses manualtesting. Your boss wants you to put forth three testing tools and their featuresto create awareness about the testing tools in the top management. Suggestany three testing tools for your test Environment and why do you suggestthem?Answer:The third question is very important one. You can write about test Director, Winrunner/Load runner, McCable or any other coverage tool. Test director is useful totrack defect. WR or LR to do functionality/Load testing, Coverage tool to check thecode coverage thereby helping in White box testing.
Q. You are working on a project, where the requirements change dynamically.The data in the project comes from various ends (from various Platforms) andare inter-dependent. You see this as a big risk in the project. How would youplan accordingly?Answer:Give a Plan which takes care of the risk and is identified in the Risk Areas. Say thatthe testing scope would concentrate more on Data driven tests etc.
Q. It’s observed that the testers in your organization are performing tests on thedeliverable even after significant defects have been found. This has resultedin unnecessary testing of little value because re-testing needs to be doneafter defects have been rectified. You are the test manager and going toupdate the test plan with recommendations on when to stop testing. List therecommendations you are going to make.Answer:Following steps need to be taken .a) Acceptance criteria should tightenb) Test cases should be re-evaluated (preferably peer review)c) If possible more test cases should be added. With boundary value andequivalence class partition cases.d) More test cases with invalid condition should be addede) Stop criteria needs to be modified
Q. You are newly appointed as a test lead in an organization which uses manualtesting. Your boss wants you to put forth three testing tools and their featuresto create awareness about the testing tools in the top management. Suggestany three testing tools for your test Environment and why do you suggestthem?Answer:The third question is very important one. You can write about test Director, Winrunner/Load runner, McCable or any other coverage tool. Test director is useful totrack defect. WR or LR to do functionality/Load testing, Coverage tool to check thecode coverage thereby helping in White box testing.
Q. You are working on a project, where the requirements change dynamically.The data in the project comes from various ends (from various Platforms) andare inter-dependent. You see this as a big risk in the project. How would youplan accordingly?Answer:Give a Plan which takes care of the risk and is identified in the Risk Areas. Say thatthe testing scope would concentrate more on Data driven tests etc.
Test your Software Testing knowledge. Take this mock test
If you are preparing for CSTE testing certification exam or thinking to give the exam in coming days then this question series will help you for preparation. Here I have included some questions from CSTE objective type question papers.
The software testing/Quality assurance professionals can also take this exam to test their testing knowledge.
You can either take printout and mark the answers or note your answers somewhere on the paper serially for all 25 questions. Verify your answers at the answer page provided at the bottom of this test page.
1. Verification is: a. Checking that we are building the right systemb. Checking that we are building the system rightc. Performed by an independent test teamd. Making sure that it is what the user really wants
2. A regression test:a. Will always be automatedb. Will help ensure unchanged areas of the software have not been affectedc. Will help ensure changed areas of the software have not been affectedd. Can only be run during user acceptance testing
3. If an expected result is not specified then: a. We cannot run the testb. It may be difficult to repeat the testc. It may be difficult to determine if the test has passed or failedd. We cannot automate the user inputs
4. Which of the following could be a reason for a failure1) Testing fault2) Software fault3) Design fault4) Environment Fault5) Documentation Faulta. 2 is a valid reason; 1,3,4 & 5 are notb. 1,2,3,4 are valid reasons; 5 is notc. 1,2,3 are valid reasons; 4 & 5 are notd. All of them are valid reasons for failure
5. Test are prioritized so that: a. You shorten the time required for testingb. You do the best testing in the time availablec. You do more effective testingd. You find more faults
6. Which of the following is not a static testing technique a. Error guessingb. Walkthroughc. Data flow analysisd. Inspections
7. Which of the following statements about component testing is not true?a. Component testing should be performed by developmentb. Component testing is also know as isolation or module testingc. Component testing should have completion criteria plannedd. Component testing does not involve regression testing
8. During which test activity could faults be found most cost effectively? a. Executionb. Designc. Planningd. Check Exit criteria completion
9. Which, in general, is the least required skill of a good tester?a. Being diplomaticb. Able to write softwarec. Having good attention to detaild. Able to be relied on
10. The purpose of requirement phase is a. To freeze requirementsb. To understand user needsc. To define the scope of testingd. All of the above
11. The process starting with the terminal modules is called -a. Top-down integrationb. Bottom-up integrationc. None of the aboved. Module integration
12. The inputs for developing a test plan are taken from a. Project planb. Business planc. Support pland. None of the above
13. Function/Test matrix is a type of a. Interim Test reportb. Final test reportc. Project status reportd. Management report
14. Defect Management process does not includea. Defect preventionb. Deliverable base-liningc. Management reportingd. None of the above
15. What is the difference between testing software developed by contractor outside your country, versus testing software developed by a contractor within your country?a. Does not meet people needsb. Cultural differencec. Loss of control over reallocation of resourcesd. Relinquishments of control
16. Software testing accounts to what percent of software development costs?a. 10-20b. 40-50c. 70-80d. 5-10
17. A reliable system will be one that:a. Is unlikely to be completed on scheduleb. Is unlikely to cause a failurec. Is likely to be fault-freed. Is likely to be liked by the users
18. How much testing is enough a. This question is impossible to answerb. The answer depends on the risks for your industry, contract and special requirementsc. The answer depends on the maturity of your developersd. The answer should be standardized for the software development industry
19. Which of the following is not a characteristic for Testability? a. Operabilityb. Observabilityc. Simplicityd. Robustness
20. Cyclomatic Complexity method comes under which testing method. a. White boxb. Black boxc. Green boxd. Yellow box
21. Which of these can be successfully tested using Loop Testing methodology? a. Simple Loopsb. Nested Loopsc. Concatenated Loopsd. All of the above
22. To test a function, the programmer has to write a ______, which calls the function and passes it test data.a. Stubb. Driverc. Proxyd. None of the above
23. Equivalence partitioning is: a. A black box testing technique used only by developersb. A black box testing technique than can only be used during system testingc. A black box testing technique appropriate to all levels of testingd. A white box testing technique appropriate for component testing
24. When a new testing tool is purchased, it should be used first by: a. A small team to establish the best way to use the toolb. Everyone who may eventually have some use for the toolc. The independent testing teamd. The vendor contractor to write the initial scripts
25. Inspections can find all the following except a. Variables not defined in the codeb. Spelling and grammar faults in the documentsc. Requirements that have been omitted from the design documentsd. How much of the code has been covered
Done?
Click here to Verify your answers.
The software testing/Quality assurance professionals can also take this exam to test their testing knowledge.
You can either take printout and mark the answers or note your answers somewhere on the paper serially for all 25 questions. Verify your answers at the answer page provided at the bottom of this test page.
1. Verification is: a. Checking that we are building the right systemb. Checking that we are building the system rightc. Performed by an independent test teamd. Making sure that it is what the user really wants
2. A regression test:a. Will always be automatedb. Will help ensure unchanged areas of the software have not been affectedc. Will help ensure changed areas of the software have not been affectedd. Can only be run during user acceptance testing
3. If an expected result is not specified then: a. We cannot run the testb. It may be difficult to repeat the testc. It may be difficult to determine if the test has passed or failedd. We cannot automate the user inputs
4. Which of the following could be a reason for a failure1) Testing fault2) Software fault3) Design fault4) Environment Fault5) Documentation Faulta. 2 is a valid reason; 1,3,4 & 5 are notb. 1,2,3,4 are valid reasons; 5 is notc. 1,2,3 are valid reasons; 4 & 5 are notd. All of them are valid reasons for failure
5. Test are prioritized so that: a. You shorten the time required for testingb. You do the best testing in the time availablec. You do more effective testingd. You find more faults
6. Which of the following is not a static testing technique a. Error guessingb. Walkthroughc. Data flow analysisd. Inspections
7. Which of the following statements about component testing is not true?a. Component testing should be performed by developmentb. Component testing is also know as isolation or module testingc. Component testing should have completion criteria plannedd. Component testing does not involve regression testing
8. During which test activity could faults be found most cost effectively? a. Executionb. Designc. Planningd. Check Exit criteria completion
9. Which, in general, is the least required skill of a good tester?a. Being diplomaticb. Able to write softwarec. Having good attention to detaild. Able to be relied on
10. The purpose of requirement phase is a. To freeze requirementsb. To understand user needsc. To define the scope of testingd. All of the above
11. The process starting with the terminal modules is called -a. Top-down integrationb. Bottom-up integrationc. None of the aboved. Module integration
12. The inputs for developing a test plan are taken from a. Project planb. Business planc. Support pland. None of the above
13. Function/Test matrix is a type of a. Interim Test reportb. Final test reportc. Project status reportd. Management report
14. Defect Management process does not includea. Defect preventionb. Deliverable base-liningc. Management reportingd. None of the above
15. What is the difference between testing software developed by contractor outside your country, versus testing software developed by a contractor within your country?a. Does not meet people needsb. Cultural differencec. Loss of control over reallocation of resourcesd. Relinquishments of control
16. Software testing accounts to what percent of software development costs?a. 10-20b. 40-50c. 70-80d. 5-10
17. A reliable system will be one that:a. Is unlikely to be completed on scheduleb. Is unlikely to cause a failurec. Is likely to be fault-freed. Is likely to be liked by the users
18. How much testing is enough a. This question is impossible to answerb. The answer depends on the risks for your industry, contract and special requirementsc. The answer depends on the maturity of your developersd. The answer should be standardized for the software development industry
19. Which of the following is not a characteristic for Testability? a. Operabilityb. Observabilityc. Simplicityd. Robustness
20. Cyclomatic Complexity method comes under which testing method. a. White boxb. Black boxc. Green boxd. Yellow box
21. Which of these can be successfully tested using Loop Testing methodology? a. Simple Loopsb. Nested Loopsc. Concatenated Loopsd. All of the above
22. To test a function, the programmer has to write a ______, which calls the function and passes it test data.a. Stubb. Driverc. Proxyd. None of the above
23. Equivalence partitioning is: a. A black box testing technique used only by developersb. A black box testing technique than can only be used during system testingc. A black box testing technique appropriate to all levels of testingd. A white box testing technique appropriate for component testing
24. When a new testing tool is purchased, it should be used first by: a. A small team to establish the best way to use the toolb. Everyone who may eventually have some use for the toolc. The independent testing teamd. The vendor contractor to write the initial scripts
25. Inspections can find all the following except a. Variables not defined in the codeb. Spelling and grammar faults in the documentsc. Requirements that have been omitted from the design documentsd. How much of the code has been covered
Done?
Click here to Verify your answers.
How to prepare for software testing interview
This is the a guest post from author “Suhas R M”. Author is working as a software test engineer and having 4 years of manual testing experience.
A colleague of mine recently asked me, “Where should I start with for preparation of interview? It’s almost 2 years I have faced any interview.”
My answer was straightforward: (This will help for fresher as well as working professional who want to switch the current job)
What you need to know about software testing?First basic thing - Testing Concepts: One needs to be very good at this especially the manual testing methodologies. But only knowing different testing concepts is half work done. The next - most important thing is to know which type/technique/concept of testing can be applied at what stage of SDLC.
“What should I test and when” is very important. There might be some concepts, which are not applicable to what we, professional test in our company, but it’s always better to have an idea of all testing practices.
Many freshers and working testing professionals have might not worked on various testing domains like localization testing, time Zone testing etc. But knowing more than what you have worked on will help you better answering the different questions from the interviewer. I always try to keep my testing knowledge updated besides my current project work. This helped me a lot while switching my job some years ago. What if an interviewer asks you question on topic, which you have never worked on? For example you don’t have any experience on web based projects or client server testing and interviewer asks you to test “Yahoo mail application”. Will you be able to answer this question? You can. Even you have not worked on this type of projects. How? Your curiosity to learn the things you never done before will help you in this case. So broaden your thinking area, be curious in every work and every query you face in your daily work routine.
Knowing more is harmless and will definitely help you at least to give your thoughts on the questions asked by interviewer.
If you don’t know any testing concept, e.g. “Localization testing”, then try to learn the concepts first. Like - what would be localization testing? It’s simple; Test if the application looks local for you while using. Then go on exploring. See for used colors, content, images, culture etc, Different countries, locales have it in different way. Consider a web site that reads from right to left, is it accepted in countries other than Middle East? Obviously NO. Or can you display the same geo specific content in India what you can display in US? Again NO. This is just a simple example how you can learn unknown testing concepts.
The very essential part of a test engineer is “Thinking out of box”. If you are not capable of thinking out of box, believe me testing is not for you. What do you mean by thinking out of box? Don’t just follow the traditional methods. Implement new things in testing. Try to summarize, automate the routine testing work. Think from user perspective. Think how user will use your application. What common mistakes he can make or which tasks he can perform on your application? This way you will get insight of any application and will also help to answer the questions in depth.
Besides from “curiosity to learn” you should upgrade your skill in following areas:- Some hands-on on basic database/SQL queries and concepts- Any basic scripting language (For automation testing)- Networking and system administration concepts will help you in system domain projects.
Do not just write the UI test cases, check what is happening inside the application. For application having database connection check for data updation, retrieving and in any case there should not be loss of data.
Get hold on project. Know the application under test before starting to test it. Instead of looking in the requirement document, look into the architecture doc, design doc, sequence diagram and activity flow diagram.
Most importantly you need to be perfect in what you mention in your CV. All the questions interviewer asks will be based on what you are specifying in your CV. So do not mention the skills you have not worked on, just for the sake of decorating the CV with multiple skills.
The key point in interview is, You should make interviewer feel that it was a complex application you were testing and had lot many challenges in it for a tester!
And one last thing - If you don’t know answer for a question, say so. Don’t fool around and get into trouble.
A colleague of mine recently asked me, “Where should I start with for preparation of interview? It’s almost 2 years I have faced any interview.”
My answer was straightforward: (This will help for fresher as well as working professional who want to switch the current job)
What you need to know about software testing?First basic thing - Testing Concepts: One needs to be very good at this especially the manual testing methodologies. But only knowing different testing concepts is half work done. The next - most important thing is to know which type/technique/concept of testing can be applied at what stage of SDLC.
“What should I test and when” is very important. There might be some concepts, which are not applicable to what we, professional test in our company, but it’s always better to have an idea of all testing practices.
Many freshers and working testing professionals have might not worked on various testing domains like localization testing, time Zone testing etc. But knowing more than what you have worked on will help you better answering the different questions from the interviewer. I always try to keep my testing knowledge updated besides my current project work. This helped me a lot while switching my job some years ago. What if an interviewer asks you question on topic, which you have never worked on? For example you don’t have any experience on web based projects or client server testing and interviewer asks you to test “Yahoo mail application”. Will you be able to answer this question? You can. Even you have not worked on this type of projects. How? Your curiosity to learn the things you never done before will help you in this case. So broaden your thinking area, be curious in every work and every query you face in your daily work routine.
Knowing more is harmless and will definitely help you at least to give your thoughts on the questions asked by interviewer.
If you don’t know any testing concept, e.g. “Localization testing”, then try to learn the concepts first. Like - what would be localization testing? It’s simple; Test if the application looks local for you while using. Then go on exploring. See for used colors, content, images, culture etc, Different countries, locales have it in different way. Consider a web site that reads from right to left, is it accepted in countries other than Middle East? Obviously NO. Or can you display the same geo specific content in India what you can display in US? Again NO. This is just a simple example how you can learn unknown testing concepts.
The very essential part of a test engineer is “Thinking out of box”. If you are not capable of thinking out of box, believe me testing is not for you. What do you mean by thinking out of box? Don’t just follow the traditional methods. Implement new things in testing. Try to summarize, automate the routine testing work. Think from user perspective. Think how user will use your application. What common mistakes he can make or which tasks he can perform on your application? This way you will get insight of any application and will also help to answer the questions in depth.
Besides from “curiosity to learn” you should upgrade your skill in following areas:- Some hands-on on basic database/SQL queries and concepts- Any basic scripting language (For automation testing)- Networking and system administration concepts will help you in system domain projects.
Do not just write the UI test cases, check what is happening inside the application. For application having database connection check for data updation, retrieving and in any case there should not be loss of data.
Get hold on project. Know the application under test before starting to test it. Instead of looking in the requirement document, look into the architecture doc, design doc, sequence diagram and activity flow diagram.
Most importantly you need to be perfect in what you mention in your CV. All the questions interviewer asks will be based on what you are specifying in your CV. So do not mention the skills you have not worked on, just for the sake of decorating the CV with multiple skills.
The key point in interview is, You should make interviewer feel that it was a complex application you were testing and had lot many challenges in it for a tester!
And one last thing - If you don’t know answer for a question, say so. Don’t fool around and get into trouble.
Software testing questions and answers
This article is the part software testing question and answer series. Here I will answer some reader’s questions asked to me in comments or using contact form. If you have queries on software testing, quality assurance or career in testing then you can ask me these questions in comment section below.
It’s not possible to address each and every question in detail as I observed the questions are on vast topics, for which detail answers will itself require a new article. I will answer such questions in brief here and will also write detail articles separately if required.
So let’s get some questions answered:
Naresh A. asks:
“My past experience was related to “Test Engineer”. Recently I am appointed as Test Lead in a product based company. Currently there is no Pre-established testing process. As a TL am meant to define a standard process for the entire testing flow and I will maintain certain documents for each product.Can you help me out in establishing a process for testing, and make me know the entire responsibilities of TL and what documents I am supposed to prepare and maintain?”
As a team leader you are responsible for project planning, scheduling, communicating your project status to your manager and most important task of assigning and monitoring the project work. Your main responsibility is to build a team to achieve your project goals. You need to focus on handling the challenges in your project so that your team and project will grow and perform well.
As far as the standard testing process is considered, it’s depends on you - what procedure you want to establish. Yes some people might blame me for this point but I prefer to establish my own processes that work for me. I don’t stick to those old process definitions that are written and managed in some 90’s and most of which might not applicable nowadays.
Test lead is responsible for ensuring project plan changes are incorporated in test plan. You might write a test plan and test strategy (In some cases it might be written by senior test team member or even by project test manager) Ensure the work is going according to this test plan. Identify the risks and try to mitigate them. At the end of project testing life cycle ensure that all test objectives are accomplished and acceptance criteria is met.
More TL responsibilities includes: Test Case Review, Requirements Validation, Monitoring the execution of manual and automated test cases, Prepare test summary report and Communicate test status to seniors and prepare corresponding documents.
To know more on SQA processes read this article “SQA Processes- How to Test complete application“. Hope from this answer you will get good idea of testing processes and TL responsibilities.
Pavan Ankus asks:
“I am appearing for the QA positions in US. I would kindly request you to mail me the suitable challenging situations in manual testing and also since I don’t have domain knowledge in Insurance, finance and other financial domain experience I am finding hard to explain to the interviewer as an experienced person. In this regard I need your suitable answer as to how to face the interviewer?”
In every testing interview you will get this question: “Tell me any challenging situation you faced in your previous projects or Tell me any bug that you feel proud to find it?”
I think answers to these questions depend on your testing career. I know every one of you might have faced many challenging situations where exceptional thinking is required to solve such problems.
I will suggest to pick any such situation from you career and explain it in better way. At least it should sound challenging This will help you to face further questions from interviewer depending on your answer.
The broad challenges in manual testing are: How to ensure complete test coverage? Testing without an automation tool is itself a big challenge. You can also explain non-technical challenges in manual testing like managing the testing work in critical time (Llink to testing under time limit) i.e. completing testing before deadline and even worst case if the deadline itself is not feasible.
Explaining a challenging bug you found in your career can be also a good answer for this question. For example the bug that was difficult to find or reprove or having big impact on customer revenue etc.
Pavan you mentioned that you don’t have knowledge in banking and finance domain then how you expect from yourself to give answer on that? If you don’t have experience in banking and finance domain then do not put this as a skill in your resume just for the sake of matching your profile with employer requirements. If you really want to get into testing of BFSI (Banking, Financial services and Insurance) domain then first study this domain. Know the basic concepts in BFSI domain. See the resources I have listed on BFSI domain on our resource page. Keep in mind you can answer in detail about any question if you have worked on that.
Mitch asks:
“What is the best way to go about getting a pay rise? Is reporting and graphing bugs found compared to other team member a good idea?
Comparing the bug count with other team or team member is very bad idea to ask for pay rise. If you are working for the organization for long time then your employer know your value and importance in organization. There is no need to show how your bug count graph is higher than your counterparts.
So what is the best way to ask for good salary rise? At the time of your performance appraisal you should be able to convince to your reviewer that how you worked hard for your organization, How you succeeded in managing difficult tasks and how you enhanced your skills to better match your current work profile. If you succeed in this negotiation then you will definitely get good pay rise.
It’s not possible to address each and every question in detail as I observed the questions are on vast topics, for which detail answers will itself require a new article. I will answer such questions in brief here and will also write detail articles separately if required.
So let’s get some questions answered:
Naresh A. asks:
“My past experience was related to “Test Engineer”. Recently I am appointed as Test Lead in a product based company. Currently there is no Pre-established testing process. As a TL am meant to define a standard process for the entire testing flow and I will maintain certain documents for each product.Can you help me out in establishing a process for testing, and make me know the entire responsibilities of TL and what documents I am supposed to prepare and maintain?”
As a team leader you are responsible for project planning, scheduling, communicating your project status to your manager and most important task of assigning and monitoring the project work. Your main responsibility is to build a team to achieve your project goals. You need to focus on handling the challenges in your project so that your team and project will grow and perform well.
As far as the standard testing process is considered, it’s depends on you - what procedure you want to establish. Yes some people might blame me for this point but I prefer to establish my own processes that work for me. I don’t stick to those old process definitions that are written and managed in some 90’s and most of which might not applicable nowadays.
Test lead is responsible for ensuring project plan changes are incorporated in test plan. You might write a test plan and test strategy (In some cases it might be written by senior test team member or even by project test manager) Ensure the work is going according to this test plan. Identify the risks and try to mitigate them. At the end of project testing life cycle ensure that all test objectives are accomplished and acceptance criteria is met.
More TL responsibilities includes: Test Case Review, Requirements Validation, Monitoring the execution of manual and automated test cases, Prepare test summary report and Communicate test status to seniors and prepare corresponding documents.
To know more on SQA processes read this article “SQA Processes- How to Test complete application“. Hope from this answer you will get good idea of testing processes and TL responsibilities.
Pavan Ankus asks:
“I am appearing for the QA positions in US. I would kindly request you to mail me the suitable challenging situations in manual testing and also since I don’t have domain knowledge in Insurance, finance and other financial domain experience I am finding hard to explain to the interviewer as an experienced person. In this regard I need your suitable answer as to how to face the interviewer?”
In every testing interview you will get this question: “Tell me any challenging situation you faced in your previous projects or Tell me any bug that you feel proud to find it?”
I think answers to these questions depend on your testing career. I know every one of you might have faced many challenging situations where exceptional thinking is required to solve such problems.
I will suggest to pick any such situation from you career and explain it in better way. At least it should sound challenging This will help you to face further questions from interviewer depending on your answer.
The broad challenges in manual testing are: How to ensure complete test coverage? Testing without an automation tool is itself a big challenge. You can also explain non-technical challenges in manual testing like managing the testing work in critical time (Llink to testing under time limit) i.e. completing testing before deadline and even worst case if the deadline itself is not feasible.
Explaining a challenging bug you found in your career can be also a good answer for this question. For example the bug that was difficult to find or reprove or having big impact on customer revenue etc.
Pavan you mentioned that you don’t have knowledge in banking and finance domain then how you expect from yourself to give answer on that? If you don’t have experience in banking and finance domain then do not put this as a skill in your resume just for the sake of matching your profile with employer requirements. If you really want to get into testing of BFSI (Banking, Financial services and Insurance) domain then first study this domain. Know the basic concepts in BFSI domain. See the resources I have listed on BFSI domain on our resource page. Keep in mind you can answer in detail about any question if you have worked on that.
Mitch asks:
“What is the best way to go about getting a pay rise? Is reporting and graphing bugs found compared to other team member a good idea?
Comparing the bug count with other team or team member is very bad idea to ask for pay rise. If you are working for the organization for long time then your employer know your value and importance in organization. There is no need to show how your bug count graph is higher than your counterparts.
So what is the best way to ask for good salary rise? At the time of your performance appraisal you should be able to convince to your reviewer that how you worked hard for your organization, How you succeeded in managing difficult tasks and how you enhanced your skills to better match your current work profile. If you succeed in this negotiation then you will definitely get good pay rise.
What you need to know about BVT (Build Verification Testing)
What is BVT?
Build Verification test is a set of tests run on every new build to verify that build is testable before it is released to test team for further testing. These test cases are core functionality test cases that ensure application is stable and can be tested thoroughly. Typically BVT process is automated. If BVT fails that build is again get assigned to developer for fix.
BVT is also called smoke testing or build acceptance testing (BAT)New Build is checked mainly for two things:
Build validation
Build acceptance
Some BVT basics:
It is a subset of tests that verify main functionalities.
The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken.
Design BVTs carefully enough to cover basic functionality.
Typically BVT should not run more than 30 minutes.
BVT is a type of regression testing, done on each and every new build.
BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules. I heard many cases of application failure due to improper module integration. Even in worst cases complete project gets scraped due to failure in module integration.
What is the main task in build release? Obviously file ‘check in’ i.e. to include all the new and modified project files associated with respective builds. BVT was primarily introduced to check initial build health i.e. to check whether - all the new and modified files are included in release, all file formats are correct, every file version and language, flags associated with each file.These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.
Which test cases should be included in BVT?
This is very tricky decision to take before automating the BVT task. Keep in mind that success of BVT depends on which test cases you include in BVT.
Here are some simple tips to include test cases in your BVT automation suite:
Include only critical test cases in BVT.
All test cases included in BVT should be stable.
All the test cases should have known expected result.
Make sure all included critical functionality test cases are sufficient for application test coverage.
Also do not includes modules in BVT, which are not yet stable. For some under-development features you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point using such modules or test cases in BVT.
You can make this critical functionality test cases inclusion task simple by communicating with all those involved in project development and testing life cycle. Such process should negotiate BVT test cases, which ultimately ensure BVT success. Set some BVT quality standards and these standards can be met only by analyzing major project features and scenarios.
Example: Test cases to be included in BVT for Text editor application (Some sample tests only):1) Test case for creating text file.2) Test cases for writing something into text editor3) Test case for copy, cut, paste functionality of text editor4) Test case for opening, saving, deleting text file.
These are some sample test cases, which can be marked as ‘critical’ and for every minor or major changes in application these basic critical test cases should be executed. This task can be easily accomplished by BVT.
BVT automation suits needs to be maintained and modified time-to-time. E.g. include test cases in BVT when there are new stable project modules available.
What happens when BVT suite run:Say Build verification automation test suite executed after any new build.1) The result of BVT execution is sent to all the email ID’s associated with that project.2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.3) If BVT fails then BVT owner diagnose the cause of failure.4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if it’s a bug then what will be his bug-fixing scenario.6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.
This process gets repeated for every new build.
Why BVT or build fails?BVT breaks sometimes. This doesn’t mean that there is always bug in the build. There are some other reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures etc.You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.
Tips for BVT success:1) Spend considerable time writing BVT test cases scripts.2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.4) Automate BVT process as much as possible. Right from build release process to BVT result - automate everything.5) Have some penalties for breaking the build Some chocolates or team coffee party from developer who breaks the build will do.
Conclusion:BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.
If you have some experience in BVT process then please share it with our readers in comments below.
Build Verification test is a set of tests run on every new build to verify that build is testable before it is released to test team for further testing. These test cases are core functionality test cases that ensure application is stable and can be tested thoroughly. Typically BVT process is automated. If BVT fails that build is again get assigned to developer for fix.
BVT is also called smoke testing or build acceptance testing (BAT)New Build is checked mainly for two things:
Build validation
Build acceptance
Some BVT basics:
It is a subset of tests that verify main functionalities.
The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken.
Design BVTs carefully enough to cover basic functionality.
Typically BVT should not run more than 30 minutes.
BVT is a type of regression testing, done on each and every new build.
BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules. I heard many cases of application failure due to improper module integration. Even in worst cases complete project gets scraped due to failure in module integration.
What is the main task in build release? Obviously file ‘check in’ i.e. to include all the new and modified project files associated with respective builds. BVT was primarily introduced to check initial build health i.e. to check whether - all the new and modified files are included in release, all file formats are correct, every file version and language, flags associated with each file.These basic checks are worth before build release to test team for testing. You will save time and money by discovering the build flaws at the very beginning using BVT.
Which test cases should be included in BVT?
This is very tricky decision to take before automating the BVT task. Keep in mind that success of BVT depends on which test cases you include in BVT.
Here are some simple tips to include test cases in your BVT automation suite:
Include only critical test cases in BVT.
All test cases included in BVT should be stable.
All the test cases should have known expected result.
Make sure all included critical functionality test cases are sufficient for application test coverage.
Also do not includes modules in BVT, which are not yet stable. For some under-development features you can’t predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point using such modules or test cases in BVT.
You can make this critical functionality test cases inclusion task simple by communicating with all those involved in project development and testing life cycle. Such process should negotiate BVT test cases, which ultimately ensure BVT success. Set some BVT quality standards and these standards can be met only by analyzing major project features and scenarios.
Example: Test cases to be included in BVT for Text editor application (Some sample tests only):1) Test case for creating text file.2) Test cases for writing something into text editor3) Test case for copy, cut, paste functionality of text editor4) Test case for opening, saving, deleting text file.
These are some sample test cases, which can be marked as ‘critical’ and for every minor or major changes in application these basic critical test cases should be executed. This task can be easily accomplished by BVT.
BVT automation suits needs to be maintained and modified time-to-time. E.g. include test cases in BVT when there are new stable project modules available.
What happens when BVT suite run:Say Build verification automation test suite executed after any new build.1) The result of BVT execution is sent to all the email ID’s associated with that project.2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT.3) If BVT fails then BVT owner diagnose the cause of failure.4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if it’s a bug then what will be his bug-fixing scenario.6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.
This process gets repeated for every new build.
Why BVT or build fails?BVT breaks sometimes. This doesn’t mean that there is always bug in the build. There are some other reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures etc.You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.
Tips for BVT success:1) Spend considerable time writing BVT test cases scripts.2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.4) Automate BVT process as much as possible. Right from build release process to BVT result - automate everything.5) Have some penalties for breaking the build Some chocolates or team coffee party from developer who breaks the build will do.
Conclusion:BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.
If you have some experience in BVT process then please share it with our readers in comments below.
Types of Risks in Software Projects
Are you developing any Test plan or test strategy for your project? Have you addressed all risks properly in your test plan or test strategy?
As testing is the last part of the project, it’s always under pressure and time constraint. To save time and money you should be able to prioritize your testing work. How will prioritize testing work? For this you should be able to judge more important and less important testing work. How will you decide which work is more or less important? Here comes need of risk-based testing.
What is Risk?“Risk are future uncertain events with a probability of occurrence and a potential for loss”
Risk identification and management are the main concerns in every software project. Effective analysis of software risks will help to effective planning and assignments of work.
In this article I will cover what are the “types of risks”. In next articles I will try to focus on risk identification, risk management and mitigation.
Risks are identified, classified and managed before actual execution of program. These risks are classified in different categories.
Categories of risks:
Schedule Risk:Project schedule get slip when project tasks and schedule release risks are not addressed properly.Schedule risks mainly affect on project and finally on company economy and may lead to project failure.Schedules often slip due to following reasons:
Wrong time estimation
Resources are not tracked properly. All resources like staff, systems, skills of individuals etc.
Failure to identify complex functionalities and time required to develop those functionalities.
Unexpected project scope expansions.
Budget Risk:
Wrong budget estimation.
Cost overruns
Project scope expansion
Operational Risks:Risks of loss due to improper process implementation, failed system or some external events risks.Causes of Operational risks:
Failure to address priority conflicts
Failure to resolve the responsibilities
Insufficient resources
No proper subject training
No resource planning
No communication in team.
Technical risks:Technical risks generally leads to failure of functionality and performance.Causes of technical risks are:
Continuous changing requirements
No advanced technology available or the existing technology is in initial stages.
Product is complex to implement.
Difficult project modules integration.
Programmatic Risks:These are the external risks beyond the operational limits. These are all uncertain risks are outside the control of the program.These external events can be:
Running out of fund.
Market development
Changing customer product strategy and priority
Government rule changes.
As testing is the last part of the project, it’s always under pressure and time constraint. To save time and money you should be able to prioritize your testing work. How will prioritize testing work? For this you should be able to judge more important and less important testing work. How will you decide which work is more or less important? Here comes need of risk-based testing.
What is Risk?“Risk are future uncertain events with a probability of occurrence and a potential for loss”
Risk identification and management are the main concerns in every software project. Effective analysis of software risks will help to effective planning and assignments of work.
In this article I will cover what are the “types of risks”. In next articles I will try to focus on risk identification, risk management and mitigation.
Risks are identified, classified and managed before actual execution of program. These risks are classified in different categories.
Categories of risks:
Schedule Risk:Project schedule get slip when project tasks and schedule release risks are not addressed properly.Schedule risks mainly affect on project and finally on company economy and may lead to project failure.Schedules often slip due to following reasons:
Wrong time estimation
Resources are not tracked properly. All resources like staff, systems, skills of individuals etc.
Failure to identify complex functionalities and time required to develop those functionalities.
Unexpected project scope expansions.
Budget Risk:
Wrong budget estimation.
Cost overruns
Project scope expansion
Operational Risks:Risks of loss due to improper process implementation, failed system or some external events risks.Causes of Operational risks:
Failure to address priority conflicts
Failure to resolve the responsibilities
Insufficient resources
No proper subject training
No resource planning
No communication in team.
Technical risks:Technical risks generally leads to failure of functionality and performance.Causes of technical risks are:
Continuous changing requirements
No advanced technology available or the existing technology is in initial stages.
Product is complex to implement.
Difficult project modules integration.
Programmatic Risks:These are the external risks beyond the operational limits. These are all uncertain risks are outside the control of the program.These external events can be:
Running out of fund.
Market development
Changing customer product strategy and priority
Government rule changes.
What are the Quality attributes?
First in brief know what is Quality? Quality can be define in different manner. Quality definition may differ from person to person. But finally there should be some standards. So Quality can be defined as
Degree of excellence - Oxford dictionary
Fitness for purpose - Edward Deming
Best for the customer’s use and selling price - Feigenbaum
The totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs - ISO
How a Product developer will define quality - The product which meets the customer requirements.How Customer will define Quality - Required functionality is provided with user friendly manner.
These are some quality definitions from different perspective. Now lets see how can one measure some quality attributes of product or application.Following factors are used to measure software development quality. Each attribute can be used to measure the product performance. These attributes can be used for Quality assurance as well as Quality control. Quality Assurance activities are oriented towards prevention of introduction of defects and Quality control activities are aimed at detecting defects in products and services.
Reliability Measure if product is reliable enough to sustain in any condition. Should give consistently correct results.Product reliability is measured in terms of working of project under different working environment and different conditions.
Maintainability Different versions of the product should be easy to maintain. For development its should be easy to add code to existing system, should be easy to upgrade for new features and new technologies time to time. Maintenance should be cost effective and easy. System be easy to maintain and correcting defects or making a change in the software.
UsabilityThis can be measured in terms of ease of use. Application should be user friendly. Shouldbe easy to learn. Navigation should be simple.The system must be:
Easy to use for input preparation, operation, and interpretation of output.
Provide consistent user interface standards or conventions with our other frequently used systems.
Easy for new or infrequent users to learn to use the system.
PortabilityThis can be measured in terms of Costing issues related to porting, Technical issues related to porting, Behavioral issues related to porting.
CorrectnessApplication should be correct in terms of its functionality, calculations used internally and the navigation should be correct. This means application should adhere to functional requirements.
EfficiencyTo Major system quality attribute. Measured in terms of time required to complete any task given to the system. For example system should utilize processor capacity, disk space and memory efficiently. If system is using all the available resources then user will get degraded performance failing the system for efficiency. If system is not efficient then it can not be used in real time applications.
Integrity or securityIntegrity comes with security. System integrity or security should be sufficient to prevent unauthorized access to system functions, preventing information loss, ensure that the software is protected from virus infection, and protecting the privacy of data entered into the system.
TestabilitySystem should be easy to test and find defects. If required should be easy to divide in different modules for testing.
FlexibilityShould be flexible enough to modify. Adaptable to other products with which it needs interaction. Should be easy to interface with other standard 3rd party components.
ReusabilitySoftware reuse is a good cost efficient and time saving development way. Different code libraries classes should be generic enough to use easily in different application modules. Dividing application into different modules so that modules can be reused across the application.
InteroperabilityInteroperability of one system to another should be easy for product to exchange data or services with other systems. Different system modules should work on different operating system platforms, different databases and protocols conditions.
Degree of excellence - Oxford dictionary
Fitness for purpose - Edward Deming
Best for the customer’s use and selling price - Feigenbaum
The totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs - ISO
How a Product developer will define quality - The product which meets the customer requirements.How Customer will define Quality - Required functionality is provided with user friendly manner.
These are some quality definitions from different perspective. Now lets see how can one measure some quality attributes of product or application.Following factors are used to measure software development quality. Each attribute can be used to measure the product performance. These attributes can be used for Quality assurance as well as Quality control. Quality Assurance activities are oriented towards prevention of introduction of defects and Quality control activities are aimed at detecting defects in products and services.
Reliability Measure if product is reliable enough to sustain in any condition. Should give consistently correct results.Product reliability is measured in terms of working of project under different working environment and different conditions.
Maintainability Different versions of the product should be easy to maintain. For development its should be easy to add code to existing system, should be easy to upgrade for new features and new technologies time to time. Maintenance should be cost effective and easy. System be easy to maintain and correcting defects or making a change in the software.
UsabilityThis can be measured in terms of ease of use. Application should be user friendly. Shouldbe easy to learn. Navigation should be simple.The system must be:
Easy to use for input preparation, operation, and interpretation of output.
Provide consistent user interface standards or conventions with our other frequently used systems.
Easy for new or infrequent users to learn to use the system.
PortabilityThis can be measured in terms of Costing issues related to porting, Technical issues related to porting, Behavioral issues related to porting.
CorrectnessApplication should be correct in terms of its functionality, calculations used internally and the navigation should be correct. This means application should adhere to functional requirements.
EfficiencyTo Major system quality attribute. Measured in terms of time required to complete any task given to the system. For example system should utilize processor capacity, disk space and memory efficiently. If system is using all the available resources then user will get degraded performance failing the system for efficiency. If system is not efficient then it can not be used in real time applications.
Integrity or securityIntegrity comes with security. System integrity or security should be sufficient to prevent unauthorized access to system functions, preventing information loss, ensure that the software is protected from virus infection, and protecting the privacy of data entered into the system.
TestabilitySystem should be easy to test and find defects. If required should be easy to divide in different modules for testing.
FlexibilityShould be flexible enough to modify. Adaptable to other products with which it needs interaction. Should be easy to interface with other standard 3rd party components.
ReusabilitySoftware reuse is a good cost efficient and time saving development way. Different code libraries classes should be generic enough to use easily in different application modules. Dividing application into different modules so that modules can be reused across the application.
InteroperabilityInteroperability of one system to another should be easy for product to exchange data or services with other systems. Different system modules should work on different operating system platforms, different databases and protocols conditions.
Types of software Testing
Black box testing - Internal system design is not considered in this type of testing. Tests are based on requirements and functionality.
White box testing - This testing is based on knowledge of the internal logic of an application’s code. Also known as Glass box Testing. Internal software and code working should be known for this type of testing. Tests are based on coverage of code statements, branches, paths, conditions.
Unit testing - Testing of individual software components or modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. may require developing test driver modules or test harnesses.
Incremental integration testing - Bottom up approach for testing i.e continuous testing of an application as new functionality is added; Application functionality and modules should be independent enough to test separately. done by programmers or by testers.
Integration testing - Testing of integrated modules to verify combined functionality after integration. Modules are typically code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.
Functional testing - This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type testing geared to functional requirements of an application.
System testing - Entire system is tested as per the requirements. Black-box type testing that is based on overall requirements specifications, covers all combined parts of a system.
End-to-end testing - Similar to system testing, involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
Sanity testing - Testing to determine if a new software version is performing well enough to accept it for a major testing effort. If application is crashing for initial use then system is not stable enough for further testing and build or application is assigned to fix.
Regression testing - Testing the application as a whole for the modification in any module or functionality. Difficult to cover all the system in regression testing so typically automation tools are used for these testing types.
Acceptance testing -Normally this type of testing is done to verify if system meets the customer specified requirements. User or customer do this testing to determine whether to accept application.
Load testing - Its a performance testing to check system behavior under load. Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.
Stress testing - System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load.
Performance testing - Term often used interchangeably with ’stress’ and ‘load’ testing. To check whether system meets performance requirements. Used different performance and load tools to do this.
Usability testing - User-friendliness check. Application flow is tested, Can new user understand the application easily, Proper help documented whenever user stuck at any point. Basically system navigation is checked in this testing.
Install/uninstall testing - Tested for full, partial, or upgrade install/uninstall processes on different operating systems under different hardware, software environment.
Recovery testing - Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.
Security testing - Can system be penetrated by any hacking way. Testing how well the system protects against unauthorized internal or external access. Checked if system, database is safe from external attacks.
Compatibility testing - Testing how well software performs in a particular hardware/software/operating system/network environment and different combination s of above.
Comparison testing - Comparison of product strengths and weaknesses with previous versions or other similar products.
Alpha testing - In house virtual user environment can be created for this type of testing. Testing is done at the end of development. Still minor design changes may be made as a result of such testing.
Beta testing - Testing typically done by end-users or others. Final testing before releasing application for commercial purpose.
White box testing - This testing is based on knowledge of the internal logic of an application’s code. Also known as Glass box Testing. Internal software and code working should be known for this type of testing. Tests are based on coverage of code statements, branches, paths, conditions.
Unit testing - Testing of individual software components or modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. may require developing test driver modules or test harnesses.
Incremental integration testing - Bottom up approach for testing i.e continuous testing of an application as new functionality is added; Application functionality and modules should be independent enough to test separately. done by programmers or by testers.
Integration testing - Testing of integrated modules to verify combined functionality after integration. Modules are typically code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.
Functional testing - This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type testing geared to functional requirements of an application.
System testing - Entire system is tested as per the requirements. Black-box type testing that is based on overall requirements specifications, covers all combined parts of a system.
End-to-end testing - Similar to system testing, involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
Sanity testing - Testing to determine if a new software version is performing well enough to accept it for a major testing effort. If application is crashing for initial use then system is not stable enough for further testing and build or application is assigned to fix.
Regression testing - Testing the application as a whole for the modification in any module or functionality. Difficult to cover all the system in regression testing so typically automation tools are used for these testing types.
Acceptance testing -Normally this type of testing is done to verify if system meets the customer specified requirements. User or customer do this testing to determine whether to accept application.
Load testing - Its a performance testing to check system behavior under load. Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.
Stress testing - System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load.
Performance testing - Term often used interchangeably with ’stress’ and ‘load’ testing. To check whether system meets performance requirements. Used different performance and load tools to do this.
Usability testing - User-friendliness check. Application flow is tested, Can new user understand the application easily, Proper help documented whenever user stuck at any point. Basically system navigation is checked in this testing.
Install/uninstall testing - Tested for full, partial, or upgrade install/uninstall processes on different operating systems under different hardware, software environment.
Recovery testing - Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.
Security testing - Can system be penetrated by any hacking way. Testing how well the system protects against unauthorized internal or external access. Checked if system, database is safe from external attacks.
Compatibility testing - Testing how well software performs in a particular hardware/software/operating system/network environment and different combination s of above.
Comparison testing - Comparison of product strengths and weaknesses with previous versions or other similar products.
Alpha testing - In house virtual user environment can be created for this type of testing. Testing is done at the end of development. Still minor design changes may be made as a result of such testing.
Beta testing - Testing typically done by end-users or others. Final testing before releasing application for commercial purpose.
How Domain knowledge is Important for testers?
Looking at the current scenario from the industry it is seen that the testers are expected to have both technical testing skills as well either need to be from the domain background or have gathered domain knowledge mainly for BFSI is commonly seen.I would like to know why and when is this domain knowledge imparted to the tester during the testing cycle?”
First of all I would like to introduce three dimensional testing career mentioned by Danny R. Faught. There are three categories of skill that need to be judged before hiring any software tester. What are those three skill categories?1) Testing skill2) Domain knowledge3) Technical expertise.
No doubt that any tester should have the basic testing skills like Manual testing and Automation testing. Tester having the common sense can even find most of the obvious bugs in the software. Then would you say that this much testing is sufficient? Would you release the product on the basis of this much testing done? Certainly not. You will certainly have a product look by the domain expert before the product goes into the market.
While testing any application you should think like a end-user. But every human being has the limitations and one can’t be the expert in all of the three dimensions mentioned above. (If you are the experts in all of the above skills then please let me know ;-)) So you can’t assure that you can think 100% like how the end-user going to use your application. User who is going to use your application may be having a good understanding of the domain he is working on. You need to balance all these skill activities so that all product aspects will get addressed.
Nowadays you can see the professional being hired in different companies are more domain experts than having technical skills. Current software industry is also seeing a good trend that many professional developers and domain experts are moving into software testing.
We can observe one more reason why domain experts are most wanted! When you hire fresh engineers who are just out of college you cannot expect them to compete with the experienced professionals. Why? Because experienced professional certainly have the advantage of domain and testing experience and they have better understandings of different issues and can deliver the application better and faster.
Here are some of the examples where you can see the distinct edge of domain knowledge: 1) Mobile application testing.2) Wireless application testing3) VoIP applications4) Protocol testing5) Banking applications6) Network testing
How will you test such applications without knowledge of specific domain? Are you going to test the BFSI applications (Banking, Financial Services and Insurance) just for UI or functionality or security or load or stress? You should know what are the user requirements in banking, working procedures, commerce background, exposure to brokerage etc and should test application accordingly, then only you can say that your testing is enough - Here comes the need of subject-matter experts.
Let’s take example of my current project: I am currently working on search engine application. Where I need to know the basic of search engine terminologies and concepts. Many times I see some other team tester’s asking me questions like what is ‘publishers’ and ‘advertisers’, what is the difference and what they do? Do you think they can test the application based on current online advertising and SEO? Certainly not. Unless and until they get well familiar with these terminologies and functionalities.
When I know the functional domain better I can better write and execute more test cases and can effectively simulate the end user actions which is distinctly a big advantage.
Here is the big list of the required testing knowledge:
Testing skill
Bug hunting skill
Technical skill
Domain knowledge
Communication skill
Automation skill
Some programming skill
Quick grasping
Ability to Work under pressure …
That is going to be a huge list. So you will certainly say, do I need to have these many skills? Its’ depends on you. You can stick to one skill or can be expert in one skill and have good understanding of other skills or balanced approach of all the skills. This is the competitive market and you should definitely take advantage of it. Make sure to be expert in at least one domain before making any move.
What if you don’t have enough domain knowledge?You will be posted on any project and company can assign any work to you. Then what if you don’t have enough domain knowledge of that project? You need to quickly grasp as many concepts as you can. Try to understand the product as if you are the customer and what customer will do with application. Visit the customer site if possible know how they work with the product, Read online resources about the domain you want to test the application, participate in events addressing on such domain, meet the domain experts. Or either company will provide all this in-house training before assigning any domain specific task to testers.
There is no specific stage where you need this domain knowledge. You need to apply your domain knowledge in each and every software testing life cycle.
First of all I would like to introduce three dimensional testing career mentioned by Danny R. Faught. There are three categories of skill that need to be judged before hiring any software tester. What are those three skill categories?1) Testing skill2) Domain knowledge3) Technical expertise.
No doubt that any tester should have the basic testing skills like Manual testing and Automation testing. Tester having the common sense can even find most of the obvious bugs in the software. Then would you say that this much testing is sufficient? Would you release the product on the basis of this much testing done? Certainly not. You will certainly have a product look by the domain expert before the product goes into the market.
While testing any application you should think like a end-user. But every human being has the limitations and one can’t be the expert in all of the three dimensions mentioned above. (If you are the experts in all of the above skills then please let me know ;-)) So you can’t assure that you can think 100% like how the end-user going to use your application. User who is going to use your application may be having a good understanding of the domain he is working on. You need to balance all these skill activities so that all product aspects will get addressed.
Nowadays you can see the professional being hired in different companies are more domain experts than having technical skills. Current software industry is also seeing a good trend that many professional developers and domain experts are moving into software testing.
We can observe one more reason why domain experts are most wanted! When you hire fresh engineers who are just out of college you cannot expect them to compete with the experienced professionals. Why? Because experienced professional certainly have the advantage of domain and testing experience and they have better understandings of different issues and can deliver the application better and faster.
Here are some of the examples where you can see the distinct edge of domain knowledge: 1) Mobile application testing.2) Wireless application testing3) VoIP applications4) Protocol testing5) Banking applications6) Network testing
How will you test such applications without knowledge of specific domain? Are you going to test the BFSI applications (Banking, Financial Services and Insurance) just for UI or functionality or security or load or stress? You should know what are the user requirements in banking, working procedures, commerce background, exposure to brokerage etc and should test application accordingly, then only you can say that your testing is enough - Here comes the need of subject-matter experts.
Let’s take example of my current project: I am currently working on search engine application. Where I need to know the basic of search engine terminologies and concepts. Many times I see some other team tester’s asking me questions like what is ‘publishers’ and ‘advertisers’, what is the difference and what they do? Do you think they can test the application based on current online advertising and SEO? Certainly not. Unless and until they get well familiar with these terminologies and functionalities.
When I know the functional domain better I can better write and execute more test cases and can effectively simulate the end user actions which is distinctly a big advantage.
Here is the big list of the required testing knowledge:
Testing skill
Bug hunting skill
Technical skill
Domain knowledge
Communication skill
Automation skill
Some programming skill
Quick grasping
Ability to Work under pressure …
That is going to be a huge list. So you will certainly say, do I need to have these many skills? Its’ depends on you. You can stick to one skill or can be expert in one skill and have good understanding of other skills or balanced approach of all the skills. This is the competitive market and you should definitely take advantage of it. Make sure to be expert in at least one domain before making any move.
What if you don’t have enough domain knowledge?You will be posted on any project and company can assign any work to you. Then what if you don’t have enough domain knowledge of that project? You need to quickly grasp as many concepts as you can. Try to understand the product as if you are the customer and what customer will do with application. Visit the customer site if possible know how they work with the product, Read online resources about the domain you want to test the application, participate in events addressing on such domain, meet the domain experts. Or either company will provide all this in-house training before assigning any domain specific task to testers.
There is no specific stage where you need this domain knowledge. You need to apply your domain knowledge in each and every software testing life cycle.
What is actual testing process in practical or company environment?
What is Cookie?Cookie is small information stored in text file on user’s hard drive by web server. This information is later used by web browser to retrieve information from that machine. Generally cookie contains personalized user data or information that is used to communicate between different web pages.
Why Cookies are used?Cookies are nothing but the user’s identity and used to track where the user navigated throughout the web site pages. The communication between web browser and web server is stateless.
For example if you are accessing domain http://www.example.com/1.html then web browser will simply query to example.com web server for the page 1.html. Next time if you type page as http://www.example.com/2.html then new request is send to example.com web server for sending 2.html page and web server don’t know anything about to whom the previous page 1.html served.
What if you want the previous history of this user communication with the web server? You need to maintain the user state and interaction between web browser and web server somewhere. This is where cookie comes into picture. Cookies serve the purpose of maintaining the user interactions with web server.
How cookies work?The HTTP protocol used to exchange information files on the web is used to maintain the cookies. There are two types of HTTP protocol. Stateless HTTP and Stateful HTTP protocol. Stateless HTTP protocol does not keep any record of previously accessed web page history. While Stateful HTTP protocol do keep some history of previous web browser and web server interactions and this protocol is used by cookies to maintain the user interactions.
Whenever user visits the site or page that is using cookie, small code inside that HTML page (Generally a call to some language script to write the cookie like cookies in JAVAScript, PHP, Perl) writes a text file on users machine called cookie.Here is one example of the code that is used to write cookie and can be placed inside any HTML page:
Set-Cookie: NAME=VALUE; expires=DATE; path=PATH; domain=DOMAIN_NAME;
When user visits the same page or domain later time this cookie is read from disk and used to identify the second visit of the same user on that domain. Expiration time is set while writing the cookie. This time is decided by the application that is going to use the cookie.
Generally two types of cookies are written on user machine.
1) Session cookies: This cookie is active till the browser that invoked the cookie is open. When we close the browser this session cookie gets deleted. Some time session of say 20 minutes can be set to expire the cookie.2) Persistent cookies: The cookies that are written permanently on user machine and lasts for months or years.
Where cookies are stored?When any web page application writes cookie it get saved in a text file on user hard disk drive. The path where the cookies get stored depends on the browser. Different browsers store cookie in different paths. E.g. Internet explorer store cookies on path “C:\Documents and Settings\Default User\Cookies”Here the “Default User” can be replaced by the current user you logged in as. Like “Administrator”, or user name like “Vijay” etc.The cookie path can be easily found by navigating through the browser options. In Mozilla Firefox browser you can even see the cookies in browser options itself. Open the Mozila browser, click on Tools->Options->Privacy and then “Show cookies” button.
How cookies are stored?Lets take example of cookie written by rediff.com on Mozilla Firefox browser:On Mozilla Firefox browser when you open the page rediff.com or login to your rediffmail account, a cookie will get written on your Hard disk. To view this cookie simply click on “Show cookies” button mentioned on above path. Click on Rediff.com site under this cookie list. You can see different cookies written by rediff domain with different names.
Site: Rediff.com Cookie name: RMIDName: RMID (Name of the cookie)Content: 1d11c8ec44bf49e0… (Encrypted content)Domain: .rediff.comPath: / (Any path after the domain name)Send For: Any type of connectionExpires: Thursday, December 31, 2020 11:59:59 PM
Applications where cookies can be used:
1) To implement shopping cart:Cookies are used for maintaining online ordering system. Cookies remember what user wants to buy. What if user adds some products in their shopping cart and if due to some reason user don’t want to buy those products this time and closes the browser window? When next time same user visits the purchase page he can see all the products he added in shopping cart in his last visit.
2) Personalized sites:When user visits certain pages they are asked which pages they don’t want to visit or display. User options are get stored in cookie and till the user is online, those pages are not shown to him.
3) User tracking: To track number of unique visitors online at particular time.
4) Marketing:Some companies use cookies to display advertisements on user machines. Cookies control these advertisements. When and which advertisement should be shown? What is the interest of the user? Which keywords he searches on the site? All these things can be maintained using cookies.
5) User sessions:Cookies can track user sessions to particular domain using user ID and password.
Drawbacks of cookies:
1) Even writing Cookie is a great way to maintain user interaction, if user has set browser options to warn before writing any cookie or disabled the cookies completely then site containing cookie will be completely disabled and can not perform any operation resulting in loss of site traffic.
2) Too many Cookies:If you are writing too many cookies on every page navigation and if user has turned on option to warn before writing cookie, this could turn away user from your site.
3) Security issues:Some times users personal information is stored in cookies and if someone hack the cookie then hacker can get access to your personal information. Even corrupted cookies can be read by different domains and lead to security issues.
4) Sensitive information:Some sites may write and store your sensitive information in cookies, which should not be allowed due to privacy concerns.
This should be enough to know what cookies are. If you want more cookie info see Cookie Central page.
Some Major Test cases for web application cookie testing:
The first obvious test case is to test if your application is writing cookies properly on disk. You can use the Cookie Tester application also if you don’t have any web application to test but you want to understand the cookie concept for testing.
Test cases:
1) As a Cookie privacy policy make sure from your design documents that no personal or sensitive data is stored in the cookie.
2) If you have no option than saving sensitive data in cookie make sure data stored in cookie is stored in encrypted format.
3) Make sure that there is no overuse of cookies on your site under test. Overuse of cookies will annoy users if browser is prompting for cookies more often and this could result in loss of site traffic and eventually loss of business.
4) Disable the cookies from your browser settings: If you are using cookies on your site, your sites major functionality will not work by disabling the cookies. Then try to access the web site under test. Navigate through the site. See if appropriate messages are displayed to user like “For smooth functioning of this site make sure that cookies are enabled on your browser”. There should not be any page crash due to disabling the cookies. (Please make sure that you close all browsers, delete all previously written cookies before performing this test)
5) Accepts/Reject some cookies: The best way to check web site functionality is, not to accept all cookies. If you are writing 10 cookies in your web application then randomly accept some cookies say accept 5 and reject 5 cookies. For executing this test case you can set browser options to prompt whenever cookie is being written to disk. On this prompt window you can either accept or reject cookie. Try to access major functionality of web site. See if pages are getting crashed or data is getting corrupted.
6) Delete cookie: Allow site to write the cookies and then close all browsers and manually delete all cookies for web site under test. Access the web pages and check the behavior of the pages.
7) Corrupt the cookies: Corrupting cookie is easy. You know where cookies are stored. Manually edit the cookie in notepad and change the parameters to some vague values. Like alter the cookie content, Name of the cookie or expiry date of the cookie and see the site functionality. In some cases corrupted cookies allow to read the data inside it for any other domain. This should not happen in case of your web site cookies. Note that the cookies written by one domain say rediff.com can’t be accessed by other domain say yahoo.com unless and until the cookies are corrupted and someone trying to hack the cookie data.
8 ) Checking the deletion of cookies from your web application page: Some times cookie written by domain say rediff.com may be deleted by same domain but by different page under that domain. This is the general case if you are testing some ‘action tracking’ web portal. Action tracking or purchase tracking pixel is placed on the action web page and when any action or purchase occurs by user the cookie written on disk get deleted to avoid multiple action logging from same cookie. Check if reaching to your action or purchase page deletes the cookie properly and no more invalid actions or purchase get logged from same user.
9) Cookie Testing on Multiple browsers: This is the important case to check if your web application page is writing the cookies properly on different browsers as intended and site works properly using these cookies. You can test your web application on Major used browsers like Internet explorer (Various versions), Mozilla Firefox, Netscape, Opera etc.
10) If your web application is using cookies to maintain the logging state of any user then log in to your web application using some username and password. In many cases you can see the logged in user ID parameter directly in browser address bar. Change this parameter to different value say if previous user ID is 100 then make it 101 and press enter. The proper access message should be displayed to user and user should not be able to see other users account.
These are some Major test cases to be considered while testing website cookies. You can write multiple test cases from these test cases by performing various combinations. If you have some different application scenario, you can mention your test cases in comments below.
Why Cookies are used?Cookies are nothing but the user’s identity and used to track where the user navigated throughout the web site pages. The communication between web browser and web server is stateless.
For example if you are accessing domain http://www.example.com/1.html then web browser will simply query to example.com web server for the page 1.html. Next time if you type page as http://www.example.com/2.html then new request is send to example.com web server for sending 2.html page and web server don’t know anything about to whom the previous page 1.html served.
What if you want the previous history of this user communication with the web server? You need to maintain the user state and interaction between web browser and web server somewhere. This is where cookie comes into picture. Cookies serve the purpose of maintaining the user interactions with web server.
How cookies work?The HTTP protocol used to exchange information files on the web is used to maintain the cookies. There are two types of HTTP protocol. Stateless HTTP and Stateful HTTP protocol. Stateless HTTP protocol does not keep any record of previously accessed web page history. While Stateful HTTP protocol do keep some history of previous web browser and web server interactions and this protocol is used by cookies to maintain the user interactions.
Whenever user visits the site or page that is using cookie, small code inside that HTML page (Generally a call to some language script to write the cookie like cookies in JAVAScript, PHP, Perl) writes a text file on users machine called cookie.Here is one example of the code that is used to write cookie and can be placed inside any HTML page:
Set-Cookie: NAME=VALUE; expires=DATE; path=PATH; domain=DOMAIN_NAME;
When user visits the same page or domain later time this cookie is read from disk and used to identify the second visit of the same user on that domain. Expiration time is set while writing the cookie. This time is decided by the application that is going to use the cookie.
Generally two types of cookies are written on user machine.
1) Session cookies: This cookie is active till the browser that invoked the cookie is open. When we close the browser this session cookie gets deleted. Some time session of say 20 minutes can be set to expire the cookie.2) Persistent cookies: The cookies that are written permanently on user machine and lasts for months or years.
Where cookies are stored?When any web page application writes cookie it get saved in a text file on user hard disk drive. The path where the cookies get stored depends on the browser. Different browsers store cookie in different paths. E.g. Internet explorer store cookies on path “C:\Documents and Settings\Default User\Cookies”Here the “Default User” can be replaced by the current user you logged in as. Like “Administrator”, or user name like “Vijay” etc.The cookie path can be easily found by navigating through the browser options. In Mozilla Firefox browser you can even see the cookies in browser options itself. Open the Mozila browser, click on Tools->Options->Privacy and then “Show cookies” button.
How cookies are stored?Lets take example of cookie written by rediff.com on Mozilla Firefox browser:On Mozilla Firefox browser when you open the page rediff.com or login to your rediffmail account, a cookie will get written on your Hard disk. To view this cookie simply click on “Show cookies” button mentioned on above path. Click on Rediff.com site under this cookie list. You can see different cookies written by rediff domain with different names.
Site: Rediff.com Cookie name: RMIDName: RMID (Name of the cookie)Content: 1d11c8ec44bf49e0… (Encrypted content)Domain: .rediff.comPath: / (Any path after the domain name)Send For: Any type of connectionExpires: Thursday, December 31, 2020 11:59:59 PM
Applications where cookies can be used:
1) To implement shopping cart:Cookies are used for maintaining online ordering system. Cookies remember what user wants to buy. What if user adds some products in their shopping cart and if due to some reason user don’t want to buy those products this time and closes the browser window? When next time same user visits the purchase page he can see all the products he added in shopping cart in his last visit.
2) Personalized sites:When user visits certain pages they are asked which pages they don’t want to visit or display. User options are get stored in cookie and till the user is online, those pages are not shown to him.
3) User tracking: To track number of unique visitors online at particular time.
4) Marketing:Some companies use cookies to display advertisements on user machines. Cookies control these advertisements. When and which advertisement should be shown? What is the interest of the user? Which keywords he searches on the site? All these things can be maintained using cookies.
5) User sessions:Cookies can track user sessions to particular domain using user ID and password.
Drawbacks of cookies:
1) Even writing Cookie is a great way to maintain user interaction, if user has set browser options to warn before writing any cookie or disabled the cookies completely then site containing cookie will be completely disabled and can not perform any operation resulting in loss of site traffic.
2) Too many Cookies:If you are writing too many cookies on every page navigation and if user has turned on option to warn before writing cookie, this could turn away user from your site.
3) Security issues:Some times users personal information is stored in cookies and if someone hack the cookie then hacker can get access to your personal information. Even corrupted cookies can be read by different domains and lead to security issues.
4) Sensitive information:Some sites may write and store your sensitive information in cookies, which should not be allowed due to privacy concerns.
This should be enough to know what cookies are. If you want more cookie info see Cookie Central page.
Some Major Test cases for web application cookie testing:
The first obvious test case is to test if your application is writing cookies properly on disk. You can use the Cookie Tester application also if you don’t have any web application to test but you want to understand the cookie concept for testing.
Test cases:
1) As a Cookie privacy policy make sure from your design documents that no personal or sensitive data is stored in the cookie.
2) If you have no option than saving sensitive data in cookie make sure data stored in cookie is stored in encrypted format.
3) Make sure that there is no overuse of cookies on your site under test. Overuse of cookies will annoy users if browser is prompting for cookies more often and this could result in loss of site traffic and eventually loss of business.
4) Disable the cookies from your browser settings: If you are using cookies on your site, your sites major functionality will not work by disabling the cookies. Then try to access the web site under test. Navigate through the site. See if appropriate messages are displayed to user like “For smooth functioning of this site make sure that cookies are enabled on your browser”. There should not be any page crash due to disabling the cookies. (Please make sure that you close all browsers, delete all previously written cookies before performing this test)
5) Accepts/Reject some cookies: The best way to check web site functionality is, not to accept all cookies. If you are writing 10 cookies in your web application then randomly accept some cookies say accept 5 and reject 5 cookies. For executing this test case you can set browser options to prompt whenever cookie is being written to disk. On this prompt window you can either accept or reject cookie. Try to access major functionality of web site. See if pages are getting crashed or data is getting corrupted.
6) Delete cookie: Allow site to write the cookies and then close all browsers and manually delete all cookies for web site under test. Access the web pages and check the behavior of the pages.
7) Corrupt the cookies: Corrupting cookie is easy. You know where cookies are stored. Manually edit the cookie in notepad and change the parameters to some vague values. Like alter the cookie content, Name of the cookie or expiry date of the cookie and see the site functionality. In some cases corrupted cookies allow to read the data inside it for any other domain. This should not happen in case of your web site cookies. Note that the cookies written by one domain say rediff.com can’t be accessed by other domain say yahoo.com unless and until the cookies are corrupted and someone trying to hack the cookie data.
8 ) Checking the deletion of cookies from your web application page: Some times cookie written by domain say rediff.com may be deleted by same domain but by different page under that domain. This is the general case if you are testing some ‘action tracking’ web portal. Action tracking or purchase tracking pixel is placed on the action web page and when any action or purchase occurs by user the cookie written on disk get deleted to avoid multiple action logging from same cookie. Check if reaching to your action or purchase page deletes the cookie properly and no more invalid actions or purchase get logged from same user.
9) Cookie Testing on Multiple browsers: This is the important case to check if your web application page is writing the cookies properly on different browsers as intended and site works properly using these cookies. You can test your web application on Major used browsers like Internet explorer (Various versions), Mozilla Firefox, Netscape, Opera etc.
10) If your web application is using cookies to maintain the logging state of any user then log in to your web application using some username and password. In many cases you can see the logged in user ID parameter directly in browser address bar. Change this parameter to different value say if previous user ID is 100 then make it 101 and press enter. The proper access message should be displayed to user and user should not be able to see other users account.
These are some Major test cases to be considered while testing website cookies. You can write multiple test cases from these test cases by performing various combinations. If you have some different application scenario, you can mention your test cases in comments below.
Subscribe to:
Posts (Atom)