Why Automation testing?1) You have some new releases and bug fixes in working module. So how will you ensure that the new bug fixes have not introduced any new bug in previous working functionality? You need to test the previous functionality also. So will you test manually all the module functionality every time you have some bug fixes or new functionality addition? Well you might do it manually but then you are not doing testing effectively. Effective in terms of company cost, resources, Time etc. Here comes need of Automation.- So automate your testing procedure when you have lot of regression work.
2) You are testing a web application where there might be thousands of users interacting with your application simultaneously. How will you test such a web application? How will you create those many users manually and simultaneously? Well very difficult task if done manually.- Automate your load testing work for creating virtual users to check load capacity of your application.
3) You are testing application where code is changing frequently. You have almost same GUI but functional changes are more so testing rework is more.- Automate your testing work when your GUI is almost frozen but you have lot of frequently functional changes.
What are the Risks associated in Automation Testing?There are some distinct situations where you can think of automating your testing work. I have covered some risks of automation testing here. If you have taken decision of automation or are going to take sooner then think of following scenarios first.
1) Do you have skilled resources? For automation you need to have persons having some programming knowledge. Think of your resources. Do they have sufficient programming knowledge for automation testing? If not do they have technical capabilities or programming background that they can easily adapt to the new technologies? Are you going to invest money to build a good automation team? If your answer is yes then only think to automate your work.
2) Initial cost for Automation is very high:I agree that manual testing has too much cost associated to hire skilled manual testers. And if you are thinking automation will be the solution for you, Think twice. Automation cost is too high for initial setup i.e. cost associated to automation tool purchase, training and maintenance of test scripts is very high.There are many unsatisfied customers regretting on their decision to automate their work. If you are spending too much and getting merely some good looking testing tools and some basic automation scripts then what is the use of automation?
3) Do not think to automate your UI if it is not fixed:Beware before automating user interface. If user interface is changing extensively, cost associated with script maintenance will be very high. Basic UI automation is sufficient in such cases.
4) Is your application is stable enough to automate further testing work?It would be bad idea to automate testing work in early development cycle (Unless it is agile environment). Script maintenance cost will be very high in such cases.
5) Are you thinking of 100% automation?Please stop dreaming. You cannot 100% automate your testing work. Certainly you have areas like performance testing, regression testing, load/stress testing where you can have chance of reaching near to 100% automation. Areas like User interface, documentation, installation, compatibility and recovery where testing must be done manually.
6) Do not automate tests that run once:Identify application areas and test cases that might be running once and not included in regression. Avoid automating such modules or test cases.
7) Will your automation suite be having long lifetime?Every automation script suite should have enough life time that its building cost should be definitely less than that of manual execution cost. This is bit difficult to analyze the effective cost of each automation script suite. Approximately your automation suite should be used or run at least 15 to 20 times for separate builds (General assumption. depends on specific application complexity) to have good ROI.
Here is the conclusion:
Automation testing is the best way to accomplish most of the testing goals and effective use of resources and time. But you should be cautious before choosing the automation tool. Be sure to have skilled staff before deciding to automate your testing work. Otherwise your tool will remain on the shelf giving you no ROI. Handing over the expensive automation tools to unskilled staff will lead to frustration. Before purchasing the automation tools make sure that tool is a best fit to your requirements. You cannot have the tool that will 100% match with your requirements. So find out the limitations of the tool that is best match with your requirements and then use manual testing techniques to overcome those testing tool limitations. Open source tool is also a good option to start with automation. To know more on choosing automation tools read my previous posts here and here.
Instead of relying 100% on either manual or automation use the best combination of manual and automation testing. This is the best solution (I think) for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.
Monday, March 24, 2008
Reader Tips
James Bach - The consulting software Tester
Michael Bolton - Blog on Software Testing and Quality observations
Matt Heusser - Thoughts on life, Software development and Testing
Shrini Kulkarni - A Thinking Tester
Pradeep Soundarajan - Critical thoughts on Software Testing
Erkan Yilmaz - Good stories worth to read
Jeff Fry - Thoughts on the craft of software testing
Jonathan Kohl - Software Investigator
Cem Kaner - Blog on craft and community of software testing
Elisabeth Hendrickson - Agile Testing Guru
Debasis Pradhan - The bug hunter
Rahul Verma - Testing Perspective
Michael Hunter - The guy making developers cry since 1995
Scott Barber Better Testing, Better results
Michael Bolton - Blog on Software Testing and Quality observations
Matt Heusser - Thoughts on life, Software development and Testing
Shrini Kulkarni - A Thinking Tester
Pradeep Soundarajan - Critical thoughts on Software Testing
Erkan Yilmaz - Good stories worth to read
Jeff Fry - Thoughts on the craft of software testing
Jonathan Kohl - Software Investigator
Cem Kaner - Blog on craft and community of software testing
Elisabeth Hendrickson - Agile Testing Guru
Debasis Pradhan - The bug hunter
Rahul Verma - Testing Perspective
Michael Hunter - The guy making developers cry since 1995
Scott Barber Better Testing, Better results
Web Testing: Complete guide on testing web applications
In my previous post I have outlined points to be considered while testing web applications. Here we will see some more details on web application testing with web testing test cases. Let me tell you one thing that I always like to share practical knowledge, which can be useful to users in their career life. This is a quite long article so sit back and get relaxed to get most out of it.
Let’s have first web testing checklist.1) Functionality Testing2) Usability testing3) Interface testing4) Compatibility testing5) Performance testing6) Security testing
1) Functionality Testing:
Test for - all the links in web pages, database connection, forms used in the web pages for submitting or getting information from user, Cookie testing.
Check all the links:
Test the outgoing links from all the pages from specific domain under test.
Test all internal links.
Test links jumping on the same pages.
Test links used to send the email to admin or other users from web pages.
Test to check if there are any orphan pages.
Lastly in link checking, check for broken links in all above-mentioned links.
Test forms in all pages:Forms are the integral part of any web site. Forms are used to get information from users and to keep interaction with them. So what should be checked on these forms?
First check all the validations on each field.
Check for the default values of fields.
Wrong inputs to the fields in the forms.
Options to create forms if any, form delete, view or modify the forms.
Let’s take example of the search engine project currently I am working on, In this project we have advertiser and affiliate signup steps. Each sign up step is different but dependent on other steps. So sign up flow should get executed correctly. There are different field validations like email Ids, User financial info validations. All these validations should get checked in manual or automated web testing.
Cookies testing:Cookies are small files stored on user machine. These are basically used to maintain the session mainly login sessions. Test the application by enabling or disabling the cookies in your browser options. Test if the cookies are encrypted before writing to user machine. If you are testing the session cookies (i.e. cookies expire after the sessions ends) check for login sessions and user stats after session end. Check effect on application security by deleting the cookies. (I will soon write separate article on cookie testing)
Validate your HTML/CSS:If you are optimizing your site for Search engines then HTML/CSS validation is very important. Mainly validate the site for HTML syntax errors. Check if site is crawlable to different search engines.
Database testing:Data consistency is very important in web application. Check for data integrity and errors while you edit, delete, modify the forms or do any DB related functionality.Check if all the database queries are executing correctly, data is retrieved correctly and also updated correctly. More on database testing could be load on DB, we will address this in web load or performance testing below.
2) Usability Testing:
Test for navigation:Navigation means how the user surfs the web pages, different controls like buttons, boxes or how user using the links on the pages to surf different pages.Usability testing includes:Web site should be easy to use. Instructions should be provided clearly. Check if the provided instructions are correct means whether they satisfy purpose.Main menu should be provided on each page. It should be consistent.
Content checking: Content should be logical and easy to understand. Check for spelling errors. Use of dark colors annoys users and should not be used in site theme. You can follow some standards that are used for web page and content building. These are common accepted standards like as I mentioned above about annoying colors, fonts, frames etc.Content should be meaningful. All the anchor text links should be working properly. Images should be placed properly with proper sizes.These are some basic standards that should be followed in web development. Your task is to validate all for UI testing
Other user information for user help:Like search option, sitemap, help files etc. Sitemap should be present with all the links in web sites with proper tree view of navigation. Check for all links on the sitemap.“Search in the site” option will help users to find content pages they are looking for easily and quickly. These are all optional items and if present should be validated.
3) Interface Testing:The main interfaces are:Web server and application server interfaceApplication server and Database server interface.
Check if all the interactions between these servers are executed properly. Errors are handled properly. If database or web server returns any error message for any query by application server then application server should catch and display these error messages appropriately to users. Check what happens if user interrupts any transaction in-between? Check what happens if connection to web server is reset in between?
4) Compatibility Testing:Compatibility of your web site is very important testing aspect. See which compatibility test to be executed:
Browser compatibility
Operating system compatibility
Mobile browsing
Printing options
Browser compatibility:In my web-testing career I have experienced this as most influencing part on web site testing.Some applications are very dependent on browsers. Different browsers have different configurations and settings that your web page should be compatible with. Your web site coding should be cross browser platform compatible. If you are using java scripts or AJAX calls for UI functionality, performing security checks or validations then give more stress on browser compatibility testing of your web application.Test web application on different browsers like Internet explorer, Firefox, Netscape navigator, AOL, Safari, Opera browsers with different versions.
OS compatibility:Some functionality in your web application is may not be compatible with all operating systems. All new technologies used in web development like graphics designs, interface calls like different API’s may not be available in all Operating Systems.Test your web application on different operating systems like Windows, Unix, MAC, Linux, Solaris with different OS flavors.
Mobile browsing:This is new technology age. So in future Mobile browsing will rock. Test your web pages on mobile browsers. Compatibility issues may be there on mobile.
Printing options:If you are giving page-printing options then make sure fonts, page alignment, page graphics getting printed properly. Pages should be fit to paper size or as per the size mentioned in printing option.
5) Performance testing:Web application should sustain to heavy load. Web performance testing should include:Web Load TestingWeb Stress Testing
Test application performance on different internet connection speed.In web load testing test if many users are accessing or requesting the same page. Can system sustain in peak load times? Site should handle many simultaneous user requests, large input data from users, Simultaneous connection to DB, heavy load on specific pages etc.
Stress testing: Generally stress means stretching the system beyond its specification limits. Web stress testing is performed to break the site by giving stress and checked how system reacts to stress and how system recovers from crashes.Stress is generally given on input fields, login and sign up areas.
In web performance testing web site functionality on different operating systems, different hardware platforms is checked for software, hardware memory leakage errors,
6) Security Testing:
Following are some test cases for web security testing:
Test by pasting internal url directly into browser address bar without login. Internal pages should not open.
If you are logged in using username and password and browsing internal pages then try changing url options directly. I.e. If you are checking some publisher site statistics with publisher site ID= 123. Try directly changing the url site ID parameter to different site ID which is not related to logged in user. Access should denied for this user to view others stats.
Try some invalid inputs in input fields like login username, password, input text boxes. Check the system reaction on all invalid inputs.
Web directories or files should not be accessible directly unless given download option.
Test the CAPTCHA for automates scripts logins.
Test if SSL is used for security measures. If used proper message should get displayed when user switch from non-secure http:// pages to secure https:// pages and vice versa.
All transactions, error messages, security breach attempts should get logged in log files somewhere on web server.
Let’s have first web testing checklist.1) Functionality Testing2) Usability testing3) Interface testing4) Compatibility testing5) Performance testing6) Security testing
1) Functionality Testing:
Test for - all the links in web pages, database connection, forms used in the web pages for submitting or getting information from user, Cookie testing.
Check all the links:
Test the outgoing links from all the pages from specific domain under test.
Test all internal links.
Test links jumping on the same pages.
Test links used to send the email to admin or other users from web pages.
Test to check if there are any orphan pages.
Lastly in link checking, check for broken links in all above-mentioned links.
Test forms in all pages:Forms are the integral part of any web site. Forms are used to get information from users and to keep interaction with them. So what should be checked on these forms?
First check all the validations on each field.
Check for the default values of fields.
Wrong inputs to the fields in the forms.
Options to create forms if any, form delete, view or modify the forms.
Let’s take example of the search engine project currently I am working on, In this project we have advertiser and affiliate signup steps. Each sign up step is different but dependent on other steps. So sign up flow should get executed correctly. There are different field validations like email Ids, User financial info validations. All these validations should get checked in manual or automated web testing.
Cookies testing:Cookies are small files stored on user machine. These are basically used to maintain the session mainly login sessions. Test the application by enabling or disabling the cookies in your browser options. Test if the cookies are encrypted before writing to user machine. If you are testing the session cookies (i.e. cookies expire after the sessions ends) check for login sessions and user stats after session end. Check effect on application security by deleting the cookies. (I will soon write separate article on cookie testing)
Validate your HTML/CSS:If you are optimizing your site for Search engines then HTML/CSS validation is very important. Mainly validate the site for HTML syntax errors. Check if site is crawlable to different search engines.
Database testing:Data consistency is very important in web application. Check for data integrity and errors while you edit, delete, modify the forms or do any DB related functionality.Check if all the database queries are executing correctly, data is retrieved correctly and also updated correctly. More on database testing could be load on DB, we will address this in web load or performance testing below.
2) Usability Testing:
Test for navigation:Navigation means how the user surfs the web pages, different controls like buttons, boxes or how user using the links on the pages to surf different pages.Usability testing includes:Web site should be easy to use. Instructions should be provided clearly. Check if the provided instructions are correct means whether they satisfy purpose.Main menu should be provided on each page. It should be consistent.
Content checking: Content should be logical and easy to understand. Check for spelling errors. Use of dark colors annoys users and should not be used in site theme. You can follow some standards that are used for web page and content building. These are common accepted standards like as I mentioned above about annoying colors, fonts, frames etc.Content should be meaningful. All the anchor text links should be working properly. Images should be placed properly with proper sizes.These are some basic standards that should be followed in web development. Your task is to validate all for UI testing
Other user information for user help:Like search option, sitemap, help files etc. Sitemap should be present with all the links in web sites with proper tree view of navigation. Check for all links on the sitemap.“Search in the site” option will help users to find content pages they are looking for easily and quickly. These are all optional items and if present should be validated.
3) Interface Testing:The main interfaces are:Web server and application server interfaceApplication server and Database server interface.
Check if all the interactions between these servers are executed properly. Errors are handled properly. If database or web server returns any error message for any query by application server then application server should catch and display these error messages appropriately to users. Check what happens if user interrupts any transaction in-between? Check what happens if connection to web server is reset in between?
4) Compatibility Testing:Compatibility of your web site is very important testing aspect. See which compatibility test to be executed:
Browser compatibility
Operating system compatibility
Mobile browsing
Printing options
Browser compatibility:In my web-testing career I have experienced this as most influencing part on web site testing.Some applications are very dependent on browsers. Different browsers have different configurations and settings that your web page should be compatible with. Your web site coding should be cross browser platform compatible. If you are using java scripts or AJAX calls for UI functionality, performing security checks or validations then give more stress on browser compatibility testing of your web application.Test web application on different browsers like Internet explorer, Firefox, Netscape navigator, AOL, Safari, Opera browsers with different versions.
OS compatibility:Some functionality in your web application is may not be compatible with all operating systems. All new technologies used in web development like graphics designs, interface calls like different API’s may not be available in all Operating Systems.Test your web application on different operating systems like Windows, Unix, MAC, Linux, Solaris with different OS flavors.
Mobile browsing:This is new technology age. So in future Mobile browsing will rock. Test your web pages on mobile browsers. Compatibility issues may be there on mobile.
Printing options:If you are giving page-printing options then make sure fonts, page alignment, page graphics getting printed properly. Pages should be fit to paper size or as per the size mentioned in printing option.
5) Performance testing:Web application should sustain to heavy load. Web performance testing should include:Web Load TestingWeb Stress Testing
Test application performance on different internet connection speed.In web load testing test if many users are accessing or requesting the same page. Can system sustain in peak load times? Site should handle many simultaneous user requests, large input data from users, Simultaneous connection to DB, heavy load on specific pages etc.
Stress testing: Generally stress means stretching the system beyond its specification limits. Web stress testing is performed to break the site by giving stress and checked how system reacts to stress and how system recovers from crashes.Stress is generally given on input fields, login and sign up areas.
In web performance testing web site functionality on different operating systems, different hardware platforms is checked for software, hardware memory leakage errors,
6) Security Testing:
Following are some test cases for web security testing:
Test by pasting internal url directly into browser address bar without login. Internal pages should not open.
If you are logged in using username and password and browsing internal pages then try changing url options directly. I.e. If you are checking some publisher site statistics with publisher site ID= 123. Try directly changing the url site ID parameter to different site ID which is not related to logged in user. Access should denied for this user to view others stats.
Try some invalid inputs in input fields like login username, password, input text boxes. Check the system reaction on all invalid inputs.
Web directories or files should not be accessible directly unless given download option.
Test the CAPTCHA for automates scripts logins.
Test if SSL is used for security measures. If used proper message should get displayed when user switch from non-secure http:// pages to secure https:// pages and vice versa.
All transactions, error messages, security breach attempts should get logged in log files somewhere on web server.
Winrunner questions from Interviewer point of view
Explain WinRunner testing process?WinRunner testing process involves six main stagesi. Create GUI Map File so that WinRunner can recognize the GUI objects in the application being testedii. Create test scripts by recording, programming, or a combination of both. While recording tests, insert checkpoints where you want to check the response of the application being tested.iii. Debug Test: run tests in Debug mode to make sure they run smoothlyiv. Run Tests: run tests in Verify mode to test your application.v. View Results: determines the success or failure of the tests.vi. Report Defects: If a test run fails due to a defect in the application being tested, you can report information about the defect directly from the Test Results window.
What is contained in the GUI map?- WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
- There are 2 types of GUI Map files.i. Global GUI Map file: a single GUI Map file for the entire applicationii. GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
How does WinRunner recognize objects on the application?WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.
Have you created test scripts and what is contained in the test scripts?Yes I have created test scripts. It contains the statement in Mercury Interactive’s Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using WinRunner’s visual programming tool, the Function Generator.
How does WinRunner evaluates test results?Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.
Have you performed debugging of the scripts?Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.
How do you run your test scripts?We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results. 1. Where did you learn WinRunner and TestDirector?
If they say it was in a mercury class, ask if they can show you theircertificate of completion. If they say no, let them know you will verify itwith Mercury.
2. Have you ever created a start-up script?
If they answer “yes”, ask themwhat was in it and how they got WR to execute the start-up script when WR isinvoked.
They should answer something like this, “In the start-up script, we loaded allthe gui maps for the application, any library files we needed, and any customgui object mapping we have to do. We also loaded global variables and systemvariables here. The startup script location is added to the WR .ini file,wrun.ini located under the windows or winnt directory as wrun.ini”
3. What is the difference between writing a function and writing a script?
They should mention some of these things:1. A function goes into a “compiled module”, a script does not.2. A function follows strict “c” syntax. For instance, you have to declareall variables created and used in the function. In a script you do not.
4. What is the difference between “set_window” and “win_activate”.?When would you use “set_window” and when would you use “win_activate”?win_activate has the format win_activate(window);. The win_activate functionmakes the specified window the active window by bringing it into focus andraising it to the top of the display. (It is the equivalent to clicking on thewindow banner)
Set_window has the following format: set_window(window,[time]);The set_window function directs input to the correct application window. Thisdirects the GUI map to this window. It also sets the scope for objectidentification in the GUI map.
The most important difference is that set_window has a timing option.WinRunner will wait a maximum of the number used in the function, PLUS thesystem set timeout, to wait for the window to appear. Win_activate assumes thewindow is already on the desktop and has no timing option.
What is contained in the GUI map?- WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
- There are 2 types of GUI Map files.i. Global GUI Map file: a single GUI Map file for the entire applicationii. GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
How does WinRunner recognize objects on the application?WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.
Have you created test scripts and what is contained in the test scripts?Yes I have created test scripts. It contains the statement in Mercury Interactive’s Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using WinRunner’s visual programming tool, the Function Generator.
How does WinRunner evaluates test results?Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.
Have you performed debugging of the scripts?Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.
How do you run your test scripts?We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results. 1. Where did you learn WinRunner and TestDirector?
If they say it was in a mercury class, ask if they can show you theircertificate of completion. If they say no, let them know you will verify itwith Mercury.
2. Have you ever created a start-up script?
If they answer “yes”, ask themwhat was in it and how they got WR to execute the start-up script when WR isinvoked.
They should answer something like this, “In the start-up script, we loaded allthe gui maps for the application, any library files we needed, and any customgui object mapping we have to do. We also loaded global variables and systemvariables here. The startup script location is added to the WR .ini file,wrun.ini located under the windows or winnt directory as wrun.ini”
3. What is the difference between writing a function and writing a script?
They should mention some of these things:1. A function goes into a “compiled module”, a script does not.2. A function follows strict “c” syntax. For instance, you have to declareall variables created and used in the function. In a script you do not.
4. What is the difference between “set_window” and “win_activate”.?When would you use “set_window” and when would you use “win_activate”?win_activate has the format win_activate(window);. The win_activate functionmakes the specified window the active window by bringing it into focus andraising it to the top of the display. (It is the equivalent to clicking on thewindow banner)
Set_window has the following format: set_window(window,[time]);The set_window function directs input to the correct application window. Thisdirects the GUI map to this window. It also sets the scope for objectidentification in the GUI map.
The most important difference is that set_window has a timing option.WinRunner will wait a maximum of the number used in the function, PLUS thesystem set timeout, to wait for the window to appear. Win_activate assumes thewindow is already on the desktop and has no timing option.
Regression Testing with Regression Testing Tools and methods
What is Regression Software Testing?Regression means retesting the unchanged parts of the application. Test cases are re-executed in order to check whether previous functionality of application is working fine and new changes have not introduced any new bugs.
This is the method of verification. Verifying that the bugs are fixed and the newly added feature have not created in problem in previous working version of software.
Why regression Testing?Regression testing is initiated when programmer fix any bug or add new code for new functionality to the system. It is a quality measure to check that new code complies with old code and unmodified code is not getting affected.Most of the time testing team has task to check the last minute changes in the system. In such situation testing only affected application area in necessary to complete the testing process in time with covering all major system aspects.
How much regression testing?This depends on the scope of new added feature. If the scope of the fix or feature is large then the application area getting affected is quite large and testing should be thoroughly including all the application test cases. But this can be effectively decided when tester gets input from developer about the scope, nature and amount of change.
What we do in regression testing?
Rerunning the previously conducted tests
Comparing current results with previously executed test results.
Regression Testing Tools:Automated Regression testing is the testing area where we can automate most of the testing efforts. We run all the previously executed test cases this means we have test case set available and running these test cases manually is time consuming. We know the expected results so automating these test cases is time saving and efficient regression testing method. Extent of automation depends on the number of test cases that are going to remain applicable over the time. If test cases are varying time to time as application scope goes on increasing then automation of regression procedure will be the waste of time.
Most of the regression testing tools are record and playback type. Means you will record the test cases by navigating through the AUT and verify whether expected results are coming or not.Example regression testing tools are:
Winrunner
QTP
AdventNet QEngine
Regression Tester
vTest
Watir
Selenium
actiWate
Rational Functional Tester
SilkTest
Most of the tools are both Functional as well as regression testing tools.
Regression Testing Of GUI application:It is difficult to perform GUI(Graphical User Interface) regression testing when GUI structure is modified. The test cases written on old GUI either becomes obsolete or need to reuse. Reusing the regression testing test cases means GUI test cases are modified according to new GUI. But this task becomes cumbersome if you have large set of GUI test cases.
This is the method of verification. Verifying that the bugs are fixed and the newly added feature have not created in problem in previous working version of software.
Why regression Testing?Regression testing is initiated when programmer fix any bug or add new code for new functionality to the system. It is a quality measure to check that new code complies with old code and unmodified code is not getting affected.Most of the time testing team has task to check the last minute changes in the system. In such situation testing only affected application area in necessary to complete the testing process in time with covering all major system aspects.
How much regression testing?This depends on the scope of new added feature. If the scope of the fix or feature is large then the application area getting affected is quite large and testing should be thoroughly including all the application test cases. But this can be effectively decided when tester gets input from developer about the scope, nature and amount of change.
What we do in regression testing?
Rerunning the previously conducted tests
Comparing current results with previously executed test results.
Regression Testing Tools:Automated Regression testing is the testing area where we can automate most of the testing efforts. We run all the previously executed test cases this means we have test case set available and running these test cases manually is time consuming. We know the expected results so automating these test cases is time saving and efficient regression testing method. Extent of automation depends on the number of test cases that are going to remain applicable over the time. If test cases are varying time to time as application scope goes on increasing then automation of regression procedure will be the waste of time.
Most of the regression testing tools are record and playback type. Means you will record the test cases by navigating through the AUT and verify whether expected results are coming or not.Example regression testing tools are:
Winrunner
QTP
AdventNet QEngine
Regression Tester
vTest
Watir
Selenium
actiWate
Rational Functional Tester
SilkTest
Most of the tools are both Functional as well as regression testing tools.
Regression Testing Of GUI application:It is difficult to perform GUI(Graphical User Interface) regression testing when GUI structure is modified. The test cases written on old GUI either becomes obsolete or need to reuse. Reusing the regression testing test cases means GUI test cases are modified according to new GUI. But this task becomes cumbersome if you have large set of GUI test cases.
How to write effective Test cases, procedures and definitions
Writing effective test cases is a skill and that can be achieved by some experience and in-depth study of the application on which test cases are being written.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
How to write effective Test cases, procedures and definitions
Writing effective test cases is a skill and that can be achieved by some experience and in-depth study of the application on which test cases are being written.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
Here I will share some tips on how to write test cases, test case procedures and some basic test case definitions.
What is a test case?“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.” Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.Level 1: In this level you will write the basic test cases from the available specification and user documentation.Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
Why we write test cases?The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.
How to write test cases?Here is a simple test case format
Fields in test cases:
Test case id:Unit to test: What to be verified?Assumptions:Test data: Variables and their valuesSteps to be executed:Expected result:Actual result:Pass/Fail:Comments:
So here is a basic format of test case statement:
VerifyUsing [tool name, tag name, dialog, etc]With [conditions]To [what is returned, shown, demonstrated]
Verify: Used as the first word of the test case statement.Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.
For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.
Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.
Try writing the simple test cases as mentioned in above test case format. Generally I use Excel sheets to write the basic test cases. Use any tool like ‘Test Director’ when you are going to automate those test cases.
Subscribe to:
Posts (Atom)