Software Testing with Visual Studio Team System 2008

Saalim from Packt Publishing has offered me a chance to review their new “Software Testing in Visual Studio Team System 2008”. While I wait for the book they sent me a couple chapters with the permission to post them to my blog. I thought it would be a good break from my usual re-blogging so here you go…

This may be review for many of you, but for those who haven’t yet jumped into the testing features VS2008 provides this article/chapter will give you a nice introduction into some of what is available

Software Testing with Visual Studio Team System 2008: Subashni S: Books

ISBN: 184719558X
ISBN-13: 9781847195586

--- Content reproduced in full with permission from Publisher---

Software testing is one of the most important phases of the Software Development Life Cycle (SDLC). The end product is based on design, coding, testing, and meeting the specified requirements. The quality of the end product is measured by testing the product based on the requirement, using different testing tools and techniques. Even though we test the product with different tools, the real judgment of the product comes from the testing that simulates the real life situation, for example, by simulating the actual number of users, by simulating the load and simulating the actual production environment, and then by measuring the product. Microsoft Visual Studio 2008 provides not only the development environment and code maintenance for application but also different testing features such as Unit test, Load test, Web test, Coded tests, and Ordering tests list.

This article by Subashni.S and N Satheesh Kumar, will give us an understanding of the tools in Visual Studio and an overview of the different types of testing supported by Visual Studio.

Software testing in Visual Studio Team System 2008

Before going into the details of the actual testing using Visual Studio 2008, we need to understand the different tools provided by Visual Studio Team System (VSTS) and their usage. Once we understand the tools usage, then we should be able to perform different types of testing using VSTS. As we go along creating a number of different tests, we will encounter difficulty in managing the test similar to the code and its different versions during application development. There are different features such as the Test List Editor and Test View and the Team Foundation Server (TFS) for managing and maintaining all the tests created using VSTS. Using this Test List Editor, we can group similar tests, create number of lists, add, or delete tests from the list.

The other aspect of this article is to see the different file types getting created in Visual Studio during testing. Most of these files are in XML format, which get created automatically whenever the corresponding test is created.

The tools such as the Team Explorer, Code Coverage, Test View, and Test Results are not new to Visual Studio 2008 but actually available since Visual Studio 2005. While we go through the windows and their purposes, we can check the IDE and the tools integration into Visual Studio 2008.

Testing as part of Software Development Life Cycle

The main objective of testing is to find the defects early in the SDLC. If the defect is found early, then the cost will be less, but if the defect is found during production or implementation stage, then the cost will be higher. Moreover, testing is carried out to assure the quality and reliability of the software. In order to find the defect earlier, the testing activities should start early, that is, in the Requirement phase of SDLC and continue till the end of SDLC.

In the Coding phase, various testing activities takes place. Based on the design, the developers start coding the modules. Static and dynamic testing is carried out by the developers. Code reviews and code walkthroughs are also conducted.

Once the coding is completed, then comes the Validation phase, where different phases or forms of testing are performed.

Unit Testing: This is the first stage of testing in SDLC. This is performed by the developer to check whether the developed code meets the stated requirements. If there are any defects, the developer logs them against the code and fixes the code. The code is retested and then moved to the testers after confirming the code without any defects for the piece of functionality. This phase identifies a lot of defects and also reduces the cost and time involved in testing the application and fixing the code.
Integration Testing: This testing is carried out between two or more modules or functions together with the intent of finding interface defects between them. This testing is completed as a part of unit or functional testing, and sometimes becomes its own standalone test phase. On a larger level, integration testing can involve putting together groups of modules and functions with the goal of completing and verifying that the system meets the system requirements. Defects found are logged and later fixed by the developers. There are different ways of integration testing such as top-down testing and bottom-up testing:
- The Top-Down approach is followed to test the highest level of components and integrate first to test the high-level logic and the flow. The low-level components are tested later.
- The Bottom-Up approach is the exact opposite of the top-down approach. In this case, the low-level functionalities are tested and integrated first and then the high-level functionalities are tested. The disadvantage in this approach is that the high-level or the most complex functionalities are tested later.
- The Umbrella approach uses both the top-down and bottom-up patterns. The inputs for functions are integrated in the bottom-up approach and then the outputs for the functions are integrated in the top-down approach.
System Testing: It compares the system specifications against the actual system. The system test design is derived from the system design documents and is used in this phase. Sometimes, system testing is automated using testing tools. Once all the modules are integrated, several errors may arise. Testing done at this stage is called system testing. Defects found in this testing are logged and fixed by the developers.
Regression Testing: This is not mentioned in the testing phase, but is carried out once the defects are fixed by the developers. The main objective of this type of testing is to determine if bug fixes have been successful and have not created any new problems. Also, this type of testing is done to ensure that no degradation of baseline functionality has occurred and to check if any new functionality was introduced in the software.

Types of testing

Visual Studio provides a range of testing types and tools for software applications. Following are some of those types:

Unit test
Manual test
Web test
Load test
- Stress test
- Performance test
- Capacity Planning test
Generic test
Ordered test

In addition to these types, there are additional tools provided to manage, order the listing, and execute tests created in Visual Studio. Some of these are the Test View, Test List Editor, and the Test Results window. We will look at these testing tools and the supporting tools for managing the testing in Visual Studio 2008 in detail later.

Unit test

As soon as the developer finishes the code, the developer wants to know if it is producing the expected result before getting into any more detailed testing or handing over the component to the tester. The type of testing performed by the developers to test their own code is called Unit testing. Visual Studio has great support for Unit testing.

The main goal of the unit testing is to isolate each piece of the code or individual functionality and test if the method is returning the expected result for different set of parameter values. It is extremely important to run unit tests to catch the defects in the early stage.

The methods generated by the automated unit testing tool call the methods in the classes from the source code and test the output of each of the methods by comparing them with the expected values. The unit test tool produces a separate set of test code for the source. Using the test code we can pass the parameter values to the method and test the value returned by the method, and then compare them with the expected result.

Unit testing code can be easily created by using the code generation feature, which creates the testing source code for the source application code. The generated unit testing code will contain several attributes to identify the Test Class, Test Method, and Test Project. These attributes are assigned when the unit test code gets generated from the original source code. Then using this code, the developer has to change the values and assert methods to compare the expected result from these methods.

The Unit test class is similar to the other classes in any other project. The good thing here is that we can create new test classes by inheriting the base test class. The base test class will contain the common or reusable testing methods. This is the new Unit testing feature which helps us reduce the code and reuse the existing test classes.

Whenever any code change occurs, it is easy to figure out the fault with the help of Unit tests, rerun those tests, and check whether the code is giving the intended output. This is to verify the code change the developer has made and to confirm that it is not affecting the other parts of the application.

All the methods and classes generated for the automated unit testing are inherited from the namespace Microsoft.VisualStudio.TestTools.UnitTesting.

Manual test

Manual testing is the oldest and the simplest type of testing, but yet very crucial for software testing. It requires a tester to run all the tests without any automation tool. It helps us to validate whether the application meets various standards defined for effective and efficient accessibility and usage.

Manual testing comes to play in the following scenarios:

There is not enough budget for automation.
The tests are more complicated, or are too difficult to be converted into automated tests.
The tests are going to be executed only once.
There is not enough time to automate the tests.
Automated tests would be time-consuming to create and run.

Manual tests can be created either using a Word document or Text format in Visual Studio 2008. This is a form of describing the test steps that should be performed by the tester. The step should also mention the expected result out of testing the step.

Web tests

Web tests are used for testing the functionality of the web page, web application, web site, web services, and a combination of all these. Web tests can be created by recording the interactions that are performed in the browser. These can be played back to test the web application. Web tests are normally a series of HTTP requests (GET/POST).

Web tests can be used for testing the application performance as well as for stress testing. During HTTP requests, the web test takes care of testing the web page redirects, validations, viewstate information, authentication, and JavaScript executions.

There are different validation rules and extraction rules used in web testing. The validation rules are used for validating the form field names, texts, and tags in the requested web page. We can validate the results or values against the expected result as per business needs. These validation rules are also used for checking the time taken for the HTTP request.

At some point in time, we need to extract the data returned by the web pages. We may need the data for future use, or we may have to collect the data for testing purposes. In this case, we have to use the extraction rules for extracting the data returned by the page requested. Using this process, we can extract the form fields, texts, or values in the web page and store it in the web test context or collection.

Web tests cannot be performed only with the existence of a web page. We need some data to be populated from the database or some other source to test the web page functionality and performance. There is a data binding mechanism used in Web test, which is used for providing the data required for the requested page. We can bind the data from a database or any other data source. For example, the web page would be a reporting page that might require some query string parameters as well as the data to be shown in the page according to the parameters passed. To provide data for all these data-driven testing, we have to use the concept of data binding with the data source.

Web tests can be classified into Simple Web tests and Coded Web tests. Both these are supported by VSTS.

Simple Web tests are very simple to create and execute. It executes on its own as per the recording. Once the test is started, there won't be any intervention. The disadvantage is that it is not conditional. It's a series of valid flow of events.
Coded Web tests are bit more complex, but provide a lot of flexibility. For example, if we need some conditional execution of tests based on some values then we have to depend on this coded web test. These tests are created using either C# or Visual Basic code. Using the generated code we can control the flow of test events. But the disadvantage is its high complexity and maintenance cost.

Load test

Load testing is a method of testing used in different types of testing. The important thing with Load testing is that it is about performance. This type of testing is conducted with other types of testing, which means that it can be performed along with either Web testing or Unit testing.

The main purpose of load testing is to identify the performance of application based on different scenarios. Most of the time, we can predict the performance of the application that we develop, if it is running on one machine or a desktop. But in the case of web applications such as online ordering systems, we know the estimated maximum number of users, but do not know the connection speeds and location from where the users will access the web site. For such scenarios, the web application should support all the end users with good performance irrespective of the system they use, their Internet connection, the place, and the tool they use to access the web site.

So before we release this web site to the customers or the end users, we should check the performance of the application so that it can support the mass end user group. This is where load testing will be very useful in testing the application along with Web test or Unit test.

When a Web test is added to a Load test, it will simulate multiple users opening simultaneous connections to the same web application and making multiple HTTP requests. Load testing in Visual Studio comes with lots of properties which can be set to test the web application with different browsers, different user profiles, light loads, and heavy loads. Results of different tests can be saved in a repository to compare the set of results and improve their performance.

In case of client server and multi-tier applications, we will be having a lot of components which will reside in the server and serve the client requests. To get the performance of these components, we have to make use of a Load test with a set of Unit tests. One good example would be to test the data access service component that calls a stored procedure in the backend database and returns the results to the application that is using this service.

Load tests can be run either from the local machine or by submitting to a rig, which is a group of computers used for simulating the tests remotely. A rig consists of a single controller and one or more agents.

Load tests can be used in different scenarios of testing:

Stress testing: This checks the functionality of the application under heavy load. The resource provided to the application could vary based on the input file size or the size of the data set, for example, uploading a file which is more than 50MB in size.
Smoke testing: This checks if the application performs well for a short duration with a light load.
Performance testing: This checks the responsiveness and throughput of the application with different loads.
Capacity Planning test: This checks the application performance with various capacities.

Ordered test

As we know, there are different types of testing required to build quality software. We take care of running all these tests for the applications we develop. But we also have an order in which to execute all these different tests. For example, we do the unit testing first, then the integration test, then the smoke test, and then we go for the functional test. We can order the execution of these tests using Visual Studio.

Another example would be to test the configurations for the application before actually testing the functionality of the application. If we don't order the test, we would never know whether the end result is correct or not. Sometimes, the tests will not go through successfully if the tests are not run in order.

Ordering of tests is done using the Test View window in Visual Studio. We can list all the available tests in the Test View and choose the tests in the same order using different options provided by Visual Studio and then run the tests. Visual Studio will take care of running the tests in the same order we have chosen in the list.

So once we are able to run the test successfully in an order, we can also expect the same ordering in getting the results. Visual Studio provides the results of all the tests in a single row in the Test Results window. Actually, this single row result will contain the results of all the tests run in the order. We can just double-click the single row result to get the details of each tests run in the ordered test.

Ordered test is the best way of controlling the tests and running the tests in an order.

Generic test

We have seen different types and ways of testing the applications using VSTS. There are situations where we might end up having other applications for testing, which are not developed using Visual Studio. We might have only the executables or binaries for those applications. But we may not have the supported testing tool for those applications. This is where we need the generic testing method. This is just a way of testing third-party applications using Visual Studio.

Generic tests are used to wrap the existing tests. Once the wrapping is done, then it is just another test in VSTS.

Using Visual Studio, we can collect the test results, and gather the code coverage data too. We can manage and run the generic tests in Visual Studio just like the others tests.

Software Testing with Visual Studio Team System 2008

A comprehensive and concise guide to testing your software applications with Visual Studio Team System 2008

Test your software applications with Visual Studio Team System 2008 and rest assured of its quality
Create a structured testing environment for your applications to produce reliable products
Comprehensive yet concise guide with a lot of examples and clear explanations
No knowledge of software testing is required, only basic knowledge of Visual Studio 2008 operation is expected

http://www.packtpub.com/software-testing-with-visual-studio-team-system-2008/book

Test management and new testing features in VS 2008

Visual Studio Team 2008 has great features for working with the type of tests we've just described. These features are greatly improved from the previous versions of VSTS. The Team Explorer, Test View, and Test List Editor are some of those main IDE add-ins used for tests management

Team explorer

Team Explorer is the add-on to the VSTS IDE. Using this explorer, the developer or a user of Visual Studio can browse through the Team Projects in the TFS to which the explorer is connected. TFS is the central repository for managing and controlling the team projects. To work with any of the team projects in the TFS, first we have to connect to it using VSTS. After connecting to the server, we can use Team Explorer to browse through the details of the particular Team Project.

Team Explorer is a client of TSF and is integrated with the VSTS. Team Explorer provides features to work with the team project details such as:

Accessing project, process guidance documents that explain the process to be followed for the Team Project. This process guidance helps the team follow the process for the Team Project items such as work items, reports, queries, and work products.
Managing work items such as tasks, defects, issues, and requirements.
Importing and exporting the work items.
Accessing the project files and source control management.
Adding, removing, configuring, and managing the Team Projects.
Creating, managing, and automating the project builds.
Managing favorites which are shortcuts to the favorite nodes of the Team Project.
Managing the reports. It could be defect reporting or any type of work items reporting.

Team system testing tools introduction

Visual Studio 2008 provides many tools to create, run, debug, and view results of the tests for the applications. It is very flexible for running a test and getting the results. Given below is an overview of the tools and windows provided by Visual Studio to view the test details and test output details. We will look into the overview of each of the tools available in Visual Studio 2008.

Let us see how we can create a new testing project using Visual Studio 2008, and then we will test a sample project to get to know about the tools and features.

Open Visual Studio 2008 and open the solution of the application which is developed using C# or VB.NET. We will not go into the details of the sample application TestingApp as shown in the preceding screenshot. We will just create a test project for the sample application and see the features of the tools and windows.

Now similar to adding the projects and code files to the solution, we have to create the test project and test files and add the test project to the solution. There are different ways of creating test projects under the solution.

Select the solution and add a project using the Add | New Project.... Then select the project type as Test from the list of project types under the language, which we are going to use. Then select the template as Test Project.
Select the solution and then from the top toolbar menu option, select the option Test | New Test..., which will list down all the test type templates. Now you can select the type of template we need for the testing. You can see the option which is a drop-down saying Add to Test Project with the options as list of <projectnames>, Create a new Visual C# test project…, Create a new Visual Basic test project..., and three more options.

For our sample testing application, we will select the second option mentioned above and choose the simple Manual Test (text format) type.

The Add New Test dialog contains nine different template options for creating the tests. There are two different templates for the Manual Test with one using Text file and the other using a Word document. The Unit testing template is of three types; one is for the Database Unit Test, the second one for the general Unit Test, and the third one for the Unit Test Wizard. Following are the file extensions for each of the VSTS test types as shown in the preceding in the image:

.vb—is for the Database Unit test
.generictest—for the Generic test type
.loadtest—for test which is of type load test
.mtx—for manual test using text file format
.mht—for manual test using word document format
.orderedtest—for ordered test types
.vb—for unit test
.webtest—for web testing

After selecting the test type, give a name for the test in the Test Name field. Select the Create new Visual C# test Project… in the Add to Test Project option. We will see more details about this later. For now, consider that we have added the testing project to the existing solution.

Now, the system will ask for a name for the new test project. Let us name it TestProjectforTestingApp.

Now, we can notice that there are two files created as <ProjectName>.vsmdi, which is the metadata file, and LocalTestRun.testrunconfig file created under the solution for the test projects. The metadata file contains all the information about the tests list. It is an XML file which has all the details.

Add two more tests of any type you want. For this example, I have added one more Manual Test using Word document and one Unit test as shown in the Solution Explorer here.

Now, let us see the different windows and tools available in support of all the tests that we can perform using Visual Studio. We will take the sample Manual testing example for getting to know about the tools.

Test view

The Test View window is used for viewing the available tests and also for selecting the test and run it. We can also see the properties of each test from here. It is similar to the solution explorer from where we can get the list of class files in the project. We can also group the available tests based on the type, or owner, or project, or solution, or priority, or any other property of the test. The Test View window can be opened using the Visual Studio IDE menu option Test | Windows | Test View. Now, you can see the following Test View screen which displays a list of all the tests created in the current Test Project.

The screenshot shows the default test view with the list of tests based on the type. We can customize the test view using the filters, grouping and adding or removing the columns.

The Test View window has its own toolbar for different operations that can be performed on the list. The left-most option in the toolbar is the run and debug option, by which we can start the VSTS process for running or debugging the selected tests.

The second is the Filter option in the toolbar, which contains a drop-down listing of all the columns related to the Test View and the test properties and a box which says <Type keyword> for entering the value that can be used against the field selected in the second drop-down for filtering the list of tests shown. When the button with the arrow next to these drop-downs is clicked, the actual filtering will take place, and the grid will show the filtered list of tests. For example, the following screenshot shows a list of tests filtered with Test Type which are Manual.

The third one is the Group By option which has a drop-down with a list of test properties using which we can group the list of tests available.

In the preceding screenshot, remove the criteria entered for the filter in the second drop-down and click on the arrow which will show all the available tests.

Now select any of the values available in the Group By drop-down.
On selecting the drop-down, we can see the list groups based on the value of the property selected in the drop-down.

The three screenshots below show the tests grouped based on the type. We have two Manual tests and one Unit test. If we select the Group By as the Class Name as shown in the second screenshot, we can see that both of the Manual tests are under No Class Name and the Unit Test is under UnitTest1. Here the class name is no name because the manual tests will not have any classes. Same is the case when we select Description as the Group By field as shown in the third screenshot below. The test which is of type Unit test, and the test which is of type Manual Test are shown together because both these tests do not have a description.

The screenshot below shows the tests grouped by Test type. There are two manual tests and one Unit test.

The screenshot below shows the tests grouped by Class Name. The Manual tests does not have class name as it is not derived from any class and does not have a class.

The below screenshot shows the grouping of tests based on the description. The unit test has a description but not the manual tests as there is no description for tests which is of type Manual.

We can change the descriptions of the tests and group them together. Whenever we change the description or class name or an editable property of the test, it will be grouped under that property, if it exists. Otherwise, it will be shown as a new property with that new group. For example, if we change the description of all the tests to Sample Test and group the tests by description, then we can see all the tests grouped together as shown here:

In the Test View, we can select the available columns to display in the view, and we can also sort them using the column header.

The following steps explain how to show a column in the view display and how to sort them:

Right-click on the columns that are available in the Test View window, and select the Add/Remove Columns option. The Add/Remove Columns dialog box get displayed.
Check and uncheck the columns that need to be displayed, and those that are not to be displayed.

For sorting the columns, click on each column header to change the sorting order. If the same column is already sorted, then the sorting order will be changed.

Test list editor

All features available in Test View are also available in the Test List Editor with some additional features. Through this window, we can create the test lists by grouping the available tests. We can also select multiple tests from the list and run them using this window.

Whenever we open the Test List Editor using the menu option Test | Windows | Test List Editor, we can see the window as follows showing all the available tests:

The toolbar in the Test List Editor window has the same features that we saw in the Test View window. But you can see that the details section of the list window is split into two panes. The left pane of the window has three nodes, and the right pane displays all the tests in the lists based on the selection in the left pane. The Test List Editor displays the following three options as nodes.

A comprehensive and concise guide to testing your software applications with Visual Studio Team System 2008

Test your software applications with Visual Studio Team System 2008 and rest assured of its quality
Create a structured testing environment for your applications to produce reliable products
Comprehensive yet concise guide with a lot of examples and clear explanations
No knowledge of software testing is required, only basic knowledge of Visual Studio 2008 operation is expected

http://www.packtpub.com/software-testing-with-visual-studio-team-system-2008/book

Lists of tests

This is the first node in the list which is initially empty. It also asks us to create the list and add the tests to the list.

We can click on the here link and give the details for the new test list in the window that opens up as Create New Test List. Let's call that test list Manual Tests to group all the available manual tests.

Now in the left pane of the Test List Editor, select the second option Tests Not in a List. Do not check the option but just select the option. This will display all the available tests in the project, which are not part of any other Test list as shown below:

Now choose both the Manual tests on the right pane and just drag and drop them on the Manual Tests node on the left pane, which we created earlier. The other way of doing the same is, after selecting both manual tests, by right-clicking and choosing the New Test List... option. We will get a window similar to what we saw earlier in Create New Test List.

Since we have already created the Test list, we will just drag and drop the tests on the test list name. Now when we select the newly created Manual Tests list, we can both the manual tests listed on the right-side pane.

Now if we select the node Tests Not in a List, we can see only the other tests that we had not selected in our previous steps to add to the Manual Tests list.

Select the third node in the left pane All Loaded Tests. This option will list down all the tests available, whether it is added to a Test List or not. We can also see the Test List name to which a particular test is added; and it will remain empty for the tests which are not part of any of the Test Lists. The following screenshot shows all the loaded tests in the project:

After creating the Test List, we can run the tests based on the list we have created. The purpose of creating the Test List is to group the tests together so that we can run them all together by selecting the Test List using the checkbox option to the left of the test list name and then selecting the Run or Debug option in the toolbar. Each of the nodes in the Test List Editor has checkboxes. Using these checkboxes, we can select the individual tests, a Test List, multiple tests within the Test list, and tests that are not in a list. Then we can run the tests using the Run Checked Tests or Debug Checked Tests option in the toolbar.

There are two more options in editor toolbar: Load Metadata and Import Metadata. These are helpful in loading the existing Test Lists, which are already saved but not available in the current list. This Import and Load refer to the metadata file that gets created for the Test Project under the solution. It's the file with the extension vsmdi, which contains all the information about the tests lists.

All other options in the toolbar are the same as the options available in the Test View window such as grouping, filtering, and adding or removing columns.

Test results

This window shows the results of the tests run. It shows the error messages or the statuses of the multiple tests run. We can also reselect the tests from the Test Results window and rerun the tests. Using this window, we can export the test results or import the results already exported. This is useful in viewing the result in the Test Results window and rerun the test.

Now, as we have seen in the previous section about the Test List Editor, select the Manual Tests list and run the tests.

Now, we will consider the test, manualtestdoc as passed, and the test ManualTestText as failed. While running the tests, we can find the tests in progress and the status of the completed tests in the Test Results window as shown below. It's just the starting of the tests. So both the tests are in pending stage.

After completing both the tests, we can see the results as shown in the following screenshot. It will clearly indicate the status for each result. The window also shows the summary of the results, passed and failed. If we want to rerun the test, we don't need to go to the List Editor. From the results window, we can select the tests and rerun them. Using the checkbox before the test in the list window, we can select the test and then choose the Run or Debug option to rerun it.

In the preceding screenshot, we can also notice that there are two panes similar to the Test List Editor. The second option in the toolbar shows the list of Test Lists available and their corresponding tests in the right pane. It is the same option that we saw in the Test List Editor.

From the test result, we can also get the details of the test list run. We can get the details such as start time, end time, test location, and the user who ran the test and also the summary of the test results as shown in the following screenshot:

There is another option in the Test Results, which is the option to export the results. Whenever we finish our testing, we always would want to save the results of the tests for our further analysis. In the toolbar, we can see the option for exporting the test results. It provides two options, Export Test Run Results... to export all test results and Export Selected Test Results... to export only the selected test run result. We can also import the saved result using this window.

Once we choose any of these options, we can see the file dialog for saving the file. We can also notice that the file type is *.trx, which is the Visual Studio Test Results file for saving all the test results.

The .trx test result file is created by the VSTS on execution of the test every time. The file is named after the user, machine name, the date and time of execution. In the previous image, you can see the test name as Subashni@MyPC 2008-02-16 19:59:50, which shows the name of the user logged in and running the test, the machine name, and then the date and time of execution. The .trx file is an XML file with all information about the test result. If you open the .trx file, it will open the test result window in Visual Studio and show the details in the Test Results window. The contents of the test result file would look like the code shown below. This is just a sample and part of the generated test result file:

<?xml version="1.0" encoding="UTF-8"?>
<TestRun id="7c0e0d0a-5068-436c-887e-b4367ad893c2"
 name="Subashni@MyPC (2008-02-16 29:59:50)" runUser="Subashni"
 xmlns="http://microsoft.com/schemas/VisualStudio/
 TeamTest/2006">
 <TestRunConfiguration name="TestRunConfig1"
 id="fa788594-6ac2-495e-a484-2229faa94326">
 <Description>Test configuration file with the user defined
 scheme</Description>
 <CodeCoverage enabled="true">
 <Regular>
 <CodeCoverageItem binaryFile="C:WorkspaceCustomRulesbin
 DebugCustomRules.dll"
 pdbFile="C:WorkspaceCustomRulesbin
 DebugCustomRules.instr.pdb"
 instrumentInPlace="true" />
 </Regular>
 </CodeCoverage>
 <Timeouts runTimeout="1800000" testTimeout="2700000" />
 <Remote controllerName="&lt;Local - No controller&gt;" />
 <Deployment runDeploymentRoot="C:WorkspaceTestingApp
 TestResultsSubashni_MyPC _2008-02-16 19_59_50_">
 <DeploymentItem filename="C:WorkspaceCustomRulesbinDebug
 CustomRules.dll" />
 <DeploymentItem filename="C:WorkspaceServiceHostApp
 ServiceHostAppbinServiceHostApp.dll" />
 <DeploymentItem filename="C:WorkspaceWebTestPluginSamplebin
 DebugWebTestPluginSample.dll" />
 <DeploymentItem filename="C:WorkspaceProjectGenericsSample
 binDebugProjectGenericsSample.exe" />
 <DeploymentItem filename="Test.dll" />
 <DeploymentItem filename="C:WorkspaceTestLibrarybinDebug
 TestLibrary.dll" />
 <DeploymentItem filename="C:Program Files
 Microsoft Visual Studio 9.0Common7IDE
 PrivateAssembliesMicrosoft.VisualStudio.
 SmartDevice.TestHostAdapter.dll" />
 <DeploymentItem filename="ClassLibrary1.dll" />
 </Deployment>
 <NamingScheme baseName="Subashni@MyPerPC" useDefault="false" />
 <Scripts setupScript="C:WorkspaceTestingApp
 SetEnvironment.bat" />

Code coverage results

Code coverage provides information about codes and how many kinds of codes are covered during the test run. This is to view the statistical information about the test run. This result will show how many lines of code are executed and how many lines are not executed. We can also get the color coding for the lines run, and different color coding for the lines not run. This is also a kind of test result.

Before starting the code coverage we have to configure the code coverage. Select the Test | Edit Test Run Configurations | Local Test Run option. These configurations are stored in the configuration file, localtestrun.testrunconfig. In the configuration window, select the option Code Coverage in the left pane. On the right-side pane, we can see the list of all the available artifacts for code coverage. The artifacts could be an application, or a library, or any type of assembly. We can choose the artifacts from the list, or we can also import the artifacts using the Add Assembly option and then choose that for the code coverage. As shown in the following screenshot, let us select the test project library, TestProjectforTestingApp.dll.

When we click on apply in the above window, the code coverage is enabled only for the assemblies selected. When the application is executed, we can see the code coverage window collecting information about the source code covered in testing as shown in the following screenshot. The code coverage window is available under the menu option Test | Windows | Code Coverage Results. Lets try running the unit test application available in the test list editor under the Tests Not in a List which has the Unit test. After running the test, we can see the code coverage results as shown in the following screenshot:

The Code Coverage Results window shows each block covered, blocks not covered, and the percentage of coverage for blocks. This applies to each run of the test. Next time we run the same test, we can see a different set of results under a new name.

In the Unittest1 class, we can also notice the color coding based on the coverage. This is the code which is covered under testing, and code that is not covered will be highlighted with color coding.

After completing the test, we can also export the coverage results similar to the test results. We can also import the coverage results which were already exported. There is another new option, Merge Results. This window will list all the coverage results from which we can select the results and merge them to get the end result of the code coverage collectively from various results.

Overview of XML files for testing tools in Visual Studio 2008

There are several XML files used by the VSTS Test Edition for storing the test information. All these documents should conform to the schema defined in the TestTypes.xsd file.

We have seen some of the XML document types in the previous examples, and here is the summary of all the XML documents for different test types supported by Visual Studio.

Test Metadata file: This file stores information about all the tests in the solution. It has the extension .vsmdi.

Test Results file: This file contains the results of the tests that were run. This file will be created automatically by Visual Studio when any test is run. It has the extension .trx.

Test Run Configuration file: This file contains the settings that specify how the test runs should proceed further. It has the extension .testrunconfig.

There are other XML files for different test types with some specific extensions such as .webtest, .loadtest, .orderedtest, .generictest.

XML schema was changed from Visual Studio 2005 Team Edition to the release of Visual Studio Team System 2008 Test Edition. So the XML documents that were created using Visual Studio 2005 Team Edition for Testers need to be upgraded to VSTS 2008 Test Edition before starting to use those XML documents. These XML documents can be converted to the current format using the exe, VSTST-FileConverter.exe.

This file accepts one parameter which is the XML document for the test created using Visual Studio 2005. For example, if the test is an ordered test with the name SampleOrderedTest1, then it should be converted using the converter to:

>VSTST-FileConverter.exe SampleOrderedTest1.orderedtest

To get help on the file converter, we can use the following command:

>VSTST-FileCOnverter.exe /help

Summary

This article gave an overview of the different types of testing that we can perform using Visual Studio 2008. We have also looked at how we can create the test project in Visual Studio, and how we make use of different tools that support testing in Visual Studio. Now, we should be able to create and execute a test project and create the test lists. We got an idea of how to view the test results using the Test Results window and look into the Code Coverage Results window. This article also explained about different types of XML and configuration documents that were created during the testing.