“Visual Studio Performance Testing Quick Reference Guide” v3.6 released - 241 articles, 51 updates/additions, 253 pages, all free.
ALM and Beyond - Performance Testing Quick Reference Guide
“This version of the quick reference guide QRG is a PDF file with a total of 241 articles (51 of these are either new or updated from version 3.5) that are all meant to provide quick information about various aspects of performance testing with Visual Studio. Here is a list of categories that these articles pertain to:
…”
Visual Studio Performance Testing Quick Reference Guide -
“Welcome to the Visual Studio Performance Testing Quick Reference Guide Community Site! The purpose of this project is to build some insightful and practical guidance around Visual Studio Team Test.
What is in the package?
- How It Works
- Items new to VSTT 2010
- Configurations and Settings
- Networks, IP Spoofing, Test Startups
- Performance Counters and Data
- Performance Counter Considerations on Rigs with slow links
- Data and Results
- Errors and Known Issues
- Troubleshooting
- Extensibility
- Items not specific to the VSTS testing platform
…”
Visual Studio Performance Testing Quick Reference Guide - VS Performance Testing Quick Reference V3.6
Here’s the TOC (Yeah, I know, wow! New and Updated items in bold)
NOTE FROM THE AUTHOR 10
HOW IT WORKS 11
How Web Tests Handle HTTP Headers 11
General Info (including order of execution) of load and web test plugins and rules 11
Client Code does not execute because Web Tests Work at the HTTP Layer 14
When is the "Run unit tests in application domain" needed? 14
How the "Test Iterations" Setting impacts the total number of tests executed 14
Test timeout setting for load test configuration does not affect web tests 15
How user pacing and "Think Time Between Test Iterations" work 15
Load test warmup and cool down behaviors 15
What is the difference between Unique, Sequential and Random Data Sources 16
Simulation of Browser Caching during load tests 17
Comparing new users to return users 18
How cache settings can affect your testing and app performance 20
Goal based user behavior after the test finishes the warmup period 22
Threading models in Unit tests under load 23
The difference between Load Test Errors and Error Details 24
How parameterization of HIDDEN Fields works in a webtest 25
Testing execution order in Unit Tests 27
How machines in the test rig communicate 29
How a Load Test Plugin is executed on a test rig 30
Sharing State across agents in a load test rig is not supported out of the box 31
--NEW--Order of execution of components in a webtest 32
--NEW--401 Access Denied responses and "Requests per second" measurements 44
--UPDATED-- File Downloads, Download Size and Storage of files during Web Tests 45
--NEW-- Info on how VS records and generates web tests. 46
--NEW--IE9 and other browser emulation in VS2010 46ITEMS NEW TO VS 2010 48
"Find" feature now available in Webtest playback UI 48
"Go To Web Test" feature now available in Webtest playback UI 49
Recorder Log Available 50
Add extraction rule directly from the playback UI 51
New "Reporting Name" property for web requests 52
LoadTestResultsTables now differentiate between GET and POST requests 53
--UPDATED-- Virtual user visualization now available 54
New Excel reporting features built into load test results 60
New Load Test and Load Test Rig Licensing and configurations 61
New test mix: "Sequential Test Mix" 65
Query String and FORM POST URLs get parameterized 67
New options on Load Test Scenarios 68
Loops and Conditionals 69
Visual Studio Performance Testing Quick Reference Guide Page 5
--NEW--Load test distributions/Network emulation 71
--NEW--SilverLight recording info 71
--NEW--Unlimited agent licenses for certain MSDN subscribers 71CONFIGURATIONS AND SETTINGS 72
How to Change the Location Where Agents Store Run Files 72
How to set a proxy server for web tests 72
How to configure Web Tests so Fiddler can capture playback info 72
Controlling the amount of memory that the SQL Server Results machine consumes 73
How to configure the timeouts for deployment of load tests to agents 73
How to set the number of Load Test Errors and Error Details saved 74
Multi-proc boxes used as agents should have .NET garbage collection set to server mode 75
Location of list of all agents available to a controller 75
--NEW--Setting the Agents to run in 64 bit mode 76
--NEW--Managing Test Controllers and Agents from VS *AND* from Lab Center 76NETWORKS, IP SWITCHING, TEST STARTUPS 77
IP Address Switching anatomy (how it works) 77
Gotcha: IP Address Switching is ONLY for WEB TESTS 77
Gotcha: IP Addresses used for switching are not permanent 77
How to Setup IP Switching 78
Startup: Slowness Restarting a Test Rig with Agents Marked as "Offline" 81
Startup: Multiple Network Cards can cause tests in a rig to not start 81
Startup: Slow startup can be caused by _NT_SYMBOL_PATH environment variable 82
Startup: tests on a Rig with Agents on a Slow Link 82 --NEW-- Startup: tests on a Rig with several Agents 82
"Not Bound" Exception when using IP Switching is not really an error 82
How to configure the timeout for deployment of load tests to agents 84
Customizing and extending the available network emulation settings 85
--NEW--Socket Exception: "An operation on a socket could not be performed because…" 87
--NEW--Failed scenarios when client is in a non-trusted domain 87PERFORMANCE COUNTERS AND DATA 90
Customizing the Available Microsoft System Monitor counter sets 90
Performance Counter Considerations on Rigs with slow links 92
Increase the performance counter sampling interval for longer tests 93
Changing the default counters shown in the graphs during testing 93
Possible method for fixing "missing perfmon counters" issues 93
How and where Performance data gets collected 94
--NEW-- How to remove Controller and Agent countersets 95
--NEW-- Controller and Agent countersets show up in test even after removing them 96DATA AND RESULTS 98
Custom Data Binding in UNIT Tests 98
Verifying saved results when a test hangs in the "In Progress" state after the test has finished 98
The metrics during and after a test differ from the results seen. 99
How new users and return users affect caching numbers 100
Visual Studio Performance Testing Quick Reference Guide Page 6
data sources for data driven tests get read only once 101
Consider including Timing Details to collect percentile data 102
Consider enabling SQL Tracing through the Load Test instead of separately 103
How to collect SQL counters from a non-default SQL instance 103
How 90% and 95% response times are calculated 103
Transaction Avg. Response Time vs. Request Avg. Response Time 104
Considerations for the location of the Load Test Results Store 104
--UPDATED-- Set the recovery model for the database to simple 104
How to clean up results data from runs that did not complete 105
InstanceName field in results database are appended with (002), (003), etc. 105
Layout for VS Load Test Results Store 105
How to view Test Results from the GUI 106
SQL Server Reporting Services Reports available for download 106
How to move results data to another system 106
Load Test Results without SQL NOT stored 107
Unable to EXPORT from Load Test Repository 107
Web Test TRX file and the NAN (Not a Number) Page Time entry 108
Proper understanding of TRX files and Test Results directory 109
Understanding the Response Size reported in web test runs 110ERRORS AND KNOWN ISSUES 111
--UPDATED-- CSV files created in VS will not work as data sources 111
Incorrect SQL field type can cause errors in web tests 112
Leading zeroes dropped from datasource values bound to a CSV file 112
--UPDATED-- Recorded Think Times and paused web test recordings 112
After opening a webtest with the VS XML Editor, it will not open in declarative mode. 113
Calls to HTTPS://Urs.Microsoft.Com show up in your script 113
Possible DESKTOP HEAP errors when driving command line unit tests 113
Goal based load tests in VS 2008 do not work after applying SP1 114
Using Named Transactions in a Goal-Based Load Profile can cause errors 114
Debugging Errors in Load Tests 115
Debugging OutOfMemory Exceptions in Load Tests 115
Memory leak on load test when using HTTPS 115
"Not Trusted" error when starting a load test 116
Detail Logging may cause "Out of disk space" error 117
Error details and stack traces no longer available in VS 2010 117
VS does not appear to be using more than one processor 117
Changes made to Web Test Plugins may not show up properly 117
--UPDATED-- Socket errors or "Service Unavailable" errors when running a load test 118
Error "Failed to load results from the load test results store" 119
Hidden Field extraction rules do not handle some fields 119
Test results iteration count may be higher than the max test iterations set 119
In flight test iterations may not get reported 120
Completion of Unit Test causes spawned CMD processes to terminate 120
Bug with LoadProfile.Copy() method when used in custom goal based load tests 121
Errors in dependent requests in a Load Test do not show up in the details test log 122
Visual Studio Performance Testing Quick Reference Guide Page 7
WCF service load test gets time-outs after 10 requests 124
Loadtestitemresults.dat size runs into GBs 124
Content-Length=0 Header not sent resulting in HTTP 411 Length Required Error 125
Error that test could not run because the network emulation is required 126
Error/Crash in "Open and Manage Load Test Results" dialog 126
Calls to CaptchaGenerator.aspx fail during playback 127
Request failure with improperly encoded query strings calling SharePoint 2010 127
Network Emulation does not work in any mode other than LAN 127
Error that Browser Extensions are disabled when recording a web test 128
Error: Request failed: No connection could be made because the target machine actively refused it 129
MaxConnection value in App.Config is not honored when running a load test 129
--NEW-- Cannot change the "Content-Type" header value in a webtest 129
--NEW-- VS does not expose a method for removing specific cookies from requests. 130
--NEW-- File Upload" feature in VS does not allow you to use a stream to send file 130
--NEW--Unit tests that consume assemblies requiring MTA will fail with default test settings 130
--NEW--MSTest tests that consume assemblies requiring MTA will fail with default test settings 130
--NEW--Load Test Agent Error "Failed to open the Visual Studio v10.0 registry key" 131
--NEW--"Agent to use" property of a Load Test not acting as expected 131
--NEW--Fiddler 2 not seeing application traffic 132
--NEW--BUG: Microsoft.VisualStudio.TestTools.WebStress.LoadTestResultsCollector 132
--NEW--ASP.NET Profiling sometimes does not report after a run. 133
--NEW--System.InvalidCastException: Unable to cast COM object 133
--NEW-- Assembly could not be loaded and will be ignored. 134
--NEW--Issue with webtest login when getting 307 Temporary Redirect 134
--NEW--Data bound validation rule fails when set at the TEST level 135
--NEW--"Could not read result repository" 136
--NEW--Page response time counters disappear after test is completed 136
--NEW--WebTestContext.Clear() not clearing cookies 137
--NEW--SQL Tracing error "Could not stop SQL tracing" 137
--NEW--LoadTestCounterCategoryNotFoundException 138IN DEPTH BLOG POSTS ON DEBUGGING AND TROUBLESHOOTING 140
Web Test Authoring and Debugging Techniques 140
Troubleshooting Network Emulation 159
Troubleshooting Guide for Visual Studio Test Controller and Agent 163
Best Practice: Understanding the ROI for test automation 176
Best Practice: Blog on various considerations for web tests running under load 176
User Account requirements and how to troubleshoot authentication 177TROUBLESHOOTING 178
How to enable logging for test recording 178
--UPDATED-- Diagnosing and fixing Web Test recorder bar issues 178
How to enable Verbose Logging on an agent for troubleshooting 180
Troubleshooting invalid view state and failed event validation 180
Troubleshooting the VS Load Testing IP Switching Feature 182
--NEW--Performance Counters in .NET 4.0 help with analysis of Agent machines 183HOW TO, GOTCHAS AND BEST PRACTICES 184
How to call one coded web test from another 184
How to use methods other than GET and POST in a web test 184
How to filter out certain dependent requests 184
How to handle ASP.NET Cookie-less Sessions 185
How to use Client-side certificates in web tests 185
How to remove the "If-Modified-Since" header from dependent requests 186
How to handle custom data binding in web tests 186
How to add a datasource value to a context parameter 186
How to test Web Services with Unit Tests 187
How to add random users to web tests 187
How to add think time to a Unit Test 187
How to add details of a validation rule to your web test 188
How to mask a 404 error on a dependent request 189
How to parameterize Web Service calls within Web Tests 190
How to pass Load Test Context Parameters to Unit Tests 190
How to create Global Variables in a Unit Test 190
How to use Unit Tests to Drive Load with Command Line Apps 191
How to add Console Output to the results store when running Unit tests under load 191
How to add parameters to Load Tests 192
How to Change the Standard Deviation for a NormalDistribution ThinkTime 192
How to programmatically access the number of users in Load Tests 193
How to create a webtest plugin that will only execute on a predefined interval 193
How to support Context Parameters in a plug-in property 194
How to stop a web test in the middle of execution 195
How To: Modify the ServicePointManager to force SSLv3 instead of TLS (Default) 195
How To: Stop a Test in the PreRequest event 196
How to make a validation rule force a redirection to a new page 196
How to add a Web Service reference in a test project - testing services in Unit Tests 200
How to remotely count connections to a process 202
How to hook into LoadTest database upon completion of a load test 202
How to deploy DLLs with MSTEST.EXE 203
How to authenticate with proxy before the test iteration begins 204
How to enumerate WebTextContext and Unit TestContext objects 205
How to manually move the data cursor 205
How to programmatically create a declarative web test 206
How to modify the string body programmatically in a declarative web test 207
How to Add Agents To A Test Rig 207
How to Change the Default Port for Agent-Controller Communication 207
How to create guaranteed unique user IDs for UNIT tests 208
How to create a sync point for starting load tests 210
How to set default extensions that the WebTest recorder will ignore 210
How to get the LoadTestRunId from a load test 210
--NEW--How To: Add comments to a web recording where IE is in KIOSK mode 211
--NEW--How to access a data source before it is bound to an object 212
--NEW--How to store and view transaction times for Unit and Coded UI tests 213
--UPDATED--HOW TO: Handle 404 errors in dependent requests so the main request does not fail. 214
--NEW--HOW TO: Minimize the amount of data a webtest retains for Response Bodies 215
--NEW--HOW TO: Schedule tests to execute 215
--NEW--HOW TO: NOT send an "accept-language" in webtests 216
--NEW--How to upload a file in a Web test 217
Gotcha: Check Your Validation Level in the Load Test Run Settings 223
Gotcha: Do not adjust goals too quickly in your code 223
Gotcha: Response body capture limit is set to 1.5 MB by default 224
Gotcha: Caching of dependent requests is disabled when playing back Web Tests 224
Gotcha: VS 2008 and out of memory 224
Gotcha: Timeout attribute in coded web test does not work during a load test 224
--NEW--Gotcha: Cannot programmatically set .counterset mappings at runtime 225
Best Practice: considerations when creating a dynamic goal based load test plugin: 225
Best Practice: Coded web tests and web test plug-ins should not block threads 225
Best Practice: Add an Analysis Comment 226EXTENSIBILITY 226
New Inner-text and Select-tag rules published on Codeplex 226
How to Add Custom Tabs to the Playback UI 228
How to extend recorder functionality with plugins 235ITEMS NOT SPECIFIC TO THE VS TESTING PLATFORM 243
Stand-Alone Network Emulation and CodePlex 243
Using the VS Application Profiler 244
VS 2008 Application Profiler New Features 244
Using System.NET Tracing to debug Network issues 244
Logparser tips and tricks 245
Logparser WEB Queries 245
LogParser Non-Web Queries 246
--NEW--Keyboard shortcut for adding "USING" statements automatically 247OLDER ARTICLES 248
Content-Length header not available in Web Request Object 248
SharePoint file upload test may post the file twice 248
Some Hidden Fields are not parameterized within AJAX calls 248
(FIX) Unit Test threading models and changing them 248
Bug in VS 2008 SP1 causes think time for redirected requests to be ignored in a load test 249
New Load Test Plugin Enhancements in VS 2008 SP1 249
Four New Methods added to the WebTestPlugin Class for 2008 SP1 249INDEX 250
‘nuff said?
Posts
No comments:
Post a Comment