Switching environments, data sources, and other dynamically changeable settings in TFS Lab (on-premise) for 2013

Thanks to a post on this blog, we can access the settings inject into Lab-managed builds in TFS 2012 and 2013. Even though this is 2016 and many people are moving to the cloud with VSTS, there are still a good number of companies out there with considerable infrastructure "on-premise", and TFS can help with executing automated tests in those environments. I'm going to be building a library for helping others out with retrieving settings dynamically from TFS and I'll update my blog when it's available.

Building a developer test lab with Test Controllers and Agents in VSTS

Microsoft has recently released their DevTest Lab resources to production in Azure. This great functionality can let you quickly build up a lab where you can test your products and tear it down.

Steps:

1. Create an Azure Resource Group project in Visual Studio for managing your DTL.
2. Add a Virtual Network resource to the template
3. Add a DevTest Lab resource to the template
4. In the portal, bind the virtual network to the DTL in the Settings tab of the DTL
5. Create a Virtual Machine to host your software. Use a Formula to install any pre-requisites that you want on the machine: e.g. Chrome, Notepad++, etc.
1. As a pre-requisite for connecting to a vNext workflow, you **MUST** have the Azure PowerShell cmdlets installed on the machine as part of the Formula for the VM
6. On the virtual machine, open a PowerShell console and enable remoting along with adding a corresponding firewall rule to remove the local-subnet-access-only restriction that's set by default:
1. PS> Enable-PSRemoting
2. PS> Set-NetFirewallRule –Name "WINRM-HTTP-In-TCP-PUBLIC" –RemoteAddress Any -LocalPort 5986
3. PS> Set-ExecutionPolicy RemoteSigned
4. PS> dir WSMan:\localhost\listener\*\Port # show the port on which WSMan is currently listening
5. PS> winrm set winrm/config/Listener?Address=*+Transport=HTTP '@{Port="5986"}' # Change the port on which WSMan runs, option 1
6. PS> Set-Item WSMan:\localhost\listener\*\Port 5986 # Change the port on which WSMan runs, option 2
7. Configure the machine as per the tools here, which is essentially the same as the above steps, with some extra ones as well: https://github.com/Azure/azure-quickstart-templates/tree/master/201-vm-winrm-windows
7. From another machine, run the following to ensure that your machine is connectable:
1. Test-WSMan -ComputerName mymachinedns.westus.cloudapp.azure.com -Port 5986
2. $sessionOpt = New-PSSessionOption -SkipCACheck
3. $session = New-PSSession -ComputerName [myvmname].westus.cloudapp.azure.com -Credential (Get-Credential) -Port 5986 -UseSSL -SessionOption $sessionOpt

Regarding pre-test invocation scripts when deploying to a Test Agent with TFS

So, I learned something interesting about TFS Test Agents today and they way they handle pre-test invocation scripts when running tests on a Test Agent. In Microsoft Test Manager, do the following:

1. Connect to a Team Project
2. Change into 'Lab Center' mode
3. Click on 'Test Settings' in the top bar-ish area. This will open up the Test Settings Manager.
4. Edit or create a new 'Test Settings' item and open it up.
5. In the 'test settings' editor, under the 'Steps' column on the left-hand side, go to 'Advanced' -> 'Scripts'. You're now presented with the scripts page where you can specify scripts to be invoked before and after the execution of your test run.

Now, **here's the important thing** :

The script file(s) you specify in these boxes do not get copied to the Test Agent per se. Instead, their contents get read and merged with an automatically generated script that's created by the Test Agent. The script that actually gets run on the Test Agent will look similar to the following:

REM **************************************************************************** 

REM *  Generated by Microsoft Visual Studio 

REM *  Copyright (c) Microsoft Corporation. All rights reserved. 

REM *   

REM **************************************************************************** 

set ResultsDirectory=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1\Results 

set DeploymentDirectory=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1\DEPLOY~1 

set TestRunDirectory=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1 

set TestRunResultsDirectory=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1\Results\00C4EF~1 

set TotalAgents=1 

set AgentWeighting=100 

set AgentLoadDistributor=Microsoft.VisualStudio.TestTools.Execution.AgentLoadDistributor 

set AgentId=1 

set TestDir=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1 

set BuildDirectory=\\SOMESERVER\SomeShare\TFSDrops\HRST\CWSTAM~1.SPR\CWSTAM~3.1 

set DataCollectionEnvironmentContext=Microsoft.VisualStudio.TestTools.Execution.DataCollectionEnvironmentContext 

set TestLogsDir=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1\Results\00C4EF~1 

set ControllerName=TSTTFSTHR01:6901 

set TestDeploymentDir=C:\Users\Autotest\AppData\Local\VSEQT\QTAgent\23620\00C4EF~1\DEPLOY~1 

set AgentName=00C4EFD8-A65F-4B5E-AD5F-04F93895B543 

REM **************************************************************************** 

REM *  User Commands 

REM *   

REM **************************************************************************** 

echo "My actual user commands from my script file here"

This is important to keep in mind when you're writing the commands in the script to be executed, because no files get copied with the script file you reference in the Test Settings, and there's no mention of where that script file originally came from, so the script is effectively executed without any context except for that which is given to it by the TFS Test Agent in the prefixed lines.