Life is a Non-Deterministic Finite State Automata
Automation ? (*pGeekiness)++ : Code /eneration;

June 15, 2012

Automating the creation of standard environments using the VS2012 RC API (Update)

The full source code for this article can be downloaded here (V1.1) It is built against the 2012 RC version.

UPDATE 28-June-2012: The TF259637 error has been confirmed and sounds like it will be fixed with better UI guidance in the next release. The sample code is still at V1.1 and I will update my code at RTM.

A very welcome feature in the new edition of VS2012 Lab Center is the concept of a ‘Standard Environment’. Within Lab Centre, you can add any running machine to an environment and automatically push out the Test Agent ready for distributing tests to:

This means that VmWare, Virtual Box, Virtual PC and physical machines can easily be incorporated into your Lab Center environments. This post will show how to automate that feature using the (now public) API. No need to jump through hoops anymore like in VS2010!

However, before going on, you must at least be able to push these agents out manually using the Lab Center user interface so you know your infrastructure is set up for this: ensure that your target machine has file sharing set up; that you have fixed this rather obscure registry setting if necessary and ensure that IPSec and the Firewall aren’t getting in the way. And lots more.

We will now write some code to create a new Standard Environment and push the Agent out ready to run tests:

It looks like all of the API calls to the Lab Service and supporting infrastructure are now public in MSDN. There is some very kool stuff in there!

Getting going
The first thing is to add a few references to your project: Microsoft.TeamFoundation.DLL, Microsoft.TeamFoundation.Client.DLL and Microsoft.TeamFoundation.Lab.Client.DLL (search your machine for these):

Then it’s a simple case of connecting to a Team Project…

TeamProjectPicker picker = new TeamProjectPicker(TeamProjectPickerMode.SingleProject, false);

DialogResult result = picker.ShowDialog();
if (result != System.Windows.Forms.DialogResult.OK) return;
if (picker.SelectedProjects.Length == 0) return;

… and making the following calls to create a new environment and register it with a test controller:

LabSystemDefinition single = new LabSystemDefinition("TheMachineNameYouWantToPushTheAgentsOutTo", "TheMachineNameYouWantToPushTheAgentsOutTo", "YourMachineRole");

LabEnvironmentDefinition definition = new LabEnvironmentDefinition("The Environment Name", "The Environment Description", new List() { single });
definition.TestControllerName = "TheTestController:6901";

LabEnvironment newEnvironment = service.CreateLabEnvironment(ProjectName, definition, null, null);

There is then a nicely exposed ‘InstallTestAgent’ method that does exactly what it says on the tin:

// Download the source code to see how the credentials are set up for this call (process == null if you want to run the Test Agent as a service)
themachine.InstallTestAgent(admin, process);


The name ‘InstallTestAgent’ is a little misleading – it installs the Agent if it does not already exist and then reconfigures it.

Configuring the Agent to run tests interactively is similar: all we need to do is to provide another set of credentials that we want to run the Test Agent as on each end-point and tell the Lab Environment which machine roles require an interactive agent so the deployed agents can be configured correctly. We do this prior to creating the environment otherwise we would have to call LabService.UpdateLabEnvironment afterwards:

definition.CodedUIRole = p.MachineRoles;
definition.CodedUIUserName =  String.Format("{0}\\{1}", p.InteractiveCredentials.Domain, p.InteractiveCredentials.UserName);

I had issues getting my Lab Center to push out a test agent across *Workgroups* to run interactively even when I drive the operation manually from the Lab Center UI (not the API): this happened from a completely fresh install or otherwise. After pushing out the Agent, rebooting and automatically logging in, the Lab Center UI would keep hitting me with error “TF259637: The test machine configuration does not match the configuration on the environment. The account…”. The benefit of a public API is that it lets us investigate! The error appears to be the way Lab Center stores and/or validates its LabEnvironment.CodedUIUserName and LabSystem.Configuration.ConfiguredUserName parameters when distributing a Test Agent across Workgroups. The LabEnvironment.CodedUIUserName was set to ‘W732AGENTS\Graham’ (the value I entered in the Lab Center UI because that is what I want to run the Agent as on the end-point) whereas the LabSystem.Configuration.ConfiguredUserName property was set to .\Graham. Clearly a mismatch. To fix it, it seems all we need to do is sync the two. I need to be clear [especially given how the above code snippet obviously creates this problem!]: the issue occurs when driving the Lab Center UI manually, so it is not specific to the API or this sample. For the sample, I have chosen to mimic (what I think is) the behaviour of the Lab Center UI.

I have posted an issue with Connect with more information to seek clarification – please see ID: 749436.

If deploying Agents to Workgroups, you might get the TF2569637 error. I have left my source code to pass parameters to the API the same way that the Lab Center UI RC appears to (ie: without validation or guidance) so I attempt to deal with the error and automatically fix it post-deployment. It actually makes things more robust anyhow. I will update my code to reflect Lab Center UI changes post-RC:

var theEnvironment = service.QueryLabEnvironments(new LabEnvironmentQuerySpec() { Project = ProjectName }).First(f => f.Name == p.EnvironmentName);
var theMachine = theEnvironment.LabSystems.First(f => f.Name == p.MachineName);

string testAgentRunningAs = theMachine.Configuration.ConfiguredUserName;
string environmentThinksTestAgentRunningAsd = theEnvironment.CodedUIUserName;

if (String.Compare(testAgentRunningAs, environmentThinksTestAgentRunningAsd, true) != 0)
    // Synchronize the user names... 
    service.UpdateLabEnvironment(theEnvironment.Uri, new LabEnvironmentUpdatePack() { CodedUIUserName = testAgentRunningAs });

You can also use that snippet to fix a manually-deployed environment that is broken with error TF259637.

Putting it all together, you can can download this sample here. :

It looks like there is a slightly more flexible way of doing the installation of the test agents using a combination of the TestAgentDeploy class and the AMLCommandBase-derived classes. But perhaps more on that some other time!


March 25, 2011

Automating the Integration of VmWare with Microsoft Test Manager and Lab Center: Part 6 – Changes for Visual Studio 2010 Service Pack 1

PLEASE NOTE: This is for Visual Studio 2010. For the VS2012 version, please click here.

Mid-way through the series, Visual Studio Service Pack 1 was released. How amusing! So this is an update to incorporate the Service Pack 1 changes.

See Part 5 for the Source Code and scripts.


Do not use this code under any circumstances (should just about cover the possibilities!).

I am using an undocumented API in order to construct the Physical Environment in Lab Center and set up the Test Controller Topology. I have tested the registered environments using MTM, use it often and come to no grief. My Lab Center and TFS system appears to be stable. But you use this at your own risk! At the very least, it would be sensible to do a full back up of your TFS Installation and ideally test this prior to production deployment. Use at your own risk :)

Parts 1, 2, 4, 5 are the same: nothing changes. The only changes you will need to make are down to the installation automation in Part 3.

I will not be providing an updated script to do this but if you have been following the series and want to stick to the same structure, you need to make Service Pack 1 available under the VisualStudioGumpf directory by unpacking your ISO there:

You will probably also need to create a new BAT file to launch “setup.exe /passive” from the Service Pack 1 location. Drop this into your Golden VM at the usual place:

And then write a new function to launch that from PowerShell.

InstallServicePack1 $VmWareConnectionParameters $VmWareClonedImageLocation "$DomainName\$DomainUsername" $DomainPassword $VisualStudioGumpfUnc;

If you get problems – try it manually first! The only part where I do anything undocumented is to create the Physical Environment. If you happen to get a situation where you can do this registration process manually, but not automatically, please let me know so that I can fix it :-)

Source Code Changes
I have no idea how many lines of white powder I had up my nose when I wrote this comment:

// I am not going to check this here but only one machine in an environment can be of a given role. 
Dictionary agentsToAdd = new Dictionary();
Dictionary machineRoleInfo = new Dictionary();

But it is clearly wrong!

Apart from having to install Service Pack 1, I haven’t had to make any changes: the environment still gets created and all appears normal. You should be able to target your created environment from MTM:

And run your Unit Tests, Integration Tests and CodedUI Tests on it:


December 5, 2010

T4 Performance: From TextTemplatingHost to Preprocessed Templates

Filed under: Performance,Programming — Tags: , , , , — admin @ 7:44 am

I’ve been moving my code generators and infrastructure from Visual Studio 2008 to Visual Studio 2010 and making use of the new ‘Preprocessed T4 Templates’ feature. There’s no metrics out there I could find, so here’s mine.

Within 2008, I had my own host that loaded the TT files as they were required, used Microsoft’s Text Templating Engine to parse them on the fly and then generate my code. Performance was pretty good but it meant parsing the TT files every time. 7.5 seconds for 140+ files is admirable!

In 2010, I’ve moved over to Preprocessed Templates and the performance has improved three fold:

In all, there’s about 2 megabytes of generated C# code from the above generation job – in about 2.5 seconds. Preprocessed Templates are clearly the way to go!

My motivation for doing this isn’t speed though. As much as I would like to ship raw T4 templates with my application, it would appear it is not legal to distribute the Text Templating Engine along with your product. This means that unless you are targeting the Visual Studio development community, you cannot rely on the TextTemplating DLL’s being there. So you have to preprocess your templates and distribute a compiled version of that instead. The Engine comes with Visual Studio and various addons (in the Microsoft.VisualStudio.TextTemplating.* DLL’s) so developers don’t notice it’s missing until they distribute their application :-) It would be awesome if Microsoft could push out the T4 Engine as part of a regular update because it’s a mighty useful piece of kit.

The only way around this at the moment (apart from breaking the law or arranging something with the doods in Redmond) is to distribute the C# or VB.Net project containing the T4 files so your customers can regenerate them onsite if they need to modify the output. Or use the Open Source / reverse engineered version mentioned on the Stackoverflow link above. I don’t think either is ideal, but it seems to be the best that can be done at the moment. I would love to hear otherwise!


Powered by WordPress