Data-driven testing with MSTest via configuration-defined data sources and XML as a data source

I've done data driven testing with MSTest before. Prior to now, it's been solely with XML file data sources that have been defined inline in the [DataSource] attribute on my test.

I wanted to try using the app.config to define my data sources so that I could make them dynamic and configure them at test execution time so that I could point my tests at different environments.

I started following this tutorial on MSDN. It shows how to define data sources in the app.config for the tests. Then I followed the guidance in this post on the MSDN forums for how configure XML data sources in the app.config. Once I did that, I set up my XML data file similarly to the following:

<?xml version="1.0" encoding="utf-8" ?>
<Rows>
<dev>
<ShouldExist>True</ShouldExist>
<ProjectNumber>1721202</ProjectNumber>
</dev>
<dev>
<ShouldExist>False</ShouldExist>
<ProjectNumber>00000</ProjectNumber>
</dev>
</Rows>

The second-level "dev" elements are the "data table" in the connection parameters for the data source. By naming these appropriately, I can now select the environment in which to run my tests as part of my automated builds (yay ALM).

Resolving "The agent process was stopped while the test was running" on a TFS build agent

I'm currently working on a project that uses TFS for automated builds. Recently, our builds started failing with the error "The agent process was stopped while the test was running" while running unit tests. It wouldn't always happen during the same test, though it did frequently.

After inspecting the Event Viewer on the build machines, we found an error in the .NET Runtime that was taking down the build agent:

Application: QTAgent32_40.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: Moq.MockException

Stack:

   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<ThrowAsync>b__5(System.Object)

   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Object)

   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()

   at System.Threading.ThreadPoolWorkQueue.Dispatch()

   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

The exception that was being thrown on a ThreadPoolThread (always extremely bad, never good) was a MockException: our unit testing framework was throwing an exception that was taking down the build agent.  But where?

How We Solved It

By some miracle, one of our developers decided to run our problematic tests with mstest on the command line on his local machine. As it turns out, this was great because it showed the stack traces for the loose threads on the command line. Turns out we had a lot more loose threads than just the one that was taking down our test agent on the build machine. The command the developer used was (similar to) the following:

mstest.exe /testcontainer:"C:\path\to\my\test.dll" /noisolation

After auditing all of the errors by rooting them out with command line runs of mstest, the solutions to our problems all boiled down to one thing:

*ALWAYS* wrap the contents of 'async void' methods with a try-catch block!

System.ArgumentException: Unable to find the requested .Net Framework Data Provider when using SQLite with MSTest unit tests in Visual Studio 2012

I recently encountered the exception in the post title while migrating solutions from Visual Studio 2010 to Visual Studio 2012. Going back and forth between versions, the problem only occurred for me in Visual Studio 2012, it otherwise did not appear in 2010. I ran across this post on StackOverflow about another developer in a similar circumstance, but I didn't want to depend on whatever machine I was using having the SQLite Db Provider installed in the GAC, so I chose a different route: I added [DeploymentItem]s for each of SQLite.Interop.dll, System.Data.SQLite.dll and System.Data.SQLite.Linq.dll to the top of each of my test suites that require them.  It's perhaps not an ideal solution, but it does seem to be the most pragmatic one for my circumstances.