Getting the ETW messages for a live running instance of a Service Fabric application

As our company has recently discovered, it would be very handy to have a live running stream of diagnostic from a live running Service Fabric application when one is trying to figure out why a Service won't start.

This docs.microsoft.com article provides a handy way to enable the log streaming feature in Visual Studio for a live running service in the same manner as one would get when debugging the Service Fabric Application in your local cluster.

Finally ... how to debug the startup of a Windows Service application

Do what this guy says: http://einaregilsson.com/run-windows-service-as-a-console-program/

Pasted here for posterity, here's the example of how to write a Windows Service such that it can execute in a console and a developer can debug the startup routine:

using System;

using System.ServiceProcess;

 

public partial class DemoService : ServiceBase

{

    static void Main(string[] args)

    {

        DemoService service = new DemoService();

 

        if (Environment.UserInteractive)

        {

            service.OnStart(args);

            Console.WriteLine("Press any key to stop program");

            Console.Read();

            service.OnStop();

        }

        else

        {

            ServiceBase.Run(service);

        }

 

    }

    public DemoService()

    {

        InitializeComponent();

    }

 

    protected override void OnStart(string[] args)

    {

        // TODO: Add code here to start your service.

    }

 

    protected override void OnStop()

    {

        // TODO: Add code here to perform any tear-down

        //necessary to stop your service.

    }

}

Running and debugging Azure WebJobs locally on your development machine

Check out this article: https://github.com/Azure/azure-webjobs-sdk/wiki/Running-Locally

It shows you how to locally run and debug Azure WebJobs, which as it turns out is extremely handy because you can interact with all the Queues, Tables, Blobs etc as you normally would and get a full debugging environment.

Resolving "The agent process was stopped while the test was running" on a TFS build agent

I'm currently working on a project that uses TFS for automated builds. Recently, our builds started failing with the error "The agent process was stopped while the test was running" while running unit tests. It wouldn't always happen during the same test, though it did frequently.

After inspecting the Event Viewer on the build machines, we found an error in the .NET Runtime that was taking down the build agent:

Application: QTAgent32_40.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: Moq.MockException

Stack:

   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<ThrowAsync>b__5(System.Object)

   at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Object)

   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()

   at System.Threading.ThreadPoolWorkQueue.Dispatch()

   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

The exception that was being thrown on a ThreadPoolThread (always extremely bad, never good) was a MockException: our unit testing framework was throwing an exception that was taking down the build agent.  But where?

How We Solved It

By some miracle, one of our developers decided to run our problematic tests with mstest on the command line on his local machine. As it turns out, this was great because it showed the stack traces for the loose threads on the command line. Turns out we had a lot more loose threads than just the one that was taking down our test agent on the build machine. The command the developer used was (similar to) the following:

mstest.exe /testcontainer:"C:\path\to\my\test.dll" /noisolation

After auditing all of the errors by rooting them out with command line runs of mstest, the solutions to our problems all boiled down to one thing:

*ALWAYS* wrap the contents of 'async void' methods with a try-catch block!