Side-by-Side Tracing

Whenever you run into problems with SxS, e.g. because an assembly cannot be loaded because of missing dependencies or similar, using the tracing tool which SxS offers can help you a lot. Here’s how to use it.

Start Tracing

You start tracing by calling the following command:

sxstrace Trace –logfile:sxs.etl

This will keep the tracer running until you actually press enter. If you want to stop it through issuing another command, you should call

sxstrace Trace –logfile:sxs.etl -nostop

This will exit as soon as the tracing session was created.

Stop Tracing

In case you were using the nostop option, you’ll have to stop tracing through the following command.

sxstrace StopTrace

Transforming the Traces

As with most traces, you’ll need to transform the traces from the raw format in the ETL file to a human readable format. You’ll do this with the following command.

sxstrace Parse –logfile:sxs.etl –outfile:sxs.log

In the out file you should then be able to easily figure out if your problem is related to SxS or not.

Managed debugging with WinDbg

This is surely no surprise for many people: you can actually debug managed code with WinDbg (or CDB, if you prefer that one) by using the SOS extension (that ships with the .Net framework) in the debugger. What’s new is that today, Microsoft released Psscor2 which is pretty much an enhanced version of SOS. Now I’m sure that you’ll soon find much information on using Psscor2 on Tom’s blog. I’m much in favor of Psscor2 myself and therefore I’ll try to post useful scripts here as well (if I can come up with some ;-)).

PowerShell Script to trace OCS Components

Tracing OCS components may be vital in troubleshooting various issues you may face in your deployment. On machines where you have OCS components installed, you’ll typically find a tool called OCSLogger.exe which allows you to start/stop/view traces of OCS components. However, sometimes this is not enough, for instance when you see problems at the startup of a machine. It’s kind of hard to run the GUI if you cannot logon yet. But you can typically run a scheduled task. Or maybe you are — just like me — more like the console guy and thus want to have a script/cmdline tool for everything.

Let’s start with the config file used by the script (TraceConfig.xml) which defines the components you want to trace, to what level the traces are supposed to be and some more things. The sample given here traces mostly the components which are useful in troubleshooting issues related to the Response Group Service of OCS.

<?xml version="1.0" encoding="utf-8"?>
<Config>
    <!--
        Levels:
            TL_FATAL        1
            TL_ERROR        2
            TL_WARN         3
            TL_INFO         4
            TL_VERBOSE      5
            TL_NOISE        6

        Flags:
            TF_COMPONENT    0x00000001
            TF_PROTOCOL     0x00000002
            TF_CONNECTION   0x00000004
            TF_SECURITY     0x00000008
            TF_DIAG         0x00000010
            TF_AUTH         0x00000020
            TF_PARSE        0x00000040
            TF_NETWORK      0x00000080
            TF_STACKTRACE   0x00000100
    -->
    <Default Level="6" Flags="0xffff" />
    <Paths Tracer="C:\Program Files\Common Files\Microsoft Communications Server 2007 R2\Tracing"
           Etl="D:\Tracing"
           Log="D:\Tracing"
           TmfSearchPath="C:\Program Files\Common Files\Microsoft Communications Server 2007 R2\Tracing">
    </Paths>
    <Components>
        <Component Name="LcsWMI" Enabled="no" />
        <Component Name="LcsWMIUserServices" Enabled="no" />

        <Component Name="PowerShell" Enabled="yes" />

        <Component Name="ApplicationServer" Enabled="yes" />

        <Component Name="RgsClientsLib" Enabled="yes" />
        <Component Name="RgsCommonLibrary" Enabled="yes" />
        <Component Name="RgsDatastores" Enabled="yes" />
        <Component Name="RgsDeploymentApi" Enabled="yes" />
        <Component Name="RgsDeploymentLibrary" Enabled="yes" />
        <Component Name="RgsDiagnostics" Enabled="yes" />
        <Component Name="RgsHostingFramework" Enabled="yes" />
        <Component Name="RgsMatchMakingService" Enabled="yes" />
    </Components>
</Config>

I added the most importan trace levels and flags in the comment. Right now, the Default element defines the levels and flags for all components, but there’s no reason why you shouldn’t be able to do that per component you want to trace.

The PS1 script itself (Tracer.ps1) heavily relies on the OcsTracer.exe tool which also comes with OCS and is typically installed in the same place as OcsLogger.exe. It has four main actions:

  1. Start tracing components
  2. Stop tracing components and format the traces
  3. Format traces of ETL files (e.g. from a different machine)
  4. Show the configuration details from a particular config XML file
<#
.SYNOPSIS
        Starts or Stops tracing of Office Communications Server components.
.DESCRIPTION
        Starts or Stops tracing of Office Communications Server components.
.PARAMETER Action
        The action to perform. Must be one of 'Start', 'Stop', 'Config' or
        'Format'.
.PARAMETER ConfigPath
        The path to the configuration XML file. If not specified,
        "TraceConfig.xml" is used.
.LINK
        This script was originally posted to
        http://www.cymbeline.ch/post/2009/12/11/PowerShell-Script-to-trace-OCS-Components.aspx
.EXAMPLE
        .\Tracer.ps1 Start

        Starts tracing all the enabled components from the "TraceConfig.xml" file.
.EXAMPLE
        .\Tracer.ps1 Stop

        Stops tracing all the enabled components from the "TraceConfig.xml" file
        and formats the traces.
.EXAMPLE
        .\Tracer.ps1 Format "MyOtherConfig.xml"

        Formats the traces of the enabled components from the "MyOtherConfig.xml"
        file with all the settings from the "MyOtherConfig.xml" file.
.EXAMPLE
        .\Tracer.ps1 Config

        Shows the configuration of the "TraceConfig.xml" file.
#>
param(
    [Parameter(Mandatory=$true)]
    [ValidateSet("Start", "Stop", "Config", "Format", IgnoreCase=$true)]
    [String] $Action,
    [String] $ConfigPath = "TraceConfig.xml"
)

$configXml = ((Get-Content $ConfigPath))
$tracerPath = $configXml.Config.Paths.Tracer
$etlDir = $configXml.Config.Paths.Etl
$logDir = $configXml.Config.Paths.Log
$tmfSearchPath = $configXml.Config.Paths.TmfSearchPath

# Construct the parameters for the 'Start' command to OcsTracer.exe
function getStartParams()
{
    $ret = @()

    $configXml.Config.Components.Component |
        ? {$_.Enabled -eq "yes"} |
        foreach {
            $ret = $ret +
                ("/Component:" + $_.Name + "," + $configXml.Config.Default.Level +
                    "," + $configXml.Config.Default.Flags + " ")
        }

    return $ret
}

# Construct the parameters for the 'Stop' command to OcsTracer.exe
function getStopParams()
{
    $ret = @()

    $configXml.Config.Components.Component |
        ? {$_.Enabled -eq "yes"} |
        foreach { $ret = $ret + ("/Component:" + $_.Name) }

    return $ret
}

# Format the ETL files for enabled components to a human readable format
function formatFiles(
    [Parameter(Mandatory=$true)]
    [String] $Timestamp
)
{
    md $logDir\$timestamp -ea silentlycontinue | Out-Null

    $configXml.Config.Components.Component |
        ? {$_.Enabled -eq "yes"} |
        foreach {
            $etlFile = $_.Name + ".etl";

            if (Test-Path $etlFile)
            {
                $logFile = $Timestamp + "\" + $Timestamp + "_" + $_.Name + ".log";

                & "$tracerPath\OcsTracer.exe" Format /LogFilePath:"$etlDir\$etlFile" /OutputFile:"$logDir\$logFile" /TmfSearchPath:"$tmfSearchPath" | Write-Verbose
            }
            else
            {
                Write-Warning "File $etlFile not found.";
            }
        }
}

Write-Host "Using Config File: $ConfigPath"
$timestamp = Get-Date -format "yyyy-MM-dd_HH.mm.ss"

if ($Action -eq "start")
{
    Write-Host "Removing all .etl files ..."
    ls $etlDir *.etl | ri

    Write-Host "Start tracing components ..."
    $params = getStartParams

    & "$tracerPath\OcsTracer.exe" Start $params /LogFileFolder:"$etlDir" | Write-Verbose
}
elseif ($Action -eq "stop")
{
    Write-Host "Stop tracing components ..."
    $params = getStopParams

    md $logDir\$timestamp | Out-Null

    & "$tracerPath\OcsTracer.exe" Stop $params /OutputFile:"$logDir\$timestamp\$($timestamp)_All.log" /TmfSearchPath:"$tmfSearchPath" | Write-Verbose

    if (!$?)
    {
        rd $logDir\$timestamp | Out-Null
    }
    else
    {
        Write-Host "Sessions stopped. Start formatting ..."
        formatFiles $timestamp
    }
}
elseif ($Action -eq "format")
{
    Write-Host "Formatting traces from ETL files ..."
    formatFiles $timestamp
}
elseif ($Action -eq "config")
{
    Write-Host "Default values"
    Write-Host "--------------"
    $configXml.Config.Default | ft Level,Flags

    Write-Host "Paths"
    Write-Host "-----"
    $configXml.Config.Paths | fl

    Write-Host "Components"
    Write-Host "----------"
    $configXml.Config.Components.Component | ft Name,Enabled
}
else
{
    Write-Error "Unknown action."
}

For samples on how to run the script, please run man .\Tracer.ps1 -Examples
Have fun 🙂

Finding the Private Key File of Certificates

At work, I have created multiple tools which we used to analyse and fix issues related to certificates that we use with Office Communications Server and their respective private key files. To summarize, a user/service who wants to use the certificate for authentication needs to have read access on the private key. If it doesn’t, you’ll typically see a strange error which many people don’t relate to missing ACLs on the private key file.

Now these days, you don’t need to write such tools anymore. PowerShell allows you to pretty much do everything you need in this area. Let’s look at the following PS script (let’s call it FindPrivateKey.ps1) which accepts a parameter of type System.Security.Cryptography.X509Certificates.X509Certificate2, i.e. a reference to the certificate you want to analyze.

param(
    [Parameter(Mandatory=$true)]
    [System.Security.Cryptography.X509Certificates.X509Certificate2]
    $Certificate
)

echo "Looking for private key file of certificate"
echo $Certificate
echo ""
echo "The private key file is '$($Certificate.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName)'"
echo ""

$file = ls $env:userprofile -Filter $Certificate.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName -Recurse -Force -EA SilentlyContinue
echo "It is located at '$($file.FullName)'."

I guess I do not need to mention that you can easily find the certificate you’re interested in by running something like

pushd cert:\CurrentUser\My
ls
$cert = gci 0DAC31905AEB722D8561BFAF3F3BFD2F551AA197
popd
.\FindPrivateKey.ps1 $cert

where ‘0DAC31905AEB722D8561BFAF3F3BFD2F551AA197’ is simply the thumbprint of the certificate we’re interested in. From here on it should be easy to check what ACLs the file has (run $file.GetAccessControl()) and to modify them (run $file.SetAccessControl()).