Testing static_assert (MSBuild)

Spread the love

Now that we’ve gotten a handle on testing static_assert for CMake, let’s turn our focus to MSBuild. We will use a nearly identical project structure as what was established in the CMake stassert sample project but with all the of the, uh, “beauty” of .vcxproj and .sln files. (For good measure, we have also switched to VSTest for C++ unit tests.) Here is the result: StaticAssertSample – initial port from CMake

As before with CMake, we now need to cobble together a compiler command that would test each static_assert based on a conditional compilation symbol, for example:

#ifdef STASSERT_NAME_HAS_WRONG_TYPE
        struct WrongType
        {
            std::string name{};
        };

        TEST_METHOD(name_has_wrong_type)
        {
            WrongType val{};

            Example<WrongType> hello{ val };
        }
#endif

It turns out that, in this department, we are in luck — sort of. During a C++ project build, the compiler commands are indeed recorded, though mainly for the benefit of incremental build. The only drawback, as clearly laid out in the documentation for .tlog files (the underlying file format for tracking build inputs and outputs) is that command-line .tlog file contents are not specifically documented and “determined by the MSBuild task that produces them.”

However, it is pretty clear that the format is nearly identical to that of the “read” and “write” .tlog files which are documented and presumably stable. A typical command-line file might look like this (example adapted from the CL.command.1.tlog for the StaticAssertTest project):

^D:\SOME\PATH\PROJECTS\STATICASSERTSAMPLE\TEST\EXAMPLE_TEST.CPP
/c /I..\INC /I"C:\PROGRAM FILES\MICROSOFT VISUAL STUDIO\2022\ENTERPRISE\VC\UNITTEST\INCLUDE" /I"D:\OTHER\PATH\CPP\VCPKG\INSTALLED\X64-WINDOWS\INCLUDE" /ZI /JMC /nologo /W3 /WX- /diagnostics:column /sdl /Od /D _DEBUG /D _WINDLL /D _UNICODE /D UNICODE /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /std:c++20 /Fo"X64\DEBUG\\" /Fd"X64\DEBUG\VC143.PDB" /external:W3 /Gd /TP /FC D:\SOME\PATH\PROJECTS\STATICASSERTSAMPLE\TEST\EXAMPLE_TEST.CPP

The first line is a caret (^) followed by the full path the .cpp source file. The second line is the full argument list for CL.exe. While it may not be contractual, it seems simple enough to throw together a prototype where we assume this format. So let’s press on!

At the end of our test project, we can add some custom items and an associated targets import:

  <ItemGroup>
    <StaticAssertTest Include="STASSERT_NAME_HAS_WRONG_TYPE">
      <TestFile>example_test.cpp</TestFile>
      <Pattern>Type must have 'name' field of type `const char\*`</Pattern>
    </StaticAssertTest>
    <StaticAssertTest Include="STASSERT_NAME_HAS_WRONG_SIZE">
      <TestFile>example_test.cpp</TestFile>
      <Pattern>Type must not have size of 16</Pattern>
    </StaticAssertTest>
  </ItemGroup>
  <Import Project="StaticAssertTest.targets" />

The items lay out a series of tests that we want to perform. The Include (the “identity” metadata component) is the #define symbol; exactly one of these defines a unique test. The metadata items TestFile and Pattern specify the related information for the test — which source file it is associated with and the expected static_assert error pattern, respectively. Now we need our custom .targets file:

<Project>
  <Target Name="StaticAssertTest" AfterTargets="ClCompile">
    <ItemGroup>
      <CLCommandFile Include="$(TLogLocation)CL.command.*.tlog" />
      <StaticAssertTestCommand Include="@(StaticAssertTest)">
        <Args>-Symbol "%(Identity)" -Pattern "%(Pattern)" -TestFile "%(TestFile)"</Args>
      </StaticAssertTestCommand>
    </ItemGroup>
    <Exec Command="powershell.exe -File Test-StaticAssert.ps1 -ClExe &quot;$(ClCompilerPath)&quot; %(StaticAssertTestCommand.Args) -CommandFile &quot;@(CLCommandFile->'%(fullpath)', '&quot;,&quot;')&quot;" />
  </Target>
</Project>

This file defines one StaticAssertTest target which runs after the C++ compiler (ClCompile) target. It uses an item wildcard to select all the possible .tlog command files and another item to build up the script arguments we will need to run the test. We then simply hand off to powershell.exe to run our Test-StaticAssert.ps1 script to do the real work:

[CmdletBinding()]
param(
    [Parameter(Mandatory=$true)]
    [string]$ClExe,
    [Parameter(Mandatory=$true)]
    [string]$Symbol,
    [Parameter(Mandatory=$true)]
    [string]$Pattern,
    [Parameter(Mandatory=$true)]
    [string[]]$CommandFile,
    [Parameter(Mandatory=$true)]
    [string]$TestFile
)

Function Exit-Task($errorText) {
    Write-Host "$($PSCommandPath): error : $errorText"
    Exit 1
}

$clArgs = ''
$outPath = ''
foreach ($file in $CommandFile) {
    if (!(Test-Path $file)) {
        Exit-Task "Command file '$file' not found"
    }

    $lines = Get-Content $file
    # First line has the following format:
    # ^X:\SOME\FULL\PATH\FILE.CPP
    $cppFile = Split-Path ($lines[0].Substring(1)) -Leaf
    if ($cppFile -eq $TestFile) {
        # Second line has actual args to CL.exe
        $clArgs = $lines[1]
        break
    }
}

if (!$clArgs) {
    Exit-Task "Could not find CL args for test file '$TestFile'"
}

$clArgs += " /D$Symbol"
$cmdText = "`"$ClExe`" $clArgs"
Write-Host $cmdText
# The CL.exe path (and possibly some arguments) will be quoted. This causes
# trouble for invoking the expression directly, so invoke via cmd.exe.
$output = & cmd.exe /c $cmdText
if ($LASTEXITCODE -eq 0) {
    Exit-Task "Compilation unexpectedly succeeded for '$Symbol'"
}

$matched = $false
$output | Select-String -Pattern $Pattern | ForEach-Object {
    $matched = $true
    Write-Host "Found matching output line for '$Symbol': $_"
}

if (!$matched) {
    $output | Write-Host
    Exit-Task "Did not find output line for '$Symbol' matching '$Pattern'"
}

The script is not too different from the one we built for CMake, though with some MSBuild adaptations. For one, it tries to produce an error output that would be understood better by MSBuild which helps highlight the actual problem when something goes wrong. Also, since these command lines can have paths with spaces, there is a lot of argument quoting needed. This is somewhat of a minefield in PowerShell but I found success by just passing off to a “cmd.exe /c” expression via the call operator (&).

The end result is a set of post-build static_assert “tests” that run equally well on the msbuild.exe command line or in the Visual Studio IDE. A failing test has an output like the following:

"D:\some\path\projects\StaticAssertSample\StaticAssertSample.sln" (default target) (1) ->
"D:\some\path\projects\StaticAssertSample\test\StaticAssertTest.vcxproj" (default target) (4) ->
(StaticAssertTest target) ->
  D:\some\path\projects\StaticAssertSample\INC\example.h(13,33): error C2338: static_assert failed: 'Type must not have size of 16' [D:\some\path\projects\StaticAssertSample\test\StaticAssertTest.vcxproj]
  D:\some\path\projects\StaticAssertSample\test\Test-StaticAssert.ps1 : error : Did not find output line for 'STASSERT_NAME_HAS_WRONG_SIZE' matching 'Type must not have size of 99' [D:\some\path\projects
\StaticAssertSample\test\StaticAssertTest.vcxproj]
  D:\some\path\projects\StaticAssertSample\test\StaticAssertTest.targets(9,5): error MSB3073: The command "powershell.exe -File Test-StaticAssert.ps1 -ClExe "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\cl.exe" -Symbol "STASSERT_NAME_HAS_WRONG_SIZE" -Pattern "Type must not have size of 16" -TestFile "example_test.cpp" -CommandFile "some\path\projects\StaticAssertSample\test\x64\Debug\StaticAssertTest.tlog\CL.command.1.tlog"" exited with code 1. [D:\some\path\projects\StaticAssertSample\test\StaticAssertTest.vcxproj]

Yes, it’s somewhat lengthy, but the error condition is front and center.

If you have a need for static_assert validation and you use Visual Studio with .vcxproj files, this might be a good starting point (no promises, though!). All of the code for this sample can be found here: StaticAssertSample on GitHub

Leave a Reply

Your email address will not be published. Required fields are marked *