{"id":5901,"date":"2024-04-29T07:00:16","date_gmt":"2024-04-29T14:00:16","guid":{"rendered":"http:\/\/writeasync.net\/?p=5901"},"modified":"2024-04-28T22:40:03","modified_gmt":"2024-04-29T05:40:03","slug":"testing-static_assert-cmake","status":"publish","type":"post","link":"https:\/\/writeasync.net\/?p=5901","title":{"rendered":"Testing static_assert (CMake)"},"content":{"rendered":"

One of the many innovations in C++11 was static_assert<\/a>. This allowed, at long last, a custom error message at compile time. Sure, we’ve had the #error<\/code> directive<\/a> for a while, but that only works for conditions that one could evaluate at preprocessing time. On the contrary, static_assert<\/code> can (with enough metaprogramming<\/a>) see anything the compiler can see. Here is an example from WIL<\/a> where static_assert<\/code> provides a clear, unambiguous error (instead of an inscrutable template error<\/a>):<\/p>\n

\r\n\/\/ Blocking get_value_string template types that are not already specialized - this gives a much friendlier compiler error message\r\ntemplate <typename T>\r\nT get_value_string(HKEY \/*key*\/, _In_opt_ PCWSTR \/*subkey*\/, _In_opt_ PCWSTR \/*value_name*\/)\r\n{\r\n    static_assert(sizeof(T) != sizeof(T), "Unsupported type for get_value_string");\r\n}\r\n<\/pre>\n
Yes, static_assert<\/code> is a great feature — one which may help us get ever-closer to the ideal of making illegal states unrepresentable<\/a>. This is especially valuable for library development; we would much prefer that a misuse of a feature simply fail to compile rather than blow up at runtime<\/a>.<\/p>\n
Ah, but there is the conundrum. With runtime errors, we can easily write unit tests to assert the error behavior<\/a>. But static_assert<\/code> generates compiler<\/em> errors; how are we supposed to test for those?<\/p>\n
As it turns out, this question<\/a> has come up<\/a> more than a few times<\/a>. There is even a related cottage industry of sorts on how to achieve unit testing<\/a> at compile time<\/a>. But let’s not stray too far from our goal here — what we would like is for a static_assert<\/code> in our code to be explicitly tested just like any other feature.<\/p>\n
By far the best treatment of this subject is by Dr. Roland Bock<\/a> who gave the CppCon 2016 talk “How to test static_assert.”<\/a> His solution is somewhat involved but works quite well for his use case in his sqlpp11<\/a> library. In short, he recommends a particular formulation of static_assert<\/code> which provides predictable cross-platform behavior at the cost of a bit template magic (see for yourself in portable_static_assert.h<\/a>). This is a great solution, but I actually want to explore the option he rejected at the front of his talk — I want to test my static_assert<\/code>s with the compiler but in a way that fits better with my unit tests.<\/p>\n
Right off the bat, we need to be clear that any “test” of static_assert<\/code> is going to involve invoking the compiler. In particular, we need to provide the compiler with a program that should fail<\/em> compilation, and that failure is success<\/em> for the static_assert<\/code> test. Further, we want to know that the failure is really the one we expect; it is pretty easy to write a program that fails for any one of a million reasons but not the<\/em> reason that we care about.<\/p>\n
Let’s start with a contrived example of a static assertion — maybe not so useful in real life but good enough for demonstration purposes:<\/p>\n
\r\nstd::string get_name(const char* raw);\r\n\r\ntemplate <typename T>\r\nclass Example\r\n{\r\n    static_assert(std::is_same_v<decltype(T::name), const char*>, "Type must have 'name' field of type `const char*`");\r\n\r\npublic:\r\n    Example(const T& input) : m_name{ get_name(input.name) }\r\n    {}\r\n\r\n    std::string operator()() const\r\n    {\r\n        return "Hello, " + m_name + "!";\r\n    }\r\n\r\nprivate:\r\n    std::string m_name;\r\n};\r\n<\/pre>\n
In this imaginary library, we have a template class Example<\/code> which assumes that the type T<\/code> has a const char* name<\/code> field. Regardless of the assertion, the code would<\/em> fail to compile if we break this assumption (when trying to evaluate the input.name<\/code> expression). But let’s say we just like our custom error message better (\"Type must have 'name' field of type `const char*`\"<\/code>). (And yes, we could use C++20 concepts<\/a> for this, but bear with me here.)<\/p>\n
Here are some unit tests for this type, which are limited to testing runtime behavior only:<\/p>\n
\r\n\/\/ stassert_test.cpp\r\n\/\/ ...\r\n\r\nstruct MyVal\r\n{\r\n    const char* name{};\r\n};\r\n\r\nTEST(example_test, has_name)\r\n{\r\n    MyVal val{ .name = "world" };\r\n    Example<MyVal> hello{val};\r\n\r\n    ASSERT_EQ("Hello, world!", hello());\r\n}\r\n\r\nTEST(example_test, has_null_name)\r\n{\r\n    MyVal val{};\r\n    Example<MyVal> hello{ val };\r\n\r\n    ASSERT_EQ("Hello, <null>!", hello());\r\n}\r\n<\/pre>\n
What would be nice is if I could add the following test and somehow validate that it fails while compiling:<\/p>\n
\r\nstruct WrongType\r\n{\r\n    std::string name{};\r\n};\r\n\r\nTEST(example_test, name_has_wrong_type)\r\n{\r\n    WrongType val{};\r\n\r\n    Example<WrongType> hello{ val };\r\n}\r\n<\/pre>\n
Of course, if I add this code, my entire test suite will fail to compile (for static_assert<\/code> reasons). To get anywhere with this approach, I need conditional compilation<\/a>. Let’s enlist the help of the much-maligned preprocessor<\/a>:<\/p>\n
\r\n#ifdef STASSERT_NAME_HAS_WRONG_TYPE\r\nstruct WrongType\r\n{\r\n    std::string name{};\r\n};\r\n\r\nTEST(example_test, name_has_wrong_type)\r\n{\r\n    WrongType val{};\r\n\r\n    Example<WrongType> hello{ val };\r\n}\r\n#endif\r\n<\/pre>\n
Now I can have the rest of the test suite compile while hiding this code section, as long as no one has defined STASSERT_NAME_HAS_WRONG_TYPE<\/code>. The next bit of machinery we need is some sort of compiler invocation that #define<\/code>s this symbol and spits out the expected error message. CMake actually does have a try_compile<\/a> feature that seems like it could be useful here; in my exploration of it, though, I could not get it to work in a simple enough way to recommend it.<\/p>\n
Instead, let’s try to capture the compiler command that we already used to build our test suite. This is actually a feature of CMake, enabled with CMAKE_EXPORT_COMPILE_COMMANDS<\/a>:<\/p>\n
\r\n# put this at the top of the relevant CMakeLists.txt file\r\nset(CMAKE_EXPORT_COMPILE_COMMANDS true)\r\n<\/pre>\n
All the relevant commands would thus be saved in a compile_commands.json<\/code> file in the CMAKE_BINARY_DIR<\/code>. But we need some other process to interpret the command and invoke the modified build step to trigger our static_assert<\/code>. We’re going to need JSON parsing, text wrangling, and handling an external process — clearly a job for an advanced scripting system. If we want to be relatively cross-platform, we can write a PowerShell Core<\/a> script-based cmdlet:<\/p>\n
\r\n# Test-StaticAssert.ps1\r\n[CmdletBinding()]\r\nparam(\r\n\t[Parameter(Mandatory=$true)]\r\n\t[string]$Symbol,\r\n\t[Parameter(Mandatory=$true)]\r\n\t[string]$Pattern,\r\n\t[Parameter(Mandatory=$true)]\r\n\t[string]$WorkingDirectory,\r\n\t[Parameter(Mandatory=$false)]\r\n\t[string]$CompileCommands = 'compile_commands.json',\r\n\t[Parameter(Mandatory=$false)]\r\n\t[string]$TestFile = 'stassert_test.cpp'\r\n)\r\n\r\nif (!(Test-Path $WorkingDirectory)) {\r\n\tthrow "Working directory '$WorkingDirectory' not found"\r\n}\r\n\r\nPush-Location $WorkingDirectory\r\nWrite-Host "Set working directory '$WorkingDirectory'"\r\n\r\nif (!(Test-Path $CompileCommands)) {\r\n    throw "Compile commands file '$CompileCommands' not found"\r\n}\r\n\r\n$item = Get-Content $CompileCommands |\r\n    ConvertFrom-Json | Where-Object { $_.file -like "*$TestFile"}\r\n\r\nif (!$item) {\r\n    throw "Could not find compile command for test file '$TestFile'"\r\n}\r\n\r\n$command = $item.command -replace '@\\S+\\.modmap ', ''\r\n$command += " \/D$Symbol"\r\nWrite-Host "Invoking compilation command: $command"\r\n$output = Invoke-Expression $command\r\nif ($LASTEXITCODE -eq 0) {\r\n    throw "Compilation unexpectedly succeeded for '$Symbol'"\r\n}\r\n\r\nWrite-Host "Output: $output"\r\n\r\n$matched = $false\r\n$output | Select-String -Pattern $Pattern | ForEach-Object {\r\n    $matched = $true\r\n    Write-Host "Found matching output line for '$Symbol': $_"\r\n}\r\n\r\nif (!$matched) {\r\n    throw "Did not find output line for '$Symbol' matching '$Pattern'"\r\n}\r\n<\/pre>\n
The script here reads the relevant command from the .json file, executes a modified version of it (filtering out some sort of module map<\/a> response file that caused me trouble in practice), and tries to match the output with an expected pattern. Importantly, it fails if the compile is successful (remember, we expect a compilation error!) or if there is no matching output message.<\/p>\n
The last step is to incorporate this “test” into the CMake build. Here is one way:<\/p>\n
\r\n# our normal test suite\r\nadd_executable(stassert-test\r\n    "test\/stassert_test.cpp"\r\n)\r\n\r\n# . . .\r\n# our static_assert test infra\r\nfunction(add_stassert_test symbol err_pattern)\r\n    set(test_script "${CMAKE_CURRENT_SOURCE_DIR}\/test\/Test-StaticAssert.ps1")\r\n    string(CONCAT test_script_args\r\n        " -Symbol ${symbol}"\r\n        " -Pattern \\"${err_pattern}\\""\r\n        " -WorkingDirectory ${CMAKE_BINARY_DIR}"\r\n    )\r\n    add_custom_command(TARGET stassert-test POST_BUILD\r\n        COMMAND pwsh ${test_script} ${test_script_args}\r\n    )\r\nendfunction()\r\n\r\n# the first static_assert test!\r\nadd_stassert_test(STASSERT_NAME_HAS_WRONG_TYPE "Type must have 'name' field of type `const char\\\\*`")\r\n<\/pre>\n
We are using add_custom_command<\/a> to incorporate a post-build step for the test executable. The command simply invokes the script above based on a preprocessor symbol and the error pattern we expect.<\/p>\n
Amazingly, this actually works, as shown in the CMake build output:<\/p>\n
\r\n[47\/48] Linking CXX executable src\\stassert\\stassert-test.exe\r\nSet working directory '...\/writeasync-cpp\/out\/build\/x64-release'\r\nInvoking compilation command: ...\\VC\\Tools\\MSVC\\1439~1.335\\bin\\Hostx64\\x64\\cl.exe  \/nologo \/TP -DGTEST_LINKED_AS_SHARED_LIBRARY=1 -I...\\writeasync-cpp\\src\\stassert\\inc -external:I...\\writeasync-cpp\\out\\build\\x64-release\\vcpkg_installed\\x64-windows\\include -external:W0 \/DWIN32 \/D_WINDOWS \/GR \/EHsc \/O2 \/Ob2 \/DNDEBUG -std:c++20 -MD \/W4 -WX \/Fosrc\\stassert\\CMakeFiles\\stassert-test.dir\\test\\stassert_test.cpp.obj \/FdTARGET_COMPILE_PDB \/FS -c ...\\writeasync-cpp\\src\\stassert\\test\\stassert_test.cpp \/DSTASSERT_NAME_HAS_WRONG_TYPE\r\nOutput: stassert_test.cpp ...\\writeasync-cpp\\src\\stassert\\inc\\example.h(13): error C2338: static_assert failed: 'Type must have 'name' field of type `const char*`' ...\r\nFound matching output line for 'STASSERT_NAME_HAS_WRONG_TYPE': ...\\writeasync-cpp\\src\\stassert\\inc\\example.h(13): error C2338: static_assert failed: 'Type must have 'name' field of type `const char*`'\r\n<\/pre>\n
Most importantly, if we “break” the static_assert<\/code>, say, by erroneously allowing this code to compile, we get a clear error:<\/p>\n
\r\n  Exception: ...\\writeasync-cpp\\src\\stassert\\test\\Test-StaticAssert.ps1:38\r\n  Line |\r\n    38 |      throw "Compilation unexpectedly succeeded for '$Symbol'"\r\n       |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n  \r\n       | Compilation unexpectedly succeeded for 'STASSERT_NAME_HAS_WRONG_TYPE'\r\n<\/pre>\n
In fact, we can even use TDD now for static_assert<\/code>! Let’s say we want to also assert that we cannot handle objects of size 16. First, add a new test at the end of the CMake file with the expected symbol and error:<\/p>\n
\r\nadd_stassert_test(STASSERT_NAME_HAS_WRONG_SIZE "Type must not have size of 16")\r\n<\/pre>\n
This should fail the build step (as expected) because we haven’t added any of this code yet, and the test program would still compile. Now, let’s add the test program code:<\/p>\n
\r\n#ifdef STASSERT_NAME_HAS_WRONG_SIZE\r\nstruct WrongSize\r\n{\r\n    const char* name{};\r\n    const char* extra{};\r\n};\r\n\r\nTEST(example_test, name_has_wrong_size)\r\n{\r\n    WrongSize val{};\r\n\r\n    Example<WrongSize> hello{ val };\r\n}\r\n#endif\r\n<\/pre>\n
We are still in a failing state because the program as of now still compiles. Finally, we add the static_assert<\/code>:<\/p>\n
\r\ntemplate <typename T>\r\nclass Example\r\n{\r\n    static_assert(std::is_same_v<decltype(T::name), const char*>, "Type must have 'name' field of type `const char*`");\r\n    \/\/ new assert follows:\r\n    static_assert(sizeof(T) != 16, "Type must not have size of 16");\r\n\/\/ ...\r\n<\/pre>\n
At this point the build succeeds, which means our static_assert<\/code> works (i.e., fails the build) as expected. Check out the GitHub project with all of the associated code here: https:\/\/github.com\/brian-dot-net\/writeasync-cpp\/tree\/main\/src\/stassert<\/a><\/p>\n
One could certainly find many issues with this approach. For one, it is relatively heavyweight. A program with hundreds of static_assert<\/code> statements would need hundreds of compiler invocations to be fully tested. But this is simply the reality — the only true test of static_assert<\/code> is indeed a program that provably fails the assert. Perhaps it is not great to harvest build commands from CMake — maybe try_compile<\/code> has better odds of working in a fully cross-platform way. It’s a cool hack, but is it practical?<\/p>\n
Nevertheless, I thought this was an interesting jumping off point to TDD your asserts. What do you think?<\/p>\n","protected":false},"excerpt":{"rendered":"
One of the many innovations in C++11 was static_assert. This allowed, at long last, a custom error message at compile time. Sure, we’ve had the #error directive for a while, but that only works for conditions that one could evaluate… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[108,101,41,51],"tags":[],"class_list":["post-5901","post","type-post","status-publish","format-standard","hentry","category-build","category-native","category-tdd","category-testing"],"_links":{"self":[{"href":"https:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/5901","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5901"}],"version-history":[{"count":2,"href":"https:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/5901\/revisions"}],"predecessor-version":[{"id":5903,"href":"https:\/\/writeasync.net\/index.php?rest_route=\/wp\/v2\/posts\/5901\/revisions\/5903"}],"wp:attachment":[{"href":"https:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5901"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5901"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/writeasync.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5901"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}