A Hello, World Test

So far, we have a simple library with a single function: get_greeting() and an application that makes use of it. How can we test it?

With dds, similar to generating applications, creating a test requires adding a suffix to a source filename stem. Instead of .main, simply add .test before the file extension.

A New Test Executable

We’ll create a test for our strings component, in a file named strings.test.cpp. We’ll use an assert to check our get_greeting() function:

<root>/src/hello/strings.test.cpp

#include <hello/strings.hpp>

int main() {
  if (hello::get_greeting() != "Hello world!") {
    return 1;
  }
}

If you run dds build once again, dds will generate a test executable and run it immediately. If the test executable exits with a non-zero exit code, then it will consider the test to have failed, and dds itself will exit with a non-zero exit code.

Important

dds executes tests in parallel by default! If the tests need access to a shared resource, locking must be implemented manually, or the shared resource should be split.

Note

dds builds and executes tests for every build by default. The *.test.cpp tests are meant to be very fast unit tests, so consider their execution time carefully.

If your code matches the examples so far, the above test will fail. Keen eyes will already know the problem, but wouldn’t it be better if we had better test diagnostics?

A test_driver: Using Catch2

dds ships with built-in support for the Catch2 C and C++ testing framework.

To make use of Catch as our test driver, we simply declare this intent in the package.json5 file at the package root:

<root>/package.json5

{
  name: 'hello-dds',
  version: '0.1.0',
  namespace: 'tutorial',
  test_driver: 'Catch-Main',
}

If you now run dds build, we will get a linker error for a multiply-defined main function. When setting the test_driver to Catch-Main, dds will compile an entrypoint separately from any particular test, and the tests will link against that entrypoint. This means we cannot provide our own main function, and should instead use Catch’s TEST_CASE macro to declare our test cases.

In addition to an entrypoint, dds provides a catch.hpp header that we may use in our tests, simply by #include-ing the appropriate path. We’ll modify our test to use the Catch test macros instead of our own logic. We’ll leave the condition the same, though:

<root>/src/hello/strings.test.cpp

#include <hello/strings.hpp>

#include <catch2/catch.hpp>

TEST_CASE("Check the greeting") {
  CHECK(hello::get_greeting() == "Hello world!");
}

Now running dds build will print more output that Catch has generated as part of test execution, and we can see the reason for the failing test:

[16:41:45] [error] Test <root>/_build/test/hello/strings failed! Output:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
strings is a Catch v2.10.2 host application.
Run with -? for options

-------------------------------------------------------------------------------
Check the greeting
-------------------------------------------------------------------------------
<root>/src/hello/strings.test.cpp:5
...............................................................................

<root>/src/hello/strings.test.cpp:5: FAILED:
  CHECK( hello::get_greeting() == "Hello world!" )
with expansion:
  "Hello, world!" == "Hello world!"

===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed

[dds - test output end]

Now that we have the direct results of the offending expression, we can much more easily diagnose the nature of the test failure. In this case, the function returns a string containing a comma , while our expectation lacks one. If we fix either the get_greeting or the expected string, we will then see our tests pass successfully and dds will exit cleanly.