<?xml version="1.0"?>
<page title="Changing Cgreen Reporting" here="Cgreen reporting">
    <long_title>
        Changing the reporting mechanism of Cgreen.
    </long_title>
    <content>
        <section name="extending" title="Replacing the reporter">
            <p>
                In every test suite so far, we have run the tests
                with this line...
<c><![CDATA[
return run_test_suite(our_tests(), <strong>create_text_reporter()</strong>);
]]></c>
                We can change the reporting mechanism just by changing this
                method.
                Here is the code for <code>create_text_reporter()</code>...
<c><![CDATA[
TestReporter *create_text_reporter() {
    TestReporter *reporter = create_reporter();
    reporter->start = &text_reporter_start;
    reporter->finish = &text_reporter_finish;
    reporter->show_fail = &show_fail;
    reporter->show_incomplete = &show_incomplete;
    return reporter;
}
]]></c>
                The <code>TestReporter</code> structure contains function
                pointers that control the reporting.
                When called from <code>create_reporter()</code> constructor, these
                pointers are set up with functions that display nothing.
                The text reporter code replaces these with something more
                dramatic, and then returns a pointer to this new object.
                Thus the <code>create_text_reporter()</code> function effectively
                extends the object from <code>create_reporter()</code>.
            </p>
            <p>
                The text reporter only outputs content at the start of the first test,
                at the end of the test run to display the results, when a failure
                occours, and when a test fails to complete.
                A quick look at the <em>text_reporter.c</em> file in <em>Cgreen</em>
                reveals that the overrides just output a message and
                chain to the versions in <em>reporter.h</em>.
            </p>
            <p>
                To change the reporting mechanism ourselves, we just have to know a little
                about the methods in the <code>TestReporter</code> structure.
            </p>
        </section>
        <section name="base" title="The TestReporter structure">
            <p>
                The Cgreen <code>TestReporter</code> is a pseudo class that looks
                something like...
<c><![CDATA[
typedef struct _TestReporter TestReporter;
struct _TestReporter {
    void (*destroy)(TestReporter *);
    void (*start)(TestReporter *, const char *);
    void (*finish)(TestReporter *, const char *);
    void (*show_pass)(TestReporter *, const char *, int, const char *, va_list);
    void (*show_fail)(TestReporter *, const char *, int, const char *, va_list);
    void (*show_incomplete)(TestReporter *, const char *);
    void (*assert_true)(TestReporter *, const char *, int, int, const char *, ...);
    int passes;
    int failures;
    int exceptions;
    void *breadcrumb;
    int ipc;
    void *memo;
};
]]></c>
                The first block are the methods that can be overridden.
                <dl>
                    <dt><code>void (*destroy)(TestReporter *)</code></dt>
                    <dd>
                        This is the destructor for the default structure.
                        If this is overridden, then the overriding function
                        must call <code>destroy_reporter(TestReporter *reporter)</code>
                        to finish the clean up.
                    </dd>
                    <dt><code>void (*start)(TestReporter *, const char *)</code></dt>
                    <dd>
                        The first of the callbacks.
                        At the start of each test suite <em>Cgreen</em> will
                        call this method on the reporter with the name of the
                        suite being entered.
                        The default version keeps track of the stack of tests
                        in the <code>breadcrumb</code> pointer of
                        <code>TestReporter</code>.
                        If you make use of the breadcrumb functions, as the defaults
                        do, then you will need to call <code>reporter_start()</code> to
                        keep the book keeping in sync.
                    </dd>
                    <dt><code>void (*finish)(TestReporter *, const char *)</code></dt>
                    <dd>
                        The counterpart to the <code>(*start)()</code> call called
                        on leaving the test suite.
                        It needs to be chained to the <code>reporter_finish()</code>
                        to keep track of the breadcrumb book keeping.
                        The text reporter uses the state of the breadcrumb to decide
                        if it is ending teh top level test.
                        If so, it prints the familiar summary of passes and fails.
                    </dd>
                    <dt><code>void (*show_pass)(TestReporter *, const char *, int, const char *, va_list)</code></dt>
                    <dd>
                        This method is initially empty, so there is no need to chain
                        the call to any other function.
                        Besides the pointer to the reporter structure, <em>Cgreen</em>
                        also passes the file name of the test, the line number of failed
                        assertion, the message to show and any additional parameters to
                        substitute into the message.
                        The message comes in as <code>printf()</code> style format string,
                        and so the variable argument list should match the substitutions.
                    </dd>
                    <dt><code>void (*show_fail)(TestReporter *, const char *, int, const char *, va_list)</code></dt>
                    <dd>
                        The partner of <code>show_pass()</code>, and the one you'll likely
                        overload first.
                    </dd>
                    <dt><code>void (*show_incomplete)(TestReporter *, const char *)</code></dt>
                    <dd>
                        When a test fails to complete, this is the handler that is called.
                        As it's an unexpected outcome, no message is received, but
                        we do get the name of the test.
                        The text reporter combines this with the breadcrumb to produce the
                        exception report.
                    </dd>
                    <dt><code>void (*assert_true)(TestReporter *, const char *, int, int, const char *, ...)</code></dt>
                    <dd>
                        This is not normally overridden and is really internal.
                        It is the raw entry point for the test messages from the test suite.
                        By default it dispatches teh call to either <code>show_pass()</code>
                        or <code>show_fail()</code>.
                    </dd>
                </dl>
                The second block is simply resources and book keeping that the reporter
                can use to liven up the messages...
                <dl>
                    <dt><code>passes</code></dt>
                    <dd>The number of passes so far.</dd>
                    <dt><code>failures</code></dt>
                    <dd>The number of failures generated so far.</dd>
                    <dt><code>exceptions</code></dt>
                    <dd>The number of test functions that have failed to complete so far.</dd>
                    <dt><code>breadcrumb</code></dt>
                    <dd>This is a pointer to the list of test names in the stack.</dd>
                </dl>
                The <code>breadcrumb</code> pointer is different and needs a
                little explanation.
                Basically it is a stack, analagous to the breadcrumb trail you
                see on websites.
                Everytime the <code>start()</code> handler is invoked, the
                name is placed in this stack.
                When the <code>finish()</code> message handler is invoked, a
                name is popped off.
            </p>
            <p>
                There are a bunch of utility functions in <em>cgreen/breadcrumb.h</em>
                that can read the state of this stack.
                Most useful are <code>get_current_from_breadcrumb()</code> which
                takes the breadcrumb pointer and returns the curent test name,
                and <code>get_breadcrumb_depth()</code> which gives the current
                depth of the stack.
                A depth of zero means that the test run has finished.
            </p>
            <p>
                If you need to traverse all the names in the breadcrumb,
                then you can call <code>walk_breadcrumb()</code>.
                Here is the full signature...
<c><![CDATA[
void walk_breadcrumb(Breadcrumb *breadcrumb, void (*walker)(const char *, void *), void *memo);
]]></c>
                The <code>void (*walker)(const char *, void *)</code> is a callback
                that will be passed the name of the test suite for each
                level of nesting.
                It is also poassed the <code>memo</code> pointer that was
                passed to the <code>walk_breadcrumb()</code> call.
                You can use this pointer for anything you want, as
                all <em>Cgreen</em> does is pass it from call to call.
                This is so aggregate information can be kept track of whilst
                still being reentrant.
            </p>
            <p>
                The last parts of the <code>TestReporter</code> structure are...
                <dl>
                    <dt><code>ipc</code></dt>
                    <dd>
                        This is an internal structure for handling the messaging between
                        reporter and test suite.
                        You shouldn't touch this.
                    </dd>
                    <dt><code>memo</code></dt>
                    <dd>
                        By contrast, this is a spare pointer for your own expansion.
                    </dd>
                </dl>
            </p>
        </section>
        <section name="example" title="An example XML reporter">
            <p>
                Let's make things real with an example.
                Suppose we want to send the output from <em>Cgreen</em> in XML format,
                say for storing in a repository or for sending across the network.
            </p>
            <p>
                Suppose also that we have come up with the following format...
<pre><![CDATA[
<?xml?>
<test name="Top Level">
    <test name="A Group">
        <test name="a_test_that_passes">
        </test>
        <test name="a_test_that_fails">
            <fail>
                <message><![CDATA[A failure]]>]]<![CDATA[></message>
                <location file="test_as_xml.c" line="8"/>
            </fail>
        </test>
    </test>
</test>
]]></pre>
                In other words a simple nesting of tests with only failures
                encoded.
                The absence of failure is a pass.
            </p>
            <p>
                Here is a test script, <em>test_in_xml.c</em> that we can use to construct the
                above output...
<c><![CDATA[
#include "cgreen/cgreen.h"

void <strong>a_test_that_passes()</strong> {
    assert_true(1);
}

void <strong>a_test_that_fails()</strong> {
    assert_true_with_message(0, <strong>"A failure"</strong>);
}

TestSuite *create_test_group() {
    TestSuite *suite = <strong>create_named_test_suite("A Group")</strong>;
    add_test(suite, a_test_that_passes);
    add_test(suite, a_test_that_fails);
    return suite;
}

int main(int argc, char **argv) {
    TestSuite *suite = <strong>create_named_test_suite("Top Level")</strong>;
    add_suite(suite, create_test_group());
    return run_test_suite(suite, <strong>create_text_reporter()</strong>);
}
]]></c>
                The text reporter is used just to confirm that everything
                is working.
                So far it is.
<sh><![CDATA[
Running "Top Level"...
Failure!: A Group -> a_test_that_fails -> A failure at [test_as_xml.c] line [8]
Completed "Top Level": 1 pass, 1 failure, 0 exceptions.
]]></sh>
            </p>
            <p>
                Our first move is to switch the reporter from text, to our
                not yet written XML version...
<c><![CDATA[
#include "cgreen/cgreen.h<strong>
#include "xml_reporter.h"</strong>

...

int main(int argc, char **argv) {
    TestSuite *suite = create_named_test_suite("Top Level");
    add_suite(suite, create_test_group());
    return run_test_suite(suite, <strong>create_xml_reporter()</strong>);
}
]]></c>
                We'll start the ball rolling with the <em>xml_reporter.h</em>
                header file...
<c><![CDATA[
#ifndef _XML_REPORTER_HEADER_
#define _XML_REPORTER_HEADER_
<strong>
#include "cgreen/reporter.h"

TestReporter *create_xml_reporter();</strong>

#endif
]]></c>
                ...and the simplest possible reporter in <em>reporter.c</em>.
<c><![CDATA[
#include "xml_reporter.h"
#include "cgreen/reporter.h"

TestReporter *create_xml_reporter() {
    TestReporter *reporter = create_reporter();
    return reporter;
}
]]></c>
                One that outputs nothing.
<sh><![CDATA[
gcc -c test_as_xml.c
gcc -c xml_reporter.c
gcc xml_reporter.o test_as_xml.o cgreen/cgreen.a -o xml
./xml
]]></sh>
                Yep, nothing.
            </p>
            <p>
                Let's add the outer test tags first, so that we can see
                <em>Cgreen</em> navigating the test suite...
<c><![CDATA[
#include "xml_reporter.h"
#include "cgreen/reporter.h"<strong>
#include <stdio.h>

static void xml_reporter_start(TestReporter *reporter, const char *name);
static void xml_reporter_finish(TestReporter *reporter, const char *name);</strong>

TestReporter *create_xml_reporter() {
    TestReporter *reporter = create_reporter();<strong>
    reporter->start = &xml_reporter_start;
    reporter->finish = &xml_reporter_finish;</strong>
    return reporter;
}
<strong>
static void xml_reporter_start(TestReporter *reporter, const char *name) {
    printf("<test name=\"%s\">\n", name);
    reporter_start(reporter, name);
}

static void xml_reporter_finish(TestReporter *reporter, const char *name) {
    reporter_finish(reporter, name);
    printf("</test>\n");
}</strong>
]]></c>
                Although chaining to the underlying <code>reporter_start()</code>
                and <code>reporter_finish()</code> functions is optional, I want to
                make use of some of the facilities later.
            </p>
            <p>
                Our output meanwhile, is making it's first tentative steps...
<sh><![CDATA[
<test name="Top Level">
<test name="A Group">
<test name="a_test_that_passes">
</test>
<test name="a_test_that_fails">
</test>
</test>
</test>
]]></sh>
                We don't want a passing message, so the <code>show_fail()</code> function is all we
                need...
<c><![CDATA[
...<strong>
static void xml_show_fail(TestReporter *reporter, const char *file, int line, const char *message, va_list arguments);</strong>

TestReporter *create_xml_reporter() {
    TestReporter *reporter = create_reporter();
    reporter->start = &xml_reporter_start;
    reporter->finish = &xml_reporter_finish;<strong>
    reporter->show_fail = &xml_show_fail;</strong>
    return reporter;
}

...
<strong>
static void xml_show_fail(TestReporter *reporter, const char *file, int line, const char *message, va_list arguments) {
    printf("<fail>\n");
    printf("\t<message><![CDATA[");
    vprintf(message, arguments);
    printf("]]>]]&gt;<![CDATA[</message>\n");
    printf("\t<location file=\"%s\" line=\"%d\"/>\n", file, line);
    printf("</fail>\n");
}</strong>
]]></c>
                We have to use <code>vprintf()</code> to handle the variable argument
                list passed to us.
                This will probably mean including the <em>stdarg.h</em> header
                as well as <em>stdio.h</em>.
            </p>
            <p>
                This gets us pretty close to what we want...
<sh><![CDATA[
<test name="Top Level">
<test name="A Group">
<test name="a_test_that_passes">
</test>
<test name="a_test_that_fails">
<fail>
    <message><![CDATA[A failure]]>]]&gt;<![CDATA[</message>
    <location file="test_as_xml.c" line="9"/></fail>
</test>
</test>
</test>
]]></sh>
                For completeness we should add a tag for an incomplete test.
                We'll output this as a failure, athough we don't get a location this
                time...
<c><![CDATA[
<strong>static void xml_show_incomplete(TestReporter *reporter, const char *name) {
    printf("<fail>\n");
    printf("\t<message><![CDATA[Failed to complete]]>]]&gt;<![CDATA[</message>\n");
    printf("</fail>\n");
}</strong>
]]></c>
                All that's left then is the XML declaration and the thorny issue of
                indenting.
                Although the indenting is not strictly necessary, it would make the
                output a lot more readable.
            </p>
            <p>
                The test depth is kept track of for us with the <code>breadcrumb</code>
                object in the <code>TestReporter</code> structure.
                We'll add an <code>indent()</code> function that outputs the
                correct number of tabs...
<c><![CDATA[
static indent(TestReporter *reporter) {
    int depth = <strong>get_breadcrumb_depth((Breadcrumb *)reporter->breadcrumb)</strong>;
    while (depth-- > 0) {
        printf("\t");
    }
}
]]></c>
                The <code>get_breadcrumb_depth()</code> function just gives the
                current test depth.
                As that is just the number of tabs to output, the implementation
                is trivial.
            </p>
            <p>
                We can then use this function in the rest of the code.
                Here is the complete listing...
<c><![CDATA[
#include "xml_reporter.h"
#include "cgreen/reporter.h"
#include "cgreen/breadcrumb.h"
#include <stdio.h>
#include <stdarg.h>
<strong>
static indent(TestReporter *reporter);</strong>
static void xml_reporter_start(TestReporter *reporter, const char *name);
static void xml_reporter_finish(TestReporter *reporter, const char *name);
static void xml_show_fail(TestReporter *reporter, const char *file, int line, const char *message, va_list arguments);
static void xml_show_incomplete(TestReporter *reporter, const char *name);

TestReporter *create_xml_reporter() {
    TestReporter *reporter = create_reporter();
    reporter->start = &xml_reporter_start;
    reporter->finish = &xml_reporter_finish;
    reporter->show_fail = &xml_show_fail;
    reporter->show_incomplete = &xml_show_incomplete;
    return reporter;
}
<strong>
static indent(TestReporter *reporter) {
    int depth = get_breadcrumb_depth((Breadcrumb *)reporter->breadcrumb);
    while (depth-- > 0) {
        printf("\t");
    }
}</strong>

static void xml_reporter_start(TestReporter *reporter, const char *name) {<strong>
    if (get_breadcrumb_depth((Breadcrumb *)reporter->breadcrumb) == 0) {
        printf("<?xml?>\n");
    }
    indent(reporter);</strong>
    printf("<test name=\"%s\">\n", name);
    reporter_start(reporter, name);
}

static void xml_reporter_finish(TestReporter *reporter, const char *name) {
    reporter_finish(reporter, name);
    <strong>indent(reporter);</strong>
    printf("</test>\n");
}

static void xml_show_fail(TestReporter *reporter, const char *file, int line, const char *message, va_list arguments) {
    <strong>indent(reporter);</strong>
    printf("<fail>\n");
    <strong>indent(reporter);</strong>
    printf("\t<message><![CDATA[");
    vprintf(message, arguments);
    printf("]]>]]&gt;<![CDATA[</message>\n");
    <strong>indent(reporter);</strong>
    printf("\t<location file=\"%s\" line=\"%d\"/>\n", file, line);
    <strong>indent(reporter);</strong>
    printf("</fail>\n");
}

static void xml_show_incomplete(TestReporter *reporter, const char *name) {
    <strong>indent(reporter);</strong>
    printf("<fail>\n");
    <strong>indent(reporter);</strong>
    printf("\t<message><![CDATA[Failed to complete]]>]]&gt;<![CDATA[</message>\n");
    <strong>indent(reporter);</strong>
    printf("</fail>\n");
}
]]></c>
                And finally the desired output...
<sh><![CDATA[
<?xml?>
<test name="Top Level">
    <test name="A Group">
        <test name="a_test_that_passes">
        </test>
        <test name="a_test_that_fails">
            <fail>
                <message><![CDATA[A failure]]>]]&gt;<![CDATA[</message>
                <location file="test_as_xml.c" line="9"/>
            </fail>
        </test>
    </test>
</test>
]]></sh>
                Job done.
            </p>
            <p>
                Possible other extensions include reporters that write to <em>syslog</em>,
                talk to IDE plug-ins, paint pretty printed documents or just return a boolean
                for monitoring purposes.
            </p>
        </section>
    </content>
    <internal>
        <link>
            <a href="#extending">Replacing and extending the reporting</a> from Cgreen.
        </link>
        <link>
            The <a href="#base">Cgreen TestReporter pseudo class</a>.
        </link>
        <link>
            An <a href="#example">example of an XML Cgreen reporter</a>.
        </link>
    </internal>
    <external>
        <link>
            The <a href="http://sourceforge.net/projects/cgreen">Cgreen SourceForge page</a>.
        </link>
        <link>
            Unit testing was popularised by <a href="http://extremeprogramming.org/">eXtreme Programming</a>.
        </link>
    </external>
    <meta>
        <keywords>
            unit testing, test-first, test first, code quality, automate unit testing,
            automated testing, c programming, c tools, c unit tests, unit testing in c,
            check unit tester, test coverage, TDD, extreme programming, agile development,
            tdd in C, c regression testing, cunit, cppunit, c unit testing frameworks, sourceforge,
            open source, open source framework, lesser gnu licence, lgpl, software testing,
            executable specification, software requirements, quality assurance,
            test driven design, software decoupling, XML reports, error logging, software monitoring
        </keywords>
    </meta>
</page>