Enhanced Unit Testing

Sorry, I meant to say, 'I don't think an _extended_ unit test module is out of place'. :)

How do you think that this module compares to typical frameworks from other languages like JUnit, TestNG, CppUnit (?) or others?

How easy would it be to extend it further, e.g. with something like the DMocks library?

As for the source itself - the output shouldn't go to Stdout by default, it takes away the full utility of it as a library module.

I suppose it would be fairly easy to modify the example code to take a Print object to control output with.

Compared to jUnit, this example is fairly similar. The both allow simple adding of tests to a test suite, and a fairly simple run that defaults to stdout. jUnit also provides some customizable output.

jUnit includes some concepts that are not included in this example. jUnit Fixtures simplify writing several unit tests that utilize the same data. A fixture is essentially a class with a constructor that sets up it's class variables, and this class can be passed to multiple tests. You can accomplish the same thing with a Test unit by simply including all tests that want to use the common data in an enclosing class. This method also has the advantage of being able to have subsequent tests dependant on the results of previous. Because you can accomplish the same goal using language features, I don't think Fixtures are a concept that adds any test functionality.

jUnit also allows you to specify return values as 'expected', for example, if you are testing for an exception to be thrown, you can specify that, for testX, exceptionA is expected and is the 'success' condition. The same thing can be accomplished with this example, because each test controls it's own success or failure output. So a test can catch an exception, compare it against what it expects, and return success or failure as appropriate, which seems a bit more intuitive (the test itself knows what it's success or failure conditions are, not the test add parameter). This also keeps the success and failure conditions close to each other rather that split apart (a test far up in the code, with the test success/failure conditions far down in the adding).

So, compared to jUnit, this Test example seems quite similar, perhaps even arguably simpler to use, making use of language features to accomplish same goals.

Comparisons to TestNG and CppUnit? will follow.

TestNG provides a lot of features beyond the scope of this example Test. For example: -Test distribution to slaves -Tool and plug-in support -BeanShell?

Really, this example test is intended to cover Unit testing, whereas TestNG is quite broader. However, there are still some portions of Unit tested in TestNG that can be compared.

TestNG provides for the concept of test sub-grouping. That is, you can declare in a particular test suite that some tests are part of different groupings. You can then run only the tests in a particular group in a particular suite. The example Test does not provide for this. Really, I think this is a bit of bloat, the goal of which you can accomplish by simply creating two separate suites.

TestNG also provides for formatted output to html.

Compared to TestNG, this example Test is much simpler in scope. I think that much of the extended functionality of TestNG wouldn't quite belong in a standard library, but would be something that would be built on top of test features in the standard library.

CppUnit? is the same as jUnit, being a port, but it does have some differences which are mostly related to language differences (the need for a tearDown method in Fixtures, adding macros). It's essentially the same scope and features as the example Test.

All of these Test units follow the same common theme, you have TestUnits? that are collected and ran by a TestRunner? and return TestResults?. The example Test does the same, though perhaps a bit simpler. (Instead of a TestUnit? class, test units are simply delegates, Test itself is a TestRunner?, and TestResults? are simply an enumeration).

A possible enhancement to the example Test would be if it provided for returning a result array that one could do whatever it wanted with, and gave a printer that output those results and went to Stdout.

This too.

Sean, Kris: Any thoughts on this?

yes: it won't happen any time soon unless someone else takes the initiative, builds a proof-of-concept, and is prepared to own it. Because there's no volunteers yet, or no-one actively working on it, I'm moving this to the "nice idea to explore" milestone (currently the 2.0 one).

Regarding unit testing in general... I'll admit that I wish there were a way to further leverage the "unittest" blocks built into D, but barring that I think a a unit testing facility would be nice to have. One thing I'd like to see, however, is a way to plug in different output formatters.

Adding functionality like this fits with the general philosophy of Tango where I feel it is our goal to provide an extensible framework for users to build upon rather than just ready to go utilities. The point of this is to make it more likely that third party libraries will be compatible because they all build off the same common framework, but they'll only do that if the framework is flexible enough to handle common tasks.

That said, I agree with Kris. Great stuff, good to have, but it would need a maintainer (ie. not me).

JimPanic? is onto this - although it seems to most likely become an internal external library. This means Tango, but not part of the normal user library.