[wiki:CoreComments Leave Comments, Critiques, and Suggestions Here]
[[TOC()]]
= The Core System =

This chapter marks the beginning of the "Advanced" section of the Tango Manual.  Rather than viewing these chapters as "advanced" in terms of mental comprehension, it may be easier to understand them as providing fine grained functionality for the expert programmer: these are features that may be seen less frequently in applications yet, when used, become critical components of the software project in terms of data manipulation, process control, or general optimization.

The Core Chapter starts things off by introducing the developer to a range of convenience tools, including features that may represent the under-pinnings of some high-performance D applications. Here is a sample of what we look forward to in this chapter:

 * Low-level mechanisms for fine control of D programs: GC/memory, runtime, exceptions, arrays, and signals/slots.
 * Special data control, reflection, and binding mechanisms: variable argument lists, tuples, traits and variants
 * Highly optimized byte/bit manipulation tools: bit arrays, bit manipulation and byte swaps.  
 * Low level concurrency functionality and control mechanisms: atomics, threading, and thread pools.

Let's begin.

== Array ==

{{{
#!d
import tango.core.Array;
}}}

While arrays may be one of the most fundamental storage types in modern programming languages, their flexibility and convenience is difficult to match with higher-level constructs.  D recognizes this and augments the traditional array concept with a slice syntax that raises the efficiency and flexibility of the arrays to even greater heights.  The true utility of arrays is only realized with their easy application to algorithms however, so Tango provides ''Array'' to fill this need.

Each of the algorithms in ''Array'' belongs to one of a fairly distinct set of categories.  The most commonly used algorithms, such as ''find'' and ''count'', do not require the array to be sorted and can therefore be used in nearly any situation.  These algorithms also do not modify the array they are passed, so they may be used on array literals.

{{{
#!d
import tango.core.Array;

void main()
{
    size_t pos = find( "The quick brown fox", 'q' );
}
}}}

Here we are searching for the string index of the letter 'q', and pos should be set to 4 by this operation.  It is more common to search for words in a string however, and ''find'' supports both.  But the syntax above seems a bit awkward for a type that claims to be a proper container.  The next example will perform a traditional substring find operation and follow with an alternate syntax that D supports for functions accepting an array as their first parameter:

{{{
#!d
import tango.core.Array;

void main()
{
    size_t pos;

    pos = find( "The quick brown fox", "quick" );
    pos = "The quick brown fox".find( "quick" );
}
}}}

Now let us assume that we have a large array of numbers and we want to count all the occurrences of 2 in the array, regardless of whether each occurrence is positive or negative.  The ''count'' function seems appropriate, but since numbers are normally compared for strict equality, we must supply count with an alternate comparator to use for evaluation:

{{{
#!d
import tango.core.Array;
import tango.math.Math;

void main()
{
    bool absCompare( int x, int y )
    {
        return abs( x ) == abs ( y );
    }

    int[]  buf = [1,2,3,-2,4,2,-2,5];
    size_t num = buf.count( 2, &absCompare );
}
}}}

But what if we want to operate on elements matching a more complex set of requirements?  For example, suppose we want to remove all even-numbered integers from an array.  Comparing to a search pattern isn't necessarily appropriate, even with a user-supplied comparator, but ''removeIf'' may be an appropriate substitute.  The example also contains a delegate literal to demonstrate how to avoid declaring standalone functions for such operations:

{{{
#!d
import tango.core.Array;

void main()
{
    int[] buf = [0,1,2,3,4,5,6,7].dup;

    buf = buf[0 .. buf.removeIf( ( int x ) { return x % 2 == 0; } )];
}
}}}

The ''removeIf'' function simply moves all matching elements to the end of the array and returns the number of elements that were not moved.  This allows the routine to be used with temporary values such as slices and makes no assumptions about whether the array should be resized.  In fact, there are only two functions that alter an array's size--''pushHeap'' and ''popHeap''--which we will get to later.

Now that we have explored some of the operations available for unordered arrays, it is time to move on to the next category of operations in ''Array'': operations on sorted arrays.  To make the transition, there must be some way to sort arrays.  D provides a built-in predicate for this purpose (appropriately called ''sort''), but it is limited to using the default comparison method for the contained elements.  But as we have seen earlier, this is not always the desired behavior.  For this reason, all routines in ''Array'' allow the user to supply a comparator, and the ''sort'' routine is no exception.

{{{
#!d
import tango.core.Array;
import tango.text.Ascii;
import tango.io.Console;

void main()
{
    char[][] names;

    names ~= "Roger";
    names ~= "alice";
    names ~= "bob";
    names ~= "Stacy";

    names.sort( ( char[] lhs, char[] rhs ) { return icompare( lhs, rhs ) < 0; } );

    foreach( name; names )
        Cout (name).newline;
}
}}}

Suppose we have a sorted array of temperature measurements and we wish to isolate the range of measurements which are exactly 50 degrees.  Since the array is sorted, a binary search routine seems appropriate, and ''lbound'' (lower bound) and ''ubound'' (upper bound) are ideal for this situation:

{{{
#!d
import tango.core.Array;
import tango.io.Stdout;

void main()
{
    int[] samples = ([47, 51, 50, 48, 50, 50, 52, 49, 54, 43].dup).sort;
    int[] inRange = samples[samples.lbound( 50 ) .. samples.ubound( 50 )];

    foreach( i, s; samples )
        Stdout( i )( " = " )( s ).newline;
    Stdout.newline;
    Stdout( "There were " )( inRange.length )( " matches: " );
    foreach( i, s; inRange )
        Stdout ( s )( ' ' );
    Stdout.newline;
}
}}}

Finally, many common data structures have an array representation and in some cases it is useful to operate on data in its existing representation rather than copy it into a specialized data structure for use.  Heaps are one such data structure, and the operations on a heap are fairly straightforward: push and pop:

{{{
#!d
import tango.core.Array;
import tango.io.Stdout;

void main()
{
    int[] samples = ([47, 51, 50, 48, 50, 50, 52, 49, 54, 43]).dup;

    samples.makeHeap();
    Stdout( "The measurements in descending order are: " ).newline;
    while( samples.length )
    {
        Stdout( samples[0] )( ' ' );
        samples.popHeap();
    }
    Stdout.newline;
}
}}}

== !BitArray ==

{{{
#!d
import tango.core.BitArray;
}}}

C Bit fields are out, and the ''BitArray'' is in.  By taking advantage of D's customizable type semantics and operator overloading, Tango adopts a solution in which an abstract container type does all the work of bit storage and manipulation.  True to its name, a ''BitArray'' is meant to be a container that is equivalent to an array of bits.  With this container it is possible to perform a large number of operations with ease and flexibility.

=== Initialization ===

''BitArray'' needs to be filled by initializing it with a set of values.  Recall that D's ''bool'' type is the basis for all boolean logical expressions.  Since bit operations are an exact equivalent to these boolean operations, Tango implements a ''BitArray'' by accepting a set of bools as the preferred way to initialize the array.  Nonetheless, a ''BitArray'' may be initialized in several ways as the following section will disclose.

Examine the following way to set up a ''BitArray'':

{{{
#!d
import tango.core.BitArray;

void main()
{
	bool[] b = [1,0,0,1,1];
	BitArray bitbag;
	bitbag.init( b );
}
}}}

We create a dynamic array of bools and then initialize ''bitbag'' with the contents.

And another way:

{{{
#!d
import tango.core.BitArray;

void main()
{
	BitArray bitbag;
	bitbag.length = 3;
	bitbag[0] = 1;
	bitbag[1] = 1;
	bitbag[2] = 0;
}
}}}

''BitArray'' length is set to a maximum of 3 bits.  Then each element is set independently.  The length may be extended or reduced later, and extra bits may be added or removed to fit the length as necessary.

The last two methods for initializing the field are straightforward:

{{{
#!d
import tango.core.BitArray;

void main()
{
	BitArray bitbag1 = bits( 1,0,1,1,1,0 );
	// or
	BitArray bitbag2( 1,0,1,1,1,0 );
}
}}}

Both bitbags are initialized with the provided ''bool'' values in parentheses.  ''bits'' is a function that takes a list of booleans or boolean equivalents using the typesafe variadic function feature of D.  It converts the list into a bool array and returns a copy of a initialized ''BitArray''.  This array is then assigned to ''bitbag1''.  ''bitbag2'' is initialized such that it uses a special ''opCall'' overload for the ''BitArray'' to achieve similar results.

This example creates a ''BitArray'' and fills them with some bits from an unsigned int:

{{{
#!d
import tango.core.BitArray;

void main()
{
	static uint x = 0b110110011;
	static BitArray bitbag = { 10, &x };
}
}}}

There are some details in the example that should be mentioned.  The first thing to notice is that we have made use of the ''static'' keyword.  This tells the compiler that the following data type will have a permanent storage class.  In contrast to local variables, the static keyword ensures that the variable remains active even after the function containing it is complete, much like a global variable.  In this case, the static attribute is necessary in order to apply a static initialization of the ''bitbag'' structure. Next we notice that we are actually applying an unsigned integer value to the array.  This value is prefixed with a ''0b'', which clearly represents that the next digits will be in binary format.  The ability to do this with a ''BitArray'' is a convenience.  The final { 10, &x } sets the appropriate structure members to the designated values.

''BitArray'' has more to offer.  To complete the container's initialization functionality, a ''BitArray'' also may be filled with arrays of other D types as long as such arrays are supplied in a generic format. Achieving this is accomplished by the clever type agnostics intrinsic to the D void array.  The benefit of using the void array type is that any D array can be cast to it.  In later chapters of this manual, the reader will see numerous opportunities for taking advantage of this wonderful feature.

Here is a demonstration of using void as an initializer for a ''BitArray'':

{{{
#!d
import tango.core.BitArray;

void main()
{
	void[] v;
	ubyte[] n = [ 0b0101011, 0b111110, 0b0010001, 0b00011111 ];
	v = cast(void[]) n;

	BitArray bitbag;
	bitbag.init( v, n.length * ubyte.sizeof * 8 );

	foreach( a ; bitbag )
		Stdout (a ? "1" : "0" ).newline;
}
}}}

It is admittedly more work to handle a void array, but the technique turns out to be useful in various situations.  The above converts an unsigned byte array to a void array and the initializes ''bitbag'' with its contents.  ''bitbag'' requires that the number of bits in the array be submitted as well.  Therefore, we calculate the total number of bits and pass that to the ''init'' method.

One important detail here is that ''BitArray'' expects to be initialized with values that are equivalent to the computer architecture word size in granularity.  That means that for a 32-bit architecture we should be careful to pass a 4 byte array as a minimum.  The next step in size would be an 8-byte array.  Anything smaller than 4 bytes or in between 32-bit increments will trigger an exception in ''BitArray''.  If we cast an ''uint'' array to a void array, we have little to worry about because the default granularity is 32-bits.  This caution relates to using void arrays only.

=== Properties ===

''BitArray'' provides several features that allow the programmer to both set and get details about the state of an initialized variable. Here is an overview what they do:

''length'' returns the length in bits of the ''BitArray''.  It may also be used to increase the size of the ''BitArray'' in place by assigning a value to it.

''dim'' returns the size of the ''BitArray'' in terms of the computer architecture word.  For the 32-bits systems, a call to ''dim'' would return the size of the ''BitArray'' in 32-bit words.  On a 64-bit machine, the result would be in 64-bit words.  Thus, a ''BitArray'' with a ''length'' property set to 64-bits would return a ''dim'' of 1 on a 64-bit machine while a 32-bit machine would return a ''dim'' of 2.

The ''dup'' property accommodates duplicating a ''BitArray''.  It is useful for quick in place copying of  one ''BitArray'' to another.

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray bitbag( 1,1,1,0,1 );

	// Get the length in bits
	Stdout.format ("length: {0}", bitbag.length ).newline;

	// Set the length in bits via assignment
	bitbag.length = 6;
	// or method call style
	bitbag.length( 6 );

	// Get the dimension of bitbag
	Stdout.format ("dim: {0}", bitbag.dim ).newline;

	// Duplicate the bitbag contents and store
	// them in tempbag.
	BitArray tempbag = bitbag.dup;
}
}}}

Notice once more how we can set the ''length'' property.  D allows either an assignment style or method call style for setting properties.  Use of the ''length'' property in this way is a common theme in D programming.

Next, the ''dim'' property above should result in a 1 being printed to the display of a 32-bit machine because all the bits fit into a 32 bit value.  Finally the ''dup'' feature replicates the contents of ''bitbag'' and puts them in ''tempbag''.

Two more properties available are really just method calls that manipulate the contents of our abstract ''BitArray'' type.

''reverse'' arranges the bits in the ''BitArray'' variable in the opposite direction.  An application for ''reverse'' might be in the mirroring of a simple bitmap, as in the case of a mouse cursor on a graphics display.

''sort'' applies a sort algorithm to the bits in a ''BitArray''.  Calling ''sort'' will cause all bit 0's to bubble to the top or beginning of ''BitArray'' and all bit 1's to sink to the bottom or end of the ''BitArray''.  The effect is much like the separating action when oil and water are mixed in a glass.  The heavy oil settles below while the lighter water rises above.

An example of below shows how to use ''reverse'' and ''sort'':

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray bitbag( 1,1,1,0,1 );

	Stdout ("Before: ");
	foreach ( n ; bitbag )
		Stdout ( n ? "1" : "0" );

	bitbag.reverse;

	Stdout ("\nAfter:  ");
	foreach ( n ; bitbag )
		Stdout ( n ? "1" : "0" );

	bitbag.sort;

	Stdout ("\nAfter sort: ");
	foreach ( n, ; bitbag )
		Stdout (n ? "1" : "0" );
}
}}}

=== Bit Operators ===

Finally we arrive at the main feature of the ''BitArray''. The ''BitArray'' would be really quite useless if we could not do any boolean operations on them.  Thankfully, we need not worry because the Tango ''BitArray'' type provides the standard bitwise logical operators: &, |, !^, and ~ (also known as ''AND'', ''OR'', ''XOR'', and ''complement'').

An example:

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray a,b,c,d;
	BitArray bitbag1( 1,1,0,0,1,1 );
	BitArray bitbag2( 0,1,1,0,1,0 );
	BitArray bitbag3( 1,1,0,1,0,0 );

	// Demonstrating "AND" operation
	a  = bitbag1 & bitbag2;
	a &= bitbag3;

	// Demonstrating "OR" operation
	b  = bitbag1 | bitbag2;
	b |= bitbag3;

	// Demonstrating "XOR" operation
	c  = bitbag1 ^ bitbag2;
	c ^= bitbag3;

	// Demonstrating "Complement" operation
	d = ~ bitbag1;
}
}}}

The boolean operators work on two BitArray's. The bit arrays involved are not affected by the bit level operation so it is important to remember to store the result in a variable.

The unary complement operator likewise returns a result without affecting the contents of the involved ''BitArray''.  The result of the complement will be an inverting of all bits in bitbag1: 1's will now be 0's and 0's will now be 1's.

The example also shows a variation of the bitwise operations: a combined operator and assignment.  For those unfamiliar with C-like languages, these operators are equivalent to applying the left side value to the binary operation involving the right side expression and then assigning the result back to the left side variable: therefore, ''n &= x'' is equivalent to ''n = n & x''.

=== General Operations ===

The remainder of features available in the ''BitArray'' type are for general convenience, designed to immerse the Tango developer into the D experience.  With these operations, ''BitArray'' begins to feel like a native D type.

The first feature is the ability to loop through the contents of a ''BitArray'' by using a ''foreach'' construct.  We have already seen the ''foreach'' in action in previous examples, but let us review the functionality in a little more detail.

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray bitbag( 1,0,1,1,1,0,1 );

	foreach( abit ; bitbag )
		Stdout.format( "{0}", abit );
}
}}}

Silently working behind the scenes is an operator that allows us to step through the bits in bitbag and extract each consecutive value such that we can output the result with a ''Stdout'' call.  This construct makes iterative tasks exceptionally easy and is another common operation in the D language.

We can do comparisons between bit arrays as well.  Several operators are overloaded that allow for common comparisons:

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray bitbag1( 1,0,1,1,0,1,0 );
	BitArray bitbag2( 1,1,0,0,1,0,1 );
	BitArray bitbag3( 1,1,0 );
	BitArray bitbag4( 0,1,1,0 );
	BitArray bitbag5( 1,1,0,0,1,0,1 );

	if ( bitbag1 != bitbag2 )
		Stdout ("different").newline;

	if ( bitbag2 == bitbag5 )
		Stdout ("same").newline;

	if ( bitbag3 >= bitbag4 )
		Stdout ("greater than or equal").newline;

	if ( bitbag1 > bitbag4 )
		Stdout ("greater than").newline;
}
}}}

All relational operations are possible: <, >, >=, <=, ==, and !=.  Comparison is made between equivalent ranges.  This means that if the two !BitArrays in a comparison have a different number of bits, the comparison between elements will be restricted to the length of the smaller array.  The comparison treats the first bit in the array as the most significant bit. Thus in the examples above, the left most bit in the initialization section of each array is the most significant bit. In the example, where ''bitbag3'' is compared to bitbag4 with a >=, ''bitbag3'' will be found to be greater than ''bitbag4'', even though ''bitbag4'' has more bits in its !BitArray.

The !BitArray may be converted to a void array using a ''cast'' operation:

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray bitbag( 1,1,0,1,1 );

	void[] v = cast(void[]) bitbag;
}
}}}

Remember that a void array is like a generic container in D.  It holds any type of value.  Here, the cast fills up the void array with duplicates of the bits it contains.  The bits are not directly accessible from a ''v'' as is. Yet the void array type may be passed to other object methods, that accept the void array type, for further processing.  Tango contains objects of this sort as we will see later in the chapter on IO operations.

There may be situations where we want to combine arrays or add a single bit to them.  This operation, which is a commonly used feature in D arrays, is called a concatenation.  It has its own operator which !BitArray coveniently overloads.

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray a,b,c,d;

	BitArray bitbag1( 1,1,0,1,1 );
	BitArray bitbag2( 0,1,1,1 );

	a = bitbag1 ~ bitbag2;
	b = bitbag1 ~ true;
	c = false ~ bitbag2;

	c ~= true;
}
}}}

''bitbag1'' and ''bitbag2'' are combined, and then then the result is stored in ''a''.  The next two lines show how a single bit is concatenated to the contents of bitbag1 and bitbag2; the whole result is stored in a new !BitArray.  bitbag1 and bitbag2 are not modified by the concatenation operator in these three cases.  If we want to concatenate to the current array we use the concatenation and assign feature as shown in the last line.  There, a bit is appended to the end of the ''c'' !BitArray.

We are nearing the end of the !BitArray discussion. Another operator that has already been demonstrated but not discussed is array indexing.

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray bitbag( 1,1,1,0,0,1 );

	for (int i = 0; i < bitbag.length; i++)
	{
		Stdout.format( "{0}", bitbag[i] );
		bitbag[i] = true;
	}
}
}}}

Using the index operator, we are able to retrieve bits from specific locations in the bitarray.  Assignment of values to the appropriate location is also straight forward.  This may seem obvious, yet this would not be possible without an overloaded index operator within !BitArray.

The last operation to discuss in the subtraction operation.  A subtraction seems like an unusual feature for a bit array, but in fact it is provided as a quick way to perform a combined bitwise operation.

{{{
#!d
import tango.core.BitArray,
       tango.io.Stdout;

void main()
{
	BitArray a;
	BitArray bitbag( 1,0,1,1,0,1 );
	BitArray bitbag( 0,1,1,0,1,1 );

	a = bitbag1 - bitbag2;
	bitbag1 -= bitbag2;
}
}}}

''a'' is assigned the result of bitbag1 and bitbag2.  The operation in this case is equivalent to complementing bitbag2 and then ''AND''-ing the result with bitbag1, finally assigning the result to !BitArray ''a''.  In the second operation, we use the familiar operation and assign to do the equivalent of bitbag1 = bitbag1 & ~bitbag2.

The world of bits is only a small part of the whole story.  We now step upward from bits to bytes.

== !ByteSwap ==

{{{
#!d
import tango.core.ByteSwap;
}}}

The essence of all data in computer technology is a distinct unit of bits that we call a byte.  It represents the most basic parcel of information on most computer architectures in existence.  As discussed in the intrinsics section, one machine will fail to communicate with another if each machine does not see groups of bytes in the same order as the other.  Such communication problems would be far too common without properly written software and readily available development tools.

The Tango solution is the !ByteSwap structure, a software tool that can do fast byte reordering on byte groups so that data can be “understood” between systems not sharing the same data-direction or Endianess.

We discussed a similar topic in the Intrinsics section in which we define a machine level instruction called ''bswap()''.  While ''bswap()'' deals with the reordering of single 4 byte units by mapping to a machine level instruction operative, !ByteSwap attends to more flexible conversion by extending the intrinsic's functionality.  Software can use this object to accomplish byte reordering for up to four of the most common packet sizes: 2, 4, 8, 10 byte size increments.

{{{
#!d
import tango.io.Stdout,
       tango.core.ByteSwap;

void main()
{
      ushort x16 = 0xFF00;
      uint   x32 = 0xFFFF0000;
      ulong  x64 = 0xFFFFFFFF00000000;

      ubyte[10] x80;
      x80[0..5] = 0xff;
      x80[5..10] = 0x00;

      print( x16 );
      ByteSwap.swap16( cast(void*) &x16, x16.sizeof );
      print( x16 );

      print( x32 );
      ByteSwap.swap32( cast(void*) &x32, x32.sizeof );
      print( x32 );

      print( x64 );
      ByteSwap.swap64( cast(void*) &x64, x64.sizeof );
      print( x64 );

      printArray( x80.dup );
      ByteSwap.swap80( cast(void*) &x80, x80.sizeof );
      printArray( x80.dup );
}

void printArray( T )( T[] x )
{
      Stdout.newline;
      foreach ( n; x )
            Stdout.format ("{0:X2}", n);
}

void print( ulong x )
{
       Stdout.newline.format(" {0:X2} ", x).newline;
}
}}}

The example shows how to use each of the swap methods available in the !ByteSwap module.  We use ushort to supply 16-bit data to swap16, uint for 32-bit data and swap32, and ulong for 64-bit data and swap64.  The console output shows the appropriate reversal of all bytes in each group (in hexadecimal format):

{{{
 FF00

 00FF

 FFFF0000

 0000FFFF

 FFFFFFFF00000000

 00000000FFFFFFFF

 FFFFFFFFFF0000000000
 0000000000FFFFFFFFFF
}}}

In order to supply an 80-bit value to the last swap80() method, we need to use an array of 10 ubytes.  We can initialize this array with test values by using D array slices with a single array initializer for shorthand.  As demonstrated we fill the first half of the array with “ff” and the second half with “00”.

We supply all the various byte groups to their functions by casting the variables to a void pointer.  This allows the byte swap function to have direct access to all variables to accommodate the in-place reordering.

Up to this point, we have covered Tango utilities that handle information on the bit and byte levels.  It is now time to investigate the Tango exception mechanism.

== Exception ==

{{{
#!d
import tango.core.Exception;
}}}

Each day, we involve ourselves in mundane tasks, automated procedures, and trivial pursuits.  We think little of what our brains do during these processes and even less of the myriad of the failures and corrections that precipitate from seemingly uneventful tasks.  But, unconscious or not, what our brains accomplish during these moments amounts to finely tuned mechanisms of high speed processing and recovery from errors; it involves feedback systems, processing and filtering, and eventual recovery and correction.

Picture a dancer who steps on an uneven surface while trying to perform a carefully timed maneuver.  The brain is ever ready to receive information through afferent nervous system messages, to process balance changes through the inner ear, and to encode correcting signals through a somatic nervous system to effect coordinated motor responses in the limbs to achieve recovery. Outgoing motor signals are filtered through the cerebellum during the process to fine tune the completed execution which hopefully results in something more graceful than a finely tuned face plant.

Programming languages represent an analogous operation for computers.  Programming language history is as much a history of computer processing algorithms as it is a history of recovering from errors.  Error recovery is an inseparable part of computer science because it details what we expect to happen when the unexpected happens.  In the early days of software development, the novelty of being able to program in a higher language might have directly overshadowed any special considerations involved in recovering from exceptional circumstances.  Now, as years have passed and experience has aggregated, we find old techniques of error recovery to be proverbial duct tape on a problem domain littered with what might be called “after-thoughts.”  Thankfully, lessons have been learned over the last few decades, and the classification of errors and exceptions have been much studied: languages have adopted more elegant solutions for handling them.  D is just such a language.  Here we investigate how the Tango library uses D Exception classes to implement program recovery solutions.

... discuss exception hierarchy here ...

While error detection and recovery is commonplace, the nuances of what constitutes an error and how that error should be dealt with are largely application-specific.  The most common situation involves the handling of assertion errors.  In a typical application the proper approach is generally to terminate the application with an informative message, but such behavior is often not ideal during debugging.  Rather, the programmer would typically prefer that the debugger trap such errors and allow the program state to be inspected immediately.  On x86 CPUs, this can be achieved using inline assembler line so:

{{{
#!d
import tango.core.Exception;

void assertHandler( char[] file, size_t line, char[] msg = null )
{
    asm { int 3; }
}

void main()
{
    setAssertHandler( &assertHandler );
    assert( false );
}
}}}

The "int 3" instruction should signal the debugger to halt within the assert handler, and we can trace up the call stack normally to inspect program state.  Alternately, we may simply want to report the error and continue--perhaps during unit testing:

{{{
#!d
import tango.core.Exception;
import tango.io.Stdout;

void assertHandler( char[] file, size_t line, char[] msg = null )
{
    Stdout( "An error occurred in file " )( file )( ", line " )( line );
    if( msg )
        Stdout( ", with message:\n" )( msg );
    Stdout.newline;
}

void main()
{
    setAssertHandler( &assertHandler );
    assert( 'a' == 'b' );
    assert( 'c' == 'd', "'c' does not equal 'd'" );
    setAssertHandler( null );
    assert( 'e' == 'f' );
}
}}}

Here, we set a custom assert handler which is used to simply write the first two assertion failures as messages to Stdout, then we reset the handler to use default behavior which should throw an AssertException for the third assertion.

Another point of concern in languages with garbage collection is how to handle resource management.  If an object is discarded during normal processing and is cleaned up by the garbage collector then it may not expect that any references it has to garbage collected objects may still be valid.  However, if an object is destroyed in a deterministic manner (ie. through the use of the ''scope'' keyword or explicitly via ''delete'') then we may safely assume that its internal references are still valid and explicit cleanup of such resources is allowed.  Because resources are limited, deterministic destruction of such objects is preferred, but it is not always possible to be sure that resources objects have a deterministic lifetime.  For these reasons, Tango offers a means of intercepting and overriding the normal garbage collector clean-up mechanism like so:

{{{
#!d
import tango.core.Exception;
import tango.core.Memory;
import tango.io.Stdout;

class ClassA
{
    ~this()
    {
        Stdout( "ClassA dtor" ).newline;
    }

    void dispose()
    {
        Stdout( "ClassA dispose" ).newline;
    }
}

class ClassB
{

}

bool collectHandler( Object obj )
{
    if( obj.classinfo is ClassA.classinfo )
    {
        Stdout( "ClassA encountered, performing special clean-up" ).newline;
        (cast(ClassA) obj).dispose();
        return false;
    }
    Stdout( "Collecting: " )( obj.classinfo.name ).newline;
    return true;
}

void main()
{
    GC.collectHandler( &collectHandler );
    {
        ClassA a = new ClassA;
        ClassB b = new ClassB;
    }
    {
        scope ClassA a = new ClassA;
    }
    GC.collect();
    Stdout( "done" ).newline;
}
}}}

Here, we have chosen to clean up ''ClassA'' differently based on whether its lifetime ends in a deterministic or a non-deterministic manner.  If ClassA is cleaned up by the garbage collector it will be passed to collectHandler, which calls the dispose method if the object is of type ClassA.  However, if ClassA is cleaned up in a deterministic manner then the garbage collector will never see it and collectHandler will not be called.  The return value for collectHandler specifies whether the garbage collector should call the object's dtor and the dtors of its parents: ''true'' indicates that the object's dtor should be called as normal, and ''false'' indicates that it should not.

An interesting side-effect of using a collection handler is that it may be used for inspection or reporting purposes as easily as for altering cleanup behavior.  In applications where certain objects must have a deterministic lifetime, it may be useful to set a collect handler which simply checks to make sure that no objects of that type are ever encountered by the garbage collector.  In essence, when combined with purely deterministic programming, the collectHandler becomes an in-language form of leak detection.

== intrinsic ==

{{{
#!d
import tango.std.intrinsic;
}}}

The intrinsic module is owner to several basic functions useful to those interested in performance code that would normally be relegated to the realm of an assembler.  By importing this module, the programmer is able to call functions that the compiler vendor can convert directly to an assembler operation.

Most of these functions constitute methods to manipulate, scan, or test bits; yet having these immediately available avoids much of the less efficient bit shifting and comparisons in the high level language.

Let us look at the first of these functions:

{{{
#!d
int bsf( uint v );
int bsr( uint v );
}}}

The ''bsf'' function accepts an argument ''v'' to be scanned.  It scans each bit from lowest to highest searching for the first bit that is set.  It returns the number of that bit. Think of it as an abbreviation for "bit scan first."

A counterpart of the above is "bit scan reverse" which scans in the opposite direction, highest to lowest significant bit, to find the first bit in the ''uint'' that is set.  Recall that in D the ''uint'' is 32 bits, so D returns a number between 0 and 31 to indicate which bit is set.

{{{
#!d
int bt ( uint* p, uint bitnum );
int btc( uint* p, uint bitnum );
int btr( uint* p, uint bitnum );
int bts( uint* p, uint bitnum );
}}}

To test a bit, use ''bt''.  Testing a bit usually involves applying a binary ''AND'' to the argument submitted.  The test typically is used to set a flag status.  Doing so will not change the value of the argument's bit.

''bt'' takes a pointer argument. This allows the bit test to take an array of ''uint'' with arbitrary length. Submit the array to the function.  To indicate which bit to test, set ''bitnum'' to a number between 0 and the extent of the array (which is 4*32*array.length).

{{{
#!d
uint array[2] = [ 0x1010, 0x0480 ];

if ( bt(array, 43) == 0 )
    {
    // do something...
    }
}}}

Bit tests, in contrast to ''AND'' operations, are useful whenever the programmer wants to follow up with a conditional expression based on the result without modifying the original expression.  Bit tests are common in numerous fields including graphics, animation, and cryptography.

The next two bit tests take the same arguments.  ''btc'' or "bit test and complement" merely complements or toggles the bit to be tested and returns the result of that operation.  ''btr'' or "bit test and reset" resets the chosen bit to 0 and returns the result of the bit test.  The result, in this case, will be the original test.  It will not return the result of the reset which would always be 0. ''bts'' does similarly to ''btr'' except it sets the bit in ''*p'' after testing the bit at position ''bitnum''.  Thus the bit will be set to 1.

We now move from bits to bytes.

{{{
#!d
uint bswap( uint v );
}}}

The function ''bswap'' swaps bytes in a ''uint''.  Since D's ''uint'' is a 32 bit value, the byte swap operates on 4 bytes.  So if we had an ''uint'' aligned as byte 0, byte 1, byte 2, byte 3, the result would be the reordering in reverse:  byte 3, byte 2, byte 1, byte 0.  Byte swaps are extremely useful operations when working across networks of different machines or when porting software between platforms.  Some computer hardware, specifically microprocessors, read the bytes at an address in one of two directions.  This is known as an Endianness.  ''bswap'' offers an efficient method to translate byte ordering in such situations.  This in turn allows for fast cross platform interaction amongst computers from different manufacturers.

I/O port intrinsics are also included.  They allow direct access to the computer's hardware ports.

{{{
#!d
ubyte  inp ( uint port_address );
ushort inpw( uint port_address );
uint   inpl( uint port_address );
}}}

These functions are powerful and require a thorough understanding of the underlying computer architecture. Usually such functions are only useful for driver development or embedded systems.

The input functions above take a port address as an argument.  Once executed, the functions send a request to the hardware to activate the port and read the contents from the hardware bus.  The return value for these functions will be the data requested from the port. The input functions return vary in size: ''inp'' will return an 8 bit data value from the port request, ''inpw'' will return a 16 bit value, and ''inpl'' will return a 32 value.

The final port intrinsics are for output:

{{{
#!d
ubyte outp ( uint port_address, ubyte value );
ubyte outpw( uint port_address, ushort value );
ubyte outpl( uint port_address, uint value );
}}}

In this case, a command is sent to the computer hardware port, specified by ''port_address'', to output a value on the particular hardware bus.  Once again, the value may be 8 bit, 16 bit, or 32 bit.  The programmer choses the appropriate function that fits the data to be sent.

That completes the intrinsics module.  These intrinsics are purposely brief and cryptic because they are usually only useful to programmers familiar with their associated low level tasks.

== Memory ==

{{{
#!d
import tango.core.Memory;
}}}

The D language approach to memory management is perhaps nothing new to the computer world.  What is new is the fact that D is not dependent on a virtual machine yet still resorts to using a garbage collector as its primary form of memory management.  Thus, in contrast to C++ where memory management has always been a hideous ordeal of meticulous observance of matching new's with delete's, D's adoption of garbage collection as an intrinsic feature has been a new freedom that has thrown off a massive burden in the field of native-compiled languages.  And in contrast to languages like Java, D still compares favorablely because it avoids the sluggishness inherent in virtual machines yet adopts the style of programming and ease of use inherent in garbage collected languages.

Every system has its weakness.  Garbage collection, while a pleasant relief from the weighty tasks of manual memory management, still has its drawbacks when used in a small percentage of applications.  A stated goal in the D specification is an attempt to avoid making a religion of any one programming paradigm.  It is evident that D adopts this viewpoint in the area of memory managers as well.  While the default form of managing memory in D is the garbage collector, D allows finer control of memory: the programmer may disable the collector and take control of allocation tasks; the programmer may allocate memory that is outside the garbage collector's control; the programmer may allocate and deallocate as necessary without worrying about collection cycles interfering in performance sensitive code sections.

The garbage collector can also be completely replaced with different memory management technologies because D adopts a plug in architecture.  Alternatively, the collector may be disabled and enabled to allow the programmer direct control of its operation.  Pointers can be manually pinned and released as necessary.  And if all this is too much work, the programmer can just safely rely on the default operation of the garbage collector in which little interference or performance degradation will ever take place in the grand majority of D applications.

The core Memory module is Tango's interface to the garbage collector.  Through this interface, the Tango user may manipulate the native D collector in all sorts of ways.  We will discuss these features in detail in this section.

The D garbage collector is accessed directly through a global ''GC'' object which contains an assortment of important methods:

{{{
#!d
GC.enable();
GC.disable();
GC.collect();

GC.getAttr();
GC.setAttr();
GC.clrAttr();

GC.malloc();
GC.calloc();
GC.realloc();
GC.free();

GC.sizeOf();

GC.addRoot();
GC.addRange();
GC.removeRoot();
GC.removeRange();
}}}

... to be continued ...

== Thread ==

... In progress

== Vararg ==

{{{
#!d
import tango.core.Vararg;
}}}

Using a variable number of arguments in a function has long been a mainstay in the C language as demonstrated by the venerable ''printf'' function.  Here we will discover how D and Tango can offer that and more for the variadic function aficionado.

The Tango ''Vararg'' module implements a set of D templates for accessing the arguments of a D variadic function.   Though templates are used, the details of instantiating Vararg templates require little understanding of the inner workings of the D template system. If the reader is interested in further study of how D templates work, however, we refer you to the D Reference Manual where the topic is covered in detail.  The material contained here will provide the Tango user with enough information on D templates to get started.

First let us review how variable argument functions are defined.  In D their declaration is similar to the C style with the notable inclusion of the ellipses as the last argument of the group:

{{{
#!d
// D style
void foo1( int b, ...) {}
// or
int foo2(...) {}

// C style can be defined in D also
extern(C) void foo3( int b, ... ) {}
}}}

The C style requires at least one argument before the ellipses.  There is no such requirement in the D version.

Note that D actually offers three types of variadic functions:

 * C-style variadic functions
 * Variadic functions with type information
 * Typesafe variadic functions

This section discusses the second type, the one demonstrated in the sample code. This is the default D variadic function style to make use of the Vararg module.  The C-style is equivalent to the D one and shares a compatible system for obtaining the argument list, something that makes the transition from C to D even easier.  The D style has an added advantage in that the argument types and number may be obtained through the hidden ''_arguments'' ''TypInfo'' array.

The third type, D typesafe variadic functions, do not use or need these core library templates and are not discussed further here.  Please refer to the appropriate section of the D reference manual for more information concerning them.

The Vararg module defines two templates:

{{{
#!d
template va_start( T )
template va_arg( T )
}}}

and two functions:

{{{
#!d
void va_end( va_list ap );
void va_copy( out va_list dest, va_list src );
}}}

The D specification defines two hidden local variables that are accessible to the variadic functions: ''_arguments'' and ''_argptr''.  The programmer using the Vararg module makes use of these local variables to submit information to the Vararg templates, to obtain the variable list, and to acquire type information on all variables in that list.

Let us look at the following example:

{{{
#!d
module VarargDemo1;

import tango.io.Stdout,
       tango.core.Vararg;

void foo( int x, ... )
{
	va_start!(int)(_argptr, x);

	foreach( argtype; _arguments )
	{
		Stdout ( argtype.toString() ).newline;
	}
}

void main()
{
	foo( 4, "this", 8L, 3.4f );
}
}}}

''foo(...)'' is our D variadic function.  Inside it, the local variables ''_arguments'' and ''_argptr'' become implicitly available.  The first step to accessing variadic arguments is very similar to the C way of variadic management.  Using the template ''va_start'', we can initialize the argument pointer so that it points to the beginning of the argument list.  This template is mostly available for backwards compatibility with the C macro of the same name and is not required in D variadic functions.

The ''va_start'' template is instantiated in the following manner:

 * ''!(int)'' indicates the template instantiation type for the last argument before the ellipses
 * ''_argptr'' is the template function argument that we want initialized to point to the argument list
 * ''x'' is the template function argument that gives access to the last argument before the ellipses of our variadic function; this value is necessary for the template to calculate ''_argptr''.

In the example, ''va_start'' causes ''_argptr'' to point to the first variadic argument.  In order to get access to the consecutive values of the actual arguments, we still need to use sequential calls to ''va_arg'' to do the pointer arithmetic that steps to each arguments value.  Meanwhile, the ''foreach'' demonstrates a simple and effective way to loop through the length of the ''_arguments TypeInfo'' array.

However, using ''va_start'' to initialize the ''_argptr'' is actually unnecessary for D variadic functions because the ''_argptr'' is always properly initilized to the first argument automatically.  The fact that ''va_start'' acts as a compatible equivalent for both C and D is merely a useful feature for C programmers transitioning to D.

The following demonstrates the use of ''va_arg'':

{{{
#!d
module VarargDemo2;

import tango.io.Stdout,
       tango.core.Vararg;

void foo ( ... )
{
	foreach( argtype; _arguments )
	{
		Stdout ( argtype.toString() ).newline;

		if ( typeid(char[]) == argtype )
		{
			char[] word = va_arg!(char[])(_argptr);
			Stdout (word).endline;
		}
		else if ( typeid(long) == argtype )
		{
			long l = va_arg!(long)(_argptr);
			Stdout ( l ).newline;
		}
		else if ( typeid(float) == argtype )
		{
			float f = va_arg!(float)(_argptr);
			Stdout ( f ).newline;
		}
	}
}

void main()
{
	foo( 4, "this", 8L, 3.4f );
}
}}}

''argtype'' represents the value of the current ''_arguments'' index in the ''foreach'' loop.  The ''if'' condition blocks check to see if the current ''argtype'' is equivalent to a D type that the variadic function expects to handle.   If it is, we call a ''var_arg'' template instance that does several important tasks automatically:

 * It increments ''argptr'' by the appropriate amount so that it points to the next argument in the list
 * It casts ''argptr'' to the designated type as specified by the template instantiation.
 * It returns a value of the designated type so that it can be assigned to a variable

In this way, we can gain the complete list of the variable arguments in a type safe manner.

The two remaining symbols in the ''Vararg'' module are functions rather than templates.  ''va_end'' ends the initialization.  It is an visual indicator that variadic function processing is complete.  Much like a block of code is embraced by a pair of brackets, ''va_start'' and ''va_end'' may be used to provide a visual cue that variadic processing is occurring within the enclosed code.

''va_copy'' copies a source argument list pointer to a new destination pointer.  This procedure may be useful to save the current location of the ''_argptr'' which, otherwise, is lost as the variadic processing of the argument list progresses.

{{{
#!d
module VarargDemo1;

import tango.io.Stdout,
       tango.core.Vararg;

void foo( int x, ... )
{
	va_list startptr;

	va_start!(int)(_argptr, x);
	va_copy( startptr, _argptr );
	va_end( _argptr );
}

void main()
{
	foo( 4, "this", 8L, 3.4f );
}
}}}

In the above example, ''startptr'' is initialized to the current ''_argptr''.  If ''_argptr'' changes as we iterate through the argument list, the initial position is guaranteed to be saved in ''startptr''.

= In Review =

... in progress ...