Specialized Working with Files

Random IO with Separated Buffers

Why?

One reason is that it is a flexible method of reading and writing random IO that's probably as efficient as any other. It also isn't much more complex. One plausible reason for maintaining separated buffers is to maintain a linkage between the file position and the data. When data is read by a Reader it is buffered in for efficiency. The file pointer is then advanced one buffer load (or to the end of file). This causes no problems for sequential reads, but it after a buffer is read the file pointer points either just beyond the end of the buffer or to the end of file. If you care where the next read will start, this is a significant problem. Using a separated buffer during reads and writes solves the problem with little overhead.

Setting up the class

This is just one of many good ways to set up the class...but it's one that has been tested. The this methods are omitted, as there is nothing special about them.
Note that error checks have been omitted for the sake of clarity, not because they should be omitted.

import	tango.io.Buffer;
import	tango.io.FileConduit;
import	tango.io.GrowBuffer;
import	tango.io.protocol.Reader;
import	tango.io.protocol.Writer;

class ClassName
{
   uint byteLengthEstimate()
   {  
      /* Use some way to estimate the byte length of this class.  
       * Overestimate rather than underestimate.
       * This is used in allocation of the buffers.
       */
   }

Writing out the class

This code assumes that you have a Writer method defined for the components of the class. Check out the Writer Class for a list of the types that are predefined.

//  write data at a position
ulong  put(FileConduit fc, ulong pos)
{  
   FileConduit fc;     //  The file associated with the class
   ulong       fPos;   //  The current position of the file
   uint[]      buff;   //  this is the buffer that will hold the data to be written
   // note that this allocates four times the estimated requirement  (buff is uints rather than bytes)
   buff.length = byteLengthEstimate;
   fc.seek(pos);       //  attempt to move file to desired position
   fpos = fc.getPosition;  //  set file position variable to actual position
   // Here I reserve some bytes at the start of the buffer for use outside of the writer method
   Buffer  bf    = new Buffer(buff, 4);  //  reserve 4 bytes of space
   Writer  write = new Writer(bf); // create a writer on the Buffer
   write (dataItem_1);       // using the writer on a data item
   write (dataItem_...);     // using the writer on a data item
   write (dataItem_n);       // using the writer on a data item
   uint bfEnd    = cast(uint)bf.readable;  // determine how many bytes have been written
   buff[0]       = bfEnd;                  // here I use the reserved space
   fc.write(buff[0..(bfEnd+3)/4]); // write the buffer to the fileConduit
   return  fPos;                   // this method choses to return the address of the START of the data
}

Reading the class back in

Note that in this read method the buffer is not separated from the FileConduit?. Also the buffer chosen is a GrowBuffer?. This is to allow the handling of records whose size is not known ahead of time.

ClassName  get(FileContuit fc, ulong pos)
{  fc.seek(pos);
   GrowBuffer  gb  = new GrowBuffer(fc);
   Reader     read = new Reader(gb);
   uint       len;          // When I wrote the record, the first thing I wrote was the length
   read(len);
   read(data item_1);
   read(data item_...);
   read(data item_n);
   ulong oEnd  =  gb.getPosition;   // One can accurately tell the position in the buffer 
   if (len != oEnd)      //  Since I knew the lenght written, length read can be an error check
   {  //  Handle error somehow
   }
   return  this;         //  The value read from the file is now the value of this class
}