SHORT-FORMAT
FILES
| Created On: |
13 July, 1997 |
| Last Modified: |
4 November, 1997 |
Overview
Introduction
The native file format of CleanVec and other software in the Laboratory for
Turbulence and Complex Flow is short format. Short format was developed by Prof. Carl D. Meinhart
as a compact binary representation for two-dimensional fields of PIV data.
However, because of its flexible data storage capability, it can be used for
much more than standard PIV data. In fact, it is capable of storing almost any
type of 1-D or 2-D data field, including
- Vorticity (or any other type of scalar field)
- Planar 3-D vector fields (as used by stereoscopic and holographic PIV)
- Probability density functions
- Reynolds stress fields
- and many more
The only assumption made in short format is that the underlying grid of data
is uniform Cartesian, which is not restrictive since most (if not all)
interrogation software outputs data onto this type of grid.
File Content
Since short format is a binary format, the content of this file type is
outlined below for reference. Each variable is named and described. Their data
types and sizes (in bytes) are listed, as well. The order in which the variables
appear in the table below is the same as in the file.
| Variable |
Type |
Size
(bytes) |
Description |
| Columns |
Integer |
4 |
Number of columns on the grid. |
| Rows |
Integer |
4 |
Number of rows on the grid. |
| Values |
Integer |
4 |
Number of data values stored at each grid point. |
| Photos |
Integer |
4 |
Number of photographs (realizations) used to generate the values in
this file. This variable is primarily used for statistical analysis
software that performs ensemble averages. |
| X0 |
Real |
4 |
x-location of grid origin (lower-left corner). |
| Y0 |
Real |
4 |
y-location of grid origin (lower-left corner). |
| Delta X |
Real |
4 |
Spacing between grid points in the x-direction. |
| Delta Y |
Real |
4 |
Spacing between grid points in the y-direction. |
| Comments |
Integer |
4 |
Number of comments contained in the comment history. |
| Comment History |
String Array |
80 * Comments |
Comment history. These comments indicate what operations have been
performed on the short-format file. Used primarily by statistical analysis
software. |
| Data Values |
Real Array |
4 * Rows * Columns * Values |
Array of data values stored at each grid point. |
| Data Weights |
Integer Array |
4 * Rows * Columns |
Array of data weights at each grid point. Raw data normally has a
weight of "1" at each grid point. Used primarily by statistical analysis
software. |
Sample Code for Reading and Writing Short-Format Files
Below is a listing of source code that contains two functions which can be
used to read and write short-format files. These functions are written in C++
and make use of the Microsoft® Foundation Classes (MFC) library (version 4.0 or
higher), although it could be easily changed to make use of another library,
such as the standard C++ library.
/////////////////////////////////////////////////////////////////////////////
//
// HOW TO READ AND WRITE SHORT-FORMAT FILES
//
//---------------------------------------------------------------------------
//
// This sample contains two subroutines that illustrate how one could read
// and write short-format files. Both routines use the CShortFormatContent
// structure, which, as the name implies, is responsible for storing the
// content of a single short-format file.
//
// The sample has been written in C++ using the Microsoft(R) Foundation
// Classes (MFC) library. If you are not familiar with MFC, most of the
// constructs can be easily extended to other libraries, such as the standard
// C++ library.
//
/////////////////////////////////////////////////////////////////////////////
#include // MFC core components
#include // MFC templates
/////////////////////////////////////////////////////////////////////////////
// Define Alias Types
typedef float REAL;
typedef CArray CREALArray;
typedef CArray CIntArray;
/////////////////////////////////////////////////////////////////////////////
// Define Constants
static const int COMMENT_LENGTH = 80;
/////////////////////////////////////////////////////////////////////////////
// User-Defined Types
/////////////////////////////////////////////////////////////////////////////
//
// CShortFormatContent Structure
//
//---------------------------------------------------------------------------
//
// This structure stores the contents of a short-format file. It is used
// by the routines ReadShortFormatFile() and WriteShortFormatFile().
//
/////////////////////////////////////////////////////////////////////////////
struct CShortFormatContent
{
// The following attributes represent the components of the short-format
// file's header.
//
int nRows, // Number of rows in the grid
nCols, // Number of columns in the grid
nValues, // Number of values stored at each grid point
nPhotos; // Number of photos (realizations) used to
// generate the values in this file
REAL rX0, // x-coordinate of origin (real units)
rY0, // y-coordinate of origin (real units)
rDeltaX, // Grid point spacing in x-direction (real units)
rDeltaY; // Grid point spacing in y-direction (real units)
CStringArray astrComments; // Comment history
// The following attributes represent the data content of the short-format
// file.
//
CREALArray arValues; // Implicit 3-D array that stores all the data values
// at each grid point
CIntArray anPoints; // Implicit 2-D array of weights for each grid point
// (defaults to 1)
};
/////////////////////////////////////////////////////////////////////////////
// Function Prototypes
BOOL ReadShortFormatFile( LPCTSTR, CShortFormatContent& );
BOOL WriteShortFormatFile( LPCTSTR, const CShortFormatContent& );
/////////////////////////////////////////////////////////////////////////////
// Function Definitions
//===========================================================================
//
// ReadShortFormatFile()
//
//---------------------------------------------------------------------------
//
// PURPOSE: This function reads in the specified short-format file and stores
// its contents in the CShortFormatContent structure specified by
// content.
//
// INPUTS: lpszFileName: Path to the short-format file to be read.
// content: Structure which will receive the contents of the short-
// format file.
//
// RETURNS: [BOOL] Nonzero if the file was read successfully; otherwise 0.
//
//===========================================================================
BOOL ReadShortFormatFile( LPCTSTR lpszFileName, CShortFormatContent& content )
{
/////////////////////////////////////////////////////////////////////////
// VARIABLE DECLARATIONS //
CFile file; // File used to read object from disk
char szBuffer[ COMMENT_LENGTH ]; // Buffer used for reading header comments
int nComments, // Number of comments in header
nI; // Loop control variable
// //
/////////////////////////////////////////////////////////////////////////
// Attempt to open the file in read mode
if( !file.Open( lpszFileName, CFile::modeRead | CFile::shareDenyWrite ) )
return( FALSE );
try
{
// Read in the general parameters from the short-format header
file.Read( &content.nCols, sizeof( int ) );
file.Read( &content.nRows, sizeof( int ) );
file.Read( &content.nValues, sizeof( int ) );
file.Read( &content.nPhotos, sizeof( int ) );
file.Read( &content.rX0, sizeof( REAL ) );
file.Read( &content.rY0, sizeof( REAL ) );
file.Read( &content.rDeltaX, sizeof( REAL ) );
file.Read( &content.rDeltaY, sizeof( REAL ) );
// Read in the number of comments and set the size of the comment array
file.Read( &nComments, sizeof( int ) );
content.astrComments.SetSize( nComments );
// Read in each comment
for( nI = 0; nI < nComments; nI++ )
{
file.Read( szBuffer, COMMENT_LENGTH );
content.astrComments[ nI ] = szBuffer;
}
// Allocate memory for the Values and Points arrays and read in each one
content.arValues.SetSize( content.nCols * content.nRows * content.nValues );
content.anPoints.SetSize( content.nCols * content.nRows );
file.Read( content.arValues.GetData(), content.arValues.GetSize() * sizeof( REAL ) );
file.Read( content.anPoints.GetData(), content.anPoints.GetSize() * sizeof( int ) );
// Close the file object
file.Close();
// File was read successfully
return( TRUE );
}
catch( CFileException* pE )
{
// Do default exception processing
pE->ReportError();
pE->Delete();
return( FALSE );
}
catch( CMemoryException* pE )
{
// Do default exception processing
pE->ReportError();
pE->Delete();
return( FALSE );
}
}
//===========================================================================
//
// WriteShortFormatFile()
//
//---------------------------------------------------------------------------
//
// PURPOSE: This function writes the CShortFormatContent structure specified
// by content to the file given by lpszFileName.
//
// INPUTS: lpszFileName: Path to the short-format file to be created.
// content: Structure which contains the contents of the short-
// format file.
//
// RETURNS: [BOOL] Nonzero if the file was written successfully; otherwise 0.
//
//===========================================================================
BOOL WriteShortFormatFile( LPCTSTR lpszFileName, const CShortFormatContent& content )
{
/////////////////////////////////////////////////////////////////////////
// VARIABLE DECLARATIONS //
CFile file; // File used to write object to disk
int nComments, // Number of comments in the header
nI; // Loop control variable
// //
/////////////////////////////////////////////////////////////////////////
// Attempt to open the file in write mode
if( !file.Open( lpszFileName, CFile::modeWrite | CFile::modeCreate | CFile::shareDenyWrite ) )
return( FALSE );
try
{
// Write the general parameters of the short-format header
file.Write( &content.nCols, sizeof( int ) );
file.Write( &content.nRows, sizeof( int ) );
file.Write( &content.nValues, sizeof( int ) );
file.Write( &content.nPhotos, sizeof( int ) );
file.Write( &content.rX0, sizeof( REAL ) );
file.Write( &content.rY0, sizeof( REAL ) );
file.Write( &content.rDeltaX, sizeof( REAL ) );
file.Write( &content.rDeltaY, sizeof( REAL ) );
// Write the number of comments
nComments = content.astrComments.GetSize();
file.Write( &nComments, sizeof( int ) );
// Write each comment (NOTE: we must make sure each comment is COMMENT_LENGTH
// characters long, otherwise we face a possible invalid address problem)
for( nI = 0; nI < nComments; nI++ )
{
file.Write( content.astrComments[ nI ].GetBuffer( COMMENT_LENGTH ), COMMENT_LENGTH );
content.astrComments[ nI ].ReleaseBuffer();
}
// Write the Values and Points arrays
file.Write( content.arValues.GetData(), content.arValues.GetSize() * sizeof( REAL ) );
file.Write( content.anPoints.GetData(), content.anPoints.GetSize() * sizeof( int ) );
// Close the file object
file.Close();
// File was written successfully
return( TRUE );
}
catch( CFileException* pE )
{
// Do default exception processing
pE->ReportError();
pE->Delete();
return( FALSE );
}
}
Ordering of Multi-Dimensional Arrays
Both the Data Values and the Data Weights arrays implicitly represent
multi-dimensional arrays. Specifically, the Data Values array is
three-dimensional, while the Data Weights array is two-dimensional. The ordering
of elements in these arrays is described below.
Data Values Array
The elements in the Data Values array are indexed as (column, row, value).
Where the first array index (column) varies the fastest. The sample code below
illustrates how one would iterate through the Data Values array.
// nCol, nRow, nVal: Loop control variables
// nCols, nRows: Size of grid
// nVals: Number of values stored at each grid point
// nIndex: Equivalent 1-D index given a set of 3-D indices
// arValues: 1-D array of data values
// rValue: Current data value
for( nCol = 0; nCol < nCols; nCol++ )
{
for( nRow = 0; nRow < nRows; nRow++ )
{
for( nVal = 0; nVal < nVals; nVal++ )
{
nIndex = nCols * (nVal * nRows + nRow) + nCol;
rValue = arValues[ nIndex ];
}
}
}
Data Weights Array
The elements in the Data Weights array are indexed as (column, row). Where
the first array index (column) varies the fastest. The sample code below
illustrates how one would iterate through the Data Weights array.
// nCol, nRow, nVal: Loop control variables
// nCols, nRows: Size of grid
// nIndex: Equivalent 1-D index given a set of 2-D indices
// anWeights: 1-D array of data weights
// nWeight: Current data weight
for( nCol = 0; nCol < nCols; nCol++ )
{
for( nRow = 0; nRow < nRows; nRow++ )
{
nIndex = nCols * nRow + nCol;
nWeight = anWeights[ nIndex ];
}
}
Download Tech Note with Source Code