FARGOS/VISTA Object Management Environment Core  ..
FARGOS/VISTA Object Management Environment Core Table of Contents
Read_And_Process_File Class Reference

Fundamental class that can read data from a memory-mapped region, file or socket and process data in lines, fixed-sized blocks or as data is available. More...

#include <read_file.hpp>

+ Inheritance diagram for Read_And_Process_File:

Classes

struct  FileTypeReaderSelector
 Describes magic numbers needed to identify a file's type and the routine capable of decoding a file's contents. More...
 

Public Types

enum  ReadModes {
  READ_GUESS =0, READ_NORMAL =1, READ_ZLIB =2, READ_LZ4 =4,
  READ_PCAP =8, WRITE =16, WRITE_APPEND =WRITE | 32, WRITE_TRUNCATE =WRITE | 64,
  READ_FROM_SOCKET =128
}
 
typedef int(* ReadDataFP) (class Read_And_Process_File *input, unsigned char *bfr, uint32_t bfrLen)
 

Public Member Functions

 Read_And_Process_File (OS_HANDLE_TYPE srcDescriptor, ReadModes mode=READ_NORMAL, const FileTypeReaderSelector *selectorTable=nullptr)
 Construct from an existing file descriptor. More...
 
 Read_And_Process_File (const char *fileName, ReadModes mode=READ_NORMAL, const FileTypeReaderSelector *selectorTable=nullptr)
 Construct given the name of a file to be opened. More...
 
virtual ~Read_And_Process_File ()
 
void setReadRoutine (ReadDataFP altRoutine) OME_ALWAYS_INLINE
 Set a new file read routine. More...
 
virtual void noteDataRead (const unsigned char *bfr, size_t bfrLen) const
 user-exit to see original copy of any data read More...
 
int readAndProcessBlocksFromFile (size_t recordLength)
 Process fixed length records. More...
 
int findAndProcessNextLine (File_Buffer *fileBfr, bool hasHeaderLine=false)
 Process next text line from buffer. More...
 
int readAndProcessTextLines (bool hasHeaderLine=false)
 Process text lines. More...
 
int readIntoFileBuffer (File_Buffer *bfr)
 
int readAndProcessFile ()
 Process file contents with no imposed structure. More...
 
virtual int beginFile ()
 
virtual int completedFile (int recordsSeen)
 
virtual int processLine (const unsigned char *line, size_t lineLen)
 
virtual int processHeaderLine (unsigned char *line, size_t lineLen)
 Interface to handle special case processing of header lines. More...
 
virtual int_fast32_t processBlock (unsigned char *block, size_t blockLen)
 
virtual int processBuffer (File_Buffer *bfrState)
 

Static Public Member Functions

static int defaultReadRoutine (Read_And_Process_File *input, unsigned char *bfr, uint32_t bfrLen)
 Default read routine. More...
 
static const FileTypeReaderSelectorfindTypeOfFile (const char *fileName, const FileTypeReaderSelector *selectorTable)
 
static OS_HANDLE_TYPE openFile (const char *fileName, ReadModes mode=READ_NORMAL)
 Open the indicated file. More...
 
static int closeFile (OS_HANDLE_TYPE fd, ReadModes mode=READ_NORMAL)
 Close an native operating system file handle. More...
 
static int findFileInPathsWithSuffixes (char *path, uint_fast32_t pathLen, const char *searchRootPaths, const char *possibleFilenames="", const char *possibleSuffixes="")
 Search for a file using a combination of directory roots and file suffixes. More...
 

Public Attributes

ReadDataFP readRoutine
 
void * auxData
 
OS_HANDLE_TYPE descriptor
 
uint32_t recordsProcessed
 
enum ReadModes readMode
 
uint32_t _explicitAlignmentPadding
 

Protected Member Functions

virtual int readIntoBuffer (unsigned char *bfr, size_t bfrLen)
 

Static Protected Member Functions

static uint32_t matchFileHeader (const FileTypeReaderSelector *criteria, const unsigned char *fileHdr, const size_t hdrLen)
 

Protected Attributes

File_BufferintermediateBuffer
 

Detailed Description

Fundamental class that can read data from a memory-mapped region, file or socket and process data in lines, fixed-sized blocks or as data is available.

Member Typedef Documentation

◆ ReadDataFP

typedef int(* Read_And_Process_File::ReadDataFP) (class Read_And_Process_File *input, unsigned char *bfr, uint32_t bfrLen)

Member Enumeration Documentation

◆ ReadModes

Enumerator
READ_GUESS 
READ_NORMAL 
READ_ZLIB 
READ_LZ4 
READ_PCAP 
WRITE 
WRITE_APPEND 
WRITE_TRUNCATE 
READ_FROM_SOCKET 

Constructor & Destructor Documentation

◆ Read_And_Process_File() [1/2]

Read_And_Process_File::Read_And_Process_File ( OS_HANDLE_TYPE  srcDescriptor,
ReadModes  mode = READ_NORMAL,
const FileTypeReaderSelector selectorTable = nullptr 
)
explicit

◆ Read_And_Process_File() [2/2]

Read_And_Process_File::Read_And_Process_File ( const char *  fileName,
ReadModes  mode = READ_NORMAL,
const FileTypeReaderSelector selectorTable = nullptr 
)
explicit

Construct given the name of a file to be opened.

Parameters
fileNamespecifies the file to be opened
modespecifies the mode for the file.
selectorTablespecifies a table of magic number prefixes and associated routines to be used if mode READ_GUESS is used.

The special mode READ_GUESS can be used to automatically select a file decoding mechanism based on the initial bytes of a file. This does not work on pipes or devices that cannot be reopened.

References _INVALID_DESCRIPTOR, auxData, defaultReadRoutine(), descriptor, findTypeOfFile(), intermediateBuffer, INVALID_HANDLE_VALUE, Read_And_Process_File::FileTypeReaderSelector::modeToSelect, openFile(), READ_GUESS, Read_And_Process_File::FileTypeReaderSelector::readInterfaceRoutine, readMode, readRoutine, and recordsProcessed.

◆ ~Read_And_Process_File()

Read_And_Process_File::~Read_And_Process_File ( )
virtual

Member Function Documentation

◆ beginFile()

virtual int Read_And_Process_File::beginFile ( )
inlinevirtual

◆ closeFile()

int Read_And_Process_File::closeFile ( OS_HANDLE_TYPE  fd,
ReadModes  mode = READ_NORMAL 
)
static

Close an native operating system file handle.

Parameters
fdspecifies the native file handle.
modespecifies the mode in which the file was opened.

References _INVALID_DESCRIPTOR, fd, OS_SOCKET_TYPE, and READ_FROM_SOCKET.

Referenced by findTypeOfFile(), and ~Read_And_Process_File().

◆ completedFile()

virtual int Read_And_Process_File::completedFile ( int  recordsSeen)
inlinevirtual

◆ defaultReadRoutine()

int Read_And_Process_File::defaultReadRoutine ( Read_And_Process_File input,
unsigned char *  bfr,
uint32_t  bfrLen 
)
static

Default read routine.

References descriptor, LOG_CERR, LOG_ENDLINE, OS_SOCKET_TYPE, READ_FROM_SOCKET, and readMode.

Referenced by Read_And_Process_File().

◆ findAndProcessNextLine()

int Read_And_Process_File::findAndProcessNextLine ( File_Buffer fileBfr,
bool  hasHeaderLine = false 
)

◆ findFileInPathsWithSuffixes()

int Read_And_Process_File::findFileInPathsWithSuffixes ( char *  path,
uint_fast32_t  pathLen,
const char *  searchRootPaths,
const char *  possibleFilenames = "",
const char *  possibleSuffixes = "" 
)
static

Search for a file using a combination of directory roots and file suffixes.

Parameters
pathspecifies a buffer into which the found file name will be stored.
pathLenspecifies the length of the buffer.
searchRootPathsspecifies a set of directories to be searched.
possibleFilenamesspecifies a set of potential file name components.
possibleSuffixesspecifies a set of possible file name suffixes.

Paths are separated by ":" on Unix systems and by ";" under Windows.

Returns
the length of the found file is returned. The path buffer will contain the file name.
Return values
-1indicates the file could not be found.

References io(), LOG_COMPONENT_CERR, LOG_ENDLINE, and R_OK.

◆ findTypeOfFile()

◆ matchFileHeader()

uint32_t Read_And_Process_File::matchFileHeader ( const FileTypeReaderSelector criteria,
const unsigned char *  fileHdr,
const size_t  hdrLen 
)
staticprotected

◆ noteDataRead()

virtual void Read_And_Process_File::noteDataRead ( const unsigned char *  bfr,
size_t  bfrLen 
) const
inlinevirtual

user-exit to see original copy of any data read

Referenced by CSV_File::processPacket(), and readIntoBuffer().

◆ openFile()

OS_HANDLE_TYPE Read_And_Process_File::openFile ( const char *  fileName,
ReadModes  mode = READ_NORMAL 
)
static

Open the indicated file.

Parameters
fileNamespecifies the file name.
modespecifies the how to open the file.
Returns
A native operating system handle to the open file.
Return values
_INVALID_DESCRIPTORis returned if the file could not be opened.

References _STDERR_FD, _STDIN_FD, _STDOUT_FD, fd, INVALID_HANDLE_VALUE, LOG_CERR, LOG_ENDLINE, NULL, OS_HANDLE_TYPE, WRITE, WRITE_APPEND, and WRITE_TRUNCATE.

Referenced by findTypeOfFile(), and Read_And_Process_File().

◆ processBlock()

virtual int_fast32_t Read_And_Process_File::processBlock ( unsigned char *  block,
size_t  blockLen 
)
inlinevirtual

◆ processBuffer()

virtual int Read_And_Process_File::processBuffer ( File_Buffer bfrState)
inlinevirtual

Reimplemented in ReadBinaryLog.

Referenced by readAndProcessFile().

◆ processHeaderLine()

virtual int Read_And_Process_File::processHeaderLine ( unsigned char *  line,
size_t  lineLen 
)
inlinevirtual

Interface to handle special case processing of header lines.

Return values
0= processed header, remaining lines are normal data
1= processed header, still more header lines to consume
2= not a header, need to process as normal data line

Reimplemented in CSV_File.

Referenced by findAndProcessNextLine().

◆ processLine()

virtual int Read_And_Process_File::processLine ( const unsigned char *  line,
size_t  lineLen 
)
inlinevirtual

Reimplemented in ReadBinaryLog.

Referenced by findAndProcessNextLine().

◆ readAndProcessBlocksFromFile()

int Read_And_Process_File::readAndProcessBlocksFromFile ( size_t  recordLength)

◆ readAndProcessFile()

int Read_And_Process_File::readAndProcessFile ( )
inline

Process file contents with no imposed structure.

NOTE: intermediateBuffer needs to be set appropriately by the subclass.

References beginFile(), completedFile(), intermediateBuffer, processBuffer(), readIntoFileBuffer(), and recordsProcessed.

◆ readAndProcessTextLines()

int Read_And_Process_File::readAndProcessTextLines ( bool  hasHeaderLine = false)

◆ readIntoBuffer()

int Read_And_Process_File::readIntoBuffer ( unsigned char *  bfr,
size_t  bfrLen 
)
protectedvirtual

References LOG_CERR, LOG_ENDLINE, and noteDataRead().

Referenced by readIntoFileBuffer().

◆ readIntoFileBuffer()

◆ setReadRoutine()

void Read_And_Process_File::setReadRoutine ( ReadDataFP  altRoutine)
inline

Set a new file read routine.

References readRoutine.

Member Data Documentation

◆ _explicitAlignmentPadding

uint32_t Read_And_Process_File::_explicitAlignmentPadding

◆ auxData

void* Read_And_Process_File::auxData

Referenced by Read_And_Process_File().

◆ descriptor

OS_HANDLE_TYPE Read_And_Process_File::descriptor

◆ intermediateBuffer

◆ readMode

enum ReadModes Read_And_Process_File::readMode

◆ readRoutine

ReadDataFP Read_And_Process_File::readRoutine

◆ recordsProcessed


The documentation for this class was generated from the following files:
Generated: Tue Jul 28 2020 16:03:27
Support Information