storage.h File Reference

#include <sys/types.h>
#include "globals.h"

Data Structures

Defines

Typedefs

Functions


Define Documentation

#define MALLOCED   2
#define memblob_allocate   alloc_mblob
#define memblob_clear   init_mblob
#define memblob_free   mfree
#define memblob_read_from_file   read_file_into_blob
#define memblob_write_to_file   write_file_from_blob
#define MMAPPED   1

Flag: use mmap() to allocate memory.

Referenced by load_component(), mfree(), read_file_into_blob(), and write_file_from_blob().

#define PAGED   3

Referenced by mfree().

#define SIZE_BIT   0

Referenced by alloc_mblob(), and read_file_into_blob().

#define SIZE_BYTE   sizeof(char)
#define SIZE_INT   sizeof(int)

Referenced by creat_rev_corpus_idx().

#define SIZE_LONG   sizeof(long)
#define SIZE_SHINT   sizeof(short)
#define UNALLOCATED   0

Typedef Documentation

typedef struct TMblob MemBlob

The MemBlob object.

This object, unsurprisingly, represents a blob of memory.


Function Documentation

int alloc_mblob ( MemBlob blob,
int  nr_items,
int  item_size,
int  clear_blob 
)

Allocates memory for a blob of the requested size.

A block of memory holding nr_items of size item_size is created in the specified MemBlob.

Parameters:
blob The MemBlob in which to place the memory.
nr_items The number of items the MemBlob is to hold as data.
item_size The size of one item.
clear_blob boolean: if true, all bytes in the data space will be initialised to 0
Returns:
boolean: true 1 if OK, false on error

References TMblob::allocation_method, TMblob::changed, cl_calloc(), cl_malloc(), TMblob::data, TMblob::fname, TMblob::fsize, TMblob::item_size, MALLOCED, TMblob::nr_items, TMblob::offset, TMblob::size, SIZE_BIT, and TMblob::writeable.

void init_mblob ( MemBlob blob  ) 

Clears all fields in a MemBlob, regardless of their usage, and puts the blob back to its virginal state.

Note that it doesn't free blob->data - just sets it to NULL.

References TMblob::allocation_method, TMblob::changed, TMblob::data, TMblob::fname, TMblob::fsize, TMblob::item_size, TMblob::nr_items, TMblob::offset, TMblob::size, UNALLOCATED, and TMblob::writeable.

Referenced by declare_component(), and mfree().

caddr_t mallocfile ( char *  filename,
size_t *  len_ptr,
char *  mode 
)

Maps a file into memory.

This function does virtually the same as mmapfile (same parameters, same return value), but the memory is taken with malloc(3), not with mmap(2).

See also:
mmapfile

References cl_malloc().

Referenced by read_file_into_blob().

void mfree ( MemBlob blob  ) 

Frees the memory used by a MemBlob.

This works regardless of the method used to allocate the blob.

References TMblob::allocation_method, TMblob::data, TMblob::fname, init_mblob(), MALLOCED, MMAP_EMPTY_LEN, MMAPPED, PAGED, TMblob::size, and UNALLOCATED.

Referenced by comp_drop_component().

caddr_t mmapfile ( char *  filename,
size_t *  len_ptr,
char *  mode 
)

Maps a file into memory in either read or write mode.

Parameters:
filename Name of the file to map.
len_ptr The number of bytes the returned pointer points to.
mode Can be either "r" or "w", nothing else. If mode is "r", len_ptr is taken as an input parameter (*len_ptr bytes are allocated) {NB I copied this from existing notes but surely shouldn't the last comment apply if mode is "w" not "r"? -- AH}
Returns:
The contents of file in filename as a pointer to a memory area.

References MMAP_EMPTY_LEN, and MMAPFLAGS.

Referenced by read_file_into_blob().

void NreadInt ( int *  val,
FILE *  fd 
)

Reads an integer from file, converting from network byte order.

This function does all the error checking for you, and will abort the program if the int cannot be read.

Parameters:
val Location to put the resulting int.
fd File handle to read from

References word.

Referenced by decode_check_huff(), and ReadHCD().

void NreadInts ( int *  vals,
int  nr_vals,
FILE *  fd 
)

Reads an array of integers from file, converting from network byte order.

This function does all the error checking for you, and will abort the program if the requested number of ints cannot be read.

Parameters:
vals Pointer to location to put the resulting array of ints.
nr_vals Number of integers to read.
fd File handle to read from

References word.

Referenced by ReadHCD().

void NwriteInt ( int  val,
FILE *  fd 
)

Writes an integer to file, converting to network byte order.

Other than the byte order conversion, this is the same as fwrite(&val, sizeof(int), 1, fd) .

Parameters:
val The integer to write.
fd File handle to write to.

References word.

Referenced by addline(), close_range(), compress_reversed_index(), compute_code_lengths(), creat_rev_corpus(), main(), write_region_to_disk(), and WriteHCD().

void NwriteInts ( int *  vals,
int  nr_vals,
FILE *  fd 
)

Writes an array of integers to file, converting to network byte order.

Other than the byte order conversion, this is the same as fwrite(vals, sizeof(int), nr_vals, fd) .

Parameters:
vals Pointer to the location of the block of integers to write.
nr_vals Number of integers to write.
fd File handle to write to.

References word.

Referenced by creat_rev_corpus(), write_file_from_blob(), and WriteHCD().

int read_file_into_blob ( char *  filename,
int  allocation_method,
int  item_size,
MemBlob blob 
)

Reads the contents of a file into memory represented by blob.

You can choose the memory allocation method - MMAPPED is faster, but writeable areas of memory should be taken with care. MALLOCED is slower (and far more space consuming), but writing data into malloced memory is no problem.

Parameters:
filename The file to read in.
allocation_method MMAPPED or MALLOCED (see function description)
item_size This is used for MemBlob access methods, it is simply copied into the MemBlob data structure.
blob The MemBlob to read the file into. It must not be in use -- the fields are overwritten.
Returns:
0 on failure, 1 if everything went fine.

References TMblob::allocation_method, TMblob::changed, TMblob::data, TMblob::item_size, MALLOCED, mallocfile(), mmapfile(), MMAPPED, TMblob::nr_items, TMblob::size, SIZE_BIT, UNALLOCATED, and TMblob::writeable.

Referenced by creat_freqs(), creat_sort_lexicon(), and load_component().

int write_file_from_blob ( char *  filename,
MemBlob blob,
int  convert_to_nbo 
)

Writes the data stored in a blob to file.

Parameters:
filename The file to write to.
blob The MemBlob to write to file.
convert_to_nbo boolean: if true, data is converted to network byte order before it's written.
Returns:
0 on failure, 1 if everything went fine.

References TMblob::allocation_method, TMblob::changed, TMblob::data, MALLOCED, MMAPPED, NwriteInts(), TMblob::size, and UNALLOCATED.

Referenced by creat_freqs(), creat_rev_corpus_idx(), and creat_sort_lexicon().


Generated on Sun Feb 28 18:08:04 2010 for CWB by  doxygen 1.6.1