ZTCL


Node:Top, Next:, Up:(dir)

ZTCL

This file documents ZTCL version 1.0.4, a TCL binding to the ZLIB library. ZTCL provides access to the ZLIB compression and decompression functionalities, as long as the capability to read and write files in the .gz format, the one used by gzip.

ZTCL has been tested with ZLIB version 1.2.1 and TCL version 8.4. To compile ZTCL the library (libz.so.1.2.1 on UNIX-like systems) and the header file (zlib.h) must be installed.

ZTCL depends on TCLMORE: another extension to the TCL language. A version of TCLMORE must be installed on the system to use ZTCL; TCLMORE should be available on the Net from the same site of ZTCL. Version 0.7 is required.

This document is copyright © 2002, 2003, 2004 by Marco Maggi.


Node:Overview, Next:, Previous:Top, Up:Top

Overview of the library

ZTCL provides access to the ZLIB functions at both the TCL and C language levels:

When using ZTCL from C, the stub mechanism can be used to access all the functions in the interface: that way you can code extensions that do not need to be recompiled when using a new version of ZTCL.

If you only need to access ZLIB from C code, you better use directly the ZLIB library and leave ZTCL out of your project: the ZLIB functions are easy to use.


Node:Package, Next:, Previous:Overview, Up:Top

Accessing the functions from TCL

To load the package execute the command:

package require zlib ?VERSION?

the package name is not ztcl, it's really zlib. There's a single TCL command called [zlib] provided by the package zlib.

zlib subcommand ?options? Command
Interface to the library functionalities.


Node:Compress and Decompress, Next:, Previous:Package, Up:Top

Compressing and decompressing chunks of data

zlib compress data ?options? Command
Compresses the data stored in data, returns the compressed data. The data is treated as a ByteArray object.

Recognised options follow.

-level level
Select the compression level, possible values: default, a compromise between speed and size; 0 for no compression at all; a number between 1 and 9 (1 gives best speed, 9 gives best compression); best and fast.

zlib decompress data ?-size size? Command
Decompresses the ByteArray representation of data. If given, size is a guess of the size of the decompressed data: this can speed up the decompression operation.

Examples

Compress and decompress a chunk of data:

package require zlib

set size     [string length $data]
set zipped   [zlib  compress $data]
set unzipped [zlib  decompress $zipped -size $size]


Node:GZ Files, Next:, Previous:Compress and Decompress, Up:Top

Reading and writing files in GZ format

The ZLIB interface to .gz files is similar to the one offered by the stdio.h C language module. The ZTCL interface is similar to the standard TCL interface for files.

Examples

Commands

zlib open fileName mode Command
Opens a file named fileName compressed in .gz format for reading or writing. mode must be RDONLY or WRONLY. Returns a token that identifies the channel.

All the TCL built-in channel commands can be used on the ZTCL channels. Some options are configurable for the channel with the [fconfigure] command.

-level level
Configures the compression level (Compress and Decompress for details).
-strategy name
Configures the compresstion strategy. Accepted values are: default, filtered, huffman and rle (run-length encoding); see the documentation of ZLIB for details.


Node:Streams, Next:, Previous:GZ Files, Up:Top

Compressing streams

ZTCL allows the compression and decompression of streams of data. We can read or write data from or to a channel and compress/decompress it; this works by stacking upon the channel a transformation layer.

Streams are used to send data from one software entity to another; the concept of TCL transformation has to be interpreted as a way to process all the data flowing between the two entities, not only chunks of it: if we need to send compressed data in chunks intermixed with normal bytes, we have to use the [zlib compress] and [zlib decompress] commands, not the transformation.

A stacked transformation can be detached with the [more unstack] command, provided by TCLMORE (See Channel interfaces, for details).

Examples

Command

zlib stream channel mode ?option ...? Command
Stacks a transformation upon an already existent channel. mode can be RDONLY, WRONLY or RDWR, and must be compatible with the open mode of channel. Both the directions can be in compress or decompress mode, even both in compress or both in decompress mode.

Description of supported options follows.

-input compress
-input decompress
Selects a compression or decompression stream for the input direction. The default if this option is not used is decompression.
-output compress
-output decompress
Selects a compression or decompression stream for the output direction. The default if this option is not used is compression.
-level LEVEL
Configures the compression level.

The compression strategy is not configurable for streams.

A stacked transformation accepts the following configuration options through [fconfigure].

-flush input
-flush output
Forces the flush operation on data in the internal context of the ZLIB stream.

For output streams: this causes as much data as possible to be flushed from the internal context to the output buffer and sent to the underlying channel, we should invoke [flush $channel] before this.

For input streams: this causes as much data as possible to be flushed from the internal context to the output buffer and to be available for reading; no new data is read from the underlying channel.

-finish input
-finish output
Forces the finalisation of the stream.

For input streams: this causes all the data stored in the internal stream context to be available for reading; after this: no more data can be read from the underlying channel until the transformation is unstacked. No data is read from the underlying channel.

For output streams: all the data is flushed to the output buffer as if [flush $channel] has been invoked, and an attempt is made to write all of it to the underlying channel; after this: no data can be written to the transformation.

-bufferSize SIZE
Configures the output buffer size. If the buffer is currently allocated and the number of bytes in it can fit in a buffer of new size, the buffer is reallocated (Stream Functions for details on the output buffer).
-bufferDelta NUMBER
Configures the new hysteresis parameter for the reallocation of the output buffer. When the buffer has less than NUMBER free bytes, it is reallocated enlarging it by NUMBER bytes. When the buffer has at least a number of unused bytes equal to two times NUMBER, the buffer is restricted, leaving at least NUMBER bytes free.

The following options are available in read-only mode to query the state of the channel.

-endOfInputStream
-endOfOutputStream
The value returned by [fconfigure] is true if the stream is in decompression mode and the end of stream has been detected; this means that the stream will no more process data: it is locked until we unstack the transformation. In this state we can read data from the internal context of the stream until all of it has been consumed. Some unprocessed data may have been read from the underlying channel: it is lost.


Node:Misc Commands, Next:, Previous:Streams, Up:Top

Miscellaneous commands

zlib isstrategy string Command
Returns true if string is a valid ZTCL compression strategy.

zlib islevel string Command
Returns true if string is a valid ZTCL compression level.

zlib info version Command
Returns the version of ZLIB in use.


Node:Error Codes, Next:, Previous:Misc Commands, Up:Top

Error code strings

In case of errors, the ZTCL commands set the ::errorCode variable to one of the following values: LOGIC, represents errors that should be catched when debugging the program (both TCL or C); RUNTIME, denotes errors occurred at runtime.


Node:Library, Next:, Previous:Error Codes, Up:Top

Accessing the functions from C

The C API of ZTCL provides the same functionalities of the TCL commands at the C level. We can access the functions including the installed include/ztcl.h header file.


Node:Data Types, Next:, Previous:Library, Up:Top

ZTCL data types

Ztcl_GzFile Opaque Pointer
A file descriptor: holds data used by ZLIB to describe an opened .gz file. Memory for file descriptors is allocated and freed by ZTCL functions.

Ztcl_Strategy Opaque Pointer
A compression strategy descriptor: it's a reference to a statically allocated block of informations describing a strategy and mapping values used by ZTCL to values used by ZLIB.

Ztcl_Config Struct Typedef
A structure used to hand compression configuration to the interface functions.
int level
The compression level.
Ztcl_Strategy strategy
The compression strategy.

More_Error Typedef
An opaque data type that represents an error. Error descriptors of this type hold references to values for the errorCode and errorInfo variables. This type is provided by TCLMORE (See Error Reporting, for details).

More_Block Struct Typedef
Type of input and output memory blocks used as source and destination for write and read operations on buffers. It is declared by TCLMORE. Fields description follows.
int len
The number of bytes in the block. This is an int, rather than a size_t, to make it easy to interface with TCL code.
More_BytePtr ptr
Pointer to the first byte in the block.

More_BytePtr Macro
Macro that represents a pointer to byte type. It is declared by TCLMORE.


Node:Defines, Next:, Previous:Data Types, Up:Top

Preprocessor symbols

Compression strategies, values of type Ztcl_Strategy.

ZTCL_STRATEGY_DEFAULT Macro
The default algorithm.

ZTCL_STRATEGY_FILTERED Macro
An algorithm.

ZTCL_STRATEGY_HUFFMAN Macro
Another algorithm.

ZTCL_STRATEGY_RLE Macro
Another algorithm.

The compression levels are integers in the range 0-9, the following constants can be used to select values:

ZTCL_LEVEL_DEFAULT Macro
A compromise between compression size and speed.

ZTCL_LEVEL_FAST Macro
Lower compression, better speed.

ZTCL_LEVEL_BEST Macro
High compression, worse speed.

Other symbols.

ZTCL_DEFAULT_CONFIG Macro
A default value for the Ztcl_Config structure. Selects the default compression level and strategy.


Node:Compress Functions, Next:, Previous:Defines, Up:Top

Compress/decompress functions

More_Error Ztcl_CompressObj (srcObj, config, dstObjVar) Function
Compresses data stored in a Tcl_Obj and stores it in a new Tcl_Obj.

The source buffer is the ByteArray representation of the source object; if it's not already a ByteArray object it's converted to this type: this is required because the string representation of an object is in UTF format (while the ByteArray representation is in UNI char), so if we want to store binary data into an object and extract it later intact we have to use a ByteArray. The new object will be of ByteArray type.

To learn more about how TCL treats ByteArray objects look for the UpdateStringOfByteArray() and SetByteArrayFromAny() functions in generic/tclBinary.c in the TCL source code.

The destination buffer size upper limit is the INT_MAX constant defined for the platform: this is because the ByteArray size can be that value at maximum. The maximum size of the destination buffer depends on how much memory is required to compress it, and it cannot be determined because the space required depends on the source data.

Parameters:

Tcl_Obj *srcObj
pointer to the object that holds the data to compress;
Ztcl_Config * config
the configuration for the compression; actually only the compression level is used;
Tcl_Obj ** dstObjVar
pointer to a variable that will hold a reference to the result object; in case of error the variable is left untouched.

Returns NULL or an error descriptor. The new object is produced only if no error occurs, and it will have reference counter left to zero.

More_Error Ztcl_DecompressObj (srcObj, size, dstObjVar) Function
Decompresses data from a Tcl_Obj to a Tcl_Obj.

The source and destination objects will be of ByteArray type.

The uncompress() function of ZLIB has no way to predict the size of the decompressed data: if the caller can guess a good value (for example because during the compression operation the size of the original data has been recorded somewhere) it can pass it in size.

If the caller cannot guess a good value it can pass size==0 and Ztcl_Decompress() will guess a size, then will loop reallocating the buffer until a sufficient size is found.

The destination buffer size upper limit is the INT_MAX constant defined for the platform: this is because the ByteArray size can be that value at maximum.

Parameters:

Tcl_Obj * srcObj
pointer to the object holding the compressed data;
int size
an initial guess of the decompressed data size;
Tcl_Obj ** dstObjVar
pointer to a variable that will hold a reference to the result object; in case of error the variable is left untouched.

The new object with the decompressed data will have the reference counter set to zero.

The uncompress() function will operate on the first chunk of compressed data: if the data in srcObj has been built with two or more calls to compress(), only the first chunk will be decompressed. The size parameter must be the size of the decompressed first chunk.

Returns an error descriptor, NULL if no error occurs. The new object is produced only if no error occurs, and it will have reference counter left to zero.


Node:File Functions, Next:, Previous:Compress Functions, Up:Top

File functions

Tcl_Channel Ztcl_MakeChannel (Ztcl_File token) Function
Opens a new channel interface to a ZTCL file. Returns the channel token.

More_Error Ztcl_Open (fileName, mode, tokenVar) Function
Opens a .gz file for reading or writing, not both; the file is always opened in binary mode.

Parameters:

CONST char * fileName
pointer to the string holding the file name;
int mode
TCL_READABLE or TCL_WRITABLE;
Ztcl_File * tokenVar
pointer to a variable that will hold the file descriptor.

Returns NULL or an error descriptor. The error descriptor is produced only if no error occurs.

More_Error Ztcl_Close (Ztcl_File token) Function
Closes a file previously opened with Ztcl_Open(). The file descriptor, all the resources are released. Returns NULL or an error descriptor.

More_Error Ztcl_Read (Ztcl_File token, More_Block * blockVar) Function
Reads decompressed data from a file and stores it in a memory block. It's an error to call this function for a file opened for writing.

blockVar must reference a block structure allocated by the caller.

Before the call
The len field must hold the number of decompressed bytes to be read; the ptr field must be the pointer to a memory block allocated by the caller, large enough to hold len bytes.

If the len value is zero or negative no bytes will be read.

After the call
The len field will hold the number of bytes actually read, or zero if an error occurred or no bytes were read; the ptr field is left untouched.

Returns NULL or an error descriptor.

More_Error Ztcl_ReadObj (Ztcl_File token, int number, Tcl_Obj ** dstObjVar) Function
Reads data from a file and stores it in a new ByteArray object. The number of bytes read is number and a pointer to the new object is stored in the variable referenced by dstObjVar, with reference counter set to zero. Returns NULL or an error descriptor.

More_Error Ztcl_Write (Ztcl_File token, More_Block block) Function
Compresses data from a memory block and writes the result in a file. It's an error to call this function for a file opened for reading.

The len field of the block structure must be the number of bytes to compress, the ptr field must reference a the first byte in a buffer of at least len bytes.

Returns NULL or an error descriptor.

More_Error Ztcl_WriteObj (Ztcl_File token, Tcl_Obj * srcObj) Function
Compresses data from an object into a file. The source data is the ByteArray representation of the object. Returns NULL or an error descriptor.

int Ztcl_Eof (Ztcl_File token) Function
Returns true when end-of-file has previously been detected reading the given input stream, otherwise returns false.

More_Error Ztcl_Flush (Ztcl_File token) Function
Flushes data for a file. It is an error to invoke this function for a readable file.

More_Error Ztcl_Seek (Ztcl_File token, int offset, int whence) Function
Changes the position of the cursor for reading operations. See the documentation of ZLIB for details on seeking.

More_Error Ztcl_Tell (Ztcl_File token, int * positionVar) Function
Retrieves the current position of the read cursor. See the documentation of ZLIB for details on seeking.

More_Error Ztcl_SetLevel (Ztcl_File token, int level) Function
Configures the compression level for a file descriptor. The compression level is an unsigned between 0 and 9. Returns NULL or an error descriptor.

int Ztcl_GetLevel (Ztcl_File token) Function
Returns the integer representing the compression level selected for the file. This value makes sense only for files opened for writing.

More_Error Ztcl_SetStrategy (Ztcl_File token, Ztcl_Strategy strategy) Function
Configures the compression strategy for a file descriptor. strategy is one of the ZTCL_STRATEGY_* constants (Defines for details). Returns NULL or an error descriptor.

Ztcl_Strategy Ztcl_GetStrategy (Ztcl_File token) Function
Returns a value of type Ztcl_Strategy representing the compression strategy selected for the file. This value makes sense only for files opened for writing.

int Ztcl_ReadableFile (Ztcl_File token) Function
Returns true if the open mode for a file is reading.

int Ztcl_WritableFile (Ztcl_File token) Function
Returns true if the open mode for a file is writing.


Node:Stream Functions, Next:, Previous:File Functions, Up:Top

Stream functions

From the point of view of a programming interface, compression and decompression are transformations with two fundamental properties: data is accumulated in a context; it is impossible to predict how many input bytes will be required to read a given number of bytes from the output.

ZLIB stream interface functions define and manage the internal compression or decompression context, we have to supply the input and output buffers.


Node:Stream Functions Overview, Next:, Up:Stream Functions

Overview

Allocate only the output buffer, process input data immediately consuming all of it. This is a simpler and less efficient solution than the first one.

we supply                      the stream
an input                       has an
block                          output buffer
 -                                -
 |             --------------     |              we copy data
 | ---------> | compression/ |    | -----------> directly from
 |  Write()   |decompression |++> |    Read()    the buffer
 -            |   context    |    |
               --------------     |
                                  -

We can push data until the system has memory to allocate to the process for the output buffer. At each write operation the internal context is modified, this may slow down the execution if we write many little chunks of data.

We can pull data from the stream until the output buffer is empty. Some data may still be in the internal context.


Node:Stream Functions Examples, Next:, Previous:Stream Functions Overview, Up:Stream Functions

Examples

Processing data

Ztcl_Stream     token;
More_Block      buffer, input, output;
More_Error      error;
Ztcl_Config     config;
int             compress, numberOfBytesUsed;


compress = ...;
/* fill "config" */

error = Ztcl_StreamInit(&token, compress, &config);
if (error) { ... }

More_BlockAlloc(buffer, 4096*32);

for (input = buffer, fill_a_block_with_data(&input);
     input.len;
     input = buffer, fill_a_block_with_data(&input))
  {
    error = Ztcl_StreamWrite(token, &input);
    if (error)
      {
        goto Error; /* example: corrupted data */
      }
    if (input.len)
      {
        /*
          Some data was not absorbed: it is still in "buffer",
          referenced by "input", so do something with it.

            For compression streams: this has to be considered
          an error so: goto Error. This should never happen
          anyway.

            For decompression streams: the end of stream was
          found. This is not an exception: it is normal operation
          when we read data from a source that supplies the stream
          and other data after it.
        */
        break;
      }

    output = Ztcl_StreamOutput(token);
    if (output.len)
      {
        numberOfBytesUsed = use_the_processed_data(output);
        Ztcl_StreamRead(token, numberOfBytesUsed);
      }
    else
      {
        break;
      }
  }

Ztcl_StreamFinish(token);
output = Ztcl_StreamOutput(token);
use_the_processed_data(output);

Error:
More_BlockFree(buffer);
Ztcl_StreamFinal(token);


Node:Stream Functions Interface, Next:, Previous:Stream Functions Examples, Up:Stream Functions

Interface functions

All the stream memory is managed with the ckalloc(), ckrealloc() and ckfree() functions.

In case of error the functions return an error descriptor of type More_Error; this type is declared by TCLMORE. The only thing to do after an error is to call the finalisation function to release all the resources.

The integer data error-specific field of the error descriptor is set to an appropriate (more or less) errno value. This is to allow the use of the stream functions with the transformation layer implemented by TCLMORE; errno codes are poor information but this is what the TCL channel interface offers. In the future an -error option for [fconfigure] may be implemented, somewhat like the one offered by [socket].


Node:Stream Functions Interface Init and Final, Next:, Up:Stream Functions Interface

Initialisation and finalisation

More_Error Ztcl_StreamInit (tokenVar, compress, config) Function
Initialises a (de)compression stream. Memory is completely managed by ZTCL.
Ztcl_Stream * tokenVar
Pointer to a variable that will hold the reference to the new stream descriptor.
int compress
If true the new stream will compress data, else will decompress.
Ztcl_Config *config
Pointer to the structure holding the configuration (Data Types for details).

Returns NULL or an error descriptor.

void Ztcl_StreamFinal (Ztcl_Stream token) Function
Releases all the resources. This function may be used to abort a stream, at any instant, or to clean up after the stream has been finished.


Node:Stream Functions Interface Writing, Next:, Previous:Stream Functions Interface Init and Final, Up:Stream Functions Interface

Writing data

More_Error Ztcl_StreamWrite (Ztcl_Stream token, More_Block *blockPtr) Function
Writes data into a stream. blockPtr must reference a variable referencing the input block: its pointer field must reference a block of memory, its length field must be the number of bytes to write into the stream from the block.

For compression streams: all the data will be absorbed, the block structure will be cleared to zero.

For decompression streams:

  • if the end of stream is not detected, all the data is absorbed and the block structure is cleared to zero;
  • if the end of stream is found: no more data is processed after it, the internal context is finalised, no error is returned and data is available from the output buffer.

If end-of-stream is detected the block is updated to reference the portion of data that has not been processed: before:

              end of stream
                    v
|---------------------------------------|
^
blockPtr->ptr

.........................................
             blockPtr->len

after:

              end of stream
                    v
|---------------------------------------|
                     ^
                   blockPtr->ptr

                     ....................
                       blockPtr->len

Returns NULL or an error descriptor.


Node:Stream Functions Interface Reading, Next:, Previous:Stream Functions Interface Writing, Up:Stream Functions Interface

Reading data

More_Block Ztcl_StreamOutput (Ztcl_Stream token) Function
Accesses the output buffer. This function is meant to be used just before Ztcl_StreamRead(). Returns a block representing the output buffer; it may reference a NULL buffer.

void Ztcl_StreamRead (Ztcl_Stream token, int number) Function
Declares that a number of bytes has been read from the output buffer. Shifts the unread data to the beginning of the output buffer. Possibly restricts the output buffer.


Node:Stream Functions Interface Flush and Finish, Next:, Previous:Stream Functions Interface Reading, Up:Stream Functions Interface

Flusing and finishing

More_Error Ztcl_StreamFlush (Ztcl_Stream token) Function
Flushes as much data as possible from the internal context. It is required if we need to send data somewhere quickly, and usually it is not used. Its use is discouraged because it slows down the operations and may degrade the compression ratio. After the flush operation has been carried out, new data may have been added to the output buffer, and so it can be read. Returns NULL or an error descriptor.

More_Error Ztcl_StreamFinish (Ztcl_Stream token) Function
For compression streams: flushes data from the internal context to the output buffer, marking it as end of stream.

For decompression streams: flushes data from the internal context until the end of stream is found; this works only if we have written all the data into the stream, else a "corrupted data" error is raised.

After a call to this function data is available with Ztcl_StreamOutput(). When data has been read the only allowed operation is finalisation.


Node:Stream Functions Interface Output Buffer, Next:, Previous:Stream Functions Interface Flush and Finish, Up:Stream Functions Interface

Output buffer

The dimension of the output buffer can be configured after the stream has been initialised: the buffer is allocated only at the first invocation to Ztcl_StreamWrite().

Repeated invocations to the write function will enlarge the buffer if the unused space is less than a configurable hysteresis value. The formula used to compute the new size follows.

newSize = oldSize + oldSize % hysteresis + hysteresis;

The size will always be a multiple of the hysteresis value, and after reallocation there always be a number of unused bytes greater or equal to the hysteresis value. Invocations to the write function will never shrink the buffer.

Invocations to Ztcl_StreamRead() may reallocate the buffer, shrinking it. The reallocation will take place if the number of unused bytes is greater than twice the configured hysteresis value. The formula used to compute the new size follows.

newSize = usedSpace + usedSpace % hysteresis + hysteresis;

The default value for both the size and hysteresis is 4096*4.

void Ztcl_StreamSetBufferSize (Ztcl_Stream token, int size) Function
Selects a new size for the output buffer.

If the buffer is not allocated: size will be its initial size.

If the buffer is allocated at the time of the call and the number of used bytes is less than size: the buffer is reallocated to the new size.

void Ztcl_StreamSetBufferHysteresis (Ztcl_Stream token, int hysteresis) Function
Selects a new hysteresis for the reallocation of the buffer.

int Ztcl_StreamGetBufferSize (Ztcl_Stream token) Function
Returns the current buffer size.

int Ztcl_StreamGetBufferHysteresis (Ztcl_Stream token) Function
Returns the current hysteresis value.

int Ztcl_StreamBufferAllocated (Ztcl_Stream token) Function
Returns true if the buffer is allocated.


Node:Stream Functions Interface Inspection, Previous:Stream Functions Interface Output Buffer, Up:Stream Functions Interface

Inspection

int Ztcl_StreamGetLevel (Ztcl_Stream token) Function
Returns the compression level.

int Ztcl_StreamFinished (Ztcl_Stream token) Function
Returns true if the stream has been finished: explicitly with Ztcl_StreamFinish() or because the end of stream was detected in a write or flush operation.


Node:Stream Functions Transformation, Next:, Previous:Stream Functions Interface, Up:Stream Functions

Transformation

The stream transformation can be stacked upon an existing channel. To remove it use Tcl_UnstackChannel().

Tcl_Channel Ztcl_MakeTransform (input, output, subChannel) Function
Creates a new channel interface to already created stream descriptors. The transformation is stacked upon an existing channel. Arguments description follows.
Ztcl_Stream input
The input stream token. Can be NULL if the transformation is write-only.
Ztcl_Stream output
The output stream token. Can be NULL if the transformation is read-only.
Tcl_Channel subChannel
The underlying channel token.

Returns the transformation's channel token.

The transformation module is implemented by TCLMORE. Ztcl_MakeTransform() is a wrapper for More_MakeStreamTransform() (See Public interface, for details).


Node:Stream Functions Internals, Previous:Stream Functions Transformation, Up:Stream Functions

Internals

The source code file is generic/stream.c. This module is composed by four elements.

Compression/decompression context
It is a data structure holding the internal state of the stream; it is handled mainly by the ZLIB functions.

Data is absorbed registering into the context a reference to a block of data supplied by the user of the module, and extracted from it registering a reference to unused space in the output buffer. The processing functions of ZLIB will transfer the data. There must always be an output block to which send data; an input block must be present when processing data in normal mode, while when flushing or finishing a context input data must not be present.

Compression/decompression drivers
The ZLIB functions are wrapped by "drivers": submodules whose functions are collected in a table of function pointers, to provide a common interface for compression and decompression. These tables may be viewed as implementations of virtual functions (in the C++ sense).

The driver instances are statically allocated, have no version field and the logic used to select and call them is hard-coded: there is no plan to split the stream module to allow plugging in of other drivers.

The functions offered by the ZLIB interface are three for compression and three for decompression: initialisation, finalisation, processing; the processing functions can be invoked requesting different operations: common processing, flushing, finishing. The driver functions are four: initialisation, finalisation, writing, flushing.

Output buffer
It is where the data goes after being processed by the stream. When data is written to the stream: it is processed and accumulated in the output buffer; some of it may be accumulated in the internal context.

Reading data from the stream does not involve usage of the drivers: it is an operation that acts upon the output buffer only.

An effort was attempted to split completely the output buffer submodule from the rest: the result was that the synchronisation required between the buffer and the internal context nearly doubled the size of the code for some operations, so the idea was abandoned by the author. In the end, the output buffer functions are just wrappers for ckalloc() and ckrealloc().

Public interface
These functions are all prefixed with Ztcl_. Basically: they are wrappers for the driver functions; at initialisation time a driver is selected and its functions are invoked through the pointers in it.

Data types

Stream Struct Typedef
The basic data structure used to handle the stream, in it are allocated the internal context and output buffer instances.

The first argument to most of the functions in the module is a pointer to a structure of this type. This is the case of the opaque pointer used as first argument for the public interface functions.

Driver Struct Typedef
It is the table of function pointers used to provide a common interface for both compression and decompression.

Each driver function has its own type declaration, to make it easy to declare the prototypes.

Operations

Initialisation

It is straightforward: a Stream data structure is allocated and initialised with default values; the configuration values are stored in it along with a reference to the selected driver: compression or decompression. The initialisation function of the driver is invoked to prepare the internal context. The output buffer is not allocated: this will be done at the first write operation.

Writing

Steps: the input block supplied by the user is registered in the context; the processing function of ZLIB is invoked multiple times in normal mode until all the data has been absorbed; the input block is unregistered.

Invoking the processing function multiple times is required because, after each call, the output buffer may be full but some unread data may still be in the input block; the buffer is checked prior to each invocation to make sure that room is available.

Compression
Writing never ends a stream, we can process data at will.
Decompression
Each writing operation may find the end of stream in the middle of the input block; if this happens: the stream is finished correctly and a reference to the unread data is handed to the caller, so that it can be put back where it came from.
Flushing

This operation attempts to extract data from the internal context, without processing new data.

Compression
The processing function is invoked in a loop in flushing mode until it signals that all the data has been processed, or an error occurs.
Decompression
The processing function is invoked in a loop in flushing mode until: the end of stream is found, no more data is appended to the output buffer, or an error occurs. If the end of stream is detected the internal context is correctly finished and subsequent operations different from reading will be ignored.

It is not clear from the ZLIB documentation if the decompression accumulates data in an internal context or not; when the decompression function does not append new data to the output buffer, the operation is considered carried out.

Finishing

Data still in the internal context is processed and appended to the output buffer. The processing function of ZLIB is invoked in finish mode until all data has been flushed.

Compression
A stream can be terminated at any time.
Decompression
A stream can be terminated successfully only if all the compressed data has been absorbed in the internal context. ZLIB refuses to absorb data past the end of the stream, so there is no problem of handling superfluous data.

When a stream is finished: the event is recorded in the descriptor, so that no other operation will be performed on it.

Output buffer allocation

void OutputBufferEnlarge (Stream * stream) Function
If the buffer has not been allocated yet: allocates it with the configured size. Else, if required, enlarge the allocated data block.

When the number of unused bytes in the buffer is less than the configured hysteresis value, the block is reallocated. The new size is a multiple of the hysteresis value, and it is such that at least a number of bytes equal to the hysteresis value is unused.

This function is invoked before processing data through the stream. A reference to the unused space is registered in the context of the stream.

void OutputBufferShrink (Stream * stream) Function
Shrinks the output buffer to reduce memory usage. If the number of unused bytes is at least twice the configured hysteresis value, the block is reallocated. The new size is computed in the same way as when the block is enlarged.

This function is invoked after data has been read from the output buffer.

ZLIB memory allocation

void * AllocFunc (void *opaque, int number, int size) Function
Memory allocator used by the ZLIB library to manage its internal context. It is a wrapper for ckalloc().

void FreeFunc (void * opaque, void * block) Function
Memory releaser used by the ZLIB library to manage its internal context. It is a wrapper for ckfree().


Node:Misc Functions, Next:, Previous:Stream Functions, Up:Top

Miscellaneous functions

int Ztcl_GetLevelFromObj (interp, obj, levelPtr) Function
Extracts a valid ZLIB compression level from an object; valid string values are: best, fast, default, any number in the range [0-9].

Parameters:

Tcl_Interp * interp
if not NULL, the interpreter in which report errors;
Tcl_Obj * obj
pointer to the object holding the value;
int * levelPtr
pointer to the variable that will hold the integer identifying the level.

Returns TCL_OK if a good value is found, otherwise returns TCL_ERROR and leaves an error message in the interpreter (the error code is not set).

int Ztcl_IsLevel (int level) Function
Tests a value: returns true if level is acceptable as compression level, otherwise returns false.

int Ztcl_GetStrategyFromObj (interp, obj, strategyPtr) Function
Extracts a valid ZTCL compression strategy from an object; valid string values are: default, filtered, huffman, rle. See the ZLIB documentation for details.

Parameters:

Tcl_Interp * interp
if not NULL, the interpreter in which report errors;
Tcl_Obj * obj
pointer to the object holding the value;
Ztcl_Strategy * strategyPtr
pointer to the target variable.

Returns TCL_OK if a good value was found; otherwise TCL_ERROR and leaves an error in the interpreter (the error code is not set).

int Ztcl_IsStrategy (Ztcl_Strategy strategy) Function
Tests a value: returns true if strategy is acceptable as compression strategy, otherwise returns false.

CONST char * Ztcl_Version (void) Function
Returns a pointer to a statically allocated string representing the ZLIB version number.


Node:Using stubs, Next:, Previous:Misc Functions, Up:Top

Using the stubs mechanism

The stubs mechanism allows us to dynamically link a client extension to a version of ZTCL and to use it with future versions, without recompiling, as long as the future versions do not change the interface.

To do this we link our client extension with ZTCL's stub library (an object file whose name is something like liztclstub...) and compile our code with the symbol USE_ZTCL_STUB defined. Our client library's initialisation function must contain the following code:

#include "ztcl.h"

...

int
Client_Init (...)
{
...

#ifdef USE_ZTCL_STUB
  if (Ztcl_InitStubs(interp, "1.0", 0) == NULL)
    {
      return TCL_ERROR;
    }
#endif

...
}

where 1.0 is the version of ZTCL that the client library is supposed to use.


Node:Package License, Next:, Previous:Using stubs, Up:Top

BSD style license

Copyright © 2002, 2003, 2004 Marco Maggi.

The author hereby grant permission to use, copy, modify, distribute, and license this software and its documentation for any purpose, provided that existing copyright notices are retained in all copies and that this notice is included verbatim in any distributions. No written agreement, license, or royalty fee is required for any of the authorized uses. Modifications to this software may be copyrighted by their authors and need not follow the licensing terms described here, provided that the new terms are clearly indicated on the first page of each file where they apply.

IN NO EVENT SHALL THE AUTHOR OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHOR HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE AUTHOR AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHOR AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.


Node:Documentation License, Next:, Previous:Package License, Up:Top

Documentation license

This document is copyright © 2002, 2003, 2004 by Marco Maggi.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.


Node:Credits, Next:, Previous:Documentation License, Up:Top

Who wrote what

ZTCL was written by Marco Maggi with no third party code.

From the ZLIB README file:

zlib 1.1.4 is a general purpose data compression library. All the code is thread safe. The data format used by the zlib library is described by RFCs (Request for Comments) 1950 to 1952 in the files <http://www.ietf.org/rfc/rfc1950.txt> (zlib format), rfc1951.txt (deflate format) and rfc1952.txt (gzip format). ...

Questions about zlib should be sent to zlib@gzip.org, or to Gilles Vollant info@winimage.com for the Windows DLL version. The zlib home page is <http://www.zlib.org> or <http://www.gzip.org/zlib/>. Before reporting a problem, please check this site to verify that you have the latest version of zlib; otherwise get the latest version and check whether the problem still exists or not.

PLEASE read the zlib FAQ <http://www.gzip.org/zlib/zlib_faq.html> before asking for help. ...

The deflate format used by zlib was defined by Phil Katz. The deflate and zlib specifications were written by L. Peter Deutsch. Thanks to all the people who reported problems and suggested various improvements in zlib; they are too numerous to cite here. ...

The library has been entirely written by Jean-loup Gailly and Mark Adler; it does not include third-party code. ...

A ZLIB binding for TCL written by Andreas Kupries is available at:

<http://www.westend.com/~kupries/doc/trf/man/man.html>

other bindings can be found by doing a search on:

<http://wiki.tcl.tk/>


Node:Concept Index, Previous:Credits, Up:Top

An entry for each concept, command, function and data type

Table of Contents