tclmore


Node:Top, Next:, Up:(dir)

tclmore

TCLMORE is a C language extension library for TCL. It provides additional commands to a TCL interpreter and a set of C API functions accessible through the stub mechanism. This file documents version 0.7.1 of TCLMORE.


Node:Overview, Next:, Previous:Top, Up:Top

Overview

This file documents TCLMORE version 0.7.1. This package adds some commands to a TCL interpreter and provides a table of functions implementing useful features. The functions are accessible through the stub mechanism.

This package is mostly a base for other extensions: the most important modules are the one exposed at the C language level, not the ones at the TCL level. The purposes of the modules is to make the TCL C API more friendly (example: TCL variable functions), or more organised (example: channel driver facilities), or to allow automation of code generation (example: the TCL command interface).

Officially TCLMORE does not support threads.


Node:Loops, Next:, Previous:Overview, Up:Top

Additional loop statements

more do body expr Command
Evaluates body in the scope of the caller while expr is true. Honors the [break], [continue], [error] and [return] exceptions. body is evaluated at least once. The return value is the result of the last command in body.
more do {
   # do something
} {
   #condition
}

more loop integer body Command
Evaluates body in the scope of the caller integer times. Honors the [break], [continue], [error] and [return] exceptions. The return value is the result of the last command in body. This is a little faster than a [for] loop.
more loop 10 {
   # repeat this 10 times
}


Node:Channels, Next:, Previous:Loops, Up:Top

Channel interfaces

TCLMORE adds commands that make use of the TCL channel interface to read and write data to and from variables. These functionalities can be used to debug TCL code modules that interface with channels. Another module allows the creation of "pipes": couples of channels linked together that can be shared among interpreters and threads.

more unstack channel Command
Removes a transformation from the top of a channel's stack.


Node:Varchan, Next:, Up:Channels

Channel interface to variables

A "variable channel" (varchan) is a channel linked to a variable that allows data to be read from and written to it. The bytes written are appended to the variable's content interpreted as a byte array; the bytes read are extracted from the variable's content.

Example:

set channel [more varchan target RDONLY]
set target  "some text"
set text [read $channel 4]
close $channel
# $text -> "some"

Command

more varchan variable mode Command
Opens a new variable channel and returns its identifier. variable is the name of the variable, mode can be: RDONLY, WRONLY, RDWR. Write-only varchans allow to write bytes into the channel and to find them later in the content of a variable; read-only varchans allow to store bytes into a variable and read them later from the channel.

If the variable does not exist: it is created by this command and set to the empty string. The variable must be: readable, writable and free of traces that raise errors.

For readable channels: data present in the variable when [varchan] is executed is immediately sent to the channel.

set target abc
set channel [more varchan target RDONLY]
set data [read $channel]; # $data -> abc

For writable channels: data present in the variable when [varchan] is executed is kept in the variable.

set target abc
set channel [more varchan target WRONLY]
puts -nonewline $channel def
# $target -> abcdef

Special options are recognised by a varchan.

-inputBufferSize SIZE
-outputBufferSize SIZE
The internal buffers are chains of memory blocks. These options configure the size in bytes of each block, allowing an attempt in minimisation of memory reallocation.

Variable name

The name can reference an automatic variable or namespace variable: the resolution will always be from the scope of the invoking command. That is, in the following script:

proc green { } {
    set chan [more varchan target WRONLY]
    fconfigure $chan -buffering none
    puts -nonewline $chan "string"
    white $chan
    close $chan
}

proc white { var chan } {
    upvar $var $var
    puts -nonewline $chan " gulp"
    # [set $var] -> "string gulp"
}

green

the [puts] and [close] commands will find the correct target variable (notice that the two arguments of [upvar] are equal); in the following script:

proc green { } {
    set chan [more varchan target WRONLY]
    puts $chan "string"
    return $chan
}

set chan [green]
puts $chan wo
close $chan

after the termination of the procedure execution, the variable is unset and so detached from the channel; an attempt to access the channel to read will return zero bytes and the "end of file" condition; an attempt to access the channel to write will raise an error, with EPIPE as POSIX code (probably causing the error message to be broken pipe); note that an invocation to [close] on the channel is correct even if the linked variable no longer exists.

If we do:

proc green { } {
    set chan [more varchan target WRONLY]
    fconfigure $chan -buffering none
    puts -nonewline $chan "string"
    white $chan
    close $chan
    # $target -> "string gulp"
}

proc white { chan } {
    puts -nonewline $chan " gulp"
}

the [puts] command in the [white] procedure will write the string into the varchan internal buffer: the variable is not involved in this operation; when, back in [green], the variable is read all the text is there.

If the name is qualified: the code will look for a variable in a namespace; with a qualified variable name we can share the channel between procedures:

namespace eval red {
    variable   target
}

proc red::green { } {
    variable   target
    set chan [more varchan target WRONLY]
    puts $chan "string"
    white $chan
    close $chan
}

Summary: if we access a variable channel in the scope of a procedure we have to make the correct variable accessible from the procedure, with [global], [upvar], [variable] or by using a qualified name.

Operations

In this section a description of varchan operations is given. To understand how a varchan works we have to know that data is buffered internally for both the reading and writing directions; this buffering is completely separated from the TCL buffering on channels: even when [fconfigure] is invoked with the -buffering none option, a varchan does its own buffering.

The data in the variable is always seen as a byte array.

Reading the variable
When the variable is read and the associated channel is open for writing: a trace is executed and variable is filled with data currently in the output buffer of the varchan. Each time the variable is read the trace extracts all the bytes from the internal buffer and it appends it to the current variable contents.

If the channel is closed: reading the variable will consume all the bytes still in the internal output buffer: when all the data has been consumed the internal buffer is released.

Setting the variable
When the variable is set and the associated channel is open for reading: a trace is executed and the variable's contents extracted from the variable and written in the varchan input buffer. If the variable is set to the empty string: the internal input buffer is left untouched.

After the channel is closed: the first time the variable is set the trace will detect the situation and release the input buffer.

Reading the channel
When a read operation is executed: the requested number of bytes is extracted from the TCL input buffer, if any, and if not enough from the internal varchan input buffer.

If the variable is unset: at the first read operation on the channel that requests bytes and finds none available from the internal input buffer, the end-of-file condition is returned.

If blocking mode is on, the buffers are empty and a [read] or [gets] command is evaluated on the channel: the operation will block; if the variable and the channel are in the same thread: this will block forever, else it will block until a script in the other thread writes or unsets the variable.

If blocking mode is off, a read command is evaluated on the channel and not enough bytes are in the buffers: the command will return all the available bytes, with the behaviours described in the TCL manual pages. [eof] will return false in this case.

Writing the channel
When data is flushed to the channel: it is written in the internal output buffer.

If the variable is unset: the command that caused the flush ([puts], [close], [flush]) will raise an error with POSIX code EPIPE.

At present blocking mode does not affect output on a varchan.


Node:Teechan, Next:, Previous:Varchan, Up:Channels

T stacked channel

A "tee channel" (teechan) is a transformation channel stacked above another channel, readable and/or writable; it duplicates all the data flowing through it to a couple of log variables. The data is sent unchanged to the underlying channel.

Example:

set target abcdefg
set channel [more varchan target RDWR]
more teechan $channel
fconfigure $channel -invar input -blocking no
set result [read $channel]
close $channel
# $result -> abcdefg
# $input  -> abcdefg

See the varchan description for a discussion of variable name resolution.

Command

more teechan channel Command
Stacks a new tee layer over the already existent channel; the return value is channel. The open mode of the transformation defaults to the same mode of the underlying channel.

Some options are configurable for a tee stacked channel.

-invar varname
Selects the name of the variable to which all the data read from the channel is appended to.
-outvar varname
Selects the name of the variable to which all the data written to the channel is appended to.

New log variables may be attached to a teechan any number of times: the new variable will replace the old one; if the empty string is used as argument to a log variable option: the current log variable is detached and no new one attached.

Operations

In this section a brief description of tee channel behaviour is given. The driver tries to behave in the most intuitive way, but there are cases in which knowing what a tee channel is doing (or not doing) is useful.

Reading data
When the [read] command is used on a stacked tee channel, data is read from the underlying channel and sent to the upper level. If an error occurs in the underlying channel: the same error is returned by the tee driver.

If an error occurs updating the variable: an error is raised by the command that acted on the channel; EINVAL is used as POSIX error code, probably causing the generic channel layer to use the invalid argument error description; if such an error happens: data has been read from the underlying channel and so it will be lost (unless the upper level ignores the error, but this is not standard behaviour).

Writing data
When the [puts] command is used on a stacked tee channel, data is written to the underlying channel. If an error occurs in the underlying channel: the same error is returned by the tee driver.

If an error occurs updating the variable: an error is raised by the command that acted on the channel; EINVAL is used as POSIX error code, probably causing the generic channel layer to use the invalid argument error description; if such an error happens: data has been written to the underlying channel with no errors.

Channel events
Channel events are almost ignored by the tee channel: requests sent to the driver through the [fileevent] command are handed to the underlying channel. If a request does not comply with the open mode of the tee channel, it is discarded: a readable event is not serviced by write-only stacked channels; a writable event is not serviced by read-only stacked channels.
Closing channel
When the channel is closed: it is detached from the logging variables, if present. The [close] command will close the whole stack of channels; if we want to remove only the tee channel, we have to use [more::unstack] (Channels for details).
Log variables
If log variables are used: they are traced with the purpose to detach them from the channel whenever they are unset. Even if variables are linked in an interpreter and then the channel is transferred to another interpreter, the unset trace should fire and detach the variable in a safe way.


Node:Pipechan, Previous:Teechan, Up:Channels

Pipe channels

A "pipe channel" (pipechan) is a pair of channels linked together. Writing into one makes data available for reading to the other one. Pipe channels can be shared among different threads.

more pipechan outvar1 outvar2 mode Command
Creates a couple of channels linked together. The return value is the empty string.

mode is one of: RDONLY, WRONLY, RDWR. outvar1 is the name of a variable that will hold the first channel identifier, outvar2 is the name of a variable that will hold the second channel identifier.

If RDONLY is selected: the first channel will be read-only and the second one write-only; if WRONLY is selected: the opposite. If RDWR is selected both the channels will be read-write.

The channels are linked together through internal buffers: this buffering is completely separated from the one TCL does on channels.

Reading the channel

When a read operation is executed: the requested number of bytes is extracted from the TCL input buffer, if any, and if not enough from the internal pipechan input buffer.

If the writer channel is unset: at the first read operation on the reader channel that requests bytes from the internal input buffer: the end-of-file condition is returned.

If blocking mode is on, the buffers are empty and a [read] or [gets] command is evaluated on the channel: the operation will block; if both the channels are in the same thread: this will block forever, else it will block until a script in the other thread writes or closes the other channel.

If blocking mode is off, a read command is evaluated on the channel and not enough bytes are in the buffers: the command will return all the available bytes, with the behaviours described in the TCL manual pages. [eof] will return false in this case.

Writing the channel

When data is flushed to the channel: it is written in the internal output buffer.

If the reader channel is closed: the command that caused the flush ([puts], [close], [flush]) will raise an error with POSIX code EPIPE.

At present blocking mode does not affect output on a pipechan.

Threads and channel events

When channel events are used (command [fileevent]) on a channel: the other channel will queue channel events in the event loop of the requesting thread; the requesting channel is correctly notified of events.


Node:Callback Functions, Next:, Previous:Channels, Up:Top

Callback function operations

This module defines data types and preprocessor macros used to handle callback functions. A callback is a pair: function pointer, opaque data pointer; the data pointer is used as argument for the function call.

A callback is meant to be registered in a state context and executed by the context's controlling module to trigger the execution of an operation in another module, possibly over another context. This is typical when a program implements the observer design pattern.

More_Callback Struct Typedef
Type of a callback entity. Fields description follows.
More_CallbackFunction * function
Pointer to the callback function.
ClientData data
An opaque value to be used as argument to the callback.

More_CallbackFunction Function Prototype
Type of the callback function.
void More_CallbackFunction (ClientData D);

More_CallbackInit (callback, function, data) Macro
Initialises a callback structure. Arguments description follows.
More_Callback callback
The callback structure (not a pointer to it).
More_CallbackFunction * function
Pointer to the callback function. If NULL the callback is reset.
ClientData data
Opaque value to use as argument to the callback function.

More_CallbackPresent (More_Callback callback) Macro
Evaluates to true if a callback is registered.

More_CallbackReset (More_Callback callback) Macro
Resets a callback structure.

More_CallbackInvoke (More_Callback callback) Macro
Invokes the callback.


Node:Memory Blocks, Next:, Previous:Callback Functions, Up:Top

Handling memory blocks


Node:Memory Blocks Data Types, Next:, Up:Memory Blocks

Data types

More_Block Struct Typedef
Type of input and output memory blocks used as source and destination for write and read operations on buffers. Fields description follows.
int len
The number of bytes in the block. This is an int, rather than a size_t, to make it easy to interface with TCL code.
More_BytePtr ptr
Pointer to the first byte in the block.

More_BytePtr Macro
Macro that represents a pointer to byte type.

MORE_BLOCK_NULL_VALUE Macro
The initialisation value for an empty More_Block. This symbol can be used to initialise the state invocation structure for a TCL command implemented with the TCLMORE infrastructure (Command Interface for details).


Node:Memory Blocks Allocation, Previous:Memory Blocks Data Types, Up:Memory Blocks

Allocation

More_BlockAlloc (More_Block block, int size) Macro
Allocates a new block of memory with ckalloc(). Fills the fields of block.

More_BlockRealloc (More_Block block, int newSize) Macro
Reallocates a block with ckrealloc().

More_BlockFree (More_Block block) Macro
Releases the block with ckfree().

More_BlockIsNull (More_Block block) Macro
Returns true if the block is empty.

More_BlockReset (More_Block block) Macro
Resets to zero the fields of block.

More_BlockFromByteArray (More_Block block, Tcl_Obj * objPtr) Macro
Fills a block with the pointer and length of a byte array.

More_ByteArrayFromBlock (Tcl_Obj * objPtr, More_Block block) Macro
Builds a new byte array object from a block.


Node:Variable Interface, Next:, Previous:Memory Blocks, Up:Top

Interface to TCL variables

A TCL variable is always associated to the interpreter to which it belongs. A variable exists in a context of the interpreter, for example: automatic variables exists only in the context of a procedure execution. In the use of functions in this module we must take care to ensure that the variable we are acting on exists in the current context of the interpreter.

Interpreters are associated to a single thread, so when a TCL command implemented with a C function is executed, we can be sure that an existent variable will exists for the whole duration of the call, provided that the command will not evaluate scripts or enter the event loop of the thread.

Functions in this module should not be used with variables to which error raising traces are attached: errors accessing the variable are ignored.

More_Variable Struct Typedef
The type of the structure used to represent a variable. It is declared as an array of a single element.

void More_VariableInit (variable, interp, name) Function
Initialises a variable structure.
More_Variable variable
Pointer to the variable instance.
Tcl_Interp * interp
The interpreter to which the variable belongs.
Tcl_Obj * name
Pointer to an object holding the variable name as a string.

void More_VariableCopy (More_Variable dest, More_Variable source) Function
Duplicates an instance into another one.

void More_VariableFinal (More_Variable variable) Function
Finalises an instance, releasing the associated resources.

int More_VariableExists (More_Variable variable) Function
Tests if a variable exists in the current context of the interpreter. Returns true if the variable exists.

int More_VariableTest (More_Variable variable) Function
Tries to retrieve the object in the variable. If this operation succeeds the variable exists and can be considered accessible until the next script is evaluated in the interpreter.

void More_VariableSet (More_Variable variable, Tcl_Obj * objPtr) Function
Stores a new value in the variable.

void More_VariableSetBlock (More_Variable variable, More_Block block) Function
Stores a byte array in the variable.

void More_VariableAppend (More_Variable variable, Tcl_Obj * objPtr) Function
Appends an object to the one in the variable.

void More_VariableAppendBlock (More_Variable variable, More_Block block) Function
Appends a byte array to the content of the variable.

void More_VariableClear (More_Variable variable) Function
Sets variable's content to the empty string.

Tcl_Obj * More_VariableGet (More_Variable variable) Function
Returns the object currently in the variable.

More_Block More_VariableGetBlock (More_Variable variable) Function
Returns the content of a variable seen as a byte array.

CONST char * More_VariableName (More_Variable variable) Function
Returns a pointer to the variable name.


Node:Command Interface, Next:, Previous:Variable Interface, Up:Top

Implementing TCL commands

TCLMORE provides an infrastructure to declare a set of TCL commands in an XML document and automatically generate the source code to interface the commands to the internal functions.

This infrastructure assumes that the "real work" is performed by a set of functions provided by the package (possibly the ones also exported through a stub table); so the callback function of the commands, the Tcl_ObjCmdProc, only has to implement command line arguments parsing and the interface to the algorithm function.


Node:Command Interface Introduction, Next:, Up:Command Interface

Introduction

The data types and functions described in this section are used to automate the parsing of command line arguments. A TCL command using this module has three kinds of arguments: command selection, mandatory values and options.

Example: the constructor of a TK widget, say the [entry] command, has: a single command selector ([entry]), a single mandatory argument (the widget pathname) and supports a set of options (like -foreground <color>). The syntax of a command like [puts] is not supported.


Node:Command Interface Using, Next:, Previous:Command Interface Introduction, Up:Command Interface

Using the infrastructure

The set of files required to use the infrastructure are stored in the xml subdirectory of the source tree. They are not installed along with TCLMORE. Most are TCL scripts, and we need TCL version 8.4 to use them. Files description follows.

xmlpp.tcl
Converts an XML document into a TCL script that can be executed if one provides a set of procedures.
tclcommand.tcl2data
A TCL script that executes the output of xmlpp.tcl to produce a set of TCL variable declaration representing the data in the original XML document.
tclcommand.data2code
A TCL script that reads the output of tclcommand.tcl2code to produce a C language source file, conaining the declarations of functions and data structures representing the data in the original document.
Makefile
A file for GNU make that controls everything. Inspection of this (short) file should enlighten on the use of the scripts.
tclcommand.dtd
The document type declaration of the XML document used to build the commands source code.

To use the infrastructure we can copy the files in a directory of our project, then, in the same directory, we create one or more XML documents with our command declarations. We do make all and we have the source code.

For example: lets say that we described a set of commands in declaration.xml; we do:

$ make all

and we find declaration.c in the same directory; inspection of this file is mandatory as it contains comments on how to use the generated code.


Node:Command Interface DTD, Next:, Previous:Command Interface Using, Up:Command Interface

Document Type Declaration

We will do this through examples. It is impossible to fully understand this section without reading the source code produced as output by the infrastructure scripts. In the source code of TCLMORE there are example declarations of commands: read this section, then read the examples in the xml directory (both the .xml and .c files).

Command hierarchy

First: a set of commands is a forest: an ensemble of trees. XML is good at representing trees. So here is an example of a document declaring a main command with two subcommands.

<?xml version="1.0"?>
<!DOCTYPE tclcommand SYSTEM "tclcommand.dtd">
<tclcommand table="CommandsTable">
  <maincommand name="one">
     <command name="red"></command>
     <command name="white"></command>
  </maincommand>
</tclcommand>

To call these commands:

one red
one white

The name attribute selects the name of the command selectors. The table attribute of the <tclcommand> markup is the name of an array of structures in the generated code: this array is used as argument to More_CreateCommands(), the function that adds commands to an interpreter.

Now two main commands and two commands.

<?xml version="1.0"?>
<!DOCTYPE tclcommand SYSTEM "tclcommand.dtd">
<tclcommand table="CommandsTable">
  <maincommand name="one"><!-- omission --></maincommand>
  <maincommand name="two"><!-- omission --></maincommand>
  <command name="three"></command>
  <command name="four"></command>
</tclcommand>

We can nest <maincommand> markups at will, to create complex hierarchies; going above the third level (three strings to select a command) is highly discouraged, though, better try to do everything in two levels: a root <maincommand> with nested <command> markups.

Both <command> and <maincommand> support the safe attribute, which can have values yes and no; this attribute selects whether the command or main command is available in a safe interpreter. The default is to make all the commands safe.

Arguments

So far our commands have no arguments nor options. Lets see how to declare a command entity with two arguments.

<struct name="State">
  <member name="integer" type="int" default="123"/>
  <member name="number" type="double" default="1.2"/>
</struct>
<command name="four" synopsis="integer number" struct="State">
  <argument extractor="GetIntFromArg" member="integer"/>
  <argument extractor="GetDoubleFromArg" member="number"/>
</command>

Lets examine <command> first: synopsis selects the string argument for Tcl_WrongNumArgs(); struct selects the name of a structure type, whose fields are used to hold the values extracted from command line arguments and options. The name of the structure must match the value of the name attribute in one of the <struct> entities.

The <struct> entity declares the structure fields with data type and default value. The entity in the example will be converted in the following chunk of code.

typedef struct State {
  int      integer;
  double   number;
} State;

static CONST State StateDefaults = { 123, 1.2 };

When a command is called: an instance of its invocation state structure is allocated and initialised with the selected defaults. The informations in <argument> are used to parse command line arguments.

The order in which <argument> markups are declared in the content of <command> is the same as the one of matching command line arguments. The value of the extractor attribute is the name of a function of type More_Extractor that we must implement to get a value from a Tcl_Obj. The function will have, as parameter, a pointer to the corresponding field in the invocation structure instance.

Options

Now lets see how to declare a command with options.

<struct name="State">
  <member name="integer" type="int" default="123"/>
  <member name="number" type="double" default="1.2"/>
  <member name="mode" type="int" default="0"/>
</struct>
<command name="three" synopsis="?options?" struct="State">
  <option name="-integer" hasarg="yes" extractor="GetIntFromArg"
    member="integer"/>
  <option name="-number" hasarg="yes" extractor="GetDoubleFromArg"
    member="number"/>
  <option name="-on" hasarg="no" extractor="GetModeFromArg"
    member="mode"/>
  <option name="-off" hasarg="no" extractor="GetModeFromArg"
    member="mode"/>
</command>

<option> entities are similar to <argument> entities; the name attribute selects the option selector; the hasarg attribute, which can be yes or no, tells if the option has argument or no.

As we can see: options with no arguments still have a structure field associated, and we can use many options to store different values into the same structure field.

Implementation functions

For each <command> we must write an implementation function. It is a function invoked after all the command line arguments and options have been parsed with no errors, and the extracted values have been stored in the invocation structure fields. The prototype of this function is already in the generated code.

Value extractors

Each command line argument and option has an extractor associated to it. Extractors are functions that we must implement, they get as parameters: a pointer to the interpreter, a pointer to the source object, a pointer to the field in the invocation state structure.

If an extractor returns with code TCL_ERROR the command returns with an error condition; it is responsibility of the extractor to put an error message in the interpreter.


Node:Command Interface Typedefs, Next:, Previous:Command Interface DTD, Up:Command Interface

Data types

Implementation function

int More_CmdFunc (More_CommandFrame frame) Function
The type of the command implementation function.

More_CommandFrame Struct Pointer Typedef
Parameter for the implementation function. Fields description follows.
ClientData stateStructPtr
Pointer to the command invocation structure. The More_CmdFunc has to cast it to the correct type.
ClientData D
The client data of the root command.
Tcl_Interp * interp
Pointer to the interpreter in which the command invocation took place.
int objc
The number of arguments on the command line.
Tcl_Obj *CONST * objv
The arguments on the command line.

Value extractor

int More_Extractor (More_ExtractorFrame frame) Function
The prototype of the extractor functions.

More_ExtractorFrame Struct Pointer Typedef
The parameter for the extractor. Fields description follows.
ClientData fieldPtr
Pointer to the field in which the value must be stored. The extractor has to cast it to the correct type.
Tcl_Obj * srcObj
Pointer to the object from which the value must be extracted.
More_CommandFrame commandFrame
The value that will be used as argument for the implementation function. From the referenced structure the extractor can get a pointer to the interpreter in which register errors.


Node:Command Interface Functions, Previous:Command Interface Typedefs, Up:Command Interface

Interface functions

More_Error More_CreateCommands (interp, namespName, table) Function
Creates a namespace and a set of commands in it; exports the commands matching the pattern [a-z]*. Returns NULL or an error descriptor.
Tcl_Interp * interp
The target interpreter.
CONST char * namespName
Namespace name. Can be NULL if the target namespace is the root one.
More_CommandTreeNode * table
Pointer to the declaration table: an array of structures. This value is a pointer to the table of root nodes whose name was selected with the table attribute of the <tclcommand> markup.

Argument extractors

The following extractors return TCL_OK or TCL_ERROR. In case of error they leave a message in the interpreter and set the error code to LOGIC.

int More_GetObjFromArg (More_ExtractorFrame frame) Function
Extracts a pointer to an object from a command argument and stores it into the selected struct field.

int More_GetStringFromArg (More_ExtractorFrame frame) Function
Extracts a string from an object command argument and stores it into the selected struct field. The string field is allocated as a More_String instance. The string can be empty, that is: the source object can be the empty string; in this case the More_String fields is set to NULL pointer and zero length.

int More_GetAStringFromArg (More_ExtractorFrame frame) Function
Extracts a non-empty string from an object command argument and stores it into the selected struct field. The string field is allocated as a More_String instance. An error is raised if the string is empty.

int More_GetBlockFromArg (More_ExtractorFrame frame) Function
Extracts a byte array from an object command argument and stores it into the selected struct field. The field is allocated as a More_Block instance.

int More_GetABlockFromArg (More_ExtractorFrame frame) Function
Extracts a non-empty byte array from an object command argument and stores it into the selected struct field. The field is allocated as a More_Block instance.

int More_GetIntFromArg (More_ExtractorFrame frame) Function
Extracts an integer from a command argument and stores it into the selected struct field.

int More_GetWideIntFromArg (More_ExtractorFrame frame) Function
Extracts an integer from a command argument and stores it into the selected struct field.

int More_GetUnsignedFromArg (More_ExtractorFrame frame) Function
Extracts an unsigned integer from a command argument and stores it into the selected struct field.

int More_GetFloatFromArg (More_ExtractorFrame frame) Function
Extracts a float number from a command argument and stores it into the selected struct field.

int More_GetDoubleFromArg (More_ExtractorFrame frame) Function
Extracts a double number from a command argument and stores it into the selected struct field.

int More_GetSizeTFromArg (More_ExtractorFrame frame) Function
Extracts a float number from a command argument and stores it into the selected struct field.


Node:Error Reporting, Next:, Previous:Command Interface, Up:Top

Returning error conditions


Node:Error Descriptors, Next:, Up:Error Reporting

Propagating error informations

The function and data types described in this section are meant to be used to propagate error informations from a nested function call, up to the point where the information can be presented as result of the invocation of a TCL command.

The scenario for which this system was designed is the following. We have a C language extension that provides an interface to an external library; two layers are available: a set of commands created in an interpreter; a set of C API functions accessible from other extension libraries. The TCL layer is built upon the API functions.

When a function in the external library returns an error, the API builds an error descriptor with local informations and returns. Up level functions get the descriptor, process it and return, too. At some point a TCL command callback function is reached and the error is swallowed by the state of an interpreter.

A description of informations in an error descriptor follows.

More_Error Struct Pointer Typedef
Error descriptor: holds an error code and an error description. Members description follow.
CONST char * errorCode
Pointer to a statically allocated, NULL-terminated string representing the error code. Typical values: LOGIC, to indicate an error in the way a command or function has been invoked (example: a precondition has not been satisfied or an argument is invalid); RUNTIME, to indicate things that can happen (example: a file does not exist or there's not enough memory).
Tcl_Obj * errorInfo
Pointer to an object holding the error description as a string. We use an object to allow modules to have localised descriptions. A function is provided to prepend text.
ClientData data.data
int data.integer
Error specific informations; this is a union. The format of the information must be known to the code receiving the error descriptor: propagating this kind of information through many layers of nested function invocations is a bad idea. Better share this data between only two functions: the one that builds it and the one that uses it, the latter invoking directly the former.

A problem with this field is how to release the resources associated to it; example: this field could be a pointer to a dynamically allocated structure. The TCLMORE code will take no action to do this: the responsibility of freeing the resources is completely delegated to the user's code.

A useful example of usage of the data.integer field is to store the return value of Tcl_GetErrno().

More_Error More_ErrorNew (void) Function
Allocates and returns a new error descriptor. The block is allocated with a call to ckalloc(). All the members are initialised to zero.

void More_ErrorForget (More_Error e) Function
Frees the resources associated to the descriptor and releases the descriptor itself with a call to ckfree().

int More_ErrorResult (Tcl_Interp *interp, More_Error e) Function
Makes error informations become part of the internal state of the interpreter, then destroys the descriptor with a call to More_ErrorForget(). If interp is NULL: the error descriptor is destroyed and its informations lost. Returns TCL_ERROR.

This function may be the one used to return from the callback of a TCL command. The code may look like the following:

return More_ErrorResult(interp, error);

in this case all the resources used by the callback function must be freed before the invocation.

void More_ErrorLogic (More_Error e, Tcl_Obj *info) Function
If the descriptor is clean: sets the error code to LOGIC and sets the error description to info. If the descriptor has already been initialised with some data: changes the error code to LOGIC and prepends info to the information text, separating the two with a colon and a space.

void More_ErrorRuntime (More_Error e, Tcl_Obj *info) Function
If the descriptor is clean: sets the error code to RUNTIME and sets the error description to info. If the descriptor has already been initialised with some data: changes the error code to RUNTIME and prepends info to the information text, separating the two with a colon and a space.

void More_ErrorLogicStr (More_Error e, CONST char * info) Function
Wrapper for More_ErrorLogic() that accepts the information as a string and builds an object for it.

void More_ErrorRuntimeStr (More_Error e, CONST char * info) Function
Wrapper for More_ErrorRuntime() that accepts the information as a string and builds an object for it.

void More_ErrorPrepend (More_Error e, Tcl_Obj * info) Function
Prepends info to the information text, separating the two with a colon and a space.

void More_ErrorPrependStr (More_Error e, CONST char * info) Function
Wrapper for More_ErrorPrepend() that accepts the text has a string and builds an object for it. The string is duplicated.

int More_ErrorIsLogic (More_Error e) Function
Returns true if the error code is set to logic.

int More_ErrorIsRuntime (More_Error e) Function
Returns true if the error code is set to run time.

More_ErrorSetData (More_Error e, ClientData data) Macro
Fills the error specific data field.

More_ErrorGetData (More_Error e) Macro
Returns the error specific data field.

More_ErrorSetInt (More_Error e, int integer) Macro
Fills the error specific integer field.

More_ErrorGetData (More_Error e) Macro
Returns the error specific integer field.

int More_ErrorCodeAndForget (More_Error error) Function
Returns the integer error specific integer, destroys the error descriptor. This function is useful in channel drivers using modules that return error descriptors: the modules can use the integer data to register a POSIX code describing the error.

More_Error More_ErrorNoMemory (void) Function
Allocates a new descriptor and initialises it with a run time error code and the string not enough memory. Returns the descriptor. Localisation of this string is currently not possible (but you have the code).

More_Error More_ErrorErrno (void) Function
Builds a new error descriptor with the informations of errno. The error info is the string associated to the current value of errno; the integer value of the error specific data is the return value of Tcl_GetErrno().


Node:Error Codes, Next:, Previous:Error Descriptors, Up:Error Reporting

Setting error codes

int More_LogicError (Tcl_Interp * interp) Function
Signals a logic error and returns an error code. interp is the interpreter in which the error is reported, if it is NULL nothing happens. Sets LOGIC as error code and returns TCL_ERROR.

int More_RuntimeError (Tcl_Interp * interp) Function
Signals a run time error and returns an error code. interp is the interpreter in which the error is reported, if it is NULL nothing happens. Sets RUNTIME as error code and returns TCL_ERROR.

int More_LogicErrorStr (Tcl_Interp * interp, CONST char * errorInfo) Function
Like More_LogicError(), but in addition stores errorInfo in interpreter's result.

int More_RuntimeErrorStr (Tcl_Interp * interp, CONST char * errorInfo) Function
Like More_RuntimeError(), but in addition stores errorInfo in interpreter's result.

Example: signaling a logic error.

if (! precondition())
  {
     return More_LogicError(interp);
  }

Example: signaling a run time error.

if (! commit_success())
  {
     abort_transaction();
     return More_RuntimeError(interp);
  }


Node:Predefined Errors, Previous:Error Codes, Up:Error Reporting

Signaling common errors

int More_WrongNumArgs (interp, objcount, objv, message) Function
Sets up the wrong # args error message; it is a wrapper for Tcl_WrongNumArgs(). Arguments description follow.
Tcl_Interp * interp
Pointer to the interpreter structure in which report the error.
int objcount
Number of leading arguments from objv to include in error messages.
Tcl_Obj *CONST objv[]
Arguments to a command that had the wrong number of arguments.
CONST char * message
Additional error informations to print, may be NULL.

Returns TCL_ERROR, leaves an error message as result in the interpreter and sets a logic error code (Error Codes for details on the logic error).

Example:

if (objc < 3)
  {
    return More_WrongNumArgs(interp, 2, objv, "data ?options?");
  }

int More_OptionRequiresArg (interp, option) Function
Builds the option requires argument error message. This function is meant to be used when parsing command line options of a TCL command. Arguments description follow.
Tcl_Interp * interp
Pointer to the interpreter in which report the error.
CONST char * option
Pointer to a string representing the option name that was used without argument.

Returns TCL_ERROR, leaves an error message as result in the interpreter and sets a logic error code (Error Codes for details on the logic error).


Node:Identifiers Table, Next:, Previous:Error Reporting, Up:Top

Handling tables of identifiers

More_IdTable Struct Typedef
Table of identifiers: basically a wrapper for the TCL hash table structure. It's typically embedded in the interpreter-specific package-associated data structure. Members description follow.
Tcl_HashTable table
Maps from identifier names to identifier references.
unsigned counter
Keeps the count of released identifiers; it's used to build the next identifier's unique name.
CONST char * tmplPtr
Pointer to a statically allocated, NULL-terminated, string representing the template for identifier names. Something like: channel%u; a single %u printf() code must be included in the template. This must be unique to the extension and the value is selected at initialisation time.
unsigned tmplLen
The number of characters in the string referenced by tmplPtr; it's used to compute the size of the buffer to be allocated for the identifier's string. This value is automatically computed when the structure is initialised.
More_IdDestructor destructor
Pointer to the function used to close the descriptor and release all the resources associated to it. It's selected at initialisation time.

More_IdDestructor Function Pointer
Pointer to the function used to destroy the values associated to identifiers in the table. The declaration is:
typedef void (*More_IdDestructor) _ANSI_ARGS_((ClientData D));

Must be a function provided by the extension, and it cannot fail. The single argument is the value associated to the identifier.

It's used only when the table is destroyed, for example: if the structure is embedded in the assoc data of an interpreter, when the interpreter is finalised the registered Tcl_InterpDeleteProc must call More_DeleteIdTable() which, in turn, will call the destructor for each registered identifier.

void More_InitIdTable (table, template, destructor) Function
Initialises the identifiers table. Arguments description follow.
More_IdTable * table
Pointer to an already allocated table struct.
CONST char * template
Pointer to a string holding the identifier template, something like file%u.
More_IdDestructor destructor
Pointer to a function that could be used to destroy the data associated to an identifier. Can be NULL to signal that the data must be left untouched.

The data structure is initialised. The hash table is initialised.

void More_DeleteIdTable (More_IdTable * table) Function
Deletes all the objects in the table. Extracts all the elements from the table and destroys each one with a call to the registered destructor (if not NULL).

Tcl_Obj * More_AttachId (More_IdTable *table, ClientData D) Function
Inserts a new value in the table. Builds and returns a new identifier's string object, the data is associated to it in the hash table.

void More_DetachId (More_IdTable * table, CONST char *id) Function
Extracts an identifier from the table. Extracts the data from the table; the data is just extracted, not destroyed. If the identifier is not in the table, nothing happens. After the invocation to this function the identifier is no longer valid.

ClientData More_GetDataFromId (More_IdTable *table, CONST char *id) Function
Extracts the data associated to a key in the identifiers table. id is the pointer to a string identifier. Returns the required data or NULL if the identifier is not present in the table.

An unrealistic example:

More_IdTable    table;
Tcl_Obj *       idObj;
SomeData *      data;

data = ...;

More_InitIdTable(&table, "nugget%u", Tcl_Free);
idObj = More_AttachId(&table, data);
Tcl_IncrRefCount(idObj);

/* ... do something ... */
data = More_GetDataFromId(&table, Tcl_GetString(idObj));

More_DetachId(&table, Tcl_GetString(idObj));
Tcl_DecrRefCount(idObj);
More_DeleteIdTable(&table);


Node:Object Extractors, Next:, Previous:Identifiers Table, Up:Top

Extracting values from objects


Node:Object Extractors Unsigned, Next:, Up:Object Extractors

Unsigned Values

Tcl_Obj * More_NewUnsignedObj (unsigned value) Function
Returns the pointer to a new object holding an unsigned value.

int More_GetUnsignedFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, unsigned * valVar) Function
Extracts an unsigned integer from an object. Behaves like TCL builtin extractors.

int More_GetUnsignedInRangeFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, unsigned min, unsigned max, int * valVar) Function
Extracts a integer in an inclusive range from an object. Behaves like TCL builtin extractors.


Node:Object Extractors Int, Next:, Previous:Object Extractors Unsigned, Up:Object Extractors

Integer Values

int More_GetIntRangeInFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, int min, int max, int * valVar) Function
Extracts a integer in an inclusive range from an object. Behaves like TCL builtin extractors.


Node:Object Extractors WideInt, Next:, Previous:Object Extractors Int, Up:Object Extractors

Wide Integer Values

int More_GetWideIntInRangeFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, Tcl_WideInt min, Tcl_WideInt max, Tcl_WideInt * valVar) Function
Extracts a wide integer in an inclusive range from an object. Behaves like TCL builtin extractors.


Node:Object Extractors Size, Next:, Previous:Object Extractors WideInt, Up:Object Extractors

Size Values

Tcl_Obj * More_NewSizeTObj (size_t value) Function
Returns the pointer to a new object holding an size_t value.

int More_GetSizeTFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, size_t * valVar) Function
Extracts a size_t integer from an object. Behaves like TCL builtin extractors.

int More_GetSizeTInRangeFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, size_t min, size_t max, size_t * valVar) Function
Extracts a size_t in an inclusive range from an object. Behaves like TCL builtin extractors.


Node:Object Extractors Float, Previous:Object Extractors Size, Up:Object Extractors

Float Values

Tcl_Obj * More_NewFloatObj (float value) Function
Returns the pointer to a new object holding an float value.

int More_GetFloatFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, float * valVar) Function
Extracts a float number from an object. Behaves like TCL builtin extractors.

int More_GetFloatInRangeFromObj (Tcl_Interp * interp, Tcl_Obj * objPtr, float min, float max, float * valVar) Function
Extracts a float in an inclusive range from an object. Behaves like TCL builtin extractors.


Node:Buffers, Next:, Previous:Object Extractors, Up:Top

Queuing data in a buffer

A set of functions is provided to manage buffer objects. Buffers are used in the implementation of the [varchan], [pipechan] and [teechan] commands and the interface is exposed in the stub table. All the function names in this module are prefixed with More_Buffer.

Buffers have features to share instances among different tasks in a process. Buffers are shared among two, and only two, entities: a writer and a reader; the two are associated to two contexts in a process. The two contexts may belong to the same or to different threads. The usage pattern of a shared buffer is:

  1. allocate a new shared buffer; this operation cannot fail;
  2. attach the buffer to the two contexts giving them the reader and writer roles; if an error occurs in this step the buffer is released;
  3. the writer writes, the reader reads; these operations are asynchronous: it is the whole point of using a buffer;
  4. one between the writer and the reader detaches itself from the buffer: the other entity senses this and detaches itself, too, causing the buffer to be released.

Deallocation of the buffer is similar to the reference counting pattern: when the two entities detach themselves from the buffer, the buffer's module automatically releases the resources.

The shared buffer has no knowledge about the reader and writer: it only knows if they have detached themselves.

More_Buffer Opaque Pointer
A token used to reference a shared buffer. The data type includes: a mutex to ensure exclusive access to each instance; a set of flags to keep track of entities attached as reader and writer to the buffer.

Buffers have a base struct that keeps a reference to a linked list of memory blocks; the blocks have a meta-data area at the beginning and a data area at the end. Interface functions allow to select the data area size in bytes.

Memory for the blocks is allocated with ckalloc() and released with ckfree(), it is responsibility of the module's functions to do that.

When a block is full a new one is appended to the chain; when data in the first block in the chain is consumed, the block is released and the second one takes its place.

Automatic notification of events

After an operation is performed on a shared buffer, a notification function, private in this module, is invoked: its purpose is to test the conditions for notification of the reader and writer about buffer events. A couple of callback functions can be registered in the shared buffer to be invoked to do the notification.

Of course if an entity has detached itself or no callback is registered: the notification is not done. The callback is invoked synchronously: it is guaranteed that when the callback is invoked the entity has not detached itself yet.

It is not safe to access the buffer from the callback functions. A callback function should only register the event somewhere or queue an event in the loop.

If the buffer is shared among two threads: the reader callback is invoked in writer's thread, the writer callback is invoked in reader's thread. In this case it is responsibility of the callback to queue an event in the loop of the other thread.

The reader callback is invoked if one of the following conditions are true:

For the writer, the callback is invoked if the buffer is writable: a buffer is always writable, so the notification is sent each time a module's function is invoked.

Note that notifications are sent to an entity even if the other entity has detached itself.

The callbacks are responsible of keeping track of queued events in their state. This is required for the following reasons:

A good way to manage creation and deletion of events is to use timer handlers: Tcl_CreateTimerHandler(), Tcl_DeleteTimerHandler().

Interface description

More_Buffer More_BufferAlloc (void) Function
Allocates a new buffer. Returns buffer's token.

void More_BufferFree (More_Buffer buffer) Function
Releases all the resources associated to the buffer. It is safe to call this function only before having assigned the buffer to the writer and reader entities.

void More_BufferReaderCallback (More_Buffer buffer, More_Callback callback) Function
Registers the callback used to notify the reader that the buffer is readable. If the function pointer is NULL the callback is reset. The notification function is invoked.

void More_BufferWriterCallback (More_Buffer buffer, More_Callback callback) Function
Registers the callback used to notify the writer that the buffer is writable. If the function pointer is NULL the callback is reset. The notification function is invoked. If the writer is attached: the function is invoked immediately because a buffer is always writable.

void More_BufferDetachReader (More_Buffer buffer) Function
Detaches the reader from the buffer. If the writer has already detached itself: the buffer is released. The notification function is invoked.

void More_BufferDetachWriter (More_SharedBuffer base) Function
Detaches the writer from the buffer. If the reader has already detached itself: the buffer is released. The notification function is invoked.

int More_BufferRead (More_Buffer buffer, More_Block block) Function
Extracts data from the buffer and places it in the block; the length field of the block is the number of requested bytes, the pointer field must reference a block of memory wide enough. Returns the number of bytes read which can be zero if no data is available. The notification function is invoked.

More_Block More_BufferReadAllBlock (More_Buffer buffer) Function
Reads all the data and stores it in a block, then returns the block. The memory in the block is allocated with ckalloc(), so it must be released by the caller with ckfree().

Tcl_Obj * More_BufferReadAllObj (More_Buffer buffer) Function
Reads all the data and stores it in a new byte array object.

void More_BufferWrite (More_Buffer buffer, More_Block block) Function
Write data from the block to the buffer. The notification function is invoked.

int More_BufferEmpty (More_Buffer buffer) Function
Returns true if the buffer is empty.

void More_BufferSetSize (More_Buffer buffer, size_t bufferSize) Function
Selects a new value for the next allocated internal block.

size_t More_SharedBufferGetSize (More_SharedBuffer buffer) Function
Acquires the current internal block size.

int More_BufferEof (More_Buffer buffer) Function
Tests the "end of file" condition on a shared buffer: the condition is true if the writer has detached itself and the buffer is empty. If this function returns true: the reader can detach itself because no more data will be available from the buffer; this will cause the buffer to be finalised.

int More_BufferAlive (More_Buffer buffer) Function
Tests the effect of writing data to the buffer: if the reader has detached itself the operation is useless. If the reader is still present the function returns true. If this function returns false the writer can detach itself from the buffer because writing data is useless; this will cause the buffer to be finalised.

Examples

Meaningless example of writing and reading data.

More_Buffer    buffer;
More_Block     block;
int            readNum;


buffer = More_BufferAlloc();

block.len = ...;
block.ptr = ckalloc(block.len);
memcpy(block.ptr, ..., block.len);

if (More_BufferAlive(buffer))
  {
    More_BufferWrite(buffer, block);
  }

if (! More_BufferEof(buffer))
  {
    readNum = More_BufferRead(buffer, block);
  }

ckfree((char *) block.ptr);
More_BufferDetachReader(buffer);
More_BufferDetachWriter(buffer);


Node:Channel Interface, Next:, Previous:Buffers, Up:Top

Channels and transformations


Node:Channel Creators, Next:, Up:Channel Interface

Channel Creators

ClientData More_CreateBufferVariable (variable, input, output) Function
Links an existent variable to a buffer. The variable must be: existent, accessible, free of traces raising errors. This function is used to implement the [varchan] command (Varchan:: for details). Arguments description follows.
More_Variable variable
Pointer to a variable token.
More_Buffer input
Pointer to the shared input buffer: when the variable is read, the value returned is the data currently in the buffer. Can be NULL.
More_Buffer output
Pointer to the shared output buffer: when the variable is set, the data goes to this buffer. Can be NULL.

Returns the client data of the trace: it can be used together with the variable to remove the trace.

If data is present in the variable and the output buffer is not NULL: the data is immediately sent to the output buffer, if any.

If both the input and output buffers are NULL, nothing happens and the return value is NULL.

The variable is traced and the operations are:

  • when the variable is written: all the data is removed from the variable and appended to the output buffer;
  • when the variable is read: all the data in the input buffer is extracted and appended to the variable's content.

No callback is registered in the shared buffers: when the owner of the variable wants data it gets it. If one needs notifications, he can use a pipe channel.

void More_DeleteBufferVariable (More_Variable variable, ClientData D) Function
Deletes the link between a variable and the associated buffers. The variable itself is not touched. D is the client data of the variable trace. Removes the trace on the variable, causing the buffers to be detached; this can cause the buffers deletion.

Tcl_Channel More_OpenBufferChannel (More_Buffer input, More_Buffer output) Function
Opens a new channel interface for a couple of buffers.
 --------  puts   ----------
| TCL    |------>| channel  |----->output buffer
| interp |<------| instance |<-----input buffer
 --------  read   ----------

Arguments description follows.

inputBuffer
The token of the input buffer. Can be NULL if the channel is write-only.
outputBuffer
The token of the output buffer; can be NULL if the channel is read-only.
channelVar
Pointer to the variable that will hold the channel token.

Creates the channel linked to the buffers. Returns the channel token.

Tcl_Channel More_OpenVarChannel (More_Variable variable, int modeMask) Function
Opens a new channel interface for the contents of a variable. Makes use of More_CreateBufferVariable() and More_OpenBufferChannel. variable is the variable token; modeMask is an OR-ed combination of TCL_READABLE and TCL_WRITABLE. Returns the channel token.

If TCL_READABLE is used: the variable can be used to read data from the channel; if TCL_WRITABLE is used: the variable can be used to write data to the channel.

Tcl_Channel More_StackTeeChannel (Tcl_Interp *interp, Tcl_Channel subChannel, int modeMask) Function
Stacks a new channel on top of a selected one. Arguments description follows.
interp
The interpreter used to report errors.
subChannel
The token of the underlying channel.
modeMask
An OR-ed combination of TCL_READABLE and TCL_WRITABLE.

Returns the new channel token.

Tcl_Channel More_OpenPipeChannel (int modeMask, Tcl_Channel *channelVar) Function
Opens a new couple of channels, linked together. If modeMask is TCL_READABLE: the first channel will be read-only and the second one write-only; if mode is TCL_WRITABLE: the opposite. If mode is TCL_READABLE OR-ed with TCL_WRITABLE both the channels will be read-write. modeMask must not be zero.

Returns the first channel token; stores the second channel token in the variable referenced by channelVar.


Node:Stream Transform, Next:, Previous:Channel Creators, Up:Channel Interface

Stream transformation

The stream transformation provides a stackable channel that uses user supplied functions to implement data processing. Each transformation can have two streams: one for input and one for output.

We consider a stream transformation with two fundamental properties:


Node:Stream Overview, Next:, Up:Stream Transform

Overview

We imagine to have an external library that provides some sort of stream processing with interface functions like the following:

init()
initialises a new context structure used to process data;
final()
finalises a context freeing all the resources;
register_input_block()
registers in the context structure a block of memory holding input data, the input buffer;
register_output_block()
registers in the context structure a block of memory to which output data should be written, the output buffer;
process()
process data from the input block, through the context, to the output block;
flush()
flushes as much data as possible from the internal context to the output buffer;
finish()
finalises the transformation flushing all the data from the internal context to the output buffer; after this function has been called the only allowed operation is final().

This interface is, more or less, the one offered by libraries such as zlib and BZip2.

We want to make this transformation available at the TCL level, so we have to define a C API that wraps the external library and that we can use to implement the transformation through the TCL channel interface.

First possible solution (not implemented)

Allocate the two buffers at the beginning then: accumulate data in the input buffer, process it and accumulate data in the output buffer. When reading or writing data we do not want to manage memory allocation, so the stream module has to take care of reallocating buffers automatically.

we supply       the stream           the stream        we supply
an input        has an input         has an            an output
block           buffer               output buffer     block
 -               -                        -              -
 |               |     --------------     |              |
 | ------------> |    |   stream     |    | -----------> |
 |  CopyWrite()  | ++>|   context    |++> |  CopyRead()  |
 -               |     --------------     |              -
                 -                        -

This mode of operation allows us to delay processing until there is "enough" input data to make it efficient. For example: we may select the size of the input buffer and process data when it is full.

When reading and writing it is possible that data goes only to the input buffer or comes only from the output one, without modification of the internal context.

We can pull data from the stream until the input buffer is empty, the internal context has flushed as much bytes as possible and the output buffer is empty; then data is no more extracted, but some of it may still be inside the internal context.

Using a copy operation to read and write data from and to the input and output buffers is easy, because it requires a single function call; but sometimes it may require to allocate a block of memory to hold incoming/outgoing data. For example:

We may choose to request to the stream module, to make room in the input buffer so we can write data to it, and to provide us a reference to a portion of the output buffer that we can read data from. This involves a transaction protocol, possibly with locking of the stream; two function calls to the stream module are requested. Example:

Second possible solution (implemented)

Allocate only the output buffer, process input data immediately consuming all of it. This is a simpler and less efficient solution than the first one for transformations that acts better on many bytes at once, like compression.

we supply                 the stream
an input                  has an
block                     output buffer
 -                           -
 |             ---------     |              we copy data
 | ---------> | stream  |    | -----------> directly from
 |  Write()   | context |++> |    Read()    the buffer
 -             ---------     -

We can push data until the system has memory to allocate to the process for the output buffer. At each write operation the internal context is modified, this may slow down the execution if we write many little chunks of data.

We can pull data from the stream until the output buffer is empty. Some data may still be in the internal context.


Node:Fantasy Stream Example, Next:, Previous:Stream Overview, Up:Stream Transform

Examples

Remark: in this section we have to remember that TCL never fails to allocate memory.

The following is a fantasy example, with no error detection, of the way the stream interface should be used. All the imaginary stream functions and data type names are prefixed with Dream_.

Dream_Stream    token;
More_Block      buffer, input, output;
int             numberOfBytesUsed;


Dream_StreamInit(&token);

More_BlockAlloc(buffer, INPUT_BLOCK_SIZE);

/* Read input until no more data is supplied. */
for (input = buffer, fill_a_block_with_data(&input);
     input.len;
     input = buffer, fill_a_block_with_data(&input))
  {
    Dream_StreamWrite(token, &input);
    output = Dream_StreamOutput(token);
    /* If output was produced: use it. */
    if (output.len)
      {
        numberOfBytesUsed = use_the_processed_data(output);
        Dream_StreamRead(token, numberOfBytesUsed);
      }
  }

/* Flush data from the internal context and finish. */
Dream_StreamFinish(token);
output = Dream_StreamOutput(token);
use_all_the_processed_data(output);

/* Free resources. */
More_BlockFree(buffer);
Dream_StreamFinal(token);

Now we see the same code modified to take care of the following case: if some data is not absorbed by Dream_StreamWrite(): it is still in buffer, and also referenced by input, so we can do something with it.

For some streams this may be considered an error, for example: a compression or encryption stream is supposed to absorb data with no problems. The following example assumes this and also does error detection.

Dream_Stream    token;
More_Block      buffer, input, output;
int             numberOfBytesUsed;
More_Error      error = NULL;


error = Dream_StreamInit(&token);
if (error) { ... }

More_BlockAlloc(buffer, INPUT_BLOCK_SIZE);

for (input = buffer, fill_a_block_with_data(&input);
     input.len;
     input = buffer, fill_a_block_with_data(&input))
  {
    error = Dream_StreamWrite(token, &input);
    if (error || input.len)
      {
        goto Error;
      }

    output = Dream_StreamOutput(token);
    if (output.len)
      {
        numberOfBytesUsed = use_the_processed_data(output);
        Dream_StreamRead(token, numberOfBytesUsed);
      }
  }

error = Dream_StreamFinish(token);
if (error) { ... }
output = Dream_StreamOutput(token);
use_the_processed_data(output);

Error:
More_BlockFree(buffer);
Dream_StreamFinal(token);
if (error)     { ... }
if (input.len) { ... }

For other streams unabsorbed data may mean that the end of stream was found; for example: a compression may mark the end of compressed data, so that the corresponding decompression stream is able to detect it and finalise the transformation. This is not an exception, it is normal operation. The following example assumes this and also does error detection.

Dream_Stream    token;
More_Block      buffer, input, output;
int             numberOfBytesUsed;
More_Error      error = NULL;


error = Dream_StreamInit(&token);
if (error) { ... }

More_BlockAlloc(buffer, INPUT_BLOCK_SIZE);

for (input = buffer, fill_a_block_with_data(&input);
     input.len;
     input = buffer, fill_a_block_with_data(&input))
  {
    error = Dream_StreamWrite(token, &input);
    if (error)
      {
        goto Error;
      }
    if (input.len)
      {
        break;
      }

    output = Dream_StreamOutput(token);
    if (output.len)
      {
        numberOfBytesUsed = use_the_processed_data(output);
        Dream_StreamRead(token, numberOfBytesUsed);
      }
  }

error = Dream_StreamFinish(token);
if (error) { ... }
output = Dream_StreamOutput(token);
use_the_processed_data(output);

if (input.len)
  {
    /* Can do something with unabsorbed data. */
  }

Error:
More_BlockFree(buffer);
Dream_StreamFinal(token);
if (error) { ... }


Node:Stream Drivers, Next:, Previous:Fantasy Stream Example, Up:Stream Transform

Drivers

TCLMORE defines driver data types to allow extensions to implement stream modules; these drivers can be used by the TCLMORE transformation exported at the TCL level.


Node:Stream Driver Type 1, Up:Stream Drivers
Driver type 1

The imaginary stream module shown in the examples section could be used as type 1 driver with the following declaration:

static More_ChannelDriverSetOptionProc DreamSetOption;
static More_ChannelDriverGetOptionProc DreamGetOption;

static CONST More_ChannelDriverOption optionTable[] = {
  { "-option", DreamSetOption, DreamGetOption },
  { NULL, NULL, NULL }
};

static CONST More_StreamDriver Dream_StreamDriver = {
  "1",
  "dream",
  Dream_StreamFinal,
  Dream_StreamOutput,
  Dream_StreamRead,
  Dream_StreamWrite,
  Dream_StreamFlush,
  Dream_StreamFinish,
  optionTable
};

in this example we have imagined that the driver has a configuration option which is accessible through the [fconfigure] option -option.

The following is the description of the driver members. All the functionalities described must be provided by the stream module, they are not offered by TCLMORE. Remarks:

More_Stream Opaque Pointer
The token used as reference to a stream structure. It is an alias to ClientData.

More_StreamDriver Struct Typedef
The type of driver for the transformation. Fields description follows.
CONST char * version
Version field; the version number is the one of the TCLMORE functions to be used to handle the structure. It must be 1 if the driver described in this section is to be used.
CONST char * type
A string used to identify the driver. Can be anything, even NULL.
More_StreamFinal * final
More_StreamOutput * output
More_StreamRead * read
More_StreamWrite * write
More_StreamFlush * flush
More_StreamFinish * finish
Pointers to implementation functions. Notice that the initialisation function is not present: the constructor may have any prototype.
CONST More_ChannelDriverOption * optionTable
Pointer to the table of transformation specific options, Channel Driver for details. These options must act only on the input and output streams, in no way they can directly change the behaviour of the transformation.

An example of option that may be offered is the configuration of the initial size of the stream output buffers.

The pointer can be NULL if the streams have no options.

void More_StreamFinal (More_Stream token) Function
Type of function used to finalise a stream. Must free all the resources still in the stream descriptor including the stream descriptor structure referenced by token. This function is used to abort a stream or to finalise a stream after the finish function has been invoked and data consumed. This function can not fail.

More_Block More_StreamOutput (More_Stream token) Function
Type of function used to access the output buffer of the stream. Must return a block referencing the internal output buffer; data will be read from the block directly.

This function should not do any processing on data, only provide a reference to the output buffer.

void More_StreamRead (More_Stream token, int number) Function
Type of function used to register that a number of bytes has been read from the output buffer acquired with the More_StreamOutput function.

More_Error More_StreamWrite (More_Stream token, More_Block * blockPtr) Function
Type of function used to make the stream process input data. blockPtr must reference a block of input data.

When the function returns with no error, it must have modified the block structure to reference unread data: bytes that, for some reason, have not been absorbed by the stream. If all the data has been absorbed: the block must be cleared to zero.

It is responsibility of the caller to keep another reference to the memory block, so that it can be released (Fantasy Stream Example, see how this was done in the examples section).

More_Error More_StreamFlush (More_Stream token) Function
Type of function used to flush as much data as possible from the internal context to the output buffer.

More_Error More_StreamFinish (More_Stream token) Function
Type of function used to finish a stream: must flush all the data from the internal context to the output buffer and release all the resources, with the exception of the output buffer itself.

After this function is invoked, the only legal operations on the stream are: reading data and finalising the stream.

This function must be able to return an error, for example: if the end of stream is required to finish but it has not been written to the context with More_StreamWrite().

More_StreamIO Struct Typedef
A pointer to an instance of this type is used as client data to the set-option and get-option functions registered in the optionTable field of the stream driver. Members description follows.
More_Stream input
The token of the input stream.
More_Stream output
The token of the output stream.


Node:Stream Transform Public Interface, Next:, Previous:Stream Drivers, Up:Stream Transform

Public interface

A function is needed to create a new transformation channel from a couple of existing input and an output streams.

Tcl_Channel More_MakeStreamTransform (driver, input, output, subChannel) Function
Creates a new transformation stacked upon an already existing channel.
CONST More_StreamDriver * streamPtr
Pointer to the table of functions that implement the transformation algorithm.
More_Stream input
The token of the input stream. Can be NULL if the transformation is write-only.
More_Stream output
The token of the output stream. Can be NULL if the transformation is read-only.
Tcl_Channel subChannel
The token of the underlying channel.

Returns the token of the transformation channel.

We must supply input and output streams according to the open mode of the underlying channel: if it is read-only we must pass output == NULL, if it is write-only we must pass input == NULL (Channel Object Extractors, for the functions used to detect the open mode of an underlying channel).


Node:Stream TCL Interface, Next:, Previous:Stream Transform Public Interface, Up:Stream Transform

TCL stream interface

It is responsibility of the user code to create a TCL command that initialises the streams; the command implementation must use More_MakeStreamTransform() to build the transformation channel and stack it upon an existing channel.

TCLMORE will add to the channel the support for a set of options through the [fconfigure] command; these will be in addition to the options declared with the optionTable field of the driver.

-flush input
-flush output
Forces the flush operation, causes the More_StreamFlush function to be invoked for the input or output stream.

For output streams: this causes as much data as possible to be flushed from the internal context to the output buffer and sent to the underlying channel, we should invoke [flush $channel] before this.

For input streams: this causes as much data as possible to be flushed from the internal context to the output buffer and to be available for reading; no new data is read from the underlying channel.

-finish input
-finish output
Forces the finalisation of the input or output stream.

For input streams: this causes all the data stored in the internal stream context to be available for reading; after this: no more data can be read from the underlying channel until the transformation is unstacked. No data is read from the underlying channel.

For output streams: all the data is flushed to the output buffer as if [flush $channel] has been invoked, and an attempt is made to write all of it to the underlying channel; after this: no data can be written to the transformation.


Node:Stream Transform Internals, Previous:Stream TCL Interface, Up:Stream Transform

Internals

In inspecting the internals of the transformation channel module, it is useful to remember that:

Channel creation

The open mode for the transformation is implicitly declared by supplying a NULL or non-NULL input or output stream token: if the token is NULL the corresponding direction is disabled. TCL imposes no restriction for transformations: if the underlying channel is read-only, TCL may invoke the output function of the transformation; if the underlying channel is write-only TCL may invoke the input function. The transformation has to take care of itself.

No check is done to ensure that the transformation mode complies with the underlying channel mode: this means that an incompatible mode (example: read-only transformation above write-only channel) will cause a crash (probably).

Close function

Invoked by the generic layer to clean-up driver related informations when the channel is closed. Frees all the resources associated to the channel. Must return zero if the operation is successful, or a non-zero POSIX error code if an error occurs; always returns zero.

This is the only function that finalises the streams. Finalising does not mean finishing. Finishing of the streams must be explicitly requested by the user of the channel.

Input function

The input function is invoked by the generic layer to read data from the device. The general behaviour of the function is: read data from the underlying channel, process it with the stream, put processed data in the memory block supplied by the caller.

The return condition is represented by a pair of integers: the return value that must be the number of read bytes or -1; an output variable that must be set to zero or a POSIX error code.

Tcl_ReadRaw() is used to read data from the underlying channel, so that no translation is performed.

In blocking mode the steps are:

  1. if the output buffer of the stream holds enough data to satisfy the request: use that data and return;
  2. loop reading the underlying channel and processing data with the stream until enough data is in the output buffer;
  3. fill the output block supplied by the caller with output data.

If, while processing data with the stream, the end of stream is detected: the stream driver finishes the stream sending all the data to its output buffer; this operation is completely transparent to the input function. If some data is left unread in the input block: that data is lost.

This behaviour allows the end of file condition on the underlying channel to be handled correctly, that is: if

set data [read $channel]

is executed, all the data is read from the underlying channel and the transformation internal buffer is consumed.

Output function

Invoked by the generic layer to transfer data from an internal buffer to the output device. Must return a non-negative integer indicating how many bytes were written; in case of error must return -1.

If a channel output function returns a partial write in blocking mode: the generic layer invokes it again and again until the request is satisfied. In non-blocking mode the data is left in the output buffer. We do not know if the underlying channel can accept data until we try to write, so: data is always absorbed. Data goes in the output stream and accumulates in its output buffer.

Attempts are done to write data from the output buffer to the underlying channel. In blocking mode the function will loop waiting for the underlying channel to absorb all the data; in non-blocking mode only a single attempt is done.

A transformation channel is always writable; normally: writable events are notified to the generic layer by the underlying channel. If data is absorbed in the stream, no data is sent to the underlying channel and the generic layer is interested in writable events: a timer event is scheduled to notify TCL about the writability of the channel. If TCL revokes interest in the event before it is consumed: the watch function will delete the event.

If an error occurs writing bytes to the underlying channel, data is lost: this means that the stream gets corrupted and we have to close it.

Watch function

Invoked by the generic layer to initialise the event notification mechanism. The event mask supplied by the generic layer is cleared from unsupported events (example: unreadable transformations do not support readable events) and then sent to the underlying channel's watch function.

If TCL is interested in readable events and there is data in the output buffer of the input stream: a timer event is scheduled to notify the channel with the purpose of flushing the stream buffer. If the generic layer aborts its interest the timer is deleted.

If TCL aborts its interest in channel writability and a timer event was scheduled by the output function: the event is deleted.

Handler function

Invoked by the generic layer to notify the transformation about events on the underlying channel. Returns the event mask unchanged: this transformation does not need to absorb events.

Seeking

Seeking is not implemented. If one has to seek it must accumulate data in memory: using a TCL variable or a channel provided by the MEMCHAN extension.

Setting and getting options

Four special options are supported for flushing and finishing the input and output streams. Other options are sent to the functions registered in the driver.

The transformation specific functions will need only the input and output stream tokens to do their job: a little structure is allocated and a pointer to it sent to the driver function.

Special options do not show up when the user requests the channel configuration with [fconfigure $channel].

If a request to flush the input stream is done: as much data as possible is transferred from the internal context to the output buffer and is available to be read.

If a request to flush the output stream is done: as much data as possible is transferred from the internal context to the output buffer, then the output function is invoked with no data to output: this will cause data to be written from the output buffer to the underlying channel, according to the current blocking mode. Requesting such a flush in non-blocking mode is probably a bad idea because we have no way to determine what data is flushed from the TCL buffer to the output stream.

If a request to finish the input stream is done: the input stream is finished without trying to read more data from the underlying channel.

If a request to finish the output stream is done: the stream is finished without including data from the TCL buffer for the channel, then the output function is invoked with no data to output: this will cause data to be written from the output buffer to the underlying channel, according to the current blocking mode. The same remark of the flush operation applies here.


Node:Channel Object Extractors, Previous:Stream Transform, Up:Channel Interface

Object Extractors

int More_GetOpenModeFromObj (interp, objPtr, flags, mode) Function
Extracts a channel open mode from an object. Valid values are the strings: RDONLY, WRONLY, RDWR. The result is stored in the variable referenced by mode, and is an OR-ed combination of TCL_READABLE and TCL_WRITABLE.
Tcl_Interp * interp
The interpreter in which report errors.
Tcl_Obj * objPtr
Pointer to the source object.
int flags
If flags is TCLMORE_READ_OR_WRITE, rather than zero, only RDONLY and WRONLY are accepted.
int * mode
Pointer to the target variable.

Returns TCL_OK or TCL_ERROR. In case of error leaves a message in the interpreter's result and stores LOGIC in the error code variable.

int More_GetTransformOpenModeFromObj (interp, objPtr, subModeMask, flags, modeMaskvar) Function
Extracts a transformation open mode from an object. Valid values are the strings: RDONLY, WRONLY, RDWR. The result is stored in the variable referenced by mode, and is an OR-ed combination of TCL_READABLE and TCL_WRITABLE.

The mode has to be compatible with the mode of the underlying channel: if the channel is read-only, the transformation cannot be write-only; if the channel is write-only the transformation cannot be read-only; a read-only or write-only transformation can be stacked upon a read-write channel.

If the channel mode is read-only and the requested mode is read-write, the extracted value is TCL_READABLE; if the channel mode is write-only and the requested mode is read-write, the extracted value is TCL_WRITABLE. The open mode can be restricted.

Arguments description follow.

Tcl_Interp * interp
The interpreter in which report errors.
Tcl_Obj * objPtr
Pointer to the source object.
int subModeMask
The open mode mask of the underlying channel.
int flags
If flags is TCLMORE_READ_OR_WRITE, rather than zero, only RDONLY and WRONLY are accepted.
int * modeMaskVar
Pointer to the target variable.

Returns TCL_OK or TCL_ERROR. In case of error leaves a message in the interpreter's result and stores LOGIC in the error code variable.

int More_GetChannelFromObj (interp, objPtr, channelVar, modeMaskVar) Function
Extracts a channel token from an object. Arguments description follows.
Tcl_Interp * interp
Pointer to the interpreter to which the channel belongs. Cannot be NULL.
Tcl_Obj * objPtr
Pointer to the source object.
Tcl_Channel * channelVar
If not NULL: pointer to the variable that will hold the channel token.
int * modeMaskVar
If not NULL: pointer to the variable that will hold the open mode mask.

Returns TCL_OK or TCL_ERROR. If an error occurs an error message is left in the interpreter result and the error code is set to LOGIC.


Node:Channel Driver, Next:, Previous:Channel Interface, Up:Top

Channel Driver Facilities


Node:Channel Driver Options, Up:Channel Driver

Driver specific options

This module can be used to organise channel driver options: the ones accessed with the [fconfigure] command.


Node:Channel Driver Options Data Types, Next:, Up:Channel Driver Options

Data Types

More_ChannelDriverOption Struct Typedef
Structure used to declare a channel driver option available through the [fconfigure] command. Fields description follows.
CONST char * name
A string representing the option name. Example: -inputBufferSize.
More_ChannelDriverSetOptionProc * setOptionProc
Pointer to the function used to configure the option.
More_ChannelDriverGetOptionProc * getOptionProc
Pointer to the function used to retrieve the value of the option.

More_ChannelDriverSetOptionProc Function Typedef
Prototype of the function used to configure an option.
int More_ChannelDriverSetOptionProc (ClientData D,
				     Tcl_Interp *interp,
				     CONST char *optionName,
				     CONST char *optionValue)

The arguments have the same meaning of the ones in the declaration of Tcl_DriverSetOptionProc.

More_ChannelDriverGetOptionProc Function Typedef
Prototype of the function used to retrieve an option value.
int More_ChannelDriverGetOptionProc (ClientData D,
				     Tcl_Interp *interp,
				     CONST char *optionName,
				     Tcl_DString *optionValue)

The arguments have the same meaning of the ones in the declaration of Tcl_DriverGetOptionProc.


Node:Channel Driver Options Functions, Next:, Previous:Channel Driver Options Data Types, Up:Channel Driver Options

Public Interface Functions

int More_ChannelDriverSetOption (table, channel, D, interp, optionName, optionValue) Function
Selects the correct option from the table and invokes the set option function. Arguments description follows.
CONST More_ChannelDriverOption * table
Pointer to an array of structures describing the supported options. It is used as argument for Tcl_GetIndexFromObjStruct().
Tcl_Channel channel
The channel token. If the channel is a transformation and the option is not recognised by its driver, the request is propagated to the underlying channel.
ClientData D
The channel instance data.
Tcl_Interp * interp
If not NULL, the interpreter in which report errors.
CONST char * optionName
Pointer to a string representing the option name.
CONST char * optionValue
Pointer to a string representing the option value.

Returns the code returned by the set-function or TCL_ERROR if an unsupported option is selected. In case of error leaves an error in the interpreter.

int More_ChannelDriverGetOption (table, channel, D, interp, optionName, optionValue) Function
Selects the correct option from the table and invokes the get option function. Arguments description follows.
CONST More_ChannelDriverOption * table
Pointer to an array of structures describing the supported options. It is used as argument for Tcl_GetIndexFromObjStruct().
Tcl_Channel channel
The channel token. If the channel is a transformation and the option is not recognised by its driver, the request is propagated to the underlying channel.
ClientData D
The channel instance data.
Tcl_Interp * interp
If not NULL, the interpreter in which report errors.
CONST char * optionName
Pointer to a string representing the option name.
Tcl_DString * optionValue
Pointer to a dynamic string that must be filled with the option value or the option/value pairs.

Returns the code returned by the get-function or TCL_ERROR if an unsupported option is selected. In case of error leaves an error in the interpreter.


Node:Channel Driver Options Examples, Previous:Channel Driver Options Functions, Up:Channel Driver Options

Examples

Usage example follows.

static CONST More_ChannelDriverOption optionTable[] = {
  {  "-inputBufferSize", SetOptionBufferSize, GetOptionBufferSize },
  { "-outputBufferSize", SetOptionBufferSize, GetOptionBufferSize },
  { NULL, NULL, NULL }
};

int
BufchanSetOption (D, interp, optionName, newValue)
     ClientData		D;
     Tcl_Interp *	interp;
     CONST char *	optionName;
     CONST char *	newValue;
{
  ChannelInstance *     instance = (ChannelInstance *) D;

  return More_ChannelDriverSetOption(optionTable, instance->channel,
                                     D, interp, optionName, newValue);
}

int
BufchanGetOption (D, interp, optionName, newValue)
     ClientData		D;
     Tcl_Interp *	interp;
     CONST char *	optionName;
     Tcl_DString *	optionValue;
{
  ChannelInstance *     instance = (ChannelInstance *) D;

  return More_ChannelDriverGetOption(optionTable, instance->channel,
                                     D, interp, optionName, optionValue);
}

int
SetOptionBufferSize (D, interp, optionName, newValue)
     ClientData		D;
     Tcl_Interp *	interp;
     CONST char *	optionName;
     CONST char *	newValue;
{
   ...
}

int
BufchanGetOption (D, interp, optionName, optionValue)
     ClientData		D;
     Tcl_Interp *	interp;
     CONST char *	optionName;
     Tcl_DString *	optionValue;
{
   ...
}


Node:Dynamic Strings, Next:, Previous:Channel Driver, Up:Top

Dynamic Strings

void More_DStringPrintf (Tcl_DString * string, CONST char * format, ...) Function
A printf() version that appends the result to a dynamic string.

void More_DStringAppendSizeT (Tcl_DString * string, size_t size) Function
Appends a the string representing a size_t value to a dynamic string.


Node:Delayed Scripts, Next:, Previous:Dynamic Strings, Up:Top

Queuing delayed script execution

This module allows to queue the evaluation of a script in an interpreter. The script can be evaluated more than once and with additional arguments different at each execution.

The main problem when queuing a task in the event loop is that, when the event is consumed, the subject of the task may have been destroyed. So we need at least one of the following solutions: the ability to remove an event from the queue before it is consumed; the ability to test if the subject of the task still exists.

In the case of this module there may be a number of subjects: the TCL interpreter and whatever object the script acts upon.

A key role in the implementation of this module is delegated to the Tcl_Preserve() and Tcl_Release() functions: they allow simple reference counting on data instances without the need to insert a reference counter field in the data structure itself. Examples of structures that are handled with this mechanism in the TCL core are: interpreters (Tcl_Interp), channels (Tcl_Channel) and their associated instance structures (whatever they are).

Handling interpreter data instances

Inspection of generic/tclBasic.c (TCL core version 8.4.5) reveals that when a Tcl_Interp instance is destroyed by Tcl_DeleteInterp(), the following code is evaluated:

Tcl_EventuallyFree((ClientData) interp, \
      (Tcl_FreeProc *) DeleteInterpProc);

so if we have preserved the interpreter:

Tcl_Preserve(interp);

actually it will not be deleted by Tcl_DeleteInterp(), but only marked for deletion; only when we:

Tcl_Release(interp);

the free function will be invoked. If we do not preserve the interpreter: Tcl_EventuallyFree() will immediately call the free function.

The fact that the data instance is still there when an event is consumed does not mean that the interpreter still exists: we have to test this:

if (Tcl_InterpDeleted(interp)) { ... }

and we can be sure that interp is a valid pointer because we have preserved the instance.

The delayed script module functions encapsulate this mechanism to make sure that, when the event is consumed, the script is evaluated only if the interpreter is still there. If the interpreter has been marked for deletion the script is, silently, not executed.

Queuing tasks in the event loop

Inspection of Tcl_DoOneEvent() in generic/tclNotify.c (TCL core 8.4.5) reveals that when TCL enters the event loop a list of sources are queried for events to be consumed: the first that has at least an event ready is the selected one.

TCL itself defines a set of sources, and we can add sources if we need it. The importance of having an event source is that whole classes of events can be removed from the loop simply be detaching a source: doing this with a simple list of events coming from different modules would imply the iteration over all the queued items and could be inefficient.

The delayed script module does not need an event source of its own: it makes use of timer events. This is because the module is simple and the interface of timer event is very easy: only two functions, Tcl_CreateTimerHandler() and Tcl_DeleteTimerHandler().

This module queues timer events with a delay of one millisecond: this is somewhat like queuing an event at the end of the queue. Each instance of delayed script stores the token of the timer event in its state; this allows the caller to abort the evaluation of the script before the event is consumed. This raises a problem: who releases the resources allocated to the delayed script instance?

If the module originating the script does not keep a reference to it: when the event is consumed, the script instance is finalised: no problem. If the originating module keeps a reference in a context, there are two scenarios: the event is consumed, the event is aborted.

  1. When the event is consumed the originating module must be informed about the event, so that it can release the reference to the instance.
  2. The module directly calls the finalisation function: this causes the timer event to be destroyed and no problem arises.

To solve this problem the delayed script instance can register a callback in its internal context.

Functions

More_DScript Opaque Struct Pointer
A reference to a delayed script instance. A module may store this value in a context handling the reference with Tcl_Reserve() and Tcl_Release().

More_DScript More_DScriptInit (Tcl_Interp * interp, Tcl_Obj * body) Function
Allocates and initialises a new delayed script, returns a reference to the instance. The script is not queued. interp is the pointer to the interpreter in which the script must be evaluated; the referenced interpreter is preserved. body is a pointer to the object holding the script body.

More_DScript More_DScriptCopy (More_DScript token) Function
Duplicates an instance. This function is useful if we need to queue a script multiple times.

void More_DScriptFinal (More_DScript token) Function
Finalises an instance. This function may be used: to release an instance that is not queued; to abort a queued script; to free the resources after the timer event has been consumed.

void More_DScriptQueue (More_DScript token, More_Objects args) Function
Queues the script in the event loop. args holds additional argument to be appended to the script body, there may be no arguments.

void More_DScriptCallback (More_DScript token, More_Callback cb) Function
Registers a callback in the context of the instance: the callback is invoked after the event has been consumed.


Node:Miscellaneous Funcs, Next:, Previous:Delayed Scripts, Up:Top

Miscellaneous functions

int More_EqualVarnames (name, var, key) Function
Compares a variable name in a string with a pair array name/array key. The arguments are all of type CONST char *: name, pointer to the variable name; var, pointer to the array name; key, pointer to the key name. Returns true if the two variable names are equal, false otherwise.

CONST char * More_MakeName (Tcl_Interp *interp, CONST char *template) Function
Builds a new name for an item, unique for an interpreter.

A hash table is a member of the data associated to the interpreter: its keys are the name templates and its values are the associated counters; at the first invocation of this function it is initialised.

If the string referenced by template is not a key in the table, a new entry is created and its value set to 1; else its value is retrieved, incremented by one and updated.

The counter is used together with the template to build a new name: the template should be a printf() format string with a single field selector for an unsigned integer %u. The new name is stored in a dynamically allocated buffer, acquired with ckalloc().

The return value is the pointer to the buffer.

CONST char * More_DupAllocString (CONST char * string) Function
Duplicates a string into dynamically allocated buffer. string is the pointer to the source string. Returns the pointer to the new buffer, allocated with ckalloc().

CONST char * More_Asprintf (CONST char * format, ...) Function
An sprintf() version that allocates enough memory for the string. Returns a pointer to the output string, allocated with ckalloc().

CONST char * More_Bsprintf (CONST char * format, va_list ap) Function
An sprintf() version that allocates enough memory for the string. Returns a pointer to the output string, allocated with ckalloc().

Tcl_Obj * More_ObjPrintf (CONST char * format, ...) Function
An sprintf() version that outputs the string into a new string object. Returns a pointer to the new string object, with reference counter set to zero. If format is NULL the returned object is empty.


Node:Miscellaneous Types, Next:, Previous:Miscellaneous Funcs, Up:Top

Miscellaneous types

String

More_String Struct Typedef
Type of NULL-terminated string. This type exists only to make it easy to extract a string from an argument for a TCL command implemented with the TCLMORE infrastructure.
int len
The number of characters.
char * ptr
Pointer to the first character.

MORE_STRING_NULL_VALUE Macro
The initialisation value for an empty More_String. This symbol can be used to initialise the state invocation structure for a TCL command implemented with the TCLMORE infrastructure (Command Interface for details).


Node:Stub Mechanism, Next:, Previous:Miscellaneous Types, Up:Top

Using the stub mechanism

The stubs mechanism allows us to dynamically link a client extension to a version of TCLMORE and to use it with future versions, without recompiling, as long as the future versions do not change the interface.

To do this we link our client extension with TCLMORE's stub library (an object file whose name is something like libtclmorestub...) and compile our code with the symbol USE_TCLMORE_STUB defined. Our client library's initialisation function must contain the following code:

#include "tclmore.h"

...

int
Client_Init (...)
{
...

#ifdef USE_TCLMORE_STUB
  if (More_InitStub(interp, "1.0", 0) == NULL) {
    return TCL_ERROR;
  }
#endif

...
}

where 1.0 is the version of TCLMORE that the client library is supposed to use.


Node:Package License, Next:, Previous:Stub Mechanism, Up:Top

BSD style license

Copyright © 2002, 2003, 2004 Marco Maggi.

The author hereby grant permission to use, copy, modify, distribute, and license this software and its documentation for any purpose, provided that existing copyright notices are retained in all copies and that this notice is included verbatim in any distributions. No written agreement, license, or royalty fee is required for any of the authorized uses. Modifications to this software may be copyrighted by their authors and need not follow the licensing terms described here, provided that the new terms are clearly indicated on the first page of each file where they apply.

IN NO EVENT SHALL THE AUTHOR OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHOR HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE AUTHOR AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHOR AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.


Node:Documentation License, Next:, Previous:Package License, Up:Top

Documentation license

This document is copyright © 2002, 2003, 2004 by Marco Maggi.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.


Node:Concept Index, Previous:Documentation License, Up:Top

An entry for each concept

Table of Contents