 |
smax-clib v1.0
A C/C++ client library for SMA-X
|
|
smax-clib v1.0
A C/C++ client library for SMA-X
|
A free C/C++ client library and toolkit for the SMA Exchange (SMA-X) structured real-time database
Author: Attila Kovacs
Updated for version 1.0 and later releases.
If you use SMA-X for your project, please cite as:
You may also find the citation record on ADS as 2025JATIS..11a7001K.
The SMA Exchange (SMA-X) is a free, high performance, and versatile real-time data sharing platform for distributed software systems. It is built around a central Redis (or equivalent) database, and provides efficient atomic access to structured data, including specific branches and/or leaf nodes, with associated metadata. SMA-X was developed at the Submillimeter Array (SMA) observatory, where we use it to share real-time data among hundreds of computers and nearly a thousand individual programs. The smax-clib library is free to use, in any way you like, without licensing restrictions.
SMA-X consists of a set of server-side LUA scripts that run on Redis (or one of its forks / clones such as Valkey or Dragonfly); a set of libraries to interface client applications; and a set of command-line tools built with them. Currently we provide client libraries for C/C++ (C99) and Python 3. We may provide Java and/or Rust client libraries too in the future.
There are no official releases of smax-clib yet. An initial 1.0.0 release is expected early/mid 2025. Before then the API may undergo slight changes and tweaks. Use the repository as is at your own risk for now.
The SMA-X C/C++ library has a build and runtime dependency on the xchange and RedisX libraries also available at the Sigmyne Github repositories:
Additionally, to configure your Redis (or Valkey / Dragonfly) servers for SMA-X, you will need the Smithsonian/smax-server repo also.
The smax-clib library can be built either as a shared (libsmax.so[.1]) and as a static (libsmax.a) library, depending on what suits your needs best.
You can configure the build, either by editing config.mk or else by defining the relevant environment variables prior to invoking make. The following build variables can be configured:
After configuring, you can simply run make, which will build the shared (lib/libsmax.so[.1]) and static (lib/libsmax.a) libraries, local HTML documentation (provided doxygen is available), and performs static analysis via the check target. Or, you may build just the components you are interested in, by specifying the desired make target(s). (You can use make help to get a summary of the available make targets).
After building the library you can install the above components to the desired locations on your system. For a system-wide install you may simply run:
Or, to install in some other locations, you may set a prefix and/or DESTDIR. For example, to install under /opt instead, you can:
Or, to stage the installation (to /usr) under a 'build root':
Provided you have installed the shared (libsmax.so, libredisx.so, and libxchange.so) or static (libsmax.a, libredisx.a, and libxchange.so) libraries in a location that is in your LD_LIBRARY_PATH (e.g. in /usr/lib or /usr/local/lib) you can simply link your program using the -lsmax -lredisx -lxchange flags. Your Makefile may look like:
(Or, you might simply add -lsmax -lredisx -lxchange to LDFLAGS and use a more standard recipe.) And, in if you installed the smax-clib, RedisX, and/or xchange libraries elsewhere, you can simply add their location(s) to LD_LIBRARY_PATH prior to linking.
The smax-clib library provides two basic command-line tools:
The tools can be compiled with make tools. Both tools can be run with the --help option for a simple help screen on usage. E.g.:
These command-line tools provide a simple means to interact with SMA-X from the shell or a scripting language, such as bash, or perl (also python though we recommend to use the native Sigmyne/smax-python library instead).
Bu default, the library assumes that the Redis server name used for SMA-X is either stored in the environment variable SMAX_HOST or is smax (e.g. you may assign smax to an IP address in /etc/hosts), and that the Redis is on the default port 6379/tcp. However, you can configure to use a specific host and/or an alternative Redis port number also, e.g.:
Also, while SMA-X will normally run on database index 0, you can also specify a different database number to use. E.g.:
(Note, you can switch the database later also, but beware that if you have an active subscription client open, you cannot switch that client until the subscriptions are terminated.)
You can also set up the authentication credentials for using the SMA-X database on the Redis server:
By default smax_clib will connect both an interactive and a pipeline (high-throughput) Redis client. However, if you are planning to only use interactive mode (for setting an queries), you might not want to connect the pipeline client at all:
And finally, you can select the option to automatically try reconnect to the SMA-X server in case of lost connection or network errors (and keep track of changes locally until then):
You can also use SMA-X with a TLS encrypted connection. (We don't recommend using TLS with SMA-X in general though, since it may adversely affect the performance / throughput of the database.) When enabled, Redis normally uses mutual TLS (mTLS), but it may be configured otherwise also. Depending on the server configuration, and the level of security required, you may configure some or all of the following options:
The SMA-X configuration is activated at the time of connection (see below), after which it persists, through successive connections also. That means, that once you have connected to the server, you cannot alter the configuration prior to another connection attempt, unless you call smaxReset() first while disconnected. smaxReset() will discard the currently configured Redis instance, so the next connection will create a new one, with the current configuration.
Once you have configured the connection parameters, you can connect to the configured (or default) server by:
And, when you are done, you should disconnect with:
The user of the smax-clib library might want to know when connections to the SMA-X server are established, or when disconnections happen, and may want to perform some configuration or clean-up accordingly. For this reason, the library provides support for connection hooks – that is custom functions that are called in the even of connecting to or disconnecting from a Redis server.
Here is an example of a connection hook, which simply prints a message about the connection to the console.
And, it can be activated prior to the smaxConnect() call.
The same goes for disconnect hooks, using smaxAddDisconnectHook() instead.
For SMA-X we use the terms sharing and pulling, instead of the more generic get/set terminology. The intention is to make programmers conscious of the fact that the transactions are not some local access to memory, but that they involve networking, and as such may be subject to unpredictable latencies, network outages, or other errors.
At the lowest level, the library provides two functions accordingly: smaxShare() and smaxPull(), either to send local data to store in SMA-X, or to retrieve data from SMA-X for local use, respectively. There are higher-level functions too, which build on these, providing a simpler API for specific data types, or extra features, such as lazy or pipelined pulls. These will be discussed in the sections below. But, before that, let's look into the basics of how data is handled between you machine local data that you use in your C/C++ application and its machine-independent representation in SMA-X.
Here is an example for a generic sharing of a double[] array from C/C++:
Pulling data back from SMA-X works similarly, e.g.:
The metadata argument is optional, and can be NULL if not required.
For every variable (structure or leaf node) in SMA-X there is also a set of essential metadata that is stored in the Redis database, which describe the data themselves, such as the native type (at the origin); the size as stored in Redis; the array dimension and shape; the host (and program) which provided the data; the time it was last updated; and a serial number.
The user of the library has the option to retrieve metadata together with the actual data, and thus gain access to this information. The header file smax.h defines the Xmeta type as:
One nifty feature of the library is that as a consumer you need not be too concerned about what type or size of data the producer provides. The program that produces the data may sometimes change, for example from writing 32-bit floating point types to a 64-bit floating point types. Also while it produced data for say 10 units before (as an array of 10), now it might report for just 9, or perhaps it now reports for 12.
The point is that if your consumer application was written to expect ten 32-bit floating floating point values, it can get that even if the producer changed the exact type or element count since you have written your client. The library will simply apply the necessary type conversion automatically, and then truncate, or else pad (with zeroes), the data as necessary to get what you want.
The type conversion can be both widening or narrowing. Strings and numerical values can be converted to one another through the expected string representation of numbers and vice versa. Boolean true values are treated equivalently to a numerical value of 1 and all non-zero numerical values will convert to boolean true.
And, if you are concerned about the actual type or size (or shape) of the data stored, you have the option to inspect the metadata, and make decisions based on it. Otherwise, the library will just take care of giving you the available data in the format you expect.
Often enough we deal with scalar quantities (not arrays), such as a single number, boolean value, or a string (yes, we treat strings i.e., char *, as a scalar too!).
Here are some examples of sharing scalars to SMA-X. Easy-peasy:
Or pulling them from SMA-X:
The generic smaxShare() function readily handles 1D arrays, and the smaxPull() handles native (monolithic) arrays of all types (e.g. double[][], or boolean[][][]). However, you may want to share multidimensional arrays, noting their specific shapes, or else pull array data for which you may not know in advance what size (or shape) of the storage needed locally for the values stored in SMA-X for some variable. For these reasons, the library provides a set of functions to make the handling of arrays a simpler too.
Let's begin with sharing multi-dimensional arrays. Instead of smaxShare(), you can use smaxShareArray() which allows you to define the multi-dimensional shape of the data beyond just the number of elements stored. E.g.:
Note, that the dimensions of how the data is stored in SMA-X is determined solely by the '2' dimensions specified as the 4th argument and the corresponding 2 elements in the shape array. The data could have been any pointer to an array of floats containing at least the required number of element (8 in the example above).
For 1D arrays, you have some convenience methods for sharing specific types. These can be convenient because they eliminate one potential source of bugs, where the type argument to smaxShare() does not match the pointer type of the data. Using, say, smaxShareFloats() instead to share a 1D floating-point array instead of the generic smaxShare() will allow the compiler to check and warn you if the data array is not the float * type. E.g.:
Similar functions are available for every built-in type (primitives, plus strings and booleans). For pulling arrays without knowing a-priori the element count or shape, there are also convenience functions, such as:
As illustrated, the above will return a dynamically allocated array with the required size to hold the data, and the size and shape of the data is returned in the metadata that was also supplied with the call. After using the returned data (and ensuring that it is not NULL), you should always call free() on it to avoid memory leaks in your application.
You can share entire data structures, represented by an appropriate XStructure type (see the xchange library for a description and usage):
Or, you can read a structure, including all embedded substructures in it, with smaxPullStruct():
Note, that the structure returned by smaxPullStruct() is in the serialized format of SMA-X. That is, all leaf nodes are stored as strings, just as they appear in the Redis database. Hence, we used the smaxGet...Field() methods above to deserialize the leaf nodes as needed on demand. If you want to use the methods of xchange to access the structure, you will need to convert to binary format first, using smax2xStruct(XStructure *).
Note also, that pulling large structures can be an expensive operation on the Redis server, and may block the server for longer than usual periods, causing latencies for other programs that use SMA-X. It's best to use this method for smallish structures only (with, say, a hundred or so or fewer leaf nodes).
What happens if you need the data frequently? Do you pound on the database at some high-frequency? No, you probably no not want to do that, especially if the data you need is not necessarily changing fast. There is no point on wasting network bandwidth only to return the same values again and again. This is where 'lazy' pulling excels.
From the caller's perspective lazy pulling works just like regular SMA-X pulls, e.g.:
or
But, under the hood, it does something different. The first time a new variable is lazy pulled it is fetched from the Redis database just like a regular pull. But, it also will cache the value, and watch for update notifications from the SMA-X server. Thus, as long as no update notification is received, successive calls will simply return the locally cached value. This can save big on network usage, and also provides orders of magnitude faster access so long as the variable remains unchanged.
When the variable is updated in SMA-X, our client library will be notified, and one of two things can happen:
The choice between the two is yours, and you can control which suits your need best. The default behavior for lazy pulls is (1), but you may call smaxCache() after the first pull of a variable, to indicate that you want to enable background cache updates (2) for it. The advantage of (1) is that it will never serve you outdated data even if there are significant network latencies – but you may have to wait a little to fetch updates. On the other hand (2) will always provide a recent value with effectively no latency, but this value may be outdated if there are delays on the network updating the cache. The difference is typically at the micro-seconds level on a local LAN. However, (2) may be preferable when you need to access SMA-X data from timing critical code blocks, where it is more important to ensure that the value is returned quickly, rather than whether it is a millisecond too old or not.
You can also explicitly select the second behavior by using smaxGetCached() instead of smaxLazyPull():
In either case, when you are done using lazy variables, you should let the library know that it no longer needs to watch updates for these, by calling either smaxLazyEnd() on specific variables, or else smaxLazyFlush() to stop watching updates for all lazy variables. (A successive lazy pull will automatically start watching for updates again, in case you wish to re-enable).
The regular pulling of data from SMA-X requires a separate round-trip for each and every request. That is, successive pulls are sent only after the responses from the prior pull has been received. A lot of the time is spent on waiting for responses to come back. With round trip times in the 100 μs range, this means that this method of fetching data from SMA-X is suitable for obtaining at most a a few thousand values per second.
However, sometimes you want to get access to a large number of values faster. This is what pipelined pulling is for. In pipelined mode, a batch of pull requests are sent to the SMA-X Redis server in quick succession, without waiting for responses. The values, when received are processed by a dedicated background thread. And, the user has an option of either waiting until all data is collected, or asking for as callback when the data is ready.
Again it works similarly to the basic pulling, except that you submit your pull request to a queue with smaxQueue(). For example:
Pipelined (batched) pulls have dramatic effects on performance. Rather than being limited by round-trip times, you will be limited by the performance of the Redis server itself (or the network bandwidth on some older infrastructure). As such, instead of thousand of queries per second, you can pull 2-3 orders of magnitude more in a given time, with hundreds of thousands of pull per second this way.
After you have submitted a batch of pull request to the queue, you can create a synchronization point as:
A synchronization point is a marker in the queue that we can wait on. After the synchronization point is created, you can submit more pull request to the same queue (e.g. for another processing block), or do some other things for a bit (since it will take at least some microseconds before the data is ready). Then, when ready you can wait on the specific synchronization point to ensure that data submitted prior to its creation is delivered from SMA-X:
The alternative to synchronization points and waiting, is to provide a callback function, which will process your data as soon as it is available, e.g.:
Then submit this callback routine to the queue after the set of variables it requires with:
If you might still have some pending pipelined pulls that have not received responses yet, you may want to wait until all previously submitted requests have been collected. You can do that with:
The LUA scripts that define SMA-X interface on the Redis server send out PUB/SUB notifications for every variable on their own dedicate PUB/SUB channel whenever the variable is updated. By default, lazy access methods subscribe to these messages and use them to determine when to invalidate the cache and fetch new values from the database again. However, you may subscribe and use these messages outside of the lazy update routines also. The only thing you need to pay attention to is not to unsubscribe from update notifications for those variables that have multiple active monitors (including lazy updates).
One common use case is that you want to execute some code in your application when some value changes in SMA-X. You can do that easily, and have two choices on how you want to trigger the code execution: (1) you can block execution of your current thread until an update notification is received for one of the variables (or patterns) of interest to which you have subscribed, or (2) you can let smax_clib call a designated function of your application when such an update notification is captured. We'll cover these two cases separately below.
However, in either case you will have to subscribe to the variable(s) or pattern(s) of interest with smaxSubscribe() before updates can be processed, e.g.
and/or pattern(s):
You can subscribe to any number of variables or patterns in this way. smax_clib will receive and process notifications for all of them. (So beware of creating unnecessary network traffic.)
The first option for executing code conditional on some variable update is to block execution in the current thread and wait until the variable(s) of interest change(s) (or until some timeout limit is reached). There is a group of functions smaxWaitOn...() that do exactly that, provided you have already subscribed to receiving updates for the desired variables / patterns. For example:
Similar methods allow you to wait for updates on any subscribed variable in selected tables, or the update of select variables in all subscribed tables, or any of the subscribed variables to change for the wait to end normally (with return value 0).
The last parameter (NULL in the above example) allows to pass a pointer to a POSIX sempahore, which can be used for gating another thread, which might also want exclusive access to the SMA-X notifications, but we do not want it to accidentally block entering the wait in a timely manner.
Sometimes, you don't want to block execution, but you want to make sure some code executes when the a variable or variables of interest get updated. For such cases you can designate your own RedisSubscriberCall callback function, e.g.:
Once you have defined your callback function, you can activate it with smaxAddSubscriber(), e.g.:
When you no longer need to process such updates, you can simply remove the function from being called via smaxRemoveSubscriber(), and if you are absolutely sure that no other part of your code needs the subscription(s) that could trigger it, you can also unsubscribe from the trigger variables/pattern to eliminate unnecessary network traffic.
One word of caution on callbacks is that they are expected to:
If the above two conditions cannot be guaranteed, it's best practice for your callback to place a copy of the callback information on a queue, and then spawn or notify a separate thread to process the information in the background, including discarding the copied data if it's no longer needed. Alternatively, you can launch a dedicated processor thread early on, and inside it wait for the updates before executing some complex action. The choice is yours.
It is possible to use SMA-X for remote control of programs on distributed systems. In effect, any client can set designated control variables / values. These variables are monitored by an appropriate server program, which acts to changes to the 'commanded' values accordingly, and report the result back in a related other SMA-X variable. The client thus can obtain confirmation from the response variable after it submits it requested 'command' variable.
On the server side, you will need a function (of SMAXControlFunction type), which acts when some control variable changes. E.g.:
It is important to remember that each action taken on a control variable should the set the designated response variable exactly once, and only when the action is completed, so that the calling client can use it to confirm the completed action. The same action action is free to set any number of other values in SMA-X before signaling completion, thus 'returning' further data to the caller, if needed.
Next, all you have to do is specify what control variable to use this function with and what optional pointer argument to pass on to it:
The above call will subscribe for updates to system:subsystem:control_value and will call my_control_function with actual_value as the optional argument. You may change the function called later, or undefine it by calling smaxSetControlFunction() with NULL as the function pointer.
Clearly, the same processing function can be used with multiple control values, if convenient, or you may specify different control functions to every control value if it makes more sense for the implementation.
One thing to watch out for on the server-side implementation is that control functions are called asynchronously and immediately, each time the control variable updates. As such, a control function may be called while another, or even the same one, is in the middle of performing its task for a prior 'command'. You should therefore use mutexes as necessary to prevent the concurrent execution of program controls as appropriate. E.g.
From the client side, you control the above server by setting system:subsystem:control_value to an appropriate new value, and then wait for the response in system:subsystem:actual_value. You can do that via smaxControl() or one of its type-specific variants. Since in the above server example, we use integer control value and reply, we'll use smaxControlInt():
The NULL as the 3rd argument is a shorthand to indicate that we expect the reply in the same hash table in which we set control_value (that is in system:subsystem). If we expect the response in some other location, we can specify the appropriate table name instead of the NULL pointer in the example above.
As hinted earlier, the remote control via SMA-X relies on a single control variable, and a single response variable provides confirmation of completion back to the client. This scheme does not preclude passing multiple values to the server, or receiving multiple values as a response. A client may set a number of call parameters in SMA-X before triggering the action on them by the designated control variable. The server will not act on the parameters alone. Instead, only when the designated control (trigger) variable is set it proceeds to read all associated parameter values from SMA-X.
Similarly, the server may set a number of return parameters during its action, before finally setting the designated response variable to indicate completion. For example, consider locking a local oscillator to a designated frequency and sideband (two parameters). The client might do that by:
And the server might process the request as:
SMA-X also provides a standard for reporting program status, warning, and error messages via the Redis PUB/SUB infrastructure.
Broadcasting program messages to SMA-X is very simple using a set of dedicated messaging functions by message type. These are:
| smax_clib function | Description |
|---|---|
| smaxSendStatus(const char *msg, ...) | sends a status message |
| smaxSendInfo(const char *msg, ...) | sends an informational message |
| smaxSendDetail(const char *msg, ...) | sends optional status/information detail |
| smaxSendDebug(const char *msg, ...) | sends a debugging messages |
| smaxSendWarning(const char *msg, ...) | sends a warning message |
| smaxSendError(const char *msg, ...) | sends an error message |
| smaxSendProgress(double fraction, const char *msg, ...) | sends a progress update and message |
All the above methods work like printf(), and can take additional parameters corresponding to the format specifiers contained in the msg argument.
By default, the messages are sent under the canonical program name (i.e. set by _progname on GNU/Linux systems) that produced the message. You can override that, and define a custom sender ID for your status messages, by calling smaxSetMessageSenderID() prior to broadcasting, e.g.:
On the receiving end, other applications can process such program messages, for a selection of hosts, programs, and message types. You need to prepare you message processor function(s) first, e.g.:
The processor function does not return any value, since it is called by a background thread, which does not check for return status. The XMessage type, a pointer to which is the sole argument of the processor, is defined in smax.h as:
Once you have your message consumer function, you can set it to be called for messages from select hosts, programs, and/or select message types, using smaxAddMessageProcessor(), e.g.:
Each string argument (host, prog, and type) may take an asterisk ("*") or NULL as the argument to indicate that the processor function should be called for incoming messages for all values for the given parameter.
The processor function can also inspect what type of message it received by comparing the XMessage type value against one of the predefined constant expressions in smax.h:
| XMessage type | Description |
|---|---|
| SMAX_MSG_STATUS | status update |
| SMAX_MSG_INFO | informational program message |
| SMAX_MSG_DETAIL | additional detail (e.g. for verbose messages). |
| SMAX_MSG_PROGRESS | progress update. |
| SMAX_MSG_DEBUG | debug messages (also e.g. traces) |
| SMAX_MSG_WARNING | warning message |
| SMAX_MSG_ERROR | error message |
Once you no longer need to process messages by the given processor function, you can remove it from the call list by passing its ID number (<0) to smaxRemoveMessageProcessor().
The principal error handling of the library is an extension of that of xchange, with further error codes defined in smax.h and redisx.h. The functions that return an error status (either directly, or into the integer designated by a pointer argument), can be inspected by smaxErrorDescription(), e.g.:
You can enable verbose output of the library with smaxSetVerbose(boolean). When enabled, it will produce status messages to stderrso you can follow what's going on. In addition (or alternatively), you can enable debug messages with xSetDebug(boolean). When enabled, all errors encountered by the library (such as invalid arguments passed) will be printed to stderr, including call traces, so you can walk back to see where the error may have originated from. (You can also enable debug messages by default by defining the DEBUG constant for the compiler, e.g. by adding -DDEBUG to CFLAGS prior to calling make).
For helping to debug your application, the xchange library provides two macros: xvprintf() and xdprintf(), for printing verbose and debug messages to stderr. Both work just like printf(), but they are conditional on verbosity being enabled via xSetVerbose(boolean) and xSetDebug(boolean), respectively. Applications using this library may use these macros to produce their own verbose and/or debugging outputs conditional on the same global settings.
Some obvious ways the library could evolve and grow in the not too distant future:
If you have an idea for a must have feature, please let me (Attila) know. Pull requests, for new features or fixes to existing ones, are especially welcome!
A predictable release schedule and process can help manage expectations and reduce stress on adopters and developers alike.
smax-clib will try to follow a quarterly release schedule. You may expect upcoming releases to be published around March 1, June 1, September 1, and/or December 1 each year, on an as-needed basis. That means that if there are outstanding bugs, or new pull requests (PRs), you may expect a release that addresses these in the upcoming quarter. The dates are placeholders only, with no guarantee that a new release will actually be available every quarter. If nothing of note comes up, a potential release date may pass without a release being published.
New features are generally reserved for the feature releases (e.g. 1.x.0 version bumps), although they may also be rolled out in bug-fix releases as long as they do not affect the existing API – in line with the desire to keep bug-fix releases fully backwards compatible with their parent versions.
In the weeks and month(s) preceding releases one or more release candidates (e.g. 1.0.1-rc3) will be published temporarily on GitHub, under Releases, so that changes can be tested by adopters before the releases are finalized. Please use due diligence to test such release candidates with your code when they become available to avoid unexpected surprises when the finalized release is published. Release candidates are typically available for one week only before they are superseded either by another, or by the finalized release.
Copyright (C) 2025 Attila Kovács
1.17.0