Receiver Instrument

© 2007-2021 Kevan Hashemi Open Source Instruments Inc.

Contents

Description
Receiver Messages
Data Acquisition
Receiver Script
Message Display
Channel Selection
Message Analysis
Error Handling
Data Processing

Description

Note: This manual applies to the Receiver Instrument bundled with LWDAQ 10.2.12+.

Note: The Receiver Instrument was formerly called the Recorder Instrument.

The Receiver Instrument is available in the Instrument Menu of the LWDAQ program, which you can download for free here. The Receiver Instrument downloads signals from a particular class of device that we call data receivers. Data receivers accumulate messages and make them available for download. The messages accumulated by any given data receiver are of fixed length: a one-byte channel number, a two-byte sample value, a one-byte timestamp, and an optional payload. The table below lists the existing data receivers.

AssemblyNamePayload
(Bytes)
Description
A3018 Data Receiver 0 SCT receiver, single antenna input, obsolete, still supported,
specify internet addres and driver socket for readout.
A3027 Octal Data Receiver 0 SCT receiver, eight antenna inputs, in production,
specify internet addres and driver socket for readout.
A3032 Animal Location Tracker 16 SCT tracker, requires assistant receiver, obsolete, not supported,
specify internet addres and driver socket for readout.
A3038 Animal Location Tracker 16 SCT receiver and tracker, in production, on-board channel selection
supported, specify internet addres only for readout.
Table: Existing Data Receivers. Older data receivers must be connected to a LWDAQ Driver for power and internet communication. Newer data receivers need only a power-over-ethernet connection.

A sequence of messages with the same channel number is a signal. Signals are generated by devices like Subcutaneous Transmitters (SCTs), Implantable Stimulators (ISS), and Implantable Inertial Sensors (IIS). These devices transmit their messages with microwaves that the data receiver picks up with its antennas. The Receiver Instrument makes contact with the data receiver over ethernet. Some data receivers provide a direct power-over-ethernet (PoE) connection, others require that we use a LWDAQ Driver as an intermediary between ethernet and the receiver. We specify an internet protocol (IP ) sddress with daq_ip_addr. We specify a driver socket number, if required, with daq_driver_socket. When we press Reset and Configure and the Receiver Instrument connects to the data receiver, determines its type and firmware version number. The receiver type allows the Receiver Instrument to set the payload automatically in payload_length. The payload, if it exists, will carry additional information that is synchronous with sample value and associated with the same channel number.

Example: The Animal Location Tracker (A3038) payload is sixteen bytes. The channel number and sample value are generated by an SCT over the tracker platform. The first fifteen payload bytes are the microwave power received from the SCT during its transmission by each of fifteen antenna coils arranged in a grid. The sixteenth byte is the power received by an auxilliary detector that may or may not be connected to an external antenna.

The Receiver Instrument plots the signals it downloads, but does not attempt to interpret the payload bytes. The sixteen-bit samples always represent a time-varying quantity, but the payload can represent any kind of information. The plot can be of three forms: simple, centered, and normalized, as we describe below.


Figure: Receiver Instrument Panel on MacOS. Radio buttons and entry boxes control data acquisition and display. Here we have the simple plot (SP) of the signals we obtain from several SCTs on a bench top while we tap the bench to get their wires moving.

The Reset and Configure button resets the data receiver's data buffers and message clock. It reads a segment of data from the receiver, identifies the receiver type by examining this segment, and configures the Receiver Instrument accordingly. When we execute the reset, one or more lights should flash briefly on our data receiver, depending upon its type.

The data receiver keeps track of time by inserting clock messages into the message stream. The clock messages are those with channel number zero. In place of a signal value, they contain a sixteen-bit count of the clock message number. All existing data receivers insert one clock message every 7.8125 ms, so the clock message frequency is 128 Hz. The clock message number starts at zero after reset, increments by one every 7.8125 s, reaches 65536 in 511.9921875 s, and returns to zero after exactly 512 s. The sixteen-bit clock message value, combined with a signal message's own eight-bit timestamp, allows us to determine the time of a sample with precision of 30 μs.

The Receiver Instrument acquires data in discrete blocks. It assembles these blocks in memory until it can compose an interval with exactly daq_num_clocks clock messages. With daq_num_clocks set to 128 and a data receiver with a clock rate of 128-Hz, the interval will be exactly one second long. As soon as the Receiver Instrument has a complete interval, it plots the interval's signals in accordance with the plot controls and analysis_channels string.

The Acquire button obtains one interval of data from the data receiver. The Loop button obtains repeated intervals until you press Stop. The Activity line lists all active channels, giving channel numbers and number of messages from each channel. Channel zero should always be present, with a number of messages equal to daq_num_clocks. Other channels will be listed so long as they contain more than active_threshold messages. The active_threshold parameter is available in the Info panel.

With analysis_enable = 1, the Receiver Instrument analyses each interval it acquires, plots the signals and generates a result string. With the same parameter set to zero, we see a gray-scale image of the message bytes, no plotting takes place, and no result string is generated. You can see an example of a result string in the screen shot above. To understand the meaning of numbers in the result string, set verbose_result to 1, and acquire an interval. You will get something like this:

Receiver_1
Channel Number: 1
Number of Samples Recorded: 512
Average Value: 43758.6
Standard Deviation: 117.9

The data receiver receives messages from up to 255 channels. Our subcutaneous transmitters use channel numbers 1-223 for their signals, with the exception of those that have remainder zero or fifteen when divided by sixteen. Those with remainder fifteen we have reserved for auxilliary, slow data. Those with remainder zero we have reserved for no particular purpose. We control which channels the Receiver Instrument displays with analysis_channels . By default, this string is the wildcard character, "*", which selects all channels for which there are at least activity_threshold messages in the interval. Each channel receives four numbers in the result string. The first is the channel number, the second is the number of messages received from this channel, the third is the average value of the message samples, and the fourth is the standard deviation of the sample values. The numbers provided for clock messages are slightly different. Instead of the average and standard deviation, the Receiver Instrument provides the minimum and maximum clock values.

Receiver Messages

The messages in the data receiver's message buffer are each four bytes long followed by a payload of dependent upon the data receiver. The bytes of each message are listed in the table below. The Channel Number is used to identify the source of the message. Channel number zero is reserved for clock messages. Following the channel number, each message contains a sixteen-bit data word. In the case of SCTs, the sixteen-bit data word is a digitized voltage. The last byte of the message is a timestamp.

ByteContents
0Channel Number
1Most Significant Data Byte
2Least Significant Data Byte
3Timestamp or Version Number
4 to (L+3)Payload Bytes
Table: Contents of One Message. Each message consists of four bytes followed by a payload. Here the number of payload bytes is L.

All data receivers have a clock. The clock is a twenty-four bit counter that counts clock ticks. The standard data receiver clock runs at 32.768 kHz, and is provided by a ±1 ppm 32.768 kHz oscillator. The bottom eight bits of the counter provide the timestamp of individual non-clock messages. Every time the timestamp wraps around to zero again, which is every 256 ticks of the clock, the data receiver stores a clock message in its message buffer. The timestamp of a clock message would always be zero, so the data receiver stores a version number in the timestamp of clock messages, and this version number allows the Receiver Instrument to figure out what kind of data receiver it is dealing with, and set its payload length. The upper sixteen bits of the clock message counter increment every time the data receiver stores a clock message, and reset to zero every 512 s.

Note: The data receiver will never store a message with channel number zero unless that message comes from the clock. All messages with channel number zero are guaranteed to be clocks.

The Receiver Instrument supports the use of auxiliary channels, which transmitting devices can use for acknowledgements or slow data. All channel numbers which, when divided by sixteen, have remainder fifteen, are auxiliary channel numbers. They are channel numbers 15, 31, 47, 63, 79, 95, 111, 127, 143, 159, 175, 191, 207, and 223. The aux_list_name parameter is the name of the string that contains a list of auxiliary messages. This list will include only the most recent aux_num_keep messages. Other processes waiting for data or acknowledgement can search the auxiliary message list for the messages they need. When they find such a message, they delete it from the list.

Each auxiliary message contains sixteen bits of data. The Receiver Instrument assumes these sixteen bits are arranged as a four-bit source identifier, a four-bit field address, and an eight-bit data byte. The four-bit source identifiers we combine with the top four bits of the auxiliary channel number to obtain the channel number of the transmitter that produced the auxiliary message. The four-bit field addresses 0 and 15 are reserved. The others may be used to identify instructions, registers, or a device group. The data has no constraints upon its value, and can be used for any purpose. Each entry in the auxiliary message list contains four unsigned integers: channel number, field address, data, and timestamp. The timestamp is the Data Receiver time at which the auxiliary message was received. Its units are clock ticks. The timestamp allows other processes to use the precise timing of auxiliary messages as a form of data addressing. The Receiver Instrument provides the timestamp modulo 216, which is sufficient for any timestamp-based addressing between two clocks running within ±20 ppm of one another.

Example: An Implantable Stimulator-Transponder (A3036) with data transmit channel number 68 uses channel number 71 to send command acknowledgments. The identifier bits are the lower four bits of the IST data transmit channel. The data byte is a value passed to the IST by the acknowledgement request. Once the Stimulator Tool has detected the acknowledgement in the Receiver Instrument's auxilliary message list, it deletes the acknowledgement from the list.

The following table gives the existing definitions of field address values and the meaning of the data that follows them.

Field
Address
Field
Function
Data
Function
0ReservedUndefined
1AcknowledgementAcknowledgement Key
2SynchronizeDevice Timestamp
3StatusFlags for Device Status
4Time-Addressed DataData Byte
5..14UndefinedUndefined
15ReservedUndefined
Table: Defined Field Addresse and Data Functions.

The time-addressed data is data transmitted with a particular timing pattern that allows the receiving process to figure out which byte is which. Such a scheme might be as simple as transmitting the high and low byte of a sixteen-bit temperature measurement on consecutive clock ticks, or it might be as complex as a pseudo-random sequence of time delays that carries thousands of unique bytes of data over a period of several minutes. In all cases, the intent is to allow slow data to pass from transmitter to receiver without interrupting the continuous flow of faster data, and in such a way as to tolerate loss of some of the data. Thus we use timing as a way to detect the loss of a message, which allows the receiver to request that missing bytes be transmitted again.

Data Acquisition

To acquire fresh data from a data receiver, you must set up a LWDAQ (Long-Wire Data Acquisition) system with a data receiver, and make it accessible to your computer. We describe how to set up an LWDAQ in the LWDAQ User Manual. Once you have a LWDAQ available, consult the manual of your data receiver for information on how to connect it to the LWDAQ. Chances are, all you have to do is plug it into one of the LWDAQ Driver sockets with a CAT-5 patch cable. The Receiver Instrument provides a set of lebeled text entry boxes on the right side of its window. Among these, you will see several that begin with daq_. We describe each of these in the table below. You change the value of the named parameters by entering new values in the text entry boxes.

ParameterFunction
daq_ip_addrIP address of LWDAQ driver
daq_driver_socketdriver socket number used for data receiver
daq_mux_socketmultiplexer socket used for data receiver (if any)
daq_num_clocksnumber of clock messages to accumulate per acquisition
payload_lengthnumber of payload bytes following message core
daq_channelsa list of channels for the receiver to select
Table: Receiver Instrument Data Acquisition Parameters. You are most likely connecting your data receiver directly to the driver with a cable, so you won't have to bother with daq_mux_socket at all.

When you press the Acquire button, the Receiver Instrument obtains a new interval of messages from the data receiver. In LWDAQ terminology, the Receiver Instrument acquires an image. This image contains exactly the number of clock messages you specify in daq_num_clocks. If you specify n, the first message in the image is clock message 0 the last message is the one that preceeds the n+1'th clock message. The next image the Receiver Instrument acquires will begin with this same n+1'th clock message, but in the next image this clock message will be clock message 0. The time it takes the data receiver to generate daq_num_clocks clock messages is the image time. The image contains all messages stored by the data receiver during the image time.

In order to assemble images with a specified number of clock messages, the Receiver Instrument reads blocks of messages out of the data receiver and adds them to its own message buffer. When the message buffer contains enough clock messages, the Receiver Instrument extracts the desired interval from the message buffer and stores these messages in the new image. It leaves unused messages in the message buffer for inclusion in the next image. No messages are lost unless you leave the data receiver too long and it's message buffer overflows, in which case the oldest messages are lost because they are over-written by new messages. But the Receiver Instrument does not throw any data away unless you press the Reset button. When you press the Reset button, the Receiver Instrument clears its message buffer and clears the data receiver message buffer as well.

Example: We have an Octal Data Receiver (A3027) connected to a LWDAQ driver. Its eight Loop Antennas (A3015C) are receiving signals from four Subcutaneous Transmitters (A3028B). Each SCT transmits 512 messages per second and occupies one channel number. The channel number is given by the second part of the SCT serial number, visible through its silicone encapsulation. The data receiver stores 128 clock messages per second and all signal messages it receives. When we set daq_num_clocks to 128, the image period is one second. In this case, the image contains 128 clock messages. Assuming perfect reception, there will be 2048 transmitter messages. In practice, there will be fewer, and there may also be more, because the duplicate elimination the A3027 applies to its eight independent receivers is imperfect. The combined size of the ideal interval is 8704 bytes. We turn off all the transmitters and acquire again. We see the transmitter traces. We acquire again. They are still there. We are reading data that is several seconds, or even several minutes, old. We press the Reset and Configure button and acquire again. Now we see only a red trace. This red trace is the clock message trace. It has a slight slope to it.

Suppose your data receiver is recording messages faster than the Receiver Instrument can read them out. Eventually, you will start to lose messages. The data receiver's message buffer will overflow. When the buffer overflows, the data receiver will over-write the oldest messages with the newest messages. The Receiver Instrument will recognise that the latest block of messages it downloads from the data receiver does not match up with its message buffer. When the Receiver Instrument detects this or any other kind of error in the data it receives, it resets the data receiver's buffer and message clock.

Example: We turn on our four transmitters. Our Data Receiver (A3027) stores 8704 bytes per second. But our download speed drops to 4000 bytes per second because of some problem. After a hundred seconds, the receiver's 512 KB buffer is full. The buffer overflows. The Receiver Instrument detects a jump in the clock messages of the interval it is trying to compose, discards the data it has not yet analyzed, and resets the data receiver's buffer and clock.

Set analysis_enable to zero and acquire again. The message traces disappear. All you see is a black image with some of the top rows filled in with random pixels. These pixels are your messages. They usually occupy only a small fraction of the display area, although they can overflow the image area when we are recording from a location tracker. Press the Info button. You will see another window with many more Receiver Instrument parameters (screen shot). Among them are daq_image_height and daq_image_width. These determine the size of the image that holds our messages. By default they are 300 and 400. The first row of the image we use for a header, so that leaves 119,600 bytes for messages, which is space for 29,000 messages.

Example: With four transmitters active, each transmitting 512 messages per second, we we obtain 2048 transmitter messages and 128 clock messages, making a total of 2176 messages of four bytes each, or 8704 bytes. The A3018's message buffer is 512 KBytes, so it will hold up to one minute of data for us. Because of this ample buffer in the data receiver, our instantaneous TCP/IP download speed from the LWDAQ driver can drop to zero for ten or twenty seconds, and we will not lose any data. So long as our average download rate is around 8 Kbytes/s (64 kbits/s) or greater, we will sustain the data acquisition without loss. We use the Neuroarchiver to store the dta to disk, we will accumulate roughly 8 Kbytes/s, or 30 Mbytes/hr.

When you save a Receiver Instrument image to disk with the Write button, we recommend you give it the extension .daq. That way, LWDAQ will store the image in its DAQ format. The black part of the image following the actual messages will be left off the file. Only the header and the actual messages will be stored. But the Neuroarchiver, uses the NDF format to store messages. As it acquires new messages, it appends them to the data section of an existing NDF file. When you read Receiver Instrument images from disk, the Receiver Instrument displays them as if they were freshly-acquired. But it changes the image_source parameter to "file". You will find the meaning of the image_source parameter and its values described here. Change image_source back to "daq" to acquire fresh data from the data receiver.

Receiver Script

If you press the Info button in the Receiver Panel, the Info Window opens with an array of Receiver parameters and their current values. These parameters are the elements of the LWDAQ_info_Receiver array. The Receiver uses these parameters to control data acquisition, analysis, and its own windows. Feel free to change them. The worst thing that can happen is the program freezes and you have to re-start it.

Press the Script button in the Info window and a text window will open up showing you the TCL script that defines the Receiver Instrument functions. All LWDAQ instruments define themselves with a few standard routines. In this case the standard routines are LWDAQ_daq_Receiver, and LWDAQ_analysis_Receiver. The former acquires a new image from the data acquisition system, the latter applies Receiver Instrument analysis to an image. The additional routin LWDAQ_create_controls_Receiver defines the Receiver Instrument's custom controls. These are the display entries and the reset button. The function of the reset button is defined by LWDAQ_reset_Receiver.

If you want to learn how, in detail, the Receiver Instrument communicates with the data receiver device, look at the script for LWDAQ_daq_Receiver. You will see lines that open a socket to the LWDAQ driver over TCPIP, write to the command word on the data receiver, and download blocks of data. The basic act of downloading data from the data receiver into the LWDAQ driver's RAM, and then from the driver RAM to the LWDAQ software, is straightforward. What complicates the Receiver acquisition is the buffering required to parse the data receiver message buffer contents into image periods, and to do so with minimal delay.

Message Display

The Receiver Instrument is a generic downloader and sorter of sixteen-bit values stored in a generic data receiver. Nowhere in the Receiver Instruments parameters will you see any mention of the data receiver clock speed. Nor will you see any mention of the relationship between the sixteen-bit samples and any physical quantities. The sixteen-bit samples originate from some sort of sensor or transmitter. Consult the sensor or transmitter manual for the physical meaning of the sample values.

Example: The Subcutaneous Transmitter (A3028B) provides one analog input at 512 SPS. The gain in the analog input amplifiers is ×100, and the input range of its 16-bit ADC is 0 V to 2.7 V. Each ADC count is 2.7 / 65536 / 100 = 412 nV. For more details see the Analog Inputs of the A3028 manual.

There are three ways to plot the voltage in the Receiver Instrument window. The simple plot we select with the SP button. The range sets the range of the plot from bottom to top in ADC counts. The offset sets the voltage at the bottom of the display. The centered plot uses range in the same way, but ignores the offset. The plot of each signal is centered upon the window, so that the average value of the signal is exactly half-way up. The normalized plot ignores both range and offset and fits the signal exactly into the height of the display. The horizontal axis is time. The full range from left to right covers the recording interval. No matter what you do with the display, the data in the image is unaffected.

The message display uses a different color to display each channel. With analysis_channels set to its default value of * (which means "all messages"), the Receiver Instrument displays all messages in the current image (as we say in the previous section: in LWDAQ terminology, a block of messages assembled and returned by the Recoder Instrument is an image). The message display is the product of analysis of the image. Whether you display all the channels or not, they remain in the image. The Receiver Instrument will not delete them from the image.

Each channel number has its own color. The colors are unique, but it's unlikely that you could tell them apart if you had messages from all 256 possible channel numbers displayed at the same time. Subcutaneous transmitters us only 196 of the 256 available channels for their data transmissions.


Figure: Channel Color Numbers Used By The Receiver Instrument.

The horizontal axis of the display covers the image time. The Receiver Instrument does not know how to translate daq_num_clocks into image seconds, but it does know how to translate the clock messages and message time stamps into time in units of data receiver clock ticks. So it plots the message values against the time, in clock ticks, at which the data receiver stored the message. If there are missing messages, you will see a straight line joining the messages on either side of the missing messages.

Channel Selection

If you have many active data channels, we may wish to isolate one or more of them for display by the Receiver Instrument. Channel selection is supported by the Animal Location Tracker (A3038), but not by the Octal Data Receiver (A3027) or Data Receiver (A3018). Enter "*" to selecct all channels, "0" to select none, or list the channels you want to record a string of numbers seprated by spaces. When we Reset and Configure the Receiver Instrument sends the commands that configure channel selection to the data receiver. The selection will either be ignored or honored. For data receivers that ignore channel selection, the Receiver Instrument sets them up to receive from all channels.

A second stage of selection takes place in the Recoder Instrument, by means of the analysis_channels string. If this is set to *, the Receiver Instrument analyzes and displays all channels in the image. If it is set to a list of channel numbers separated by spaces, only these channels are analyzed and displayed.

The result string returned by the Receiver Instrument changes with the value of analysis_channels. When you select all channels with *, every channel with one or more messages is listed in the line. When you select a specific subset of channels, only these channels, in order of ascending channel number, appear in the result string. At the end of this specific list of channels, the Receiver Instrument adds an entry for channel number −1, which is "all other non-zero channels", and gives you the number of messages in these other channels.

Message Analysis

By default, the Receiver Instrument analyzes every new data image it acquires. The analysis consists of plotting, listing, and summarizing the acquired data. You can turn off analysis of the image by setting analysis_enable to zero. With the analysis turned off, you will see the bytes of the messages drawn as individual pixels in the top part of the image. When you enable the analysis, you get the message value plots. The analysis does not change the image data.

The Receiver Instrument's image analysis routine is called LWDAQ_analysis_Receiver. This routine calls lwdaq_receiver, which in turn calls lwdaq_A3007_receiver. The lwdaq_sct_receiver routine is defined in a Pascal source filed called electronics.pas. If you want to learn about the analysis in detail, as a guide to writing your own reconstruction program, take a look at the Pascal file, where the algorithm is explained in detail in the comments.

The lwdaq_receiver command is available at the LWDAQ console. Its arguments are an image name and a command string. The command string begins with an instruction. The following table lists the lwdaq_receiver instructions.

InstructionFunctionUsed By Receiver
listLists active channels in an image.Yes
printPrints message contents and lists errors.Yes
plotPlots messages in the image overlay and returns summary result.Yes
clocksCounts clock messages and locates specific clock messages.Yes
extractExtracts messages from a single channel.No
reconstructElminates bad messages and adds substitute messages.No
purgeRemoves duplicate messages.Yes
Table: Receiver Analysis Instructions.

The purge instruction is the only instruction that alters the data downloaded from the data receiver. It eliminates duplicate messages, whether they are consecutive in the data or not. All other instructions leave the data in unaffected.

The Receiver uses the plot instruction to plot the channels in the image overlay and to generate the summary string. It uses the list instruction to obtain a list of active channels. Internally, as it is accumulating an image with the correct number of clock messages, the Receiver uses the clocks instruction to determine how many clock messages are in a new packet of data, and where the first and last clock messages are in the data. When the Receiver encounters errors in acquired data, it uses the plot instruction to obtain a list of these errors and their locations within the data, so that it can attempt to remove the errors and correct mis-alignment of message boundaries in the data receiver. Such errors occur during rare electrical events such as static discharge on the data acquisition hardware or power interruptions. You can see for yourself how the Receiver uses the lwdaq_receiver instructions in the Receiver Instrument source code.

You can apply the lwdaq_receiver instructions yourself, using the console and the name of the current Receiver image. After you acquire a Receiver image, look at the value of memory_name. Suppose it's Receiver_1. Go to the File Menu and open the console. Enter the following at the console prompt.

(LWDAQ) 1 % lwdaq_receiver Receiver_1 "clocks"
0 128 3166 

The result we show is the one we get when we apply a clock instruction to this image. There are 0 errors in the clock sequence in this image, there are 128 clock messages, and 3166 messages in total. Each message in an image has a message index, which is its position in the sequence of all messages. The first message has message index zero (0). Each message also has a channel index, which is its position in the sequence of messages in its channel. The following command returns the message index of each of three clock messages. The command specifies the clock messages by channel index.

(LWDAQ) 2 % lwdaq_receiver Receiver_1 "clocks 0 64 128"
0 128 3166 0 1583 -1  

Clock message zero also has message index zero. The first message in the image is a clock message. Clock message 64 has message index 3166. There is no clock message 128, so the command returns −1 to indicate the absence of the specified clock message.

The print instruction instructs lwdaq_receiver to print out the messages for us, as shown below. Each message gets a separate line. On each line is the message index (the 0'th message is the first one in the image), the channel number, the data value, and the timestamp. After that comes the hexadecimal representation of the four bytes that make up the message in the block of data we acquired from the data receiver.

(LWDAQ) 3 % lwdaq_receiver Receiver_1 "print 0 10"
Data Receiver Version 5.
Total of 3166 messages and 128 clocks.
Encountered 0 errors in data block.
Messages 0 to 10 (index id value timestamp $hex):
    0   0  1281   5 $00050105
    1   5 42860   8 $05A76C08
    2   6 40972  18 $06A00C12
    3   9 40654  22 $099ECE16
    4   3 30275  33 $03764321
    5   7 37119  37 $0790FF25
    6   4 46759  60 $04B6A73C
    7   5 43183  72 $05A8AF48
    8   6 41065  73 $06A06949
    9   9 40063  87 $099C7F57
   10   3 30456  97 $0376F861
End of Messages

Here we see the print instruction used to print out the first eleven messages. If we don't specify the message range, the print instruction will print the contents of all the messages. The first message is a clock message. It's channel number is zero and its timestamp is 5. The timestamp of a clock message gives us the data receiver version, which is in this case 5, meaning an A3018 with firmware 5. In the following print command, we see the messages and payload bytes produced by an Animal Location Tracker (ALT, A3032C, with firmware 9. The ALT produes sixteen payload bytes. In clock messages, the timestamp is the ALT version and the last payload byte is the data receiver version. Here, the ALT version is 69, meaning A3032C with firmware 9, and the data receiver version is 43, meaning A3037E with firmware 13. In all other messages, the payload consists of fifteen power measurements from the ALT detector coils and a reserved byte, which this version of the ALT sets to zero.

Data Receiver Version 69.
Total 1132 messages, 128 clocks, 0 errors, and 0 null messages.
Messages 0 to 10 (index id value timestamp $hex):
    0   0 33266  69 $0081F245 1414141414141414141414141414142B
    1  39 40457   6 $279E0906 5C39656B7B681B737F5560645E676A00
    2 230 43263  64 $E6A8FF40 3E23464B432D597559374A4A51564000
    3  39 40440  67 $279DF843 5E38666C7D6920768156636860686E00
    4 230 43211 134 $E6A8CB86 4029493B422D5B7B573B4B4951584300
    5  39 40463 139 $279E0F8B 5834657179663E687F4E5B6659656300
    6 230 43221 188 $E6A8D5BC 3E22465043295771582C494C4F534100
    7  39 40463 198 $279E0FC6 57356470796543677E4E5B6657646300
    8   0 33267  69 $0081F345 1414141414141414141414141414142B
    9 230 43255   1 $E6A8F701 3F244853432858735A26494E4F543F00
   10  39 40458   6 $279E0A06 5D39656A7C681774805661655E676C00
End of Messages

If you want to extract messages from a single channel, and get them in a format that's easy for plotting on the screen, or pasting into a spreadsheet, use the extract instruction. Here is an example, obtained from our Receiver_hum.daq data in image Receiver_1. We use the ellipsis ("...") to indicate lines we have cut from the console output.

(LWDAQ) 4 % lwdaq_receiver Receiver_1 "extract 1"
18 40972
73 41065
135 40923
208 40652
265 40586
...
32525 44508
32593 44287
32658 44226
32708 44208

The extract instruction takes one parameter: the channel number you want to extract from the image data. The extraction result does not contain the channel number. It returns the time and value of each message on a separate line. The time is in clock ticks since the start of the image interval. You can cut and paste the extraction result into a spreadsheet and plot the values that way, or you can use LWDAQ library routines to plot the same data in a LWDAQ window.

Each LWDAQ image provides an overlay in which you can draw graphs. The following lines use an extract instruction to obtain channel five (5), plot it in the overlay using lwdaq_graph, and refresh the display of Receiver_1 in the Receiver Instrument window using lwdaq_draw.

(LWDAQ) 5 % lwdaq_graph [lwdaq_receiver Receiver_1 "extract 5"] Receiver_1 -fill 1
(LWDAQ) 6 % lwdaq_draw Receiver_1 receiver_photo

The plot instruction plots messages in an image's overlay in the same as lwdaq_graph. Here we tell lwdaq_receiver to plot messages from channel three (3) and five (5) in the overlay.

(LWDAQ) 7 % lwdaq_receiver Receiver_1 "plot 0 65535 DC 3 5"
3 511 42461.0 3256.6 5 482 43415.2 522.4 -1 2045
(LWDAQ) 8 % lwdaq_draw Receiver_1 receiver_photo

The "0 65535" means that the display should show data values in the range 0 to 65535. The "DC" means that the coupling for the display should be direct. With "AC", the analysis will subtract the average of each channel's data value before it plots. The "3 5" means plot channels 3 and 5. If you say "*" instead of giving channel numbers, lwdaq_receiver will plot all available channels. In the results string we see that channel 3 provided 511 samples with average value 42461.0 and standard deviation 3256.6. Channel −1 is for "all other channels except channel zero". Channel zero (0) is, of course, the clock message channel. There were 2045 samples in the other channels.

The reconstruct instruction attempts to eliminate bad messages from the message stream and replace missing messages with substitute messages. The value of a substitute message will be the value of the previous message in the message stream. The reconstruct instruction's task is complicated by the fact that most sources of messages, such as the Subcutaneous Transmitter (A3028A) displace the instant of each individual message transmission by a random time. We call this random displacement transmission scatter. Transmission scatter makes it possible for two message sources with the same period to share the same transmission medium without corrupting a significant fraction of one another's messages. We discuss transmisison scatter in more detail in Collisions.

Example: Two Subcutaneous Transmitters (A3013A) share the same 900-MHz ISM radio-frequency band. Each A3013A transmission takes 7 μs. A 32.768 kHz clock runs in the transmitter. Every 64 clock ticks, the A3013A prepares to transmit a message. But it delays transmission by 0 to 15 clock ticks at random, which is what we call transmission scatter. The A3013A uses the lower four bits coming from its digital to analog converter as the random number. These bits being dominated by random noise, they make a good random number generator. When two A3013As drift into synchronicity, so that the 64-tick sampling periods begin within a few microseconds of one another, transmission scatter guarantees that only one in sixteen messages will be collide.

With transmission scatter, each message transmission occurs at some random time within a transmission window centered upon a nominal transmission time. The nominal transmission times, and therefore the transmission windows, are spaced by an exact and well-known transmission period.

A reconstruct instruction requires that we specify one message stream for reconstruction by giving its channel number. We specify the period of this message stream. The reconstruction deduces the transmission scatter from the period. The only units of time used by lwdaq_receiver instructions are data-receiver clock ticks. So we specify the period in units of clock ticks.

Example: A Subcutaneous Transmitter (A3013A) transmissions occur somewhere within 480 μs of the rising edge of its 512-Hz message clock. The distribution of the transmission time is random and uniform. The nominal transmission time is the center of this distribution, and is therefore 240 μs after the rising edge of the message clock. The transmission scatter is ±240 μs on either side of the nominal transmission time. The Data Receiver (A3018A) that receives our A3013A messages has a reference clock of 32.768 kHz. Each clock tick is 30 μs. The A3013A transmission scatter is therefore &plusm;240 / 30 = ±8 clock ticks. The A3013A's transmission frequency is 512 SPS, so its period is 64 clock ticks. The time between two consecutive messages could be as little as 48 ticks and as great as 80 ticks, or 1.5 ms to 2.4 ms. You can see transmission scatter between consecutive A3013A messages here.

Here is an example reconstruct command, applied to Receiver_hum.daq example image in Receiver_1.

(LWDAQ) 1 % lwdaq_receiver Receiver_1 "reconstruct 1 64 30000"
-7 30000
59 41175
121 44466
183 47057
246 46943
317 44541
380 40901
432 38665
498 39453
...
32693 38335
(LWDAQ) 2 % 

We see the reconstruction of channel 1 with period 64 ticks. The third number in the command is the standing value, which we set to 30000. The standing value of the channel is the value we want the reconstruction to use if it determines that there is a missing message at the beginning of the image's message stream. In this case, the reconstruction finds that there is a transmission window overlapping the beginning of the image's time interval. The center of this transmission window is −7 clock ticks before the beginning of the images's time interval, and therefore overlaps this time interval because the transmission scatter is ±8 clock ticks. The reconstructed message stream contains a message at time −7 with value equal to the standing value we specified.

A reconstruct instruction allows us to apply reconstruction to a continuous message stream. Each time we call reconstruct we apply it to one image only, but we pass information to it from previous images using the command string. The standing value is the value of the final message from the previous image. The final message value in our example reconstruct result is 38335.

Suppose the reconstruction gets to the end of a message stream in an image and finds that the final transmission window overlaps the end of the image time. This happens in our example above. The final transmission window is centered at time 32761 clock ticks. Our extraction returned a final message "32765 38205", which lies in this window. You will note that our reconstruct instruction did not return this final message. It did not return the message because it cannot be confident of the status of the message. What if it is a bad message, and the valid message for that transmission window occurs at the beginning of the next image time interval? In the general case, there can be one or more messages within the final transmission window, and the reconstruction must somehow pass these messages to the next reconstruct instruction that will be applied to the next image in the message stream.

The extract and reconstruct instructions both use the image results string to store meta-data, such as a list of unaccepted sample values. The image results string is part of the image data block, as described here. The extract instruction sets the image results string to the number of clocks in the image followed by the number of messages extracted from the selected channel. The reconstruction writes four or more numbers to the image results string.

(LWDAQ) 2 % lwdaq_image_results Receiver_1
128 512 4 5 38205 

The lwdaq_image_results command returns an image's results string. We obtain the above result if we call lwdaq_image_results immediately after our execution of a reconstruc instruction. The first number is the number of clocks in the image. Then comes the number of messages in the reconstructed message stream. The third number is the number of messages the reconstruction rejected as bad messages. In this case, it rejected 4. The fourth number is the number of missing messages for which the reconstruction generated substitutes. In this case, it inserted five substitute messages. The final number is the value of the unaccepted message we discussed above. If there were several such unaccepted messages, the image results string would contain their values one after another. In this case, we have only one value, 38205.

We pass these unaccepted values to the next reconstruction simply by appending them to the end of the command string, like so:

(LWDAQ) 1 % lwdaq_receiver Receiver_2 "reconstruct 1 64 38335 38205"

Here we assume that the name of the next image, one it has been acquired, is Receiver_2. We specify 38335 for the standing value, because that was the value of the last message returned by reconstruction when it operated upon Receiver_1. At the end we give the single unaccepted value.

In our first reconstruct instruction, we did not pass the correct standing value, nor any unaccepted values. That's why we ended up with the "−7 30000" sample, which cannot be real. As you can see, reconstruction does recreate a message stream, but there may be one more or one less messages in each section of the message stream it returns.

The extract and reconstruct instructions create a list of samples and their payloads for subsequent use by routines such as lwdaq_locator command. The locator command takes the payload and extracts from it coil power measurements which it uses to measure the location of a subcutaneous transmitter on a tracker platform.

Error Handling

It is possible to receive corrupted data from a data receiver. The most common cause of such corruption is static discharge, electrical power glitches, and interruption of TCPIP communication between the data acquisition computer and the receiver. The corruption usually takes the form of extra bytes inserted into the data, but may involve missing bytes as well. Another form of corruption occurs when the Receiver Instrument cannot keep up with the data produced by the data receiver, such as when the network connection is slow. The message buffer on the data receiver overflows and the Receiver Instrument sees a block of data go missing.

When the Receiver Instrument detects an error in the data it has downloaded, it makes no attempt to correct the error or recover the data the block might contain. The best way to handle even the slightest error is to reset the data receiver and try again. When we reset the data receiver, we reset the clock value, which confuses the Neuroarchiver's calculation of time within an archive. In order to nagivate correctly through an archive with corruption, the Neuroarchiver provides the sequential navigation option, which is slow but sure. In the Neuroarchiver we can use the Receiver Instrument's analysis routines to display errors in the messages recorded to disk, as they are after error-correction.

The show_errors and show_error_extent parameters instruct the Receiver Instrument to print out a table of raw messages when it encounters an error. If show_errors is set, the Receiver will print out show_error_extent messages before and after the first error encountered in a block of data downloaded from the receiver. The print-out will include a description of the error, and the corruption of the data will be visible in the hexadecimal printout of the raw message contents.

Data Processing

The Neuroarchiver Tool uses the Receiver Instrument to obtain data from a data receiver. The Neuroarchiver uses the lwdaq_receiver routine's the reconstruct instruction to clean up and patch up the message stream received from one or more sources. It stores raw data, reconstructed data, and fourier-transformed data to disk. The Neuroarchiver is intended for use with Subcutaneous Transmitters and Data Receivers, as described here. Other data receiver systems will require their own tools. If you plan to write such a tool, start by looking over the Neuroarchiver.tcl source code, which you will find in the Tools directory of the latest LWDAQ distribution.