Neuroplayer Analysis Library
[15-JUL-25] This is a library of interval analysis programs. These programs operate upon characteristics lines produced by Neuroplayer interval processing. We run the analysis programs within LWDAQ, but we do not run them directly with the Neuroplayer. Most of them run in the LWDAQ Toolmaker. We paste the script into the Toolmaker window and press Execute to run them. The objective of interval analysis is to produce data worth of a plot, or a table, so as to produce a conclusion or demonstration.
The progression from telemetry reception to the completion of interval analysis is as follows. We begin with telemetry signals recorded to disk by the Neurorecorder. The disk file will be a Neuroscience Data Format (NDF) archive. We read the NDF archive with the Neuroplayer. The Neuroplayer reads the signals from disk in discrete, non-overlapping playback intervals. For each playback interval, the Neuroplayer applies interval processing to each channels specified in its channel select string. This interval processing produces some kind of output, such as a string of numbers, and the Neuroplayer writes these to disk. Interval processing applies itself to these strings of numbers, for which there is one string per playback interval, with the objective of producing summarizing results.
Standard processors produce one line of text for each playback interval. We call this the characteristics line. The standard characteristics line has a fixed format. It begins with the NDF file name and the interval start time within the archive. The first interval has time zero. After that, each processed channel will be represented by its channel number followed by one or more characteristics. A characteristic is either a word or a real-valued numbers, but never an integer. The only integers in the result string are the channel numbers. In order to support interval analysis, the Neuroplayer writes the characteristics lines to a characteristics file. Most of our interval analysis programs operate upon characteristics files in which each line is a standard characteristics line. Some, however, operate upon non-standard characteristics files.
[15-JUL-25] This interval analysis program produces a histogram of a real-valued characteristic. It assumes that it has standard characteristics files to operate upon, such as those that are produced by event classification processors. This particular configuration of the histogram program produces a histogram of the spikiness metric produced by the ECP20V4R1.tcl event classification processor. This processor adds nine elements to the characteristics line for each channel. These are channel number, interval classification letter code, power, coastline, intermittency, coherence, asymmetry, and spikiness. The metric histogram program is a Toolmaker script, so cut and paste it into the Toolmaker panel and press Execute to run it.
set num_params 9
set metric_index 8
set fnl [LWDAQ_get_file_name 1]
set histogram [list]
for {set i 0} {$i <= 100} {incr i} {
lappend histogram 0
}
foreach fn $fnl {
set f [open $fn r]
set count 0
while {[gets $f line] > 0} {
incr count
set line [lrange $line 2 end]
while {[llength $line] > 0} {
set metric [expr round(100*[lindex $line $metric_index])]
lset histogram $metric [expr [lindex $histogram $metric]+1]
set line [lrange $line $num_params end]
}
}
close $f
LWDAQ_print $t "Read $count lines from [file tail $fn]."
}
for {set i 0} {$i < 100} {incr i} {
LWDAQ_print $t "[format %.2f [expr 0.01*$i]] [lindex $histogram $i]"
}
[15-JUL-25] This analysis program calculates the average power in a sequence of frequency bands during consecutive intervals in time. You specify the length of these intervals in the script, in units of seconds. The averaging interval can be minutes, hours, or days. Before running this analysis program, we must generate power band characteristics files using the Neuroplayer with a power band processor such as Consecutive Power Bands or Arbitrary Power Bands. This analysis program is available for download as PBAV4.tcl. It can be adapted to calculate the average of any characteristic produced by a standard interval processor. We could apply it to the characteristics generated by an event classification processor to obtain the average value of metrics in one-minute intervals. This program is similar to our Reception Average analysis program, but this program supports simultaneous averaging of multiple characteristics from the same signal channel. Run the program in the Toolmaker. A file browser window will open up. Select your characteristics files in the file browser. Cut and paste from the window to plot in Excel.
# Power Band Average, Version 4 [04-FEB-20].
# A Neuroplayer Analysis Script, for execution in the LWDAQ Toolmaker.
#
# This analysis produces a graph of average power in various frequency bands with
# a sampling period we define in seconds. As input, the analysis takes a series
# of characteristics files that we select with a file browser. Each line in these
# files must be of the following format.
#
# archivename archivetime id band_1 band_2 ...band_n id...
#
# This analysis supports any number of power bands, provided that each channel
# has the same number of bands. The playback interval in all characteristics files
# must be constant if the analysis is to be correct in its calculation of average
# band power. The analysis deduces the playback interval by looking at the first
# two lines in the first file.
#
# The output of this script is written to a file at the location given by
# ofl. Each line in the file is the average power in all bands for the selected
# channels, calculated over the averaging interval. When the power in the
# characteristics line is exactly "0.00", the line is ignored, because this
# suggests that there was no data for the transmitter in the interval.
#
# During execution, each line in the execution window is an index of the
# averaging interval just written to the output file.
#
# The calculatiom_type parameter instructs the program to calculate the
# average, median, or maximum power for values 0, 1, and 2. The band_offset
# allows us to skip over some leading characteristics that are not real-
# valued, or which we want to ignore.
# Number of power bands for each active channel.
set num_bands 1
# Number of seconds over which you wish to take the average power.
set averaging_interval 60
# Channels to analyze.
set select "1 6"
# Output file.
set ofl "~/Desktop/PBA_Data.txt"
# Set to 1 if you want median rather than average. Set to 2 if you want the
# maximimum rather than the average.
set calculation_type 0
# The band offset allows us to skip over characteristics that are not
# real-valued power measurements or metrics. Event classification
# processors produce a letter as the first characteristic, so we set
# the band offset to 1 to skip over the letter.
set band_offset 0
# We select a list of files in a browser. We sort them in order of
# increasing time stamp. The name of the files is assumed to be
# of the form Mx.ndf, where
set fnl [lsort -dictionary [LWDAQ_get_file_name 1]]
# We initialize some variables used in the analysis.
set previous_clock_seconds 0
foreach id $select {
for {set band 1} {$band <= $num_bands} {incr band} {
set power_$id\_$band [list]
}
}
set interval_num 0
# We go through the files and set our start time based upon the file
# name.
foreach fn $fnl {
if {![regexp -nocase {M([0-9]{10}).*?\.txt} $fn match file_clock_seconds]} {
error "File [file tail $fn] is NDF, not text."
}
if {$previous_clock_seconds == 0} {
set previous_clock_seconds $file_clock_seconds
}
# Read the characteristics from the file into a list.
set f [open $fn r]
set characteristics [string trim [read $f]]
close $f
set characteristics [split $characteristics \n]
# Go through all the characteristics lines.
for {set i 0} {$i < [llength $characteristics]} {incr i} {
# We get the i'th characteristics line from the list.
set r [lindex $characteristics $i]
# We go through each selected id, find the elements in the characteristics
# line that correspond to this id, if any, and add all append the band
# powers to their interval list, which we will later use to obtain average,
# median, or maximum values during the interval.
foreach id $select {
set index [lsearch -start 2 $r $id]
for {set band 1} {$band <= $num_bands} {incr band} {
set new_power [lindex $r [expr $index + $band + $band_offset]]
if {$new_power != "0.00"} {
lappend power_$id\_$band $new_power
}
}
}
# We obtain the time of this line from the file name and the time given
# in the second element of the characteristics line.
set clock_seconds [expr $file_clock_seconds + [lindex $r 1]]
# Whenever we have accumulated an averaging interval's worth of data,
# we print the averages to the screen along with the time from the start
# of our calculation.
if {[expr $clock_seconds - $previous_clock_seconds] >= $averaging_interval} {
incr interval_num
LWDAQ_print $t "Interval $interval_num,\
Time [expr round($clock_seconds)],\
Length [expr round($clock_seconds - $previous_clock_seconds)]."
set result "[expr round($clock_seconds)] "
foreach id $select {
append result "$id "
for {set band 1} {$band <= $num_bands} {incr band} {
set len [llength [set power_$id\_$band]]
if {$len > 0} {
switch $calculation_type {
0 {
set sum 0.0
foreach p [set power_$id\_$band] {
set sum [expr $sum + $p]
}
lappend result [format %.1f [expr 1.0 * $sum / $len]]
}
1 {
set sorted [lsort -increasing -real [set power_$id\_$band]]
lappend result [format %.1f [lindex $sorted [expr round(0.5*$len)]]]
}
2 {
set sorted [lsort -increasing -real [set power_$id\_$band]]
lappend result [format %.1f [lindex $sorted end]]
}
}
} {
append result "0.0"
}
append result " "
}
}
set previous_clock_seconds $clock_seconds
foreach id $select {
for {set band 1} {$band <= $num_bands} {incr band} {
set power_$id\_$band ""
}
}
LWDAQ_print $ofl $result
}
LWDAQ_support
if {![winfo exists $t]} {break}
}
}
[15-JUL-25] The Reception Average (RA.tcl) analysis program calculates average reception during consecutive intervals of time. It is similar to the Power Band Average script in the way it reads in characteristics files one after another and prints its results to the screen. We can adapt this program to calculate the average of any single characteristic produced by a standard processor. We could use this program to calculate, for example, the average battery voltages of our transmitters in ten-minute intervals, after using our Generic Processor to produce characteristics files. To run the Reception Average analysis program, cut and paste the script into the Toolmaker window and press Execute. A file browser window will open up. Select your characteristics files in the file browser.
# This interval analysis produces a graph of average reception efficiency, as a
# percentage of the messages transmitted, during consecutive averaging periods
# we define in seconds. As input, the analysis takes a series of characteristics
# files that we select with a file browser. Each line in these files must be of
# the following format.
# archivename archivetime id reception c1 c2 ...c_n id...
# Here c1..cn are n characteristics generated by some processor, but the first
# characteristic is reception as a percentage of nominal. Many processors
# produce an output of this form. This reception-averaging script is designed to
# extract the reception values from these characteristics lines and generate the
# average reception in, for example, hour-long intervals so we can plot
# reception versus time. The analysis supports any number of other
# characteristics being present in the line, or even n=0 where reception is the
# only characteristic in the line. All characteristics must be real-valued
# numbers with a decimal point, so this script can find the reception after the
# ID number in each line without knowing how many other real-valued
# characteristics are present.
# The analysis deduces the playback interval by looking at the first two lines
# in the first file and subtracting their archive times.
#
# The output of this script appears in the Toolmaker execution window. Each line
# in the window is the average reception for selected channels, calculated over
# the one averaging interval.
# The select string allows us to list the transmitter channels we want to
# examine.
# Number of seconds over which you wish to take the average power.
set averaging_interval 300
# Channels to analyze.
set select "117 118 119 120 133 134 135 136"
# We write a title line to the screen for the benefit of spreadsheet plots.
LWDAQ_print $t "Time $select"
# We select a list of files in a browser. We sort them in order of
# increasing time stamp.
set fnl [lsort -dictionary [LWDAQ_get_file_name 1]]
# We initialize some variables used in the analysis.
set previous_clock_seconds 0
set start_clock_seconds 0
set count 0
foreach id $select {
set reception_$id 0
}
# We go through the files and set our start time based upon the file
# name.
foreach fn $fnl {
if {![regexp -nocase {M([0-9]{10}).*?\.txt} $fn match file_clock_seconds]} {
error "File [file tail $fn] has invalid name."
}
if {$previous_clock_seconds == 0} {
set previous_clock_seconds $file_clock_seconds
set start_clock_seconds $file_clock_seconds
}
# Read the characteristics from the file into a list.
set f [open $fn r]
set characteristics [string trim [read $f]]
close $f
set characteristics [split $characteristics \n]
# Go through all the characteristics lines.
for {set i 0} {$i < [llength $characteristics]} {incr i} {
# We get the i'th characteristics line from the list.
set r [lindex $characteristics $i]
# We go through each selected id, find the reception value that corresponds
# to this id, and store it.
foreach id $select {
set index [lsearch -start 2 $r $id]
if {$index >= 0} {
set reception_$id [expr [set reception_$id] + [lindex $r [expr $index + 1]]]
}
}
# Increment the interval counter.
incr count
# We obtain the time of this line from the file name and the time given
# in the second element of the characteristics line.
set clock_seconds [expr $file_clock_seconds + [lindex $r 1]]
# Whenever we have accumulated an averaging interval's worth of data,
# we print the averages to the screen along with the time from the start
# of our calculation. To plot the averages, we copy from the Toolmaker
# execution window into a spreadsheet.
if {[expr $clock_seconds - $previous_clock_seconds] >= $averaging_interval} {
set result "[expr round($clock_seconds) - $start_clock_seconds] "
foreach id $select {
append result [format %.1f [expr [set reception_$id] / $count]]
append result " "
}
set previous_clock_seconds $clock_seconds
foreach id $select {set reception_$id 0}
set count 0
LWDAQ_print $t $result
}
LWDAQ_support
if {![winfo exists $t]} {break}
}
}
The Reception Failure (RF.tcl) script looks for periods of reception failure and writes an event list to the Toolmaker execution window. Cut and past the list into a file to make an event list the Neuroplayer can step through. Cut and paste the script into the Toolmaker window and press Execute to run.
# Reception Failure, a Neuroplayer Analysis Script.
# Run this script in the LWDAQ Toolmaker.
#
# This analysis produces a list of reception failure events.
#
# Channels to analyze.
set select "3"
# Percentage reception below which failure occurs.
set failure_threshold 80
# We select a list of files in a browser. We sort them in order of
# increasing time stamp.
set fnl [lsort -dictionary [LWDAQ_get_file_name 1]]
# We initialize the failure description string.
set fd ""
# We go through the files and set our start time based upon the file
# name.
foreach fn $fnl {
if {![regexp -nocase {M([0-9]{10}).*?\.txt} $fn match file_time]} {
error "File [file tail $fn] has invalid name."
}
# Read the characteristics from the file into a list.
set f [open $fn r]
set characteristics [string trim [read $f]]
close $f
set characteristics [split $characteristics \n]
# Go through all the characteristics lines.
for {set i 0} {$i < [llength $characteristics]} {incr i} {
# We get the i'th characteristics line from the list.
set r [lindex $characteristics $i]
# We go through each selected id, find the reception that corresponds
# to it, or set the reception to zero, and either start or continue
# a failure event. We skip over the file name and archive time when we
# look for id (elements 0 and 1 in the characteristics line). We will
# not mistake power or reception for an id because both are written
# with one digit after a decimal point.
set failure 0
foreach id $select {
set index [lsearch -start 2 $r $id]
if {$index >= 0} {
set reception [lindex $r [expr $index + 1]]
} {
set reception 0
}
if {$reception < $failure_threshold} {
set failure 1
if {$fd == ""} {
set fd "$file_time [lindex $r 1] $id"
} {
if {![string match *$id\* [lindex $fd 2]]} {
lset fd 2 [list [lindex $fd 2] $id]
}
}
}
}
if {(!$failure) && ($fd != "")} {
set fl [expr $file_time + [lindex $r 1] - [lindex $fd 0] - [lindex $fd 1]]
set fd "M[lindex $fd 0]\.ndf [lindex $fd 1] $id \
\"reception failures lasting $fl seconds\""
LWDAQ_print $t $fd
set fd ""
}
}
LWDAQ_support
if {![winfo exists $t]} {break}
}
The Bad Characteristics Sifter (BCS.tcl) script goes through characteristics files and extracts those corresponding to one particular channel, provided that the characteristics meet certain user-defined criteria, such as minimum or maximum power in various frequency bands. Cut and paste the script into the Toolmaker window and press Execute to run.
# Begin Sifting Script.
#
# Tell the script how many power bands are present
# for each channel in the characteristics file. The
# script assumes that each channel gets its channel
# number followed by reception as a percentage, then
# the power bands.
set num_bands 3
# Channel to analyze. Specify only one channel. The
# characteristics for this channel will be extracted
# from the files you select, and saved in a single file
# called Selected_id.
set id 1
# Select charcteristics files.
LWDAQ_print $t "Reading characteristics files..."
set characteristics ""
set fnl [lsort -dictionary [LWDAQ_get_file_name 1]]
foreach fn $fnl {
if {[string match *.ndf $fn]} {
error "file [file tail $fn] has invalid name"
}
set f [open $fn r]
append characteristics [string trim [read $f]]\n
close $f
}
set characteristics [split [string trim $characteristics] \n]
# Form the output file name using one of the
# characteristics file names. Lines will be
# appended to this file.
set ofn [file join [file dirname $fn] Sifted_$id\.txt]
set of [open $ofn a]
# Sift the characteristics lines.
LWDAQ_print $t "Sifting..."
set counter 0
set rejected 0
foreach c $characteristics {
if {[expr $counter % 100] == 0} {LWDAQ_print $t "$counter lines"}
LWDAQ_support
# Find the selected channel number in the line, and extract all
# its band powers into variables p1..pn, where n is the number
# of bands. We locate the band powers for channel id within the
# the line by looking for its id after the file name and archive
# time. We start at element 2 in the line. Band powers are stored
# as real numbers. The channel ids are integers, so they will not be
# confused with the band powers, regardless of the value of the band
# power. The first real number after the id is the reception
# efficiency. After that come the band powers.
set index [lsearch -start 2 $c $id ]
for {set n 1} {$n <= $num_bands} {incr n} {
set p$n [lindex $c [expr $index + 1 + $n]]
}
# Here we use the power bands to accept or reject a characteristic
# line. If we accept the line, we write the data for our selected
# channel to the output file. The first check we make is to see if
# data for the channel exists, which is to check that the index
# location is greater than zero.
if {($index > 0) && ($p1 > 0) && ($p1 < 500) && ($p3 < 60)} {
puts $of "[lrange $c 0 1] [lrange $c $index [expr $index + 1 + $num_bands]]"
} {
incr rejected
}
incr counter
}
# Close the output file.
close $of
LWDAQ_print $t "Read $counter lines and rejected $rejected."
LWDAQ_print $t "Accepted lines written to $ofn"