Experiment Prototype¶

The experiment_prototype package contains the building blocks of experiments, which includes the ExperimentPrototype base class, the interface_classes subpackage including the InterfaceClassBase classes, and the ExperimentException. There is also a list_tests module which is used by the ExperimentPrototype class.

Experiment Prototype Package¶

experiment_prototype¶

This is the base module for all experiments. An experiment will only run if it inherits from this class.

copyright: 2018 SuperDARN Canada
author: Marci Detwiller

class src.experiment_prototype.experiment_prototype.ExperimentPrototype(cpid, rx_bandwidth=5000000.0, tx_bandwidth=5000000.0, comment_string='')[source]¶

Bases: object

The base class for all experiments. This class is used via inheritance to create experiments.

A prototype experiment class composed of metadata, including experiment slices (exp_slice) which are dictionaries of radar parameters. Basic, traditional experiments will be composed of a single slice. More complicated experiments will be composed of multiple slices that interface in one of four pre-determined ways, as described under interface_types.

Some variables shouldn’t be changed by the experiment, and their properties do not have setters. Some variables can be changed in the init of your experiment, and can also be modified in-experiment by the class method ‘update’ in your experiment class. These variables have been given property setters.

The following are the user-modifiable attributes of the ExperimentPrototype that are used to make an experiment. Other parameters are set in the init and cannot be modified after instantiation.

slice_dict: modifiable only using the add_slice, edit_slice, and del_slice methods.
interface: modifiable using the add_slice, edit_slice, and del_slice methods, or by
updating the interface dict directly.

Parameters

cpid (int) – Unique id necessary for each control program (experiment). Cannot be changed after instantiation.
rx_bandwidth (float) – The desired bandwidth for the experiment. Directly determines rx sampling rate of the USRPs. Cannot be changed after instantiation. Default 5.0 MHz.
tx_bandwidth (float) – The desired tx bandwidth for the experiment. Directly determines tx sampling rate of the USRPs. Cannot be changed after instantiation. Default 5.0 MHz.
comment_string (str) – Description of experiment for data files. This should be used to describe your overall experiment design. Another comment string exists for every slice added, to describe information that is slice-specific.

Raises

ExperimentException – if cpid is not an integer, cannot be represented by a 16-bit signed integer, is not unique, or is not a positive value
ExperimentException – if output sample rate is too high
ExperimentException – if transmit bandwidth is too large or not an integer multiple of USRP clock rate
ExperimentException – if receive bandwidth is too large or not an integer multiple of USRP clock rate

add_slice(exp_slice, interfacing_dict=None)[source]¶

Add a slice to the experiment.

Parameters

exp_slice (dict) – a slice (dictionary of slice_keys) to add to the experiment.
interfacing_dict (dict) – dictionary of type {slice_id : INTERFACING , … } that defines how this slice interacts with all the other slices currently in the experiment.

Returns

the slice_id of the new slice that was just added.

Return type

int

Raises

ExperimentException – if slice is not a dictionary or if there are errors in setup_slice.

build_scans()[source]¶

Build the scan information, which means creating the Scan, AveragingPeriod, and Sequence instances needed to run this experiment.

Will be run by experiment handler, to build iterable objects for radar_control to use. Creates scan_objects in the experiment for identifying which slices are in the scans.

calculate_center_freq(chosen_freq)[source]¶

Calculates the closest actual center frequency based on the USRP device clock and the desired center frequency.

Parameters: chosen_freq (float) – the center frequency desired
Return actual_center_freq: valid center frequency for USRP device
Rtype actual_center_freq: float

check_new_slice_interfacing(interfacing_dict)[source]¶

Checks that the new slice plays well with its siblings (has interfacing that is resolvable). If so, returns a new dictionary with all interfacing values set.

The interfacing assumes that the interfacing_dict given by the user defines the closest interfacing of the new slice with a slice. For example, if the slice is to be ‘CONCURRENT’ combined with slice 0, the interfacing dict should provide this information. If only ‘SCAN’ interfacing with slice 1 is provided, then that will be assumed to be the closest and therefore the interfacing with slice 0 will also be ‘SCAN’.

If no interfacing_dict is provided for a slice, the default is to do ‘SCAN’ type interfacing for the new slice with all other slices.

Parameters: interfacing_dict (dict) – the user-provided interfacing dict, which may be empty or incomplete. If empty, all interfacing is assumed to be = ‘SCAN’ type. If it contains something, we ensure that the interfacing provided makes sense with the values already known for its closest sibling.
Returns: full interfacing dictionary.
Return type: dict
Raises: ExperimentException – if invalid interface types provided or if interfacing can not be resolved.

property comment_string¶

A string related to the experiment, to be placed in the experiment’s files.

Returns: comment_string - read-only
Return type: str

static compare_freq_to_band(freq_list, center_freq_band, valid=True, down_shift=False)[source]¶

Checks if a list of frequencies fall within a frequency band and returns a boolean indicating the result. Since the radar should not operate in a 50 kHz band centered around the center frequency, the center band is split into two bands above and below the center frequency. If the frequency list under test is a range (eg: cfs_range) then the function will also check if the frequency range overlaps with the 50kHz band around the center freq that should not be used.

Parameters

freq_list (list, 1 or 2 elements long) – List of frequencies to check
center_freq_band (list of lists [[band 1], [band 2]]) – The frequency bands a corresponding to a particular center frequency. Split into and upper and lower band because of the 50kHZ band around the center frequency that should not be used for operation.
valid (boolean) – Tracks if the frequencies meet the conditions
down_shift (boolean) – Indicates if the freq_list encompasses a null in center_freq_band

Returns

valid, down_shift

Return type

boolean, boolean

property cpid¶

This experiment’s CPID (control program ID, a term that comes from ROS).

Returns: cpid - read-only, only modified at runtime by set_scheduling_mode() to set to a negative value if the embargo flag was set in the schedule
Return type: int

del_slice(remove_slice_id)[source]¶

Remove a slice from the experiment.

Parameters: remove_slice_id (int) – the id of the slice you’d like to remove.
Returns: a copy of the removed slice.
Return type: dict
Raises: ExperimentException – if remove_slice_id does not exist in the slice dictionary.

edit_slice(edit_slice_id, **kwargs)[source]¶

Edit a slice.

A quick way to edit a slice. In reality this is actually adding a new slice and deleting the old one. Useful for quick changes. Note that using this function will remove the slice_id that you are changing and will give it a new id. It will account for this in the interfacing dictionary.

Parameters

edit_slice_id (int) – the slice id of the slice to be edited.
kwargs (dict) – slice parameter to slice values that you want to change.

Returns

the new slice id of the edited slice, or the edit_slice_id if no change has occurred due to failure of new slice parameters to pass experiment checks.

Return type

int

Raises

ExperimentException – if the edit_slice_id does not exist in slice dictionary or the params or values do not make sense.

property experiment_name¶

The experiment class name.

Returns: experiment_name
Return type: str

get_slice_interfacing(slice_id)[source]¶

Check the experiment’s interfacing dictionary for all interfacing that pertains to a given slice, and return the interfacing information in a dictionary.

Parameters: slice_id (int) – Slice ID to search the interface dictionary for.
Returns: interfacing dictionary for the slice.
Return type: dict

property interface¶

The dictionary of interfacing for the experiment slices.

Interfacing should be set up for any slice when it gets added, ie. in add_slice, except for the first slice added. The dictionary of interfacing is setup as: [(slice_id1, slice_id2) : INTERFACING_TYPE, (slice_id1, slice_id3) : INTERFACING_TYPE, …] for all current slice_ids.

Returns: interface
Return type: dict

property new_slice_id¶

The next unique slice id that is available to this instance of the experiment.

This gets incremented each time it is called to ensure it returns a unique ID each time.

Returns: new_slice_id
Return type: int

property num_slices¶

The number of slices currently in the experiment.

Will change after methods add_slice or del_slice are called.

Returns: num_slices
Return type: int

property options¶

The config options for running this experiment.

These cannot be set or removed, but are specified in the config.ini, hdw.dat, and restrict.dat files.

Returns: options
Return type: Options

property rx_bandwidth¶

The receive bandwidth for this experiment, in Hz.

Returns: rx_bandwidth - read-only
Return type: float

property rxrate¶

The receive bandwidth for this experiment, or the receive sampling rate (of I and Q samples) In Hz.

Returns: rxrate - read-only
Return type: float

property scan_objects¶

The list of instances of class Scan for use in radar_control.

These cannot be modified by the user, but are created using the slice dictionary.

Returns: scan_objects
Return type: list

property scheduling_mode¶

Return the scheduling mode time type that this experiment is running in. Types are listed in possible_scheduling_modes. Initialized to ‘unknown’ until set by the experiment handler.

Returns: scheduling_mode
Return type: str

self_check()[source]¶

Check that the values in this experiment are valid. Checks all slices.

Raises: ExperimentException – if any self check errors occur

set_center_frequencies()[source]¶

Determines and sets a tx and rx center frequency for any slices the user did not set manually. First the slices with unset center frequencies will be determined and compiled. Then a while loop will be entered. It also creates a list of all CONCURRENT and SEQUENCE interfaces slices that must have the same center frequencies.

While not all the slices in the experiment have center frequencies, the lowest unset slice frequency will be selected and a center frequency that just satisfies that slice will be picked. The loop then goes through all the other unset slices to find which other slices are also satisfied by the center frequency. In the event that a slice has a frequency range that overlaps with the band around the center frequency that should not be used for operation, instead of setting the center frequency for all slices, the loop will shift the center frequency down a bit and then go back through the loop again. Once no frequency ranges overlap with the center frequency, then all slices that are satisfied by the current selected center frequency will have the tx and rx center frequencies set.

Raises: ExperimentException – if while loop continues for too long (10000 iterations)

slice_beam_directions_mapping(slice_id)[source]¶

A mapping of the beam directions in the given slice id.

Parameters: slice_id (int) – id of the slice to get beam directions for.
Returns: enumeration mapping dictionary of beam number to beam direction(s) in degrees off boresight.
Return type: dict

property slice_dict¶

The dictionary of slices.

The slice dictionary can be updated in add_slice, edit_slice, and del_slice. The slice dictionary is a dictionary of dictionaries that looks like:

{ slice_id1 : {slice_key1 : x, slice_key2 : y, …}, slice_id2 : {slice_key1 : x, slice_key2 : y, …}, …}

Returns: slice_dict
Return type: dict

property slice_ids¶

The list of slice ids that are currently available in this experiment.

This can change when add_slice, edit_slice, and del_slice are called.

Returns: slice_ids
Return type: list

property slice_keys¶

The list of slice keys available.

This cannot be updated. These are the keys in the current ExperimentPrototype slice_keys dictionary (the parameters available for slices).

Returns: slice_keys
Return type: frozenset

property transmit_metadata¶

A dictionary of config options and experiment-set values that cannot change in the experiment, that will be used to build pulse sequences.

Returns: transmit_metadata
Return type: dict

property tx_bandwidth¶

The transmission sample rate to the DAC (Hz), and the transmit bandwidth.

Returns: tx_bandwidth - read-only
Return type: float

property txrate¶

The transmission sample rate to the DAC (Hz).

Returns: txrate - read-only
Return type: float

src.experiment_prototype.experiment_prototype.interface_types = ('SCAN', 'AVEPERIOD', 'SEQUENCE', 'CONCURRENT')¶

Interfacing in this case refers to how two or more slices are meant to be run together. The following types of interfacing between slices are possible, arranged from highest level of experiment building-block to the lowest level:

SCAN

The scan-by-scan interfacing allows for slices to run a scan of one slice, followed by a scan of the second. The scan mode of interfacing typically means that the slice will cycle through all of its beams before switching to another slice.

If any slice in the experiment specifies a value for scanbound, all other slices must also specify a value for scanbound. The values do not have to be the same, however.
AVEPERIOD
This type of interfacing allows for one slice to run its averaging period (also known as integration time or integration period), before switching to another slice’s averaging period. This type of interface effectively creates an interleaving scan where the scans for multiple slices are run ‘at the same time’, by interleaving the averaging periods.

Slices which are interfaced in this manner must share:
- the same scanbound value.
SEQUENCE
Sequence interfacing allows for pulse sequences defined in the slices to alternate between each other within a single averaging period. It’s important to note that data from a single slice is averaged only with other data from that slice. So in this case, the averaging period is running two slices and can produce two averaged datasets, but the sequences within the averaging period are interleaved.

Slices which are interfaced in this manner must share:
- the same scanbound value.
- the same intt or intn value.
- the same rx_beam_order length (scan length).
- the same txctrfreq value.
- the same rxctrfreq value.
CONCURRENT
Concurrent interfacing allows for pulse sequences to be run together concurrently. Slices will have their pulse sequences summed together so that the data transmits at the same time. For example, slices of different frequencies can be mixed simultaneously, and slices of different pulse sequences can also run together at the cost of having more blanked samples. When slices are interfaced in this way the radar is truly transmitting and receiving the slices simultaneously.

Slices which are interfaced in this manner must share:
- the same scanbound value.
- the same intt or intn value.
- the same rx_beam_order length (scan length).
- the same txctrfreq value.
- the same rxctrfreq value.
- the same decimation_scheme.

class src.experiment_prototype.experiment_slice.ExperimentSlice[source]¶

These are the keys that are set by the user when initializing a slice. Some are required, some can be defaulted, and some are set by the experiment and are read-only.

Slice Keys Required by the User

beam_angle required: list of beam directions, in degrees off azimuth. Positive is E of N. The beam_angle list length = number of beams. Traditionally beams have been 3.24 degrees separated but we don’t refer to them as beam -19.64 degrees, we refer as beam 1, beam 2. Beam 0 will be the 0th element in the list, beam 1 will be the 1st, etc. These beam numbers are needed to write the [rx|tx]_beam_order list. This is like a mapping of beam number (list index) to beam direction off boresight.
cfs_range required or freq required: range for clear frequency search, should be a list of length = 2, [min_freq, max_freq] in kHz.
first_range required: first range gate, in km
freq required or cfs_range required: transmit/receive frequency, in kHz. Note if you specify cfs_range it won’t be used.
intt required or intn required: duration of an averaging period (integration), in ms. (maximum)
intn required or intt required: number of averages to make a single averaging period (integration), only used if intt = None
num_ranges required: Number of range gates to receive for. Range gate time is equal to pulse_len and range gate distance is the range_sep, calculated from pulse_len.
pulse_len required: length of pulse in us. Range gate size is also determined by this.
pulse_sequence required: The pulse sequence timing, given in quantities of tau_spacing, for example normalscan = [0, 14, 22, 24, 27, 31, 42, 43].
rx_beam_order required: beam numbers written in order of preference, one element in this list corresponds to one averaging period. Can have lists within the list, resulting in multiple beams running simultaneously in the averaging period, so imaging. A beam number of 0 in this list gives us the direction of the 0th element in the beam_angle list. It is up to the writer to ensure their beam pattern makes sense. Typically rx_beam_order is just in order (scanning W to E or E to W), ie. [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]. You can list numbers multiple times in the rx_beam_order list, for example [0, 1, 1, 2, 1] or use multiple beam numbers in a single averaging period (example [[0, 1], [3, 4]], which would trigger an imaging integration. When we do imaging we will still have to quantize the directions we are looking in to certain beam directions. It is up to the user to ensure that this field works well with the specified tx_beam_order or tx_antenna_pattern.
rxonly read-only: A boolean flag to indicate that the slice doesn’t transmit, only receives.
tau_spacing required: multi-pulse increment (mpinc) in us, Defines minimum space between pulses.

Defaultable Slice Keys

acf defaults

flag for rawacf generation. The default is False. If True, the following fields are also used: - averaging_method (default ‘mean’) - xcf (default True if acf is True) - acfint (default True if acf is True) - lagtable (default built based on all possible pulse combos) - range_sep (will be built by pulse_len to verify any provided value)

acfint defaults

flag for interferometer autocorrelation data. The default is True if acf is True, otherwise False.

align_sequences defaults

flag for aligning the start of the first pulse in each sequence to tenths of a second. Default False.

averaging_method defaults

a string defining the type of averaging to be done. Current methods are ‘mean’ or ‘median’. The default is ‘mean’.

cfs_duration defaults

Amount of time a clear frequency search will listen for in ms.

cfs_scheme defaults

Decimation scheme to be used in analyzing data collected during a clear frequency search when determining transmit frequencies for CFS experiment slices

cfs_stable_time defaults

Amount of time in seconds clear frequency search will not change the slice frequencies for. Ensures a minimum stable time, but does not force a change in frequency once the stable time has elapsed.

cfs_pwr_threshold defaults

Difference in power (in dB) that clear frequency search must see in the measured frequencies before it changes the cfs slice frequencies. This can be trigered either when another frequency in the cfs range is found to have a lower power than the currently set frequency, or when the currently set frequency has increased in power by the threshold value since it was selected as an operating frequency.

cfs_fft_n defaults

Sets the number of elements used in the fft when processing clear frequency search results. Determines the frequency resolution of the processing following the formula; frequency resolution = (rx rate / total decimation rate) / cfs fft n value

cfs_freq_res defaults

Defines the desired frequency resolution of clear frequency search results. The frequency resolution is used to calculate the number of elements used in the fft when processing and sets that value to cfs_fft_n. Note the actual frequency resolution set will differ based on the nearest integer value of n corresponding to the requested resolution.

cfs_always_run defaults

If true always run the cfs sequence, otherwise only run after cfs_stable_time has expired

comment defaults

a comment string that will be placed in the borealis files describing the slice. Defaults to empty string.

lag_table defaults

used in acf calculations. It is a list of lags. Example of a lag: [24, 27] from 8-pulse normalscan. This defaults to a lagtable built by the pulse sequence provided. All combinations of pulses will be calculated, with both the first pulses and last pulses used for lag-0.

pulse_phase_offset defaults

a handle to a function that will be used to generate one phase per each pulse in the sequence. If a function is supplied, the beam iterator, sequence number, and number of pulses in the sequence are passed as arguments that can be used in this function. The default is None if no function handle is supplied.

encode_fn(beam_iter, sequence_num, num_pulses):: return np.ones(size=(num_pulses))

The return value must be numpy array of num_pulses in size. The result is a single phase shift for each pulse, in degrees.

Result is expected to be real and in degrees and will be converted to complex radians.

range_sep defaults

a calculated value from pulse_len. If already set, it will be overwritten to be the correct value determined by the pulse_len. Used for acfs. This is the range gate separation, in the radial direction (away from the radar), in km.

rxctrfreq defaults

Center frequency, in kHz, used to mix to baseband. Since this requires tuning time to set, it is the user’s responsibility to ensure that the re-tuning time does not detract from the experiment implementation. Tuning time is set in the usrp_driver.cpp script and changes to the time will require recompiling of the code.

rx_intf_antennas defaults

The antennas to receive on in interferometer array, default is all antennas given max number from config.

rx_main_antennas defaults

The antennas to receive on in main array, default is all antennas given max number from config.

rx_antenna_pattern defaults

Experiment-defined function which returns a complex weighting factor of magnitude <= 1 for each beam direction scanned in the experiment. The return value of the function must be an array of size [beam_angle, antenna_num]. This function allows for custom beamforming of the receive antennas for borealis processing of antenna iq to rawacf.

scanbound defaults

A list of seconds past the minute for averaging periods in a scan to align to. Defaults to None, not required. If one slice in an experiment has a scanbound, they all must.

seqoffset defaults

offset in us that this slice’s sequence will begin at, after the start of the sequence. This is intended for CONCURRENT interfacing, when you want multiple slice’s pulses in one sequence you can offset one slice’s sequence from the other by a certain time value so as to not run both frequencies in the same pulse, etc. Default is 0 offset.

txctrfreq defaults

Center frequency, in kHz, for the USRP to mix the samples with. Since this requires tuning time to set, it is the user’s responsibility to ensure that the re-tuning time does not detract from the experiment implementation. Tuning time is set in the usrp_driver.cpp script and changes to the time will require recompiling of the code.

tx_antennas defaults

The antennas to transmit on, default is all main antennas given max number from config.

tx_antenna_pattern defaults

experiment-defined function which returns a complex weighting factor of magnitude <= 1 for each tx antenna used in the experiment. The return value of the function must be an array of size [num_beams, main_antenna_count] with all elements having magnitude <= 1. This function is analogous to the beam_angle field in that it defines the transmission pattern for the array, and the tx_beam_order field specifies which “beam” to use in a given averaging period.

tx_beam_order defaults, but required if tx_antenna_pattern given

beam numbers written in order of preference, one element in this list corresponds to one averaging period. A beam number of 0 in this list gives us the direction of the 0th element in the beam_angle list. It is up to the writer to ensure their beam pattern makes sense. Typically tx_beam_order is just in order (scanning W to E or E to W, i.e. [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]. You can list numbers multiple times in the tx_beam_order list, for example [0, 1, 1, 2, 1], but unlike rx_beam_order, you CANNOT use multiple beam numbers in a single averaging period. In other words, this field MUST be a list of integers, as opposed to rx_beam_order, which can be a list of lists of integers. The length of this list must be equal to the length of the rx_beam_order list. If tx_antenna_pattern is given, the items in tx_beam_order specify which row of the return from tx_antenna_pattern to use to beamform a given transmission. Default is None, i.e. rxonly slice.

wait_for_first_scanbound defaults

A boolean flag to determine when an experiment starts running. True (default) means an experiment will wait until the first averaging period in a scan to start transmitting. False means an experiment will not wait for the first averaging period, but will instead start transmitting at the nearest averaging period. Note: for multi-slice experiments, the first slice is the only one impacted by this parameter.

xcf defaults

flag for cross-correlation data. The default is True if acf is True, otherwise False.

Read-only Slice Keys

cfs_flag read-only: A boolean flag to indicate that a clear frequency search will be done. Not currently supported.
cpid read-only: The ID of the experiment, consistent with existing radar control programs. This is actually an experiment-wide attribute but is stored within the slice as well. This is provided by the user but not within the slice, instead when the experiment is initialized.
slice_id read-only: The ID of this slice object. An experiment can have multiple slices. This is not set by the user but instead set by the experiment when the slice is added. Each slice id within an experiment is unique. When experiments start, the first slice_id will be 0 and incremented from there.
slice_interfacing read-only: A dictionary of slice_id : interface_type for each sibling slice in the experiment at any given time.

check_slice()[source]¶: Checks and verifies all fields at any time after instantiation.

experiment_exception¶

This is the exception that is raised when there are problems with the experiment that cannot be remedied using experiment_prototype methods.

copyright: 2018 SuperDARN Canada
author: Marci Detwiller

exception src.experiment_prototype.experiment_exception.ExperimentException(message, *args)[source]¶

Bases: Exception

Is raised for the exception where an experiment cannot be run due to setup errors.

Interface Class Package¶

interface_class_base¶

This is the base module for all InterfaceClassBase types (iterable for an experiment given certain parameters). These types include the Scan class, the AveragingPeriod class, and the Sequence class.

copyright: 2018 SuperDARN Canada
author: Marci Detwiller

class src.experiment_prototype.interface_classes.interface_class_base.InterfaceClassBase(object_keys, object_slice_dict, object_interface, transmit_metadata)[source]¶

Bases: object

The base class for the classes Scan, AveragingPeriod, and Sequence. Scans are made up of AveragingPeriods, these are typically a 3sec time of the same pulse sequence pointing in one direction. AveragingPeriods are made up of Sequences, typically the same sequence run ave. 20-30 times after a clear frequency search. Sequences are made up of pulses, which is a list of dictionaries where each dictionary describes a pulse.

Parameters

object_keys (list) – slice_ids that need to be included in this interface_class_base type.
object_slice_dict (dict) – the slice dictionary that explains the parameters of each slice that is included in this interface_class_base type. Keys are the slice_ids included and values are dictionaries including all necessary slice parameters as keys.
object_interface (dict) – the interfacing dictionary that describes how to interface the slices that are included in this interface_class_base type. Keys are tuples of format (slice_id_1, slice_id_2) and values are of interface_types set up in experiment_prototype.
transmit_metadata (dict) –
a dictionary of the experiment-wide transmit metadata for building pulse sequences. The keys of the transmit_metadata are:
- ’output_rx_rate’ [Hz],
- ’tr_window_time’ [s],
- ’main_antenna_locations’,
- ’intf_antenna_locations’,
- ’main_antenna_count’
- ’intf_antenna_count’
- ’main_antenna_spacing’ [m],
- ’intf_antenna_spacing’ [m],
- ’pulse_ramp_time’ [s],
- ’max_usrp_dac_amplitude’ [V peak],
- ’rx_sample_rate’ [Hz],
- ’min_pulse_separation’ [us],
- ’txrate’ [Hz],
- ’intf_offset’ [m,m,m],
- ’dm_rate’

get_nested_slice_ids()[source]¶

Organize the slice_ids by interface.

This method is inherited by child classes and organizes all slices in each child class which should be combined by the class. For example, all slices in a Scan should be combined if they share an AveragingPeriod or Sequence or are concurrent.

Returns a list of lists where each inner list contains the slices that are combined inside this object. e.g. for InterfaceClassBase: len(nested_slice_list) = # of scans in this experiment, len(nested_slice_list[0]) = # of slices in the first scan

Returns: A list that has one element per scan. Each element is a list of slice_ids signifying which slices are combined inside that scan. The list returned could be of length 1, meaning only one scan is present in the experiment.
Return type: list of lists

prep_for_nested_interface_class()[source]¶

Retrieve the params needed for the nested class (also with base InterfaceClassBase).

This class reduces duplicate code by breaking down the InterfaceClassBase class into the separate portions for the nested instances. For Scan class, the nested class is AveragingPeriod, and we will need to break down the parameters given to the Scan instance because there may be multiple AveragingPeriods within. For AveragingPeriod, the nested class is Sequence.

Returns: params for the nested class’s instantiation.
Return type: list

static slice_combos_sorter(list_of_combos, all_keys)[source]¶

Sort keys of a list of combinations so that keys only appear once in the list.

This function modifies the input list_of_combos so that all associated slices are in the same list. For example, if input is list_of_combos = [[0,1], [0,2], [0,4], [1,4], [2,4]] and all_keys = [0,1,2,4,5] then the output should be [[0,1,2,4], [5]]. This is used to get the slice dictionary for nested class instances. In the above example, we would then have two instances of the nested class to create: one with slices 0,1,2,4 and another with slice 5.

Parameters

list_of_combos (list) – list of lists of length two associating two slices together.
all_keys (list) – list of all keys included in this object (scan, ave_period, or sequence).

Returns

list of combos that is sorted so that each key only appears once and the lists within the list are of however long necessary

Return type

list

scans¶

This is the module containing the Scan class. The Scan class contains the InterfaceClassBase members, as well as a scanbound (to be implemented), a beamdir dictionary and scan_beams dictionary which specify beam direction angle and beam order in a scan, respectively, for individual slices that are to be combined in this scan. Beam direction information gets passed on to the AveragingPeriod.

copyright: 2018 SuperDARN Canada
author: Marci Detwiller

class src.experiment_prototype.interface_classes.scans.Scan(scan_keys, scan_slice_dict, scan_interface, transmit_metadata)[source]¶

Bases: InterfaceClassBase

Set up the scans.

A scan is made up of AveragingPeriods at defined beam directions, and some other metadata for the scan itself.

The unique members of the scan are (not a member of the interfaceclassbase):

scanbound: A list of seconds past the minute for scans to align to. Must be increasing, and it is possible to have values greater than 60s.

prep_for_nested_interface_class()[source]¶

Override of base method to give more information about beamorder and beamdir.

Beam order and beamdir are required for instantiation of the nested class AveragingPeriod so we need to extract this information as well to fill self.aveperiods.

Returns: Parameters that can be directly passed into the nested InterfaceClassBase type, AveragingPeriod. The params_list is of length = # of AveragingPeriods in this scan.
Return type: list of lists

averaging_periods¶

This is the module containing the AveragingPeriod class. The AveragingPeriod class contains the InterfaceClassBase members, as well as cfs_flag, intn (number of integrations to run), or intt(max time for integrations), and it contains sequences of class Sequence.

copyright: 2018 SuperDARN Canada
author: Marci Detwiller

class src.experiment_prototype.interface_classes.averaging_periods.AveragingPeriod(ave_keys, ave_slice_dict, ave_interface, transmit_metadata, slice_to_beamorder_dict, slice_to_beamdir_dict)[source]¶

Bases: InterfaceClassBase

Set up the AveragingPeriods.

An averaging period contains sequences and integrates one or multiple pulse sequences together in a given time frame or in a given number of averages, if that is the preferred limiter.

The unique members of the averagingperiod are (not a member of the interfaceclassbase):

slice_to_beamorder: passed in by the scan that this AveragingPeriod instance is contained in. A dictionary of slice: beam_order for all slices contained in this aveperiod.
slice_to_beamdir: passed in by the scan that this AveragingPeriod instance is contained in. A dictionary of slice: beamdir(s) for all slices contained in this aveperiod.
intt: The priority limitation. The time limit (ms) at which time the aveperiod will end. If None, we will use intn to end the aveperiod (a number of sequences).
intn: Number of averages (# of times the sequence transmits) to end after for the averagingperiod.
sequences: The list of sequences included in this aveperiod. This does not indicate how many averages will be transmitted in the aveperiod. If there are multiple sequences in the list, they will be alternated between until the end of the aveperiod.
one_pulse_only: boolean, True if this averaging period only has one unique set of pulse samples in it. This is true if there is only one sequence in the averaging period, and all pulses after the first pulse in the sequence have the isarepeat key = True. This boolean can be used to speed up the process of sending data to the driver which means we can get more averages in less time.

set_beamdirdict(beamiter)[source]¶

Get a dictionary of ‘slice_id’ : ‘beamdir(s)’ for this averaging period.

At a given beam iteration, this averagingperiod instance will select the beam directions that it will shift to.

Parameters: beamiter (int) – the index into the beam_order list, or the index of an averaging period into the scan
Returns: dictionary of slice to beamdir where beamdir is always a list (may be of length one though). Beamdir is azimuth angle.
Return type: dict

class src.experiment_prototype.interface_classes.averaging_periods.CFSAveragingPeriod(ave_keys, ave_slice_dict, ave_interface, transmit_metadata, slice_to_beamorder_dict, slice_to_beamdir_dict)[source]¶

Bases: AveragingPeriod

A variation of AveragingPeriod that conducts a clear frequency search to determine the frequency to use for some or all of the slices within the averaging period.

build_cfs_sequence()[source]¶: Builds an empty rx only pulse sequence to collect clear frequency search data

check_update_freq(cfs_spectrum, cfs_slices, threshold, beam_iter)[source]¶

Checks if any scanned frequencies have power levels that exceed the current power of each cfs slice based on a threshold

Params cfs_packet: Results of the CFS analysis
Params cfs_slices: Slice ids of each cfs slice to be checked
Params threshold: Power threshold (dB) used in check
Params beam_iter: current beam index

select_cfs_freqs(cfs_packet)[source]¶

Accepts the analysis results of the clear frequency search and uses the passed frequencies and powers to determine what frequencies to set each clear frequency search slice to.

Loops through all sequences that have CFS slices:
- Records all non-CFS slice frequencies
Loops through each CFS slice in the averaging period:
- Calculates the tx bandwidth for the slice pulses;
- Creates a mask of the CFS frequency band
- Masks restricted frequencies
- Masks already-used frequencies by other slices
- Masks around the N200 tuned center frequencies (rx and tx)
- Masks frequencies outside requested CFS range
- Sorts measured spectrum by power
- Sets slice frequency to the least powerful frequency
- Updates already-used frequency list
- Stores the frequency mask
Builds each CFS sequence
Return the frequency masks

Parameters: cfs_packet (ProcessedSequenceMessage) – Analyzed CFS sequence data

update_cfs_freqs()[source]¶

src.experiment_prototype.interface_classes.averaging_periods.log = <BoundLoggerLazyProxy(logger=None, wrapper_class=None, processors=None, context_class=None, initial_values={}, logger_factory_args=('<frozen runpy>',))>¶: Scans are made up of AveragingPeriods, these are typically a 3sec time of the same pulse sequence pointing in one direction. AveragingPeriods are made up of Sequences, typically the same sequence run ave. 20-30 times after a clear frequency search. Sequences are made up of pulse_time lists, which give timing, slice, and pulsenumber. CPObject provides channels, pulseshift (if any), freq, pulse length, beamdir, and wavetype.

sequences¶

This is the module containing the Sequence class. The Sequence class contains the InterfaceClassBase members, as well as a list of pulse dictionaries, the total_combined_pulses in the sequence, power_divider, last_pulse_len, ssdelay, seqtime, which together give sstime (scope synce time, or time for receiving, and numberofreceivesamples to sample during the receiving window (calculated using the receive sampling rate).

author: Marci Detwiller

class src.experiment_prototype.interface_classes.sequences.Sequence(seqn_keys, sequence_slice_dict, sequence_interface, transmit_metadata)[source]¶

Bases: InterfaceClassBase

Set up the sequence class.

The members of the sequence are:

ssdelay

delay past the end of the sequence to receive for (us) - function of num_ranges and pulse_len. ss stands for scope sync.

seqtime

the amount of time for the whole sequence to transmit, until the logic signal switches low on the last pulse in the sequence (us).

sstime

ssdelay + seqtime (total time for receiving) (us).

numberofreceivesamples

the number of receive samples to take, given the rx rate, during the sstime.

first_rx_sample_start

The location of the first sample for the RX data from the start of the TX data (in number of samples, unitless). This will be calculated as the center sample of the first occurring pulse (uncombined).

blanks

A list of sample indices that should not be used for acfs because they were samples taken when transmitting.

basic_slice_pulses

A dictionary that holds pre-computed tx samples for each slice. Each dictionary value is a multi-dimensional array that holds a beamformed set of samples for each antenna for all beam directions.

combined_pulses_metadata

This list holds dictionary metadata for all pulses in the sequence. This metadata holds all the info needed to combine pulses if pulses are mixed.

start_time_us - start time of the pulse in us, relative to the first pulse in sqn.
total_pulse_len - total length of the pulse that includes len of all combined pulses.
pulse_sample_start - The tx sample number at which the pulse starts.
tr_window_num_samps - The number of tx samples of the tr window.
component_info - a list of all the pre-combined pulse components (incl their length and start time) that are in the combined pulseAlso in us.
pulse_transmit_data - dictionary hold the transmit metadata that will be sent to driver.

output_encodings

This dict will hold a list of all the encodings used during an aveperiod for each slice. These will be used for data write later.

Parameters

seqn_keys (list) – slice_ids that need to be included in this sequence.
sequence_slice_dict (dict) – the slice dictionary that explains the parameters of each slice that is included in this sequence. Keys are the slice_ids included and values are dictionaries including all necessary slice parameters as keys.
sequence_interface (dict) – the interfacing dictionary that describes how to interface the slices that are included in this sequence. Keys are tuples of format (slice_id_1, slice_id_2) and values are of interface_types set up in experiment_prototype.
transmit_metadata (dict) – metadata from the config file that is useful here.

build_sequence_pulses()[source]¶

build_tx_phases(slice_id, exp_slice, freq_khz)[source]¶

find_blanks()[source]¶: Finds the blanked samples after all pulse positions are calculated.

get_rx_phases(beam_iter)[source]¶

Gets the receive phases for a given beam

Parameters: beam_iter (int) – The beam iter in a scan.
Returns: The receive phases for each possible beam for every main and intf antenna
Return type: dict for both main and intf

make_sequence(beam_iter, sequence_num)[source]¶

Create the samples needed for each pulse in the sequence. This function is optimized to be able to generate new samples every sequence if needed. Modifies the samples_array and isarepeat fields of all pulse dictionaries needed for this sequence for radar_control to use in operation.

Parameters

beam_iter (int) – The beam iterator
sequence_num (int) – The sequence number in the ave period

Returns

Transmit data for each pulse where each pulse is a dict, including timing and samples

Return type

list

Decimation Scheme¶

decimation_scheme¶

This file contains classes and functions for building decimation schemes

class src.experiment_prototype.experiment_utils.decimation_scheme.DecimationScheme(rxrate, output_sample_rate, stages=None)[source]¶

Bases: object

Class for DSP filtering and decimation scheme.

class src.experiment_prototype.experiment_utils.decimation_scheme.DecimationStage(stage_num, input_rate, dm_rate, filter_taps)[source]¶: Bases: object

src.experiment_prototype.experiment_utils.decimation_scheme.calculate_num_filter_taps(sampling_freq, trans_width, k)[source]¶

Calculates the number of filter taps required for the filter, using the sampling rate, transition width desired, and a k value which impacts filter performance (3 typical).

Parameters

sampling_freq (float) – sampling rate, Hz.
trans_width (float) – transition bandwidth between cutoff of passband and stopband (Hz)
k (int) – value increasing filter performance with increasing value.

Returns

num_taps, number of taps for the FIR filter.

Return type

int

src.experiment_prototype.experiment_utils.decimation_scheme.create_default_cfs_scheme()[source]¶: Wide passband decimation scheme for listening only (high output sample rate, beware!) This default clear frequency search scheme is designed for a 300kHz range.

src.experiment_prototype.experiment_utils.decimation_scheme.create_default_scheme()[source]¶

Previously known as create_test_scheme_9 until July 23/2019! Create four stages of FIR filters and a decimation scheme. Returns a decimation scheme of type DecimationScheme. This filter will have a wider receive bandwidth than the previous. Pasha recommends a 10kHz bandwidth for the final stage. I believe there will be aliasing caused by this but perhaps the concern is not critical because of the small bandwidth overlapping. I will test this anyways.

Returns: a decimation scheme for use in experiment.
Return type: DecimationScheme

src.experiment_prototype.experiment_utils.decimation_scheme.create_firwin_filter_by_attenuation(sample_rate, transition_width, cutoff_hz, ripple_db, window_type='kaiser')[source]¶

Create a firwin filter.

Parameters

sample_rate (float) – sample rate
transition_width (float) – transition bandwidth between cutoff of passband and stopband (Hz)
cutoff_hz (float) – cutoff frequency, in hz
ripple_db (float) – The desired attenuation in the stop band, in dB.
window_type (str) – Window for the filter. Defaults to kaiser

Returns

taps, coefficient of FIR filter

Return type

ndarray

src.experiment_prototype.experiment_utils.decimation_scheme.create_firwin_filter_by_num_taps(sample_rate, cutoff_hz, num_taps, window_type=('kaiser', 8.0))[source]¶

Create a firwin filter.

Parameters

sample_rate (float) – sample rate
cutoff_hz (float) – cutoff frequency, in hz
num_taps (int) – The desired number of filter taps
window_type (str) – Window for the filter. Defaults to kaiser

Returns

taps, coefficient of FIR filter

Return type

ndarray

src.experiment_prototype.experiment_utils.decimation_scheme.create_remez_filter(sampling_freq, cutoff_freq, trans_width)[source]¶

Creates the default filter for the experiment, if the filter taps are not provided.

Parameters

sampling_freq (float) – sampling rate, Hz.
cutoff_freq (float) – cutoff frequency, or edge of passband, Hz.
trans_width – transition bandwidth between cutoff of passband and stopband (Hz)

Rtype trans_width

float

Returns

lowpass filter taps created using the remez method.

Return type

ndarray