File Formats

file_formats

Contains the dataclass SliceData defining the data and metadata that is stored in files produced by borealis.

SliceData fields contain associated metadata that determines which file types (rawrf, antennas_iq, bfiq, rawacf) should contain the aforementioned field, and at which level (file or record) the field should be written. Fields at the file level are written only once, with the associated data immutable throughout the experiment for the given slice. Fields at the record level are written to each averaging period, within the record for that averaging period.

class src.utils.file_formats.SliceData[source]

Bases: object

This class defines all fields that need to be written by any type of data file. The ‘groups’ metadata lists the applicable file types for each field.

Each field contains metadata that determines how the field is written to file.

description

A description of the field.

dim_labels

If applicable, a brief descriptor for each dimension of the dataset. This could be different for different group values. If so, this metadata will be a dict, with the keys being the group name and the values the associated list of dimension labels.

dim_scales

If applicable, dimension scales will be associated to the field. These are datasets that match one of the dimensions of the data, such as timestamps to go along with an array of collected data. Note that some dimensions may be associated with multiple fields. If a dimension has no associated dataset, the list will have a None entry.

groups

The types of data file that need this field to be written.

level

The level within the file that the data will be stored at. Either file or record, indicating that the field is either written once per file, or once per record.

nickname

A nickname for the field, used for making Dimension Scale names.

units

Units for the data.

classmethod all_fields(file_type)[source]

Returns a list of names for all fields which belong in ‘file_type’ files.

Parameters:

file_type (str)

classmethod optional_fields(file_type)[source]

Returns a list of names for all fields which are optional for file_type files.

Parameters:

file_type (str)

classmethod required_fields(file_type)[source]

Returns a list of names for all fields which are required for file_type files.

Parameters:

file_type (str)

to_dmap()[source]

Converts data from self into a valid DMAP record.

to_hdf5(group, metadata_group, data_type)[source]

Converts data from self of relevance for the data type into a group and metadata_group

Parameters:

  • group (h5py.Group) – HDF5 group that the data will be placed into

  • metadata_group (h5py.Group) – HDF5 group which file-level metadata should exist in

  • data_type (str) – Type of data that is being written.

agc_status_word: int
antenna_arrays: list[str]
antenna_locations: ndarray
antennas: ndarray
antennas_iq_data: ndarray
averaging_method: str
beam_azms: list[float]
beam_nums: list[int]
bfiq_data: ndarray
blanked_samples: ndarray
borealis_git_hash: str
cfs_freqs: ndarray
cfs_masks: ndarray
cfs_noise: ndarray
cfs_range: ndarray
data_normalization_factor: float
experiment_comment: str
experiment_id: int
experiment_name: str
first_range: int
first_range_rtt: float
freq: float
global_coord: list[str]
gps_locked: bool
gps_to_system_time_diff: float
int_time: float
intf_acfs: ndarray
lag_numbers: ndarray
lag_pulse_descriptors: list[str]
lag_pulses: ndarray
lags: ndarray
local_coord: list[str]
lp_status_word: int
main_acfs: ndarray
num_sequences: int
num_slices: int
pulse_phase_offset: ndarray
pulses: ndarray
range_gates: ndarray
range_sep: int
rawrf_data: ndarray
rx_antennas: ndarray
rx_center_freq: float
rx_intf_antennas: list[int]
rx_intf_excitations: list[complex]
rx_main_antennas: list[int]
rx_main_excitations: list[complex]
rx_sample_rate: float
sample_time: ndarray
samples_data_type: str
scan_start_marker: bool
scheduling_mode: str
slice_comment: str
slice_id: int
slice_interfacing: dict
sqn_timestamps: list[float]
station: str
station_location: ndarray
tau_spacing: float
tx_antennas: list[int]
tx_excitations: ndarray
tx_pulse_len: float
xcfs: ndarray