Behavior

Audio

class AudioInterface(file_paths: list, verbose: bool = False)

Bases: BaseTemporalAlignmentInterface

Data interface for writing .wav audio recordings to an NWB file.

Data interface for writing acoustic recordings to an NWB file.

Writes acoustic recordings as an AcousticWaveformSeries from the ndx_sound extension.

Parameters

• file_paths (list of FilePathTypes) – The file paths to the audio recordings in sorted, consecutive order. We recommend using natsort to ensure the files are in consecutive order.

  >>> from natsort import natsorted
  >>> natsorted(file_paths)
  

• verbose (bool, default: False)

get_metadata_schema() → dict

Retrieve JSON schema for metadata.

get_metadata() → dict

Child DataInterface classes should override this to match their metadata.

get_original_timestamps() → ndarray

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

get_timestamps() → Optional[ndarray]

Retrieve the timestamps for the data in this interface.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

set_aligned_timestamps(aligned_timestamps: List[ndarray])

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float)

Align all starting times for all audio files in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_starting_time (float) – The common starting time for all temporal data in this interface. Applies to all segments if there are multiple file paths used by the interface.

set_aligned_segment_starting_times(aligned_segment_starting_times: List[float])

Align the individual starting time for each audio file in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_segment_starting_times (list of floats) – The relative starting times of each audio file (segment).

align_by_interpolation(unaligned_timestamps: ndarray, aligned_timestamps: ndarray)

Interpolate the timestamps of this interface using a mapping from some unaligned time basis to its aligned one.

Use this method if the unaligned timestamps of the data in this interface are not directly tracked by a primary system, but are known to occur between timestamps that are tracked, then align the timestamps of this interface by interpolating between the two.

An example could be a metronomic TTL pulse (e.g., every second) from a secondary data stream to the primary timing system; if the time references of this interface are recorded within the relative time of the secondary data stream, then their exact time in the primary system is inferred given the pulse times.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

• unaligned_timestamps (numpy.ndarray) – The timestamps of the unaligned secondary time basis.

• aligned_timestamps (numpy.ndarray) – The timestamps aligned to the primary time basis.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, stub_test: bool = False, stub_frames: int = 1000, write_as: Literal['stimulus', 'acquisition'] = 'stimulus', iterator_options: Optional[dict] = None, compression_options: Optional[dict] = None, overwrite: bool = False, verbose: bool = True)

Parameters

• nwbfile (NWBFile) – Append to this NWBFile object

• metadata (dict, optional)

• stub_test (bool, default: False)

• stub_frames (int, default: 1000)

• write_as ({‘stimulus’, ‘acquisition’}) – The acoustic waveform series can be added to the NWB file either as “stimulus” or as “acquisition”.

• iterator_options (dict, optional) – Dictionary of options for the SliceableDataChunkIterator.

• compression_options (dict, optional) – Dictionary of options for compressing the data for H5DataIO.

• overwrite (bool, default: False)

• verbose (bool, default: True)

Return type

NWBFile

DeepLabCut

write_subject_to_nwb(nwbfile: NWBFile, h5file: FilePathType, individual_name: str, config_file: FilePathType, timestamps: Optional[Union[List, ndarray]] = None)

Given, subject name, write h5file to an existing nwbfile.

Parameters

• nwbfile (pynwb.NWBFile) – The in-memory nwbfile object to which the subject specific pose estimation series will be added.

• h5file (str or path) – Path to the DeepLabCut .h5 output file.

• individual_name (str) – Name of the subject (whose pose is predicted) for single-animal DLC project. For multi-animal projects, the names from the DLC project will be used directly.

• config_file (str or path) – Path to a project config.yaml file

• timestamps (list, np.ndarray or None, default: None) – Alternative timestamps vector. If None, then use the inferred timestamps from DLC2NWB

Returns

nwbfile – nwbfile with pes written in the behavior module

Return type

pynwb.NWBFile

class DeepLabCutInterface(file_path: FilePathType, config_file_path: FilePathType, subject_name: str = 'ind1', verbose: bool = True)

Bases: BaseTemporalAlignmentInterface

Data interface for DeepLabCut datasets.

Interface for writing DLC’s h5 files to nwb using dlc2nwb.

Parameters

• file_path (FilePathType) – path to the h5 file output by dlc.

• config_file_path (FilePathType) – path to .yml config file

• subject_name (str, default: “ind1”) – the name of the subject for which the NWBFile is to be created.

• verbose (bool, default: True) – controls verbosity.

get_metadata()

Child DataInterface classes should override this to match their metadata.

get_original_timestamps() → ndarray

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

get_timestamps() → ndarray

Retrieve the timestamps for the data in this interface.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

set_aligned_timestamps(aligned_timestamps: Union[List, ndarray])

Set aligned timestamps vector for DLC data with user defined timestamps

Parameters

timestamps (list, np.ndarray) – alternative timestamps vector.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None)

Conversion from DLC output files to nwb. Derived from dlc2nwb library.

Parameters

• nwbfile (NWBFile) – nwb file to which the recording information is to be added

• metadata (dict) – metadata info for constructing the nwb file (optional).

FicTrac

class FicTracDataInterface(file_path: FilePathType, radius: Optional[float] = None, configuration_file_path: Optional[FilePathType] = None, verbose: bool = True)

Bases: BaseTemporalAlignmentInterface

Data interface for FicTrac datasets.

Interface for writing FicTrac files to nwb.

Parameters

• file_path (a string or a path) – Path to the .dat file (the output of fictrac)

• radius (float, optional) – The radius of the ball in meters. If provided the radius is stored as a conversion factor and the units are set to meters. If not provided the units are set to radians.

• configuration_file_path (a string or a path, optional) – Path to the .txt file with the configuration metadata. Usually called config.txt

• verbose (bool, default: True) – controls verbosity. True by default.

get_metadata()

Child DataInterface classes should override this to match their metadata.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, compression: Optional[str] = 'gzip', compression_opts: Optional[int] = None)

Parameters

• nwbfile (NWBFile) – nwb file to which the recording information is to be added

• metadata (dict) – metadata info for constructing the nwb file (optional).

get_original_timestamps()

Retrieve and correct timestamps from a FicTrac data file.

This function addresses two specific issues with timestamps in FicTrac data:

1. Resetting Initial Timestamp: In some instances, FicTrac replaces the initial timestamp (0) with the system time. This commonly occurs when the data source is a video file, and OpenCV reports the first timestamp as 0. Since OpenCV also uses 0 as a marker for invalid values, FicTrac defaults to system time in that case. This leads to inconsistent timestamps like [system_time, t1, t2, t3, …]. The function corrects this by resetting the first timestamp back to 0 when a negative difference is detected between the first two timestamps.

2. Re-centering Unix Epoch Time: If timestamps are in Unix epoch time format (time since 1970-01-01 00:00:00 UTC), this function re-centers the time series by subtracting the first timestamp. This adjustment ensures that timestamps represent the elapsed time since the start of the experiment rather than the Unix epoch. This case appears when one of the sources of data in FicTrac (such as PGR or Basler) lacks a timestamp extraction method. FicTrac then falls back to using the system time, which is in Unix epoch format.

Returns

An array of corrected timestamps, in seconds.

Return type

np.ndarray

Notes

• The issue of the initial timestamp replacement appears in FicTrac 2.1.1 and earlier versions.

• Re-centering is essential for timestamps in Unix epoch format as timestamps in an NWB file must be relative

to the start of the session. The heuristic here is to check if the first timestamp is larger than the length of a 10-year experiment in seconds. If so, it’s assumed that the timestamps are in Unix epoch format.

References

Issue discussion on FicTrac’s timestamp inconsistencies: rjdmoore/fictrac#29

get_timestamps()

Retrieve the timestamps for the data in this interface.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

set_aligned_timestamps(aligned_timestamps)

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time)

Align the starting time for this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_starting_time (float) – The starting time for all temporal data in this interface.

extract_session_start_time(file_path: FilePathType, configuration_file_path: Optional[FilePathType] = None) → Optional[datetime]

Extract the session start time from a FicTrac data file or its configuration file.

The session start time is determined from the data file if the timestamps are in Unix epoch format. If not, the function defaults to extracting the date from the configuration file and assuming that the start time is midnight. If neither of these methods works, the function returns None.

The session start time, has two different meanings depending on the source of the FicTrac data: - For video file sources (.avi, .mp4, etc.), the session start time corresponds to the time when the FicTrac analysis commenced. That is, the session start time reflects the analysis time rather than the actual start of the experiment. - For camera sources (such as PGR or Basler), the session start time is either the time reported by the camera or the system time if the camera’s SDK does not provide timestamps to Fictrac. In both cases, this time is the experiment start time, barring synchronization issues.

Parameters

• file_path (FilePathType) – Path to the FicTrac data file.

• configuration_file_path (Optional[FilePathType]) – Path to the FicTrac configuration file. If omitted, the function defaults to searching for “fictrac_config.txt” in the same directory as the data file.

Returns

The session start time of in UTC as a datetime object. None if the session start time cannot be extracted.

Return type

datetime | None

parse_fictrac_config(file_path: FilePathType) → dict

Parse a FicTrac configuration file and return a dictionary of its parameters.

Parameters

file_path (str, Path) – Path to the configuration file in txt format.

Returns

A dictionary where the keys are the parameter names and the values are the parameter values.

Return type

dict

LightningPose

class LightningPoseConverter(file_path: FilePathType, original_video_file_path: FilePathType, labeled_video_file_path: Optional[FilePathType] = None, image_series_original_video_name: Optional[str] = None, image_series_labeled_video_name: Optional[str] = None, verbose: bool = True)

Bases: NWBConverter

Primary conversion class for handling Lightning Pose data streams.

The converter for Lightning Pose format to convert the pose estimation data along with the original and the optional labeled video added as ImageSeries to NWB.

Parameters

• file_path (a string or a path) – Path to the .csv file that contains the predictions from Lightning Pose.

• original_video_file_path (a string or a path) – Path to the original video file (.mp4).

• labeled_video_file_path (a string or a path, optional) – Path to the labeled video file (.mp4).

• image_series_original_video_name (string, optional) – The name of the ImageSeries to add for the original video.

• image_series_labeled_video_name (string, optional) – The name of the ImageSeries to add for the labeled video.

• verbose (bool, default: True) – controls verbosity. True by default.

classmethod get_source_schema()

Compile input schemas from each of the data interface classes.

get_conversion_options_schema() → dict

Compile conversion option schemas from each of the data interface classes.

get_metadata() → DeepDict

Auto-fill as much of the metadata as possible. Must comply with metadata schema.

run_conversion(nwbfile_path: Optional[str] = None, nwbfile: Optional[NWBFile] = None, metadata: Optional[dict] = None, overwrite: bool = False, reference_frame: Optional[str] = None, confidence_definition: Optional[str] = None, external_mode: bool = True, starting_frames_original_videos: Optional[list] = None, starting_frames_labeled_videos: Optional[list] = None, stub_test: bool = False) → None

Run the NWB conversion over all the instantiated data interfaces. :Parameters: * nwbfile_path (FilePathType) – Path for where to write or load (if overwrite=False) the NWBFile.

If specified, the context will always write to this location.

• nwbfile (NWBFile, optional) – An in-memory NWBFile object to write to the location.

• metadata (dict, optional) – Metadata dictionary with information used to create the NWBFile when one does not exist or overwrite=True.

• overwrite (bool, default: False) – Whether to overwrite the NWBFile if one exists at the nwbfile_path. The default is False (append mode).

• conversion_options (dict, optional) – Similar to source_data, a dictionary containing keywords for each interface for which non-default conversion specification is requested.

SLEAP

class SLEAPInterface(file_path: FilePathType, video_file_path: Optional[FilePathType] = None, verbose: bool = True, frames_per_second: Optional[float] = None)

Bases: BaseTemporalAlignmentInterface

Data interface for SLEAP datasets.

Interface for writing sleap .slp files to nwb using the sleap-io library.

Parameters

• file_path (FilePathType) – Path to the .slp file (the output of sleap)

• verbose (bool, default: True) – controls verbosity. True by default.

• video_file_path (FilePath, optional) – The file path of the video for extracting timestamps.

• frames_per_second (float, optional) – The frames per second (fps) or sampling rate of the video.

get_original_timestamps() → ndarray

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

get_timestamps() → ndarray

Retrieve the timestamps for the data in this interface.

Returns

timestamps – The timestamps for the data stream.

Return type

numpy.ndarray

set_aligned_timestamps(aligned_timestamps: ndarray)

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None)

Conversion from DLC output files to nwb. Derived from sleap-io library.

Parameters

• nwbfile (NWBFile) – nwb file to which the recording information is to be added

• metadata (dict) – metadata info for constructing the nwb file (optional).

Video

class VideoInterface(file_paths: list, verbose: bool = False)

Bases: BaseDataInterface

Data interface for writing videos as ImageSeries.

Create the interface for writing videos as ImageSeries.

Parameters

file_paths (list of FilePathTypes) – Many video storage formats segment a sequence of videos over the course of the experiment. Pass the file paths for this videos as a list in sorted, consecutive order.

get_metadata_schema()

Retrieve JSON schema for metadata.

get_metadata()

Child DataInterface classes should override this to match their metadata.

get_original_timestamps(stub_test: bool = False) → List[ndarray]

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns

• timestamps (numpy.ndarray) – The timestamps for the data stream.

• stub_test (bool, default: False) – This method scans through each video; a process which can take some time to complete.

  To limit that scan to a small number of frames, set stub_test=True.

get_timing_type() → Literal['starting_time and rate', 'timestamps']

Determine the type of timing used by this interface.

Returns

timing_type – The type of timing that has been set explicitly according to alignment.

If only timestamps have been set, then only those will be used. If only starting times have been set, then only those will be used.

If timestamps were set, and then starting times were set, the timestamps will take precedence as they will then be shifted by the corresponding starting times.

If neither has been set, and there is only one video in the file_paths, it is assumed the video is regularly sampled and pre-aligned with a starting_time of 0.0 relative to the session start time.

Return type

‘starting_time and rate’ or ‘timestamps’

get_timestamps(stub_test: bool = False) → List[ndarray]

Retrieve the timestamps for the data in this interface.

Returns

• timestamps (numpy.ndarray) – The timestamps for the data stream.

• stub_test (bool, default: False) – If timestamps have not been set to this interface, it will attempt to retrieve them using the .get_original_timestamps method, which scans through each video; a process which can take some time to complete.

  To limit that scan to a small number of frames, set stub_test=True.

set_aligned_timestamps(aligned_timestamps: List[ndarray])

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

aligned_timestamps (list of numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float, stub_test: bool = False)

Align all starting times for all videos in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

• aligned_starting_time (float) – The common starting time for all segments of temporal data in this interface.

• stub_test (bool, default: False) – If timestamps have not been set to this interface, it will attempt to retrieve them using the .get_original_timestamps method, which scans through each video; a process which can take some time to complete.

  To limit that scan to a small number of frames, set stub_test=True.

set_aligned_segment_starting_times(aligned_segment_starting_times: List[float], stub_test: bool = False)

Align the individual starting time for each video (segment) in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters

• aligned_segment_starting_times (list of floats) – The relative starting times of each video.

• stub_test (bool, default: False) – If timestamps have not been set to this interface, it will attempt to retrieve them using the .get_original_timestamps method, which scans through each video; a process which can take some time to complete.

  To limit that scan to a small number of frames, set stub_test=True.

add_to_nwbfile(nwbfile: NWBFile, metadata: Optional[dict] = None, stub_test: bool = False, external_mode: bool = True, starting_frames: Optional[list] = None, chunk_data: bool = True, module_name: Optional[str] = None, module_description: Optional[str] = None, compression: Optional[str] = 'gzip', compression_options: Optional[int] = None)

Convert the video data files to ImageSeries and write them in the NWBFile. Data is written in the ImageSeries container as RGB. [times, x, y, 3-RGB].

Parameters

• nwbfile (NWBFile, optional) – nwb file to which the recording information is to be added

• metadata (dict, optional) – Dictionary of metadata information such as names and description of each video. Metadata should be passed for each video file passed in the file_paths. If storing as ‘external mode’, then provide duplicate metadata for video files that go in the same ImageSeries container. Should be organized as follows:

  metadata = dict(
      Behavior=dict(
          Videos=[
              dict(name="Video1", description="This is the first video.."),
              dict(name="SecondVideo", description="Video #2 details..."),
              ...
          ]
      )
  )
  

  and may contain most keywords normally accepted by an ImageSeries (https://pynwb.readthedocs.io/en/stable/pynwb.image.html#pynwb.image.ImageSeries). The list for the ‘Videos’ key should correspond one to the video files in the file_paths list. If multiple videos need to be in the same ImageSeries, then supply the same value for “name” key. Storing multiple videos in the same ImageSeries is only supported if ‘external_mode’=True.

• stub_test (bool, default: False) – If True, truncates the write operation for fast testing.

• external_mode (bool, default: True) – ImageSeries may contain either video data or file paths to external video files. If True, this utilizes the more efficient method of writing the relative path to the video files (recommended).

• starting_frames (list, optional) – List of start frames for each video written using external mode. Required if more than one path is specified per ImageSeries in external mode.

• chunk_data (bool, default: True) – If True, uses a DataChunkIterator to read and write the video, reducing overhead RAM usage at the cost of reduced conversion speed (compared to loading video entirely into RAM as an array). This will also force to True, even if manually set to False, whenever the video file size exceeds available system RAM by a factor of 70 (from compression experiments). Based on experiments for a ~30 FPS system of ~400 x ~600 color frames, the equivalent uncompressed RAM usage is around 2GB per minute of video. The default is True.

• module_name (str, optional) – Name of the processing module to add the ImageSeries object to. Default behavior is to add as acquisition.

• module_description (str, optional) – If the processing module specified by module_name does not exist, it will be created with this description. The default description is the same as used by the conversion_tools.get_module function.

• compression (str, default: “gzip”) – Compression strategy to use for hdmf.backends.hdf5.h5_utils.H5DataIO. For full list of currently supported filters, see https://docs.h5py.org/en/latest/high/dataset.html#lossless-compression-filters

• compression_options (int, optional) – Parameter(s) for compression filter. Currently, only supports the compression level (integer from 0 to 9) of compression=”gzip”.