File I/O¶
| Author: | Kyle M. Douglass |
|---|---|
| Contact: | kyle.m.douglass@gmail.com |
| organization: | École Polytechnique Fédérale de Lausanne (EPFL) |
| revision: | $Revision: 0 $ |
| date: | 2017-01-24 |
| abstract: | A brief explanation of file input and output in B-Store. |
Table of Contents
I/O in B-Store¶
Overview¶
B-Store uses two types of objects to read files from the disk that contain localization microscopy data:
- Parsers
- Readers
A Parser reads the filename of a file and assigns IDs to it that are used to identify the dataset inside a HDF file. A Reader reads the actual data contained in the file and converts it to an internal Python datatype. This datatype serves as an intermediary step before saving the data to the HDF file.
Readers are convenience tools that allow the same functionality for reading files to be applied to multiple datasetTypes. For instance, the Localizations, FiducialTracks, and AverageFiducial datasetTypes are all internally represented as Pandas DataFrames. Readers also allow B-Store to be easily extended to new types of files. The purpose in this is that users can store their localization microscopy data regardless of the software program that generated it. All that would be needed would be a new Reader instance that would know how to interpret the data in the files.
Important Note¶
Reader functionality was added in version 1.1.0 and is not yet integrated with all dataset types. This will change in future versions.
There is also no arbitrary file output from the HDF files as of version 1.1.0, except for that generated by CSVBatchProcessor. This too should change in upcoming versions.
Built-in Parsers¶
Two types of Parsers are currently built-in to B-Store:
The SimpleParser interprets files names in the format PREFIX_ACQID.<file_type> where the prefix and acquisition ID are separated by an underscore. The PositionParser splits up a filename by a specified character, e.g. ‘_’, and assigns DatasetIDs based on the elements of the filename occupying integer positions separated by this character, starting with 0 at the left-most position.
Built-in Readers¶
There are currently two Readers built-in to B-Store:
These readers use functionality from Pandas and read generic .csv and .json files, respectively. They are highly customizable with optional parameters provided by Pandas read_csv and read_json functions. This means that anything you can load with these two functions, you can load into B-Store.
Code Examples for Custom Parsers/Readers¶
The following Jupyter notebooks demonstrate how to write custom Parsers and Readers.