TomoFlows: Pre-Processing Workflows For Cryo-Electron Tomography

Common challenges for the data management at regional, national, and international cryo-electron microscopy (cryo-EM) facilities include annotating data, managing the lifecycle of the data, and evaluating data collections quickly. Feedback by pre-processing micrographs on-the-fly and evaluating the quality of the incoming data with metrics such as maximum resolution in CTF estimations, overall motion, and astigmatism estimates inform collection strategies. CryoSPARC-Live [1] and RELION Schedules [2] can provide live data processing with user interaction to select data, configure different processing jobs, and view results. Annotating datasets with metadata to describe sample descriptions, optics parameters, and collection strategies is necessary to provide context for the data collections and tracking of the project work. Additional methods for preprocessing workflows for tomographic data collections would similarly enable facilities to determine the on-going quality of data collections. We have developed Python modules to wrap tasks such as motion correction, commands of IMOD, and machine-learning approaches such as AreTomo [3] to be submitted to a grid computing pool of GPU-aided servers. Non-interactive jobs can be launched as micrographs transfer to a common network storage. Common tasks in the pipeline including conversion of imaging formats, motion correction, CTF estimation, assembly of image stacks, and generation of a tomogram from an image stack. Workflow outputs can produce collections of pre-processed micrographs, image stacks ready to generate a tomogram externally by the user, or preview tomograms with AreTomo. Data processing workflows are being extended with interactive web-based sessions for the selection of the input data, viewing of results, and tracking of results in a database. TomoFlows will be available via GitHub at https://github.com/cemrc-uw-madison/TomoFlows [6] Pipeline Description. Python modules wrap tasks such as motion correction, commands of IMOD, and machine-learning approaches such as AreTomo that are submitted to a computing cluster. Visual UI Adaptation. Design for a web-based UI for an individual task of a processing pipeline, to allow user to specify parameter values and inputs. Each task will be runnable and updates the UI to display progress as the processing step completes.