Synthetic Data Generation Pipeline for Multi-Task Deep Learning-Based Catheter 3D Reconstruction and Segmentation from Biplanar X-Ray Images

Catheter three-dimensional (3D) position reconstruction is a technology that reconstructs spatial positions from multiple two-dimensional (2D) images. It plays a pivotal role in endovascular surgical navigation, guiding surgical catheters during minimally invasive procedures within vessels. While deep learning approaches have demonstrated significant potential for catheter 3D reconstruction, their clinical applicability is limited due to the lack of annotated datasets. In this work, we propose a synthetic data generation pipeline coupled with a multi-task deep learning framework for simultaneous catheter 3D reconstruction and segmentation from biplanar 2D X-ray images. Our pipeline begins with a novel synthetic data generation methodology that creates realistic catheter datasets with precise ground truth annotations. We next present a combined catheter segmentation and 3D reconstruction architecture, utilizing shared encoder features, in the context of a multi-task deep learning framework. Finally, our work demonstrates the effectiveness of the synthetic data generation method for training deep learning models for 3D reconstruction and segmentation of medical instruments.