3DPro - Scalable 3D Data Management and Queries for 3D Tissue Maps at Extreme Scale

High resolution, high throughput imaging technologies are producing unprecedented amount of information at cellular and subcellular level from human tissues. The explosion of 3D spatial objects such as nuclei, cells and vessels pose significant challenges for spatial data management and queries: 1) Explosion of data: a single tissue may contain millions of cells, and a tissue volume may contain tens of millions of 3D objects, with an estimated 37 trillion cells per human body. High throughput is demanded for processing such amounts of data; 2) Data often come from multi-modalities representing different types of knowledge, where image registration and data integration is needed; and 3) The data are represented in complex geometric shapes such as 3D meshes, which are highly computational intensive, in particular, for complex objects such as vessels.

We developed 3DPro, an efficient and scalable 3D data management and querying system for 3D objects. 3DPro represents 3D objects with multi-levels of details (LODs) with progressive compression, with progressive LOD query conditions.

A skeleton based approach is used to preserve the structure for partitioning complex structures such as vessels, which guides the creation of multiple into subobject MBBs for fine grained approximation of the shape. This significantly reduces distance based computations.

3DPro uses a novel Filter-Progressive-Refine paradigm to minimize geometric computation, which can provide early returns of accurate results from lower resolution whenever possible. If a result could not be determined from a LOD, the query will further proceed to the next level of LOD with higher resolution. A memory centered approach is used for data management and querying processing to mitigate I/O cost. Meanwhile, 3DPro is designed for parallelization, supporting both GPU based parallelism and CPU level parallelism.