Example Queries: Spatial Join

We take the use case of 3D digital pathology image as an example, and present spatial join as a representative spatial query to illustrate the 3D query process in iSPEED.

3D spatial join / cross-matching problem involves identifying and comparing 3D objects from different observations or analyses. In 3D pathology imaging, spatial cross-matching is often used to compare and evaluate 3D image segmentation or reconstruction results, iteratively develop high quality image analysis algorithms, and consolidate multiple analysis results from different approaches to generate more confident results.

Example Data sets

Here we use the dataset of 3D nuclei from pathology image analysis as the test dataset.

Data Preparation

  1. Download data. Download the example dataset to your local directory.
  2. Stage data to HDFS. Run:
    hadoop fs -mkdir your_hdfs_path/3Ddata/spjoin/testdata
    hadoop fs -mkdir your_hdfs_path/3Ddata/spjoin/testdata/d1
    hadoop fs -put your_path/3Ddata1.off your_hdfs_path/3Ddata/spjoin/testdata/d1
    hadoop fs -mkdir your_hdfs_path/3Ddata/spjoin/testdata/d2
    hadoop fs -put your_path/3Ddata2.off your_hdfs_path/3Ddata/spjoin/testdata/d2

Spatial Join Execution

  1. Change directory into the iSPEED base directory.
  2. Download the script run_spatial_join, and change its mode to executable. The detailed explains of commands in the script are in the Features page.
  3. Execute the framework of 3D query processor with the script. Run:
  4. After all Map Reduce jobs are completed, check the results on HDFS: your_hdfs_path/3Ddata/spjoin/testdata/output_joinout.
  5. To list the results, run:
     hadoop fs -ls your_hdfs_path/3Ddata/spjoin/testdata/output_joinout 

    Or to view the results, run:

     hadoop fs -cat your_hdfs_path/3Ddata/spjoin/testdata/output_joinout/part-* 

    Or to download the results from HDFS, run:

     hadoop fs -get  YOUR_HDFS_PATH/containmentout ./