Spider benchmark results

Introduction

We evaluate the performance of the Spider dosimetry pipeline by applying it to publicly available datasets.

Time-integrated activity image

Spider was used to compute a time-integrated activity (TIA) image from the SPECT/CT scans of patient 4 in the SNMMI Lu-177 Dosimetry Challenge 2021 dataset. This benchmark dataset also includes a TIA image that, according to the dataset's documentation, was computed using the proprietary MIM MRT Dosimetry package.

The TIA image from Spider was compared with the TIA image from the benchmark dataset. We calculated the per-pixel ratio of TIA values between the Spider and benchmark images. The distribution of the base-10 logarithm of this ratio is shown in the histogram below. Note that a value of 0 corresponds to equal voxel TIA values.

tia/hist-.svg

If we exclude voxels that have TIA less than 1010 disintegrations mL-1 (≈ 2.8 MBq h mL-1) in the benchmark image, the distribution becomes:

tia/hist-1e10.svg

Provenance

These results were obtained with Spider commit 6da76fd28c502fc54a8408a4c1d7a5e3e93a7ead. To reproduce the experiments, check out this commit of the Spider repository, then run guix build -f benchmark/guix.scm using the following revision of Guix:

(list (channel
        (name 'guix)
        (url "https://git.guix.gnu.org/guix.git")
        (branch "master")
        (commit
          "7cd78468a214f262403c369f1b7a7aa8e8c73a8e")
        (introduction
          (make-channel-introduction
            "9edb3f66fd807b096b48283debdcddccfea34bad"
            (openpgp-fingerprint
              "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"))))
      (channel
        (name 'guix-science)
        (url "https://codeberg.org/guix-science/guix-science.git")
        (branch "master")
        (commit
          "ba3583a812916e368f34dc8bb68390143aa07733")
        (introduction
          (make-channel-introduction
            "b1fe5aaff3ab48e798a4cce02f0212bc91f423dc"
            (openpgp-fingerprint
              "CA4F 8CF4 37D7 478F DA05  5FD4 4213 7701 1A37 8446")))))

To run the experiments on Unix-like operating systems without Guix, build Spider with -DSPIDER_BUILD_BENCHMARKS=ON, then run benchmark/run.sh in the build directory. But running the experiments this way is not reproducible.

License

Copyright © 2026 South Australia Medical Imaging. This document is published under the terms of the CC BY 4.0 license.