Patient-wise Dataset Splitting

This notebook shows how to use group_by_patient() and split(by_patient=True) on a DICOM dataset.

The Problem: Data Leakage

Standard random splitting shuffles individual resources. If a patient has multiple scans (e.g. a baseline CT and a follow-up CT), both scans can end up in different splits (one in train, one in test). Because the images share patient anatomy, the model effectively sees the test patient during training, inflating metrics.

Patient-wise splitting fixes this by shuffling patients instead of resources: every scan belonging to a patient lands in the same split.

Resource-wise (risky)       Patient-wise (safe)
──────────────────────      ──────────────────────
train: scan_A1, scan_B1     train: patient_A (scan_A1, scan_A2)
test:  scan_A2, scan_B2     test:  patient_B (scan_B1, scan_B2)
       ↑ same patient!             ↑ clean boundary

What You’ll Learn

1. How patient_id is exposed on resources

2. group_by_patient() — organise a dataset by patient for inspection or custom splits

3. split(by_patient=True) — an option for the split

4. How to persist patient-wise splits to the server and reload them reproducibly

5. How to handle resources without a patient_id

1. Setup

We use a small set of DICOM files bundled with pydicom as example data. Each file already carries a PatientID tag in its header.

The upload runs only once: if the project already exists we skip straight to loading the dataset.

from pathlib import Path

import pydicom
import pydicom.data

from datamint import Api
from datamint.dataset import ImageDataset

PROJECT_NAME = "split_save_test"

# 8 patients selected from pydicom's
SELECTED_PATIENTS = {
    '4MR1', '8NM1', '13US1', 'CQ500-CT-310',
    '1CT1', '021234567', '11-05-25-142825', '204',
}

def _collect_dicom_files() -> list[str]:
    data_root = Path(pydicom.data.DATA_ROOT) / 'test_files'
    selected = []
    for f in sorted(data_root.glob('*.dcm')):
        try:
            ds = pydicom.dcmread(str(f), stop_before_pixels=True, force=True)
            if getattr(ds, 'PatientID', None) in SELECTED_PATIENTS:
                selected.append(str(f))
        except Exception:
            pass
    return selected

api = Api()

project = api.projects.get_by_name(PROJECT_NAME)
if project is None:
    project = api.projects.create(PROJECT_NAME, description="Patient Split Demo")
    dicom_files = _collect_dicom_files()
    print(f"Uploading {len(dicom_files)} DICOM files for {len(SELECTED_PATIENTS)} patients...")
    api.resources.upload_resources(
        dicom_files,
        publish_to=project,
        assemble_dicoms=False,
        progress_bar=True,
    )
    print("Upload complete.")
else:
    print(f"Project '{PROJECT_NAME}' already exists, skipping upload.")

dataset = ImageDataset(project=project, include_unannotated=True)
print(f"\nLoaded {len(dataset)} resources")

Project 'split_save_test' already exists, skipping upload.

Loaded 20 resources

2. Inspecting patient_id

For DICOM files the platform extracts PatientID from the DICOM header automatically at upload time and stores it as a top-level field on the resource. No manual assignment needed.

for r in dataset.resources[:5]:
    print(f"{r.filename:40s}  patient_id={r.patient_id!r}")

MR_truncated.dcm                          patient_id='4MR1'
MR_small_RLE.dcm                          patient_id='4MR1'
JPEG2000-embedded-sequence-delimiter.dcm  patient_id='8NM1'
JPEG2000.dcm                              patient_id='8NM1'
MR_small.dcm                              patient_id='4MR1'

3. group_by_patient()

group_by_patient() returns a dict mapping each patient ID to a sub-dataset containing only that patient’s resources.

Handling patient_id=None

Well-formed DICOM files always carry a PatientID tag, so all resources here should have one. In edge cases, anonymised DICOMs where the tag was stripped, patient_id will be None. The none_patient_id_strategy parameter controls what to do:

Strategy	Behaviour
'individual' (default)	Each None-patient resource becomes its own group
'group'	All grouped together under key None
'skip'	Excluded from the output entirely
'error'	Raises ValueError immediately

groups = dataset.group_by_patient()

sizes = [len(g) for g in groups.values()]
print(f"Patient groups : {len(groups)}")
print(f"Total resources: {sum(sizes)}")
print(f"Per-group size : min={min(sizes)}, max={max(sizes)}")
print()
for pid, g in groups.items():
    print(f"  {pid:20s}  {len(g):2d} resource(s)")

Patient groups : 8
Total resources: 20
Per-group size : min=1, max=9

  4MR1                   9 resource(s)
  8NM1                   4 resource(s)
  1CT1                   1 resource(s)
  13US1                  2 resource(s)
  CQ500-CT-310           1 resource(s)
  11-05-25-142825        1 resource(s)
  021234567              1 resource(s)
  204                    1 resource(s)

4. split(by_patient=True) — Compute Patient-wise Splits Locally

When passing by_patient=True, the split shuffles patients instead of individual resources. Internally it shuffles patients, assigns patient buckets to splits by ratio, then collects all resources from each bucket.

parts = dataset.split(
    train=0.7,
    val=0.15,
    test=0.15,
    by_patient=True,
    seed=42,
)

for name, ds in parts.items():
    print(f"{name:6s}: {len(ds):4d} resources")

train :    7 resources
val   :    9 resources
test  :    4 resources

5. Persist Splits to the Server

The parts object returned by split() is a SplitResult — it behaves exactly like a dict, but it also knows how to save itself to the server via .save().

Once persisted, anyone can reload exactly the same split (without re-running the patient split logic) by calling dataset.split() with no arguments.

If the project already has split assignments, .save() raises an error. Pass force=True to overwrite.

parts.save()

# If the project already has splits, this raises:
#   ValueError: Project '...' already has split assignments. Use save(force=True) to overwrite.
# To overwrite:
# parts.save(force=True)

6. Reload Splits from the Server

After persisting, the splits are loaded back just like any project-scoped split, no by_patient, no ratios. The split_as_of_timestamp returned on each sub-dataset can be stored and replayed later to recover the exact same snapshot.

# Reload the dataset fresh (simulates a new session)
dataset_fresh = ImageDataset(project=project, include_unannotated=True)

# No by_patient, no ratios, reads from project API
reloaded = dataset_fresh.split()

for name, ds in reloaded.items():
    print(f"{name:6s}: {len(ds):4d} resources")

val   :    9 resources
test  :    4 resources
train :    7 resources

# Replay the exact same snapshot at any later point
snapshot_ts = reloaded['train'].split_as_of_timestamp

replayed = dataset_fresh.split(as_of_timestamp=snapshot_ts)
print(f"Replaying split snapshot from {snapshot_ts}")
print({name: len(ds) for name, ds in replayed.items()})

Replaying split snapshot from 2026-06-19T16:37:40.622014Z
{'val': 9, 'test': 4, 'train': 7}