{ "cells": [ { "cell_type": "markdown", "id": "3a6a4720", "metadata": {}, "source": [ "# VolumeDataset Tutorial\n", "\n", "This notebook is a comprehensive guide to the `VolumeDataset` class and its 2D slicing functionality in Datamint.\n", "\n", "**What you'll learn:**\n", "1. Loading 3D medical volumes from a Datamint project\n", "2. Inspecting resource metadata\n", "3. Accessing full 3D items and visualizing them\n", "4. Slicing volumes along anatomical axes into 2D datasets\n", "5. Working with segmentation masks on slices\n", "6. Pre-processing with **albumentations**\n", "7. Accessing DICOM metadata from resources" ] }, { "cell_type": "markdown", "id": "e1ae885a", "metadata": {}, "source": [ "## 1. Installation & Imports\n", "\n", "Install the `datamint` package if it is not already available, then import all required modules.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "d955876f", "metadata": {}, "outputs": [], "source": [ "%pip install -U datamint --quiet" ] }, { "cell_type": "code", "execution_count": 8, "id": "17456f16", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "\n", "from datamint.dataset import VolumeDataset" ] }, { "cell_type": "markdown", "id": "b5b16478", "metadata": {}, "source": [ "## 2. Loading a VolumeDataset\n", "\n", "`VolumeDataset` is a PyTorch `Dataset` backed by Datamint. Pass a project name and it will fetch all volumetric resources from that project. Data is loaded lazily — nothing is downloaded until you actually index into the dataset." ] }, { "cell_type": "code", "execution_count": null, "id": "a88d4a18", "metadata": {}, "outputs": [], "source": [ "PROJECT_NAME = \"your-project-name\" # <-- Replace with your project name\n", "\n", "vol_ds = VolumeDataset(\n", " project=PROJECT_NAME,\n", " include_unannotated=True, # Whether to include volumes without any annotations (default: False)\n", " allow_external_annotations=True, # Whether to allow annotations from other projects/sources (default: False)\n", " # return_as_semantic_segmentation=True, # False by default. Converts instance segmentations to semantic segmentation masks.\n", " # semantic_seg_merge_strategy=\"union\"\n", ")\n", "\n", "# alternatively, you can use the build_dataset function that is generic and can be used for both volume and image datasets.\n", "#vol_ds = build_dataset(\n", "# PROJECT_NAME,\n", "# include_unannotated=True, \n", "# allow_external_annotations=True,\n", "#)\n", "\n", "print(vol_ds)\n", "print(f\"\\nNumber of volumes: {len(vol_ds)}\")\n", "print(f\"Segmentation labels: {vol_ds.segmentation_labels_set}\")\n", "print(f\"Image-level labels : {vol_ds.image_labels_set}\")" ] }, { "cell_type": "markdown", "id": "09c2b9ca", "metadata": {}, "source": [ "## 3. Inspecting Volume Resources\n", "\n", "Each `VolumeDataset` exposes a `.resources` list of `Resource` objects. These carry rich metadata — filenames, modality, DICOM UIDs, size, tags and more.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "0a66e49f", "metadata": {}, "outputs": [], "source": [ "vol_ds.resources[0]" ] }, { "cell_type": "markdown", "id": "d843b91c", "metadata": {}, "source": [ "## 4. Accessing 3D Volume Items\n", "\n", "Indexing into `VolumeDataset` returns a dictionary with:\n", "- `'image'` — `np.ndarray` of shape `(C, D, H, W)`\n", "- `'metainfo'` — dict with file-level metadata\n", "- `'masks'` — dict `{annotator_id -> np.ndarray}` of shape `(#instances, D, H, W)` (when `return_segmentations=True`)\n", "- `'image_labels'` — dict `{annotator_id -> Tensor}` of binary label vectors\n" ] }, { "cell_type": "code", "execution_count": null, "id": "36eca250", "metadata": {}, "outputs": [], "source": [ "item = vol_ds[0] # loads and caches the first volume\n", "\n", "img = item['image'] # shape: (C, D, H, W)\n", "print(f\"Image shape : {img.shape} (C, D, H, W)\")\n", "print(f\"Patient ID: {item['metainfo'].get('PatientID')}\")\n", "\n", "if item.get('masks'):\n", " for author, seg in item['masks'].items():\n", " print(f\"\\nSegmentation by '{author}': shape={seg.shape} (instances, D, H, W)\")" ] }, { "cell_type": "code", "execution_count": null, "id": "10e7d093", "metadata": {}, "outputs": [], "source": [ "from datamint.utils.visualization import show\n", "\n", "show(img[0, 0], normalize=True)" ] }, { "cell_type": "markdown", "id": "719c9321", "metadata": {}, "source": [ "## 5. Slicing Volumes into 2D Datasets\n", "\n", "`VolumeDataset.slice()` returns a `SlicedVolumeDataset` that expands every 3D volume into individual 2D slices. Each slice becomes one dataset item with shape `(C, H, W)`.\n", "\n", "Supported axes:\n", "- `'axial'` (or `0`) — superior–inferior slices\n", "- `'coronal'` (or `1`) — anterior–posterior slices\n", "- `'sagittal'` (or `2`) — left–right slices" ] }, { "cell_type": "code", "execution_count": 15, "id": "e4860d05", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Volumes in parent dataset : 12\n", "Total Axial slices : 6144\n" ] } ], "source": [ "# Create a sliced dataset for each anatomical plane\n", "axial_ds = vol_ds.slice(axis='axial')\n", "# coronal_ds = vol_ds.slice(axis='coronal')\n", "# sagittal_ds = vol_ds.slice(axis='sagittal')\n", "\n", "print(f\"Volumes in parent dataset : {len(vol_ds)}\")\n", "print(f\"Total Axial slices : {len(axial_ds)}\")\n" ] }, { "cell_type": "markdown", "id": "26dda52c", "metadata": {}, "source": [ "## 6. Iterating Over 2D Slices\n", "\n", "`SlicedVolumeDataset` is a standard PyTorch `Dataset`, so it works seamlessly with `DataLoader`. Each item is a dict with at least:\n", "- `'image'` — `np.ndarray` or `Tensor` of shape `(C, H, W)`\n", "- `'image_labels'` — dict of annotator → label tensor\n" ] }, { "cell_type": "code", "execution_count": 18, "id": "8eb6d830", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Slice image shape: (1, 512, 94) (C, H, W)\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/luan/Desktop/Datamint/Codes/datamint-python-api/datamint/env/lib/python3.12/site-packages/torchvision/transforms/functional.py:282: RuntimeWarning: invalid value encountered in cast\n", " npimg = (npimg * 255).astype(np.uint8)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFcAAAGFCAYAAACISW/8AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjksIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvJkbTWQAAAAlwSFlzAAAPYQAAD2EBqD+naQAABBpJREFUeJzt2EFq3EgUgOGnpr0U3vdZcoIc2CfITRr6AlradGWTmUDASTOTn5mY7wOtVILHT1EF2tZaa0ic/usBPjJxQ+KGxA2JGxI3JG7o/Mii+/0+t9tt9n2fbdvqmf731lpzHMdcLpc5nX6yP9cDrtfrmhnPD8/1ev1pt4d27r7vMzPzaT7PeZ4e+eRDe5vX+TIvf3d5z0Nx/zoKzvM0503c+fbD4FdHpAstJG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obEDYkbEjckbkjckLghcUPihsQNiRsSNyRuSNyQuCFxQ+KGxA2JGxI3JG5I3JC4IXFD4obOjyxaa83MzNu8zqx0nj/C27zOzPcu73ko7nEcMzPzZV7+5Vgfy3Ec8/z8/O77bf0q/8zc7/e53W6z7/ts2/ZbB/wTrbXmOI65XC5zOr1/sj4Ul3/GhRYSNyRuSNyQuCFxQ+KGvgIKp8p1Hi+c/wAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Single-item access\n", "slice_item = axial_ds[1]\n", "slice_img = slice_item['image'] # shape: (C, H, W)\n", "print(f\"Slice image shape: {slice_img.shape} (C, H, W)\")\n", "show(slice_img[0], normalize=True)" ] }, { "cell_type": "markdown", "id": "e7d4dbd3", "metadata": {}, "source": [ "## 7. Accessing Segmentation Masks\n", "\n", "When `return_segmentations=True` (the default), `SlicedVolumeDataset.__getitem__` returns:\n", "\n", "- `item['masks']` — dict `{annotator_id -> np.ndarray}` of shape `(#instances, H, W)` (instance masks, boolean)\n", "- `item['mask_labels']` — dict `{annotator_id -> np.ndarray}` of integer class codes per instance\n", "\n", "if `return_as_semantic_segmentation=True`, then instance masks are converted to a single semantic mask:\n", "- `item['masks']` — `Tensor` of shape `(num_classes+1, H, W)` with one-hot encoding (background=0, classes start at 1).\n" ] }, { "cell_type": "code", "execution_count": null, "id": "7e961bf5", "metadata": {}, "outputs": [], "source": [ "seg_ds = vol_ds.slice(axis='axial') # return_segmentations is inherited from vol_ds\n", "\n", "seg_item_idx = 3\n", "item = seg_ds[seg_item_idx]\n", "img_slice = item['image'] # (C, H, W)\n", "segs = item['masks'] # dict {author -> (#instances, H, W)}\n", "seg_labels = item.get('mask_labels', {})\n", "\n", "print(f\"Slice index : {seg_ds.resources[seg_item_idx].slice_index}\")\n", "for author, masks in segs.items():\n", " labels = seg_labels.get(author, [])\n", " if (masks==0).all(): # Check if there are any non-empty masks\n", " print(f\" Author '{author}': No segmentation instances (all masks are empty).\")\n", " else:\n", " print(f\" Author '{author}': {masks.shape[0]} instance(s), labels={labels.tolist()}\")\n" ] }, { "cell_type": "markdown", "id": "7b567df8", "metadata": {}, "source": [ "## 8. Pre-processing with Albumentations\n", "\n", "Pass an **albumentations** transform via `alb_transform` to apply augmentations on-the-fly. The library is slice-aware: the same geometric transform is replayed on all segmentation masks so spatial consistency is guaranteed.\n", "\n", "> **Important:** albumentations expects images in `(H, W, C)` format. `VolumeDataset` / `SlicedVolumeDataset` handles the transposition automatically.\n" ] }, { "cell_type": "code", "execution_count": 20, "id": "44c552bb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Augmented dataset: 6144 slices\n" ] } ], "source": [ "import albumentations as A\n", "\n", "# Define an augmentation pipeline for training\n", "train_transform = A.Compose([\n", " # A.RandomCrop(height=256, width=256),\n", " A.HorizontalFlip(p=0.5),\n", " A.RandomBrightnessContrast(p=0.3),\n", " A.GaussNoise(p=0.2),\n", "])\n", "\n", "# Build a dataset with transforms\n", "augmented_vol_ds = VolumeDataset(\n", " project=PROJECT_NAME,\n", " return_segmentations=True,\n", " alb_transform=train_transform,\n", ")\n", "augmented_ds = augmented_vol_ds.slice(axis='axial')\n", "\n", "# You can also set or swap the transform after construction\n", "augmented_ds.set_transform(train_transform)\n", "\n", "print(f\"Augmented dataset: {len(augmented_ds)} slices\")\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b731e999", "metadata": {}, "outputs": [], "source": [ "# Visualize before/after augmentation on the same slice index\n", "IDX = 3\n", "\n", "# Without augmentation (original)\n", "orig_item = axial_ds[IDX]\n", "orig_img = orig_item['image'] # (C, H, W)\n", "\n", "# With augmentation (may be cropped / flipped)\n", "aug_item = augmented_ds[IDX]\n", "aug_img = aug_item['image'] # (C, H_crop, W_crop)\n", "\n", "fig, axes = plt.subplots(1, 2, figsize=(10, 4))\n", "axes[0].imshow(orig_img[0], cmap='gray')\n", "axes[0].set_title(f\"Original {orig_img.shape[1:]}\")\n", "axes[0].axis('off')\n", "\n", "axes[1].imshow(aug_img[0], cmap='gray')\n", "axes[1].set_title(f\"Augmented {aug_img.shape[1:]}\")\n", "axes[1].axis('off')\n", "\n", "plt.suptitle(\"Before vs After Augmentation\", fontsize=13)\n", "plt.tight_layout()\n", "plt.show()\n" ] }, { "cell_type": "markdown", "id": "4baf74ce", "metadata": {}, "source": [ "## 9. Accessing DICOM Metadata\n", "\n", "Every `Resource` (and by delegation, every `SlicedVolumeResource`) exposes structured metadata through its `.metadata` dict and top-level shortcut properties such as `instance_uid`, `series_uid`, `study_uid`, and `patient_id`.\n", "\n", "For richer DICOM tag access, you can load the raw pydicom `Dataset` directly via `resource.fetch_file_data(auto_convert=True)` for DICOM series resources.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "4a75d3b6", "metadata": {}, "outputs": [], "source": [ "# --- High-level shortcut properties available on every Resource ---\n", "for i, resource in enumerate(vol_ds.resources[:2]):\n", " print(f\"\\nResource {i}: {resource.filename}\")\n", " print(f\" modality : {resource.modality}\")\n", " print(f\" instance_uid : {resource.metadata['instance_uid']}\")\n", " print(f\" series_uid : {resource.metadata['series_uid']}\")\n", " print(f\" study_uid : {resource.metadata['study_uid']}\")\n", " print(f\" patient_id : {resource.metadata['patient_id']}\")\n", " print(f\" metadata keys : {list(resource.metadata.keys())}\")" ] }, { "cell_type": "markdown", "id": "ba7368cb", "metadata": {}, "source": [ "---\n", "\n", "## Summary\n", "\n", "| Feature | Code |\n", "|---|---|\n", "| Load 3D dataset | `VolumeDataset(project='...')` |\n", "| Inspect resources | `dataset.resources[i].filename`, `.modality`, `.tags` |\n", "| Access 3D item | `dataset[i]['image']` → `(C, D, H, W)` |\n", "| Slice to 2D | `vol_ds.slice(axis='axial')` |\n", "| 2D item | `sliced_ds[i]['image']` → `(C, H, W)` |\n", "| Segmentation masks | `sliced_ds[i]['masks']` → `{author -> (#inst, H, W)}` |\n", "| Apply transforms | `VolumeDataset(..., alb_transform=my_transform)` |\n", "| DICOM tags | `resource.fetch_file_data(auto_convert=True)` → pydicom Dataset |\n", "| Resource metadata | `resource.metadata` |\n", "| Resource tags | `resource.tags` |\n" ] } ], "metadata": { "kernelspec": { "display_name": "env", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }