{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7faad80e",
   "metadata": {},
   "source": [
    "# Exploring Data Tutorial\n",
    "\n",
    "This tutorial demonstrates how to explore your data in Datamint. You'll learn how to:\n",
    "\n",
    "- Connect to the Datamint API\n",
    "- List and select projects\n",
    "- Fetch resources from a project\n",
    "- Download and display resource images\n",
    "- Fetch annotations and their segmentation files\n",
    "\n",
    "**Note:** This tutorial focuses on read-only operations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dca275e3",
   "metadata": {},
   "source": [
    "## 1. Setup and API Connection\n",
    "\n",
    "First, let's import the necessary libraries and connect to the Datamint API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0161696e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "\n",
    "from datamint import Api\n",
    "\n",
    "# Initialize the API client\n",
    "# Make sure you have setup your api key\n",
    "api = Api()\n",
    "\n",
    "print(\"✓ Successfully connected to Datamint API\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1129f9d",
   "metadata": {},
   "source": [
    "## 2. Exploring Projects\n",
    "\n",
    "Let's start by listing all available projects and selecting one to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17120f60",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get all projects\n",
    "projects = api.projects.get_all()\n",
    "\n",
    "print(f\"Found {len(projects)} projects:\\n\")\n",
    "for i, project in enumerate(projects, 1):\n",
    "    print(f\"{i}. {project.name}\")\n",
    "    print(f\"   Description: {project.description}\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bee8a5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select a project to work with (by name or index)\n",
    "# Option 1: By name\n",
    "project_name = \"Your Project Name\"  # Replace with your project name\n",
    "selected_project = api.projects.get_by_name(project_name)\n",
    "\n",
    "# Option 2: By index (uncomment to use)\n",
    "# project_index = 0  # First project\n",
    "# selected_project = projects[project_index]\n",
    "\n",
    "if selected_project:\n",
    "    print(f\"Selected project: {selected_project.name}\")\n",
    "else:\n",
    "    print(f\"Project '{project_name}' not found. Please check the project name.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88474cf5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display selected project details in your web browser\n",
    "selected_project.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac0d9aa4",
   "metadata": {},
   "source": [
    "## 3. Getting Resources from a Project\n",
    "\n",
    "Now let's fetch all resources associated with the selected project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9810380e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get all resources from the selected project\n",
    "resources = api.projects.get_project_resources(selected_project)\n",
    "\n",
    "print(f\"Found {len(resources)} resources in project '{selected_project.name}':\\n\")\n",
    "\n",
    "# Display first 5 resources\n",
    "for i, resource in enumerate(resources[:3], 1):\n",
    "    print(f\"{i}. Filename: {resource.filename}\")\n",
    "    print(f\"   Created at: {resource.created_at}\")\n",
    "    print(f\"   Created by: {resource.created_by}\")\n",
    "    print(f\"   MIME type: {resource.mimetype}\")\n",
    "    print(f\"   Status: {resource.status}\")\n",
    "    print()\n",
    "\n",
    "if len(resources) > 3:\n",
    "    print(f\"... and {len(resources) - 3} more resources\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c19d45ce",
   "metadata": {},
   "source": [
    "## 4. Downloading and Displaying Resource Images\n",
    "\n",
    "Let's download and display the image data from a resource using the `download_resource_file` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32f2fdf3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select a resource to work with\n",
    "selected_resource = resources[0]  # First resource\n",
    "\n",
    "print(f\"Selected resource: {selected_resource.filename}\")\n",
    "\n",
    "# Download the resource file with auto-conversion\n",
    "# This will automatically convert DICOM to numpy array, etc.\n",
    "image_data = selected_resource.fetch_file_data(\n",
    "    auto_convert=True,\n",
    "    use_cache=True  # cache at \"~/.datamint/\" for faster access next time. Automatically checks for updates.\n",
    ")\n",
    "\n",
    "print(f\"\\nImage data type: {type(image_data)}\")\n",
    "if isinstance(image_data, np.ndarray):\n",
    "    print(f\"Image shape: {image_data.shape}\")\n",
    "elif hasattr(image_data, 'pixel_array'):  # DICOM\n",
    "    print(f\"DICOM pixel array shape: {image_data.pixel_array.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6a1c631",
   "metadata": {},
   "outputs": [],
   "source": [
    "# show the resource in your web browser\n",
    "selected_resource.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11b62643",
   "metadata": {},
   "source": [
    "This code below caches all resources data in the project locally (~/.datamint/ on Linux by default), so subsequent accesses will be much faster.\n",
    "\n",
    "```python\n",
    "selected_project.cache_resources(progress_bar=True)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b3a9d88",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the image\n",
    "fig, ax = plt.subplots(1, 1, figsize=(10, 10))\n",
    "\n",
    "if isinstance(image_data, np.ndarray):\n",
    "    # NumPy array\n",
    "    if len(image_data.shape) == 3:\n",
    "        # RGB or multi-frame\n",
    "        if image_data.shape[0] == 3:  # RGB format (C, H, W)\n",
    "            ax.imshow(image_data.transpose(1, 2, 0))\n",
    "        else:\n",
    "            # Multi-frame, show first frame\n",
    "            ax.imshow(image_data[0], cmap='gray')\n",
    "    else:\n",
    "        # Grayscale\n",
    "        ax.imshow(image_data, cmap='gray')\n",
    "elif hasattr(image_data, 'pixel_array'):\n",
    "    # DICOM\n",
    "    ax.imshow(image_data.pixel_array[0], cmap='gray')\n",
    "elif isinstance(image_data, Image.Image):\n",
    "    # PIL Image\n",
    "    ax.imshow(image_data)\n",
    "else:\n",
    "    print(f\"Unsupported image format: {type(image_data)}\")\n",
    "\n",
    "ax.set_title(f\"Resource: {selected_resource.filename}\")\n",
    "ax.axis('off')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c109f89",
   "metadata": {},
   "source": [
    "## 5. Getting Annotations for Resources\n",
    "\n",
    "Now let's fetch annotations for our resources, including segmentation files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56a1a323",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get annotations for a single resource\n",
    "annotations = api.annotations.get_list(\n",
    "    resource=selected_resource,\n",
    "    load_ai_segmentations=True\n",
    ")\n",
    "\n",
    "print(f\"Found {len(annotations)} annotations for resource '{selected_resource.filename}':\\n\")\n",
    "\n",
    "for i, annotation in enumerate(annotations, 1):\n",
    "    print(f\"{i}. Annotation ID: {annotation.id}\")\n",
    "    print(f\"   Type: {annotation.type}\")\n",
    "    print(f\"   Identifier: {annotation.identifier}\")\n",
    "    print(f\"   Scope: {annotation.scope}\")\n",
    "    if hasattr(annotation, 'frame_index') and annotation.frame_index is not None:\n",
    "        print(f\"   Frame index: {annotation.frame_index}\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51bc186a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get annotations for multiple resources\n",
    "# This is more efficient than calling get_list for each resource individually\n",
    "multiple_resources = resources[:3]  # First 3 resources\n",
    "\n",
    "# Option 1: Get all annotations in a flat list\n",
    "all_annotations = api.annotations.get_list(\n",
    "    resource=multiple_resources,\n",
    "    load_ai_segmentations=True\n",
    ")\n",
    "\n",
    "print(f\"Found {len(all_annotations)} total annotations across {len(multiple_resources)} resources\\n\")\n",
    "\n",
    "# Option 2: Group annotations by resource\n",
    "grouped_annotations = api.annotations.get_list(\n",
    "    resource=multiple_resources,\n",
    "    load_ai_segmentations=True,\n",
    "    group_by_resource=True\n",
    ")\n",
    "\n",
    "print(\"Annotations grouped by resource:\")\n",
    "for i, (resource, resource_annotations) in enumerate(zip(multiple_resources, grouped_annotations), 1):\n",
    "    print(f\"\\n{i}. Resource: {resource.filename}\")\n",
    "    print(f\"   Number of annotations: {len(resource_annotations)}\")\n",
    "    for j, ann in enumerate(resource_annotations, 1):\n",
    "        print(f\"   {j}. {ann.identifier} ({ann.type})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ab00105",
   "metadata": {},
   "source": [
    "## 6. Downloading and Displaying Segmentation Files\n",
    "\n",
    "Let's download segmentation files for annotations that have them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee7e3cab",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Filter for segmentation annotations\n",
    "segmentation_annotations = [ann for ann in annotations if ann.type == 'segmentation']\n",
    "\n",
    "if segmentation_annotations:\n",
    "    print(f\"Found {len(segmentation_annotations)} segmentation annotation(s)\")\n",
    "    \n",
    "    # Download the first segmentation\n",
    "    seg_annotation = segmentation_annotations[0]\n",
    "    print(f\"\\nDownloading segmentation: {seg_annotation.identifier}\")\n",
    "    \n",
    "    # Download segmentation file\n",
    "    seg_data = seg_annotation.fetch_file_data(use_cache=True) # cache at \"~/.datamint/\" for faster access next time\n",
    "    # seg_data is a numpy array\n",
    "\n",
    "    print(f\"Segmentation shape: {seg_data.shape}\")\n",
    "    print(f\"Unique values: {np.unique(seg_data)}\")\n",
    "else:\n",
    "    print(\"No segmentation annotations found for this resource\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e914677e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the original image and segmentation side by side\n",
    "if segmentation_annotations:\n",
    "    fig, axes = plt.subplots(1, 2, figsize=(15, 7))\n",
    "    \n",
    "    # Original image\n",
    "    if isinstance(image_data, np.ndarray):\n",
    "        if len(image_data.shape) == 3 and image_data.shape[0] == 3:\n",
    "            axes[0].imshow(image_data.transpose(1, 2, 0))\n",
    "        else:\n",
    "            axes[0].imshow(image_data[0] if len(image_data.shape) == 3 else image_data, cmap='gray')\n",
    "    elif hasattr(image_data, 'pixel_array'):\n",
    "        axes[0].imshow(image_data.pixel_array, cmap='gray')\n",
    "    elif isinstance(image_data, Image.Image):\n",
    "        axes[0].imshow(image_data)\n",
    "    \n",
    "    axes[0].set_title(f\"Original: {selected_resource.filename}\")\n",
    "    axes[0].axis('off')\n",
    "    \n",
    "    # Segmentation\n",
    "    axes[1].imshow(seg_data, cmap='jet', alpha=0.7)\n",
    "    axes[1].set_title(f\"Segmentation: {seg_annotation.identifier}\")\n",
    "    axes[1].axis('off')\n",
    "    \n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b75c655",
   "metadata": {},
   "source": "### Quick plots\n\nA resource can plot itself with its annotations, one method per annotation family:\n\n```python\nresource.plot_segmentations()                    # masks over the image, all annotators\nresource.plot_shapes(axis='axial')               # boxes, lines, points and polylines\nresource.plot_labels()                           # categories, labels and values as a text box\n```\n\n- `axis` picks the plane on volumes (`'axial'`, `'coronal'`, `'sagittal'`). `plot_shapes` requires it on volumes; on 2D images it is not needed.\n- `index` picks the slice. If omitted: the slice with the biggest mask, the slice with most shapes, or the middle slice.\n- `ax` is optional. Each method returns the matplotlib `Axes`; pass it to another method to draw on the same image:\n\n```python\nax = resource.plot_segmentations(axis='axial', index=50)\nresource.plot_shapes(axis='axial', index=50, ax=ax)\n```"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67d4af46",
   "metadata": {},
   "outputs": [],
   "source": [
    "# index=None picks the slice with the biggest mask\n",
    "selected_resource.plot_segmentations()\n",
    "plt.show()\n",
    "\n",
    "# volumes: choose plane and slice, and draw shapes on the same axes\n",
    "# ax = selected_resource.plot_segmentations(axis='axial', index=50)\n",
    "# selected_resource.plot_shapes(axis='axial', index=50, ax=ax)\n",
    "# plt.show()\n",
    "\n",
    "#selected_resource.plot_labels()\n",
    "#plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6eecc932",
   "metadata": {},
   "source": [
    "## 7. Bulk Download Multiple Segmentations\n",
    "\n",
    "For efficiency, you can download multiple segmentation files at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ee3b132",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a directory to save segmentations\n",
    "output_dir = \"./downloaded_segmentations\"\n",
    "os.makedirs(output_dir, exist_ok=True)\n",
    "\n",
    "# Download multiple segmentation files\n",
    "if len(segmentation_annotations) > 1:\n",
    "    # Prepare save paths\n",
    "    save_paths = [\n",
    "        os.path.join(output_dir, f\"{ann.identifier}_{ann.id}.png\")\n",
    "        for ann in segmentation_annotations\n",
    "    ]\n",
    "    \n",
    "    # Bulk download (faster than downloading one by one)\n",
    "    results = api.annotations.download_multiple_files(\n",
    "        segmentation_annotations,\n",
    "        save_paths\n",
    "    )\n",
    "    \n",
    "    # Check results\n",
    "    successful = [r for r in results if r['success']]\n",
    "    failed = [r for r in results if not r['success']]\n",
    "    \n",
    "    print(f\"Downloaded {len(successful)} segmentation files successfully\")\n",
    "    if failed:\n",
    "        print(f\"Failed to download {len(failed)} files:\")\n",
    "        for f in failed:\n",
    "            print(f\"  - {f['annotation_id']}: {f.get('error', 'Unknown error')}\")\n",
    "else:\n",
    "    print(\"Not enough segmentations for bulk download demo\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b5a9721",
   "metadata": {},
   "source": [
    "## 8. Filtering Annotations\n",
    "\n",
    "You can filter annotations by various criteria."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "175ee2f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import date, timedelta\n",
    "\n",
    "# Filter by annotation type\n",
    "category_annotations = api.annotations.get_list(\n",
    "    resource=selected_resource,\n",
    "    annotation_type='category'\n",
    ")\n",
    "print(f\"Category annotations: {len(category_annotations)}\")\n",
    "\n",
    "# Filter by date range\n",
    "date_to = date.today()\n",
    "date_from = date_to - timedelta(days=30)  # Last 30 days\n",
    "\n",
    "recent_annotations = api.annotations.get_list(\n",
    "    resource=selected_resource,\n",
    "    from_date=date_from,\n",
    "    to_date=date_to\n",
    ")\n",
    "print(f\"Annotations from last 30 days: {len(recent_annotations)}\")\n",
    "\n",
    "# Filter by status\n",
    "published_annotations = api.annotations.get_list(\n",
    "    resource=selected_resource,\n",
    "    status='published'\n",
    ")\n",
    "print(f\"Published annotations: {len(published_annotations)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa093997",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "In this tutorial, you learned how to:\n",
    "\n",
    "1. ✓ Connect to the Datamint API\n",
    "2. ✓ List and select projects\n",
    "3. ✓ Fetch resources from a project using `get_project_resources()`\n",
    "4. ✓ Download and display resource images using `download_resource_file()`\n",
    "5. ✓ Fetch annotations for resources (single or multiple)\n",
    "6. ✓ Download segmentation files using `download_file()` or `download_multiple_files()`\n",
    "7. ✓ Filter annotations by type, date, and status\n",
    "\n",
    "All operations were read-only, demonstrating how to explore your data without making any modifications."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}