{ "cells": [ { "cell_type": "markdown", "id": "d0a6c6af", "metadata": {}, "source": [ "# Project-Scoped Dataset Splits Tutorial\n", "\n", "This notebook shows how to use the project-scoped split API in the Datamint Python client. It covers:\n", "\n", "- Listing project split assignments with `api.projects.get_splits()`\n", "- Assigning resources to a split with `api.projects.assign_splits()`\n", "- Inspecting one resource with `api.projects.get_resource_split()`\n", "- Loading a project-backed dataset and using `dataset.split()` with the new project-split mode\n", "\n", "For project-backed datasets, `dataset.split()` now prefers project-scoped assignments when you call it without ratios or explicit mode flags." ] }, { "cell_type": "markdown", "id": "0d6fcd9b", "metadata": {}, "source": [ "## Setup\n", "\n", "Update `datamint` first if needed, then configure your API key." ] }, { "cell_type": "code", "execution_count": null, "id": "c895412f", "metadata": {}, "outputs": [], "source": [ "%pip install -U datamint --quiet" ] }, { "cell_type": "code", "execution_count": null, "id": "57d4d6f7", "metadata": {}, "outputs": [], "source": [ "from datamint import Api\n", "\n", "PROJECT_NAME = \"FracAtlas\"\n", "\n", "api = Api()\n", "project = api.projects.get_by_name(PROJECT_NAME)\n", "if project is None:\n", " raise ValueError(f\"Project '{PROJECT_NAME}' was not found.\")\n", "\n", "project" ] }, { "cell_type": "markdown", "id": "defa40fc", "metadata": {}, "source": [ "## Fetch The Project Resources\n", "\n", "Project-scoped split assignments operate on project membership, so start by fetching the resources that belong to the project." ] }, { "cell_type": "code", "execution_count": null, "id": "2d5611ba", "metadata": {}, "outputs": [], "source": [ "resources = list(project.fetch_resources())\n", "print(f\"Loaded {len(resources)} project resources\")\n", "resources[:3]" ] }, { "cell_type": "markdown", "id": "b8c19409", "metadata": {}, "source": [ "## Assign Project Splits\n", "\n", "This example builds a deterministic `train` / `val` / `test` assignment from the project resource IDs, then writes those assignments back through `api.projects.assign_splits()`.\n", "\n", "Re-running the cell is safe for the listed resources because the split name is reassigned for the IDs you send." ] }, { "cell_type": "code", "execution_count": null, "id": "c2a8b56f", "metadata": {}, "outputs": [], "source": [ "from random import Random\n", "\n", "\n", "def partition_ids(resource_ids: list[str], ratios: dict[str, float], seed: int = 42) -> dict[str, list[str]]:\n", " total = sum(ratios.values())\n", " if abs(total - 1.0) > 1e-6:\n", " raise ValueError(f\"Split ratios must sum to 1.0, got {total}\")\n", "\n", " shuffled_ids = list(resource_ids)\n", " Random(seed).shuffle(shuffled_ids)\n", "\n", " assignments: dict[str, list[str]] = {}\n", " start = 0\n", " items = list(ratios.items())\n", " for index, (split_name, ratio) in enumerate(items):\n", " if index == len(items) - 1:\n", " end = len(shuffled_ids)\n", " else:\n", " end = start + round(ratio * len(shuffled_ids))\n", " assignments[split_name] = shuffled_ids[start:end]\n", " start = end\n", "\n", " return assignments\n", "\n", "resource_ids = [resource.id for resource in resources]\n", "split_ids = partition_ids(resource_ids, {\"train\": 0.7, \"val\": 0.15, \"test\": 0.15}, seed=42)\n", "{name: len(ids) for name, ids in split_ids.items()}" ] }, { "cell_type": "code", "execution_count": null, "id": "a7334f3a", "metadata": {}, "outputs": [], "source": [ "for split_name, split_resource_ids in split_ids.items():\n", " if not split_resource_ids:\n", " continue\n", " api.projects.assign_splits(split_resource_ids, split_name, project=project)\n", "\n", "print(\"Project-scoped split assignments updated.\")" ] }, { "cell_type": "markdown", "id": "726317a1", "metadata": {}, "source": [ "## Inspect Split Assignments\n", "\n", "Use `get_splits()` to list the assignments for the whole project or a single split, and `get_resource_split()` to inspect one resource." ] }, { "cell_type": "code", "execution_count": 9, "id": "b955bbf5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total assignments: 4083\n", "Train assignments: 2858\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", "
\n", "
Entity
\n", "
\n", "

ProjectResourceSplit

\n", "
\n", "
\n", "\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Split Name\n", " test\n", "
Project Id\n", " f97c6fc8-f5fe-4191-a37b-eccfbb81dac7\n", "
Resource Id\n", " 106748f4-7774-43e2-8fb3-017edef01e10\n", "
Created At\n", " 2026-04-21T13:09:34.912Z\n", "
Created By\n", " datamint-dev@mail.com\n", "
\n", "
\n", "\n", "
" ], "text/plain": [ "ProjectResourceSplit(id='8e36bdef-1567-4cf2-b832-eb2e5bf8a1b6', split_name='test', project_id='f97c6fc8-f5fe-4191-a37b-eccfbb81dac7', resource_id='106748f4-7774-43e2-8fb3-017edef01e10', created_at='2026-04-21T13:09:34.912Z', created_by='datamint-dev@mail.com', deleted_at=None, deleted_by=None)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "all_assignments = api.projects.get_splits(project)\n", "train_assignments = api.projects.get_splits(project, split_name=\"train\")\n", "\n", "print(f\"Total assignments: {len(all_assignments)}\")\n", "print(f\"Train assignments: {len(train_assignments)}\")\n", "all_assignments[0]" ] }, { "cell_type": "code", "execution_count": null, "id": "9435723a", "metadata": {}, "outputs": [], "source": [ "example_resource = resources[0]\n", "resource_split = api.projects.get_resource_split(example_resource, project=project)\n", "resource_split" ] }, { "cell_type": "markdown", "id": "2ef2b50c", "metadata": {}, "source": [ "## Load A Dataset And Split It\n", "\n", "Choose the dataset class that matches your project data. This example uses `ImageDataset`; for video or volume projects, swap in `VideoDataset` or `VolumeDataset`.\n", "\n", "For a project-backed dataset, calling `split()` with no arguments now prefers the project split API automatically." ] }, { "cell_type": "code", "execution_count": null, "id": "e8ecd3ad", "metadata": {}, "outputs": [], "source": [ "from datamint.dataset import ImageDataset\n", "\n", "dataset = ImageDataset(\n", " project=project,\n", " include_unannotated=True,\n", ")\n", "\n", "project_parts = dataset.split()\n", "{name: len(part) for name, part in project_parts.items()}" ] }, { "cell_type": "markdown", "id": "9179ee15", "metadata": {}, "source": [ "## Explicit Split Modes\n", "\n", "You can still force either mode explicitly:\n", "\n", "- `use_project_splits=True` to always read the project split API (default for project-backed datasets)\n", "- `as_of_timestamp=\"2026-04-21T12:34:56Z\"` to replay the exact project split snapshot used previously\n", "- ratio kwargs such as `train=0.8, val=0.2` for a local random split" ] }, { "cell_type": "code", "execution_count": null, "id": "22d454cf", "metadata": {}, "outputs": [], "source": [ "explicit_project_parts = dataset.split(use_project_splits=True)\n", "{name: len(part) for name, part in explicit_project_parts.items()}" ] }, { "cell_type": "markdown", "id": "027f6aaf", "metadata": {}, "source": [ "## Reuse A Historical Split Snapshot\n", "\n", "Project-scoped splits now expose the effective snapshot timestamp on each split dataset via `split_as_of_timestamp`. The same value is also carried into MLflow dataset lineage during training, so you can reuse it later to resolve the same split assignment state again." ] }, { "cell_type": "code", "execution_count": null, "id": "4f797e92", "metadata": {}, "outputs": [], "source": [ "split_timestamp = project_parts['train'].split_as_of_timestamp\n", "replayed_parts = dataset.split(as_of_timestamp=split_timestamp)\n", "\n", "print(f\"Replaying project split snapshot from {split_timestamp}\")\n", "{name: len(part) for name, part in replayed_parts.items()}" ] }, { "cell_type": "code", "execution_count": null, "id": "a42412ed", "metadata": {}, "outputs": [], "source": [ "local_parts = dataset.split(train=0.5, val=0.5, seed=7)\n", "{name: len(part) for name, part in local_parts.items()}" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.13" } }, "nbformat": 4, "nbformat_minor": 5 }