Project-Scoped Dataset Splits Tutorial

This notebook shows how to use the project-scoped split API in the Datamint Python client. It covers:

• Listing project split assignments with api.projects.get_splits()

• Assigning resources to a split with api.projects.assign_splits()

• Inspecting one resource with api.projects.get_resource_split()

• Loading a project-backed dataset and using dataset.split() with the new project-split mode

For project-backed datasets, dataset.split() now prefers project-scoped assignments when you call it without ratios or explicit mode flags.

Setup

Update datamint first if needed, then configure your API key.

%pip install -U datamint --quiet

from datamint import Api

PROJECT_NAME = "FracAtlas"

api = Api()
project = api.projects.get_by_name(PROJECT_NAME)
if project is None:
    raise ValueError(f"Project '{PROJECT_NAME}' was not found.")

project

Fetch The Project Resources

Project-scoped split assignments operate on project membership, so start by fetching the resources that belong to the project.

resources = list(project.fetch_resources())
print(f"Loaded {len(resources)} project resources")
resources[:3]

Assign Project Splits

This example builds a deterministic train / val / test assignment from the project resource IDs, then writes those assignments back through api.projects.assign_splits().

Re-running the cell is safe for the listed resources because the split name is reassigned for the IDs you send.

from random import Random


def partition_ids(resource_ids: list[str], ratios: dict[str, float], seed: int = 42) -> dict[str, list[str]]:
    total = sum(ratios.values())
    if abs(total - 1.0) > 1e-6:
        raise ValueError(f"Split ratios must sum to 1.0, got {total}")

    shuffled_ids = list(resource_ids)
    Random(seed).shuffle(shuffled_ids)

    assignments: dict[str, list[str]] = {}
    start = 0
    items = list(ratios.items())
    for index, (split_name, ratio) in enumerate(items):
        if index == len(items) - 1:
            end = len(shuffled_ids)
        else:
            end = start + round(ratio * len(shuffled_ids))
        assignments[split_name] = shuffled_ids[start:end]
        start = end

    return assignments

resource_ids = [resource.id for resource in resources]
split_ids = partition_ids(resource_ids, {"train": 0.7, "val": 0.15, "test": 0.15}, seed=42)
{name: len(ids) for name, ids in split_ids.items()}

for split_name, split_resource_ids in split_ids.items():
    if not split_resource_ids:
        continue
    api.projects.assign_splits(split_resource_ids, split_name, project=project)

print("Project-scoped split assignments updated.")

Inspect Split Assignments

Use get_splits() to list the assignments for the whole project or a single split, and get_resource_split() to inspect one resource.

all_assignments = api.projects.get_splits(project)
train_assignments = api.projects.get_splits(project, split_name="train")

print(f"Total assignments: {len(all_assignments)}")
print(f"Train assignments: {len(train_assignments)}")
all_assignments[0]

Total assignments: 4083
Train assignments: 2858

Entity

ProjectResourceSplit

Split Name	test
Project Id	f97c6fc8-f5fe-4191-a37b-eccfbb81dac7
Resource Id	106748f4-7774-43e2-8fb3-017edef01e10
Created At	2026-04-21T13:09:34.912Z
Created By	datamint-dev@mail.com

example_resource = resources[0]
resource_split = api.projects.get_resource_split(example_resource, project=project)
resource_split

Load A Dataset And Split It

Choose the dataset class that matches your project data. This example uses ImageDataset; for video or volume projects, swap in VideoDataset or VolumeDataset.

For a project-backed dataset, calling split() with no arguments now prefers the project split API automatically.

from datamint.dataset import ImageDataset

dataset = ImageDataset(
    project=project,
    include_unannotated=True,
)

project_parts = dataset.split()
{name: len(part) for name, part in project_parts.items()}

Explicit Split Modes

You can still force either mode explicitly:

• use_project_splits=True to always read the project split API (default for project-backed datasets)

• as_of_timestamp="2026-04-21T12:34:56Z" to replay the exact project split snapshot used previously

• ratio kwargs such as train=0.8, val=0.2 for a local random split

explicit_project_parts = dataset.split(use_project_splits=True)
{name: len(part) for name, part in explicit_project_parts.items()}

Reuse A Historical Split Snapshot

Project-scoped splits now expose the effective snapshot timestamp on each split dataset via split_as_of_timestamp. The same value is also carried into MLflow dataset lineage during training, so you can reuse it later to resolve the same split assignment state again.

split_timestamp = project_parts['train'].split_as_of_timestamp
replayed_parts = dataset.split(as_of_timestamp=split_timestamp)

print(f"Replaying project split snapshot from {split_timestamp}")
{name: len(part) for name, part in replayed_parts.items()}

local_parts = dataset.split(train=0.5, val=0.5, seed=7)
{name: len(part) for name, part in local_parts.items()}