AI Task Builder

AI Task Builder provides two distinct approaches for human data work: Batches for annotating and evaluating existing data, and Collections for gathering original data from participants.

Both approaches integrate with Prolific’s participant pool and share core functionality like instruction types and quality controls, but they’re designed for fundamentally different workflows.

Batches vs Collections

The key difference comes down to the direction of data flow:

• AI Task Builder Batch: You provide data, participants evaluate it
• AI Task Builder Collection: Participants provide data, you receive it

When to use a Batch

Use an AI Task Builder Batch when you have existing data that needs human judgement. Typical use cases include:

• Labeling and classification — categorizing text, images, or other content
• Model evaluation — rating AI-generated outputs for quality, accuracy, or safety
• Pairwise comparison — selecting the better of two options
• Quality assurance — verifying generated content meets requirements
• Ground truth creation — building labeled datasets for model training

With Batches, you upload a dataset (typically via CSV), and AI Task Builder distributes datapoints across participants via Taskflow. Each participant sees a subset of your data, and you can configure how many annotators evaluate each datapoint.

When to use a Collection

Use an AI Task Builder Collection when you need participants to provide original content. Typical use cases include:

• Image collection — gathering photos for training data (e.g., medical images, handwriting samples)
• Data donation — collecting files or documents from participants
• Structured surveys with uploads — multi-step forms that include file submissions
• Content creation — gathering written responses, recordings, or other original material

With Collections, you define pages containing instructions and optional reference content. All participants complete the same flow, submitting their responses and uploads as they progress through each page.

Core concepts

Both Batches and Collections share some common building blocks:

Instructions

Instructions define what you’re asking participants to do. Available instruction types include:

• Multiple choice — single selection from options
• Checkbox — multiple selections from options
• Free text — open-ended text input
• File upload — participants submit files (images, documents, etc.)

Publishing

Both Batches and Collections are attached to a Prolific study for publishing. When creating the study, you specify:

• data_collection_method: Either AI_TASK_BUILDER_BATCH or AI_TASK_BUILDER_COLLECTION
• data_collection_id: The ID of your Batch or Collection

1
{
2
  "name": "My Study",
3
  "data_collection_method": "AI_TASK_BUILDER_BATCH",
4
  "data_collection_id": "{{BATCH_OR_COLLECTION_ID}}",
5
  // ... other study configuration
6
}

Next steps

By using AI Task Builder, you agree to our AI Task Builder Terms.