All page routes must have URL statefulness, for example, if the tab property is specified in the url, the default tab should be that value (if valid) and changes to tabs should update the URL search params accordingly.

Layout (All)

Must be responsive for both mobile and desktop

Display a consistent header that has the company logo on the left, light mode toggle & profile image on the right

Display a collapsible sidebar with links to pages and sub-links in a dropdown (if applicable).

Sign In (/sign-in)

Typical OAuth login screen, authenticate with supabase. Magic link or OAuth options

Dashboard (/)

Display responsive paginated list of projects for user in the form of cards with a loading skeleton using server-side rendering only.

Each project card has:

  • Title
  • Description
  • Task Count
  • (Tasks / Tasks) * 100 With Annotations (displayed as %)
  • Status

When clicking into the project card, navigate to /p/${projectId}

API for projects section:

NOTE

Must get a list of project ids for the user before querying.

1. Get all project ids for the user from supabase

  • await supabase.from(user_projects).select().eq(“user_id”, user.id)

Query parameters

ordering: created_at ids: [stringified ids] page: integer page_size: 18

Returns

{
  "count": 1,
  "results": [
    {
      "id": 1,
      "title": "title",
      "description": "description",
      "label_config": "label_config",
      "expert_instruction": "expert_instruction",
      "show_instruction": true,
      "show_skip_button": true,
      "enable_empty_annotation": true,
      "show_annotation_history": true,
      "organization": 1,
      "color": "color",
      "maximum_annotations": 1,
      "is_published": true,
      "model_version": "model_version",
      "is_draft": true,
      "created_at": "2024-01-15T09:30:00Z",
      "min_annotations_to_start_training": 1,
      "start_training_on_annotation_update": "start_training_on_annotation_update",
      "show_collab_predictions": true,
      "num_tasks_with_annotations": 1,
      "task_number": 1,
      "useful_annotation_number": 1,
      "ground_truth_number": 1,
      "skipped_annotations_number": 1,
      "total_annotations_number": 1,
      "total_predictions_number": 1,
      "sampling": "Sequential sampling",
      "show_ground_truth_first": true,
      "show_overlap_first": true,
      "overlap_cohort_percentage": 1,
      "task_data_login": "task_data_login",
      "task_data_password": "task_data_password",
      "control_weights": {
        "key": "value"
      },
      "parsed_label_config": {
        "key": "value"
      },
      "evaluate_predictions_automatically": true,
      "config_has_control_tags": "config_has_control_tags",
      "skip_queue": "REQUEUE_FOR_ME",
      "reveal_preannotations_interactively": true,
      "pinned_at": "2024-01-15T09:30:00Z",
      "finished_task_number": 1,
      "queue_total": "queue_total",
      "queue_done": "queue_done"
    }
  ],
  "next": "next",
  "previous": "previous"
}

NOTE

If no projects, then display create project button which navigates them to /create-project

Create Project (/create-project)

Create a form that guides the user through creating a project. A project is a configuration that allows a user to upload a data set and start outsourcing annotations.

The form fields are:

  • Title
  • Description (optional)
  • Annotation type (Select)

The label_config and instructions are predefined according to the annotation type. We don’t allow the user to customise the label config for the MVP.

After submit, redirect to /p/${projectId}

Project Page (/p/${projectId})

The project page shows information about a specific project, allows the user to create tasks by uploading media and view tasks and their statuses.

UI/UX: Project stats are displayed in a header, end of header is ‘export’ The stats displayed are (these come directly from the API):

  "num_tasks_with_annotations": 10,
  "task_number": 100,
  "total_annotations_number": 10,

There are 3 tabs

  1. Tasks tab (default if tasks exist) - empty content is a button that navigates to upload tab
  2. Annotations tab - empty content is a button that navigates to upload tab
  3. Upload tab (default if tasks do not exist)

Tasks tab content:

  • Show paginated task rows without annotations and predictions

Annotations tab content:

  • Show paginated task rows with annotations and predictions

Upload tab content:

  • A large drag and drop file upload component that spans the width of the container and about 40% height
  • Separator
  • Header for uploaded file that shows total uploaded files, in progress and total file size
  • Rows of upload data with image preview, created_at, file_size in MB and upload status (progress or check for done)

Before querying the API, perform a check to confirm the user has access to the project

const {data, error} = await supabase.from('user_project').select().eq("", user.id).eq("project_id", urlParams.projectId).single()
if(error) throw error
if(!data) notFound()

Returns (project API)

This is the API response from https://api.labelstud.io/api-reference/api-reference/projects/get

{
  "id": 1,
  "title": "My project",
  "description": "My first project",
  "label_config": "<View>[...]</View>",
  "expert_instruction": "Label all cats",
  "show_instruction": true,
  "show_skip_button": true,
  "enable_empty_annotation": true,
  "show_annotation_history": true,
  "organization": 1,
  "color": "#FF0000",
  "maximum_annotations": 1,
  "is_published": true,
  "model_version": "1.0.0",
  "is_draft": false,
  "created_by": {
    "id": 1,
    "first_name": "Jo",
    "last_name": "Doe",
    "email": "manager@humansignal.com",
    "avatar": "avatar"
  },
  "created_at": "2023-08-24T14:15:22Z",
  "min_annotations_to_start_training": 0,
  "start_training_on_annotation_update": "start_training_on_annotation_update",
  "show_collab_predictions": true,
  "num_tasks_with_annotations": 10,
  "task_number": 100,
  "useful_annotation_number": 10,
  "ground_truth_number": 5,
  "skipped_annotations_number": 0,
  "total_annotations_number": 10,
  "total_predictions_number": 0,
  "sampling": "Sequential sampling",
  "show_ground_truth_first": true,
  "show_overlap_first": true,
  "overlap_cohort_percentage": 100,
  "task_data_login": "user",
  "task_data_password": "secret",
  "control_weights": {
    "key": "value"
  },
  "parsed_label_config": {
    "key": "value"
  },
  "evaluate_predictions_automatically": false,
  "config_has_control_tags": "config_has_control_tags",
  "skip_queue": "REQUEUE_FOR_ME",
  "reveal_preannotations_interactively": true,
  "pinned_at": "2023-08-24T14:15:22Z",
  "finished_task_number": 10,
  "queue_total": "queue_total",
  "queue_done": "queue_done"
}

Returns (tasks API)

this is the API response from https://api.labelstud.io/api-reference/api-reference/tasks/list

{
  "tasks": [
    {
      "id": 1,
      "predictions": [
        {
          "result": [
            {
              "from_name": "sentiment",
              "to_name": "text",
              "type": "choices",
              "value": {
                "value": {
                  "choices": [
                    "POSITIVE"
                  ]
                }
              }
            }
          ],
          "score": 0.9,
          "model_version": "1.0",
          "task": 1,
          "created_at": "2021-01-01T00:00:00Z",
          "updated_at": "2021-01-01T00:00:00Z"
        }
      ],
      "annotations": [
        {
          "result": [
            {
              "from_name": "sentiment",
              "to_name": "text",
              "type": "choices",
              "value": {
                "value": {
                  "choices": [
                    "POSITIVE"
                  ]
                }
              }
            }
          ],
          "created_at": "2021-01-01T00:00:00Z",
          "updated_at": "2021-01-01T00:00:00Z",
          "completed_by": 1,
          "updated_by": 1,
          "was_cancelled": false,
          "ground_truth": false,
          "lead_time": 12.34
        }
      ],
      "drafts": [
        {
          "key": "value"
        }
      ],
      "annotators": [
        1
      ],
      "inner_id": 1,
      "cancelled_annotations": 1,
      "total_annotations": 1,
      "total_predictions": 1,
      "completed_at": "2024-01-15T09:30:00Z",
      "file_upload": "file_upload",
      "storage_filename": "storage_filename",
      "avg_lead_time": 1.1,
      "draft_exists": true,
      "updated_by": [
        {
          "key": "value"
        }
      ],
      "data": {
        "key": "value"
      },
      "meta": {
        "key": "value"
      },
      "created_at": "2024-01-15T09:30:00Z",
      "updated_at": "2024-01-15T09:30:00Z",
      "is_labeled": true,
      "overlap": 1.1,
      "comment_count": 1,
      "unresolved_comment_count": 1,
      "last_comment_updated_at": "2024-01-15T09:30:00Z",
      "project": 1,
      "comment_authors": [
        1
      ]
    }
  ],
  "total": 1,
  "total_annotations": 1,
  "total_predictions": 1
}