Skip to main content
Your instructions are the single most important resource for your labeling team. This is where you move beyond simple definitions and provide the context, examples, and rules needed to handle real-world complexity. Well-crafted instructions are the first line of defense against inconsistent or inaccurate annotations.

​
Create labeling instructions

To add labeling instructions:
  1. Go to your project overview.
  2. Click on Labeling instructions.
  3. In the Instructions section, you may write the instructions directly in the text box, upload a PDF or HTML file, or add a video link.
What to include in your instructions:
  • Detailed Definitions: Go beyond the feature name. What exactly constitutes a “partially occluded vehicle” versus a “fully visible vehicle”?
  • Visual Examples (The Good and The Bad): Show clear examples of correctly labeled data. Just as importantly, show examples of common mistakes or edge cases that should be labeled differently or ignored.
  • Edge Case Guidance: Your data will never be perfect. Provide rules for how to handle blurry images, rare objects, or situations where multiple interpretations are possible.
  • “When in Doubt” Rules: Give your team a clear default action to take when they are unsure, such as flagging the asset for review.
Use the following template as a reference (or download this PDF file).
Changes to instructions are globalRemember that instructions live with the ontology, not the project. A single ontology can be shared across many projects. When you update the instructions for an ontology, those changes will immediately be reflected in every project that uses it.
If you need different instructions for different projects, the correct approach is to create a separate copy of your ontology. Each copy can then have its own unique instructions tailored to that specific project’s needs.

​
Use quizzes for quality

Quizzes are a project-level quality check for labelers and Alignerrs. One common use is confirming that contributors understand your instructions; you can also use quizzes to reinforce quality standards, test edge cases, or reassess contributors after guidelines change. When a quiz is required, labelers must pass it before they can start labeling or start the Hubstaff timer. You can also make a quiz optional so contributors can still begin work if they do not pass. When both instructions and a quiz are due, the contributor workflow is:
  1. Review instructions: The labeler is first prompted to carefully read the instructions you’ve provided.
  2. Take the quiz: After confirming instructions, they answer the quiz questions.
  3. Receive AI-powered feedback: Each answer is scored by an AI on a 1–5 scale. A score of 3 or higher means that individual question is correct.
  4. Pass the quiz: The quiz passes when the share of correctly answered questions meets the configured pass threshold (%). By default the threshold is 100% (every question must pass).
  5. Iterate or proceed: If a labeler fails and retries are still allowed, they can retake the quiz. Pass/fail and scores are always shown; per-question grader feedback is shown when enabled.
Confirming instructions and passing the quiz are independent. Passing the quiz does not confirm instructions, and confirming instructions does not mark the quiz complete. To create and manage a project quiz—including AI generation, attempt policies, required vs optional gating, force retakes, and lockouts—see Quiz.