Learn how to use in-editor tools like AI critic and code runner to identify grammar errors and validate code in multimodal chat evaluation and prompt-response projects.
Labelbox offers in-editor assistance tools to help identify and fix grammar and code issues for AI model evaluation projects, including multimodal chat evaluation and prompt and response generation. Currently, the following tools are available:
AI critic provides in-editor AI assistance that helps identify grammar issues and code inconsistencies. It can assist in the following example areas:
Both multimodal chat editor and prompt-response generation editor automatically apply AI suggestions when you add or edit prompt and response fields. AI critic is available for all workflow tasks, including labeling, reviewing, and reworking.
To use AI critic for your multimodal chat or prompt and response generation projects:
When you make changes in the Markdown editor, AI critic runs automatically and provides real-time suggestions. You can also click the refresh button to re-generate AI critic comments.
Example AI critic suggestions
Code runner lets you run and test code in model-generated and labeler-written responses for live and offline multimodal chat projects. It detects the programming language, sends the code to the appropriate runtime environment, and provides results in an easily analyzable format, including standard output (stdout), standard error (stderr), execution time, and runtime warnings or errors.
The code runner allows you to pass environment variables (ENV) for credentials to connect with external databases and APIs. It also supports multi-file projects for testing complex code structures. Currently, it supports the following languages:
The code runner clears all execution environments and related resources to ensure security and performance.
To use the code runner for your multimodal chat projects that involve coding:
In the project Overview > Workflow tasks, click Start labeling.
Send a prompt to generate a response that includes code.
Code runner automatically detects the runtime based on your code language. Click CODE > RUN CODE to execute the code or select a different runtime.
View the Outputs field at the bottom of the response to see code execution results or errors. Update your response to fix any issues.
Example code runner outputs
You can use the following code editors integrated into Labelbox when adding your own responses:
Code editors are available when you write your own response for a prompt using the Markdown editor.
To enable the Monaco Editor, add a code block with a specified language in the Markdown editor:
If the syntax is correct, a < > button appears next to the code block. Click this button to open the code editor and modify the block.
To enable the VS Code IDE, click CODE > EDIT IN VS CODE. See Visual Studio Code documentation to get familiar with its offerings and learn how to code faster with the AI Copilot.