Labelbox documentation

Webhooks

You can use webhooks to set up workflows that require quick response times. Webhooks are useful for providing automatic notifications when an event happens in Labelbox, such as label updates, reviews, creations, etc. Every time a webhook is triggered, Labelbox will send a POST payload with the relevant information. A notification will be recorded to capture the request and response itself, along with all relevant metadata about the POST. Webhooks can be enabled organization-wide or on specific projects you choose.

This tutorial walks you through the steps to set up a simple webhook workflow with Labelbox.

Before you start

  • Install flask

  • Install ngrok

Setup steps

  1. Create a mock server that represents the endpoint for Labelbox to hit. Copy this code into your text editor and save as server.py. This is your server file. The secret can be anything you wish and should be specified when you create your webhook in step 4.

    from flask import Flask, request
    import json
    import hmac
    import hashlib
    app = Flask(__name__)
    
    @app.route('/')
    def hello_world():
        return 'Hello, World!'
    
    # Make sure this is the same secret you provided in the webhook creation
    secret = 'example_secret'
    
    @app.route('/webhook-endpoint', methods=['POST'])
    def print_webhook_info():
        payload = request.data
        computed_signature = hmac.new(secret, msg=payload, digestmod=hashlib.sha1).hexdigest()
        if request.headers['X-Hub-Signature'] != 'sha1='+computed_signature:
            print('Error: computed_signature does not match signature provided in the headers')
            return 'Error', 500, 200
    
        print('=========== New Webhook Delivery ============')
        print('Delivery ID: %s' % request.headers['X-Labelbox-Id'])
        print('Event: %s' % request.headers['X-Labelbox-Event'])
        print('Payload: %s' % json.dumps(json.loads(payload.decode('utf8')),indent=4))
        return 'Success'
    
    if __name__ == '__main__':
        app.run(host='0.0.0.0', port=3000, debug=True)
  2. Run the file from your terminal and when you visit http://0.0.0.0:3000 you should see “Hello, World!”.

    python server.py
  3. Because Labelbox cannot send messages to an endpoint on your local server, you can either deploy this server or use ngrok to provide a public HTTP proxy to your local endpoint. You should get a public endpoint that looks like this: http://a5625d01fdab.ngrok.io.

    ./ngrok http 3000
  4. Run this 

    createWebhook mutation and append /webhook-endpoint to the end of the URL.

    mutation CreateWebhook {
        createWebhook(data:{
            project:{
                id:"<PROJECT_ID>"
            },
            url:"http://a5625d01fdab.ngrok.io/webhook-endpoint",
            secret:"example_secret",
            topics:{set:[LABEL_CREATED, LABEL_UPDATED, LABEL_DELETED]}
            # topics:{set:[REVIEW_CREATED, REVIEW_UPDATED]}
        }){
            id
        }
    }

    Where:

    • id (OPTIONAL) indicates a specific project for which notifications should be sent. If None, notifications are sent for all events in your organization.

    • url (REQUIRED) is the URL to which notifications should be sent by the Labelbox server.

    • secret (REQUIRED) is the secret you specified in your server file in step 1.

    • topics (REQUIRED) indicates a list of topics this webhook should get notifications for. Must include at least one topic:

      • LABEL_CREATED - user presses the "Submit" button in the editor.

      • LABEL_UPDATED - user presses the "Save" button in the editor.

      • LABEL_DELETED - user deletes a label via the activity table, deletes the

        dataRow that a label is associated with, or deletes a dataset that contains a dataRow that a label is associated with.

      • REVIEW_CREATED - user submits a review in the queue-based review or open review.

      • REVIEW_UPDATED - user updates a review in open review mode.

  5. Test this out by going to your project and create or modify a label to trigger the webhook. You should see the webhook response in your terminal. The following is a sample webhook response for a LABEL_CREATED event.

    =========== New Webhook Delivery ============
    Delivery ID: ckfgcfsjk09jn06624cy7gb26
    Event: LABEL_CREATED
    Payload: {
        "benchmarkAgreement": null,
        "project": {
            "name": "Image project",
            "deleted": false,
            "updatedAt": "2020-09-15T21:19:52Z",
            "id": "ckf4fx97mel7z0729qm72dhyv",
            "createdAt": "2020-09-15T21:00:27Z",
            "description": "Image editor demo"
         },
         "deleted": false,
         "agreement": null,
         "label": "{\"objects\":[{\"featureId\":\"ckfgcff8n01v50z9n54gr2a8a\",\"schemaId\":\"ckf4g0be000fm0y3e8oavh4fv\",\"title\":\"Tree\",\"value\":\"tree\",\"color\":\"#e0ff00\",\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckfgcff8n01v50z9n54gr2a8a\",\"classifications\":[{\"featureId\":\"ckfgcff8n01v60z9neim61644\",\"schemaId\":\"ckf4g0beu00fw0y3ehrrv3u75\",\"title\":\"Is the tree healthy?\",\"value\":\"is_the_tree_healthy?\",\"answer\":{\"featureId\":\"ckfgcff8n01v70z9n2f1p1opu\",\"schemaId\":\"ckf4g0bfe00fy0y3ef8puftut\",\"title\":\"Yes\",\"value\":\"yes\"}}]},{\"featureId\":\"ckfgcfnm901m00y7y9kjybi3p\",\"schemaId\":\"ckf4g0be000fm0y3e8oavh4fv\",\"title\":\"Tree\",\"value\":\"tree\",\"color\":\"#e0ff00\",\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckfgcfnm901m00y7y9kjybi3p\",\"classifications\":[{\"featureId\":\"ckfgcfpyd04al0y8ncvdoc9wl\",\"schemaId\":\"ckf4g0beu00fw0y3ehrrv3u75\",\"title\":\"Is the tree healthy?\",\"value\":\"is_the_tree_healthy?\",\"answer\":{\"featureId\":\"ckfgcfpyy04am0y8n19b0favh\",\"schemaId\":\"ckf4g0bfe00g00y3e1l0c3tc8\",\"title\":\"No\",\"value\":\"no\"}}]}],\"classifications\":[]}",
         "secondsToLabel": 9.581,
         "dataRow": {
            "rowData": "https://storage.labelbox.com/ck52rvx0mpw4g0766wgqvjtl9%2Fb56e90f7-0e9d-05f5-cd5e-37cf32f314d5-red-zeppelin-IjXxL75E1Io-unsplash.jpg?Expires=1602132968&KeyName=labelbox-assets-key-1&Signature=w07YrjJqdRdiMo7pgF2DEFYQogo=",
            "externalId": "red-zeppelin-IjXxL75E1Io-unsplash.jpg",
            "updatedAt": "2020-09-15T21:01:19Z",
            "deletedAt": null,
            "id": "ckf4fyd7i0itb0bo5gjgjavwf",
            "createdAt": "2020-09-15T21:01:19Z"
        },
        "user": {
            "id": "ck52rvx1kqazb07700mq27x4l",
            "email": "alexandra+1@labelbox.com"
        },
        "updatedAt": "2020-09-24T04:56:08Z",
        "dataset": {
            "name": "lawn-drone-view.jpg",
            "deleted": false,
            "updatedAt": "2020-09-15T21:01:19Z",
            "id": "ckf4fycvwf10f0811b6p2cj3b",
            "createdAt": "2020-09-15T21:01:19Z",
            "description": ""
         },
         "id": "ckfgcfsbi00003h5ziwnd8qt7",
         "createdAt": "2020-09-24T04:56:08Z"
    }
    
    127.0.0.1 - - [23/Sep/2020 21:56:08] "POST /webhook-endpoint HTTP/1.1" 200 -

Modify webhooks

To view the GraphQL mutations for creating, updating, and deleting webhooks, see our GraphQL docs on Webhooks.