A webhook is an endpoint on your server that receives requests from Labelbox. The purpose of a webhook event is to notify your application whenever something interesting happens in one of your Labelbox projects. Rather than periodically polling the Labelbox API to check for any updates in Labelbox, setting up webhooks means your application will receive updates automatically.

For security purposes, Labelbox includes a signature in each webhook event it sends to your application endpoint. Setting up your webhooks configuration to verify the signature in each webhook message allows you to verify that the events were sent by Labelbox. See an example here.

Any traffic coming out of Labelbox will be coming from this IP address: 35.223.142.181. Make sure this IP address is whitelisted when you are setting up your webhooks.

Labelbox offers the following events either per Project or Org-Wide (all projects):

LABEL_CREATED
LABEL_UPDATED
LABEL_DELETED
REVIEW_CREATED
REVIEW_UPDATED

📘
When a review is removed from a label, the REVIEW_UPDATED event will be triggered.

Manage webhooks in the UI

You can manage your webhooks from the application by going to the Webhooks tab in your Account section. Here you can create and manage your webhooks without having the use GraphQL at all.

Manage webhooks via the Python SDK

Code sample

public_url = "https://example.com/webhook-endpoint" # Where the messages will be sent
secret = b"CHANGE-ME" # Use to verify Labelbox is sending the message

client = Client()
project = client.get_project("<project-id>")

# Create a project specific webhook
topics = {topic.value for topic in Webhook.Topic} # Specify 1+ topics
webhook = Webhook.create(
  client,
  topics=topics,
  url=public_url,
  secret=secret.decode(),
  project=project
)

# --- Example Flask Server Code ---

@app.route("/webhook-endpoint", methods=["POST"])
def print_webhook_info():
  
    payload = request.data
    computed_signature = hmac.new(
      secret,
      msg=payload,
      digestmod=hashlib.sha1
    ).hexdigest()
    
    if request.headers["X-Hub-Signature"] != "sha1=" + computed_signature:
        print(
            "Error: computed_signature does not match signature provided in the headers"
        )
        return "Error", 500, 200

    print("=========== New Webhook Delivery ============")
    print("Delivery ID: %s" % request.headers["X-Labelbox-Id"])
    print("Event: %s" % request.headers["X-Labelbox-Event"])
    print("Payload: %s" % json.dumps(json.loads(payload.decode("utf8")), indent=4))
    return "Success"

Sample response

{
    "id": "cl0sjr69j9rrf0z8wffuke9xk",
    "createdAt": "2022-03-15T20:03:25Z",
    "updatedAt": "2022-03-15T20:03:25Z",
    "secondsToLabel": 89.836,
    "label": "{\"objects\":[{\"featureId\":\"cl0skaz9p00043e6erua7q8r4\",\"schemaId\":\"ckyau5pte20n30zai65yzh0st\",\"title\":\"Dog\",\"value\":\"dog\",\"color\":\"#ff0000\",\"bbox\":{\"top\":69,\"left\":86,\"height\":133,\"width\":191},\"instanceURI\":\"https://api.labelbox.com/masks/feature/cl0skaz9p00043e6erua7q8r4\"}],\"classifications\":[],\"relationships\":[]}",
    "agreement": null,
    "benchmarkAgreement": null,
    "deleted": false,
    "project": {
        "id": "cl0sjof3x9r6k0z8w6ntwcrqz",
        "createdAt": "2022-03-15T19:45:50Z",
        "updatedAt": "2022-03-15T19:46:25Z",
        "name": "webhooks",
        "description": "",
        "deleted": false
    },
    "dataRow": {
        "id": "ckyufmlh55jso10t62tvx5zeb",
        "createdAt": "2022-01-25T18:08:35Z",
        "updatedAt": "2022-01-25T18:08:35Z",
        "deletedAt": null,
        "externalId": "dog_0700.jpg",
        "rowData": "https://storage.labelbox.com/ckhmnux5zeutm0825bt0kj34y%2Ffbc2bb6f-efe5-b369-7208-84532368ab51-dog_0700.jpg?Expires=1648584205&KeyName=labelbox-assets-key-3&Signature=gRsiXtEeXpzuoDSCPB9FVT3Tcg4="
    },
    "dataset": {
        "id": "ckyufmdelex2610bmf0c4fkou",
        "createdAt": "2022-01-25T18:08:25Z",
        "updatedAt": "2022-01-25T18:08:35Z",
        "name": "dogs",
        "description": "",
        "deleted": false
    },
    "user": {
        "id": "ckhmnux6hi0p907898jext199",
        "email": "[email protected]"
    }
}

{
    "id": "ckmb8h50t008f3h683tugqsap",
    "createdAt": "2021-03-15T23:44:18Z",
    "updatedAt": "2021-03-17T13:51:59Z",
    "secondsToLabel": 79.173,
    "label": "{\"objects\":[{\"featureId\":\"ckmb8gvd9008b3h68l5tcmrt1\",\"schemaId\":\"ckm3se1a5010y0y6rcdc0bvvl\",\"title\":\"animal\",\"value\":\"animal\",\"color\":\"#1CE6FF\",\"bbox\":{\"top\":325,\"left\":233,\"height\":266,\"width\":413},\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckmb8gvd9008b3h68l5tcmrt1\"},{\"featureId\":\"ckmbdp94300013h68y14zluto\",\"schemaId\":\"ckmbdol5s1ybk0y96fje86ktu\",\"title\":\"human\",\"value\":\"human\",\"color\":\"#FF34FF\",\"bbox\":{\"top\":75,\"left\":527,\"height\":453,\"width\":155},\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckmbdp94300013h68y14zluto\"},{\"featureId\":\"ckmbdpdda00043h688pqrsoku\",\"schemaId\":\"ckmbdol5s1ybk0y96fje86ktu\",\"title\":\"human\",\"value\":\"human\",\"color\":\"#FF34FF\",\"bbox\":{\"top\":81,\"left\":666,\"height\":396,\"width\":134},\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckmbdpdda00043h688pqrsoku\"},{\"featureId\":\"ckmbdpgd200073h68cu3vhjf2\",\"schemaId\":\"ckmbdol5s1ybk0y96fje86ktu\",\"title\":\"human\",\"value\":\"human\",\"color\":\"#FF34FF\",\"bbox\":{\"top\":0,\"left\":722,\"height\":535,\"width\":259},\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckmbdpgd200073h68cu3vhjf2\"},{\"featureId\":\"ckmdi72l700043g68i52bq31i\",\"schemaId\":\"ckm3se1a5010y0y6rcdc0bvvl\",\"title\":\"animal\",\"value\":\"animal\",\"color\":\"#1CE6FF\",\"bbox\":{\"top\":94,\"left\":65,\"height\":104,\"width\":67},\"instanceURI\":\"https://api.labelbox.com/masks/feature/ckmdi72l700043g68i52bq31i\"}],\"classifications\":[{\"featureId\":\"ckmb8h0t7008e3h68hhmuf3y4\",\"schemaId\":\"ckm3se1a4010w0y6rd579gyl3\",\"title\":\"image\",\"value\":\"image\",\"answers\":[{\"featureId\":\"ckmb8h0t6008d3h689yq6cnm6\",\"schemaId\":\"ckm3se1aw01120y6r7zkcd93v\",\"title\":\"contains_human\",\"value\":\"contains_human\"}]}]}",
    "agreement": null,
    "benchmarkAgreement": null,
    "deleted": false,
    "project": {
        "id": "ckm4xyfncfgja0760vpfdxoro",
        "createdAt": "2021-03-11T14:03:12Z",
        "updatedAt": "2021-03-11T18:22:08.523Z",
        "name": "animal_demo_proj",
        "description": "",
        "deleted": false
    },
    "dataRow": {
        "id": "ckm4y6s5a1s7e0rb6bgf00oqa",
        "createdAt": "2021-03-11T14:09:41Z",
        "updatedAt": "2021-03-11T14:09:41Z",
        "deletedAt": null,
        "externalId": "uploaded_images/177.jpg",
        "rowData": "https://storage.labelbox.com/ckk4q1vgapsau07324awnsjq2%2F7b7f4198-3ea4-68db-9bf7-01dfd0bf3e8c-177.jpg?Expires=1617198719&KeyName=labelbox-assets-key-1&Signature=9mjtUoc61rooqsI8I2unQNU4N-o="
    },
    "dataset": {
        "id": "ckm4xyfua04cf0z7a3wz58kgj",
        "createdAt": "2021-03-11T14:03:12Z",
        "updatedAt": "2021-03-11T14:03:12Z",
        "name": "animal_demo_ds",
        "description": "",
        "deleted": false
    },
    "user": {
        "id": "ckm570mgm7q880795an0rfzmz",
        "email": "[email protected]"
    }
}

{
    "id": "cl0sjr9nb9tfi0zatagsq5zw1",
    "createdAt": "2022-03-15T20:07:41Z",
    "updatedAt": "2022-03-16T20:35:57Z",
    "secondsToLabel": 32.4,
    "label": "{\"objects\":[{\"featureId\":\"cl0skgf0j00073e6ez9ihrwj9\",\"schemaId\":\"ckyau5pte20n30zai65yzh0st\",\"title\":\"Dog\",\"value\":\"dog\",\"color\":\"#ff0000\",\"bbox\":{\"top\":48,\"left\":88,\"height\":177,\"width\":162},\"instanceURI\":\"https://api.labelbox.com/masks/feature/cl0skgf0j00073e6ez9ihrwj9\"},{\"featureId\":\"cl0u0vcej00013e6ebfjt42h0\",\"schemaId\":\"ckyau5pte20n30zai65yzh0st\",\"title\":\"Dog\",\"value\":\"dog\",\"color\":\"#ff0000\",\"bbox\":{\"top\":199,\"left\":20,\"height\":108,\"width\":124},\"instanceURI\":\"https://api.labelbox.com/masks/feature/cl0u0vcej00013e6ebfjt42h0\"}],\"classifications\":[],\"relationships\":[]}",
    "agreement": null,
    "benchmarkAgreement": null,
    "deleted": true,
    "project": {
        "id": "cl0sjof3x9r6k0z8w6ntwcrqz",
        "createdAt": "2022-03-15T19:45:50Z",
        "updatedAt": "2022-03-15T19:46:25Z",
        "name": "webhooks",
        "description": "",
        "deleted": false
    },
    "dataRow": {
        "id": "ckyufmlh55jss10t63ve6dk3v",
        "createdAt": "2022-01-25T18:08:35Z",
        "updatedAt": "2022-01-25T18:08:35Z",
        "deletedAt": null,
        "externalId": "dog_0699.jpg",
        "rowData": "https://storage.labelbox.com/ckhmnux5zeutm0825bt0kj34y%2Fae770f47-0d7b-198c-3736-3bdcfbe3e5f5-dog_0699.jpg?Expires=1648672558&KeyName=labelbox-assets-key-3&Signature=dNFjefWzAoLFUsfzVstehZFMLhc="
    },
    "dataset": {
        "id": "ckyufmdelex2610bmf0c4fkou",
        "createdAt": "2022-01-25T18:08:25Z",
        "updatedAt": "2022-01-25T18:08:35Z",
        "name": "dogs",
        "description": "",
        "deleted": false
    },
    "user": {
        "id": "ckhmnux6hi0p907898jext199",
        "email": "[email protected]"
    }
}

{
    "id": "cl0skj9ul9xhm0z8w840sayja",
    "createdAt": "2022-03-15T20:09:50Z",
    "updatedAt": "2022-03-15T20:09:50Z",
    "score": 1,
    "deleted": false,
    "label": {
        "id": "cl0sjr9nb9tfi0zatagsq5zw1"
    },
    "user": {
        "id": "ckhmnux6hi0p907898jext199",
        "email": "[email protected]"
    }
}

{
    "id": "cl0u0xib60xfp0z7794le97rb",
    "createdAt": "2022-03-16T20:36:34Z",
    "updatedAt": "2022-03-16T20:36:45Z",
    "score": -1,
    "deleted": false,
    "label": {
        "id": "cl0sjr69j9rrf0z8wffuke9xk"
    },
    "user": {
        "id": "ckhmnux6hi0p907898jext199",
        "email": "[email protected]"
    }
}

Complete Python SDK tutorial

labelbox-python/webhooks.ipynb at develop · Labelbox/labelbox-python

github.com

labelbox-python/webhooks.ipynb at develop · Labelbox/labelbox-python

Manage webhooks via the GraphQL API

Visit our Webhooks docs in our GraphQL API section to learn how to manage webhooks via the GraphQL API.

FAQ

What is the outgoing rate for webhooks?

The rate would be determined by the configuration (the subscribed topics) and their usage (how many labels are being created)

Is there an upper limit on the size of the webhook payload?

No, because payload is function of the number of annotations on a label and we don’t limit number of annotations on a label.

Does Labelbox use any non-https connections to send webhook events?

All our webhook events are sent in https

Is Content-Type header from the labelbox webhook events only “application/json”?

Yes, the Content-Type will only ever be application/json

Is the “id” nested under the “payload” field unique for each payload?

Event: LABEL_CREATED
Payload: {
    "id": "cl0sjr9nb9tfi0zatagsq5zw1",

Is the order of events from the labelbox webhook guaranteed to be the same as the order in which the label/review was created/updated/deleted?

No we have no guarantees on delivery sequencing