Configure HTTP Source

The HTTP Source Source A connection to an external system that Drasi monitors for data changes. Learn more exposes HTTP endpoints that external applications can use to push change events (insert/update/delete) into Drasi Server Drasi Server A standalone Data Change Processing server running as a process or Docker container. Learn more . It supports two mutually exclusive modes:

• Standard Mode — structured event endpoints for applications that produce Drasi change events directly.
• Webhook Mode — configurable routes with template-based payload transformation for receiving arbitrary payloads from external services like GitHub, Shopify, or custom applications.

When to use the HTTP source

• Accept change events from systems that can’t use gRPC (simple HTTP clients, webhooks).
• Build a lightweight ingestion endpoint for custom apps and scripts.
• Feed Drasi from an event bridge that transforms external events into Drasi’s change-event schema.
• Receive webhooks from external services (GitHub, Shopify, Stripe, etc.) and transform them into graph events.
• Connect third-party REST APIs to Drasi continuous queries.

Prerequisites

• Your producer can reach the HTTP endpoint (host:port).
• For Standard Mode: you can generate JSON payloads matching the event schema below.
• For Webhook Mode: you know the payload format of the external service sending webhooks.

Modes

The HTTP source operates in one of two mutually exclusive modes:

Standard Mode

Quick example

Drasi Server source configuration uses camelCase keys.

sources:
  - kind: http
    id: events-api
    autoStart: true

    host: 0.0.0.0
    port: 9000

    # Optional
    endpoint: null
    timeoutMs: 10000

If you run Drasi Server in Docker, remember to publish the HTTP source port:

# docker-compose.yml (snippet)
services:
  drasi-server:
    image: ghcr.io/drasi-project/drasi-server:latest
    ports:
      - "8080:8080"   # Drasi Server REST API
      - "9000:9000"   # HTTP source

Endpoints

The HTTP source exposes:

• GET /health — health check (returns service status and enabled features)
• POST /sources/{sourceId}/events — submit a single event
• POST /sources/{sourceId}/events/batch — submit multiple events

{sourceId} must match the source id from your Drasi Server config (otherwise requests are rejected).

Event schema

Events are JSON objects with a tagged-union operation field:

• operation: "insert" | "update" uses an element (a node or relation).
• operation: "delete" uses id (and optionally labels).

Insert/update: node

{
  "operation": "insert",
  "element": {
    "type": "node",
    "id": "user-123",
    "labels": ["User"],
    "properties": {
      "email": "alice@example.com",
      "age": 30,
      "active": true
    }
  },
  "timestamp": 1738650000000000000
}

Insert/update: relation

{
  "operation": "insert",
  "element": {
    "type": "relation",
    "id": "follows-1",
    "labels": ["FOLLOWS"],
    "from": "user-123",
    "to": "user-456",
    "properties": {
      "since": "2024-01-01"
    }
  }
}

Relationship direction semantics are:

(from)-[relation]->(to)

Delete

{
  "operation": "delete",
  "id": "user-123",
  "labels": ["User"],
  "timestamp": 1738650001000000000
}

Response format

Success:

{
  "success": true,
  "message": "All 2 events processed successfully",
  "error": null
}

Partial success (batch):

{
  "success": true,
  "message": "Processed 8 events successfully, 2 failed",
  "error": "Invalid element type"
}

Error:

{
  "success": false,
  "message": "All 1 events failed",
  "error": "Source name mismatch"
}

curl examples

Single event:

curl -X POST http://localhost:9000/sources/events-api/events \
  -H 'Content-Type: application/json' \
  -d '{
    "operation":"insert",
    "element":{
      "type":"node",
      "id":"test-1",
      "labels":["Test"],
      "properties": {"message":"hello"}
    }
  }'

Batch:

curl -X POST http://localhost:9000/sources/events-api/events/batch \
  -H 'Content-Type: application/json' \
  -d '{
    "events": [
      {"operation":"insert","element":{"type":"node","id":"1","labels":["Test"],"properties":{}}},
      {"operation":"update","element":{"type":"node","id":"1","labels":["Test"],"properties":{"status":"updated"}}}
    ]
  }'

Health check:

curl http://localhost:9000/health

Webhook Mode

Webhook Mode enables the HTTP source to receive arbitrary payloads from external services and transform them into graph events using Handlebars templates.

Quick example

sources:
  - kind: http
    id: webhook-source
    autoStart: true

    host: 0.0.0.0
    port: 9000

    webhooks:
      errorBehavior: acceptAndLog
      routes:
        - path: "/github/events"
          methods: ["POST"]
          auth:
            signature:
              type: hmac-sha256
              secretEnv: GITHUB_WEBHOOK_SECRET
              header: X-Hub-Signature-256
              prefix: "sha256="
          mappings:
            - when:
                header: X-GitHub-Event
                equals: push
              operation: insert
              elementType: node
              effectiveFrom: "{{payload.head_commit.timestamp}}"
              template:
                id: "commit-{{payload.head_commit.id}}"
                labels: ["Commit", "GitHubPush"]
                properties:
                  message: "{{payload.head_commit.message}}"
                  author: "{{payload.head_commit.author.name}}"
                  branch: "{{payload.ref}}"

Webhook configuration options

Top-level settings

CORS configuration

Enable CORS (Cross-Origin Resource Sharing) for browser-based clients:

webhooks:
  cors:
    enabled: true
    allowOrigins: ["*"]
    allowMethods: ["GET", "POST", "PUT", "DELETE", "OPTIONS"]
    allowHeaders: ["Content-Type", "Authorization"]
    exposeHeaders: []
    allowCredentials: false
    maxAge: 3600
  routes:
    # ...

Route settings

Authentication

HMAC signature verification

Used by services like GitHub and Shopify that sign webhook payloads:

auth:
  signature:
    type: hmac-sha256      # or hmac-sha1
    secretEnv: SECRET_VAR  # Environment variable containing the shared secret
    header: X-Hub-Signature-256
    prefix: "sha256="      # Optional prefix to strip before verifying
    encoding: hex          # hex (default) or base64

Bearer token

auth:
  bearer:
    tokenEnv: API_TOKEN    # Environment variable containing the expected token

Mapping settings

Each mapping transforms a matched request into a graph event.

Conditions (when)

Conditions currently support matching against header (HTTP headers) and field (payload fields). Path parameters and query parameters are available in templates via {{route.param}} and {{query.param}}, but cannot be used in when conditions.

# Match an HTTP header value
when:
  header: X-GitHub-Event
  equals: push

# Match a payload field (dot notation for nested fields)
when:
  field: event.type
  equals: order.created

# Match with contains
when:
  header: Content-Type
  contains: json

# Match with regex
when:
  field: action
  regex: "^(created|updated)$"

Template structure

Node template

template:
  id: "{{payload.id}}"
  labels: ["Order", "{{payload.type}}"]
  properties:
    customer: "{{payload.customer.name}}"
    total: "{{payload.total}}"
    items: "{{payload.line_items}}"

Relation template

template:
  id: "rel-{{payload.id}}"
  labels: ["PURCHASED"]
  from: "customer-{{payload.customer_id}}"
  to: "product-{{payload.product_id}}"
  properties:
    quantity: "{{payload.quantity}}"

Spread entire payload as properties

Use a template string instead of an object to spread all fields from the incoming payload as node/relation properties:

template:
  id: "event-{{payload.id}}"
  labels: ["Event"]
  properties: "{{payload}}"

With this configuration, an incoming payload like {"id": "123", "name": "Test", "status": "active"} creates a node with properties id, name, and status.

You can also spread a nested object:

template:
  id: "{{payload.id}}"
  labels: ["Event"]
  properties: "{{payload.data}}"

Template context variables

Templates have access to these variables:

Note: Header names are lowercase in the headers context. Access Content-Type via {{headers.content-type}}.

Template helpers

Property value types

Template values preserve their original types from the payload:

To force a value to a JSON string, use the json helper:

metadata: "{{json payload.complex_object}}"

Timestamp handling (effectiveFrom)

The effectiveFrom field sets the element’s effective timestamp in milliseconds since Unix epoch. It auto-detects format:

Error behavior

Controls how mapping errors are handled:

Set globally in webhooks.errorBehavior or override per-route with routes[].errorBehavior.

Multi-format payload support

The webhook mode parses request bodies based on Content-Type:

Example: GitHub webhooks

webhooks:
  routes:
    - path: "/github/events"
      methods: ["POST"]
      auth:
        signature:
          type: hmac-sha256
          secretEnv: GITHUB_WEBHOOK_SECRET
          header: X-Hub-Signature-256
          prefix: "sha256="
      mappings:
        # Push events
        - when:
            header: X-GitHub-Event
            equals: push
          operation: insert
          elementType: node
          effectiveFrom: "{{payload.head_commit.timestamp}}"
          template:
            id: "commit-{{payload.head_commit.id}}"
            labels: ["Commit"]
            properties:
              message: "{{payload.head_commit.message}}"
              author: "{{payload.head_commit.author.name}}"
              repo: "{{payload.repository.full_name}}"
              branch: "{{payload.ref}}"

        # Issue events
        - when:
            header: X-GitHub-Event
            equals: issues
          operationFrom: payload.action
          elementType: node
          template:
            id: "issue-{{payload.issue.id}}"
            labels: ["Issue"]
            properties:
              title: "{{payload.issue.title}}"
              state: "{{payload.issue.state}}"
              author: "{{payload.issue.user.login}}"

Example: Shopify webhooks

webhooks:
  routes:
    - path: "/shopify/orders"
      methods: ["POST"]
      auth:
        signature:
          type: hmac-sha256
          secretEnv: SHOPIFY_WEBHOOK_SECRET
          header: X-Shopify-Hmac-SHA256
          encoding: base64
      mappings:
        - when:
            header: X-Shopify-Topic
            equals: orders/create
          operation: insert
          elementType: node
          template:
            id: "order-{{payload.id}}"
            labels: ["Order"]
            properties:
              order_number: "{{payload.order_number}}"
              total: "{{payload.total_price}}"
              currency: "{{payload.currency}}"
              customer_email: "{{payload.email}}"
              line_items: "{{payload.line_items}}"

Example: Generic REST API with path parameters

webhooks:
  routes:
    - path: "/api/:resource/:id"
      methods: ["POST", "PUT", "DELETE"]
      auth:
        bearer:
          tokenEnv: API_SECRET_TOKEN
      mappings:
        - when:
            field: resource_type
            equals: users
          operation: insert
          elementType: node
          template:
            id: "user-{{route.id}}"
            labels: ["User"]
            properties:
              name: "{{payload.name}}"
              email: "{{payload.email}}"

Configuration reference (Drasi Server)

Core settings

Adaptive batching settings

Webhook settings

When the webhooks key is present, Webhook Mode is activated and Standard Mode endpoints are disabled.

webhooks (top-level)

webhooks.cors

webhooks.routes[]

webhooks.routes[].auth.signature (HMAC)

webhooks.routes[].auth.bearer

webhooks.routes[].mappings[]

webhooks.routes[].mappings[].when (condition)

Only one of equals, contains, or regex should be specified per condition.

webhooks.routes[].mappings[].template (element template)

Node template:

Relation template:

Adaptive batching

The HTTP source includes intelligent batching that automatically adjusts based on throughput:

Tuning guidelines

• Low latency (real-time dashboards): set adaptiveMaxWaitMs: 10 and adaptiveMinBatchSize: 1.
• High throughput (bulk ingestion): set adaptiveMaxBatchSize: 5000 and adaptiveMaxWaitMs: 500.
• Disable adaptive batching: set adaptiveEnabled: false.

Performance tuning notes

• Use the /events/batch endpoint when you can; it reduces HTTP overhead.
• For bursty workloads, increase adaptiveMaxBatchSize (note: internal buffering scales with this).
• For lower latency, reduce adaptiveMaxWaitMs.
• If you want stable behavior (no adaptive adjustment), set adaptiveEnabled: false.
• Monitor the /health endpoint for production deployments.

Troubleshooting

Connection refused

• Ensure the source is started (autoStart: true or started via the server API).
• Verify Docker port publishing and firewall rules.

400 “Source name mismatch” (Standard Mode)

• The {sourceId} path segment must match the configured id.

404 on standard endpoints

• You have webhooks configured, which activates Webhook Mode and disables standard endpoints. Remove the webhooks key to use Standard Mode.

Invalid event format (Standard Mode)

• Ensure JSON is valid.
• Ensure operation is one of insert, update, delete.
• For insert/update, include element.type, element.id, and element.labels.
• For relations, include from and to.

Authentication failures (Webhook Mode)

• Verify the environment variable referenced by secretEnv / tokenEnv is set and correct.
• For HMAC signatures, ensure the prefix and encoding match what the external service sends.

Template rendering errors (Webhook Mode)

• Ensure Handlebars expressions reference valid paths in the payload.
• Use {{default payload.field "fallback"}} to handle optional fields.

Known limitations

• Standard Mode does not provide built-in authentication/TLS; use a gateway/reverse-proxy if you need them.
• Webhook Mode provides HMAC and Bearer token auth, but not TLS; use a reverse proxy for TLS termination.
• Standard Mode and Webhook Mode are mutually exclusive.

Producer examples

Python

import requests
import json
from datetime import datetime

DRASI_URL = "http://localhost:9000"
SOURCE_ID = "events-api"

def send_event(operation, element=None, element_id=None, labels=None):
    """Send a change event to Drasi HTTP source."""
    url = f"{DRASI_URL}/sources/{SOURCE_ID}/events"
    
    if operation in ("insert", "update"):
        payload = {
            "operation": operation,
            "element": element
        }
    else:  # delete
        payload = {
            "operation": "delete",
            "id": element_id,
            "labels": labels or []
        }
    
    response = requests.post(url, json=payload)
    response.raise_for_status()
    return response.json()

# Insert a new user
send_event("insert", element={
    "type": "node",
    "id": "user-123",
    "labels": ["User"],
    "properties": {
        "name": "Alice",
        "email": "alice@example.com",
        "created_at": datetime.now().isoformat()
    }
})

# Update the user
send_event("update", element={
    "type": "node",
    "id": "user-123",
    "labels": ["User"],
    "properties": {
        "name": "Alice Smith",
        "email": "alice.smith@example.com"
    }
})

# Delete the user
send_event("delete", element_id="user-123", labels=["User"])

Node.js

const axios = require('axios');

const DRASI_URL = 'http://localhost:9000';
const SOURCE_ID = 'events-api';

async function sendEvent(event) {
  const url = `${DRASI_URL}/sources/${SOURCE_ID}/events`;
  const response = await axios.post(url, event);
  return response.data;
}

// Insert a node
await sendEvent({
  operation: 'insert',
  element: {
    type: 'node',
    id: 'order-456',
    labels: ['Order'],
    properties: {
      customer_id: 'user-123',
      total: 99.99,
      status: 'pending'
    }
  }
});

// Send batch of events
const batchUrl = `${DRASI_URL}/sources/${SOURCE_ID}/events/batch`;
await axios.post(batchUrl, {
  events: [
    { operation: 'insert', element: { type: 'node', id: 'item-1', labels: ['Item'], properties: { name: 'Widget' }}},
    { operation: 'insert', element: { type: 'node', id: 'item-2', labels: ['Item'], properties: { name: 'Gadget' }}}
  ]
});

Documentation resources