Trackr

Automation & Webhooks

Event-driven architecture and automation integrations in Trackr.

Automation & Webhooks in Trackr

Welcome to the comprehensive guide on the Automation and Webhooks systems within the Trackr SaaS application. As a modern tracking platform, Trackr is built with a robust, scalable event-driven architecture at its core. This design philosophy not only ensures high performance and decoupled internal services but also provides our users with powerful tools to connect their Trackr workflows with external platforms seamlessly.

In this document, we will deeply explore the event-driven architecture, the internal dispatcher mechanism, the webhook delivery lifecycle, strategies for retries and dead letter queues, and finally, how to integrate Trackr with popular automation platforms like n8n, Zapier, and Make.

Event-Driven Architecture (EDA)

At the heart of Trackr's responsiveness and scalability is an Event-Driven Architecture (EDA). Rather than relying solely on synchronous, request-response communication between microservices or application modules, Trackr leverages asynchronous event broadcasting. When a significant action occurs within the system—such as a user creating a new job application, moving a candidate to a new stage, or deleting a record—an event is emitted.

Core Principles of Trackr's EDA

  1. Decoupling: Services that produce events (publishers) do not need to know about the services that consume them (subscribers). This allows us to add new features or integrations without modifying the core domain logic.
  2. Asynchrony: The user experience remains fast and fluid because the main request thread is not blocked by secondary tasks like sending emails, updating search indexes, or firing webhooks.
  3. Auditability & Replayability: Events act as a historical log of everything that has happened in the system. This is invaluable for debugging, analytics, and recovery.

Architecture Diagram

Below is a Mermaid diagram illustrating the high-level event flow within Trackr:

graph TD
    subgraph Core Domain
        API[API Gateway / Controllers]
        Service[Domain Services]
        DB[(Primary Database)]
    end
 
    subgraph Event Bus
        Dispatcher{Event Dispatcher}
        Queue[(Message Broker / Redis)]
    end
 
    subgraph Internal Workers
        EmailWorker[Email Notifications]
        SearchWorker[Search Indexing]
        AnalyticsWorker[Analytics Aggregation]
    end
 
    subgraph Webhook Delivery System
        WebhookWorker[Webhook Dispatcher]
        OutboundHttp[HTTP Client]
        DLQ[(Dead Letter Queue)]
    end
 
    API -->|1. Request| Service
    Service -->|2. Persist State| DB
    Service -->|3. Emit Event| Dispatcher
    
    Dispatcher -->|Publish| Queue
    
    Queue -->|Consume| EmailWorker
    Queue -->|Consume| SearchWorker
    Queue -->|Consume| AnalyticsWorker
    Queue -->|Consume| WebhookWorker
    
    WebhookWorker -->|POST Payload| OutboundHttp
    OutboundHttp -->|Success| End((Done))
    OutboundHttp -->|Fail / Retry| WebhookWorker
    WebhookWorker -->|Max Retries Exceeded| DLQ

In this flow, an API request triggers domain logic which first persists state and then emits a domain event (e.g., job_application.created). The Event Dispatcher catches this and places it onto a Message Broker queue. Various internal workers, as well as the Webhook Delivery System, consume these events independently.

The Event Dispatcher

The Event Dispatcher is the central nervous system of Trackr's automation suite. It is responsible for accepting domain events, serializing them into a standard JSON format, and ensuring they are safely committed to our underlying message broker (typically Redis or RabbitMQ in production environments).

Standardized Event Payload

Every event flowing through the dispatcher adheres to a strict schema. This ensures predictability for both internal consumers and external webhook receivers. A typical event payload looks like this:

{
  "event_id": "evt_01H9J2X8P4FGT5R3ZQW1K4M9N",
  "event_type": "job_application.status_updated",
  "timestamp": "2023-10-27T10:30:00Z",
  "tenant_id": "org_55xyz789",
  "actor": {
    "user_id": "usr_99abc123",
    "name": "Jane Doe"
  },
  "data": {
    "application_id": "app_12345",
    "job_title": "Senior Frontend Developer",
    "company": "Tech Innovators Inc.",
    "previous_status": "Interview",
    "new_status": "Offer Extended"
  }
}

This standardized wrapper (event_id, event_type, timestamp, tenant_id, actor) surrounds the event-specific data payload, providing context and preventing consumers from needing to fetch additional data synchronously.

Webhook Delivery Lifecycle

While internal workers process events for Trackr's own functionality, the Webhook Delivery System acts as the bridge to the outside world. It allows users to register their own HTTP endpoints to receive real-time notifications when specific events occur in their Trackr workspace.

Registration and Verification

Users can register webhooks via the Trackr API or the user interface. When registering, they specify:

  • The Target URL (must be HTTPS).
  • The Events they wish to subscribe to (e.g., * for all, or specific ones like candidate.created).
  • An optional Secret Key for payload signing.

To prevent abuse, Trackr requires endpoint verification before sending live data. Upon registration, a challenge request is sent to the target URL, and the endpoint must respond with the correct challenge token.

Payload Security & Signatures

Security is paramount when pushing data across the open internet. Trackr employs HMAC (Hash-based Message Authentication Code) with SHA-256 to sign all webhook payloads.

When a webhook is dispatched, Trackr generates a signature using the user's Secret Key and includes it in the Trackr-Signature HTTP header.

POST /api/webhooks/trackr-events HTTP/1.1
Host: customer-app.com
Content-Type: application/json
Trackr-Signature: t=1698402600,v1=a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4y5z6
Trackr-Event-Id: evt_01H9J2X8P4FGT5R3ZQW1K4M9N

The receiving server is expected to:

  1. Extract the timestamp (t) and the signature (v1).
  2. Recreate the signature base string: timestamp.payload_body.
  3. Compute the HMAC SHA-256 hash using their secret key.
  4. Compare the computed hash with the provided signature to verify authenticity and integrity.

Delivery Diagram

sequenceDiagram
    participant Trackr as Trackr Webhook System
    participant Customer as Customer Endpoint
 
    Trackr->>Customer: POST /webhook (Attempt 1)
    alt Success (200 OK)
        Customer-->>Trackr: 200 OK
        Trackr->>Trackr: Mark Delivery as Successful
    else Failure (e.g., 500 Internal Error or Timeout)
        Customer-->>Trackr: 500 Internal Server Error
        Trackr->>Trackr: Schedule Retry with Backoff
        Note over Trackr,Customer: After Backoff Duration
        Trackr->>Customer: POST /webhook (Attempt 2)
    end

Retries and the Dead Letter Queue (DLQ)

The internet is unreliable. Endpoints go down, rate limits are hit, and network partitions occur. To ensure that our users do not lose critical event data, Trackr implements a robust retry mechanism coupled with a Dead Letter Queue.

Exponential Backoff Retry Strategy

When an HTTP POST request to a webhook endpoint fails (e.g., returns a 4xx or 5xx status code, or times out), the dispatcher does not immediately give up. Instead, the delivery attempt is rescheduled using an exponential backoff algorithm with jitter.

The standard retry schedule is as follows:

  1. Immediate retry: (if connection fails)
  2. Retry 2: 1 minute later
  3. Retry 3: 5 minutes later
  4. Retry 4: 30 minutes later
  5. Retry 5: 2 hours later
  6. Retry 6: 8 hours later
  7. Retry 7: 24 hours later

The inclusion of "jitter" (a small amount of randomization in the retry delay) prevents the "thundering herd" problem, where a recovered server is immediately overwhelmed by a synchronized wave of retried requests.

The Dead Letter Queue (DLQ)

If all retry attempts are exhausted and the delivery still fails, the event is routed to a Dead Letter Queue (DLQ). The DLQ is a specialized, persistent storage area for failed messages.

The purpose of the DLQ is to:

  1. Preserve Data: Ensure that no event data is permanently lost due to a prolonged outage on the receiver's end.
  2. Provide Visibility: Allow Trackr users to inspect failed deliveries via the dashboard. The DLQ view displays the event payload, the target URL, the HTTP status code received, and the complete response body from the failed attempts.
  3. Enable Manual Replay: Once the user has resolved the issue on their end (e.g., fixed a bug in their endpoint or restored server capacity), they can manually select events in the DLQ and initiate a "Replay". This places the events back into the primary dispatcher queue for a fresh delivery attempt.
stateDiagram-v2
    [*] --> InitialDeliveryAttempt
    InitialDeliveryAttempt --> Success: 2xx Status
    InitialDeliveryAttempt --> RetryQueue: Failure/Timeout
    
    RetryQueue --> NextAttempt: Backoff timer expires
    NextAttempt --> Success: 2xx Status
    NextAttempt --> RetryQueue: Failure/Timeout (Count < Max)
    NextAttempt --> DeadLetterQueue: Failure/Timeout (Count >= Max)
    
    DeadLetterQueue --> UserInspection: User views dashboard
    UserInspection --> ManualReplay
    ManualReplay --> InitialDeliveryAttempt: Requeue Event
    
    Success --> [*]

Third-Party Integrations

While custom webhooks are powerful for developers, many users prefer no-code or low-code solutions to automate their workflows. Trackr's event-driven architecture is designed to integrate seamlessly with the holy trinity of automation platforms: n8n, Zapier, and Make (formerly Integromat).

Integrating with n8n

n8n is a fair-code workflow automation tool that appeals to technical users who want to host their own automation infrastructure.

How to integrate:

  1. In your n8n workspace, create a new workflow.
  2. Add a Webhook Node as the trigger.
  3. Set the HTTP Method to POST and configure the authentication if desired (Trackr supports passing custom headers or Basic Auth in the webhook URL).
  4. Copy the generated Webhook URL from n8n.
  5. In the Trackr Dashboard, navigate to Settings > Integrations > Webhooks, and paste the n8n URL.
  6. Select the events you want to route to n8n.
  7. In n8n, click "Listen for Event" and trigger a test event from Trackr to map the JSON payload to subsequent nodes in your workflow.

Integrating with Zapier

Zapier is the industry standard for connecting web apps. Trackr provides a native Zapier app (currently in beta) that simplifies this process.

How to integrate:

  1. Log into Zapier and click Create a Zap.
  2. Search for Trackr in the App Event trigger.
  3. Authenticate your Trackr account using your API key.
  4. Select a specific Trigger Event (e.g., "New Job Application", "Application Stage Changed").
  5. Zapier will automatically configure the webhook in the background.
  6. Pull in sample data to test the trigger, and then connect it to any of Zapier's 5000+ app integrations (e.g., sending a Slack message, updating a Google Sheet, or adding a contact in Salesforce).

Integrating with Make (Integromat)

Make offers a visual, non-linear approach to building integrations, which is excellent for complex logic and data manipulation.

How to integrate:

  1. In Make, create a new Scenario.
  2. Add the Webhooks app and select Custom Webhook.
  3. Click "Add" to create a new webhook and copy the provided URL.
  4. Go to the Trackr Dashboard and register this URL as a new webhook endpoint.
  5. In Make, wait for the webhook to determine the data structure.
  6. Trigger a test event from Trackr. Make will automatically determine the JSON schema of the incoming Trackr event.
  7. You can now drag and drop data fields from the Trackr event into subsequent modules in your Make scenario.

Conclusion

The Automation and Webhooks system in Trackr transforms the platform from a standalone tracking tool into a dynamic, central hub for your entire operational workflow. By embracing an Event-Driven Architecture, ensuring reliable delivery through intelligent retries and Dead Letter Queues, and providing deep integrations with top-tier automation tools, Trackr empowers users to build complex, responsive, and highly efficient business processes with confidence.