Skip to main content

Overview

What callbacks are, streaming vs final delivery, and data completeness guarantees.

Payloads

Callback JSON structure, run vs callback statuses, custom_data, and examples.

Delivery

Idempotency, missed-callback detection, and replaying failed deliveries.

​
Handling Callbacks & Idempotency

Even with a single callback (final mode), you should implement idempotency to handle potential retries. For streaming mode, this is even more critical since you receive multiple callbacks.

​
Best Practices

  • Use callback_ref_uid for deduplication: This stable identifier is the same across retries, making it the recommended way to detect and ignore duplicate callbacks
  • Track processed results using meaningful keys (e.g., linkedin_profile_id) or run_uid/batch_uid
For streaming mode (on: "all"):
  • Aggregate progressively as you receive callbacks with run.status: RUNNING (these contain partial results)
  • Finalize only when you receive a callback with run.status: SUCCEEDED (this indicates the run completed)
  • Verify completeness by comparing the run output_count (from GET /runs/{run_uid}) with the total number of results received in callbacks
  • If counts don’t match, identify and replay missing callbacks or fetch results via the API
For final mode (on: "final"):
  • You receive a single callback indicating the run status (SUCCEEDED, PARTIAL_SUCCEEDED, or FAILED)
  • Fetch all results using GET /runs/{run_uid}/outputs after receiving the callback
  • No aggregation needed - the API returns the complete result set

​
Benefits

This approach allows you to:
  • Stream results progressively (streaming mode) or receive status then fetch results (final mode)
  • Handle partial failures gracefully
  • Ensure data consistency even with retries
  • Verify data completeness using run output count (streaming mode) or API retrieval (final mode)

​
Managing Callbacks

Edges provides endpoints to track, retrieve, and manage your callbacks. Even with a reliable setup, callbacks may not always reach your endpoint, and that is usually invisible from your side — a missed callback means missing data, not an error you can see.
If you rely solely on your callback URL logs, you might never detect missed callbacks, as they were never delivered to your system.

​
Common Reasons for Missed Callbacks

  • Network interruptions between Edges and your callback URL
  • Temporary downtime of your server or API endpoint
  • Gateway or firewall restrictions blocking Edges IPs
  • TLS/SSL handshake issues (expired certificates, protocol mismatch)
  • Slow responses from your server causing timeouts
  • Transient cloud provider issues on either side

​
How to Detect & Resolve Issues

You can use the List Callbacks endpoint to programmatically detect missing or failed callbacks:
  1. Schedule a periodic check (e.g., once per day) to call GET /runs/callbacks filtered by status=FAILED.
For example:
The number of callbacks can be adjusted with the limit parameter, up to 100. You can use the offset param if needed to paginate through results and retrieve all failed callbacks until a date.If you track the run_uid, you can also filter by run_uid to check for specific runs and verify the callbacks were successfully received run by run.
  1. Analyze the http_status field to identify the root cause (e.g., connection refused, timeout). If the callback reached your endpoint but ended with an error, inspect your own server logs to diagnose the issue.
http_status is the HTTP status code returned by your endpoint. It is meaningful to identify what’s happening on your callback URL. Refer to the HTTP Status Codes documentation for more details.While it will be enough to identify most issues, you may need to check your own server logs for more details in some cases:
  • 4xx errors: Client-side issues (e.g., authentication, bad request)
  • 5xx errors: Server-side issues (e.g., internal server error,…)
Note that you may also have “silent” failures if you have logic issues in your callback processing code that do not return an error status code but still fail to process the data correctly. So best practice is to implement explicit error handling on your endpoint and always log the received callbacks and their processing results.
  1. Fix the issue(s) (e.g., adjust firewall, fix SSL, improve server response time).
While you can automatically replay failed callbacks, it’s crucial to understand why they failed first. This avoids flooding your endpoint with retries before the underlying issue is fixed, and keeps you within the replay limit.Use the replay functionality carefully — you can only replay the same callback up to 3 times.
  1. Replay the affected callbacks with POST /runs/callbacks/{callback_uid}/replay.

​
Daily Monitoring Example

Recommended Daily Check: Run a cron job that:
  • Fetches all failed callbacks from the past 24 hours
  • Logs the details for investigation
  • Automatically retries transient failures using the replay endpoint
Regularly checking the callback history helps ensure no data is silently lost and allows you to maintain a robust, fault-tolerant integration.

​
Additional Use Cases for Callback History

  • Post-incident recovery: After downtime, retrieve missed callbacks and replay them to backfill data. You can also use GET /runs/{run_uid}/inputs to recover all input-level data including errors that aren’t available in the outputs endpoint.
  • Audit & compliance: Keep a complete log of all callbacks sent and their statuses for troubleshooting or audits.
  • Performance monitoring: Track the proportion of successful vs failed callbacks over time to improve infrastructure reliability.