Handling incoming Webhook with PHP

Tram Ho

More and more apps offer webhooks as a built-in, often in addition to an API. The classic example, familiar to most developers, is GitHub webhooks that can notify your other systems, such as CI tools, that a new commit has been added to a branch. If you imagine how many repositories exist on GitHub and how many other systems react to per-repository changes… there’s a reason they’re excellent with webhooks! Whether it’s your source control, an update from your IoT sensor, or an event coming from another component of your app, I have some Opinions(TM) on webhook handling , so I thought I’d write them down and include some code, as I think this is an area where many applications will need to work.

Receive and respond

Most of the problems I’ve had or have created when working with incoming webhooks is trying to do too much synchronously – so do all the processing when the hook arrives. This leads to problems for two reasons:

The incoming web connection remains open while processing is in progress. There are a limited number of web connections, so when we run out, the next connection will have to wait, making the system slower…. you see. Kind of what makes up the “hockey” graph shape we see on the web, where everything gets slower and then makes everything else slower and it’s all snowballs.

If something goes wrong in the middle, you have no way to retry that piece of data.

So my advice is to immediately store and then record incoming data, then process it asynchronously. The best solution here is to use a queue, but if adding new dependencies to your application is not simple, then you can totally start with a simple database. Store a record for each incoming webhook, with some sort of unique identifier, a timestamp of when it arrived, maybe some status fields to indicate if it was processed, and the entire data payload webhook when you receive. It can also be useful to put some key fields from the incoming payload into their own columns, such as account number or event type, depending on the type of data you’re dealing with.

Here’s a quick snippet that I use in one of my talks on the subject, which uses PHP to receive an incoming webhook and store it into CouchDB (adjust as required if you’re not using CouchDB). , this will also work perfectly with MySQL , this is just from a project using CouchDB).

This script starts by trying to guess if we have incoming JSON data or a regular form post – and either way creating a $data array that is the incoming payload of the webhook. It also outputs this for debugging purposes, helping to see what has come. If there is any uncertainty about the reliability of the data format or if you are integrating with a third-party system, you may also want to store the actual contents of file_get_contents(“php:/ /input”) verbatim, in case they are needed for debugging or debating who broke what. !

With the data in hand, this script also sets up a $metavariable, with additional fields to store (in this case, just the state and user-agent). The database itself will give our record a unique identifier. Finally, the POST request set up on line 18 will be how we insert data into our database.

It’s not explicitly called here, but when a PHP script completes successfully, it returns a 200 OK response. Note that there are no extra steps here, no validation or checking of fields or fetching more data. Just accept and when it is successfully archived, return “Thanks!” (or rather 200 OK status).

Make a treatment plan

With this data in place, you can handle webhooks asynchronously. If you use a queue instead of a database, then you will set up a few workers to process incoming data. With a solution like the one above, I would recommend doing a cron job to select the unprocessed jobs and actually process the data. You can always return webhooks when they’re done if you need to provide a notification about whether the data was successfully received and processed. One more piece of advice here: set a limit on the number of unprocessed jobs selected and mark them as “in progress”. If the system is under heavy load then more than one of these processes will be useful, so being able to get several jobs waiting for each process would be very helpful!

Hopefully, the example here will help illustrate the point I’ve tried to make about incoming webhooks. For a scalable system, each part of the system wants to be as independent as possible, and the tactics outlined here have worked for me in the past – hopefully, they’re also useful for friend.

#Source: ST

Share the news now

Source : Viblo