We have a growing support team and recently started work on an internal tool to help them better serve our users, this combines data from our main application and Helpscout, which we use extensively. We initially worked with scheduled API imports but quickly felt the need for real-time data coming in from Helpscout via webhooks. Unfortunately they only support setting up a single endpoint for their webhooks, as do some other services. As this endpoint was already in use, this posed a problem.
To solve this issue, we decided to build something to mirror these incoming webhooks to any number of endpoints. Normally with webhooks you will want to verify that an incoming request was actually from the configured service by using a shared secret key to hash the body with, so the goal was to mirror it exactly, using the same headers and POST body.
Trying out Lambda
We actually started out with a solution on AWS
Lambda running on Node, but the
problem there was that you do not get easy access to the headers and request
body, and have to define mapping templates. That ended up slightly changing the
http:\/\/ for example, which of course changes
the verification signature.
So back to the drawing board, Lambda seemed really awesome as you don’t have any servers to take care of, and webhooks really seem to fit nicely, but the Amazon API Gateway that needs to be in front of it felt annoying and tough to configure, we really wanted easy access to the raw request, and configure everything in code over clicking through endless and identical-looking forms.
We decided to write Forwardhook using Go and it’s been serving our needs since. We are running this on AWS Container Services using our provided docker image. We picked Go as we like the benefits of a single binary, cross-compiliation and the low resource usage.
As the goal is quite simple, this tool is as well. It mirrors incoming requests, using the same headers and body, and will retry a failed connection up to 10 times. Failing requests (non-20x) are not retried. Besides this it’s fire and forget, so we are always returning a 200 to the original requester.
Endpoints are configurable via
FORWARDHOOK_SITES, which makes it easy to spin
up multiple containers to point to other endpoints. We have two running at the
moment for our Helpscout accounts as they are currently split up per country. A
possible improvement could be to configure the endpoints via query parameters,
to be able to run a single instance for however many webhooks you need. For
now, we simply run two containers and listen on two seperate ports.
So give it a spin if you ever need to mirror incoming webhooks to multiple locations! :-)
Want to work at the biggest learning source in the world?