We have a growing support team and recently started work on an internal tool to help them better serve our users, this combines data from our main application and Helpscout, which we use extensively. We initially worked with scheduled API imports but quickly felt the need for real-time data coming in from Helpscout via webhooks. Unfortunately they only support setting up a single endpoint for their webhooks, as do some other services. As this endpoint was already in use, this posed a problem.
To solve this issue, we decided to build something to mirror these incoming webhooks to any number of endpoints. Normally with webhooks you will want to verify that an incoming request was actually from the configured service by using a shared secret key to hash the body with, so the goal was to mirror it exactly, using the same headers and POST body.
Trying out Lambda
We actually started out with a solution on AWS Lambda running on Node, but the problem there was that you do not get easy access to the headers and request body, and have to define mapping templates. That ended up slightly changing the body, escaping
http:\/\/for example, which of course changes the verification signature.
So back to the drawing board, Lambda seemed really awesome as you don’t have any servers to take care of, and webhooks really seem to fit nicely, but the Amazon API Gateway that needs to be in front of it felt annoying and tough to configure, we really wanted easy access to the raw request, and configure everything in code over clicking through endless and identical-looking forms.
We decided to write Forwardhook using Go and it’s been serving our needs since. We are running this on AWS Container Services using our provided docker image. We picked Go as we like the benefits of a single binary, cross-compiliation and the low resource usage.
As the goal is quite simple, this tool is as well. It mirrors incoming requests, using the same headers and body, and will retry a failed connection up to 10 times. Failing requests (non-20x) are not retried. Besides this it’s fire and forget, so we are always returning a 200 to the original requester.
Endpoints are configurable via
FORWARDHOOK_SITES, which makes it easy to spin up multiple containers to point to other endpoints. We have two running at the moment for our Helpscout accounts as they are currently split up per country. A possible improvement could be to configure the endpoints via query parameters, to be able to run a single instance for however many webhooks you need. For now, we simply run two containers and listen on two seperate ports.
So give it a spin if you ever need to mirror incoming webhooks to multiple locations! :-)Read more
During my first months at Springest, as a fast-learning junior developer, I will be diving into the workings and usage of popular gems and posting here my experiences with each of them once in a while. Simple Form is the gem of this week!
Forms are essential for user input in web applications. However, they can become tedious to write and maintain because of the need to handle form control naming and its numerous attributes.
That’s where the Simple Form gem comes in handy. It aims to be as flexible as possible while helping you with powerful components to create your forms. And don’t worry, because you will still be able to define your layout the way you do now.
What We Are Building
In this article we will be building a model-centric form for creating and editing companies including the following fields: a
text fieldfor its name, a
text areafor its description, a
select boxfor its size (number of employees) and a
checkboxes collectionfor its different categories.
Defining Relationships Between Models
To be able to build this form we will first need a
categorymodels. Beside these two models we will also have a
characterizationjoin model, which will be in charge of connecting the
categorymodels. Any particular row in the
characterizationstable will effectively join a row in the
companiestable and a row in the
Check out the following diagram showing the models associations:
The following code shows the corresponding validations and how the structure would look like:
Let’s Build the Form
The form will be scoped to the
companymodel and to start using Simple Form you just have to add the helper it provides:
Adding a Text Field and a Text Area
When using the Rails Form Builder you need to specify one by one the type for each of the different form helpers and add a corresponding label to them. For the
text areafields you then have to pass in the name of the attribute to be called on the
companyobject (name and description in this case).
One of the nicest things about Simple Form is that you don’t need to specify the type of the different form elements, but it will figure that out by itself looking at the column type in the database and use an appropriate input for the column. For example, a column created with type
:textin the database (like the company’s description in our case) will use a
textareainput by default. You can check here a complete list of the available input types and defaults for each column type.
A label element will also be added by default to each of the fields, transforming the above code into this:
Adding a Select Box
In the case of a select box, Rails Form Builder will also need the options array to be passed in as a second parameter.
With Simple Form if you want to create a select containing the different sizes of a company you can do it overriding the
:collectionoption. Collections can be arrays or ranges, and when a
:collectionis given the
:selectinput will be rendered by default, so no need to pass the
Adding a Checkboxes Collection
To generate a collection of checkboxes with Rails Form Builder we will need to pass in the following parameters to the form helper:
- Name of the attribute or method to be called on the
companyobject (as in the previous form elements)
- Options array or collection to be shown
- Name of the attribute to be used as the checkbox value
- Name of the attribute to be used as the checkbox label text
To deal with these type of associations, Simple Form can generate either select inputs or a series of radios buttons or checkboxes. We will only need to pass in the
categoriescollection as a parameter. The association helper would render a
:selectinput by default, which can be changed into radio buttons or checkboxes like we did in this case:
What We Got
Our form will end up looking very short and simple indeed ☺ :
If you didn’t have enough and you are actually interested in doing more complicated things with Simple Form, you will be happy to hear that:
- You can create your own custom inputs
- You can also add some extra helpers
- And it works nicely with the country_select gem
- Name of the attribute or method to be called on the
Elastic Beanstalk offers a lot of bang for your buck. Deployment, auto scaling, load balancing and database management are just some of the features Elastic Beanstalk offers out-of-the-box.
It’s hard to find a software platform that isn’t supported by Elastic Beanstalk, it currently offers support for Ruby, PHP, Python, Java, .NET, Node.JS and Go. But it doesn’t stop there, for every platform there is a multitude of platform versions and servers to run your application. Just for the Ruby environment you can choose between several versions of Ruby in combination with Passenger or Puma.
But it doesn’t stop there! Elastic Beanstalk has one more trick up it’s sleeve: The ability to run your application inside a Docker container.
There are a lot of advantages to using Docker for running your application. A very incomplete list:
- Less overhead on the environment running your application.
- Test and run your application in the same environment.
- Easy to debug. You can export a faulty container, download it and start it locally.
- Simplify configuration, less need for customising the host environment using .ebextensions.
- Faster deployments and auto scaling.
We had been experimenting with using Docker to run our stack for a while. Our first try was using Deis to run our staging environment. We saw a lot of potential and spent quite a lot of time on getting our stack to work but, in the end we simply didn’t feel comfortable with the level of stability Deis was offering.
This made us little weary of trying it again on Elastic Beanstalk but, after some initial experimentation we were sold!
If you are interested in knowing more about our experiences using Deis please let us know in the comments. If we receive enough interest we might devote an article to the subject.
There’s more than one way to go about deploying an application on an Elastic Beanstalk Docker environment. First of all, you have the choice between running a single or multi-docker container setup. For our purpose the Single Docker setup works just fine.
First, we are going to need a base docker image that we can use for our deployments:
You should of course replace
awesome_appwith the name of your own organisation on the Docker repository.
Next we will create the
Dockerrun.aws.jsonfile. This file holds the configuration needed for Elastic Beanstalk to deploy your Docker container:
The file includes the following sections:
AWSEBDockerrunVersion- Specifies the version number. 1 for single and 2 for multi.
Ports- Which ports to expose on the Docker container.
Volumes- Which volumes should be mounted from host to the Docker container.
Logging- Where to mount the host log directory to the docker container. The host log directory can be found here:
If you are already familiar with the structure of the
Dockerrun.aws.jsonyou might notice we left out the
Imagesection. The above
Dockerfilewill be used as a base for a Docker image that will be built during deployment by Elastic Beanstalk. For this we need to create a different Dockerfile which will be sent along with all other application files to Elastic Beanstalk during deployment.
Create the following
Dockerfileand place it in
In this example we start the application by running
pumabut you could just as easily start any other application server.
For the final step we will need to pull it all together. We will be deploying from inside the
awesome-app/base:0.1docker container. For this we will create a simple script called
In the example the EC2 region has been hardcoded to
eu-central-1but this can be changed to the region of your liking.
The script creates two files needed by the Elastic Beanstalk CLI to deploy your application:
- .elasticbeanstalk/config.yml - This file specifies the target environment for deployment
- /root/.aws/config - This file specifies the AWS credentials file needed for authentication
For the script to work it will need two environment variables:
These can be generated from the IAM console in your AWS console.
Don’t forget to make the script executable:
Finally, let’s run the script. First we will need to start a new docker container based on the
If everything went as planned you should be logged into the docker container as root.
To run the script we first have to
/webappdirectory so we can run the deploy script:
When the script is finished your application is pushed to Elastic Beanstalk for propagation to your EC2 instances, shaving a lot of time on spinning up new EC2 instances during auto scaling.
Even though running the deployment from your local machine works perfectly fine, it would be a lot more efficient to let your CI environment take care of this. At Springest we use Wercker and their Ewok infrastructure to run our specs and deploy script.
This article won’t go into detail how Wercker can be used to make the process even more efficient. If you are interested in the details please leave a comment. If we receive enough interest we might devote an article about it in the future.Read more
Note: This project is no longer maintained by our team, but it’s still cool to learn from. If you would like to take it over, email us, or check out one of its maintained forks.
Amazon’s Elastic Beanstalk is a very nice platform to easily host simple applications, while it also gives you enough tools to customise your environment for more complex setups. One simple customisation we made here at Springest is a script to push deploy notifications to NewRelic, AppSignal, and our development Slack channel.
Example Script for Slack
For argument’s sake, we are going to use the Slack script as an example. The other scripts you can find in our repo.
Below is an example script to post to Slack. It takes a couple of arguments. Read the usage info for more details. Note that this was heavily inspired by the scripts from @apancutt’s deploy notification script.
Create a Webhook Integration in Slack
First off, you need to create a webhook integration in Slack. You can do so by clicking the name of the channel you’re in and select Add a service integration.
Create a file in
script/slack_deploy.shwith the code below and make it executable with
chmod +x script/slack_deploy.sh:
When you try to use the script above, it will fail with a:
So let’s fix that.
Creating the REVISION file
AppSignal and Newrelic need a revision for best results, so we’re going to create one. We don’t have access to the git history directly while we run these scripts on EB, so let’s make sure they get created while we have it.
You can either (1) update the REVISION file before you deploy with
eb deployor (2) have your CI / deploy tool (we use Wercker) write and commit the file. Either way, this is the command to update the file:
You can find the above script, and the scripts for NewRelic and AppSignal in the repository we created for it. Let us know if you have any fixes or improvements!
Running the scripts during deploys
To run the script, you need to tell EB which command to run. Additionally, you’ll want to make sure that it only runs once per deploy, and not on every instance (no need to have 40 deploy messages if you have 40 instances running your web cluster ;)
So we’re going to use
container_commandsfor that. Create a file called
.ebextensions/003_notifications.configand provide the following input:
Setting the necessary ENV vars on Elastic Beanstalk
As you can see from the example above, we use a couple of environment variables to call these scripts. To set these you can just use the
ebcommand line tool:
If all goes well, you will see the notification in Slack:
subscribe via RSS