Keeping alive the messenger architecture on free heroku dynos

Heroku is a cloud application platform – a new way of building and deploying web apps and a PaaS service to host applications in various programming languages and frameworks on its cloud. The entire architecture behind the messengers has been written with nodejs and deployed to heroku as its production for one main reason which is that it provides a verified and signed SSL Certificate along with its deployment which is useful for facebook messenger integration of susi as well as useful for telegram to trigger webhooks on SSL.

A major problem with the dynos (apps deployment) is that for free users the deployment automatically goes to sleep if no one is using it for a short while and only wakes up when the endpoint is hit and also is available in a day for upto 18 hours. This could be problematic when the deployment is made, so the best way is to have maximum utilization of the resources and keep it awake as and when required. This was resolved with facebook messenger because of incoming event webhooks that are available which request the server to wake up in case they sleep but what about Slack ? The slack server doesn’t send a notification event to the server when a message is sent by a user mentioning susi which is just like every other user and can be added to a channel.

To fix this problem there are multiple approaches that we’ve taken, the first step is to ensure that every fixed time interval the server pings itself so that it’s kept alive. This is accomplished with this fraction of the code

setInterval(function() {
		http.get(heroku_deploy_url);
	}, 1800000); //pings the deployment url every 30 minutes

where the heroku_deploy_url is an environment variable that can be set by the user depending on the URL of the deployment.

Another option was to use the New Relic insights and their availability tracker to keep sending requests after every fixed interval of time and use it to keep the server alive. This can be accomplished by doing the following on the heroku toolbelt.

heroku addons:add newrelic:standard
heroku addons:open newrelic

then using the following ruby script and setting the PING_URL as heroku config:add PING_URL=http://longweburl.herokuapp.com

desc "Pings PING_URL to keep a dyno alive"
    task :dyno_ping do
      require "net/http"

      if ENV['PING_URL']
        uri = URI(ENV['PING_URL'])
        Net::HTTP.get_response(uri)
      end
    end

and then performing the execution of the task by doing

heroku addons:add scheduler:standard
heroku addons:open scheduler
rake dyno_ping

The last option was however to use the existing loklak.net server setup a cronjob on that to query the heroku instance periodically so that the instance quota doesn’t get over and at the same time has as much uptime as majorly required. The best option however would be to upgrade to a hobby plan and purchase a dyno to host the resource.

Keeping alive the messenger architecture on free heroku dynos

Improved documentation for Loklak repos

Its the final week of GSoC 2016. All the projects are nearing their completion stage. Since one of the plugins from FOSSASIA (https://wordpress.org/plugins/tweets-widget/) is already in WordPress directory, I took this opportunity to write some documentation for other plugins and the plugin maintenance repo.

The documentation now verbosely describes the complete Heroku deployment procedure directly from Github as well as using the Git-Heroku toolbelt (see this).

Selection_023

Docs for updating to a newer version of WordPress have also been added.

Selection_022

Screenshots and relevant documentation regarding Loklak API was added to several plugins.

 

Selection_025
readme.txt of https://github.com/fossasia/wp-recent-tweet

Some screenshots of the plugin (wp-recent-tweet) added in the readme

Selection_026

Improved documentation for Loklak repos

Using Heroku-WordPress Buildpack to test Loklak integration in WordPress plugins

Loklak support was added to several wordpress plugins this summer. In order to properly handle and structure further development in this area, a common repository has been created to host the incumbent plugins. Please refer to this repository. To ensure rigorous testing in the internet environment, all plugins have been installed (loklak_wordpress_plugins/wp-content/plugins) in a sample WordPress site deployed on Heroku.

Heroku is a platform as a service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud.

Heroku provides several buildpacks for different deployment patterns. After careful research I found a useful WordPress+Heroku buildpack here. To create your own wordpress instance to test the plugins follow the below instructions:

Deploy from github directly:

  1. Click on Deploy to Heroku button in the Readme file of your fork of loklak_wordpress_plugins repo. This step would install wordpress on Heroku and setup your database.
  2. Give your website a name and input your time-zone and add authentication information for .htpasswd to access wp-admin page (admin privileges). See the screenshot below. blog2_1
  3. Click on Deploy for Free. Once the app is deployed, Click on ‘Manage App’. Go to ‘Deploy’ tab and choose ‘Deployment method’ as Github
  4. Connect your forked loklak_wordpress_plugins repo to Heroku. blog2_2
  5. To automate the deployment process when the github repo is updated, enable automatic deploys. Now deploy the master branch of your repo and you are good to go. blog2_3

Deploy using Heroku Toolbelt:

To deploy using Heroku-git toolbelt, please refer to this.

After we are done with the deployment, we need to setup our plugins for using and testing:

  1. Follow the steps to install WordPress (as they appear on your computer screen) on your chosen app/domain name.
  2. Once WP is installed, change the language, time-zone and preferred date-time format in ‘general settings’.blog2_4
  3. Now you can activate plugins from ‘plugins’ as per your need and test their functionality!blog2_5
Using Heroku-WordPress Buildpack to test Loklak integration in WordPress plugins

Loklak Server just a click away – One Click Deployments

As a person with an Android background working on the loklak project for the first time last month, I really did have a lot to pick up, and somehow I had to get started off.

I checked out the loklak docs which listed the process of deploying the server on your local machine or on Heroku / DigitalOcean / AWS, and all of those processes were lengthy and needed a bit of time to get set up, and besides I had never worked with Heroku before so I had to read up how it worked. There was no other feature to speed up the deployment process at that time, and that is when we decided to add a new feature: One Click Deploy!

One Click Deploy is the easiest, most convenient method to start up your own loklak server. Instead of going through the long drawn setting up through the terminal, you can simply deploy your server and get loklak running with a single click. We extended our list of supported platforms, and today, loklak server can be started onto Heroku, Scalingo, Docker Cloud and Bluemix, just with a single click. And all you need to do? Give your deployed app a name. That’s it. Quite literally.

Screen Shot 2016-06-01 at 12.08.47 AM

Integrating One Click Deploy into loklak, at first, wasn’t too big of a task. For Heroku, I just created an app.json file and linked it to the custom buildpack we had here. For Scalingo, I similarly had to create a scalingo.json, however I had issues as they didn’t have a JDK for Heroku Buildpacks, so I contacted them and they mended their own codebase to accommodate Heroku, and it finally worked. The Bluemix deployment was a bit more tedious: we had to make a manifest.yml and a pipeline.yml file, and then added a Bluemix start shell script and a Bluemix buildpack to make it work.

However, we did end up facing a lot of challenges after a period of time. After some days, our deployed servers started crashing. I communicated with the Scalingo team over email and they told us that our servers used an Xmx (maximum memory) of 600 MB while Scalingo containers worked with 512 MB, thus increasing swapping and reducing performance hereby crashing the applications. This was sorted by keeping an environment variable $ENVXmx which set the Xmx level based on the environment.

Finally, we decided to update our existing codebase to support Java 8, and as a result, we modified our buildpacks (Bluemix and Heroku) to accommodate for this change. This took a couple of days but finally it was done.

So at the end of it all, loklak finally has a much needed feature with which you can experience it in a matter of seconds, and a way with which you can set up multiple servers in just a few minutes, to multiple platforms. We have plans for having a deployment to Azure as well, and will plan on implementing it in the days to come 🙂

Loklak Server just a click away – One Click Deployments