I have been getting in to Cloud Foundry for a little while now and being a fan of using Resque to handle background tasks in Rails apps I was keen on getting this all working on my new favorite PaaS platform. So for the sake of the rest of the Cloud Foundry community and others trying to achieve the same, I thought it would be a good idea to get the process on paper.

As an example I have create a very simple Rails 3.2 application that has one model, Tweets. The purpose of this application is to archive the latest tweet from the Twitter timeline when the index page is requested. Instead of reading the Twitter timeline when the index page is loaded, a Resque job is created which performs exactly that task. The saved tweets are displayed in a table on the index page of the appication.

You can see the repository here

So, first step, create the a Rails application with a MySQL backend;

$ rails new resque_test -d mysql

Next, edit the Gemfile and add Resque and HTTParty (for getting the timeline)

1
2

gem 'resque'
gem 'httparty'

A quick “bundle install” and we’re good to start setting up Resque, obviously it’s good to get this working locally first, Resque is backed by Redis, if you don’t have that installed then head on over to http://redis.io/ and get it installed. If like me, you’re using Mac OS you can install it using homebrew;

$ brew install redis

Okay, now you have done all this we can actually have our Rails app start and mount Resque and give it is own route, this is easily achieved by modifying your routes.rb file so it looks like this;

1
2
3
4
5
6
7
8
9

require 'resque/server'

ResqueTest::Application.routes.draw do

  mount Resque::Server.new, :at => "/resque"

  root :to => 'home#index'

end

Note the include at the top of the file and the “mount” statement on line 5. Assuming that Redis is running on the local machine, you should be able to start the Rails application and view the Resque dashboard at http://localhost:3000/resque

Next, we create our model to store the archived tweets, not going to go too deep in to this but in the example the migration looks like this;

1
2
3
4
5
6
7
8
9
10

class CreateTweets < ActiveRecord::Migration
  def change
    create_table :tweets do |t|
        t.string :account_name
        t.string :content
        t.datetime :posted_at
      t.timestamps
    end
  end
end

And also a controller and view for displaying the archived tweets;

1
2
3
4
5
6
7
8
9
10
11

require './lib/twitter_reader'

class HomeController < ApplicationController
    def index

        @tweets = Tweet.all

        # queue up next read
        Resque.enqueue(TwitterReader)
    end
end

Note the require to our Resque job class on line 1 that will actually read and archive the latest tweet. Also note how we enqueue a new instance of that job on line 9.

1
2
3
4
5
6
7
8
9
10
11

<h1>Tweets</h1>

<table>
    <% @tweets.each do |t| %>
        <tr>
            <td><%= t.account_name %></td>
            <td><%= t.content %></td>
            <td><%= t.posted_at %></td>
        </tr>
    <% end %>
</table>

So, next create the class that Resque will use to undertake the queued task, I always create these in the lib folder;

1
2
3
4
5
6
7
8
9
10
11
12
13
14

class TwitterReader
    include HTTParty
    @queue = :twitter_reader

  def self.perform()
    tweet = get("http://twitter.com/statuses/public_timeline.json").parsed_response.first

    user = tweet["user"]["screen_name"]
    content = tweet["text"]
    posted_at = tweet["created_at"]

    Tweet.create :account_name => user, :content => content, :posted_at => posted_at
  end
end

I think this class is pretty self-explanatory, notice I have actually set a name for the job queue (twitter_reader), for a more detail on how to use Resque take a look at https://github.com/defunkt/resque. At this stage the application should run just fine, however we don’t currently have a Resque worker assigned to that “twitter_reader” queue to actually process the jobs. To do this we need to run one of the two Resque rake tasks, by default these are not included in the Rails project. You will need to include them in your Rakefile, mine looks like this;

1
2
3
4
5
6
7

#!/usr/bin/env rake
# Add your own tasks in files placed in lib/tasks ending in .rake,
# for example lib/tasks/capistrano.rake, and they will automatically be available to Rake.

require File.expand_path('../config/application', __FILE__)
require 'resque/tasks'
ResqueTest::Application.load_tasks

Notice I have included resque/tasks on line 6. Now we can launch a Resque job worker process, the Resque site advises us to do this like so;

$ QUEUE=twitter_reader rake resque:work

This works fine if the application is a simple rack based app but I found this doesn’t work with Rails and produces the following error “undefined method `require_dependency’”. However, launching the rake task as follows seems to work just fine!;

$ QUEUE=twitter_reader rake environment resque:work

Run the application, refresh the index page and you should start to see Resque process jobs and create record entries for the Tweet model. On to deployment to Cloud Foundy!

For this example app I am using an account on cloudfoundry.com, deploy the application as normal using vmc;

$ vmc push resque-test --runtime ruby19

Remember to create a new Redis service to bind to the application and to also save the configuration to manifest.yml. Once this is deployed you should have a working Rails app but we still need a worker process to service those requests. The best way to do this is re-deploy the application again using a different name and configuration much the same as is detailed (here)[http://blog.cloudfoundry.com/post/13481011636/running-resque-workers-on-cloud-foundry] for other rack-based apps.

Before doing this step make sure you have the latest version of vmc but re-installing the gem, the latest version has the option of deploying an application as a standalone app.

Move the manifest.yml out of the way so if we need to we can redploy the web server Rails app if needs be;

$ mv manifest.yml server-manifest.yml

Now redeploy the application using the same vmc command as last time with a different name but when asked if it is a Rails application, say no and then select “Standalone”, you will then be asked what command to execute to start the application, in this case it should be;

bundle exec rake environment VERBOSE=true QUEUE=twitter_reader resque:work

If this all deploys ok then the application should process all the queued jobs just fine! My demo application should still be running at http://resque-test.cloudfoundry.com/

Sometimes, if your working with Git on your own you might find yourself in the situation where you want to merge a branch back in to “master” without having to manually merge in changes. I have been using Git with Unity 3D lately, Unity is a 3D programming platform that allows you to use the Mono port of C#, so when starting the project, the first thought that came to mind was - “How do we version all the scripts?”

Needless to say, I started using Git and everything seem to go OK despite the large amount of binary changes included in every commit, it all worked well. Recently, I encountered a point in the project where it was going to be useful to branch the repository as I was going to continue making small code changes whilst also refactoring a large chunk of existing functionality. I branched and continued on my way as before, although I couldn’t find a way of changing my post-receive hook on my remote repository to export that particular branch, I made a new archive of the branch head every time I commit for the client to download.

A week passed and I found myself in the situation where I wanted to go back to working off of the master branch, if anything, to get the export hook working again. I now faced a small dilemma, do I somehow manage to reconcile the differences between the two branches (how does this work with binaries? not sure). I was sure the head of the new branch was correct, so I needed a way of merging in and skipping the diffing.

I had read about the “Ours” merging strategy but I wanted something the other way round. I happened across this article which details just how to do this.

And here it is;

1
2
3

git merge -s ours ref-to-be-merged
git diff --binary ref-to-be-merged | git apply -R --index
git commit -F .git/COMMIT_EDITMSG --amend

Blogging this for my own sake, I know I will need it again!

Having heard how great Unicorn and NginX are together for hosting Rack-based applications I decided I would need a decent way of deploying applications to a brand new EC2 instance, quickly and efficiently.

I knew the solution would be made up of a Capistrano recipe and a pre-built EC2 server image.

Rather than describe the whole process on here I thought it best to do a screencast (you will need to watch this in HD to read any of the text!), however I have transcribed the main steps below the video :). Get a coffee though, it’s near 15 minutes of me talking!

Okay, so the basic steps to deploying the application are;

1) Develop your rack app

2) Capify the application

3) Get hold of the ec2-unicorn recipe from https://raw.github.com/danhigham/ec2-unicorn/master/ec2-unicorn-recipe.rb and include it in your deploy.rb

4) Create an instance of the UnicornNginX AMI which at the time of writing has an AMI id of ami-1310df7a

5) Change your deploy.rb so it looks something like this;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

require File.join(File.dirname(__FILE__), 'ec2-unicorn-recipe.rb')

set :application, "example-app"
set :repository,  <your apps code repo>
set :deploy_to, "/var/www/example-app"
set :scm, :git
set :user, "ec2-user"
set :use_sudo, false

set :host_header, "_"

# needed to run bundler
default_run_options[:pty] = true

# set this to point to your EC2 keys
ssh_options[:keys] = ["#{ENV['HOME']}/.ssh/<security key pair from Amazon>.pem"]

role :web, "ec2-x-x-x-x.compute-1.amazonaws.com"                          # Your HTTP server, Apache/etc
role :app, "ec2-x-x-x-x.compute-1.amazonaws.com"                          # This may be the same as your `Web` server
role :db,  "ec2-x-x-x-x.compute-1.amazonaws.com", :primary => true # This is where Rails migrations will run

6) Run cap deploy:setup to setup the live environment

7) Run cap deploy to deploy and start Unicorn

8) Run ec2_unicorn:restart_nginx to restart nginx

And that’s it! Any comments, please do let me know via Github or catch me on Twitter, @danhigham

Whilst working on a project this evening I translated a rather handy javascript function that builds an object from form elements in to CoffeeScript. This is pretty handy when you want to post data from a form as JSON using AJAX.

I found the original Javascript solution here

And here is my CoffeeScript version

Simples!

A month or so in to a project with Unity on OS X, and I was beginning to feel like version 2.4 of MonoDevelop just wasn’t cutting the mustard! I discovered that Unity actually keep all of their own builds of MonoDevelop on Github

The one we are obviously interested in, is MonoDevelop

So, here is a quick guide on how to get this beast of a project built and running in OS X;

Pre-requisites;

• Mono Framework version 2.6.7 for compiling Unity Projects (actually optional but Unity projects historically use 2.6)
• Mono Framework version 1.10.4 SDK for building MonoDevelop
• XCode, if your using Lion like me, it may be worth moving GCC back to the GNU version, this can be done by symlinking /usr/bin/gcc -> /usr/bin/gcc-4.2
• Git, which comes with OS X nowadays anyway

First things first, grab a copy of the repository;

1

git clone https://github.com/Unity-Technologies/monodevelop.git

We are only really interested in the most up-to-date version, so switch branches to ‘bleeding-edge’;

1

git checkout remotes/origin/bleeding-edge

As per the instructions at http://monodevelop.com/Developers/Mac_Support/Building_MonoDevelop_on_OS_X set up the following two environment variables as follows;

1
2

export ACLOCAL_FLAGS="-I /Library/Frameworks/Mono.framework/Versions/Current/share/aclocal"
export PATH="/Library/Frameworks/Mono.framework/Versions/Current/bin:$PATH"

Configure the build with the mac profile

1

./configure --profile=mac

Then make the project

1

make

Once this has built, here comes the important bit, move in to the OS X build directory and run make to build the OS X app package;

1

cd main/build/MacOSX && make MonoDevelop.app

If you navigate to the application with Finder and try and run it you will notice this fails. Navigate in to the app package;

1

cd MonoDevelop.app/Contents/MacOS

Use your favourite editor to edit the monodevelop script

1

mate ./monodevelop

Change

1

MONO_FRAMEWORK_PATH="$FINAL_INSTALL_DIR/Versions/Current"

to

1

MONO_FRAMEWORK_PATH="/Library/Frameworks/Mono.framework/Versions/Current"

and change

1

export DYLD_FALLBACK_LIBRARY_PATH=$MONO_FRAMEWORK_PATH/lib:/lib:/usr/lib

to

1

export DYLD_FALLBACK_LIBRARY_PATH=$MONO_FRAMEWORK_PATH/lib:$DYLD_FALLBACK_LIBRARY_PATH:/usr/lib

Hopefully, when you try and launch the MonoDevelop.app file from Finder it should work, copy it to your Applications folder if you wish!

Any problems, let me know; I am lurking on #unity3d on freenet or mail me - dan at high.am

Since registering the domain “high.am”, setting up email against it and also a nifty URL shortener I wanted to put some content up too!

Having always been a fan of Github, I was keen to give Octopress a shot. This is where I will be giving it that shot! As a contract programmer, I need somewhere to hang my hat online as an individual and showcase some of the projects I have been doing of late.

Octopress seemed to fit the bill nicely. So expect to see plenty of reviews, how tos and other crap, a general mish mash of stuff I come across in my daily grind…