No items found.
February 20, 2015
/
8
min

Replacing the Rails asset pipeline with Gulp: Using Gulp to compile and cache our assets

At Bugsnag, we used Rails’s asset pipeline to compile and deploy our assets. It was really convenient since the asset pipeline is bundled into Rails, and it solved many problems for us right out of the box. It dealt with caching our assets and generating the correct URLs for them, plus cache busting. Not only that, it also compiled and minified all  our assets. We also discovered Rails Assets, a way to bridge Bundler with Bower, which came in handy for handling our front-end dependencies easily. This allowed us to load in our bower components as gems.

Gulp

So this all sounds perfect, right? Why would we change what we had? Unfortunately, the asset pipeline had significant issues. Any time we compiled locally or deployed, it was always painfully slow. We were also frustrated because it was extremely hard to debug and test what was going on since so much magic was happening behind the scenes with Rails. Plus, the asset pipeline didn’t allow us to harness sourcemaps to fix up our JavaScript stacktraces which we knew would make debugging our JavaScript way easier. We decided we wanted a better way.

What is Gulp?

After some research, we decided to use a tool called Gulp for our asset compilation needs. Gulp is a Node.js streaming build system, strictly providing streams and a basic task system. This means that we can use Gulp to automate common tasks we need in our website, like compiling assets for deployment. We decided to go with Gulp because of its intuitive code-over-configuration stance, as well as its simple but effective plugins.

To install Gulp, check out Gulp’s getting started document.

Setting up for development

In development, we don’t need to worry about caching, deployment, or any of the other tasks associated with pushing out to production. For now, we’ll want to pull out the Rails asset pipeline, install Gulp, and get Gulp building our development assets and serving them to [CODE]public/assets[/CODE].

Pulling out the Rails asset pipeline

In Rails 4, you can turn asset pipeline off easily. In your [CODE]config/application.rb[/CODE], you just need to disable your assets:

-- CODE language-ruby --
config.assets.enabled = false

And if you use generators, you’ll want to prevent the cli generator from creating assets when generating a scaffold:

-- CODE language-ruby --
config.generators do |g|
 g.assets false
end

Hooking up Gulp

Gulp allows us to separate our backend Rails app from our frontend app. We’ll want to create a [CODE]frontend[/CODE] directory in the root of the application in which to keep our assets. From there, we’ll compile a [CODE]public/assets[/CODE] directory so the assets can be served up with Rails. We’ll have to be in the [CODE]frontend[/CODE] directory whenever we run any of our gulp commands.

Our [CODE]frontend[/CODE] directory will also contain our [CODE]gulpfile[/CODE], which will contain the code for our tasks. When we run [CODE]$ gulp[/CODE] from the command line, we want our default [CODE]gulp[/CODE] task to:

-- CODE language-coffeescript --
gulp.task "default", ["js", "sass", "images", "fonts"]

Concatenate all JS bower files, vendor files, and source files into one file; add to public; create JS sourcemaps

-- CODE language-coffeescript --
sourceMaps  = require "gulp-sourcemaps"
liveReload     = require "gulp-livereload"
filter               = require "gulp-filter"
coffee            = require "gulp-coffee"

gulp.task "js", ->
 gulp.src("paths/to/your/js")
   .pipe(sourceMaps.init())
   .pipe(coffeeFilter)
   .pipe(coffee())
   .pipe(coffeeFilter.restore())
   .pipe(concat("application.js"))
   .pipe(sourceMaps.write("."))
   .pipe(gulp.dest("../public/assets"))
   .pipe(liveReload())

Create Sass sourcemaps, add to public    

-- CODE language-coffeescript --
sass = require "gulp-sass"
autoPrefixer = require "gulp-autoPrefixer"
sourceMaps = require "gulp-sourcemaps"
liveReload = require "gulp-livereload"

gulp.task "sass", ->
 gulp.src("sass/dashboard/application.css.scss")
   .pipe(sourceMaps.init())
   .pipe(sass({includePaths: "path/to/code", sourceComments: true, errLogToConsole: true}))
   .pipe(autoprefixer())
   .pipe(rename("application.css"))
   .pipe(sourceMaps.write("."))
   .pipe(gulp.dest("../public/assets"))
   .pipe(liveReload())

Add our images to public    

-- CODE language-coffeescript --
liveReload = require "gulp-livereload"

gulp.task "images", ->
 gulp.src("images/**/*")
   .pipe(gulp.dest("../public/assets/images/"))
   .pipe(liveReload())

Add our fonts to public    

-- CODE language-coffeescript --
liveReload = require "gulp-livereload"

gulp.task "fonts", ->
 gulp.src("fonts/**/*")
   .pipe(gulp.dest("../public/assets/fonts/"))
   .pipe(liveReload())

After our default task finishes, we’ll have our assets compiled into [CODE]public/assets[/CODE] in a way we can use for development.

Live Reloading

We recommend setting up a [CODE]reload[/CODE] task that includes live reloading for your assets. We use this so that any time someone edits a JavaScript or CSS file, their browser will automatically refresh after compiling, saving time and effort:

-- CODE language-coffeescript --
gulp.task "reload", ["watch", "js", "sass", "images", "fonts"]

Since we’re using the Ruby gem rack-livereload, our [CODE]watch[/CODE] task can now enable live reloading:

-- CODE language-coffeescript --
gulp.task "watch", ->
 liveReload.listen
 gulp.watch "paths/to/your/sass", { interval: 500 }, ["sass"]
 gulp.watch "paths/to/your/js",  { interval: 500 }, ["js"]

Setting up for production

Next we need to set up for getting our code out to production. This entails setting up caching and cache busting, sending our assets up to S3, and setting up Capistrano for deployment.

Adding production Gulp tasks

gulp js

In our [CODE]js[/CODE] gulp task, we’ll want to add in uglification of JavaScript after concatenation.

-- CODE language-coffeescript --
sourceMaps = require "gulp-sourcemaps"
liveReload = require "gulp-livereload"
filter     = require "gulp-filter"
coffee     = require "gulp-coffee"
uglify     = require "gulp-uglify"

coffeeFilter = filter ["**/*.coffee"]

gulp.task "js", ->
 stream = gulp.src("paths/to/your/js")
   .pipe(sourceMaps.init())
   .pipe(coffeeFilter)
   .pipe(coffee())
   .pipe(coffeeFilter.restore())
   .pipe(concat("../public/assets"))

 if ["production, staging"].indexOf(railsEnv) != -1
   stream = stream
     .pipe(uglify())

 stream.pipe(gulp.dest("../public/assets"))
   .pipe(sourceMaps.write("."))
   .pipe(liveReload())

gulp sass

Similar to our [CODE]js[/CODE] gulp task, we’ll want to add in minification of Sass.

-- CODE language-coffeescript --
sass = require "gulp-sass"
autoPrefixer = require "gulp-autoPrefixer"
sourceMaps = require "gulp-sourcemaps"
liveReload = require "gulp-livereload"
minifyCSS  = require "gulp-minify-css"

gulp.task "sass", ->
 stream = gulp.src("sass/dashboard/application.css.scss")
   .pipe(sourceMaps.init())
   .pipe(sass({includePaths: "path/to/code", sourceComments: true, errLogToConsole: true}))
   .pipe(autoprefixer())
   .pipe(rename("application.css"))

 if ["production, staging"].indexOf(railsEnv) != -1
   stream = stream
     .pipe(minifyCSS())

 stream.pipe(gulp.dest("../public/assets"))
   .pipe(sourceMaps.write("."))
   .pipe(liveReload())

gulp production

This task will run after we’ve built our production assets using [CODE]RAILS_ENV=production gulp[/CODE]. In addition to those tasks, gulp production will also have to do some production-specific things. We’ll use [CODE]gulp-rev-all[/CODE] to set up our files for caching, and to create a cache manifest to map our cached files to the real URLs.

-- CODE language-coffeescript --
gulp.task "production", ->
 gulp.src(["../public/*assets/**"])
   .pipe(revAll()
   .pipe(gulp.dest("../public/assets/production"))
   .pipe(revAll.manifest())
   .pipe(gulp.dest("../public/assets/production"))

gulp publish

We’re using AWS to store our assets. We use Cloudfront as our CDN, which is set up to read from our AWS bucket. This task uses gulp-awspublish to gzip, cache our upload, and send our assets to AWS:

-- CODE language-coffeescript --
gulp.task "publish", ->
 publisher = awspublish.create(bucket: 'your-s3-bucket-name')

 gulp.src("../public/assets/production/**")
   .pipe(awspublish.gzip())
   .pipe(parallelize(publisher.publish({"Cache-Control": "max-age=31536000, public"}), 20))
   .pipe(publisher.cache())
   .pipe(awspublish.reporter())

Monkeypatching Rails

Since we’re changing where all of our assets are, we need to update some of our helper methods to find the correct assets from our Gulp manifests. Our [CODE]rev-manifest.json[/CODE] will map our cached URLs to the originals. We’ll need our Rails URL helpers to intercept the cached URLs from the manifest first if they’re available. We’re going to make an [CODE]AssetManifest[/CODE] class with all of the methods we need in [CODE]config/initializers/asset_manifest.rb[/CODE]:

-- CODE language-ruby --
class AssetManifest
 def self.manifest
   if File.exists?("rev-manifest.json")
     @manifest ||= JSON.parse(File.read("rev-manifest.json"))
   end
 end

 def self.stylesheet_path(url)
   if AssetManifest.manifest
     url += ".css" unless url.end_with?(".css")
     AssetManifest.manifest[url] || url
   else
     url
   end
 end

 def self.javascript_path(url)
   if AssetManifest.manifest
     url += ".js" unless url.end_with?(".js")
     AssetManifest.manifest[url] || url
   else
     url
   end
 end

 def self.asset_path(url)
   if AssetManifest.manifest
     AssetManifest.manifest[url] || url
   else
     url
   end
 end
end

This class helps us by mapping our asset filenames to the cached versions of the filenames produced by [CODE]gulp-rev-all[/CODE].

And then in your [CODE]app/helpers/application_helper.rb[/CODE]:

-- CODE language-ruby --
 def stylesheet_link_tag(url, options={})
   url = AssetManifest.stylesheet_path(url)

   super(url, options)
 end

 def crossorigin_javascript_include_tag(url, options={})
   url = AssetManifest.javascript_path(url)

   super(url, options)
 end

 def image_tag(url, options={})
   url = AssetManifest.asset_path(url)

   super(url, options)
 end

 def image_path(url, options={})
   url = AssetManifest.asset_path(url)

   super(url, options)
 end

 def image_url(url, options={})
   url = AssetManifest.asset_path(url)

   super((ActionControllerBase::.asset_host || "") + url, options)
 end

This way, we’ll call our [CODE]AssetManifest[/CODE] methods, giving us the correct URLs, and then call [CODE]super[/CODE] so we call the original Rails helper methods.

Setting up Capistrano 3 to use Gulp

We now need to hook up deployment of our site. For this, we’ll use Capistrano 3. In your [CODE]config/deploy.rb[/CODE] we’ll have to run our precompile script and upload our Gulp manifest to our machines:

-- CODE language-ruby --
namespace :assets do
 desc "Publish assets"
 task :publish do
   run_locally do
     execute "script/precompile.sh #{fetch(:current_revision)} #{fetch(:stage)}"
   end
 end

 desc "Transfer asset manifest"
 task :manifest do
   on roles(:all) do
     # All roles need to be able to link to assets
     upload! StringIO.new(File.read("public/assets/production/rev-manifest.json")), "#{release_path.to_s}/rev-manifest.json"
   end
 end

 after "deploy:updated", "assets:publish"
 after "deploy:updated", "assets:manifest"
end

The [CODE]deploy.rb[/CODE] file is going to look for a [CODE]precompile.sh[/CODE] script, where precompiling assets will take place. That’s where you’ll run your [CODE]gulp[/CODE] tasks to compile your assets.

-- CODE language-bash --
#!/bin/bash

NEW_SHA=$1
RAILS_ENV=$2

mkdir -p precompile

cd precompile
git clone .. . || git fetch
git checkout $NEW_SHA

cd frontend
cp -r ../path/to/frontend/dependencies .

export RAILS_ENV=$RAILS_ENV
gulp install
gulp default
gulp production
gulp publish

Retrospective

We’re really happy with the results we’ve had since switching over to Gulp. Asset compilation times have gone down from minutes to seconds which is an amazing improvement. Local development is running more smoothly due to our site live reloading. And we now have access to our JavaScript sourcemaps which makes JavaScript debugging way easier. Not to mention we support JavaScript sourcemaps for better stacktraces at Bugsnag!

———

Hooking up Gulp as our asset pipeline has been very helpful for us. Let us know if this post ends up helping you change your asset pipeline by tweeting at us or emailing us!

BugSnag helps you prioritize and fix software bugs while improving your application stability
Request a demo