stderr.timfischbach.de

Nested Helper Classes vs Private Helper Methods

2014-12-26T00:00:00+00:00

While criticism towards Rails helpers has almost become folklore (procedural, global scope, junk drawers), I still find myself reaching for them as a low ceremony way of making templates more concise and building simple view layer APIs.

Obviously, the greatest downside of helper modules is how dangerous extracting private methods becomes.

module UsersHelper
  def user_section(user)
    content_tag(:section, user.name, class: css_class(user))
  end

  private

  def css_class(user)
    ['user', user.role].join(' ')
  end
end

module PostsHelper
  def post_section
    content_tag(:section, post.text, class: css_class(post))
  end

  private

  # collides with UsersHelper#css_class
  def css_class(post)
    'post'
  end
end

While I always liked extracting classes to keep helper methods short, only recently did I realize how a nuance in Ruby's constant look-up mechanism helps prevent namespace collisions as above.

Let's look at a variant of the above helpers that uses nested classes instead of private methods.

module UsersHelper
  def user_section(user)
    Section.new(user).render(self)
  end

  class Section < Struct.new(:user)
    def render(template)
      template.content_tag(:section, user.name, class: css_class)
    end

    private

    def css_class
      ['user', user.role].join(' ')
    end
  end
end

module PostsHelper
  def post_section(post)
    Section.new(post).render(self)
  end

  class Section < Struct.new(:post)
    def render(template)
      template.content_tag(:section, post.text, class: css_class)
    end

    private

    def css_class
      'post'
    end
  end
end

At first glance, one could expect the same problem as above. Instead of both defining an equally named private method, both modules now contain a nested class with the same name. Once both modules get included into the view object only one can win, right? That's sure the way it looks when we try to reference one of the nested classes from a template:

  # app/views/users/index.html.erb
  <%= Section == PostsHelper::Section # => true %>

But in contrast to method invocations, which are resolved by the object handling the call, constant look-up uses the lexical scope at the point where the constant is referenced. So referring to a Section class inside UsersHelper still resolves to UsersHelper::Section, no matter whether there is a colliding PostsHelper::Section class when including helper modules into the view context.

Nested helper classes thus provide a safe space for refactoring towards small methods. Helper methods end up containing only wiring code, keeping implementation details from leaking into tests or templates.

YARD Documentation with Emacs

2014-10-12T00:00:00+00:00

In Emacs, the fill-paragraph command lets you reformat a chunk of text to break lines at a certain width. Bound to M-q by default, it is a handy way to enforce consistent line widths inside text documents.

Recently, I've been editing a lot of inline documentation in Ruby code using YARD. A typical doc comment looks like this:

# Reverses the contents of a String or IO object. 
# 
# @param [String, #read] contents the contents to reverse 
# @return [String] the contents reversed lexically 
def reverse(contents) 
  # ...
end

As descriptions got longer, I found myself wishing to reformat paragraphs. While Emacs knows how to fill paragraphs inside Ruby comments, hitting M-q still turned text into unreadable gibberish:

# Reverses the contents of a String or IO object.  @param [String,
# #read] contents the contents to reverse @return [String] the
# contents reversed lexically

Turns out there is an easy way to tell Emacs how to recognize paragraphs: paragraph-separate and paragraph-start. Let's define a function that sets these variables to YARD specific values. The concatenate call is only there to improve readability.

(defun yard-paragraph-boundaries ()
  (interactive)
  ; Paragraphs are separated by lines containing only a # character
  (setq paragraph-separate "[ \t]*#[ \t]*$")
  ; Paragraphs start with YARD tags or list items
  (setq paragraph-start
      (concatenate
       'string
       "^[ \t]*"         ; some whitespace
       "#[ \t]*"         ; a # character followed by whitespace
       "\\("
         "@[[:alpha:]]+" ; a YARD tag
       "\\|"             ; or
         "-"             ; a list item
       "\\)"
       "\\([ \t]+.*\\)?" ; an optional text
       "[ \t]*$")))      ; some more whitespace

Finally, we can call the function whenever ruby-mode is entered:

(add-hook 'ruby-mode-hook
      '(lambda ()
        (yard-paragraph-boundaries)))

Now text inside YARD comments can be reformatted with a single key stroke. Emacs even preserves custom indentation levels:

# @param [String] A really long description text which spans
#   multiple lines and has a custom indentation, which is
#   preserved even when the paragraph is reformatted.

Resetting Has One Assocations

2014-09-08T00:00:00+00:00

Tested with Rails 4.0

Just a quick note since I could not find this information anywhere. Making sure a has_many association is reloaded on next access is easy. Just call the reset method provided by the collection proxy:

class Post
  has_many :comments
end

post.comments.reset
post.comments # fetches comments from database

But how to accomplish the same for has_one associations?

class Post
  has_one :last_comment, -> { order 'created_at' }, class_name: "Comment"
end

Calling post.last_comment just returns a comment (or nil). Of course, you can pass true as the force_reload parameter:

post.last_comment(true) # fetches comment from database

But what if there is no need to reload the association now, only next time it is used? Turns out the underlying association object can be accessed via the associations method:

post.associations(:last_comment).reset

Now the cache is cleared and the next call to post.last_comment will fetch the record from the database.

Introducing Env Lint

2014-02-20T00:00:00+00:00

We've been following the 12 factor approach of configuring our Rails apps through environment variables. Tweaking options through the env:set task provided by recap feels much more light weight than manually fiddling with yml files on the server. Moreover, the dotenv gem makes controlling your development environment very comfortable. While the .env file itself is not checked into version control, there is a .env.example file which acts as a starting point for new developers and documents the available options.

Still, there were some pain points left:

Whenever new variables were added to the app's configuration, the process of updating the staging and production environments turned out to be rather error prone. Even having carefully compared the lists of defined variables, only after booting the newly deployed app could one be sure that nothing was missing.
This was even harder as small spelling errors in variable names either in the code or on the command line were easy to miss.
Finally, the .env.example file would quickly become outdated, incomplete or simply incorrect.

To resolve all of these issues we came up with the env_lint gem, which is comprised of three components:

A wrapper around ENV called LintedEnv which fails loudly whenever variables are not defined or misspelled variable names are used.
A capistrano task to verify all required environment variables are defined on the production machine --- even before deploying a new revision of the app.
A rake task which helps developers make sure that their local environment is in shape.

For all of this the application's .env.example file becomes the primary source of truth. Only variables explained there can be used in the code or passed to tasks like env:set. The gem also parses the description comments to generate helpful error messages. A typical .env.example file looks like this:

# Host name used in absolute urls 
HOST_NAME=my.app.com

# Redirect users to the https site
# ENFORCE_SSL=true

Commented out assignments signify optional variables. LintedEnv makes sure you always supply a default value when fetching one of these.

linted_env.fetch(:enforce_ssl)
# => raises an error telling you to supply a default

All in all, it's almost impossible for your example file to rot. And when it's time to deploy, a simple

$ cap production env:lint

makes sure your app won't crash after restart only because an environment variable is missing.

More details can be found in the README.

Finally, the ci-config-generator is nice addition to this team, letting you define a .env.ci file which contains a concrete, version controlled configuration which is verified in your continuous integration runs.

Chef Solo Provisioning with Capistrano Roundsman and Berkshelf

2013-11-29T00:00:00+00:00

Chef helps make server provisioning reproducible and self documenting. Even more, a lot of the benefits can be gained without maintaining the infrastructure surrounding a chef server. Especially if you do not have an ops team carefully curating a set of cookbooks representing your server landscape, a more application-centric approach might work for you.

We tend to deploy our applications to phoenix servers which run as Linux Containers (lxc) using libvirt. Each application contains an application cookbook describing the required deployment environment. Here is an excerpt from the typical directory layout of our projects.

some_app/
  Berksfile
  Berksfile.lock
  Capfile
  deploy/
    tasks
      provision.rb
    cookbooks/
      main/
        attributes/
        recipes/
          default.rb
          nginx.rb
          mysql.rb
        templates/
        metadata.rb

Let's take a looks at the different ingredients one by one.

The Application Cookbook

The cookbooks directory normally only contains a single cookbook. It provides recipes to setup everything required to run the app: ruby versions, databases, web servers. The recipes mostly contain include_recipe directives and some lightweight resources. Here is an excerpt from the mysql recipe:

# deploy/cookbooks/main/recipes/mysql.rb
include_recipe 'mysql::server'
include_recipe 'mysql::client'
include_recipe 'mysql::ruby'

mysql_connection_info = { ... }

mysql_database node[:application] do
  connection mysql_connection_info
  action :create
end

mysql_database_user node[:application] do
  connection mysql_connection_info
  password   node[:database_password]
  action     :grant
end

Dependent cookbooks can simply be listed in the cookbook's metadata.rb file:

# deploy/cookbooks/main/metadata.rb
name              "main"
maintainer        "Codevise Solution"
maintainer_email  "me@example.com"
description       ""
long_description  ""
version           "0.0.0"

depends "rvm"
depends "mysql", "3.0.12"
depends "database"
depends "nginx"

supports "ubuntu"

Sometimes other custom cookbooks shall be bundled in the application repository. We then place them next to main in the cookbooks directory. Most of the time though it's easier to automatically resolve dependencies.

Dependency Resolution

Just like bundler resolves gem dependencies, Berkshelf can be used to install required cookbooks. In the Berksfile alternative sources to fetch cookbooks from can be specified.

# Berksfile
site :opscode

# Application cookbook
cookbook 'main', :path => 'deploy/cookbooks/main'

# Alternative sources
cookbook 'rvm', :git => 'https://github.com/fnichol/chef-rvm'

Once you run berks install, all required cookbooks are downloaded into a shared directory and a Berksfile.lock is placed in the project root. Pretty familiar.

One could just as well use Librarian Chef to manage dependencies. But since as of recently Berkshelf appears to be backed by Opscode, I figure it is a good choice.

Controlling Chef Solo

Finally we need a tool to actually run the recipes on the app's deployment server. This is where the Capistrano Roundsman gem comes in. It bootstraps the server with a version of ruby and the chef gem, uploads the cookbooks and invokes a chef solo run.

Note that the ruby version used to run chef needs not be the one used by passenger to later run the app. We usually have a chef recipe install rvm on server to gain flexibility.

The following capistrano tasks wires everything together. When we run cap provision, we first tell berkshelf to unpack the required cookbooks to a tmp directory. The :cookbooks_directory option tells roundsman to pick up on those cookbooks. Finally the roundsman.run_list command triggers the chef solo run.

By default, roundsman installs a rather outdated version of chef. This can easily be fixed by setting the :chef_version option.

# deploy/tasks/provision.rb
require 'roundsman/capistrano'

set :application, 'some_app'
server 'someapp.example.com', :app
set :user, 'ubuntu'

set :cookbooks_directory, ['tmp/cookbooks']
set :chef_version, '11.4.0'

namespace :provision do
  desc 'Install cookbooks and provision server'
  task :default do
    install
    apply
  end

  desc 'Install cookbooks with berkshelf'
  task :install do
    run_local "bundle exec berks install --path #{cookbook_directory}"
  end

  desc 'Provision server'
  task :apply do
    roundsman.run_list 'recipe[main]'
  end
end

def run_local(command)
  system(command)
  if($?.exitstatus != 0) then
    puts 'exit code: ' + $?.exitstatus.to_s
    exit
  end
end

Happy provisioning!

Decorating Active Record Models

2013-10-08T00:00:00+00:00

In the quest against god object active records, decorators provide a nice way to group functionality according to features of your application. They help to keep your models lean and ease separation of responsibilities.

A simplified View

Often certain parts of an application only need to act upon a simplified projection of the actual domain model. Imagine for example an Entry model which consists of various Revisions. Furthermore, let's say, for each entry, there is always at most one published revision:

class Entry < ActiveRecord::Base
  has_many :revisions
  has_one :published_revision, -> { published }, class: 'Revision'
end

class Revision < ActiveRecord::Base
  scope :published, -> { where(published: true) }
end

Only the contents of the published revision is supposed to be visible on the public site. Instead of making our controllers and views deal with the complexity of finding the right revision to display, we introduce a decorator to handle the delegation.

class PublishedEntry
  def initialize(entry)
    @entry = entry
    @published_revision = entry.published_revision
  end

  delegate :title, :description, :to => :@published_revision
end

Code using the decorator can now remain totally unaware of the revision concept. Even better, once we decide to change the logic determining the correct revision for public display, there is a single place to make that change.

A Place for Finders

Since our decorator class is tailored for a rather specific use case, we know what data is going to be needed. This knowledge can be captured in a custom finder which performs appropriate eager loading.

class PublishedEntry
  // ...

  def self.find(id)
    new(Entry.find(id).includes(:published_entry));
  end
end

No need to either scatter these details across controller actions, nor let a bunch of unrelated finder methods pile up in the Entry model.

Quacking like a Model

From the controller's point of view, the decorator looks a lot like the ActiveRecord model.

class EntriesController < ApplicationController
  def show
    @entry = PublishedEntry.find(params[:id])
  end
end

To preserve Rail's magic when passing records to routing, render and form helpers, we delegate the required ActiveModel methods to the decorated Entry.

class PublishedEntry
  include ActiveModel::Conversion
  extend ActiveModel::Naming

  delegate :to_model, :to_key, :persisted?, :to => :@entry

  def initialize(entry)
    @entry = entry
    @published_revision = entry.published_revision
  end

  // ...
end

PublishedEntry now basically looks like Entry to ActionPack helpers.

A Word about Testing

Whether it makes sense to test presentational decorators in isolation, really depends on the kind of logic you end up encapsulating in them. The more tightly they are integrated with active record specific constructs like scopes or associations, the more a black box approach makes sense verifying that all parts are wired up correctly.

Making Rails 4 and Sinatra share a Session

2013-09-14T00:00:00+00:00

In our latest project, I wanted to restrict access to the resque web interface by reusing the Warden/Devise/CanCan based authorization stack of the main Rails app. I quickly found some posts explaining how to mount the resque Sinatra app side by side with the Rails app in the config.ru file. Still, I always ended up with an empty env['rack.session'] inside the Sinatra app.

As it turns out, Rails 4 changes the session middleware to use encrypted cookies. It no longer relies on Rack::Session::Cookie, but uses ActionDispatch::Session::CookieStore instead.

Unfortunately, the new CookieStore middleware cannot live on its own since the ActionDispatch::Cookies::CookieJar it uses to get cookies requires an action_dispatch.key_generator to be present on the env. If it is missing, the session cookie cannot be decrypted leading to silent failure.

Rails::Application objects place the key generator on an env_config hash, which is merged into the request's env before processing a request. To make the Rails session available to our Sinatra app, we have to insert a middleware which mimics this behaviour:

# rails_env_config_middleware.rb 
class RailsEnvConfigMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    env.merge!(Rails.application.env_config)
    @app.call(env)
  end
end

The final rack up file then looks like this:

# config.ru
require ::File.expand_path('../config/environment',  __FILE__)

map '/' do
  run Rails.application
end

map '/resque' do
  # This is the important line
  use RailsEnvConfigMiddleware

  use ActionDispatch::Session::CookieStore, :key => '_<app_name>_session', 
    :path => '/', :secret => '<secret_key_base>'

  use Warden::Manager do |manager|
    manager.failure_app = Pageflow::Application
    manager.default_scope = Devise.default_scope
  end

  run AdminResqueServer.new
end

Now we can access the authenticated user via warden and authorize with CanCan:

# admin_resque_server.rb
require 'resque/server'
require 'resque_scheduler/server'

class AdminResqueServer < Resque::Server
  before do
    redirect '/admin/login' unless authenticated?
    redirect '/' unless can?(:manage, Resque)
  end

  private

  def can?(*args)
    Ability.new(user).can?(*args)
  end

  def authenticated?
    request.env['warden'].authenticated?
  end

  def user
    request.env['warden'].user
  end
end

In our production environment, I faced a final bug. For some reason the rack-protection middleware which comes as part of the Sinatra stack kept dropping the session as a reaction to some falsely detected attack. Deactivating protection inside the Sinatra app solved the issue for now.

class AdminResqueServer < Resque::Server
  disable :protection

  ...
end

This is of course not the desired solution. I'll post an update once I have figured out the details.

Organizing Integration Test Helpers with Dominos

2013-09-10T00:00:00+00:00

The domino gem by Nick Gauthier provides a great way to organize your integration test support methods. Before, I used to package test helpers as mixins.

# spec/features/admin/user_locking_spec.rb
feature 'user locking' do
  include UsersTestHelper

  scenario 'locking a user account' do
    user = create(:user)

    visit user_path(user)
    lock_user_button.click

    expect(page).to have_locked_notice
  end
end

# spec/support/integration/users_test_helper.rb
module UsersTestHelper
  extend RSpec::Matchers::DSL

  def lock_user_button
    page.find('section.user [data-rel=lock]')
  end

  matcher :have_locked_notice do |*args|
    # check if there is an element matching .lock_notice
  end
end

While those usally made for quite readable tests, there were two major downsides:

Findability — It quickly became hard to tell where a certain helper method was defined as soon as more than one or two mixins were used in a test.
Naming clashes — Extracting private methods to clean up test helper methods came with the risk of introducing name collisions.

With dominos you can take a more object oriented approach:

# spec/support/dominos/user_page.rb
module Dom 
  UserSection < Domino
    selector 'section.user'

    def lock_button
      node.find('[data-rel=lock]')
    end

    def has_locked_notice?
      node.has_selector?('.lock_notice')
    end
  end
end

The test from above now becomes:

# spec/features/admin/user_locking_spec.rb
feature 'user locking' do
  scenario 'locking a user account' do
    user = create(:user)

    visit user_path(user)
    Dom::UserSection.first.lock_button.click

    expect(Dom::UserSection.first).to have_locked_notice
  end
end

While this arguably reads a bit more cryptic at first, there are a couple of wins:

It is now directly obvious where each methods is defined.
Private methods can freely be extracted inside dominos.
Custom matchers can be replaced by RSpec's built in predicate matchers.
It is easy to adapt a component based mental model of your UI.

While there are other great features, the domino codebase only clocks in at about 160 LOC including doc comments. So its a rather lightweight dependency to add to your testing stack.

Automating MySQL Setup for Jenkins Jobs

2013-09-07T00:00:00+00:00

Creating databases is one of those repetitive tasks when setting up continuous integration for a project. At work we use Jenkins. So we've developed the jenkins-mysql-job-databases-plugin to automate the required steps.

The plugin creates a database and a user for the job, granting access only to the job's database. User credentials and database name are exported to environment variables, which is a great fit with the ciconfiggenerator gem we saw in a previous post. In a Rails project, for example, you could commit the following file to wire everything up:

# database.yml.ci
test:
  adapter: mysql2
  encoding: utf8
  database: %{MYSQL_DATABASE}
  username: %{MYSQL_USER}
  password: %{MYSQL_PASSWORD}

The plugin itself is written in Ruby. Since it is unclear how to handle global configuration options in Ruby plugins, the credentials of the MySQL user that creates databases and grants access are hard coded in the plugin at the moment. Apart from this caveat, the plugin has worked well for us so far.

Generating Configuration Files for CI

2013-09-05T00:00:00+00:00

When setting up continuous integration for a project, manually editing host specific configuration has always annoyed me. The ciconfiggenerator gem sets out to free you from fiddling with config files in your ci job's workspace directory.

Instead, it lets you create templates for ignored config files. For each of those .yml files, simply commit an accompanying .yml.ci file and run the ci:config:generate rake task at the beginning of your ci job. While copying files, the task interpolates environment variables:

# zencoder.yml.ci
test:
  output_bucket: "com.example.test"
  api_key: "%{API_KEY}"

That way, passwords and other secret bits of data do not need to be stored in the repository, but can be configures as build parameters via your ci servers admin interface.

As an additional benefit, new developers on your team can follow a self documented and continuously tested path to obtain a passing test suite.

Sass Mixins and Icon Fonts

2013-09-01T00:00:00+00:00

Sass mixins are a great tool for building abstractions and writing intention revealing CSS code. One application lies in using them with icons fonts. Instead of littering your HTML with style specific icon classes, we can define mixins to apply icons in a declarative manner:

a.edit {
  @include pencil-icon;
}

The basic idea of the mixin definition is the following:

@mixin icon($font-name, $char) {
  &:before {
    content: $char;
    font-family: $font-name;
  }
}

@mixin pencil-icon { @include icon("entypo", "\270E"); }
@mixin warning-icon { @include icon("entypo", "\26A0"); }

This approach also makes it easy to swap icon fonts later on by limiting duplication.

Taking it further

Extracting other patterns concerning icon placement and styling, an expressive language can be built up. The following example uses some simple mixins defined in this gist to display icons in the background of navigation links.

nav a {
  @include background-icon($font-size: 20px);
}

a.edit {
  @include pencil-icon;
}

a.errors {
  @include warning-icon;
  @include background-icon-color(#d00);
  @include background-icon-animation(pulse);
}

A careful selection of such mixins might make a great gem in the vein of Bourbon or other mixin libraries.