installing git is not fun but it is more fun than a fork in your eye

Here is more or less what I did to install git on OS X and on Media Temple. Newer versions of the below files will exist by the time you are reading this i.e. now.

I got the below mostly from http://dysinger.net/2007/12/30/installing-git-on-mac-os-x-105-leopard/

# GPG (if you didn’t have it already)
curl ftp://ftp.gnupg.org/gcrypt/gnupg/gnupg-1.4.7.tar.bz2 | tar xj
cd gnupg-1.4.7
./configure
make
sudo make install
cd ..

# GetText
curl http://mirrors.usc.edu/pub/gnu/gettext/gettext-0.17.tar.gz | tar xz
cd gettext-0.17
./configure
make
sudo make install
cd ..

# GIT
curl http://kernel.org/pub/software/scm/git/git-1.5.4.4.tar.bz2 | tar xj
cd git-1.5.4.4
./configure
make
sudo make install
cd ..
curl http://www.kernel.org/pub/software/scm/git/git-manpages-1.5.4.4.tar.bz2 |
sudo tar xj -C /usr/local/share/man

# Setup GIT
git config –global user.name ‘My Name’
git config –global user.email me@mydomain.net

#can you see two dashes before “global”?

deployment on mediatemple with capistrano and git is not fun but it is more fun than doing your taxes

HAPPY TAX DAY!

Deploying to Media Temple isn’t easy. You have to add a domain, make an alternate domain and configure a lot of things through something called “mtr”. In order to deploy there with capistrano and git I first compiled git on my server to the following location:

/home/####/users/.home/usr/bin/git

Then I capify my project and copy in the below deploy.rb with my custom settings and passwords. There are some tasks included such as mtr_init, mtr_generate_htaccess and mtr_create_link that help with media temple commands.

 

deploy.rb:

require 'mt-capistrano'

#gem install mt-capistrano --source=http://gems.mediatemple.net/

 

set :site,         "SITE_NUMBER (4 digits probably)"

set :application,  "APPLICATION"

set :webpath,      "APPLICATION.com"

set :domain,       "MEDIA TEMPLE DOMAIN"

set :user,         "MEDIA TEMPLE USER"

set :password,     "MEDIA TEMPLE PASSWORD"

 

set :scm, :git

set :scm_command, "/home/####/users/.home/usr/bin/git"

set :repository, "git://github.com/ckhsponge/remindblast.git"

set :deploy_to,  "/home/#{site}/containers/rails/#{application}"

set :current_deploy_dir, "#{deploy_to}/current"

set :tmp_dir, "#{deploy_to}/tmp"

 

set :checkout, "export"

 

role :web, "#{domain}"

role :app, "#{domain}"

role :db,  "#{domain}", :primary => true

 

task :after_update_code, :roles => :app do

  put(File.read('config/database.yml'), "#{release_path}/config/database.yml", :mode => 0444)

end

 

task :mtr_init, :roles => :app do

  run "cd $HOME/../../containers && mkdir -p rails && cd rails && mkdir -p #{application} && cd #{application} && mkdir -p current && cd current && mtr add #{application} $PWD #{webpath}"

  run "mkdir -p $HOME/../../containers/rails/#{application}/shared"

  run "mkdir -p $HOME/../../containers/rails/#{application}/shared/log"

end

 

task :mtr_create_link, :roles => :app do

  run "mtr create_link #{application}"

end

 

namespace :deploy do

task :restart, :roles => :app do

  #run "mtr restart #{application} -u #{user} -p #{password}"

  #run "mtr generate_htaccess #{application} -u #{user} -p #{password}"

  run "mtr restart #{application} -u #{user} -p #{password}"

  run "mtr generate_htaccess #{application} -u #{user} -p #{password}"

  #migrate

end

end

Capistrano Remote Cache Easter Party ‘08

Thankfully we all survived Easter this year. To celebrate the resurrection of our savior we resurrected our servers into a new cluster at Engine Yard. The new cluster has plenty of room for us to grow as we need more resources. We didn’t kill our servers on Good Friday but we did have two hours of easter downtime right before midnight as we copied the database.

We discovered some new deployment recipes in the process.

Engine Yard has a gem eycap that allows remote caching and dramatically speeds up deployment for our plugin laden application. It works by checking out our code on the remote system. For every deploy the code is updated and then copied minus the .svn directories. That means an entire svn checkout is not needed!

First install the gem:

gem install eycap –source http://gems.engineyard.com

Then add this to your deploy.rb:

require ‘eycap/recipes’

set :deploy_via, :filtered_remote_cache

set :repository_cache, “/var/cache/somewhere”

The license included says anyone who has the software can use it so I think I’m allowed talk about it. Enjoy.

A Lazy Javascript, a Ruby Load and a Sponge plugin to a bar

Lazy Load Content

Sometimes a portion of your page takes a really long time to generate and render. Instead of allowing the user to spin his thumbs, you can display the bulk of content to user and send off a separate ajax request to gather the “slow” data. Lazy Load Content makes late loading a portion of your page through ajax as easy as setting up a remote_function call. With options to render fragment cached if they exist, and perform the late load if not, your app will fly like the sloth eating eagle.

Install

ruby script/plugin install  http://arperftoolkit.rubyforge.org/svn/trunk/lazy_load_content/

Copy the file javascripts/lazyLoadContent.js into your public/javascripts folder and include it with prototype.js in your views.

The Basics:

In your rhtml or haml file:

<h1> I love giraffes.. basically </h1>
<% lazy_load(:remote => {:update => 'lazyLoadContent', :url => {:action => 'lazy_load_action'}})do -%>
  <div id = "lazyLoadContent"> We wait while the giraffes are smelling themselves. </div>
<% end -%>

In your controller define the specified function

def lazy_load_action
  render :text => 'The giraffes are ready!'
end

When the page loads, the loading text is displayed: We wait while the giraffes are smelling themselves.
Then after the ajax call we see: The giraffes are ready!

…….. and more……….

ruby-xslt for windows

Porting native ruby gems to Win32 is a bitch. I have been using ruby-xslt for our XSLT needs on OS X for a while now and finally had to port it to Windows to support our minority of Windows developers.

Here is my advice for if you have to do this sort of thing:

Download the Windows SDK - turns out you can download the Windows development tools for free now. The web install wasted many hours of my time getting clogged and refusing to install and it would have been faster to just download the ISO in the first place. You don’t need all the .Net stuff.

Try to get rid of any shell scripts in the gem build process. There was a little shell to build the arguments to the linker to include the xslt libraries for example. I wasted a bunch of time trying to get it to run in the build process somehow when the real best answer was to look at the output in cygwin and put the equivalent flags straight into my Makefile.

There is a magic thing you have to do with manifests that I don’t even pretend to understand:

mt.exe -manifest my_lib_name.so.manifest -outputresource:my_lib_name.so;2

These blog posts by Al Hoang explain a lot.

You probably have to include msvcr80.dll with your thing which is installed and used by the SDK.

Replace non-portable library calls with portable calls back into Ruby, not with Windows native calls. There were a bunch of POSIX file calls in the ruby-xslt library for opening and reading libraries. Those kinds of things are really easy to replace by just calling methods on File from C.

I’d like to get this all cleaned up and made available with ruby-xslt so it can be used straight from the gem by Windows users. But it is extra work and gem install stuff is confusing and weird. Email me if ruby-xslt for Windows is something you are looking for though and maybe I will try to put some effort into it.

Oh, and these are the libxml2 and libxslt binaries precompiled for Win32.

Rails is spewing sql in my soup (better sql logging for optimizing)

Im all pissy today because Rails is spewing all these sql statements out and I have no idea where they are coming from so I can add my little optimizations or what have you. As if I wasn’t already crying in my soup.

log_sql_caller(:include => 'SELECT count(*) from giraffes', :log_level => :error) { ... }


So I wrote a little plugin to print the callstack in the log so I don’t have to read eleventy billion lines of code! Or enable some debugger of pain.

ruby script/plugin install http://arperftoolkit.rubyforge.org/svn/trunk/log_sql_caller/

Que bueno! Now our soup is thick and moist like goat stew. Yum.

Delegate to the Giraffe

I have gotten very tired of writing the same code over and over. And apparently I was so tired I forgot to publish this last year.
def foo_bar
my_member ? my_member.foo_bar : nil
end

The rails delegate method allows you to delegate a method to another class. Neat! There are two tweaks that I made to suite my salty tastes.

1. Nil freaks out. Sure you can write something like
:to => (my_member or return nil) as pointed out on the rails ticket. But that is sort of groddy. boo!

2. I want to rename the destination method sometimes. Avoid collisions, make new friends. Win-Win.

Here is delegate_x. Delegate extra! You can list out the delegated methods or, to rename them, put them in a hash of destination_method_name => target_method_name

class WebTwoPointOMGCorp
attr_accessor :giraffe

delegate_x :hovercraft, :to => :giraffe
delegate_x :to => :giraffe, :giraffe_spots => :num_spots
end

The first is equivalent to:
def hover_craft
giraffe ? giraffe.hover_craft : nil
end

and the second

def giraffe_spots
giraffe.num_spots if giraffe
end

With ActiveRecord :has_one and :belongs_to associations, check out Stefan Kaes’s piggyback plugin to piggy back attributes (define methods) to queried records.

Sweet. A little (terribly formatted) code below. Jam it in the lib or something. Make it a plugin. Just remember, the hovercraft owned by the giraffe is really the hovercraft owned by WebTwoPointOMGCorp.

(Actually I just wrote this whole post to show Joel’s awesome new spongecell giraffe that breathes fire! We love giraffes and so should you.)

Finding and Fixing Slow and Leaky Requests this MLK

Happy MLK! For this year’s MLK Spongecell has released a new ultra-beta Boxed Calendar. It’s so ultra-beta we haven’t even mentioned it on our main blog yet.

If The Reverend had been a system administrator he definitely would not have allowed slow or leaky requests on his system. Spongecell was recently experiencing a very bad leaky request. Luckily we were logging PIDS to support Rawk so simply watching our mongrels and seeing when a process exploded let me know when and which process was bad. I looked at the logs and saw a few requests for that process at that time. I ran those requests on our staging server and very quickly saw a bad sql query. Turns out some new code found some very old code and the mix wasn’t pleasant.

I’ve also been spending a little bit of time optimizing requests that Rawk reports have shown our server spends a lot of time on. For one of these MySql had decided to start using the wrong index on a paginated join. Thanks to Blythe, our resident ActiveRecord expert, we were able to change the has_many relationship to use an index hint. Here’s what the code looked like for User:

has_many(:email_addresses, :foreign_key => :user_id, :index_hint => :fk_ea_user …

Engine Yard also put in fair scheduling into our Nginx setup. By default nginx will distribute requests round-robin which is bad if one request is slow. When nginx sends to that slow mongrel the request will queue and wait for completion. With fair scheduling Nginx sends to available mongrels first thus less waiting.

I found this picture on the internet. “Leaky pipes are the biggest wasters.”

The complete nginx solution to sending flowers and files with rails

Rails send_file is effed. It may send you flowers and whisper sweet nothings into your ear, but believe me, it will steal your memory, your lover and your freetime. If you are using apache or lighttpd, all the cool kids are using the x_send_file plugin. Nginx fans, check out my monkey patch so you too can send files, flowers and love without crashing mongrels with nginx’s equivalent X-Accel-Redirect header.

1. Install the x_send_file plugin.

ruby script/plugin install http://john.guen.in/svn/plugins/x_send_file

2. Copy monkey patch to

Replace vendor/plugins/x_send_file/lib/x_send_file/controller.rb with http://www.acidlunchbox.com/blythe/rails/x_send_file/controller.rb

3. Configure things

nginx.conf

remember to also add to server section if sending over ssl

location /files {

root /data/spongecell;

internal;

}

environment.rb

#send files through nginx

XSendFile::Plugin.options.update(

:header => ‘X-Accel-Redirect’,

:root => ‘/data/spongecell’,

:file_paths => ‘/data/spongecell/files’

)

In this example, x_send_file will create the following response header

>> x_send_file ‘/data/spongecell/files/snow/iheartbigdumps.jpg’

>> response.headers['X-Accel-Redirect']

=> ‘/files/snow/iheartbigdumps.jpg’

 

Other resources:

Nginx X-Accel-Redirect

Nginx X-Accel-Redirect with rails and php examples

 

 

 

Spongecell in Vegas or Why Good Technology doesn’t always equal a Good Product

Trick or Treat?

Spongecell was in Vegas last weekend! I’m sure you’ve all seen Shakira’s Spongecell calendar on her myspace page by now. We were fortunate enough to get some face time with her, er, we ran into her in the lobby of our hotel. Other highlights were seeing Daft Punk at Vegoose and participating in MIA’s intimate performance at the Hard Rock benefitting breast cancer.

We did a lot more than just schmoozing with big names while we were there. Here is Blythe fixing bugs with me by the pool at Ceasar’s Palace. Also is a picture of James at the Silent Disco. His pic didn’t come out too well but I posted it anyways because it reminded me of a previous posting.

We also were reminded of a valuable lesson while in Vegas: good technology doesn’t always equal a good product. We went to a Silent Disco. Everyone gets headphones and the music is radioed in from the DJ. I imagine that I could have liked this event. Unfortunately they decided to book the DJ from my high school prom.