Obama Day Caching Funtime

January 30, 2009
Obama Day was caching day!

Obama Day was caching day!

Obama day was sort of intense! Not only did Aretha Franklin have a sweet hat, but all the internet traffic on CNN Live put the hurt on old skool Spongecell Promote. Chris  saved the day by quickly page caching the content that was in high demand. So basically some files were copied and everyone was happy (except maybe Texas?)

This trick is super obvi, but hella useful, yo. But, <tear>, its not a permanent fix. Alas, not every day can be Obama Day ! So I set out to fix this permanently. We need to page cache for one customer who always sends the same request, expire the page when the data changes, and action cache for everyone else who is not logged in.

So I wrote a tiny little conditional page cache plugin to let you choose your caching type at runtime. You want to page cache giraffe 1, action cache giraffe 2 when they are green and serve everything else straight up.

class PictureController

  conditionally_cache :giraffe

  def page_cache?; @giraffe == 1; end
  def action_cache?; @giraffe = 2 && @giraffe_color = 'blue'; end

  def giraffe
    @giraffe = blah_blah
  end
end

To clarify the temporary caching trick:
Suppose you have: http://snowgiraffe.com/pictures/giraffe/1.html

Then you simply move the file to:
#{RAILS_ROOT}/public/pictures/giraffe/1.html


Fastest CDN to Sacramento

August 27, 2008

I tested content delivery to a server we have at Engine Yard in Sacramento.

We are testing CDN’s for our flash delivery so I tested with a 160kb swf file. Media Temple doesn’t claim to be a CDN but I tested it anyways on it’s budget grid server. Panther and Edgecast are CDNs. I did all testing using apache bench from a slice in Engine Yard cluster EY01. The Intranet test was with content from a slice in their cluster EY02. This was a baseline since the servers are probably only meters from each other. S3′s latency is a big reason we’re looking for a good CDN.

.mac was the surprise contender and they only cost $100 a year. Perhaps we were just geographically lucky. They limit to 100gb a month and we hope to be serving more flash than that so .mac is not a realistic solution. Their consumer agreement is not sufficient as well but in reality neither is S3′s.

I tested 10,000 requests with a concurrency of 10. I repeated the ~4 minute test to make sure numbers were settled. The huge bias was location and perhaps time of day. Tests were done at noon on 8/27/08.

Conclusion: the two CDNs performed better than the non-CDN delivery. This data isn’t fair nor significant enough to rule one CDN better than the other.


Embedded Calendars and Github Trolling

June 7, 2008

I was trolling github to see what kind of calendar projects were out there and found calendar_helper. It looks decent. I would probably use it if I needed to display a simple calendar.

For displaying a rich calendar in html the best solution is of course to use Spongecell’s Monthly Online Calendar:

These calendars can easily be created at Spongecell by creating some events and then getting code for your site. For a dynamic calendar built into your web application these calendars can be modified and created using the Spongecell API. A good open source example of this can be found at Spongewolf.

Github is my new favorite social network. My friends are defined by people watching my projects and the projects I am watching. There’s a lot less trashiness and nudity than on other social networks I try to avoid such as MySpace and LinkedIn.


deployment on mediatemple with capistrano and git is not fun but it is more fun than doing your taxes

April 15, 2008

HAPPY TAX DAY!

Deploying to Media Temple isn’t easy. You have to add a domain, make an alternate domain and configure a lot of things through something called “mtr”. In order to deploy there with capistrano and git I first compiled git on my server to the following location:

/home/####/users/.home/usr/bin/git

Then I capify my project and copy in the below deploy.rb with my custom settings and passwords. There are some tasks included such as mtr_init, mtr_generate_htaccess and mtr_create_link that help with media temple commands.

 

deploy.rb:

require 'mt-capistrano'

#gem install mt-capistrano --source=http://gems.mediatemple.net/

 

set :site,         "SITE_NUMBER (4 digits probably)"

set :application,  "APPLICATION"

set :webpath,      "APPLICATION.com"

set :domain,       "MEDIA TEMPLE DOMAIN"

set :user,         "MEDIA TEMPLE USER"

set :password,     "MEDIA TEMPLE PASSWORD"

 

set :scm, :git

set :scm_command, "/home/####/users/.home/usr/bin/git"

set :repository, "git://github.com/ckhsponge/remindblast.git"

set :deploy_to,  "/home/#{site}/containers/rails/#{application}"

set :current_deploy_dir, "#{deploy_to}/current"

set :tmp_dir, "#{deploy_to}/tmp"

 

set :checkout, "export"

 

role :web, "#{domain}"

role :app, "#{domain}"

role :db,  "#{domain}", :primary => true

 

task :after_update_code, :roles => :app do

  put(File.read('config/database.yml'), "#{release_path}/config/database.yml", :mode => 0444)

end

 

task :mtr_init, :roles => :app do

  run "cd $HOME/../../containers && mkdir -p rails && cd rails && mkdir -p #{application} && cd #{application} && mkdir -p current && cd current && mtr add #{application} $PWD #{webpath}"

  run "mkdir -p $HOME/../../containers/rails/#{application}/shared"

  run "mkdir -p $HOME/../../containers/rails/#{application}/shared/log"

end

 

task :mtr_create_link, :roles => :app do

  run "mtr create_link #{application}"

end

 

namespace :deploy do

task :restart, :roles => :app do

  #run "mtr restart #{application} -u #{user} -p #{password}"

  #run "mtr generate_htaccess #{application} -u #{user} -p #{password}"

  run "mtr restart #{application} -u #{user} -p #{password}"

  run "mtr generate_htaccess #{application} -u #{user} -p #{password}"

  #migrate

end

end

ruby-xslt for windows

March 14, 2008

Porting native ruby gems to Win32 is a bitch. I have been using ruby-xslt for our XSLT needs on OS X for a while now and finally had to port it to Windows to support our minority of Windows developers.

Here is my advice for if you have to do this sort of thing:

Download the Windows SDK – turns out you can download the Windows development tools for free now. The web install wasted many hours of my time getting clogged and refusing to install and it would have been faster to just download the ISO in the first place. You don’t need all the .Net stuff.

Try to get rid of any shell scripts in the gem build process. There was a little shell to build the arguments to the linker to include the xslt libraries for example. I wasted a bunch of time trying to get it to run in the build process somehow when the real best answer was to look at the output in cygwin and put the equivalent flags straight into my Makefile.

There is a magic thing you have to do with manifests that I don’t even pretend to understand:

mt.exe -manifest my_lib_name.so.manifest -outputresource:my_lib_name.so;2

These blog posts by Al Hoang explain a lot.

You probably have to include msvcr80.dll with your thing which is installed and used by the SDK.

Replace non-portable library calls with portable calls back into Ruby, not with Windows native calls. There were a bunch of POSIX file calls in the ruby-xslt library for opening and reading libraries. Those kinds of things are really easy to replace by just calling methods on File from C.

I’d like to get this all cleaned up and made available with ruby-xslt so it can be used straight from the gem by Windows users. But it is extra work and gem install stuff is confusing and weird. Email me if ruby-xslt for Windows is something you are looking for though and maybe I will try to put some effort into it.

Oh, and these are the libxml2 and libxslt binaries precompiled for Win32.


Rails is spewing sql in my soup (better sql logging for optimizing)

February 12, 2008

Im all pissy today because Rails is spewing all these sql statements out and I have no idea where they are coming from so I can add my little optimizations or what have you. As if I wasn’t already crying in my soup.

log_sql_caller(:include => 'SELECT count(*) from giraffes', :log_level => :error) { ... }

So I wrote a little plugin to print the callstack in the log so I don’t have to read eleventy billion lines of code! Or enable some debugger of pain.

ruby script/plugin install http://arperftoolkit.rubyforge.org/svn/trunk/log_sql_caller/

Que bueno! Now our soup is thick and moist like goat stew. Yum.


Spongecell in Vegas or Why Good Technology doesn’t always equal a Good Product

November 1, 2007

Trick or Treat?

Spongecell was in Vegas last weekend! I’m sure you’ve all seen Shakira’s Spongecell calendar on her myspace page by now. We were fortunate enough to get some face time with her, er, we ran into her in the lobby of our hotel. Other highlights were seeing Daft Punk at Vegoose and participating in MIA’s intimate performance at the Hard Rock benefitting breast cancer.

We did a lot more than just schmoozing with big names while we were there. Here is Blythe fixing bugs with me by the pool at Ceasar’s Palace. Also is a picture of James at the Silent Disco. His pic didn’t come out too well but I posted it anyways because it reminded me of a previous posting.

Ceasar’s PalaceSilent Disco James

We also were reminded of a valuable lesson while in Vegas: good technology doesn’t always equal a good product. We went to a Silent Disco. Everyone gets headphones and the music is radioed in from the DJ. I imagine that I could have liked this event. Unfortunately they decided to book the DJ from my high school prom.


Up all nite with the sponge

November 15, 2006

Every developer have been up all night doing something completely ridiculous: forgetting a semi-colon and screwing your JS compression, writing hot new code for your acts_as_idiot functionality that unbeknownst to you the RoR team has already implemented and included in the latest version. Once in the C++ days I even forgot a bracket and to deallocate memory. Regardless, there are so many tricks I wish I knew before I was up all night with the sponge.

Let’s take our kimonos off and run around naked. I will show you mine if you show me yours.Kimono
Read the rest of this entry »


Follow

Get every new post delivered to your Inbox.