Ruby on Rails and Unicode (Nothing is perfect)

Unfortunately Ruby on Rails (and Ruby in general) does not support Unicode out of the box. This sucks. It is possible to use Unicode in Rails projects as shown on Rails Wiki. After 2 hours of testing I've found that almost everything works good. I even asked Julik (author of unicode_hacks plugin) if it's true what the wiki says about bugs triggered by these hacks. He told me to be careful around ActionMailer. I instantly did some testing and found it works ok (besides I had to explictly set base64 encoding for email body, but it's ActionMailer problem).

One thing that annoyed me was Google WSDL driver (by soap4r) which no longer works. This issue is connected to $KCODE variable which is set to 'UTF8' to make all this Unicode stuff work. When it's set GoogleSearch doesn't work. I tried to set proper soap envelope options but it didn't work so I've come with this ugly hack:

 begin
   # utf-8 workaround
   kcode = $KCODE # no longer needed
   $KCODE = "" # Updated: no longer needed
   driver = SOAP::WSDLDriverFactory.new("http://api.google.com/GoogleSearch.wsdl").create_rpc_driver

   # UPDATED: following line is no longer needed
   #driver.options["soap.envelope.use_numeric_character_reference"] = true
   #driver.wiredump_dev = STDOUT
 rescue
   puts "Error creating rpc driver: " + $!
   return
 ensure
   $KCODE = kcode # Updated: no longer needed
 end

I don't like it. If you know the better way, let me know.


UPDATE:

I turned off unicode_hacks and now everything works. Soap4r was broken because of overriden versions of String methods. So now I have SOAP, ActionMailer and Rails in general working. Only things (but essential!) that left are broken (unicode-unaware) String methods (I live with jcode).

The reason why all was broken before, was unicode versions of String methods provided by Julik's unicode_hacks (among others tweaks). They're used deeply in the code of Soap4r (which is anyway aware of existence of Unicode) despite these methods don't know anything about multibyte characters.

Don't get me wrong: unicode_hacks are the only way if you want to do some serious work with multibyte character strings. You can also take a glance at Unicode Library for Ruby by Yoshida Masato.

I have to do more tests but my recommendation for today is:

• read HowToUseUnicodeStrings
  
• set $KCODE = 'UTF8' (shorthand version: 'u')
• put encoding: utf8 to your database.yml
• set your db of choice to use utf-8 to store data
• use jcode / Unicode library if you need

Note: I assume you're livin' on the EdgeRails or at least Rails 1.1.0.

Track keywords in Ruby on Rails using Google API

Below is essential part of code which tracks ranks of keywords in Google. As almost always in Ruby world, this source code is self-explanatory. It was partially inspired by this Bernard Peh's article. If you are PHP developer and willing to become programming language polyglot, compare length of relevant code and its readablity. If you think this snippet sucks in any way, comment on! Some variables in the code are defined outside of its scope, but it's not hard to find which.

Bonus tip: Ruby SOAP translates Google API field URL from resultElement structure to method uRL!

  def update_link_rank(iterations = 10)
    # sanity chceck
    iterations = 10 if iterations < 1 || iterations > 100
    
    require 'soap/wsdlDriver'

    begin
      driver = SOAP::WSDLDriverFactory.
new("http://api.google.com/GoogleSearch.wsdl").create_rpc_driver
    rescue
      puts "Error creating rpc driver: " + $!
      return
    end

    for i in (0...iterations) do
      begin
        result = driver.doGoogleSearch(
user.google_api_key, anchor, 0 + i, 10 + i, true, "", false, "", "", "")
      rescue SOAP::FaultError
        puts "Got Fault: " + $!
        return
      end
      
      partial_rank = 0
      for resultElement in result.resultElements do
        partial_rank += 1
        if resultElement.uRL.include? href then
          value = i * 10 + partial_rank
          rank = LinkRank.new
          rank.link = self
          rank.value = value
          rank.save
          return value
        end
      end

      # shorter than 10 means no next result
      return if result.resultElements.size < 10 
    end
  end

Ruby on Rails served by Lighttpd behind reverse proxy on Apache 2.0

Summary: If you want Apache 2.0, PHP and Rails, do yourself a favor, use reverse proxy and Lighttpd.

After few unproductive hours spent fighting FastCGI and Apache 2.0 and far too much coffee drunk, I was forced to make a decision.

Our Apache server hosts PHP applications for a dozen of our clients and needs to work very stable and uninterruptedly. There is also an urgent need to host Rails applications on the same server, namely the same IP and in some cases simultaneously in the same URL namespace with PHP applications. This is a bit tricky but useful setup if you take the possibilities into account.

O.K. Lets get to work. We have the Apache server (btw. on Debian, so we will do it Debian way). We want to minimize changes in its configuration. We definitely need to enable mod_proxy:

a2enmod proxy

Then we need to configure it, so we create file /etc/apache2/sites-available/lighty-rproxy and put some date in it:

<VirtualHost *>
  ServerName example.com

  <Proxy *>
    Order deny,allow
    Allow from all
  </Proxy>

  <Location />
    ProxyPass http://localhost:81/
    ProxyPassReverse http://localhost:81/
  </Location>

  ProxyPreserveHost On
</VirtualHost>

Here we enable this vhost:

a2ensite lighty-rproxy
/etc/init.d/apache2 force-reload

If you now point your browser to configured domain (we use famous example.com) you should see information about "Bad Gateway". Don't panic. We have all under control. Inhale... Exhale... Yes. Like this.

What we just saw was reaction of Apache which wanted to redirect everything inside exmaple.com to another http server on the same host (localhost) but on port 81. So now we need to put something there.

We install Lighttpd:

apt-get install lighttpd

and configure it. Below are excerpts from /etc/lighttpd/lighttpd.conf we need to change. First we enable mod_rewrite and mod_redirect:

server.modules              = (
            "mod_access",
            "mod_alias",
            "mod_accesslog",
            "mod_rewrite",
            "mod_redirect",
#           "mod_status",
#           "mod_evhost",
#           "mod_compress",
#           "mod_usertrack",
#           "mod_rrdtool",
#           "mod_webdav",
#           "mod_expire",
#           "mod_flv_streaming",
#           "mod_evasive"
 )

Then we bind lighty to localhost:81 to correspond to Apache configuration:

## bind to port (default: 80)
server.port               = 81

## bind to localhost only (default: all interfaces)
#server.bind                = "localhost"

In file /etc/lighttpd/conf-available we have to comment fastcgi server definition for php (unless we have php4-cgi installed) and enable fastcgi in Lighty by invoking:

lighty-enable-mod fastcgi

Last thing we need is our humble vhost configuration for Rails app which sits in /etc/lighttpd/conf-available/11-rails-app.conf and looks like this:

var.app = "/home/necro/rails/linkpro"

$HTTP["host"] == "example.com" {
  server.document-root = var.app + "/public"
  url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )
  server.error-handler-404 = "/dispatch.fcgi"
  fastcgi.server = ( ".fcgi" =>
    ( "localhost" =>
      ( "min-procs" => 2,
        "max-procs" => 2,
        "socket" => "/tmp/app.fcgi.socket",
        "bin-path" => var.app + "/public/dispatch.fcgi",
        "bin-environment" => ( "RAILS_ENV" => "development" )
      )
    )
  )
}

As you can see this configuration is pretty simple but works and is a good start for more complicated setups where you separate application server from http server or create other sophisticated configurations. Just remember to properly set rights to log and tmp directories in your Rails app. Lighty needs to write to them. I overlooked this at first and lost some time so you don't have to.

Feedback strongly appreciated.

Sunday, bloody sunday

To name a few, I've installed and partially tested and tuned the configuration of this software (for security reasons):

• apache 2
• mysql 4.1
• php 4.1
• phpmyadmin
  
• cpan
  
• vhcs 2
• nagios
• snort
• snoopy
• trac
• subversion
• ruby
• rails
• etc.
  

If you have any suggestions for software (and/or libraries) you'd like to use one our shiny hosting, drop us a note. We're here for you.

We've (Tomek and me) also started translating vhcs into Polish because of our legacy base of customers. There are 1,000 strings to carefully translate so it will take one week at least.

Today's question is: what software or feature would you like to have on your hosting plan of dreams served by Trix? Be creative answering this.

News from the field

This one is about time and that's why is so short. As you probably know, we're moving the blog (and everything connected to Trix) somewhere else. First we thought to move it to our virtual server but now we're buying a dedicated server at hetzner.de. We can not stretch the time so it will take longer. We are testing now our server environment (read Debian) and next week will be deploying it on the shiny new dedicated server at hetzner. Stay tuned.