302 Permanently Moved!

We're now at trixnews.com. Please update your links. This blog will not be updated anymore.

Happy deployment with Capistrano and Rails 1.1

Today is busy as always but I have some info I want to share :)

We have finally migrated to our new co-located server (on 4Gbit/s connection yeah!) and started to prepare it for our incoming app. It's being built on Rails. Our setup is almost typical:

• Apache2 as a reverse proxy to Lighttpd
• Lighttpd
• FCGI listeners running independently from Lighty (with spawner/reaper)
  
• Rails (1.1/Edge) app deployed with Capistrano
• MySQL 5.0
• etc.

Everything is working almost out of the box. However one thing was tricky to set up: Capistrano deployment and spawner/reaper. There is no spinner i Rails 1.1 (I don't remember when it disappeared) but spawner can respawn fcgi processes if they die.

The problem is that spawner writes pid files into the tmp/pids directory of your current deployed version of app. When you deploy new version, you want to invoke reaper script to restart fcgi listeners, but the pid files are now in the previous release (yes it sounds complicated). We don't want to kill poor spawner and his beloved kids. We want them to live forever! So the trick is to copy pid files from previous release to current one (simple huh?). Below is a custom Capistrano restart task:

desc "Restarts fcgi listeners"
task :restart, :roles => :app do
  puts "Restarting fcgi listeners..."
  run <<-CMD     
    cp -r #{previous_release}/tmp/pids #{current_path}/tmp/ &&                  
    #{current_path}/script/process/reaper   
  CMD 
end 

One thing that annoyed me (not any more) was the fact fcgi processes listen on wildcard ip address (0.0.0.0) which is not wanted for obvious reasons (we are on a single machine). This is another custom Capistrano task which is invoked only on so-called "cold_deploy":

desc "Starts fcgi listeners"
task :spawn, :roles => :app do
  puts "Starting fcgi listeners..."
  run "#{current_path}/script/process/spawner -r 5 -s \"/usr/bin/env spawn-fcgi -a 127.0.0.1\""
end

Although on a single machine now, we're prepared for moving application servers to another box if the demand for our services grows beyond expectations (and we hope it will!).

Ruby on Rails and Unicode (Nothing is perfect)

Unfortunately Ruby on Rails (and Ruby in general) does not support Unicode out of the box. This sucks. It is possible to use Unicode in Rails projects as shown on Rails Wiki. After 2 hours of testing I've found that almost everything works good. I even asked Julik (author of unicode_hacks plugin) if it's true what the wiki says about bugs triggered by these hacks. He told me to be careful around ActionMailer. I instantly did some testing and found it works ok (besides I had to explictly set base64 encoding for email body, but it's ActionMailer problem).

One thing that annoyed me was Google WSDL driver (by soap4r) which no longer works. This issue is connected to $KCODE variable which is set to 'UTF8' to make all this Unicode stuff work. When it's set GoogleSearch doesn't work. I tried to set proper soap envelope options but it didn't work so I've come with this ugly hack:

 begin
   # utf-8 workaround
   kcode = $KCODE # no longer needed
   $KCODE = "" # Updated: no longer needed
   driver = SOAP::WSDLDriverFactory.new("http://api.google.com/GoogleSearch.wsdl").create_rpc_driver

   # UPDATED: following line is no longer needed
   #driver.options["soap.envelope.use_numeric_character_reference"] = true
   #driver.wiredump_dev = STDOUT
 rescue
   puts "Error creating rpc driver: " + $!
   return
 ensure
   $KCODE = kcode # Updated: no longer needed
 end

I don't like it. If you know the better way, let me know.


UPDATE:

I turned off unicode_hacks and now everything works. Soap4r was broken because of overriden versions of String methods. So now I have SOAP, ActionMailer and Rails in general working. Only things (but essential!) that left are broken (unicode-unaware) String methods (I live with jcode).

The reason why all was broken before, was unicode versions of String methods provided by Julik's unicode_hacks (among others tweaks). They're used deeply in the code of Soap4r (which is anyway aware of existence of Unicode) despite these methods don't know anything about multibyte characters.

Don't get me wrong: unicode_hacks are the only way if you want to do some serious work with multibyte character strings. You can also take a glance at Unicode Library for Ruby by Yoshida Masato.

I have to do more tests but my recommendation for today is:

• read HowToUseUnicodeStrings
  
• set $KCODE = 'UTF8' (shorthand version: 'u')
• put encoding: utf8 to your database.yml
• set your db of choice to use utf-8 to store data
• use jcode / Unicode library if you need

Note: I assume you're livin' on the EdgeRails or at least Rails 1.1.0.

Track keywords in Ruby on Rails using Google API

Below is essential part of code which tracks ranks of keywords in Google. As almost always in Ruby world, this source code is self-explanatory. It was partially inspired by this Bernard Peh's article. If you are PHP developer and willing to become programming language polyglot, compare length of relevant code and its readablity. If you think this snippet sucks in any way, comment on! Some variables in the code are defined outside of its scope, but it's not hard to find which.

Bonus tip: Ruby SOAP translates Google API field URL from resultElement structure to method uRL!

  def update_link_rank(iterations = 10)
    # sanity chceck
    iterations = 10 if iterations < 1 || iterations > 100
    
    require 'soap/wsdlDriver'

    begin
      driver = SOAP::WSDLDriverFactory.
new("http://api.google.com/GoogleSearch.wsdl").create_rpc_driver
    rescue
      puts "Error creating rpc driver: " + $!
      return
    end

    for i in (0...iterations) do
      begin
        result = driver.doGoogleSearch(
user.google_api_key, anchor, 0 + i, 10 + i, true, "", false, "", "", "")
      rescue SOAP::FaultError
        puts "Got Fault: " + $!
        return
      end
      
      partial_rank = 0
      for resultElement in result.resultElements do
        partial_rank += 1
        if resultElement.uRL.include? href then
          value = i * 10 + partial_rank
          rank = LinkRank.new
          rank.link = self
          rank.value = value
          rank.save
          return value
        end
      end

      # shorter than 10 means no next result
      return if result.resultElements.size < 10 
    end
  end

Ruby on Rails served by Lighttpd behind reverse proxy on Apache 2.0

Summary: If you want Apache 2.0, PHP and Rails, do yourself a favor, use reverse proxy and Lighttpd.

After few unproductive hours spent fighting FastCGI and Apache 2.0 and far too much coffee drunk, I was forced to make a decision.

Our Apache server hosts PHP applications for a dozen of our clients and needs to work very stable and uninterruptedly. There is also an urgent need to host Rails applications on the same server, namely the same IP and in some cases simultaneously in the same URL namespace with PHP applications. This is a bit tricky but useful setup if you take the possibilities into account.

O.K. Lets get to work. We have the Apache server (btw. on Debian, so we will do it Debian way). We want to minimize changes in its configuration. We definitely need to enable mod_proxy:

a2enmod proxy

Then we need to configure it, so we create file /etc/apache2/sites-available/lighty-rproxy and put some date in it:

<VirtualHost *>
  ServerName example.com

  <Proxy *>
    Order deny,allow
    Allow from all
  </Proxy>

  <Location />
    ProxyPass http://localhost:81/
    ProxyPassReverse http://localhost:81/
  </Location>

  ProxyPreserveHost On
</VirtualHost>

Here we enable this vhost:

a2ensite lighty-rproxy
/etc/init.d/apache2 force-reload

If you now point your browser to configured domain (we use famous example.com) you should see information about "Bad Gateway". Don't panic. We have all under control. Inhale... Exhale... Yes. Like this.

What we just saw was reaction of Apache which wanted to redirect everything inside exmaple.com to another http server on the same host (localhost) but on port 81. So now we need to put something there.

We install Lighttpd:

apt-get install lighttpd

and configure it. Below are excerpts from /etc/lighttpd/lighttpd.conf we need to change. First we enable mod_rewrite and mod_redirect:

server.modules              = (
            "mod_access",
            "mod_alias",
            "mod_accesslog",
            "mod_rewrite",
            "mod_redirect",
#           "mod_status",
#           "mod_evhost",
#           "mod_compress",
#           "mod_usertrack",
#           "mod_rrdtool",
#           "mod_webdav",
#           "mod_expire",
#           "mod_flv_streaming",
#           "mod_evasive"
 )

Then we bind lighty to localhost:81 to correspond to Apache configuration:

## bind to port (default: 80)
server.port               = 81

## bind to localhost only (default: all interfaces)
#server.bind                = "localhost"

In file /etc/lighttpd/conf-available we have to comment fastcgi server definition for php (unless we have php4-cgi installed) and enable fastcgi in Lighty by invoking:

lighty-enable-mod fastcgi

Last thing we need is our humble vhost configuration for Rails app which sits in /etc/lighttpd/conf-available/11-rails-app.conf and looks like this:

var.app = "/home/necro/rails/linkpro"

$HTTP["host"] == "example.com" {
  server.document-root = var.app + "/public"
  url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )
  server.error-handler-404 = "/dispatch.fcgi"
  fastcgi.server = ( ".fcgi" =>
    ( "localhost" =>
      ( "min-procs" => 2,
        "max-procs" => 2,
        "socket" => "/tmp/app.fcgi.socket",
        "bin-path" => var.app + "/public/dispatch.fcgi",
        "bin-environment" => ( "RAILS_ENV" => "development" )
      )
    )
  )
}

As you can see this configuration is pretty simple but works and is a good start for more complicated setups where you separate application server from http server or create other sophisticated configurations. Just remember to properly set rights to log and tmp directories in your Rails app. Lighty needs to write to them. I overlooked this at first and lost some time so you don't have to.

Feedback strongly appreciated.

Sunday, bloody sunday

To name a few, I've installed and partially tested and tuned the configuration of this software (for security reasons):

• apache 2
• mysql 4.1
• php 4.1
• phpmyadmin
  
• cpan
  
• vhcs 2
• nagios
• snort
• snoopy
• trac
• subversion
• ruby
• rails
• etc.
  

If you have any suggestions for software (and/or libraries) you'd like to use one our shiny hosting, drop us a note. We're here for you.

We've (Tomek and me) also started translating vhcs into Polish because of our legacy base of customers. There are 1,000 strings to carefully translate so it will take one week at least.

Today's question is: what software or feature would you like to have on your hosting plan of dreams served by Trix? Be creative answering this.

News from the field

This one is about time and that's why is so short. As you probably know, we're moving the blog (and everything connected to Trix) somewhere else. First we thought to move it to our virtual server but now we're buying a dedicated server at hetzner.de. We can not stretch the time so it will take longer. We are testing now our server environment (read Debian) and next week will be deploying it on the shiny new dedicated server at hetzner. Stay tuned.

Buffering...

If you click play on the video from last post you will not be happy. Your eyes will be given a black screen to watch and nice sentence to read: "This video is currently not available. Please try again later". Ain't cool, huh? WTF? I've clicked the video two days ago and watched it successfully. Apparently the one who uploaded this clip to Google Video erased it later. How come? Well...

It was this person's free choice. This is so Web2.0. You add content. You build smart mob, collective intelligence etc., finally, you have ability to erase content whenever you want. Content is yours. Google Video provides a platform. Nothing else.

Hmmm... So what about permalinks? How to accept these two excluding pillars of Web2.0?
Can anybody tell me? Anyone?

Who is really behind all this Web2.0 thing (watch the video)

Any comments?

Wireless entertainment considered cool

I've just put PSPRadio on my PSP and I can listen to my favorite Frisky Radio virtually everywhere in my house, even in my backyard which is very practical considering incoming summer :) This is what I call technology. Great. I just need some PDF reader for my shiny PSP. Great work Raf! PSP Radio is great piece of software! Keep it up!

Just before the move aka migration

This saturday I'm finally going to finish my little blog engine in Rails. This was a busy week and I've had no time to write antyhing although so much happened. I hope I'll be able to write more as soon as we move to new domain and new engine. I know I could use some already well known blog engine, but hell, how would I learn anything?

So be patient and enjoy the weekend!

Short note only tonight.

Our goals are more clear than ever. We are starting to see the essence. 

Good Luck and Good Night
(This "blog item" was written using MacOSX Widget :)

U. S. Patents BS (Balthaser case)

I just took a glance at the recently acquired Balhaser's patent (which is valid in USA anyway) and it made me laugh. It was filled in 2001 and they've just described normal process of developing and using desktop application with one exception: they've added this expression: "via Internet". I won't comment on this more because I have to fill my own patent form. I want to patent "Methods, systems and processes for the design and creation of emotion-rich applications via Internet". And I don't care about prior act. Any comments? A little flame war?

Update: I've just signed up for free version of Balthaser's "design system". It's just a website editor in Flash. It looks nice but it's closed and I doubt it will get wide acceptance. So I was quite right that they've just brought application development to your browser. They've embedded IDE in Macromedia Flash presentation. And indeed, prior art for their lame patent exists.

Extracting the essence

Recently I've been overwhelmed (Have you too?) by Web2.0 stuff. Clearly, I needed some time to aggregate and filter all the information on this exciting topic. So this post is my try to extract and enumerate the most important characteristics (at least for me) of the before mentioned topic and how to exploit them in a business way. (read: how to make money with Web2.0).

Because the difference between "taxonomy" and "folksonomy", the very important thing was achieved: ease of building logical and searchable structure of user-contributed content. When you compare, let's say the biggest catalogue of Web1.0 (especially the way it's built) and possibilities of simple tagging (del.icio.us) from the point of view of average user, you will admit it's the brilliant way to exploit social nature of human.

Nowadays, everyone and everything is going mobile, we want to use everything on the go. Personally, I read e-books on my way to work and back, take pictures and immediately send them to my flickr account, play games, call friends and co-workers, and so on. I try to use every moment spent alone to do something (even if it's having fun or resting actively). Before you ask, I am not some freaky geek without social life. I just don't like wasting my time. I took myself as an example of this trend which escalates among youth even faster. Implementing Web2.0 principles in mobile world we can further exploit all this "social thing". Possibilities here are enormous. I bet you can come with some bright ideas quickly (don't hesitate to comment on them).

Today I've read article describing research on strength of Internet ties and the way Internet users socialize comparing to ones who don't use The Network at all. The results can be condensated into this sentence: "People who use Internet are far more social beings". They mention previous research results which imply something opposite, but are much older and I think just not fresh and thus irrelevant.

The whole Web2.0 thing is about delivering easier, more useful ways to enrich our social lives by creating the infinite flow of information, knowledge, content, interaction etc. interconnected with links. And content is still the king. Links are the base of The World Wide Web. Everyone (even my father) knows what link is. The Web2.0 is still about links. Links are still base of The Web.

I think I'll need at least one more try to come with more clearer extract of business and social aspects of Web2.0 movement. This is for now. Comments are appreciated.

Structured Blogging, Web2.0 essence

Some time ago I've read an article. It was full of terms such as semantic web, intelligent content and other buzzwords. I have to admit I've not understood a word. It was so blurry then. I don't really remember when it was - one or two years ago? Sounds probably. One thing I remember is this (famous now) Web2.0.

Today, after 2 hours of brainstorming with one of my friends (a very good marketer) we've begun to ask ourselves what is the essence of Web2.0 and how Web2.0 looks or will look like.

We are in the middle of making a list of features for our new product (hopefully killer app). Last months we've been moving around the concepts of Ajax and have used the term Ajax interchangeably with Web2.0.

We were missing the point.

Web2.0 is not about Ajax. It's about those buzzwords I stumbled upon last year ago. Semantic web is by far the more important one. Semantic stands for meaning. You may think: but the meaning is already here, in every sentence on the Web. It's true. But for machines it's not. They don't understand the meaning of sentences written in natural human readable form. They need something else. Hints. Decorating the content of the Web1.0 with hints for computers makes Web2.0 come true. Structured Blogging is the first place where you can read more on this topic and be finally enlightened. It's a good word considering the impact Web2.0 will have on Internet.

My prediction is simple: Revolution comes. Don't sleep through this opportunity to make the world better (and get rich).

Any comments? Ideas on how to ehm.. get rich "using this tools and techniques"? (David Deangelo's beloved question)

Home sweet home

After approx. 13h of work today, I am finally home. 13h of pure adrenaline fighting with so called database. You guess, I will only tell that it sometimes let you have the same value in unique column twice. I have witnesses.

In few weeks this nightmare will come to an end. Let's hope to the happy end. And how this imaginary happy end looks like?

1. TCO is much lower than now.
2. Maintenance is easier.
3. Stability of rock.
4. No licensing issues.
   
5. No more 13h days at work.

I don't want more. This would be enough. After the switch we finally will be free from expensive licensing although I don't consider this main reason this migration. I've just lost too many evenings like this one using this piece of software. It can be good, but not for our purposes. Our needs developed greatly and we have to move on. We will never miss you. Ever.

Farewell,
adieu,
au revoir,
ciao,
adios,
bye-bye,
so long,
see you later,
see you,
sayonara,
bon voyage,
cheers,
toodle-oo,
żegnaj

Microsoft Access!!!

Skype, MacOS X, Bluetooth headset. Holy grail?

I always wanted to work at home. To have the safety of being next to my bed and my computer at the same moment. To have the ability of taking a short nap every moment I want. Some of you may think it's hard to maintain discipline. And you're right.

But let's go back to the topic. If you work at home you're obliged to have a rather good Internet connection. Lousy dial-ups won't do any good. 512Kb/s will satisfy almost all of your needs (except downloading these Oracle databases from production servers).

Next thing you'll need is a very comfortable operating system. You have a choice. I use MacOS X myself and find it excellent. You also have to set up all applications you use to work with. Sometimes it's can become tricky easily when you use very advanced and rare tools which are impossible to install on a typical home workstation but in most cases there is a solution.

Then you will need some way to cooperate with your boss and co-workers. I would recommend here some of tools made by 37signals. Personally Basecamp and recently created Campfire are my favorites.

Last thing you need is some voice instant messaging. Skype is the most popular one, but there's one catch. Windows versions are far more stable than MacOS X, and Linux versions. I used all of them. Linux and MacOS X versions are far behind considering functionality. I don't blame Skype programmers. They're have a business to run and Microsoft Windows users are the vast majority. To make Skype work you need some headset of course. I personally use Bluetooth headset from Logitech, but sometimes it drives me crazy. This is common to all wireless devices I know. Batteries will run out of energy only when you're speaking with your boss about the recent crash of your program or whatever you do to pay your bills. They also tend to loose connection without reason. Murphy's Laws at their glory.


These would be all you need.

I work in office mostly and at home partially, mostly Saturdays and when it rains too heavily. I think I am more productive working this way. Changing work conditions helps me to come with good and innovative ideas more often. And if I have worse day I can sleep longer and do my assignments later but more enjoyable.


If you don't agree drop some comment, they're always read and answered.

Airport successfully installed. Two screws left unused.

Yesterday, after long day at work I finally came home, ate dinner and unpacked mysterious package received that day. It was earlier ordered AirPort for iMac mini. The wrapping was tight, everything looked ok.
I took my two special iPutty knives and opened iMac. When looking inside I was wandering which screws should I remove to get to the motherboard. Of course I could google for some videocasts and tutorials on this topic, but my only computer at home was already dismembered. And besides who needs to RTFM? Last but not least I found three critical screws and after few seconds the motherboard was mine. On these iMac mini photos you can see the finesse the motherboard was built with. Next, I removed the daughterboard with Bluetooth module, and installed AirPort module on it. Notice, now I got two daughterboards. One came with AirPort and the other was installed because of Bluetooth module. Someone wants to buy the spare daughterboard? Let me know. The most interesting and funny thing is Apple uses sticky tape inside their computers widely. The AirPort chipset is taped to daughterboard, almost all cables are taped . I think it's ok, but somehow it doesn't fit to the trendy and sophisticated Apple design. Ok, back to AirPort. After reassembling iMac, I had no blood on hands but two screws left. And you know what? iMac works without them. So, besides I've upgraded this piece of hardware I've also improved the mobility by lowering overall weight.
After those interesting moments with screwdriver I came with this conclusion: You can always remove some parts and a device will still be working fine. And you can use these parts to assemble some new revolutionary hardware. Any questions?

Big Money research

A lot of people on SEO and e-marketing forums are not sure if AdSense "fraudsters" are really so successful as all the buzz implies. So we've come with a little experiment. We set up two AdSense accounts and on one of them we act ehical and obey all the rules. On the second one we're not so saint. The experiment is about who will make more money in short and long term. We are of course conscious about breaking some rules (and laws!) on that second account and we know it can be blocked any minute. Of course we won't hire any Africans to click on our ads for 5c per hour. We will use some more advanced fraud techniques. After publishing results of this experiment we will donate all the money from the second not-so-saint account to some open-source project. Any suggestions are welcomed.

We're moving to dedicated hosting soon.

Unfortunately, due to slow blogger.com hosting and other issues and limitations, we're moving to dedicated hosting soon. Expect new blog engine, new design and of course more juicy news. Our next goal is to give you aproximately 5 news daily and one bigger article on a weekly basis.

We expect the transition will be smooth, despite the fact we probably will use some new and unexplored tools by us. As some of you readers may guess we're moving to RubyOnRails.

Stay tooned!

Update: If you know some affordable hosting companies selling RubyOnRails accounts, drop us a note.

Thursday Trix meetings

Last thurstay we've initiated our weekly company meetings. We discuss all aspects of our work, present our brilliant ideas and drink famous polish beer. Everyone is invited. Feel free to come. Cafe Rene, Pszczyna, Poland.

Accidental ethernet cable disconnection testing

Today we are testing update for our major software product. As always on saturdays, I'm at home skyping with boss. He's in Germany. We want to see how our application behaves when someone accidentally pulls the ethernet plug out. So, here's what I hear in my uber-cool bt headset:

- Jacek, I'm going to pull the plug now.
...
- Yo, boss, are you there??? Helllloooooo!!!
...
...

After five clicks he's skyping me again:
- Jacek, I've lost Internet connection. Everything worked perfectly.

Can you imagine? By pulling the plug out of the ethernet card you actually lose Internet connection. That's great. Now, the question. How can you make money using this "technique"?

Camperaphones. Was it worth it?

A little shitty new nokia smartphone...
Originally uploaded by Rstless.

Notice the violet glare in the center of this flick. My old Agfa e-photo behaves much better and was bought few years ago.

More to come!

I will drop here some random sentences about healthcare. I hope Google AdSense will notice and show some relevant ads. We all know they pay more for "Pennis enlargement pills" ads.

Liberator’s line of pillows will heighten your lovemaking -- literally. The new angles and elevations equal new possibilities for pleasure.

Ehh, I think it's pointless anyway...