snarez.com

A website about many wonderous things…

Hamachi vs. The Internet

It took me nearly 2 days, but I finally tracked down the cause of some internet connectivity problems I was having lately. The symptoms were rather strange: a few sites were completely unavailable to me, in a very no-route-to-host-like way. Even stranger was the fact that all three of my colleagues, on same LAN and gateway, could access those sites without any problems. All the “is-site-down” sites claimed everything was fine. A few of my buddies whom I IM-ed could acces them. But one friend couldn’t. Furthermore, one more site was unavailable to him, and turned out to be unavailable to me too.

You know WHOIS to blame

Well, it’s in the title of this post. But it wasn’t so obvious to me at the time. First of all, I had installed Hamachi some time ago, and haven’t had any connectivity issues since then. Furthermore, the errors seemed too random: 1 out of 12 servers my webhost uses, and one completely unrelated webpage. At first, it seemed that there was some problem with a large hosting provider in Germany, but after a lot of pinging, tracerouting, WHOISing and reverse WHOISing I finally saw it: all the affected websites were in 5.0.0.0/8 block.

Further digging discovered this:

1) Hamachi uses the 5.0.0.0./8 block for all client “virtual” addresses. Its virtual adapter sets the metric so it gets all the traffic going to that host

2) 5.0.0.0. was unused on internet

3) The 5.0.0.0/8 address block was allocated by IANA to RIPE NCC in November 2010

4) On April 23, 2012, RIPE started to give out the addresses from this prefix.

So, while having an active Hamachi adapter, it is impossible to connect to some hosts. Mind you, it’s not necessary to be CONNECTED to hamachi network, it is enough  for the adapter to be present and active. So for now, it means sudo ipconfig ham0 down for me whenever I want to access those sites. And sudo ipconfig ham0 up when I need hamachi. And hoping those 2 activities don’t overlap…

We’re running out of internets…

That’s right folks. We were warned a long time ago, and we were warning others for a long time. But it’s becoming more and more real. True, Hamachi WAS violating internet specifications, but for me, this was a taste of what happens when address spaces begin to collide. It will be interesting to see which prevails in the future: another NAT layer on top of the whole internet, or an implementation of IPv6..

P.S.
Hamachi offers a “solution”: put it into IPv6-only mode

but pay attention to this part:
Note:  Some Software programs may not support IPv6 addressing and may require configuring a Gateway Network.

P.P.S
There might be other 0-config VPNs which behave like that; apparently EasyVPN is one.

RSpecing the mass assignment whitelist

There was quite a stir during the past few days in the rails community regarding the recent github “hacking” issue. The whole thing resulted in a lot of finger-pointing and name-calling, and a few proposed solutions. While I think that the issue was beneficial in a way, as it led to some people showing their true (im)maturity, I’m not going to talk about that. There was enough ranting already, and almost everything has already been said – relevant or not. Instead, I’d like to take a look at the technical side of the issue.

The status quo is not good. Sure, you can “just use the attr_accessible, noob”. Or slice the params, as DHH suggests. But in the end, people are bound to forget something (as illustrated by github “hack”), or somebody is going to change something, and it all goes to hell. Even worse, a vulnerability could be introduced and remain undetected for a long time (again, as illustrated by github “hack”). So, let’s look at it from another perspective.

Now, I’m nowhere near a ruby nor rails expert, so I may be completely wrong, in concept or in implementation – and please DO point out any mistakes I make. But one thing I know is that all good rubysts/railists almost fanaticaly use testing and TDD/BDD. It should be possible to write such tests which would notify us if a mass assignment vulnerability was introduced. Such a test would basically consist of 2 parts:

  1. Verify that the specified attributes are all mass assignable
  2. Verify that no other attribute is mass assignable

The first part is easily achieved using the shoulda-matchers and allow_mass_assignment_of matcher.

Update: I think better when being away from computer for a while

The second part seems to be tricky when virtual attributes are involved; at least I haven’t been able to come up with an elegant solution how to obtain a list of relevant virtual attributes. My current method is to get all the model instance methods and select only those which and in =, as they are obviously writer methods. Also, eliminate all methods with name beginning with underscore. There are still a few “uninteresting” methods left, so they are removed “by hand” – I made an array with those method names, which I compiled through some inspect calls and console use. In the end, column names are added. After all the possible candidates for mass assignmnet are collected, it is easy to write the second part of the test.

Obviously, there are some problems with the described procedure, worst of them being “manual” weeding out of methods. That step most likely breaks depending on rails version (which can be partially mitigated by writing a method which returns that array based on rails version) and possibly because of some libraries modifying ActiveRecord::Base. Also, it would ignore virtual attributes whose name begins with underscore, but I suppose it is not much of an issue.

In order to verify the second part, we should obtain a list of all attributes which could be mass assignable, including the virtual ones. Virtual writers are defined in model, and can be obtained as model’s class instance methods minus superclass class instance methods. This array still contains a few extra methods, but they can be filtered out by selecting only those which end with = and don’t begin with an underscore. Adding in the list of table columns, and the complete list of possibly mass-assignable attributes is complete. Finally, we need to check all the candidate attributes which were not specified in the whitelist; if any of them is mass assignable, the test should fail.

All in all, it is a quick-and-dirty (more dirty than quick) solution which works for me and my current setup, and hopefully somebody can come up with something better. To better illustrate my point, i wrote a whitelist matcher, so check it out and come back with comments :). And forgive me for my weak ruby-fu, I’m still learning.

Rails 3.2, rvm, Ruby Enterprise Edition, nginx and PostgreSQL on Debian

Yet another step-by-step guide

There are a lot of tutorials/installation logs online about installing any of the mentioned software and combinations of them, but I sincerely couldn’t find a guide for this exact combination. Not a big deal, it is always possible to open a few tutorials and follow them in parallel. However, sometimes the tutorials disagree on certain points. Sometimes things simply refuse to work. And my bookmarks aren’t becoming any less messy as time goes by. So, I’m going to document a process which works for me, and hopefully it will work for somebody else.

Disclamer

This guide assumes you’ll be the only person using the machine, or that you can trust other persons 100%, even with your most sensitive data (probably not a good assumption). I will try to note where some of the steps might create the risk if this is not the case, but please DO your homework in such environments. Also, it might be a good idea to have/hire/consult an expert if you’re planning on having a shared and untrusted environment. This setup is intended with solo developers in mind, and for development environment only. Do NOT use this exact configuration for production.

Debian

I use Debian for a lot of servers, and am quite happy with it. In this guide I’m using 6.0.3-i386-netinst, but any 6.0.x should do. For development environment I use virtual machines + ssh, and would recommend you to do the same. Hint: taking snapshots before installing/upgrading (a group of) something(s) is the key to less frustration (and less blog posts like this one).

If you don’t have your preferred partitioning scheme, guided partitioning on whole disk with everything in one partition works good enough for development. For packages, I chose SSH server and standard system utilities.

Additional tools and configuration

Depending on Debian version and package selection, sudo is most likely not installed. Therefore:

1
2
3
4
5

$ su -
Password:
# apt-get install sudo
# usermod -G sudo snarez
# exit
$ su -
Password:
# apt-get install sudo
# usermod -G sudo snarez
# exit

You may need to relog after this.

Now, time to install the editor of choice for config files etc. You may be happy with default available editors (vi, nano..), but I use vim for everything including development. If you don’t have a favorite CLI editor, consider familiarizing yourself with one – vi is a good choice as it is present on almost any machine.
Also, now is a good time to install git, curl, and SSH if you haven’t selected it from packages.

1

$ sudo apt-get install vim git curl ssh
$ sudo apt-get install vim git curl ssh

RVM and ruby

As can be seen on official rvm website, the best way to install rvm is with curl. After install, you need to reload the .profile using source or by relogging:

1
2

$ bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
$ source ~/.profile
$ bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
$ source ~/.profile

Now it's time to install your favorite ruby version, in my case it is ruby enterprise edition. It requires some dev headers and a "non-broken c++ compiler", so install them first.

1
2

$ sudo apt-get install build-essential zlib1g-dev libssl-dev libreadline5-dev
$ rvm install ree
$ sudo apt-get install build-essential zlib1g-dev libssl-dev libreadline5-dev
$ rvm install ree

One of the things that I really like about ree (and passenger) is that they do sanity checks before installing, and print nicely colored instructions in case something is missing. Kudos to phusion for making a really user-friendly CLI installer.

Passenger and nginx

To install passenger with our ree we'll use the gem installation, as mentioned in official install instructions. First, it is necessary to install curl dev headers with SSL support, openssl or gnutls. I personally use openssl.
Also, make sure you're using the desired ruby version to install passenger into.

1
2
3
4

$ sudo apt-get install libcurl4-openssl-dev
$ rvm use ree
$ gem install passenger
$ rvmsudo passenger-install-nginx-module
$ sudo apt-get install libcurl4-openssl-dev
$ rvm use ree
$ gem install passenger
$ rvmsudo passenger-install-nginx-module

Default installation and options (should) work just fine - I never had any problems.

An init script for nginx is really useful, you may find one here. There are many similar scripts available on the internet, but pay attention to PATH and DAEMON variables.
Download to /etc/init.d, allow execution and make sure it starts/stops with your machine:

1
2
3

$ sudo wget https://gist.github.com/gists/2000616/download -P /etc/init.d
$ sudo chmod 755 /etc/init.d/nginx
$ sudo update-rc.d -f nginx defaults
$ sudo wget https://gist.github.com/gists/2000616/download -P /etc/init.d
$ sudo chmod 755 /etc/init.d/nginx
$ sudo update-rc.d -f nginx defaults

PostgreSQL

Debian being debian, PostgreSQL 8.4 is still the latest available version to install out-of-the-box. In order to install the 9.1, you have to add backports by adding the following to /etc/apt/sources.list

deb http://backports.debian.org/debian-backports squeeze-backports main

Update apt and install postgreSQL from backports. Also, in order for pg gem to be able to connect to database, we need libpq-dev headers:

1
2

$ sudo apt-get update
$ sudo apt-get -t squeeze-backports install postgresql-9.1 libpq-dev
$ sudo apt-get update
$ sudo apt-get -t squeeze-backports install postgresql-9.1 libpq-dev

Now, configure the postgresql to allow local access, so rails can use it.

WARNING: potetntial security risk if untrusted persons/apps are using the same computer.

Edit the /etc/postgresql/9.1/main/pg_hba.conf, near the end of file (line 90)

local   all             all                                     peer

and change peer to trust

local   all             all                                     trust

and restart postgresql server

1

$ sudo /etc/init.d/postgresql restart
$ sudo /etc/init.d/postgresql restart

Project-specific rails

Create a directory structure. I use ~/rails/<rails_version>/<project_name>. Then create a gemset for use by desired ruby version, instruct rvm to use it in that directory, and to trust the .rvmrc:

1
2
3
4
5
6
7
8
9

$ mkir -p rails/3.2/testapp
$ cd rails/3.2/testapp
$ rvm use ree
$ rvm gemset create testapp
$ echo "rvm use ree@testapp" >> .rvmrc
$ rvm rvmrc trust
$ rvm source .rvmrc
$ rvm current
ree-1.8.7-2011.12@testapp
$ mkir -p rails/3.2/testapp
$ cd rails/3.2/testapp
$ rvm use ree
$ rvm gemset create testapp
$ echo "rvm use ree@testapp" >> .rvmrc
$ rvm rvmrc trust
$ rvm source .rvmrc
$ rvm current
ree-1.8.7-2011.12@testapp

Time to install rails (finally)

1

$ gem install rails -v=3.2
$ gem install rails -v=3.2

I prefer using project-specific rails, but you may install it in global gemset if you're sure you'll be using the same version across all your apps. Project specific gemsets are almost a must have.

Creating and configuring the app

Create a new project in curent folder, using postgresql as database. We'll skip the bundler installation, as some modifications to Gemfile are required anyway.

1

$ rails new . --database=postgresql --skip-bundle
$ rails new . --database=postgresql --skip-bundle

It's time to create the database and user for our app. In order to do it, we need to run the psql command line as user postgres:

1
2
3
4
5
6
7
8
9

$ sudo -u postgres psql
=# create database testapp_db;
CREATE DATABASE
=# create user testapp_user with encrypted password 'testapp_pass';
CREATE ROLE
=# grant all privileges on database testapp_db to testapp_user;
GRANT
=# \q
$
$ sudo -u postgres psql
=# create database testapp_db;
CREATE DATABASE
=# create user testapp_user with encrypted password 'testapp_pass';
CREATE ROLE
=# grant all privileges on database testapp_db to testapp_user;
GRANT
=# \q
$

For development environments, I always use <appname>_db for database name and <appname>_user, <appname>_pass for username and password. Do I need to tell you to not to use this scheme in production?

Next step is to configure our rails app to use the same credentials. Open the config/database.yml and change the development section:

development:
  adapter: postgresql
  encoding: unicode
  database: testapp_db
  pool: 5
  username: testapp_user
  password: testapp_pass

Configure nginx to know about the app by editing /opt/nginx/conf/nginx.conf and adding the following at the end, before the last }

server {
  listen 8000;
  passenger_enabled on; 
  passenger_spawn_method smart;
  rails_env development;
  root /home/snarez/rails/3.2/testapp/public;
}

And finally, start nginx to make the app available: 

1

$ sudo /etc/init.d/nginx start
$ sudo /etc/init.d/nginx start

Of course, if it was already started, or when you change the configuration (e.g. you create another app), use reload instead of start.

In order to allow correct loading of gems with rvm, you will need a initializer script

1

$ wget https://gist.github.com/gists/2000626/download -P config/
$ wget https://gist.github.com/gists/2000626/download -P config/

Add rake and therubyracer gems to your Gemfile (why is this not included by default is a mystery to me...):

gem 'rake', '0.9.2.2'
gem 'therubyracer'

Time to install all the gems

1

$ bundle install
$ bundle install

And finally, your app should be ready to use. Navigate to http://<server_ip>:<port> and click the "About your application’s environment" link and it should work.

Just to make sure, let's do a final test

1
2

$ rails g scaffold test foo:integer bar:string baz:text
$ rake db:migrate
$ rails g scaffold test foo:integer bar:string baz:text
$ rake db:migrate

Navigate to http://<server_ip>:<port>/tests
That should be it :)

For testing purposes, you will need to grant privileges to create databases to user. Again, this is NOT recommended in production environment:

1
2
3
4
5
6

$ sudo su postgres
$ psql
=# alter user testapp_user createdb;
ALTER ROLE
=# \q
$ exit
$ sudo su postgres
$ psql
=# alter user testapp_user createdb;
ALTER ROLE
=# \q
$ exit

Hello, world

Such an unimaginative title, yet what’s better to start a blog, being  a programmer?

A few years ago (5 according to whois), I decided to write about something. Or anything.I bought the domain, but got distracted – great ideas for custom made CMS, great designs and whatnot – but never found the time to actually do it. Kinda sad, taking into account all the countless hours I wasted during that period. I’m not really THAT busy. Nobody is.

So today I decided I’m going to stop dreaming about reinventing the wheel – something I do too often – and use a good enough existing solution. And if it turns out not to be really good enough, then I’ll just create something better. One day.

In the mean time, I’ll start with writing about things I should have written about some time ago, and see where it goes: personal reminders, rants, opinions and cool things I learned (mostly) about technology. So stay tuned future readers!