meso·pixel    

Early May

Bumpin' Time Machine Blues

It’s a bit surreal to think that just one year ago, the BumpTop team was still in Toronto studying for the on-sites and getting ready for another release after a successful Mac launch.

bumptop team in san francisco 2010
BumpTop team in San Francisco (2010), courtesy of Justin.

Also, Google I/O starts tomorrow at the Moscone Center in San Francisco, and Pat will be giving an interesting talk on Android Memory Management later this week. Many of the talks will be streamed live as well, so be sure to check out the site for more info.

 

Late April

Server-side up, with a side of AWS

In the six years since I wrote the first site, the web and the technologies that power it has changed significantly – and for me, this refresh of the site is also a great opportunity to learn more about new server-side technologies and tools, such as Amazon’s cloud-based web services. With a number of constraints in mind, I ended up deciding to host the site on AWS because of the ease of administration, deployment, scaling, and reliability, even if this site doesn't necessarily stress all those factors. This page you are reading now is proudly served from a micro EC2 instance, with most of the static content including images and the old site archive are served directly from S3.

  • EC2 (Elastic Cloud Compute) is a service that allows you to run a virtual machine on Amazon’s distributed network. The pricing is based on the specifications of the machine that your instance runs on, as well as the running time, which is measured in hours. This sounds weird at first, because most sites run 24/7, but this flexibility allows you to scale up the number of servers to handle variable load as your site grows (or shrinks).
  • S3 (Simple Storage Service) is a distributed host for serving static content like images, styleshees, scripts and even web pages. This service helps reduce demand on your server by distributing requests for common files to multiple machines, all of which can be handled in parallel. The pricing for S3 is based on bandwidth, and the number of requests made.

By taking advantage of Amazon’s free tier, access to these services (and many others) are free for a full year. Setup was relatively painless using Amazon’s AWS Management Console along with some helpful advice from Rhys and Justin.

Each instance starts bare (depending on the machine image that it is based on), which allows you complete freedom to install whatever services you need. In my case, the primary packages we needed were nginx, PHP, the Kohana MVC framework, MySQL, and Memcached. We’ve had experience using nginx at BumpTop, and it is a reliable and efficient (in both cpu and memory) alternative to Apache – which is important on our micro EC2 instance. Kohana is, in my limited experience, a great MVC framework that provides a nice balance between the convenience and utility of a framework, and the freedom of being able to work around certain framework features. I have not yet found myself fighting the framework to do what I need, as I did while experimenting with web.py, django (and less so in Google App Engine), and Kohana is noticeably lighter than other heavy PHP frameworks like Zend, Symfony and Cake. The only downsides I’ve encountered so far are the limited number of tutorials and API instability between minor versions (eg. 3.0 vs 3.1) due to the rapid pace of development. In general, the machine setup was mainly a lesson in learning the linux tools (apt-get is your friend), getting around the linux filesystem, configuring and reconfiguring services, editing config files, and reacquainting myself with vim. Maybe I’ll make a quick post later describing my vim configuration, as I find it super handy when working directly on the server.

Some setup issues I ran into that were frustrating (with potential workarounds):

  • My .bash_profile only gets applied once – That’s because .bash_profile only gets executed for login shells, so if you change user using sudo or su, then only .bashrc gets executed. You can source one from the other for a quick fix
  • Old packages on the AMI – Run apt-get update and apt-get upgrade first to make sure you have the latest tools/packages
  • EC2 instance IP keeps changing – Attach an Elastic IP to your instance (it’s free)
  • Binding a domain name to an EC2 instance – Add/change the A record in the DNS entries where your domain was registered
  • My site is not loading – Ensure that the server_name regex is correct, and that you are not falling back to the default server config (rm /etc/nginx/sites-enabled/default)
  • I’m not seeing my newly-installed module in phpinfo() – You need to restart php-fastcgi for the new module to be picked up
  • Problems with Kohana – Read the docs again, remember to add a rewrite rule in nginx to map uris through the Kohana index file (rewrite ^(.*)$ /index.php?kohana_uri=$1 last;)
  • Certain pages show permissions denied – Add your user to www-data (using adduser), and make sure the www-data group owns /var/www and has access to it (chmod -R 770)
  • Can’t find AWS-SDK classes – Add the AWS-SDK directory to your path, or require it at the top of the PHP file (require_once("$root/aws-sdk/sdk.class.php");)
  • Uploads greater than N megabytes fail – You need to edit php.ini (found in /etc/php5/cgi/php.ini) and increase the values post_max_size, upload_max_filesize, max_file_uploads (if it is a result of too many files at once), as well as client_max_body_size in your nginx server config file (eg. /etc/nginx/sites-available/yoursite.com)
  • I’m tired of typing my IP – Add an entry in your /etc/hosts to point yoursite.com to the elastic IP
  • Git fails with Permission Denied – You need to add the git user to your sshd_config file, and use the correct public key when initializing gitosis (and make sure that key has the right permissions)

And a quick cheat-sheet:

configuration: EC2 micro-instance based on ami-ad7e2ee8 image (US-West region, Ubuntu 10.10 Maverick)
apt-get install: zip unzip mysql-server mysql-client php5-cli php5-cgi php5-mysql php5-gd php5-curl php5-memcache nginx memcached git git-core speedometer iftop
important directories:
  • /etc/init.d – Where the start/stop scripts are located (eg. /etc/init.d/nginx restart or /etc/init.d/php-fastcgi stop)
  • /etc/nginx – The nginx configuration directory, and where you can put your server configurations in (eg. /etc/nginx/sites-available and linked from /etc/nginx/sites-enabled)
  • /var/www – A common default directory for you to put your website contents (eg. /var/www/yourside.com with subdirectories log/ and public/ for example)

And that's it. Easy right? :)

 

“Risky to change. Riskier not to change.”

What a great quote. It’s not always clear the best time for change, but what has been clear is that the recent proliferation of Random Posts and Picture Dumps on my part is not quite the direction I had wanted the site to go. I think there needs to be a higher standard both for what I write and share – which is what I’m going to try and commit to along with this refresh. Most of the changes you see were done after considering both what the old site did and did not do right. One change on the front page is that you can now find pictures, and tidbits in the Waterfall off to the right. This way, you won’t have to scroll through twenty pages of pictures in between posts, and browsing images is now a bit easier from within the gallery.

A little history tidbit; the original incarnation of mesopixel.com was borne out of a strong curiosity in learning a bit more about web development and infrastructure as well as the eventual necessity of having to move of the University of Toronto servers. For six years, the site ran on Dreamhost’s (rather respectable) shared-hosting plan[1], and I somehow ended up making 146 posts and uploading over 1050 pictures and comics along the way! It’s still probably one of the longer commitments I’ve made outside of school, which is pretty neat.

A few things I learned from the first site:

  • It’s really difficult from a motivation standpoint to maintain a site and or gallery if your tools aren’t dirt easy to use. Every time you make the choice between having to tweak with CSS to make something look the way you want it to, and doing something else more fun, it kills your motivation just a tiny bit and over time, your clever brain begins to dread having to update the site.
  • Spammers really suck, but I ended up removing the ability to comment completely for other reasons. Having meaningful conversation on a personal blog is really difficult due to synchronization (expectation of responses), privacy (what you can/can not write in a comment), and context (does the conversation even make sense for anyone else?). I’m sure there's a better solution, but am still thinking about this one.
  • Search engines don’t have very good image recognition, and having proper accessibility attributes and descriptions on images/content/links allow for people to find them easier. Apparently, people really like to search for pictures of the 1000 Islands (in Ontario), as well as people getting slapped in the face (some image I linked to ages ago).
  • People don’t like clicking off the front page. On average, people spent about a minute and a half on the old site, but the probability they clicked on any other page was really, really tiny (like 1.6% on certain pages). Perhaps I contributed to this by having all the good stuff on the front page, or perhaps I was convinced that the only way for content to be seen was to put it on the front page. I’m not sure which predates which.
  • Having a dark theme results in higher contrast when looking at ordinary photos, and really brings out the vibrance of the colours in those photos! This is also why you see a lot of modern displays displays with dark bezels.
  • Sometimes you just don’t feel like writing and laziness will evolve into complacency. This is really difficult to break out of. Best not to post while in this state.
  • Information and opinions get stale. This is a widely acknowledged problem that I might not try and tackle. There is also a bit of charm and nostalgia in being able to go back in time and reflect on how you felt about things before, and how you feel about them now.
  • Code is all easy to write, but difficult to read. Simplifying the design, and organizing the code and services better goes a long way to maintaining a site.

I’m hoping that this kind of experience will help guide what I do with this new site over the next few years.

My next post is going to be more technical in nature, and I’ll be describing some of the technologies that this new site is built on. For a bit of nostalgia, you can always find an archive of the old site in the mesopixel archive.

Apologies in advance if you hit an error page, as the site is still very much under development.

[1] Also known as the "Crazy Insane Domain" hosting plan

Waterfall

Ah, the memories of MIDI

This is almost artistic, cruise ships from an ariel view

Auto-complete Bash history using arrow keys (probably the best Bash tip I know)

Pong

Remember and Big Shiny Tunes and Much Dance? Good times.

Worst office fear: Rolling over your own toes with your computer chair.

Don't say Disney won't go to great lengths to optimize their animatronics...

Like horse racing but for nerds and biologists, Genetic Cars.

Monterey 2013 (4)
Monterey 2013 (4)