meso·pixel    

Late October

Stops-r-us

One of the annoying things about my bike ride to work is the frequency of the stops along the way, which can often make a six mile ride take upwards of 30 minutes one way. My suspicion was that a significant amount of time was spent at the Moffet & Central intersection right where the Mountain View Caltrain station was, but since I had the Strava data from a couple posts ago, I decided to take a closer at where the time was actually going.

To get an approximation of the stopping intersections along a ride, you can take advantage of the fact that Strava slows down the recording of waypoints as the device stops moving (or in other words, the time between subsequent waypoints increases while stopped). And at those stops, you can find the time spent there by taking the time difference between the next moving waypoint and the last moving waypoint at some threshold distance. From this, the data suggested that I had to stop an average of 23.4 times for a total of 7 minutes 10 seconds per commute, which is almost 25% of my total commute time!

To break that down even more, you can take the approximate intersection waypoints and cross reference them with the closest actual streets (using the Geonames API) and find out exactly how much time you are cumulatively spending at each intersection. Here are the top 15 intersections along my commute to work for example:

IntersectionTime spent waiting
Castro St & W El Camino Real 4 hours 27 minutes
California St & Castro St 4 hours 24 minutes
Central Expy & Moffett Blvd 3 hours 41 minutes
Charleston Rd & Huff Ave 3 hours 31 minutes
N Shoreline Blvd & Pear Ave 3 hours 12 minutes
Cuesta Dr & Lassen St 2 hours 51 minutes
Moffett Blvd & W Middlefield Rd 2 hours 34 minutes
Cuesta Dr & Miramonte Ave 1 hours 38 minutes
Cuesta Dr & S El Monte Ave 1 hours 31 minutes
Moffett Blvd & W Valley Fwy 1 hours 4 minutes
Castro St & Villa St 1 hour
Castro St & Miramonte Ave 58 minutes
Cuesta Dr & Tyndall St 51 minutes
Central Ave & Moffett Blvd 51 minutes
Castro St & W Evelyn Ave 47 minutes

Surprisingly, the longest time waiting isn't actually at Moffet & Central Expressway but rather at the other end of downtown Mountain View at the intersection of Castro & El Camino Real. And it's true, that intersection also has a long wait since El Camino is such a high-traffic thoroughfare for the peninsula.

Castro and El Camino Real, the intersection where I spent most of my time on my commute
Castro & El Camino Real, the longest intersection wait time of my commute.

As a side note, if I were to sum up all the time spent at each intersection, it would be approximately 42 hours 24 minutes – that's almost two whole days spent just waiting for the cars to pass!

 

Late June

Socket dreams

uWSGI is one of those things in the Python world that Just Works™, I use it mainly in Emperor mode to monitor and manage the spawning of app processes but it turns out it has a full Websocket implementation, which really piqued my interest. Now that the world has largely moved onto modern browsers (with Websocket support), I thought it would be fun to test out the uWSGI implementation by making a tiny interactive visualization.

What you see above is a shared space where each heart beat line represents a visitor to this page. Clicking anywhere on the space causes your user's line to pulse, which is reflected to all other user's views (you can try it out by having two tabs open). It's a trivial visualization, but just complex enough to test it out from top to bottom, with some observations noted below:

  • Handling each socket Python isn't really built for parallelism; spawing off a new process for each connection is prohibitively memory intensive and spawning threads generally provides no benefit due to the GIL. Which really just leaves fibers/greenlets in the form of the Gevent library. Luckily, uWSGI has built in support for Gevent and they can be easily configured to work with Websockets.
  • Number of connections The visualization artificially caps the number of simultaneous connections to five, but theoretically there can be an arbitrary number of websocket connections to the server (and they are not multiplexed on the browser side like with HTTP/2 connections). And uWSGI happily sends them through, so the app really has to handle this itself. In this case, we just put the extra connections into a queued, sleeping state, waking only when it is promoted to a full connection or to send the requisite Websocket ping/pong frame to keep the connection alive.
  • Output buffering By default Nginx will buffer data before sending it to the client, and you will want to disable this by setting "proxy_buffering" to false.
  • Client messaging Like a chat room, messages from a client are received by the server and forwarded out to all the other clients. To save a bit of processing, each of the server connections also debounces messages before sending out to their respective clients. This adds latency, but it allows for reducing traffic by batching messages (for later experiments).
  • Visualization On the client side, the animation frames drive the rotation for each line, and the pulses are just animated amplitudes on the circle (radius + amplitude * periodic f(t)). I kinda like how it actually turned out!

Overall, Websockets mostly just work out of the box now, and while it's not appropriate for everything (ie. real-time, low latency is difficult on TCP), it's pretty cool to not have to fake all this with long-poll anymore.

Waterfall

Ah, the memories of MIDI

This is almost artistic, cruise ships from an ariel view

Auto-complete Bash history using arrow keys (probably the best Bash tip I know)

Pong

Remember and Big Shiny Tunes and Much Dance? Good times.

Worst office fear: Rolling over your own toes with your computer chair.

Don't say Disney won't go to great lengths to optimize their animatronics...

Like horse racing but for nerds and biologists, Genetic Cars.

Monterey 2013 (4)
Monterey 2013 (4)