Mindstab.net

During the course of updating my home computer I rebooted it because of a kernel update. Later that week at work I went to connect to my home computer and discovered that it’s dynamic IP had changed and it’s DNS name was invalid.

So following common advice to “fix a problem two ways to prevent it in the future” I fixed the DNS, but I also wanted an automated way to track my computers when and if their IPs changed.

So the first thing I needed was shared place to store the IP information. Thinking about it I realized that Dropbox would work well for that. So all I needed was a simple script.

So the solution was to put a script that determined the IP of the computer in Dropbox and have cron on all the computers run it. Each user can call cron with

$ crontab -e

And I created a crontab directory that I could add more scripts to later if need be with and run them hourly with the following entry

0 * * * * cd /home/dan && run-parts Dropbox/cron

The script itself was a file called getip and it used whatsmyip.com’s automation detection script.

getip

#!/bin/sh 
 
wget -O /tmp/`hostname`.ip http://automation.whatismyip.com/n09230945.asp
tmp_file=/tmp/`hostname`.ip
dst_file=Dropbox/var/log/`hostname`.ip
if ! diff -q ${tmp_file} ${dst_file} > /dev/null  ; then
  `cp ${tmp_file}  ${dst_file}`
fi

Then I just created Dropbox/var/log and installed the crontab on all my computers, and volia, homebrew IP tracking for all my compters accessible to me from anywhere.

A coworker just got a new box and tried a fresh Ubuntu 11.10 install (the rest of us have stayed safely with 10.04). He found this link and after looking through it I definitely wanted to save it for later if I need it:

Things to Tweak After Installing Ubuntu 11.10

A whole list of commands to get rid of global menus, modify all kinda of Unity WM behavior and Gnome 3 behavior (like get screensavers back) and even how to install Gnome Shell on Ubuntu and tweak that. Looks like a great collection for anyone underwhelmed by Ubuntu 11.10′s default offerings.

About 3 weeks ago Lennart Poettering posted A Plumber’s Wish List for Linux to his blog, which is one of the many Linux related blogs I read. Included on that list was a request to “expose CAP_LAST_CAP somehow in the running kernel at runtime”.

Now I’d never contributed to the Linux kernel before, but in college we had hacked on the Minix kernel to extend it and in University we had written our own kernel and implemented virtual memory, task switching, message passing and preemptive processes; so I was at least familiar and comfortable with kernel level development. Looking at this task, I immediately thought, “hey, that is totally within my capabilities and I can probably squeeze it into my schedule”.

So I did. I checked out the latest copy of the torvalds Linux repository and implemented the feature. The actual implementation didn’t take too long. Mostly looking for the right place. What did take a day or two was getting up to speed on Linux coding practices and patch submission protocol and procedure. They have a good process in place and it’s best not to step on toes so I did my research as best I could. When I was ready I sent in my patch.

Andrew Morton picked it up, asked for a few enhancements, and those submitted, added it to his -mm tree for testing. Then a few weeks later Linux 3.1 launched and the new merge window opened and today Andrew submitted his diff from the -mm branch including my patch to Linus and it was merged to his tree!

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=73efc0394e148d0e15583e13712637831f926720

I am now a Linux kernel contributor. I have a small piece of code in the Linux kernel that will ship with 3.2. It’s not huge, but I did it, and that gives me a real sense of pleasure and accomplishment. Thanks to all who helped!

So after a few failed attempts at installing Ubuntu 11.04 on my parents computer I thought I’d give Fedora 15 a try. It worked.

The install was pretty painless, and no weird unresponsive 10 to 20 minute pauses, so that was nice. Rebooted, logged into Gnome Shell.

It’s ok. It’s not bad. I like it better than Unity, but both of them suffer from the developer “my hands are always on the keyboard, why can’t I launch apps by search” syndrome. That’s cool for you, but sometimes if I have one hand on the mouse I just want to use the mouse to lauch apps and having to fling or click the top left of the screen, then move to the other side to click applications and the find my app in an unsorted list that over fills a screen is not optimal. At least Gnome Shell obviously offers the categories on the side to filter by. But at this point it is way way slower than the old Gnome menu was. Also, not offering an window list for me to easily toggle between apps? Either I again have to fling or click the top left and the select my window from a different display or Alt-Tab it. It’s like these new exciting GUIs are less friendly to traditional graphical interface tools like… a mouse. They are more friendly to keyboard combos.

Still, ranting aside, Gnome Shell was ok, and definatly better than Unity. And it and Fedora were more stable, why, this system has been runing for over an hour, and no random crashes. There are a few visual artifacts but I guess that’s just what I get for runing Linux on a new AMD Radeon 6500. Maybe in half a year there won’t be visual artifacts.

What really got me though was how much worse package management is on Fedora than it is on Ubuntu. It’s a colossal pain. Ubuntu / Debian have really spoiled me. Flash didn’t come with the system, so I followed online advice and downloaded the YUM and then it installed and nothing happened. But then flash was in the package manager, except it just sat there with a “Waiting in queue” message doing nothing for like 10 minuets with no way to fix. Awesome. And searching for apps? *Office also doesn’t come installed, and good luck trying to find it. I search Libre Office all kinds of ways. Among the truly large amount of not Libre Office results, the one that was closest that I usually got was “Libre Office development”… I eventually gave up and found a .doc file and tried to open it and Fedora then asked me if I wanted to install Libre Office Writer. Why yes I would. Then the installer went to work, it looked like it froze on the download, but with so little feed back it was hard to tell so I left it alone for a while and it sorted it self out. It’s just really not remotely helpful.

Finally, one of my mom’s favorite features on the computer is making a screen saver of all her family photos (they had several thousand photos and slides digitized recently) and it appears Gnome Shell has abolished the screen saver…. Really? Also, one of the other winning features of Gnome 2.0 for my Mom was I could place absurdly large icons of her few apps (Web Browser, Office, Picture folder) on her desktop. Not any longer.

All this new “usability” seems to make things more of a nuisance for me and simply remove key features my mom relied on. Critical regressions. I’m pretty sad.

Family got a new computer and Mom wanted Ubuntu on it. A good sign. They’ve won her over. The battle should be done right?

• Time spent waiting while the installer is unresponsive and gives no indication it is working or if it is dead: 30 minutes
• First install booted up with a thin stripe of graphics about 1cm tall at the top of the screen and blank black for the rest. Completely unuseable
• Second install, I chrooted it from the liveCD, updated all the software and installed fglrx
• Second install booted, started using Unity (the new Ubuntu interface):
  • Only a few visual artifacts :/
  • Software discover-ability is about zero compared the the old well structured Gnome menu :(
  • Having to type the names of the apps I want or sort through a giant unstructured alphabetical list of all software installed is not the way I want to launch my programs and is slow and cumbersome
    • Also, after typing there is a noticable pause before the search list populates. Live AJAX web services like google search which are remote are/feel more responsive :(
  • New sidebar buttons are huge, greatly limiting the number of shortcuts I can pin to it
  • No smooth and wobbly windows anymore.
  • No applets
• After about 30-45 minutes use the screen blanked. I changed to a shell (alt-f1) and rebooted. It booted up and the monitor got no signal. Repeated with same result. It completely died. (remounting from the liveCD Xorg.0.log reports some problem with the fglrx driver not loading (but it worked from the liveCD and for the first while of the fresh install?). What happened to the good old indestructible bullet proof X Ubuntu touted a few years ago? Do modern graphics cards no longer support fall back graphics modes like VESA?)
• Mom is stuck with Windows 7 because it works

:(

Let’s say you did a presentation that was recorded and you’d like to post it to your website. Sadly, let’s now say there are some problems, like that your 5 minute presentation is part of a nearly 2 hour video only available in a flash player that doesn’t even have a time display so you couldn’t even point people to the video and say jump to 1 hour and 15 minutes to see me. It sucks. Technically your presentation is available online, but it’s not really accessible. So here is how you might rescue it!

It turns out there are two ways flash players server videos these days. The first and easiest is that a simple flash player loads in your browser, and uses your browser to make a GET request to the server to load a .flv file (FLash Video). This is relatively easy to intercept, there are lots of tools and plugins for Firefox that do this automatically for you. Even better, on Linux for example, these videos are usually stored in /tmp so your browser does the whole job and gives them to you. No work required.

The other more complicated but more secure option is that the flash player connects to a dedicated rtmp server that streams flash video. The flash plugin does the networking and there is no file to save, it’s a stream.

If you are lucky enough to have a player using the first option, you are done. Assuming you have the second option, then your fun has just begun.

First we need to try and figure out where the server that your flash video is.

My first approach was to use wireshark to sniff the traffic. Through this I discovered the basics, like the address the server and the port, 1935.

Next I installed rtmpdump. RTMP is the Real Time Messaging Protocol and rtmpdump is a program that can connect to an RTMP server, get a stream and save it to a file. Sadly the data I got from wireshark didn’t have all the parameters I needed to get the file. Or I couldn’t read it properly. So while I knew where the server was and could now connect to it, I still didn’t know how to ask for the video I wanted.

Thankfully rtmpdump comes with several other utilities. After reading its README I went the rtmpsuck route. I set local redirecting of all port 1935 requests to localhost with iptables and ran the rtmpsuck proxy server. In theory it was supposed to intercept all calls from the flash player to the rtmp server, decode and spit them out, and then forward them along. Even better, it would try to save the stream on the way back as it passed through it.

# iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner --uid-owner OWNER_UID  -j REDIRECT
$ ./rtmpsuck

Where OWNER_UID is the uid of the user running rtmpsuck. With this running I just reloaded the page with the player (twice, it’s a bit glitchy) and then tried to skip to where my part was so it would save the stream from there.

It was partially successful. It spit out on the console all the pertinent path parameters about the video on the server, but it kept chocking on bad packets of data and stopped recording. Also for some reason the video it did store was very large, space-consuming wise.

Armed with the right parameters though I was able to use rtmpdump to suck down the whole video from the server surprisingly quickly and in a reasonably sized format.

$ ./rtmpdump  -r rtmp://server.net/app_name/blah/event/date/date-1 -o video.flv

Now the video was liberated from its flash interface and in my possession, I just had to cut out my small part and then convert it to a more common format.

$ mencoder -ss 1:15:50  -endpos 0:05:57  -ovc copy -oac copy video.flv -o result.flv
$ ffmpeg -i result.flv result.avi

And volia. I now have just my part of the video and in a common format. I mean you hypothetically do! Yes…

Completely unrelatedly, you can expect to see my presentation on my project Cortex from the BCNet Broadband Innovation Challenge (where I got second place) online soon.

So I’m setting up a new website for a client (an artist) so the easiest what I’ve always done is use Gallery. I’ve used 2.* for years and now 3.0 is out, and has been for a few months. So I figured why not give it a try.

I’ve never knows a website to have intermittent bugs, but Gallery3 has a good couple of them. Sometimes the spacing around photo/album items is just way too big, and after a mouse over, they jump position. A good half the time trying to delete an item takes you to a blank white page with a single option, “delete” and then that takes you to a dead end page with clearly an AJAX reply. But only sometimes. How do you track down bugs that only happen half the time? The default theme, wind, seems to lean heavily on jquery and I think, but am not sure, this is where the instability is coming from, but having not boned up to the level of jquery master, I certainly can’t dive in. Also apparently some of these issues aren’t even unknown, but still haven’t been fixed so we could assume better minds than mine have looked.

So that’s a bit disappointing and a waste of a days work that I get to eat. Gallery3 is not stable and usable. Back to Gallery2.

(Also, I’m not even really a fan of hiding things like item names by default and only showing on mouse over, it’s bad for the kind of galleries I’m putting up, but there isn’t even an option about that, and again, not recoding a ton of jquery code.)

Frustrated.

So 2010′s been a year.

Nearer it’s start some friends and I competed for the second time in MIT’s BattleCode, this time getting second non-MIT spot, or 18th overall. These competitions have been good for us: they are fun, it’s a good group project, and we work on our group project skills like planning, coordinating, and so forth. We spend so much time on our own, or in school even, working solo it’s good to work these skills as they will be needed later. Also it’s fun to learn about and catch up on low level AI stuff, like swarming and flocking movement/coordination techniques etc.

I also entered a school project into BCNet’s Broadband Innovation Challenge and got awarded second place. My project was “Cortex” a P2P processing app that runs with no software install entirely in your web browser. It was comprised of a small Java Applet webserver used as a backbone for communication and then a JavaScript front end, with all the control logic of the P2P network also written in JavaScript. I pretty much wrote a P2P app in JavaScript just using Java only to get around the AJAX/Server of Origin security policy issue. It was an interesting and challenging project and I’m pleased with how it did in the competition.

Over the summer I was in China which was amazing.

Then in the fall while finishing off my degree in CS once and for all I also competed in the Google sponsored University of Waterloo AI Contest. This, while being a simpler solo competition, was notable for me as it was my third major project undertaken in Lisp. I thoroughly enjoyed the challenge and again learned lots more about Lisp and again improved my Lisp style. Lisp and the emacs environment just take longer to learn and wrap my head around. And since I don’t get to work in them constantly, between work and school, it takes time. I’m by no means a master, but after convincing a friend to take a stab at the same competition in Lisp for his first try with Lisp, I at least see how far I’ve come. I’m getting more used to thinking functionally, especially with respect to using Lisp mapping functions instead of loops to modify, filter, or build on data. I placed disappointingly poorly due to lack of time, but I’m satisfied with what I learned (and also proud by association that the winner was a Lisp entry!). It was a good experience. I look forward to being able to undertake some more Lisp projects in the new year.

I also boned up on my Python this fall for a small work project, a multi threaded web crawler for a client. Played successfully with Python’s threading, so that was fun.

And that brings us to now. I’m in Colombia for the holidays, and in my vacation spare time I’ve finally gotten around to looking at the codebase to my school project “Cortex”. As school projects are, it worked, and well, but the codebase was a bit of a mess due to strong time constraints. Now that I have some time I’m doing some massive cleanups and adding a few features I’d wanted to but didn’t have time to. Hopefully early in the new year it’ll be in shape that I can release it. That would be nice.

So 2010 was a great year. I got to write a lot of cool code in several different language. I got to travel more than I ever have before, and I read a lot more than 2009 (traveling facilitates a lot of reading :)). It’s been a good year.

For 2011 though, now that I’m done with school, I’d like to start by releasing more code, starting with Cortex; getting more paying work; and looking at maybe starting a startup. I’d like to spend more time working on both AI (if you hadn’t noticed, obviously a hobby of mine) and in Lisp, starting with getting back into my signed copy of Peter Norvig’s “Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp” (yes getting it signed was awesome and a ridiculously geeky moment) and moving on from there. I’d like to at least keep up with the reading. I have high hopes for it to be an interesting year.

So here’s to 2010, you’ve been great, lets see if I can’t build on that for a more amazing 2011.

So early this year I bought my first Android phone, an LG Eve, from Rogers Mobile. It turns out it came with Android 1.5 which was a year old at the time, the current version was 2.1 and 2.2 was released that spring. Rogers has been promising an update to 1.6 for the phone pretty much since I bought it and still no release. It was pushed back twice and has since gone silent. This could be the end and a sad story at that, with just over 2 more years left on my contract.

However, this is Android! And so heroically the CyanogenMod people have been following the Android code base and releaseing an free up to date version that works on some more common phones. The good folks of Open Etna have taken that work and customized it and made sure it works on the LG Eve specifically. So following the instructions on their and other sites, I have been able to upgrade my phone to Android 2.2, and it’s awesome. There is a lot more software available, voice commands, live wallpapers, and a JIT compiler so software should run faster, and more that I’m just discovering (ships with a good terminal app). And just generally newer default apps. So thank you very much to those communities of volunteers for doing vastly more than what my Phone company was incapable of.

And now briefly what I did:

I followed the Open Etna Installation Guide. It’s pretty straight forward and simple. However there were a few other resources. I can’t stress enough how important good backups are, so make sure your contacts are synced to your google account and make a list of your favorite apps because you are about to wipe everything clean. Better instructions on how to run the backup procedure are at www.zacpod.com/p/71. Bellow the videos on the Open Etna guide is a important reminded to install the Google apps. Considering how important this is I’m surprised they placed it a bit out of the way, and it’s important because it allows you to then resync all your contacts and access all the other Google goodies that help make these phones great (and who wants to reenter all their contacts when all you have to do is hit a button to restore them).

So that’s about it. Now my phone is to date and has a whole lot more life to it. Thanks Android community!

So now Yahoo is shutting down Delicious. I really can’t think of a single service that has been made better under Yahoo, and plenty that have died horrible slow deaths after taking off so quickly and getting bought for decent prices. Yahoo is where you go to die, even if you are healthy.

Anyways, a quick and simple backup command to get an XML file of your bookmarks from Delicious care of Ixiaus 9 on Hacker News

curl https://{your username}:{your password}@api.del.icio.us/v1/posts/all > bookmarks.xml

Ha, coupled with massive lay offs just before xmas I think Yahoo’s holiday spirit can be summed up in “Bah Humbug!”