Mindstab.net

So over the last month I’ve started up learning new tech again. Most of last year I was in and out of coursera classes learning new… “things” (crypto, Saas, model thinking, game theory, Quantum Computing intro theory, Functional programming with Scala…), but now it’s back to new tech, because there is just so much out there and it’s growing near exponentially.

First up is trying to get into Django. I’ve been a Drupal programmer by trade now for two years and wanted to get a better idea of what some of the other web offerings had to offer. Also again it was another chance to work on my Python programming which has never had a chance to be amazing. I ran through their good but somewhat limited intro tutorial. But that didn’t get me far enough along to really sit down and start my own thing, I needed something more. I found many tutorials that looked good but so many were a few versions out of date. After a lot of googleing though I finally seem to have found a good and recent one in Building Ribbit in Django, part of a series of implementing Twitter clones in various frameworks. So I’ve started that.

Django however wasn’t enough, (and by web standards, relatively old) so I’ve also decided to dive into the more cutting edge worlds of Node.js and Backbone.js. And there’s a lot there to learn, and those are practically umbrella names for a whole host of support technologies you need to learn as well, like Underscore.js. Then the internet indicated good practice also points to learning RequireJS and more too. There are even less tutorials here than for Django and out closer to the cutting edge of tech they fall stale and out of date even faster. But I’m slowly getting a handle on the big picture and how to make things work. Lots of Stackoverflow for this learning. Also, if I have time, Backbone Fundamentals soon published by O’Reilly looks full of potential too.

So that’s what’s been filling lots of my spare time recently… well that and Steam for Linux and Starcraft 2… :)

It’s all Rob’s fault. He got a Kinect and was playing with it and also wanted to bone up on his Python so he ended up trying to write some visual processing code in Python and it was unusably slow, so he had to switch back to C. No problem. But then I wanted to get an idea for just “how much” slower it might have been. So I busted out my terrible tried and true language/math benchmark suite Primes and ran the C and Python parts. Except they both failed.

Primes dates back to about 2004 and has seen very rare updates since then. Seems that some of the languages have changed since then. Who knew. So tonight I sat down, installed as many compilers as I could get my hands on, ran the suite, and started attacking broken things. I fixed the following:

• Python range() strictness now requires ints and won’t take floats (or they have to match…)
• gcc (but not g++) now requires linking specified AFTER the source file on the command line (gcc -lm primes.c failes now)
• fixed open cobol compiler arguments (-O to -x)
• Changed Objective C syntax: alloc not part of Object, have to manually write it, and now no need to subclass Object. And add stdio.h
• Removed lua4 support (the auto version detect code (that is bad) was messing up), and really, who now would be running lua4?
• Updated haskell includes for new 2010 library standard
• Switched Fortran compiler from g77 to gfortran and tweaked a few type bugs that came from that

And all the updates can be found in the Github repository for primes. I still find it a handy crappy benchmark and a handy Rosetta stone library. Good to keep it vaguely up to date and still running. :)

So 3 years ago as I was learning Lisp and looking for things to do I noticed it seemed to lack perl style pack/unpack encoding functions that many other languages shared (like PHP, Ruby, Python) so I wrote cl-pack (github) to fill that void. I released it back at the end of the summer of 2009 and moved on. Then this week on github I got a pull request. It seems cl-pack has found some use by people and a github user Taksatou had noticed a bug in the BER encoding and fixed it! So awesome, and thank you! The fix is merged and cl-pack continues to fill a small gap in the Lisp ecosystem :)

During the course of updating my home computer I rebooted it because of a kernel update. Later that week at work I went to connect to my home computer and discovered that it’s dynamic IP had changed and it’s DNS name was invalid.

So following common advice to “fix a problem two ways to prevent it in the future” I fixed the DNS, but I also wanted an automated way to track my computers when and if their IPs changed.

So the first thing I needed was shared place to store the IP information. Thinking about it I realized that Dropbox would work well for that. So all I needed was a simple script.

So the solution was to put a script that determined the IP of the computer in Dropbox and have cron on all the computers run it. Each user can call cron with

$ crontab -e

And I created a crontab directory that I could add more scripts to later if need be with and run them hourly with the following entry

0 * * * * cd /home/dan && run-parts Dropbox/cron

The script itself was a file called getip and it used whatsmyip.com’s automation detection script.

getip

#!/bin/sh 
 
wget -O /tmp/`hostname`.ip http://automation.whatismyip.com/n09230945.asp
tmp_file=/tmp/`hostname`.ip
dst_file=Dropbox/var/log/`hostname`.ip
if ! diff -q ${tmp_file} ${dst_file} > /dev/null  ; then
  `cp ${tmp_file}  ${dst_file}`
fi

Then I just created Dropbox/var/log and installed the crontab on all my computers, and volia, homebrew IP tracking for all my compters accessible to me from anywhere.

So starting around October last year Stanford started the first few of its free online geared classes. I successful took and completed and passed the Artificial Intelligence class and the Machine Learning class. That, coupled with full time work, pretty much left me with no time for anything else. I was crazy busy for that 10 weeks but I learned a bunch and it was fun. Over the winter break, even though I’ve been busy, I’ve more or less recovered, and just in time, because in a few weeks a whole new batch of free online classes will be kicking off. I’m going to see if it’ll be feasible for me to do 3 this time, but I might have to settle for 2. So If I vanish against for another 10 weeks, that’ll be why.

This stuff is amazing and a lot of it simply isn’t available at the university I went to, so to have it now for free is a pretty big opportunity so I’m eager to cram in as much as I can before it goes for pay (it has the feel of a beta/demo that will go that way once material is built up and kinks worked out).

So cheers! and see you again in the spring!

About 3 weeks ago Lennart Poettering posted A Plumber’s Wish List for Linux to his blog, which is one of the many Linux related blogs I read. Included on that list was a request to “expose CAP_LAST_CAP somehow in the running kernel at runtime”.

Now I’d never contributed to the Linux kernel before, but in college we had hacked on the Minix kernel to extend it and in University we had written our own kernel and implemented virtual memory, task switching, message passing and preemptive processes; so I was at least familiar and comfortable with kernel level development. Looking at this task, I immediately thought, “hey, that is totally within my capabilities and I can probably squeeze it into my schedule”.

So I did. I checked out the latest copy of the torvalds Linux repository and implemented the feature. The actual implementation didn’t take too long. Mostly looking for the right place. What did take a day or two was getting up to speed on Linux coding practices and patch submission protocol and procedure. They have a good process in place and it’s best not to step on toes so I did my research as best I could. When I was ready I sent in my patch.

Andrew Morton picked it up, asked for a few enhancements, and those submitted, added it to his -mm tree for testing. Then a few weeks later Linux 3.1 launched and the new merge window opened and today Andrew submitted his diff from the -mm branch including my patch to Linus and it was merged to his tree!

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=73efc0394e148d0e15583e13712637831f926720

I am now a Linux kernel contributor. I have a small piece of code in the Linux kernel that will ship with 3.2. It’s not huge, but I did it, and that gives me a real sense of pleasure and accomplishment. Thanks to all who helped!

Well, aside from being pretty busy with work, I spent some more time playing with OpenGL and Lisp. I took my demo test of a spinning diamond up to the beginnings of a space flight sim. Now we have a ship that has a thruster that applies forward thrust, there is the beginnings of physics. The next logical step would be to add turning, but I wanted to do it with physics, which meant writing torque code and that’s about where I got side tracked and have stopped for the moment. Not super interesting yet, but not bad for someone who has never done 3D programming before. The code is at https://github.com/dballard/flight-sim.

I then got distracted by the idea of a checkers AI by a few articles I read in August. I decided to dust off my Ruby and give that a go but it didn’t get too far before I realized that I was going to have to get side tracked from search in to aggressive pruning and heuristics. I was satisfied with the project even though it doesn’t really play because at it renewed me Ruby which had been shelved since some time last year and reminded me that search of this kind is bloody hard. The code is at https://github.com/dballard/checkers. It can simulate a full game to any depth if you have the time. If you have less than days or years per turn, it’s pretty much no good.

Now I’m getting further distracted by a few ideas in different directions yet again and I’m starting the Stanford AI class and Machine Learning class that they are offering online for free on monday. That should keep me pretty much tied up until years end and I’m looking forward to it all!

Also thanks to my Kobo Touch eReader I’ve been reading a ton, which is nice. I may have to give up my pace of a book every week or two for the next month or two to keep up with the new classes, we shall see.

• Zach's Journal – Making a small Lisp project with quickproject and Quicklisp
  Zach outlines how to start projects using quicklisp and quickproject
  (tags: lisp quicklisp project depends)

I woke up today to a message from a physics PhD student in the US asking about my old Lisp Go AI bot code (from the Mindstab Go AI competition of 2008) and if I was ever planning on releasing it or if he could get a look at it since Go and programming were two of his hobbies.

“fink”, as my Go AI bot was called, was my frist real Lisp project, and in hindsight probably not that good, but I learned a lot from it and it was fun, which is all that really matters. Possibly it’s most notable feature now is that it’s a Lisp program that speaks GTP and may provide a base for someone to write a better bot with. So if someone else can learn something from it or use it, that would be pretty cool. So here it is

https://github.com/dballard/fink

There’s some notes about it in it’s README. I don’t recall if I documented it much but hopefully it can be made sense of :)

So as you may know I’m trying to keep up with programming exploratory side projects in my spare time, somewhat spurred on by among many things, The Cult of Done. After learning a bit about HTML5 and Canvas and writing the The Game of Life for Canvas I wanted to move on to something more meaty. That’s about as far as I’d ever gone in graphics programming so I wanted to keep pushing on in that vein for a bit. So I figured why not check out 3D land. I briefly looked at WebGL but it’s not entirely ready just yet. Some of the tech demos look decent but for one thing, there’s almost no interactivity in the more complicated ones like flight sims, I assume because a good IO system isn’t in place, so I figured why not go for straight up OpenGL.

The defacto language for OpenGL programming is C/C++ however I’ve done my share of C programming and I’m enjoying higher level languages for many things these days. I have an ok relationship with Python but it doesn’t get me excited or anything, and I’m still working at getting better at Lisp which I am fond of so I thought why not use this as another excuess to work in Lisp? And so against some people’s advice I’ve been exploring OpenGL programmign in Lisp.

I Googled around and found that the basic package is cl-opengl and using Quicklisp got that installed and found a very basic tutorial for getting a window to show with a tutorial at nklein.com from NeHe. It worked okish but not really. Then I found 3bb.cc’s cl-opengl tutorials which were better. They used SDL instead of GLUT to make a window and the first one worked and I had a raindow triangle. Instead of going on, I started looking at the opengl.com tutorial’s in C and tried translating them. I worked through the cube.c one, but it didn’t quite work, so after a bit I gave up.

I went back to the 3bb code and instead of continuing on with the tutorials, I just started working on it myself and googling for parts I needed and getting some help from a friend who’d done some heavier OpenGL programming in C. I made the color change over time (like I learned for the game of life). Then I made the triangle spin. Then I added a second back triangle and made them spin. Then I fleshed it out and made pyramid, then a diamond that spun. Then I finally made the spinning color changing diamond that also moved in a circle, refactoring the code a few times in the process.

Now I have a nice simple tech demo and some code and understandings of the basics of OpenGL and 3D programming. For about 8 days work in spare time I am very pleased. I have more plans to move on to from here, but I’ll probably start a new project for it. The code for all my demos is on github at https://github.com/dballard/cl-opengl-tests. The only one that really works is 3bb.lisp and you need cl-opengl and lispbuilder-SDL.

Something I discovered is that the linear algebra scene for Lisp is confusing and a mess (IMHO). There are a bunch of projects for it, all kind of a mess with no apparent blessed/defacto standard library. I looked at L-math, but the version in quicklisp seemed a bit broken :/. This whole Lisp philosophy of always using the latest dev code instead of stable releases seems… against better sense. And seemed to fail me here. I looked at Femlisp/matlisp but their own FAQ talked about how their linear algebra implementation was slow and that’s all I wanted it for. So in the end I just rolled my own. It was fun and I had to remember some basic math from a few years ago.

Here’s the final product: