Mindstab.net

So after being alterted to the existence of DKIM by this article posted on HackerNews I wanted to implement it immediatly on my server. DKIM is Domain Keys for Identified Mail, a crypo signing protocol where a pub key sits in your DNS and your mail servers sign your mail as it passes through your server. Seems a little stronger than SPF from a few years ago for authenticating mail’s origin so I was keen to adopt it.

So I found the freshest instructions on the Gentoo wiki and followed them. They were a bit spartan so I went looking for a bit more material and found this Ubuntu tutorial which had some helpful suggestions like the testing section.

After giving the OpenDKIM instructions a first run through I gave the testing a try.

First using dkimcore.org/tools/ I found that the Gentoo OpenDKIM config tool had spat out invalid TXT. It had spat out

v=DKIM1;=rsa; p=MIGfM......

And after some quick internet consultation I found out I needed to fix it to

v=DKIM1; k=rsa; p=MIGfM.....

The second test from the Ubuntu docs was an auto-respond test email system that along with wikipedia I learned about ADSP from. So I added

_adsp._domainkey.mindstab.net. IN TXT "dkim=discardable"

to my Bind config as well. (I’m still not 100% about the final ‘.’). Also it seems the autoresponder email tool doesn’t update its DNS too often so I may have to wait a bit to retest.

So now it seems I should have DKIM signed/valid email! :) Just another step to make sure my email is valid, slightly less spoofable and liked/accepted by the big email providers.

Also, seeing results like this from Gmail after receiving my email seems good:

Received-SPF: pass (google.com: domain of dan@mindstab.net designates 69.164.214.81 as permitted sender) client-ip=69.164.214.81;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan@mindstab.net designates 69.164.214.81 as permitted sender) 
  smtp.mail=dan@mindstab.net; dkim=pass header.i=@mindstab.net

During the course of updating my home computer I rebooted it because of a kernel update. Later that week at work I went to connect to my home computer and discovered that it’s dynamic IP had changed and it’s DNS name was invalid.

So following common advice to “fix a problem two ways to prevent it in the future” I fixed the DNS, but I also wanted an automated way to track my computers when and if their IPs changed.

So the first thing I needed was shared place to store the IP information. Thinking about it I realized that Dropbox would work well for that. So all I needed was a simple script.

So the solution was to put a script that determined the IP of the computer in Dropbox and have cron on all the computers run it. Each user can call cron with

$ crontab -e

And I created a crontab directory that I could add more scripts to later if need be with and run them hourly with the following entry

0 * * * * cd /home/dan && run-parts Dropbox/cron

The script itself was a file called getip and it used whatsmyip.com’s automation detection script.

getip

#!/bin/sh 
 
wget -O /tmp/`hostname`.ip http://automation.whatismyip.com/n09230945.asp
tmp_file=/tmp/`hostname`.ip
dst_file=Dropbox/var/log/`hostname`.ip
if ! diff -q ${tmp_file} ${dst_file} > /dev/null  ; then
  `cp ${tmp_file}  ${dst_file}`
fi

Then I just created Dropbox/var/log and installed the crontab on all my computers, and volia, homebrew IP tracking for all my compters accessible to me from anywhere.

So at work we use Drupal a lot. And one of our clients has a Google Search Appliance (mini google in a box installed in their datacenter for their personal use) and runs sites that operate in multiple languages. So we installed the Drupal GSA module, flicked it on, and had Google searches. Of course since we had multiple languages, we set up GSA collections, one collection for each language. Easy enough. But by default the Drupal GSA module only takes one collection in it’s admin page. So we did what we always do in this situation, we made the collection a multilingual variable, a kind of Drupal hack that enables that variable to have different values transparently depending on the site’s language. Then we input each GSA collection into each language/site’s GSA config. Done and good to go? Nope. All the searches were still in English for all the sites. So on the French site we cleared the cache and then the searches were in French! … for all the sites. The Drupal GSA module was deeply not designed to work with multiple languages/collections it turned out.

After about 2 hours debugging we figured it all out. When the module is initialized, it pulls out all the config data, puts it in a hash, and hands it off to Drupal in the menu hook. Drupal stores it forever, and hands it off to the search function whenever it is called and so what ever language is active when the module is initialized determines what colleciton is used forever for all languages. Not cunning. The fix however was straight forward and one line: in the search function, just disregard the passed in settings value for the collection and re-pull it.

For what it’s worth I filed a bug and patch (against 6.x-2.0-beta1, though it should work fine for all versions) with the module at http://drupal.org/node/1208892 but it seems it’s suffering from low maintainership.

Here it is as well, though it’s probably just as easy to copy/paste the line in directly.

Index: google_appliance.module
===================================================================
--- google_appliance.module	(revision 14982)
+++ google_appliance.module	(working copy)
@@ -1055,6 +1055,9 @@
  *   Themed search results.
  */
 function google_appliance_search_view($search_base, $client, $collection, $keys = NULL, $title = NULL) {
+  
+  $collection =  trim(variable_get('google_appliance_default_collection', 'default_collection'));
+  
   $form = drupal_get_form('google_appliance_search_form', NULL, $search_base, $client, $collection, $keys);
   // When POSTing back to an existing search-results page, the original
   // URL is accessed (which re-runs that search) and then the redirect for

Now it works just fine and all our different language sites have searches and results in just their language. Splendid!

Let’s say you did a presentation that was recorded and you’d like to post it to your website. Sadly, let’s now say there are some problems, like that your 5 minute presentation is part of a nearly 2 hour video only available in a flash player that doesn’t even have a time display so you couldn’t even point people to the video and say jump to 1 hour and 15 minutes to see me. It sucks. Technically your presentation is available online, but it’s not really accessible. So here is how you might rescue it!

It turns out there are two ways flash players server videos these days. The first and easiest is that a simple flash player loads in your browser, and uses your browser to make a GET request to the server to load a .flv file (FLash Video). This is relatively easy to intercept, there are lots of tools and plugins for Firefox that do this automatically for you. Even better, on Linux for example, these videos are usually stored in /tmp so your browser does the whole job and gives them to you. No work required.

The other more complicated but more secure option is that the flash player connects to a dedicated rtmp server that streams flash video. The flash plugin does the networking and there is no file to save, it’s a stream.

If you are lucky enough to have a player using the first option, you are done. Assuming you have the second option, then your fun has just begun.

First we need to try and figure out where the server that your flash video is.

My first approach was to use wireshark to sniff the traffic. Through this I discovered the basics, like the address the server and the port, 1935.

Next I installed rtmpdump. RTMP is the Real Time Messaging Protocol and rtmpdump is a program that can connect to an RTMP server, get a stream and save it to a file. Sadly the data I got from wireshark didn’t have all the parameters I needed to get the file. Or I couldn’t read it properly. So while I knew where the server was and could now connect to it, I still didn’t know how to ask for the video I wanted.

Thankfully rtmpdump comes with several other utilities. After reading its README I went the rtmpsuck route. I set local redirecting of all port 1935 requests to localhost with iptables and ran the rtmpsuck proxy server. In theory it was supposed to intercept all calls from the flash player to the rtmp server, decode and spit them out, and then forward them along. Even better, it would try to save the stream on the way back as it passed through it.

# iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner --uid-owner OWNER_UID  -j REDIRECT
$ ./rtmpsuck

Where OWNER_UID is the uid of the user running rtmpsuck. With this running I just reloaded the page with the player (twice, it’s a bit glitchy) and then tried to skip to where my part was so it would save the stream from there.

It was partially successful. It spit out on the console all the pertinent path parameters about the video on the server, but it kept chocking on bad packets of data and stopped recording. Also for some reason the video it did store was very large, space-consuming wise.

Armed with the right parameters though I was able to use rtmpdump to suck down the whole video from the server surprisingly quickly and in a reasonably sized format.

$ ./rtmpdump  -r rtmp://server.net/app_name/blah/event/date/date-1 -o video.flv

Now the video was liberated from its flash interface and in my possession, I just had to cut out my small part and then convert it to a more common format.

$ mencoder -ss 1:15:50  -endpos 0:05:57  -ovc copy -oac copy video.flv -o result.flv
$ ffmpeg -i result.flv result.avi

And volia. I now have just my part of the video and in a common format. I mean you hypothetically do! Yes…

Completely unrelatedly, you can expect to see my presentation on my project Cortex from the BCNet Broadband Innovation Challenge (where I got second place) online soon.

So early this year I bought my first Android phone, an LG Eve, from Rogers Mobile. It turns out it came with Android 1.5 which was a year old at the time, the current version was 2.1 and 2.2 was released that spring. Rogers has been promising an update to 1.6 for the phone pretty much since I bought it and still no release. It was pushed back twice and has since gone silent. This could be the end and a sad story at that, with just over 2 more years left on my contract.

However, this is Android! And so heroically the CyanogenMod people have been following the Android code base and releaseing an free up to date version that works on some more common phones. The good folks of Open Etna have taken that work and customized it and made sure it works on the LG Eve specifically. So following the instructions on their and other sites, I have been able to upgrade my phone to Android 2.2, and it’s awesome. There is a lot more software available, voice commands, live wallpapers, and a JIT compiler so software should run faster, and more that I’m just discovering (ships with a good terminal app). And just generally newer default apps. So thank you very much to those communities of volunteers for doing vastly more than what my Phone company was incapable of.

And now briefly what I did:

I followed the Open Etna Installation Guide. It’s pretty straight forward and simple. However there were a few other resources. I can’t stress enough how important good backups are, so make sure your contacts are synced to your google account and make a list of your favorite apps because you are about to wipe everything clean. Better instructions on how to run the backup procedure are at www.zacpod.com/p/71. Bellow the videos on the Open Etna guide is a important reminded to install the Google apps. Considering how important this is I’m surprised they placed it a bit out of the way, and it’s important because it allows you to then resync all your contacts and access all the other Google goodies that help make these phones great (and who wants to reenter all their contacts when all you have to do is hit a button to restore them).

So that’s about it. Now my phone is to date and has a whole lot more life to it. Thanks Android community!

A while ago I tried with mixed success to get Lisp onto my Gentoo Hardened server. I had to go a binary only route and kind of stopped there not taking it any farther. Now, 2 years later, I need the full meal deal, lisp + emacs + slime, on my server, which is now a Xen VPS with as much hardening as I could get (much less kernel based hardening since it’s the VPS’s kernel). It was still too much for SBCL to compile in portage so here’s what I did to get it all working.

So you need an out of tree binary copy of SBCL. Live with it. It works. The problem with going with out of tree software, especially for a language, is that what ever binary you get isn’t supported and hasn’t been tested against all the software in-tree. For instance I initially tried the newest version of SBCL (1.0.42) but ran into problems with portage’s stable slime.
Ultimately I went with the closest I could get to portage’s stable version. Portage has 1.0.19 marked as the most recent stable version so I went out and downloaded the binary of that version

$ wget  http://sourceforge.net/projects/sbcl/files/sbcl/1.0.19/sbcl-1.0.19-x86-linux-binary.tar.bz2/download
$ tar -xjf sbcl-1.0.19-x86-linux-binary.tar.bz2

So change into the directory and check out INSTALL. Basically installation is easy. Binary SBCL is configured around installing into /usr/local but that can be gotten around. So we’ll go with a more traditional install into /usr

Note: My test box is a VPS with a Xen kernel not a hardened kernel so I didn’t have any PaX problems, but my notes for the last time I tired this on a full hardened install mention that you need do disable some PaX features before SBCL will work:

$ paxctl -p -e -m -r -x -s " on src/runtime/sbcl

Install to /usr

# INSTALL_ROOT=/usr sh install.sh

Now SBCL is installed but it won’t work because the binary is preconfigured to look for the core in /usr/local. So we’ll borrow the gentoo SBCL config files to get that setup properly.

/etc/env.d/50sbcl

SBCL_HOME=/usr/lib/sbcl
SBCL_SOURCE_ROOT=/usr/lib/sbcl/src

# env-update

The above file and command set up the system environment variables to tell SBCL where it’s really installed. Now is as good a time as and to ‘source /etc/profile‘ to get those changes.

Now SBCL is installed and working, we need to let portage know that. There used to be a ‘emerge –inject’ method, but that’s been deprecated in place of a new provides file

/etc/portage/profile/package.provided

dev-lisp/sbcl-1.0.19

Now portage knows about our SBCL so we can start installing things that depend on it like the rest of our tool chain

# emerge cl-asdf emacs slime -va

So now we have all the pieces, all they need is some gluing together. Again we’ll borrow from the Gentoo SBCL files.

/etc/sbclrc

;;; The following is required if you want source location functions to
;;; work in SLIME, for example.
 
(setf (logical-pathname-translations "SYS")
    '(("SYS:SRC;**;*.*.*" #p"/usr/$(get_libdir)/sbcl/src/**/*.*")
          ("SYS:CONTRIB;**;*.*.*" #p"/usr/$(get_libdir)/sbcl/**/*.*")))
 
;;; Setup ASDF
(load "/etc/gentoo-init.lisp")

/etc/gentoo-init.lisp

(in-package #:cl-user)
#+(or sbcl ecl) (require :asdf)
#-(or sbcl ecl) (load #p"/usr/share/common-lisp/source/asdf/asdf.lisp")
(push #p"/usr/share/common-lisp/systems/" asdf:*central-registry*)
(asdf:oos 'asdf:load-op :asdf-binary-locations)
(setf asdf:*centralize-lisp-binaries* t)
(setf asdf:*source-to-target-mappings* '((#p"/usr/lib/sbcl/" nil) (#p"/usr/lib64/sbcl/" nil)))

Now everything should work. You just need to set up your emacs and slime

~/.emacs

; your SLIME directory
(add-to-list 'load-path "/usr/share/emacs/site-lisp/slime/")
; your Lisp system
(setq inferior-lisp-program "/usr/bin/sbcl")
(require 'slime)
(slime-setup)
 
(global-set-key (kbd "C-c C-q") 'slime-close-all-parens-in-sexp)

Now It’s all glued together, give it a go

$ emacs
M-x slime

If you don’t get any compilation errors you should be in emacs + slime.

And there you have it, SBCL emacs and slime on Gentoo Hardened.

Cavets

1) For some reason this approach adds some annoying extra text to vanilla SBCL start up that I can’t seem to get rid of

$ sbcl
This is SBCL 1.0.19, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http: //www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
; loading system definition from
; /usr/share/common-lisp/systems/asdf-binary-locations.asd into
; #<package "ASDF0">
; registering #<system ASDF-BINARY-LOCATIONS {AAF8F51}> as ASDF-BINARY-LOCATIONS
* 

2) The system I tested this on is a VPS so the kernel is a Xen kernel, not a hardened kernel, so there may be additional complications on a full hardened install. Please let me know if you have any, and especially any working solutions.

CLC or Common Lisp Controller is a system that other Lisp systems user to keep track of ASDF systems (a mouthful I know). By default, system packages are installed to /usr/share/common-lisp/ but what happens if you don’t have root access but still want to leverage the ease of use CLC installed ASDF packages provide?

clc-register-user-package to the rescue! Create ~/.clc/source and put your ASDF package there, then simply run

$ clc-register-user-package ~/.clc/source/package/package.asd 

and volia, you can (require :package) from any of your Lisp systems there after.

It’s pretty awesome. :)

I decided I really needed to work on a server backup system, so here are my notes on the system I have now.

First things first, the files I want to backup are owned by all different users, so the only user who can run the backup process is root. Therefore I can’t just run rsync from my local machine and grab the files from the server, the backup process has to be run on the server and backup to the backup machine. Now in my case this was a bit of a trick because the backup machine was behind a firewall, so the server had no direct line of communication to it. So I wrote a script to turn on a reverse ssh port forward.

recv-serv-backup.sh

#!/bin/sh
 
ssh -R 8000:127.0.0.1:22 -N user@kvasir.mindstab.net

When run on the backup machine behind a firewall, it connects to the server (kvasir) and listens on port 8000. When ssh on kvasir connects to port 8000 it redirects that traffic to local port 22, the ssh port of the backup machine. This is how the firewall is gotten around. Reverse port mapping is a cool trick to master.

Next as root on kvasir I generated a public ssh key and put it on the backup machine so root could automatically log on repeatedly to the back up machine (think lots of rsync calls) once the key was loaded once. Then I hooked up the key to keychain. That is all better outlined at:
Gentoo Linux Documentation: OpenSSH key management, Part 1 and
Gentoo Linux Documentation: OpenSSH key management, Part 2.

The all I needed to do was poke the rsync syntax to use the nonstandard ssh port for backup. The best method I found was

rsync -e "ssh -p 8000" -av 

rsync for those of you who don’t know is a great little backup tools. It’s like a smart (in that it only copies files that have been modified since the last backup) network aware (since it can use ssh) copy tool. Simple but really useful.

So with that I wrote a backup script on the server

server-backup.sh

#!/bin/sh
 
backup () {
        echo "$1..."
        DST=`echo "$1" | awk '{split($0,a,"/");  result = "/"; for (i=2 ; i < length(a)  ; i++)   result = result "/" a[i];  print result;  };'`
        rsync -e "ssh -p 8000" -acv $1 dan@127.0.0.1:~/kvasir$DST ;
 
}
 
backup "/svn"
backup "/git"
...

There is one caveat, rsync won’t create subdirectories on the other side specified in the path so you need to create the basic directory structure.

 rsync -e "ssh -p 8000" -av /git user@127.0.0.1:~/kvasir/git 

is fine because it will create /git just fine, but

 rsync -e "ssh -p 8000" -av /home/user user@127.0.0.1:~/kvasir/home/user 

will fail if ~/kvasir/home doesn't exist. So you'll need to create the basic directory structure or enhance the backup function to strip out extra directories in the target path.

Finally, I didn't want anyone and everyone to potentially be able to gain access to private data on the backup machine, so the target directory needed to be encrypted. There are a lot of options, but I opted for the easy encFS route and just installed "cryptkeeper" and had it setup the directory.

Now all I have to do is mount the encrypted backup directory, run the script to turn on the reverse ssh tunnel, and run the backup script, and I have an encrypted backup solution for my server that gets around firewalls.

Not bad.

References

• What Goes Out Can Come Back In -- SSH tricks
• Google

I don’t know who to be annoyed at actually. Microsoft just upgrade their hotmail interface, and hotmail is about the last MS product I use (that and the MSN server though I obviously don’t use their client). Anyways, suddenly hotmail wasn’t working for me which was a rather large pain. I searched around and found the solution.

In Firefox, goto “about:config” and put a filter of “vendor” in and hit enter. Change general.useragent.vendor from “Ubuntu” to “Firefox”.

Then it works fine. It’s annoying MS is still doing such browser specific tweeking and it breaks on something so small, but then again, it does seem that they at least support vanilla Firefox. I don’t know how important the vendor string is but if it’s breaking compatibility maybe Canoncial should leave it alone?

Edit (2010-09-16): I tried again with much better success. Read the new version of this guide at:
http://www.mindstab.net/wordpress/archives/806

My server, mindstab.net, runs Hardened Gentoo. I like it. It provides nice features from grsecurity and PaX like memory randomization, non executable writable memory, etc. However, it really doesn’t get along so well with Lisp. Lisp in general seems to like executable and writable memory, and SBCL at least also doesn’t like randomized memory. So it took a bit of work to get Lisp onto my server.

Approach 1: Failure
I spent a bunch of time trying to patch the build process in portage to coax SBCL into building. First, of course, I used gcc-config to disable the hardened gcc profile, and just use the vanilla one. Then I created a suid root shell script to call “paxctl -m -p -r -e $1” so that the sandboxed build process could disable PaX features on the SBCL binaries. I added the command to the ebuild, and created a patch to insert the command into SBCL’s build process. The process goes like this, portage download’s the SBCL source and a pre-compiled SBCL binary. The patched ebuild then calls my suid root script which disables PaX on the pre-compiled binary so it actually runs (as opposed to crashing under PaX) and then a new SBCL binary is built from the source and the pre-compiled binary builds a core file from the SBCL lisp source. The patched SBCL make.sh then again calls the suid root script on the new binary, so it will run. Then it should load the new core and recompile the system for itself. Sadly, while it runs at least, it chokes on the core file and hangs while using 100% cpu. I couldn’t get past this so I eventually gave up. If anyone has any suggestions that’d be great.

Approach 2: Success
So the actually solution was as follows: Download the most recent precompiled SBCL binary from the website (1.0.15 for x86), run “paxctl -p -e -m -r -x -s ” on src/runtime/sbcl (to cover all the bases). Then run “sh install.sh” to install SBCL to /usr/local. That’s it.

The problem with this is you can’t emerge lisp packages in portage, you have to install them by hand (unless maybe you want to fake inject the package into the portage database).

I downloaded a copy of slime, untarred it and popped it in my .emacs and I had a full lisp environment ready to go, and on my hardened machine no less. Not so bad.