So more spam has been reaching my inbox lately so I’ve been looking to fix that. I updated my spamassassin and amavis and tweeked their rules but that still can only go so far. The way to do better is to make it more aggressive but I don’t want false positives. So the first step seems to be to make it autowhite list people in my address book. To that end I started with the instructions at http://www.iredmail.org/forum/topic2195-iredmail-support-postfixamavisdroundcube-whitelist-addressbooks.html. They tell you how to get amavis to read a file for whitelisting. In my case, my contact list in roundcube is in a different talbe because it’s coming from ownCloud, so I wrote the following script to dump it daily
/usr/bin/mysql -u USER -pPASSWORD -D roundcube -e 'select email from carddav_contacts;' | sed 's/, /\n/' | sed '1d' > /etc/whitelist.amavis
then tossed it into cron and ran it. Now amavis should whitelist people I correspond list. The next step was to dramatically reduce the sa_tag2_level_deflt (I’m trying 3) and set sa_kill_level_deflt to it’s own value (6.31 for me) and then make sure $final_spam_destiny = D_PASS so that I get the spam for now and can file it as spam and ham accordingly to train it. Finally uncomment the @spam_subject_tag2_maps = (‘***SPAM*** _SCORE_ (_REQD_) ‘) line to make sure it’s flagged in the subject.
Now hopefully I should have less false positives from important people, and a lower threshold for detection and I can train it a bit better manually with time.
We shall see.
Because I’m a very DIY kind of guy, when I recently caught wind of ownCloud I knew I wanted to play with it. They’ve done an amazing job making it easy to install and use.
To start with, since my server is a Gentoo server I ‘emerge owncloud -va‘-ed to get the latest version of the webapp and its dependencies. I already had a SSL enabled vhost setup with apache for hosting other sensitive webapps like Roundcube for webmail, so I created a new owncloud directory there and copied the app into it ‘cp -r /usr/share/webapps/owncloud/5.0.11/htdocs ./owncloud/‘. Then I pointed my browser to the webapp. It had a simple one step setup. It asked for an admin username and password to setup and a database database and user/pass (which I added in myqsl and gave it). And then it was done. Files, contacts, and calendar were installed by default. ownCloud has great on site docs so it was trivial to find the desktop sync-clients list and info and get that install on my Ubuntu laptop. Volia, my own personal dropbox! I was very pleased and impressed.
Contacts have always been a pain for me so I was curious if I could push owncloud a bit. Previous versions (4) appear to have had a built in roundcube email client, but I have my own already and I’m running the newest version 5 of ownCloud. But with these simple instructions I was able to install a roundcube plugin, get it talking to ownCloud contacts, and then in roundcube I drag and dropped my cotnacts over to my owncloud address book and it was done. Also, the ownCloud docs have instructions for getting Android and iOS to get contacts from ownCloud.
So with very little work I’ve got some nice new functionality (contact syncing between webmail and phone) and my own personal dropbox. For less than an hours work I’m quite impressed and pleased. Massive kudos to the ownCloud team.
Update: Be sure in apache to add “AllowOverride all” to the Directory directive for ownCloud so it can secure its data/USER/files directorys with “deny all”s. Otherwise even without logging in anyone can browse your files (and worse, execute php).
So I like running a full computer on my TV. It’s just convenient to be able to easily to Youtube, torrent stuff directly on it, copy files to it over sftp, play any media file I can find, etc etc. Our last “tv box” was a small nettop that was a little under powered: it chocked on high def video files and full screen youtube. So I’ve been waiting for a replacement that fit the following parameters: cheap (less than $400), small nettop form factor and light on power consumption, and more powerful. The Lenovo Q190 hit the mark with dual core and 4gb ram. My only concern was it was a Windows 8 box so it’d be the first time installing Linux on a secure boot machine.
The good news is it went really well. First note, the Windows 8 partition resizer may be the best thing about Windows 8. I remember being stoked when Windows got its own partition resizer back in Vista or Windows 7 days. The only slight con was that it was pretty hungry. If you were using only 20GB on a fresh install, it wouldn’t shrink much lower than 60GB… But the new Windows 8 one is hilarious and will let you shrink down to 100% full. Also it’s unbelievably fast, like I had to reload it to double check it had actually done the shrink.
After that, rebooted the machine and jammed F1 (ok actually I jammed F1, F2, F8, F9, F10, F11, F12, and ESC) to bring up the “BIOS”. There I turned off Secure boot in the security menu, and turned off Quick Boot. Then the USB stick booted Ubuntu fine. The install ran fine too. I did have to go back into the BIOS after to reorder the boot drives. Ubuntu manager to name its partition in a way the BIOS could recognize so I just moved that in priority above Windows 8 and next reboot got Grub!
The device runs fine, all the high def files I could through at it it ran fine and so far even full screen youtube on higher def seems to be ok.
The box is supposed to have Wifi but that doesn’t seem to have been recognized out of the box but that’s ok, I can either poke at it or leave it plugged into ethernet, not a deal breaker. It also came with an adorable hand held keyboard/mouse device but it’s bluetooth also doesn’t seem to be supported. But it fulfils what I need well so I’m pleased and secureboot was less of a pain to work around than I’d worried about, so yeah!
So I was checking syslog for something when I noticed it wasn’t working. In fact it hadn’t updated a log file in /var/log since when I upgraded to 13.04. I did some light googleing, and double checked, yes, rsyslog was still installed, so I restarted it, and it was running fine, and of course without logs it was a bit hard to see what the problem might be. But then I did an ls -l on /var/log and saw its files were owned by “messagebus” but in /etc/rsyslog.conf it was now set to run as the “syslog” user. So I ran “ls -l | grep messagebus” on /var/log and deleted those files and again restarted rsyslog and volia! Logging was working, it created the log files it needed as the new user and went to work.
So over the last month I’ve started up learning new tech again. Most of last year I was in and out of coursera classes learning new… “things” (crypto, Saas, model thinking, game theory, Quantum Computing intro theory, Functional programming with Scala…), but now it’s back to new tech, because there is just so much out there and it’s growing near exponentially.
First up is trying to get into Django. I’ve been a Drupal programmer by trade now for two years and wanted to get a better idea of what some of the other web offerings had to offer. Also again it was another chance to work on my Python programming which has never had a chance to be amazing. I ran through their good but somewhat limited intro tutorial. But that didn’t get me far enough along to really sit down and start my own thing, I needed something more. I found many tutorials that looked good but so many were a few versions out of date. After a lot of googleing though I finally seem to have found a good and recent one in Building Ribbit in Django, part of a series of implementing Twitter clones in various frameworks. So I’ve started that.
Django however wasn’t enough, (and by web standards, relatively old) so I’ve also decided to dive into the more cutting edge worlds of Node.js and Backbone.js. And there’s a lot there to learn, and those are practically umbrella names for a whole host of support technologies you need to learn as well, like Underscore.js. Then the internet indicated good practice also points to learning RequireJS and more too. There are even less tutorials here than for Django and out closer to the cutting edge of tech they fall stale and out of date even faster. But I’m slowly getting a handle on the big picture and how to make things work. Lots of Stackoverflow for this learning. Also, if I have time, Backbone Fundamentals soon published by O’Reilly looks full of potential too.
So that’s what’s been filling lots of my spare time recently… well that and Steam for Linux and Starcraft 2… :)
Heads up, wine 1.5.23 from the Ubuntu ppa seems to break Starcraft 2. So I had to apt-get remove wine1.5 wine1.5-i386 wine1.5-amd64 (or use synaptic to remove it) and then from the cached deb files in /var/cache/apt/archives reinstall (dpkg -i) wine1.5 wine1.5-amd64 and wine-1.5-i386. Then all was good again. Now I have to not take that update as well.
I recommend following the instructions on version locking for those 3 packages to prevent updating wine the next time the update manager pops up. Launch synaptic, search for wine1.5, select the 3 packages and use the menu package->lock version.
Or follow the illustrated instructions here on package holding:
It’s all Rob’s fault. He got a Kinect and was playing with it and also wanted to bone up on his Python so he ended up trying to write some visual processing code in Python and it was unusably slow, so he had to switch back to C. No problem. But then I wanted to get an idea for just “how much” slower it might have been. So I busted out my terrible tried and true language/math benchmark suite Primes and ran the C and Python parts. Except they both failed.
Primes dates back to about 2004 and has seen very rare updates since then. Seems that some of the languages have changed since then. Who knew. So tonight I sat down, installed as many compilers as I could get my hands on, ran the suite, and started attacking broken things. I fixed the following:
- Python range() strictness now requires ints and won’t take floats (or they have to match…)
- gcc (but not g++) now requires linking specified AFTER the source file on the command line (gcc -lm primes.c failes now)
- fixed open cobol compiler arguments (-O to -x)
- Changed Objective C syntax: alloc not part of Object, have to manually write it, and now no need to subclass Object. And add stdio.h
- Removed lua4 support (the auto version detect code (that is bad) was messing up), and really, who now would be running lua4?
- Updated haskell includes for new 2010 library standard
- Switched Fortran compiler from g77 to gfortran and tweaked a few type bugs that came from that
And all the updates can be found in the Github repository for primes. I still find it a handy crappy benchmark and a handy Rosetta stone library. Good to keep it vaguely up to date and still running. :)
So today is the 11th year anniversary of the site. It’s really starting to get some age on it :) Last year was pretty light on posts and side project work progress as work and coursera classes ate most of my time but I got some stuff done. Also a lot of reading.
As for this year? Who knows. I always intend to blog more than I do but at least I also haven’t given up and that’s the most important thing. The site still pulls in some little traffic from google so google analytics tells me so that’s good too.
This year has started off on vacation in Colombia with my girlfriend’s family. We spent new years in Santa Marta and it was beautiful. Now we’re relaxing back home in Bogota. I head back to Vancouver and work at the end of next week. The break has been good and 17 hour car trips and chilling at the beach have gotten the year off to a good start reading wise too.
This year I really would like to spend more time on projects than learning I think and continue to poke more at python, I’m still pretty green at it compared to say my PHP lately. We shall see.
In a lot of ways outside of work last year was pretty chill, I’d like to shake it up a bit more this year, again, we shall see.
Ah well, Happy New years to every one and here’s to doing the things we want to!
So 3 years ago as I was learning Lisp and looking for things to do I noticed it seemed to lack perl style pack/unpack encoding functions that many other languages shared (like PHP, Ruby, Python) so I wrote cl-pack (github) to fill that void. I released it back at the end of the summer of 2009 and moved on. Then this week on github I got a pull request. It seems cl-pack has found some use by people and a github user Taksatou had noticed a bug in the BER encoding and fixed it! So awesome, and thank you! The fix is merged and cl-pack continues to fill a small gap in the Lisp ecosystem :)
So after being alterted to the existence of DKIM by this article posted on HackerNews I wanted to implement it immediatly on my server. DKIM is Domain Keys for Identified Mail, a crypo signing protocol where a pub key sits in your DNS and your mail servers sign your mail as it passes through your server. Seems a little stronger than SPF from a few years ago for authenticating mail’s origin so I was keen to adopt it.
So I found the freshest instructions on the Gentoo wiki and followed them. They were a bit spartan so I went looking for a bit more material and found this Ubuntu tutorial which had some helpful suggestions like the testing section.
After giving the OpenDKIM instructions a first run through I gave the testing a try.
First using dkimcore.org/tools/ I found that the Gentoo OpenDKIM config tool had spat out invalid TXT. It had spat out
And after some quick internet consultation I found out I needed to fix it to
v=DKIM1; k=rsa; p=MIGfM.....
The second test from the Ubuntu docs was an auto-respond test email system that along with wikipedia I learned about ADSP from. So I added
_adsp._domainkey.mindstab.net. IN TXT "dkim=discardable"
to my Bind config as well. (I’m still not 100% about the final ‘.’). Also it seems the autoresponder email tool doesn’t update its DNS too often so I may have to wait a bit to retest.
So now it seems I should have DKIM signed/valid email! :) Just another step to make sure my email is valid, slightly less spoofable and liked/accepted by the big email providers.
Also, seeing results like this from Gmail after receiving my email seems good:
Received-SPF: pass (google.com: domain of firstname.lastname@example.org designates 22.214.171.124 as permitted sender) client-ip=126.96.36.199;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of email@example.com designates 188.8.131.52 as permitted sender)
firstname.lastname@example.org; dkim=pass email@example.com