Posts for Friday, August 21, 2009

avatar

You’re putting your mum on Gentoo? You’re mad.

This is the second time I’m putting Gentoo on my mum’s computer. The first time was a good year or so ago – however my own old laptop got a hardware failure soon after and so I *ahem* took her computer. (I’m innocent I swear!) She’s decently computer illiterate and has always wanted to learn. She recently got a new laptop, an Acer Aspire 4535 (it comes without Windows pre-installed).

Had to install it using the SystemRescueCD as Gentoo’s minimal install didn’t have the module for my NIC. Xorg is compiling, the holidays are almost over, and it’s time to overload my schedule again.

To make this a bit more computer-relevant, I ask you: what do you suggest I do to help make it “easier” to use for someone like my mum? I am planning a cron-scheduled usual sync, update, revdep-rebuild. I don’t think I can automate the etc-update but that could pretty easy to train I think. Kernel updates is going to be a hassle. She wants KDE and so that means unstable packages.

Sounds fun.

Related posts:

  1. Gentoo installed (again).

Posts for Thursday, August 20, 2009

searchpairpos

How did I live this long without knowing about searchpairpos() in Vim? I hate when I write a clumsy, slow reimplementation of something that already exists as a standard function.

The only bad thing about Vim and Emacs both is that the feature list is about a mile and a half long (and that's a bad thing only in the sense of being an overwhelming amount of good things).

I have read almost the entire Vim manual at this point but there are corners that remain unexplored, and sometimes they contain treasure. One thing I love doing is answering Vim questions on Stack Overflow because most of the time I don't know the answer right off the bat, and so looking it up or figuring it out teaches me something new.

Emacs is another story... Emacs remains a mystery to me in many ways, in spite of having used it for about a year now. I very much plan to read the whole Emacs manual. I've already read parts of it but I seem to have barely made a dent. There are things I know should be simple to do or that there are already built-in options for, but I don't know how to do them.

  • How do I kill a word and also kill the whitespace immediately after it so it yanks properly later?
  • When I kill-whole-line and paste that line elesewhere, I lose a newline and screw up indenting. Sometimes it works how I expect and sometimes it doesn't.
  • There are so many things I can do in Vim but can't in Emacs... marks, multiple registers, abbreviations, sensibly configured per-filetype indentation.

etc. etc. I know there are ways to do these things once I have time to just sit down and read the darned manual. And learn elisp's syntax and semantics (which can't be harder than learning Vim script). My ~/.vimrc is currently twice as long as my ~/.emacs, which says a lot.

On a related note, I'm in the process of putting my Vim and Emacs configs on github.

Read "Little Brother"

The title of this post can be either read as a statement or a command, and initially I wanted to write a lenghtly sloopy post about how some books change your life or just mean a lot to someone etc. etc. etc. ...but I won't!!

It is a command.

Read it!

If you know me, you know that I don't make recomendations lightly and that I have a very good reason to do so now. (Yes, it was the first book in years that made my heart pound wildly and I didn't want to part with it...)

If you don't know me or just don't care what I think or don't want to follow what I say, read it anyway! XD

I won't tell you why, just do it! Now!!!

Bloody download the bloomin' book and read it!!!

hook out >> read it, enjoyed it, have to study now

P.S. Cory Doctorow, you are an evil evil man to have written such a brilliant book. If I fail these exams, I am sooooo totally blaming your evil genius for making me procrastinate!!! XD
<!--break-->

"What's in a name? That which we call a rose / By any other name would smell as sweet."

The quote that forms the heading for this post is from William Shakespeare's play "Romeo and Juliet", this post is about names.

Today _why died. But _why is not a real person, he is the virtual personality of someone. That someone decided to delete _why's Twitter account, his websites and similar things for unclear reasons (and those reasons are not actually the topic here either). _why's "death" created quite a lot of fuss in the Ruby community where _why was … well not a star, more like a saint actually ;-).

If I would describe this to my dad he'd call those guys nuts: _why is not a "real person" and _why was not his "real name", so why talk about it? What's it with those "nicknames" anyways?

A few days ago I wrote about our authorship fetish and how some people can't deal with the fact that the "brilliant creator" doesn't exist anymore (and never really has) because we just merge all our influences and communication to "create". The fetish for the "author" is only outmatched by our fetish for names.

When we are born a name is given to us, we even call it our "given name". The rest of our name is inherited from our parents: We have no choice and we are not asked. But all our life we are forced to use that name, everywhere. When I want to publish "serious" work I have to put my "real name" on it. When I order something online, I have to give my "real name". Many have gotten used to people that "live online" having "aliases" just as we allow "artists" to have stage names. But those are not "real" in our minds. They are just "made up" and therefore are just toys.

In older times people used to use "invented" names because they thought that if someone knew your real name that person had "power" over you, could cast magic spells on you. Today probably only very few people still believe in that direct relation but the way we see "real names" still hasn't changed: If tomorrow I would ask everybody to call me "WASD" or "1234" or "Peter" I'd be looked at like I was a mad man (even more than usual). I can change my "official" name for a lot of money (and only if I have a very serious reason and can convince some bureaucrat to agree that I should be allowed to change my name. And even then I cannot really chose freely, I have to choose something "appropriate", something that looks like a "real name").

Online I usually use the handle "tante" (if it's not already taken which happens more often than I like ;-)) and there's a bunch of people who either only know me by that name or who would never call me something else even if they know my real name. If you want to get my attention in real life your chances are best if you yell out "tante" and not "Jürgen" or "Mr. Geuter". "tante" is my name. I picked it, it was not chosen for me, so why should it be any less "real" than the name my parents picked for me? And why shouldn't I be allowed to drop it any time I want and chose another one?

Names don't have mystical powers, they're just handles. As any government official will tell you, they're not even very good identifiers, cause we have so many name collisions. Which is why official databases use something else as identifier. So if names are no longer identifiers (a function they might have had in the past) why should we force people to stick to the name they didn't even chose for themselves? We don't force people to stick to the religion that their parents might have chosen for them by having them christened as babies. Why with names?

Yeah it might be a pain in the ass to learn a new name if someone you've known for a long while changes his/her handle, but in this case, it's not about you.

Names are like fashion. A chosen name says something about the person that chose it. Maybe it just says something about what that person likes soundwise. Maybe it says more. But we'd never say that some man isn't allowed to wear a skirt if he wants to, so why shouldn't I be allowed to change my "real name" to "f(x)=Π*6/x"?

I propose the following new rule: Everybody can always change his/her name. There's a fee to cover the actual costs (like changing passports and stuff) but that's it. If you wanna be called "Poopy the dogmaster" so be it.

Shoes 2 packaged for Ubuntu: My first package

So you may have noticed a few days ago a link to an article on teaching to program in a newish language called Shoes. Its a cute language on top of Ruby for whipping up fun cute little GUI apps, event oriented and good for introductions to programming. So I wanted to play with it but Ubuntu and Debian ship the old version 1 and version 2 has been out since December 2008. So I checked Ubuntu's bugzilla and sure enough, there was a bug from April asking for a version bump with no response. So I figured it might be time to step up to the plate. So I brought up the Ubuntu Packaging Guide and gave it a read. Turns out Shoes wasn't trivial to package but with the old version 1 Deb package as a starting point I was able to get version 2 packaged! It's now sitting in Ubuntu's bugzilla at https://bugs.launchpad.net/ubuntu/+source/shoes/+bug/359031 and if you just want the Shoes 2 i386 deb, its at http://launchpadlibrarian.net/30491206/shoes_0.r1134-1_i386.deb. So yeah, check it out, give it a whirl, have fun.

As a side note, I've found Ubuntu's bugzilla to be sporadically responsive which sucks a bit, but does encourage one to step up some... But looking at Debian, where this package actually originated from is even worse. They have no web interface for entering bugs, they only accept them via email or a command line tool. It does seems like a epic usability fail. So here's hoping that now that Shoes 2 has been packaged as a .deb we'll see it in Ubuntu sooner rather than later. Maybe I should just make a new bug for it?

(I hate to say it, but I still have found the Gentoo bugzilla to be blazingly responsive and have fond memories of it. I wish other communities could learn from it, what ever it is they are doing right.)

GitHub Linking part 2


This is more involved explanation of Part 1:

The basics of what’s going on here is pretty simple. Take our original link:

http://github.com/soliver/vim-files/blob/56f54383deecebf113e8e828db0fb3792a331de0/filetype.vim

The first couple of parts is pretty simple

GitHub’s URL:
http://github.com/

Followed by the user:
soliver/

… the repo name:
vim-files/

… the type (This is basically what you’re looking at. If you’re looking at the repository itself it’ll say “tree”):
blob/

… the jibberish:
56f54383deecebf113e8e828db0fb3792a331de0/

If you look real close as “the jibberish” you’ll notice it’s simply the commit number. The problem then stems if you want to link to the latest version you obviously don’t want to link to an individual commit. If you do then if you ever update you’ll have to update your link too! So instead we replace the commit key with “master” which instructs it to goto the latest version.

and finally the file:
filetype.vim

I hope that helped!

Enjoy the Penguins!

How to link to GitHub


You’re probably thinking this is going to be the most retarded post ever because anyone can copy and paste out of the address bar in Firefox, but after a little experimentation I’ve discovered it’s not quite that easy.

I’ll use my own github site’s URL as an example. So for example, if you wanted to see my filetype.vim because you need to know how to make vim recognize those crazy .fal files for your falcon code. So you goto my github site and you find it so you can send it to your friends to because your gracious and actually credit me with showing you how. So you copy and paste the following link:

http://github.com/soliver/vim-files/blob/56f54383deecebf113e8e828db0fb3792a331de0/filetype.vim

Well, not only is that hideous and impossible to remember but your not even sure it’ll work for them and you like to show off like the pimp you really are. Well, if you take the above link remove the jibberish in the middle:

/56f54383deecebf113e8e828db0fb3792a331de0/

and replace it with:

/master/

so that you final link looks like:

http://github.com/soliver/vim-files/blob/master/filetype.vim

You can now easily link to any file you want from anyone’s github website.

Enjoy the Penguins!

Posts for Wednesday, August 19, 2009

Falcon Vim Syntax Update


I have been testing my vim syntax file for falcon since I posted about it and I have corrected a lot of issues with it. It now highlights single line comments. And it also no recognizes the preprocessor statement at the top of the script. So if you’re programming in falcon using vim I suggest you download my update. There other minor fixes to but they’re not really worth mentioning. Finally, I posted my falcon syntax file on Vim’s website so you can find in the official vim repositories though I can’t promise I’m going to update it there every time so I recommend you check github if you want a copy.

I’m working on creating an indent file for falcon as well. Indent files are more complicated though so it’s taking me longer to hash one out. In the mean time though I believe they do have one on the falcon svn though I do not have a link for it. I’ll post again once I get one worth publishing.

Enjoy the Penguins!

avatar

Fixing image distortion on websites using Firefox/Iceweasel 3.5 on Debian testing with intel xorg driver

Lately I noticed some image distortion appearing on some websites using my laptop with Debian squeeze. Menus on swiftfox did not appear as they should, some logos appeared out of their place and there were artifacts and other annoying things. For example Planet Gnome looked like this:
image-distortion
When using iceweasel 3.0.12 everything looked fine. Then I followed a guide to install Iceweasel 3.5 from experimental to my system. Images looked distorted again. So there must have been a problem with the latest xulrunner….

After some googling I bumped into Debian bug #491871 – [965GM EXA] display corruption with xulrunner 1.9. Following post #67 on that thread I was able to repair my xorg.conf to something that fixed the image distortion. Now Planet Gnome looks like this:
no-image-distortion

Some info:

# apt-cache policy iceweasel xserver-xorg-video-intel xulrunner-1.9.1
iceweasel:
Installed: 3.5.1-1
Candidate: 3.5.1-1
Version table:
*** 3.5.1-1 0
1 http://ftp.debian.org experimental/main Packages
100 /var/lib/dpkg/status
3.0.12-1 0
500 http://ftp.de.debian.org squeeze/main Packages
99 http://ftp.de.debian.org sid/main Packages
xserver-xorg-video-intel:
Installed: 2:2.3.2-2+lenny6
Candidate: 2:2.3.2-2+lenny6
Version table:
2:2.8.0-2 0
99 http://ftp.de.debian.org sid/main Packages
*** 2:2.3.2-2+lenny6 0
500 http://ftp.de.debian.org squeeze/main Packages
100 /var/lib/dpkg/status
xulrunner-1.9.1:
Installed: 1.9.1.1-2
Candidate: 1.9.1.1-2
Version table:
*** 1.9.1.1-2 0
1 http://ftp.debian.org experimental/main Packages
100 /var/lib/dpkg/status

Posts for Tuesday, August 18, 2009

Touchscreens suck

Touchscreens are all the rage: Most modern smartphones use them, surf-tablets or other appliances integrate them. They allow the developer (and sometimes the user) to customize the interface a lot more than what they are used to: Buttons are not mapped to actual buttons on a keyboard anymore so you can change their design or size any given time allowing really pretty interfaces. We even start to integrate touchscreens that can register more than one contact (so-called "multitouch") to trigger actions with certain complex hand gestures. Touchscreens are so neat. And they suck so bad.

Let me elaborate. We as human beings come with a set of features and limitations as well as a few basic principles that are hardwired into our brains. We do for example know that when we grab something we have to close our hand around it. That idea has found its way into our language, too: "grasp" for example means to actually grab a physical object as well as understanding something (as in grabbing it with you mind), the same phenomenon can be found in German ("begreifen") and Latin ("capere"), it seems to be a correlation that comes to us quite "naturally".

Now let us apply that idea to touchscreens. When I want to move an object in the real world, I grab it, I move my hand and I let the object go. With a conventional mouse I click (which is a movement similar to closing the hand if you look at it), I hold the button (I keep the hand closed), I move my hand and then I release the button (I "open my hand"). The way the mouse works is very similar to how the real-world process works. On a touchscreen I just point at the thing I want to move. I don't grab it, I don't even have to touch it "harder" or anything, I just touch. Then I move the finger and move the finger away from the screen. That is not natural at all.

Let's look at another thing: Typing. At home I use a Lenovo Thinkpad. I bought it for a bunch of reasons but one main reason was that the keyboard is awesome: The buttons have a really good feeling, I am never unsure whether a keypress registered or not. The resistance of the keys is pretty brilliant. And it makes sense because we have learned how buttons work since we were little children: You press the thingy, you hear a "click" or feel it and now you know that an action will happen (or if it doesn't you know that something is broken). If your button has no such haptic response people will be confused and check whether the button really worked all the time. Welcome to touchscreen land: Pressing a button either gives you no haptic feedback (it makes no difference whether you hit the button or the area next to it) or the device starts doing something weird (like vibrating for example, seriously, if my buttons at home would vibrate when I push them I'd go nuts). The reaction of the button to the activation is completely counterintuitive (especially considering that the screen usually still shows me that I "pushed the button down" when I obviously didn't. Typing on touchscreen interfaces kinda works for short texts but not for real typing due to the lack of feedback which leads to your brain being irritated and making you hit the surface harder than it has to. The resistance that real keys have is also a buffer that helps the brain to act properly.

Well now we have looked at two aspects and maybe you consider both of them invalid rendering my whole rant moot. Good thing that I kept an ace up my sleeve, the absolute killer argument for why touchscreens suck.

I don't know your hands, but here's one of mine. As you can see, I have rather slim fingers. I'm just saying that cause it is somewhat relevant to the next complaint: Touchscreens suck cause you have your bloody fingers in that area where the content you are trying to read is, covering it. I don't know who thought it was a good idea but it is the main complaint against touchscreens. Screen real estade is precious, especially on mobile devices where the amount of pixels is quite limited. Why would I now put my hand over the small area where the stuff I wanna see is? I don't know about you but my hand is quite opaque.

Touchscreens look nice and allow some real eye-candy but from a usuability standpoint and from a common sense standpoint they are completely retarded. And that, my friends and readers, is why touchscreens suck.

Now let's see if someone can bring some good arguments for touchscreens, post them into the comments.

ATH-AD700 Review

Recently I got my ATH-AD700 headphones. I've been FAR more excited than anyone has a right to be, waiting for these things to show up, like Christmas in August. Sweet, sweet anticipation. It was well worth the wait.

The only other headphones I have to compare these with are my Grado SR80's (which have really seen better days) and some Shure "noise-cancelling" earbuds which are nice but are not comparable to either. So I'll compare the AD700 to the SR80's. ATH-AD700's are pictured left, Grado SR80's are right.

Headphones Headphones

(Note: Nowhere in this article shall I refer to anything as "cans". I reserve the right to retain some level of self-righteous, snobbish disdain for the audiophile community.)

ATH-AD700 in two words: Freaking Huge.

One cannot understate how enormous the AD700's are. I thought the SR80's were big but the AD700's make me feel like a toddler. They literally engulf your face like the hand of a giant. If you have a tiny head you might have problems even keeping them on your head.

These are the kind of headphone that completely surround your ear rather than sit on your ear. With the AD700's I could probably fit 2 or 3 more ears into the cups along with mine.

There is no way you will wear these and not look completely ridiculous to those around you.

And yet, freaking comfortable.

In spite of their size, the AD700's are very light. They seem to be made of some kind of thin plastic with aluminum grated sides and a few metal finishing bits. They barely feel like anything when you put them on. I've worn them for many hours without discomfort.

And they feel wonderful. The pads are some kind of soft comfy velour fabric. These headphones are not manually adjustable; instead there are little 3-D flaps on top that auto-adjust on springs, and they seem to help equally distribute weight around your head so it isn't all bearing down directly on your ears. The lack of a proper "band" probably contributes to keeping them light. When you put the AD700's on, and you feel everything magically shift around to fit your head, it's a freakish (yet strangely entertaining) experience. I felt like a cyborg.

By comparison, you can't forget you're wearing SR80's. They are mostly metal and thick heavy plastic and they hurt after a half hour. The cups are hard plastic and the foam pads are oddly shaped so that your ear inevitably sits directly on the poky, scratchy plastic of the drivers. From the first day I owned the SR80's there was no mistaking that they were painful, and they've gotten far worse over time. I put up with the SR80's in spite of this because they sound great.

Which brings us to...

Sound quality

The AD700's really do sound awesome. I had my doubts how much different they'd be from my SR80's, but there is definitely a noticeable difference.

The AD700's are very detailed compared to the SR80's. The SR80's have an overwhelming amount of bass and it drowns out the vocals on a lot of my songs. I'd never noticed until I put on the AD700's and heard the difference.

My music of choice is metal, industrial, hard rock, soft rock, a bit of techno and J-pop, and they all sound great. I don't have to screw around with the equalizer settings on my MP3 player just to be able to hear the vocals clearly, as I sometimes did with the SR80's. The AD700's are probably what people call "neutral".

When I listened to one song of a live concert on the AD700's, I actually heard a police siren in the background as a cop car apparently drove down the street outside the concert hall. I'd listened to that song probably 50 times on my SR80's, and never heard that. There were actually many times this week when I was sitting in my office at work and heard what I thought was a sound behind me, and as I looked around trying to find what was making that noise, I realized it was in the music. It's a bit unnerving.

If your main criteria is bass, the SR80's are probably better. I thought I really liked bass to the exclusion of all else, but maybe I'm getting old or maybe my tastes are changing, because the bass on the AD700's is more than good enough for me. It's definitely weaker but it's also clearer.

Anything else I can say about these is going to be even more subjective and unhelpful than what I already wrote, but I think I do prefer the sound of the AD700's over the SR80's at this point. To be clear though, both of these headphones sound amazingly good and I was very happy with my SR80's for years and years. (The AD700's also have the advantage of being shiny and new and I'm sure this skews my opinion.)

Note that these are "open" headphones, so they will leak noise. People sitting next to you will hear your music. This isn't an issue for me but it may be for some.

Build quality

I won't be able to make a real comparison until I bang the AD700's around for four years in my briefcase like I did with my SR80's, but at a glance they certainly look and feel sturdy. Some of the ridiculous design flaws of the SR80's (like the ever-spinning cups that result in crimped and broken wires) are joyously absent in the AD700's. The headphone cord comes out of only one side of the headphones, which helps you not to feel like you're being strangled by two cords meeting under your chin as with the SR80's. The headphone wire itself is thinner than the SR80's but also feels more flexible and hopefully less likely to snap.

(The cord on both the SR80's is way too long, and I end up looping it and twist-tying it to avoid tripping over it or running it over in my office chair. But too long is better than too short.)

Even the box the AD700's came in was impressive. It had nice Japanese writing all over, and to open it was like unfolding origami.

Price

I got the AD700's for less than $80, new. The MSRP is supposedly $250. I don't know if I got an insanely good deal or if the MSRP is artificially inflated, but you can still get the AD700's on Amazon for around $80 if you look around.

This is $10-20 cheaper than Grado SR80's. I don't think the price difference is significant. I think both headphones are easily worth $80-100. Are they worth $250? Er... maybe not.

Conclusion

ATH-AD700: I love these things. I suggest, nay, demand that you buy them. They feel and sound very good. I am glad I didn't get replacement Grado SR80's as I originally planned.

I think it is easily worth spending $100 to get a "good" pair of headphones. Even if all you listen to is a crappy MP3 player, it makes a huge difference in how much you will enjoy your music. But I also use headphones when I'm at my computer, or even when I'm gaming. For me music is essential for avoiding distractions while programming, and these headphones are excellent for that purpose (especially because of the comfort).

The only bad thing about the AD700's is how ridiculous I look wearing novelty-sized, bright purple headphones in public. Personally, I will pay the price of bearing that shame.

Falcon Programming Language


I have discovered a new programming language called Falcon. Pretty nice little programming language. It’s written in C with a little C++ and is very confused about what kind of language it wants to  be. It’s pretty fun to play around because I won’t lie, I have no honest use for it. But all jokes aside I like it pretty well, I think I’ll replace all my ruby shenanigans with falcon. Not that I write a lot of code in Ruby either but when I want to play I play in Ruby land. But from now on I’m going to play in Falcon land where everything is written in C/C++ (not just part of it like Ruby) and scripts are compiled in binaries before they are actually ran. Pretty nice. The only downside to it is there is no vim syntax file for it that I can find. But fear not! I have written one… err I have started one. It’s not perfect by any means and I’m sure it leaves a lot out and it is probably (okay I know it’s broken) in several places. But that’s the beauty of open source. So please help me with it if you know anything about vim syntax files and have an interest in Falcon.

Falcon.vim

Which brings me to my final point for the post. I have finally broken down (or got off my ass, however you look at it) and created an account at github. Now you can see just how worthless an open source contributor I really am :) .

Enjoy the Penguins!

UPDATE: Nothing makes you look more like a tool then to have someone correct you the moment after you post to the world. A syntax file does exist for vim. I think I’m going to stick to mine but update the broke parts based on his (this is open source after all). The only place I know to find what I suppose is the “official” version  is on falcon’s svn:

svn://falconpl.org/falcon/docs/trunk/SyntaxHilight/vim

Posts for Monday, August 17, 2009

Dedicated Home Partition


Header

Intro

How to use the same home partition for reinstalling a distro or using a new distro but wanting to use the same home partition (i.e. preferences, Documents…)

Before Installing

Find the user and group id’s (uid and gid) on your current distro before reinstalling/adding new distro and write them down:

id

And the username of your regular user:

whoami

Get your home partition fstab entry:

grep home /etc/fstab

If your fstab uses a UUID, keep in mind that this will change if you change your partition map.

Note: If only wanted to change the permissions on the home partition to match a new install, see the final step.

Install

During the installation process the you will be asked about partitioning. You should not partition unless you know what you are doing. Some distro’s will safely allow resizing and adding new partitions. Some partitions tools can safely shrink, expanded, and add new partition. If you need a new partition or a reorganization of partitions consider adding gparted to the installer CD if it isn’t already and partitioning with that.

When you got your disk partitioned, start the installer and manually set the partitions you plan to use. Don’t use the dedicated home partition or the installer will likely erase it. Also during the install don’t use the username from your previous install, likely the installer will choose a different uid and gid so this is best not to, later matching id’s will be set. Finish installation and reboot.

After Reboot

Exit if automatically logged in and goto console Ctl+Alt+F1. Log in as root (or ‘su -‘ to root from regular user if on Ubuntu) and find out what the newly created user uid/gid is.

id <new-username>

Keep a note of the groups the distro added and also be sure the new user didn’t get the same uid as the one you already had.

Add a new user:

adduser

For username select your old username, for uid match the old one. If you’d like to prevent possible uid conflicts in the future, consider using a higher uid like 1050. Enter gid to match the one you are using on the new system. Then add the groups that match the user that the distro created.

You can also use the ‘useradd‘ command but I find the former easier. For example on my gentoo system:

useradd -d /home/user --uid 1050 -G adm,audio,cdrom,cdrw,fcron,portage,users,usb,video,wheel -s /bin/bash

Delete distro created user:

userdel <username>

And delete the folder in the home directory for that user:

rm -rf /home/*

This will delete everthing in the home folder (it is not sane to mount a partition on an folder containing contents.)

Add home partition to fstab

Add the home partition to be loaded at boot (if already not added). For example:

nano /etc/fstab ... /dev/sda5 /home ext4 defaults 0 1

Besure to enter the correct filesytem type and settings.

Now reboot and login to your new user.

Match your home partition to your new distro ids

Warning: If you’ve done the above you’re already done, don’t this.

Mount the home partition and change to the directory of the dedicated home partition:

mount /dev/<home-partition> cd /mnt/<home partition>

Then change the old user and group id’s to the new one:

find . -uid <old-uid-number> -gid <old-gid-number> -exec chown -h <username>:<usergroup> {} +

This will change permissions on all files/folders/links that have both the old uid number and gid number. Some (very few) files will not match but most programs will eventually write to them and update them. To update all file/folders/links:

find . -exec chown -h <username>:<usergroup> {} +
avatar

Random updates

Some fun with my Trac installs today. First of all, I converted them all to use the inherited .ini file, which avoids fun with “Why isn’t this change having any effect - oh, it’s not even looking at that!”

I also set up non-SSL access to the trac installs, and with a bit of rewrite magick, have it force logged in users to use SSL. This is actually pretty easy - set “secure_cookies=True” in the trac.ini, then setup the following rewrite rule in the non-SSL virtual host:

RewriteEngine On
RewriteRule ^/([^/]+)/login https://trac.allenjb.me.uk/$1/login [L]

Logging in now always redirects to the SSL address, and the cookies are only valid for SSL. (Don’t forget to clear any existing cookies)

For some strange reason, the Icekap trac instance had decided to randomly corrupt itself - it continually claimed it needed upgrading while trac-admin said it didn’t. I solved this by, in the trac installs directory:

mv icekap icekap.bak
../trac-create icekap "Icekap" nosvn
cp icekap.bak/db/trac.db icekap/db/
cp icekap.bak/conf/trac.ini icekap/conf/

trac-create is my own script which optionally creates an svn repo, then creates the trac instance for it - params are: directory “Project Name” [nosvn]

The PPF MediawikiRC plugin has been slowly progressing - it can now handle delete actions nicely, as well as marking minor changes and new articles. There’s probably lots it’s still missing (for example, user blocks is one thing I know it could have extra handling for), but I plan to implement these on a “when I can be bothered” basis now - it’s “good enough” for what I use it for at the moment and I’d rather move on to other projects.

links for 2009-08-17

Posts for Sunday, August 16, 2009

Adding Common Lisp to Apache Thrift

Well, with cl-pack sitting in a (hopefully) finished state, I'm turning my attention back to what got me started on it in the first place: attempting to write Lisp support for Apache Thrift. Thrift is the RPC framework Facebook uses internally and they open sourced it a bit ago, and it made its way into the Apache incubator where it resides now getting all kinds of attention. Several languages have been added to it, and its been generally cleaned up. After watching a video on Facebook architecture I got interested in Thrift. When I found out there was no Lisp support I figured I'd take a stab at it. Apparently others have tried but disappeared, so as to weather I'll finish, we can only wait and see but it seems like a good challenge and something I'd very much like to see done.

The digression onto cl-pack was a wonderful little trip. I learned a bit more about Lisp and lots more about packaging software for Common Lisp. It was a good little project to cut my teeth on and hopefully better prepare me to see this through.

So wish me luck, I'll probably need it for this larger undertaking. Approach #1 is reading the Ruby code and then writing similar CLOS Lisp code. It seems like a decent approach off the top of my head.

Moved to Linode

My web host for a good long while was Futurehosting. My OS was Debian 4.0 (Etch). Strike one: as of now there's still no option to upgrade to a newer version of Debian. Debian lags so much to begin with, it's really painful ify ou want to use anything released in the past two years.

I had an unmanaged VPS. I ran a bunch of funky non-standard stuff on there and it ran mostly OK. I had to upgrade to get more RAM just so SBCL would run on it, which sucked but I don't know that another host would've been any better.

The good thing about Futurehosting was that they responded very fast to tickets. The bad thing was the fact that I had ample opportunity to know this. The server would go down randomly once every month or two. I'd open a ticket saying "Hi my server is down", then things would be working again in a half hour, but why did this happen so often? I don't know. An awful lot of "failed switches". I wonder how often this happened without my knowing about it, given how often it happened in the middle of my using the server for something.

With all the hardware they were burning through I would've expected upgrades or price reductions over time, given that I was a steady customer for so long and that disk space and memory keeps becoming cheaper and cheaper in the world. But the prices always stayed the same, which was another strike.

Being hosted there was annoying but never annoying enough to switch. And migrating all of my sites and data to another server seemed like a huge pain. Momentum: the worst enemy of progress.

I moved to a new host on a whim recently: Linode. It was far less painful than I expected. Thanks to Linux and plaintext config files, it was mostly a SCP-it-all-over and tweak process. It took me one evening and a bit of time the next morning. Linode offers a lot of OSes which is also nice.

I pay less for Linode than I did at FH (and I get fewer resources at Linode, but I don't need much). Thus far I'm astonished how much faster things are running on the server. Even goofing off at a terminal, the shell is more responsive. My email loads instantly in kmail instead of lagging for a second. I never knew what I was missing. Linode's DNS control panel is also pretty braindead simple to use.

Futurehosting gets a C+ from me. It worked and my website existed, but it didn't knock my socks off. Hopefully Linode is better.

Install Lisp ASDF packages as a user with CLC

CLC or Common Lisp Controller is a system that other Lisp systems user to keep track of ASDF systems (a mouthful I know). By default, system packages are installed to /usr/share/common-lisp/ but what happens if you don't have root access but still want to leverage the ease of use CLC installed ASDF packages provide?

clc-register-user-package to the rescue! Create ~/.clc/source and put your ASDF package there, then simply run

$ clc-register-user-package ~/.clc/source/package/package.asd

and volia, you can (require :package) from any of your Lisp systems there after.

It's pretty awesome. :)

Posts for Saturday, August 15, 2009

Comcast blocks port 25

Testing a remote SMTP server is kind of hard when Comcast is secretly blocking all inbound and outbound traffic on port 25. I only figured it out when telnet wouldn't even work. I could've saved myself an hour of frustration if I'd known this. Thanks, Comcrap.

Windows Powershell: Can you handle the power?

MS Powershell is Microsoft's ripoff of Bash. I don't think this is a bad thing necessarily. Bash is a good tool and it's open source. If Windows bundled a sensible, full-fledged Bash and got rid of CMD.EXE I would dance for joy.

So Powershell lets you refer to your home directory as ~ and a bunch of commands have *nix aliases like ls and cat. This is nice for those who have *nix commands firmly internalized. You have to use .\foo.bat to run things in the current directory instead of just foo.bat, which I thought was cool.

But Powershell is not without its problems. For one thing I see this a lot:

The redirection operator '<' is not supported yet.

How hard is it to implement input redirection, really? For another thing, tab completion continues to be broken. When you hit Tab it still doesn't put a slash at the end of the text it inserts. You have to type a manual \ every time you hit Tab, to continue tabbing your way through directories. Thus doubling the number of keystrokes you're forced to type. This continues to drive me crazy. There's a small amount of evidence that someday it'll be fixed, but I'm not holding my breath.

You also usually can't bundle flags together. e.g. rm -rf would have to be rm -r -f in Powershell. This is just annoying enough to bother me, but I can look past it.

Sadly, Powershell also runs slower than a geriatric sea turtle. I don't understand what it's doing that takes 10-20 seconds to startup. Or why tab completion often lags for 5+ seconds itself.

My happiest surprise was when I tried to uninstall Powershell (so I could try version 2) and got this:

/random/powershell.png

This dialog listed every program installed on my computer (in random order) including every Windows Update I'd ever installed. The worst part is that I couldn't even dismiss this dialog as an error. For all I know, uninstalling Powershell could cause every program on my computer to stop working. I've seen stranger things happen in Windows.

A person walking past my office when I saw this would have heard the crazed, maniacal, tortured laughter that only the experience of being forced to use Windows can elicit.

Lightweight compositing in X

It might seem as if one needs a whole desktop environment in order to enjoy the luxury of compositing and its so called real transparency, but in fact this is not the case.

All one needs is to have X compiled with composite support (USE="xcomposite" in Gentoo) and two small X applications (both are in the official Portage tree):

  • xcompmgr — a lightweight compositing manager
  • transset-df — an application with which one can select the desired transparency levels (this is a patched/improved version of the original X.org's transset)

Although xcompmgr does not have all the nifty effects as KDE or GNOME or Compiz do, it does pack real transparent windows (as well as panels etc.), basic shadows and a few more tricks.

The man pages of these two are short and quite self-explenatory, so I find it obsolete to describe the usage here.
<!--break-->

cl-pack 0.2

cl-pack 0.2 is out and can be gotten at the regular place. I've packed a lot of new features into it. More data types supported, and a lot of new syntax and formating rules including grouping and '/' templates. Additionally some bugs were fixed including signed number support.

cl-pack covers most of Perl's pack's features ignoring as far as I'm concerned only weird esoteric stuff. Granted thats up to interpretation but I am very happy with what cl-pack supports and so I am pushing out 0.2 and calling it "feature complete" unless I hear from anyone soon needing anything or with bugs. If not I'll rerelease it in a bit as 1.0. It's a pretty solid piece of work IMHO.

Posts for Friday, August 14, 2009

Submit your Exherbo repositories

In Exherbo we employ a distributed development model where users can easily create their own repositories containing packages not otherwise available. Most users want to share their repository so other people can benefit from their work and quite often contribute updates and fixes in return.

But how do you find all these repositories? To solve that problem we've created the unavailable and unavailable-unofficial repositories. These two repositories index all the repositories Exherbo knows about (and is deemed of sufficient quality of course) and this makes it possible for paludis to know which repositories packages like powerpc-utils or bzflag are in.

All this is for nothing if you don't submit your repository for inclusion in unavailable* however so I'd like to remind people to submit their repositories. All you need to do is ask for it to be added in the #exherbo channel on freenode.

And you have a lot to gain by submitting your repository - here's just a short list of what you get in return:
- A reasonable thorough review of you repository with comments on things that needs improvement and explanations as needed
- The review usually teaches a lot of things about Exherbo and the exheres package format
- You get higher quality packages
- Other users can contribute to your packages in the future - you should expect updates, bugfixes and improved packaging from your fellow users
- Inclusion on Summer

So keep on creating new repositories and packages but make sure you submit them to unavailable-unofficial - there's lots of people that wants to help with your packages.

Backup around firewalls with ssh and rsync to encrypted destinations

I decided I really needed to work on a server backup system, so here are my notes on the system I have now.

First things first, the files I want to backup are owned by all different users, so the only user who can run the backup process is root. Therefore I can't just run rsync from my local machine and grab the files from the server, the backup process has to be run on the server and backup to the backup machine. Now in my case this was a bit of a trick because the backup machine was behind a firewall, so the server had no direct line of communication to it. So I wrote a script to turn on a reverse ssh port forward.

recv-serv-backup.sh

 
#!/bin/sh
 
ssh -R 8000:127.0.0.1:22 -N user@kvasir.mindstab.net
 

When run on the backup machine behind a firewall, it connects to the server (kvasir) and listens on port 8000. When ssh on kvasir connects to port 8000 it redirects that traffic to local port 22, the ssh port of the backup machine. This is how the firewall is gotten around. Reverse port mapping is a cool trick to master.

Next as root on kvasir I generated a public ssh key and put it on the backup machine so root could automatically log on repeatedly to the back up machine (think lots of rsync calls) once the key was loaded once. Then I hooked up the key to keychain. That is all better outlined at:
Gentoo Linux Documentation: OpenSSH key management, Part 1 and
Gentoo Linux Documentation: OpenSSH key management, Part 2.

The all I needed to do was poke the rsync syntax to use the nonstandard ssh port for backup. The best method I found was

rsync -e "ssh -p 8000" -av

rsync for those of you who don't know is a great little backup tools. It's like a smart (in that it only copies files that have been modified since the last backup) network aware (since it can use ssh) copy tool. Simple but really useful.

So with that I wrote a backup script on the server

server-backup.sh

 
#!/bin/sh
 
backup () {
        echo "$1..."
        DST=`echo "$1" | awk '{split($0,a,"/");  result = "/"; for (i=2 ; i < length(a)  ; i++)   result = result "/" a[i];  print result;  };'`
        rsync -e "ssh -p 8000" -acv $1 dan@127.0.0.1:~/kvasir$DST ;
 
}
 
backup "/svn"
backup "/git"
...
 

There is one caveat, rsync won't create subdirectories on the other side specified in the path so you need to create the basic directory structure.

 rsync -e "ssh -p 8000" -av /git user@127.0.0.1:~/kvasir/git

is fine because it will create /git just fine, but

 rsync -e "ssh -p 8000" -av /home/user user@127.0.0.1:~/kvasir/home/user

will fail if ~/kvasir/home doesn't exist. So you'll need to create the basic directory structure or enhance the backup function to strip out extra directories in the target path.

Finally, I didn't want anyone and everyone to potentially be able to gain access to private data on the backup machine, so the target directory needed to be encrypted. There are a lot of options, but I opted for the easy encFS route and just installed "cryptkeeper" and had it setup the directory.

Now all I have to do is mount the encrypted backup directory, run the script to turn on the reverse ssh tunnel, and run the backup script, and I have an encrypted backup solution for my server that gets around firewalls.

Not bad.

References

    Posts for Thursday, August 13, 2009

    avatar

    Open Source Days 2009: Talk Suggestions

    Peter Toft has made an announcement on his weblog about Open Source Days 2009. Open Source Days is the largest Danish open source conference that targets both developers and end-users. This year the conference will take place the 24’th of October. It is only going to be a “small” community conference and then there will be a larger conference in the beginning of 2010. I am already very excited about this conference and I am looking forward to meet a lot of Danish open source people.

    Peter Toft also suggested that people should let the coordination team know which talks they would like to hear about. Here are my suggestions:

    • Julia Lawall from DIKU about Coccinelle. Julia did perhaps the most interesting talk at Open Source Days 2008 about her research on semantic patches and how they are used. I would like to hear what has happened since then, especially regarding the focus on bug hunting using Coccinelle.

    • Bryan Østergaard about open source project management. Bryan is the leader of the Exherbo project and used to work on Gentoo before finally starting his own project. Bryan has written an interesting blogpost about how to manage an open source project the right way and getting more contributors and more useful contributions. It would be interesting to hear more about this as a talk at the conference.

    • Jesper Louis Andersen and I are working on a Git tutorial spread out over three workshops and we are planning to present them during the winter at SSLUG’s location in Frederiksberg. We could do a one hour talk at Open Source Days, and I am sure a lot of people would find it interesting to hear.

    • A lightning talk about communities on IRC. This is not super important, but if someone found it interesting I would be more than willing to do it.

    Looking forward to see you at the conference. Remember to follow the Zebrapig!

    Planet Larry is not officially affiliated with Gentoo Linux. Original artwork and logos copyright Gentoo Foundation. Yadda, yadda, yadda.