Planet Larry

I got it from my sister’s digicam. Thanks to Leerz for the walkthrough. This worm is really annoying especially if you are more comfortable doing stuffs in the console.

1. Check for any bar311.exe, Autorun.inf, pc-off.bat files in mounted drives.
2. Delete if found.
3. Edit the following entries in the registry.

HKCU\Software\Microsoft\Command Processor\"Autorun"
HKLM\Software\Microsoft\Command Processor\"Autorun"

Alternatively, you can download Noob.Killer run it, then watch and learn.

I symlinked my .vimrc to my local mirror of my website so that every time I rsync it (which is pretty often) it'll automatically update my the vimrc on this server. So that should be fun. I experiment with things in there all the time so at any given moment there are likely to be things horribly broken, but maybe someone can use some of it.

This mirror of Ciaran McCreesh's vimrc which I found linked from here (edit: updated version here) has lots of good stuff in it. In particular using :set listchars to display tabs and trailing whitespace as some funky Unicode characters is a really good idea. When I first tried that good idea I realized my favorite font ProggySquare didn't properly display most Unicode characters, which was part of my motivation to switch to Terminus. (That, and those tiny Proggy fonts aren't so great on a 1920x1200 monitor.)

After a long time putting it off, I finally hunkered down one day and figured out how the heck Vim script works. The difference between statements and expressions in Vim script language confused me for a while, which goes to show that I'm far too used to Ruby and Lisp where almost everything or everything returns a value as an expression. Vim expects expressions in certain places and colon-prefixed commands in others. But then there's normal and eval and execute and "= some of which let you do things from one mode in another mode if you mix and match them. But I think I've gotten a handle on it now.

Today I came across Limp which is a recent attempt to get Lisp to work well with Vim. It seems quite new and buggy and had dependencies on things I had to guess until I was able to install it (like rlwrap), but I still was excited about it. Until I realized that it's just a wrapper around GNU screen. SBCL runs separately, and some keystrokes send stuff from Vim to screen, but that's about it. Nice, but not nearly as nice as SLIME in Emacs. So that disappointed me. In the back of my mind I always think about how Vim could possibly be integrated with Lisp like SLIME does but I don't see any good way. Vim doesn't have the ability to embed shells like Emacs and it doesn't look like it will gain that ability any time soon. Ah well.

By some people’s logic, this how the economy is supposed to work:

1. New companies emerge all the time.
2. No companies ever close.
3. Consumers always buy the cheaper and better products.
4. No products ever become obsoleted and force the company to go out of business.

Sounds perfectly reasonable, doesn’t it?

When a new company opens in a town and provides a thousand new jobs, there’s noone protesting that this is unfair, we didn’t do anything to deserve this, that you can’t just suddenly create new jobs out of nothing, there aren’t people complaining that it’s not right, we didn’t get jobs at the new company. No, people accept it with great fanfare. Great, the economy is growing, our town will prosper! People will have more money, there’ll be less unemployment, we’ll be able to afford a higher standard of living.

And yet when, after 40 years, the company goes out of business or moves their production to a cheaper location, people say this is outrageous, 1000 jobs will be lost! There’s anger and pandemonium, how can they do this to us, we were loyal to the company for 40 years. People appeal to some sort of higher ethical body; you can’t take our livelihood away, what are we going to do with ourselves? And the town itself, which never had much industry, and really just had that one company that employed everyone in town, starts to regress. People move out in search of jobs, young people leave and don’t come back, noone moves in because there’s no local economy.

It’s a sensitive topic. Losing your livelihood is one of the more challenging life situations. But before you start screaming that it’s those damn crooked politicians and those greedy executives that have stolen your life, take a moment to think about why you had that job in the first place. In fact, let’s start with the basics: what does it mean to have a job?

It means that you are producing a product or offering a service that someone is willing to buy. It does not mean any of these things:

1. Someone is being nice to you.
2. You deserve this.
3. You’re going to keep your job because you’ve been loyal to the company.

If you actually believed any of that then you were under a complete misapprehension. Sure, sentimental concerns do come into it sometimes, like the boss’s son getting a summer job because he’s family. But in the long run, the only thing that matters is the economic reality.

If you think that’s a raw deal, think about this. Most artists aren’t wealthy, in fact most artists are struggling to get enough work to live on. A painter may think that he deserves to live a decent life as a painter, but if noone is willing to buy his work, well he’s not going to. Is that unfair? No, it isn’t, because if he’s not producing anything of value, why should anyone have to pay to keep him in business? So if a painter can’t do the job he wants to, why should it be any different for you making shoes, or catching fish, or whatever it is you do?

There used to be people working in elevators who would press the buttons. We don’t need them anymore. Shepherds aren’t in great demand either. Neither are telegraph operators. These professions have all be superseded and they’re not coming back. Many others still exist, but have been moved to where production is cheaper, like textiles.

It’s always a turbulent transition, you can be sure of that. We don’t have hunters anymore, we have domesticated animals now, no need to chase them in the woods. Think about how many hunters were out of work when this happened. But what should they have done, lynch the guy who came up with the idea of keeping animals on the property? Compared to the hunters’ relatively narrow interests (although there were many of them), domestic animals were very beneficial to the village. For one thing, you didn’t wonder where dinner was coming from, the animals were right there. So should the villagers have discarded this new idea just to make sure the hunters could keep their jobs?

I’ve got news for you. The very same thing you’re protesting against, your job being taken away, you’re doing the same thing to people everyday. That’s right, you’re not so innocent yourself. Have you ever bought a car from a different automaker, because it was cheaper? Did you ever buy peaches from Spain instead of domestic apples? Well, I’m sure it must have been a very gruelling decision for you, right? I mean to think that you could be putting car makers and farmers out of business because you’re not buying their products, that’s a tough one to swallow.

And what did you get out of it? You could afford to buy more things, because the new products were cheaper. And they didn’t break as quick, so you could use them longer. And they had some functions that the old products didn’t have, which made you happy. And just as this was happening, the old companies that couldn’t stay competitive were going out of business one by one, people were losing their jobs. But hey, you got a pretty good deal out of it, didn’t you?

Here’s what it comes down to. You’re not entitled to your job. You’ll only have it for as long as people are willing to buy your product. And even if you’ve had it for 40 years, that doesn’t mean the global market won’t make it obsolete tomorrow. There was a demand for your product, now there isn’t. You didn’t do anything to deserve getting it, and you didn’t do anything to deserve losing it.

Debian and Ubuntu are not random enough

There is has been a bug in random number generator on Debian (from Etch onwards) or Ubuntu (Feisty onwards). You should already have a security update for the number generator. If you have not yet accepted the update then do so.

If you are on an operating system that has apt-get then you probably want to look at what is going on. If you are on Gentoo or another distribution then for now you can just smile quietly to yourself.

In theory, bugs in the number generator are bad mojo because there are less combinations, depending on the severity of the bug, it makes a brute-force attack go from almost completely impossible, to either still very improbable down to theoretically possible with a supercomputer. SSH is often the first point of entry to a Linux machine (but not the last line of defense) so bugs here are particularly prominent.

However, lets not have a panic attack about it. There are a dozen easier ways to get into someone's machine. In proprietary software land, they probably would have just ignored this kind of theoretical exploit to keep their marketing team happy. For a proprietary software company, still existing in five years time is a higher priority than a theoretical brute-force attack using hardware of the future. Free/Open Source Software forces good security, your dirty laundry is washed in public. Today's theoretically possible attacks are tomorrow's malware. If we ignore all these things then we end up with an operating system akin to Windows.

If you are on Debian or Ubuntu, the security updates means that any new keys will be to the desired level of randomness, but your existing ones need to be ditched. The update manager does not do this for you in case you are then left unable to log into remote systems.

Swapping out your SSH keys

Cleaning this up is easy. Run:

sudo ssh-vulnkey -a

This outputs a line for each SSH key on your system:

Not blacklisted: 2048 <key fingerprint> <filename>
Not blacklisted: 1024 <key fingerprint> <filename>
COMPROMISED: 2048 <key fingerprint> <filename>
COMPROMISED: 2048 <key fingerprint> <filename>
Not blacklisted: 2048 <key fingerprint> <filename>

So the ones that came from Gentoo or another Linux distribution are okay as far as we know. The two Ubuntu ones we must delete or archive somewhere else. To delete the keys use rm.

Now we might like to generate replacements, so we can still use SSH as before:

ssh-keygen

So to make this simpler, one of the lines was: COMPROMISED: 2048 49:37:38:f4:86:28:ac:b1:7e:a6:df:bd:1d:a4:da:81 /home/warrior/.ssh/id_rsa.pub

That is the public key of the local machine. So we get rid of it:

rm /home/warrior/.ssh/id_rsa rm /home/warrior/.ssh/id_rsa.pub

Now we want a new one:

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/warrior/.ssh/id_rsa):

The brackets mean that is the default, so I press enter.

Next it finds an existing key (the private half of the existing keypair);

/home/warrior/.ssh/id_rsa already exists.
Overwrite (y/n)?

We want to overwrite it so we say we yes.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/warrior/.ssh/id_rsa.
Your public key has been saved in /home/warrior/.ssh/id_rsa.pub.

Now we are done.

1. Build a cheap and ultra portable machine that looks cool but that does not have everything in. Don't compromise too much to get millions of hardware features. A 10" display with 1000x600-ish resolution, a processor that gets the browser running at an OK speed, a very good touchpad. 1GB of RAM so you don't need swap to run. Only very little solid state harddrive.


2. Set up a service that is connected into every aspect of the machine: Allow people to have their "home directory" on your server for backups. Let them pay for it. Allow people to buy all kinds of neat services. Don't get greedy by trying to lock them into one operating system or platform but offer benefits: Build a service that allows people to have their pidgin conversation logs on your server and guarantee backups. Guarantee encryption for their personal data. Do similar stuff for other software (while still keeping the base system free of charge and updates cost-free, too).

If you have signed up to my site's RSS feed, please update to http://commandline.org.uk/feeds/full/ as soon as possible. Then you will be sure not to miss any of my exciting adventures.

This main feed should work the same as the old one, however, I have provided a number of extra feed options. if that is your bag.

Update: I am hoping all the old feeds should do something now.

I took a few minutes today and cleaned up a few small bugs on Planet Larry and friends.

One thing I get asked for every now and then is if I have archives of past posts. Well, I do now. I just copy the HTML file of the last post to $date one each run. A simple and unelegant solution. I’ll be doing the same thing for Planet Gentoo soon.

Also, fixed the FeedBurner link on the main page — I didn’t even realize it was broken. While I was at it, I created one for Universe as well.

On the packages website, I finally fixed it so you can search against just packages again. That’s been annoying me for a while. By default the search is way too wide, I think. It will search the full atom, the package description and the package name. I have to do the package name twice because of regular expressions (starting with, ending with, exact matches, etc.). And there’s still no simple way to search for packages containing multiple words, which is also an annoying little bug. Advanced searches for GPNL and Packages has been something I’ve wanted to do for a long time, but have been putting off since I started the projects. Sheesh. Every time I sit down and start to poke at it, though, I realize just how big a beast it is, based on what I’d like to accomplish. I really need something for the interim, though.

Anyway, I better quit before this post gets any more boring. One last thing — we can use more users who are Gentoo users and have a blog on Planet Larry. Just drop me an e-mail and I’ll get you setup.

or how not to serve robots.txt with PyBlosxom

So as you may have noticed, I moved this site from PyBlosxom to Django, which depending on your perspective is a fabulous thing to do or is tantamount to treason on the high seas. I will explain more about that later.

Old links to the site should, in the main, still work hopefully as I have done some regular expressions jujitsu which should hopefully send everyone to where they were supposed to be going.

However, some posts and comments will have their formatting up the creek. So I want the old version of the site to be available (at archive.commandline.org.uk) for a while longer.

Because the archived version is deprecated and on the way out, I do not want the search engines to index it. Therefore I needed to make a robots.txt file for that subdomain excluding them from indexing it.

The last version of this site, like many dynamic sites, is composed of a number of layers, part of which was a lot of my own nonsense code doing various things. Ignoring that, when a request for a packet came in it would go to WSGI which would then pass the request on to PyBlosxom which was at the bottom of it all doing the hard work.

To deploy it properly, one would normally put Apache at the front as well, but I never got around to that. In theory this is a bad thing to do. But in practice it worked really well without the huge and complicated server that is Apache in the mix. It actually ran fine for a year without stopping, and blazing fast too; if it also confused a few comment spam bots then all the merrier.

So I tried putting Apache into the mix so I could use a Location directive to direct /robots.txt to somewhere with the robots.txt file, but no joy, this would have required doing a lot of what I never got around to before.

So I then looked into how the test server was deploying the site, thinking that I could do some kind of smart regular expressions type matching like in Django or Pylons. But nope.

Hack for the win

So the next step down is PyBlosxom, so I looked out of chance in Pyblosxom/pyblosxom.py and saw the following:

def __call__(self, env, start_response):
    """
    Runs the WSGI app.
    """
    # ensure that PATH_INFO exists. a few plugins break if this is
    # missing.
    if "PATH_INFO" not in env:
        env["PATH_INFO"] = ""

    p = PyBlosxom(self.config, env)
    p.run()

    pyresponse = p.getResponse()
    start_response(pyresponse.status, list(pyresponse.headers.items()))
    pyresponse.seek(0)
    return [pyresponse.read()]

Bingo! As soon as I saw it, I just somehow, on auto pilot, typed in the following lines before the line p = PyBlosxom(self.config, env):

if env["PATH_INFO"] == "/robots.txt":
    start_response('200 OK', [('Content-type','text/plain')])
    return ["""User-agent: * \nDisallow: /"""]

And unbelievably it worked. What I had subconsciously done was to see that we have some kind of string referred to by env["PATH_INFO"]. Then further on we have an object called start_response which is being passed a status and some headers. Then we are returning the response.

I was kidding around more than anything so I just replaced everything I didn't know about with reasonable looking constants (you will know these well if you have ever done Python CGI programming).

I am sure there are millions of far better ways to serve robots.txt with PyBlosxom. But this hack works for me until I no longer need the old site anymore.

Mughlai and Jalfrezi are better than gruel

A few hundred years ago, the great mass of the British poor ate gruel, while the middle class ate bland over-boiled vegetables. However, as a naval people, the British went out around the world with their empire, and brought food and foreign chefs back with them. Now British people can and do eat food originating from the whole world. Not just the rich, normal working class people will regularly eat curry, Cantonese food, kebabs and so on that would not have been imaginable before.

If I tried to explain this to a 14th Century peasant eating his gruel, then he probably would just ignore me, having no idea what I am talking about. At best he might look at me strangely, and then go back to his life of gruel.

Windows is the gruel of the digital world. There are certain people that understand this fact and have moved on to greater and better things, however most people take what they are given and swallow it as best they can.

What is an operating system?

If you don't know what a computer program is, think about cooking. In cooking you have tools, such as an oven and a blender and you have ingredients such as vegetables and meat. The recipe allows you to use the tools to take your food (ingredients) and turn them into to other types of food (meals).

In computers, you have tools (called hardware) such as a DVD player, a screen, a keyboard, a hard drive and so on. The computer program is the mathematical recipe that allows you to take your data (text, pictures, videos, etc) and then do various things with it. So for example, you might use a computer program that takes a song from the hard-drive and then plays it out of the speakers.

An operating system is a set of computer programs that makes your computer hardware do the basic things (put text on the screen, play sound, and so on). You might then run other programs to do more advanced things. It is like your recipe book.

People who are in to cooking will try out lots of recipe books, and in doing so, they do not starve because they have bought a different recipe book, indeed the opposite happens, they cook so much that they do not have time to eat it all themselves so have to give food to their family and neighbours.

By changing recipe book, they don't suddenly become unable to cook, they in fact get better as they move on to better and more advanced recipe books.

If you turn on you computer and see Windows, then Windows is your current 'operating system'. If you have only ever used Windows, don't you think it is time to give up the gruel and try a new recipe book?

I think so, and if you think this way then you have come to the right place!

About Windows

Windows started on home computers and was commonly used for playing computer games. Though some shifty business deals in the 1980s and 1990s, Windows became pre-installed on the PC and so became the main operating system used by non-technical users.

However, there a lot of people that think that this situation is not good for humanity and we need to progress past it.

Why? Well lets look at some of the reasons.

Firstly, Windows does not promote a competitive industry. Only Microsoft can sell Windows, only Microsoft can really provide complete support for Windows. Mainstream PC shops may only stock Windows PCs.

So you have one company earning billions of their monopoly, with these excess profits, Microsoft can then give campaign contributions to politicians to make sure they don't make the industry competitive or hold Microsoft to account when they break the law.

Weak, bribed, politicians allow Microsoft to use the educational system as a giant marketing tool, indoctrinating a new generation to become helpless and passive recipients of Microsoft's, and only Microsoft's, products.

Secondly, each version of Windows is developed in secret, and then launched with billions of dollars worth of marketing to make you believe the magic; however like all magic, it is no replacement for public peer review. Microsoft don't like public peer review because they know that when compared fairly to other operating systems, Windows always loses.

The fact that there is no public peer review, and no effective competitive pressures, means that Windows is not very well engineered. When the main architecture of DOS and Windows was created in the 80s, it was already 20 years behind the state of computer science; and it has not really changed that much since.

This 'closed-off from the world in my own cave' approach to software engineering means that Windows is plagued with security problems, it uses computer resources inefficiently, wasting electricity and requiring unnecessary replacement of perfectly fine computers that could have lasted another five to ten years.

Fourthly, a software mono-culture, like a biological monoculture, is not very healthy. If a future Windows virus wipes out all of the world's Windows PCs, then 90% of the computer using population are offline, without their data and without access to government services, Internet commerce and digital information. Businesses would collapse and the western world would be plunged into a digital dark age.

Fourthly, because what Windows is doing is a secret; if you use Windows, then you are not in control of your computer, Microsoft is. Windows reports back lots of data to the USA which is then made available to whomever Microsoft wants to share it with. While most of us are not interesting to the US security agencies; Microsoft can sell your private information to anyone.

In short, Windows leaves your backdoor open to Microsoft, but even if you trust Microsoft, the US government, and all companies that Microsoft might sell your information too; the fact there are built-in backdoors means that anyone, criminals, terrorists, anyone, can potentially walk through Microsoft's backdoor to access your private data or install viruses or tracking software on your PC.

There is another way...

...Indeed there are lots of them! The opposite to Windows slavery is software freedom. And with freedom comes lots of choices, and choices are good! If you have spent a lifetime eating gruel then you might resent choice, but then remember the intolerant character in Dr Seuss' classic "Green Eggs and Ham", who resists and resists trying out new things for unjustified reasons.

The operating system I currently use is GNU/Linux (commonly just called Linux), which started out in Universities and parts were contributed by thousands of volunteers over the world wide web; others soon joined in, such as small and large companies, charities and even the American military.

Unlike Windows, any company or individual can share, sell, give away or provide services for GNU/Linux, anyone can change it, and there is complete public peer review. There are no hidden traps and you are in control of your own computer.

Also the way Linux executes programs is based on a completely different architecture. There is no concept of an untrusted, unknown program having access to everything. The problems that plague Windows, viruses, spywhere, malware and worms, do not exist in the Linux world. They have never existed and will never exist, because the architecture of the system is not designed that way.

So as I said before, anyone can give out Linux, so lots of people do (remember: choice is good) most versions are free and you can legally share them with your friends and neighbours without having to ask anyone.

Give it a go!

Vor ungefaehr 2 Wochen war auf einmal eine Festplatte mit 80GB nicht mehr ansprechbar. Ich konnte das Problem mit dem Austausch des IDE-Kabels beheben. Die Platte war kurze Zeit spaeter wieder nicht ansprechbar und liess sich auch nicht mehr zum ansprechen ueberreden. Jetzt ist mir auch noch eine zweite 80er Platte kaputt gegangen. Jaja, ich weiss, sie waren alt, und auch die letzten 5 Jahre im Dauereinsatz…

Dennoch ist es aergerlich fuer mich auf einen Schlag zwei Platten zu verlieren, die ich u.a. dafuer genutzt habe meine Backups drauf zu schieben.

Ich moechte eigentlich ungern wieder feste Platten in einen Rechner verbauen. Externe finde ich da irgendwie praktischer. Schon lange liebaeugel ich mehr oder weniger mit einem Linksys NSLU2 und bei den aktuellen Plattenpreisen… Ich stell mir das gerade vor mit z.B. zwei 500GB Platten von WD (zum Beispiel diese hier). Ich finde das Nett! Das sind mal schnell zusammengefasst 1TB Speicher verfuegbar im Netzwerk incl. eines sehr netten Geek-Toys fuer ~225€…

Bevor ich mir jedoch ernsthaft Gedanken ueber eine evtl. Finanzierung mache, moechte ich wissen, ob das Ding auch von der Hardwareleistung her vernuenftig Streamen kann. Interessant ist es ja fuer mich z.B. im Zusammenspiel mit meinem XBMC. Laut Aussagen in Foren (z.B. da) geht das… Noch weitere Stimmen?

I look at Vim 7 or 8 hours a day, so it's nice if the colors don't give me a headache. I've used ps_color for years but recently I decided it's a bit too washed-out and it has some quirks that make it hard to read Ruby code. It's hard to find anything else that's any better though. inkpot is good but it's a bit too monochrome for me. I like things to have a very distinct hue rather than rely on saturation or subtle differences.

So I started writing my own color scheme. For some reason that's beyond me, I seem to gravitate toward purple and green. Green is my favorite color, but why purple? I think it might be due to Gentoo brainwashing, so I called this color scheme Gentooish. I've been using it for a week or so and I keep changing things that annoy me, which will probably continue, but it's non-sucky enough to upload at this point probably.

Download gentooish.vim.

I've never written a color scheme before, but it's not difficult. inkpot had nice clean source code so I used that as a basis. ps_color's source is horrific.

Sadly I'm not 100% sure how vim color schemes map to colors in a terminal. Konsole, urxvt, xterm, and a real terminal all show me different colors when using the same color scheme. So I didn't bother with it.

…und die auswirkungen der bodennahen sonnenstrahlen auf die psyche der zentraleuropäischen hochgebirgsbewohnenden landbevölkerung

klingt ja schon mal interessant…

könnte auch interessant werden, sofern man es wagt, sich bei den aktuell herrschenden wetterbedingungen dem kontakt mit der angeführten bewohnerschaft des mittleren und unteren rheintales sowie der bevölkerung des vorderen bregenzerwaldes auszusetzen…

nun, ich tat es und ich tu es nach wie vor. warum? versuch dir mal einen biker im wohnzimmer vorzustellen! geht nicht? nein? na dann wirst ihn wohl auf der strasse antreffen…

so, und jetzt nach diesem kurzen exkurs ins reich der phantasie zurück zum thema: das wetter, die damit verbundenen physikalischen realitäten und der mensch des unteren, mittleren und des oberen rheintales sowie des vorderen bregenzerwaldes. ach ja, und die touristen, eigentlich, so wie sie sich verhalten, besser als terroristen bezeichnet, die darf man auch nicht vergessen!

nun, was zeigt uns das alltägliche leben? mit steigender temperatur steigt der rocksaum der jungen zentraleuropäerinnen. ob das nun positiv oder negativ zu werten ist, sei voerst einmal aus der betrachtung ausgeklammert. eine folge des rocksaumanstiegs ist auf jeden fall der anstieg des hormonspiegels der dazugehörigen, jungen und vorzugsweise männlichen hochgebirgslandjäger. daraus folgt naturgegebener maßen ein drastischer abfall der hemmschwelle, einhergehend mit einem ebenso drastischen schwund der zurechnungsfähigkeit. das ist relativ leicht am dauernd geöffneten mund eines jungen, vorzugsweise männlichen gebirgseuropäers.

klingt nun, als wären davon nur junge alpenhirschen betroffen, aber nein, bei abhandengekommenseins der angetrauten senn- und alpbewohnenden weiblichen bevölkerung tritt dieses phänomän der offenen klapparatur auch bei alpendjangos der gehobeneren altersklasse ebenfalls auf. das geht soweit, bis der offene mund und der gekrümmte rücken mit hilfe von sogenannten oder auch heilbehelfen gestützt werden muß. dann erst kann dieses phänomän nicht mehr beobachtet werden. scheint, als ob das sture auf-überbreite-gürtel-glotzen in direktem zusammenhang mit dem sinn- und unsinnlosen öffnen der klapparatur steht.

zurück zur höhe des rocksaums und der positiven oder negativen bewertung: auch hier konnten schon diverse beobachtugnen gemacht werden, die zwar unlogisch, aber in der natur immer häufiger vorkommen und demzufolge schon die wandlung von der these zur theorie vollzogen haben.

die höhe des rocksaums ist quadratisch proportional zum durchmesser der beine, die darunter hervor hängen. das bedeutet, de größer der durchmesser der beine ist, desto höher, und das sogar quadratisch!, ist der rocksaum angeordnet.

was heißt das nun? also, schlanke beine werden sehr häufig durch jeans oder knöchellange beinkleider anderer macharten versteckt und damit regen die beinträgerinnen die phantasie der männlichen alpenhirschen an, während bei kurzen, sehr groß bemessenen durchmesserstarken beinen grundsätzlich möglichst nahe an den scharnieren der rocksaum zu finden ist. eine drehbewegung der männlichen alpendjango-köpfe ist sehr oft die folge. allerdings nicht hin zu diesen beinen, sondern weg! man könnte dies auch mit fluchtinstinkten umschreiben.

welche der beiden varianten nun als positiv und welche als negativ bezeichnet werden kann, ist nun schwer zu definieren. die weiblichen alpendjangos und terroristinnen haben da sehr häufig gegenteilige ansichten als die männlichen exemplare.

wie sieht das nun mit einem erkennbaren defizit an gehirnbetätigung aus? und woran erkennt man so ein defizit an einem freilaufenden alpeneuropäer oder terroristen?

nun, das ist recht einfach: sogenannte importeuropäer, erkennbar an gelackter kopfbehaarung in dunklerer ausführung, drehen allen beinen den kopf nach. egal, wie sie aussehen und wo der rocksaum zu finden ist, hauptsache, es handelt sich um eine beinträgerin! nach erfolgter eroberung wird diese dann mittels kopftuch und bettwäsche vor der umwelt geschützt. oder die umwelt wird vor ihr geschützt, diesbezüglich gibt es keine gezielten beobachtungsergebnisse. anders ist es bei den ursptrünglichen alpeneuropäern: diese beginnen bei anblick eines massiv-gebirgstauglichen zentnerbeines mit den augenbrauen zu zucken und noch undeutlicher zu lallen als unter dem gewöhnlichen alkoholeinfluß. und schlußendlich die erkennbarkeit bei terroristen, die ist sehr einfach: zusammenkommen von sauerstoffmangel und frei sichtbaren beinen jeglicher form und jeglichen formates lösen plötzliches fallenlassen von müll jeder art und gleichzeitigem zücken eines fotoapparates an allen möglichen und unmöglichen lokalitäten aus. blitz und bruzzel und feddisch issa, der olle touri…

gibt es sonst noch auffälligkeiten zur herrschenden großwetterlage, dem drohenden stauwochenende und den europäern? klar! europäer werden in blechdosen hingekarrt, wohin auch immer asphaltbänder führen, vorzugsweise zu straßenräubern des 21. jahrhunderts, den italienern. auf dem weg in deren reich werden, soweit es nur irgendwie machbar ist, alle kreuzungen verstopft, klaustrophobieanfälle und kreislaufkollapse en masse und dazu weißwürste und weißbier, was auch immer das sein soll. und ist dieses nicht verfügbar, ein hefeweizen tut’s auch… und die angestammte alpenbewohnerschaft dreht am rad…

und ist der spuk vorbei, bleiben einige kfz-mechaniker auf ihren rechnungen und die landbevölkerung auf zigarettenstummeln sitzen… und die filmrollen in den mittelalterlichen radargeräten müssen ausgewechselt und schöner erinnerungsfotos ausgearbeitet werden.

fazit: touristen, oder auch terroristen, sind mindestens so bradngefährlich wie elefanten in miniröcken… und alles dreht sich im kreise…

Ok, so I know we announced it at around Christmas, but *now* the Mindstab AI Go Competition is starting to get under way. Both Rob and I now have entries that can compete, though mine is mostly just an over engineered random bot, until I get even more framework in place.
Still, everything has really started happening this week as now that school is over, I've had some time to devote to this, so my bot finally got off the ground. And then we got together to day and hacked on the server some more cleaning it up, and making an 0.2 release of out Go client/server software.
Finally, and most fun, we whipped together a Matchs page where you can see all the results of games so far. See, real proof that things are happening!
So with this out of the way I can really start to focus on the bot. To that end I've defiantly come up against some quicks in Lisp and SBCL. But then today we were also working in C, Python and PHP and we certainly came up against some quirks in the first two (oddly PHP really does manage to get out of your way and let you do your thing). Still, I think I have things mostly worked out and I can focus on the bot. Which is fun because I'm finding for the most part I'm really enjoying coding in Lisp and Slime+Emacs is pretty rocking. As for the bot, I'm excited. I've got some fun plans for it and I haven't done nearly enough fun coding since school's been on.

Right, so when doing C development on Ubuntu I suddenly noticed something was missing in the pan pages department. Like all the C api.

apt-get install manpages-dev

ah that's better. Someone might want to make it part of the 'build-essentials' package.

The Spring '08 semester has finally drawn to a close. I'm so glad I'll be able to go back to getting reasonable amounts of sleep, and not going to school for several hours a day, then working close to 40 hours a week as well. I'll be much happier being able to split my time between BIOSLEVEL.com and Buffalo Wild Wings.

For BIOSLEVEL, Sapphire recently sent us new AMD 780G chipset motherboard, Radeon HD3450, and an overclocked Radeon HD3870. Between the motherboard and Radeon HD3870, there's a pretty powerful machine just waiting to be powered. Several companies sent us additional parts for the review, which included 2GB of RAM, a quad-core Phenom CPU, and a slew of heatsinks. The reviews of the motherboard and videocard have been posted, and I've got several reviews lined up for the donated components as well. I'm just amazed at the piled of stuff that was shipped to us to power the motherboard.

Simply put, I'm surprised at how well the motherboard's ATI-based integrated graphics performed in Linux. This won't be a motherboard I let out of my sight anytime soon. Since my last final on Wednesday, I've completed an entire two reviews in a single 24-hour period. It's so ridiculously nice to have time away from school again.

Maybe this'll finally be that summer I do something truly productive (and profitable). I've got all the ideas lined up, but do I have the time or motivation to do it? Time will tell.

In Linux when I use SSH I usually pass the host and port and username on the command line and then type the password when prompted. (In those rare cases I don't use certificates to log in without a password.) In Windows, PuTTY makes you pick a host and port and then prompts you for the username AND password.

This leads to unpleasant results. I'm so conditioned to open SSH and type my password at the prompt and hit Enter that I often end up typing my password as my username in PuTTY. Bad.

I've sometimes opened webpages that have some stupid Javascript bullcrap that tries to auto-focus the username field in a login form. But if you're a fast typist (and mouse-ist) like I am, you can focus the field, type your username, and hit tab to get to the password field before the long-loading Javascript bloat has a time to load and run. Which can result in auto-re-focusing the username field, which if it happens at just the right instant, results in my typing the password into it and pounding Enter before I have a chance to notice what's happened. Bad bad bad.

I use a computer far too often to have time too read every prompt, which leads to bad things. Anyone who's used to flying around an interface at light-speed by instinct and repeated learned behavior has experienced this kind of thing I'm sure.

This is horrendously bad because these programs often log the usernames of login attempts in plaintext in logs that lots of potentially evil people have the ability to read. The logs don't usually log the passwords of login attempts, but if you type a password AS a username, oops, you're screwed. Thankfully I'm root on most or all of the machines I ever SSH to, and I can go into /var/log and erase my mistake from the logs before anyone can see. But that doesn't help for web pages I don't know. And I wonder how often this kind of thing happens to other people. I wonder how many people who aren't familiar with computers accidentally send their password as their username to a bunch of websites.

After all the effort we go to to try to secure computer applications, these kinds of stupid human factors can still so easily ruin everything.

The Good

Firefox 3 Beta

Wubi

The Bad

Bootsplash

Drive encryption

The Ugly

Pulseaudio

Summary

A few days ago, I spoke with John Phillips (he is a computer science professor and adviser here at Mansfield) and I found out that I had made three class-selection errors in my Fall 2008 schedule. Luckily, he was able to set me on the right track. My new schedule is linked below. Instead of Accounting II (sorry to Dave and Vinnie), Data Structures (sorry to Dan McKee), and Honors Research (sorry Sara, Andrew, and others), I will be taking Computer Organization, Finite Math, and Principles of Microeconomics.

Revised Fall 2008 Schedule (PDF).

After the big download, I kick-started the installer and I couldn’t believe my eyes: That service pack wants to have 1475 MB free space while installation - true, that number sounds scary, but it wasn’t better with the second service pack.

Since the machine boots again, I am looking through application changes - in other words, what does not work any more. But luckily it is only the ATI-Catalyst 8.2 having problems: screen rotation does not work any more. I have not spotted anything else acting strange by now…

After one week delay, the third service pack for Microsoft Windows XP arrived. You won’t believe it: It’s even available via automatic update.

But what’s inside? First of all, it’s a huge file being 313 MB which means a rather big download and a longer installation. But what’s under the hood? Can’t tell by now. I’m just installing.

http://world.std.com/~franl/worm.html

A Slashdot article reminded me of one of my favorite technical articles on the Internet entitled “A Tour of the Worm", an in-depth historical and technical look at the Morris worm. The Morris worm, mistakenly unleashed in 1988, was one of the first significant worms to strike the Internet, and it caused enough damage that it arguably has done the most relative damage of any worm since then.

Check it out if you haven’t read it, or even if you have; it’s a fascinating look at the early days of the Internet. http://world.std.com/~franl/worm.html

Here’s the thing. I hate stating the obvious. It really annoys me. On the other hand, obvious things are sometimes things that most need to be repeated. So I wrestle with myself and I finally decide that I should, because there is a shockingly large number of people out there who don’t realize how obvious this is. See if you can learn something from this mock dialog.

Vendor: Good morning, is this Harry, the CTO*, I’m speaking to?
Client: Yes, how may I help you?
Vendor: Hey Harry, this is Steve from Microsoft. I would like to talk to you about Windows Vista.
Client: What’s that?
Vendor: Why, it’s the brand new version of our Windows operating system.
Client: Oh, that.
Vendor: I was wondering if I could interest you in our product.
Client: You know what, I don’t think so, we are a very security sensitive company, and..
Vendor: But that’s precisely the reason I’m calling, I would like to tell you how you can enhance your security with Windows Vista. You see, we’ve built the operating system with security in mind and it’s the state of the art in operating systems.
Client: Hey, that sounds pretty exciting. So how does this work now, you ship us the source code and…
Vendor: No no, we don’t distribute the source code.
Client: You don’t?!?
Vendor: No, you see it’s a trade secret. (my precious etc)
Client: You’re kidding, right?
Vendor: No, really.
Client: So how do we know that it’s actually secure if we can’t see for ourselves? How do we know there isn’t anything malicious in it?
Vendor: Well you’ll just have to trust us.
*Harry hangs up*
Vendor: Hello? Harry?
*CTO - the highest placed person who makes technical decisions in a company.

How did it go? Did you get it? It was kind of a long thing, huh? Ok, stop racking your brains, I’ll give you the answer: no source code, no security.

Here’s how that works. It’s simple economics, so try to keep up. If they give you the source code, then they put their cards on the table. You can see what the code does, and if it’s doing something stupid (security hole) or nasty (like sending your data to back to the vendor), then you’ll be able to check for this. Now you may say “I don’t know how to check”, and that’s okay. But just by giving you the source code the vendor knows that you can see everything the code is doing. And if you find something nasty in there, they know you’ll never trust them again. So it doesn’t really matter if *you* don’t know how to check, because there are others who do, and sooner or later someone will find the nasty code if it’s in there. Thus, if the vendor gives you the source code, then he’ll be a lot more careful about what’s in there, because he’s risking losing your trust and your business forever. That will keep him honest.

Is there then anything surprising about finding out that Microsoft is putting in backdoors in Windows? No, because how would you know it’s there? You don’t have the source code! In case you were wondering, the words “security” and “backdoor” are mutually exclusive.

So what have we learned today? Is there somehow we could summarize all this in just one sentence? There is: If you want security, ask for the source code. If you can’t get the source code, you know that the vendor isn’t taking security seriously.

  * ERROR: net-fs/nfs-utils-1.1.2-r1 failed.
 * Call stack:
 *       misc-functions.sh, line 652:  Called install_qa_check
 *       misc-functions.sh, line 360:  Called die
 * The specific snippet of code:
 *   		[[ ${abort} == "yes" ]] && hasq stricter ${FEATURES} && die "poor code kills airplanes"
 *  The die message:
 *   poor code kills airplanes

I had to undelete someone's files from a FAT partition today. My first thought was to use good ol' Windows to do so, given that Windows is the unholy ground which spawned FAT to begin with. I remember there used to be an UNDELETE command of some sort in some old version of DOS. But this doesn't seem to exist in XP any longer.

There are however lots and lots of third-party "shareware" programs which can do this kind of thing, as Google reveals. There is in fact an overwhelming number of such shareware programs. Most of these programs are total crap and cost around $30. One program required me to burn a CD and reboot my computer from the CD before I could run it. Many of the programs "intelligently" scan a partition looking for chunks of things that look like JPEGS or WMVs. I tried a few "demos" before I gave up, not having an hour to waste finding the one program that would work. Thus bringing the current score to Windows: 948, Brian: 0.

Instead I brought the drive home and plugged it into Gentoo and used this post as a guide. I dd'ed the partition to a file, fscked around with it a bit, mounted it via loopback, and had my files back. Took 10 minutes, and worked as expected. And it didn't cost me $30.

The moral of this story: I need to burn a Knoppix disk to take to work with me.

My only quibble is that I can never ever remember what Gentoo package contains fsck.vfat. Note to self, it's dosfstools. I can never think of the search terms even to locate that package. I had to google it.

My ninth call to Westinghouse today, about my Westinghouse L2410NM 24" LCD monitor which I RMA'ed back in March, revealed that they did in fact shipmy monitor, supposedly to my house, on April 4th or so. A UPS tracking number confirms it. There are are a few things wrong with this.

1. In spite of the fact that I asked for a phone call to be updated on the status of my monitor whenever it was shipped, I received no such phone call.
2. During the four phone calls (or was it five?) I made to Westinghouse in April, AFTER my monitor was supposedly shipped to my house, no one at the company had any record that it shipped. I was told that by multiple representatives over the past four weeks that my monitor was "in processing".
3. I asked for my monitor to be shipped to workplace, not my house. My nice, safe, cozy workplace with human beings who can sign for large expensive packages. Not my empty house in a neighborhood full of drug addicts, in the property theft capital of the west. In addition to telling the phone representative this, I actually taped a 8.5 x 11 inch sheet of paper directly to the monitor itself (as well as the outside of the box) specifying SHIP TO: and my work address. Even such drastic measures were not enough to catch the attention of whatever magical monitor-repair fairies work at Westinghouse, apparently. Perhaps I should've carved that information directly into the monitor screen.
4. I could possibly overlook the above, except that, as you may have surmised, at the present time, I do not, in fact, have my monitor.

After calling up UPS to ask why their driver left a $450 computer monitor, in a shiny bright blue and white box with pictures of a computer monitor all over it, sitting on my front porch while I was at work without getting my signature, I placed call number ten (yes, I've finally hit double digits!) to Westinghouse, and managed to escalate my issue to the Westinghouse corporate office. Supposedly in 7-10 business days they will send me a brand new monitor.

Oh how I wish I had any confidence that I'm ever going to see that monitor.

In the meantime, this guy was on sale at the local store, so I bought one. Time will tell whether LG brand is any better than Westinghouse. This time, I also bought the extended warranty, having learned my lesson that it can, indeed, be worth an extra $60 to save myself some pain and aggravation later. I'm also going to think twice about buying things like this over the internet in the future. There is something to be said about being able to drive 10 minutes down the road to have your property serviced or replaced by real-life human beings, rather than paying to have things shipped around the world for a month.

I’ve made an awkward and difficult decision that I hope doesn’t make anyone feel bad: effective immediately I’ve removed any feeds from Planet Larry / Larry the Universe that were from developers who have retired from the Gentoo project.

I setup the planet feeds mainly so that regular users of Gentoo Linux could talk about and share their experiences, and I want to get back to keeping it with them primarily in mind. I tend to think that ex-developers have more weight with their posts, which tends to cause some unbalance that I was never comfortable with.

Speaking of users, I am always looking for new bloggers to get added, so if you are a Gentoo user who blogs about computer experiences, be sure to sign up.

That’s all for now.

Edit: I originally took this post down, and am now restoring it. I still haven’t decided if this is a permanent change or not. I might create a planet just for ex-developers, or reintegrate them somehow. I dunno. Comments and suggestions are welcome.

Computers are a love/hate thing for me. I love all things digital, but I desperately need to get away from it sometimes too. So I had a nice vacation away from my computer last week. I couldn't keep myself from reading some mailing lists and hitting Slashdot once a day, but I didn't write a single line of code and didn't give my websites or work projects or anything much thought.

But now my vacation is over, and it's so easy to fall back into old habits, endlessly looking at webcomics and reading articles about Common Lisp unit testing suites and cringing at the latest drama amongst Gentoo devs and minding my message board like a crusty old beat cop making his rounds. It's the life I've chosen, and I do like it, but I do like getting away sometimes too.

I fulfilled one of my dreams last week when I finally caved and ordered a solid glass mousepad. They're pretty cheap on newegg.com, depending on the color you want. I happened to want green, and it happened to be the cheapest, so all is well. It looks very nice, and it's big and hopefully the surface won't degrade over time; I tend to eat through mousepads via a slow yet inexorable process of erosion.

Unfortunately my laser mouse doesn't work on it. However, I have learned that if I upgrade my mouse's firmware, it will magically be able to work on a solid glass mousepad. Who would've thought my mouse had updateable firmware, let alone that updating the firmware would allow it to work on new surfaces? Not I.

The bad thing is that I need freaking Windows XP to upgrade the firmware on my mouse. I don't have any computer that has XP on it and I'm afraid to try anything in a virtual machine that involves something as dangerous as fiddling with the innards of connected peripherals. So I tried to install XP on my laptop, desperate times calling for desperate measures. But of course the install failed because my XP install CD is so old (pre-SP1, received free from my college 7 years ago) that it didn't recognize most of my hardware. In fact, the XP install CD blue-screened, which set a new record for how low Windows could sink in my opinion.

So I tried slipstreaming SP2 into my install CD. But it failed because, get this, the filenames of some drivers on the CD, namely usbehci.sys, ended up in lower case rather than uppercase and the CD's install program couldn't locate them. I kid you not. Since when is anything in Windows case-sensitive? Is it running Linux? I had to burn another CD after renaming all the files into uppercase. Then the CD worked, but it couldn't find my hard drive, probably due to missing SATA drivers. At that point I gave up, and plan to take my mouse to work tomorrow to upgrade the firmware on a work machine that has XP on it.

And so the score up to this point in my life is Windows: 947, Brian: 0. Windows remains undefeated.

Thanks go out to Logitech for not letting me use Vista (or, say, LINUX) to upgrade my mouse's firmware, and of course to Microsoft, for yet another gloriously broken and frustrating computing experience.

Google’s SketchUp is an incredibly easy-to-use architectural and 3D-modeling tool. The Computer Science Club here at Mansfield has been talking about modeling the university for a 3D perspective in Google Earth, similar to what Google Earth provides for New York City and other metropolitan areas. Recently, I have been experimenting with SketchUp. A free version is available from the website and a professional version can be purchased for about $500. People enrolled in Universities can obtain the professional licenses for $50 a year. The $50 counts towards purchasing a full license, so after 10 years, it’s yours forever. Or, you can pay for the educational license for 3 years and pay the rest of the money up front.

At a first glance, SketchUp seems to be too simple to be worth anything. Upon further investigation, the simplicity seems to come from Google’s innovational perspective. Google has outdone themselves with SketchUp. It is amazingly easy to pick up and create simple objects. More complicated objects can be created with some practice. I have spent about an hour and a half working with the program. First, I watched the beginning tutorials. Then, I went straight into making objects and refining them. It’s amazingly simple. Just make a shape, pull it up to give it depth, draw other objects on it, and manipulate them. It’s amazingly simple.

In about twenty minutes, I was able to make the desk that I use here at school. Keep in mind that it isn’t 100% perfect, nor is it 100% to scale. I have never seen a program this simple. With an hour and a half of experience, I was easily able to make this (click to enlarge):

Here is the front of it:

From an angle:

From the side (notice the arches in the drawers):

And here it is next to a person:

I consider OLPC to be one of the most exciting initiatives of the last few years. When the idea was first circulated it was such an exciting call to arms to do something about the lack of education in poor regions of the world. And the project has produced what appears to be a pretty incredible product, the research of which is now recycled back into the general hardware industry, so it has brought advances that wouldn’t otherwise have happened (now).

I recall pondering the real purpose of the project, asking what is going to be achieved with these laptops. The OLPC project had a very good answer to this. They said the laptops will promote learning in areas where school books are a luxury. Furthermore, the laptop itself is completely tweakable, you press a special key and the source code of the current program pops up. This will promote learning through tweaking and experimentation, so that eventually an industry can be built on these foundations, in regions where little industry exists today and where perhaps the potential for one (in terms of natural resources) is bleak. A beautiful dream, one that could change the world in big ways.

Now Negroponte has changed his tune. Visionary that he is, he failed to convince the clients of the value of free software. So now he’s humming “forget open source, it’s all about the kids!” while preparing to run Windows on the laptop. There is a new smoke screen being constructed:

Negroponte says that the organization is working to ensure that Sugar can run smoothly on Windows.

Riiiight, running Sugar on Windows. Tell me, what exactly is the value of running Windows with an all free software stack? It’s completely useless, that’s what. The whole value of Windows is as a platform, not merely as an operating system. People buy Windows to run Windows applications, not for Windows itself. Or are we actually buying that Egyptian officials are eager to purchase Windows licenses in order to run the free software suite?

Congratulations, Negroponte, you’ve just become a licensed Windows vendor. The kids will no doubt have fun clicking on the Start menu and playing Solitaire. There is a great deal to learn from that, just nothing about the operating system or the applications, you know, actual learning.

OLPC in its original form was about empowering the users, with Windows that capability is entirely destroyed. The fact you cannot mix learning with trade secrets should be blindly obvious to anyone. Open souce is important, but it’s especially important when you want people to learn something.

Furthermore, learning doesn’t happen in isolation. It’s accelerated when it happens in a community of ideas and impulses that flow freely. Resigning OLPC president gets it when he says:

“What comes part and parcel with open source is a culture, and it’s the culture that I’m interested in,” he says. “It’s a culture of expression and critique, sharing, collaboration, appropriation.” And this culture can and should spill into classrooms, he says.

For a very long while now, several years actually, I've been a bit annoyed by the behaviour of terminals under X when you doubleclick links. What the UI considers a word is selected. Selection 'starts' at the point that is doubleclicked and 'spreads' in each direction, stopping at a char it considers to be a word delimiter. A space is probably always considered a delimiter. Sometimes a '?' too, and often ',' as well. This has been very annoying for me as I spend quite alot of time in irc (irssi in a screen, its lovely in combination with bitlbee; icq, irc etcetc in the same screen.) and every now and then someone pastes an url which I want to doubleclick and then paste into my browser. Now the selection stops prematurely as it's not uncommon for a hyperlink to contain one of the characters that is in the word delimiter list. I've thought of this as a limitation to the system I use and, although annoyed, never gave it much thought. Maybe I've been to susceptible to propagandaists telling me gnu/linux is user-unfriendly.

A couple of weeks ago I started to think a little about it and the solution is rather simple. I use Eterm more or less exclusively.

In ~/.Eterm/user.cfg I've put this:

<eterm-0.9>
begin misc
cut_chars "\t\\\`\\\"\'() *,;<>[]{|}"
end misc

This might not be perfect but it's certainly served it's purpose this far as I have been able to doubleclick on the links and then middleclick in the address field (or open a new tab) in the browser. My OS is even better =)

Graduate School Information Request

---

Your request for information has been processed.

Thank you for your interest in the University of Massachusetts Amherst.

Applications directed to addresses within the United States are mailed twice weekly with first class postage; allow 10 days for delivery. Applications directed to non-U.S. addresses are mailed weekly and sent air mail; allow at least 3 weeks for receipt. There is no charge for any of these materials.

The following materials will be sent to you here:

Department of Computer Science
KICT, IIUM
Gombak, 53100 MYS

• CMPSC program materials

Sometimes you may want to check that an email address is not syntactically invalid, i.e. it looks like a recognisable email address. I use this approach in my zetact contact form processor.

Of course, it does not mean the address actually leads anywhere, but at least you know are dealing with an email address that could exist.

This is the code I have been using, albeit I have changed it from a class method to a simple function to make this post simpler.

"""Email check using regex."""
    def invalidreg(emailkey):
        """Email validation, checks for syntactically invalid email
        courtesy of Mark Nenadov.
        See
        http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65215"""
        import re
        emailregex =
        "^.+\\@(\\[?)[a-zA-Z0-9\\-\\.]+\\.([a-zA-Z]{2,3}|[0-9]{1,3\
    })(\\]?)$"
        if len(emailkey) > 7:
            if re.match(emailregex, emailkey) != None:
                return False
            return True
        else:
            return True

I decided it would be more Pythonic to try to do this using the built-in string methods, rather than importing the re module and using a monster regular expression. Here was my first attempt.

"""Email checks using string methods - simple version."""
    def invalidemail(emailaddress):
        """Checks for a syntactically invalid email address."""
        try:
            emailitems = emailaddress.rsplit('@', 1)
            emailitems.extend(emailitems[1].rsplit('.', 1))
        except IndexError:
            return True

        if [x for x in emailitems if not x.replace(".","").isalnum()] \
                and emailaddress >= 7:
            return True
        else:
            return False

After a bit of testing and playing with this, a friend pointed me towards the relevant RFC on restrictions of email addresses. While the standard allows the use of many different special characters, in practice email addresses have to be much stricter if you actually want people in the real world to be able to send email to you.

For example, if we allow the email address []@commandline.org.uk, will whatever receives the output of this function be able to use it? As pointed out by Jan Goyvaerts, most software won't actually be able to handle obscure special characters.

We also don't want to water down the syntax check and allow junk for the sake of theoretical but non-existent addresses.

My compromise is to allow these special symbols -_.%+. in the local-part of the email address, and -_. in the domain name. I also do sanity checking on the top-level domain, it needs to be either a generic name or two characters long (country codes are all two letters).

So below is my current version, I added lots of comments and white space to make it easy to read.

"""Ditch nonsense email addresses."""

    GENERIC_DOMAINS = "aero", "asia", "biz", "cat", "com", "coop", \
        "edu", "gov", "info", "int", "jobs", "mil", "mobi", "museum", \
        "name", "net", "org", "pro", "tel", "travel"

    def invalid(emailaddress, domains = GENERIC_DOMAINS):
        """Checks for a syntactically invalid email address."""

        # Email address must be 7 characters in total.
        if len(emailaddress) < 7:
            return True # Address too short.

        # Split up email address into parts.
        try:
            localpart, domainname = emailaddress.rsplit('@', 1)
            host, toplevel = domainname.rsplit('.', 1)
        except ValueError:
            return True # Address does not have enough parts.

        # Check for Country code or Generic Domain.
        if len(toplevel) != 2 and toplevel not in domains:
            return True # Not a domain name.

        for i in '-_.%+.':
            localpart = localpart.replace(i, "")
        for i in '-_.':
            host = host.replace(i, "")

        if localpart.isalnum() and host.isalnum():
            return False # Email address is fine.
        else:
            return True # Email address has funny characters.

    # Start the ball rolling.
    if __name__ == "__main__":
        print invalid("warrior@example.com")

Discuss this post - Leave a comment

[update] Question solved, see bottom of post.

Since Python 2.5 the language got a new built-in method ‘all’ (and it’s nephew ‘any’). I wanted to play around with this a little, combined with generators, so I created a little testcase to test performance.

Here’s the test-case: take a list L of X random numbers in a given range [A, B], and check whether

• all elements in L are >= A
• all elements in L are >= (A + Z) where Z is a number in [0, (B - A)]

The first test should always result True, the second test could result to False.

Here’s the output of a test-run:

In [1]: import random, sys

In [2]: a = [random.randint(100, sys.maxint) for i in xrange(2000000)]

In [3]: len(a)
Out[3]: 2000000

In [4]: #Check whether all elements are >= 100 

In [5]: %timeit all(i >= 100 for i in a)
10 loops, best of 3: 515 ms per loop

In [6]: %timeit any(i < 100 for i in a)
10 loops, best of 3: 454 ms per loop

In [7]: def f(l):
   ...:     for i in l:
   ...:         if i < 100:
   ...:             return False
   ...:     return True
   ...: 

In [8]: %timeit f(a)
10 loops, best of 3: 292 ms per loop

In [9]: #Same thing for 100000, since now the list shouldn't be completely iterated

In [10]: %timeit all(i >= 100000 for i in a)
100 loops, best of 3: 4.73 ms per loop

In [11]: %timeit any(i < 100000 for i in a)
100 loops, best of 3: 4.29 ms per loop

In [12]: def g(l):
   ....:     for i in l:
   ....:         if i < 100000:
   ....:             return False
   ....:     return True
   ....: 

In [13]: %timeit g(a)
100 loops, best of 3: 2.82 ms per loop

In [14]: #For reference

In [15]: %timeit False in (i >= 100 for i in a)
10 loops, best of 3: 531 ms per loop

In [16]: %timeit False in (i >= 100000 for i in a)
100 loops, best of 3: 5.03 ms per loop

It’s as if ‘all’, ‘any’ or ‘in’ don’t break/return when a first occurence of False (or True, obviously) is found. Is this the desired behaviour, and if it is, why? The calculation time difference between using all/any/in or a custom-made function (which is, unlike all etc, not written in C) which breaks whenever it can, is pretty astonishing.

[update] Question solved. It’s pretty normal the function-based approach performs better, since it combines what ‘all’ and the generator provided to ‘all’ do, taking away the generator function-call overhead. Damn

If you’ve been dealing with files for a while you will have noticed that there is a slight semantic gap between how humans see files and how computers do. If you’ve ever seen a file list like this you know what I mean:

Lecture10.pdf
Lecture11.pdf
Lecture12.pdf
Lecture1.pdf
Lecture2.pdf
…

Numbering these files was done in good faith, and a user understands what it means, but the computer doesn’t get it. Sorting in dictionary order produces the wrong order as far as the user is concerned. The reason is that the digits in these filenames are not treated and compared as integers, merely as strings. (Actually, . comes before 0 in ASCII, what’s going on here?)

While we’re not expecting our computers to wisen up about this anytime soon, there is the obvious fix:

Lecture01.pdf
Lecture02.pdf
…
Lecture10.pdf
Lecture11.pdf
Lecture12.pdf

You’ve probably done this by hand once or twice, while cursing.

On the upshot, this is very easy to fix with a few lines of code:

#!/usr/bin/env python
#
# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 3.
#
# revision 1 - support multiple digit runs in filenames
 
import os, string, glob, re, sys
 
def renseq():
    if (len(sys.argv) != 2):
        print "Usage:\\t" + sys.argv[0] + " <num_digits>"
    else:
        ren_seq_files(sys.argv[1])
 
 
def ren_seq_files(num_digits):
    files = glob.glob("*")
    for filename in files:
        m = re.search("(.*)(\\..*)", filename)
        ext = ""
        if m: (filename, ext) = m.groups()
 
        digit_runs = re.finditer("([0-9]+)", filename)
        spans = [m.span() for m in digit_runs if digit_runs]
        if spans:
            spans.reverse()
            arr = list(filename)
            for (s, e) in spans:
                arr[s:e] = string.zfill(str( int(filename[s:e]) ), int(num_digits))
            os.rename(filename+ext, "".join(arr)+ext)

 
 
if __name__ == "__main__":
    renseq()

Download this code: renseq.py

This works on all the files in the current directory. Pass an integer to renseq.py and it will change all the numbers in a filename (if there are any) to the same numbers, padded with zeros if they have fewer digits than the amount you want. So on the example

renseq.py 2

will turn the first list into the second list.

If say, there are filenames with numbers of three digits and you pass 2 to renseq.py, the numbers will be preserved (so it’s not a destructive rename), you’ll just revert to your incorrect ordering as it was in the beginning.

renseq.py will rewrite all the numbers in a filename, but not the extension. So mp3 won’t become mp03.

I had to fix a problem at my dad's company...
"The network was broken."

It was a NetBEUI network connecting some windows stations - it has been running for years - and now suddenly the nodes couldn't find eachother.
One of the boxes (windows 2000 iirc) had 2 network cards, one for the network, the other not used for anything (not even connected). Disabling the latter - not even touching the former - fixed half of the network.

read more

Someone in #lh today told me about Sender Policy Framework, which sounds like a badly-needed enhancement to the Internet’s email protocols. Basically, the idea is to provide a DNS record that informs MTAs “don’t trust emails claiming to be from this domain unless they’re coming from one of my actual servers".

In DNS, this looks like (in my case):

emptymatter.org. IN TXT "v=spf1 a mx ~all"

Some MTAs support SPF but need to be configured, I believe Gentoo’s postfix is one of them. If I’m going to expect other mail servers to support it, I probably should myself. I’ll have to tackle that another day…

Since Heise announced that Microsoft will release the Windows XP Service Pack 3 on the 29th, I didn’t sleep too well, as I really want to grab it as soon as possible. Sure, I use Linux, but this doesn’t mean that I am not dangling with windows boxes from time to time and I am fed up applying about 100 patches before I can even think of security.

But I was laughing too soon - as just one day after the 29th (today), I have seen a posting on the Washington Post Blog that Microsoft has delayed the start of the service pack again. In a written statement they say:

“In order to make sure customers have the best possible experience we have decided to delay releasing Windows XP SP3 to Windows Update and Microsoft Download Center.”

In other words, there seems to be no release date yet. Well - in the meantime I’m rolling my own update pack using the Heise Offline Update. Thanks a ton, guys.

Nothing exciting here. Got my tax refunds. Might build a home-made NAS with a couple terabytes of disk and put it in the basement.

On the DS, I’ve been playing Rondo of Swords and The World Ends With You. Rondo is a pleasant find, a difficult but still reasonable strategy RPG that makes one think and plan ahead, unlike games such as Revenant Wings which are much more “bring a healer and just mob everyone at the thing they’re strong against!” Also, I have a crush on Atlus by this point. There’s no denying it now. I draw their name with little hearts all around when I’m in meetings.

The World Ends With You is refreshingly original, one of those games that, even with it being Square Enix, is a bit surprising that it made it to the States. Very Japanese, and the game makes few concessions to the English audience. Sure, long gone are the times of gratitutous name changes, but even the j-pop/j-rock soundtrack remains intact, and that is, to my slightly jaded mind, a bit commendable. Now, if only the main character didn’t suffer from two vile Square Enix staples: unimaginable thinness and nearly sickening teenage angst. Neku is supposed to get better with the latter; I hope it is soon.

My games to beat are now Etrian Odyssey and Rondo of Swords, one I must beat before Etrian Odyssey II (guess which one) is released here, and the other before the Final Fantasy IV remake reaches the States. I’m excited. If I have time before those, Final Fantasy III and The World Ends With You are my RPGs to beat. FF3 is a cakewalk thus far, but its ease and its crude mechanics compared to Final Fantasy V make it hard to stay with for long.

I didn’t really intend this to become all about video games. I’ve been working on a Gentoo Wiki page for the HP 2133 which has kind of slowed down as most of the parts I’m interested in are supported as best they can be without new versions of drivers, I think. There’s some other hardware that I need to try out (the webcam, for example), but I don’t really care that much, so it’s low priority. Notebooky stuff works.

I have a Waterfield Designs bag coming soon, which I’m excited about. Don’t think it will be suitable for gaming books, but I still have that backpack which is going on 5+ years. The little trooper.

I’ve been meaning to survey the gaming group and associated friends to see what they’re using for IM these days. I think the answer for some is “nothing", with a couple saying “AIM on occasion” or “I idle on Google Talk", so I’ve not really been motivated to test those waters. I want to get a private Jabber conference room running for the group, since the IRC thing kind of sputtered off and died (I still idle there!), but I know it means getting people to switch to Jabber (or at least Google Talk) and then getting them to use a non-Google Talk client (Pidgin, I bet, but maybe Trillian would work). Sigh. If anyone has interest in switching to one network (I highly suggest a Jabber-like ["XMPP” for the techies]), or trying out conferences, or whatever, email/IM me and we’ll play around.

This really is getting rambly, and people might expect me to write long posts all the time. So I’m wrapping this up by saying that spring is finally here, and that’s why it snowed yesterday.

Use Readline shortcuts

At the bash prompt, you can use the default readline keybindings, these are similar to Emacs ones. Many of these are also available within other programs that use readline, such as the Python interpreter.

Here are some useful ones:

Ctrl-A Beginning of Line

Ctrl-E End of Line

Ctrl-U Kill (cut) everything left of cursor

Ctrl-K Kill (cut) everything right of cursor

Ctrl-W Kill (cut) the single word before the cursor

Ctrl-Y Yank (paste) the text back

Ctrl-L Clear Screen

Ctrl-D Exit

Ctrl-R Reverse interactive-search, (attempt to complete what is currently being typed using the history file)

SSH without Passwords

If you login to a remote machine often and you get bored of typing the password, then you can use public key cryptography instead.

The way it works is that the remote machine has a copy of your local machine's public key, it can then use that to check that your local machine is really your machine, and so let you in.

To start with, on the local machine, see if you already have a key pair:

ls ~/.ssh/id_?sa.pub

If not, then make one:

ssh-keygen -t dsa

Now you need to copy your public key to the remote host. On the local machine run:

scp ~/.ssh/id_?sa.pub remotehost:

Now we login to the remote server:

ssh remotehost

Append the public key to your authorized keys file

cat id_?sa.pub >> ~/.ssh/authorized_keys

Now you can login without passwords. Make sure the security of your machines is well thought out. Use disk encyption if possible.

Create a script directory in home directory

I often talk about random Python or bash scripts. The easy way to use them on Linux is to make a dedicated script directory for these.

mkdir ~/bin

Add it to your shell's path. Edit ~/.bashrc and add:

export PATH=$HOME/bin:$PATH

Now all the scripts that you add to ~/bin are always available. This makes things a lot more flexible and fun as you can try out various scripts by dropping them in ~/bin and then deleting them when you are bored of them.

Discuss this post - Leave a comment

So Ubuntu hardy doesn't ship with the handy git wrapper/tool cogito because git has all the features in it incoperated... somewhere...

But documentation is surprisingly sparse. Anyways, if you want to set up a git repo nowadays using just git, it should go as something like follows:

root@server # cd /git
root@server # mkdir newrepo
root@server # chgrp git newrepo
root@server # chmod g+ws newrepo
root@server # cd newrepo
root@server # git init

And if this is a public repository

root@server # touch git-daemon-export-ok

On the client side.

user@client $ cd project
user@client $ git init
user@client $ git add *
user@client $ git commit -m "Initial code dump"
user@client $ git remote add origin ssh://user@git.server.com/git/reponame
user@client $ git push origin master

and after that regular

user@client $ git push

works just fine.

1. Logged into a lab computer via XDMCP: I could've used VNC, but that would've required someone to be logged-in on the lab comp.


2. Using my laptop's PulseAudio as the lab computer's default PulseAudio sink: This makes the lab computer's PulseAudio send all sound to my laptop's PulseAudio by default.


3. Connected to my laptop's DAAP share from the lab computer's Rhythmbox: The music on my laptop becomes accessible from the lab computer's Rhythmbox.

This has probably happened to you. You come to a web page that has links to a bunch of pictures, or videos, or documents that you want to download. Not one or two, but all. How do you go about it? Personally, I use wget for anything that will take a while to download. It’s wonderful, accepts http, https, ftp etc, has options to resume and retry, it never fails. I could just use Firefox, and if it’s small files then I do just that, and click all the links in one fell swoop, then let them all download on their own. But if it’s larger files then it’s not practical. You don’t want to download 20 videos of 200mb each in parallel, that’s no good. If Firefox crashes within the next few hours (which it probably will) then you’ll likely end up with not even one file successfully downloaded. And Firefox doesn’t have a resume function (there is a button but it doesn’t do anything ).

So there is a fallback option: copy all the links from Firefox and queue them up for wget: right click in document, Copy Link Location, right click in terminal window. This is painful and I last about 4-5 links before I get sick of it, download the web page and start parsing it instead. That always works, but I have to rig up a new chain of grep, sed, tr and xargs wget (or a for loop) for every page, I can never reuse that and so the effort doesn’t go a long way.

There is another option. I could use a Firefox extension for this, there are some of them for this purpose. But that too is fraught with pain. Some of them don’t work, some only work for some types of files, some still require some amount of manual effort to pick the right urls and so on, some of them don’t support resuming a download after Firefox crashes. Not to mention that every new extension slows down Firefox and adds another upgrade cycle you have to worry about. Want to run Firefox 3? Oh sorry, your download extension isn’t compatible. wget, in contrast, never stops working. Most limiting of all, these extensions aren’t Unix-y. They assume they know what you want, and they take you from start to end. There’s no way you can plug in grep somewhere in the chain to filter out things you don’t want, for example.

So the problem is eventually reduced to: how can I still use wget? Well, browsers being as lenient as they are, it’s difficult to guarantee that you can parse every page, but you can at least try. spiderfetch, whose name describes its function: spider a page for links and then fetch them, attacks the common scenario. You find a page that links to a bunch of media files. So you feed the url to spiderfetch. It will download the page and find all the links (as best it can). It will then download the files one by one. Internally, it uses wget, so you still get the desired functionality and the familiar output.

If the urls on the page require additional post-processing, say they are .asx files you have to download one by one, grab the mms:// url inside, and mplayer -dumpstream, you at least get the first half of the chain. (Unlikely scenario? If you wanted to download these freely available lectures on compilers from the University of Washington, you have little choice. You could even chain spiderfetch to do both: first spider the index page, download all the .asx files, then spider each .asx file for the mms:// url, print it to the screen and let mplayer take it from there. No more grep or sed. )

Features

• Spiders the page for anything that looks like a url.
• Ability to filter urls for a regular expression (keep in mind this is still Ruby’s regex, so .* to match any character, not * as in file globbing, (true|false) for choice and so on.)
• Downloads all the urls serially, or just outputs to screen (with --dump) if you want to filter/sort/etc.
• Can use an existing index file (with --useindex), but then if there are relative links among the urls, they will need post-processing, because the path of the index page on the server is not known after it has been stored locally.
• Uses wget internally and relays its output as well. Supports http, https and ftp urls.
• Semantics consistent with for url in urls; do wget $url… does not re-download completed files, resumes downloads, retries interrupted transfers.

Limitations

• Not guaranteed to find every last url, although the matching is pretty lenient. If you can’t match a certain url you’re still stuck with grep and sed.
• If you have to authenticate yourself somehow in the browser to be able to download your media files, spiderfetch won’t be able to download them (as with wget in general). However, all is not lost. If the urls are ftp or the web server uses simple authentication, you can still post-process them to: ftp://username:password@the.rest.of.the.url, same for http.

Download spiderfetch:

• spiderfetch.rb

Recipes

To make the use a bit clearer, let’s see some concrete examples.

Recipe: Download the 2008 lectures from Fosdem:

spiderfetch.rb http://www.fosdem.org/2008/media/video 2008.*ogg

Here we use the pattern 2008.*ogg. If you first run spiderfetch with --dump, you’ll see that all the urls for the lectures in 2008 contain the string 2008. Further, all the video files have the extension ogg. And whatever characters come in between those two things, we don’t care.

Recipe: Download .asx => mms videos

Like it or not, sometimes you have to deal with ugly proprietary protocols. Video files exposed as .asx files are typically pointers to urls of the mms:// protocol. Microsoft calls them metafiles. This snippet illustrates how you can download them. First you spider for all the .asx urls, using the pattern \.asx$, which means “match on strings containing .asx as the last characters of the string”. Then we spider each of those urls for actual urls to video files, which begin with mms. And for each one we use mplayer -dumpstream to actually download the video.

#!/bin/bash
 
mypath=$(cd $(dirname $0); pwd)
webpage="$1"
 
for url in $($mypath/spiderfetch.rb $webpage "\\.asx$" --dump); do
	video=$($mypath/spiderfetch.rb $url "^mms" --dump)
	mplayer -dumpstream $video -dumpfile $(basename $video)
done
 

Download this code: asx_spiderfetch.sh

Living Code

David Parker famously said that texts are living, once they leave the pen of the author then they have a life of their own, you never know where the text will end up or how it will be modified. For Python code that is even more true.

The beauty of Python is that you can write code fast, share code and modify code. For this to work, your code needs to be readable. Writing code is easy, reading other people's code is much harder, or even reading your own code after a few months or years has past.

Therefore the aim is to make code as readable as possible, even if it causes a little more work when you write it. The way to make your Python code most readable is to keep to the Style Guide for Python Code, also known as PEP8.

Pylint for the Win

It is far easier to keep your code valid to PEP8 as you go along, than to try to move a large codebase to PEP8 at the end. I recommend the use of a tool called pylint.

Pylint is available from all Linux distributions' package managers (e.g. apt-get install pylint or emerge pylint). Here are some instructions for Windows.

If you have ever made a webpage you probably know about HTML-tidy or the online W3C Validator tool. These tell you everything wrong with your HTML.

Pylint is similar, it goes through and tells you both syntax errors and also how your code differs from the PEP8 standard.

There are some corner cases in which you will need to give pylint the finger, but doing it consciously for good reason is better than because you are sloppy.

PEP8 is better than your crappy style

People often don't use PEP8. This is for a variety of (bad) reasons.

Firstly, sometimes people are tourists from another programming language, they do not know any better so they write their Python code like it was Java or C code.

Secondly, Sometimes people think their (cl)own style is better than PEP8 in some technical way. Well that does not matter. I might have a better way to design a plug socket, but if I implemented my better plug socket, I would not be able to buy any electrical devices.

There can only be one standard, and PEP8 is that standard. If you want to change that standard then bribe, sleep with or kill Guido Van Rossum.

Not following the standard makes your code less readable to others, this prevents the quick reuse that Python is designed for (see above).

If you are a free-software/open-source project, then you particularly should be ashamed if you write hard to read code, because allowing other people to read, understand and modify your code is the whole point.

Lastly, some people don't use PEP8 because the document is too circular and verbose for them to remember. I feel your pain, below are the main points in 12 easy rules.

The 12 commandments

Guido, who brought you out of the land of Visual Basic, out of the land of slavery, spake all these words to thee:

• Module names should be in all lowercase - hello.py.
• Class names should be in CamelCase.
• Methods and functions should be in lower_with_underscores
• Implementation-specific 'private' methods _single_underscore_prefix
• Especially private non-subclassable methods __double_underscore_prefix
• Top level constants (i.e. those that are not in a function or class) should be in BLOCKCAPITALS. Overuse of these constants may make your code less reusable.
• If a variable inside a function or method is so temporary and disposable that you cannot give it a name, then use i for the first one, j for the second and k for third.
• Indentation is four spaces per level. No tabs. If you break this rule then you must be stoned in the village square.
• Lines are never more that 80 characters wide. Tip, break lines with a backward slash . You do not need to do this if there are parentheses, brackets or braces. Don't add extra parentheses just to break lines, use instead.
• Spaces after commas, (green, eggs, and, ham)
• Spaces around operators i = i + 1
• Write docstrings for all public modules, functions, classes, and methods. Python is an international community, so use English for docstrings, object names and comments. If you want to provide local translations then use a proper localisation library.

Discuss this post - Leave a comment

Monit is a daemon that looks simple enough and does one thing really good: Making sure services stay up (or: Making sure you know when they don’t). It does some other things as well (like checking up on files and system resources), but I just use it for important processes which could crash (or be killed). Now, I’m not gonna go into how to configure monit since it is quite easy and the config-file can be understood by simply looking at it. You can start monit like any other service via scripts in /etc/init.d and symlinks from the appropriate runlevel-directories. The question that immediatly comes to mind is what would happen if monit itself crashed (Hint: Well, nothing, it would stay down and you wouldn’t even receive an email ;). So, instead of using some home-brewed solution with cronttab that checks monit, I’d suggest just invoking it with init. To do that just append a line like this to your /etc/inittab:mo:2345:respawn:/usr/local/sbin/monit -Ic /etc/monitrcThe -I switch means it runs interactively, which is necessary for processes started by init. To initialize it without rebooting try telinit Q which should already spawn the process. If you kill the monit process now it will be respawned automatically. Voila! Oh, and if you ever want to stop some monitored service (which monit would restart) you can just unmonitor it from the webinterface.
So I’d suggest you give monit a try, even if you think you already have an appropriate solution. Don’t be scared by the looks of the monit-website. And if you think monit doesn’t do everything you need you’re bound to notice it after 15 minutes max, which imho isn’t much time wasted.