Posts for Wednesday, August 12, 2009

Scala tail recursion and decompiler adventures

I’ve been into Scala lately. More about it will follow later, but there’s something I found out which I really like.

Last couple of days I wrote some very basic Scala snippets, containing constructs which would be non-trivial or ‘unusual’ to write in Java, compile it to a class file, and then use a Java decompiler to figure out how the Scala compiler maps those constructs to JVM bytecodes.

There’s one thing which took my attention: looks like (basic) tail-recursive functions are optimized into while-loops! This only happens if the last call of a function is a call to itself (the most basic form of tail recursion), but it’s an interesting feature anyway… No more need to put socket accept handling in an infinite while loop :-)

A little demo. First, here’s a Scala object which implements a very basic ‘reduce’ function:

object Reducer {
  def reduce[T, V](fun: (V, T) => V, values: List[T], initial: V): V = {
    if(values isEmpty)
      return initial
    val next = fun(initial, values head)
    return reduce(fun, values tail, next)
  }

  def main(args: Array[String]): Unit = {
    val values = List(1, 2, 3, 4)
    val sum = reduce[Int, Int]((x, y) => x + y, values, 0)
    println("Result: " + sum)
  }
}

We can compile and run this, and it’ll output the expected result ‘10′:

MacBook:reduce nicolas $ scalac Reducer.scala
MacBook:reduce nicolas $ scala Reducer
Result: 10

Now we can open the generated class files in JD. There are a couple of them (it’s interesting to take a look at all of them and figure out what they represent exactly), but in this case we need ‘Reducer$.class’, which contains the implementations of our public functions, including ‘reduce’.

Here’s the Java version of the ‘reduce’ function:

public <T, V> V reduce(Function2<V, T, V> fun, List<T> values, V initial)
{
  while (true)
  {
    if (values.isEmpty())
      return initial;
    Object next = fun.apply(initial, values.head());
    initial = next;
    values = values.tail();
  }
}

‘Function2′ is a built-in Scala type which represents a function taking 2 parameters. As you can see, this code does exactly the same as our Scala version and is most likely the way we’d write the code manually as well (the only thing I don’t get is why ‘next’ is an Object and not a ‘V’, I might figure that out later), but without forcing us to write the imperative code, whilst still producing bytecodes which will most likely show the best performance on the JVM (which currently has no tail recursion optimization support (although that might change one day)).

I like it :-)

Backup Your Xing Contacts Using Bash

So, you have bloody Xing account and you want to backup your contacts? But you don't want to upgrade to become a premium member (which I would recommend if you're a regular Xing user and want to backup your contacts on a regular basis) because you plan on leaving the platform anyway? If that's the case the following might be of interest to you ;-).

Yesterday I decided that I want to leave Xing for the better. I just didn't get any use out of the platform. In the 3 years I was a member I didn't participate in any groups, nor did I actively manage my network. I accepted contacts when they approached me, but that's about it. Most of my contacts are either work colleagues or people I know in person and I think I really don't need to be a Xing member to just, well, manage that data. On the other hand I have a linkedin account as well. I might need to put some work into that though. In any case, I consider my linkedin account more valuable in terms of networking than my Xing account and most of my important contacts are on linkedin too anyway.

So I needed a way get all my contacts of the platform, just to be sure my local address book isn't out of sync (I'm not really into contact management so it is probably way out of sync). Xing however, provides a function to export all vCards in one go to premium members only.

Sorry guys, but I don't want to go premium just before leaving Xing.

IMPORTANT: Xing's TOS don't allow to use scripts to interact with the Xing website. I would have used an API, as stated in the TOS, but I couldn't find any references if this API does even exist :-/. If I'm wrong please point me into the right direction, but, I share tante's opinion on the matter. So, use this script on your own risk! I am not responsible for any implications the usage of this script has regarding your relations with Xing.

Okay, back to the story: I wrote a little bash script which utilises wget and sed to fetch the vCards of my Xing contacts.

To run the script just download it from below, put it in your ~/bin directory and make it executable. The script will download the vCards of your contacts into the current directory when you execute it. It accepts one parameter, namely the number of pages it needs to check to get all your contacts. To find out about that number, just go to your Xing contacts page and switch to vCard view. The last number of the pagination links below the vCards is the number you need. And last not least you also have to set your username and password inside the script to make it work.

The script will sleep 2 seconds after each fetched vcard, this is to prevent the script from hammering the Xing servers, so if you have lots of contacts you might go and do something else while it's running. The error handling of the script is pretty basic, it won't let you know if a download fails, but you can check if everything worked correctly by just counting the number of vCard files against your number of contacts when the script has finished.

That's it! Have fun leaving Xing!

%> ~/bin/xingscrape.sh 7
Authenticating ...
Fetching vcards ...
Peter Lustig ...
...
...
Done!
%> ls -l | wc -l
77

And here's the script:

xingscrape.sh
#!/bin/bash
 
user='XINGUSERNAME'
pass='XINGPASSWORD'
 
# script start
if [[ $# -lt 1 ]]; then
    echo "Error: missing offset"
    echo
    echo "Usage: $0 [offset]"
    exit 1
fi
count=$1
url='https://www.xing.com/app/contact?notags_filter=0;card_mode=1;search_filter=;tags_filter=;offset='
useragent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5"
 
# authenticate and save cookie
echo "Authenticating ..."
wget --cookies=on \
     --keep-session-cookies \
     --save-cookies=cookie.txt \
     --post-data="login_user_name=${user}&login_password=${pass}&remember-login=1&op=login" \
     --user-agent=${useragent} \
     --quiet \
     -O - \
     https://www.xing.com/app/user \
     > /dev/null
 
# dumb check if auth worked
if [ $(wc -l cookie.txt | cut -c 1) -lt 6 ]; then
    echo -n "Boo, auth failed :-/!"
    exit 1
fi
 
# lets do all the work
echo "Fetching vcards ..."
for i in $(seq 0 $count); do
    # fetch vcard urls
    vcards=$(wget --referer="https://www.xing.com/app/user?op=home" \
         --cookies=on \
         --load-cookies=cookie.txt \
         --keep-session-cookies \
         --save-cookies=cookie.txt \
         --user-agent=${useragent} \
         --quiet \
         -O - \
         "https://www.xing.com/app/contact?notags_filter=0;card_mode=1;search_filter=;tags_filter=;offset=$(($i*12))" \
         | sed -n 's/.*href="\([^"]*vcard[^"]*\)".*/\1/p')
 
    # fetch each vcard
    for vcard in $vcards; do
        fname=$(echo $vcard | cut -d \= -f3)
        response=$(wget --referer="https://www.xing.com/app/user?op=home" \
             --cookies=on \
             --load-cookies=cookie.txt \
             --keep-session-cookies \
             --save-cookies=cookie.txt \
             --user-agent=${useragent} \
             -O $fname \
             https://www.xing.com${vcard} 2>&1)
 
        # check HTTP response if we got the vcard
        echo $response | egrep -i "200 OK" > /dev/null
        if [ $? == 0 ]; then
            # give vcard file a proper name
            contact=$(head -n4 $fname | tail -n1 | cut -d \: -f2 | tr -d [:cntrl:] | iconv -f ISO-8859-1 -t UTF-8)
            mv $fname "$contact"
            echo "$contact ..."
        else
            rm $fname
        fi
 
        # don't hammer their servers
        sleep 2
    done
done
 
# delet cookie etc
rm cookie.txt
echo "Done!"

Of course there's room for improvements. If you have an idea how to make this script better just let me know in the comments ;-).

Read or add comments to this article · Save to del.icio.us
avatar

Dell Inspiron 9100 infra red port

My old Dell laptop has a built-in infra red port on the front. In the 4 or more years that I've had it, I've had no need for the port.

Now that it is becoming my (MythTV) media centre, the infra red port becomes useful! Here are the steps I used to get it working. I had some small hassle with the built-in serial port driver taking over.

  1. First, make sure it's enabled in the BIOS, and take note of the serial port it's attached to.

    There are no other serial ports on this laptop, so anything that looks like a serial port is likely to be the infra red port.

  2. I used lirc to receive and configure the port.

    In Gentoo enable the lirc global USE flag (global means put it in make.conf because many programs can make use of it). Also add this line to make.conf:
    LIRC_DEVICES="sir"

    In other distros, you have to somehow build the sir driver.

    Also add the lirc USE flags hardware-carrier and transmitter. You can do this in /etc/make.conf, or better use /etc/portage/package.use, or better still, make package.use a directory, and add the file /etc/portage/package.use/lirc-20090806 with the contents:
    app-misc/lirc hardware-carrier transmitter

    (You can of course make any-named file in /etc/portage/package.use)

  3. (re)compile! This includes lirc, MythTV, mplayer and anything else that might be able to take advantage of your new lirc USE flag.

    In Gentoo, you could use the command:
    emerge --update --newuse --deep world
    to find and rebuild all packages that have changed USE flags.

  4. Now you need to load the lirc module. For com1, run this command:
    modprobe lirc_sir io=0x2f8 irq=3 threshold=5

    options lirc-sir io=0x2f8 irq=3 threshold=5
    Also add this line to remove any conflicts:
    install lirc_sir setserial /dev/ttyS0 uart none; modprobe --ignore-install lirc_sir

    Make sure you run update-modules; then unload and reload the lirc_sir module after any changes to lirc.conf

  5. Test your configuration with mode2. Run mode2 as root and then press buttons on a remote. You should see some output like
    pulse 591
    space 96900
    pulse 9042
    space 2277
    pulse 591
    space 96900
    pulse 9042
    space 2273
    pulse 594
    space 96901
    pulse 9042
    space 2274
    pulse 593


    If not, make sure you've unloaded any serial related module. Check dmesg after loading the lirc module to make sure your hardware is recognised:
    $ dmesg | grep lirc_sir
    lirc_sir: I/O port 0x03e8, IRQ 4.
    lirc_sir: Installed.


  6. You're almost there! Now you can download remote codes from the LIRC website, or make your own with irrecord. I made my own, since my remote wasn't there.

    Copy the remote code file to /etc/lircd.conf

    Leave the /etc/conf.d/lircd blank or all commented out if you have the same hardware

    Start lircd: /etc/init.d/lircd
And that should be that!

Many thanks to:

Posts for Monday, August 10, 2009

It's alive!

Elbereth - my old laptop is alive! :D

It turned out that what made it look completely dead was some faulty RAM. Without those 256 MiB (the 1 GiB left is enough for my needs anyway), it boots up and works just fine :] Thanks Maq for fixing my precious lappy! :D

So what did I learn from this disaster:

  • don't panic, it can probably be fixed if you keep your cool and check everything
  • faulty RAM shows the same symptoms as clinical death ...especially since it seems that on laptops you don't get those tell-tale beeps when RAM is foo
  • it's still nicer to have your old lappy working then buying a new one
  • reliable backups are essential
  • PortableApps are a nice solution when needs be ...but, boy!, am I happy to have my KDE an Awesome back! XD
  • Amarok is not only amazing, but most probably the best tool out there to listen to free music and disover new artists and tracks
  • now I understand (again) why people use FaceBook and Flash games ...man, Windows is such a boring OS ...I actually caught myself searching for flash games! O_o
  • what people say about the ThinkPad's keyboards is true! They are fantastic!! — the bounce, the sound, the layout — it all makes sense, when you get used to it! If my dad didn't let me occasionall borrow his old ThinkPad, I'd never believe just how much of a difference there is!
  • 14.1" is enough (erm, I'm talking about the screen diagonal, BTW)
  • when Elbereth finally gives up and dies — hopefully not for at least a year! — ThinkPad T400 will be in my narrowest selection for a successor (probably called Fangorn)

hook out >> lalalalala, lappy working, eix-sync; emerge -DNu world, studying obligation law while stuff compiles...

P.S. Maq (a.k.a. MaQmigh, Dexter) is a hardware wiz, a co-worker at Cyberpipe, friend and an all-round great guy
<!--break-->

New Shoes

After reading splitbrains latest article about his new shoes and a closer look at this:

3807114243_0d8a61368b.jpg

3807116783_0dbcd73e20.jpg

(I wore those shoes through 2 summers and 2 winters.)

I decided that it might be at the time to get myself some new shoes as well (allthough I was comfortable with the fact that everyone could see my socks when I was walking around the rain issue was a tad bit suboptimal at times ;-)).

My only problem is that I'm a big fan of Vans since I was a teenager. My first really comfy shoes where a pair of Sk8-His. There's just no alternative (standard Adidas Samba shoes do it too though). So I googled arround if Vans shoes are produced under fair ecologial conditions but found no references. So I hope I'm on the safe side and my new pair lasts at least another 2 years (if you know about something which speaks against Vans please let me know in the comments).

3807919120_9e1683dc0d.jpg

BTW, nope, that the color of my new shoes somehow matches the link color I use on my blog or the Arch Linux Logo color was not intentional ;-).

Read or add comments to this article · Save to del.icio.us
avatar

Arch Linux 2009.08 & Froscon 2009

So, the Arch Linux 2009.08 release is now behind us, nicely on schedule.
I hope people will like AIF because it was a lot of work and we didn't receive much feedback. I personally like it to apply my fancy backup restoration approach.
But I'm sure if more people would look at the code we would find quite some design and implementation things that could be improved. (With uzbl I was amazed how much difference it can make if many people all have ideas and opinions about every little detail)

Later this week I'm off to the Counting Cows festival in France, and the week after that (august 22-23) I'm going to FrOSCon in Germany where I will meet some of my Arch Linux colleagues in real life, which I'm really looking forward to.

If anyone wants a ride to froscon let me know. But note I'll try to maximize my time there (leave saturday early and come back late on sunday. I even took a day off on monday so I might stay a day longer if I find more intersted people to hang out there)

Desktop Yank


Every now and then I get the urge to put up my desktop, so here it is. Not much different (like the kcheckgmail and ktorrent thing). Haven’t got to updating to 4.3 yet because I’m going to have to do a full reinstall soon (don’t really need RTM anyhow). I got a computer that didn’t come with install disks for Vista so I gotta find a way to restore vista from the restore disks created by the emachines restore program (phew). I also got a partition at the beginning of the drive called PQSERVICE that I have no idea what it does. To my dismay I have previously tried to use the restore vista disks and got a big fail. Don’t think the program is able to wipe the disk with Linux partitions. Drat. Anyways, before I veer too far, here’s my desktop:

Clean

Dirty

Ok, I lied. Just updated:

Not too bad. KDE 4.3 doesn’t start up as fast but that might be because I’m still using the same ~/.kde4 folder. Doesn’t really have to tools I need yet (web browser, sound-recorder, network manager…) but a few good bugs got rid, so it’s a good update.

Posts for Sunday, August 9, 2009

avatar

Jabber

It is still vacation and I decided that I needed something today on this not to sunny day. So after cycling trough the city and discovering that some places in Amsterdam could really use some attention in OSM (will do that later this week), I decided to setup my own jabber(XMPP) server.

After browsing trough the portage tree landed on jabberd2. It is writen in C (no java or whatever on my server!) and easy to setup. This basically means that I could register and login using kopete in no time!

Now the things I like about open protocols (and software of course) is that most of the times you can do way cooler things with it. Take the MSN (of Live) protocol for example: sending messages in plain text over the net! I prefer secure connections, all the time actually, and that is possible with jabber. It is even specified in the RFC. And not just my connection to my server, also the server2server connections are secure. I mean that is just awesome! Since I see no point in securing only one part of the route.

Now of course some of the people I chat with can still not use SSL which would mean kind of a security risk, and we do not want that. But since you can use a client of your liking I can also use OTR encryption while using jabber.

Now I already told you that jabber is open. Which means anyone with enough time (and some skills) can write a library for this protocol. Which in turn makes it easy to use for status messages of your bash scripts or for elog messages from portage! Now I have not checked on any of this but I could imaging jabber plugins for torrent/usenet deamons (informing you of your finished download) or even bootup messages from computers that you maintain informing you of several useful things!

Long story short. I like Jabber!

Edit: I am now using ejabberd (or recomandation of a fried) and the webinterface rocks!

Posts for Saturday, August 8, 2009

sms is a garbage protocol

  • no error on transmission failure
  • no failure on invalid recipient
  • no assurance of delivery to temporarily disconnected recipient

All of which amounts to the result that when you do send a text you have no idea if the number is correct, if the message went through, and in the event the person’s phone is turned off if he’ll ever see it.

Now, I really don’t care if the flaws are in the protocol, the phones or the services we pay for; what I do know is that it all makes for an appalling technology whose reliability imitates Microsoft’s infamous msn messenger where messages constantly go missing without notice. I hear all the time about messages I never saw and apparently when my phone is off for a few days there is zero chance of getting messages sent in that interval.

Compare it to email, which doesn’t have any of these flaws, and it seems incredible that we are actually paying to use this.

And that’s to say nothing of the horrendous user interface for typing messages on a phone to begin with.

This entry belongs in a comprehensive tome I have been writing in my head for a decade, detailing just how much cell phones suck. But the field is so vast that it overwhelms my ability to articulate it.

Renoise

I've been playing with Renoise since quite while now. Time to write sth. about it (this is by no means a review, rather a short recap of my impressions).

Renoise is a modern music Tracker. Unlike other sequencers like Pro Tools or Cubase, trackers follow a different approach for music creation. The arrangement is based on patterns, and the main focus is on working with samples instead of recording instruments (though, it is of course possible as well). In case of Renoise, musical information isn't represented as MIDI notes, but as hex values (which in turn represent MIDI notes of course). You can modify lots of parameters of a played sample like speed, pitch, pan, volume or even the playback offset by using different control commands in the pattern editor. This makes Renoise ideal for beat programming/slicing as you don't have to cut a loop for example into different pieces and then rearrange everything. Instead you just play one and the same loop from different positions (yes, you can jump around inside a loop in real time by just setting events which triggers the loop at different playback positions).

Here's a screenshot of how the main window of renoise looks like:

Okay, hex instead of MIDI sounds a little bit complicated at first, and I have to admit that I had a hard time at first to get used to this completely different approach of sequencing. But, after some time it gets really intuitive. I definitely recommend an external MIDI controller/keyboard to ease the setting of new events though.

Like you can guess from the screenshot, Renoise also supports automation of lots of parameters of used samples and plugins (btw. Renoise comes with a set of really nice plugins) and also provides automatic delay compensation for used plugins and can render MIDI tracks to patterns inside a session. The mixer provides enough routing options but could be optimized to be a little bit more intuitive.

But the best thing about Renoise, is that it's absolutely cheap. A license costs about 50€ - which is nothing compared to the pricing of other DAWs. If you're into music creation and maybe into electronic music and have never worked with a tracker before I can definitely recommend to give Renoise a try. It's well worth the 50 bucks and it runs on Windows, OSX and Linux!

Btw, in case you, like me, wondered how some of those Breakcore/Drum and Bass artists make their crazy music without editing their brains out in a DAW like Pro Tools or Cubase: Some just use trackers, and some, like Venetian Snares use Renoise (which makes completely sense now that I know it ;-) - the following video shows the Renoise session of the Venetion Snares track Vache):

Of course I made a test track as well. It's some kind of breakcore style electronic piece. It's not really a complete track in itself, but rather a playground for all the possibilities Renoise offers. I'm definitely plan on making some more music with Renoise in the future.

As always, enjoy (or not)!

Download: chizm_lies.mp3

Read or add comments to this article · Save to del.icio.us

Reading "Little Brother" is good procrastination

I've read about Cory Doctorow when his first novel Down and Out in the Magic Kingdom came out, but forgot about it and haven't read it at that time.

Now I've stumbled upon him again and started reading his Little Brother — in short, a novel about kids hacking to get their freedom back, because they feel too oppressed, too monitored and in short robbed of their basic freedoms in the almost Orwellian society.

It might sound like sci-fi and the first few pages did feel like it as well, but both the technology and the state of the society are almost identical to what our current lives. And, being an EU-citizen, once I got over that it's located in San Francisco and got used to some of the specifics of the slang, it's quite engulfing!

The story itself is quite exciting and I'd gobble it up in one day, if I was still in high-school, but its strongest point is how it describes all the technology we use today (TOR, PGP/GPG, SSL, encrypted disks, etc. etc.) to try to ensure at least some privacy in our online lives as well as the technology that's used to spy on us (be it just advertising cookies, credit cards or biometrics, CCTV or worse).

So far I've read halfway through it — although I wanted to read only a chapter a day, I couldn't help myself! XD — and if most books about "hackers" show them in just one light, this one seems to try to cover as many aspects as possible. From the hacker's/kid's views, the normal users', worried parents' and those who think that privacy is a small price to pay for the terrorists to be caught. It also shows the possible good as well as the bad consequences that could happen when you deviate from the norm or even fight for your rights.

It's one of those rare books that come out at exactly the right time and are aimed at exactly the right crowd — the high-school kids, as they are the biggest users of the internet, therefore the most vulnerable and also the ones that can make the biggest change. Just today I read in the local paper's NYT supplement on what all crap the advertising and profiling companies (try to) take from us on the 'net. And if it's already normal news in the papers, the shit's hit the proverbial fan already — it's high time the broader public knows about it (especially the kids).

If you haven't yet, read it — go download it for free (it's under CC BY-NC-SA) or rent it or buy it in a bookstore. Give it to a kid you know at least occasionally uses the internet (yeah, every little bugger, I know :P), share it with your friends, spread it around...

I seldom get this passionate about a book, but — by George! — if I was a kid, that book'd be a real eye-opener for me!

hook out >> have to stop reading ... must learn for exams ... have to stop reading ... must learn for exams XD
<!--break-->

Week of bash scripts – rps and commentstrip


Header

These two scripts will respectively: find if a program is running, and strip-comments from text files. The first is useful if you need to see if the program is running or if you need to kill the process with it’s id, comment strip is a good tool to use if posting configurations on forums as often developers or advanced users already know what the settings actually do.

rps

#!/bin/bash # rps - running program search ps aux | grep $@ | grep -v grep | grep -v ".bin/rps" | column -t

commentstrip

Commentstrip will display the output to the terminal, but if you got xclip installed the ‘x’ flag can be used to copy the output to the clipboard (i.e. commentstrip x file).

#!/bin/bash # commentstrip - outputs file without comments or blanklines if [ -z $1 ]; then   echo "commentstrip <*x> <filename> - outputs file without comments or blanklines"   exit fi case $1 in   x | clipboard ) shift                   if [[ `whoami` == root ]]; then                     echo "Must be regular user to copy to clipboard."; else                     grep -vh '^[[:space:]]*\(#\|$\)' "$@" | xclip -selection c                   fi                   ;;   * )             grep -vh '^[[:space:]]*\(#\|$\)' "$@"                   ;; esac

The final day of week of bash scripts… phew! I’d like to thank everyone that posted comments, and to those that stopped by this week. Enjoy.

Posts for Friday, August 7, 2009

avatar

If the auto industry makes cars like Microsoft makes Windows?

This is an old one (and I take no credit for it), but cracks me up every time I read it and I’m sure that I’m not the first. However it’s great to share, and here it is … again :)

Bill Gates reportedly compared the computer industry with the auto industry and stated, “If GM had kept up with technology like the computer industry has, we would all be driving $25.00 cars that got 1,000 miles to the gallon.”

In response to Bill’s comments, General Motors issued a press release stating:

If GM had developed technology like Microsoft, we would all be driving cars with the following characteristics:

  1. For no reason whatsoever, your car would crash twice a day.
  2. Every time they repainted the lines in the road, you would have to buy a new car.
  3. Occasionally your car would die on the freeway for no reason. You would have to pull to the side of the road, close all of the windows, shut off the car, restart it, and reopen the windows before you could continue. For some reason you would simply accept this.
  4. Occasionally, executing a maneuver such as a left turn would cause your car to shut down and refuse to restart, in which case you would have to reinstall the engine.
  5. Macintosh would make a car that was powered by the sun, was reliable, five times! as fast and twice as easy to drive – but would run on only five percent of the roads.
  6. The oil, water temperature, and alternator warning lights would all be replaced by a single “This Car Has Performed An Illegal Operation” warning light.
  7. The airbag system would ask “Are you sure?” before deploying.
  8. Occasionally, for no reason whatsoever, your car would lock you out and refuse to let you in until you simultaneously lifted the door handle, turned the key and grabbed hold of the radio antenna.
  9. Every time a new car was introduced car buyers would have to learn how to drive all over again because none of the controls would operate in the same manner as the old car.
  10. You’d have to press the “Start” button to turn the engine off.

Related posts:

  1. Windows 7 Review
  2. Windows 7 “feature”: let’s include XP!
  3. Windows 7

Week of bash scripts – grok and cdf


Header

These two scripts are two different find commands. The first (grok) will list all files in a directory recursively that contain matched text, and the second will locate a file/folder and the change to it’s directory. Neither of these are mine (though slightly edited), I’ve gotten them from the Arch forums where they have a great thread called Post your handy self made command line utilities.

Grok

This one is by rebugger and I call it grok. Syntax is:

grok <keyword> <*location>

If no location is given it begins the search in the current directory.

#!/bin/bash # grok - search file(s) for keyword # Author: Gen2ly # Text color variables TXTBLD=$(tput bold)     # Bold TXTUND=$(tput sgr 0 1)  # Underline TXTRED=$(tput setaf 1)  # Red TXTGRN=$(tput setaf 2)  # Green TXTYLW=$(tput setaf 3)  # Yellow TXTBLU=$(tput setaf 4)  # Blue TXTPUR=$(tput setaf 5)  # Purple TXTCYN=$(tput setaf 6)  # Cyan TXTWHT=$(tput setaf 7)  # White TXTRST=$(tput sgr0)     # Reset if [ -z "$1" ]; then   echo "grok <keyword> <location/file> - search file(s) for keyword (recursive)."   exit fi  if [ -z "$2" ]; then   echo "${TXTBLD}${TXTGRN} * ${TXTRST}Files found in ${TXTUND}`pwd`${TXTRST} that contain ${TXTYLW}"$1"${TXTRST}:"   grep -Ilr "$1" .   exit fi echo "${TXTBLD}${TXTGRN} * ${TXTRST}Files found in ${TXTUND}"$2"${TXTRST} that contain ${TXTYLW}"$1"${TXTRST}:" grep -Ilr "$1" "$2"

cdf

This one is by segoe that uses locate to find a file and that cd’s to the first match found.

This one I put in my ~/.bashrc:

cdf () { cd "$(dirname "$(locate -i "$*" | head -n 1)")"

Posts for Thursday, August 6, 2009

The timeless Riviera

summer_vacation_2009_flags

The beauty of the camping vacation is the enduring feeling of being on site. Air travel is very insular that way, I feel like I’m in the same place right up until I get off the plane, but when you’re on the road it’s a whole different feeling and more authentic in a sense. It’s tiresome if you have to go far to get there, but once you cross the border into where you’re going there is a real sense of expectation that gradually bears out. And you can stop anywhere you want along the way.

summer_vacation_route_2009

Locations of interest:

  • Wien (classical concert)
  • Verona (late night outdoor opera)
    Camping right near Lago di Garda – a summer camping hotspot
  • La Spezia (all day visit to the pictoresque villages in Cinque Terre national park)
  • Genova (city walk and a visit to Europe’s largest aquarium/oceanography museum)
  • San Remo (walk along the promenade and swimming at the beach)
  • Monaco (city walk, swimming at the beach)
  • Nice (walk along the promenade, swimming)
  • Cannes (walk along the promenade)
  • Ramatuelle by St. Tropez
    Camping at the unbeatable Les Tournels
  • St. Tropez (city walk)
  • Marseille (city walk)
    Camping in Aix-en-Provence – a nice town in itself
  • Les Baux-de-Provence (a splendid ancient castle)
  • Orange (the best preserved Roman theater in Europe)
  • Geneve (CERN museum, city walk)
  • Lausanne (city walk)
  • Bern (Einestein museum)
  • Zurich (natural earth museum and a robotics museum)
  • Munich (Deutches museum – world’s largest museum of science and technology)

And that’s a way to spend three weeks.

Audiophail

My Grado SR-80 headphones are more electrical tape than headphone at this point. Inexplicably, sound continues to come out of them. The wires have so many breaks that I'm not sure how this is physically possible.

Also the top of the band is also splitting apart and the pads are worn down and fall off constantly and there are pokey plastic bits that hurt my ears a lot. After prying them apart with a hammer and screwdriver to fix the wires a few times, they look like they've been through a wood chipper. I love those things but it is time for retirement.

Researching headphones can suck up months of your time if you let it, especially if you believe the bullcrap. Going to an "audiophile" site like Head-Fi is like entering a new world. I have no idea what any of the vocabulary means.

"Detailed", "neutral", "open soundstage", I can figure those out to some degree. But what the hell do "sweet" and "dark" and "thick" and "smooth" mean with regard to headphone quality? Are we talking about music or chocolate? What do "forward" and "recessed" and "transparent" and "analytical" mean? These are rhetorical questions, I don't care what they mean. I have my doubts that they even mean anything objective.

Then there are strange beliefs, like that letting your headphones run for 100 hours to "burn them in" when they're new will make them sound better. I'd really like to see that theory put to a proper scientific test. I have strong doubts that it's anything more than people's minds fooling themselves. It sounds like voodoo. At least it's not as bad as $7,000 speaker wires.

I can definitely and easily tell the difference between cheapo $5 headphones and my Grado's, but beyond that I really start to doubt that it matters. Spending hundreds or thousands of dollars on headphones seems like insanity to me.

For this reason I decided to do next to no research, and went and bought the first pair that I found online that looked comfy, got mostly good reviews, got a couple good reviews on head-fi (as far as I could decipher) and had a price of around $100. I ended up ordering ATH-AD700's from Amazon. Should be here in a week.

Posts for Wednesday, August 5, 2009

Week of bash scripts – newx and bgcmd


Header

Here are two scripts: one, that helps improve gaming performance, and the other to free up the terminal. I’ve talked about the later before, but I got a new trick up my sleeve. However, before I go any further, I’d like to point out how to use bash scripts so that they are easily accessible, yet (at the same time) out of the way.

Organization

I like to keep my bash script out of plain view. Some people like to put their bash scripts in a folder called scripts in their home folder. This works good but keeping the home folder just for documents can help reduce clutter:

Think about putting your scripts in a hidden directory in your home folder. For example, I use a folder called ~/.bin. To make the scripts executable anywhere, create a path to them in your ~/.bashrc:

export PATH="/home/gen2ly/.bin:/home/gen2ly/.bin/root/arch:\ /home/gen2ly/.bin/backup:$PATH"

To quickly enter the script directory, create an alias to them:

alias cdb="cd /home/gen2ly/.bin && ls -h"

Now, reload the bash environment:

source ~/.bashrc

To get to your script directory you can now just do:

NewX

A basic script but useful for gamers that don’t have the most powerful graphic cards. Compositing can zap game performance pretty thoroughly. Rather than digging through menus and disable compositing this command will just start a new xorg server:

#!/bin/bash DISPLAY=:1.0 xinit $* -- :1

By typing newx a new xorg server will be opened on the eighth virtual console (Ctl+Alt+F8). This will also open a terminal where commands can be entered. Typing exit will exit the new xorg server and return you to your original. Alternatively you can type newx urbanterror and urbanterror will be loaded.

Bgcmd

bgcmd will background a program so that it doesn’t overtake the terminal. I’ve written about this before but I’ve discovered how to add bash-completion to it. I’ve updated the page to reflect such:

Background a Process

Enjoy!

Ballmer at it again...

I just got done reading about and listening to a wonderful audio-bite from good ol' Steve Ballmer.  Honestly, I don't understand why Microsoft keeps him employed.

Anyway...  According to Mr. Ballmer, Linux is no longer free.  So, with that in mind, please do not try to download Linux from here, or here, or here, or here, or here, or here, or here, or here, because somehow, they're gonna charge you for it.  Seriously...  Don't even think about trying to download any version of Linux from here, because I believe in Mr. Ballmer when he says about Linux, "of course, it has a price."

Hmm...  I guess I just didn't give gentoo my mailing address for them to send me the invoice for using gentoo Linux these past 6 years.  Silly me.

All this time, I thought it was free.

Oh wait.  Nevermind.  I forgot who I was referencing all this time.

Me, being a developer, am fully aware of what has a price, and what doesn't.

Heh....  "of course it has a price."  sheesh.
avatar

SynCE 0.14 updates

SynCE 0.14 ebuilds are now available in the overlay for all SynCE packages. (See the instructions here for details)

You should have no trouble compiling and installing (of course, let someone know if you do!)

Most of the SynCE problems seem to come from configuring and using (or trying to). So before you report any bugs, please try these steps. Note: this is a short summary! Please read the SynCE wiki for full instructions.
  1. First verify you have version 0.14 of everything, using eix, equery, genlop or whatever.
  2. Backup all your Contacts, Calendar, Tasks and Files. (I use Sprite Backup that came with my Treo Pro). This is important as Windows Mobile 6 will delete all data associated with a deleted partnership.
  3. Remove the multisync group (your command may vary)
    $ msynctool --delgroup synce-sync
  4. Remove the partnership
    $ delete_partnership.py
  5. Delete the ~/.opensync and ~/.synce directories (copy any local config files first)
    $ rm -rf ~/.opensync ~/.synce/*
  6. Create a new partnership (change the name and objects to synchronise)
    $ create_partnership.py "TreoPro" "Calendar"
  7. Create a new opensync group, and add members (alter members to suit your needs)
    $ msynctool --addgroup synce-sync
    $ msynctool --addmember synce-sync evo2-sync
    $ msynctool --addmember synce-sync synce-opensync-plugin
    $ msynctool --showgroup synce-sync

    If these steps fail, check the output from --listplugins
    $ msynctool --listplugins
    Available plugins:
    evo2-sync
    synce-opensync-plugin
    testmodule
If you upgrade sys-apps/hal to >= 0.5.13 (which is now in ~x86) you will have to compile synce-hal again. This is because the hal scripts directory has changed, and synce-hal only detects this at install-time.

Errors similar to https://www.opensync.org/ticket/1131 were fixed for me by following the steps above.

Let me know of your success / failure. SynCE 0.14 will be in portage in the fullness of time!

Way to go, Arch

KDE 4.3 is available in Arch already. If the Arch MB is to be believed, it was available for install a few hours before KDE even announced it. Good job Arch devs. Arch seems to have pretty fast turnaround on new packages.

The stable release of KDE 4.3 is looking good too. I wiped my KDE profile because I think I've been running the same one since KDE 3.1 and the cruft was becoming noticeable. That helped resolve a lot of things (I can add widgets to the main panel without crashing things now).

One thing that has caused me tons of problems historically is CJK input in KDE. In KDE3 Skim worked OK, but I couldn't get it to work in KDE4. Instead there's UIM which so far has been better than Skim in terms of stability and predictability of interface.

As per a comment by knef on a recent post, you can set per-desktop wallpapers now (as in virtual desktops, workspaces, whatever you call them). You have to:

  1. Zoom out (via the cashew)
  2. Hope it doesn't crash
  3. Go into the Plasma options there
  4. Pray it doesn't crash
  5. Enable the setting to make each desktop have its own "activity"
  6. Yeah you probably crashed right here. In the off chance you didn't crash, once you zoom in you can set per-desktop wallpapers now.

Not just wallpapers, but widgets in general. This is kind of good, kind of bad. If you want to go back to a single wallpaper per desktop, you have to go back and screw with activities. Also I don't think you can "sticky" a widget to span all desktops. It's either everything per-desktop or everything global. I'd bet this will change in future versions.

But I read a suggestion somewhere (probably Slashdot) to set up a different Folder View on each desktop, each pointing to a different folder, and that's actually a great idea. You can set kwin to always open Gimp on a certain desktop and have a folder view of your pictures folder underneath, or something.

Kudos KDE devs, KDE is awesome and keeps getting better.

Posts for Tuesday, August 4, 2009

On crossover work

We are idiots. Most of us. Cause we are trained to be.

When we pick something to study we dive into it, we learn more and more in that specific area. And the world is complex and gets more complex every day, just as much as the topics people work on get more and more specific: You used to study computer science and kinda knew a lot about every branch of computer science but nowadays every branch in itself is pretty much its own little science with its own language, its own concepts and its own … well bullshit.

And that is not a computer science specific problem: In order to know enough about one specific thing we dive into that topic. If we don't we might not be able to solve our problems or we might not pass our exams. Even schools (at least here in Germany) allow pupils to drop certain things they don't like, allow pupils to specialize.

Specialization is something our society loves: Whenever a problem emerges we ask the experts, the specialists, cause they know what to do, right? They've worked so much on that topic, they will make the right recommendation or decision.

But our focus on experts has a dark side, too: We lose the overview; our focus on the details blurs our views on the general.

The problem is simple: Everything is connected, everybody is connected. I have said this in a post before: Nobody is an island. And our actions aren't either.

I studied computer science and philosophy. For my dissertation I am currently diving into social science and systems theory and guess what? It's fun. We tend to chose simple ways to learn new stuff, we choose areas that are connected to what we already know, but I can just encourage you to make a leap: Learn something that has nothing to do with your normal area of expertise.

Don't listen to naysayers telling you what a waste of time it is, how unproductive it is. When getting into a different topic you learn now ways to think, new perspectives and you will start seeing your old knowledge in a new way, you will find new connections, new ideas on how to mix stuff up will form.

Focussing only on one thing limits what you can do in that area. The more you specialize on one topic the less creative you will be in that area. We need input to be great, new ideas, perspectives, thoughts. And those come from connecting the unconnected.
avatar

Hello Planet KDE!

Hello all KDE users out there! If nothing got borked in the process this blog post should be aggregated in the Planet KDE feed :)

Obviously, this is my first post, and that warrants some introduction to any new readers out there.

My name is Dion Moult, I am a KDE-user (running on my favourite flavour Gentoo) and have joined Hans Chen in his path to become a KDE developer. My posts would document my progress and should be interesting as we do come from very different backgrounds. I actually described a bit about myself in my first post about “The Road to KDE Devland Moult Edition #0” – so I don’t see any benefit of repeating myself here.

The posts that appear here will be manually filtered from my blog posts with the tag “planetkde” – this means that you will not benefit from the vibrant and unpredictable publications on the full monty of my blog, but this should still add to the community that KDE has managed to create – something unquantifiable relative to “lines of code”.

In case you bothered to click that link to my first road-to-kde-devland post, you would know that I do graphic design too – and since nobody likes walls of text, here is a blue-ish wallpaper dedicated to KDE (not really, it was a design I had started on then stopped because I didn’t like it) and of course, it’s made with GIMP (sorry haven’t tried Krita yet):

horrible

In a nutshell: “hello” (I really should apply for law school or something)

Related posts:

  1. The Road to KDE Devland (Moult Edition) #0
  2. If the auto industry makes cars like Microsoft makes Windows?

Computer e-Recycling (an I.T. WTF Odyssey)

Story Time: Computer e-Recycling

an I.T. WTF Odyssey

I had the displeasure of working at an erecycler several years ago. Even watching stuff come off the trucks, it was very hard to get anything before it was utterly destroyed by the yard goons. Inserting a forklift blade into a 19″ rack cabinet was common practice and I witnessed on numerous occasions the dropping of them in this fashion. Once, they even rolled a forklift off the ramp. These events were always followed by a flurry of Spanish profanity and I usually had to check my pants for continence afterward from laughing so hard. It is a miracle nobody has ever been seriously maimed there to the best of my knowledge.

The highlight of this job experience was when the greedy goons resold a defective Siemens blood handling instrument of some sort that was sent specifically to them to be destroyed (as was EVERYTHING, in theory). The serial number was traced back to them and there was an all out shitstorm. A team of inspectors was flown out from Germany. The goons put on a particularly hilarious show buying hardhats, safety vests, warning signs, identification badges, and more. Somehow, they kept the contract (it was probably just “check you ass” on Siemen’s part — dumping obsolete X-Ray, MRI machines, medical waste, etc for free must be hard to pass up) and all of this change disappeared within a couple days.

What is remarkable is that the above business got multi-millions of dollars of inventory for FREE every year. I really can’t think of any other business model like it. The owner is completely incompetent and morally bankrupt. Very little was put back into the company’s facilities, employees (except maybe a couple at the top), or development. None of the employees had an IQ above room temperature, and everybody seemed content to keep it that way.

There is a happy ending though. A skid of IBM RS/6000 7012 systems from Intel came in at one time. Among the 20-30 machines was a lone -397 that I snagged. This is my favorite collectors box to date (it is the same POWER2 CPU type used in the famous Deep Blue super computer).

Share and Enjoy: Digg del.icio.us Slashdot Facebook Reddit StumbleUpon Google Bookmarks Print this article! E-mail this story to a friend! Turn this article into a PDF! TwitThis

Related posts:

  1. One Small Step for QT, One Giant Leap for Free Software QT Software, under the graces of Nokia, has released the...
  2. Retrocomputing for Fun and Profit Buy Old Computers ??? Profit What is retrocomputing? I...
  3. Xen 3.3 in RHEL/CentOS 5 and more Link Aggregation Fun RHEL 5 includes the now ancient Xen 3.0 hypervisior.  A...

avatar

Introduction to Twelf

I always spend some time during my summer holidays learning a new programming language. First was Python a couple of years ago, then Ruby and finally Haskell last summer. This summer I have decided to learn Twelf. The only reason that I have heard of Twelf is because of my friend, Jesper Louis, who keeps talking about it, due to his use of it for our Ex-SML project and this really annoyed me because I did not know what he was talking about.

Twelf is very different from any other programming language that I have ever played with. Twelf is an implementation of the logical framework – called LF – and it is used for logical programming and to formalise programming language theory.

From Twelf’s website:

Twelf is a language used to specify, implement, and prove properties of deductive systems such as programming languages and logics. Large research projects using Twelf include the TALT typed assembly language, a foundational proof-carrying-code system, and a type safety proof for Standard ML.

Twelf’s website is very useful and is, honestly, the only website available on the internet with a plethora of information about the language. You can find some decent tutorials if you look at some of the university websites out there (CMU especially has some very interesting slides about Twelf on their website).

I am going to give a short example on how to implement the factorial function in Twelf and hopefully this will enable you to understand Twelf a bit better than before you read this post.

Example

There are tons of small examples on Twelf’s website, but here is another one. We are going to implement the factorial function in Twelf. The factorial function is, for functional programming languages, what “Hello World” is for imperative programming languages – the first function you will see when you open a book about the language.

The factorial function can be defined recursively as this:

equation

That is easy to implement in most programming languages. In a functional programming language, like Haskell, it can be implemented with pattern-matching so:

factorial :: Int -> Int
factorial 0 = 1
factorial n | n > 0 = n * factorial (n - 1)

In particular, notice how much the pattern-matching method looks like the formula listed above.

It is nowhere near as easy to implement this in Twelf. The first issues we will encounter are:

  • Twelf has no knowledge of numbers. We really take that for granted in any modern programming language (for a very good reason: adding numbers to a language every time you have to write anything feels a bit like reinventing the wheel over and over again);
  • Twelf has no arithmetic operators at all. We need to define all of those and the numbers via the type-system of Twelf.

The Natural Numbers

The first thing we need to define is the natural numbers (equation). We are going to do that with Peano axioms which map well into the Twelf type-system. First we need to define a type for the natural numbers, and then two relations for it.

In Twelf we write:

nat : type.
nat/zero : nat.
nat/succ : nat -> nat.

This says: nat is a type, nat/zero is of the type nat and is going to be the type that represents the natural number zero, nat/succ is the successor of the former natural number. This way we have zero as nat/zero, one as nat/succ nat/zero, two as nat/succ (nat/succ nat/zero) and so on. It is already beginning to get a bit disturbing, right?

We also need to be able to perform some basic arithmetic on the Peano numbers. We have to implement both addition and multiplication (since the latter requires the former) to be able to finally implement factorial in Twelf.

Addition on Peano numbers is defined recursively as:

equation

We need to convert this from the infix-notation above to something that fits into Twelf (Twelf is able to do infix-operators via the %infix-statement). Putting it together in Twelf will result in this:

plus : nat -> nat -> nat -> type.
plus/zero : plus nat/zero B B.
plus/succ : plus (nat/succ A) B (nat/succ C) <- plus A B C.

Multiplication is defined using addition and this is also done recursively, like so:

equation

And in Twelf:

mult : nat -> nat -> nat -> type.
mult/zero : mult nat/zero B nat/zero.
mult/succ : mult (nat/succ A) B C' <- plus B C C' <- mult A B C.

The only operation that we are missing is to be able to do equation, which can easily be defined with:

pred : nat -> nat -> type.
pred/s : pred (nat/succ A) A.

This works for what we are going to use, but it won’t work for someone who decides to do a equation, since our system does not have any knowledge of negative numbers. However, we are never going to reach this in our factorial function (since we have a case for equation) and we are simply going to ignore this issue for now.

The Factorial Relation

The final relation that we have to define in Twelf is our factorial-relation, which can be defined in the form of the previous defined relations thusly:

factorial : nat -> nat -> type.
factorial/z : factorial nat/zero (nat/succ nat/zero).
factorial/s : factorial N R <- pred N N' <- factorial N' R' <- mult N R' R.

Testing

Twelf has a very nice declaration called %query where we can query (Prolog-style) a relation for a value. In most programming languages, we are used to functions being one-way. For instance, a function takes an argument and returns a value. With Twelf you can do it the other way around and ask for the input value for a given output value.

To query Twelf for the value of X in the equation equation, we write:

%query 1 1 plus (nat/succ (nat/succ nat/zero)) (nat/succ nat/zero) X.

And Twelf will reply with the answer:

---------- Solution 1 ----------
X = nat/succ (nat/succ (nat/succ nat/zero)).

One thing I discovered pretty quickly after I started playing around with Twelf, is that it becomes really tedious to work with Peano axioms like this. To make it a bit less annoying you can do the following:

0 : nat = nat/zero.
1 : nat = nat/succ 0.
2 : nat = nat/succ 1.
3 : nat = nat/succ 2.

And so on. This makes it possible to use “normal” numbers in your queries, but the answer will still be in the annoying format.

Getting the result of the factorial relation can be done in the same manner. With factorial we have two interesting cases. One where the input is zero and one where the input is larger than zero.

The output from Twelf with equation and equation:

%query 1 1 factorial 0 X.
---------- Solution 1 ----------
X = nat/succ nat/zero.

%query 1 1 factorial 3 X.
---------- Solution 1 ----------
X = nat/succ (nat/succ (nat/succ (nat/succ (nat/succ (nat/succ nat/zero))))).

Both results are correct, but you will soon begin to hate the output-format – especially if you start trying with an equation larger than equation.

Conclusion

Twelf is fun to play with, but I do not have much use for it other than to be able to read the code. It is worth playing with if you want to try “a completely different programming language”.

We are going to use it in the Ex-SML project (a Moscow ML fork using LLVM as backend) to prove that the internal Lambda language is actually type-safe. Currently Moscow ML converts SML into the (currently) untyped Lambda language before it is turned into bytecode that runs on a virtual machine named Zinc. Our problem is that LLVM’s assembly language is typed, so we actually need to keep the type-information during every phase of the compilation.

Planet Larry is not officially affiliated with Gentoo Linux. Original artwork and logos copyright Gentoo Foundation. Yadda, yadda, yadda.