Planet Larry

What the open source community can learn from Devops

For the sake of getting a point across, I'll simplify some things.

First, a crashcourse into Devops...

A commonly used organisatorial idiom used in tech companies is that of developers and operations.

Developers:

• develop a product
• improve their product based on feedback from production usage

Operations:

• put the product in production, making it available for users/customers
• give feedback to devs

Experience shows this model often falls short. 'Dev' and 'Ops' being too artificially separated from each other, resulting in improper communication, clashing procedures and tools,
resulting in devs disliking ops ("we need to push this out to users, ops are holding us back"), and the other way around ("again new code that will cause trouble, when will they finally release something stable")
It doesn't take a genious to see this is pretty ineffective. There's a better way: integrating and reconciling dev and ops, so that all involved know the hard parts of each others' jobs, and in fact letting each other do the others' job. (developers being responsible for their own checkouts, ops working on the code, etc). Most of all it's about culture over processes. About being smart and nice human beings.
The exact methods are still being experimented with and preached about under the name of "Devops".
There is a really good Devops explanation online, with more details. Read it.

Often enough we're talking about teams working for the same company, usually under the same roof, so it isn't too terribly hard to implement these visions.

Now, let's look at the open source community

Open source developers ("upstream"):

• develop stuff
• improve their stuff based on feedback from end users

Distributions ("downstream"):

• package software and make it available to end users
• get bugreports, which often get forwarded to upstream

Looks familiar?

The problems are similar too...

Some upstreams:

• like to use "weird" (home grown) build systems
• violate FHS
• use home grown packaging systems. Languages and applications with plugins like to do this
• mix bugfixes, security patches and feature additions in the same code branch (often there is not enough manpower to maintain them in separation, and the need for it is dependent on how/when downstreams ship it anyway)
• run into the chicken/egg problem: they need to release software to have it shipped and tested, but it should only be released after being properly tested. ("Release early, release often" alleviates this, but it's not always that easy)

..making it hard for downstreams.
Even for each other: unannounced/frequent API changes come to mind.

Dowstreams, often:

• Lack discipline and/or tools to properly report back to upstream
• Have to make hard choices. Not shipping software at all or patching beyond recognition
• Don't contribute patches back to upstream. Posting them on some obscure albeit "public" mailing list or code archive isn't the most effective either

Nothing pleases an upstream more then complaints from end users running into problems that only happen witch patches applied by the distributor (patches that are deemed necessary to make the app work properly in the distro. The irony..) Runner up is getting plenty of complaints from users, but no patches.

Some distributions focus on shipping "only stable software", causing them to be obsolete by definition. (Time to production often extends in the order of years), and are forced to apply so many patches that they are essentially forking their upstreams.
Other distributions limit their role to giving you the real open source software experience in it's current state, and that state is not always pretty.

but they are much harder to solve

• Upstream and downstream are separated much more, resulting in very little communication between both parties. So the incompatibilities manifest themselves even harder.
• Among distributions, there are very different visions on and implementations of tools and processes. Pretty much each distro has a vision which separates it from the others.
  Among upstreams, there are as well some different ideas on how things should be done. Luckily enough upstream developers agree on some things. But there are some "clusters" doing things their - often radically different - way (freedesktop.org and suckless.org come to mind)
  The amount of incompatibilities is pretty much the carthesian product of the amount of distributions with the amount of "different visions" among upstreams
• Despite their differences, some upstreams and downstreams actually do have some common ground, but as they don't involve each other in tools nor processes, they hardly benefit from each other

So, in contrast to popular belief, open source is not a magical wonderland where everyone works nicely together.

Tech companies are usually on their way if they understand and can introduce agile and devops, but I think in the open source ecosystem it's much harder to bring unity.

Luckily, some smart people are already working on bridging the gap between up- and downstream, and between each other.
some examples:

• transifex.net provides a common translation infrastructure and service
• launchpad.net provides code hosting and cross-project issue tracking

I also think about Fosdem's cross-distro miniconf and the freedesktop.org project, which encourage closer cooperation between different downstreams and desktop projects, respectively.

I don't know if we should try to go much beyond "some" common infrastructure and some best practices. People will always have different opinions on how things should be done. And that's a good thing, it's the very definiton of the open source community: scratch your own itch.
But at the very least, I find it an interesting topic.

Linux Sea: log file management and backups

I’ve added two more chapters to the Linux Sea book. The first one is about Log file management, the second one about Taking Backups. They’re far from finished, but I thought that those two topics are important for day-to-day Gentoo usage and shouldn’t be left out of the Linux Sea saga.

Exherbo Repository Update

I have updated the exheres for the Go Programming Language. Hopefully it will work now. It’s the first one I’ve written from scratch so odds are it won’t work.

I have also started, but not finished, an exheres for LDC. A D programming language compiler. It’s actually part of LLVM. Either way it’s all here:

http://github.com/steveno/exheres

Patches are recommended!

cvechecker 0.5 released

A new intermediate release of cvechecker is now released. The tool is reported to build properly on NetBSD and FreeBSD as well (although much user experience there is still welcome), introduces a cvereport command (example output), has lowered its initial dependency requirements and pullcves now only loads the CVE XML changes in the database, rather than iterating across all CVE XML entries.

Many thanks to Nigel Horne for his continuous testing/hammering on the tool.

Lazyweb: Simple GNOME backup solution

Dear lazyweb, I’m currently looking for a simple GUI-based (GTK) backup solution. A view key requirements:

• I want to back up to a USB drive and want a graphical reminder in case it’s not plugged in
• I want to back my $HOME dir into an encrypted archive
• My /etc and a few other files are to be backed up in a different, non-encrypted archive
• I’d like to keep around a few iterations
• Simple restauration of a whole image or single files from the backup.

Suggestions?

Tech tip #7: Browse Amarok’s embedded MySQL database.

Amarok can probably be called one of Linux’s flagship programs. However since the upgrade from Amarok 1 to Amarok 2 there have been quite a number of controversial changes. One of these changes is that Amarok switched from an SQLite database to a MySQL database to store song information, however whether or not this was the right move is not the topic of this post (why yes, it was a good move, thanks for asking).

With this new database, users were given two choices – an external database or an embedded database. The embedded database was created to simplify the setup for users who weren’t comfortable with the idea of manually setting up a MySQL database. But what happens if you have an embedded database and afterwards you do want to mess around with it and look inside? For whatever purpose be it bugfixing, locating a specific bit of information, or bulk song management (nothing beats a good query!), sometimes you’d want to do this.

Amarok stores its embedded database information in $KDEHOME/share/apps/amarok/mysqle/amarok/ – where $KDEHOME is usually ~/.kde. So as long as you have MySQL setup elsewhere, all you have to do is create a blank database, and dump all of these files where MySQL stores its information. This location is MySQL’s datadir, which is set inside the my.cnf configuration file, normally placed in /etc/mysql/my.cnf. In a regular install, your data dir will be in /var/lib/mysql/ – and will contain one directory per database. So just copy over Amarok’s database files into the database’s directory. The final step is to ensure the files are owned by the mysql user, done by chown mysql:mysql.

Now you can browse the database normally through your preferred method (command line, PHPMyAdmin, or other MySQL client)

That’s it! I hope this is useful to somebody.

Related posts:

1. Tech tip #4: Copy a random set of files from a directory.
2. Tech Tip #5: Rotate a video by 90 degrees with mencoder
3. Playing a song as a background process in Windows

Ogg-seeding bovines and ease of use

Every time I see someone seed an Ogg/Vorbis album on Jamendo's trackers I get that urge to stretch my arms all through the wires and hug whomsoever is on the other side impersonated by that IP my BitTorrent client shows and shout through my monitor: "Thank you! Thank you for caring."

With that out of my system, let's focus on code. I promissed a while ago to write a KTorrent script to help share and seed free (i.e. mostly CC) music, but my work has been stalled by KTorrent not yet supplying the API calls I need. This doesn't mean nothing I'm doing nothing though!

What I did do so far is create a project on Gitorious^[1] to host it — Dashing Freemooina Cow^[2] — and as a subproject a small script called "moo-cow".

Dashing Freemooina Cow is part of a bigger plan to make sharing free music as easy as possible and will initially only handle Jamendo albums, but will be later expanded to any and all free netlabels I can find. I hope to add more advanced features like creating and uploading yet non-existing torrents to trackers and integration with Libre.FM as well.

But until I get my wish concerning the API, you'll have to make use of the spartan shell scrip that is moo-cow. To make use of the script, just download it and edit the MUSIC_DIR to point to where you keep (and seed!) your music; write Jamendo album ID's^[3] each in a sepratate line into album_list and run moo-cow.sh. I've already tested it on about 50 albums and it works!

...so, get the code, download your albums, share and seed to others and don't forget to be dashing! ;)

hook out >> sipping recycled tea and off to bed...

Vim :ruby and :rubydo scope

Note to self. In old Vim (tested in 7.2.320), I could do this:

:ruby x='foo'
:rubydo $_=x

Now every line in the file says foo. But in Vim 7.3 I get an error:

NameError: undefined local variable or method `x' for main:Object

The scoping rules for Ruby in Vim must have changed somewhere along the line. I was abusing this feature to do some handy things, so this is sad.

A workaround is to use global variables in Ruby instead. So this still works:

:ruby $x='foo'
:rubydo $_=$x

Phew.

Beware of warnings about default Apache2 config for PHP

Planet PHP recently popped up a post by Ilia with a warning about the configuration of PHP using AddHandler instead of AddType.

Now I’m all for people publshing warnings about security issues, but they really should at least read the official documentation to check their information first.

In this case, to summarise the documentation:

• AddHandler is used for server-side content handling – it associates a handler with the specified content
• AddType is used for determining the Content-Type in relation to the client request (ie. the default Content-Type identified to the browser)
• BOTH obey multiple extensions, but a response can only have one Content-Type while a file can be handled by multiple Handlers.

This means that while, when using AddType, “test.php.gif” no longer works, “test.php.something” will (assuming .something doesn’t have an associated Type), because .php is the last extension encountered which has an associated Type. So the so called “fix” doesn’t really fix the problem at all.

Additionally, when using AddType, instead of the default Content-Type being text/html, it becomes “application/x-httpd-php”, which is technically incorrect and may result in your website not being viewable in browsers or by search engine bots.

If you really want to make it so that only files ending with .php are handled by PHP, then you should use SetHandler instead of AddHandler. In fact, this is what the current official PHP documentation recommends.

qemu monitor cd change

I’ve been playing around with kvm (which uses qemu) to try out other operating systems and Linux distributions. Up until now, little progress on that part (not because it is difficult, just little time) but there are a few things worth mentioning. For this post, let’s start with a quicky on CD changes.

qemu’s integrated monitor is very nice and powerful. To go to the monitor from inside the vnc session, press Ctrl+Alt+2 (to go back, use Ctrl+Alt+1). Then you can query for attached hardware, add new devices, remove others, cpu’s, etc. Something I found necessary was to switch CD/DVD images. With info block I found the device. I then ran eject ide1-cd0 followed by change ide1-cd0 /path/to/new/image et voila, new CD available.

Getting SmartCard reader working under Gentoo and SSH authentication via OpenPGP

About two weeks ago I ordered a SmartCard reader and got it to work. It's really nifty and I now use I use my Fellowship OpenPGP/member card to sign (and encrypt) my e-mail and log via SSH.

Flameeyes has already reported how he got his working under Gentoo and in short my installation just partially deviates from it, but I'll add more how to chose and use it as well.

First things first — getting your hardware. You can safely skip the next paragraph and just pick one from this list.

In Slovenia we're not that hot on SmartCards, so anything else then ActivIdentity is pretty hard to get and for my taste those are too clunky. So what I did was call up the local importer Crea and was completely taken aback by the treatment I got! I know this sounds like an advertisement, but it's not. It's just that rarely never have I met such a competent and friendly service. Not only do they know about GNU/Linux, they test every model they sell on it as well, e-mailed me a bunch of useful links and even suggested a solution that I didn't think of before. This was a trully nice experience :]

Now that we got the HW it's time to set up the system. In my case I had GnuPG already installed and emerged just pcsc-lite and ccid. You need GnuPG for the obvious reason of handling the GPG/PGP keys and while it is reported that many OpenPGP card readers should work with pure GnuPG, for me this didn't prove the case. What I needed to do is to get PCSC-Lite middleware and the CCID driver. Note: USE="pcsc-lite" pulls in the ebuild for sys-apps/pcsc-lite, but you still need to emerge app-crypt/ccid yourself.

So here's the list of the ebuilds I used:

• sys-apps/pcsc-lite-1.6.1
  with USE="hal usb -static" (on HAL vs. USB flags: I have tried both and they both worked flawlessly)
• app-crypt/gnupg-2.0.15
  with USE="bzip2 ldap nls pcsc-lite smartcard -adns -caps -doc -openct -selinux -static" (if you set pcsc-lite Portage automatically pulls in sys-apps/pcsc-lite as a dependancy; no idea why there isn't a flag to do the same for app-crypt/ccid though)
• app-crypt/ccid-1.3.13-r1(because 1.3.11 doesn't compile)
  with USE="usb -twinserial"

Note that I had to pull in CCID from the testing branch because the stable didn't manage to compile, the other two are from stable branch. (Update: app-crypt/ccid-1.3.13-r1 is now stable) Also the installation method on other distributions varies of course and the PC/SC middleware package has some other names as well.

The most delicate part of course is getting the key(s) on the card. Probably the best HOWTO resides on Fellowship Wiki — I've followed it with just a few alterations, namely:

• take into account that you are using GnuPG 2.x, so you don't have to kill the agent while generating subkeys;
• the HOWTO presumes you already have a few keys, so don't be confused if there's a key extra which you don't have (e.g. the already existing auth key);
• adding an auth key to the OpenPGP card is not handled by the HOWTO and is done by the addcardkey command in the gpg --edit-key interface. Then when asked which kind of key, you just select Authentication key and you're almost set to log into SSH sessions with it!

If, as me (and Flameeyes), you get problems with gpg-agent and/or scdaemon not running (basically the most common problem), it's easily solved. In my case, I edited the /etc/kde/startup/agent-startup.sh file to include this loop:

if [ -x /usr/bin/gpg-agent ]; then
  eval "$(/usr/bin/gpg-agent --daemon --enable-ssh-support)"
fi

The good bit about it is that it works flawlessly throughout KDE and the whole X session — KMail, Dolphin, Kopete, etc. etc.; the caveat though is that if I want to use OpenPGP in a pure TTY (≠ terminal emulator) at the same time I have to kill gpg-agent and run that loop by hand in that TTY. If you happen to mix X with TTY often, you should try Flameeyes' solution with a wrapper. Our methods differ because one's better for one scenario and the other for another — chose whichever suits you better.

As a final treat — authenticating SSH sessions (i.e. logging via SSH with just your OpenPGP card), which is a most cool thing indeed, I followed Greve's instructions, which basically boil down to:

1. make sure you have created the authentication key on the OpenPGP card as explained above. To check that it's working run gpg --card-status and ssh-add -l and see if the Serial number of the first output matches with cardno. of the second;
2. run ssh-add -L to see the SSH public key(s) and copy the one which states the (right) cardno. entry;
3. log onto the server you wish to authenticate to with OpenPGP and paste the SSH public key into ~/.ssh/authorized_keys and there you are!
4. now log off the server and the next time you SSH to that server you'll be using your OpenPGP key and should at most be asked for your PIN.

This trick with OpenPGP authentication works also with Git over SSH, as e.g. used by Gitorious.

Big help provided by: bremner and others on #GnuPG; ph3-der-loewe, jhelwig and jg71 on #FSFEurope; thiago on #KDE and last but not least Flameeyes on his blog and via XMPP.

Flameeyes and me have already decided to update the Fellowship Wiki Card HOWTOs on this matter, but we've still to find time for it.

hook out >> sipping Ceylon Vanilla Bourbon tea and either hacking on moo-cow or studying ...will see
<!--break-->

Application design polish.

Free software is great. Everybody loves free stuff. However there’s one common flaw experienced by a lot of free software – they look ugly.

The reason behind this isn’t because we have too many programmers (yes, we know you never have enough programmers) and have too little artists – no, the problem is a lot more subtle. The real problem is that there is no clear hierachy within the artists. There is no control. There is no clear structure, focus, and branding. The question so many artists fail to ask ourselves as a contributor to free software is – What do we want to communicate?

To illustrate my point, I would like to use Ubuntu as an example. Regardless of your prejudices for the distribution and/or Canonical, they did do something right – they have a brand. They have a clear, recognisable pallette and style – from colourschemes to typefaces. Why don’t you see it for yourself: go and visit Ubuntu.com. Notice the colours. Notice the icon styles. Notice the typography.

Another example of a project taking the steps in the right direction is KDE and their Oxygen iconset + plasma "Air" attempt. However there is still far to go.

However the issue does not lie with such large FOSS projects such as the above mentioned. Instead the real problem lies with smaller software and application created by smaller developer groups. The reason is because these small applications rarely have to worry about problems such as branding – instead they have to focus on creating an elegant application. Design elegance can only rely so far on the design of widgets in the UI toolkit used. The rest is really up to the developer. Allow me to give a quick visual example of Blogilo, a blog client which I’m using to type out this post. Take a look:

The untrained eye would not see any problem with the screenshot – however the application design above screams complexity. There is no elegance. There is no simplicity – no "flow" (a clear step by step separation of functions). A blog client is not a complex application like an IDE. It exists for you to add, edit, and delete blog posts. Nothing more. When stripped down to its basics, a blog client is naught more but a rich text editor with a few extra options. Instead we have frames within frames, accordion panels, tabs, and buttons strewn about. Overkill, in my humble opinion.

Design polish is a very hard topic to separate what is ugly and what isn’t. It’s blends over into many neighbouring topics such as usability, a macro-view of marketing (in this case, Blogilo is part of KDE), and functionality. If you are interested, however, I would like to direct you to this very interesting blog by Troy Sobotka, one of the folks behind Ubuntu, who discusses this in much more clarity and detail than I am capable of.

Related posts:

1. thinkMoult blog design updated.
2. KDE.org relaunch with a brand new design!
3. Ubuntu 10.04 LTS (Lucid Lynx) – packed with goodies.

Windows Powershell Profile

At work, like most of this world, I’m forced to use Windows. Which isn’t always a bad thing just not preferable. At the same time though Microsoft has made this much easier for me by introducing Windows PowerShell. It’s actually been around for quite some time and I’ve actually been using for a while as well, but only recently have I been really getting into it and learning more about.

One of the nice things about PowerShell as opposed to cmd which you’re used to in Windows is that you can configure it just like you can Bash. Well, I have of course, spent some time setting mine up to basically work more or less exactly like I like my Bash setup to work. I keep in my dotfiles on GitHub if anyone wants to steal some code from it.

http://github.com/steveno/dotfiles

Migrating to XMPP/Jabber (and AIM woes)

Alright, so I've decided to finally drop proprietary IM protocols.

To some extend because promoting free software and open standards while at the same using proprietary protocols of companies that don't respect your privacy is a bit, well, let's be honest, hypocritic. On the other hand though because I've increasingly grown fed up with WLM/MSN/ICQ/YIM/AIM, their quirks and the inability to provide me with what I want.

I've already stopped using YIM and AIM with the rest following soon. What I've learnt from this so far is how very scary proprietary IM actually is. Since (for now) I still use Yahoo as my spambox e-mail provider, I just disabled the messenger component and it was very simple.

But when I tried to delete my AIM account, stuff got difficult. Brace yourselves, this gets pretty ugly! First of all, the general AOL's idea on how to "cancel AIM" is "just stop using it". The second odd thing is that they have more then one official help/support portal (yes, portal. it's big!). That being said the official help I found on how to cancel an AOL account was that if you're a paid customer, it's just a click away; if you're a sad sap who got the free account only to communicate to some other bloke on AIM, ...weeeeeheeheheheeeell, then it depends, maaaaaybeee you can delete it, maybe you can't. As luck has it, in my case (and in case of many people out there) my free account doesn't allow me to cancel it. A quick search on the aforementioned help portal says it could be because I have a child account. Which is kinda odd, since a child account, as I understand it, is a subaccount made by an adult person with their own full account and I can remember registring my account while already full age. Setting the problem aside that AIM just called me a bastard (= a parentless child), the problem stays that I cannot cancel my account. Of course, live (= phone, e-mail etc.) help is only available to paid customers. Well, I'm cooked there. I searched the web a bit and found out one of many self-help groups which lists a few solutions. For now I'm trying the contact-AOL-UK-and-hope-they-comply method, but if that fails — people report it fails more often then not — they go so far as to suggest to break AOL's ToS and wait for them to suspend your account and then not log in for half a year. ...bloody hell!?!?! The only half-way sure way I can get my AOL account removed is by contract breach?!? That alone should send shivvers down anyone's spine! And people are wondering why I'm against such things.

Below is the notice I'm sending to people on my contact list to explain why I am migrating and informing them of the benefits the XMPP protocol brings:

This is just a friendly notice that I'm leaving <insert_proprietary_IM> (and other proprietary IM protocols). If you want to keep in touch via instant messaging you can add my XMPP/Jabber ID to your contacts: matija [döt] suklje [ät] gabbler [døt] org or you can always send me an e-mail at matija [æt] suklje [doŧ] name.

There are many reasons why I'm switching to XMPP (Jabber) and here I'll list just a few of its advantages.

XMPP is:

• open standard — XMPP is the only IM protocol that is an internet standard and it's free software as well, so anyone can use, adapt or write from scratch their own XMPP client, server or other program that uses it.
• decentralised — so it doesn't matter on which server you are, you can always keep in touch with people on other servers, as is already true with e-mail (e.g. if you have GMail, you can just add my Jabber ID to your contacts, since GTalk is just Google's XMPP server); this also means you can chose any server you like best or even build your own and you won't lose the ability to chat with your friends. So you're not locked to just one provider.
• extensible and flexible — XMPP stands for Extensible Messaging and Presence Protocol, so you can easily use it to provide additional functionailty. E.g. people already use it to (micro)blog, get news and status updates, issue commands to remote computers, collaborate, play games over it and all sorts of stuff.
• security — the protocol itself uses industry-standard TLS and SASL for security and many XMPP servers pledge to never log your what you chat. That is not something you can expect from proprietary IM like MSN/WLM which stores whom you chatted with and what eternally (sic!). This is also a reason why even though you can use GTalk, I would advise you to chose a server that expects your privacy — Jabber.org and Gabbler.org are such examples, but there's many more. And since you can always build your own, you can be in complete control of your own data.
• can communicate with other IM — if you desperately need to use other IM protocols, you can just tell the XMPP server to forward the messages to and from your other IM accounts.

Well, that's it from me. If you decide that a) this sounds good; or at least b) you'd like to keep in IM contact with me, here's a (non-complete!) list of XMPP servers where you can make your account: http://xmpp.org/services/ (Jabber.org is the oldest and most popular); and add me: matija [döt] suklje [ät] gabbler [døt] org

There is also a great number of clients (programs to connect to XMPP) you can use: http://xmpp.org/software/clients.shtml (I use Kopete, but chose whichever you like best; there's even web-based clients if you're in a cybercafe)

In case you got really excited about it, you can also run your own server: http://xmpp.org/software/servers.shtml

Cheers and hope to chat with you someday again,

Matija "hook" Šuklje

If anyone has done or plans to do a similar move, I'd be happy to hear about your experiences.

hook out >> hmmm, tea with honey's nice, but sadly honey usually overpowers the delicate taste of tea
<!--break-->

An unwinnable battle

For as long as I follow discussions and event around the internet and similar technology (networked databases, datamining &ct) the topic of privacy has been quite dominant (especially here in Germany!). Privacy itself is not a new concept, we’ve had the option to put curtains in front of our windows for a while now, we could close our doors and we could decide who to share our holiday pictures with. We had a lot of privacy and whenever a government agency or other entity was stepping in to rob us of it, there were complaints, demonstrations and public disobedience.

Now with the Internet being very dominant in many of our lives, the discussion of privacy comes up almost weekly: Facebook, 4Square, Google Street View … we could go on for ages here and list all the different “Privacy Disasters” we’ve had in the last year or so. But lately I’ve come to the conclusion that it’s a battle that cannot be won. Why? Because we’ve said it for years now, just in a different context.

Let’s step away from privacy and look at a different topic: DRM. DRM stands for Digital Rights (some say Restriction) Management, DRM is what tries to make copying a DVD or a video game not simple, DRM is what locks you out of the music you bought when you buy a second computer. What DRM does is (quoting Cory Doctorow here) “trying to make bits harder to copy” and as Cory so precisely nails:

“There is no such thing as a copy-proof bit. There aren’t even copy-resistant bits. Copying is what bits are for. They will never, ever get any harder to copy.

If you want people to be able to listen to something at home, or watch something at home, you lose control over it. There are always ways to intercept data and copy it, even if you think it is encrypted, take a video file for example: Let’s say you encrypt it with a secret key so people can only play it after having contacted your servers and authenticated to get a key to unencrypt it. That’s safe, right? Except for the fact that at some point the data has to be decrypted to be playable on some sort of screen and at that point somebody can intercept it and save the unencrypted data. Even if you move the encryption to the TV, people will just start soldering a few contacts in and get the data some other way or they’ll use cameras to save the image. And yes, the quality suffers, but when I was young and we copied CDs for each other we used tapedecks and it was good enough. Quality loss will stop some people but not most of them.

So we just established that DRM does not work, right? It inconveniences only the “honest” people because the “dishonest” people rip the content out of its DRM-cell anyways (That’s why even people who buy a game use NoCD Cracks and other piracy-related technologies: They make using the product easier!). Now comes the problem: What we want as only privacy is just another layer of DRM, just that this time we are the guys asking for it.

Of course there always is the way of the luddite: We call all this fancy internet stuff crap and do not use it, we don’t publish our data here, we do not post our pictures, we just stay out of it. The problem is: When it comes to personal data you are not the only one that can enter it. If I put up a photo of me and a few friends on the internet and put a text with each person’s name next to it, all those people now have a part of “their” private data public (Remark: I don’t believe that you can own data, not even personal data). So keeping private data private means not just making sure that you are not leaking it, but also making sure nobody else leaks it. What happens if your bank has a data leak and your whole account data is online? There nothing you could have done, it’s not your fault but your data is public. And the net does not forget. The bits will have been copied and you will never get them back. All you can do is try staying ahead in the race and switch banks to get a new “private” account. Until that bank has a leak.

But most of us are no luddites: We enjoy putting stuff online. When me and Annette moved I took some pictures of our new house so I could send the link around to family and friends who don’t live in Oldenburg, I wanted to share something with others and the internet (and it’s great ability to copy bits!) made that easy for me. Now let’s make an experiment based on this.

People talking about “data ownership” would say that we need technology where we can say: “Only these 6 people can view these pictures.” This way we could use the internet to share data but keep privacy intact. Cool. So how do I make sure that one of those people does not save the image and uploads it on Facebook saying “Look what house my friend tante got!”? Do we really want to pretend that in this case where we want DRM for ourselves it would work?

“Privacy” as a concept has the same issues that “property” as a concept has when being transferred from the real into the online world: Property does not work because you can make indistinguishable copies without cost so the idea of scarcity that property is based on fails. Privacy is an idea based on control and especially on the internet where everything is virtual and easy to copy control is impossible.

Back in the days you had a photograph and would show it only to people you wanna show it to. I they tried making a photo of your photo you would see that and could intervene. A JPEG does not give you that control, if somebody except you has seen it you can assume the data to have leaked (at least you cannot guarantee that it didn’t if you didn’t turn the virtual data into a physical thing by forcing people to copy to your place and having them look at it on your screen).

DRM does not work. It does not work to make copying video games impossible, it does not work to keep movies from being copied online and it also does not work to keep your data private.

Paludis 0.54.2 Released

Paludis 0.54.2 has been released:

• Upgrading from older versions to 0.54.1 would produce strange errors. This does not happen for 0.54.2.
• Blockers and uninstalls for chroots are now handled.

Paludis 0.54.1 Released

Paludis 0.54.1 has been released:

• User-defined cave commands are now passed all command line arguments correctly.
• ‘cave show’ now has ‘–description-keys’, and ‘cave search’ uses this where appropriate.
• Paludis key=value configuration files can now use ‘$ENV{FOO}’ to refer to an environment variable.
• output.conf can now set log_path, and user defined output managers will now override builtin output managers with the same name.
• When running in ‘quiet’ mode, status messages are now shown.
• Warning and error messages are now shown at the end, along with info and log messages.
• The ‘tee’ output manager handler now has ‘stdout_children’ and ‘stderr_children’ for only forwarding stdout or stderr.
• Where supported, ‘fallocate’ is now used when merging files.
• Where supported, dirent->d_type is now used to reduce the number of stat() calls that are made.

PixieLive

A friend of mine, Christian, has released PixieLive, a slax based live linux distro build with gentoo.
It’s one of the few live distributions that support Intel GMA500, popular on netbooks like some Asus EEEPC.

PixieLive

Audio transcoding in bash

A proper transcode must have a lossless audio format as a source, a good one is flac ,stands for Free Lossless Audio Codec, because it does the job well and has a good licence ( BSD and GPL ).

If you have some flacs and want them in mp3 format (you need lame and flac) you can use this one liner:
for FILE in *.flac; do flac -cd "${FILE}" | lame - "${FILE/.flac/.mp3}"; done

the above line decodes every file that ends in .flac to the standard output (screen) and pipes it to lame that will encode it. “${FILE/.flac/.mp3}” will be the output file, in bash that means: take the variables “$FILE” (that is the original filename), remove “.flac” and substitute “.mp3″.

Nowadays I transcode quite often to 320 kbit/s constant bit rate (CBR)

• achieving the best quality for an mp3 and still being highly portable, some players don’t like variable bitrate of high quality ( like the V0 preset)
• doing something unefficient since mp3 is not a good codec for that quality but if your stereo reads just mp3 you don’t have any choices

I simply need to use lame with the proper switches , the above one liner becomes:
for FILE in *.flac; do flac -cd "${FILE}" | lame --cbr --preset insane - "${FILE/.flac/.mp3}"; done

To rip my compact discs and encode my music in flac I use a program for windows called Exact Audio Copy (EAC), my Wine build doesn’t like it for some reasons and Virtualbox has historical problems in handling cdroms correctly, the solution was VMWare. The native solution is RubyRipper

Added “iw” support to Linux Sea

The wireless driver developers are actively working on a new wireless toolset called “iw”, slowly deprecating the older wireless-tools toolset (which contains the “iwconfig” command). Kasumi_Ninja reported to me in the Gentoo Forums that it would be nice to add information on iw to Linux Sea, so I did. I must admit though that my systems don’t (or hardly) support iw, so I had to be lead by examples throughout the Internet.

Apart from the iw addition, I’ve reorganized the sections on Software Management as it was becoming too crowded. I’ve also started a ChangeLog for those who want to track the changes to the document.

Now if anyone can recommend me a good spell- and grammar checker…

cvechecker 0.4 released

Albeit with less updates than 0.3 had, cvechecker 0.4 brings in internal project files reorganization (more to the liking of the GNU autoconf/automake standards – I think), fixes a databaseleak (instead of memoryleak ;-) bug and introduces a teenie weenie bit more intelligent pullcves command (with multiple return code behavior to improve automation efforts) as was mentioned in the feature request list. All documentation is also updated and a pullcves manual page has been added.

the "Community Contributions" section on the Arch Linux forums is a goldmine

The Community contributions subforum of the Arch Linux forums is awesome.
It is the birthplace of many applications, most of them not Arch Linux specific.
File managers, media players, browsers, window managers, text editors, todo managers, and so on. Many shell scripts, urxvt extensions and dwm patches aswell.
Most of the apps are designed after suckless/KISS principles, but there are also some GUI programs.

If you like to discover new apps and tools, check it out.

Using PlayOnLinux to run applications already installed by vanilla WINE.

RTS is an acronym for real-time strategy. It’s a game genre. There’s a classic RTS game known as StarCraft. Although only a casual gamer (say, might I recommend Machinarium?) I have played (and somewhat enjoyed) a few more "hardcore" games – like for instance, StarCraft. Although being pretty pathetic at it, I did enjoy it and appreciated the balanced strategy between the various "races" you could control in the game. With StarCraft II already out (hell, it’s about time!) I decided to revisit the original StarCraft: Brood War game to refresh my memory.

I downloaded the game from Battle.net, installed the latest version of WINE (1.3.0), ran the Blizzard downloader flawlessly, ran the installer flawlessly – and true to its Gold ranking on the WINE AppDB, ran the game flawle- no wait. It was laggy. Not particularly laggy. But it wasn’t as fast as it should’ve been, and too many critters on the screen would make it choke. I tried all the lag-fixes suggested on the AppDB submission to no effect. It was a sort of phantom, website-loadingish lag.

However one comment on the AppDB page said that using PlayOnLinux to install WINE 0.9.14 to run StarCraft fixed all lag issues. I decided to give it a shot. Turns out any version below 1.0 is no longer supported (as any sane developer would do) and no longer available in portage. PlayOnLinux was, however – and PlayOnLinux did still allow WINE 0.9.14 to be installed. Unfortunately it wasn’t particularly intuitive to tell PlayOnLinux to run my installed .exe file with WINE 0.9.14, so perhaps this blog post might help others in my situation. Using these steps I was able to install a prehistoric version WINE via PlayOnLinux, and tell it to run the already installed version of StarCraft on my computer:

1. Install PlayOnLinux, Tools -> Manage wine versions -> install v0.9.14. Should be straightforward.
2. mkdir /home/username/.PlayOnLinux/wineprefix/starcraft && env WINEPREFIX=/home/username/.PlayOnLinux/wineprefix/starcraft wineboot
3. cp -r /home/username/.wine/drive_c/Program\ Files/StarCraft /home/username/.PlayOnLinux/wineprefix/starcraft/drive_c/Program\ Files/ (or mv it, doesn’t make a difference)
4. touch /home/username/.PlayOnLinux/configurations/installed/StarCraft
5. Place the following in the StarCraft file:

#!/bin/bash
export WINEPREFIX="/home/username/.PlayOnLinux/wineprefix/starcraft"
export WINEDEBUG="-all"
cd "/home/username/.PlayOnLinux/wineprefix/starcraft/drive_c/Program Files/StarCraft"
wine "StarCraft.exe" $@

And that’s it! Run PlayOnLinux and you’ll be able to run the program from there. This guide should be able to work for other scenarios as well so feel free to adapt it.

Oh, and as for the lag? Yep – oddly enough it did get completely fixed. Regression time.

Related posts:

1. Top 10 Windows Mobile Applications
2. A scalable and adaptable standardised user file structure?

I remain impressed by the free software community

My current personal projects, Linux Sea and cvechecker, are actively being watched by the free software community. For the Linux Sea book, I get nice feedback and ideas on the Gentoo Forums and on the cvechecker application, people such as Nigel Horne are helping out in various ways – including feature requests of all sorts.

I must admit, I remain impressed.

Small changes have already been squeezed in in the Linux Sea document. A larger change (the use of the iw tools for wireless connectivity) is being investigated (sadly, my broadcom-sta device doesn’t support the new nl80211 API so the documentation change is slower to integrate than expected). I’m also planning to make some updates on the software management chapter as it is currently becoming quite crowded.

In the cvetool, most changes are bugfixes and output enhancements as expected. I’m not going to add more functionality now – I first want to get a stable 1.0 release out there. But first continue to squash bugs and add rules to the versions.dat file so that it is usable on various systems (release 0.4 is around the corner).

Paludis 0.54.0 Released

Paludis 0.54.0 has been released:

• ‘cave resolve –remove-if-dependent’ will now cause dependent packages to be removed from world, if no versions remain.
• ‘cave resolve –make’ now defaults to ‘chroot’ when the environment’s preferred root is not ‘/’.
• ‘cave display-resolution –show-option-descriptions’ now works as documented.
• ‘cave fix-linkage’ may take multiple ‘–library’ arguments.
• New ‘cave’ subcommands: ‘report’, ‘print-id-environment-variable’.
• ‘cave print-’ commands are now consistent in handling ‘–all’ / ‘-a’ and ‘–best’ / ‘-b’.
• ‘cave show’ will now always show keys specified by ‘-k’, even if those keys are internal-only or complex.
• ‘cave search’ now takes an optional SQLite index, which can be created using ‘cave manage-search-index’.
• ‘cave search’ now has a ‘–visible’ option.
• The ‘importare’, ‘inquisitio’ and ‘reconcilio’ clients are now deprecated in favour of ‘cave import, ‘cave search’ and ‘cave fix-linkage’ respectively.
• Output managers and output.conf are now documented features, and a new ‘command’ output manager handler was added.
• Hooks are now run using an output manager, where appropriate. The API for .so hooks has been changed to allow this.
• The demo hooks have been removed, since none are particularly useful with ‘cave’.
• ‘cave’ now has a global ‘–colour’ option, and coloured output is disabled by default when outputting to a non-tty.
• The output format for certain cave commands can now be tinkered with by the user. Use ‘cave dump-cave-formats-conf’ to create ~/.cave/formats.conf and then edit as desired.
• The way sub-programs are executed has changed to be somewhat less convoluted.