How to live with both GnuPG key formats, gpg1 and gpg2

GNU Privacy Guard (GPG) changed the format in which keys are stored, the so-called keyring directory, somewhere around version 2.1. I am faced with a situation where both, older and newer versions of GPG have to be used with the same keys. This is possible, in principle. But there are a few caveats and don'ts.

First create a temporary keyring directory. We don't want to mess with productive keyrings in the following:

  GNUPGHOME=$(mktemp -d); chmod 700 $GNUPGHOME; export GNUPGHOME

For this demo only: set the shell prompt to show our GPG version:

  PS1=$(gpg --version | head -1 | sed 's/ .GnuPG. /-/')'> '

Now generate a key pair:

 gpg --batch --gen-key <<EOF
 %no-protection
 Key-Type:1
 Key-Length:2048
 Subkey-Type:1
 Subkey-Length:2048
 Name-Real: foo
 Name-Email: foo@bar.com
 Expire-Date:0
 EOF

This HERE file notation is still needed in GPG2, it's a shame.

gpg-1.2.1:
gpg: keyring `/tmp/tmp.npsekdHuuE/secring.gpg' created
gpg: keyring `/tmp/tmp.npsekdHuuE/pubring.gpg' created
gpg: skipping control `%no-protection' ()
...........+++++
gpg: /tmp/tmp.npsekdHuuE/trustdb.gpg: trustdb created


gpg-1.4.20:
gpg: keyring `/tmp/tmp.CTWQLCtLlJ/secring.gpg' created
gpg: keyring `/tmp/tmp.CTWQLCtLlJ/pubring.gpg' created
gpg: skipping control `%no-protection' ()
.....+++++
gpg: /tmp/tmp.CTWQLCtLlJ/trustdb.gpg: trustdb created
gpg: key 12B2E56A marked as ultimately trusted

gpg-2.2.4:

gpg: keybox '/tmp/tmp.RJ0dQbUWsJ/pubring.kbx' created

gpg: /tmp/tmp.RJ0dQbUWsJ/trustdb.gpg: trustdb created

gpg: key B7659D91346F6245 marked as ultimately trusted

gpg: directory '/tmp/tmp.RJ0dQbUWsJ/openpgp-revocs.d' created

gpg: revocation certificate stored as '/tmp/tmp.RJ0dQbUWsJ/openpgp-revocs.d/508DB21AA7490081AB326E26B7659D91346F6245.rev'

We see that "key ring" is replaced by "keybox" in gpg2.

Compatibility tests:

gpg-1.2.1> gpg --armor --export > public-1.2.1

gpg-1.4.20> gpg --import public-1.2.1 

gpg-2.2.4> gpg --import public-1.2.1 

gpg-1.4.20> gpg --armor --export > public-1.4.20

gpg-1.2.1> gpg --import public-1.4.20 

gpg-2.2.4> gpg --import public-1.4.20 

gpg-2.2.4> gpg --armor --export > public-2.2.4

gpg-1.2.1> gpg --import public-2.2.4 

 gpg: key 346F6245: invalid self-signature on user id "foo <foo@bar.com>" 

 gpg: key 346F6245: invalid subkey binding

 gpg: key 346F6245: no valid user IDs

 gpg: this may be caused by a missing self-signature

gpg-1.4.20> gpg --import public-2.2.4 

A key generated on gpg-2.2.4 can be imported by 1.4.20, but not by 1.2.1.

Perl stubbing (mocking) a non-method subroutine

This is about Perl5 unit testing.

In Unit testing, you often want to mock some method of somebody else's code, i.e. you replace the method call with one that executes your own code, more appropriate to the test at hand.

This technique is called "mocking" if the faked subroutine is a class method. In the seemingly simpler case, where there are no classes and methods, only packages and subroutines, authors prefer the term "stub" to "mock", even though the idea is exactly the same.

However, creating a subroutine stub in Perl5 seems to be exceedingly difficult. All available documentation and CPAN modules refer to OO classes. Those modules that also mention stubbing (Mock::Modules, Test::Speck::Mock, and maybe others) do so only vaguely and provide no examples.

Here I present a generic solution to the problem. I create a package PackageA exporting a subroutine subA that simply prints "The Original" to stdout. The PackageB contains a subB which calls subA from PackageA, i.e. it also prints "The Original", but that's what we want to change. In a third package, called MyTest, we first call subB, then we replace the global reference of subA with our own subroutine that prints "The Stub". Finally we call subB again, but now in the stubbed version. Here is the complete example in one file:

package PackageA;
sub subA {
    print "The Original\n";
}

package PackageB;
sub subB {
    PackageA::subA();
}

package MyTest;
PackageB::subB();
no warnings;
local *PackageA::subA = sub {
    print "The Stub\n";
};
use warnings;
PackageB::subB();

This prints:

The Original
The Stub

Good! This is exactly what we want. - Sadly it does not work if we naively split the above code across three files using the well-known Exporter.

File PackageA.pm:
package PackageA;
use Exporter;
@ISA = qw(Exporter);
@EXPORT = qw(subA);
sub subA {
    print "The Original\n";
}
1;

File PackageB.pm:
package PackageB;
use PackageA qw(subA);
sub subB {
    subA();
}
1;

File MyTest.t (does not work):

use PackageA qw(subA);

use PackageB;

PackageB::subB();

no warnings;

local *PackageA::subA = sub {

    print "The Stub\n";

};

use warnings;

PackageB::subB();

Executing perl MyTest.t prints:

The Original

The Original

No stubbing takes place, the original Package::subA is executed in both cases! How comes? - The reason is the way the Exporter/use modifies the global symbol table. Importing a subroutine does not simply create a new subroutine that calls the original one (this would be an unnecessary redirection). Importing a foreign symbol copies its reference into the own namespace. All this happens before any statements are executed, so at the time of printing, the entry *PackageA::subA still points to the original subroutine, while there is now a new entry *PackageB::subA, created by "use PackageA", that also points to the original subA, and that's the one we want to replace:

local *PackageB::subA = sub {

With this modification, the stub is actually executed: perl MyTest.t prints

The Original
The Stub

The correct MyTest.t:

use PackageA qw(subA);

use PackageB;

PackageB::subB();

no warnings;

local *PackageB::subA = sub {

    print "The Stub\n";

};

use warnings;

PackageB::subB();

I shall be happy to use a CPAN module that does the above, as soon as somebody points me to one. Until then, my unit tests will use the technique outlined here.

Access USB drive (USB storage, USB stick) from the Linux command line

I am using Ubuntu, but the problem is the same for most current Linux desktops: When inserting a USB storage device, it is automatically reconized, a file browser window pops up and allows you access to the files and directories inside. Great!

That is, USB storage can be seen by the file browser and other GUI applications, but not the command line. shame!

For us who live in the shell, two additional steps are required:

1. find the block device name of the usb drive (usually /dev/sdb1)
2. mount it

For 1.) either fdisk -l, lsblk, or blkid can be used. To recap:

• sudo fdisk -l
• mount /dev/sda1 /mnt

Don't forget to unmount before removing the USB device:

• umount /mnt

Ubuntu 16.04 - Macintosh Tastatur - Schweizerdeutsches Layout

This post is in Swiss German for obvious reasons.

I han chrüzlech so ne ultra-cooli Mac-Alu-Taschtatur übercho. En Mac-Mini hani scho, aber MacOS hani ned wölle... Ubuntu funktioniert rächt guet uf Mac-Hardware. Die folgendi Aaleitig sött aber ou for Mac-Taschtature a nid-Macs funktioniere.

Mac-Tastatur aschliesse: d'Helfti vo de Taschte sind falsch beleit. Compi unbruchbar.

Ubuntu> sudo dpkg-reconfigure keyboard-configuration

-> "Generische PC-Tastatur mit 105 Tasten (Intl)"
-> Swizerland
-> Switzerland - Deutsch (Schweiz, Macintosh)
-> Der Standard für die Tastenbelegung
-> Keine Compose-Taste

Denn neu boote (Login isch ned gnueg).

Erklärig: De Trick isch die erschti Iischtellig "Generischi 105-Taschte", und nid öppe "Macintosh", wie me jo vermuete chönnt.

Jetzt chunnt no d'Fiinarbet.

Es sind no zwöi Taschte vertuuscht: Paragraph/Grad linggs obe, und grösser/chliner linggs unde. Die chamer zom Biispil so tuusche:

echo "keycode  49 = less greater less greater lessthanequal greaterthanequal lessthanequal" >> .Xmodmap
echo "keycode  94 = section degree section degree NoSymbol NoSymbol NoSymbol NoSymbol section degree NoSymbol NoSymbol section degree" >> .Xmodmap
echo /usr/bin/xmodmap $HOME/.Xmodmap >> .xinitrc

Jetz sind alli Symbol uf de richtige Taschte. Leider sind aber nid alli aagschribe. Als Programmierer fähled mir do extrem wichtegi Symbol wie: gschweifti und eggigi Chlammere, Backslash, Tilde usw.

Mit Mac-Benutzer diskutiere nützt nüt. Die findet das guet. Sie chönnd ihre Taschte usswändig und findet, wer das nid chönn, söll sich doch en PC choufe, oder wenigschtens e Logitech-Taschtatur. Mac-User sind e Ploog.

Ich ha drum en wasserunlösliche Folieschriiber gnoh und folgendi Taschte annotiert:

N: Tilde
S: Doppel-S ("esszett")
3: Gartehaag
5: offni eggigi Chlammere
6: gschlossni eggigi Chlammere
7: sänkrächte Strich (unde), Backslash (obe)
8: offni gschweifti Chlammere
9: gschlossni gschweifti Chlammere
fn: Insert
Pfiil lings ufe: Home
Pfiil rächts abe: End

Jetz isch die Taschtatur einigermasse bruuchbar, wemme sich merkt, dass me die undere Symbol mit der rächte "alt"-Taschte erreicht, die obere Symbol mit rächts-alt-Shift.

S'isch echli blöd, dass me für eis vo de meischtbenützte Symbol, de Backslash, grad drüü Taschte gliichziitig trücke mues. Vilich mach ich mer do schpäter ou no en "modmap".

Wiitere Knackpunkt: alt-Space macht nid öppen es normals Läärzeiche wie die normal PC-Taschtatur, sondern es Unicode-Läärzeiche, wo uusgseeht wienes normals, aber imene Programm-File zum Biischpil es Riesen-Unheil aarichte chann:

> file prog.c
prog.c: C source, UTF-8 Unicode text

Das git Ärger. Such die Ziile mitem Unicode-Läärzeiche:

> grep -P "[\x80-\xFF]" prog.c

und ersetzes

> vi prog.c
> file prog.c
prog.c: C source, ASCII text

grettet!

Extract subversion metadata directly from checkout, without subversion tools

Hello readers!

By now, you suspect that I am an avid subversion user, or at least a professional programmer, so yes. I might as well admit it.

The following trick is not for the showroom. I had to work with incompatible subversion versions. It usually is a pain to work with different versions of subversion. Checkouts are incompatible.

Anyway, I had to find the author and revision number of the last commit in a given checkout, without having the proper subversion command line tool at hand. It can be done, because subversion metadata resides in a sqlite3 database in the .svn directory.

This is the magic incantation:

sqlite3 .svn/wc.db "select changed_author,changed_revision from NODES order by changed_revision desc limit 1;"

There are many other interesting items to be found in .svn/wc.db and the database schema seems to be fairly stable since subversion 1.6 or so.

following code through history, in subversion and git

Both, git and subversion, have the famous "blame" command, which allows me to see, line by line, who changed this line of code last.

Sometimes it is not the latest modifier of a line who is its true author. For example when a programmer added or removed whitespace for better readability, she will then be "blame"d as it's author, even though she is not.

In this case, it is possible to give the blame command an additional agrument, saying "look at the file in this revision, and tell me who brought it up to that state, line by line".

in subversion:

$svn blame filename
   649     joe  today's code
$svn blame filename -r648
  171     jim previous code
$svn blame filename -r170
 svn: Unable to find repository location for 'filename' in revision 170

in git:

$ git blame filename
3c9fb2a17 (Joe 2016-09-01 16:43:11 +0200 1) today's code
$ git blame filename 3c9fb2a17^
 98bcec40b (Jack 2016-06-23 18:04:55 +0200 1) previous code
$ git blame filename 98bcec40b^
fatal: no such path filename in 98bcec40b^

Note the caret (^) you have to add to the end of the git hash. It means "the previous commit from that designated by the hash". In subversion, where everything is nicely ordered, you simply subtract one in order to get the same effect.

One day I might write a script that traces the history of a single line of code back to its creation. Please tell me if this has already been done.

Subversion: find all files with keyword but without the svn:keywords property

The subversion source control system replaces certain keywords like $Id$ with some very useful metadata, if and only if the file in question is configured to do so. Normally, file properties are added automatically to subversion at the time the file is added. Unless..., unless there was a mistake. Mistakes happen.

Unfortunately, subversion is not very keen to discover and fix such inconsistencies. That's why I came up with this one-liner:

find . -name .svn -prune -o -type f | while read f; do if egrep -q '\$Id|\$HeadURL' $f; then test "$(svn pget svn:keywords $f)" = "Id HeadURL" || echo $f; fi ; done

This finds all files that contain the $Id$ or $HeadURL$ keywords anywhere in the text, and at the same time do not have the svn:keywords property set to "Id HeadURL".

You might be using other subversion keywords: just change the regexp and the property string accordingly.

This helped me a lot, lately, and I thought I wanted to share it.