Wiedi in Wonderland

Parallel git and cvs workflow

Wed 08 July 2015

This describes a workflow for using git to develop on pkgsrc. As pkgsrc upstream uses cvs you'll need two parallel copies of the tree: one using git and one using cvs. To get changes from git into cvs it uses git-cvsexportcommit. Git has many benefits like local commits, easy branching, atomic commits across multiple files, etc. I also like to prepare my commit messages and review them before setting them in stone in the eternal history.


To setup a basic working environment I use configuration like follows.

Add to ~/.bash_profile:

export CVSROOT=$LOGNAME@cvs.NetBSD.org:/cvsroot
export CVS_RSH=ssh

Add to ~/.cvsrc

# recommended CVS configuration file from the pkgsrc guide
cvs -q -z2
checkout -P
update -dP
diff -upN
rdiff -u
release -d


get your cvs tree:

mkdir ~/cvs ~/tmp
cd ~/cvs
cvs checkout pkgsrc


checkout Jörgs git conversion:

cd ~/tmp
git clone https://github.com/jsonn/pkgsrc.git


the git tree will be your working tree:

cd ~/tmp/pkgsrc/bootstrap
./bootstrap --unprivileged

add to ~/.bash_profile

export PATH="~/pkg/bin:~/pkg/sbin:$PATH"

create your change

as an example let's do a simple update of nano:

cd ~/tmp/pkgsrc/editors/nano

make changes and test them:

$EDITOR Makefile
bmake package

stage your changes and commit:

git add Makefile distinfo PLIST
git commit

you now get to review your change as one entitiy, including the commit message:

git show -1
commit 3b861ca6563af68aa3c175ebae151d3870c9b5d2
Author: Sebastian Wiedenroth <wiedi@frubar.net>
Date:   Wed Jul 8 22:59:39 2015 +0200

    Update nano to 2.4.2

    2015.07.05 - GNU nano 2.4.1 "Portorož" is released.  This release
            includes several fixes, including the ability to resize
            when in modes other than the main editing window,
            proper displaying of invalid UTF-8 bytes, new syntax
            definitions for Elisp, Guile, and PostgreSQL, and
            better display of shortcuts in the help menu and file
            browser.  Thanks for your patience and using nano!

diff --git a/editors/nano/Makefile b/editors/nano/Makefile
index 62fb2c1..c2a4b12 100644
--- a/editors/nano/Makefile
+++ b/editors/nano/Makefile
@@ -1,10 +1,10 @@
 # $NetBSD: Makefile,v 1.46 2015/06/05 01:32:38 wiedi Exp $

-DISTNAME=  nano-2.4.1
+DISTNAME=  nano-2.4.2
 CATEGORIES=    editors
 MASTER_SITES=  http://www.nano-editor.org/dist/v2.4/

-MAINTAINER=    pkgsrc-users@NetBSD.org
+MAINTAINER=    wiedi@frubar.net
 HOMEPAGE=  http://www.nano-editor.org/
 COMMENT=   Small and friendly text editor (a free replacement for Pico)
 LICENSE=       gnu-gpl-v3
diff --git a/editors/nano/PLIST b/editors/nano/PLIST
index 7d4f944..1ecfdd4 100644
--- a/editors/nano/PLIST
+++ b/editors/nano/PLIST
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.18 2015/06/05 01:32:38 wiedi Exp $
+@comment $NetBSD$
@@ -50,10 +50,12 @@ share/nano/cmake.nanorc
@@ -70,6 +72,7 @@ share/nano/patch.nanorc
diff --git a/editors/nano/distinfo b/editors/nano/distinfo
index 6b372a5..3420686 100644
--- a/editors/nano/distinfo
+++ b/editors/nano/distinfo
@@ -1,6 +1,6 @@
 $NetBSD: distinfo,v 1.20 2015/06/05 01:32:38 wiedi Exp $

-SHA1 (nano-2.4.1.tar.gz) = 422958cb700cc8cedc9a6b5ec00bf968c0fa875e
-RMD160 (nano-2.4.1.tar.gz) = 84bd54e50b5e8c6457d983dc7ef730b5a0303bf8
-Size (nano-2.4.1.tar.gz) = 1890805 bytes
+SHA1 (nano-2.4.2.tar.gz) = bcf2bb3fcc04874cb38c52cfd8feebce61dd5e0a
+RMD160 (nano-2.4.2.tar.gz) = 6a3d0569740c223230af6ae88f8ef0797402c4c2
+Size (nano-2.4.2.tar.gz) = 1898633 bytes
 SHA1 (patch-configure) = 3a63b02a39000d5a15087739648b82e999d14f56

You can take this diff and apply it to different systems for testing. This is very easy with git.

Once you are happy you can commit to CVS.


cd ~/tmp/pkgsrc/
git cvsexportcommit -w ~/cvs/pkgsrc/ -pcv 3b861ca6563af68aa3c175ebae151d3870c9b5d2

This will commit (-c) if the change with the id 3b861ca6563af68aa3c175ebae151d3870c9b5d2 applies cleanly (-p for paranoid).

Keepking CHANGES and TODO up to date:

bmake changes-entry
cd ../../doc
cvs diff
cvs commit CHANGES-2015

The reason to not prepare this in the git commit is that the CHANGES file is updated very frequently and will conflict with the strict settings used with cvsexportcommit.

keeping both trees updated

before starting a new change update your git tree:

git pull

As you now have git you can also rebase easily onto more recent changes from upstream.

Before commiting update your cvs tree:

cvs update -d

Building illumos-gate on OmniOS

Thu 02 July 2015

Illumos-gate is the source repository of illumos. It is comparable with the Linux kernel in that you usually don't install from illumos-gate direclty but one of the distributions like SmartOS, OmniOS or OpenIndiana.

Unlike Linux the stable interface is not at the syscall layer but libc. So it kinda makes sense that the repository also contains some userland software like libc and core utilities that make the operating system.

Illumos has strict requirements for its build environment. Currently you need a specific version of gcc. In the past the only distribution where you could do a clean build of unmodified illumos-gate was OpenIndiana. Thanks to Dan McDonald it recently became also possible to build on OmniOS.

For detailed information see the "How to build illumos" page in the wiki. If you are starting out with OpenIndiana Ryan Zezeski has very helpful instructions. These are the steps I used on OmniOS.

Install dependencies

First start out by making sure you have all required software installed:

sudo pkg install -v \
 pkg:/developer/astdev \
 pkg:/developer/build/make \
 pkg:/developer/build/onbld \
 pkg:/developer/gcc44 \
 pkg:/developer/sunstudio12.1 \
 pkg:/developer/gnu-binutils \
 pkg:/developer/java/jdk \
 pkg:/developer/lexer/flex \
 pkg:/developer/object-file \
 pkg:/developer/parser/bison \
 pkg:/developer/versioning/mercurial \
 pkg:/developer/versioning/git \
 pkg:/developer/library/lint \
 pkg:/library/glib2 \
 pkg:/library/libxml2 \
 pkg:/library/libxslt \
 pkg:/library/nspr/header-nspr \
 pkg:/library/perl-5/xml-parser \
 pkg:/library/security/trousers \
 pkg:/runtime/perl \
 pkg:/runtime/perl-64 \
 pkg:/runtime/perl/module/sun-solaris \
 pkg:/system/library/math \
 pkg:/system/library/install \
 pkg:/system/library/dbus \
 pkg:/system/library/libdbus \
 pkg:/system/library/libdbus-glib \
 pkg:/system/library/mozilla-nss/header-nss \
 pkg:/system/header \
 pkg:/system/management/snmp/net-snmp \
 pkg:/text/gnu-gettext \

Get the code

Next clone the repository:

cd ~
git clone git://github.com/illumos/illumos-gate.git
cd illumos-gate

Closed source binaries

Some closed source binaries are still required:

wget -c \
 https://download.joyent.com/pub/build/illumos/on-closed-bins.i386.tar.bz2 \
tar xjvpf on-closed-bins.i386.tar.bz2
tar xjvpf on-closed-bins-nd.i386.tar.bz2

Setup illumos.sh environment file

The build environment is configured in a script that is passed to nightly.sh as an argument. Start out by copying the example:

cp usr/src/tools/env/illumos.sh .

Then adjust it to your needs. My diff looks like this:

--- usr/src/tools/env/illumos.sh        Sat May 30 16:58:22 2015
+++ illumos.sh  Mon Jun  1 14:18:16 2015
@@ -58,10 +58,10 @@

 # This is a variable for the rest of the script - GATE doesn't matter to
 # nightly itself
-export GATE='testws'
+export GATE='illumos-gate'

 # CODEMGR_WS - where is your workspace at (or what should nightly name it)
-export CODEMGR_WS="$HOME/ws/$GATE"

 # Maximum number of dmake jobs.  The recommended number is 2 + NCPUS,
 # where NCPUS is the number of logical CPUs on your build system.
@@ -206,7 +206,9 @@
 # exists to make it easier to test new versions of the compiler.
 export BUILD_TOOLS='/opt'
 #export ONBLD_TOOLS='/opt/onbld'
-export SPRO_ROOT='/opt/SUNWspro'
+# Help OmniOS find lint
+export SPRO_ROOT='/opt'

 # This goes along with lint - it is a series of the form "A [y|n]" which
@@ -230,15 +232,26 @@

 # Comment this out to disable support for IPP printing, i.e. if you
 # don't want to bother providing the Apache headers this needs.

 # Comment this out to disable support for SMB printing, i.e. if you
 # don't want to bother providing the CUPS headers this needs.

 # If your distro uses certain versions of Perl, make sure either Makefile.master
 # contains your new defaults OR your .env file sets them.
 # These are how you would override for building on OmniOS r151012, for example.
-#export PERL_VERSION=5.16.1
-#export PERL_ARCH=i86pc-solaris-thread-multi-64int
-#export PERL_PKGVERS=-5161
+export PERL_VERSION=5.16.1
+export PERL_ARCH=i86pc-solaris-thread-multi-64int
+export PERL_PKGVERS=-5161
+# OmniOS places GCC 4.4.4 differently.
+export GCC_ROOT=/opt/gcc-4.4.4/
+export ONNV_BUILDNUM=151014
+# GCC only
+export CW_NO_SHADOW=1
+export ONLY_LINT_DEFS=-I${SPRO_ROOT}/sunstudio12.1/prod/include/lint
+export __GNUC=""

Also copy nightly.sh and make it executeable:

cp usr/src/tools/scripts/nightly.sh .
chmod +x nightly.sh

I am not sure why this is needed:

sudo ln -s /opt/sunstudio12.1/ /opt/SUNWspro


Now all you need to start a build is:

./nightly.sh illumos.sh


If all went well you now have a working development environment and already did your first build. Next you can start making changes. The usual workflow is documented in the excellent illumos developer's guide.

For code review illumos also has a reviewboard which can be used instead of webrev since a few months.

Finally there is also a chapter on getting code upstream.

Installing OmniOS

Wed 01 July 2015

OmniOS is one of the illumos distributions. Created by OmniTI it is primarily designed as a server operating system and comes with a minimal set of software installed by default.

Installation is straight forward and configuration well documented. So in general you should not need this article. But as I was using VMware Fusion I hit a few issues that might be worth mentioning for others. The rest is mostly so I have something to copy & past in the future.

The first issue you might run into using VMware is that you need to add an empty floppy disk to your vm or else installation will hang.

installing OmniOS in VMware

The second issue took me a bit longer to figure out. There was a bug in VMware Fusion that led to kernel panics. An update of Fusion solved this, so make sure you're using the latest version.

Configure Network

The installer, like the system in general, is very minimal. Network configuration is left for the admin:

ipadm create-if e1000g0
ipadm create-addr -T dhcp e1000g0/v4
echo 'nameserver' >> /etc/resolv.conf
cp /etc/nsswitch.conf{,.bak}
cp /etc/nsswitch.{dns,conf}

Create a user

Add yourself a user to work with:

useradd -s /usr/bin/bash -d /export/home/wiedi wiedi
mkdir /export/home/wiedi
chown wiedi:other /export/home/wiedi
passwd wiedi
vim /etc/sudoers

Probably you also want to add your ssh key to the ~/.ssh/authorized_files.

To get a nicer bash prompt and and all the tools you need add this to your ~/.bash_profile:

export PATH=$PATH:/opt/omni/bin/
export PS1'=\u@\h \w: '


pkg install pkg:/package/pkg
pkg update -v


More non-core packages can be found in the "managed services" repository. To install nano use:

sudo pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.com
sudo pkg install nano

Take half a pull request and keep the author

Thu 25 September 2014

So someone contributes a fantastic bugfix to your project on github and you're happy. But there's a problem: besides the bugfix there are other changes too that you might not want to merge.

So what you do is you start cherry-picking the good stuff. On Stackoverflow there is actually a great answer on how to only cherry-pick some changes from one commit. Sadly, once you do it like in that answer you become the author of that new commit.

Someone took the effort to write a fix for your project so proper attribution is important. The solution is git commit -c <commit> which reuses the log message and the authorship information (including the timestamp) when creating the commit.

So the complete thing looks something like that:

git cherry-pick -n <commit> # get your patch, but don't commit (-n = --no-commit)
git reset                   # unstage the changes from the cherry-picked commit
git add -p                  # make all your choices (add the changes you do want)
git commit -c <commit>      # make the commit and keep the author


Tue 01 June 2010

I haven't blogged in a while, but that doent's mean nothing happened.
So here are some quick updates. ;-)

Since the first released of libmaia 2 years ago a lot of people have mailed me patches. This is really great, thanks for that! To make it easier to contribute the source is now (in addition to subversion) also available on github. So if you want to add some new feature or fix a bug you can simply fork it. Github also offers issue tracking which was requested a few times, since managing bug-reports in blog comments of course isn't ideal.

In february we upgraded our ircd and are now reachable over IPv6! Connectivity is provided by SkyLime ;-)

Now also IPv6 ready is the Freamware jabber server. And we've switched our server software to ejabberd.

We had an awesome python workshop and a nice Linux Install Party. This semester I'm working on a cluster management project which involves django and "something" with the semantic web and linked data. Great stuff!

Barcamp Bodensee:
This weekend drscream, Maex and myself will be in Konstanz for the 2.0 version of the Barcamp Bodensee.

Good night! :D

delicious tags

Sun 30 August 2009

Wenn man seine bookmarks schon online sortiert kann man das natürlich als tagcloud verwursten: *hübsch*
wied0r delicious tags
(via drscream)


Sat 29 August 2009

Die lelith hat was buntes gefunden sich selbst darzustellen:
Das bin dann also ich, soso ;)

ALIX Router

Mon 29 June 2009

Seit gut einer Woche läuft jetzt mein neuer Router.
Die erste Überlegung war ein Soekris Board zu kaufen, wegen dem Preis hat dann aber das Alix 2d13 gewonnen.
Im Varia-Store gibt es günstige Komplettpakete inkl. Case und allem was man braucht.

Auf dem Alix ist ein AMD Geode mit 500 MHz verbaut, es hat drei Ethernet Ports und bootet von CF.

Weil sich Router alleine schnell einsam fühlen haben sich drscream und Boris auch einen gekauft. Wir haben dann Voyage Linux, ein Debian Derivat, auf die CF Card installiert.
Das war einfach und schnell gemacht.
Voyage ist speziell für solche embedded x86 Geräte angepasst.
Hat also Kernel Patches um die LEDs und Sensoren anzusteuern. Das System ist sinnvoll vorkonfiguriert.

Damit die CF card nicht gleich stirbt werden /var/log etc. als tmpfs gemounted und via init Script gesynced.
Der ISC dhcpd hat sein lease-file allerdings in /var/lib/dhcp3/. Es ist also eine gute Idee den Ordner in /etc/default/voyage-util bei VOYAGE_SYNC_DIRS einzutragen.

Ausserdem läuft noch ein OpenVPN Client und Quagga für OSPF drauf.

Boris hat heute noch eine Wifi Karte via miniPCI angeschlossen.
Was man sonst noch alles damit basteln kann wird sich zeigen. Ist auf jedenfall ein schönes Gerät - sehr unkompliziert.

BarCamp Dornbirn 2009

Sun 07 June 2009

Heute war der erste Tag vom BarCamp Dornbirn.
Mal wieder ein BarCamp gleich ums eck. Relativ klein und gemütlich, sehr nett.
Die erste Session, "Entscheidungsfindung in Gruppen" von Thomas Zahreddin, war eine interessante Diskussion über Soziokratie. Weiter ging's mit PHP-Security von Alexander Fend.

Nachmittags hat Christopher Clay über seine Startup-Erfahrungen mit Soup.io erzählt.
Das war mal wieder eine Session für die ich BarCamps so toll finde: zum einen Austausch über Firmengründungen, aber besonders weil man neue Websites kennen lernt.
Soup.io ist eine Mischung aus Bloghoster, Feed Aggregator, Social Network und ein paar anderen Ideen. Das ganze klingt auf den ersten Blick nicht sonderlich spannend, ist aber sehr gut umgesetzt. Jetzt gibts seit eben den Suppenwiedi. Das User Interface ist genial und sehr einfach. Man kann direkt loslegen.

Ich hab mich dann auch gleich mal ein paar Stunden in der Suppe verloren. Viel guter Content, der von der Qualität je nach Gruppe fast an FFFFOUND! heranreicht. Aber für mehr als nur Bilder.

Das einzige was mich abhält hier alles auf Soup umzustellen ist, dass ich meine Sachen selber hosten möchte. Mal sehen wie sich das noch entwickelt.

Morgen geht's weiter, aber jetzt brauch ich erstmal etwas schlaf.

La Fonera 2.0 beta

Wed 04 February 2009

La Fonera 2.0 beta box

Nachdem ich gestern endlich mit meiner Bachelor Thesis komplett abschliessen konnte hab ich heute via UPS neues Spielzeug aus Spanien bekommen: eine La Fonera 2.0 beta. Danke Fon!

Die neue Fonera hat jetzt 32 mb RAM, was hoffentlich ausreicht um zuverlässig Quagga darauf laufen zu lassen.
Allgemein ist mein Plan das ganze "IPv6 ready" zu bekommen. Interessant ist auch der USB Port - mal schauen was sich damit noch alles basteln lässt ;-)

fon spot

la fonera 2.0 beta

fonera leds

Ach ja, und übers Wochenende bin ich in Brüssel, Belgien - ist ja FOSDEM! ;-)