[jump to content][No Software Patents]FAQ [ Home | Scylla+Charybdis old home | Tools (old) | Tools (new) | FAQ | Downloads ]

Moving to GitHub, slowly

The software on this pages will slowly be moved to GitHub https://github.com/hilbix/. The CVS repository will be migrated to GIT as well, so the history will be preserved, a bit. See FAQ.

Scylla and Charybdis, FAQ

If you think you found a bug or your question is not answered, you can leave me a message I will fix it or add the answer here if it is important.
If you need to send me a file, then please additionally leave me a message (the upload does not notify me yet).

Q: hi how r u
A: Well, fine, thanks. ;)


Q: could i have some more info? how it came to be, who the parents were (how's related to creation of world and titans and all that). could you describe it for me plz?
A: Well .. I only used the names to represent data transport. Charybdis sucks in the data and drives it away to Scylla, who eats it with her many heads. For more info about the myth background, perhaps see Wikipedia about Scylla and Charybdis.


Q: xml2sql-v does not compile/link under CygWin
A: There is a bug in the Makefile. Replace (the word) LDFLAGS with LDLIBS in the Makefile. It then links.


Q: ./xml2sql-v.exe test < xmlfile.xml gives
#0 # '<?xml version=\"1.0\" encoding=\"ISO-8859-2\"?>'
parse error line=1 column=30: unknown encoding
Likewise under Linux ..
A: Obviously expat does not know XML encoding for ISO-8859-2, sigh ;( This means, my tool must be extended to support the unknown encoding. I currently don't have the time to fix this.
But luckily there is a bad trick to still convert it without hacking my C code:
ISO-8859-2 is very similar to ISO-8859-1. So the file parses OK if the encoding is changed to ISO-8859-1. This can be accomplished with following line (under CygWin):

sed '1s/ISO-8859-2/ISO-8859-1/' xmlfile.xml | ./xml2sql-v.exe " 'test' " >xmlfile.sql

Expat then does not complain for wrong characters and still shall output ISO-8859-2 code (as it does not convert the characters).
OK, yes, it's quite ugly, but - hopefully - it shall work this way (no guarantees!) ;)


Q: make spits out errors like Makefile.awk:256: fatal: match() cannot have 3 arguments
A: Your AWK is too old to recreate the Makefile. The build is done based on the distributed Makefile. Everything shall compile successfully though.

So if the compile succeeds just ignore such errors, please.


Q: How to use a signature (.asc) file to check the download?
A: Get GnuPG.
First (you only need to do this once per login) download my key from a keyserver, for example (note that the key ID can change in future):
gpg --keyserver pgp.mit.edu. --recv-keys 2C6BE86C
Now you can check the signature using the .asc file. For example, if you have downloaded
diskus-0.5.0-20081214-023903.tar.gz and diskus-0.5.0-20081214-023903.tar.gz.asc
then you can check it with
gpg diskus-0.5.0-20081214-023903.tar.gz.asc
If you dislike the cryptic output of GnuPG, then try
gpg --batch diskus-0.5.0-20081214-023903.tar.gz.asc 2>/dev/null </dev/null; echo $?
  • If this gives you "0" (for OK), then everything is OK.
  • If you see "1" you probably have a corrupt downlod (or mixed file names).
  • If you see "2", then you need to download the key from the keyserver. The ID of the missing key is printed to STDERR by gpg, use the former variant to see it.

If you are puzzled how to enter this commands under Windows, consider installing CygWin. If you can understand German, perhaps you want to read my rants about GnuPG.


Q: What does a good signature tell me?
A: Nothing!

The only thing it tells you is:

  • The download worked.
  • Somebody, probably me, once has uploaded a proper key to the keyserver. Note that keyservers do not even check the eMail address.
  • Somebody, probably me, was able to create a proper signature file and uploaded it to this server here.
  • If somebody else, who is not me, is able to access my system unauthorized, it is very likely that this one can also fool you by creating a false signature and change all references to the signature to his one.
  • Also it is a weak key (read: poorly defended), so it is possible that some evil person can gain access to my signing key somehow (in Germany, where I live, the state nowadays is allowed to compromize any computer's security using the Bundestrojaner, I am not able to protect my keys against that threat, therefor my key must be considered compromized, too)
  • To get some more safety from all that key checking is an extremely complicated task, which even is error prone to absolute experts. So it is beyond any FAQ.
  • So if you ask yourself, what this key checking everywhere is all about, then I only can say, it is plain snake oil
  • Even I (only the rightfully owner of the signature) have limited checks to proof the validity of the signature. This check cannot be automated (any automation can be broken) and I can only check what I know of, so I cannot even find out if my keyfile was stolen somehow (as it is used for automatic signing, I cannot hide it away).
  • You (the downloader) can tell from the signature IDs, that the files are somewhat related and that a newer version, which is signed by the same key, probably is not compromized, if the previous version you have, isn't compromized, too.


Q: Have you tried to get them into coreutils?Have you tried to get them into moreutils?
A: No, but thanks for noting (as I did not know of moreutils before).

Please note, that I will support anybody, who tries to change my utilities such, that they can be added to some standard-package. Also note, that there is the RSS feed to stay informed about any changes to this site.

But I (probably) never will submit any of my tools to some standard- or semi-standard-package. This is for several reasons:

  • I simply lack the time to do quality assurance. Sometimes changes break CygWin compatibility or backward compatibility (to S.u.S.E. 5.3 for example). I will fix this as soon as I notice (or need this feature on some other platform), but often this will take months to years.
  • The tools seem to share a common code base (my library), but this library changes frequently and often breaks things. So the tools are not on the same level of the common code base. If the tools are combined somewhere, it would be right to base them on the same common code level. That's too much effort for me now.
  • Everything is continuously evolving. I add things as soon as I need something or when I am on it, find the time and think, it would be nice to have. This is bad for some "standard" distribution, as it is irritating if tools are not "stable".
  • I want to be able to change every aspect of my tools. As soon as they become somewhat "standard" this will make this widely impossible. It's hard to drop a feature against a big userbase. Today you can simply require a certain version of my tool (all old versions can be downloaded here as well).
  • Many of my tools have a really weird or very unusual way to do things. But they are lacking a common default way to do it. For example "printargs", "md5chk" and "sq3" each have different defaults when it comes to how to escape things to the shell. This is confusing, so it probably would be wise to fix that first. If I had the time to do it, I would have done this already.
  • Doing it in several small tools perhaps is stupid. I want to combine things into a better suited toolset somehow, but I am not really sure where this will lead me to. Note that I am working (very slowly) on something called "shit", which means "SHell-IT" or "SHell Intetgrated Tools", which will - as the name says - provide some tool-into-tool-integration, which probably make several of my tools superfluous, as some functions can be reached in a more common (perhaps a little bit more elaborate) way.
  • I often replace standard tools (like tee) with my own version (like unbuffered), just because the standard tool does not do what I expect. For 99% of all tasks the standard tool is enough, but for the 1% lacking I need to re-invent the wheel. (For example I really have trouble with that /bin/sh does not support coprocesses well. Ksh and bash5 are of no relief, as either it is not available or not widely deployed.) This is not the right way to do it. The right way to do it to get the standard utilitiy to support my needs. But I will never succeed with such a task, so the tools are written for solely my own purpose for now and perhaps can help out others.
  • Some tools are badly implemented versions of more useful utilities, for example "socat" is extremely powerful when it comes to networking needs. However often I need just a small subset of this possibilities, and the capacity of the machine might be limited (like an NSLU2 or some 16 MB machine which is over 10 years old), such that I need such a stripped down tool. Adding it to some package would be redundant.
  • Also I am not happy with quite some of them, but I am lacking the time to make them fit together in a better way. I cannot submit anything I am not completely happy with. For example: gcopy was done in a hurry, and therefor was based on mvatom, which means, it is unsuitable for what I want it to do on the long term, so there is a long way to go until I would suggest it to others.
Again to stress it: Anybody is welcome by me, who wants to contribute, for example by adding one of my utilities to some other package. Everything here is GPL or CLL (some weird PD style license from me). However my support is somewhat limited, so somebody else must do all the packaging stuff etc.


Q: diskus gives:fatal error: FTLM100E 102400 alloc failed
A: This is due to some old POSIX library I think. The posix_memalign() function fails if the blocksize is not a power of two. There is a workaround: Give option -bs 1M or -bs 256M


Q: cc1: Invalid option `-Wno-unused-function'
A: Your gcc is too old. Just remove -Wno-unused-function from Makefile, but leave the rest of the two lines intact.


Q: Why the move to GitHub?
A: Because GitHub is better suited to distribute source only software. Also this domain probably can lead there, too, if the transition has finished. Thanks to the distributed nature of GIT, this step cannot be false. If GitHub ever goes out of business, it is easy to distribute the repositories somewhere else.


Q: Why GIT?
A: When GIT came around it was difficult to use, lacked a lot of features and was clumsy to use. So I left it alone because it was not mature. Other VCS failed for me, like mercurial, monotone or SVN. There are others, but I never came around to test all.

In the meanwhile GIT evolved, and suddenly, there it was, a cross platform VCS monster which was easily to tame and use. Some time ago I gave GIT another chance and I was surprised how powerful it had become. The right tool to look at it, again.

  • GIT is lacking CVS tags. That's the only drawback I know of.
  • However GIT is a configuration monster, so they could be brought back, if really needed.
  • I am a die hard CVS user. CVS suits all my personal needs, and never left me alone. The server was easy to set up (inetd), works over socket tunnels, does not need any other high level packages, is easy to maintain (for a single user), is fast enough for me (small projects) and works out of the box.
  • The only real drawback of CVS is, that it is nearly impossible to synchronize two (diverging!) CVS servers. CVS is not suited for "the cloud" (whatever one thinks "the cloud" is).
  • So, sadly, CVS cannot be used as a platform for easy source code distribution, in a failing world which needs redundancy.
  • OTOH GIT, except for CVS-Tags, can replace all what I want from CVS. And it continues to be relatively easy to use. (At least from my point of view. It might differ if you are suited to SVN or other monolitic CVS).
  • GIT even outclasses CVS by far: It has a staging area so you can combine different files to commit more easily, is cryptographically secure, can link other git repositories as submodules, integrates SSH as native authentication and transport mechanism, works in a distributed world and made it possible to have services like GitHub which are easy to integrate and keep up to date.
Not to forget, it's also easy to migrate from CVS to GIT without loosing history. After starting with GIT it quickly became an essential part of my software development.


Q: Which things are missing from GIT?
  • CVS tags. CVS tags are increadibly helpful to distinguish software versions. Currently I did not find a good way to support this with GIT, too, so I have to maintain versions manually, which is error prone.
  • Automatic version numbering. I would like to have automatic version numbers. This is like CVS revisions, such that you cannot forget about them. Version numbers could be manually maintained as tags, but this is - again - clumsy.
  • Bootstrapping. If you get a GIT repository without GIT you are doomed. A good VCS should offer a way to bootstrapp from nothing. It should include something like a rosettastone, which allows you to decipher what you have found. Bootstrapping does not need to be easy, but you must be able to follow it, without prior knowledge or access to GIT. Imaginge a future alien race who discovers a, perhaps partly broken, GIT repository on our planet, when nobody is around to explain what GIT is.
  • Perhaps more, but I will not maintain this list here.


Please note that this page probably will be replaced by Wikis on GitHub, slowly.

[end of page - jump back to content][hacker culture]
Last modified: 2011-09-12 by Valentin Hilbig [ Imprint / Impressum ]