“bad subifd directory” Replacing/removing bad SubIFD

Handling bad SubIFD entries in photos you want to geo tag / the problem

I just converted some of my RAW photos in RawTherapee and polished them in Photoshop CS. Of course I also wanted to GeoTag them (using GeoSetter) before putting the images in my archive. Fortunately, a GPS point was found in my trace for all photos. Unfortunately, I couldn’t write the Exif-GPS position due to a “bad subifd directory”. That’s what exiftool (which geosetter uses) tells me.

So – how to get the geo information into exif – or how to get the bad SubIFD directory out?

The solution:

In the end the issue was rather easy to solve:

  1. copy your images to a linux machine (or vm or whatever)
  2. copy all images to a directory “conv” AND in parallel to “conv2“
  3. enter “conv2“
  4. remove all exif information from the conv2 images by
    find ./*.jpg -exec exiftool -exif:all= {} ;
  5. enter “conv1” & copy the exif information (except SubIFD) to the cleaned images in “conv2” by
    find ./*.jpg -exec exiftool -tagsfromfile {} -exif:all --subifd:all ../conv2/{};
  6. you’re done.
  7. (fire up geo setter and geo tag your photos)

Be happy!

Convert Mercurial repository to Git

I have converted yet another googlecode Mercurial repository to Git – and as it took me (again) a bit too much time, here is my recipe:

Prepare

Log into a Linux shell (Windows will hardly work – at least it didn’t work for me). If you don’t have some remote shell, download an Ubuntu VM and fire it up (you might need to install VirtualBox if you haven’t installed it already).

Check out your HG repo:

hg clone https://code.google.com/p/MyHgProject/

Check if you have to remap some author information:
hg log MyHgProject

If you want/have to remap, simple create an author map file
oldemail@example.com=Firstname Lastname

Convert

Get fast-export and convert your repo:
git clone git://repo.or.cz/fast-export.git
mkdir new_git_repo
cd new_git_repo
git init
/path/to/hg-fast-export.sh -A ../authormap -r /path/to/MyHgProject
git checkout HEAD

Push

Now change the source setting in your google code repository to git and push the local codebase:
git remote add origin https://code.google.com/p/MyHgRepo/
git push --all

At this point, your upload might fail with something like “error: RPC failed; result=35, HTTP code = 0”.
This can happen if your upload takes too long and this is a documented bug.
I simply solved this issue by pushig the git repo from a shell with a fast enough upload speed as my local connection obviously was too slow.

Wiki

Now you might realize that you have “lost” your complete wiki – don’t worry, it’s still there!
Switch back your repo setting to Mercurial and repeat the process for your wiki which you can usually find at https://code.google.com/p/yourProjectName/source/checkout?repo=wiki

That’s it. You should now be ready to use Git!

Thanks to following sites hedonismbot, scrambled tofu

How to write useful error messages

Each programmer has to deal with error mesages in some way. Either we have to check our data (and possibly raise an error/exception) or we are using libraries and have to deal with error messages or exceptions that are raised in these libraries. The purpose of this post is to point out how meaningfull error messages can save you time. Either your own time (by avoiding answering support questions) or your own debugging time.

So how can an error message save or waste time? Imagine the situation where you try to open a file and all you get is an error. What do you do next? I guess you’ll probably check if you’re trying to open the correct path. Maybe you did, then you possibly check if the permissions are okay, if you’re trying to open a directory or a file, etc until you found the problem that is causing the error. Actually this can be quite some wasted time – why? Becasue you’re doing all the stuff that the system already checked. The system decided that one of the condidtions failed and exited. So – shouldn’t the system just TELL the exact reason? And even better: shouldn’t it also just tell you a possible solution? Well – actually I do not blame “the system” but the guy who write the line that threw the error without any further information. Everytime I get an absolutely uninformative error I think “was it really so damn hard to add one more line of code that just mentions WHAT went wrong?!”.

Assume the situation where you try to open a file with the path being configured in an external config.properties file.
I think the quality of an error message can be categorized in one of the three categories:

  1. Reporting: The system just reports that there was an error.
    Example: IO Exception occured.
  2. Informing: Same as 1, just with more information.
    Example:  IO Exception occured while opening file: <filename>, null=false, exists=true, is file=false, is directory=false, readable=false, writable=false
  3. Supporting: same as 2 with additional information of how to solve the problem.
    Example: IO Exception occured while opening file: <filename>, null=false, exists=true, is file=false, is directory=true, readable=false, writable=false; Was the pat set correctly in config.properties?

In the past weeks I have seen very useful error messages in Google’s Android (“… Have you declared this activity in your AndroidManifest.xml?”) or in twitter4j (Displaying a shortURL to an FAQ page explaingin a very common error).
Bad examples are plain NullPointerExceptions when 3 parameters were checked, one of them was null and the according is just followed by a throw new NullPointerException so that you even did not recognize from the message WHICH parameter was null.

Conclusion: Please write robust code and please provide helpful and supportive error messages to the folks using your code!

An introduction to the Logging framework (a.k.a. System.out.println is evil)

Motivation

One of the first lines a programmer will write in a new language is surely “Hello World”. In Java you can write to the console or to the error stream quite easy with a simple System.out.println("Hello World") or System.err.println("Hello Error"). Great! When the code grows, bugs creep into the code and make live a bit harder. At this point programmers should definately start a deep and loving relationship with the debugger that is delivered with the IDE instead of using System.out/err.println() as debug method. Nevertheless – there are at cases, where a Debugger cannot (or hardly) be applied:

  1. The code runs in the IDE but not if startet directly. – What the hell’s going wrong?
  2. Handling of exceptions. An exception indicates a state that should not have happened and therefore it might be considered to be logged.
  3. The code is deployed to someone else and you cannot attach the debugger to his/her machine.

At either point, beginners tend to use System.out/err.println() to trace the execution path. While this might be okay if the onlyone that is using the code is the programmer alone, this can be very annoying if you are working in a team: If you forget to remove the debug messages, you’re polluting someone elses console output. Even worse: if the code is deployed to a client which reports an error, you cannot raise/lower debug levels or just enable/disable debugging. Do you really want to send a “debug version”? (No you wouldn’t.)

Continue reading An introduction to the Logging framework (a.k.a. System.out.println is evil)

Rooting and upgrading my HTC Tattoo to CyanogenMod 7

Yesterday I decided that the days of my HTC Tattoo running original HTC-Android-1.6 were numbered – finally. Actually the thought about updating was in my mind for QUITE some time. But with all the different how-to-update guides and “oh my god I broke my phone”-posts, I was really frustrated and unsure if I should really risk to “brick” my phone (a.k.a.: turn it totally unusable).

Well okay – so I crawled the web once more for an up-to-date update guide. Well – to make a (rather) long story short: I achieved my goal!

Continue reading Rooting and upgrading my HTC Tattoo to CyanogenMod 7

Parse Error: There is a problem parsing the package / Beim Parsen des Pakets ist ein Problem aufgetreten

Just finished my first (real) Android App and wanted to deploy it in my phone (not just in the emulator).
So I sent an email to myself with the .apk attached as several tutorials said that you can just open it from your mail to install the app.
On the phone, I opened my mail app K9 and opened the .apk. But instead of installing I unfortunately just got

Beim Parsen des Pakets ist ein Problem aufgetreten.
(in english) Parse Error: There is a problem parsing the package.

even though the minSdkVersion was set correctly.

The solution was rather simple: Do NOT open the .apk in K9 directly but save it to the sdcard and open it via a filemanager like Astro.

Knowing: A Generic Data Analysis Application

We got another demo accepted:

Knowing: A Generic Data Analysis Application

Thomas Bernecker, Franz Graf, Hans-Peter Kriegel, Nepomuk Seiler, Christoph Türmer, Dieter Dill
To appear at 15th International Conference on Extending Database Technology (2012)
March 27-30, 2012, Berlin, Germany

Abstract:

Extracting knowledge from data is, in most cases, not restricted to the analysis itself but accompanied by preparation and post-processing steps. Handling data coming directly from the source, e.g. a sensor, often requires preconditioning like parsing and removing irrelevant information before data mining algorithms can be applied to analyze the data. Stand-alone data mining frameworks in general do not provide such components since they require a specified input data format. Furthermore, they are often restricted to the available algorithms or a rapid integration of new algorithms for the purpose of quick testing is not possible. To address this shortcoming, we present the data analysis framework Knowing, which is easily extendible with additional algorithms by using an OSGi compliant architecture. In this demonstration, we apply the Knowing framework to a medical monitoring system recording physical activity. We use the data of 3D accelerometers to detect activities and perform data mining techniques and motion detection to classify and evaluate the quality and amount of physical activities. In the presented use case, patients and physicians can analyze the daily activity processes and perform long term data analysis by using an aggregated view of the results of the data mining process. Developers can integrate and evaluate newly developed algorithms and methods for data mining on the recorded database.

BibTex

@INPROCEEDINGS{BerGraKriSeietal12,
  AUTHOR     = {T. Bernecker and F. Graf and H.-P. Kriegel and N. Seiler and C. Tuermer and D. Dill},
  TITLE      = {Knowing: A Generic Data Analysis Application},
  BOOKTITLE  = {Proceedings of the 15th International Conference on Extending Database Technology (EDBT), Berlin, Germany},
  YEAR       = {2012}
}

More informations will be published at the official publication site at the LMU.

Research Idea: Evaluation of Traffic Lane Detection with OpenStreetMap GPS Data

I am soon leaving University and thus the time for pure research will soon be over. Unfortunately I still have some ideas for possible research. I’ve tried getting them out of my head as this has not yet worked out, I’ll try to write them down – maybe somewone finds them interesting enough for a Bachelor-/Masterthesis or something like that …

Introduction

OpenStreetMap creates and provides free geographic data such as street maps to anyone who wants them. The project was started because most maps you think of as free actually have legal or technical restrictions on their use, holding back people from using them in creative, productive, or unexpected ways. The OpenStreetMap approach is comparable to Wikipedia where everyone can contribute content. In openStreetMap, registered users can edit the map directly by using different editors or indirectly by providing ground truth data in terms of GPS tracks following pathes or roads. A recent study shows, that the difference between OpenStreetMap’s street network coverage for car navigation in Germany and a comparable proprietary dataset was only 9% in June 2011.

In 2010, Yihua Chen and John Krumm have published a paper at ACM GIS about “Probabilistic Modeling of Traffic Lanes from GPS Traces“. Chen and Krum apply Gaussian micture Models (GMM) on a data set of 55 shuttle vehicles driving between the Microsoft corporate buildings in the Seattle area. The vehicles were tracked for an average of 12.7 days resulting in about 20 million GPS points. By applying their algorithm to this data, they were able to infer lane structures from the given GPS tracks.

Adding and validating lane attributes completely manually is a rather tedious task for humans – especially in cases of data sets like OpenStreetMap. Therefore it should be evaluated if the proposed algorithm could be applied to OpenStreetMap data in order to infer and/or validate lane attributes on existing data in an automatic or semiautomatic way.

Continue reading Research Idea: Evaluation of Traffic Lane Detection with OpenStreetMap GPS Data

SRTM Plugin for OpenStreetMap

One of the features of our TrafficMining Project at the LMU was to use additional attributes in routing queries. Standart routing queries usually just use time and distance for calculating the fastest/shortest routes. In order to have an additional attribute we decided to use evelation data as this might be an issue if you also want to take fuel cost into account or if you’re planning a bike tour (instead of crossing a hill, you might want to consider a longer tour that avoids crossing the hill).

The problem just was that data nodes from OpenStreetMap are defined mostly by id, latitude and longitude, which is totally enough for painting 2D maps and standard routing queries. As the elevation is not added to OpenStreetMap data directly (and it is also not intended to be added to the OSM data base), a component was needed that parses both Nasa SRTM data as well as OSM data files in order to combine the data.

In the first version, we parsed the SRTM data directly and addied it to the nodes of the OSM-Graph directly. During one refactoring, we decided to integtrate Osmosis into the project in order to be able to read PBF files (another OpenStreetMap file format). During this integration we decided to separate the SRTM parsing into a separate module so that other people can make use of it as well. The plugin was open sourced some time ago at google code as the “osmosis-srtm-plugin” under an LGPL licence.

Relevant Links:

Drupal shows warnings after migrating to new server

Recently we migrated a Drupal managed website to  a new server. – Everything was fine, just on some pages, some php-warnings showed up that did not show up on the original site.
After some investigation I checked php’s error_reporting setting and realized that the settings of the new new server were more sensitive than the old one.

The solution was pretty simple. As I definately didn’t want to start hacking around in Drupal, I just turned off showing the errors to the user:

Administer > Site configuration > Error reporting
(or in german:) Verwalten > Einstellungen > Error reporting
/admin/settings/error-reporting
Error Reporting: Write errors to the log (instead: Write errors to the log and to the screen).

Pretty easy.