IllegalStateException: Content has been consumed

When working with Android or (to be more general) Apache HttpComponents, one should keep in mind that it depends on the HttpResponses (Api: Apache, Android) if they can be consumed multiple times or not. And if the response is not, then better triple check your code, what you’re doing.

I experiences the (very ugly) issue during the apparently simple task of reading the content of the response. )For ease of simplicity, I also use Apache commons-io, to read buffers etc.)

HttpResponse response = ...;
String content = IOUtils.toString(response.getEntity().entity.getContent());

Okay, how many possible Nullpointer Exceptions do you see? I didn’t care until I experienced the first one, so I made the code NPE safe:

HttpEntity entity = response.getEntity();
// entity can be null according to the docs!
if (entity != null) { 
    // The interface doesn't say that getContent() MUST return non-null
    InputStream stream = entity.getContent(); 
        if (stream != null) {
            tempContent = IOUtils.toString(entity.getContent());
        }
}

And suddenly I was faced with IllegalStateException: Content has been consumed. As I also did changes somewhere else, I assumed the error in some toString()-Methods that would read the content of the response during debugging. But as the error also showed up without debugging, I had a closer look to my “improvement”.

Well, the error was the call IOUtils.toString(entity.getContent()); which tried to re-aquire the Input Stream. But as I just aquired it two lines above for the null check, the content was already marked as consumed. So the (now hopefully) correct and robust code is:

HttpEntity entity = response.getEntity();
// entity can be null according to the docs!
if (entity != null) { 
    // The interface doesn't say that getContent() MUST return non-null
    InputStream stream = entity.getContent(); 
        if (stream != null) {
            tempContent = IOUtils.toString(<strong>stream</strong>);
        }
}

And the moral of the story

Be very carefull when reading HttpResponses! Also avoid pretty-printing the content in toString() – this might suddenly also consume your content. And good luck finding the point where you consume the content in such cases.

But .. why?! Please avoid returning null!

Yet I still wonder, why so many methods are allowed to return null instead of just an empty stream or something. All the Null-checks don’t make the code more readable. Some programmers might even be tempted to simply put an catch(NullPointerException e) around the part of

response.getEntity().entity.getContent()

. Nothing I’d really recommend but I could also understand if I’d see something in other code.

Android HTTP POST authentication error with Basic Auth

In one of my Android apps, a user should be able to push JSON data to the server. Of course this should only be allowed if the user is authorized by his credentials. For simplicity, I decided to use Basic Auth via HTTPS.

As I usually do not deal with Connectsions directly, I prefer to use the Apache Http classes to make life easier. So I used code like the following:

HttpParams httpParams = new BasicHttpParams();
HttpProtocolParams.setContentCharset(httpParams, HTTP.UTF_8);
HttpProtocolParams.setHttpElementCharset(httpParams, HTTP.UTF_8);
HttpConnectionParams.setConnectionTimeout(httpParams, 5000);
client = new DefaultHttpClient(httpParams);
client.getCredentialsProvider().setCredentials(
    new AuthScope("localhost",80),
    new UsernamePasswordCredentials(username, password)
);

This was fine and worked in my local test environment and also in the live stage with GET requests. But when I started to POST data to the server I always got a “401 Unauthorized”. First I thought It’d be malformated JSON or encoding issues – but even with an empty string as payload I simply didn’t get through. Also the “same” POST request from the commandline using CURL worked like a charm! So it was very likely to be an issue with my handling of the Apache HTTP Lib. Stackoverflow also showed various people with the same problem. The proposed solution that came up several times was to manually set up the auth header. Not really the solution that I thought would be the greatest one.

After some more StackOverflow Posts and API crawling I found the very simple solution:

client.getCredentialsProvider().setCredentials(
   new AuthScope(AuthScope.ANY),
   new UsernamePasswordCredentials(username, password)

And yes, it was that easy! Just set the Auth scope correctly.

I’m just very surprised that I only encountered issues with the POST request. The very same configuration worked with a GET without problems. This was actually the point that made me sure that this part of the code was correct.

RaspberryPi Weatherstation – The Sensor Station (Part II)

The second post of this series describes the RasperryPi that is connected to the sensors and used to record the data.


Part 1Part 2Part 3


Hardware

As described in the first part, I decided to use sensors from Tinkerforge as they are comparatively cheap, require no soldering, no electrical know how and they also provide a convenient API to get the data from the sensors. Ideal for people like me who want to spend more time with using the data instead of playing around with wires, resistors, etc.

The required hardware was just.

Raspberry Pi - Sensor Station

Continue reading RaspberryPi Weatherstation – The Sensor Station (Part II)

My first RaspberryPi project: TV-Controlled Weatherstation (Part I)

Part 1Part 2Part 3


History

For quite some time I’ve been playing with the thought about bying a Raspberry Pi. Yet the biggest issue was: I had no use case for it!

I always liked information about the weather – especiallly I wanted to know in the morning how cold it was at night etc. So the use case was found: a weather station! Yet I was pretty distracted from all examples that I found on the web as I wanted a visually appealing UI so that it is fun to look at the data. Usually the samples just controlled a small LED display to display some values but – honestly – I don’t need a display at the weatherstation directly – I mean – shouldn’t it be somewhere outside and not in the room?

Continue reading My first RaspberryPi project: TV-Controlled Weatherstation (Part I)

Tomcat MongoDB Authentication Realm

Tomcat MongoDB Authentication Realm

Tomcat provides a nice and powerful mechanism for authentication by the so called Tomcat Authentication Realms.
In the Tomcat Docs, Realms are explained as:

A Realm is a “database” of usernames and passwords that identify valid users of a web application (or set of web applications), plus an enumeration of the list of roles associated with each valid user. You can think of roles as similar to groups in Unix-like operating systems, because access to specific web application resources is granted to all users possessing a particular role (rather than enumerating the list of associated usernames). A particular user can have any number of roles associated with their username.

Continue reading Tomcat MongoDB Authentication Realm

A basic Java WebApp example project supporting an xml free Jersey REST config

RESTful Webservices are an essential and convenient part in current web applications when Information should be exchanged. Additionally the main exchange formats in current webservices are either JSON or XML.

So when we want to implement the serverside of a web application using Java and especially REST services, we might end up serializing and deserializing objects to and from XML and JSON. Honestly not a very desirable task. But fortunately there is the Jersey project which can help us in this task.

Continue reading A basic Java WebApp example project supporting an xml free Jersey REST config

“Easy going” vs “Taking care”

Freitag Abend. Heimfahrt in der BOB (BayerischeOberlandBahn). Wochenende here I come!

In Bad Tölz (eine Station vor meinem “Zielflughafen”) höre ich “Hausham? Da sind sind im falschen Zug.” (Hätte man vor einer halben Std in Holzkirchen umsteigen müssen). Ich denke mir “Arme Sau – jetzt fährst erst mal wieder zurück – oder du kennst hier jemanden der dich fährt – Ist mir aber wurscht – denn ICH bin gleich daheim. HA-HA!”.

Der Zug steht noch – ein junges asiatisches Mädchen das gerade ins Teenageralter eingetreten sein muss läuft etwas hektisch herum und spricht mich dann an “schuldi-gung – Hausham?”
Ohoh. “Hm tut mir leid, falscher Zug, steig schnell um in den Zug am anderen Gleis. Du musst zurückfahren.” Verzweifelter Blick ihrerseits. “ik ver-stehe nicht?”
Oh verdammt – kurzer Versuch in Englisch – nada. Noch maximum 2-3min bis der andere Zug fährt. Oh – scheiße – das bekomme ich jetzt nie erklärt. Die Hoffnung stirbt zuletzt: Noch ein Versuch es ihr zu erklären. Auch die Taxioption versteht sie natürlich nicht.

Option A) Schultern zucken, mich nicht weiter kümmern, das Mädel bis nach Lenggries fahren lassen (wo sie nicht weiter von Hausham weg sein könnte) und gleich daheim sein (also ich – sie sicher nicht). Ist ja schließlich nicht mein Kind.
Option B) Zusammenpacken, das Mädel in den anderen Zug setzen und hoffen, dass mich irgendwer fahren kann (oder dass diesmal wenigstens Taxis da sind) weil MEIN Zug dann ganz sicher weg ist.

Continue reading “Easy going” vs “Taking care”

“bad subifd directory” Replacing/removing bad SubIFD

Handling bad SubIFD entries in photos you want to geo tag / the problem

I just converted some of my RAW photos in RawTherapee and polished them in Photoshop CS. Of course I also wanted to GeoTag them (using GeoSetter) before putting the images in my archive. Fortunately, a GPS point was found in my trace for all photos. Unfortunately, I couldn’t write the Exif-GPS position due to a “bad subifd directory”. That’s what exiftool (which geosetter uses) tells me.

So – how to get the geo information into exif – or how to get the bad SubIFD directory out?

The solution:

In the end the issue was rather easy to solve:

  1. copy your images to a linux machine (or vm or whatever)
  2. copy all images to a directory “conv” AND in parallel to “conv2
  3. enter “conv2
  4. remove all exif information from the conv2 images by
    find ./*.jpg -exec exiftool -exif:all= {} ;
  5. enter “conv1” & copy the exif information (except SubIFD) to the cleaned images in “conv2” by
    find ./*.jpg -exec exiftool -tagsfromfile {} -exif:all --subifd:all ../conv2/{};
  6. you’re done.
  7. (fire up geo setter and geo tag your photos)

Be happy!

Convert Mercurial repository to Git

I have converted yet another googlecode Mercurial repository to Git – and as it took me (again) a bit too much time, here is my recipe:

Prepare

Log into a Linux shell (Windows will hardly work – at least it didn’t work for me). If you don’t have some remote shell, download an Ubuntu VM and fire it up (you might need to install VirtualBox if you haven’t installed it already).

Check out your HG repo:

hg clone https://code.google.com/p/MyHgProject/

Check if you have to remap some author information:
hg log MyHgProject

If you want/have to remap, simple create an author map file
oldemail@example.com=Firstname Lastname

Convert

Get fast-export and convert your repo:
git clone git://repo.or.cz/fast-export.git
mkdir new_git_repo
cd new_git_repo
git init
/path/to/hg-fast-export.sh -A ../authormap -r /path/to/MyHgProject
git checkout HEAD

Push

Now change the source setting in your google code repository to git and push the local codebase:
git remote add origin https://code.google.com/p/MyHgRepo/
git push --all

At this point, your upload might fail with something like “error: RPC failed; result=35, HTTP code = 0”.
This can happen if your upload takes too long and this is a documented bug.
I simply solved this issue by pushig the git repo from a shell with a fast enough upload speed as my local connection obviously was too slow.

Wiki

Now you might realize that you have “lost” your complete wiki – don’t worry, it’s still there!
Switch back your repo setting to Mercurial and repeat the process for your wiki which you can usually find at https://code.google.com/p/yourProjectName/source/checkout?repo=wiki

That’s it. You should now be ready to use Git!


Thanks to following sites hedonismbot, scrambled tofu

No more windows backup after uninstalling Acronis True Image Home 2011

I gave Acronis True Image Home 2010 and 2011 really quite several chances but it seems that either the product cannot reliably perform regular backups for my computer or I am just incomatible with it. Anyways – I finally wanted to draw a final stroke, get rid of Acronis True Image Home (ATIH), uninstall it and try Windows’ native Backup and Restore.

After unsintalling TIH I wanted to configure Windows’ backup and restore tool.
But – the according window just didn’t open!? Is TIH still bothering me AFTER I uninstalled it?

After quite some googling, I found one article in the Acronis Knowledge Base “14741: Acronis True Image Home 2011: Disabling “Integrate True Image into Windows” Does Not Work“. Unfortunately this only applies to an installed Version of TIH where Dis-Integrating TIH from windows backup does not work. – It’s quite a while since I tried it – but if I remember correctly, this didn’t work out for me. So another round at google was waiting for me…

Continue reading No more windows backup after uninstalling Acronis True Image Home 2011