How to fix: IntelliJ IDEA and Scala being awfully slow on Windows 8.1

At work we are working mostly in Scala and most of us are using IntelliJ IDEA for coding. The choice of the operating system is up to the developer. As I am quite convenient with Windows (and use MS Office quite often), I am a happy Windows 8.1 user (btw: who the hell needs a start button when you have the windows key!? Anyways … different story).

The Problem

After a while when I started a Scalding project, IntelliJ became very slow and often turned to be non responsive for some seconds about once per minute. So over all it was a very inconvenient and unproductive situation.
Continue reading How to fix: IntelliJ IDEA and Scala being awfully slow on Windows 8.1

RaspberryPi Weatherstation – The TV Station (Part III)

The third and last post of this series describes the RasperryPi that is connected to the TV and displays the sensor data in a visually appealing way.

Part 1 – Part 2 – Part 3

In the past blog post, the first Raspberry Pi was connected to several sensors. Tiny Python scripts poll the data from the sensors regularly and save them to simple text files that can be copied via SSH/SCP. Current data can be obtained from the sensors by directly connecting to the Brick Daemon which runs on this RasPi.

The main focus of this post is the visualization of the data via a JavaFX application and how to control the FX application by using the regular remote control of the TV. But – before we’re divig into the details, I want to teas you with a screen shot of the final result 😉

Wetterstation blue edition

Nice isn’t it? So let’s get started

Things to discuss

The most important question first: What do I want to see and do?
I had a pretty clear intention already: I want to see the current values of all sensors in a small overview and I want to be able to toggle through time series of the past days.

Toggling should be done by the TV’s remote control as I would have the remote at hand already when I switch to the Weather data display. I’ve seen several guides where an IR-receiver and an additional remote were used to control the RasPi. Yet I simply do not like the idea of having another remote control on the table – right next to the other ones. Also controlling the app via mobile phone wasn’t what I wanted as we usually still simply use the TV’s own remote to control the TV. – Also I know from Raspbmc that it is possible to accomplish this by using libCEC – somehow.

Next question to clarify: How should the data be visualized on the TV. This was easy: JavaFX.
Colleagues asked me why I chose FX instead of HTML5. Well mainly there were three reasons:

  • I simply wanted to do something in JavaFX
  • I’m not a fan of fizzling around with JavaScript and CSS. And the standard way to displaying webapps on the RasPi seems to be using Midori. A browser I never worked with before – and even thinking about possible CSS/JS incompatibilities totally turned me off.
  • I wanted to control the UI with the TV’s remote control. That has to be done somehow using the CEC-commands that are sent through HDMI. I’ve never done that before and anticipated my chances to accomplish this in pure Java ways higher than somehow creating input events that I had to redirect to the browser.

Checking the Hardware

Prior to coding, I did some research if and how the CEC commands are sent through HDMI. And – unfortunately – besides all software problems, a lot of people reported problems to even get the signal to the RasPi! Possible problems mentioned throughout several posts included the TV software having disabled the CEC functionality or HDMI cables that were blocking / not forwarding the CEC signals.

What I found the easiest way to test the CEC compatibility of the setup was Raspbmc. Raspbmc is a mediacenter for Raspberry Pis, which is easy to install and supports CEC. So: download the image, flash it to an SD card, connect the Pi to your TV with the HDMI cable that you want to use later as well and start the Pi. If you see your TV’s remote working: Great! Otherwise: you have my sympathy. Good luck in finding out what is wrong with your technical setup.

Install libCEC

So the CEC signals are technically consumable by the Pi. Let’s istall libCEC.
DO NOT install the packages from the pulse-eight website. And also DO NOT simply recompile and install libCEC following the guide on the GitHub page. It’s important to compile the lib for the Pi! Just follow one of the step by step guides to compile and install:

$ cd /tmp
$ sudo apt-get install build-essential autoconf liblockdev1-dev libudev-dev git  libtool pkg-config
$ git clone git://github.com/Pulse-Eight/libcec.git
$ cd libcec
$ sudo ./bootstrap
$ sudo ./configure --with-rpi-include-path=/opt/vc/include --with-rpi-lib-path=/opt/vc/lib --enable-rpi
$ sudo make
$ sudo make install
$ sudo ldconfig
$ cec-client -l

LibCEC is now installed including cec-client (seen in the last call). Play around a bit with cec-client. Up to now everything was fine on our Samsung TV. But beware, I’ve seen posts where people with TVs from other vendors faced serious problems … If you want to play around with cec-client, CEC-O-MATIC might also be a reference you should have a look at.

Install Java 8

At the time of writing, Java 8 is already released. At the time I was writing the code, it was still an EA release (Early Access). I read several posts that recommended installing the ARM version of Java 8. So I simply followed the OpenJDK guide to install OpenJDK 8:

  1. download Java 8 ARM from Oracle
  2. unpack the file: sudo tar zxvf jdk-8-linux-arm-vfp-hflt.gz -C /opt
  3. and check if Java 8 got installed: /opt/jdk1.8.0/bin/java -version
  4. Set default java and javac to JDK 8: 
    $ sudo update-alternatives --install /usr/bin/javac javac /opt/jdk1.8.0/bin/javac 1
$ sudo update-alternatives --install /usr/bin/java java /opt/jdk1.8.0/bin/java 1

$ sudo update-alternatives --config javac
$ sudo update-alternatives --config java

$ java -version
$ javac -version

    java and javac should link to 1.8.0 now.

Next adjust the memory split option as also mentioned on the page to 256mb (gpu_mem=256 in /boot/config.txt). It’s useful to read the page. The page also gives a note which is pretty important to avoid frustration:

Note that the default configuration of JavaFX on the Raspberry Pi does not use X11. Instead JavaFX works directly with the display framebuffer and input devices. So you should not have the X11 desktop running when starting JavaFX.

JDK 8 EA builds for the Raspberry Pi include full support for hardware accelerated graphics, with everything from the base, graphics, controls and FXML modules. Media and Web modules are not included.

So configure the Pi to boot just to the shell (via sudo raspi-config) and do not try to see the JavaFX output via VNC!

Building the GUI

Designing the UI (or: how I started to love JavaFX)

Before I even started to code, I wanted to set up the layout. Coming from a Java-Swing background I was really excited to try this JavaFX thingy that was said to be so much cooler than Swing.

After setting up the project in NetBeans and installing JavaFX Scene Builder I was very positively surprised! By the way, I was only working with Scene Builder 1.1, the current Version 2 is said to be even better. After initializing the project, my focus quickly went into the src/main/resources folder. At this location you can find fxml/FXMLDocument.fxml and styles/base.css (later renamed to metro.fxml/.css). Designing the Gui was about 90% done just in Scene Builder (for the .fxml) and Netbeans (for the.css). The really really nice thing is that it is a true WYSIWYG editor. Also whenever the css file is being edited, the effect is immediately visible in Scene Builder. Simply great compared to what I was used to from my Swing experience.

After the main layouting was done, I started to write some Java code in FXMLDocumentController.java. Accessing UI components is simply done by annotating the according fields. The fields themselves are injected automatically. A nice decoupling of view and controller. Also I was able to compare two very different layouts by just switching one single line (the one referencing the fxml file) without further refactoring.

Time for some user tests! Honestly, I was a bit afraid what others would say. But I wanted to make a NICE UI, so I asked my wife and friends on Facebook and Google+ for feedback. And this decision turned out to be damn right! I got very valuable feedback about colors and layout. I definately recommend asking users – and value their feedback!

Afterwards I implemented the logic for reading the CSV files including filling the model classes. These model classes were then rendered into the graphs. So I also had some test data and continued styling the graph lines via CSS. At this stage I remembered the talks of Gerrit Grunwald. Gerrit is pretty experienced in JavaFX on the Pi and mentioned in some talks that animations on the Pi can be really really slow in the beginning until everything gets hot spot compiled. – I can definitely confirm this! The trick was to simply set animated="false" on the line chart that would only be animated at the beginning (which I didn’t really need or want anyways).

Show the UI on the TV

Time for a first test on the TV! At this point, remember the last part from “Install Java8” above: We do not need an X-server as JavaFX on the Pi paints directly to the framebuffer! So just compile and build the Jar, copy it over to the Pi and start it with java -jar ./ Weatherstation-1.0-SNAPSHOT.jar.

I made the experience that my regular monitor is not the same as the display of the TV. So I spent some time with adjusting css font sizes until I was content with the way everything was displayed on the TV. It feels a bit like webdesign and testing with different browsers – just a bit less painful.

Connecting the sensors

With the gui being done so far, the next step had to be done. When the program is started, archived data is read from files. Current data should be fetched directly from the remote sensors. This was actually plain easy. With the API provided by tinkerforge, everything I had to do was starting a thread off the Event dispatching thread that polled new sensor values periodically and pushed the data to the models. the models then simply updated their UI elements.

Remote control Java

The key strokes should be used to toggle through the values of humidity, temperature, ambient light and air pressure.

Everything that was left now was the remote control part. Unfortunately, libcec is a C-binary only. So I had the choices to try some JNI and talk to the lib directly or to start a Java Thread wrapping a process that just called the cec-client program:
/usr/local/bin/cec-client -d 8 -t prta -o Wetter. The d parameter defines the verbosity of the output, t defines the type of device the cec-client simulates (recorder, …), and o finally defines the string that is shown in the On Screen Display of the TV when brwosing through the HDMI input devices. When a cec-client listens with this command, just switch the TV to the according “device” and press some buttons on the remote control. The key presses are shown on the console.

The console output again is parsed directly from the Java process and calls the appropriate methods in the controller. This works brilliant in my case. Yet PulseEight do not recommend using cec-client in a production environment as it is intended for tests only. But well – I didn’t want to invest even more time into coding a perfect cec-java-bridge. Especially as my C-knowledge is very basic. So chances are high that my implementation would be far from “production ready”, too.

WE ARE DONE

That’s it! And of course, you can fork the complete project on GitHub!

Part 1 – Part 2 – Part 3

How to make Html Links in Android Text View work

The task itself is easy: You have a TextView which should show a clickable link to open a WebPage. As ususal, there are several ways to achieve the goal. The nasty thing is: if you mix them, they might no longer work. And also, some sometimes work, sometimes they don’t.

As it took me a little while to figure my final settings, I’ll note them here. A StackOverflow Post was a hint into a very good direction.

First, display the links as clickable links in the textview:

textView.setText(Html.fromHtml("Here is a <a href=\"http://www.Locked.de\">link</a>"));

Now there are two options:
Either make the link clickable programmatically by adding the following code:

textView.setMovementMethod(android.text.method.LinkMovementMethod.getInstance());

or add android:autoLink="web" to the according XML-element:

<TextView
android:id="@+id/myId"
android:autoLink="web"/>

If one of the options doesn’t work, try the other one, but don’t mix them. In one of my apps there are TextViews in two different activities with very similar setup. Yet, the android:autoLink="web" solution didn’t work in both TextViews whereas the setMovementMethod works in both. In the StackOverflow Post, some users also mention that both solutions didn’t work, and that android:linksClickable="true" was a solution.

My learning was: there are multiple ways to achieve the goal, but test it on each component separately as I do not yet completely trust the setMovementMethod way.

IllegalStateException: Content has been consumed

When working with Android or (to be more general) Apache HttpComponents, one should keep in mind that it depends on the HttpResponses (Api: Apache, Android) if they can be consumed multiple times or not. And if the response is not, then better triple check your code, what you’re doing.

I experiences the (very ugly) issue during the apparently simple task of reading the content of the response. )For ease of simplicity, I also use Apache commons-io, to read buffers etc.)

HttpResponse response = ...;
String content = IOUtils.toString(response.getEntity().entity.getContent());

Okay, how many possible Nullpointer Exceptions do you see? I didn’t care until I experienced the first one, so I made the code NPE safe:

HttpEntity entity = response.getEntity();
// entity can be null according to the docs!
if (entity != null) { 
    // The interface doesn't say that getContent() MUST return non-null
    InputStream stream = entity.getContent(); 
        if (stream != null) {
            tempContent = IOUtils.toString(entity.getContent());
        }
}

And suddenly I was faced with IllegalStateException: Content has been consumed. As I also did changes somewhere else, I assumed the error in some toString()-Methods that would read the content of the response during debugging. But as the error also showed up without debugging, I had a closer look to my “improvement”.

Well, the error was the call IOUtils.toString(entity.getContent()); which tried to re-aquire the Input Stream. But as I just aquired it two lines above for the null check, the content was already marked as consumed. So the (now hopefully) correct and robust code is:

HttpEntity entity = response.getEntity();
// entity can be null according to the docs!
if (entity != null) { 
    // The interface doesn't say that getContent() MUST return non-null
    InputStream stream = entity.getContent(); 
        if (stream != null) {
            tempContent = IOUtils.toString(<strong>stream</strong>);
        }
}

And the moral of the story

Be very carefull when reading HttpResponses! Also avoid pretty-printing the content in toString() – this might suddenly also consume your content. And good luck finding the point where you consume the content in such cases.

But .. why?! Please avoid returning null!

Yet I still wonder, why so many methods are allowed to return null instead of just an empty stream or something. All the Null-checks don’t make the code more readable. Some programmers might even be tempted to simply put an catch(NullPointerException e) around the part of 

response.getEntity().entity.getContent()

. Nothing I’d really recommend but I could also understand if I’d see something in other code.

Android HTTP POST authentication error with Basic Auth

In one of my Android apps, a user should be able to push JSON data to the server. Of course this should only be allowed if the user is authorized by his credentials. For simplicity, I decided to use Basic Auth via HTTPS.

As I usually do not deal with Connectsions directly, I prefer to use the Apache Http classes to make life easier. So I used code like the following:

HttpParams httpParams = new BasicHttpParams();
HttpProtocolParams.setContentCharset(httpParams, HTTP.UTF_8);
HttpProtocolParams.setHttpElementCharset(httpParams, HTTP.UTF_8);
HttpConnectionParams.setConnectionTimeout(httpParams, 5000);
client = new DefaultHttpClient(httpParams);
client.getCredentialsProvider().setCredentials(
    new AuthScope("localhost",80),
    new UsernamePasswordCredentials(username, password)
);

This was fine and worked in my local test environment and also in the live stage with GET requests. But when I started to POST data to the server I always got a “401 Unauthorized”. First I thought It’d be malformated JSON or encoding issues – but even with an empty string as payload I simply didn’t get through. Also the “same” POST request from the commandline using CURL worked like a charm! So it was very likely to be an issue with my handling of the Apache HTTP Lib. Stackoverflow also showed various people with the same problem. The proposed solution that came up several times was to manually set up the auth header. Not really the solution that I thought would be the greatest one.

After some more StackOverflow Posts and API crawling I found the very simple solution:

client.getCredentialsProvider().setCredentials(
   new AuthScope(AuthScope.ANY),
   new UsernamePasswordCredentials(username, password)

And yes, it was that easy! Just set the Auth scope correctly.

I’m just very surprised that I only encountered issues with the POST request. The very same configuration worked with a GET without problems. This was actually the point that made me sure that this part of the code was correct.

RaspberryPi Weatherstation – The Sensor Station (Part II)

The second post of this series describes the RasperryPi that is connected to the sensors and used to record the data.

Part 1 – Part 2 – Part 3

Hardware

As described in the first part, I decided to use sensors from Tinkerforge as they are comparatively cheap, require no soldering, no electrical know how and they also provide a convenient API to get the data from the sensors. Ideal for people like me who want to spend more time with using the data instead of playing around with wires, resistors, etc.

The required hardware was just.

Raspberry Pi - Sensor Station

Continue reading RaspberryPi Weatherstation – The Sensor Station (Part II)

My first RaspberryPi project: TV-Controlled Weatherstation (Part I)

Part 1 – Part 2 – Part 3

History

For quite some time I’ve been playing with the thought about bying a Raspberry Pi. Yet the biggest issue was: I had no use case for it!

I always liked information about the weather – especiallly I wanted to know in the morning how cold it was at night etc. So the use case was found: a weather station! Yet I was pretty distracted from all examples that I found on the web as I wanted a visually appealing UI so that it is fun to look at the data. Usually the samples just controlled a small LED display to display some values but – honestly – I don’t need a display at the weatherstation directly – I mean – shouldn’t it be somewhere outside and not in the room?

Continue reading My first RaspberryPi project: TV-Controlled Weatherstation (Part I)

Tomcat MongoDB Authentication Realm

Tomcat MongoDB Authentication Realm

Tomcat provides a nice and powerful mechanism for authentication by the so called Tomcat Authentication Realms.
In the Tomcat Docs, Realms are explained as:

A Realm is a “database” of usernames and passwords that identify valid users of a web application (or set of web applications), plus an enumeration of the list of roles associated with each valid user. You can think of roles as similar to groups in Unix-like operating systems, because access to specific web application resources is granted to all users possessing a particular role (rather than enumerating the list of associated usernames). A particular user can have any number of roles associated with their username.

Continue reading Tomcat MongoDB Authentication Realm

A basic Java WebApp example project supporting an xml free Jersey REST config

RESTful Webservices are an essential and convenient part in current web applications when Information should be exchanged. Additionally the main exchange formats in current webservices are either JSON or XML.

So when we want to implement the serverside of a web application using Java and especially REST services, we might end up serializing and deserializing objects to and from XML and JSON. Honestly not a very desirable task. But fortunately there is the Jersey project which can help us in this task.

Continue reading A basic Java WebApp example project supporting an xml free Jersey REST config

“Easy going” vs “Taking care”

Freitag Abend. Heimfahrt in der BOB (BayerischeOberlandBahn). Wochenende here I come!

In Bad Tölz (eine Station vor meinem “Zielflughafen”) höre ich “Hausham? Da sind sind im falschen Zug.” (HĂ€tte man vor einer halben Std in Holzkirchen umsteigen mĂŒssen). Ich denke mir “Arme Sau – jetzt fĂ€hrst erst mal wieder zurĂŒck – oder du kennst hier jemanden der dich fĂ€hrt – Ist mir aber wurscht – denn ICH bin gleich daheim. HA-HA!”.

Der Zug steht noch – ein junges asiatisches MĂ€dchen das gerade ins Teenageralter eingetreten sein muss lĂ€uft etwas hektisch herum und spricht mich dann an “schuldi-gung – Hausham?”
Ohoh. “Hm tut mir leid, falscher Zug, steig schnell um in den Zug am anderen Gleis. Du musst zurĂŒckfahren.” Verzweifelter Blick ihrerseits. “ik ver-stehe nicht?”
Oh verdammt – kurzer Versuch in Englisch – nada. Noch maximum 2-3min bis der andere Zug fĂ€hrt. Oh – scheiße – das bekomme ich jetzt nie erklĂ€rt. Die Hoffnung stirbt zuletzt: Noch ein Versuch es ihr zu erklĂ€ren. Auch die Taxioption versteht sie natĂŒrlich nicht.

Option A) Schultern zucken, mich nicht weiter kĂŒmmern, das MĂ€del bis nach Lenggries fahren lassen (wo sie nicht weiter von Hausham weg sein könnte) und gleich daheim sein (also ich – sie sicher nicht). Ist ja schließlich nicht mein Kind.
Option B) Zusammenpacken, das MĂ€del in den anderen Zug setzen und hoffen, dass mich irgendwer fahren kann (oder dass diesmal wenigstens Taxis da sind) weil MEIN Zug dann ganz sicher weg ist.

Continue reading “Easy going” vs “Taking care”