Friday, September 7, 2012

Clean Code by Example: Unit Tests and Mindfulness

In this posting I going to do a bit of a code review. I am trying to apply the guidelines given in Robert C. Martin's «Clean Code» to spot problems in a little sample application of mine, and then try to fix them.

Only recently I applied for a job at a major financial service provider that has its headquarters in the heart of Berlin. It is a real pearl, and I am in high hopes still to get hired. (Update 09/18/2012: On the evening of the very same day I posted this article, I received a phone call that I would not get the job—bummer!) Part of their rather thorough procedure was a little programming challenge. The assignment was to compute the overall score for a bowling game. In an afterward interview, the chief architect of the department went through the code with me. He pointed out spots that could be improved, and in many of them I agree.

The developers of that financial service provider follow Robert C. Martin's book «Clean Code» as a coding guideline. I did not know it at the time, and so, I guess, it is high time I read it now. In this code review I will follow both the comments of the chief architect and the principles of «Uncle Bob»—which may overlap. Also, I will try to mend those parts which I do not like about my code—there are a few.

The program code that went with my job application you can find on Github. For those not familiar with the scoring rules of ten pin bowling, Wikipedia has a sum-up.

The chief architect focused on the following issues. Maybe there were more, but these I remember:
  1. Missing dependencies.
  2. Test-cases are too large.
  3. Test-cases contain redundant code.
  4. Something is fishy about the exceptions thrown. (Posted on 09/24/2012)
  5. There are too much comments.
  6. There are too few comments.
  7. Some methods or to long,
  8. Some methods are hard to understand.
I ordered them by increasing complexity and difficulty to fix. Some of them are related, like the second or fourth, or the last four items. I will try to recollect the point the chief architect made about them. I also will rationalize—reason I mean—why I programmed them that way in the fist place. In this blog posting I will only cover the first three items, covering the other in later articles.

Missing dependencies

This is not really a code issue, but it is an issue nonetheless. The chief architect complained that my solution depended on external libraries (JUnit here), but neither did I package them with my solution nor did I provide a way to easily satisfy the dependency, like a Maven project. The odd fact is, first thing I did, when I had my solution ready, was to put it in a Maven project. Before that, Eclipse did all the magic for me. But I was familiarizing myself with Maven at the time, so it was the natural thing to do.

I deliberately did not send the Maven project along with the source code, and that is the real problem. It is a problem specifically if you are in for an employment, mind you. Because in such a situation you might want to consider, what the employing party is expecting of you. Also you certainly do not want to annoy them in a task they are actually doing for you, like evaluating your working samples. I failed in both aspects, because I neither minded the small but avoidable trouble the missing libraries would cause. Nor did I reason, that a professional Java programmer would be familiar with the prevalent build tool of his field. My line of thinking went more in the direction of avoiding a zip-archive that contained considerably more directories than files, which was the first thing I disliked about Maven in the beginning (but got fairly quick used to).

There nothing to fix here—anymore. So let's go on to the real stuff.

Test-cases are too large and are full of redundant code

Have a look at BowlingGameTest.java. It contains a few short test-cases but also three test-cases that are between 40 and 70 lines long, the largest containing 27 assertions!

When I was asked, why they were so long, I started blabbering something about Python doc-tests and telling stories with a test-case, and non-sense like that. I was taken off my guard; then trying to make up some reasons on the spot because I could not remember or reconstruct my original line of thinking. A better reaction in an job interview would be to either admit that you see the point, or alternatively be honest and say that it seemed reasonable then, but you cannot recollect, why, now.

In truth, I was too lazy to split the test-cases into separated entities. This would have meant to come up with meaningful names. And that would have taken time. Instead I grouped related tests under one name.

The problem with such bloated test-cases is, that you are never sure how many tests are actually failing. Because with JUnit test-cases it is all or nothing. As soon as the first assertion of a test-case is violated the whole test-case will be aborted. You will never now, whether the following assertions would hold, or not. This is not a problem if the assertion are strongly related and test a single aspect of the program. In an aggregated test-case this is rarely the case. Look at this:

@Test
public void gameProgress() throws BowlingException {
  BowlingGame game = new BowlingGame();

  for (int i = 0; i < 9; i++) {
    assertEquals(i + 1, game.getCurrentFrame());
    assertEquals(1, game.getCurrentRoll());
    game.addRoll(2);
    assertTrue(!game.isComplete());

    assertEquals(2, game.getCurrentRoll());
    game.addRoll(3);
    assertTrue(!game.isComplete());
  }

  assertEquals(10, game.getCurrentFrame());
  assertEquals(1, game.getCurrentRoll());
  game.addRoll(4);
  assertTrue(!game.isComplete());

  assertEquals(2, game.getCurrentRoll());
  game.addRoll(5);
  assertTrue(game.isComplete());

  // 10th frame is spare
  game = new BowlingGame();
  for (int i = 0; i < 9; i++) {
    game.addRoll(2);
    game.addRoll(3);
  }
  assertEquals(10, game.getCurrentFrame());
  …
}

…and so on for forty odd lines. There is, of cause, the problem of poor intelligibility. What does the first for-loop up to the comment actually test for? But I will go into this matter later. Clearly, the loop after the comment tests a completely independent aspect, as it creates a whole new BowlingGame object. And thanks to the comment the reader might guess what it actually does. Without it, the reader has to figure out the meaning of the test for himself. But that's the job of good, succinct method name, right?

Let's see what Uncle Bob has to say about the matter. Chapter 9 of «Clean Code» is dedicated to unit testing. Two rules apply for certain: minimize the number asserts per test (he is not advertising strict one-assert-per-test discipline), and a single concept per test. Fewer assertions make «those tests come to a single conclusion that is quick and easy to understand» (page 130). The single concept rule states basically what I wrote in the last paragraph.

So let's split the huge methods up, starting with «invalidRolls()», which originally looked like this:

@Test
public void invalidRolls() {
  BowlingGame game;
  boolean caught;

  game = new BowlingGame();
  caught = false;
  try {
    game.addRoll(-1);
  } catch (BowlingException x) {
    caught = true;
  }
  if (!caught)
    fail("A bowling exception should have been thrown.");

  game = new BowlingGame();
  caught = false;
  try {
    game.addRoll(11);
  } catch (BowlingException x) {
    caught = true;
  }
  if (!caught)
    fail("A bowling exception should have been thrown.");

  game = new BowlingGame();
  caught = false;
  try {
    game.addRoll(2);
    game.addRoll(9);
  } catch (BowlingException x) {
    caught = true;
  }
  if (!caught)
    fail("A bowling exception should have been thrown.");

  game = new BowlingGame();
  caught = false;
  try {
    game.addRoll(2);
    game.addRoll(-2);
  } catch (BowlingException x) {
    caught = true;
  }
  if (!caught)
    fail("A bowling exception should have been thrown.");
}

This one shows with in-your-face directness that laziness and ugliness go hand in hand. Each individual test-case of the four aggregated ones expects an BowlingException to be thrown. JUnit test can express this intend explicitly by annotating the test-case with «@Test(expected=BowlingException.class)». The aggregate test-case cannot make use of this because the first thrown exception would successfully terminate the test-case leaving all the following code un-executed. Refactoring the four kinds of invalid rolls into separate methods will not only make their nature more obvious, it will make the test-case a great deal shorter and more to the point, as the whole exception-catching and flagging construct disappears.

I started to extract the methods using the equally named refactoring tool of the Eclipse IDE, but so much code had become unnecessary that I just did one test-case and copied it three times. The result is a stunning improvement:

@Test(expected=BowlingException.class)
public void invalidRollBelowZero() throws BowlingException {
  BowlingGame game = new BowlingGame();
  game.addRoll(-1);
}

@Test(expected=BowlingException.class)
public void invalidRollAboveTen()  throws BowlingException {
  BowlingGame game = new BowlingGame();
  game.addRoll(11);
}

@Test(expected=BowlingException.class)
public void invalidSecondRollBelowZero() throws BowlingException {
  BowlingGame game = new BowlingGame();
  game.addRoll(2);
  game.addRoll(-2);
}

@Test(expected=BowlingException.class)
public void invalidFrameAboveTen()  throws BowlingException {
  BowlingGame game = new BowlingGame();
  game.addRoll(2);
  game.addRoll(9);
}

Next up is «gameInProgress()», part of which you already saw above. Again the code is highly redundant. First, nine frames are rolled in a loop without spare or strike. The last frame is special, as a spare or a strike allows for extra rolls to compute the bonuses. A strike, a spare, or none of the former make it three test-cases in total. Splitting by «extract method» is straight forward, yielding three almost identical tests, that look like the following:

@Test
public void tenthFrameIsStrike() throws BowlingException {
  BowlingGame game;
  game = new BowlingGame();
  for (int i = 0; i < 9; i++) {
    game.addRoll(2);
    game.addRoll(3);
  }

  assertEquals(10, game.getCurrentFrame());
  assertEquals(1, game.getCurrentRoll());
  game.addRoll(10);
  assertTrue(!game.isComplete());

  assertEquals(10, game.getCurrentFrame());
  assertEquals(2, game.getCurrentRoll());
  game.addRoll(10);
  assertTrue(!game.isComplete());

  assertEquals(10, game.getCurrentFrame());
  assertEquals(3, game.getCurrentRoll());
  game.addRoll(10);
  assertTrue(game.isComplete());
}

One of them differs a bit as it also makes some assertions during the rolling of the first nine frames. The latter test-cases do not do this as the executed code would be exactly the same. Clearly, the former test-case violates the «one concept per test-case» rule, as it test first the correct filling of the first nine frames, and only then its advertised concept, the rolling of the final frame. Resolving this puts us in a tight spot between minimizing redundancy and having one concept per test-case, because these are the options I see:
  1. Make the rolling of the first nine frames a separate test-case. That would solve the one-concept-per-test-case-issue. Still, rolling nine frames is required before testing the rolling of the tenth frame. Which would indicate to…
  2. isolate the rolling of the first nine frames as an utility operation. This utility cannot be a test-case itself. This would violate the I of the F.I.R.S.T. virtues of unit-testing that «Clean Code» declares: Fast, Independent, Repeatable, Self-Validating, and Timely (pp. 132). Independent means, no test-case must depend on another. This again implies that we have a test-case and a utility operation, both having virtually the same code. Which throws us into redundancy hell. The third option would be to…
  3. add some assertions to the utility operation, call it from a test-case that covers the rolling of the first nine-frame, and also use it to prepare the testing of the tenth frame. The price to pay for this is the overhead introduced by implicitly testing the rolling of the first nine frames over and over again.
But what is life without ambivalence? Hence, I went for option three as the lesser of the evils, and ended up with code like this:

protected BowlingGame rollNineFrames() throws BowlingException {
  BowlingGame game = new BowlingGame();
  for (int i = 0; i < 9; i++) {
    assertEquals(i + 1, game.getCurrentFrame());
    assertEquals(1, game.getCurrentRoll());
    game.addRoll(2);
    assertTrue(!game.isComplete());

    assertEquals(2, game.getCurrentRoll());
    game.addRoll(3);
    assertTrue(!game.isComplete());
  }
  assertEquals(10, game.getCurrentFrame());
  assertEquals(1, game.getCurrentRoll());
  return game;
}

@Test
public void firstNineFramesNoStrikeNoSpare() throws BowlingException {
  rollNineFrames();
}

@Test
public void tenthFrameIsStrike() throws BowlingException {
  BowlingGame game = rollNineFrames();

  game.addRoll(10);
  assertTrue(!game.isComplete());

  assertEquals(10, game.getCurrentFrame());
  assertEquals(2, game.getCurrentRoll());
  game.addRoll(10);
  assertTrue(!game.isComplete());

  assertEquals(10, game.getCurrentFrame());
  assertEquals(3, game.getCurrentRoll());
  game.addRoll(10);
  assertTrue(game.isComplete());
}

Now, should the tests that cover the rolling of the final frame be split up further? Uncle Bob advises so, because they still cover more than one concept. The first is the way the final frame is rolled: without clearing all pin, with a strike, or with a spare. The latter two continue the game. The second concept is the correct handling of the bonus rolls.

But is the splitting mandatory? I would argue, it is! But only if following a strike, spare, or none of the former, there would be more than one test-relevant way to roll the bonus balls. As a matter of fact, I realized there is at least one: after a spare in the last frame, a following «strike» should not allow for another bonus roll.

Dutifully I extracted three more utility operations for rolling ten frames: with the last frame being either a spare, a strike, or none of them.

Again the utility methods make assertions. That bothers me for the following reason. In their current form, the utilities test whether the game is complete, which frame is rolled and which roll within a frame is due. Now suppose, we decided that we would like to check the game score as well. Computing the game score is the most expensive operation of the application, and now it would be called again and again, most of the time testing the very same aspect of the application. Admittedly, the scoring function is not really expensive in computation complexity terms, in fact its complexity is O(1), i.e. constant time. But suppose it was not cheap, what then? We would violate the F of the F.I.R.S.T. virtues: fastness. I see no generally viable way out of this dilemma, so I let it rest for now.

Only, two test-cases are left to go. First I split «strikesAndSpares()» in two, right at the «And». This test-case checks the valid flagging of frames as strike or spares. Then I realized there is a third concept tested: neither spare nor strike was rolled. That made it three test-cases, that look like this one:

@Test
public void frameIsSpare() throws BowlingException {
  BowlingGame game = new BowlingGame();

  game.addRoll(5);
  game.addRoll(5);
  assertTrue(game.isSpare(1));
  assertTrue(!game.isStrike(1));
}

The last test-case «score()» feeds a list of rolls into a game. Then it checks where the score is computed correctly:

@Test
public void score() throws BowlingException {
  BowlingGame game = new BowlingGame(new int[] { 10, 10, 10, 10, 10, 10,
      10, 10, 10, 10, 10, 10 });
  assertEquals(300, game.getScore());

  game = new BowlingGame(new int[] { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
      0, 0, 0, 0, 0, 0, 0, 0 });
  assertEquals(0, game.getScore());

  game = new BowlingGame(new int[] { 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2,
      3, 4, 5, 1, 2, 3, 4, 5 });
  assertEquals(60, game.getScore());
  …

Eleven combinations of rolls are tested altogether. Again, multiple, though admittedly related, concepts are merged into one test-case. Again, comments are required to signify the role of the rolls. Again, the first failing assert will veil the outcome of the following. Nurse, scissors!

@Test
public void scoreAllZeroLastFrameStrikeExtraRollsStrikes()
    throws BowlingException {
  BowlingGame game = new BowlingGame(new int[] { 0, 0, 0, 0, 0, 0, 0, 0,
      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 10, 10 });
  assertEquals(30, game.getScore());
}

@Test
public void midGameScoreInterruptedStrikeStreak() throws BowlingException {
  BowlingGame game = new BowlingGame(new int[] { 10, 10, 10, 5 });
  assertEquals(75, game.getScore());

The resulting eleven test-cases are crisp and short, but not without flaw. The names are awful! They all start with either «score» or «midGameScore». Clearly this is grouping prefix. Classes lend themselves much better for grouping, and since the BowlingGameTest has «grown» from five to twenty eight methods, it seems sensible to break the class up.

I created two classes «CompleteGameScoringTests» and «IncompleteGameScoringTests» and moved the methods by drag-and-drop in the Eclipse package explorer. The method names are much more readable now:

public class IncompleteGameScoringTest {

  @Test
  public void interruptedStrikeStreak() throws BowlingException {
    BowlingGame game = new BowlingGame(new int[] { 10, 10, 10, 5 });
    assertEquals(75, game.getScore());
  }
  …

Done! It is quite possible than further splitting of the BowlingGameTest class would be wise, but I defer this until later, when I know more about «Clean Code.»

Conclusion

Applying the rules of «Clean Code» to the test-cases of even such a simple piece of code as mine, already greatly improved its readability—and yes, its beauty. But there is further room for improvement. Because Robert C. Martin states that the most important thing about automated testing is that the tests are first-class citizen, just like the application code is, and must be kept clean by the same high standards (page 124). That's why I will frequently revisit my tests during the course of cleaning up my Bowling example.

Tuesday, August 21, 2012

Modelio – a free and open-source UML modeling tool


It has been a while since I last fired up a UML CASE-tool. I hardly can remember, which one I used back then. Was it Poseidon or MagicDraw? Most of the free and open-source tools sucked back then. I was quite surprised—not pleasantly—when I saw how little matters progressed. Well, at least I found one product, which did not completely frustrate within the first fifteen minutes.

After some rather annoying experiences with (free) Eclipse UML modeling tools, like Papyrus and Umlet, I stumbled over «Modelio». To be fair, Umlet is a nice tool also, and I really like how rapid one can enter properties to a class. I failed however to create an association class, and that was that. Papyrus was a complete usability disaster. It took me about fifteen minutes to just figure out how to add a property to a class and how to name it. And even when I got it, it took a dozen or so clicks to add simple attribute to class.

Modelio, on the other hand, is rather convenient to use and produces nice looking diagrams. Properties cannot be added as rapidly as with Umlet, but the procedure is still acceptable. It provides the modeling elements I need at the moment, and that contents me.

Maybe you ask yourself, how can all of a sudden a free and open-source project blossom a quality CASE tool? Well simple, it used to be a commercial product. But now it goes the «freemium» way. The base product is free (and open-source) but more professional features, like round-trip engineering for C++, C# and (commercial) Java projects must be bought.

Unlike other freeium products or so called «community editions», Modelio appears to to be uncripled for open source Java development, including round-tip engineering, though I have not yet given it a try. It even has Hibernate support, which I will test-drive presently.

I like Modelio's business model because it gives open-source developers a professional tool at hand, while getting financed by commercial software producers, who usually can afford the license fee. Thumbs up!

Saturday, August 11, 2012

Creating and filling MySQL database tables

In a previous posting I summarized the steps required to start your own private instance of the MySQL daemon. Hibernate needs a few things more than just the bare DBMS instance, however. You will need a database, at least one table in it, and a user with permission to access that table. The user is not strictly needed. The user «root» has full permission to do anything. And this is precisely the reason why to create a dedicated user.

All the stuff following is not fit for a production environment, due to the lax security precautions.

Creating a database and and user

Creating a database is very simple. Connect to your daemon:
mysql -u root mysql
This will only work if the Unix socket or IP-address and port are specified in either a configuration file or by a environment variable. Then, from within the mysql client, create a database,
create database simple;
and a user:
create user 'hibernate'@'localhost';
My user «hibernate» may only connect from the local host. He has no password set, though. I am not going to change that, as it is more convenient during development.

Then grant him full permission on the database you created earlier:
grant all privileges on simple.* to 'hibernate'@'localhost';
The MySQL manual explains the whole topic on creating and confining users in greater detail.

Filling the database

Initializing a database is a cumbersome and error-prone task. It usually involves sending quite a few SQL commands to the DBMS instance. All the repeatedly needed stuff better goes into a plain text file for re-use. By convention one with the file extension «sql». I have one sql file that creates the tables. It is named «simple_db_create_tables.sql» and looks something like this:
drop table if exists songs;
drop table if exists albums;

create table albums (
  id integer primary key auto_increment,
  title varchar(20),
  publisher varchar(20)
);

create table songs (
  id integer primary key auto_increment,
  title varchar(40),
  artist varchar(20),
  performer varchar(20),
  album integer,

  foreign key (album) references albums (id)
);
Yes, it is about music, as my Android example. Another sql script loads sample data from a text file. It utilizes the «load data» statement for that purpose. My sample data is arranged as list of comma-separated values, one file per table to be filled, e.g. in this example one for albums and one for songs. The album data file contains this:
Contains sample values for the "simple" database table "albums".
Values separated by comma. First two lines ignored.
1, Distraction Pieces, Speech Development
2, A Canticle for Leibowitz, "Blackstone Audio, Inc."
The «load data» statement is quite flexible as to the structure of the input data. My CSV files are loaded by statements like this:
load data local infile 'simple_db_albums.csv'
      replace
      into table simple.albums
      fields terminated by ','
      ignore 2 lines;
Did you wonder why the data file above explicitly sets the «id» field for each row? Should not «auto_increment» accomplish that? Well, it would. Just leave the first field empty and it kicks in. But «id» is a foreign key used by the table «songs». That means each song must give the exact id of its album. Furthermore, subsequent loading of the CSV-file would happily insert row after row of the same two albums into the database, each having a different id.

All scripts can be executed with the «mysql» command-line tool, e.g.:
mysql -u hibernate simple < simple_db_create_tables.sql
The tutorial chapter of the MySQL documentation explains creating and filling tables much more thoroughly.

Eclipse SQL Explorer

All the things above—and much more—can be accomplished conveniently from within Eclipse with the help of the Eclipse SQL Explorer extension. Installing the JDBC-driver for MySQL (or any other DBMS) is—while not overly complicated–also not exactly trivial. Mark E. DeYoung tells you what needs to be done.

SQL Explorer sucks however when it comes to executing «load data» statements, though not entirely to its own fault. The file-name given to that statement can be either an absolute or a relative path. In the former case the behavior of «load data» is pretty obvious. In the latter case however, the path is interpreted relative to the directory where the executing command—the mysql client or in this case Eclipse—has been started. With the mysql client you would simply change directory to where the data file resides, but as for Eclipse this is not within you power. Most likely Eclipse had been started with your home directory as current working directory, and this is where «load data» would resolve the relative path from. Setting the path of «load data» relative to your home directory will most likely not be viable because anyone else sharing the sql script would have to have the data file at the exact same relative path within their home directories too. For the same  reason absolute paths fail altogether.
In consequence, it means that for loading data the «mysql» command is still the best.

With tables created and filled, one can proceed to finally accessing them from a Java program. But not in this posting.

Updates

08/14/2012, 21:00 UTC Changed «grant all privileges on simple…» to «grant all privileges on simple.*…» Otherwise MySQL would interpret «simple» as a table name in the currently selected database, which would most likely be the «mysql» database.
Also, my MySQL user is no longer called «eclipse» because it is mostly not Eclipse accessing my database. Chose «hibernate» instead.

Friday, August 10, 2012

Showing Delicious booksmarks in a Blogger widget

I just added a widget to this blog which shows my recent Delicious bookmarks. My first thought was to look for a dedicated Delicious widget, but there seems to be none in Blogger's widget library. Google search lists a few rather dated do-it-yourself articles. They all stem from a time before the «new» Delicious. That is why I did not even attempt them. It is much easier to show the bookmarks with the off-the-shelf RSS-widget.

There was as slight problem however. I mark all the bookmarks relating to the topic of this blog with a «The Convalescent Coder» tag. I could not see a direct way to show the URL for just this restricted RSS-feed. The obvious appoach seemed to be to take the browsing URL which was «http://www.delicious.com/jhunovis/the-convalescent-coder» and insert «v2/rss» just before the user-name. Surprisingly this produced an empty feed—but only for tags that contain spaces. Some unsuccessful web-searches later, I simply tried to replace the dashes by escaped spaces, and—it worked. For instance, the RSS feed for my bookmarks tagged «The Convalescent Coder» is «http://www.delicious.com/jhunovis/the%20convalescent%20coder».

Thursday, August 9, 2012

Running a private mysqld instance and keeping the databases in your home directory

I needed a DBMS running for some experiments with Hibernate. I wanted to use the
packages delivered by Fedora but did not want to run it as a system service,
because this usually requires special user permission; in the worst case root
permission. Running the DBMS as a system service has further
disadvantages. Programmers are of the lazy folk and tend to run the service at
boot time. It saves a few keystroke. It also wastes computing resources, opens
potential security holes into your computer, and slows down the start-up time of
the system. As a further detriment this approach fails on systems where you
do not have administrator permissions. You know, the ones at work. So, it is best
to run it as local user from the start.

I chose MySQL for my DBMS for no other reason that I have used it before
somewhere in far past. Installing it on Fedora is simple:

su -c "yum install mysql-server"

The MySQL manual contains detailed installation instructions for all major
distributions
.

Then copy the default MySQL configuration file at /etc/my.cnf into your home
directory as ~/.my.cnf and adapt it. Mine is looking like this:

[mysqld]
datadir=/home/jhunovis/tmp/mysqld
socket=run/mysqld.sock

[mysqld_safe]
log-error=log/error.log
pid-file=run/mysqld.pid

It overrides the settings of the global configuration file. The «datadir» is the
location where MySQL stores its run-time data, like the actual databases, but
also log-files and the Unix socket. Note that all the file options are relative
to «datadir». Unfortunately, the tilde is not accepted for referring to a home
directory.

Update 08/09/12, 21:30 UTC: I just realized, that the command «mysqlshow» will show the directories «log» and «run» as databases, which is awkward, if not disturbing. I then decided to place all the non-database files into a sibling directory of the datadir. End update.

Update 08/10/12, 13:15 UTC: When I was filling some tables with sample data I
saw that my database used the character set «latin1» instead of
«utf8». Furthermore it used * Swedish* for collating, which was just weird (I am
far from Sweden). Adding the line «character_set_server=utf8» to the
configuration file fixed this. You can check your character settings by issuing:

mysqladmin variables | grep -E 'character|collation'

End update.

You will have to create those directories prior starting the MySQL
daemon:

mkdir -p ~/tmp/mysqld/{run,log}

Starting mysqld at this point will fail because the system databases are still
missing. Create them by issuing on the command line:

mysql_install_db

Now start the service:

mysqld_safe

Then you can connect to your database:

mysql -u root -S ~/tmp/mysqld/run/mysqld.sock

You have to pass the socket to the MySQL client to which it should connect to.
Alternatively, you may add a section like the following to your ~/.my.conf that
saves you the trouble:

[mysql]
socket=/home/jhunovis/tmp/mysqld/run/mysqld.sock

You will have to repeat this for every MySQL command you
use. Better point the shell environment variable «MYSQL_UNIX_PORT» to that
socket file and be done with it once and for all:

export MYSQL_UNIX_PORT=~/tmp/mysqld/run/mysqld.sock

Your private MySQL service instance is now ready. You may now test it, adjust
permissions
, and create a database for your own purposes.

Update 08/10/12, 13:15 UTC: Changed title from «Starting a private mysqld instance» to «Running a private mysqld instance and the keeping databases in your home directory».

Thursday, July 5, 2012

Never start or shutdown a service in android.test.ServiceTestCase.setUp() or tearDown()

Setting up your service setUp() or stopping it in tearDown() will break your test-case. Most probably a NullPointerException will be thrown somewhere. Both errors happen independently, but have in essence the same cause: the internal state keeping of ServiceTestCase.

The class Service does not keep track of the service life-cycle state, i.e. whether the service has been created, started, or is about to be destroyed. That is why the ServiceTestCase defines a number of state variables for tracking the state: mServiceAttached, mServiceCreated, mServiceStarted, and mServiceBound. The problem lies with mServiceCreated. Let's check it out in detail.

Calling startService() in setUp()

Starting the service in setUp() might cause an error during testServiceTestCaseSetUpProperly()—and for that matter, during any other of your own tests.

In my case it was only testServiceTestCaseSetUpProperly() that failed with an error. It is the sole test-case defined by ServiceTestCase. Its purpose is to detect improperly configured tests. I does only one thing: calling setupService().

My service was still very rudimentary, resembling the local service example from the API docs of class Service. The test failed during the tearDown(). Per default, tearDown() shuts down the service. This includes invoking onDestroy() on the service. There occurred a NullPointerException. The implementation of my onDestroy() method is as follows:

@Override
public void onDestroy() {
    // Cancel the persistent notification.
    mNM.cancel(NOTIFICATION);
    // Tell the user we stopped.
    Toast.makeText(this, R.string.music_player_stopped, Toast.LENGTH_SHORT).show();
}

Surprisingly the variable mNM (of class NotificationManager) was null, though it was initialized properly in onCreate():

@Override
public void onCreate() {
    mNM = (NotificationManager)getSystemService(NOTIFICATION_SERVICE);
    // Display a notification about us starting.  We put an icon in the status bar.
    showNotification();
}

So, when did it went null again? Well, never! Due to calling startService(…) in setUp() it just never had been initialized! It was initialized once, all right. And that was during startService(…). But right after that, testServiceTestCaseSetUpProperly() called setupService(), which will create a new service instance, but will not start it—i.e. onCreate() will not be invoked upon it. So my precious—and initialized—service instance was silently exchanged against an uninitialized one. That alone would not be fatal, were not the state variable ServiceTestCase.mServiceCreated still set to true, thus signaling that onCreate() had been invoked on the service. And because of that, tearDown() will happily invoke onDestroy() upon a service instance whose onCreate() method never had been called. Smells a bit like a bug to me…

Calling shutdownService() in tearDown()

In the case of the shutdown the cause of the error is easy to pinpoint: ServiceTestCase.tearDown() also shuts down the service—and the API docs clearly say so. It is understandable that a second attempt will fail, finding its underlying service gone. Or is it? The error happens in ServiceTestCase.shutdownService(), in line 10 of the following listing:

protected void shutdownService() {
    if (mServiceStarted) {
        mService.stopSelf();
        mServiceStarted = false;
    } else if (mServiceBound) {
        mService.onUnbind(mServiceIntent);
        mServiceBound = false;
    }
    if (mServiceCreated) {
        mService.onDestroy();
    } 
}

The call to ServiceTestCase.tearDown() has just executed this very same code once and afterwards nullified the service instance. However, mServiceCreated is (again) still true, although the service is already destroyed. In fact, this status variable is never re-set to false anywhere in ServiceTestCase. Thus onDestoy() will be invoked on a null reference.

Conclusion

The moral is, never call startService(…) in setUp() and never shutdownService() in tearDown(). It would be nice however, if the official documentation would impress a little more upon that. Also, a little less unforgiving implementation seems possible. Basically that would mean to just re-set mServiceCreated to false in two or so places.

Tuesday, June 12, 2012

Backlight control causes hard reboot on Macbook Pro

Since installing Fedora on my aged first-generation Macbook Pro I was plagued by intermittent reboots. They mostly occurred during the boot process, while logging into KDE, and—less often— when the system was up and running. No need to mention this was a disaster and it pretty much cured me from using a file system that deems a fsck-tool of no great import.

At least for the latter crash-type, I was able to pin-point the culprit: it was the black-light control. Whenever KDE dimmed the panel there was a solid chance the system would reboot—no kernel panic, no stopping. Not even kdump would break its fall. Sure enough, disabling dimming would cure that.

Display dimming can be disabled in KDE's «Engery Saving Settings»

Now I am waiting for the next crash during reboot before blacklisting the «apple_bl» kernel driver. But maybe the Linux kernel 3.4 that ships with Fedora 17 has mended this issue already. Lets wait and see.