Harsh J

Alright, so you’ve got here; you’ve got to 2010; its been roughly five years since you bought that fascinating brick. Now what?

If you’re here and you don’t have a Nokia 6681, continue your journey across the web. This post is strictly for people who’ve been messed up in their minds about buying a newer phone, cause everyone else seems to be getting one of those shiny new S60s or Androids.
Read the rest of this entry »

The default plugin offered by Hadoop for Eclipse, which allows browsing the DFS and running jobs among other things, does not work really well with newer versions of Eclipse. The issue is that the plugin (found under HADOOP_HOME/src/contrib/eclipse-plugin) is written for a fairly old version of Eclipse (Europa) and things have changed in the newer releases (Ganymede, Galileo and Helios).

To fix it, first get the patch from MAPREDUCE-1280 (direct download here), and apply it into your Hadoop installation. The following commands describe the process for getting the plugin built:

$ cd $HADOOP_HOME
$ patch -p0 < ~/eclipse.patch
$ # The following is where my eclipse resides.
$ export ECLIPSE_HOME=/usr/share/eclipse
$ ant -Declipse.home=$ECLIPSE_HOME binary
$ cp build/contrib/eclipse-plugin/hadoop-*.jar $ECLIPSE_HOME/plugins

That’s it. The last line in the commands above had copied it to your Eclipse installation, which you can then restart and switch to Hadoop MapReduce perspective and run jobs flawlessly. Credits go to Alex Kozly for mentioning the solution here.

Working with an SCM like Git, SVN, etc is a great experience all in all. Its more fun than just the part which manages your files for you. Its almost like a huge data source in itself, which you can mine for rich data (which may be useless, or may turn out some great workflow-changing points).

Git has some excellent command line tools to query log, branch and other data. After seeing what GitX does using a plain git repository folder on Mac OS X, I was hunting for a Linux tool that does similar things with respect to branches. I came across gitg which is a fantastic UI for Git on Linux (GTK).

It looks like the following:

gitg - Rich visualizations for git workflow

Get gitg here. Or install it via your package manager and have fun pointing and laughing at your merge stupidness!

Another way to dig into the data is by looking at SCM file-level data such as Author, Domain of an author, Distribution of commits over the week, etc. There are two tools available for this, Gitstats and Gitstat. I liked the former for its ease of setup and the number of statistics shown. The latter is a fairly more advanced tool but with fewer features and requires a Database to work over a web server.

gitstats - Statistics for # of commits over a week's days (For the git project itself)

Get gitstats here and just do a ‘gitstats [cloned directory] [output directory for html files]‘ or get gitstat here and have fun setting it up. I’d recommend adding the former as a cron job based on your commit interval statistics and getting done with it.