lefed
08 Dec 2013 / posted by Nikku

Recently I was looking at a project and wanted to capture a number of code metrics to see how the project developed over time. It turns out some interesting metrics can easily be extracted with basic shell scripting.

I wanted to figure out, whether the project grew or shrinked over time and how that relates to test code and the number of test cases.

A few metrics that I was interested in were

  • source code size (lines of code and files)
  • test code size (lines of code and files)
  • number of tests

What I came up with is number of bash snippets that collected the information form me.

Bash for the win

Given I wanted to scan the current directory . for source files matching *.java we may implement the metrics as follows:

Number of Files

find . -type f -name "*.java" | wc -l

Lines of Code (LOC)

find . -type f -name "*.java" \
  -exec cat {} \; | sed '/^\s*$/d' | wc -l

This excludes empty lines (sed '/^\s*$/d').

Number of Tests

find . -type f -name "*.java" \
  -exec cat {} \; | grep @Test | wc -l

It scans for unit tests (in my case indentified by a @Test annotation).

Over Time Metrics

Based on the three metrics, over time data an be collected by browsing through the projects versioning history. I combined the above snippets in a small script that checks out a number of commits to collect the metrics of a project over a series of commits.

It logs commit_hash, src_files, test_files, src_loc, test_loc and num_tests as comma separated values into a specified file. To use it, collect all commits you would like to analyse via git rev-list first_commit..last_commit first.

This results in a CSV file with all information in place, ready to be visualized.

Summary

As it turned out, shell scripting is an appropriate environment do do quick and dirty extraction of source code metrics. The performance is well enough for a nightly job, too. On my developer machine for around 90k lines of code project, the scanning takes aproximately five seconds per commit.

Happy analyzing!


13 Jun 2013 / posted by Nikku

So there is an open source project on GitHub. And you forked it already. And there is this bug XYZ you would like to fix, throught a pull request. This post is a short checklist on how to do it.

Use Feature Branches

Get ready to work with feature branches. Those isolate the work you do for a particicular pull request from all the other things you want to contribute.

Create a branch for the issue XYZ where you will implement the bug fix.

git checkout master        # starting point is your master
git checkout -b XYZ        # create and checkout branch XYZ
                           # you are going to work on

Do non-stop coding. All day all night. Mind coding styles of the project you are working on. Use the right tools.

At some point you are done.

git log                    # check what you have done

Cleanup using git rebase

You may have produced several commits. And there may be typos in your commits. Or you commited try to fix bug (1..n). Squash (i.e. combine) all of your commits and give it a nice commit message. This makes it easier for reviewers to see the actual changes and to eventually merge the pull request.

Your tool for the job is interactive rebase. It allows you to reword, remove, combine and alter commits.

git rebase -i HEAD~n       # n=number of commits from HEAD
                             # you created

This pops up an interactive console

pick 906eb32 yea, nearly done
pick 8a45bb9 ok, late night trying again
pick b86deaf got ya

Combine all of your commits into one by assigning s or squash as the rebase operation.

pick 906eb32 yea, nearly done
s 8a45bb9 ok, late night trying again
s b86deaf got ya

Finish the job, i.e. press [ESC], [:wq] to write the changes and close the editor.

You will be asked to write a commit message that describes everything you did. Make it a good one (check your projects contribution guidelines, too).

Again, finish the job with [ESC], [:wq].

You are done and your changes are ready to be published!

Push, Pull Request, restart

Push the stuff to your remote:

git push origin XYZ

If you have pushed parts of the work before, you may need to override these changes by applying force (-f):

git push -f origin XYZ      # overrides history on the server
                            # branch so take care what you do

Create a pull request from your origin/XYZ branch to the OSS projects development branch.

Switch back to your master branch

git checkout master

And create a new feature branch for this AABB issue you wanted to fix for such a long time already.

git checkout -b AABB

There is more

There exist many more resources worth checking out on the topic: