Developer activity metrics Part 1: Gitinspector

Measuring developer productivity is one topic destined to be a cause of heated discussions. Leaving all sentiments aside, here’s a short list of tools with ambition to provide analytical analysis to the developer activity.

Gitinspector – free and down-to-earth statistical tool for Git repositories
Hello2morrow – builds code dependencies graphs and identifies software architecture rules violations
Semmle – a commercial tool with main focus on ability to view results of static code analysis under different angles
BlueOptima -Developer Efficiency Analyzer – a commercial tool that based on the company’s algorithm is able to translate number of committed lines into hours of developer effort

What we can measure with Gitinspector:

1. The size of a pull request

Historical commit information by author actually has an
interesting twist:

   (insertions + deletions) / number of commits 
                         = the size of each commit

This is an objective indicator of cries of pain you might’ve let escape your chest while facing overwhelming pull requests.

2. Efficiency of the change

Number of rows from each author that have survived and are still intact in the current revision Generate the report for a time frame of a sprint or a release development cycle, and the tool will display a number of survived lines for each author. So, if during the sprint, you add or modify 100 lines, but only 10 of them survive at the end of the sprint, it’s time to focus on quality and pertinence of
your work.

3. Cadence of the change

Ignore most of the History timeline info and focus on Modified Rows for each month:

Modified Rows / Total Code Base Rows = % of monthly change
% of monthly change * Coefficient of the ave test time = 
estimated time you need to test your changes

Yes, this is the approximate metric, but given a few months on the project, you’ll be able to determine the coefficient and hence estimate the testing time

4. Areas of expertise

Is mostly responsible for section can be easily called Authors’ areas of expertise – we always tend to spend more time coding what we like to code, or we end up becoming experts in what we have to code. So, file extensions indicate language preferences, and file locations show level of comfort with the code base – after all, making a change to the framework core carries more responsibility than implementing a new handler or a view.

Screen Shot 2016-05-08 at 3.24.03 PM

May 8, 2016 | by An Nadein | 2 Comments

Continuous Delivery

Look at Bamboo with Jenkins fan’s eyes

For the past 10 years, my projects’?Continuous Integration were done on Hudson/Jenkins. Cannot say I’d ever been a Jenkins fan though, but I haven’t had a chance to work?with?alternatives either. Until I stumbled over Atlassian Bamboo.

At the end of the day, both?Bamboo and Jenkins are Continuous Integration (CI) tools,?but while Jenkins is doing its best to support Continuous Delivery (CD) via newly developed plugins, Bamboo is built with CD in?its backbone.?

?Native support for Continuous Delivery?Pipeline

Bamboo CD pipeline is a chain of stages. Each stage contains 1 ore more jobs. Each job consists of one or more tasks.

For example, a project continuous delivery pipeline consist of the following stages: package, deploy to QA, run tests in QA, deploy to UAT, smoke tests in UAT.

“Package” stage contains one “Build and Package” job that has “compile”, “unit test”, and “install into Artifactory” tasks.

“Run Tests in QA” stage contains 2 jobs that are executed in parallel: “Run Regression tests” and “Run Functional tests?.

All jobs in one stage have to be completed successfully before moving to the next stage.

Steps can be configured to pause for manual input.

As a part of a pipeline, a release candidate can be manually approved and promoted to higher environments. For example, a build that successfully passed all stages including a smoke test in UAT can be marked as approved by UAT team and promoted to Prod by Operate team. Bamboo groups permissions would ensure correct access to these functions.

Many daily CI features are put on auto-pilot:

Choose one of two merge models to?avoid manual merging?? to trigger a feature build, Bamboo will use either the ?Branch Updater? to update a feature branch with the integration branch or the ?Gatekeeper? to update the integration with the feature branch.
Enable ?push on success? to merge a pull request to the feature branch via Branch Updater or to the integration branch via the Gatekeeper.
Configure Bamboo to automatically delete configurations for branches that are deleted in Stash

Tight integration with JIRA:

Generate Release Notes for each build and shows a difference between current and any previous builds.
Link issues to dev branches to see real-time build status inside the issue
Get related build results inside issues and the issue’s vital stats on the build result in Bamboo.
Track each issue’s deployment status throughout your environments.
See a roll-up of all the JIRA?issues and commits included in each release candidate.
Create an issue from Bamboo failed job to resolve it

Tight support of Docker build agents and tasks:

Build a Docker image, run a Docker container, and push the image to Docker Registry
Run as a Bamboo Agent inside a Docker container to ease up distribution of changes to Bamboo instances and avoid conflicts between remote agents on the same host

November 1, 2015 | by An Nadein | 1 Comment

Continuous Delivery

A month with Git or life after SVN

? Our’s is a typical old-fashioned large-scaled?project with a bloated codebase that reached 1,000?lines of code in about 3 years and keeps moving with a velocity of 300 lines a day. Yes, we do have design and code quality issues, but that?s the topic for another day.

One sunny weekend? we moved to Git with GitFlow workflow? covered in details by revered “A successful Git branching model and heavily supported by Atlassian

The first dust had settled in a month. We still couldn?t offer ?The? answer to life, the universe and everything? better than a 42, but we surely learned a few things about life in Git era.

?Two heads are not necessarily better than one….there is always someone to disagree with you.

Mistake #1: Pick a single?Git client to begin with.

We gave the team an option to choose from one of 3 Git clients – TortoiseGit, Eclipse EGit, and command line client. And we’ve run into a classic example of “The Paradox of Choice – Why More Is Less“.

Every time someone was stuck, they had to either find a buddy from their preaching camp or revise their tool allegiance.? That is, if there was someone around to help… If no one who knew ?how to? was available, the guys would attempt to find a solution on their own and with any tool ending up joggling between all three Git clients. You can imagine the slope of the learning curve, the level of productivity degradation, and the volume of pain cries.

Majority of folk chose TortoiseGit. It is a breather to get up and running with and it feels like home for those who come from using TortoiseSVN. And that was the very disaster – people felt they’d got a hang of Git without understanding the fundamental differences. Lion’s share of bad merges, lost files, and delayed pushes came from this group.

The curious ones stuck to Eclipse EGit. It can do a lot of things a command line client can. One limitation that we found out quite early was EGit could not prune dead branches, but we could live with that for a while. The real problem was ? the tool was very very buggy. We had to watch numerous tutorials on Youtube in order to find workarounds to merge files, resolve conflicts, reset branches, and much more.

The old school used the?command line client – the all-purpose tool until it came to conflicts resolution where a graphical client was much needed. At that point, it was a split of schools again: Eclipse, TortoiseGit Diff, or any other stand-alone Git compare tool.

?It may disturb you. It scares the willies out of me.

Mistake #2: Do not try to stop people from making mistakes.

Barely overcoming the ancient fear of cutting a branch, we couldn?t bring ourselves up to easily exposing off them. ?What if we need to come back in a week to fix a feature bug?? or ?My bugfix hasn?t passed QA yet!? were cried out a lot too often. Yes, we did study theory and conduct hands-on sessions, but obviously they were largely useless. What worked was to step back, let them keep original branches for as long as they like and then watch them cut a new branch when a bug in that feature had been discovered or the fix had failed QA.

?Not entirely unlike

Mistake #3: Don’t count on a tool upgrade to upgrade mindsets

The habit of isolating one?s changes from the rest of us by delaying syncing with SVN trunk till the ?you?ve got to complete this now? time caused us to update our local code bases once in a week or two. The pain of conflicts resolution was significant and regressions were introduced. Yes, there are ways to monitor and discourage such behavior, but once again ? we did have serious continuous delivery challenges. Indeed, many of us had accepted the fact that to avoid that pain we had to sync our code bases daily. Feature branches at first glance looked like a fallback to ?isolated development?. The point of daily syncing features with the develop branch and bugfixes with the release branch had to be hammered into each coding brain.

?If you want to survive out here, you’ve got to know where your towel is.

So to get ?The answer to life, the universe and everything?, we still have a few mountains to climb:

We don?t see how we can ease up the burden of merging bugfix branches into both release and develop branches. In theory – yes, we push bugfix into release, and after enabling the auto-merge the release will be pushed to develop. But by then, our develop is way ahead of release and the conflicts are the matter-of-fact.

Another unanswered question is competing pull requests. Say, I created a perfectly valid pull request, but my peers haven?t reviewed it on time and someone else?s request was merged before mine causing mine become a conflicting request. The next thing I know is that after I had moved onto another super urgent task, I have to stop my current work, switch to the old feature branch, withdraw the old request, merge conflicts, and repeat the push request again.

No questions about Git?s benefits, but porting a fast-paced large-scoped and organically grown project is definitely not a task for the faint-hearted.

August 15, 2015 | by An Nadein | 1 Comment