Category

Agile

Agile, Continuous Delivery

Why do DevOps?

Background:

According to the Puppet’s 2016 State of DevOps Report DevOps is no longer a mere fad or buzz word but an understood set of practices and cultural patterns.

DevOps was originally coined by Patrick Debois and Andrew Shafer in 2008. The velocity conference  community made DevOps a known term with the famous 10+ deploys per day: Dev and Ops cooperation at Flickr in 2009.

What is DevOps?:

Daniel Greene in techcrunch defines DevOps as a set of practices, tool and policies that lead to improved quality and automated delivery. But I really like Gene Kim’s definition in his book the phoenix project where he refers to DevOps as the outcome of applying Lean principles to the IT value stream. These principles are age old but are now used to accelerate flow of work in software development through product management, development, test, operations and Information Security.  Adapting lean principles and shift left, Toyota’s mantra is to stop and fix quality issues before they manifest at the end and also deliver just in time parts and support.

ProductionSystem_tcm-11-406919

Source: Toyota website

Imagine applying these principles to a software development lifecycle and while coding and running automated testing, checking for known vulnerabilities, testing for security, continuously integrating code and deploying several times a day in various environments including production. This is shifting left the activities that were performed at the very end, which caused problems to be discovered late and teams could never build in quality. This lead to developers spending less time reworking their code while Ops spending time in late nights and weekends to deploy code into production. In fact according to the phoenix project, DevOps has benefited from an astounding convergence of philosophical movements such as Lean Startup, Innovation Culture, Toyota Kata, Rugged Computing and the Velocity community.

What do the development and operations typically see?:

What Operations sees? What development sees?
Fragile applications are prone to failure More urgent, date-driven projects put into the queue
Long time required to figure out “which bit got flipped” Even more fragile code put into production
Detective control is a salesperson More releases have increasingly “turbulent installs”
Too much time required to restore service Release cycles lengthen to amortize “cost of deployments”
Too much firefighting and unplanned work Failing bigger deployments more difficult to diagnose
Planned project work cannot complete  Most senior and constrained IT ops resources have less time to fix underlying process problems
 Frustrated customers leave  Ever increasing backlog of infrastructure projects that could fix root cause and reduce costs
 Market share goes down  Ever increasing amount of tension between IT Ops and Development

 

Source: 2011 06 15 velocity conf from visible ops to dev ops final

The above leads to a mindset that eventually breeds mistrust and a not so healthy competition between these teams.

Lets see how DevOps changes this:

Looking at some statistics of companies showed high performers delivered on faster time to market, increased customer satisfaction and market share, team and employee productivity and happiness as well as allowing organizations to win in the marketplace.

Amazon in 2011 deployed code every 11.6 seconds, Netflix several times a day and Google several time a week.  Gary Gruver’s description of their transformation of HP’s laserjet division where 400 people distributed across 300 continents is able to integrate around 150 changes or 100,000 lines of code  into the trunk on their 10m lines of code base every day.

BenefitsofDevOps

We know our quality within 24 hours of any fix going into the system and we can broadly test for even last minute fixes to ensure that a bug fix does not cause unexpected failures

Check out Guy Gruver’s talk in 2013 on how they did it.

Summary:

DevOps transformation at scale is hard and needs changes in both organization and technology bringing together teams that have been in the past operated separately and introduces innovation that automates end to end software development pipeline.

Agile, Continuous Delivery

Developer activity metrics Part 1: Gitinspector

Measuring developer productivity is one topic destined to be a cause of heated discussions. Leaving all sentiments aside, here’s a short list of tools with ambition to provide analytical analysis to the developer activity.

  1. Gitinspector – free and down-to-earth statistical tool for Git repositories
  2. Hello2morrow – builds code dependencies graphs and identifies software architecture rules violations
  3. Semmle – a commercial tool with main focus on ability to view results of static code analysis under different angles
  4. BlueOptima -Developer Efficiency Analyzer – a commercial tool that based on the company’s algorithm is able to translate number of committed lines into hours of developer effort

What we can measure with Gitinspector:

1. The size of a pull request

Historical commit information by author actually has an
interesting twist:

   (insertions + deletions) / number of commits 
                         = the size of each commit

This is an objective indicator of cries of pain you might’ve let escape your chest while facing overwhelming pull requests.

Screen Shot 2016-05-08 at 2.21.14 PM

2. Efficiency of the change

Number of rows from each author that have survived and are still intact in the current revision Generate the report for a time frame of a sprint or a release development cycle, and the tool will display a number of survived lines for each author. So, if during the sprint, you add or modify 100 lines, but only 10 of them survive at the end of the sprint, it’s time to focus on quality and pertinence of
your work.
Screen Shot 2016-05-08 at 3.06.28 PM

3. Cadence of the change

Ignore most of the History timeline info and focus on Modified Rows for each month:

Modified Rows / Total Code Base Rows = % of monthly change
% of monthly change * Coefficient of the ave test time = 
estimated time you need to test your changes

Yes, this is the approximate metric, but given a few months on the project, you’ll be able to determine the coefficient and hence estimate the testing time

Screen Shot 2016-05-08 at 3.06.28 PM

4. Areas of expertise

Is mostly responsible for section can be easily called Authors’ areas of expertise – we always tend to spend more time coding what we like to code, or we end up becoming experts in what we have to code. So, file extensions indicate language preferences, and file locations show level of comfort with the code base – after all, making a change to the framework core carries more responsibility than implementing a new handler or a view.

Screen Shot 2016-05-08 at 3.24.03 PM

Agile, Innovation

How do you Innovate?

IMG_5066

A former colleague of mine recently asked me how I would calculate ROI for a new product/service they are planning to build and launch in their organization.

I enthusiastically provided some measures that I had used to calculate ROI in my prior life  after successfully building and deploying a financial services adapter at a past employer.

It seems like my colleague’s team while working on the vision of a new product and were also asked to calculate the potential ROI.  A recent progress report showed that the product was still far from being viable and one of the key improvement was to conduct mini proof of concepts to let the architecture emerge.

Reading that, I was quickly reminded of the LEAN Enterprise book by Jez Humble, Joanne Molesky and Barry O’Reilly and the MVP (Minimum Viable Product) vs Product Vision post by Marty Cagan.

According to Cagan, vision and MVP are related but not the same. Vision provides the roadmap (type of service and customer) and an innovative organization should be prepared to create many MVPs as we need to get to the final product.

Building MVPs allows us to focus on the right thing to build and provides valuable information on how to evolve, adapt, and pivot to meet customer needs that are discovered through experimentation.

So how do you measure?

ROI and customer churn are lagging metrics that measure output after the fact. Scott Cook, founder of Intuit suggests to use “love metrics” after every the delivery of every MVP- how much people loved the product or how often they come back and how delighted they are in the early stages. If these metrics do not indicate high activity then its time to pivot and experiment again. Agile 101?

Would love to get your thoughts and feedback…

Agile

5 agile tools that fascinate – week ending September 5th 2015

Every weekend I search the web for new agile or devops tools that fascinate and engage me. Here are the 5 agile tools I tried this week

  • Learn from yesterday, live for today and hope for tomorrow with Sensei:

Fun and effective retrospectives using Sensei for distributed agile teams. Sensei guides teams through the entire retrospective and times team members to be prompt. Sensei is able to record all feedback and insights and turn them to actionable steps and follow up on prior commitments. Check out their product tour from slideshare

  • Alone we can do so little; together we can do so much with Symphonical: Symphonical

Symphonical uses google hangouts to create virtual scrum boards, team to do lists, meeting agenda and many more templates. Check this cool collaboration tool that makes working in distributed teams across geographies seamless. The website has a nice tour https://www.symphonical.com/tour/

 

  • If you can dream it, you can do it with Lino:

Lino is a great tool for planning with distributed teams using sticky notes. Teams can be in front of a desktop or on the go, lino enables real time planning and brainstorming. Check out their product video.

 

 

  • Dream big, start small, but most importantly start with scrumwiseScrumwise

This neat and intuitive tool can kickstart scrum for any organization in a matter of minutes. If your organization is starting its agile transformation and has not picked a tool of choice, scrumwise is a nice platform with built in support for backlogs, sprints, metrics and much more. Check out their instant demo at https://www.scrumwise.com/scrum

 

  • Plan for what is difficult while it is easy, do what is great while it is small with mingle:  mingle-logo

Mingle from thoughtworks is a great tool that guides you through scrum. It integrates with Github and the plus version has program management and planning features which is an afterthought among most tools that I have come across.