Tag

devops

Agile, Continuous Delivery

Why do DevOps?

Background:

According to the Puppet’s 2016 State of DevOps Report DevOps is no longer a mere fad or buzz word but an understood set of practices and cultural patterns.

DevOps was originally coined by Patrick Debois and Andrew Shafer in 2008. The velocity conference  community made DevOps a known term with the famous 10+ deploys per day: Dev and Ops cooperation at Flickr in 2009.

What is DevOps?:

Daniel Greene in techcrunch defines DevOps as a set of practices, tool and policies that lead to improved quality and automated delivery. But I really like Gene Kim’s definition in his book the phoenix project where he refers to DevOps as the outcome of applying Lean principles to the IT value stream. These principles are age old but are now used to accelerate flow of work in software development through product management, development, test, operations and Information Security.  Adapting lean principles and shift left, Toyota’s mantra is to stop and fix quality issues before they manifest at the end and also deliver just in time parts and support.

ProductionSystem_tcm-11-406919

Source: Toyota website

Imagine applying these principles to a software development lifecycle and while coding and running automated testing, checking for known vulnerabilities, testing for security, continuously integrating code and deploying several times a day in various environments including production. This is shifting left the activities that were performed at the very end, which caused problems to be discovered late and teams could never build in quality. This lead to developers spending less time reworking their code while Ops spending time in late nights and weekends to deploy code into production. In fact according to the phoenix project, DevOps has benefited from an astounding convergence of philosophical movements such as Lean Startup, Innovation Culture, Toyota Kata, Rugged Computing and the Velocity community.

What do the development and operations typically see?:

What Operations sees? What development sees?
Fragile applications are prone to failure More urgent, date-driven projects put into the queue
Long time required to figure out “which bit got flipped” Even more fragile code put into production
Detective control is a salesperson More releases have increasingly “turbulent installs”
Too much time required to restore service Release cycles lengthen to amortize “cost of deployments”
Too much firefighting and unplanned work Failing bigger deployments more difficult to diagnose
Planned project work cannot complete  Most senior and constrained IT ops resources have less time to fix underlying process problems
 Frustrated customers leave  Ever increasing backlog of infrastructure projects that could fix root cause and reduce costs
 Market share goes down  Ever increasing amount of tension between IT Ops and Development

 

Source: 2011 06 15 velocity conf from visible ops to dev ops final

The above leads to a mindset that eventually breeds mistrust and a not so healthy competition between these teams.

Lets see how DevOps changes this:

Looking at some statistics of companies showed high performers delivered on faster time to market, increased customer satisfaction and market share, team and employee productivity and happiness as well as allowing organizations to win in the marketplace.

Amazon in 2011 deployed code every 11.6 seconds, Netflix several times a day and Google several time a week.  Gary Gruver’s description of their transformation of HP’s laserjet division where 400 people distributed across 300 continents is able to integrate around 150 changes or 100,000 lines of code  into the trunk on their 10m lines of code base every day.

BenefitsofDevOps

We know our quality within 24 hours of any fix going into the system and we can broadly test for even last minute fixes to ensure that a bug fix does not cause unexpected failures

Check out Guy Gruver’s talk in 2013 on how they did it.

Summary:

DevOps transformation at scale is hard and needs changes in both organization and technology bringing together teams that have been in the past operated separately and introduces innovation that automates end to end software development pipeline.

Continuous Delivery

5 tips on developing your first Atlassian server plugin

We at Nimble love to develop and experiment with different technologies and share our experiences with our community. This time around developing an Atlassian plugin became our fancy. Atlassian of course offers tools for development, collaboration and continuous delivery such as Jira, Confluence, Bitbucket and Bamboo. Atlassian also offers a rich REST API to build integrations and plugins that one can add/extend their basic tool functionality.

So I started on my journey and found a few gotchas that I think are interesting if you happen to go down that path.

1. The basics

a. Sign up for an account at http://developers.atlassian.com. Atlassian offers two plugin development SDKs for their cloud based and their in-house server based versions.

b. Consider signing up for an account at bitbucket.org if you don’t already have a SCM or if you want to try out a nice cloud based code collaboration tool that provides Git repos to host your code and also offers integration with several tool suites including a cloud based continuous integration and delivery tool. More on this later…

2. Issues with the tutorial

I started exploring the server side development and accessed the tutorial at https://developer.atlassian.com/docs/getting-started/set-up-the-atlassian-plugin-sdk-and-build-a-project. The tutorial does a great job at the setup and gets you going, but it is dated.

Most of my time really went into trouble shooting issues with dependencies in pom.xml and conflicting directives in the plugin config file: atlassian-plugin.xml

Following a few sub links in the tutorial:

You are directed to convert the plugin component to a servlet model at https://developer.atlassian.com/docs/getting-started/learn-the-development-platform-by-example/convert-component-to-servlet-module by adding the following lines in the atlassian-plugin.xml

<component key="myPluginComponent" class="com.atlassian.plugins.tutorial.refapp.MyPluginComponent" public="true">
 <interface>com.atlassian.plugins.tutorial.refapp.MyPluginServlet</interface>
</component>
<servlet name="adminUI" class="com.atlassian.plugins.tutorial.refapp.MyPluginServlet" key="test">
 <url-pattern>/test</url-pattern>
</servlet>

Next there are SAL (Secure Access Layer) class components that need to be imported into the project. Look at https://developer.atlassian.com/docs/getting-started/learn-the-development-platform-by-example/control-access-with-sal which directs you to add the following lines to your atlassian-plugin.xml

<component-import key="templateRenderer" interface="com.atlassian.templaterenderer.TemplateRenderer" filter=""/>
<component-import key="userManager" interface="com.atlassian.sal.api.user.UserManager" filter=""/>
<component-import key="loginUriProvider" interface="com.atlassian.sal.api.auth.LoginUriProvider" filter=""/>
<component-import key="pluginSettingsFactory" interface="com.atlassian.sal.api.pluginsettings.PluginSettingsFactory" filter=""/>

Compiling this code promptly delivers an error messsage while using the latest version of Atlassian spring scanner and JDK 1.8.  The error message is that servlet directive and the component import tags are not compatible

After a lot of time troubleshooting, I hit upon the reason. The latest version of spring scanner supports annotations instead of tags. Take a look at the revamp suggested at https://bitbucket.org/atlassian/atlassian-spring-scanner

The Atlassian-plugin.xml file should not contain any of the <component-import> tags  and in your servlet code add annotations such as the below

@ComponentImport
private final UserManager userManager;
	
@ComponentImport
private final LoginUriProvider loginUriProvider;
	
@ComponentImport
private final TemplateRenderer templateRenderer;
	
@ComponentImport
private final PluginSettingsFactory pluginSettingsFactory;

This needs to be updated in the tutorial.

3. Use a RefApp

Instead of developing for a particular tool like jira or confluence, developing for RefApp gives you the flexibility of completing your plugin development irrespective of the tool and then generating a plugin version for the target tool. RefApp provides the shared framework between all Atlassian applications. This means that you can develop your plugin without accidentally relying on dependencies or features specific to one application, or encountering an application-specific bug later on. Developing a plugin with RefApp eliminates guesswork about the functionality of your project. You can rest assured that since all Atlassian applications share at least the framework present in RefApp, your plugin will work as expected.

 4. Leverage bitbucket pipeline

One of the other advantages of hosting your code at bitbucket.org is that you can use Bitbucket pipeline. Bitbucket pipleline is a new offering and still in beta. I happen to be part of the beta program and got an opportunity to kick the tires. In short the tool is a continuous integration and continuous delivery tool that relies on the presence of a bitbucket-pipelines.yml file in your top level of your project. The yml file with the below directives helped me build my plugin on the cloud. The pipeline relies on running builds, tests and deployment with pre-configured docker images. Anytime you update your code, the pipleline runs based on the directives as in the below yml file.

# You can use a Docker image from Docker Hub or your own container
# registry for your build environment.

image: translucent/atlassian-plugin-sdk

pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - cd adminUI
          - pwd
          - atlas-version
          - atlas-mvn clean install

 

5. Develop an Atlassian plugin like its 2016

Some interesting ideas to explore once you have the basic tutorial up and running is how to take your plugin to the next level with ideas on how data should be retrieved only using the REST API and UI should be rendered on the client site by adding javascript libraries such as Babel and Webpack

Take a look at the details at the following two write ups….

https://developer.atlassian.com/blog/2016/06/jira-add-on-dev-2016-part-1/

https://developer.atlassian.com/blog/2016/06/jira-add-on-dev-2016-part-2/

You can find my latest code and use to kick start your plugin development at https://github.com/aniljaising/atlassian-plugin-tutorial

I would love to get your feedback on this article as well feel free to ask any questions…