Building a new Integration

Creating a new integration in CloudMunch is quite simple. The following sections will cover how using a declarative model and some code new integrations can be created. 

  1. Integrations Primer
  2. Add new Integration type
  3. Create Resource definition
  4. Add new Interface definition

Integrations Primer

CloudMunch has several out-of-the-box integrations supported today. These integrations cover a spectrum of the most popular open-source and commercial DevOps tools. If there are tools that are not supported out of the box, it is easy to build a new integration.  

Before diving into how to extend the CloudMunch platform by building a new integration, let's understand the key components of the platform that connect to the tools and get data.    

  1. Integration: Defines a way to connect to the source. This allows definition of security mode (basic, OAuth, API key based, ssh key based etc), It stores the url to connect to them. It allows users to connect to key chains and get relevant data for its connectivity.
  2. Resource: Defines the details needed for any insight plugin to pull information from an integration. This involves details of what entity for which insights to be provided (example a project_id in JIRA, or a job in Jenkins). Insights are always generated for a set of resources created in the application connected to an integration.
  3. Interface: Interface holds mainly two things, the key metrics that are created for this integration, and actions that can be performed on the integration once it is created. Interface provides a powerful way to configure interface actions, like support for authentication at the source and support for data transformation. 
  4. Steps/Plugins: Plugins are created to help use the information provided by a user in the template, and implement functionality to either automate a specific action for the integration in question or generated insights for the integration.
  5. Templates: Templates are a group of integrations, tasks, metrics and dashboards that serves as a blueprint for an application using the wizard.
  6. Wizard: Wizard provides a way of inserting inputs in templates to enable integration to be created with right set of data, pass the right data to each steps in the plugins to enable right information flow for the tasks to run.

Add new Integration type

The first step in adding a new integration type is to define the configuration that needs to be capture for an integration. This configuration will be used by the methods that are invoked on the integration to connect and pull data.

Identify your integration structure

To add a new integration type, first you need to identify all the fields you need to collect as part of integration. We call this as a integration definition. A sample integration definition is as below

{
  "id": "kubernetes",
  "label": "kubernetes",
  "interface": "kubernetes",
  "application_import": "no",
  "display": "yes",
  "status": "enabled",
  "documentation": {
    "description": "Kubernetes"
  },
  "registrationFields": {
    "kubeconfig": {
      "type": "fileUpload",
      "display": "yes",
      "label": "Kube config",
      "mandatory": true,
      "hint": "kubernetes configuration",
      "dependency": []
    }
  }
}

The following are the key fields for you to define integration definition

Field Field Description
id (read-only) Unique id for the integration. ID is generated when API is used to create the integration
interface Identifier of the Interface to which the integration connects to for execution of its methods
application_import Does this integration come up as part of application import. If this is marked yes, the integration shows up in the wizard for application creation lis
display Setting "yes" will cause the integration to show up in the UI. If you are still creating the integration and debugging the integration, mark it "no" 
status If the integration is enabled or disabled. If disabled, the integration cannot be used in an application either through the UI or API
registration fields Is the list of attributes that we need to capture from the user for an integration. 

Save the definition into CloudMunch using API

Once the definition is identified, call the following API to register this definition into your instance of CloudMunch

POST /definitions/integrations

POST data: 
{"data":{"id":"newone","name":"newone","label":"newone","interface":"newone",
"application_import":"no","display":"yes","status":"enabled",
"documentation":{"description":"newone"},"events_definition":{},
"registrationFields":{"kubeconfig":{"type":"fileUpload","display":"yes",
"label":"Kube config","mandatory":true,"hint":"kubernetes configuration",
"dependency":[]}}}} Expected response: { "data": { "id": "DEF2016082209112397481", "name": "newone", "label": "newone", "interface": "newone", "application_import": "no", "display": "yes", "status": "enabled", "documentation": { "description": "newone" }, "registrationFields": { "kubeconfig": { "type": "fileUpload", "display": "yes", "label": "Kube config", "mandatory": "true", "hint": "kubernetes configuration" } }, "created_date": "2016-08-22 09:11:23.63887", "created_by": "you@yourcompany.com", "updated_by": "you@yourcompany.com", "updated_date": "2016-08-22 09:11:23.63887" }, "request": { "request_id": "R2016082209112263469", "response_time": "2.32 seconds", "status": "SUCCESS" } }

Validate the added definition

To validate what has been entered try the following command and ensure that the filter is added

GET /definitions/integrations?filter={"name":"newone"}

The expected response should be:

{
    "data": [
        {
            "id": "DEF2016082209112397481",
            "name": "newone",
            "label": "newone",
            "interface": "newone",
            "application_import": "no",
            "display": "yes",
            "status": "enabled",
            "documentation": {
                "description": "newone"
            },
            "registrationFields": {
                "userconfig": {
                    "type": "text",
                    "display": "yes",
                    "label": "Configuration Name",
                    "mandatory": "true",
                    "hint": "Enter the configuration information"
                }
            },
            "created_date": "2016-08-22 09:11:23.63887",
            "created_by": "you@yourcompany.com",
            "updated_by": "you@yourcompany.com",
            "updated_date": "2016-08-22 09:11:23.63887"
        }
    ],
    "request": {
        "request_id": "R2016082209143953658",
        "response_time": "1.29 seconds",
        "status": "SUCCESS"
    }
}

Note: Logo of the integration will be defaulted to CloudMunch at the time of writing and no API will be provided to upload a logo for custom integrations.

Add a new resource type

Resource types are objects that hold all information needed to pull metrics from the integration added. Ex: A resource could be a project name and details for a JIRA integration.

Design the structure for your resource

Here is a sample definition of a resource for Kubernetes. Identify all information you would need and also all the graphics you want the end user to see as part of the resource details, in this structure.

{
  "id": "kubernetes",
  "type": "kubernetes",
  "tags":[ "kube" ],
  "label": "Kubernetes",
  "category": [ "Deployment" ],
  "component": "Kubernetes integration",
  "description": "Choose if you'd like insights about your Kubernetes configuration",
  "carouselText": "These are the insight cards you will be seeing for Kubernetes",
  "carouselItems":[
    {
        "image":"images/resource_types/kubernetes/AllResourceCount.png",
        "caption": "",
        "first":true
    },
    {
        "image":"images/resource_types/kubernetes/DeploymentsAge.png",
        "caption": ""
    },
    {
        "image":"images/resource_types/kubernetes/KubeServicesAge.png",
        "caption": ""
    }
  ]
}

The following are the key fields and their descriptions

Field Field Desctiption
id A unique id for the integration
tags This links to the template that uses this resource when the template is being used. This can be used to filter what template will show what resources
category The list of resources are categorized under this category
component to be documented
carousel_text the text that comes up for the description for this resource
carousel_items the list of images that shows up when the user selects details for this resources, and also when he is filling up the details of this resource

Note: Logo of the integration will be defaulted to CloudMunch at the time of writing and no API will be provided to upload a logo for custom integrations.

Save Resource definition 

Use similar procedure used in (1b) to push this definition. The key change will be the definition_type which will be in this case be resource_type. so the actual call to post will be as below

POST /definitions/resource_types

Add a new Interface for the integration added

The last step in building a new integration is the creating the interface definition.

Identify the key metrics and actions provided by this integration

Configure the list of key metrics that will be generated by the integration in the interface, and also the list of actions that can be run directly on the integration.

A sample structure is as below

{
  "id": "jira",
  "name": "JIRA",
  "description": "Interface file for communicating with jira",
  "actions" : {
    "test" : {
      "base_url" : "{configuration->jira_url}",
      "path" : "/rest/api/latest/search",
      "method": "GET"
    },
    "list_rapid_boards" : {
      "base_url" : "{configuration->jira_url}",
      "path" : "/rest/greenhopper/1.0/rapidview",
      "method": "GET"
    },
    "list_project_versions" : {
      "base_url" : "{configuration->jira_url}",
      "path" : "/rest/agile/1.0/board/{rapidBoardID}/version",
      "method": "GET"
    }
  },
  "key_metrics" : {
    "sprint" : {
      "id" : "sprint",
      "name" : "Active sprint",
      "status" : "ENABLED",
      "compliance" : "YES",
      "tolerance":{
        "applied":"NO",
        "upper":"",
        "lower":""
      }
    },
    "velocity" : {
      "id" : "velocity",
      "name" : "Velocity",
      "status" : "ENABLED",
      "compliance" : "YES",
      "tolerance":{
        "applied":"YES",
        "upper":"90",
        "lower":"100"
      }
    },
    "volatility" : {
      "id" : "volatility",
      "name" : "Volatility",
      "status" : "ENABLED",
      "compliance" : "YES",
      "tolerance":{
        "applied":"NO",
        "upper":"",
        "lower":""
      }
    },
    "defect_density" : {
      "id" : "defect_density",
      "name" : "Defect density",
      "status" : "ENABLED",
      "compliance" : "YES",
      "tolerance":{
        "applied":"YES",
        "upper":"1",
        "lower":"0.1"
      }
    }
  }
}

Field Field Description
id Unique identifier for the interface
name Name to the interface
actions List of actions that can be performed on the integration - these are typically the methods that would be invoked on the integration or take action
actions structure Each section contains the details of an action - base_url (Base URL of the API endpoint), path (resource path on the endpoint) and method (HTTP method to invoke on the endpoint) 
key_metrics List of key metrics created for this interface by the plugins
Key_metrics structure Each section contains the definition of a Key Metric