coshsh

Tags:

Coshsh - A Config Generator for Naemon/Shinken/Nagios/Icinga/Prometheus/Whatever

Coshsh

How to pronounce it?

ʦɔʃ:

What is coshsh?

Coshsh is a framework that assists in the automatic creation of configuration files.

Features

  • Coshsh is extremely fast. (depending on the cmdb performance ~100000 services in 10s)
  • Coshsh can be easily extended. Coshsh reads only hosts and applications. Creation of services is very flexible.

Who uses it?

Lidl, Kaufland, Bühler, City of Munich and several others generate their monitoring configuration files with Coshsh. Users usually have in common:

  • they have a very big monitoring landscape.
  • they want automation and no manual editing of configs.
  • they have a highly dynamic cmdb (hundreds of hosts added and deleted every day)
  • they want a small team.

Download

github releases
Python Package Index
The suggested and supported (if i am having a good day) way to use Coshsh is installing OMD. With OMD, batteries and everything else you need to run Coshsh is included.

Support

Support and consulting are available from ConSol

Changelog

The Changelog can be found on github

How does it work?

Coshsh reads data from various sources with the help of adapters, containing information about hosts and the applications installed on them. The host and service definitions are generated by filling in placeholders in template files. So template files can be transformed to Naemon cfg files, but also to yaml files containing Prometheus scrape definitions.

Basic concepts of Coshsh

Coshsh differs from other Nagios config tools in that users do not interactively click together individual objects in a GUI. With Coshsh, rules are created in advance and applied to the objects to be monitored, which come from so-called datasources.

Datasource

The raw data processed by coshsh comes from datasources. In most cases, there will be only one datasource per installation. This can be a set of CSV files or any data collection. Typically, it is the company’s own CMDB or another database containing hosts and applications for monitoring purposes. Each type of datasource must be read in a unique way. It is crucial that the raw data is transformed into lists of host and application objects and delivered to coshsh. To do this, a file named datasource_.py must be created, containing the necessary code. It is, so to speak, the “adapter” that connects a datasource to coshsh. Datasources are described in the coshsh configuration file as follows:

[datasource_cmdb]
type = mycmdb
hostname = dbsrv1
username = cfggen
password = secret

[datasource_extraapps]
type = csv
files = /omd/sites/gen/data

The different parameters arise because each datasource is different and is opened and read in different ways.

Class

In the datasource adapter, rows are read from database tables or files. These represent hosts and their associated applications. To further process them, they need to be converted into Python objects by calling the constructors Host() and Application(). For each type of application to be monitored, a class must be defined that inherits from the parent class Application.

import coshsh
from coshsh.application import Application
from coshsh.templaterule import TemplateRule
from coshsh.util import compare_attr


def __mi_ident__(params={}):
    if compare_attr("type", params, ".*applxy.*"):
        return XyApp


class XyApp(coshsh.application.Application):
    template_rules = [
        TemplateRule(needsattr=None,
            template="app_xy_default"),
    ]

Now when a datasource reads a record from a cmdb, which represents the application XY installed on host ABC, then the constructor Application() is called with a dictionary like { ‘host_name’: ‘ABC’, ’name’: ’the app xy #1’, type: ‘vendor123 applxy vers0815’}. Because the method _\mi_ident__ of the class XyApp matches the record and returns XyApp, the call Application() automatically returns an object of class XyApp

Template

The whole generation with coshsh is about creating configuration files for Nagios (or Shinken or Icinga or…). Each application is monitored with a specific set of services. These are thematically grouped in so-called tpl files. These are templates for the future configuration files, containing placeholders. The template_rules in the class definitions determine which tpl file(s) will contain the future services for a certain type of application. In the paired curly brackets, the attributes of the respective application object are referenced. At this point, the real value (coming from the datasource) will be placed.

{{ application|service("app_xy_default_check_alive") }}
  host_name                       {{ application.host_name }}
  use                             app_xy_default
  check_command                   check_xy!60
}


define service {
  service_description             app_xy_default_check_users
  host_name                       {{ application.host_name }}
  use                             app_xy_default
  max_check_attempts              5
  check_command                   check_xy_users!10!20
}
Recipe

The heart of coshsh is a recipe. Analogous to a cooking recipe, it consists of ingredients. It describes what ingredients are needed to create a Nagios configuration:

  • Datasource(s) - hosts and the applications installed on them are read from these. Contact and detail information also come from here. It is possible to mix datasources, e.g., hosts and applications from a CMDB and details (filesystems, tablespaces, interfaces, etc.) from an Excel sheet.
  • Classes directory - this is where the Python files containing the application classes are located which act as a bridge to the templates.
  • Templates directory - this is where the templates for the future configuration files are located.
  • Objects directory - the configuration files rendered from the templates for every application/host object are written to this directory.
Cookbook

A cookbook is a configuration file containing the recipes.

Examples

Datasource
from coshsh.host import Host
from coshsh.datasource import Datasource
from coshsh.application import Application
from coshsh.util import compare_attr

logger = logging.getLogger('coshsh')

def __ds_ident__(params={}):
    if coshsh.util.compare_attr("type", params, "mycmdb"):
        return MyCMDB

class MyCMDB(coshsh.datasource.Datasource):
    def __init__(self, **kwargs):
        # kwargs wird mit den Werten aus dem DATASOURCE-Abschnitt
        # des Cookbooks belegt
        self.name = kwargs["name"]
        self.password = kwargs["password"]
        ...

    def open(self, filter=None, objects={}, **kwargs):
        # optional
        # kann verwendet werden, um eine DB aufzumachen

    def close(self, filter=None, objects={}, **kwargs):
        # und am Schluss wieder zuzumachen
        # auch optional

    def read(self, filter=None, objects={}, **kwargs):
        logger.info('read items from datasource')
        self.objects = objects
        hostdata = {
            'host_name': 'test_host_0',
            'address': '127.0.0.9',
            'type': 'test',
            'os': 'Red Hat 6.3',
            'hardware': 'Vmware',
            'virtual': 'vs',
            'notification_period': '7x24',
            'location': 'esxsrv10',
            'department': 'test',
        }
        self.add('hosts', MyHost(hostdata))
        appdata = {
            'name': 'os',
            'type': 'Red Hat',
            'component': '',
            'version': '6.3',
            'patchlevel': '',
            'host_name': 'test_host_0',
            'check_period': '7x24',
        }
        self.add('applications', coshsh.application.Application(appdata))
        ....

Slides

Now is a opportune moment to learn German. Once you’ve acquired the language, return here, click on “fullscreen” and enjoy the slides.