Archive on ConSol Monitoring

Devoxx Video: Prometheus Monitoring for Java Web Applications w/o Modifying Source Code

Tue, 07 Nov 2017 00:00:00 +0000

The Prometheus monitoring tool follows a white-box monitoring approach: Applications actively provide metrics about their internal state to the Prometheus server. In order to instrument an application with Prometheus metrics, you have to add a metrics library and call that library in the application’s source code. However, DevOps teams do not always have the option to modify the source code of the applications they are running.

At this year’s Devoxx conference, Fabian Stäber did a talk on how to instrument Java Web Applications with Prometheus metrics without modifying the application’s source code.

Simulating 3rd party services with Spring Boot and Citrus

Fri, 22 Sep 2017 00:00:00 +0000

When developing software that exchanges data with other components or services you may be confronted with the proper simulation of those foreign services during integration testing. This is because you need to connect with a foreign service that is simply not available on your local machine or in a test environment.

For unit testing purpose you can use mocks that help out to simulate proper responses. There will be times where your software is deployed to a test environment in order to perform some acceptance tests with your stakeholders before going to a final release. Usually this is also done with the customer exploring the software through manual testing. In these situations traditional service mocking is not a good option and you need a real simulator instance that receives requests and responds with proper test data.

This is exactly what the Citrus simulator project provides for you. Standalone simulation and complex request/response processing with solid validation capabilities. The Citrus simulator provides a very easy and reliable definition of inbound and outbound messages for different scenarios. Good news is that this is not only for Http REST interfaces but also for SOAP WebService, JMS, RMI, mail messaging and many more. So you can use the simulator whenever you need to integrate with another service that is simply not available on your local machine or in your test environment.

Docker Headless VNC Container 1.2.0 Released

Tue, 19 Sep 2017 10:00:00 +0200

Docker Headless VNC Container 1.2.0 has been released today. The different Docker images contains a complete VNC based, headless UI environment for testautomation like Sakuli does or simply for web browsing and temporary work in a throw-away UI container. The functionality is pretty near to a VM based image, but can be started in seconds instead of minutes. Each Docker image has therefore installed the following components:

Monitoring Plugin Language Comparison

Wed, 13 Sep 2017 12:00:00 +0200

Which programming language should we use to write monitoring check_plugins? This question rose some discussion and this post is trying to give some hints.

Monitoring-Workshop 2017

Tue, 12 Sep 2017 00:00:00 +0000

Workshop 2017

Veranstaltungsdaten

Ablauf

Vorträge

Anmeldung

Anreise / Hotel

Vorabendtreff

Abendveranstaltung

Bis zum nächsten Mal

Reunite separate git repositories

Fri, 08 Sep 2017 00:00:00 +0000

I recently had to deal with two projects that have a common origin but separated at some point in time. I now had to try to bring them back together again - basically merging the changes. Sounds like a pretty standard git merge or git rebase job.

Unfortunately the separation was done in a not so clever way. Someone cloned the original repository, checked out some branch, made some first refactoring steps, got rid of the git stuff (probably rm -rf .git) and started a new git repository with this status. Rumors are that the situation at that time was so tense that people wanted to make a clear cut - which they did in a technical way.

Quite some time later it was my task to try to get the projects together again. The only input I had was two git URLs and the above story.

NEB Modules with Go

Fri, 01 Sep 2017 14:00:00 +0200

Have you ever written a NEB (Nagios Event Broker) module? This article will explain a tool which makes this a lot easier, especially if the reason was that you are not familiar with C or C++. In this case the “Go NEB Wrapper” could come very handy and if you are new to this topic it is a good point to start with.

OMD 2.60 Labs Edition Released

Mon, 21 Aug 2017 10:00:00 +0200

OMD Labs Edition 2.60 has been released today. The OMD Labs Edition is based on the standard OMD but adds some more useful addons like Grafana and Prometheus or additional cores like Icinga 2 and Naemon. This release updates many of the shiped components and adds some interesting options when resolving update conflicts.

Automated CI/CD Build Pipeline with Jenkis in OpenShift

Mon, 17 Jul 2017 00:00:00 +0000

The GitHub repository toschneck/openshift-example-bakery-ci-pipeline contains, the sourcecode for the examples of the talk Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines at the Redhat/ConSol OpenShift-Day:

Introduction to GitLab CI with Maven

Fri, 14 Jul 2017 00:00:00 +0000

At ConSol we use GitLab as our central Git server and I am quite happy with its functionality. Lately, I have been playing around with GitLab CI with the objective of finding out if we can use it instead of Jenkins, our current CI server of choice.

Since most of our projects use Maven, I was particularly interested in setting up a simple Maven build job.

To cut a long story short, yes, I would use GitLab CI in my next project. We’ll later see why, but first I want to give a quick walkthrough of GitLab CI.

Introducing Citrus Admin Web UI

Tue, 11 Jul 2017 00:00:00 +0000

It has been a while since the last release in the Citrus universe. It took us some time to get the new Citrus release 2.7.2 ready for you. Of course we were not being lazy in that time. Besides the new Citrus 2.7.2 release we are proud to announce a new player in the Citrus team. The Citrus administration UI is a web-based user interface that helps you to manage your Citrus projects and test cases.

Often users complained about the complexity of having to learn all about Citrus and the Spring framework in particular as Citrus uses Spring for configuration and dependency injection. Especially non-developers had problems to master the learning curve for Citrus and Spring when starting to use the framework. Also people asked for a way to have a user interface for managing components and tests.

We heard you and introduced a new administration user interface for Citrus! There is a detailed Citrus Admin documentation (which is still ongoing). However I would like to outline the main features of that web UI here in a short post for you.

Prometheus Monitoring for Java Web Applications without Modifying Their Source Code

Mon, 10 Jul 2017 00:00:00 +0000

The Prometheus monitoring tool follows a white-box monitoring approach: Applications actively provide metrics about their internal state to the Prometheus server. In order to instrument an application with Prometheus metrics, you have to add a metrics library and use that library in the application’s source code. However, DevOps teams do not always have the option to modify the source code of the applications they are running.

Promagent is a Java agent using Bytecode manipulation for instrumenting Java Web applications without modifying their source code. Promagent allows you to get white-box metrics for Java Web applications even if these applications do not implement any metrics library out-of-the-box.

OMD 2.40 Labs Edition Released on Raspberry Pi

Wed, 14 Jun 2017 14:00:00 +0200

OMD Labs Edition 2.40 for the Raspberry Pi has been released today. A month and a broken SD card (excessive use of /var/swap during the builds) after the release of the x86 version it is now possible to run a full-blown monitoring system on your ARM boards. It was tested on Raspberry 2 and Raspberry 3. If you want to run OMD on one of the older models, you might experience performance problems, especially when you enable InfluxDB and Grafana.

OMD 2.40 Labs Edition Released

Wed, 17 May 2017 10:00:00 +0200

OMD Labs Edition 2.40 has been released today. The OMD Labs Edition is based on the standard OMD but adds some more useful addons like Grafana and Influxdb or additional cores like Icinga 2 and Naemon. This releases focus is on security and maintainance and removes some recently discovered CVEs in Nagios, Icinga and Naemon.

Angular, we have to talk!

Tue, 28 Mar 2017 00:00:00 +0000

About three or four years ago I had the first contact with AngularJs (obviously V1.x) and what should I say? I loved it! It perfectly added the missing piece that JQuery wasnt able to solve: Bind data to the Dom in an easy way. Since this days there where a lot of evolution in Javascriptland. A lot of new Frameworks for entire SPAs, new techniques like functional and reactive programming, a lot (!) of build systems / task manager and even the language itself developed towards an serious programming language (I know some people have a different opinion). At least with the power of Typescript (or Flow) Javascript projects doenst have to be a Pain. Angular2+ took many of the the mordern aspects to provide a good and productive developer experience to develop application in time, quality and budget. Dont get me wrong: They made a very good job! But I have personally some concerns which I want to point out in this post.

Protecting Passwords in Java Properties Files on Windows

Mon, 27 Mar 2017 00:00:00 +0000

Typical Java backend applications need to integrate with existing 3rd party services. In most cases, calls to these 3rd party services are authenticated. Frequently, Java applications are required to use login credentials for authenticated calls: A username and a password.

This scenario raises a problem: How can we store the password needed for calling the 3rd party service? We could store it in a properties file, but then everyone with access to the properties file learns the password. We could provide the password as a command line parameter or environment variable, but then everyone with access to the startup script learns the password. We could hard-code it in our application, but then everyone with access to the JAR file learns the password. We could encrypt the password using a master key, but then we have the same problem again: How to store the master key?

The common solution is to use a secure data store provided by the operating system. Our application runs on Windows Server, so we use the Windows Data Protection API (DPAPI) for protecting our secret passwords. This blog post shows how to use the DPAPI in Java applications.

Ansible limit pitfalls

Wed, 15 Mar 2017 00:00:00 +0000

Mit steigender Zahl der im Ansible-Inventory gepflegten Hosts verlängert sich die Laufzeit eines Playbooks. Ansible erkennt zwar, welche Tasks nicht ausgeführt müssen (z.B. weil bestimmte Pakete bereits installiert sind), jedoch kostet auch diese Überprüfung Zeit. Früher oder später wird man deshalb den Playbook-Parameter --limit|-l einsetzen - und sich wundern, warum Teile des Playbooks plötzlich nicht mehr funktionieren. Dieser Blogpost zeigt, in welche Probleme man laufen kann bzw. wie man sie vermeidet und löst.

Getting Started With Java 9's New HTTP Client

Tue, 14 Mar 2017 00:00:00 +0000

If you ever needed to request HTTP resources with Java, you probably came across several solutions put together from a surprising number of lines. And you probably ended up with using a third party library to achieve your goal in a reasonable manner.

Good news: besides Java 9 modules, the next JDK version comes with a brand new HTTP client, and it not only brings support for HTTP/2, but also a sleek and comprehensive API. Let’s have a closer look at the new features.

Ansible meets DokuWiki

Mon, 13 Mar 2017 00:00:00 +0000

Ansible meets DokuWiki

Dokumentation belegt in der Rangliste der beliebtesten Arbeiten eines Administrators sicher einen der hinteren Plätze. Neben der Beliebtheit der Aufgabe ist es auch mit zunehmender Anzahl der vorhandenen Systeme immer aufwändiger, die Dokumentation auf einem aktuellen Stand zu halten. Ein klassischer Fall also für Automatisierung.

Das Ziel in diesem Blog soll es sein, für jedes System eine DokuWiki Seite automatisch zu erzeugen. Weiter soll auf jeder Seite noch die Möglichkeit bestehen, individuelle Dokumentation mit einzufügen.

Prometheus und die Fritzbox

Wed, 08 Mar 2017 00:00:00 +0000

Prometheus ist ein quelloffenes Monitoring- und Alarmierungs-Werkzeug. Seine Basis bildet eine Zeitreihen-Datenbank, auf deren Daten mit einer eingebauten, sehr mächtigen Abfragesprache zugegriffen werden kann.

Prometheus verfolgt den Ansatze des sogenannten “whitebox-monitoring”. Anwendungen stellen hier entweder nativ Metriken zur Verfügung, oder alternativ macht ein “exporter” Applikations- oder Geräte-Metriken für Prometheus abfragbar.

In diesem Artikel möchte ich zeigen, wie man mit Hilfe des fritzbox_exporter und des speedtest_exporter im Zusammenspiel mit Grafana Einblicke in die Performance seines Heimnetzwerks und seines Internetanschlusses bekommen kann. Die Hardware-Basis für dieses Projekt stellt ein RaspberryPi.

Testing Logstash Configuration with Citrus and Docker

Wed, 08 Mar 2017 00:00:00 +0000

The ELK-Stack is a good option to aggregate and visualize distributed logging-data. It basically based on

Logstash for extracting and distributing the data
Kibana as visualization frontend.

The core of the most ELK applications is the Logstash configuration. A user defines here which data (inputs) is processed, how (filter) the data is processed and where it will go afterwards (outputs). Especilly this configuration contains a lot of logic which is unfortunally not easy to test. In this article I want to show you how to setup a testing environment for your Logstash configuration.

Portchannel-Monitoring mit check_nwc_health

Sun, 26 Feb 2017 00:00:00 +0000

Version 6.0 von check_nwc_health ist erschienen und hat neben Aufräumarbeiten unter der Haube ein paar neue Features zu bieten:

interface-etherstats
F5 Wide IPs
Juniper VSD Memberstatus
interface-stack-status

SNMP-Traps und OMD - Teil 1

Sat, 25 Feb 2017 00:00:00 +0000

SNMP-Traps und Nagios ist eins der Themen, um das man bislang gerne einen großen Bogen gemacht hat. Grundsätzlich gibt es seit etlichen Jahren die AddOns SNMPTT und Nagtrap, deren Konfiguration aber ein wenig mühsam ist. In einem Projekt, bei dem es um die Überwachung von mehreren Tausend Storage-Systemen ging, entstand eine Methode, welche ressourcenschonend und einfach automatisierbar ist. Im ersten Teil dieses Artikels geht es um die entsprechende Vorbereitung eines OMD-Servers. Genauer gesagt darum, wie man dafür sorgt, daß ein eingehender Trap gleichzeitig an mehrere OMD-Sites (Test, Produktion, …) zugestellt wird.

Using .gitignore the Right Way

Wed, 22 Feb 2017 00:00:00 +0000

Have you ever wondered what kind of patterns .gitignore allows? Was it **/*/target, target/* or *target*?? Read on and find out!

PNP4Nagios and Grafana

Tue, 21 Feb 2017 16:00:00 +0200

Many Nagios folks use PNP4nagios to store performance data and draw graphs. Nowadays time series databases like Influxdb are quite trendy and have their own pros and cons. But In the end they visualize the stored metrics with Grafana which comes with a plugable datasource api, so wouldn’t it be nice to use the already collected RRD data in Grafana as well? This combines the speed of rrd with the more modern graphs from Grafana.

Getting Started with Java 9 Modules

Mon, 13 Feb 2017 00:00:00 +0000

So, 2017 has arrived - this is the year when Java 9 will finally be released. And with it, the brand new module system called Jigsaw. In January, Marc Reinhold has announced that JDK 9 is feature complete, so we have every reason to be optimistic that the final release will actually ready in July. So it is about time to get acquainted with project Jigsaw, also known as Java 9 modules.

Local Kubernetes Development with Minikube

Fri, 10 Feb 2017 00:00:00 +0000

Getting started with Kubernetes can be intimidating at first. Installing Kubernetes is not the easiest of tasks and can get quite frustrating.¹ Luckily, there is an out-of-the box distribution called Minikube which makes toying around with Kubernetes a bliss.

FOSDEM Video: Implementing 'tail -f'

Tue, 07 Feb 2017 00:00:00 +0000

At this year’s FOSDEM conference I did a 20 minutes presentation on how to implement tail -f in Go. The video is available below.

Abstract: As part of a log file monitoring tool, I implemented a file tailer that keeps reading new lines from log files. This turned out to be much more challenging than I thought, especially because it should run on multiple operating systems and it should be robust against logrotate. In this 20 Minutes talk I will present the lessons learned, the pitfalls and dead-ends I ran into.

Hamcrest - More readable Tests

Tue, 07 Feb 2017 00:00:00 +0000

The probably best written tests are those which can be understood by anyone understanding some English, right?

Hamcrest is an anagram of the word “Matchers” and a paradigm of encapsulating matching logic and corresponding error messages in objects we could use and reuse in the tests. They hide “matching”-implementation details and get self explanatory names we can seamless integrate in our tests. And of course we are also able to write tests for our matchers!

Hamcrest itself isn’t only intended to be used in the context of tests. It’s available for: Java, Python, Ruby, Objective-C, PHP, Erlang, Swift.

Undertow - How to setup a HTTP server

Tue, 24 Jan 2017 00:00:00 +0000

Undertow is an open-source lightweight, flexible and performant Java server, they say. I can confirm that it’s

flexible: always feel free to provide your own implementations or use Undertow helpers to delegate usual server glue code to a more specific implementation you provide

I didn’t check or compare performance. It is the default server implementation of Wildfly Application Server and sponsored by JBoss.