Pandora FMS server with docker-compose

Docker-compose is amazing, this tool allows you to literally deploy complex clusters of containers with one command. Previously, I had a seamless experience running ELK (Elastic, Logstash, Kibana) with docker-compose and now decided to give a try to Pandora with it.

Pandora FMS is a great tool for monitoring and securing the infrastructure since it provides insights into anomalies that may happen to your servers. And it is open source and free!

To start with, I found 3 containers required to run the Pandora Server on the Docker Hub of Pandora FMS : MySQL DB instance initialized with Pandora DB, Pandora Server, and Pandora Console.

All the deployment magic happens within each container, and my task was only to create some infrastructure and orchestration for them with help of docker-compose.

I put the result on Github – pandora-docker-compose.

Here is a quick overview of  what it does:

• Creates a dedicated network and assigns IPs to containers
• Configures Postfix for sending emails to admins (int he default container it was not working)
• Synchronizes time with Docker host
• Maps Pandora DB files to local host folder so that you can back them up and restore

 

DSC Pull Server for managing… Linux nodes

Let’s face it, Powershell DSC and Linux is definitely not the most favorite duo when speaking about CM. In this article, I will explain how to set up Linux node with existing DSC Pull Server and why this is a win-win for Win-dominated tech stack.

Powershell DSC is well-documented for Windows, you may find some information about using it in the Push mode (when the configuration is applied TO the node FROM the server via CIM session) but Linux Pull Server is rarely getting more than just a paragraph of attention. So, I’m here to fix it.

The main prerequisite of this exercise is a configured DSC Pull Server – you may want to check out this excellent MSDN tutorial. We run it on Windows Seever 2012 R2, with WMF (Windows Management Framework 5.0). The hostname for the means of this exercise is set to DSCPULLSERVER. Also, you got a Linux node – in rhis case, CentOS 7, with a hostname: DSCPULLAGENT.

The first discovery about DSC On Linux is that the client doesn’t run in Powershell for Linux. It is actually a set of Python and shell scripts that provide an interface to OMI (Open Management Infrastructure) CIM Server and control an execution of pulled configurations. Therefore, it works in a slightly different way comparing to “native” DSC.

To start with, we would need to obtain both OMI and DSC rpm’s in accordance with an $ openssl version installed – it might be either 0.98 (then  use rpms with ssl_98 in the name) or >=1.00 (use ssl_100 rpms).

rpm -Uvh https://github.com/Microsoft/PowerShell-DSC-for-Linux/releases/download/v1.1.1-294/dsc-1.1.1-294.ssl_100.x64.rpm
rpm -Uvh omi-1.1.0.ssl_100.x64.rpm

After installing the packages, DSC files can be found under /opt/microsoft/dsc/.

The next step is to prepare a DSC Local Configuration manager file. It tells DSC client where to pull the configuration from and which configuration. To generate it (and all kinds of other .mof files), I use a separate Windows machine.

Here is an example PS DSC script that creates a local configuration mof.

[DSCLocalConfigurationManager()]
configuration LinuxPullAgent
{
Node localhost
Settings {
RefreshMode = 'Pull'
ConfigurationMode = 'ApplyAndAutocorrect'
}
ConfigurationRepositoryWeb main {
ServerURL = 'https://DSCPULLSERVER:8080/PSDSCPullServer.svc'
RegistrationKey = "xxx-xxx-xxxxxx-xxxxxx"
ConfigurationNames = @('DSCLINUXSTATE')
}
}
}
LinuxPullAgent

Here, DSCLINUXSTATE is the name of the configuration placed to a Configurations folder of the DSC Pull Server. RegistrationKey is the key you obtain from the Pull Server. Also, you can specify here how ofter to pull the configuration, and a bunch of other parameters – read more here.

After running this snippet on a machine with PS 4.0 or higher, it produces a local configuration manager file localhost.mof that you need transfer to the Linux node. Basically, it should be the same file for all your Linux nodes using the DSCLINUXSTATE configuration.

Assume, the transferred file is located under ~/localhost.mof. Now, you need to install this Pull Server configuration on the node. We navigate to /Scripts where you can found various Python scripts for different tasks – for example, do not run Register.py, it is intended for use with Azure DSC server. Also, GetDscConfiguration.py and StartDscConfiguration.py – are the scripts for running DSC in the Push mode.

So here we are interested only in SetDscLocalConfigurationManager.py. This script performs a configuration of the pull agent basing on the .mof files generated for this agent. The second command in the next listing displays all the parameters of an active configuration of the LCM and helps to troubleshoot issues related to it.

SetDscLocalConfigurationManager.py -configurationmof ~/localhost.mof
GetDscLocalConfigurationManager.py

After installing the configuration, the pulls are done periodically with a 30-minute minimal interval. To trigger the pull of the configuration manually we need to execute

/opt/microsoft/dsc/bin/ConsistencyInvoker

This command pulls the configuration state from the pull server that we just installed and check if the current state corresponds to it.

And this is it! If everything went well, your Linux is now managed by the DSC. Using nx, nxNetworking and nxComputerManagement modules from Powershell Gallery, you can cover basic scenarios of configuration for Linux machines.

You can troubleshoot the issues related to DSC states or server connectivity by checking the error messages in the log file /var/opt/omi/log/dsc.log. OMI server instance log is located under /var/opt/omi/log/omiserver.log.

Pros and cons of DSC on Linux:

• Pros:
  • Powershell and DSC ecosystem – very convenient when the company’s tech stack mainly includes Windows machines. Managing Linux with the same set of technologies sounds like a dream for decreasing complexity and adding transparency to the CM solution.
  •  Simplicity configuration of the Pull model – server only provides the state, the node does all the work.
  • Extensibility of DSC framework – allows to create custom scripts and utilize existing modules, but for Linux!
  • Microsoft’s open-source direction – it looks very determined, and after Powershell release for Linux, I believe, the integration between these two worlds will only improve, adding more functionality and features to both. Therefore, it might be that tomorrow DSC for Linux will be the next big thing.
  • Free of charge comparing to 120$+/node for Puppet or Chef.
• Cons:
  • nx family of modules is still limited comparing to what can do native CM tools (Puppet, Chef, Ansible, Salt).
  • Poor logging and transparency of a node state – it is hard to trace the results of applying states, especially from the DSC Pull server. Logs need to be examined on the agent. Also, requires setting up a reporting server for DSC which is also not being the most user-friendly interface.
  • Requires an orchestration tool for machine deployment (in the case of using VMs). DSC can provision a new node by itself.

In general, it worked for us quite well – we were aware of general DSC limitations and didn’t find anything unexpected in its Linux specific implementation – given our tech stack it turned out to be a perfect solution.

In the next post, I will cover the examples of using nx modules for configuring CentOS node.

Ansible: loops and registering results of commands

Ansible has a great feature of registering results of a performed step. But recently I stumbled upon a need to do it against actions from a loop and then using the registered result in the next steps which are also looped. Most often, it can be used to perform some action only in case if a previous action changed a file.

Firstly, we register `action_state` variable. Its sub-dictionary `results` contains attribute `changed` which can be accessed as `action_state.results.changed`. Now, we need to get it into use in the next looped step. It can be done with ease using `with_together` operator:

- name: copy a file
  copy:
    src: '{{ source }}'
    dest: '{{ target }}'
  with_items: yourlist
  when: item.state == 'present'
  register: action_state

- name: create .ssh dir if file changed in the previous step
  sudo_user: '{{ item.0.user }}'
  file:
    state: directory
    path: '/home/{{ item.0.user }}/.ssh'
    mode: 0700
  with_together:
  - yourlist
  - action_state.results
  when: item.1.changed and item.0.state == 'present'

Here, for the `yourlist` we copied some files to a target machine (i.e. public keys to some temp location) in the loop and then for each of the files performed an action only if a file was changed.