Playing with SaltStack and external classifiers

We have been discussing this a lot lately. How do we structure our SaltStack-config in a way that lets us do changes without possibly breaking abseloutly everything. Finding a good hierarchy is not always easy. How do we build it so that we can open it up later…. How can we manage to mostly leave the top.sls file alone and how do we include the right config for the right minions without having to maintain a list of minions in the config and make it so large its unreadable.

Turns out… the solution was pretty easy for us, as soon as we came up with the idea.

We allready have an internally written CMDB-solution and we wanted to use that as an external classifier.
First we had to write a simple module that made pillars from the data in our CMDB. More about that some other time. This post is about the structure we went for.
Anyway, our cmdb-module creates pillars for all hosts containing hostname, status (dev,qa,prod) and product (or role if you prefer).
This will typically look like this for a dev-server:

ourservername:
status: development
product: ourwebsite


So what we ended up with, was using file_roots in salt, matching each our environments like this:
(top.sls)
base:
"*"
- somebasestuff
"cmdb:status:prod*":
- match: pillar
- role
"cmdb:status:qa":
- match: pillar
- role
"cmdb:status:dev*":
- match: pillar
- role

This will match the role.sls file in all 3 environments.
In all 3 environments we have 2 subfolders. “products” and “services”.
The “product” folders contain the state of the final product, using services
from the “services” folder. For instance, say you have a product called “yourwebsite”.
It will probably contain installation and configuration of web, cache and db. Those 3 are
reusable services under the services folder and doesnt change much.
In our role.sls we are now matching on the pillar “product” in our CMDB like this:

include:
- products/{{ pillar.get('cmdb', {}).get('product') }}

What this will do, is look for the CMDB-value for “product” and then include the matching item in the products-folder …and so, we do not need to maintain the top.sls OR any hostnames in the salt-config. So far we think it is a good idea, but we will see in a few weeks if it actually lives up to our expectations.

Anyways, figured I should share our thoughts.