EDIT

We will use collectd + facette for now:

Test installation on edit-test as client and edit-staging as master to collect the time series data

The facette server is running at: http://edit-staging.bgbm.fu-berlin.de/facette/

collectd¶

• collectd Wiki

https://www.21x9.org/de/serverueberwachung-mit-collectd-und-facette.html
https://www.tecmint.com/configure-collectd-as-central-monitoring-server-for-clients/2/

installation¶

Collectd will be installed on the client and master

apt-get install collectd

Configuration¶

create types.db file with custom types (needed for GenericJMX), create this file for the server and client machine

echo "jmx_memory      value:GAUGE:0:U
time_ms      value:GAUGE:0:U" > /etc/collectd/my_types.db    

and uncomment the below line in /etc/collectd/collectd.conf

TypesDB "/usr/share/collectd/types.db" "/etc/collectd/my_types.db"

Apache

https://collectd.org/wiki/index.php/Plugin:Apache

enable and configure mod_status:

a2enmod status # should be enabled already
systemctl reload apache2 # only if needed

in /etc/apache2/mods-available/status.conf:

ExtendedStatus on
<Location /mod_status>
  SetHandler server-status
</Location>

MySQL

https://collectd.org/wiki/index.php/Plugin:MySQL

1. create mysql user collectd (see link above)
2. configure collectd:

<Plugin mysql>
  <Database localhost>
    Port "3306"
    User "collectd"
    Password "*****"
    MasterStats true
  </Database>
</Plugin>

Docker

No official collectd plugin so far. So are using a contributed plugin https://github.com/lebauce/docker-collectd-plugin. A good installation guide is found in the fork https://github.com/signalfx/docker-collectd-plugin

Facette¶

https://github.com/facette/facette

Documentation¶

There is currently only sparse documentation on the configuration of facette. The only source of documentation are the issues:

• RRD Provider setup in Admin Panel
• RRD Provider setup via API
• facette-graph-templates
• Add collections templates #170
• create new graph/collection -> create new graph/collection template
• Graphs/Collections and Templates
• Aggregation function over metric group in a graph template

Templates:

• A graph or a collection is considered being a template once markups are detected in certain fields
• Both graphs and collections can be templates (e.g. if you use {{ .source }} in the source field of a metric in a graph, you'll be able a reuse this graph for multiple hosts, providing a value for source in either a graph created from a template or in a collection referencing it);
• Create new graph/collection will be templates only if template markups are detected (e.g. {{ .var }}). Create from template is used to create resources based on existing templates, it'll ask you to provide values for markups variables existing in the source template;

dump and restore of facette setups

facettectl -a http://$(hostname -I):12003 library dump -o facette-lib_$(date -I).tar.gz
facettectl -a http://$(hostname -I):12003 library restore -i facette-lib****.tar.gz

Installation¶

installing facette from binaries¶

https://github.com/facette/facette/releases

mkdir /root/installer
wget https://github.com/facette/facette/releases/download/0.5.1/facette_0.5.1_stretch-amd64.deb
dpkg -i facette_0.5.1_stretch-amd64.deb

Install from sources¶

apt-get install build-essential golang-go librrd-dev pkg-config pandoc protobuf-compiler

install node_js:

https://github.com/nodesource/distributions/blob/master/README.md#deb

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs
npm install yarn -g

....

Setup and configuration¶

edit /etc/facette/facette.yaml:

listen: 160.45.63.177:12003

restart facette:

systemctl restart facette.service 

Admin panel¶

The admin panel is usually set readonly. see /etc/facette/facette.yaml

  # Allow/Prevent modifications through API calls
  read_only: true

Configure the provider

(See https://github.com/facette/facette/issues/336)

curl -s -H 'Content-Type: application/json' http://160.45.63.177:12003/api/v1/providers -d '                                                                                                                ─╯
{
  "name": "collectd",
  "description": null,
  "created": "2017-06-14T06:09:19Z",
  "modified": "2017-06-14T06:09:19Z",
  "connector": "rrd",
  "settings": {
    "path": "/var/lib/collectd",
    "pattern": "rrd/(?P<source>[^/]+)/(?P<metric>.+)\\.rrd"
  },
  "filters": [
    {"action": "sieve", "target": "metric", "pattern": "/average$", "into": ""},
    {"action": "rewrite", "target": "metric", "pattern": "/average$", "into": ""},
    {"action": "rewrite", "target": "metric", "pattern": "/value$", "into": ""},
    {"action": "rewrite", "target": "metric", "pattern": "/", "into": "."},
    {"action": "rewrite", "target": "metric", "pattern": "^cpu\\.percent-", "into": "cpu.percent."},
    {"action": "rewrite", "target": "metric", "pattern": "^df-(.+)\\.(?:df_)?(.+)-(.+)", "into": "df.$1.$2.$3"},
    {"action": "rewrite", "target": "metric", "pattern": "^interface-(.+).if_(.+)\\.", "into": "net.$1.$2."},
    {"action": "rewrite", "target": "metric", "pattern": "^load\\.load\\.", "into": "load."},
    {"action": "rewrite", "target": "metric", "pattern": "^memory\\.memory-", "into": "memory."}
  ],
  "refresh_interval": 0,
  "priority": 0,
  "enabled": true
}
'

the new provider should now be available at http://edit-staging:12003/admin/providers/