MozDef Services

MozDef services can be broken up into 3 different groups (Alert, Ingest, Web). Each group of services should be run on the same machine, with all of the Ingest services able to run on N number of machines, allowing for a more distrubuted environment.

Note

It’s recommended in a distributed environment, to have only 1 Alert Service node, 1 Web Service node, and N Ingest Service nodes.

MozDef Service Service Type Required Service(s)
Alerts Alert Elasticsearch, RabbitMQ, MozdefRestAPI
Alert Actions Alert RabbitMQ
Bot Alert RabbitMQ
Loginput Ingest RabbitMQ, Nginx
MQ Workers Ingest Elasticsearch, RabbitMQ
RestAPI Web Elasticsearch, Mongodb, Nginx
Meteor Web Mongodb, MozdefRestAPI, Nginx
Kibana Web Elasticsearch, Nginx

RestAPI

The MozDef RestAPI service is an HTTP API that works alongside the Mozdef Web Service.

Copy over systemd file:

cp /opt/mozdef/envs/mozdef/systemdfiles/web/mozdefrestapi.service /usr/lib/systemd/system/mozdefrestapi.service

Start loginput service:

systemctl start mozdefrestapi
systemctl enable mozdefrestapi

Verify service is working:

curl http://localhost:8081/status

You should see some json returned!

Alerts

The alerts service searches Elasticsearch for specific terms (defined in specific alerts), and will create an Alert document if any of the alerts found events.

Note

The alerts service depends on Elasticsearch, RabbitMQ, AND the MozdefRestAPI.

Let’s edit the configuration file:

vim /opt/mozdef/envs/mozdef/alerts/lib/config.py

The ALERTS dictionary is where we define what alerts are running, and with what schedule they run on. The dictionary key consists of 2 fields, the alert filename (excluding the .py extension), and the alert classname. The dictionary value is a dictionary containing the schedule. An example:

ALERTS = {
    'bruteforce_ssh.AlertBruteforceSsh': {'schedule': crontab(minute='*/1')},
    'unauth_ssh.AlertUnauthSSH': {'schedule': crontab(minute='*/1')},
    'get_watchlist.AlertWatchList': {'schedule': crontab(minute='*/1')},
}

Copy over systemd file:

cp /opt/mozdef/envs/mozdef/systemdfiles/alert/mozdefalerts.service /usr/lib/systemd/system/mozdefalerts.service

Start alerts service:

systemctl start mozdefalerts
systemctl enable mozdefalerts

Look at logs:

tail -f /var/log/mozdef/supervisord/alert_errors.log

Alert Actions

The Alert Actions service consists of pluggable modules that perform certain actions if certain alerts are detected. These actions are simply python files, so actions like sending an email or triggering a pagerduty notification are possible.

These actions are stored in /opt/mozdef/envs/mozdef/alerts/actions/

Let’s edit the configuration file:

vim /opt/mozdef/envs/mozdef/alerts/lib/config.py

The ALERT_ACTIONS list is where we define what alert actions are running. Each entry is simply the filename of the alert action to run (excluding the .py extension). An example:

ALERT_ACTIONS = [
    'pagerDutyTriggerEvent',
]

Copy over systemd file:

cp /opt/mozdef/envs/mozdef/systemdfiles/alert/mozdefalertactions.service /usr/lib/systemd/system/mozdefalertactions.service

Start alert actions service:

systemctl start mozdefalertactions
systemctl enable mozdefalertactions

Look at logs:

tail -f /var/log/mozdef/alertactions.log

Bot

The MozDef Bot service is a method for sending alerts to either an IRC or slack channel(s).

The source code for this service is broken up into multiple directories, depending on if you want to use Slack or IRC. For this example, we’re going to be using the Slack functionality.

Let’s edit the configuration file and set our channel and secrets accordingly:

vim /opt/mozdef/envs/mozdef/bot/slack/mozdefbot.conf

Copy over systemd file:

cp /opt/mozdef/envs/mozdef/systemdfiles/alert/mozdefbot.service /usr/lib/systemd/system/mozdefbot.service

Start bot service:

systemctl start mozdefbot
systemctl enable mozdefbot

Look at logs:

tail -f /var/log/mozdef/mozdefbot.log

Cron

Crontab is used to run periodic maintenance tasks in MozDef.

MozDef cron entries can be broken up similarly to the 3 Service Groups (Alerts, Ingest, Web).

Note

You should run the Ingest Related Tasks on each ingest host that you have in your MozDef environment.

Recommended Mozdef Crontab:

su mozdef
crontab -e

With the following contents:

## Alert Related Tasks ##
*/15 * * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/eventStats.sh
0 0 * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/esMaint.sh
0 8 * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/pruneES.sh
0 0 * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/update_geolite_db.sh
0 1 * * 0 /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/closeIndices.sh

## Ingest Related Tasks ##
* * * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/healthAndStatus.sh
0 0 * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/update_geolite_db.sh

## Web Related Tasks ##
* * * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/healthToMongo.sh
* * * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/collectAttackers.sh
* * * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/syncAlertsToMongo.sh
# Uncomment if running multiple Elasticsearch nodes
#0 * * * * /opt/mozdef/envs/mozdef/cron/cronic /opt/mozdef/envs/mozdef/cron/esCacheMaint.sh

Loginput

The MozDef Loginput service is an HTTP API to send events to MozDef by external sources.

Copy over systemd file:

cp /opt/mozdef/envs/mozdef/systemdfiles/consumer/mozdefloginput.service /usr/lib/systemd/system/mozdefloginput.service

Start loginput service:

systemctl start mozdefloginput
systemctl enable mozdefloginput

Verify service is working:

curl http://localhost:8080/status

You should see some json returned!

MQ Workers

MQ Workers are the main service to pull events into MozDef. These workers can pull from a queue from RabbitMQ, Cloudtrail, Amazon SQS, Amazon SNS and Papertrail.

The MQ worker files are stored in /opt/mozdef/envs/mozdef/mq/.

For this example, we’re going to focus on configuring and running the eventtask worker. This specific workers pull from a RabbitMQ queue, which will be populated by the MozDef Loginput service.

Each MQ worker service has the following associated files:

1. A .ini file used to control certain properties of the worker service (number of processes, logging directory, etc). Modify eventtask ini file:

vim /opt/mozdef/envs/mozdef/mq/eventtask.ini

Note

The mules key is used to determine how many “processes” the worker service will run. Depending on the amount of incoming messages, you may need to duplicate this line (thus adding more processes to run).

2. A .conf file used to store credentials and other configuration options for the worker process. Modify eventtask conf file:

vim /opt/mozdef/envs/mozdef/mq/esworker_eventtask.conf

3. A systemd file used to start/stop/enable the specific MQ worker. Copy systemd file into place:

cp /opt/mozdef/envs/mozdef/systemdfiles/consumer/mworker-eventtask.service /usr/lib/systemd/system/mworker-eventtask.service

Start worker:

systemctl start mworker-eventtask
systemctl enable mworker-eventtask

Web

Meteor is a javascript framework used for the realtime aspect of the web interface.

Install requirements:

export NODE_VERSION=8.11.4
export METEOR_VERSION=1.8

cd /opt/mozdef
gpg="gpg --no-default-keyring --secret-keyring /dev/null --keyring /dev/null --no-option --keyid-format 0xlong"
rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
rpm -qi gpg-pubkey-f4a80eb5 | $gpg | grep 0x24C6A8A7F4A80EB5
curl --silent --location https://rpm.nodesource.com/setup_8.x | bash -
rpmkeys --import /etc/pki/rpm-gpg/NODESOURCE-GPG-SIGNING-KEY-EL
rpm -qi gpg-pubkey-34fa74dd | $gpg | grep 0x5DDBE8D434FA74DD
yum install -y \
   make \
   glibc-devel \
   gcc \
   gcc-c++ \
   libstdc++ \
   zlib-devel \
   nodejs
curl --silent --location https://static-meteor.netdna-ssl.com/packages-bootstrap/$METEOR_VERSION/meteor-bootstrap-os.linux.x86_64.tar.gz \
  | tar --extract --gzip --directory /opt/mozdef .meteor
ln --symbolic /opt/mozdef/.meteor/packages/meteor-tool/*/mt-os.linux.x86_64/scripts/admin/launch-meteor /usr/bin/meteor
install --owner mozdef --group mozdef --directory /opt/mozdef/envs /opt/mozdef/envs/mozdef
chown -R mozdef:mozdef /opt/mozdef/envs/mozdef/meteor
chown -R mozdef:mozdef /opt/mozdef/.meteor

Let’s edit the configuration file:

vim /opt/mozdef/envs/mozdef/meteor/imports/settings.js

Note

We’ll need to modify the rootURL and kibanaURL variables in settings.js

The default setting will use Meteor Accounts, but you can just as easily install an external provider like Github, Google, Facebook or your own OIDC:

mozdef = {
  ...
  authenticationType: "meteor-password",
  ...
}

or for an OIDC implementation that passes a header to the nginx reverse proxy (for example using OpenResty with Lua and Auth0):

mozdef = {
  ...
  authenticationType: "OIDC",
  ...
}

In addition, environment variables can also be set instead of requiring modification of the settings.js file.:

OPTIONS_METEOR_ROOTURL is "http://localhost" by default and should be set to the dns name of the UI where you will run MozDef
OPTIONS_METEOR_PORT is 80 by default and is the port on which the UI will run
OPTIONS_METEOR_ROOTAPI is http://rest:8081 by default and should resolve to the location of the rest api
OPTIONS_METEOR_KIBANAURL is http://localhost:9090/app/kibana# by default and should resolve to your kibana installation
OPTIONS_METEOR_ENABLECLIENTACCOUNTCREATION is true by default and governs whether accounts can be created
OPTIONS_METEOR_AUTHENTICATIONTYPE is meteor-password by default and can be set to oidc to allow for oidc authentication
OPTIONS_REMOVE_FEATURES is empty by default, but if you pass a comma separated list of features you'd like to remove they will no longer be available.

Install mozdef meteor project:

su mozdef
export MONGO_URL=mongodb://localhost:3002/meteor
export ROOT_URL=http://localhost
export PORT=3000

mkdir -p /opt/mozdef/envs/meteor/mozdef
cd /opt/mozdef/envs/mozdef/meteor
meteor npm install
meteor build --server localhost:3002 --directory /opt/mozdef/envs/meteor/mozdef
ln --symbolic /opt/mozdef/envs/meteor/mozdef/node_modules /opt/mozdef/envs/mozdef/meteor/node_modules
cd /opt/mozdef/envs/meteor/mozdef/bundle/programs/server
npm install

Copy over systemd file (as root):

cp /opt/mozdef/envs/mozdef/systemdfiles/web/mozdefweb.service /usr/lib/systemd/system/mozdefweb.service
systemctl daemon-reload

Start loginput service:

systemctl start mozdefweb
systemctl enable mozdefweb