Umbra Quickstart

Using Umbra to apply AI to a system is intended to be as easy as possible. When making an AI/ML system there are many considerations, not just the actual AI/ML systems that are going to be used. Umbra seeks to make these steps re-usable and easy to apply to multiple types of applications.

Installation

To get started with Umbra, first install the application:

pip install penumbra

Remember, umbra requires python 3.6 or later to run!

Making a Flow

Note

Umbra can be run as any user on your system. This intro will assume that you are running as root for simplicity, but is running as a non-root user the configurations used here will be available in that user’s home directory under ~/.umbra

The first thing to do is to make a flow. Umbra defines the AI pipeline in flow files. Start by making a flow file in /etc/umbra/flows/shell.yml:

shell:
  ingress:
    json: [/root/shell.json]
  data: salt_event
  model: knn
  egress: cli
  train_for: 10000

This will set up Umbra to read in the dataset from the file located at /root/shell.json. This is the ingress point, it will be sent through the salt_event data manager which makes the dataset readable for AI, then it will used the knn model to analize the data. Finally the egress system is the cli, so the results will just be spit out to the command line. train_for tells us how many data points to use for training before we start to predict if the following data points are anomalies.

You have just defined an Umbra Pipe called shell. You can run multiple pipes at the same time and create as many flow files as you like.

The shell.json file can be generated by running the mkdata.py script found in the demo directory. It will use your shell history to create a larger dataset.

Now you can run Umbra and see the anomalies:

umbra

Give it a few moments to train on the dataset, and then it will tell you what outliers it found in the generated shell.json data.