Kubernetes Deployment

Red Canary provides a Docker container image that can be used to deploy our Linux EDR via a DaemonSet controller to nodes in a Kubernetes cluster. The sensor and plugin processes (cfbrkd, cfpmid) are the only services that run within the container. 

Please note that the container must be privileged because it requires full access to the system to monitor, and it must be included in the host namespace.

Red Canary Linux EDR (AKA Canary Forwarder) Container Image

The filesystem image consists of a standard alpine base image (mostly to provide a shell for easy examination via kubectl exec) and a small set of extra files and directories rooted at /opt/redcanary:

/opt/redcanary/
  cfsvcd
  etc/
  var/
  tmp/
  log/
  hostfs/

The directories are all defined as volume mount points, several of which need attention in the design of your deployment.

etc/

• Linux EDR’s config.json file must be mounted as a filesystem in the container for the service daemon to find. Discover how a ConfigMap volume can be used for this purpose so that a group of nodes can share the same configuration.

var/

• It is used for node-linked persistent state, and must be provided by mounting a filesystem from the pod's host node into the container.

/tmp and /log

• These directories are both for transient storage, in that nothing written to them needs to remain long-term, and no files in them are needed for startup. However, if they don’t persist across container restarts, you may lose telemetry or debug log information in the case of a crash or unexpected shutdown.

hostfs/

• This directory gives the service daemon access to the host node’s root filesystem. Features like telemetry file hashing (MD5) need to be mounted within the container in order to function effectively.

Example ConfigMap or Secret

An easy way to distribute the config.json file for Linux EDR across a group of nodes is to use the Kubernetes ConfigMap and Secret resources.

The example below displays a ConfigMap definition that directly embeds Linux EDRs config.json, which can be mounted to the etc/ mount of a Linux EDR container.

apiVersion: v1
kind: ConfigMap
metadata:
 name: groupconfig
data:
 config.json: |
 {
   "access_token": "",
   "subscription_plan": "Managed",
   "offload_target": "Outpost",
   "outpost_auth_token": ""
 }

This approach allows anyone with access to the storage for the ConfigMap to see the secrets stored within it as plaintext.

It is possible to obfuscate the config.json file instead by creating a Secret:

$ mkdir etc
$ cp config.json etc
$ kubectl create secret generic  --from-file=etc

This creates the resource directly in your cluster; the name you chose for the Secret corresponds to the metadata.name value from the example ConfigMap yaml file.

Example Daemonset

The DaemonSet configuration below is meant as a starting point.

1. Decide between a ConfigMap or Secret and choose an appropriate name for that resource.

2. Place the container image in a private registry accessible to your cluster.

3. Replace the container image to pull with the correctly-named one.

4. Disable AuditD on the nodes where the DaemonSet pods are deployed.

You should also consider the best type of storage for the tmp/ and log/ volumes. If they are left as emptyDir volumes their contents will be lost across pod restarts.

Note: The container must be privileged. Linux EDR requires access to proc filesystem elements that, in the Docker security model, cannot be granted to an unprivileged container via capabilities.

apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: canary-forwarder
spec:
 selector:
   matchLabels:
     name: "canary-forwarder"
 template:
   metadata:
     labels:
       name: "canary-forwarder"
   name: canary-forwarder
 spec:
   hostNetwork: true
   hostPid: true
   tolerations:
     - key: node-role.kubernetes.io/master
       operator: Exists
       effect: NoSchedule
   containers:
     - image: /canary_forwarder:
       imagePullPolicy: Always
       name: canary-forwarder
       securityContext:
         privileged: true
       command: ["/opt/redcanary/cfsvcd"]
       volumeMounts:
         - name: groupconfig
           mountPath: /opt/redcanary/etc
         - name: nodestate
           mountPath: /opt/redcanary/var
         - name: tmp
           mountPath: /opt/redcanary/tmp
         - name: log
           mountPath: /opt/redcanary/log
         - name: hostfs
           mountPath: /opt/redcanary/hostfs
   volumes:
     - name: groupconfig
       secret:
         secretName: 
     - name: nodestate
       hostPath:
         path: /var/lib/misc
     - name: tmp
       emptyDir: {}
     - name: log
       emptyDir: {}
     - name: hostfs
       hostPath:
         path: /