• Overview
• Pre-requisites
• Steps
  • Step 1: Create an API key in CSE’s Cloud Command Center with ReadOnly Access
  • Step 2: Store API key in Filebeat store
  • Step 3: Update filebeat.inputs section of the filebeat.yaml file
• Additional Information

Overview

Cloud Secure Edge Events can be piped to your ELK (Elasticsearch, Logstash, Kibana) Stack for log storage and analysis. We do this by integrating with Filebeat and using Cloud Secure Edge’s (formerly Banyan’s) API for events. This integration surfaces events from the console into your ELK Stack and uses Cloud Secure Edge’s (CSE) standard events terminology and definitions.

Pre-requisites

• Filebeat server access
• Rights to store API keys and make configuration changes to the filebeat.yaml file

Steps

Step 1: Create an API key in CSE’s Cloud Command Center with ReadOnly Access

1.1 In the Command Center, navigate from Settings > API Keys.

1.2 Add an API Key, assigning a Name, a Description, and a ReadOnly scope.

Step 2: Store API key in Filebeat store

2.1 Log into the Filebeat system.

2.2 Enter the following in your CLI:

(a) filebeat keystore create

Note: This is only needed if the filebeat key store does not exist.

(b) filebeat keystore add BANYAN_API_KEY

(c) the string generated for the ReadOnly API key from Step 1

Step 3: Update filebeat.inputs section of the filebeat.yaml file

3.1 Enter the following into the filebeat.inputs section:

  - type: httpjson
    config_version: 2
    interval: 1m
    request.url: 'https://net.banyanops.com/api/v1/events'
    request.transforms:
    - set:
        target: header.Authorization
        value: 'Bearer ${BANYAN_API_KEY}'
    - append:
        target: url.params.after
        value: '[[ .cursor.last_created_at ]]'
        default: '[[ (now (parseDuration "-5m")).UnixMilli ]]'
    - append:
        target: url.params.order
        value: 'ASC'
    - append:
        target: url.params.severity
        value: 'INFO'
    - append:
        target: url.params.limit
        value: '1000'
  response.split:
    target: body.data
  cursor:
    last_created_at:
      value: '[[ printf "%d" (add (toInt (index .last_event "created_at")) 1) ]]'
  fields_under_root: true
  fields:
    event.dataset: banyan

3.2 Save your file.

3.3 Restart your Filebeat service.

Note: For more information on how to start Filebeat on each platform type, visit here.

Additional Information

The filebeat.yaml file contains editable fields. We recommend the code block above (in Step 3.1). Some of these fields are described below:

type: httpjson

• Note: Our API for event logs returns in JSON format.

interval: 1m

• Note: Pulling once per minute prevents being rate limited from the API; If you require a shorter interval, please contact support.

value: 'Bearer ${BANYAN_API_KEY}

• Note: BANYAN_API_KEY is the name of the key stored in Step 1. We recommend entering the key into a secure space (i.e., Filebeat keystore) rather than entering the plaintext of the key in the filebeat.yaml file.

value: '[[ .cursor.last_created_at ]]'

• Note: Using our last_created_at value in the events JSON will provide the last time the events database consumed data.

default: '[[ (now (parseDuration "-5m")).UnixMilli ]]'

• Note: If the last_created_at field is unavailable (most often occurs during the first launch of integration), we recommend only obtaining the last 5 minutes of data so that you don’t exceed the request size limit. If you require more, please contact support.

event.dataset: banyan

• Note: We recommend calling the events something specific for easy searching and filtering (e.g., ‘banyan’).