manifest.yml

title: Cisco Umbrella logs
type: logs
streams:
  - input: aws-s3
    enabled: false
    title: Cisco Umbrella logs
    description: Collect Cisco Umbrella logs
    template_path: aws-s3.yml.hbs
    vars:
      - name: queue_url
        type: text
        title: Queue URL
        multi: false
        required: false
        show_user: true
        description: URL of the AWS SQS queue that messages will be received from. For Cisco Managed S3 buckets or S3 without SQS, use Bucket ARN.
      - name: bucket_arn
        type: text
        title: Bucket ARN
        multi: false
        required: false
        show_user: true
        description: >-
          Required for Cisco Managed S3. If the S3 bucket does not use SQS, this is the address for the S3 bucket, one example is `arn:aws:s3:::cisco-managed-eu-central-1`. For a list of Cisco Managed buckets, please see https://docs.umbrella.com/mssp-deployment/docs/enable-logging-to-a-cisco-managed-s3-bucket.
      - name: access_point_arn
        type: text
        title: Access Point ARN
        multi: false
        required: false
        show_user: true
        description: This is an alternative to the Bucket ARN when fetching logs from S3 without SQS. For Cisco Managed S3 use Bucket ARN instead.
      - name: region
        type: text
        title: Bucket Region
        multi: false
        required: false
        show_user: true
        description: >-
          Required for Cisco Managed S3. The region the bucket is located in.
      - name: bucket_list_prefix
        type: text
        title: Bucket List Prefix
        multi: false
        required: false
        show_user: true
        description: "Required for Cisco Managed S3. This sets the root folder of the S3 bucket that should be monitored, found in the S3 Web UI. Example value: `1235_654vcasd23431e5dd6f7fsad457sdf1fd5`. "
      - name: number_of_workers
        type: text
        title: Number of Workers
        multi: false
        required: false
        show_user: true
        default: 1
        description: Required for Cisco Managed S3. Number of workers that will process the S3 objects listed. Minimum is 1.
      - name: start_timestamp
        type: text
        title: "Start Timestamp"
        multi: false
        required: false
        show_user: false
        description: If set, only read S3 objects with last modified timestamp newer than the given timestamp. Accepts a timestamp in `YYYY-MM-DDTHH:MM:SSZ` format. For example, "2020-10-10T10:30:00Z" (UTC) or "2020-10-10T10:30:00Z+02:30" (with zone offset).
      - name: ignore_older
        type: text
        title: "Ignore Older Timespan"
        multi: false
        required: false
        show_user: false
        description: If set, ignore S3 objects whose Last-Modified time is before the ignore older timespan. Timespan is checked from the current time to S3 object's Last-Modified time. Accepts a duration like `48h`, `2h30m`.
      - name: bucket_list_interval
        type: text
        title: Bucket List Interval
        multi: false
        required: false
        show_user: true
        description: Time interval for polling listing of the S3 bucket. Defaults to 120s.
      - name: shared_credential_file
        type: text
        title: Shared Credential File
        multi: false
        required: false
        show_user: false
        description: Directory of the shared credentials file.
      - name: credential_profile_name
        type: text
        title: Credential Profile Name
        multi: false
        required: false
        show_user: false
      - name: access_key_id
        type: text
        title: Access Key ID
        multi: false
        required: false
        show_user: true
        secret: false
      - name: secret_access_key
        type: password
        title: Secret Access Key
        multi: false
        required: false
        show_user: true
        secret: true
      - name: session_token
        type: password
        title: Session Token
        multi: false
        required: false
        show_user: true
        secret: true
      - name: role_arn
        type: text
        title: Role ARN
        multi: false
        required: false
        show_user: false
      - name: endpoint
        type: text
        title: Endpoint
        multi: false
        required: false
        show_user: false
        default: ""
        description: URL of the entry point for an AWS web service.
      - name: default_region
        type: text
        title: Default AWS Region
        multi: false
        required: false
        show_user: false
        default: ""
        description: Default region to use prior to connecting to region specific services/endpoints if no AWS region is set from environment variable, credentials or instance profile. If none of the above are set and no default region is set as well, `us-east-1` is used. A region, either from environment variable, credentials or instance profile or from this default region setting, needs to be set when using regions in non-regular AWS environments such as AWS China or US Government Isolated.
      - name: visibility_timeout
        type: text
        title: Visibility Timeout
        multi: false
        required: false
        show_user: false
        description: The duration that the received messages are hidden from subsequent retrieve requests after being retrieved by a ReceiveMessage request. The maximum is 12 hours. Valid time units are h, m, s.
      - name: api_timeout
        type: text
        title: API Timeout
        multi: false
        required: false
        show_user: false
        description: The maximum duration of AWS API can take. The maximum is half of the visibility timeout value. Valid time units are h, m, s.
      - name: fips_enabled
        type: bool
        title: Enable S3 FIPS
        default: false
        multi: false
        required: false
        show_user: false
        description: Enabling this option changes the service name from `s3` to `s3-fips` for connecting to the correct service endpoint.
      - name: proxy_url
        type: text
        title: Proxy URL
        multi: false
        required: false
        show_user: false
        description: URL to proxy connections in the form of http\[s\]://<user>:<password>@<server name/ip>:<port>
      - name: tags
        type: text
        title: Tags
        multi: true
        required: true
        show_user: false
        default:
          - cisco-umbrella
          - forwarded
      - name: preserve_original_event
        required: true
        show_user: true
        title: Preserve original event
        description: Preserves a raw copy of the original event, added to the field `event.original`.
        type: bool
        multi: false
        default: false
      - name: file_selectors
        type: yaml
        title: File Selectors
        multi: false
        required: false
        show_user: false
        description: >-
          If the S3 bucket will have events that correspond to files that this integration shouldn't process, file_selectors can be used to limit the files that are downloaded. This is a list of selectors which are made up of regex and other options such as expand_event_list_from_field or parsers. If file_selectors is given, then any global expand_event_list_from_field or parsers values are ignored in favor of the ones specified in the file_selectors. Regexes use [RE2 syntax](https://pkg.go.dev/regexp/syntax). Files that don’t match one of the regexes will not be processed. For more information see [File Selectors](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-aws-s3.html#_file_selectors)
        default: |
          - regex: "dnslogs/.+"
          - regex: "proxylogs/.+"
          - regex: "firewalllogs/.+"
          - regex: "cloudfirewalllogs/.+"
          - regex: "iplogs/.+"
          - regex: "intrusionlogs/.+"
          - regex: "dlplogs/.+"
          - regex: "auditlogs/.+"
            parsers:
              - multiline:
                  # Prevent multiline CSV being split in audit logs.
                  type: pattern
                  pattern: '\"$'
                  negate: true
                  match: before
      - name: parsers
        type: yaml
        title: Parsers
        multi: false
        required: false
        show_user: false
        description: >-
          This option expects a list of parsers that the payload has to go through. If file_selectors is given, parsers must be set for each regex entry in the file_selectors box as this global option will be ignored. For more information see [Parsers](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-aws-s3.html#input-aws-s3-parsers)
        default: |
          - multiline:
              # Prevent multiline CSV being split in audit logs.
              type: pattern
              pattern: '\"$'
              negate: true
              match: before
      - name: processors
        type: yaml
        title: Processors
        multi: false
        required: false
        show_user: false
        description: >-
          Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.