Adding fireeye nx package #1887

emnp · 2021-10-10T09:32:15Z

What does this PR do?

Adds FireEye NX package.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

Author's Checklist

Data Stream Added
Kibana Visualizations & Dahsboards

How to test this PR locally

# cd integrations/packages/fireeye
# elastic-package build
# elastic-package stack up -d -v
# eval "$(elastic-package stack shellinit)"
# elastic-package test  -v

Related issues

Screenshots

cla-checker-service · 2021-10-10T09:32:18Z

💚 CLA has been signed

elasticmachine · 2021-10-10T09:35:56Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-10-27T10:18:20.313+0000
Duration: 15 min 24 sec
Commit: 784e198

Test stats 🧪

Test	Results
Failed	0
Passed	7
Skipped	0
Total	7

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

emnp · 2021-10-10T10:14:54Z

/test

elasticmachine · 2021-10-11T13:03:51Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

P1llus

Thank you very much for this integration @emnp It is looking awesome already 👍

There is some comments below, but they are all super small, so no need to worry!

packages/fireeye/data_stream/nx/_dev/test/pipeline/test-common-config.yml

packages/fireeye/data_stream/nx/agent/stream/tcp.yml.hbs

packages/fireeye/data_stream/nx/manifest.yml

packages/fireeye/manifest.yml

packages/fireeye/changelog.yml

P1llus · 2021-10-13T11:07:23Z

/test

P1llus · 2021-10-13T11:24:55Z

Thanks for the updates @emnp .

It seems that the CI just has some small formatting issues left.

Could you run these two commands in the folder of the specific package:
elastic-package build
elastic-package format && elastic-package lint && elastic-package check

It will fix any formatting issues that you can commit, or point out what is missing.

emnp · 2021-10-13T11:50:35Z

Thanks for the updates @emnp .

It seems that the CI just has some small formatting issues left.

Could you run these two commands in the folder of the specific package: elastic-package build elastic-package format && elastic-package lint && elastic-package check

It will fix any formatting issues that you can commit, or point out what is missing.

@P1llus I check with these commands and formatting issues are clear now. And also all tests are working well. Could you please check the updates that I pushed?

P1llus · 2021-10-13T11:52:42Z

/test

P1llus · 2021-10-13T12:09:16Z

@emnp Seems like it is still complaining about integrations/packages/fireeye/data_stream/nx/manifest.yml. Could you run the second command again and push the changes it does? Might be a missing whitespace or something

P1llus · 2021-10-14T14:40:35Z

/test

P1llus · 2021-10-14T14:41:02Z

If the build doesn't pass now I might try to fix it and commit it to this PR if it is okay?

emnp · 2021-10-15T03:41:04Z

If the build doesn't pass now I might try to fix it and commit it to this PR if it is okay?

Sorry, @P1llus. I missed your comment. Did the build pass or did you try to fix it?

packages/fireeye/data_stream/nx/_dev/test/pipeline/test-nx.log-expected.json

packages/fireeye/data_stream/nx/fields/fields.yml

packages/fireeye/data_stream/nx/elasticsearch/ingest_pipeline/renaming-raws.yml

P1llus

I added a few small changes that would be nice to have, while we wait for the CI to pass @emnp

packages/fireeye/data_stream/nx/agent/stream/stream.yml.hbs

packages/fireeye/data_stream/nx/agent/stream/tcp.yml.hbs

packages/fireeye/data_stream/nx/agent/stream/udp.yml.hbs

packages/fireeye/manifest.yml

P1llus · 2021-10-25T11:27:45Z

Currently waiting for this to finish before triggering CI again: #2017

Thanks for fixing the last few comments :)

P1llus · 2021-10-25T18:40:05Z

/test

P1llus · 2021-10-26T06:03:55Z

Did you run one last format/lint/package build afterwards? If not I can do it as well of course @emnp .

emnp · 2021-10-26T08:04:20Z

Did you run one last format/lint/package build afterwards? If not I can do it as well of course @emnp .

Yes, @P1llus I did run these commands before pushing to git.

P1llus · 2021-10-26T09:34:07Z

Hey again @emnp 👋
I went through it, and the issue was that since we moved the observer fields to an ingest pipeline, the output of the pipeline test files was slightly different, so just had to regenerate those.

I also added some minor changes in capitalization of letters, and since all the required pipeline and system tests are available, I moved it from experimental to GA.
Would you be able to look through my changes and see if you agree? Feel free to comment on anything I did change.

P1llus · 2021-10-26T10:02:32Z

/test

emnp · 2021-10-26T10:21:58Z

Hey @P1llus, yes we moved the observer fields to ingest pipeline and I forgot to regenerate again :(.
And also this is my first experience. Thanks for correcting me.

P1llus · 2021-10-26T10:23:17Z

Hey @P1llus, yes we moved the observer fields to ingest pipeline and I forgot to regenerate again :(. And also this is my first experience. Thanks for correcting me.

No problem at all, we greatly appreciate it! :) So are you also okay with the changes that I performed @emnp?

emnp · 2021-10-26T10:38:12Z

Hey @P1llus, yes we moved the observer fields to ingest pipeline and I forgot to regenerate again :(. And also this is my first experience. Thanks for correcting me.

No problem at all, we greatly appreciate it! :) So are you also okay with the changes that I performed @emnp?

Yes, it's okay.

marc-gr

As a suggestion, could be interesting to add a community_id processor (https://www.elastic.co/guide/en/elasticsearch/reference/master/community-id-processor.html) using the available fields. Other that that and the couple of comments, it looks good 👍

packages/fireeye/_dev/deploy/docker/docker-compose.yml

packages/fireeye/data_stream/nx/agent/stream/stream.yml.hbs

P1llus · 2021-10-27T09:26:39Z

Adding the small fixes now

P1llus · 2021-10-27T12:28:46Z

Thank you very much for the contribution @emnp :) It is now merged and will be published with the rest of the 7.16 packages. Hope to work with you again in the future 👍

emnp added 2 commits October 10, 2021 15:13

Adding fireeye nx package

438d3dd

add PR link

98c6160

default

5768ab1

jamiehynds added the Team:Security-External Integrations label Oct 11, 2021

jamiehynds added the New Integration Issue or pull request for creating a new integration package. label Oct 11, 2021

P1llus reviewed Oct 11, 2021

View reviewed changes

emnp added 6 commits October 12, 2021 16:13

remove comments

31606f6

update tcp stream

794fc78

update manifest

f415f35

change version

c01d113

change github owner

080535a

add pr link

019b671

emnp added 3 commits October 13, 2021 18:07

fix linting package error

cfde7b4

retest pipeline with kibana 7.16

c40347f

retest system with kibana 7.16

8e938ca

emnp added 2 commits October 13, 2021 21:31

update manifest

4a2f736

update manifest

15e34a3

efd6 reviewed Oct 18, 2021

View reviewed changes

P1llus reviewed Oct 25, 2021

View reviewed changes

emnp added 7 commits October 25, 2021 17:21

update file

768cf23

update file

d01cc05

remove fields part

907be6a

remove fields part

ae1bad1

add set processors

11e617c

remove sample img

c720b96

remove screenshots part

169b5b6

adding some minor changes

bfcc131

running format and update generated docs

3bad8a9

P1llus requested a review from a team October 26, 2021 10:53

marc-gr requested changes Oct 27, 2021

View reviewed changes

packages/fireeye/_dev/deploy/docker/docker-compose.yml Outdated Show resolved Hide resolved

packages/fireeye/data_stream/nx/agent/stream/stream.yml.hbs Show resolved Hide resolved

P1llus added 2 commits October 27, 2021 11:42

adding changes from PR comments, and fixing some small pipeline issues

ac544ae

linting

f44f474

P1llus requested a review from marc-gr October 27, 2021 09:44

marc-gr approved these changes Oct 27, 2021

View reviewed changes

P1llus added 2 commits October 27, 2021 11:53

adding related.ip field

c8c87c3

update docs

784e198

P1llus merged commit c77af22 into elastic:master Oct 27, 2021

Adding fireeye nx package #1887

Adding fireeye nx package #1887

Uh oh!

Conversation

emnp commented Oct 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

cla-checker-service bot commented Oct 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Oct 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

emnp commented Oct 10, 2021

Uh oh!

elasticmachine commented Oct 11, 2021

Uh oh!

P1llus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

P1llus commented Oct 13, 2021

Uh oh!

P1llus commented Oct 13, 2021

Uh oh!

emnp commented Oct 13, 2021

Uh oh!

P1llus commented Oct 13, 2021

Uh oh!

P1llus commented Oct 13, 2021

Uh oh!

P1llus commented Oct 14, 2021

Uh oh!

P1llus commented Oct 14, 2021

Uh oh!

emnp commented Oct 15, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

P1llus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

P1llus commented Oct 25, 2021

Uh oh!

P1llus commented Oct 25, 2021

Uh oh!

P1llus commented Oct 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emnp commented Oct 26, 2021

Uh oh!

P1llus commented Oct 26, 2021

Uh oh!

P1llus commented Oct 26, 2021

Uh oh!

emnp commented Oct 26, 2021

emnp commented Oct 10, 2021 •

edited

Loading

cla-checker-service bot commented Oct 10, 2021 •

edited

Loading

elasticmachine commented Oct 10, 2021 •

edited

Loading

P1llus commented Oct 26, 2021 •

edited

Loading

P1llus commented Oct 26, 2021 •

edited

Loading