Skip to content

Conversation

zmoog
Copy link
Contributor

@zmoog zmoog commented Nov 13, 2024

Proposed commit message

Mitigates #11729

If the azure.platformlogs.properties field contains a string instead of the expected object, the pipeline renames it as azure.platformlogs.properties.raw field.

This allows:

  • Elasticsearch to index the log events instead of dropping them (status 400)
  • Users to customize parsing of the .raw field in the logs-azure.platformlogs@custom pipeline

This avoids dropping log events while we build a complete and more robust invalid JSON handling solution with #11728

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

@zmoog zmoog changed the title [azure] rename azure.platformlogs.properties as .raw when it contains a string [azure] rename azure.platformlogs.properties as .raw when it contains a string value Nov 13, 2024
@zmoog zmoog added Integration:azure Azure Logs bugfix Pull request that fixes a bug issue Team:obs-ds-hosted-services Observability Hosted Services team [elastic/obs-ds-hosted-services] labels Nov 13, 2024
@zmoog zmoog self-assigned this Nov 13, 2024
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Nov 13, 2024

🚀 Benchmarks report

Package azure 👍(8) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
platformlogs 5649.72 4651.16 -998.56 (-17.67%) 💔

To see the full report comment with /test benchmark fullreport

@zmoog zmoog marked this pull request as ready for review November 13, 2024 22:34
@zmoog zmoog requested review from a team as code owners November 13, 2024 22:34
Copy link

@elasticmachine
Copy link

💚 Build Succeeded

cc @zmoog

@zmoog zmoog merged commit baa3937 into elastic:main Nov 14, 2024
5 checks passed
@zmoog zmoog deleted the zmoog/azure-logs-platformlogs-fallback-as-raw branch November 14, 2024 17:12
@elastic-vault-github-plugin-prod

Package azure - 1.19.2 containing this change is available at https://epr.elastic.co/package/azure/1.19.2/

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
… a string value (elastic#11732)

Mitigates elastic#11729

If the `azure.platformlogs.properties` field contains a string instead of the expected object, the pipeline renames it as `azure.platformlogs.properties.raw` field. 

This allows:

- Elasticsearch to index the log events instead of dropping them (status 400)
- Users to customize parsing of the `.raw` field in the `logs-azure.platformlogs@custom pipeline`

This avoids dropping log events while we build a complete and more robust [invalid JSON](elastic#11729) handling solution with elastic#11728
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
… a string value (elastic#11732)

Mitigates elastic#11729

If the `azure.platformlogs.properties` field contains a string instead of the expected object, the pipeline renames it as `azure.platformlogs.properties.raw` field. 

This allows:

- Elasticsearch to index the log events instead of dropping them (status 400)
- Users to customize parsing of the `.raw` field in the `logs-azure.platformlogs@custom pipeline`

This avoids dropping log events while we build a complete and more robust [invalid JSON](elastic#11729) handling solution with elastic#11728
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:azure Azure Logs Team:obs-ds-hosted-services Observability Hosted Services team [elastic/obs-ds-hosted-services]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants