Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 185 additions & 0 deletions .generator/schemas/v1/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6182,6 +6182,190 @@ components:
type: string
x-enum-varnames:
- ARITHMETIC_PROCESSOR
LogsArrayMapArithmeticSubProcessor:
description: |-
An arithmetic sub-processor for use inside an array-map processor.
Unlike the top-level arithmetic processor, `is_enabled` is not supported.
properties:
expression:
description: Arithmetic operation to perform.
example: ""
type: string
is_replace_missing:
default: false
description: Replace missing attribute values with 0.
type: boolean
name:
description: Name of the sub-processor.
type: string
target:
description: Target attribute path for the result.
example: ""
type: string
type:
$ref: "#/components/schemas/LogsArithmeticProcessorType"
required:
- expression
- target
- type
type: object
LogsArrayMapAttributeRemapper:
description: |-
An attribute remapper sub-processor for use inside an array-map processor.
Unlike the top-level attribute remapper, `is_enabled`, `source_type`, and
`target_type` are not supported.
properties:
name:
description: Name of the sub-processor.
type: string
override_on_conflict:
default: false
description: Override the target element if already set.
type: boolean
preserve_source:
default: false
description: Remove or preserve the remapped source element.
type: boolean
sources:
description: Array of source attribute paths.
example:
- ""
items:
type: string
type: array
target:
description: Target attribute path.
example: ""
type: string
target_format:
$ref: "#/components/schemas/TargetFormatType"
type:
$ref: "#/components/schemas/LogsAttributeRemapperType"
required:
- sources
- target
- type
type: object
LogsArrayMapCategorySubProcessor:
description: |-
A category sub-processor for use inside an array-map processor.
Unlike the top-level category processor, `is_enabled` is not supported.
properties:
categories:
description: Array of filters to match against a log and the corresponding value to assign.
items:
$ref: "#/components/schemas/LogsCategoryProcessorCategory"
type: array
name:
description: Name of the sub-processor.
type: string
target:
description: Target attribute path for the category value.
example: ""
type: string
type:
$ref: "#/components/schemas/LogsCategoryProcessorType"
required:
- categories
- target
- type
type: object
LogsArrayMapProcessor:
description: |-
The array-map processor transforms each element of a source array by applying
sub-processors in order and collecting the results into a target array.
Results can be written to a new array, to the source array (in-place), or to
an existing target array. Sub-processors can read from `$sourceElem.<field>`
(object element field), bare `$sourceElem` (primitive element), or any parent
log attribute path. Sub-processors write to `$targetElem.<field>` (object
output field) or bare `$targetElem` (primitive output).
properties:
is_enabled:
default: false
description: Whether or not the processor is enabled.
type: boolean
name:
description: Name of the processor.
type: string
preserve_source:
default: true
description: |-
When `false` and `source != target`, the source attribute is removed after
processing. Cannot be `false` when `source == target`.
type: boolean
processors:
description: |-
Sub-processors applied to each element. Allowed types: `attribute-remapper`,
`string-builder-processor`, `arithmetic-processor`, `category-processor`.
items:
$ref: "#/components/schemas/LogsArrayMapSubProcessor"
type: array
source:
description: |-
Attribute path of the source array. Elements are read-only via `$sourceElem`
inside sub-processors.
example: detail.resource.s3BucketDetails
type: string
target:
description: |-
Attribute path of the output array. Sub-processors write to `$targetElem`
(or `$targetElem.<field>`) to build each output element.
example: ocsf.resources
type: string
type:
$ref: "#/components/schemas/LogsArrayMapProcessorType"
required:
- source
- target
- processors
- type
type: object
LogsArrayMapProcessorType:
default: array-map-processor
description: Type of logs array-map processor.
enum:
- array-map-processor
example: array-map-processor
type: string
x-enum-varnames:
- ARRAY_MAP_PROCESSOR
LogsArrayMapStringBuilderSubProcessor:
description: |-
A string builder sub-processor for use inside an array-map processor.
Unlike the top-level string builder processor, `is_enabled` is not supported.
properties:
is_replace_missing:
default: false
description: Replace missing attribute values with an empty string.
type: boolean
name:
description: Name of the sub-processor.
type: string
target:
description: Target attribute path for the result.
example: ""
type: string
template:
description: Formula with one or more attributes and raw text.
example: ""
type: string
type:
$ref: "#/components/schemas/LogsStringBuilderProcessorType"
required:
- template
- target
- type
type: object
LogsArrayMapSubProcessor:
description: |-
A sub-processor used inside an array-map processor.
Allowed types: `attribute-remapper`, `string-builder-processor`,
`arithmetic-processor`, `category-processor`.
oneOf:
- $ref: "#/components/schemas/LogsArrayMapAttributeRemapper"
- $ref: "#/components/schemas/LogsArrayMapArithmeticSubProcessor"
- $ref: "#/components/schemas/LogsArrayMapStringBuilderSubProcessor"
- $ref: "#/components/schemas/LogsArrayMapCategorySubProcessor"
LogsArrayProcessor:
description: |-
A processor for extracting, aggregating, or transforming values from JSON arrays within your logs.
Expand Down Expand Up @@ -7237,6 +7421,7 @@ components:
- $ref: "#/components/schemas/LogsDecoderProcessor"
- $ref: "#/components/schemas/LogsSchemaProcessor"
- $ref: "#/components/schemas/LogsExcludeAttributeProcessor"
- $ref: "#/components/schemas/LogsArrayMapProcessor"
LogsQueryCompute:
description: Define computation for a log query.
properties:
Expand Down
49 changes: 49 additions & 0 deletions docs/datadog_api_client.v1.model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1957,6 +1957,55 @@ datadog\_api\_client.v1.model.logs\_arithmetic\_processor\_type module
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_arithmetic\_sub\_processor module
---------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_arithmetic_sub_processor
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_attribute\_remapper module
--------------------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_attribute_remapper
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_category\_sub\_processor module
-------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_category_sub_processor
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_processor module
----------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_processor
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_processor\_type module
----------------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_processor_type
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_string\_builder\_sub\_processor module
--------------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_string_builder_sub_processor
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_map\_sub\_processor module
---------------------------------------------------------------------

.. automodule:: datadog_api_client.v1.model.logs_array_map_sub_processor
:members:
:show-inheritance:

datadog\_api\_client.v1.model.logs\_array\_processor module
-----------------------------------------------------------

Expand Down
46 changes: 46 additions & 0 deletions examples/v1/logs-pipelines/CreateLogsPipeline_104735144.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
"""
Create a pipeline with Array Map Processor with preserve_source false returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v1.api.logs_pipelines_api import LogsPipelinesApi
from datadog_api_client.v1.model.logs_array_map_attribute_remapper import LogsArrayMapAttributeRemapper
from datadog_api_client.v1.model.logs_array_map_processor import LogsArrayMapProcessor
from datadog_api_client.v1.model.logs_array_map_processor_type import LogsArrayMapProcessorType
from datadog_api_client.v1.model.logs_attribute_remapper_type import LogsAttributeRemapperType
from datadog_api_client.v1.model.logs_filter import LogsFilter
from datadog_api_client.v1.model.logs_pipeline import LogsPipeline

body = LogsPipeline(
filter=LogsFilter(
query="source:python",
),
name="testPipelineArrayMapNoPreserve",
processors=[
LogsArrayMapProcessor(
type=LogsArrayMapProcessorType.ARRAY_MAP_PROCESSOR,
is_enabled=True,
name="map and remove source",
source="items",
target="out",
preserve_source=False,
processors=[
LogsArrayMapAttributeRemapper(
type=LogsAttributeRemapperType.ATTRIBUTE_REMAPPER,
sources=[
"$sourceElem.id",
],
target="$targetElem.uid",
),
],
),
],
tags=[],
)

configuration = Configuration()
with ApiClient(configuration) as api_client:
api_instance = LogsPipelinesApi(api_client)
response = api_instance.create_logs_pipeline(body=body)

print(response)
57 changes: 57 additions & 0 deletions examples/v1/logs-pipelines/CreateLogsPipeline_1185292896.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
"""
Create a pipeline with Array Map Processor using category sub-processor returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v1.api.logs_pipelines_api import LogsPipelinesApi
from datadog_api_client.v1.model.logs_array_map_category_sub_processor import LogsArrayMapCategorySubProcessor
from datadog_api_client.v1.model.logs_array_map_processor import LogsArrayMapProcessor
from datadog_api_client.v1.model.logs_array_map_processor_type import LogsArrayMapProcessorType
from datadog_api_client.v1.model.logs_category_processor_category import LogsCategoryProcessorCategory
from datadog_api_client.v1.model.logs_category_processor_type import LogsCategoryProcessorType
from datadog_api_client.v1.model.logs_filter import LogsFilter
from datadog_api_client.v1.model.logs_pipeline import LogsPipeline

body = LogsPipeline(
filter=LogsFilter(
query="source:python",
),
name="testPipelineArrayMapCategory",
processors=[
LogsArrayMapProcessor(
type=LogsArrayMapProcessorType.ARRAY_MAP_PROCESSOR,
is_enabled=True,
name="categorize items",
source="items",
target="out",
processors=[
LogsArrayMapCategorySubProcessor(
type=LogsCategoryProcessorType.CATEGORY_PROCESSOR,
target="$targetElem.level",
categories=[
LogsCategoryProcessorCategory(
filter=LogsFilter(
query="@$sourceElem.status:error",
),
name="error",
),
LogsCategoryProcessorCategory(
filter=LogsFilter(
query="*",
),
name="info",
),
],
),
],
),
],
tags=[],
)

configuration = Configuration()
with ApiClient(configuration) as api_client:
api_instance = LogsPipelinesApi(api_client)
response = api_instance.create_logs_pipeline(body=body)

print(response)
Loading
Loading