From f22a8d747e23de3cb663db382291fc86ae2d4975 Mon Sep 17 00:00:00 2001 From: "deivid.garcia.garcia" Date: Thu, 20 Nov 2025 15:17:53 +0100 Subject: [PATCH 1/7] tail: New keep_file_handle and fstat_interval_nsec options for tail input #11151 Signed-off-by: David Garcia Signed-off-by: deivid.garcia.garcia --- pipeline/inputs/tail.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 7d1ac7fa1..29596ee1b 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -37,6 +37,8 @@ The plugin supports the following configuration parameters: | `tag_regex` | Set a regular expression to extract fields from the filename. For example: `(?[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?[^_]+)_(?.+)-(?[a-z0-9]{64})\.log$`. | _none_ | | `static_batch_size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` | | `file_cache_advise` | Set the `posix_fadvise` in `POSIX_FADV_DONTNEED` mode. This reduces the usage of the kernel file cache. This option is ignored if not running on Linux. | `on` | +| `keep_file_handle` | Keep file handles open while tailing files. When set to `false`, file handles are closed after reading, which is useful for SMB/SAMBA storage backends that prevent file deletion when handles are open. **Note:** When set to `false`, log rotation detection does not work as it requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | +| `fstat_interval_nsec` | Set the interval for checking file status (`fstat`). This controls how often file metadata is checked, which is important for cloud storage backends that are sensitive to IOPS (Input/Output Operations Per Second). The value accepts time units (for example, `1s`, `500ms`, `1000000000ns`) and will be converted to nanoseconds internally. | `250ms` | | `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` | | `Unicode.Encoding` | Set the Unicode character encoding of the file data. This parameter requests two-byte aligned chunk and buffer sizes. If data isn't aligned for two bytes, Fluent Bit will use two-byte alignment automatically to avoid character breakages on consuming boundaries. Supported values: `UTF-16LE`, `UTF-16BE`, and `auto`. | `none` | | `Generic.Encoding` | Set the non-Unicode encoding of the file data. Supported values: `ShiftJIS`, `UHC`, `GBK`, `GB18030`, `Big5`, `Win866`, `Win874`, `Win1250`, `Win1251`, `Win1252`, `Win2513`, `Win1254`, `Win1255`, and `Win1256`. | `none` | @@ -474,6 +476,38 @@ While file rotation is handled, there are risks of potential log loss when using {% endhint %} +## Tailing files from SMB/SAMBA + +When tailing files from SMB/SAMBA shares on POSIX systems or Kubernetes CSI volume mounts, file handles can prevent file deletion. Although the `FILE_SHARE_DELETE` flag is supported on Windows, it's not available on POSIX systems or when using intermediate abstractions such as Kubernetes CSI volume mounts. Set `keep_file_handle` to `false` to allow file deletion while files are being tailed. Note that this disables log rotation detection. + +Example configuration: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: tail + path: /mnt/samba/logs/*.log + keep_file_handle: false + fstat_interval_nsec: 1s +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name tail + Path /mnt/samba/logs/*.log + Keep_File_Handle false + Fstat_Interval_Nsec 1s +``` + +{% endtab %} +{% endtabs %} + ## Character encoding conversion This feature allows Fluent Bit to convert logs from various character encodings into the standard UTF-8 format. From b1dfc96e99459b9164a2ed490bec15d8a164c793 Mon Sep 17 00:00:00 2001 From: David Date: Sat, 22 Nov 2025 09:26:08 +0100 Subject: [PATCH 2/7] Update pipeline/inputs/tail.md Co-authored-by: Alexa Kreizinger Signed-off-by: David --- pipeline/inputs/tail.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 29596ee1b..5e193d91f 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -37,7 +37,7 @@ The plugin supports the following configuration parameters: | `tag_regex` | Set a regular expression to extract fields from the filename. For example: `(?[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?[^_]+)_(?.+)-(?[a-z0-9]{64})\.log$`. | _none_ | | `static_batch_size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` | | `file_cache_advise` | Set the `posix_fadvise` in `POSIX_FADV_DONTNEED` mode. This reduces the usage of the kernel file cache. This option is ignored if not running on Linux. | `on` | -| `keep_file_handle` | Keep file handles open while tailing files. When set to `false`, file handles are closed after reading, which is useful for SMB/SAMBA storage backends that prevent file deletion when handles are open. **Note:** When set to `false`, log rotation detection does not work as it requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | +| `keep_file_handle` | Keep file handles open while tailing files. If `false`, file handles are closed after reading, which is useful for SMB/Samba storage backends that prevent file deletion when handles are open. However, if `false`, log rotation does not work because that requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | | `fstat_interval_nsec` | Set the interval for checking file status (`fstat`). This controls how often file metadata is checked, which is important for cloud storage backends that are sensitive to IOPS (Input/Output Operations Per Second). The value accepts time units (for example, `1s`, `500ms`, `1000000000ns`) and will be converted to nanoseconds internally. | `250ms` | | `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` | | `Unicode.Encoding` | Set the Unicode character encoding of the file data. This parameter requests two-byte aligned chunk and buffer sizes. If data isn't aligned for two bytes, Fluent Bit will use two-byte alignment automatically to avoid character breakages on consuming boundaries. Supported values: `UTF-16LE`, `UTF-16BE`, and `auto`. | `none` | From 3240bcab243ccd33a7982abb0c8002a460a12478 Mon Sep 17 00:00:00 2001 From: David Date: Sat, 22 Nov 2025 09:26:25 +0100 Subject: [PATCH 3/7] Update pipeline/inputs/tail.md Co-authored-by: Alexa Kreizinger Signed-off-by: David --- pipeline/inputs/tail.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 5e193d91f..132ac9bb7 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -476,7 +476,7 @@ While file rotation is handled, there are risks of potential log loss when using {% endhint %} -## Tailing files from SMB/SAMBA +## Tailing files from SMB/Samba When tailing files from SMB/SAMBA shares on POSIX systems or Kubernetes CSI volume mounts, file handles can prevent file deletion. Although the `FILE_SHARE_DELETE` flag is supported on Windows, it's not available on POSIX systems or when using intermediate abstractions such as Kubernetes CSI volume mounts. Set `keep_file_handle` to `false` to allow file deletion while files are being tailed. Note that this disables log rotation detection. From 96b09280b6a62f557d61694d092456822fa99be0 Mon Sep 17 00:00:00 2001 From: David Date: Sat, 22 Nov 2025 09:26:53 +0100 Subject: [PATCH 4/7] Update pipeline/inputs/tail.md Co-authored-by: Alexa Kreizinger Signed-off-by: David --- pipeline/inputs/tail.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 132ac9bb7..4615811f6 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -478,7 +478,7 @@ While file rotation is handled, there are risks of potential log loss when using ## Tailing files from SMB/Samba -When tailing files from SMB/SAMBA shares on POSIX systems or Kubernetes CSI volume mounts, file handles can prevent file deletion. Although the `FILE_SHARE_DELETE` flag is supported on Windows, it's not available on POSIX systems or when using intermediate abstractions such as Kubernetes CSI volume mounts. Set `keep_file_handle` to `false` to allow file deletion while files are being tailed. Note that this disables log rotation detection. +When tailing files from SMB/Samba shares on POSIX systems or Kubernetes CSI volume mounts, file handles can prevent file deletion. Although the `FILE_SHARE_DELETE` flag is supported on Windows, it's not available on POSIX systems or when using intermediate abstractions such as Kubernetes CSI volume mounts. Set `keep_file_handle` to `false` to allow file deletion while files are being tailed. However, keep in mind that this disables log rotation detection. Example configuration: From ae0b9d15c6487126ca48f321720c8e653adcca5d Mon Sep 17 00:00:00 2001 From: Alexa Kreizinger Date: Mon, 24 Nov 2025 11:21:11 -0800 Subject: [PATCH 5/7] Update pipeline/inputs/tail.md Signed-off-by: Alexa Kreizinger --- pipeline/inputs/tail.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 4615811f6..82f2920e8 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -37,7 +37,7 @@ The plugin supports the following configuration parameters: | `tag_regex` | Set a regular expression to extract fields from the filename. For example: `(?[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?[^_]+)_(?.+)-(?[a-z0-9]{64})\.log$`. | _none_ | | `static_batch_size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` | | `file_cache_advise` | Set the `posix_fadvise` in `POSIX_FADV_DONTNEED` mode. This reduces the usage of the kernel file cache. This option is ignored if not running on Linux. | `on` | -| `keep_file_handle` | Keep file handles open while tailing files. If `false`, file handles are closed after reading, which is useful for SMB/Samba storage backends that prevent file deletion when handles are open. However, if `false`, log rotation does not work because that requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | +| `keep_file_handle` | Keep file handles open while tailing files. If `false`, file handles are closed after reading, which is useful for SMB/Samba storage backends that prevent file deletion when handles are open. However, if `false`, log rotation does not work because that requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | | `fstat_interval_nsec` | Set the interval for checking file status (`fstat`). This controls how often file metadata is checked, which is important for cloud storage backends that are sensitive to IOPS (Input/Output Operations Per Second). The value accepts time units (for example, `1s`, `500ms`, `1000000000ns`) and will be converted to nanoseconds internally. | `250ms` | | `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` | | `Unicode.Encoding` | Set the Unicode character encoding of the file data. This parameter requests two-byte aligned chunk and buffer sizes. If data isn't aligned for two bytes, Fluent Bit will use two-byte alignment automatically to avoid character breakages on consuming boundaries. Supported values: `UTF-16LE`, `UTF-16BE`, and `auto`. | `none` | From 67aee562055db1edf44bf8dabbf8023f00a3815e Mon Sep 17 00:00:00 2001 From: David Date: Sun, 8 Feb 2026 17:36:34 +0100 Subject: [PATCH 6/7] lint fixes --- pipeline/inputs/tail.md | 2 +- vale-styles/FluentBit/Acronyms.yml | 2 ++ vale-styles/FluentBit/Headings.yml | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 58856661e..68dfd612e 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -29,7 +29,7 @@ The plugin supports the following configuration parameters: | `ignore_active_older_files` | Ignore files that are older than the value set in `ignore_older` even if the file is being ingested. | `false` | | `ignore_older` | Ignores files older than `ignore_older`. Supports `m`, `h`, `d` (minutes, hours, days) syntax. | Read all. | | `inotify_watcher` | Set to `false` to use file stat watcher instead of `inotify`. | `true` | -| `keep_file_handle` | Keep file handles open while tailing files. If `false`, file handles are closed after reading, which is useful for SMB/Samba storage backends that prevent file deletion when handles are open. However, if `false`, log rotation does not work because that requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | +| `keep_file_handle` | Keep file handles open while tailing files. If `false`, file handles are closed after reading, which is recommended for SMB/Samba storage backends that prevent file deletion when handles are open. However, if `false`, log rotation doesn't work because that requires an active handle to determine where the original file was rotated to. Rotation will be detected as a truncation, so some logs might be lost. | `true` | | `key` | When a message is unstructured (no parser applied), it's appended as a string under the key name `log`. This option lets you define an alternative name for that key. | `log` | | `mem_buf_limit` | Set a memory limit that the Tail plugin can use when appending data to the engine. If the limit is reached, it will be paused. When the data is flushed, it resumes. | _none_ | | `offset_key` | If enabled, Fluent Bit appends the offset of the current monitored file as part of the record. The value assigned becomes the key in the map. | _none_ | diff --git a/vale-styles/FluentBit/Acronyms.yml b/vale-styles/FluentBit/Acronyms.yml index 066c76d17..dd91b57a8 100644 --- a/vale-styles/FluentBit/Acronyms.yml +++ b/vale-styles/FluentBit/Acronyms.yml @@ -30,6 +30,7 @@ exceptions: - CPU - CRI - CRD + - CSI - CSS - CSV - DCO @@ -122,6 +123,7 @@ exceptions: - SIMD - SLA - SLES + - SMB - SNI - SQL - SSH diff --git a/vale-styles/FluentBit/Headings.yml b/vale-styles/FluentBit/Headings.yml index 96df03ba6..89cd3922f 100644 --- a/vale-styles/FluentBit/Headings.yml +++ b/vale-styles/FluentBit/Headings.yml @@ -142,7 +142,9 @@ exceptions: - SignalFx - SIMD - Slack + - SMB - SQL + - Tailing files from SMB/Samba - SSL - SSM - StatsD From f25709e775c2ee7220a10bcb0a8981e8bbce2641 Mon Sep 17 00:00:00 2001 From: David Date: Sun, 8 Feb 2026 17:38:57 +0100 Subject: [PATCH 7/7] Fix lint issues --- vale-styles/FluentBit/Headings.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vale-styles/FluentBit/Headings.yml b/vale-styles/FluentBit/Headings.yml index 89cd3922f..8420df90f 100644 --- a/vale-styles/FluentBit/Headings.yml +++ b/vale-styles/FluentBit/Headings.yml @@ -144,12 +144,12 @@ exceptions: - Slack - SMB - SQL - - Tailing files from SMB/Samba - SSL - SSM - StatsD - Studio - Sumo Logic + - Tailing files from SMB/Samba - Tanzu - TCP - Telemetry Pipeline