Azure Monitor Agent -Data Collection Rules / Custom Text Logs

Question

Azure Monitor Agent -Data Collection Rules / Custom Text Logs

Minas Stam 20

We are experiencing an issue with Azure Monitor Agent (AMA) Custom Text Log ingestion

During testing, we observed that AMA correctly identifies new log records when the hour is zero-padded (for example 01:55:00 PM), but fails to consistently recognize timestamps when the hour is not zero-padded (for example 1:55:00 PM).

As a result, multiline log entries are either split into multiple records or merged into adjacent events, causing inaccurate ingestion of the audit data.

To validate the issue, we created multiple test files using different timestamp combinations and confirmed that the behavior is specifically related to single-digit hour values. The issue persists on the latest AMA version.

We would like to understand whether:

This behavior is expected for the M/D/YYYY HH:MM:SS AM/PM timestamp format.
This is a known issue.
There is a supported workaround that would allow AMA to correctly process timestamps with single-digit hour values.

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Answer accepted by question author

0 additional answers

Your answer

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Answer 1

Bharath Y P 10,180 Microsoft External Staff Moderator

Hello Minas Stam, you are using Azure Monitor Agent (AMA) custom text log ingestion with timestamp-based multiline detection, and observing that Works: 01:55:00 PM (zero‑padded hour) but Fails: 1:55:00 PM (single-digit hour). This leads to Incorrect multiline splitting/merging. Log records being grouped incorrectly.

While the format M/D/YYYY HH:MM:SS AM/PM is listed as supported, the implementation of AMA parsing is strict and pattern-sensitive. Based on the version notes Earlier AMA versions had parsing inconsistencies for this format, especially affecting multiline detection.

AMA uses strict timestamp pattern matching (internally regex-based) for multiline boundary detection. The mismatch occurs because the parser expects a fixed-width hour (HH) rather than a flexible (H) when interpreting M/D/YYYY HH:MM:SS AM/PM. As a result 01:55:00 PM > matches expected pattern, 1:55:00 PM > does not match strictly, causing new record NOT detected and Lines appended or merged incorrectly.

You can try with normalize log format, Modify log generation to enforce zero-padded hours 01:55:00 PM Supported reliably. This is the most stable and officially aligned approach Guarantees correct multiline parsing
Use alternative timestamp format, If feasible, switch to a format with better parsing stability YYYY-MM-DD HH:MM:SS (preferred in enterprise scenarios). ISO 8601 (yyyy-MM-ddTHH:mm:ssK) These formats are more consistently handled by AMA
Upgrade AMA agent, ensure you are on a recent AMA version (>= May 2025 builds). The release notes confirm fixes for multiline timestamp parsing issues in M/D/YYYY HH:MM:SS AM/PM, However even with fixes, strict pattern matching may still require zero-padding.

This is not purely a configuration issue, but a known limitation/behavior of AMA timestamp parsing, especially in older versions. The most reliable fix is to enforce zero-padded hours or migrate to a standardized timestamp format (ISO / 24‑hour).

Hope this helps! thank you

Suchitra Suregaunkar 15,130 Reputation points Microsoft External Staff Moderator

2026-06-10T18:58:46.5066667+00:00

Hello Minas Stam,

If the answer is helpful, could you please click "Accept Answer" and this helps other community members who may encounter a similar issue in the future.

If you’re still experiencing the problem or need further clarification, please feel free to share additional information so we can continue investigating and assist you further.

Thanks,

Suchitra
Minas Stam 20 Reputation points

2026-06-10T20:45:25.77+00:00

Hi,

Thank you for the detailed response. We have tried the following approaches:

Modifying the fluent-bit parsers.conf — we manually updated the timestamp-format2 regex to use \d{1,2} instead of \d\d for the hour field to support single-digit hours. While the fix worked when applied manually, AMA overwrites the parsers.conf automatically when it refreshes its configuration, making this approach unsustainable.

Modifying the DCR — we attempted to pass a custom regex via the DCR JSON using recordStartTimestampCustomFormat and multiline properties, however Azure stripped these properties on save, confirming they are not supported at the ARM level.

Switching timestamp formats — unfortunately the system generating the logs does not support alternative timestamp formats such as ISO 8601 or 24-hour format, so migrating to a more standardized format is not an option in our case.

Given these constraints, we would like to ask:

Is there a planned update to AMA that would fix the timestamp-format2 regex to natively support single-digit hours?

Is there any other workaround you would recommend that does not require modifying the log source or manually editing AMA configuration files?

Thank you
Suchitra Suregaunkar 15,130 Reputation points Microsoft External Staff Moderator

2026-06-11T17:46:24.94+00:00
Hello Minas Stam,

Thank you for the detailed follow-up and for thoroughly testing all three approaches. I understand the constraints you're working under, the log source cannot be modified, manual parsers.conf edits are overwritten by AMA on config refresh, and custom regex properties like recordStartTimestampCustomFormat are not supported at the ARM/DCR level.

To address your questions directly:

1. Is there a planned update to fix the timestamp-format2 regex?

As of the latest AMA versions (Windows 1.39.0 / Linux 1.38.x — October 2025 release), there is no documented fix or planned update specifically for supporting single-digit hours in the M/D/YYYY HH:MM:SS AM/PM format. The recordStartTimestampFormat values supported in the DCR are predefined and do not accept custom regex patterns. I recommend submitting a feature request via the Azure feedback portal https://feedbackportal.microsoft.com/feedback/post/ under Azure Core CSS to request native support for single-digit hours (H:MM:SS) in the timestamp parser. This helps the product team prioritize the fix.

2. Recommended workarounds that do not require modifying the log source or manually editing AMA config:

Since modifying the log source or AMA config files is not viable, here are two alternative approaches:

Pre-processing with a lightweight script: Deploy a scheduled task (Windows) or cron job (Linux) that reads the original log file, zero-pads the single-digit hours using a simple regex replacement (\b(\d): → 0$1:), and writes the output to a separate monitored directory. Configure the AMA DCR filePatterns to point to the processed file location instead of the original. This approach does not alter the source application or the AMA configuration files, it simply creates a normalized copy that AMA can reliably parse.

Use the Logs Ingestion API: Instead of relying on AMA file-based collection for this specific log, use a PowerShell or Python script to read the log file, parse records correctly with your own regex, and send them to the Log Analytics workspace via the https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-ingestion-api-overview. This gives you full control over timestamp parsing and multiline record boundaries. The script can be scheduled via Azure Automation, a scheduled task, or cron.

Both approaches work within the current AMA limitations without requiring changes to the log-generating application or unsupported modifications to AMA internals.

Thanks,

Suchitra.