ADF pipeline getting triggered for old blob event

Question

ADF pipeline getting triggered for old blob event

JigarkumarPatel-5373 46

Hi All,

I have a storage event trigger attached to a pipeline in ADF. On 4-jun-2026 between 4:00 and 4:05 PM UTC around 70k blob json files were written to the ADLS2 path monitored by storage event trigger.

Due to this 10000 runs of pipeline started as 10000 is the limit

User's image

I then stopped the storage event trigger in ADF and cancelled all the runs.

After this upon restarting the storage event trigger, another set of 10000 runs started and this runs were from the older blob part of 70k blobs already received. I have verified no new blobs were added after restarting the storage event trigger.

I then stopped the storage event trigger in ADF and cancelled all the runs.

Now I again restarted the storage event trigger on 23-jun-2026 i.e after 9 days and immediately 400 runs of the pipeline started all for older blobs of 70k events.

I am not able to understand what is going wrong here ?

This is the output of KQL query proving that ADF runs were started for older blob event

User's image

Can someone help me understand what is happing here ?

JigarkumarPatel-5373 46 Reputation points

2026-06-25T14:04:02.96+00:00

Hi @Manoj Kumar Boyini ,

Thank you for the reply.

We indeed stopped the current trigger and create new trigger to process new events.

Is there a way to check how many events are queued ?

Also is there documentation confirming this behaviour as the https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers explain about how to create trigger.
Manoj Kumar Boyini 17,950 Reputation points Microsoft External Staff Moderator

2026-06-25T16:50:17.09+00:00

Hi @JigarkumarPatel-5373

Regarding the queued events, Azure Data Factory does not currently expose a metric or interface that shows the exact number of queued Storage Event Trigger events. The closest approach is to use Azure Monitor / Log Analytics to review TriggerRuns and compare the original event timestamp with the trigger execution timestamp to identify delayed processing. PipelineRuns with a status of "Queued" can also be reviewed; however, this reflects queued pipeline executions rather than the underlying storage events themselves.

Regarding the documented behavior, the relevant guidance is available in the Event-based Trigger section of the Azure Data Factory trigger documentation. The documented behavior states that if an event-based trigger is stopped and started again, it resumes the previous trigger pattern, which can result in older events being processed. To start without historical events, the recommended approach is to delete the existing event trigger and create a new one. A newly created trigger starts without prior trigger history.

Reference: https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers#event-based-trigger

Based on the timestamps you shared earlier, the behavior appears consistent with previously queued events from the original blob event burst being processed after the trigger was reactivated.

Please let us know if you have any further questions.
Manoj Kumar Boyini 17,950 Reputation points Microsoft External Staff Moderator

2026-06-26T10:47:23.8966667+00:00

Hi @JigarkumarPatel-5373

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

Answer accepted by question author

Manoj Kumar Boyini 17,950 Microsoft External Staff Moderator

Hi @JigarkumarPatel-5373

Based on the trigger behavior and the timestamps shown in your query results, this appears to be consistent with previously queued Storage Event Trigger events being processed after the trigger was re-enabled.

The blob events were originally generated on 4-Jun-2026 when approximately 70,000 files were written to the monitored path. Since the number of generated events exceeded the number of pipeline runs that could be started immediately, a backlog of events was created.

When the trigger was stopped and later restarted, the remaining queued events were not automatically discarded. As a result, ADF continued processing events that had originally been generated on 4-Jun, which explains why pipeline runs were started on 23-Jun even though no new blobs had been added.

The KQL output supports this interpretation, as the eventTime corresponds to the original blob creation date while the trigger execution occurred much later.

If the intention is to ignore historical queued events and process only new files, the recommended approach is to stop the current trigger and create a new trigger for future events, as previously queued events cannot be purged from the existing trigger.

Please let us know if you have any questions.

References:
https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers

0 comments

1 additional answer

Your answer

JigarkumarPatel-5373 46 Reputation points

2026-06-25T14:04:02.96+00:00

Hi @Manoj Kumar Boyini ,

Thank you for the reply.

We indeed stopped the current trigger and create new trigger to process new events.

Is there a way to check how many events are queued ?

Also is there documentation confirming this behaviour as the https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers explain about how to create trigger.
Manoj Kumar Boyini 17,950 Reputation points Microsoft External Staff Moderator

2026-06-25T16:50:17.09+00:00

Hi @JigarkumarPatel-5373

Regarding the queued events, Azure Data Factory does not currently expose a metric or interface that shows the exact number of queued Storage Event Trigger events. The closest approach is to use Azure Monitor / Log Analytics to review TriggerRuns and compare the original event timestamp with the trigger execution timestamp to identify delayed processing. PipelineRuns with a status of "Queued" can also be reviewed; however, this reflects queued pipeline executions rather than the underlying storage events themselves.

Regarding the documented behavior, the relevant guidance is available in the Event-based Trigger section of the Azure Data Factory trigger documentation. The documented behavior states that if an event-based trigger is stopped and started again, it resumes the previous trigger pattern, which can result in older events being processed. To start without historical events, the recommended approach is to delete the existing event trigger and create a new one. A newly created trigger starts without prior trigger history.

Reference: https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers#event-based-trigger

Based on the timestamps you shared earlier, the behavior appears consistent with previously queued events from the original blob event burst being processed after the trigger was reactivated.

Please let us know if you have any further questions.
Manoj Kumar Boyini 17,950 Reputation points Microsoft External Staff Moderator

2026-06-26T10:47:23.8966667+00:00

Hi @JigarkumarPatel-5373

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

Answer 1

JigarkumarPatel-5373 46

Hi @Manoj Kumar Boyini

Thanks for your reply. My query is resolved by the information provided by you.

0 comments