Power Automate Apply To Each Loops 20X Faster

Power Automate Apply To Each Loops 20X Faster

The Power Automate apply to each loop often runs slowly because it executes actions sequentially one-by-one. We can improve the performance of Apply To Each by using parallelism to have multiple threads running at the same time. Parallelism increases the speed of a flow and reduces completion time by 20x or more. In this article I will show you how to run the Power Automate apply to each action faster.

Table of Contents
• Setup The SharePoint ListRun Apply To Each Loop Actions One-At-A-TimeImprove Apply To Each Speed 20X With ParallelismProblems With Variables While Using Parallelism In LoopsReplace Set Variable In Apply To Each Action With Compose




Introduction: The Past Due Invoices Flow

A plumbing company uses a Power Automate flow to check if an invoice is past due and then update its status. The flow applies invoice updates concurrently instead of sequentially to speed up processing times.




Setup The SharePoint List

Create a new SharePoint List called Company Invoices with the following columns:

  • InvoiceNumber (number)
  • VendorName (single-line text)
  • InvoiceAmount (number)
  • IsPastDue (yes/no)



Populate the Company Invoices list with these items:

InvoiceNumberVendorNameInvoiceAmountIsPastDue
1001Wordify105No
1002Twitterwire350No
1003Bubblebox240No
1004Twitterlist335No
1005Skipstorm250No
1006Flashset165No
1007Chatterpoint110No
1008Kazio355No
1009Abata175No
1010Gabspot130No
1011Dabtype210No
1012Linkbridge335No
1013Gigazoom175No
1014Skidoo415No
1015Wikizz470No
1016Edgepulse200No
1017Viva465No
1018Vimbo500No
1019Mycat225No
1020Fliptune265No



Once setup the SharePoint list should look like this.




Run Apply To Each Loops Actions One-At-A-Time

We want to build a flow that gets all items in the Company Invoices SharePoint list and updates the yes/no column IsPastDue to equal “Yes.” The first flow we build will run the apply to each action sequentially one-by-one. Then we will improve the flow using parallelism to execute many apply to each actions at the same time.

Create the following flow in Power Automate. Set the IsPastDue column to Yes in the Update Item: Change Past Due To Yes action.



Test the flow to check how fast it runs. In this test the flow took 9s to complete.




Improve Apply To Each Speed 20X With Parallelism

The Apply To Each action runs very slowly in our current flow. We can improve it to run lighting-fast by using parallelism. Parallelism allows us to execute multiple threads at once. Enable parallelism for the Apply To Each action by opening its Settings.



Then turn on Concurrency control. The default degree of parallelism is 20 concurrent threads. If we wanted to increase it further, it would be a good idea to check the throttling limits for the Power Automate SharePoint connector. Too many concurrent API can cause the flow to run slower.

After selecting the degree of parallelism press Done.



Now test the flow to see how fast it runs. The flow takes only 2s to complete compared to the original time of 9s. An amazing improvement!




Problems With Variables While Using Parallelism In Loops

Let’s review another scenario to understand the limitations of parallelism and how to deal with them. In this flow we want to increment the InvoiceAmount column for each Company Invoice by 10 as a penalty for non-payment.



We have added an Initialize Variable step at the start of the flow and set varUpdatedAmount to 0. Then we included a Set Variable step inside the apply to each loop with the following code.

add(items('Apply_to_each:_Company_Invoice')?['InvoiceAmount'], 10)



When we create a Set variable step inside an apply to each loop Power Automate gives us a warning. Variables should not be used within a loop while parallelism is enabled. The danger is our variable would be set by multiple concurrent threads running at once and provide an incorrect result to the Update Item action.




Replace Set Variable In Apply To Each Action With Compose

How can we solve this problem? We do need to use variables within an apply to each action. Fortunately there is a workaround.

Remove the Initialize Variable from the start of the flow. Then replace the Set Variable action with a Compose action inside the Apply to Each loop.



The Compose action outputs its result as a text data type. Therefore, we must write a flow expression in the Update Item action InvoiceAmount field to change the type to a float. A float is a decimal number.

float(outputs('Compose:_Updated_Invoice_Amount'))



Now flow no longer shows a warning and has the desired result. Test the flow to make sure it still works.




Questions?

If you have any questions or feedback about Power Automate Apply To Each Loops 20X Faster please leave a message in the comments section below. You can post using your email address and are not required to create an account to join the discussion.

Matthew Devaney

Subscribe
Notify of
guest

9 Comments
Oldest
Newest
Inline Feedbacks
View all comments
Alex MacVean
1 year ago

Hi Matthew, as I understood it, the problem with using variables inside a Apply to Each loop in Power Automate is that it places a lock on the variable when it’s being used. This is done to prevent concurrent threads from changing the value out-of-sequence and thus providing inconsistent results (as you mention in this blog post).

So in fact, the result should be correct, but the issue is that you end up with a performance bottleneck – only one thread can access the variable at any given point in time and so it doesn’t matter how many concurrent threads you run, they’ll always be stuck waiting to use the variable one-by-one.

A very useful technique that I’ve implemented over the past few years is described by Pieter Veenstra here: The advanced Pieter’s method in Power Automate (sharepains.com). It takes advantage of undocumented ways the Power Automate platform actually functions behind the scenes to deliver exceptional performance gains in scenarios where you’d commonly use an Apply to Each loop.

Jason Templeton
Jason Templeton
1 year ago

Hi Matthew, Would there be any issues with adding the Concurrency contral when creating an excel table eg: adding in 1500 rows to an excel table with a flow run, would it still be able to add all the items and not miss anything if doing concurrently

Jason Templeton
Jason Templeton
1 year ago

Excellent Thanks Matthew!

Vishal
Vishal
1 year ago

Hi Matthew,
Now I am facing the below error after replicating your solution.
Flow save failed with code ‘InvalidTemplate’ and message ‘The template validation failed: ‘The repetition action(s) ‘Apply_to_each:_Sp_List_Name_Spain_Certificates_HC’ referenced by ‘inputs’ in action ‘Compose:_update_ID_Number’ are not defined in the template.’.’.

Joshma
Joshma
7 months ago

Hi Matthew, I am getting a Save conflict while trying to update sharepoint item within the loop. Do you know how we can solve that?

Valentin
Valentin
2 months ago

Hi, I am currently doing a power automate flow and I noticed when we do a compose outside de Apply Each and handwriting outputs(ComposeInApplyEach) it returns an array with all compose iteration output.
Is it good to use that methods to return messages to gain performance with no use of variables ?