Power Automate Apply To Each Loops 20X Faster
The Power Automate apply to each loop often runs slowly because it executes actions sequentially one-by-one. We can improve the performance of Apply To Each by using parallelism to have multiple threads running at the same time. Parallelism increases the speed of a flow and reduces completion time by 20x or more. In this article I will show you how to run the Power Automate apply to each action faster.
Table of Contents
• Setup The SharePoint List
• Run Apply To Each Loop Actions One-At-A-Time
• Improve Apply To Each Speed 20X With Parallelism
• Problems With Variables While Using Parallelism In Loops
• Replace Set Variable In Apply To Each Action With Compose
Introduction: The Past Due Invoices Flow
A plumbing company uses a Power Automate flow to check if an invoice is past due and then update its status. The flow applies invoice updates concurrently instead of sequentially to speed up processing times.
Setup The SharePoint List
Create a new SharePoint List called Company Invoices with the following columns:
- InvoiceNumber (number)
- VendorName (single-line text)
- InvoiceAmount (number)
- IsPastDue (yes/no)
Populate the Company Invoices list with these items:
InvoiceNumber | VendorName | InvoiceAmount | IsPastDue |
1001 | Wordify | 105 | No |
1002 | Twitterwire | 350 | No |
1003 | Bubblebox | 240 | No |
1004 | Twitterlist | 335 | No |
1005 | Skipstorm | 250 | No |
1006 | Flashset | 165 | No |
1007 | Chatterpoint | 110 | No |
1008 | Kazio | 355 | No |
1009 | Abata | 175 | No |
1010 | Gabspot | 130 | No |
1011 | Dabtype | 210 | No |
1012 | Linkbridge | 335 | No |
1013 | Gigazoom | 175 | No |
1014 | Skidoo | 415 | No |
1015 | Wikizz | 470 | No |
1016 | Edgepulse | 200 | No |
1017 | Viva | 465 | No |
1018 | Vimbo | 500 | No |
1019 | Mycat | 225 | No |
1020 | Fliptune | 265 | No |
Once setup the SharePoint list should look like this.
Run Apply To Each Loops Actions One-At-A-Time
We want to build a flow that gets all items in the Company Invoices SharePoint list and updates the yes/no column IsPastDue to equal “Yes.” The first flow we build will run the apply to each action sequentially one-by-one. Then we will improve the flow using parallelism to execute many apply to each actions at the same time.
Create the following flow in Power Automate. Set the IsPastDue column to Yes in the Update Item: Change Past Due To Yes action.
Test the flow to check how fast it runs. In this test the flow took 9s to complete.
Improve Apply To Each Speed 20X With Parallelism
The Apply To Each action runs very slowly in our current flow. We can improve it to run lighting-fast by using parallelism. Parallelism allows us to execute multiple threads at once. Enable parallelism for the Apply To Each action by opening its Settings.
Then turn on Concurrency control. The default degree of parallelism is 20 concurrent threads. If we wanted to increase it further, it would be a good idea to check the throttling limits for the Power Automate SharePoint connector. Too many concurrent API can cause the flow to run slower.
After selecting the degree of parallelism press Done.
Now test the flow to see how fast it runs. The flow takes only 2s to complete compared to the original time of 9s. An amazing improvement!
Problems With Variables While Using Parallelism In Loops
Let’s review another scenario to understand the limitations of parallelism and how to deal with them. In this flow we want to increment the InvoiceAmount column for each Company Invoice by 10 as a penalty for non-payment.
We have added an Initialize Variable step at the start of the flow and set varUpdatedAmount to 0. Then we included a Set Variable step inside the apply to each loop with the following code.
add(items('Apply_to_each:_Company_Invoice')?['InvoiceAmount'], 10)
When we create a Set variable step inside an apply to each loop Power Automate gives us a warning. Variables should not be used within a loop while parallelism is enabled. The danger is our variable would be set by multiple concurrent threads running at once and provide an incorrect result to the Update Item action.
Replace Set Variable In Apply To Each Action With Compose
How can we solve this problem? We do need to use variables within an apply to each action. Fortunately there is a workaround.
Remove the Initialize Variable from the start of the flow. Then replace the Set Variable action with a Compose action inside the Apply to Each loop.
The Compose action outputs its result as a text data type. Therefore, we must write a flow expression in the Update Item action InvoiceAmount field to change the type to a float. A float is a decimal number.
float(outputs('Compose:_Updated_Invoice_Amount'))
Now flow no longer shows a warning and has the desired result. Test the flow to make sure it still works.
Did You Enjoy This Article? 😺
Subscribe to get new Power Apps & Power Automate articles sent to your inbox each week for FREE
Questions?
If you have any questions or feedback about Power Automate Apply To Each Loops 20X Faster please leave a message in the comments section below. You can post using your email address and are not required to create an account to join the discussion.
Hi Matthew, as I understood it, the problem with using variables inside a Apply to Each loop in Power Automate is that it places a lock on the variable when it’s being used. This is done to prevent concurrent threads from changing the value out-of-sequence and thus providing inconsistent results (as you mention in this blog post).
So in fact, the result should be correct, but the issue is that you end up with a performance bottleneck – only one thread can access the variable at any given point in time and so it doesn’t matter how many concurrent threads you run, they’ll always be stuck waiting to use the variable one-by-one.
A very useful technique that I’ve implemented over the past few years is described by Pieter Veenstra here: The advanced Pieter’s method in Power Automate (sharepains.com). It takes advantage of undocumented ways the Power Automate platform actually functions behind the scenes to deliver exceptional performance gains in scenarios where you’d commonly use an Apply to Each loop.
Alex,
This is a great comment. I am going to update my article to mention “locking the variable” and restricting to 1 concurrent thread in that scneario.
I also enjoy Pieter’s method in Power Automate. It’s a much better way to build arrays than “append to a variable” 🙂
Appreciate the comment. Thank again for sharing.
Hi Matthew, Would there be any issues with adding the Concurrency contral when creating an excel table eg: adding in 1500 rows to an excel table with a flow run, would it still be able to add all the items and not miss anything if doing concurrently
Jason,
There are two things you will want to keep in mind when using concurrency with Excel:
1. There is a limit of 100 API calls per minute for Excel. Set your degrees of parallelism at a value that is unlikely to surpass this. (It’s normal to deal with this while using concurrency).
2. The rows will not populate in the expected order 1, 2, 3, 4, 5. If this is important, you will need to sort the rows after they are populated in Excel using some subsequent action.
Excel Connector documentation:
https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/
Excellent Thanks Matthew!
Hi Matthew,
Now I am facing the below error after replicating your solution.
Flow save failed with code ‘InvalidTemplate’ and message ‘The template validation failed: ‘The repetition action(s) ‘Apply_to_each:_Sp_List_Name_Spain_Certificates_HC’ referenced by ‘inputs’ in action ‘Compose:_update_ID_Number’ are not defined in the template.’.’.
Vishal,
Try to recreate my tutorial exactly as shown and then apply it to your situation. It will work!
I think you are likely mistyping an action name in your expressions.
Hi Matthew, I am getting a Save conflict while trying to update sharepoint item within the loop. Do you know how we can solve that?
Hi, I am currently doing a power automate flow and I noticed when we do a compose outside de Apply Each and handwriting outputs(ComposeInApplyEach) it returns an array with all compose iteration output.
Is it good to use that methods to return messages to gain performance with no use of variables ?