Extract Invoice Details With Power Automate And AI Builder

Power Automate can use AI Builder’s prebuilt model to effortlessly identify and extract the details from an invoice. Then additional actions can be added to write the invoice data to an Excel spreadsheet, a SharePoint document library or even use Power Automate Desktop to enter them directly into an Accounting system. There is great potential here to drastically reduce manual data-entry. In this article I will show you how to automatically extract invoices details using Power Automate & AI Builder.

Table of Contents
• Introduction: The Invoices Library With Automated Data Extraction
• Create A SharePoint Document Library
• Get An Invoice PDF File From The Document Library
• Extract Invoice Details Using AI Builder
• Test The Invoice Data Extraction Flow
• Determine Whether An Invoice Requires Manual Review With Confidence Scores
• Tag An Invoice As Needing Human Review In SharePoint
• Run The Flow To Test The Needs Human Review Feature>

Introduction: The Invoices Library With Automated Data Extraction

The Accounting department of a company receives vendor invoices and uploads them to a SharePoint document library. Once the invoice is uploaded AI Builder extracts the invoice details such as Invoice Number, Invoice Date, Vendor Name & Invoice Total and writes them to the file’s metadata in SharePoint.

Download a copy of the sample invoice to follow along with this tutorial.

Create A SharePoint Document Library

Go to SharePoint and create a new document library to store invoices named Vendor Invoices.

Add the following columns to the Document Library:

Get An Invoice PDF File From The Document Library

We want to extract the invoice details when a file is uploaded to the document library. To do this, create a new flow with an automated trigger. Select the SharePoint – When A File Is Created (Properties Only) trigger.

Point the SharePoint – When A File Is Created Trigger to the Vendor Invoices document library.

Then add the SharePoint – Get File Content action to the flow and target the Invoice PDF using the File Identifier.

Extract Invoice Details Using AI Builder

Now that we have the invoice’s file content we can use AI Builder’s pre-built invoices model to extract its details. Pre-built models do not require any training before using them in a flow. Add an AI Builder – Extract Information From Invoices action and load the File Content into the Invoice File field.

Note: Invoices file format can be JPG, PNG or PDF.

The AI Builder – Extract Information From Invoices action outputs all of the invoice details. We want to take those details any apply them to the metadata of the invoice document in SharePoint. Create a SharePoint – Update File Properties action populate its fields as shown below. Include the corresponding Confidence Score fields to indicate accuracy.

Test The Invoice Data Extraction Flow

At this point we can run the flow to test it’s ability to extract data from the invoice. Turn the flow on and upload an invoice to the document library. A few moments later the invoice details should appear beside it.

The AI Builder – Extract Information From Invoices action outputs more information than what we are currently tracking in the document library. To see the full set of data extracted, review the flow action outputs below.

Determine Whether An Invoice Requires Manual Review With Confidence Scores

A confidence score is a number between 0 and 1 that predicts whether the information extracted from the invoice is correct. There is a confidence score for each piece of data extracted from the invoice. Data with a low confidence score should be reviewed manually by a human. We can limit the amount of human-effort by setting a confidence score threshold and asking them to only review any data that scored below it. Not the entire invoice

Insert a new Data Operations – Compose action into the flow.

Then create an array of field names and confidence scores using the following code.

[
  {
    "Name": "Invoice ID",
    "Score": @{outputs('Extract_information_from_invoices')?['body/responsev2/predictionOutput/result/fields/invoiceId/confidence']}
  },
  {
    "Name": "Vendor Name",
    "Score": @{outputs('Extract_information_from_invoices')?['body/responsev2/predictionOutput/result/fields/vendorName/confidence']}
  },
  {
    "Name": "Invoice Date",
    "Score": @{outputs('Extract_information_from_invoices')?['body/responsev2/predictionOutput/result/fields/invoiceDate/confidence']}
  },
  {
    "Name": "Invoice Total",
    "Score": @{outputs('Extract_information_from_invoices')?['body/responsev2/predictionOutput/result/fields/invoiceTotal/confidence']}
  }
]

Now that we have an array we can filter it to determine which fields have low confidence scores. Add a Data Operations – Filter Array action to the flow. Filter for any confidence scores that are less than 75%.

Use this code in the left-side of the logical comparison to target the Score field of the array.

item()?['Score']

Once we have filtered the records we want to create a simple array that includes only the field names. The field names will tell the user which fields to review. We no longer need the confidence scores.

Insert a Data Operations – Select action into the flow. Switch the map field to text mode.

Use this code in the action’s Map field to build an array of field names.

item()?['Name']

Finally, we will convert the simple array to a text string of comma-separated field names.

Write this expression to create the comma-separated text string of field names.

join(body('Select:_Invoice_Field_Names'),', ')

Tag An Invoice As Needing Human Review In SharePoint

Any invoices having a field with a low confidence score (less than 75%) must be tagged for manual review. Add a Condition action to the flow.

Then use this code to determine if the manual review fields text string is empty.

empty(outputs('Compose:_Manual_Review_Fields'))

When the manual review fields text string is empty it means the invoice does not require any review. Fill-in the If Yes block of the condition as shown below. Set the invoice Status to Completed.

Otherwise, the invoice will require a human-review. Fill in the If no block of the condition as shown below. Set the invoice status to Needs Human Review and load the Manual Review Fields property with the outputs of the last Compose: Manual Review Fields action.

Run The Flow To Test The Needs Human Review Feature

We’re done building the flow. Upload an invoice to the SharePoint document library to test the Needs Human Review feature we built. If the invoice has any fields with a confidence score lower than 75% the Status Needs Human Review will appear.

Download a copy of the sample invoice to test the flow.

Once the invoice has been reviewed by a human they set they manually set the Status to Completed.

Questions?

If you have any questions or feedback about Extract Invoice Details With Power Automate And AI Builder please leave a message in the comments section below. You can post using your email address and are not required to create an account to join the discussion.