Book a demo

For full terms & conditions, please read our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Navigating Intelligent Document Processing (IDP) Tools: A Cheatsheet

Miranda Hartley
January 12, 2024

Intelligent document processing (IDP) is the automated capture and structuring of data from documents. In a world where one person generates 1.7MB of data per second, sophisticated data management tools – such as IDP – have evolved from a luxury afforded by the largest enterprises to a necessity for most firms in financial services.

Finding and implementing an IDP solution may seem daunting, not the least due to the influx of IDP vendors. As of 2023, Gartner estimates that over 60 IDP companies are delivering solutions. Even more, as the SaaS sales process can span weeks, we’d recommend asking the right questions at the initial demo.

In this article, we’ll break down the right questions to ask, enabling your company to unlock the multiple benefits of IDP.

1. Key Functionality

First, it’s best to determine the limits of an IDP tool’s functionality. Consider asking a vendor the following questions: 

1. Does the system provide confidence scores for each extracted data field? 

Confidence scores validate data quality. For each data point, a score indicates the likelihood that the AI has captured and attributed the data correctly.

2. Is the captured data automatically outputted in the required layout? 

Some IDP tools release unstructured data or data in fixed formats (such as CSV), which may not be immediately actionable and require manual conversion.

3. Is the service end-to-end? 

A fully end-to-end data extraction service will include creating and updating the existing AI extraction models to fit your company’s use case, adding a human-in-the-loop (HITL) system, implementing the model (including any post processing) and providing ongoing support.

2. Outcomes

Even if the tool you’re considering offers impressive functionality, it’s important to check that its quality extends to its outputs. Consider verifying a few fundamental facts before proceeding.

The system should learn from its mistakes.

If the system struggles to learn from incorrectly captured data, then it’s likely some kind of rule-bound OCR, not true AI. An IDP vendor should be able to prove the AI model’s learning capabilities during a demo.

Accuracy – under a managed service – should be guaranteed. 

IDP technology may be packaged as a managed service, where human analysts or annotators validate the captured data. If so, the vendor should be able to guarantee complete accuracy.

AI has advanced beyond the stage where 99% accuracy is acceptable. Therefore, it’s important to assess how IDP vendors can vouch for the product's data quality.

3. Testing the Vendor

So far, during the demo, the IDP company representative has supplied basic information. Now it’s time to ask them questions specific to your business’s use case.

The vendor should have experience deploying projects on all scales. 

Ask them to talk you through a case study matching your company’s use case. In particular, you might ask:

  1. What did you learn by working with [Customer X]? Is there anything you would have done differently?
  1. Has your solution changed since working with [this customer]? If so, how did you ensure the technology you initially provided was sustainable and scalable?

For managed services, their annotators/analysts must have marked experience with the required documents.

For example, you might ask:

  • What qualifications do your annotators possess?
  • How many [document types] have your annotators reviewed in total?
  • Do you have a dedicated team of annotators for [document type]? Or are all annotators trained across all document types?

In addition to assuring you that their annotators possess demonstrable experience with the document type(s) your business requires, it’s also important the IDP vendor answers your questions confidently. A well-informed and connected team is the only way a data extraction vendor can provide the level of service your business deserves.

4. Technical Questions

At this point, you’ll have acquired an overview of the IDP service. The next step is to explore the product’s technical details. Enquiring about these details will be useful on two levels. For one, it will allow you to check the robustness of their tool, and it will provide the vendor with a better understanding of your business’s use case.

Let’s explore some of the technical details you’ll want to discuss with the vendor.

The API’s capabilities should be concrete. 

Part of the API’s accessibility can be demonstrated through seamless integration with your company’s architecture. What do we mean by ‘seamless integration? Firstly, no IDP tool should require a lengthy configuration period. The tool should also be minimally disruptive to your company’s IT architecture.

A poorly implemented AI technology will inevitably cause issues downstream due to its slow performance or network and storage complications. Therefore, it’s important that a vendor’s API demonstrates robustness, which it can do by:

  • Being carefully and comprehensively documented, including a versioning strategy
  • Incorporating strong security protocols
  • Offering support options (e.g., a dedicated customer service team, and more)

Users should be able to annotate documents and train the model.

This capability distinguishes sophisticated intelligent document processing platforms from LLMs like GPT-4, which are extremely unwieldy to train.

The vendor must assure the confidentiality, integrity and availability of your data. 

When it comes to this requirement, there are no compromises.

Some of the top red flags to watch out for with a potential vendor include whether they:

  • Have an inadequate or non-existent incident response plan in the event a data breach occurs.
  •  Fail to show data governance policies that detail how they store and handle data.

5. Security & Data Privacy

Understandably, you may have reservations about letting a third party access your company’s data. Checking that the IDP vendor commits to data security is essential, so it’s good practice to check that the following are in place:

  • The vendor possesses a recognised security certificate. ISO27001 is the leading international certification for information security.
  • A secure understanding of and adherence to General Data Protection Regulation (GDPR) policies.
  • The vendor has performed a recent penetration test. A penetration test is a simulated cyberattack designed to identify the weaknesses of a system.
  • Data is encrypted at rest and in transit. Encryption ensures that the data is secure while stored and moved between platforms.
  • Data loss prevention (DLP) policies are in place. DLP tools monitor the movement of information, preventing unauthorised access to sensitive data.

6. Commercials & TCO

Once you’ve validated the technical features of the IDP tool, it’s time to negotiate a price. There are several pertinent questions that will determine whether the IDP platform’s pricing matches your project’s requirements, such as whether the vendor offers a volume discount. Let’s explore the others.

The accessibility of the pricing list

Transparent pricing – whether per page or by license – is essential to make an informed purchasing decision.

The tool’s setup costs

Some IDP vendors will charge a hefty upfront fee, which you’ll need to factor into the project’s costs.

Self-service vs. managed service

Self-service refers to the IDP tool itself, whereas a managed service will deploy a HITL system to validate the data before it is returned. Consider which is best suited to your company culture, your budget and your current IT architecture.

Summary & Next Steps

When you’re ready to book a discovery call with an IDP vendor, here are a few general tips:

  • Include as many different shareholders as possible on the initial demo call to avoid wasting your time repeating the demo in the future.
  • If possible, send your documents ahead so you can watch the IDP technology extract data from them in real time. Using your own documents tests whether the AI is well-trained and ready to deploy.
  • Ensure that you’ve already accessed the pricing, or they’ve committed to emailing the pricing sheet to you after the call to prevent miscommunication down the line.

In summary, though shopping for an IDP solution may feel like a tedious trawl through 60+ technologies, with an organised approach, you can find the right tool for your company’s requirements.

If you have any questions or comments, please feel free to reach out at hello@evolution.ai

You can also download a PDF of our IDP Buyer’s Guide here.

Share to LinkedIn