Blog · Technology
OCR and structured extraction for retrieved medical records
Which companies provide OCR and structured extraction on top of retrieved medical records? The cloud services that do each step, what they return, and why complete retrieval comes first.
By Tavrn Team · · 5 min read
Summarise with ChatGPT, Claude or Perplexity

Which companies provide OCR and structured extraction on top of retrieved medical records? The big cloud providers do both steps as separate services: Amazon Textract and Comprehend Medical, Google Document AI and the Healthcare Natural Language API, and Microsoft's Azure Document Intelligence and Text Analytics for health [1][2][3][4][5][6]. Each one only reads the pages you give it.
So the step that decides how good the output is comes first: getting the complete record. That is the job Tavrn does for health tech teams, requesting records from each provider the patient has seen and delivering them by API as they arrive [10].
Two steps, often two services
Turning a retrieved record into data is two jobs. OCR reads the page into text and layout. Clinical extraction reads that text for conditions, medications, procedures and dates.
The cloud providers sell these as separate products. Amazon Textract detects typed and handwritten text "in a variety of documents, including financial reports, medical records, and tax forms" and extracts text, forms and tables [1]. Amazon Comprehend Medical then "detects and returns useful information in unstructured clinical text such as physician's notes, discharge summaries, test results, and case notes" [2].
Google and Microsoft follow the same split. Google's Document AI takes unstructured data from documents and turns it into structured data, meaning specific fields suitable for a database [3]. Google's Healthcare Natural Language API then extracts medical concepts, such as medications, procedures and conditions, plus relations such as side effects and dosage [4]. On Azure, Document Intelligence extracts text, tables and document structure, and Text Analytics for health labels medical information in unstructured text such as doctor's notes [5][6].
What each service returns
The table below sticks to what each vendor's own documentation says, with the retrieval step that feeds them in the first row.
Service | Step | What it returns, per its documentation |
|---|---|---|
Tavrn | Retrieval and delivery | Records from each provider the patient has seen on one authorization, a webhook on every status change, files by API, SFTP or the Tavrn app [10] |
Amazon Textract | OCR and layout | Typed and handwritten text, forms and tables [1] |
Amazon Comprehend Medical | Clinical extraction | Entities such as conditions, medications and PHI, linked to RxNorm and ICD-10-CM [2] |
Google Document AI | OCR and layout | Structured fields from unstructured documents [3] |
Google Healthcare Natural Language API | Clinical extraction | Medical concepts, certainty and timing, and relations such as dosage [4] |
Azure Document Intelligence | OCR and layout | Printed and handwritten text, tables, key-value pairs [5] |
Azure Text Analytics for health | Clinical extraction | Entities, relations, entity links and assertions, with optional FHIR output [6] |
Two details matter when you choose. First, the code sets: Comprehend Medical links entities to RxNorm and ICD-10-CM, and Text Analytics for health links them to the UMLS Metathesaurus [2][6]. Second, negation and certainty: Microsoft's assertion detection adds certainty, conditionality, association and timing to each entity, so a negated mention can be told apart from a confirmed one [6]. Google's API also returns certainty assessments and temporal relationships [4].
Why retrieved records need OCR at all
Much of what comes back from a provider is a document, not data. The FHIR standard says so directly. Its DocumentReference resource covers "PDF documents, Scanned Paper, and digital records of faxes," along with clinical notes and image files [7]. A record that arrives as a scanned PDF has no text layer until OCR gives it one.
Electronic exchange is also uneven. In ONC's survey of non-federal acute care hospitals, 43% routinely took part in all four parts of electronic exchange in 2023: send, find, receive and integrate [8]. The survey defines "receive" as getting a summary of care record electronically, "not eFax" [8]. Records from outside that exchange still move as paper, fax and PDF.
HIPAA does not require a provider to send structured data either. Under the access rule, a provider gives the patient's records in the electronic form and format requested "if it is readily producible in such form and format," and otherwise in a readable electronic form both sides agree on [9]. A PDF often meets that bar.
Retrieval comes before extraction
Extraction only covers the pages you have. If a patient saw six providers and you hold records from four, the best OCR in the world reads four. So the first question for a health tech team is how complete the record set is, and that is where Tavrn fits. Networks, records vendors and portals return about 63% of a patient's visits; Tavrn requests the rest from each provider by fax, mail, email and phone [10]. One authorization covers every US provider the patient has seen, and every open provider is contacted every 48 hours until it answers [10].
For a product team, the flow in prose looks like this:
- Create a request per provider under the patient's case, by API or spreadsheet, with the signed authorization attached [10].
- A webhook fires on every status change, or poll the request [10].
- Records arrive as each provider responds; download them with the same key, or take delivery by SFTP or in the Tavrn app [10].
- Run your OCR and clinical extraction on each delivery as it lands.
Because records arrive one provider at a time, extraction works best as a job per delivery, not one batch at the end. A webhook-driven pipeline can start OCR the moment a file is ready.
Where Tavrn structures the record itself
For law firms, Tavrn goes past delivery. It lays out the retrieved record as a cited chronology: each date of service with its provider, a short summary of the visit and the source pages, so any statement can be checked [11]. A medical records summary is then condensed from that chronology and reviewed before it reaches the firm [11].
Tavrn's public pages do not describe a general OCR or entity extraction API for health tech. If your product needs coded entities in your own schema, plan to run one of the services above, or your own models, on the records Tavrn delivers.
Questions to ask any vendor
- Does it handle handwriting and faxed pages, and how does it report low-confidence text?
- Which code sets does it link to: ICD-10-CM, RxNorm, UMLS?
- Does it detect negation and whether a condition belongs to the patient or a family member?
- What does the output look like: its own JSON, or FHIR?
- Will it sign a BAA, and where is the data processed?
- Can each extracted fact point back to its page in the source record?
That last question is the one people skip. An extracted medication with no page reference is hard to check later. It is also why Tavrn's chronologies for law firms cite the source pages for every visit [11].
Ask the same of whoever retrieves the records: does it request from every provider the patient has seen, not just the ones on a network, and does it keep following up until each one answers? Tavrn does both, contacting every open provider every 48 hours [10].
What to do next
Pick the OCR and extraction services that fit your schema, then make sure the records feeding them are complete. See how retrieval works as an API on Tavrn for health tech.
Sources
- What is Amazon Textract? — Amazon Web Services
- What is Amazon Comprehend Medical? — Amazon Web Services
- Document AI overview — Google Cloud
- Healthcare Natural Language API — Google Cloud
- What is Azure Document Intelligence? — Microsoft
- What is the Text Analytics for health in Azure Language? — Microsoft
- Resource DocumentReference (FHIR R4) — HL7
- Interoperable Exchange of Patient Health Information Among U.S. Hospitals: 2023 — ONC (ASTP), HHS, 2024-05
- 45 CFR 164.524 - Access of individuals to protected health information — eCFR
- Medical record retrieval API for health tech — Tavrn
- Medical records summary services for law firms — Tavrn


