Identify AI Workloads · Topic 3 of 3
Extracting Information from Content
AI-901 information extraction: OCR and layout, fields with confidence scores, entity extraction, transcripts from audio, and scenes from video.
- 4 min
- OCR reads text from images. Layout analysis finds paragraphs, tables and structure.
- Field extraction returns named values from a schema, by extracting, classifying or generating.
- Confidence scores show how sure the system is, and grounding shows where a value came from.
What it is
Most business information is locked inside content that a database cannot read: scanned forms, photographs, recorded calls and video. Information extraction turns that content into structured data, such as a set of fields and values, that software can store and use.
Each kind of content needs its own technique, and a modern service can apply them together.
How it works
Extraction by content type
| Content | Techniques | What you get |
|---|---|---|
| Text | Entity detection, key phrase extraction, classification | Names, dates, amounts and topics |
| Documents and forms | OCR, layout analysis, field extraction | Text, tables, paragraphs and named fields such as an invoice total |
| Images | OCR, image description, object and chart recognition | Text in the image, a description, the type of chart |
| Audio | Speech recognition, summarisation, sentiment | A transcript, a summary and the mood of a call |
| Video | Transcription, scene or segment detection, visual description | A transcript and a description of each segment |
Optical character recognition and layout
OCR reads printed or handwritten text from an image or a scanned page. Layout analysis goes further by finding structure: paragraphs, headings, tables and figures, and the reading order. Good layout analysis keeps a table as a table, which makes later extraction far more accurate than a flat stream of text.
Field extraction
A field is a named value you want, such as InvoiceTotal, VendorName or DueDate. Extraction can work in three ways.
- Extract: copy a value as it appears in the content, such as a date on a receipt.
- Classify: choose from a set of categories, such as the sentiment of a call or the type of a chart.
- Generate: create a value from the content, such as a short summary of a conversation.
You describe the fields you want in a schema, and the service returns them as structured output, usually JSON.
Confidence scores and grounding
Each extracted value can carry a confidence score from 0 to 1 that says how sure the system is. High scores allow straight through processing without a person, and low scores can be sent for review. Grounding records where in the source each value came from, such as the region of the page, so a reviewer can check it quickly.
Audio and video
For audio, the key steps are transcription, which writes down what was said, then summarisation and sentiment, which condense and interpret it. A call centre can use these to find topics, mood and action items.
For video, the content is first segmented, which divides it into scenes or topics. Each segment can then have a transcript, a description of the people, places and actions, and a summary. That makes long videos searchable.
Document types
| Document | Typical fields |
|---|---|
| Invoice | Vendor, invoice number, date, line items, total |
| Receipt | Merchant, date, items, total |
| Identity document | Name, date of birth, document number, expiry |
| Contract | Parties, effective date, terms |
A worked scenario
A finance team receives thousands of supplier invoices as scanned PDFs. OCR and layout analysis read the pages, field extraction returns vendor, date and total for each, and a confidence score is attached to each value. Invoices where every field scores high are paid automatically, and the rest go to a clerk, who sees the source region highlighted for quick checking. The same team records its supplier calls. Transcription and summarisation turn each call into text and a short summary, and sentiment shows which calls were tense.
What to remember for the exam
- OCR reads text from images. Layout analysis finds paragraphs, tables and structure.
- Field extraction returns named values from a schema, by extracting, classifying or generating.
- Confidence scores show how sure the system is, and grounding shows where a value came from.
- Audio becomes a transcript, a summary and sentiment. Video is segmented into scenes with transcripts and descriptions.
When a question needs structured fields from a form, think field extraction. When it needs the words from a call, think transcription.
How the exam asks about this
- "Read the text from a scanned page." OCR.
- "Estimate how reliable an extracted value is." A confidence score.
- "Break a long video into meaningful sections." Segmentation.
- "Produce a transcript, a scene list and a summary of each recorded session." Video analysis.
Common mistakes
Using OCR alone and expecting tables. Layout analysis keeps the table structure.
Trusting every value equally. Low confidence scores deserve a human check.
Treating audio analysis as plain transcription. Summaries and sentiment come from analysing the transcript.
Using a speech service for a scanned form. A form needs OCR and field extraction.
Check yourself
What does layout analysis add to OCR?
It finds the structure of the page, such as paragraphs, tables and headings.
What does a confidence score tell you?
How sure the system is that an extracted value is correct, from 0 to 1.
How is a long video made searchable?
It is segmented into scenes, and each segment gets a transcript and a description.
Topic 3 of 3 in Identify AI Workloads
Test yourself on Identify AI Workloads
The full Microsoft AI-901 question bank has 511 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.