Ways to Represent Data

What it is

Data is any recorded fact: a price, a name, a photo, a sensor reading. The exam sorts data into three shapes by how much structure it has: structured, semi structured and unstructured.

The shape decides which store you pick and how you query it, so this is the first idea to master.

How it works

Structured data

Structured data follows a fixed schema. Every row has the same columns, and every column has a data type such as integer, text or date. The natural home is a relational database table, and you query it with SQL.

A CSV file is also treated as structured when each line holds the same fields in the same order.

Semi structured data

Semi structured data has some organisation but no fixed table layout. Each item carries its own labels, so one record can have fields another lacks. Common forms are JSON documents, XML files, key value pairs and graphs.

{
  "id": 101,
  "name": "Ana Silva",
  "phones": ["+351 21 000 0000"],
  "address": { "city": "Lisbon" }
}

The same data as XML uses opening and closing tags, and an attribute such as id="101" sits inside the tag itself.

<customer id="101"><name>Ana Silva</name><city>Lisbon</city></customer>

Because the structure travels with the data, it is called self describing. You can add a new field to one record without altering any other.

Unstructured data

Unstructured data has no predefined model. Examples are documents, PDF files, images, video, audio and the free text of an email. You cannot place a photo into rows and columns, so it is kept as a file or a binary large object, called a blob.

To get meaning from it you use other tools, such as image recognition or text analysis, or you attach a little structured information to it, such as a title and a date.

Comparing the three shapes

A worked scenario

An online shop keeps three kinds of information. Orders and prices sit in tables with fixed columns, so that is structured data. Each product has a flexible set of attributes stored as a JSON document, so that is semi structured. Product photos and customer review videos are files with no internal schema, so they are unstructured. One shop therefore uses all three shapes at once.

Spotting the shape from a description

Metadata bridges the gap

A photo is unstructured, but it can carry metadata: the file name, the date taken, the camera and a few tags. That small amount of structured data lets you search and filter a large set of photos without analysing the pictures themselves. Blob storage works this way, with the content kept as an object and descriptive properties held beside it.

Moving between shapes

Data is often reshaped on its way to analysis. A nested JSON order can be flattened into rows for a table, and the text of a document can be analysed to pull out names and dates, turning unstructured content into structured fields. The reverse is also common: a relational row is exported as a JSON document for a web application. Knowing which shape you start with tells you which tool to use first.