Exploring Data Transformation with Google Cloud · Topic 5 of 5
Analytics, Looker and Streaming Data
How Looker and BigQuery democratize data, why real-time streaming analytics matters, and what Pub/Sub, Dataflow and Managed Service for Apache Spark do.
- 4 min
- Includes a diagram
- Looker: governed, self-service BI with a semantic model; integrates data into workflows.
- Looker with BigQuery: real-time dashboards on large datasets.
- Streaming processes events as they happen; batch processes them later.
What it is
Business intelligence (BI) turns data into reports and dashboards people use to decide. Looker is Google Cloud's BI platform. Streaming analytics processes data as it arrives instead of in nightly batches. Google Cloud's main pipeline products are Pub/Sub (ingest messages), Dataflow (process streams and batches) and Managed Service for Apache Spark (run Spark and Hadoop jobs).
How it works
Looker democratizes data
Democratizing data means giving everyone in the organization access to trusted data, not only specialists. Looker does this in several ways:
| Looker feature | Business value |
|---|---|
| Semantic model (LookML) | Business metrics such as "revenue" are defined once, so every team gets the same answer |
| Self-service exploration | Business users explore data and build reports without writing SQL |
| Dashboards and reports | Share up-to-date views across teams |
| Embedded analytics and integrations | Put data into the apps and workflows people already use, and trigger actions |
| Connects to the data where it lives | Queries the database directly, such as BigQuery, instead of copying extracts |
Looker Studio is a separate, simpler tool for quick reports and dashboards from many data sources.
Looker with BigQuery
Analyzing BigQuery data in Looker combines BigQuery's ability to query very large datasets quickly with Looker's governed metrics and dashboards. The value for the business is real-time reports and dashboards on fresh data, one version of the truth, and the ability to integrate data into workflows, for example sending an alert or updating another system when a metric crosses a threshold.
When a question stresses "consistent definitions of metrics across the company", the answer is Looker's semantic model.
Why real-time streaming matters
Batch processing collects data and processes it later, for example overnight. Streaming processes each event within seconds of it happening.
| Use case | Why real time matters |
|---|---|
| Fraud detection | Block a suspicious payment before it completes |
| Inventory | Update stock as items sell to avoid overselling |
| Personalization | Recommend products while the customer is still on the site |
| Operations | Alert when a machine sensor shows a fault |
| Logistics | Reroute deliveries as traffic changes |
Modern customers expect instant responses, and many decisions lose value with every minute of delay.
The pipeline products
| Product | What it does | When to choose it |
|---|---|---|
| Pub/Sub | Global messaging service that ingests streams of events and delivers them to many subscribers | Collect events from apps and devices and decouple producers from consumers |
| Dataflow | Serverless processing for streaming and batch pipelines, based on Apache Beam | Transform, enrich and aggregate data in real time without managing servers |
| Managed Service for Apache Spark | Managed clusters for Apache Spark and Hadoop | Run existing Spark or Hadoop jobs in the cloud with little change |
| BigQuery | Serverless data warehouse; can receive streaming data | Store and analyze results with SQL |
Managed Service for Apache Spark was previously named Dataproc, a name you may still see in older material.
A worked scenario
An online retailer wants to stop fraudulent orders and show live sales to managers. Orders are published to Pub/Sub, Dataflow scores each order and enriches it in real time, results land in BigQuery, and Looker dashboards show live sales by region with governed metric definitions. The data science team still runs its existing Spark jobs on Managed Service for Apache Spark for weekly model training.
How the exam asks about this
- "Business users need self-service access to consistent metrics." Looker.
- "Ingest millions of events per second from devices." Pub/Sub.
- "Process streaming data without managing infrastructure." Dataflow.
- "Run existing Spark and Hadoop jobs." Managed Service for Apache Spark.
- "Why stream instead of batch?" Decisions such as fraud blocking need data now.
What to remember for the exam
- Looker: governed, self-service BI with a semantic model; integrates data into workflows.
- Looker with BigQuery: real-time dashboards on large datasets.
- Streaming processes events as they happen; batch processes them later.
- Pub/Sub ingests, Dataflow processes, BigQuery stores and analyzes, Looker presents.
- Managed Service for Apache Spark runs Spark and Hadoop jobs.
Pub/Sub transports messages; it does not transform or analyze them. Processing is Dataflow's job.
Common mistakes
Choosing Managed Service for Apache Spark for new serverless streaming. Dataflow is the serverless choice; Spark clusters suit existing Spark and Hadoop work.
Thinking dashboards alone democratize data. Shared metric definitions and self-service access are what make data trustworthy for everyone.
Using batch for fraud detection. By the next morning the money is gone.
Check yourself
Which product ingests streams of events and delivers them to many subscribers?
Pub/Sub.
What does Looker's semantic model give a business?
Metrics defined once, so every team sees the same numbers.
Which serverless product processes streaming and batch data pipelines?
Dataflow.
Topic 5 of 5 in Exploring Data Transformation with Google Cloud
Test yourself on Exploring Data Transformation with Google Cloud
The full Google Cloud Digital Leader question bank has 512 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Google Cloud. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.