Amazon RedShift vs RedShift Spectrum vs Amazon EMR - A comparison - AWS Certification Cheat Sheet

Let’s get a quick overview of the big data options in AWS - Amazon RedShift vs RedShift Spectrum vs Amazon EMR. We will look at important certification questions regarding Amazon RedShift vs RedShift Spectrum vs Amazon EMR.

You will learn

  • What are big data options in AWS?
  • How do you choose - Amazon RedShift vs RedShift Spectrum vs Amazon EMR?
  • Comparison - Amazon RedShift vs RedShift Spectrum vs Amazon EMR

Table of Contents

Amazon Redshift

Redshift is a relational database ( tables and relationships).

What is the need for another relational database?

RDS is optimized for online transaction processing. It is optimized to provide a balance between both reads and write operations. OLAP workloads have exponentially larger reads on the databases compared to writes.

The questions are:

  • Can we use a different approach to design the database?
  • How about creating a cluster and splitting the execution of the same query across several nodes?

Redshift is a petabyte-scale distributed data ware house based on PostgreSQL

Three important characteristics of Redshift:

  • Massively parallel processing (MPP) - storage and processing can be split across multiple nodes
  • Columnar data storage
  • High data compression

Amazon Redshift Spectrum

Amazon Redshift Spectrum helps you run SQL queries against datasets in Amazon S3. It does need for any intermediate data stores.

Here are some of the important characteristics:

  • Auto scales based on your queries
  • Scale storage and compute independently
  • Metadata defined in Amazon Redshift
    • Avro, CSV, Ion, JSON, ORC, Parquet formats supported
  • Eliminate expensive data transfers from S3 to data warehousing solutions (Cost effective)
  • Integrates with Amazon Athena
  • Query against Amazon EMR (as well)

Amazon EMR - Elastic MapReduce

Amazon EMR is a managed Hadoop service with high availability and durability.

Here are some of the important characteristics:

  • EMR gives access to underlying OS => You can SSH into it
  • Important tools in Hadoop eco system are natively supported:
    • Examples: Pig, Hive, Spark or Presto
  • Install others using bootstrap actions

Here are some of the important Use cases of Amazon EMR:

  • Log processing for insights
  • Click stream analysis for advertisers
  • Genomic and life science dataset processing

Amazon EMR - Storage Types

The table below shows the different Storage Types in EMR and the differences between them.

Feature Hadoop Distributed File System (HDFS) EMR File System (EMRFS)
Standard for Hadoop X
Data Storage EBS or instance storage S3
Data Survival on cluster shutdown Yes for EBS. No for Instance Storage Yes
Persistent Clusters running 24 X 7 analysis ✓ (low latency on instance storage)  
Transient Clusters running Infrequent big data jobs   ✓(Run MapReduce jobs against S3 bucket)

Comparison and Certification Questions - Amazon Redshift and EMR

When do you use EMR? When do you use Redshift?

Let’s look at a quick table:

Alternative Scenario
Amazon EMR For big data frameworks like Apache Spark, Hadoop, Presto, or Hbase to do large scale data processing that needs high customization
For example: machine learning, graph analytics etc
Amazon Redshift Run complex queries against data warehouse - housing structured and unstructured data pulled in from a variety of sources
Amazon Redshift Spectrum Run queries directly against S3 without worrying about loading entire data from S3 into a data warehouse
Amazon Athena Quick ad-hoc queries without worrying about provisioning a compute cluster (serverless)
Amazon Redshift Spectrum is recommended if you are executing queries frequently against structured data

Certification - Recommended Reading

Cloud Certifications - AWS, Azure and Google Cloud - Top 8 FAQ For Me

Google Cloud For Beginners - How to choose a Database Service?

Teaching Cloud Certifications - Top 6 Learnings

Google Cloud For Beginners - How to choose a Compute Service?

Important Kubernetes Concepts Made Easy

Introduction to Google Cloud - For AWS Professionals

Getting Started with Docker - 5 Easy Steps

GCP PubSub - GCP Certification Cheat Sheet

GCP IAM - GCP Certification Cheat Sheet

GCP App Engine - GCP Certification Cheat Sheet


Congratulations on reading this article!

Wondering what to learn next?


Keep Learning Every Day

Check Out Our Amazing ROADMAPS