Migrating databases in AWS - Data Pipeline vs DMS vs SCT - AWS Certification

Let’s get a quick overview of different options in migrating databases from/to AWS - Data Pipeline vs Database Migration Service DMS vs Schema Conversion Tool SCT.

You will learn

  • What is different options in migrating databases from/to AWS?
  • Comparison - Data Pipeline vs Database Migration Service DMS vs Schema Conversion Tool SCT
  • When to use - Data Pipeline vs Database Migration Service DMS vs Schema Conversion Tool SCT

Table of Contents

AWS Certification - 25 PDF Cheat Sheets + Free Course

Each cheat sheet contains:

  • FAQs and Tutorials with 5-20 slides
  • Notes to quickly review and prepare for certification exam
  • Certification Exam Tips
  • Certification and Interview Scenario Questions

AWS Data Pipeline

AWS Data Pipeline helps you to process and move data (ETL) between S3, RDS, DynamoDB, EMR, On-premise data sources.

Here are few important things to remember:

  • AWS Data Pipeline can create complex data processing workloads that are fault tolerant, repeatable, and highly available
  • AWS Data Pipeline launches required resources and tear them down after execution
  • REMEMBER : AWS Data Pipeline is NOT for streaming data!

AWS Database Migration Service

AWS Database Migration Service is used to migrate databases to AWS while keeping source databases operational.

Two types of migrations:

  • Homogeneous Migrations (ex: Oracle to Oracle)
  • Heterogeneous Migrations (ex: Oracle to Amazon Aurora, MySQL to Amazon Aurora)

Here are some of the important characteristics:

  • Free for first 6 months when migrating to Aurora, Redshift or DynamoDB
  • (AFTER MIGRATION) You can keep databases in sync and pick right moment to switch

Here are some the important use cases:

  • Consolidate multiple databases into a single target database
  • Continuous Data Replication can be used for Disaster Recovery

AWS Schema Conversion Tool

AWS Schema Conversion Tool is used to migrate data from commercial databases and data warehouses to open source or AWS services.

AWS Schema Conversion Tool is preferred option for migrating data warehouse data to Amazon Redshift.

You can migrate database schema (views, stored procedures, and functions) to compatible targets.

Here are some of the important features:

  • SCT assessment report
    • Analyze a database to determine the conversion complexity
  • Update source code (update embedded SQL in code)
  • Fan-in (multiple sources - single target)
  • Fan-out (single source - multiple targets)

Database Migration Service VS Schema Conversion Tool

Remember that SCT is part of DMS service.

Here are some of the important recommendations:

  • DMS is preferred for homogeneous migrations
  • SCT is preferred when schema conversion are involved
  • DMS is for smaller workloads (less than 10 TB)
  • SCT preferred for large data warehouse workloads
    • Prefer SCT for migrations to Amazon Redshift
  • Only DMS provides continuous data replication after migration


Certification - Recommended Reading

Amazon RDS - Relational Database Service and Amazon Aurora

Multi-AZ vs Multi-Region vs Read replicas - Amazon RDS

Database Primer - Availability vs Durability vs RTO vs RPO vs Consistency

Amazon RedShift vs RedShift Spectrum vs Amazon EMR - A comparison

AWS Data Lakes - Simplified Big Data Solutions

Amazon RedShift - Big Data in AWS

Amazon Cognito - User Pools, Identity Pools

Amazon S3 Storage Classes - Standard vs Standard-IA vs One Zone vs Intelligent-Tiering vs Glacier

Amazon S3 Glacier - vs Amazon S3

Choosing AWS Databases - RDS vs DynamoDB vs Redshift

Free Videos - Get Started with AWS Certification

Complete Course - AWS Certified Solutions Architect Associate

Get our amazing course pursued by thousands of learners

  • 400+ Lectures
  • 28 hours on-demand video
  • 18 articles
  • 2 downloadable resources
  • Full lifetime access