Migrating databases in AWS - Data Pipeline vs DMS vs SCT - AWS Certification

Let’s get a quick overview of different options in migrating databases from/to AWS - Data Pipeline vs Database Migration Service DMS vs Schema Conversion Tool SCT.

You will learn

  • What is different options in migrating databases from/to AWS?
  • Comparison - Data Pipeline vs Database Migration Service DMS vs Schema Conversion Tool SCT
  • When to use - Data Pipeline vs Database Migration Service DMS vs Schema Conversion Tool SCT

Table of Contents

AWS Certification - 25 PDF Cheat Sheets + Free Course

Each cheat sheet contains:

  • FAQs and Tutorials with 5-20 slides
  • Notes to quickly review and prepare for certification exam
  • Certification Exam Tips
  • Certification and Interview Scenario Questions

AWS Data Pipeline

AWS Data Pipeline helps you to process and move data (ETL) between S3, RDS, DynamoDB, EMR, On-premise data sources.

Here are few important things to remember:

  • AWS Data Pipeline can create complex data processing workloads that are fault tolerant, repeatable, and highly available
  • AWS Data Pipeline launches required resources and tear them down after execution
  • REMEMBER : AWS Data Pipeline is NOT for streaming data!

AWS Database Migration Service

AWS Database Migration Service is used to migrate databases to AWS while keeping source databases operational.

Two types of migrations:

  • Homogeneous Migrations (ex: Oracle to Oracle)
  • Heterogeneous Migrations (ex: Oracle to Amazon Aurora, MySQL to Amazon Aurora)

Here are some of the important characteristics:

  • Free for first 6 months when migrating to Aurora, Redshift or DynamoDB
  • (AFTER MIGRATION) You can keep databases in sync and pick right moment to switch

Here are some the important use cases:

  • Consolidate multiple databases into a single target database
  • Continuous Data Replication can be used for Disaster Recovery

AWS Schema Conversion Tool

AWS Schema Conversion Tool is used to migrate data from commercial databases and data warehouses to open source or AWS services.

AWS Schema Conversion Tool is preferred option for migrating data warehouse data to Amazon Redshift.

You can migrate database schema (views, stored procedures, and functions) to compatible targets.

Here are some of the important features:

  • SCT assessment report
    • Analyze a database to determine the conversion complexity
  • Update source code (update embedded SQL in code)
  • Fan-in (multiple sources - single target)
  • Fan-out (single source - multiple targets)

Database Migration Service VS Schema Conversion Tool

Remember that SCT is part of DMS service.

Here are some of the important recommendations:

  • DMS is preferred for homogeneous migrations
  • SCT is preferred when schema conversion are involved
  • DMS is for smaller workloads (less than 10 TB)
  • SCT preferred for large data warehouse workloads
    • Prefer SCT for migrations to Amazon Redshift
  • Only DMS provides continuous data replication after migration


Certification - Recommended Reading

AWS Security Groups and Network Access Control List - A Comparison

AWS VPN vs AWS Direct Connect vs Software VPN vs CloudHub - AWS Certification

VPC and Subnet Route Tables - Routing in AWS

Moving Data between AWS and On-premises - Snowball vs Snowmobile vs DataSync

AWS Storage Gateway - File vs Tape vs Volume - Stored vs Cached

Security Groups vs NACL - A Comparison

Public Subnet vs Private Subnet - Routing and Internet Gateway

Private Subnets - NAT Gateway vs NAT Instance

VPC Endpoints - Gateway vs Interface, VPC Peering and VPC Flow Logs

Virtual Private Cloud and Subnet Fundamentals - VPC

Free Videos - Get Started with AWS Certification

Complete Course - AWS Certified Solutions Architect Associate

Get our amazing course pursued by thousands of learners

  • 400+ Lectures
  • 28 hours on-demand video
  • 18 articles
  • 2 downloadable resources
  • Full lifetime access