DETAILED COURSE OUTLINE
Course 1. Data Engineering Foundations:Modern Data Warehouse & Data Lake
Data Sources-ERP systems, CRM systems. ETL Process, Data Storage, Data Ingestion, Data Processing - Hadoop, Spark. Architecture of Modern Data Warehouses, Cloud Data Warehouses, Architecture of Data Lakes, Key Technologies Data Warehouses and Data Lakes Tools - Amazon Redshift, AWS Lake Formation etc.
Course 2. Data Modeling
Introduction to Data Modeling, Data Modeling for Data Warehouses, Steps in Data Warehouse Modeling, Requirement Analysis - Understand business requirements and objectives.Conceptual Modeling, Logical Modeling, Physical Modeling. Data Modeling for Data Lakes, Steps in Data Lake Modeling. Tools for Data Modeling-ERwin Data Modeler, Microsoft Visio, Power Designer etc.
Course 3. Design Pattern of Data Pipeline
Introduction to Data Pipelines, Key Components of Data Pipelines, APIs, flat files, streaming sources. Data Storage, Data Orchestration, Data Monitoring. Common Data Pipeline Design Patterns - ETL (Extract, Transform, Load), Use Cases: Batch processing, data warehousing. Tools and Technologies for Data Pipelines. Lambda Architecture.
Course 4. On-Premise and Cloud Base Data engineering tool
Introduction to Data Engineering tool, On-Premise Data Engineering Tools- SSIS, Talend, Pentaho. Cloud-Based Tools-AWS Glue, AWS Lambda. Server-less compute for event-driven processing. Google Dataflow, Azure Data Factory. Comparison: On-Premise vs. Cloud-Based Tools based on Control, Scalability, Cost and Maintenance.
Course 5. SQL & Python Programming for Data Engineer
Introduction to SQL and Python for Data Engineer. Introduction to SQL Programming for Data Engineers. SQL Basics and Advances. Practical SQL Exercises. Python Programming for Data Engineers - Introduction to Python: Basic programming concepts, Data Types, Data dictionaries, Functions, Modules and Packages, libraries. Data Manipulation with Pandas etc
On
Course Features
Certificate of Completion from NED
Access to NED University resource