Scribd
Upload
75 views

Azure Data Bricks & Factory

Uploaded by

vishwaraghu3025
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
75 views

Azure Data Bricks & Factory

Uploaded by

vishwaraghu3025
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Trainer : Deeksha Sharma

Email : [email protected]

Azure Storage & SQL:


(Required as per defining Source & destination for ADF):

• Different cloud concepts?


• Types of Clouds (Public, Private, Hybrid)
• What is Azure?
• IaaS, PaaS, SaaS
• Regions and Region Pairs
• What is Resource Groups & Subscriptions
• Different types of storage
• Access Tiers & Redundancy
• Different authentication & security option
• Understand Life cycle management
• Working with Azure Storage Containers
• Different types of azure sql & understand the use cases
• Different service tiers for sql
• Understand dynamic data masking
• Row level security
• Encryption & threat protection

Azure Data Bricks:

• Introduction to Azure Databricks


• Read and write data in Azure Databricks
• Data processing in Azure Databricks
• Work with DataFrames in Azure Databricks
• Platform architecture, security, and data protection in Azure Databricks
• Delta Lake
• Analyze streaming data and create workloads
• Create a data architecture
• Implement data pipelines for azure databricks
• Azure databricks for data warehouse
• Azure databricks for intrusion detection
• data ingestion and data processing techniques
• working with various note books like python, scala, spark etc.
• databricks and runtime ML
• using apache spark with .Net

Azure Data Factory:

• Azure Data Factory Architecture


• ADF Pipeline Design, Publish, Trigger
• ADF Architecture, Pipelines & ETL
• DIU : Data Integration Units; Concurrency
• Linked Service, Dataset & Activities
• Staging Data - Advantages and Pricing
• Polybase Indexes, Compression Options
• Mapping Data Flow, Wrangling Data Flow
• Incremental Loads
• Incremental Load Pipeline Design in ADF
• Working with Azure Storage Containers
• Pipeline Executions, Incremental Schedules
• Regular Schedules & Tumbiling Windows
• Binay Copy, Last Modified Date in Blob
• Pipeline Trigger Schedules, Modifications
• Incompatable Rows Skips, Fault Tolerance
• Incremental Loads with Mutliple Tables
• Stored Procedures, Loops in ADF Pipelines
• Configure ETL Sources, Pre-Copy Scripts
• Using @{item() with Dyanamic Connections
• Table_Schema for Column Mapping
• Writing Expressions For Dynamic Loads
• Staging and Performance for ADF Loads
• Mapping Data Flow in ADF
• Data Flow Task Creation in ADF Pipelines
• Transformation Editor and Parameters
• Comparing ADF Pipelines and Data Flow
• Debugging: ADF Managed Executions
• Apache Spark Clusters @ ADF Debugging
• Authoring Data Flow, Graph, Configuration
• Transformation Setting, Optimize, Inspect
• Conditional Split Transformation in ADF
• Pivot Transformation in Mapping Data Flow
• Pivot Column & Aggregation Functions
• Pivot Transformation, Pivot Settings
• Pivot Key Value, Enabling Null Values
• Pivoted Columns, Pattern, Optimize
• Column Prefix, Help Graphic, Metadata

You might also like