Data Engineer

Handle interruptions

Configure exception handling

Upsert data

Replay archived stream data

Trigger batches

Handle failed batch loads

Manage data pipelines in Azure Data Factory or Azure Synapse Pipelines

Validate batch loads

Schedule data pipelines in Data Factory or Azure Synapse Pipelines

Implement version control for pipeline artifacts

Manage Spark jobs in a pipeline

Implement data masking

Encrypt data at rest and in motion

Implement row-level and column-level security

Implement Azure role-based access control (RBAC)

Implement POSIX-like access control lists (ACLs) for Data Lake Storage Gen2
1 2 3 4 5 6