Looking for a Cloud Data Engineer that has proven Azure Cloud skills based around the design and development of multiple data pipelines going between legacy on premise and cloud environments. They will leverage their coding skills to enable multiple sources to move data through an Azure Data Lake Storage Gen2, Azure Synapse Analytics environments using tools such as Talend cloud and more.Responsibilities
- Uses Talend cloud to create multiple complex pipelines and activities using both Azure and On-Prem data stores for full and incremental data loads into a Azure Synapse Analytics.
- Establishes data pipelines with automated batch, micro-batch, and incremental data refreshes.
- Migrates on premise applications/solutions into the cloud
- Provisions, deploys, and scales Azure resources.
- Creates technical design documentation in adherence to business, source system, target system, and solution architecture requirements.
- 2+ years recent experience developing and implementing Data cloud-based solutions in a Microsoft Azure environment
- Experience managing Azure Data Lakes (ADLS) and Azure Synapse Analytics and an understanding of how to integrate with other Azure Services.
- Experience with Python or Spark.
- Experience with JSON, API calls.
- Experience with Data warehousing technologies such Azure Synapse Analytics /Azure SQL DW, or Redshift/Snowflake,
- Experience using SQL, T-SQL
- Experience with Azure DevOps, Azure DevOps Git
Nice to have
- Experience with message brokers such as Event Hubs or Kafka
- Experience with Azure Stream Analytics, Azure Functions, Azure Data Factory