Azure Data factory( Python RestApi, Pyspark) Experience 6+years Location Pan india immediate joiner
Orcapod
Bengaluru, Karnataka
•
On-site
On-Site
Full-Time
Bengaluru, Karnataka
India
Skills
REST APIs
Microsoft Azure
Python (Programming Language)
Continuous Integration and Continuous Delivery (CI/CD)
PySpark
Azure Data Factory
About the Role
About the Company - We are looking for a skilled Data Engineer with hands-on experience in Azure Data Factory (ADF), PySpark, and REST API development using Python. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and integrations across various sources.
About the Role - Key Responsibilities:
Design and implement robust ETL pipelines using Azure Data Factory.
Develop and optimize distributed data processing applications using PySpark on Azure Databricks or HDInsight.
Build and consume RESTful APIs using Python (Flask/FastAPI/Django) to interact with external systems and services.
Integrate data from various structured and unstructured sources into the data lake or data warehouse.
Monitor and troubleshoot data pipelines, ensuring data quality and integrity.
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders.
Implement best practices in data engineering, security, and compliance on Azure cloud.
Qualifications - Required Skills:
6+ years of experience in Azure Data Factory (ADF) for pipeline orchestration and data integration.
Strong hands-on experience with PySpark and distributed data processing.
Proficient in Python with experience in developing and consuming REST APIs.
Good understanding of Azure ecosystem – Azure Data Lake, Azure Blob Storage, Azure SQL, Key Vault, etc.
Experience with version control systems like Git and CI/CD tools like Azure DevOps.
Knowledge of data warehousing concepts, data modeling, and performance tuning.
Pay range and compensation package -40%hike to current
Location Pan India
Notice Period :- immediate joiner to 30days
About the Role - Key Responsibilities:
Design and implement robust ETL pipelines using Azure Data Factory.
Develop and optimize distributed data processing applications using PySpark on Azure Databricks or HDInsight.
Build and consume RESTful APIs using Python (Flask/FastAPI/Django) to interact with external systems and services.
Integrate data from various structured and unstructured sources into the data lake or data warehouse.
Monitor and troubleshoot data pipelines, ensuring data quality and integrity.
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders.
Implement best practices in data engineering, security, and compliance on Azure cloud.
Qualifications - Required Skills:
6+ years of experience in Azure Data Factory (ADF) for pipeline orchestration and data integration.
Strong hands-on experience with PySpark and distributed data processing.
Proficient in Python with experience in developing and consuming REST APIs.
Good understanding of Azure ecosystem – Azure Data Lake, Azure Blob Storage, Azure SQL, Key Vault, etc.
Experience with version control systems like Git and CI/CD tools like Azure DevOps.
Knowledge of data warehousing concepts, data modeling, and performance tuning.
Pay range and compensation package -40%hike to current
Location Pan India
Notice Period :- immediate joiner to 30days