Remote
Contract
India
Skills
Microsoft Azure
Data Warehousing
Hadoop
Data Engineering
Hive
Machine Learning
Databases
Data Science
Extract, Transform, Load (ETL)
Apache Spark
Data Modeling
Data Analytics
About the Role
Company Description
ThreatXIntel is a startup cyber security company that focuses on protecting businesses and organizations from cyber threats. We offer services in cloud security, web and mobile security testing, cloud security assessment, and DevSecOps. Our goal is to provide customized and affordable solutions to meet the specific needs of our clients, regardless of their size.
Role Description
We are seeking a skilled Freelance Data Engineer to support our real-time data engineering initiatives. The ideal candidate will have strong expertise in Apache Flink (mandatory), Python, Databricks, and Microsoft Azure. You will be responsible for building real-time and batch data processing solutions, optimizing ETL workflows, and contributing to scalable data architecture in a cloud-native environment.
Tech Stack & Tools
Programming Languages: Python (required), SQL and NoSQL (required)
Big Data & Streaming Tools: Apache Flink (mandatory), Apache Spark (required), Apache Kafka (preferred)
Cloud Platform: Microsoft Azure (required)
Data Processing & ETL: Databricks (required), ETL pipeline design & optimization (required)
Orchestration & Containerization: Kubernetes (preferred)
Version Control: Git or similar (required)
Data Storage: Azure Blob Storage, ADLS
Databases: SQL databases for structured data, NoSQL databases (e.g., MongoDB, Cosmos DB) for unstructured data
Key Responsibilities
Design and build real-time data pipelines using Apache Flink
Develop scalable ETL workflows in Databricks using Python and Spark
Implement high-performance, fault-tolerant streaming systems
Manage large-scale data processing across structured and unstructured datasets in cloud environments
Ensure data quality, integrity, and security best practices are maintained
Collaborate with internal teams to optimize and automate data workflows
Use Azure-native tools and Kubernetes for orchestration and deployment
Preferred Experience
Hands-on experience with real-time analytics systems
Familiarity with cluster computing and distributed architecture
Proven ability to work in cross-functional teams
Understanding of CI/CD pipelines and DevOps practices
ThreatXIntel is a startup cyber security company that focuses on protecting businesses and organizations from cyber threats. We offer services in cloud security, web and mobile security testing, cloud security assessment, and DevSecOps. Our goal is to provide customized and affordable solutions to meet the specific needs of our clients, regardless of their size.
Role Description
We are seeking a skilled Freelance Data Engineer to support our real-time data engineering initiatives. The ideal candidate will have strong expertise in Apache Flink (mandatory), Python, Databricks, and Microsoft Azure. You will be responsible for building real-time and batch data processing solutions, optimizing ETL workflows, and contributing to scalable data architecture in a cloud-native environment.
Tech Stack & Tools
Programming Languages: Python (required), SQL and NoSQL (required)
Big Data & Streaming Tools: Apache Flink (mandatory), Apache Spark (required), Apache Kafka (preferred)
Cloud Platform: Microsoft Azure (required)
Data Processing & ETL: Databricks (required), ETL pipeline design & optimization (required)
Orchestration & Containerization: Kubernetes (preferred)
Version Control: Git or similar (required)
Data Storage: Azure Blob Storage, ADLS
Databases: SQL databases for structured data, NoSQL databases (e.g., MongoDB, Cosmos DB) for unstructured data
Key Responsibilities
Design and build real-time data pipelines using Apache Flink
Develop scalable ETL workflows in Databricks using Python and Spark
Implement high-performance, fault-tolerant streaming systems
Manage large-scale data processing across structured and unstructured datasets in cloud environments
Ensure data quality, integrity, and security best practices are maintained
Collaborate with internal teams to optimize and automate data workflows
Use Azure-native tools and Kubernetes for orchestration and deployment
Preferred Experience
Hands-on experience with real-time analytics systems
Familiarity with cluster computing and distributed architecture
Proven ability to work in cross-functional teams
Understanding of CI/CD pipelines and DevOps practices
Apply for this position
Application Status
Application Draft
In Progress
Submit Application
Pending
Review Process
Expected within 5-7 days
Similar Jobs
