On-Site
Full-Time
Hyderabad, Telangana
India
Skills
Python (Programming Language)
SQL
Apache Spark
Azure Databricks
Scala
About the Role
Compelling Opportunity for Senior Data Engineer with Innovative Entity in Insurance Industry
Employment | Immediate
Location: Hyderabad, India
Reporting Manager: Head of Analytics
Work Pattern: Full Time, 5 days in the office
Minimum Experience as a data engineer: 8 years
Responsibilities
· Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights
· Constructing infrastructure for efficient ETL and ELT processes from various sources and storage systems
· Knowledge of how to create and improve data sets, “big data” data pipelines, and infrastructures
· Being able to do root cause analysis on data and procedures both internally and outside to find possibilities for improvement and provide clarification
· Outstanding analytical abilities related to coping with unstructured datasets.
· Capacity to develop procedures that support task management, data structures, dependence, and metadata
· Strong project management and organisational skills
· Collaborating closely with Product Managers and Business Managers to design technical solutions aligned with business requirements
· Leading the implementation of algorithms and prototypes to transform raw data into useful information
· Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations
· Creating innovative data validation methods and data analysis tools
· Ensuring compliance with data governance and security policies
· Interpreting data trends and patterns to establish operational alerts
· Developing analytical tools, programs, and reporting mechanisms
· Conducting complex data analysis and presenting results effectively
· Preparing data for prescriptive and predictive modeling
· Continuously exploring opportunities to enhance data quality and reliability
· Applying strong programming and problem-solving skills to develop scalable solutions
· Collaborate with data scientists, data analysts and architects on several products and projects
Skills
· 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines
· Technical expertise with data models, data mining, and segmentation techniques for both structured and unstructured data
· High proficiency in Scala/Java and Spark for applied large-scale data processing
· Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive
· Solid understanding of batch and streaming data processing techniques
· Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion
· Expert-level ability to write complex, optimized SQL queries across extensive data volumes
· Experience with RDBMS and OLAP databases like MySQL, Redshift
· Familiarity with Hadoop, HBase, MapReduce, and other suitable platforms.
· Excellent understanding of operating systems like UNIX, Linux, and Windows
· Familiarity with Agile methodologies
· Obsession for service observability, instrumentation, monitoring, and alerting
· Knowledge or experience in architectural best practices for building data lakes
· Hands-on data management in any of the hyperscalers – Azure, AWS, GCP
Good to Have:
· Passion for testing strategy, problem-solving, and continuous learning.
· Willingness to acquire new skills and knowledge
· Possess a product/engineering mindset to drive impactful data solutions
· Experience working in distributed environments with global teams
· Great numerical and analytical skills
Soft Skills
· Analytical thinking skills
· Excellent verbal and written communication skills
· Problem-solving skills
Tools and Programs (Min 3 of these)
· Python
· SQL
· Databricks
· Kedro
· Airflow
· Luigi
· PowerBI
· Tableau
· Java
· C#
Education
A bachelor’s degree in computer science, data science, statistics or related field. A master’s degree in data management, information science or similar is preferred. Certifications from IBM, Google or Microsoft is desired.
Screening Criteria
· A bachelor’s degree in computer science, data science, statistics or related field
· 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines
· High proficiency in Scala/Java and Spark for applied large-scale data processing
· Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive
· Solid understanding of batch and streaming data processing techniques
· Expert-level ability to write complex, optimized SQL queries across extensive data volumes
· Expertise in atleast 3 of the following
o Python
o SQL
o Databricks
o Kedro
o Airflow
o Luigi
o PowerBI
o Tableau
o Java
o C#
· Exposure to data models, data mining, and segmentation techniques for both structured and unstructured data
· Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion
· Experience with RDBMS and OLAP databases like MySQL, Redshift
· Excellent understanding of operating systems like UNIX, Linux, and Windows
· Hands-on data management in any of the hyperscalers – Azure, AWS, GC
· Available to work from office in Hyderabad
· Available to join within 30 days
Considerations
· Location – Hyderabad
· Working from office
· 5 day working
Evaluation Process
Round 1 – HR Round
Round 2 & 3 – Technical Round
Round 4 – Discussion with CEO
Interested Profiles,
Kindly Apply
Note
o Additional inputs to be gathered from the candidate to put together the application
Employment | Immediate
Location: Hyderabad, India
Reporting Manager: Head of Analytics
Work Pattern: Full Time, 5 days in the office
Minimum Experience as a data engineer: 8 years
Responsibilities
· Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights
· Constructing infrastructure for efficient ETL and ELT processes from various sources and storage systems
· Knowledge of how to create and improve data sets, “big data” data pipelines, and infrastructures
· Being able to do root cause analysis on data and procedures both internally and outside to find possibilities for improvement and provide clarification
· Outstanding analytical abilities related to coping with unstructured datasets.
· Capacity to develop procedures that support task management, data structures, dependence, and metadata
· Strong project management and organisational skills
· Collaborating closely with Product Managers and Business Managers to design technical solutions aligned with business requirements
· Leading the implementation of algorithms and prototypes to transform raw data into useful information
· Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations
· Creating innovative data validation methods and data analysis tools
· Ensuring compliance with data governance and security policies
· Interpreting data trends and patterns to establish operational alerts
· Developing analytical tools, programs, and reporting mechanisms
· Conducting complex data analysis and presenting results effectively
· Preparing data for prescriptive and predictive modeling
· Continuously exploring opportunities to enhance data quality and reliability
· Applying strong programming and problem-solving skills to develop scalable solutions
· Collaborate with data scientists, data analysts and architects on several products and projects
Skills
· 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines
· Technical expertise with data models, data mining, and segmentation techniques for both structured and unstructured data
· High proficiency in Scala/Java and Spark for applied large-scale data processing
· Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive
· Solid understanding of batch and streaming data processing techniques
· Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion
· Expert-level ability to write complex, optimized SQL queries across extensive data volumes
· Experience with RDBMS and OLAP databases like MySQL, Redshift
· Familiarity with Hadoop, HBase, MapReduce, and other suitable platforms.
· Excellent understanding of operating systems like UNIX, Linux, and Windows
· Familiarity with Agile methodologies
· Obsession for service observability, instrumentation, monitoring, and alerting
· Knowledge or experience in architectural best practices for building data lakes
· Hands-on data management in any of the hyperscalers – Azure, AWS, GCP
Good to Have:
· Passion for testing strategy, problem-solving, and continuous learning.
· Willingness to acquire new skills and knowledge
· Possess a product/engineering mindset to drive impactful data solutions
· Experience working in distributed environments with global teams
· Great numerical and analytical skills
Soft Skills
· Analytical thinking skills
· Excellent verbal and written communication skills
· Problem-solving skills
Tools and Programs (Min 3 of these)
· Python
· SQL
· Databricks
· Kedro
· Airflow
· Luigi
· PowerBI
· Tableau
· Java
· C#
Education
A bachelor’s degree in computer science, data science, statistics or related field. A master’s degree in data management, information science or similar is preferred. Certifications from IBM, Google or Microsoft is desired.
Screening Criteria
· A bachelor’s degree in computer science, data science, statistics or related field
· 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines
· High proficiency in Scala/Java and Spark for applied large-scale data processing
· Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive
· Solid understanding of batch and streaming data processing techniques
· Expert-level ability to write complex, optimized SQL queries across extensive data volumes
· Expertise in atleast 3 of the following
o Python
o SQL
o Databricks
o Kedro
o Airflow
o Luigi
o PowerBI
o Tableau
o Java
o C#
· Exposure to data models, data mining, and segmentation techniques for both structured and unstructured data
· Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion
· Experience with RDBMS and OLAP databases like MySQL, Redshift
· Excellent understanding of operating systems like UNIX, Linux, and Windows
· Hands-on data management in any of the hyperscalers – Azure, AWS, GC
· Available to work from office in Hyderabad
· Available to join within 30 days
Considerations
· Location – Hyderabad
· Working from office
· 5 day working
Evaluation Process
Round 1 – HR Round
Round 2 & 3 – Technical Round
Round 4 – Discussion with CEO
Interested Profiles,
Kindly Apply
Note
o Additional inputs to be gathered from the candidate to put together the application