MapReduce, HDFS, Spark - Pyspark, ETL Fundamentals, SQL, SQL (Basic + Advanced), Spark - Scala, Python, Data Warehousing, Hive, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures, Oozie
Job requirements
• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
• Design, develop, and maintain data pipelines and ETL processes using Python, Hive, Spark, and other relevant technologies.
• Good Hands on Exp with java
• Minimum of 6+ years of experience as a Data Engineer, with proven expertise in Python, Hive, and Spark. • Advance SQL experience and working experience with Big data Hadoop .
• Strong understanding of data engineering concepts, including data modelling, data integration, and data warehousing. • Hands-on experience with tools such as Apache Airflow.
• Experience with Docker and containerization technologies is a plus.
• Strong SQL skills and experience with relational databases, such as PostgreSQL, MySQL, or similar.
• Familiarity with Apache Spark.
• Knowledge of data processing frameworks such as Apache Beam, Apache Flink, or similar.
• Strong problem-solving and troubleshooting skills, with attention to detail.
• Excellent communication and interpersonal skills, with the ability to work collaboratively in a team environment.
