Senior Lead Data Engineer - R01557349

San Francisco, United States
Job ID: R01557349

SAVE JOB APPLY

Senior Lead Data Engineer - R01557349

Senior Lead Data Engineer
Primary Skills
  • Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL
    • Specialization
  • Hadoop and Spark SQL
    • Job requirements
  • Key Responsibilities:
  • Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL
  • - Manage and optimize the infrastructure ensuring reliability and performance
  • -Design and implement data models and schemas for enterprise data warehouse and data marts
  • -Write and maintain scripts in Python for various applications
  • -Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases
  • - Work closely with analysts and product managers to understand data requirements and deliver high-quality data
  • -Provide technical guidance and mentorship to other team members fostering best practices in data engineering
  • -Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability

    • Required Skills:
  • -High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing
  • -Good knowledge in Python for data manipulation and scripting
  • -Mastery of advanced SQL and deep experience working with relational databases and datawarehouses
  • -Hands-on experience with workflow management tools like Apache Airflow
  • -Familiarity with Unix shell scripting for automation and system management

    • Similar Jobs

    Jobs based on your browsing history

    Together, we create the future you always aspired to. Explore your next career opportunity.

    SEE ALL OPEN POSITIONS