Lead Data Engineer

  • Free
  • Published date: December 7, 2022
    • New South Wales, Australia


Role: Data Engineer


Location: (Remote Role)


Mandatory Skills: Unix, Jenkins/GitLab, SQL, Python, (Pyspark/Scala),
Databricks & Spark

Mandatory Responsibilities: Delta Tables, Merging & Design

Desired Skills: IDE

Good To Have: Python API, Terraform & AWS (IAM & S3)

YOE: 10-18

 


Basic Unix Knowledge: (Mandatory)
Must be able to work in mac
Resolve environment setup issues with minimal help from others
Basic Devops skills: (Mandatory)
Understanding of source control, feature branch, resolve code conflict
issues
Experience in working with build pipelines using Jenkins or GitLab
Must be comfortable working with IDE
Basic SQL skills: (Mandatory)
Inner join, left joins etc
Basic AWS skills: (Secondary)
Understanding of IAM, S3
Resolve access issues related to IAM Policy, trust relationship etc.
At least 5 years of development experience using Python (Mandatory)
At least 1 year of experience working with Pyspark or Scala:
Basic understanding of ETL (Mandatory)
Basic knowledge of Databricks clusters (Mandatory)
Basic knowledge of spark (Mandatory)
Must have worked in Delta Tables, Merging (Mandatory)
Must have used python api to call elasticseaarch (Secondary)
Basic Design skills. Must be able to talk about the design work done in
earlier projects (Mandatory)
Must be willing to work with fluid requirements in fast paced environment
(Mandatory)
Must be comfortable in working with 5-6 code bases at a time (Mandatory)
Basic Terraform knowledge (Secondary)

Reference : Lead Data Engineer jobs

Useful information

  • Avoid scams by acting locally or paying with PayPal
  • Never pay with Western Union, Moneygram or other anonymous payment services
  • Don't buy or sell outside of your country. Don't accept cashier cheques from outside your country
  • This site is never involved in any transaction, and does not handle payments, shipping, guarantee transactions, provide escrow services, or offer "buyer protection" or "seller certification"


post to multiple job boards
Now take benefits of multiple job boards and search engine posting.
Try it.

Related listings

  • 100% Remote - Cloud Platform/ Data Engineer (Cloud Operations team)

    Technology (New South Wales) December 7, 2022 Free

    Cloud Platform/ Data Engineer (Cloud Operations team) Location : Remote Responsibilities:      Experience with datalakes and data pipelines      Strong understanding of data and data-integration concepts (SQL, data governance, enterprise patterns)   ...

  • QA Manager with Healthcare for Remote.

    Technology (New South Wales) December 7, 2022 Free

    We have an opening for QA Manager with Healthcare Position: QA Manager with Healthcare Location: Remote Duration: Contract Experience: 9+ years Interview mode: Video. Requirements: - Few MMIS implementation exp - Working with various stakeholders - Q...

  • Azure Cyber Security Lead

    Technology (New South Wales) December 7, 2022 Free

    Position: Azure Cyber Security Lead/Engineer Location: Remote Azure Cyber Security Lead/Engineer - 10+ years? experience in IT along with minimum of 6+ years of experience in Azure Cloud. - Experience in Terraform, Infrastructure of a Code, JSON, Pow...