Data
Engineer
Bradenton, FL
Contract
Bradenton, FL
Contract
Job Description:
The Data Engineer will be responsible for building,
maintaining data pipelines and data products to ingest, process large volume of
structured / unstructured data from various sources. The Data engineer will
work on analyzing the data needs, migrating the data into an Enterprise data
lake, build data products and reports. The role requires experience with
building real time and batch based ETL pipelines with strong understanding of
big data technologies and distributed processing frameworks with.
Skill Needs
·
Expertise working with large scale distributed
systems (Hadoop, Spark).
·
Strong understanding of the big data cluster, and its
architecture
·
Experience building and optimizing big data ETL
pipelines.
·
Advanced programming skills with Python, Java, Scala
·
Good knowledge of spark internals and performance
tuning of spark jobs.
·
Strong SQL skills and is comfortable operating with
relational data models and structure.
·
Capable of accessing data via a variety of
API/RESTful services.
·
Experience with messaging systems like Kafka.
·
Experience with No SQL databases. Neo4j, mongo, etc.
·
Expertise with Continuous Integration/Continuous
Delivery workflows and supporting applications.
·
Exposure to cloud environments and architectures.
(preferably Azure)
·
Ability to work collaboratively with other
teams.
·
Experience with containerization using tools such as
Docker.
·
Strong knowledge of Linux and Bash. Can
interact with the OS at the command line and create shell scripts to automate
workflows.
·
Advanced understanding of software development and
collaboration, including experience with tools such as Git.
·
Excellent written and verbal communication skills,
comfortable presenting in front of non-technical audiences.
Post a Comment