NewYorkRecruiter Since 2001
the smart solution for New York jobs

Senior Data Engineer

Company: Tavant Technologies Inc
Location: New York
Posted on: September 25, 2022

Job Description:

The Senior Data Engineer opportunity is located in Santa Monica CA/ Seattle, WA/NYC, NY and This team is considered Hybrid so looking for someone that's close to Cal/Seattle/NYC - Can be remote to start but the expectation is in the office.No sponsorship at this moment. What you will do: Sr Data Engineer (SQL, Python, Airflow) AWSThink and communicate critically about architecture, design, and best practices and guide your team to adopt them.Design data systems that allow managed growth of the data model to minimize risk and cost of change.Write transformation and validation code that applies complex data aggregation and calculation using SQL and PythonDrive implementation of automated testing for data pipelines within a CI environmentCreate new pipelines or rewrite existing pipelines and build reusable components at scale to support accounting functions, as well as reporting & analytics.Collaborate with other Companies and teams to identify and document shifting data requirements while also advocating for a minimal change set for your team.Solve complex data issues and perform root cause analysis to proactively resolve product and operational issues.Collaborate with leadership and other engineers to develop technical story backlog derived from high-level business requirements and design collaboration and estimating story points.BS or MS in Computer Science, a related field, or equivalent industry experience3 years of professional experience engineering complex, high-volume data pipelines using SQL, Python, and Airflow3 years of experience building cloud scalable and high-performance data lake/data warehouse solutions using AWS products - S3, Athena, Glue, and EMRExperience with binary data serialization formats such as ParquetDeep understanding of data structures and algorithmsUnderstanding of code versioning tools such as GITHave a passion for data solutions Nice to Have:Exposure to AWS cloud data pipeline tools such as Managed Airflow and GlueExperience integrating with Ad Tech platforms such as Operative and STAQExposure and opinions regarding alternate orchestration tooling beyond AirflowUnderstanding of SOX compliance needs and how they affect system design.Have worked with a variety of Airflow Operator types, including REST, Lambda, ECSCan flex between Python and Javascript/Typescript. Technical EnvironmentAurora/Hive (databases)Spark (large-scale data processing)Airflow (workflow management)Docker (software packaging and delivery)AWS (development and hosting)

Keywords: Tavant Technologies Inc, New York , Senior Data Engineer, Accounting, Auditing , New York, New York

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category

Log In or Create An Account

Get the latest New York jobs by following @recnetNY on Twitter!

New York RSS job feeds