Job Description
We are currently seeking an experienced Data Engineer to join the Big Data and Advanced Analytics department. As part of the Data Engineering team, the Lead Data Engineer will architect, design, and implement a cloud based modern enterprise data platform to support data analytic use cases for the midstream oil and gas business units. This individual will lead the development of enterprise-wide data strategies enabling advanced analytical uses case and driving innovation through modern cloud technologies. Responsibilities include:
- Architect, design, and implement comprehensive cloud-based data architectures
- Develop data models, data flow diagrams, and technical specifications for complex data ecosystem
- Create reference architectures and design patterns for data ingestion, processing, storage, and analytics
- Lead cross-functional teams in the design and implementation of data products
- Collaborate with data engineers, analysts, data scientists, and stakeholders to translate business requirements into technical solutions
- Provide technical guidance on cloud data services including data lakes, data warehouses, and streaming platforms
- Create and implement strategies for migrating on-premise data platforms to cloud environments
- Develop and implement long-term data architecture roadmaps aligned with business objectives
- Define and implement data integration strategies across multiple systems and platforms
- Implement complex modules and core services of the data platform ensuring correctness, reliability, scalability, and stability
- Mentor junior data engineers in design and code reviews and implementation best practices
The successful candidate will meet the following qualifications:
- 7 years of experience as a hands-on Data Engineer designing and implementing enterprise-wide data platform architectures
- Expert level experience in Python and SQL
- Expert level experience in dimensional modeling, data vault methodology, and modern data modeling techniques
- Deep expertise in at least one major cloud provider (AWS, Azure, GCP) with knowledge of their data services
- Experience in various data integration patterns including ETL, ELT, Pub/Sub, Streaming, and Change Data Capture
- Experience with common Python Data Engineering packages including pandas, numpy, pyarrow pytest, scikit-learn, and boto3
- Experience with Infrastructure as Code principals and tools such as Terraform and Cloud Formation
- Experience in software development practices such as Design Principles and Patterns, Modularity, Unit Testing, Refactoring, CI/CD, and version control
- Experience building data products leveraging Snowflake, DBT, and Airflow
- Experience in implementing Data Lakehouse patterns using Apache Iceberg or Delta Lake
- Excellent communication skills with ability to present complex technical concepts to non-technical stakeholders
Job Tags
Permanent employment,