Senior Data Engineer - (GCP & Lakehouse Architecture) - On Site - Octopus by RTG
We are seeking an experienced Senior Data Engineer to lead the design, development, and scaling of our next-generation, cloud-native Data Lakehouse platform on Google Cloud Platform (GCP). In this role, you will be instrumental in modernizing our data infrastructure, building robust ELT/ETL pipelines, and establishing enterprise-grade data modeling standards to empower analytics, machine learning, and business intelligence. If you are passionate about modern open-source table formats, high-performance distributed computing, and data reliability, we would love to have you on the team.
Key Responsibilities
• Lakehouse Architecture & Development: Design, deploy, and maintain a scalable GCP-based Lakehouse ecosystem leveraging Apache Iceberg for transactional consistency, time-travel, and efficient storage management.
• Pipeline Engineering: Architect, build, and optimize high-throughput ETL/ELT data pipelines using Python and Apache Airflow for orchestrating complex dependency workflows.
• Big Data Processing: Implement large-scale distributed data processing workloads and transformation pipelines using Apache Spark.
• Data Warehousing & Analytics: Leverage BigQuery for high-performance analytical queries, cost optimization, partition/cluster management, and serving structured data to downstream consumers.