Skip to main content

Senior Data Engineer - (GCP & Lakehouse Architecture) - On Site - Octopus by RTG

posted by: spj_bot

We are seeking an experienced Senior Data Engineer to lead the design, development, and scaling of our next-generation, cloud-native Data Lakehouse platform on Google Cloud Platform (GCP). In this role, you will be instrumental in modernizing our data infrastructure, building robust ELT/ETL pipelines, and establishing enterprise-grade data modeling standards to empower analytics, machine learning, and business intelligence. If you are passionate about modern open-source table formats, high-performance distributed computing, and data reliability, we would love to have you on the team.

Key Responsibilities

• Lakehouse Architecture & Development: Design, deploy, and maintain a scalable GCP-based Lakehouse ecosystem leveraging Apache Iceberg for transactional consistency, time-travel, and efficient storage management.

• Pipeline Engineering: Architect, build, and optimize high-throughput ETL/ELT data pipelines using Python and Apache Airflow for orchestrating complex dependency workflows.

• Big Data Processing: Implement large-scale distributed data processing workloads and transformation pipelines using Apache Spark.

• Data Warehousing & Analytics: Leverage BigQuery for high-performance analytical queries, cost optimization, partition/cluster management, and serving structured data to downstream consumers.

Job Skills

dataflow datasets apache airflow reliability high throughput product managers embedding usability build tool performance tuning operator testable design patterns data scalable data engineer GCP big datum data modeling distribute computing semantic management query performance apache spark dimensional modeling sql IAM write data infrastructure cloud native writing data services data quality high performance low latency analytical experience design structured data bigquery senior engineer data engineering open source python google cloud platform data processing automated testing best practices analytics workflows collaboration programming data warehousing concepts storage management business intelligence data data google cloud cloud platform data governance cloud storage machine learning ETL tracking scale workflow data warehousing data pipelines snowflake schema transformation
View the job post & apply

SPJ is not just a platform; it's a transformative force in the maritime sector. We reinvent job discovery and collaboration, leveraging cutting-edge AI to create a space where careers thrive and innovations set sail.

Featured Posts