[Hiring] Senior Data Engineer @Sequoia Connect
Job at a glance
Role Description We are currently searching for a Senior Data Engineer : Work with cutting-edge big data platforms (e.g., Databricks, Apache Spark) at a large scale, pushing the boundaries of data processing and model enablement. Build and maintain robust ETL/ELT pipelines for the ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms. Develop high-performance data processing jobs using PySpark/Spark and Python on data platforms such as Cloudera and Databricks.
Optimize pipeline performance and cost through partitioning, file formats, compute tuning, and efficient query patterns. Contribute to CI/CD for data workflows (testing, code reviews, deployment automation), promoting engineering best practices and maintainable codebases. Partner with Product Managers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features.
Qualifications Strong hands-on experience in data engineering, building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms, specifically Databricks). High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), and SQL to build Big Data products. Hands-on experience with cloud data platforms such as Databricks and Snowflake (Databricks preferred).
Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend. Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability. Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders. Experience developing Java-based applications is an added advantage.
Familiarity with cloud-native foundations or AI coding assistants. Requirements Advanced Oral English for seamless collaboration with global teams. Advanced Spanish. Benefits Flexible work arrangement: Remote.