Distributed Systems Engineer (Apache Spark Internals) – Remote

About the role

We are looking for a Distributed Systems Engineer specialized in Apache Spark internals to work on the core engine, not on user‑level pipelines. The role focuses on optimizing and extending Spark’s core components such as Catalyst, Tungsten, the DAG scheduler, shuffle and memory management at petabyte scale.

Key responsibilities

• Contribute production‑grade code to Apache Spark, especially Spark SQL and Structured Streaming.
• Debug, profile and optimize Spark internals (Catalyst, Tungsten, DAG Scheduler, Shuffle, memory management) for petabyte‑scale workloads.
• Influence architectural direction to improve Spark performance and scalability.
• Profile and tune JVM behavior (GC, memory layout, concurrency) at the engine level.
• Collaborate with cross‑functional engineering teams and open‑source committers on integrations and ecosystem projects.
• Mentor senior engineers and raise the engineering bar through code reviews and design critiques.

Required profile

• 6+ years of software development experience.
• Strong Java and/or Scala programming skills.
• Experience with distributed systems and concurrent or parallel programming.
• Working knowledge of Spark internals (Catalyst, Tungsten, DAG Scheduler, Shuffle, or memory management).
• Familiarity with JVM performance characteristics (GC, memory, threading).
• Advanced level of English.

Required skills

• Java
• Scala
• Apache Spark (core, Spark SQL, Structured Streaming)
• Catalyst Optimizer
• Tungsten execution engine
• DAG Scheduler
• Shuffle subsystem
• Memory management
• JVM performance tuning (GC, concurrency)
• Distributed systems

What we offer

• Fully remote work.
• Competitive compensation in USD or local currency.
• Hardware and software setup for home office.
• Flexible schedule – create your own hours.
• Paid parental leave, vacation and national holidays.
• Innovative, multicultural environment.