Staff Software Engineer, Data

Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than 800 of the world's leading enterprises, Astronomer lets businesses do more with their data. To learn more, visit www.astronomer.io.

About this role:

At Astronomer, we’re redefining how companies run Apache Airflow at scale. Our R&D organization is home to some of the most innovative minds in cloud infrastructure and open-source software.

You can’t spell “Data Orchestration” without “Data”, and we’re looking to add a world-class Staff+ level engineer to our team, to set out our Data and Database story as we level up our platform’s Reliability, cost profile and growth trajectory.

You get to go in at the ground level of how our production infrastructure is designed, built, tested and deployed. Your work will directly influence how we build Astro, Observe and our IDE product, as well as how global organizations orchestrate data pipelines at scale—making them faster, more reliable, and easier to manage.

What you get to do:

• Be a subject-matter expert in how we treat Data at scale. Astronomer has a number of different databases and data sources at work providing our platform, and our needs are evolving. We’re looking for a database and data platforms expert to map out how we store, retrieve, keep safe, and otherwise be responsible curators of ours and customers’ data.
• Astronomer’s stack is fairly heavily Postgres-based, with some blob storage and some specialised options for certain data types - you get to recommend, design and later lead building the data systems that will help us continue to scale. This is very much a technical role; you’ll be just as active in building these systems and ensuring they’re fit for purpose as specifying and designing.

• You'll be at the forefront of how we work together as a Platform Engineering team and an R&D group more broadly.

  • Blaze a Trail: Own and develop our Database and overall data strategy and practice, with sponsorship and responsibility to match – this role reports directly to the VP of Reliability.

  • Be an Owner: Be directly involved in decision-making on what we work on, as well as how we work on it. Make promises, and keep them.
  • Do Sensible Things: Be directly involved in determining how our platform works. Make build vs. buy assessments, and advocate for the right tools for the right job when it comes to data.
  • Garage Door Open: Create and maintain comprehensive internal documentation and decision records for systems and processes, ensuring clarity and accessibility. Participate in Architectural forums and discussion and make principled, open decisions.

What you bring to the role:

• Hands-on experience designing, developing, and scaling production infrastructure.
• Extensive knowledge of Postgres and Postgres-like cloud offerings (AWS, GCP, Azure)
• Extensive and recent experience with building low-level data systems and/or managed data platforms.
• In-depth knowledge and practical experience of the product and technology space in Database and related fields (Blob, NoSQL, Timeseries, Graph, Vector).
• Experience defining requirements and making and justifying technology choices around the data space.
• Strong experience in Non-Abstract Systems design and implementation.
• Strong proficiency in Golang and in-depth experience with Kubernetes.
• Strong communication skills, both written and verbal, with experience in working with a globally distributed team in delivery.

Bonus points if you have:

• Experience working with Spanner, AlloyDB and/or other cloud-native databases, including hands-on experience with provisioning, development practices and migration of data.
• Experience working on a SaaS/PaaS product across multiple cloud providers.
• Experience building internal data platforms from cloud-native component parts - we have a healthy mix of build vs. buy; sometimes building is the right choice.
• Experience with Apache Airflow.

At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.