Senior Software Engineer, Data Governance and Foundations

Overview

Instacart’s Data Governance and Foundations team builds the core infrastructure that keeps our data trusted, secure, and compliant across every region where we operate. We partner closely with Legal, Compliance, Security, and Engineering to translate regulatory requirements like SOX and GDPR into resilient, scalable systems that power safe data access and accelerate the business.

As a Senior Software Engineer on this team, you will design and deliver platform capabilities for policy management, role-based data access, auditing, data classification, and lineage across Instacart’s modern data stack (Snowflake, Databricks, Kafka, Iceberg, Trino). Your work will directly enable product teams, analysts, and data scientists to innovate confidently while meeting stringent privacy and compliance standards.

You’ll join a collaborative team of 6–7 engineers focused on foundational infrastructure, with opportunities to lead high-impact architecture, influence standards, and mentor others. 

About the Job

• Design, build, and operate services and tooling for role-based and attribute-based access control, policy enforcement, auditing, and data classification across Snowflake, Databricks, Trino, and Iceberg-backed datasets.
• Develop event-driven controls and metadata pipelines (e.g., Kafka-based) to automate governance workflows, detect drift, and continuously validate compliance posture.
• Collaborate with Legal, Compliance, and Security to translate requirements (e.g., GDPR, SOX, PII handling) into technical controls, runbooks, and demonstrable evidence for audits.
• Own projects end-to-end—from architectural design and implementation to reliability, performance, and operability—setting and meeting clear SLOs in a fast-paced environment.
• Standardize and evangelize best practices for data privacy, schema evolution, cataloging/lineage, and policy-as-code across Instacart’s data platform and partner teams.

About You

Minimum Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field, or equivalent practical experience.
• 5+ years of software engineering experience building backend or platform services for data-intensive or distributed systems.
• Proficiency in one or more programming languages such as Python, Java, Scala, or Go, plus strong SQL skills.
• Hands-on experience with modern data platforms, including at least two of the following: Snowflake, Databricks, Trino/Presto, Apache Iceberg, Apache Kafka.
• Demonstrated experience implementing data security controls (e.g., RBAC/ABAC, row/column-level security, tokenization, or dynamic data masking) in production.
• Experience operating services in cloud environments (AWS, GCP, or Azure), including monitoring, incident response, and on-call participation.
• Proven track record of delivering complex, multi-stakeholder projects in a rapidly changing environment.

Preferred Qualifications

• Deep knowledge of distributed systems concepts (consensus, partitioning, streaming semantics, exactly-once processing, schema compatibility).
• Familiarity with data privacy and compliance frameworks (e.g., GDPR, SOX) and building auditable technical controls and evidence pipelines.
• Experience with data cataloging and lineage (e.g., Unity Catalog, DataHub, OpenLineage) and policy frameworks (e.g., OPA/Rego, Apache Ranger, Lake Formation).
• Exposure to Infrastructure-as-Code (e.g., Terraform) and policy-as-code practices to standardize governance across environments.
• Ability to lead design reviews, mentor engineers, and drive alignment across Legal, Compliance, Security, and Engineering partners.