Senior Machine Learning Engineer II, AI Special Projects

Overview

AI Special Projects is a newly formed, agile team driving the next wave of innovation at Instacart. The team’s charter is to reimagine our core consumer experience with AI. We are tackling some of the company’s biggest growth bets, including generative recommendation systems, hyper-personalized marketing content, and unique customer experiences such as personalized meals. Additionally, we are paving the way for AI-native development across the company by driving advancements in event-driven architectures and other paradigms that will impact a broad range of applications.

We build with a modern, AI-native Python stack with minimal legacy tech debt. This empowers us to move quickly, experiment, and focus on pursuing the highest-impact opportunities.

As a technical lead on the team, you'll play a pivotal role in shaping the future of Instacart's AI platform offerings. This role provides the opportunity to elevate AI at Instacart by applying advanced industry architectures, innovating upon them, and deploying them to improve experiences for our customers and retailers.

From Day 1, you will arrive as a leader on the team with significant scope and influence. This includes contributing through your product mindset, technical vision, strong execution, and mentorship. If a culture of bottom-up innovation and deep ownership resonates with you, we’re excited to meet you!

About the Job
In this role, you will:

• Build the team vision: Partner with cross-functional leaders to define key business challenges and design AI-first solutions that unlock substantial value for customers and stakeholders.
• Drive hands-on execution: Lead the development of personalized content and recommendation systems by leveraging a mix of generative and traditional machine learning architectures to drive core business outcomes.
• Deploy AI systems at scale: Drive end-to-end software development, including building ML services, agentic workflows, managing experiment lifecycles, and deploying production-ready AI solutions at scale.
• Innovate: Stay at the cutting edge of AI innovation by conducting literature reviews and exploring novel paradigms in recommendation systems (RecSys). Identify opportunities for customization and apply these advancements to meet Instacart’s unique needs.
• Elevate the team: Mentor and guide junior team members, fostering learning, engagement, and technical growth within the team.

About You
Minimum Qualifications:

• Expertise in Generative AI technologies, including prompt engineering, model fine-tuning, and familiarity with emerging AI trends and AIQA processes.
• Strong background in traditional machine learning and deep learning fundamentals, with hands-on experience building efficient models using frameworks like PyTorch or TensorFlow.
• Professional software development skills, including building data pipelines (SQL/Spark), managing CI/CD workflows, and deploying models into production environments.
• Product-focused mindset with a bias for action and proven ability to define and solve complex business problems through cross-functional collaboration.
• Exceptional communication skills, with the ability to influence stakeholders and translate technical insights into actionable business strategies.
• An innovation-driven mindset, rooted in curiosity and a passion for pushing the boundaries of applied AI.

Preferred Qualifications:

• Experience building scalable ranking and recommendation systems.
• Application of ML models within an e-commerce environment.
• Cloud infrastructure expertise (e.g., AWS, GCP), particularly in managing large GPU workloads for training and inference.
• Demonstrated success as a technical leader, with the proven ability to guide teams toward strategic, high-impact outcomes.
• Technical influence evidenced by contributions to blogs, published papers, or industry thought leadership.