Staff Software Engineer, AI Search

RDQ427R351

At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business.

Search plays a foundational role in this mission. Whether through keyword-based retrieval, semantic similarity via vector embeddings, or hybrid approaches that combine both, our Search technologies help customers find, discover, and understand information across massive, complex datasets. These capabilities power everything from Retrieval Augmented Generation (RAG), AI assistants, and recommendation systems to enterprise knowledge management, in-product search, and data exploration.

As a Staff Engineer for Search, you’ll play a critical role in building and scaling our next-generation Search product — spanning Vector Search, Keyword Search, and Search Quality. You’ll drive the design and evolution of a highly-performant, cost-efficient, and developer-friendly Search stack that enables customers to build AI applications with accuracy, speed, and scale. Beyond hands-on contributions, you’ll help define the long-term vision, mentor senior engineers, collaborate across teams, and lead strategic efforts that expand the technical and business impact of Search at Databricks.

Check out our recent blog post: https://www.databricks.com/blog/decoupled-design-billion-scale-vector-search

The impact you will have:

• Drive the architecture and evolution of the Search product, delivering scalable, low-latency, and high-quality retrieval across massive multimodal datasets.
• Identify and solve challenges in indexing, storage, query execution, and ranking — blending innovations in vector search, keyword search, and hybrid retrieval techniques.
• Partner with infrastructure, product, and research teams to shape APIs and developer experiences that make it easy to build production-grade, AI-powered and search-driven applications.
• Lead end-to-end engineering efforts — from design to production rollout — ensuring high standards for performance, correctness, and reliability.
• Make direct, high-impact contributions to core systems powering search, discovery, and recommendation experiences.
• Champion software engineering excellence and mentor other engineers through design reviews, technical discussions, and collaborative development.
• Contribute to long-term planning and influence Databricks’ technical direction in search, AI infrastructure, and data systems.

What we look for:

• 10+ years of experience building and operating large-scale distributed systems.
• Deep expertise in Search technologies (vector, keyword, hybrid) and related domains such as indexing, ranking, retrieval infrastructure, and relevance optimization.
• Familiarity with storage systems, recommendation engines, and database internals.
• Strong foundation in algorithms, data structures, and system design as applied to large-scale retrieval and data systems.
• Proven ability to deliver technically complex, high-impact initiatives that create customer or business value.
• Experience leading architecture for performance-sensitive, large-scale systems.
• Strong communication skills and ability to collaborate across teams in fast-moving environments.
• Strategic and product-oriented mindset with the ability to align technical execution with long-term vision.
• Passion for mentoring, growing engineers, and fostering technical excellence.