Principal Software Engineer - Intelligent Decision, Evaluation & Review Platforms
Microsoft
Microsoft
Architect and build scalable platforms for agentic evaluation, production audits, and human-in-the-loop review. Develop workflows in which AI agents gather evidence, analyze cases, compare decisions, and assist reviewers while preserving human oversight and accountability. Build reviewer experiences that present the right evidence, explanations, uncertainty, and recommended actions at the right time. Design distributed orchestration systems for long-running workflows involving models, agents, tools, automated checks, and human decisions. Provide technical leadership across engineering, applied science, product, design, operations, and policy teams. Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, with 8+ years of relevant industry experience. Strong production-coding skills in at least one of C++, C#, Java, Python, or TypeScript. Demonstrated experience designing, building, and operating large-scale distributed services or software platforms. Strong system-design skills across APIs, workflow orchestration, event-driven systems, storage, concurrency, fault tolerance, and service reliability. Experience building AI evaluation, agent orchestration, human-in-the-loop, audit, experimentation, or LLMOps platforms. Experience owning substantial systems from initial architecture through rollout, adoption, and production operations. Background in advertising, trust and safety, fraud, responsible AI, search, recommendations, or another high-scale complex decision domain.
Ready to apply?
Sign up first — takes a minute — and you get Hiro’s take on this role, a resume tailored to it, and (if available) a referral from a real employee at Microsoft. All free with your Pro gift.
Sign up to apply