Senior ML Engineer at Microsoft

I build distributed infrastructure that makes LLM agents reliable in production.

Senior ML systems engineer with 12+ years building distributed execution, model serving, evaluation, and data platforms. I focus on latency, throughput, and fault recovery for agent and post-training workloads.

Explore my work Connect on LinkedIn
Portrait of Mrityunjay Kumar

Based in San Francisco Bay Area

Millions-scale request workloads

100M+ documents generated

100M DAU product reach

Applied science evaluation · ML · systems

Selected experience

Production systems. Measurable outcomes.

12+ years across ML infrastructure, distributed storage, and data platforms at Microsoft, VMware, Talentica, and MediaTek.

  1. Microsoft

    Senior ML Engineer

    Mountain View, CA

    Building production ML and agent infrastructure for PowerPoint, spanning model evaluation, fine-tuning pipelines, and the distributed serving and execution systems that run these workloads at scale with low latency and high reliability.

    Accomplishments

    • Scaled the agent's distributed execution and serving infrastructure to thousands of concurrent containers under strict latency SLOs, with load balancing, autoscaling, and automated fault detection and recovery.
    • Built PowerPoint's first end-to-end fine-tuned model workflow, including synthetic-data generation, batched data preparation, secure deployment, inference, and integration with PowerPoint workflows.
    • Designed extensible slide-generation and evaluation systems with configurable models and datasets, end-to-end telemetry, and parallel execution that reduced evaluation time by 4.6x.
    • Designed and operated PowerPoint cloud services across commercial and government environments, adding automated validation, observability, and incident-response tooling.
    • ML systems
    • Inference & serving infrastructure
    • Model evaluation
    • Fine-tuning & MLOps

  2. VMware

    Member of Technical Staff III

    Palo Alto, CA

    Led the design of the FSCK user-space tool infrastructure for VMware's distributed storage (vSAN); core member of the snapshot dev team.

    Accomplishments

    • Conceptualized and built distributed FSCK for analyzing and repairing storage metadata with cross-consistency matrices across B+ trees, bitmaps, and segment usage tables, in online and offline modes via lazy evaluation.
    • Built a cloud-native microservice validating the filesystem key-value store with pluggable validation algorithms; reused for snapshots, unmap, and segment cleaning.
    • Implemented serialized transactional snapshot crash consistency by replaying committed reads from persisted write-ahead logs.
    • Built aggregated snapshot capacity statistics from scratch; p99 latency ≈ 30 ms.
    • Distributed storage
    • Crash consistency
    • Metadata integrity
    • C++

  3. Stony Brook University

    Graduate Research Assistant

    Stony Brook, NY

    Co-authored Rolis, published at EuroSys 2022, and implemented core mechanisms for replicated multi-core transactions.

    • Distributed systems research

  4. Talentica Software

    Senior Software Engineer

    Pune, India

    Data Science team, with three years across search, ML pipelines, file sync, chatbots, and network analytics, applying ML to network-based problems.

    Accomplishments

    • Led file-sync tool development: a delta-sync service in C++ substantially reduced sync-server load; also built a stateless updater, a multi-tenant client update framework, and a rate limiter that improved sync frequency 30%.
    • Built a financial-document search engine with +45% search relevancy, +30% online ontology enhancer, with an NLP pipeline for clustering, keyword extraction, and text classification.
    • Designed an ML model object-storage / delivery pipeline (gRPC, protobuf, Redis, RabbitMQ, S3): +40% deployment frequency, clients in Java / Python / Go / C++, weekly release cadence.
    • BLE indoor location: 95% region classification for static assets, 68% regression accuracy via RSSI triangulation + interference correction; a live asset-view portal added 70% correctness to site calibration.
    Full detail Hide detail

    More Talentica work

    • Network traffic estimation: an ensembled regression model for bandwidth prediction from SLA metrics and native Speedtest feedback.
    • Network traffic identification: a multi-modal ML model (regression ensemble + auto-encoders) classifying audio vs. video streams.
    • Real-time ML model deployment pipeline on Storm, Kafka, Python, Java.
    • AWS Athena data-dump tool: an ad-hoc Storm-topology client feeding a disk + batch pipeline (Java, Spark).
    • Single-cell identity classification: an auto-encoder network, 83% accuracy for cell-based print technology.
    • Data platforms
    • Machine learning
    • NLP / search
    • Distributed pipelines

  5. MediaTek

    Software Engineer

    Noida, India

    Multimedia team, focused on audio-player development for feature phones; led in-house stress-test tooling for Wi-Fi / Bluetooth / GPS.

    Accomplishments

    • Improved the audio-player playlist sub-module and integrated the Bluetooth stack into the MMI layer.
    • Built the SW-layer for a combo (GPS-Wi-Fi-BT) stress-test tool with an event-driven asynchronous architecture for the packet-data interface.
    • Embedded systems
    • Multimedia
    • C / C++

Research & systems

Ideas tested in code.

My academic work sharpened the same questions I bring to production: where contention appears, what failure means, and how correctness is verified.

Graduate Research Assistant · Stony Brook University · 2019–2020 · Advisor Prof. Shuai Mu

Replicating multi-core transactions without giving up performance.

Co-authored Rolis (EuroSys 2022) and implemented core mechanisms for its replicated multi-core transaction engine: asynchronous multi-process Paxos, checkpoint log truncation, and serializable replay. Built a header-only C++ logging library (~1200 LOC) and evaluation benchmarks profiled with mutrace and gprof.

Visit the research group

  1. Publication

    Rolis: a software approach to efficiently replicating multi-core transactions

    EuroSys 2022 · systems implementation & evaluation

  2. Patent

    Verification of metadata consistency across snapshot COW B+ tree logical maps

    US11573860B1

  3. Publication

    Learning to fingerprint the latent structure in question articulation

    R. Guntur, M. Kumar · IEEE ICMLA 2018 · pp. 73–80

  4. Teaching

    Teaching Assistant: Introduction to Data Science (CSE-351)

    Stony Brook University · with Prof. Martin Radfar · 2019

  5. Systems project

    Sharded, replicated, fault-tolerant key-value store on Raft

    CSE 535 Asynchronous Systems Lab

  6. Systems project

    Distributed MapReduce library with worker-failure recovery

    CSE 535 Asynchronous Systems Lab

  7. Kernel project

    Versioned stackable backup filesystem for the Linux kernel

    Stony Brook FSL · queue-based versioning + retention policy

  8. Kernel project

    AES encrypt / decrypt system call via the kernel crypto API

    Stony Brook FSL

  9. Project

    Mood- and activity-based music recommendation

    Data Visualization Lab

About

How I approach systems work.

I care about systems that remain understandable under pressure: explicit tradeoffs, useful observability, and designs that account for failure from the start.

I take complex systems from 0 → 1: turning ambiguous problems into clear architectures, building the critical path, and carrying them into production with measurable outcomes.

Languages
Python · C# · C++ · C · Java · Go
ML systems
Agent infrastructure · Model evaluation · Fine-tuning pipelines
Distributed systems
Inference serving · Load balancing & autoscaling · Transactional storage
Production
Latency & throughput · Fault detection & recovery · Observability · Azure

Contact

Let's talk about production AI infrastructure.

  1. Inference engines
  2. Rigorous evals
  3. RL systems at scale
Email LinkedIn GitHub Résumé