I build the RL Infrastructure and Applied AI systems that keep RL honest.

Shipped product & organizational adoption

• Owned review-model integration, deployment coordination, telemetry, and production rollout for an RL-trained model shipped to PowerPoint customers.
• Built a PowerPoint-native execution platform adopted beyond the immediate team to generate 100M+ documents for model pre-training.
• Migrated per-datapoint stateful VMs to stateless session pools: valid-sample rate ≈40% → ≈99%, RL-loop time ≈7 days → 24–36 hours, environment acquisition 2–5 min → ≈100 ms, and per-edit payload ≈16 MB → ≈100 bytes.

RL reward signal & evaluation

• Built LLM-as-judge reward grading for multi-turn agent trajectories.
• Built a document-render-to-image visual grading pipeline; ETag optimistic concurrency dedupes rewards across parallel graders.
• Drove an org-wide checkpoint pre-screening gate; RL candidates simulated before end-to-end integration, cutting compute wasted on non-viable checkpoints.
• Established the first per-slide latency profile (33–75s/slide), isolating LLM reasoning (7–30s) from tool execution (5–8s/turn).

Multimodal RL environment & tool execution

• Integrated an image-generation engine as an agent-callable tool, extending the agent's action space to a new modality.
• Owned the tool-execution stack: client-to-daemon architecture, per-trajectory VM provisioning, async multimodal pipeline; held P50 < 60s / P95 < 120s / P99 < 180s at 7,000 containers.
• Unified Anthropic Claude, Azure OpenAI (incl. O-series), and internal serving endpoints behind one provider-adapter interface.

Production platform, scale & observability

• Engineered the platform: nginx-balanced Python orchestration over a .NET/COM PowerPoint execution tier, Redis store + queue, 20+ containers on ACI.
• 5.7x throughput / 7.8x worker utilization (12% → 94%): base64 upload elimination, COM-pool redistribution, CPU-quota cache-thrash fix. 200-file batch 285s → 49.8s.
• Cleared two RL-scale blockers (platform throttling at 1.4M req/10 min; VM-lifecycle leak blocking 20k-VM parallelism); end-to-end OpenTelemetry/Jaeger tracing, per-turn attributable.

ML platform & fine-tuning

• Built PowerPoint's first end-to-end fine-tuned model workflow, covering synthetic-data generation, batched data preparation, secure deployment, inference, and workflow integration.
• Authored a model-first platform architecture (ingestion, grading, eval, observability) adopted as org blueprint.
• Designed a multi-turn SFT data strategy: three-phase curation, full training schema, DAG execution cutting agent latency ≈100s → 37s.

Technical leadership & team enablement

• Mentored five engineers and a summer intern through architecture reviews, code reviews, and production debugging; the intern shipped cloud-hosted grading and tagging tools.
• Led service and ML-system design walkthroughs, documentation, and engineering sessions that accelerated onboarding and distributed operational knowledge across the team.

More Talentica work

• Network traffic estimation: an ensembled regression model for bandwidth prediction from SLA metrics and native Speedtest feedback.
• Network traffic identification: a multi-modal ML model (regression ensemble + auto-encoders) classifying audio vs. video streams.
• Real-time ML model deployment pipeline on Storm, Kafka, Python, Java.
• AWS Athena data-dump tool: an ad-hoc Storm-topology client feeding a disk + batch pipeline (Java, Spark).
• Single-cell identity classification: an auto-encoder network, 83% accuracy for cell-based print technology.