This episode is for builders experimenting in AI. We dive deep into technical aspects of agent based workflows, data models, and interesting new fundamental research.
Madhav Singhal is a lead AI researcher at Replit focused on the frontier of what AI can enable for Replit users. In this episode, we dive into specific applications of models across code generation, testing, programming languages and more. We get fairly technical into how data maps to ML models and the types of agent based workflows that work today or may work in the future. We cover why focusing time on new model architectures may not be as relevant for founders to spend time on and how everything with new technology always maps to thinking about what solves the core problem efficiently. Enjoy this deep dive into the technical aspects of agent based workflows.
Where to Find Madhav:
Twitter: https://x.com/madhavsinghal_
LinkedIn: https://www.linkedin.com/in/ms337/
Where to Find Shomik:
Twitter: https://twitter.com/shomikghosh21
LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/
Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.
In this episode, we cover:
(00:40) – What does Replit do?
(02:40) – Tackling Data Bottlenecks and the Shape of the Data
(05:50) – Evolution of LLM Usage & Orchestration
(07:58) – Synthetic Data Use Cases
(11:25) – Background Behind Replit’s Program Repair Agent
(14:48) – Sampling Petabytes of Data a Day for ML
(16:06) – Handling Models Across Programming Languages
(17:12) – When to Pretrain Models and Use Smaller Models
(21:53) – Narrower Problems Means Less Hallucinations
(23:05) – Real-Time Inference
(25:51) – How Madhav Allocates His Research Time
(28:00) – Choosing Mixture of Experts or Model Merging Based on Product Use Case
(30:00) – Multi-Modal Applicability for Code
(33:25) – Why New Model Architectures Don’t Necessarily Benefit Users
(34:46) – Binary Embeddings Improving Memory & Performance for Agents
(36:18) – Agent Based vs CoPilot Approaches
(38:45) – Replit’s Culture & How They Ship So Fast
(41:57) – Simple vs Complex Agent Use Cases
(43:56) – New Paradigm: IDE Specific Workflow Models
(49:20) – Wrap Up
Show Notes:
Frontier AI Research & Agents in the IDE - Madhav Singhal (AI Engineer, Replit)