Diptanu Gon Choudhury

I work in the intersection of Distributed Systems and AI/ML. I have worked on Facebook's Machine Learning/AI training platform FBLearner, some larger scale inference services which serve NLP models for online inference, large scale data preparation and feature serving service and on training and optimizing PyTorch NLP models to run efficiently on servers. In the past I have built large scale distributed cluster schedulers Nomad and Titus at HashiCorp and Netflix. I have also built foundational cloud infrastructure software such as highly available service discovery systems, traffic routing services, RPC libraries, etc. Papers - [1] Cross Lingual LID using Self Supervised Learning - https://arxiv.org/pdf/2107.04082.pdf [2] Designing cluster schedulers for internet-scale services - https://queue.acm.org/detail.cfm?id=3199609 [3] XDP - Programable Data Path in the Kernel - https://www.usenix.org/system/files/login/articles/login_spring18_05_choudhury.pdf Talks - [1] Chaos Engineering and design patterns for building highly available services https://www.youtube.com/watch?v=sYlWtTbpHQI [2] Reliably shipping containers in a resource rich world using Titan https://www.youtube.com/watch?v=V3OfAATYksM [3] https://www.youtube.com/watch?v=gnNp0t2JAjg [4] https://www.youtube.com/watch?v=Blh608p2M1E [5] https://speakerdeck.com/diptanu/distributed-scheduling-with-apache-mesos-in-the-cloud