Friendly interviewers interesting work and problems. Would have been to work on LLM, would recommend applying if interested. Expect programming and maths questions. Possibly ML too. Not much else to say.
The technical round hit hard with a question to implement multi-head self-attention from scratch in PyTorch. I felt the pressure as I explained the time and memory complexity, but the real surprise came when I recalled a similar problem I had practiced on PracHub just days before. Following that, I had a casual chat about the role and my experience, and then wrapped up with a behavioral interview. Once everything was done, I received an offer, which I gladly accepted. Overall, a challenging yet rewarding experience.
Interview questions [1]
Question 1
Implement multi-head self-attention from scratch in PyTorch and explain the time/memory complexity, then describe how FlashAttention reduces memory usage
