InstaDeep interview question

How do transformers work, what is the main drawback.