How to make the most of your masked language model for protein engineering

Summary

Despite the recent release of numerous protein language models, relatively little work has addressed how to best sample from them to optimize desired biological properties. This paper fills that gap by proposing a flexible and effective sampling method for masked language models (MLMs), and by systematically evaluating models and methods both in silico and in vitro in the context of antibody therapeutics campaigns. The authors propose sampling with stochastic beam search, exploiting the fact that MLMs can efficiently evaluate the pseudo-perplexity of the entire 1-edit neighborhood of a sequence. Reframing generation as whole-sequence evaluation enables flexible guidance with multiple optimization objectives. Their extensive in vitro head-to-head evaluation for antibody engineering reveals that the choice of sampling method can have a substantial impact, motivating further research into this under-explored area.