New algorithm maps protein mutations to understand how language models parse biological sequences

Researchers have introduced a novel approach to decoding how machine learning systems should parse protein sequences—by identifying the fundamental units that make up the protein "language" and tracking how they evolve through mutation.

The method, called PUMA (Protein Units via Mutation-Aware Merging), borrows an insight from linguistics: just as words in human languages have roots that morph into different forms, protein units can take on various forms when genetic mutations occur. By mapping these relationships, the algorithm creates families of functionally related protein segments that behave consistently during evolution.

How PUMA learns protein units#

PUMA works by analyzing protein sequences and using substitution matrices—charts that score how likely one amino acid is to replace another during evolution—to group protein units into lineages. Rather than treating each variation as independent, the algorithm recognizes that certain mutations preserve the original unit's role and significance. These grouped units form what researchers call a "genealogy" of related variants.

Better predictions of harmful mutations#

A key finding from the research is that mutations staying within a PUMA family tend to be less disruptive than standard substitution matrices would suggest on their own. This suggests the algorithm captures something real about protein structure and function that goes beyond simple amino acid swap statistics. The genealogies also improved how well molecular functions were represented compared to analyzing units separately.

Practical advantages for AI models#

When used as a tokenizer—the component that breaks sequences into processable chunks—PUMA matched the performance of existing methods on downstream machine learning tasks. The benefit came in efficiency: grouping units into families shrunk the embedding table size and accelerated training time, making the approach practical for real-world deployment.

Toward understanding the protein language#

The work represents progress on a fundamental question: what actually constitutes the "vocabulary" that biological language models should recognize? By grounding the vocabulary in evolutionary and mutational principles rather than arbitrary divisions, PUMA offers a more biologically plausible framework for how language models might best interpret protein sequences.