Research questionHow can pretrained transformer embeddings capture peptide properties from scarce labels without distorting their geometry?Scarce labeled peptide data make it difficult to extract task-relevant signals from pretrained embeddings. Adaptation can either reshape the original representation substantially or fail to capture properties needed for downstream design tasks.