Researchers present IDiom, an autoregressive protein language model trained on 54 million predicted intrinsically disordered protein regions (IDRs) from the AlphaFold Database, alongside RL-SAE, a reinforcement learning post-training method that rewards sequences activating specified sparse autoencoder features. The approach enables interpretable, composable design of IDRs, which structure-based methods and full-sequence protein language models handle poorly. On eight IDR design tasks, RL-SAE substantially outperforms activation steering and supervised fine-tuning on targeted feature activation and predicted functional outcomes.
A new arXiv paper describes a failure mode in inference-time guidance of protein language models: strong guidance pushes activations off the natural manifold, producing sequences that are hard to fold. The authors identify a cheaply detectable signature of this collapse, which could help practitioners avoid it.
A new arXiv paper describes a failure mode in inference-time guidance of protein language models: strong guidance pushes activations off the natural manifold, producing sequences that are hard to fold. The authors identify a cheaply detectable signature of this collapse, which could help practitioners avoid it.
Researchers introduced a framework combining sparse dictionary learning with causal intervention to extract, validate, and causally test interpretable features in genomic foundation models. They trained top-k sparse autoencoders on hidden activations of two architecturally distinct models, Nucleotide Transformer and another, to identify transcription-factor binding features. The method provides a principled procedure to verify that apparent concepts inside models are real rather than artifacts of sequence composition.
Researchers introduced a framework combining sparse dictionary learning with causal intervention to extract, validate, and causally test interpretable features in genomic foundation models. They trained top-k sparse autoencoders on hidden activations of two architecturally distinct models, Nucleotide Transformer and another, to identify transcription-factor binding features. The method provides a principled procedure to verify that apparent concepts inside models are real rather than artifacts of sequence composition.
SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
A paper introduces CODesign, a de novo protein binder co-design framework that jointly generates sequences and structures with improved cross-modal consistency. It combines a consistency-distilled dimer dataset (~105,000 dimers), a multimodal joint flow model over sequences, backbones, and atomic configurations, and a consistency-aware joint resampling strategy. CODesign reports state-of-the-art in silico success rates on protein- and ligand-target binder design, with code, weights, and dataset to be fully open-source.
A paper introduces CODesign, a de novo protein binder co-design framework that jointly generates sequences and structures with improved cross-modal consistency. It combines a consistency-distilled dimer dataset (~105,000 dimers), a multimodal joint flow model over sequences, backbones, and atomic configurations, and a consistency-aware joint resampling strategy. CODesign reports state-of-the-art in silico success rates on protein- and ligand-target binder design, with code, weights, and dataset to be fully open-source.
A new arXiv paper establishes a three-stage framework to empirically study inference-time sampling strategies for multimodal protein language models (pLMs), evaluating vanilla sampling, classifier-free guidance, and reward-guided beam search across three pLMs and four tasks. The work shifts focus from training to inference, offering practical guidance for improving generation performance.
A new arXiv paper establishes a three-stage framework to empirically study inference-time sampling strategies for multimodal protein language models (pLMs), evaluating vanilla sampling, classifier-free guidance, and reward-guided beam search across three pLMs and four tasks. The work shifts focus from training to inference, offering practical guidance for improving generation performance.