Get Started
Home
Topics
Search
Library
Research questionHow can fully time-to-first-spike spiking language models encode normalization and attention while preserving language-modeling quality?In time-to-first-spike coding, each neuron emits at most one spike within a time window, making operations such as layer normalization and attention difficult to represent. Supporting these operations is necessary for a fully time-to-first-spike language model that retains conventional language capabilities.
AI
Inference Optimization
Machine Learning
Natural Language Processing
Neural and Evolutionary Computing
Research Paper
Latest papersRecent research connected to this question, newest first.Large Language Models with At Most One Spike per NeuronThe study uses end-to-end trained spiking language models with encodings for embeddings, layer normalization, attention-related operations, and dropout. Evidence covers BERT- and GPT-2-based models, including a 1.5-billion-parameter model, on natural language understanding, commonsense reasoning, and language-modeling perplexity; energy results are spike-count estimates under a cost model rather than neuromorphic-hardware measurements.research paper · Sep 4, 2026
Related questions
How can sparse autoencoders capture language-model features that persist across token sequences?How can hybrid language models aggressively quantize Gated DeltaNet layers while preserving quality and long-context behavior?How can language models sparsely access a growing internal state without limiting it to fixed-size memory?How can language-model attention remain reliable beyond its training context?