A developer released Event2Vec, an open-source library that learns embeddings of discrete event sequences (purchases, logs, clinical codes) as a running sum of event vectors, so any linear score on a history decomposes exactly into per-event terms. Training is self-supervised next-event prediction with a reconstruction term, no labels needed. It grew out of root-cause analysis of fault sequences at Berkeley Lab's Advanced Light Source and is backed by a NeurReps@NeurIPS workshop paper (arXiv 2509.12188).
An arXiv paper introduces Universal Multi-Modal Traceformer (UMT), a Transformer-based framework that incorporates heterogeneous context — numerical, categorical, textual, and metadata features — into next-event prediction on event logs. Experiments on 13 real-world event logs show improvements over existing approaches in both next-event activity and time prediction.
An arXiv paper introduces Universal Multi-Modal Traceformer (UMT), a Transformer-based framework that incorporates heterogeneous context — numerical, categorical, textual, and metadata features — into next-event prediction on event logs. Experiments on 13 real-world event logs show improvements over existing approaches in both next-event activity and time prediction.
A new arXiv paper presents exact solutions for how linear concept representations emerge during neural network training, providing a mathematical framework for the dynamics of abstraction. This work formalizes the linear representation hypothesis, which underpins interpretability methods like linear probes and activation steering, and could guide future training and control techniques.
A new arXiv paper presents exact solutions for how linear concept representations emerge during neural network training, providing a mathematical framework for the dynamics of abstraction. This work formalizes the linear representation hypothesis, which underpins interpretability methods like linear probes and activation steering, and could guide future training and control techniques.
A Medium article explains rotary position embeddings (RoPE) through hand calculations, building on a previous piece about sinusoidal absolute positional encoding. It aims to demystify how RoPE encodes position information in transformer models.
A Medium article explains rotary position embeddings (RoPE) through hand calculations, building on a previous piece about sinusoidal absolute positional encoding. It aims to demystify how RoPE encodes position information in transformer models.
A developer is prototyping RHEA, a ~1B-parameter reactive hypergraph event architecture, and considering combining it with NORD, a recurrent/spiking architecture, to create a non-Transformer language model. The hybrid aims to replace fixed Transformer stacks with sparse, event-driven computation.
A developer is prototyping RHEA, a ~1B-parameter reactive hypergraph event architecture, and considering combining it with NORD, a recurrent/spiking architecture, to create a non-Transformer language model. The hybrid aims to replace fixed Transformer stacks with sparse, event-driven computation.
A new framework, EvtGraph, aligns computation with temporal salience in multimodal time series by compressing sequences into event-level tokens under explicit budget constraints. It uses event-adaptive compression, node budget selection, and temporally constrained sparse graph reasoning to transform dense sequences into structured computation over salient events.
A new framework, EvtGraph, aligns computation with temporal salience in multimodal time series by compressing sequences into event-level tokens under explicit budget constraints. It uses event-adaptive compression, node budget selection, and temporally constrained sparse graph reasoning to transform dense sequences into structured computation over salient events.