New method trims unnecessary variables from RL agent state representations, cutting inference time by up to 2.6x

Computer scientists have introduced a new technique for streamlining how reinforcement learning agents represent their surrounding environment. The core problem they tackle is identifying which observable variables truly matter for decision-making and which are simply noise.

The Challenge of State Representation#

In reinforcement learning, agents learn to maximize rewards by observing state variables and taking actions. However, not all observable information is equally important. Many environments provide redundant or irrelevant data that clutters the agent's state representation, slowing down both training and real-time execution. Selecting the right subset of variables can dramatically improve efficiency without sacrificing performance.

How TERC Works#

The new approach, called Transfer Entropy Redundancy Criterion (TERC), applies information theory to identify which variables actually influence an agent's policy during learning. Rather than relying on hand-crafted feature engineering, TERC measures the flow of entropy—or information—from observable state variables to the actions the agent selects. Variables that fail to transfer entropy to actions can be safely removed from the state representation.

The researchers developed an algorithm based on TERC that mathematically guarantees the exclusion of irrelevant state variables. Critically, the method is policy-dependent and algorithm-agnostic, meaning it works regardless of which learning method the agent uses.

Demonstrated Results#

Testing across multiple learning paradigms—tabular Q-learning, Actor-Critic methods, and Proximal Policy Optimization (PPO)—the team showed consistent gains. Reduced state representations achieved inference speedups of up to 2.6 times faster while maintaining decision quality. The method also improved retraining efficiency on the trimmed state space.

To validate their approach against existing feature selection techniques, the researchers first ran controlled experiments on synthetic data before evaluating performance in real decision-making tasks. They also formalized the state variable selection problem as a Bayesian network, offering a compact mathematical representation of how information flows from observations to actions.

This work addresses a practical bottleneck in deploying reinforcement learning systems where computational resources or latency constraints are critical.