Backprop-free Pong: PC + distributional Hebbian plasticity vs. PPO: 57% vs. 59%, ~1500 lines from scratch [P]

Wanted to see how close a fully bio-plausible agent could get to PPO on Pong.

Setup

• Custom Pong environment (pygame, no gym)
• PPO baseline: paper-faithful, from scratch
• Hebbian agent: PPO policy replaced with Hebbian value estimation
  • engineered features → 61%
• BioAgent: Predictive Coding for feature learning + distributional Hebbian plasticity for value (Dabney et al. 2020) → 57% Zero backprop anywhere in the pipeline.

Key observations

1. The 2% gap is real but small. The bottleneck wasn't the lack of backprop because it was catastrophic forgetting under non-stationary opponent dynamics during self-play.
2. Distributional value encoding (à la Dabney) helped stability vs. a scalar Hebbian baseline, but not enough to match PPO under self-play.
3. Self-play exposed the plasticity–stability dilemma hard: Hebbian rules that adapt fast forget fast. This is the real wall for bio-plausible RL in non-stationary settings.

Not claiming novelty in the architecture as this is a from-scratch exploration of whether bio-plausible rules can handle a real RL task. Short answer: yes, mostly, with one clear failure mode.

Code: github.com/nilsleut/Biologically-Plausible-RL-Plays-Pong

Happy to answer questions about the PC implementation, the Hebbian value estimator, or the self-play setup.