The eighty five percent rule for optimal learning
https://www.nature.com/articles/s41467-019-12552-4
https://github.com/bobUA/EightyFivePercentRule
Researchers and educators have long wrestled with the question of how best to teach their clients be they humans, non-human animals or machines.
Here, we examine the role of a single variable, the difficulty of training, on the rate of learning. In many situations we find that there is a sweet spot in which training is neither too easy nor too hard, and where learning progresses most quickly. We derive conditions for this sweet spot for a broad class of learning algorithms in the context of binary classification tasks.
For all of these stochastic gradient-descent based learning algorithms, we find that the optimal error rate for training is around 15.87% or, conversely, that the optimal training accuracy is about 85%.
We demonstrate the efficacy of this ‘Eighty Five Percent Rule’ for artificial neural networks used in AI and biologically plausible neural networks thought to describe animal learning.