Fast Learning Requires Good Memory: A Time-Space Lower Bound for Parity Learning

We prove that any algorithm for learning parities requires either a memory of quadratic size or an exponential number of samples. This proves a recent conjecture of Steinhardt, Valiant and Wager [SVW15] and shows that for some learning problems a large storage space is crucial. More formally, in the problem of parity learning, an unknown string x ∈ {0, 1}n was chosen uniformly at random. A learner tries to learn x from a stream of samples (a1, b1), (a2, b2) . . ., where each at is uniformly distributed over {0, 1}n and bt is the inner product of at and x, modulo 2. We show that any algorithm for parity learning, that uses less than n2 25 bits of memory, requires an exponential number of samples. Previously, there was no non-trivial lower bound on the number of samples needed, for any learning problem, even if the allowed memory size is O(n) (where n is the space needed to store one sample). We also give an application of our result in the field of bounded-storage cryptography. We s...

• Type: Journal
• Year: 2016
• Where: ECCC
• Authors: Ran Raz
• Keywords: ECCC 2016