Can a Neural Network Learn Exact Arithmetic?
A calculator multiplies two thousand-digit numbers and takes the remainder without thinking about it. Ask a neural network to do the same thing and it becomes an open research question, one that Terence Tao helped set as a competition. The answer has to be exact, because a remainder that is off by one is simply wrong. This is what I submitted to the SAIR Foundation’s Modular Arithmetic Challenge, why the obvious approach cannot work, and the one idea that made the difference.