The Ensemble of Memory and Now