Persistent neural activity is observed in many systems, and is thought to be a neural substrate for holding memories over time delays of a few seconds. Recent work has addressed two issues. First, how can networks of neurons robustly hold such an active memory? Computer systems obtain significant robustness to noise by approximating analogue quantities with discrete digital representations. In a similar manner, theoretical models of persistent activity in spiking neurons have shown that the most robust and stable way to store the short-term memory of a continuous parameter is to approximate it with a discrete representation. This general idea applies very broadly to mechanisms that range from biochemical networks to single cells and to large circuits of neurons. Second, why is it commonly observed that persistent activity in the cortex can be strongly time-varying? This observation is almost ubiquitous, and therefore must be taken into account in our models and our understanding of how short-term memories are held in the cortex.