Spiking neural networks (SNNs) may enable low-power intelligence on the edge by combining the merits of deeplearning with the computational paradigms found in the humanneo-cortex. The choice of neuron model is an open researchtopic. Many spiking models implement neural dynamics frombiology that involve one or more exponential decay functions.Previous work focused on accurate modeling of exponentialdecay functions on neuromorphic hardware. In this paper, weexplore the limits of error resilience in SNNs by aggressivelyapproximating their exponential decay functions. Three approx-imation techniques are presented and their hardware cost andinference accuracy on benchmark applications are compared. Wealso explore retraining for reduced inference accuracy as wellas programmable hardware for time constant parameters. Ourresults show resilience to lossy approximation, fast retraining,and a low energy consumption of 47 fJ per operation