This paper describes a miniature edge device that performs neural network inference with different exit options depending on available energy. In addition to the main-exit path, it provides an alternative, early-exit path that requires less computation and thus increase the number of inference operations for given energy. To compensate its degraded accuracy, the proposed device provides entropy as a confidence level for the early exit. The network is implemented with a custom low-power 180 nm CMOS processor chip and a 90 nm embedded flash memory chip and tested by images from CIFAR-10 dataset.