Deep convolutional neural networks have recently emerged as a state-of-the art tool in detection of seizures. However, neural networks are susceptible to confounding artifacts commonly present in EEG signals and are notoriously difficult to interpret. We present a neural-network based algorithm for seizure detection which leverages recent advances in information theory to construct a signal representation containing the minimal amount of information necessary to discriminate between seizure and normal brain activity. We show our approach automatically learns representations that ignore common signal artifacts and which encode medically relevant information from the raw signal.