Alzheimer’s disease (AD) is a progressive brain disorder affecting millions of people worldwide. An accurate diagnosis of AD plays a significant role in identifying the progression of the disease at its prodromal mild cognitive impairment (MCI) stage. In this paper, we propose a lightweight deep model using separable and attention-based convolution operations to classify the patients into diagnostic groups, AD vs. normal control (NC) or progressive MCI (pMCI) vs. stable MCI (sMCI), with high accuracy, using MRI data. Experimental results show that compared to existing deep models, the proposed method provides significantly improved classification accuracy with reduced number of parameters.