Health problems related to stress are increasing globally and significantly affect the mental health, quality of life of human beings. Continuous suffering from stress may lead to serious psychological and physical health problems. But no effective and reliable stress detection methods are still available. In this paper, a novel wearable device is designed to measure electroencephalogram (EEG) and electrocardiogram (ECG) simultaneously in a non-invasive approach. This system includes an analog front end integrated with a machine learning-based digital backend processor for mental stress prediction using only 3 electrodes. A PCB prototype is developed using the commercial off-the-shelf components. The developed prototype shows excellent noise performance of 0.1μVrms and predicts the mental stress with a classification accuracy of 92.7%. The proposed system is lightweight and easily wearable (behind the ear). The data is acquired from 25 participants for different stress scenarios including the arithmetic tests and Stroop color-word test.