Dysphagia is a symptom of many neurological disorders. Existing diagnosis systems are either invasive or require swallowing liquids, which are costly and harmful to humans. In this work, we design a smart dysphagia detection system based on speech signals. Rather than the voice data acquired by traditional microphones, we apply a bone conduction headset for vibration signal acquisition from the throat to get cleaner speech signals. After speech feature extraction, under-sampling is performed to deal with the imbalanced data problem, and principal component analysis is used for dimensionality reduction. In this paper, we construct an ensemble adaptive boosting classifier to detect the dysphagia patient. Experimental results show that the testing classification accuracy of the proposed system reaches 71.2 %. Sensitivity and specificity can reach 66.6 % and 76 %, respectively.