The continuing effect of COVID-19 pulmonary infection has highlighted the importance of machine-aided diagnosis for its initial symptoms such as fever, dry cough, fatigue, and dyspnea. This paper attempts to address respiratory-related symptom, cough detection using low power scalable software and hardware framework. We propose CoughNet, a flexible low power CNN-LSTM processor that can take audio recordings as input to detect cough sounds in audio recordings. We analyze the three different publicly available dataset and use those as part of our experiment to detect cough sound in audio recordings. We perform windowing and hyperparameter optimization on the software side with regard to fitting the network architecture to the hardware system. The hardware prototype is designed to handle different numbers of processing engines for parallel processing using Verilog HDL on Xilinx Kintex-7 160t FPGA with hardware scalability extension. The proposed implementation of hardware has a low power consumption of o 290 mW and energy consumption of 2.12 mJ.