In this work, we propose a hardware Trojan inserted into the mathematical operations of individual layers of CNN, which propagates erroneous operations in all the subsequent CNN layers that lead to misclassification. Five different attack scenarios with respect to each CNN layer are designed and evaluated. Our results for two CNN architectures show that in all the attack scenarios, additional latency is negligible (<0.61%), increment in DSP, LUT, FF is also less than 2.36%. Three attack scenarios do not require any additional BRAM resources, while in two scenarios BRAM increases which compensates with the corresponding decrease in FF and LUTs.