First, a new CNN model – LightPose is proposed for an efficient and lightweight processing of human pose estimation. Second, by using a quantization-aware training method, an 8-bit quantization is conducted on LightPose with an only 0.5% drop in average precision. Finally, with specially designed computing engines and pipelined modules, we build LightPose on a Xilinx xc7k325t FPGA together with a RISC-V CPU for system management and external communications. Experiments show that our design achieves 411.6 FPS in speed and 0.546 in average precision, surpassing the existing peer works in both processing rate and power efficiency.