Video recording has become one of the key features of digital cameras nowadays. Though the resolution and quality of frames always draw the attentions, the stability issue of amateur videos is less addressed. The amount of data involved in video stabilization makes memory bandwidth a bottleneck. Hence existing hardware accelerators are usually designed using a simple approach of displacement compensation, which has limited performance. In this work, we design a special processor capable of conducting a better stabilization process in real time. The hardware techniques implemented include pipelining, parallel processing, memory interleaving, and data ordering. The chip is synthesized with TSMC 40nm technology. A 11X speedup can be achieved when compared to the software version.