Exploiting neural network sparsity is one of the most important directions to accelerate CNN executions. Plenty of neural network pruning techniques are proposed to exploit neural network sparsity, where spatial-wise pruning is quite effective for input image. However, previous spatial-wise pruning methods need nontrivial hardware overhead for dynamic execution, due to layer-by-layer binary sampling and online execution scheduling. This paper proposes a structured configured, spatial-wise pruning technique, which could measure region importance for multiple levels. Numerous computation will be saved by skipping those unimportant region. By using a unified importance map, the computing graph could be compiled in advance to make it quite efficient for a wide range of vision tasks. On image classification task, the method can have around 50% fewer top1 accuracy drop than previous spatial wise pruning methods at similar sparse level. On super resolution and image deraining task, the method can bring 5x to 19x acceleration while causing neglectable effect on reconstruction quality. A hardware architecture supporting this accelerating technique is also included.