The state-of-the-art methods employ deformable alignment to gather similar information from multiple neighborhood frames to enhance target frame quality. However, they always align multiple frames to the target frame simultaneously which brings repeat or useless information because of multiple and imperfect alignments. In this paper, we propose a recurrent deformable fusion method which considers the alignment quality distortion caused by time distance from the target frame. Compared with the previous multi-frame alignment approach, our method can avoid obtaining a lot of useless and repeat information. Experiment results confirm that our method achieves the state-of-the-art performance on the standard test sequences.