We propose a novel hybrid neural network model based on multi-level attention fusion for multimodal DMR. The proposed model use convolutional neural networks and gated recurrent unit networks to extract temporal-spatial features from multimodal sensing signals and propose the multi-level attention fusion to explore the significant patterns over local and global periods. In addition, we design three different levels of fusion (early, late, and full) to explore the effects of different attention fusions on the model. Extensive experiments show that the proposed model achieves superior performance to the baseline methods, and multi-level attention fusion brings 6.17% gain to the F1-score.