In this paper, we study the impact of padding in learned image compression. From the experimental results, padding causes serious performance drops. To compensate for padding effects, we propose a padding-aware training strategy, adapting networks to padded images during the training stage. In addition, we notice that images with different resolutions fit different padding modes. Based on this observation, we propose to conduct padding mode decision during the encoding stage through rate-distortion optimization. Experimental results demonstrate that both the padding-aware training strategy and the padding mode decision boost the compression performance of padded images.