Federated learning has successfully mitigated the privacy concerns for deep learning over distributed data, but suffers from performance decay led by the non-identical distribution of data. In this paper, we propose a novel framework, namely FedExg, to improve federated learning for non-IID data with model exchange. Two strategies, i.e., FedExg-S and FedExg-A, are developed to realize model exchange in a privacy-preserving fashion. FedExg-S randomly shuffles the models for broadcasting, while FedExg-A improves FedExg-S with partial aggregation to avoid model inversion attack. Theoretical analysis demonstrates that FedExg achieves lower regret bound than traditional FedAvg. Experimental results show that FedExg improves image classification tasks on MNIST and CIFAR-10 dataset.