In this work, we build a crossbar model based on experimentally characterized device statistics in large crossbar arrays. We identified imperfections including statistical device relaxation, fluctuation, peripheral circuits, etc. The experimentally validated model is then used to co-optimize analog matrix multiplication and neural network applications. Specifically, we propose and implement defect-aware training and verify that the neural network trained with our algorithm can provide better accuracy and reliability when deployed on physical crossbars.