In this paper, we present a novel NN based video coding framework by leveraging the supervised trained NN models for multiple modules in the hybrid coding framework, from the predictive coding to the in-loop filtering. Specifically, NN based intra prediction models the non-linear mapping from contextual pixels to the predictions. The inter prediction efficiency is enhanced by introducing a virtual reference frame (VRF) network. The convolutional neural network based loop filtering (CNNLF) with discriminative model selection exploits the texture adaptivity. The experimental results show that the combined three NN coding tools reveal that around 13% YUV BD-rate reduction could be obtained compared with AVS reference software HPM13.0. The proposed framework opens novel sights for next generation video coding from the intelligent coding perspective.