The goal of this paper is to find hardware supporting a broad suite of NNs and to construct NNs which can be efficiently deployed on broad set of hardware platforms. This paper, therefore, brings 3 contributions: 1) an assessment methodology across a suite of networks and architectures; 2) deployment of this analysis for a set of architecture on popular networks for the ImageNet task; 3) derivation of insights from this study on characteristic of optimal architectures for modern NN models, and characteristic of optimal network topologies for modern processor architectures.