In safety-critical applications, the reliability of CNNs becomes a priority. To reduce the cost of protecting CNNs for resource-limited embedded systems, we proposed to replace a strong CNN with an ensemble of pruned versions, and designed a combiner to exclude faulty base learner from the ensemble. VGG16 is taken as a case study to demonstrate the idea. Experiment results reveal that the ensemble of pruned VGG16 networks could achieve higher reliability than the TMR protected VGG16 with an overhead lower than double modular redundancy. Complementarity analysis between different pruned networks has been performed to explain the reliability improvement.