This paper presents a tiny charge injection-Successive Approximation A/D converter to be integrated at the periphery of analog Matrix Vector Multiplication accelerators for Deep Neural Network inference. Derived from the ci-SAR ADC, this converter exploits a single charge injecting cell to minimize area and energy consumption. The ADC exhibits a signal-to-noise and distortion ratio of 30.5 dB, at 5 bits of nominal resolution. The energy per conversion is 86fJ, running at 34 MS/s, with a silicon area of 75um2, in 22 nm technology node. From the results of our analytical framework, an SRAM-based Analog in-Memory Compute array, including the proposed ADC at 5 bits of resolution, can achieve an energy efficiency of 1650 TOPs/W.