Recently, there is an increased demand for high-linearity applications, where the requirements on the baseband Analog-to-Digital Converter (ADC) Spurious Free Dynamic Range (SFDR) represents a challenge. Most ADC architectures suffer from element mismatch which limits the linearity, and require significant calibration, whose resolution is ultimately limiting the performance. The exception is the single-bit Delta-Sigma Modulator (DSM), where the internal Digital-to-Analog Converter (DAC) is intrinsically linear and ideally no calibration is required, making it the architecture of choice in recent high linearity implementations. However, when the Over Sampling Ratio (OSR) is constrained to low values, the modulator linearity becomes intrinsically limited by the quantized operation of the modulator, even with perfectly linear DAC and ideal components in the loop filter. In this article we focus on this limitation of linearity by the quantizer, discuss how to best model it, and evaluate what are the key elements the designers have to master in order to design for best linearity.