We have witnessed the revolutionary progress of learned image compression despite a short history of this field. Some challenges still remain such as computational complexity that prevent the practical application of learning-based codecs. In this paper, we address the issue of heavy time complexity from the view of arithmetic coding. We make use of channel-adaptive codebooks that cover more appropriate ranges to reduce the runtime of compression. Experimental results demonstrate that both the arithmetic encoding and decoding can be accelerated while preserving the rate-distortion performance of learned compression model.