Deep neural networks have been recently applied to point cloud compression (PCC). Feature extractions implemented via deep neural networks are essential for the compression performance. Different from high level tasks like point cloud classification which homogenizes descriptors within same classes, PCC requires low level features discriminative for point-level 3D reconstructions. With this motivation, we first adopt to model the feature elements by Gaussians, and then propose a deep distribution-aware network (DDA-Net) to manipulate such distributions on-the-fly to favor the point cloud reconstruction with high fidelity. The proposed DDA-Net is successfully incorporated into an end-to-end PCC system.