The graph convolutional network (GCN) based 3D point cloud semantic segmentation (PCSS) processor for mobile devices is proposed. GCN based 3D PCSS requires a lot of computation, making it unsuitable for real-time operation in mobile devices. For real-time 3D PCSS on mobile devices, this paper proposes two key features: 1) a sparse grouping based dilated graph convolution (SG-DGC) which reduces 71.7% of the overall computation of GCN by simply dividing input point cloud into multiple sparse point cloud. 2) group-level pipelining which improves low pipeline utilization due to the computation imbalance of GCN. Finally, the proposed GCN processor is simulated in 65 nm CMOS technology and occupies 4.0 mm2. The proposed processor consumes 176mW and shows 54.7 frames-per-second (fps) for the 3D point cloud semantic segmentation of indoor scene with 4k points.