Polyps are one of the major causes of colorectal cancer. Polyp Segmentation is a fundamental task for computer aided gastrointestinal disease detection. Nowadays, it is very essential to develop an intelligent system for early diagnosis and detection of polyps which could result in successful treatment. Unlike the existing deep learning segmentation methods which required huge amount of labeled data, this paper presents a semi-supervised segmentation method called SemiSegPolyp based on the use graph signal processing (GSP) which requires few labeled data. The proposed approach includes many steps: instance segmentation, texture polyp features for the nodes of the graph, graph construction using K-nearest neighbors, and semi- supervised semantic segmentation based on the Total Variation Minimization tool. We evaluate our method on two popular polyp datasets: CVC-ClinicDB and Kvasir-SEG. SemiSegPolyp outperforms semi-supervised and supervised methods although the use of small amount of labeled data