It is generally accepted that robotic surgery has increased in popularity which has revolutionized the medical field over the past few decades. Despite many successes and development, there are still a lot of challenges what was remained in the surgeries such as the requirement of haptic feedback system has become a raising issue due to a lot of failure cases during robotic surgery processes, for example: suture breakage because of pulling action in knot-tying, slipped on a needle of grasping action or inadequate of force. In this paper, an image-based force sensing feedback system is presented. which can reduce instances of surgery failure without negatively impacting the surgeon’s performance. Visual images-based were employed to watching the distance changing of plasticity sensor during the time it was suffering an opposite reaction force from grasping action of a surgery robot arm. From the experiments, this visual images-based evaluation method can detect a minimum force value of 0.01 Newton. This system provides extraordinary accuracy of force sensing feedback and also hands controlling for a robot arm, the accuracy shows up to 99.96%.