This work evaluates, for the first time, the effect of permanent faults in the register file of a GPU and the impact of several performance optimizations on the DNN\'s reliability. The reliability evaluation relies on software-based fault campaigns deployed on the NVIDIA RTX 3060Ti GPU with Ampere architecture. The results show that setting up optimization configurations, with reduced usage of the number of registers/thread and computations on the Stream Processors only, can improve the reliability, against permanent faults in the register file, by up to 20% for most of the evaluated DNNs