This paper presents a novel architecture for the Singular Value Decomposition (SVD) algorithm. The architecture embraces the reductions offered by the use of Approximate Computing (AxC) as a trade-off between complexity and accuracy. A shallow Neural Network (NN) consisting of three layers is used to compute the SVD of an input matrix, offering a comparable Mean Squared Error (MSE) with exact computations. The NN is implemented using High-Level Synthesis (HLS) on a Virtex- 7 FPGA device. When compared to an exact implementation of the SVD algorithm, the proposed architecture achieves a computational speedup between 5x and 19x with an average reduced hardware area of up to 80% with a noticeable 6 reduction in the DSP usage.