Extensive study on the effects of one of the most important design parameters in Remote Photoplethysmosgraphy (RPPG) systems, sliding window (SW) size, for a variety of algorithms, in order to quantify the trade-off between computational cost in time and accuracy in root-mean-squared-error (RMSE), using a standardized public database. We also studied how different face detection and region-of-interest selection affected these results. Finally, based on these, we came up with a new and simple metric that takes into account both computation and accuracy.