For regression tasks, the most common cost functions are Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). I'll be using MAE here. It is defined as follows:
Very simply, MAE is the average unsigned error for all examples in the dataset. It's very similar to RMSE; however, we use the absolute value of the difference between y and instead of the square root of the average squared error: