To determine which line fits the data most optimally, we first need to define the criteria in which we evaluate a line. In machine learning, this is called the cost function. The cost function will compare a given line with the data and output a number that represents how well the line fits it.
A line that poorly fits the data will have a very high cost function:
Whereas a line that fits the data well will have a low cost function:
So how can we go about creating a cost function for linear regression? The most intuitive approach is to find the average vertical distance between the line and each data point:
As you can see, the vertical distances are on average pretty large, which is why the first line has a rather big cost function. On the other hand, the vertical distances are pretty small on average for the second line, leading to a low cost function:
Mathematically, we can define the cost function we've just discussed as follows:
$$ J=\frac{1}{n}\sum^{n}_{i=1}|\text{pred}_i-y_i| $$
Here, $n$ is the number of data points there are, $y_i$ is the y-value of the $i$-th point, and $\text{pred}_i$ is the y-value of the line at the x-value of the $i$-th point. By finding the sum of $|\text{pred}_i-y_i|$ for every single data point, we sum up all of the distances between the points and the line. Then, by dividing that sum by $n$ we find the average distance between each point and the line — hence the cost function we originally wanted.
While this cost function is probably the most intuitive and seemingly the most effective, statisticians have found the following cost function to be even more effective for linear regression:
$$ J=\frac{1}{n}\sum^{n}_{i=1}(\text{pred}_i-y_i)^2 $$
By squaring $\text{pred}_i-y_i$ instead of taking its absolute value, we punish the model a lot more for very large distances between each point and the line. Thus, a line that optimizes this cost function will ensure a good fit by prioritizing minimizing especially large errors.
We will be using the above cost function for linear regression. That means that when we try fitting a line to any 2-dimensional dataset, we'll be looking for a line such that $J$ is minimized.
To actually find a line that minimizes the cost function, we can use a method called gradient descent. Gradient descent is an incredible optimization method courtesy of statistics.