Simple Linear Regression fits a line to the data. The reason it is "simple" is that it only works for two-dimensional data. This means that the line fitted to the data has only one feature. Here's what it could look like:
Source: Wikipedia
In this case, the x-axis could represent the input to the machine learning model and the y-axis could represent the output. By fitting a line through the data, we can now create new predictions with inputs we've never seen before by finding the point on the line that corresponds with those inputs. For instance, if our input was 35, our model would output ~10.
Now let's try to get a better intuition of how this model works. First, Let's remind ourselves what the equation for a line is:
$$ y=mx+c $$
$x$ and $y$ are the two variables that change. $x$ is independent and $y$ is dependent. However, their values do not determine the shape of the model. Instead, the model maps values of $x$ to values for $y$. $m$ and $c$, however, are the two numbers that determine the shape of the model. As a reminder, $m$ is the slope of the line, and $c$ is the y-intercept.
You may remember that parameters also determine the shape of the model, and so in this particular type of model, $m$ and $c$ are parameters**.** As a means to standardize our parameters, we'll rename $c$ to be $\theta_0$ and $m$ to be $\theta_1$, so that the equation is:
$$ y=\theta_0+\theta_1x $$
Now, for any set of two-dimensional data, our goal is to find optimal values for $\theta_0$ and $\theta_1$ such that they produce a line that fits the data well. We'll cover that in the next section.
5.3 Optimization Approaches: Gradient Descent and Ordinary Least Squares
<aside> ⚖️ Copyright © 2021 Code 4 Tomorrow. All rights reserved. The code in this course is licensed under the MIT License.
If you would like to use content from any of our courses, you must obtain our explicit written permission and provide credit. Please contact [email protected] for inquiries.
</aside>