With supervised learning, the prediction equation is an important step required to perform both training and testing. So let's dive into it!
First of all, we have to consider vectorization, an important technique used to significantly speed up calculations. This is the standard practice in Machine Learning, and you should at least theoretically understand how it works.
Vectorization just means that the lists of features and parameters will be represented as matrices. For example, if we had the following features when trying to predict the price of a house:
We would write this as a matrix $X$, called the feature matrix, where the columns are various features, and the rows are different examples/observations. Similarly, we have a weight parameter assigned to each feature. We would put all of these parameters together in a vector called $\theta$.
Given that the vector $y$ represents the predictions of the ML model for each input, it is clear that some combination between $\theta$ and $X$ produces $y$. Before we decide what operation this is, let's consider the dimensions of $X$ and $\theta$.
The matrix $X$ has $m$ rows and $n$ columns, where $m$ is the number of observations or examples, and $n$ is the number of features.
The vector $\theta$ has $m$ rows and 1 column. Each row contains the weight $\theta_i$ for the corresponding feature $x_i$.
Therefore, by performing the matrix multiplication $\theta^TX$, we multiply our transposed weights (which just transforms $\theta$ into a vector with 1 row and $m$ columns) by the feature values. We have to transpose $\theta$ so that we are able to multiply it with $X$. This produces the model's predictions, which we can then evaluate or improve on, depending on what stage of the ML process we are on.