Extension: Visualising KNN Regression 1D (k = 2)

4.6. Extension: Visualising KNN Regression 1D (k = 2)#

We can apply the same logic for different values of k, but it does get a bit trickier.

Let’s look at the case where k = 2. First we identify the neighbourhoods. If you look at the diagram below, this is the neighbourhood identified by the two red crosses.

../../_images/knn_flat_7a.png

Moving to the left of this neighbourhood will change the two closest neighbours as shown below.

../../_images/knn_flat_7b.png

Similarly, moving to the right of this neighbourhood will change the two closest neighbours as shown below.

../../_images/knn_flat_7c.png

You’ll have noticed that the edge of the neighbourhood correspond to midpoints between pairs of samples that are at least one apart.

../../_images/knn_flat_7d.png

Again we can draw the neighbourhoods. Note that the predicted values are the average of the 2 nearest neighbours.

../../_images/knn_flat_7e.png

Visualising the final result we get:

../../_images/knn_neighbourhoods_4.png

The same logic will apply for k = 3, and so on…

What you need to do is determine the neighbourhoods in which the neighbours don’t change, calculate the average of the neighbours, and that’s your prediction. As a result KNN models look like ‘steps’ where

  • The location of the steps correspond to when the neighbourhood changes.

  • The height corresponds to the average value of the neighbours.