Implementation of K-nearest Neighbor Classification Model
Installing Panda, numpy, scipy, sklearn libraries
- Go to terminal in Pycharm
- Type pip install pandas
- Or you can install all by going to Settings -> Project Interpreter -> +-> sklearn -> Install Package
In pattern recognition and machine learning, k-nearest neighbors (KNN) is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure (e.g. distance). KNN is a non-parametric method where the input consists of the k closest training examples in the feature space. The output is a class membership. An object is classified by a majority vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors (k is a positive integer, typically small). If k = 1, then the object is simply assigned to the class of that single nearest neighbor.
KNN Classification Model
The algorithm for KNN classification model is given below:
|Input: Training data X |
Training data labels Y
Sample x to classify
Output: Decision yp about sample x
fori <-– 1 to m do
Compute distance between training sample Xi and unlabeled sample x i.e. d(Xi, x) end for
Compute set I containing the indices for the k smallest distances d(Xi, x)
Compute the decision class yp by measuring the majority label Y from I
Fit does training on Training Data (X_train, Y_Train)
Predict does testing on Test data (X_test) and predicts outputs
Write a program to implement KNN classifier and classify given vector. (for k = 3)
Modify Task 1 and perform it for all odd values of k from 1 to 10.
- Find accuracy for each value of k and display.
- Find the best k which gives highest value of accuracy
- Also compute confusion matrix for the best value of k.
Implement KNN algorithm yourself in python for Iris Dataset without using built-in KNN classifier library.
- Load dataset
- Split dataset into test and train sets
- Perform KNN algorithm to make predictions for k=5
- Compute accuracy and confusion matrix