Naïve Bayes Classifier. Neural Network. Benchmarks. Gradient-Based Learning Applied to Document Recognition by LeCun, Y., Bottou, L., Bengio, Y., Haffner, ...

浅色棱发布于2018/06/13

注脚

展开查看详情

1.Analysis of Classification Algorithms In Handwritten Digit Recognition Logan Helms Jon Daniele

2.Classification Algorithms Template Matching Naïve Bayes Classifier Neural Network

3.Benchmarks Gradient-Based Learning Applied to Document Recognition by LeCun, Y., Bottou , L., Bengio , Y., Haffner , P. Comparison of Machine Learning Classifiers for Recognition of Online and Offline Handwritten Digits by Omidiora , E., Adeyanju , I., Fenwa , O.

4.MNIST Training Set: 60,000 samples Test Set: 10,000 samples Accuracy: Number of correctly guessed test samples/ 10,000

5.Naïve Bayes classifier

6.Naïve Bayes Classifier Each pixel value (on/off) is independent of any other pixel value​ Each pixel has a probability associated with being on or off in any given digit class​ The probability of each pixel is used to determine the probability of an unknown digit being classified in one of the known classes

7.Naïve Bayes Classifier Training set: 60000 digits​ Test set: 10000 digits​ Success rate:​ Abysmal: 08.13% correct classification rate​ Benchmark:​ WEKA: Multimodal Naive Bayes: 83.65%​ 08.13% <<<<<<<<< 83.65%

8.Naïve Bayes Classifier Challenges:​ Pixel probabilities change according to the shape of the digit​ Are pixels the best feature set by which to compare different digits?​ Input size:​ 28x28 digit image results 786 pixels​ Requirements for matrix manipulation

9.Naïve Bayes Classifier Improvements​ Discarding extraneous pixel data​ Pixel values are mainly contained in a 20x20 matrix​ Using a binary pixel value vs a range of pixel values (0-255)​ Edge detection​ Incorporate feature extractor(s) and evaluate images based on those features

10.Neural network

11.Neural Network Type: Feed Forward Training: Back-propagation algorithm Response Function: Architectures: Name Input Layer Hidden Layer Output Layer NN300 784 300 10 NN1000 784 1000 10

12.Training

13.NN300 Training time: ~17 hours (~52 mins/epoch) Learning rate: Epoch Rate 1, 2 0.0005 3, 4, 5 0.0002 6, 7, 8 0.0001 9, 10, 11, 12 0.00005 13, 14, 15, 16, 17, 18, 19, 20 0.00001

14.NN1000 Training time: ~2.5 days (~3 hrs/epoch) Learning rate: Epoch Rate 1, 2 0.0005 3, 4, 5 0.0002 6, 7, 8 0.0001 9, 10, 11, 12 0.00005 13, 14, 15, 16, 17, 18, 19, 20 0.00001

15.Results After 20 epochs Network Accuracy Benchmark 1 95.30% NN300 75.82% Benchmark 2 75.12% Network Accuracy Benchmark 1 95.50% NN1000 -

16.Benchmark 1 95.30% On MNIST test set as is. 96.4% Generated more training data by using artificial distortions 98.4% When using deslanted images

17.Future Work Further training of NN300 with the MNIST test set has increased accuracy to 84.01% Experiment with hidden neuron count and multiple hidden layers Research other types of neural networks