Coursera_Machine_Learning_Course_project
Project Goal
The goal of this project is to predict the manner in which they did the exercise.
http://web.archive.org/web/20161224072740/http:/groupware.les.inf.puc-rio.br/har.
This is the “classe” variable in the training set. You may use any of the other variables to predict with. You should create a report describing how you built your model, how you used cross validation, what you think the expected out of sample error is, and why you made the choices you did. You will also use your prediction model to predict 20 different test cases.
Background
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement – a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks.
One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants.
They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here: http://web.archive.org/web/20161224072740/http:/groupware.les.inf.puc-rio.br/har (see the section on the Weight Lifting Exercise Dataset).
The data for this project come from this source: http://web.archive.org/web/20161224072740/http:/groupware.les.inf.puc-rio.br/har.
Thanks for sharing!!!!!
Data
The training data for this project is available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv
The test data is available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv
Load Data
traindf <- read.csv(url("https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv"),header=TRUE)
dim(traindf)## [1] 19622 160testData <- read.csv(url("https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv"),header=TRUE)
dim(testData)## [1] 20 160Installing Packages and loading Libraries
- Loading libraries and dependencies
- library(caret)
- library(tidyverse)
- library(randomForest)
- library(e1071)
- library(scales)
## Loading required package: lattice## Loading required package: ggplot2## -- Attaching packages ------------------------------------------------------------------------------------------- tidyverse 1.3.0 --## v tibble 2.1.3 v dplyr 0.8.5
## v tidyr 1.0.2 v stringr 1.4.0
## v readr 1.3.1 v forcats 0.5.0
## v purrr 0.3.3## -- Conflicts ---------------------------------------------------------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
## x purrr::lift() masks caret::lift()## randomForest 4.6-14## Type rfNews() to see new features/changes/bug fixes.##
## Attaching package: 'randomForest'## The following object is masked from 'package:dplyr':
##
## combine## The following object is masked from 'package:ggplot2':
##
## margin##
## Attaching package: 'scales'## The following object is masked from 'package:purrr':
##
## discard## The following object is masked from 'package:readr':
##
## col_factorThe next code chunk Sets Seed for reproducibility
set.seed(123)Removing variables where N/A values is higher than 95% of observations
traindf <- traindf[, colSums(is.na(traindf)) < nrow(traindf) * 0.95]
dim(traindf)## [1] 19622 93Removing some variables not interesting in;
- Variables 1-7
- X
- user_name
- raw_timestamp_part_1
- raw_timestamp_part_2
- cvtd_timestamp
- new_window
- num_window
and all the variables with Nearly Zero Variance
traindf <- traindf[,c(-1:-7)]
# remove variables with Nearly Zero Variance
nzv_cols <- nearZeroVar(traindf)
if(length(nzv_cols) > 0) traindf <- traindf[, -nzv_cols]
dim(traindf)## [1] 19622 53Creating a validation dataset
inTrain <- createDataPartition(traindf$classe, p=0.7, list=F)
trainData <- traindf[inTrain, ]
validationData <- traindf[-inTrain, ]
remove(traindf)- So we have 3 datasets
- testData for testing the prediction model
- trainData for creating the prediction model
- validationData to validate the prediction model
Train couple of random forest models using different methods
control <- trainControl(method="repeatedcv", number =3, repeats = 3)
rf_model <- train(classe ~., data=trainData, method="rf", trControl=control)
print(rf_model)## Random Forest
##
## 13737 samples
## 52 predictor
## 5 classes: 'A', 'B', 'C', 'D', 'E'
##
## No pre-processing
## Resampling: Cross-Validated (3 fold, repeated 3 times)
## Summary of sample sizes: 9157, 9159, 9158, 9160, 9157, 9157, ...
## Resampling results across tuning parameters:
##
## mtry Accuracy Kappa
## 2 0.9876003 0.9843123
## 27 0.9888136 0.9858490
## 52 0.9825772 0.9779586
##
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was mtry = 27.control <- trainControl(method="cv", 10)
rf_model_1 <- train(classe ~., data=trainData, method="rf", trControl=control)
print(rf_model_1)## Random Forest
##
## 13737 samples
## 52 predictor
## 5 classes: 'A', 'B', 'C', 'D', 'E'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 12364, 12364, 12362, 12362, 12364, 12365, ...
## Resampling results across tuning parameters:
##
## mtry Accuracy Kappa
## 2 0.9924296 0.9904235
## 27 0.9923566 0.9903313
## 52 0.9867499 0.9832371
##
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was mtry = 2.Train a naive bayes model
control <- trainControl(method="cv", 10)
naive_bayes_model <- train(classe ~., data=trainData, method="naive_bayes", trControl=control)
print(naive_bayes_model)## Naive Bayes
##
## 13737 samples
## 52 predictor
## 5 classes: 'A', 'B', 'C', 'D', 'E'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 12363, 12364, 12364, 12364, 12363, 12362, ...
## Resampling results across tuning parameters:
##
## usekernel Accuracy Kappa
## FALSE 0.5003299 0.3838413
## TRUE 0.7392408 0.6669212
##
## Tuning parameter 'laplace' was held constant at a value of 0
## Tuning
## parameter 'adjust' was held constant at a value of 1
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were laplace = 0, usekernel = TRUE
## and adjust = 1.Train a Stochastic Gradient Boosting
control <- trainControl(method = "cv", 10)
GBM_model <- train(classe ~ ., data=trainData, method = "gbm", trControl = control, verbose = FALSE)
print(GBM_model)## Stochastic Gradient Boosting
##
## 13737 samples
## 52 predictor
## 5 classes: 'A', 'B', 'C', 'D', 'E'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 12363, 12362, 12363, 12363, 12363, 12364, ...
## Resampling results across tuning parameters:
##
## interaction.depth n.trees Accuracy Kappa
## 1 50 0.7535111 0.6876329
## 1 100 0.8172795 0.7687334
## 1 150 0.8525120 0.8133724
## 2 50 0.8533138 0.8142147
## 2 100 0.9061632 0.8812204
## 2 150 0.9325883 0.9147030
## 3 50 0.8968464 0.8693947
## 3 100 0.9409615 0.9252917
## 3 150 0.9607608 0.9503621
##
## Tuning parameter 'shrinkage' was held constant at a value of 0.1
##
## Tuning parameter 'n.minobsinnode' was held constant at a value of 10
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were n.trees = 150, interaction.depth =
## 3, shrinkage = 0.1 and n.minobsinnode = 10.The best model is rf_model_1 (Mtry=2) with an accuracy of 99.24% and kappa over .8 (0.9904), that is a random forest with 10 folds,lets have a look in deep about this model and also using this model to predict against validation dataset;
validation <- predict(rf_model_1, newdata = validationData)
confusionMatrix(validation, validationData$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1674 3 0 0 0
## B 0 1134 3 0 0
## C 0 2 1023 14 4
## D 0 0 0 950 3
## E 0 0 0 0 1075
##
## Overall Statistics
##
## Accuracy : 0.9951
## 95% CI : (0.9929, 0.9967)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9938
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 1.0000 0.9956 0.9971 0.9855 0.9935
## Specificity 0.9993 0.9994 0.9959 0.9994 1.0000
## Pos Pred Value 0.9982 0.9974 0.9808 0.9969 1.0000
## Neg Pred Value 1.0000 0.9989 0.9994 0.9972 0.9985
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2845 0.1927 0.1738 0.1614 0.1827
## Detection Prevalence 0.2850 0.1932 0.1772 0.1619 0.1827
## Balanced Accuracy 0.9996 0.9975 0.9965 0.9924 0.9968We can see after validating the model against the validation dataset that the accuracy is 99.51%, that is pretty close with our prediction, in fact is even better (99.24%)
# create a function to plot the confusion matrix in a clearer way
ggplotConfusionMatrix <- function(df){
mytitle <- paste("Accuracy", label_percent(accuracy = 0.001)(df$overall[1]),
"Kappa", label_percent(accuracy = 0.001)(df$overall[2]))
mytitle2 <- "Model Eficiency - Random Forest - 10 folds cross validation"
p <-
ggplot(data = as.data.frame(df$table) ,
aes(x = Reference, y = Prediction)) +
geom_tile(aes(fill = log(Freq)), colour = "white") +
scale_fill_gradient(low = "white", high = "steelblue") +
geom_text(aes(x = Reference, y = Prediction, label = Freq)) +
theme(legend.position = "none") +
ggtitle(mytitle2, mytitle)
return(p)
}
plot(rf_model_1)
conf_Matrix <- (confusionMatrix(validation, validationData$classe))
ggplotConfusionMatrix(conf_Matrix)
Applying the model to testData
predict.testData <- predict(rf_model_1, testData)
print(predict.testData)## [1] B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E