Through HAR or Human Activity Recognition, you can utilize artificial intelligence to discern human activities. With the help of smart devices (such as smartwatches), you will be able to record human activities and generate raw data. Multiple sensors that are installed in the smart devices create signals whenever an individual carries out any kind of action.
Some of the sensors that continuously collect information include magnetometers, accelerometers, gyroscopes, etc. The application of Human Activity Recognition is vast. It can help sick people get assistance and at the same time, you can also analyze the skills of athletes in the sports industry.
Human Activity Recognition can be mainly divided into two separate categories: Mobile sensors and Fixed sensors. In this article, we will discuss how raw data produced by sensors are used to recognize human activities.
Some of the activities that we will be anticipating are:
- Sitting
- Walking
- Standing
- Downstairs
- Upstairs
- Jogging
To recognize the above-listed human activities we will make use of CNN or Convolutional Neural Network and LSTM or Long Short-Term Memory.
An Overview of LSTM-CNN
You may think that we can use the machine learning methods or basic neural network method instead of LSTM-CNN to perform the above-mentioned activity but there are several reasons why you should opt for the LSTM-CNN model.
For analyzing the human activity recognition tasks machine learning methods are heavily reliant upon heuristic manual feature extraction whereas the basic neural network method will not provide you with the precision you will receive from a deep neural network (DNN) method.
A deep neural network with the combination of CNN and LSTM can easily extricate and categorize activity features using very simple parameters and this entire process simplifies the feature extraction.
Performing human activity recognition with CNN-LSTM
One of the ideal datasets for this project would be the WISDM dataset which has around 1,100,000 samples. We will be utilizing this dataset and extracting its features using different data preprocessing techniques. Then we will be feeding the data to the model to train it and find out the reliability of the model.
Libraries to import
First, we will have to import all of the important libraries that we will require for the project. We can make use of different frameworks such as Keras, Sklearn, NumPy, TensorFlow, and SciPy to build our model and process the data. For loading the data we can use the Pandas module and for the data visualization process, we will use Matplotlib.
Dataset
Suppose we are using raw data given by an accelerometer (mobile device) which is carried by an individual around their waist. As mentioned above the dataset file is WISDM and Pandas will be used to load the dataset into a DataFrame. For final preparation, we will need to convert the data for it to float.
Data preprocessing
To prepare our raw data, data preprocessing is a crucial step. It can help the raw data to efficiently disintegrate in our model. The methods of data preprocessing in this case are:
- Data Split
- Label Encoding
- One-Hot Encoding
- Normalization
- Linear Interpolation
- Segmentation
Label Encoding
The model is unable to support any non-numerical labels as an input therefore we will have to transform them into numerical labels as given below:
- Sitting (2)
- Walking (5)
- Standing (3)
- Downstairs (0)
- Upstairs (4)
- Jogging (1)
Linear Interpolation
With the help of Linear Interpolation, we will be able to recover the lost data during the data collection process. Linear Interpolation will infuse the missing values for your convenience.
Data Split
Depending on the user IDs the data will be split. In this project, the total number of user IDs is less than or equal to 25 and the number is ideal for the test.
Normalization
Now our training data features will be normalized in the range of 0 to 1. The method used for normalization in this case is:
Xi = (Xi - xi min) / (xi max - xi min)
N denotes the number of channels
xi max denotes the maximum number of values in the i-th channel and
xi min denotes the minimum number of values in the i-th channel.
Segmentation
Through Segmentation we reshape the data frame and separate the features and labels.
One-Hot Encoding
It is the last step of data preprocessing where we will be storing our labels.
Result
After the training has been done our model will provide the preciseness of 98.02 percent with a loss of 0.58 percent. The F1 score for the entire training process is 0.96 percent.
Key takeaways
For Human Activity Recognition, you can certainly choose the CNN-LSTM model because it performs a lot better than any other basic neural network model or machine learning model. To know more about Human Activity Recognition you can study the application of artificial intelligence in detail.
Reference links:
https://ieeexplore.ieee.org/document/9065078
https://medium.com/@tanmaychauhan111/human-activity-recognition-using-lstm-cnn-8ccb1a42cb81