Multi-User Activity Recognition Using Plot Images Based on Ambiental Sensors

Artificial intelligence has increasingly taken over various aspects of daily life, resulting in the proliferation of smart devices and the development of smart living and working environments. One significant domain within this technological advancement is human activity recognition, which includes a broad spectrum of applications such as patient monitoring and supervision of children’s activities. In this research, we endeavor to design a human activity recognition system that effectively analyzes multi-user data through a machine learning framework centered on graphical plot images. The proposed methodology uses a PIR sensor-based system. This system uses a two-stage process; the first one involves generating new image datasets as density map images and graphical representations based on the Kyoto CASAS multi-user dataset. In the second stage, the generated data are provided to a sequential convolutional neural network, which predicts the 16 activities developed by two users. To generate the new datasets, we only used data from ambient sensors, which were organized in windows. We tested many types of window dimensions and extra features such as temporal aspect and the limitation of two activities in one window. The neural network was optimized by increasing the deconvolutional layers and adding the AdamW optimizer. The results demonstrate the viability of this method, evidencing an accuracy rate of 83% for multi-user activity and an accuracy rate of 99% for single-user activity. This study successfully achieved its objective of identifying an efficient activity recognition methodology and a data image representation. Furthermore, future enhancements are anticipated by integrating data sourced from PIR sensors, with information gathered from user-personal devices such as smartphones. This approach is also applicable to real-time recognition systems.