Description:FIELD OF INVENTION
The main field of invention is to identify the areas of need that older people have, and the available solutions. In particular, the robotic solutions are explored and critiqued and areas for future development identified. The literature was reviewed for factors that influence admission to nursing home care, and for technological solutions to these factors. The main issues facing older people are physical decline, cognitive decline, health management, and psychosocial issues.
BACKGROUND OF INVENTION
Scientific advances, in general terms, have improved living conditions as reflected in such basic aspects as longevity. According to several studies, there are expected to be around 125 million people over the age of 65 in the EU in 2030. Life expectancy has indeed improved considerably in recent years. Spain is one of the countries at the top of this list despite a decline of 1.6 years in 2020 due to the COVID 19 pandemic, as has happened in other countries. Increased longevity is not always accompanied by greater happiness. Cities are getting bigger and bigger and at the same time there are more lonely people. According to the survey published by the INE (the Spanish National Institute of Statistics) 42.7% of women over the age of 85 lived alone, compared to 23.6% of men. Since humans are naturally social, this is becoming a significant psychological problem. A clear example is the current COVID-19 pandemic, which has brought about social isolation with repercussions on mental health. Lockdowns and quarantines have forced even more of the elderly to live alone. INE statistics show that the number of people over 65 who lived alone in 2020 was 2,131,400 compared to 2,009,100 in 2019, so that loneliness is already postulated as one of the main epidemics of the 21st century. Despite the loneliness involved, older people defend their right to stay at home and in many cases they refuse to move in with relatives or to go to residences for the elderly, even though their mobility and ability to look after themselves could be progressively reduced. Although assistance can be given by home caregivers, it will be difficult to cover all demands for home assistance in the near future due to a shortage of available health workers and doctors as a result of greater life expectancy and low birth rate. There is thus an urgent need for innovative forms of support and health care for the elderly to maintain their physical and mental well-being, for which society should be prepared by adapting to the greater health assistance and monitoring requirements. The latter is an advantage since it allowed reaching populations with scarce resources, difficult to reach by traditional means. In recent years, the contributions of psychology have made sense with the intention of promoting interest in health and well-being during aging. Being relevant that people live longer, with quality and above all with autonomy, health and well-being.
LITERATURE SURVEY
The RobWell work was begun in 2019 with the aim of finding technological solutions to these needs. The team is made up of a multidisciplinary group of researchers, engineers and psychologists from the Technical University of Cartagena, the University of Murcia (Spain) and the University of Örebro (Sweden) and the University of Kuyshu (Japan). It is part of the larger HIMTAE work in cooperation with the University Carlos III of Madrid, which includes physical assistance in the kitchen with a manipulating robot (not described in this work). The RobWell work includes a mobile robotic platform integrated with ambient intelligence in a smart home that also includes estimating the user’s mood through wearables with a proposal for emotional coaching strategies. From the user’s point of view, the idea is to create a relatively simple system that can be installed in homes to monitor the user’s daily activities and his/her state of health and mood by means of distributed home automation sensors, medical devices and smart bands. The data is interpreted by artificial intelligence to estimate the habits and state of mood of those living alone. If strange behavior or a low mood is detected, emotional coaching strategies are proposed by a small robot with the size of a robot vacuum cleaner [9] through smart speakers. Alerts can also be generated to reach caregivers, family members or emergency services if necessary. Strange behavior would be considered, for example, as spending too long in bed, not going to the bathroom for a long time, or not opening the refrigerator all day. Emotional coaching strategies are recommended such as: “You should go for a walk”, “Why don’t you call someone on the phone” or “You haven’t eaten anything for a long time”. The robotic-based ambient intelligent system will not only detect situations that suggest that the person needs help but can suggest activities to improve the person’s mood, such as leaving the house to meet other people.
SUMMARY
This system requires expertise ambient assisted living, sensor data analysis, and machine learning. There is an additional distributed artificial intelligent system that coordinates all the subsystems, with symbiosis of physical and virtual robotic and AI subsystems to assist the elderly living alone during their daily activities. Advances in smart home devices and activity wristbands may make it possible to address the design of assistive environments with low-cost devices to achieve a relatively simple system that can be marketed at an affordable price to potential users. One of the most important aspects of this work is the artificial intelligence system involved in mood recognition. However, there are many problems with making these experiments a reality. The biggest impediments encountered are the number of uncontrollable variables involved in daily activities and the inconvenience of including the sensors used in the studies in daily tasks. Advances have been made in wearable devices, including the possibility of transferring these studies to other scenarios. The main objective of this article is to present the RobWell work, including the system architecture and integration of the distributed sensor ecosystem, data collection and method of mood prediction using machine learning algorithms. To do so, the following points will be discussed: General scheme, both software and hardware proposed. This section will present the general hardware/software composition including elements such as the robotic platform, the sensors for home automation and user monitoring, the actuators, the data acquisition system for mood estimation, the user interface and so forth. Integration of the proposed elements. The proposed test system and the integration achieved in the field of mobile robotic platform and home automation will be discussed. Robustness of the navigation of the robotic platform. The navigation system implemented on the robotic platform will be discussed together with the continuous operation tests. In this way it will be possible to discuss whether autonomous navigation needs more sophisticated navigation strategies. The autodocking algorithm offered for the Kobuki robotic base will also be tested for robustness in continuous operation. With the continuous running test, the autonomy of the robotic system will be observed. Mood prediction. The data acquisition system designed for conducting the experiment in an everyday environment is discussed. On the other hand, the machine learning strategy to be used for mood prediction is also discussed. In addition, the psychological tools to be used (questionnaires and EMAs) are presented.
DETAILED DESCRIPTION OF INVENTION
This field is attracting growing interest and is now focused on care in hospitals, nursing homes, rehabilitation clinics and even people’s homes. In this way, the sick, disabled or elderly can have continuous care and supervision. Assistive robotics can include ADL (Activities of Daily Living) tasks such as cleaning, administering medication and assisting healthcare personnel, general assistance and feeding, among others. Some outstanding ADL works include the Hospi Rimo robot, already installed in many Japanese hospitals, or Moxi, which is being tested in hospitals in Texas. Tasks such as feeding and personal hygiene represent a great challenge in terms of assistive robotics. In the case of feeding, there are challenges such as the manipulation of the food itself and tools of different geometry and size. With regard to personal grooming, there are designs for robotic toilets with a wall mounted motorized chair with three degrees of freedom while the Cody robot provides a solution to keep bedridden patients clean. In the field of mobile robotics applied to monitoring the elderly, Samsung proposes Bot Care, a small robot that aims to ensure that the owner is in an optimal state of health by analyzing data such as breathing, heart rate, activity levels, stress, quantity and quality of sleep, and so forth, alerting emergency services if necessary. Its functions include giving pills at the programmed time, monitoring physical condition and vital signs, and locating keys and mobile phones. It can also advise when it is time to eat, carry objects, recommend healthy dishes, assist the elderly person at bedtime, tuck them in or help them into bed.
The term smart home is defined in the literature according to the field of application to which it refers. Attempting to group and generalize the definition, Ehsan Kamel and Ali M. Memari determined that a smart home is a dwelling in which data related to the home environment and its residents are obtained from sensors, electrical devices or home gateways, and transferred using communication tools and networks for the purpose of monitoring devices and executing units, either to help decide actions or for the execution of so called services, these being the activities carried out by the smart home. Although there is no unified and definitive criterion for the differentiation of a smart home based on services, many authors agree on the following fields: health, well-being, security (internal agents), safety (external agents), entertainment and energy management. In most cases, the name of the service speaks for itself. However, a distinction can be made between wellness and health, with the former referring to trying to act before an illness occurs and the latter to monitoring the user’s vital signs to provide data to medical staff, promoting telemedicine. With regard to safety, a clarification is made in brackets. Internal agents refer to those found inside the home (leaving the iron plugged in or the cooker on) While external agents detect intruders. To conclude the review of related articles, it is worth mentioning, which presents an ROS based Integration of Smart Space and a Mobile Robot as the Internet of Robotic Things involving the new concept of IoRT (Internet of Robotic Things). This term encompasses and combines the terms robotic cloud and IoT. It shows how the key features of robotic technology, manipulation, intelligence and autonomy are related to scenarios in which IoT is applied. To do this, compatible communication protocols are used to connect the robot (or robots) with the smart space. This work aims to demonstrate how a smart home can provide motion assistance through robots that do not have vision systems, image processing and intensive calculations for decision making, to prove that sensors mounted on the robot can be replaced by others found in the environment. One of the most interesting works that has used ROS as middleware dates back. In this work, a general architecture was designed and implemented for the integration of different elements with reasoning capable planning modules for application to a smart office. This work is important because it implements a similar idea to the one proposed in our work, including the same mobile robot (Turtlebot). It is important to mention that this system has been tested in an environment emulating a smart office. In the case of the HIMTAE work, the environment is an elderly person’s house, which can become more complex. However, future work in this article reflects a number of practices to investigate that may be interesting to carry out our system, such as having different coordinated ROS masters or attempting mutual cooperation of different elements (an event addressed in the HIMTAE/RobWell work.
Wearables in Affective Computing
Emotional assessment and regulation has been one of the promising wearable applications since the pioneering works of Picard and Healey, and the presentation of their affective sensing device at the first IEEE International Symposium on Wearable Computers. According to these authors, wearables allow continuous emotional state estimation as they can be in close, long term physical contact with the user, enabling measurement of their condition on a scale of observations per person per minute rather than in a period of months or even years through current metrics and methods. As Pentland later pointed out in Social Physics. This specific characteristic could also enable long term data gathering for an individual “in the wild”, rather than the usual short-term data gathering for a group of people in the laboratory. After these initial basic technology research works (Technology readiness level -TRL- 1), it was just a matter of time for the technological offer to reach the maturity level re-quired for the proposal of research works for evaluating the feasibility of using these systems as feedback sources for human-robot interaction (TRL 2–3). In this regard, the Universal Emotion Recognizer (UER) used in the psychophysiological control architecture for human-robot coordination proposed can be considered as the first integrated device for personal emotional assessment after demonstrating its potential for inferring the emotional state of a particular individual after her observed physical expressions in a given perceivable context. Later works by this team at Vanderbilt University proposed the development of personal robots designed to act as understanding companions to humans, boosting research in the application of psychophysiology measurements for Human-Robot Interaction. When devices such as the first FitBit were introduced, authors such as Swan clearly identified the potential of these wearables for developing new patient driven health care services, including those related to patient quantified self-tracking for health applications and emotional support. With the advent of Ambient Intelligence (AmI) and Ambient Assisted Living -AAL-, initial requirements for user-centered design of Ambient Assisted Emotional Regulation systems were proposed after the usability analysis of this first generation of commercially available smart wearable devices. The electronics industry quickly reacted to the consumers’ expectations, increasing the availability and diversity of new electronic sensors specially conceived for integration in the second generation of these devices. Besides the “classical” MeMs-based accelerometers, inertial units (integrating triaxial magnetometers, gyroscopes and accelerometers), pulse sensors and biopotential front-ends developed by the main industry stakeholders, enabling the design and commercialization of advanced experimental wearable devices intended for sampling biosignals at higher frequencies with better resolution. The adoption of open operating systems, APIs and SDKs contributed to the spread of these technologies, boosting the development of new systems and solutions, that later lead to the advent of the era of the Internet of Wearable Things (IoWT). The design of this new generation of devices (TRL 9) considered not only functional requisites but also others such as aesthetics and usability to extend their use to other population groups such as the elderly raisings user awareness on other emerging topics such as privacy related to lifelogging. As happened previously with the spread of smartphone use, after the introduction of these advanced devices, research was then oriented toward the development of health care solution. Initially, most of them were centered on clinical assessment of chronic conditions and neurodegenerative diseases such as Parkinson’s disease or Multiple Sclerosis. These studies contributed to the knowledge related to subrogated alterations in the user’s condition (e.g., balance, gait or sleep patterns, etc.) useful for diagnostic and screening purposes.
Wearables and Emotional Biomarkers
Most of emotion or affective estimation works are based upon the analysis of subrogated physiological responses related to the different dimensions of the emotional model. When looking for emotional biomarkers, typical in lab experiments have involved recording psychophysiological responses of experimental subjects after the presentation of certain stimuli, either relevant or not. Tonic responses are measured as baseline values or resting levels of each signal in the absence of relevant stimuli while phasic responses gather the effect of certain stimuli in relation to the baseline values (i.e., increase/decrease, slope, etc.). In opposition to these, spontaneous or non-specific responses are measured when there is no known stimulus presented, being the typical kind of data acquired “in the wild”, as in this work. Since the initials works on affective computing by Picard and Healey, researchers have tried to acquire the same biosignals set as that used in lab conditions, including respiration (chest expansion measured with resistive/hall sensors), skin conductivity (GSR galvanic skin response), temperature, photoplethysmography (BVP, blood volume pressure), heart rate (measured after the BVP signal/ECG) and muscular activity (though the acquisition of the electromyogram, EMG,), brain activity (through the electroencephalograph, EEG) gait analysis and others (inertial parameters, light level, etc.).
Heart Rate: Heart rate is controlled by the autonomic nervous system, which comprises both sympathetic and parasympathetic (vagal) branches, whose action is normally balanced. When exposed to a stressor, this balance is lost, increasing the activity of the sympathetic branch, which leads to higher heart rates. In order to have a continuous indicator of this regulation, heart rate variability analysis is performed by studying the frequency contents of the spectral representation of the Inter-Beat-Interval time series (IBI). Mean and median frequency values of the power spectral density (PSD) of the IBI spectrum have been proposed as indicators. Special care should be taken when performing the spectral transformation since this series is not evenly sampled by nature. Time-domain parameters such as RMSSD, NN50 and pNN50 are also considered.
Respiration: As happens with heart rate, the respiration rate is also controlled by the autonomic nervous system, and thus is susceptible to be used as a subrogated indicator of the onset of stress and anxiety.
Photoplethysmography: Easily acquired by means of non-invasive sensors, blood volume pressure (BVP) has been widely used in affective detection experiments and instantaneous heart rate calculation.
Electromyography: Anxiety and stress can lead to increased muscled tension, as in the masseter and the cervical trapezius, being muscle electrical activity (usually quantified as the root-mean value of the EMG signal) a subrogated measurement of these anxiety and stress levels.
Skin Temperature: As above, finger blood capillaries vasoconstrict because of anxiety and stress, reducing the effective blood flow, which leads to decreased skin temperature, which is simple to measure using small mass thermistors.
Galvanic Skin Response: Stress and anxiety lead to an increase in sweating activity, which reduces the electrical resistance of the skin, which can be determined by injecting a minimal known DC-current and then measuring the voltage across the electrodes.
Gait analysis: Although gait analysis has been widely used for studying neurological degeneration related to Parkinson’s disease, Multiple Sclerosis and Alzheimer’s, only a few works have tried to find correlations of gait indicators with the emotional state of the subjects studied.
Brain activity: Through there are some works related to emotion detention in-the-wild based on electroencephalogram (EEG) analysis, there are some serious concerns limiting the practical application of these methods. Most of them are related to the reduced signal-to-noise-ratio achieved in real life conditions due to the small amplitude of the recorded signals and their susceptibility to artifacts (i.e., eyeblinks, heartbeats, jaw and forehead muscle tension) and electromagnetic perturbations.
With the advent of smartphone integrated sensors, some studies have taken advantage of them to build huge lifelogging datasets including signals from the built-in inertial sensors (accelerometer, gyroscope ad magnetometers), compass sensors, environmental parameters sensors such as sound and light levels and air pressure, humidity, and temperature as well as other as location and phone state.

Figure 1: Hardware schematic of the system
The main problem with commercial devices is that, even though the devices may comply with the open Zigbee standard, each manufacturer has his own gateway and user application (Ikea, LIDL, Xiaomi, Philips, etc.). This makes it impossible to use devices with another brand’s gateways and the aim of this work was to integrate devices of different brands. One of the main software tasks was thus to provide inter-device interoperability and transparency. As can be seen in Figure 2, the system is composed of a variety of software components for different domains. The elements that need specific software are as follows: robotic platforms, home automation system, positioning system, data acquisition system for predicting affective state and integration software. Of all these elements most are in the process of development or improvement, except for the indoor positioning system, which is currently under development. The rest of the systems are listed below with a brief description: Robotics platforms. Of the robotic platforms in Figure 1, only the emotional coaching platform will be explained because the other platform is being developed by the in the HIMTAE work in the Carlos III University of Madrid. The teleoperated mapping, autonomous navigation, autodocking and power management software to determine its performance in continuous operation has already been implemented and tested. Further features and enhancements are under development.
Smart home: The software that will run all the home automation logic is the Home Assistant operating system, selected because of its integration tools and the associated community for troubleshooting. It will run a Node-RED server and a Mosquitto MQTT broker. In addition, it also allows the use of the CC2531 USB dongle.
Acquisition system for affective state prediction. As can be seen in Figure 1, at the hardware level, this system consists of an Empatica E4 medical device and a smartphone. Two Android applications are used for the extraction of the data used in mood prediction. The first one, E4 RealTime, is the official application offered by Empatica for the extraction of physiological data. There is also a self-developed application for conducting tests and questionnaires to relate physiological data to mood states. The final idea is that this whole process will take place in a single application.
General integration system: This is the system in charge of carrying out the integration between the elements. There will only be differentiation at the level of sensors and actuators, regardless of the brand or manufacturer. This whole system will be developed on Ubuntu 16.04 and ROS. The use of ROS to carry out this task is justified by the fact that the integration, in some cases, is direct. Furthermore, being structured in nodes, the topics and following the publisher/subscriber policy makes development more accessible. The development of this application is not yet complete. For the time being, the integration paths have been basically developed and tested in a lightweight way. For example, the integration between Node-RED and ROS has been tested, as has the integration between elements with ROS, the integration between Zigbee and Node-RED elements with MQTT and between Android and Node-RED using MQTT. The integration application is currently in the conceptual development phase.

Figure 2: Software schematic of the system
The system introduced in this work is based on two elements: on the one hand, we have a wristband type device that can be used to obtain information on physiological variables and, on the other hand, we have an android application that collects user responses to the tests and questionnaires carried out. This wristband is provided with sensors to monitor blood volume pulse (BVP), electrodermal activity (EDA), and peripheral skin temperature. It is also equipped with a 3-axis accelerometer and a built-in application to derive the heart rate (HR) and interbeat interval (IBI) from the BVP signal. The emotional state of the monitored person is collected following the model proposed which is a multi-dimensional approach that defines emotions by two dimensions: arousal (or activeness) and pleasure (or happiness), pleasure being the range of negative and positive emotions; and arousal representing their active or passive degree. Therefore, each emotional state can be placed as a point; in this case, a bidimensional space. The collection of those two dimensions is done via a mobile app. The participant is asked to fill in the happiness and activeness felt at a certain time on a 5-point Likert scale to quantify his/her emotional well-being by asking only two concise questions, known as an Ecological Momentary Assessment (EMA). The small mental burden of this technique makes it suitable for the environment proposed in this work: emotion recognition during daily activities. The user is asked to answer this questionnaire five times a day and more answers can be included if required. The extracted features from the signals and the emotional state collected with the mobile APP define the classification problem proposed to estimate mood states.
DETAILED DESCRIPTION OF DIAGRAM
Figure 1: Hardware schematic of the system
Figure 2: Software schematic of the system , Claims:1. An Autonomous Mobile Robot System Monitoring Emotional and Physical Health Parameters and a Method Thereof claims that artificial intelligence and home automation is especially relevant for therapeutic applications in mental health services.
2. The interventions were especially directed towards caring for the elderly. It should be noted that our work focused on innovating by providing a responsible approach (under the supervision of a trained mental health professional) and taking into account the ethical implications of incorporating artificial intelligence and home automation in people’s lives (ecological sensors in the house so as not to interfere, the use of bracelets etc.).
3. The most important thing is that our work escapes from the laboratory since it allows the subjects to carry out their daily activities by means of smart devices.
4. Another of the fundamental aspects is the daily monitoring, after a previous evaluation, to be able to evaluate moods and mental states and to be able to make diagnoses where appropriate.
5. From an ethical and responsible perspective, this work will bring important benefits from the application of robotics and artificial intelligence to mental health, which will allow new modes of treatment, opportunities to involve hard-to-reach populations, improve adherence to patient response and free up time for specialists through combined care models.
6. That is why we argue that the union of artificial intelligence and home automation is a promising approach in the entire field of mental health, especially in innovative mental health care.