caring4U - A study on people activity in private spaces: towards a multisensor network that meets privacy requirements
Ambient assisted living (AAL) makes "use of information and communication technologies (ICT) in a person's daily living and working environment to enable them to stay active longer, remain socially connected and live independently into old age" (http://www.aal-europe.eu). Research efforts in AAL have increased rapidly to support independent living, as the social and economic impact of aging population has become a more concrete problem in our European society. The impact of the European demographic change is widely recognised making critical the need to address the problem both from a societal and economic standpoint. In the light of this, research into aging, age-related conditions and into supporting an aging population has become a priority for many governments around the world.
AAL aims to provide assistive solutions for people affected by a wide range of physical and cognitive challenges, in particular elderly persons. AAL applies the paradigm of the Internet of Things where sensing technology is embedded in objects, or in the environment or worn on the person to promote health and enhance wellbeing, or to help maintain an independent life at home. Captured sensor data are analysed to detect activity and infer knowledge about the physical or cognitive status of a monitored person; recognising and classifying patterns, detecting trends, and unusual or anomalous behaviour.
Video cameras are being used in AAL more frequently. The advances in the field of computer vision motivate this. Computer vision allows to monitor an environment and report on visual information, which is generally the most direct and natural way of describing an event, a person, an object, actions and interactions. However, the deployment of smart cameras in private spaces infringes people privacy rights. Surely, inhabitants would feel constantly observed and, in some way, they will be reluctant to privacy loss. This is a problem that it is currently of interest as new consumer electronics with wearable cameras, as the Google Glass, are reaching the market.
caring4U deals with these aspects, designing and developing intelligent vision systems to analyse human behaviour in private environments by considering ethical criteria to ensure privacy protection. caring4U considers a privacy-by-context approach, proposing a level-based visualisation scheme to protect privacy. Each level establishes how the image is displayed and, therefore, the provided protection degree. In practice it consists in choosing, according to the context, between different visualisation models, that is, different ways of modifying an image before it is displayed. The context is composed of different elements: the person being monitored, the location (kitchen, bathroom, bedroom...), the on-going activity, event or alarm; the observer (a doctor, a professional carer, a relative, a friend...). Then, the assisted person can decide which visualisation level assigns to each context. This visualisation scheme aims to balance privacy protection and information utility, and at the same time it can be adapted to a personal feeling of privacy.
For instance, different levels of alarm can trigger different visualisations:
- No alarm: a virtual image of the room is presented without showing any person;
- Low-level alarm: a virtual image is displayed of both the room and the person showing position but not posture (the person would be in a standard or fixed posture, avoiding to display the real one);
- High-level alarm: a virtual image of both the room and the person (in the real posture) are presented. For instance, if a person falls in the bathroom, a carer of a telecare service would be able to see this image; and
- Very high-level alarm: the system shows the real image of the room and the person. For instance, if after a fall the person does not answer to the professional carer, a close relative with the appropriate permissions would be able to observe the real scene in order to assess and confirm the level of alarm.
Therefore, work in caring4U has being focused in both areas of research: human action recognition in order to establish the on-going activity or event, and visual privacy preservation techniques.
The first approach to human action recognition was using RGB cameras. Based on a human silhouette extracted from the image, the method learns the features that make up the most characteristic poses, the so-called key poses. These can be acquired from single- or multi-view data, which makes the method suitable for scenarios with one or more cameras. Using the ground truth data, the sequences of key poses corresponding to the labelled videos are obtained. These sequences are matched later with the current test sequence based on Dynamic Time Warping. caring4U system has been designed so as to run at a frame rate close to real-time and to support online recognition. Since the target application is human monitoring at home for AAL services, these were both essential premises.
Recently, interest has grown on affordable devices as the Microsoft Kinect or the ASUS Xtion Pro, which can capture depth quite reliably. These image sensors provide a depth image (D), besides the regular colour image (RGB). The resulting RGB-D data can be used to obtain marker-less body pose estimation. Specifically, a skeleton model consisting of a set of joints is generated. This data can be used in order to learn and classify human poses, actions or even activities of daily living. These sensors have become popular due to their low cost, high sample rate and capability of combining visual and depth information. Usage can be found both in research and commercial applications. Although they were initially designed for gaming purposes, other applications, where natural human-computer interaction (HCI) is required, are extensively employing these technologies. In particular, RGB-D devices are used in ambient assisted living for fall detection, physical rehabilitation, medical image exploration in operating rooms and gait analysis among other applications.
Most of the existing works for human action recognition use all the available features obtained by the devices (whole contour or silhouette in the case of cameras and all the joints in the case of RGB-D devices). However, some actions or gestures involve moving the whole body, whereas others are performed using only the arms or the hands. Therefore, it is interesting to determine which features have a greater value to the success of the recognition method being used, and which ones can be discarded because they are not relevant for a specific application, since they introduce confusion or noise and reduce the recognition rate. So, caring4U proposes a cooperative co-evolutionary algorithm for the selection of the optimal subset of training instances, relevant features and parameters in order to improve action recognition.
In the area of privacy preservation different visualisations have been implemented in caring4U in order to be associated with a specific context (figure 2). These filters are: blur, pixelating, emboss, solid silhouette, 3D virtual avatar, and invisibility. The system employs a RGB-D device to extract the silhouette and the posture of the person, and is able to perform in real-time in a standard PC.
A survey run at the end of the project shows that people accept more to install cameras at home if the level of protection is increased. Therefore, using a virtual avatar, which avoids seeing the real person, could be a good solution for privacy preservation and acceptance by users in almost any context.
The use of ethical vision systems for AAL will enable a breakthrough in the development of services and technologies for intelligent and self-adaptive environments with the aim to support independent living for everyone, but in particular for the elderly or people who require the constant attention of a carer, allowing their better quality of life and greater efficiency of their care. The maintenance of privacy will also provide greater acceptance and confidence of people when using the proposed technologies.