Keywords

1 Introduction

With the worldwide increase in an ageing population [1], many related cognitive impairment issues, such as dementia, and Parkinson’s Disease, are critical societal challenges that need to be solved [2]. Therefore, there has been a rapid increase in investment in the research and development of related digital technologies (mental health applications and digital therapeutics) that can help tackle the concerning cognition and mental health issues [3].

The purpose of this study is to develop serious game content that can test cognitive ability by using digital media such as smartphones, beam projection, and a robot. The main idea of the serious game is that the physical movements of the elderly control the developed game. The different scenarios of the game promote the physical activity of the elderly while at the same time measuring the cognitive characteristics of the elderly through their behavioural reaction to the contents. Additionally, weekly and daily practice of these interactive contents is to monitor the elderly user’s neurocognitive function in terms of memory loss, spatial-temporal awareness. The contents are aimed at the use of the general healthy adult population to prevent and screen for dementia and cognitive decline. In contrast, the content is to help delay the onset of symptoms for the mild cognitive impairment group, whereas, for the more severe it will be used to help manage and slow the progression of the cognitive impairments in memory loss and temporal-spatial awareness domains.

2 Content Development Method

There are a vast number of studies highlighting the secure link between the various physical activity intensities and types and the neurotransmitters in the brain [4,5,6]. Additionally, there are strong links between the cognitive training contents and improvement in cognition by the varying ages of the participants from university students to the elderly [7,8,9]. We, therefore, propose a mixture of these two methods to aid in the stimulation of cognitive-based physical activity. From this point of view, the game will allow the elderly users to perform the tasks in reaction to the content with their whole-body movement (Fig. 1).

Fig. 1.
figure 1

The three scenarios to be tested and trained using the proposed contents for dementia detection, screening, prevention, and management.

Full size image

The contents were developed to be able to manage with users of different severities in the stages of dementia. We targeted the users from the asymptomatic stage to mild cognitive impairment, early dementia, mild dementia, moderate dementia, and severe dementia. The game will present a high level of difficulty for the lower severities of the mental/cognitive decline, whereas, for the more severe patients with dementia, the level of content will be easier. Through the continuous practice of this whole-body interactive content, we expect that the contents will be applicable for slowing down progression and the prevention of cognitive and physical decline. The content was designed based on the mini-mental state examination for cognitive domains and on sit to walk gait analysis for a screening of both physical and mental issues. Gait analysis is a common method that can be used to screen for cognitive decline and predict frailty in the elderly population [10,11,12].

The interactive program opens with the choice of the user to select between 3 choices, gait analysis (top), memory evaluation (middle), and temporal-spatial awareness (bottom) (Fig. 2).

Fig. 2.
figure 2

Opening screen with the three options to choose form.

Full size image

For the gait analysis task, the user is instructed to walk around the projection of the path on the floor while their time is measured. Here are two screenshots showing the path and the user’s walking time. The gait test is to replicate the sit to walk test, which has been validated in research to be able to predict cognitive decline in the elderly [13] (Fig. 3).

Fig. 3.
figure 3

The user starts on the bottom left of the screen projected on the floor then must walk around the edge as shown.

Full size image

For the memory loss task, the user a choice of 3 levels. Each time the level increases the time that the position of the animals on the 3 × 3 grid becomes shorter. The level 1 test shows the full grid of the three different animals, the rabbit, turtle, and tiger for 10 s then will disappear. Then two of the same animals are shown and the user must go and stand on the position of the same third animal. The user must go to the designated square and stand there until the next option is given. The second level is the same, but the time to show the position of the animals becomes shorter, and for the third level, the time showing the positions of the animals is even shorter again (Fig. 4).

Fig. 4.
figure 4

(Left) Screenshot with the 3 levels to choose from, (right) the user must stand on the position where the third rabbit was shown in the previous screen

Full size image

For the spatial-temporal awareness task, the user has the choice to do all three levels or choose one of the levels. The difficulty of the levels increases by showing a path projected on the ground, either in a straight line, curving line or a combination of both. The difficulty is increased by showing the path for a shorter time and making a more complex path with more turns (Fig. 5).

Fig. 5.
figure 5

(Left) Screenshot with the 3 levels to choose from, (right) the user must follow the path projected onto the ground correctly

Full size image

3 Validity of Content

The reliability and the face validity of the content will be checked through a series of clinical pilot tests. As the contents is based on the cognitive tests used by neurologists to screen and diagnose any cognitive decline, the contents will be checked by neurologists by comparing the results of the behavior recorded in the clinical trials with the results of the existing data that they have from the participants regular hospital check-up. In other words, the existing clinical data and the results of the user’s reactions and performance of the proposed contents will be used to validate the effectiveness of the interaction intervention contents. Currently, all people over 65 years of age are expected to attend a clinic to fill out a mini mental state examination and other cognitive related questionnaires every year [12, 14].

4 User Analysis

For user analysis, we plan to measure the user’s behaviors and interaction with the reaction-stimulating media (game) content. Their behavior will be recorded using Kinect depth cameras (RGB-D) and their movement is used to control the interactive contents. Though the analysis of their movements, such as reaction times and movement speeds, we plan to measure the state of the cognitive function in the various domains (short- and medium-term memory loss, and temporal-spatial awareness).

5 Future Work

The intervention contents developed in this study for the screening, prevention, management of cognitive decline for the elderly needs to be continually improved and additional contents for other cognitive domains needs to be further developed. Furthermore, content based on the innovative methods in this study should be examined for other target groups of people and other mental/cognitive related declines, such as depression, worry, anxiety, etc. By using a system that is controlled by the user’s full body movement the behavior and movement data can be used to develop a digital database of human behavior norms during various set scenarios. As explanatory artificial intelligence develops this behavior database will possibly be used to develop learning algorithms that may detect abnormal or possibly troubling behavior queues. It is important to note here that the privacy of the user during all of these future endeavors must confirm to the highest standards to protect any user or participant.