JP2022127234A - Information processing method, information processing system, and program - Google Patents

Abstract

To provide an information processing method and the like that can determine the state of a cognitive function in consideration of characteristics such as a user's strong field or weak field.SOLUTION: An information processing system includes a robot and a display device communicatively connected to the robot. The information processing system outputs a program for determining a physical condition to a target person through the display device and obtains answer information about the target person's answer to the program through the display device or the robot. Based on the obtained answer information, the information processing system identifies a category related to the target person's cognitive function, and outputs content related to the cognitive function through the display device according to the identified category.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to an information processing method, an information processing system, and a program.

In recent years, the number of elderly people is increasing. For example, elderly people who live alone or live alone in a facility have few opportunities to interact with others in their daily lives, and it is difficult to grasp changes in their physical condition. In particular, it is difficult to perceive deterioration of cognitive functions such as memory, language, or judgment by oneself. Therefore, Patent Literature 1 discloses a technology capable of performing a cognitive function test as part of daily health care. In the technique disclosed in Patent Literature 1, for example, a cognitive function test is performed using the opportunity of blood pressure measurement.

JP 2020-151045 A

With respect to cognitive functions, each user has a different field of strength or weakness. It is difficult to determine the state of cognitive function by

The present disclosure has been made in view of such circumstances, and its purpose is to determine the state of cognitive function in consideration of characteristics such as the user's strong field or weak field. An object of the present invention is to provide an information processing method and the like.

An information processing method according to an aspect of the present disclosure includes a robot and a display device communicably connected to the robot, and outputs a program for determining a physical condition to a subject through the display device. , through the display device or robot, acquire response information about the subject's response to the program, identify the classification of the cognitive function of the subject based on the obtained response information, and recognize according to the identified classification A process of outputting content related to the function through the display device is included.

In one aspect of the present disclosure, it is possible to determine the state of cognitive function in consideration of characteristics such as the user's strong field or weak field, and the user according to the determination result reflecting the user's characteristics It is possible to provide content suitable for

1 is a schematic diagram showing a configuration example of an information processing system; FIG. FIG. 2 is a block diagram showing an internal configuration example of an information processing device and a robot; 4 is a schematic diagram showing a configuration example of a test DB stored in an information processing device; FIG. 4 is a schematic diagram showing a configuration example of a history DB stored in an information processing apparatus; FIG. It is a block diagram which shows the internal structural example of a server. 4 is a schematic diagram showing a configuration example of a DB stored by a server; FIG. 4 is a schematic diagram showing a configuration example of a DB stored in a server; FIG. FIG. 4 is a schematic diagram showing a configuration example of a learning model; FIG. 11 is a flowchart showing an example of a learning model generation processing procedure; FIG. FIG. 11 is a flowchart showing an example of a content provision processing procedure; FIG. It is a schematic diagram which shows an example of a screen. It is a schematic diagram which shows a screen example. It is a schematic diagram which shows an example of a screen. FIG. 12 is a flowchart showing an example of a content providing processing procedure according to the second embodiment; FIG. FIG. 12 is a flowchart showing an example of a content provision processing procedure according to the third embodiment; FIG. FIG. 11 is a schematic diagram showing an example of a screen according to the third embodiment; FIG. FIG. 16 is a flowchart showing an example of a content provision processing procedure according to the fourth embodiment; FIG. FIG. 12 is a schematic diagram showing an example of a screen according to Embodiment 4; FIG. 12 is a schematic diagram showing a configuration example of an information processing system according to Embodiment 5; FIG. 21 is a flow chart showing an example of a content provision processing procedure according to the fifth embodiment; FIG. FIG. 11 is a schematic diagram showing an example of a screen according to Embodiment 5; FIG. 12 is a block diagram showing an example of the internal configuration of an information processing apparatus according to Embodiment 6; 4 is a schematic diagram showing a configuration example of a user information DB stored by a server; FIG. FIG. 16 is a flow chart showing an example of a content provision processing procedure according to the seventh embodiment; FIG. FIG. 21 is a schematic diagram showing an example of a screen according to Embodiment 7; FIG. 11 is a flowchart showing an example of an extraction processing procedure for users with low training results; FIG.

An information processing method, an information processing system, and a program according to the present disclosure will be described below based on drawings showing embodiments thereof.

(Embodiment 1)
Information that outputs (issues) a program (test program) to the user, identifies the type of cognitive function of the user according to the behavioral tendency of the user when answering, and provides content according to the identified type A processing system will be described. FIG. 1 is a schematic diagram showing a configuration example of an information processing system. The information processing system of the present embodiment includes an information processing device 10, a robot 30, a server 40, etc. The information processing device 10 and the server 40 are connected to a network N such as the Internet, and transmit and receive information via the network N. conduct. The information processing device 10 and the robot 30 are configured to be capable of short-range wireless communication such as Bluetooth (registered trademark), and transmit and receive information by wireless communication. The information processing device 10 and the robot 30 are used as one set, for example, installed in the user's home. Accordingly, the information processing system includes multiple sets of information processing devices 10 and robots 30 . Note that the information processing device 10 and the robot 30 can be installed at any location.

The information processing device 10 is a general-purpose information processing device such as a tablet terminal, a personal computer, or a smart phone, but may be configured as a dedicated terminal. The information processing device 10 is a display device having a display unit 16 such as a liquid crystal display or an organic EL (Electro Luminescence) display. In the information processing apparatus 10, a camera 17 is provided at a position where the front side (display screen side) can be photographed, and a microphone 18 is provided at a position where the sound on the front side can be collected. Note that the camera 17 may be provided at a position where the upper half of the body (especially the face) of the user in front of the information processing apparatus 10 can be photographed, and the microphone 18 is positioned at a position where the voice uttered by the user in front of the information processing apparatus 10 can be collected. It is sufficient if it is provided in

The robot 30 has a small exterior that resembles a bird. The robot 30 is mounted, for example, on a mounting table provided at the upper end (upper surface) of the information processing apparatus 10 . A charging base (not shown) for the robot 30 may be provided on the mounting base, and the robot 30 is operated by power supplied via the charging base or by a battery built into the robot 30. do. Further, the robot 30 is provided with a speaker 34, a light emitting unit 35, a human sensor 36 for detecting the approach of a person, and a microphone 37 on the front side (the same side as the display screen side of the information processing device 10). In the example shown in FIG. 1, a robot 30 shaped like a bird has a speaker 34 on its head, two light-emitting units 35 on its eyes, and a human sensor 36 on its chest. A microphone 37 is arranged below the human sensor 36 . Note that the speaker 34 may be provided at a position where the sound can be delivered to the user on the front side of the robot 30, and the light emitting unit 35 may be provided at a position where the user on the front side can visually recognize the emitted light. good. Also, the human sensor 36 may be provided at a position where it can detect the approach of the user on the front side, and the microphone 37 may be provided at a position where it can collect the uttered voice of the user on the front side. In this embodiment, the robot 30 may be configured without the light emitting unit 35 and the human sensor 36 .

The server 40 is an information processing device capable of various information processing and transmission/reception of information, and is configured using, for example, a server computer or a personal computer. A plurality of servers 40 may be provided for distributed processing, may be realized by a plurality of virtual machines provided in one server, or may be realized using a cloud server. .

In the information processing system of the present embodiment, the information processing device 10 outputs to the user a program (test content) for determining the physical condition of the user (subject). The program for determining the physical state includes, for example, cognitive function tests and physical function tests, and includes questions and problems for determining the state of cognitive function and physical function. Below, outputting a program to a user may be expressed as "giving a question". The test content includes image data and audio data, the image data is output to the display unit 16 and displayed, and the audio data is output to the speaker 22 and output as audio. Note that the audio data may be output to the robot 30 and output as audio from the speaker 34 . The user answers the program (questions to be set) output from the information processing apparatus 10, and at that time, the user is photographed by the camera 17. FIG. The information processing apparatus 10, from a photographed image of the user, detects movements of the user's gaze (visual line) when the user is working on a problem (while answering), movements of the mouth, behavior such as the degree of swaying, and facial expressions. Detects information (response information) containing The information processing device 10 then transmits the detected information (response information) to the server 40 . Based on the answer information received from the information processing device 10, the server 40 identifies the user's cognitive function type, and provides advice or content (for example, training content) according to the identified type to the user via the information processing device 10. provide to

FIG. 2 is a block diagram showing an internal configuration example of the information processing device 10 and the robot 30. As shown in FIG. Information processing apparatus 10 includes control unit 11, storage unit 12, first communication unit 13, second communication unit 14, input unit 15, display unit 16, camera 17, microphone 18, reading unit 19, behavior detection unit 20, facial expression A detection unit 21, a speaker 22, etc. are included, and these units are connected to each other via a bus. The control unit 11 has one or more processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), or a GPU (Graphics Processing Unit). The control unit 11 appropriately executes a control program 12P stored in the storage unit 12 to perform various information processing and control processing that the information processing apparatus 10 should perform. The storage unit 12 includes a RAM (Random Access Memory), flash memory, hard disk, SSD (Solid State Drive), and the like. The storage unit 12 stores in advance a control program 12P executed by the control unit 11 and various data necessary for executing the control program 12P. The storage unit 12 also temporarily stores data and the like generated when the control unit 11 executes the control program 12P. The storage unit 12 also stores a test DB (database) 12a and a history DB 12b, which will be described later. The test DB 12a and the history DB 12b may be stored in another storage device connected to the information processing device 10, or may be stored in another storage device with which the information processing device 10 can communicate.

The first communication unit 13 is a communication module for connecting to the network N by wired communication or wireless communication, and transmits and receives information to and from other devices via the network N. The second communication unit 14 is a communication module for wireless communication using Bluetooth, and transmits and receives information to and from the robot 30 by wireless communication. The second communication unit 14 may be configured to perform wireless communication according to a communication standard other than Bluetooth, or may be configured to transmit and receive information to and from the robot 30 by wired communication via a cable.

The input unit 15 receives an operation input by the user and sends a control signal corresponding to the operation content to the control unit 11 . The display unit 16 displays various information according to instructions from the control unit 11 . The input unit 15 and the display unit 16 may be a touch panel that is laminated and integrally configured. The camera 17 is a photographing unit that photographs according to an instruction from the control unit 11, and sends acquired photographed images (still image and moving image data) to the storage unit 12 for storage. The microphone 18 is a sound collecting unit that collects surrounding sounds according to an instruction from the control unit 11, and transmits the acquired sound data to the storage unit 12 for storage. The speaker 22 is an audio output unit that outputs audio according to instructions from the control unit 11 , and outputs messages according to instructions from the control unit 11 . The camera 17, the microphone 18, and the speaker 22 may be built in the information processing apparatus 10 as shown in FIGS. 1 and 2, or may be externally attached to the information processing apparatus 10. For example, the information processing apparatus 10 may be configured to include a camera connection unit to which an external camera can be connected, or a camera communication unit that performs wireless communication with the external camera. In this case, the camera connection unit or the camera communication unit accepts input of image data acquired by the external camera, and sends the input image data to the storage unit 12 for storage. The information processing apparatus 10 may also be configured to include a microphone connection unit to which an external microphone can be connected, or a microphone communication unit that performs wireless communication with an external microphone. In this case, the microphone connection unit or the microphone communication unit accepts input of audio data acquired by the external microphone, and sends the input audio data to the storage unit 12 for storage. Further, the information processing apparatus 10 may be configured to include a speaker connection unit to which an external speaker can be connected or a speaker communication unit that performs wireless communication with the external speaker. In this case, the control unit 11 instructs the external speaker to output the voice message via the speaker connection unit or the speaker communication unit.

The reader 19 reads information stored in a portable storage medium 1a including CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, USB (Universal Serial Bus) memory, SD (Secure Digital) card, etc. read. The control program 12</b>P and various data stored in the storage unit 12 may be read from the portable storage medium 1 a by the control unit 11 via the reading unit 19 and stored in the storage unit 12 . Also, the control program 12P and various data stored in the storage unit 12 may be downloaded from another device by the control unit 11 via the first communication unit 13 and stored in the storage unit 12 .

The behavior detection unit 20 detects the position of the user's line of sight based on the face image of the user captured by the camera 17 . For example, when a question and answer options are displayed on the display screen, the position of the line of sight indicates whether the user is looking at the part of the question, the part of the option, or other part. . The behavior detection unit 20 can detect the position of the line of sight in chronological order based on the moving image captured by the camera 17, and as a result, can detect the movement of the line of sight. The behavior detection unit 20 also detects movements of the user's mouth based on a moving image (face image) of the user's face captured by the camera 17 . Furthermore, the behavior detection unit 20 detects the degree of swaying of the user based on the video of the user's whole body or upper body captured by the camera 17 . The behavior detection unit 20 of the present embodiment is configured to detect user behavior such as the position and movement of the user's eyes, the movement of the mouth, and the degree of swaying, but the behavior of the user is not limited to these. The facial expression detection unit 21 detects facial expressions appearing on the user's face based on a facial image of the user's face captured by the camera 17 . Facial expressions include facial expressions expressing emotions such as joy, sadness, anger, surprise, and fear, and serious faces (expressionless). The facial expression detection unit 21 may detect facial expressions based on still images captured by the camera 17, may detect facial expressions based on moving images, or may detect changes in facial expressions over time.

The behavior detection unit 20 and the facial expression detection unit 21 may be software functions realized by the control unit 11 executing an application program stored in the storage unit 12, and may be realized by dedicated hardware circuits. good too. Moreover, the facial expression detection unit 21 may be configured to detect the facial expression of the user from the user's captured image (face image) using a neural network constructed by machine learning. For example, it is composed of a CNN (Convolution Neural Network) model, and when a photographed image is input, it outputs information indicating which of a plurality of preset facial expressions the subject (user) in the photographed image has. A learning model trained to do so may be used. In this case, the facial expression detection unit 21 can input the user's facial image to the learned model, and identify the user's facial expression in the captured image based on output information from the learned model. The behavior detection unit 20 also uses a neural network built by machine learning to detect the user's behavior (movement of eyes, movement of mouth, degree of swaying, etc.) from captured images of the user. good too. For example, when a photographed image is input using a CNN model, the user's behavior (eye movement, mouth movement, degree of swaying) in the photographed image is determined by a plurality of preset behaviors (eye movement, mouth movement, etc.). A learning model that has been trained to output information indicating whether it is the movement of the body or the degree of sway may be used. In this case, the behavior detection unit 20 can input the photographed image of the user to the learned learning model, and identify the behavior of the user in the photographed image based on the output information from the learning model. In addition to the configuration described above, the information processing apparatus 10 may be configured to include a gesture detection unit that detects an action (gesture) performed by the user based on a captured image of the user. In this case, the information processing apparatus 10 can be configured to receive an input operation (gesture input) based on a user's gesture.

The robot 30 includes a control unit 31, a storage unit 32, a communication unit 33, a speaker 34, a light emitting unit 35, a human sensor 36, a microphone 37, etc. These units are interconnected via a bus. The control unit 31, the storage unit 32, the communication unit 33, the speaker 34, and the microphone 37 of the robot 30 are similar to the control unit 11, the storage unit 12, the second communication unit 14, the speaker 22, and the microphone 18 of the information processing device 10, respectively. , detailed description is omitted.

The light emitting unit 35 is, for example, an LED (Light Emitting Diode), and emits light (lights up or blinks) with luminance and color according to instructions from the control unit 31 . The human sensor 36 is a sensor that detects the presence or absence of a person within a predetermined detection area. When this is detected, the detection result is sent to the control unit 31 .

In the present embodiment, the information processing apparatus 10 includes the camera 17, the microphone 18, and the speaker 22, and the robot 30 includes the speaker 34, the light emitting unit 35, the human sensor 36, and the microphone 37. Not limited. Each of the camera 17, the microphones 18 and 37, the speakers 22 and 34, the light emitting unit 35, and the human sensor 36 may be provided only in the information processing apparatus 10, may be provided only in the robot 30, or both may be provided. may be provided in

FIG. 3 is a schematic diagram showing a configuration example of the test DB 12a stored in the information processing apparatus 10. As shown in FIG. The test DB 12a includes cognitive function tests, physical function tests, etc., and stores information on tests for determining physical conditions such as cognitive functions and physical functions of the user. The test DB 12a shown in FIG. 3 includes an item column, a test ID column, a test content column, a correct answer column, an answer method column, and the like, and stores information on the test for each item for each item to be judged. The item column stores identification information (item name) for identifying each item to be determined. In the example shown in FIG. 3, items such as memory ability, calculation ability, language ability, judgment ability, and physical ability are registered, but the types of items are not limited to these. The test ID column stores identification information (test ID) for identifying each test, and the test content column stores content data of each test. The content data may be image data, audio data, or may include image data and audio data. In addition to being stored in the test DB 12a, the content data may be stored in a predetermined area of the storage unit 12 or another storage device. file name indicating the storage location). The correct answer column stores the correct answer for each test, and the correct answer may be information on the correct answer, or in the case of a test (question) that requires multiple answers, information indicating the correct answer option (choice number, etc.). may The answering method column stores the answering method set for each test. The answering method includes, for example, a touch operation via the input unit 15, voice input via the microphone 18, input by line of sight (line of sight), input by gesture, and the like. The items stored in the test DB 12a are stored in advance, and are additionally stored by the control unit 11 when a new item is registered. The test ID stored in the test DB 12a is issued and stored by the control unit 11 when a new test is registered according to each item. Other information stored in the test DB 12 a is added by the control unit 11 when the control unit 11 acquires an instruction to add each information via the first communication unit 13 or the reading unit 19 . The contents stored in the test DB 12a are not limited to the example shown in FIG. The test contents stored in the test DB 12a include, in addition to functional tests for determining cognitive and physical functions, "What is your name?", "How old are you?" It may also include an interview test such as "Are you sure?"

FIG. 4 is a schematic diagram showing a configuration example of the history DB 12b stored by the information processing apparatus 10. As shown in FIG. The history DB 12b stores history information about tests performed on users using the information processing device 10 and the robot 30 . The history DB 12b shown in FIG. 4 includes a start date and time column, a test content column, an answer information column, a determination result column, etc., and in association with the start date and time, the status of the performed test and the determination result determined from the test result. Remember. Note that the history DB 12b stores information about a series of exchanges between the information processing apparatus 10 and the user in association with the start date and time. The start date and time column stores the date and time when the user started using the information processing apparatus 10 . The test content column stores identification information (test ID) of test content given to the user by the information processing apparatus 10 . The answer information column stores information indicating the user's situation when the user is answering the given test (question). Incorrect answers (correct or incorrect), behaviors, facial expressions, time zones in which questions were set, user's conversation tendencies, etc. are associated and stored. The tendency of questions includes the types of questions, answering methods, etc. Examples of question types include questions to be answered by looking at pictures, questions to be answered by reading letters, questions involving touch operations according to the content, and physical movement. Including the type of problem, etc. The behavior includes information such as the position and movement of the user's line of sight (whether the user is staring or whether the line of sight is moving), movement of the mouth, degree of swaying, and the like. Facial expressions include facial expressions expressing emotions such as joy, sadness, anger, surprise, and fear, and straight faces (expressionless). The determination result column stores the user's situation (determination result) determined based on the user's answer information, specifically, the type of cognitive function of the user, training to be provided to the user (training ID) and advice (advice ID) are associated with each other and stored. Note that the determination result is specified by the server 40 that has received the reply information from the information processing device 10 . The start date and time stored in the history DB 12b is stored by the control unit 11 when the information processing apparatus 10 starts providing content to the user. As for the test content stored in the history DB 12b, the test ID of the test content is stored by the control unit 11 when the control unit 11 outputs the test content via the display unit 16. FIG. As the answer information stored in the history DB 12b, when the control unit 11 outputs the test content, the control unit 11 stores the problem tendency and the time zone of the output test content, and after receiving the answer from the user, the answer The control unit 11 stores the time, whether the answer is correct or not, the result of detection by the behavior detection unit 20 (behavior of the user), and the result of detection by the facial expression detection unit 21 (facial expression of the user). The determination result stored in the history DB 12 b is stored by the control unit 11 when the control unit 11 acquires each information of the determination result from the server 40 via the first communication unit 13 . The contents stored in the history DB 12b are not limited to the example shown in FIG. For example, when the information processing apparatus 10 poses a question to the user, the tendency of the user's dialogue is extracted from the contents of the dialogue performed with the user, and the information indicating the tendency of the dialogue (dialogue tendency) is used as the answer information. It may be stored in the history DB 12b including. Also, information indicating the results of training performed by the user on training content provided to the user according to the determination result may be stored in the history DB 12b.

FIG. 5 is a block diagram showing an internal configuration example of the server 40. As shown in FIG. The server 40 includes a control section 41, a storage section 42, a communication section 43, an input section 44, a display section 45, a reading section 46, etc. These sections are interconnected via a bus. The control unit 41, the storage unit 42, the communication unit 43, the input unit 44, the display unit 45, and the reading unit 46 of the server 40 correspond to the control unit 11, the storage unit 12, the first communication unit 13, the input Since the configuration is the same as that of the unit 15, the display unit 16, and the reading unit 19, detailed description thereof will be omitted. In addition to the control program 42P executed by the control unit 41, the storage unit 42 of the server 40 stores a learning model 42M trained on training data by, for example, machine learning, a type DB 42a, a training DB 42b, an advice DB 42c, and a dialogue DB 42d, which will be described later. I remember.

The learning model 42M is a trained model that has been learned to output a type related to the user's cognitive function determined by this answer information when the user's answer information collected by the information processing device 10 is input. . The learning model 42M is assumed to be used as a program module that constitutes artificial intelligence software. The learning model 42M performs a predetermined calculation on the input value and outputs the calculation result. In the storage unit 42, data such as the coefficient of the function that defines this calculation and the threshold value are stored as the learning model 42M. be done. The type DB 42a, the training DB 42b, the advice DB 42c, and the dialogue DB 42d may be stored in another storage device connected to the server 40, or may be stored in another storage device with which the server 40 can communicate.

6 and 7 are schematic diagrams showing configuration examples of the DBs 42a to 42d stored in the server 40. FIG. 6A shows the type DB 42a, FIG. 6B shows the training DB 42b, FIG. 6C shows the advice DB 42c, and FIG. 7 shows the dialogue DB 42d. The type DB 42a stores various features related to cognitive function in association with types (type IDs) assigned to the features. Each type of characteristic includes, for example, being weak or good at problems related to memory, arithmetic, verbal ability, judgment, physical ability, and the like. In addition, the characteristics of each type are characteristics related to hobbies and preferences such as interest in gardening, interest in automobiles, interest in cooking, etc., less frequency of going out, less frequency of conversation, less frequency of exercise. Features related to daily life such as less, continuing training every day, training once every two days, etc. may be included. In the example shown in FIG. 6A, a characteristic of being weak or good at one item such as "not good at memory problems" is stored, but the configuration is not limited to this. For example, a plurality of items such as "I am not good at memory problems and calculation problems", "I am not good at calculation problems and good at language problems", are collectively stored as one feature. may The stored contents of the type DB 42a may be registered in advance, or may be added as appropriate when a new type appears. The storage contents of the type DB 42a are not limited to the example shown in FIG. 6A.

The training DB 42b stores information on training for improving cognitive function and physical function. The training DB 42b shown in FIG. 6B includes an item column, a training ID column, a training content column, a type column, and the like, and stores information on training for each item for each item. The item column stores the item name of each item related to cognitive function in the same manner as the item column of the test DB 12a shown in FIG. The training ID column stores identification information (training ID) for identifying each training, and the training content column stores content data of each training. The training content data may be image data, audio data, or may include image data and audio data. In addition to being stored in the training DB 42b, the content data may be stored in a predetermined area of the storage unit 42 or another storage device. file name indicating the storage location). The type column stores the type (type ID) of the user's cognitive function to which each training content should be provided. With such a configuration, in the training DB 42b, training content to be provided to the user is associated and registered according to the type of cognitive function. The items stored in the training DB 42b are stored in advance, and are additionally stored by the control unit 41 when a new item is registered. The training ID stored in the training DB 42b is issued and stored by the control unit 41 when new training is registered according to each item. The training content and type stored in the training DB 42b are added by the control unit 41 when the control unit 41 acquires an addition instruction via the input unit 44 or the communication unit 43. FIG. The storage contents of the training DB 42b are not limited to the example shown in FIG. 6B.

The advice DB 42c stores information on advice provided to the user according to the type of cognitive function. The advice DB 42c shown in FIG. 6C includes an advice ID column, an advice content column, a type column, and the like, and stores information on advice in association with the advice ID. The advice ID column stores preset identification information (advice ID) for identifying each piece of advice. The advice content column stores the content data of each advice corresponding to the type of cognitive function. The advice content data may be image data, audio data, or may include image data and audio data. The advice may be, for example, content related to lifestyle habits such as diet and exercise, training, etc., and may include, for example, the content of meals according to the time of day, the purpose and effect of each training, and the like. Such advice may be content set by experts such as doctors, nurses, pharmacists, public health nurses, registered dietitians, psychological counselors, and exercise instructors according to the type and characteristics of cognitive function. Further, the advice includes the test result (correctness of the answer) for the test content output from the information processing device 10, the judgment result (cognitive function type) judged based on the answer information, the commentary on the test content or how to solve it, and the test content. It may include the reason for selection or purpose of questioning. Again, the content data may be stored in a predetermined area of the storage unit 42 or another storage device in addition to being stored in the advice DB 42c. In this case, the advice content string is information for reading out the content data ( For example, a file name indicating the storage location of data) is stored. The type column stores the type (type ID) of the user's cognitive function for which each piece of advice should be provided. With such a configuration, advice to be provided to the user is associated and registered in the advice DB 42c according to the type of cognitive function. The advice ID stored in the advice DB 42c is issued and stored by the control unit 41 when new advice is registered. The advice contents and types stored in the advice DB 42 c are added by the control section 41 when the control section 41 acquires an addition instruction via the input section 44 or the communication section 43 . The storage contents of the advice DB 42c are not limited to the example shown in FIG. 6C.

The dialogue DB 42d stores information related to dialogue with the user. The dialog DB 42d shown in FIG. 7 includes a dialog ID column, a dialog content column, a dialog content column, and the like, and stores information about a dialog to be spoken to the user in association with the dialog ID. The dialogue ID column stores identification information (dialogue ID) for identifying each pre-registered dialogue content. The dialogue content column stores information about the content of the dialogue with the user, such as greetings (morning greetings, afternoon greetings, evening greetings) or weather. The dialog content column stores content data of dialogs spoken to the user. The dialogue content data may be image data, audio data, or may include image data and audio data. For example, dialogue content whose dialogue content is "morning greeting" includes a display image and/or audio data of a message such as "Good morning. Did you sleep well?" Again, the content data may be stored in a predetermined area of the storage unit 42 or another storage device in addition to being stored in the dialog DB 42d. In this case, the dialog content string is information for reading the content data ( For example, a file name indicating the storage location of data) is stored. The dialogue ID stored in the dialogue DB 42d is issued and stored by the control unit 41 when new dialogue content is registered. The dialogue content and dialogue content stored in the dialogue DB 42 d are added by the control unit 41 when the control unit 41 acquires an addition instruction via the input unit 44 or the communication unit 43 . The storage contents of the dialogue DB 42d are not limited to the example shown in FIG. For example, the dialogue DB 42d may store, in association with each dialogue content, a time zone (dialogue time zone) during which the dialogue content should be output, or information about the user's utterance content for which the dialogue content should be output. The information processing apparatus 10 acquires dialogue contents registered in the dialogue DB 42 d from the server 40 and outputs them from the display unit 16 or the speaker 22 to speak to the user and to listen to the user's uttered voice through the microphone 18 . It is configured to have a dialogue with the user by acquiring from the server 40 and outputting from the display unit 16 or the speaker 22 the dialogue content corresponding to the acquired utterance content. Note that the server 40 may generate a message (dialogue content) according to the content of the user's utterance by using a known chatbot instead of the dialogue DB 42d.

FIG. 8 is a schematic diagram showing a configuration example of the learning model 42M. The learning model 42M is a trained model that has been learned to output information indicating the user's cognitive function type based on the input answer information of the user collected by the information processing apparatus 10. be. The learning model 42M is configured using an algorithm such as random forest, decision tree, SVM (Support Vector Machine), or the like. In addition to these, the learning model 42M may be configured using algorithms such as CNN and RNN (Recurrent Neural Network), or may be configured by combining a plurality of algorithms. The server 40 performs machine learning for learning predetermined training data to generate a learning model 42M in advance. The server 40 inputs the user's answer information acquired from the information processing apparatus 10 to the learning model 42M, and identifies the type of user's cognitive function based on the output information from the learning model 42M.

The learning model 42M shown in FIG. 8 is composed of an input layer, an intermediate layer and an output layer. The input layer has multiple input nodes, and each input node is associated with each piece of information included in the answer information. 42M. In the example shown in FIG. 8, the question tendency, answer time, correct/incorrect information, behavior information, facial expression information, time period, and dialogue tendency information included in the answer information are input. may be configured to input other information if the answer information includes other information. The intermediate layer uses various functions and threshold values to calculate an output value from each piece of information input via the input layer, and outputs the calculated output value to the output layer. Note that, for example, when the intermediate layer has multiple layers, the nodes of each layer calculate the output value based on the input information using the functions and thresholds between each layer, and the calculated output value is sequentially applied to the subsequent layers. input to the node of The intermediate layer sequentially inputs the output values of the nodes of each layer to the nodes of the succeeding layer, and finally gives each output value to each output node of the output layer. The output layer has a plurality of output nodes each associated with a preset type, and each output node outputs a discrimination probability for the associated type. In this embodiment, each output node outputs the discrimination probability for each type registered in the type DB 42a, for example, the discrimination probabilities for types with type IDs TY01, TY02, . . . The discrimination probability output by each output node indicates the possibility (probability) that the associated type is appropriate for the user of the input answer information. The output value of each output node in the output layer is, for example, a value between 0 and 1, and the sum of the discrimination probabilities output from each output node is 1.0. With the configuration described above, the learning model 42M outputs information (discrimination probability for each type) related to the user's cognitive function type when the user's answer information is input.

In the learning model 42M described above, the server 40 identifies the output node that outputs the maximum output value (discrimination probability) among the output values from the output layer, and identifies the type corresponding to the identified output node. Be specific to the type of function. The output layer of the learning model 42M has a structure having a plurality of output nodes that output the discrimination probability for each type, and also has one output node that outputs the type with the highest discrimination probability (type ID). may be configured.

The learning model 42M learns using training data in which information indicating the type of cognitive function of the user (correct label) is added to the answer information of the user. As the correct label type, for example, a cognitive function type determined by an expert such as a doctor based on the user's answer information can be used. As the answer information and type for training data, answer information obtained from the information processing apparatus 10 in the past and a type determined based on this answer information can be used. When the answer information included in the training data is input to the learning model 42M, the output value from the output node corresponding to the correct label (correct type) included in the training data approaches 1, and from other output nodes is learned so that the output value of approaches 0. Specifically, the learning model 42M performs an operation in the intermediate layer based on the input answer information, and calculates an output value from each output node in the output layer. Then, the learning model 42M compares the calculated output value of each output node with the value (0 or 1) corresponding to the correct label, and performs intermediate Optimize the parameters in the layer and the output layer. The learning model 42M optimizes data such as coefficients and threshold values of various functions used for calculations in the intermediate layer and the output layer, using, for example, error backpropagation. As a result, a learning model 42M that has been learned to output the discrimination probability for each type of cognitive function when the user's answer information is input is obtained.

Learning of the learning model 42M may be performed by a learning device other than the server 40 . A learned learning model 42M generated by learning in another learning device is downloaded from the learning device to the server 40 via, for example, the network N or the portable storage medium 4a and stored in the storage unit 42. The learning model 42M is not limited to the configuration shown in FIG.

Processing for learning the training data and generating the learning model 42M will be described below. FIG. 9 is a flow chart showing an example of a processing procedure for generating the learning model 42M. The following processing is performed by the control unit 41 of the server 40 according to the control program 42P stored in the storage unit 42, but may be performed by another learning device.

The control unit 41 of the server 40 acquires training data in which the type of correct answer is given to the answer information of a certain user (S11). The training data may be stored in a training data DB (not shown) in association with, for example, answer information received from the information processing apparatus 10 in the past and a type determined based on the answer information. In this case, the control unit 41 acquires training data from the training data DB. The control unit 41 performs learning processing of the learning model 42M using the acquired training data (S12). Here, the control unit 41 inputs each information of the answer information included in the training data from each input node of the input layer to the learning model 42M, and acquires the output value from each output node of the output layer. The control unit 41 compares the output value from each output node with the value corresponding to the correct answer type (1 for the output node corresponding to the correct answer type and 0 for the other output nodes), and the two are approximated. , the parameters in the hidden and output layers are optimized using, for example, error backpropagation. That is, the control unit 41 learns the learning model 42M so that the output value from the output node corresponding to the correct answer type approaches 1 and the output value from the other output nodes approaches 0.

The control unit 41 determines whether or not there is unprocessed training data (S13). For example, the control unit 41 determines whether or not there is training data that has not yet been used in the learning process among the training data stored in the training data DB. If it is determined that there is unprocessed training data (S13: YES), the control unit 41 returns to the processing of step S11, performs the processing of steps S11 and S12 on the training data that has not undergone the learning processing, and Repeat the learning process using If it is determined that there is no unprocessed data (S13: NO), the control unit 41 terminates the learning process. Thus, when the user's answer information is input, the learning model 42M learned to output the discrimination probability for each of the types registered in advance in the type DB 42a as the discrimination result of the user's cognitive function type. is generated. The learning model 42M that has already been trained can also be re-learned by performing the above-described processing, and in this case, a learning model 42M with higher discrimination accuracy can be generated. By repeating the learning process described above, the learning model 42M can be optimized, and when the learning model 42M is learned using a huge amount of training data, the type of user's cognitive function can be determined more accurately. A learning model 42M that can be used can be realized.

Below, a process will be described in which the information processing apparatus 10 identifies the type of cognitive function of the user from the answer information collected by asking questions to the user, and provides the user with content corresponding to the identified type. In the following process, server 40 identifies the type of user's cognitive function from the answer information received from information processing apparatus 10 using learning model 42M generated by the above-described process. FIG. 10 is a flow chart showing an example of a content provision processing procedure, and FIGS. 11 to 13 are schematic diagrams showing examples of screens. In FIG. 10, the processing performed by the information processing apparatus 10 is shown on the left side, and the processing performed by the server 40 is shown on the right side. The following processing is executed by the control unit 11 according to the control program 12P stored in the storage unit 12 of the information processing device 10, and is executed by the control unit 41 according to the control program 42P stored in the storage unit 42 of the server 40. be. A part of the following processing may be realized by a dedicated hardware circuit.

In the information processing system of the present embodiment, the control unit 11 of the information processing device 10 outputs standby content in a standby state in which test content, training content, or the like is not output (S21). The standby content includes image data or audio data, and when image data is included, the control unit 11 outputs the image data to the display unit 16 to display a standby screen (not shown). The standby screen displays, for example, the message "Let's touch the screen to start." When audio data is included in the standby content, the control unit 11 outputs the audio data to the speaker 22 or the robot 30 and outputs audio from the speaker 22 or the speaker 34 of the robot 30 . For example, the message "Let's start by touching the screen" is output by voice. Note that the standby content can be set for each user, and may be, for example, content set by the user's family or guardian, or content that the user is interested in. Further, in the standby state, the robot 30 may be configured to emit light with a color and brightness corresponding to the standby state from the light emitting unit 35, and may be configured to output predetermined music from the speaker 34. may In the processing shown in FIG. 10, the processing by the information processing apparatus 10 is started by a touch operation on the screen (touch panel), but the configuration is not limited to this. For example, the information processing apparatus 10 may start processing by voice input by the user uttering a predetermined message such as "start". The configuration may be such that the processing device 10 starts processing.

The control unit 11 determines whether or not a touch operation has been received via the input unit 15 (S22), and if it determines that it has not been received (S22: NO), it continues outputting the standby content. If it is determined that the touch operation has been received (S22: YES), the control unit 11 starts the process of asking the user the test content. Specifically, the control unit 11 specifies test content to be presented to the user from the test content stored in the test DB 12a (S23). For example, the control unit 11 may specify test content of the same item as the test content presented last time, or may specify test content of a different item from the test content presented last time. In addition, the control unit 11 may take into account correct/incorrect information (correct answers or incorrect answers) at the time of past questions, and identify, for example, test content that was incorrect. Furthermore, the control unit 11 may specify the test content to be set in consideration of the previous determination result (cognitive function type). The control unit 11 (program output unit) outputs the identified test content (S24). The test content includes image data, and the control section 11 outputs the image data to the display section 16 to display a test screen as shown in FIG. 11A, for example. Audio data may be included in the test content, and in this case, the control unit 11 outputs the audio data from the speaker 22 or the speaker 34 of the robot 30 . Before outputting the test content, the control unit 11 may output content for relaxing the user, such as weather or seasonal topics, news or sports topics, topics of interest to the user, and the like.

The test screen (test content) displays a message to the user, for example, as shown in FIGS. 11A to 11D, and each screen is switched in the order of FIGS. 11A to 11D at predetermined time intervals. It should be noted that a "next" button may be displayed on each of the test screens, and the next screen may be displayed when the user performs a touch operation on the "next" button. Note that the "next" button can be operated not only by a touch operation, but also by voice input by the user uttering "next", or by line-of-sight input by the user looking at the "next" button. may be configured to The test screen shown in FIG. 11B is the first screen of the test content. For example, when a predetermined period of time has passed so that the user can finish reading the displayed message, the test screen shown in FIG. 11C is displayed. 11D is displayed when a predetermined time has passed since the screen in FIG. 11C was displayed, and the test screen shown in FIG. 12A is displayed when a predetermined time has passed since the screen in FIG. 11D was displayed. A screen appears. On the screen of FIG. 12A, a message "You watched A well" is displayed, and a "Yes" button for selecting "Yes" and a "No" for selecting "No" are displayed as answers to this message. button is displayed. When the "Yes" button is selected on the screen of FIG. 12A, A is selected as the answer to the question set on the screen of FIG. 11D, and when the "No" button is selected, the answer to the question is B will be selected as Therefore, when answering A, the user operates the "Yes" button on the screen of FIG. 12A, and when answering B, operates the "No" button on the screen of FIG. 12A. When the "Yes" button or the "No" button is operated on the screen of FIG. 12A, the control section 11 displays the test screen shown in FIG. 12B. The correct answer is displayed on the screen of FIG. 12B. Note that, when the screen of FIG. 12B is displayed, the control unit 11 determines whether the user's answer received through the screen of FIG. 12A is correct or incorrect. After a predetermined period of time has passed since the screen of FIG. 12B was displayed, the control section 11 displays the test screen shown in FIG. 12C, and shifts to the display of the test screen of the next question. Each screen in FIGS. 11C to 12B is a test screen for one question, and the control unit 11 sequentially switches and displays the test screens at predetermined time intervals or according to the operation of buttons on the screen. In the screens of FIGS. 11C to 12B as well, each button may be operated not only by touch operation but also by voice input or line-of-sight input. As a result, the user can operate in a method that he/she is good at, and can reduce the operation burden.

Although the screen shown in FIG. 12A is configured to receive an answer from the user based on the position and movement of the user's line of sight on the screen of FIG. 11D, the configuration is not limited to this. For example, the control unit 11 may accept the user's answer by a touch operation on the image A or the image B on the screen of FIG. The user's answer may be accepted by voice input by doing.

When each test screen of FIGS. 11C to 12B is displayed (when a question is given), the control unit 11 captures the user with the camera 17 (S25), and acquires, for example, a captured image of the user's upper body. do. Further, the control unit 11 detects the behavior of the user by the behavior detection unit 20 based on the captured images of the user that are sequentially acquired (S26). The behavior includes, for example, whether the part the user is looking at is the display part of the problem, the display part of the option, or the part unrelated to the problem, or whether the user is staring, whether the line of sight is moving, and so on. Movement, the open/closed state of the user's mouth, the degree of swaying of the user's upper body or whole body, and the like are detected, and the control unit 11 acquires the behavior information detected by the behavior detection unit 20 . Further, the control unit 11 detects the user's facial expression using the facial expression detection unit 21 based on the captured images of the user that are sequentially acquired (S27). As the facial expression, for example, emotions such as joy, sadness, anger, surprise, and fear appearing on the user's face, or expressionlessness (straight face) are detected, and the control unit 11 acquires facial expression information detected by the facial expression detection unit 21. do. Further, the control unit 11 acquires information on the correct answer or incorrect answer (correct or incorrect) for the contents of the user's answer (S28). For example, the control unit 11 accepts the user's answer via the screen of FIG. 12A, and determines whether the input user's answer matches (is it correct) or does not match (is it correct) displayed on the screen of FIG. 12B? whether the answer is incorrect or not), and obtains correct/incorrect information for the user's answer.

The control unit 11 performs steps S23 to S28 for one question, for example. Therefore, for one question, the control unit 11 (acquisition unit) obtains the tendency of the test content (problem tendency), the response time required for the user to answer, the correct/incorrect information on the user's answer obtained in step S28, and the The behavior information acquired in step S27, the facial expression information acquired in step S27, the time period when the question was asked, and the conversation tendency are acquired as answer information. The dialogue tendency may include, for example, a tendency regarding dialogue with the user before outputting the test content, or a tendency regarding dialogue with the user during outputting the test content. Then, the control unit 11 stores answer information including the above-described information in the history DB 12b in association with the test ID of the given test content for one question. Note that the control unit 11 may be configured to acquire the user's impression of the question after the screen of FIG. 12B is displayed, for example. In this case, the acquired impression is also included in the answer information and stored in the history DB 12b. may The control unit 11 determines whether or not to end the questioning process (test) of the test content (S29). For example, the control unit 11 terminates the test content questioning process when a predetermined time has passed since the start of the test content questioning process for the user, or when a predetermined number of test content questions have been given to the user. judge that it should. If it is determined that the test content questioning process should not end (S29: NO), the control unit 11 returns to the process of step S23, and selects the next test content to be asked from the test content stored in the test DB 12a. Specify (S23). Then, the control unit 11 performs the processing of steps S24 to S28 for the specified test content, and the answer information (problem tendency, answer time, correct/incorrect information, behavior information, facial expression information, time zone, conversation tendency) for the set test content. is stored in the history DB 12b.

If it is determined that the process of setting the test content should be finished (S29: YES), the control unit 11 reads the answer information for each set of test content from the history DB 12b and transmits it to the server 40 (S30). When the control unit 41 (specification unit) of the server 40 acquires the user's answer information for each test content from the information processing device 10, based on the acquired answer information, the type of the user's cognitive function (classification related to cognitive function) is specified (S31). Here, the control unit 41 inputs the user's answer information to the learning model 42M, and identifies the type of user's cognitive function based on the output information from the learning model 42M. For example, the control unit 41 may input answer information to the learning model 42M for each test content to specify the type of the user's cognitive function, and determine the user's type from the specified type. Alternatively, for example, the number of test contents specified for each type (specified number) may be counted for each type, and the type with the largest specified number may be determined as the user type. The type of number (eg, two or three) may be determined to the type of user. Note that the process of specifying the type of cognitive function of the user from the answer information is not limited to the configuration using the learning model 42M. For example, for each type of cognitive function, the tendencies of behaviors that are highly likely to be performed by each type of user are registered in the DB, and based on the registered contents of the DB, the behavior tendencies indicated by the user's answer information are handled. You may specify the type of Also, a specialist such as a doctor may specify the type of cognitive function based on the tendency of behavior indicated by the user's answer information.

The control unit 41 identifies advice content to be provided to the user from the advice DB 42c according to the identified user type (S32), and identifies training content to be provided to the user from the training DB 42b (S33). For example, the control unit 41 identifies advice content stored in the advice DB 42c in association with the identified type, and identifies training content stored in the training DB 42b in association with the identified type. Then, the control unit 41 transmits the specified type, and the specified advice content and training content (determination result) to the information processing device 10 (S34). Note that the control unit 41 may generate (customize) advice content according to the characteristics of the user from the advice contents specified from the advice DB 42c, and may customize training contents according to the characteristics of the user from the training contents specified from the training DB 42b. It may be generated (customized).

When acquiring the determination result from the server 40, the control unit 11 of the information processing device 10 stores the acquired determination result in the history DB 12b (S35). Specifically, the control unit 11 associates the type ID of the type included in the determination result, the training ID of the training content, and the advice ID of the advice content, and stores them in the determination result column of the history DB 12b. After that, the control unit 11 (content output unit) outputs the advice content obtained from the server 40 (S36), and also outputs the training content obtained from the server 40 according to the user's operation via the input unit 15. Output (S37). The advice content includes image data, and the control section 11 outputs the image data to the display section 16 to display, for example, an advice screen as shown in FIG. 13A. Audio data may be included in the advice content, and in this case, the control unit 11 outputs the audio data from the speaker 22 or the speaker 34 of the robot 30 . The advice screen shown in FIG. 13A displays a "next" button, and when the user operates the "next" button, the control unit 11 outputs training content, for example. The training content also includes image data, and the control section 11 outputs the image data to the display section 16 to display a training screen such as that shown in FIG. 13B or 13C, for example. Note that the training content may also include audio data. In this case, the control unit 11 outputs the audio data to the speaker 22 or the robot 30 and outputs audio from the speaker 22 or the speaker 34 of the robot 30 . The screen in FIG. 13B is, for example, a training content screen for memory training, and the screen in FIG. 13C is a training content screen for oral cavity training (patacara exercises). When the training screens shown in FIGS. 13B and 13C are displayed, the control unit 11 provides the training content to the user by displaying the next screen of the training content after the lapse of a predetermined time.

Through the above-described processing, the information processing apparatus 10 can pose a question (test content) to the user and collect answer information indicating the state of the user's answer. Further, the server 40 can identify the type of cognitive function of the user based on the answer information collected by the information processing device 10 and provide the user with advice and training according to the identified type through the information processing device 10 . In this embodiment, the type of cognitive function is specified based not only on the correctness or wrongness of the user's answer to the question, but also on the tendency of the user's behavior when answering (answer information). It is possible to identify the type. In the present embodiment, when performing the questioning process of the test content next time, the test content can be identified by considering the determination result (type) stored in the history DB 12b. It is possible to provide test content based on Therefore, test contents (questions) can be set in consideration of the user's weak field or strong field. In this way, by repeatedly providing test content, advice content, and training content considering the type of user's cognitive function, the user's cognitive function can be determined more accurately, and personalized content suitable for the user can be provided. can. In addition, since it is possible to provide such determination processing and training at the user's home or the like, it is possible to easily perform determination and training of cognitive function.

In this embodiment, a problem is presented to the user, the type of cognitive function is determined based on the answer information at that time, and advice content and training content are provided according to the determination result. is not limited to For example, once a month, the process of setting test content according to the previous judgment result (type) is performed, and training content is provided for one month according to the new judgment result based on the user's answer information to the test content. Can be configured. Further, it may be configured such that the process of setting test content questions is performed at an arbitrary timing in accordance with an instruction from the user. In this case, the determination process can be performed at the timing when the user wants to take the test, and the content of the training can be changed according to the result of the determination process performed at any timing.

In the present embodiment, the information processing device 10 may be configured to perform the process of identifying the cognitive function type of the user based on the user's answer information. For example, by storing the learning model 42M in the storage unit 12 of the information processing apparatus 10, when the control unit 11 acquires the user's answer information, the learning model 42M is used to obtain the user's answer information from the acquired answer information. can identify the type of cognitive function of In such a configuration, the information processing apparatus 10 may be configured to transmit the specified type to the server 40 and acquire advice content and training content corresponding to the specified type from the server 40. . Further, when the training DB 42b and the advice DB 42c are stored in the storage unit 12 of the information processing device 10, the control unit 11 identifies advice content and training content corresponding to the identified type without inquiring the server 40. can be output as

In this embodiment, when the type of cognitive function is specified from the answer information acquired from the information processing device 10, the server 40 associates the answer information with the specified type (type ID) and stores them in the DB. You can In this case, it is possible to use the answer information and types accumulated in the DB as training data when generating the learning model 42M.

In this embodiment, the input unit 15 and the display unit 16 of the information processing apparatus 10 are touch panels, and the user inputs an answer to the question by a touch operation on the touch panel. Note that the user's answer may be input by voice input, input by line of sight (line of sight), input by gesture, or the like. In addition, when the robot 30 is configured to accept voice input via the microphone 37 , the configuration may be such that the user's answer is input by voice input via the robot 30 microphone 37 . In this case, the robot 30 transmits the voice data acquired via the microphone 37 to the information processing device 10, and the information processing device 10 converts the voice data into text data to convert the voice input information (user's answer) may be obtained. Alternatively, the control unit 31 of the robot 30 may convert the acquired voice data into text data and then transmit the text data to the information processing apparatus 10 , so that the information processing apparatus 10 acquires the user's answer. Also, the robot 30 may be configured to acquire the user's answer information. For example, the robot 30 is provided with a camera, and the camera of the robot 30 photographs a user who answers test content provided through the information processing device 10, and information is processed by detecting the user's behavior and facial expression from the photographed user's face image. It may be transmitted to device 10 .

In the present embodiment, the user type is specified based on the user's answer information to the test content from the preset cognitive function type, but the present invention is not limited to such a configuration. For example, based on the user's answer information, the user's weak or good at each item (field) such as memory power, calculation power, language ability, judgment power, physical ability, weak or good at each answer method such as touch operation, voice input, eye line input , Problems that move the mouth or body, problems with long problem sentences, problems that require understanding and answering diagrams, etc. However, it may be configured to identify a classification (type) related to cognitive function based on the user's characteristics. In addition, characteristics such as the fact that the answers to the questions are often incorrect, but they continue training every day, the answers take time, and they are good at two-choice questions are extracted, and recognition is based on these characteristics. A functional classification may be specified.

In the present embodiment, the server 40 may be configured to associate, for each user, the response information acquired from the information processing device 10 with the type of cognitive function specified from the response information, and store them in the DB. . In this case, the type identified from the user's answer information can be managed in chronological order, and the past determination result (type) can be taken into consideration when newly identifying the user's type. In addition, in the case of such a configuration, the server 40 can output the past judgment results to the information processing device 10 together with the judgment results (type, training, advice), and the user can change the new judgment results to the past judgment results. It can be confirmed by comparing with the result. In addition, by registering attributes such as age and gender, past medical history (medical history), treatment history, medication history, etc. as information of each user in the DB, it is possible to identify the type of cognitive function of the user. , can also take into account these information of the user.

(Embodiment 2)
A description will be given of an information processing system in which the information processing apparatus 10 starts the process of setting test content when the human sensor 36 of the robot 30 detects the approach of the user to the robot 30 and the information processing apparatus 10 . The information processing system of the present embodiment can be realized by devices similar to those of the information processing system of the first embodiment, so description of the configuration of each device will be omitted.

FIG. 14 is a flow chart showing an example of a content provision processing procedure according to the second embodiment. The process shown in FIG. 14 is obtained by adding steps S41 to S43 instead of step S22 in the process shown in FIG. Description of the same steps as in FIG. 10 will be omitted. In FIG. 14, illustration of steps other than steps S21 and S23 to S24 in FIG. 10 is omitted. In FIG. 14, the processing performed by the robot 30 is shown on the left side, and the processing performed by the information processing apparatus 10 is shown on the right side.

In the information processing system of the present embodiment, the controller 31 of the robot 30 determines whether or not the human sensor 36 has detected the approach of the user in a standby state where there is no user near the robot 30 (S41). ), if it is determined that it is not detected (S41: NO), it waits until it is detected. When determining that the user's approach has been detected (S41: YES), the control unit 31 notifies the information processing device 10 of the user's approach (S42). In the present embodiment, when the control unit 11 of the information processing apparatus 10 is outputting standby content (S21), it determines whether or not the robot 30 has notified the user of the approach through the second communication unit 14. (S43). If it is determined that the user's approach has not been notified (S43: NO), the control unit 11 returns to the process of step S21 and continues outputting the standby content. If it is determined that the user's approach has been notified (S43: YES), the control unit 11 shifts the process to step S23, and starts the process of setting the test content to the user. That is, the control unit 11 of the information processing device 10 and the control unit 41 of the server 40 perform the processes of steps S23 to S37 in FIG.

According to the process described above, in this embodiment, when the human sensor 36 of the robot 30 detects the user's approach, a test screen as shown in FIG. Questions (test content) for . Also in this embodiment, the information processing apparatus 10 sequentially displays the screens of FIGS. Collect (problem tendency, answer time, correct/incorrect information, behavior information, facial expression information, time period, conversation tendency). Thus, also in this embodiment, based on the user's answer information, it is possible to identify the type of cognitive function that better reflects the characteristics of the user, and provide the user with advice and training according to the identified type. be able to.

In this embodiment, the same effects as those of the first embodiment described above can be obtained. Further, in this embodiment, when the human sensor 36 of the robot 30 detects the user's approach, the information processing apparatus 10 starts the process of setting the test content, so the user performs an operation to start the test. you don't have to. In this embodiment, the robot 30 may be configured to emit light from the light emitting unit 35 or output sound from the speaker 34 when the human sensor 36 detects the user's approach. In this case, the presence of the robot 30 can be appealed to the user who has approached the robot 30, and training can be promoted.

(Embodiment 3)
An information processing system that extracts a user's hobbies, preferences, interests, or the like based on the content of a conversation uttered by the user, and uses the content of the extracted content as standby content will be described. The information processing system of the present embodiment can be realized by devices similar to those of the information processing system of the first embodiment, so description of the configuration of each device will be omitted.

FIG. 15 is a flowchart showing an example of a content providing processing procedure according to the third embodiment, and FIG. 16 is a schematic diagram showing an example of a screen according to the third embodiment. The process shown in FIG. 15 is obtained by adding steps S51 to S55 in place of step S21 in the process shown in FIG. Description of the same steps as in FIG. 10 will be omitted. In FIG. 15, illustration of steps other than steps S22 to S24 in FIG. 10 is omitted.

In the information processing system of the present embodiment, the control unit 11 of the information processing device 10 speaks to the user by outputting a message from the speaker 22 or the speaker 34 of the robot 30, for example, and uses the microphone 18 or the microphone 37 of the robot 30. A user's uttered voice is acquired through the speaker 22 or 34, and a message corresponding to the acquired uttered content is output as voice from the speaker 22 or 34, thereby allowing dialogue with the user. The message according to the content of the user's utterance is provided by the server 40, but may be generated by a known chatbot. The chatbot function may be installed in any of the robot 30 , the information processing device 10 , or the server 40 . The information processing apparatus 10 of the present embodiment extracts (estimates) the user's hobbies, interests, or the like, based on the content of dialogue with the user.

Therefore, the control unit 11 of the information processing apparatus 10 outputs the information (eg, text data) voice-output from the speaker 22 or 34 and the utterance content (eg, text data) generated from the user's utterance voice acquired via the microphone 18 or 37 data) has been acquired (S51), and if it is determined that it has not been acquired (S51: NO), it waits until it is acquired. At this time, the control unit 11 may output preset standby content, as in step S21 in FIG.

If it is determined that the dialogue information has been acquired (S51: YES), the control unit 11 extracts information about the user's hobbies and preferences, such as matters in which the user is interested, matters that the user likes, etc., from the acquired dialogue information. (S52). For example, if the number of times the user responds to a conversation about cooking is large, or if the user seems to be enjoying themselves, the information about the user's hobbies and preferences is "cooking" (that the user is interested in cooking). to extract Further, the control unit 11 may be configured to identify information about the user's tastes and preferences from dialogue information using a neural network constructed by machine learning. For example, a learning model configured by a CNN or RNN model and trained to output information (information on tastes and preferences) contained in dialogue content when dialogue information is input may be used. In this case, the control unit 11 can input dialogue information to a learned learning model and specify information about the user's tastes and preferences contained in the dialogue content based on the output information from the learning model.

When the information about the user's tastes and preferences can be extracted, the control unit 11 stores the extracted information about the tastes and preferences in the storage unit 12, for example (S53). Further, the control unit 11 generates content (second content, hereinafter referred to as hobby preference content) corresponding to the user's interest and preference based on the extracted interest and preference information (S54). For example, the control unit 11 searches websites open to the public via the network N for matters in which the user is interested, and uses the information of the searched websites as hobby and taste contents for the user. may Further, for example, when video content such as a photograph or a moving image taken or obtained by the user is stored in the storage unit 12, the control unit 11 reads the video content related to the matter in which the user is interested from the storage unit 12, It may also be used as hobby and taste content for the user. Note that when video content is classified into categories such as travel, cats, and automobiles and stored in the storage unit 12, the control unit 11 identifies a category corresponding to the matter in which the user is interested, You may use the video content of the said category for a user's taste preference content. Further, for example, when video content captured or acquired by the user is accumulated in a cloud server (not shown), the control unit 11 reads out video content related to matters in which the user is interested from the cloud server and provides it to the user. hobbies and tastes content.

When the user's hobby/preference content is generated, the control unit 11 outputs the generated hobby/preference content (S55). The hobby content here includes image data, and the control unit 11 outputs the image data to the display unit 16 to display a content screen as shown in FIG. 16, for example. The screen shown in FIG. 16 is a content screen generated when the user is interested in shrines and temples, sightseeing spots, travel, etc., and displays images of shrines. Also, the screen of FIG. 16 is superimposed on the content screen corresponding to the user's tastes and preferences, and displays the message "Touch the screen to start!". Therefore, the user can view content according to his or her tastes and preferences, and can start cognitive function testing or training by performing a touch operation on the screen. Note that, if the content according to the user's tastes and preferences includes audio data, the control unit 11 may output the audio data from the speaker 22 or the speaker 34 of the robot 30 as sound.

When the control unit 11 of the information processing apparatus 10 outputs the hobby/preference content for the user generated by the process described above, the process proceeds to step S22 and determines whether or not the touch operation is received via the input unit 15. (S22). Note that in this embodiment, since the information on the user's tastes and preferences extracted from the content of the dialogue with the user is stored in the storage unit 12, when the control unit 11 specifies the test content to be provided to the user in step S23, In addition, it is possible to specify the test content according to the user's tastes and preferences. For example, for each category such as cooking, beauty, travel, sightseeing spots, automobiles, electrical appliances, etc., test content for each category is prepared and registered in the test DB 12a. can be asked. In addition, by preparing training content for each category in the server 40 and registering it in the training DB 42b, it is possible to provide training content according to the tastes and preferences of the user. Further, by preparing advice contents for each category and registering them in the advice DB 42c, it is possible to provide advice contents according to the tastes and preferences of the user.

In the process shown in FIG. 15, the processes of steps S51 to S54 are executed when the information processing apparatus 10 has interacted with the user, and the process of step S55 is executed when the information processing apparatus 10 is in a standby state. may be executed. As a result, when the information processing apparatus 10 interacts with the user, the user's tastes and preferences can be extracted, and content corresponding to the user's tastes and preferences can be generated. Can be used as content.

In this embodiment, the same effects as those of the above-described embodiments can be obtained. Further, in the present embodiment, it is possible to output, as standby content, content corresponding to the user's hobbies and preferences specified from the content of dialogue with the user. Further, in the present embodiment, when test content, training content, or advice content is prepared in association with each of a plurality of categories, test content, training content, or advice content according to the user's tastes and preferences can be obtained. can be provided. In addition to the contents according to the tastes and preferences of the user, the standby contents may be contents according to the user's age, gender, hometown, etc., for example. In this case, for example, content associated with the user, such as video of what was popular when the user was a child, video of the user's hometown, etc., can be used as standby content. The configuration of the present embodiment can also be applied to the information processing system of the second embodiment, and similar effects can be obtained even when applied to the information processing system of the second embodiment. In addition, in the information processing system of this embodiment, it is possible to apply the modified examples appropriately described in each of the above-described embodiments.

(Embodiment 4)
An information processing system that causes the robot 30 to perform an action according to the determination result (cognitive function type) determined based on the user's answer information will be described. The information processing system of the present embodiment can be realized by devices similar to those of the information processing system of the first embodiment, so description of the configuration of each device will be omitted.

FIG. 17 is a flow chart showing an example of a content provision processing procedure according to the fourth embodiment, and FIG. 18 is a schematic diagram showing an example of a screen according to the fourth embodiment. The process shown in FIG. 17 is obtained by adding steps S61 and S62 after step S37 in the process shown in FIG. Description of the same steps as in FIG. 10 will be omitted. Also, in FIG. 17, the illustration of steps S21 to S33 in FIG. 10 is omitted. 17, the processing performed by the robot 30 is shown on the left side, the processing performed by the information processing apparatus 10 is shown in the center, and the processing performed by the server 40 is shown on the right side.

In the information processing system of this embodiment, the control unit 11 of the information processing device 10 and the control unit 41 of the server 40 perform the processes of steps S21 to S37 in FIG. As a result, the information processing apparatus 10 can set test content questions to the user, collect answer information when the user answers, and determine the result (cognitive function type) based on the answer information of the user, and give advice. and training can be obtained from server 40 . After the process of step S37, the control unit 11 of the information processing device 10 instructs the robot 30 to perform an operation according to the determination result (type) obtained from the server 40 (S61). In the information processing apparatus 10 of the present embodiment, for example, the DB stores the color or brightness of the light output from the light emitting unit 35 of the robot 30 in association with each type specified by the server 40 . Therefore, the control unit 11 can specify from the DB the color or brightness corresponding to the determination result obtained from the server 40, and instruct the robot 30 to perform the light emission process with the specified color or brightness. Further, the information processing apparatus 10 may store music output as voice from the speaker 34 of the robot 30 in association with each type. music corresponding to is specified from the DB, and the robot 30 is instructed to output the specified music.

When the controller 31 of the robot 30 is instructed to perform an action according to the determination result from the information processing device 10, the controller 31 performs the instructed action (S62). For example, when instructed to perform light emission processing with a color or luminance corresponding to the determination result, the control unit 31 causes the light emitting unit 35 to emit light of the instructed color or luminance. Further, when instructed to output music according to the determination result, the control unit 31 outputs the instructed music from the speaker 34 . The information processing apparatus 10 not only instructs the robot 30 to perform an action according to the determination result, but also changes the display mode of the advice content or the training content displayed on the display unit 16 according to the determination result. good too. For example, FIG. 18A shows an example of a display mode when the determination result is a standard type, and FIG. 18B shows an example of a display mode when the determination result is a better than standard type. In this way, while the advice content or training content is displayed in a manner corresponding to the type of the judgment result, the robot 30 performs an action according to the judgment result, thereby providing the judgment result to the user in a manner that makes it easy to recognize whether the judgment result is good or bad. can do.

In this embodiment, the same effects as those of the above-described embodiments can be obtained. Further, in the present embodiment, the user can be notified of the determination result (cognitive function type) determined based on the user's answer information from the robot 30 and the information processing device 10 in a manner corresponding to the determination result. Therefore, the user can easily grasp the quality of his own cognitive function. In particular, if the determination result is good and the robot 30 is caused to output music or light that raises the user's tension, it can be expected that motivation for training will be improved. The configuration of this embodiment can also be applied to the information processing systems of the second and third embodiments, and similar effects can be obtained even when applied to the information processing systems of the second and third embodiments. In addition, in the information processing system of this embodiment, it is possible to apply the modified examples appropriately described in each of the above-described embodiments.

In the present embodiment, the robot 30 may be configured so that the head in the shape of a bird can rotate with respect to the body about the vertical direction. By rotating the part, the robot 30 can be caused to swing. With such a configuration, it is possible to cause the robot 30 to perform a swing motion according to the determination result. Also, the robot 30 may be configured to perform the swing motion randomly or in a preset pattern. When the robot 30 performs the swinging motion, the user can be given the impression of being alive, and the user can sense that the robot 30 is reacting to him/herself and feel familiarity. This allows the user to feel a sense of familiarity with the robot 30 and relax while training. The shape of the robot 30 may be, for example, cat-shaped, dog-shaped, or human-shaped, and preferably has a small shape that gives the user a sense of familiarity.

(Embodiment 5)
An information processing system that allows a medical practitioner such as a doctor to perform a remote diagnosis after performing a test (examination) on cognitive function will be described. FIG. 19 is a schematic diagram showing a configuration example of an information processing system according to the fifth embodiment. The information processing system of this embodiment includes the information processing device 10, the robot 30, and the server 40, as well as a doctor's terminal 50 used by a medical worker such as a doctor. The information processing device 10, the robot 30, and the server 40 have the same configurations as the information processing device 10, the robot 30, and the server 40 of the first embodiment, and thus the description thereof is omitted. The doctor's terminal 50 is a general-purpose information processing device such as a personal computer, tablet terminal, or smart phone. The doctor's terminal 50 has a configuration including each section other than the behavior detection section 20, the facial expression detection section 21, and the second communication section 14 in the information processing apparatus 10 shown in FIG. 2, so detailed description of the configuration is omitted. . The storage unit of the doctor's terminal 50 stores an application program for videophone or video chat via the network N, in addition to the control program executed by the control unit. The storage unit 12 of the information processing apparatus 10 of this embodiment also stores an application program for videophone or video chat via the network N. FIG.

In the information processing system of the present embodiment, as in the information processing system of the first embodiment, the type of cognitive function of the user is specified based on the user's answer information to the test content given by the information processing apparatus 10, and is specified. A result is provided from the server 40 to the information processing device 10 . The user can request remote examination by a medical professional such as a doctor via advice content or training content provided by the server 40, and when the user makes a request, the information processing device 10 and the doctor terminal 50 Telemedicine is conducted by telephony or video chat.

FIG. 20 is a flowchart showing an example of a content provision processing procedure according to the fifth embodiment, and FIG. 21 is a schematic diagram showing an example of a screen according to the fifth embodiment. The process shown in FIG. 20 is obtained by adding steps S71 to S72 after step S34 and steps S73 to S78 after step S37 in the process shown in FIG. Description of the same steps as in FIG. 10 will be omitted. Also, in FIG. 20, illustration of steps S21 to S33 in FIG. 10 is omitted. 20, the processing performed by the information processing apparatus 10 is shown on the left side, the processing performed by the server 40 is shown in the center, and the processing performed by the doctor's terminal 50 is shown on the right side.

In the information processing system of this embodiment, the control unit 11 of the information processing device 10 and the control unit 41 of the server 40 perform the processes of steps S21 to S37 in FIG. After the processing of step S34, the control unit 41 of the server 40 transmits the information (user information) on the user of the information processing device 10 and the determination result to the doctor's terminal 50 (S71). The doctor's terminal 50 is a terminal registered in advance, and is a terminal in which a doctor (medical worker) can perform a remote diagnosis. When a plurality of doctors' terminals 50 are registered, the control unit 41 selects an arbitrary doctor's terminal 50 or a doctor's terminal 50 capable of remote diagnosis, and sends user information to the selected doctor's terminal 50. and send the judgment result. In addition to the user information and the determination result, the control unit 41 may also transmit to the doctor's terminal 50 the content of advice content and training content provided to the user according to the determination result. When the user information and the determination result are received from the server 40, the doctor's terminal 50 stores the received user information and the determination result in the storage unit (S71) and waits. Thereafter, the doctor's terminal 50 waits until receiving a connection request for remote diagnosis from the information processing apparatus 10, for example.

On the other hand, after the process of step S37, the control unit 11 of the information processing apparatus 10 determines whether or not an instruction to perform remote diagnosis by the medical staff has been received via the advice screen or the training screen displayed on the display unit 16 ( S73). The advice screen or the training screen of the present embodiment displays, for example, a "remote examination" button as shown in FIG. 21A. When the user operates the "remote examination" button, the control unit 11 , to receive a remote medical examination execution instruction. If the control unit 11 determines that the remote diagnosis execution instruction has not been received (S73: NO), the process returns to step S36 to continue outputting the advice content or the training content.

If it is determined that the execution instruction for remote diagnosis has been received (S73: YES), the control unit 11 requests connection to the doctor's terminal 50 (S74). A URL (Uniform Resource Locator) for connecting to the doctor's terminal 50 is linked to the "remote medical examination" button on the advice screen or the training screen, and the "remote medical examination" button is operated. In this case, the control unit 11 accesses the linked URL. In addition, when requesting connection to the doctor's terminal 50 , the control unit 11 may transmit to the doctor's terminal 50 the details and results of training performed by the user using the training content. Further, the content and results of the user's training are transmitted from the information processing device 10 to the doctor's terminal 50, and are also transmitted from the information processing device 10 to the server 40 and transmitted from the server 40 to the doctor's terminal 50. It may be a configuration that As a result, a doctor or the like can be notified of not only the user's cognitive function type (determination result) but also the content of the training provided to the user and the result of the training via the doctor's terminal 50 .

When receiving a connection request from the information processing apparatus 10, the doctor terminal 50 displays, for example, a message indicating that the information processing apparatus 10 is requesting connection on the display unit, and asks the medical staff for permission to connect. Accepted by the input unit. The doctor's terminal 50 determines whether or not permission for connection has been received from the medical staff (S75), and if it determines that it has not been received (S75: NO), it waits until it is received. Note that, for example, when the connection is refused by the medical staff, the doctor's terminal 50 may notify the information processing apparatus 10 of the connection rejection and abandon the remote diagnosis.

If it is determined that the connection permission from the medical staff has been received (S75: YES), the doctor terminal 50 transmits the connection permission to the information processing device 10 (S76), and performs remote diagnosis with the information processing device 10. Start (S77). On the other hand, when receiving the connection permission from the doctor's terminal 50, the control unit 11 of the information processing device 10 starts remote diagnosis with the doctor's terminal 50 (S78). Specifically, the control unit 11 of the information processing device 10 sends the user's captured image captured by the camera 17 and the user's uttered voice (data related to remote diagnosis) acquired via the microphone 18 to the doctor via the network N. to the terminal 50. Similarly, the control unit of the doctor's terminal 50 processes the captured image of the medical staff captured by the camera and the spoken voice of the medical staff (data related to remote diagnosis) acquired via the microphone via the network N. Send to device 10 . Note that the control unit 11 of the information processing device 10 displays the photographed image of the medical staff received from the doctor terminal 50 on the display unit 16, and outputs the spoken voice of the medical staff from the speaker 22 or the speaker 34 of the robot 30. do. Further, the control unit of the doctor's terminal 50 displays the captured image of the user received from the information processing device 10 on the display unit, and outputs the user's uttered voice from the speaker.

By the above-described processing, the display unit 16 of the information processing apparatus 10 displays, for example, a photographed image of the medical staff as shown in FIG. be able to.

In this embodiment, the same effects as those of the above-described embodiments can be obtained. In addition, in this embodiment, after the determination result (cognitive function type) determined by the server 40 is notified to the user, it becomes possible for the medical staff to conduct a remote diagnosis according to the user's request. As a result, the user can easily receive a medical examination by a medical professional, and if he or she has any concerns or worries, he or she can consult with the medical professional and resolve the concern. The configuration of this embodiment can also be applied to the information processing systems of the second to fourth embodiments, and similar effects can be obtained even when applied to the information processing systems of the second to fourth embodiments. In addition, in the information processing system of this embodiment, it is possible to apply the modified examples appropriately described in each of the above-described embodiments.

In this embodiment, the server 40 may be configured to transmit the determination result for the user to the terminal of the user's family or guardian, in addition to transmitting the result to the doctor's terminal 50 . In this case, each time the type of cognitive function of the user is identified, the determination result can be shared with family members, guardians, or the like.

(Embodiment 6)
An information processing system in which only the information processing device 10 executes the processing that was executed by the information processing device 10 and the robot 30 in the information processing systems of the first to fifth embodiments described above will be described. The information processing system of this embodiment includes an information processing apparatus 10 and a server 40, and the server 40 has the same configuration as the servers 40 of the first to fifth embodiments. The information processing apparatus 10 of this embodiment has the functions of the robot 30 in addition to the functions of the information processing apparatuses 10 of the first to fifth embodiments described above. Descriptions of the same configurations as those of the first to fifth embodiments described above will be omitted.

FIG. 22 is a block diagram showing an internal configuration example of the information processing apparatus 10 of the sixth embodiment. The information processing apparatus 10 of this embodiment has a human sensor 23 instead of the second communication unit 14 in the configuration shown in FIG. The human sensor 23 has the same configuration as the human sensor 36 of the robot 30 shown in FIG. Note that the information processing apparatus 10 of this embodiment does not have to include the human sensor 23 .

The information processing apparatus 10 of the present embodiment having the configuration described above is capable of executing the processing of the first embodiment shown in FIG. Therefore, also in the present embodiment, the information processing apparatus 10 presents test content to the user, collects the user's answer information, and acquires the type of cognitive function determined based on the user's answer information from the server 40. be able to. Therefore, the information processing apparatus 10 can provide the user with advice and training according to the type of cognitive function that reflects the characteristics of the user.

Further, the information processing apparatus 10 of the present embodiment performs the processes of the robot 30 (steps S41 to S42) and the processes of the information processing apparatus 10 (steps S21, S43, S23 to S30, S35) in the process of the second embodiment shown in FIG. to S37) can be executed. In the process shown in FIG. 14, the control unit 11 of the information processing apparatus 10 outputs standby content (S21) and determines whether or not the human sensor 23 detects the approach of the user ( S41), when the approach of the user is detected (S43: YES), the process after step S23 is executed. Also in this case, the same effects as in the second embodiment can be obtained.

Further, the information processing apparatus 10 of the present embodiment performs the processing by the robot 30 (step S62) and the processing by the information processing apparatus 10 (steps S21 to S30, S35 to S37, S61) in the processing of the fourth embodiment shown in FIG. It is executable. Note that in the process shown in FIG. 17, the control unit 11 of the information processing device 10 displays a screen in a display mode according to the determination result on the display unit 16 instead of instructing the robot 30 to perform an action according to the determination result. Then, music corresponding to the determination result is output from the speaker 22.例文帳に追加Also in this case, the same effects as in the third embodiment can be obtained.

Furthermore, the configuration of this embodiment can also be applied to the information processing systems of the third and fifth embodiments, and similar effects can be obtained even when applied to the information processing systems of the third and fifth embodiments. Specifically, the information processing apparatus 10 of the present embodiment can execute the processing (steps S51 to S55, S22 to S30, S35 to S37) by the information processing apparatus 10 in the processing of the third embodiment shown in FIG. be. Further, the information processing apparatus 10 of the present embodiment is capable of executing the processing (steps S21 to S30, S35 to S37, S73 to S74, S78) by the information processing apparatus 10 in the processing of the fifth embodiment shown in FIG. . As described above, the processing by the information processing device 10 and the robot 30 in the information processing systems of Embodiments 1 to 5 can be configured to be executed only by the information processing device 10, and similar effects can be obtained. Furthermore, in the information processing system of the present embodiment, it is possible to apply the modified examples appropriately described in each of the above-described embodiments.

Further, in the present embodiment, the learning model 42M, the training DB 42b, and the advice DB 42c stored in the storage unit 42 of the server 40 are stored in the storage unit 12 of the information processing device 10, and the server 40 uses the user's answer information to The information processing apparatus 10 may perform the process of identifying the type of cognitive function of the user and the process of identifying advice content and training content corresponding to the type. With such a configuration, the information processing apparatus 10 can be caused to execute not only the processing executed by the robot 30 but also the processing executed by the server 40 in each of the first to fifth embodiments described above.

(Embodiment 7)
An information processing system that gives points to a user according to the implementation status of cognitive function determination processing and training performed by the user via the information processing device 10 and the robot 30 will be described. Further, in the information processing system of the present embodiment, the server 40 determines the cognitive function and collects a service usage fee for a service that provides the user with training according to the determination result. If the result of training is not good (if the result of training is low), an insurance that refunds the service usage fee is sold. Therefore, when using the service provided by the server 40, the user can choose whether or not to take out insurance. The information processing system of the present embodiment can be realized by devices similar to those of the information processing system of the first embodiment, so description of the configuration of each device will be omitted. The server 40 of this embodiment stores a user information DB 42e in the storage unit 42 in addition to the configuration shown in FIG.

FIG. 23 is a schematic diagram showing a configuration example of the user information DB 42e stored by the server 40. As shown in FIG. The user information DB 42e determines the type of cognitive function by the server 40 based on the user's answer information acquired by the information processing apparatus 10, and information on the user registered as a user to acquire advice content and training content from the server 40. memorize The user information DB 42e shown in FIG. 23 includes a user ID column, an insurance policy content column, a determination result column, a point column, etc., and stores each piece of information about the user in association with the user ID. The user ID column stores identification information (user ID) for identifying a user who has registered with the server 40 . The insurance contract content column stores information indicating the content of the insurance that the user subscribes (contracts to). The content of the insurance is, for example, a refund of the service fee if the result of the training provided by the server 40 is low, but is not limited to this. The determination result column stores the user's determination results (type, training, advice) determined based on the user's answer information in chronological order. The point string stores the points given to the cognitive function test or training performed by the user via the information processing device 10, specifically, the points given to each cognitive function test or training. (additional value) and total points (total value) are stored. A user ID stored in the user information DB 42 e is issued and stored by the control unit 41 when a new user is registered with the server 40 . The content of the insurance contract is stored by the control unit 41 when the control unit 41 acquires it via the input unit 44 or the communication unit 43 . As for the determination result, when the control unit 41 performs determination processing based on the answer information acquired from the information processing device 10 , each piece of information of the determination result is stored by the control unit 41 . Points are stored by the control unit 41 when the control unit 41 gives points to each user. The contents stored in the user information DB 42e are not limited to the example shown in FIG. For example, the user information DB 42e may store various types of information input by the user at the time of user registration, such as the user's name, age, and gender.

FIG. 24 is a flowchart showing an example of a content providing processing procedure according to the seventh embodiment, and FIG. 25 is a schematic diagram showing an example of a screen according to the seventh embodiment. The process shown in FIG. 24 is obtained by adding step S81 between steps S33 and S34 in the process shown in FIG. Description of the same steps as in FIG. 10 will be omitted. In FIG. 24, illustration of steps other than steps S32 to S37 in FIG. 10 is omitted.

In the information processing system of this embodiment, the control unit 11 of the information processing device 10 and the control unit 41 of the server 40 perform the processes of steps S21 to S33 in FIG. Accordingly, the server 40 can determine the type of cognitive function based on the user's answer information acquired from the information processing device 10, and can specify advice and training according to the determination result. After the process of step S33, the control unit 41 of the server 40 specifies points to be given to the user who has performed the determination process, and adds the specified points to the user (S81). For example, based on the answer information acquired from the information processing device 10, the control unit 41 determines the number of questions answered by the user (the number of questions output), the number of advice content or training content provided to the user (the number of content outputs). ) or the like to specify the points to be given to the user. Points may be set for each question answered by the user or for each training content. In this case, the points to be given to the user are determined by adding the points for each question or training content answered by the user. Identify. The control unit 41 stores the identified points in the user information DB 42e in association with the user ID. At this time, the control unit 41 calculates the total value of points, and stores the calculated total value in the user information DB 42e.

After that, the control unit 41 proceeds to the process of step S34, and transmits the determination result (type, advice content, and training content) to the information processing apparatus 10 together with the points added to the user in step S81 (S34). The information processing apparatus 10 that has acquired such a determination result displays an advice screen as shown in FIG. 25 on the display unit 16, for example. The advice screen shown in FIG. 25 displays advice for the test performed by the user and points added in the current test. As a result, it is possible to notify the user of the points given for the test and training performed by the user, and it can be expected that the user's motivation for training will continue. Note that what is given to the user is not limited to points, and may be various benefits, coupons, or the like.

In the information processing system of the present embodiment, the server 40 accumulates determination results for each user, so it is determined whether training results (effects) have been obtained based on the time-series determination results for each user. can. In the information processing system of the present embodiment, insurance is sold in which the service usage fee will be returned if the results of the training are not obtained. A process of extracting users who have not obtained results of training on a regular basis is performed. FIG. 26 is a flow chart showing an example of a procedure for extracting users with low training results. The following processing is executed by the control unit 41 according to the control program 42P stored in the storage unit 42 of the server 40. FIG.

In the information processing system of the present embodiment, the control unit 41 of the server 40 performs extraction processing of users with low training results periodically, for example, once a month. Specifically, the control unit 41 identifies users who have insurance based on the information of each user stored in the user information DB 42e (S91). In the user information DB 42e shown in FIG. 23, a user whose information on insurance details is registered in the insurance contract details column is specified. Then, the control unit 41 acquires determination results before and after the predetermined period from the determination results of the specified user (S92). As the determination results before and after the predetermined period, for example, the determination result (cognitive function type) corresponding to the most recent date and time and the determination result (cognitive function type) corresponding to the date and time one month before the most recent date and time are acquired. . Note that the predetermined period is not limited to one month, and may be three months, half a year, one year, or the like.

The control unit 41 identifies the results of the training based on the determination results before and after the predetermined period (S93). For example, the control unit 41 specifies that the result of the training is obtained (high result) when the result of the determination after the predetermined period is better than the result of determination before the predetermined period. In addition, the control unit 41 may specify that the result of the training was not obtained (the result was low) when the result of the determination after the predetermined period was not good compared to the result of determination before the predetermined period. Further, when the determination result is indicated by a level, the control unit 41 may calculate the level difference between the determination result before and after the predetermined period, and use the difference in level as the result of training. In this case, if the level difference is negative, we identify that the training was not successful (low outcome), and if it is positive, the training was successful (high outcome). and specify.

The control unit 41 determines whether or not the result of the specified training is low (S94), and if it is determined to be low (S94), stores the user ID of the user specified in step S91 (S95). For example, a list of users with low achievements is prepared in the storage unit 42, and the control unit 41 registers the user ID of the identified user in this user list. If it is determined that the result is high (S94: NO), the control unit 41 skips the process of step S95 and determines whether or not there is an unprocessed user among the users who have insurance (S96). If it is determined that there is an unprocessed user (S96: YES), the control unit 41 returns to the process of step S91 and performs the processes of steps S91 to S95 for the unprocessed user.

When it is determined that there is no unprocessed user (S96: NO), the control unit 41 outputs the information (user ID) of the user registered in the list of users with low results (S97). For example, the control unit 41 may display a user list (a list of user IDs) on the display unit 45 and notify the operator of the server 40 of the user list. Further, the control unit 41 may transmit a user list (a list of user IDs) to pre-registered terminal devices and notify the operator of the terminal device of the user list. The person in charge (operator) notified of the user list determines whether or not the service usage fee is subject to return based on the status of each user's determination result. In addition to returning the service usage fee, the insurance company is requested to pay insurance money (insurance money corresponding to the service usage fee to be returned to the user). With the above-described processing, it is possible to return the service usage fee to the user according to the training result of each user, so that the user can easily start using the service. Also, the loss caused by returning the service usage fee to the user can be compensated by the insurance company.

In this embodiment, the same effects as those of the above-described embodiments can be obtained. Further, in the present embodiment, since points can be given according to the user's usage status, it is possible to keep the user motivated. In addition, in this embodiment, in cooperation with an insurance company, it is possible to return the service usage fee to a user who cannot obtain an effect (result) from the service provided by the server 40, and the user can use the service. It is possible to lower the barrier when starting The configuration of this embodiment can also be applied to the information processing systems of the second to sixth embodiments, and similar effects can be obtained even when applied to the information processing systems of the second to sixth embodiments. In addition, in the information processing system of this embodiment, it is possible to apply the modified examples appropriately described in each of the above-described embodiments.

The embodiments disclosed this time are illustrative in all respects and should not be considered restrictive. The scope of the present invention is indicated by the scope of the claims rather than the meaning described above, and is intended to include all modifications within the scope and meaning equivalent to the scope of the claims.

10 information processing device 11 control unit 12 storage unit 13 first communication unit 14 second communication unit 15 input unit 16 display unit 17 camera 20 behavior detection unit 21 facial expression detection unit 30 robot 31 control unit 32 storage unit 33 communication unit 36 human feeling Sensor 40 Server 41 Control Unit 42 Storage Unit 43 Communication Unit 12a Test DB
12b History DB
42a type DB
42b Training DB
42c Advice DB
42M learning model

Claims (14)

A robot and a display device communicably connected to the robot,
outputting a program for judging a physical condition to a subject through the display device;
Obtaining response information about the subject's response to the program through the display device or robot;
Based on the obtained answer information, identify the classification of the cognitive function of the subject,
An information processing method including a process of outputting content related to cognitive function through the display device according to the identified classification. Inputting the obtained answer information into a learning model trained to output a classification about cognitive function when the answer information is input, and identifying the classification about the cognitive function of the subject of the answer information. The information processing method according to claim 1, comprising: 1 or 2. The information processing method according to 2. Based on the answer information, classifying into one of a plurality of types related to the cognitive function,
4. The information processing method according to any one of claims 1 to 3, further comprising a process of outputting the content according to the classified type through the display device. The program for determining the physical condition includes cognitive function tests and physical function tests,
5. The information processing method according to any one of claims 1 to 4, further comprising outputting the result of the cognitive function test or the physical function test through the display device. Output the specified classification to the doctor's terminal,
Sending and receiving data related to remote diagnosis to and from the doctor's terminal,
6. The information processing method according to any one of claims 1 to 5, further comprising outputting the data related to the remote diagnosis received from the doctor's terminal through the display device. 7. The information processing method according to any one of claims 1 to 6, including a process of outputting advice according to the specified classification. 8. The information processing method according to any one of claims 1 to 7, further comprising a process in which the robot executes an action corresponding to the identified classification. Outputting the obtained answer information to the server,
9. The information processing method according to any one of claims 1 to 8, comprising acquiring a classification related to the cognitive function specified by the server based on the answer information. estimating the hobbies and preferences of the subject;
outputting to the subject the program in a category corresponding to the subject's hobbies and tastes among a plurality of categories;
10. The information processing method according to any one of claims 1 to 9, further comprising a process of outputting the content of a category according to the subject's hobbies and tastes. 11. Any one of claims 1 to 10, including a process of outputting second content according to the subject's tastes and preferences through the display device during a time period in which the program or the content is not being outputted through the display device. The information processing method described in . from claim 1, including a process of giving points to the subject based on the number of times the program is output to the subject or the number of times the content is output according to the cognitive function classification based on the response information of the subject 11. The information processing method according to any one of 11 to 11. In an information processing system comprising a robot and a display device communicably connected to the robot,
a program output unit that outputs a program for determining a physical condition to a subject through the display device;
an acquisition unit that acquires answer information about the subject's answer to the program through the display device or the robot;
an identifying unit that identifies the classification of the subject's cognitive function based on the acquired answer information;
An information processing system comprising: a content output unit that outputs content related to cognitive function through the display device according to the identified classification. outputting a program for judging the state of the body to the subject through a display device communicably connected to the robot;
Obtaining response information about the subject's response to the program through the display device or robot;
Based on the obtained answer information, identify the classification of the cognitive function of the subject,
A program that causes a computer to execute a process of outputting content related to cognitive function through the display device according to the identified classification.

Images