JP2021015465A - Watching service system - Google Patents

Abstract

To automatize state observation of a user as much as possible, and to reliably deal with a problem.SOLUTION: A user's terminal 14 performs state observation of a user 12, and automatically determines a state thereof to automatically confirm the state with previously prepared question-and-answer. Notification is made to an operator's terminal 32 only when determining that there is a problem, and thus a human load for the state observation is reduced. Determination accuracy can be enhanced by learning a determination database 22 via returning of a confirmation result and so forth of an operator 30 to a watching control unit 24 of the user's terminal 14.SELECTED DRAWING: Figure 1

Description

The present invention relates to a monitoring service system capable of observing a user's condition using a network and ensuring a safe and secure life for the user.

In recent years, there has been an increasing demand for watching services for the elderly living alone. For this purpose, for example, a method of observing the usage status of a hot water supply pot or other household electric appliances that a user uses daily is known through a network. In addition, devices have been developed that can observe the user's condition in more detail by using an AI device such as a robot that interacts with the user or a smart speaker at the user's home (Patent Document 1) (Patent Document 2).
U.S. Patent US100963319B1 Japanese Unexamined Patent Publication No. 2019-103073

However, even if the user's condition can be predicted from the usage status of the equipment, it is often not known until the user is actually met. Relatives who live in remote areas cannot handle this, and the role is delegated to welfare staff. Welfare staff also visit users' homes on a regular basis, so they know the emergency situation from the usage status of the equipment and respond, but there are many false alarms when the equipment is not turned on or is out of order. .. Even if the data acquired by the robot or the like is transmitted to the persons concerned as it is, the information becomes excessive and it is difficult to make an appropriate judgment. The present invention has been made to solve the above problems.

The following configurations are means for solving the above problems, respectively.

<Structure 1>
A watching service system with the following configuration.
(1) An emergency contact number in the event of a user's accident can be sent to the user terminal itself installed in the user's home to be monitored or the storage device of the cloud server connected to the user terminal via the network. Memorize the emergency call destination to rescue the user.
(2) Either a video camera, an image sensor, a proximity sensor, a temperature sensor, or a microphone, which is arranged at a plurality of places in the user's house and connected to the user terminal to acquire the user's behavior pattern. It has a group of interfaces including.
(3) A determination database including a data group showing a correspondence relationship between the user's behavior pattern, the words spoken by the user, and the user's state is stored in the storage device.
(4) For the above user terminal or cloud server, the user's state is referred to the above judgment database by using the user's behavior pattern acquired from the above interface group and the words spoken by the user as keys. Is provided with a monitoring control unit that determines at least one of "normal", "observation required", "dispatch of rescuer", or "notification to close relatives".
(5) The above-mentioned user terminal or cloud server is provided with a speaker, a display, or a message transmitting device for calling out to the user when the judgment result of the user's condition is "observation required". The above-mentioned judgment database includes data indicating the correspondence between the question-and-answer data for calling and the user's state, and the above-mentioned monitoring control unit calls out to the user to execute the question and answer, and the judgment database. Refer to to determine whether the user's status is "observation required", "rescue dispatch" or "notify close relatives".
(6) When the above-mentioned monitoring control unit determines that the judgment result of the user's condition is "observation required", "dispatch of rescuer" or "notification to close relatives", the user terminal sends the operator terminal to the operator terminal. , Transfer this judgment result and its basis data.

<Structure 2>
On the display of the operator terminal, the above-mentioned monitoring control unit displays the judgment result and reason for the user's condition, a list of emergency contacts and emergency call destinations, and a send button that automatically starts the notification process. The monitoring service system according to the configuration 1, characterized in that it is displayed.

<Structure 3>
The above-mentioned monitoring control unit adds data relating the above-mentioned user's behavior pattern, words spoken by the user, automatic judgment result, operator's judgment, and subsequent measures to the above-mentioned judgment database. The monitoring service system according to the configuration 1 or 2, wherein the monitoring service system is described.

<Structure 4>
The monitoring described in Configuration 1 or 2, characterized in that the time when a specific word of the user is spoken when interacting with the user via a voice interface is included in the judgment database as the action time of the user. Service system.

<Structure 5>
When a habitual exchange of words between the user terminal and the user is performed via the voice interface, the monitoring described in the configuration 4 is characterized in that the exchange of these words is included in the judgment database as a life pattern. Service system.

<Structure 6>
When a user makes a device operation request to the user terminal via a voice interface, the control time of the device is recorded, and a set of standard patterns that are continuously performed in a short period of time are displayed. The monitoring service system according to configuration 4, characterized in that it is included in a judgment database as a user's life pattern.

<Structure 7>
A computer program that causes a computer of a user terminal or a cloud server to function as a monitoring control unit and a judgment database described in any one of configurations 1 to 6.

<Structure 8>
A computer-readable recording medium on which the computer program according to configuration 7 is recorded.

The user terminal observes the user's condition, automatically determines the condition, automatically confirms the condition with the questions and answers prepared in advance, and notifies the operator terminal only when it is determined that there is a problem. Therefore, the human load for observing the state is reduced.
Furthermore, since the operator makes the final decision, it is possible to minimize notifications to emergency contacts and emergency call destinations and prevent false alarms.
By returning the result of the post-treatment such as the confirmation result of the operator to the monitoring control unit of the user terminal, the judgment database can be learned and the judgment accuracy can be improved.

It is the schematic of the whole system for carrying out this invention. It is explanatory drawing of the structure of the judgment database and the learning function. It is a screen block diagram which shows an example of the screen displayed on the terminal device of an operator. It is a flowchart of the computer program which operates the system of this invention. It is explanatory drawing of an example of a life pattern.

First, the outline of the system of the present invention will be described with reference to FIG. For example, it is assumed that the user 12 lives alone at home. At this time, the user terminal 14 is installed at the home. It is assumed that the user terminal 14 has a built-in interface group 16 such as a speaker, a display, and a microphone. This can be achieved, for example, by using a speaker that has a well-known AI function and can talk with the user 12.

The user terminal 14 is connected to the network 28. Then, it is assumed that it is also controlled by the cloud server 18 connected to the network 28. The storage device 20 of the cloud server 18 incorporates a watching control unit 24 and a message transmitting device 26 that are operated by a computer program in order to watch over the user 12. Further, in the cloud server 18, a determination database 22 is stored in order to determine the state of the user 12.

Further, an operator terminal 32 is connected to the network 28 in order for the operator 30 to perform a watching service for a large number of users 12. It is the user terminal 14 that observes the state of the user 12, and the cloud server 18 first automatically determines the state of the user 12.

An important result of this determination is transmitted to the operator terminal 32. The operator 30 takes necessary measures for the determination result. For example, the security company 34 is requested to rescue the user 12 through the network 28. In addition, the close relatives 36 are notified of the determination result through the network 28 and requested to take action.

Here, as shown in FIG. 2, the monitoring control unit 24 automatically determines the state of the user 12 with reference to the determination database 22. The determination database 22 records, for example, data for specifically determining the state of the user 12 with respect to the behavior pattern 8 of the user 12 and the words uttered by the user 12.

These data are associated with each other so that the state of the user 12 can be determined as to whether it is a normal state, a state requiring observation, a state in which a rescuer should be dispatched, or a state in which a close relative 36 should be notified. Has been done. Further, when a rescuer is dispatched, the telephone number of the hospital or security company 34 and a message for processing a notification are stored. Further, when it is necessary to notify the close relative 36, a telephone number for notification processing, a notification message, and the like are stored.

The monitoring control unit 24 notifies the operator 30 of the automatic determination result 38 and the determination reason 40 that led to the result. After that, the operator's judgment 48 and the subsequent measures 50 and the like are acquired as data and a site. By reflecting these data in the determination database 22, a highly accurate determination result can be obtained. This system has such a learning function.

As shown in FIG. 3, on the screen of the terminal device operated by the operator 30, for example, the above-mentioned automatic determination result 38 and the determination reason 40 are displayed. Further, the emergency contact number 44, the emergency call destination 42, the telephone number thereof, and the like are displayed. The operator 30 looks at the contents, and if it determines that it is necessary, operates the send button 46 to automatically send the necessary message to the corresponding contact. The measures taken thereafter are input to this operation screen by the operator 30. In this way, the learning data of the determination database 22 is acquired and saved. More specific system functions and operation examples will be described below.

As shown in FIG. For the purpose of observing the user 12, the user terminal 14 installed in the user 12's house has an emergency contact number 44 and an emergency call destination 42 (security company 34, local government) when an accident occurs in the user 12 at the time of installation. , Medical institutions, etc.), etc. are stored. This information is stored in the user terminal 14 main body or the cloud server 18 connected to the user terminal 14 through the network 28.

The monitoring control unit 24 using AI (artificial intelligence) of the user terminal 14 or the cloud server 18 determines the state of the user 12 from the behavior pattern of the user 12 and the words spoken by the user 12. The behavior pattern of the user 12 is detected by a microphone, a video camera, and various sensors. The words uttered by the user 12 are detected and analyzed by the microphone. There may be a plurality of user terminals 14. It is preferable that one of them has a built-in video camera for photographing the user 12, a proximity sensor for detecting the movement of the user 12, and the like.

The judgment database 22 contains data showing the correspondence between the behavior pattern of the user 12 and the words uttered and the state of the user 12, and the data is updated and evolved over time by the machine learning and deep learning functions. A learning function unit is provided. The behavior pattern of the user 12 includes a life pattern such as the daily routine of the user 12 and a routine life pattern in which the same work and word exchange are habitually repeated. In addition, an abnormal behavior or facial expression when the user 21 is not feeling well, which is momentary, is also used as an action pattern.

The determination result of the state of the user 12 is, for example, "normal", "observation required", "dispatch of rescuer", "notification to relatives 36", and the like. This determination result can also be fed back to the database to improve the determination accuracy in consideration of the unique characteristics of the user 12.

In the "normal" or "observation required" state, various calls are made in order to more accurately grasp the state of the user 12. Calls are made using speakers and displays. The mail by the message sending device 26 may be used together. The above-mentioned determination database 22 includes data indicating the correspondence between the question-and-answer data for calling out and the state of the user 12, and this data also evolves with time by the machine learning and deep learning functions. Therefore, it is preferable that this call is performed by the automatic sounding device as frequently as possible.

When it is determined that "observation required", "dispatch of rescuer", or "notify relatives 36", the automatic determination result 38 and the determination reason 40 are transferred from the user terminal 14 to the operator terminal 32 and notified. To. At the same time, information on the emergency contact number 44 and the emergency call destination 42 (security company 34, local government, medical institution, etc.) when an abnormality occurs in the user 12 is transmitted to the operator terminal 32.

At this time, except in the case of a false alarm, the emergency contact 44 and the emergency call destination 42 are always notified. Therefore, the operator 30 either performs an operation for notification processing by himself or the notification processing is performed. It is responsible for checking whether it was executed automatically and reliably.

For example, the system may be configured so that a list of emergency contact numbers 44 and emergency call destinations 42 is displayed on the display of the operator terminal 32, and the operator 30 may click the transmission button 46. When the learning period of the monitoring control unit 24 of the user terminal 14 is short, there are many cases where the determination result is not appropriate. As a result, it is possible to reduce the problem that the persons concerned frequently rush due to unnecessary reports.

That is, the notification process can be optimized by the final judgment of the operator 30. On the other hand, after the monitoring control unit 24 of the user terminal 14 has learned for a sufficient period of time, the state of the user 12 may be determined more accurately than the operator 30. If the determination reason 40 or the like is displayed on the operator terminal 32, the operator 30 can also confirm the credibility of the determination result.

The operator 30 can see the content of the report, for example, directly call out to the user 12, and confirm the state of the user 12 by himself / herself. This is to confirm the presence or absence of false alarms. You may use the phone or email.

The operator 30 makes the final judgment in order to prevent false alarms, but the watching control unit 24 is made to learn by returning the confirmation result of the operator 30 to the watching control unit 24 of the user terminal 14 to improve the judgment accuracy. It also has the effect of being able to.

The user terminal 14 observes the state of the user 12, automatically determines the state, and notifies the operator terminal 32 only when there is a problem and asks for confirmation. Load is reduced. Further, only at this time, the information of the emergency contact number 44 and the emergency call destination 42 (security company 34, local government, medical institution, etc.) when an abnormality occurs in the user 12 is transmitted from the user terminal 14 or the cloud server 18 to the operator terminal 32. If the notification is made, the operator 30 does not need to manage the personal information of all the users 12 to be observed. That is, it is not necessary to store the personal information of all the users 12 on the operator terminal 32 side.

When the operator 30 determines that it is necessary to contact the persons concerned, the operator 30 appropriately selects the emergency contact number 44 and the emergency call destination 42 (security company 34, local government, medical institution, etc.) and contacts by telephone or e-mail. Execute. For example, if a message is automatically sent from the cloud server 18 to the emergency contact 44 or the emergency call destination 42 when the button for notification processing is operated, the contact information of the user 12 is sent to the operator terminal 32. You don't have to notify. In this way, personal information can be protected.

For example, contact the close relatives 36 and the security guard, have them go to the user 12's house, and take necessary relief measures. When the measure 50 or the like taken as a result of the contact is also fed back to the user terminal 14 or the cloud server 18 through the operator terminal 32, the learning effect further enhances the accuracy of determining the state of the user 12. That is, by including in the database data relating the behavior pattern of the user 12, the words uttered by the user 12, the automatic judgment result 38, and the measure 50 taken as a result of contacting the operator's judgment 48. The judgment system can be further enhanced.

The operation flow of the system will be described with reference to the flowchart of FIG.
First, in step S11, the user terminal 14 detects the voice (image) of the user 12. In step S12, at the same time, the words and action patterns of the user 12 are recognized. In step S13, the determination database 22 is referred to.

In step S14, the watching control unit 24 determines the state of the user 12. In step S15, it is determined whether or not the determination result is "normal". If the result of this judgment is yes, the process returns to step S11 and the watching is continued. If no, the process proceeds to step S16.

In step S16, it is determined whether or not the determination result is "observation required". If the result of this determination is yes, the process proceeds to step S17, and if no (state is normal), the process proceeds to step S20. In the case of "observation required", confirm the specific condition by the following voice and determine whether it is normal or not. If it is not "observation required", it is always reported to the operator 30, so the process proceeds to step S20.

In step S17, the question and answer to the user 12 is executed. This process is performed by the monitoring control unit 24 by reading the data stored in the determination database 22. In step S18, the state of the user 12 is redetermined. In step S19, it is determined whether or not the state of the user 12 is "normal". If the result of this judgment is yes, go back to the beginning and continue watching. If no, the process proceeds to step S20.

In step S20, the watching control unit 24 notifies the operator 30 of the determination result. In step S21, the emergency call destination 42 is displayed if the rescuer is dispatched. In step S22, if the close relative 36 is notified, the emergency notification destination is displayed. In step S23, it is detected that the operator 30 clicks the send button 46 after confirming the content. In step S24, the message transmitting device 26 makes an automatic call using the data stored in the determination database 22.

This completes the processing on the operator 30 side. After that, as described above, the processing result is reflected in the determination database 22 for learning. As described above, the behavior pattern of the user 12 in the present invention includes a life pattern like the daily routine of the user 12, a habitual exchange of words and a routine life pattern, and an abnormality of the user 21. Including words and actions and facial expressions. An example will be described below.

(Recording life patterns and learning AI)
FIG. 5 illustrates an example of a life pattern registered in the determination database 22 and subject to determination. A more specific example of the update process of the determination database 22 will be described with reference to this figure. First, before starting to use the service, the user 12 is asked to register each life pattern in the determination database 22 for each day of the week such as weekdays, Saturdays, Sundays, and holidays. For example, register data such as "get up at 7:00 am on weekdays", "go out (go to work) at 8:00 am on weekdays", "go home at 7:00 pm on weekdays", and "go to bed at 11:00 pm on weekdays". The permissible time difference of this time is set to, for example, 1 hour.

For a certain period of time, for example, one month after the start of using the service, the registered life pattern and the actual time of getting up, going out, returning home, going to bed, etc. of the user 12 are individually compared with each other. If any of them deviates from the time considering the above allowable time difference, it is determined to be in the "observation required" state.

After the lapse of one month, the average value is calculated from the history of the actual wake-up, going out, returning home, bedtime, and the like of the user 12, and the determination database 22 is updated. For example, "Wake up at 7 am on weekdays" is modified to "Wake up at 7:15 am on weekdays". Further, for example, when the variation in bedtime is large, the allowable time difference for bedtime is expanded to, for example, one and a half hours. In this way, the accuracy of the determination database 22 is improved by the history.

For example, if you register "Wake up at 7:00 am on weekdays" and register the allowable time difference as 1 hour, wake up before 6:00 am and wake up after 8:00 am "Observation required". It is judged as a state. Further, even if the wake-up time of the user 12 is not recorded even though the wake-up time is registered, it is determined that the state is "observation required".

To record the action time of the user 12 such as getting up or going out, the voice interface of the user terminal 14 is used. For example, when waking up, the user 12 utters "Good morning" to the voice interface. The user terminal 14 uses the voice interface to answer "Good morning. It's fine weather today." With such an automatic answer, it is possible to prompt the user 12 to speak at the time of waking up every morning and obtain the wake-up time. The operation (control) of the device may be automatically detected by using the IoT function and transferred to the user terminal 14. If the tone of the voice of the user 12 with respect to the voice interface can be recognized, this can also be used for grasping the state of the user 12.

FIG. 5 shows an example of a life pattern that is repeated when waking up. The detection time of the user 12, such as a lighting instruction such as "turn on the electricity" and a power-on instruction of the television such as "turn on the TV", is recorded as the wake-up time. If such utterances are habitual, if the life pattern data of the item of habit when waking up is automatically generated and registered, it is possible to accurately detect the abnormal time of the user 12. It is possible to systematically and automatically learn the life patterns of the users 12 by analyzing the weather forecast, news search, and specific utterances multiple times in a day. For example, such various utterance time and behavior data can be analyzed by statistical methods such as regression analysis and Poisson distribution.

By such automatic learning, for example, the time interval from waking up to turning on the light becomes longer than usual, or the weather forecast search is suddenly stopped from a certain time, and so on. It becomes possible to detect. In this case, as shown in the figure, it is recognized that the lighting and the television are turned on after the wake-up recognition, and it is further determined whether or not this series of actions is executed within, for example, 10 minutes. It differs from the observation of daily life patterns in that it is not an individual judgment. That is, it is advisable to observe both daily life patterns and routine life patterns.

In addition to this, when going out, detection of utterances such as "I'm going" to the voice interface, instructions to turn off the lights such as "turn off the lights in the room", and instructions to turn off the TV such as "turn off the TV". It is advisable to record the time as the time of going out. When returning home, it is advisable to record the utterance of "I'm home" to the voice interface, the instruction to turn on the lights in the room, the instruction to turn on the TV, etc. as the time to return home. At bedtime, it is advisable to record the utterance of "good night" to the voice interface, the instruction to turn off the lights in the bedroom, the instruction to turn off the TV, etc. as the bedtime. The utterance time for requesting the operation of the device and the on / off time of the controlled device may be recorded as the action time.

(Correspondence to observation required)
When the user terminal 14 detects the "observation required" state, for example, the following processing is automatically executed. First, in the first stage, a dialogue with the user 12 is executed fully automatically. For example, if there is no voice from the user 12 when waking up after 8 o'clock, a question such as "I have no voice today, how are you feeling?" Is asked. When there is a response from the user 12 such as "I feel sick", the operator 30 is immediately notified of the response result. The same applies when observing abnormal behavior or facial expressions. It is also possible to parse the response from the user 12 and, if it is determined that the user 12 has no abnormality, only record the result.

In addition, the voice interface cannot handle the case where the user 12 is absent. For example, there is a case where the user 12 returns home late. At this time, a text message such as an e-mail with the content "When will you be back?" Is automatically sent to the mobile phone of the user 12. If there is a reply to this e-mail, use it. The response from person 12 may be parsed and the above processing may be performed.

When the call center operator 30 is notified, the second stage is entered, and the operator 30 confirms the safety of the user 12 by telephone or the like. When the safety is confirmed, the state of "observation required" is canceled and the result is recorded. On the other hand, when it is determined that further confirmation of the state of the user 12 is necessary, the security company 34 is notified. If the close relative 36 can rush to the user 12 apologies, only the close relative 36 may be notified. Since the coping method for each user 12 is displayed on the operator terminal 32 device, the instruction may be followed.

In this third stage, the operator 30 automatically notifies the relatives 36, the security company 34, the local government, the medical institution, etc. designated in advance by the user 12 by operating one button. You can also. As the means of notification, communication means such as e-mail, telephone, and messaging application may be set according to the notification destination. If the notification destination notifies the operator 30 that the response to the user 12 has been successfully completed, the operator 30 may record the response result and cancel the "observation required" of the user 12. Display the release button on the user terminal 14. It may be possible to cancel "observation required" with one touch.

When a user terminal 14 having a function of controlling various devices by voice as shown in Patent Documents 1 and 2 is used, not only the words of the user 12 but also the time when the user 12 controls the device can be obtained. Can be recorded accurately. As a result, the life pattern of the user 12 can be recorded more accurately.

For example, a series of work such as a series of work when waking up in the morning and a series of work before and after a meal have a flow determined by the habits of each user 12. When this flow is disturbed, it can be determined that the user 12 has some mental or physical abnormality.

In other words, there is a macro life pattern that takes place throughout the day, such as weather, going out, going home, and going to bed, and a group of micro-typical patterns that take place in a short period of time triggered by some action. Both are recorded in the determination database 22 as life patterns. By comparing and verifying the degree of these daily changes, it becomes possible to automatically observe the user 12 in more detail.

Moreover, when the life pattern is different from usual, instead of immediately notifying the concerned parties, by automatically executing a prepared dialogue with the user 12 and confirming the reason. , Excessive reporting can be suppressed. If the user 12 is informed that the life pattern is different from usual through a dialogue prepared in advance, the user 12 himself may notice the change in his / her physical condition. This is also an important effect.

Twelve users do not notice by having them use a simple device that automatically measures body temperature, blood pressure, heart rate, etc., rather than simply having a dialogue, and comparing the results with normal values. Abnormal physical condition can also be detected.

For example, when interacting with the user 12 through the interface connected to the user terminal 14, the state of the user 12 is reached by touching the device for measuring body temperature, heart rate, etc. connected to the user terminal 14. Can be observed accurately at the same time.

In the above example, the behavior of the user 12 is acquired as a life pattern. For example, if various tools and furniture arranged in the living room of the user 12 are always in a certain place, this is assumed. Should be recorded as a reference image and always compared with the current state. It is considered that this is useful for detecting the abnormality of the user 12 by detecting, for example, a case where the state of the living room is significantly changed.

12 User 14 User terminal 16 Interface group 18 Cloud server 20 Storage device 22 Judgment database 24 Watching control unit 26 Message transmission device 28 Network 30 Operator 32 Operator terminal 34 Security company 36 Close relatives 38 Automatic judgment result 40 Judgment reason 42 Emergency Report destination 44 Emergency contact information 46 Send button 48 Operator judgment 50 Measures taken as a result of contact

Claims (8)

A watching service system with the following configuration.
(1) An emergency contact number in the event of a user's accident can be sent to the user terminal itself installed in the user's home to be monitored or the storage device of the cloud server connected to the user terminal via the network. Memorize the emergency call destination to rescue the user.
(2) Either a video camera, an image sensor, a proximity sensor, a temperature sensor, or a microphone, which is arranged at a plurality of places in the user's house and connected to the user terminal to acquire the user's behavior pattern. It has a group of interfaces including.
(3) A determination database including a data group showing a correspondence relationship between the user's behavior pattern, the words spoken by the user, and the user's state is stored in the storage device.
(4) For the above user terminal or cloud server, the user's state is referred to the above judgment database by using the user's behavior pattern acquired from the above interface group and the words spoken by the user as keys. Is provided with a monitoring control unit that determines at least one of "normal", "observation required", "dispatch of rescuer", or "notification to close relatives".
(5) The above-mentioned user terminal or cloud server is provided with a speaker, a display, or a message transmitting device for calling out to the user when the judgment result of the user's condition is "observation required". The above-mentioned judgment database includes data indicating the correspondence between the question-and-answer data for calling and the user's state, and the above-mentioned monitoring control unit calls out to the user to execute the question and answer, and the judgment database. Refer to to determine whether the user's status is "observation required", "rescue dispatch" or "notify close relatives".
(6) When the above-mentioned monitoring control unit determines that the judgment result of the user's condition is "observation required", "dispatch of rescuer" or "notification to close relatives", the user terminal sends the operator terminal to the operator terminal. , Transfer this judgment result and its basis data. The above-mentioned monitoring control unit automatically notifies the emergency contact and the emergency call destination on the display of the operator terminal, the judgment result and the reason for the judgment of the user status, the list of the emergency contact and the emergency call destination. The monitoring service system according to claim 1, wherein a send button is displayed. The above-mentioned monitoring control unit puts data relating the above-mentioned user's behavior pattern, words spoken by the user, automatic judgment result, operator's judgment, and subsequent measures into the above-mentioned judgment database. The monitoring service system according to claim 1 or 2, wherein the monitoring service system is added. The invention according to claim 1 or 2, wherein the time when a specific word of the user is spoken when interacting with the user via a voice interface is included in the judgment database as the action time of the user. Watching service system. The fourth aspect of claim 4, wherein when a habitual exchange of words between a user terminal and a user is performed via a voice interface, the exchange of these words is included in a judgment database as an action pattern. Watching service system. When a user makes a device operation request to the user terminal via a voice interface, the control time of the device is recorded, and a set of standard patterns that are continuously performed in a short period of time are displayed. The monitoring service system according to claim 4, wherein the user's life pattern is included in the judgment database. A computer program that causes a computer of a user terminal or a cloud server to function as a monitoring control unit and a judgment database according to any one of claims 1 to 6. A computer-readable recording medium on which the computer program according to claim 7 is recorded.

Images