WO2023248937A1

WO2023248937A1 - Terminal, notification system, notification method, and program

Info

Publication number: WO2023248937A1
Application number: PCT/JP2023/022353
Authority: WO
Inventors: 琢也神田; 公海 ▲高▼橋; 麻美宮島; 康雄石榑
Original assignee: 日本電信電話株式会社
Priority date: 2022-06-22
Filing date: 2023-06-16
Publication date: 2023-12-28
Also published as: WO2023248392A1

Abstract

This terminal comprises a detection information acquisition unit that acquires information about a detected motion of a user, and a vibration control unit that generates vibrations for notification to the user such that vibrations are generated during the action of the user on the basis of the information about the detected motion of the user.

Description

Terminal, notification system, notification method, and program

The present invention relates to a terminal, a notification system, a notification method, and a program.

Techniques exist for systems to encourage action by sending notifications to users. Many of these notification methods that prompt users to take action may distract the user's attention and make the user feel bothered by the notification. Techniques have been developed to reduce the annoyance that users feel when a system sends notifications and engages users.

For example, in Patent Document 1, in order to reduce the annoyance that the user feels when the system sends notifications to the user or other actions, a threshold value or a mental burden function based on the mental burden of presenting information accompanying the action is proposed. A technique has been disclosed that determines the content and presentation timing of information to be presented to a user when a condition indicated by a threshold value is satisfied.

JP 2019-185389 Publication

In the above-mentioned conventional technology, there are not many breaks in actions that can be intervened in, timings with a high possibility of interruption, etc. in daily life, so the frequency of notifications is low compared to the frequency that should be notified. Therefore, there is a problem that it is not possible to notify the user at an appropriate timing.

The disclosed technology aims to notify the user at an appropriate timing according to the user's actions.

The disclosed technology includes a detection information acquisition unit that acquires information on detecting a user's motion, and a sensor configured to vibrate while the user is performing a motion based on the information on the detected motion of the user. This terminal is equipped with a vibration control unit that generates vibrations to notify the user.

It is possible to notify the user at an appropriate timing according to the user's actions.

1 is a diagram showing an example of a hardware configuration of a wearable terminal according to an embodiment of the present invention. 1 is a diagram illustrating an example of a functional configuration of a wearable terminal according to Example 1 of the embodiment of the present invention. It is a flowchart which shows an example of the flow of the notification process based on Example 1 of embodiment of this invention. FIG. 3 is a diagram for explaining an operation start time and an operation end time according to Example 1 of the embodiment of the present invention. FIG. 2 is a diagram for explaining an acceleration prediction method according to Example 1 of the embodiment of the present invention. FIG. 3 is a diagram showing an example of a functional configuration of a wearable terminal according to Example 2 of the embodiment of the present invention. 12 is a flowchart illustrating an example of the flow of notification processing according to Example 2 of the embodiment of the present invention. FIG. 7 is a diagram for explaining a vibration adjustment method according to Example 2 of the embodiment of the present invention. It is a figure which shows an example of the functional structure of the wearable terminal based on Example 3 of embodiment of this invention. 12 is a flowchart illustrating an example of the flow of notification processing according to Example 3 of the embodiment of the present invention. It is a figure which shows an example of the hardware structure of the electronic terminal based on Example 4 of embodiment of this invention. It is a figure which shows an example of the functional structure of the electronic terminal and wearable terminal based on Example 4 of embodiment of this invention. 12 is a flowchart illustrating an example of the flow of learning processing according to Example 4 of the embodiment of the present invention. FIG. 7 is a diagram for explaining an example of learning processing according to Example 4 of the embodiment of the present invention. It is a figure which shows an example of the functional structure of the learning device based on Example 4 of embodiment of this invention. 12 is a flowchart illustrating an example of the flow of inference processing according to Example 4 of the embodiment of the present invention.

Hereinafter, an embodiment of the present invention (this embodiment) will be described with reference to the drawings. The embodiments described below are merely examples, and embodiments to which the present invention is applied are not limited to the following embodiments.

(Problems with conventional technology)
First, problems with the conventional technology will be explained. There are many technologies that encourage behavior by sending notifications from the system side. Many of the notification methods that encourage these actions steal the user's attention and make the user feel bothered. Many technologies have been developed to reduce the hassle of sending notifications from the system side and encouraging users. For example, Patent Document 1 discloses a technique for reducing the annoyance felt by a user when a system sends a notification to the user.

There is a method that uses a wearable terminal to send reminder notifications by vibrating the wearable terminal. While these types of vibrations make it easy to notice reminder notifications, they also take away a lot of people's attention. There are methods to reduce the annoyance of notifications, such as a method of detecting breaks in behavior and a method of estimating interruptibility and issuing notifications. However, the problem is that there are not many breaks in actions that can be intervened in, or timings where there is a high possibility of interruption, in daily life, and the frequency of notifications is small compared to the frequency of notifications.

Additionally, there are many methods in which the modality of notification itself is devised. For example, there are methods that use changes in heat to notify the user of the presence or absence of a notification, and methods that notify the user by lighting an LED in an ambient manner. However, the notification actuators used in these methods are not installed in many smartwatches currently on the market, and it is extremely costly to install new actuators for the sole purpose of notifications. Conceivable.

(Summary of this embodiment)
The timing at which the user does not feel bothered by notification vibrations is when the part of the body that feels the vibrations is moving. Therefore, in order to address the problems of the conventional technology described above, in this embodiment, we assume that notifications occur more frequently in daily life, and the parts of the user who is wearing a terminal that vibrates when a notification moves significantly. An example of vibration for notification at the appropriate timing will be explained. Notifications are made by vibrating during periods when the part of the body the device is attached to is moving significantly, and the vibrations end during periods when the body parts are moving significantly, thereby reducing the annoyance that users feel due to vibrations during notifications. can be reduced.

Hereinafter, Examples 1 to 4 will be described as specific examples of this embodiment.

(Example 1)
In this embodiment, an example of a terminal that vibrates as a notification while the user is moving his or her body will be described.

FIG. 1 is a diagram showing an example of the hardware configuration of a wearable terminal according to an embodiment of the present invention. The wearable terminal 10 may be a device that is in constant contact with the user's body, such as a smart watch. They don't have to be in contact. The wearable terminal 10 is an example of a terminal that has a function of notifying a user by generating vibrations.

The wearable terminal 10 includes a CPU 101, a memory 102, a communication device 103, a sensor 104, and a vibration device 105, which are interconnected via a bus.

The memory 102 stores programs and the like that describe the processing contents described in this embodiment. The CPU 101 executes processing written in a program stored in the memory 102.

The communication device 103 is a device for transmitting and receiving signals with other devices.

The sensor 104 is a device that performs various measurements to detect the movements of the user wearing the wearable terminal 10. The sensor 104 may include, for example, an acceleration sensor that measures acceleration, a touch sensor that detects contact, a camera sensor that captures an image, and the like.

The vibration device 105 is a device that generates vibrations in response to instructions from the CPU 101. Vibration device 105 may be composed of one or more vibrators.

Note that the wearable terminal 10 may further include a display device for displaying messages. In that case, in the wearable terminal 10, the vibration device 105 may generate vibration and the display device may display a message when notifying the wearable terminal 10.

FIG. 2 is a diagram illustrating an example of the functional configuration of a wearable terminal according to Example 1 of the embodiment of the present invention. The wearable terminal 10 includes a storage section 11 , a detection information acquisition section 12 , a motion start confirmation section 13 , an acceleration prediction section 14 , a motion end estimation section 15 , and a vibration control section 16 . Each of these parts is realized by a CPU 101 and a memory 102 that constitute a computer. Specifically, each part is realized by the CPU 101 executing processing described in a program stored in the memory 102.

The storage unit 11 stores information indicating the message content of the notification, information indicating the timing of notification, information indicating the vibration duration d at the time of notification, etc.

The detection information acquisition unit 12 acquires detection information indicating the result of detecting the user's motion by the sensor 104. For example, the detection information acquisition unit 12 acquires information indicating acceleration. Note that the wearable terminal 10 may be premised on moving in space in unison with the body part of the user wearing the wearable terminal 10. That is, the acceleration of the wearable terminal 10 is considered to indicate the movement of the body part of the user wearing the wearable terminal 10.

The motion start confirmation unit 13 determines whether the user's motion has started based on the detection information acquired by the detection information acquisition unit 12. Then, the operation start confirmation unit 13 estimates the operation start time T _ms . Note that the motion start confirmation unit 13 may determine whether or not the user's motion has started based on detection information obtained by any type of sensor, as long as the information is related to the user's motion. good.

The acceleration prediction unit 14 predicts how the acceleration will change from the operation start time T _ms based on the detection information acquired by the detection information acquisition unit 12. For example, the acceleration prediction unit 14 may predict acceleration based on acceleration included in detection information acquired by an acceleration sensor. Note that the acceleration prediction unit 14 may predict acceleration based on detection information obtained by any type of sensor as long as the information is related to prediction of acceleration.

The motion end estimating unit 15 estimates the time at which the user's motion ends (motion end time T _me ) based on the acceleration prediction result. The operation end estimating unit 15 then estimates the vibration based on the time to start the vibration (vibration start time T _ns ), the time to continue the vibration (vibration duration d), the time to end the vibration (vibration end time T _ne ), etc. , it is determined whether the vibration due to the notification ends before the operation end time _Tme .

The vibration control unit 16 controls the vibration device 105 to generate vibrations based on the vibration start time T _ns , the vibration duration time d, the vibration end time T _ne , the determination result by the operation end estimation unit 15, and the like.

FIG. 3 is a flowchart illustrating an example of the flow of notification processing according to Example 1 of the embodiment of the present invention. The notification process is started when a reason for notifying the user occurs.

The operation start confirmation unit 13 monitors the operation (step S101). Specifically, the motion start confirmation unit 13 determines whether the user's motion has started based on the detection information acquired by the detection information acquisition unit 12. Then, the operation start confirmation unit 13 estimates the operation start time T _ms .

Next, the acceleration prediction unit 14 predicts the acceleration of the wearing part (the part where the user wears the wearable terminal 10) (step S102). Specifically, the acceleration prediction unit 14 predicts how the acceleration will change from the operation start time T _ms based on the detection information acquired by the detection information acquisition unit 12.

Subsequently, the motion end estimating unit 15 estimates the end time of the motion (step S103). Specifically, the motion end estimating unit 15 estimates the motion end time _Tme based on the acceleration prediction result.

Then, the operation end estimating unit 15 determines whether the duration of the notification (vibration) is before the end time of the operation (step S104). Specifically, the operation end estimating unit 15 determines whether the vibration due to the notification will end before the operation end time _Tme based on the vibration start time _Tns , vibration duration d, vibration end time _Tne, etc. Determine whether

When the operation end estimating unit 15 determines that the duration of the notification (vibration) is not before the end time of the operation (step S104: NO), it ends the notification process. On the other hand, if the operation end estimating unit 15 determines that the duration of the notification (vibration) is before the end time of the operation (step S104: YES), the vibration control unit 16 generates vibration (step S105).

FIG. 4 is a diagram for explaining the operation start time and operation end time according to Example 1 of the embodiment of the present invention. For example, when the user lifts his arm while wearing the wearable terminal 10 on his arm, the movement start time T _ms is the time at which he starts the movement with his arm lowered. The motion end time _Tme is the time when the motion of raising the arm ends and the arm becomes in a raised state.

FIG. 5 is a diagram for explaining an acceleration prediction method according to Example 1 of the embodiment of the present invention. FIG. 5 shows a method for predicting acceleration in the user's motion shown in FIG. The vertical axis in FIG. 5 indicates acceleration, and the horizontal axis indicates time. FIG. 5 shows an operation start time T _ms , an operation end time T _me , a vibration start time T _ns , a vibration end time T _ne , and a vibration duration time d.

Further, the storage unit 11 may store in advance information indicating the vibration duration d for notification and a threshold value A _th for determination regarding acceleration. In step S101 of the notification process, the operation start confirmation unit 13 monitors the operation and acquires information indicating the acceleration of the measurement result. A graph 901 is an example of acceleration as a measurement result.

Then, the operation start confirmation unit 13 detects the start of the operation using the threshold value A _th based on the information indicating the acceleration of the measurement result. For example, the operation start confirmation unit 13 may determine that the operation has started when the acceleration exceeds the threshold value A _th . The operation start confirmation unit 13 sets the time when the start of the operation is detected as the operation start time T _ms .

Next, in step S102 of the notification process, the acceleration prediction unit 14 predicts how the acceleration will change after the start of the motion is detected. For example, as shown in FIG. 5, the acceleration prediction unit 14 calculates that the acceleration changes from the point at which the acceleration reaches the maximum by the same amount of decrease as the amount of increase over time until the acceleration reaches the maximum. The acceleration may be predicted based on the model that A graph 902 is an example of acceleration as a prediction result. Hereinafter, the acceleration of the prediction result will be referred to as A(t).

Subsequently, in step S103 of the notification process, the motion end estimation unit 15 estimates the motion end time _Tme based on the acceleration prediction result. For example, the motion end estimation unit 15 may estimate the time when the predicted acceleration A(t) becomes 0 as the motion end time T _me . Note that the motion end estimating unit 15 may estimate the time when the predicted acceleration A(t) becomes equal to or less than the threshold value A _th as the motion end time T _me .

The movement end time _Tme can be estimated by any other method as long as it is possible to estimate the movement end time _Tme , such as by using a musculoskeletal model formula, by machine learning, or by using a statistical model. This method may also be used.

In step S104 of the notification process, the motion end estimating unit 15 checks whether the vibration end time T _ne , which is the vibration start time T _ns plus d, is earlier than the estimated motion end time T _me . Then, if the vibration end time T _ne is earlier than the estimated operation end time T _me , the vibration control unit 16 generates vibration in step S105 of the notification process.

Note that the method of this embodiment may be implemented in combination with the conventional techniques described above, such as a method of detecting a break in behavior, a method of estimating a timing with high interruptibility, and issuing a notification. good.

According to this embodiment, vibration for notification can be generated while the user is moving the part to which the wearable terminal 10 is attached. This can reduce the annoyance felt by the user.

(Example 2)
Example 2 will be described below with reference to the drawings. The second embodiment differs from the first embodiment in that the intensity of vibration is adjusted according to the user's movement. Therefore, in the following explanation of the second embodiment, the differences from the first embodiment will be mainly explained, and parts having the same functional configuration as the first embodiment will be designated by the same reference numerals as used in the explanation of the first embodiment. A symbol is given and the explanation thereof is omitted.

FIG. 6 is a diagram illustrating an example of a functional configuration of a wearable terminal according to Example 2 of the embodiment of the present invention. The wearable terminal 10 according to the present embodiment has a configuration in which a vibration adjustment section 17 is added to the wearable terminal 10 according to the first embodiment.

The vibration adjustment unit 17 adjusts the intensity of the vibration to be generated based on the prediction result of the acceleration estimated by the acceleration prediction unit 14.

FIG. 7 is a flowchart illustrating an example of the flow of notification processing according to Example 2 of the embodiment of the present invention. The flow of the notification process according to the present embodiment is the same as the notification process according to the first embodiment with step S201 added.

In step S104, if the operation end estimating unit 15 determines that the duration of the notification (vibration) is before the end time of the operation (step S104: YES), the vibration adjustment unit 17 adjusts the intensity of the vibration (step S201). Then, in step S105, the vibration control unit 16 generates vibration with the adjusted intensity.

FIG. 8 is a diagram for explaining a vibration adjustment method according to Example 2 of the embodiment of the present invention. FIG. 8 shows a method for adjusting vibrations in the user's actions shown in FIG.

In step S201 of the notification process according to the second embodiment, the vibration adjustment unit 17 adjusts the vibration as shown in equation (1) below in response to the time-series acceleration A(t) predicted by the acceleration prediction unit 14. Adjust the strength V _s (t).

V _s (t)=αA(t)...Formula (1)

Note that α is a constant. Graph 903 shows the intensity of vibration adjusted in this way.

Further, the vibration adjustment unit 17 may divide the acceleration predicted by comparison with one or more reference threshold values into groups, and adjust the intensity to be different for each group.

Sensitivity to vibrations is thought to change depending on the movement of the worn part of a smartwatch, etc. Therefore, conventional smart watches etc. send notifications with a constant vibration regardless of the user's actions, and do not respond to changes in the sensitivity of the user receiving notifications to vibrations, so notification vibrations can be annoying. Conceivable.

According to the wearable terminal 10 according to the present embodiment, it is possible to obtain information on the acceleration of the part of the user to which the wearable terminal 10 is attached using the installed sensor, and to generate vibrations with an intensity corresponding to the magnitude of the acceleration. . This can further reduce the annoyance felt by the user.

(Example 3)
Example 3 will be described below with reference to the drawings. The third embodiment differs from the first embodiment in that information indicating a notification schedule (schedule information) is acquired and vibration is generated based on the schedule. Therefore, in the following explanation of the third embodiment, the differences from the first embodiment will be mainly explained, and the same reference numerals as used in the explanation of the first embodiment will be used for parts having the same functional configuration as the first embodiment. A symbol is given and the explanation thereof is omitted.

FIG. 9 is a diagram illustrating an example of the functional configuration of a wearable terminal according to Example 3 of the embodiment of the present invention. The notification system according to this embodiment includes a wearable terminal 10 (first terminal) and an electronic terminal 20 (second terminal).

The electronic terminal 20 is, for example, a mobile terminal, a server device, etc., and stores schedule information. Note that the electronic terminal 20 may be any other device as long as it can store schedule information.

The electronic terminal 20 includes a storage section 21 and a communication section 22. The storage unit 21 stores schedule information. The schedule information may be information indicating, for example, the date and time of notification, day of the week, frequency, number of times, etc.

The communication unit 22 acquires schedule information from the storage unit 21 and transmits it to the wearable terminal 10.

The wearable terminal 10 according to the present embodiment has a configuration in which a communication unit 18 is added to the wearable terminal 10 according to the first embodiment. The communication unit 18 receives schedule information from the electronic terminal 20 and stores it in the storage unit 11 .

Further, the vibration control unit 16 according to the present embodiment generates notification vibration at an appropriate timing using the method of the first embodiment, as long as it is within the schedule time, based on the schedule information stored in the storage unit 11.

FIG. 10 is a flowchart illustrating an example of the flow of notification processing according to Example 3 of the embodiment of the present invention. The flow of the notification process according to the present embodiment is a flow in which step S301 and step S302 are added to the notification process according to the first embodiment.

When the notification process is started, the communication unit 18 receives notification schedule information from the electronic terminal 20 (step S301). Schedule information is stored in the storage unit 11.

Next, the wearable terminal 10 determines whether it is within the scheduled time (step S302). When the wearable terminal 10 determines that it is within the scheduled time (step S302: YES), the process proceeds to step S101.

On the other hand, if the wearable terminal 10 determines that it is not within the scheduled time (step S302: NO), it ends the notification process.

Incidentally, if the scheduled time has passed because the notification could not be made within the scheduled time, the vibration control unit 16 may immediately generate a notification vibration.

According to this embodiment, it is possible to set the vibration timing of the wearable at an appropriate timing when notifying a message such as a reminder that has a predetermined notification time period.

(Example 4)
Example 4 will be described below with reference to the drawings. Healthcare apps have become commonly used in recent years. A terminal equipped with a healthcare application can, for example, collect data related to a user's health (such as data when performing a certain action) and manage the user's health. In the fourth embodiment, it is assumed that a health care application is used on the wearable terminal 10 or the electronic terminal 20.

In the fourth embodiment, as in the third embodiment, a wearable terminal 10 (for example, a smart watch) and an electronic terminal 20 (for example, a smartphone) are used.

In the fourth embodiment, the wearable terminal 10 is connected to the electronic terminal 20 or the wearable terminal in order to have the user perform actions such as running and answering a questionnaire (hereinafter referred to as recommended actions) necessary for the operation of the healthcare application. The feasibility of implementing each recommended action is estimated based on user state information (eg, action recognition information, time, GPS, etc.) that can be obtained by 10.

Then, the wearable terminal 10 selects an appropriate recommended action based on the feasibility of implementing each recommended action, and uses the technology of the third embodiment to provide a reminder at an appropriate timing and content.

In Example 4, a neural network (NN) model (feasibility estimation model) is used to estimate the feasibility of each recommended action. Here, a softmax function is used as the activation function of the output layer of the model, and the feasibility (also called probability) of each recommended action is output as an output of the softmax function.

In the following explanation of the fourth embodiment, the differences from the first and third embodiments will be mainly explained, and those having the same functional configuration as the first and third embodiments will be explained. The same reference numerals as those used in the explanation of 3 will be given, and the explanation thereof will be omitted.

<About hardware configuration>
The hardware configuration of the wearable terminal 10 is as described in the first embodiment. The electronic terminal 20 may be a smartphone, a tablet, a PC (personal computer), or other devices. FIG. 11 shows an example of the hardware configuration of the electronic terminal 20 in the fourth embodiment. Although FIG. 11 is a diagram assuming a computer such as a PC, a smartphone or a tablet has the same basic configuration as shown in FIG. 11 and has the configuration of a computer. Note that the learning device 100, which will be described later, also has a computer configuration shown in FIG. 11 as a hardware configuration.

The electronic terminal 20 can be realized by using hardware resources such as a CPU and memory built into a computer to execute a program corresponding to the processing performed by the electronic terminal 20. The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. It is also possible to provide the above program through a network such as the Internet or e-mail.

The computer shown in FIG. 11 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and a sensor 1009, which are interconnected by a bus BS. etc.

A program that realizes processing on the computer is provided, for example, by a recording medium 1001 such as a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily need to be installed from the recording medium 1001, and may be downloaded from another computer via a network. The auxiliary storage device 1002 stores installed programs as well as necessary files, data, and the like.

The memory device 1003 reads and stores the program from the auxiliary storage device 1002 when there is an instruction to start the program. The CPU 1004 implements functions related to the electronic terminal 20 according to programs stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network or the like. A display device 1006 displays a GUI (Graphical User Interface) and the like based on a program. The input device 1007 is composed of a keyboard, a mouse, buttons, a touch panel, or the like, and is used to input various operation instructions. An output device 1008 outputs the calculation result.

The sensor 1009 includes, for example, one or more of an acceleration sensor that measures acceleration, a touch sensor that detects contact, a camera sensor that captures an image, a position sensor (GPS device), or all of the following. But that's fine.

<Functional configuration example>
FIG. 12 is a diagram showing an example of the functional configuration of the wearable terminal 10 and the electronic terminal 20 in the fourth embodiment. In the following description, it is assumed that the electronic terminal 20 is a terminal such as a smartphone that is carried and used by a user.

The electronic terminal 20 according to the fourth embodiment has a configuration in which a detection information acquisition section 23 and a recommended action implementation recognition section 24 are added to the electronic terminal 20 according to the third embodiment.

The detection information acquisition unit 23 acquires detection information indicating the result of detecting the user's motion by the sensor 1008. Further, the detection information acquisition unit 23 can also acquire the user's position information acquired by the sensor 1008 (for example, a GPS device) as the detection information, or can acquire information such as the user's posture acquired by the sensor 1008 (for example, a camera). It can be acquired as detection information, or other information can be acquired.

The recommended behavior implementation recognition unit 24 checks whether the user has implemented the recommended behavior based on the information acquired by the detection information acquisition unit 23.

The wearable terminal 10 according to the fourth embodiment has a configuration in which a recommended action implementation recognition section 31, a user state estimation section 32, an implementation possibility estimation section 33, and a recommended action selection section 34 are added to the wearable terminal 10 according to the third embodiment. . Note that the recommended behavior selection section 34 may also be referred to as the behavior selection section 34. Further, the recommended behavior implementation recognition unit 31 may be referred to as the behavior implementation recognition unit 31.

The recommended behavior implementation recognition unit 31 checks whether the user has implemented the recommended behavior based on the information acquired by the detection information acquisition unit 12.

It is assumed that several actions are determined in advance as recommended actions in the healthcare application assumed in this example. In addition, many of the recommended actions in the healthcare app are based on the presence or absence (for example, walking) of the wearable terminal 10 or the electronic terminal 20 using detection information (information obtained by the sensor) using the action recognition API. You can check the presence or absence of running, running, etc.). Furthermore, it is also possible to check whether or not the user has responded to a questionnaire, confirmed a notification, taken medication, etc., based on operation logs or responses in a healthcare application used on the wearable terminal 10 or the electronic terminal 20.

Both the recommended action implementation recognition unit 24 and the recommended action implementation recognition unit 31 can detect implementation of the recommended action by the user, for example, by using the above-mentioned action recognition API.

The user state estimation unit 32 estimates the user state based on the information acquired by the detection information acquisition unit 12. The user state estimation unit 32 may use the information acquired by the detection information acquisition unit 12 as the user state information as it is.

The feasibility estimating unit 33 includes a storage unit that stores weight data of the NN that estimates the feasibility of each recommended action. The feasibility estimation unit 33 can operate the NN in which the weight data is set by reading the weight data (weight parameters) from the storage unit.

The user state obtained by the user state estimating unit 32 is input to the NN as input data, and the NN outputs the feasibility of each recommended action.

As mentioned above, in the output layer of the NN, a softmax function, which is often used in multi-level classification, is used. The softmax function outputs the feasibility of each category of recommended actions in the range of 0 to 1, and outputs the feasibility of all categories so that the sum of the possibilities of action becomes 1. Examples of outputs are: Survey: 0.4, Running: 0.3, Medication: 0.3. The above NN will be referred to as a feasibility estimation model.

In the fourth embodiment, there is a learning phase in which a feasibility estimation model is learned, and an inference phase in which inference is made using the learned feasibility estimation model. Each operation will be described later.

The recommended action selection unit 34 selects one recommended action from among the plurality of recommended actions that have implementation possibilities and are output from the implementation possibility estimation unit 33 (implementability estimation model). The specific operation will be described later.

<Learning phase>
Next, an example of the operation of the system (electronic terminal 20 and wearable terminal 10) in the learning phase will be described. Here, the implementation feasibility estimation unit 33 will be described as including a function of learning the implementation feasibility estimation model (creating weight data). The feasibility estimating unit 33 during learning of the feasibility estimation model may be referred to as a “learning unit”.

Note that the learning of the feasibility estimation model (creation of weight data) may be performed by a device (referred to as a learning device) separate from the electronic terminal 20 and the wearable terminal 10.

In the learning phase, "the information obtained by the user state estimation section 32 and the information obtained by the recommended action implementation recognition section 31" in the wearable terminal 10 and the information obtained by the recommended action implementation recognition section 24 in the electronic terminal 20 are used. Using the obtained information, the feasibility estimating unit 33 (learning unit) of the wearable terminal 10 performs learning of the feasibility estimation model. In other words, the implementability estimation unit 33 (learning unit) determines that the implementability estimation model into which the information obtained by the user state estimation unit 32 is input is the correct recommended action (recommended action implementation recognition unit 31/recommended action implementation recognition unit). The weight data (parameters) of the feasibility estimation model are adjusted so as to output the behavior obtained by the unit 24 (that is, so that the probability of implementing the correct recommended behavior is 1).

Note that learning may be performed using information obtained by either the information obtained by the recommended action implementation recognition unit 31 or the information obtained by the recommended action implementation recognition unit 24.

The operation of the electronic terminal 20 and the operation of the wearable terminal 10 in the learning phase will be described with reference to the flowchart in FIG. 13. Hereinafter, S401 to S403 will be explained before S501 to S503, but this is for convenience of explanation. S401 to S403 and S501 to S503 are basically performed in parallel.

In S401, the recommended action implementation recognition unit 24 acquires detection information from the detection information acquisition unit 23. In S402, the recommended behavior implementation recognition unit 24 detects implementation of the recommended behavior. In S403, the content of the detected recommended action is transmitted from the communication unit 22 to the communication unit 18.

In S501, the recommended action implementation recognition unit 31 and the user state estimation unit 32 each acquire detection information from the detection information acquisition unit 12. In S502, the recommended behavior implementation recognition unit 31 detects implementation of the recommended behavior based on the detection information. In S503, the user state estimation unit 32 acquires data indicating the user state based on the detection information.

In S504, the feasibility estimating unit 33 (learning unit) determines that the feasibility estimation model that is the learning target is a recommended behavior class that is implemented from among predefined recommended behaviors using user state data as input. Create weight data for the model so that it can be estimated. Here, the weight data is created by using the recommended action recognized by the recommended action implementation recognition unit 24/recommended action implementation recognition unit 31 as the correct answer. The created weight data (learned model) is stored in a storage unit included in the feasibility estimating unit 33.

The learning operation will be explained more specifically using an example.

When the recommended action implementation recognition unit 31 of the wearable terminal 10 or the recommended action implementation recognition unit 24 of the electronic terminal 20 detects the user's implementation of the recommended action, the user state at the start timing of the detected recommended action implementation is determined by Simultaneously with the timing, it is recorded, for example, in the storage section of the feasibility estimating section 33.

Then, the feasibility estimating unit 33 trains the feasibility estimation model using the recommended behavior and the user state at the same timing as the timing at which the recommended behavior was detected.

An example will be explained using FIG. 14. In the example of FIG. 14, it is assumed that the user holds a smartphone as the electronic terminal 20 and a smart watch as the wearable terminal 10.

Here, the recommended action implementation recognition unit 24 or the recommended action implementation recognition unit 31 detects the implementation of “questionnaire input” as a user action (recommended action).

At the start of implementation of the "questionnaire input", the user state estimating unit 32 determines the user state as follows: the behavior is "standing still", the location is "outdoors", the schedule is "no plans", and the electronic terminal 20 (smartphone) is Data such as the status "Active" and the heart rate "82" are acquired and recorded in the storage section of the feasibility estimating section 33.

Note that the user state estimating unit 32 constantly acquires the above data, and records the acquired user state data in the storage unit at the timing of recommended action detection. At the same time, the detected recommended actions are also recorded in the storage unit.

The implementability estimating unit 33 (learning unit) inputs the user state at the start timing of the recorded recommended action to the implementability estimation model, and sets the implementability of the category of the recommended action performed by the user to 1 and that. The weight data of the feasibility estimation model is learned by setting the feasibility of other categories to 0. In the example of FIG. 14, the weight data of the feasibility estimation model is learned so that the feasibility of the category "questionnaire" is 1. Learning of weight data itself is an existing technique, and can be performed using, for example, error backpropagation.

<Other configuration examples in the learning phase>
In the above example, the feasibility estimation model is learned in the wearable terminal 10, but the present invention is not limited to such a form.

The learning of the feasibility estimation model may be performed by a device other than the wearable terminal 10 (this will be referred to as the learning device 100).

FIG. 15 shows a configuration example of the learning device 100. As shown in FIG. 15, the learning device 100 includes an input section 110, a learning section 120, an output section 130, and a storage section 140.

The recommended behavior detected in the electronic terminal 20 or the wearable terminal 10 and data on the user state at the start timing of implementation of the recommended behavior (data acquired in the wearable terminal 10) are input to the input unit 110. The input recommended actions and user status data are stored in the storage unit 140.

The storage unit 140 stores weight data of a feasibility estimation model to be learned. The learning unit 120 inputs the user state at the start timing of the recommended action to the implementation feasibility estimation model, and sets the implementation possibility of the category of recommended actions performed by the user to 1 and the implementation possibility of other categories to 0. , learn the weight data of the feasibility estimation model. Learning is performed using, for example, error backpropagation.

The learned weight data is output from the output unit 130 and input to the feasibility estimation unit 33 in the wearable terminal 10.

<Inference phase>
In the inference phase, the recommended action implementation recognition unit 24/recommended action implementation recognition unit 33 may not be provided in the electronic terminal 20 and the wearable terminal 10. Further, in the inference phase, it is assumed that the feasibility estimating unit 33 holds a learned feasibility estimation model (specifically, weight data). The feasibility estimating unit 33 itself may be considered to be a trained feasibility estimation model.

In the inference phase, the feasibility estimation unit 33 inputs the user state information obtained by the user state estimation unit 32 to the feasibility estimation model. The feasibility estimation model (trained NN) outputs the feasibility of each recommended action defined in advance. The feasibility estimation unit 33 inputs the feasibility of each recommended action outputted from the feasibility estimation model to the recommended action selection unit 34 .

The recommended behavior selection unit 34 acquires notification schedule time information from the recommended behavior reminder data obtained from the storage unit 11, and acquires information on recommended behaviors whose current time is within the schedule time. The recommended action selection unit 34 selects the recommended action that is most likely to be implemented within the scheduled time from among the one or more recommended actions output from the implementation possibility estimation unit 33.

For example, the recommended action whose current time is within the scheduled time is (taking medication, questionnaire), and the feasibility of each recommended action category is (questionnaire: 0.4, running: 0.5, taking medication: 0.1). If so, the recommended action selection unit 34 selects the questionnaire.

Thereafter, in the same manner as in the third embodiment (and the first embodiment), the vibration control unit 16 sends a message to the user at a notification timing that reduces the annoyance detected by the operation start confirmation unit 13 and the operation end estimation unit 15. Notification vibrations will be transmitted. The method of notifying at a timing that reduces annoyance is the same as the method described in the first embodiment.

In the case of notification in the fourth embodiment, information on the selected recommended action (for example, vibrating at a frequency corresponding to the questionnaire) may be included in the notification information. Further, in the case of the notification in the fourth embodiment, the selected recommended action (for example, a questionnaire) may be displayed as a message on the display of the wearable terminal 10 or the electronic terminal 20 at the same time as the notification (vibration).

With reference to the flowchart in FIG. 16, an example of a processing procedure for selecting a recommended action and notifying the user at a convenient timing will be described.

In S601, the electronic terminal 20 receives notification schedule information from, for example, a server. The schedule information is stored in the storage unit 21. In S602, the communication unit 22 transmits notification schedule information to the wearable terminal 10, and in S603, the communication unit 18 receives the notification schedule information. Schedule information is stored in the storage unit 11.

In S604 (S610), the wearable terminal 10 determines whether the current time is within the scheduled time. When the wearable terminal 10 determines that the current time is within the scheduled time, it executes S605 to S609 and S611. On the other hand, if the wearable terminal 10 determines that the current time is not within the scheduled time, it ends the notification process.

In S605, the user state estimating unit 32 acquires user state data. In S606, the implementability estimation unit 33 (implementability estimation model) outputs the implementability of each recommended action based on the user status, and the recommended action selection unit 34 outputs the implementability of each recommended action based on the user status. Among the recommended actions above, the recommended action that is most likely to be implemented within the scheduled time is selected.

Thereafter, through the same process as described in Example 1 (S607 to S609, S611), notification (vibration) is performed at a timing that is not bothersome, and at a timing when the user is performing an action. S607 to S609 and S611 correspond to S101 to S105 in FIG. 3 in the first embodiment.

According to this embodiment, it is possible to notify (reminder) the user at a time when there is a high possibility that the recommended action has been implemented, so the user can receive health care information without the user feeling bothered. It is possible to have the user perform the actions necessary for the operation of the app.

Note that in Example 4, an example was explained in which a NN is used as the feasibility estimation model, but the feasibility estimation model is not limited to a NN. Any model that can predict categories may be used.

Furthermore, when using a NN, the output function is not limited to a softmax function. Any method may be used as long as it can output the probability of implementation (feasibility of implementation) for each category.

Furthermore, although the fourth embodiment assumes the use of a healthcare app, the technology of the fourth embodiment can be applied to various uses regardless of whether a healthcare app is used.

(Summary of embodiments)
This specification describes at least the terminal, notification system, notification method, and program described in each section below.
(Section 1)
a detection information acquisition unit that acquires information on detected user motion;
a vibration control unit that generates vibration for notification to the user so as to vibrate while the user is performing the action, based on information obtained by detecting the user's action;
terminal.
(Section 2)
an operation start confirmation unit that determines whether the user has started an operation based on information obtained by detecting the user's operation;
an acceleration prediction unit that predicts a change in acceleration in the user's movement based on information obtained by detecting the user's movement;
further comprising a motion end estimation unit that estimates an end time of the user's motion based on a predicted change in acceleration in the user's motion;
When the vibration control unit determines that the vibration will end before the estimated end time of the user's motion based on a predefined vibration duration, the vibration control unit may be configured to send a notification to the user. generate vibrations,
The terminal described in paragraph 1.
(Section 3)
further comprising a vibration adjustment unit that adjusts the intensity of the vibration to be generated according to a predicted change in acceleration in the user's motion;
The terminal described in Section 2.
(Section 4)
a user state estimation unit that estimates the state of the user;
an implementation possibility estimating unit that estimates the implementation possibility of one or more predetermined actions based on the state;
The terminal according to claim 1, further comprising: an action selection unit that selects one action from the one or more actions based on the feasibility.
(Section 5)
further comprising an action implementation recognition unit that detects implementation of the action by the user,
The implementation possibility estimation model used in the implementation possibility estimation section is obtained by the user state estimation section based on the action whose implementation was detected by the action implementation recognition section and the timing at which the implementation of the action was detected. The terminal according to item 4, wherein the terminal is a model learned using the user's state.
(Section 6)
A notification system comprising a first terminal and a second terminal,
The first terminal is
a communication unit that receives information indicating a schedule for notification to the user from the second terminal;
a detection information acquisition unit that acquires information on detected user motion;
a vibration control unit that generates a vibration for notification to the user, based on information obtained by detecting the user's movement, at a scheduled notification timing, so as to vibrate while the user is performing the movement; and,
The second terminal is
comprising a communication unit that transmits information indicating a schedule for notification to the user to the first terminal;
Notification system.
(Section 7)
A notification method performed by a terminal,
a step of acquiring information on detecting a user's motion;
a step of generating vibration for notification to the user so as to vibrate while the user is performing the action, based on information obtained by detecting the user's action;
Notification method.
(Section 8)
A program for causing a computer to function as each part of the terminal according to any one of items 1 to 3.

Although the present embodiment has been described above, the present invention is not limited to such specific embodiment, and various modifications and changes can be made within the scope of the gist of the present invention as described in the claims. It is.

This patent application claims priority based on the international patent application PCT/JP2022/024966 filed on June 22, 2022, and the entire content of the international patent application PCT/JP2022/024966 is incorporated into this application.

10 Wearable terminal 11 Storage unit 12 Detection information acquisition unit 13 Operation start confirmation unit 14 Acceleration prediction unit 15 Operation end estimation unit 16 Vibration control unit 17 Vibration adjustment unit 18 Communication unit 20 Electronic terminal 21 Storage unit 22 Communication unit 23 Detection information acquisition unit 24 Recommended action implementation recognition unit 31 Recommended action implementation recognition unit 32 User state estimation unit 33 Implementability estimation unit 34 Recommended action selection unit 100 Learning device 110 Input unit 120 Learning unit 130 Output unit 140 Storage unit 101 CPU
102 Memory 103 Communication device 104 Sensor 105 Vibration device 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device 1008 Output device 1009 Sensor

Claims

a detection information acquisition unit that acquires information on detected user motion;
a vibration control unit that generates vibration for notification to the user so as to vibrate while the user is performing the action, based on information obtained by detecting the user's action;
terminal.
an operation start confirmation unit that determines whether the user has started an operation based on information obtained by detecting the user's operation;
an acceleration prediction unit that predicts a change in acceleration in the user's movement based on information obtained by detecting the user's movement;
further comprising a motion end estimation unit that estimates an end time of the user's motion based on a predicted change in acceleration in the user's motion;
When the vibration control unit determines that the vibration will end before the estimated end time of the user's motion based on a predefined vibration duration, the vibration control unit may be configured to send a notification to the user. generate vibrations,
The terminal according to claim 1.
further comprising a vibration adjustment unit that adjusts the intensity of the vibration to be generated according to a predicted change in acceleration in the user's motion;
The terminal according to claim 2.
a user state estimation unit that estimates the state of the user;
an implementation possibility estimating unit that estimates the implementation possibility of one or more predetermined actions based on the state;
The terminal according to claim 1, further comprising: an action selection unit that selects one action from the one or more actions based on the feasibility.
further comprising an action implementation recognition unit that detects implementation of the action by the user,
The implementation possibility estimation model used in the implementation possibility estimation section is obtained by the user state estimation section based on the action whose implementation was detected by the action implementation recognition section and the timing at which the implementation of the action was detected. The terminal according to claim 4, wherein the terminal is a model learned using the user's state.
A notification system comprising a first terminal and a second terminal,
The first terminal is
a communication unit that receives information indicating a schedule for notification to the user from the second terminal;
a detection information acquisition unit that acquires information on detected user motion;
a vibration control unit that generates a vibration for notification to the user, based on information obtained by detecting the user's movement, at a scheduled notification timing, so as to vibrate while the user is performing the movement; and,
The second terminal is
comprising a communication unit that transmits information indicating a schedule for notification to the user to the first terminal;
Notification system.
A notification method performed by a terminal,
a step of obtaining information on detected user motion;
a step of generating vibration for notification to the user so as to vibrate while the user is performing the action, based on information obtained by detecting the user's action;
Notification method.
A program for causing a computer to function as each part of the terminal according to any one of claims 1 to 5.