JP2022089139A - Solution provision system and portable terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a solution provision system and a portable terminal, with which it is possible to analyze a user's feeling or fatigue state and provide an appropriate solution simply that corresponds to the user's feeling or fatigue state.

SOLUTION: A solution storage unit stores multiple pieces of solution information for improving a user's state which is one or both of the user's feeling and fatigue state; an answer acceptance unit accepts input of an answer to a question about the state of the user's face; a user information acquisition unit acquires information regarding the user's heart beat from the image of the user's face; a user state information generation unit acquires information that indicates the use's fatigue stage or an autonomic nerve balance information on the basis of information regarding the heart beat; and a solution provision unit provides the user with solution information determined in accordance with the information that indicates the fatigue state based on the heart beat or the autonomic nerve balance information and the user state information generated based on the answer information.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a system for providing a solution for improving a user's emotional or fatigued state.

Conventionally, a mental and physical state guidance device is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2020-62282

According to the invention described in Patent Document 1, it is possible to specify the physical condition of the user from the biological information of the user and emit a stimulus for inducing a respiratory rhythm according to the physical condition.

It is conceivable to apply the invention described in Patent Document 1 to a system that analyzes a user's emotion or fatigue state and provides a solution according to the user's emotion or fatigue state.

However, since the device of Patent Document 1 is composed of a plurality of devices such as a biological sensor, a respiratory guidance device, an HCU, and an operation device, preparation and operation for use are complicated, and for example, a user who is not good at operating the device. It is difficult for people to properly prepare and operate the system.

If the user is not properly used, the user's condition cannot be obtained properly, and the provided solution will not be the appropriate solution according to the user's condition. Or if the operation is complicated, it will not be used by the user in the first place.

The solution providing system of the first invention is
A computer system consisting of mobile terminals that users can carry
A solution storage unit that stores information on a plurality of solutions that can be provided to the user in order to improve the user condition, which is one or both of the user's emotional and fatigue states.
A user information acquisition unit that acquires user information including information obtained from the user's body or movement, and
A user state information generation unit that generates user state information indicating the user state of the user based on information acquired from the body or movement acquired by the user information acquisition unit.
A solution decision unit that determines a solution according to the user status information from the plurality of solutions,
It is characterized by including a solution providing unit that provides the solution determined by the solution determining unit to the user.

The solution providing system of the second invention is
A computer system that includes a mobile terminal that can be carried by a user and a server.
The mobile terminal includes a user information acquisition unit and a solution provision unit.
The server includes a solution storage unit, a user status information generation unit, and a solution determination unit.
The solution storage unit stores information on a plurality of solutions that can be provided to the user in order to improve the user state, which is one or both of the user's emotional and fatigue states.
The user information acquisition unit acquires user information including information obtained from the user's body or movement, and transmits the user information to the server.
The user state information generation unit generates user state information indicating the user state of the user based on the information acquired from the body or movement acquired by the user information acquisition unit.
The solution determination unit determines a solution according to the user status information from the plurality of solutions and transmits the solution to the mobile terminal.
The solution providing unit is characterized in that the solution determined by the solution determining unit is provided to the user.

There are various timings when a user wants to be provided with a solution for improving his / her condition. Therefore, it is desirable to acquire the user's status using a portable terminal and easily receive the solution via the terminal.

According to the present invention, the user information acquisition unit of the mobile terminal acquires user information including information obtained from the user's body or movement, and the user state information generation unit obtains the information acquired from the body or movement. User status information indicating the user status of the user is generated.

Then, the solution determination unit determines a solution according to the user status information from a plurality of solutions, and the solution provider provides the determined solution to the user from the mobile terminal.

Further, as in the second invention, if the analysis of user information and the determination of the solution are performed on the server, the load for processing on the mobile terminal can be reduced accordingly.

As described above, according to the present invention, it is possible to analyze a user's emotion or fatigue state and easily provide an appropriate solution according to the user's emotion or fatigue state.

In the solution providing system of the second invention
The mobile terminal includes a history storage unit that stores the solution provided to the user for a predetermined period or a predetermined number of times.
When the determined solution is stored in the history storage unit, the solution determination unit does not send the determined solution to the mobile terminal.
When the solution determined by the solution determination unit is stored in the history storage unit, the solution providing unit acquires the determined solution from the history storage unit and provides the solution to the user. Is preferable.

It is thought that the user status will not improve rapidly but will improve gradually. Therefore, in the last few times, it is expected that the solution provided to the user will be the same solution.

According to the present invention, the information of the solution provided to the user is stored in the history storage unit of the mobile terminal for a predetermined period or a predetermined number of times.

If the solution determined by the solution determination unit is stored in the history storage unit, the solution determination unit does not send the solution to the mobile terminal, and the solution provider acquires the solution from the history storage unit. Since it is provided to the user, the amount of communication between the mobile terminal and the server is reduced, and the solution is promptly provided to the user.

In the solution providing system of the second invention
The mobile terminal includes an imaging unit and has an imaging unit.
The user information acquisition unit acquires user information including an image of the user's face via the image pickup unit.
It is preferable that the mobile terminal does not store the image of the user's face and the user state information included in the user information.

When the solution providing system is composed of a server and a mobile terminal as in the second invention, the user is an image of the user's face acquired by using the solution providing system and the analysis result thereof. It is assumed that we do not want information such as user status information to be recorded in the mobile terminal.

For example, users want to reduce the risk that mobile terminals will be invaded by computer viruses and the image of the user's face and the user status information that is the analysis result will be spread on the Internet. Is.

According to the present invention, since the mobile terminal does not store the image of the user's face and the user status information included in the user information, it resists the user who does not want such information to be recorded in the mobile terminal. It can be used without feeling.

In the solution providing system of the present invention
The mobile terminal includes a vibration generating unit.
The solution determined by the solution determination unit includes a predetermined periodic vibration according to the user state information.
It is preferable that the solution providing unit provides the user with the predetermined periodic vibration via the vibration generating unit.

The vibration of a predetermined cycle has an effect of relieving a person's feeling of tension and relieving fatigue.

According to the present invention, the solution determined by the solution determination unit includes a predetermined periodic vibration according to the user state of the user.

According to the present invention, the emotional or fatigued state of the user can be effectively improved.

In the solution providing system of the present invention
The mobile terminal includes an audio reproduction unit and has an audio reproduction unit.
The solution determined by the solution determination unit includes a human voice, a natural sound, or a voice including music according to the user state information.
It is preferable that the solution providing unit provides the voice to the user via the voice reproducing unit.

Human voices and natural sounds have the effects of activating the parasympathetic nerves, relieving human tension, and relieving fatigue. In addition, music has the effect of relieving a person's tension or activating the function of the sympathetic nerve to increase the person's tension. It will be more effective if these are provided in an appropriate combination according to the user's condition.

According to the present invention, abundant voice solutions can be appropriately combined according to the user's condition to more effectively improve the user's emotional or fatigued state.

In the solution providing system of the present invention
The mobile terminal includes a scent generator and has a scent generator.
The solution determined by the solution determination unit contains a predetermined scent according to the user status information.
It is preferable that the solution providing unit provides the scent to the user via the scent generating unit.

The predetermined scent has the effect of relieving a person's tension and relieving fatigue.

According to the present invention, since the solution determined by the solution determination unit contains a predetermined scent according to the user state of the user, the emotional or fatigue state of the user can be effectively improved.

In the solution providing system of the present invention
The mobile terminal includes a display unit and has a display unit.
The solution determined by the solution determination unit includes display information including information for promoting a predetermined operation or an image for improving the user state according to the user state information.
It is preferable that the solution providing unit provides the display information to the user via the display unit.

Taking a predetermined action or viewing an image to improve the user's condition has the effect of relieving a person's tension and relieving fatigue, or activating the sympathetic nerve function to activate a person's tension. It has the effect of enhancing the feeling. It will be more effective if these are provided appropriately according to the user's condition.

According to the present invention, the solution determined by the solution determination unit includes information that promotes a predetermined operation or display information including an image for improving the user state according to the user state information, and thus the user's emotions. Alternatively, the fatigue state can be effectively improved.

In the solution providing system of the present invention
The mobile terminal includes an imaging unit and has an imaging unit.
The user information acquisition unit acquires user information including an image of the user's face via the image pickup unit.
It is preferable that the user state information generation unit generates the user state information based on the image of the user's face acquired by the user information acquisition unit.

Since the face image is easy to acquire, the user's resistance to the image acquisition is low.

According to the present invention, a user can easily receive a solution according to his / her emotional or fatigue state by having a mobile terminal acquire an image of his / her face.

In the solution providing system of the present invention
The mobile terminal includes an imaging unit and has an imaging unit.
The user information acquisition unit
Recognizing whether the user is in the first situation where the image of the skin of the fingers should be acquired or the second situation where the image of the skin of the face should be acquired.
When the user is in the first situation, the user information including an image of the skin of the finger of the user is acquired, and when the user is in the second situation, an image of the skin of the user's face is obtained. Acquire the user information including
When the user is in the first situation, the user state information generation unit generates the user state information based on the image of the user's face acquired by the user information acquisition unit, and the user is said to be the user. In the second situation, the user state information is generated based on the image of the user's face acquired by the user information acquisition unit.
Is preferable.

There are various situations in which users who want to use the solution provision system are placed. For example, if you need to get an image of your skin, in some situations you may not want the user to get an image of your face, or you may want to get an image of your face but it is not in the right situation. Conceivable.

In such a case, according to the present invention, the user's emotion or fatigue state is accurately analyzed while changing the information to be acquired according to the user's situation, and the user's emotion or fatigue state is determined according to the user's emotion or fatigue state. It is possible to easily provide an appropriate solution.

In the solution providing system of the present invention
The mobile terminal includes an image pickup unit, a display control unit, a display unit, a response reception unit, and an input unit.
The user information acquisition unit acquires user information including an image of the user's face via the image pickup unit.
The display control unit displays an image of the user's face captured by the imaging unit on the display unit, and correlates with the emotion or fatigue state displayed on the user's face displayed on the display unit. The question about the condition of the face is displayed on the display unit.
The input unit is arranged at a position where the user can operate while looking at the image of the user's face displayed on the display unit and the question.
The answer receiving unit receives an input of an answer by the user to a question about the state of the user's face via the input unit in a state where the image of the face is displayed on the display unit.
It is preferable that the user state information generation unit generates the user state information based on the information including the answer.

The user's emotions and fatigue are clearly visible on the user's face.

According to the present invention, the user answers a question about the condition of the face while looking at his / her own face, so that the answer is easy. The question is a question that has a correlation with the emotion or fatigue state appearing on the user's face. In addition, since you will have the opportunity to calmly look back on your facial condition, you will be promoted to be aware of the facial condition that is closely related to your emotions or fatigue, and you may be able to answer your facial condition accurately. It will increase.

Further, according to the present invention, the user can easily receive a personalized solution according to his / her emotional or fatigued state by getting the answer regarding the facial image and the facial condition.

As described above, according to the present invention, it is possible to analyze a user's emotion or fatigue state and easily provide an appropriate solution according to the user's emotion or fatigue state.

The block diagram which shows the whole picture of the solution providing system of this invention. The flowchart which shows the processing content of the solution providing system of this invention. The flowchart which shows the processing content of the solution providing system of this invention. The flowchart which shows the processing content of the solution providing system of this invention. The block diagram which shows an example of the modification embodiment of the solution providing system of this invention.

<Configuration of solution provision system>
First, the configuration of the solution providing system of the present embodiment will be described with reference to FIG. The same components may be designated by the same reference numerals and description thereof may be omitted.

The solution providing system of this embodiment is a computer system including a mobile terminal 10 and a server 30. The mobile terminal 10 and the server 30 are connected to each other so as to be able to communicate with each other by a communication network 90 such as a mobile phone network. The number of mobile terminals 10 and the number of servers 30 may be plural.

The mobile terminal 10 includes a mobile terminal control unit 110, an output unit 130, or further includes an image pickup unit 150, an input unit 170, and a history storage unit 190, for example, a smartphone, a tablet terminal, or the like. It is a general-purpose terminal. Alternatively, the mobile terminal 10 may be a dedicated terminal that can be carried by the user.

The mobile terminal control unit 110 is composed of an arithmetic processing unit such as a CPU (Central Processing Unit), a memory, an I / O (Input / Output) device, and the like. By reading and executing a predetermined program, the mobile terminal control unit 110 functions as, for example, a user information acquisition unit 111, a solution provision unit 113, or further, a display control unit 115, and a response reception unit 117.

The user information acquisition unit 111 acquires user information including information obtained from the user's body or movement via, for example, the image pickup unit 150 and the input unit 170.

The solution providing unit 113 provides the solution determined by the solution determining unit 313 to the user via the output unit 130.

The display control unit 115 displays the image of the user's face captured by the image pickup unit 150 on the display unit 133, and displays a question about the state of the user's face displayed on the display unit 133 on the display unit 133. To control. The question is a question that has a correlation with the emotion or fatigue state appearing on the user's face.

The answer receiving unit 117 accepts the input of the answer by the user to the question about the state of the user's face via the input unit 170 in a state where the image of the face is displayed on the display unit.

The output unit 130 includes, for example, a vibration generation unit 131, a display unit 133, a voice reproduction unit 135, and a scent generation unit 137, and provides a solution to the user by controlling the solution provision unit 113 and the like. ..

The vibration generation unit 131 is, for example, a vibrator that provides the user with a predetermined periodic vibration when the solution determined by the solution determination unit 313 includes a predetermined periodic vibration according to the user state information. ..

The display unit 133 is a display that displays the display information and provides it to the user when the display information including information for promoting a predetermined operation or an image for improving the user state is included according to the user state information. ..

The voice reproduction unit 135 is a speaker that reproduces the voice and provides it to the user when the solution includes a voice including a human voice, natural sounds, or music according to the user status information.

The scent generation unit 137 generates a scent and provides it to the user when a predetermined scent according to the user state information is included in the solution.

The image pickup unit 150 is, for example, a camera, and acquires an image of the user's face.

The input unit 170 is, for example, an input mechanism for characters such as a touch panel and a keyboard, a selection result, and the like, and is used when the answer reception unit 117 receives an input of an answer by a user to a question about a user's facial condition. The input unit 170 is arranged at a position where the operation can be performed while looking at the image of the user's face displayed on the display unit 133 and the question about the state of the face.

The history storage unit 190 includes, for example, a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an HDD (Hard Disk Drive). The history storage unit 190 stores the information of the solution provided to the user for a predetermined period or a predetermined number of times, and also stores the information necessary for the processing of the mobile terminal, the processing result of the mobile terminal, and the like. ..

The history storage unit 190 is configured not to store the face image and the user state information included in the user information, for example.

The server 30 is, for example, a computer including a server control unit 310 and a solution storage unit 330.

The server control unit 310 is composed of an arithmetic processing unit such as a CPU, a memory, an I / O device, and the like. The server control unit 310 functions as, for example, a user status information generation unit 311 and a solution determination unit 313 by reading and executing a predetermined program.

The user state information generation unit 311 generates user state information indicating the user state of the user based on the information acquired from the body or movement acquired by the user information acquisition unit 111.

The solution determination unit 313 determines a solution according to the user status information from a plurality of solutions stored in the solution storage unit 330.

The solution storage unit 330 is composed of a storage device such as a ROM, a RAM, or an HDD. The solution storage unit 330 provides information on a plurality of solutions that can be provided to the user in order to improve the user condition, which is one or both of the user's emotional and fatigue states, as well as information and solutions necessary for processing the solution providing system. It stores the processing results of the provided system.

The solution information stored in the solution storage unit 330 may be digital data itself such as music, voice, video, a predetermined periodic vibration pattern, or code information such as an ID that identifies a predetermined scent. It may be there. When these solutions are provided to the user alone or in combination, they function as a solution for improving the user's emotional and / or fatigued state.

The user's emotions include so-called psychological states such as emotions and negative states, as well as the state of autonomic nervous balance as one of the indicators of whether the person is in an excited state or a relaxed state, and awakening and rest. It is a concept that includes the state of self-awareness by the user such as, calmness, and liberation.

<Outline of processing>
Next, the processing contents of the solution providing system of this embodiment will be described. First, a series of processes by the solution providing system will be described with reference to FIG.

<User information acquisition process>
When a series of processes are started, the user information acquisition unit 111 of the mobile terminal 10 first acquires the user information and transmits it to the server 30 (FIG. 2 / S110).

More specifically, for example, as shown in FIG. 3, the user information acquisition unit 111 acquires an image of the user's face via the image pickup unit 150 (FIG. 3 / S111).

Then, the display control unit 115 displays the image of the user's face captured by the image pickup unit 150 on the display unit 133, and displays the question about the state of the user's face displayed on the display unit 133 on the display unit 133. (FIG. 3 / S113).

The input receiving unit 155 receives the input of the answer by the user to the question about the state of the user's face via the input unit 170 in a state where the image of the face is displayed on the display unit 133. (FIG. 3 / S115).

For example, the input receiving unit 155 displays a plurality of questions about the state of the user's face and answer options in order for each question, and accepts the options selected by the user as answers to the question.

The user information acquisition unit 111 acquires the answer to the question about the facial condition received in this way as the user status information.

The answer to the question about the condition of the face may be input by, for example, letting the user select which of the five-grade evaluation corresponds to, or may be input by voice or text. Further, the question may be one or more.

Then, the user information acquisition unit 111 analyzes the image of the face (FIG. 3 / S117). That is, for example, the user information acquisition unit 111 acquires information necessary for acquiring information on the heartbeat from the face image as an analysis result of the face image.

Various known methods may be used as a method for the user information acquisition unit 111 to acquire information necessary for acquiring information related to the heartbeat from the face image, and the user information acquisition unit 111 is, for example, a face image. Information on changes in the brightness of each color component of RGB is acquired from a moving image or a continuously captured still image.

Then, the user information acquisition unit 111 answers the user's answer to the question about the facial condition of the user who has received the input in this way, and the analysis result of the facial image (for example, RGB which is information necessary for acquiring information about the heartbeat). Information on changes in the brightness of each color component of the above) and is transmitted to the server 30 as user information (FIG. 3 / S119).

<User information analysis processing>
Then, the user state information generation unit 311 of the server 30 receives the user information (FIG. 2 / S210), and the user information such as the answer by the user to the question about the user's face state and the analysis result of the face image. The information contained in is analyzed (FIG. 2 / S230).

More specifically, for example, the user state information generation unit 311 acquires information on the user's heartbeat from the information on the change in the luminance of each color component of RGB, which is the information necessary for acquiring the information on the heartbeat (FIG. 4 / S231). ).

Various known methods may be used as a method for the user state information generation unit 311 to acquire information on the user's heartbeat from the information necessary for acquiring the information on the heartbeat, and the user state information generation unit 311 is, for example, RGB. The user's heartbeat is acquired by analyzing the change in the brightness of a specific color component (for example, G (green)) included in the information on the change in the brightness of each color component in the above using a predetermined algorithm.

Further, the user state information generation unit 311 calculates and acquires, for example, a heart rate, a heart rate interval, or an index value such as CVR-R or CCVTP from these information from the user's heart rate.

Then, the user state information generation unit 311 uses a predetermined algorithm from index values such as CVR-R and CCVTP to indicate a fatigue state based on the user's heartbeat in 0.1 increments between 0 and 10. Is calculated.

After that, the user state information generation unit 311 acquires the autonomic nerve balance information based on the information related to the heartbeat (FIG. 4 / S233).

Various methods may be used as the method for the user state information generation unit 311 to acquire the autonomic nerve balance information. That is, for example, the user state information generation unit 311 frequency-analyzes the fluctuation of the heartbeat, and the ratio (LF / HF) of LF (Low Frequency), which is a low frequency component of the fluctuation of the heartbeat, and HF (High Frequency), which is a high frequency component. ) Is acquired as autonomic nerve balance information.

Then, the user state information generation unit 311 calculates an autonomic nerve balance score indicating the balance of the user's autonomic nerve from 0 to 10 in increments of 0.1 using a predetermined algorithm from the autonomic nerve balance information.

Then, the user state information generation unit 311 analyzes the answer by the user to the question about the state of the user's face included in the user state information (FIG. 4 / S235).

More specifically, the user state information generation unit 311 obtains, for example, a value obtained by adding the values associated with each option selected by the user for each question about the state of the user's face.
The signs of fatigue appearing on the user's face are quantified as a quantified fatigue sign score, and the fatigue sign score is determined as an analysis result.

<User status information generation process>
Then, the user state information generation unit 311 generates user state information indicating the user state of the user based on the analysis result in the user information analysis process (FIG. 2 / S250).

For example, the user state information generation unit 311 uses the combination of the fatigue degree score based on the heartbeat, the autonomic nerve balance score, and the fatigue sign score obtained above as the user state information.

<Solution decision process>
Then, the solution determination unit 313 determines a solution according to the user status information and transmits it to the mobile terminal 10 (FIG. 3 / S270).

More specifically, for example, a combination of a fatigue degree score based on a heartbeat, an autonomic nerve balance score, and a fatigue sign score and a correspondence table with the solution are stored in the solution storage unit 330 in advance, and the solution determination unit 313 stores the combination. , A solution provided to the user by recognizing a solution corresponding to the combination of the fatigue score based on the heart rate, the autonomic nerve balance score, and the fatigue sign score obtained as described above by referring to the corresponding table. To decide.

That is, for example, if the fatigue score based on the heartbeat or the sign of fatigue score is high, a predetermined periodic vibration having an effect of lowering the fatigue, a human voice, a voice including natural sound or music, a predetermined scent, and a predetermined The information that promotes the operation or the image for improving the user condition is determined alone or a combination thereof according to the user condition as a solution to be provided to the user.

Alternatively, when the autonomic nerve balance score indicates a state in which the autonomic nerve balance is not well-balanced, for example, the sympathetic nerve or the parasympathetic nerve is excessively dominant, a predetermined periodic period having an effect of adjusting the autonomic nerve balance is obtained. Vibration and the like are determined alone or a combination of these according to the user's condition as a solution to be provided to the user.

<Solution provision process>
Then, the solution providing unit 113 receives the solution from the server 30 (FIG. 3 / S310) and provides the received solution to the user (FIG. 3 / S330).

More specifically, the solution providing unit 113 passes through the vibration generating unit 131, the display unit 133, the voice reproducing unit 135, and the scent generating unit 137 according to the content of the solution received from the server 30, and has a predetermined periodicity. Tactile stimuli such as vibrations, human voices, natural sounds (bird's voice, stream babbling, etc.) or auditory stimuli such as voice including music, odor stimuli such as predetermined scents, to improve user status It provides users with passive stimuli such as visual stimuli such as images, and active stimuli containing information that encourages users to perform predetermined actions.

That is, for example, if the received solution is digital data of music, the solution providing unit 113 causes the voice reproduction unit 135 to play the music. Alternatively, for example, if the received solution has an ID that specifies a predetermined scent, the solution providing unit 113 generates the scent corresponding to the ID in the scent generating unit 137. Or, for example, if the received solution is information that encourages the user to perform a certain action, "Let's do a simple mindfulness meditation while feeling the vibration" that changes at regular intervals, "Stretch your back and correct your posture. A combination of sentences that encourage breathing or meditation in a predetermined action or in a predetermined manner, such as "Let's relax your shoulders and breathe slowly through your nose", is displayed by the display unit 133, or the voice reproduction unit 135. Played by.

As described above, according to the present invention, it is possible to analyze a user's emotion or fatigue state and easily provide an appropriate solution according to the user's emotion or fatigue state.

The embodiments of the present invention have been described above, but the present invention is not limited thereto. Various changes can be made without departing from the spirit of the present invention.

That is, for example, it is considered that the user state does not improve rapidly but gradually improves. Therefore, it is expected that the solutions of the last few times or the last period will be the same solution.

Therefore, for example, the mobile terminal 10 may be configured to store the solution provided to the user in the history storage unit 190 for a predetermined period or a predetermined number of times.

In this case, the solution determination unit 313 is configured not to transmit the determined solution to the mobile terminal 10 when the determined solution is stored in the history storage unit 190.

Then, when the solution determined by the solution determination unit 313 is stored in the history storage unit 190, the solution provision unit 113 acquires the determined solution from the history storage unit 190 and provides the solution to the user. It is composed.

As a method for the solution determination unit 313 to determine whether or not the determined solution is stored in the history storage unit 190 of the mobile terminal 10, various methods may be adopted.

That is, for example, the solution determination unit 313 stores information specifying a solution determined in the past in the solution storage unit 330 for a predetermined period or a predetermined number of times. The predetermined period or the predetermined number of times is the same as the period or the number of times stored in the history storage unit 190 of the mobile terminal 10.

Then, the solution determination unit 313 determines whether or not the solution determined this time matches any of the solutions stored in the solution storage unit 330, and when the matching solution is stored in the solution storage unit 330. In addition, it is determined that the determined solution is stored in the history storage unit 190 of the mobile terminal 10.

In this case, the solution determination unit 313 does not send the determined solution to the mobile terminal 10, and instead, the information that identifies which solution the determined solution is stored in the history storage unit 190. Is transmitted to the mobile terminal 10.

Then, the solution providing unit 113 receives information from the server 30 that identifies which solution is stored in the history storage unit 190, and obtains the corresponding solution from the history storage unit 190. Get it and provide it to the user.

Alternatively, for example, as shown in FIG. 5, the server 30 may be omitted, and the mobile terminal 10 may be provided with all the functions described above by the server 30.

Alternatively, for example, the user information acquisition unit 111 inputs an answer to the question by the user while showing the image of the user's face reflected in the mirror unit that reflects the user's face, instead of displaying the image of the face on the display unit 133. It may be configured to accept.

In this case, the display unit 133 is arranged at a position where the user can see the user's face reflected in the mirror unit at the same time, and the input unit 170 looks at the user's face reflected in the mirror unit and the question displayed on the display unit 133. It is placed in a position where it can be operated.

Alternatively, for example, the user information acquisition unit 111 recognizes whether the user is in the first situation where the image of the skin of the fingers should be acquired or the second situation where the image of the skin of the face should be acquired. When the user is in the first situation, the user information including the image of the skin of the finger of the user is acquired, and when the user is in the second situation, the user information including the image of the skin of the user's face is acquired. It may be configured as follows.

In this case, the user state information generation unit 311 generates user state information based on the image of the user's face acquired by the user information acquisition unit 111 when the user is in the first situation, and the user is in the second situation. In the case of, the user state information is generated based on the image of the user's face acquired by the user information acquisition unit 111.

For example, the user information acquisition unit 111 displays a selection button for selecting whether the user is in the first situation or the second situation on the display of the mobile terminal 10 and causes the user to select the user. Recognize whether you are in the 1st situation or the 2nd situation. Alternatively, for example, the user information acquisition unit 111 may recognize whether the user is in the first situation or the second situation based on the current time zone. That is, for example, the user information acquisition unit 111 recognizes that it is the first situation in the daytime time zone and the second situation in the night time zone before going to bed.

10 ... Mobile terminal, 30 ... Server, 111 ... User information acquisition unit, 113 ... Solution provision unit, 115 ... Display control unit, 117 ... Answer reception unit, 131 ... Vibration generation unit, 133 ... Display unit, 135 ... Voice playback unit , 137 ... Fragrance generation unit, 150 ... Imaging unit, 170 ... Input unit, 190 ... History storage unit, 311 ... User status information generation unit, 313 ... Solution determination unit, 330 ... Solution storage unit.

The present invention relates to a system and a mobile terminal for providing a solution for improving a user's emotional or fatigued state.

By the way, in order to effectively improve the user's emotions and fatigue by using the system and mobile terminal that provide the solution to improve the user's emotions or fatigue, it is necessary to acquire sufficient information about the user's condition. , It is necessary to determine and provide a solution according to the user's condition.

However, the information that the device of Patent Document 1 identifies the user's physical condition is limited to the user's biological information, and is insufficient as information on the user's condition for effectively improving the user's emotion and fatigue state. ..

If the information about the user's condition is insufficient, the provided solution will not be the appropriate solution according to the user's condition. Therefore, the present invention provides a solution providing system and a mobile terminal capable of acquiring sufficient information for grasping the user's condition and providing an effective solution according to the grasped user's condition. The purpose is to provide.

The solution providing system of the first invention is
A computer system consisting of a mobile terminal that can be carried by a user, a server, and so on.
The mobile terminal includes an image pickup unit, a display control unit, a display unit, a response reception unit, an input unit, a user information acquisition unit, and a solution provision unit.
The server includes a solution storage unit, a user status information generation unit, and a solution determination unit.
The solution storage unit stores information on a plurality of solutions that can be provided to the user in order to improve the user state, which is one or both of the user's emotional and fatigue states.
The display control unit displays an image of the user's face captured by the imaging unit on the display unit, and correlates with the emotion or fatigue state displayed on the user's face displayed on the display unit. The question about the condition of the face is displayed on the display unit.
The answer receiving unit receives input of an answer by the user to a question about the facial condition of the user via the input unit.
The user information acquisition unit
The information of the image of the user's face captured by the image pickup unit is acquired, and the information is obtained.
The information about the heartbeat of the user is acquired from the image of the face, and the information about the heartbeat and the answer are transmitted to the server.
The user status information generation unit is
Based on the information about the heartbeat acquired by the user information acquisition unit, information indicating a fatigue state based on the user's heartbeat or autonomic nerve balance information indicating the autonomic nerve balance of the user is acquired.
Based on the information indicating the fatigue state or the autonomic nerve balance information based on the acquired heartbeat and the answer received by the answer receiving unit, the user state information indicating the user state of the user is generated.
The solution determination unit determines solution information corresponding to the user state information generated by the user state information generation unit from the plurality of solution information and transmits the solution information to the mobile terminal.
The solution providing unit provides the user with the solution information determined by the solution determining unit.
It is characterized by that.

Further, the mobile terminal of the second invention is
A mobile terminal used in a computer system consisting of a mobile terminal that can be carried by a user, a server, and the like.
The server includes a solution storage unit, a user status information generation unit, and a solution determination unit.
The solution storage unit stores information on a plurality of solutions that can be provided to the user in order to improve the user state, which is one or both of the user's emotional and fatigue states.
The user status information generation unit is
Based on the information about the user's heartbeat acquired by the user information acquisition unit provided in the mobile terminal, the information indicating the fatigue state based on the user's heartbeat or the autonomic nerve balance information indicating the user's autonomic nerve balance is obtained. Acquired,
The user state of the user is determined based on the information indicating the fatigue state based on the acquired heartbeat or the autonomic nerve balance information, and the answer by the user to the question about the face state of the user from the mobile terminal. Generates the user state information shown and
The solution determination unit determines solution information corresponding to the user state information generated by the user state information generation unit from the plurality of solution information and transmits the solution information to the mobile terminal.
Used in a system that provides the solution information to the user from the solution providing unit provided in the mobile terminal.
It is equipped with an image pickup unit, a display control unit, a display unit, a response reception unit, an input unit, a user information acquisition unit, and a solution provision unit.
The display control unit displays an image of the user's face captured by the imaging unit on the display unit, and correlates with the emotion or fatigue state displayed on the user's face displayed on the display unit. The question about the condition of the face is displayed on the display unit.
The answer receiving unit receives input of an answer by the user to a question about the facial condition of the user via the input unit.
The user information acquisition unit
The information of the image of the user's face captured by the image pickup unit is acquired, and the information is obtained.
Information about the user's heartbeat is acquired from the face image, and the information about the heartbeat and the answer are transmitted to the server.
The solution providing unit provides the user with the solution information determined by the solution determining unit.
It is characterized by that.

Further, the mobile terminal of the third invention is
A mobile terminal that can be carried by the user
The mobile terminal includes a solution storage unit, a display control unit, an imaging unit, a display unit, an answer reception unit, an input unit, a user information acquisition unit, a user status information generation unit, a solution determination unit, and the like. With a solution provider
The solution storage unit stores information on a plurality of solutions that can be provided to the user in order to improve the user state, which is one or both of the user's emotional and fatigue states.
The display control unit displays an image of the user's face captured by the imaging unit on the display unit, and correlates with the emotion or fatigue state displayed on the user's face displayed on the display unit. The question about the condition of the face is displayed on the display unit.
The answer receiving unit receives input of an answer by the user to a question about the facial condition of the user via the input unit.
The user information acquisition unit
The information of the image of the user's face captured by the image pickup unit is acquired, and the information is obtained.
Obtaining information about the user's heartbeat from the face image,
The user status information generation unit is
Based on the information about the heartbeat acquired by the user information acquisition unit, information indicating a fatigue state based on the user's heartbeat or autonomic nerve balance information indicating the autonomic nerve balance of the user is acquired.
Based on the information indicating the fatigue state or the autonomic nerve balance information based on the acquired heartbeat and the answer received by the answer receiving unit, the user state information indicating the user state of the user is generated.
The solution determination unit determines solution information according to the user state information generated by the user state information generation unit from the plurality of solution information.
The solution providing unit provides the user with the solution information determined by the solution determining unit.
It is characterized by that.

Objective data such as the user's heartbeat information and the user's own subjective sensation are both important indicators for grasping the user's emotion and fatigue state.

According to the present invention, the user state information generation unit correlates objective information such as fatigue state information based on the user's heartbeat or autonomic nerve balance information with emotions or fatigue states appearing on the user's face. Both the subjective information of the user's own answer to the question about the condition of the face is obtained. In addition, the user status information generation unit generates user status information indicating the user status of the user based on both of these information.

Then, the solution providing unit provides the user with solution information according to the user state information generated by the user state information based on the information indicating the fatigue state based on the heartbeat or the autonomic nerve balance information and the answer information. ..

As a result, objective information regarding the user's emotional state and / or user's state, which is one or both of the user's emotions and fatigued state, which is obtained from the user's face (information indicating the fatigued state based on the user's heartbeat or autonomic nerve balance information). And subjective information (the user's answer to the question about facial condition), and by considering both, it is possible to provide an appropriate solution according to these combinations and effectively improve the user condition. ..

As described above, according to the present invention, it is possible to acquire sufficient information for grasping the state of the user and provide an effective solution according to the grasped state of the user. Further, as in the first and second inventions, if the analysis of user information and the determination of the solution are performed on the server, the load for processing on the mobile terminal can be reduced accordingly.

In the present invention
The user information acquisition unit acquires information on changes in the brightness of each color component of RGB as information on the heartbeat from the image of the user's face captured by the image pickup unit.
Is preferable.

According to the present invention, when acquiring information on the heartbeat, the user only needs to have the image pickup unit acquire an image of his / her face, so that the operation is easy. Therefore, it is possible to easily acquire information on the heartbeat used for grasping the user's state.

In the present invention
The user state information generation unit acquires the value of the user's CVR-R or CCVTP based on the information regarding the heartbeat acquired by the user information acquisition unit, and is based on the value of the user's CVR-R or CCVTP. Acquires information indicating a fatigue state based on the heartbeat.
Is preferable.

The value of CVR-R or CCVTP calculated by analyzing the information about the heartbeat is closely related to the fatigue state of a person.

According to the present invention, the user state information generation unit acquires the value of the user's CVR-R or CCVTP based on the information about the heartbeat, and the fatigue state based on the heartbeat based on the value of the user's CVR-R or CCVTP. Since the information indicating the user's fatigue state is acquired, it is possible to properly grasp the user's fatigue state and provide an effective solution according to the user's fatigue state.

In the present invention
The user state information generation unit acquires the LF / HF value of the user based on the information regarding the heartbeat acquired by the user information acquisition unit, and the autonomic nerve balance information is based on the LF / HF value of the user. To get
Is preferable.

The LF / HF value calculated by analyzing the information regarding the heartbeat has a close relationship with the human autonomic nervous balance.

According to the present invention, the user state information generation unit acquires the user's LF / HF value based on the information related to the heartbeat, and the autonomic nerve balance information is acquired based on the user's LF / HF value. It is possible to properly grasp the state of the autonomic nerve balance and provide an effective solution according to the state of the user's autonomic nerve balance.

In the present invention
The image pickup unit and the display unit are provided on the same surface of the mobile terminal.
Is preferable.

With such a configuration, it is possible to answer questions about the condition of the face while surely showing the user's own face.

Further, the mobile terminal of the second invention is
A mobile terminal used in a computer system consisting of a mobile terminal that can be carried by a user, a server, and the like.
The server includes a solution storage unit, a user status information generation unit, and a solution determination unit.
The solution storage unit stores information on a plurality of solutions that can be provided to the user in order to improve the user state, which is one or both of the user's emotional and fatigue states.
The user status information generation unit is
Based on the information about the user's heartbeat acquired by the user information acquisition unit provided in the mobile terminal, the information indicating the fatigue state based on the user's heartbeat or the autonomic nerve balance information indicating the user's autonomic nerve balance is obtained. Acquired,
The user of the user based on the information indicating the fatigue state or the autonomic nerve balance information based on the acquired heartbeat and the information of the answer by the user to the question about the facial condition of the user from the mobile terminal. Generates user state information that indicates the state
The solution determination unit determines solution information corresponding to the user state information generated by the user state information generation unit from the plurality of solution information and transmits the solution information to the mobile terminal.
Used in a system that provides the solution information to the user from the solution providing unit provided in the mobile terminal.
The mobile terminal includes an image pickup unit, a display control unit, a display unit, a response reception unit, an input unit, a user information acquisition unit, and a solution provision unit. The display control unit is imaged by the image pickup unit. The image of the user's face is displayed on the display unit, and a question about the facial condition that correlates with the emotional or fatigue state displayed on the user's face displayed on the display unit is asked. Display on the display
The answer reception unit accepts input of answer information by the user to a question about the face condition of the user via the input unit.
The user information acquisition unit
The information of the image of the user's face captured by the image pickup unit is acquired, and the information is obtained.
The information about the heartbeat of the user is acquired from the image of the face, and the information about the heartbeat and the information of the answer are transmitted to the server.
The solution providing unit is characterized in that the solution information determined by the solution determining unit is provided to the user.

Claims (11)

A computer system consisting of mobile terminals that users can carry
A solution storage unit that stores information on a plurality of solutions that can be provided to the user in order to improve the user condition, which is one or both of the user's emotional and fatigue states.
A user information acquisition unit that acquires user information including information obtained from the user's body or movement, and
A user state information generation unit that generates user state information indicating the user state of the user based on information acquired from the body or movement acquired by the user information acquisition unit.
A solution decision unit that determines a solution according to the user status information from the plurality of solutions,
A solution providing system including a solution providing unit that provides the solution determined by the solution determining unit to the user. A computer system that includes a mobile terminal that can be carried by a user and a server.
The mobile terminal includes a user information acquisition unit and a solution provision unit.
The server includes a solution storage unit, a user status information generation unit, and a solution determination unit.
The solution storage unit stores information on a plurality of solutions that can be provided to the user in order to improve the user state, which is one or both of the user's emotional and fatigue states.
The user information acquisition unit acquires user information including information obtained from the user's body or movement, and transmits the user information to the server.
The user state information generation unit generates user state information indicating the user state of the user based on the information acquired from the body or movement acquired by the user information acquisition unit.
The solution determination unit determines a solution according to the user status information from the plurality of solutions and transmits the solution to the mobile terminal.
The solution providing unit is a solution providing system characterized in that the solution determined by the solution determining unit is provided to the user. In the solution providing system according to claim 2,
The mobile terminal includes a history storage unit that stores the solution provided to the user for a predetermined period or a predetermined number of times.
When the determined solution is stored in the history storage unit, the solution determination unit does not send the determined solution to the mobile terminal.
When the solution determined by the solution determination unit is stored in the history storage unit, the solution providing unit acquires the determined solution from the history storage unit and provides the solution to the user. A solution providing system featuring. In the solution providing system according to claim 2 or 3.
The mobile terminal includes an imaging unit and has an imaging unit.
The user information acquisition unit acquires user information including an image of the user's face via the image pickup unit.
The mobile terminal is a solution providing system characterized in that it does not store the image of the user's face and the user status information included in the user information. In the solution providing system according to any one of claims 1 to 4.
The mobile terminal includes a vibration generating unit.
The solution determined by the solution determination unit includes a predetermined periodic vibration according to the user state information.
The solution providing unit is a solution providing system characterized by providing the user with the predetermined periodic vibration via the vibration generating unit. In the solution providing system according to any one of claims 1 to 5.
The mobile terminal includes an audio reproduction unit and has an audio reproduction unit.
The solution determined by the solution determination unit includes a human voice, a natural sound, or a voice including music according to the user state information.
The solution providing unit is a solution providing system characterized in that the voice is provided to the user via the voice reproducing unit. In the solution providing system according to any one of claims 1 to 6.
The mobile terminal includes a scent generator and has a scent generator.
The solution determined by the solution determination unit contains a predetermined scent according to the user status information.
The solution providing unit is a solution providing system characterized in that the scent is provided to the user via the scent generating unit. In the solution providing system according to any one of claims 1 to 7.
The mobile terminal includes a display unit and has a display unit.
The solution determined by the solution determination unit includes display information including information for promoting a predetermined operation or an image for improving the user state according to the user state information.
The solution providing unit is a solution providing system characterized in that the display information is provided to the user via the display unit. In the solution providing system according to any one of claims 1 to 8.
The mobile terminal includes an imaging unit and has an imaging unit.
The user information acquisition unit acquires user information including an image of the user's face via the image pickup unit.
The user state information generation unit is a solution providing system characterized in that the user state information is generated based on the image of the user's face acquired by the user information acquisition unit. In the solution providing system according to any one of claims 1 to 8.
The mobile terminal includes an imaging unit and has an imaging unit.
The user information acquisition unit
Recognizing whether the user is in the first situation where the image of the skin of the fingers should be acquired or the second situation where the image of the skin of the face should be acquired.
When the user is in the first situation, the user information including an image of the skin of the finger of the user is acquired, and when the user is in the second situation, an image of the skin of the user's face is obtained. Acquire the user information including
When the user is in the first situation, the user state information generation unit generates the user state information based on the image of the user's face acquired by the user information acquisition unit, and the user is said to be the user. In the second situation, the user state information is generated based on the image of the user's face acquired by the user information acquisition unit.
A solution providing system characterized by that. In the solution providing system according to any one of claims 1 to 9.
The mobile terminal includes an image pickup unit, a display control unit, a display unit, a response reception unit, and an input unit.
The user information acquisition unit acquires user information including an image of the user's face via the image pickup unit.
The display control unit displays an image of the user's face captured by the imaging unit on the display unit, and correlates with the emotion or fatigue state displayed on the user's face displayed on the display unit. The question about the condition of the face is displayed on the display unit.
The input unit is arranged at a position where the user can operate while looking at the image of the user's face displayed on the display unit and the question.
The answer receiving unit receives an input of an answer by the user to a question about the state of the user's face via the input unit in a state where the image of the face is displayed on the display unit.
The user state information generation unit is a solution providing system characterized in that the user state information is generated based on the information including the answer.

Images