JP2021189573A - Information processing system and information processing method - Google Patents

Abstract

To provide an information processing system in which a user can be prevented from feeling uncomfortable with sensing.SOLUTION: An information processing system 10 includes: an acquisition unit 32a that acquires environmental information about a facility 80 and trigger information; a control unit 32b that, when the trigger information is acquired, starts sensing of a user by means of an image sensor 37 set in the facility 80; and an output unit 32c that outputs recommend information for giving a recommendation to the user on the basis of the acquired environmental information and sensing information acquired by the sensing.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system and an information processing method.

Various techniques for presenting information to users have been proposed. Patent Document 1 discloses a recommendation information presentation system capable of presenting recommendation information to the user in response to a change in the action schedule of the user.

Japanese Unexamined Patent Publication No. 2017-134635

A configuration is conceivable in which the user is sensed in the facility and information is recommended to the user. In such a configuration, the user may be reluctant to constantly perform sensing in the facility.

The present invention provides an information processing system or the like that can suppress a user from feeling resistance to sensing.

The information processing system according to one aspect of the present invention includes an acquisition unit for acquiring at least one of environmental information in the facility and equipment information indicating the state of the equipment installed in the facility, and trigger information. Based on the control unit that causes the sensor installed in the facility to start sensing by the user when the trigger information is acquired, at least one of the acquired information, and the sensing information acquired by the sensing. Further, it is provided with an output unit that outputs recommendation information for making recommendations to the user.

The information processing method according to one aspect of the present invention is an information processing method executed by a computer, and is information on at least one of environmental information in the facility and equipment information indicating the state of the equipment installed in the facility. When the trigger information is acquired, the sensor installed in the facility is started to sense the user, and at least one of the acquired information and the sensing are used. Based on the acquired sensing information, the recommendation information for making a recommendation to the user is output.

The program according to one aspect of the present invention is a program for causing a computer to execute the information processing method.

The information processing system and the like of the present invention can suppress the user from feeling resistance to sensing.

FIG. 1 is a block diagram showing a functional configuration of an information processing system according to an embodiment. FIG. 2 is a diagram showing an example of registration information. FIG. 3 is a flowchart of an operation example 1 of the information processing system according to the embodiment. FIG. 4 is a diagram showing an example of a notification screen. FIG. 5 is a diagram showing a first example of a recommended image. FIG. 6 is a diagram showing a second example of the recommended image. FIG. 7 is a diagram showing an example of information showing the correspondence relationship between the hot / cold feeling report and the recommendation information. FIG. 8 is a diagram showing a third example of the recommended image. FIG. 9 is a flowchart of operation example 2 of the information processing system according to the embodiment. FIG. 10 is a diagram showing a fourth example of the recommended image. FIG. 11 is a flowchart of an operation example 3 of the information processing system according to the embodiment. FIG. 12 is a flowchart of an operation example 4 of the information processing system according to the embodiment. FIG. 13 is a diagram showing a fifth example of the recommended image.

Hereinafter, embodiments will be specifically described with reference to the drawings. It should be noted that all of the embodiments described below are comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, the order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present invention. Further, among the components in the following embodiments, the components not described in the independent claims are described as arbitrary components.

It should be noted that each figure is a schematic view and is not necessarily exactly shown. Further, in each figure, the same reference numerals may be given to substantially the same configurations, and duplicate explanations may be omitted or simplified.

(Embodiment)
[composition]
First, the configuration of the information processing system according to the embodiment will be described. FIG. 1 is a block diagram showing a functional configuration of an information processing system according to an embodiment.

As shown in FIG. 1, the information processing system 10 includes a plurality of sensors 20, a control device 30, a plurality of devices 40, and a server device 50. The information processing system 10 is a system that can make recommendations (output of recommendation information) to users in the facility 80 based on the sensing results of a plurality of sensors 20. The recommendation is, for example, a recommendation regarding the control of the device 40, a recommendation regarding an action to be taken by the user, and the like. The facility 80 is, for example, a house such as an apartment house or a detached house. A plurality of sensors 20, a control device 30, and a plurality of devices 40 are installed in the facility 80. The server device 50 is a cloud (in other words, a cloud server).

First, a plurality of sensors 20 will be described. The plurality of sensors 20 include a temperature sensor 21, a humidity sensor 22, a fine particle concentration sensor 23, an open / close sensor 24, and a thermal image sensor 25. Although the plurality of sensors 20 are devices different from the control device 30, at least a part of the plurality of sensors 20 may be built in the control device 30. Further, at least a part of the plurality of sensors 20 may be built in a device other than the control device 30. For example, the temperature sensor 21, the humidity sensor 22, the thermal image sensor 25, and the like may be built in the air conditioner 41.

The temperature sensor 21 senses the temperature inside the facility 80 and outputs temperature information indicating the sensed temperature. The temperature sensor 21 is realized by, for example, a temperature measuring element such as a thermistor or a thermocouple.

The humidity sensor 22 senses the humidity in the facility 80 and outputs humidity information indicating the sensed humidity. The humidity sensor 22 is realized by, for example, a humidity measuring element such as a capacitance type humidity sensitive element or a resistance change type humidity sensitive element.

The fine particle concentration sensor 23 senses the concentration of fine particles (for example, pollen or PM2.5) in the facility 80, and outputs fine particle concentration information indicating the sensed fine particle concentration. The fine particle concentration sensor 23 is, for example, an optical sensor that senses the fine particle concentration based on the scattering of light emitted by a light source such as an LED. In the following embodiment, the fine particle concentration sensor 23 is described as a sensor that senses the concentration of pollen.

Each of the temperature sensor 21, the humidity sensor 22, and the fine particle concentration sensor 23 is an example of an environment sensor that senses the environment in the facility 80 and outputs environmental information indicating the sensed environment. Examples of other environmental sensors include an illuminance sensor that senses the illuminance in the facility 80, and the information processing system 10 may include these sensors.

The open / close sensor 24 senses the opening / closing of a window or door provided in the facility 80, and outputs information indicating the opened / closed state of the sensed window or door. The open / close sensor 24 is an example of a sensor that senses the state of equipment (for example, fittings such as windows or doors) installed in the facility 80. The open / close sensor 24 is attached to, for example, a magnet attached to one of the window and the window frame (or one of the door and the door frame) and the other of the window and the window frame (or the other of the door and the door frame). It is a magnetic sensor realized by a magnet sensor.

The thermal image sensor 25 senses the distribution of heat (infrared rays) in the facility 80 and outputs it as thermal image information. The thermal image sensor 25 is an example of a sensor for sensing the behavior of the user in the facility 80 (for example, where the user is located in the facility 80). The thermal image sensor 25 is realized by, for example, an infrared detection element arranged in an array, but may be realized by an image sensor capable of detecting infrared rays.

As the sensor that senses the behavior of the user in the facility 80, a radio wave sensor or the like is exemplified in addition to the thermal image sensor 25, and the information processing system 10 may include these sensors.

Next, the control device 30 will be described. The control device 30 is, for example, a HEMS (Home Energy Management System) controller having an energy management function, and is installed in the facility 80 to manage the power consumption of the device 40 installed in the facility 80. The control device 30 is a local controller installed in the facility 80 as a device separate from the device 40 (that is, not built in the device 40). Further, the control device 30 acquires and displays the state of the device 40, and controls the device 40 installed in the facility 80 (or in the premises of the facility 80). The control device 30 is not limited to the HEMS controller, and may be another home controller or a gateway device that does not have an energy management function. Specifically, the control device 30 includes a display unit 31, an information processing unit 32, a storage unit 33, a first communication unit 34, and a second communication unit 35.

The display unit 31 is a display device having an image display function and a function of accepting manual input by the user. The display unit 31 is realized by a touch panel and a display panel such as a liquid crystal panel or an organic EL panel. The touch panel is, for example, a capacitance type touch panel, but may be a resistance film type touch panel.

The information processing unit 32 controls the display of an image on the display unit 31, controls the device 40, and the like. The information processing unit 32 is realized by, for example, a microcomputer, but may be realized by a processor. Specifically, the information processing unit 32 has an acquisition unit 32a, a control unit 32b, and an output unit 32c. Specific functions of the acquisition unit 32a, the control unit 32b, and the output unit 32c will be described later.

The storage unit 33 is a storage device that stores a computer program or the like executed by the information processing unit 32. The storage unit 33 is realized by, for example, a semiconductor memory. Further, the registration information is stored in advance in the storage unit 33. FIG. 2 is a diagram showing an example of registration information.

The registration information is stored in the storage unit 33 by a plurality of users using the control device 30 inputting their own information in advance by operating the display unit 31 or the like. As shown in FIG. 2, the registration information is, for example, information in which a face image, an utterance voice feature amount, a footstep voice feature amount, a warm / cold feeling report, and the presence / absence of pollinosis are associated with a user ID. be. The face image is generated based on the image taken by the image sensor 37, for example, and the utterance voice feature amount and the footstep voice feature amount are generated based on the voice signal acquired by the voice sensor 38. ..

The first communication unit 34 is a communication module (communication circuit) for the control device 30 to communicate with a plurality of sensors 20 and a plurality of devices 40 via a local communication network. The communication performed by the first communication unit 34 is, for example, wireless communication, but may be wired communication. The communication standard used for communication is also not particularly limited. The communication between the first communication unit 34 and the device 40 conforms to, for example, ECHONET Lite (registered trademark).

The second communication unit 35 is a communication module (communication circuit) for the control device 30 to communicate with the server device 50 via a wide area communication network 70 such as the Internet. The communication performed by the second communication unit 35 is, for example, wireless communication, but may be wired communication. The communication standard used for communication is also not particularly limited.

The motion sensor 36 senses the presence or absence of a person around the control device 30 (that is, in the facility 80), and outputs information indicating the presence or absence of the person to the information processing unit 32. The motion sensor 36 is, for example, a pyroelectric sensor that senses infrared rays emitted from a human body. The motion sensor 36 may be separate from the control device 30. That is, the motion sensor 36 may be included in the plurality of sensors 20.

The image sensor 37 is a sensor that captures an image of the surroundings of the control device 30 (that is, in the facility 80), in other words, a camera. The image sensor 37 is realized by, for example, a CMOS image sensor. The image sensor 37 may be separate from the control device 30. That is, the image sensor 37 may be included in the plurality of sensors 20.

The voice sensor 38 is a sensor that senses (acquires) the voice around the control device 30 (that is, in the facility 80), in other words, it is a microphone. The voice sensor 38 may be separate from the control device 30. That is, the voice sensor 38 may be included in the plurality of sensors 20.

Next, the device 40 will be described. The device 40 is a device to be controlled by the control device 30. The device 40 includes, for example, a communication module (communication circuit) for communicating with the control device 30, and operates according to a control signal received from the control device 30. The device 40 includes, for example, an air conditioner 41, a ventilation device 42, an air purifier 43, and the like.

The air conditioner 41 is a device that takes in the air in the space inside the facility 80, adjusts at least one of the temperature and humidity, and discharges the air into the space. A heat exchanger or the like is used to adjust the temperature. The air conditioner 41 is, for example, an air conditioner for general household use, but may be an industrial air conditioner.

The ventilation device 42 is a device for ventilating the space in the facility 80. The ventilation device 42 is, for example, an energy recovery ventilator (ERV), but may be a ventilation device such as a ventilation fan that does not involve heat exchange. Further, the ventilation device 42 may be a window opening / closing device provided in the facility 80. In the following embodiment, the ventilation device 42 is described as a ventilation device that exhausts air from the inside of the facility 80 to the outside of the facility 80, but is a ventilation device that supplies air from the outside of the facility 80 to the inside of the facility 80. You may.

The air purifier 43 is a device that collects the air in the space inside the facility 80, filters the collected air with an air filter such as a HEPA (High Effective Particulate Air) filter, and discharges the collected air into the space. The air purifier 43 may be a device that collects the air in the space inside the facility 80, sterilizes the collected air with hypochlorous acid, and discharges the collected air into the space.

Next, the server device 50 will be described. The server device 50 is a cloud server that manages environmental information (temperature information, humidity information, fine particle concentration information, weather information (weather information), etc.) outside the facility 80 and provides it to the control device 30.

[Operation example 1]
Next, operation example 1 of the information processing system 10 will be described. FIG. 3 is a flowchart of an operation example 1 of the information processing system 10.

When the human sensor 36 of the control device 30 detects a person around the control device 30 (S11), information indicating the detection result (that is, information indicating that a person exists around the control device 30 in the facility 80). ) Is output to the information processing unit 32 (acquisition unit 32a).

The acquisition unit 32a acquires this information as trigger information (S12). The output unit 32c outputs the notification information to the display unit 31 when the trigger information is acquired. As a result, the display unit 31 displays a notification screen notifying that the user identification process (such as shooting of the user by the image sensor 37 described later) is started (S13). FIG. 4 is a diagram showing an example of a notification screen.

As shown in FIG. 4, on the notification screen, the avatar notifies in a conversational manner that the user identification process is started. With such a notification screen, it is possible to prevent the user from feeling resistance to the start of the user identification process. As shown in FIG. 4, the notification screen may include an operation object for the user to approve the start of the user identification process, and the process after step S13 is approved by the user. It may be done only when.

Next, the control unit 32b activates the image sensor 37 and causes the image sensor 37 to start shooting by the user (S14). That is, before the trigger information is acquired, the image sensor 37 is in the off state (or sleep state) and does not capture an image around the control device 30. As described above, according to the configuration in which the image sensor 37 is activated by the trigger information, the user resists shooting as compared with the configuration in which the image around the control device 30 is constantly captured by the image sensor 37. It is possible to suppress the feeling of.

Next, the acquisition unit 32a acquires the user's face image information from the user's image taken by the image sensor 37 (S15). The output unit 32c identifies the user by collating the acquired face image information with a plurality of face image information included in the registration information (FIG. 2) stored in the storage unit 33 (S16).

Next, the acquisition unit 32a acquires the environmental information in the facility 80 from the environmental sensor among the plurality of sensors 20 via the first communication unit 34 (S17), and the output unit 32c uses the acquired environmental information. Based on this, it is determined whether or not a recommendation to the user is necessary (S18).

For example, the output unit 32c determines whether or not the temperature indicated by the temperature information acquired from the temperature sensor 21 is within a predetermined range. When the temperature indicated by the temperature information acquired from the temperature sensor 21 is within a predetermined range, the output unit 32c determines that the recommendation regarding the temperature adjustment is unnecessary (No in S18), and does not perform the process of step S19.

On the other hand, when the temperature indicated by the temperature information acquired from the temperature sensor 21 is out of the predetermined range, the output unit 32c determines that a recommendation regarding temperature adjustment is necessary (Yes in S18), and identifies the user in step S16. Based on the result, the recommendation information is output to the display unit 31. As a result, the recommended image is displayed on the display unit 31 (S19). For example, when the user is identified as the user A in step S16, the warm / cold feeling report of the user A is "normal" in the registration information, so that the display unit 31 is air-conditioned with the set temperature as a specified value. A recommendation image that recommends operating the machine 41 is displayed. FIG. 5 is a diagram showing a first example of a recommended image. On the other hand, when the user is identified as the user B in step S16, since the hot / cold feeling report of the user B is "hot" in the registration information, the set temperature is set lower than the specified value on the display unit 31. A recommendation image that recommends operating the air conditioner 41 is displayed. FIG. 6 is a diagram showing a second example of the recommended image. In this way, the output unit 32c makes the content of the recommended image different according to the identification result of the user.

In the storage unit 33, the hot / cold feeling report and the recommendation information (recommended image) to the user having the hot / cold feeling report are stored in association with each other. FIG. 7 is a diagram showing an example of information indicating such a correspondence relationship (hereinafter, also referred to as correspondence information). The output unit 32c can output the recommendation information as described above based on the user identification result, the registration information, and the information indicating the correspondence relationship.

Further, as shown in FIGS. 5 and 6, the recommended image may include an operation object for the user to execute the control of the device 40 proposed by the recommended image. When an operation is performed on such an operation object, the control unit 32b controls the device 40 (air conditioner 41 in the operation example 1) by causing the first communication unit 34 to transmit a control signal to the device 40. ..

As described above, the information processing system 10 causes the image sensor 37 installed in the facility 80 to start sensing (shooting) by the user when the trigger information is acquired. Further, the information processing system 10 recommends information to the user based on the environmental information in the facility 80 and the user's identification information (specifically, face image information) acquired by the sensing. Is output. Such an information processing system 10 can suppress the user from feeling resistance to sensing (shooting).

In the above operation example 1, the content of the recommendation information (recommendation image) is changed according to the identification result of the user, but the criterion for determining whether or not the recommendation is necessary is changed according to the identification result of the user. You may. Hereinafter, a specific example of such an operation will be described.

For example, in step S16, when the user is identified as the user A, the registration information stipulates that the user A is not pollinosis. Therefore, in step S17, the output unit 32c determines whether or not the fine particle concentration (= regarded as the pollen concentration) acquired from the fine particle concentration sensor 23 is equal to or higher than the first predetermined value. When the fine particle concentration indicated by the fine particle concentration information acquired from the fine particle concentration sensor 23 is less than the first predetermined value, the output unit 32c determines that the recommendation regarding the air quality in the facility 80 is unnecessary (No in S18). The process of step S19 is not performed.

When the fine particle concentration indicated by the fine particle concentration information acquired from the fine particle concentration sensor 23 is equal to or higher than the first predetermined value, the output unit 32c determines that a recommendation regarding the air quality in the facility 80 is necessary (Yes in S18). The recommendation information is output to the display unit 31. As a result, the recommended image is displayed on the display unit 31 (S19). FIG. 8 is a diagram showing a third example of the recommended image, and in the recommended image, it is recommended to turn on the ventilation device 42 and the air purifier 43.

On the other hand, in step S16, when the user is identified as the user B, the registration information stipulates that the user B has hay fever. Therefore, in step S17, the output unit 32c determines whether or not the fine particle concentration acquired from the fine particle concentration sensor 23 is equal to or higher than the second predetermined value smaller than the first predetermined value. That is, the recommended image of FIG. 8 is more likely to be displayed when the user is identified as the user B who has hay fever than when the user is identified as the user A who does not have hay fever in step S16. ..

[Operation example 2]
In the operation example 1, the environmental information in the facility 80 is acquired, but instead of the environmental information in the facility 80, the equipment information indicating the state of the equipment installed in the facility 80 may be acquired. Further, in addition to the environmental information inside the facility 80, the environmental information outside the facility 80 may be further acquired. Hereinafter, operation example 2 of such an information processing system 10 will be described. FIG. 9 is a flowchart of the operation example 2 of the information processing system 10.

The processing of steps S11 to S16 is the same as that of the operation example 1. After step S16, the acquisition unit 32a acquires equipment status information indicating the equipment status installed in the facility 80 from the equipment sensor among the plurality of sensors 20 via the first communication unit 34 (S21). The unit 32a acquires environmental information outside the facility 80 from the server device via the second communication unit 35 (S22).

Then, the output unit 32c determines whether or not a recommendation to the user is necessary based on the acquired equipment state information and the acquired environmental information outside the facility 80 (S23).

For example, in the output unit 32c, the concentration of fine particles outside the facility 80 acquired from the server device 50 is higher than a predetermined value (that is, the air quality outside the facility 880 is poor), and the user identified in step S16. It is determined whether or not the information acquired from the opening / closing sensor 24 of the window of the room satisfies the requirement that the window is open. The window open / close sensor 24 is installed in each user's room in the facility 80, and the control device 30 grasps which open / close sensor 24 is installed in which user's room. And.

When the above requirement is not satisfied, the output unit 32c determines that the recommendation regarding opening / closing of the window is unnecessary (No in S23), and does not perform the process of step S19.

On the other hand, when the above requirement is satisfied, the output unit 32c determines that a recommendation regarding opening / closing of the window is necessary (Yes in S23), and outputs the recommendation information to the display unit 31. As a result, the recommended image is displayed on the display unit 31 (S24). FIG. 10 is a diagram showing a fourth example of the recommended image.

As described above, the information processing system 10 makes recommendations to the user based on the equipment status information of the facility 80, the environmental information outside the facility 80, and the user's identification information (specifically, facial image information). It can be carried out.

In the operation example 2, the environmental information in the facility 80 may be further acquired. For example, in the determination in step S23, it is required that the concentration of fine particles outside the facility 80 acquired from the server device 50 is higher than a predetermined value, but the concentration of fine particles outside the facility 80 is higher than the concentration of fine particles inside the facility 80. (That is, the air quality outside the facility 80 is worse than that inside the facility 80) may be a requirement.

[Operation example 3]
In the operation example 1 and the operation example 2, the face image information is acquired as the identification information for identifying the user, but it is not essential to identify the user. For example, the information processing system 10 may estimate the emotion of an arbitrary user from the face image information regardless of who the user is, and display the recommended image based on the estimation result. FIG. 11 is a flowchart of an operation example 3 of the information processing system 10.

The processing of steps S11 to S15 is the same as that of the operation example 1. After step S15, the output unit 32c estimates the user's emotion based on the acquired facial image information (S31). The face image information in this case is an example of emotional information indicating the user's emotions. As a specific method for estimating the user's emotion based on the face image information, any existing method may be used.

The output unit 32c determines whether or not the user's emotion is an emotion suitable for recommendation (S32). For example, in step S31, when the user is presumed to be angry or the user is presumed to be sad, the output unit 32c does not indicate that the user's emotion is suitable for the recommendation. The determination is made (No in S32), and the processing after step S33 is not performed. On the other hand, in step S31, when the user is presumed to be relaxed, the output unit 32c determines that the user's emotion is suitable for the recommendation (Yes in S32), and steps S33 and subsequent steps are performed. Processing is done. The acquisition unit 32a acquires the environmental information in the facility 80 from the environmental sensor among the plurality of sensors 20 via the first communication unit 34 (S33), and the output unit 32c obtains the environmental information in the facility 80 based on the acquired environmental information. It is determined whether or not a recommendation to the user is necessary (S34).

For example, the output unit 32c determines whether or not the temperature indicated by the temperature information acquired from the temperature sensor 21 is within a predetermined range. When the temperature indicated by the temperature information acquired from the temperature sensor 21 is within a predetermined range, the output unit 32c determines that the recommendation regarding the temperature adjustment is unnecessary (No in S34), and does not perform the process of step S35.

On the other hand, when the temperature indicated by the temperature information acquired from the temperature sensor 21 is out of the predetermined range, the output unit 32c determines that a recommendation regarding temperature adjustment is necessary (Yes in S34), and the recommendation information is displayed on the display unit 31. Is output. As a result, the recommended image (for example, the recommended image of FIG. 5) is displayed on the display unit 31 (S35).

As described above, the information processing system 10 makes recommendations to the user based on the environmental information in the facility 80 and the user's emotional information (specifically, facial image information) acquired by the sensing. be able to.

In Operation Example 3, it is determined whether or not to perform the recommendation based on the estimation result of the user's emotion, but the content of the recommendation image may be changed based on the estimation result of the user's emotion. For example, the message content in the recommendation image (for example, the tone of the avatar) may be changed based on the estimation result of the user's emotion, or the avatar in the recommendation image may be changed based on the estimation result of the user's emotion. You may.

Further, in the operation example 3, the user's emotion is estimated instead of the user's identification, but the user's emotion may be estimated in addition to the user's identification.

[Operation example 4]
In the operation example 1 and the operation example 2, the face image information is acquired as the identification information for identifying the user, but it is not essential to identify the user. For example, the information processing system 10 may acquire thermal image information output by a thermal image sensor for sensing the user's behavior and display a recommended image based on the acquired thermal image information. FIG. 12 is a flowchart of an operation example 4 of the information processing system 10.

The processing of steps S11 to S13 is the same as that of the operation example 1. After step S13, the control unit 32b activates the thermal image sensor 25 by causing the first communication unit 34 to transmit a control signal to the thermal image sensor 25, and causes the thermal image sensor 25 to start sensing by the user (S41). .. That is, before the trigger information is acquired, the thermal image sensor 25 is in the off state (or sleep state) and does not sense the user. In this way, according to the configuration in which the thermal image sensor 25 is activated by the trigger information, the user feels resistance to sensing as compared with the configuration in which the inside of the facility 80 is constantly sensed by the thermal image sensor 25. It is possible to prevent the image from being lost.

Next, the acquisition unit 32a acquires the thermal image information output by the thermal image sensor 25 (S42). The output unit 32c estimates the user's body temperature based on the acquired thermal image information (S43). The output unit 32c estimates, for example, a region in which the user's head is reflected in the thermal image by performing pattern matching processing on the thermal image information, and estimates that the temperature in this region is the user's body temperature.

Further, the acquisition unit 32a acquires the environmental information in the facility 80 from the environment sensor among the plurality of sensors 20 via the first communication unit 34 (S44). The output unit 32c determines whether or not a recommendation to the user is necessary based on the estimated body temperature of the user and the acquired environmental information (S45).

In the output unit 32c, for example, the temperature and humidity in the facility 80 determined by the acquired environmental information when the user's body temperature is equal to or higher than a predetermined value (that is, the user is feverish) increases the virus survival rate. It is determined whether or not the requirement of belonging to a predetermined range (low temperature and low humidity) is satisfied.

When the above requirement is not satisfied, the output unit 32c determines that the recommendation is unnecessary (No in S45), and does not perform the process of step S19.

On the other hand, when the above requirements are satisfied, the output unit 32c determines that the user needs a recommendation because it is estimated that the user has a cold or influenza (Yes in S45), and the display unit 31 displays the recommendation information. Is output. As a result, the recommended image is displayed on the display unit 31 (S46). FIG. 13 is a diagram showing a fifth example of the recommended image.

As described above, the information processing system 10 makes recommendations to the user based on the environmental information in the facility 80 and the user's behavior information (specifically, thermal image information) acquired by the above sensing. be able to.

[Transformation example of trigger information]
In the above operation examples 1 to 4, the information indicating that a person exists in the facility 80 output by the motion sensor 36 was used as the trigger information. However, the trigger information is not limited to such information.

For example, the control device 30 may support voice recognition, and the voice recognition function of the control device 30 may be turned on by the user speaking a trigger word (in other words, an activation word or a wakeup word). Be done. In such a case, the voice information indicating the trigger word, which is output by the voice sensor 38 by sensing the trigger word, may be used as the trigger information.

Further, for example, the information transmitted from the display unit 31 to the information processing unit 32 when a predetermined operation is performed on the display unit 31 by the user may be used as the trigger information.

[Variation example of user identification]
In the above operation examples 1 and 2, face image information was used as the identification information for identifying the user. However, the user identification method is not limited to the method based on the face image information. For example, the user identification may be performed based on the voice information of the user's utterance sound acquired by the voice sensor 38. That is, voice information may be used as identification information. As shown in FIG. 2, if the voice feature amount of each utterance sound of a plurality of users is registered in advance in the registration information, the output unit 32c is obtained from the voice information of the utterance sound output by the voice sensor 38. The user can be identified by collating the voice feature amount of the utterance sound to be produced with the voice feature amount of the utterance sound included in the registration information.

Further, the user identification may be performed based on the voice information of the user's footsteps acquired by the voice sensor 38. As shown in FIG. 2, if the voice features of the footsteps of the plurality of users are registered in advance in the registration information, the output unit 32c is an utterance obtained from the voice information of the footsteps output by the voice sensor 38. The user can be identified by collating the voice feature amount of the sound with the voice feature amount of the footstep included in the registration information.

In this way, when the user is identified by using the voice sensor 38, the control unit 32b activates the voice sensor 38 when the trigger information is acquired, and the voice sensor 38 acquires the user's voice. To start. That is, before the trigger information is acquired, the voice sensor 38 is in the off state (or sleep state) and does not acquire the voice. In this way, according to the configuration in which the voice sensor 38 is activated by the trigger information, the user feels resistance to the acquisition of the voice as compared with the configuration in which the voice is constantly acquired by the voice sensor 38. It can be suppressed.

[Modified example of user's emotion estimation]
In the above operation example 3, the user's emotion is estimated based on the facial image information. However, the method of estimating the user's emotion is not limited to the method based on the facial image information. For example, the estimation of the user's emotion may be performed based on the user's utterance sound acquired by the voice sensor 38. For example, the user's emotion may be estimated based on the user's utterance content specified by voice recognition, or the user's emotion may be estimated based on the voice feature amount of the utterance sound.

[Modification example of output method of recommendation information]
In the operation example 1 or the like, the output unit 32c is a reference in which registration information (FIG. 2) and correspondence information (FIG. 7) are combined (that is, identification information of a plurality of users and recommendation information are associated with each other). The recommendation information was output based on the information). However, it is not essential that the reference information be used to output the recommendation information.

For example, the output unit 32c may output the recommendation information using a machine learning model. The machine learning model includes user identification information, recommended images presented to the user (recommended information), information output by a plurality of sensors 20 when the recommended images are presented (environmental information, equipment status information, etc.), It is constructed by learning in advance data associated with the result of presenting the recommended image (for example, whether or not the user approves the control of the recommended device 40). The output unit 32c can also output the recommendation information using such a machine learning model. The machine learning model may be updated (re-learned) as appropriate.

[Effects, etc.]
As described above, the information processing system 10 acquires at least one of the environmental information in the facility 80 and the equipment information indicating the state of the equipment installed in the facility 80, and the trigger information 32a. And, when the trigger information is acquired, the control unit 32b that causes the sensor installed in the facility 80 to start sensing by the user, at least one of the acquired information, and the sensing information acquired by sensing. Based on this, it is provided with an output unit 32c that outputs recommendation information for making recommendations to the user.

In such an information processing system 10, by limiting the time and performing sensing, it is possible to suppress the user from feeling resistance to sensing. Further, the configuration in which the sensing is performed with the time limited in this way suppresses the power consumption and the amount of information processing as compared with the configuration in which the sensing is constantly performed.

Further, for example, the trigger information is information indicating that a person exists in the facility 80 output by the motion sensor 36, or voice information indicating a predetermined trigger word output by the voice sensor 38.

Such an information processing system 10 can start sensing by the user when the motion sensor 36 reacts (detects a person) or when the user utters a trigger word.

Further, for example, the sensing information is identification information for distinguishing a user from other users.

Such an information processing system 10 can make recommendations according to who the user is.

Further, for example, the sensor is an image sensor 37, and the identification information is face image information sensed by the image sensor 37.

Such an information processing system can identify a user by using facial image information.

Further, for example, the sensor is a voice sensor 38, and the identification information is voice information sensed by the voice sensor 38.

Such an information processing system can identify a user using voice information.

Further, for example, the information processing system 10 further includes a storage unit 33 in which reference information in which identification information and recommendation information of a plurality of users are linked is stored. The output unit 32c outputs the recommendation information based on the at least one of the information acquired by the acquisition unit 32a, the sensing information, and the reference information.

Such an information processing system 10 can output recommendation information using reference information stored in the storage unit 33.

Further, for example, the sensing information is emotional information indicating the emotion of the user.

Such an information processing system 10 can make recommendations according to the emotions of the user.

Further, for example, the sensor is an image sensor, and the emotion information is face image information sensed by the image sensor 37.

Such an information processing system 10 can estimate a user's emotion using facial image information.

Further, for example, the sensor is a sensor that senses the user's behavior. Such an information processing system 10 can make recommendations according to the user's behavior.

Further, for example, the sensor is a thermal image sensor and outputs thermal image information as the sensing information.

Such an information processing system 10 can make recommendations according to thermal image information.

Further, for example, the acquisition unit 32a further acquires the environmental information outside the facility 80, and the control unit 32b further uses the acquired environmental information outside the facility 80 to output the recommendation information.

Such an information processing system 10 can make recommendations according to environmental information outside the facility 80.

Further, for example, the information processing system 10 further includes a display unit 31 that displays a notification screen for notifying that the user's sensing is started when the trigger information is acquired. On the notification screen, the notification is performed in a conversation format with the avatar displayed on the notification screen.

Such an information processing system 10 can prevent the user from feeling resistance to sensing by performing notification in a conversational format with the avatar displayed on the notification screen.

Further, the information processing method executed by a computer such as the information processing system 10 acquires at least one of the environmental information in the facility 80 and the equipment information indicating the state of the equipment installed in the facility 80, and triggers the information. When the information is acquired and the trigger information is acquired, the sensor installed in the facility 80 is started to sense the user, and the acquired at least one of the information and the sensing information acquired by the sensing are used. Based on this, the recommendation information for making a recommendation to the user is output.

In such an information processing method, it is possible to suppress the user from feeling resistance to the sensing by performing the sensing for a limited time. Further, the configuration in which the sensing is performed with the time limited in this way suppresses the power consumption and the amount of information processing as compared with the configuration in which the sensing is constantly performed.

(Other embodiments)
Although the embodiments have been described above, the present invention is not limited to the above embodiments.

For example, the operation examples 1 to 4 described in the above embodiment may be arbitrarily combined. For example, the estimation of the user's emotion in the operation example 3 and the determination of the suitability of the recommendation may be combined with any one of the operation example 1, the operation example 2, and the operation example 4.

For example, in the above embodiment, the facility is described as a house such as an apartment house or a detached house, but it may be a nursing care facility or an office, and the facility is specifically any kind of facility. Is not particularly limited.

Further, in the above embodiment, the information processing system is realized by a plurality of devices, but may be realized as a single device. For example, the information processing system may be realized as a single device corresponding to a control device. When the information processing system is realized by a plurality of devices, the components included in the information processing system may be distributed to the plurality of devices in any way.

For example, the communication method between the devices in the above embodiment is not particularly limited. Further, in the communication between the devices, a relay device (including a broadband router) (not shown) may intervene.

Further, in the above embodiment, another processing unit may execute the processing executed by the specific processing unit. Further, the order of the plurality of processes may be changed, or the plurality of processes may be executed in parallel. Further, in the above embodiment, the operation examples may be arbitrarily combined.

Further, in the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

In addition, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form one circuit as a whole, or may be separate circuits from each other. Further, each of these circuits may be a general-purpose circuit or a dedicated circuit.

In addition, general or specific embodiments of the present invention may be realized in a recording medium such as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. Further, it may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program and a recording medium.

For example, the present invention may be realized as an information processing method executed by a computer such as an information processing system, or may be realized as a program for causing a computer to execute such an information processing method. Further, the present invention may be realized as a computer-readable non-temporary recording medium in which such a program is recorded.

In addition, it is realized by a form obtained by applying various modifications to each embodiment that a person skilled in the art can think of, or by arbitrarily combining the components and functions in each embodiment within the range not deviating from the gist of the present invention. Also included in the present invention.

10 Information processing system 25 Thermal image sensor (sensor)
31 Display unit 32a Acquisition unit 32b Control unit 32c Output unit 33 Storage unit 36 Motion sensor 37 Image sensor (sensor)
38 Voice sensor (sensor)

Claims (14)

An acquisition unit that acquires at least one of the environmental information in the facility and the equipment information indicating the state of the equipment installed in the facility, and the trigger information.
A control unit that causes a sensor installed in the facility to start sensing by the user when the trigger information is acquired.
An information processing system including at least one of the acquired information and an output unit that outputs recommendation information for making recommendations to the user based on the sensing information acquired by the sensing. The information according to claim 1, wherein the trigger information is information output by a motion sensor indicating the presence of a person in the facility, or information indicating a predetermined trigger word output by a voice sensor. Processing system. The information processing system according to claim 1 or 2, wherein the sensing information is identification information for distinguishing the user from other users. The sensor is an image sensor and
The information processing system according to claim 3, wherein the identification information is face image information sensed by the image sensor. The sensor is a voice sensor and
The information processing system according to claim 3, wherein the identification information is voice information sensed by the voice sensor. Further, it is provided with a storage unit that stores reference information in which identification information and recommendation information of a plurality of users are linked.
The output unit is described in any one of claims 3 to 5, which outputs the recommendation information based on the at least one information acquired by the acquisition unit, the sensing information, and the reference information. Information processing system. The information processing system according to claim 1 or 2, wherein the sensing information is emotion information indicating the emotion of the user. The sensor is an image sensor and
The information processing system according to claim 7, wherein the emotion information is face image information sensed by the image sensor. The information processing system according to claim 1 or 2, wherein the sensor is a sensor that senses the behavior of the user. The information processing system according to claim 9, wherein the sensor is a thermal image sensor and outputs thermal image information as the sensing information. The acquisition unit further acquires environmental information outside the facility.
The information processing system according to any one of claims 1 to 10, wherein the control unit further uses the acquired environmental information outside the facility to output the recommendation information. Further, it is provided with a display unit for displaying a notification screen for notifying that the sensing of the user is started when the trigger information is acquired.
The information processing system according to any one of claims 1 to 11, wherein the notification is performed in the form of conversation with the avatar displayed on the notification screen. An information processing method executed by a computer
Acquire at least one of the environmental information in the facility and the equipment information indicating the state of the equipment installed in the facility.
Get the trigger information,
When the trigger information is acquired, the sensor installed in the facility is started to sense the user.
An information processing method that outputs recommendation information for making recommendations to the user based on the acquired at least one of the information and the sensing information acquired by the sensing. A program for causing a computer to execute the information processing method according to claim 13.

Images