JPWO2019130977A1 - Cleanup support system and program - Google Patents

Abstract

The cleanup support system (10) includes an information acquisition unit (110) that acquires in-house status data indicating the result of sensing the in-house condition, and a clutter status recognition unit (120) that recognizes the in-house clutter status based on the in-home status data. ), The user behavior recognition unit (130) that recognizes the behavior of the user in the house based on the in-house situation data, and the user is presented to the user according to a predetermined rule regarding cleaning up in the house based on the clutter situation and the user's behavior. A cleanup support signal for outputting the determined cleanup support information as at least one of voice information and visual information is output to the presentation information determination unit (140) that determines the cleanup support information and the information presenter in the house. It includes an output unit (150).

Description

The present invention relates to a technique for presenting information on tidying up to assist the user in tidying up.

A device has been proposed that outputs data that supports the user's language learning from the recognition result of the user's behavior by the signal of the image taken by the camera (see Patent Document 1).

JP-A-2002-207413

In this way, there is a technique for presenting information that should be considered to be useful to the user by changing it according to the recognition result of the user's behavior.

However, the information is not really useful to the user unless the user is willing or motivated to use the information.

Therefore, the present disclosure provides a system for supporting user behavior, especially personal cleanup behavior, by presenting the above information to the user and motivating the user to take an action based on the information. The purpose.

In order to achieve the above object, the cleanup support system according to one aspect of the present disclosure includes an information acquisition unit that acquires in-home situation data indicating the sensing result from the sensor that senses the in-home situation, and the in-home situation data. Based on the clutter situation recognition unit that recognizes the clutter situation in the house, the user action recognition unit that recognizes the behavior of the user in the house based on the home situation data, and the clutter situation and the user's behavior. Then, according to a predetermined rule regarding the tidying up in the house, the tidying up support information is transmitted to the presentation information determination unit for determining the tidying up support information presented to the user and the information presenter in the house for voice information and visual information. It is provided with an output unit that outputs a cleanup support signal for output as at least one.

Further, in the cleanup support program according to one aspect of the present disclosure, the processor provided in the computer is made to acquire the in-house situation data indicating the sensing result from the sensor that senses the in-house situation, and the in-house situation data is used as the basis for the in-house situation data. The clutter situation is recognized, the behavior of the user in the house is recognized based on the in-house situation data, and the user is presented to the user according to a predetermined rule regarding the tidying up in the house based on the clutter situation and the user's behavior. The information presenter that determines the cleanup support information to be performed and presents at least one of the voice information and the visual information to the user in the house outputs at least one of the voice information and the visual information representing the content of the cleanup support information. To output a cleanup support signal for

According to one aspect of the present invention, it is provided a system or the like that supports a user's tidying up behavior by presenting information indicating a tidying up target to the user and motivating the user to take a tidying up behavior based on the information. With the goal.

FIG. 1 is a block diagram showing a functional configuration example of the cleanup support system according to the first embodiment. FIG. 2 is a block diagram for explaining a hardware configuration example of the cleanup support system according to the first embodiment. FIG. 3 is a schematic diagram showing an example of in-home situation data acquired by the cleanup support system according to the first embodiment. FIG. 4 is a diagram for explaining the recognition of the clutter situation from the home situation data executed in the cleanup support system according to the first embodiment. FIG. 5 is a diagram for explaining recognition by estimation of behavior using a convolutional neural network, which is executed in the cleanup support system according to the first embodiment. FIG. 6 is a diagram showing an example of a data structure of a cleanup rule used in the cleanup support system according to the first embodiment. FIG. 7 is a sequence diagram for explaining the operation of the cleanup support system according to the first embodiment. FIG. 8 is a block diagram showing a functional configuration example of the cleanup support system according to the second embodiment. FIG. 9 is a flow chart showing the operation of the user action recognition unit in the cleanup support system according to the second embodiment. FIG. 10 is a block diagram showing a functional configuration example of the cleanup support system according to the third embodiment. FIG. 11 is a block diagram showing a functional configuration example of the cleanup support system according to the fourth embodiment. FIG. 12 is a block diagram showing a functional configuration example of the cleanup support system according to the modified example. FIG. 13 is a sequence diagram for explaining the operation of the cleanup support system according to the modified example. FIG. 14A is a block diagram showing another example of the functional configuration of the cleanup support system according to the modified example. FIG. 14B is a block diagram showing another example of the functional configuration of the cleanup support system according to the modified example. FIG. 14C is a block diagram showing another example of the functional configuration of the cleanup support system according to the modified example. FIG. 15 is a block diagram showing a functional configuration example of a cleanup support system according to one of the other embodiments, each of which includes a plurality of robots that function as information presenters. FIG. 16 is a functional block diagram showing a functional configuration example of an information presenter which is a robot.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments and modifications thereof described below are all specific examples of the present invention. Therefore, the numerical values, shapes, materials, components, arrangement positions of the components, connection forms, etc. shown in the following embodiments and modifications thereof are merely examples, and are not intended to limit the present invention. Therefore, among the components in the following embodiments and modifications thereof, the components not described in the independent claims are described as arbitrary components.

It should be noted that each figure is a schematic view and is not necessarily exactly illustrated. Further, in each figure, the same reference numerals are given to substantially the same configurations, and duplicate description will be omitted or simplified.

(Embodiment 1)
[Overview]
In the tidying up behavior support system according to the present embodiment, the cluttered situation in the house and the behavior of the user in the house are recognized from the data of the situation in the house detected by using the sensor. Then, according to the recognized clutter situation and the user's behavior, and the predetermined rule regarding the tidying up, the content of the information presented to the user and the timing of the presentation to support the tidying up in the house are determined and presented to the user. Will be done.

[Constitution]
FIG. 1 is a block diagram showing a functional configuration example of the cleanup support system 10 according to the present embodiment. Further, FIG. 2 is a block diagram for explaining a hardware configuration example of the cleanup support system according to the first embodiment.

The cleanup support system 10 includes a situation detection unit 200, an information acquisition unit 110, a clutter situation recognition unit 120, a user behavior recognition unit 130, a presentation information determination unit 140, an output unit 150, a storage unit 160, and an information presenter 300.

The situation detection unit 200 collects data used for recognizing the clutter situation in the house and the behavior of the user in the house. Such data (hereinafter referred to as home situation data) is the result of sensing by a sensor that senses the situation in the house, and the situation detection unit 200 uses a device equipped with this sensor, for example, a camera 200a as shown in FIG. Realized using. In the case of the camera 200a, the sensor is specifically an image sensor. Other examples of sensors used to realize the situation detection unit 200 include a distance image sensor and a Doppler sensor. These sensors may be used in combination. Further, the camera 200a is a stereo camera including a plurality of image sensors, and the situation detection unit 200 may be realized by using the stereo camera.

The information acquisition unit 110 receives the input of the signal output by the situation detection unit 200, and acquires the in-house situation data indicating the result of sensing by the situation detection unit 200. The in-house situation data is, for example, an image taken by an image sensor, a distance image showing a distance from the sensor to an object or a person in the house, or the like. FIG. 3 is a schematic view of an image taken by an image sensor, which is an example of such in-house situation data. This image is one frame image included in a still image or a moving image of a room in a house taken from one direction.

The clutter situation recognition unit 120 recognizes the clutter situation in the house based on the in-house situation data acquired by the information acquisition unit 110. FIG. 4 is a diagram for explaining the recognition of the clutter situation from the home situation data by the clutter situation recognition unit 120. In this description, the captured image shown in FIG. 3 and the depth (distance from the sensor or camera to each subject) information (not shown) of the captured image are used as examples of home situation data.

The clutter situational awareness unit 120 divides the captured image shown in FIG. 3 into a plurality of sub-regions as shown in FIG. 4A. Then, each sub-region is Fourier transformed to obtain a frequency image for each sub-region as shown in FIG. 4B. The arrow in FIG. 4 shows an example of the correspondence between the sub region and its frequency image. However, there is also a one-to-one correspondence between the other six sub-regions not connected by the arrows and their frequency images. In the example of FIG. 4, the captured image is divided into nine sub-regions, but the number of sub-regions is not limited to nine.

Subsequently, the clutter situational awareness unit 120 obtains the product of the sum of the high frequency component intensities of the frequency image for each sub region and the average depth. This is because when there is a difference in perspective in the two sub-regions of the same size, the high-frequency component intensity is higher in the sub-region that captures the distant range, even if the degree of scattering is the same. .. The difference in the ease of producing high-frequency components due to such a difference in distance is corrected by multiplying the average depth of each sub-region.

Further, the clutter situational awareness unit 120 sums the products obtained for each sub-region. The value obtained in this way is used in the cleanup support system 10 as a value indicating the degree of clutter in the entire space reflected in the captured image. This value is output to the presentation information determination unit 140 as a recognition result of the clutter situation. The product obtained for each sub-region may be used as a value indicating the degree of clutter in the range reflected in each sub-region.

The user behavior recognition unit 130 recognizes the behavior of the user in the house based on the in-house situation data acquired by the information acquisition unit 110. Recognition of user behavior is performed, for example, by using a convolutional neural network technique. FIG. 5 is a diagram for explaining recognition by estimating behavior using a convolutional neural network.

The convolutional neural network used in the user behavior recognition unit 130 shown in FIG. 5 (b) is a pre-learned model for behavior recognition from an image. The input data to the convolutional neural network shown in FIG. 5A is, for example, the red brightness value, the green brightness value, and the blue brightness value, that is, the RGB value of the captured image which is the home situation data. The output data shown in c) includes the user's behavior, which is the recognition result by the convolutional neural network, and its probability. In the example of FIG. 5, laundry drying is estimated as the most probable behavior, and the output data includes the estimated recognition results following cleaning and cooking. The recognition result of the user's behavior is output to the presentation information determination unit 140.

The presentation information determination unit 140 provides cleanup support presented to the user based on the recognition result of the clutter situation received from the clutter situation recognition unit 120 and the recognition result of the user's behavior received from the user behavior recognition unit 130. Determine the information. For this determination, the presentation information determination unit 140 further uses a predetermined rule. The cleanup rule held in the storage unit 160 shown in FIG. 1 is an example of this predetermined rule in the present embodiment. Further, FIG. 6 is a diagram showing an example of the data structure of the cleanup rule.

In the example of FIG. 6, the contents of each row of the table correspond to one cleanup rule. Each cleanup rule consists of clutter, user behavior, and cleanup support information associated with each other.

The presentation information determination unit 140 searches for a line containing the most probable result of the recognition result of the clutter situation input from the clutter situation recognition unit 120 and the recognition result of the user's behavior input from the user behavior recognition unit 130. .. Then, when such a line is found, the user behavior recognition unit 130 determines the content of the cleanup support information in the line as the cleanup support information presented to the user (application of the cleanup rule). The content of the cleanup support information may include a message recommending cleanup according to the recognized user's behavior, as shown in FIG. 6, for example. This message describes the effect of performing or not cleaning up on the user's own behavior or mood. Receiving such a message motivates the user to clean up.

In addition, there may be priorities applied between such predetermined rules. For example, the order of each row in the table as shown in FIG. 6 may be in order of priority, or the table may further include a column indicating the priority of the cleanup rule in each row.

The presented information determination unit 140 notifies the output unit 150 of the determined cleanup support information.

The output unit 150 transmits a signal to the information presenter 300 for outputting the notified cleanup support information to the information presenter 300 as at least one of audio information and visual information. The information presenter 300 is a device capable of presenting voice information or visual information to the user in the house where the situation detection unit 200 collects in-house situation data, that is, a device capable of outputting voice or displaying characters or video. This is realized, for example, in a speaker, a display, or an information processing terminal including these. Further, the tidying up support information may be visually presented to the user by changing the lighting mode of the lighting fixture in the house. In this case, the information presenter 300 is also realized by the lighting fixture (for example, the lighting fixture 300a in FIG. 2). obtain.

The signal transmitted to the information presenter 300 is, for example, a signal indicating voice data (not shown) stored in the storage unit 160 to be presented to the user, and the output unit 150 presents the voice information. Is sent to a device that can. Alternatively, when the information presenter 300 holds such voice data, it is a signal for identifying the voice data and causing the information presenter 300 to reproduce the voice data.

For the presentation of specific cleanup support information by voice, even if the user in the house does not see or cannot see the screen of the information presenter 300 illustrated above, the user is cleaned up. It has the beneficial effect of being able to give a chance.

Further, in order for the information presenter 300 to display an image such as characters or figures, a signal indicating what is presented to the user among the image data stored in the storage unit 160 is displayed from the output unit 150. Is sent to a device that can. Alternatively, when the information presenter 300 holds such video data, a signal for identifying the video data and causing the information presenter 300 to reproduce the video data is transmitted.

Further, in order to change the lighting mode of the information presenter 300 as a lighting fixture, for example, a signal for controlling the lighting fixture is transmitted from the output unit 150 to the lighting fixture.

In this way, the signal transmitted from the output unit 150 to the information presenter 300 in order to cause the information presenter 300 to perform an operation to support the user's tidying up is hereinafter also referred to as a tidying up support signal regardless of the contents indicated. ..

Here, as shown in FIG. 2, the information acquisition unit 110, the clutter status recognition unit 120, the user behavior recognition unit 130, the presentation information determination unit 140, the output unit 150, and the storage unit 160 are arithmetic units such as a processor, for example. In one or more information processing devices 100a including a storage device such as a semiconductor memory, the processor executes a program that describes a processing procedure related to the operation described later. The storage device includes a temporary or non-temporary storage medium in which the program read and executed by the processor, or intermediately generated data or data generated as a result of the execution is electromagnetically recorded. The type and installation location of the information processing device can be various. For example, it may be an information processing terminal such as a desktop computer, a laptop computer, a tablet computer, or a smartphone in the house where the situation detection unit 200 collects in-house situation data. Alternatively, it may be a server-type computer located in a place other than the home, or a combination thereof.

The situation detection unit 200 and the information presenter 300 are as described above regardless of whether the information processing device 100a is wireless or wired, or includes a communication line provided by the business operator. Connected to be communicable in order to exchange information (signals indicating).

Alternatively, the situation detection unit 200, the information processing device 100a, and the information presenter 300 do not have to be individual devices that communicate with each other as described above. For example, a tablet computer with a camera in the house may provide a camera 200a, an information processing device 100a, and an audio output device 300b. Further, in order to realize the tidying up support system 10, one device having the functions of the camera 200a and the information processing device 100a may be provided.

[motion]
The operation of the tidying up support system 10 will be described. FIG. 7 is a sequence diagram for explaining the operation of the tidying up support system 10.

First, the home situation data indicating the sensing result output by the sensor that senses the home situation is transmitted from the situation detection unit 200 to the information processing device 100a (step S201).

In the information processing device 100a, when the information acquisition unit 110 acquires the in-house situation data (step S101), the clutter situation recognition unit 120 recognizes the in-house clutter situation (step S102) based on the in-house situation data. The recognition unit 130 recognizes the user's behavior (step S103).

Next, in the information processing apparatus 100a, the presentation information determination unit 140 determines whether the recognition results in steps S103 and S105 satisfy the conditions included in any of the cleanup rules (step S104). If the conditions are not satisfied for any of the cleanup rules (NO in step S104), the processing target of the information processing device 100a is the next home status data. When the condition included in any of the cleanup rules is satisfied (YES in step S104), the presentation information determination unit 140 determines the cleanup support information included in the cleanup rule as the cleanup support information to be presented to the user. (Step S105).

When the cleanup support information to be presented to the user is determined by the presentation information determination unit 140, the output unit 150 transmits a cleanup support signal for causing the information presenter 300 to present the cleanup support information by image, character, light, or voice. , Output to the information presenter 300 (step S106).

The information presenter 300 that has received the cleanup support signal (step S301) outputs an image, characters, light, or voice based on the cleanup support signal to present the cleanup support information (step S302).

By the operation including such a series of procedures, the user is presented with the cleanup support information selected and determined based on the clutter situation in the house and the user's situation.

In the cleanup support system 10, the determination using the conditions included in the cleanup rule, the content of the cleanup support information presented to the user, and the timing of the presentation are not limited to the examples so far.

For example, in the above description, the clutter situation is described as a situation relating to the degree of clutter in the house to be sensed, but the present invention is not limited to this. As the clutter situation, the situation detection unit 200 may recognize a part of the sensing target in the house where the sensing target is cluttered, that is, a place in the house to be cleaned up. Alternatively, as a clutter situation, things in the house that should be cleaned up may be recognized. Such places or objects can also be recognized by the image analysis method as described above.

The result of these recognitions is, for example, information that identifies a sub-region including a place or an object to be cleaned up on an image that is home situation data used for recognition, or coordinates set in this image or a space in the house in advance. It may be included in the form of coordinates or the like indicating these places or positions in the system. In addition, the recognition result includes the recognized place such as "on the table", "floor", "sofa" for the place, "newspaper", "toy", "clothing" for the thing. Information on the specific name of the object may be included.

The tidying up support information presented in this case may be information indicating a place or thing to be tidied up in this house. Specific examples of such cleanup support information include a name indicating the place or thing in the case of voice or character information. Further, the cleanup support signal is a signal for displaying such voice or characters on the information presenter 300.

Further, if the information is based on the lighting mode of the lighting fixture as the information presenter 300, the lighting fixture illuminates the specific place or object brighter than other places, or illuminates the specific place or object with light of a color different from that of other places. Then, it may be highlighted. It should be noted that such a lighting mode may be realized by one lighting fixture capable of switching the irradiation direction and illuminating, or may be realized by a plurality of lighting fixtures that illuminate different places in the house. Further, the tidying up support signal is a signal for causing the luminaire to perform lighting in such an embodiment. This signal may be indicated, for example, by a predetermined section or coordinates representing the direction of a place or object to be highlighted, or may further indicate a mode of highlighting. It is conceivable that enhanced lighting may be performed in different ways depending on what should be cleaned up.

Further, for example, in the above example, the content of the tidying up support information presented to the user is determined when the recognized user's action matches the specific action indicated as a condition. As an example of the variation, for example, the result of recognition of a predetermined action related to tidying up (hereinafter, also referred to as tidying up related action) may be used to determine the timing of presenting the tidying up support information to the user.

The cleanup-related behavior is, for example, an behavior in which the user can start cleaning up without interrupting work such as housework or taking care of someone or dealing with the other party. When such an action is recognized, the presentation information determination unit 140 may determine that it is time to present the cleanup support information, and output the cleanup support signal from the output unit 150 to the information presenter 300. .. As a result, it is possible to encourage the user to clean up at a timing when the user is likely to take a cleanup action. Therefore, it is possible to maintain or improve the effectiveness of the cleanup support system 10 in which the user actually cleans up with reference to the presented cleanup support information.

On the contrary, the cleanup-related action may be an action in an inconvenient state such that the user needs to interrupt some work in order to start the cleanup. When such an action is recognized, even if the tidying up support information is determined based on the clutter situation, it is determined that it is not the timing to present the tidying up support information, and the output unit 150 is sent to the information presenter 300. The cleanup support signal is not output. As a result, it is possible to avoid making the user uncomfortable or annoyed by rushing to clean up or giving a psychological burden. Further, if such feelings are invited, the reliability of the user for the tidying up support system 10 is impaired, and the user may not refer to the tidying up support information presented by the tidying up support system 10. Avoiding the presentation of the tidying up support information at such a timing leads to the maintenance or improvement of the effectiveness of the tidying up support system 10.

When the timing of presenting the cleanup support information is determined according to whether or not the cleanup-related action is being executed, the content of the cleanup support information is determined only based on the clutter situation. You may. In this case, the content of the tidying up support information includes information on the degree of clutter in the house without information related to the recognized user's behavior as illustrated in FIG. 6, or a place where it is further cluttered. It may be only information about things.

[effect]
As described above, the cleanup support system 10 according to the present embodiment outputs the situational awareness unit 200, the information acquisition unit 110, the clutter situation recognition unit 120, the user behavior recognition unit 130, the presentation information determination unit 140, and the output. A unit 150 and an information presenter 300 are provided.

The situation detection unit 200 includes a sensor that senses the situation in the house, outputs the in-house situation data indicating the sensing result, and the information acquisition unit 110 acquires the in-house situation data. Based on the home situation data, the clutter situation recognition unit 120 recognizes the clutter situation in the home, and the user behavior recognition unit 130 recognizes the behavior of the user in the home. The presentation information determination unit 140 determines the cleanup support information presented to the user according to a predetermined rule regarding cleanup in the house based on the clutter situation and the user's behavior. The output unit 150 outputs a cleanup support signal for outputting at least one of the voice information and the visual information representing the content of the cleanup support information to the information presenter 300 that presents at least one of the voice information and the visual information to the user in the house. To do. The information presenter 300 presents audio information or visual information to the user based on the received cleanup support signal.

This allows the user to know the clutter in the house and the positive impact of performing the cleanup on their behavior at that time, and is motivated to clean up.

Further, the recognized user action is a predetermined action related to the tidying up performed by the user, and the presentation information determination unit 140 determines whether or not the user is executing this predetermined action according to a predetermined rule. Depending on the situation, the timing of presenting the tidying up support information may be determined.

As a result, the user is urged to clean up at a timing or timing that is easy for the user to execute, and the effectiveness of the cleanup support system 10 is maintained or improved.

In addition, the clutter situation is a situation relating to at least one of the degree of clutter in the house, the things to be cleaned up in the house, and the place to be cleaned up in the house, and the cleanup support information may include information indicating the clutter situation.

By presenting more specific information about the target of such tidying up, it is possible to give the user a stronger motivation for tidying up.

In addition, the clutter situation is a situation relating to at least one of the things to be cleaned up in the house and the place to be cleaned up in the house, and the cleanup support signal is cleaned up in the house by the lighting equipment in the house as the information presenter 300. It is a signal for highlighting an object or place.

As a result, the object to be cleaned up is visually emphasized, which can give the user a stronger motivation for cleaning up.

(Embodiment 2)
Next, the tidying up support system according to the second embodiment will be described. The cleanup support system 10A according to the present embodiment is different from the cleanup support system 10 according to the first embodiment in that the user's behavior is recognized based on the in-home situation data acquired over a predetermined time. Hereinafter, the cleanup support system 10A according to the present embodiment will be described with reference to the drawings, focusing on the differences from the cleanup support system 10 according to the first embodiment.

[Constitution]
FIG. 8 is a block diagram showing a functional configuration example of the cleanup support system 10A according to the present embodiment. Since the hardware configuration of the cleanup support system 10A can be the same as that of the hardware configuration example of the cleanup support system 10 according to the first embodiment, the illustration and description are omitted, and the common reference is given below. It is shown by a reference numeral.

As shown in FIG. 8, in the tidying up support system 10A, the storage unit 160A accumulates time-series home situation data unlike the storage unit 160 in the first embodiment. Such a storage unit 160A is an example of a home situation data storage unit in the present embodiment, and the storage unit 160A is realized by using a storage device included in the information processing device 100a like the storage unit 160.

The time-series home situation data is home situation data showing the result of sensing over a predetermined time, and is, for example, data of a plurality of frame images included in a still image or a moving image taken at different times. The fact that the user behavior recognition unit 130 recognizes the user's behavior based on this time-series home situation data is also a difference from the cleanup support system 10 according to the first embodiment of the cleanup support system 10A.

As a result, it is possible to recognize the user's behavior with higher accuracy or more detail than the recognition based on the home situation data indicating the momentary home situation.

The path of data exchange between the components indicated by the arrows in FIG. 8 is an example and can be modified. For example, the information acquisition unit 110 may extract and acquire the home status data from the time-series home status data stored in the storage unit 160A instead of from the information acquisition unit 110. Further, the user behavior recognition unit 130 may acquire the in-home situation data from the information acquisition unit 110, and may acquire the time-series in-home situation data stored in the storage unit 160A as needed.

[motion]
Since the overall flow of the operation of the cleanup support system 10A is the same as the operation of the cleanup support system 10 according to the first embodiment shown in FIG. 7, the description thereof is omitted. Here, the difference is the user behavior recognition. The operation of recognizing the user's behavior by the unit 130 will be described with reference to a specific example. FIG. 9 is a flow chart of the operation of the user action recognition unit 130 in the cleanup support system 10A.

For example, it is assumed that images captured at time t, time t-a, and time t-b (b> a) are available.

First, the feature quantity (hereinafter referred to as feature A) is extracted by nonlinearly converting the RGB value vector of the image taken at the oldest time t−b (step S91).

Next, the composite vector of the feature A and the RGB value vector of the image taken at the next oldest time t-a is non-linearly converted to extract the feature amount (hereinafter referred to as feature B) (step S93). ..

Finally, using the model of the convolutional neural network trained using the feature B, the recognition of the user's behavior is executed by inputting the RGB value of the image taken at the latest time t as an input (step S95).

[effect]
By recognizing the user's behavior based on the image data of a plurality of still images along the time series in this way, the user's behavior can be recognized with higher accuracy or in more detail. For example, it is possible to distinguish and recognize the action of holding the remote control to start watching TV and the action of cleaning up the remote control, which cannot be accurately distinguished only by the image of the user holding the remote control.

As a result, the cleanup support system 10A can more accurately determine the timing at which the cleanup support action should be presented to the user.

(Embodiment 3)
Next, the tidying up support system according to the third embodiment will be described. The cleanup support system 10B according to the present embodiment is different from the cleanup support system 10 according to the first embodiment in that the clutter status is recognized based on the continuation status of the clutter status recognized in the past. .. Hereinafter, the cleanup support system 10B according to the present embodiment will be described with reference to the drawings, focusing on the differences from the cleanup support system 10 according to the first embodiment.

[Constitution]
FIG. 10 is a block diagram showing a functional configuration example of the cleanup support system 10B according to the present embodiment. Since the hardware configuration of the cleanup support system 10B can be the same as that of the hardware configuration example of the cleanup support system 10 according to the first embodiment, the illustration and description are omitted, and the common reference is given below. It is shown by a reference numeral.

As shown in FIG. 10, in the tidying up support system 10B, unlike the storage unit 160 in the first embodiment, the storage unit 160B accumulates the recognized clutter situation in time series by the clutter situation recognition unit 120. Such a storage unit 160B is an example of a cluttered situation storage unit in the present embodiment, and the storage unit 160B is realized by using a storage device included in the information processing device 100a like the storage unit 160.

The clutter situation accumulated in time series is information on the clutter situation recognized by the clutter situation awareness unit 120 and the time or time order of the home situation data on which the recognition is based.

[motion]
Since the overall flow of the operation of the cleanup support system 10B is the same as the operation of the cleanup support system 10 according to the first embodiment shown in FIG. 7, the description thereof will be omitted. Regarding the present embodiment, the contents of the operation of the clutter situation awareness unit 120 regarding the time-series home situation data, which is a difference, will be described.

The clutter situation recognition unit 120 not only inputs the clutter situation recognized based on the home situation data to the presentation information determination unit 140, but also stores it in the storage unit 160B that functions as the clutter situation storage unit. Further, the clutter situation recognition unit 120 recognizes the clutter situation in the house based on the latest in-house situation data input from the information acquisition unit 110 and the clutter situation accumulated in the clutter situation storage unit. As a result, for example, when an object that has begun to be recognized at a certain time in the past continues to be recognized at the same location for a certain period of time, it is recognized and recognized by the situational awareness unit 120 as an object to be cleaned up. The result is input to the presentation information determination unit 140.

It should be noted that the above-mentioned thing to be cleaned up is an example of the clutter situation in the house which is further recognized based on the accumulated clutter situation, and is not limited to this. For example, an area that is cluttered for more than a certain period of time may be recognized as a place to be cleaned up, and whether or not the degree of clutter in the entire house exceeding a predetermined threshold exceeds a certain time, or the duration thereof. It may be input to the presentation information determination unit 140 as a clutter situation.

Upon receiving the input of the recognition result, the presentation information determination unit 140 may specify the item to be cleaned up and determine the information to be shown to the user as the cleanup support information. The content of such cleanup support information may be presented to the user by including the name or place of the object to be cleaned up in the audio information output from the information presenter 300, or as visual information. It may be done by highlighting what should be cleaned up.

In addition, information about the duration of the ongoing clutter may be included in the cleanup support information. More specifically, it may be done by including the time in the audio information output from the information presenter 300, or as visual information, the longer the time left uncleaned, the longer the object or place. Intensified lighting, such as illumination with brighter or more prominent colors of light, may be applied.

[effect]
By presenting more specific information about the subject of tidying up, the user can be given a stronger motivation for tidying up.

(Embodiment 4)
Next, the tidying up support system according to the fourth embodiment will be described. The tidying up support system 10C according to the present embodiment is different from the tidying up support system 10 according to the first embodiment in that the tidying up support information is determined based on the information indicating the attribute related to the clutter situation in the user or the house. Hereinafter, the cleanup support system 10C according to the present embodiment will be described with reference to the drawings, focusing on the differences from the cleanup support system 10 according to the first embodiment.

[Constitution]
FIG. 11 is a block diagram showing a functional configuration example of the cleanup support system 10C according to the present embodiment. Since the hardware configuration of the cleanup support system 10C can be the same as that of the hardware configuration example of the cleanup support system 10 according to the first embodiment, illustration and description thereof will be omitted.

As shown in FIG. 11, the cleanup support system 10C further includes an attribute information acquisition unit 170. The attribute information acquisition unit 170 acquires attribute information indicating attributes related to the clutter status of the user or the house.

Examples of the content of the attribute information include the number or configuration (for example, family configuration) of users in the place where the cleanup support system 10C is used as a user attribute. In addition, as an example of the attribute related to the clutter situation in the house, the use of the room (living room, study, children's room, etc.) can be mentioned.

Such attribute information is input to the information processing device 100a by the user using, for example, the user interface of the computer which is the information processing device 100a. Then, this attribute information is acquired as an input by the attribute information acquisition unit 170 realized by the processor executing the program in the information processing apparatus 100a. The attribute information acquired by the attribute information acquisition unit 170 is input to the presentation information determination unit 140.

In the fourth embodiment, the presentation information determination unit 140 determines the cleanup support information based on the attribute information in addition to the recognition result of the clutter situation and the recognition result of the user's behavior used in the first embodiment.

For example, when the family structure includes an infant or an infant, the presentation information determination unit 140 may use the condition regarding the degree of clutter relaxed from that included in the cleanup rule, as compared with the case where the infant or the infant is not included. Alternatively, the presentation information determination unit 140 may use different sets of cleanup rules depending on whether the family structure includes or does not include an infant or toddler. In this case, the set of tidying rules used when the family structure includes an infant or toddler has lesser conditions regarding the degree of clutter.

As a result, for example, it is possible to suppress the occurrence of a situation in which the user who lives in a family structure that is easily messed up is urged to clean up too frequently and is stressed, or the information presented by the cleanup support system 10C is ignored.

Further, the presentation information determination unit 140 may change the condition regarding the degree of clutter included in the tidying up rule according to the use of the room indicated by the attribute information. Alternatively, the presentation information determination unit 140 may use a different set of cleanup rules depending on the use of the room indicated by the attribute information.

As a result, it is possible to make adjustments so that the user is prompted to clean up at an appropriate frequency according to the amount of objects, the ease of cluttering, the degree of need for cleaning up, etc.

[effect]
In this way, by making it possible to adjust the frequency of presenting the tidying up support information to the user, it is possible to give motivation for tidying up so as not to burden the user.

(Modification example)
The cleanup support system according to each of the above embodiments also considers the effectiveness of prompting the user to clean up, but as a modification, a cleanup support system having a configuration for further improving the effectiveness or maintaining the effectiveness. Will be described.

[Constitution]
FIG. 12 is a block diagram showing a functional configuration example of the cleanup support system 1010 according to the present modification. Since the hardware configuration of the cleanup support system 1010 can be the same as that of the hardware configuration example of the cleanup support system 10 according to the first embodiment, the illustration and description are omitted, and the common reference is given below. It is shown by a reference numeral. Hereinafter, the cleanup support system 1010 will be described focusing on the differences from the cleanup support system 10 according to the first embodiment.

As shown in FIG. 12, the tidying up support system 1010 includes a presentation information management unit 190 in addition to the configuration of the tidying up support system 10.

The presentation information management unit 190 changes the cleanup rule based on the clutter in the house, the user's behavior, or both after the cleanup support information is once presented to the user by the information presenter 300 in the form of voice or the like. The presentation information management unit 190 is also realized by the processor executing a program in which the processing procedure related to the operation described later is described in the information processing device 100a.

[motion]
In the cleanup support system 1010, once the cleanup support information is presented to the user by the series of operations shown in FIG. 7, it is determined whether or not the mode of presenting the cleanup support information is effective for the user, and the result of this determination is determined. It operates according to. The following describes the operation of the cleanup support system 1010 including this determination. FIG. 13 is a sequence diagram for explaining the operation of the cleanup support system 1010, and shows an operation following the operation shown in FIG. 7. The steps common to the operation shown in FIG. 7 are shown by using a common reference code, and the description thereof may be omitted or simplified.

After the tidying up support information is presented in step S302 in FIG. 7, the situation in the house is sensed by the sensor provided in the situation detection unit 200. Then, the home situation data indicating the sensing result output by this sensor is transmitted from the situation detection unit 200 to the information processing device 100a (step S201).

In the information processing device 100a, the information acquisition unit 110 acquires the home status data (step S101). Then, based on the home situation data, the clutter situation recognition unit 120 recognizes the clutter situation in the house (step S102), and the user behavior recognition unit 130 recognizes the user's behavior (step S103).

Next, in the information processing device 100a, the presentation information management unit 190 changes the cleanup rule based on the recognition results in steps S102 and S103.

More specifically, first, the presentation of the previous cleanup support information (step S302 in FIG. 7) is effective from the clutter situation in the house, the user's behavior, or both received by the presentation information management unit 190 as a recognition result. Determine if it was the target.

For example, if the degree of clutter in the house indicated by the in-house situation data is lower than the previous time, that is, the recognition result in step S102 in FIG. 7, even if it is determined that the presentation of the tidying up support information was effective. Good (YES in step S114). Alternatively, it may be determined that the presentation of the cleanup support information was effective when the user's behavior indicated by the home status data is cleanup in the home (YES in step S114). In these cases, the procedure of operation in the information processing apparatus 100a proceeds to step S104 by the presentation information determination unit 140.

Further, for example, when the degree of clutter in the house indicated by the house situation data is the same as or increased from the previous time, it is determined that the presentation of the tidying up support information is not effective (NO in step S114). Alternatively, if the user's behavior indicated by the in-home situation data is not the in-house tidying up, it may be determined that the presentation of the tidying up support information was not effective (NO in step S114). In these cases, the presentation information management unit 190 modifies the cleanup rule determined to satisfy the condition in step S104 in FIG. 7, which was executed last time (step S115). For example, in the previous step S104, if the condition included in the first row cleanup rule among the cleanup rules shown in FIG. 6 is satisfied, in step S115, the condition included in the first row cleanup rule is changed. More specifically, for example, the numerical value of the degree of clutter included in the condition of the clutter situation may be increased. In other words, this condition after the change is satisfied when the degree of clutter in the house is not the same as before the change and is more cluttered. The degree of increase may be predetermined by, for example, a program executed by the processor or by user setting.

Next, in the information processing apparatus 100a, the presentation information determination unit 140 determines whether or not the recognition results in step S102 and step S103 in FIG. 13 satisfy the conditions included in any of the cleanup rules this time (step). S104). If the conditions are not satisfied for any of the cleanup rules (NO in step S104), the processing target of the information processing device 100a is the home status data to be received next. When the condition included in any of the cleanup rules is satisfied (YES in step S104), the presentation information determination unit 140 determines the cleanup support information included in the cleanup rule as the cleanup support information to be presented to the user (YES in step S104). Step S105).

When the tidying up support information to be presented to the user is determined, the output unit 150 sends the tidying up support signal to the information presenter 300 for presenting the tidying up support information to the information presenter 300 by image, character, light or voice. Output (step S106).

The information presenter 300 that has received the cleanup support signal (step S301) outputs an image, characters, light, or voice based on the received cleanup support signal, and presents the cleanup support information to the user (step S302).

By operating the information processing device 100a in this way, the degree of clutter in which the cleanup support information is output for a certain action of the user is increased. For example, when the user is cooking or watching TV, if the cleanup support information is presented once according to a certain cleanup rule, but the effect of the presentation is not seen, then this cleanup rule is applied. In accordance with this, the tidying up support information is presented only after the house becomes more cluttered.

As a result, the frequency with which the tidying up support information is presented can be suppressed according to some circumstances of the user, for example, the priority, physical condition, or preference of the user's behavior. In addition, if a cleanup rule including a clutter condition with an increased degree of clutter is applied and the user takes a cleanup action in response to the presentation of cleanup support information according to the changed cleanup rule, the condition of the clutter situation will be Be maintained. That is, the situation in which the tidying up support information is presented is adjusted according to the circumstances of the user. As a result, the timing of presenting the cleanup support information by the information processing device 100a is optimized for the user. As a result, for example, the presentation of the cleanup support information from the information processing device 100a prevents the user from being offended, or the user ignores the presentation of the cleanup support information, and the effectiveness of the information processing device 100a is maintained. And prolongation is planned.

The presentation information management unit 190 described above is a modification in combination with each of the cleanup support system 10A according to the second embodiment, the cleanup support system 10B according to the third embodiment, and the cleanup support system 10C according to the fourth embodiment. The same effect is achieved in the example. FIG. 14A is a block diagram showing a functional configuration example of the cleanup support system 1010A according to a modified example of the cleanup support system 10A. FIG. 14B is a block diagram showing a functional configuration example of the cleanup support system 1010B according to a modified example of the cleanup support system 10B. FIG. 14C is a block diagram showing a functional configuration example of the cleanup support system 1010C according to a modified example of the cleanup support system 10C.

In the cleanup support system 1010B, the presentation information management unit 190 recognizes the object or place where the clutter situation recognition unit 120 is located as an object or place to be cleaned up, instead of or in addition to adjusting the threshold value indicating the degree of clutter. The reference fixed time may be adjusted. As a result, it is possible to avoid a situation in which the user feels annoyed by repeatedly presenting the cleanup support information regardless of the user's circumstances and prompting the cleanup. As a result, it is possible to optimize the timing of prompting the user to clean up by the cleanup support information.

Further, in the tidying up support systems 1010, 1010A, 1010B and 1010C, the judgment using the conditions included in the tidying up rule, the content of the tidying up support information presented to the user, the timing of the presentation, and the like are not limited to the examples so far. ..

For example, in determining whether or not there is an effect of presenting the cleanup support information in step S114, whether or not there is a decrease in the clutter status may be determined depending on whether or not the degree of clutter has decreased by a predetermined magnitude or more.

Further, in determining whether or not the presentation of the cleanup support information is effective in step S114, only one of the clutter situation and the user's behavior may be used. For example, the determination may be made only based on whether or not the home situation data is scattered and shows a decrease in the situation. Alternatively, if the user's cleanup behavior is recognized regardless of the magnitude of the change in the clutter situation, it may be determined that the presentation of the cleanup support information is effective.

Further, the route of exchanging data or information between the components indicated by the arrows in FIGS. 12 and 14A to 14C is an example, and can be modified. For example, the arrow pointing from the output unit 150 to the presentation information management unit 190 indicates the cleanup rule including the cleanup support information to be output to the information presenter 300 among the cleanup rules in FIG. 6, after step S106 in FIG. Indicates that the information is passed from the output unit 150 to the presentation information management unit 190. Such information may be passed from the presentation information determination unit 140 to the presentation information management unit 190 after step S105. Further, such information may be exchanged via the storage unit 160.

Further, based on the recognition result of the cleanup-related behavior, in the change of the cleanup rule in step S115, the degree of clutter including the condition regarding the clutter situation may be raised by a larger range. For example, when the user behavior recognition unit 130 recognizes a cleanup-related behavior that is inconvenient for the user to start cleaning up, the presentation information management unit 190 includes the clutter including the condition regarding the clutter situation as compared with the case where it is not. Change the cleanup rules to increase the degree of. As a result, the tidying up support system 1010 can avoid the occurrence of a situation in which the user is psychologically burdened by the tidying up prompt or the user is made to ignore the tidying up prompt, and the effectiveness is further improved or. Sustainability is planned.

[effect]
As described above, the cleanup support system 1010 according to the present modification includes the situation detection unit 200, the information acquisition unit 110, the clutter situation recognition unit 120, the user behavior recognition unit 130, the presentation information determination unit 140, and the output unit. It includes 150, a presentation information management unit 190, and an information presenter 300.

The situation detection unit 200 includes a sensor that senses the situation in the house, outputs the in-house situation data indicating the sensing result, and the information acquisition unit 110 acquires the in-house situation data. Based on the home situation data, the clutter situation recognition unit 120 recognizes the clutter situation in the home, and the user behavior recognition unit 130 recognizes the behavior of the user in the home. The presentation information determination unit 140 determines the cleanup support information presented to the user according to a predetermined rule regarding cleanup in the house based on the clutter situation and the user's behavior. The output unit 150 outputs a cleanup support signal that causes the information presenter 300, which presents at least one of the audio information and the visual information in the house, to output as at least one of the audio information and the visual information representing the content of the cleanup support information. To do. The information presenter 300 presents audio information or visual information to the user based on the received cleanup support signal. The presentation information management unit 190 is determined based on at least one of the clutter in the house and the user's behavior recognized by the user behavior recognition unit 130 after the information presenter 300 presents the cleanup support information to the user in the form of voice or the like. Change the rules of.

This allows the user to receive information about the clutter in the house and the positive effect of performing the tidying up on his / her behavior at that time at a timing according to his / her own circumstances. Such motivation for cleaning up to the user by the cleaning up support system 1010 can sustain higher effectiveness.

(Other embodiments)
Although the present invention has been described based on each of the above-described embodiments and modifications thereof, the present invention is not limited to the above-described embodiments and modifications thereof, and the following cases are also possible. Included in the present invention.

(1) Specifically, the above-mentioned information processing device is a computer system composed of a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or the hard disk unit. When the microprocessor operates according to the computer program, each device achieves its function. Here, a computer program is configured by combining a plurality of instruction codes indicating commands to a computer in order to achieve a predetermined function.

(2) A part or all of the components constituting each of the above devices may be composed of one system LSI (Large Scale Integration: large-scale integrated circuit). A system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, is a computer system including a microprocessor, ROM, RAM, and the like. .. A computer program is stored in the RAM. The system LSI achieves its function by operating the microprocessor according to the computer program.

Further, each part of the constituent elements constituting each of the above-mentioned devices may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them.

Further, although it is referred to as a system LSI here, it may be referred to as an IC, an LSI, a super LSI, or an ultra LSI depending on the degree of integration. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, an FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used.

Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is natural that the components may be integrated using that technology. There is a possibility of adaptation of biotechnology.

(3) A part or all of the constituent elements constituting each of the above devices may be composed of an IC card or a single module that can be attached to and detached from each device. The IC card or the module is a computer system composed of a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI. When the microprocessor operates according to a computer program, the IC card or the module achieves its function. This IC card or this module may have tamper resistance.

(4) In each of the above embodiments, as an information presenter for presenting cleanup support information to the user, a lighting fixture, a speaker, a display, or an information processing terminal is given as an example, but the present invention is not limited thereto. For example, various robots such as companion robots and cleaning robots used at home may function as information presenters. If the robot is equipped with speakers, light sources, or indicators, they may be used to present cleanup support information. In addition, in the case of a robot that can automatically move in the house, it may move to the vicinity of these places or objects in order to indicate the object or place to be cleaned up, which is the content of the cleanup support information. .. Also, instead of or in addition to voice or light, posture or movement may indicate an object or place to be cleaned up. Examples of such postures or movements include, for example, a hand-shaped or hand-shaped part that points to a tidying target, and a face-corresponding part that is directed or brought closer to the tidying target. ..

As a result, it is possible to present the tidying up support information in a more familiar atmosphere for the user.

Further, when there are a plurality of robots in the house, the tidying up support information may be indirectly presented to the user by using communication between the robots instead of the direct presentation thrown from the robot to the user. The presentation using communication between robots is executed, for example, in a dialogue or operation for communication between robots in a form in which information on an object or place to be cleaned up, which is the content of cleanup information, is represented. Alternatively, the information on the object or place to be cleaned up may appear in the communication between the user and the plurality of robots instead of the one-to-one communication between the robot and the user.

As a result, the response responsibility that makes the user feel is smaller than that of the direct information presentation from the information presenter, and it is less likely to cause trouble. As a result, the user can easily start the cleanup action according to the cleanup support information, which leads to the maintenance or improvement of the effectiveness of the cleanup support system 10.

Note that FIG. 15 is a block diagram showing a functional configuration example of the cleanup support system 10D according to the embodiment including a plurality of robots, each of which functions as an information presenter. In this example, three robots are included in the cleanup support system 10D as the information presenter 300A, the information presenter 300B, and the information presenter 300C.

Further, FIG. 16 is a functional block diagram showing a functional configuration example of the information presenter, which is a robot, using the information presenter 300A as an example.

The information presenter 300A includes a communication unit 310, a control unit 320, a drive unit 330, a display unit 340, a voice output unit 350, and a storage unit 360.

The communication unit 310 receives the cleanup support signal transmitted from the output unit 150 of the information processing device 100a wirelessly or by wire. The tidying up support signal in this case is a signal for causing the information presenter 300A to indirectly present the tidying up support information to the user, for example, a place or object to be tidied up, or the information presenter 300A. Indicates a predetermined operation mode or the like.

The control unit 320 controls the operation of the information presenter 300A by reading and executing a program realized by a processor, for example, and stored in the storage unit 360. Information on the content of the operation for cleaning up support is obtained by decoding the cleaning up support signal obtained via the communication unit 310. Then, a signal for realizing this operation is sent to the drive unit 330, the display unit 340, or the audio output unit 350.

The drive unit 330 is a functional component realized by, for example, an actuator for moving a movable part of the robot. It operates according to the signal received from the control unit 320, and realizes the movement of the robot or the movement of each part of the robot. By such a movement, the instruction of the place or object to be cleaned up may be realized, or the facial expression or gesture of the robot may be expressed.

The display unit 340 is realized by a display, a projector, or the like, operates according to a signal received from the control unit 320, and presents information visually perceived to the user. The presentation of visually perceived information here may be the display of characters, figures or symbols indicating the place or object to be cleaned up, or the expression of the robot's facial expression or information by images. Good.

The voice output unit 350 is realized by a speaker, operates according to a signal received from the control unit 320, and provides voice-based cleanup support information. In addition, the voice does not have to represent the human language, and for example, the bark of an animal or a fictitious character, an electronic sound, or the like is emitted to attract the user's attention or indicate the occurrence of communication between robots. May be good.

The storage unit 360 is realized by a storage device such as a hard disk drive or a semiconductor memory, and stores, for example, a program read by the control unit 320. In addition, video data displayed on the display unit 340 and audio data output from the audio output unit 350 may be stored.

The configuration of the information presenter, which is a robot, is not limited to the above example. For example, it is not necessary to include a part of the drive unit 330, the display unit 340, and the audio output unit 350. Further, for example, a sensor may be further provided. The output of this sensor may be data of information processed for control in the robot, or may be used as in-home situation data provided to the information acquisition unit 110 through the communication unit 310. That is, in the latter case, the sensor provided by the robot functions as the situation detection unit 200.

Further, the method of moving the robot is not limited to traveling on the floor surface or the like, and may be movement by flight, for example, as performed by a drone.

(5) The present invention may be the method shown above. Further, it may be a computer program that realizes these methods by a computer, or it may be a digital signal composed of the above computer program.

The present invention also relates to a computer-readable recording medium such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, or a BD (Blu-ray (Blu-ray). It may be recorded on a registered trademark) Disc), a semiconductor memory, or the like. Further, it may be the digital signal recorded on these recording media.

Further, the present invention may transmit the computer program or the digital signal via a telecommunication line, a wireless or wired communication line, a network typified by the Internet, data broadcasting, or the like.

Further, the present invention is a computer system including a microprocessor and a memory, and the memory may store the computer program, and the microprocessor may operate according to the computer program.

Further, it is carried out by another independent computer system by recording the program or the digital signal on the recording medium and transferring the program, or by transferring the program or the digital signal via the network or the like. May be.

(6) Each of the above embodiments and the above modifications may be combined. For example, the storage device included in the information processing device 100a may realize both a home status data storage unit that stores time-series home information data and a clutter status storage unit that stores clutter status in time series. Further, the attribute information acquisition unit may be realized by executing the program by the processor included in the information processing apparatus 100a.

(7) Although the cleanup support system in each of the above embodiments is described as including a situation detection unit 200 realized by a camera or the like provided with a sensor and an information presenter 300 realized by a speaker or the like. The present invention may be provided as one or more information processing devices that do not include the situation detection unit 200 or the information presenter 300 and provide other components.

The term "cleanup support system" in the present application is used for convenience without distinguishing between a plurality of such realization modes of the present invention, and a realization mode realized by one information processing device is also substantially used. Please note that this is a concept that includes.

10, 10A, 10B, 10C, 10D, 1010, 1010A, 1010B, 1010C Cleanup support system 100a Information processing device 110 Information acquisition unit 120 Scattered situation recognition unit 130 User behavior recognition unit 140 Presentation information determination unit 150 Output unit 190 Presentation information management Department

Claims (14)

An information acquisition unit that acquires in-home status data showing the sensing results from a sensor that senses the in-home status,
A clutter situation recognition unit that recognizes the clutter situation in the house based on the home situation data,
A user behavior recognition unit that recognizes the behavior of a user in the home based on the home status data,
A presentation information determination unit that determines cleanup support information presented to the user according to a predetermined rule regarding cleanup in the house based on the clutter situation and the behavior of the user.
A cleanup support system including an information presenter in the house and an output unit that outputs a cleanup support signal for outputting the cleanup support information as at least one of audio information and visual information. The user's behavior is a predetermined behavior related to the tidying up performed by the user.
The cleanup support system according to claim 1, wherein the presentation information determination unit determines the timing of presenting the cleanup support information according to whether or not the user is executing the predetermined action according to the predetermined rule. .. The clutter situation is a situation relating to at least one of the degree of clutter in the house, the things to be cleaned up in the house, and the place to be cleaned up in the house.
The cleanup support system according to claim 1 or 2, wherein the cleanup support information includes information indicating the clutter status. Further equipped with an in-home situation data storage unit that accumulates the in-home situation data acquired over a predetermined time as time-series in-home situation data.
The cleanup support system according to any one of claims 1 to 3, wherein the user behavior recognition unit recognizes the user's behavior based on the accumulated time-series home situation data. Further provided with a mess situation storage unit that accumulates the recognized mess situation in chronological order.
The cleanup support system according to any one of claims 1 to 4, wherein the presentation information determination unit further determines the content of the cleanup support information based on the accumulated clutter. Further provided with an attribute information acquisition unit for acquiring the attribute of the tendency of the clutter in the house or the attribute information indicating the attribute of the user.
The cleanup support system according to any one of claims 1 to 5, wherein the presentation information determination unit further determines the cleanup support information based on the attribute information. The clutter situation is a situation relating to at least one of the things to be cleaned up in the house and the place to be cleaned up in the house.
The tidying up support signal is a signal for highlighting an object to be tidied up in the house or a place to be tidied up in the house to a lighting fixture in the house as an information presenter. The cleanup support system described in any one of the items. The content of the cleanup support information is a voice output of a message prompting cleanup from a voice output device having a function of outputting voice, which is located in the house as the information presenter.
The cleanup support system according to any one of claims 1 to 7, wherein the output unit transmits a control signal for causing the voice output device to execute the voice output of the message. Further, the information presenter includes a presentation information management unit that changes the predetermined rule based on at least one of the clutter situation in the house and the behavior of the user recognized after the information presenter presents the cleanup support information to the user. The cleanup support system according to any one of claims 1 to 8. It is effective for the presentation information management unit to present the tidying up support information to the user from the clutter situation in the house or the behavior of the user recognized after the information presenter presents the tidying up support information to the user. Judge whether it was or not,
The cleanup support system according to claim 9, wherein when the presentation of the cleanup support information to the user is not effective, the presentation information management unit changes the predetermined rule. The information presenter is a plurality of robots capable of communicating with each other at least one of dialogue and operation, and the content of the cleanup support information is expressed in the dialogue or action between the plurality of robots to clean up. The cleanup support system according to any one of claims 1 to 10, wherein the support information is presented to the user. The information presenter is an automatically movable robot, and the tidying up support information is provided to the user by using an operation including the robot moving to an object to be tidied up in the house or a place to be tidied up in the house. The cleanup support system according to any one of claims 1 to 10 presented. The cleanup support system according to any one of claims 1 to 12, further comprising the sensor and the information presenter. For the processor of the computer
In-house situation data showing the sensing result is acquired from the sensor that senses the in-house situation.
Recognize the cluttered situation in the house based on the house situation data
Recognize the behavior of the user in the house based on the in-house situation data.
Based on the clutter situation and the behavior of the user, the cleanup support information presented to the user is determined according to a predetermined rule regarding the cleanup in the house.
A program that causes an information presenter in the house to output a cleanup support signal for outputting the cleanup support information as at least one of audio information and visual information.

Images