JP2020017258A - Information processing method, information processing apparatus, and information processing program - Google Patents

Abstract

To provide an information processing method capable of causing a self-propelled apparatus to perform a predetermined operation according to the situation in which an article has been damaged, an information processing apparatus, and an information processing program.SOLUTION: An information processing method in a server device 3 includes the steps of: obtaining a piece of first information obtained in at least one of the one or more sensors installed in a space; detecting damage of an article existing in the space based on the first information; estimating the situation where the article is damaged based on the first information; and outputting a piece of second information for causing the self-propelled cleaner 21 to execute a predetermined operation in the space according to the estimated situation.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an information processing method, an information processing device, and an information processing program for causing a device to execute a predetermined operation.

  Conventionally, during a cleaning operation, image recognition is performed by comparing a captured image captured by the imaging unit with an image of a registered foreign substance stored in the storage unit, and the registered foreign substance stored in the storage unit is compared. There is a known electric cleaning device that recognizes (see, for example, Patent Document 1). This electric cleaning device, when recognizing a registered foreign substance, controls the suction drive unit based on the control mode stored in the storage unit corresponding to the recognized foreign substance, and removes the recognized foreign substance. A display for identifying what is is displayed on the display screen.

Japanese Patent No. 5771885

  However, in the above-described conventional technology, the situation in which the article is damaged is not estimated, and further improvement is required.

  The present disclosure has been made in order to solve the above-described problem, and has an information processing method, an information processing apparatus, and an information processing method that can cause a self-propelled device to execute a predetermined operation according to a situation in which an article is damaged. It is an object to provide an information processing program.

  An information processing method according to an aspect of the present disclosure is an information processing method in an information processing device, wherein the first information acquired by at least one of one or more sensors installed in a space is acquired, and the first information is acquired. 1 based on the information, detects damage to the article present in the space, estimates a situation in which the article has been damaged based on the first information, and, based on the estimated situation, The second information for causing the mobile device to execute a predetermined operation is output.

  According to the present disclosure, when an article existing in a space is damaged, the self-propelled device can execute a predetermined operation according to the situation in which the article has been damaged.

1 is a diagram illustrating a configuration of a device control system according to a first embodiment of the present disclosure. FIG. 2 is a diagram illustrating a configuration of a server device according to Embodiment 1 of the present disclosure. FIG. 3 is a diagram illustrating an example of device operation information stored in a device operation information storage unit according to the first embodiment. FIG. 3 is a diagram illustrating an example of article information stored in an article information storage unit according to the first embodiment. FIG. 3 is a diagram illustrating an example of movie information stored in a service information storage unit according to the first embodiment. FIG. 3 is a diagram illustrating an example of restaurant information stored in a service information storage unit according to the first embodiment. 6 is a first flowchart for describing an operation of the server device according to Embodiment 1 of the present disclosure. 6 is a second flowchart for describing an operation of the server device according to Embodiment 1 of the present disclosure. FIG. 7 is a diagram for describing an operation of the device in the first situation in which an article slips from a person's hand during a daily operation in the first embodiment. FIG. 5 is a diagram for describing an operation of the device in a second situation in which a plurality of persons are fighting in the first embodiment. FIG. 7 is a diagram for describing an operation of the device in the third situation in which a suspicious person has entered in the first embodiment. FIG. 10 is a diagram illustrating a configuration of a first server device according to a second embodiment of the present disclosure. FIG. 13 is a diagram illustrating a configuration of a second server device according to Embodiment 2 of the present disclosure.

(Knowledge underlying the present disclosure)
In the above conventional technology, an image of a foreign substance other than the object to be cleaned is registered in the storage unit in advance, and a captured image captured during the cleaning operation is compared with the image of the foreign substance, and the registered foreign substance is recognized, A display for identifying the recognized foreign matter is displayed on the display screen. The foreign substances include, for example, “plastic bags”, “documents”, “cords”, and “screws” from the viewpoint of avoiding breakdown or breakage of the vacuum cleaner. From the viewpoint of securing and avoiding contamination and breakage, "documents", "micro SD cards", "banknotes", "jewels" and the like are mentioned. That is, in the related art, although a foreign substance other than the object to be cleaned is recognized, a cleaning object such as a damaged mug is not recognized.

  Further, in the related art, when an article is damaged, a situation in which the article is damaged is not estimated. Therefore, the related art does not disclose or suggest that the vacuum cleaner performs a predetermined operation according to a situation in which the article is damaged.

  In order to solve the above problems, an information processing method according to an embodiment of the present disclosure is an information processing method in an information processing device, the method being obtained by at least one of one or more sensors installed in a space. Acquiring first information, detecting damage to an article present in the space based on the first information, estimating a situation in which the article has been damaged based on the first information, And outputs second information for causing the self-propelled device to execute a predetermined operation in the space.

  According to this configuration, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation where the article has been damaged is estimated, and the space is determined according to the estimated situation. Inside, the second information for causing the self-propelled device to execute a predetermined operation is output. Therefore, when an article present in the space is damaged, the self-propelled device can cause the self-propelled device to execute a predetermined operation according to the situation in which the article is damaged.

  Further, in the above information processing method, the second information may be information for causing the self-propelled device to output a predetermined sound in the space according to the estimated situation.

  According to this configuration, the predetermined sound can be output to the self-propelled device in the space according to the situation in which the article is damaged. For example, if the situation in which the article is damaged is a situation where a plurality of persons are fighting, the self-propelled device may perform an operation of moving while outputting sound soothing a plurality of fighting persons. it can.

  Further, in the above information processing method, third information for causing the presentation device to present information for changing the estimated situation may be output.

  According to this configuration, since the third information for causing the presentation device to present information for changing the estimated situation is output, it is possible to change the situation in which the article is damaged. For example, when the situation in which the article is damaged is a situation in which a plurality of persons are fighting, information suitable for a plurality of fighting persons to make up can be presented.

  In the above information processing method, the self-propelled device is a self-propelled cleaner, and the second information is transmitted to the self-propelled cleaner in the space according to the estimated situation. The information may be information for cleaning a damaged article.

  According to this configuration, the self-propelled device is a self-propelled cleaner, and can cause the self-propelled cleaner to clean the damaged article in the space according to the situation where the article has been damaged. For example, when the situation in which the article has been damaged is a situation in which the article has slipped from the hand of a person during a daily operation, the operation of causing the self-propelled cleaner to clean the damaged article can be performed.

  In the above information processing method, the estimated situation is a situation where a suspicious person has entered the space, and the second information disturbs the suspicious person by the self-propelled device in the space. Information for executing the operation may be used.

  According to this configuration, when the situation in which the article is damaged is a situation in which a suspicious person has entered the space, the self-propelled device can perform an operation of disturbing the suspicious person in the space.

  In the above information processing method, further, image data obtained by imaging the suspicious person from an imaging device arranged in the space is obtained, and further, the obtained image data and the suspicious person are present. Notification information for notification may be transmitted.

  According to this configuration, image data obtained by imaging the suspicious individual is acquired from the imaging device arranged in the space, and the acquired image data and notification information for notifying that the suspicious individual is present are transmitted. Therefore, it is possible to notify another person that a suspicious person exists.

  Further, in the above information processing method, further, when damage information indicating that the self-propelled device has been damaged is obtained, fourth information for requesting repair of the self-propelled device may be output. Good.

  According to this configuration, when the self-propelled device is damaged, repair of the self-propelled device can be automatically requested.

  Further, in the above information processing method, furthermore, fifth information for causing a presentation device to present information on an article for suppressing occurrence of the estimated situation may be output.

  According to this configuration, since information about the article for suppressing the occurrence of the situation where the article has been damaged is presented to the presentation device, the occurrence of the situation where the article has been damaged can be suppressed. For example, if the situation in which the article has been damaged is a situation in which a suspicious person has entered the space, information about security equipment must be presented as an article to suppress the occurrence of a situation in which the suspicious person has entered the space. Can be.

  In the above information processing method, the one or more sensors include at least one of a microphone device and an imaging device installed in the space, and the first information includes sound data and sound data acquired by the microphone device. The situation may be estimated based on at least one of the sound data and the image data, and the situation may be estimated based on at least one of the sound data and the image data.

  According to this configuration, based on at least one of the sound data obtained by the microphone device installed in the space and the image data obtained by the imaging device installed in the space, it is highly likely that the article has been damaged. It can be estimated with accuracy.

  Further, in the above information processing method, the obtaining of the first information obtains the first information at predetermined time intervals, and the estimation of the situation is performed based on a predetermined time based on a point in time when the article is damaged. The situation in which the article has been damaged may be estimated based on a plurality of pieces of first information acquired during a period.

  According to this configuration, the first information is obtained at a predetermined time interval, and the damage of the article occurs based on the plurality of pieces of first information obtained within a predetermined period based on the time when the damage of the article occurs. Since the situation is estimated, for example, the situation in which the article has been damaged can be estimated with higher accuracy by using past image data for a predetermined period from the time when the article has been damaged.

  An information processing apparatus according to another aspect of the present disclosure includes an acquisition unit configured to acquire first information acquired by at least one of one or more sensors installed in a space, and the space based on the first information. A detecting unit for detecting the damage of the article present in the vehicle, an estimating unit for estimating a situation in which the article is damaged based on the first information, and a self-propelled vehicle in the space according to the estimated situation. An output unit that outputs second information for causing the expression device to execute a predetermined operation.

  According to this configuration, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation where the article has been damaged is estimated, and the space is determined according to the estimated situation. Inside, the second information for causing the self-propelled device to execute a predetermined operation is output. Therefore, when an article present in the space is damaged, the self-propelled device can cause the self-propelled device to execute a predetermined operation according to the situation in which the article is damaged.

  An information processing program according to another aspect of the present disclosure acquires first information acquired by at least one of one or more sensors installed in a space, and stores the first information in the space based on the first information. Detecting the damage of the article to be performed, estimating the situation where the article has been damaged based on the first information, and causing the self-propelled device to execute a predetermined operation in the space according to the estimated situation. Computer to output the second information for the computer.

  According to this configuration, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation where the article has been damaged is estimated, and the space is determined according to the estimated situation. Inside, the second information for causing the self-propelled device to execute a predetermined operation is output. Therefore, when an article present in the space is damaged, the self-propelled device can cause the self-propelled device to execute a predetermined operation according to the situation in which the article is damaged.

  Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. The following embodiment is an example of the present disclosure, and does not limit the technical scope of the present disclosure.

(Embodiment 1)
FIG. 1 is a diagram illustrating a configuration of a device control system according to Embodiment 1 of the present disclosure. As shown in FIG. 1, the device control system includes a server device 3, a gateway (GW) 5, a first sensor 11, a second sensor 12, a self-propelled cleaner 21, and a display device 22.

  The gateway 5, the first sensor 11, the second sensor 12, the self-propelled cleaner 21 and the display device 22 are arranged in the house 10. The gateway 5 is connected to the first sensor 11, the second sensor 12, the self-propelled cleaner 21 and the display device 22 so as to be able to communicate wirelessly. The gateway 5 is communicably connected to the server device 3 via the network 4. The network 4 is, for example, the Internet.

  The first sensor 11, the second sensor 12, the self-propelled cleaner 21 and the display device 22 are communicably connected to the server device 3 via the gateway 5. The first sensor 11, the second sensor 12, the self-propelled cleaner 21 and the display device 22 may be connected to the server device 3 so as to be able to directly communicate with each other without passing through the gateway 5.

  FIG. 2 is a diagram illustrating a configuration of the server device according to Embodiment 1 of the present disclosure. The server device 3 is communicably connected to a sensor group 1 including a plurality of sensors arranged in the house 10 and a device group 2 including a plurality of devices arranged in the house 10. The sensor group 1 includes various sensors such as a first sensor 11 and a second sensor 12. The device group 2 includes various devices such as a self-propelled cleaner 21, a display device 22, and an information device 23. In FIG. 2, the gateway 5 is omitted.

  The first sensor 11 is, for example, a microphone device, collects sounds in the house 10, and transmits sound data to the server device 3. The second sensor 12 is, for example, an imaging device, which images the inside of the house 10 and transmits image data to the server device 3. Note that the sensor group 1 may include a thermal image sensor and a vibration sensor. The sensors constituting the sensor group 1 may be installed on walls, floors, and furniture of the house 10 or may be mounted on any of the devices in the device group 2.

  The self-propelled cleaner 21 is an example of a self-propelled device, and performs suction cleaning while moving autonomously. The self-propelled cleaner 21 cleans the floor surface while autonomously moving on the floor surface in the house 10. Usually, the self-propelled cleaner 21 is connected to a charging device (not shown) installed at a predetermined place in the house 10, and a cleaning start button provided on the main body of the self-propelled cleaner 21 is provided. When pressed by the user or when cleaning instruction information is received from the server device 3, the user leaves the charging device and starts cleaning. The self-propelled cleaner 21 includes a control unit (not shown), a camera, a speaker, a driving unit, a cleaning unit, and a communication unit.

  The control unit controls a cleaning operation by the self-propelled cleaner 21. The drive unit moves the self-propelled cleaner 21. The driving unit includes a driving wheel for moving the self-propelled cleaner 21 and a motor for driving the driving wheel. The drive wheels are installed at the bottom of the self-propelled cleaner 21. The cleaning unit is installed at the bottom of the self-propelled cleaner 21 and sucks the suction target.

  The camera captures an image of the traveling direction of the self-propelled cleaner 21. The communication unit transmits image data captured by the camera to the server device 3. In addition, the communication unit receives cleaning instruction information for starting cleaning from the server device 3. The control unit starts cleaning when the cleaning instruction information is received by the communication unit. Note that the cleaning instruction information includes a damaged position where the article 6 is damaged in the house 10. The broken position is a position where the article 6 such as a mug and a plate is broken. After moving to the damaged position, the self-propelled cleaner 21 captures an image of the suction target present at the damaged position, and transmits the captured image data to the server device 3. Then, the self-propelled cleaner 21 cleans the damaged position and returns to the charging device.

  The loudspeaker outputs a predetermined sound according to the situation in which the article is damaged. For example, if the situation in which the article is damaged is a situation in which a plurality of persons are fighting, the speaker outputs a sound soothing the plurality of fighting persons.

  In the first embodiment, the device control system includes the self-propelled cleaner 21 as an example of a self-propelled device. However, the present disclosure is not particularly limited thereto, and the self-propelled cleaner such as a pet-type robot may be used. A mobile robot may be provided as an example of a self-propelled device. The self-propelled robot has functions other than the cleaning function of the self-propelled cleaner 21.

  The display device 22 is arranged on a wall of a predetermined room in the house 10. Further, the device control system according to the first embodiment may include a plurality of display devices 22. The plurality of display devices 22 may be arranged on a wall of each room such as a living room, kitchen, bedroom, bath, toilet, and entrance. The display device 22 may be an information terminal such as a smartphone or a tablet computer. The display device 22 includes a communication unit, a display unit, and an input unit (not shown).

  The communication unit receives information indicating a state of a device from a device arranged in the house 10. Further, the communication unit receives the presentation information from the server device 3.

  The display unit is, for example, a liquid crystal display device and displays various information. The display unit displays information related to devices arranged in the house 10. The display unit displays, for example, the current state of the washing machine, and displays the current state of the air conditioner. The display unit displays the presentation information received by the communication unit.

  The input unit is, for example, a touch panel, and receives an input operation by a user. The input unit receives an input of an operation instruction for a device arranged in the house 10. The input unit receives, for example, an input of an operation instruction to an air conditioner or an input of an operation instruction to a lighting device. The communication unit transmits the operation instruction input by the input unit to the device.

  The information device 23 is, for example, a smartphone, a tablet computer, a personal computer, or a mobile phone, and has a function of communicating with the outside. The information device 23 includes a communication unit (not shown). The communication unit receives image data of the suspicious person who has entered the house 10 and notification information for notifying that the suspicious person is present from the server device 3, and transmits the received image data and notification information to the police. To the server managed by the server. The server device 3 transmits the image data and the notification information to the server device managed by the police via the information device 23 in the house 10, so that the police can specify the transmission source of the image data and the notification information. .

  In the first embodiment, the server device 3 transmits the image data and the notification information to the server device managed by the police via the information device 23. However, the present disclosure is not particularly limited thereto, and the server device 3 transmits the notification data to the server device. The device 3 may directly transmit the image data and the notification information to the server device managed by the police, without passing through the information device 23.

  The device group 2 includes a washing machine, a lighting device, an air conditioner, an electric shutter, an electric lock, an air cleaner, and the like, in addition to the self-propelled cleaner 21, the display device 22, and the information device 23. The devices that make up the device group 2 include, for example, home appliances, information devices, and home equipment.

  The server device 3 includes a communication unit 31, a processor 32, and a memory 33.

  The communication unit 31 includes a sensor data receiving unit 311, a control information transmitting unit 312, a presentation information transmitting unit 313, and a notification information transmitting unit 314. The processor 32 includes a damage detection unit 321, a situation estimation unit 322, a damage position identification unit 323, a device operation determination unit 324, a damaged item identification unit 325, an alternative item identification unit 326, a presentation information generation unit 327, a service identification unit 328, and a notification. An information generation unit 329 is provided. The memory 33 includes a device operation information storage unit 331, an article information storage unit 332, and a service information storage unit 333.

  The sensor data receiving unit 311 acquires sensor data (first information) acquired by at least one of one or more sensors installed in the house 10 (space). The sensor data receiving unit 311 receives sensor data from each sensor of the sensor group 1. The sensor data (first information) includes sound data acquired by the first sensor 11 (microphone device) and image data acquired by the second sensor 12 (imaging device). The sensor data receiving unit 311 receives sound data from the first sensor 11 as sensor data, and receives image data from the second sensor 12 as sensor data.

  Further, the sensor data receiving unit 311 receives sensor data from each device of the device group 2. In the device group 2, there is also a device provided with a sensor. The device provided with the sensor transmits the sensor data to the server device 3. As described above, the self-propelled cleaner 21 includes the camera. Therefore, the sensor data receiving unit 311 receives image data from the self-propelled cleaner 21 as sensor data. Further, the display device 22 may include a microphone and a camera, and the sensor data receiving unit 311 may receive sound data and image data from the display device 22 as sensor data.

  The damage detection unit 321 detects damage to an article present in the house 10 based on the sensor data (first information) received by the sensor data reception unit 311. When the sound data received from the first sensor 11 includes a feature of a sound generated when the article is damaged, the damage detection unit 321 detects that the article is damaged. The memory 33 may store in advance, for example, frequency components of a plurality of breakage sounds such as a sound of breaking ceramics or glass. The breakage detecting unit 321 compares the frequency component of the sound data received from the first sensor 11 with the frequency components of the plurality of breakage sounds stored in the memory 33. If the two frequency components match, the article is detected. Detects damage.

  Note that the damage detection unit 321 uses the sound data at the time of the occurrence of the article damage and the occurrence of the article damage as teacher data using a prediction model that is machine-learned as the teacher data, and receives the sound received from the first sensor 11. From the data, it may be estimated that the article has been damaged. In this case, the prediction model is stored in the memory 33 in advance.

  Further, the damage detection unit 321 may detect that the article has been damaged from the image data captured by the second sensor 12. For example, the sensor data receiving unit 311 may acquire temporally continuous image data from the second sensor 12. The damage detection unit 321 analyzes the acquired image data, and if the image data includes a state in which the article falls from a person's hand in the house 10 and the article is broken on the floor, the article is damaged. May be detected.

  Further, the damage detection unit 321 may detect that the article has been damaged using sensor data from another sensor such as a vibration sensor. Further, the damage detection unit 321 may detect that the article has been damaged using sensor data from a plurality of sensors in the sensor group 1.

  The situation estimating unit 322 estimates the situation in which the article has been damaged based on the sensor data (first information). The situation estimation unit 322 estimates the situation in which the article has been damaged based on at least one of the sound data and the image data. That is, the sensor data receiving unit 311 acquires image data at predetermined time intervals. The situation estimation unit 322 estimates the situation in which the article has been damaged based on a plurality of image data acquired within a predetermined period based on the point in time when the article has been damaged. The damage detection unit 321 can identify the point at which the article has been damaged by recognizing the characteristic component of the sound of the damaged article from the sound data. The situation estimating unit 322 estimates the situation in which the article has been damaged based on a plurality of pieces of image data obtained within a predetermined period in the past from the time when the specified article has been damaged.

  For example, if an article slips off a person's hand during a person's daily activities, the article is damaged. Further, for example, when a plurality of persons are fighting, the article is damaged by one of the plurality of persons throwing the article. In addition, for example, when a suspicious person enters the house 10, the suspicious person destroys the article, and the article is damaged. Therefore, the situation in which the article has been damaged includes, for example, the first situation in which the article has slipped from the hand of a person during a daily operation, the second situation in which a plurality of persons are fighting, and the intrusion of a suspicious person. A third situation exists. The situation estimating unit 322 determines that the situation in which the article has been damaged includes the first situation in which the article slipped from the hand of the person during the daily operation, the second situation in which a plurality of persons are fighting, and the suspicious person. Estimate which of the third situations is invading.

  The situation estimating unit 322 analyzes a plurality of image data immediately before the time when the article is damaged, and recognizes a person's hand and the article included in the plurality of image data. When the article drops from the person's hand, the situation estimation unit 322 estimates that the situation in which the article has been damaged is the first situation in which the article has slipped off the person's hand during daily operations. I do. The situation estimation unit 322 acquires the sound data immediately before the time when the article is damaged, and if the sound data includes a surprised voice of a person, the situation in which the article is damaged is regarded as a daily state. It may be presumed that the first situation is that the article has slipped off the hand of the person during the operation.

  In addition, the situation estimation unit 322 analyzes a plurality of image data immediately before the time when the article is damaged, and recognizes a plurality of person's hands and the article included in the plurality of image data. Then, when one of the plurality of persons throws the article, the situation estimation unit 322 estimates that the situation in which the article has been damaged is the second situation in which the plurality of persons are fighting. Note that the situation estimation unit 322 acquires the sound data immediately before the damage of the article occurs, and if the sound data includes voices of a plurality of people arguing, the situation in which the damage of the article has occurred is more than one. It may be estimated that the second situation is that the person is fighting.

  In addition, the situation estimation unit 322 acquires the sound data immediately before the time when the article is damaged, and when the sound data is equal to or more than the volume threshold of the voices of a plurality of persons included in the sound data, the situation in which the article is damaged is determined. Alternatively, it may be estimated that this is the second situation where a plurality of persons are fighting. Further, the situation estimation unit 322 may acquire a plurality of image data immediately before the time when the article is damaged, and recognize a motion in which a plurality of persons included in the plurality of image data are fighting. Further, the situation estimation unit 322 may detect a vibration generated when a plurality of persons are fighting by the vibration sensor.

  Further, the situation estimating unit 322 analyzes a plurality of image data immediately before the time when the article is damaged, and recognizes a person included in the plurality of image data. Then, when a person who is not a resident of the house 10 registered in advance is recognized, the situation estimation unit 322 estimates that the situation in which the article has been damaged is the third situation in which a suspicious person has entered. . In addition, the situation estimation unit 322 determines that the situation in which the damage of the article occurs when a person who is not a resident of the pre-registered house 10 is recognized and a resident of the pre-registered house 10 is not recognized. It may be estimated that this is the third situation in which a suspicious person has entered.

  The situation estimating unit 322 uses the prediction model that is machine-learned using the image data immediately before the article is damaged and the situation where the article is damaged as teacher data, and the article is damaged. The situation where the article has been damaged may be estimated from the image data immediately before the time point. Note that the prediction model is stored in the memory 33 in advance.

  The situation in which the article is damaged in the first embodiment is not limited to the first to third situations described above.

  The damaged position specifying unit 323 specifies the damaged position of the article in the house 10. The memory 33 may store in advance, for example, a floor plan of the house 10 represented in a two-dimensional coordinate space. The self-propelled cleaner 21 may create a floor plan by moving in the house 10 and transmit the created floor plan to the server device 3. The damage position specifying unit 323 specifies the coordinates on the floor plan of the sound source where the article is damaged as the damage position.

  The damaged position specifying unit 323 can more accurately specify the source of the sound of the damaged article by collecting the sound of the damaged article by the plurality of microphones. Further, the damaged position specifying section 323 may specify the position where the article is damaged from the image data captured by the second sensor 12.

  The appliance operation determining unit 324 determines a predetermined operation to be executed by the self-propelled cleaner 21 (self-propelled device) in the house 10 (space) according to the situation estimated by the situation estimation unit 322. Further, the device operation determining unit 324 may determine a predetermined operation to be performed by another device other than the self-propelled cleaner 21 according to the situation estimated by the situation estimation unit 322. Further, the device operation determining unit 324 determines a predetermined operation to be executed by the sensors configuring the sensor group 1 according to the status estimated by the status estimation unit 322 as well as the devices configuring the device group 2. Is also good.

  The device operation information storage unit 331 stores, in advance, device operation information that associates a situation in which an article has been damaged with an operation to be executed by the device.

  FIG. 3 is a diagram illustrating an example of device operation information stored in the device operation information storage unit according to the first embodiment.

  As shown in FIG. 3, an operation to be executed by the device is associated with a situation in which the article is damaged. In the first situation where an article slips from a person's hand during a daily operation, an operation of causing the self-propelled cleaner 21 to suck a damaged article and an operation of causing the display device 22 to present a substitute article for the damaged article Are associated with each other. Also, in the second situation where a plurality of persons are fighting, the self-propelled cleaner 21 outputs the soothing voices of the fighting persons while outputting the soothing voice, and the display device 22 has a restaurant suitable for reconciliation. Alternatively, an operation for presenting a movie is associated with the operation. Further, in the third situation where a suspicious person has entered, an operation of moving the self-propelled cleaner 21 while disturbing the feet of the suspicious person and an operation of imaging the suspicious person with the imaging device (second sensor 12). And an operation of transmitting, to the information device 23, image data obtained by imaging the suspicious individual and notification information for notifying that the suspicious individual exists to the police.

  The operation of the device associated with the situation where the article has been damaged is not limited to the above.

  The device operation determination unit 324 refers to the device operation information storage unit 331 and determines a predetermined operation to be executed by a device associated with the situation estimated by the situation estimation unit 322.

  When the first situation in which the article has slipped from the hand of the person during the daily operation is estimated, the device operation determining unit 324 performs the operation of the self-propelled cleaner 21 that sucks the damaged article and the replacement of the damaged article. The operation of the display device 22 for presenting the article is determined. In addition, when the second situation in which a plurality of people are fighting is estimated, the device operation determining unit 324 outputs the operation of the self-propelled vacuum cleaner 21 that moves while outputting sound soothing the plurality of fighting people. , The operation of the display device 22 for presenting a restaurant or movie suitable for reconciliation. In addition, when the third situation in which the suspicious person has entered is estimated, the device operation determination unit 324 causes the operation of the self-propelled cleaner 21 to move while disturbing the feet of the suspicious person and image the suspicious person. The operation of the imaging device (second sensor 12) and the operation of the information device 23 that causes the police to transmit image data obtained by imaging the suspicious individual and notification information for notifying that the suspicious individual is present are determined.

  Further, the appliance operation determination unit 324 controls the operation of the self-propelled cleaner 21. The appliance operation determining unit 324 generates control information (second information) for causing the self-propelled cleaner 21 (self-propelled device) to execute a predetermined operation in the space according to the estimated situation. Here, the control information is cleaning instruction information for causing the self-propelled cleaner 21 to clean damaged articles in the house 10 (space) according to the estimated situation. When determining the operation of the self-propelled cleaner 21 to suck the damaged article, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323, and The cleaning instruction information for causing the self-propelled cleaner 21 to clean the damaged article at the position is generated.

  The control information transmission unit 312 is control information (second information) for causing the self-propelled cleaner 21 (self-propelled device) to execute a predetermined operation in the house 10 (space) according to the estimated situation. Is output. The control information transmitting unit 312 transmits the cleaning instruction information generated by the device operation determining unit 324 to the self-propelled cleaner 21. Upon receiving the cleaning instruction information, the self-propelled cleaner 21 moves to the damaged position, captures an image of the suction target that is a damaged article at the damaged position, transmits the captured image data to the server device 3, and Aspirate things. The sensor data receiving unit 311 acquires information on an object to be suctioned by the self-propelled cleaner 21. For example, the information on the suction target is information on the appearance of the suction target. The information on the appearance includes an image captured by a camera included in the self-propelled cleaner 21. The image includes the suction target. The sensor data receiving unit 311 receives the image data of the suction target transmitted by the self-propelled cleaner 21.

  When the operation of the display device 22 for presenting the substitute article for the damaged article is specified by the device operation determining section 324, the damaged article specifying section 325 is based on the image data including the suction target received from the self-propelled cleaner 21. Then, the damaged article constituted by the suction target is specified. The suction target is a damaged article. The damaged article specifying unit 325 specifies the damaged article based on the appearance of the suction target. The damaged article specifying unit 325 recognizes the image including the suction target, and specifies the damaged article from the recognized image. The memory 33 may store in advance a table in which images of a plurality of articles are associated with the names (product names) of the plurality of articles. The damaged article specifying unit 325 compares the image data obtained by imaging the suction target with the images of the plurality of articles stored in the memory 33, and corresponds to the image of the article that partially matches the image of the suction target. The name of the attached article is specified as the name of the damaged article.

  The article information storage unit 332 stores article information on articles.

  FIG. 4 is a diagram illustrating an example of article information stored in the article information storage unit according to the first embodiment.

  As shown in FIG. 4, the article information includes a product number for identifying the article, a product name of the article, an article type, an article category, an article color, an article size, an article weight, an article material, and an article name. Includes prices, article manufacturers, and article sales outlets. The article information storage unit 332 stores a product number, an article name, an article type, an article category, an article color, an article size, an article weight, an article material, an article price, an article manufacturer, and an article sale. The article information associated with the store is stored.

  The article information shown in FIG. 4 is an example, and may include other information such as an image of the article. Further, all of the article information may be managed in one table, or may be managed in a distributed manner in a plurality of tables.

  The substitute article specifying unit 326 specifies a substitute article related to the damaged article based on the article information about the damaged article. The substitute article specifying unit 326 acquires the article information of the damaged article from the article information storage unit 332.

  For example, the replacement article may be the same article as the damaged article. In this case, the substitute article specifying unit 326 specifies the same article as the damaged article as a substitute article.

  Further, for example, the substitute article may be an article having the same attribute as the damaged article. In this case, the substitute article specifying unit 326 specifies an article having the same attribute as the damaged article as a substitute article. The attribute is, for example, the color, size, weight, or material of the article. The substitute article specifying unit 326 specifies an article having the same color, size, weight, and material of the damaged article as a substitute article. Note that the substitute article specifying unit 326 may specify, as a substitute article, an article having at least one of the same color, size, weight, and material of the damaged article.

  Further, for example, the substitute article may be an article having an attribute similar to the attribute of the damaged article. In this case, the substitute article specifying unit 326 specifies an article having an attribute similar to the attribute of the damaged article as a substitute article. The attribute is, for example, the color, size, or weight of the article. The substitute article specifying unit 326 specifies an article similar to at least one of the color, size, and weight of the damaged article as a substitute article. For example, a color similar to blue is bluish purple or the like, and a similar color is stored for each color in advance. In addition, the article having a size similar to the size of the damaged article is, for example, an article having a size within a range of −1 to +1 cm in width, depth, and height of the damaged article. In addition, the article of the size similar to the size of the damaged article is not limited to the article whose size is within the predetermined value range as described above. For example, the width, the depth, and the height of the damaged article are from −10% to + 10%. The size may be an article whose size is within a predetermined ratio range, such as an article whose size is within the range of%. Further, an article having a weight similar to the weight of the damaged article is, for example, an article having a weight within the range of −10 grams to +10 grams of the weight of the damaged article. Note that the article having a weight similar to the weight of the damaged article is not limited to, for example, an article whose size is within a predetermined value range as described above, but is within a range of −10% to + 10% of the weight of the damaged article. It may be an article whose size is within a predetermined ratio range, such as a heavy article.

  Further, for example, the substitute article may be an article having the same attribute as the damaged article and having a higher strength than the damaged article. In this case, the substitute article specifying unit 326 specifies, as a substitute article, an article having the same attribute as the damaged article and having a higher strength than the damaged article. The attribute is, for example, the color of the article. When the material of the damaged article is pottery, the substitute article specifying unit 326 specifies a metal article having the same color as the damaged article and having higher strength than the damaged article as a substitute article.

  Note that the memory 33 may include a user information storage unit that stores user information about the user. The user information includes a user ID for identifying the user, the user's name, the user's address, the user's date of birth, the user's blood type, the user's family structure, and the user's possession. The substitute article specifying unit 326 may specify a user who owns the damaged article specified by the damaged article specifying unit 325, and may acquire user information of the specified user from the user information storage unit. Then, the substitute article specifying unit 326 may specify a substitute article related to the damaged article based on the article information about the damaged article and the user information about the owner of the damaged article.

  For example, the user information includes owned article information indicating a plurality of owned articles owned by the owner. The substitute article specifying unit 326 specifies an article different in type from the damaged article from among the plurality of owned articles indicated by the owned article information, and at least identifies the specified article from among the plurality of sold articles sold. An article having the same attribute and the same type as the damaged article may be specified as a substitute article. Further, the user information may include house information indicating the position of the house of the owner. The substitute item specifying unit 326 may specify a substitute item that can be purchased at a store within a predetermined range from the position of the house indicated by the house information.

  The presentation information generation unit 327 generates presentation information on the substitute article specified by the substitute article specification unit 326. The presentation information includes an image indicating the appearance of the substitute article. In addition, the presentation information may include an object image for ordering a substitute article, together with an image indicating the appearance of the substitute article.

  The presentation information transmitting unit 313 outputs presentation information on the substitute article specified by the substitute article specifying unit 326. That is, the presentation information transmission unit 313 transmits the presentation information generated by the presentation information generation unit 327 to the display device 22. The display device 22 displays the received presentation information in a predetermined manner.

  When the second situation in which a plurality of persons are fighting is estimated, the control information generated by the device operation determining unit 324 indicates that the self-propelled cleaning is performed in the house 10 (space) according to the estimated situation. This is audio output instruction information for causing the machine 21 (self-propelled device) to output a predetermined audio. When the device operation determining unit 324 determines the operation of the self-propelled cleaner 21 that moves while outputting a sound that soothes a plurality of fighting persons, the self-propelled cleaner is moved to the damaged position specified by the damaged position specifying unit 323. In addition to moving the vacuum cleaner 21, voice output instruction information for moving the cleaner 21 to the self-propelled cleaner 21 while outputting a soothing voice to the self-propelled cleaner 21 at the damaged position is generated.

  The control information transmitting unit 312 transmits the audio output instruction information generated by the device operation determining unit 324 to the self-propelled cleaner 21. Upon receiving the voice output instruction information, the self-propelled cleaner 21 moves from the charging position to the damaged position, and outputs a sound soothing a plurality of fighting persons at the damaged position. At this time, the self-propelled cleaner 21 may output a sound soothing a plurality of fighting persons, and may suck the suction target (damaged article). The self-propelled cleaner 21 returns to the charging position after the suction of the suction target is completed or after a predetermined time has elapsed since the start of the sound output.

  When the operation of the display device 22 for presenting a restaurant or movie suitable for reconciliation is specified by the device operation determining unit 324, the service specifying unit 328 refers to the service information storage unit 333 and presents it to a plurality of fighting persons. Identify restaurants or movies to do.

  The service information storage unit 333 stores in advance service information including movie information on a movie being shown and restaurant information on a restaurant.

  FIG. 5 is a diagram illustrating an example of movie information stored in the service information storage unit according to the first embodiment.

  As shown in FIG. 5, the movie information includes a movie title, a movie genre, a screening place, a screening schedule, and vacant seat information. The service information storage unit 333 stores movie information in which movie titles, movie genres, screening locations, screening schedules, and vacant seat information are associated with each other.

  Note that the movie information shown in FIG. 5 is an example, and may include other information such as an actor appearing in a movie. Further, all of the movie information may be managed in one table, or may be managed in a distributed manner in a plurality of tables.

  The service specifying unit 328 specifies a movie to be presented to a plurality of fighting persons. The service specifying unit 328 specifies, for example, a movie that is a genre suitable for reconciliation and that can be viewed. Whether or not the movie is a genre suitable for makeover is preferably included in the movie information in advance. A movie that can be watched is a movie whose screening start time is after the current time and there are vacant seats. In addition, when the memory 33 stores in advance user information about each of a plurality of fighting persons, and the user information includes information about a favorite movie genre, the service specifying unit 328 refers to the user information. Alternatively, a movie corresponding to a favorite movie genre common to a plurality of fighting persons may be specified.

  FIG. 6 is a diagram illustrating an example of restaurant information stored in the service information storage unit according to the first embodiment.

  As shown in FIG. 6, the restaurant information includes a restaurant name, a cuisine genre, a restaurant location, a restaurant opening time, and vacant seat information. The service information storage unit 333 stores restaurant information in which restaurant names, cuisine genres, restaurant locations, restaurant hours, and vacant seat information are associated.

  Note that the restaurant information illustrated in FIG. 6 is an example, and may include other information such as a menu and an image in the restaurant. Further, all of the restaurant information may be managed in one table, or may be managed in a distributed manner in a plurality of tables.

  The service specifying unit 328 specifies a restaurant to be presented to a plurality of fighting persons. The service specifying unit 328 specifies, for example, a restaurant that is a genre suitable for reconciliation and that can eat. It is preferable that whether the restaurant is a genre suitable for reconciliation is included in the restaurant information in advance. In addition, a restaurant that can be eaten is a restaurant whose current time is within business hours and has vacant seats. Further, when the memory 33 previously stores user information on each of a plurality of fighting persons, and the user information includes information on a favorite genre of food, the service specifying unit 328 refers to the user information. Alternatively, a restaurant corresponding to a favorite cooking genre common to a plurality of fighting persons may be specified.

  In the first embodiment, the service specifying unit 328 may specify any one of a restaurant and a movie to be presented to a plurality of fighting persons, or to present it to a plurality of fighting persons. Restaurant and movie may be specified.

  In the first embodiment, the device operation determining unit 324 determines the operation of the display device 22 that presents a restaurant or a movie suitable for reconciliation when the second situation in which a plurality of persons are fighting is estimated. However, the present disclosure is not limited to this, and the device operation determining unit 324 may display the service suitable for reconciliation when the second situation in which a plurality of persons are fighting is estimated. May be determined. The service specifying unit 328 refers to the service information storage unit 333 and presents it to a plurality of fighting persons when the operation of the display device 22 for presenting a service suitable for reconciliation is specified by the device operation determining unit 324. Identify services.

  The presentation information generation unit 327 generates presentation information (third information) for causing the display device 22 (presentation device) to present information for changing the estimated situation. The presentation information generation unit 327 generates the presentation information on the restaurant or movie suitable for reconciliation specified by the service specification unit 328.

  The presentation information transmission unit 313 outputs presentation information for causing the display device 22 (presentation device) to present information for changing the estimated situation to the display device 22 (presentation device). Here, the presentation information transmission unit 313 transmits the presentation information relating to the restaurant or the movie specified by the service specification unit 328 to the display device 22. That is, the presentation information transmission unit 313 transmits the presentation information generated by the presentation information generation unit 327 to the display device 22. The display device 22 displays the received presentation information in a predetermined manner.

  When the third situation in which a suspicious person has entered is estimated, the control information generated by the appliance operation determination unit 324 is transmitted to the self-propelled cleaner 21 (self-propelled device) in the house 10 (space). This is disturbing operation instruction information for executing an operation for disturbing a suspicious individual. If the device operation determining unit 324 determines that the self-propelled cleaner 21 moves the suspicious person while disturbing the feet, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323. At the same time, disturbing operation instruction information for disturbing and moving the suspicious person's feet to the self-propelled cleaner 21 at the damaged position is generated.

  The control information transmitting unit 312 transmits the disturbance operation instruction information generated by the device operation determining unit 324 to the self-propelled cleaner 21. Upon receiving the disturbance operation instruction information, the self-propelled cleaner 21 moves from the charging position to the damaged position, and moves while disturbing the suspicious person's feet at the damaged position. At this time, the self-propelled cleaner 21 measures the distance to the suspicious person with a distance sensor, and moves while maintaining a predetermined distance from the suspicious person. Then, after the suspicious person has left the house 10 (space), the self-propelled cleaner 21 sucks the suction target (damaged article) and returns to the charging position. In addition, the self-propelled cleaner 21 may move so as to block the entrance of the house 10 in order to keep the suspicious person in the house 10 until the police arrive.

  In addition, when the operation of causing the imaging device (the second sensor 12) to image the suspicious person is specified, the sensor data receiving unit 311 images the suspicious person from the second sensor 12 disposed in the house 10 (space). Get image data. The sensor data receiving unit 311 outputs the image data to the notification information generating unit 329.

  The notification information generation unit 329 generates notification information for notifying that a suspicious individual exists. The notification information includes, for example, information indicating that a suspicious individual exists and the address of the house 10. The notification information generation unit 329 outputs the image data acquired by the sensor data reception unit 311 and the notification information to the notification information transmission unit 314.

  When the operation of the information device 23 for transmitting the image data of the suspicious person and the notification information for notifying that the suspicious person is present to the police is determined, the notification information transmitting unit 314 sends the sensor data receiving unit 311 Is transmitted to the information device 23, and the image data acquired as described above and notification information for notifying that a suspicious individual exists. The information device 23 receives the image data and the notification information from the server device 3 and transmits the received image data and the notification information to the server device managed by the police.

  When the self-propelled cleaner 21 detects that the self-propelled cleaner 21 has been damaged, the self-propelled cleaner 21 may transmit damage information indicating that the self-propelled cleaner 21 has been damaged to the server device 3. The sensor data receiving unit 311 may receive the damage information transmitted by the self-propelled cleaner 21. The notification information generator 329 may generate notification information (fourth information) for requesting the manufacturer to repair the self-propelled cleaner 21 when the damage information is received. The notification information transmitting unit 314 may transmit notification information for requesting the manufacturer to repair the self-propelled cleaner 21 to the information device 23. The information device 23 may receive notification information for requesting the manufacturer to repair the self-propelled cleaner 21 from the server device 3 and transmit the received notification information to a server device managed by the manufacturer.

  FIG. 7 is a first flowchart illustrating the operation of the server device according to Embodiment 1 of the present disclosure, and FIG. 8 is a first flowchart illustrating the operation of the server device according to Embodiment 1 of the present disclosure. 2 is a flowchart of FIG. Although FIG. 7 illustrates an example in which the damage is detected based on the sound data, the damage may be detected based on other sensor data such as image data as described above.

  First, the sensor data receiving unit 311 receives sound data from the first sensor 11 as sensor data (Step S1).

  Next, the damage detection unit 321 determines whether damage to an article in the house 10 has been detected using the sound data received by the sensor data reception unit 311 (step S2). At this time, when the frequency component of the sound data received from the sensor data receiving unit 311 matches the frequency component of the sound data in which the stored article is damaged, the damage detection unit 321 determines that the article is damaged. Detect. Here, when it is determined that the damage of the article has not been detected (NO in step S2), the process returns to step S1.

  On the other hand, when it is determined that the damage of the article has been detected (YES in step S2), the sensor data receiving unit 311 has been acquired within a predetermined period in the past from the time when the occurrence of the damage of the article was detected. A plurality of image data is received from the second sensor 12 as sensor data (step S3). Note that the sensor data receiving unit 311 requests the second sensor 12 to transmit a plurality of image data acquired within a predetermined period in the past from the time when the occurrence of the damage of the article is detected, and responds to the request. May receive a plurality of image data transmitted by the second sensor 12. Further, the sensor data receiving unit 311 may periodically receive image data from the second sensor 12 and store the received image data in the memory 33. When it is detected that the article has been damaged, the situation estimation unit 322 stores a plurality of image data acquired within a predetermined period in the past from the time when the occurrence of the article damage is detected. 33 may be read.

  Next, the situation estimating unit 322 estimates the situation in which the article has been damaged based on a plurality of image data acquired within a predetermined period in the past from the time when the occurrence of the article damage has been detected ( Step S4). In the first embodiment, the situation estimating unit 322 determines that the situation in which the article has been damaged is the first situation in which the article has slipped from the hand of a person during daily operation, and the situation in which a plurality of persons are fighting. It is estimated which of the two situations and the third situation where the suspicious person is invading.

  Next, the situation estimating unit 322 determines whether or not the estimation result is the first situation in which the article has slipped from the hand of the person during daily operation (step S5).

  Here, when the estimation result is determined to be the first situation in which the article has slipped from the hand of the person (YES in step S5), the damaged position specifying unit 323 specifies the damaged position of the article in the house 10. (Step S6).

  Next, the device operation determination unit 324 refers to the device operation information storage unit 331 and determines the operation of the device associated with the first situation estimated by the situation estimation unit 322 (Step S7). Here, when the estimation result is the first situation in which the article has slipped from the hand of the person, the device operation determining unit 324 determines the operation of the self-propelled cleaner 21 for sucking the damaged article, and the replacement article for the damaged article. And the operation of the display device 22 for presenting the

  Next, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323, and causes the self-propelled cleaner 21 to clean the damaged article at the damaged position. Is generated (step S8).

  Next, the control information transmitting unit 312 transmits the cleaning instruction information generated by the device operation determining unit 324 to the self-propelled cleaner 21 (Step S9). The self-propelled cleaner 21 receives the cleaning instruction information from the server device 3 and moves toward the damaged position included in the cleaning instruction information. When the self-propelled cleaner 21 arrives at the damaged position, the self-propelled cleaner 21 takes an image of the suction target with a camera, and transmits the image data of the taken image to the server device 3. When the self-propelled cleaner 21 transmits the image data including the suction target to the server device 3, the self-propelled cleaner 21 suctions the suction target.

  Next, the sensor data receiving unit 311 receives, as sensor data, image data including the suction target from the self-propelled cleaner 21 (step S10).

  Next, based on the image data including the suction target received from the self-propelled cleaner 21, the damaged product specifying unit 325 specifies the damaged product constituted by the suction target (Step S <b> 11). The damaged article specifying unit 325 compares the image of the plurality of articles stored in advance with the image of the suction target included in the image data, and recognizes a damaged article formed by the suction target. For example, when the suction target is a piece of a ceramic mug, the damaged article specifying unit 325 recognizes an image of the article that partially matches the image of the piece included in the image data, and corresponds to the image of the recognized article. Items to be damaged are specified as damaged items.

  Next, the substitute article specifying unit 326 acquires the article information of the damaged article from the article information storage unit 332 (Step S12).

  Next, the substitute article specifying unit 326 specifies a substitute article related to the damaged article based on the article information about the damaged article (Step S13). For example, the substitute article specifying unit 326 specifies the same article as the damaged article as a substitute article.

  Next, the presentation information generation unit 327 generates presentation information on the substitute article specified by the substitute article specification unit 326 (Step S14).

  Next, the presentation information transmission unit 313 transmits the presentation information generated by the presentation information generation unit 327 to the display device 22 (Step S15). The display device 22 receives the presentation information transmitted by the server device 3 and displays the received presentation information. The display device 22 displays the presentation information while the suction target is being sucked by the self-propelled cleaner 21. The display device 22 may display the presentation information together with the start of the suction of the suction target by the self-propelled cleaner 21. Further, the display device 22 may continuously display the presentation information even after the suction of the suction target by the self-propelled cleaner 21 is completed.

  FIG. 9 is a diagram for explaining the operation of the device in the first situation in which the article slips from the person's hand during a daily operation in the first embodiment.

  If the article 6 is broken as a result of slipping from the hand of the person 61 during daily operation, the self-propelled cleaner 21 moves to the damaged position and sucks the damaged article 6. Further, the display device 22 installed in the room displays the presentation information 221 including the image 222 for confirming whether or not to purchase the same substitute article as the damaged article 6.

  As shown in FIG. 9, the presentation information 221 includes, for example, a sentence “Would you buy a new mug?”, An image showing the appearance of the substitute article, and a button for switching to an order screen for ordering the substitute article. And an image 222 containing

  When generating the cleaning instruction information, the device operation determining unit 324 may generate presentation information for notifying the user that cleaning is to be started. In this case, the control information transmitting unit 312 transmits the cleaning instruction information generated by the appliance operation determining unit 324 to the self-propelled cleaner 21 and sends the presentation information generated by the appliance operation determining unit 324 to the display device 22. May be sent. The display device 22 may display presentation information for notifying a user that cleaning is to be started. In this case, the presentation information includes, for example, a sentence "Are you okay? Not introduced? I'm going to clean now."

  On the other hand, when it is determined in step S5 of FIG. 7 that the estimation result is not the first situation in which the article has slipped from the hand of the person (NO in step S5), the situation estimation unit 322 determines that the estimation result It is determined whether or not the second situation is that the person is fighting (step S16).

  Here, when it is determined that the estimation result is the second situation in which a plurality of persons are fighting (YES in step S16), the damaged position specifying unit 323 specifies the damaged position of the article in the house 10. (Step S17).

  Next, the device operation determination unit 324 refers to the device operation information storage unit 331 and determines the operation of the device associated with the second situation estimated by the situation estimation unit 322 (Step S18). Here, when the estimation result is the second situation in which a plurality of people are fighting, the device operation determining unit 324 outputs the self-propelled cleaner 21 that moves while outputting a sound soothing the plurality of fighting people. And the operation of the display device 22 for presenting a restaurant or movie suitable for reconciliation.

  Next, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323, and a plurality of persons fighting with the self-propelled cleaner 21 at the damaged position. Then, voice output instruction information for moving while outputting a soothing voice is generated (step S19).

  Next, the control information transmitting unit 312 transmits the voice output instruction information generated by the device operation determining unit 324 to the self-propelled cleaner 21 (Step S20). Upon receiving the voice output instruction information, the self-propelled cleaner 21 moves from the charging position to the damaged position, and outputs a sound soothing a plurality of fighting persons at the damaged position. Then, the self-propelled cleaner 21 sucks the suction target (damaged article). The self-propelled cleaner 21 returns to the charging position after the suction of the suction target is completed or after a predetermined time has elapsed since the start of the sound output.

  Next, the service specifying unit 328 refers to the service information storage unit 333 and specifies a restaurant to be presented to a plurality of fighting persons (step S21). The service specifying unit 328 refers to the user information stored in advance, specifies a genre of a favorite dish common to a plurality of fighting persons, and specifies a restaurant corresponding to the specified genre and capable of eating.

  In the process of FIG. 8, the service specifying unit 328 specifies a restaurant to be presented to a plurality of fighting persons. However, the present disclosure is not particularly limited to this. A movie to be presented may be specified.

  Next, the presentation information generation unit 327 generates presentation information on the restaurant suitable for reconciliation specified by the service specification unit 328 (Step S22).

  Next, the presentation information transmission unit 313 transmits the presentation information generated by the presentation information generation unit 327 to the display device 22 (Step S23). The display device 22 receives the presentation information transmitted by the server device 3 and displays the received presentation information. The display device 22 displays the presentation information while the sound is being output by the self-propelled cleaner 21. The display device 22 may display the presentation information together with the start of the output of the sound by the self-propelled cleaner 21. Further, the display device 22 may continuously display the presentation information even after the output of the sound by the self-propelled cleaner 21 is completed.

  FIG. 10 is a diagram for explaining the operation of the device in the second situation in which a plurality of persons are fighting in the first embodiment.

  When the article 6 is damaged while the plurality of persons 62 and 63 are fighting, the self-propelled cleaner 21 moves to the damaged position and outputs a sound soothing the fighting persons 62 and 63. . In FIG. 10, for example, the self-propelled cleaner 21 outputs a sound “OK, calm down”. Further, the display device 22 installed in the room displays presentation information 223 for presenting a restaurant suitable for reconciliation between the plurality of fighting persons 62 and 63.

  As illustrated in FIG. 10, the presentation information 223 includes, for example, a sentence “Do you not eat Italian food at M restaurant?” And a reservation button for reserving a restaurant. When the reservation button is pressed, the screen shifts to a reservation screen for reserving a restaurant.

  On the other hand, when it is determined in step S16 of FIG. 8 that the estimation result is not the second situation in which a plurality of persons are fighting (NO in step S16), the situation estimation unit 322 determines that the estimation result is a suspicious individual. It is determined whether or not is the third situation in which is invading (step S24).

  Here, if it is determined that the estimation result is not the third situation in which a suspicious person has entered (NO in step S24), that is, if it is not possible to estimate the situation in which the article has been damaged, the process proceeds. finish. Note that, when it is determined that the estimation result is not the third situation in which a suspicious person has entered, the device operation determining unit 324 may determine the operation of the self-propelled cleaner 21 that sucks the damaged article. .

  On the other hand, when it is determined that the estimation result is the third situation in which a suspicious person has entered (YES in step S24), the damaged position specifying unit 323 specifies the damaged position of the article in the house 10 (step S24). S25).

  Next, the device operation determination unit 324 refers to the device operation information storage unit 331 and determines the operation of the device associated with the third situation estimated by the situation estimation unit 322 (Step S26). Here, if the estimation result is the third situation in which a suspicious person has entered, the device operation determination unit 324 determines whether the operation of the self-propelled cleaner 21 that moves the suspicious person while disturbing his or her feet and the suspicious person Operation of the imaging device (second sensor 12) that causes the police to transmit image data of the suspicious individual and notification information for notifying that the suspicious individual exists to the police. I do.

  Next, the device operation determining unit 324 moves the self-propelled cleaner 21 to the damaged position specified by the damaged position specifying unit 323, and disturbs the foot of the suspicious person to the self-propelled cleaner 21 at the damaged position. The disturbance operation instruction information for moving while moving is generated (step S27).

  Next, the control information transmitting unit 312 transmits the disturbance operation instruction information generated by the device operation determining unit 324 to the self-propelled cleaner 21 (Step S28). Upon receiving the disturbance operation instruction information, the self-propelled cleaner 21 moves from the charging position to the damaged position, and moves while disturbing the suspicious person's feet at the damaged position. Then, after the suspicious person has left the house 10 (space), the self-propelled cleaner 21 sucks the suction target (damaged article) and returns to the charging position.

  Next, the sensor data receiving unit 311 receives image data of the suspicious individual captured from the second sensor 12 disposed in the house 10 (space) (Step S29).

  Next, the notification information generation unit 329 generates notification information for notifying that a suspicious individual exists (step S30).

  Next, the notification information transmission unit 314 transmits the image data acquired by the sensor data reception unit 311 and the notification information generated by the notification information generation unit 329 to the information device 23 (Step S31). The information device 23 receives the image data and the notification information from the server device 3 and transmits the received image data and the notification information to the server device managed by the police.

  FIG. 11 is a diagram for describing the operation of the device in the third situation in which a suspicious person has entered in the first embodiment.

  If the article 6 is damaged when the suspicious person 64 enters, the self-propelled cleaner 21 moves to the damaged position and moves while disturbing the feet of the suspicious person 64. In FIG. 11, for example, the self-propelled cleaner 21 moves around the suspicious individual 64 while maintaining a predetermined distance from the suspicious individual 64. Further, the second sensor 12 transmits image data of the suspicious person 64 to the server device 3. Further, the information device 23 receives image data of the suspicious individual 64 and notification information for notifying that the suspicious individual 64 is present from the server device 3, and the received image data and notification information are managed by the police. Send to server.

  After the information device 23 transmits the image data and the notification information to the server device managed by the police, the display device 22 installed in the room displays the presentation information 224 for presenting the fact that the police were notified. Is also good. As shown in FIG. 11, the presentation information 224 includes, for example, a sentence “reported to the police”.

  As described above, based on the first information acquired by at least one of the one or more sensors installed in the space, the situation where the article has been damaged is estimated, and according to the estimated situation, Second information for causing the self-propelled cleaner 21 to execute a predetermined operation is output. Therefore, when an article present in the space is damaged, the self-propelled cleaner 21 can cause the self-propelled cleaner 21 to execute a predetermined operation according to the situation in which the article is damaged.

  Note that the presentation information generation unit 327 may generate presentation information (fifth information) for causing the display device 22 (presentation device) to present information about the article for suppressing the occurrence of the estimated situation. Further, the presentation information transmission unit 313 transmits (outputs) presentation information (fifth information) for causing the display device 22 (presentation device) to present information on the article for suppressing the occurrence of the estimated situation to the display device 22 (presentation device). Is also good. For example, when the third situation in which a suspicious person is intruding is estimated, the presentation information generation unit 327 may generate presentation information on security products and transmit it to the display device 22.

(Embodiment 2)
The device control system according to Embodiment 1 includes one server device, but the device control system according to Embodiment 2 includes two server devices.

  FIG. 12 is a diagram illustrating a configuration of a first server device according to Embodiment 2 of the present disclosure, and FIG. 13 is a diagram illustrating a configuration of a second server device according to Embodiment 2 of the present disclosure.

  The device control system according to the second embodiment includes a first server device 3A, a second server device 3B, a gateway 5 (not shown), a first sensor 11, a second sensor 12, a self-propelled cleaner 21, a display device 22, and An information device 23 is provided. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. The sensor group 1 includes various sensors such as a first sensor 11 and a second sensor 12. The device group 2 includes various devices such as a self-propelled cleaner 21, a display device 22, and an information device 23. 12 and 13, the gateway 5 is omitted.

  The first server device 3A is communicably connected to the sensor group 1, the device group 2, and the second server device 3B via a network. The second server device 3B is communicably connected to the device group 2 and the first server device 3A via a network.

  The first server device 3A is operated by, for example, a platformer. The second server device 3B is operated by, for example, a third party.

  The first server device 3A includes a communication unit 31A, a processor 32A, and a memory 33A.

  The communication unit 31A includes a sensor data receiving unit 311, a control information transmitting unit 312, a notification information transmitting unit 314, a damaged article information transmitting unit 315, and a device operation information transmitting unit 316. The processor 32A includes a damage detection unit 321, a situation estimation unit 322, a damage position identification unit 323, a device operation determination unit 324, a damaged article identification unit 325, and a notification information generation unit 329. The memory 33A includes a device operation information storage unit 331.

  The damaged article information transmitting unit 315 transmits damaged article information indicating the damaged article specified by the damaged article specifying unit 325 to the second server device 3B.

  The device operation information transmission unit 316 transmits device operation information indicating the operation of the display device 22 for presenting a restaurant or movie suitable for reconciliation determined by the device operation determination unit 324 to the second server device 3B.

  The second server device 3B includes a communication unit 31B, a processor 32B, and a memory 33B.

  The communication unit 31B includes a presentation information transmission unit 313, a damaged article information reception unit 317, and a device operation information reception unit 318. The processor 32B includes a substitute article specifying unit 326 and a presentation information generating unit 327. The memory 33B includes an article information storage unit 332.

  The damaged article information receiving unit 317 receives the damaged article information transmitted by the first server device 3A. The substitute article specifying unit 326 specifies a substitute article related to the damaged article based on the damaged article information received by the damaged article information receiving unit 317.

  The device operation information receiving unit 318 receives the device operation information transmitted by the first server device 3A. When the device operation information indicating the operation of the display device 22 for presenting a restaurant or a movie suitable for reconciliation is received by the device operation information receiving unit 318, the service specifying unit 328 refers to the service information storage unit 333 and performs a fight. A restaurant or movie to present to a plurality of persons.

  In the second embodiment, the first server device 3A transmits the damaged article information and the device operation information to the second server device 3B. However, the present disclosure is not particularly limited thereto, and the second server device 3B Transmits a request for damaged item information and device operation information to the first server device 3A, and the first server device 3A transmits damaged item information and device operation information to the second server device 3B in response to the request. Good.

  In the second embodiment, the device control system may include a plurality of second server devices 3B.

  In each of the above embodiments, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  Some or all of the functions of the device according to the embodiment of the present disclosure are typically realized as an LSI (Large Scale Integration) that is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include some or all of them. Further, the integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, some or all of the functions of the device according to the embodiment of the present disclosure may be realized by a processor such as a CPU executing a program.

  In addition, the numbers used above are merely examples for specifically describing the present disclosure, and the present disclosure is not limited to the illustrated numbers.

  Further, the order in which the steps shown in the above flowchart are executed is merely an example for specifically describing the present disclosure, and may be any other order as long as a similar effect is obtained. . Also, some of the above steps may be performed simultaneously (in parallel) with other steps.

  The information processing method, the information processing device, and the information processing program according to the present disclosure can cause the self-propelled device to execute a predetermined operation according to a situation in which an article is damaged, and thus the device performs the predetermined operation. It is useful as an information processing method, an information processing apparatus, and an information processing program to be executed.

REFERENCE SIGNS LIST 1 sensor group 2 device group 3 server device 3A first server device 3B second server device 4 network 5 gateway 6 article 10 house 11 first sensor 12 second sensor 21 self-propelled cleaner 22 display device 23 information device 31, 31A , 31B communication unit 32, 32A, 32B processor 33, 33A, 33B memory 311 sensor data receiving unit 312 control information transmitting unit 313 presentation information transmitting unit 314 notification information transmitting unit 315 damaged article information transmitting unit 316 equipment operation information transmitting unit 317 breakage Item information receiving unit 318 Device operation information receiving unit 321 Damage detecting unit 322 Situation estimating unit 323 Damage position specifying unit 324 Device operation determining unit 325 Damaged item specifying unit 326 Alternative item specifying unit 327 Presentation information generating unit 328 Service specifying unit 329 Notification information Generator 331 Device Work information storage unit 332 article information storage unit 333 service information storage unit

Claims (12)

An information processing method in an information processing device,
Acquiring first information acquired in at least one of the one or more sensors installed in the space,
Based on the first information, detecting breakage of an article present in the space,
Based on the first information, estimating a situation in which the article has been damaged,
Outputting second information for causing the self-propelled device to execute a predetermined operation in the space according to the estimated situation;
Information processing method. The second information is information for outputting a predetermined sound to the self-propelled device in the space according to the estimated situation,
The information processing method according to claim 1. Further, outputting third information for causing the presentation device to present information for changing the estimated situation,
The information processing method according to claim 1. The self-propelled device is a self-propelled vacuum cleaner,
The second information is information for causing the self-propelled cleaner to clean the damaged article in the space according to the estimated situation,
The information processing method according to claim 1. The estimated situation is a situation in which a suspicious person has entered the space,
The second information is information for causing the self-propelled device to perform an operation of disturbing the suspicious individual in the space.
The information processing method according to claim 1. Further, acquiring image data of the suspicious person imaged from an imaging device arranged in the space,
Further, transmitting the obtained image data and notification information for notifying that the suspicious individual exists,
The information processing method according to claim 5. Further, when damage information indicating that the self-propelled device has been damaged is obtained, outputting fourth information for requesting repair of the self-propelled device,
The information processing method according to claim 5. Further, outputting fifth information for causing the presentation device to present information on the article for suppressing the occurrence of the estimated situation,
The information processing method according to claim 1. The one or more sensors include at least one of a microphone device and an imaging device installed in the space,
The first information includes at least one of sound data acquired by the microphone device and image data acquired by the imaging device,
The estimation of the situation is based on at least one of the sound data and the image data, and estimates the situation in which the article has been damaged.
The information processing method according to claim 1. The obtaining of the first information includes obtaining the first information at predetermined time intervals,
The estimation of the situation is based on a plurality of pieces of first information acquired within a predetermined period based on the time when the damage of the article occurs, and estimates the situation where the damage of the article has occurred.
The information processing method according to claim 1. An acquisition unit configured to acquire first information acquired in at least one of the one or more sensors installed in the space;
Based on the first information, a detection unit that detects damage to an article present in the space,
An estimating unit that estimates a situation in which the article has been damaged based on the first information;
An output unit that outputs second information for causing the self-propelled device to perform a predetermined operation in the space according to the estimated situation;
An information processing apparatus comprising: Acquiring first information acquired in at least one of the one or more sensors installed in the space,
Based on the first information, detecting breakage of an article present in the space,
Based on the first information, estimating a situation in which the article has been damaged,
An information processing program for causing a computer to function to output second information for causing a self-propelled device to execute a predetermined operation in the space according to the estimated situation.

Images