JP2022165192A - Area recognition system, moving object and area recognition device - Google Patents

Abstract

To provide an indoor floor plan recognition technology capable of coping with life scenes.SOLUTION: A floor plan recognition system 1 includes: a home appliance measurement unit 6 that acquires home appliance information such as sound, image, and temperature related to home appliances installed in a home; a home appliance estimation unit 21 for estimating home appliance attribute information of the home appliance and a position of the home appliance based on the home appliance information; a map processing unit 10 that divides an indoor map into a plurality of unit areas; a floor plan determination unit 11 that assigns a label to each divided unit area where the floor plan determination unit 11 associates each unit area with the position of the home appliance, and assigns a label corresponding to the home appliance attribute information to the unit area based on the home appliance attribute information of home appliances installed in each unit area.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for identifying attributes of unit areas that make up an area and recognizing the corresponding area. In particular, it relates to a technology for autonomously recognizing the layout of a facility such as a house (inside the house).

Note that typical examples of areas and unit areas include facilities such as buildings and houses, rooms, and floors.

At present, with the increase in the penetration rate of so-called IoT (Internet of Things) home appliances (including household mobile units), recognition technology for unit areas that make up areas such as the floor plan of a house is becoming important. For example, it is possible to purify the air in the bedroom with a mobile air purifier via a smart speaker, or use a monitoring robot to check the situation in the living room (such as an elderly person falling or a suspicious person entering). be done. In such a case, without floor plan information such as "bedroom" and "living room", the moving body cannot move to a designated place or operate at a designated place. Therefore, it is considered important to grasp the information about the unit area, for example, the floor plan information of the house.

Here, several techniques for acquiring floor plan information have been proposed. For example, Patent Literature 1 describes "a step of displaying an image captured by a camera of a mobile robot within an area of the house for the room identifications for at least two of the room identifications; accepting a selection of a room identification marker among the displayed plurality of room identification markers; assigning the selected room identification marker to the room identification; and displaying a user map based on the generated map. and allowing the mobile robot to navigate the planned path to the location corresponding to the designation of the destination room identification marker."

In Patent Document 2, "The floor plan identification unit includes a line segment detection unit that detects line segments corresponding to walls on the floor plan, a segmentation processing unit that identifies room regions that correspond to rooms on the floor plan, and a floor plan image. A character recognition unit that recognizes a character string contained in ".

JP 2016-103277 A Japanese Unexamined Patent Application Publication No. 2018-81548

However, in Patent Document 1, the user is presented with room identification markers that are icons and character strings of the living room, bedroom, and bathroom, the user selects the room identification marker, and the selected room identification marker is the room layout. For this reason, in Patent Document 1, there is a problem that the user needs to select the room identification marker, which is troublesome for the user.

On the other hand, in Patent Document 2, a character string (Western-style room, Japanese-style room, etc.) included in the image of the floor plan is recognized, and the recognized character string is used as the name of each room. In Patent Document 2, floor plan information can be automatically registered without user's trouble. Here, the usage of the room differs depending on the household. For this reason, the rooms are not always used according to the character strings on the floor plan. For example, a room described as a Western-style room in the floor plan image may be used as a kitchen with a refrigerator in some households, or may be used as a living room with a TV and a sofa in some households. Therefore, Patent Document 2 has a problem that the floor plan cannot be set according to the life scene.

Another problem of Patent Document 2 is that it cannot deal with floor plans and indoor maps that do not contain character strings. In particular, there are many cases where the character string shown in Patent Document 2 is not included in a home map created by a mobile object such as a robot cleaner. In such a case, it is difficult to deal with the technology disclosed in Patent Document 2.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to realize region recognition that is more realistic. As an example, the object is to provide a room layout recognition technology that can respond to various life scenes.

In order to solve the above problems, the present invention recognizes an area by identifying the "attribute=type" of the unit areas that make up the area based on the attributes of the object placed in the area. Also, it is desirable to use environmental information of the object to specify the attributes of the object. A more specific configuration of the present invention is as follows.

In an area recognition system for recognizing an area composed of a plurality of unit areas, an object measuring unit that acquires environmental information of an object existing in the area, and an object that indicates attributes of the object using the measured environmental information. estimating a unit area in which the object exists, using an object estimating unit that estimates attribute information and a position, an area processing unit that divides the area into a plurality of partial unit areas, and the estimated position of the object; and an area recognition system having an attribute identification unit that identifies an attribute of an estimated unit area using the object attribute information.

Further, the present invention includes a moving body and an area recognition device that constitute an area recognition system. A region recognition method using these is also an aspect of the present invention. Further, a program or program product for making them function as a computer, for example, a storage medium storing the program is also an aspect of the present invention.

According to the present invention, it is possible to recognize an area in accordance with the actual state of the target area.

1 is a schematic structural diagram of a floor plan recognition system according to a first embodiment; FIG. 1 is a schematic perspective view of a moving body according to Example 1. FIG. 6 is a flowchart showing home appliance attribute information and estimation processing for estimating the position of the home appliance according to the first embodiment. FIG. 4 is an explanatory diagram illustrating estimation of a direction of a sound source (home appliance) according to the first embodiment; FIG. 4 is an explanatory diagram illustrating estimation of a position of a sound source (home appliance) according to the first embodiment; 6 is a flowchart showing floor plan determination processing according to the first embodiment; FIG. 4 is a diagram showing an occupancy grid map used in Example 1; FIG. 4 is a diagram showing an occupancy grid map subjected to binarization processing in Example 1; FIG. 10 is a diagram showing an occupancy grid map subjected to contraction processing in Example 1; 4 is a diagram showing an occupancy grid map subjected to quadrilateral detection processing in Example 1. FIG. FIG. 10 is a diagram showing a home appliance list according to the first embodiment; FIG. FIG. 4 is a diagram showing room attribute determination rules according to the first embodiment; FIG. 11 is a schematic structural diagram of a floor plan recognition system according to a second embodiment; FIG. 10 is a diagram showing an occupation grid map including indoor obstacles according to Example 2; 10 is a flowchart showing home appliance attribute information, household goods attribute information, and a process of estimating the positions of home appliances and household goods according to Example 2. FIG. FIG. 11 is a flow chart showing floor plan determination processing according to the second embodiment; FIG. It is a figure which shows the equipment list|wrist which concerns on Example 2. FIG. FIG. 10 is a diagram showing room attribute determination rules according to the second embodiment; FIG. 11 is a flowchart showing a modification of the object attribute information estimation process according to the second embodiment; FIG.

Embodiment 1 of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a schematic structural diagram of a floor plan recognition system 1 according to the first embodiment. Hereinafter, in the present embodiment and the second embodiment, an example in which a house is a recognition target will be described as an area. Therefore, in each embodiment, the attribute of a room in a house is identified as a unit area. Therefore, in each embodiment, the floor plan recognition system is used as an example of the area recognition system.

As shown in FIG. 1(a), a floor plan recognition system 1 includes a moving object 2 for acquiring environmental information about objects in a house, and a floor plan analyzer 3 for recognizing the floor plan based on the environmental information and a home map. Consists of

Here, the mobile object 2 uses the home appliance information, which is a type of environmental information obtained by measuring the home appliance (hereinafter referred to as home appliance) by the home appliance measurement unit 6 provided therein, to determine the home appliance attribute information and its position. to estimate As described above, in this embodiment, home electric appliances are used as objects. Note that the objects include home appliances, furniture, and other household equipment, as well as objects from which environmental information can be acquired, such as people and animals.

In addition, the floor plan analysis device 3 uses the estimation result of the moving body 2 to specify a label, which is one type of attribute of each room. For this purpose, the floor plan analysis device 3 divides the indoor map of the house into a plurality of unit areas, and uses the position of the home appliance to associate each divided unit area with the home appliance. That is, the home appliances installed in each unit area are specified. Then, the floor plan analysis device 3 assigns a label, which is one type of attribute, to each unit area based on the home appliance attribute information of the home appliances installed in the unit area. As a result, the attribute of each unit area is identified, that is, the number of rooms is specified. Next, details of each component will be described. In addition, in this embodiment, since a home appliance is used as an object, home appliance attribute information, which is a kind of object attribute information, is used.
<Moving body 2>
The functional block of the moving body 2 is shown in FIG. 1(a), and the external appearance is shown in FIG. The moving object 2 includes a driving unit 4, a detecting unit 5, a home appliance measuring unit 6, a moving object control unit 7, and a communication unit 8, as shown in FIG. 1(a). Here, the drive unit 4 has a function of moving (running) the moving body 2 . Here, as shown in FIG. 2, the moving body 2 has a pair of driving wheels 15 and 16 and an auxiliary wheel 17 (not shown) provided in the lower part of the housing S of the moving body 2 . The driving wheels 15 and 16 are rotationally driven by wheel units (not shown) each composed of a traveling motor and a speed reducer. By rotating the driving wheels 15 and 16 according to the command from the moving body control unit 7, the moving body 2 is advanced, retreated, and turned. The auxiliary wheels 17 are driven wheels (casters) that freely rotate according to the drive wheels 15 and 16, and are provided near the outer periphery of the main body. Thus, the drive unit 4 is realized by the drive wheels 15 and 16, the auxiliary wheels 17, and the wheel unit.

The detection unit 5 has a function of measuring surrounding obstacles, steps, and the like when the moving body 2 travels. The detection unit 5 mainly includes an environment measurement sensor 18 for measuring obstacles, a floor distance measurement sensor 19 (not shown) for measuring the distance to the floor, and a sensor for detecting collisions with obstacles. It is implemented with a bumper sensor 20 (not shown).

Among them, the environmental measurement sensor 18 emits a laser beam from a light emitting part and emits the laser beam substantially horizontally from a rotating mirror (laser beam scanning type sensor). A part of the laser light is reflected by an obstacle such as a home appliance and returns, and is received by the light receiving section of the environmental measurement sensor 18 . Then, the environmental measurement sensor 18 calculates the distance to the obstacle from the time difference between the emission of the laser beam and the reception of the reflected light. Since the laser beam is generated while changing its direction from moment to moment due to the rotation of the mirror, it is possible to acquire distance data of obstacles within a predetermined angle range by using the environmental measurement sensor 18 . It should be noted that the wider the predetermined angular range, the wider the range of obstacles that can be detected, so 360 degrees is desirable. Further, the environment measurement sensor 18 may be an image sensor capable of measuring the status of obstacles in the environment, or a distance measurement device using millimeter waves or the like, other than the laser beam scanning type sensor.

The floor distance measuring sensor 19 is a distance measuring sensor using infrared rays for measuring the distance to the floor, and is installed on the lower surface of the mobile housing S. As shown in FIG. By detecting a large level difference such as a stair using the floor ranging sensor 19, it is possible to prevent the moving body 2 from falling or running off the wheel.

Further, the bumper sensor 20 is, for example, a photocoupler, and when the moving body 2 collides with an obstacle such as a wall, the moving body control unit 7 changes the running direction to the front/rear and left/right according to the force acting from the outside. , move away from obstacles.

Next, the home appliance measurement unit 6 has a function of performing measurement for estimating the home appliance attribute information and the position of the home appliance. Here, the home appliance attribute information is information identifying the type, which is the attribute of the home appliance, such as "refrigerator, washing machine". Therefore, the home appliance measurement unit 6 can be realized by, for example, a microphone. First, the microphone measures the sound generated when the home appliance is driven, and acquires the sound information as the home appliance information. Here, since the characteristics of the sound generated by the home appliance differ depending on the type, the home appliance attribute information can be specified by measuring the sound of the home appliance and acquiring the sound information. Furthermore, the position of the home appliance can also be calculated from the sound localization method.

Also, the home appliance measurement unit 6 may be an image sensor capable of acquiring an image of the home appliance. While the moving body 2 is running, the image of the home appliance in the running environment is measured. By image processing and machine learning of the measured image, the home appliance attribute information and the position of the home appliance can be estimated.

Also, the home appliance measurement unit 6 may be an infrared thermometer capable of measuring the temperature of the home appliance. For example, when an oven operates, the temperature inside the oven gradually rises and reaches a constant temperature. Here, the oven has the feature that the temperature of the upper and lower heating portions (heaters) is the highest. In this way, the temperatures and temperature distributions that occur when home appliances are in operation are different. Therefore, by acquiring temperature data during operation with an infrared thermometer and analyzing the temperature distribution, the home appliance attribute information and the position of the home appliance can be obtained. can be estimated.

The home appliance attribute information and the position estimation of the home appliance are performed by the home appliance estimation unit 21, which will be described later. For this reason, the home appliance estimation unit 21 can estimate by comparing the measurement result of the home appliance measurement unit 6 with pre-stored standard information. Standard information includes, for example, standard home appliance sounds.

In this embodiment, details of the home appliance measurement unit 6 will be described with a microphone array (composed of a plurality of microphones) as an example. The home appliance measurement unit 6 is configured with a microphone 50 and a microphone 51 as shown in FIG. The microphones 50 and 51 are devices that convert mechanical vibrations caused by sound waves into electrical signals, and collect sounds in the environment. If the microphones 50 and 51 have omnidirectionality, they can collect sounds in all directions, so that sounds can be measured over a wide range. Moreover, in the case of a directional microphone, it is desirable to install the microphone so that it can be directed in the direction of the greatest sensitivity in order to effectively collect sound. In addition, it is desirable to provide a moving structure such as a motor so that the microphone can be directed in the direction of the greatest sensitivity, and installed so that the sound collecting direction of the microphone can be changed.

Next, the moving body control unit 7 has a function of controlling the moving body 2, and can be realized by a microcomputer. As shown in FIG. 1B, the moving body control unit 7 includes a home appliance estimation unit 21, a self-position estimation unit 22, a map generation unit 23, an operation control unit 24, and a storage unit 25.

First, the home appliance estimation unit 21 uses the sensor data collected by the home appliance measurement unit 6 to estimate home appliance attribute information and its position. In this embodiment, the home appliance estimation unit 21 analyzes the frequency characteristics of the sound information of the home appliance acquired by the two microphones 50 and 51, and estimates the type of the home appliance. That is, the home appliance estimation unit 21 estimates home appliance attribute information. Also, the home appliance estimation unit 21 estimates the positional relationship between the home appliance and the home appliance measurement unit 6 based on the sound localization. A specific estimation method will be described later with reference to FIG. Note that the home appliance estimation unit 21 is a kind of object estimation unit that estimates an object.

Next, the self-position estimator 22 determines the self-position and posture of the moving body 2 from the arrangement of obstacles in the running environment. The self-position estimating unit 22 reads distance data to obstacles acquired by the environmental measurement sensor 18 while the mobile body 2 is moving, and calculates the self-position and attitude of the mobile body 2 from changes in the distance data. In addition, when calculating the self-position and orientation, the self-position estimation unit 22 uses information on the amount of movement of the moving body 2 detected by sensors such as encoders, lasers, and IMUs (Inertial Measurement Units) to estimate the self-position and orientation. can be used for calculations. Note that the self-position calculated here is the center point position of the moving body 2 (shown as the SP position in FIG. 2).

Next, the map creation unit 23 creates an environment map from the obstacle position information measured by the environment measurement sensor 18 . Here, the created map is stored in the storage unit 25 . This environmental map is an occupancy grid map in which a plane is divided into grids and the probability of an obstacle existing in each grid is expressed. The higher the probability that an obstacle exists, the larger the grid value, and the lower the probability that an obstacle exists, the smaller the grid value.

Next, the motion control unit 24 controls various motions of the moving body 2 . For example, the operation control section 24 controls the drive section 4 based on the driving environment information measured by the detection section 5 . In addition, the operation control unit 24 generates a travel route for collecting sounds, and controls operations such as making the moving body 2 travel.

Each function of the home appliance estimation unit 21, the self-position estimation unit 22, the map creation unit 23, and the operation control unit 24 can be realized by a program or a dedicated hardware circuit. When realized by a program, the program is stored in the storage unit 25, which will be described later, and the processor executes the calculation for realizing the function. Moreover, when it implement|achieves by a dedicated hardware circuit, FPGA (Field-Programmable Gate Array) and ASIC (Application Specific Integrated Circuit) can be utilized.

Next, the storage unit 25 stores information necessary for various types of control, such as a home map, which is map data of a house (home) to be recognized, and home appliance information.

The communication unit 8 is capable of inputting/outputting various signals. In this embodiment, the communication unit 8 realizes wireless communication including short-range wireless communication such as NFC (Near Field Communication). In this way, information transmission between units in the moving body 2 and information transmission between the moving body 2 and the floor plan analysis device 3 are performed via the communication unit 8 .

The above configurations described with reference to FIGS. 1(a), 1(b), and 2 are merely examples, and the configuration of the moving body 2 is not limited to these. In particular, the moving body 2 may have additional configurations to meet its usage. For example, if the moving body 2 is a vacuum cleaner or an air purifier, it may have a dust collector. Moreover, when the moving body 2 is a sterilization device, it may be provided with a sterilization unit. Moreover, when the moving body 2 is a housekeeping robot, it may be provided with an operation unit such as a hand or an arm. Moreover, when the moving object 2 is a watching robot, it may be provided with an output unit (speaker, display, etc.) for interacting with the user. Furthermore, the moving body 2 may further include sensors and devices including a temperature sensor, an illuminance sensor, and a gyro sensor. As described above, the sensors and devices included in the moving body 2 can be flexibly changed according to their use and operation.
<Floor plan analyzer 3>
Next, the floor plan analysis device 3, which is a type of area recognition device, will be described. The floor plan analysis device 3 is a device that can be realized by a computer, which is a type of information processing device. For example, software, data, etc. can be realized by cloud computing that can be used remotely through a communication network, or by user devices such as smartphones, tablets, and personal computers. Also, the moving body control section 7 of the moving body 2 may be provided with the same function.

Note that when the floor plan analysis device 3 is realized by cloud computing, that is, by a server, it is connected to the moving body 2 via the Internet. Also, this connection may be realized via a terminal device such as a smart phone. Furthermore, it is desirable for this server to process a plurality of moving bodies 2 and a plurality of houses.

Here, the floor plan analysis device 3 includes a communication section 9, a map processing section 10, and a floor plan determination section 11, as shown in FIG.

First, the communication unit 9 communicates with other devices and enables input/output of various signals and information. Information such as map data (in-house map), home appliance information, sensor data of the moving body 2 and operation status are received by the floor plan analysis device 3 via the communication unit 9 .

Next, the map processing unit 10 analyzes the indoor map input by the communication unit 9 . In the analysis, binarization processing, contraction processing, and quadrilateral detection are mainly performed by image processing to divide the indoor map, that is, the interior of the house into a plurality of unit areas. Specific functions of the map processing unit 10 will be described later with reference to FIG. The indoor map used by the map processing unit 10 uses map data created by the moving body 2 based on information from the home appliance measuring unit 6 or map data owned by the developer. This is an example of a region processing unit that divides a region.

The floor plan determination unit 11 associates each of the plurality of unit areas divided by the map processing unit 10 with the position of the home appliance. This association includes listing to create a list of unit areas and appliance locations. Then, the floor plan determination unit 11 assigns a label to each unit area based on the associated result and the room attribute rule. Here, the label in this embodiment is the layout of the room, and is an example of an attribute including the type of room such as "kitchen", "living room", and "bedroom". As a result, the room, which is the unit area, is identified, and the house, which is the area, can be recognized. That is, the floor plan determination unit 11 is an example of an attribute identification unit. The details of the floor plan determination process, which is the process of the floor plan determination unit 11, will be described later with reference to FIGS. 8 and 9. FIG.

A map processing unit 10 and a floor plan determination unit 11 are provided. These functions can be implemented by programs or dedicated hardware circuits. When implemented by a program, the program is stored in a storage device or storage medium (not shown), and the processor of the floor plan analysis device 3 executes calculations for implementing the function. Moreover, when it implement|achieves by a dedicated hardware circuit, FPGA (Field-Programmable Gate Array) and ASIC (Application Specific Integrated Circuit) can be utilized.

The above configuration is merely an example, and the configuration of the floor plan recognition system 1 is not limited to this example. At least part of the functions of the moving body 2 may be performed by the floor plan analyzer 3 . Also, at least part of the functions of the floor plan analysis device 3 may be performed by the moving body 2 . For example, the function of the map creation unit 23 of the moving body 2 is executed by the floor plan analysis device 3 . In this case, the sensor data acquired from the environment measurement sensor 18 is transmitted to the floor plan analysis device 3 via the communication units 8 and 9 . Then, the floor plan analysis device 3 creates a map. The configuration has been described above, and the processing of this embodiment will be described in detail below with reference to the drawings.
<Home appliance attribute information and home appliance location estimation processing>
First, an overview of the processing of this embodiment will be described. In this embodiment, the home appliance attribute information and the position of the home appliance are estimated, and using this result, the floor plan determination process for recognizing the area is performed. First, in the process of estimating judgment home appliance attribute information and the position of the home appliance, the home appliance attribute information and the position of the home appliance are estimated from the input signal obtained from the home appliance measurement unit 6 provided in the moving body 2 . In the floor plan determination process, the floor plan analysis device 3 divides the home map into a plurality of unit areas, associates the divided unit areas with the positions of the home appliances, and based on this correspondence relationship, labels corresponding to the home appliance attribute information. is given to each unit area.

The details of this embodiment will be described below. FIG. 3 is a flowchart showing an estimation process for estimating home appliance attribute information and the position of the home appliance.

In this estimation process, the home appliance attribute information and the position of the home appliance are estimated using the sound information obtained from the home appliance measurement unit 6 provided in the moving body 2 . Details of each step of the processing will be described below.

First, when this estimation process is activated in the moving object 2, sounds in the environment are measured by the microphones 50 and 51, which are an example of the home appliance measurement unit 6 (step S1). Here, there are three examples of the timing for starting this estimation process. (1) First, it is the timing at which the moving body 2 is caused to run inside the house at a specified time and the sound is detected during the running. In this case, by running the moving body 2, the sound of each place in the house can be measured. (2) Also, it is the timing at which the IoT home appliance among the home appliances transmits to the moving object 2 driving information indicating that an operating sound is being generated, and the moving object 2 acquires this driving information. For example, the moving body 2 may activate the determination process at the timing of receiving information that the washing operation of the washing machine has been activated. By using the operation information of the home appliance, the moving body 2 can more reliably acquire the operation sound of the home appliance. (3) It is also the timing at which the microphones 50 and 51 detect new sounds in the environment. For example, the determination process may be activated at the timing when it is detected that the washing operation of the washing machine has been activated in a quiet environment and the sound pressure level has increased.

By activating the estimation process at the above timing, it is possible to more reliably measure the operation sound of the home appliance. Here, the operation sound of the home appliance is the sound generated when the home appliance is operated. For example, operating sounds of a washing machine include motor sound and vibration sound during operation, presentation sound, and stirring sound of water and clothes during operation.

Next, the home appliance estimation unit 21 determines whether or not sound information has been obtained by measuring sound (step S2). If it is determined that the sound information has not been acquired (No), the process returns to step S1, and the home appliance measurement unit 6 continues to measure the sound. If it is determined that the sound information has been acquired (Yes), the process proceeds to step S3.

Next, the home appliance estimation unit 21 estimates the direction of the sound source using the acquired sound information (step S3). In other words, the home appliance estimation unit 21 estimates the direction of the home appliance emitting the sound. This direction estimation will be described below. In this embodiment, the home appliance estimating unit 21 estimates the direction of the sound source based on the arrival time difference of the sound measured by the microphones 50 and 51 .

Here, estimation of the direction of the sound source will be described with reference to FIG. In FIG. 4, a sound source N on the upper right (θ direction) of the moving body 2 (housing S) in the drawing causes the operation sound of the home appliance. Here, a sound wave that reaches the housing S from the θ direction at the speed of sound c is received by two microphones 50 and 51 arranged at an interval d. At this time, the arrival time difference T between the microphones 50 and 51 of the sound waves is as shown in (Equation 1).
T=d cos(θ)/c (Equation 1)
Therefore, the direction θ of the sound source can be expressed by Equation 2) below. In other words, Equation 2 makes it possible to estimate the direction of the sound source.
θ = arccos(T·c/d) (equation 2)
Returning to FIG. 3, the description of the estimation process is continued. The motion control unit 24 moves the moving body 2 in the direction of the sound source while avoiding obstacles (step S4). For this purpose, first, the traveling environment of the moving object 2 is measured by the detection unit 5 . For example, the environment measurement sensor 18 detects an obstacle or a wall, and the floor distance sensor 19 detects a step. Then, the operation control unit 24 uses the traveling environment to determine the location where the vehicle cannot travel and the location where the vehicle can travel. Then, the operation control unit 24 moves the moving body 2 in the direction θ of the sound source while avoiding the place where travel is impossible or passing through the place where travel is possible. During this movement, the self-position estimator 22 calculates the position and orientation of the moving body 2 . Then, the motion control unit 24 controls the movement of the moving body 2 using the calculated position and orientation.

Next, the home appliance estimation unit 21 estimates the home appliance attribute information of the home appliance that is the sound source and its position (step S5). For this purpose, the self-position estimation unit 22 first detects that the moving body 2 has reached the vicinity of the sound source. Here, the neighborhood may satisfy a predetermined condition. The motion controller 24 then moves the mobile 2 to a number of different locations within the vicinity. Next, the microphone 50 or the microphone 51 collects the operation sound of the home appliance. Then, the home appliance estimation unit 21 estimates the home appliance attribute information and the position of the home appliance from the collected operating sound.

In this way, by moving the moving body 2 to the vicinity of the sound source and collecting the sound in the vicinity, it is possible to collect the operating sound of the home appliance used for home appliance identification with good quality. Furthermore, in order to acquire the operation sound of the home appliance with high quality, it is desirable that the operation control unit 24 stops the motors of the dust collection unit and the drive wheels of the moving body 2, for example. In other words, it is desirable to reduce the sound generated by the moving body 2 .

Details of the estimation of home appliance attribute information will now be described. In order to perform this estimation, operation sounds of various home appliances are stored as standard home appliance sounds in a database such as the storage unit 25 for each attribute (type). Then, the home appliance estimation unit 21 collates the stored operation sound with the frequency characteristics of the operation sound collected by the home appliance measurement unit 6 of the moving object 2 . As a result, the home appliance estimation unit 21 determines the home appliance attribute information. For example, when the home appliance is a washing machine, the frequency is wide in the range of 1 kHz to 8 kHz during the washing operation, and the frequency is in the range of 40 to 50 Hz during the dehydration operation. Also, the microwave oven is characterized by operating at a constant frequency of about 2 kHz.

For this reason, the home appliance estimation unit 21 compares the operation sound collected by the home appliance measurement unit 6 with the standard home appliance sound, and determines the home appliance attribute of the standard home appliance sound with the smallest total squared error by a method such as the least squares method. Adopt information. In this way, the home appliance can be identified using the characteristics of each home appliance.

Also, a neural network such as a CNN (Convolutional neural network) may be trained using stored operating sounds (standard household sounds) of various home appliances as learning data to generate feature models of operating sounds of home appliances. Then, the home appliance estimation unit 21 may use a method of inputting the collected operation sound into the feature model and estimating the attribute of the home appliance that matches the feature of the collected operation sound. In this process, since the operation sound differs depending on the attribute of the home appliance, the home appliance can be identified using the characteristics of the operation sound. In the present embodiment, home appliance attribute information is used in floor plan determination processing, which will be described later. This is because the home appliances to be placed in the room differ depending on how the room is used. For this reason, if it is possible to estimate the electrical appliances placed in each room, it is possible to determine the attribute of the room, that is, to assign a label.

Next, estimation of the position of the home appliance in step S5 will be described. The position of the home appliance, that is, the sound source is estimated by the home appliance estimation unit 21 using the position of the moving body 2 estimated by the self-position estimation unit 22 and the above (Equation 1) and (Equation 2). Details of this will be described with reference to FIG. As shown in FIG. 5(a), the moving body 2 uses the microphone 50 and the microphone 51 to pick up the operation sound near the sound source (on the right side of the drawing). Then, the home appliance estimation unit 21 applies the collected sound result to (Equation 1) and (Equation 2) to calculate the sound source direction θ1. After that, the operation control unit 24 moves the moving body 2 to a place different from the place where the sound was collected (left side of the drawing). That is, the center point position of the moving body 2 changes from SP1 to SP2. Next, the movement amount L of the center position of the moving body 2 is calculated by the self-position estimation unit 22 . Also, the microphone 50 and the microphone 51 are used to pick up the operation sound again at the destination. Then, the home appliance estimation unit 21 applies the collected sound result to (Equation 1) and (Equation 2) to calculate the sound source direction θ2.

Finally, as shown in FIG. 5B, the home appliance estimation unit 21 estimates the position of the sound source N relative to the moving object 2 based on the sine law using the sound source direction θ1, the sound source direction θ2, and the movement amount L. . Specifically, before the movement, the distance L1 between the sound source N and the center position SP1 of the moving body 2 can be calculated by the following (Equation 3).
L1=(L×sinθ2)/sin(π+θ1) (Equation 3)
After the movement, the distance L2 between the sound source N and the center position SP2 of the moving body 2 can be calculated by (Formula 4).
L2=(L×sin(π-θ1))/sin(π+θ1) (Formula 4)
Note that estimation of the position of the sound source, that is, the home appliance is not limited to this method. For example, the home appliance estimation unit 21 may estimate the position of the sound source using the phase difference or volume difference of the operation sound. This completes the description of step S5, and the description of the flowchart shown in FIG. 3 is continued.

Next, the home appliance estimation unit 21 estimates the indoor position of the home appliance (step S6). Here, the in-house position indicates the relative position in the in-house area, that is, the position in the coordinate system of the in-house map of the storage unit 25 . Note that the in-house position may be directly estimated in step S5.

The details of step S6 will be described below. As a flow for estimating the indoor position of the home appliance, first, the self-position estimation unit 22 estimates the indoor position (coordinates) of the moving body 2 in the coordinate system of the indoor map. After that, the home appliance estimating unit 21 coordinate-transforms the position of the sound source (home appliance) by coordinate transformation of the position of the sound source indicated by the positional relationship between the sound source and the moving body 2 calculated in step S5. That is, the home appliance estimation unit 21 can calculate the indoor position of the home appliance.

This completes the description of the process of estimating the home appliance attribute information and the position of the home appliance according to the present embodiment, but the present process is not limited to the above description. In this embodiment, the operation sound is used as the home appliance information, but other sounds such as the guidance voice of the home appliance may be used. Also, images, humidity, and temperature may be used in addition to sound. Alternatively, the identification information received by the moving body 2 from the home appliance via short-range wireless communication or the like may be used. Furthermore, a plurality of pieces of home appliance information may be combined and used. Details of this will be described later in a third embodiment.

Furthermore, objects other than home electric appliances may be used as objects. For example, equipment such as a person or a water supply may be used. Note that the home appliance information is the same in the second and third embodiments, which will be described later.

By the above processing, the home appliance attribute information and the position of the home appliance can be estimated.
<Room layout recognition processing>
Next, the floor plan recognition processing will be described. First, an outline of the floor plan determination process will be described. In this determination process, the floor plan analysis device 3 divides the home map into a plurality of unit areas, associates the plurality of divided unit areas with the position of the home appliance, and determines the home appliance attribute of the home appliance installed in each unit area. Based on the information, a label corresponding to the home appliance attribute information is given to the corresponding unit area.

Next, the details of the floor plan recognition processing by the floor plan analyzer 3 will be described with reference to the flowchart of FIG. 6 showing the floor plan recognition processing.

First, when the room layout recognition process is activated by the floor plan analysis device 3, this process is started. Then, the communication unit 9 acquires a home map, which is an example of a recognition target area, from the storage unit 25 of the mobile object 2, the server, or the user device (step S10).

Next, the map processing unit 10 determines whether an in-house map can be acquired and whether there is an in-house map in the storage unit 25, server, or user device (step S11). If it is determined that the indoor map cannot be acquired (NO), the process proceeds to step S12. Then, the map processing unit 10 transmits an instruction to create a home map to the moving body 2 via the communication unit 9 (step S12). As a result, the moving body 2 creates an indoor map.

If it is determined in step S11 that the in-house map has been obtained (YES), the process proceeds to step S13.

Next, the map processing unit 10 divides the indoor map (step S13). The in-house map division processing in step S13 will be described in detail below with reference to FIGS. 7A to 7D. First, the home used in this embodiment will be described. In this embodiment, the occupancy grid map shown in FIG. 7A is used as the indoor map. In this occupancy grid map, the higher the probability that an obstacle exists, the larger the grid value, and the lower the probability that an obstacle exists, the smaller the grid value. As shown in FIG. 7A, the wall 30, the object 31, and the object 32 are obstacles, so the grid value is large, and the grid value is small in the unit area 33 where there is no obstacle and the vehicle can travel. Therefore, in FIG. 7A, the color depth of each pixel of the image represents the magnitude of the value of the random variable of each grid of the occupancy grid map. That is, grids without obstacles are represented in white, and grids with obstructions are represented in black. Here, FIGS. 7A to 7D show that the house (house) to be recognized is composed of three unit areas (rooms) of unit area 38, unit area 39, and unit area 40. FIG.

The processing of step S13 will be described below. The map processing unit 10 divides the map into several unit areas by performing binarization processing, contraction processing, and quadrilateral detection processing on the occupation grid map shown in FIG. 7A). Although it is usually divided into a plurality of unit areas, it may become a single unit area.

First, in the binarization process, the map processing unit 10 sets the luminance value of each pixel of the image to 1 (white) when the luminance value of each pixel in the image is equal to or greater than the threshold, and sets the luminance value to 0 (white) when the luminance value is less than the threshold. black). FIG. 7B shows the result of the binarization process performed by the map processing unit 10 on the occupation grid map shown in FIG. 7A. In FIG. 7B, luminance values of 34, 35, and 36 corresponding to wall 30, object 31, and object 32 are 0, and luminance value of location 37 corresponding to unit area 33 without obstacles is 1. In FIG. By the binarization process, the luminance value of the image becomes a binary value of 0 or 1 (black pixel or white pixel), thereby reducing the amount of data. This facilitates image processing, which will be described later.

In the next erosion process, the map processing unit 10 slides the kernel (used for filtering and 2D convolution) on the binarized image of FIG. 7B. In other words, this process is a process of replacing a pixel of interest with a black pixel if there is even one black pixel in a predetermined neighborhood such as 4 or 8 neighborhoods of the pixel of interest. This shrinking process can remove small obstacles in the map. As a result, objects 35 and 36 are erased as shown in FIG. 7C.

In the next quadrilateral detection process, the map processing unit 10 converts the black pixels in FIG. 7C into white pixels and the white pixels into black pixels by binarization processing. The map processing unit 10 then divides FIG. 7C into several unit areas using the Watershed algorithm. Here, the Watershed algorithm is widely used as an image area segmentation method. This algorithm was first introduced in geology to describe geological morphology and features. The luminance values of pixels in the image and their minimum and maximum values correspond to the altitude above sea level, valley bottom and peak, respectively. Considering the image as a terrain, water is injected from the bottom of the valley, and the adjoining parts of multiple bodies of water formed are boundaries called watersheds. Specifically, it is flooded, and the area below the water level is called a marker. Then, when the water level rises, the markers expand, and when the markers overlap each other, the position is specified as a watershed. In other words, in the present embodiment, the map processing unit 10 uses the threshold value of the pixel luminance value as the water level, and while increasing the threshold value, tracks the unit areas below the threshold value, searches for the boundary where different unit areas overlap, and obtains the image split the After that, the quadrilaterals in the divided area are extracted. Specifically, a quadrilateral is determined under the following conditions. Condition 1: The number of corners is four. Condition 2: A convex shape. FIG. 7D shows the result of the above processing, that is, the result of quadrangle detection processing. FIG. 7D shows that unit areas 38 to 42 have been detected.

After that, the map processing section 10 obtains the area of each divided unit area, extracts the unit area whose area is equal to or larger than the threshold value (1 m ² ), and assigns a number to it. Since the area of a room in a general household is often 1 m ² or more, a unit area of 1 m ² or more is regarded as a room in this embodiment. Therefore, the map processing unit 10 assigns numbers such as "1" and "2" to the unit areas 38, 39, and 40 in FIG. 7D because they are 1 m ² or more. Note that 1 m ² is an example of a threshold, and other numerical values may be used. Unit regions 41 and 42 in FIG. 7D are not numbered because their areas are 1 m ² or less.

Then, the map processing unit 10 stores in the memory the occupied grid map shown in FIG. 7A and the unit areas 38 to 40 numbered in the quadrilateral detection process. Here, the memory is a storage unit provided in the floor plan analysis device 3 .

Note that the above map division processing is an example, and other methods may be used. For example, a neural network may be trained using the occupancy grid map shown in FIG. 7A and the coordinate array of each unit area in the map as training data to create a model capable of identifying quadrilateral features. A new map is input to the created model, and the quadrangle unit area in the map is detected by the model. A map may be divided into unit areas by detecting rectangles using such machine learning.

With this, the description up to step S13 is completed, and returning to FIG. 6, the description of the floor plan recognition processing is continued. After the home map is divided in step S13, the home appliance attribute information and the home position of the corresponding home appliance are acquired by the communication unit 9 (step S14). That is, the communication unit 9 receives from the mobile body 2 the home appliance attribute information estimated by the mobile body 2 according to the flowchart shown in FIG. Note that step S13 and step S14 may be processed continuously, or the process may be temporarily terminated or interrupted at step S13, and the process after step S14 may be executed as a separate process.

Next, the floor plan determination unit 11 associates each of the divided unit areas and home electric appliances arranged in each unit area (step S15). It is desirable that the floor plan determination unit 11 creates a home appliance list 80 shown in FIG. 8 in accordance with this correspondence.

The electric home appliance list 80 shown in FIG. 8 will be described below, and then the preparation of the electric home appliance list 80 in step S15 will be explained using FIG.

First, FIG. 8 is a diagram showing a home appliance list 80. As shown in FIG. The first column of the table in FIG. 8 indicates the unit area number that identifies each unit area. is. In FIG. 8, this unit area No. , the unit area 38 in FIG. 7D is numbered "1", and the unit area 39 is numbered "2". The second column of the table in FIG. 8 is the unit area range indicating the range of each unit area. The unit area range is represented by a coordinate array on the indoor map. In this embodiment, the lattices within the unit area are represented by 1, and the lattices outside the unit area are represented by 0s.

The creation of the home appliance list 80 will be described below. The floor plan determination unit 11 compares the coordinate array of each unit area with the position of each home appliance, and extracts home appliances arranged in each unit area. The floor plan determination unit 11 enters the extracted home appliances in the third column (household appliances in unit area) of the table in FIG. For example, when it is determined that a range, a refrigerator, and an oven are arranged in the unit area "1", the floor plan determination unit 11 assigns the range, Fill the refrigerator and oven. Then, the floor plan determination unit 11 stores the home appliance list 80 created in this manner in the storage unit of the floor plan analysis device 3 .

This completes the description of step S15, returning to FIG. 6 to continue the description of the floor plan determination process. The floor plan determination unit 11 assigns a label to each unit area according to the room attribute determination rule 90 (step S16). That is, in this step, the floor plan determination unit 11 identifies an attribute indicating the type of each unit area.

Here, FIG. 9 shows a room attribute determination rule 90. As shown in FIG. The room attribute determination rule 90 is a type of area attribute determination rule and is stored in the storage unit of the floor plan analyzer 3 . The room attribute determination rule 90 indicates the correspondence relationship between labels and electrical appliances. Here, the label is a kind of attribute of the unit area and indicates the attribute of the room that is the unit area. Also, the home appliance in the room attribute determination rule 90 indicates home appliance attribute information, which is the type of home appliance. A specific example thereof will be described below with reference to FIG. "Kitchen", "laundry", and "living room" shown in FIG. 9 are labels representing room attributes. In addition, microwave ovens, refrigerators, and electric fryers are home appliances that correspond to "kitchens." This room attribute determination rule 90 is determined for a room attribute and a home electric appliance having a larger correlation than a reference for the room attribute. For example, in "kitchen", there is a high possibility that electric appliances such as a range, refrigerator, oven, dishwasher, rice cooker, coffee maker, and electric fryer are present, and the correlation is high with "kitchen". Thus, the above various household appliances correspond to the label "kitchen". Other labels are determined similarly.

Then, in step S16, the floor plan determination unit 11 collates the room attribute determination rule 90 with the home appliance list 80, and determines the unit area No. of the home appliance list 80. A corresponding label is given to each unit area indicated by . For example, the unit area "1" in the home appliance list 80 includes a range, a refrigerator, and an oven. Also, since it matches the home appliance corresponding to "kitchen" in the room attribute determination rule 90, the floor plan determination unit 11 assigns the room attribute "kitchen" to the unit area "1".

Here, even if the electrical appliance list 80 and the room attribute determination rule 90 are collated in each unit area, they may not completely match each other. In that case, the label with the largest number of matching electrical appliances is assigned to the unit area. For example, unit area "1" in FIG. 8 may contain a washing machine in addition to a range, a refrigerator, and an open space. Among these four home appliances, the three home appliances corresponding to "kitchen" in the room attribute determination rule of FIG. 9 are the range, refrigerator, and open. A home appliance corresponding to "laundry" in the room attribute determination rule 90 is a washing machine. Since the number of home appliances that match "kitchen" is greater than the number of home appliances that match "laundry", "kitchen" is assigned as a label for the unit area. As described above, in this embodiment, using the room attribute determination rule 90, which is a type of attribute determination rule, a label, which is a type of attribute of a unit area, is determined according to the number of home electric appliances, which are objects indicated by the object attribute information. can be identified. Through the above processing, the room attribute of each unit area in the house can be determined. Furthermore, the "household appliance" of the room attribute determination rule 90 may be provided with "essential". In other words, when a home appliance described as essential is presumed, a corresponding label is given.

In addition, there are cases where the user wants to change the floor plan or the floor plan is erroneously identified depending on the life scene. This embodiment also provides a means for the user to modify the floor plan. For example, the floor plan analysis device 3 accepts a user's vocal floor plan correction instruction using a microphone connected to the floor plan analysis device 3, in addition to the microphones 50 and 51, and changes the label. In this case, information such as "change the bedroom to a child's room" is transmitted to the communication unit 9 of the floor plan analyzer 3 through the above-described microphone, and the floor plan analyzer 3 changes the label. Also, the user may change the label of the floor plan on the map of the house. For example, the indoor map and floor plan information may be transmitted to an output device such as a smart phone or tablet, and the label may be changed by the user on the output device. Furthermore, the floor plan analysis device 3 or the input device of the moving body 2 may receive correction instructions other than voice.

In this case, the home appliance estimation unit 21 determines the home appliance attribute information and its position from the input signal obtained from the home appliance measurement unit 6 provided on the moving body 2 . Then, the map processing unit 10 divides the indoor map into a plurality of unit areas, associates the plurality of unit areas with the positions of the home appliances, and calculates the home appliance attributes based on the home appliance attribute information of the home appliances installed in the unit areas. By assigning a label corresponding to information to the unit area, the unit area can be identified as a room. Further, by identifying the floor plan of each unit area using the home appliance information, it is possible to acquire the floor plan information that is applied to the life scene. This completes the description of the first embodiment.

Next, Example 2 will be described. In the first embodiment, region recognition, that is, room layout determination processing is performed using home appliance attribute information based on home appliance information. On the other hand, in the second embodiment, the facility equipment attribute information including the home appliance attribute information and household goods attribute information is used as the object attribute information to perform the floor plan determination process. This household property attribute information can be acquired by an obstacle analysis device, an image sensor, an odor measurement device, a humidity measurement device, or the like. In other words, these devices are used in addition to the microphones 50 and 51. FIG. By combining the home appliance attribute information and the family attribute information, the accuracy of floor plan recognition can be improved. In a second embodiment, a technique for realizing floor plan recognition using a device capable of acquiring room attribute information other than the home appliance measurement unit 6 will be described.

FIG. 10 is a schematic structural diagram of the floor plan recognition system of this embodiment. As shown in FIG. 10, a furniture attribute information acquisition unit 60 is added to the floor plan recognition system of the first embodiment. The household property information acquisition unit 60 will be described below. The household property information acquisition unit 60 includes an analysis device that can analyze obstacles in the house, an image sensor that can acquire equipment such as furniture and indoor structures, an odor measurement device that can acquire the smell and humidity in the house, and a humidity measurement device. can be realized by For example, the household property information acquisition unit 60 uses an obstacle analysis device to analyze the size, distribution, and shape of obstacles in the home. This makes it possible to identify obstacles. FIG. 11 shows an occupancy grid map with identified indoor obstacles. This occupation grid map is created by the map creating section 23 . As shown within the dashed frame in FIG. 11, there are 16 scattered small obstacles (black dots within the frame in FIG. 11). The size of each obstacle in this example is approximately 3 cm to 5 cm. In a general dining set, the long diameter of the legs of the table and chairs is 3 cm to 5 cm, and the tables and chairs are also distributed within a close range. This feature is consistent with the features of the 16 scattered small obstacles shown in the dotted box in FIG. 11 (black dots in the box in FIG. 11).

Therefore, by using the obstacle analysis device, the home appliance estimation unit 21 can analyze the distribution and dimensions of obstacles in the home, and determine furniture such as a dining set and a bed. For example, when an image sensor is used as the household property information acquiring unit 60, the household property information acquiring unit 60 measures objects in the environment while the moving body 2 is running. Then, the home appliance estimation unit 21 can identify furniture such as a sofa, a chair, a table, a door, and equipment in the house from the measured image. The household property information acquisition unit 60 may be integrated with the home appliance measurement unit 6 as an object measurement unit. In this case, home appliance attribute information and household property attribute information can also be treated as equipment attribute information.

Next, home appliance attribute information, household goods attribute information, home appliance and household goods position estimation processing (hereinafter, simply estimation processing), and room layout recognition processing in this embodiment will be described with reference to FIGS. 12 and 13. FIG. The outline of the processing of this embodiment will be described below, and then the processing of each step will be described according to the flowchart.

First, an overview of the processing of this embodiment will be described. In the estimation process, from the input signals (environmental information) obtained from the home appliance measurement unit 6 and the household property information acquisition unit 60 installed in the moving body 2, equipment including household goods such as home appliances and furniture and their positions are estimated. do. In this estimation process, the floor plan analysis device 3 divides the home map into a plurality of unit areas, associates the plurality of divided unit areas with the positions of the equipment, and based on this correspondence relationship, determines the home appliance attribute information and the facility attributes. A label corresponding to information is assigned to each unit area. The details of the estimation process and the floor plan recognition process will be described below.
<Estimation processing>
FIG. 12 is a flow chart showing processing for estimating home appliance attribute information, household goods attribute information, and positions of home appliances and household goods according to the present embodiment. This flowchart differs from FIG. 3 in that step S20 is added. Therefore, the difference from FIG. 3 will be mainly described below.

Steps S1 to S4 perform the same processing as in FIG. Next, while the moving body 2 is moving, the household goods attribute information acquisition unit 60 identifies furniture, doors, and other household goods in the house and determines their positions (step S20). For example, the environmental measurement sensor 18 can detect and identify obstacles (household goods) such as dining sets and beds. Also, an image acquired from an image sensor can be processed to identify an object in the image, and the position of the object can be estimated by triangulation. In addition, the acquisition of the household property attribute information by the household property information acquisition unit 60 may be performed by, for example, driving the moving body 2 into the house at a specified time to acquire the household property attribute information. The moving body 2 may acquire household property attribute information when performing work. For example, in the case of a robot cleaner, the moving object 2 acquires property attribute information during cleaning. Furthermore, the household property attribute information and the position of the household property may be estimated in the same manner as the estimation of the home appliance attribute information in steps S5 and S6. In this case, household property attribute information and household appliance attribute information may be acquired together as equipment attribute information. In addition, the position of the household goods and the position of the home appliance may also be estimated as the position of the equipment.

This completes the description of step S20. Thereafter, steps S5 and S6 similar to those in FIG. 3 are executed. By the above processing, the attribute and the position of equipment, which is a type of indoor object, can be estimated.
<Floor plan determination processing>
FIG. 13 is a flowchart illustrating floor plan determination processing according to the second embodiment. This flowchart differs from FIG. 6 in that step S30 is added and step S15 is changed. Therefore, the difference from FIG. 6 will be mainly described below.

Steps S10 to S14 perform the same processing as in FIG. Next, the communication unit 9 acquires property attribute information from the property property information acquisition unit 60 (step S35).

Next, the floor plan determining unit 11 associates each of the divided unit areas and equipment including household appliances and household goods arranged in each unit area (step S15'). It is desirable that the floor plan determination unit 11 creates an equipment list 140 shown in FIG. 14 in accordance with this correspondence. Note that step S15' differs from step S15 in that the presence of household goods is also taken into account.

The equipment list 140 shown in FIG. 14 will be described below. FIG. 14 is a diagram showing the equipment list 140. As shown in FIG. The equipment list 140 has substantially the same configuration as the home appliance list 80 . The third column of the home appliance list 80 is changed from "household appliance in unit area" to "equipment in unit area". In other words, the equipment list 140 records equipment including household goods in addition to household appliances. Specifically, as shown in FIG. 14, in addition to home appliances, equipment including household goods such as furniture, doors, and house structures is recorded. It is stored in the storage unit of the analysis device 3 .

Thereafter, the floor plan determination unit 11 assigns a label to each unit area according to the room attribute determination rule 91 (step S16). Although step S16 performs the same processing as in FIG. 6, the room attribute determination rule used is different. Therefore, the room attribute determination rule 91 used in this embodiment will be described below.

FIG. 15 is a diagram showing the room attribute determination rule 91. As shown in FIG. The room attribute determination rule 91 has substantially the same configuration as the room attribute determination rule 90 . In the second column, the room attribute determination rule 90 is changed from "household appliances" to "facility equipment". In other words, household effects can also be taken into account in order to determine room attributes. Specifically, as shown in FIG. 15, in addition to home appliances, facility equipment including home appliances such as furniture, doors, and indoor structures are also recorded in the second column. Note that the room attribute determination rule 91 is also stored in the storage unit of the floor plan analysis device 3 .

With the above processing, it is possible to determine the room attribute for each unit area in the house, taking household goods into consideration. In addition, there are cases where the user wants to change the floor plan or the floor plan is erroneously identified depending on the life scene. This embodiment can also provide a means for the user to modify the floor plan. The floor plan analysis device 3 may receive a user's vocal floor plan correction instruction using, for example, a microphone connected to the floor plan analysis device 3 in addition to the microphones 50 and 51, and change the label. In addition, the floor plan analysis device 3 may accept a user's correction instruction on the home map and change the label of the floor plan.

Furthermore, the home appliance list in each unit area is collated with the room attribute determination rule, and there are cases where it does not completely match the room attribute determination rule. In this case, as in the first embodiment, it is desirable to assign the label with the largest number of matching objects to the unit area. Furthermore, the "equipment" of the room attribute determination rule 91 may be set to "essential". In other words, when equipment described as essential is presumed, a corresponding label is given.

According to the second embodiment described above, from the input signal (environmental information) obtained from the home appliance measurement unit 6 and the household property information acquisition unit 60 installed in the moving body 2, equipment such as home appliances and household goods and their positions are determined. do. Then, the map processing unit 10 divides the indoor map into a plurality of unit areas, associates the plurality of divided unit areas with the positions of the equipment and the positions of the equipment, and determines the location of the equipment installed in the unit areas. A label corresponding to the equipment attribute information can be assigned to each unit area based on the equipment attribute information and the position. In addition, it can be applied to people's lives, and by fusing multiple types of information, it is possible to improve the accuracy of floor plan recognition.
<Modified example of estimation processing>
Next, a modification of the object attribute information estimation process according to the second embodiment will be described. In this modified example, when object attribute information cannot be estimated from certain environmental information, other environmental information is used. In this modified example, since it is necessary to use a plurality of pieces of environment information, the same configuration as in the second example is also used in this example.

The estimation processing of the object attribute information in this modified example will be described below. Also in this modified example, steps S1 to S4 and step S6 are the same processing. Therefore, processing corresponding to steps S5 and S20 will be described here. Also, since the floor plan determination process is the same as that in the first and second embodiments, the description thereof will be omitted.

FIG. 16 is a flowchart showing object attribute information according to this embodiment. Note that the estimation of the position of the object is the same as in the first and second embodiments.

First, the home appliance estimation unit 21 determines whether object attribute information can be estimated from the environment information acquired by the home appliance measurement unit 6 and the household property information acquisition unit 60 (step S101). As a result, if it can be estimated (YES), the process proceeds to step S6. When it cannot estimate (NO), it changes to step S102.

Next, the home appliance estimation unit 21 activates parts other than the part from which the environment information was acquired in step S1. For example, if the home appliance measurement unit 6 detects sound in step S1, the household property information acquisition unit 60 is activated to acquire environmental information other than sound. In other words, environment information of a different type from the environment information acquired in step S1 can be acquired. Therefore, when a plurality of types of environment information can be acquired by one part, another function of the same part is activated.

Next, other environmental information is obtained from the part activated in step S102, and the process returns to step S101. By processing in this way, it is possible to cope with the case where one piece of object attribute information cannot be estimated.

Although the description of each embodiment is finished above, the present invention is not limited to these, and can be applied to various modifications and applications.

1 floor plan recognition system 2 moving body 3 floor plan analysis device 6 home appliance measurement unit 7 mobile body control unit 21 home appliance estimation unit 22 self-position estimation unit 23 map creation unit 24 operation control unit 25 storage unit 10 map processing unit 11 floor plan determination unit 50, 51 Microphone 60 Household property attribute information acquisition unit

Claims (15)

In an area recognition system that recognizes an area composed of a plurality of unit areas,
an object measurement unit that acquires environmental information of an object existing in the area;
an object estimation unit for estimating object attribute information indicating an attribute of an object and a position using the measured environment information;
a region processing unit that divides the region into a plurality of partial unit regions;
An area recognition system having an attribute identification section that uses the estimated position of the object to estimate a unit area in which the object exists, and uses the object attribute information to identify attributes of the estimated unit area. In the area recognition system according to claim 1,
The area recognition system, wherein the object is a home appliance. In the area recognition system according to claim 2,
The object measurement unit measures the sound generated by the home appliance as the environmental information to acquire sound information of the home appliance,
The area recognition system, wherein the object estimation unit estimates an attribute of the home appliance as the object attribute information using the sound information. In the area recognition system according to claim 3,
further comprising a moving body including the object measuring unit and a driving unit that moves itself,
The drive unit moves the moving body in a direction of a sound source of sound generated by the home appliance,
The object measurement unit is a region recognition system that measures the sound generated by the home appliance at the destination of the moving object. In the area recognition system according to any one of claims 1 to 4,
a storage unit that stores in advance an area attribute determination rule indicating a correspondence relationship between the attribute of the unit area and the object indicated by the object attribute information;
The area recognition system, wherein the attribute identifying section identifies the attribute of the unit area according to the number of objects indicated by the object attribute information using the area attribute determination rule. In a moving object for recognizing an area composed of a plurality of unit areas,
an object measurement unit that acquires environmental information of an object existing in the area;
an object estimation unit for estimating object attribute information indicating an attribute of an object and a position using the measured environment information;
Having a communication unit that outputs the object attribute information and the position,
The region recognition device divides the region into a plurality of partial unit regions, estimates a unit region in which the object exists using the estimated position of the object, and uses the object attribute information to determine the estimated unit region. A moving body that enables identification of the attributes of a unit area. In the mobile body according to claim 6,
The object is a moving body that is a home appliance. In the mobile body according to claim 7,
The object measurement unit measures the sound generated by the home appliance as the environmental information to acquire sound information of the home appliance,
The object estimating unit uses the sound information to estimate an attribute of the home appliance as the object attribute information. In the mobile body according to claim 8,
further comprising the object measuring unit and a driving unit that moves the moving object,
The drive unit moves the moving body in a direction of a sound source of sound generated by the home appliance,
The object measuring unit is a moving body that measures a sound generated by the home electric appliance at a destination of the moving body. In the moving body according to any one of claims 6 to 9,
In the area recognition device, using an area attribute determination rule indicating a correspondence relationship between an attribute of the unit area and an object indicated by the object attribute information in advance, according to the number of objects indicated by the object attribute information, the A moving object that identifies attributes of a unit area. In an area recognition device for recognizing an area composed of a plurality of unit areas,
a communication unit that communicates with a mobile body and receives from the mobile body object attribute information indicating the attributes of an object existing in the region and the position thereof;
a region processing unit that divides the region into a plurality of partial unit regions;
An area recognition apparatus having an attribute identification unit that estimates a unit area in which the object exists using the estimated position of the object, and identifies an attribute of the estimated unit area using the object attribute information. The region recognition device according to claim 11,
The area recognition device, wherein the object is a home appliance. 13. The region recognition device according to claim 12,
The area recognition device, wherein the communication unit receives, as the object attribute information, home appliance attribute information indicating an attribute of the home appliance estimated based on the sound generated by the home appliance from the moving object. 14. The region recognition device according to claim 13,
The area recognition device, wherein the communication unit receives the home appliance attribute information estimated based on the sound measured at the destination of movement of the moving body in the direction of the sound source from the moving body. The region recognition device according to any one of claims 11 to 14,
a storage unit that stores in advance an area attribute determination rule indicating a correspondence relationship between the attribute of the unit area and the object indicated by the object attribute information;
The attribute identification unit uses the area attribute determination rule to identify an attribute of the unit area according to the number of objects indicated by the object attribute information.

Images