JP2021140583A - Object image display device, object image display system, and object image display program - Google Patents

Abstract

To display a layout of an object in a space.SOLUTION: An object image display device 120 includes: a drawing information acquisition unit 171 for acquiring first drawing information created based on measurement by a mobile robot 130 autonomously moving within a predetermined dwelling 200, and second drawing information on an inside of the dwelling 200 created before creation of the first drawing information; an object separation unit 173 for individually separating object information showing an object arranged in the dwelling 200 based on a difference between the first drawing information and the second drawing information; and a display unit for displaying a superimposed image 300 in which an object image 310 corresponding to the object information is superimposed on a space image 301 corresponding to at least one of the first drawing information and the second drawing information.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to an object image display device, an object image display system, and an object image display program for displaying the layout of an object in a predetermined space.

Conventionally, by attaching a wireless tag to home appliances, furniture, etc. placed in a house and having a robot moving in the house read the information of the wireless tag, the arrangement of the object and the information of the object are associated with each other. A display technique is disclosed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2004-34553

However, with only the viewpoint of the robot moving in the house and the information such as furniture, the size is enough to measure the distance to the wall to purchase new furniture and home appliances to be installed in the house. It was necessary to confirm in advance whether it would fit. There is also a desire to determine before purchase whether furniture can be installed in a home.

The present disclosure provides an object image display device capable of displaying the layout of an object in space, and an object image display program.

The object image display device of the present disclosure relates to first drawing information created based on measurement by a mobile robot that autonomously moves in a predetermined space, and the inside of the space created before the creation of the first drawing information. A drawing information acquisition unit that acquires the second drawing information, and an object separation unit that individually separates object information indicating an object arranged in the space based on the difference between the first drawing information and the second drawing information. It has a display unit that displays a superposed image in which an object image corresponding to the object information is superimposed on a spatial image corresponding to at least one of the first drawing information and the second drawing information.

The object image display system of the present disclosure includes a mobile robot that autonomously moves in a predetermined space, a terminal device that can communicate with the mobile robot, first drawing information created based on measurement by the mobile robot, and A drawing information acquisition unit that acquires a second drawing information about the inside of the space created before the creation of the first drawing information, and an arrangement in the space based on the difference between the first drawing information and the second drawing information. A superposition in which an object image corresponding to the object information is superimposed on a space image corresponding to at least one of the first drawing information and the second drawing information and an object separation unit that individually separates the object information indicating the object. An object image display system including a display unit for displaying an image.

The object image display program of the present disclosure relates to the first drawing information created based on the measurement by a mobile robot autonomously moving in a predetermined space, and the inside of the space created before the creation of the first drawing information. A drawing information acquisition unit that acquires the second drawing information, and an object separation unit that individually separates object information indicating an object arranged in the space based on the difference between the first drawing information and the second drawing information. Realized by causing a computer to execute a display unit that displays a superposed image in which an object image corresponding to the object information is superimposed on a spatial image corresponding to at least one of the first drawing information and the second drawing information. do.

According to the present disclosure, it is possible to output a layout showing the size and arrangement state of an object installed in the space on the screen.

FIG. 1 is a diagram showing an object image display system according to an embodiment together with space. FIG. 2 is a side view showing the appearance of the mobile robot according to the embodiment. FIG. 3 is a bottom view showing the appearance of the mobile robot according to the embodiment. FIG. 4 is a block diagram showing each functional unit of the mobile robot system according to the embodiment. FIG. 5 is a diagram showing a display device on which a spatial image, an object image, and the like are displayed. FIG. 6 is a diagram showing a display image of the object movement mode state. FIG. 7 is a diagram showing a display image of the measurement mode state. FIG. 8 is a diagram showing a display image of the object integrated mode state. FIG. 9 is a diagram showing a display image in the additional mode state. FIG. 10 is a diagram showing an object creation screen. FIG. 11 is a diagram showing a state in which a new object image is added to the superimposed image.

Next, an embodiment of the object image display device, the object image display system, and the object image display program in the present disclosure will be described with reference to the drawings. The following embodiments are merely examples of the object image display device, the object image display system, and the object image display program in the present disclosure. Therefore, the present disclosure is defined by the wording of the claims with reference to the following embodiments, and is not limited to the following embodiments. Therefore, among the components in the following embodiments, the components not described in the independent claims indicating the highest level concept of the present disclosure are not necessarily necessary for achieving the subject of the present disclosure, but more. Described as constituting a preferred form.

In addition, the drawings are schematic drawings in which emphasis, omission, and ratio are adjusted as appropriate to show the present disclosure, and may differ from the actual shape, positional relationship, and ratio.

FIG. 1 is a plan view showing a space in which the mobile robot included in the object image display system according to the embodiment moves together with the mobile robot. In the case of the present embodiment, the space in which the mobile robot 130 moves is the dwelling 200, and the mobile robot 130 autonomously moves on the floor 201 of the dwelling 200. The dwelling 200 is a space surrounded by a wall 202, and has a protruding portion 203, a door portion 204, and the like in a part thereof. Objects such as furniture and home appliances (first objects 211 to fifth objects 215) are arranged on the floor 201. The object image display system 100 includes a mobile robot 130 and a terminal device 160. In the case of the present embodiment, the object image display device 120 is realized by executing the object image display program on the terminal device 160 (see FIG. 4).

FIG. 2 is a side view showing the appearance of the mobile robot according to the embodiment. FIG. 3 is a bottom view showing the appearance of the mobile robot according to the embodiment. As shown in these figures, the mobile robot 130 according to the embodiment is a cleaning robot that performs cleaning while autonomously traveling on the floor 201, and sucks and cleans the dust existing on the floor 201. ..

In the case of the present embodiment, the mobile robot 130 includes a body 131 on which various components are mounted, a drive unit 132 for moving the body 131, a cleaning unit 134 for collecting dust existing on the floor 201, and dust inside the body 131. It is provided with a suction unit 133, a drive unit 132, a cleaning unit 134, a control unit 135 for controlling the suction unit 133, and a positional relationship detecting means 136.

The body 131 is a housing that houses the drive unit 132, the control unit 135, and the like. The upper part is removable with respect to the lower part. A bumper 139 that can be displaced with respect to the body 131 is attached to the outer peripheral portion of the body 131. Further, as shown in FIG. 3, the body 131 is provided with a suction port 138 for sucking dust into the body 131.

The drive unit 132 is a device for running the mobile robot 130 based on an instruction from the control unit 135. The drive unit 132 includes a wheel 140 that travels on the cleaning surface, a traveling motor (not shown) that applies torque to the wheel 140, and a housing 141 that houses the traveling motor. Caster 142 is provided as an auxiliary wheel on the bottom surface of the body 131. By independently controlling the rotations of the two wheels 140, the mobile robot 130 can freely travel in a straight line, backward, counterclockwise, clockwise, and the like.

The cleaning unit 134 is a unit for sweeping dust and sucking dust from the suction port 138, and includes a rotary brush arranged in the vicinity of the suction port 138, a brush drive motor for rotating the rotary brush, and the like.

The suction unit 133 is a unit that sucks dust from the suction port 138 and holds the dust sucked into the body 131, and has an electric fan (not shown) and a dust holding portion 143. The electric fan sucks the air inside the dust holding portion 143 and discharges the air to the outside of the body 131 to suck the dust from the suction port 138. It is a device that detects the positional relationship between the mobile robot 130, the wall 202, an object, and the like on the floor 201. The positional relationship detecting means 136 detects the direction, distance, and the like of an object such as a wall 202 and furniture existing around the body 131, and acquires 2.5-dimensional information. Further, the mobile robot 130 can also grasp its own position from the information of the direction and the distance detected by the positional relationship detecting means 136. The type of the positional relationship detecting means 136 is not particularly limited, but for example, LiDAR (Light Detection and Ranking), which detects the position and the distance based on the light that radiates light and is reflected by an obstacle and returned. A ToF (Time of Reflect) camera and the like can be exemplified. Further, as the positional relationship detecting means 136, a compound eye camera or the like that acquires illumination light or natural light reflected by an obstacle as an image and acquires a position and a distance based on parallax can be exemplified. In the case of the present embodiment, the mobile robot 130 is provided with a LiDAR on the upper portion and a camera on the side portion as the positional relationship detecting means 136.

The mobile robot 130 may include a sensor in addition to the positional relationship detecting means 136. For example, a floor surface sensor that is arranged at a plurality of locations on the bottom surface of the body 131 and detects whether or not the floor 201 is present may be provided. Further, the drive unit 132 may be provided with an encoder for detecting the respective rotation angles of the pair of wheels 140 rotated by the traveling motor. Further, an acceleration sensor that detects the acceleration when the mobile robot 130 travels and an angular velocity sensor that detects the angular velocity when the mobile robot 130 turns may be provided. Further, a dust amount sensor for measuring the amount of dust accumulated on the floor surface may be provided. A contact sensor may be provided that detects the displacement of the bumper 139 and detects that an obstacle has collided. Further, an obstacle sensor such as an ultrasonic sensor other than the positional relationship detecting means 136 that detects an obstacle existing in front of the body 131 may be provided.

The terminal device 160 is a computer or the like that can realize various functions by executing a program. In the case of the present embodiment, the terminal device 160 is a device called a so-called smartphone or tablet terminal, and information can be exchanged with the mobile robot 130 by communication. In the case of the present embodiment, the terminal device 160 includes a display device 161 (see FIG. 5) and an input device 162 (FIG. 5) such as a touch sensor arranged on the surface of the display device 161 and two-dimensionally detecting contact with fingers or the like. 4) and.

FIG. 4 is a block diagram showing each functional unit of the object image display system according to the embodiment. As shown in the figure, the object image display system 100 includes a drawing information acquisition unit 171, an object separation unit 173, and a display unit 174 as functional units of the object image display device 120. In the case of the present embodiment, the object image display system 100 further includes a positional relationship acquisition unit 172 and a drawing information creation unit 175 as functional units of the mobile robot 130, and further as a functional unit of the object image display device 120. It includes an object image moving unit 177, a distance display unit 178, an object information integration unit 179, an object information acquisition unit 180, and an input unit 181.

The positional relationship acquisition unit 172 acquires the positional relationship between the mobile robot 130 and an object such as furniture, a wall, or the like on the floor 201 from the positional relationship detecting means 136. In the case of the present embodiment, the positional relationship between the self-position of the mobile robot 130 and an object or the like is acquired by calculation from the measurement data from the LiDAR, which is the positional relationship detecting means 136, and at least one of the cameras.

The drawing information creation unit 175 creates drawing information based on the positional relationship between the mobile robot 130 acquired by the positional relationship acquisition unit 172 and an object or a wall. The method of creating the drawing information is not particularly limited, but in the case of the present embodiment, the drawing information is created by the SLAM (Simultaneus Localization and Mapping) technique. The created drawing information is stored in the storage unit 176.

The drawing information acquisition unit 171 acquires drawing information which is information indicating the shape of a predetermined space or the like. The drawing information acquisition unit 171 acquires at least two types of drawing information. The first drawing information, which is one of the drawing information, is drawing information created based on the measurement by the mobile robot 130 that autonomously moves in the house 200. The second drawing information is drawing information regarding the residence 200 created before the creation of the first drawing information.

The location where the first drawing information is created is not particularly limited, and the data detected by the positional relationship detecting means 136 of the mobile robot 130 is obtained by the server via the network 250, and the first drawing information is obtained by the server or the like. You may create it. In the case of the present embodiment, the first drawing information is created by the drawing information creation unit 175 included in the mobile robot 130 based on the data detected by the positional relationship detecting means 136 of the mobile robot 130. The drawing information acquisition unit 171 acquires the first drawing information from the mobile robot 130.

The location where the second drawing information is created is not particularly limited. For example, the second drawing information may be created on a server or the like based on the data created at the time of designing the residence 200. Further, the mobile robot 130 may be run on the floor 201 of the residence 200 in which furniture or the like is not installed, and the second drawing information may be created based on the data detected by the positional relationship detecting means 136. The second drawing information is preferably composed of information on the wall of the dwelling 200 in which an object such as furniture is not installed. In the case of the present embodiment, the drawing information acquisition unit 171 acquires a so-called floor plan as the second drawing data via the network 250.

The object separation unit 173 provides object information indicating the first object 211 to the fifth object 215 arranged in the residence 200 based on the difference between the first drawing information and the second drawing information acquired by the drawing information acquisition unit 171. Separate individually. The method of separating them individually is not particularly limited, but the mobile robot 130 becomes independent by traveling around the third object 213, for example, as in the case of the third object 213 which is an elliptical sofa. Information that can be recognized as one individual and that does not correspond to the second drawing information is separated from the first drawing information as one of the object information. In addition, when the mobile robot 130 sneaks in like the fourth object 214 which is a table, the leg may be recognized as one individual based on the measurement data of the four legs. Further, when the object is placed in close contact with the wall 202 or another object and the measurement data of the entire circumference of the object cannot be obtained from the movement path of the mobile robot 130, the information corresponding to the second drawing information from the first drawing information is obtained. Is removed, corners and the like are extracted from the remaining information, and externally attached to a rectangle or the like, the object separation unit 173 generates object information.

FIG. 5 is a diagram showing a display device on which a spatial image, an object image, and the like are displayed. The display unit 174 superimposes the object image 310 corresponding to the object information separated by the object separation unit 173 on the spatial image 301 corresponding to at least one of the first drawing information and the second drawing information acquired by the drawing information acquisition unit 171. The superimposed image 300 is displayed on the display device 161.

The input unit 181 processes a signal from the input device 162 provided in the terminal device 160. In the case of the present embodiment, the input device 162 includes a touch panel, and the input unit 181 accepts user operations such as "tap", "drag", and "pinch out" on the touch panel.

The object image moving unit 177 is a processing unit that moves the object image 310 in the superimposed image 300. In the case of the present embodiment, as shown in FIG. 6, when the user of the terminal device 160 taps the button on which the characters "move the object" are described based on the input device 162, the object movement mode is set and the object to be moved is changed. By tapping the portion of the object image 310 that becomes, the tapped object image 310 can be moved by dragging to a position that does not overlap with other object images 310 in the superimposed image 300.

The distance display unit 178 displays the actual distances corresponding to the two points in the superimposed image 300. In the case of the present embodiment, as shown in FIG. 7, when the user of the terminal device 160 taps the button on which the characters "measure the distance" are written based on the input device 162, the distance measurement mode is set and the superimposed image 300 is set. When two places are tapped inside, the actual distance corresponding to the tapped object image 310 is displayed on the display device 161 via the display unit 174. The user can also pinch out to enlarge and display the superimposed image 300.

The object information integration unit 179 performs a process of integrating corresponding object information based on a plurality of object images 310. In the case of the present embodiment, as shown in FIG. 8, when the user of the terminal device 160 taps the button on which the characters "combine a plurality of objects" are described based on the input device 162, the object integration mode is set and the user Can use the input device 162 to enclose a plurality of object images 310 in the superimposed image 300 with one figure such as a rectangle. The plurality of enclosed object images 310 are subsequently treated as one object image 310. That is, when moving, the object image 310 moves together with the enclosed image.

The object information acquisition unit 180 acquires newly added object information. The method for acquiring the object information is not particularly limited, and for example, the object information such as furniture and home appliances may be acquired from the web via the network 250. In the case of the present embodiment, as shown in FIG. 9, when the user of the terminal device 160 taps the button on which the characters "add a new object" are described based on the input device 162, the addition mode is set, and FIG. 10 shows. The transition to the object creation screen as shown can be performed, and the user can create a desired shape using the object creation screen. When the user taps the button on the object creation screen on which the characters "Next" are written, the object information acquisition unit 180 acquires the shape created by the user as the object information, and the display unit 174 acquires the object information acquisition unit. The object image 310 corresponding to the object information acquired by 180 is superimposed and displayed on the superimposed image 300 as in the highlighted object image 310 in FIG.

The newly added object image 310 can also be moved like the previously displayed object image 310, and is a target for distance measurement.

According to the object image display system 100 according to the above embodiment, after the object is installed in the residence 200, the mobile robot 130 such as a robot vacuum cleaner is moved to acquire the first drawing information, and before the object is installed. By acquiring the second drawing information indicating the living space, the layout of the movable object on the screen of the terminal device 160 can be confirmed together with the living space.

Further, the object image 310 corresponding to the object can be recognized in a one-to-one correspondence with the actual object, and the distance between the objects and the actual distance from the object to the wall 202 can be confirmed on the screen. Is possible.

Further, the object image 310 can be arbitrarily moved, and the layout of the object can be changed. Therefore, it is possible to contribute to the remodeling plan in the house 200.

Further, since new object information can be added and displayed as an object image 310 in the superimposed image 300, it is possible to confirm whether new furniture or the like can be installed in the residence 200. Further, since the positions of the object images 310 including the new object image 310 can be changed, it is possible to confirm whether or not the new furniture or the like fits in the house 200 depending on the layout.

The present disclosure is not limited to the above embodiment. For example, another embodiment realized by arbitrarily combining the components described in the present specification and excluding some of the components may be the embodiment of the present disclosure. The present disclosure also includes modifications obtained by making various modifications that can be conceived by those skilled in the art within the scope of the gist of the present disclosure, that is, the meaning indicated by the wording described in the claims, with respect to the above-described embodiment. Is done.

For example, the object image display device 120 may be able to erase the object image 310 in the superimposed image 300.

Further, although the case where the object image display system 100 includes the terminal device 160 has been described, in the object image display system 100, each functional unit of the object image display device 120 is realized by the mobile robot 130 without providing the terminal device 160. It doesn't matter.

Further, the mobile robot 130 may have other functions such as a pet robot, a monitoring robot, and a transfer robot as well as a cleaning robot.

Further, although the residence 200 is illustrated as the space, the space may be a relatively large space such as a hotel lobby, an airport, a mass retailer, or a factory.

Further, a part or all of the processing unit realized by executing the program may be provided in either the mobile robot 130 or the terminal device 160, or on a server connected via a network. You may be prepared for.

Further, although the case where the first drawing information is created by using one mobile robot 130 has been described, a part of the first drawing information may be created by running a plurality of mobile robots 130. .. In this case, the terminal device 160 may be able to communicate with a plurality of mobile robots 130, and the terminal device 160 may supervise the information and create the first drawing information.

The object image display device, the object image display system, and the object image display program of the present disclosure can be used for layout display of an object arranged in space.

100 Object image display system 120 Object image display device 130 Mobile robot 131 Body 132 Drive unit 133 Suction unit 134 Cleaning unit 135 Control unit 136 Positional relationship detection means 138 Suction port 139 Bumper 140 Wheel 141 Housing 142 Caster 143 Dust holder 160 Terminal device 161 Display device 162 Input device 171 Drawing information acquisition unit 172 Positional relationship acquisition unit 173 Object separation unit 174 Display unit 175 Drawing information creation unit 176 Storage unit 177 Object image movement unit 178 Distance display unit 179 Object information integration unit 180 Object information acquisition unit 181 Input part 200 Residential 201 Floor 202 Wall 203 Protruding part 204 Door part 211 First object 213 Third object 214 Fourth object 215 Fifth object 250 Network 300 Superimposed image 301 Spatial image 310 Object image

Claims (7)

First drawing information created based on measurements by a mobile robot that autonomously moves in a predetermined space, and drawing information for acquiring second drawing information about the inside of the space created before the creation of the first drawing information. Acquisition department and
An object separation unit that individually separates object information indicating an object arranged in the space based on the difference between the first drawing information and the second drawing information.
A display unit that displays a superimposed image in which an object image corresponding to the object information is superimposed on a spatial image corresponding to at least one of the first drawing information and the second drawing information.
An object image display device having. The object image display device according to claim 1, further comprising an object image moving unit that moves the object image in the superimposed image. The object image display device according to claim 1 or 2, further comprising a distance display unit that displays an actual distance corresponding to two points in the superimposed image. The object image display device according to any one of claims 1 to 3, further comprising an object information integration unit that integrates corresponding object information based on the plurality of object images. Equipped with an object information acquisition unit that acquires object information
The object image display device according to any one of claims 1 to 4, wherein the display unit displays an object image corresponding to the object information acquired by the object information acquisition unit by superimposing the object image on the superimposed image. A mobile robot that autonomously moves in a predetermined space,
A terminal device capable of communicating with the mobile robot and
A drawing information acquisition unit that acquires the first drawing information created based on the measurement by the mobile robot and the second drawing information about the inside of the space created before the creation of the first drawing information.
An object separation unit that individually separates object information indicating an object arranged in the space based on the difference between the first drawing information and the second drawing information.
A display unit that displays a superimposed image in which an object image corresponding to the object information is superimposed on a spatial image corresponding to at least one of the first drawing information and the second drawing information.
An object image display system equipped with. First drawing information created based on measurements by a mobile robot that autonomously moves in a predetermined space, and drawing information for acquiring second drawing information about the inside of the space created before the creation of the first drawing information. Acquisition department and
An object separation unit that individually separates object information indicating an object arranged in the space based on the difference between the first drawing information and the second drawing information.
A display unit that displays a superimposed image in which an object image corresponding to the object information is superimposed on a spatial image corresponding to at least one of the first drawing information and the second drawing information.
An object image display program realized by letting a computer execute.

Images