KR100993241B1 - System and method for monitoring movable robot - Google Patents

Abstract

A mobile robot monitoring system, and method are disclosed. The mobile robot monitoring system includes a route image input unit, a guideline generator, and an image output unit. The image input unit is mounted on a mobile robot to input an image on a path along which the moving object moves, the guide line generator generates a guide line for displaying a position separated from the image input unit by a predetermined distance on the image, and the image output unit is input. The guideline is displayed on the printed image and output. This configuration effectively recognizes the exact working radius or the boundary distance to the operator of the mobile robot without the need for additional expensive devices, thereby improving work efficiency and preventing safety accidents.

Robot, Construction, Monitoring, Barrel Distortion

Description

System and method for monitoring movable robot}

TECHNICAL FIELD The present invention relates to a robot monitoring system and method, and more particularly, to a system and method for monitoring a working radius or a boundary distance of a movable robot.

In general, when a building is constructed, most of the work is performed by manpower. Therefore, there is a problem in that it takes a lot of work manpower and takes a long time for transportation and construction of construction materials. Is being studied.

Until now, when construction robots move or work in construction site, due to technical limitations, all work cannot be done automatically, and the operator uses the method of moving or working the robot by direct control from near or far. At this time, the manipulator to control the robot is generally based on the image information obtained from the image camera installed in the construction robot.

However, in this case, the plane image obtained from the camera installed in the robot makes it difficult for the robot operator to recognize the exact distance from the robot's working environment, thereby reducing the efficiency of the robot's work or causing a safety accident.

In addition, the distance from the robot can be obtained by using a separate distance measuring device or a method of calculating the distance from the image data to a specific point, but each requires a separate distance measuring device or a high performance computing device. However, there is a problem that does not effectively provide the robot operator with general information about the working environment of the robot.

The present invention has been made to solve the above-mentioned conventional problems, to effectively recognize the correct working radius or boundary distance to the operator of the mobile robot without a separate expensive device to improve the work efficiency and prevent safety accidents It is an object to provide a system, and a method.

In order to achieve the above object, the mobile robot monitoring system according to the present invention includes a route image input unit, a guideline generator, and a projection output unit.

The image input unit is mounted on a mobile robot and inputs an image on a path along which the mobile body moves. The guide line generator generates a guide line for displaying a position separated from the image input unit by a predetermined distance on the image, and the image output unit displays and outputs a guide line on the input image.

Since the work guideline away from the robot is displayed and printed on the image provided to the robot controller, the robot can effectively recognize the exact working radius or boundary distance of the mobile robot without the need for an additional expensive device, improving work efficiency and safety. Help prevent accidents.

The guide line generator may generate a guide line by using an installation height from a path of the image input unit and an installation angle with respect to the path. Since the distance from the image input unit is calculated using only the installation height and the installation angle information of the image input unit, it is possible to effectively generate a work guide line without a separate expensive computing device.

The guide line generator may be generated by deforming the guide line according to the characteristics of the image input unit, and the deformation of the guide line may be barrel distortion. This configuration can generate accurate guide lines corresponding to the distorted image even when the distorted image is provided according to the characteristics of the image input unit.

In addition, an invention is disclosed in which the system is implemented in the form of a method.

The present invention provides an adjustment guide method of the robot that does not require complex calculation according to the type of the robot, the type of the camera, the installation position, etc., and thus has the effect of providing quick guide data, and can be manufactured using relatively inexpensive components. It has the effect of making it possible.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram of one embodiment of a mobile robot monitoring system according to the present invention.

The mobile robot monitoring system includes a route image input unit, a guideline generator, and an image output unit.

In FIG. 1, a path image input unit is displayed as a camera image input unit 300, an image output unit is displayed as a display output unit 500, and the robot monitoring system further includes a robot information input unit 100 and a camera information input unit 200. It is included.

The camera image input unit 400 is mounted on the mobile robot and inputs an image on a path along which the moving object moves.

FIG. 2 is a diagram illustrating a use state of a mobile robot provided with a camera image input unit of FIG. 1.

As shown in FIG. 2, the camera installed in the robot is photographed at an angle to the construction site at an oblique angle. As a result of the pinhole camera, there is a phenomenon in which the image becomes smaller as the object to be photographed is farther away. Various coordinate models have been studied for this purpose, but because they include a rather complicated computational process, there is a burden of using a high-end microprocessor. In the present invention, in order to express only the distance information between the guideline and the robot, a new image plane coordinate model expressed simply in two dimensions is used.

2 is a schematic diagram of a camera installation for generating a guide image for adjusting the construction robot, in which the camera installed on the robot tilts the ground of the construction site by a pitch angle Φ. In addition, because the pinhole camera has a phenomenon in which the image becomes smaller as the object to be photographed is farther away, the new image surface coordinate model shown in FIG. 3 is used to express this.

The guide line generator 300 generates a guide line for displaying a position away from the camera image input unit 400 by a predetermined distance on the image. In particular, the guideline generator 300 generates a guideline by using the installation height from the path of the camera image input unit 400 and the installation angle with respect to the path.

As such, since the distance from the image input unit is calculated using only the installation height and the installation angle information of the camera image input unit 400, it is possible to effectively generate a work guide line without a separate expensive computing device.

FIG. 3 is a diagram illustrating a 2D image plane coordinate model presented by the mobile robot monitoring system of FIG. 1. Where X is the coordinate representing the ground and Y is the vertical coordinate of the ground. D is the minimum working radius of the robot and is a value input during the robot information input process. F is the focal length of the camera, H is the mounting height of the camera, θ is the vertical wide angle of the camera, Φ is the value input during the camera information input process. The unknown angle angle Ψ is obtained by the following method.

First, if a point of the ground photographed at the top of the image is Lmax and a point of the ground photographed at the bottom of the image is Lmin, each value is obtained by the following equation.

(1) Lmin = H tan (φ-0.5θ)

(2) Lmax = H tan (φ + 0.5θ)

D is the minimum work radius of the robot, which can be arbitrarily changed, and can be obtained based on a value input during the robot information input process.

(3) ψ = arctan (D / H)-(φ-0.5θ)

In the image photographed on the basis of the image plane coordinates shown in FIG. Where D is the actual distance from the ground and D 'is the distance in the image.

(4) D '= F tan (ψ)

In addition, 2D 'and 3D' can be obtained by substituting each ψ obtained by substituting 2D and 3D in Equation (3) into Equation (4).

The display output unit 500 displays and outputs guide lines on the input image.

FIG. 4 is a schematic diagram of image data of a guideline, which is a result obtained from the image surface coordinate model of FIG. 3.

The guide line generator 300 may generate the guide line by deforming the guide line according to the characteristics of the camera image input unit 400, and the deformation of the guide line may be barrel distortion. This configuration can generate accurate guide lines corresponding to the distorted image even when the distorted image is provided according to the characteristics of the image input unit.

The camera installed in front of the robot may use a wide-angle lens such as a fisheye lens to secure a wide field of view, and when the camera image input unit 400 uses a wide-angle lens, it is applied to a barrel distorted image according to the use of the wide-angle lens. The barrel needs to be distorted to match.

5 is a barrel distortion plot diagram of image data of a work guideline of the robot of FIG. 4.

In this case, a barrel distortion method is used to distort using a general barrel distortion method, and thus an input image is barrel distorted. In order to represent the robot's work guideline in this barrel distorted image, the guideline must also be displayed in a barrel distorted state. That is, the work guideline of the robot should be distorted to match the distorted image input, and the work guideline of the distorted robot is overlaid with the barrel distorted image input for easy display by the operator.

6 is a screen output by overlaying an operation guideline of the barrel distorted robot of FIG. 5 onto an image obtained from a wide-angle camera.

The present invention relates to a monitoring device that overlays and outputs a guideline for assisting the work of a construction robot that is put into a construction site as shown in FIG. 6, and visualizes a certain radius from an outer boundary of the robot to promote convenience and safety of work. It is about an apparatus for doing so.

The construction guideline monitoring device of the construction robot uses the camera installed in front of the robot to go beyond the level of displaying the screen, and outputs the operation guideline information on the camera image to guide the operator of the robot's work and boundary radius. .

In this way, since the operation guideline away from the robot positioned on the robot's movement path is displayed and output on the image provided to the robot controller, the operator of the mobile robot can effectively adjust the correct working radius to the operator of the mobile robot without an additional expensive device. Recognize the boundary distance to improve work efficiency and prevent safety accidents.

FIG. 7 is a schematic diagram of a work guideline representation of the construction robot of FIG. 1.

Here, D is the minimum value of the working radius of the robot, which can be arbitrarily changed, and the 2D and 3D distances, which are doubled according to the risk, are expressed as guidelines, using red, yellow, and green, respectively. In addition, the camera used is a wide-angle camera that generates a barrel distortion, such as fisheye lens to secure a wide viewing angle.

8 is a schematic flowchart for performing an embodiment of a mobile robot monitoring method according to the present invention.

As shown in Figure 8, the present invention is a robot information input process for inputting the information of the robot (S100), a camera information input process (S200) for inputting the information of the camera installed in the robot, the operation of the robot through the input information Guideline generation process for generating image data for displaying a radius (S300), a display process (S400) for overlaying the output of the guideline generated in the process on the camera image (S400), the guideline data output in the process It consists of a result data storage process for storing (S500).

Robot information input process (S100) inputs the appearance information, such as height, length, width, wheel size, wheelbase distance of the robot to the robot information input unit 100, the working radius 102, the speed of each wheel ( 103 and the direction of travel 104 of the robot.

The camera information input process (S200) inputs camera information such as a focal length, a wide angle, an installation height, a pitch angle, and a camera resolution of a camera installed in a robot to the camera information input unit.

The operation guideline generation process of the robot (S300) generates a guideline using the new image surface coordinates shown in FIG. 3 based on the robot information and the camera information input in the above process, and the barrel distortion according to the use of the wide-angle lens. Barrel distortion may also be made to the robot's working guidelines to match the image.

The display process (S400) of overlaying the guideline on the camera image and outputting outputs the image by overlaying the work guideline of the barrel-distorted robot generated in the above process onto the image obtained from the wide-angle camera.

The result data storing process stores the work guideline data of the robot converted into the image form in the above process.

The present invention is a technology for providing image information to the coordinator of the robot through the camera of the construction robot as a method for generating a work guideline representing the work radius or boundary distance information of the robot and overlaying the work guideline on the acquired image It relates to a monitoring device that outputs.

The present invention allows the operator to monitor the overlaid image by matching the barrel distorted image obtained from the wide-angle camera installed in the robot with the robot's work guidelines in real time.

Robot information input process for inputting the information of the robot: Camera information input process for inputting the information of the camera installed in the robot: Guide for generating image data for displaying the working radius of the robot through the input information Line generation process: a display process of overlaying a guideline, which is a result generated in the above process, on a camera image and outputting the result: a result data storing process of storing the guideline data output in the above process.

The present invention provides a method for generating a work guideline of a robot through information of a robot and installation information of a camera installed in the robot, and converts the generated result into data in an image format and monitors the data using an image processing board. do.

In addition, the present invention provides a device for monitoring the overlaid image in real time by reducing the complexity of the process of matching the barrel distorted image obtained from the wide-angle camera installed in the robot with the robot's work guidelines.

The present invention has an effect of providing a quick guide data because it provides a guide method for adjusting the masterpiece robot that does not require complex calculation according to the type of construction robot, the type of camera, the installation position.

The present invention has the effect of providing the operation guide information of the robot by a relatively simple method without complicated calculations, so that the adjustment device of the construction robot can be manufactured using relatively low-cost parts.

The present invention has an effect that can be used as an adjustment guide device for various industrial robots as well as construction robots by providing a method for generating a work guideline of the robot regardless of the type of the robot or the type of camera.

Although the invention has been described above by way of some preferred embodiments, the scope of the invention should not be limited by this, but modifications or improvements of the embodiments which are supported by the claims should extend.

1 is a block diagram of one embodiment of a mobile robot monitoring system according to the present invention;

FIG. 2 is a view illustrating a use state of a mobile robot in which a camera image input unit of FIG. 1 is installed.

3 is a diagram illustrating a 2D image plane coordinate model presented by the mobile robot monitoring system of FIG. 1.

4 is a schematic diagram of image data of a guideline which is a result obtained from the image surface coordinate model of FIG. 3.

5 is a barrel distortion plot diagram of image data of a work guideline of the robot of FIG. 4.

6 is a screen output by overlaying the operation guideline of the barrel distorted robot of FIG. 5 to the image obtained from the wide-angle camera.

7 is a schematic diagram of a work guideline representation of the construction robot of FIG.

8 is a schematic flowchart for performing an embodiment of a mobile robot monitoring method according to the present invention;

Claims (8)

delete An image input unit mounted on a mobile robot and inputting an image on a path along which the mobile robot moves; A guide line generation unit generating a guide line for displaying a position separated from the image input unit by a predetermined distance on the image; A mobile robot monitoring system comprising an image output unit to display and output the guide line on the input image. The guide line generator generates the guide line by using the installation height from the path and the installation angle from the path of the image input unit. 3. The method of claim 2, The guide line generation unit may be generated by modifying the guide line according to the characteristics of the image input unit. The method of claim 3, wherein And the deformation of the guideline is barrel distortion. delete An image input step of receiving an image on a path along which the mobile robot moves from an image input apparatus mounted on the mobile robot; A guide line generating step of generating a guide line for displaying a position separated from the image input unit by a predetermined distance on the image; A mobile robot monitoring method comprising an image output step of displaying and outputting the guide line on the input image. The generating of the guide line may include generating the guide line by using an installation height from the path of the image input apparatus and an installation angle with respect to the path. The method of claim 6, The generating of the guide line may include generating the guide line by modifying the guide line according to the characteristics of the image input apparatus. The method of claim 7, wherein And the deformation of the guide line is barrel distortion.

Images