KR20140099622A - Robot localization detecting system using a multi-view image and method thereof - Google Patents

Abstract

The present invention relates to a system for measuring the position of a robot inside a reactor using multiple images and a measuring method thereof. The system comprises: an underwater robot (100) which includes a vertical and a horizontal thruster (101) at a fixed position in an internal space (10) of a reactor in order to move front and back, left and right, and up and down, and includes an underwater camera (102) at one end thereof in order to observe the internal space (10) of the reactor; multiple cameras (200) which are installed on the top (20) of a reactor vessel; a position measuring unit (300) which processes multiple images through the multiple cameras (200) in order to measure the position and posture of the underwater robot (100); and a remote control unit (400) which controls by converting data on the position and posture of the underwater robot (100) measured by the position measuring unit (300) into feedback signals. The position measuring unit (300) includes a DVR facility for obtaining images, and a PC for driving algorithm. Therefore, the system for measuring the position of a robot inside a reactor using multiple images and the measuring method thereof can accurately measure the position of the robot inside the reactor in real time in order to provide user convenience and to realize accuracy in remote operations.

Description

TECHNICAL FIELD [0001] The present invention relates to a robot position measurement system and a position measurement method using a multi-

The present invention relates to a system and method for measuring a position of a robot inside a reactor using multiple images, and more particularly, to a method for measuring the position of a robot by installing several cameras on the top of a reactor vessel, To a system and method thereof.

Reactor pressure vessel is a high radiation area where human access is restricted, and remote operation using a robot is essential.

Generally, robots are equipped with various sensors to allow remote workers to recognize their surroundings and perform desired tasks. Sensors such as cameras and pressure sensors are used for underwater robots. By observing the object intuitively like the human eye, the camera enables remote operation to be performed smoothly, and the pressure sensor allows the depth information of the underwater robot to be grasped.

Underwater robots for reactor operation are operated remotely from a remote location to prevent users from being exposed to radiation. Users operate underwater robots through a camera installed on the robot, a camera installed on the reactor pressure vessel, and inspect the inside of the pressure vessel do. At this time, it is very difficult to manipulate the robot to reach a desired position using only image information. It provides location information of the robot and provides it to the operator so that remote work can be performed smoothly. In general, an inertial measuring device (IMU) or a pressure sensor is used as a method of measuring the position of a robot.

First, in the case of the inertial measurement device, accumulation of the error can not be avoided because the position is estimated by integrating the change of the acceleration and the tilt, and the position accuracy is lowered as the operation time of the underwater robot becomes longer. Second, the pressure sensor is difficult to cope with the small height change of the underwater robot, and it is difficult to judge the correct position due to the influence of the surrounding temperature or flow rate.

It is an object of the present invention to provide a system for measuring the position of a robot by installing a plurality of cameras on the top of a reactor vessel having a small radiation influence for measuring the position of the robot, .

It is another object of the present invention to provide a robot position measuring system and method for measuring the robot position accurately in real time in a nuclear reactor, thereby realizing the convenience of the operator and the accuracy of the remote operation.

It is still another object of the present invention to provide a robot position measurement system and a method of measuring the position of a robot which can double the reliability of a remote operation by allowing an intuitive robot position observation as well as a quantitative robot position measurement.

According to an aspect of the present invention for achieving the above object, the present invention provides a thruster in vertical and horizontal directions so as to move back and forth, right and left, and up and down at an arbitrary position in a nuclear reactor interior space, An underwater robot in which an underwater camera is installed at one end; A plurality of cameras provided at an upper end of the reactor vessel; A position measuring unit for processing a plurality of images through the plurality of cameras and measuring a position and an attitude of the underwater robot; And a remote control unit for controlling the position and the attitude of the underwater robot measured by the position measuring unit as feedback signals. The position measuring unit includes a DVR for acquiring an image and a PC for driving an algorithm.

According to another aspect of the present invention, there is provided an image processing method including the steps of: (a) acquiring multiple images from a plurality of cameras; (b) determining positions of an image plane and a spatial position of the multiple images through camera correction; (c) measuring the silhouette of the robot in the multi-view image using hue detection; (d) performing elliptical fitting on the measured silhouette area; (e) obtaining an intersection point only for a ray passing through a center point of the robot obtained by performing the elliptic fitting; (f) obtaining the position data of the robot by determining the intersections; And (g) performing low-pass filtering to remove noise of the obtained data to obtain a position of a final robot. The step (b) is characterized in that the step (b) is divided into a process of obtaining an internal parameter that matches the actual space and a pixel space of the camera, and a process of obtaining an external parameter that acquires a rotation matrix and a movement matrix of a camera center point in a global coordinate system.

As described above, the present invention proposes a method for measuring the position of a robot in three dimensions by installing multiple cameras on the upper part of a reactor vessel having a small radiation influence for measuring the position of the robot, deriving multiple images through the cameras.

Accordingly, since the robot position can be accurately measured in real time in the reactor, it is possible to improve the quality of the work and increase the speed of the work by implementing the convenience of the operator and the accuracy of the remote work.

In addition, the present invention provides a position sensing unit in the low radiation area outside the reactor vessel so as to prevent direct exposure to the nuclear reactor in the high radiation area.

Accordingly, the present invention utilizes vision for robot position measurement, thereby enabling quantitative robot position measurement as well as intuitive robot position observation, thereby doubling the convenience and reliability of remote operation.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a robot position measuring system according to a preferred embodiment of the present invention.
2 is a diagram showing a measurement method performed by a position measuring unit of the robot position measuring system,
3 is a conceptual diagram of a visual hull,
4 is a conceptual diagram showing a camera model in Zhang's calibration,
5 is a photograph showing coordinate pairs measured in an image,
6 is a photograph showing the camera calibration using the reactor pressure vessel bottom surface hole,
7A is a photograph showing the position of an actual camera,
7B is a photograph showing the position of the camera obtained through external calibration,
8 is a front view showing an actual robot,
9 is a photograph showing the silhouettes acquired by the respective robots,
10 is a photograph showing the result of performing RANSAC elliptic fitting,
Fig. 11 is a photograph showing a ray passing through the center point of the robot not exactly intersecting,
FIG. 12 is a conceptual diagram of a method for obtaining a position of a robot,
13 is a photograph showing the position acquisition of the robot using the intersection measurement algorithm,
14A to 14D show the actual positions of the robots in the experimental graphs of 0.1 m, 0.2 m, 0.3 m and 0.4 m from the bottom,
FIG. 15 is a graph of the position estimation in the constant velocity motion of the robot.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a robot position measuring system and a measuring method thereof according to preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a robot position measuring system according to a preferred embodiment of the present invention, and FIG. 2 is a diagram illustrating a measuring method performed by a position measuring unit of the robot position measuring system.

Referring to FIG. 1, the robot position measurement system according to the preferred embodiment of the present invention includes a thruster 101 in vertical and horizontal directions to move back and forth, right and left, and up and down at an arbitrary position in a reactor interior space 10 An underwater robot (100) in which an underwater camera (102) observing the nuclear reactor internal space (10) is installed at one end; A plurality of cameras (200) provided on the upper end (20) of the reactor vessel; A position measuring unit 300 for processing the multiple images through the plurality of cameras 200 and measuring the position and the posture of the underwater robot 100; And a remote controller 400 for controlling the position and orientation data of the underwater robot 100 measured by the position measuring unit 300 as a feedback signal.

Since the underwater robot 100 must be able to maintain an arbitrary angle at an arbitrary position in the reactor internal space 10, it can be moved in the vertical and horizontal directions of the underwater robot 100 so as to be able to move back and forth, Thrusters 101 are arranged so as to have a certain angle so as to have a 4-degree of freedom. In addition, the underwater robot 100 may be provided with an underwater camera 102 at one end thereof for observing the reactor internal space 10 so as to obtain an image desired by the operator.

Next, a plurality of cameras 200 may be installed in the upper reactor 20 to obtain multiple images. The plurality of cameras 200 preferably have three or more cameras, more preferably four or more cameras, to obtain a more precise multiple image.

The position measuring unit for processing the multiple images through the plurality of cameras 200 and measuring the position and attitude of the underwater robot 100 is installed in the reactor low external radiation region. The radiation exposure can be reduced. The position measuring unit 300 may include a DVR for acquiring an image and a PC for driving an algorithm. In the position measuring unit 300 having such a configuration, the position of the robot is three-dimensionally Will be described in detail below with reference to the robot position measurement method.

The remote control unit 400 for controlling the position and orientation data of the underwater robot 100 measured by the position measuring unit 300 as a feedback signal is installed in a safe area free from radiation exposure.

The remote control unit 400 can control the robot 100 to move to a desired position or to maintain a desired posture in order to inspect the inside of the nuclear reactor vessel based on the data measured by the position measuring unit.

Hereinafter, a method of measuring the robot position using the 3D image restoration algorithm using the multiple images performed by the position measuring unit 300 will be described in detail. For reference, a method of acquiring multiple images using four cameras and obtaining the position of the robot 100 through the four cameras will be described in detail.

FIG. 2 is a diagram illustrating a method of measuring a robot position according to the present invention. Referring to FIG. 2, the measuring method includes: (a) acquiring multiple images from a plurality of cameras; (b) determining positions of an image plane and a spatial position of the multiple images through camera correction; (c) measuring the silhouette of the robot in the multi-view image using hue detection; (d) performing elliptical fitting on the measured silhouette area; (e) obtaining an intersection point only for a ray passing through a center point of the robot obtained by performing the elliptic fitting; (f) obtaining the position data of the robot by determining the intersections; And (g) performing low-pass filtering to remove noise of the obtained data to obtain a position of a final robot.

Generally, the 3D shape restoration algorithm using multi-view images obtained from multiple cameras has various algorithms depending on the information used in the image such as visual hull, photo hull, voxel coloring, space carving, stereo vision, do. In the present invention, a visual hull algorithm is used for the robot position measurement. In this method, the silhouette of the object to be measured is acquired and the object is cross-projected on the three-dimensional space to acquire the intersection space to be.

FIG. 3 is a conceptual diagram of a visual hull. FIG. 3 (a) is a visual hull conceptual diagram in which a light ray in a space passing through a pixel of a camera origin and a silhouette image is obtained and the intersection thereof is obtained by using a voxel, a lookup table, an epipolar constraint, And a step of acquiring multiple images from a plurality of cameras is constituted.

       Next, (b) in the step of determining the position of the image plane and the spatial position of the multiple images through camera correction, the camera correction is a process of matching the actual space and the pixel space of the camera, Intrinsic calibration for obtaining internal parameters such as distortion coefficient, and extrinsic calibration for obtaining the rotation matrix and movement matrix of the camera center point in the actual spatial position of the camera, that is, the global coordinate system. After completing the camera calibration process, the projection matrix is finally obtained and the position of a point in space in the pixel space on the camera image can be known.

      Intrinsic calibration is performed using Zhang's camera calibration method, which is mainly used to obtain internal parameters. In Zhang's calibration, the camera model as shown in FIG. 4 is defined. Referring to FIG. 4, M represents a point on the world coordinate, m represents a pixel coordinate projected on the image, c represents a principal point, α and β represent an aspect ratio (x, y) aspect ratio, and is multiplied by the focal length. As a result, the relationship between the world coordinate M and the pixel coordinate m is defined as in equation (1).

      Where s is the scale factor, γ is the skew parameter, r and t are the rotation parameter and the translation parameter. Here, the intrinsic matrix is the first matrix of the right side, and is composed of focal length, principal point, and skew parameter reflecting the aspect ratio. Therefore, in order to obtain the intrinsic matrix, a solution is to be obtained by using a plurality of pairs of world-pixel coordinates. As shown in FIG. 5, a method of obtaining a pair of coordinates by measuring a previously known object in an image is used.

      Table 1 shows the internal parameters of four cameras using Zhang's calibration. These internal parameters need to be performed only once, unless the focal length and magnification of the camera are manipulated.

(Table 1)

Next, in order to obtain the external parameters, four points of coordinates known in the actual space are required. In general, since a cube or a pattern for camera correction is used, a camera correction cube or a pattern can not be installed inside the reactor vessel when the robot is actually operated. Therefore, camera calibration is performed using the reactor pressure vessel bottom hole as shown in FIG. . At this time, the origin of the global coordinate axis is the center of the bottom of the reactor vessel.

FIG. 7A shows the position of the actual camera, and FIG. 7B shows the position of the camera obtained through extrinsic calibration in three dimensions. The position of the camera can be obtained by the equation (2) using the rotation matrix and the movement matrix obtained as a result of extrinsic calibration.

C = - R _- T = - R _T T (2)

Where C is the position of the camera on world coordinates, R and T are the rotation and translation matrix obtained through extrinsic calibration.

Next, the step of (c) measuring the silhouette of the robot in the multi-view image using the color tone detection will be described.

       As a method of measuring the silhouette of the robot in each of multiple images, hue detection was used as the assumption that the underwater robot is unique in the color vessel inside the reactor vessel. This is a method of converting the input image into the HSI color space and then measuring it using the hue range of the robot. When one pixel of the image satisfies the condition of (4), it is determined that the corresponding point is a silhouette area. At this time, the morphology operation is performed several times in order to remove the noise and fill the inside of the silhouette. Equation (3) shows how to convert from RGB color space to HIS color space.

Where I _xy , S _xy , and H _xy are the illuminance, chroma, and hue values at the x and y positions of the image, and I _Th and S _Th are the thresholds at that time. These two values are adaptively determined according to the amount of ambient light, and H min and H max are determined by prior observation values.

      Fig. 8 is a front view showing an actual robot, and Fig. 9 is a result of acquiring a robot silhouette in each camera.

       (D) performing elliptical fitting on the measured silhouette area; And (e) obtaining an intersection point only for a ray passing through a center point of the robot obtained by performing the elliptic fitting.

       The ellipse fitting is performed on the robot silhouette region obtained in the above step (c), and the intersection points are obtained only for the rays passing through the center point of the obtained robot. In this way, it is possible to acquire the actual coordinates of the robot and only four intersection points can be obtained.

        Referring to FIG. 10, the RANSAC ellipse fitting is performed on the silhouette image. The RANSAC elliptic fitting is to obtain a conic parameter satisfying Eq. (5) using the five points given in the conic equation. In this case, the five points used as in the general RANSAC algorithm are randomly selected in the silhouette image, and the optimal ellipse is searched for until the inlier becomes maximum or the predetermined number of times is reached.

       The green circle shown in FIG. 10 represents the optimal ellipse found through the elliptic fitting algorithm, and the yellow circle at the center represents the robot center point. At this time, the rays passing through the center points of the robots in the respective multi-view images do not exactly cross each other as shown in Fig. This is due to the camera calibration error and the center point coordinate error of the robot obtained by the elliptic fitting. Therefore, an algorithm for finding the intersection point of light rays is required in consideration of such an error, and the proposed method is as follows.

     1) Find the equation of the straight line passing through the camera origin and the robot center point.

     2) Find the intersection of any plane and straight lines that are parallel to the x and y axes of the global coordinate system and have a z-axis height.

     3) Increase the z-axis height of the plane and repeat 2) to find the height of the z-axis and the midpoint of the intersection when the width of the square of each intersection is minimum.

      In this case, if the intersection point of the reference plane and each ray is Pi = (xi, yi), the width of the rectangle formed by the intersection points can be obtained by the slanting formula of the following equation (6). The z-axis height of the plane that minimizes the width is the z-axis coordinate of the robot, and the center point of the intersection points is the x- and y-axis coordinates of the robot.

       FIG. 12 is a conceptual diagram of a method for obtaining the position of a robot through intersection measurement. When the position of the robot is obtained by such a method, generally, it is superior in terms of speed and accuracy to a cross-area determination technique using a three- Performance. 13 shows the position of the robot using the proposed intersection measurement algorithm.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a robot position measuring system and a measuring method thereof according to the present invention will be described in detail.

Experiments were carried out in a 1/5 scale reactor pressure vessel mock-up, and the mock-up consists of a pressure vessel inner shape, a pressure vessel bottom hole, an underwater robot, and an underwater robot lifting device. At the top of the reactor vessel, four color cameras with a resolution of 352x240 were installed. DVR equipment was used for image acquisition. The specification of PC for algorithm driving is Intel i7 CPU and 16GB of memory. The total algorithm execution speed is about 22 frame / sec, which takes about 45 msec in one loop.

     The experiments were carried out in the following order. First, the actual position of the robot is moved 0.1m, 0.2m, 0.3m, and 0.4m from the floor, and the measurement results are obtained for 600 frames at each height. Then, the mean, maximum and minimum errors of the z-axis height are obtained and shown in Table 2 and the graph. Figs. 14A to 14D are graphs showing the experiment from 0.1 m to 0.4 m in order. The coordinates of the center point of the robot can not be accurately measured each time, and the value fluctuates. However, in Table 2, Can be seen.

    (Table 2)

       Next, FIG. 15 shows the result of the position estimation when the robot is moved at a constant speed using a motor mounted in a reactor mockup. It can be confirmed that the measurement result linearly follows the movement of the robot. The robot position detection experiment was performed in the 1/5 scale reactor pressure vessel mockup, and the accuracy of the underwater robot height accuracy within a few millimeters was obtained. In addition, the accuracy of the proposed algorithm is verified by showing the linearity of the position detection result of the underwater robot obtained by moving the robot at the constant height in the vertical direction, and it can be utilized for the remote control of the reactor and the control of the underwater robot in the future.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

The present invention is applied to a reactor internal inspection equipment or a field for remotely and precisely controlling a robot.

10: reactor interior 20: reactor upper end
100: underwater robot 101: thruster
102: underwater camera 200: plural cameras
300: Position measuring section 400: Remote control section

Claims (4)

A vertical and horizontal thruster 101 is provided to move back and forth, left and right, and up and down at an arbitrary position in the reactor internal space 10, and an underwater camera 102 observing the reactor internal space 10 is provided at one end An underwater robot 100 installed;
A plurality of cameras (200) provided on the upper end (20) of the reactor vessel;
A position measuring unit 300 for processing the multiple images through the plurality of cameras 200 and measuring the position and the posture of the underwater robot 100; And
And a remote control unit (400) for controlling the position and attitude data of the underwater robot (100) measured by the position measuring unit (300) as a feedback signal,
The method according to claim 1,
The position measuring unit 300 includes a DVR equipment for image acquisition and a PC for driving an algorithm.
(a) acquiring multiple images from a plurality of cameras;
(b) determining positions of an image plane and a spatial position of the multiple images through camera correction;
(c) measuring the silhouette of the robot in the multi-view image using hue detection;
(d) performing elliptical fitting on the measured silhouette area;
(e) obtaining an intersection point only for a ray passing through a center point of the robot obtained by performing the elliptic fitting;
(f) obtaining the position data of the robot by determining the intersections; And
(g) performing low-pass filtering to remove noise of the obtained data to obtain a position of a final robot
4. The method of claim 3, wherein step (b)
Obtaining an internal parameter that matches the actual space and the pixel space of the camera, and obtaining an external parameter that obtains a rotation matrix and a moving matrix of the camera center point in the global coordinate system.

Images