KR101923581B1 - Normal vector extraction apparatus and method thereof based on stereo vision for hull underwater inspection using underwater robot - Google Patents

Abstract

The present invention can extract a normal vector of a tangent plane viewed by a camera in order to enable a submersible robot to keep a distance and an attitude from a surface to be inspected constantly while following a hull inspection operation using an underwater robot The present invention relates to a stereo vision-based normal vector extraction apparatus and a method thereof for underwater inspection of a ship using an underwater robot.
The stereo vision-based normal vector extraction apparatus 100 includes a stereo-vision-based normal vector extraction unit 100 for receiving a stereo image including left and right images of a target plane photographed through a stereo camera, (110); A stereo vision unit 120 for calculating three-dimensional coordinates of stereo corresponding points based on the camera coordinate system included in the left and right images with improved image quality; And a plane determination unit 130 for determining a plane using three-dimensional coordinates obtained by the stereo vision unit 120 and extracting and outputting a normal vector.

Description

TECHNICAL FIELD [0001] The present invention relates to a stereo vision-based normal vector extracting apparatus and a method thereof for a submersible inspection of a hull using an underwater robot. [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hull underwater inspection, and more particularly, to a hull inspection method using an underwater robot, in which a camera is hung in order to keep a distance and a posture constant Based normal vector extracting apparatus and method thereof for an underwater inspection of a ship using an underwater robot capable of extracting a normal vector on a tangent plane.

Periodic underwater hull inspection is a very important process in terms of maintenance of the ship. The stability and efficiency of the existing hull inspection work performed by the divers can be greatly improved by introducing an underwater robot system. A hull surface image developed as a result of a hull inspection using an underwater robot can be obtained by applying an automatic image mosaicking algorithm to partially acquired images. In order to perform a more precise image mosaicking, the underwater robot needs to keep the distance and posture constant from the surface to be inspected and follow it. In order to realize this, it is essential to measure the normal vector of the tangent plane viewed by the camera.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the conventional technology, and an object of the present invention is to provide a method and apparatus for performing accurate image mosaicking on partially acquired images in order to obtain a hull- The normal vector of the object plane, which enables the robot to obtain the position and attitude information for aligning the image plane of the camera and the object plane so that the distance and posture from the surface to be inspected are kept constant, Based normal vector extracting apparatus and method thereof for an underwater inspection of a hull using an underwater robot capable of extracting the normal vector based on the characteristics of the underwater robot.

According to an aspect of the present invention, there is provided a stereovision-

An underwater image enhancement unit 110 that receives a stereo image including left and right images of a target plane captured through a stereo camera, and then changes the brightness distribution to improve the image quality;

A stereo vision unit 120 for calculating three-dimensional coordinates of stereo corresponding points based on the camera coordinate system included in the left and right images with improved image quality; And

And a plane determining unit 130 for determining a plane using the three-dimensional coordinates obtained by the stereo vision unit 120 and extracting and outputting a normal vector.

The underwater image enhancement unit 110 includes:

The contrast limited adaptive histogram equalization method may be applied to improve the image quality by changing the brightness distribution of the inputted left and right images.

The stereo vision unit (120)

It is possible to apply epipolar geometry and stereo matching to the left and right images whose image quality has been improved by the underwater image improving unit 120 so as to calculate the three-dimensional coordinates of the stereo corresponding points with respect to the camera coordinate system .

The plane determining unit 130 determines,

(RANDAM SAmple Consensus) technique is applied to the three-dimensional coordinate information obtained by the stereo vision unit 120 to determine a plane using residual coordinates and to derive a normal to the determined plane .

According to another aspect of the present invention, there is provided a stereovision-

An underwater image enhancement process (S100) in which an underwater image enhancement unit (110) receives a stereo image including left and right images of a target plane shot through a stereo camera, and then changes the brightness distribution to improve the image quality;

A three-dimensional coordinate calculation step (S200) of calculating the three-dimensional coordinates of the stereo corresponding points based on the camera coordinate system in which the stereo vision unit 120 is included in the left and right images with improved image quality; And

The plane determining unit 130 determines a plane using three-dimensional coordinates obtained from the stereo vision unit 120, extracts a normal vector, and outputs the extracted normal vector (S300).

The submerged image enhancement process (S100)

The contrast limited adaptive histogram equalization method may be applied to improve the image quality by changing the brightness distribution of the input left and right images.

The three-dimensional coordinate calculation process (S200)

Dimensional coordinates of stereo corresponding points on the basis of the camera coordinate system by applying epipolar geometry and stereo matching to the left and right images whose image quality is improved by the underwater image improving unit 120 .

The planarization process (S300)

A method of applying a RANSAC (Solidarity Consensus) technique to the three-dimensional coordinate information obtained by the stereo vision unit 120 to remove anomalous points, determining a plane using residual coordinates, and deriving a normal to the determined plane Lt; / RTI >

The present invention having the above-described structure can extract the normal vector of the target object plane from which the position and orientation information for aligning the image plane of the camera and the target object plane can be obtained by using the characteristics of stereo vision , The underwater robot can keep track of the distance and attitude from the surface to be inspected and follow the underwater robot, so that the partially captured images for obtaining the hull image development result, which is the result of the hull inspection using the underwater robot, It is possible to perform accurate image mosaicing, thereby providing an effect of greatly improving the efficiency of the hull inspection work using an underwater robot.

1 is a block diagram of a stereoscopic vision-based normal vector extraction apparatus 100 according to an embodiment of the present invention.
FIG. 2 is a diagram showing a result of applying a Contrast Limited Adaptive Histogram Equalization Method to a stereo image; FIG.
3 is a diagram illustrating a process of calculating three-dimensional coordinates of stereo corresponding points based on a camera coordinate system.
4 is a diagram showing a result of normal vector estimation;
5 is a flowchart showing a process of a stereo-based-based normal vector extraction method according to an embodiment of the present invention;

In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The embodiments according to the concept of the present invention can be variously modified and can take various forms, so that specific embodiments are illustrated in the drawings and described in detail in the specification or the application. It is to be understood, however, that the intention is not to limit the embodiments according to the concepts of the invention to the specific forms of disclosure, and that the invention includes all modifications, equivalents and alternatives falling within the spirit and scope of the invention. Also, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary " is not necessarily to be construed as preferred or advantageous over other aspects.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises ", or" having ", or the like, specify that there is a stated feature, number, step, operation, , Steps, operations, components, parts, or combinations thereof, as a matter of principle.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings showing embodiments of the present invention.

1 is a block diagram of a stereoscopic vision-based normal vector extraction apparatus 100 according to an embodiment of the present invention.

As shown in FIG. 1, the stereo vision-based normal vector extraction apparatus 100 includes an underwater image enhancement unit 110, a stereo vision unit 120, and a plane determination unit 130.

The underwater image enhancement unit 110 is configured to improve a picture quality by changing a brightness distribution after receiving a stereo image including left and right images of a target plane shot through a stereo camera. At this time, the underwater image enhancement unit 110 may be configured to apply the Contrast-limited Adaptive Histogram Equalization (HALF) technique to improve the image quality by changing the brightness distribution of the inputted left and right images.

FIG. 2 is a diagram illustrating a result of applying a Contrast Limited Adaptive Histogram Equalization Method to a stereo image. FIG.

As shown in FIG. 2, the Contrast-limited Adaptive Histogram Equalization Method is a method of improving the image quality by changing the brightness distribution of an image, minimizing the distortion problem of image data caused by the concentration of light when using artificial illumination in an underwater environment can do.

The stereo vision unit 120 is configured to calculate the three-dimensional coordinates of the stereo corresponding points based on the camera coordinate system included in the left and right images with improved image quality. The stereoscopic vision unit 120 applies epipolar geometry and stereo matching to the left and right images with improved image quality by the underwater image enhancement unit 120 so that stereoscopic images of three- And calculate the coordinates.

3 is a diagram illustrating a process of calculating three-dimensional coordinates of stereo corresponding points based on a camera coordinate system.

As shown in FIG. 3, if stereo matching is applied in a state where two image planes are aligned by a stereo calibration process in the stereo vision unit 120 to obtain an image point Pl and a corresponding point Pr, as shown in FIG. 3, The coordinate P can be obtained.

X: the x-axis coordinate of the corresponding point belonging to the left image, yl: the y-axis coordinate of the corresponding point belonging to the left image, and Ol: the left image, Iright: Y: the y axis coordinate of the corresponding point belonging to the right image, Or: the origin of the right image coordinate axis, X: the X axis of the camera reference coordinate system, Y: the origin of the right image coordinate axis, Z represents the Z axis of the camera reference coordinate system, f represents the focal length, T represents the distance between the coordinate axes of the two cameras, and P represents the three-dimensional coordinate of the corresponding point expressed in the camera reference coordinate system.

The plane determining unit 130 is configured to determine a plane using three-dimensional coordinates obtained from the stereo vision unit 120 and to extract and output a normal vector. The plane determining unit 130 determines a plane using residual coordinates by applying RANSAC (Random Absolute Consensus) technique to the three-dimensional coordinate information obtained by the stereo vision unit 120, And to derive a normal to the determined plane.

Specifically, the plane in which the three-dimensional point group corresponding to each pixel belongs is calculated through the three-dimensional coordinate information obtained by the stereo vision unit 120. Since the 3D coordinate information calculated through the image obtained in the underwater environment may contain many outliers, we use the RANSAC (Consecutive Random Space Consensus) method to robustly remove it. Through the application of the RANSAC technique, it is possible to extract steepest inliers with respect to the dominant plane. The following is an example of the pseudo-code of the RANSAC algorithm applied for the plane calculation.

4 is a diagram showing a result of normal vector estimation.

FIG. 4 shows a normal vector extraction result obtained by the stereo vision-based normal vector extraction apparatus 100 of the present invention. In FIG. 4, the green dot represents the group of inliers used for the plane calculation, and the red dot represents the points classified by the RANSAC algorithm as outliers. The blue line is the normal vector of the estimated plane through which the relative distance and attitude information of the camera to the reference plane can be calculated.

5 is a flowchart illustrating a process of a stereo-vector-based normal vector extraction method according to an embodiment of the present invention.

As shown in FIG. 5, the stereo vision-based normal vector extraction method includes an underwater image enhancement process S100, a three-dimensional coordinate calculation process S200, and a plane determination process S300.

First, an underwater image enhancement unit 110 receives a stereo image including left and right images of a target plane photographed through a stereo camera, and then changes the brightness distribution to improve an image quality. . The underwater image enhancement process S100 may be performed by applying a Contrast Limited Adaptive Histogram Equalization Method to improve the image quality by changing the brightness distribution of the input left and right images.

Next, the stereo vision unit 120 performs a three-dimensional coordinate calculation process (S200) of calculating three-dimensional coordinates of stereo corresponding points based on the camera coordinate system including the left and right images with improved image quality. In the three-dimensional coordinate calculation step (S200), epipolar geometry and stereo matching are applied to the left and right images with improved image quality by the underwater image improving unit 120, and a stereo corresponding point Dimensional coordinates of the target object.

Thereafter, the plane determining unit 130 determines a plane using the three-dimensional coordinates obtained by the stereo vision unit 120, extracts a normal vector, and performs a plane determining process (S300). The plane determining process S300 is a process of determining a plane using residual coordinates by applying a RANSAC (Random Absolute Consensus) technique to three-dimensional coordinate information obtained by the stereo vision unit 120, And deriving a normal to the determined plane.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: Stereo-based normal vector extraction device

Claims (8)

An underwater image enhancement unit 110 for improving the image quality by receiving a stereo image including left and right images of a target plane photographed through a stereo camera using an underwater robot for underwater inspection, ;
A stereo vision unit 120 for calculating three-dimensional coordinates of stereo corresponding points based on the camera coordinate system included in the left and right images with improved image quality; And
A plane determination unit (130) for determining a plane using the three-dimensional coordinates obtained by the stereo vision unit (120) and extracting and outputting a normal vector;
The underwater image enhancement unit 110 includes:
A stereo vision-based normal vector extraction device configured to apply a contrast-limited adaptive histogram equalization method to improve image quality by changing a brightness distribution of inputted left and right images. delete The stereoscopic vision system according to claim 1, wherein the stereoscopic vision unit (120)
Dimensional coordinates of stereo corresponding points on the basis of the camera coordinate system by applying epipolar geometry and stereo matching to the left and right images whose image quality is improved by the underwater image improving unit 120,
The plane determining unit 130 determines,
(RANDAM SAmple Consensus) technique is applied to the three-dimensional coordinate information obtained by the stereo vision unit 120 to determine a plane using residual coordinates and to derive a normal to the determined plane Stereo Vision Based Normal Vector Extraction Device. delete An underwater image enhancement process (S100) in which an underwater image enhancement unit (110) receives a stereo image including left and right images of a target plane shot through a stereo camera, and then changes the brightness distribution to improve the image quality;
A three-dimensional coordinate calculation step (S200) of calculating the three-dimensional coordinates of the stereo corresponding points based on the camera coordinate system in which the stereo vision unit 120 is included in the left and right images with improved image quality; And
A plane determining step (S300) of determining the plane by using the three-dimensional coordinates obtained by the stereo vision unit (120), extracting a normal vector and outputting the normal vector;
The submerged image enhancement process (S100)
A stereo vision-based normal vector extraction method performed by applying a Contrast-limited Adaptive Histogram Equalization (AMC) technique to improve image quality by varying the brightness distribution of inputted left and right images. delete The method of claim 5, wherein the three-dimensional coordinate calculation step (S200)
Calculating three-dimensional coordinates of stereo corresponding points on the basis of the camera coordinate system by applying epipolar geometry and stereo matching to the left and right images whose image quality is improved by the underwater image improving unit 120;
The planarization process (S300)
A method of applying a RANSAC (Solidarity Consensus) technique to the three-dimensional coordinate information obtained by the stereo vision unit 120 to remove anomalous points, determining a plane using residual coordinates, and deriving a normal to the determined plane Stereo Vision Based Normal Vector Extraction Method.
delete

Images