KR20220060283A - Method and system for obtaining 3d optical volume model for underwater objects - Google Patents

Abstract

Disclosed is a method for obtaining a 3D optical volume model for an underwater object. According to an exemplary embodiment of the present invention, the method for obtaining the 3D optical volume model for the underwater object using an optical camera and an imaging sonar comprises: a step of obtaining an optical volume model from an optical image generated by using the optical camera; a step of obtaining an acoustic volume model from the acoustic image generated by using the imaging sonar; and a step of correcting the optical volume model based on information on the acoustic volume model.

Description

METHOD AND SYSTEM FOR OBTAINING 3D OPTICAL VOLUME MODEL FOR UNDERWATER OBJECTS

The present invention relates to a method and system for acquiring a three-dimensional optical volume model, and more particularly, to a method and system for acquiring a three-dimensional optical volume model for an underwater object using an optical camera and an imaging sonar.

In general, SLAM (Simultaneous Localization And Mapping) is performed with an optical camera for three-dimensional reconstruction of underwater objects.

In this regard, shooting multiple angles around the target object in clear water enables precise SLAM with high-definition optical images in RGB colors provided by the optical camera. However, at depths of 50 m or more, only the blue light of the sun reaches, and at depths of 70 m or more, the sun hardly reaches. Therefore, the use of optical cameras is limited without a separate lighting system.

Also underwater, fragments of marine life, including the corpse of bacteria called marine snow, slowly settle to the sea floor, and near the sea floor, the visibility becomes cloudy due to this snow. In addition, since the optical image does not provide perspective information on the object on the image, in order to accurately estimate the distance between the optical camera and the object, the target object must be photographed from various angles. However, if the position of the optical camera is frequently changed on the seabed where GPS cannot be used, the error of the position measurement sensor used to estimate the position of the optical camera accumulates, and the position information of the optical camera becomes inaccurate. Do.

In comparison, the imaging sonar is less sensitive to the measurement environment, such as light scattering or water turbidity, and the acoustic image provided by the imaging sonar provides perspective and azimuth information about the position of the imaged object. However, there is a disadvantage in that the sound image is of low quality in black and white and does not provide information on the elevation of an object.

Therefore, there is a need for research on a method of reconstructing an underwater object in three dimensions more accurately by using an optical camera and an imaging sonar at the same time, but by compensating for each other's shortcomings.

Republic of Korea Patent Registration No. 10-2095009

An embodiment of the present invention is to provide a method of obtaining a three-dimensional optical volume model for an underwater object capable of increasing accuracy by minimizing perspective errors.

According to an aspect of the present invention, there is provided a method for obtaining a three-dimensional optical volume model for an underwater object using an optical camera and an imaging sonar, the method comprising: obtaining an optical volume model from an optical image generated using the optical camera; A three-dimensional optical volume model for an underwater object, comprising: obtaining an acoustic volume model from an acoustic image generated using the imaging sonar; and calibrating the optical volume model based on information in the acoustic volume model; A method of obtaining is provided.

In this case, in the step of correcting the optical volume model based on the information of the acoustic volume model, any one point included in the optical volume model may be moved to a point on the adjacent acoustic volume model.

In this case, the one point may mean a point located in an area where light reflection is difficult to occur when it is determined based on the acoustic volume model.

In this case, the one point may be moved to an intersection point where the reference point at which the optical camera is located and a straight line connecting the one point and the acoustic volume model meet.

In this case, the optical volume model may be obtained by estimating the three-dimensional coordinates of the point clouds constituting the surface of the underwater object using the optical image through a SLAM algorithm.

In this case, the step of obtaining the acoustic volume model from the acoustic image generated using the imaging sonar is setting a first boundary surface including a point where the first reflection of the acoustic beam occurs based on the position of the imaging sonar. and setting a second boundary surface including a point at which a final reflection of the sound beam occurs based on the position of the imaging sonar.

In this case, each of the first boundary surface and the second boundary surface may be formed to include a part of a spherical surface centered on a reference point where the imaging sonar is located.

In this case, the first boundary surface and the second boundary surface may be continuously updated according to the advancement of the image sonar.

In this case, in the step of correcting the optical volume model based on the information of the acoustic volume model, a first point existing inside the first boundary surface of the acoustic volume model among the optical volume model and a reference point at which the optical camera is located The first point may be moved to an intersection point where a straight line connecting , and the acoustic volume model meet.

At this time, in the step of correcting the optical volume model based on the information of the acoustic volume model, a second point located outside the second boundary surface of the acoustic volume model among the optical volume model and the optical camera are located The second point may be moved to an intersection point where the straight line connecting the reference points and the acoustic volume model meet.

According to another aspect of the present invention, there is provided an optical volume model acquisition system for acquiring a three-dimensional optical volume model for an underwater object using an optical camera and an imaging sonar, wherein the optical volume model is obtained from an optical image generated using the optical camera. an optical volume model unit to acquire; Based on the location of the imaging sonar from the sound image generated using the imaging sonar, the first boundary surface including the point at which the first reflection of the sound beam occurs, and the point at which the last reflection of the sound beam occurs and an acoustic volume model unit to obtain an acoustic volume model including an included second boundary surface, and correcting the optical volume model based on information of the acoustic volume model, wherein the correcting unit includes the acoustic volume model among the optical volume models. Moving the one point to the intersection point where a straight line connecting any one point located inside the first boundary surface of the volume model or outside the second boundary surface and the reference point where the optical camera is located and the acoustic volume model meet , an optical volume model acquisition system is provided.

The method of obtaining an optical volume model according to an embodiment of the present invention may obtain a more accurate optical volume model by correcting an error included in the optical volume model using the acoustic volume model.

In addition, in the method of obtaining an optical volume model according to an embodiment of the present invention, errors existing in various spaces in the optical volume model by subdividing the space by considering not only the first boundary surface but also the second boundary surface in relation to the acoustic volume model. can be corrected.

1 is a flowchart illustrating each step of a method of acquiring an optical volume model according to an embodiment of the present invention.
2 is a block diagram illustrating an optical volume model acquisition system according to an embodiment of the present invention.
3 is a diagram illustrating an example of forming an optical volume model using a SLAM algorithm.
4 is a diagram illustrating an optical volume model and an acoustic volume model obtained by an optical volume model acquisition system according to an embodiment of the present invention by superimposing them.
5 is a flowchart illustrating detailed steps of obtaining an acoustic volume model of a method of obtaining an optical volume model according to an embodiment of the present invention.
6 and 7 are diagrams for explaining acquiring an acoustic image using an imaging sonar.
8 is a view for explaining that the acoustic volume model unit of the optical volume model acquisition system according to an embodiment of the present invention updates the acoustic volume model.
9 and 10 are diagrams for explaining an acoustic volume model acquired by an optical volume model acquisition system according to an embodiment of the present invention.
11 is a diagram for explaining that the correction unit of the optical volume model acquisition system corrects points included in the optical volume model according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily carry out the present invention. The present invention may be embodied in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and the same reference numerals are given to the same or similar components throughout the specification.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or by circuit configurations for a given function. Also, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as an algorithm running on one or more processors. In addition, the present disclosure may employ prior art for electronic configuration, signal processing, and/or data processing, and the like. Terms such as “mechanism”, “element”, “means” and “configuration” may be used broadly and are not limited to mechanical and physical components.

1 is a flowchart illustrating each step of a method of obtaining an optical volume model according to an embodiment of the present invention. 2 is a block diagram illustrating an optical volume model acquisition system according to an embodiment of the present invention. 3 is a diagram illustrating an example of forming an optical volume model using a SLAM algorithm. 4 is a diagram illustrating an optical volume model and an acoustic volume model obtained by an optical volume model acquisition system according to an embodiment of the present invention by superimposing them.

A method of obtaining a three-dimensional optical volume model for an underwater object according to an embodiment of the present invention (hereinafter referred to as 'a method of obtaining an optical volume model') obtains a three-dimensional optical volume model from an optical image, and It is a method capable of improving the accuracy of the three-dimensional optical volume model by correcting it based on an acoustic image. In this case, each step of the method for acquiring an optical volume model according to an embodiment of the present invention may be performed by the optical volume model acquisition system 10 according to an embodiment of the present invention.

Hereinafter, each step of the method for obtaining an optical volume model according to an embodiment of the present invention will be mainly described, but also for the subject performing each step of the method for obtaining the optical volume model in the optical volume model obtaining system 10 . to describe together.

First, referring to FIGS. 1 and 2 , in a method of acquiring an optical volume model according to an embodiment of the present invention, the optical volume model is obtained by using the optical volume model unit 20 of the optical volume model acquisition system 10 . and acquiring. (S10)

In this case, the optical volume model unit 20 may obtain a 3D optical volume model from the 2D optical image. Here, the two-dimensional optical image may be a high-definition optical image of RGB color captured by the optical camera to the underwater object (M) in the water.

Specifically, in relation to the acquisition of the optical volume model, the optical volume model unit 20 processes the optical image through an open source-based SLAM (Simultaneous Localization and Mapping) algorithm, thereby forming the surface of the underwater object M. It is possible to estimate the three-dimensional coordinates of the cloud (K).

Here, referring to FIG. 3, the SLAM algorithm is a computer vision (CV)-based simultaneous localization and mapping algorithm, without requiring any prior knowledge of the surrounding environment, and the camera's position and / Or a technique for determining orientation and creating a geometrical model of the real surrounding environment. Since such a SLAM algorithm is a well-known technique in the field of location estimation, a detailed description thereof will be omitted herein.

In this case, the optical volume model unit 20 may estimate a position related to the underwater object M using a large-scale direct SLAM (LSD-SLAM) algorithm. This is to estimate the position more quickly by using Semi-Dense without going through a process of detecting a feature point with respect to the optical image generated by the optical camera.

In one embodiment of the present invention, the optical volume model may be shown in the form of a point cloud K along the surface of the underwater object M as shown in FIG. 4 . However, keep in mind that not all points constituting the point cloud K mean the surface of the underwater object M due to the perspective error of the optical volume model itself. The method of obtaining an optical volume model according to an embodiment of the present invention is a method of increasing the accuracy of an optical volume model by correcting a point cloud generated by the aforementioned perspective error using an acoustic volume model, which will be described later.

5 is a flowchart illustrating detailed steps of obtaining an acoustic volume model of a method of obtaining an optical volume model according to an embodiment of the present invention. 6 and 7 are diagrams for explaining acquiring an acoustic image using an imaging sonar. 8 is a view for explaining that the acoustic volume model unit of the optical volume model acquisition system according to an embodiment of the present invention updates the acoustic volume model. 9 and 10 are diagrams for explaining an acoustic volume model acquired by an optical volume model acquisition system according to an embodiment of the present invention.

Next, the method of acquiring an optical volume model according to an embodiment of the present invention includes acquiring an acoustic volume model using the acoustic volume model unit 30 of the optical volume model acquiring system 10 . (S20)

In this case, the acoustic volume model unit 30 may acquire the acoustic volume model by using the acoustic image generated using the imaging sonar.

이내에서 균등한 간격으로

개의 음향 빔을 발사하고(고도각은

), 상기 음향 빔이 수중 물체(M) 등에 의해 반사되어 돌아온 음향 빔을 감지함으로써 음향 이미지를 생성할 수 있다. 여기서, 이미징 소나가 감지할 수 있는 음향 빔의 반사 지점 거리 범위는

(최소 반사 지점 거리)에서 최대

(최소 반사 지점 거리) 일 수 있다.In one embodiment of the present invention, the imaging sonar is azimuthal, as shown in FIG.

equally spaced within

Fire a sound beam of dogs (the elevation angle is

), the sound beam may be reflected by an underwater object (M) and the like, and by detecting the returned sound beam, an acoustic image may be generated. Here, the range of the reflection point distance range of the acoustic beam that the imaging sonar can detect is

(minimum reflection point distance) to maximum

(minimum reflection point distance).

-번째 음향 빔이 거리

에서 반사된다면, 음향 이미지의

-번째 열,

-번째 행에 해당하는 픽셀의 밝기 값은 반사돼 돌아온 음향 빔의 강도가 강할수록 높아지며, 최종적으로

크기의 음향 이미지가 생성된다. (여기서,

,

,

은 샘플링 수)More specifically, referring to FIG. 7 , each pixel in an acoustic image may have different contrasts according to the intensity of the reflected and returned acoustic beam. in other words,

-th sound beam distance

If reflected from

-th column,

The brightness value of the pixel corresponding to the -th row increases as the intensity of the reflected and returned sound beam becomes stronger, and finally

An acoustic image of the size is created. (here,

,

,

is the number of samples)

In this regard, the meaning of each pixel in the sound image will be described as follows. Specifically, the acoustic image may be divided into three types of regions. Referring to (a) of FIG. 8, it can be seen that each column of the acoustic image is composed of a black pixel (S1), a white or gray pixel (S2), and another black pixel (S3) from a position close to the imaging sonar. there is.

At this time, the first black pixel S1 corresponds to a space in which there is no sound beam reflected back because an object does not exist. And the white or gray pixel S2 corresponds to the space where the sound beam is reflected by the object or the seabed. The last black pixel S3 corresponds to a space where the sound beam cannot reach because the object or the sea floor is blocked. Since light cannot reach the area S3 because it is blocked by the same object or the sea floor, the space S3 corresponds to a space where even an optical camera cannot be observed.

-번째 열에 대해서만 살펴보면,

-번째 열에서 이미징 소나와 가장 가까운 지점(

)에서 시작되어 가장 마지막으로 연속적인 검은 픽셀(

)은

-번째 열에서 음향 빔의 첫 번째 반사(first reflection, 이하, 'f.r.')가 발생한 지점까지의 거리(

)와 대응된다. Continue to refer to (a) of Figure 8, but for convenience of explanation,

If we look only at the first column,

The point closest to the imaging sonar in the -th column (

) to the last consecutive black pixel (

)silver

- the distance from the first column of the acoustic beam to the point where the first reflection (hereafter, 'fr') occurs (

) corresponds to

-번째 열에서 이미징 소나와 가장 먼 지점(

)에서 시작되어 가장 마지막으로 연속적인 검은 픽셀(

)은 음향 빔의 마지막 반사(last reflection, 이하, 'l.r.')가 발생한 지점까지의 거리(

)와 대응된다. 이와 관련하여,

-번째 빔의

과

는 다음 수학식 1을 이용하여 계산될 수 있다.Similarly,

In the -th column, the point farthest from the imaging sonar (

) to the last consecutive black pixel (

) is the distance to the point where the last reflection (hereinafter, 'lr') of the acoustic beam occurs

) corresponds to In this regard,

- of the second beam

class

can be calculated using Equation 1 below.

[Equation 1]

,

,

.

.

Next, a process in which the acoustic volume model unit 30 acquires a three-dimensional acoustic volume model from the acoustic image will be described in detail.

The process of obtaining a three-dimensional acoustic volume model from the acoustic image largely includes the steps of setting a first boundary surface A1 including a point at which the first reflection of the acoustic beam occurs based on the position of the imaging sonar (S21); , it can be divided into a step (S22) of setting the second boundary surface A2 including the point at which the last reflection of the acoustic beam occurs.

-번째 열(

-번째 음향 빔에 의한 음향 이미지)에 대하여, 첫 번째 반사(f.r)와 마지막 반사(l.r)가 발생한 반사 지점을 포함하는 구면을 상정하여 음향 체적 모델의 일부를 생성할 수 있다. 이 때, 상기 구면은 이미징 소나가 위치하는 기준점을 중심으로 한다.In order to set the first boundary surface A1 or the second boundary surface A2, the acoustic volume model unit 30 first

-th column (

For the acoustic image by the -th acoustic beam), a part of the acoustic volume model can be generated by assuming a sphere including the reflection points where the first reflection (fr) and the last reflection (lr) occur. In this case, the spherical surface is centered on the reference point where the imaging sonar is located.

)과 고도각(

)을 고려하여 수학식으로 나타내면 다음과 같다.Part of this acoustic volume model is the azimuth of the acoustic beam (

) and the elevation angle (

) in consideration of the equation, it is as follows.

[Equation 2]

Sphere containing first point of reflection =

Sphere containing last reflection point =

,

,

.

,

,

.

또는

)가 동일하다면 이들은 음향 이미지 상에서 동일한 픽셀로서 표현될 수 있기 때문이다.As such, in an embodiment of the present invention, constructing an acoustic volume model assuming a spherical surface including the reflection point where the reflection occurs without specifying one coordinate specifying the position of the first or last reflection point is the reflection point. It is due to the characteristics of the imaging sonar that the information on the elevation angle of the is not known. That is, even if the elevation angle of the reflection point is different as shown in FIG. 9, the distance to the reflection point (

or

) are the same, because they can be expressed as the same pixel on the sound image.

-번째 열에 대하여 음향 체적 모델의 일부를 생성하는 작업을,

개의 전체 음향 빔에 대하여 모두 수행한 후, 이들을 일체로 연결하면 제 1 경계면(A1)과 제 2 경계면(A2)이 형성될 수 있다. 여기서, 제 1 경계면(A1)은 첫 번째 반사 지점을 포함하는 구면

을 연결하여 형성된 것이며, 제 2 경계면(A2)은 마지막 반사 지점을 포함하는 구면

을 연결하여 형성된 것임은 물론이다. 도 8에는 상술한 제 1 경계면(A1)과 제 2 경계면(A2)의 일단면이 도시되어 있음을 유의해야 한다.As above, the sound image

-creating part of the acoustic volume model for the first column,

After performing all of the sound beams, the first boundary surface A1 and the second boundary surface A2 may be formed by integrally connecting them. Here, the first boundary surface A1 is a spherical surface including the first reflection point.

It is formed by connecting , and the second boundary surface A2 is a spherical surface including the last reflection point.

Of course, it is formed by connecting . It should be noted that one end of the above-described first interface A1 and second interface A2 is illustrated in FIG. 8 .

Meanwhile, as the imaging sonar advances in the direction of the underwater object M, the first interface A1 and the second interface A2 may be continuously updated.

In more detail, the imaging sonar repeatedly radiates a sound beam while moving in the direction of the underwater object (M) to expand the detection range of spatial information. 2 The interface A2 may be repeatedly generated.

In this regard, the acoustic volume model unit 30 cumulatively connects the individual first and second boundary surfaces A1 and A2 generated over time, and the first boundary surface A1 and the second boundary surface. (A2) can be newly updated. 8( a ) to (e) of FIG. 8 , the updating process of the first boundary surface A1 and the second boundary surface A2 is arranged in chronological order. By the above-described updating process of the boundary surfaces A1 and A2, the final first boundary surface A1 and the second boundary surface A2 can be determined as shown in FIG. 10, and an acoustic volume model including them can be generated. can

Next, with reference to FIG. 5 , the meaning of each space in the acoustic volume model divided by the first boundary surface A1 and the second boundary surface A2 will be examined. In this specification, in defining the location of the space divided by the boundary plane in the present specification, a place relatively adjacent to the imaging sonar is defined as the inside, and a location far from the imaging sonar is defined as the outside.

First, the first space S'1 existing between the first boundary surfaces A1 is a space in which the reflection of the acoustic beam does not occur, and may be a space in which the underwater object M does not exist.

And, the second space S'2 existing between the first boundary surface A1 and the second boundary surface A2 is a space in which the underwater object M is likely to exist. At this time, the underwater object M may not exist in the interior of the second space S'2, which is, as described above, that the boundary surfaces A1 and A2 of the acoustic volume model include the reflection point of the acoustic beam. This is because it consists of a comprehensive spherical surface.

Finally, the third space S'3 existing outside the second boundary surface A2 means a space that cannot be reached because the propagation of the acoustic beam is blocked by the underwater object M or the sea floor. Accordingly, the third space S'3 may mean a space occupied by the real underwater object M, or may mean an empty space beyond the underwater object M or a virtual space under the seabed.

As such, in the optical volume model acquisition system 10 according to an embodiment of the present invention, in configuring the acoustic volume model, not only the first interface A1 including the point at which the first reflection occurs, but also the last reflection f.r. By considering the second boundary surface A2 including the point where ?

11 is a view for explaining that the correction unit of the optical volume model acquisition system corrects points included in the optical volume model according to an embodiment of the present invention.

Meanwhile, the method of obtaining the optical volume model according to an embodiment of the present invention includes the step of the correction unit 40 correcting the optical volume model based on the above-described information on the acoustic volume model. (S30)

In an embodiment of the present invention, the corrector 40 of the optical volume model acquisition system 10 may move any one point included in the optical volume model to a point on an adjacent acoustic volume model. At this time, among the points included in the optical volume model, the points P1 and P2 moving to the points on the acoustic volume model are points located in an area where light reflection is difficult to occur when judged based on the acoustic volume model. can do.

More specifically, referring to FIG. 10 , as described above, the underwater object M may be an area inside the first boundary surface A1 of the acoustic volume model, that is, in the first space S′1, in which the underwater object M does not exist. . Therefore, among the point clouds K constituting the optical volume model, it can be determined that the first point P1 existing inside the first boundary A1 of the acoustic volume model is generated by the error of the optical volume model. In addition, by moving the first point P1 to an adjacent point on the acoustic volume model or excluding it from the optical volume model altogether, the accuracy of the optical volume model can be increased.

In addition, the outside of the second boundary surface A2, that is, the third space S'3, may be an area in which light cannot be reflected because light cannot reach by the underwater object M or the seabed. Therefore, since the second point P2 existing inside the third space S'3 may also be generated by an error of the optical volume model, it may be moved to an adjacent point on the acoustic volume model or a point of the optical volume model By excluding the cloud K, the accuracy of the optical volume model can be improved.

,

)으로 제 1 점(P1) 또는 제 2 점(P2)을 이동시킬 수 있다. 이 경우, 광학 카메라를 향하는 광의 진행 경로를 최대한 고려하여 점구름을 이동시키므로, 광학 체적 모델이 가진 원근 오차를 효과적으로 해결할 수 있는 장점이 있다.Referring back to FIG. 11 , as a specific example of moving the first point P1 and the second point P2 due to the error of the optical volume model, the reference point where the optical camera is located and the first point P1 or the second The intersection of the straight line (L1, L2) connecting the points (P2) and the acoustic volume model (

,

) to move the first point P1 or the second point P2. In this case, since the point cloud is moved by considering the propagation path of light toward the optical camera as much as possible, there is an advantage in that the perspective error of the optical volume model can be effectively resolved.

As another example of moving the first point (P1) or the second point (P2), the first point (P1) or the second point (P2) is selected only in any one of the X-axis, Y-axis and Z-axis directions. It can also be moved to a point on the acoustic volume model that meets it when moved. In the case of such a movement method, since only movement in one axial direction is considered, the amount of calculation related to movement is remarkably reduced, and there is an advantage in that it is possible to reduce the time required to correct the optical volume model.

As described above, in the method of obtaining an optical volume model according to an embodiment of the present invention, a more accurate optical volume model can be obtained by correcting a perspective error included in the optical volume model using the acoustic volume model.

In addition, the method of obtaining an optical volume model according to an embodiment of the present invention subdivides the space by considering not only the first boundary surface A1 but also the second boundary surface A2 in relation to the acoustic volume model to be within the optical volume model. A perspective error existing in the third space S'3 can also be eliminated, so that a more accurate optical volume model can be obtained.

Although one embodiment of the present invention has been described above, the spirit of the present invention is not limited to the embodiments presented herein, and those skilled in the art who understand the spirit of the present invention can add components within the scope of the same spirit. , changes, deletions, additions, etc. may easily suggest other embodiments, but this will also fall within the scope of the present invention.

,

교차점

최소 반사 지점 거리

최대 반사 지점 거리 S'1 제 1 공간
S'2 제 2 공간 S'3 제 3 공간
S1, S2, S3 음향 이미지의 픽셀10 Optical volume model acquisition system 20 Optical volume model unit
30 Acoustic volume model unit 40 Correction unit
M underwater object A1 first interface
A2 2nd interface K point cloud
P1 first point P2 second point

,

crossing

Minimum Reflection Point Distance

Maximum Reflection Point Distance S'1 First Space
S'2 second space S'3 third space
Pixels of the S1, S2, S3 sound image

Claims (11)

A method for obtaining a three-dimensional optical volume model for an underwater object using an optical camera and an imaging sonar, comprising:
obtaining an optical volume model from an optical image generated using the optical camera;
obtaining an acoustic volume model from an acoustic image generated using the imaging sonar; and
and correcting the optical volume model based on information in the acoustic volume model. The method of claim 1,
In the step of correcting the optical volume model based on the information of the acoustic volume model, any one point included in the optical volume model is moved to a point on the adjacent acoustic volume model, a three-dimensional optical volume model for an underwater object. how to obtain it. 3. The method of claim 2,
The one point is a method of obtaining a three-dimensional optical volume model for an underwater object, which means a point located in an area where light reflection is difficult to occur when judged based on the acoustic volume model. 4. The method of claim 3,
The one point is a reference point where the optical camera is located and a straight line connecting the one point, and the acoustic volume model is moved to an intersection point, How to obtain a three-dimensional optical volume model for an underwater object. The method of claim 1,
The optical volume model is obtained by estimating the three-dimensional coordinates of a point cloud constituting the surface of the underwater object using the optical image through a SLAM algorithm, a method of obtaining a three-dimensional optical volume model for an underwater object. The method of claim 1,
Acquiring the acoustic volume model from the acoustic image generated using the imaging sonar comprises:
setting a first boundary surface including a point at which a first reflection of an acoustic beam occurs based on the position of the imaging sonar; and
based on the position of the imaging sonar, setting a second boundary surface including a point at which the last reflection of the sound beam occurs. 7. The method of claim 6,
The method for acquiring a three-dimensional optical volume model for an underwater object, wherein the first interface and the second interface each include a part of a sphere centered on a reference point on which the imaging sonar is located. 7. The method of claim 6,
wherein the first interface and the second interface are continuously updated according to the advancement of the image sonar. 7. The method of claim 6,
In the step of correcting the optical volume model based on the information of the acoustic volume model, a first point existing inside the first interface of the acoustic volume model among the optical volume model and a reference point at which the optical camera is located are connected. A method of obtaining a three-dimensional optical volume model for an underwater object, moving the first point to an intersection point where a straight line and the acoustic volume model meet. 7. The method of claim 6,
In the step of correcting the optical volume model based on the information of the acoustic volume model, a second point located outside the second boundary surface of the acoustic volume model among the optical volume model and a reference point at which the optical camera is located are connected A method of obtaining a three-dimensional optical volume model for an underwater object, moving the second point to an intersection point where a straight line and the acoustic volume model meet. An optical volume model acquisition system for acquiring a three-dimensional optical volume model for an underwater object using an optical camera and an imaging sonar, comprising:
an optical volume model unit for obtaining an optical volume model from an optical image generated using the optical camera;
Based on the location of the imaging sonar from the sound image generated using the imaging sonar, the first boundary surface including the point at which the first reflection of the sound beam occurs, and the point at which the last reflection of the sound beam occurs an acoustic volume model unit to obtain an acoustic volume model including the included second boundary surface; and
a correction unit for correcting the optical volume model based on the information of the acoustic volume model;
The correction unit includes a straight line connecting any one point located inside the first boundary surface or outside the second boundary surface of the acoustic volume model among the optical volume model and a reference point at which the optical camera is located, and the acoustic volume model is An optical volume model acquisition system that moves the one point to an intersection point where it meets.

Images