KR101952291B1 - Object appearance detection method using multi-beam sonar camera - Google Patents

Abstract

The present invention relates to a method for detecting appearance of an object in an image of a multi-beam sonar camera comprising: (a) a step of emitting a plurality of ultrasonic beams in a row direction by a multi-beam sonar camera, and receiving reflection waves to determine a pixel value representing brightness of a pixel row in accordance with strength; (b) a step of inserting data of a first length (Z) with a pixel value of 0 in front of and behind the determined pixel value to generate first and second shift data; (c) a step of setting a pulse train having the same length as the first and second shift data and having a variable amplitude; (d) a step of adding the pulse train to the first and second shift data to calculate first and second sum data, and considering consistency (correlation) of the first and second sum data while changing the amplitude of the pulse train to determine a threshold amplitude; and (e) a step of relatively comparing threshold amplitudes of the pixel rows to determine that an object has appeared in a corresponding row if the threshold amplitude is increased.

Description

[0001] The present invention relates to an object appearance detection method using a multi-beam sonar camera,

The present invention relates to a method for detecting the appearance of an object in an image of a multi-beam ultrasonic camera, and more particularly, to a method for detecting the appearance of an object in a multi-beam ultrasonic camera, To a method for detecting the appearance of an object.

Generally, an ultrasonic camera is one of devices for acquiring an underwater terrain of a poor view or an underwater object as an image, and is used for the purpose of underwater exploration and underwater object detection. An ultrasound camera emits ultrasound waves and produces images using reflected waves reflected from underwater features.

FIG. 1 is a schematic view showing an ultrasonic emission mode of a general multi-beam ultrasonic camera. In FIG. 1, a plurality of sector beams 2 are emitted forward in a multi-beam ultrasonic camera 1, and the echo is measured to generate an image.

In order to detect an image of a certain area forward from the position of the multi-beam ultrasonic camera, the sector beam is fired in the form of an ultrasonic beam having a narrow width in the left and right direction and a spreading shape in the up and down direction.

The number of the sector beams 2 of the multi-beam ultrasonic camera determines the number of columns.

2 is an explanatory view for explaining the meaning of columns in an image of a general multi-beam ultrasonic camera.

Referring to FIG. 2, since an image obtained using a multi-beam ultrasonic camera forms one pixel column C, the number of sector beams 2, which are ultrasound beams of a multi-beam ultrasonic camera, is the number of columns in the image .

Cn represents the closest point, and Cd represents the furthest point.

A multi-beam ultrasound camera measures the reflected wave for a set time after emitting the beam. At this time, the reflected wave reflected at the farthest point (Cd) is different in time from the nearest point (Cn), and the distance can be obtained by multiplying the time difference by the velocity of the ultrasonic wave.

The brightness of the pixel is brighter as the intensity of the reflected wave is larger.

Fig. 3 is a graph showing a difference between a row in which an object is photographed and a row in which an object is not photographed.

Referring to FIG. 3, there is no object 3 in the column C12 of the 12th ultrasonic beam of the multi-beam ultrasonic camera, and the object is located in the column 46 of the 46th ultrasonic beam.

At this time, the pixel values of the 12th column (C12) are uniformly distributed. However, since strong reflection occurs on the surface of the object in the 46th column (C46) where the object (3) is scanned, .

However, in order to automatically detect that an object is detected in a scanning process, a pixel value indicating brightness according to a change in intensity of a reflected wave must be detected as shown in the graph of FIG. 3, but there is no actual object due to the material of the bottom surface. It is difficult to distinguish the accurate object, for example, a reflected wave having a strength level of about a certain level can not be received.

Registration No. 10-1696086 (registered on January 6, 2017, method and apparatus for extracting object part in sonar image)

An object of the present invention is to provide a method of detecting the appearance of an object in an image of a multi-beam ultrasonic camera capable of automatically detecting that an object appears in a scan region of a multi-beam ultrasonic camera.

According to an aspect of the present invention, there is provided a method of detecting the appearance of an object in an image of a multi-beam ultrasonic camera, comprising the steps of: a) firing a plurality of ultrasonic beams in a column direction in a multi-beam ultrasonic camera, Determining a pixel value representing the brightness of the pixel column according to the intensity; b) inserting data of a first length (Z) having a pixel value of 0 before and after the determined pixel value, respectively, C) setting a pulse train having the same length as the first shift data and the second shift data and capable of varying the amplitude; d) The first sum data and the second sum data are calculated by adding pulse trains to the data and the second shift data, respectively, and the match degree of the first sum data and the second sum data Inertia) step and, e) to determine the amplitude threshold in consideration of a step of determining that if the threshold amplitude increased compared relative to a threshold amplitude of the pixel column, the column is an object appeared.

According to an embodiment of the present invention, the pulse trains in the step c) may include a first pulse having a non-zero amplitude in the first pulse width W and a single pulse having the same amplitude in a period in which the amplitude is zero, Shift data and the length of the second shift data.

According to an embodiment of the present invention, the length of the first pulse width W may be twice the first length Z of the data.

According to an embodiment of the present invention, the match degree of the step d) may be obtained by superimposing the first sum data and the second sum data directly on the first match data to obtain the first match degree, The second sum data may be shifted by the first length Z and superimposed with the first sum data to obtain the second match degree.

According to an embodiment of the present invention, the threshold amplitude may be set such that, when the second match degree is larger than the first match degree, the amplitude of the pulse train is changed so that the first match degree becomes larger than the second match degree, Lt; / RTI >

According to an embodiment of the present invention, the second match degree is greater than the first match degree, the pixel value of the pixel column of step a) may be larger than the amplitude of the pulse train.

According to an embodiment of the present invention, the state where the first match degree is larger than the second match degree may be such that the amplitude of the pulse train is larger than the pixel value of the pixel column of the step a).

According to an embodiment of the present invention, when the second match degree is greater than the first match degree, the value of the pulse train is increased to make the first match degree greater than the second match degree, It can be a large value.

A method of detecting the appearance of an object in an image of a multi-beam ultrasonic camera according to the present invention includes adding a pulse train to a pixel value determined according to the intensity of a reflected wave of each of a plurality of ultrasonic beams, The value of the pulse train whose value becomes higher than the value of the column pixel is detected and the value of the detected pulse train is compared with the value of the pulse train so that the object is detected Thereby, it is possible to recognize that an object appears automatically in the scan area.

1 is a schematic view for explaining a general multi-beam ultrasonic camera.
2 is an explanatory view for explaining the meaning of columns in an image of a general multi-beam ultrasonic camera.
Fig. 3 is a graph showing a difference between a row in which an object is photographed and a row in which an object is not photographed.
4 is a flowchart of a method of detecting the appearance of an object in an image of a multi-beam ultrasonic camera according to a preferred embodiment of the present invention.
In the graph of FIG. 5, pixel values for index k in one column are shown.
6 is a graph of shift data.
7 is a waveform diagram of a pulse train.
8 and 9 are illustrations of first sum data and second sum data, respectively.
Fig. 10 is an example of the critical amplitude of each column in the absence of an object.
Fig. 11 is an example of the critical amplitude of each column when there is an object.

Hereinafter, a method for detecting the appearance of an object in an image of a multi-beam ultrasonic camera according to the present invention will be described in detail with reference to the accompanying drawings.

The embodiments of the present invention are provided to explain the present invention more fully to those skilled in the art, and the embodiments described below can be modified into various other forms, The scope of the present invention is not limited to the following embodiments. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a", "an," and "the" include plural forms unless the context clearly dictates otherwise. Also, " comprise " and / or " comprising " when used herein should be interpreted as specifying the presence of stated shapes, numbers, steps, operations, elements, elements, and / And does not preclude the presence or addition of one or more other features, integers, operations, elements, elements, and / or groups. As used herein, the term " and / or " includes any and all combinations of one or more of the listed items.

Although the terms first, second, etc. are used herein to describe various elements, regions and / or regions, it is to be understood that these elements, parts, regions, layers and / . These terms do not imply any particular order, top, bottom, or top row, and are used only to distinguish one member, region, or region from another member, region, or region. Thus, the first member, region or region described below may refer to a second member, region or region without departing from the teachings of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings schematically showing embodiments of the present invention. In the figures, for example, variations in the shape shown may be expected, depending on manufacturing techniques and / or tolerances. Accordingly, embodiments of the present invention should not be construed as limited to any particular shape of the regions illustrated herein, including, for example, variations in shape resulting from manufacturing.

4 is a flowchart of a method of detecting the appearance of an object in an image of a multi-beam ultrasonic camera according to a preferred embodiment of the present invention.

Referring to FIG. 4, the method includes determining (S10) a pixel value indicating the brightness of a pixel in each column, and inserting data of a predetermined length having a pixel value of 0 before and after the determined pixel value, A step (S30) of setting a pulse train having the same length as the first shift data and the second shift data and capable of varying the amplitude, (S40) of adding a pulse train to the first shift data and the second shift data to determine a critical amplitude while changing the value of the pulse train, comparing the critical amplitudes of the respective columns with each other, (S50) that an object is detected in the corresponding column when the relative increase is detected.

Hereinafter, the structure and operation of a method for detecting the identity of an object in an image of a multi-beam ultrasonic camera according to a preferred embodiment of the present invention will be described in more detail.

First, in step S10, a pixel value indicating the brightness of the pixel is determined according to the intensity of the reflected waves received from the multi-beam ultrasonic camera.

In the graph of FIG. 5, pixel values for index k in one column are shown.

Here, the index can be understood as an array of pixel values between near and far from one column as in Fig.

In Fig. 5, Ci denotes the i-th column, and M-1 is a constant.

The multi-beam ultrasound camera moves in a specific direction for capturing an underwater terrain, and pixel values of the corresponding column are determined to be different values depending on the existence of an object, the material and shape of the object, and the material and form of the bottom surface.

In step S20, as shown in FIG. 6, data having a pixel value of 0 and a length of Z are inserted before and after the determined pixel value, respectively, and the first shift data 10 and the second shift Data 20 is generated.

The first shift data has a pixel value of 0 and a length of Z is inserted in front of the data of FIG. 5 to shift the index backward. The second shift data has the same data inserted backward, And is shifted forward relative to the data 10. [

The first shift data 10 and the second shift data 20 have the same length of M + Z-1. Here, M is the length of the index, and Z is the length of the data in which the pixel value inserted in the above-described pixel column is zero.

로 표시되며 여기서 U는 열의 앞쪽에 데이터가 삽입되어 시프트된 것을 나타내기 위한 것이며, 제2시프트 데이터(20)는

로 표시되어 있으며 여기서 D는 열의 뒤쪽에 데이터가 삽입되어 시프트된 것을 나타낸다.6, the first shift data 10 is

Where U is for indicating that data is inserted and shifted in front of the column, and the second shift data 20 is for

, Where D indicates that data is inserted and shifted to the rear of the column.

The length of the index (k) of the first shift data 10 and the second shift data 20 is M + Z-1.

In the first shift data 10, the pixel value is 0 as the inserted data from 0 to Z-1, and the pixel values from Z to M + Z-1 follow the result of the scan.

In the second shift data 20, the index k indicates the pixel value corresponding to the result of the scan from 0 to M-1, and the pixel value from M to M + Z-1 is zero.

Therefore, the first shift data 10 and the second shift data 20 do not overlap each other in a state in which they are not shifted separately.

Next, in step S30, a pulse train having the same length as the first shift data 10 and the second shift data 20 and capable of varying the amplitude is set.

An example of the pulse train 30 is shown in Fig. The pulse train 30 (? (K)) is a series of single pulses having a pulse width W and an amplitude A, whose length is the same value as the first shift data 10 and the second shift data 20 .

The pulse width W is set to be twice the length Z of the data having the pixel value of 0.

Accordingly, the pulse trains 30 are repeated at intervals equal to the length of Z so that A and 0 are not zero.

Next, in step S40, a pulse train 30 is added to the first shift data 10 to obtain the first sum data 40 of FIG. 8, and a pulse train (second shift data) 30) to obtain the second sum data 50 of FIG.

(k)와 Ω(k)의 성분 합이며, 제2합데이터(50) Gi(k)는 제2시프트 데이터(20)

(k)와 Ω(k)의 성분 합이다.The first sum data 40 (Fi (k)

(k) and? (k), and the second sum data 50 (Gi (k)

(k) and? (k).

At this time, the pulse trains 30 added to the first shift data 10 and the second shift data 20 are all shifted, and the first sum data 40 and the second sum data 50 are not shifted It must be superimposed to be a conforming shape. For example, when all the pixel values of the first shift data 10 and the second shift data 20 are 0, the first sum data 40 and the second sum data 50 are both supplied to the pulse train 30, Are overlapped with each other.

If the pulse width maximum value of the pulse train 30 is much larger than the amplitudes of the first shift data 10 and the second shift data 20, The amplitude of the pulse train 30 does not affect the pulse width of the pulse train 30. When the first sum data 40 and the second sum data 50 are overlapped without shifting, the degree of matching is high.

Conversely, when the pulse width maximum value of the pulse train 30 is very small compared with the amplitudes of the first shift data 10 and the second shift data 20, the amplitude of the pulse train 30 is smaller than the amplitude of the first sum data 40 The first sum data 40 and the second sum data 50 are shifted by Z in a specific direction and then superimposed so that the degree of matching is high.

Therefore, when the first summation data 40 and the second summation data 50 are superimposed without shifting, the degree of agreement and the degree of agreement after the shift are obtained. When it is confirmed that the degree of agreement in each case is higher, The relative magnitude of the amplitude of the first shift data 10 or the second shift data 20 can be measured.

The match degree X (k) of the first sum data 40, Fi (k) and the second sum data 50, Gi (k) can be calculated from the following equation (1).

X _{i, 1} (k) and X _{i, 2} (k) in Equation 1 represent the correlation (degree of agreement) between the pixel value of the i-th column and the pulse train.

X _{i, 3} (k) and X _{i, 4} (k) represent the cross-correlation of the pixel values of the i-th column and the pulse trains.

(K), X _{i, 1} (k), X _{i, 2} (k), X _{i, 3} (k) and X _{i, 4} (k)

In the above Equation (2), when the index k is equal to Z, X _{i, 1} (Z) is the pixel value of the i-th column and can be expressed by Equation (3) below.

In Equation (3), Pi represents the magnitude of the pixel value as a correlation of the pixel values in the i-th column.

In Equation (2), when the index k is equal to Z, the value of X _{i, 2} (k) becomes zero. This is because the pulse width W of the pulse train 30 is twice the Z, and the product of the element units of? (T + Z) and? (T) is zero.

The sum of X _{i, 3} (k) and X _{i, 4} (k) defined in Equation (2) can be expressed as Equation (4) below.

The correlation Xi (Z) is calculated by substituting Pi, which is a value of X _{i, 1} (k) obtained in Equation (3) and 0 being a value of X _{i, 2} (k) into Equation 1, Can be expressed by the following equation (5).

The above correlation is obtained when the index k is Z in the i-th column, and when k is 0, X _{i, 1} (k) can be expressed by the following equation (6).

That is, in Equation (6), Qi indicates the correlation between the data in which the data having the pixel value of 0 is inserted for the period Z and the data not inserted.

When k is 0, X _{i, 2} (0) is equal to the size of the pulse train and can be expressed by Equation (7) below.

The sum of X _{i, 3} (k) and X _{i, 4} (k) at this time is expressed by the following equation (8).

As a result, when k is 0, the correlation Xi (0) is expressed by the following equation (9).

When the amplitude of the pulse train 30 is varied and the pulse train 30 dominates and the first sum data 40 and the second sum data 50 are superimposed on each other without a shift, The amplitude of the pulse train 30 when the match degree when the first sum data 40 and the second sum data 50 are overlapped becomes higher is defined as a threshold amplitude Ath.

Expressing this through the equation, the correlation Xi (0) when the index k is 0 in the i-th column means the degree of agreement when superimposed in the non-shifted state, and when the index k is Z, .

Therefore, the critical amplitude (Ath) is the point at which the maximum agreement degree changes from Xi (Z) to Xi (0), and Xi (O) and Xi (Z) are identical.

According to the above definition, the mathematical definition of the critical amplitude (Ath) can be expressed by the following equation (10).

Next, in step S50, a critical amplitude is calculated for the pixel values converted into the brightness information of the pixels according to the intensity of the reflected waves of all the ultrasonic beams of the multi-beam ultrasonic camera. That is, after determining the critical amplitudes of a plurality of pixel columns, the critical amplitudes of the respective columns are compared with each other to determine that an object is detected in the corresponding column if the critical amplitude is relatively increased above a set value.

The threshold amplitude (Ath) obtained above is different for each pixel column of the multi-beam ultrasonic camera. In the case of not including an object, the pixel column has only a gentle change due to slight bending or physical properties of the bottom surface. Therefore, as described above, when the characteristics of the pulse train 30 are expressed and they are superimposed in a non-shifted state, the degree of agreement is higher.

Therefore, the pulse train 30 can be dominant even at a relatively low pulse intensity (amplitude A), and the critical amplitude (Ath) is small.

The state at this time is shown in Fig.

FIG. 10 is a graph showing the critical amplitudes calculated for 96 total columns, in which the critical amplitudes of the respective columns are all low and thus no object appears.

Conversely, when an object is included, the pixel value has a sharp pixel value change due to the object surface reflection, and the influence of the pixel value detected is larger than that of the pulse train 30.

Accordingly, the degree of agreement between the first sum data 40 and the second sum data 50 is increased when shifting by Z is performed.

At this time, since the critical amplitude Ath is a point where the degree of agreement after shifting is high and the degree of matching when shifting without shifting is switched to a high state, the intensity of the pulse train 30 must be increased , And the critical amplitude (Ath) is increased.

The change in the critical amplitude (Ath) in the state where the object is detected is shown in Fig.

In FIG. 11, the critical amplitudes in the 60th to 80th columns are higher than the critical amplitudes of the other columns, and it can be seen that the object is detected in the 60th to 80th columns.

When the critical amplitude is used, the noise component can be removed and the appearance of the object can be accurately detected as compared with the method using the intensity of the reflected wave directly.

The processing or execution means of each of the steps described in the above-described embodiments of the present invention can be a computing means such as a microprocessor, and the microprocessor can access memory means for temporary or permanent storage of data. The calculating means and the memory means may be included in means for controlling the multi-beam ultrasonic camera or means for receiving and processing the image photographed from the multi-beam ultrasonic camera.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention will be.

10: first shift data 20: second shift data
30: Pulse train 40: First sum data
50: second sum data

Claims (8)

a) emitting a plurality of ultrasound beams in a column direction in a multi-beam ultrasound camera, receiving each of the reflected waves and determining a pixel value indicative of the brightness of the pixel column according to the intensity;
b) generating first shift data and second shift data by inserting data of a first length (Z) having a pixel value of 0 before and after the determined pixel value, respectively;
c) setting a pulse train having the same length as the first shift data and the second shift data and capable of varying the amplitude;
d) adding the pulse trains to the first shift data and the second shift data, respectively, to calculate first sum data and second sum data, and changing the amplitude of the pulse train, Determining a critical amplitude in consideration of the degree of matching of the input signal; And
e) comparing the critical amplitudes of the pixel columns with each other to determine that an object appears in the column when the critical amplitude increases, and detecting the appearance of the object in the image of the multi-beam ultrasound camera. The method according to claim 1,
The pulse trains of step c)
The single pulse having a length equal to the length of the first shift data and the length of the second shift data is equal to the length of the first shift data and the second shift data. A method for detecting the appearance of an object in an image of a multi-beam ultrasonic camera. 3. The method of claim 2,
Wherein the length of the first pulse width (W) is twice the first length (Z) of the data. The method according to claim 1,
The match degree of the step d)
The first sum data and the second sum data are superimposed on each other to obtain a first match degree,
The first sum data is shifted by a first length Z to overlap the second sum data to determine a match degree or the second sum data is shifted by a first length Z to overlap the first sum data, Wherein the degree of agreement is determined based on the degree of agreement. 5. The method of claim 4,
The threshold amplitude may be determined by:
And the amplitude of the pulse train at the boundary where the first match degree is larger than the second match degree by changing the amplitude of the pulse train in a state where the second match degree is larger than the first match degree, A method for detecting the appearance of an object in a vehicle. 5. The method of claim 4,
And the second degree of match is greater than the first degree of match,
Wherein the pixel value of the pixel column in the step a) is larger than the amplitude of the pulse train. 5. The method of claim 4,
When the first match degree is greater than the second match degree,
Wherein the amplitude of the pulse train is larger than the pixel value of the pixel column of the step a). 5. The method of claim 4,
And when the second degree of match is larger than the first degree of match, the value of the pulse train is increased so that the first degree of match is made larger than the second degree of match and the threshold amplitude becomes a relatively large value. A method for detecting the appearance of an object in a beam ultrasound camera image.

Images