KR20190089489A - Apparatus and method for detecting object candidate of image using Additive Kernel BING - Google Patents

Abstract

The present invention relates to an apparatus and a method for detecting an object candidate which approximate an output of an AK SVM by calculation of binary vectors, and detect an object candidate in an input image by applying the AK SVM to BING. The apparatus of the present invention comprises: an index information calculating unit for calculating index information with respect to pixels on the basis of approximate values obtained by calculating NG values of the pixels related to an NG feature map obtained from an input image and a reference value; an index information updating unit for updating index information on an adjacent pixel adjacent to a reference pixel on the basis of index information on a reference pixel whenever the reference pixel is selected among pixels; and an object candidate detecting unit for detecting at least one object candidate from the input image on the basis of an inner product between a first vector related to a first pixel selected from the pixels and a second vector related to a second pixel selected from the pixels when the index information on the pixels is updated.

Description

[0001] APPARATUS AND METHOD FOR DETECTING IMAGE OBJECT CANDIDATE USING AKBING [0002]

The present invention relates to an apparatus and method for detecting object candidates. More particularly, the present invention relates to an apparatus and method for detecting an object candidate in an image.

Binarized Normed Gradients (BING) is one of the proposed methods for video-based candidates. It gives a score to candidates in all areas, and suggests candidates with high scores as final candidates.

BING is scored using a linear SVM (Support Vector Machine) based on the NG (Normed Gradients) feature containing edge information. Such a BING has the advantage of processing at a high speed since all the processes of extracting features and calculating scores are approximated by operations using atom manipulation.

However, applying kernel SVM instead of linear SVM to BING can improve the accuracy of BING. However, since the kernel SVM requires operation with a support vector for each input data, the calculation time is long.

Korean Registered Patent No. 1,569,411 (Notification date: Nov. 27, 2015)

SUMMARY OF THE INVENTION The present invention has been conceived to solve the problems described above, and it is an object of the present invention to approximate an output of an AK (Additive Kernel) SVM (Support Vector Machine) by the operation of binary vectors, apply AK SVM to Binarized Normed Gradients And an object candidate detecting apparatus and method for detecting an object candidate in an image.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

SUMMARY OF THE INVENTION The present invention has been made in order to achieve the above object, and it is an object of the present invention to provide an image processing apparatus and a method thereof, which are capable of obtaining, based on approximate values obtained by calculating NG values and reference values of pixels related to an NG (Normed Gradient) feature map, An index information calculation unit for calculating index information; An index information updating unit updating the index information of the adjacent pixels adjacent to the reference pixel based on the index information of the reference pixel every time the reference pixel is selected from the pixels; And updating at least one object in the input image based on an inner product between a first vector associated with a first pixel selected from the pixels and a second vector associated with a second pixel selected from the pixels, And an object candidate detecting unit for detecting an object candidate.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: calculating index information on the pixels based on approximate values obtained by calculating NG values and reference values of pixels related to an NG (Normed Gradient) feature map obtained from an input image; Updating index information of an adjacent pixel adjacent to the reference pixel based on index information of the reference pixel every time a reference pixel is selected among the pixels; And updating at least one object in the input image based on an inner product between a first vector associated with a first pixel selected from the pixels and a second vector associated with a second pixel selected from the pixels, And detecting a candidate of an object candidate.

The present invention can achieve the following effects through the configurations for achieving the above object.

First, the processing speed and accuracy can be improved at the same time when the candidate group is selected from the image.

Second, when the present invention is applied to an ADAS (Advanced Driver Assistance System), a vehicle can recognize a pedestrian quickly and accurately, thereby reducing vehicle accidents.

1 is an exemplary diagram showing a table operation using a lookup table.
FIGS. 2 and 3 are exemplary diagrams illustrating a process of approximating a table operation using a lookup table by weighting and obtaining an output value for an input value.
FIGS. 4 to 6 are exemplary diagrams illustrating a process of updating i _{x, y,} and R _{x, y} using an NG feature map having a W × H size
FIG. 7 is a conceptual diagram schematically illustrating an internal configuration of an object candidate detection apparatus according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

Based on the score calculated using linear SVM (Support Vector Machine) based on the NG (Normed Gradients) feature, the proposed method of the existing image based candidate group proposes a candidate group. Where NG denotes the Euclidean norm of the gradient. However, this method has a problem that the accuracy is somewhat low.

In the present invention, AKBING (Additive Kernel Binarized Normed Gradients) algorithm, in which an additive kernel SVM is applied to Binary (Binarized Normed Gradients) in order to obtain high accuracy with high processing speed, is proposed.

With AK SVM, it is possible to perform fast operation with excellent performance using the kernel. The algorithm proposed in the present invention is able to effectively select candidates (improve accuracy) while maintaining the fast processing speed of BING by applying AK SVM having stronger performance than linear SVM to BING.

Hereinafter, the present invention (proposal of candidates using AKBING) will be described in detail with reference to the drawings.

SVM is a binary classifier with robust performance, which is used in many fields. The existing linear SVM can be regarded as a problem of obtaining a hyperplane that maximizes the margin between classes.

However, linear SVM has a disadvantage that the performance is degraded for nonlinear data. To solve this problem, kernel SVM using kernel has been suggested. In the case of the kernel SVM, the optimization problem can be expressed as Equation 1 based on the training dataset.

First, the training data set can be expressed as follows.

S = {(g ⁽ⁱ⁾ , y ⁽ⁱ⁾ )} _{i = 1} ^L

In the g ⁽ⁱ⁾ and y ⁽ⁱ⁾ is a training ^{data, g (i) = [g} 1 (i), g 2 (i), ... , g _N ⁽ⁱ⁾ ] ^T ∈ R ^N and y ⁽ⁱ⁾ ∈ {-1, +1}. i denotes an index, and L denotes the number of input data.

The optimization problem can be expressed on the basis of this training data set as follows.

In the above optimization equation, K means the kernel matrix. K can be expressed by the following equation.

_{K i, j = κ (g} (i), g (j)) = φ (g (i)) · φ (g (j)) ∈ R

Also, α means a dual variable used for optimization of the kernel SVM. As a result of the above optimization, α is calculated as follows.

α = [α ⁽¹⁾ , α ⁽²⁾ , ... , α ^(L) ] ^T ∈ R

The classification equation of the kernel SVM can be expressed as the following equation (2) with the above α.

Where b is the bias and b ∈ R. Also, κ means kernel trick.

As can be seen from Equation (2), unlike the linear SVM, the kernel SVM requires a kernel operation with all the training data in one test sample. Therefore, there is a problem that the kernel SVM has to carry all the training data. In addition, the testing time and the calculation are long when each sample is classified.

An additive kernel is proposed to solve these computational problems. The additive kernel means a kernel in which the value of the kernel is the sum of kernel-level values of the data as shown in Equation (3).

In the above, κ _n (g _n , z _n ) denotes a kernel value of each dimension. g and z can be expressed as follows.

g = {g ₁ , g ₂ , ... , g _N } ∈ R ^N

z = {z ₁ , z ₂ , ... , z _N } ∈ R ^N

Where N is the number of kernels.

The main advantage of the additive kernel is that it can reduce the amount of computation required for the testing classification process to a very simple operation using tables. The classification equation of the kernel SVM can be converted into the sum of the one-dimensional function values by dimension as shown in Equation (4) using the characteristics of the additive kernel.

As can be seen from Equation 4, the one-dimensional function h _n (g _n ) consists of α, training data g ⁽ⁱ⁾ and y ⁽ⁱ⁾ resulting from the kernel SVM optimization, The output values can be used as a table for testing purposes.

In other words, if the output values of the one-dimensional function h _n (g _n ) for each dimension are set as a table, the output value can be easily obtained as the table value corresponding to the input value without calculating the output value separately for the test input value .

1 is an exemplary diagram showing a table operation using a lookup table (LUT). 1 shows a process of obtaining an output value h _n (g _n ) for an input value g _n through a table operation (LUT _n (g _n )).

_{g = [g 1, g 2} , ... , g ₆₄ ] ∈ [0, 255] ⁶⁴

In the case of BING, 64 NG features are used as shown in the above equation. If AK SVM is used, 64 table operations must be repeated for each candidate group. To change this efficiently, it is necessary to simplify the table operation.

In the present invention, a table operation for obtaining an output value of a corresponding input value is approximated by multiplying it by a weight. FIGS. 2 and 3 are exemplary diagrams illustrating a process of approximating a table operation using a lookup table by weighting and obtaining an output value for an input value.

FIG. 2 is an example of a table value approximated by eight values. In FIG. 2, reference numeral 110 denotes an LUT _n , and reference numeral 120 denotes an approximated LUT _n . As shown in FIG. 2, since the table has a gentle curved shape, it can be approximated with a small error even with eight values. Assuming that the table LUT _n is approximated by eight values, LUT 8 _n , the operation through the approximated table can be changed to the inner form of the eight vectors as shown in the following equation (5).

Refers to g _n are n dimensional features NG (NG n th dimension value) in the, and, xi _n means a binary index vector (binarized index vector). xi _n is defined by the following Equation (6).

In the above, ┗┛ denotes a floor function. FIG. 3 shows a process of obtaining h _n (g _n ) using <LUT 8 _n , x i _n >.

Since the NG features are all 64-dimension as described above, 64 approximated tables and binary index vectors can be connected to one vector as shown in Equation (7).

And the output of the AK SVM can be approximated as shown in Equation (8).

To approximate the LUT with a linear combination of binary vectors to obtain fast computational effects of atomic operation,

In the above, N _L means the number of binary vectors, and N _L ? 512. a _Lj is a jth binary vector, a _Lj ∈ { _-1,1 } ⁵¹² , and β _Lj is a constant corresponding to each binary vector.

Substituting the LUT of Equation (9) into Equation (8) yields Equation (10).

In the XI is _{XI = [xi 1, xi 2} , ... , xi ₆₃ , xi ₆₄ ], and one for each dimension. Thus, | XI | is always equal to 64 regardless of the input data g.

Also, a _Lj ⁺ is a _Lj ⁺ ∈ {1, 0} ⁵¹² and can be obtained by the following equation (11).

The output value calculation formula for the final AK BING is shown in Equation (11).

Where N _w denotes the number of binary vectors, and N _w ≤ 64.

<a _Lj ^+, XI> is capable of fast operation with atomic operation since the operation between the operation of the binary vector. Also, the XI input to the input in Equation (12) can be efficiently and quickly extracted using atomic operation using a binary vector, i.e., bitwise shift.

Figs. 4 to 6 are exemplary diagrams showing a process of updating i _{x, y} and R _{x, y} using an NG feature map (g ^{W x H} ) having a W x H size. Here, i _{x, y} means each xi _n , and R _{x, y} means a collection of 8 x i _n adjacent to each i _{x, y} . For example, R _{x, y} is [x i ₁ , x i ₂ , ... , xi ₇ , xi ₈ ], [xi ₉ , xi ₁₀ , ... , xi ₁₅ , xi ₁₆ ], and so on.

When the first pixel is g _1,1 = 133, the NG value corresponds to the fifth interval from the left of the eight approximation values (see FIGS. 2 and 3) when calculating based on Equation 6, so xi ₁ = [0 0 0 0 1 0 0 0]. Similarly, when the next pixel is g _2,1 = 97, the NG value is encoded as xi ₂ = [0 0 0 1 0 0 0], since it corresponds to the fourth value from the left of the eight approximation values.

To avoid loop computation, the eight adjacent LUT indices ([xi ₁ , xi ₂ , ..., xi ₇ , xi ₈ ]) in the same row are mapped to each other in INT64 (64 variables) . For example, the last 8 bits of R _1,1 are xi ₁ = [0 0 0 0 1 0 0 0] as shown at 210 in FIG. 4, and the last 16 bits of R _2,1 are Are interconnected by xi ₁ = [0 0 0 0 1 0 0 0] and xi ₂ = [0 0 0 1 0 0 0 0] as shown at 220. As a result of this connection, we can obtain R _1,1 , R _{2,1 and} so on with AK BING features.

In the present invention, i _{x, y} can be obtained by BYTE (eight variables), and R _{x, y} can be obtained by INT 64 (64 variables).

i _{x, y} can be obtained by an 8-bit index of the NG value g _{x, y} , and can be directly calculated from g _{x, y} using a bit shift (BITWISE SHIFT) and a multiplication. R _{x, y} are adjacent to R _{x, y} using a bitwise operation such as BITWISE SHIFT, BITWISE AND, etc. _, and R _{x-1, y} . &lt; / RTI &gt;

The process shown in FIG. 4 can be summarized as follows.

First, given an NG image (8 bits) with W × H size, it creates an 8-bit image i and a 64-bit image R with the same size.

Then, each pixel i _{x, y} of i is given an 8-bit binary index for the g _{x, y} values at the same position.

Then, R _{x, y,} which is 64 bits, inserts R _x-1 located at the left side, i _{x, y} which is 8 bit at _y , as bit shifts.

If the process is repeated using the process described with reference to FIG. 4, the final shape is as shown in FIG. 5 is an exemplary diagram showing a process of updating the R matrix extracted from the NG feature map.

As shown in FIG. 5, R _{x, y} contains eight left binary indexes based on the corresponding position. Accordingly, the final XI can be extracted by connecting eight R _{x, y} in the vertical direction on the basis of the position as shown in FIG. I.e. final XI is R _{x, y} in the _{[R x, y-7,} R x, y-6, ... , R _{x, y-2} , R _{x, y-1} ].

The method proposed in the present invention efficiently extracts XI since all i _{x, y} and R _{x, y} are updated by one or eight binary bits to the side adjacent feature through BITWISE SHIFT.

The algorithm proposed by the present invention described above is described as follows.

Algorithm: Score computation of AKBING W × H positions

Input: normed gradient map g ^{W x H}

Ouput: score matrix s ^{W x H}

Initialize: R ^{W × H} = {0} ^{W × H} , s ^{W × H} = {0} ^{W × H}

// Compute R _{x, y} ^{W x H} of the top 7 rows

for y = 1 to 7 do

for x = 1 to W do

If x = 1 do

R _{x, y} ^{W x H} = ({0} ^{64 & lt} ; 8) | i _{x, y}

else

R _{x, y} ^{W x H} = (R _{x-1, y?} 8) | _{x, y}

end

end

end

// Compute score s _{x, y}

// using [Rx _{, y-} ^7WxH , _{Rx, y-} ^6WxH , ... , R _{x, y-1} ^{W x H} , R _{x, y} ^{W x H} ]

for y = 8 to H do

for x = 1 to W do

If x = 1 do

R _{x, y} ^{W x H} = ({0} ^{64 & lt} ; 8) | i _{x, y}

else

R _{x, y} ^{W x H} = (R _{x-1, y?} 8) | _{x, y}

end

XI = [Rx _{, y-} ^7WxH , _{Rx, y-} ^6WxH , ... , R _{x, y-1} ^{W x H} , R _{x, y} ^{W x H} ]

end

end

The present invention relates to a method for proposing a candidate group through BING (AK BING) using an additive kernel as described above. AK SVM requires calculation using a table. In the present invention, by developing a method of approximating the operation of binary vectors such as atom manipulation of BING, the accuracy of BING can be improved and the accuracy can be improved.

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention has been described with reference to Figs. Best Mode for Carrying Out the Invention Hereinafter, preferred forms of the present invention that can be inferred from the above embodiment will be described.

FIG. 7 is a conceptual diagram schematically illustrating an internal configuration of an object candidate detection apparatus according to a preferred embodiment of the present invention. 7, the object candidate detection apparatus 300 includes an index information calculation unit 310, an index information update unit 320, an object candidate detection unit 330, a power supply unit 340, and a main control unit 350.

The power supply unit 340 performs a function of supplying power to the respective constituent elements of the object candidate detection apparatus 300.

The main control unit 350 performs a function of controlling the overall operation of each constituent constituting the object candidate detection apparatus 300.

The index information calculation unit 310 calculates index information on pixels based on approximate values obtained by calculating NG values and reference values of pixels related to an NG (Normed Gradient) feature map obtained from an input image . The index information calculation unit 310 may perform the above-mentioned function in updating i _{x, y} from g _{x, y} .

The index information calculation unit 310 may calculate index information as index information in binary form.

The index information calculation unit 310 may include any one of binary numbers to calculate index information as index information in binary form. The index information calculation unit 310 may calculate the index information as index information in binary form by changing the position of any one number according to the magnitude of the approximate values.

The index information calculation unit 310 may calculate index information as index information in binary form using a bitwise operation. The index information calculation unit 310 may use a bit shift (BITWISE SHIFT) as a bit operation.

The index information calculation unit 310 may calculate approximate values by calculating the NG values and the reference value using a floor function. The index information calculation unit 310 may calculate approximate values by applying a floor function to a value obtained by dividing the NG values by the reference value.

The index information updating unit 320 updates the index information of the adjacent pixels adjacent to the reference pixel based on the index information of the reference pixel every time the reference pixel is selected from among the pixels. The index information updating unit 320 may perform the above-mentioned function in updating R _{x, y} from i _{x, y} .

The index information updating unit 320 can update the index information of the adjacent pixels based on the index information of the reference pixel when acquiring the index information whose bit number is increased than before. For example, the index information updater 320 can perform the above-described function when acquiring index information whose bit number is increased from 8 bits to 64 bits.

The index information updating unit 320 may update the index information of the adjacent pixels by inserting the index information of the reference pixel into the index information of the adjacent pixel by using the bit operation. The index information updating unit 320 can use bit shift and bitwise AND as bit operations.

The index information updating unit 320 may update the index information of the adjacent pixels by inserting the index information of the reference pixels before the index information of the adjacent pixels.

The index information updating unit 320 can use the pixel located on the same row as the adjacent pixel as the reference pixel when updating the index information of the adjacent pixel.

When the index information of the pixels is updated by the index information updater 320, the object candidate detector 330 detects a first vector associated with the first pixel selected from the pixels and a second vector associated with the second pixel selected from among the pixels. And detects at least one object candidate in the input image based on the inner product. The object candidate detection unit 330 can perform the above function based on Equation (12).

The object candidate detection unit 330 calculates a score for the pixels based on the inner product between the first vector and the second vector, the number of vectors related to the pixels, constants related to the vectors, The object candidate can be detected from the input image.

The object candidate detection apparatus 300 described above can operate when detecting a pedestrian positioned ahead in the vehicle in which the vehicle is traveling.

Next, an operation method of the object candidate detection apparatus 300 will be described.

First, the index information calculation unit 310 calculates index information for the pixels based on the approximate values obtained by calculating the NG values and the reference values of the pixels related to the NG (feature map) feature map obtained from the input image ( STEP A).

The index information updating unit 320 updates the index information of the adjacent pixels adjacent to the reference pixel based on the index information of the reference pixel every time the reference pixel is selected among the pixels (STEP B).

When the index information of the pixels is updated, the object candidate detecting unit 330 detects at least one of the first vector associated with the first pixel selected from the pixels and the second vector associated with the second pixel selected from the pixels, One object candidate is detected (STEP C).

It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, or the like can be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (16)

An index information calculation unit for calculating index information on the pixels based on approximate values obtained by calculating NG values and reference values of pixels related to an NG (Normed Gradient) feature map obtained from an input image;
An index information updating unit updating the index information of the adjacent pixels adjacent to the reference pixel based on the index information of the reference pixel every time the reference pixel is selected from the pixels; And
Wherein when the index information of the pixels is updated, based on an inner product between a first vector associated with a first pixel selected from the pixels and a second vector associated with a second pixel selected from the pixels, an object candidate detector for detecting an object candidate,
And an object candidate detection unit for detecting an object candidate of an image. The method according to claim 1,
Wherein the index information calculation unit calculates the index information as index information in a binary number format. The method according to claim 1,
Wherein the index information calculation unit includes one of the binary numbers and calculates the index information as index information in binary form. The method of claim 3,
Wherein the index information calculation unit changes the position of any one of the numbers according to the magnitude of the approximate values, and calculates the index information as the index information of the binary number type. The method according to claim 1,
Wherein the index information calculation unit calculates the index information as binary index information using a bitwise operation. 6. The method of claim 5,
Wherein the index information calculation unit uses a bit shift (BITWISE SHIFT) as the bit operation. The method according to claim 1,
Wherein the index information calculation unit calculates the approximate values by calculating the NG values and the reference value using a floor function. 8. The method of claim 7,
Wherein the index information calculation unit calculates the approximate values by applying the floor function to a value obtained by dividing the NG values by the reference value. The method according to claim 1,
Wherein the index information updating unit updates the index information of the adjacent pixel based on the index information of the reference pixel when acquiring index information in which the number of bits is increased. The method according to claim 1,
Wherein the index information updating unit updates the index information of the adjacent pixels by inserting the index information of the reference pixels into the index information of the adjacent pixels using a bit operation. 11. The method of claim 10,
Wherein the index information updating unit uses bit shift and bitwise AND in the bit operation. 11. The method of claim 10,
Wherein the index information updating unit updates the index information of the adjacent pixels by inserting the index information of the reference pixels before the index information of the adjacent pixels. The method according to claim 1,
Wherein the index information updating unit uses a pixel located on the same row as the adjacent pixel as the reference pixel when updating the index information of the adjacent pixel. The method according to claim 1,
Wherein the object candidate detection unit calculates a score for the pixels based on an inner product between the first vector and the second vector, a number of vectors related to the pixels, constants related to the vectors, And detects the object candidate in the input image based on the score. The method according to claim 1,
Wherein the object candidate detecting apparatus operates when detecting a pedestrian positioned ahead in the vehicle in which the vehicle is traveling. Calculating index information for the pixels based on approximate values obtained by calculating NG values and normalized values of pixels related to an NG (Normed Gradient) feature map obtained from an input image;
Updating index information of an adjacent pixel adjacent to the reference pixel based on index information of the reference pixel every time a reference pixel is selected among the pixels; And
Wherein when the index information of the pixels is updated, based on an inner product between a first vector associated with a first pixel selected from the pixels and a second vector associated with a second pixel selected from the pixels, a step of detecting an object candidate
And detecting the candidate of the object of the image.

Images