KR20180082299A - Method and apparatus for accelerating foreground and background separation in object detection using stereo camera - Google Patents

Abstract

The present invention relates to a method for separating a foreground and a background by extracting a depth image through a stereo camera, forming an occupancy lattice map based on the depth image, calculating a free space, and calculating a membership value. The method comprises the following steps: setting a threshold value of a foreground object existing in the predicted free space boundary area; determining whether the membership value reaches the threshold value of the foreground object while calculating the membership value; terminating calculation of the membership value when the membership value being calculated in the determination step reaches the threshold value of the foreground object; and separating a foreground and a background through the calculated membership value.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for accelerating foreground / background classification in an object detection method using a stereo camera,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of classifying foreground and background of an object detection method using a stereo camera, and a background and object classification acceleration method and apparatus using occupancy lattice information according to a prior probability distribution.

In order to detect an object using a conventional stereo camera, foreground and background classification techniques are presented in various ways. Among them, the "stile world" scheme proposed by Daimler is widely used.

The technique adopts a method of classifying a region similar to a depth value (distance value) in a free space boundary region into a foreground region after confirming a free space region.

More precisely, a membership value is assigned to a similar region classification, a method of accumulating the corresponding value in the vertical direction in the free space boundary region is adopted, and the cumulative value first maximum value is set as the foreground boundary value region Setting.

However, the prior art has a disadvantage in that the calculation amount of the foreground / background region detection increases because the cumulative value of the corresponding region must be calculated in all the vertical directions.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems, and an object of the present invention is to provide a method and apparatus for accelerating a background and an object by using occupancy lattice information according to a probability distribution of a dictionary, And to provide such a device.

The present invention sets a threshold value of an object in a free space boundary area based on a prior probability information value constituting a occupation grid map and calculates a membership value based on an object threshold value in an object height classification step And a method and apparatus for reducing the amount of membership calculation for separating background and object.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, a method for accelerating foreground / background classification in an object detecting method using a stereo camera according to an embodiment of the present invention includes extracting a depth image through a stereo camera, Setting a threshold value of a foreground object existing in the predicted free space boundary area by dividing foreground and background by calculating a membership value after predicting a free space; Determining whether a membership value has reached a threshold value of the foreground object during the membership value calculation; Ending the calculation of the membership value when the calculated membership value reaches the threshold value of the foreground object in the determining step; And separating foreground and background through the calculated membership value.

A foreground / background classifying accelerator in an object detecting method using a stereo camera according to an embodiment of the present invention includes a pair of stereo cameras for capturing an image; A captured grid image forming unit for extracting a depth image from an image photographed by the stereo camera and forming a dominant grid map based on the depth image; A free space processor for predicting a free space in the formed occupancy grid map; A threshold value setting unit for setting a threshold value of the foreground object for termination of the calculation of the membership value; A membership calculation unit for calculating a membership value of the image; A membership determination unit for determining whether a membership value has reached a threshold value of the foreground object during the membership value calculation; A membership controller for terminating the calculation of the membership value when the calculated membership value reaches a threshold value of the foreground object in the determining step; And a separator for separating the foreground and background from the calculated membership value.

According to the embodiment of the present invention, there is an effect of speeding up the method of separating the background and the object by using the occupancy grid information according to the probability distribution of the dictionary, thereby reducing unnecessary calculation amount and optimizing the foreground / background separation method having a relatively large amount of calculation .

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

FIG. 1 is a functional block diagram for explaining an apparatus for detecting a stick cell object according to the related art; FIG.
BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to an object detection method using a stereo camera, and more particularly,
3 is a flowchart illustrating a method of accelerating a foreground / background classification in a method of detecting an object using a stereo camera according to an embodiment of the present invention.
4 is a flowchart illustrating threshold setting of a foreground object in an embodiment of the present invention;
5 is another flow chart for illustrating threshold setting of foreground objects in an embodiment of the present invention;
6 is a reference diagram for explaining a step-by-step process of an embodiment of the present invention.
FIG. 7 is a reference diagram for explaining a process for setting a threshold value of a foreground object in an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, advantages and features of the present invention and methods of achieving them will be apparent from the following detailed description of embodiments thereof taken in conjunction with the accompanying drawings.

The present invention may, however, be embodied in many different forms and should not be construed as being limited to the exemplary embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, And advantages of the present invention are defined by the description of the claims.

It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. &Quot; comprises "and / or" comprising ", as used herein, unless the recited component, step, operation, and / Or added.

Before describing the embodiments of the present invention, for the purpose of facilitating understanding of those skilled in the art, an object detection method according to a conventional method will be described, and a background / object using a prior probability distribution value of occupancy lattice information according to an embodiment of the present invention The acceleration method of the separation method will be described.

Object detection methods using a stereo camera in a conventional ADAS system for a vehicle have been variously presented. Among them, a stick cell type object detection method proposed by Daimler has been widely used.

Hereinafter, a conventional stick cell type object detection method will be described with reference to FIG.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a simplified flow diagram of a stereo camera based object detection method according to the prior art. As shown in FIG. 1, in the conventional stick cell type object detection method, when a depth image is extracted using a stereo camera, the occupancy grid forming unit 10 calculates a probability based occupancy grid map .

Here, the occupancy lattice is obtained by dividing the earth surface on the world coordinate system into cells of the grid type and calculating the lattice likelihood value (Lij) in each cell. Thus, the occupancy grid is defined as the occupancy likelihood, which corresponds to the measured value expressed by the depth image.

This can be expressed by the following equation (1).

Where j is the depth configuration, Lij is the gap likelihood value, mk is the measured value, u and v are the image coordinates, d is the disparity Value, and T means a value for changing the position of a row and a column in a matrix.

The occupancy grid may be represented by a Cartesian Occupancy Grid, a Column / Disparity Occupancy Grid, and a Polar Occupancy Grid. Among them, Polar occupancy grid, which can be expressed as [Equation 2], is most utilized in the vehicle object extraction method. As a result, all the columns can be searched simply as shown in FIG. 6 (a).

Where Gmk is a multi-variate Gaussian function, u and v are image coordinates, d is a disparity value, and T is a pixel value in a matrix. And a value for changing the position of the column.

The multivariable Gaussian (Gm) can be expressed as a probability value as shown in Equation (3), so that a lossless multivariate Gaussian can be universally used.

는 실제 측정된 공분산 행열을 의미며,

는 에러를 포함하는 확율 값이다. Here, Gmk denotes a multi-variate Gaussian function,

Is the actual measured covariance matrix,

Is a probability value that includes an error.

Thereafter, the free space processing unit 20 performs a free space prediction process (S12). That is, the free space in the ADAS (Advanced Driver Assistance System) field means an area on the world coordinate system in which the vehicle can move without collision, and the conventional method is based on the occupancy grid map defined above, Predict free space as shown.

The method of predicting the free space is described in more detail. The method is a method of extracting the cells initially occupied in the front direction of the vehicle, and the free space area is extracted using dynamic programming methodically. The dynamic programming applied to the present invention is known from US-A-08548229 (Method for detecting objects), and thus a detailed description thereof will be omitted.

Next, the membership cost calculation unit 30 performs height classification of the object adjacent to the predicted free space boundary area as shown in FIG. 6 (c). Here, height class means to find upper boundary of object based on free space boundary area to classify foreground and background.

In the conventional height classification method, the upper boundary value of the object is extracted based on the free area boundary region in the corresponding depth image according to the free region prediction result. In detail, voting is performed on an object similar to the depth information in the object boundary region based on the free region boundary region.

In other words, it is set to a positive value (+) if it does not deviate from a certain maximum distance from an expected value (critical value) corresponding to an object, and a membership value expressed as a negative value in the opposite case. Equation 4].

Here, M (u, f) is a membership cumulative value of the membership from the free space region (v) to the boundary region (vb) of the object.

Hereinafter, the object calculates the membership value accumulated from the free space boundary area calculated above as an object value, which can be expressed by Equation (5) below. In this way, a free space area can be obtained as shown in Fig. 6 (d).

Where M (u, b) is the accumulated value of membership from the free space boundary region vt to the boundary region vb of the object, and M (u, b) It is a cumulative membership total value from the boundary area (0) to the end (v-1) of the object.

Then, the foreground / background separator 40 classifies the boundary value for the height of the object by using dynamic programming again based on the converted cost value.

Finally, the Stixel generation unit 500 expresses the object by defining the "stixel " by expressing the classified object as a pixel of a uniform shape of a stack shape applied in Daimler.

Since the object extraction method of the conventional Daimler method can accurately represent the object extraction result and can express the extraction result in a regular form, there is an advantage that it is widely used in the ADAS forward surveillance field using the stereo camera, but the dynamic programming and the membership value calculation And so on.

According to an embodiment of the present invention, a method for efficiently calculating a membership value by using prior probability information is proposed to improve the performance of the conventional scheme.

In the step of generating the conventional occupancy grid map, the values assigned to the occupancy grid are assigned cumulative values for likelihood values to be included in the corresponding grid after the pixels in the depth image are converted into world coordinates.

This is because, since the pixel values mapped to similar world coordinates in the depth image are accumulated within the same occupancy lattice range, the occupancy lattice values provide clues about the height information for each object.

Accordingly, in the present invention, a method of setting a threshold value of an object in a free space boundary area based on a prior probability information value constituting a occupancy grid map, and setting a threshold value-based membership value in a height classification step of an object do.

2 is a functional block diagram for explaining a foreground / background classifying accelerator in an object detecting method using a stereo camera according to an embodiment of the present invention. 2, the foreground / background classifying accelerator in the object detecting method using a stereo camera according to an embodiment of the present invention includes a stereo camera 100, a occupancy grid map forming unit 200, A membership determination unit 500, a membership determination unit 600, a membership control unit 700, and a separation unit 800.

The pair of stereo cameras 100 take a picture of the same point through a pair of cameras and acquire the corresponding image information, as shown in Fig. 7 (a). It is preferable that such a pair of stereo cameras 100 are mounted on both sides of the front of the vehicle.

The occupancy grid map forming unit 200 extracts a depth image from an image photographed through the stereo camera 100 and forms a occupancy grid map based on the depth image. At this time, the occupancy grid forming unit 200 may represent the objects as pointers p in the occupancy grid map, as shown in FIG. 7B.

The free space processing unit 300 predicts the free space in the formed occupancy grid map. Here, the method of predicting the free space in the occupied grid map by the free space processing unit 300 is a well known technique, and a detailed description thereof will be omitted.

The threshold value setting unit 400 sets the threshold value of the foreground object so that the membership value after the lightning object is not calculated in order to classify the foreground (o) and the background (b). As described above, the threshold value of the foreground object is set through the threshold value setting unit 400 employed in the embodiment of the present invention, so that the throughput of the membership value to be calculated by the membership calculation unit 500 is reduced, It has the advantage of accelerating the classification of the foreground and background in the images.

Also, the membership calculation unit 500 calculates a membership value of the image.

The membership determination unit 600 determines whether the membership value reaches the threshold value of the foreground object through the membership calculation unit 500 during the membership value calculation.

The membership controller 700 terminates the calculation of the membership value when the membership determining unit 600 determines that the membership value being calculated reaches the threshold value of the foreground object.

According to an embodiment of the present invention, membership values after a threshold of a foreground object are not calculated through a membership calculator, a membership determination unit, and a membership controller as shown in Equation (6) below, The classification can be accelerated.

M (u, b) is the cumulative membership value from the free space region (v) to the boundary region (vb) of the object, and M Is the membership cumulative value from the threshold value Th (u, dh) to the ending point (v-1)

The separator 800 separates the foreground and background from the calculated membership value. That is, the separator 800 performs the same function as the conventional foreground / background separator 40. The separator 800 classifies boundary values for the height of the object by using dynamic programing based on the converted membership cost, The object classified through the unit 500 is represented by pixels of an equal interval stack shape applied in Daimler, and the object is represented by defining "stixel ".

Therefore, according to the embodiment of the present invention, by accelerating the method of separating the background and the object by using the occupancy grid information according to the probability distribution of the dictionary, it is possible to optimize the foreground / background separation method which reduces the unnecessary calculation amount and has a relatively large amount of calculation It is effective.

Meanwhile, the threshold value setting unit 400 employed in the embodiment of the present invention may set the threshold value of the foreground object based on the prior-art information value. In this way, the threshold value setting unit 400 sets the threshold value of the foreground object based on the prior-art information value, thereby reducing errors when it is determined whether the membership value reaches the threshold value of the foreground object.

The threshold value setting unit 400 employed in the embodiment of the present invention sets the number of object pointers p existing in the grid of the generated probability based occupancy grid map through the occupancy grid forming unit 200 Calculates the height of the object from the number of object pointers p, and sets the height of the calculated object to the threshold value of the foreground object.

As described above, according to the threshold value setting unit 400 employed in the embodiment of the present invention, the height of the object can be easily detected by detecting the number of object pointers existing for each grid of the probability based occupancy grid map , And the threshold value of the foreground object can be set through this.

The threshold value setting unit 400 may further include an additional correction value in a threshold of the foreground object to be set, as in Equation (7).

값으로 정규화하는 과정이며, Tho는 여분의 추가 보정값이다. 하지만, 본 발명에 따른 문턱치 설정은 아래 식에 국한하지는 않는다.Here, Th (u, db) is a threshold value, and the value in the boundary region is represented by

, And Tho is an extra correction value. However, the threshold setting according to the present invention is not limited to the following equation.

As described above, according to the threshold value setting unit 400 employed in the embodiment of the present invention, since an additional correction value is further included in the threshold value of the set foreground object, it is possible to reduce errors in foreground object and background separation .

Meanwhile, the threshold value setting unit 400 adopted in the embodiment of the present invention performs a profiling on foreground object values by voting objects having similar depth information when calculating foreground object membership values from the free space boundary region And detects a maximum value from the profiled foreground object values and determines the threshold value as a foreground object value.

3 is a flowchart illustrating a foreground / background classification acceleration method according to an embodiment of the present invention. As shown in FIG. 3, the foreground / background classification acceleration method according to an embodiment of the present invention is preferably performed by a single processor.

First, a threshold value of a foreground object existing in the predicted free space boundary area is set (S100). In one embodiment of the present invention, it is desirable to set the threshold value of the foreground object based on the prior probability information value.

Then, it is determined whether the membership value reaches the threshold value of the foreground object during the membership value calculation (S200).

If the membership value being calculated reaches the threshold value of the foreground object (YES), the calculation of the membership value is terminated (S300).

Then, foreground and background are separated through the calculated membership value (S400).

As described above, according to the embodiment of the present invention, by setting the threshold value of the foreground object in the calculation of the foreground object membership value, the calculation of the membership value distinguishable from the foreground after the threshold is not performed, There is an advantage.

The step of setting the threshold value in the free space boundary value area adopted in the embodiment of the present invention (S100) is preferably set through the above-mentioned equation (7).

Hereinafter, in operation S200 of determining whether the membership value reaches the threshold value of the foreground object during the membership value calculation, the conventional membership value is calculated and accumulated in the vertical direction , It is determined whether or not the set threshold value has been reached, and the amount of membership calculation is reduced by stopping accumulation in the vertical direction. That is, Equation (6) shows an embodiment of transforming a threshold-based membership value.

As described above, according to the embodiment of the present invention, in calculating the cost map value for the membership, the cost map value for membership is not calculated for M (u, b) after the threshold value of the object , It is effective to reduce unnecessary calculation amount and to optimize the foreground / background separation method having a relatively large amount of calculation.

 4 is a flowchart illustrating a threshold setting of a foreground object in an embodiment of the present invention. As shown in FIG. 4, the threshold value (threshold value) of the foreground object employed in an embodiment of the present invention may include a number of object pointers for each grid of the generated probability based occupancy grid map (S110).

Next, the height of the object is calculated from the number of object pointers (S120), and the height of the calculated object is set as a threshold value of the foreground object (S130).

As described above, according to the embodiment of the present invention, the height of the object can be easily detected by detecting the number of object pointers existing for each grid of the probability-based occupancy grid map, and the threshold value of the foreground object can be set There is an advantage.

In step S200, it is determined whether a membership value has reached a threshold value of the foreground object in the membership value calculation. In operation 200, when the membership value is calculated from the free space boundary area, And performs profiling on foreground object values (S101).

Then, the maximum value is detected from the profiled foreground object value and determined as a threshold value of the foreground object (S220).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood that various modifications and applications not illustrated in the drawings are possible. For example, each component specifically shown in the embodiments of the present invention can be modified and implemented.

It is to be understood that all changes and modifications that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims (12)

A method for extracting a depth image through a stereo camera, forming a occupancy grid map based on a depth image, predicting a free space and calculating a membership value to separate foreground and background,
Setting a threshold value of a foreground object existing in the predicted free space boundary area;
Determining whether a membership value has reached a threshold value of the foreground object during the membership value calculation;
Ending the calculation of the membership value when the calculated membership value reaches the threshold value of the foreground object in the determining step; And
And separating the foreground and the background through the calculated membership value. The method of claim 1,
The method according to claim 1,
The step of setting a threshold value (threshold value) of the foreground object includes:
A method for accelerating foreground / background classification in an object detection method using a stereo camera, wherein a threshold value of a foreground object is set based on a priori probability information value.
3. The method of claim 2,
The step of setting a threshold value (threshold value) of the foreground object includes:
Detecting a number of object pointers existing for each grid of the generated probability based occupancy grid map;
Calculating a height of the object from the number of object pointers; And
And setting a height of the calculated object to a threshold value of the foreground object.
The method of claim 3,
The step of setting a threshold value (threshold value) of the foreground object includes:
Wherein the threshold value of the foreground object to be set is further provided with an additional correction value.
3. The method of claim 2,
Performing profiling on a foreground object value by voting on an object having similar depth information when calculating a membership value for the object from the free space boundary region; And
Detecting a maximum value in the profiled foreground object value and determining the maximum value as a threshold value of the foreground object.
6. The method of claim 5,
Determining whether there is an interval in which the profiling foreground object value re-increases within a predetermined interval; And
And setting a detected foreground object value as a maximum value and setting the detected foreground object value as a threshold value of the foreground object if there is no interval in which the foreground object value is increased again in the determining step How to accelerate foreground / background classification.
A pair of stereo cameras for capturing images;
A captured grid image forming unit for extracting a depth image from an image photographed by the stereo camera and forming a dominant grid map based on the depth image;
A free space processing unit for predicting a free space in the formed occupancy grid map;
A threshold value setting unit for setting a threshold value of the foreground object for termination of the calculation of the membership value;
A membership calculation unit for calculating a membership value of the image;
A membership determination unit for determining whether a membership value has reached a threshold value of the foreground object during the membership value calculation;
A membership controller for terminating the calculation of the membership value when the calculated membership value reaches a threshold value of the foreground object in the determining step; And
And a separator for separating the foreground and background from the calculated membership value.
8. The method of claim 7,
The threshold value setting unit may set,
A foreground / background classification accelerator in an object detection method using a stereo camera, wherein a threshold value of a foreground object is set based on a priori probability information value.
9. The method of claim 8,
The threshold value setting unit may set,
Detecting a number of object pointers existing for each grid of the generated probability based occupancy grid map, calculating a height of the object from the number of object pointers, and setting a height of the calculated object as a threshold value of the foreground object Accelerating device for foreground / background classification in object detection method using stereo camera.
10. The method of claim 9,
The threshold value setting unit may set,
Wherein the threshold value of the foreground object to be set is further provided with an additional correction value.
8. The method of claim 7,
The threshold value setting unit may set,
In calculating the membership value of the foreground object from the free space boundary region, the foreground object value is subjected to voting by voting on an object having similar depth information, and a maximum value is detected from the profiled foreground object value, Wherein the object is detected by a stereo camera.
12. The method of claim 11,
The threshold value setting unit may set,
Judges whether there is a section in which the profiling foreground object value re-increases within a preset section,
And setting a detected foreground object value as a maximum value and setting the detected foreground object value as a threshold value of the foreground object when there is no interval in which the foreground object value is increased again as a result of the determination Foreground / background acceleration accelerator.

Images