KR101527886B1 - Method for detecting puddle on the road based on stereo image - Google Patents

Method for detecting puddle on the road based on stereo image Download PDF

Info

Publication number
KR101527886B1
KR101527886B1 KR1020140128666A KR20140128666A KR101527886B1 KR 101527886 B1 KR101527886 B1 KR 101527886B1 KR 1020140128666 A KR1020140128666 A KR 1020140128666A KR 20140128666 A KR20140128666 A KR 20140128666A KR 101527886 B1 KR101527886 B1 KR 101527886B1
Authority
KR
South Korea
Prior art keywords
distance
road
pixel
water
model
Prior art date
Application number
KR1020140128666A
Other languages
Korean (ko)
Inventor
김은태
김지수
백정현
최혁두
Original Assignee
연세대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 연세대학교 산학협력단 filed Critical 연세대학교 산학협력단
Priority to KR1020140128666A priority Critical patent/KR101527886B1/en
Application granted granted Critical
Publication of KR101527886B1 publication Critical patent/KR101527886B1/en

Links

Images

Landscapes

  • Traffic Control Systems (AREA)

Abstract

The present invention relates to an apparatus and method for automatically grasping a state of a road.
The present invention relates to a stereoscopic camera that photographs a stereo road image using a stereo camera, generates a parallax map from the photographed stereo road image, obtains a distance map using the parallax map, An apparatus for automatically detecting a water sludge as a risk element on the road more cost effectively by detecting a water sludge portion on the road by using the estimated trend model and the calculated threshold value after calculating a threshold value, Provide the method.

Figure R1020140128666

Description

Technical Field [0001] The present invention relates to a device for detecting a water puddle using a stereo image,

The present invention relates to an apparatus and method for automatically grasping a state of a road.

In the past, the driver manually controlled the direction and speed of the car, but in recent years, the development of automobile technology has accelerated the development of automobiles without the help of the driver or with minimal driver involvement. In addition, various technologies have been developed to provide safe driving performance by automatically controlling the vehicle in accordance with a given environment and situation without using a driver's control by using various sensing technologies.

Especially, for the safe operation and the automated operation of the automobile, it is necessary to grasp the state of the road on which the automobile travels. For example, if there is an obstacle on the road, or if there is a water puddle or a ice sheet on the road, the automobile needs to run safely in response to such an unexpected road situation.

Conventional sensing technologies for detecting the state of the road include sensing technology using a near-infrared camera and sensing technology using a radar.

First, the road condition detection technology using a near-infrared camera captures the road using a near infrared ray camera, identifies the temperature on the photographed road image, and effectively grasps the position of the ice sheet or accumulated snow on the road. However, the above-mentioned method of detecting the temperature by sensing the road condition does not detect the water puddle on the road well, and the price of the near infrared camera is high.

In addition, a conventional technology for detecting a road condition using a radar sensor discloses a method of detecting a state of a road by comparing a ratio of back scattering values by sensing a difference in permittivity according to a road state. However, There is a problem in that it must be used.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a stereoscopic image display apparatus which photographs a stereo road image using a stereo camera, generates a parallax map from the taken stereo road image, obtains a distance map using the parallax map, And the threshold value is calculated. Then, by detecting the part of the water puddle on the road using the estimated trend model and the calculated threshold value, the water puddle, which is a dangerous element on the road, can be automatically And a method thereof.

According to an aspect of the present invention, there is provided an apparatus for detecting a water puddle using a stereo image, comprising: a stereo road image pickup unit that receives stereo road images photographed using at least two cameras, A distance map obtaining unit for calculating distance information between a subject photographed by the camera and the camera for each pixel and obtaining a distance map in which the signal value in each pixel coordinate is the calculated distance information; And a water shedding detector that receives the distance map and detects a water soul located on the road in the stereo road image using the coordinates of each pixel of the inputted distance map and the distance information.

Here, the road puddle detecting device using the stereo image may include a first camera and a second camera that are spaced apart from each other by a predetermined distance, and photographs the road using the first camera and the second camera And a stereo camera unit for acquiring a first road image and a second road image for the road, respectively, wherein the stereo road image includes the first road image and the second road image .

Here, the distance map obtaining unit may include: a parallax map generating unit that receives the stereo road image from the stereo camera unit and generates a parallax map using the stereo road image; And a distance information calculation unit that receives the parallax map from the parallax map generation unit and calculates the distance information using the parallax value of each pixel of the parallax map.

Here, the parallax map generation unit may compare the first road image and the second road image of the stereo road image, and generate a parallax map corresponding to each pixel of the first road image and the second road image, And calculating the parallax representing a coordinate difference between corresponding pixels representing the same subject to generate the parallax map.

According to an aspect of the present invention, there is provided an apparatus for detecting a water puddle using a stereo image, the apparatus comprising: means for detecting the water puddle detected by the water puddle detecting unit based on any one of the stereo road images; And a water droplet image acquiring unit for acquiring a water droplet image in which coordinates corresponding to coordinates of the water droplet are displayed to have a constant image signal value.

Here, when the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map, for each pixel of the distance map, And determining a pixel of the distance map as the part of the water pool to detect the water pool located on the road.

Here, the water shedding detecting unit may set a trending model indicating the tendency of the distance information according to the coordinates of the pixels of the distance map, and may calculate the distance between the coordinates of sample pixels of the predetermined number of distance maps, Calculating a coefficient of the trend model, and detecting the water sump located on the road using the trend model calculated by the coefficient.

Here, the trend model may be set such that the coordinates of the pixels of the distance map and the distance information are used as input variables.

Here, the trend model may be set as shown in Equation (1).

Equation 1.

Figure 112014091556803-pat00001

(here

Figure 112014091556803-pat00002
( X, y ) is the coordinate of the pixel of the distance map, d is the distance information of the pixel of the distance map,
Figure 112014091556803-pat00003
Is a coefficient of the tendency model, and S = [ x, y, d ] is a vector representing the coordinates of the pixel of the distance map together with the distance information.

Here, the water shedding detection unit may include a trend model obtaining unit that sets the trend model, calculates coefficients of the trend model using the coordinates of the sample pixels of the distance map and the distance information, and obtains the trend model; A threshold value calculation unit for calculating a constant threshold value; And applying the pixels of the distance map to the trend model obtained by the trend model obtaining unit to calculate a result value, and when the calculated result value is larger than the threshold value, And a water pool determination unit for determining the water pool determination unit.

Here, the trend model acquisition unit may include a sample pixel acquiring unit for arbitrarily selecting a predetermined number of pixels among the pixels of the distance map and selecting the selected pixels as the sample pixels; A tendency model coefficient estimating unit that estimates a coefficient of the tendency model using the sample pixels selected by the sample pixel acquiring unit; And a tendency model evaluation unit for evaluating the coefficients of the tendency model estimated by the tendency model coefficient estimating unit and determining coefficients of the tendency model.

According to another aspect of the present invention, there is provided an apparatus for detecting a water puddle using a stereo image, the apparatus comprising: means for detecting distance information on each pixel of a stereo road image photographed using at least two cameras; And a water shedding detector for detecting water shedding on the road in the stereo road image using the coordinates of each pixel of the inputted distance map and the distance information.

Here, when the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map, for each pixel of the distance map, And determining a pixel of the distance map as the part of the water pool to detect the water pool located on the road.

Here, the water shedding detecting unit may set a trending model indicating the tendency of the distance information according to the coordinates of the pixels of the distance map, and may calculate the distance between the coordinates of sample pixels of the predetermined number of distance maps, Calculating a coefficient of the trend model, and detecting the water sump located on the road using the trend model calculated by the coefficient.

Here, the trend model may be set such that the coordinates of the pixels of the distance map and the distance information are used as input variables.

Here, the water shedding detection unit may include a trend model obtaining unit that sets the trend model, calculates coefficients of the trend model using the coordinates of the sample pixels of the distance map and the distance information, and obtains the trend model; A threshold value calculation unit for calculating a constant threshold value; And applying the pixels of the distance map to the trend model obtained by the trend model obtaining unit to calculate a result value, and when the calculated result value is larger than the threshold value, And a water pool determination unit for determining the water pool determination unit.

According to one aspect of the present invention, there is provided a method of detecting a road puddle using a stereo image, the method comprising the steps of: capturing the road using at least two cameras and generating a stereo image An image acquisition step; A distance map obtaining step of calculating distance information between a subject photographed by the camera and the camera for each pixel of the stereo road image and obtaining a distance map in which a signal value in each pixel coordinate is the calculated distance information, ; And a water puddle detecting step of receiving the distance map and detecting a water sump located on the road in the stereo road image using the pixel coordinates and the distance information of the inputted distance map.

Here, if the distance information value of the pixel of the distance map is greater than or equal to the distance information value of the surrounding pixels of the pixel of the distance map, for each pixel of the distance map, And determining a pixel of the distance map as the part of the water pool to detect the water pool located on the road.

The water puddle detecting step may set a trend model indicating the tendency of the distance information according to the coordinates of the pixels of the distance map and may calculate the water puddle using coordinates of sample pixels of the predetermined number of the distance maps, Wherein the trend model detects the water puddle located on the road by using the trend model calculated by the coefficient, and the trend model calculates coordinates of the pixel of the distance map and the distance information as input variables Is set to the following expression.

Here, the water shedding detection step may include a step of setting the trend model, calculating a coefficient of the trend model using the coordinates of the sample pixels of the distance map and the distance information, and obtaining the trend model ; A threshold value calculation step of calculating a constant threshold value; And applying each pixel of the distance map to the trend model obtained in the trend model obtaining step to calculate a result value, and when the calculated result value is larger than the threshold value, And a water-droplet determination step of determining the water-droplet as a part.

According to the present invention, there is provided an apparatus and method for detecting a water puddle using a stereo image, comprising: capturing a stereo road image using a stereo camera; acquiring a parallax map and a distance map from the photographed stereo road image; It is possible to cost-effectively and more robustly detect a water sump as a risk element on the road.

1 is a block diagram of an apparatus for detecting a water puddle using a stereo image according to an embodiment of the present invention.
2 is a reference diagram for explaining the operation principle of a road puddle detecting apparatus using a stereo image according to the present invention.
3 is a reference view showing a stereo puddle image and a distance map obtained by a road puddle detecting apparatus using a stereo image according to the present invention.
FIG. 4 is a reference diagram for explaining a trend model of a road puddle detecting apparatus using a stereo image according to the present invention.
5 is a detailed block diagram of a water puddle detecting unit of a road puddle detecting apparatus using a stereo image according to the present invention.
6 is a detailed block diagram of a trend model acquisition unit of a road puddle detection apparatus using a stereo image according to the present invention.
FIG. 7 is a reference view showing a water sump image detected by a road water sump detection apparatus using a stereo image according to the present invention.
FIG. 8 is another reference view showing a water sump image detected by a road water sump detection apparatus using a stereo image according to the present invention.
FIG. 9 is a flowchart of a method of detecting a water puddle using a stereo image according to another embodiment of the present invention.
10 is a detailed flowchart of a water puddle detection step in a road puddle detection method using a stereo image according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

1 is a block diagram of an apparatus for detecting a water puddle using a stereo image according to an embodiment of the present invention.

The apparatus for detecting a water puddle using a stereo image includes a stereo camera unit 100, a distance map obtaining unit 200, a water shedding detecting unit 300, and a water shedding image obtaining unit 400 can do. The above embodiment is an optimal embodiment of the present invention, and another embodiment of the present invention includes a water pool detecting unit 300, and the remaining camera 100, the distance map obtaining unit 200, The unit 400 may be included or omitted as needed. For example, a road puddle detecting apparatus using stereo images according to the present invention may include a distance map obtaining unit 200 and a water shedding detecting unit 300. In another example, the apparatus for detecting a road puddle using a stereo image according to the present invention may include a stereo camera unit 100, a distance map obtaining unit 200, and a water shedding detector 300. Hereinafter, an optimal embodiment including the stereo camera unit 100, the distance map acquisition unit 200, the water shedding detection unit 300, and the water shedding image acquisition unit 400 will be described in detail below .

The stereo camera unit 100 acquires a stereo road image for the road. A stereo image means an image of the same subject photographed using two different cameras. Here, the road is referred to as a stereo road image because the road is a target of the photographing.

Here, the stereo camera unit 100 includes a first camera 110 and a second camera 120 that are spaced apart from each other by a predetermined distance, and the first camera 110 and the second camera 120 are used And the first road image and the second road image for the road are obtained, respectively. Here, the first road image and the second road image thus obtained may be the stereo road image.

The distance map acquiring unit 200 receives the stereo road image, calculates distance information between the camera and a subject photographed by the camera for each pixel of the stereo road image, And obtains a distance map that is the calculated distance information.

Here, the distance map means a depth map in the stereo image field. That is, the distance information is a value indicating the distance between the subject and the camera, and is calculated for each pixel of the stereo road image. The distance map represents the value of the distance information in two dimensions. That is, the signal value of each pixel of the distance map is the value of the distance information in each pixel of the stereo road image.

The water shedding detection unit 300 receives the distance map, and detects a water sump located on the road in the stereo road image using the pixel-by-pixel coordinates of the input distance map and the distance information.

The water-droplet image acquiring unit 400 calculates the coordinates of the water droplet detected by the water-droplet detecting unit 300 on the basis of any one of the stereo road images, and sets the pixel corresponding to the coordinates of the water droplet as a constant And acquires a water pool image displayed so as to have a video signal value.

Here, the water shed image acquiring unit 400 may be omitted as needed.

Hereinafter, the distance map obtaining unit 200 will be described in more detail.

The distance map obtaining unit 200 may include a parallax map generating unit 210 and a distance information calculating unit 220 as shown in FIG.

The parallax map generating unit 210 receives the stereo road image from the stereo camera unit 100 and generates a parallax map using the stereo road image. Here, the disparity map means a disparity map in a stereo image field.

Here, the parallax map generation unit 210 compares the first road image and the second road image of the stereo road image, and calculates the same subject for each pixel of the first road image and the second road image And a parallax representing a coordinate difference between corresponding pixels representing the same subject can be calculated. The parallax map generating unit 210 may generate the parallax map in which the calculated parallax is the signal value of each pixel.

The distance information calculator 220 receives the parallax map from the parallax map generator 210 and calculates the distance information using the parallax values of the pixels of the parallax map.

In a parallax map obtained from a stereo image captured by a stereo camera, the distance between the camera and the subject is closer to the camera, and the distance from the camera to the subject is smaller when the parallax value is smaller. The distance information calculating unit 220 preferably calculates the distance information from the parallax using the characteristics of the stereo camera.

Hereinafter, the water shedding detection unit 300 will be described in more detail.

When the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map, for each pixel of the distance map, It is possible to determine the pixels of the distance map as the part of the water shed and detect the water shed located on the road.

Hereinafter, the principle of operation of the water sump detector 300 will be described with reference to FIGS. 2 and 3. FIG.

2 is a reference diagram for explaining the operation principle of a road puddle detecting apparatus using a stereo image according to the present invention.

Referring to FIG. 2, the water sump on the road reflects light incident on the water sump. Therefore, rather than representing the shape of the road on which the actual water sump is located, the shape of the remote object corresponding to the light incident on the water sump . Therefore, the distance information value of the part where the water puddle part is taken from the stereo image is larger than the road part around the water puddle. That is, referring to FIG. 2, since the water shed (a100) reflects the shape of the remote building a200, when the distance map is calculated for the stereo image including the water shed (a100) The distance d2 corresponding to the distance information becomes larger than the value of d1 or d3 corresponding to the distance information of the peripheral portion of the water sump a100.

FIG. 3 is a reference view showing a stereoscopic water puddle image and a distance map obtained by the road puddle detecting apparatus using the stereo image according to the present invention.

Referring to FIG. 3, as described above with reference to FIG. 2, a portion where a water sump is formed in (a) of FIG. 3 corresponds to a distance information value As shown in Fig.

Therefore, the water puddle detecting unit 300, as described above, determines whether or not the water puddle corresponds to each of the pixels of the distance map by using the above-described characteristics of the stereo road image, When the distance information value of the pixel of the map is larger than the distance information value of the surrounding pixels by a predetermined amount or more, the pixel of the distance map to be judged can be determined as the water pool portion.

The operation of the water puddle detecting unit 300 as described above can be implemented more precisely using the trend model as described below.

That is, the water shedding detection unit 300 sets a trending model indicating the tendency of the distance information according to the coordinates of the pixels of the distance map, and calculates a trend model using the coordinates of sample pixels of the predetermined number of the distance maps and the distance information A coefficient of the trend model may be calculated and a water sump located on the road may be detected using the trend model in which the coefficient is calculated.

4 is a reference diagram for explaining the trend model.

Referring to FIG. 4, as described above with reference to FIG. 3, the portion corresponding to the water shed in the distance map has a larger value of the distance information when compared with the portion not surrounding the water shed. In the stereo image obtained when the road is photographed from the viewpoint of the stereo camera attached to the object running on the road, the upper part of the image represents a distant image and the lower part of the image represents a near image as shown in FIG. 4 (a) . Therefore, the upper part of the image has a larger distance information value and the lower part of the image has a smaller distance information value. The tendency of the distance information on the above distance map is shown in Fig. 4 (b) when the left upper end point of the distance map as shown in Fig. 4 (a) is taken as the origin. In the graph of FIG. 4 (b), the x coordinate represents the Y coordinate of the distance map, the y coordinate represents the distance information, and the curved portion b100 of the graph represents each pixel of the distance map of FIG. Is displayed on the graph in accordance with the Y coordinate value of the distance map and corresponding distance information, and the straight line portion b200 in the graph is a straight line indicating the tendency of the curve portion. Referring to FIG. 4B, it can be confirmed that the distance information shows a constant tendency according to the coordinates of the pixels of the distance map in the distance map, and it is confirmed that the above-mentioned tendency is in contradiction in the water pool portion .

The trend model is for modeling the above-mentioned tendency of the distance map.

The water shedding detection unit 300 determines whether each of the pixels of the distance map corresponds to a water pool, and when the corresponding pixel is substituted into the trending model, the pixel corresponding to the trending model A pixel not corresponding to the water pool, and a pixel violated the trend model can be determined as a pixel corresponding to the water pool.

Here, the trend model may be set by using coordinates of pixels of the distance map and the distance information as input variables.

Preferably, the trend model can be set as shown in Equation (1).

Figure 112014091556803-pat00004

(here

Figure 112014091556803-pat00005
( X, y ) is the coordinate of the pixel of the distance map, d is the distance information of the pixel of the distance map,
Figure 112014091556803-pat00006
Is a coefficient of the tendency model, and S = [ x, y, d ] is a vector representing the coordinates of the pixel of the distance map together with the distance information.

Here, the water shedding detection unit 300 can arbitrarily select a predetermined number of pixels from among the pixels of the distance map, select the sample pixels, and calculate the coefficient of the trending model using the sample pixels.

Then, the water shedding detection unit 300 can detect the water shedding by using the above-mentioned trend model in which the coefficients are calculated. Here, the water shedding detecting unit 300 applies the respective pixels of the distance map obtained from the stereo road image to the trend model to calculate a result value, and when the calculated result value is larger than a predetermined threshold value, It is preferable to determine the pixel of the water pool as the water sump portion.

The operation of the water shedding detector 300 as described above will be described in more detail with reference to FIGS. 5 and 6 below.

5 is a detailed block diagram of the water shedding detection unit 300. As shown in FIG.

The water shedding detection unit 300 may include a trending model obtaining unit 310, a threshold calculating unit 320, and a water shedding determining unit 330.

The trend model obtaining unit 310 may set the trend model and calculate the coefficient of the trend model using the coordinates of the sample pixels of the distance map and the distance information to obtain the trend model.

Here, the trend model obtaining unit 310 may calculate the coefficients of the trend model so that the trend model can optimally represent the distribution of the sample pixels. The operation of the trend model obtaining unit 310 will be described in more detail below with reference to FIG.

6 is a detailed block diagram of the trend model obtaining unit 310. As shown in FIG.

The trend model obtaining unit 310 may include a sample pixel obtaining unit 311, a trend model coefficient estimating unit 312, and a trend model evaluating unit 313. [

Here, the tendency model obtaining unit 310 may operate the sample pixel obtaining unit 311, the tendency model coefficient estimating unit 312, and the tendency model evaluating unit 313 in order, and repeatedly operate the tendency model estimating unit 313 a predetermined number of times or more have. The tendency model acquiring unit 310 repeats the operation of consecutively operating the sample pixel acquiring unit 311, the tendency model coefficient estimating unit 312 and the tendency model evaluating unit 313 by the predetermined number of times in this way, Model coefficients can be obtained.

Here, the number of times that the tendency model obtaining unit 310 repeats the operations of the sample pixel obtaining unit 311, the tendency model coefficient estimating unit 312, and the tendency model estimating unit 313 can be determined as shown in the following equation (2).

Figure 112014091556803-pat00007

(Where N is the number of times that the tendency model obtaining unit 310 repeats the operation, m is the number of the selected sample pixels, and p is the time when the tendency model obtaining unit 310 repeats the Nth operation and selecting the sample from the sample pixel pixel acquisition unit 311 at least once to a probability of selecting the m number of sample pixels in the non-puddle area,

Figure 112014091556803-pat00008
Is the ratio of the area of the street map that is not a water pool.)

The sample pixel obtaining unit 311 can arbitrarily select a predetermined number of pixels among the pixels of the distance map and select the pixel as the sample pixel.

The trend model coefficient estimator 312 can estimate the coefficients of the trend model using the sample pixels selected by the sample pixel obtaining unit 311. [

Here, the tendency model coefficient estimator 312 calculates a coefficient of the tendency model so that the sum of squares of the resultant values when the coordinates of the pixels of the sample pixels and the distance information are substituted into the tendency model of Equation (1) .

Here, the tendency model coefficient estimator 312 can estimate the tendency model coefficient as shown in the following Equation (3) for the tendency model as shown in Equation (1).

Figure 112014091556803-pat00009

(here

Figure 112014091556803-pat00010
Is a vector representing the coordinates of the sample pixel together with the distance information, m is the number of the sample pixels,
Figure 112014091556803-pat00011
Is the trend model, and A is a vector representing the coefficients of the trend model.

The tendency model evaluating unit 313 can evaluate the coefficients of the tendency model estimated by the tendency model coefficient estimating unit 312 to determine the coefficients of the tendency model.

Here, the trend model evaluation unit 313 preferably leaves the trend model coefficient estimated by the trend model coefficient estimator 312 when the performance is superior to the existing trend model coefficient, and discards it if it is not excellent. Here, the performance of the estimated trend model coefficient can be measured by how much the trend model using the estimated coefficient models the coordinates of the pixels of the distance map and the distance information. For example, when a predetermined number of pixels are arbitrarily selected from the distance map and the selected pixels are assigned to a trend model expressed by Equation 1 to which the estimated trend model coefficient is applied, The smaller the sum is, the better the performance of the estimated tendency model coefficient is. Or the tendency model evaluation unit 313 assigns all the pixels of the distance map to the tendency model to which the estimated tendency model coefficient is applied. When the sum of the squares of the results is smaller, the performance of the estimated tendency model coefficient May be considered superior.

Here, the operation of the tendency model evaluation unit 313 may be expressed by the following equation (4).

Figure 112014091556803-pat00012

(Where PF Is an index indicating the performance of the estimated tendency model coefficient,

Figure 112014091556803-pat00013
( X , y ) of the pixel selected in the distance map and the distance information d , m is the number of the sample pixels,
Figure 112014091556803-pat00014
Is the trend model, and A ' Is a vector representing the coefficients of the estimated trend model.)

Or the trend model evaluation unit 313 may determine the performance of the estimated trend model coefficient according to how much data the estimated trend model includes among the pixels of the distance map. In this case, as the number of pixels of the distance map corresponding to a certain distance in the tendency model according to the estimated coefficient increases, the estimated tendency model coefficient shows better performance.

The threshold value calculating unit 320 may calculate a threshold value of a predetermined size. The threshold value calculating unit 320 may set the threshold value to an arbitrary value as required. Or threshold value calculation unit 320 applies the threshold value to all the pixels of the distance map to the trend model obtained by the trend model obtaining unit 310 to calculate each result value, .

Here, the operation of the threshold value calculation unit 320 may be expressed as Equation (5).

Figure 112014091556803-pat00015

(Where T is the threshold,

Figure 112014091556803-pat00016
( X , y ) of the pixel of the distance map and the distance information d , and M Is the total number of pixels included in the distance map,
Figure 112014091556803-pat00017
Is the trend model obtained by the trend model obtaining unit 310,
Figure 112014091556803-pat00018
Is a vector representing the coefficients of the trend model obtained by the trend model obtaining unit 310.)

The water sill determination unit 330 applies each pixel of the distance map to the trend model obtained by the trend model obtaining unit 310 to calculate a result value, and when the calculated result value is larger than the threshold value, The pixel of the distance map may be determined as the water sump portion.

Here, the water shedding determination unit 330 may determine the water shedding portion as shown in Equation (6).

Figure 112014091556803-pat00019

(Where S is a vector representing the coordinates of the pixel of the distance map and the distance information together,

Figure 112014091556803-pat00020
Is the trend model, T is the threshold value, and W ( S ) is a function for determining whether the pixel of the distance map corresponds to the water sump.

The water shedding image acquiring unit 400 sets the image signal value of the portion where W (S) is determined to be a constant value and the image signal value of the portion where W (S) is determined to be 0, It is possible to generate a water droplet image having a constant value and a different value. For example, in the images of FIGS. 8B and 8B, a portion where W (S) is determined as 1 is a white signal value, and a portion where W (S) is determined as 0 is a black signal value A water-droplet image is generated.

FIG. 7 and FIG. 8 are further reference views showing a water sump image detected by a road water sump detection apparatus using a stereo image according to the present invention.

7 (a) and 8 (a) respectively show one of the stereo road images, and FIGS. 7 (b) and 8 (b) It is a water puddle image in which a water puddle portion is detected using a road water puddle detecting device and the detected water puddle portion is expressed by an image. As described above, it can be confirmed that the device according to the present invention is robustly detecting the water sump portion.

The apparatus for detecting a water puddle using a stereo image according to another embodiment of the present invention may include a water sump detector 300. The water shedding detecting unit 300 in this embodiment can operate in the same manner as the water shedding detecting unit 300 in the embodiment described above in detail. Therefore, the overlapping parts are omitted and briefly explained below.

The water shedding detection unit 300 receives a distance map indicating distance information of each pixel of a stereo road image photographed by using at least two cameras on the road, The controller can detect the water sump located on the road in the stereo road image.

Here, if the distance information value of the pixel of the distance map is greater than or equal to the distance information value of the surrounding pixels of the pixel of the distance map, for each pixel of the distance map, , A pixel of the distance map is determined to be the part of the water puddle, and the water puddle located on the road can be detected.

Here, the water shedding detecting unit 300 sets a trending model indicating the tendency of the distance information according to the coordinates of the pixels of the distance map, and uses the coordinates of the sample pixels of the predetermined number of arbitrary distance maps and the distance information And calculating a coefficient of the trend model, and using the calculated trend model, the water sump located on the road can be detected.

Here, it is preferable that the trend model is set by an expression using the coordinates of the pixels of the distance map and the distance information as input variables.

Here, the water shedding detection unit 300 may include a current state model acquisition unit 310, a threshold value calculation unit 320, and a water sump determination unit 330.

The trend model obtaining unit 310 may set the trend model and calculate the coefficient of the trend model using the coordinates of the sample pixels of the distance map and the distance information to obtain the trend model.

The threshold value calculation unit 320 can calculate a certain threshold value.

The water sill determination unit 330 applies each pixel of the distance map to the trend model obtained by the trend model obtaining unit 310 to calculate a result value, and when the calculated result value is larger than the threshold value, The pixel of the distance map may be determined as the water sump portion.

FIG. 9 is a flowchart of a method of detecting a water puddle using a stereo image according to another embodiment of the present invention.

The method of detecting a road puddle using a stereo image according to another embodiment of the present invention may include a stereo image acquisition step (S100), a distance map acquisition step (S200), and a water shedding detection step (S300). The road puddle detection method using the stereo image can operate in the same manner as the method of operating the road puddle detection apparatus using the stereo image described above in detail. Therefore, the overlapping parts are omitted and briefly explained below.

In the stereo image acquisition step S100, a road is photographed using at least two cameras, and a stereo road image for the road is obtained.

The distance map obtaining step S200 calculates the distance information between the camera and the subject photographed by the camera for each pixel of the stereo road image, and calculates a distance Obtain a map.

The water puddle detection step S300 receives the distance map and detects the water sump located on the road in the stereo road image using the coordinates of the pixel of the inputted distance map and the distance information.

Here, the water droplet detection step (S300) may be such that, for each pixel of the distance map, the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map A pixel of the distance map may be determined as the part of the water puddle, and the water puddle located on the road may be detected.

Here, the water shedding detection step S300 may set a trending model indicating the tendency of the distance information according to the coordinates of the pixels of the distance map, and may set coordinates of sample pixels of a predetermined number of randomly selected distance maps, And the water puddle located on the road can be detected using the trend model in which the coefficient is calculated.

Here, it is preferable that the trend model is set by an expression using coordinates of pixels of the distance map and the distance information as input variables.

10 is a detailed flowchart of the water sludge detection step 300. As shown in FIG.

The waterdrop detection step S300 may include a trend model acquisition step S310, a threshold calculation step 320, and a waterdrop determination step 330. [

The trend model acquisition step S310 may set the trend model, calculate the coefficients of the tendency model using coordinates of the sample pixels of the distance map and the distance information, and obtain the tendency model.

The threshold calculation step S320 can calculate a certain threshold value.

The water pool determination step S330 may include applying each pixel of the distance map to the trend model obtained in the trend model acquisition step S310 to calculate a result value, and when the calculated result value is larger than the threshold value, The pixel of the distance map may be determined as the water sump portion.

It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them.

In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like can be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (20)

An apparatus for determining the state of a road,
A stereo road image obtained by photographing the road using at least two or more cameras, calculating distance information between the camera and a subject photographed by the camera for each pixel of the stereo road image, A distance map acquiring unit for acquiring a distance map in which the signal value of the distance information is the calculated distance information; And
Setting a trend model indicating the tendency of the distance information according to the coordinates of pixels of the distance map using coordinates of pixels of the inputted distance map and the distance information, And a water sump detecting unit for detecting a water sump located on the road in the stereo road image using the stereo image.
The method according to claim 1,
And a first camera and a second camera located at a predetermined distance from each other, the first camera capturing the road using the first camera and the second camera, And a stereo camera unit,
Wherein the stereo road image includes the first road image and the second road image.
3. The apparatus according to claim 2,
A parallax map generation unit that receives the stereo road image from the stereo camera unit and generates a parallax map using the stereo road image; And
And a distance information calculation unit that receives the parallax map from the parallax map generation unit and calculates the distance information using the parallax value of each pixel of the parallax map, .
The method of claim 3,
The parallax map generation unit compares the first road image and the second road image of the stereo road image to generate a corresponding pixel representing the same object for the pixels of the first road image and the second road image And calculates the parallax representing a coordinate difference between corresponding pixels representing the same subject to generate the parallax map.
The method according to claim 1,
A controller for calculating the coordinates of the water shed detected by the water shedding detecting unit based on any one of the stereo road images and for displaying the pixels corresponding to the coordinates of the water shed to have a constant image signal value, And a water droplet image acquiring unit for acquiring the water droplet image acquired by the image acquiring unit.
The method according to claim 1,
When the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map by a predetermined amount or more for each pixel of the distance map, And the water puddle located on the road is detected by determining the pixel of the water puddle as the water sump portion.
7. The method according to any one of claims 1 to 6,
Wherein the water shedding detection unit calculates the coefficients of the trending model using the coordinates of the sample pixels of the predetermined number of the distance maps and the distance information which are arbitrarily selected and calculates the coefficients of the trending model using the calculated trending models, Wherein the water puddle detecting unit detects the water puddle.
8. The method of claim 7,
Wherein the trend model is set such that the coordinates of the pixels of the distance map and the distance information are used as input variables.
9. The method of claim 8,
Wherein the trend model is set as shown in Equation (1).
Equation 1.
Figure 112014091556803-pat00021

(here
Figure 112014091556803-pat00022
( X, y ) is the coordinate of the pixel of the distance map, d is the distance information of the pixel of the distance map,
Figure 112014091556803-pat00023
Is a coefficient of the tendency model, and S = [ x, y, d ] is a vector representing the coordinates of the pixel of the distance map together with the distance information.
9. The water treatment system according to claim 8,
A tendency model acquiring unit that sets the tendency model, calculates coefficients of the tendency model using the coordinates of the sample pixels of the distance map and the distance information, and obtains the tendency model;
A threshold value calculation unit for calculating a constant threshold value; And
Wherein each pixel of the distance map is applied to the trend model obtained by the trend model obtaining unit to calculate a result value, and when the calculated result value is larger than the threshold value, And a water puddle determination unit for determining the water puddle based on the determination result.
11. The apparatus of claim 10,
A sample pixel acquiring unit for arbitrarily selecting a predetermined number of pixels among the pixels of the distance map and selecting the pixel as the sample pixel;
A tendency model coefficient estimating unit that estimates a coefficient of the tendency model using the sample pixels selected by the sample pixel acquiring unit; And
And a trend model evaluation unit for evaluating the coefficients of the tendency model estimated by the tendency model coefficient estimating unit and determining the coefficients of the tendency model.
An apparatus for determining the state of a road,
A distance map indicating distance information of each pixel of the stereo road image photographed by using at least two cameras on the road, receiving coordinates of pixels of the inputted distance map and the distance information, And a puddle detecting unit for setting a trending model indicating the tendency of the distance information according to the coordinates of the pixels of the map and detecting the waterdrops located on the road in the stereo road image using the trending model A road puddle detection device using a stereo image.
13. The method of claim 12,
When the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map by a predetermined amount or more for each pixel of the distance map, And the water puddle located on the road is detected by determining the pixel of the water puddle as the water sump portion.
14. The method according to any one of claims 12 to 13,
Wherein the water shedding detection unit calculates the coefficients of the trending model using the coordinates of the sample pixels of the predetermined number of the distance maps and the distance information which are arbitrarily selected and calculates the coefficients of the trending model using the calculated trending models, Wherein the water puddle detecting unit detects the water puddle.
15. The method of claim 14,
Wherein the trend model is set such that the coordinates of the pixels of the distance map and the distance information are used as input variables.
16. The water treatment system according to claim 15,
A tendency model acquiring unit that sets the tendency model, calculates coefficients of the tendency model using the coordinates of the sample pixels of the distance map and the distance information, and obtains the tendency model;
A threshold value calculation unit for calculating a constant threshold value; And
Wherein each pixel of the distance map is applied to the trend model obtained by the trend model obtaining unit to calculate a result value, and when the calculated result value is larger than the threshold value, And a water puddle determination unit for determining the water puddle based on the determination result.
A method for determining the state of a road,
A stereo image acquiring step of photographing the road using at least two cameras and acquiring a stereo road image for the road;
A distance map obtaining step of calculating distance information between a subject photographed by the camera and the camera for each pixel of the stereo road image and obtaining a distance map in which a signal value in each pixel coordinate is the calculated distance information, ; And
Setting a trend model indicating the tendency of the distance information according to the coordinates of pixels of the distance map using coordinates of pixels of the inputted distance map and the distance information, And a water puddle detecting step of detecting a water puddle located on the road in the stereo road image using the stereo puddle detection method
18. The method of claim 17,
When the distance information value of the pixel of the distance map is larger than the distance information value of the surrounding pixels of the pixel of the distance map by at least a certain amount for each pixel of the distance map, Determining a pixel of the map as the water sump portion, and detecting a water sump located on the road; and detecting the water sump by using the stereo image.
19. The method according to any one of claims 17 to 18,
Wherein the water shedding detecting step calculates the coefficients of the trending model using the coordinates of the sample pixels of the predetermined number of the distance maps and the distance information which are arbitrarily selected and calculates the coefficients of the trending model using the calculated trending models A water sump is detected,
Wherein the trend model is set such that the coordinates of the pixels of the distance map and the distance information are used as input variables.
20. The method according to claim 19,
A tendency model acquisition step of setting the tendency model, calculating coefficients of the tendency model using the coordinates of the sample pixels of the distance map and the distance information, and obtaining the tendency model;
A threshold value calculation step of calculating a constant threshold value; And
Wherein each pixel of the distance map is applied to the trend model obtained in the trend model obtaining step to calculate a result value, and when the calculated result value is larger than the threshold value, And determining a water puddle based on the determination of the water puddle.
KR1020140128666A 2014-09-25 2014-09-25 Method for detecting puddle on the road based on stereo image KR101527886B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020140128666A KR101527886B1 (en) 2014-09-25 2014-09-25 Method for detecting puddle on the road based on stereo image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020140128666A KR101527886B1 (en) 2014-09-25 2014-09-25 Method for detecting puddle on the road based on stereo image

Publications (1)

Publication Number Publication Date
KR101527886B1 true KR101527886B1 (en) 2015-06-10

Family

ID=53505789

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020140128666A KR101527886B1 (en) 2014-09-25 2014-09-25 Method for detecting puddle on the road based on stereo image

Country Status (1)

Country Link
KR (1) KR101527886B1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101030211B1 (en) * 2010-01-07 2011-04-22 쓰리에이치비젼주식회사 System and method for detecting road and object
JP2012123495A (en) * 2010-12-07 2012-06-28 Hitachi Automotive Systems Ltd Traveling environment recognition device
JP2014089547A (en) * 2012-10-30 2014-05-15 Sharp Corp Road surface level difference detection method, road surface level difference detection device, and vehicle equipped with the road surface level difference detection device
JP2014512291A (en) * 2011-02-14 2014-05-22 コンティ テミック マイクロエレクトロニック ゲゼルシャフト ミット ベシュレンクテル ハフツング Friction coefficient estimation by 3D camera

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101030211B1 (en) * 2010-01-07 2011-04-22 쓰리에이치비젼주식회사 System and method for detecting road and object
JP2012123495A (en) * 2010-12-07 2012-06-28 Hitachi Automotive Systems Ltd Traveling environment recognition device
JP2014512291A (en) * 2011-02-14 2014-05-22 コンティ テミック マイクロエレクトロニック ゲゼルシャフト ミット ベシュレンクテル ハフツング Friction coefficient estimation by 3D camera
JP2014089547A (en) * 2012-10-30 2014-05-15 Sharp Corp Road surface level difference detection method, road surface level difference detection device, and vehicle equipped with the road surface level difference detection device

Similar Documents

Publication Publication Date Title
US20200334843A1 (en) Information processing apparatus, control method for same, non-transitory computer-readable storage medium, and vehicle driving support system
KR101411668B1 (en) A calibration apparatus, a distance measurement system, a calibration method, and a computer readable medium recording a calibration program
EP3367084B1 (en) Road surface state determination device and method
EP3205084B1 (en) Image processing method
US10290080B2 (en) Method for displaying a vehicle environment of a vehicle
JP6744336B2 (en) Detection of lens contamination using expected edge trajectories
CN110737266B (en) Automatic driving control method and device, vehicle and storage medium
CN108028023A (en) Information processor, information processing method and program
US20140071240A1 (en) Free space detection system and method for a vehicle using stereo vision
US20170220875A1 (en) System and method for determining a visibility state
EP3082068B1 (en) Traveling road surface detection device and traveling road surface detection method
US10165246B2 (en) Method and device for processing stereoscopic data
CN106851238A (en) Method for controlling white balance, white balance control device and electronic installation
CN103770704A (en) System and method for recognizing parking space line markings for vehicle
CN110341621B (en) Obstacle detection method and device
KR20170064652A (en) Egomotion estimation system and method thereof
JP4344860B2 (en) Road plan area and obstacle detection method using stereo image
KR20160142137A (en) Device for detecting moving object and method thereof
EP2913998A1 (en) Disparity value deriving device, equipment control system, movable apparatus, robot, disparity value deriving method, and computer-readable storage medium
US9898676B2 (en) Method for determining the level of degradation of a road marking
KR20100066952A (en) Apparatus for tracking obstacle using stereo vision and method thereof
JP6982865B2 (en) Moving image distance calculation device and moving image distance calculation program
Elzayat et al. Real-time car detection-based depth estimation using mono camera
JP6398218B2 (en) Self-position calculation device and self-position calculation method
CN105021573A (en) Method and device for tracking-based visibility range estimation

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20180605

Year of fee payment: 4