KR101163042B1 - Apparatus and Method for Extracting Vehicle - Google Patents

Abstract

According to the present invention, a parallax map is generated by extracting distance information using parallax information of an object formed on an image photographing a road using a stereo camera, and a v-displacement map is generated from the parallax map generated by the parallax map generator. Only the road components are extracted from the generated v-displacement map, and the obstacles are detected by comparing each row of the parallax map based on the extracted road components, and the position of each obstacle is detected. Provided is a vehicle detection apparatus and method for detecting a vehicle by calculating one or more than one of the magnitudes of parallax values of obstacles.

Stereo, Vision, Parallax Maps,

Description

Apparatus and Method for Extracting Vehicle}

The present invention relates to a vehicle detection apparatus and method. More particularly, the present invention relates to a vehicle detection apparatus and method for detecting a vehicle using stereo vision.

Obstacle Localization in three-dimensional space is one of the important problems in intelligent vehicle and robot navigation. This is especially important in the implementation of Adaptive Cruise Control (ACC) and Lane Keeping System (LKS), which are one of the essential functions in intelligent vehicles. In order to determine the location of obstacles, various sensors such as cameras, sonar, radar, and ray radar are used. Recently, data processing time has been reduced due to the improvement of computer performance and integration technology, and many researchers are using computer vision to perform obstacle positioning.

Computer vision has some problems with data processing time, but there are many advantages that are similar to human vision, where a lot of information is available from a single sensor. Recently, Stereo Vision (Stereo Vision) has been studied to overcome many problems of Mono Vision.

Obstacles can be detected using stereo vision, such as a V-disparity map, inverse perspective mapping (IPM), and column detection. However, these methods have a problem in that the recognition performance of the obstacle itself is poor and there is almost no ability to detect only a specific obstacle.

Therefore, the specific obstacle is detected through post-processing of the obstacle detection result. Neural network-based recognition technology is most commonly used as a representative post-process. However, it is good in performance, but most of it requires learning the recognition module through prior learning, which requires a lot of effort and time. In addition, the actual recognition requires a lot of time, it is too difficult to apply the actual.

In the existing obstacle detection method, obstacles are first detected using a parallax map. In the detection step, since the recognition performance is poor, various obstacles such as not only a vehicle but a cover, a roadside road, a road surface, etc. are detected. In order to detect only specific obstacles (eg, vehicles), post-processing is performed on the detected results. Usually, a neural network recognition module based on database learning is used.

However, this method not only requires a lot of data when learning, but also takes a lot of time, and it requires a lot of time even in actual execution, which makes it difficult to apply in real time.

SUMMARY OF THE INVENTION In order to solve the above problems, an object of the present invention is to provide an apparatus and method for detecting only a vehicle by using a feature contrasting with surrounding obstacles in detecting a vehicle on a road.

Another object of the present invention is to provide an apparatus and method for detecting a vehicle in real time in consideration of signs, roads, road surfaces, median separators, and the like, and the size and location of the vehicle.

In order to achieve the above object, the present invention provides a parallax map generator for generating a parallax map by extracting distance information using parallax information of an object formed on an image of a road using a stereo camera; A v-displacement map generator for generating a v-displacement map from the disparity map generated by the disparity map generator; A column detector extracting only road components from the generated v-displacement map and comparing obstacles with each row of the parallax map based on the extracted road components; And a post-processor for detecting a vehicle by detecting a position of each obstacle and calculating one or more of a road contact point, a horizontal / vertical ratio, and a size compared to a parallax value of each obstacle.

In another aspect, the present invention, a parallax map generation step of generating a parallax map by extracting the distance information using the parallax information of the object formed on the image of the road using a stereo camera; A v-displacement map generation step of generating a v-displacement map from the disparity map generated in the disparity map generation step; A column detection step of extracting only road components from the generated v-displacement map and comparing obstacles with each row of the parallax map based on the extracted road components; And a post-processing step of detecting a location of each obstacle and detecting a vehicle by calculating one or more of a road contact point, a horizontal / vertical ratio, and a size compared to a parallax value of each obstacle. do.

As described above, according to the present invention, there is an effect that only the vehicle can be detected by using a feature that contrasts with the surrounding obstacle in detecting the vehicle on the road.

In addition, according to the present invention, there is another effect that can detect the vehicle in real time in consideration of the characteristics, size, location, etc. of the sign, roadside, road surface, the central separator and the like on the road.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but between components It will be understood that may be "connected", "coupled" or "connected".

In the following description of an embodiment of the present invention, it is assumed that any object in the image is a road. That is, for example, the computer vision system mounted on a vehicle may photograph a road using two or more cameras and extract feature information of the road to detect an obstacle in an image of the photographed road. Explain. However, these assumptions are merely for convenience of explanation, and the present invention may be any object in the image, as well as roads, objects such as sidewalks, desks, indoors, playgrounds, or various body parts such as faces, cheeks, abdomen, and legs. It may be an object in the form.

1 is a flowchart illustrating a vehicle detection method according to an embodiment of the present invention.

The vehicle detection method includes a parallax map generation step (S10) of generating distance parallax by extracting distance information using parallax information of an object formed on an image of a road by using a stereo camera; A v-displacement map generation step (S20) of generating a v-displacement map from the disparity map generated in the disparity map generation step (S10); A column detection step (S30) of extracting only road components from the generated v-displacement map and comparing obstacles with each row of the parallax map based on the extracted road components (S30); And a post-processing step (S40) of detecting the position of each obstacle and detecting the vehicle by calculating one or more of a road contact point, a horizontal / vertical ratio, and a size compared to a parallax value of each obstacle.

FIG. 2 is a flow chart of a vehicle detection method more embodying the vehicle detection method of FIG. 3 and 4 are stereo vision based left and right images. 5 is an exemplary diagram illustrating a disparity map from left and right images. 6 is an exemplary diagram illustrating a v-displacement map for a case where there are many obstacles.

First, a parallax map is formed by receiving left and right images based on stereo vision (S110). There are various methods of forming a parallax map. A parallax map expresses distance information as a brightness value, and a near point is represented by a bright value and a far point is represented by a dark value.

The stereo vision system obtains parallax information of an object formed on an image of a road as an object by using two left and right cameras (ie, a stereo camera), extracts distance information by using the acquired parallax information, and extracts the extracted distance information. A parallax map can be generated by expressing the distance information as a gray value.

Referring to FIGS. 3 and 4, it can be seen that the left and right images are partially moved horizontally based on the stereo vision. This is to create a parallax map. FIG. 5 shows a result of generating a parallax map from left and right images, and the brightness is different for each distance. It can be seen that the near distance is bright and the far is dark.

Next, a v-displacement map is generated from the generated parallax map (S112).

Referring to FIG. 6, the horizontal axis of the v-displacement map generated from the parallax map represents the parallax value (0 to 255), and the vertical axis represents the same axis as the vertical line of the image. In other words, the horizontal lines corresponding to the vertical lines of the parallax map (disparity values) are accumulated. As a result, as shown in FIG. 6, the planar component appears as a diagonal line on the v-displacement map, and the obstacle appears as a vertical line on the v-displacement map.

Planar components such as roads can be seen that the parallax value increases as the vertical line value increases. This is based on the image input to the camera, the lower portion of the large vertical line value is a distance closer to the distance is higher, the parallax value is higher, the lower portion of the vertical line value is far from the distance is smaller parallax value.

Since the planar component gradually changes in distance, as shown in FIG. 6, most of the planar components appear as diagonal straight lines, and obstacles appear as vertical components because they have a specific peak value.

Next, road feature information is extracted using the above-described properties of the generated v-displacement map (S114).

On the other hand, since detecting an obstacle directly on the v-displacement map generates a large number of obstacles and a lot of errors when the road is bent, the present embodiment does not directly detect an obstacle on the v-displacement map, but on the v-displacement map. Only road components are extracted and a column detection method using this information is performed.

Accordingly, only road components are extracted from the generated v-displacement map, and the obstacles are more robustly detected by comparing the road components with each row of the disparity map (S116).

FIG. 7 is a diagram illustrating a process of detecting an obstacle by extracting only a road component from a generated v-displacement map and comparing it with each row of a parallax map based on this.

Referring to FIG. 7, the middle view of FIG. 7 is the same parallax map as in FIG. 5, and the values are read for each row (blue line) and compared with the road components.

7 is a diagram in which a road component (red dotted line) as a reference and an arbitrary row value (blue line) of the parallax map are expressed together on the v-displacement map. As shown in the left figure of FIG. 7, it can be seen that the components A and B corresponding to the obstacles in one row value of the parallax map are larger than the reference road components (red dotted line). This is used to detect obstacles on the road, which is done in all rows of the parallax map.

Looking at the overall result it can be seen that only the obstacle portion is detected as shown in the right view of FIG. However, this is because there is no recognition function as described above, it can be seen that all obstacles on the road is detected. In this case, the detection of the road surface shows that a part which is not an actual obstacle may be detected due to an error on a parallax map.

Post-processing is performed to detect only the vehicle in the region of interest (ROI) detected through the above process (S118).

First, the joongyoung is removed (S120), and the position of each obstacle is detected (S122).

Thereafter, additional segmentation is performed in each of the separated obstacle areas (S124). This is for more accurate obstacle detection through additional separation and merging within the separated obstacle area.

Next, whether or not the vehicle is determined based on all obstacles finally detected.

First, since the vehicle is in contact with the road surface, it is determined whether the road contact (S126). When the vehicle is in contact with the road, the parallax value at the contact point will be the same. That is, the parallax value of the lower portion of the obstacle on the parallax map will be similar to the parallax value of the surrounding road.

8 is a diagram illustrating a process of checking whether or not an obstacle contacts a road.

Referring to FIG. 8, the middle view of FIG. 8 illustrates obstacles extracted on the parallax map, and the parallax value of each lower portion of each obstacle is read to determine whether the road contact point is performed (A1, B1, C1, and D1).

This is compared with the parallax value of the road around each obstacle. For this, the road component value is used on the v-displacement map. Referring to the left figure of FIG. 8, the parallax value at the bottom of each obstacle, that is, the parallax value of the sign A1, the vehicle B1, the road surface C1, and the roadside number D1, corresponds to the v-displacement corresponding to the vertical line. When the difference is small compared to the road component values A, B, C, and D on the map, the obstacle is in contact with the road. For example, in the case of a sign, the parallax value A1 at the lower part of the sign has a certain value, but since the parallax value A at the vertical position on the v-displacement map is close to 0, the difference abs (A -A1) is a big value, so the sign determines that you are not on the road. The result is shown in the right figure of FIG. Importantly, when reading A1 ~ D1, 0 value is excluded from calculation because it is background, not obstacle.

In addition, the aspect ratio of the separated obstacle is calculated (S128). Typically, the vehicle has a horizontal / vertical value between 1 and 2. People or roadside trees have more values and can be removed.

In addition, the size (number of pixels) of the parallax value of each obstacle is calculated (S130). The larger the parallax value is, the nearer it is, so the size of the obstacle is larger, and the smaller the parallax value is, the farther it is, so the size of the obstacle is smaller. In other words, the presence or absence of a vehicle is determined using the general distance-to-distance of the vehicle.

It is determined whether the vehicle targets all obstacles detected using the three steps S126, S128, and S130 (S132).

Further checks are made for obstacles that are recognized as vehicles using the three steps of S126, S128 and S130. Obstacles such as road surfaces or median separators detected by errors in the parallax maps are also in contact with the roads, and their proportions and sizes may be similar to vehicles.

To remove this, texture and symmetry information are calculated (S134). First, the symmetry shows that the vehicle has a characteristic of being almost symmetrical when viewed from the rear. When the vehicle is viewed as a gray image, there is a significant change in the image value therein. In this case, contrast, entropy, dissimilarity are high, and homogeneity is low.

However, obstacles such as road surfaces or median have little change in image values within the gray image, resulting in low values such as contrast, entropy, dissimilarity, and homogeneity values. Homogeneity is high. In other words, by using the texture information, the vehicle can be detected more accurately. However, if there is a background other than a desired obstacle in the ROI, texture calculation may be an error factor.

Therefore, in this embodiment, reliability is improved by calculating the texture using only obstacles excluding the background.

9 is a diagram illustrating a process of detecting a vehicle through texture / symmetry calculation.

Referring to FIG. 9, (a) of FIG. 9 is an extraction result of an obstacle on a parallax map, and a background is separated (zero value). 9B illustrates a result of detecting an obstacle in a gray image, but the background is not separated. FIG. 9C is a result generated by using FIGS. 9A and 9B, and shows a result of separating a background from a gray image.

By calculating the texture using this, a more accurate value can be obtained, which leads to an improvement in detection performance. Of course, in this process, the value of zero is excluded from the calculation because it is a background, not an obstacle. 9 (d) shows the final vehicle detection result detected through symmetry and texture calculation.

Finally, it is determined whether or not the desired obstacle (S136), only the vehicle is detected (S138).

FIG. 10 shows a result of removing other obstacles as a result of the proposed final vehicle detection and properly detecting only the vehicle.

11 is a block diagram schematically illustrating an obstacle detecting apparatus according to another embodiment of the present invention.

The obstacle detecting apparatus 1000 according to the exemplary embodiment may include a parallax map generator 1010, a v-displacement map generator 1020, a column detector 1030, and a post processor 1040.

The obstacle detecting apparatus 1000 may be implemented as a computer vision system or an image processing apparatus in a computer vision system. Such a computer vision system or image processing apparatus includes an input device such as a keyboard or a mouse that receives data or commands, a monitor for outputting data, a printer, an output device such as a liquid crystal display, and a memory for storing data or a program for processing an image. The apparatus may include a microprocessor for processing an image according to an algorithm of a program stored in a memory by using input data or instructions or data stored in a memory.

The parallax map generator 1010 acquires parallax information of an object formed on an image photographing a road as an object using two left and right cameras (ie, a stereo camera), and extracts distance information using the obtained parallax information. The parallax map may be generated by expressing the extracted distance information as a gray value.

The v-displacement map generator 1020 generates a v-displacement map from the disparity map generated by the disparity map generator 1010. The v-displacement map generator 1020 represents a parallax value (0 to 255) on the horizontal axis and accumulates the horizontal line value (disparity value) corresponding to each vertical line of the parallax map. As a result, as shown in FIG. 6, the planar component appears as a diagonal line on the v-displacement map, and the obstacle appears as a vertical line on the v-displacement map.

In the parallax map shown in FIG. 5, the plane such as a road gradually decreases in distance so that its brightness is constantly reduced. If an obstacle is placed on the plane, unlike the plane, the phenomenon that the brightness value of the obstacle portion increases due to the obstacle. Therefore, the obstacle can be detected using the phenomenon on the parallax map.

The column detector 130 detects obstacles by extracting only road components from the generated v-displacement map and comparing them with each row of the disparity map.

Referring to FIGS. 11 and 7 (a), the column detector 1030 indicates that the components A and B corresponding to the obstacles in one row of the parallax map have a larger value than the reference road component (red dotted line). To detect obstacles on the road. As described above, FIG. 7C illustrates a result of detecting only an obstacle portion by the column extractor 1030.

The post-processor 1040 performs post-processing to detect only the vehicle in the detected region of interest (ROI). The post processor 1040 first removes the ghosting and detects the position of each obstacle. Post-processor 1040 then performs additional segmentation within each of the separated obstacle regions.

Next, the post-processor 1040 determines whether the vehicle targets all the obstacles finally detected. The post-processor 1040 calculates the road contact status, the ratio of the width / length, and the disparity value of each obstacle, and performs an additional checking process on the obstacles recognized as vehicles. 8 shows the result of removing the obstacle through the road contact determination.

The post-processor 1040 uses texture and symmetry information to remove obstacles such as road surfaces or central separators that are detected due to errors in the parallax map. Symmetry uses the symmetry of the vehicle left and right when viewed from the rear, and there is little change in the image value in the gray image of the vehicle, such that the values such as contrast, entropy and dissimilarity are low. Homogeneity and the like can more accurately detect a vehicle using high texture information. In this case, the reliability can be improved by calculating the texture using only obstacles excluding the background. 9 (d) shows the final vehicle detection result detected through symmetry and texture calculation.

FIG. 10 shows a result of removing other obstacles as a result of the proposed final vehicle detection and properly detecting only the vehicle.

In addition, although not shown in FIG. 11, when the vehicle detection apparatus 1000 is not connected to the stereo vision system or a parallax map is not input from the outside, the vehicle detection apparatus 1000 may disparity the object, that is, a camera photographing a road and an object of a photographed image. The apparatus may further include a stereo matcher which extracts distance information from the information to generate a parallax map.

According to the embodiments described above, in detecting a vehicle on the road, only the vehicle is detected by using a feature that contrasts with surrounding obstacles, and features, sizes, locations, etc. of signs, roads, road surfaces, median separators, and the like on the road are detected. In consideration of this, the vehicle can be detected in real time.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, a program in which some or all of each component is selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a module. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

Furthermore, the terms "comprises", "comprising", or "having" described above mean that a component can be implanted unless otherwise specifically stated, But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Terms used generally, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meaning of the related art, and shall not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present invention.

The description above is merely illustrative of the technical idea of the present invention and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

1 is a flowchart illustrating a vehicle detection method according to an embodiment of the present invention.

FIG. 2 is a flowchart of a vehicle detection method in which the vehicle detection method of FIG. 1 is further embodied.

3 and 4 are stereo vision based left and right images.

5 is an exemplary diagram illustrating a parallax map from left and right images.

6 is an exemplary diagram illustrating a v-displacement map for a case where there are many obstacles.

FIG. 7 is a diagram used to detect an obstacle by extracting only road components from the generated v-displacement map and comparing them with each row of the parallax map based on the extracted road components.

FIG. 7 is a diagram illustrating a process of detecting an obstacle by extracting only a road component from a generated v-displacement map and comparing it with each row of a parallax map based on this.

8 is a diagram illustrating a process of checking whether or not an obstacle contacts a road.

9 is a diagram illustrating a process of detecting a vehicle through texture / symmetry calculation.

FIG. 10 shows a result of removing other obstacles as a result of the proposed final vehicle detection and properly detecting only the vehicle.

11 is a block diagram schematically illustrating an obstacle detecting apparatus according to another embodiment of the present invention.

Claims (8)

A parallax map generator for generating a parallax map by extracting distance information by using parallax information of an object formed on an image of a road photographed using a stereo camera; A v-displacement map generator for generating a v-displacement map from the disparity map generated by the disparity map generator; A column detector extracting only road components from the generated v-displacement map and comparing obstacles with each row of the parallax map based on the extracted road components; And And a post-processor for detecting a vehicle by detecting a location of each obstacle and calculating one or more of a road contact point, a horizontal / vertical ratio, and a size relative to a parallax value of each obstacle. The method of claim 1, The post processor detects the vehicle by further calculating one or more of symmetry information or texture information. The method of claim 2, And the texture information is one or more of contrast, entropy, dissimilarity, and homogeneity. The method of claim 3, wherein The post processor calculates a texture using only obstacles excluding a background. Generating a parallax map by extracting distance information by using parallax information of an object formed on an image of a road by using a stereo camera; A v-displacement map generation step of generating a v-displacement map from the disparity map generated in the disparity map generation step; A column detection step of extracting only road components from the generated v-displacement map and comparing obstacles with each row of the parallax map based on the extracted road components; And And a post-processing step of detecting a vehicle by detecting a location of each obstacle and calculating one or more of a road contact point, a horizontal / vertical ratio, and a size relative to a parallax value of each obstacle. 6. The method of claim 5, The post-processing step is a vehicle detection method, characterized in that for detecting the vehicle by further calculating one or more of the symmetry information or texture information. The method of claim 6, And the texture information is one or more of contrast, entropy, dissimilarity, and homogeneity. The method of claim 7, wherein The post-processing step is a vehicle detection method, characterized in that for calculating the texture using only obstacles excluding the background.

Images