KR100631082B1 - Method for acquiring three-dimensional image information using single camera and video traffic monitoring method using the same - Google Patents

Abstract

Provided are a method for acquiring three-dimensional information from a two-dimensional image obtained using only one camera, and a method for acquiring traffic information such as traffic volume using the method. In the present invention, a moving object is extracted from an image of consecutive frames photographed by one camera, and the distance the moving object is moved between the images of two consecutive frames is calculated. In addition, assuming that the first image is taken by the first camera in two consecutive frames of images, the second image is spaced apart from the first camera in a direction opposite to the moving direction of the moving object by the distance corresponding to the moving distance of the moving object. Assuming that the image is taken at the same time as the first image by the virtual second camera, three-dimensional information of the moving object is extracted using these two images. The three-dimensional information is used to correct the traveled distance of the moving body calculated above, and the moving speed of the moving body can be calculated from the corrected distance to calculate more accurate traffic volume and traveling speed.

Image detection, traffic volume, speed, vehicle, moving object, camera, 3d, height

Description

Method for acquiring three-dimensional image information using single camera and video traffic monitoring method using the same}

FIG. 1A is a view showing a camera installed in a general image detection system, FIG. 1B is an example of an image captured by an image detection system, and FIG. 1C is a view for explaining a problem of speed measurement using a 2D image.

2 is a block diagram showing the configuration of a general image detection system.

3 is a flowchart showing an image detection method according to the method of the present invention.

4 is an example of a captured image.

5 is a diagram illustrating feature points extracted from the image of FIG. 4.

6 is a view for explaining the principle of the three-dimensional image information acquisition method of the present invention.

The present invention relates to a three-dimensional image information acquisition method using a single camera and an image detection method using the same, in particular, a method for acquiring three-dimensional information from a two-dimensional image obtained using only one camera and using this method It relates to a method for obtaining traffic information such as traffic volume.

In order to determine the amount of traffic on the road, as shown in FIG. 1A, a video camera 10 is installed at a high position on the road side, and an image detection device for capturing an image of the road as shown in FIG. 1B and measuring the speed of each vehicle is known.

2 shows an example of the configuration of such an image detecting apparatus. In general, the image detecting apparatus converts an image obtained by using the camera 10 installed at a high position on the roadside into digital data using the A / D converter 20, and then stores the image in the memory 40. By using a digital signal processor (Digital Signal Processor, DSP) (30) to perform the signal processing on the image to extract the desired information. Meanwhile, the extracted information is transmitted to a traffic control center using a communication means such as a modem 50.

Examples of such image detection apparatus include Korean Patent Publication Nos. 1998-079232, 1999-0047365, 2002-078653, and the like.

Korean Patent Laid-Open Publication No. 1998-079232 and 1999-0047365 both detect a vehicle using a two-dimensional image obtained by photographing a road image with a single camera. However, when measuring the speed of the vehicle based on a two-dimensional image taken using one camera, the height information of the vehicle may not be known, and thus an error may occur in the speed measurement. For example, when measuring the speed based on the floor of the vehicle and when measuring the speed based on the upper part of the vehicle, an error in the speed measurement occurs because a difference occurs in the angle of the camera according to the movement of the vehicle. .

This will be described with reference to FIG. 1C. When the vehicle moves as shown in FIG. 1C, the camera angle between the upper portion and the bottom portion of the vehicle corresponds to θ1 at the position before the movement of the vehicle, but the camera angle between the upper portion and the bottom portion of the vehicle is θ2 at the position after the movement of the vehicle. Will correspond to On the other hand, the movement angle on the camera angle according to the movement of the vehicle when the upper portion of the vehicle is referred to corresponds to α, and becomes β when the reference is to the bottom portion of the vehicle. By the way, as can be seen from the figure, since θ1> θ2, α> β is obtained.

In terms of the distance on the captured image, assuming that the vehicle moved the distance of 100 pixels at the bottom during time t, the upper portion of the vehicle would move by less than, for example, 90 pixels. Will be.

On the other hand, in order to measure the actual moving distance from the moving distance on the captured image, it is common to use environmental information that converts how much the pixel distance in the portion on the screen is the actual distance based on the road surface. That is, since the environment information is generated based on the road surface, an error is inevitably generated when the moving distance is measured by catching the upper part or the middle part of the vehicle as a feature rather than the bottom part of the vehicle.

In view of this, a method of using two cameras to obtain 3D information including height information of a vehicle can be considered. However, in the case of using two cameras, there is a problem that the installation cost, the cost of the device is increased, and setting of environmental information according to the angle difference between the two cameras is very troublesome.

In order to solve this problem, Korean Patent Laid-Open Publication No. 2002-78653 proposes a method of converting a two-dimensional image obtained by using one camera to three-dimensional. In this method, a reference object knowing information about the size is installed at a specific position on the road, and information for three-dimensional modeling is extracted from the shape of the reference object photographed in the captured image. However, in order to use this method, since a reference object must be installed at a specific position of the road, there is a problem that the installation cost is high and it is difficult to apply on the road where it is difficult to install the reference object.

SUMMARY OF THE INVENTION The present invention has been made in view of this point, and an object thereof is to provide a method for acquiring three-dimensional information from a two-dimensional image obtained using only one camera.

Another object of the present invention is to provide an image detection method capable of measuring the traffic volume in real time using the three-dimensional information obtained by the above method.

The present invention for achieving this object, the first step of extracting the moving object from the image of the continuous frame photographed by one camera, and the second step of calculating the distance traveled by the moving object between the image of two consecutive frames And assuming that the first image is taken by the first camera in two consecutive frames of images, the second image is spaced apart from the first camera in a direction opposite to the moving direction of the moving object by the distance corresponding to the moving distance of the moving object. A third step of assuming that the image is taken simultaneously with the first image in the virtual second camera, and using the image photographed by the first camera and the image photographed by the virtual second camera, And a fourth step of extraction.

The first step may be implemented by extracting a background image from an image of a continuous frame, and extracting a moving object by comparing the image of the continuous frame with the background image.

The second step includes extracting the feature points of the extracted moving object, and for each moving object, the moving object having information about the characteristic point in the image of a specific frame of the moving object and information on the characteristic point that matches a predetermined criterion or more is successively performed. And searching for an image of a next frame, and calculating a moving distance from the position of the moving object in the image of the specific frame and the position of the searched moving object in the image of the subsequent next frame.

Also, the step, and obtaining a difference image by subtracting the image of the previous frame from the image of the current frame, one among the difference in binary image comprises: obtaining an image S _N and the image S _N and the image S _N-1 And performing an AND operation and extracting an outline from the image of the multiplication operation.

In this case, the second step may include extracting feature points for the moving object from which the outline is extracted, and for each moving object, to a feature point that matches at least a predetermined criterion with information about the characteristic point in the image of a specific frame of the moving object. Retrieving a moving object having information on the next frame of successive images, and calculating a moving distance from the position of the moving object in the image of the specific frame and the position of the retrieved moving object in the image of the subsequent frame; It may include.

As a method of obtaining a feature point, the outline of the moving object is approximated by a straight line, and the straight line having a predetermined length or more is extracted from these straight lines to determine the point where three or more straight lines meet and the point where two straight lines meet within a predetermined angle. Method can be used. The information about the feature point includes the location of the feature point and a connection relationship between the feature points.

After extracting three-dimensional information from successive two-dimensional images taken by a camera as described above, the three-dimensional information is used to correct the moving distance of the moving object calculated above, and the moving speed of the moving object is calculated from the corrected distance. By doing so, it is possible to calculate more accurate traffic volume and speed.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. 3 is a flowchart showing an image detection method according to the method of the present invention.

First, an image on a road is photographed using a camera (step S310). The image is input by being composed of consecutive frames. An image of an input continuous frame is stored in a memory and used in the following procedure. In addition, the input image may be compressed and transmitted to a traffic control center in real time through a modem.

The moving object is extracted from the image frame stored in the memory (S320). As a method of separating a moving object from an image, a method of comparing a photographed image with a background image is generally used.

In the present invention, such a conventional moving object separation method can be used. However, according to the conventional method, when the vehicle is congested or the movement of the camera occurs, separation of the vehicle may fail due to an incorrect background.

In the present invention, the moving object is preferably extracted by the following method. That is, S _N is an image obtained by binarizing a difference image obtained by subtracting the input image I _N-1 of the previous frame from I _N from the image sequentially coming from the camera. At this time, it is also possible to take only the value of each pixel in the difference image more than a predetermined value. That is, a pixel in which a difference value between each pixel in the previous image and the current image does not exceed a predetermined threshold is regarded as no difference between the two images.

Conventionally, although the area of the moving object is obtained through the S _N image, in this case, the movement of the moving object can be known, but the exact outline of the object cannot be obtained. According to the present invention obtained from the previous input S _N-1 (that is, a binarizing a difference image obtained by subtracting the I _N-2 at I _N-1 image), and the logical product between the S _N (logical AND) by the operation I _N-1 The moving object is extracted by obtaining the outline of the moving object in the image. If the obtained image is O _N-1 , these relations can be expressed by the following equation.

* TH: Threshold

In addition, it is also possible to take only those that are close to the shape of the object obtained and determined to be a vehicle. In addition, by filling the convex hull of the moving object determined to be a vehicle, it is possible to obtain the area of the complete moving object.

On the other hand, the background image may be taken in advance when there is no vehicle, or may be obtained by taking an average of a plurality of consecutive frames, for example, 100 frames, and may use various conventional background image extraction methods. The present invention is not limited to a specific background image extraction method.

The feature points of the objects present in the region are extracted from the extracted region of the moving object and the same objects are matched between the images of consecutive frames (step S330).

Conventionally, the feature points are obtained for the entire image, but when the feature points are obtained for the entire image, it is difficult to implement the real time due to the increase in the amount of computation. Therefore, in the present invention, the feature of the fixed background is eliminated by obtaining the feature point only for the moving object obtained in step S320, and only the feature of the moving vehicle can be obtained, thereby greatly reducing the amount of computation.

As a method of extracting feature points, various conventional methods may be used, or the following method may be used. First, the edge is obtained only for the moving object and approximated by a straight line. Among these straight lines, only the straight lines having a predetermined length or more are selected, and the point where three or more line segments meet and the two line segments meet at an angle within a certain range are defined as feature points.

In the present invention, the same moving object is matched in images of consecutive frames using information about the feature point including the position of the feature point and the connection relationship between the feature points. That is, the moving object having the best matching feature point is determined to be the same moving object by comparing the information about the feature point of the moving object found in the image of the previous frame with the information about the feature point of the moving objects found in the image of the next frame. In this case, since the moving object may disappear on the screen according to the movement of the moving object, it is determined that the object disappears from the image of the next frame if the moving object having a matching degree of information about a feature point does not exist in the image of the next frame. . In addition, since a part of the moving body may be covered by another moving object, rather than comparing the entire feature points of each moving object, the comparison of the characteristic points of each part part is ignored so that only a specific part is not matched. desirable. In addition, when the moving direction of the moving object is known, it is also possible to compare only the object in the moving direction of the moving object. FIG. 4 is an example of a captured image, and FIG. 5 shows an example in which feature points extracted from the image of FIG. 4 are displayed.

In this way, after it is determined in which position the specific moving object is located in the image of each frame, three-dimensional information of the moving object is extracted from the image in each frame image of the moving object (step S340).

The principle of this method is explained with reference to FIG. 6 is a view for explaining the principle of the three-dimensional image information acquisition method of the present invention.

If t _N moment the point on the actual material coordinates and moves between the next instant t _{N + 1} Δ P _N, this time in an actual camera C2 point to t _N moment P _N, t _{N + 1} is projected onto P _{N + 1} do. On the other hand, the camera C2 is virtually - Δ P _N which project the same object to t _N moment in the virtual camera C1 to move by P _'N image, and the P _{N + 1} projected on t _{N + 1} time the camera C2 same. Therefore, the moving object can obtain the effect of the image acquired at the same time through the real camera C2 and the virtual camera C1. In addition, since the object rotation of the short time can be ignored due to the movement characteristics of the vehicle, it can be approximated that the coordinate system of the camera has only moved without rotation. Since a method of acquiring three-dimensional information from an image photographed using two cameras is widely known, the conventional method is applied to an image at two viewpoints photographed by one camera to obtain three-dimensional information of a moving object. You can get it.

That is, assuming that the first image is taken by the first camera in two consecutive frames of images, the second image is spaced apart from the first camera in a direction opposite to the moving direction of the moving object by the distance corresponding to the moving distance of the moving object. Assuming that the image is taken at the same time as the first image from the virtual second camera, the 3D information of the moving object may be extracted using the image photographed by the first camera and the image photographed by the virtual second camera. .

On the other hand, when extracting the three-dimensional information of the moving object it is possible to infer the three-dimensional structure of the object from the structural features between the feature points of the moving object. In addition, it is important to accurately determine the vehicle's motion vector Δ P _N for reliable results. Since the motion vector of the floor shadow of the vehicle is the same, it is preferable to find the motion vector of the feature point that is found to be a shadow. When there is no shadow, it can be estimated by the movement of a bonnet, a bumper, a license plate, and the like.

On the other hand, using the three-dimensional information, such as the height of the vehicle extracted in this way it is possible to correct the moved distance of the previously calculated moving object. That is, as described with reference to FIG. 1C, a difference occurs in the moving distance between pixels of the moving body according to the height of the vehicle. This difference can be corrected by knowing the height of the vehicle. Therefore, the moving distance of the moving object can be corrected using the calculated three-dimensional information. In this way, by calculating the moving speed of the moving object from the corrected distance, it is possible to calculate a more accurate traffic volume and running speed.

As mentioned above, although this invention was demonstrated using the Example, these Example is an illustration and is not limited. Those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit of the invention.

According to the present invention, since three-dimensional information can be obtained from a two-dimensional image obtained using only one camera, accurate traffic volume information can be obtained at a relatively low cost.

In addition, since only the feature points of the moving object are used to determine the moving object, the computation time is greatly reduced, thereby obtaining accurate traffic information using 3D information in real time.

Claims (14)

A first step of extracting a moving object from images of consecutive frames photographed by one camera, Calculating a distance traveled by the moving object between two consecutive frames of image; Assuming that the first image is taken by the first camera in two consecutive frames of images, the second image is spaced apart from the first camera in the opposite direction to the moving direction of the moving object by the distance corresponding to the moving distance of the moving object. A third step of assuming that the image photographed simultaneously with the first image in the second camera; A fourth step of extracting three-dimensional information of the moving object by using the image photographed by the first camera and the image photographed by the virtual second camera 3D image information acquisition method using a single camera having a. The method of claim 1, wherein the first step is Extracting a background image from an image of a continuous frame; Extracting a moving object by comparing the image of the continuous frame and the background image 3D image information acquisition method using a single camera, characterized in that it comprises a. The method of claim 1, wherein the second step is Extracting feature points of the extracted moving object; Retrieving, in each moving object, a moving object having information about a feature point in an image of a specific frame of the moving object and information on a feature point that matches a predetermined criterion or more in an image of a subsequent frame; Calculating a moving distance from the position of the moving object in the image of the specific frame and the position of the retrieved moving object in the image of the next subsequent frame. 3D image information acquisition method using a single camera, characterized in that it comprises a. The method of claim 1, wherein the first step is Obtaining a difference image obtained by subtracting an image of a previous frame from an image of a current frame; Binarizing the difference image to obtain an image S _N ; Performing a logical AND operation between the image S _N and the image S _N-1 ; Extracting an outline from the multiplicative image 3D image information acquisition method using a single camera, characterized in that it comprises a. The method of claim 4, wherein the second step is Extracting feature points of the moving object from which the outline is extracted; Retrieving, in each moving object, a moving object having information about a feature point in an image of a specific frame of the moving object and information on a feature point that matches a predetermined criterion or more in an image of a subsequent frame; Calculating a moving distance from the position of the moving object in the image of the specific frame and the position of the retrieved moving object in the image of the next subsequent frame. 3D image information acquisition method using a single camera, characterized in that it comprises a. The method of claim 5, wherein the feature point Approximating the outline of the moving body with a straight line; Extracting straight lines having a predetermined length or more from the straight lines; Determining a point where at least three straight lines meet and two straight lines meet within a predetermined angle among the extracted straight lines 3D image information acquisition method using a single camera, characterized in that the extraction through. The method of claim 6, The information regarding the feature point is a method of obtaining 3D image information using a single camera, characterized in that the position of the feature point and the connection between the feature points. A first step of shooting an image of a continuous frame with one camera; Extracting a moving object from the image of the continuous frame; A third step of calculating a distance traveled by the moving object between two consecutive frames of image; Assuming that the first image is taken by the first camera in two consecutive frames of images, the second image is spaced apart from the first camera in the opposite direction to the moving direction of the moving object by the distance corresponding to the moving distance of the moving object. A fourth step in which the second camera assumes an image captured simultaneously with the first image, A fifth step of extracting three-dimensional information of the moving object by using the image photographed by the first camera and the image photographed by the virtual second camera; A sixth step of correcting the moved distance of the moving object calculated in the third step by using three-dimensional information of the moving object extracted in the fifth step; A seventh step of calculating the moving speed of the moving object from the corrected distance; Image detection method comprising a. The method of claim 8, wherein the second step is Extracting a background image from an image of a continuous frame; Extracting a moving object by comparing the image of the continuous frame and the background image Image detection method comprising a. The method of claim 8, wherein the third step is Extracting feature points of the extracted moving object; For each moving object, retrieving a moving object having information about a feature point in an image of a specific frame of the moving object and information on a feature point that matches a predetermined criterion or more in an image of a subsequent frame; Calculating a moving distance from the position of the moving object in the image of the specific frame and the position of the retrieved moving object in the image of the next subsequent frame. Image detection method comprising a. The method of claim 8, wherein the second step is Obtaining a difference image obtained by subtracting an image of a previous frame from an image of a current frame; Binarizing the difference image to obtain an image S _N ; Performing a logical AND operation between the image S _N and the image S _N-1 ; Extracting an outline from the multiplicative image Image detection method comprising a. The method of claim 11, wherein the third step is Extracting feature points of the moving object from which the outline is extracted; Retrieving, in each moving object, a moving object having information about a feature point in an image of a specific frame of the moving object and information on a feature point that matches a predetermined criterion or more in an image of a subsequent frame; Calculating a moving distance from the position of the moving object in the image of the specific frame and the position of the retrieved moving object in the image of the next subsequent frame. Image detection method comprising a. The method of claim 12, wherein the feature point Approximating the outline of the moving body with a straight line; Extracting straight lines having a predetermined length or more from the straight lines; Determining a point where at least three straight lines meet and two straight lines meet within a predetermined angle among the extracted straight lines Image detection method characterized in that the extraction through. The method of claim 13, And the information about the feature point includes a location of the feature point and a connection relationship between the feature points.

Images