JP2001357489A - Traveling vehicle detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a traveling vehicle detector capable of obtaining the highly accurate motion track of an object area by using plural camera video images for which the mapping function of a space position and a pixel position within respective images is known, extracting template position information for indicating a vehicle area and then subjecting the information to a multi-view-point processing and a time sequential processing. SOLUTION: This traveling vehicle detector is constituted of an input means 1, a moving area extraction processing means 2, a labeling processing means 3, a vehicle detection processing means 4, a template matching means 5, a general processing means 6 and a moving object tracing means 7. By using the intersection of a straight line connecting the diagonals of the rectangular area of a template obtained from the template matching means of respective camera video processing systems in a real space in a general processing means, the accurate position of a moving vehicle is extracted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a traveling vehicle detection device for detecting a traveling vehicle from a moving image and accurately detecting and tracking the traveling vehicle in a monitoring device or the like.

[0002]

2. Description of the Related Art In recent years, in wide area monitoring and equipment, etc.,
Surveillance systems using a large number of cameras are increasing. Therefore, development of an automatic monitoring system capable of grasping the situation by capturing the movement or the like of a target object by excluding an inefficient monitoring work in which an observer monitors a plurality of images is desired. In particular, as a method of tracking a moving object in a wide observation area with multiple camera images, the area of the target object in the image is accurately extracted, the foot position of the target object is obtained, and that position is determined in the observation space. Convert to a planar position and track the target object, associate the same object in two images by shooting the same observation area with two cameras, and use the parallax of the two cameras Then, there is a stereo method for performing stereo processing for obtaining a depth distance of the target object.

[0003] Regarding this stereo technique, "3D image measurement" (Yuichi Ota, System / Control / Information, vol. 38, N
o.1, pp. 21 to 27, 1994). In the stereo method, as shown in FIG. 5, on a pair of left and right images taken at different positions, a pair of images Pl and Pr corresponding to the same point Ps in a three-dimensional space is obtained. The position of the point Ps based on the image position can be determined based on the principle of. At this time, the coordinate conversion between the three three-dimensional coordinate systems fixed to the left and right imaging systems needs to be known.

One of the important properties of the stereo method is epipolar constraint that restricts the range of the pair of left and right images on the image plane. This is a property that, for example, an image on the right image that should correspond to the point Pl must exist on one straight line depending on the point Pl on the right image, and this straight line is called an epipolar line. This constraint is apparent from the fact that both the point Pl and the point r must be present on a plane in a three-dimensional space determined by the two lens centers Ol, Or and the point Ps. The epipolar line is actually the intersection of this plane (called an epipolar plane) and the image plane.
From another viewpoint, the points Pl and Pr and the lens center Ol and
The condition that the four points Or are placed on the same plane is a condition for determining the point Ps. The condition that these four points are placed on the same plane is called a coplanar condition. This is the most basic principle of restoring three-dimensional information from.

[0005]

However, the prior art as described above has a large problem. For example, the following four points can be raised as problems in the stereo method. (1) It is not easy to find corresponding points on left and right images. (2) The target object cannot be used in a portion where no feature point exists. (3) The measurement accuracy depends on the direction of the edge on the target object. (4) It cannot be used in hidden parts that are visible on the one hand but not visible on the other.

In view of the above-mentioned problems of the prior art, the present invention proposes a multi-view integration method based on a template position based on an automatically generated template matching method with a small processing load. An object of the present invention is to provide a traveling vehicle detection device that can accurately extract a moving object by introducing processing.

[0007]

An input means for inputting a camera image, a moving area extracting means for detecting a moving area from the video data obtained from the input means, and a moving area extracting means for obtaining the moving image. Labeling processing means for detecting information on the moving area such as the size of the area and the number of pixels for each moving area from the moving area, and using the information on each moving area detected by the labeling processing means, the moving area is regarded as a target. Shape information extracting means for extracting information indicating how similar the object is to the object to be processed; and operating the template based on video data obtained from the input means and information on a moving region obtained from the shape feature extracting means. A camera image processing system having a template processing unit for performing the processing, an integrated processing unit for integrating information from the plurality of camera image processing systems, and a moving object obtained from the integrated processing unit. Processing the location and classification information is composed from the output processing means for displaying on a display or the like.

Thus, the present invention solves the above-described problem by converting a plurality of camera images having a known mapping function between a spatial position and a pixel position in each image in a processing procedure utilizing the characteristics of moving image processing. By extracting rough foot position information of the target area, and performing multi-viewpoint processing and time-series processing on this unstable information, a highly accurate motion trajectory (trace) of the target area can be obtained. A moving object can be extracted.

According to a second aspect of the present invention, in each of the observation areas photographed by a plurality of cameras, each area is photographed by two or more cameras. This is a traveling vehicle detection device, and has the effect that it is possible to more accurately track a vehicle with simple processing.

According to a third aspect of the present invention, when a plurality of cameras are installed, four cameras constitute one set, and when these cameras are used as a camera 1, a camera 2, a camera 3, and a camera 4, the cameras 1 and 2 and cameras 3 and 4 are arranged side by side or inward, and cameras 1 and 2 and cameras 3 and 4 are installed facing each other, so that any observation area can be photographed with two or more cameras. The traveling vehicle detection device according to claim 1 or 2, characterized in that the number of cameras is increased in a long time. It has the effect that it can be performed in the area.

According to a fourth aspect of the present invention, there is provided the traveling vehicle detecting apparatus according to the first to third aspects, wherein the input means is a visible camera, an infrared camera, or a combination thereof. This is a limitation of the input means for achieving the effect of the invention described in any one of 1 to 3, and has an effect that the effect can be obtained by compensating for the advantages and disadvantages of both the visible camera and the infrared camera.

According to a fifth aspect of the present invention, in the integrated processing means, a mapping function of an installation position of each camera in an observation space, a pixel in an image and a plane position of the observation space is known, and a template of each image processing system is provided. The diagonal line of the rectangular area of the template obtained by the matching means is obtained. The traveling vehicle detecting apparatus according to any one of claims 1 to 4, wherein intersections of the straight lines obtained from a plurality of video processing systems are set as position candidates of the target object. There is an effect that the position accuracy of the moving object can be improved by the integration processing means for performing the integration / identification processing of the vehicle detection information.

According to a sixth aspect of the present invention, the correspondence is established in the shortest distance process in the observation space using the position candidate of the target object obtained from the integration processing means and the position information of the vehicle obtained up to that time. The vehicle detection apparatus according to claim 5, wherein the object detection is performed by the target association means according to the following.
This has the effect of reducing erroneous recognition of a moving object on the time axis for the identification processing.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIGS. FIG. 1 shows a block configuration diagram of a traveling vehicle detection device according to an embodiment of the present invention. In FIG. 1, reference numerals 7 to 9 denote camera image processing systems, 1 denotes an input means for inputting a camera image, 2 denotes a moving area extracting means for detecting a moving area from the video data obtained from the input means 1, and 3 denotes a moving area extracting means. Labeling processing means for detecting information on the moving area such as the size of the area and the number of pixels for each moving area from the moving area obtained from the area extracting means 2, and information on each moving area detected by the labeling processing means 3 The vehicle detecting means 5 for determining whether or not the moving area is a vehicle by using, when the vehicle detecting means determines that the moving area is a vehicle area,
6 is a template matching means for generating a template or performing template matching on an existing template, 6 is an integrated processing means for integrating information from a plurality of camera video processing systems 10 to 12, and 7 is an integrated processing means 6
And a moving object tracing means for displaying the position of the traveling vehicle obtained from the moving object on a display or the like.

The large flow from the input means 1 to the template matching means 5, which are the camera image processing systems 10 to 12 of the present invention, is a process in which a template matching method is combined to detect a vehicle and track the vehicle. . The template matching is a process that is stable with respect to a change in the imaging environment and that has a relatively light calculation load. Therefore, in the present invention, when a traveling vehicle is detected, the vehicle area is searched, and if there is no template, a template is generated.If a template exists, template matching is performed, and tracking of a moving object is performed. I do.

Hereinafter, the traveling vehicle detecting device will be described in detail for each processing means. The input means 1 uses, for example, a visible camera or an infrared camera as a camera. In the present invention, a plurality of cameras may be mixed. Selection can be made according to the application. In addition, the observation area photographed by a plurality of cameras is subject to the condition that every area is photographed in duplicate by two or more camera images.
As an example of installation, as shown in FIG. 2, four cameras are set as one set, and these cameras are referred to as camera 1, camera 2, camera 3,
In the case where the camera 4 is used, the cameras 1 and 2 and the cameras 3 and 4 are arranged side by side or inward, and the cameras 1 and 2 and the cameras 3 and 4 are installed facing each other. There is also a method of increasing the number of cameras so that the camera images can be overlapped and photographed. As the monitoring area in FIG. 2, an automobile road including an expressway is assumed.

In the present invention, it is important for the moving area extraction processing means 2 to detect the entire target object, and it is preferable that the entire area including the foot is extracted as one area. When extracting a moving area from video data, if the rate of separation of the area of the target object is small, a method of subtracting one frame of the video or a background video is calculated, and the background is used to calculate the current video. Is used. Further, in the case where the region of the target object is separated by the above-described method, an inter-frame cumulative difference that accumulates the difference result between one frame of the video over several frames is performed, and then binarized by the optimal threshold. It is assumed that a method of extracting a motion region is used.

The labeling processing means 3 separates a target object area such as a person or a vehicle from the binarized moving area detected by the moving area extracting means 2 by labeling processing. In general, the motion area detected by the motion area extraction unit 2 may include an area other than the target object, and examples thereof include a shadow, reflection, and noise added to the video. Therefore, in this means, these unnecessary portions are removed, and a portion close to the actual target object area is set to one.
It also has a role as an area shaping that is extracted as one area.

The vehicle detecting means 4 includes a labeling processing means 3
The moving area detected by means includes a case where the vehicle overlaps, separates, or contains a shadow. Therefore, in the present invention, it is determined whether or not a vehicle is a vehicle to be tracked by using the shape characteristics such as the size of the moving region, the Feret ratio, and the inclination of the whole, based on the size of the moving region and whether or not there is one vehicle. Is calculated, and a rule for making a determination based on the similarity is set. This is determined from the fact that the actual size of the target vehicle is substantially determined. This determination rule will be described below.

The size Size of the person on the image is related to the actual size by projective transformation via a distance from the camera. Therefore, this projective transformation is represented by a linear expression Size = a × Ybottom
+ B can be approximated. Here, Ybottom is the coordinates on the image of the lower end of the moving area, and almost corresponds to the position of the foot of a person. The size having the dimension of length includes the width w and height h of the circumscribed rectangle of the moving region, the square root r of the area (number of pixels) of the moving region, and the like. Each of w, h, and r can be well approximated by the above linear equation. Therefore, w, h, and r are 1 by the principal component analysis.
Information can be compressed into two quantities. Then, Size is changed to Size = αw × w
+ Αh × h + r × r. Where αw, αh,
αr is a projection coefficient onto the first principal component axis, and αw2 + αh2 + α
r2 = 1 is satisfied. Since the size can be approximated by the above-mentioned linear expression, an appropriate fitting method (least square method, etc.)
The coefficients “a” and “b” of the above equation were obtained by the following equation, and the “standard size of the vehicle” was determined by this equation.

Using the standard size, a discrimination rule is determined as follows. The vehicle similarity L in the moving area is defined as L = 100 (1-
| Size−Size0 | / Size0), and a person is determined when L ≧ Th is satisfied. Here, Size0 is a standard size (a × Ybottom + b) obtained from Ybottom of the moving area,
Th is an appropriate threshold. Note that the above shows the rules for persons, but in the judgment of a car and a person, Ybottom
There is also a method of using a first-order approximation formula of the square root r of the area of the moving region and determining whether a person or a car is in a vertical relationship of the approximation formula.

In the template matching means 5, when the vehicle detecting means 4 determines that the moving area is a vehicle, a template search is performed using the template creation candidate position. Generate. Further, for an existing template, template matching is performed to change the position of the template.

FIG. 3 shows a search for a template.
A position in the detected vehicle area where a template should be generated is searched for, and if no template exists in that range, a new template is generated. If a template already exists in the search range, template matching is performed. In addition, template matching is performed on other already existing templates, and the position is updated.

The template matching is performed on a luminance image obtained from each frame image and an edge image extracted by a Sobel filter or the like. Therefore, the pixel values stored in the template are the values extracted from the luminance image and the luminance image, and the values extracted from the edge image when the edge image is used. In the template matching method, as a method of searching for a pattern similar to a template, the shortest distance method / SSDA (Sequential Similarit
y Detection Algorithm), correlation method, statistical method, etc. In the present invention, SSDA having high speed is also used.

The integrated processing means 6 includes a camera video processing system 7
9 are integrated by a multi-viewpoint method of the positions of the templates generated in the above-described processing. A specific example is shown in FIG. In FIG. 6, the camera 1 and the camera 2 photograph substantially the same observation area from different angles. At this time, the image of the camera 1 and the image of the camera 2 track the vehicle by template matching. This tracking is performed by a template, and the position of the template indicates a part of the vehicle. For this reason, even if it is known how the camera is installed in the real space, the position of the vehicle in the real space (on the road) cannot be accurately obtained. Therefore, the diagonal line of the rectangular area of the template is mapped on the real space. For example, in FIG. 4, camera image 1 (FIG.
In (a)), the diagonal line AB of the rectangular area of the template, and in the camera image 2 (FIG. 4B), points A 'and B' correspond to points A and B of the camera image 1. If it is assumed that the camera installation position is known for each point extracted from each image, if the image distortion is not so large, it can be converted by a linear equation.

An image diagram (FIG. 6C) of the real space for tracing the object is shown on the right side of FIG. In this real space, a straight line connecting points A and B and points A 'and B' is plotted in the real space. In this case, assuming that the template processing of each video is correctly performed, an intersection is set at the position of the corresponding target object in the real space except when the target object is on a line connecting the two cameras. It can be proved geometrically to have. Therefore, this intersection can be set as the position of the target object in the real space.

After the intersection is calculated by the intersection calculating means 8 in the integration processing means 6, the target object can be traced by plotting the shortest distance between the position of the target object which existed in the past and the intersection along the time axis. In the template matching process, there are cases where the position of the template does not always accurately indicate the position of the target object, but robustness is ensured by the correspondence based on the shortest distance. In addition, when an intersection is not determined or a plurality of target objects exist, a fictitious intersection may occur, but the processing is performed by association using past histories. The tracking result of the vehicle is displayed on a display or the like on the moving object tracing means.

Next, embodiments of the present invention will be described below.
As an example of FIG. 1, there is a system for monitoring a vehicle. At this time, it is often difficult to correctly extract and track a person with one camera. For example, there are cases where the floor area is large, a part of the vehicle is hidden, or a reflection or shadow of the vehicle is present in the real space. So 2
A high-accuracy and simple system is realized by installing multiple cameras and tracking multiple moving objects by integrating and identifying multi-viewpoint moving object information using an automatically generated template matching method. it can.

Although the width is not so wide as shown in FIG. 2, a surveillance system on a highway or a general road which extends horizontally has a camera area as shown in FIG. By taking multiple shots and using the shot video to track and track multiple cars, using an auto-generated template matching method to integrate and identify multi-view car information, high accuracy and simple The system is realized.

[0030]

As described above, in the present invention, the features of moving image processing are utilized by integrating and identifying multi-viewpoint moving object information by using a template matching method for tracking a vehicle in an automatic monitoring system or the like. It is possible to track the vehicle with high accuracy by simple processing, and it is possible to easily obtain the position of the overlap of the vehicle and the position of the stationary object. is there.

Further, it is possible to omit the difficulty and inefficiency of pattern matching found in stereo processing. Further, there is an effect that observation positions in a wide area can be observed by dispersing the installation positions of the cameras.

[Brief description of the drawings]

FIG. 1 is a block configuration diagram of a traveling vehicle detection device according to Embodiment 1 of the present invention.

FIG. 2 is a layout example of a plurality of cameras in the traveling vehicle detection device according to the first embodiment of the present invention.

FIG. 3 is a diagram showing an example of searching for a template in a vehicle area according to Embodiment 1 of the present invention.

FIG. 4 is a diagram illustrating a processing example of an integrated processing unit of the moving object extraction device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Input means 2 Moving area extraction means 3 Labeling processing means 4 Vehicle detection means 5 Template matching means 6 Integration processing means 7 Moving object tracing means 8 Intersection calculation means 9 Target association means 10, 11, 12 Camera image processing system

Continuing from the front page (72) Inventor Tomonori Narioka 1006 Kazuma Kadoma, Osaka Prefecture Inside Matsushita Electric Industrial Co., Ltd. Reference) 5B057 AA06 AA16 BA02 CE09 CE12 DA07 DA08 DA16 DC05 DC33 5H180 AA01 CC02 CC04 DD01 5L096 BA02 CA05 EA35 EA43 FA06 FA10 FA54 GA28 GA34 HA05 HA09 JA03 JA09

Claims (6)

[Claims] 1. An apparatus for inputting a plurality of camera images and tracking a moving object in an observation area which can be photographed by the camera images, comprising: an input unit for inputting a camera image; and video data obtained from the input unit. Moving area extracting means for detecting a moving area; labeling processing means for detecting information on the moving area such as the size of the area and the number of pixels for each moving area from the moving area obtained from the moving area extracting means; Using the information of each moving area detected by the processing means, it is determined whether or not the moving area is a vehicle, and a vehicle detection output means for extracting only the vehicle area and a template are generated at a position representative of the vehicle area. A camera image processing system including a template matching unit that matches an already existing template in accordance with the movement of the vehicle; and A traveling vehicle detection device comprising: integrated processing means for integrating information; and moving object tracing means for processing the position and classification information of the vehicle obtained from the integrated processing means and displaying the processed information on a display or the like. 2. The observation area photographed by a plurality of cameras,
2. The traveling vehicle detection device according to claim 1, wherein each area is photographed by two or more camera images in an overlapping manner. 3. When installing a plurality of cameras, four cameras are set to one.
As a set, when these cameras are camera 1, camera 2, camera 3, and camera 4, camera 1 and camera 2 and camera 3 and camera 4 are arranged side by side or inward.
3. The camera according to claim 1, wherein the cameras 1 and 2 and the cameras 3 and 4 are installed opposite to each other, and the number of cameras is increased so that any observation area can be photographed with two or more cameras. The traveling vehicle detection device as described in the above. 4. The traveling vehicle detection device according to claim 1, wherein the input means is a visible camera, an infrared camera, or a combination thereof. 5. The integrated processing means obtains a line segment having a rectangular area of the template as a diagonal line using the position of the template from the template matching means of each camera image processing system, and obtains the line segment obtained from each camera image. Is mapped onto a real space in the presence of a vehicle, and the position of the vehicle is estimated from the intersection of the line segment.
The traveling vehicle detection device according to any one of the above. 6. The output processing means performs shortest distance processing in the observation space using information on the position candidate of the target object obtained from the integration processing means, the position of the target object obtained so far, and the target classification of the template. The traveling vehicle detection device according to claim 5, wherein the detection is performed according to an association time axis.