JP2006293629A - Method and device for detecting height of mobile object, and method for determining object shape - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To minutely know an object shape on the basis of height information by calculating the height of the object with the use of a device with a simple structure. <P>SOLUTION: A camera for surveying a place where the object is moved periodically photographs the place. When the mobile object is detected in a photographed image, the speed of the mobile object and the feature line of the mobile object, which is expressed in the image and is vertical with respect to the mobile direction of the mobile object, are detected. Then the height of the feature line is calculated by tracking the feature line on the image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method and apparatus for detecting the height of a moving object that can detect height information of the moving object by image processing. The present invention also relates to a method for determining the shape of an object based on the detected height information of the object.

Detailed traffic information provision is required for smooth traffic signal control and traffic investigation support, especially by providing detailed vehicle type information, signal control according to the mixing rate of the vehicle type and tracking of specific vehicles It becomes possible.
For this reason, there is a need for a method for providing more detailed information with higher accuracy and lower cost than conventional large / small two-car model information.

Japanese Patent Application Laid-Open No. 5-307695 discloses a method for determining the vehicle type based on the apparent size of the vehicle on the camera image. However, there is a problem that a small vehicle with a large vehicle height is misdetected as a large vehicle. Only two small models can be identified.
In Japanese Patent Application Laid-Open No. 11-175880, a plurality of cameras are installed on the road, and the distance from the camera is calculated from the parallax between the cameras, and the detailed vehicle shape is calculated. , Equipment cost will be high. Moreover, since the weight increases from one camera, the installation cost also increases. Furthermore, it is necessary to strictly adjust the direction of the optical axis between the cameras, and handling is difficult.

  Also, in the literature “Determination of vehicle type by recognizing cross-sectional shape from monocular image”, Electric theory D, Volume 120, No. 10, pp.1182-1188, 2000 It is a method to detect the position and calculate the vehicle height from the deviation on the screen, but in the first place, it is difficult to detect the center point within the bonnet range when the bonnet and the side color are the same in the left and right symmetrical point detection is there.

In Japanese Patent Laid-Open No. 10-63987, a detailed image of the entire passing vehicle is synthesized from a plurality of continuous images, and the vehicle type is determined. However, in the image synthesis, the front part and the side part of the vehicle are estimated from only the edge information, but it is generally difficult to realize.
JP-A-5-307695 JP-A-11-175880 JP-A-10-63987 Matsuzaki, Ozawa, “Vehicle Identification by Recognizing Cross-sectional Shapes from Monocular Images,” D & D, Volume 120, No. 10, pp.1182-1188, 2000

Therefore, the present inventor has paid attention to the fact that the height information of an object that crosses the visual field range of one camera at a predetermined speed can be calculated.
In addition, if the height information of the object is used, the outer shape (silhouette) of the vehicle can be easily synthesized.
Accordingly, an object of the present invention is to provide a moving object height detection method and apparatus capable of calculating the height of an object using an apparatus having a simpler configuration than the prior art.

  It is another object of the present invention to provide an object shape determination method that can know the shape of an object in detail based on the height information.

  According to the moving object height detection method of the present invention, when a moving object is detected periodically in a photographed image, the moving object is periodically photographed by a camera overlooking the place where the object moves. This is a method of calculating the height of the feature line by detecting the feature line of the moving object perpendicular to the moving direction of the moving object, which appears in the image, and tracking the feature line on the screen.

In this method, if the place is photographed by a single camera that looks down on the place where the object moves, and the object moves at a known speed in real space, the moving object The time or speed across the camera screen is different.
Therefore, by detecting the speed of the moving object, detecting the feature line of the moving object that appears in the image perpendicular to the moving direction of the moving object, and tracking the feature line on the screen, The height can be calculated.

In the procedure of calculating the height of the feature line, the height of the feature line is calculated based on the speed of the moving object and the time for the feature line to pass from a predetermined position to a predetermined position on the shooting screen. Also good.
Also, by preparing time trajectory patterns of feature lines with different speeds and heights in advance and fitting the time trajectory points of the measured feature lines to the same speed pattern group, the height of the feature lines is calculated. May be. In general, detecting a combination of positions of the same feature line at two separate times may cause erroneous extraction of the line position or omission of extraction due to effects of illumination fluctuations, noise, etc. It is even more difficult to calculate the combination with high accuracy. For this reason, a fitting method has been proposed as a method that is resistant to the effects of illumination fluctuations, noise, and the like by detecting a plausible trajectory.

  The moving object height detection device of the present invention includes a camera installed at a predetermined height so as to overlook the road surface, and a measurement device that captures and processes a captured image of the camera, and the measurement device includes: When a moving object is detected within the detection range set on the camera screen, speed detection means for detecting the speed of the moving object, and features of the moving object that appear in the image perpendicular to the moving direction of the moving object Feature line detection means for detecting a line and height calculation means for calculating the height of the feature line by tracking the feature line of the vehicle on the screen.

According to this apparatus, a plurality of lines that are characteristic of an object shape are extracted from an image of one camera, the trajectory of these lines in a continuous image is detected, and the moving speed information calculated by the speed detecting means is used. The height of each horizontal line can be calculated with high accuracy.
The object shape determination method of the present invention is a method of generating a silhouette image of a moving object by combining a plurality of feature lines detected based on the moving object height detection method.

  When the moving object is a vehicle, the vehicle type can be determined based on the silhouette image of the vehicle.

  As described above, according to the present invention, since the height and shape of an object can be calculated with only one camera, the equipment cost, installation cost, and adjustment cost can be reduced. In addition, since the characteristic line is tracked a plurality of times, it is resistant to noise and height information can be calculated with high accuracy.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a configuration diagram of an object shape determination apparatus including a moving object height detection apparatus according to the present invention.
The camera 2 is installed at a predetermined height so as to overlook the road surface above the road. The visual field range 5 of the camera 2 is on one lane of the road.
The captured image of the camera 2 is input to the measuring device 4 through the video cable 3. The measuring device 4 has communication means and can provide the measurement result to a remote processing center.

In FIG. 2, the example of the picked-up image image | photographed with the camera 2 is shown.
As shown in FIG. 2, the field of view 5 of the camera 2 is a field of view that is easy to extract a horizontal line that is perpendicular to the moving direction of a moving object such as a vehicle bumper or a windshield (left and right direction 2 to 3 m, up and down Direction 2 to 3 m).
FIG. 3 is a diagram illustrating a configuration example of the measuring device 4.

The measuring device 4 captures the image of the camera 2, determines the presence or absence of a vehicle within the vehicle detection measurement range set on the screen of the camera 2, and extracts and tracks a horizontal line when a vehicle is detected. Then, arithmetic processing such as object shape determination is executed.
The measurement device 4 includes an image input unit 41, an A / D conversion unit 42, an image memory 43, a calculation unit 44, and a communication unit 45.

  The image input unit 41 provides an interface for capturing an image of the camera 2. The A / D converter 42 digitally converts the image signal. The image memory 43 temporarily stores the digitally converted image data. The calculation unit 44 executes calculation processes such as horizontal line extraction, tracking, and object shape determination. The communication part 45 communicates with a traffic light, a display board, a traffic control center, etc., and conveys the information of an object shape.

The camera 2 and the measuring device 4 may be integrated.
FIG. 4 is a flowchart illustrating an object shape calculation processing method performed by the measurement device 4.
The calculation unit 44 captures an image through the image input unit 41 and stores it in the image memory 43 (step S1). The image capture interval is, for example, 30 images per second.
Based on the image stored in the image memory 43, the calculation unit 44 detects whether or not a vehicle is present on the screen (step S2).

FIG. 5 is a diagram showing a measurement range R for vehicle detection. The measurement range R is indicated by a rectangular thick frame on the screen.
Within this frame, the presence or absence of the vehicle is determined.
For example, a road surface image (hereinafter referred to as a background image) when a vehicle is not present is created in advance, a brightness difference from the captured image (hereinafter referred to as an input image) is calculated, and a change that exceeds a threshold value If the vehicle is detected, the vehicle is present, and if the change is less than or equal to the threshold, the vehicle is absent (step S3).

Alternatively, the position of a portion of the road surface pattern where the brightness changes (hereinafter referred to as an edge) is stored in advance, the edge is also calculated for the input image, the difference in sharpness between the edges is calculated, When the difference is detected, the vehicle can be present, and when the difference is less than or equal to the threshold, the vehicle can be absent.
If it is determined that there is a vehicle, the vehicle speed is detected (step S4). A well-known means can be used for vehicle speed detection. For example, an ultrasonic Doppler radar can be installed under the camera 2 to measure the speed of the vehicle. In addition, it is possible to know the speed of the vehicle by installing two embedded vehicle detectors on the road and measuring the time when the vehicle passes through these vehicle detectors.

Next, feature line extraction processing is performed (step S5).
FIG. 6 is a detailed flowchart for explaining the feature line extraction processing.
First, for each pixel, the brightness difference between the previous time and the current time is calculated. If the difference is equal to or greater than a threshold, it is determined that there is a time difference change (step T1). This is to erase an edge of a road surface that does not move, such as a road marking.

Next, the presence / absence of a horizontal edge is calculated for a pixel with a time difference change (step T2). For example, as shown in FIG. 7, when the Sobel filter processing is performed on the target pixel and its nine neighboring pixels and the absolute value of the calculated value D is equal to or larger than the threshold value, the target pixel (*) In FIG.
The calculation formula is, for example, D = I11 × 1 + I12 × 2 + I13 × 1-I31 × 1-I32 × 2-I33 × 1
(Ijk represents the luminance of the pixel in the j-th row and the k-th column of the matrix).

Next, a thinning process is performed on pixels that have a time difference change and constitute a horizontal edge (step T3). This is a process for removing adjacent edges and facilitating extraction of feature lines. For example, a horizontal edge image is thinned using Hilditch's thinning algorithm (see “Image Processing and Recognition”, Aiin et al., Shosodo).
FIG. 8 is a diagram illustrating an example of a thinned horizontal edge image.

Next, the number of horizontal edges is calculated in the horizontal direction of the screen, and a histogram is created as shown at the right end of FIG. 8 (step T4).
Based on this histogram, from the upper part to the lower part of the screen (or from the lower part to the upper part), a line whose number of horizontal edges is greater than a threshold value is extracted as compared with the horizontal lines adjacent to the top and bottom of the horizontal line. This corresponds to a sharp peak portion surrounded by a circle in FIG. This pointed line is called “characteristic line”.

When the feature line extraction process is completed in this manner, the process returns to FIG. 4, the vehicle detection count is incremented by 1 (step S 6), and the process returns to step S 1.
Thereafter, the processing from step S2 onward is repeated based on the next captured image.
If it is determined in step S3 that there are no vehicles in the image, the process proceeds to step S7, where the number of vehicle detections so far is compared with a threshold value n.

The threshold value n of the number of vehicle detections is set to about “3”, for example. This is to stop the following feature line height calculation process because it is doubtful whether the vehicle is really a vehicle if continuous vehicle detection can be performed only three times or less.
If the number of vehicle detections exceeds the threshold value n, the process proceeds to step S8, and the feature line height calculation process is entered.

Hereinafter, the feature line height calculation process will be described in detail.
FIG. 9 is a graph in which the position of the detected feature line on the screen is on the vertical axis and the time is on the horizontal axis. The case where three feature lines are shown is illustrated.
In FIG. 9, the trajectory of each feature line is an apparent movement on the screen and is a projection of the position in real space at each time. The characteristic line moves from the top to the bottom of the screen as the vehicle travels.

Here, a conversion formula between coordinates in the real space and the imaging plane is defined.
As shown in FIG. 10, the coordinate system in the real space is (X, Y, Z), and the coordinate system on the imaging surface of the camera 2 is (P, Q).
In real space, Z is the vertical direction, Y is the direction in which the road extends, and X is the direction that crosses the road.
The camera 2 is installed on an extended line of the road, and on the photographing surface thereof, the horizontal axis P is parallel to the road crossing direction X, and the vertical axis Q is perpendicular to the horizontal axis P. The vertical axis Q is inclined with respect to the vertical direction Z of the road by an angle α corresponding to the depression angle.

Here, since it is assumed that the camera 2 is installed on an extension line of the road and the vehicle approaches the camera 2, the movement in the road crossing direction X in the real space is not considered. On the imaging plane, only the position coordinate q of the feature line is considered, and the coordinate p orthogonal thereto is ignored. Therefore, the coordinates of the vehicle in the real space are (y, z) and q on the photographing surface.
The relational expressions (1) and (2) are established for the coordinate q (unit: number of pixels) on the imaging surface and the coordinate point (y, z) on the real space.

However, the ratio of the focal length f of the camera 2 lens, the installation height H of the camera 2, the height Sq in the Q-axis direction on the imaging surface, and the number Nq of pixels in the Q-axis direction of the imaging surface corresponding to Sq (one Pixel size) is known and treated as a constant value.
z = H− (ftanα−ξ) y / (ξtanα + f) (1)
ξ = (Sq / Nq) q (2)
Here, ξ is the height of the feature line on the imaging surface. Since the moving speed v of the vehicle within the minute time (the time passing through the upper and lower range of the screen) has been detected, the Y coordinate y (t) at each time can be calculated based on the elapsed time from the predetermined time.

The height z of the feature line in the real space is assumed to be a certain constant value.
Then, from equation (1), ξ (t) can be calculated as a function of time based on y (t). Using this ξ (t), q (t) can be obtained as a function of time from the equation (2).
Therefore, the curve U can be drawn using q (t) as a function of time.
If the height z of the feature line in the real space is a variable (parameter) and the height z is changed, a curve group depicting a plurality of q (t) can be formed.

FIG. 11 is a graph showing a curve group U of these q (t) using the height z of the feature line in the real space as a variable.
In the figure, the height on the photographing surface at time t0 is unified to the same value qs for each curve. In the same figure, the height z has a relationship of z0>z1> z2.
It can be seen that the curve U0 corresponding to the height z0 passes through the imaging surface from the upper end to the lower end (referred to as “field of view of the camera 2” from the upper end to the lower end of the imaging surface) in a short time. This phenomenon corresponds to an empirical rule that a high object such as a vehicle roof crosses the field of view of the camera 2 in a relatively short time.

On the other hand, an object with a low height, such as a bumper of a vehicle, passes through the field of view of the camera 2 in a relatively long time even when traveling at the same speed. In the graph of FIG. 11, the curve U2 corresponding to the height z2 passes through the imaging surface over a long time.
Therefore, assuming that the moving speed of the vehicle is known, the height of the object can be obtained by measuring the time T during which the characteristic line passes through the field of view of the camera 2.

Further, assuming that the moving speed of the vehicle is known, instead of the time T when the feature line passes through the field of view of the camera 2, the apparent speed at which the feature line passes through the field of view of the camera 2 (from the upper end to the lower end of the screen). The height of the object can also be obtained based on the distance (q0−q1) divided by the time T).
For example, in FIG. 11, it is assumed that an example of the trajectory of the feature line from time to time is plotted, and it takes time T for these points to pass through the field of view of the camera 2. The curve U ^* passing through the field of view of the camera 2 over this time T can be specified. If the height corresponding to the curve U ^* is z ^* , the z ^* is the height of the feature line in real space.

Further, instead of measuring the time T and the apparent speed, the trajectory of the feature line is known every moment (that is, the shape of the curve U). Therefore, the shape of this trajectory is used as each curve constituting the curve group U. The curve having the most similar inclination may be specified by fitting to. Then, the height corresponding to this curve is known.
For example, in FIG. 11, the trajectory of the photographed feature line is plotted every moment, but these points are assumed to fit the curve U ^* . When the height corresponding to the curve U ^* is z ^* , the z ^* is obtained as the height of the feature line in the real space.

Since the shape of the curve group is different if the moving speed of the vehicle is different, a curve group must be prepared for each moving speed of the vehicle.
However, if a group of curves corresponding to one moving speed is provided, a group of curves of other moving speeds can be easily obtained simply by extending or reducing the time scale.
In the above description, as shown in FIG. 11, the time is measured on the graph and the coincidence of the curve shapes is examined, but these processes are actually performed by using the computer processing function. Needless to say. For example, in order to check the coincidence of the curve shapes, a known maximum likelihood estimation method such as a least square method may be used.

As described above, when the height z of the feature line in the real space is known, the distance y from the vehicle to the camera 2 is calculated by the equation (1) using the position q every moment on the screen. It can be calculated. In this way, the coordinates (y, z) in the real space of the feature line are known.
Therefore, by combining the feature line positions, a silhouette image of the vehicle can be created as shown in FIG.

From the calculated silhouette image, the vehicle length, vehicle height, and shape can be discriminated, and as shown in FIG. 13, the vehicle type such as large vehicle / small vehicle, sedan / wagon type can be determined.
Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to the above-described embodiments. For example, although a vehicle traveling on a road is assumed as an object, a pedestrian or an aircraft may be used as long as it is a moving object. Moreover, even if it is a ship which navigates on the water surface, the shape can be determined. In addition, various modifications can be made within the scope of the present invention.

It is a block diagram of an object shape determination apparatus. It is a figure which shows the picked-up image example. 3 is a block diagram illustrating a configuration example of a measuring device 4. FIG. It is a flowchart which shows a vehicle shape calculation processing method. It is a figure which shows the measurement range R for vehicle detection. It is a detailed flowchart for demonstrating a feature line extraction process. It is a figure which shows the mask for a detection of a horizontal edge. It is a figure which shows the thinned horizontal edge image and horizontal edge number histogram. It is a locus diagram on the time space of each characteristic line position reflected on the screen. It is the figure which defined the coordinate on real space and an imaging surface. It is a graph which shows the relationship q (t) between each feature line position q reflected on the screen and time t, using the height z of the feature line in the real space as a variable. It is the figure which combined the characteristic line position and created the silhouette image of the vehicle. It is a figure which shows the result of having discriminate | determined the vehicle length, the vehicle height, and the shape from the silhouette image.

Explanation of symbols

2 Camera 3 Video cable 4 Measuring device 41 Image input unit 42 A / D conversion unit 43 Image memory 44 Calculation unit 45 Communication unit

Claims (6)

(A) Using a camera overlooking the place where the object moves, the place is periodically photographed;
(B) When a moving object is detected in the captured image, the speed of the moving object and the feature line of the moving object perpendicular to the moving direction of the moving object appearing in the image are detected;
(C) A moving object height detection method for calculating the height of a feature line by tracking the feature line on a screen. In the procedure (c),
The height detection method of the moving object according to claim 1, wherein the height of the feature line is calculated based on the speed of the moving object and a time during which the feature line passes from a predetermined position to a predetermined position on the shooting screen. In the procedure (c),
Claims: The height of the feature line is calculated by preparing time trajectory patterns of feature lines with different speeds and heights in advance and fitting the time trajectory points of the measured feature lines to the same speed pattern group. 2. A method for detecting a height of a moving object according to 1. A camera installed at a predetermined height to overlook the road surface;
A measuring device that captures and processes captured images of the camera;
When a moving object is detected within the detection range set on the camera screen, it is provided with speed detecting means for detecting the speed of the moving object,
The measurement device includes a feature line detection unit that detects a feature line of a moving object that appears in the image and is perpendicular to the moving direction of the moving object;
A moving object height detection apparatus comprising: a height calculation unit that calculates a height of the feature line by tracking the feature line of the vehicle on a screen.   An object shape determination method for generating a silhouette image of a moving object by combining a plurality of feature lines detected based on the moving object height detection method according to claim 1.   The object shape determination method according to claim 5, wherein the moving object is a vehicle, and a vehicle type is determined based on a silhouette image of the vehicle.