JP3456339B2 - Object observation method, object observation device using the method, traffic flow measurement device and parking lot observation device using the device - Google Patents

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION The present invention relates to a method and apparatus for observing an object at a predetermined observation position,
Further, the present invention observes the vehicles on the road, and based on the temporal transition of the observation results, the number of vehicles passing, the vehicle speed,
Data such as vehicle type (collectively referred to as "traffic flow data" below)
The present invention relates to a traffic flow measuring device for measuring a traffic flow and a parking lot observation device for observing a vehicle parked in a parking space.

[0002]

2. Description of the Related Art Conventionally, as a device for measuring traffic flow data, a camera is installed above a road to capture an image of a predetermined position on the road. A device for measuring is being developed. In this device, an image in which an object does not exist on the road is stored in advance as a background image, and the difference between the image from the camera and this background image is extracted as an image related to the vehicle, and the extraction result is extracted. The flow of the vehicle on the road is measured by sequentially tracking it (Technical Report of the Institute of Electrical Engineers of Japan No. 512, pp. 80-81). In addition, instead of storing the background image, a method of storing the image captured one step before and subtracting the immediately preceding image from the latest image to extract only the change in the vehicle position has also been proposed.

Furthermore, in recent years, a technology (DTT method) for creating a spatiotemporal image in which input images from a camera are accumulated every hour and measuring traffic flow data has been announced (Journal of the Institute of Electronics, Information and Communication Engineers Vol.J77- D-II No. 10 p2019-2026, 1994). According to this method, after the characteristic amount data such as the contour of the vehicle is extracted from the images for each time, the one-dimensional data is created by projecting the characteristic amount data on the direction axis parallel to the moving direction of the vehicle. . This one-dimensional data is further arranged in time series to create a two-dimensional image (DDT image) consisting of a direction axis and a time axis, and an object moving in a predetermined direction is extracted from this DDT image.

In order to determine whether or not each parking area of the parking lot is vacant, generally, the above-mentioned first or second method, that is, the input image obtained by imaging the observation position and the background image are compared. Whether or not the vehicle is parked in the parking area is determined based on the difference or the difference between the input image and the image captured one step before.

[0005]

When the difference between the input image and the background image is used for measuring the traffic flow, it is necessary to store the background image according to various environments such as day and night, fine weather, and rainy weather. There is a limit in assuming the required environment and updating the background image according to the change in the environment, and it is not possible to extract the vehicle with high accuracy. In the case of each of the first, second, and third methods described above, a shadow that moves on the road such as a shadow of a vehicle or a tree swaying by the wind, or a light reflected at night or reflected on a wet road For example, there is a problem that a motion other than the vehicle is erroneously detected as the vehicle.

Further, in the case of each of the conventional methods, when a large vehicle is present on the road or a traffic jam occurs, the vehicles cannot be distinguished on the screen, and there is a risk that a plurality of vehicles will be erroneously detected as one vehicle. is there. In addition, if the image of the shadow of a large vehicle overlaps with a small vehicle in the back to reach the adjacent lane, the shadow or small vehicle will be erroneously detected as a large vehicle, and the number and type of vehicles on the road can be accurately determined. It is impossible to do.

The above-mentioned problem also applies to the parking lot observation device, and there is a possibility that the discrimination accuracy may be deteriorated due to the change of the illuminance at the observation position, or the shadow of the vehicle in the adjacent area may be erroneously detected. .

In addition, these conventional methods require a memory for accumulating various background images and images for each time, and a hardware configuration for performing arithmetic processing such as subtraction and cumulative addition of each image. However, there is also a problem that the structure of the device becomes large and the manufacturing cost becomes high.

The present invention has been made in view of the above problem. The observation position is imaged by a plurality of cameras, the characteristic portion in each obtained image is three-dimensionally measured, and the height data is used. By discriminating the target object with the purpose, the object is to observe the target object with high accuracy.

Further, according to the present invention, the three-dimensional coordinates of the characteristic portion in the image obtained from each of the cameras are projected on an imaginary vertical plane perpendicular to the plane in contact with the object, whereby the shape of the object is projected. The purpose is to accurately grasp the existence, number, and type of objects at the observation position.

A further object of the present invention is to observe the inside of a road or a parking space by using the above method, and to perform highly accurate traffic flow measurement and vehicle discrimination based on the observation result.

[0012]

According to a first aspect of the present invention, in a method of observing an object, two or more image pickup devices are simultaneously imaged toward a predetermined observation position, and each of the obtained images is A first step of extracting a characteristic part for each image, a second step of associating the extracted characteristic part of each image between the images, and a three-dimensional measurement for each associated characteristic part. And a third step of extracting the three-dimensional coordinates of each characteristic part, and a fourth step of projecting the extracted three-dimensional coordinates of each characteristic part onto a virtual vertical plane that is perpendicular to the plane in contact with the object. And the fifth step of discriminating the object by comparing the projection data with predetermined two-dimensional model data.

In the invention of claim 2, as the projection process of the fourth step, the three-dimensional coordinates of each characteristic portion are divided into groups based on the positional relationship on the plane with which the object is in contact, and then, for each group, The three-dimensional coordinates included in the group are projected on the virtual vertical plane.

According to the third aspect of the invention, as the comparison process of the fifth step, for each projection point generated on the virtual vertical plane by the projection process of the fourth step, a predetermined point including the projection point is specified. A range is set, and each point within this range is weighted based on the positional relationship with the projection point and compared with the two-dimensional model data.

In the object observing method according to a fourth aspect, the first step similar to the first aspect, and the second step of specifying the points showing the representative features of the extracted feature portions of each image, respectively. And a third step of associating the specified representative points between images, and a three-dimensional measurement for each associated representative point to extract three-dimensional coordinates of each representative point. 4, the fifth step of projecting the extracted three-dimensional coordinates of each representative point on a virtual vertical plane that is perpendicular to the plane in contact with the object, and the projection data of the predetermined two-dimensional model. The sixth step of discriminating the object by comparing with the data is performed in series.

In the invention of claim 5, the same process as in claim 2 is executed as the projection process, and in the invention of claim 6, the same process as in claim 3 is executed as the comparison process.

According to a seventh aspect of the present invention, there is provided an object observing apparatus, which comprises two or more imaging devices arranged toward an observation position, and a feature extracting means for extracting a characteristic part from each image simultaneously captured by each imaging device. And associating means for associating between the images the characteristic portions of the images extracted by the characteristic extracting means, and the three-dimensional measurement of the respective characteristic portions associated by the associating means. Coordinate extraction means for extracting the three-dimensional coordinates of the portion, storage means for storing the two-dimensional model data of the object, and a virtual vertical plane are set so as to be positioned perpendicular to the plane in contact with the object. Projecting means for projecting the three-dimensional coordinates of each characteristic portion extracted by the coordinate extracting means on a vertical plane, and a projection result by the projecting means stored in the storage means. And a comparing means for comparing the Rudeta.

An object observing apparatus according to a tenth aspect of the present invention is a representative of two or more image pickup apparatuses and feature extracting means as in the seventh aspect, and a characteristic portion of each image extracted by the characteristic extracting means. Associated with the representative point specifying means for specifying a point indicating the characteristic of the image, the associating means for associating the representative points specified by the representative point specifying means between the images, and the associating means. Coordinate extraction means for performing three-dimensional measurement for each representative point to extract three-dimensional coordinates of each representative point, storage means for storing two-dimensional model data of the object, and a position perpendicular to a plane in contact with the object. A virtual vertical plane to be set, a projection means for projecting the three-dimensional coordinates of each representative point extracted by the coordinate extraction means on the virtual vertical plane, and a projection result by the projection means for the storage means. And a comparing means for comparing the stored two-dimensional model data.

According to the invention of claims 8 and 11, the projection means divides the three-dimensional coordinates extracted by the coordinate extraction means into groups based on the positional relationship on the plane with which the object is in contact, and then, for each group. In addition, the three-dimensional coordinates included in the group are projected on the virtual vertical plane.

In the ninth and twelfth aspects of the present invention, the comparing means sets, for each projection point generated on the virtual vertical plane by the projection processing by the projection means, a predetermined range including the projection point. The points in this range are weighted based on the positional relationship with the projection point and are compared with the two-dimensional model data.

The thirteenth and fourteenth aspects of the present invention relate to an apparatus for observing the flow of a vehicle on a road and measuring traffic flow data on the road based on the temporal transition of the observation result. The traffic flow measuring device according to the invention of claim 13 is
Two or more image pickup devices arranged above the road toward an observation position on the road, feature extraction means for extracting a feature part from each image simultaneously captured by the image pickup devices at predetermined time intervals, An associating unit that associates the characteristic portions of each image extracted by the characteristic extracting unit between the images, and three-dimensional measurement by performing three-dimensional measurement for each of the characteristic portions associated by the associating unit. A coordinate extracting means for extracting the dimensional coordinates, a virtual vertical plane along the road, and a projecting means for projecting the three-dimensional coordinate of each characteristic portion extracted by the coordinate extracting means on the virtual vertical plane; For each type of vehicle, by comparing the shape model of the side surface of each vehicle with the storage means, and comparing the projection result by the projection means with each model stored in the storage means, And it includes a vehicle discriminating means for discriminating vehicles, and observation means for observing the flow of vehicles in the vehicle discriminating means a discrimination result successively tracking to by on the road.

A traffic flow measuring apparatus according to a fourteenth aspect of the present invention comprises an image pickup apparatus, a feature extracting means, a correlating means, and a coordinate extracting means similar to those described above, and each characteristic portion extracted by the coordinate extracting means. Among the three-dimensional coordinates, the vehicle discrimination means for discriminating the vehicle on the road by using the relative positional relationship of the three-dimensional coordinates satisfying the predetermined height, and the discrimination result by the vehicle discrimination means are sequentially traced on the road. And an observing means for observing the flow of the vehicle.

The inventions of claims 15 and 16 relate to an apparatus for observing a vehicle parked in an area of a predetermined shape. A parking lot observation device according to the invention of claim 15 is characterized in that two or more image pickup devices arranged toward the area above the area and feature portions are extracted from each image simultaneously captured by each image pickup device. Three-dimensional measurement is performed on the extraction unit, the association unit that associates the feature portions of each image extracted by the feature extraction unit between the images, and the feature portions associated by the association unit. Coordinate extraction means for extracting the three-dimensional coordinates of each characteristic portion and a virtual vertical plane positioned perpendicular to the installation surface of the vehicle in the area are set, and extracted on the virtual vertical plane by the coordinate extraction means. The projection means for projecting the three-dimensional coordinates of each characteristic portion, the storage means for storing the shape model of each vehicle for a plurality of types of vehicles, and the projection result by the projection means are described above. And a discriminating means for discriminating vehicles within said area by comparing with the respective models stored in means.

A parking lot observing device according to a sixteenth aspect comprises an image pickup device, a feature extracting means, an associating means, and a coordinate extracting means similar to those described above, and the three-dimensional structure of each characteristic portion extracted by the coordinate extracting means. Among the coordinates, there is provided a judging means for judging the vehicle in the area by using three-dimensional coordinates satisfying a predetermined height condition.

[0025]

According to the invention of claims 1 and 7, the characteristic portions of each image are extracted from the images obtained by picking up the observation positions by two or more image pickup devices, and the correspondence between the images of these characteristic portions causes After the three-dimensional coordinates of the characteristic portion are extracted, the shape of the object is extracted by projecting the extracted three-dimensional coordinates on a virtual vertical plane that is positioned perpendicular to the plane in contact with the object. . Therefore, by comparing this projection result with the model data of the object, the type, position, number, etc. of the object at the observation position can be grasped.

According to the fourth and tenth aspects of the present invention, when the characteristic portions are respectively extracted from the images by the two or more image pickup devices in the same manner as described above, a point showing a representative characteristic for each characteristic portion (eg, The center point of the characteristic part) is specified. Similar to the above, three-dimensional coordinates are extracted for each of the identified representative points, and the object is determined based on the result obtained by projecting the extraction result on the virtual vertical plane.

In each of the inventions of claims 2, 5, 8 and 11, the extracted three-dimensional coordinates are divided into groups based on the positional relationship on the plane in contact with the object, and then each group is moved to the virtual vertical plane. Since the projection processing is performed, even when there are a plurality of objects, it is possible to individually determine them according to their positional relationship.

According to the inventions of claims 3, 6, 9 and 12, for each projection point on the virtual vertical plane, for each point within a predetermined range including the projection point, a weight based on the distance from the projection point is used. The error generated during the three-dimensional measurement is removed by adding the symbol and comparing with the model data.

According to the thirteenth aspect of the present invention, two or more image pickup devices are provided above the road and the images obtained by picking up the observation position are captured at predetermined time intervals.
After extracting the three-dimensional coordinates of the characteristic portion in each image, the extraction result is projected on a virtual vertical plane along the road. As a result, the shape of the side surface of the object located on the road is projected on the virtual vertical plane. On the other hand, since the storage means stores the side surface shape models of a plurality of types of vehicles,
By comparing these models with the projection result, the type, number and position of vehicles on the road can be determined.

In the fourteenth aspect of the present invention, after the characteristic portions are extracted from the images obtained from the two or more image pickup devices arranged above the road, the respective characteristic portions are associated with each other in the same manner as described above, and the three-dimensional measurement is performed. After executing, the vehicle on the road is discriminated by using the relative positional relationship of the three-dimensional coordinates satisfying the predetermined height condition. Therefore, for example, it is necessary to extract the light of the vehicle when driving at night and execute the discrimination processing. You can Therefore, even when it is difficult to extract an image showing the shape of the vehicle at night,
Vehicles on the road can be identified, and highly accurate traffic flow measurement can be performed.

In the inventions of claims 15 and 16, two or more image pickup devices are arranged toward an area of a predetermined shape and imaged at the same time, and the obtained images are three-dimensionally characterized by the same method as described above. Extract the coordinates. After that, in the invention of claim 15, each of the extracted three-dimensional coordinates is projected onto a virtual vertical plane that is positioned perpendicular to the installation surface of the vehicle in the area, and the projection result and the shape models of various vehicles are obtained. Since the comparison is made, it is possible to determine even the vehicle type of the vehicle parked in the area. In the invention of claim 16, depending on whether or not each extracted three-dimensional coordinate satisfies a predetermined height condition,
It is determined whether or not there is a vehicle in the area.

[0032]

BEST MODE FOR CARRYING OUT THE INVENTION Five embodiments will be shown below as various modes according to the present invention. In each of the embodiments, two cameras 3a and 3b are simultaneously imaged toward a road or a parking area to be observed, and after characteristic portions such as edge constituent points are extracted from the input image of each camera, The three-dimensional coordinates of the characteristic portions are calculated by associating these characteristic portions with each other. Further, the three-dimensional coordinates are projected on a virtual vertical plane, or coordinate points having predetermined height data thereof are extracted, and the processing result is used to execute the observation processing of the object (vehicle in this case).

[0033]

FIG. 1 shows an example of installation of a traffic flow measuring device according to an embodiment of the present invention. This traffic flow measuring device is configured by disposing an F-shaped support 2 right next to a road 1 and mounting two cameras 3a and 3b and a control processing device 4 on the support 2. Road 1 by cameras 3a and 3b
By processing the image obtained by imaging the above from the upper side by the control processing device 4, the number and types of vehicles passing through each road on the road 1 can be discriminated, the passing speed of the specific vehicle can be measured, and the vehicle is illegally parked. Is detected.

The pillar 2 is arranged such that its cross rail portion projects on the road, and the cameras 3a and 3b are vertically mounted on a vertical rod 6 fixedly arranged between the cross rails. . Further, the control processing device 4 is attached near the base of the column 2 for the necessity of maintenance and inspection.

FIG. 2 shows an installation example of the cameras 3a and 3b. Each of the cameras 3a and 3b has a lens having the same focal length, and each side surface is supported by long support plates 7 and 7 attached to the vertical rod 6 at a predetermined angle. At this time, the optical axes of the cameras 3a and 3b are arranged parallel to each other in a direction orthogonal to the length direction of the support plate 7, that is, in the direction of the road, and the respective image pickup surfaces are located on the same plane. The mounting positions of the cameras 3a and 3b are adjusted. The camera 3a of the support plate 7
A short reinforcing plate 8 is connected between the supporting position of the supporting plate 7 and the supporting position of the camera 3b. By fixing the other end of the reinforcing plate 8 to the vertical rod 6, the inclination of the supporting plate 7 is maintained. To be done.

FIG. 3 shows another installation example of the cameras 3a and 3b. In this example, each of the cameras 3a and 3b includes two support plates 7a, which are mounted orthogonally to the vertical rod 6.
Both side surfaces are supported by 7b, and each support plate 7a, 7b
By maintaining b in the horizontal direction by the reinforcing plate 9, the positions of the cameras 3a and 3b are stabilized. Also in this case, the positions of the cameras 3a and 3b are adjusted so that their optical axes are parallel to each other toward the road.

In the case of this example, the image pickup surface of the upper camera 3a is arranged at a position retracted from the image pickup surface of the lower camera 3b, so that the camera 3a can be processed before the image processing described later.
It is necessary to magnify the image in accordance with the receding distance. The installation examples of the cameras 3a and 3b are not limited to the above,
As shown in FIG. 4, an integrated device in which the cameras 3a and 3b are housed in the box-shaped unit 25 may be prepared and supported by the support plate 7 and the reinforcing plate 8.

Although the road is imaged by two cameras in this embodiment, the present invention is not limited to this, and three or more cameras may be used. Further, the support column to which the camera is attached is not limited to the above-mentioned F-shaped support column 2, and an existing telegraph pole or illumination pole may be modified and used.

FIG. 5 shows an electrical configuration of the above traffic flow measuring device. The control processing unit 4 includes an A / D conversion unit 10a,
10b, image memories 11a and 11b, edge image memories 12a and 12b, edge extraction unit 13, and association processing unit 1
4, three-dimensional coordinate calculation unit 15, camera parameter storage unit 1
6, projection processing unit 17, lane data storage unit 18, model data storage unit 19, comparison determination unit 20, processing result storage unit 2
1, includes a tracking processing unit 22 and the like.

The A / D converters 10a and 10b are for converting analog video signals from the cameras 3a and 3b into grayscale image signals, and the converted image signals are respectively stored in the image memory 11a. , 11b. In the following description, the image of the camera 3a stored in the image memory 11a is referred to as “first image”, the image memory 11
The image of the camera 3b stored in b will be referred to as a "second image".

FIGS. 6A and 6B show the first image,
As a specific example of the second image, image data including images such as lanes on the road, vehicles, and shadows of the vehicles are generated.

The edge extraction unit 13 is provided for each image memory 1
The first image and the second image are extracted from 1a and 11b, respectively, to generate an edge image. A Laplacian filter as shown in FIG. 7 is scanned for each image to extract the zero-crossing point. Furthermore, the edge extraction unit 13
The coordinates of the zero-crossing point are calculated, and a binary edge image in which the pixel at this coordinate position is a black pixel and the other pixels are white pixels is generated. Note that the above edge extraction processing is not limited to the Laplacian filter, and an edge extraction filter such as a Sobel filter may be used.

The edge image generated for the first image (hereinafter referred to as "first edge image") is stored in the first edge image memory 12a as the edge image generated for the second image (hereinafter referred to as "second edge image"). Are stored in the second edge image memory 12b. Figure 8 (1)
6 (2) shows the results of the above edge extraction processing performed on the first and second images shown in FIGS. 6 (1) and 6 (2), which correspond to the contour of the vehicle and the shadow of the vehicle. Edge components have been extracted.

When the above-described edge image generation and storage processing in the edge image memories 12a and 12b are completed, the association processing unit 14 then performs processing for associating edge constituent points in each edge image. The three-dimensional coordinate calculation unit 15 performs a three-dimensional coordinate calculation process for each edge composing point.

9 (1) to 9 (4) show specific examples of the associating process of the associating processor 14, and the details of the process of the associating processor 14 will be described below with reference to the drawings. To do. First, the association processing unit 14 extracts a predetermined edge composing point p from the first edge image E1 (shown in FIG. 9A) of the edge image memory 12a. Next, the association processing unit 14 pays attention to the second edge image E2 (shown in FIG. 9 (2)) of the edge image memory 12b, and in the edge image E2, the second edge image E2 is placed on the epipolar line L of the edge composing point p. Positioned edge constituent points q ₁ , q ₂ , q ₃ , q ₄ , q
₅ is extracted as a corresponding candidate point of the edge composing point p. In this case, since the cameras 3a and 3b are arranged vertically as described above, the epipolar line L is perpendicular to the y axis, and the corresponding candidate points can be easily extracted.

Next, the association processing unit 14 sets the same coordinates (x, _yu ) as the edge composing point p on the first image G1 (shown in FIG. 9 (3)) in the first image memory 11a. A window W _u of a predetermined size with the point P located at
To set. Further, the correspondence processing unit 14 also has the same coordinates (x, x) as the correspondence candidate points q _{1 to} q ₅ on the second image G2 (shown in FIG. 9 (4)) in the second image memory 11b.
y ₁ ), (x, y ₂ ), (x, y ₃ ), (x, y ₄ ) _,
Extract the Q ₁ to Q ₅ point located respectively on (x, y _5), and around these points Q ₁ to Q _5, and the window W ₁ to _W-5 having the same size as the window W _u It is set on the image G2.

When each window is set, the association processing unit 14 executes the following equation (1) for each window W _{1 to} W ₅ on the second image, and each window and the first image on the first image. The difference C with the window W _u is calculated.
In the following equation, g _U (x, y) is the window W _U
, G _L (x, y) is the brightness value of a predetermined pixel in the window W _L (L = 1 to 5),
Shown respectively. Further, i and j are variables that vary depending on the size of each window.

[0048]

[Equation 1]

The correlation processing unit 14 determines that each window W ₁
The difference C with the window W _U obtained for W ₅
Are compared with each other, and the window having the smallest difference is determined to correspond to the window W _U. Then, a point q _L on the second edge image on the same coordinates (x, y _L ) as the center point Q _{L of the} window is determined as a corresponding point of the edge constituent point p of the first edge image.

FIG. 10 shows the result of calculating the difference between the windows W _{1 to} W ₅ and the window W _U. In the illustrated example, the degree of difference in the second window W ₂ is the smallest, and therefore, the corresponding point in the second edge image with respect to the edge composing point p is regarded as q ₂ .

Instead of the above difference, the normalized cross-correlation operation with the window W _U is performed for each of the windows W _{1 to} W ₅ , and the window having the highest correlation value is set as the window W _U. It may be determined that they correspond.

When the above-mentioned associating process is performed for all edge constituent points in both edge images, the three-dimensional coordinate calculating section 15 determines the corresponding edge constituent points p, q between the respective edge images E1, E2. The coordinates (x, y _U ) _, (x, y _L ) and each camera 3 stored in the camera parameter storage unit 16.
Using the parameters a and 3b, three-dimensional coordinates corresponding to each edge composing point are calculated based on the principle of triangulation measurement.

FIG. 11 shows the principle of the above-mentioned triangulation measurement.
In the figure, P indicates a predetermined feature point on the object 23 (vehicle in this case) on the roadway, and the object point image P _u of this feature point P on the first image G1 and the second image G2. , P _L have appeared. In the figure, U is the focus of the camera 3a and L is the camera 3a.
The focal points of b are shown respectively.

The three-dimensional coordinates (X, Y, Z) corresponding to the corresponding object point images P _U and P _L correspond to the spatial position of the characteristic point P. Therefore, if the above method is executed for all the characteristic points of the object 23, the three-dimensional shape of the object 23 can be grasped. Based on this principle, the three-dimensional coordinate calculation unit 15 executes the following expressions (2) to (4) to calculate the three-dimensional coordinates corresponding to each edge composing point.

[0055]

[Equation 2]

[0056]

[Equation 3]

[0057]

[Equation 4]

In the above equations, B is the base length of each camera, F is the focal length of the lens of each camera, H is the height data of the second camera 3b, and θ is the depression angle of the camera. Shown respectively.

When the three-dimensional coordinates corresponding to all the edge constituent points are calculated, the projection processing unit 17 sets a virtual vertical plane along the length direction of the road, and the three-dimensional coordinate calculation unit calculates the virtual vertical plane. 3D coordinates are projected on this virtual vertical plane.

FIG. 12 shows an example of the projection process. In the figure,
X, Y, and Z are spatial coordinate axes for defining each three-dimensional coordinate point, and the virtual vertical plane (indicated by R in the drawing) is
A horizontal axis H parallel to the Z axis at a position along the road, and Y
And a vertical axis V parallel to the axis.

On the virtual vertical plane R, points reflecting the height data (y coordinate) of each three-dimensional coordinate point and the depth data (z coordinate) when viewed from the front of the captured image are projected. To be done. Therefore, when the virtual vertical plane R is viewed from the front, a state in which each of the edge components is viewed from the side of the road is extracted as shown in FIG.

The lane data storage section 18 stores data (ie, x coordinate and z coordinate) indicating the positional relationship of each lane on the road viewed from the observation position. If you do the projection process,
Each three-dimensional coordinate can be projected for each lane. Further, instead of the lane data storage unit 18, the three-dimensional coordinate calculation unit 1
5 is arranged between the projection processing unit 17 and the projection processing unit 17, the three-dimensional coordinates are divided into groups based on the positional relationship in the road width direction, and the projection processing is performed for each group. May be.

FIG. 14 shows the model data storage section 19.
As shown in (1), (2) and (3), two-dimensional data MD1, MD2, MD3.
.. is stored. The comparison / determination unit 19 sequentially scans these side surface shape models (hereinafter, simply referred to as “models”) on the virtual vertical plane R to show what the projection point represented on the virtual vertical plane R is. To determine if.

It should be noted that the model of each vehicle is not limited to the one representing the entire side surface shape, but as shown in FIGS. 15 (1) to 15 (3), two-dimensional data ( The part shown by the solid line in the figure) may be used as a model.

FIG. 16 shows an example in which the projection result shown in FIG. 13 is subjected to comparison processing using the model MD1 of FIG. 14 (1). The comparison and determination unit 20 sequentially scans the model MD1 in the H-axis direction with the origin O of the virtual vertical plane R as an initial position, counts the number n of projection points included in the model MD1 at each scanning position, and calculates the projections. When the score n is equal to or larger than the predetermined threshold value TH, the following expression (5) is executed to evaluate the evaluation value E of the projection portion overlapping with the model MD1.
Calculate h _i .

In the equation (5), h _k and v _k indicate the coordinates of a point on the virtual vertical plane R, h _i is the H coordinate of the left end point of the model, LN is the length of the model in the H axis direction, V (h) represents a function representing the height of the model at the coordinate h (0 <h <LN). Further, S _i is represented by the equation (6) described later.

[0067]

[Equation 5]

[0068]

[Equation 6]

On the other hand, when the number n of projection points overlapping the model falls below the threshold value, the comparison and determination section 20 sets the maximum value MAX as the evaluation value at that scanning position. The comparison determination unit 20 scans the virtual vertical plane R with all the models to calculate the evaluation value at each scanning position, and projects the vehicle type of the model from which the minimum evaluation value is obtained from the obtained results. Then, the coordinate h ₀ (shown in FIG. 16) at the time when the minimum evaluation value is obtained is determined as the head position of the vehicle.

The above determination result is stored in the processing result storage unit 21. Thereafter, similar processing is executed every predetermined time, and the processing results are sequentially accumulated in the processing result storage unit 21. The processing result storage unit 21, as shown in FIG.
Each table has a table TB (hereinafter referred to as a “process result storage table TB”) for storing the head position (coordinate h _{0 in} FIG. 16) of each vehicle extracted for each lane, and each process time t ₁ , A numerical value indicating the head position of the extracted vehicle is sequentially stored for each time t ₂ , t ₃ .

The tracking processing unit 22 takes out the position of the vehicle detected at each processing time from the processing result storage table TB and associates it with any detected value at the processing time one step before. , Determine the position change of the vehicle over time. J at the current processing time t _i
It is assumed that one vehicle is detected, of which jth (1 ≦
When the head position of the detected vehicle in j ≦ J) was h _j (t _i), the tracking processing unit 22 in each detection result at the time of the next processing time t _{i + 1,} wherein h _j ( from among those whose difference with t _i ) is less than or equal to a predetermined threshold value, h
_A detection result that minimizes the distance from _j (t _i ) is extracted, and this is extracted as the head position h of the j-th vehicle at the processing time t _{i + 1} .
Recognize as _j (t _{i + 1} ).

Incidentally, in this associating process, data satisfying the above-mentioned condition is a predetermined detection line Det (FIG. 18 (1)).
The process ends when (2) is exceeded. Further, at the processing time t _{i + 1} , when there is no data satisfying the above conditions, the tracking processing unit 21 stops the subsequent associating processing.

FIGS. 18 (1) and 18 (2) show the position change of each vehicle determined based on the above-mentioned correspondence processing. In the figure, ×
The mark shows the detection results for each processing time.
The locus indicated by each broken line shows the corresponding detection result, that is, the position change of each vehicle.

The detection line Det corresponds to a predetermined horizontal line on the road, and the tracking processing unit 21 determines that each locus has crossed the detection line Det (in FIG. 18 (1),
At t ₁ , t ₂ , and t ₃ ), the passing vehicle is counted, and the detection value immediately after passing and the detection results of several times immediately before that are sampled, and the vehicle is detected using these detection results. Calculate the passing speed of.

The u-th locus (u = 1, 2, 3, ...
·) For sampling the N number of the detected values, coordinates (tu _n detected values sampled in these n-th (1 ≦ n ≦ N), when the hu _n), the passage of vehicles indicated by the locus The velocity V _u is expressed by the following equation (7).

[0076]

[Equation 7]

Incidentally, in the state where each vehicle is traveling smoothly, the locus shown in FIG. 18 (1) is obtained,
If you are waiting for a traffic light or a traffic jam,
As shown in (2), the result that the detected value at each processing time hardly changes is obtained.

According to the above method, there is no possibility of erroneously detecting planar data such as the shadow of a vehicle as a vehicle, and it is possible to perform highly accurate measurement processing by accurately discriminating the vehicle. Further, in addition to the speed of a moving vehicle, a vehicle or an object stopped on the road can be detected, so that it is possible to accurately grasp a traffic jam condition and to illegally park or extract an obstacle.

FIG. 19 shows another configuration example of the traffic flow measuring device. Also in this embodiment, the two cameras 3a, 3a,
Vehicles traveling in each lane on the road are discriminated from the images simultaneously captured by 3b, and the traffic flow is measured from the time transition of the discrimination result.
In addition to the above configuration, by adding a feature selection unit 30, a representative point extraction unit 31, a classification processing unit 32, a coordinate selection unit 33, a weighting range setting unit 34, etc. to the configuration of the control processing device 4 as processing, The time is reduced and highly accurate measurement processing is performed.

The feature selection unit 30 uses the edge image memory 12
This is for selecting a characteristic portion as a processing target candidate from each edge image stored in a and 12b. Here, a portion in which a predetermined number of edge constituent points are connected for each edge image is set as a characteristic portion. I try to take it out.

The representative point extracting section 30 extracts edge constituent points corresponding to the center points of the respective characteristic portions as one point representing the characteristic portions of the selected characteristic portions.
FIGS. 20 (1) and 20 (2) show an example in which the feature selection processing is applied to the edge images shown in FIGS. 8 (1) and 8 (2), and FIG. The result of extracting a representative point (indicated by X in the figure) in the image within the area surrounded by the broken line is shown. In the following processing, only these representative points are processed, so that the processing speed can be greatly improved.

The associating processor 14 executes the associating process between the respective edge images for each of the extracted representative points,
The three-dimensional coordinate calculation unit 15 calculates three-dimensional coordinates for each representative point by using this association result. Classification processing unit 32
Performs clustering processing using each x coordinate on each of the calculated three-dimensional coordinates, and groups each representative point into groups according to the processing result.

FIG. 22 shows an example of the calculation result of the three-dimensional coordinates for each representative point, in the space near the boundary between the lanes L1 and L2 and on the XZ plane of the first lane L1. A plurality of representative points are extracted.

FIG. 23 shows the result of grouping the representative points in FIG. 22. The representative points located near the boundary between the lanes L1 and L2 are in the first group G1 and on the xz plane. The representative points located at are the second group G2
Are classified into two categories.

When the grouping process is completed, the projection processing section 17 executes the projection process for each group with respect to the three-dimensional coordinates of the representative points included in each group. The coordinate selection unit 33 selects a representative point to be the final processing target from the projection processing results of each group. In this embodiment, the y-coordinate corresponding to the bumper position of the vehicle is used as the threshold value, and the coordinate selection unit 33 selects a height higher than the threshold value from the representative points projected on the virtual vertical plane R. A point having data is selected as a candidate point of the vehicle.

FIGS. 24 (1) and 24 (2) show the projection results of the groups G1 and G2 after the coordinate selection processing is performed. The projection points of the group G2 are removed as noise. There is. By the above processing, the projection points on the object located on the road plane, such as the shadow of the vehicle, are removed. The remaining projection points are weighted by the weighting range setting unit 34 as described below. A range setting process is performed, and thereafter, the comparison processing unit 20 sequentially executes a comparison process between each point included in the setting range and each model stored in the model data storage unit 19.

Details of the weighting range setting process and the comparison process will be described below with reference to FIGS. 25 to 29. As shown in FIG. 25, the weighting range setting unit 34 sets each projection point as a range corresponding to the width of the error generated at the time of measuring the three-dimensional coordinates, and sets the projection point in the direction based on the central camera parameter. The line segment WL expanded by the length is set.

Let the coordinates of the i-th projection point W _{i on} the HV plane be (h _i , v _i ), and y of the point corresponding to this projection point W _i on each input image obtained by the cameras 3a and 3b. If the coordinates are y _Ui and y _Li , respectively, the coordinates (h _{i +} , v _{i +} ), (h _i− , v of the end points W _i ⁺ , W _i ⁻ of the extended line segment WL with the projection point W _i as the center point are set. _i- ) is calculated by the following equations (8) to (11). In the formulas below, B, F, H, and θ represent the same camera parameters as used in the formulas (2) to (4). Also in each formula,
e indicates the actual length indicated by one pixel on the input image.

[0089]

[Equation 8]

[0090]

[Equation 9]

[0091]

[Equation 10]

[0092]

[Equation 11]

The comparison / determination unit 20 is the model data storage unit 1.
Each model M of 9 is sequentially read out, and as shown in FIG.
A range SR in which two or more projection points are included in the range of the width r of the model M is set as the scanning range of the model.

Thereafter, as shown in FIG. 27, the comparison / determination section 20 makes an intersection Z _i between the extension line segment WL and the model M for each scanning position h _i of the model M within the scanning range R.
With determination of the coordinates, the distance D _i between the intersection point Z _i and the projection point W _i, and the end point W _i which is located the intersection point Z _i side ⁺
(Or W _i ^-) and calculates the distance E _i to the projection point W _i. Further, the comparison / determination unit 20 applies the calculated ratio of D _i and E _{i to} the projection point W _i to the following equation (12), so that the model M actually exists at the position of this intersection Z _i. After calculating the probability sp _i (shown in FIG. 28) as the evaluation value of the projection point W _i , these evaluation values sp _i
The sum SP _i, is calculated as a goodness of fit with the model of the projection data in the scanning position h _i.

[0095]

[Equation 12]

In the above equation, ε means the base of the natural constant, and σ takes a value of 2 or 3.

According to the above method, the model M and each extension line segment WL are compared with each other by weighting according to the distance between each intersection Z _i and the projection point W _i. The error at the time of coordinate calculation can be removed. Note that this weighted comparison configuration can also be applied to the apparatus of the first embodiment.

FIG. 29 shows the result of the comparison processing in the scanning range SR. The comparison / determination unit 20 indicates that the head of the vehicle type corresponding to the model M is located at the scanning position h ₀ where the evaluation value SP _i is the maximum value. Judge that the part is located.

The determination result is stored in the processing result storage unit 21, and the tracking processing unit 22 executes processing such as temporal position change and speed measurement of each vehicle as in the first embodiment. .

In each of the above two embodiments, edge constituent points are extracted from the input images from the cameras 3a and 3b,
Based on the extraction result, the characteristic part on the image is projected on the virtual vertical plane along the road to identify the vehicle, but the image part such as the ceiling and the windshield, which is the characteristic of the vehicle at night, is detected. Under difficult conditions, it is effective to binarize each input image to extract the image of the vehicle light as a characteristic portion and measure the traffic flow using the extracted result.

FIG. 30 shows an example of the structure of a traffic flow measuring device corresponding to the processing at night. The control processing unit 4 includes the edge extracting unit 13 and the edge image memory 1 of the first embodiment.
Instead of 2a and 12b, a binarization processing unit 35 and binary image memories 36a and 36b are included as components.

FIG. 31 (1) (2) and FIG. 32 (1)
(2) shows examples of input images obtained by the cameras 3a and 3b, respectively. In all examples, the drawing with (1) shows the first input image stored in the first image memory 11a, and the drawing with (2) shows the second input image memory 1a.
2b shows a second input image stored in 1b.

The binarization processing unit 35 operates the image memory 11
The input images in a and 11b are subjected to processing for converting pixels whose brightness values exceed a predetermined threshold value into black pixels, and the resulting binary image is stored in the binary image memory 36.
a and 36b. FIGS. 33 (1) and (2) show images obtained by binarizing the input images shown in FIGS. 31 (1) and (2), and FIGS. 1)
The image obtained by binarizing each input image in (2) is
In each of the binary images, the light portion of the vehicle is extracted as a black pixel.

The associating processing unit 14 performs the associating process between the images for the black pixels in each binary image in the same manner as described above, and the three-dimensional coordinate calculating unit 15 carries out each associating process. The calculation process of the three-dimensional coordinates is executed for each pixel. As a result, the position of the light portion included in each input image is extracted and output to the comparison determination unit 20.

The model data storage unit 19 stores ordinary vehicles,
Data relating to the number of lights and installation positions for each vehicle such as a motorcycle and a large vehicle is stored, and the comparison / determination unit 20
By comparing the stored data with the calculation result by the three-dimensional coordinate calculating unit 15, it is possible to determine which kind of vehicle is located at which position.

FIG. 35 shows the calculation result of the three-dimensional coordinates using the binary image of FIGS. 33 (1) and 33 (2), and FIG.
The calculation results of the three-dimensional coordinates using the binary images of 4 (1) and (2) are shown respectively.

In the figure, the mark * indicates a point corresponding to the center point of each extracted feature portion (hereinafter referred to as "feature point"). The comparison / determination unit 20 sets the recognition range from the y-coordinate corresponding to the standard height position of the bumper to a predetermined height position, and sets the feature point whose y-coordinate value is within the recognition range to the vehicle light. It is extracted as the indicated feature point candidate. Furthermore, the comparison and determination unit 2
0 compares the extracted positional relationship of each characteristic point with the stored data in the model data storage unit 19 to determine the vehicle type and position of the vehicle on the road. In each figure, a dotted line portion indicates a lane boundary line based on the position information in the lane data storage unit 18, and it is possible to recognize in which lane the determined vehicle is located based on this position information. .

For example, a set of characteristic points located at a position directly above the bumper and parallel to the width direction of the road at an interval corresponding to the interval of lights of a passenger car (for example, in FIG. 35). Of the feature points o ₁ and o ₂ and the feature points o ₃ and o ₄ ) of the
Recognizes the positions of these feature points as the head position of the ordinary passenger car. If there is a single feature point (for example, feature point o _{5 in} FIG. 35) having height data corresponding to the position directly above the bumper, the comparison and determination unit 20 determines that the motorcycle is located at this feature point. To do. Further, when the interval of the feature points corresponding to the lights is larger than the interval of the lights of the ordinary passenger car and a large number of feature points are located near the feature point set (each feature point in FIG. 36). Is the comparison determination unit 2
0 determines that this feature point group indicates a large vehicle, and further determines the vehicle type (truck, van, bus, etc.) from the positional relationship of each feature point.

On the other hand, when the shadow of the light shifts on the road at the time of rainfall or the image of the night light or the sign along the road is extracted as the feature points, the y coordinates of these feature points are outside the recognition range. Therefore, it is determined that the feature point does not reflect the vehicle. As a result, there is no risk of erroneously detecting an object other than the vehicle as the vehicle, and the vehicle can be discriminated with high accuracy even at night.

The determination results of the comparison / determination unit 20 are sequentially stored in the processing result storage unit 21.
The same processing as in the first and second embodiments is executed.

Although all of the above-described embodiments are related to the traffic flow measuring device, the above-mentioned vehicle observation method can be applied not only to a running vehicle but also to a parked vehicle. .

FIG. 37 shows an example in which the present invention is applied to a parking lot observation device. This parking lot observation device is located in the vicinity of a parking lot consisting of a plurality of parking zones (two zones in the illustrated example),
The vehicle is used to determine the parking status, and the parking sections 40, 40 arranged at predetermined intervals are respectively 4 sections in the length direction (direction shown by arrow A in the figure) and 2 sections in the width direction. A total of eight parking areas AR are provided by dividing the areas.

In this embodiment, for each parking section 40, each parking area AR in the parking section 40 is observed, and the cameras 3a, An observation device composed of 3b and a control processing device 4'is provided.

In this embodiment, the cameras 3a and 3b are used.
Are attached to the lighting columns 41 in the same positional relationship as described above. Further, the measurement result by each control processing device 4'is transmitted to a computer of a management center (not shown) to grasp the usage status of the entire parking lot.

FIG. 38 shows the electrical configuration of the above observation device. As in the case of FIG. 5, the control processing device 4 ′ includes A / D conversion units 10a and 10b, image memories 11a and 11b, edge image memories 12a and 12b, an edge extraction unit 13, an association processing unit 14, and three-dimensional coordinate calculation. Unit 15, camera parameter storage unit 16, projection processing unit 17, model data storage unit 1
9. In addition to having a comparison and determination unit 20, each parking area AR
The area data storage unit 42 for storing the position information (for example, the coordinates of each vertex of the area AR) is included as a configuration.

When the input images as shown in FIGS. 39 (1) and (2) are stored in the image memories 11a and 11b, the edge extracting unit 13 determines the input images according to the first and second embodiments. The edge constituent points in the horizontal direction are extracted by a method similar to. As a result, edge images as shown in FIGS. 40 (1) and 40 (2) are generated, and the edge image memory 12
a and 12b.

Next, the correspondence processing unit 14 and the three-dimensional coordinate calculation unit 15 perform the correspondence processing between the images of the edge constituent points and the corresponding points by the same method as in each of the embodiments. A three-dimensional coordinate calculation process is performed.

The projection processing unit 17 sets a virtual vertical plane R'along the lengthwise direction A of the parking section, that is, the widthwise direction of each parking area AR, and calculates each calculated three-dimensional coordinate on this virtual vertical plane. Project on R '. It should be noted that this projection process is performed for each row of the parking section 40 based on the storage data in the area data storage unit 42.

FIG. 41 shows an example of the above-mentioned projection processing.
Shows a state in which the virtual vertical plane R ′ subjected to this projection process is viewed from the front, and projection data indicating the shape of the front or back of the vehicle in each parking area is separated and extracted.

FIG. 43 shows the model data storage section 19.
As shown in (1) and (2), a model indicating the front shape or the back shape of each vehicle is stored, and the comparison / determination unit 20 uses the same method as that of the first embodiment to calculate the virtual vertical direction. Each model is sequentially scanned on the plane R ′ to identify the vehicle. In this case, when the comparison / determination unit 20 determines the position where the vehicle corresponding to any of the models exists, the comparison / determination unit 20 further collates the stored data in the area data storage unit 42, and the determined position indicates which parking area. Finally, the information indicating which kind of vehicle is parked in which parking area is finally transmitted to the management center.

In the above embodiment, the virtual vertical plane R'is set in the width direction of the parking area to project the front shape of each vehicle. However, the present invention is not limited to this, and the length of the parking area is not limited to this. A virtual vertical plane may be set in the direction to project the side surface shape of each vehicle. Further, in the above-described embodiment, the projection processing is performed simultaneously for the parking areas arranged in the same row, but the projection processing may be performed individually for each parking area. Furthermore, in addition to this parking lot observation processing, the vehicle running on the passage around each parking section 40 is
According to the embodiment described above, it is possible to determine the vehicle that is about to park or the vehicle that leaves the parking lot, and the entire parking lot can be managed in an integrated manner.

The above parking lot observing device performs vehicle discrimination based on the projection processing of three-dimensional coordinates. Instead, it discriminates vehicles based on the height data indicated by the calculated three-dimensional coordinates. It may be configured as follows. Figure 44 shows
An example of a device configuration in this case is shown, and a control processing device 4 '
Has a parking presence / absence determining unit 45 for determining whether or not the vehicle is parked in each parking area based on the calculated three-dimensional coordinates and the position information of each parking area.
In addition, other than that, detailed description is omitted by assigning the same reference numerals to the same configurations as the respective embodiments.

The parking presence / absence determining section 45 divides the calculated three-dimensional coordinates into groups for each parking area based on the position information of each parking area AR stored in the area data storage section 42. For example, if the calculation results of the three-dimensional coordinates shown in FIG. 41 are divided into groups, the results shown in FIGS. 45 (1) and 45 (2) are obtained.

Further, the parking presence / absence determining unit 45 calculates the number PT _m of the three-dimensional coordinate points included in the group g _m and the average value AD _m of the height data of these coordinate points for the group g _m classified into the m-th place. Then, the calculated data P
By comparing each of T _m and AD _m with a predetermined threshold value, it is determined whether or not there is a vehicle in the parking area AR corresponding to this group g _m .

The number PT _m of the three-dimensional coordinate points is compared with a threshold value TH ₁ set to a value corresponding to the average number of edge constituent points of the vehicle. The height average value AD _m is 2
The threshold values TH ₂ and TH ₃ are compared with each other. Among them, the threshold value TH ₂ is a value corresponding to the average height of an ordinary passenger vehicle, and the threshold value TH ₃ is a value of a large vehicle. A value corresponding to the average height is set respectively.

In the group g _m , the PT _m exceeds the threshold TH ₁ and the AD _m is the threshold TH _2.
And between the threshold value TH ₃ and the parking presence / absence determining unit 4
5 determines that an ordinary passenger vehicle exists in the parking area AR corresponding to this group g _m . When PT _m exceeds the threshold value TH ₁ and AD _m takes a value equal to or greater than the threshold value TH _2, it is determined that a large vehicle exists in the parking area AR corresponding to this group g _m. It When PT _m has a value equal to or less than the threshold value TH ₁ or when AD _m has a value equal to or less than the threshold value TH _2, it is determined that no vehicle exists in the corresponding parking area.

Although each of the above-described embodiments discriminates the vehicle on the road or in the parking area, it is not limited to the vehicle and can discriminate various objects or change the position of the object with time. It goes without saying that the same method can be applied to the measurement of.

[0128]

According to the first and seventh aspects of the present invention, the characteristic portions of each image are extracted from the images obtained by picking up the observation positions by two or more image pickup devices, and the correspondence relation between the images of these characteristic portions is extracted. After extracting the three-dimensional coordinates of each feature part from, the extracted three-dimensional coordinates are projected on a virtual vertical plane that is perpendicular to the plane in contact with the object, and the projection result is model data of the object. The target object is discriminated by comparing with. As a result, the type, position, number, etc. of the object at the observation position can be accurately grasped, and the planar data such as the shadow of the object is not erroneously recognized as the object, so that the observation accuracy is greatly improved.

According to the inventions of claims 4 and 10, after the characteristic portions are respectively extracted from the images by the two or more image pickup devices in the same manner as described above, the points showing the representative characteristic of each characteristic portion are specified and specified. Since the projection processing and the comparison / discrimination processing similar to those described above are performed on the three-dimensional coordinates of the respective representative points, the processing speed can be significantly reduced.

In each of the inventions of claims 2, 5, 8 and 11, the extracted three-dimensional coordinates are divided into groups based on the positional relationship on the plane in contact with the object, and then each group is moved to the virtual vertical plane. Since the projection processing is performed, even when there are a plurality of objects, it is possible to individually determine them according to their positional relationship.

According to the inventions of claims 3, 6, 9 and 12, for each projection point on the virtual vertical plane, for each point within a predetermined range including the projection point, a weight based on the distance from the projection point is used. By adding the symbol and comparing with the model data, it is possible to remove the error that has occurred during the three-dimensional measurement and improve the discrimination accuracy of the object.

In the thirteenth aspect of the present invention, two or more image pickup devices are provided above the road to capture images obtained by picking up the observation position at predetermined time intervals, and by the same method as described above,
After extracting the three-dimensional coordinates of the characteristic part in each image, the extraction result is projected on a virtual vertical plane along the road, and the projection result is compared with the shape models of the side surfaces of a plurality of types of vehicles. The traffic flow can be accurately measured by discriminating the type, number and position of vehicles on the road. Further, according to the above-mentioned comparison processing, it is possible to accurately extract the vehicle even when the vehicles overlap due to traffic congestion. Furthermore, since this comparison process is performed on a two-dimensional plane, the hardware configuration can be reduced, the processing speed can be increased, and a highly accurate traffic flow measuring device can be provided at low cost.

In the fourteenth aspect of the present invention, after the characteristic portions are extracted from the images obtained from the two or more image pickup devices arranged above the road, the respective characteristic portions are associated with each other in the same manner as described above, and the three-dimensional measurement is performed. After executing, the vehicle on the road is discriminated by using the three-dimensional coordinates that satisfy the predetermined height condition. Therefore, it is possible to perform highly accurate traffic flow measurement based on the positional relationship of the light portion of the vehicle even at night. it can.

According to the fifteenth aspect of the present invention, two or more image pickup devices are arranged toward an area having a predetermined shape, and images are picked up at the same time.
For each of the obtained images, the 3
Dimensional coordinates are extracted, and each extracted three-dimensional coordinate is projected on a virtual vertical plane that is perpendicular to the installation surface of the vehicle in the area, and the projection result is compared with the shape models of various vehicles. Therefore, the vehicle in the area can be accurately identified. According to the sixteenth aspect of the present invention, whether or not a vehicle exists in the area can be easily determined by whether or not each of the extracted three-dimensional coordinates satisfies a predetermined height condition. Further, also in the inventions of claims 15 and 16, the hardware configuration can be reduced and the processing speed can be increased, and the conventional problems in the parking lot measurement can be solved at once.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing an installation example of a traffic flow measuring apparatus according to a first embodiment of the present invention.

FIG. 2 is a perspective view showing an installation example of two cameras.

FIG. 3 is a perspective view showing another installation example of the camera.

FIG. 4 is a perspective view showing an example in which a camera is integrated.

FIG. 5 is a block diagram showing an electrical configuration of a traffic flow measuring device.

FIG. 6 is an explanatory diagram showing an example of an input image from each camera.

FIG. 7 is an explanatory diagram showing a Laplacian filter.

FIG. 8 is an explanatory diagram illustrating an example of extracting an edge image.

FIG. 9 is an explanatory diagram showing a method of association processing.

FIG. 10 is an explanatory diagram exemplifying a result of a dissimilarity calculation for association processing.

FIG. 11 is an explanatory diagram showing the principle of triangulation measurement.

FIG. 12 is an explanatory diagram showing a projection method of three-dimensional coordinates.

FIG. 13 is an explanatory diagram illustrating the result of projection onto a virtual vertical plane.

FIG. 14 is an explanatory diagram showing a side surface shape model.

FIG. 15 is an explanatory diagram showing another setting example of the model.

FIG. 16 is an explanatory diagram showing a comparison determination processing result.

FIG. 17 is an explanatory diagram showing a data structure of a processing result storage unit.

FIG. 18 is an explanatory diagram showing a result of associating detected values of vehicle positions at respective processing points.

FIG. 19 is a block diagram showing a configuration of a traffic flow measuring device according to a second embodiment.

FIG. 20 is an explanatory diagram showing a result of feature selection.

FIG. 21 is an explanatory diagram showing a result of extracting representative points.

FIG. 22 is an explanatory diagram showing a calculation result of three-dimensional coordinates for a representative point.

23 is an explanatory diagram showing a result of a classification process on each three-dimensional coordinate in FIG. 22.

FIG. 24 is an explanatory diagram showing a result of projecting three-dimensional coordinates for each group.

FIG. 25 is an explanatory diagram showing a method of setting a weighting range.

FIG. 26 is an explanatory diagram showing a scanning range of a model.

FIG. 27 is an explanatory diagram showing the principle of a weighted comparison processing method.

FIG. 28 is an explanatory diagram showing the concept of the evaluation value of each projection point.

FIG. 29 is an explanatory diagram showing a result of comparison / determination processing.

FIG. 30 is a block diagram showing a configuration of a traffic flow measuring device according to a third embodiment.

FIG. 31 is an explanatory diagram showing an example of an input image from each camera.

FIG. 32 is an explanatory diagram showing an example of an input image from each camera.

FIG. 33 is an explanatory diagram showing a result of binarizing each input image of FIG. 31.

34 is an explanatory diagram showing a result of binarizing each input image of FIG. 32. FIG.

FIG. 35 is an explanatory diagram showing a result of calculating three-dimensional coordinates of feature points.

FIG. 36 is an explanatory diagram showing a result of calculating three-dimensional coordinates of feature points.

FIG. 37 is an explanatory diagram showing an installation example of the parking lot observation device according to the fourth embodiment.

FIG. 38 is a block diagram showing a configuration of a parking lot observation device.

FIG. 39 is an explanatory diagram showing an example of an input image from each camera.

FIG. 40 is an explanatory diagram illustrating an example of extracting an edge image.

FIG. 41 is an explanatory diagram showing a relationship between a calculation result of three-dimensional coordinates and projection processing.

FIG. 42 is an explanatory diagram showing a projection processing result.

FIG. 43 is an explanatory diagram showing the shape of a model.

FIG. 44 is a block diagram showing a configuration of a parking lot observation device according to a fifth embodiment.

FIG. 45 is an explanatory diagram showing a result of grouping three-dimensional coordinates.

[Explanation of symbols]

3a, 3b cameras 4 Control processor 13 Edge extractor 14 Correlation processing unit 15 3D coordinate production tool 17 Projection processing unit 18 Model data storage 20 Comparison judgment section 31 Representative point extractor 34 Weighting range setting section 35 Binarization processing unit 45 Parking presence / absence determination unit

Front page continued (58) Fields surveyed (Int.Cl. ⁷ , DB name) G01B 11/00-11/30 G06T 1/00 G06T 7/00 G08G 1/00-9/02 H04N 7/18

Claims (16)

(57) [Claims] 1. A method for observing a target object, wherein two or more imaging devices are simultaneously imaged toward a predetermined observation position, and a characteristic portion in each obtained image is extracted for each image. Steps, a second step of associating the extracted feature portions of each image between the images, and three-dimensional measurement of each associated feature portion to extract three-dimensional coordinates of each feature portion. And a fourth step of projecting the extracted three-dimensional coordinates of each feature part onto a virtual vertical plane that is positioned perpendicular to the plane in contact with the object, and this projection data An object observing method, characterized in that a fifth step of discriminating the object by comparing it with three-dimensional model data is carried out in series. 2. The projection process of the fourth step is performed by grouping the three-dimensional coordinates of each characteristic portion into groups based on the positional relationship on the plane with which the object is in contact, and then including each group within the group. The object observing method according to claim 1, wherein the object observing method is a process of projecting the three-dimensional coordinates described above onto the virtual vertical plane. 3. The comparison process of the fifth step sets, for each projection point generated on the virtual vertical plane by the projection process of the fourth step, a predetermined range including the projection point. 3. The object observing method according to claim 1, wherein the object observing method is a process of weighting each point within this range based on a positional relationship with the projection point and comparing the weight with the two-dimensional model data. . 4. A method of observing an object, wherein first and second imaging devices are simultaneously imaged at predetermined observation positions, and a characteristic part in each obtained image is extracted for each image. A second step of identifying a point indicating a representative feature of each of the extracted characteristic portions of each image, and a third step of associating the identified representative points between images And the fourth step of performing three-dimensional measurement for each associated representative point to extract the three-dimensional coordinates of each representative point, and Fifth projection onto an imaginary vertical plane that is perpendicular to the tangent plane
And a sixth step of discriminating the target object by comparing the projection data with predetermined two-dimensional model data. 5. The projection process of the fifth step, after grouping the three-dimensional coordinates of each representative point on the basis of the positional relationship on the plane in contact with the object, includes each group within the group. The object observing method according to claim 4, which is a process of projecting the three-dimensional coordinates described above onto the virtual vertical plane. 6. The comparing process of the sixth step sets, for each projection point generated on the virtual vertical plane by the projection process of the fifth step, a predetermined range including the projection point. The object observing method according to claim 4 or 5, wherein each point within this range is weighted based on the positional relationship with the projection point and compared with the two-dimensional model data. . 7. An apparatus for observing an object, characterized in that two or more imaging devices arranged toward an observation position, and feature extraction for extracting a characteristic part from each image simultaneously captured by each imaging device. Means, associating means for associating the characteristic portions of each image extracted by the characteristic extracting means between the images, and three-dimensional measurement for each of the characteristic portions associated by the associating means. Coordinate extraction means for extracting the three-dimensional coordinates of the characteristic portion, storage means for storing the two-dimensional model data of the object, and a virtual vertical plane are set so as to be positioned perpendicular to the plane in contact with the object. Projection means for projecting the three-dimensional coordinates of each characteristic portion extracted by the coordinate extraction means on a virtual vertical plane, and a projection result by the projection means is stored in the storage means. Object observation device comprising a comparison means for comparing the two-dimensional model data. 8. The projection means divides the three-dimensional coordinates of each characteristic portion extracted by the coordinate extraction means into groups based on a positional relationship on a plane with which the object is in contact, and then groups each group. The object observation device according to claim 7, wherein the three-dimensional coordinates included in the image are projected on the virtual vertical plane. 9. The comparing means sets, for each projection point generated on the virtual vertical plane by the projection processing by the projection means, a predetermined range including the projection point, and each point within this range. 9. The object observation apparatus according to claim 7, wherein a weight is added based on a positional relationship with respect to the projection point and the comparison is performed with the two-dimensional model data. 10. A device for observing an object, characterized in that two or more image pickup devices arranged toward an observation position, and feature extraction for extracting a characteristic part from each image simultaneously captured by each image pickup device. Means, a representative point specifying means for specifying a characteristic point of each image extracted by the characteristic extracting means, and an image for each representative point specified by the representative point specifying means. 3 for each of the representative points associated by the associating means and the associating means
Coordinate extraction means for performing three-dimensional measurement to extract three-dimensional coordinates of each representative point, storage means for storing two-dimensional model data of the object, and virtual vertical plane positioned perpendicular to the plane in contact with the object A projection means for projecting the three-dimensional coordinates of each representative point extracted by the coordinate extraction means on the virtual vertical plane; and two-dimensional model data in which the projection result by the projection means is stored in the storage means. An object observation apparatus comprising: a comparison means for comparing. 11. The projection means divides the three-dimensional coordinates of each representative point extracted by the coordinate extraction means into groups based on the positional relationship on the plane with which the object is in contact, and then groups each group. The object observation device according to claim 10, wherein the three-dimensional coordinates included in the image are projected onto the virtual vertical plane. 12. The comparison means sets, for each projection point generated on the virtual vertical plane by the projection processing by the projection means, a predetermined range including the projection point, and each point within this range. The object observing device according to claim 10 or 11, wherein a weight based on a positional relationship with the projection point is added and the comparison is performed with the two-dimensional model data. 13. A device for observing a flow of a vehicle on a road and measuring traffic flow data on the road based on a temporal transition of the observation result, which is directed toward an observation position on the road above the road. Two or more image pickup devices provided, feature extraction means for extracting feature portions from each image simultaneously captured by each image pickup device at a predetermined time, and feature portions of each image extracted by the feature extraction means With respect to each image, coordinate extraction means for performing three-dimensional measurement on each characteristic portion associated with the association means, and extracting three-dimensional coordinates of each characteristic portion, An imaginary vertical plane is set along the virtual vertical plane, and 3 of each of the characteristic portions extracted by the coordinate extracting means on the imaginary vertical plane.
By projecting the dimensional coordinates, storing means for storing the shape model of the side surface of each vehicle for a plurality of types of vehicles, and comparing the projection result by the projecting means with each model stored in the storing means. A traffic flow measuring device comprising: a vehicle discrimination means for discriminating a vehicle on the road; and an observation means for observing the flow of the vehicle on the road by sequentially tracing the discrimination results by the vehicle discrimination means. 14. A device for observing a flow of a vehicle on a road and measuring traffic flow data on the road based on a temporal transition of the observation result, which is directed toward an observation position on the road above the road. Two or more image pickup devices provided, feature extraction means for extracting feature portions from each image simultaneously captured by each image pickup device at a predetermined time, and feature portions of each image extracted by the feature extraction means And a coordinate extraction unit that performs three-dimensional measurement on each characteristic portion associated by the association unit, and extracts three-dimensional coordinates of each characteristic portion, Vehicle discriminating means for discriminating the vehicle on the road by using the relative positional relationship of the three-dimensional coordinates satisfying a predetermined height among the three-dimensional coordinates of the respective characteristic parts extracted by the extracting means. , Traffic flow measurement device comprising a observation means for observing the flow of vehicles in the vehicle discriminating means by the discrimination result successively tracking to the road. 15. A device for observing a vehicle parked in an area of a predetermined shape, comprising two or more image pickup devices arranged toward the area above the area, and captured by the respective image pickup devices at the same time. The feature extracting means for extracting the characteristic part from each image, the associating means for associating the characteristic parts of the images extracted by the feature extracting means between the images, and the associating means. The coordinate extraction means for performing three-dimensional measurement of each characteristic portion to extract the three-dimensional coordinates of each characteristic portion, and the virtual vertical plane positioned perpendicular to the installation surface of the vehicle in the area are set, and the virtual vertical plane Projection means for projecting the three-dimensional coordinates of each characteristic portion extracted by the coordinate extraction means on a plane, and storage means for storing a shape model of each vehicle for a plurality of types of vehicles And a discriminating means for discriminating a vehicle in the area by comparing a projection result by the projecting means with each model stored in the storing means. 16. A device for observing a vehicle parked in an area of a predetermined shape, comprising two or more image pickup devices arranged toward the area above the area, and captured by the respective image pickup devices at the same time. The feature extracting means for extracting the characteristic part from each image, the associating means for associating the characteristic parts of the images extracted by the feature extracting means between the images, and the associating means. And a coordinate extracting means for performing three-dimensional measurement on each characteristic portion to extract the three-dimensional coordinates of each characteristic portion, and a predetermined height condition among the three-dimensional coordinates of each characteristic portion extracted by the coordinate extracting means. A parking lot observing device comprising: a discriminating means for discriminating a vehicle in the area using the three-dimensional coordinates that are satisfied.