JPH1055445A - Vehicle monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a system for estimating the image pickup position of a driver from license plate position information by one camera. SOLUTION: Image input/storage means 2 and 3 A/D-converts an image pickup video signal 11 which is inputted from the camera 1 for picking-up the image of a passage vehicle from a sufficiently separated upper part at every prescribed image pickup cycle time and stores it. A license plate detecting means 4 detects a license plate image and generates license plate position information 13 against the image pickup digital images 12 of the image input/storage means 2 and 3. A driver position is estimated based on windshield position information which is estimated by a driver position estimating means 5 from license plate position information 13. A display means 6 executes partial enlargement monitor display as against the estimated driver position image 1 and an image storing means 7 stores it.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle monitoring apparatus which contributes to security improvement by monitoring vehicles such as entrances and exits of factories.

[0002]

2. Description of the Related Art For example, a conventional vehicle monitoring device disclosed in Japanese Patent Laid-Open No. 4-152497 has a security camera 1a, a license plate camera 1b, and a vehicle sensor 1c as shown in FIG.
Captures an image of the front of the vehicle including the driving vehicle and the vehicle license plate, and detects the vehicle. The image input means 2a receives the vehicle detection timing signal 11c from the vehicle sensor 1c,
A / D conversion is performed on the video signals 11a and 11b input from the security camera 1a and the license plate camera 1b. The image storage unit 3a stores the vehicle / license plate imaged digital image signal 12a from the image input unit 2a. The license plate detection unit 4a cuts out the license plate image 13 from the license plate captured digital image 12b from the image storage unit 3a. The license plate character recognizing means 8 includes a license plate detecting means 4.
Then, the individual characters are cut out from the license plate image 13 from the image data and recognized, and a character code is output as a recognition result 15. The license plate database search means 9 searches the license plate database 10 in accordance with the recognition result 15 from the license plate character recognition means 8 and issues an alarm when matching with the registered vehicle, and at the same time, passes the vehicle passing time and license plate information. 16 and a digital image 12 captured from the front of the vehicle including the driver stored in the image storage means 3a.
A monitor is displayed on the display means 6a for c. The image storage unit 7a stores the digital image 12c of the front of the vehicle including the driver stored in the image storage unit 3a in association with the recognition result 17 from the license plate database search unit 9.

[0003] The above-mentioned conventional vehicle monitoring device employs a system in which a front image of a vehicle including a driver and a license plate are simultaneously imaged.

[0004]

The above-mentioned conventional vehicle monitoring system employs a system in which the front of the vehicle including the driving vehicle and the license plate are simultaneously imaged. Needs to be spread over the entire front surface of the vehicle, and the driver cannot be identified because of the unclear face image due to the reduction in resolution. On the other hand, in order to clearly capture the face image of the driver, the front of the vehicle including the driver and the license plate may be captured by separate cameras, or the memory capacity for storing the captured image may increase and the number of imaging pixels may increase. There is a problem that a high-resolution camera with many is required.

The problem to be solved by the present invention is to use a method of estimating the image pickup position of a driver from license plate position information with one camera without using an expensive high-resolution camera in a vehicle monitoring device (driver image pickup). Position estimation method).

[0006]

The vehicle monitoring apparatus according to the present invention is characterized in that the following means are provided to solve the above-mentioned problems, and employs a driver imaging position estimation method.

The camera is installed so as to look down from the front of the vehicle from a sufficient distance above, and captures an image of the vehicle so as to move from the top to the bottom of the imaging range as the vehicle approaches / passes every predetermined imaging cycle time. I do. Alternatively, a swing operation capable of changing an imaging range is enabled.

[0008] The image input / storage means performs A / D conversion of an imaged video signal input from the camera, and stores it.

The license plate detecting means detects a license plate image from the captured digital image in the image storage means and generates license plate position information. Alternatively, the type of the passing vehicle is determined based on the license plate in the detected license plate image, and the vehicle type information is also output.

The windshield detecting means detects a windshield image from the captured digital image in the image storage means and generates windshield position information.

[0011] The driver position estimating means may include: license plate position information from the license plate detecting means or windshield position information estimated from the license plate position information and the vehicle type information; or windshield position information from the windshield detecting means; The driver position is estimated based on both the former and the latter. Alternatively, the lighting flag is set according to the presence / absence of the windshield in the captured image.
Alternatively, the windshield left / right end position information estimated from the license plate left / right end position information as license plate position information from the license plate detecting means is also output.

The lighting means determines the necessity / non-necessity of lighting in accordance with the lighting flag from the driver position estimating means, and turns on / off the lighting device.

The camera control means controls the camera's swing operation in accordance with the windshield left / right end position information from the driver position estimating means.

The display means and the image storage means display and store a partially enlarged monitor on the driver position image from the driver position estimation means.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A vehicle monitoring apparatus according to one embodiment of the present invention is installed as shown in FIG. 1, and a camera 1 is installed so that the front of the vehicle can be looked down sufficiently from above as shown in FIG. In addition, the vehicle moving at a constant speed is made to appear to move at a substantially constant speed even in the captured image. Fig. 2 (b) as the vehicle approaches and passes
(I) only the road surface before entering the imaging range A as ~ (v), so that an image is obtained moves from top to bottom of the image with passage, in each predetermined imaging cycle time T _s The vehicle is imaged. The image input unit 2 performs A / D conversion of the captured video signal 11 input from the camera 1. The image storage unit 3 stores the captured digital image signal 12 from the image input unit 2. The license plate detecting means 4
From the captured digital image 12 of the image storage means 3, for example, a license plate image is obtained by a method disclosed in a known document (Kato et al .: License Plate Recognition Technology, Mitsubishi Electric Technical Report, vol. 62, no. 2, 1988). Is detected, and Y coordinate P
_bottom (see FIG. 2C) / license plate position information 13 of the undetectable code is generated. The driver position estimating means 5 estimates the driver position based on the windshield position information estimated from the license plate position information 13 from the license plate detecting means 4. The display unit 6 and the image storage unit 7 display and store a partially enlarged monitor display of the estimated driver position image 14 from the driver position estimation unit 5. Even if the windshield and the license plate are not included in the captured image at the same time, the image including the windshield as the driver position can be estimated, and the driver can be efficiently imaged with one camera. Further, by partially enlarging and displaying the driver's position, the driver's confirmation by the supervisor can be easily performed, and the driver's image can be used for post-searching and the like. Also, saving only the driver position image can save memory capacity.

The vehicle monitoring apparatus according to the above-described embodiment does not use an expensive high-resolution camera, and estimates the image pickup position of the driver from license plate position information with one camera (driver image pickup position estimation method). Take.

As shown in FIG. 3, the driver position estimating means 5 first initializes the license plate upper end Y coordinate P _top (see FIG. 2C) and the license plate detection flag to α (a sufficiently large negative value) and 0. Thereafter, the license plate position of the license plate detecting means 4 for detecting a license plate image from within the captured digital image 12 of the image storage means 3 for performing A / D conversion of the captured video signal 11 by the image input means 2 from the camera 1 and storing it. The presence or absence of a license plate in the captured image is determined based on the information 13 (procedures S1 to S5 in FIG. 3). If there is a license plate in the captured image, first, the license plate lower end Y coordinate P
_{From the bottom} and, for example, the distance C1 / C2 obtained by adding a fixed margin to the average value of the distance between the lower end of the license plate of the general vehicle and the upper / lower coordinates of the windshield,
The lower end Y coordinate f _top / f _bottom (see FIG. 2 (c))
_The estimation is performed as follows: _top = P _bottom −C1, f _bottom = P _bottom −C2 (step S6 in FIG. 3). Next, the presence / absence of license plate detection at the time of the previous imaging is determined, and when no (number plate detection flag = 0), it indicates that the vehicle has entered the imaging range A for the first time, so the license plate detection flag is set to 1 do. When present (license plate detection flag = 1), it indicates that the same vehicle license plate has been continuously imaged. Therefore, the difference value between the _bottom Y coordinate P _bottom of the license plate detected last time and the current time detected at the time of imaging is determined by the imaging cycle time T _s. It is calculated as the vehicle movement distance Δd in the image between (steps S7 to S9 in FIG. 3). If there is no license plate in the captured image, the license plate detection flag is first cleared to 0 (step S10 in FIG. 3). Next, since the license plate has moved out of the imaging range, the upper / lower windshield Y is calculated from the calculated difference value Δd.
Coordinates f _top / f _bottom are calculated as f _top = f _top + Δd, f
_Bottom = f _bottom + Δd is re-estimated (step S11 in FIG. 3). Further, it is determined whether or not the windshield including the driver is completely within the captured image, that is, whether or not _ftop ≧ 0 and the height of the captured image (see FIG. 2C) h> _fbottom. Is present in the captured image, the captured image at the driver position having a display range from f _top to f _bottom is enlarged and displayed on the monitor by the display means 6 and stored by the image storage means 7. . After the above procedure is continuously repeated for each imaging cycle time T _s. Even when the windshield is not present in the captured image,
The above procedure is repeated (the above steps S12 to S14 in FIG. 3).

The display means 6 and the image storage means 7 may combine and display and store an image acquisition date / time / place, a license plate image and the like together with the above-mentioned driver image. It goes without saying that the image may be compressed and stored.

In the embodiment of the present invention shown in FIG. 1, instead of the license plate detecting means 4 and the driver position estimating means 5, as shown in FIG. A windshield detecting means 4a for detecting a glass image and generating windshield position information 13a of the area inside the boundary line, and a driver position for estimating a driver position based on the windshield position information from the windshield detecting means 4a. You may comprise as estimating means 5a. The driver's position can be specified accurately without depending on the position of the license plate of the vehicle. As shown in FIG. 5, the windshield detecting means 4a first performs a differential operation on all the pixels of the captured digital image 12 of the image storage means 3 for performing A / D conversion of the captured video signal 11 by the image input means 2 from the camera 1 and storing the same. subjecting, for example, using the spatial second derivative filter (3 × 3) shown in FIG. 6 (a), Sekiwachi P of the mask value corresponding to the processing target pixel P and the neighboring pixel P _a ~P _h '= |
_{P b + P d + P f} + P h -4 · P | creates a differential image and the differential value of the target pixel to be processed P, and edge detection (steps S2 and S3 and S21 in FIG. 5). Next, the differential image is binarized with a predetermined threshold value so that, for example, pixels equal to or greater than / less than a predetermined threshold value are set to 1 (black point) / 0 (white point),
For example, as shown in FIG. 6B, a differential binary image from which contour components of a vehicle, a windshield, a light, and the like are obtained is created.
Unnecessary image information such as a road surface is deleted (step S2 in FIG. 5).
2). In addition, for the contour components in the differential binary image, for example, a well-known document (Tamura et al .: Introduction to Computer Image Processing,
The contour is traced by the boundary line tracking method disclosed in Soken Shuppan, 1985), and a series of closed coordinate points corresponding to the boundary portion of the vehicle, windshield, light, etc. is extracted as the boundary line (procedure S23 in FIG. 5). ). Further, whether the windshield is present in the captured image, that is, both the boundary line area extracted from the captured image and the characteristic values of the general windshield area obtained in advance, for example, the width W _i and the height H _i and the area S _i, and the difference between the width W _std , the height H _std and the area S _std are | W _std −W _i | <W _th , | H _std
−H _i | <H _th , | S _std −S _i | <S _th (i = 1,
2,..., N), the predetermined threshold values W _th and H _th , respectively.
Determines whether less than S _th and (procedure of FIG. 5 S2
4). When the windshield is not present in the captured image, the above procedure is repeated. When the windshield is present, for example, as shown in FIG.
In the differential image of the above, since the area of the light portion boundary line region is sufficiently smaller than the windshield and the height of the vehicle portion boundary line region is sufficiently large, the difference value becomes equal to or larger than a predetermined threshold value, and the region is determined to be not a windshield. . Since the difference value is less than the predetermined threshold value, the boundary inner area is generated as windshield position information 13a (step S25 in FIG. 5). Although the binarization processing of the differential image shown in the procedure S22 in FIG. 5 has been described as using a predetermined threshold value, for example, an average linear expression method for obtaining a threshold by applying a linear expression to the average density of an image, a method of calculating an image density histogram It goes without saying that a mode method using the density value of the valley of the two peaks as a threshold value and various other proposed threshold value determination methods may be used. As shown in FIG. 5, the driver position estimating means 5a determines whether the windshield is completely in the captured image based on the windshield position information 13a of the windshield detecting means 4a, that is, f _bottom
It is determined whether or not −H _f ≧ 0 and f _top + H _f <h (H _f represents, for example, a distance obtained by adding a certain margin to the average value of the height of the windshield of a general vehicle), and is determined in the captured image. At this time, the enlarged and displayed image of the driver's position is displayed in the same manner as in steps S13 and S14 in FIG. After the above procedure is continuously repeated for each imaging cycle time T _s. The above procedure is repeated even when the image is not in the captured image (the above-described procedure S2 in FIG.
6, S13 and S14).

In the embodiment of the invention shown in FIG. 1, the driver position estimating means 5 has been described as using only the license plate position information 13 from the license plate detecting means 4, but as shown in FIG. Windshield position information 13 from windshield detecting means 4a shown in FIG.
driver position estimating means 5b for estimating the driver position using a
It may be constituted as. The driver position is estimated by integrating the license plate and the windshield position information 13 and 13a, and the driver position can be specified with one camera with high accuracy.
As shown in FIG. 8, the driver position estimating means 5b determines whether the windshield including the driver is completely in the captured image in the same manner as in steps S1 to S12 in FIG. S1 to S12). If the windshield does not exist in the captured image, the above procedure is repeated. If the windshield exists, the windshield is included in the captured image based on the windshield position information 13a from the windshield detection means 4a in the same manner as in step S26 in FIG. , That is, whether f _bottom −H _f ≧ 0 and f _top + H _f <h (steps S31 and S32 in FIG. 8). When it is within the captured image, the enlarged display and accumulation of the driver position captured image are performed in the same manner as in steps S13 and S14 in FIG.
Steps S13 and S14). After the above procedure is continuously repeated for each imaging cycle time T _s.

In the embodiment of the invention shown in FIG. 1 or FIG. 7, the license plate detecting means 4 and the driver position estimating means 5 or 5b are used in the detected license plate image as in steps S41 and S42 in FIG. Considering the vehicle type information, which determines and outputs whether the license plate height H (see FIG. 10A) is within the predetermined upper / lower limit value for large and medium-sized passing vehicles, for example, general Distance C1 _large and C1 _middle / C2 _large and C2 _middle (Fig. 1) with a certain margin added to the average of the distance between the lower edge of the license plate and the coordinates of the upper and lower edges of the windshield of large and medium-sized vehicles.
0 (b)), f _top = P _bottom -C1 _large or C1 _middle , f _bottom = P _bottom -C2 _large or C
_Windshield top / bottom Y coordinate f _top like 2 _middle
/ F _bottom may be estimated. Using the fact that different license plate heights H are defined for large and small passing vehicles (H = 220 and 165 mm), the driver's position can be specified accurately.

In the embodiment of the invention shown in FIG. 7 (or FIG. 1), the driver position estimating means 5c as shown in FIG.
Is determined in accordance with the lighting flag 15 from step S15. When the lighting flag 15 is 1/0, the lighting device is turned on / off (steps S51 to S53 in FIG. 12). The estimating means 5b performs steps S54 and S54 in FIG.
As in 55, it is determined whether the windshield including the driver is completely in the captured image in the same manner as in steps S1 to S12 in FIG. 8 described above, and the lighting flag is determined based on the presence / absence of the windshield in the captured image. It may be configured as a driver position estimating unit 5c that sets 15 to 1/0. By illuminating only while the passing vehicle exists in the captured image, a clear driver image can be obtained. Further, the life of the lighting device can be prolonged, and power consumption for lighting can be saved.

In the embodiment of the invention shown in FIG. 7 (or FIG. 1), the driver position estimating means 5d as shown in FIG.
Windshield left / right end X coordinate f _left / f _right
A camera control unit 9 for controlling the head swing operation of the camera 1a according to the embodiment 16 is provided separately. The camera 1a enables the head swing operation capable of changing the imaging range. The driver position estimating unit 5b uses the procedure S63 in FIG. The license plate position information 13 from the license plate detecting means 4 as shown in FIG.
License plate left / right end X coordinate P _left / P
C3 / C which is obtained by adding a fixed margin to the _right and, for example, the average value of the distance between the left / right coordinates of the license plate of a general vehicle.
4 and the actual windshield left / right end X coordinate f _left
/ F _right 16 also f _left = P _left -C3, f _right = P
_It may be configured as a driver position estimating means 5d that estimates and outputs like _right- C4. Even if the windshield area of the passing vehicle protrudes from the imaging range of the camera, the license plate or windshield detection means 4 or 4a
The windshield area existing based on the output result of is obtained, the camera is swung so that the windshield area falls within the imaging range, and good driving is performed irrespective of the camera installation conditions and the width of the road on which the vehicle passes. Image is obtained. As shown in FIG. 14, the camera control means 9 first initializes a separately determined camera swing amount (pixel) L to 0, and then θ = ta
camera swing angle 17 calculated as n ^-1 (L / D) (D represents a predetermined camera installation location and inter-vehicle distance)
When no windshield or license plate is detected, the swing camera 1 is used as the swing angle initialization signal 17 so as to return the camera imaging range to the standard position.
a (steps S61 and S62 in FIG. 14). Next, of the windshield left / right end X coordinates f _left / f _right 16 from the driver position estimating means 5d, L = f _left when f _left <0, and L when f _right ≧ captured image width (pixel) W
= F _right -W, otherwise calculated as L = 0 (procedure S64 in FIG. 14). For the camera swing amount L for changing the imaging range right / left according to the positive / negative of the value, the windshield is It does not exist within the imaging range (procedure S in FIG. 14)
When the process branches to step S62 in steps S12 and S32), the calculated swing amount L is used. Further, when the windshield is present in the imaging range, the actual windshield left / right end X estimated as in the following equations (1) and (2) based on the windshield position information 13a from the windshield detection means 4a. Of the coordinates f ' _left / f' _right , f ' _left
<0, L = f ' _left , f' _right ≥W, L =
f ′ _right −W, otherwise, the camera swing amount L calculated again as L = 0 is used (procedure S65 in FIG. 14). f ′ _left = Min (f _right −W _f , f _left ) (1) f ′ _right = Max (f _left + W _f , f _right ) (2) where W _f is, for example, Indicates the average distance of the width of the windshield of a general vehicle.

[0024]

As described above, the vehicle monitoring apparatus of the present invention employs a method of estimating the image pickup position of the driver from license plate position information with one camera without using an expensive high-resolution camera. The following effects are obtained for each invention as compared with the conventional method of simultaneously imaging the front of the vehicle including the driver and the license plate. (1) Even if the windshield and the license plate are not included in the captured image at the same time, the image including the windshield as the driver position can be specified, and the driver can be efficiently imaged with one camera. (2) The driver's position can be specified accurately without depending on the license plate position of the vehicle. (3) The driver position is estimated by integrating the license plate and windshield position information, and the driver position can be accurately specified by one camera. (4) It is possible to accurately identify the driver's position by using different license plate format information for large and medium-sized passing vehicles. (5) By illuminating only while the passing vehicle exists in the captured image, a clear driver image can be obtained. Further, the life of the lighting device can be prolonged, and power consumption for lighting can be saved. (6) Even if the windshield area of the passing vehicle protrudes from the imaging range of the camera, an existing windshield area is obtained based on the output result of the license plate or the windshield detection means, and the windshield area is included in the imaging range. The camera is swung so as to enter, and a good driving vehicle image can be obtained without being affected by the installation conditions of the camera and the width of the road on which the vehicle passes. (7) By partially enlarging and displaying the driver's position, the driver's confirmation by the supervisor can be easily performed, and the driver's image can be used for post-search. In addition, by saving only the driver position image, the required memory capacity can be saved.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a vehicle monitoring device according to an embodiment of the present invention.

FIG. 2 is a view for explaining image pickup installation conditions of the camera shown in FIG. 1, an image picked up under the conditions, and a positional relationship between a license plate and a windshield of a passing vehicle.

FIG. 3 is a flowchart illustrating functions of a driver position estimating unit shown in FIG. 1;

FIG. 4 is a functional block diagram showing another embodiment of the present invention.

FIG. 5 is a flowchart for explaining the function of the windshield detecting means shown in FIG.

FIG. 6 is a view for explaining an edge detection method using a spatial secondary differential filter and an example of a differential binary image of a vehicle image.

FIG. 7 is a functional block diagram showing another embodiment of the present invention.

FIG. 8 is a flowchart illustrating functions of a driver position estimating unit illustrated in FIG. 7;

FIG. 9 is a flowchart illustrating a function of a license plate detecting unit according to another embodiment of the present invention.

FIG. 10 is a diagram illustrating a method of determining the type of a passing vehicle.

FIG. 11 is a functional block diagram showing another embodiment of the present invention.

FIG. 12 is a flowchart illustrating functions of a driver position estimating unit and a lighting unit illustrated in FIG. 11;

FIG. 13 is a functional block diagram showing another embodiment of the present invention.

FIG. 14 is a flowchart illustrating the functions of a windshield detection unit and a camera control unit illustrated in FIG. 13;

FIG. 15 is a functional block diagram of a vehicle monitoring device showing a conventional technique.

[Explanation of symbols]

1, 1a camera, 2 image input means, 3 image storage means, 4 license plate detection means, 4a windshield detection means, 5, 5a, 5b, 5c driver position estimation means, 6 display means, 7 image storage means, 8 Illumination means, 9 Camera control means, 11 Imaging video signal, 12
Digital image signal of 13 images, 13 license plate position information, 13a windshield position information, 14 driver position image, 15 lighting flag, 16 windshield right and left end coordinates, 17 swing angle initialization signal. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (7)

[Claims] 1. An image of a vehicle is installed so that the front of the vehicle is looked down from a sufficiently distant upper position, and the vehicle moves from the top to the bottom of the imaging range as the vehicle approaches / passes every predetermined imaging cycle time. A camera and an image input / output unit that performs A / D conversion of an imaged video signal input from the camera and stores the converted video signal;
Storage means; license plate detection means for detecting license plate images from within the digital image captured by the image storage means to generate license plate position information; and windshield position estimated from license plate position information from the license plate detection means. A vehicle monitoring device comprising: driver position estimating means for estimating a driver position based on information. 2. A windshield detecting means for detecting a windshield image from an imaged digital image stored in an image storage means and generating windshield position information in place of the license plate detecting means. 2. A driver's position is estimated based on windshield position information from a glass detecting means.
The vehicle monitoring device according to claim 1. 3. The windshield detecting means according to claim 2 is separately provided together with the license plate detecting means, and the windshield position information estimated by the driver position estimating means from the license plate position information from the license plate detecting means and the windshield are provided. The vehicle monitoring device according to claim 1, wherein the driver's position is estimated based on both the windshield position information from the glass detecting means. 4. The license plate detecting means determines the type of the passing vehicle based on the license plate in the detected license plate image and outputs the vehicle type information, and the driver position estimating means outputs the license plate position information from the license plate detecting means. The vehicle monitoring device according to claim 1 or 3, wherein windshield position information estimated from the vehicle type information is used. 5. Illumination means for judging necessity / non-necessity of illumination in accordance with the illumination flag from the driver position estimating means and turning on / off the illuminating device is separately provided. 5. The vehicle monitoring device according to claim 1, wherein the lighting flag is set according to presence / absence. 6. A camera control means for separately controlling a camera swing operation in accordance with windshield position information from a driver position estimating means, enabling a camera to perform a swing operation capable of changing an imaging range. 6. The windshield left / right end position information estimated from the license plate left / right end position information as the license plate position information from the license plate detection unit by the estimating means is also output. The vehicle monitoring device according to claim 1. 7. One or both of display means for displaying a partially enlarged monitor display of the driver position image from the driver position estimation means and image storage means for storing the image are separately provided. Claims 1, 2, 3, 4,
7. The vehicle monitoring device according to 5 or 6.