JP3550874B2 - Monitoring device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a monitoring device using an image processing technique for recognizing and monitoring a target object from an image captured by an imaging device.
[0002]
[Prior art]
In a monitoring device using image processing technology that is required to operate all day outdoors or the like, an image captured by an imaging device such as a monitoring camera is used to extract features of an object using an image processing technology such as background subtraction, and based on the extracted feature. The target is recognized and monitored.
[0003]
In this case, since the average luminance value of the image is significantly different between daytime and nighttime, day / night determination is performed based on the captured image, and based on the determination result, the target is monitored using different image processing techniques between daytime and nighttime. It is carried out. As a method of determining day and night, an average luminance value of the entire image is obtained, and the determination is made based on the magnitude of the average luminance value.
[0004]
[Problems to be solved by the invention]
However, the conventional day / night determination method based on the average luminance value of the entire image has a disadvantage that the determination accuracy is reduced in a place or an area where the target object is easily affected by the surrounding environment or the like.
[0005]
Further, when an image is binarized by setting a different binarization threshold (fixed value) between daytime and early morning / dusk, and an object is recognized, the luminance average value such as in early morning / dusk suddenly increases. In the changing time zone, there is a problem that noise is extracted when the binarization threshold is too low, and that the object cannot be recognized when the binarization threshold is too high.
[0006]
The present invention has been made in order to solve such a conventional problem, and improves the determination accuracy when determining day and night from an image, and a time zone in which the average luminance value suddenly changes, such as in the early morning and at twilight. However, it is an object of the present invention to provide a monitoring device using an image processing technology capable of accurately recognizing and monitoring an object.
[0007]
[Means for Solving the Problems]
According to a first aspect of the present invention, there is provided a monitoring apparatus comprising: an image input unit for inputting an image captured by an imaging device; a luminance detection unit for obtaining an average luminance value of a specific designated area in the image; Using the auxiliary information as the auxiliary information , day and night determining means for determining day and night from the average brightness value of the designated area, and from the image using different image processing techniques between daytime and nighttime based on the determination result by the day and night determining means Recognition means for recognizing the object.
[0008]
According to the first aspect, day / night determination is performed based on the average luminance value of a specific designated area instead of the entire area of the image, and the object is recognized using different image processing techniques day / night based on the determination result. Therefore, if the designated area is appropriately set, the area other than the designated area is not subjected to the determination of day and night, so that the influence of the surrounding environment on the target object can be reduced, thereby improving the determination accuracy of day and night. The accuracy of object recognition and the accuracy of monitoring can be improved. Further, in the vicinity of day / night switching (early morning / dusk), the aperture of the imaging device is released, and the shutter speed becomes the lowest. Therefore, using this information reduces erroneous determination when the luminance variance value increases in the daytime. be able to.
[0009]
According to a second aspect of the present invention, there is provided a monitoring apparatus comprising: an image input unit configured to input an image captured by an imaging device; a luminance detection unit configured to calculate a luminance average value and a luminance variance value of a specific designated area in the image; Day and night determining means for determining day and night from the two-dimensional distribution position of the average luminance value and the luminance variance value of the designated area using the exposure information of And a recognizing means for recognizing a target object from an image by using a different image processing technique between nighttime and nighttime.
[0010]
According to the second aspect, day / night determination is performed based on the designated area, and day / night determination is performed based on two pieces of information of the luminance average value and the luminance variance value. Therefore, the determination accuracy can be further improved. Further, in the vicinity of day / night switching (early morning / dusk), the aperture of the imaging device is released, and the shutter speed becomes the lowest. Therefore, using this information reduces erroneous determination when the luminance variance value increases in the daytime. be able to.
[0011]
According to a third aspect of the present invention, there is provided a monitoring apparatus comprising: an image input unit that inputs an image captured by an imaging device ; a luminance detection unit that calculates a luminance average value and a luminance variance value of a specific designated area in the image; When daytime, early morning, and twilight are determined from the average brightness value, and when the daytime, early morning, and twilight are determined, a corrected average brightness value is obtained from a two-dimensional distribution position of the average brightness value and the brightness distribution value. Threshold value determination means for determining a binarization threshold value from the corrected average luminance value; binarization processing means for binarizing the image using the binarization threshold value to obtain a binarized image; And recognition means for recognizing the object .
[0012]
According to the third aspect of the present invention, the determination of daytime, early morning, or twilight is performed based on the data in the designated area. Therefore, if the designated area is appropriately set, the influence of the surrounding environment on the target object is reduced, and the determination accuracy is improved. Can be achieved. In addition, since the determination is performed based on two pieces of information of the average luminance value and the luminance variance value, the determination accuracy can be improved. Furthermore, since the time zones such as early morning twilight which brightness average value is suddenly changed automatically varies the binarization threshold, it is Rukoto obtain the binary image in the optimum binarization threshold.
[0013]
According to a fourth aspect of the monitoring device of the present invention, the threshold value determining means in the third aspect determines a binarization threshold from the corrected average luminance value within a range set with respect to the past binarization threshold. Is configured to determine .
[0014]
According to the fourth aspect, since the present binarization threshold is determined in consideration of the past binarization threshold, the binarization threshold can be stabilized even for a sudden change in the luminance average value .
[0015]
According to a fifth aspect of the present invention, there is provided a monitoring apparatus comprising: an image input unit for inputting an image captured by an imaging device; a luminance detecting unit for obtaining a luminance average value and a luminance variance value of a specific designated area in the image; Using the exposure information of the apparatus as an auxiliary, a threshold determining means for determining daytime, early morning, and dusk from the average luminance value of the designated area and determining a binarization threshold suitable for them, and a binarization threshold The image processing apparatus includes a binarization processing unit that binarizes an image to obtain a binarized image, and a recognition unit that recognizes an object from the binarized image.
[0016]
According to the fifth aspect, the determination of daytime, early morning, or twilight is performed based on the data in the designated area. Therefore, if the designated area is appropriately set, the influence of the surrounding environment on the target object is reduced, and the determination accuracy is improved. Can be achieved. In addition, since the determination is performed based on two pieces of information of the average luminance value and the luminance variance value, the determination accuracy can be improved. Further, in the early morning and at twilight near the switching between day and night, the aperture of the imaging device is released and the shutter speed is minimized. Therefore, unnecessary fluctuation of the binarization threshold can be reduced by using this information. .
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
(Embodiment 1)
FIG. 1 is a block diagram of the monitoring device according to the first embodiment of the present invention. This device performs object recognition using different recognition methods for day and night.By improving the accuracy of day / night determination, the accuracy of selecting the object recognition method is improved, thereby monitoring the object. It is intended to improve the accuracy.
[0027]
In FIG. 1, an image input unit 1 captures an image captured by an imaging device (not shown) such as a monitoring camera installed outdoors as a monitoring image, and temporarily stores the captured image in an internal image memory.
[0028]
The brightness detection unit 2 obtains the brightness of each pixel in a predetermined designated area from the monitoring image stored in the image memory, obtains a value obtained by dividing the sum by the number of pixels in the designated area, and obtains a brightness average value. A luminance variance value is obtained from a luminance distribution of each pixel.
[0029]
FIG. 2 shows the relationship between the entire area E1 and the designated area E2 of the monitoring image stored in the image memory. Conventionally, the average of the brightness of each pixel in the entire area E1 indicated by the solid line is determined to determine day or night. However, in the present embodiment, the average of the brightness of the designated area E2 indicated by the one-dot chain line or the average of the brightness is determined. Day and night are determined from the values and the luminance variance values. Since the surveillance camera is fixed, if the designated area E2 is appropriately set, pixels other than the designated area E2 are not subjected to day / night determination, so that the influence of the surrounding environment on the object can be reduced. The determination accuracy between day and night can be improved.
[0030]
When the designated area E2 is divided into a plurality of small areas E3 as shown by a broken line in the figure, a luminance average value and a luminance variance value are obtained for each small area E3, and a representative value is determined from the average value and the luminance variance value. The luminance average value and the luminance variance value of the designated area E2 are set. The representative value includes an average value other than the intermediate value, the minimum value, or the maximum value. As a result, noise in the designated area E2 can be reduced.
[0031]
The day / night determining unit 3 determines day / night from the average luminance value obtained by the luminance detecting unit 2 or the average luminance value and the luminance dispersion value. When the day and night are determined from the average brightness value, the average brightness value is compared with a predetermined determination threshold. If the average brightness value is larger than the determination threshold, daytime is determined. When determining day and night based on the luminance average value and the luminance variance value, the judgment is made from the two-dimensional distribution position of the luminance average value and the luminance variance value. Details of these will be described later.
[0032]
The daytime recognition unit 4a and the nighttime recognition unit 4b perform image processing of the monitoring image stored in the image input unit 1 using different image processing techniques between daytime and nighttime based on the determination result of the daytime / nighttime determination unit 3. And recognize the object. In the daytime, the target object is recognized by the daytime recognition unit 4a, for example, by background matching, frame difference, pattern matching by edge extraction, or the like. In the case of nighttime, if the target is a vehicle, the nighttime recognition unit 4b recognizes the target vehicle by pattern matching of the headlight part.
[0033]
The output unit 5 transmits the data generated by the daytime recognition unit 4a and the nighttime recognition unit 4b to a central monitoring device (not shown) via a communication line. For example, if the object is a passerby, the number of persons is transmitted, and if the object is an automobile, the number of passing persons is transmitted.
[0034]
Next, the details of the day / night determination in the day / night determination unit 3 will be described. FIG. 3 is a graph showing a change in the average luminance value at each time from daytime to nighttime. As is clear from this figure, at time TM, the average luminance value becomes smaller than the determination threshold value TH. Therefore, before time TM, daytime is determined, and thereafter, nighttime is determined.
[0035]
In addition, if the luminance average value at each time is taken, the fluctuation is remarkable. If the average is taken at a predetermined time and smoothed over time, as shown in FIG. Since the variation in the determination in the vicinity is reduced, the switching between day and night processing is stabilized.
[0036]
Next, when performing day / night determination from the luminance average value and the luminance variance value, as shown in FIG. 5, the two-dimensional distribution positions of the luminance average value and the luminance variance value at each time are plotted on the coordinates. In the figure, black circles indicate data at each time in the daytime, and white circles indicate data at each time in the night.
[0037]
Next, a determination function F is drawn on the coordinates. This determination function F can be expressed as “y = ax−b” (where a and b are positive constants), where the luminance average value is the x coordinate and the luminance variance is the y coordinate. If the data is located on the low luminance average value side (left side in the figure) with respect to the straight line of the determination function F, it is determined to be night, and if the data is located on the high luminance average value side (right side in the figure), it is determined to be daytime. Therefore, for example, when the object is an automobile, it is determined that the vehicle is at night if the luminance variance is large even if the luminance average value is increased by the headlight. As described above, since the day and night are determined using the luminance average value and the luminance dispersion value, highly accurate day and night determination can be performed.
[0038]
In order to accurately perform the day / night determination in the early morning and the day / night determination in the twilight, as shown in FIG. The determination may be performed by setting two determination thresholds, the determination threshold THND to be detected.
[0039]
Also, when performing day / night determination from the luminance average value and the luminance variance value, as shown in FIG. 7, a determination function FDN for detecting a transition from daytime to night and a determination function FND for detecting a transition from nighttime to daytime. The determination may be performed using the two determination functions.
[0040]
(Embodiment 2)
FIG. 8 is a block diagram of the monitoring device according to the second embodiment of the present invention. This device binarizes an original image or a background image with a binarization threshold and recognizes an object from the obtained binarized image. During the time period, the binarization threshold is automatically varied, and a binarized image is obtained with the optimal binarization threshold, thereby improving the monitoring accuracy of the object.
[0041]
8, an image input unit 11 captures an image captured by an imaging device (not shown) such as a monitoring camera installed outdoors as a monitoring image, and temporarily stores the captured image in an internal image memory.
[0042]
The luminance detection unit 12 calculates the luminance of each pixel of the predetermined designated area E2 (FIG. 2) from the monitoring image stored in the image memory, and divides the sum by the number of pixels of the designated area E2 to calculate the average luminance value. . Further, a luminance variance value of the designated area E2 is obtained from the luminance distribution of each pixel of the designated area E2.
[0043]
When the designated area E2 is divided into a plurality of small areas E3, a luminance average value and a luminance variance value are obtained for each of the small areas E3, and a representative value is determined therefrom to determine a luminance average value and a luminance variance value of the designated area E2. And By specifying the monitoring area in this way, pixels other than the specified area E2 are not subjected to the determination, thereby reducing the influence of the surrounding environment. These are the same as in the above-described luminance detection unit 2.
[0044]
The threshold determination unit 13 determines a binarization threshold from the luminance average value and the luminance variance obtained by the luminance detection unit 12. The binarization threshold includes a daytime threshold (fixed value) and an early morning / dusk threshold (variable value). Details of the method of determining the binarization threshold will be described later.
[0045]
The binarization processing unit 14 binarizes the monitoring image stored in the image memory according to the binarization threshold determined in this way to obtain a binarized image. The binarization threshold is used for binarization of the original image, binarization after background difference, binarization after frame difference, binarization after differentiation, and the like.
[0046]
The recognition unit 15 recognizes an object by pattern matching or the like from the binarized image obtained by the binarization processing unit 14, and generates data related to the object based on the object. The output unit 16 transmits the data generated by the recognition unit 15 to a central monitoring device (not shown) via a communication line. For example, if the object is a passerby, the number is counted, and if the object is a vehicle, the number is counted and transmitted to a central monitoring device via a communication line.
[0047]
Next, a method of determining the binarization threshold in the threshold determination unit 13 will be described. FIG. 9 is a graph showing changes in the average luminance value at each time from daytime to nighttime. If the average luminance value is greater than the upper luminance limit value BU, it is determined that the day is daytime, and a binarization threshold value for daytime is determined. If the average luminance value is less than the lower luminance limit value BL, the lower limit binarization threshold value for early morning / dusk is used. To determine.
[0048]
When the average luminance value is located in the area BR between the upper limit value BU and the lower limit value BL, it is determined that the morning is early in the morning and at dusk, and the binarization threshold is determined by the procedure described below. It should be noted that, if the luminance average value at each time is taken, the fluctuation is drastic. If the average is taken at a predetermined time and smoothed over time, the abrupt fluctuation is reduced as shown in FIG.
[0049]
FIG. 11 is a diagram in which the two-dimensional distribution positions of the luminance average value and the luminance variance value at each time are plotted on coordinates. The upper right position in the figure is data on a sunny daytime, and the lower position is data on a cloudy daytime. It becomes data when the dusk deepens as it moves to the lower left of the figure.
[0050]
Next, an upper limit function FU and a lower limit function FL are drawn on the coordinates. Assuming that the luminance average value is the x coordinate, the luminance variance value, and the y coordinate, the upper limit function FU is “y = −cx + d”, and the lower limit function FL is “y = −cx + e” (e <d). Can be expressed. When the data is located between the upper limit function FU and the lower limit function FL, as shown in FIG. 12, the data is set in parallel with the upper limit function FU and the lower limit function FL to determine the binarization threshold for each data. Assume a function FD passing through D. This function FD can be expressed as “y = −cx + f” (e <f <d). Then, the luminance average value on the function FD when the luminance variance value is 0 is set as the corrected luminance average value BD.
[0051]
Next, as shown in FIG. 13, the binarization threshold value THD is determined from the corrected luminance average value BD using a threshold calculation function Fth that indicates the relationship between the luminance average value and the binarization threshold value. The corrected brightness average value BD is located in the variable area BR between the upper limit value BU and the lower limit value BL of the brightness average value, and the determined binarization threshold value THD is equal to the daytime binarization threshold value THU and the early morning / dusk. Is set so as to linearly fluctuate between the threshold value THL and the lower limit binarization threshold value THL. The image is binarized by the binarization processing unit 14 using the binarization threshold value THD obtained in this manner, and a binarized image is obtained.
[0052]
In order to consider the past binarization threshold, as shown in FIG. 14, the threshold variation effective area THR is set for the past binarization threshold (black circle in the figure), and the binarization threshold obtained this time is set. If the THD is within the effective area THR, it is determined as it is, if it exceeds the upper limit THRU of the effective area THR, the upper limit is determined as the binarization threshold, and if it falls below the lower limit THRL, the lower limit is set. It is determined as a binarization threshold. As a result, a rapid change in the average luminance value can be suppressed.
[0053]
As another method for suppressing a rapid change in the average luminance value, as shown in FIG. 15, if the past binarization threshold is the daytime binarization threshold THU, the current binarization threshold THU is used. Is calculated from the first threshold value calculation function Fth1. Next, when the calculated binarization threshold becomes the night-time binarization threshold THL, it is calculated from the second threshold calculation function Fth2. The threshold calculation function Fth2 is used until the calculated binarization threshold becomes the daytime binarization threshold THU. Hereinafter, this is repeated.
[0054]
In this case, the variable region includes two variable regions, a variable region BR1 (average luminance values BL1 to BU1) when the threshold value calculation function Fth1 is used and a variable region BR2 (average luminance values BL2 to BU2) when the threshold value calculation function Fth2 is used. Is set.
[0055]
In the above-described embodiment, the determination of day and night or the determination of day and early morning / dusk is made only from the monitoring image.However, the aperture of the monitoring camera is released in the early morning / dusk near day / night switching. Since the shutter speed becomes the lowest, use of this information as auxiliary information can reduce unnecessary fluctuation of the binarization threshold.
[0056]
【The invention's effect】
According to the present invention, when the recognition of an object is performed day and night using different recognition methods, the determination of day and night is performed according to the designated area. Therefore, by appropriately setting the designated area, the surrounding environment for the object is determined. Can be reduced, and the determination accuracy can be improved. In addition, since the determination of day and night is made based on two pieces of information, the luminance average value and the luminance variance value, the determination accuracy can be further improved. Thereby, the monitoring accuracy of the object can be improved.
[0057]
Further, according to the present invention, when an original image or the like is binarized by a binarization threshold and a target object is recognized from the obtained binarized image, a luminance average value such as early morning or dusk changes suddenly. Since the binarization threshold is automatically changed in the time zone, the recognition accuracy of the target object can be improved in any time period in the daytime, early morning, and twilight, thereby improving the monitoring accuracy of the target object. Improvement can be achieved.
[Brief description of the drawings]
FIG. 1 is a block diagram of a monitoring device according to a first embodiment of the present invention.
FIG. 2 is a diagram showing an entire area, a designated area, and a small area of a monitoring image.
FIG. 3 is a graph showing a change in a luminance average value.
FIG. 4 is a graph obtained by temporally smoothing a change in the average luminance value shown in FIG.
FIG. 5 is a diagram in which two-dimensional distribution positions of a luminance average value and a luminance variance value at each time are plotted, in which black circles indicate daytime data and white circles indicate nighttime data.
FIG. 6 is a graph showing a change in an average luminance value, and is a graph for determining day and night based on two determination thresholds.
FIG. 7 is a diagram in which two-dimensional distribution positions of a luminance average value and a luminance variance value at each time are plotted, and is a diagram for determining day or night by two determination functions.
FIG. 8 is a block diagram of a monitoring device according to a second embodiment of the present invention.
FIG. 9 is a graph showing a change in an average luminance value.
FIG. 10 is a graph obtained by temporally smoothing a change in the average luminance value shown in FIG. 9;
FIG. 11 is a diagram in which two-dimensional distribution positions of a luminance average value and a luminance variance value at each time are plotted.
FIG. 12 is an explanatory diagram for obtaining a corrected average luminance value from data in the early morning and at twilight.
FIG. 13 is a diagram illustrating a threshold calculation function representing a relationship between a luminance average value and a binarization threshold.
FIG. 14 is an explanatory diagram for determining a binarization threshold in consideration of a past binarization threshold;
FIG. 15 is an explanatory diagram for determining a binarization threshold in consideration of a past binarization threshold;
[Explanation of symbols]
REFERENCE SIGNS LIST 1 image input unit 2 luminance detection unit 3 day / night determination unit 4a daytime recognition unit 4b nighttime recognition unit 5 output unit 11 image input unit 12 luminance detection unit 13 threshold value determination unit 14 binarization processing unit 15 recognition unit 16 output unit E1 Overall area E2 Designated area E3 Small area

Claims (5)

Image input means for inputting an image taken by the imaging device;
Brightness detection means for obtaining a brightness average value of a specific designated area in the image,
Day / night determining means for determining day / night from the average luminance value of the designated area using exposure information of the imaging device as auxiliary information ,
Recognition means for recognizing a target object from the image using different image processing techniques between daytime and nighttime based on the determination result of the day / night determination means,
A monitoring device comprising: Image input means for inputting an image taken by the imaging device;
Brightness detection means for calculating a brightness average value and a brightness variance value of a specific designated region in the image,
Day / night determination means for determining day / night from the two-dimensional distribution position of the average luminance value and the luminance variance value of the designated area using the exposure information of the imaging device as auxiliary information ,
Recognition means for recognizing a target object from the image using different image processing techniques between daytime and nighttime based on the determination result of the day / night determination means,
A monitoring device comprising: Image input means for inputting an image taken by the imaging device;
Brightness detection means for calculating a brightness average value and a brightness variance value of a specific designated region in the image,
Daytime and early morning / dusk are determined from the average luminance value of the designated area, and when the daytime, early morning / dusk is determined, the corrected average luminance value is calculated from the two-dimensional distribution position of the average luminance value and the luminance distribution value. Threshold value determining means for determining a binarization threshold value from the corrected luminance average value;
A binarization processing unit that binarizes the image using the binarization threshold to obtain a binarized image;
Recognizing means for recognizing an object from the binarized image;
Monitoring device, characterized in that it comprises a. Threshold determination means, correcting the luminance average value from the monitoring device according to claim 3, wherein that you determined within a range set for the past binarization threshold in determining a binarization threshold. Image input means for inputting an image taken by the imaging device;
Brightness detection means for calculating a brightness average value and a brightness variance value of a specific designated region in the image,
Threshold determining means for determining daytime, early morning and dusk from the average luminance value of the designated area and determining a binarization threshold suitable for them by using the exposure information of the imaging device as an auxiliary ,
A binarization processing unit that binarizes the image using the binarization threshold to obtain a binarized image;
Recognizing means for recognizing an object from the binarized image;
A monitoring device comprising:

Images