JP5279475B2 - Surveillance camera device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily specify a subject, while imaging the subject without being affected by peripheral light. <P>SOLUTION: The monitoring camera apparatus includes: a modulation light radiating section 20 for irradiating a target area with modulation light; an optical modulation image generating section 21 for receiving reflection light reflected by a subject and generating an optical modulation image, with only a reflected light component, corresponding to the modulation light, in the reflection light as a pixel value; and an image processing section 4 for performing image processing on the optical modulation light, to extract a feature amount of the subject. The image processing section 4 selects an optimal frame for selecting the relevant subject from among the optical modulation images of a plurality of frames generated within a predetermined period, on the basis of an extraction result of the feature amount. Consequently, the subject can be imaged without being affected by a periheral light, such as, reflection onto a glass door or a windshield of an automobile, by using the optical modulation image, with only the reflection light component corresponding to the modulation light as the pixel value; and further, only optimal frame is selected, based on the result of the feature amount extraction due to the image processing means, thereby readily specifying the subject. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a surveillance camera device.

  Conventionally, to capture people (intruders) who invade offices and stores, and cars that pass through toll road interchanges (tollgates) such as expressways without paying fees (traffic vehicles) As described above, surveillance camera devices are installed at entrances and exits of offices and stores, interchange toll booths, and the like (see, for example, Patent Document 1 or Patent Document 2).

  Here, in the conventional surveillance camera device, when a subject (person, driver) is imaged through a glass door provided at the entrance of an office or a store or a windshield of an automobile, the surroundings are placed around the glass door or the windshield. The subject's face may not be captured normally because of the scenery.

On the other hand, the present applicant irradiates the target area with modulated light obtained by high-speed modulation of infrared light emitted from the light emitting diode, and receives the reflected light component of the modulated light and the reflected light of the ambient light received by the image pickup device composed of the CCD There has already been proposed a light modulation camera device that distinguishes components from each other and generates an image (light modulation image) having a reflected light component of only modulated light as a pixel value (see Patent Document 3). Since the reflection on the glass door or the windshield is due to ambient light, the subject can be imaged without being affected by the reflection by using the light modulation camera device for the monitoring camera device.
JP 2008-197738 A JP 2008-250677 A JP 2006-121617 A

  By the way, even if the influence of ambient light such as reflection can be excluded by the light modulation image, the face of the driver as the subject is not facing the front (front) but facing sideways or down, or the face is hidden by the hand In such a case, it is difficult to specify a person (driver).

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a surveillance camera device that can easily identify a subject while imaging the subject without being influenced by ambient light.

In order to achieve the above object, the invention according to claim 1 receives the modulated light irradiating means for irradiating the target area with the modulated light and the reflected light reflected by the subject existing in the target area and modulating the reflected light. A light-modulated image generating means for generating a light-modulated image having only a reflected light component corresponding to light as a pixel value, an image processing means for extracting the feature quantity of the subject by image processing the light-modulated image, and image processing Selection means for selecting an optimal frame for identifying the subject from a plurality of light modulation images generated within a predetermined period based on the extraction result of the feature amount by the means, and reflection reflected by the subject Non-light modulation image generating means for receiving light and generating a non-light modulation image having the reflected light component as a pixel value, and the image for extracting the feature quantity of the subject by image processing the non-light modulation image And a management unit, said selecting means, if the extraction result of the feature amount of the light modulated image and the non-light modulated image is not observed superiority is characterized by selecting a non-light modulated image preferentially.

According to the first aspect of the present invention, an object can be detected without being affected by ambient light such as reflection on a glass door or an automobile windshield by a light modulation image having only a reflected light component corresponding to the modulated light as a pixel value. Since an image can be picked up, and only the optimal frame based on the feature amount extraction result by the image processing means is selected, the subject can be easily specified. In addition, if superiority or inferiority is not found in the feature extraction results of light-modulated images and non-light-modulated images, that is, if the influence of ambient light such as reflection is small, the reflected light component including ambient light is used as the pixel value. By preferentially selecting a non-light modulated image having a relatively higher resolution than the modulated image, the subject can be identified more easily.

According to a second aspect of the present invention, in the first aspect of the invention, the optical modulation is performed when the moving object is detected by the moving object detecting means and the moving object detecting means for detecting the presence or absence of the moving object present before the subject. The pixel area where the moving object is imaged is detected by performing image processing on the image, and the moving object is imaged based on pixel information of the pixel area where the moving object is imaged between light modulation images of a plurality of frames. And image processing means for correcting with pixel information of the pixel area that is not.

According to the invention of claim 2 , when there is a moving object (for example, raindrops, snow, etc.) in front of the subject, the pixel region in which the moving object is imaged between light modulation images of a plurality of frames By correcting the pixel information with the pixel information of the pixel area where the moving object is not imaged, it is possible to obtain a light modulation image suitable for specifying the subject without being affected by the presence of the moving object.

According to a third aspect of the present invention, in the second aspect of the present invention, the light modulation image generating unit is configured such that when the moving object is detected by the moving object detection unit, the light modulation image generating unit is not compared with the case where the moving object is not detected. It is characterized in that the period for generating the light modulation image is shortened.

According to the invention of claim 3 , since the number of frames of the light modulation image obtained per unit time is increased by shortening the cycle for generating the light modulation image, the light modulation image that is a candidate for correcting the pixel information As a result, the accuracy of correction can be improved.

According to a fourth aspect of the present invention, in the first aspect of the invention, the image processing means detects a regular reflection portion in which the pixel value is equal to or greater than a predetermined threshold in the light modulation image, and the selection means is the image processing means. However, when the specular reflection portion is detected and the feature amount of the subject cannot be sufficiently extracted, the light modulation image is excluded from the selection target.

According to the invention of claim 4 , the modulated light may be specularly reflected on the glass door or the windshield of the automobile, and when such a specularly reflected portion overlaps the subject (particularly the face), the feature amount of the subject Cannot be extracted sufficiently, and in this case, the accuracy of specifying the subject is improved by excluding the light modulation image from the selection target.
According to a fifth aspect of the present invention, in order to achieve the above object, the modulated light irradiating means for irradiating the target area with the modulated light and the reflected light reflected by the subject existing in the target area are received and modulated among the reflected light. A light-modulated image generating means for generating a light-modulated image having only a reflected light component corresponding to light as a pixel value, an image processing means for extracting the feature quantity of the subject by image processing the light-modulated image, and image processing Selection means for selecting an optimal frame for specifying the subject from a plurality of light modulation images generated within a predetermined period based on the extraction result of the feature amount by the means, and the image processing means Detects a specular reflection part in which the pixel value is greater than or equal to a predetermined threshold value in the light modulation image, and the selecting means detects the specular reflection part by the image processing means and cannot sufficiently extract the feature amount of the subject. Wherein the exclusion can the light modulated image from the selected target.
According to the fifth aspect of the present invention, the object can be detected without being affected by ambient light such as reflection on a glass door or a windshield of a car by a light modulation image having only a reflected light component corresponding to the modulated light as a pixel value. Since an image can be picked up, and only the optimal frame based on the feature amount extraction result by the image processing means is selected, the subject can be easily specified. Also, the modulated light may be specularly reflected on the glass door or the windshield of the car, and when such a specular reflection part overlaps the subject (especially the face), the subject's feature quantity cannot be extracted sufficiently, In that case, the accuracy of specifying the subject is improved by excluding the light modulation image from the selection target.

According to a sixth aspect of the present invention, in the fourth or fifth aspect of the present invention, in the case where the person sitting in the driver's seat of the automobile is the subject, the image processing means The position of the driver's seat of the automobile is estimated by performing image processing on the modulated image, and pixel information of the pixel area corresponding to the position of the driver's seat is corrected between the light modulated images of a plurality of frames.

  According to the sixth aspect of the present invention, even when the feature amount of the person (driver) seated in the driver's seat cannot be extracted because the modulated light is regularly reflected on the windshield, it is estimated by image processing of the light modulation image. By correcting the pixel information of the pixel area corresponding to the position of the driver's seat, the subject identification accuracy is improved.

  According to the present invention, it is possible to easily identify a subject while imaging the subject without being affected by ambient light.

  Hereinafter, an embodiment in which the technical idea of the present invention is applied to a monitoring camera device for imaging a driver of a car passing through a toll gate on a toll road such as an expressway will be described. However, the monitoring camera device to which the technical idea of the present invention can be applied is not limited to this embodiment. For example, the monitoring camera device for imaging a person passing through an entrance of an office, a store, etc. The technical idea of the present invention can be applied.

  As shown in FIG. 1, the monitoring camera device 1 of the present embodiment includes a light modulation camera unit 2 that captures a light modulation image, a non-light modulation camera unit 3 that captures a non-light modulation image, a light modulation image, and a non-light. An image processing unit 4 that performs various image processing to be described later on the modulated image, and controls the image processing unit 4 and outputs a light modulated image or a non-light modulated image output from the image processing unit 4 to an external storage device 6. And a control unit 5 to be stored.

  The light modulation camera unit 2 includes a modulated light irradiating unit 20 that irradiates modulated light onto a target region, and an image sensor that is sensitive to the modulated light, and the reflection corresponding to the modulated light among the reflected light received by the image sensor. And a light modulation image generation unit 21 that generates a light modulation image having only light components as pixel values. The modulated light irradiation unit 20 includes a light emitting diode (not shown) that emits infrared light, and is driven by a modulation signal with a very high frequency (for example, 10 megahertz) output from the light modulation image generation unit 21. Then, modulated light is irradiated onto a target area (an area including an automobile lane passing through a toll booth) where a subject (car driver) exists. The light modulation image generation unit 21 includes an image sensor having sensitivity to infrared light (for example, a dedicated semiconductor image sensor described in Patent Document 3), and is synchronized with the light emission period of the light emitting diode by the modulation signal. Then, a process of extracting only the reflected light component of infrared light from the output of each pixel of the image sensor (hereinafter referred to as “synchronous detection process”) is performed, and the reflected light component extracted by the synchronous detection process is used as the pixel value. A light modulation image is generated. Here, in the light modulation image generation unit 21, the number of frames of the light modulation image generated per unit time (1 second) is variable (in other words, the cycle for generating the light modulation image is variable).

  The non-light modulation camera unit 3 has an image sensor (for example, a general-purpose CCD image sensor or a CMOS image sensor) having sufficiently higher resolution than the image sensor of the light modulation camera unit 2 and mainly reflects light corresponding to ambient light. A non-light modulation image having a light component as a pixel value is generated. However, since a general CCD camera or the like can be used for such a non-light modulation camera unit 3, illustration and description of a detailed configuration are omitted.

  The image processing unit 4 includes a microcomputer or a microprocessor dedicated to image processing (image processing LSI) and a frame memory. Here, in the image processing unit 4, the feature amount of the driver's face (for example, eyes, eyes) is calculated from the light modulation image output from the light modulation camera unit 2 and the non-light modulation image output from the non-light modulation camera unit 3. In addition to feature amount extraction processing that extracts the edges of facial organs such as the nose and mouth, the driver's face orientation and driver behavior (close eyes and scratch the face with hands) based on the extracted feature amounts Determination processing, frame evaluation processing for evaluating the extraction result of the feature amount for each frame of the light modulation image and the non-light modulation image, moving object detection processing for detecting the presence or absence of a moving object such as rain or snow, Image information correction processing for correcting pixel information (pixel values) of a specific pixel region between a plurality of frames of the light modulation image is performed.

  The control unit 5 includes a microcomputer, a memory, and the like, and is an image that has not been subjected to image processing by the image processing unit 4 (light modulation image or non-light modulation image), or an image that has undergone image processing by the image processing unit 4 (light modulation image). Alternatively, a process of storing the non-light modulated image) in the storage device 6 is executed. The storage device 6 has a storage medium such as a rewritable non-volatile semiconductor memory and a magnetic disk, and is a conventionally known device that writes and reads an image (image data) on the recording medium.

  Next, the operation of the monitoring camera device 1 of this embodiment will be described with reference to the flowchart of FIG.

  The light modulation camera unit 2 and the non-light modulation camera unit 3 generate a light modulation image and a non-light modulation image at a predetermined frame rate (frames per second) and output them to the image processing unit 4 (steps S1 and S2). The image processing unit 4 performs feature amount extraction processing on each frame of the light modulation image and the non-light modulation image (steps S3 and S4), and then extracts the feature amount extraction result of the light modulation image and the feature of the non-light modulation image. The frame evaluation process is executed on the amount extraction result to relatively evaluate the feature amount extraction result of the light modulation image and the feature amount extraction result of the non-light modulation image (step S5). For example, if all feature amounts of the three types of facial organs of eyes, nose and mouth can be extracted from the light modulation image, but only the mouth feature amount can be extracted from the non-light modulation image, the light modulation image In the relative evaluation of the feature value extraction result of the image and the feature value extraction result of the non-light modulation image, it is determined that the feature value extraction result of the light modulation image is recognized as superior, and both the light modulation image and the non-light modulation image are determined. If all the features of the three types of facial organs of eyes, nose, and mouth are extracted, superiority or inferiority is recognized in the relative evaluation of the feature extraction result of the light modulation image and the feature extraction result of the non-light modulation image It is determined that it is not possible (step S6).

  The image processing unit 4 determines that the feature amount extraction result of the light modulation image is superior in the relative evaluation of the feature amount extraction result of the light modulation image and the feature amount extraction result of the non-light modulation image. When there is an influence of the reflection of the windshield in the light modulation image, a determination process is executed on the light modulation image (step S7), and when it is determined that superiority or inferiority is not recognized in both feature amount extraction results, If the non-light modulation image is hardly affected by the reflection of the windshield, the determination process is executed on the non-light modulation image (step S8). In this determination process, the image processing unit 4 determines the driver's ability based on the feature amount extraction result of the target image (light modulated image or non-light modulated image) and the feature amount extraction result of the image of several frames before and after the image. Judgment of face orientation and driver's behavior (close eyes, scratching face with hand, etc.) and excluding frames determined as being closed by the driver or hiding the face with hands At the same time, the image of the frame closest to the front direction (front direction with respect to the monitoring camera device 1) of the driver's face is selected from the remaining frames and output to the control unit 5 (step S9). Then, the control unit 5 records the image (light modulated image or non-light modulated image) received from the image processing unit 4 on the recording device 6 (step S10).

  Here, since the light-modulated image is generated as a grayscale image of the reflected light component reflected from the subject or the like, a normal image (non-modulated light image) composed of the reflected light component reflected from ambient light such as sunlight. It looks different. Therefore, when the non-light modulation image is hardly affected by the reflection of the windshield, the subject (driver) can be easily identified by preferentially selecting and recording the non-light modulation image with respect to the light modulation image. It can be done.

  In this embodiment, a non-light modulation camera unit 3 for capturing a non-light modulation image is provided separately from the light modulation camera unit 2, but the light modulation image generation unit 21 of the light modulation camera unit 2 has ambient light. Alternatively, a non-light modulation image having only the reflected light component as a pixel value may be generated. With such a configuration, there is an advantage that the monitoring camera device 1 can be downsized because only one camera unit is required. However, an image sensor for capturing a light-modulated image requires a dedicated image sensor and has fewer pixels than a general image sensor that captures a non-light-modulated image. The resolution of the modulated image is relatively low. For example, the resolution of the image sensor of the light modulation camera unit 2 is QVGA, and the resolution of the image sensor of the non-light modulation camera unit 3 is VGA or higher. Therefore, according to the configuration of the present embodiment separately including the non-light modulation camera unit 3 for capturing the non-light modulation image, the light modulation image is more effective than the case of the influence of ambient light such as the reflection of the windshield. By selecting a non-light modulated image with high resolution, there is an advantage that the identification accuracy of the driver is improved.

  By the way, since the surveillance camera device 1 of this embodiment is used outdoors, it naturally captures moving objects such as rain, snow, insects, and small birds. If the image is superimposed on the face of the driver, the feature amount extraction of the driver's face will be hindered. Therefore, in the present embodiment, the image processing unit 4 executes a moving object detection process for detecting the presence or absence of a moving object such as rain or snow. For example, since raindrops and snow have a relatively high reflectance, it is considered that the pixel value of the pixel area where raindrops and snow are captured in the light-modulated image is relatively large. A moving object (raindrop or snow) can be detected by extracting a pixel region that exceeds (so-called whiteout).

  Further, the image processing unit 4 corrects the pixel value (pixel information) of the pixel area where the moving object is detected by the moving object detection process by the image information correction process. For example, if the left frame and the mouth are hidden by the moving object in the first frame, only the mouth is hidden by the moving object in the second frame, and only the left eye is hidden by the moving object in the third frame, the image The processing unit 4 replaces the pixel value of the pixel area including the left eye of the first frame with the pixel value of the pixel area including the left eye of the second frame, and the pixel value of the pixel area including the mouth of the first frame Is replaced with the pixel value of the pixel region including the mouth of the third frame to correct the pixel value of the pixel region including the left eye and the pixel region including the mouth in the first frame, so that the left eye and the mouth are moving objects. A normal light modulation image that is not hidden is synthesized and output to the control unit 5. However, in order to perform such image information correction processing, the image processing unit 4 instructs the light modulation image generation unit 21 at the time when a moving object such as rain or snow is detected, and performs light processing at a frame rate higher than normal. It is desirable to generate a modulated image.

  As described above, even when a moving object is imaged on the driver's face, a normal light modulation image in which the moving object does not overlap the driver's face is synthesized by the image information correction process by the image processing unit 4. As a result, a light modulation image suitable for specifying the subject (driver's face) can be obtained without being affected by the presence of the moving object.

  By the way, since the inclination angle of the windshield is different for each vehicle type, the modulated light emitted from the modulated light irradiation unit 20 is regularly reflected on the windshield depending on the vehicle type and enters the image sensor of the light modulation image generation unit 21. There are cases. And since the pixel value of the pixel area where the specularly reflected light component (hereinafter referred to as specular light component) is incident becomes very large, the pixel area overlaps the pixel area where the driver's face is imaged. This will hinder the extraction of the feature amount of the driver's face.

  Accordingly, the image processing unit 4 compares the pixel values of all the pixel regions of the light modulation image with a predetermined threshold value, and the pixel value exceeds the threshold value (so-called whiteout) pixel region. Specular reflection part detection processing is executed to detect as a regular reflection part. Furthermore, the image processing unit 4 performs the feature amount extraction process on the light modulation image in which the specular reflection portion is detected by the specular reflection portion detection process. As a result, the feature amount of the driver's face cannot be sufficiently extracted. The light modulation image is excluded from selection targets. Alternatively, as in the case of the moving object described above, if the image processing unit 4 executes the image information correction process and corrects the regular reflection portion in the light modulation image with the pixel value of the pixel area in the light modulation image of another frame, Even when the specular reflection part is superimposed on the driver's face, it is possible to synthesize a normal light modulation image that does not overlap the specular reflection part on the driver's face, and this affects the presence of the specular reflection part. Accordingly, a light modulation image suitable for specifying the subject (driver's face) can be obtained.

  Here, the pixel area of the regular reflection portion is relatively large, and it may be difficult to specify the pixel area including the driver's face from the light modulation image. In such a case, the image processing unit 4 performs image processing on the light modulation image to estimate the position of the driver's seat of the automobile, and a pixel region corresponding to the position of the driver's seat between the light modulation images of a plurality of frames. The pixel information is corrected. Specifically, the position of the driver's seat may be estimated by executing conventionally known image processing on the light modulation image and recognizing the shape of the vehicle body or the windshield of the automobile. In this way, even when the feature amount of the driver seated in the driver's seat cannot be extracted because the modulated light is regularly reflected on the windshield, the position of the driver's seat estimated by the image processing of the light modulation image It is possible to correct the pixel information of the pixel area corresponding to, and specify the pixel area including the driver's face. If the pixel area including the driver's face can be specified, the image processing unit 4 can execute the image information correction process to obtain a light modulation image suitable for specifying the subject (driver's face).

It is a block diagram which shows embodiment of this invention. It is a flowchart for operation | movement description same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Surveillance camera apparatus 2 Light modulation camera unit 3 Non-light modulation camera unit 4 Image processing part (image processing means, selection means)
20 Changed light irradiation unit (modulated light irradiation means)
21 Light modulation image generation unit (light modulation image generation means)

Claims (6)

Modulated light irradiation means for irradiating modulated light to the target area, and light modulation that receives reflected light reflected by a subject existing in the target area and uses only the reflected light component corresponding to the modulated light among the reflected light as a pixel value A light modulation image generation unit that generates an image, an image processing unit that extracts a feature amount of the subject by performing image processing on the light modulation image, and a predetermined period based on the extraction result of the feature amount by the image processing unit Selection means for selecting an optimal frame for identifying the subject from a plurality of light modulation images to be generated; and non-light that receives reflected light reflected by the subject and uses the reflected light component as a pixel value A non-light modulated image generating means for generating a modulated image; and the image processing means for extracting a feature amount of the subject by image processing the non-light modulated image ;
The monitoring camera device according to claim 1, wherein the selection means preferentially selects the non-light modulation image when superiority or inferiority is not recognized in the extraction result of the feature amount of the light modulation image and the non-light modulation image . Moving object detection means for detecting the presence or absence of a moving object present in front of the subject, and the moving object is imaged by performing image processing on a light modulation image when the moving object is detected by the moving object detection means The image processing for detecting a pixel area and correcting pixel information of the pixel area in which a moving object is imaged between light modulation images of a plurality of frames with pixel information of the pixel area in which the moving object is not imaged surveillance camera apparatus according to claim 1, characterized in that a means. The light modulation image generation means shortens a period for generating a light modulation image when a moving object is detected by the moving object detection means compared to when a moving object is not detected. The surveillance camera device according to claim 2 . The image processing means detects a specular reflection part in which the pixel value is equal to or greater than a predetermined threshold in the light modulation image, and the selection means detects the specular reflection part and the feature amount of the subject is sufficient. it excludes the light modulated image from the selection target when it is unable to extract the surveillance camera apparatus according to claim 1, wherein. Modulated light irradiation means for irradiating modulated light to the target area, and light modulation that receives reflected light reflected by a subject existing in the target area and uses only the reflected light component corresponding to the modulated light among the reflected light as a pixel value A light modulation image generation unit that generates an image, an image processing unit that extracts a feature amount of the subject by performing image processing on the light modulation image, and a predetermined period based on the extraction result of the feature amount by the image processing unit Selecting means for selecting an optimum frame for identifying the subject from the generated light modulation images of a plurality of frames;
The image processing means detects a specular reflection part in which the pixel value is equal to or greater than a predetermined threshold in the light modulation image, and the selection means detects the specular reflection part and the feature amount of the subject is sufficient. monitoring camera apparatus characterized by excluding the light modulated image from the selection target when it can not extract the. In the case where a person seated in the driver's seat of the automobile is a subject, the image processing means estimates the position of the driver's seat of the automobile by performing image processing on the light modulation image when the feature amount of the subject cannot be extracted. 6. The monitoring camera device according to claim 4 , wherein pixel information of a pixel area corresponding to the position of the driver's seat is corrected between the light modulation images of a plurality of frames.

Images