JP2009042098A - Object detector of image processing type - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detector of image processing type performing object detection using a photographed image obtained from an imaging apparatus, wherein even if a detection object receives external light, the object detection then is performed accurately. <P>SOLUTION: An illumination device 6 intermittently irradiates the detection object 1 with artificial lighting. A camera 5 photographs an image without floodlight during a period, when the illumination device 6 does not irradiate the detection object 1 with the artificial lighting, and photographs an image with floodlight during a period, when the illumination device 6 irradiates the detection object 1 with the artificial lighting. A controller 3 derives a floodlight differential image that is the difference between the image without floodlight and the image with floodlight; calculates a correction image in which luminance unevenness that may arise in the photographed image, when a part of the detection object 1 receives the external light 10 is suppressed low, by using the image without floodlight or the floodlight differential image; and performs processing for successively extracting the detection object 1 from the correction image, each time a photographed image is acquired. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing type object detection apparatus that uses a captured image obtained from an imaging device such as a camera and performs object detection by image processing that extracts a specific image reflected in the captured image.

  2. Description of the Related Art Conventionally, there is an image processing type object detection device that detects a specific detection target by performing image processing with an imaging device such as a camera and extracting a specific image reflected in a captured image obtained at that time. There are various types of this type of image processing type object detection device. One type of the image processing type object detection device is when the detection target 82 is not illuminated by the illumination device 83 as in the image processing type object detection device 81 shown in FIG. An image (hereinafter referred to as a non-projected image 84) is captured by the camera 85, and an image (hereinafter referred to as a projected image 86) when the detection target 82 is illuminated by the illumination device 83 is captured by the camera 85. There is an image difference formula that extracts a light projection difference image 87 that is a difference between the images 84 and 86 and detects a detection target 82 from the light projection difference image 87.

  In this type of image difference formula, the camera 85 captures an image not only on the detection target 82 but also on the background 88 thereof. By the way, since the illumination device 83 is arranged for the purpose of illuminating the detection target 82, the distance between the detection target 82 and the illumination device 83 is shorter than the distance between the illumination device 83 and the background 88. Is inevitable. Therefore, the amount of light from the lighting device 83 to the detection target 82 is larger than the amount of light from the lighting device 83 to the background 88, and when the lighting device 83 irradiates the detection target 82 and the background 88, detection is performed as a result. The object 82 is illuminated brighter than the background 88. For this reason, since the brightness difference of the projection difference image 87 becomes large at a location corresponding to the detection target 82, an image region in which the brightness difference is equal to or greater than a threshold is recognized as a detection target image 89 in which the detection target 82 is reflected. Then, the object detection is performed by extracting the image 89 as the detection target 82.

In this manner, in the image difference type image processing type object detection device 81, the background 88 located far from the camera 85 and the illumination device 83 is easily removed from the image, so that the image is not included in the captured image. Only the detection target image 89 reflected as the detection target 82 can be easily extracted. Therefore, an object detection method using this kind of image difference formula is widely used as a detection method when performing object detection using image data captured by the camera 85. An example of this type of image difference type image processing object type detection device is disclosed in, for example, Japanese Patent Application Laid-Open No. H10-228707.
Japanese Patent Laid-Open No. 9-259278

  By the way, the detection target 82 that is the object to be photographed by the camera 85 may be illuminated not only by the illumination device 83 but also by external light 90 such as sunlight as shown in FIG. Depending on the degree of the sheath irradiation angle, it may be assumed that a strong external light 90 hits a part of the detection target 82. As described above, when the external light 90 that is partially strong is irradiated to the detection target 82, the external light 90 is irradiated to the detection target 82 a that appears in the non-projected image 84 as shown in FIG. 10. The luminance (concentration value) of the portion where the luminance is relatively high (hereinafter referred to as the external light irradiation unit 91) is the portion where the luminance is relatively low where the external light 90 is not irradiated (hereinafter, the external light non-existing portion). This will be referred to as the irradiation unit 92).

  Similarly, as shown in FIG. 10, the projected image 86 obtained when the detection target 82 is artificially illuminated by the illumination device 83 is increased in the external light non-irradiation unit 92 than in the external light irradiation unit 91. Has a characteristic of increasing brightness. That is, the amount of change in luminance between the non-projected image 84 and the projected image 86 in the external light irradiation unit 91 and the luminance between the non-projected image 84 and the projected image 86 in the external light non-irradiated unit 92. When comparing with the amount of change of the external light, the luminance change amount of the external light non-irradiation unit 92 is larger than the luminance change amount of the external light irradiation unit 91. This is because, for example, when a small amount of light is applied to a dark place, it looks bright even with a small amount of light if it is originally dark, and on the other hand, even if a light is applied to a bright place, the increase in brightness feels dull. It can be seen from the phenomenon.

  By the way, when an image difference formula is used in the image processing type object detection device 81, a projection difference image 87 between the non-projection image 84 and the projection image 86 is obtained when detecting an object from the photographed image. Although the detection is performed, the light projection difference image 87 is a simple difference between the two images 84 and 86, and thus the light projection difference image 87 is also affected by the external light 90 in the same manner as the images 84 and 86. It becomes an image. That is, as illustrated in FIG. 10, in the detection target 82 c reflected in the light projection difference image 87, the external light irradiation unit 91 irradiated with the external light 90 is not irradiated with the external light 90. The luminance (density value) is lower than 92, and the brightness difference image 87 also has nonuniform luminance on one image. In this case, the detection result of the detection target 82 is different depending on the presence or absence of the external light 90, so that there is a problem that the object detection cannot be performed with high accuracy by the method of taking a simple image difference.

  An object of the present invention is to perform object detection using a photographed image obtained from a photographing device. For example, even if a detection target receives external light, the object detection at that time can be performed with high accuracy. To provide an apparatus.

  In order to solve the above problems, in the present invention, at least the detection target can be photographed by the photographing means, and at the time of photographing, at least the detection target can be illuminated by the illumination means, and the detection means is defined by the illumination means. The non-light-projecting image captured by the photographing means when not illuminated and the light-projected image photographed by the photographing means when the detection target is illuminated by the illumination means are continuously acquired, and the non-light-projecting In the image processing type object detection device that detects the detection target from the captured image using a projected difference image that is a difference between the image and the projected image, the captured image captured by the capturing unit, and the captured image Even if external light is partially irradiated to a part of the detection target that is a subject of the photographing unit using the light projection difference image derived from the image, it is caused by the external light in the photographed image. A corrected image creating means for creating a corrected image with reduced brightness non-uniformity, and the detection target by extracting an image area corresponding to the detection target from the corrected image created by the corrected image creating means And extracting means for detecting from the photographed image.

  According to this configuration, when the imaging unit captures the detection target, the illumination unit alternately turns on and off the artificial illumination on the detection target, and the imaging unit captures the detection target when the illumination unit does not artificially illuminate the detection target. Thus, the non-projected image is captured, and the projected image is captured by capturing the detection target when the illumination unit artificially illuminates the detection target. At this time, the corrected image creating means uses the photographed image (for example, a non-projected image) photographed by the photographing means and a projected difference image that is a difference between the non-projected image and the projected image. Even if a part of the detection target is partially irradiated with external light, a corrected image in which this influence is suppressed to be low in the captured image is created. Then, the extraction unit detects the detection target from the photographed image of the photographing unit by cutting out an image region corresponding to the detection target from the corrected image.

  For this reason, when the detection method for detecting the detection target from the captured image of the imaging unit as in the present configuration is taken, the captured image captured by the imaging unit is assumed to be, for example, sunlight on a part of the detection target that is the subject of the imaging unit. Even if the external light is partially irradiated, the corrected image processing means adds image processing to the captured image, thereby reducing the brightness non-uniformity of the captured image due to the external light irradiation to a corrected image. It is corrected. Therefore, the detection target appears in the captured image captured by the capturing unit with less uneven brightness, and this can be accurately performed when detecting the detection target from the captured image of the capturing unit.

  In the present invention, the corrected image creating means includes the luminance of the external light irradiation unit that is an image region in which the external light is reflected in the non-projected image and an external image that is an image region in which the external light is not reflected. The difference between the luminance of the light non-irradiation part and the luminance of the external light irradiation part which is the image area in which the external light is reflected in the projection difference image and the image area in which the external light is not reflected By dividing the difference between the brightness of a certain external light non-irradiated portion and obtaining a constant that can suppress the brightness non-uniformity on the photographed image due to the external light, one pixel of the projected difference image Multiplying the constant and adding this to one pixel of the non-projected image corresponding to the one pixel is performed for each pixel of the photographed image, and an image composed of these pixel groups is created as the corrected image The gist is to do.

  According to this configuration, the constant necessary for creating the corrected image is calculated by calculating the luminance difference between the various images and dividing this luminance difference, and the corrected image derived using this correction value is: It is created by multiplying one pixel of the projected difference image by a constant and adding this to one pixel of the non-projected image corresponding to the one pixel for each pixel. Therefore, the correction image can be created by a simple calculation of adding, subtracting, and dividing various numerical values.

The gist of the present invention is that the luminance is an average value of luminance output by each detection unit of the photographing unit in the corresponding image area of the irradiation unit.
According to this configuration, while calculating the luminance average of the image area of the corresponding irradiation unit (external light irradiation unit, external light non-irradiation unit) in various images, the constant is calculated using this luminance average, It is possible to derive the luminance of the external light irradiation unit and the external light non-irradiation unit more accurately. Therefore, if a corrected image is created using constants having high characteristics in this way, the accuracy of the corrected image can be made higher, and detection when detecting a detection target from the captured image of the imaging means. The accuracy can be further improved.

  According to the present invention, when performing object detection using a photographed image obtained from a photographing device, for example, even if the detection target receives external light, the object detection at that time can be accurately performed.

Hereinafter, an embodiment of an image processing type object detection apparatus embodying the present invention will be described with reference to FIGS.
As shown in FIG. 1, for example, as a detection device for detecting a detection target 1 such as a vehicle driver, an image processing type object detection device for detecting the detection target 1 itself or its movement from a captured image taken by an imaging device. There are two. The image processing type object detection device 2 includes a controller 3 that functions as a control unit that performs overall control of the image processing type object detection device 2, a camera 5 that can photograph the detection target 1 and its background 4, and the detection target 1 and its An illumination device 6 capable of irradiating the background 4 is provided. The camera 5 corresponds to the photographing unit, and the lighting device 6 corresponds to the lighting unit.

  The camera 5 forms an image on the imaging surface 8 through the lens 7 so that the light emitted from the detection target 1 is recognized in order to recognize the image of the detection target 1, and converts the brightness level on the imaging surface 8 into an electrical signal and outputs the electrical signal. The camera 5 has a plurality of image sensors 9, 9... Arranged in parallel at the location of the imaging surface 8, and each pixel of the image sensors 9, 9. An electric signal having a value corresponding to the luminance of the image is output as image data Dpc. When the camera 5 is a color type, there are a three-plate type in which light is divided into RGB by a prism, a single-plate type in which color filters are arranged for each of the image sensors 9, 9. The camera 5 is an analog video output method in which the output format of the image data Dpc is an analog signal as a video output method.

  When the camera 5 receives the shooting request signal Skp from the controller 3 as shown in FIG. 2, the camera 5 takes an image with the signal input timing as the shutter timing, outputs the shot image Pic to the controller 3, and receives the shooting request signal Skp. Each time an image is input, an image is captured and the captured image is output, and the captured image Pic is continuously output to output a moving image signal Sdp to the controller 3. Note that the photographed image Pic is a group of image data Dpc expressed as one image when the image data Dpc is displayed on the screen for convenience. By the way, if the imaging request signal Skp is output from the controller 3 to the camera 5 with multiple pulses per unit time, the camera 5 outputs the captured image Pic to the controller 3 at a short interval. The shorter the number, the larger the number of captured images within a unit time, and the more continuous video can be obtained. The time interval W is generally determined to have a minimum value, for example, 1/30 seconds is generally used.

  Further, the image processing type object detection device 2 is provided with an illumination device 6 capable of irradiating the detection target 1 over the background 4 with artificial illumination when the camera 5 captures the detection target 1. The illumination device 6 is disposed at a position adjacent to the camera 5 and uses, for example, an LED or a lamp as a light source. The illumination device 6 uses pulse illumination in which on / off of illumination is switched corresponding to the shutter timing of the camera 5 when the camera 5 captures the detection target 1 (including the background 4). When the camera 5 captures an image by the interlace method, the detection target 1 is irradiated by switching on / off for each even / odd field in the interlace method. The illumination intensity of the illuminating device 6 is set to such an intensity that the brightness does not matter even if the irradiation operation is performed on the detection target 1 when the detection target 1 is not irradiated with the external light 10.

  The controller 3 is connected to the camera 5 via a first cable 11 that is a transmission path for various data of the camera system, and performs overall control of the camera 5 via this cable 11. The controller 3 is connected to the lighting device 6 via the second cable 12 that is a transmission path for various data of the lighting device system, and controls the lighting device 6 through the cable 12. The controller 3 includes various devices such as a CPU 13 and a memory 14 (ROM, RAM), for example, and is unitized so that these component groups can be handled as one component.

  The memory 14 of the controller 3 stores an image processing program Dpr for detecting the detection target 1 by performing image processing on the image data Dpc acquired from the camera 5 and extracting the detection target 1 from the image data Dpc. The image processing program Dpr in this example includes a non-projection image 15 (see FIG. 4) obtained when the detection target 1 (including the background 4) is photographed by the camera 5 when the illumination device 6 is turned off, and the illumination device. 6 is obtained, and a projection image 16 (see FIG. 5) obtained when the detection target 1 (including the background 4) is photographed by the camera 5 when the illumination is turned on is obtained. An optical difference image 17 (see FIG. 6) is extracted, and a corrected image 18 in which luminance non-uniformity (image unevenness) that can occur in the photographed image Pic when the detection target 1 receives the external light 10 as a part thereof is suppressed to a low level. (See FIG. 7) is a program for calculating the projection difference image 17 and extracting the detection target 1 from the corrected image 18 continuously every time a captured image is acquired. Note that the non-projected image 15 and the projected image 16 constitute a captured image.

  As shown in FIG. 3, the controller 3 receives from the system control unit 19 a system control unit 19 that controls the entire image processing type object detection device 2 and image data Dpc taken by the camera 5, and performs image processing on this. In addition, an image processing unit 20 that detects the detection target 1 reflected in the image data Dpc is provided. The system control unit 19 and the image processing unit 20 are functionally generated by the cooperation of the CPU 13 and the image processing program Dpr. In this example, this is expressed by a block diagram. Yes. The image processing unit 20 constitutes a corrected image creating unit and an extracting unit.

  When the system control unit 19 outputs a shooting request signal Skp to the camera 5 in accordance with the image processing program Dpr and causes the camera 5 to take an image, the system control unit 19 takes an image by the camera 5 using an interlace method as its shooting scanning method. The interlace method is to reduce the flickering of the screen with respect to the eyes when displaying the image of the camera 5 on the screen, and scans every other horizontal scanning line while scanning the camera and repeats this twice. This is a scanning method that completes a single screen.

  Further, when the system control unit 19 causes the illumination device 6 to perform artificial illumination according to the image processing program Dpr, the detection target 1 is not artificially illuminated by switching the illumination device 6 on and off for each even-odd field in the interlace method. The non-projected image 15 that is the captured image Pic of the camera 5 at the time and the projected image 16 that is the captured image Pic of the camera 5 when the detection target 1 is artificially illuminated are acquired, and these images 15 and 16 are obtained. The image is output to the image processing unit 20. The system control unit 19 causes the camera 5 to perform a shooting operation at a short time interval so that the non-projected image 15 and the projected image 16 acquired at the time of shooting by the camera 5 are substantially the same image, and the non-projected image The image 15 and the projected image 16 are acquired continuously.

  When the image processing unit 20 acquires the non-projected image 15 and the projected image 16 from the system control unit 19, the image processing unit 20 derives a projected difference image 17 that is a difference between the images 15 and 16. When the image processing unit 20 derives the light projection difference image 17, even if the external light 10 is partially irradiated on the detection target 1 on the captured image Pic of the camera 5 and the brightness nonuniformity occurs on the captured image Pic. Then, correction is made to the captured image Pic to newly create a corrected image 18 in which the uneven brightness is not noticeable. Then, the image processing unit 20 extracts an image area where the detection target 1 appears from the corrected image 18 and detects this as the detection target 1.

Next, a procedure for creating the corrected image 18 in the image processing type object detection device 2 of this example will be described.
By the way, depending on the irradiation angle of the external light 10 such as sunlight or indoor light, a part of the detection target 1 is irradiated by the external light 10 and the amount of light irradiated on the detection target 1 is partially uneven. There is a case. At this time, as shown in FIG. 1, the detection target 1 that is the subject of the camera 5 includes an external light irradiation spot 21 (non-decorative area in FIG. 1) where the external light 10 is irradiated on the detection target 1, and an external The brightness differs depending on the external light non-irradiation point 22 (point area in FIG. 1) where the light 10 is not irradiated.

  Therefore, when the non-projected image 15 is acquired by the camera 5, as shown in FIG. 4, as shown in FIG. The luminance of the irradiation unit 23 (hereinafter referred to as non-projected image light no-side luminance La1) is the image area (hereinafter referred to as the non-projected image 15) where the external light 10 is illuminated by the detection target 1. It becomes a state that takes a value lower than the luminance of the external light irradiation unit 24 (denoted by the non-projected image light presence luminance La2).

  The same can be said for the projected image 16 as shown in FIG. 5. As shown in FIG. 5, the brightness of the external light non-irradiation part 23 in the projected image 16 (hereinafter referred to as the projected image light no-side brightness Lb1). Is a state that takes a value lower than the luminance of the external light irradiation unit 24 in the projected image 16 (hereinafter, referred to as the projected image light presence luminance Lb2). When the illumination device 6 artificially illuminates the detection target 1, the luminance of the external light non-irradiation unit 23 (projected image light non-side luminance Lb1) and the luminance of the external light irradiation unit 24 (projected image light present) The side luminance Lb2) is not necessarily the same luminance, and the brightness relationship between the originally dark portion and the originally bright portion does not change even when artificial lighting is applied. And a brightness difference Lb2 with the projected image light. Further, when the illumination device 6 artificially illuminates the detection target 1, the luminance does not increase with the same increase amount together with the external light non-irradiation unit 23 and the external light irradiation unit 24, but as described in the background art, The amount of increase in luminance in the external light non-irradiation unit 23 is the same as when light is applied to a dark place, even if a slight amount of light appears bright, and it does not feel so bright when light is originally applied to a bright place. This is larger than the increase in luminance in the external light irradiation unit 24.

  For this reason, when the projection difference image 17 which is the difference between the non-projection image 15 and the projection image 16 is taken in a state where the external light 10 is partially irradiated on the detection target 1, FIG. As shown, the brightness of the external light non-irradiation unit 23 in the light projection difference image 17 (hereinafter referred to as the light projection image light no-side luminance Lc1) is the brightness of the external light irradiation unit 24 in the light projection difference image 17 (hereinafter referred to as the following). The brightness is higher than the brightness Lc2 of the projected image light. That is, in the light projection difference image 17, unlike the non-light projection image 15 and the light projection image 16, an image having a luminance difference in which the external light non-irradiation unit 23 appears brighter than the external light irradiation unit 24. It becomes.

  Therefore, when the external light irradiation unit 24 is viewed, the non-projected image light-side luminance La2 that is the luminance of the non-projected image 15 takes a relatively high value and is the luminance of the projected difference image 17. The brightness Lc2 with the projected image light is relatively low. When the external light non-irradiation unit 23 is viewed, the non-projected image light no-side luminance La1 that is the luminance of the non-projected image 15 takes a relatively low value, and the luminance of the projected difference image 17 A certain projected image light no-side luminance Lc1 takes a relatively high value. Therefore, the relationship between the luminance of the non-light-projected image 15 in the external light irradiation unit 24 and the luminance of the light-projected differential image 17, and the luminance of the non-light-projected image 15 in the external light non-irradiated unit 23 and the light-projected differential image 17. It can be seen that the relationship between the brightness and the brightness is in contrast. That is, a bright image area in the non-projected image 15 becomes a dark image area in the projected difference image 17, and a dark image area in the non-projected image 15 becomes a bright image area in the projected difference image 17.

  Therefore, by utilizing the property that the luminance is opposite in magnitude, as shown in the following equation (1), the luminance of a predetermined pixel in the non-projected image 15 is set as the non-projected image luminance La, and the light is projected. When the luminance of a predetermined pixel in the difference image 17 is the projection difference image luminance Lc and the constant necessary for obtaining the corrected image 18 in which the luminance non-uniformity due to the external light 10 is eliminated is C, the projection difference A new brightness Lx is calculated by adding a correction value Px obtained by multiplying the image brightness Lc by a constant C to the non-projection image brightness La of the corresponding pixel, and the captured image Pic caused by the external light 10 is calculated. The corrected image 18 in which the luminance non-uniformity is suppressed to a low level is created. The new luminance Lx is obtained as the luminance corrected to be significantly brighter in the dark portion of the non-projected image 15 for each pixel of the captured image Pic, and in the bright portion of the non-projected image 15. This is a value obtained as the brightness corrected brightly only by a small amount.

New brightness Lx = non-projection image brightness La + (constant C × projection difference image brightness Lc) (1)
(However, constant C × light projection difference image luminance Lc → correction value Px)
Therefore, the image in which the new luminance Lx obtained for each pixel of the captured image Pic is gathered is that the image (pixel) of the non-irradiated portion of the external light 10 is greatly corrected for brightness and the external light 10 is irradiated. The partial image (pixel) should be an image whose brightness is corrected by a small amount, that is, an image in which the influence of the external light 10 is suppressed to a low level. Accordingly, when the image processing unit 20 acquires the non-projected image 15 and the projected image 16 to be paired from the system control unit 19, the image processing unit 20 calculates a new luminance Lx for each pixel and includes the collection of the luminance Lx. The image is calculated as a corrected image 18, and the corrected image 18 is calculated each time the non-projected image 15 and the projected image 16 forming a set are acquired.

  Further, the constant C is a new luminance Lx of the external light non-irradiation part 23 in the captured image Pic and the external light in the captured image Pic in order to make the corrected image 18 an image with less luminance nonuniformity due to the external light 10. It is necessary to set the value so that the new luminance Lx of the irradiation unit 24 takes the same level. Therefore, the average value of the external light irradiating unit 24 in the non-projected image 15 is set as the non-projected image bright luminance average Ka1, and the average value of the external light non-irradiating unit 23 in the non-projected image 15 is set as the non-projected image dark luminance average. Ka2, the average value of the external light non-irradiation unit 23 in the light projection difference image 17 is the light projection difference image bright luminance average Kb1, and the average value of the external light irradiation unit 24 in the light projection difference image 17 is the light projection difference image dark luminance. Assuming average Kb2, the constant C is calculated by the following equation (2).

Constant C = (Ka1-Ka2) / (Kb1-Kb2) (2)
That is, the constant C is the difference (absolute value) between the non-projected image bright brightness average Ka1 and the non-projected image dark brightness average Ka2, and the projected difference image bright brightness average Kb1 and the projected difference image dark brightness average. It is the value divided by the difference (absolute value) from Kb2. In other words, when the luminance value of each corresponding pixel on the projection difference image 17 is added to each pixel of the non-projection image 15, the constant C is obtained by adding the added luminance to the original projection difference. This corresponds to a coefficient that determines how many times the pixel luminance on the image 17 is added.

  The non-projected image bright luminance average Ka1 is a value that is greater than or equal to the first threshold value α in the non-projected image 15 in the entire image area of the non-projected image 15 and has a constant luminance in the projected difference image 17. It is obtained by deriving the average luminance of the image area where pixels equal to or larger than β are gathered. That is, here, by extracting an image area having a luminance equal to or higher than the first threshold value α in the non-projected image 15, an image area irradiated with the external light 10 is extracted from the non-projected image 15. In the detection object 1 reflected in the non-projection image 15, the image area in which the detection object 1 is reflected is extracted from the light projection difference image 17 by cutting out the image area having the luminance equal to or higher than the second threshold value β. The image area of the external light irradiation unit 24 (the thin dot area in FIG. 4) is extracted, and the average luminance of the image area is calculated.

  Further, the light projection difference image dark luminance average Kb2 is equal to the first threshold value α in the non-light projection image 15 as in the case of calculating the non-light projection image bright luminance average Ka1 in the entire image area of the light projection difference image 17. As described above, it is obtained by deriving the luminance average of the image area in which the pixels having the luminance of the fixed value β or more in the light projection difference image 17 are gathered. That is, here, the image area of the external light irradiation unit 24 (dark dot area in FIG. 6) in the detection target 1 reflected in the image 17 is extracted from the light projection difference image 17 and the average luminance of the image area is calculated. is doing.

  Further, the non-light-projected image dark luminance average Ka2 derives the luminance average of the image area in which the pixels having the luminance of the third threshold value γ or more in the light-projected difference image 17 are gathered in the entire image area of the non-light-projected image 15. It is obtained by. That is, in this case, the external light non-irradiation unit is detected in the detection target 1 reflected in the non-projected image 15 by cutting out the image region having the luminance equal to or higher than the third threshold γ in the projected difference image 17 in the non-projected image 15. The image area 23 (the dark dot area in FIG. 4) is extracted, and the average luminance of the image area is calculated.

  Further, the light projection difference image bright luminance average Kb1 derives the luminance average of the image region in which the pixels having the luminance equal to or higher than the third threshold value γ in the light projection difference image 17 in the entire image region of the light projection difference image 17. It is determined by that. That is, here, by cutting out an image region having a luminance equal to or higher than the third threshold value γ in the light projection difference image 17, the image region of the external light non-irradiation part 23 (the thin dot region in FIG. 6) is extracted from the light projection difference image 17. The average luminance of the image area is calculated by extracting. Note that the background 4 that appears behind the detection target 1 has a very small light projection difference value, and thus is not included in the luminance averages Ka1, Ka2, Kb1, and Kb2.

Next, the operation of the image processing type object detection device 2 configured as described above will be described.
For example, when a camera shooting start request operation is performed, for example, when a start start switch of the image processing type object detection device 2 is turned on, the system control unit 19 executes the image processing program Dpr and starts the start. . At this time, the system control unit 19 outputs the photographing request signal Skp to the camera 5 to cause the camera 5 to photograph the detection target 1 and its background 4. The camera 5 captures an image every time it acquires a shooting request signal Skp from the system control unit 19, and outputs the moving image signal Sdp by continuously outputting the captured image Pic to the system control unit 19.

  In addition, when the system controller 19 causes the camera 5 to capture the detection target 1 (including the background 4), the illumination device 6 is switched on and off for each even / odd field in the interlace method, and the detection target 1 is artificially illuminated. give. Therefore, the system control unit 19 acquires the captured image Pic obtained from the camera 5 as the non-projected image 15 when the illumination device 6 is not artificially illuminating the detection target 1, and the illumination device 6 artificially illuminates the detection target 1. Sometimes, a captured image Pic obtained from the camera 5 is acquired as a projected image 16, and these images 15 and 16 are output to the image processing unit 20. Note that the non-projected image 15 and the projected image 16 are images taken at a very short shutter interval, and therefore take substantially the same image.

  When the image processing unit 20 obtains a pair of the non-projected image 15 and the projected image 16 from the camera 5, the image processing unit 20 derives a projected difference image 17 that is a difference between the images 15 and 16. Subsequently, the image processing unit 20 that has derived the light projection difference image 17 performs calculation processing using the above formulas (1) and (2), and obtains a new luminance for each pixel of the captured image Pic obtained from the camera 5. Lx is calculated. As a calculation process performed by the image processing unit 20 at this time, first, the image processing unit 20 calculates a constant C using the above equation (2). That is, the image processing unit 20 uses the non-projected image 15 and the projected difference image 17 to use the non-projected image bright luminance average Ka1, the non-projected image dark luminance average Ka2, and the projected difference image bright luminance average Kb1. The light projection difference image dark luminance average Kb2 is calculated, and the constant C is calculated from these luminance averages Ka1, Ka2, Kb1, and Kb2.

  The image processing unit 20 that has calculated the constant C then obtains a new luminance Lx for each pixel in the captured image Pic acquired from the camera 5 using the above equation (1). That is, the image processing unit 20 multiplies the light projection difference image 17 by a constant C by the light projection difference image luminance Lc for each pixel, and calculates the correction value Px derived for each pixel by this multiplication as a non-light projection. By adding to the corresponding pixels in the image 15, a new luminance Lx is calculated for each pixel in the captured image Pic of the camera 5. The image processing unit 20 recognizes an image including a pixel group having the new luminance Lx calculated as described above as the corrected image 18.

  As shown in FIG. 7, the corrected image 18 is an image in which a portion irradiated with the external light 10 is reflected even if the external light 10 is generated in a part of the detection target 1 that is a photographing target of the camera 5. A small amount of brightness correction is performed on the portion, and a large amount of brightness correction is performed on the image portion in which the portion that is not irradiated with the external light 10 is reflected. It is created as an image in which the luminance non-uniformity caused by is eliminated. For this reason, the detection target 1 reflected in the corrected image 18 does not take, for example, an image in which a part is extremely bright or dark, but appears in a state where the entire detection target 1 can be clearly seen in normal light and dark. become.

  When the corrected image 18 is created, the image processing unit 20 detects the detection target 1 from the photographed image Pic of the camera 5 by extracting the detection image 1 as a detection target 1 in the corrected image 18. . Each time the image processing unit 20 acquires the non-projected image 15 and the projected image 16 that form a pair, the image processing unit 20 creates the corrected image 18 and extracts an image region in which the detection target 1 is reflected from the corrected image 18. Processing to detect the detection target 1 from the captured image Pic is performed. The image processing unit 20 recognizes the position of the detection target 1 by detecting the detection target 1 from the captured image Pic of the camera 5, or detects the movement of the detection target 1 by looking at the temporal position change of the detection target 1. To recognize.

  Accordingly, in this example, every time the paired non-projected image 15 and the projected image 16 are obtained from the camera 5, the projected difference image 17 which is the difference between these images 15 and 16 is derived and the above equation is obtained. Using (1) and (2), a corrected image 18 in which the influence of the external light 10 is kept low is created, and the image portion of the detection target 1 is extracted from the corrected image 18 to detect the detection target 1. For this reason, for example, even if a part of the detection target 1 is irradiated with the external light 10, the corrected image 18 finally extracted from the captured image Pic of the camera 5 is an image in which the influence of the external light 10 is suppressed to a low level. Derived. Therefore, since the final captured image Pic (corrected image 18) obtained from the camera 5 is an image in which the detection target 1 is clearly reflected, the detection target 1 is reflected in the captured image Pic more accurately. It becomes possible to detect the detection target 1 from the captured image Pic with high accuracy.

According to the configuration of the present embodiment, the following effects can be obtained.
(1) A non-projection image 15 photographed by the camera 5 when the illumination target 6 does not artificially illuminate the detection target 1 and a projection image 16 photographed by the camera 5 when the illumination target 6 artificially illuminates the detection target 1 A light projection difference image 17 that is a difference is obtained, and the light projection difference image 17 and the non-light projection image 15 are used to cause a part of the detection target 1 to be irradiated with the external light 10. A constant C that can suppress the brightness non-uniformity of the captured image Pic is calculated, and the original captured image Pic is corrected using the constant C, thereby correcting the brightness non-uniformity on the image to a low level. Create For this reason, the corrected image 18 is an image from which the influence of the external light 10 has been removed, so that the detection target 1 is clearly reflected in the corrected image 18 with little luminance nonuniformity. Therefore, when detecting the detection target 1 from the captured image Pic of the camera 5, if the detection target 1 is extracted from the corrected image 18 after image processing is applied to the captured image Pic, the detection target 1 is accurately detected. Can be detected.

  (2) The constant C necessary for creating the corrected image 18 is the difference between the luminance average Ka1 of the external light irradiation unit 24 and the luminance average Ka2 of the external light non-irradiation unit 23 in the non-projected image 15; It is calculated by dividing the difference between the luminance average Kb1 of the external light non-irradiation unit 23 and the luminance average Kb2 of the external light irradiation unit 24 in the projected image 16. Further, the corrected image 18 is obtained by multiplying one pixel of the light projection difference image by a constant C and calculating a new luminance Lx in addition to the one pixel corresponding to the non-light projection image 15. It is created by performing each pixel of the image Pic. Therefore, the corrected image 18 can be created by a simple calculation of adding, subtracting, dividing, and dividing various numerical values.

  (3) The constant C is the difference between the luminance of the external light irradiation unit 24 and the luminance of the external light non-irradiation unit 23 in the non-projection image 15, and the luminance of the external light non-irradiation unit 23 in the projection difference image 17. It is calculated by dividing the difference between the luminance of the external light irradiation unit 24 and the luminance of each irradiation unit 23, 24 used at this time is the luminance average Ka1, Ka2, Kb1, Kb2 in each image region. Is used. Therefore, this type of average value (brightness average Ka1, Ka2, Kb1, Kb2) is a value that more suitably reflects the luminance of each irradiation unit 23, 24 in each image 15, 17, and thus the luminance average Ka1. , Ka2, Kb1, and Kb2, the constant C can be calculated with high accuracy, and the corrected image 18 can be created with higher accuracy.

  (4) Since the corrected image 18 is created every time the non-projected image 15 and the projected image 16 forming a set are photographed, the corrected image 18 is sequentially created every time the image is acquired. Therefore, when the detection target 1 is detected from the captured image Pic of the camera 5, the correction image 18 is always used. Therefore, when the detection target 1 is detected from the captured image Pic of the camera 5, this is highly accurate. Can be done.

Note that the embodiment is not limited to the configuration described so far, and may be modified as follows.
The constant C used to create the corrected image 18 is the luminance difference between the external light irradiation unit 24 and the external light non-irradiation unit 23 of the non-projected image 15, and the external light irradiation unit 24 and the external light of the light projection difference image 17. It is not limited to the value obtained by dividing the luminance difference from the non-irradiation part 23. For example, the luminance difference between the external light irradiation unit 24 and the external light non-irradiation unit 23 of the projected image 16 and the luminance difference between the external light irradiation unit 24 and the external light non-irradiation unit 23 of the light projection difference image 17 are divided. The brightness difference between the external light irradiation unit 24 and the external light non-irradiation unit 23 of the non-projected image 15 and the luminance between the external light irradiation unit 24 and the external light non-irradiation unit 23 of the projected image 16 may be used. It may be a value divided by the difference.

  When the constant C used to create the corrected image 18 is calculated by dividing the luminance difference between the irradiation units 23 and 24 in each of the images 15 and 17, the luminance of each irradiation unit 23 and 24 at this time is not necessarily It is not limited to the average brightness of the image area in the irradiation units 23 and 24. For example, the luminance at this time may be determined by specifying the pixel having the highest luminance among the pixels in the image areas of the irradiation units 23 and 24, and the constant C may be calculated using this luminance.

-When calculating the constant C, α, β, γ are used as thresholds, but the values of the thresholds α, β, γ may be freely changed.
Various types of cameras 5 can be used. For example, a CCD camera using a CCD as the image sensor 9 or a CMOS camera using a CMOS sensor as the image sensor 9 may be used.

  The image processing type object detection device 2 is not limited to detecting the vehicle driver, for example, as the detection target 1, and it is necessary to detect the detection target 1 by image processing from the captured image Pic taken by the camera 5. It can be widely applied to devices and equipment.

Next, technical ideas that can be grasped from the above-described embodiment and other examples will be described below together with their effects.
(1) In any one of claims 1 to 3, the corrected image creating means is configured to calculate at least two of the non-light-projected image, the light-projected image, and the light-projected difference image, and the external light. The correction image is created by obtaining a constant capable of suppressing the luminance non-uniformity on the photographed image due to the image and correcting the photographed image photographed by the photographing means using the constant. Also with this configuration, it is possible to obtain the effects specific to the present invention in the same manner.

  (2) In any one of claims 1 to 3, the correction image creation means sequentially creates the correction image every time the non-light-projected image and the light-projected image that the photographing means forms a set. . According to this configuration, since a suitable corrected image suitable for the time is created, it becomes possible to detect the detection target with higher accuracy.

1 is a configuration diagram illustrating a schematic configuration of an image processing type object detection device according to an embodiment. The wave form diagram which shows the waveform of the various signals which a controller communicates with a camera. The block diagram which expressed the image processing type object detection apparatus with the functional block. The image figure of the non-projection image image | photographed with a camera. The image figure of the light projection image image | photographed with a camera. The image figure of the light projection difference image which is a difference between a non-light projection image and a light projection image. The image figure of the correction image by which the brightness | luminance nonuniformity was reduced by correct | amending a picked-up image. The block diagram which shows the specific structure of the image processing type object detection apparatus in the past. The image figure which shows the state by which external light was irradiated to a part of detection object. The conceptual diagram which shows the magnitude relationship of the brightness | luminance in various images.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Detection object, 2 ... Image processing type object detection apparatus, 5 ... Camera as imaging | photography means, 6 ... Illumination device as illumination means, 10 ... External light, 15 ... Non-projection image which comprises a picked-up image, 16 ... Projected image constituting a projected image, 17... Projected difference image, 18... Corrected image, 20... Image processing unit constituting corrected image creating means and extracting means, 23. Part, Pic ... photographed image, C ... constant.

Claims (3)

At least the detection target can be photographed by the photographing means, and at the time of photographing, at least the detection target can be illuminated by the illumination means, and the non-projection photographed by the photographing means when the detection target is not illuminated by the illumination means. A light image and a projected light that is a difference between the non-projected image and the projected image are obtained by continuously acquiring a light image and a projected image captured by the photographing unit when the detection unit is illuminated by the illumination unit. In the image processing type object detection device that detects the detection target from the captured image using a difference image,
Using the captured image captured by the imaging unit and the light projection difference image derived from the captured image, a part of the detection target that is a subject of the imaging unit is partially irradiated with external light. A corrected image creating means for creating a corrected image in which brightness non-uniformity caused by the external light in the captured image is suppressed, and
An image processing expression comprising: an extraction unit that detects the detection target from the captured image by extracting an image region corresponding to the detection target from the correction image generated by the correction image generation unit. Object detection device.   The corrected image creating means includes an external light non-irradiation unit that is an image region in which the external light is not reflected and luminance of the external light irradiation unit that is an image region in which the external light is reflected in the non-projected image Between the brightness of the external light and the brightness of the external light irradiation unit, which is the image area in which the external light is reflected, in the light projection difference image and the external light non-existence, which is an image area in which the external light is not reflected By dividing the difference between the brightness of the irradiation unit and the brightness, a constant that can suppress brightness non-uniformity on the captured image caused by the external light is obtained, and one pixel of the light projection difference image is multiplied by the constant. Then, adding this to one pixel of the non-projected image corresponding to the one pixel is performed for each pixel of the photographed image, and an image composed of these pixel groups is created as the corrected image. The image processing type object detection device according to claim 1.   3. The image processing type object detection device according to claim 2, wherein the luminance is an average value of luminance output by each detection unit of the photographing unit in the corresponding image area of the irradiation unit.

Images