JP2012095342A - Pseudo-gray-image generating device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To stably obtain an image excellent in visibility.SOLUTION: A first photographing device 12 photographs a front side of one's own vehicle so as to output a visible-light image. A second photographing device outputs a near-infrared image. An image acquiring section 22 acquires the visible-light image and the near-infrared image from the first photographing device 12 and the second photographing device 14. A pseudo-gray-image generating section 24 generates a pseudo gray image from the visible-light image and the near-infrared image by using luminance information of one image having a brighter pixel for each pixel.

Description

  The present invention relates to a pseudo grayscale image generating apparatus and program, and more particularly, to a pseudo grayscale image generating apparatus and program for generating a pseudo grayscale image.

  2. Description of the Related Art Conventionally, a color photographing apparatus that can be used continuously day and night with both daytime color reproducibility and nighttime sensitivity is known (Patent Document 1). In this color photographing apparatus, one G filter among the RGBG4 pixels constituting one unit of a normal Bayer array is replaced with an IR filter, and the RGB filter is assigned to the first mode and the IR filter is assigned to the second mode. ing. In addition, an infrared cut filter is applied to the three pixels of RGB.

  A color image reproduction apparatus that reproduces a color image in which both color information and luminance information are optimal is known (Patent Document 2). In this color image reproducing device, infrared information is removed from visible image data, color information (hue, saturation) is extracted, and synthesized with infrared luminance information to generate a pseudo color image.

Japanese Patent Laid-Open No. 2005-6066 JP 2007-184805 A

  However, in the technique described in Patent Document 1, since the RGB filter and the IR filter are switched and used depending on the mode, when the surroundings are dark such as at night, only the monochrome image using only the IR filter and having low visibility is used. There is a problem that it cannot be provided.

  Further, in the technique described in Patent Document 2, when photographing with a color camera and an infrared camera, if there is brightness such as illumination, the amount of infrared light may be smaller than the amount of visible light. When the infrared brightness is selected, there is a problem that the brightness becomes darker than the visible light brightness and the visibility deteriorates.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a pseudo gray-scale image generation apparatus and program capable of stably obtaining an image with good visibility.

  In order to achieve the above object, a pseudo gray-scale image generating apparatus according to a first aspect of the present invention captures a predetermined region, a visible light image in a first wavelength band including visible light, and a band is the first wavelength band. And an image acquisition unit that acquires the visible light image and the infrared image from an imaging unit that outputs an infrared image of a second wavelength band that is different and includes infrared light, and the visible image acquired by the image acquisition unit For each block obtained by dividing the visible light image and the infrared image into a plurality of blocks from the optical image and the infrared image, the block of the visible light image is compared with the block of the infrared image, and the visible image Of the light image and the infrared image, when the luminance information of the block with the brighter block is equal to or greater than the threshold, the weighted average of the luminance information of the block in the visible light image and the infrared image is used. Pseudo-dark It is configured to include a generating means for generating an image.

  A program according to a second invention is a program for photographing a predetermined area, a visible light image of a first wavelength band including visible light, and a band different from the first wavelength band and including infrared light. From the imaging means for outputting the infrared image of the second wavelength band, the image acquisition means for acquiring the visible light image and the infrared image, and the visible light image and the infrared image acquired by the image acquisition means, For each block obtained by dividing the visible light image and the infrared image into a plurality of blocks, the block of the visible light image is compared with the block of the infrared image, and among the visible light image and the infrared image, When the luminance information of the block of the brighter image is greater than or equal to the threshold value, a raw grayscale image is generated using a weighted average of the luminance information of the block in the visible light image and the infrared image. Is a program for functioning as a unit.

  According to 1st invention and 2nd invention, the imaging | photography means image | photographed the predetermined area | region, the visible light image of the 1st wavelength band containing visible light, and a band differ from the said 1st wavelength band, and An infrared image in the second wavelength band including infrared light is output. A visible light image and an infrared image are acquired from the imaging unit by the image acquisition unit.

  Then, with respect to each block obtained by dividing the visible light image and the infrared image from the visible light image and the infrared image acquired by the image acquiring unit by the generating unit, the block of the visible light image and the infrared When the luminance information of the block of the image having the brighter block among the visible light image and the infrared image is greater than or equal to a threshold value, the visible light image and the infrared image are compared with the block of the image A pseudo grayscale image is generated using the weighted average of the luminance information of the block at.

  As described above, by generating the pseudo gray image using the luminance information of the brighter image for each block from the visible light image and the infrared image, it is possible to stably obtain an image with good visibility. In addition, saturation of the luminance value can be prevented.

  The block can be a pixel unit.

  As described above, according to the pseudo grayscale image generation apparatus and program of the present invention, the pseudo grayscale image is generated from the visible light image and the infrared image using the luminance information of the brighter image for each block. As a result, it is possible to obtain an effect that it is possible to stably obtain an image with high visibility.

It is a block diagram which shows the structure of the imaging | photography system which concerns on the 1st Embodiment of this invention. (A) The graph which shows the brightness | luminance of each pixel of a near-infrared image and a visible light image, and (B) The graph which shows the brightness | luminance of each pixel which selected the higher one. It is a flowchart which shows the content of the pseudo gray image generation process routine in the imaging | photography system which concerns on the 1st Embodiment of this invention. (A) An image diagram showing a visible light image, (B) An image diagram showing a near-infrared image, and (B) An image diagram showing a pseudo gray image. It is a graph which shows the brightness | luminance of the pixel which selected the higher one about each pixel, or the average of a pixel. It is a flowchart which shows the content of the pseudo | simulation grayscale image generation process routine in the imaging | photography system which concerns on the 2nd Embodiment of this invention. It is an image figure which shows a frequency image. (A) An image diagram showing a frequency image of a small region of interest in a visible light luminance image, and (B) An image diagram showing a frequency image of a small region of interest in a near-infrared luminance image. It is a flowchart which shows the content of the pseudo gray image generation process routine in the imaging | photography system which concerns on the 3rd Embodiment of this invention. (A) The graph which shows the brightness | luminance of each pixel of a near-infrared image and a visible light image, and (B) The graph which shows the brightness | luminance of each pixel of a pseudo | simulation gray image. It is a flowchart which shows the content of the pseudo gray image generation process routine in the imaging | photography system which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the structure of the imaging | photography system which concerns on the 5th Embodiment of this invention. It is a flowchart which shows the content of the pseudo | simulation grayscale image generation process routine in the imaging | photography system which concerns on the 5th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to an imaging system that is mounted on a vehicle and generates and displays a grayscale image from a captured front image will be described as an example.

  As shown in FIG. 1, the imaging system 10 according to the first embodiment includes a first imaging device 12 including a CCD camera that images the front of the host vehicle and outputs a visible light image, and incident light. Of these, a second imaging device 14 that includes a filter that transmits light in the near-infrared wavelength band and a CCD camera, and that captures the front of the vehicle and outputs a near-infrared image; A pseudo gray image by performing image processing on the infrared light source 16 that irradiates the included light forward, the visible light image captured by the first imaging device 12 and the near-infrared image captured by the second imaging device 14. And a display device 20 for displaying a pseudo gray image output from the computer 18.

  The 1st imaging device 12 and the 2nd imaging device 14 are attached so that the front of the own vehicle may be imaged. In addition, the first imaging device 12 and the second imaging device 14 include a plurality of light receiving elements that output signals corresponding to light incident on each of the plurality of pixels arranged in a two-dimensional manner. The output signal is converted into a digital signal, and image data including pixel values of a plurality of pixels is generated. Note that the front side of the host vehicle corresponds to a predetermined area captured by the imaging means of the present invention.

  A filter (not shown) that transmits light in the near-infrared light wavelength band is provided so as to face the lens surface of the second imaging device 14. The front of the vehicle is photographed, and a near-infrared image that is an image in the wavelength band of near-infrared light is output.

  Since the first imaging device 12 is not provided with a filter, the first imaging device 12 outputs a visible light image that is an image in a wavelength band including visible light.

  The infrared light source 16 irradiates light including near-infrared light in front of the host vehicle when photographing with the second photographing device 14.

  The computer 18 includes a CPU, a ROM that stores a program for a pseudo gray image generation processing routine, which will be described later, a RAM that stores data, and a bus that connects these. If the computer 18 is described in terms of functional blocks divided for each function realization means determined based on hardware and software, as shown in FIG. An image acquisition unit 22 that acquires an image and a near-infrared image output from the second imaging device 14, and a visible light image and a near-infrared image that are the output of the image acquisition unit 22 are combined to generate a pseudo gray image. And a pseudo grayscale image generation unit 24 for outputting to the display device 20.

  The image acquisition unit 22 includes, for example, an A / D converter and an image memory that stores image data.

  The pseudo grayscale image generation unit 24 generates a pseudo grayscale image based on the visible light image and the near-infrared image and outputs the pseudo grayscale image to the display device 20 as described below.

  First, for each pixel of the visible light image and the near-infrared image, the luminance Ir of the target pixel in the near-infrared image is compared with the luminance Ic of the target pixel at the same position in the visible light image. select. When Ir ≧ Ic, the luminance Ir is selected as the luminance information of the target pixel. When Ir <Ic, the luminance information of the target pixel is selected without selecting the luminance Ir of the near-infrared image. Select the luminance Ic. Then, the grayscale pseudo image is generated using the luminance information of each selected pixel.

  For example, as shown in FIG. 2B, the luminance of each pixel of the visible light image and the luminance of each pixel of the near-infrared image as shown in FIG. The higher luminance information is selected to generate a grayscale pseudo image.

  Next, the operation of the imaging system 10 according to the present embodiment will be described. In the computer 18 when near infrared light is irradiated forward by the infrared light source 16 and the front of the host vehicle is continuously photographed by each of the first photographing device 12 and the second photographing device 14. The pseudo grayscale image generation processing routine shown in FIG. 3 is executed.

  First, in step 100, the data of the visible light image captured by the first imaging device 12 is acquired, and the data of the near-infrared image captured by the second imaging device 14 is acquired. In step 102, the target pixel is set at the same position for each of the visible light image and the near-infrared image acquired in step 100.

  In the next step 104, the luminance Ic of the target pixel in the visible light image is compared with the luminance Ir of the target pixel in the near-infrared image, and the higher luminance is selected as the luminance information of the target pixel.

  In step 108, it is determined whether or not the processing in steps 102 and 104 has been completed for all pixels. If all the pixels have not been completed, the processing returns to step 102 and is still a processing target. Other pixels that are not set are set as the target pixel. On the other hand, if it is determined in step 108 that the luminance information has been selected for all pixels, a pseudo gray image is generated in step 110 using the luminance information for each pixel selected in step 104. Then, the data is output to the display device 20, and the process returns to step 100.

  When the pseudo gray-scale image generation processing routine is executed as described above, based on the visible light image as shown in FIG. 4A and the near-infrared image as shown in FIG. A pseudo gray image as shown in FIG. 3C is generated, and the pseudo gray image continuously output from the computer 18 is continuously displayed by the display device 20.

  As described above, according to the photographing system according to the first embodiment of the present invention, the luminance information of the brighter image is obtained for each pixel from the visible light image and the near-infrared image generated by photographing. By using and generating a pseudo gray image, it is possible to obtain a stable and highly visible image.

  Further, when there is illumination or the like in the vicinity, a pseudo gray image can be generated without reducing the brightness due to the visible light component. Moreover, the pseudo gray image which also displayed the LED traffic light etc. which do not contain the brightness of a near-infrared component can be produced | generated.

  Next, a second embodiment will be described. Note that the configuration of the imaging system according to the second embodiment is the same as that of the first embodiment, and a description thereof will be omitted.

  The second embodiment is different from the first embodiment in that when the higher luminance is equal to or higher than the threshold, the weighted average of luminance is used for the target pixel.

  In the imaging system according to the second embodiment, the pseudo gray image generation unit 24 generates a pseudo gray image based on the visible light image and the near-infrared image as described below, and displays the display device 20. Output to.

  First, for each pixel of the visible light image and the near-infrared image, the luminance Ir of the pixel of interest in the near-infrared image is compared with the luminance Ic of the pixel of interest in the visible light image. Compare If Ir ≧ Ic and Ir ≧ Th1 (a threshold value set in advance based on the saturation region), the luminance Ir is saturated, so the average of the luminance Ir and the luminance Ic Used as pixel luminance information. If Ir ≧ Ic and Ir <Th1, the luminance Ir is selected as the luminance information of the target pixel.

  Further, when Ir <Ic and Ic ≧ Th2 (a threshold value set in advance based on the saturation region), the luminance Ic is saturated, and therefore the average of the luminance Ic and the luminance Ir is calculated. And used as luminance information of the pixel of interest. If Ir <Ic and Ic <Th2, the luminance Ic is selected as the luminance information of the target pixel.

  For example, as shown in FIG. 5, from the luminance of each pixel of the visible light image and the luminance of each pixel of the near-infrared image as shown in FIG. Brightness information is selected, or a grayscale pseudo image is generated using the average brightness.

  Further, smoothing or sharpening processing is performed on the image composed of the luminance information of each pixel obtained by the above processing.

  Next, a pseudo grayscale image generation processing routine according to the second embodiment will be described with reference to FIG. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  First, in step 100, visible light image data is acquired from the first imaging device 12, and near-infrared image data is acquired from the second imaging device 14. In step 102, the target pixel is set at the same position for each of the acquired visible light image and near-infrared image.

  In the next step 200, the luminance Ic of the target pixel in the visible light image is compared with the luminance Ir of the target pixel in the near-infrared image to determine whether the luminance Ir is equal to or higher than the luminance Ic.

  If the luminance Ir is equal to or higher than the luminance Ir, it is determined in step 202 whether or not the luminance Ir is equal to or higher than the threshold Th1, and if the luminance Ir is equal to or higher than the threshold Th1, the luminance Ir is saturated. In step 204, the average of the luminance Ir and the luminance Ic is determined as the luminance information of the target pixel, and the process proceeds to step 212.

  If the luminance Ir is less than the threshold value in step 202, the luminance Ir is selected as luminance information of the target pixel in step 206, and the process proceeds to step 212.

  In step 200, if the luminance Ir is less than the luminance Ic, it is determined in step 208 whether or not the luminance Ic is greater than or equal to the threshold Th2, and if the luminance Ic is greater than or equal to the threshold Th2, the luminance Ic is determined. In step 204, the average of the brightness Ir and the brightness Ic is determined as the brightness information of the target pixel, and the process proceeds to step 212.

  If the brightness Ic is less than the threshold Th2 in step 208, the brightness Ic is selected as the brightness information of the target pixel in step 210, and the process proceeds to step 212.

  In step 212, it is determined whether or not the processing in steps 200 to 210 has been completed for all the pixels. If all the pixels have not been completed, the process returns to step 102 and has not yet been processed. Other pixels are set as the target pixel. On the other hand, if it is determined in step 212 that the luminance information has been obtained for all pixels, in step 214, the luminance information for each pixel obtained in step 204, 206, or 210 is used to simulate the luminance information. A grayscale image is generated and output to the display device 20, and the process returns to step 100.

  In this way, by generating a pseudo gray image using luminance information of a brighter image for each pixel from a visible light image and a near-infrared image generated by photographing, stable visibility is high. An image can be obtained.

  In addition, when the higher luminance is equal to or higher than the threshold, by generating a pseudo gray image using the average luminance for the pixel, a saturated region can be displayed without being saturated.

  In the above embodiment, when the higher luminance is equal to or higher than the threshold, the weights for the visible light image and the near-infrared image are made the same, and the average luminance is used as the luminance information of the target pixel. However, the present invention is not limited to this, and the weighted average of the luminance may be used as the luminance information of the target pixel with different weights for both images.

  Next, a third embodiment will be described. Note that the configuration of the imaging system according to the third embodiment is the same as that of the first embodiment, and a description thereof will be omitted.

  In the third embodiment, frequency conversion is performed on a small region of a luminance image, low frequency components are compared, and a pseudo gray image is generated using a spatial frequency component having a higher low frequency component. This is mainly different from the first embodiment.

  In the imaging system according to the third embodiment, the image acquisition unit 22 acquires a visible light image output from the first imaging device 12 and generates a visible light luminance image including luminance information of the visible light image. . Moreover, the image acquisition part 22 acquires the near-infrared image output from the 2nd imaging device 14, and similarly produces | generates the near-infrared luminance image which consists of the luminance information of a near-infrared image.

  As described below, the pseudo grayscale image generation unit 24 generates a pseudo grayscale image based on the visible light luminance image and the near infrared luminance image output from the image acquisition unit 22 and outputs the pseudo grayscale image to the display device 20. To do.

  First, each of the visible light luminance image and the near-infrared luminance image is divided into a plurality of small regions, and frequency conversion is performed on each of the visible light luminance image and the near-infrared luminance image for each small region to obtain a small size. A plurality of spatial frequency components are obtained for each region. As a frequency conversion method, Fourier transform, DCT (discrete cosine transform), or the like may be used. The small area is an example of the predetermined area of the present invention.

  For example, frequency conversion using DCT is performed according to the following equation (1).

  In the present embodiment, a frequency image of 8 pixels × 8 pixels as shown in FIG. 7 is generated by frequency conversion. In this frequency image, the value of each pixel represents each spatial frequency component, and the upper left pixel is the lowest frequency spatial frequency component, which is a direct current component (DC component). The other pixels are alternating current components (AC components), and the lower right pixel is the highest frequency spatial frequency component.

  Then, based on the frequency image of the attention small area of the visible light luminance image as shown in FIG. 8A and the frequency image of the attention small area of the near-infrared luminance image as shown in FIG. 8B. Then, the frequency components are compared by comparing the DC components and selecting the frequency image with the higher DC component as the frequency image of the small region of interest. This frequency synthesis is performed for each of a plurality of small regions.

  The inverse frequency transform is performed on each small area frequency image obtained by the above-described frequency synthesis to convert each small area image into an image of each small area.

  For example, inverse frequency conversion using IDCT (Inverse Discrete Cosine Transform) is performed according to the following equation (2).

  When the luminance value of the small area image obtained by the inverse frequency conversion exceeds the saturation luminance value, the luminance value of the image is normalized.

  A pseudo gray image is generated using the image of each small area obtained as described above.

  Next, a pseudo grayscale image generation processing routine according to the third embodiment will be described with reference to FIG.

  First, in step 300, visible light image data is acquired from the first imaging device 12, and a visible light luminance image including luminance information is generated. Further, near-infrared image data is acquired from the second imaging device 14, and near-infrared luminance information including luminance information is generated. In step 302, for each of the visible light luminance image and near-infrared luminance image generated in step 300, a small region of interest is set at the same position from a plurality of small regions obtained by dividing the image.

  In the next step 304, frequency conversion is performed on a small region of interest in the visible light luminance image to convert it into a frequency image F representing a plurality of spatial frequency components, Frequency conversion is performed on the region to convert it into a frequency image G representing a plurality of spatial frequency components.

  In step 306, the direct current component of the frequency image G of the small region of interest in the near-infrared luminance image is compared with the direct current component of the frequency image F of the small region of interest in the visible light luminance image. Is determined to be equal to or greater than the DC component of the frequency image F.

  If the direct current component of the frequency image G is equal to or greater than the direct current component of the frequency image F, the frequency image G is selected as a frequency image of the small region of interest in step 308 and inverse frequency conversion is performed on the frequency image G. The image is converted into an image, and the process proceeds to step 312. On the other hand, if the direct current component of the frequency image G is less than the direct current component of the frequency image F in step 306, the frequency image F is selected as the frequency image of the small region of interest in step 310, and the frequency image F Then, inverse frequency conversion is performed to convert it into an image, and the process proceeds to step 312.

  In step 312, if the luminance value is saturated with respect to the image of the small region of interest obtained in step 308 or 310, the luminance value of the image is normalized.

  In step 314, it is determined whether or not the processing in steps 302 to 312 has been completed for all the small regions. If all the small regions have not been completed, the process returns to step 302 and is still a processing target. Other small areas not set are set as the small areas of interest. On the other hand, if it is determined in step 314 that the above processing has been completed for all of the small regions, in step 316, pseudo gray images are obtained using the images of the small regions obtained in step 312. Generate and output to the display device 20, and return to step 300.

  As described above, according to the imaging system according to the third embodiment, a small frequency image having a larger low frequency component is selected for each small region from the visible light image and the near infrared image. By synthesizing the spatial frequency components of the region and using the obtained image to generate a pseudo grayscale image, it is possible to stably obtain an image with good visibility.

  Next, a fourth embodiment will be described. In addition, since the structure of the imaging | photography system which concerns on 4th Embodiment is the same as that of 1st Embodiment, description is abbreviate | omitted.

  In the fourth embodiment, frequency conversion is performed on a small region of a luminance image, and a pseudo gray image is generated using the higher one for each frequency component. Is different.

  In the imaging system according to the fourth embodiment, the image acquisition unit 22 acquires a visible light image output from the first imaging device 12 and generates a visible light luminance image. Moreover, the image acquisition part 22 acquires the near-infrared image output from the 2nd imaging device 14, and produces | generates a near-infrared luminance image similarly.

  As described below, the pseudo grayscale image generation unit 24 generates a pseudo grayscale image based on the visible light luminance image and the near infrared luminance image output from the image acquisition unit 22 and outputs the pseudo grayscale image to the display device 20. To do.

  First, each of the visible light luminance image and the near infrared luminance image is divided into a plurality of small regions, and for each of the visible light luminance image and the near infrared luminance image, frequency conversion is performed for each small region, A plurality of spatial frequency components are obtained for each small region.

  Then, the frequency image of the attention small area of the visible light luminance image as shown in FIG. 8A and the frequency image of the attention small area of the near infrared luminance image as shown in FIG. Based on the above, each spatial frequency component (pixel value of each pixel) is compared, a higher one is selected for each spatial frequency component, and a frequency image of a small region of interest is generated to perform frequency synthesis. This frequency synthesis is performed for each of a plurality of small regions.

  Each frequency image of each small region obtained by frequency synthesis is subjected to inverse frequency conversion to convert it to an image of each small region.

  When the luminance value of the small area image obtained by the inverse frequency conversion exceeds the saturation luminance value, the luminance value of the image is normalized.

  A pseudo gray image is generated using the image of each small area obtained as described above.

  For example, from the visible light image and the near-infrared image as shown in FIG. 10 (A), as shown in FIG. 10 (B), the higher one of the plurality of spatial frequency components is selected for each small region. Thus, a grayscale pseudo image is generated.

  Next, a pseudo grayscale image generation processing routine according to the fourth embodiment will be described with reference to FIG. In addition, about the process similar to 3rd Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, in step 300, a visible light image data is acquired from the first imaging device 12, a visible light luminance image is generated, and a near-infrared image data is acquired from the second imaging device 14, and a near-infrared image data is acquired. Infrared brightness information is generated. In step 302, a small region of interest is set at the same position for each of the generated visible light luminance image and near-infrared luminance image.

  In the next step 304, frequency conversion is performed on the target small region of the visible light luminance image to convert it to the frequency image F, and frequency conversion is performed on the target small region of the near-infrared luminance image. To convert to a frequency image G.

In step 400, each spatial frequency component of the frequency image G of the target small region in the near-infrared luminance image is compared with each spatial frequency component of the frequency image F of the target small region in the visible light luminance image. For each spatial frequency component (each pixel value of the frequency image), the higher one is selected to generate a frequency image of a small region of interest.
In the next step 402, the frequency image of the small region of interest generated in step 400 is converted into an image by performing inverse frequency transformation. In step 312, the small region of interest obtained in step 402 is obtained. If the brightness value is saturated for the image of, the brightness value of the image is normalized.

  In step 314, it is determined whether or not the processing in steps 302, 304, 400, 402, and 312 has been completed for all the small areas. If all the small areas have not been completed, the process returns to step 302. Other small areas that are not yet processed are set as the attention small areas. On the other hand, if it is determined in step 314 that the above processing has been completed for all of the small regions, in step 316, pseudo gray images are obtained using the images of the small regions obtained in step 312. Generate and output to the display device 20, and return to step 300.

  As described above, according to the imaging system according to the fourth embodiment, the larger one of the plurality of spatial frequency components of the frequency image is selected for each small region from the visible light image and the near-infrared image. By synthesizing the spatial frequency components of the small area, and generating a pseudo gray image using the obtained image, it is possible to obtain a stable and good-visibility image and represent a fine pattern change. Images can be obtained.

  In the above embodiment, the larger one is selected for each pixel (each spatial frequency component) from the frequency image of the small region of the visible light luminance image and the frequency image of the small region of the near infrared luminance image. The case where frequency synthesis is performed has been described as an example. However, the present invention is not limited to this, and the frequency synthesis may be performed by performing weighted averaging of values for each pixel (each spatial frequency component). As a result, a saturated region can be appropriately displayed.

  Next, a fifth embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The fifth embodiment is mainly different from the first embodiment in that a single imaging device captures a visible light image and a near-infrared image.

  As shown in FIG. 12, an imaging system 510 according to the fifth embodiment includes an imaging device 512 including a CCD camera that images the front of the host vehicle and outputs a captured image, and light incident on the imaging device 512. A filter 514 in which a near-infrared filter that transmits light in the wavelength band of near-infrared light and a visible light filter that transmits light in the wavelength band of visible light are two-dimensionally arranged, an infrared light source 16, The computer 518 which performs image processing with respect to the picked-up image image | photographed with the imaging device 512, and produces | generates a pseudo gray image, and the display apparatus 20 are provided.

  The imaging device 512 includes a plurality of light receiving elements that output signals corresponding to light incident on each of a plurality of pixels arranged in two dimensions, and converts signals output from the plurality of light receiving elements into digital signals. Thus, image data composed of pixel values of a plurality of pixels is generated.

  A filter 514 in which a near-infrared light filter and a visible light filter are two-dimensionally arranged is provided so as to face the lens surface of the imaging device 512, and a plurality of near-infrared light filters and a plurality of visible light are arranged. Filters are arranged corresponding to a plurality of light receiving elements (a plurality of pixels) of the photographing apparatus 512.

  The imaging device 512 images the front of the host vehicle via the filter 514 and generates a captured image composed of near-infrared component data in the near-infrared light wavelength band and visible light component data in the visible-light wavelength band. Output.

  The computer 518 generates an visible light image and a near-infrared image from an image acquisition unit 522 that acquires a captured image output from the imaging device 512, and a captured image that is an output of the image acquisition unit 522. A pseudo grayscale image generation unit 524 that synthesizes infrared images to generate a pseudo grayscale image and outputs the pseudo grayscale image to the display device 20 is provided.

  As described below, the pseudo grayscale image generation unit 524 generates a visible light image and a near infrared image, generates a pseudo grayscale image based on the visible light image and the near infrared image, and displays the display device 20. Output to.

  First, the near-infrared light component data in the near-infrared light wavelength band is extracted from the pixels corresponding to the near-infrared filter of the captured image, and the visible light wavelength band is visible from the pixel corresponding to the visible light filter. Extract light component data.

  An interpolation process is performed on the near infrared component data to generate a near infrared image. In addition, an interpolation process is performed on the visible light component data to generate a visible light image.

  For each pixel of the visible light image and the near-infrared image, the luminance Ir of the target pixel in the near-infrared image is compared with the luminance Ic of the target pixel in the visible light image, and the luminance information with the higher luminance is selected. Then, the grayscale pseudo image is generated using the luminance information of each selected pixel.

  Next, a pseudo grayscale image generation processing routine according to the fifth embodiment will be described with reference to FIG. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, in step 550, captured image data is acquired from the imaging device 512, and in step 552, near-infrared component data in the near-infrared light wavelength band is extracted from the captured image acquired in step 550, and An interpolation process is performed on the near infrared component data to generate a near infrared image. In step 554, visible light component data in the visible light wavelength band is extracted from the captured image acquired in step 550, and a visible light image is generated by performing interpolation on the visible light component data. .

  In step 102, the target pixel is set at the same position for each of the visible light image and the near-infrared image generated in steps 552 and 554. In the next step 104, the luminance Ic of the target pixel in the visible light image is compared with the luminance Ir of the target pixel in the near-infrared image, and the higher luminance information is selected.

  In step 108, it is determined whether or not the processing in steps 102 and 104 has been completed for all pixels. If all the pixels have not been completed, the processing returns to step 102 and is still a processing target. Other pixels that are not set are set as the target pixel. On the other hand, if it is determined in step 108 that high luminance information has been selected for all pixels, a pseudo gray image is generated in step 110 using the luminance information for each pixel selected in step 104. Then, the data is output to the display device 20, and the process returns to step 100.

  As described above, according to the imaging system according to the fifth embodiment, the brighter image for each pixel from the visible light image and the near-infrared image generated from the captured image captured by one imaging apparatus. By generating a pseudo gray image using the luminance information, it is possible to stably obtain an image with good visibility.

  In the first embodiment, the second embodiment, and the fifth embodiment described above, an example has been described in which a block serving as a luminance determination unit is a pixel unit. However, the present invention is not limited, and an area composed of a plurality of pixels may be a block. In this case, the average luminance of each block may be compared between the visible light image and the near-infrared image.

  Moreover, although the case where a pseudo gray image is generated using a visible light image and a near infrared image has been described as an example, the present invention is not limited to this, and a visible light image and a far infrared image are used. Alternatively, a pseudo gray image may be generated. In this case, a far-infrared image is generated using a far-infrared camera, or the front of the host vehicle is photographed using a far-infrared filter that transmits the wavelength band of far-infrared light. do it. Moreover, what is necessary is just to image | photograph the front of the own vehicle, without using an infrared light source.

  Moreover, although the case where the image acquisition unit and the pseudo grayscale image generation unit are realized by a computer has been described as an example, the present invention is not limited to this, and the image acquisition unit and the pseudo grayscale image generation unit realize the function of each unit. A plurality of computers or one or a plurality of electronic circuits may be used.

  It is also possible to provide the program of the present invention by storing it in a storage medium.

10, 510 Imaging system 12 First imaging device 14 Second imaging device 16 Infrared light source 18, 518 Computer 22, 522 Image acquisition unit 24, 524 Pseudo gray image generation unit 512 Imaging device 514 Filter

Claims (3)

Photographing that captures a predetermined region and outputs a visible light image in a first wavelength band including visible light, and an infrared image in a second wavelength band that has a band different from the first wavelength band and includes infrared light. Image acquisition means for acquiring the visible light image and the infrared image from a means;
For each block obtained by dividing the visible light image and the infrared image from the visible light image and the infrared image acquired by the image acquisition means, the block of the visible light image and the infrared image Of the visible light image and the infrared image, if the luminance information of the block of the brighter image of the visible light image and the infrared image is greater than or equal to a threshold value, the visible light image and the infrared image Generating means for generating a pseudo gray image using a weighted average of luminance information of the block;
A pseudo gray-scale image generating apparatus including:   The pseudo gray image generating apparatus according to claim 1, wherein the block is a pixel unit. Computer
Photographing that captures a predetermined region and outputs a visible light image in a first wavelength band including visible light, and an infrared image in a second wavelength band that has a band different from the first wavelength band and includes infrared light. An image acquisition unit that acquires the visible light image and the infrared image from a unit, and a plurality of the visible light image and the infrared image from the visible light image and the infrared image acquired by the image acquisition unit. For each divided block, the block of the visible light image is compared with the block of the infrared image, and the block of the brighter image of the visible light image and the infrared image is compared with the block of the visible light image and the infrared image. A program for functioning as a generation unit that generates a pseudo gray image by using a weighted average of luminance information of the block in the visible light image and the infrared image when the luminance information is greater than or equal to a threshold value.