JP2006072401A - Image compounding device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable to obtain an image by compounding the luminance information of a gray scale image while leaving the color information of an RGB color image as it is. <P>SOLUTION: A gray scale image acquiring means 11 acquires a gray scale image, and an RGB image acquiring means 12 acquires an RGB color image, and a luminance information color information separating means 13 separates the acquired RGB color image into luminance information and color information. An image compounding means 14 calculates composite luminance information by operating an inter-image arithmetic operation to the separated luminance information and the acquired gray scale image, and a luminance information color information compounding means 15 prepares a composite RGB image by compounding the calculated composite luminance information with the color information separated from the RGB color image, and a display means 16 displays the prepared composite RGB image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image composite technology, and in particular, an image mounted on an aircraft, mounted on a vehicle, fixedly installed, or the like having both RGB color image acquisition means such as a visible image sensor and gray scale image acquisition means such as an infrared image sensor. The present invention relates to an image composition apparatus and an image composition method for performing image composition in order to display images obtained from two image acquisition means on one display device.

  In recent years, imaging devices equipped with two or more image acquisition means have been developed based on the improvement in performance of image acquisition devices such as infrared sensors and advances in image processing technology and control technology using microprocessors. In order to improve the performance, there is a demand for an image combination apparatus that can combine and display as one image while retaining the information of each image.

  Conventionally, as a display means in an apparatus having a plurality of image acquisition means including an RGB color image, a part of information of another grayscale image is displayed as a symbol on the RGB color image, and a part of the RGB color image is displayed. An image displayed by switching to a gray scale image (superimpose) is known.

  As a specific prior art, the following Patent Document 1 is intended to perform a diagnosis based on a thermal image quickly and accurately, and based on images of a pair of visible cameras and a pair of infrared cameras, luminance from the infrared image is determined. A technique for extracting data, applying a predetermined color according to the luminance, and synthesizing it with a stereoscopic visible image obtained from a visible camera is described.

  Patent Document 2 describes a technique for complementing a region that is dazzled by the headlight of an oncoming vehicle and disappears with an infrared image by superimposing a corresponding portion of the infrared image on a part of the visible image and synthesizing it. ing.

Patent Document 3 discloses a shear process (water vapor) in the steel industry by showing both images obtained by a TV camera that captures the infrared light region and the visible light region on one screen in a superimposition. This is a technique for observing a state invisible.
JP 2001-31718 A Japanese Patent Laid-Open No. 11-308609 JP-A-53-118309

  Among the above-described conventional techniques, in the technique of displaying a part of information of other grayscale images on the RGB color image, the information of the grayscale image other than the target on which the feature extraction processing has been performed, There is a problem that the information of the RGB color image is not reflected.

  In addition, the superimpose technique does not reflect the RGB color image information in the superimpose region and the grayscale image information outside the superimpose region, which may cause the observer to miss the object. There is.

  In addition, the technique described in Patent Document 1 gives a predetermined color according to luminance data in order to show, for example, an inflamed part on a three-dimensional image of a human body, and is actually seen by humans. The color will be completely different from the color. The technique described in Patent Document 2 is only pasting an infrared image on a part of a visible image, and the technique described in Patent Document 3 is a method of superimposing two screens in one screen. None of these techniques is a technique for displaying a single image while maintaining both RGB color image and grayscale image information.

  In other words, since the conventional technology does not display information as both an RGB color image and a grayscale image as a single image, for example, when searching for a life raft in case of a marine disaster in a dim situation, for example. , There is a problem that convenience is low.

  The present invention is intended to solve such a problem of the prior art, and is an image composite technique capable of obtaining an image in which luminance information of a gray scale image is combined while the color information of the RGB color image is maintained as it is. The purpose is to provide.

  In order to solve the above-described problems, the present invention extracts luminance information from an RGB color image and combines the luminance information so as not to leak information having a gray scale image, thereby generating composite luminance information. By using the composite luminance information and the color information extracted from the RGB color image, a composite RGB image is created, thereby creating one composite image substantially holding the information of each original image.

  That is, the present invention is an image composite device that combines an RGB color image and a grayscale image, the RGB image acquisition means for acquiring an RGB color image, the grayscale image acquisition means for acquiring a grayscale image, Luminance information color information separating means for separating the acquired RGB color image into luminance information and color information, image combining means for performing a combined process of the acquired grayscale image and the separated luminance information, and An image composite apparatus comprising: luminance information color information synthesis means for creating a composite RGB image using the luminance information obtained as a result of the composite processing and the separated color information.

  Further, according to the present invention, in the image composition device, the image composition means includes first frequency decomposition means for decomposing the acquired gray scale image into a high frequency component and a low frequency component, and the separated luminance. A second frequency decomposing means for decomposing information into a high frequency component and a low frequency component; a high frequency component obtained by decomposing by the first frequency decomposing means; and a high frequency obtained by decomposing by the second frequency decomposing means. A high frequency calculation means for calculating and combining the components, a low frequency component obtained by the decomposition by the first frequency decomposition means, and a low frequency component obtained by the decomposition by the second frequency decomposition means And a low frequency computing means for synthesizing, and a frequency synthesizing means for synthesizing the output of the high frequency computing means and the output of the low frequency computing means.

  Further, the present invention provides the image composite apparatus, wherein the filter size used when the gray scale image is further decomposed into a high frequency component and a low frequency component by the first frequency decomposing means, or the second frequency. It is characterized by comprising filter size setting means for setting one or both of the filter sizes used when the luminance information is decomposed into a high frequency component and a low frequency component by the decomposition means.

  Further, the present invention provides the image composite apparatus, wherein the high frequency component obtained by the first frequency resolving means and the high frequency obtained by the second frequency resolving means by the high frequency calculating means. The weight parameter used when calculating the component, or the low frequency component obtained by the decomposition by the first frequency resolving means by the low frequency calculating means and the low frequency obtained by the decomposition by the second frequency resolving means. Weighting parameter setting means for setting one or both of the weighting parameters used when calculating the frequency component is provided.

  Furthermore, a synthesis parameter setting unit may be provided for setting a synthesis parameter used when the output of the high frequency calculation means and the output of the low frequency calculation means are synthesized by the frequency synthesis means.

  The present invention is also an image composition method for combining an RGB color image and a grayscale image, the step of acquiring an RGB color image, the step of acquiring a grayscale image, and the acquired RGB color image. Separating the luminance information and the color information, performing a combined process of the acquired grayscale image and the separated luminance information, and the luminance information obtained as a result of the combined process and the separated And a step of creating a composite RGB image using the color information.

  According to the present invention, the luminance information possessed by the RGB color image and the gray scale image is synthesized without leaking, and the color information possessed by the RGB color image is further synthesized, so that the maximum information is provided on one composite image. And the visibility of the object can be improved.

  According to the present invention, it is possible to obtain an image in which luminance information of a gray scale image is combined while maintaining the color information of the RGB color image. For example, when searching for a life preserver in case of a marine disaster in a dimly lit situation, the color information remains as it is, and the object (an object that is hotter than the surroundings) is made brighter than the background and displayed with high contrast. This makes it possible to see clearly on the screen.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows the basic configuration of the present invention. The image composite device 1 of the present invention includes a grayscale image acquisition unit 11 that acquires a grayscale image, an RGB image acquisition unit 12 that acquires an RGB color image, and separates the acquired RGB color image into luminance information and color information. Luminance information color information separation means 13, image composite means 14 for calculating composite brightness information based on the separated brightness information and the grayscale image acquired by the grayscale image acquisition means 11, the calculated composite brightness information and brightness Luminance information color information synthesizing means 15 for synthesizing the color information separated by the information color information separating means 13 to create a composite RGB image, and display means 16 for displaying the created composite RGB image.

  The gray scale image acquisition unit 11 is, for example, an infrared video sensor or an infrared camera, and the RGB image acquisition unit 12 is, for example, a visible video sensor or a visible color camera.

  FIG. 2 is a diagram illustrating an example of an image composite processing flow. First, the gray scale image acquisition unit 11 acquires a gray scale image (step S1), and the RGB image acquisition unit 12 acquires an RGB color image (step S2).

  The RGB color image acquired by the RGB image acquisition unit 12 is separated into luminance information and color information by the luminance information color information separation unit 13 (step S3). For example, the RGB color system is converted into the HSI color system and separated into luminance information (I) and color information (H, S), or the RGB color system is converted into the YCrCb color system to obtain the luminance. Information (Y) and color information (Cr, Cb) are separated.

  The luminance information obtained in this way and the grayscale image acquired by the grayscale image acquisition means 11 are subjected to an inter-image calculation (composition without leaking the respective information) in the image combination means 14 to obtain the composite luminance information. Calculate (step S4).

  Then, the luminance information color information synthesizing unit 15 synthesizes the calculated composite luminance information and the color information separated from the RGB color image in step S3 to create an RGB color system composite RGB image (step S5). ), The display means 16 displays the created composite RGB image (step S6).

  FIG. 3 shows a configuration example of the first embodiment of the present invention. In the image composite apparatus 2 shown in FIG. 3, 100 is a visible color camera that captures a visible color image, 101 is an amplifying unit that amplifies the visible color image signal, and 102 converts the amplified visible color image signal into a digital image signal. An A / D converter 103 is an RGB-HSI converter for separating the output of the A / D converter 102 into an HSI color system luminance signal and a color signal, and 104 is a high-frequency component of the spatial frequency of the separated luminance signal And a frequency resolving unit that separates into a low frequency component.

  Reference numeral 105 denotes an infrared camera that picks up an infrared image, 106 denotes an amplification unit that amplifies the infrared image signal, 107 denotes an A / D converter that converts the amplified infrared image signal into a digital image signal, and 108 denotes the infrared image signal. This is a frequency resolving unit that separates the spatial frequency into a high frequency component and a low frequency component.

  Reference numeral 109 denotes a high-frequency operation unit that synthesizes the high-frequency component of the luminance signal output from the frequency decomposition unit 104 and the high-frequency component of the infrared image signal output from the frequency decomposition unit 108, and 110 denotes the output from the frequency decomposition unit 104. A low-frequency operation unit 111 synthesizes the low-frequency component of the luminance signal and the low-frequency component of the infrared image signal output from the frequency decomposition unit 108, and 111 represents the output (high-frequency component) of the high-frequency operation unit 109 and the low-frequency operation unit 110. A frequency synthesizer for synthesizing the output (low frequency component) to generate a composite luminance signal, and 112 synthesizes the generated composite luminance signal and the color signal separated by the RGB-HSI converter 103 to generate a composite RGB signal. An HSI-RGB conversion unit 113 for creating an image is a monitor for displaying the created composite RGB image.

  In the first embodiment, the visible color image signal captured by the visible color camera 100 is amplified by the amplifying unit 101, converted into a digital image signal by the A / D converter 102, and the infrared image signal captured by the infrared camera 105 is also used. Amplified by the amplifying unit 106 and converted into a digital image signal by the A / D converter 107.

  The visible color image signals of R (red), G (green), and B (blue), which are the outputs of the A / D converter 102, are separated into luminance signals and color signals by the RGB-HSI converter 103, and the luminance The signal is output to the frequency resolution unit 104, and the color signal is output to the HSI-RGB conversion unit 112. In the frequency resolution unit 104, the luminance signal is separated into a high frequency component and a low frequency component of the spatial frequency, and the high frequency component is output to the high frequency calculation unit 109 and the low frequency component is output to the low frequency calculation unit 110.

  The infrared digital image signal that is the output of the A / D converter 107 is separated into a high frequency component and a low frequency component of the spatial frequency by the frequency resolution unit 108, and the high frequency component is sent to the high frequency calculation unit 109. Is output to the low frequency calculation unit 110.

  In the signal separation in the frequency resolving units 104 and 108, for example, an image filtered using an n × n average value filter is created, the image is set as a low-frequency component, and an image obtained by subtracting the low-frequency component from the original image has a high frequency. Processing with components.

  The high frequency components separated by the frequency decomposing units 104 and 108 are combined and processed by the high frequency calculation unit 109 and the low frequency components are combined by the low frequency calculation unit 110 at a predetermined ratio. Further, the output of the high frequency calculation unit 109 (high frequency component) and the output of the low frequency calculation unit 110 (low frequency component) are frequency synthesized by the frequency synthesis unit 111, and a composite luminance signal is generated.

  The composite luminance signal obtained in the frequency synthesizer 111 and the color signal separated in the RGB-HSI converter 103 are combined in the HSI-RGB converter 112 to create a composite RGB image and output to the monitor 113.

  A configuration example of the second embodiment of the present invention is shown in FIG. 4 includes filter size setting units 114 and 115, weight parameter setting units 116 and 117, and a synthesis parameter setting unit 118 in addition to the configuration of the image composite device 2 in FIG.

  In the second embodiment, the filter size setting unit 114 sets the filter size for separating the luminance signal separated by the RGB-HSI conversion unit 103 into a low frequency component and a high frequency component based on external input information. change. Based on the input information, the filter size setting unit 114 determines the filter size of the average value filter as 3 × 3, 5 × 5, or 7 × 7, and sends the filter size setting signal to the frequency resolution unit 104. The frequency resolution unit 104 generates, for example, a 3 × 3 average value filter image, a 5 × 5 average value filter image, or a 7 × 7 average value filter image according to the filter size setting signal, and the generated image is spatially generated. The low frequency component of the frequency.

  Similarly, the filter size setting unit 115 sets and changes the filter size when separating the infrared digital image signal into a low frequency component and a high frequency component. The filter size setting unit 114 and the filter size setting unit 115 may be shared and configured as one filter size setting unit.

  In the second embodiment, the weight parameter setting unit 116 outputs the high-frequency component of the luminance signal output from the frequency decomposition unit 104 and the frequency decomposition unit 108 in the high-frequency calculation unit 109 based on external input information. A weighting parameter that determines a ratio for combining the high-frequency component of the infrared image signal is set.

  Similarly, the weight parameter setting unit 117 synthesizes the low frequency component of the luminance signal output from the frequency resolution unit 104 and the low frequency component of the infrared image signal output from the frequency resolution unit 108 in the low frequency calculation unit 110. Set the weight parameter that determines the ratio.

  Further, in the second embodiment, the synthesis parameter setting unit 118 performs output (high frequency component) of the high frequency calculation unit 109 and output (low frequency component) of the low frequency calculation unit 110 in the frequency synthesis unit 111 based on input information from the outside. ) And a synthesis parameter for determining a high frequency gain and a low frequency gain when the frequency synthesis is performed.

  Since the high-frequency component of each image indicates detailed details (edges and minute points) of the object included in the image, the synthesis parameter setting unit 118 increases the high-frequency gain used during frequency synthesis. The visibility of the object is improved. In addition, the low frequency component of each image indicates the luminance of the background, and since this contains less information, the average luminance of the image is adjusted by lowering the low frequency gain to prevent the display image from being saturated. .

  Further, the frequency separated into the low frequency component and the high frequency component can be adjusted by the filter size setting units 114 and 115. Further, whether to display an image in which the luminance signal obtained from the visible color image signal strongly appears or to display an image in which the infrared image signal strongly appears is adjusted by the high-frequency and low-frequency weight parameter setting units 116 and 117, respectively. be able to.

  As can be understood from the above, the features of the embodiment of the present invention are described as follows.

(Appendix 1) An image compounding device that combines a color image and a grayscale image,
Color image acquisition means for acquiring a color image;
A grayscale image acquisition means for acquiring a grayscale image;
Luminance information color information separating means for separating the acquired color image into luminance information and color information;
Image combining means for performing a combined process of the acquired grayscale image and luminance information separated from the color image;
An image composite apparatus comprising: luminance information color information synthesis means for creating a composite color image using luminance information obtained as a result of the composite processing and color information separated from the color image.

(Appendix 2) In the image composite device described in Appendix 1,
The image combining means includes
First frequency resolving means for decomposing the acquired gray scale image into a high frequency component and a low frequency component;
Second frequency resolving means for decomposing luminance information separated from the color image into a high frequency component and a low frequency component;
High-frequency calculation means for calculating and synthesizing a high-frequency component obtained by decomposition by the first frequency decomposition means and a high-frequency component obtained by decomposition by the second frequency decomposition means;
Low frequency computing means for computing and synthesizing a low frequency component obtained by the decomposition by the first frequency decomposing means and a low frequency component obtained by the decomposition by the second frequency decomposing means;
An image composite apparatus comprising: frequency synthesis means for synthesizing the output of the high frequency computation means and the output of the low frequency computation means.

(Appendix 3) In the image composite device described in Appendix 2,
A filter size used when the gray scale image is decomposed into a high frequency component and a low frequency component by the first frequency decomposition means, or luminance information is decomposed into a high frequency component and a low frequency component by the second frequency decomposition means. An image composite apparatus comprising: filter size setting means for setting one or both of the filter sizes used when performing the processing.

(Appendix 4) In the image composite device described in Appendix 2 or Appendix 3,
The weight parameter used when the high-frequency component obtained by the decomposition by the first frequency decomposing unit and the high-frequency component obtained by the decomposing by the second frequency decomposing unit are calculated by the high-frequency calculating unit, or the low-frequency component Any one of the weighting parameters used when the low frequency component obtained by the decomposition by the first frequency resolving means and the low frequency component obtained by the decomposition by the second frequency resolving means are calculated by the frequency calculating means. An image composite apparatus comprising weight parameter setting means for setting both of them.

(Supplementary Note 5) In the image composite device described in Supplementary Note 2, Supplementary Note 3 or Supplementary Note 4,
An image composite apparatus comprising: a synthesis parameter setting unit configured to set a synthesis parameter used when the output of the high frequency calculation means and the output of the low frequency calculation means are synthesized by the frequency synthesis means.

(Appendix 6) An image combination method for combining a color image and a grayscale image,
Obtaining a color image;
Acquiring a grayscale image;
Separating the acquired color image into luminance information and color information;
Performing a combined process of the acquired grayscale image and luminance information separated from the color image;
And a step of creating a composite color image using luminance information obtained as a result of the composite processing and color information separated from the color image.

(Appendix 7) In the image composition method described in Appendix 6,
Performing a combined process of the acquired grayscale image and the luminance information separated from the color image,
A first frequency decomposition step of decomposing the acquired grayscale image into a high frequency component and a low frequency component;
A second frequency decomposition step for decomposing the luminance information separated from the color image into a high frequency component and a low frequency component;
A high-frequency operation step for calculating and synthesizing a high-frequency component obtained by the decomposition by the first frequency decomposition step and a high-frequency component obtained by the decomposition by the second frequency decomposition step;
A low frequency calculation step for calculating and synthesizing a low frequency component obtained by the decomposition by the first frequency decomposition step and a low frequency component obtained by the decomposition by the second frequency decomposition step;
An image composition method comprising: a frequency synthesis step of synthesizing the output of the high frequency computation step and the output of the low frequency computation step.

(Supplementary Note 8) In the image composition method described in Supplementary Note 7,
The filter size used when the grayscale image is decomposed into a high frequency component and a low frequency component by the first frequency decomposition step, or the luminance information is decomposed into a high frequency component and a low frequency component by the second frequency decomposition step. Setting a filter size to be used when
A weighting parameter used in calculating the high frequency component obtained by the decomposition in the first frequency decomposition step and the high frequency component obtained by the decomposition in the second frequency decomposition step by the high frequency calculation step, or the low A step of setting a weight parameter used when calculating a low frequency component obtained by the decomposition by the first frequency decomposition step and a low frequency component obtained by the decomposition by the second frequency decomposition step by the frequency calculating step. When,
An image compositing method comprising: setting a synthesis parameter used when synthesizing the output of the high frequency calculation step and the output of the low frequency calculation step by the frequency synthesis step.

It is a figure which shows the fundamental structure of this invention. It is a figure which shows an example of an image composite processing flow. It is a figure which shows the structural example of Example 1 of this invention. It is a figure which shows the structural example of Example 2 of this invention.

Explanation of symbols

1, 2 and 3 Image Composite Device 11 Grayscale Image Acquisition Unit 12 RGB Image Acquisition Unit 13 Luminance Information Color Information Separation Unit 14 Image Composite Unit 15 Luminance Information Color Information Synthesis Unit 16 Display Unit 100 Visible Color Camera 101, 106 Amplification Unit 102 , 107 A / D converter 103 RGB-HSI converter 104, 108 Frequency resolving unit 105 Infrared camera 109 High frequency arithmetic unit 110 Low frequency arithmetic unit 111 Frequency synthesizing unit 112 HSI-RGB conversion unit 113 Monitor 114, 115 Filter size setting unit 116, 117 Weight parameter setting unit 118 Composite parameter setting unit

Claims (5)

An image compounding device that combines a color image and a grayscale image,
Color image acquisition means for acquiring a color image;
A grayscale image acquisition means for acquiring a grayscale image;
Luminance information color information separating means for separating the acquired color image into luminance information and color information;
Image combining means for performing a combined process of the acquired grayscale image and luminance information separated from the color image;
An image composite apparatus comprising: luminance information color information synthesis means for creating a composite color image using luminance information obtained as a result of the composite processing and color information separated from the color image. The image composite device according to claim 1,
The image combining means includes
First frequency resolving means for decomposing the acquired gray scale image into a high frequency component and a low frequency component;
Second frequency resolving means for decomposing luminance information separated from the color image into a high frequency component and a low frequency component;
High-frequency calculation means for calculating and synthesizing a high-frequency component obtained by decomposition by the first frequency decomposition means and a high-frequency component obtained by decomposition by the second frequency decomposition means;
Low frequency computing means for computing and synthesizing a low frequency component obtained by the decomposition by the first frequency decomposing means and a low frequency component obtained by the decomposition by the second frequency decomposing means;
An image composite apparatus comprising: frequency synthesis means for synthesizing the output of the high frequency computation means and the output of the low frequency computation means. The image composite device according to claim 2, further comprising:
A filter size used when the gray scale image is decomposed into a high frequency component and a low frequency component by the first frequency decomposition means, or luminance information is decomposed into a high frequency component and a low frequency component by the second frequency decomposition means. An image composite apparatus comprising: filter size setting means for setting one or both of the filter sizes used when performing the processing. The image composite apparatus according to claim 2 or 3, further comprising:
The weight parameter used when the high-frequency component obtained by the decomposition by the first frequency decomposing unit and the high-frequency component obtained by the decomposing by the second frequency decomposing unit are calculated by the high-frequency calculating unit, or the low-frequency component Any one of the weighting parameters used when the low frequency component obtained by the decomposition by the first frequency resolving means and the low frequency component obtained by the decomposition by the second frequency resolving means are calculated by the frequency calculating means. An image composite apparatus comprising weight parameter setting means for setting both of them. An image composition method that combines a color image and a grayscale image,
Obtaining a color image;
Acquiring a grayscale image;
Separating the acquired color image into luminance information and color information;
Performing a combined process of the acquired grayscale image and luminance information separated from the color image;
And a step of creating a composite color image using luminance information obtained as a result of the composite processing and color information separated from the color image.

Images