JP2007214933A - Color image pickup device and its control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a leaf image part to be bright by using one image pickup element. <P>SOLUTION: When a natural photographing mode is selected (YES in S21) in a control method of color imaging device, a subject image is photographed in a state where an infrared light cut filter is inserted to the front part of a CCD (S24, S28) and the subject image is photographed in a state where the infrared light cut filter is evacuated (S25, S30). The leaf image part in the subject is detected, based on the subject image with the infrared light cut therefrom, which is obtained in inserting the infrared light cut filter, and the subject image including the infrared light, which is obtained in evacuating the infrared light cut filter. Correction is performed to allow the detected leaf image part to be bright (S27). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a color imaging device and a control method thereof.

When a landscape or the like is imaged, plant leaves may be included. In the image obtained by imaging, it is necessary to detect the image portion of the leaf in order to make correction so that the leaf is bright. For example, incident light is split into visible light and infrared light using a spectroscopic mirror, and is received using a visible light image sensor and an infrared light image sensor. The resulting visible light image and infrared light image are obtained. There is one that detects a part such as a leaf by comparing with (Patent Document 1).
Japanese Patent Laid-Open No. 2002-5842

  However, two image sensors for a visible light image and an infrared light image are required.

  An object of this invention is to recognize the image part of a leaf using one image sensor.

  A color image pickup device according to the present invention is a solid-state electronic image pickup device that picks up a subject and outputs color image data representing a color subject image, and an infrared ray cut A filter, and the infrared cut filter is inserted in front of the light receiving surface of the solid-state electronic imaging device to image a subject, and outputs first color image data representing a first color subject image, and the infrared cut filter Are retracted from the front of the light receiving surface of the solid-state electronic image pickup device to image a subject, and the solid-state electronic image pickup device and the infrared cut filter are arranged to output second color image data representing a second color subject image. Imaging control means for controlling, and a color sky between corresponding pixels in the first color subject image and the second color subject image Based on the difference between the upper, characterized in that it comprises a detection means for detecting an image portion of the leaf is included in the first color object image or the second color object image.

  The present invention also provides a control method suitable for the color imaging apparatus. That is, this method uses a solid-state electronic image pickup device to pick up an image of a subject, obtains color image data representing a color object image, and inserts and retracts an infrared cut filter in front of the light receiving surface of the solid-state electronic image pickup device. The infrared cut filter is inserted in front of the light receiving surface of the solid-state electronic image pickup device to pick up an image of the subject, output first color image data representing a first color subject image, and the infrared ray The solid-state electronic image pickup device and the infrared ray cut-out are arranged so that the cut filter is retracted from the front of the light receiving surface of the solid-state electronic image pickup device to pick up an image of a subject and output second color image data representing a second color subject image. Controlling a filter and based on a difference in color space between corresponding pixels in the first color subject image and the second color subject image; And detects an image portion of the leaf is included in the serial first color object image or the second color object image.

  According to the present invention, the infrared cut filter that can be inserted and retracted is supported in front of the light receiving surface of the solid-state electronic imaging device. The infrared cut filter is inserted in front of the light receiving surface of the solid-state electronic image pickup device to image a subject to obtain first color image data representing a first color subject image, and the infrared cut filter is connected to the infrared cut filter. The subject is retreated from the front side of the light receiving surface of the solid-state electronic imaging device to image the subject, and second color image data representing a second color subject image is obtained. A leaf included in the first color subject image or the second color subject image based on a difference in color space between corresponding pixels in the first color subject image and the second color subject image. Are detected. In the image portion of the leaf, the difference in color space between corresponding pixels in the first color subject image and the second color subject image is large. For this reason, the image portion of the leaf can be detected based on this difference. Moreover, the image part of a leaf can be detected using one solid-state electronic imaging device.

  It is preferable to further include correction means for correcting the first color subject image data or the second color subject image data so that the leaf image portion detected by the detection means becomes clear.

  The solid-state electronic imaging device outputs RGB color image data, for example. In this case, color space conversion means for converting the first color image data and the second color image data obtained under the control of the imaging control means into image data in the Lab color space is further provided. In this case, the detection means includes a difference in the Lab color space between corresponding pixels in the first color subject image and the second color subject image that have been subjected to Lab color space conversion by the color space conversion means. Based on this, the image portion of the leaf included in the first color subject image or the second color subject image will be detected.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing a part of the electrical configuration of a digital still camera.

  The entire operation of the digital still camera is controlled by the CPU 8.

  The digital still camera according to this embodiment can detect an image of a leaf when the subject image includes a leaf image, and can perform correction suitable for the image of the leaf. Thus, the shooting mode for performing correction suitable for the leaf image is the natural shooting mode, and is selected by the mode selection dial 9. Signals indicating the natural shooting mode, the normal shooting mode, and other modes selected by the mode selection dial 9 are input to the CPU 8.

  An imaging lens 1 and an IR (infrared) cut filter 2 are provided in front of the CCD 3 as an imaging device. The IR cut filter 2 is supported by a motor 5 so as to be inserted and retracted in front of the light receiving surface of the CCD 3. The operation of the motor 5 is controlled by an IR cut / filter operation control circuit 6 controlled by the CPU 8. Specifically, as will be described later, when the natural shooting mode is set, shooting of a subject with the IR cut filter 2 inserted in front of the light receiving surface of the CCD 3 and IR cut filter from the front of the light receiving surface of the CCD 3 are performed. The subject is photographed in the state in which 2 is retracted. The image part of the leaf of the subject image is detected using the two frames of the subject image obtained by these photographing, and the detected image part of the leaf is corrected.

  The CCD 3 is controlled by the shooting control circuit 7 and the subject is shot by the CCD 3. On the photodiode of the CCD 3, a filter that transmits a red light component, a filter that transmits a green light component, and a filter that transmits a blue light component are alternately provided. When the subject is imaged, an RGB color video signal representing the subject image is output from the CCD 3 and converted into RGB digital color image data by the analog / digital conversion circuit 4. The converted digital color image data is given to a display device (not shown), so that a subject image obtained by photographing is displayed.

  When the shutter release button is pressed when the normal shooting mode is set, the RGB color image data converted in the analog / digital conversion circuit 4 as described above is subjected to predetermined signal processing. It is recorded on a memory card (not shown). When the natural shooting mode is set, when the shutter release button is pressed, as described above, an image representing the subject image in which the subject is imaged with the IR cut filter inserted in front of the CCD 3 Data and color image data representing a subject image in which the subject is imaged with the IR cut filter retracted from the front of the CCD 3 are obtained. The obtained color image data representing the two subject images is given to the memory 10 and temporarily stored. Image data representing two frames of the subject image is read from the memory 10 and input to the image recognition / processing circuit 11. In the image recognition / processing circuit 11, the leaf image portion is detected as described above, and the detected leaf image portion is corrected. Color image data in which the leaf image portion is corrected (color image data obtained when the IR cut filter 2 is inserted in front of the CCD 3 will be corrected. The image recognition / processing circuit 11 outputs the color image data obtained when the image data is saved from the image processing unit 11 may be corrected). The outputted corrected color image data is outputted through the memory 10, and is given to the memory card through predetermined signal processing and recorded.

  FIG. 2 is a flowchart showing a photographing processing procedure of the digital still camera.

  It is confirmed by the mode selection dial 9 whether or not the natural shooting mode is set (step 21).

  If the natural shooting mode is set (YES in step 21), it is confirmed whether the IR cut filter 2 is inserted in front of the light receiving surface of the CCD 3 (step 22).

  If the IR cut filter 2 is inserted in front of the light receiving surface of the CCD 3 (YES in step 22), the subject is photographed in the inserted state (step 28), and color image data representing the subject image is stored in the memory 10. Remembered. Subsequently, the IR cut filter 2 is retracted from the front of the light receiving surface of the CCD 3 (step 29), and the subject is photographed again (step 30). Color image data obtained by photographing is stored in the memory 10.

  If the IR cut filter 2 is not inserted in front of the light receiving surface of the CCD 3 (NO in step 22), it is inserted in front of the light receiving surface of the CCD 3 (step 23) and the subject is photographed (step 24). Color image data obtained by photographing is stored in the memory 10. Subsequently, the IR cut filter 2 is retracted from the front of the light receiving surface of the CCD 3 (step 25), and the subject is photographed again. Color image data obtained by photographing is stored in the memory 10.

  In this way, when the natural shooting mode is set, image data obtained by shooting the subject with the IR cut filter 3 inserted in front of the light receiving surface of the CCD 3 and the IR cut filter 3 Two frames of image data are obtained, together with image data obtained by photographing the subject in a state where the image is retracted. Color correction processing is performed on the image portion of the leaf using the obtained two-frame image data (step 27).

  If the natural shooting mode is not set (NO in step 21), it is confirmed whether the IR cut filter 2 is inserted in front of the light receiving surface of the CCD 3 (step 31). If not inserted (NO in step 31), the IR cut filter 2 is inserted in front of the light receiving surface of the CCD 3 (step 32). With the IR cut filter 3 inserted in front of the light receiving surface of the CCD 3 (YES at step 31, 32), the subject is photographed (step 33).

  FIG. 3 is an example of a subject image obtained by photographing a subject with the IR cut filter 3 inserted in front of the light receiving surface of the CCD 3.

  In FIG. 3, in order to show that the subject image I1 is obtained by photographing the subject with the IR cut filter 3 inserted in front of the light receiving surface of the CCD 3, and the infrared component is cut off, the entire subject image I1 is shown. The hatching is shown across. The subject image I1 includes a leaf image portion 41.

  FIG. 4 is an example of a subject image obtained by photographing a subject with the IR cut filter 3 retracted from the front side of the light receiving surface of the CCD 3.

  4 is different from FIG. 3 in order to show that the subject image I2 is obtained by photographing the subject with the IR cut filter 3 retracted from the front of the light receiving surface of the CCD 3 and includes an infrared component. Further, hatching is not shown in the subject image I2. Since the subject image I2 is the same as the subject image I1, a leaf image portion 42 is included.

  FIG. 5 is a flowchart showing a processing procedure of color correction processing.

  Lab conversion is performed on the data representing all the pixels constituting the first subject image I1 obtained by cutting the infrared component obtained as described above according to the equations 1 to 3 (step 51). In Equation 1, R, G, and B are data levels of RGB color image data, and N is the bit precision of the image, for example, 8 bits.

L (R) = R / N
L (G) = G / N Formula 1
L (B) = B / N

  The evaluation value on the ab plane in the Lab color space for the first subject image I1 subjected to Lab conversion is recorded in the memory 10 (step 52). The evaluation value is a coordinate position on the ab plane in the Lab color space where a specific pixel in the first subject image I1 subjected to the Lab conversion is obtained from Expression 3.

  FIG. 6 shows the ab plane in the Lab color space.

  It is assumed that a specific pixel c1 of the first subject image I1 is indicated at (a1, b1) on the ab plane. Then, the evaluation value of the specific pixel c1 is represented by (a1, b1).

  Returning to FIG. 5, the Lab conversion is performed on the data representing all the pixels constituting the second subject image I2 including the infrared component (step 53). An evaluation value on the ab plane in the Lab color space for the second subject image I2 subjected to Lab conversion is recorded in the memory 10 (step 54).

  Referring again to FIG. 6, the specific pixel c2 of the second subject image I2 corresponding to the specific pixel c1 of the first subject image I1 is represented by (a2, b2) in the ab plane. Even if the pixel represents the same part of the subject, the pixel c1 in the subject image I1 from which infrared rays are cut and the pixel c2 in the subject image I2 that contains infrared rays differ from each other by a distance (difference) Δc on the ab plane. Since the difference Δc is large for the leaf portion of the subject, the leaf image portion of the subject image I1 or I2 is detected based on the difference Δc.

Referring again to FIG. 5, the difference Δc is calculated based on Equation 4 (step 55).

  If the calculated difference Δc is greater than or equal to the threshold (YES in step 56), the pixel is considered to represent a leaf. For this purpose, the pixel portion is recognized as a leaf and stored (step 57). If the calculated difference Δc is less than the threshold (NO in step 56), the process in step 57 is skipped. For all the regions of the subject images I1 and I2, comparison of the evaluation values from Steps 55 to 57 (determination as to whether or not the difference Δc is equal to or greater than a threshold value) is performed (Step 58). When the comparison of the evaluation values is completed for all regions (YES in step 58), color correction (saturation correction) is performed so that the image portion determined to be a leaf has a vivid color (step 59). As described above, this color correction is normally performed on the subject image I1 from which infrared rays are cut, but may be performed on the subject image I2 including infrared rays.

  In the embodiment described above, the difference is detected after converting the RGB color image data to the Lab color space. However, the difference can also be detected on the RGB color space without converting to the Lab color space.

  FIG. 7 is a flowchart showing a processing procedure when color correction is performed by detecting a difference in the RGB color space. In this figure, the same processes as those shown in FIG.

  Evaluation values in the RGB color space are stored in the memory 14 for all the pixels of the first subject image I1 (step 61). The evaluation values are coordinate values R1, G1, and B1 in the RGB color space of the pixels constituting the first subject image I1. Subsequently, the evaluation values in the RGB color space are stored in the memory 14 for all the pixels of the second subject image I2 (step 62).

A difference Δc between the evaluation value of the specific pixel of the first subject image I1 and the evaluation value of the specific pixel of the second subject image I2 corresponding to the pixel is calculated according to Equation 5 (step 63). If the calculated Δc is greater than or equal to the threshold, the pixel portion is recognized as a leaf (step 57).

  In this manner, the leaf image portion can be detected in the RGB color space state without performing the Lab conversion.

It is a block diagram which shows a part of electrical structure of a digital still camera. It is a flowchart which shows the process sequence of a digital still camera. It is an example of a subject image obtained by inserting an IR cut filter in front of the light receiving surface of a CCD. It is an example of the subject image obtained by retracting the IR cut filter from the front side of the light receiving surface of the CCD. It is a flowchart which shows a color correction process procedure. The ab plane in the Lab color space is shown. It is a flowchart which shows a color correction process procedure.

Explanation of symbols

2 Infrared cut filter 3 CCD
7 Shooting control circuit 8 CPU
11 Image recognition / processing circuit

Claims (4)

A solid-state electronic imaging device that images a subject and outputs color image data representing a color subject image;
An infrared cut filter supported so as to be inserted and retracted in front of the light receiving surface of the solid-state electronic imaging device;
The infrared cut filter is inserted in front of the light receiving surface of the solid-state electronic imaging device to image a subject, and outputs first color image data representing a first color subject image, and the infrared cut filter is The solid-state electronic image pickup device and the infrared cut filter are controlled so as to retreat from the front of the light receiving surface of the solid-state electronic image pickup device to pick up an image of the subject and output second color image data representing a second color subject image. Based on a difference in color space between corresponding pixels in the first color subject image and the second color subject image, and the imaging control means, the first color subject image or the second color subject Detecting means for detecting an image portion of a leaf included in the image;
A color imaging apparatus comprising:   2. The color according to claim 1, further comprising correction means for correcting the first color subject image data or the second color subject image data so that a leaf image portion detected by the detection means becomes clear. Imaging device. The solid-state electronic imaging device outputs RGB color image data,
Color space conversion means for converting the first color image data and the second color image data obtained under the control of the imaging control means into image data in the Lab color space;
The detection means is based on a difference in the Lab color space between corresponding pixels in the first color subject image and the second color subject image that have been subjected to Lab color space conversion by the color space conversion means. Detecting an image portion of a leaf included in the first color subject image or the second color subject image;
The color imaging device according to claim 1. Using a solid-state electronic imaging device, image the subject to obtain color image data representing the color subject image,
An infrared cut filter is supported in front of the light receiving surface of the solid-state electronic image pickup device so that it can be inserted and retracted.
The infrared cut filter is inserted in front of the light receiving surface of the solid-state electronic imaging device to image a subject, and outputs first color image data representing a first color subject image, and the infrared cut filter is The solid-state electronic image pickup device and the infrared cut filter are controlled so as to retreat from the front of the light receiving surface of the solid-state electronic image pickup device to pick up a subject and output second color image data representing a second color subject image. ,
A leaf included in the first color subject image or the second color subject image based on a difference in color space between corresponding pixels in the first color subject image and the second color subject image. Detect the image part of
A method for controlling a color imaging apparatus.

Images