JP2006005500A - Imaging device and imaging method, and imaging element - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device and an imaging method capable of carrying various processes without the need for using a particular sensor while obtaining a high cost reduction effect, and to provide an imaging element used for them. <P>SOLUTION: A pixel P0 without an ordinary color filter, two kinds of pixels P1, P2 respectively with particular filters including an ultraviolet ray eliminating function and an infrared ray eliminating function in place of the ordinary color filter are arranged by one pixel each to an imaging region 21 on which an optical image of an object of the imaging element is formed together with ordinary pixels R, G, B comprising photoelectric conversion elements provided with R, G, B color filters in response to the color arrangement of the Bayer arrangement. Further, one set of the particular pixels P0, P1, P2 is respectively provided to each of division areas 21a resulting from dividing the imaging region 21 into nine. In the case of imaging, the infrared ray component can be eliminated from an imaging signal and a type of a light source under a photographing environment can be discriminated on the basis of the pixel signals of the particular pixels P0, P1, P2 included in the imaging signal, and the imaging element can be driven by using a drive circuit of prior arts. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging device and an imaging method suitable for use in, for example, a digital camera, and an imaging device used therefor.

  2. Description of the Related Art Conventionally, many digital cameras, which are a kind of imaging device, display a subject image captured using a solid-state imaging device such as a CCD in a shooting standby state on a liquid crystal monitor as a through image regardless of a shutter operation. Such digital cameras are generally provided with a white balance adjustment function in order to accurately reproduce the color of the subject.

  The white balance adjustment is performed, for example, by controlling the gain of the image pickup signal output from the solid-state image pickup device for each of the R, G, and B color components. It is necessary to perform control according to the imaging environment (light source type). As a method for determining the light source type, for example, an infrared sensor that detects infrared light and a UV sensor that detects ultraviolet light are provided separately from the solid-state imaging device, and the light source type is determined from the detection results. There is a way to judge. However, such a method has a cost disadvantage due to an increase in the number of parts.

In order to solve this problem, for example, in Patent Document 1 below, there is another function that functions as an infrared sensor for detecting an infrared component between light receiving elements serving as pixels constituting an image on a light receiving surface of a CCD. There is described an imaging apparatus in which a light receiving element is provided in advance and a light source type is determined based on a pixel signal output from the light receiving element when the CCD is driven.
JP 2004-64468 A

  However, the above-described technique requires a CCD having a special basic structure composed of a light-receiving element for constructing an image and a light-receiving element for a sensor. Therefore, although the number of parts can be reduced, the cost of the CCD itself is high. Inevitably, a new drive circuit for driving it is also required. Therefore, it is difficult to effectively use hardware resources used in the conventional imaging apparatus, and there is a problem that sufficient cost merit cannot be obtained.

  The present invention has been made in view of the conventional problems, and an imaging apparatus and imaging method capable of performing various processes without using a special sensor while obtaining a high cost reduction effect, and an imaging used for the imaging apparatus. An object is to provide an element.

  In order to solve the above-mentioned problem, in the invention of claim 1, together with the first pixel in which the color filter corresponding to the color arrangement of the predetermined arrangement is provided in the imaging region where the optical image of the subject is formed, An image sensor in which a second pixel provided with a special filter instead of the color filter is arranged, and a predetermined process is executed based on a pixel signal of the second pixel included in an image signal output from the image sensor And a means for interpolating the second pixel in the image based on the imaging signal from a plurality of the first pixels located in the vicinity thereof.

  In such a configuration, the sensor according to the original imaging function and the filter function of the special filter provided in the second pixel is applied to the imaging element having the same basic structure as the conventional one other than the filter provided to the pixel in the imaging region. Functions, and various image processing using the sensor function can be performed. Accordingly, by appropriately selecting the special filter provided in the second pixel, various processes can be performed based on the imaging signal without providing a special sensor, and at the same time, the conventional hardware is applied to the driving circuit of the imaging element. Hardware resources can be used effectively.

  In the invention of claim 2, a plurality of types of the second pixels provided with special filters having different filter functions are provided in the imaging region of the imaging device, and the processing means Is configured to execute the predetermined processing based on pixel signals of a plurality of types of second pixels included in the imaging signal output from the imaging element.

  In this configuration, the image sensor can have a plurality of types of sensor functions, and various processes and accurate processes can be performed without providing a special sensor.

  In the invention of claim 3, the processing means determines the shooting environment based on the pixel signal of the second pixel included in the imaging signal output from the imaging element.

  In such a configuration, it is possible to determine the shooting environment based on the imaging signal without providing a special sensor.

  In the invention of claim 4, the processing means determines the type of the light source based on the pixel signal of the second pixel included in the imaging signal output from the imaging device.

  In such a configuration, the type of the light source can be determined based on the imaging signal without providing a special sensor.

  In the invention according to claim 5, the processing means removes an infrared component included in the imaging signal based on the pixel signal of the second pixel included in the imaging signal output from the imaging element. It was supposed to be.

  In this configuration, since the infrared component included in the imaging signal can be removed based on the imaging signal, the optical filter provided in the optical system for the purpose of removing the infrared component can be eliminated.

  According to a sixth aspect of the present invention, a third pixel in which neither the color filter nor the special filter is provided is further arranged in the imaging region of the imaging device, and the processing means The infrared component is removed from the imaging signal by subtracting the difference between the pixel signal of the second pixel and the pixel signal of the third pixel included in the signal from the imaging signal.

  In such a configuration, it is possible to more accurately remove the infrared component contained in the imaging signal.

  In the invention according to claim 7, the second pixels are arranged in the image pickup device in a distributed manner in a plurality of different areas of the image pickup area.

  In such a configuration, various processes can be performed based on information included in the imaging signal and obtained from the entire imaging region of the imaging device.

  According to an eighth aspect of the present invention, there is provided an imaging method for a subject in an imaging apparatus including an imaging device, wherein a predetermined array of colors is formed in an imaging region where an optical image of the subject of the imaging device is formed. A second pixel provided with a special filter instead of the color filter is arranged in advance together with the first pixel provided with a color filter corresponding to the arrangement, and is output from the image sensor when the image sensor is driven. Predetermined processing based on the pixel signal of the second pixel included in the captured image signal and interpolation of the second pixel in the image based on the captured image signal from the plurality of first pixels located in the vicinity thereof Interpolation processing was performed.

  According to such a method, an image pickup device having the same basic structure as the conventional one other than the filter provided in the pixels in the image pickup region is adapted to the original image pickup function and the filter function of the special filter provided in the second pixel. A sensor function can be provided, and various processes using the sensor function can be performed. Accordingly, by appropriately selecting the special filter provided in the second pixel, various processes can be performed based on the imaging signal without providing a special sensor, and at the same time, the conventional hardware is applied to the driving circuit of the imaging element. Hardware resources can be used effectively.

  According to a ninth aspect of the present invention, there is provided an image pickup device used in an image pickup apparatus, wherein a color filter corresponding to a predetermined color arrangement is provided in an image pickup region where an optical image of a subject is formed. In addition to the first pixel, a second pixel in which a special filter is provided instead of the color filter is arranged, and at the time of driving, a predetermined value based on the pixel signal of the second pixel included in the imaging signal output by itself And an interpolation process for interpolating the second pixel in the image based on the imaging signal from the plurality of the first pixels located in the vicinity thereof.

  In such a configuration, the basic structure other than the filter provided in the pixels in the imaging region is the same as the conventional one, and in use, it corresponds to the original imaging function and the filter function of the special filter provided in the second pixel. The sensor function can be exhibited. At the same time, conventional hardware resources in the imaging apparatus such as the drive circuit can be effectively used.

  According to a tenth aspect of the present invention, in addition to the first pixel in which a color filter corresponding to a predetermined color arrangement is provided in an imaging region where an optical image of a subject is formed, the color filter An imaging device in which a second pixel that is not provided is arranged, processing means for executing a predetermined process based on a pixel signal of a second pixel included in an imaging signal output from the imaging device, and the imaging signal And interpolating means for interpolating the second pixel in the image based on the plurality of the first pixels located around the second pixel.

  In such a configuration, an original imaging function, an original imaging function, and a sensor function using the second pixel as a sensor are added to an imaging element having the same basic structure as the conventional one other than the filter provided in the pixels in the imaging area. And various processes using the sensor function can be performed. Therefore, various processes can be performed based on the imaging signal without providing a special sensor, and at the same time, conventional hardware resources can be effectively used for the driving circuit of the imaging device.

  As described above, according to the imaging apparatus and imaging method of the present invention, it is possible to perform various processes based on the imaging signal without providing a special sensor, and at the same time, to the conventional hardware in the driving circuit of the imaging element, etc. Made it possible to use resources effectively. Therefore, various processes can be performed without using a special sensor while obtaining a high cost reduction effect. Moreover, it becomes possible by using the image sensor of the present invention.

  In particular, in the image pickup apparatus according to the second aspect, the image sensor can have a plurality of types of sensor functions, and various processes and accurate processes can be performed without providing a special sensor.

  According to another aspect of the present invention, the imaging environment can be determined based on the imaging signal. Further, according to the imaging apparatus of the fourth aspect, the type of the light source can be determined based on the imaging signal. Therefore, accurate white balance control and the like are possible.

  Further, in the imaging device according to claim 5, the optical filter provided in the optical system for the purpose of removing the infrared component can be eliminated. Further, according to the imaging device according to claim 6, it is included in the imaging signal. Infrared components can be removed more accurately.

  In the imaging device according to the seventh aspect, various processes can be performed based on information included in the imaging signal and obtained from the entire imaging region of the imaging element.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a digital camera according to the present invention.

  This digital camera has a CCD 2 as an image pickup device, and the CCD 2 is disposed behind a lens 1 inside the camera body. FIG. 2 is a schematic diagram showing an imaging area 21 of the CCD 2 on which an optical image of a subject is formed by the lens 1. In the imaging area 21, pixels (R, G, B) made of photoelectric conversion elements provided with R, G, B color filters corresponding to the color arrangement of the Bayer array are two-dimensionally arranged. However, in each divided area 21a divided into 9 (vertical 3 × horizontal 3) in the imaging region 21, the pixel (P0) in which the normal color filter is not provided and the normal color filter are substituted. Two types of pixels (P1, P2) provided with a special filter are provided at a predetermined location with a distance from each other (one set). One of the special filters (P1) is a filter having an ultraviolet removing function, and the other (P2) is a filter having an infrared removing function.

  That is, in the present embodiment, among the pixels in the imaging region 21, the pixels indicated by R, G, and B in the drawing are the first pixels in the present invention, and similarly, the pixel indicated by P0 is the third pixel, P1, The pixel indicated by P2 is the second pixel. The pixel P0 is the third pixel and at the same time the second pixel in another invention. In the following description, each pixel is represented as R, G, B, P0, P1, and P2, R, G, and B are called normal pixels, and P0, P1, and P2 are called specific pixels. To distinguish. Further, the number of pixels in the divided area 21a shown in FIG. 2 and the arrangement of the specific pixels P0, P1, and P2 in the divided area 21a are shown for convenience, and the positions and mutual positions of the specific pixels P0, P1, and P2 are shown. It is assumed that the interval between them does not interfere with pixel interpolation described later.

  On the other hand, in the digital camera, the CCD 2 is scanned and driven by the timing signal generator 3 and the vertical driver 4 at a predetermined cycle, and the optical image of the subject imaged in the imaging region 21 is photoelectrically converted for each pixel, and the intensity of light. An analog imaging signal made up of pixel signals corresponding to is output to the analog processing unit 5. The analog processing unit 5 includes a correlated double sampling circuit (CDS circuit) that removes noise included in the imaging signal input from the CCD 2 and an A / D converter that converts the imaging signal from which noise has been removed to a digital signal. Thus, the digital imaging signal converted by the analog processing unit 5, that is, Bayer data, is output to the gain adjustment circuit 6.

  The gain adjustment circuit 6 includes an R amplifier 6a, a G amplifier 6b, and a B amplifier 6c for each of the R, G, and B color components, and each amplifier 6a, 6b, The white balance of the image is controlled by controlling the gain of 6c and adjusting (weighting) each component of RGB. The imaging signal amplified by the gain adjustment circuit 6 is sent to the color process circuit 9 via the subtraction circuit 7 and the pixel interpolation circuit 8.

  The subtraction circuit 7 includes a register 7 a into which a pixel value for subtraction is written by the control unit 16, and subtracts the pixel value written in the register 7 a from the sequentially input pixel data and outputs it to the pixel interpolation circuit 8. To do.

  The pixel interpolation circuit 8 is the interpolation means of the present invention. The address memory 8a in which an address indicating a predetermined pixel position is written by the control unit 16 and the input image data for a predetermined number of pixels, that is, a thick frame in FIG. And an address buffer 8c for knowing the position of the pixel position of the input pixel data. When the pixel data of the pixel at the address written in the address memory 8a is input, the pixel interpolation circuit 8 performs pixel interpolation that converts the pixel value into an average value of eight pixels adjacent to the pixel, The processed pixel data is output, and the pixel data of the pixel of the address not written in the address memory 8a is output as it is.

  The color process circuit 9 performs color interpolation processing for generating RGB data for each pixel based on the image data (Bayer data) output from the pixel interpolation circuit 8, and a digital luminance signal (Y signal) from the RGB data for each pixel. YUV conversion processing for generating YUV data including color difference signals (Cb signal, Cr signal) is performed. The generated YUV data is transferred to a designated area of the DRAM 11 via the DMA controller 10 and developed as image data.

  The video encoder 12 generates a video signal based on the YUV data written in the DRAM 11 and outputs it to the display device 13 composed of an LCD. Thereby, in the imaging standby state in the REC mode for image recording, the captured subject is displayed on the display device 13 as a through image. Further, when recording is performed when the shutter key is pressed in this state, YUV data for one frame written in the DRAM 11 is sent to the compression / expansion unit 14, subjected to compression processing, and recorded in the flash memory 15. Further, when the PLAY mode for image reproduction is set, the image data recorded in the flash memory 15 is sent to the compression / decompression unit 14 via the DMA controller 10 and decompressed to be reproduced as YUV data. The recorded image is displayed on the display device 13.

  The control unit 16 includes a CPU, a ROM in which program data is stored, and a working RAM. The control unit 16 controls the operation of each unit described above according to a predetermined program and corresponds to a status signal sent from the operation unit 17. Control each part of the digital camera. Further, when the CCD 2 is driven, the processing described later is performed so that the subtracting circuit 7 functions as the processing means of the present invention, and itself functions as the processing means of the present invention.

  The operation unit 17 is provided with various keys such as the shutter key and a key for switching between the REC mode and the PLAY mode, and sends a state signal corresponding to the key operation to the control unit 16. A plurality of gain values set in the R amplifier 6a, the G amplifier 6b, and the B amplifier 6c of the gain adjustment circuit 6 are expected in the gain storage unit 18 when the control unit 16 performs white balance control to be described later. It is stored corresponding to various imaging environments (light source types). Specifically, gain values of ratios corresponding to three types of light sources of sunlight, room light (natural light), and tungsten are stored. The gain values of R, G, and B are empirical values. Further, the flaw defect address storage unit 19 stores an address written in the address memory 8a of the pixel interpolation circuit 8, that is, a flaw defect unique to the CCD 2 (a black flaw not having photoelectric sensitivity or a white flaw having photoelectric sensitivity). ) And the address data indicating the positions of the specific pixels P0, P1, and P2.

  FIG. 3 is a flowchart showing a procedure of signal control processing executed by the control unit 16 in accordance with the setting of the REC mode by the user.

  When the REC mode is set, the control unit 16 immediately reads out the defective pixel and the address data of the specific pixels P0, P1, P2 stored in the defective defect address storage unit 19 and reads them out of the pixel interpolation circuit 8. Write to the address memory 8a (step SA1). Thereby, in the pixel interpolation circuit 8, the Bayer data sent to the color process circuit 9 by the CCD 2 is the same as the pixel data of the above-mentioned specific pixels P0, P1, and P2 included in the Bayer data included in the Bayer data. To be interpolated.

  Subsequently, the control unit 16 is provided with a pixel value of the specific pixels P0 and P2 in the Bayer data output from the analog processing unit 5, that is, a pixel (P0) not provided with a color filter, and a filter having an infrared ray removing function. Acquisition of the pixel value of the current pixel (P2) is started (step SA2). Eventually, when the pixel values of all the specific pixels P0 and P2 (for nine divided areas 21a) can be stored (YES in step SA3), the average pixel values P0v and P2v, which are the respective average pixel values, are calculated (step SA4) Further, a difference between both average pixel values P0v and P2v is calculated (step SA5). Thereafter, the calculated pixel value of the difference is written in the register 7a of the arithmetic circuit 7 (step SA6). As a result, the arithmetic circuit 7 subtracts the difference between the average pixel values P0v and P2v from the pixel value of each pixel in the Bayer data input from the gain adjustment circuit 6, thereby removing the infrared component from the Bayer data, that is, the imaging signal. Is done. Thereafter, the control unit 16 repeats the processes of steps SA2 to SA6 until the REC mode is canceled.

  FIG. 4 shows a process related to white balance control that is performed in parallel with the signal control process described above by the control unit 16 at a predetermined frame period in order to ensure white balance of the captured image during the driving of the CCD 2 in the REC mode. It is the flowchart which showed the procedure.

  The control unit 16 starts acquiring pixel values of the specific pixels P0, P1, and P2 in the Bayer data output from the analog processing unit 5 (step SB1). Eventually, when the pixel values of all the specific pixels P0, P1, P2 (for nine divided areas 21a) can be stored (YES in step SB2), the average which is the average pixel value of each of the specific pixels P0, P1, P2 Pixel values P0v, P1v, and P2v are acquired (step SB3). Then, based on the relationship between the acquired average pixel values P0v, P1v, and P2v, the type of the light source is determined as follows.

  That is, when “P0v> P1v” and the average pixel value P0v of the normal pixel P0 is larger than the average pixel value P1v of the pixel P1 from which ultraviolet rays have been removed (YES in step SB4), shooting outdoors And the light source in the current shooting environment is determined to be sunlight (step SB5). Further, although “P0v <P1v” and “P0v> P2v”, the average pixel value P0v of the normal pixel P0 is smaller than the average pixel value P1v of the pixel P1 from which ultraviolet rays are removed, but infrared rays are removed. If it is larger than the average pixel value P2v of the existing pixel P2 (NO in step SB4, YES in step SB6), the light source in the current shooting environment is determined to be tungsten light (step SB7). Further, “P0v <P1v” and “P0v <P2v”, the average pixel value P0v of the normal pixel P0, the average pixel value P1v of the pixel P1 from which ultraviolet rays are removed, and the pixel P2 from which infrared rays are removed If the average pixel value is smaller than the average pixel value P2v (NO in step SB6), it is determined that the shooting is indoor, and the light source in the current shooting environment is determined as room light (step SB8).

  Thereafter, the R, G, and B gain values corresponding to any of the light sources determined as described above are read from the gain storage unit 18 and are read from the R amplifier 6a, the G amplifiers 6b, and B of the gain adjustment circuit 6. The white balance is adjusted by setting the amplifier 6c for use (step SB9).

  In the present embodiment configured as described above, the light source type is accurately set based on the imaging signal output from the CCD 2 without providing an infrared sensor for detecting infrared light or a UV sensor for detecting ultraviolet light. In addition, accurate white balance control according to the determined light source type can be performed. At the same time, the infrared light component contained in the imaging signal can be electrically removed, so that the infrared light that has been conventionally provided in the optical system and optically removes the infrared light that adversely affects the captured image. Cut filter can be abolished. Further, even in the configuration using the infrared cut filter, if the infrared light caused by the filter cannot be completely removed, it can be compensated. In addition, since the specific pixels P0, P1, and P2 that function as sensors in the CCD 2 are interpolated by the same process as the defective pixels, an image when they have normal color filters corresponding to the color arrangement of the Bayer array An inferior image can be obtained.

  Moreover, since the CCD 2 has the same basic configuration as the general one used conventionally except for the filter configuration shown in FIG. 2, only the color and presence of the filter are changed at the time of manufacture. The CCD 2 can be manufactured at a low cost. In addition, the conventional drive circuit (timing signal generator 3 and vertical driver 4) of the CCD 2 can be used as it is. Therefore, since conventional hardware resources can be used effectively, the infrared component that adversely affects the image can be removed and a more accurate white can be obtained without using a special sensor while obtaining a high cost reduction effect. Balance control can be performed.

  In the present embodiment, the infrared light component included in the imaging signal is removed by the subtracting circuit 7. However, for example, the infrared light component of the infrared light component can be removed by data processing by the control unit 16 without providing the subtracting circuit 7. It is good also as a structure which removes. In that case, it is possible to obtain the same effect as that of the present embodiment only by changing the program of the control unit 16 (so-called firmware), and therefore, a higher cost reduction effect can be obtained. About this point, unlike this Embodiment, it is the same also when not removing an infrared-light component.

  In the present embodiment, the average pixel values of the specific pixels P0, P1, and P2 arranged in the plurality (nine) of divided areas 21a in the white balance control by the control unit 16 described above. Only the average pixel values P0v, P1v, and P2v are used to determine the type of light source in the current shooting environment, but together with the average pixel values P0v, P1v, and P2v, other normal pixels R, G, B, In particular, the type of the light source may be determined using pixel values of the pixels R and B (for example, an average value or an integral value of all the pixels) that are effective in knowing the white balance state. In that case, the light source type can be determined more accurately.

  Further, in the present embodiment, the imaging area 21 of the CCD 2 is divided into nine and each of the divided areas 21a is not provided with the above three types of specific pixels P0, P1, P2, that is, normal color filters or special filters. The pixel P0, the pixel P1 provided with the filter having the ultraviolet ray removing function, and the pixel P2 provided with the filter having the infrared ray removing function are arranged one by one, but the specific pixels P0, P1, and P2 are arranged in the imaging region 21. Only one set may be provided at any location. However, the arrangement at a plurality of locations in the imaging region 21 performs the above-described signal processing and white balance control based on information obtained from the entire imaging region 21, that is, processing such as removal of infrared components in the imaging signal and determination of the light source type. There is an advantage that you can. Note that, when the specific pixels P0, P1, and P2 are arranged at a plurality of locations in the imaging region 21 as in the present embodiment, the specific pixels P0 provided at one or a plurality of locations (for example, locations corresponding to the focus area). , P1 and P2 may be selectively used to perform the above processing.

  Further, in the present embodiment, the CCD 2 is provided with three types of specific pixels P0, P1, and P2, and based on the pixel values, that is, the respective pixel signals included in the imaging signal, the infrared component in the imaging signal is removed. However, it may be configured such that either one of the processes or other processes are performed.

  Furthermore, depending on the content of the processing to be performed, the pixel P0 that is not provided with a normal color filter is abolished, or other filters are used instead of the pixels P1 and P2 that are provided with a filter having an ultraviolet removing function or an infrared removing function. One or a plurality of types of pixels provided with a special filter having a function may be provided. That is, at least one of the pixel P0 not provided with the normal color filter and one or more kinds of pixels provided with the special filter is provided, and the pixel P0 corresponds to a pixel other than the normal pixels R, G, and B. Arbitrary processing (including infrared component removal and light source type determination) may be performed.

  For example, in the present embodiment, the pixel values of both the pixel P0 not provided with the color filter and the pixel P2 provided with the filter having the infrared ray removing function (average pixel values P0v and P2v). In the above description, the infrared component is removed from the imaging signal by subtracting the difference between the pixel values of the normal pixels R, G, and B. However, the infrared component is removed from the imaging signal as follows. can do. That is, only a pixel provided with a special filter having a function of transmitting only infrared light is provided in the CCD 2, and the pixel value (pixel signal) of the pixel is set to the pixel value (imaging signal) of each of the normal pixels R, G, B. Or the difference between the pixel value (pixel signal) of the pixel P2 provided with the filter having the infrared ray removing function and the pixel value (pixel signal) of the normal pixel R is subtracted from the normal pixel R, G, B The infrared component can be removed from the imaging signal also by subtracting from each pixel value (imaging signal).

  Depending on the content of the required processing, the CCD 2 may be provided with a combination of pixels that are not provided with normal color filters and pixels that are provided with special filters other than normal color filters, or different types of special filters. It is better to provide a combination of a plurality of types of pixels provided, or a combination of a pixel that is not provided with a normal color filter and a plurality of types of pixels that are provided with different types of special filters as in this embodiment. Various processing and accurate processing can be performed.

  Further, in the present embodiment, the light source type is determined by software processing in the control unit 16, but a configuration in which these are performed by dedicated hardware may be employed. The present invention can be applied not only to a CCD but also to a MOS type solid-state image pickup device and an image pickup apparatus using the same, and is not limited to the above-described digital camera as long as it has a solid-state image pickup device. The present invention can also be applied to other imaging devices such as a mobile phone with a camera and a PDA with a camera.

1 is a block diagram of a digital camera according to the present invention. It is a schematic diagram which shows the imaging area of CCD. It is the flowchart which showed the procedure of the signal control process by a control part. It is the flowchart which showed the processing procedure of white balance control.

Explanation of symbols

1 Lens 2 CCD
21 imaging area 21a divided area 5 analog processing unit 6 gain adjustment circuit 7 subtraction circuit 8 pixel interpolation circuit 16 control unit 18 gain storage unit 19 flaw defect address storage unit R, G, B normal pixel (first pixel)
P0 specific pixel (third pixel)
P1, P2 specific pixel (second pixel)

Claims (10)

A second pixel in which a special filter is provided in place of the color filter, together with a first pixel in which a color filter corresponding to a predetermined color arrangement is provided in an imaging region where an optical image of a subject is formed An image sensor in which
Processing means for executing predetermined processing based on the pixel signal of the second pixel included in the imaging signal output from the imaging element;
An imaging apparatus comprising: interpolation means for interpolating the second pixel in the image based on the imaging signal from the plurality of first pixels located in the vicinity thereof. In the imaging region of the imaging device, a plurality of types of the second pixels provided with special filters having different filter functions are provided,
The imaging apparatus according to claim 1, wherein the processing unit performs the predetermined process based on pixel signals of a plurality of types of second pixels included in an imaging signal output from the imaging element.   The imaging apparatus according to claim 1, wherein the processing unit determines an imaging environment based on a pixel signal of a second pixel included in an imaging signal output from the imaging element.   The imaging apparatus according to claim 1, wherein the processing unit determines a type of a light source based on a pixel signal of a second pixel included in an imaging signal output from the imaging element.   The said processing means removes the infrared component contained in an image pick-up signal based on the pixel signal of the 2nd pixel contained in the image pick-up signal output from the said image pick-up element. Imaging device. In the imaging region of the imaging device, a third pixel in which neither the color filter nor the special filter is provided is further arranged,
The processing means removes an infrared component from the imaging signal by subtracting a difference between the pixel signal of the second pixel and the pixel signal of the third pixel included in the imaging signal from the imaging signal. The imaging apparatus according to claim 5, wherein:   The imaging device according to any one of claims 1 to 5, wherein the second pixel is arranged in the imaging element in a distributed manner in a plurality of different areas of the imaging area. An imaging method of a subject in an imaging device provided with an imaging element,
A special filter is provided in place of the color filter in the imaging area where the optical image of the subject of the imaging element is formed, together with the first pixel provided with a color filter corresponding to a predetermined color arrangement. Pre-arrange the second pixel,
When driving the image sensor,
A predetermined process based on a pixel signal of the second pixel included in the imaging signal output from the imaging element;
An imaging method comprising: interpolating the second pixel in the image based on the imaging signal from a plurality of the first pixels located in the vicinity thereof. An imaging element used in an imaging device,
A second pixel in which a special filter is provided in place of the color filter, together with a first pixel in which a color filter corresponding to a predetermined color arrangement is provided in an imaging region where an optical image of a subject is formed Is placed,
At the time of driving, a predetermined process based on the pixel signal of the second pixel included in the imaging signal output by itself, and the second pixel in the image based on the imaging signal are a plurality of the second pixels positioned around the second pixel. An imaging device, wherein an interpolation process for interpolating from one pixel is performed by an imaging device. In the imaging region where the optical image of the subject is formed, the second pixel not provided with the color filter is arranged together with the first pixel provided with the color filter corresponding to the color arrangement of the predetermined arrangement. An image sensor;
Processing means for executing predetermined processing based on the pixel signal of the second pixel included in the imaging signal output from the imaging element;
An imaging apparatus comprising: interpolation means for interpolating the second pixel in the image based on the imaging signal from the plurality of first pixels located in the vicinity thereof.

Images