Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
  • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
  • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
  • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/80—Camera processing pipelines; Components thereof
  • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
  • H04N23/843—Demosaicing, e.g. interpolating colour pixel values
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2209/00—Details of colour television systems
  • H04N2209/04—Picture signal generators
  • H04N2209/041—Picture signal generators using solid-state devices
  • H04N2209/042—Picture signal generators using solid-state devices having a single pick-up sensor
  • H04N2209/045—Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
  • H04N2209/046—Colour interpolation to calculate the missing colour values

Definitions

• a SOLID STATE COLOR IMAGE SENSOR USING A FIELD-STAGGERED COLOR FILTER PATTERN BACKGROUND OF THE INVENTION Field of the Invention This invention relates to a color video camera, and in particular to a camera of the type employing a single color image sensor providing interlaced scanning of luminance and chrominance information for both fields of a television frame.
• Luminance and chrominance information can be obtained from a single solid-state sensor by covering the photosites forming the sensor with a repetitive array of color filters. Since luminance is a combination of color information with a strong bias toward green, it is frequently based on just the green information provided by the sensor. The chrominance (color) content is frequently based on differences between the luminance (i.e., green information) and other primary colors (i.e., red and blue information) . Each separate color is isolated z ⁇ passing the scene light through an appropriate color filter before it strikes the sensor. As disclosed in "Color Filters and Processing Alternatives for One-Chip Cameras," by K. A.
• interlaced readout does not require that this sampling be the same, but only that each field include both red and blue photoelements.
• An interlaced image sampling pattern can still be obtained if the chrominance sample positions are shifted from one field to another.
• U.S.Patent No. 4,479,143 discloses a further modified Bayer pattern (see Fig. 4) in which red and blue sensor elements are provided at the same sampling rate as the green sensor elements, when horizontal alignment is considered. This arrangement provides improved chrominance resolution i ⁇ the horizontal direction.
• the color filter shown in Figure 3 is .t,i _esentative of a newer type of filter used in a single sensor camera for improved image quality.
• this filter which is described in relation to a camera disclosed in U.S. Patent No. 4,663,661
• the green information is sampled three times as frequently as red and blue information because it contributes more significantly to image resolution.
• an image sensor using this filter contains three green "luminance" pixels for every one red or blue "chroma" pixel.
• One problem is the occurrence of some amount of color edge artifacts (also referred to as aliasing) on horizontal chrominance edge transitions.
• the present invention involves an improved interlaced readout of color filter arrays of the type exemplified by Figure 3, wherein at least 75% of the image sensing elements are luminance sensing elements and the rest are either of two types of chrominance sensing elements.
• the chrominance sensing elements of the same type are separated in the horizontal scanning direction by at least three luminance sensing elements.
• the horizontal resolution can be improved by using interlaced readout patterns of color elements that form a repetitive "grid" of samples in which the chrominance sensing elements are disposed in different columns of the sensor.
• the invention improves upon the existing arrays by arranging the horizontal lines containing the chrominance sensing elements in line pairs with the chrominance sensing elements appearing in different columns of the sensor, thereby increasing the horizontal sampling frequency of the chrominance sensing elements.
• the lines are staggered in each line pair such that chrominance sensing elements of the first line of each line pair are shifted by half the distance between the chrominance sensing elements of the second of each line pair, thereby appearing in every other column of the array.
• alternate line pairs are themselves staggered in a "grid" of elements such that chrominance sensing elements of the same type appear in every column of the array.
• Figure 1 is an example of an image sensor employing a known "Bayer" checkerboard color filter pattern
• Figure 2 is an image sensor employing the color filter pattern of Fig. 1 as modified to provide both television fields of an interlaced scanning pattern;
• Figure 3 is an image sensor employing a known color filter pattern in which 75% of the color elements are responsive to luminance;
• Figure 4 is an image sensor employing a known variation of the color filter pattern of Figure 2;
• Figure 5 is a block diagram of a video camera incorporating a single color image sensor th color filtering "staggered" according to the sntion;
• Figure 6 is a plan view of the image sensor shown in Figure 5;
• Figure 7 is a diagram of a preferred color filter array used in connection with the image sensor shown in Figure 5;
• Figure 8 is a first alternative embodiment of a color filter array useful with the image sensor shown in Figure 5; and Figure 9 is a second alternative embodiment of a color filter array useful with the image sensor shown in Figure 5.
• Figure 5 identifies the basic elements of a video camera including a single solid state color i age sensor of the type embodied by the invention.
• Other elements of the camera may be readily selected from like elements known in the art.
• the exposure control, recorder and display sections of the camera which are not essential to an understanding of the invention, may be provided by ordinary components well known in this art.
• the microprocessor control system and the timing circuit may be implemented in a wholly conventional manner using available components and techniques.
• the color image sensor is disclosed in connection with a video camera, the invention is not necessarily related to this mode of image capture and, for example, could be incorporated into a film-to-video converter or a still video system.
• the video camera includes an optical section 10 for directing image _ht from a subject (not shown) to a solid state *-r image sensor 11.
• the image sensor 11 includes an array 12 of color filter elements covering a like array 14 of image sensing elements.
• a diaphragm 16 regulates the optical aperture through which image light passes.
• exposure is a function of light intensity and sensor integration time, which are respectively controlled by the diaphragm 16 and the timing circuit 29.
• a sample of the scene light is input to an exposure control circuit 20, which automatically operates the diaphragm 16.
• the image sensor 11 is shown in plan view in Figure 6.
• the sensor 11 includes an active image area 22 having a horizontal resolution of 768 pixels and a vertical resolution of 484 lines.
• the pixels are arranged in a known interline transfer configuration in which separate fields are transferred line by line to an output register 24.
• a preferred interline transfer image sensor having the resolution of Figure 6 may be obtained from the Eastman Kodak Co., Rochester, New York as catalog item KAI0370.
• a cross-hatched area 26 shows a region of dark lines and dark pixels which are used to obtain an indication of the sensor "dark current" level.
• Figure 7 shows a preferred color filter geometry according to the invention for the color filter array 12, which is an improvement upon a "three-green" color filter geometry of the type disclosed in U.S. Patent No. 4,663,661 and shown in Figure 3, which is assigned to the assignee of the present invention and incorporated by reference into the present description.
• This type of filter produces red, green and blue signals, which can also be considered luminance (green) and chrominance (red or blue) colors.
• a blurring filter 28 ( Figure 5) is placed in front of the color filter 12 to prefilter the image light in order to provide a known spread function. (It is to be further noted, however, that with other processing techniques, the blurring filter 28 may be altered or omitted.)
• the "three-green" color filter array is additionally arranged in a "field-staggered" geometry in which the red and blue pixels are spaced every second pixel horizontally, rather than every fourth pixel (as in Figure 3).
• the chrominance sensing elements of the same type e.g., red
• the chrominance sensing elements of the same type are arranged in line pairs (e.g., lines 1 and 2, 5 and 6, etc.) in which chrominance sensing elements of the first line of each pair are shifted by half the distance between the chrominance sensing elements of the second line, thereby appearing in every other column of the sensor.
• the color filter array 12 as shown in Figure 7 is disclosed in relation to a specific embodiment of the invention, and alternative configurations are possible. Two alternative configurations are shown in Figures 8 and 9. In both of these patterns, the horizontal lines containing the chrominance sensing elements are £. ?ed in a repetitive grid of lines with the chrominance sensing elements appearing in each column of the sensor.
• the repetitive grid includes two line pairs (e.g., line pairs 1 and 2, and 5 and 6) containing chrominance sensing elements of a first type (e.g., red) interleaved between two line pairs (e.g., line pairs 3 and 4, and 7 and 8) containing chrominance sensing elements of a second type (e.g., blue) such that chrominance sensing elements of the same type appear in every column of the sensor.
• a first type e.g., red
• line pairs 3 and 4, and 7 and 8 containing chrominance sensing elements of a second type (e.g., blue)
• red and blue samples occur in every column of the array, thus further improving the horizontal resolution of the red and blue records.
• the "field-staggered" pattern of Figure 8 may be preferred in some situations because the sampling is more uniformly arranged.
• a timing circuit 29 is provided for clocking the image signal from the output register 24 ( Figure 6) of the image sensor 11 to an analog-to-digital (A/D) converter circuit 30 and for generally sequencing other sections of the camera.
• the digital signal generated by the A/D converter 30 is output as a stream of color signals comprising, for the color filter array of Figure 7, a sequence of red (or blue) signals separated by three green signals.
• the quantized color signals are applied to a digital signal processing circuit 32 composed of two sections: a pre-processor section 34 and a post-processor section 36.
• the signal provided to the digital processing circuit 32 is partial resolution color data from the image sensor 11, that is, an un-interpolated sensor signal incorporating any anomalies due to the black level of the sensor or to any color and density irregularity in the scene.
• the signal output from the circuit 32 is a fully corrected, full resolution multi-color signal.
• the pre-processor section 34 initially provides black-level clamping and luminance interpolation for the quantized image signals.
• a stable sensor black reference value for black-level clamping is based on a sample of dark current signal values from pixels in the non-displayed and non-imaged dark area 26 of the image sensor 11 (Fig. 6).
• the clamped signals Prior to luminance interpolation, the clamped signals are processed for defect concealment by substituting, for defective pixels, the value of the closest previous horizontally adjacent pixel of the same color.
• the chroma values are converted to hues, that is, the ratio of red or blue to green, and chroma interpolation is performed upon the hue values.
• a white balance offset and a gain offset are also applied to the hue signal.
• the output of the pre-processor 34 are red, green, and blue signals formed from a combination of the interpolated hue values and the interpolated green values.
• the black level of the red, green, and blue signals is corrected for :.ens flare.
• the corrected signals are then color matrixed in order to properly correct the spectral sensitivities of the image sensor for the chromaticities of the display, and then gamma-corrected to adjust the non-linear contrast relationship between signal voltages at the input (sensor 11) and the light values at the output (display) of the system.
• the further corrected signals are then edge-enhanced and output from the post-processor 36.
• the digital processor 32 is further described in copending Serial No.
• the fully corrected color signals from the digital processor 32 are converted to analog signals by a digital-to-analog (D/A) converter 38 and are sent to either a conventional recorder 40 or to a display device 42. (The broken line 44 is meant to indicate that the color signals may be directly recorded in a digital storage device, if such is the choice.)
• a microprocessor 46 controls the operation of the camera and allows for user entry of pre-processing information, such as gain offset and color balance, to the pre-processor 34.
• the invention has been described in detail with particular reference to a presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.
• primary color filters that is, red, green, and blue
• complementary color filters for example, the red and blue filters can be replaced with yellow and cyan filters. In such a filter grid, green might be replaced by white.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• Spectroscopy & Molecular Physics (AREA)
• Color Television Image Signal Generators (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=24591888

Family Applications (1)

Country Status (3)

Cited By (2)

Families Citing this family (9)

Citations (3)

• 1992
  • 1992-01-22 WO PCT/US1992/000519 patent/WO1992013423A1/en not_active Application Discontinuation
  • 1992-01-22 JP JP92505695A patent/JPH05505718A/ja active Pending
  • 1992-01-22 EP EP92905733A patent/EP0522143A1/en not_active Withdrawn

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events