WO2012117584A1 - カラー撮像装置 - Google Patents
カラー撮像装置 Download PDFInfo
- Publication number
- WO2012117584A1 WO2012117584A1 PCT/JP2011/067420 JP2011067420W WO2012117584A1 WO 2012117584 A1 WO2012117584 A1 WO 2012117584A1 JP 2011067420 W JP2011067420 W JP 2011067420W WO 2012117584 A1 WO2012117584 A1 WO 2012117584A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- color
- filter
- pixels
- array
- Prior art date
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 59
- 239000003086 colorant Substances 0.000 claims description 48
- 238000000034 method Methods 0.000 claims description 31
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000000875 corresponding effect Effects 0.000 description 28
- 235000019557 luminance Nutrition 0.000 description 28
- 238000010586 diagram Methods 0.000 description 21
- 230000003287 optical effect Effects 0.000 description 7
- 230000000295 complement effect Effects 0.000 description 3
- 238000004040 coloring Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 241000579895 Chlorostilbon Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 229910052876 emerald Inorganic materials 0.000 description 1
- 239000010976 emerald Substances 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000010363 phase shift Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4015—Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/1462—Coatings
- H01L27/14621—Colour filter arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/843—Demosaicing, e.g. interpolating colour pixel values
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2209/00—Details of colour television systems
- H04N2209/04—Picture signal generators
- H04N2209/041—Picture signal generators using solid-state devices
- H04N2209/042—Picture signal generators using solid-state devices having a single pick-up sensor
- H04N2209/045—Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
Definitions
- the present invention relates to a color imaging apparatus, and more particularly to a color imaging apparatus capable of suppressing the occurrence of color moire.
- an optical low-pass filter made of a birefringent material such as crystal is disposed in front of the color image pickup device, and this is avoided by optically dropping a high frequency.
- coloring due to folding of the high-frequency signal can be reduced, but there is a problem that the resolution is lowered due to its adverse effect.
- the color filter arrangement of the color image sensor is determined based on an arrangement restriction condition in which any target pixel is adjacent in any one of three colors including the color of the target pixel and four sides of the target pixel.
- Patent Document 1 A color image sensor having a three-color random array that satisfies the above has been proposed.
- Patent Document 2 an image sensor having a color filter array alternately arranged at the second predetermined period in the other diagonal direction has been proposed (Patent Document 2).
- Patent Document 3 obtains the correlation in the horizontal, vertical, and diagonal (NE, NW) directions (four directions) using the peripheral pixels of the target pixel of the Bayer-array mosaic image, and A technique for performing pixel interpolation by performing weighting according to a ratio is described.
- Patent Document 4 an image pickup apparatus including a color image pickup device in which R and B of the three primary colors of RGB are arranged every three pixels in the horizontal and vertical directions and G is arranged between the R and B is proposed.
- the color image sensor described in Patent Document 4 has a larger number of G pixels that contribute the most to obtain a luminance signal than RB pixels because the resolution of the color difference signal may be lower than the luminance signal. Thus, the horizontal and vertical resolutions can be increased.
- JP 2000-308080 A JP 2005-136766 A JP 2010-104019 A JP-A-8-23543
- Patent Document 1 The three-color random arrangement described in Patent Document 1 is effective for low-frequency color moire, but is not effective for false colors in the high-frequency part.
- R, G, and B filters are periodically arranged in the horizontal and vertical lines of the color filter array.
- the invention extracts a local area of a predetermined image size centered on a target pixel when performing a synchronization process on a mosaic image output from an image sensor having the color filter array, and determines the color of the target pixel in the local area.
- An estimated value of another color at the target pixel position is calculated.
- the invention described in Patent Document 2 requires a calculation of a statistic (covariance value) relating to a color distribution shape and a regression calculation process, and has a problem that image processing becomes complicated.
- the pixel interpolation method described in Patent Document 3 is applied to a mosaic image having a Bayer array.
- G pixels are not continuous in the horizontal and vertical directions.
- the correlation degree in the horizontal and vertical directions cannot be obtained. For example, when a high frequency of vertical stripes or horizontal stripes is input, the correlation degree is erroneously determined, and pixel interpolation cannot be performed with high accuracy.
- the color image sensor described in Patent Document 4 is configured such that the ratio of the G pixel to the RB pixel is higher than that of the Bayer array, and the resolution in the horizontal and vertical directions can be increased.
- the imaging apparatus described in Patent Document 4 has a problem that false colors are likely to occur because the RGB signals output from the color imaging element are uniformly synchronized by the interpolation filter.
- the present invention has been made in view of such circumstances, and an object of the present invention is to provide a color imaging apparatus capable of suppressing the generation of false colors in a high-frequency part by simple image processing.
- an invention includes a plurality of pixels including photoelectric conversion elements arranged in a horizontal direction and a vertical direction, and a predetermined color disposed on the plurality of pixels.
- a color image sensor of a single plate type in which a color filter of a filter array is provided, wherein the color filter array includes a first filter corresponding to a first color that contributes most to obtain a luminance signal, and the And second filters corresponding to two or more second colors other than the first color are periodically arranged, and the first filters are arranged in horizontal, vertical, and diagonal (NE, NW) directions.
- an image acquisition unit that acquires a mosaic image corresponding to the color filter array from the color imaging device, and a target pixel for synchronization processing extracted from the mosaic image Pixel values of pixels adjacent to the first pixel in the vicinity of the target pixel and adjacent in the horizontal, vertical, and diagonal (NE, NW) directions are acquired, and pixel values of the adjacent pixels are obtained.
- a direction determining unit that determines whether the luminance correlation direction is one of the horizontal, vertical, and diagonal (NE, NW) directions, and a target pixel of the synchronization process extracted from the mosaic image
- a synchronization processing unit that calculates pixel values of other colors at the pixel position of the pixel, and calculates using the pixel values of the pixels of other colors existing in the correlation direction determined by the direction determination unit A processing unit; and a control unit that repeatedly operates the direction determination unit and the synchronization processing unit while moving the target pixel of the synchronization processing extracted from the mosaic image for each target pixel unit of the synchronization processing.
- the first filter that contributes most to obtain a luminance signal has two or more adjacent portions in the horizontal, vertical, and diagonal (NE, NW) directions (four directions). Therefore, it is possible to determine which direction of the correlation direction of the luminance is in the four directions based on the pixel values of the pixels adjacent to each direction with the minimum pixel interval. Then, when calculating the pixel value of the other color at the pixel position of the target pixel of the synchronization process extracted from the mosaic image, the pixel of the pixel of the other color existing in the correlation direction determined as described above By using the values, the pixel values of the pixels of other colors can be estimated with high accuracy, and the occurrence of false colors in the high frequency part can be suppressed.
- the predetermined color filter array of the color imaging device includes a basic array pattern in which the first filter and the second filter are arrayed, and the basic array A pattern is repeatedly arranged in the horizontal and vertical directions, and one or more of the first filter and the second filter are arranged in each line in the horizontal and vertical directions in the basic array pattern. Since one or more of the first filter and the second filter are arranged in the horizontal and vertical lines in the basic array pattern, horizontal and vertical color moire (false color) is generated. Can be suppressed and high resolution can be achieved.
- this color filter array can be processed according to the repeated pattern when performing the synchronization (interpolation) process in the subsequent stage.
- the color filter array is configured such that the first filter is arranged at a center and four corners in a 3 ⁇ 3 pixel group, and the 3 ⁇ 3 pixel group is arranged in a horizontal direction and It is preferable that they are repeatedly arranged in the vertical direction. Since the first filters are arranged at the four corners of the 3 ⁇ 3 pixel group, when the 3 ⁇ 3 pixel group is repeatedly arranged in the horizontal direction and the vertical direction, the color filter array is the first filter.
- a square array corresponding to 2 ⁇ 2 pixels composed of the above-described filters is included, and the pixel value of the 2 ⁇ 2 pixels is used, and the correlation among the horizontal, vertical, and diagonal (NE, NW) directions is high. The direction can be determined.
- the first filter in the color filter array, is arranged vertically and horizontally with a central filter in a 3 ⁇ 3 pixel group, and the 3 ⁇ 3 pixel group Are preferably arranged repeatedly in the horizontal and vertical directions. Since the first filter is arranged vertically and horizontally with the filter at the center of the 3 ⁇ 3 pixel group interposed therebetween, when the 3 ⁇ 3 pixel group is repeatedly arranged in the horizontal direction and the vertical direction, the color filter The arrangement is such that the first filter is adjacent to each other in the horizontal and vertical directions by sandwiching the filter at the center of the 3 ⁇ 3 pixel group. Pixel values of pixels corresponding to these first filters (a total of 8 pixels) can be used to determine the correlation direction of the four directions.
- the direction determination unit calculates a difference absolute value of pixel values of adjacent pixels for each of the horizontal, vertical, and oblique (NE, NW) directions. Then, the smallest direction among the absolute difference values for each direction is determined as the correlation direction.
- the direction determination unit calculates a ratio of pixel values of adjacent pixels for each of the horizontal, vertical, and oblique (NE, NW) directions, Of the ratios for each direction, the direction closest to 1 is determined as the correlation direction.
- a plurality of absolute difference values or ratios may be calculated for each direction, and a sum or average value of a plurality of absolute difference values or an average value of ratios may be calculated for each direction.
- the correlation direction can be determined with higher accuracy.
- the synchronization processing unit directly uses the pixel values of the pixels of the other colors existing in the correlation direction determined by the direction determination unit as the pixels of the target pixel.
- the synchronization processing unit is arranged in the vicinity of the target pixel when the other color pixel does not exist in the correlation direction determined by the direction determination unit.
- the pixel value of the other color is calculated by interpolating the pixel value of the target pixel based on the color difference or color ratio at the pixel position of the other color pixel.
- the color difference or color ratio at the pixel position of the other color pixel is the difference (color difference) between the pixel value of the pixel at the pixel position and the pixel value of the other color already estimated by the direction determination of the correlation direction. Or a ratio (color ratio).
- the color filter includes an R filter, a G filter, and a B filter corresponding to red (R), green (G), and blue (B) colors
- the target pixel of the synchronization processing is a G pixel
- the pixel value is G
- the target pixel of the synchronization processing is R pixel
- the pixel value is R
- there is no G or B pixel in the correlation direction determined by the direction determination unit If the pixel values of the G and B pixels are G and B, and the R pixel
- the color filter includes an R filter, a G filter, and a B filter corresponding to red (R), green (G), and blue (B) colors
- the target pixel of the synchronization processing is a G pixel
- the pixel value is G
- the target pixel of the synchronization processing is R pixel
- the pixel value is R
- there is no G or B pixel in the correlation direction determined by the direction determination unit If the pixel values of the G and B pixels are G and B, and
- the direction determination unit correlates when the difference values of the pixel values of pixels adjacent in the horizontal, vertical, and diagonal (NE, NW) directions are equal.
- the direction determining unit determines that there is no direction, and the direction determining unit determines that there is no correlation direction, the pixel value of another color at the pixel position of the target pixel of the synchronization process is determined as the pixel position. Is calculated using pixel values of pixels of other colors existing in the vicinity.
- the color filter includes an R filter, a G filter, and a B filter corresponding to red (R), green (G), and blue (B) colors
- the filter array is a first array corresponding to 3 ⁇ 3 pixels, in which G filters are arranged at the center and four corners, B filters are arranged above and below the center G filter, and R filters are arranged on the left and right Are arranged in the first array and the second array corresponding to 3 ⁇ 3 pixels, and G filters are arranged at the center and four corners, and R filters are arranged above and below the center G filter.
- the second array in which B filters are arranged on the left and right are alternately arranged in the horizontal direction and the vertical direction.
- the color filter includes an R filter, a G filter, and a B filter corresponding to red (R), green (G), and blue (B) colors
- the filter array is a first array corresponding to 3 ⁇ 3 pixels, an R filter is disposed at the center, B filters are disposed at four corners, and G filters are vertically and horizontally sandwiched with the center R filter interposed therebetween.
- the center pixel (R) of the 5 ⁇ 5 pixels is extracted.
- the first filter that contributes most to obtain a luminance signal has a color filter that includes two or more adjacent portions in the horizontal, vertical, and oblique (NE, NW) directions (four directions).
- a color imaging device wherein the first filter and a second filter corresponding to a second color other than the first color are periodically arranged in each of the horizontal and vertical lines. Since the color imaging element is used and the correlation direction of the luminance is determined based on the difference value between the pixel values of pixels adjacent to each direction, the correlation direction is determined using the pixel value of the minimum pixel interval. be able to.
- the pixel of the pixel of the other color existing in the correlation direction determined as described above Since the pixel value of the other color at the pixel position of the target pixel is calculated using the value, the pixel value of the pixel of the other color can be accurately estimated, and the generation of false colors in the high frequency part Can be suppressed.
- FIG. 1 is a block diagram showing an embodiment of a color imaging device according to the present invention
- FIG. 2 is a diagram showing a color filter array of color filters provided in the color image sensor of the first embodiment
- FIG. 3 is a diagram showing a basic array pattern included in the color filter array of the color image sensor of the first embodiment
- FIG. 4 is a diagram illustrating a state in which the basic array pattern of 6 ⁇ 6 pixels included in the color filter array of the color image sensor according to the first embodiment is divided into an A array and a B array of 3 ⁇ 3 pixels;
- FIG. 1 is a block diagram showing an embodiment of a color imaging device according to the present invention
- FIG. 2 is a diagram showing a color filter array of color filters provided in the color image sensor of the first embodiment
- FIG. 3 is a diagram showing a basic array pattern included in the color filter array of the color image sensor of the first embodiment
- FIG. 4 is a diagram illustrating a state in which the basic array pattern of 6 ⁇ 6 pixels included in the color filter array
- FIG. 5 is a diagram showing a state in which the basic array pattern of 6 ⁇ 6 pixels included in the color filter array of the color image sensor of the first embodiment is divided into an A array and a B array of 3 ⁇ 3 pixels, and these are arranged. Is;
- FIG. 6 is a diagram used for explaining a method of determining a correlation direction of luminance and a pixel interpolation method at the time of synchronization processing;
- FIG. 7A is a diagram showing a case where an oblique high-frequency image is incident on a color image sensor;
- FIG. 7B is a diagram used for explaining a method of determining a correlation direction when an oblique high-frequency image is incident on a color image sensor;
- FIG. 8 is a diagram showing a second embodiment of a color image sensor applied to the present invention
- FIG. 9 is a diagram showing a third embodiment of a color image sensor applied to the present invention
- FIG. 10 is a diagram showing a fourth embodiment of a color image sensor applied to the present invention
- FIG. 11 is a diagram showing a fifth embodiment of a color image sensor applied to the present invention
- FIG. 12 is a diagram showing a sixth embodiment of a color image sensor applied to the present invention
- FIG. 13 is a diagram showing a seventh embodiment of a color image sensor applied to the present invention
- FIG. 14 is a diagram used for explaining a problem of a color image sensor having a color filter with a conventional Bayer array.
- FIG. 1 is a block diagram showing an embodiment of a color imaging apparatus according to the present invention.
- the subject is imaged by the photographing optical system 10, and a light image indicating the subject image is formed on the light receiving surface of the color image sensor 12 (the color image sensor of the first embodiment).
- the color image sensor 12 includes a plurality of pixels (not shown) made of photoelectric conversion elements arranged in a horizontal direction and a vertical direction (two-dimensional arrangement), and a predetermined color filter arranged on the light receiving surface of each pixel.
- This is a single-plate type color image pickup device composed of an array of color filters.
- the color image sensor 12 has a color filter array in which filters of all colors of red (R), green (G), and blue (B) are periodically arranged in the horizontal and vertical lines.
- the G filter corresponding to G that contributes most to obtain a luminance signal includes two or more adjacent portions in the horizontal, vertical, and oblique (NE, NW) directions. Details of the color image sensor 12 will be described later.
- the subject image formed on the color image sensor 12 is converted into a signal charge corresponding to the amount of incident light by the photoelectric conversion element.
- the signal charge accumulated in each photoelectric conversion element is sequentially read out from the color imaging element 12 as a voltage signal (image signal) corresponding to the signal charge based on a drive pulse given from the drive unit 18 according to a command from the control unit 20.
- the image signals read from the color image sensor 12 are R, G, and B signals that indicate R, G, and B mosaic images corresponding to the color filter array of the color image sensor 12.
- the color image pickup device 12 is not limited to a CCD (Charge Coupled Device) color image pickup device, but may be another type of image pickup device such as a CMOS (Complementary Metal Oxide Semiconductor) image pickup device.
- CCD Charge Coupled Device
- CMOS Complementary Metal Oxide Semiconductor
- the image signal read from the color imaging device 12 is input to the imaging processing unit 14.
- the imaging processing unit 14 is a correlated double sampling circuit (CDS) for removing reset noise included in the image signal, and an AGC (Automatic Gain Control) circuit for amplifying the image signal and controlling it to a certain level. , And an A / D converter.
- CDS correlated double sampling circuit
- AGC Automatic Gain Control circuit
- the input image signal is subjected to correlated double sampling processing and amplified, and thereafter, RAW data obtained by converting the input image signal into a digital image signal is output to the image processing unit 16.
- the image processing unit 16 includes a white balance correction circuit, a gamma correction circuit, and a synchronization processing circuit according to the present invention (all RGB colors for each pixel from an RGB mosaic image associated with the color filter array of the single-plate color image sensor 12).
- the RAW data is subjected to required signal processing to generate image data (YUV data) composed of luminance data (Y data) and color difference data (Cr, Cb data).
- the image data generated by the image processing unit 16 is compressed by a compression / decompression processing circuit for a still image according to the JPEG (joint photographic experts group) standard, and the moving image is MPEG2 (moving). After being subjected to compression processing conforming to the standards (experts group) standard, it is recorded on a recording medium (memory card), and is also output and displayed on a display device (not shown) such as a liquid crystal monitor.
- a recording medium memory card
- the color filter array of the color image sensor 12 has the following features (1), (2), and (3).
- FIG. 2 is a diagram showing a color filter array of color filters provided in the color image sensor 12.
- the color filter array of the color image sensor 12 includes a basic array pattern P (pattern indicated by a thick frame) composed of a square array pattern corresponding to 6 ⁇ 6 pixels. It is repeatedly arranged in the horizontal and vertical directions. That is, in this color filter array, R, G, and B color filters (R filter, G filter, and B filter) are arrayed with a predetermined periodicity.
- the R filter, G filter, and B filter are arranged with a predetermined periodicity in this way, when performing R, G, B signal readout processing from the color image sensor 12, processing is performed according to a repetitive pattern. It can be performed.
- FIG. 3 shows a state in which the basic array pattern P shown in FIG. 2 is divided into 4 ⁇ 3 ⁇ 3 pixels.
- the basic array pattern P includes a 3 ⁇ 3 pixel A array surrounded by a solid frame and a 3 ⁇ 3 pixel B array surrounded by a broken frame as shown in FIG. It can also be understood that the array is arranged alternately in the vertical direction.
- G filters which are luminance system pixels, are arranged at the four corners and the center, and are arranged on both diagonal lines.
- the R filter is arranged in the horizontal direction with the central G filter interposed therebetween, and the B filter is arranged in the vertical direction.
- the B filter is arranged in the horizontal direction with the central G filter interposed therebetween.
- the R filters are arranged in the vertical direction. That is, in the A array and the B array, the positional relationship between the R filter and the B filter is reversed, but the other arrangements are the same.
- the basic arrangement pattern P of the color filter arrangement shown in FIG. 2 includes a portion where the G filter is adjacent to two or more in the horizontal, vertical, and oblique (NE, NW) directions.
- the color filter array in which the basic array pattern P is arrayed in the horizontal direction and the vertical direction includes a square array corresponding to 2 ⁇ 2 pixels composed of G filters.
- the G filters that are luminance system pixels are arranged at the four corners and the center in the 3 ⁇ 3 pixels in the A array or the B array, and the 3 ⁇ 3 pixels alternate in the horizontal direction and the vertical direction.
- a square array of G filters corresponding to 2 ⁇ 2 pixels is formed.
- the pixel value of 2 ⁇ 2 pixels formed by these G filters can be used for calculation (direction determination) of the correlation direction of the luminance in the horizontal direction, the vertical direction, and the oblique direction (NE, NW) direction.
- 2 ⁇ 2 G pixels corresponding to the G filter are extracted from the mosaic image output from the color image sensor 12, and the pixel values of the G pixels are set to G1, G2 in order from the upper left to the lower right.
- the direction determination circuit included in the synchronization processing circuit calculates a difference absolute value for each direction.
- the absolute difference value in the vertical direction is (
- the absolute difference value in the horizontal direction is (
- the difference absolute value in the direction is
- the difference absolute value in the upper left diagonal direction is
- the direction discriminating circuit discriminates that there is a correlation (correlation direction) in the direction of taking the smallest difference absolute value among these four correlation absolute values.
- the correlation direction is determined from the 2 ⁇ 2 G pixels adjacent to each other as described above, the correlation direction can be determined with the minimum pixel interval. That is, the correlation direction can be accurately determined without being affected by high frequency.
- the direction in which the change in luminance is the smallest is determined based on the difference value between the pixel values of adjacent G pixels.
- the direction in which the change in luminance is the smallest may be determined based on the ratio of the pixel values of the G pixels.
- the ratio (the average value of the ratios when calculating a plurality of ratios for each of the four directions) is the direction closest to 1. It is determined that the direction of the change in luminance is small.
- the synchronization processing circuit determines the determined correlation direction when calculating a pixel value of another color at the pixel position of the target pixel of the synchronization process.
- the pixel values of the pixels of other colors existing in are used.
- pixels of all colors of R pixel, G pixel, and B pixel exist in the horizontal direction and the vertical direction, if it is determined that there is a correlation direction in the horizontal direction or the vertical direction, Pixel values of pixels of other colors in the vicinity of the target pixel that are present in the direction or the vertical direction are acquired. Then, the obtained pixel value of one pixel or the interpolated value of the pixel values of a plurality of pixels is set as a pixel value of another color at the pixel position of the target pixel.
- the R pixel value at the pixel position of the G22 pixel is the R12 or R32 pixel value as it is, or the R12 and R32 pixels. Determined by interpolating from values.
- the pixel value of B at the pixel position of the pixel of G22 is determined by using the pixel value of B02 or B42 as it is, or by interpolating from the pixel values of B02 and B42.
- the R pixel value at the pixel position of the B02 pixel is determined by using the R12 pixel value as it is or interpolating from the R12 and R32 pixel values.
- the G pixel value at the pixel position of the B02 pixel is set to the G22 pixel value as it is, or is interpolated from the G pixel value and the G22 pixel value at the same horizontal position in the adjacent B array. To decide.
- the color filter array of the color image sensor 12 of the first embodiment has only G pixels in the diagonal direction (diagonal direction) centering on the G pixels of the 3 ⁇ 3 pixel A array, the correlation is present. If it is determined that the direction is a continuous diagonal direction of G pixels, the pixel value of the target pixel is interpolated by the color difference between the pixel values of the R and B pixels in the vicinity where the G pixel value is calculated. Pixel value is calculated.
- the pixel value G11 of G11 is interpolated using the pixel values R12 and R21 of the pixels R12 and B21 in the vicinity of the pixel of G11 and the color difference between the G pixel values G12 ′ and G21 ′ at the pixel positions of these pixels. , R and G pixel values R11 ′ and B11 ′ at the pixel position of the G11 pixel are calculated.
- interpolation is performed using the following formula.
- R11 ′ G11 + (R12 ⁇ G12 ′) (a)
- B11 ′ G11 + (B21 ⁇ G21 ′) (b)
- the pixel values R22 ′ and B22 ′ of the R and B pixels at the pixel position of the G22 pixel are obtained by using the color difference between the adjacent R12 and R32 pixels and the B21 and B23 pixel positions. Are interpolated to calculate R and G pixel values R22 ′ and B22 ′ at the pixel position of the G22 pixel.
- interpolation is performed using the following formula.
- R22 ′ G22 + ⁇ (R12 + R32) / 2 ⁇ (G12 ′ + G32 ′) / 2 ⁇ (c)
- B22 ′ G22 + ⁇ (B21 + B23) / 2 ⁇ (G21 ′ + G23 ′) / 2 ⁇ (d)
- the target pixel of the synchronization process is a G pixel, and when calculating the R and B pixel values at this pixel position, there are only G pixels in an oblique direction.
- the target pixel of the synchronization process is an R pixel, and when calculating the G and B pixel values at this pixel position, there are G pixels or B pixels in an oblique direction.
- the target pixel of the synchronization processing is a B pixel, and there is no G pixel or R pixel in the diagonal direction when calculating the G and R pixel values at this pixel position.
- the pixel value of the target pixel is interpolated by the color difference of the pixel values of the neighboring pixels for which the RGB pixel values are calculated, and other colors Pixel values can be calculated.
- the method of calculating the pixel values of other colors by interpolating the pixel values of the target pixel based on the color difference of neighboring pixels is as follows.
- the target pixel of the synchronization processing is a G pixel
- the pixel value is G
- there is no R or B pixel in the correlation direction determined by the direction determination circuit the R and B pixels in the vicinity of the G pixel the values R, B, pixel values G R of G at the pixel positions of these pixels, when G B, R at the position of the target pixel, the pixel value R G and B pixels, the B G, calculated by the following equation To do.
- the target pixel of the synchronization process is an R pixel
- the pixel value is R
- there is no G or B pixel in the correlation direction determined by the direction determination circuit G, B in the vicinity of the R pixel the pixel value of the pixel G, B, a pixel value R G of R at the pixel positions of these pixels, when R B, G at the position of the target pixel, the pixel value G R and B pixels, the B R, following Calculate by the formula.
- the pixel values G, R, a pixel value of B at the pixel position of the pixel B G, and B R, G at the position of the target pixel, the pixel value G B of the R pixel, the R B the following equation Calculated by
- the pixel value of the target pixel is determined by the color difference of the other color pixels in the vicinity.
- the pixel values of the other colors are calculated by interpolating the pixel values of the other pixels.
- the present invention is not limited to this. You may make it calculate.
- the target pixel of the synchronization processing is a G pixel
- the pixel value is G
- there is no R or B pixel in the correlation direction determined by the direction determination circuit the R and B pixels in the vicinity of the G pixel the values R, B, pixel values G R of G at the pixel positions of these pixels, when G B, R at the position of the target pixel, the pixel value R G and B pixels, the B G, calculated by the following equation To do.
- the pixel values G, R, a pixel value of B at the pixel position of the pixel B G, and B R, G at the position of the target pixel, the pixel value G B of the R pixel, the R B the following equation Calculated by
- the present invention is premised on a color image pickup device in which R, G, B pixels are periodically arranged in each line in the horizontal direction and the vertical direction, but R, G also in an oblique (NE, NW) direction.
- NE, NW oblique
- FIG. 8 is a diagram showing a second embodiment of a color image sensor applied to the present invention, and particularly shows a color filter array of color filters provided in the color image sensor.
- the color filter array of this color image sensor includes a basic array pattern composed of square array patterns corresponding to 6 ⁇ 6 pixels, and this basic array pattern is repeatedly arranged in the horizontal direction and the vertical direction. Yes. That is, in this color filter array, R, G, and B color filters (R filter, G filter, and B filter) are arrayed with a predetermined periodicity.
- filters of all colors of R, G, and B are arranged in the horizontal and vertical lines in the basic array pattern.
- the A array of 3 ⁇ 3 pixels surrounded by a solid line frame and the B array of 3 ⁇ 3 pixels surrounded by a broken line frame can be regarded as a color filter array arranged alternately in the horizontal and vertical directions.
- an R filter is arranged at the center of 3 ⁇ 3 pixels, B filters are arranged at four corners, and G filters are arranged on the upper, lower, left, and right sides of the center R filter.
- B filters are arranged at the center of three pixels, R filters are arranged at the four corners, and G filters are arranged at the top, bottom, left and right with the center B filter interposed therebetween. That is, in the A array and the B array, the positional relationship between the R filter and the B filter is reversed, but the other arrangements are the same.
- the color filter array shown in FIG. 8 includes two adjacent portions of the G filter in the horizontal, vertical, and diagonal (NE, NW) directions.
- the G filter which is a luminance system pixel is arranged vertically and horizontally with the filter at the center of the 3 ⁇ 3 pixel group in the A or B array, the 3 ⁇ 3 pixel group is repeated in the horizontal and vertical directions. When arranged, two pixels are adjacent to each other in the horizontal and vertical directions across the filter at the center of the 3 ⁇ 3 pixel group.
- the correlation of the luminance in each of the horizontal, vertical, and diagonal (NE, NW) directions is obtained based on the pixel values of the pixels corresponding to the G filter which is a luminance system pixel (total 8 pixels). This can be determined by the pixel value of the minimum pixel interval.
- the color filter array of the color image sensor of the second embodiment has the same characteristics as the characteristics (1), (2) and (3) of the color filter array of the color image sensor 12 of the first embodiment. Thus, direction determination and pixel interpolation according to the present invention can be performed.
- the R pixel at the center of the A array or the B pixel at the center of the B array is the target pixel for simultaneous processing, and the diagonal direction is the correlation direction. That is, there is no G pixel in the oblique direction.
- the G pixel value can be calculated based on the above-described equations (2) and (3) or equations (5) and (6).
- FIG. 9 is a diagram showing a third embodiment of a color image sensor applied to the present invention, and particularly shows a color filter array of color filters provided in the color image sensor.
- the color filter array of this color imaging device includes a basic array pattern (pattern indicated by a thick frame) composed of a square array pattern corresponding to 5 ⁇ 5 pixels, and this basic array pattern is in the horizontal direction. And repeatedly arranged in the vertical direction. That is, in this color filter array, R, G, and B color filters (R filter, G filter, and B filter) are arrayed with a predetermined periodicity.
- filters of all the colors R, G, and B are arranged in the horizontal and vertical lines in the basic array pattern.
- two or more continuous G filters are arranged in the horizontal, vertical, and diagonal directions (NE, NW) in the basic array pattern. Based on the pixel values of these continuous G pixels, the correlation between the luminances in the horizontal, vertical, and diagonal (NE, NW) directions can be determined with the minimum pixel interval.
- the color filter array of the color image sensor of the third embodiment has the same characteristics as the characteristics (1), (2) and (3) of the color filter array of the color image sensor 12 of the first embodiment. Thus, direction determination and pixel interpolation according to the present invention can be performed.
- FIG. 10 is a diagram showing a fourth embodiment of a color image sensor applied to the present invention, and particularly shows a color filter array of color filters provided in the color image sensor.
- the color filter array of this color imaging device includes a basic array pattern (pattern indicated by a thick frame) composed of a square array pattern corresponding to 7 ⁇ 7 pixels, and this basic array pattern is in the horizontal direction. And repeatedly arranged in the vertical direction. That is, in this color filter array, R, G, and B color filters (R filter, G filter, and B filter) are arrayed with a predetermined periodicity.
- filters of all the colors R, G, and B are arranged in each line in the horizontal direction and the vertical direction in the basic array pattern.
- continuous G filters are arranged in the horizontal, vertical, and diagonal directions (NE, NW), respectively.
- NE, NW diagonal directions
- the correlation between the luminances in the horizontal, vertical, and diagonal (NE, NW) directions can be determined by the minimum pixel interval based on the pixel values of these consecutive G pixels.
- the color filter array of the color image sensor of the fourth embodiment has the same characteristics as the characteristics (1), (2) and (3) of the color filter array of the color image sensor 12 of the first embodiment. Thus, direction determination and pixel interpolation according to the present invention can be performed.
- FIG. 11 is a diagram showing a fifth embodiment of a color image sensor applied to the present invention, and particularly shows a color filter array of color filters provided in the color image sensor.
- the color filter array of this color image sensor includes a basic array pattern (pattern indicated by a thick frame) consisting of a square array pattern corresponding to 8 ⁇ 8 pixels, and this basic array pattern is in the horizontal direction. And repeatedly arranged in the vertical direction. That is, in this color filter array, R, G, and B color filters (R filter, G filter, and B filter) are arrayed with a predetermined periodicity.
- filters of all the colors R, G, and B are arranged in the horizontal and vertical lines in the basic array pattern.
- continuous G filters are arranged in the horizontal, vertical, and diagonal directions (NE, NW), respectively. That is, there are four groups of G pixels including 2 ⁇ 2 pixels that are continuous vertically and horizontally in the basic array pattern.
- the correlation between the luminances in the horizontal, vertical, and diagonal (NE, NW) directions can be determined by the minimum pixel interval based on the pixel values of these consecutive G pixels.
- the color filter array of the color image sensor of the fifth embodiment has the same features as the characteristics (1), (2) and (3) of the color filter array of the color image sensor 12 of the first embodiment. Thus, direction determination and pixel interpolation according to the present invention can be performed.
- FIG. 12 and FIG. 13 are views showing sixth and seventh embodiments of the color image sensor applied to the present invention, respectively, and particularly show the color filter arrangement of the color filters provided in the color image sensor. .
- the color filter array of the color image pickup device of the sixth embodiment shown in FIG. 12 is obtained by arranging the A array of 3 ⁇ 3 pixels shown in the first embodiment shown in FIG. 3 side by side in the horizontal and vertical directions. It is configured.
- the color filter array of the color image sensor of the seventh embodiment shown in FIG. 13 is arranged in the horizontal and vertical direction by arranging the B array of 3 ⁇ 3 pixels shown in the first embodiment shown in FIG. Has been configured.
- the color filter array of the color image sensor of the sixth and seventh embodiments has an advantage that the basic array pattern is as small as 3 ⁇ 3 pixels, and the simultaneous processing of R, G, and B signals becomes easy.
- the filters of all the colors R, G, and B are not arranged in the horizontal and vertical lines in the basic arrangement pattern, and the color filter array of the color image sensor 12 of the first embodiment.
- it has the same features as the features (1) and (3), and can perform direction discrimination and pixel interpolation according to the present invention.
- RGB primary colors + other colors for example, emerald
- the present invention can also be applied to a color image pickup apparatus including a color image pickup element having four color filters of E)).
- the present invention provides a color image pickup including a color image pickup element having a color filter of four complementary colors, in which G is added to C (cyan), M (magenta), and Y (yellow) which are complementary colors of the primary colors RGB. It can also be applied to devices.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Theoretical Computer Science (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Electromagnetism (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- Computer Hardware Design (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
Description
RG=G+(R-GR),BG=G+(B-GB)
により算出し、前記同時化処理の対象画素がR画素、その画素値がRであり、前記方向判別部により判別された相関方向にG、又はB画素が存在しない場合、前記R画素の近傍のG、B画素の画素値をG,B、これらの画素の画素位置におけるRの画素値をRG、RBとすると、前記対象画素の位置におけるG、B画素の画素値GR,BRを、次式、
GR=R+(G-RG),BR=R+(B-RB)
により算出し、前記同時化処理の対象画素がB画素、その画素値がBであり、前記方向判別部により判別された相関方向にG、又はR画素が存在しない場合、前記B画素の近傍のG、R画素の画素値をG,R、これらの画素の画素位置におけるBの画素値をBG、BRとすると、前記対象画素の位置におけるG、R画素の画素値GB,RBを、次式、
GB=B+(G-BG),RB=B+(R-BR)
により算出している。
RG=G×(R/GR),BG=G×(B/GB)
により算出し、前記同時化処理の対象画素がR画素、その画素値がRであり、前記方向判別部により判別された相関方向にG、又はB画素が存在しない場合、前記R画素の近傍のG、B画素の画素値をG,B、これらの画素の画素位置におけるRの画素値をRG、RBとすると、前記対象画素の位置におけるG、B画素の画素値GR,BRを、次式、
GR=R×(G/RG),BR=R×(B/RB)
により算出し、前記同時化処理の対象画素がB画素、その画素値がBであり、前記方向判別部により判別された相関方向にG、又はR画素が存在しない場合、前記B画素の近傍のG、R画素の画素値をG,R、これらの画素の画素位置におけるBの画素値をBG、BRとすると、前記対象画素の位置におけるG、R画素の画素値GB,RBを、次式、
GB=B×(G/BG),RB=B×(R/BR)
により算出している。
図1は本発明に係るカラー撮像装置の実施形態を示すブロック図である。
カラー撮像素子12のカラーフィルタ配列は、下記の特徴(1)、(2)及び(3)を有している。
図2はカラー撮像素子12に設けられているカラーフィルタのカラーフィルタ配列を示す図である。図2に示すように、カラー撮像素子12のカラーフィルタ配列は、6×6画素に対応する正方配列パターンからなる基本配列パターンP(太枠で示したパターン)を含み、この基本配列パターンPが水平方向及び垂直方向に繰り返し配置されている。即ち、このカラーフィルタ配列は、R、G、Bの各色のフィルタ(Rフィルタ、Gフィルタ、Bフィルタ)が所定の周期性をもって配列されている。
図2に示すカラーフィルタ配列は、R、G、Bの全ての色のフィルタが、基本配列パターンP内の水平方向及び垂直方向の各ライン内に配置されている。これにより、後述するように、同時化処理時に相関方向が水平方向又は垂直方向と判別されると、水平方向又は垂直方向に存在する他の色の画素の画素値を使用して補間することができるため、色モワレ(偽色)の発生を抑圧することができる。また、上記のようにして偽色の発生を抑制することができるため、光学ローパスフィルタを光学系の入射面から撮像面までの光路に配置しないようにでき、又は光学ローパスフィルタを適用する場合でも偽色の発生を防止するための高周波数成分をカットする働きの弱いものを適用することができ、解像度を損なわないようにすることができる。
図2に示すカラーフィルタ配列の基本配列パターンPは、Gフィルタが、水平、垂直、及び斜め(NE,NW)方向に2以上隣接する部分を含んでいる。この基本配列パターンPが水平方向及び垂直方向に配列されたカラーフィルタ配列は、Gフィルタからなる2×2画素に対応する正方配列を含んでいる。
次に、画像処理部16の同時化処理回路の処理内容について説明する。
B11’=G11+(B21-G21’) …(b)
上記式(a)及び(b)において、G12’=G01,G21’=G10である。即ち、画素値G12’,G21’は、左斜め方向の方向判別により推定したR12,B21の画素の画素位置におけるGの画素値である。
B22’=G22+{(B21+B23)/2-(G21’+G23’)/2} …(d)
尚、G32’=G43,G23’=G34である。
この式(1)は、前述した式(a),(b)に相当するものである。
また、同時化処理の対象画素がB画素、その画素値がBであり、方向判別回路により判別された相関方向にG、又はR画素が存在しない場合、前記B画素の近傍のG、R画素の画素値をG,R、これらの画素の画素位置におけるBの画素値をBG、BRとすると、前記対象画素の位置におけるG、R画素の画素値GB,RBを、次式により算出する。
尚、上記式(1)~(6)に限らず、同時化処理の対象画素の近傍に、補間して求めようとする色と同じ色の画素が複数存在する場合には、式(c)~(d)と同様に、複数の画素の平均の色差を使用して補間してもよい。
同様に、同時化処理の対象画素がR画素、その画素値がRであり、方向判別回路により判別された相関方向にG、又はB画素が存在しない場合、前記R画素の近傍のG、B画素の画素値をG,B、これらの画素の画素位置におけるRの画素値をRG、RBとすると、前記対象画素の位置におけるG、B画素の画素値GR,BRを、次式により算出する。
また、同時化処理の対象画素がB画素、その画素値がBであり、方向判別回路により判別された相関方向にG、又はR画素が存在しない場合、前記B画素の近傍のG、R画素の画素値をG,R、これらの画素の画素位置におけるBの画素値をBG、BRとすると、前記対象画素の位置におけるG、R画素の画素値GB,RBを、次式により算出する。
一方、本発明は、水平方向及び垂直方向の各ライン内にR,G,B画素が周期的に配置されるカラー撮像素子を前提とするが、斜め(NE,NW)方向にもR,G,B画素が周期的に配置されるカラー撮像素子を使用する場合には、上記式(1)~(6)等により他の色の画素値を補間算出する必要はない。
図8は本発明に適用されるカラー撮像素子の第2の実施形態を示す図であり、特にカラー撮像素子に設けられているカラーフィルタのカラーフィルタ配列に関して示している。
図9は本発明に適用されるカラー撮像素子の第3の実施形態を示す図であり、特にカラー撮像素子に設けられているカラーフィルタのカラーフィルタ配列に関して示している。
図10は本発明に適用されるカラー撮像素子の第4の実施形態を示す図であり、特にカラー撮像素子に設けられているカラーフィルタのカラーフィルタ配列に関して示している。
図11は本発明に適用されるカラー撮像素子の第5の実施形態を示す図であり、特にカラー撮像素子に設けられているカラーフィルタのカラーフィルタ配列に関して示している。
図12及び図13はそれぞれ本発明に適用されるカラー撮像素子の第6及び第7の実施形態を示す図であり、特にカラー撮像素子に設けられているカラーフィルタのカラーフィルタ配列に関して示している。
上記実施形態では、RGBの3原色のカラーフィルタを有するカラー撮像素子を備えたカラー撮像装置について説明したが、本発明は、これに限らず、RGBの3原色+他の色(例えば、エメラルド(E))の4色のカラーフィルタを有するカラー撮像素子を備えたカラー撮像装置にも適用できる。
Claims (13)
- 水平方向及び垂直方向に配列された光電変換素子からなる複数の画素と、該複数の画素上に配設された所定のカラーフィルタ配列のカラーフィルタとが配設されてなる単板式のカラー撮像素子であって、前記カラーフィルタ配列は、輝度信号を得るために最も寄与する第1の色に対応する第1のフィルタと前記第1の色以外の2色以上の第2の色に対応する第2のフィルタとが周期的に配置され、かつ前記第1のフィルタは、水平、垂直、及び斜め(NE,NW)方向に2以上隣接する部分を含むカラー撮像素子と、
前記カラー撮像素子から前記カラーフィルタ配列に対応するモザイク画像を取得する画像取得部と、
前記モザイク画像から抽出される同時化処理の対象画素に対し、該対象画素の近傍の前記第1のフィルタに対応する画素であって、前記水平、垂直、及び斜め(NE,NW)方向に隣接する画素の画素値を取得し、それぞれ隣接する画素の画素値に基づいて輝度の相関方向が、前記水平、垂直、及び斜め(NE,NW)方向のうちのいずれの方向かを判別する方向判別部と、
前記モザイク画像から抽出される同時化処理の対象画素の画素位置における他の色の画素値を算出する同時化処理部であって、前記方向判別部により判別された相関方向に存在する他の色の画素の画素値を使用して算出する同時化処理部と、
前記モザイク画像から抽出する同時化処理の対象画素を前記同時化処理の対象画素単位毎に移動させながら前記方向判別部及び同時化処理部を繰り返し動作させる制御部と、
を備えたカラー撮像装置。 - 前記カラーフィルタ配列は、前記第1のフィルタと前記第2のフィルタとが配列された基本配列パターンを含み、該基本配列パターンが水平及び垂直方向に繰り返して配置され、
前記第1のフィルタ及び第2のフィルタは、それぞれ前記基本配列パターン内において水平及び垂直方向の各ライン内に1つ以上配置されている請求項1に記載のカラー撮像装置。 - 前記カラーフィルタ配列は、前記第1のフィルタが3×3画素群において中心と4隅に配置され、該3×3画素群が水平方向及び垂直方向に繰り返し配置されている請求項1に記載のカラー撮像装置。
- 前記カラーフィルタ配列は、前記第1のフィルタが3×3画素群において中心のフィルタを挟んで上下左右に配置され、該3×3画素群が水平方向及び垂直方向に繰り返し配置されている請求項1に記載のカラー撮像装置。
- 前記方向判別部は、前記水平、垂直、及び斜め(NE,NW)方向の各方向別に、隣接する画素の画素値の差分絶対値を算出し、各方向別の差分絶対値のうち最小となる方向を相関方向として判別する請求項1から4のいずれか1項に記載のカラー撮像装置。
- 前記方向判別部は、前記水平、垂直、及び斜め(NE,NW)方向の各方向別に、隣接する画素の画素値の比を算出し、各方向別の比のうち最も1に近い方向を相関方向として判別する請求項1から4のいずれか1項に記載のカラー撮像装置。
- 前記同時化処理部は、前記方向判別部により判別された相関方向に存在する他の色の画素の画素値を、そのまま前記対象画素の画素位置における他の色の画素値とし、又は前記方向判別部により判別された相関方向に存在する複数の他の色の画素の画素値を補間した値を、前記対象画素の画素位置における他の色の画素値とする請求項1から6のいずれか1項に記載のカラー撮像装置。
- 前記同時化処理部は、前記方向判別部により判別された相関方向に前記他の色の画素が存在しない場合には、前記対象画素の近傍の前記他の色の画素の画素位置における色差又は色比により、前記対象画素の画素値を補間して前記他の色の画素値を算出する請求項1から7のいずれか1項に記載のカラー撮像装置。
- 前記カラーフィルタは、赤(R)、緑(G)、青(B)の色に対応するRフィルタ、Gフィルタ及びBフィルタを有し、
前記同時化処理部は、
前記同時化処理の対象画素がG画素、その画素値がGであり、前記方向判別部により判別された相関方向にR、又はB画素が存在しない場合、前記G画素の近傍のR、B画素の画素値をR,B、これらの画素の画素位置におけるGの画素値をGR、GBとすると、前記対象画素の位置におけるR、B画素の画素値RG,BGを、次式、
RG=G+(R-GR),BG=G+(B-GB)
により算出し、前記同時化処理の対象画素がR画素、その画素値がRであり、前記方向判別部により判別された相関方向にG、又はB画素が存在しない場合、前記R画素の近傍のG、B画素の画素値をG,B、これらの画素の画素位置におけるRの画素値をRG、RBとすると、前記対象画素の位置におけるG、B画素の画素値GR,BRを、次式、
GR=R+(G-RG),BR=R+(B-RB)
により算出し、前記同時化処理の対象画素がB画素、その画素値がBであり、前記方向判別部により判別された相関方向にG、又はR画素が存在しない場合、前記B画素の近傍のG、R画素の画素値をG,R、これらの画素の画素位置におけるBの画素値をBG、BRとすると、前記対象画素の位置におけるG、R画素の画素値GB,RBを、次式、
GB=B+(G-BG),RB=B+(R-BR)
により算出する請求項8に記載のカラー撮像装置。 - 前記カラーフィルタは、赤(R)、緑(G)、青(B)の色に対応するRフィルタ、Gフィルタ及びBフィルタを有し、
前記同時化処理部は、
前記同時化処理の対象画素がG画素、その画素値がGであり、前記方向判別部により判別された相関方向にR、又はB画素が存在しない場合、前記G画素の近傍のR、B画素の画素値をR,B、これらの画素の画素位置におけるGの画素値をGR、GBとすると、前記対象画素の位置におけるR、B画素の画素値RG,BGを、次式、
RG=G×(R/GR),BG=G×(B/GB)
により算出し、前記同時化処理の対象画素がR画素、その画素値がRであり、前記方向判別部により判別された相関方向にG、又はB画素が存在しない場合、前記R画素の近傍のG、B画素の画素値をG,B、これらの画素の画素位置におけるRの画素値をRG、RBとすると、前記対象画素の位置におけるG、B画素の画素値GR,BRを、次式、
GR=R×(G/RG),BR=R×(B/RB)
により算出し、前記同時化処理の対象画素がB画素、その画素値がBであり、前記方向判別部により判別された相関方向にG、又はR画素が存在しない場合、前記B画素の近傍のG、R画素の画素値をG,R、これらの画素の画素位置におけるBの画素値をBG、BRとすると、前記対象画素の位置におけるG、R画素の画素値GB,RBを、次式、
GB=B×(G/BG),RB=B×(R/BR)
により算出する請求項8に記載のカラー撮像装置。 - 前記方向判別部は、前記水平、垂直、及び斜め(NE,NW)方向に隣接する画素の画素値の差分値がそれぞれ等しい場合には相関方向がないと判別し、
前記同時化処理部は、前記方向判別部により相関方向がないと判別されると、同時化処理の対象画素の画素位置における他の色の画素値を、該画素位置の近傍に存在する他の色の画素の画素値を使用して算出する請求項1から10のいずれか1項に記載のカラー撮像装置。 - 前記カラーフィルタは、赤(R)、緑(G)、青(B)の色に対応するRフィルタ、Gフィルタ及びBフィルタを有し、
前記フィルタ配列は、3×3画素に対応する第1の配列であって、中心と4隅にGフィルタが配置され、中心のGフィルタを挟んで上下にBフィルタが配置され、左右にRフィルタが配列された第1の配列と、3×3画素に対応する第2の配列であって、中心と4隅にGフィルタが配置され、中心のGフィルタを挟んで上下にRフィルタが配置され、左右にBフィルタが配列された第2の配列とが、交互に水平方向及び垂直方向に配列されて構成されている請求項1から11のいずれか1項に記載のカラー撮像装置。 - 前記カラーフィルタは、赤(R)、緑(G)、青(B)の色に対応するRフィルタ、Gフィルタ及びBフィルタを有し、
前記フィルタ配列は、3×3画素に対応する第1の配列であって、中心にRフィルタが配置され、4隅にBフィルタが配置され、中心のRフィルタを挟んで上下左右にGフィルタが配置された第1の配列と、3×3画素に対応する第2の配列であって、中心にBフィルタが配置され、4隅にRフィルタが配置され、中心のBフィルタを挟んで上下左右にGフィルタが配置された第2の配列とが、交互に水平方向及び垂直方向に配列されて構成されている請求項1から11のいずれか1項に記載のカラー撮像装置。
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012524427A JP5054858B1 (ja) | 2011-02-28 | 2011-07-29 | カラー撮像装置 |
BR112012027309A BR112012027309A2 (pt) | 2011-02-28 | 2011-07-29 | aparelho de geração de imagens coloridas |
CN201180022147.5A CN102959958B (zh) | 2011-02-28 | 2011-07-29 | 彩色成像设备 |
RU2013139022/07A RU2556022C2 (ru) | 2011-02-28 | 2011-07-29 | Устройство формирования цветного изображения |
EP11859950.5A EP2683167B1 (en) | 2011-02-28 | 2011-07-29 | Color imaging device |
US13/560,458 US8878970B2 (en) | 2011-02-28 | 2012-07-27 | Color imaging apparatus having color imaging element |
US14/501,949 US9288454B2 (en) | 2011-02-28 | 2014-09-30 | Color imaging apparatus having color imaging element |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011-042831 | 2011-02-28 | ||
JP2011042831 | 2011-02-28 | ||
JP2011-162413 | 2011-07-25 | ||
JP2011162413 | 2011-07-25 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/560,458 Continuation US8878970B2 (en) | 2011-02-28 | 2012-07-27 | Color imaging apparatus having color imaging element |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012117584A1 true WO2012117584A1 (ja) | 2012-09-07 |
Family
ID=46757538
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/067420 WO2012117584A1 (ja) | 2011-02-28 | 2011-07-29 | カラー撮像装置 |
Country Status (7)
Country | Link |
---|---|
US (2) | US8878970B2 (ja) |
EP (1) | EP2683167B1 (ja) |
JP (2) | JP5054858B1 (ja) |
CN (1) | CN102959958B (ja) |
BR (1) | BR112012027309A2 (ja) |
RU (1) | RU2556022C2 (ja) |
WO (1) | WO2012117584A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102938843A (zh) * | 2012-11-22 | 2013-02-20 | 华为技术有限公司 | 图像处理方法、装置以及成像设备 |
WO2014188950A1 (ja) * | 2013-05-23 | 2014-11-27 | 富士フイルム株式会社 | 画素補間装置およびその動作制御方法 |
CN104969545A (zh) * | 2013-02-05 | 2015-10-07 | 富士胶片株式会社 | 图像处理装置、摄像装置、图像处理方法以及程序 |
Families Citing this family (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8488031B2 (en) * | 2011-01-14 | 2013-07-16 | DigitalOptics Corporation Europe Limited | Chromatic noise reduction method and apparatus |
CN102884796B (zh) | 2011-02-21 | 2014-07-09 | 富士胶片株式会社 | 彩色成像元件 |
WO2012117584A1 (ja) | 2011-02-28 | 2012-09-07 | 富士フイルム株式会社 | カラー撮像装置 |
CN102870405B (zh) | 2011-03-09 | 2015-06-17 | 富士胶片株式会社 | 彩色成像元件 |
WO2012124184A1 (ja) * | 2011-03-11 | 2012-09-20 | 富士フイルム株式会社 | 撮像装置およびその動作制御方法ならびに撮像システム |
CN104335571B (zh) * | 2012-06-08 | 2016-05-18 | 富士胶片株式会社 | 图像处理装置、摄像装置、计算机及图像处理方法 |
WO2014007279A1 (ja) * | 2012-07-06 | 2014-01-09 | 富士フイルム株式会社 | カラー撮像素子および撮像装置 |
CN104412579B (zh) * | 2012-07-06 | 2016-08-24 | 富士胶片株式会社 | 彩色摄像元件以及摄像装置 |
JP5981824B2 (ja) * | 2012-09-28 | 2016-08-31 | 株式会社メガチップス | 画素補間処理装置、撮像装置、プログラムおよび集積回路 |
JP5777825B2 (ja) * | 2012-12-05 | 2015-09-09 | 富士フイルム株式会社 | 撮像装置、異常斜め入射光検出方法及びプログラム、並びに記録媒体 |
JP5749409B2 (ja) * | 2012-12-07 | 2015-07-15 | 富士フイルム株式会社 | 撮像装置、画像処理方法及びプログラム |
US9172743B2 (en) * | 2012-12-31 | 2015-10-27 | Futurewei Technologies, Inc. | Scalable storage systems with longest prefix matching switches |
JP2016510108A (ja) | 2013-02-13 | 2016-04-04 | 3シェイプ アー/エス | 色を記録する焦点走査装置 |
CN103179407B (zh) * | 2013-03-11 | 2015-03-11 | 浙江大学 | 应用于去马赛克算法方向插值的实现装置及方法 |
US9042643B2 (en) * | 2013-06-20 | 2015-05-26 | Himax Imaging Limited | Method for demosaicking |
JP6276580B2 (ja) * | 2013-12-20 | 2018-02-07 | 株式会社メガチップス | 画素補間処理装置、撮像装置、プログラムおよび集積回路 |
JP2015162706A (ja) | 2014-02-26 | 2015-09-07 | ソニー株式会社 | 画像処理装置および方法、並びにプログラム |
KR102134636B1 (ko) | 2014-07-14 | 2020-07-16 | 삼성전자주식회사 | 이미지 센서의 단위 픽셀 및 이를 포함하는 이미지 센서 |
KR102282457B1 (ko) * | 2014-12-05 | 2021-07-28 | 한화테크윈 주식회사 | 컬러 모아레 저감 방법, 컬러 모아레 저감 장치 및 영상 처리 장치 |
TWI585726B (zh) * | 2015-03-25 | 2017-06-01 | 鴻海精密工業股份有限公司 | 畫素結構 |
US10412356B2 (en) * | 2015-03-26 | 2019-09-10 | Sony Corporation | Image processing apparatus and image processing method |
CN106454286A (zh) * | 2016-09-29 | 2017-02-22 | 杭州雄迈集成电路技术有限公司 | 一种g模式色彩滤波阵列 |
CN106507066B (zh) * | 2016-11-22 | 2018-04-17 | 深圳开阳电子股份有限公司 | 抑制图像伪彩色的方法、装置 |
KR20210130972A (ko) * | 2020-04-23 | 2021-11-02 | 삼성전자주식회사 | 전자 장치의 컬러 필터, 및 그 전자 장치 |
FR3120989B1 (fr) * | 2021-03-16 | 2024-03-08 | Commissariat Energie Atomique | Motif couleur et proche-infrarouge pour capteur d’images |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0823543A (ja) | 1994-07-07 | 1996-01-23 | Canon Inc | 撮像装置 |
JPH10243407A (ja) * | 1997-02-27 | 1998-09-11 | Olympus Optical Co Ltd | 画像信号処理装置及び画像入力処理装置 |
JP2000308080A (ja) | 1999-04-15 | 2000-11-02 | Olympus Optical Co Ltd | カラー撮像素子及びカラー撮像装置 |
JP2005136766A (ja) | 2003-10-31 | 2005-05-26 | Sony Corp | 画像処理装置および画像処理方法 |
JP2006024999A (ja) * | 2004-07-06 | 2006-01-26 | Megachips Lsi Solutions Inc | 画素補間方法 |
JP2007037104A (ja) * | 2005-06-21 | 2007-02-08 | Sony Corp | 画像処理装置及び画像処理方法、撮像装置、並びにコンピュータ・プログラム |
JP2010104019A (ja) | 2009-12-17 | 2010-05-06 | Mega Chips Corp | 画素補間方法 |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS60263592A (ja) * | 1984-06-11 | 1985-12-27 | Toshiba Corp | 固体撮像装置 |
JP2739586B2 (ja) | 1989-02-10 | 1998-04-15 | 富士写真フイルム株式会社 | カラー固体撮像デバイス |
EP0522143A1 (en) * | 1991-01-25 | 1993-01-13 | Eastman Kodak Company | A solid state color image sensor using a field-staggered color filter pattern |
JP3268891B2 (ja) * | 1992-08-14 | 2002-03-25 | オリンパス光学工業株式会社 | 内視鏡撮像装置 |
JPH0823542A (ja) | 1994-07-11 | 1996-01-23 | Canon Inc | 撮像装置 |
EP0930789B1 (en) | 1998-01-20 | 2005-03-23 | Hewlett-Packard Company, A Delaware Corporation | Colour image pickup device |
AU4897499A (en) * | 1998-09-15 | 2000-04-03 | Phase One A/S | A method and a system for processing images |
US6396873B1 (en) | 1999-02-25 | 2002-05-28 | Envision Advanced Medical Systems | Optical device |
JP4487351B2 (ja) * | 1999-07-15 | 2010-06-23 | ソニー株式会社 | 固体撮像素子およびその駆動方法並びにカメラシステム |
JP4043197B2 (ja) * | 2000-03-24 | 2008-02-06 | 三洋電機株式会社 | 単板式カラーカメラの色分離回路 |
EP1148735A1 (en) * | 2000-04-20 | 2001-10-24 | Koninklijke Philips Electronics N.V. | Camera with color filter |
US7146042B2 (en) * | 2000-09-07 | 2006-12-05 | Nikon Corporation | Image processing apparatus and colorimetric system conversion method |
EP2768223B8 (en) | 2001-01-09 | 2016-11-09 | Sony Semiconductor Solutions Corporation | Image pick up device |
WO2002093939A1 (fr) * | 2001-05-15 | 2002-11-21 | Matsushita Electric Industrial Co., Ltd. | Dispositif d'imagerie et procede de traitement de signaux associe |
US7027091B1 (en) | 2001-09-17 | 2006-04-11 | Pixim, Inc. | Detection of color filter array alignment in image sensors |
US6970597B1 (en) | 2001-12-05 | 2005-11-29 | Pixim, Inc. | Method of defining coefficients for use in interpolating pixel values |
JP4451044B2 (ja) * | 2002-02-21 | 2010-04-14 | 株式会社メガチップス | 混成画素補間装置および混成画素補間方法 |
JP4019417B2 (ja) | 2003-01-14 | 2007-12-12 | ソニー株式会社 | 画像処理装置および方法、記録媒体、並びにプログラム |
JP2004266369A (ja) | 2003-02-21 | 2004-09-24 | Sony Corp | 固体撮像装置およびその駆動方法 |
EP1650979A1 (en) | 2004-10-21 | 2006-04-26 | STMicroelectronics S.r.l. | Method and system for demosaicing artifact removal |
JP4253634B2 (ja) | 2004-11-30 | 2009-04-15 | 富士フイルム株式会社 | デジタルカメラ |
JP4840740B2 (ja) | 2004-12-01 | 2011-12-21 | 株式会社メガチップス | 画素補間方法および画像判定方法 |
US8139130B2 (en) * | 2005-07-28 | 2012-03-20 | Omnivision Technologies, Inc. | Image sensor with improved light sensitivity |
US7821553B2 (en) | 2005-12-30 | 2010-10-26 | International Business Machines Corporation | Pixel array, imaging sensor including the pixel array and digital camera including the imaging sensor |
US7755682B2 (en) | 2006-03-29 | 2010-07-13 | Ite Tech. Inc. | Color interpolation method for Bayer filter array images |
JP4662883B2 (ja) | 2006-05-15 | 2011-03-30 | 富士フイルム株式会社 | 二次元カラー固体撮像素子 |
US7769230B2 (en) | 2006-11-30 | 2010-08-03 | Eastman Kodak Company | Producing low resolution images |
US7701496B2 (en) | 2006-12-22 | 2010-04-20 | Xerox Corporation | Color filter pattern for color filter arrays including a demosaicking algorithm |
US8199812B2 (en) | 2007-01-09 | 2012-06-12 | Qualcomm Incorporated | Adaptive upsampling for scalable video coding |
JP5082528B2 (ja) | 2007-03-23 | 2012-11-28 | ソニー株式会社 | 固体撮像装置及び撮像装置 |
JP4946581B2 (ja) | 2007-04-05 | 2012-06-06 | ソニー株式会社 | 画像処理装置 |
US7825965B2 (en) * | 2007-09-07 | 2010-11-02 | Seiko Epson Corporation | Method and apparatus for interpolating missing colors in a color filter array |
US7859580B2 (en) | 2007-10-16 | 2010-12-28 | Aptina Imaging Corporation | Method and apparatus for anisotropic demosaicing of image data |
RU2383967C2 (ru) | 2008-04-10 | 2010-03-10 | Общество с ограниченной ответственностью ООО "Юник Ай Сиз" | Фотоприемная матрица цветного изображения |
EP4336447A1 (en) | 2008-05-20 | 2024-03-13 | FotoNation Limited | Capturing and processing of images using monolithic camera array with heterogeneous imagers |
US8004588B2 (en) | 2008-08-27 | 2011-08-23 | Seiko Epson Corporation | Image processing procedure for receiving mosaic image data and calculating vertical and horizontal-direction color difference components for each pixel |
US8164042B2 (en) | 2008-11-06 | 2012-04-24 | Visera Technologies Company Limited | Color filter arrays and image sensors using the same |
JP5149143B2 (ja) | 2008-12-24 | 2013-02-20 | シャープ株式会社 | 固体撮像素子およびその製造方法、電子情報機器 |
JP5574615B2 (ja) | 2009-04-20 | 2014-08-20 | キヤノン株式会社 | 画像処理装置、その制御方法、及びプログラム |
US8203633B2 (en) | 2009-05-27 | 2012-06-19 | Omnivision Technologies, Inc. | Four-channel color filter array pattern |
JP5471117B2 (ja) * | 2009-07-24 | 2014-04-16 | ソニー株式会社 | 固体撮像装置とその製造方法並びにカメラ |
JP5120441B2 (ja) | 2009-11-26 | 2013-01-16 | 株式会社ニコン | 画像処理装置 |
CN102884796B (zh) | 2011-02-21 | 2014-07-09 | 富士胶片株式会社 | 彩色成像元件 |
WO2012117584A1 (ja) * | 2011-02-28 | 2012-09-07 | 富士フイルム株式会社 | カラー撮像装置 |
-
2011
- 2011-07-29 WO PCT/JP2011/067420 patent/WO2012117584A1/ja active Application Filing
- 2011-07-29 JP JP2012524427A patent/JP5054858B1/ja active Active
- 2011-07-29 CN CN201180022147.5A patent/CN102959958B/zh active Active
- 2011-07-29 BR BR112012027309A patent/BR112012027309A2/pt not_active IP Right Cessation
- 2011-07-29 EP EP11859950.5A patent/EP2683167B1/en active Active
- 2011-07-29 RU RU2013139022/07A patent/RU2556022C2/ru active
-
2012
- 2012-07-26 JP JP2012165734A patent/JP5872408B2/ja active Active
- 2012-07-27 US US13/560,458 patent/US8878970B2/en active Active
-
2014
- 2014-09-30 US US14/501,949 patent/US9288454B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0823543A (ja) | 1994-07-07 | 1996-01-23 | Canon Inc | 撮像装置 |
JPH10243407A (ja) * | 1997-02-27 | 1998-09-11 | Olympus Optical Co Ltd | 画像信号処理装置及び画像入力処理装置 |
JP2000308080A (ja) | 1999-04-15 | 2000-11-02 | Olympus Optical Co Ltd | カラー撮像素子及びカラー撮像装置 |
JP2005136766A (ja) | 2003-10-31 | 2005-05-26 | Sony Corp | 画像処理装置および画像処理方法 |
JP2006024999A (ja) * | 2004-07-06 | 2006-01-26 | Megachips Lsi Solutions Inc | 画素補間方法 |
JP2007037104A (ja) * | 2005-06-21 | 2007-02-08 | Sony Corp | 画像処理装置及び画像処理方法、撮像装置、並びにコンピュータ・プログラム |
JP2010104019A (ja) | 2009-12-17 | 2010-05-06 | Mega Chips Corp | 画素補間方法 |
Non-Patent Citations (1)
Title |
---|
See also references of EP2683167A4 |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102938843A (zh) * | 2012-11-22 | 2013-02-20 | 华为技术有限公司 | 图像处理方法、装置以及成像设备 |
US9374567B2 (en) | 2012-11-22 | 2016-06-21 | Huawei Technologies Co., Ltd. | Image processing method and apparatus, and imaging device |
CN104969545A (zh) * | 2013-02-05 | 2015-10-07 | 富士胶片株式会社 | 图像处理装置、摄像装置、图像处理方法以及程序 |
WO2014188950A1 (ja) * | 2013-05-23 | 2014-11-27 | 富士フイルム株式会社 | 画素補間装置およびその動作制御方法 |
JP5877931B2 (ja) * | 2013-05-23 | 2016-03-08 | 富士フイルム株式会社 | 画素補間装置およびその動作制御方法 |
US9727948B2 (en) | 2013-05-23 | 2017-08-08 | Fujifilm Corporation | Pixel interpolation device and operation control method |
Also Published As
Publication number | Publication date |
---|---|
JPWO2012117584A1 (ja) | 2014-07-07 |
JP5872408B2 (ja) | 2016-03-01 |
JP5054858B1 (ja) | 2012-10-24 |
EP2683167B1 (en) | 2018-05-02 |
RU2556022C2 (ru) | 2015-07-10 |
CN102959958A (zh) | 2013-03-06 |
RU2013139022A (ru) | 2015-04-10 |
BR112012027309A2 (pt) | 2016-08-02 |
US20150029367A1 (en) | 2015-01-29 |
US8878970B2 (en) | 2014-11-04 |
EP2683167A1 (en) | 2014-01-08 |
EP2683167A4 (en) | 2014-11-26 |
US20120293696A1 (en) | 2012-11-22 |
JP2013048410A (ja) | 2013-03-07 |
CN102959958B (zh) | 2015-05-27 |
US9288454B2 (en) | 2016-03-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5872408B2 (ja) | カラー撮像装置及び画像処理方法 | |
JP5054857B1 (ja) | カラー撮像装置 | |
US8922683B2 (en) | Color imaging element and imaging apparatus | |
JP5095040B1 (ja) | カラー撮像素子 | |
WO2012114558A1 (ja) | カラー撮像素子 | |
US9159758B2 (en) | Color imaging element and imaging device | |
US9184196B2 (en) | Color imaging element and imaging device | |
US9324749B2 (en) | Color imaging element and imaging device | |
US9219894B2 (en) | Color imaging element and imaging device | |
US9185375B2 (en) | Color imaging element and imaging device | |
US9143747B2 (en) | Color imaging element and imaging device | |
US8711257B2 (en) | Color imaging device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180022147.5 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012524427 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 8954/CHENP/2012 Country of ref document: IN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11859950 Country of ref document: EP Kind code of ref document: A1 |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112012027309 Country of ref document: BR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011859950 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2013139022 Country of ref document: RU Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 112012027309 Country of ref document: BR Kind code of ref document: A2 Effective date: 20121024 |