US20100165157A1 - Apparatus for demosaicing colors and method thereof - Google Patents
Apparatus for demosaicing colors and method thereof Download PDFInfo
- Publication number
- US20100165157A1 US20100165157A1 US12/648,929 US64892909A US2010165157A1 US 20100165157 A1 US20100165157 A1 US 20100165157A1 US 64892909 A US64892909 A US 64892909A US 2010165157 A1 US2010165157 A1 US 2010165157A1
- Authority
- US
- United States
- Prior art keywords
- edge
- information
- data
- color
- missing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
- H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/843—Demosaicing, e.g. interpolating colour pixel values
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
Definitions
- An image capture device capable of capturing a digital image such as a digital camera, a cellular phone, and the like includes an image sensor.
- the image sensor includes a 2-dimensional array sensor. Each sensor is located at ‘pixel position’. Each sensor detects intensity of one color of light. In general, there may be a green sensor, a red sensor and a blue sensor.
- red and blue may be sub-sampled for sampling of green by an image sensor.
- An uppermost row of a sensor includes alternating green and red pixels.
- a row next to the uppermost row includes green and blue pixels.
- the order of the sensors is generated row by row vertically below in an image sensor.
- the number of green pixels G is greater than that of red or blue pixels R or B.
- the number of the green pixels G is equal to a sum of the numbers of red and blue pixels R and B.
- the number of the green pixels G is relatively greater than that of the red or blue pixels R or B. This is because a green color is closest to a luminance component and because a human eye reacts to the green color most sensitively.
- the image sensor shown in FIG. 1 has more green sensors than red or blue sensors. This can be called ‘red and blue sub-sampling’.
- One color sample value is taken from each pixel position on an image sensor only. For instance, each color sample value may have an 8-bit value for example.
- an image has one pixel constructed with red, green and blue. Yet, referring to FIG. 1 , the Bayer pattern has one pixel constructed with one color such as red, green and blue. In order to reconstruct a full image, color information missed from each pixel should be interpolated using neighbor pixels.
- a Hamilton edge classifier may be used for the edge sensing, which is represented as Formula 1.
- An image sensor includes a 2-dimensional array of color sensors for rows and columns. Each of the color sensors is located at one pixel position and then detects one color. In this example, each of the color sensors generates an 8-bit color subpixel value.
- the indication ‘Gij’ is used to indicate a color subpixel value.
- a front capital letter indicates a detected color. If this letter is ‘R’, a detected color is red. If this letter is ‘G’, a detected color is green. If this letter is ‘B’, a detected color is blue.
- a notation by two numerals ‘ij’ follows the front letter. In this notation, a first numeral ‘i’ indicates a row of a subpixel value and a second numeral T indicates a column off the subpixel value.
- a missed green value can be found according to Formula 2 by finding horizontal and vertical gradients and then determining whether an edge component in a horizontal direction or an edge component in a vertical direction is dominant.
- Embodiments relate to an image sensor, and more particularly, to an apparatus for demosaicing colors in an image sensor and method thereof.
- Embodiments relate to an apparatus for demosaicing an image and method thereof, by which a high-quality image can be provided.
- Embodiments relate to a demosaicing apparatus and method thereof, by which information on a neighbor pixel can be acquired by performing edge sensing in a high-frequency region accurately.
- Embodiments also relate to a demosaicing apparatus and method, by which a disadvantage caused by edge-sensing error occurring in a region where a successive 1-line narrow edge exists.
- Embodiments relate to an apparatus for demosaicing colors which may include: include an image sensor containing information on a color signal detected from each pixel, a first line memory receiving and storing output data from the image sensor, a missing green information extractor extracting missing green pixel information from the data of the first line memory, a delayer receiving the data of the first line memory, the delayer delaying the received data for a prescribed duration, the delayer outputting the delayed data, a second line memory temporarily storing the data outputted from the missing green information extractor and the data provided via the delayer, and a missing red/blue information outputter extracting missing red/blue information from the data of the second line memory.
- the missing green information extractor may include a gradient calculating unit finding a gradient in a current pixel, a line edge determining unit comparing a minimum/maximum difference value of green pixel values within a kernel to a threshold, the line edge determining unit determining a presence or non-presence of a successive 1-line edge region according to a result of the comparison, a directionality determining unit determining directionality of an edge in a current center pixel through edge sensing for 8 neighbor pixels centering on a center pixel, an edge classifying unit discriminating one value selected from the group consisting of a plain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region based on values provided by the gradient calculating unit and the line edge determining unit, and an edge refining unit modifying a neighbor edge direction using values provided by the edge classifying unit and the directionality determining unit and a threshold.
- Embodiments relate to a method of demosaicing colors which may include a first color information storing step of receiving and storing information on a color signal detected in each cell from an image sensor, a missing green information extracting step of extracting missing green pixel information in the first color information storing step, a delaying step of delaying the first color information data for a prescribed duration and then outputting the delayed data, a second color information storing step of storing data extracted through the missing green information extracting step and data provided via the delaying step, and a missing red/blue information outputting step of extracting missing red/blue information from the data stored in the second color information storing step.
- the missing green information extracting step may include the steps of finding a gradient in a current pixel, comparing a minimum/maximum difference value of green pixel values within a kernel to a threshold and then determining a presence or non-presence of a successive 1-line edge region according to a result of the comparison, determining directionality of an edge in the current center pixel through edge sensing for 8 neighbor pixels centering on a center pixel, an edge classifying step of discriminating one value selected from the group consisting of a plain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region based on the gradient value and a presence or non-presence of a 1-line edge region, and an edge refining step of modifying a neighbor edge direction using a threshold based on the classified edge and directionality of the edge.
- FIG. 1 is a diagram for an example of Bayer pattern.
- Example FIG. 2 is a block diagram of a color demosaicing apparatus according to embodiments.
- Example FIG. 3 is a detailed block diagram of a color demosaicing apparatus according to embodiments.
- Example FIG. 4 is a exemplary diagram of green component information in kernel having a successive narrow edge region.
- Example FIG. 5 is an exemplary diagram of an edge directionality map.
- Example FIG. 6 is a flowchart for a progress of a color demosaicing method according to embodiments.
- Example FIG. 7 is a flowchart for a detailed progress of a color demosaicing method according to embodiments.
- a color demosaicing apparatus may include an image sensor 10 containing information on a color signal detected from each pixel, a first line memory 20 receiving and storing output data from the image sensor 10 , a missing green information extractor 30 extracting missing green pixel information by supplementing a fault of incorrect edge sensing in a region, in which 1-line narrow edge successive in data of the first line memory 20 exists, a delayer 40 receiving the data of the first line memory 20 , the delayer 40 delaying the received data for a prescribed duration, the delayer 40 outputting the delayed data, a second line memory 50 temporarily storing the data outputted from the missing green information extractor 30 and the data provided via the delayer 40 , and a missing red/blue information outputter 60 extracting missing red/blue information from the data of the second line memory 50 .
- An edge classifier exists in the missing green pixel information extractor 30 shown in FIG. 2 .
- the edge classifier extracts green, as shown in FIG. 2 , and then extracts red and blue using corresponding values.
- Example FIG. 3 is a detailed block diagram of a color demosaicing apparatus according to embodiments.
- the missing green information extractor may include a gradient calculating unit 31 finding a gradient in a current pixel, a line edge determining unit 32 comparing a minimum/maximum difference value between green pixel values within a kernel to a threshold, the line edge determining unit 32 determining a presence or non-presence of a successive 1-line edge region according to a result of the comparison, a directionality determining unit 33 determining directionality of an edge in a current center pixel through edge sensing for 8 neighbor pixels centering on a center pixel, an edge classifying unit 34 discriminating one of a plain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region based on values provided by the gradient calculating unit 31 and the line edge determining unit 32 , and an edge refining unit 35 modifying a neighbor edge direction using values provided by the edge classifying unit 34 and the direction
- the gradient calculating unit 31 may be a block for finding a gradient value in a current pixel.
- the gradient calculating unit 31 uses a calculation scheme resulting from modifying a gradient calculating scheme using a color difference proposed by ‘Color Demosaicing Using Variance Of Color Correlation’. Namely, in ‘Color Demosaicing Using Variance Of Color Correlation’ according to a related art, a variance of color differences of neighbor pixels within a kernel is found for more accurate edge sensing. Yet, since this method requires heavy operational load, Formula 3 and Formula 4 are applied.
- K U H [( img i,j ⁇ 1 +img i,j+1 )>>]+[( img i,j ⁇ 2 +2 *img i,j +img i,j+2 )>>2]
- Formula 4 indicates a color difference between neighbor pixels and is represented as follows.
- the line edge determining unit 32 is a block newly added to the related art edge classifier.
- a color difference (dH, dV) is represented as a very small value. So, it is difficult to discriminate whether this region is a region having an edge exist therein or a plain plane region with no edge.
- Example FIG. 4 is an exemplary diagram of green component information in kernel having a successive narrow edge region. Referring to FIG. 4 , even if a gradient for green is found in a vertical or horizontal direction, it is able to expect an almost same value.
- the edge classifier 34 of embodiments finds minimum and maximum values of green values within the kernel. If the difference value exceeds a specific threshold (narrowEdgeThr), it is determined as a region whether the 1-line edge exists.
- the directionality determining unit (local gradient) 33 is a block newly added to the related art edge classifier. Simple edge sensing is performed on 8 neighbor pixels with reference to a center pixel within the kernel. In this case, the corresponding edge classification uses Hamilton edge sensing scheme like that in Formula 1 and Formula 2.
- the edge classifier 34 preferentially detects directionality of an edge in a current center pixel using the inputted gradient values and the minimum/maximum difference of the green values within the kernel. Subsequently, a reference for performing refinement, one more time, on whether the currently found directionality of the edge is a correct value, is prepared.
- the edge classifier 34 compares the values provided by the gradient calculating unit 31 and the line edge determining unit 32 to the PlainThr, StrongThr, NarrowEdgeThr values provided by the first threshold providing unit 36 to discriminate into one of a lain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region.
- ⁇ H is equal to or smaller than a plain threshold plainThr
- ⁇ V is equal to or smaller than plainThr
- a difference diffMinMaxVal between minimum and maximum is equal to or smaller than a narrowedge threshold narrowEdgethr
- ⁇ H is equal to or smaller than a plain threshold plainThr
- ⁇ V is equal to or smaller than plainThr
- a difference diffMinMaxVal between minimum and maximum is greater than a narrowedge threshold narrowEdgethr
- ⁇ H is greater than a value resulting from adding a strong threshold to ⁇ V, it is determined as a vertical edge region.
- ⁇ V is greater than a value resulting from adding a strong threshold to ⁇ H, it is determined as a horizontal edge region.
- edge refining unit (edge refinement) 35 uses the gradient value found by the directionality determining unit (local gradient block) 33 as a reference.
- Example FIG. 5 shows an example of configuring an edge direction map of neighbor pixels using values outputted from the directionality determining unit (local gradient block) 33 .
- An edge direction of a current center pixel is determined as a horizontal direction, while neighbor pixels have edge direction mainly determined as a vertical direction.
- the edge directionality of the rest of pixels except the center sets the criterion for an edge direction by applying the following rules.
- edge directionality information of a center pixel is modified.
- edge RefinementThr a threshold provided by the second threshold providing unit 37
- edge directionality of a center pixel is modified into ‘vertical (V)’.
- Example FIG. 6 is a flowchart for a progress of a color demosaicing method according to embodiments. Referring to FIG. 6 , information on a color signal detected from each pixel is received from an image sensor and is then stored (1st color data storing) [S 10 ].
- mixing green information for extracting missing green pixel information is extracted (Missing green pixel estimation) [S 20 ].
- the stored color information data are delayed for prescribed duration and are then outputted (delaying) [S 30 ].
- the data extracted through the missing green information extracting process and the data provided through the delaying are stored (2nd color data storing) [S 40 ].
- missing red and blue information is extracted from the data after completion of the mixing green information extraction (Missing Red and Blue pixel estimation) [S 50 ].
- FIG. 7 is a flowchart for a detailed progress of a color demosaicing method according to embodiments. Particularly, FIG. 7 shows the missing green pixel estimation in detail. Referring to FIG. 7 , a gradient value in a current pixel is calculated (calculating gradient using color diff) [S 21 ].
- embodiments provide the following effects and/or advantages.
- embodiments are able to accurately find edge directionality of a current pixel in the Bayer pattern.
- embodiments are able to accurately detect a 1-line edge, thereby enhancing performance of an edge classifier.
- the accuracy of color demosaicing of in Bayer pattern is improved.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Color Television Image Signal Generators (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
Description
- The present application claims priority under 35 U.S.C. 119 to Korean Patent Application No. 10-2008-0136928 (filed on Dec. 30, 2009), which is hereby incorporated by reference in its entirety.
- An image capture device capable of capturing a digital image such as a digital camera, a cellular phone, and the like includes an image sensor. The image sensor includes a 2-dimensional array sensor. Each sensor is located at ‘pixel position’. Each sensor detects intensity of one color of light. In general, there may be a green sensor, a red sensor and a blue sensor.
- Various kinds of pixel array types adopted by CCD or CMOS image sensors exist. Yet, due to the facilitation of pixel design, excellent color reproducibility and process yield, most of the image sensors adopt the Bayer pattern shown in
FIG. 1 . According to a method of visualizing colors and images, red and blue may be sub-sampled for sampling of green by an image sensor. An uppermost row of a sensor includes alternating green and red pixels. A row next to the uppermost row includes green and blue pixels. The order of the sensors is generated row by row vertically below in an image sensor. - The, the number of green pixels G is greater than that of red or blue pixels R or B. The number of the green pixels G is equal to a sum of the numbers of red and blue pixels R and B. Thus, the number of the green pixels G is relatively greater than that of the red or blue pixels R or B. This is because a green color is closest to a luminance component and because a human eye reacts to the green color most sensitively.
- Therefore, the image sensor shown in
FIG. 1 has more green sensors than red or blue sensors. This can be called ‘red and blue sub-sampling’. One color sample value is taken from each pixel position on an image sensor only. For instance, each color sample value may have an 8-bit value for example. - Generally, an image has one pixel constructed with red, green and blue. Yet, referring to
FIG. 1 , the Bayer pattern has one pixel constructed with one color such as red, green and blue. In order to reconstruct a full image, color information missed from each pixel should be interpolated using neighbor pixels. - Many efforts are ongoing to be made to interpolate the missed color information accurately. Yet, if incorrect neighbor pixel information in a high-frequency area of an image is used, an artificial video of an image is displayed.
- To solve this problem, there are studied techniques described in Color Demosaicing Using Variance Of Color Correlation, Effective Color Interpolation in CCD Color Filter Arrays Using Signal Correlation, etc. Regarding these techniques, edge sensing reduces incorrect pixel information but enables more precise pixel information to be acquired. Therefore, interpolation (demosaicing) is performed by acquiring precise pixel information through edge sensing.
- A Hamilton edge classifier may be used for the edge sensing, which is represented as Formula 1. An image sensor includes a 2-dimensional array of color sensors for rows and columns. Each of the color sensors is located at one pixel position and then detects one color. In this example, each of the color sensors generates an 8-bit color subpixel value.
- The indication ‘Gij’ is used to indicate a color subpixel value. A front capital letter indicates a detected color. If this letter is ‘R’, a detected color is red. If this letter is ‘G’, a detected color is green. If this letter is ‘B’, a detected color is blue. A notation by two numerals ‘ij’ follows the front letter. In this notation, a first numeral ‘i’ indicates a row of a subpixel value and a second numeral T indicates a column off the subpixel value.
-
ΔH i,j =|G i,j−1 −G i,j+1|+|2R i,j −R i,j−2 −R i,j+2| -
ΔV i,j =|G i,j−1 −G i,j+1|+|2R i,j −R i−2,j −R i+2,j| [Formula 1] - In Formula 1, if a current position of a pixel for finding missed color information from an image is (i, j), a missed green value can be found according to Formula 2 by finding horizontal and vertical gradients and then determining whether an edge component in a horizontal direction or an edge component in a vertical direction is dominant.
-
- Even if Formula 1 and Formula 2 are used, if a successive 1-line narrow edge exists, Formula 1 is insufficient to perfectly discriminate accurate directionality of an edge. Hence, error of demosaicing occurs.
- For this, an edge-sensing scheme in a 9×9 kernel using color correlation is introduced in Color Demosaicing Using Variance Of Color Correlation. Although ‘Effective Color Interpolation in CCD Color Filter Arrays Using Signal Correlation’ introduces a scheme of supplementing the above-described Hamilton edge-classifier, if a successive 1-line narrow edge exists, there still exists an incorrect edge sensing.
- Embodiments relate to an image sensor, and more particularly, to an apparatus for demosaicing colors in an image sensor and method thereof. Embodiments relate to an apparatus for demosaicing an image and method thereof, by which a high-quality image can be provided. Embodiments relate to a demosaicing apparatus and method thereof, by which information on a neighbor pixel can be acquired by performing edge sensing in a high-frequency region accurately. Embodiments also relate to a demosaicing apparatus and method, by which a disadvantage caused by edge-sensing error occurring in a region where a successive 1-line narrow edge exists.
- Embodiments relate to an apparatus for demosaicing colors which may include: include an image sensor containing information on a color signal detected from each pixel, a first line memory receiving and storing output data from the image sensor, a missing green information extractor extracting missing green pixel information from the data of the first line memory, a delayer receiving the data of the first line memory, the delayer delaying the received data for a prescribed duration, the delayer outputting the delayed data, a second line memory temporarily storing the data outputted from the missing green information extractor and the data provided via the delayer, and a missing red/blue information outputter extracting missing red/blue information from the data of the second line memory.
- The missing green information extractor may include a gradient calculating unit finding a gradient in a current pixel, a line edge determining unit comparing a minimum/maximum difference value of green pixel values within a kernel to a threshold, the line edge determining unit determining a presence or non-presence of a successive 1-line edge region according to a result of the comparison, a directionality determining unit determining directionality of an edge in a current center pixel through edge sensing for 8 neighbor pixels centering on a center pixel, an edge classifying unit discriminating one value selected from the group consisting of a plain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region based on values provided by the gradient calculating unit and the line edge determining unit, and an edge refining unit modifying a neighbor edge direction using values provided by the edge classifying unit and the directionality determining unit and a threshold.
- Embodiments relate to a method of demosaicing colors which may include a first color information storing step of receiving and storing information on a color signal detected in each cell from an image sensor, a missing green information extracting step of extracting missing green pixel information in the first color information storing step, a delaying step of delaying the first color information data for a prescribed duration and then outputting the delayed data, a second color information storing step of storing data extracted through the missing green information extracting step and data provided via the delaying step, and a missing red/blue information outputting step of extracting missing red/blue information from the data stored in the second color information storing step.
- The missing green information extracting step may include the steps of finding a gradient in a current pixel, comparing a minimum/maximum difference value of green pixel values within a kernel to a threshold and then determining a presence or non-presence of a successive 1-line edge region according to a result of the comparison, determining directionality of an edge in the current center pixel through edge sensing for 8 neighbor pixels centering on a center pixel, an edge classifying step of discriminating one value selected from the group consisting of a plain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region based on the gradient value and a presence or non-presence of a 1-line edge region, and an edge refining step of modifying a neighbor edge direction using a threshold based on the classified edge and directionality of the edge.
-
FIG. 1 is a diagram for an example of Bayer pattern. - Example
FIG. 2 is a block diagram of a color demosaicing apparatus according to embodiments. - Example
FIG. 3 is a detailed block diagram of a color demosaicing apparatus according to embodiments. - Example
FIG. 4 is a exemplary diagram of green component information in kernel having a successive narrow edge region. - Example
FIG. 5 is an exemplary diagram of an edge directionality map. - Example
FIG. 6 is a flowchart for a progress of a color demosaicing method according to embodiments. - Example
FIG. 7 is a flowchart for a detailed progress of a color demosaicing method according to embodiments. - A color demosaicing apparatus according to embodiments, as shown in
FIG. 2 , may include animage sensor 10 containing information on a color signal detected from each pixel, afirst line memory 20 receiving and storing output data from theimage sensor 10, a missinggreen information extractor 30 extracting missing green pixel information by supplementing a fault of incorrect edge sensing in a region, in which 1-line narrow edge successive in data of thefirst line memory 20 exists, adelayer 40 receiving the data of thefirst line memory 20, thedelayer 40 delaying the received data for a prescribed duration, thedelayer 40 outputting the delayed data, asecond line memory 50 temporarily storing the data outputted from the missinggreen information extractor 30 and the data provided via thedelayer 40, and a missing red/blue information outputter 60 extracting missing red/blue information from the data of thesecond line memory 50. - An edge classifier exists in the missing green
pixel information extractor 30 shown inFIG. 2 . The edge classifier extracts green, as shown inFIG. 2 , and then extracts red and blue using corresponding values. - Example
FIG. 3 is a detailed block diagram of a color demosaicing apparatus according to embodiments. Referring toFIG. 3 , the missing green information extractor may include agradient calculating unit 31 finding a gradient in a current pixel, a lineedge determining unit 32 comparing a minimum/maximum difference value between green pixel values within a kernel to a threshold, the lineedge determining unit 32 determining a presence or non-presence of a successive 1-line edge region according to a result of the comparison, adirectionality determining unit 33 determining directionality of an edge in a current center pixel through edge sensing for 8 neighbor pixels centering on a center pixel, anedge classifying unit 34 discriminating one of a plain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region based on values provided by thegradient calculating unit 31 and the lineedge determining unit 32, and anedge refining unit 35 modifying a neighbor edge direction using values provided by theedge classifying unit 34 and thedirectionality determining unit 34 and a threshold. - The
gradient calculating unit 31 may be a block for finding a gradient value in a current pixel. Thegradient calculating unit 31 uses a calculation scheme resulting from modifying a gradient calculating scheme using a color difference proposed by ‘Color Demosaicing Using Variance Of Color Correlation’. Namely, in ‘Color Demosaicing Using Variance Of Color Correlation’ according to a related art, a variance of color differences of neighbor pixels within a kernel is found for more accurate edge sensing. Yet, since this method requires heavy operational load,Formula 3 andFormula 4 are applied. -
K U H=[(img i,j−1 +img i,j+1)>>]+[(img i,j−2+2*img i,j +img i,j+2)>>2] -
K U V=[(img i−1,j +img i+1,j)>>]+[(img i−2,j+2*img i,j +img i+2,j)>>2] [Formula 3] -
- , where horizontal and vertical weights are Ki,j h and, Ki,j V respectively.
-
Formula 4 indicates a color difference between neighbor pixels and is represented as follows. -
dH=|K i,j H −K i,j−2 H |+|K i,j H −K i,j+2 H| -
dV=|K i,j V −K i−2,j V |+|K i,j V −K i+2,j V| [Formula 4] - The line
edge determining unit 32 is a block newly added to the related art edge classifier. In a high frequency region where 1-line narrow edges appear successively, a color difference (dH, dV) is represented as a very small value. So, it is difficult to discriminate whether this region is a region having an edge exist therein or a plain plane region with no edge. - Example
FIG. 4 is an exemplary diagram of green component information in kernel having a successive narrow edge region. Referring toFIG. 4 , even if a gradient for green is found in a vertical or horizontal direction, it is able to expect an almost same value. - Yet, if minimum and maximum values of green values within the kernel are found and a difference between the minimum and maximum values is then found, the value difference is considerably big unlike the case of the plain plane. Accordingly, in order to discriminate the successive 1-line edge region and the plain plane region from each other, the
edge classifier 34 of embodiments finds minimum and maximum values of green values within the kernel. If the difference value exceeds a specific threshold (narrowEdgeThr), it is determined as a region whether the 1-line edge exists. - The directionality determining unit (local gradient) 33 is a block newly added to the related art edge classifier. Simple edge sensing is performed on 8 neighbor pixels with reference to a center pixel within the kernel. In this case, the corresponding edge classification uses Hamilton edge sensing scheme like that in
Formula 1 andFormula 2. - The
edge classifier 34 preferentially detects directionality of an edge in a current center pixel using the inputted gradient values and the minimum/maximum difference of the green values within the kernel. Subsequently, a reference for performing refinement, one more time, on whether the currently found directionality of the edge is a correct value, is prepared. - The
edge classifier 34 compares the values provided by thegradient calculating unit 31 and the lineedge determining unit 32 to the PlainThr, StrongThr, NarrowEdgeThr values provided by the firstthreshold providing unit 36 to discriminate into one of a lain plane, a successive 1-line edge region, a vertical edge region, a horizontal edge region and a texture edge region. - According to
Formula 2 of the related art, by comparing the vertical gradient value to the horizontal gradient value, an edge according to one of ΔH>ΔV, ΔH<ΔV and ΔH=ΔV is determined. Yet, embodiments make this determination in a following manner. - 1. If ΔH is equal to or smaller than a plain threshold plainThr, ΔV is equal to or smaller than plainThr and a difference diffMinMaxVal between minimum and maximum is equal to or smaller than a narrowedge threshold narrowEdgethr, it is determined as a plain plane.
- 2. If ΔH is equal to or smaller than a plain threshold plainThr, ΔV is equal to or smaller than plainThr and a difference diffMinMaxVal between minimum and maximum is greater than a narrowedge threshold narrowEdgethr, it is determined as a successive 1-line edge region.
- 3. If ΔH is greater than a value resulting from adding a strong threshold to ΔV, it is determined as a vertical edge region.
- 4. If ΔV is greater than a value resulting from adding a strong threshold to ΔH, it is determined as a horizontal edge region.
- 5. Otherwise, it is determined as a texture edge region.
- Meanwhile, the edge refining unit (edge refinement) 35 uses the gradient value found by the directionality determining unit (local gradient block) 33 as a reference.
- The corresponding example is shown in
FIG. 5 . ExampleFIG. 5 shows an example of configuring an edge direction map of neighbor pixels using values outputted from the directionality determining unit (local gradient block) 33. An edge direction of a current center pixel is determined as a horizontal direction, while neighbor pixels have edge direction mainly determined as a vertical direction. In this case, the edge directionality of the rest of pixels except the center sets the criterion for an edge direction by applying the following rules. -
edgeCriterion = 0; for indxR = −1 : 1 for indxC = −1 : 1, if (indxR != 0) & (indxC != 0) if edgeMap(i+indxR,j+indxC == V edgeCriterion += 2; elseif edgeMap(i+indxR,j+indxC == A_V edgeCriterion += 1; elseif edgeMap(i+indxR,j+indxC == A_H edgeCriterion −= 1; elseif edgeMap(i+indxR,j+indxC == H edgeCriterion −= 2; End End End End - Thus, if the edge criterion value is equal to or greater than a predetermined threshold, edge directionality information of a center pixel is modified. In the example of
FIG. 5 , if an edge criterion is 6 and a threshold (edge RefinementThr) provided by the secondthreshold providing unit 37 is 5, as the edge criterion exceeds the threshold, edge directionality of a center pixel is modified into ‘vertical (V)’. - Example
FIG. 6 is a flowchart for a progress of a color demosaicing method according to embodiments. Referring toFIG. 6 , information on a color signal detected from each pixel is received from an image sensor and is then stored (1st color data storing) [S10]. - By supplementing a fault of incorrect edge sensing on a region having a successive 1-line narrow edge exist therein, mixing green information for extracting missing green pixel information is extracted (Missing green pixel estimation) [S20]. The stored color information data are delayed for prescribed duration and are then outputted (delaying) [S30]. The data extracted through the missing green information extracting process and the data provided through the delaying are stored (2nd color data storing) [S40]. Finally, missing red and blue information is extracted from the data after completion of the mixing green information extraction (Missing Red and Blue pixel estimation) [S50].
- Example
FIG. 7 is a flowchart for a detailed progress of a color demosaicing method according to embodiments. Particularly,FIG. 7 shows the missing green pixel estimation in detail. Referring toFIG. 7 , a gradient value in a current pixel is calculated (calculating gradient using color diff) [S21]. - By comparing a minimum/maximum difference value of green pixel values within a kernel to a threshold, it is determined whether a successive 1-line edge region exists according to a result of the comparison (Diff=Gmax−Gmin) S22). Through edge sensing on 8 neighbor pixels with reference to a center pixel, directionality of an edge in a current center cell is determined (Local gradient at neighbor 8 pixels) [S23]. Based on the gradient value and a presence or non-presence of the 1-line edge region, it is discriminated into one of plain plane, successive 1-line edge region, vertical edge region, horizontal edge region and texture edge region (Edge−classifier) [S24]. Based on the classified edge and the directionality of the edge, a neighbor edge direction is modified using a threshold (Edge-sensing refinement) [S25].
- Accordingly, embodiments provide the following effects and/or advantages. First of all, embodiments are able to accurately find edge directionality of a current pixel in the Bayer pattern. Secondly, embodiments are able to accurately detect a 1-line edge, thereby enhancing performance of an edge classifier. Thirdly, the accuracy of color demosaicing of in Bayer pattern is improved.
- It will be obvious and apparent to those skilled in the art that various modifications and variations can be made in the embodiments disclosed. Thus, it is intended that the disclosed embodiments cover the obvious and apparent modifications and variations, provided that they are within the scope of the appended claims and their equivalents.
Claims (20)
dH=|K i,j H −K i,j−2 H |+|K i,j H −K i,j+2 H|
and
dV=|K i,j V −K i−2,j V |+|K i,j V −K i+2,j V|,
K U H=[(img i,j−1 +img i,j+1)>>]+[(img i,j−2+2*img i,j +img i,j+2)>>2]
and
K U V=[(img i−1,j +img i+1,j)>>]+[(img i−2,j+2*img i,j +img i+2,j)>>2],
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020080136928A KR101580168B1 (en) | 2008-12-30 | 2008-12-30 | a color demosaicing device |
KR10-2008-0136928 | 2008-12-30 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20100165157A1 true US20100165157A1 (en) | 2010-07-01 |
US8363132B2 US8363132B2 (en) | 2013-01-29 |
Family
ID=42284465
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/648,929 Active 2031-04-23 US8363132B2 (en) | 2008-12-30 | 2009-12-29 | Apparatus for demosaicing colors and method thereof |
Country Status (4)
Country | Link |
---|---|
US (1) | US8363132B2 (en) |
KR (1) | KR101580168B1 (en) |
CN (1) | CN101771883A (en) |
TW (1) | TW201026079A (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120257821A1 (en) * | 2009-10-20 | 2012-10-11 | Yasushi Saito | Image processing apparatus and image processing method, and program |
US20130050235A1 (en) * | 2011-08-22 | 2013-02-28 | Chih-Yuan Yang | Color Information Interpolation Method |
US9875678B2 (en) | 2014-06-10 | 2018-01-23 | Samsung Display Co., Ltd. | Image display method |
US20190020411A1 (en) * | 2017-07-13 | 2019-01-17 | Qualcomm Incorporated | Methods and apparatus for efficient visible light communication (vlc) with reduced data rate |
CN111062331A (en) * | 2019-12-18 | 2020-04-24 | 银河互联网电视有限公司 | Mosaic detection method and device for image, electronic equipment and storage medium |
CN114466170A (en) * | 2021-08-27 | 2022-05-10 | 锐芯微电子股份有限公司 | Image processing method and system |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5113929B1 (en) * | 2011-06-24 | 2013-01-09 | 楽天株式会社 | Image providing apparatus, image processing method, image processing program, and recording medium |
JP2013143730A (en) * | 2012-01-12 | 2013-07-22 | Sony Corp | Imaging element, imaging device, electronic apparatus, and imaging method |
TWI517099B (en) * | 2013-11-15 | 2016-01-11 | 瑞昱半導體股份有限公司 | A consecutive thin edge detection system and method for enhancing a color filter array image |
KR102512828B1 (en) | 2016-01-22 | 2023-03-22 | 삼성전자주식회사 | Event signal processing method and apparatus |
CN106296773B (en) * | 2016-08-12 | 2019-11-29 | 中国科学院计算技术研究所 | A kind of method and apparatus generating multi-level polar plot |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20010045988A1 (en) * | 1999-12-20 | 2001-11-29 | Satoru Yamauchi | Digital still camera system and method |
US20030011708A1 (en) * | 2000-12-14 | 2003-01-16 | Hideaki Kawamura | Image angle detector and scanning line interpolating apparatus |
US20040017852A1 (en) * | 2002-05-29 | 2004-01-29 | Diego Garrido | Predictive interpolation of a video signal |
US6882365B1 (en) * | 1998-07-01 | 2005-04-19 | Ricoh Company, Ltd. | Direction-sensitive correction method and system for data including abrupt intensity gradients |
US20090109296A1 (en) * | 2004-12-02 | 2009-04-30 | Tetsuya Kuno | Pxiel signal processing apparatus and pixel signal processing method |
US20090115870A1 (en) * | 2006-06-29 | 2009-05-07 | Olympus Corporation | Image processing apparatus, computer-readable recording medium recording image processing program, and image processing method |
US20090147093A1 (en) * | 2007-12-10 | 2009-06-11 | Hon Hai Precision Industry Co., Ltd. | Color processing circuit |
US7554583B2 (en) * | 2004-11-04 | 2009-06-30 | Mitsubishi Denki Kabushiki Kaisha | Pixel signal processor and pixel signal processing method |
-
2008
- 2008-12-30 KR KR1020080136928A patent/KR101580168B1/en active IP Right Grant
-
2009
- 2009-12-28 TW TW098145358A patent/TW201026079A/en unknown
- 2009-12-29 US US12/648,929 patent/US8363132B2/en active Active
- 2009-12-30 CN CN200910266067A patent/CN101771883A/en active Pending
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6882365B1 (en) * | 1998-07-01 | 2005-04-19 | Ricoh Company, Ltd. | Direction-sensitive correction method and system for data including abrupt intensity gradients |
US20010045988A1 (en) * | 1999-12-20 | 2001-11-29 | Satoru Yamauchi | Digital still camera system and method |
US20030011708A1 (en) * | 2000-12-14 | 2003-01-16 | Hideaki Kawamura | Image angle detector and scanning line interpolating apparatus |
US20040017852A1 (en) * | 2002-05-29 | 2004-01-29 | Diego Garrido | Predictive interpolation of a video signal |
US7554583B2 (en) * | 2004-11-04 | 2009-06-30 | Mitsubishi Denki Kabushiki Kaisha | Pixel signal processor and pixel signal processing method |
US20090109296A1 (en) * | 2004-12-02 | 2009-04-30 | Tetsuya Kuno | Pxiel signal processing apparatus and pixel signal processing method |
US20090115870A1 (en) * | 2006-06-29 | 2009-05-07 | Olympus Corporation | Image processing apparatus, computer-readable recording medium recording image processing program, and image processing method |
US20090147093A1 (en) * | 2007-12-10 | 2009-06-11 | Hon Hai Precision Industry Co., Ltd. | Color processing circuit |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120257821A1 (en) * | 2009-10-20 | 2012-10-11 | Yasushi Saito | Image processing apparatus and image processing method, and program |
US8755640B2 (en) * | 2009-10-20 | 2014-06-17 | Sony Corporation | Image processing apparatus and image processing method, and program |
US9609291B2 (en) | 2009-10-20 | 2017-03-28 | Sony Corporation | Image processing apparatus and image processing method, and program |
US20130050235A1 (en) * | 2011-08-22 | 2013-02-28 | Chih-Yuan Yang | Color Information Interpolation Method |
US8665287B2 (en) * | 2011-08-22 | 2014-03-04 | Novatek Microelectronics Corp. | Color information interpolation method |
US9875678B2 (en) | 2014-06-10 | 2018-01-23 | Samsung Display Co., Ltd. | Image display method |
US20190020411A1 (en) * | 2017-07-13 | 2019-01-17 | Qualcomm Incorporated | Methods and apparatus for efficient visible light communication (vlc) with reduced data rate |
CN111062331A (en) * | 2019-12-18 | 2020-04-24 | 银河互联网电视有限公司 | Mosaic detection method and device for image, electronic equipment and storage medium |
CN114466170A (en) * | 2021-08-27 | 2022-05-10 | 锐芯微电子股份有限公司 | Image processing method and system |
Also Published As
Publication number | Publication date |
---|---|
US8363132B2 (en) | 2013-01-29 |
KR20100078623A (en) | 2010-07-08 |
CN101771883A (en) | 2010-07-07 |
KR101580168B1 (en) | 2015-12-24 |
TW201026079A (en) | 2010-07-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8363132B2 (en) | Apparatus for demosaicing colors and method thereof | |
EP2359604B1 (en) | Modifying color and panchromatic channel cfa image | |
EP1761072B1 (en) | Image processing device for detecting chromatic difference of magnification from raw data, image processing program, and electronic camera | |
JP4374488B2 (en) | Digital image system and method for combining demosaicing and bad pixel correction | |
CN101090458B (en) | Image processing apparatus, image capture apparatus, image output apparatus, and method and program for these apparatus | |
WO2019148912A1 (en) | Image processing method, apparatus, electronic device and storage medium | |
US9111365B2 (en) | Edge-adaptive interpolation and noise filtering method, computer-readable recording medium, and portable terminal | |
US20030179418A1 (en) | Producing a defective pixel map from defective cluster pixels in an area array image sensor | |
US20130188876A1 (en) | Automatic image straightening | |
CN101511031B (en) | Method for Bayer Data interplation | |
GB2364461A (en) | Correcting defective pixels in an image | |
EP2107521B1 (en) | Detecting a border region in an image | |
US20170257584A1 (en) | Image processing device, image processing method, and image processing system | |
US8854508B2 (en) | Adaptive image processing method and related device | |
US9147257B2 (en) | Consecutive thin edge detection system and method for enhancing a color filter array image | |
US9131097B2 (en) | Method and system for black bar identification | |
CN105451015B (en) | Detection method and device for image dead pixels | |
US6987577B2 (en) | Providing a partial column defect map for a full frame image sensor | |
EP2830020B1 (en) | Object detection device and program | |
CN114500850B (en) | Image processing method, device, system and readable storage medium | |
US8416328B2 (en) | Image processing apparatus and image processing method for processing bayer pattern image data | |
CN113313707A (en) | Original image processing method, device, equipment and readable storage medium | |
JPH099170A (en) | Pattern display region discrimination device | |
JPH09288737A (en) | Onboard picture processor | |
JP2010219862A (en) | Face detection device, and imaging apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: DONGBU HITEK CO., LTD.,KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HYUN, JI CHEOL;JEON, SEUNG HEON;REEL/FRAME:023714/0450 Effective date: 20091228 Owner name: DONGBU HITEK CO., LTD., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HYUN, JI CHEOL;JEON, SEUNG HEON;REEL/FRAME:023714/0450 Effective date: 20091228 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: DB HITEK CO., LTD., KOREA, REPUBLIC OF Free format text: CHANGE OF NAME;ASSIGNOR:DONGBU HITEK CO., LTD.;REEL/FRAME:044559/0819 Effective date: 20171101 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |