WO2007075052A1 - Display driver and image prosessing apparatus for interpolating color and control method thereof - Google Patents

Display driver and image prosessing apparatus for interpolating color and control method thereof Download PDF

Info

Publication number
WO2007075052A1
WO2007075052A1 PCT/KR2006/005781 KR2006005781W WO2007075052A1 WO 2007075052 A1 WO2007075052 A1 WO 2007075052A1 KR 2006005781 W KR2006005781 W KR 2006005781W WO 2007075052 A1 WO2007075052 A1 WO 2007075052A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
bayer rgb
display driver
rgb data
bit
Prior art date
Application number
PCT/KR2006/005781
Other languages
French (fr)
Inventor
Hun-Joon Jung
Original Assignee
Mtekvision Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mtekvision Co., Ltd. filed Critical Mtekvision Co., Ltd.
Publication of WO2007075052A1 publication Critical patent/WO2007075052A1/en

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/02Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the way in which colour is displayed
    • G09G5/04Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the way in which colour is displayed using circuits for interfacing with colour displays
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/2092Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G09G3/2096Details of the interface to the display terminal specific for a flat panel
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/02Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the way in which colour is displayed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • H04N23/84Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/843Demosaicing, e.g. interpolating colour pixel values
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/02Improving the quality of display appearance
    • G09G2320/0271Adjustment of the gradation levels within the range of the gradation scale, e.g. by redistribution or clipping
    • G09G2320/0276Adjustment of the gradation levels within the range of the gradation scale, e.g. by redistribution or clipping for the purpose of adaptation to the characteristics of a display device, i.e. gamma correction
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/06Handling electromagnetic interferences [EMI], covering emitted as well as received electromagnetic radiation
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/003Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G09G5/006Details of the interface to the display terminal
    • G09G5/008Clock recovery
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2209/00Details of colour television systems
    • H04N2209/04Picture signal generators
    • H04N2209/041Picture signal generators using solid-state devices
    • H04N2209/042Picture signal generators using solid-state devices having a single pick-up sensor
    • H04N2209/045Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
    • H04N2209/046Colour interpolation to calculate the missing colour values

Definitions

  • the present invention is related to a display driver, an image processing
  • CMOS complementary metal-oxide semiconductor
  • the image processing technology cause the quick extension of the digital camera market.
  • Mobile apparatuses such as mobile phones and PDAs, having digital camera functions
  • An image processing apparatus captures an image related to a photographic subject by using an image sensor such as the CCD and the CMOS.
  • the image sensor represents all colors of the nature by use of red (R), green
  • the image sensor includes a plurality of unit pixels that convert a light signal
  • one unit pixel of the image sensor detects
  • converting means such as a digital analog converter (DAC) before being outputted to DAC
  • ISP image signal processor
  • CCP camera signal processor
  • this digital signal is
  • the ISP or the CCP conventionally performs an actual
  • the data which is color-interpolated in the ISP or CCP, passes through a
  • backend chip (not shown) and is transferred to a display driver.
  • FIG. 2 is a schematic illustrating the display driver in accordance with the prior art. As illustrated in FIG. 2, the display driver in accordance with the prior art receives
  • the data has an amount of information 3 times as much as the Bayer RGB data.
  • EMI electromagnetic interference
  • the memory in the display driver must have the size 3 times as large as
  • the image processing apparatus such as a camera in a small size.
  • the data bus consists of a lot of data lines.
  • the data bus which consists of 18 lines, that is, 3 times as much as the bit
  • the present invention which is designed to solve the above problems, provides
  • the present invention provides a display driver, an image processing
  • the present invention provides a display driver, an image processing
  • the present invention provides a display driver, an image processing
  • a display driver outputting an image to a display, the image being captured
  • an image sensor having a plurality of unit pixels includes a system interface
  • Bayer RGB data having one of red (R), green (G) and blue (B) data that are captured by
  • the unit pixel a graphic memory, storing the received n bit Bayer RGB data; and a color
  • interpolation unit interpolating the n bit Bayer RGB data stored in the graphic memory
  • the display driver of the present invention can further include a register, which
  • the display driver can further include a write latch, which writes the
  • the data bus can have n lines
  • the display can be one of a liquid crystal device (LCD), an organic
  • EL electroluminescence display
  • PDP plasma display panel
  • CTR field emission display
  • FED field emission display
  • an image sensor having a plurality of unit pixels includes receiving n bit Bayer RGB data detected in the image sensor through a data bus, the Bayer RGB data having one of
  • processing apparatus includes an image sensor, having a plurality unit pixels, and
  • Bayer RGB data having one of red (R), green (G)
  • controller controlling the image sensor and the display driver.
  • FIG. 1 illustrates a color filter array in accordance with a prior art
  • FIG. 2 is a schematic illustrating a display driver in accordance with a prior art
  • FIG. 3 illustrates the operation of the nearest neighborhood replication
  • FIG. 4 illustrates the operation of an edge sensing interpolation
  • FIG. 5 and FIG. 6 illustrate the operation of determining G5 by a pixel B5
  • FIG. 7 illustrates each pattern type in a pattern recognition interpolation
  • FIG. 8 and FIG. 9 illustrate the operation of a pattern recognition interpolation.
  • FIG. 10 illustrates images before and after interpolation
  • FIG. 11 is a block diagram illustrating an image processing apparatus in
  • FIG. 12 is a block diagram illustrating a display drive in accordance with an
  • the present invention can increase the system processing efficiency by
  • the actual color interpolation allows one unit pixel to have
  • B values are generated for the unit pixel with reference to the R and B values in the
  • FIG. 3 through FIG. 9 illustrate the various interpolation methods.
  • FIG. 3 illustrates the operation of the nearest neighborhood replication.
  • the pixels having G values are Compared with the color filter array of FIG. 2, the pixels having G values are
  • the nearest neighborhood replication determines R, G, and/or B values, which
  • n th pixel is represented as R(n), G(n) and B(n).
  • the pixels G8, Gl 1, G14 and Gl 8 of 9 pixels has no value.
  • the nearest neighborhood replication method determines each incomplete value as
  • the G8 is determined as G7, the G12 is as Gl 1, the G14 is as G13 and the
  • G18 is as G17.
  • the G8 is determined by the formula of (G3+G7+G9+G13)/4 through the values of left, right, upper and lower pixels.
  • the smooth hue transition method first calculates a green value by using the
  • B13 G13/2*(B8/G8+B18/G18), B12-G12/4*(B6/G6+B8/G8+B16/G16+B18/G18).
  • the edge interpolation method first determines each changed amount of the
  • the G8 is determined through the following discriminant by using the
  • the 5 x 5 interpolation is used.
  • the edge sensing interpolation method defines the vertically and horizontally
  • the G5 in the pixel B 5, the G5 can be determined through the following formula 3 and
  • the discriminant is as follows;
  • G5 (G2 +G8)/2 + (B5-Bl +B5 -B9) /4 ; Else
  • the method of calculating the red and blue values has the following 3
  • the discriminant is as follows;
  • R5 (K3+ R7) n +(G5 -G3 +G5 ⁇ G7) /2 ;
  • L refers to pixels having values smaller than
  • the pertinent image can be classified into patterns as illustrated in FIG. 6.
  • the patterns (a) and (d) are defined as an edge pattern; the (b) is as a
  • M refers to SUM(G)/4
  • S refers to SUM(X)/8
  • 'clip' refers to a
  • the M refers to median ⁇ H's, L's ⁇ and the S SUM(X)/4.
  • the algorithm of the aforementioned edge sensing interpolation method is
  • delta_H abs(l/2*(inB(ij-2)+inB(ij+2))-inB(i,j));
  • delta_V abs(l/2*(inB(i-2,j)+inB(i+2J))-inB(i J)); if delta_H ⁇ delta_V,
  • outR(i,2:2:n) outG(i,2:2:n)+l/2*(inR(i-l,2:2:n)-outG(i-l,2:2:n)+inR(i+l,2:2:n)-outG(i+l,2:2:n));
  • FIG. 10 illustrates images before and after interpolation. In detail, the original
  • the ISP or CCP conventionally performs the actual color image interpolation.
  • the display driver controls the display driver
  • FIG. 11 is a block diagram illustrating an image processing apparatus in
  • image processing apparatus of the present invention can have an image sensor 10, an image processing signal processor (ISP) 20, a backend chip 30, a display driver 40 and a
  • the image sensor 10 has a color filter array and a plurality of unit pixels.
  • the data detected in each unit pixel refers to Bayer RGB data.
  • the image processing operation can include at lease one of gamma
  • the backend chip 30 which controls all kinds of elements of the image
  • processing apparatus can have a controller such as a central processing unit (CPU) or a
  • the backend chip 30 can have a codec, which
  • the backend chip 30 can perform the aforementioned image processing
  • the ISP or the backend chip performs the actual color
  • the operation of transferring to the display driver requests the high clock
  • the display has the 6 bit resolution and the size of 176*240, in
  • the image processing apparatus of the present invention To solve the problem, the image processing apparatus of the present invention
  • the display driver 40 of the present invention receives Bayer RGB data, which
  • the ISP 20 or the backend chip 30 converts the actual-color
  • FIG. 12 is a block diagram illustrating a display drive in accordance with the embodiment of the present invention. Referring to FIG. 12, a system interface 100
  • the included in the display driver receives the Bayer RGB data trough a data bus.
  • the ISP 20 or the backend chip 30 in the case of having the 6 bit resolution, the ISP 20 or the backend chip 30
  • the data bus consists of 18 lines.
  • the data bus can consist of only 6 lines because the actual color
  • the Bayer RGB data which is received in the system interface 100, is stored in
  • the register can have an address register, which stores address
  • a counter which counts the size of the transferred
  • control register which outputs a control signal for writing and reading the
  • the display driver further includes a read latch 106, which reads the data stored
  • the Bayer RGB data stored in the graphic memory 108 is
  • the interpolated data is
  • the display 50 can be one of a liquid crystal device (LCD), an organic
  • EL electroluminescence display
  • PDP plasma display panel
  • CTR field emission display
  • FED field emission display
  • the graphic memory size is 6 x 176
  • the size is decreased 1/3 times as much as the
  • the graphic memory size of 8 bits in the above example is 8 x 176 x 240 / 8
  • the present invention can lower the clock speed of an
  • the present invention can prevent a wrong operation due to the
  • EMI electromagnetic interference
  • the present invention can decrease the memory size of a display driver.
  • the present invention can make an image processing apparatus in a

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Color Image Communication Systems (AREA)

Abstract

The present invention is related to a display driver, an image processing apparatus, and a control method thereof for color interpolation. According to an embodiment of the present invention, a display driver, outputting an image to a display, the image being captured in an image sensor having a plurality of unit pixels includes a system interface, receiving n bit Bayer RGB data detected in the image sensor through a data bus, the Bayer RGB data comprising one of red (R), green (G) and blue (B) data that are captured by the unit pixel; a graphic memory, storing the received n bit Bayer RGB data; and a color interpolation unit, interpolating the n bit Bayer RGB data stored in the graphic memory to 3n bit actual color data. Accordingly, the present invention can increase the system efficiency through the color interpolation of the display driver.

Description

[DESCRIPTION]
[Invention Title]
DISPLAY DRIVER AND IMAGE PROCESSING APPARATUS FOR
INTERPOLATING COLOR AND CONTROL METHOD THEREOF
[Technical Field]
The present invention is related to a display driver, an image processing
apparatus, and a control method thereof for color interpolation, more specifically to a
display driver, an image processing apparatus, and a control method for color
interpolation that can improve system performance through an efficient interpolation of
actual color.
[Background Art]
Today' s high integration technology of a charged-coupled devices (CCD) and a
complementary metal-oxide semiconductor (CMOS) and remarkable development of
the image processing technology cause the quick extension of the digital camera market.
Mobile apparatuses, such as mobile phones and PDAs, having digital camera functions
are familiarized.
An image processing apparatus captures an image related to a photographic subject by using an image sensor such as the CCD and the CMOS.
The image sensor represents all colors of the nature by use of red (R), green
(G) and blue (B). As illustrated in FIG. 1, a typical image sensor uses a color filter array
(CFA) to reduce a manufacturing cost.
The image sensor includes a plurality of unit pixels that convert a light signal
related to the photographic subject to an electrical signal and output the converted signal.
In the case of using the aforementioned CFA, one unit pixel of the image sensor detects
a value of one among each of the R, G and B components.
R, G and B values, detected by each unit pixel, are converted to digital signals
by converting means such as a digital analog converter (DAC) before being outputted to
an image signal processor (ISP) or a camera signal processor (CCP).
Actual colors related to the photographic subject can be represented by using
the combination of the RGB components. However, since the data detected by each unit
pixel has information related to only one of the RGB components (this digital signal is
referred to as Bayer RGB data), the ISP or the CCP conventionally performs an actual
color interpolation such that one unit pixel can have the information related to all R, G
and B components by using the R, G and B data, detected by each unit pixel.
The data, which is color-interpolated in the ISP or CCP, passes through a
backend chip (not shown) and is transferred to a display driver.
FIG. 2 is a schematic illustrating the display driver in accordance with the prior art. As illustrated in FIG. 2, the display driver in accordance with the prior art receives
data interpolated with the actual color through a data bus 200 and stores the data in a
memory 202 to output it to a display unit (not shown).
Since the data interpolated with the actual color has all RGB data for each unit
pixel, the data has an amount of information 3 times as much as the Bayer RGB data. In
accordance with the prior art, the actual color interpolation is performed prior to the
display driver. Accordingly, the high clock speed is requested to transfer the actual
color interpolated data from the ISP to the backend chip and from the backend chip to
the display driver.
As such, the high clock speed makes it more difficult to design an internal chip.
Also, since the power consumption is increased 3 times as much during the signal
transferring process, there is a high possibility of performing a wrong operation due to
the electromagnetic interference (EMI).
Further, the memory in the display driver must have the size 3 times as large as
the data detected in the image sensor to store the actual color interpolated data, to
thereby increase the size of the display driver. Accordingly, it becomes difficult to make
the image processing apparatus such as a camera in a small size.
Still, the data bus consists of a lot of data lines. In the case of setting the 6 bit
resolution, the data bus, which consists of 18 lines, that is, 3 times as much as the bit
number 6, causes increasing the image processing apparatus in size. [Disclosure]
[Technical Problem]
The present invention, which is designed to solve the above problems, provides
a display driver, an image processing apparatus, and a control method for color
interpolation that can lower the clock speed of the image processing apparatus.
Also, the present invention provides a display driver, an image processing
apparatus, and a control method for color interpolation that can prevent a wrong
operation due to the electromagnetic interference (EMI) by reducing the power
consumption during the internal signal transferring process of the image processing
apparatus.
Also, the present invention provides a display driver, an image processing
apparatus, and a control method for color interpolation that can decrease the memory
size of the display driver.
Also, the present invention provides a display driver, an image processing
apparatus, and a control method for color interpolation that can make the image
processing apparatus in a small size by reducing the size of the display driver and
decreasing the number of data bus lines.
[Technical Solution] To achieve the above objects, according to an embodiment of the present
invention, a display driver, outputting an image to a display, the image being captured
in an image sensor having a plurality of unit pixels includes a system interface,
receiving n bit Bayer RGB data detected in the image sensor through a data bus, the
Bayer RGB data having one of red (R), green (G) and blue (B) data that are captured by
the unit pixel; a graphic memory, storing the received n bit Bayer RGB data; and a color
interpolation unit, interpolating the n bit Bayer RGB data stored in the graphic memory
to 3n bit actual color data.
The display driver of the present invention can further include a register, which
has address information on the graphic memory, and controls to read / write the data
between the system interface and the graphic memory.
Also, the display driver can further include a write latch, which writes the
Bayer RGB data received in the system interface, in the graphic memory.
Preferably, the data bus can have n lines
Also, the display can be one of a liquid crystal device (LCD), an organic
electroluminescence display (EL), a plasma display panel (PDP), a cathode-ray tube
(CRT) display and a field emission display (FED).
According to another embodiment of the present invention, a method of
controlling a display driver, outputting an image to a display, the image being captured
in an image sensor having a plurality of unit pixels includes receiving n bit Bayer RGB data detected in the image sensor through a data bus, the Bayer RGB data having one of
red (R), green (G) and blue (B) data that are captured by the unit pixel; storing the
received n bit Bayer RGB data; and interpolating the n bit Bayer RGB data stored in the
graphic memory to 3n bit actual color data.
According to another embodiment of the present invention, an image
processing apparatus includes an image sensor, having a plurality unit pixels, and
detecting n bit Bayer RGB data, the Bayer RGB data having one of red (R), green (G)
and blue (B) data that are captured by the unit pixel; a display driver, receiving and
storing the n bit Bayer RGB data detected in the image sensor through a data bus, and
interpolating the stored n bit Bayer RGB data to 3n bit actual color data; and a
controller, controlling the image sensor and the display driver.
[Description of Drawings]
FIG. 1 illustrates a color filter array in accordance with a prior art;
FIG. 2 is a schematic illustrating a display driver in accordance with a prior art;
FIG. 3 illustrates the operation of the nearest neighborhood replication;
FIG. 4 illustrates the operation of an edge sensing interpolation;
FIG. 5 and FIG. 6 illustrate the operation of determining G5 by a pixel B5;
FIG. 7 illustrates each pattern type in a pattern recognition interpolation;
FIG. 8 and FIG. 9 illustrate the operation of a pattern recognition interpolation. FIG. 10 illustrates images before and after interpolation;
FIG. 11 is a block diagram illustrating an image processing apparatus in
accordance with an embodiment of the present invention; and
FIG. 12 is a block diagram illustrating a display drive in accordance with an
embodiment of the present invention.
[Mode for Invention]
Hereinafter, an exemplary embodiment related to a display driver, an image
processing apparatus, and a control method for color interpolation in accordance with
the present invention will be described in detail with reference to the accompanying
drawings.
The present invention can increase the system processing efficiency by
improving the actual color interpolation method of Bayer RGB data detected in an
image sensor unlike a conventional art.
As described above, the actual color interpolation allows one unit pixel to have
all information related to RGB components by using the R, G and B data of each pixel.
For example, in case that there is a G value only for a unit pixel at the beginning, R and
B values are generated for the unit pixel with reference to the R and B values in the
periphery of the unit pixel.
This actual color interpolation can be performed depending on various methods. FIG. 3 through FIG. 9 illustrate the various interpolation methods.
The operation of the actual color interpolation will be described in detail with
reference to each pertinent drawing.
FIG. 3 illustrates the operation of the nearest neighborhood replication.
Compared with the color filter array of FIG. 2, the pixels having G values are
selectively illustrated.
The nearest neighborhood replication determines R, G, and/or B values, which
are incomplete in a particular pixel, through the corresponding values in the periphery
of the unit pixel.
For the sake of convenience in this description, the RGB data corresponding to
nth pixel is represented as R(n), G(n) and B(n).
Referring to FIG. 3, the pixels G8, Gl 1, G14 and Gl 8 of 9 pixels has no value.
The nearest neighborhood replication method determines each incomplete value as
pertinent values in the periphery of the unit pixel. Through the nearest neighborhood
replication, the G8 is determined as G7, the G12 is as Gl 1, the G14 is as G13 and the
G18 is as G17.
Alternatively, there can be provided a bilinear interpolation method, which
determines each of incomplete R, G and/or B values as the average values of the
pertinent values in the periphery of the unit pixel.
Also, referring to FIG. 3, the G8 is determined by the formula of (G3+G7+G9+G13)/4 through the values of left, right, upper and lower pixels.
Also, there can be provided a smooth hue transition method.
The smooth hue transition method first calculates a green value by using the
bilinear interpolation method and then determines red and blue values by using hue
values of B/G and R/G.
Referring to the color filter array of FIG. 2, in the case of determining the Blue
value, the B7, B13 and B12 determined by the formulas of B7=G7/2*(B6/G6+B8/G8),
B13=G13/2*(B8/G8+B18/G18), B12-G12/4*(B6/G6+B8/G8+B16/G16+B18/G18).
Also, there can be provided an edge sensing interpolation method, which is
described below by way of example of determining the green value in the pixel B8 of
FIG. 2.
The edge interpolation method first determines each changed amount of the
horizontal and vertical pixels of the desired pixel by the following formula 1.
[Formula 1 ]
ΔH - P? -O9| awl Δ¥ « |O3 - O13|
Then, the G8 is determined through the following discriminant by using the
value of the smaller changed mount. If Λ< 4? f
01» (07 +09) /2» EIm W M > AΨt
EIw
G8 - (3S3 +Ϊ37 +O9
End
In another edge sensing interpolation method, the 5 x 5 interpolation is used.
As described in FIG. 4, by way of example of determining the G5 in the pixel
B5, the edge sensing interpolation method defines the vertically and horizontally
changed amounts through the following formula 2.
[Formula 2]
Figure imgf000011_0001
Also, there can be provided a linear interpolation, performed with Lapacian
second-order color correction together.
As described in FIG. 5 and FIG. 6, by way of example of determining the G5
in the pixel B 5, the G5 can be determined through the following formula 3 and
discriminant. [Formula 3]
S- B3→. BS » B?| ΔV = |G2 - GS|+ |B5 - Bl + BS - B?
The discriminant is as follows;
If ZH < ΔV.
G5 = (G2 +G8)/2 + (B5-Bl +B5 -B9) /4 ; Else
05- (02404 *m + OS ) /4+ (BJ -Bl +B5 -B3 *W -B7 + 1 End
Then, the method of calculating the red and blue values has the following 3
cases depending on the position.
1) R4=(Rl+R7)/2+(G4-Gl+G4-G7)/4
2) R2=(Rl+R3)/2+(G2-Gl+G2-G3)/4
3) To determine the red value in the pixel B 5, the diagonally changed amount
can be defined through the following formula 4, and then the following discriminant can
be used
[Formula 4] ΔW = |R1 - R9j+ |G5 " GU G5
Figure imgf000013_0001
+ GS
The discriminant is as follows;
11» if ΔN > ΔP,
R5 =(K3+ R7) n +(G5 -G3 +G5^G7) /2 ;
Else
HJ -(RI +R3 +R7 +19) /4+<GJ -Ol +05 »03 +05 -0? +05 -<#) /4 ;
Also, there can be provided a pattern recognition interpolation method.
The case of determining the green value in the pixel Rl 2 of FIG. 2 will be
described by way of example.
In case that the green value is determined in the pixel Rl 2, if H refers to pixels
having values larger than an average and L refers to pixels having values smaller than
the average, the pertinent image can be classified into patterns as illustrated in FIG. 6.
In FIG. 7, the patterns (a) and (d) are defined as an edge pattern; the (b) is as a
strip pattern; and the (c) is as a corner pattern.
In the case of the edge pattern of the (a) and (d), the Gl 2 is determined through
the following formula 5. [Formula 5]
G12=median{G7,Gl l,G13,G17}=(B+C)/2
where the value B and C refers to the second and third largest numbers,
respectively.
Then, in the case of the stripe pattern, referring to FIG. 8, the G?? is
determined through the following formula 6.
[Formula 6]
Figure imgf000014_0001
where M refers to SUM(G)/4, S refers to SUM(X)/8 and 'clip' refers to a
function allowing the result value to be between B and C like the aforementioned
formula 5.
Finally, in the case of the corner pattern, referring to FIG. 9, the G?? is
determined through the aforementioned formula 6 like the stripe pattern, but unlikely,
the M refers to median{H's, L's} and the S SUM(X)/4. The algorithm of the aforementioned edge sensing interpolation method is
follows;
m = size(in,l); n = size(in,2); inR = in(:,:,l); inG = in(:,:,2); inB = in(:,:,3); out
= in; outR = inR; outG = inG; outB = inB; % G channel for i=4:2:m-2, for
j=3:2:n-3, delta_H = abs(l/2*(inB(ij-2)+inB(ij+2))-inB(i,j));
delta_V = abs(l/2*(inB(i-2,j)+inB(i+2J))-inB(i J)); if delta_H < delta_V,
outG(ij) = l/2*(inG(i,j-l)+inG(i,j+l)); elseif delta H > delta_V,
outG(i,j) = l/2*(inG(i-lj)+inG(i+l,j)); else outG(ij) =
l/4*(inG(iJ-l)+inG(i,j+l)+inG(i-l,j)+inG(i+l,j)); end end end for
i=3:2:m-3, forj=4:2:n-2, delta H =
abs(l/2*(inR(i,j-2)+inR(i,j+2))-inR(i,j)); delta_V =
abs(l/2*(inR(i-2,j)+inR(i+2,j))-inR(i J)); if delta_H < delta_V,
outG(ij) = l/2*(inG(ij-l)+inG(ij+l)); elseif delta H > delta_V,
outG(ij) = l/2*(inG(i-lj)+inG(i+l,j)); else outG(ij) =
l/4*(inG(ij-l)+inG(i,j+l)+inG(i-l,j)+inG(i+l,j)); end end end outG =
round(outG); ind = fmd(outG>255); outG(ind) = 255;
% R channel for i=l:2:m-l, outR(i,3:2:n-l) = outG(i,3:2:n-l) +
l/2*(inR(i,2:2:n-2)-outG(i,2:2:n-2)+inR(i,4:2:n)-outG(i,4:2:n)); end for i=2:2:m-2,
outR(i,2:2:n) = outG(i,2:2:n)+l/2*(inR(i-l,2:2:n)-outG(i-l,2:2:n)+inR(i+l,2:2:n)-outG(i+l,2:2:n));
outR(i,3:2:n-l) =
outG(i,3:2:n-l)+l/4*(inR(i-l,2:2:n-2)-outG(i-l,2:2:n-l)+inR(i-l,4:2:n)-outG(i-l,4:2:n)
+inR(i+l,2:2:n-2)-outG(i+l,2:2:n-2)+inR(i+l,4:2:n)-outG(i+l,4:2:n)); end outR =
round(outR); ind = fmd(outR>255); outR(ind) = 255; ind = find(outR<0); outR(ind) =
0; % B channel for i=2:2:m, outB(i,2:2:n-2) =
outG(i,2:2:n-2)+l/2*(inB(i,l:2:n-3)-outG(i,l:2:n-3)+inB(i,3:2:n-l)-outG(i,3:2:n-l));
end for i=3 :2 :m- 1 , outB(i, 1 :2 :n- 1 ) =
outG(i,l :2:n-l)+l/2*(inB(i-l,l:2:n-l)-outG(i-l,l:2:n-l)+inB(i+l,l:2:n-l)-outG(i+l,l :2:
n-1)); outB(i,2:2:n-2) =
outG(i,2:2:n-2)+l/4*(inB(i-l,l:2:n-3)-outG(i-l,l :2:n-3)+inB(i-l,3:2:n-l)-outG(i-l,3:2:n
-l)+inB(i+l,l:2:n-3)-outG(i+l,l:2:n-3)+inB(i+l,3:2:n-l)-outG(i+l,3:2:n-l)); end outB
= round(outB); ind - fmd(outB>255); outB(ind) = 255; ind = fmd(outB<0); outB(ind) =
0; out(:,:,l) = outR; out(:,:,2) = outG; out(:,:,3) = outB;
FIG. 10 illustrates images before and after interpolation. In detail, the original
of the captured image and the images interpolated according to the Laplacian
interpolation and edge sensing interpolation methods are illustrated. The following table
shows the mean square error (MSE) and the computional cost of each aforementioned
interpolation. [Table 1 ]
Figure imgf000017_0001
Referring to Table 1, it is recognized that the edge sensing II and the Laplacian
second-order color correction must be suitably used in accordance with the image
quality and the computational cost.
When these various interpolations are performed in the image processing
apparatus, the ISP or CCP conventionally performs the actual color image interpolation.
However, in accordance with an embodiment of the present invention, the display driver
receives the Bayer RGB data and performs the actual color image interpolation.
FIG. 11 is a block diagram illustrating an image processing apparatus in
accordance with the embodiment of the present invention. As illustrated in FIG. 11, the
image processing apparatus of the present invention can have an image sensor 10, an image processing signal processor (ISP) 20, a backend chip 30, a display driver 40 and a
display 50.
The image sensor 10 has a color filter array and a plurality of unit pixels. Each
of the unit pixels detects one of R, G and B data.
The data detected in each unit pixel refers to Bayer RGB data. The Bayer RGB
data, which is inputted into the ISP 20, passes through a predetermined image
processing operation.
Here, the image processing operation can include at lease one of gamma
correction, which corrects a gamma value set for the display 50; histogram equalization,
which makes the histogram of a pixel value outputted from the image sensor, and
adjusts a small Max value to 255 to uniformly stretch it if the Max value is small, edge
enhancement, subsampling, auto exposure and auto white balance.
Through the aforementioned image processing operation, there can be provided
the more improved image related to the photographic subject.
The backend chip 30, which controls all kinds of elements of the image
processing apparatus, can have a controller such as a central processing unit (CPU) or a
micro controller unit (MCU). Also, the backend chip 30 can have a codec, which
compresses a processed image, and a built-in memory, which stores compressed data.
Also, the backend chip 30 can perform the aforementioned image processing
operation by having built-in means such as the CCP. In the prior art, the ISP or the backend chip performs the actual color
interpolation of the Bayer RGB data and transfers the interpolated data to the display
driver. However, in case that the actual color interpolation is performed prior to the
display driver, the operation of transferring to the display driver requests the high clock
speed due to increasing amount of data and increases the graphic memory size of the
display driver.
For example, if the display has the 6 bit resolution and the size of 176*240, in
accordance with the prior art, the graphic memory size requested for the display driver
becomes 6 x 3 x 176 x 240 / 8 bit, that is 90,040 bite.
Accordingly, the prior art requests very large graphic memory size of the
display drive. The higher the resolution of the display is, the larger the size is.
To solve the problem, the image processing apparatus of the present invention
provides the display driver 40 equipped with the color interpolation unit 60.
The display driver 40 of the present invention receives Bayer RGB data, which
is not actual-color-interpolated, from the ISP 20 or the backend chip 30.
Even though the ISP 20 or the backend chip 30 performs the actual color
interpolation operation, the ISP 20 or the backend chip 30 converts the actual-color
-interpolated data to Bayer RGB data and the Bayer RGB data to the display driver 40.
FIG. 12 is a block diagram illustrating a display drive in accordance with the embodiment of the present invention. Referring to FIG. 12, a system interface 100
included in the display driver receives the Bayer RGB data trough a data bus.
Compared with the schematic of FIG. 2, for example in accordance with the
prior art, in the case of having the 6 bit resolution, the ISP 20 or the backend chip 30
transfers image data in 18 bits due to performance of the actual color interpolation.
Accordingly, as illustrated in FIG. 2, the data bus consists of 18 lines.
However, in accordance with the present invention, in the case of having the 6
bit resolution, the data bus can consist of only 6 lines because the actual color
interpolation is not performed prior to the display driver 40.
In other words, in the case of representing each value in n bits, the data bus can
have n lines only.
The Bayer RGB data, which is received in the system interface 100, is stored in
the graphic memory 108 through the control operation of a register 102 and the writing
operation of a write latch 104.
Here, the register can have an address register, which stores address
information on the graphic memory, a counter, which counts the size of the transferred
data, and a control register, which outputs a control signal for writing and reading the
data.
The display driver further includes a read latch 106, which reads the data stored
in the graphic memory 108, in addition to the write latch 104. On the other hand, the Bayer RGB data stored in the graphic memory 108 is
actual-color-interpolated through a color interpolation unit 110. The interpolated data is
passed through an image outputting unit 112 and outputted to the display 50.
Here, the display 50 can be one of a liquid crystal device (LCD), an organic
electroluminescence display (EL), a plasma display panel (PDP), a cathode-ray tube
(CRT) display and a field emission display (FED), but not limited thereto.
As such, in case that the display driver performs the actual color interpolation,
if the resolution is set as 6 bit as the above example, the graphic memory size is 6 x 176
x 240 / 8, that is, 31,810. In other words, the size is decreased 1/3 times as much as the
prior art having the same bit resolution.
Also, the transmission of data having the size decreased 1/3 times as much as
the prior art does not request the increase of the clock speed, to thereby improve the
apparatus efficiency remarkably. Even though the resolution is set higher than 6 bits
(e.g. 8 bits), the graphic memory size of 8 bits in the above example is 8 x 176 x 240 / 8,
that is, 42,240. Accordingly, the color can be more detailedly displayed even in the
graphic memory size more decreased than the prior art.
The drawings and detailed description are only examples of the present
invention, serve only for describing the present invention. Thus, any person of ordinary
skill in the art shall understand that a large number of permutations and other equivalent
embodiments are possible. The true scope of the present invention must be defined only by the spirit of the appended claims.
[Industrial Applicability]
As described above, the present invention can lower the clock speed of an
image processing apparatus by reducing the amount of data transferred to a display
driver.
Also, the present invention can prevent a wrong operation due to the
electromagnetic interference (EMI) by reducing the power consumption during the
internal signal transferring process of an image processing apparatus.
Also, the present invention can decrease the memory size of a display driver.
In addition, the present invention can make an image processing apparatus in a
small size by reducing the size of the display driver and decreasing the number of data
bus lines.

Claims

[CLAIMS]
[Claim 1 ]
A display driver, outputting an image to a display, the image being captured in
an image sensor having a plurality of unit pixels, the display driver comprises:
a system interface, receiving n bit Bayer RGB data detected in the image
sensor through a data bus, the Bayer RGB data comprising one of red (R), green (G)
and blue (B) data that are captured by the unit pixel;
a graphic memory, storing the received n bit Bayer RGB data; and
a color interpolation unit, interpolating the n bit Bayer RGB data stored in the
graphic memory to 3n bit actual color data.
[Claim 2]
The display driver of Claim 1, further comprising a register, which comprises
address information on the graphic memory, and controls to read / write the data
between the system interface and the graphic memory.
[Claim 3]
The display driver of Claim 1 , further comprising a write latch, which writes
the Bayer RGB data received in the system interface, in the graphic memory.
[Claim 4]
The display driver of Claim 1, wherein the data bus comprises n lines.
[Claim 5]
The display driver of Claim 1, wherein the display is one of a liquid crystal
device (LCD), an organic electroluminescence display (EL), a plasma display panel
(PDP), a cathode-ray tube (CRT) display and a field emission display (FED).
[Claim 6]
A method of controlling a display driver, outputting an image to a display, the
image being captured in an image sensor having a plurality of unit pixels, the method
comprises:
receiving n bit Bayer RGB data detected in the image sensor through a data bus,
the Bayer RGB data comprising one of red (R), green (G) and blue (B) data that are
captured by the unit pixel;
storing the received n bit Bayer RGB data; and
interpolating the n bit Bayer RGB data stored in the graphic memory to 3n bit
actual color data.
[Claim 7]
The method of Claim 6, wherein in the receiving of the n bit Bayer RGB data,
the Bayer RGB data is received through the data bus of n lines.
[Claim 8]
An image processing apparatus comprising:
an image sensor, comprising a plurality unit pixels, and detecting n bit Bayer
RGB data, the Bayer RGB data comprising one of red (R), green (G) and blue (B) data
that are captured by the unit pixel;
a display driver, receiving and storing the n bit Bayer RGB data detected in the
image sensor through a data bus, and interpolating the stored n bit Bayer RGB data to
3n bit actual color data; and
a controller, controlling the image sensor and the display driver.
[Claim 9]
The image processing apparatus of Claim 8, wherein the display driver
comprises a graphic memory, which stores the received n bit Bayer RGB data; and
a color interpolation unit which interpolates the n bit Bayer RGB data, stored in
the graphic memory, to 3n bit actual color data [Claim 10]
The image processing apparatus of Claim 8, wherein the data bus comprises n
lines.
PCT/KR2006/005781 2005-12-27 2006-12-27 Display driver and image prosessing apparatus for interpolating color and control method thereof WO2007075052A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2005-0130522 2005-12-27
KR1020050130522A KR100723046B1 (en) 2005-12-27 2005-12-27 Display driver and image prosessing apparatus for interpolating color and control method thereof

Publications (1)

Publication Number Publication Date
WO2007075052A1 true WO2007075052A1 (en) 2007-07-05

Family

ID=38218238

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2006/005781 WO2007075052A1 (en) 2005-12-27 2006-12-27 Display driver and image prosessing apparatus for interpolating color and control method thereof

Country Status (2)

Country Link
KR (1) KR100723046B1 (en)
WO (1) WO2007075052A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100833190B1 (en) * 2006-11-16 2008-05-28 삼성전자주식회사 Response Time Accelerator and method thereof in LCD Timing Controller

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1093762A (en) * 1996-09-18 1998-04-10 Fuji Photo Film Co Ltd Image data storing method and device therefor
US20040263650A1 (en) * 2003-06-26 2004-12-30 Samsung Electro-Mechanics Co., Ltd. Line buffer and method of providing line data for color interpolation
US20050030396A1 (en) * 2003-08-07 2005-02-10 Samsung Electro-Mechanics Co., Ltd. Apparatus for and method of edge enhancement in image processing

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100390847B1 (en) * 2001-06-30 2003-07-12 주식회사 하이닉스반도체 Color interpolation apparatus in image sensor

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1093762A (en) * 1996-09-18 1998-04-10 Fuji Photo Film Co Ltd Image data storing method and device therefor
US20040263650A1 (en) * 2003-06-26 2004-12-30 Samsung Electro-Mechanics Co., Ltd. Line buffer and method of providing line data for color interpolation
US20050030396A1 (en) * 2003-08-07 2005-02-10 Samsung Electro-Mechanics Co., Ltd. Apparatus for and method of edge enhancement in image processing

Also Published As

Publication number Publication date
KR100723046B1 (en) 2007-05-30

Similar Documents

Publication Publication Date Title
KR101269344B1 (en) Temporal filtering techniques for image signal processing
KR101258039B1 (en) System and method for detecting and correcting defective pixels in an image sensor
KR101303415B1 (en) System and method for demosaicing image data using weighted gradients
RU2543974C2 (en) Auto-focus control using image statistics data based on coarse and fine auto-focus scores
RU2537038C2 (en) Automatic white balance processing with flexible colour space selection
US9070185B2 (en) Noise filtering method and apparatus considering noise variance and motion detection
US7589748B2 (en) Image sensor with scaler and image scaling method thereof
EP1067507A1 (en) Image display
US20110090370A1 (en) System and method for sharpening image data
US8194155B2 (en) Information processing apparatus, buffer control method, and computer program
US8059169B2 (en) Apparatus and method for compensating color, and image processor, digital processing apparatus, recording medium using it
TWI399085B (en) Imaging apparatus
US10841488B2 (en) Combined monochrome and chromatic camera sensor
US20040257453A1 (en) Apparatus for and method of interfacing between an image sensor and an image processor
US8798386B2 (en) Method and system for processing image data on a per tile basis in an image sensor pipeline
TW201324488A (en) Pixel data conversion apparatus and method for display with delta panel arrangement
US20080292216A1 (en) Method and system for processing images using variable size tiles
WO2007075066A1 (en) Image processor, apparatus and method for lens shading compensation
US20060204138A1 (en) Image scaling device using a single line memory and a scaling method thereof
WO2007075052A1 (en) Display driver and image prosessing apparatus for interpolating color and control method thereof
US20050285813A1 (en) Display controller, electronic equipment, and image data supplying method
EP4102828A1 (en) Image sensor including image signal processor and operating method of the image sensor
US9058668B2 (en) Method and system for inserting software processing in a hardware image sensor pipeline
US9013626B2 (en) Signal processing circuit of solid-state imaging element, signal processing method of solid-state imaging element, and electronic apparatus
US20060007456A1 (en) Image signal processing device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06835482

Country of ref document: EP

Kind code of ref document: A1