KR100723046B1 - Display driver and image prosessing apparatus for interpolating color and control method thereof - Google Patents

Abstract

According to an aspect of the present invention, there is provided a color interpolation display driver, an image processing apparatus, and a control method thereof, and a display driver for outputting an image captured by an image sensor including a plurality of unit pixels to a display A system interface for receiving n-bit Bayer RGB data detected by the image sensor via a data bus, wherein the Bayer RGB data includes at least one of R (Red), G (Green) and B One containing -; A graphics memory for storing the received n bits of Bayer RGB data; And a color interpolator for interpolating n-bit Bayer RGB data stored in the graphic memory into 3n-bit actual color data. According to the present invention, there is an advantage that system performance can be improved by color interpolation in a display driver.

Color, interpolation, Bayer RGB data, display driver

Description

TECHNICAL FIELD [0001] The present invention relates to a color interpolation display driver, an image processing apparatus, and a control method thereof.

1 shows a color filter array according to the prior art;

2 is a circuit diagram of a display driver according to the prior art;

FIG. 3 is a diagram illustrating a nearest neighborhood replication process. FIG.

4 is a diagram for explaining an edge detection interpolation process;

FIGS. 5A and 5B are diagrams for explaining a process of determining G5 in a pixel B5. FIG.

6 is a view showing each pattern type in pattern recognition interpolation;

FIGS. 7A and 7B are diagrams for explaining a pattern recognition interpolation process; FIG.

8 is a diagram showing an image before and after interpolation;

9 is a block diagram of an image processing apparatus according to a preferred embodiment of the present invention.

10 is a block diagram of a display driver in accordance with a preferred embodiment of the present invention;

The present invention relates to a color interpolation display driver, an image processing apparatus, and a control method thereof, and more particularly, to a color interpolation display driver, an image processing apparatus, and a control method thereof capable of improving system performance through efficient color interpolation will be.

Recently, the digital camera market is rapidly expanding due to the high integration of CCD (Charged-Coupled Device) and CMOS (Complementary Metal-Oxide Semiconductor) and image processing technology, and digital cameras It is becoming common to provide functions.

The image processing apparatus captures an image related to a subject using an image sensor such as a CCD and a CMOS.

The image sensor uses red (R), green (G), and blue (B) to represent all of the colors present in nature. Typical image sensors use a color filter array Color Filter Array (CFA).

The image sensor includes a plurality of unit pixels for outputting a light signal relating to a subject as an electric signal. In the case of using the CFA as described above, one unit pixel of the image sensor has a unit pixel for each of R, G, Value.

The R, G and B values detected in each unit pixel are converted into digital signals by a means such as DAC (Digital Analog Converter) included in the image sensor and output to an image signal processor (ISP) or a camera signal processor Signal Processor, CCP).

The actual color of the object can be represented by a combination of R, G, and B. The data detected by each unit pixel of the image sensor includes only information about one of R, G, and B Therefore, conventionally, an actual color interpolation process is performed so that one unit pixel includes R, G, and B information using the R, G, and B data detected by the ISP or the CCP in each unit pixel.

The color interpolated data in the ISP or CCP is transmitted to the display driver via the back-end chip (not shown).

2, a display driver according to the prior art receives data interpolated in real color through the data bus 200 and supplies it to the memory 202. The display driver, And outputs it to a display unit (not shown).

Since the interpolated data in the actual color includes all the R, G and B data for each unit pixel, the amount of information is three times larger than that of the Bayer RGB data. According to the related art, There is a problem that the clock speed must be increased in order to transmit the actual color interpolated data from the ISP to the back-end chip and from the back-end chip to the display driver.

When the clock speed is high, the internal chip design becomes more difficult, and the power consumption increases by three times during the signal transmission between the chip and the chip, which increases the possibility of malfunction due to EMI (Electromagnetic Interference).

In addition, since the memory included in the display driver must have a memory size three times higher than the data detected by the image sensor in order to store the actual color interpolated data, the size of the display driver becomes large, There is a problem in that a problem occurs.

In addition, when the data bus is composed of many lines, for example, when the resolution is set to 6-bit resolution, the data bus is composed of 18 lines, which is three times larger than the data bus.

In order to solve the problems of the related art as described above, the present invention proposes a color interpolation display driver, an image processing apparatus and a control method thereof that can lower the clock speed of an image processing apparatus.

It is another object of the present invention to provide a color interpolation display driver, an image processing apparatus, and a control method thereof, which can prevent a malfunction due to EMI or the like by reducing power consumption in an internal signal transfer process of an image processing apparatus.

It is another object of the present invention to provide a color interpolation display driver, an image processing apparatus, and a control method thereof that can reduce the memory size of a display driver.

Another object of the present invention is to provide a color interpolation display driver, an image processing apparatus, and a control method thereof that can reduce the size of an image processing apparatus by reducing the size of a display driver and reducing data bus lines.

In order to achieve the above object, according to a preferred embodiment of the present invention, there is provided a display driver for outputting an image captured by an image sensor including a plurality of unit pixels to a display, Wherein the Bayer RGB data includes one of R (Red), G (Green), and B (Blue) data collected by the unit pixel; A graphics memory for storing the received n bits of Bayer RGB data; And a color interpolator for interpolating n-bit Bayer RGB data stored in the graphic memory into 3n-bit actual color data.

The display driver according to the present invention may further include a register for controlling the read / write of data between the system interface and the graphic memory, including address information of the graphic memory.

The system interface may further include a write latch for writing the received Bayer RGB data to the graphic memory.

Preferably, the data bus may have n lines.

Also, the display may be any one of an LCD, an organic EL, a PDP, a CRT, and a FED.

According to another aspect of the present invention, there is provided a method of controlling a display driver for outputting an image captured by an image sensor including a plurality of unit pixels to a display unit, the method comprising the steps of: (a) Receiving Bayer RGB data, wherein the Bayer RGB data includes one of R (Red), G (Green), and B (Blue) data collected by the unit pixel; (b) storing the received n bit Bayer RGB data in a graphics memory; And (c) interpolating n-bit Bayer RGB data stored in the graphics memory into 3n-bit actual color data.

According to another aspect of the present invention, there is provided an image sensor including a plurality of unit pixels and detecting n-bit Bayer RGB data, wherein the Bayer RGB data includes R (Red), G (Green) Blue) data; A display driver for receiving and storing n-bit Bayer RGB data detected by the image sensor through a data bus, and interpolating the stored n-bit Bayer RGB data into 3n-bit actual color data; And a control unit for controlling the image sensor and the display driver.

Hereinafter, preferred embodiments of a color interpolation display driver, an image processing apparatus, and a control method thereof according to the present invention will be described in detail with reference to the accompanying drawings.

The present invention improves the system processing performance by improving the method of interpolating the Bayer RGB data detected by the image sensor into actual colors differently from the conventional method.

As described above, the actual color interpolation is to include all information about RGB for one unit pixel by using R, G, and B data for each unit pixel of the image sensor. For example, The R and B values of the unit pixel are generated by referring to the R and B values of the peripheral unit pixel.

This actual color interpolation can be performed in various ways, and FIGS. 3 to 7 are diagrams for explaining various interpolation methods.

The actual color interpolation process will be described in detail below with reference to the respective drawings.

FIG. 3 is a diagram showing a nearest neighborhood replication (nearest neighbor interpolation) process, and FIG. 3 is a view selectively showing a pixel including a G value when compared with the color filter array of FIG.

The nearest interpolation is a method of determining R, G, and B values that are not present in a specific pixel through neighboring pixel values.

For convenience of explanation, RGB data corresponding to the n-th pixel is represented by R (n), G (n) and B (n).

Referring to FIG. 3, G8, G11, G14, and G18 do not exist among nine pixels. The nearest interpolation method determines each pixel value as a neighboring pixel value. After interpolation, G8 is determined as G7 and G12 G11, G14, and G18 are determined as G13 and G17, respectively.

As another interpolation method, there is a bilinear interpolation method, which is a method of determining R, G, and B which are not present in a specific pixel as an average value of neighboring pixels.

Referring also to FIG. 3, G8 is determined by (G3 + G7 + G9 + G13) / 4 through pixel values on the top and bottom and on the left and right.

Another method of interpolation is the Smooth Hue Transition interpolation method.

The smooth Hue transition interpolation method first determines the Green value as a bilinear interpolation method, and then uses the B / G and R / G values as the Blue and Red values.

B8 = G7 / 2 * (B6 / G6 + B8 / G8) and B13 = G13 / 2 * (B8 / G8 + B18 / G18) , B12 = G12 / 4 * (B6 / G6 + B8 / G8 + B16 / G16 + B18 / G18).

As another interpolation method, there is an edge detection interpolation process, and the Green value is obtained at the B8 pixel in FIG.

The edge interpolation first determines the amount of change in the horizontal and vertical pixels of the pixel to be determined through the following equation (1).

Then, G8 is determined using the value of the smaller change amount through the following discriminant.

Another edge detection interpolation method is to use 5 x 5 interpolation.

As shown in FIG. 4, for example, in the case of determining G5 in the B5 pixel, edge detection interpolation defines the vertical and horizontal variation amounts by the following equation (2).

As another interpolation method, there is a linear interpolation method which performs Laplacian second-order color correction together.

Referring to FIGS. 5A and 5B, for example, G5 can be determined through the following equation (3) and the discrimination equation for determining G5 in the pixel B5.

The discriminant equation is as follows.

Next, there are three ways to obtain Red and Blue values at different pixels depending on the position.

1) R4 = (R1 + R7) / 2 + (G4-G1 + G4-G7) / 4

2) R2 = (R1 + R3) / 2 + (G2-G1 + G2-G3) / 4

3) In order to determine the Red value at B5 pixel, the following discriminant can be used after defining the amount of diagonal change by the following expression (4).

The discriminant equation is as follows.

Another method of interpolation is pattern recognition interpolation.

In FIG. 2, the case of obtaining the Green value at the R12 pixel will be described as an example.

In the case of determining the Green value at R12 pixels, if H is larger than the average and L is defined as smaller than the average, the pattern can be classified as shown in Fig.

In FIG. 6, (a) and (d) can be defined as an edge pattern, (b) as a stripe pattern, and (c) as a corner pattern.

If it is an edge pattern ((a) and (d)), G12 is determined via equation (5).

G12 = median {G7, G11, G13, G17} = (B + C) / 2

Where B and C are the second and third largest numbers of G7, G11, G13, and G17

Next, in the case of a stripe pattern, referring to FIG. Is determined by Equation (6) below.

Where M is SUM (G) / 4, S is SUM (X) / 8, and the function 'clip' is a function such that the resultant value is between B and C as in Equation 5 above.

Finally, referring to FIG. 7B in the case of a corner pattern, G ?? Is determined by the above equation (6) is the same as the stripe pattern, but the difference is that M is median {H's, L's} and S is determined as SUM (X) / 4.

The edge detection interpolation algorithm is as follows.

m = size (in, 1); n = size (in, 2); inR = in (:,:, 1); inG = in (:,:, 2); inB = in (:,:, 3); out = in; outR = inR; outG = inG; outB = inB; (I, j-2) + inB (i, j + 2) (2) )) - inB (i, j)); delta_V = abs (1/2 * (inB (i-2, j) + inB (i + 2, j) - inB (i, j)); (i, j-1) + inG (i, j + 1)); if delta_H <delta_V, outG (i, j) = 1/2 * elseif delta_H> delta_V, outG (i, j) = 1/2 * (inG (i-1, j) + inG (i + 1, j)); (i, j + 1) + inG (i, j) + inG (i, j) = 1/4 * (inG (i, j-1) + inG (i, j-2) + inR (i, j + 2) (i, j-2) )) - inR (i, j)); delta_V = abs (1/2 * (inR (i-2, j) + inR (i + 2, j) - inR (i, j)); (i, j-1) + inG (i, j + 1)); if delta_H <delta_V, outG (i, j) = 1/2 * elseif delta_H> delta_V, outG (i, j) = 1/2 * (inG (i-1, j) + inG (i + 1, j)); (i, j + 1) + inG (i, j) + inG (i, j) = 1/4 * (inG (i, j-1) + inG end end end outG = round (outG); ind = find (outG >255); outG (ind) = 255;

(I, 2: n-1) = R channel for i = 1: 2: m-1, outR 2: n-2) -outG (i, 2: 2: n-2) + inR (i, 4: 2: n) -outG (i, 4: 2: n); 2: n: inR (i-2: 2: n) = outG (i, 2: ) -outG (i-1, 2: 2: n) + inR (i + 1, 2: 2: n) -outG (i + 1; 2: n-2) -outG (i, 3: 2: n-1) = outG 2: n-1) + inR (i-1,4: 2: n) -outG ) -outG (i + 1, 2: 2: n-2) + inR (i + 1, 4: 2: n- end outR = round (outR); ind = find (outR >255); outR (ind) = 255; ind = find (outR <0); outR (ind) = 0; 2: 2: 2B (i, 2: 2: n-2) = B channel for i = 2: 2: m, outB : n-3) -outG (i, 1: 2: n-3) + inB (i, 3: 2: n-1) -outG (i, 3: 2: n- (i, 1: 2: n-1) + 1/2 * inB (i-1, 1: 2: n-1) -outG (i-1,1: 2: n-1) + inB (i + 1,1: One)); 2: n-2) = outG (i, 2: 2: n-2) 1: 2: n-3) + inB (i-1, 3: 2: n-1) 1: -outG (i + 1, 1: 2: n-3) + inB ); end outB = round (outB); ind = find (outB >255); outB (ind) = 255; ind = find (outB <0); outB (ind) = 0; out (:,:, 1) = outR; out (:,:, 2) = outG; out (:,:, 3) = outB;

FIG. 8 is a diagram showing an image before and after interpolation. FIG. 8 is a view showing an original image of a captured image, an image obtained by Laplacian interpolation and edge sensing interpolation. Table 1 below shows mean square error (MSE) and computational cost at each interpolation.

Interpolation type MSE Computed load Laplacian color correction 3.1 14.3 Edge Detection II 3.8 6 Pattern recognition 5.3 5.3 By Linear 5.7 2 Smooth Hue Transition 5.7 4

Referring to Table 1, it can be seen that edge sensing II and laplacian secondary color correction should be appropriately used depending on image quality and calculation load.

Although various interpolation methods are performed in an image processing apparatus, conventionally, image interpolation in actual colors is performed in a configuration such as ISP or CCP. However, according to a preferred embodiment of the present invention, a display driver receives Bayer RGB data And performs an interpolation process to the actual color.

9, an image processing apparatus according to the present invention includes an image sensor 10, an ISP 20, a back-end chip (not shown) 30, a display driver 40, and a display 50.

The image sensor 10 includes a color filter array and a plurality of unit pixels, and one unit pixel detects one of R, G, and B data.

The data detected in each unit pixel is referred to as Bayer RGB data, and the Bayer RGB data is input to the ISP 20 and subjected to a predetermined image processing process.

In this case, the image processing is performed by gamma correction for correcting the gamma value set for the display 50, a histogram is generated for the pixel values output from the image sensor, and when the Max value is small, the Max value is set to 255 And may include at least one of Histogram Equalization, Edge Enhancement, Subsampling, Auto Exposure, and Auto White Balance processes that evenly stretch the image. have.

Through this image processing process, a more improved image of the subject can be obtained.

The back-end chip 30 functions to control various means constituting the image processing apparatus. The back-end chip 30 may include a control unit such as a CPU (Central Processing Unit) or an MCU (Micro Controller Unit) A built-in memory for storing compressed data, and the like.

In addition, the back-end chip 30 may incorporate a means such as a CCP so that the image processing process described above may be performed in the back-end chip 30. [

Conventionally, in an ISP or a back-end chip, Bayer RGB data is interpolated in real color and delivered to the display driver. However, when the actual color interpolation process is performed in front of the display driver, the data amount in the transmission process to the display driver is high A clock speed is required, and the size of the graphic memory of the display driver is increased.

For example, if the display has a 6-bit resolution and the display size is 176 * 240, according to the prior art, the graphic memory size required for the display driver is 6x3x176x240 / 8bit, 90,040 bytes.

Thus, conventionally, the size of the graphic memory of the display driver is required to be very large, and the size of the display is further increased as the resolution of the display is increased.

In order to solve such a problem, the image processing apparatus according to the present invention provides a display driver 40 having a color interpolation unit 60. [

The display driver 40 according to the present invention receives Bayer RGB data that is not actually color interpolated from the ISP 20 or the backend chip 30. [

Even if the ISP 20 or the back-end chip 30 actually performs the color interpolation process, the means 20 and 30 convert the actual color-interpolated data into the Bayer RGB data again, .

10 is a block diagram of a display driver according to an exemplary embodiment of the present invention. Referring to FIG. 10, a system interface 100 included in a display driver receives Bayer RGB data through a data bus .

As compared with the circuit diagram of FIG. 2, conventionally, for example, when the actual color interpolation is performed in the ISP or the back-end chip in the case of having a resolution of 6 bits, image data is transmitted in 18 bits, Similarly, it consists of 18 lines.

However, according to the present invention, since the interpolation from the previous stage of the display driver 40 to the actual color is not performed, if the resolution is 6 bits, a data bus composed of only 6 lines can be provided.

That is, when each value of RGB is represented by n bits, the data bus has only n lines.

Bayer RGB data received in the system interface 100 is stored in the graphic memory 108 through the control of the register 102 and the writing process of the write latch circuit 104.

Here, the register may include an address register for storing address information of the graphic memory, a counter for counting the size of data transmitted in data transmission, and a control register for outputting a control signal for writing and reading data.

The display driver includes a read latch circuit (104) and a read latch circuit (106) for reading data stored in the graphic memory (108).

Meanwhile, the Bayer RGB data stored in the graphic memory 108 is interpolated into the actual color through the color interpolation unit 110, and the interpolated data is output to the display 50 via the image output unit 112.

Here, the display 50 may be any one of a liquid crystal device (LCD), an organic electroluminescence display (EL), a plasma display panel (PDP), a cathode ray tube (CRT), and a field emission display , But are not necessarily limited thereto.

As described above, when the resolution is set to 6 bits, the size of the graphics memory becomes 6x176x240 / 8 = 31,680 bytes in the case of performing the interpolation from the display driver to the actual color, / 3.

In addition, since the data that is reduced by 1/3 of that of the conventional display driver is transmitted, there is no need to increase the clock speed, and the device performance can be remarkably improved. Even if the resolution is set to be higher than 6 bits, only 8x176x240 / 8 = 42,240 bytes, the color can be further segmented and displayed even in the reduced graphics memory size.

It will be apparent to those skilled in the relevant art that various modifications, additions and substitutions are possible, without departing from the spirit and scope of the invention as defined by the appended claims. The appended claims are to be considered as falling within the scope of the following claims.

As described above, according to the present invention, there is an advantage that the amount of data to be transmitted to the display driver is reduced and the clock speed of the image processing apparatus can be lowered.

In addition, according to the present invention, it is possible to prevent malfunction due to EMI or the like by reducing power consumption in an internal signal transfer process of the image processing apparatus.

Further, according to the present invention, there is an advantage in that the memory size of the display driver can be reduced.

Further, according to the present invention, there is an advantage in that the size of the image processing apparatus can be reduced by reducing the data bus line.

Claims (11)

A display driver for outputting an image captured by an image sensor including a plurality of unit pixels to a display, A system interface for receiving n-bit Bayer RGB data detected by the image sensor via a data bus, the Bayer RGB data being data detected by each unit pixel, each unit pixel including R (Red), G (Green) And B (Blue) data; A graphics memory for storing the received n bits of Bayer RGB data; And And a color interpolator for interpolating n-bit Bayer RGB data stored in the graphic memory into 3n-bit actual color data. The method according to claim 1, Further comprising: a register for storing address information of the graphic memory and controlling data read / write between the system interface and the graphic memory. The method according to claim 1, And a write latch for writing the received Bayer RGB data into the graphics memory at the system interface. The method according to claim 1, The data bus having n lines. Claim 5 has been abandoned due to the setting registration fee. The method according to claim 1, Wherein the display is one of an LCD, an organic EL, a PDP, a CRT, and a FED. A method of controlling a display driver for outputting an image captured by an image sensor including a plurality of unit pixels to a display unit, (a) receiving n-bit Bayer RGB data detected by the image sensor through a data bus, wherein the Bayer RGB data is data detected by each unit pixel, and each unit pixel includes R (Red), G Green) and B (Blue) data; (b) storing the received n bit Bayer RGB data in a graphics memory; And (c) interpolating n-bit Bayer RGB data stored in the graphic memory into 3n-bit actual color data. The method according to claim 6, Wherein the step (a) is for receiving Bayer RGB data through n lines of data buses. An image sensor for detecting n-bit Bayer RGB data, the Bayer RGB data being data detected by each unit pixel, wherein each unit pixel includes R (Red), G (Green), and B (Blue) data; A display driver for receiving and storing n-bit Bayer RGB data detected by the image sensor through a data bus, and interpolating the stored n-bit Bayer RGB data into 3n-bit actual color data; And And a control unit for controlling the image sensor and the display driver. 9. The method of claim 8, The display driver, A graphics memory for storing the received n bits of Bayer RGB data; And And a color interpolator for interpolating n-bit Bayer RGB data stored in the graphic memory into 3n-bit actual color data. 9. The method of claim 8, Wherein the data bus has n lines. delete

Images