WO1996013807A1

WO1996013807A1 - Method and apparatus for converting from a luminance and color difference color space to a component space

Info

Publication number: WO1996013807A1
Application number: PCT/US1995/012899
Authority: WO
Inventors: Bernd Girod; Staffan Ericsson
Original assignee: Vivo Software, Inc.
Priority date: 1994-10-31
Filing date: 1995-09-28
Publication date: 1996-05-09
Also published as: EP0789891A1

Abstract

Methods of converting color image signals from a representation including luminance signal and a pair of subsampled color difference signals to a representation including three color component signals are disclosed. The methods employ look up tables to obtain improved speed and RAM utilization as compared with prior art methods. Generally, the subsampled color difference signals are applied to one or more look up tables to obtain a color index. The color index and the luminance signal are then applied to a look up table to obtain component signals corresponding to the combination of the luminance and color difference signals.

Description

Method and apparatus for converting from a luminance and color difference color space to a component space.

BACKGROUND OF THE INVENTION

FIELD OF THE INVENTION

The present invention relates generally to the conversion of video information represented by one three-dimensional color space to another three-dimensional color space. More specifically, the present invention relates to the conversion of digitally encoded video information between a representation using one luminance signal and two color difference signals (e.g. YC_RC_B) and a representation using three color component signals (e.g. RGB).

DISCUSSION OF RELATED ART

A video signal generally comprises a representation of a sequence of frames which when displayed at a sufficiently high rate can create in a human observer the illusion of motion. Suitable frame rates used in the television arts and the personal computer arts are generally in a range of 15 to 30 frames per second.

Video signals may be either analog or digital representations of the sequence of frames. For processing in personal computers, analog video signals are first converted to digital form by an analog-to-digital (A/D) conversion process. In a digital video signal, each frame includes a matrix of picture elements or pixels, which are each assigned a luminance value for a monochrome frame and which are each assigned a color value for a color frame. In common color video signal formats, each pixel may be represented by up to three numbers, because color information is a three-dimensional physical attribute. That is, to represent any perceivable color in digital form requires three different numerical quantities.

High quality color images are typically represented by three color component signals, denoted R, G and B, corresponding to the intensities of each of the red, green and blue components of a scene. The red, green and blue component signals fully represent the visual information contained in the scene, because the intensities of red, green and blue form a coordinate system by which any point in three-dimensional color space may be identified. In one format of digital color image signal corresponding to the high quality color image signal discussed above, each pixel is represented by an RGB triplet of bytes (RGB24), wherein each byte is eight bits in length and wherein each of the red, green and blue components of a pixel is represented by one of the bytes of the triplet. In another commonly used format, each pixel is represented by an RGB triplet of five bits, each. This format (RGB 16) occupies only two bytes, but does so at the expense of poorer quantization of the intensities of the red, green and blue component signals. Regardless of the signal format, the overall luminance of a pixel is mathematically related to the individual red, green and blue intensities by the equation:

Y = 0.299R + 0.587G + 0J445 (1)

The International Radio Consultative Committee (CCIR) in Recommendation 601,

Encoding Parameters of Digital Television for Studios, CCIR XVth Plenary Assembly, Document 11/1041-E, Dubrovni , December 11, 1985, pp. 1-10, calls for a representation of color images (YC_RC_B) using digital signals representative of luminance, Y, and two chrominance or color difference components, C_R and C_B, representative of the color signal differences R - Y and B - Y, for each pixel respectively. Recommendation 601 also calls for subsampling the color difference signals by a factor of two-to-one with respect to the luminance signal. That is, each two pixels is represented by two luminance signal values, respectively, and by one pair of color difference signal values. In order to save some space when storing color digital image signals or to reduce the required communication bandwidth when communicating color digital image signals, ISO 11172 MPEG International Standard and ITU-T Recommendation H.261 require spatial subsampling of the color difference signals with respect to the luminance signal by a factor of four-to-one. That is, each frame is divided into two-by-two pixel regions, each two-by-two pixel region represented by four luminance signal values and two color difference signal values. Each pixel is associated with its own luminance signal value, but each group of four pixels is associated with only one color difference signal value for each of the two color difference signals, C_R and C_B.

Thus far, we have only defined the relationship between a luminance signal, Y, and corresponding red, R, green, G, and blue, B, color component signals. However, the entire mapping of the YC_RC_B color space to the RGB color space may be represented by the following system of linear equations:

R = Y + 1.402(C_Λ - 128), G = Y - 0J74(C_Λ - 128) - 03AA(C_B - 128), and (₂)

B = Y + \ .772(C_B - 128); where

Y, C_R and C_B are assumed to lie within a range of [0..255], a color difference value of 128 being neutral. The resulting R, G and B signals are clipped to lie within a range of [0..255] after performing the operations specified by the system of equations (2). The specified ranges correspond to the range of unsigned integers representable by a single, eight-bit byte. The mathematical operations and clipping specified by the system of equations (2) may be carried out in a personal computer or other general purpose or special purpose computer. These operations include six comparisons, four multiplications and six addition operations for each pixel. All of these operations are typical of those core operations performed by a central processing unit (CPU) in a typical personal computer or other general purpose or special purpose computer. However, the computational burden represented by the combination of operations represented by the system of equations (2) would be excessive for many personal computers now in general use. Therefore, one approach to performing this conversion of one color digital image signal has been to use look-up tables (LUTs), held in random access memory (RAM) of the personal computer, to reduce the number of operations required.

The LUTs are addressable by combinations of the luminance and color difference signals. The red and blue tables are each addressed by two of the YC_RC_B signals, while the green table is addressed by all three YC_RC_B signals. Look-up tables capable of representing each of the red and blue components of a pixel, and addressable by the YC_RC_B signals can be shown to require 2¹⁶ bytes or 64 Kbytes of RAM (1 Kbyte = 2¹⁰ bytes = 1024 bytes), when the YC_RC_B signals are each represented by one byte. But, the green LUT can be shown to require 2²⁴ bytes, or 16 Mbytes of RAM (1 Mbyte = 2²⁰ bytes). Thus, a total of over 16 Mbytes of RAM is required, just to hold these LUTs. This requirement is in excess of what is found in many personal computers in general use, which typically have 8 Mbytes or less of RAM.

Therefore, it is a general aim of the present invention to provide a computationally less burdensome method of performing color space conversion than direct, real-time execution of the operations by which the color space conversion is defined. It is a further aim of the present invention to provide such a method without an unduly large requirement for RAM.

SUMMARY OF THE INVENTION The foregoing general aims of the present invention and such other aims as will be apparent to those skilled in the art are achieved as follows. A method according to one aspect of the present invention for converting in a computer an input image signal in which color information is represented as input pixels, each input pixel including a first signal sampled at a first rate and a plurality of second signals sampled at a lower rate than the first rate to an output image signal in which color information is represented as output pixels, each output pixel including a plurality of third signals, the third signals sampled at one rate, includes the following steps: precomputing and storing in a memory in the computer a first lookup table containing index values representing possible combinations of the plurality of second signals, each index value stored at an address corresponding to the possible combinations of the plurality of second signals; precomputing and storing in a memory in the computer a secon lookup table containing output pixels including the plurality of third signals, each output pixel stored at an address corresponding to a possible combination of the first signal and the second signal; after the steps of precomputing and storing, receiving a sequence of the input pixels; addressing the first lookup table with the second signals of each input pixel received, to obtain an index value; addressing the second lookup table with the index value obtained and the first signal of each input pixel received, to obtain an output pixel; and inserting the output pixel obtained into the output image signal.

In accordance with a second aspect of the present invention, the method may further include before the step of addressing the second lookup table, the steps of: adding noise or a dither pattern in a controlled manner to the first signal; and uniformly quantizing the first signal. Uniform quantization means that the steps size between quantization values is constant.

In accordance with yet a third aspect of the present invention, the step of precomputing and storing the first lookup table of the first aspect of the invention may include steps of: generating a first sub-table corresponding to one of the second signals; and generating a second sub-table corresponding to another of the second signals.

In each of the methods disclosed herein as examples of embodiments of the present invention, the lookup tables are described as being held in random access memory of a computer. However, holding the lookup tables in random access memory is but one example of this feature of the invention. In embodiments of the invention implemented using special purpose hardware, the lookup tables may be held in a read only memory, a non-volatile random access memory, or any other suitable type of memory.

BRIEF DESCRIPTION OF THE DRAWING In the figures, in which like elements are given like reference designations: Fig. 1 is a general block diagram of a personal computer suitable for use in connection with embodiments of the present invention;

Fig. 2 is a signal flow diagram of fast color space conversion according to one aspect of the present invention; Fig. 3 is a signal flow diagram of fast color space conversion according to a second aspect of the present invention; and

Fig. 4 is a signal flow diagram of fast color space conversion according to a third aspect of the present invention.

DETAILED DESCRIPTION Some embodiments and variations disclosed herein are intended to be implemented in the form of software running on a personal computer (PC), configured substantially as shown in Fig. 1. However, the specific hardware configuration is not a limitation of the present invention. Indeed, the invention could be embodied in a sequence of instructions executed by other types of general-purpose computers, parallel processing computers or embodied in special-purpose integrated circuits. However, the PC configuration of Fig. 1 illustrates general hardware parameters suitable for practicing the invention, without unduly complicating the detailed explanation thereof. Referring to Fig. 1, the PC 101 includes a processing unit, generally a central processing unit (CPU) 103, memory 105, input/output (I/O) system 107 and mass storage 109. These are now briefly discussed, before turning to the software of the present invention.

The CPU 103 should preferably be of a type having an internal precision and register size of at least sixteen bits, such as the Intel X86 family or Motorola 680X0 devices. The minimum processing speed is not a critical parameter, but may be chosen by the skilled designer. Naturally, differences in processing speed will have a direct effect on the performance of different systems. In other embodiments the processing unit could be an ALU, a digital signal processor or other suitable device. A suitable device is a processing unit capable of performing the functions described below, under software program control. Having sufficient quantities of memory 105 available in the PC 101 will also have a direct impact on system performance. It is contemplated that the invention may be practiced using operating system/interface configurations such as Microsoft MS-DOS/Windows or IBM OS/2, both of which require several Mbytes of RAM (1 Mbyte = 220 bytes). The incremental RAM required by embodiments of the present invention is small. Data and digital signals of various kinds may be passed into or out of PC 101 through I/O system 107. An extremely wide variety of devices may be included in I/O system 107. The devices of I/O system 107 allow digital image signals to be received into or transmitted out of PC 101. For example, I/O system 107 may include a modem 108a for sending digital signals or receiving digital signals or data from remote devices and computers. Such digital signals could include digital image signals. Another possible I/O device is a frame grabber 108b, which could be used to introduce into PC 101 television or video images which have been converted to digital form. Of course, I/O system 107 may include such conventional I O devices as a monitor 108c, keyboard 108d and printer (not shown), which may be desirable.

Programs, data and digital signals which need to be stored on a more permanent basis than permitted by the RAM portion of memory 105 are stored in a mass storage device 109. The mass storage device 109 may be a hard disk, an optical disk, a CD-ROM or any permanent storage device having similar characteristics to the devices mentioned. Image signals are received into the PC 101 through one of the devices of I/O system 107.

Once inside the PC 101, the image signals will have been converted to digital form. That is, they are digital image signals. In this embodiment, it is presumed that the digital image signals are to be either stored in a mass storage device 109 or transmitted to another location via one of the I/O devices of the I/O system 107. One example of this is where the PC 101 is part of a conferencing network in which images are to be exchanged, stored and displayed on a plurality of such machines. Of course, it will be understood that the above-described architecture of PC 101 is generic and need not be limited to personal computer embodiments. The architecture, illustrated in Fig. 1 is intended to represent all similar architectures including a processing unit, memory, I/O system, etc. A first aspect of the present invention is now described in connection with Fig. 2. This aspect of the invention is fairly general, the second and third aspects of the invention including additional features which may be useful in some applications.

Conversion of YC_RC_B color digital image signals to RGB color digital image signals is now described. The characteristics of YC_RC_B color digital image signals and RGB color digital image signals discussed above are assumed. However, the method disclosed may be applied to any color space conversion wherein image signals in the source color space include one fully sampled signal and two subsampled signals, while image signals in the target color space include three fully sampled signals.

The signal flow diagram of Fig. 2 includes two LUTs, a color index LUT 201 and an RGB LUT 203. The LUTs are cascaded as follows. The color difference signals C_R and C_B are applied by the processing unit as addresses to the color index LUT 201 , which produces a color index, I. The color index I and the luminance Y are then applied by the processing unit to the RGB LUT 203, to produce three color component signals R, G and B. The operation of this method is now examined in further detail.

The color index I represents each input combination of C_R and C_B, but using fewer bits. For example C_R and C_B are each defined to be eight-bit bytes. But, the color index, I, could also be an eight-bit byte, representing 256 different combinations of C_R and C_B. A LUT such as the color index LUT 201 may be characterized as a vector quantizer. Vector quantizers are well- known in the art, and may be designed by a variety of known methods. For example, a statistical design method suitable for this application is the Generalized Lloyd Algorithm, or LBG algorithm, described in Y. Linde, et al.. An algorithm for vector quantizer design, IEEE Transactions on Communications, V. COM-28(l), pp. 84-95, January 1980.

Since the C_R and C_B signals are subsampled relative to the Y signal, as discussed above, the color index I is looked up only once for each four pixels. However, the R, G and B signals are looked up for each pixel, based on the sampling rate of the Y signal. This will be of greater significance in the later described aspects of the present invention. The RGB LUT 203 may combine several normally-distinct operations into one, simple table lookup. For example, the values stored in the RGB LUT 203 may already be clipped. Thus that step need not be separately performed. In another example, the values stored in the RGB LUT 203 may correspond to addresses into a color palette or other color display format. Thus, although red, green and blue signals may not be directly produced, signals representative of appropriate mixtures of red, green and blue intensities in a format understood to an underlying display system may be produced.

The tables 201 and 203 according to this aspect of the present invention are of very limited size, relative to the prior art. If the color index I is an eight-bit byte, and the color difference signals C_R and C_B are eight-bit bytes, then the color index LUT 201 has 64 K entries of one byte each, requiring 64 Kbytes of RAM. If RGB24 signals and an eight-bit Y signal are assumed, as above, then the RGB LUT 203 has 64 K entries of three bytes each, requiring 192 Kbytes of RAM. The total storage required by this aspect of the present invention is therefore 256 Kbytes of RAM.

Further reduction in the size of the tables 201 and 203 are possible in accordance with the second aspect of the present invention, described now in connection with Fig. 3. The size of RGB LUT 203 is a function of the number of bits by which it is addressed. By reducing the accuracy of the luminance signal Y by passing the luminance signal Y through a quantization block 301, for example to six bits of resolution, the RGB LUT 203 may be made smaller. However, this size advantage comes at a price of undesirable contouring effects in the resulting picture. These appear as steps in the picture brightness where smooth brightness transitions should appear. In order to avoid such contouring effects, the luminance signal Y may have noise or a dither pattern D added to it before passing it through quantization block 301. Dithering will also break up luminance contouring which occurs in some display formats, such as RGB 16, in which each color component signal is represented by only five bits. In one embodiment, a four- pixel dither pattern D was used, as follows:

0 4 6 2

Each number in the dither pattern D represents a value to be added to the luminance signal Y representing the pixel at the corresponding location in each two-by-two pixel region. This dither pattern D assumes that uniform quantization of the luminance is performed, for example by simply lopping off some low-order bits from the numeric representation. Such a crude quantization method causes a predictable shift in the mean of the luminance signal, which is compensated for by an equal and opposite mean shift in the dither pattern. The above dither pattern is optimized for quantization which reduces the luminance signal Y from an eight bit signal to a six bit signal. Finally, the size of the color index LUT 201 may be even further reduced, in accordance with the third aspect of the present invention, described now in connection with Fig. 4. The reduction is achieved by dividing the color index LUT into a C_u LUT 401 and a C_B LUT 403. When the CPU addresses each of the C_R LUT 401 and the C_B LUT 403 with the eight bit signals C_R and C_B, respectively, the result of the table look up is a non-uniform scalar quantization to a five bit representation of the corresponding color difference signal. The design of scalar quantizer LUTs is well-known in the art. For example, the LBG algorithm referred to above may be used to design the LUTs 401 and 403. Each component of the color index may be represented by five bits, resulting in a ten bit color index. Assuming the luminance, Y, to be represented by eight bits, as before, the total number of bits addressing the RGB table 203 is eighteen. Therefore, the RGB table 203 contains 256 K entries, each of which may be two bytes for RGB 16 format image signals. Thus, the total size of the RGB table 203 is 512 Kbytes. If the five bit color index components each occupy one byte, then each color index component LUT contains 256 bytes, or a total of 0.5 Kbytes for both. The total space occupied by all three LUTs is thus only 512.5 Kbytes for embodiments according to this aspect of the invention. However, the color index according to this aspect of the invention has ten bits of resolution as compared to only eight bits of resolution in the earlier described embodiments.

In one actual embodiment of the third aspect of the present invention, the output signals are represented in RGB 16 format. The input signals are a six bit wide luminance signal, Y, and two eight bit wide color difference signals, C_R and C_B. The color difference signals are quantized to two five bit wide component color index signals. In this embodiment, the combined size of the color index LUT is 0.5 Kbytes, as above. However, the RGB LUT 203 contains only 64 K entries, each of which is two bytes for RGB 16 format output. Thus, the total size of the RGB LUT 203 is 128 Kbytes. The combined total size of the three LUTs is 128.5 Kbytes. The present invention has been described and illustrated in connection with a number of specific examples. However, the foregoing description is not to be taken as limiting. Numerous variations and modifications contemplated as within the scope of the invention should now occur to those skilled in the art. Therefore, the scope of the invention is not to be limited except by the scope of the appended claims.

Claims

1. A method of converting in a computer an input image signal in which color information is represented as input pixels, each input pixel including a first signal sampled at a first rate and a plurality of second signals sampled at a lower rate than the first rate to an output image signal in which color information is represented as output pixels, each output pixel including a plurality of third signals, the third signals sampled at one rate, the method comprising the steps of: precomputing and storing in a memory in the computer a first lookup table containing index values representing possible combinations of the plurality of second signals, each index value stored at an address corresponding to the possible combinations of the plurality of second signals; precomputing and storing in a memory in the computer a second lookup table containing output pixels including the plurality of third signals, each output pixel stored at an address corresponding to a possible combination of the first signal and the second signal; after the steps of precomputing and storing, receiving a sequence of the input pixels; addressing the first lookup table with the second signals of each input pixel received, to obtain an index value; addressing the second lookup table with the index value obtained and the first signal of each input pixel received, to obtain an output pixel; and inserting the output pixel obtained into the output image signal

2. The method of claim 1, further including before the step of addressing the second lookup table, the steps of: adding noise in a controlled manner to the first signal; and uniformly quantizing the first signal.

3. The method of claim 2, wherein the step of adding noise in a controlled manner further comprises: adding a dither pattern to the first signal

4. The method of claim 1, wherein the first signal is a luminance signal.

5. The method of claim 1, wherein the plurality of second signals are a plurality of chrominance signals.

6. The method of claim 1, wherein the step of addressing the first lookup table is performed only once for each sample of the second signals and the step of addressing the second lookup table is performed once for each pixel.

7. The method of claim 1, wherein the step of precomputing and storing the first lookup table of the first aspect of the invention may include steps of: generating a first sub-table corresponding to one of the second signals; and generating a second sub-table corresponding to another of the second signals.

8. Apparatus for converting an input image signal to an output image signal, the input image signal comprising a sequence of input pixels in which color information is represented by sample values of a first signal sampled at a first rate and sample values of a plurality of second signals sampled at a lower rate than the first rate, and the output image signal comprising a sequence of output pixels in which color information is represented by a plurality of third signal sample values, the third signals sampled at one rate, comprising a memory containing a first lookup table of color index values, each color index value stored at an address corresponding to a combination of sample values of the plurality of second signals; a memory containing a second lookup table of output pixels, each output pixel stored at an address corresponding to a combination of one of the color index values and a sample value of the first signal; and means for addressing the first lookup table at the lower rate using the plurality of second signals, producing a color index signal, and for addressing the second lookup table at the first rate using the first signal and the color index signal, producing output pixels.

9. The apparatus of claim 8, further comprising: a quantizer connected to receive a quantizer input signal and including means for quantizing the quantizer input signal to a predefined plurality of quantization representative values and including an output of the first signal.

10. The apparatus of claim 9, further comprising: an adder having two inputs and an output, the output carrying the quantizer input signal and connected to the quantizer, a first input connected to a noise source and a second input connected to a raw input signal.

11. The apparatus of claim 10, wherein differences between sequential ones of the predefined plurality of quantization representative values are uniform.

12. The apparatus of claim 11, wherein the noise source produces a signal representative of a predefined dither pattern.

13. The apparatus of claim 8, wherein the color index values are formed of a concatenation of a first part and a second part, and wherein the memory containing the first lookup table comprises: a memory containing first partial table, receiving as addresses a first of the plurality of second signals, and when addressed producing as an output the first part of the color index values; and a memory containing second partial table, receiving as addresses a second of the plurality of second signals, and when addressed producing as an output the second part of the color index values; the color index signal formed by concatenation of the first part and the second part of the color index values.

14. The apparatus of claim 8, wherein the color index values are formed of a logical combination of a first part and a second part, and wherein the memory containing the first lookup table comprises: a memory containing first partial table, receiving as addresses a first of the plurality of second signals, and when addressed producing an output representative of the first part of the color index values; and a memory containing second partial table, receiving as addresses a second of the plurality of second signals, and when addressed producing an output representative of the second part of the color index values; the color index signal formed by logical combination of the first part and the second part of the color index values.