Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/46—Colour picture communication systems
  • H04N1/56—Processing of colour picture signals
  • H04N1/60—Colour correction or control
  • H04N1/6016—Conversion to subtractive colour signals
  • H04N1/6019—Conversion to subtractive colour signals using look-up tables
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/46—Colour picture communication systems
  • H04N1/64—Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor
  • H04N1/646—Transmitting or storing colour television type signals, e.g. PAL, Lab; Their conversion into additive or subtractive colour signals or vice versa therefor
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N9/00—Details of colour television systems
  • H04N9/64—Circuits for processing colour signals
  • H04N9/67—Circuits for processing colour signals for matrixing

Definitions

• Computer monitors emit color within a color space comprising RGB (red, green, blue) light. Although all colors of the visible spectrum can be produced by merging red, green and blue light, monitors are capable of displaying only a limited gamut (i.e., range) of the visible spectrum. Each pixel presented in the RGB format will include a separate value within a range of 0 to 255 for each of R, G, and B to produce a pixel. However, computers may also emit color within a variety of other color spaces.
• RGB red, green, blue
• another color space may comprise data consisting of luminance (Y), chrominance of the blue to yellow color content (U or C b ), and chrominance of the red to cyan color content (V or CV).
• RGB luminance
• U or C b chrominance of the blue to yellow color content
• V or CV red to cyan color content
• pixels in the YUV format are also comprised of separate values for each of Y, U, and V.
• the ranges for each value of R, G, and B do not correspond directly to the ranges for Y, U, and V.
• the range of values for Y is 16 to 235, while the ranges for both U and V is 16 to 239. Therefore, for a computer to properly display YUV video content in RGB, the YUV values for each pixel must be converted to corresponding RGB values.
• each possible component output R, G, and B may be pre- computed for all values of each possible component input Y, U, and V.
• Each contribution of Y, U, and V input may then be loaded into a register and added in parallel, without overflow, resulting in a computationally inexpensive RGB output from a YUV input.
• G, and B are retrieved from pre-computed tables.
• the YUV contributions for each value of R, G, and B are packed into three data elements and added together in parallel, resulting in a value for an RGB output.
• Fig. 1 may be an illustration of a computer that implements a method of converting YUV formatted data to RGB formatted data
• Fig. 2 may be an illustration of a method of converting YUV video data to
• Fig. 3 may be an illustration of pre-computed tables for use with a method of converting YUV video data to RGB data for computer display;
• Fig. 4 may be an illustration of memory elements for use with a method of converting YUV video data to RGB data for computer display;
• Fig. 5 may be an illustration of parallel addition of memory elements to determine an RGB output value from a YUV input.
• Figure 1 illustrates an example of a suitable computing system environment 100 that may operate to provide the method described by this specification. It should be noted that the computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the method and apparatus of the claims. Neither should the computing environment 100 be interpreted as having any dependency or requirement relating to any one component or combination of components illustrated in the exemplary operating environment 100.
• an exemplary system for implementing the blocks of the claimed method includes a general purpose computing device in the form of a computer 110.
• Components of the computer 110 may include, but are not limited to, a processing unit 120, a system memory 130, and a system bus 121 that couples various system components including the system memory 130 and video interface 190 to the processing unit 120.
• the computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180, via a local area network (LAN) 171 and/or a wide area network (WAN) 173 via a modem 172 or other network interface 170.
• a remote computer 180 via a local area network (LAN) 171 and/or a wide area network (WAN) 173 via a modem 172 or other network interface 170.
• LAN local area network
• WAN wide area network
• Computer 110 typically includes a variety of computer readable media that may be any available media that may be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media.
• the system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132.
• ROM read only memory
• RAM random access memory
• the ROM may include a basic input/output system 133 (BIOS).
• BIOS basic input/output system
• RAM 132 typically contains data and/or program modules that include operating system 134, application programs 135, other program modules 136, and program data 137.
• the computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media such as a hard disk drive 141 a magnetic disk drive 151 that reads from or writes to a magnetic disk 152, and an optical disk drive 155 that reads from or writes to an optical disk 156.
• the hard disk drive 141, 151, and 155 may interface with system bus 121 via interfaces 140, 150.
• a user may enter commands and information into the computer 20 through input devices such as a keyboard 162 and pointing device 161, commonly referred to as a mouse, trackball or touch pad.
• Other input devices may include a microphone, joystick, game pad, satellite dish, scanner, or the like.
• a monitor 191 or other type of display device may also be connected to the system bus 121 via an interface, such as a video interface 190.
• a video conversion module 192 may be connected to the system bus 121.
• the video conversion module 192 may convert or modify pixels in accordance with the method described below.
• the video conversion module 192 is a component of another element of the computer 110.
• the video conversion module 192 may be a component of the processing unit 120, and/or the video interface 190.
• computers may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 190.
• ICs integrated circuits
• FIG. 2 is an illustration of a method 200 that may convert YUV (YC b C r ) formatted pixels for display in the RGB color space.
• the method may be executed on a variety of components of the computer 110.
• the method 200 is executed on a video conversion module 192.
• the method 200 is executed on the video interface 190.
• the method is executed with the processing unit 120.
• RGB, YUV, and YC b C r may generally describe all video formats of the RBG and YUV family including, but not limited to, respectively, RGB 16, RGB24, RGB32, YUV, YUY2, YV12, YV20, and 1420.
• the method may be performed as a part of any conversion or modification that may include a per-data unit computing cost.
• the method is used to convert video from YUV to RBG wherein the conversion requires a per-pixel computing cost.
• a matrix transform may be simplified.
• the transform may be used to convert a video image from one pixel format to another or to modify pixels of a video image.
• a matrix transform to convert YUV formatted video images into RBG images is simplified.
• the matrix transform of Table 1 may be simplified to compute the contributions of each of Y, U, and V to a resulting R, G, and B value.
• a value in the RGB color space for red may be composed of some degree of luminance (Y) and some degree of chrominance in the red to cyan range (V or C 1 . ).
• green may be composed of some degree of luminance (Y), some degree of chrominance in the red to cyan range (V or C r ), and some degree of chrominance in the blue to yellow range (U or C b ).
• blue may be composed of some degree of luminance (Y), and some degree of chrominance in the blue to yellow range (U or C b ). Therefore, as in Table 2, the formula for converting YUV pixels into RGB pixels may be simplified to emphasize the contributions of the Y, U, and V components.
• each of the inputs Y, U, and V contribute, in some degree, to a resulting output of Red, Green, and Blue.
• the total contribution of each value of Y, U t and V may be added together in planar form to produce a respective Red, Green, and Blue output value.
• tables may be computed to determine all possible output values corresponding to all possible input values.
• the tables may consist of a number of values equaling the range of the target format. For example, where the target format is RGB and the range for each component is 0 to 255, there may be three tables of 256 values each representing an entire set of possible output values from an input.
• the tables may be stored in any form of computer memory described in relation to Fig. 1.
• R, G, and B output values are computed for any possible Y, U, and V(Y, C ⁇ and C r ) input value.
• the possible input values for each of 7305, C b 310 , and C r 315 may be a value of 16-235, 16-239, and 16-239, respectively. While the contribution values of Figs. 3a-c contain up to three significant figures, the contribution may be represented in any useful precision.
• the contribution of each input value to R, G, and B may be calculated using the formulae of Table 2 and inserted into a corresponding position in a set of tables 300.
• an input 305 for 7 of 17 is converted by the formula (Y - 16) x 1.164 to produce a table value 320 of 1.164 as the luminance contribution to each of R 325, G 330, and B 335.
• an input 310 for U (Q, ) may be converted by the formulae (C 6 - 128) x -0.400 and (C 6 - 128) x 2.018 to produce a table value 340 for each contribution to G 345 and B 350, respectively.
• an input 315 for V (C r ) may be converted by the formulae (C 1 . - 128) x 1.596 and
• the method 200 may receive an input value for conversion.
• pixel values are received in planar mode in response to any modification of an image that results in a per-pixel computing cost.
• Some examples of operations that may require a per-pixel computing cost are conversions from one video format to another (i.e., YUV to RGB), a shader transform, a lightness or brightness correction, a saturation change, or a color correction.
• a tuple of YUV pixel values may be received by the method 200 as a plane of Y data, a plane of U data, and a plane of V data, in response to a requested conversion from the YUV format to another format (i.e., YUV to/from RGB).
• the method 200 receives non-planar, or "chunky" data.
• the YUV data may be received as a single memory unit containing all three Y, U, and V values.
• the method 200 may consider each piece of chunky data as a 1 x 1 pixel plane. Each piece of chunky data may be converted into separate planes of data on a pixel-by-pixel basis.
• the method 200 may use the lookup tables of block 210 to find a contribution of an input to the value of an output.
• a value from a YUV tuple is used to find its contribution to the desired output.
• a Y input 305 value of 17 may contribute 1.164 to the total value of each of an R, G, and B desired output.
• a U (C b ) input 310 value of 17 may contribute 0, 44.400, and -223.998 to the total value of a respective R, G, and B desired output
• a V (C 1 . ) input 315 value of 17 may contribute -177.156, 92.241, and 0 to the total value of a respective R, G, and B desired output.
• the method 200 may store the input contributions of block 220.
• each component's contributions are stored in a particular order within a single memory location.
• the data may be stored to prevent an overflow error during a subsequent addition process.
• the method 200 may store the contributions within a memory space that may be divided into equal portions to accommodate each contribution.
• a memory space may be a register residing on a processing unit 120 of a computer 110.
• a register may be any memory location in RAM 132, or any other discrete memory unit upon which an arithmetic function may be performed using the processing unit 120.
• a memory space may be any fixed-sized group of bits that are handled or processed together by the computer 110.
• a memory space may be a 16 bit word and a double word (dword) may be 32 bits long.
• the memory space is a 32-bit dword in which each contribution is allocated ten bits 405 of the dword with a "buffer" or "gap" 410 between each value.
• the buffer may be any size that is suitable to prevent an overflow error during a subsequent addition operation.
• the buffer may be 1 bit in size.
• the memory space may be any structure that may allocate enough memory to store each ordered contribution described above wherein the number of memory spaces may be equal to the number of elements in a target format.
• the format RGB is made up of three elements; therefore, the method 200 may use three memory spaces 415, 420, 425.
• each element of a contribution tuple may be stored in the same order over multiple spaces.
• each of three memory spaces 415, 420, 425 may have data representing a Y value stored in the first ten bits, a U value in the second ten bits, and a V value in the third ten bits, with a 1-bit buffer between each value. Therefore, the contributions of Y, U (C b ,), and V(C,) to the components R, G, and B may be stored in a different memory space 415, 420, 425, and arranged in the order of contributions to R, contributions to G, and contributions to B.
• the space 415 stores, in order, the contribution of 7 to each of R, G, and B
• space 420 stores, in order, the contribution of U (C b ,) to each of R, G, and B
• space 425 stores, in order, the contribution of V (C r ) to each of R, G, and B.
• memory spaces that store the components in a particular order as well as many sizes of the components stored.
• the method 200 may calculate output pixel values.
• the method 200 performs two parallel addition operations 502, 504 employing the three memory spaces 415, 420, and 425.
• the memory spaces 415, 420, and 425 may store, in order, the contributions to R, G, and B of Y, U(C b ,), and V(C r ), respectively, for each input value 505.
• the three values may be added together in parallel 502, 504 to calculate 506 a resulting value of R, G, and B.
• the values may be added in parallel as three 32-bit words, resulting in two add operations 502, 504.
• a gap 410 prevents component values of R, G, and B from influencing each other by overflow and carrying rules associated with binary addition operations.
• a total value 510 for each contribution may result from the parallel addition process to produce an output value for RGB from a YUV input.
• the result of the parallel addition may be saved in the order R, G, and B, to a single 32-bit memory space.
• the resulting total values are clipped to ensure they remain within an acceptable range for the target format.
• the total value for R may be a negative number 515. Because the allowable component values within the RGB format are between 0 and 255, a negative result may not be acceptable. Therefore, the method 200 may determine that, when converting to the RGB format, a negative number may be clipped to zero 517 to correspond to the acceptable RGB values range. Further, decimal values 520 may be rounded to an integer 525 value corresponding to a value within an acceptable output range. Likewise, a total number 510 that is above the acceptable range may be clipped to the maximum allowable value for the format. Many other embodiments of calculating the output values also exist.
• the method 200 may determine if there are more values that need to be converted. In one embodiment, the method 200 checks a buffer into which all pre-conversion pixel values are decompressed to determine if any values remain.
• the buffer may be a FIFO queue into which YUV pixel values are loaded for processing by the method 200. If the method 200 determines that more values need to be converted, the method may return to block 220. If no values remain, the method may proceed to block 240.
• the method 200 may store the converted or modified image. In one embodiment, the method 200 stores the image directly, on a pixel-by-pixel basis, to a memory, such as a back buffer.
• a compositing layer may decode compressed video to YUV planes, convert the YUV planes to an intermediate RGB buffer, and merge/composite the RGB buffer to a back buffer for later display.
• a compositor incorporating an intermediate buffer may be useful when converting between video formats that display images at different rates.
• video playback may operate at a frames-per-second rate that is slower than a typical digital animation sequence. Because video playback is slower, some frames of the slower format may be repeatedly displayed during playback in the faster format to account for the different rates. Rather than re-converting the same frame every time it is displayed, the converted YUV/RGB data may be saved to an intermediate buffer to achieve faster recall of previously- displayed frames.
• the conversion step consumes an appreciable amount of computer processing (as when the step converts using the Table 1 matrix transform for each pixel), it may be more efficient to convert a repeated frame only once and cache the resulting data. Employing the transform at each pixel conversion may, therefore, generate an additional Read/Modify/Write command for each processed pixel.
• the processing cost of conversion may be very low.
• the conversion from YUV to RGB may involve only a table lookup for each target format component followed by parallel addition. Therefore, because the conversion cost may be very low, it may be performed as each pixel is decompressed on a pixel-by-pixel basis and the resulting data may be saved directly to a back buffer. Saving directly to a back buffer may eliminate an intermediate (RGB) buffer as well as a Read/Modify/Write command during the conversion process and may also improve the cache coherency. Further, the Write/Combine capabilities of certain processors may substantially improve the conversion and output performance.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Image Processing (AREA)
• Color Image Communication Systems (AREA)
• Processing Of Color Television Signals (AREA)
• Facsimile Image Signal Circuits (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=39644870

Family Applications (1)

Country Status (5)

Families Citing this family (10)

Citations (4)

Family Cites Families (21)

• 2007
  • 2007-01-26 US US11/627,754 patent/US7639263B2/en not_active Expired - Fee Related
• 2008
  • 2008-01-23 JP JP2009547402A patent/JP4886043B2/ja not_active Expired - Fee Related
  • 2008-01-23 EP EP08728148.1A patent/EP2111714B1/en not_active Not-in-force
  • 2008-01-23 WO PCT/US2008/051813 patent/WO2008091955A1/en not_active Ceased
  • 2008-01-23 CN CN2008800032202A patent/CN101589609B/zh not_active Expired - Fee Related
• 2011
  • 2011-07-14 JP JP2011155756A patent/JP2011223633A/ja active Pending

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events