WO2007004111A1 - Method and system of converting rgb image data to ycbcr image data - Google Patents

Method and system of converting rgb image data to ycbcr image data Download PDF

Info

Publication number
WO2007004111A1
WO2007004111A1 PCT/IB2006/052089 IB2006052089W WO2007004111A1 WO 2007004111 A1 WO2007004111 A1 WO 2007004111A1 IB 2006052089 W IB2006052089 W IB 2006052089W WO 2007004111 A1 WO2007004111 A1 WO 2007004111A1
Authority
WO
WIPO (PCT)
Prior art keywords
ycbcr
rgb
transformation matrix
color transformation
image data
Prior art date
Application number
PCT/IB2006/052089
Other languages
French (fr)
Inventor
Sandeep M. Dalal
Original Assignee
Koninklijke Philips Electronics, N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics, N.V. filed Critical Koninklijke Philips Electronics, N.V.
Publication of WO2007004111A1 publication Critical patent/WO2007004111A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46Colour picture communication systems
    • H04N1/64Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor
    • H04N1/646Transmitting or storing colour television type signals, e.g. PAL, Lab; Their conversion into additive or subtractive colour signals or vice versa therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/001Texturing; Colouring; Generation of texture or colour
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/64Circuits for processing colour signals
    • H04N9/642Multi-standard receivers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/64Circuits for processing colour signals
    • H04N9/67Circuits for processing colour signals for matrixing

Definitions

  • This invention pertains to the field of video signal processing, and more particularly, to a method and system of converting YCbCr image data to RGB image data.
  • RGB image data must be converted to YCbCr image data before transmission or distribution.
  • a 3x3 color transformation matrix is applied to the original RGB color data of the camera to produce the YCbCr image data.
  • the television industry has standardized the use of the ITU-R BT.601 color transformation matrix for transforming standard definition (SD) video signals from RGB to YCbCr prior to broadcast.
  • SD standard definition
  • a television camera in the studio creates digital gamma corrected signals - R'G ⁇ ' (where the prime indicates gamma corrected signals) - which are further transformed by the color transformation matrix.
  • the ITU-R BT.601 RGB-to- YCbCr color transformation matrix is mathematically given as:
  • the ITU-R BT.709 color transformation matrix is recommended for transforming gamma corrected high definition (HD) video signals R'G'B' to YCbCr image data in accordance with the Advanced Television Systems Committee (ATSC) standard.
  • the ITU-R BT.709 RGB-to- YCbCr color transformation matrix is mathematically given as:
  • R'G'B' are normalized to be in the range [0-1].
  • the corresponding YCbCr values are represented with, e.g., 8-bit digital video data that spans the following nominal range: Y in the range [16-235] and Cb and Cr both in the range [16-240].
  • the maximum possible 8-bit video data range of [0-255] is not utilized by the YCbCr values so as to allow for some headroom as well as to allow for timing synchronization reserved codes in the signal processing equipment used in the studio.
  • Nominal ranges for 10-bit digital video can also be found in video standards. The analysis below that refers to 256 levels can equally well apply to both 8-bit and 10-bit digital video.
  • the YCbCr values are combined into a video signal that is sent via broadcast channels (e.g. terrestrial, cable, or satellite) to a television receiver in consumer homes.
  • broadcast channels e.g. terrestrial, cable, or satellite
  • digital compression techniques are being used to send digital video over these channels.
  • the YCbCr image data is recovered from the video signal and converted back into RGB data for display or further signal processing.
  • the television receiver includes one or more color transformation matrices for converting the YCbCr image data to gamma corrected R'G'B' image data. In this process, it is important that the receiver use an appropriate color transformation matrix that corresponds to the color transformation matrix that was used at the transmit end to produce the YCbCr image data for transmission.
  • YCbCr image data it would be desirable to provide an improved method and system of converting YCbCr image data to RGB image data. It would also be desirable to provide a method and system for determining which color matrix was used by a broadcast station to encode YCbCr image data so that a receiver can apply the proper color transformation matrix for decoding the true image and ensuring accurate RGB colors on the screen. It would be further desirable to provide a method and system for recovering, directly from the YCbCr image data, the information relating to which color transformation matrix was used by the broadcast station and therefore allow selection of the proper color transformation matrix for decoding the YCbCr image data to RGB image data.
  • the present invention is directed to addressing one or more of the preceding concerns.
  • a method of converting YCbCr image data into RGB image data comprises: providing a plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix; receiving YCbCr image data; calculating histogram data for luminance data values of the YCbCr image data; performing a frequency transform on the histogram data to produce a frequency distribution of the histogram data; determining whether a localized peak occurs in the frequency distribution of the histogram data; selecting one of the sets of coefficients for the YCbCr-to-RGB color transformation matrix in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data; and applying the YCbCr-to-RGB color transformation matrix with the selected set of coefficients to the YCbCr image data to produce RGB image data.
  • a system for converting YCbCr image data to RGB image data comprises: a memory adapted to store therein plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix; a histogram generator adapted to calculate histogram data for luminance data values of received YCbCr image data; a frequency transformer adapted to perform a frequency transform on the histogram data to produce a frequency distribution of the histogram data; a peak detector adapted to determine whether a localized peak occurs in the frequency distribution of the histogram data; a matrix coefficients selector adapted to select one of the sets of coefficients for the YCbCr-to RGB color transformation matrix in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data; and a color transformer adapted to multiply the received YCbCr image data with the YCbCr-to RGB color transformation matrix to produce RGB image data.
  • a method of converting YCbCr image data into RGB image data comprises: receiving YCbCr image data; determining, from the YCbCr image data, a RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data; in response to the determination of the RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data, selecting coefficients for a YCbCr-to-RGB color transformation matrix; and multiplying the received YCbCr image data by the YCbCr-to-RGB color transformation matrix to produce RGB image data.
  • FIG. 1 shows the luminance histogram of YCbCr image data produced from a sample image using two different color transformation matrices
  • FIG. 2 shows an accumulated luminance histogram of YCbCr image data produced from over 81 different images using two different color transformation matrices
  • FIG. 3 shows the luminance histogram of YCbCr image data produced from a second sample image using two different color transformation matrices
  • FIG. 4 shows the frequency distribution of a frequency transform of the histogram data of FIG. 3
  • FIG. 5 shows the luminance histogram of YCbCr image data produced from a third sample image using two different color transformation matrices, and the corresponding frequency distribution of a frequency transform of the histogram data;
  • FIG. 6 shows the luminance histogram of YCbCr image data produced from a fourth sample image using two different color transformation matrices, and the corresponding frequency distribution of a frequency transform of the histogram
  • FIG. 7 is a functional diagram illustrating a method and system of converting YCbCr image data to RGB image data.
  • FIG. 8 illustrates a method of converting YCbCr image data to RGB image data
  • FIG. 9 is a block diagram illustrating components of a television receiver that converts YCbCr image data to RGB image data, and displays the RGB data.
  • the inventor has recognized that it would be desirable if there was some way to analyze received YCbCr image data and determine, directly from the YCbCr image data, information relating to which color transformation matrix was used by the broadcast station. This would allow selection of the proper color transformation matrix for decoding the YCbCr image data to RGB image data without requiring the decoding and processing of separate meta-data, which can be unavailable and/or unreliable in some cases.
  • FIG. 1 shows the luminance histogram for YCbCr image data produced from a sample image using both ITU-R BT.601 and ITU-R BT.709 color transformation matrices. The sample image itself is shown on the top-right corner of the figure.
  • the ITU-R BT.601 color transformation matrix generates a distribution of Y values which has a noticeably oscillatory nature compared to the ITU-R BT.709 color transformation matrix.
  • this behavior may not be repeated for any other image than the one shown in FIG. 1.
  • the accumulated histogram was calculated for YCbCr image data produced using both ITU-R BT.601 and ITU-R BT.709 color transformation matrices on 81 images randomly chosen from a set of images acquired from the INFOCOMM® 2002 test image CD-ROM. The results are shown in FIG. 2. Apart from the large spikes seen at a few gray levels, one can still observe the oscillatory nature of the histogram using the ITU-R BT.601 color transformation matrix.
  • FIG. 2 indicates that applying the ITU-R BT.601 color transformation matrix to typical RGB image data from natural images produces a preference for some luminance gray levels over others. This is typically due to the specific matrix coefficients used in the matrix and the fact that the dynamic range for the Y channel is limited to 220 levels which requires scaling, rounding and clipping transformations in the matrix calculations.
  • FIG. 3 shows the luminance histogram of YCbCr image data produced from a second sample image using the ITU-R BT.601 color transformation matrix and the ITU-R BT.709 color transformation matrix.
  • a frequency transformation e.g., a Fast Fourier Transform (FFT)
  • FFT Fast Fourier Transform
  • FIG. 4 shows the frequency distribution of a frequency transform of the histogram data of FIG. 3.
  • FIG. 4 shows the magnitude of the 256 point fast Fourier transform (FFT) of the histogram data of FIG. 3.
  • FFT 256 point fast Fourier transform
  • the frequency data produced by the ITU- R BT.601 color transformation matrix shows a prominent spike at a frequency of around 42, whereas the data produced by the ITU-R BT.709 color transformation matrix does not show any prominent peak in the same region.
  • this number indicates that approximately every 6th ( ⁇ 256/42) gray level occurs more frequently in the histogram of Y data produced by the ITU-R BT.601 color transformation matrix. This is precisely the indication seen in FIG.
  • FIGs. 5-6 show the luminance histograms of YCbCr image data produced from third and fourth sample images, respectively, using two different color transformation matrices, and the corresponding frequency distributions of frequency transforms of the histogram data.
  • the frequency data produced by the ITU-R BT.601 color transformation matrix in FIGs. 5-6 exhibits a prominent spike at a frequency of around 42, whereas the data produced by the ITU-R BT.709 color transformation matrix does not show any prominent peak in the same region.
  • FIG. 7 shows a functional diagram illustrating a method and system of converting YCbCr image data to RGB image data. Since the video signal is a continuous stream of frames, the decision on whether an image was processed by the ITU-R BT.601 color transformation matrix or the ITU-R BT.709 color transformation matrix can be made on a frame-by-frame basis and stored for application to the next frame. This eliminates any need for field/frame memories in the receiver for the sake of this process.
  • a first functional block 710 calculates a histogram of the Y component of the received data over 256 gray levels (the 256 levels correspond to 8-bit video, however one could also calculate a 256 level histogram for 10-bit video by increasing the bin size).
  • a second functional block 720 performs a frequency transform on the histogram data.
  • the frequency transform is a fast Fourier transform (FFT).
  • FFT fast Fourier transform
  • DCT discrete cosine transform
  • a functional block 730 determines whether a localized peak occurs in the frequency distribution of the histogram data. If there is a sharp peak in the FFT magnitude data at index 42, it is determined that the signal was processed by the ITU-R BT.601 color transformation matrix. If there is no sharp peak, then it is determined that the ITU-R BT.601 color transformation matrix was not used and the other assumption, which is the ITU-R BT.709 color transformation matrix was used, is applied.
  • block 740 selects a first set of coefficients for the receiver's YCbCr- to-RGB color transformation matrix in a first case where functional block 730 determines that there is a sharp peak in the FFT magnitude data at index 42, and selects a second set of coefficients for the YCbCr-to-RGB color transformation matrix in a second case where functional block 730 determines that there is no sharp peak in the FFT magnitude data at index 42.
  • the first set of matrix coefficients is selected to correspond to a case where it is determined that the received YCbCr image data was produced on the transmission end using the ITU-R BT.601 color transformation matrix.
  • the first set of matrix coefficients for the receiver's YCbCr-to-RGB color transformation matrix may be chosen as an inverse of the ITU-R BT.601 RGB-to-YCbCr color transformation matrix.
  • the actual matrix coefficients that are selected will be adjusted to compensate for the actual color gamut of the display device associated with the receiver.
  • the second set of matrix coefficients is selected to correspond to a case where it is determined that the received YCbCr image data was produced on the transmission end using the ITU-R BT.709 color transformation matrix.
  • the second set of coefficients for the receiver's YCbCr-to-RGB color transformation matrix may be chosen as an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix, but may be adjusted to compensate for the actual color gamut of the display device associated with the receiver.
  • the selected coefficients are loaded into the YCbCr-to-RGB color transformation matrix 750 and the received YCbCr image data is multiplied by the YCbCr-to-RGB color transformation matrix 750 to produce RGB data for display.
  • FIG. 8 illustrates a method 800 of converting YCbCr image data to RGB image data.
  • a receiver is provided with a plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix.
  • the first set of coefficients for the receiver's YCbCr-to-RGB color transformation matrix is chosen to correspond to the ITU-R BT.601 color transformation matrix (e.g., an inverse of the ITU-R BT.601 RGB-to-YCbCr color transformation matrix).
  • the first set of coefficients is further selected to compensate for the actual color gamut of the display device associated with the receiver.
  • the second set of matrix coefficients is chosen to correspond to the ITU-R BT.709 color transformation matrix (e.g., an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix). Also beneficially, the second set of coefficients is further selected to compensate for the actual color gamut of the display device associated with the receiver.
  • a receiver receives a video signal including YCbCr image data.
  • histogram data is calculated for luminance (Y) data values of the YCbCr image data.
  • a frequency transformation is performed on the luminance (Y) histogram data to produce a frequency distribution of the Y histogram data.
  • a 256-point FFT, DCT, etc. is performed to generate the frequency data.
  • a fifth step 850 it is determined whether a localized peak occurs in the frequency distribution of the Y histogram data. Beneficially, if a 256 point frequency transformation is performed in step 840, it is determined whether a localized peak occurs at a frequency value of 42.
  • one of the sets of coefficients for the YCbCr-to RGB color transformation matrix is selected in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data.
  • the first set of matrix coefficients is selected in the case where a localized peak is detected, and the second set of matrix coefficients is selected when no localized peak is detected.
  • the selected coefficients are loaded into the YCbCr-to RGB color transformation matrix
  • FIG. 9 is a block diagram illustrating components of a television receiver 900 that converts YCbCr image data to RGB image data, and displays the RGB data.
  • the receiver 900 includes a front-end demodulator (or decoder) 910, a color format converter 920, and a display 990.
  • Color format converter 920 includes a memory 925, a histogram generator 935, a frequency transformer 945, a peak detector 955, a matrix coefficients selector 965, and a color transformer 975.
  • the memory 925 stored a plurality of sets of coefficients for a YCbCr-to RGB color transformation matrix for the receiver 900.
  • the first set of coefficients corresponds to the ITU-R BT.601 color transformation matrix (e.g., an inverse of the ITU-R BT.601 RGB-to-YCbCr color transformation matrix)
  • the second set of matrix coefficients is chosen to correspond to the ITU-R BT.709 color transformation matrix (e.g., an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix).
  • the front-end demodulator (or decoder) 910 receives a video signal, and outputs YCbCr image data.
  • the histogram generator 935 calculates a histogram of the Y component of the received data as described above.
  • the frequency transformer 945 performs a frequency transform on the histogram data, also as described above.
  • the peak detector 955 determines whether a localized peak occurs in the frequency distribution of the histogram data.
  • the matrix coefficients selector 965 selects a first set of coefficients for the receiver's YCbCr-to-RGB color transformation matrix from the memory 925 in a first case where peak detector 955 determines that there is a localized peak in the magnitude of the frequency distribution of the histogram data, and selects a second set of coefficients for the YCbCr-to-RGB color transformation matrix from the memory 925 in a second case where peak detector 955 determines that there is no localized peak in the magnitude of the frequency distribution of the histogram data.
  • the color transformer 975 converts the received YCbCr image data to RGB image data by multiplying the received YCbCr image data by a YCbCr-to RGB color transformation matrix including the coefficient set from the memory 925 that was selected and loaded by the matrix coefficients selector 965.
  • the RGB image data is then passed to the display 990, which may include some further signal processing components as desired.
  • the histogram generator 935, frequency transformer 945, peak detector 955, matrix coefficients selector 965, and color transformer 975 may be implemented in a variety of configurations.
  • these blocks may constitute one or more software modules executed by a processor 950 (which may perform a variety of other functions) of the receiver 900.
  • a processor 950 may be a general-purpose processor, a dedicated digital signal processor (DSP), etc.
  • DSP dedicated digital signal processor
  • these blocks may be implemented by digital signal processing hardware, including, for example, an application specific integrated circuit (ASIC), or by some combination of a processor and digital signal processing hardware, etc.
  • ASIC application specific integrated circuit

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Processing Of Color Television Signals (AREA)
  • Color Image Communication Systems (AREA)

Abstract

A system (920) and method (800) of converting YCbCr image data into RGB image data includes: determining (830-850), from the YCbCr image data, a RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data; selecting (860) coefficients for a YCbCr-to-RGB color transformation matrix based on that determination; and multiplying (870) the received YCbCr image data by the YCbCr-to- RGB color transformation matrix to produce RGB image data. The system (920) and method (800) determine (830-850) the RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data by calculating (830) histogram data for luminance data values of the YCbCr image data, performing (840) a frequency transform on the histogram data to produce a frequency distribution of the histogram data, and determining (850) whether a localized peak occurs in the frequency distribution of the histogram data.

Description

METHOD AND SYSTEM OF CONVERTING RGB IMAGE DATA TO YCbCr IMAGE DATA
This invention pertains to the field of video signal processing, and more particularly, to a method and system of converting YCbCr image data to RGB image data.
Cameras used by television studios acquire real-world images using red-green- blue (RGB) colors. However, broadcast video is transmitted using YCbCr color signals. Therefore, the RGB image data must be converted to YCbCr image data before transmission or distribution. In particular, a 3x3 color transformation matrix is applied to the original RGB color data of the camera to produce the YCbCr image data.
The television industry has standardized the use of the ITU-R BT.601 color transformation matrix for transforming standard definition (SD) video signals from RGB to YCbCr prior to broadcast. A television camera in the studio creates digital gamma corrected signals - R'GΗ' (where the prime indicates gamma corrected signals) - which are further transformed by the color transformation matrix.
The ITU-R BT.601 RGB-to- YCbCr color transformation matrix is mathematically given as:
Y 65.4810 128.553 24.966
(1) Cb - 37.7968 - 74.2032 112 Cr 112 -93.786 -18.214
Figure imgf000003_0001
Meanwhile, the ITU-R BT.709 color transformation matrix is recommended for transforming gamma corrected high definition (HD) video signals R'G'B' to YCbCr image data in accordance with the Advanced Television Systems Committee (ATSC) standard. The ITU-R BT.709 RGB-to- YCbCr color transformation matrix is mathematically given as:
Y 46.5375 156.673 15.7899
(2) Cb -25.6493 -86.3507 112 Cr 112 -101.746 -10.2542
Figure imgf000003_0002
In each of the equations (1) and (2) above, R'G'B' are normalized to be in the range [0-1]. The corresponding YCbCr values are represented with, e.g., 8-bit digital video data that spans the following nominal range: Y in the range [16-235] and Cb and Cr both in the range [16-240]. The maximum possible 8-bit video data range of [0-255] is not utilized by the YCbCr values so as to allow for some headroom as well as to allow for timing synchronization reserved codes in the signal processing equipment used in the studio. Nominal ranges for 10-bit digital video can also be found in video standards. The analysis below that refers to 256 levels can equally well apply to both 8-bit and 10-bit digital video.
The YCbCr values are combined into a video signal that is sent via broadcast channels (e.g. terrestrial, cable, or satellite) to a television receiver in consumer homes. Recently, digital compression techniques are being used to send digital video over these channels.
At the television receiver, the YCbCr image data is recovered from the video signal and converted back into RGB data for display or further signal processing. In particular, the television receiver includes one or more color transformation matrices for converting the YCbCr image data to gamma corrected R'G'B' image data. In this process, it is important that the receiver use an appropriate color transformation matrix that corresponds to the color transformation matrix that was used at the transmit end to produce the YCbCr image data for transmission. For example, if the YCbCr image data was encoded with the ITU-R BT.709 matrix and the television receiver uses the ITU-R BT.601 matrix for decoding the YCbCr image data, then there will clearly be an error in the decoded gamma corrected R'G'B' image data. So, for example, when these signals when displayed on a display screen, they will produce an inaccurate image.
Therefore, for accurate signal reproduction, it is necessary for a receiver to have access to information indicating what color transformation matrix was applied to produce the YCbCr image data. One solution to this problem is to explicitly indicate which color transformation matrix was used as separate meta-data transmitted together with the digital video data.
However, there are some disadvantages to this approach. For one thing, it requires that some active step be taken on the transmit end to include this information in the form of meta-data. Inevitably, this will not always be done for every transmitted video signal, and even when it is done, it is highly subject to error. Indeed, in some broadcast facilities it may turn out to be difficult to determine for certain what color transformation matrix is applied to a particular video data stream. Furthermore, the meta-data may not be made available to every receiver.
Accordingly, it would be desirable to provide an improved method and system of converting YCbCr image data to RGB image data. It would also be desirable to provide a method and system for determining which color matrix was used by a broadcast station to encode YCbCr image data so that a receiver can apply the proper color transformation matrix for decoding the true image and ensuring accurate RGB colors on the screen. It would be further desirable to provide a method and system for recovering, directly from the YCbCr image data, the information relating to which color transformation matrix was used by the broadcast station and therefore allow selection of the proper color transformation matrix for decoding the YCbCr image data to RGB image data. The present invention is directed to addressing one or more of the preceding concerns.
In one aspect of the invention, a method of converting YCbCr image data into RGB image data, comprises: providing a plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix; receiving YCbCr image data; calculating histogram data for luminance data values of the YCbCr image data; performing a frequency transform on the histogram data to produce a frequency distribution of the histogram data; determining whether a localized peak occurs in the frequency distribution of the histogram data; selecting one of the sets of coefficients for the YCbCr-to-RGB color transformation matrix in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data; and applying the YCbCr-to-RGB color transformation matrix with the selected set of coefficients to the YCbCr image data to produce RGB image data.
In another aspect of the invention, a system for converting YCbCr image data to RGB image data, comprises: a memory adapted to store therein plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix; a histogram generator adapted to calculate histogram data for luminance data values of received YCbCr image data; a frequency transformer adapted to perform a frequency transform on the histogram data to produce a frequency distribution of the histogram data; a peak detector adapted to determine whether a localized peak occurs in the frequency distribution of the histogram data; a matrix coefficients selector adapted to select one of the sets of coefficients for the YCbCr-to RGB color transformation matrix in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data; and a color transformer adapted to multiply the received YCbCr image data with the YCbCr-to RGB color transformation matrix to produce RGB image data.
In yet another aspect of the invention, a method of converting YCbCr image data into RGB image data, comprises: receiving YCbCr image data; determining, from the YCbCr image data, a RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data; in response to the determination of the RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data, selecting coefficients for a YCbCr-to-RGB color transformation matrix; and multiplying the received YCbCr image data by the YCbCr-to-RGB color transformation matrix to produce RGB image data.
FIG. 1 shows the luminance histogram of YCbCr image data produced from a sample image using two different color transformation matrices; FIG. 2 shows an accumulated luminance histogram of YCbCr image data produced from over 81 different images using two different color transformation matrices;
FIG. 3 shows the luminance histogram of YCbCr image data produced from a second sample image using two different color transformation matrices;
FIG. 4 shows the frequency distribution of a frequency transform of the histogram data of FIG. 3;
FIG. 5 shows the luminance histogram of YCbCr image data produced from a third sample image using two different color transformation matrices, and the corresponding frequency distribution of a frequency transform of the histogram data;
FIG. 6 shows the luminance histogram of YCbCr image data produced from a fourth sample image using two different color transformation matrices, and the corresponding frequency distribution of a frequency transform of the histogram;
FIG. 7 is a functional diagram illustrating a method and system of converting YCbCr image data to RGB image data; and
FIG. 8 illustrates a method of converting YCbCr image data to RGB image data; and
FIG. 9 is a block diagram illustrating components of a television receiver that converts YCbCr image data to RGB image data, and displays the RGB data. The inventor has recognized that it would be desirable if there was some way to analyze received YCbCr image data and determine, directly from the YCbCr image data, information relating to which color transformation matrix was used by the broadcast station. This would allow selection of the proper color transformation matrix for decoding the YCbCr image data to RGB image data without requiring the decoding and processing of separate meta-data, which can be unavailable and/or unreliable in some cases.
One of the interesting properties is the distribution of luminance (Y) levels over a single image frame. This distribution is also referred to as the luminance histogram. FIG. 1 shows the luminance histogram for YCbCr image data produced from a sample image using both ITU-R BT.601 and ITU-R BT.709 color transformation matrices. The sample image itself is shown on the top-right corner of the figure.
From inspection of FIG. 1, one can notice that the ITU-R BT.601 color transformation matrix generates a distribution of Y values which has a noticeably oscillatory nature compared to the ITU-R BT.709 color transformation matrix. Of course this behavior may not be repeated for any other image than the one shown in FIG. 1. Accordingly, the accumulated histogram was calculated for YCbCr image data produced using both ITU-R BT.601 and ITU-R BT.709 color transformation matrices on 81 images randomly chosen from a set of images acquired from the INFOCOMM® 2002 test image CD-ROM. The results are shown in FIG. 2. Apart from the large spikes seen at a few gray levels, one can still observe the oscillatory nature of the histogram using the ITU-R BT.601 color transformation matrix.
FIG. 2 indicates that applying the ITU-R BT.601 color transformation matrix to typical RGB image data from natural images produces a preference for some luminance gray levels over others. This is typically due to the specific matrix coefficients used in the matrix and the fact that the dynamic range for the Y channel is limited to 220 levels which requires scaling, rounding and clipping transformations in the matrix calculations.
Another method to verify this preferential behavior of the ITU-R BT.601 color transformation matrix is to perform a frequency transformation (e.g., a Fast Fourier Transform (FFT)) on the luminance histogram data for a given image. If there is discernible peak in the magnitude of the Fourier transform, it indicates that a certain gray level differential occurs more frequently in the image data than other gray level differentials. FIG. 3 shows the luminance histogram of YCbCr image data produced from a second sample image using the ITU-R BT.601 color transformation matrix and the ITU-R BT.709 color transformation matrix.
FIG. 4 shows the frequency distribution of a frequency transform of the histogram data of FIG. 3. In particular, FIG. 4 shows the magnitude of the 256 point fast Fourier transform (FFT) of the histogram data of FIG. 3. The frequency data produced by the ITU- R BT.601 color transformation matrix shows a prominent spike at a frequency of around 42, whereas the data produced by the ITU-R BT.709 color transformation matrix does not show any prominent peak in the same region. Given knowledge of FFT properties, this number indicates that approximately every 6th (~ 256/42) gray level occurs more frequently in the histogram of Y data produced by the ITU-R BT.601 color transformation matrix. This is precisely the indication seen in FIG. 3 in the behavior of the histogram of the luminance data produced by the ITU-R BT.601 color transformation matrix, where there are small spikes observed every 6 gray levels. The FFT quantifies this behavior by capturing a peak in bin number 42 of the 256 point FFT.
FIGs. 5-6 show the luminance histograms of YCbCr image data produced from third and fourth sample images, respectively, using two different color transformation matrices, and the corresponding frequency distributions of frequency transforms of the histogram data. In similarity to FIG. 4, the frequency data produced by the ITU-R BT.601 color transformation matrix in FIGs. 5-6 exhibits a prominent spike at a frequency of around 42, whereas the data produced by the ITU-R BT.709 color transformation matrix does not show any prominent peak in the same region.
Accordingly, it is possible from the frequency transform of the histogram of the Y data to determine which color transformation matrix, between the ITU-R BT.601 and the ITU-R BT.709 color transformation matrices, was used at the broadcast source end to produce the YCbCr image data.
FIG. 7 shows a functional diagram illustrating a method and system of converting YCbCr image data to RGB image data. Since the video signal is a continuous stream of frames, the decision on whether an image was processed by the ITU-R BT.601 color transformation matrix or the ITU-R BT.709 color transformation matrix can be made on a frame-by-frame basis and stored for application to the next frame. This eliminates any need for field/frame memories in the receiver for the sake of this process. A first functional block 710 calculates a histogram of the Y component of the received data over 256 gray levels (the 256 levels correspond to 8-bit video, however one could also calculate a 256 level histogram for 10-bit video by increasing the bin size). Then, a second functional block 720 performs a frequency transform on the histogram data. Beneficially, the frequency transform is a fast Fourier transform (FFT). Alternative frequency discriminating transforms, such as discrete cosine transform (DCT), may be also applicable for computational complexity reasons. Next, a functional block 730 determines whether a localized peak occurs in the frequency distribution of the histogram data. If there is a sharp peak in the FFT magnitude data at index 42, it is determined that the signal was processed by the ITU-R BT.601 color transformation matrix. If there is no sharp peak, then it is determined that the ITU-R BT.601 color transformation matrix was not used and the other assumption, which is the ITU-R BT.709 color transformation matrix was used, is applied. Accordingly, block 740 selects a first set of coefficients for the receiver's YCbCr- to-RGB color transformation matrix in a first case where functional block 730 determines that there is a sharp peak in the FFT magnitude data at index 42, and selects a second set of coefficients for the YCbCr-to-RGB color transformation matrix in a second case where functional block 730 determines that there is no sharp peak in the FFT magnitude data at index 42.
The first set of matrix coefficients is selected to correspond to a case where it is determined that the received YCbCr image data was produced on the transmission end using the ITU-R BT.601 color transformation matrix. In that case, the first set of matrix coefficients for the receiver's YCbCr-to-RGB color transformation matrix may be chosen as an inverse of the ITU-R BT.601 RGB-to-YCbCr color transformation matrix. However, typically, the actual matrix coefficients that are selected will be adjusted to compensate for the actual color gamut of the display device associated with the receiver. Similarly, the second set of matrix coefficients is selected to correspond to a case where it is determined that the received YCbCr image data was produced on the transmission end using the ITU-R BT.709 color transformation matrix. In that case, the second set of coefficients for the receiver's YCbCr-to-RGB color transformation matrix may be chosen as an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix, but may be adjusted to compensate for the actual color gamut of the display device associated with the receiver. The selected coefficients are loaded into the YCbCr-to-RGB color transformation matrix 750 and the received YCbCr image data is multiplied by the YCbCr-to-RGB color transformation matrix 750 to produce RGB data for display.
FIG. 8 illustrates a method 800 of converting YCbCr image data to RGB image data.
In a first step 810, a receiver is provided with a plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix. Advantageously, the first set of coefficients for the receiver's YCbCr-to-RGB color transformation matrix is chosen to correspond to the ITU-R BT.601 color transformation matrix (e.g., an inverse of the ITU-R BT.601 RGB-to-YCbCr color transformation matrix). Beneficially, the first set of coefficients is further selected to compensate for the actual color gamut of the display device associated with the receiver. Meanwhile, advantageously, the second set of matrix coefficients is chosen to correspond to the ITU-R BT.709 color transformation matrix (e.g., an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix). Also beneficially, the second set of coefficients is further selected to compensate for the actual color gamut of the display device associated with the receiver. In a second step 820, a receiver receives a video signal including YCbCr image data.
In a third step 830, histogram data is calculated for luminance (Y) data values of the YCbCr image data.
In a fourth step 840, a frequency transformation is performed on the luminance (Y) histogram data to produce a frequency distribution of the Y histogram data. Beneficially, a 256-point FFT, DCT, etc. is performed to generate the frequency data.
In a fifth step 850, it is determined whether a localized peak occurs in the frequency distribution of the Y histogram data. Beneficially, if a 256 point frequency transformation is performed in step 840, it is determined whether a localized peak occurs at a frequency value of 42.
In a sixth step 860, one of the sets of coefficients for the YCbCr-to RGB color transformation matrix is selected in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data. Advantageously, the first set of matrix coefficients is selected in the case where a localized peak is detected, and the second set of matrix coefficients is selected when no localized peak is detected. The selected coefficients are loaded into the YCbCr-to RGB color transformation matrix
In a seventh step 870, the YCbCr-to RGB color transformation matrix with the selected set of coefficients is applied to the YCbCr image data to output RGB data. FIG. 9 is a block diagram illustrating components of a television receiver 900 that converts YCbCr image data to RGB image data, and displays the RGB data. The receiver 900 includes a front-end demodulator (or decoder) 910, a color format converter 920, and a display 990. Color format converter 920 includes a memory 925, a histogram generator 935, a frequency transformer 945, a peak detector 955, a matrix coefficients selector 965, and a color transformer 975.
At the outset, the memory 925 stored a plurality of sets of coefficients for a YCbCr-to RGB color transformation matrix for the receiver 900. Advantageously, the first set of coefficients corresponds to the ITU-R BT.601 color transformation matrix (e.g., an inverse of the ITU-R BT.601 RGB-to-YCbCr color transformation matrix), and the second set of matrix coefficients is chosen to correspond to the ITU-R BT.709 color transformation matrix (e.g., an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix).
The front-end demodulator (or decoder) 910 receives a video signal, and outputs YCbCr image data. The histogram generator 935 calculates a histogram of the Y component of the received data as described above. The frequency transformer 945 performs a frequency transform on the histogram data, also as described above. The peak detector 955 determines whether a localized peak occurs in the frequency distribution of the histogram data. The matrix coefficients selector 965 selects a first set of coefficients for the receiver's YCbCr-to-RGB color transformation matrix from the memory 925 in a first case where peak detector 955 determines that there is a localized peak in the magnitude of the frequency distribution of the histogram data, and selects a second set of coefficients for the YCbCr-to-RGB color transformation matrix from the memory 925 in a second case where peak detector 955 determines that there is no localized peak in the magnitude of the frequency distribution of the histogram data. The color transformer 975 converts the received YCbCr image data to RGB image data by multiplying the received YCbCr image data by a YCbCr-to RGB color transformation matrix including the coefficient set from the memory 925 that was selected and loaded by the matrix coefficients selector 965. The RGB image data is then passed to the display 990, which may include some further signal processing components as desired.
The histogram generator 935, frequency transformer 945, peak detector 955, matrix coefficients selector 965, and color transformer 975 may be implemented in a variety of configurations. In one embodiment, these blocks may constitute one or more software modules executed by a processor 950 (which may perform a variety of other functions) of the receiver 900. Such a processor 950 may be a general-purpose processor, a dedicated digital signal processor (DSP), etc. Alternatively, these blocks may be implemented by digital signal processing hardware, including, for example, an application specific integrated circuit (ASIC), or by some combination of a processor and digital signal processing hardware, etc.
While preferred embodiments are disclosed herein, many variations are possible which remain within the concept and scope of the invention. Such variations would become clear to one of ordinary skill in the art after inspection of the specification, drawings and claims herein. The invention therefore is not to be restricted except within the spirit and scope of the appended claims.

Claims

CLAIMS:
1. A method (800) of converting YCbCr image data into RGB image data, comprising: providing (810) a plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix; receiving (820) YCbCr image data; calculating (830) histogram data for luminance data values of the YCbCr image data; performing (840) a frequency transform on the histogram data to produce a frequency distribution of the histogram data; determining (850) whether a localized peak occurs in the frequency distribution of the histogram data; selecting (860) one of the sets of coefficients for the YCbCr-to-RGB color transformation matrix in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data; and applying (870) the YCbCr-to-RGB color transformation matrix with the selected set of coefficients to the YCbCr image data to produce RGB image data.
2. The method (800) of claim 1, wherein performing (840) a frequency transform on the histogram data includes performing a 256 point fast Fourier transform (FFT) or discrete cosine transform (DCT) on the histogram data.
3. The method (800) of claim 2, wherein determining (850) whether a localized peak occurs in the frequency distribution of the histogram data includes determining whether a peak occurs at a value of 42.
4. The method (800) of claim 1, wherein a first set of coefficients for the YCbCr- to-RGB color transformation matrix is an inverse of the ITU-R BT.601 RGB-to- YCbCr color transformation matrix, and a second set of coefficients for the YCbCr-to-RGB color transformation matrix is an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix.
5. The method (800) of claim 1, wherein a first set of coefficients for a YCbCr-to- RGB color transformation matrix is an inverse of the ITU-R BT.601 RGB-to- YCbCr color transformation matrix adjusted to compensate for a color gamut of a display device which displays the RGB image data, and a second set of coefficients for a YCbCr-to-RGB color transformation matrix is an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix adjusted to compensate for the color gamut of the display device which displays the RGB image data.
6. A system (920) for converting YCbCr image data to RGB image data, comprising: a memory (925) adapted to store therein plurality of sets of coefficients for a YCbCr-to-RGB color transformation matrix; a histogram generator (935) adapted to calculate histogram data for luminance data values of received YCbCr image data; a frequency transformer (945) adapted to perform a frequency transform on the histogram data to produce a frequency distribution of the histogram data; a peak detector (955) adapted to determine whether a localized peak occurs in the frequency distribution of the histogram data; a matrix coefficients selector (965) adapted to select one of the sets of coefficients for the YCbCr-to RGB color transformation matrix in response to the determination of whether a localized peak occurs in the frequency distribution of the histogram data; and a color transformer (975) adapted to multiply the received YCbCr image data with the YCbCr-to RGB color transformation matrix to produce RGB image data.
7. The system (920) of claim 6, wherein further including a processor (950), and wherein the histogram generator (935), frequency transformer (945), peak detector (955), matrix coefficients selector (965), and color transformer (975) comprise at least one software algorithm executed by the processor (950).
8. The system (920) of claim 6, wherein the frequency transform is a 256 point fast Fourier transform (FFT) or discrete cosine transform (DCT).
9. The system (920) of claim 8, wherein the peak detector (955) determines whether a peak occurs at a value of 42.
10. The system (920) of claim 6, wherein a first set of coefficients for a YCbCr-to- RGB color transformation matrix is an inverse of the ITU-R BT.601 RGB-to- YCbCr color transformation matrix, and a second set of coefficients for a YCbCr-to-RGB color transformation matrix is an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix.
11. The system (920) of claim 6, wherein a first set of coefficients for a YCbCr-to- RGB color transformation matrix is an inverse of the ITU-R BT.601 RGB-to- YCbCr color transformation matrix adjusted to compensate for a color gamut of a display device which displays the RGB image data, and a second set of coefficients for a YCbCr-to-RGB color transformation matrix is an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix adjusted to compensate for the color gamut of the display device which displays the RGB image data.
12. A method (800) of converting YCbCr image data into RGB image data, comprising: receiving (820) YCbCr image data; determining (830-850), from the YCbCr image data, a RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data; in response to the determination of the RGB -to- YCbCr color transformation matrix that was used to generate the YCbCr image data, selecting (860) coefficients for a YCbCr- to-RGB color transformation matrix; and multiplying (870) the received YCbCr image data by the YCbCr-to-RGB color transformation matrix to produce RGB image data.
13. The method (800) of claim 12, wherein determining (830-850), from the YCbCr image data, a RGB-to- YCbCr color transformation matrix that was used to generate the YCbCr image data comprises: calculating (830) histogram data for luminance data values of the YCbCr image data; performing (840) a frequency transform on the histogram data to produce a frequency distribution of the histogram data; and determining (850) whether a localized peak occurs in the frequency distribution of the histogram data.
14. The method (800) of claim 13, wherein performing (840) a frequency transform on the histogram data includes performing a 256 point fast Fourier transform (FFT) or discrete cosine transform (DCT) on the histogram data.
15. The method (800) of claim 14, wherein determining (850) whether a localized peak occurs in the frequency distribution of the histogram data includes determining whether a peak occurs at a value of 42.
16. The method (800) of claim 13, further comprising: determining (860) that a first RGB-to- YCbCr color transformation matrix was used to generate the YCbCr image data when there is a localized peak in the frequency distribution of the histogram data; and determining (860) that a second RGB-to- YCbCr color transformation matrix was used to generate the YCbCr image data when there is no localized peak in the frequency distribution of the histogram data.
17. The method (800) of claim 16, wherein the first RGB-to- YCbCr color transformation matrix is the ITU-R BT.601 color transformation matrix, and the second RGB-to- YCbCr color transformation matrix is the ITU-R BT.709 color transformation matrix.
18. The method (800) of claim 12, wherein selecting (860) coefficients for a YCbCr-to-RGB color transformation matrix, comprises: selecting a first set of coefficients for the YCbCr-to-RGB color transformation matrix in a first case when there is a localized peak in the frequency distribution of the histogram data; and selecting a second set of coefficients for the YCbCr-to-RGB color transformation matrix when there is no localized peak in the frequency distribution of the histogram data.
19. The method (800) of claim 18, wherein the first set of coefficients for the YCbCr-to-RGB color transformation matrix is an inverse of the ITU-R BT.601 RGB-to- YCbCr color transformation matrix, and the second set of coefficients for the YCbCr-to- RGB color transformation matrix is an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix.
20. The method (800) of claim 18, wherein the first set of coefficients for a YCbCr-to-RGB color transformation matrix is an inverse of the ITU-R BT.601 RGB-to- YCbCr color transformation matrix adjusted to compensate for a color gamut of a display device which displays the RGB image data, and the second set of coefficients for a YCbCr- to-RGB color transformation matrix is an inverse of the ITU-R BT.709 RGB-to- YCbCr color transformation matrix adjusted to compensate for the color palette of the display device which displays the RGB image data
PCT/IB2006/052089 2005-06-29 2006-06-26 Method and system of converting rgb image data to ycbcr image data WO2007004111A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US69524405P 2005-06-29 2005-06-29
US60/695,244 2005-06-29

Publications (1)

Publication Number Publication Date
WO2007004111A1 true WO2007004111A1 (en) 2007-01-11

Family

ID=37118424

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/052089 WO2007004111A1 (en) 2005-06-29 2006-06-26 Method and system of converting rgb image data to ycbcr image data

Country Status (1)

Country Link
WO (1) WO2007004111A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801922A (en) * 2021-04-01 2021-05-14 暨南大学 Color image-gray image-color image conversion method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0703701A2 (en) * 1990-09-28 1996-03-27 Eastman Kodak Company Color image processing system for preparing a composite image transformation module for performing a plurality of selected image transformations
EP1294179A2 (en) * 2001-09-12 2003-03-19 Canon Kabushiki Kaisha Method and apparatus for processing image data
US20050128218A1 (en) * 2003-12-11 2005-06-16 Xerox Corporation Method for determining color space of an image

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0703701A2 (en) * 1990-09-28 1996-03-27 Eastman Kodak Company Color image processing system for preparing a composite image transformation module for performing a plurality of selected image transformations
EP1294179A2 (en) * 2001-09-12 2003-03-19 Canon Kabushiki Kaisha Method and apparatus for processing image data
US20050128218A1 (en) * 2003-12-11 2005-06-16 Xerox Corporation Method for determining color space of an image

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801922A (en) * 2021-04-01 2021-05-14 暨南大学 Color image-gray image-color image conversion method
CN112801922B (en) * 2021-04-01 2021-07-27 暨南大学 Color image-gray image-color image conversion method

Similar Documents

Publication Publication Date Title
US10992898B2 (en) Display method and display device
JP6019270B1 (en) Guided color grading for extended dynamic range
KR100782818B1 (en) Method and system for luminance preserving color conversion from YUV to RGB
US8639050B2 (en) Dynamic adjustment of noise filter strengths for use with dynamic range enhancement of images
EP3242482A1 (en) Method and apparatus for encoding/decoding a high dynamic range picture into a coded bitstream
CN100394766C (en) Image signal processor and image signal processing method
EP3446284B1 (en) Method and apparatus for conversion of dynamic range of video signals
KR101080846B1 (en) Apparatus and method for color distortion correction of image by estimate of correction matrix
JP5002348B2 (en) Image processing apparatus, video receiving apparatus, and image processing method
KR101221865B1 (en) Sparkle processing
US20070222891A1 (en) Systems and methods for video data conversion
WO2007004111A1 (en) Method and system of converting rgb image data to ycbcr image data
US8792712B2 (en) Method for correcting color and apparatus using the same
KR101634652B1 (en) Method and apparatus for intensificating contrast in image
KR20170048890A (en) Method and apparatus for controlling content contrast in content ecosystem
US20090213271A1 (en) Video Signal Processing Device and Video Signal Processing Method
KR102245835B1 (en) Method for compensating picture quality
US8587725B2 (en) Method of digital signal processing
EP3281408A1 (en) Improved compression in high dynamic range video
CN118158344A (en) Audio-visual system sharing image processing program and video processing method thereof
KR20040036789A (en) Natural color reproduction method and apparatus on DTV
JP2021048522A (en) Image processing apparatus and image processing method
Fonseca et al. Quality Assessment of Video in Digital Television
JP2006325154A (en) Signal correction device, receiver having signal correction device and signal correction program
KR20030054101A (en) Method for embedding watermark which is not affected by image compression and judging the alteration or forgery of the image using thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06765870

Country of ref document: EP

Kind code of ref document: A1