WO2004110072A1

WO2004110072A1 - Apparatus and method for format converting and mixing video signal

Info

Publication number: WO2004110072A1
Application number: PCT/CN2003/000446
Authority: WO
Inventors: Tzuping Lin
Original assignee: Mediatek Inc.
Priority date: 2003-06-06
Filing date: 2003-06-06
Publication date: 2004-12-16
Also published as: DE10394246T5; AU2003246127A1; CN1778118A

Abstract

The invention provides a method for converting and mixing multiple video data in video data process apparatus. Said multiple video data have multiple sampling formats respectively, and said multiple formats include a high sampling frequency chrominance format and a low frequency chrominance format at least. Said method include: receive first video data having said low sampling frequency chrominance format and second video data having said high sampling frequency chrominance format respectively, up-sampling process the first video data having said low frequency chrominance format to make it become a first video data having high sampling frequency chrominance format; mix the first video data having high sampling frequency chrominance format and the second video data and output the mixed video data having high sampling frequency chrominance format.

Description

Format conversion for video signals and

Hybrid device and method

Technical field

The invention provides a device and method for format conversion and mixing of image data, in particular, a method for converting image data having a low sampling frequency chroma format into image data having a high sampling frequency chroma format, and then Another device and method for mixing image data with high sampling frequency and chroma format to avoid data loss. Background technique

Motion picture compression standard MPEG-1 and the new motion picture compression standard MPEG-2 are mainly used to play video signal data stored in digital storage media such as CD-ROM or DVD-ROM, such as movies and animations, or applications In high-definition digital televisions, its main function is image-based image compression technology. The MPEG-1 standard is the first standard developed by the MPEG organization. The main goal is to compress a resolution of 320 X 240 and about 20 frames per second to 1.2 Mbps, and to compress stereo music at about 250 kbps. Combined together into a movie of about 1.5Mbps, stored it in a CD-ROM, and played it at 2x CD-Player. The MPEG-2 standard is greatly improved. The shortcomings of MPEG-1 not only have significant improvements in picture and sound quality, but also multi-language, multi-language subtitles, multi-angle viewing, movie rating, and so on. In terms of sound quality, MPEG-2 is partially compatible with MPEG-1's sound compression method, and has added AAC (Advanced Audio Coding sound compression technology) with high compression ratio. In terms of picture quality, MPEG-2's picture resolution is improved To 720 X 480, and add some new compression methods and video signal sampling formats, in order to improve the picture clarity and provide a more effective compression rate.

"Compression" basically refers to the process of reducing the information capacity occupied by the content of an image or group of images by eliminating the redundant components present in the video signal data, and one of the characteristics of the video signal data is the spatially redundant component high. During the compression, which is the process of so-called spatial redundancy removal, it is necessary to identify the important elements in the video signal data and remove the duplicate and less influential elements. According to experiments, the human eye is more sensitive to changes in brightness and relatively less detectable to changes in chrominance. Therefore, MPEG-2 uses the color representation format of Luminance and Chrominance, Y Indicates the luminance value, C indicates the chrominance value (C includes CB and CR, which represent the first and second chrominance values, respectively). Before the relevant video signal data is encoded, the three primary color component signals 1, G, and B are converted into luminance. Y and chroma CB, CR form. Since reducing chroma sampling can achieve a larger data reduction effect with the least impact on vision, MPEG-2 uses reducing chroma sampling to reduce the amount of signal. There are three sampling formats defined in MPEG-2: 4: 2: 0 sampling format, 4: 2: 2 sampling format, and 4: 4: 4 sampling format, respectively representing three different chrominance sampling frequencies. 4: 2: 0 means that four luminances Y take one chroma CR—one chroma CB. The digital image data of the main screen in digital storage media such as CD-ROM or DVD-EDM, etc., is taken as 4: 2: 0 Sampling format. And 4: 2: 2 means that four luminances Y take two chroma CR two chroma CB. Similarly, 4: 4: 4 means that four luminances Y take four chroma CR four chroma CB, that is, no chroma sampling is performed and reduced. Please refer to FIG. 1. FIG. 1 is a schematic diagram of the distribution of luminance Y and chrominance C in an image plane 10 in a 4: ² : 0 sampling format. The image plane 10 includes a plurality of image elements 11 (or sampling points). Each picture element 11 may include only luminance Y, chrominance CB, CR, or both luminance Y and chrominance CB, CR after sampling. 〇 represents the sampling point of luminance Y, X represents the sampling points of chrominance CB, CR, and the entire image plane 10 can be visually composed of multiple lines (Line) 18 (to obtain a clear picture, the brightness Y sampling point 〇 and chromaticity The sampling points X of CB and CR are shown separately in each picture element 11). In the 4: 2: 0 sampling format, no matter in the vertical direction (arrow 1 or horizontal direction (arrow 14)) in the image plane 10, the # frequency of chroma C is only half of the luminance Y First, as a whole, the sampling frequency of chrominance C can be regarded as a quarter of the luminance Y. In addition, as shown in FIG. 1, it can be more obvious in a block space 16 containing four luminance Y It can be seen that the significance of the 4: 2: 0 sampling format is that four luminances Y correspond to one chromaticity C (one chromaticity CR—one chromaticity CB).

When transmitting and processing image data, in the image plane 10 shown in FIG. 1, all sampling points are scanned line by line in the direction of the arrow 14. In this way, the shortcomings of the 4: 2: 0 sampling format are exposed. As can be seen from FIG. 1, in the 4: 2: 0 sampling format, every other line is found to completely lack any chroma CB, CR sampling points ( X), the vertical resolution of chroma is severely lost. At this time, vertical interpolation of some chroma samples must be used to make up for the chroma missing during scanning. Please refer to FIG. 2. FIG. 2 is a schematic diagram of the distribution of luminance Y and chrominance C in another image plane 20 under a 4: 2: 2 sampling format. The image plane 20 also includes a plurality of image elements ²¹ , and each image element ² may include only luminance y, chroma CB, CR, or both luminance Y and chroma CB, CR after sampling, and the entire image plane 20 The view is composed of multiple lines (Eine) 28. As in the representation of FIG. 1, 0 represents a sampling point of luminance Y, and X represents a sampling point of chroma CB, CR. In current specifications, the brightness in the 4: 2: 2 sampling format The sampling frequency of Y is 13.5MHz, and the sampling frequencies of both chroma CR and CB are 6.75MHz. After making up the missing part of the chroma C, a block space 26 in the 4: 2: ² sampling format contains four luminances Y and two chroma C (two chroma CR and two chroma CB). Please refer to FIG. 3. FIG. 3 is a schematic diagram of the distribution of luminance Y and chrominance C in another image plane 30 in a 4: 4: 4 sampling format. The image plane 30 'is composed of multiple lines (Iine) ³⁸ , and includes a plurality of image elements 31, each of which includes both luminance Y (sampling point 0) and chrominance CB, CR (sampling point X). Therefore, each image block space 36 includes four luminances Y, four chroma CRs, and four chroma CBs, that is, no chroma sampling reduction is performed.

As mentioned above, the main-picture image data in a digital storage medium such as a CD-ROM or DVD-ROM adopts a 2: 0 sampling format, while the sub-picture (SP-SP) in a DVD-ROM ) Image data and On-Screen Display (OSD) data are in 4: 4: 4 sampling format. The MPEG-2 decoder pecodeir previously applied to DVD players uses an external video signal encoder (TV Encoder), and its interface must conform to the CCIR (now changed to the International Telecommunication Union (ITU) standard) specifications and be able to transmit 4: 2: 2 sampling format MPEG-2 image data, but the aforementioned sub-picture image data and screen image setting data with 4: 4: 4 sampling format cannot be transmitted, so the main picture must be transmitted to the video signal encoder before Interpolate into 4: 2: 2 format first. Therefore, when mixing the MPEG2 main screen image data in the 4: 2: 2 sampling format with the sub-picture image data in the 4: 4: 4 sampling format (Mix), the sub-picture having the 4: 4: 4 sampling format must first be mixed. The image data is first converted into the sub-picture image data in the 4: 2: 2 sampling format, and then mixed with the MPEG2 main picture image data in the 4: 2: 2 sampling format, and then the mixed image data can be transmitted from this interface channel. Based on the above-mentioned image data processing methods and related structures to complete image data format conversion and mixing, known technologies have appeared in some related literatures and patents. In US Patent No. 5,489,947, "On screen display arrangement for a digital video signal processing system", Cooper et al. 4 are sub-picture image data with a 4: 4: 4 sampling format first converted to 4: 2: 2 After the sub-picture image data in the sampling format is mixed with the MPEG2 main picture image data in the 4: 2: 2 sampling format, the image data in different formats are mixed. Similarly, Htusecky et al. In US Patent No. 6,529,244, "Digital video decode system with OSD processor for converting graphics data in 4: 4: 4 format to 4: 2: 2 format by mathematically combining chrominance values" will also have The screen image setting data OSD of the 4: 4: 4 sampling format is first converted to the 4: 2: 2 sampling format, and then the image data is mixed according to the 4: 2: 2 sampling format.

Related to the above-mentioned known patents (US Patent No. 6,529,244 and No. 5,489,947) Please refer to FIG. 4 for the basic structure. FIG. 4 is a functional block diagram of a known image data processing device 40. The image data processing device 40 includes a main picture data receiving end 42, a pair of picture data receiving ends 44, a 4: 4: 4 to 4: 2: 2 format converter 46, a data mixing device 48, and an external video The signal encoding module 50 (TV Encoding Module), the interface channel CI between the data mixing device 48 and the video signal encoding module 50 must comply with the CCIR (or ITU) specifications. The main screen data receiving end 42 is used to receive a main screen image data with a 4: ² : 2 sampling format, and as shown in FIG. 4, the main screen image data received with a 4: 2: 2 sampling format includes a main brightness Ym and a main chromaticity Cm. At this time, please refer to FIG. 5 at the same time. FIG. 5 is a schematic diagram of the brightness and chromaticity of multiple image data in the transmitted data string in FIG. 4. FIG. 5 shows the main brightness Ym and the main chroma Cm of the main screen image data with the 4: 2: 2 sampling format in FIG. 4 respectively. The data strings are transmitted in time sequence in two different data channels. The data string of Ym contains multiple main luminances YmO, Yml, Ym ² obtained from different sampling points. Similarly, the data string of main chromaticity Cm contains multiple main luminances obtained from different sampling points. Chroma CBmO, CRmO, CBm2, CRm2, ···, Since the basic concept of the 4: 2: 2 sampling format is that the sampling frequency of chroma C (CR, CB) is one-half of the luminance Y, it can be seen that the main The chrominances CBm0 and CRmO can be from the same sampling point, and correspond to the sampling points of the main luminance YmO, or the average of the main chroma of the surrounding sampling points. Similarly, the main chroma CBm2, CRm2, and main brightness Ym2 can be the average of the main chroma and main brightness from the same sampling point or the surrounding sampling points. The sub-picture data receiving end 44 in FIG. 4 may be used to receive a sub-picture image data having a 4: 4: 4 sampling format, where the sub-picture image data having a 4: 4: 4 sampling format includes a pair of brightness Ys, a The first sub-chroma CBs and a second sub-chroma CRs can be reconfirmed from FIG. 5 at the same time. Each sampling point includes a sub-chroma Ys, a first sub-chroma CBs, and a second sub-chroma CRs. Without any reduction in chroma sampling, for example, the sub-luminance Ys0, the first sub-chroma CBs0, and the second sub-chroma CRsO correspond to the same sampling point.

Please refer to FIG. 4. The 4: 4: 4 to 4: 2: 2 format converter 46 is electrically connected to the sub-picture data receiving end 44, and the sub-picture image data with the 4: 4: 4 sampling format is deleted ( After down-sampling), it is converted into sub-picture image data in 4: 2: 2 sampling format. This subtraction process is aimed at reducing the chroma sampling to achieve the effect of data reduction. Therefore, the sub-brightness Ys of the sub-picture image data that originally had the 4: 4: 4 sampling format is not affected at all, 4: 4: 4 to 4: The 2: 2 format converter 46 mainly processes the first sub-chroma CBs and the second sub-chroma CRs. Please refer to Figure 5, which shows two processing methods A and B. The subtraction A method discards the first sub-chroma CBs or the second sub-chroma CRs sequentially at each sampling point. In this example, the chroma CRsO, CBsl, CRs ² , etc. are discarded to reduce Half the amount of data on chroma. The rule of subtraction B is to completely discard the first sub-chroma CBs and second sub-chroma CRs of the sampling point every other sampling point, and then move the second sub-chroma CRs of the previous sampling point to the discarded sampling point. After consolidating into a complete continuous data string. The sub-picture image data in the 4: 2: 2 sampling format generated after the reduction processing of the 4: 4: 4 to 4: 2: 2 format converter 46 can be regarded as including a (original) sub-luminance Ys, and A pair of chroma Cs, and the secondary chroma Cs can be regarded as a result obtained after the first sub-chroma CBs and the second sub-chroma CRs are processed by the foregoing subtraction A method or subtraction: B method. A data mixing device 48 electrically connected to the 4: 4: 4 to 4: 2: 2 format converter 46 and the main screen data receiving end 42 can be used to combine the main screen image data with a 4: 2: ² sampling format and have a 4: 2: ² sampling format. After the secondary screen image data in the 2: 2 sampling format is subjected to a mixing operation, a mixed image data having the 4: 2: 2 sampling format is output. The mixed image data with a 4: 2: 2 sampling format includes a mixed luminance Yg and a mixed chrominance C _g . FIG. 5 also shows that the mixed luminance Yg and the mixed chrominance Cg are transmitted in time sequence in two different data channels, respectively. Situation.

The main screen data receiving end 42 of FIG. 4 may further include a 4: 2: 0 to 4: 2: 2 format converter 47, which is electrically connected to the main screen data receiving end 4 ² and may store one in a sampling format (in a Digital video discs, such as VCD or DVD, etc., convert the main screen digital image data into the aforementioned main screen image data with a 4: 2: 2 sampling format. In addition, the connected video signal encoding module 50 includes a 4: 2: 2 to 4: 4: 4 format converter 49 and a video signal encoder 51CTV Encoder). In this embodiment, the video signal encoder 51 The image data must conform to the 4: 4: 4 sampling format, and the 4: 2: 2 to 4: 4: 4 format converter 49 can interpolate the transmitted mixed image data with the 4: 2: 2 sampling format ( After Up-sampling processing, mixed image data (including a mixed luminance Yg, a first mixed chrominance CBg, and a second mixed chrominance CR) having a 4: 4: 4 sampling format is generated, and finally encoded by a video signal the device 51 has a 4: 4: 4 sampling format mixing video data is converted into a television video signal Ts (TV video signal) ₀

For the above method and device for converting and mixing image data of different sampling formats, for the sub-picture image data with higher resolution, the 4: 4: 4 sampling format is reduced to the 4: 2: 2 sampling format. Even if only a limited amount of chrominance information is lost in the process, it will still eventually lead to color distortion. Please refer to FIG. 5 again, if the subtractive A method is used to sequentially discard the first sub-chroma CBs or the second sub-chroma CRs of each sampling point, after that, FIG. The format converter 49 performs interpolation processing, that is, in the process of reconstructing the first chroma CB and the second chroma CR of each sampling point, the chroma CRsO, CBsl, CRs2, etc. that were originally rounded will be The duplicated chroma CRsl, CBs2, CRs3,… etc. replace the backfill. Taking the sampling point corresponding to the luminance YsO as an example, since the chrominance CRsO is lost, if the chrominance CRsl Copy and replace, the final color will show the mixed color of CBsO and CRsl, which is not the original correct combination of CBsO and CRsO, which will cause the color of the sampling point to be distorted. If the subtraction: B method completely discards the first pair of chroma CBs and the second pair of chroma CRs at every sampling point, the lost colors cannot be recovered, and the color distortion is even more serious. These colors Distortion becomes more apparent with subtle subtitle images and edge images. Even if the current DVD (or VCD) player chip mostly incorporates the video signal encoding module 50 or the video signal encoder 51 of FIG. 4 in the same chip, it does not need to pass the interface of the CCIR specification (or ITU standard), but it is still inherited. The foregoing conventional technique has the disadvantage of data loss to distort colors. Summary of the invention

Therefore, the main object of the present invention is to provide a device and method that can convert and mix multiple image data with different sampling formats (Sampling Foirmat :), so as to solve the problem.

In the present invention, based on the specifications established by the new dynamic image compression standards (MPEG-1, MPEG-2), an image data of a main screen with a 4: 2: 2 sampling format is interpolated.

(Up-sampling) becomes the main screen image data with 4: 4: 4 sampling format after processing, and then processes the main screen image data with 4: 4: 4 sampling format and a 4: 4: 4 sampling format. The secondary screen image data is mixed to avoid the loss of some chromaticity information during the conversion of the 4: 4: 4 sampling format to the 4: 2: 2 sampling format by known technologies, resulting in color distortion, and thus can fully Taking advantage of a video signal encoder built into a DVD (or VCD) player chip system, it has a significant effect on improving the picture quality.

An object of the present invention is to provide a device for converting and mixing a plurality of video data, each of which has a plurality of sampling formats (Sampling Format :). Sampling frequency chrominance format and a low sampling frequency chrominance format. The device includes a first data receiving end for receiving a first image data having the low sampling frequency chrominance format. A second data receiving Terminal for receiving a second image data having the high sampling frequency chroma format; a format conversion module electrically connected to the first data receiving terminal for receiving the first image having the low sampling frequency chroma format After the data is processed by up-sampling, it becomes the first image data having the high sampling frequency chrominance format; and a data mixing device electrically connected to the format conversion module and the second data receiving end, for After mixing (Mix) the first image data having the high sampling frequency chroma format and the second image data having the high sampling frequency chroma format, a Mixing the high sampling frequency chrominance format video data. Another object of the present invention is to provide an image data processing device, which includes a main frame data receiving end for receiving a main frame image data having a 4: ² : 2 sampling format, wherein the main frame image data has a 4: 2: 2 The main screen image data in the sampling format includes a main luminance value (Luminance) and a main chrominance value (Chrominance); a pair of screen data receiving ends is used to receive a sub screen image data with a 4: 4: 4 sampling format, The sub-picture image data with a 4: 4: 4 sampling format includes a pair of luminance values, a first pair of chrominance values, and a second pair of chrominance values; a format conversion module electrically connected to the main screen data The receiving end is used to process the main screen image data with a 4: 2: 2 sampling format into an main screen image data with a 4: 4: 4 sampling format after an Up-sampling process, where the The main screen image data in the 4: 4: 4 sampling format includes a main luminance value, a first main chrominance value, and a second main chrominance value. A data mixing device is electrically connected to the format conversion module and the auxiliary unit. Picture data receiving end 4: 4: the main screen ⁴ sampling format of image data and that has a 4: mixing 4 sampling format: 4: subpicture 4 sampling format of video data via a mixing operation (Mixing Operation), the output having a 4: 4 Image data; and a video signal encoder (TV Encoder) electrically connected to the data mixing device for converting the mixed image data having a 4: 4: 4 sampling format into a TV Video Signal .

Another object of the present invention is to provide a method for converting and mixing a plurality of video data (Video Signal) to prevent data leakage. The plurality of video data has multiple sampling formats, respectively. Format contains at least a high sampling frequency

(Chrominance) format and a low sampling frequency chroma format, the method includes receiving a first image data having the low sampling frequency chroma format and a second image data having the high sampling frequency chroma format; The first image data having the low sampling frequency chroma format converts the first image data having the high sampling frequency chroma format; and the first image data having the high sampling frequency chroma format and the high sampling frequency color After mixing (Mix) the second image data in the degree format, a mixed image data having the high sampling frequency chrominance format is output.

Another object of the present invention is to provide a method for processing image data in an image data processing device. The image data processing device includes a signal receiving module, a format conversion module, and a data mixing device. The method includes using The signal receiving module receives a main screen image data with a 4: 2: 2 sampling format and a sub screen image data with a 4: 4: 4 sampling format; using the format conversion module to sample the data with 4: 2: 2 Format the main screen image data with the 4: 4: 4 sampling format; and use the data mixing device to convert the main screen image data with the 4: 4: 4 sampling format and the 4: 4 : 4 sub-picture image data After the mixing operation, a mixed image data with a 4: 4: 4 sampling format is output. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing the distribution of luminance and chrominance in an image plane under a 4: 2: 0 sampling format. FIG. 2 is a schematic diagram showing the distribution of luminance and chrominance in an image plane in a 4: 2: 2 sampling format. FIG. ³ is a schematic diagram showing the distribution of brightness and chrominance in an image plane in a ⁴ : 4: 4 sampling format. FIG. 4 is a functional block diagram of a known image data processing.

FIG. 5 is a schematic diagram of brightness and chrominance of a plurality of image data in the transmitted data string in FIG. 4. FIG. 6 is a functional block diagram of an embodiment of a device for converting and mixing multiple image data according to the present invention.

FIG. 7 is a functional block diagram of another embodiment of an apparatus for converting and mixing multiple image data according to the present invention.

FIG. 8 is a flowchart of a method embodiment of the present invention.

FIG. 9 is a flowchart of another method embodiment of the present invention.

FIG. 10 is a functional block diagram of an image data processing device according to an embodiment of the present invention during actual implementation.

FIG. 11 is a schematic diagram of brightness and chrominance of a plurality of image data in the transmission data string in FIG. 10. FIG. 12 is a schematic diagram of an embodiment of the data mixing device in FIG. 10.

FIG. 13 is a functional block diagram of another embodiment of the image data processing apparatus in FIG. 10 when it is actually implemented.

FIG. 14 is a flowchart of another method embodiment of the present invention. DESCRIPTION OF SYMBOLS

10, 20, 30 image plane

11, 21, 31 primitives (sampling points)

12, 22, 32 (vertical) arrows

14, 24, 34 (horizontal) arrows

16, 26, 36 block space

18, 28, 38 lines

40, 80 image data processing device ''

44, 82 Main screen data receiver 44, 84 secondary picture data receivers

46 4: 4: 4 to 4: 2: 2 Format Converter

48, 68, 88 data mixing device

49, 89 4: 2: 2 to 4: 4: 4 format converter

50 video signal encoding module

51, 71, 81 video signal encoder

60, 70 devices

62, 72 First data receiving end

64, 74 Second data receiving end

65, 75, 85 format conversion module

73 first intermediate format converter

76 first intermediate format converter

87 4: 2: 2 to 4: 4: 4 format converter

Please refer to FIG. 6. FIG. 6 is a functional block diagram of an embodiment of a device 60 for converting and mixing a plurality of video data according to the present invention. Please note that this embodiment includes two image data; a first image data IS1 and a second image data IS2, and the two image data may each have two sampling formats, including a high sampling frequency color. Chrominance format and a low sampling frequency chroma format. The name of the two sampling formats shows that the sampling frequency of chroma for high sampling frequency chroma format is higher than that of low sampling frequency chroma format. For example, if Referring to the description of the basic concepts of chrominance sampling under the new motion image compression standards (MPEG-1, MPEG-2) in FIGS. 1 to 3, if the high-sampling-frequency chrominance format can correspond to the 4: 4: 4 sampling format, The low sampling frequency chroma grid test can correspond to the 4: 2: 2 sampling format or the 4: 2: 0 sampling format. The device 60 includes a first data receiving end 62, a second data receiving end 64, a format conversion module 65, and a data mixing device 68. The first data receiving end 62 is configured to receive a first image data IS1 having a low sampling frequency chroma format, and the second data receiving end 64 is configured to receive a second image data IS2 having a high sampling frequency chroma format. The format conversion module 65 is electrically connected to the first data receiving end 62, and is used to process the first image data IS1 with a low sampling frequency chroma format into an chroma format with a high sampling frequency after an up-sampling process. The interpolation process of the first image data IS1 ′ is described in detail in the following description and embodiments. Finally, the data mixing device 68 is electrically connected to the format After the conversion module 65 and the second data receiving end 64 are used to mix (Mk) the first image data ISr having a high sampling frequency chroma format and the second image data IS2 having a high sampling frequency chroma format, and output A mixed image data ISg with a high sampling frequency chroma format completes the conversion and mixing of image data with two different sampling frequency chroma formats. (Jason: It is recommended to change IS1 in the high-sampling-frequency chroma format of Figure 6 to ISr to facilitate identification)

In summary, the device 60 of the embodiment of FIG. 6 discloses one of the important technical features of the present invention, which is to convert an image data (first image data IS1) with a low sampling frequency chroma format into a high sampling frequency chroma Format image data (first image data isr), and then mix it with another image data (second image data IS2) with high sampling frequency chrominance format. During the conversion and mixing of this image data, The conversion process from "high sampling frequency chroma format to low sampling frequency chroma format" occurs, that is, there is no need to discard any chroma-related data, so there will be no loss of chroma data. Please note that the number of image data does not need to be limited to two as in this embodiment. If three or more image data with different sampling frequency and chroma formats are used, the technical features of the present invention are still applicable. One or more of the image data having a low sampling frequency chroma format is first converted into image data having a high sampling frequency chroma format, and then three or more of the image data has a high sampling frequency chroma format The image data is mixed to avoid the loss of chrominance data. Please refer to FIG. 7. FIG. 7 is a functional block diagram of another embodiment of a device 70 for converting and mixing multiple image data according to the present invention. The device 70 of the embodiment of FIG. 7 is substantially similar to the device 60 of the embodiment of FIG. 6, and the components with the same name also have the same functions, but this embodiment uses some additional components to describe the technology of the present invention in more detail. feature. Like the device 70 in FIG. 6, the device 70 in FIG. 7 also includes a first data receiving end ⁷² , a first data receiving end ⁷⁴ , a format conversion module 75, and a data mixing device "78. The receiving end ⁷² receives a first image data IS1 having a low sampling frequency chroma format, and uses the first data receiving end 74 to receive a second image data IS2 having a high sampling frequency chroma format, and uses the format conversion module 75 to have a low The first image data IS1 of the sampling frequency chroma format is converted into the first image data IS1 'of the high sampling frequency chroma format. Finally, the data mixing device 78 is used to convert the first image number isr The second image data IS2 having a high sampling frequency chroma format mixes and outputs a mixed image data ISgo having a high sampling frequency chroma format

Unlike FIG. 6, the format conversion module 75 shown in FIG. 7 is constituted by a first intermediate format converter 73 and a second intermediate format converter ^76, and this two image data (first image data and second IS1 Two image data IS2) not only have two sampling formats (high sampling frequency chroma format and low sampling Frequency chroma format), and also includes a sampling frequency chroma format, where the sampling frequency chroma format sampling frequency is between the high sampling frequency chroma format and the low sampling frequency chroma format For example, if the basic concepts of chroma sampling are described under the new motion image compression standards (MPEG-1, MPEG-2) in Figures 1 to 3, the high sampling frequency chroma format can correspond to 4: 4: 4 The sampling format, the medium sampling frequency chroma format may correspond to the 4: ² : 2 sampling format, and the low sampling frequency chroma format may correspond to the 4: 2: 0 sampling format. The setting of the first and second intermediate format converters "76" represents that the format conversion process of the first image data IS1 is a two-stage process. after a first post-interpolation process, to become the first video data IS1 "chroma format having the sampling frequency, and the second intermediate format converter 76 is electrically connected to the first intermediate format converter ^73, is used to having the The first image data isi "of the sampling frequency chrominance format is processed by a second interpolation to become the first image data isr having a high sampling frequency chrominance format. The principles of the first and second interpolation processing are the same as the aforementioned interpolation. The same processing is performed in the same way, which will be described in detail later. Please note that, as described in the embodiment of FIG. 6, the number of image data processed by the present invention is not limited, and the number of types of sampling formats is not limited. Before mixing multiple image data with different sampling frequency chrominance formats, all of them are converted to one of the highest sampling frequency chrominance formats to avoid conversion in high sampling frequency chrominance formats. The loss of chroma data during the low-sampling frequency chroma format is consistent with the technical features of the present invention. In addition, the device 70 of this embodiment further includes a video signal encoder 71 (TV Encoder), which is electrically connected to the data mixing device 78 After that, it can be used to convert the mixed image data ISg with a high chroma sampling format into a TV Video Signal Ts. This also illustrates the architecture of the video signal encoder 71 built into the system in this embodiment. (Jason: It is recommended to change IS1 in the high sampling frequency chroma format to IS1 'in Figure 7, and change IS1 in the medium sampling frequency chroma format to IS1 "to facilitate identification)

According to the device 60 of the embodiment shown in FIG. 6, a method for converting and mixing multiple image data to prevent data loss according to the present invention can be summarized in the following steps, and please refer to FIG. 8, which is a method of the present invention. Flowchart of the embodiment:

Step 100: Receive a first image data IS1 having a low sampling frequency chroma format and a second image data IS2 having a high sampling frequency chroma format, respectively.

Step 101: Convert the first image data IS1 having a low sampling frequency chroma format into the first image data isr having a high sampling frequency chroma format.

Step 102: After mixing the first image data IS with a high sampling frequency chroma format and the second image data IS2 with a high sampling frequency chroma format, output a chroma grid with high sampling frequency Mixed image data ISg;

Similarly, based on the device ^{79 in the} embodiment of FIG. 7 described above, and based on processing the first and second image data IS2 with three different sampling frequency chrominance formats of high, medium, and low, the present invention is used to convert and mix multiple An embodiment of another method for preventing image data from missing image data can be summarized in the following steps, and please refer to FIG. 9, which is a flowchart of another method embodiment of the present invention;

Step 200: Receive a first image data IS1 having a low sampling frequency chroma format and a second image data IS2 having a high sampling frequency chroma format, respectively.

Step 201: Convert the first image data IS1 having a low sampling frequency chroma format into the first image data IS1 having a high sampling frequency chroma format.

Step 202: Convert the first image data IS1 having a low sampling frequency chroma format into the first image data isr having a high sampling frequency chroma format.

Step 203: After mixing the first image data ISr with a high sampling frequency chroma format and the second image data IS2 with a high sampling frequency chroma format, output a mixed image data ISg with a high sampling frequency chroma format;

'Step 204: The mixed image data ISg having a high chroma sampling format is converted into a television video signal signal.

In fact, in actual implementation, the devices 60 and 70 shown in FIG. 6 and FIG. 7 are applied to a new type of moving image compression standard (MPEG-1, MPEG-2) and JPEG decoder (Decoder), so The aforementioned first image data IS1, second image data IS2, mixed image data ISg, high sampling frequency chroma format, medium sampling frequency chroma format, and low sampling frequency chroma sampling all conform to MPEG-1 and MPEG-2 Specifications (of which three (low, medium, high) sampling frequency chroma formats can correspond to the 4: ² : 0 sampling format, 4: ² : ² sampling format, and 4: 4 shown in Figures 1 to 3, respectively. : 4 sampling format), and the first image data IS1 may correspond to a main-picture image data of a digital video disc (VCD and DVD) format, and the second image data IS2 may correspond to a sub-image of a digital video disc format. Sub-Picture (SP) image data or screen image settings in a digital video disc format

(On-screen Display, 0SD) data (for ease of explanation, the sub-screen image data in this manual refers to these two types of data). Please refer to FIG. 10. FIG. 10 is a functional block diagram of an embodiment of an image data processing device 80 in actual implementation of the present invention, and can be regarded as a detailed embodiment of the embodiment of FIG. 6. The image data processing device 80 includes a main picture data receiving end 82, a pair of picture data receiving ends 84, a format conversion module 85, a data mixing device 88, and a video signal encoder 81. The main picture data receiving end 82 may receive a main picture image data having a 4: 2: 2 sampling format, the same as The sub-picture data receiving end 84 receives a sub-picture image data having a 4: 4: 4 sampling format. The format conversion module 85 is electrically connected to the main screen data receiving end 82, and is used for converting the main screen image data having a 4: 2: 2 sampling format into the main screen image data having a 4: 4: 4 sampling format after interpolation processing. Therefore, the format conversion module 85 can be regarded as a 4: ² : 2 to 4: 4: 4 format converter. The main screen image data with a 4: 4: 4 sampling format will be transmitted to the data mixing device 88, and after a mixing operation (Mixing Opemtion) with the sub screen image data with a 4: 4: 4 sampling format, a 4: 4: 4 Mixed image data in sampling format. Finally, the video signal encoder 81 electrically connected to the data mixing device 88 can convert the mixed image data with a 4: 4: 4 sampling format into a TV Video Signal. ₀

Please continue to refer to FIG. 10, the main screen image data with 4: 2: 2 sampling format includes a main brightness

(Luminance) Ym and a main chrominance (Cm), the sub-picture image data having a 4: 4: 4 sampling format includes a pair of luminance Ys, a first pair of chroma Ysb, and a second pair of chroma Ysr, The main screen image data having a 4: 4: 4 sampling format includes a main luminance Ym, a first main chroma CBm, and a second main chroma CRm. The first main chroma CBm and the second main chroma CRm are generated by the format conversion module 85 after the main chroma Cm is interpolated. Please refer to FIG. 11. FIG. 11 is a schematic diagram of brightness and chrominance of a plurality of image data in the transmitted data string in FIG. 10. The main luminance Ym data string contains multiple main luminances YmO, Yml, Ym2, ... obtained from different sampling points, and because the basic concept of the 4: 2: 2 sampling format is chrominance C (CR, CB) The sampling frequency is one-half of the luminance Y. Therefore, it can be seen that among the plurality of main chromaticities CBmO, CRmO, CBm2, CRm2, ... in the data string of the main luminance Cm, the main chromaticities CBm0, CRmO come from the same sampling point, And corresponding to the sampling point of the main luminance YmO, the main chroma CBm2, CRm2 and the main luminance Ym2 are from the same sampling point. The aforementioned interpolation process is to use a mathematical combination to double the sampling frequency of the main chromaticity Cm. Please refer to FIG. 11. The above numerical combination can be completed by a linear combination. Taking the newly added first main chromaticity CBma as an example, it can use the remaining first main chromaticities CBmO, CBm2, CBm4, · · · The components of. Are combined, as shown in the following mathematical formula: CBma = A-2 (nl) X CBm-2 (n- l) + '"+ A-2 X CBm- 2 + A0 X CBmO + A2 X CBm2 + · '· + A2n X CBm2n, where A-2 (n-1), ··· A-2, AO, A2, ········································· The The A2n are all constants, which respectively represent their corresponding main colors The proportion of the newly added first main chroma CBma is usually closer to the proportion of the newly added first main chroma CBma. For example, as shown in FIG. 11, the newly added first The main chroma CBma can be directly copied by copying the first main chroma CBmO or the first main chroma CBm2 (CBma = CBm0 or CBma = CBm2), or average the adjacent two first main chroma CBmO, CBm ² (Average ) To produce this new first primary color Degree CBma (CBma = 0.5 * CBmO + 0.5 * CBm2). In the same way, other first primary chromaticities and second primary chromaticities (such as CRma, CBmb, Crmb, etc. shown in FIG. 11) that need to be added can be generated by a method similar to or the same as the above.

Please continue to refer to Figure 10 and Figure 11. The mixed image data having a 4: 4: 4 sampling format generated by the data mixing device 88 has a mixed luminance Yg, a first mixed chrominance CBg, and a second mixed chrominance CRg. The mixing operation performed by the data mixing device 88 is to combine the main luminance Ym, the first main chroma CBm, and the second main chroma CRm with the sub luminance Ys, the first sub chroma CBs, and the second sub chroma CRs respectively. After a mathematical combination, the mixed brightness Yg, the first mixed chromaticity CBg, and the second mixed chromaticity CRg are generated. Please refer to FIG. 12, FIG. 12 is a schematic diagram of an embodiment of the data mixing device 88 of FIG. The main technical feature of the data mixing device 88 is that the sub-picture image data and the main-picture image data are mixed to become complete image data. The proportion of the image data, under normal circumstances, the sum of the values of A and B is 1, that is, A = (1-B). In this way, the value of the mixed brightness Yg can be expressed by the following mathematical formula: Yg = A * Ys + (l- A) * Ym; The value of the first mixed chromaticity CBg can be expressed by the following mathematical formula: CBg = A * CBs + (lA) * CBm; The value of the second mixed chromaticity CRg can be expressed by the following mathematical formula: CRg = A * CRs + (l- A) * CRm. Please refer to FIG. 13, FIG. 13 is a functional block diagram of another embodiment of the image data processing apparatus 80 of FIG. 10 in actual implementation. The format conversion module 85 in the embodiment of FIG. 13 includes a 4: 2 · 0 to 4: 2: 2 format converter 87 and a 4: 2: 2 to 4: 4: 4 format converter 89, 4: 2: The 0 to 4: 2: 2 format converter 87 can convert a main screen digital image data stored in a 4: 2: 0 sampling format (in a digital movie, such as a VCD or DVD, etc.) into the aforementioned 4: 2: The main screen image data in the 2 sampling format is then interpolated by the 4: 2: 2 to 4: 4: 4 format converter 89 to the main screen image data in the 4: 4: 4 sampling format.

According to the image data processing device 80 in the embodiments of FIG. 10 and FIG. 13, a method embodiment of the present invention in actual implementation can be summarized in the following steps in FIG. 14:

Step 300: Receive a main screen image data with a 4: 2: 2 sampling format and a

4: 4: 4 sub-picture image data in sampling format;

Step 301: Convert the main screen image data with a 4: 2: 2 sampling format into the main screen image data with a 4: 4: 4 sampling format;

Step 302: After mixing the main screen image data with a 4: 4: 4 sampling format and the sub screen image data with a 4: 4: 4 sampling format, output a mixed image data with a 4: 4: 4 sampling format; Step 303: Convert the mixed image data having a 4: 4: 4 sampling format into a television video signal signal.

Please note that in actual implementation, if the user sets up another known video signal encoder (such as the device previously applied to the moving image compression standard MPEG-1), the image data entering this known video signal encoder must be It conforms to the 4: 2: 2 sampling format. The image data processing device 80 in FIG. 10 and FIG. 13 can set a 4: 4: 4 to 4: 2: 2 format converter after the data mixing device 88, so that the user can switch between them. . It can be known from the above-mentioned embodiments that based on today's new dynamic image compression standards

(MPEG-1, MPEG-2) decoder technology, and in order to make full use of the advantages of the video signal encoder built into the DVD (or VCD) playback chip system, the main technical feature of the present invention is the sub-picture image data Before mixing with the main screen image data, both of the two image data are guaranteed to have a 4: 4: 4 sampling format (the highest sampling frequency chroma format), instead of the two as shown in the known technology of the embodiment of FIG. 4 The image data is mixed in the 2: 2 sampling format to avoid the loss of some chrominance information during the conversion of the sub-picture image data from the 4: 4: 4 sampling format to the 4: ² : 2 sampling format, resulting in Distortion of colors.

The above is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention shall fall within the scope of the patent of the present invention.

Claims

Claim

1. A device for converting and mixing a plurality of video data, each of which has a plurality of sampling formats, and the plurality of sampling formats includes at least a high sampling frequency chroma ( Chrominance) format and a low sampling frequency chroma format. The device includes:

A first data receiving end for receiving a first image data having the low sampling frequency chroma format;

A second data receiving end for receiving a second image data having the high sampling frequency chroma format;

'A format conversion module, electrically connected to the first data receiving end, for converting the first image data having the low sampling frequency chroma format into an image having the high sampling frequency after up-sampling processing. First image data in degree format; and

A data mixing device electrically connected to the format conversion module and the second data receiving end, and configured to convert the first image data having the high sampling frequency chroma format and the second image data having the high sampling frequency chroma format. After the mixing (Mk), a mixed image data having the high sampling frequency chroma format is output.

2. The device according to claim 1, further comprising a video signal encoder (TV Encoder) electrically connected to the data mixing device for converting the mixed image data having the high-chroma sampling format into a television. Video signal (TV Video Signal).

3. The device according to claim 1, which is applied to a new type of moving image compression standard (MPEG-1, MPEG-2) and JPEG decoder, and the first image data, the second The image data, the mixed image data, the high-sampling-frequency chroma format, and the low-sampling-frequency chroma sampling all conform to the MPEG-1 and MPEG-2 specifications.

4. The apparatus according to claim 3, wherein the first image data is corresponding to a digital video disc

(VCD and DVD) format main-picture (Main- Pictmre) image data, and the second image data is corresponding to a digital video disc format Sub-Picture (SP) image data or a digital video; Format screen image setting (On-sci-een Display, OSD) data.

5. The apparatus according to claim 3, wherein a Luminance-Chrominance Ratio Format (Luminance-Chrominance Ratio Format) of the high-sampling-frequency chrominance format is 4: 4: 4, and the luminance of the low-sampling-frequency chrominance format is 5. The chroma sampling ratio is 4: 2: 0.

6. The apparatus as claimed in claim 5, wherein the plurality of sampling formats further comprises a medium sampling frequency chrominance format, and a brightness chrominance sampling ratio of the medium sampling frequency chrominance format is 4: 2: 2, the format The conversion module also contains:

A first intermediate format converter for converting the first image data having the low sampling frequency chroma format into a first image data having the middle sampling frequency chroma format after a first interpolation process; and

A second intermediate format converter, electrically connected to the first intermediate format converter, for converting the first image data having the chroma format of the intermediate sampling frequency into a second sample having the high sampling frequency after a second interpolation process First image data in chroma format.

7. An image data processing device, comprising:

A main screen data receiving end is used to receive a main screen image data having a 4: ² : 2 sampling format, wherein the main screen image data having a 4: ² : 2 sampling format includes a main brightness value (Luminance) and a Main chrominance value (Chrominance);

A pair of picture data receiving end is used for receiving a sub picture image data having a 4: 4: 4 sampling format, wherein the sub picture image data having a 4: 4: 4 sampling format includes a pair of brightness values, a first pair A chroma value, and a second chroma value;

A format conversion module is electrically connected to the main screen data receiving end, and is used to interpolate the main screen image data with a 4: 2: 2 sampling format into an image with 4: 4: 4 after Upsamplin processing. Main screen image data in a sampling format, wherein the main screen image data in a 4: 4: 4 sampling format includes a main luminance value, a first main chrominance value, and a second main chrominance value;

A data mixing device is electrically connected to the format conversion and extraction module and the sub-picture data receiving end, and is used for the main screen image data having a 4: 4: 4 sampling format and the sub-picture having a 4: 4: 4 sampling format. After the frame image data is subjected to a Mixing Operation, a mixed image data having a 4: 4: 4 sampling format is output; and

A video signal encoder (TV Encoder) is electrically connected to the data mixing device, and is used for converting the mixed image data having a 4: 4: 4 sampling format into a TV Video Signal.

8. The image data processing device according to claim 7, wherein the blending operation is to respectively set the main luminance value, the first main chrominance value, and the second main chrominance value with the sub-luminance value, the first After a pair of chromaticity values and the second pair of chromaticity values are mathematically combined, a mixed luminance value, a first mixed chrominance value, and a second mixed chrominance are generated respectively. value.

9. The image data processing device according to claim 8, wherein the mixed image data having a 4: 4: 4 sampling format includes the mixed luminance value, the first mixed chrominance value, and the second mixed chrominance value.

10. The image data processing device according to claim 7, further comprising a signal format converter electrically connected to the main screen data receiving end, for storing a main screen stored in a 4: 2: 0 sampling format. The digital image data is converted into the main screen image data with a 4: ² : 2 sampling format.

11. The image data processing device according to claim 7, wherein the main screen image data is main screen image data corresponding to a digital video disc (VCD and DVD) format, and the sub screen image data is corresponding to a digital video disc Sub-picture (SP) image data in a format or on-screen display (OSD) data in a digital video disc format.

12. The image data processing device according to claim 7, which is compliant with a new type of moving image compression standard (MPEG-1, MPEG-2) and JPEG.

13. A method for converting and mixing a plurality of image data (Video Signal) to prevent data loss, each of the plurality of image data has multiple sampling formats (Sampling Fotrmat;), the multiple sampling formats include at least one high Sampling frequency chrominance format and a low sampling frequency chrominance format. The method includes:

Receiving a first image data having the low sampling frequency chroma format and a second image data having the high sampling frequency chroma format, respectively;

Converting the first image data having the low sampling frequency chroma format into the first image data having the high sampling frequency chroma format; and

After mixing (Mix) the first image data having the high sampling frequency chroma format and the second image data having the high sampling frequency chroma format, a mixed image data having the high sampling frequency chroma format is output.

14. The method according to claim 13, which is applied to a decoder (Decodeir) of a new type of moving image compression standard (MPEG-1, MPEG-2), and the first image data and the second image The data, the mixed image data, the high-sampling-frequency chroma format, and the low-sampling-frequency chroma sampling all conform to the specifications of MPEG-1 and MPEG-2.

15. The method according to claim 14, wherein the first image data is corresponding to a main-picture image data in a digital video disc (VCD, DVD) format, and the second image data is corresponding to a Sub-Picture (SP) image data in digital video disc format or On-screen Display (OSD) data in digital video format.

16. The method according to claim 14, wherein the plurality of sampling formats further comprises a sampling frequency chroma format, and the method further comprises:

Converting the first image data having the low sampling frequency chroma format into the first image data having the middle sampling frequency chroma format; and

Converting the first image data having the middle sampling frequency chroma format into the first image data having the high sampling frequency chroma format;

The medium chroma sampling format conforms to the specifications of MPEG-1 and MPEG-2.

17. The method according to claim 16, wherein a luminance chrominance sampling ratio (CLuminance-Cliirominance Ratio Format) of the high sampling frequency chrominance format is 4: 4: 4, and the luminance chrominance of the medium sampling frequency chrominance format The sampling ratio is 4: 2: 2; the luminance and chrominance sampling ratio of the low sampling frequency chrominance format is 4: 2: 0.

18. A method for processing image data in an image data processing device, the image data processing device comprising a signal receiving module, a format conversion module, and a data mixing device, the method comprising:

Using the signal receiving module to receive a main screen image data with a 4: 2: 2 sampling format and a sub screen image data with a 4: 4: 4 sampling format;

Using the format conversion module to convert the main screen image data with a 4: 2: 2 sampling format into the main screen image data with a 4: 4: 4 sampling format; and

After using the data mixing device, the main screen image data having a 4: 4: 4 sampling format and the sub screen image data having a 4: 4: 4 sampling format are subjected to a mixing operation, and a output having 4: Mixed image data in 4: 4 sampling format.

19. The method of claim 18, wherein the image data processing device further comprises a video signal encoder (TV Encoder) electrically connected to the data mixing device, and the method further comprises:

The video signal encoder is used to convert the mixed image data with a 4: 4: 4 sampling format into a TV Video Signal.

20. The method as claimed in claim 18, wherein the image data processing device further comprises a signal format converter, which is electrically connected to the signal receiving module and used for storing a master stored in a 4: 2: 0 sampling format. The picture digital image data is converted into the main picture image data having a 4: 2: 2 sampling format.

21. The method of claim 18, wherein the main screen image data having a 4: 2: 2 sampling format includes a main luminance value (Luminance) and a main chrominance value (ChtOminance); and the method has 4: 4: 4 The sub-picture image data in the sampling format includes a pair of luminance values, a first pair of chrominance values, and a second Sub-chrominance value; the main screen image data having a 4: 4: 4 sampling format includes a main luminance value, a first main chrominance value, and a second main chrominance value; the 4: 4: 4 sampling The formatted mixed image data includes a mixed luminance value, a first mixed chrominance value, and a second mixed chrominance value.

22. The method of claim 21, further comprising:

Using the format conversion module to generate the first main chroma value and the second main chroma value after processing the main chroma value by interpolation (Up-samHn ^); and

Using the data mixing device, the main luma value, the first main chroma value, and the second main chroma value with the sub chroma value, the first sub chroma value, and the second sub chroma, respectively After the values are mathematically combined, the mixed luminance value, the first mixed chrominance value, and the second mixed chrominance value are generated respectively.

23. The method of claim 18, wherein the main frame image data is main frame image data corresponding to a digital video disc (VCD, DVD) format, and the sub frame image data is a sub frame corresponding to a digital video disc format. Sub-picture (SP) image data and on-screen display (OSD) data in a digital video disc format.

24. The method according to claim 18, which is applied to a new moving image compression standard

(MPEG-1, MPEG-2) decoder.