JPH11262031A - Processing system for color image signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce artifacts such as a fogged edge in the processing of a color image signal. SOLUTION: This system includes a color coder 2 that codes a color signal into a 1st format signal, a sub-sampler 3 that converts a 1st format signal into a 2nd format signal, a re-sampler 7 that applies re-conversion to the 2nd format signal into the 1st format signal, and a color decoder 8 that decodes the 1st format signal which is re-converted into a color signal. The re-sampler 7 includes a process, where a chrominance signal is re-sampled by using a luminance signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the processing of a color image signal, and more particularly to a technique for converting a decoded color image format.

[0002]

2. Description of the Related Art Image / video coding standards, such as JPEG and MPEG, have four types of image formats:
A 2: 0 format is used. JPEG 4:
Although called the 1: 1 format, this is basically the same as the 4: 2: 0 format. Prior to encoding according to these specifications, the chrominance signal of the original sequence is subsampled by a factor of two in both the horizontal and vertical directions. Taking advantage of the lower visual sensitivity of the chrominance signals, they can be sub-sampled, which helps greatly in reducing the input data rate by half.

[0003] Once subsampling has been applied at the encoder, a type of inverse operation, resampling, is required at the decoder, particularly for displaying the output image and making a hard copy thereof. For example, taking a 4: 2: 0 format, the decoded chrominance signal may be transmitted in two directions in two directions.
Must be resampled in multiples of However, both sub-sampling and re-sampling are out of specifications, and thus a robust re-sampling algorithm is required.

In the conventional method, an interpolation function characterizing the resampling algorithm is indiscriminately applied to the entire image. The most widely used interpolation functions are nearest neighbor, bilinear, sinc, and cubic spline. From a digital signal processing perspective, these filters except the last one can be configured as FIR filters and the cubic splines are configured as IIR filters. However, the conventional method is
No matter how complicated the chosen function is, the resulting image often suffers from aliasing, edge blurring and other artifacts.

[0005]

SUMMARY OF THE INVENTION In the present invention, a novel chrominance resampling algorithm for color image coding is used. In this method, a luminance signal indicating the structure of a target (object) is used to resample the chrominance signal.

To avoid artifacts or preserve details during the resampling described above, adaptive algorithms have been introduced. In general, adaptive algorithms for image resampling are based on dynamically segmenting the image into homogeneous regions, and then applying some interpolation within the homogeneous regions. To do this, it is necessary to analyze the local structure of the input image so that a particular interpolation function in each region can be adjusted. There are many ways to make the interpolation function adaptable. Examples are block-shaped distortion models, edge-shaped distortion models, and directional sensitivity based on statistical analysis.

Conversion of a digital image from 4: 2: 0 to 4: 4: 4 is an example of such a resampling method. However, in this case, very useful information is available which tells the structure of the target image, ie the image of the original dimensions. This is a luminance signal that is available at any time during the encoding process. This means that information exhibiting adaptability can be derived from the luminance signal. Since each color component has almost no correlation, the pixel value itself of the luminance signal does not have information for this purpose. However, the luminance signal carries structural information, which helps to segment the image into homogeneous regions. One example of this process in the present invention is based on the binarization of the luminance signal. In this context, a binary index is introduced here. Each luminance pixel is classified into a low level or a high level using a threshold. Then, say 2
Apply adaptive resampling to the chrominance plane with reference to the hexadecimal exponent. The simulation result is
It shows that the scheme in the present invention is more effective than the linear resampling method of the prior art.

[0008] The color image signal processing system of the present invention comprises:
A color encoder that encodes a color signal into a first format signal; a subsampler that converts the first format signal into a second format signal; and a subsampler that converts the second format signal into a first format signal. A resampler for converting, the resampler resamples a chrominance signal using a luminance signal, and a color decoder for decoding the reconverted first format signal into a color signal.

[0009]

An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall block diagram of a color image signal processing system according to an embodiment of the present invention, in which a data format in each process is described. This figure shows that the resampler 7 performing the resampling process
It is shown that it is arranged between the image decoder 6 and the color decoder 8. YUV for both 4: 4: 4 and 4: 2: 0
Represents the image format in color space.

The color image signal processing system according to the present embodiment
A color encoder 2 for encoding a color signal into a first format signal, a subsampler 3 for converting the first format signal into a second format signal, and a second sampler for converting the second format signal into a first format signal. It includes a resampler 7 for conversion and a color decoder 8 for decoding the reconverted first format signal into a color signal. Resampler 7 includes a process for resampling the chrominance signal using the luminance signal.

The relationship between 4: 4: 4 and 4: 2: 0 formats is shown in FIG. This figure only takes up the ratio of the dimensions of the image, which ratio can be applied both at the block level and at the video level. The U and V components are also represented as Cb and Cr, respectively. A schematic example of this new algorithm in a block-level process is shown in FIG. The scheme consists of two sub-processes: index acquisition and adaptive resampling. In this scheme, 4x4 chrominance blocks are interpolated into 8x8 blocks with reference to the corresponding 8x8 luminance blocks.

The process of obtaining the exponent is similar to block truncation coding (BTC), which allows regions containing details to be well represented by a pair of sample moments. To reduce computational complexity, this sub-process must be performed once per block, but this block should be larger than the size of the filter mask or window. As an example, an index is obtained in units of 8 × 8 luminance blocks. This means that 4x4 chrominance samples are interpolated into 8x8 blocks. For each block, the threshold is first determined.
Here, we use only one threshold that must be appropriate to classify a pixel into two populations. This also helps to suppress noise around the area occupied by the edge. Assume an ideal step-shaped edge that is somewhat broken by some noise. Finding a threshold in such a large block allows the pixels to be more appropriately and easily classified into two separate populations than searching for a threshold in a smaller block. Also, 8 ×
Another reason for using eight blocks is that JPEG also uses MPEG-
1/2 also means that a block of this size is employed. There are several candidates for the threshold. That is, the mean, median, mode or middle of the dynamic range,
The center of the dynamic range is preferred as the threshold. This is because populations are generally more distinguishable than others. The threshold value τ is defined as follows, where max and min are the maximum value and the minimum value in the block, respectively.

[0013]

Τ = (max + min + 1) / 2 Next, a binary index λ (i) is obtained according to whether the luminance pixel value ν (i) is larger or smaller than a threshold value.

[0014]

(Equation 2) Tables 1 and 2 show the test image “Lenna” (4:
The pixel value and the binary index of each of the luminance component and chrominance component obtained from the 4: 4 format (512 × 480 pels) are shown. The value in parentheses indicates the exponent.

[0015]

[Table 1]

[0016]

[Table 2]

The above table shows that the luminance and chrominance components are 2
In the form of a hexadecimal exponent, it indicates that it is coherent.
In this case, it is recognized that the chrominance binary index is the reciprocal of the luminance binary index at any coordinate. This result is consistent with the hypothesis that the local structure will be common to both components. This property is called mutual chrominance coherence. However, it is not always guaranteed that pure coherence as described above is obtained. However, it is usually 85% or more certain.

The chrominance components are sub-sampled through some preprocessing in the encoder. Figure 4 shows 4:
In the 2: 0 format, the pel position of luminance and chrominance is shown, x indicates luminance, and ○ indicates pel position of chrominance. In practice, this sampling pattern is compatible with MPEG and H.264. Widely used by existing standards such as 26x. FIG. 5 shows a sample-and-hold that allows the 4: 2: 0 image to be easily converted to a 4: 4: 4 image, assuming the sampling pattern of FIG. One chrominance sample is copied to the four surrounding pels. In the present invention, this sample hold is used in the first stage of the processing.

By applying a linear sub-sampling filter to the chrominance signals of Table 2, 4 × 4 blocks are obtained as shown in Table 3. Thereafter, the 4 × 4 block can be resampled by a sample and hold as shown in Table 4. In the following experiments, the square error (SE) is used as a measure of distortion. The reference signal is as shown in Table 2.

[0020]

[Table 3]

[0021]

[Table 4]

In the flat region, the choice of the interpolation function has no significant effect on the quality of the output. That is, in such a highly correlated area, arbitrary interpolation, extrapolation, or replication of pixels is performed.
) Works well. In contrast, for details such as object boundaries and textures, the interpolation function must be carefully chosen to avoid artifacts such as edge blur. In this regard, the resampling scheme needs to be concentrated in more detail than in the flat region.

First, consider the case of estimating the edge location from the chrominance data itself. This is very similar to the work on the luminance component. There is a great deal of research in this area. Some schemes are simple nearest neighbor, linear, s
Inc, an interpolation function in a range up to a spline based on an image model is used. In a sense, they are all in the same category, addressing the scheme based on components.
However, in the present invention, a new resampling method based on an inter-component model is used. This scheme can be derived from the fact that all color components necessarily represent the same individual or object in the real world.
In the original image in 4: 4: 4 format, the location of the edge in luminance must match the same edge in the chrominance plane. Assuming this is correct, luminance may help to interpolate chrominance samples when converting the image from 4: 2: 0 to 4: 4: 4 using a resampling process.

If x and y are input and output, respectively,
The mutual chrominance resampling algorithm can be written as:

[0025]

(Equation 3) Where n represents the number of input samples required to produce an output. The value of n is related to the filter mask used.
By defining an appropriate filter mask or window, the above equation can be rewritten as:

[0026]

(Equation 4) An example of a filter mask is shown in FIG. Using the filter mask of the example of FIG. 6, the scheme according to the present invention can be adjusted as shown in Equation 5. On the other hand, linear resampling using the same mask can be expressed as in Equation 6.

[0027]

(Equation 5)

[0028]

(Equation 6) Here, an interpolated resampling signal is obtained from the subsampled chrominance signals shown in Table 4.
Table 5 shows the case of linearity, and Table 6 shows the case in the present invention. In terms of squared error, this example is one more than the linear method.
It is superior only by 1.2% ((412-343) / 617).

[0029]

[Table 5]

[0030]

[Table 6]

The scheme of the present invention is independent of the filter mask used for resampling. That is, 3 × 3 and 5
A cross-shaped or square mask such as x5 or a vertical filter is effective. Most important is the mutual chrominance
Applying any of these filters to homogeneous regions based on coherence. It is expected that the quantization error will affect the decoded image. Therefore, it is necessary to set the quantization parameter low enough to save the target structure. This condition can be met by using most image / video coding standards in practice. 1 in JPEG
Perceptually lossless playback can be achieved with compression ratios up to 5: 1 and up to 40: 1 for both MPEG-1 and MPEG-2.

In the present invention, 4: 2: 0 to 4:
It can be used in any format where the chrominance component must be resampled, such as 2: 2, and 4: 2: 2 to 4: 4: 4. However, some adjustments are needed to accommodate differences in sample format. The exponent used may take another form, ie, ternary or higher.

Table 7 shows the results of a simulation using a test image “Lenna” of 512 × 480 pels. At the video level, the conventional linear scheme has a larger square error (SE) than the sample-and-hold, but the scheme according to the present invention reduces it. Linear systems tend to smear fine details, such as hair. The preservation of details in the manner of the present invention can be said to take advantage of the benefits of mutual chrominance coherence.

[0034]

[Table 7]

The improvement in visual quality can be noticed in the case of still images, especially when using hard copies. Although the invention has been described with reference to an embodiment, the description is not limited to this.

[0036]

As described above, even when converting the format of a color image, high-definition details can be stored accurately.

[Brief description of the drawings]

FIG. 1 is a block diagram of a system according to an embodiment of the present invention.

FIG. 2 is a view for explaining the relationship between 4: 4: 4 format and 4: 2: 0 format.

FIG. 3 is a diagram illustrating resampling at a block level according to the present invention.

FIG. 4 is a diagram showing pel positions of luminance and chrominance in a 4: 2: 0 format.

FIG. 5 is a diagram showing a sample hold for the sampling pattern of FIG. 4;

FIG. 6 is a diagram showing an example of a filter mask.

[Explanation of symbols]

 Reference Signs List 1 source 2 color encoder 3 subsampler 4 video / image encoder 5 transfer and / or holding unit 6 video / image decoder 7 resampler 8 color decoder 9 display device

Claims (1)

[Claims] A color encoder that encodes the color signal into a first format signal; a subsampler that converts the first format signal into a second format signal; and a subsampler that converts the second format signal into a first format signal. A resampler that reconverts the chrominance signal using a luminance signal, and a color decoder that decodes the reconverted first format signal into a color signal. And a color image signal processing system.