JP2001177831A - Compressed image converter and compressed image conversion method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce deterioration in image and to decrease an arithmetic cost in the case of applying color conversion to a compressed image to obtain a desired compressed image. SOLUTION: The compressed converter of this invention is provided with a conversion coefficient decision means 105 that decides a modification coefficient to convert a luminance signal and a color difference signal of an image and with a spatial frequency component conversion means 106 that uses the conversion coefficient decided by the conversion coefficient decision means to convert the spatial frequency component of the image and outputs the converted spatial frequency component.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a compressed image conversion apparatus for performing a specific process on a compressed image obtained by DCT (Discrete Cosine Transform), a compressed image conversion method, and a recording medium on which a program is recorded.

[0002]

2. Description of the Related Art In recent years, the efficiency of storage and transmission of digital still images and digital moving images has been improved. In order to increase the efficiency of storage and transmission of these digital images, compressed data is often used as it is from immediately after image input to immediately before output.

A conventional compressed image conversion device for converting the color of a compressed image employs a method of decoding the compressed image into pixel components, correcting the color, and then encoding the compressed image again. The following is a brief description of a compressed image conversion device employing these techniques.

FIG. 9 is a hardware block diagram of a conventional compressed image conversion device. Here, JFIF (JPEG File Interchan) is used as image compression processing.
geFormat). The compressed image input from the input device 901 is converted into a file stream
Are converted by the Huffman decoding unit 903 and the inverse quantization unit 904. The converted compressed image is represented by N shown in FIG.
B0 blocks 10 composed of × N spatial frequency components
Output as 01. Here, a description will be given assuming that N = 8. The distribution of high-frequency components is shown from the upper left to the lower right in the drawing of the block 1001. In particular, the value of the upper left coordinate (0, 0) may be referred to as a DC (direct current) component, and the other coordinate values may be referred to as an AC (alternating current) component.

Next, the frequency component is subjected to inverse DCT (inverse discrete cosine transform) expressed by the following (Equation 1) by the inverse DCT unit 905, and is converted into an 8 × 8 pixel component as shown in a block 1002 shown in FIG. Is converted.

[0006]

(Equation 1) Y, Cb, Cr: frequency components y, cb, cr: pixel components B (B = 0 to B0-1): block Next, the pixel components are calculated using the color correction coefficients output by the conversion coefficient determination unit 906. The pixel component conversion unit 907 performs correction or conversion of luminance and hue. The converted pixel component is again converted into a spatial frequency component by a DCT unit 908 that performs DCT (discrete cosine transform) expressed by the following (Equation 2).

[0007]

(Equation 2) The data output from the DCT unit 908 is processed by the quantization unit 909, the Huffman coding unit 910, and the file stream synthesis unit 911, and is output again as a compressed image. Then, the file stream synthesizing unit 911
The compressed image output from is transmitted to the accumulation / transmission unit 912.

Japanese Patent Application No. 6-146561 discloses an improved example of such a compressed image converter. This compressed image conversion device, when there is a color change,
The DC component is corrected based on a conversion table or a conversion function obtained in advance. Then, only low frequency components including DC components are encoded. By doing so, the compressed image conversion device reduces the number of calculations.

[0009]

However, the conventional compressed image converter repeats DCT and inverse DCT every time color conversion or correction is performed. Therefore, DCT, inverse DCT
There is a problem that the calculation error accumulates due to the repetition of, causing image deterioration.

[0010] In image encoding / decoding, DCT is used.
There is also a report that the ratio of the computation cost is large. Furthermore, if the decoding block and the coding block are configured by different processors and devices, the previous quantization information,
Since the entropy coded information is not reflected, image deterioration further increases due to errors.

Further, performing color correction of only the DC component as in the compressed image conversion apparatus described in Japanese Patent Application No. Hei 6-146561 means correcting the average value of pixels of 8 × 8 blocks. are doing. Therefore, color correction of pixels other than the average value is an approximate value. For this reason, the image deteriorates. Furthermore, there is a problem in that encoding of only low-frequency components including DC components has a high possibility of generating block noise or the like peculiar to a compressed image.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to reduce the image deterioration and the calculation cost when a desired compressed image is obtained by performing color conversion on a compressed image. I do.

[0013]

According to the present invention, there is provided an image processing apparatus comprising: a conversion coefficient determining unit for determining a deformation coefficient for converting a luminance signal or a color difference signal of an image; and the conversion coefficient determined by the conversion coefficient determining unit. A spatial frequency component converting means for converting the spatial frequency component of
Was provided.

By directly converting the spatial frequency of an image in this way, it is possible to reduce the image deterioration and the calculation cost.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A compressed image conversion apparatus according to a first aspect of the present invention includes an inverse quantization means for inversely quantizing quantized data and outputting a spatial frequency component of the image, and a luminance signal of the image. And transform coefficient determining means for determining a deformation coefficient for converting a color difference signal, and using the transform coefficient determined by the transform coefficient determining means, transforms a spatial frequency component of the image output from the inverse quantization means, A spatial frequency component converting unit that outputs the converted spatial frequency component; and a quantizing unit that quantizes the spatial frequency component output by the spatial frequency component converting unit and outputs quantized data.

According to this configuration, the DCT means and the inverse DCT means are not required by directly converting the spatial frequency of the image. As a result, it is possible to reduce image deterioration and calculation cost.

According to a second aspect of the present invention, there is provided a compressed image converting apparatus for inversely quantizing quantized data and outputting a spatial frequency component of the image, and an output from the inverse quantizing means. Up-sampling correction means for correcting the spatial frequency component of the image, a conversion coefficient determination means for determining a conversion coefficient for converting a luminance signal or a color difference signal of an image, and the conversion coefficient determined by the conversion coefficient determination means. A spatial frequency component converting unit that converts a spatial frequency component of the image output by the sampling correcting unit and outputs the converted spatial frequency component; and a down-sampling correction that corrects the spatial frequency component output from the spatial frequency component converting unit. Means for quantizing the spatial frequency component output from the downsampling correction means, and outputting the quantized data. And means, equipped with a.

With this configuration, it is possible to perform the same color correction on an image in which color difference components are thinned out, such as the 4: 2: 2 format or the 4: 2: 0 format.

According to a third aspect of the present invention, there is provided a compressed image conversion apparatus for inversely quantizing quantized data and outputting a spatial frequency component of an image, and an output from the inverse quantization means. Sampling correction means for correcting the spatial frequency component of the image, a conversion coefficient determination means for determining a conversion coefficient for converting a luminance signal or a color difference signal of an image, and the sampling correction using the conversion coefficient determined by the conversion coefficient determination means. Transforms the spatial frequency component of the image output by the means,
A spatial frequency component converting means for outputting the converted spatial frequency component, and a quantizing means for quantizing the spatial frequency component output from the spatial frequency component converting means and outputting quantized data are provided.

With this configuration, it is possible to perform the same color correction on an image in which color difference components are thinned out, such as the 4: 2: 2 format or the 4: 2: 0 format. In addition, since the down-sampled color correction is performed as it is, the amount of calculation can be reduced.

According to a fourth aspect of the present invention, there is provided a compressed image conversion apparatus for separating an I picture and other pictures from a GOP of an MPEG compressed image and outputting the separated pictures, and an I / O output from the picture separating means. A variable-length decoding unit that performs variable-length decoding of a picture and outputs quantized data; and an inverse unit that inversely quantizes the quantized data output from the variable-length decoding unit and outputs a spatial frequency component of an image. Quantization means, upsampling correction means for correcting a spatial frequency component output from the inverse quantization means, conversion coefficient determination means for determining a conversion coefficient for converting a luminance signal or a color difference signal of an image, and the conversion coefficient The spatial frequency component of the image output from the up-sampling correction unit is converted using the conversion coefficient determined by the determination unit, and the converted spatial frequency is converted. A spatial frequency component converting means for outputting the spatial frequency component, a downsampling correcting means for correcting the spatial frequency component output from the spatial frequency component converting means, and a spatial frequency component output from the downsampling correcting means. Quantizing means for outputting the quantized data, variable-length coding means for performing variable-length coding on the quantized data output from the quantizing means, and output as an I-picture, and output from the variable-length coding means. And a picture combining means for combining pictures other than the I picture output from the picture separating means.

With this configuration, the compression format is MPEG
Can also respond. Thereby, an effect of approximate color space conversion can be obtained.

According to a fifth aspect of the present invention, there is provided a compressed image converting apparatus comprising: a picture separating means for separating an I picture and other pictures from a GOP of an MPEG compressed image and outputting the separated picture; and an I picture output from the picture separating means. A variable-length decoding unit that performs variable-length decoding of a picture and outputs quantized data; and an inverse unit that inversely quantizes the quantized data output from the variable-length decoding unit and outputs a spatial frequency component of an image. Quantization means, and sampling correction means for correcting the spatial frequency component output from the inverse quantization means,
A conversion coefficient determining unit for determining a conversion coefficient for converting a luminance signal or a color difference signal of the image; and a conversion coefficient determined by the conversion coefficient determination unit, and converting a spatial frequency component of the image output from the sampling correction unit. A spatial frequency component converting unit that outputs a converted spatial frequency component, a quantizing unit that quantizes the spatial frequency component output from the spatial frequency component converting unit, and outputs quantized data, Variable length coding means for performing variable length coding on the quantized data output from the means and outputting it as an I picture; an I picture output from the variable length coding means; and an I picture output from the picture separation means. Picture combining means for combining pictures other than pictures.

With this configuration, the compression format is MPEG
Can also respond. Thereby, an effect of approximate color space conversion can be obtained. In addition, since the down-sampled color correction is performed as it is, the amount of calculation can be reduced.

According to a sixth aspect of the present invention, in the compressed image conversion device according to any one of the first to fifth aspects, the inverse quantization means inversely quantizes the quantized data, The result of adding the intermediate value of the conversion width is output.

As described above, by adding the intermediate value of the quantization width to the spatial frequency component, the operation error of the requantization is reduced.

According to a seventh aspect of the present invention, in the compressed image conversion apparatus according to any one of the first to sixth aspects, the spatial frequency conversion means adjusts an offset of one or more attribute values or an attribute. At least one of the value gain adjustments is performed.

As described above, since the gain and the offset are adjusted for each component of the image, the quality of the image is improved at the time of color conversion such as brightness and contrast adjustment.

According to an eighth aspect of the present invention, in the compressed image conversion device according to any one of the first to sixth aspects, the spatial frequency conversion means performs n × n matrix conversion.

With this configuration, complicated color conversion, for example, color conversion between a standard TV color space and a high definition TV color space can be performed.

According to a ninth aspect of the present invention, in the compressed image conversion apparatus according to any one of the first to eighth aspects, the conversion coefficient determining means is configured to switch from the first color system to the second color system. Outputs the conversion coefficient to.

With this configuration, color conversion between different color systems can be performed at high speed using the deformation coefficient.

According to a tenth aspect of the present invention, in the compressed image conversion device according to any one of the first to eighth aspects, the conversion coefficient determining means outputs a coefficient for converting a white balance.

With this configuration, it is not necessary to perform inverse DCT and the number of white operations can be reduced.

According to an eleventh aspect of the present invention, an inverse quantization step of inversely quantizing the quantized data and outputting a spatial frequency component of the image, and determining a conversion coefficient for converting a luminance signal or a color difference signal of the image. Transform coefficient determining step, using the transform coefficients of the transform coefficient determining step, to transform the spatial frequency component of the image output from the inverse quantization step,
A spatial frequency component converting step of outputting the converted spatial frequency component; and a quantization step of quantizing the spatial frequency component output from the spatial frequency component converting step and outputting quantized data. This is an image conversion method.

According to a twelfth aspect of the present invention, there is provided an inverse quantization procedure in which a computer inversely quantizes quantized data to output a spatial frequency component of an image, and a conversion for converting a luminance signal or a color difference signal of the image. The procedure for determining the coefficients, using the transform coefficients determined in the transform coefficient determination procedure, to transform the spatial frequency component of the image output in the inverse quantization procedure,
A spatial frequency component conversion procedure for outputting the converted spatial frequency component, and a quantization procedure for quantizing the spatial frequency component output in the spatial frequency component conversion procedure and outputting the quantized data. Is a computer-readable storage medium storing the program.

Embodiment 1 Hereinafter, Embodiment 1 of the present invention will be described with reference to the accompanying drawings. FIG.
FIG. 2 is a hardware block diagram of the compressed image conversion device according to the first embodiment.

Image compression / conversion device 1 according to the first embodiment
00 is an input device 10 for inputting a digital compressed image.
1 is provided. The input device 101 is, for example, a digital camera. Here, JFIF (JPEG File Interchan) is used as image compression processing.
ge Format).

The image compression / conversion device 100 includes a file stream decomposing unit 102, a Huffman decoding unit 103, and an inverse quantization unit 104, which are image decompression means for decompressing a compressed image input from the input device 101. Are provided.

The file stream decomposing unit 102 converts the digitally compressed image input from the input device 101 into a JFI
The F-header is analyzed and Huffman-encoded data is output. The Huffman decoding unit 103 decodes the Huffman-encoded data by the file stream decomposing unit 102 using a Huffman table obtained from a stream (data sequence), and outputs a plurality of quantized DCT coefficient blocks. The inverse quantization unit 104 outputs a plurality of DCT coefficient blocks from the plurality of DCT coefficient blocks quantized by the Huffman decoding unit 103, using a quantization table obtained from the stream.

The compression image conversion device 100 includes a deformation coefficient determination unit 1 as color conversion means for performing color conversion of an image.
05 and a frequency component converter 106 are provided.

The conversion coefficient determining section 105 is a means for determining a color correction coefficient. The frequency component conversion unit 106 directly converts the DCT coefficient input from the inverse quantization unit 104 using the determined color correction coefficient.

The compressed image conversion apparatus 100 includes a quantization unit 107, a Huffman encoding unit 108, and a file stream synthesizing unit 109, which are image compression means for recompressing the color-converted image. I have.

The quantizing unit 107 includes a frequency component converting unit 10
The DCT coefficient block transformed by 6 is again quantized using the quantization table obtained from the stream. The Huffman coding unit 108 performs the Huffman coding again on the data quantized by the quantization unit 107 using a Huffman table obtained from the stream. The file stream synthesizing unit 109 adds a header or the like to the data that has been Huffman-coded by the quantization unit 107, and reconstructs a JFIF stream.

An image storage / transmission unit 110 for storing and transmitting the compressed image is provided in the compressed image conversion apparatus 100.
Is provided. As the image storage / transmission unit 110,
There is a web server or the like that stores and transmits JPEG images.

Next, the operation of the compressed image converter according to the first embodiment will be described. In the present embodiment, color adjustment is performed by adjusting the gain and offset of each component of an image. As described above, since the gain and the offset are adjusted for each component of the image, the quality of the image is improved during the color conversion.

In the above configuration, the JFIF image created by the input device 101 is subjected to JFIF header analysis, frame decomposition,
And a luminance component Y, chrominance components Cb and Cr, and Huffman decoding and output. The Huffman-encoded data is sent to the Huffman decoding unit 103, converted into a plurality of quantized 8 × 8 block DCT coefficients, and output.
The quantized data is processed by the inverse quantization unit 104 in FIG.
As shown in (a), inverse quantization is performed, and a DCT coefficient which is a spatial frequency component of a luminance component Y and color difference components Cb and Cr of an 8 × 8 block is output. FIG. 2A shows a case where the quantization widths are all two. The horizontal axis Q represents the quantized data, and the vertical axis Y represents the DCT coefficient. Further, as shown in FIG. 3, the conversion coefficient determination unit 105 determines a color correction coefficient for adjusting the gain and offset of each component,
Output. The following is a conversion formula (Equation 3) for converting the frequency component of a pixel. The color correction coefficient is input to the input device 10.
The one given in advance corresponding to 1 is selected. The color correction coefficient is a coefficient for adjusting brightness, contrast, or hue of an input image. By using the color correction coefficient, the color balance of the entire image can be adjusted at high speed.

[0048]

(Equation 3) The DCT coefficient is transformed by the following transformation equation (Equation 4) using the transformation coefficient (Equation 3). Here, q is a normalization coefficient for correcting the scale in the spatial domain and the frequency domain, and in this case, q = 8. The values of the gain a and the offset b in (Equation 4) are the same as those in (Equation 3).

[0049]

(Equation 4) That is, as shown in FIG. 4, the calculation is directly performed on each frequency component. As shown in (Equation 4), the offset b is DC
The component is added only to the (0,0) component. Further, the gain a is added to all components including the (0, 0) component. In other words, the frequency component conversion unit 106
A process paying attention to the principle of the orthogonal series conversion of (Expression 1) and (Expression 2) of the conversion is performed. In other words, the DCT transform equation is an equation that integrates the product of some calculated value and the orthogonal series. That is, the processed orthogonal coefficients are simply added. Also, since the series of the DCT transform satisfies the orthogonal condition, each orthogonal coefficient can be calculated independently. Therefore, the same result can be obtained by converting each frequency component to a pixel component, then performing a linear conversion, and then again converting each pixel component to a frequency component, or simply linearly converting each frequency component. become. Therefore, the same effect can be obtained by linearly converting each frequency component as in linearly converting each pixel component.

The DCT coefficient thus transformed is converted by the quantization unit 107 into an inverse operation of the inverse quantization shown in FIG. 2A, that is, in this case, the quotient divided by the quantization width 2 is obtained. Output as quantized data.

The quantized data is supplied to the Huffman encoding unit 108
And a Huffman encoding process, a luminance component Y, a chrominance component Cb, a Cr combination, a frame combination, and a JFIF header addition process by the file stream combining unit 109 and output as a JFIF image. And this output JFIF
The image is stored by the image storage / transmission unit 110 and the LA
It is transmitted on N or WAN.

However, when the quantization and the inverse quantization shown in FIG. 2A are performed, a problem occurs in the values before and after the quantization. Specifically, there is no problem when the frequency component conversion unit 106 integrates a value of 1.0 or more. However, when the value obtained by integrating the value less than 1.0 is requantized, the value to be integrated is 1. Even if the value is very close to 0 (for example, 0.999), the quantized value after the conversion is always smaller than the quantized value before the conversion. In other words, errors in the quantized value accumulate before and after the integration of less than 1.0. (Table 1) is a table showing errors occurring before and after the quantization.

[0053]

[Table 1] Therefore, as shown in FIG.
Then, の of the quantization width is added to the inversely quantized value.
The results are shown in (Table 2).

[0054]

[Table 2] As is clear from (Table 2), it is understood that the error before and after quantization when the values before and after about 1.0 are integrated is solved.

As described above, according to the first embodiment, by directly converting the spatial frequency of an image,
Not only do the CT means and the inverse DCT means become unnecessary, but also image degradation is reduced. Further, by adding the intermediate value of the quantization width to the spatial frequency component, a requantization operation error is reduced.

Also, in the first embodiment, since the decoding block and the coding block are configured by the same device,
Quantization information between the decoding block and the coding block,
Since the entropy coded information is reflected, an error hardly occurs. For this reason, the compressed image conversion device according to the first embodiment does not deteriorate in image as compared with a system in which the decoding block and the coding block of the conventional device are configured by different devices.

In the first embodiment, the input device 101
Is a digital camera, but other input devices such as a film scanner may be used. Also, the conversion coefficient determination unit 105
Is selected in advance, but the user may determine the brightness, contrast, etc. each time by a graphical user interface.

In the first embodiment, the frequency component conversion unit 106 performs the linear calculation represented by (Equation 4). However, the frequency component conversion unit 106 adds or subtracts another component or a value obtained by linearly calculating another component. You may.

Although the first embodiment deals with JFIF still images, it can be applied to still images of other formats represented by frequency components and moving images.

Further, in the first embodiment, the file stream decomposition section 102, the Huffman decoding section 103, the inverse quantization section 104, the transform coefficient determination section 105, the frequency component conversion section 1
06, the quantization unit 107, the Huffman encoding unit 108, the file stream synthesizing unit 109, and the image storage / transmission unit 110 have been described as separate processing units, but each processing unit is performed by a single processing unit. Is also conceivable. For example, it is configured by a computer including a CPU, a memory, a hard disk, a system bus, and a peripheral processor. In this case, the processing procedure performed by each processing unit may be stored as a program in a computer-readable memory, and the CPU may read this program and perform each processing. This allows the processing performed by each processing unit to be performed by a general-purpose P
C or the like.

Embodiment 2 Hereinafter, Embodiment 2 of the present invention will be described with reference to the drawings. FIG.
FIG. 9 is a hardware block diagram of a compressed image conversion device according to a second embodiment;

Image compression / conversion device 5 according to the second embodiment
00 is provided with an input device 501 for storing a compressed video stream and inputting it to another device. Input device 50
1 is, for example, a digital camera. For example, the input device 501 includes a digital video deck. Here, the DV format is used as the image compression processing.

The image compression / conversion device 500 includes a video stream decomposing unit 502, a Huffman decoding unit 503, and an inverse quantization unit 504, which are image decompression means for decompressing a compressed image input from the input device 501. , And an upsampling correction unit 505 are provided.

The video stream decomposing unit 502 analyzes a DV header, decomposes a video stream and an audio stream, and outputs Huffman-encoded frame data. The Huffman decoding unit 503 converts the data Huffman-encoded by the video stream
It decodes using the Huffman table obtained from the DV stream, and outputs frame data composed of a plurality of quantized DCT coefficient blocks. The inverse quantization unit 504 converts the data quantized by the Huffman decoding unit 503 into D
Using the quantization table obtained from the V stream, the data is output as frame data composed of a plurality of DCT coefficient blocks. The up-sampling correction unit 505 includes:
The spatial component thinning of the color difference component, such as the 1: 1 format, is corrected in the frequency domain.

The compression image conversion device 500 includes a transformation coefficient determination unit 5 serving as color conversion means for performing color conversion of an image.
06 and a frequency component converter 507 are provided.

The transform coefficient determining section 506 is means for determining a color correction coefficient, and the frequency component converting section 507 directly converts a DCT coefficient using the determined color correction coefficient.

The compressed image conversion device 500 includes a downsampling correction unit 508 as image compression means for compressing the image after color conversion again, a quantization unit 509,
Huffman encoding section 510 and video stream synthesizing section 51
1 is provided.

The downsampling correction unit 508 corrects the sampling of the color difference components subjected to the sampling correction so as to correspond to the 4: 1: 1 format again. The quantization unit 509 quantizes the transformed DCT coefficient block again using the quantization table obtained from the stream. The Huffman encoding unit 510 performs Huffman encoding on the quantized data again using a Huffman table obtained from the stream. The video stream synthesizing unit 511 includes:
A header or the like is added to the Huffman-coded data, and the data is combined with an audio stream to reconstruct a DV stream.

An image storage / transmission unit 512 for storing and transmitting the compressed image is provided in the compressed image conversion device 500.
Is provided. The image storage / transmission unit 512 stores and transmits DV video, such as a video server of a broadcasting station.

The video stream decomposing unit 50
2, Huffman decoding section 503, inverse quantization section 504, upsampling correction section 505, transform coefficient determination section 506,
Frequency component converter 507, downsampling corrector 5
08, a quantization unit 509, a Huffman encoding unit 510, a video stream synthesis unit 511, and an image storage / transmission unit 512
Is composed of a dedicated processor.

Next, the operation of the compressed image converter according to the second embodiment will be described. The second embodiment performs color space conversion of an image.

In the above configuration, the compressed video stream stored in the input device 501 is subjected to DV header analysis by the video stream decomposing unit 502, and further the video stream and the audio stream are decomposed and Huffman encoded. Output as frame data.

The operations of Huffman decoding section 503 and inverse quantization section 504 are the same as those of Huffman decoding section 103 and inverse quantization section 104 described in the first embodiment, and a description thereof will be omitted.

The DCT coefficient block obtained by the inverse quantization unit 504 has a format of FIG.
As shown in (a), the color difference components Cb,
Cr is thinned out to 1/4 of the information amount. Also, it can be seen from the spatial frequency component data that the sampling frequency is halved. Therefore, the DCT coefficient block of the chrominance component is converted by the upsampling correction unit 505 into a frequency component, interpolated into data equal to the sampling frequency of the luminance component, and output.

Each frequency component of the DCT coefficient block subjected to the sampling correction is converted and output by the conversion coefficient determination unit 506 and the frequency component conversion unit 507. The operations of the transform coefficient determining unit 506 and the frequency component converting unit 507 will be described later in detail.

The converted DCT coefficient block of the chrominance component is subjected to the reverse processing of the upsampling correction unit 505 by the downsampling correction unit 508, and the 4: 1:
It is output according to one format.

Quantizer 509, Huffman encoder 510
Operates in the quantization unit 10 described in the first embodiment.
7, since it is the same as the Huffman encoding unit 108, the description is omitted here.

The Huffman-encoded frame data is
The video stream synthesizing unit 511 synthesizes a DV header and an audio stream.
Output as video.

Next, the frequency component conversion operation according to the second embodiment will be described in detail.

Generally, a color image is represented by a luminance component and two color difference components, but the appearance of the input / output color image differs depending on the definition of the color space component. ITU Rec. Used as a standard TV color space.
The colorimetric values of the primary color components 601 are shown in (Table 3).

[0081]

[Table 3] In addition, ITU used as a color space for high-definition TV
Rec. Table 4 shows the colorimetric values of the primary color components 709.

[0082]

[Table 4] As can be seen from Tables 3 and 4, between the standard TV color space (represented by y cb cr) and the high-definition TV color space (represented by ypb pr), The relationship of 3 × 3 matrix conversion as in 5) is established.

[0083]

(Equation 5) The conversion coefficient determination unit 506 solves the simultaneous equations under the conditions of (Table 3) and (Table 4) to obtain the 3 × 3 matrix coefficient by calculation or stores the coefficient in advance.

The frequency component conversion section 507 uses this 3 × 3 matrix to obtain a D
Convert the CT coefficient data. Here, the coefficient m is represented by (Equation 5)
Is the same as the coefficient.

[0085]

(Equation 6) This transform focuses on the principle of the orthogonal series transform of (Equation 1) and (Equation 2) of the DCT transform as in the first embodiment, and performs linear sum of each frequency component to thereby obtain linear sum of pixel components. The same effect can be obtained.

Here, m11 = a1, m22 = a2, m
If 33 = a3 and the values of the other elements are set to 0, the gain adjustment portion of the first embodiment can be included. In this case, since the calculation is performed for each component, sampling correction is not necessarily required.

As described above, according to the second embodiment, in the apparatus for performing color space conversion of a compressed video, the DCT unit and the inverse D
The CT unit becomes unnecessary, and image degradation due to the DCT unit and the inverse DCT unit is reduced. Further, by adding the intermediate value of the quantization width to the spatial frequency component, the requantization operation error can be reduced.

In the second embodiment, color correction is similarly performed on an image in which color difference components such as 4: 2: 2 format and 4: 2: 0 format are thinned out by sampling correction. be able to.

Further, in the second embodiment, the chrominance component is upsampled, the matrix is calculated, and then downsampled again. However, as shown in FIG. 6B, the chrominance component is upsampled and the luminance component is downsampled. By providing a sampling correction unit for sampling, a configuration in which down-sampling correction of a color difference component at the subsequent stage is not performed may be adopted. As a result, the color difference components are subjected to a matrix operation while being down-sampled, so that the number of operations is greatly reduced, and
Calculation errors due to sampling conversion are reduced.

In the second embodiment, the DV format is used as the compression format.
May be used. In this case, the GO of the MPEG compressed image
The video stream decomposing unit 502 is used as a picture separating unit that separates and outputs an I picture and other pictures from P (group of pictures). The I-picture extracted by the video stream decomposition unit 502 is subjected to variable-length decoding by a Huffman decoding unit 503, which is a variable-length decoding unit. The variable-length decoded data is subjected to the same conversion as the above-described conversion of the frequency component by the inverse quantization unit 504,
Upsampling correction unit 505, conversion coefficient determination unit 50
6, performed by the frequency component conversion unit 507, the downsampling correction unit 508, and the quantization unit 509. And
The quantized data output from the quantization unit 509 is variable-length coded by a Huffman coding unit 510, which is a variable-length coding unit, and is output as an I-picture. The I-picture output from the Huffman code stock 510 and the I-picture other than the I-picture of the original MPEG stream output from the video stream decomposing unit 502 serving as a picture separation unit are output by a video stream synthesizing unit 511 serving as a picture decoding unit. Combine. Thereby, an effect of approximate color space conversion can be obtained. Also, in the second embodiment, JFIF (JPEG
File Interchange Format)
Still images such as the above may be applied.

In the second embodiment, the color space is converted by the conversion coefficients of the apparatus. However, the mutual color space conversion using the color management information (profile) included in the video stream and the output device information (profile) is performed. It may be used as a device. In this case, the effect of reducing the difference in color appearance when displayed on a plurality of different output devices can be obtained.

In the second embodiment, the color space of the standard TV and the color space of the high-definition TV are mutually converted.
Conversion from the first color system to the second color system such as the RGB color system, the YIQ color system, and the CIE color system may be used.

Also, in the second embodiment, the video stream decomposing unit 502, the Huffman decoding unit 503, and the inverse quantizing unit 5
04, up-sampling correction unit 505, transform coefficient determination unit 506, frequency component conversion unit 507, down-sampling correction unit 508, quantization unit 509, Huffman coding unit 5
10, the video stream synthesizing unit 511 and the image storage / transmission unit 512 are each configured by a dedicated processor.
Although described as separate processing units, a form in which each processing unit is performed by a single processing unit is also conceivable. For example, CP
U, a memory, a hard disk, a system bus, a peripheral processor and the like. In this case, the processing procedure performed by each processing unit is stored as a program in a computer-readable memory, and C
The PU may read this program and perform each process. Thus, the processing performed by each processing unit can be provided to a general-purpose PC or the like.

(Embodiment 3) Hereinafter, Embodiment 3 of the present invention will be described with reference to the drawings. FIG. 7 is a hardware block diagram of the compressed image conversion device according to the third embodiment.

The compressed image conversion device 7 according to the third embodiment
00, the file stream decomposing units 702a and 702b and the Huffman decoding units 703a and 703
3b and the inverse quantizers 704a and 704b, and the quantizers 708a and 708b, which are image compression means, and the Huffman encoders 709a and 709b have the same structure as the video stream decomposer 502, Huffman The decoding unit 503, the inverse quantization unit 504, the quantization unit 509, and the Huffman encoding unit 510 are the same as those in the first embodiment, and a description thereof is omitted.

Hereinafter, portions different from the other embodiments will be described.

A compressed image converter 7 according to the third embodiment
00 is provided with input devices 701a and 701b for inputting and storing the compressed video stream. For example, the input devices 701a and 701b are constituted by digital video cameras. Here, the DV format is used as the image compression processing.

Further, the compressed image conversion apparatus 700 includes white operation units 705a and 705b as color conversion means, conversion coefficient determination units 706a and 706b, and a frequency component conversion unit 70.
7a is provided.

The white calculation units 705a and 705b
The white point of the image is estimated from the CT coefficient block. The conversion coefficient determination units 706a and 706b determine a color correction coefficient based on the estimated white point. Further, the frequency component conversion units 707a and 707b directly convert the DCT coefficients using the determined color correction coefficients.

The file stream synthesizing section 710 adds a header or the like to the Huffman-encoded data, synthesizes the data with an audio stream, and reconstructs two DV streams into one DV stream.

The image storage / transmission unit 711 stores DV video, such as a digital video deck.

Next, the operation of the compressed image converter according to the third embodiment will be described. The device of the third embodiment is
Adjust the white balance of multiple video streams and combine them.

In the input device 701a, a picture is taken by a camera and stored as compressed video data. In the input device 701b, images are captured in the same manner under different locations and lighting conditions, and are stored as compressed image data.
The file stream synthesizing unit 710 receives the two color-adjusted compressed video streams and synthesizes them into one stream. In the image storage / transmission unit 711, the combined stream is stored as data used for broadcasting or the like.

The difference from the second embodiment is that the white calculation units 705a and 705b and the conversion coefficient determination units 706a and 706a
06b, frequency component converters 707a and 707b.
Hereinafter, these operations will be described in detail.

Various methods for estimating the white (white point) of an image have been proposed. Here, it is assumed that the average value of all pixels of the image is white. White is represented by (Equation 7).

[0106]

(Equation 7) K: The total number of pixels By the way, since the DC component of the DCT coefficient block output from the inverse quantization units 704a and 704b is an average value of 8 × 8 pixels, all the pixels are obtained by using the DC component (Equation 8). Is obtained. Here, q is a normalization coefficient for correcting the scale in the spatial domain and the frequency domain, and in this case, q = 8.

[0107]

(Equation 8) B0: Total number of 8 × 8 blocks The white calculation units 705a and 705b output white by the calculation of (Expression 8). Therefore, as compared with (Equation 7), there is no need to perform inverse DCT, and the number of operations is reduced by 1 /
(8 × 8).

Next, conversion coefficient determination units 706a and 706b
The operation of will be described. FIG. 8 shows white s estimated on the YCbCr color space.

Consider a transformation that rotates the s vector by the angle θ about the u vector and makes it coincident with the Y axis. Here, if u is a unit vector and u = (0, u2, u3), the rotation in the three-dimensional space is represented by (Equation 9).

[0110]

(Equation 9) The conversion coefficient determination units 706a and 706b calculate the actual coefficient of (Equation 8) using the white s, and output the calculated coefficients to the frequency component conversion units 707a and 707b.

The frequency component converters 707a and 707b convert the DCT coefficient data using the 3 × 3 matrix as shown in the following (Equation 10).

[0112]

(Equation 10) This transform pays attention to the principle of orthogonal series transform of (Equation 1) and (Equation 2) of DCT transformation, and obtains the same effect as linear sum of pixel components by linearly summing each frequency component. be able to.

As described above, according to the third embodiment, in the apparatus for adjusting the white balance of a plurality of video streams and synthesizing them, the spatial frequency of the image is directly converted, so that the DCT means and the inverse DCT section Is unnecessary, and the image deterioration by the DCT means and the inverse DCT unit is reduced. Further, by adding the intermediate value of the quantization width to the spatial frequency component, the requantization operation error can be reduced. In addition, in order to estimate white, it is not necessary to perform inverse DCT, and the number of white operations can be reduced to 1 / (8 × 8) of the related art.

In the third embodiment, the white is rotated by θ in order to match the white with the Y axis.
In order to match the vector corresponding to the D65 light source,
The rotation of θ ′ may be performed, thereby obtaining an effect that the illumination can be converted into a desired illumination. Further, in order to correct the white balance, the rotation calculation in the three-dimensional space is performed, but other methods such as a parallel movement of the color difference component to the luminance axis may be used.

In the third embodiment, the DV format is used as the compression mode.
May be used. In this case, picture separation means and file stream decomposing units 702a and 702b are used.
The Huffman decoding units 703a and 703b are used as the variable length decoding unit. Then, the frequency components are converted in the subsequent processing unit, and the original MPEG stream and the converted I picture are synthesized by the file stream synthesizing unit 710, whereby an approximate color space conversion effect can be obtained. In addition, JFIF (JPEG Fi
le Interchange Format).

In the third embodiment, two streams are combined, but three or more streams may be combined. Although the respective units are configured in parallel, a single unit may be configured and time-division processing may be performed while data is stored in the recording unit.

[0117]

As described above, according to the present invention,
By directly converting the spatial frequency of an image, DCT means and inverse DCT means become unnecessary, and DCT means and inverse DCT means become unnecessary.
Image deterioration due to the T means is reduced.

[Brief description of the drawings]

FIG. 1 is a hardware block diagram of a compressed image conversion device according to a first embodiment of the present invention.

FIG. 2 is a diagram for explaining an operation of the inverse quantization unit according to the first embodiment;

FIG. 3 is a diagram for explaining a color adjustment function according to the first embodiment;

FIG. 4 is a diagram for explaining frequency component conversion according to the first embodiment;

FIG. 5 is a hardware block diagram of the compressed image conversion device according to the second embodiment of the present invention.

FIG. 6A is a diagram for explaining sampling correction according to the second embodiment; FIG. 6B is a diagram for explaining sampling correction according to the second embodiment;

FIG. 7 is a hardware block diagram of a compressed image conversion device according to a third embodiment of the present invention;

FIG. 8 is a diagram for explaining white balance adjustment according to the third embodiment;

FIG. 9 is a hardware block diagram of a conventional compressed image conversion device.

FIG. 10 is a diagram for explaining operations of DCT and inverse DCT.

[Explanation of symbols]

101, 501, 701a, 701b Input device 102, 702a, 702b File stream decomposition unit 103, 503, 703a, 703b Huffman decoding unit 104, 504, 704a, 704b Inverse quantization unit 105, 506, 706a, 706b Conversion coefficient determination Units 106, 507, 707a, 707b Frequency component conversion units 107, 509, 708a, 708b Quantization units 108, 510, 709a, 709b Huffman coding unit 109 File stream synthesis unit 110, 512, 711 Image storage / transmission unit 502 Video Stream decomposition unit 505 Up-sampling correction unit 508 Down-sampling correction unit 511 Video stream synthesis unit 705a, 705b White operation unit 710 File stream synthesis unit

Claims (12)

[Claims] An inverse quantization means for inversely quantizing the quantized data and outputting a spatial frequency component of the image; a conversion coefficient determining means for determining a deformation coefficient for converting a luminance signal or a color difference signal of the image; Using the transform coefficient determined by the transform coefficient determining means, transforms a spatial frequency component of the image output from the inverse quantization means, and outputs a spatial frequency component converted, spatial frequency component transforming means, Quantizing the spatial frequency component output by the frequency component conversion means,
And a quantizing means for outputting the quantized data. 2. An inverse quantization means for inversely quantizing the quantized data and outputting a spatial frequency component of an image; an upsampling correction means for correcting the spatial frequency component output from the inverse quantization means; Transform coefficient determining means for determining a transform coefficient for converting a luminance signal or a chrominance signal of an image, and transforming a spatial frequency component of the image output by the upsampling correction means using the transform coefficient determined by the transform coefficient determining means. A spatial frequency component converting unit that outputs the converted spatial frequency component, a downsampling correcting unit that corrects the spatial frequency component output from the spatial frequency component converting unit, and a space that is output from the downsampling correcting unit. A quantizing means for quantizing the frequency component and outputting quantized data. Conversion device. 3. Inverse quantization means for inversely quantizing the quantized data and outputting a spatial frequency component of the image, sampling correction means for correcting the spatial frequency component output from the inverse quantization means, A conversion coefficient determining means for determining a conversion coefficient for converting a luminance signal or a color difference signal, and using the conversion coefficient determined by the conversion coefficient determination means, to convert a spatial frequency component of the image output by the sampling correction means, A spatial frequency component converting unit that outputs the converted spatial frequency component; and a quantizing unit that quantizes the spatial frequency component output from the spatial frequency component converting unit and outputs quantized data. A compressed image conversion device characterized by the above-mentioned. 4. A picture separating means for separating and outputting an I picture and other pictures from a GOP of an MPEG compressed image, and a variable length decoding of the I picture output from the picture separating means to obtain quantized data Output from the variable length decoding unit, inverse quantization of the quantized data output from the variable length decoding unit, and output of the spatial frequency component of the image, and output from the inverse quantization unit. Upsampling correction means for correcting the spatial frequency component of the image, a conversion coefficient determination means for determining a conversion coefficient for converting a luminance signal or a color difference signal of an image, and the upsampling using the conversion coefficient determined by the conversion coefficient determination means. A spatial frequency component converting unit that converts a spatial frequency component of the image output from the correcting unit, and outputs the converted spatial frequency component; Down-sampling correction means for correcting the spatial frequency component output from the inter-frequency component conversion means, and quantization means for quantizing the spatial frequency component output from the down-sampling correction means and outputting quantized data, A variable-length coding unit that performs variable-length coding on the quantized data output from the quantization unit and outputs the result as an I-picture; an I-picture output from the variable-length coding unit; and an output from the picture separation unit. A picture combining means for combining pictures other than I-pictures to be processed. 5. A picture separating means for separating and outputting an I picture and other pictures from a GOP of an MPEG compressed image, and performing variable length decoding of the I picture output from said picture separating means to obtain quantized data. Output from the variable length decoding unit, inverse quantization of the quantized data output from the variable length decoding unit, and output of the spatial frequency component of the image, and output from the inverse quantization unit. Sampling correction means for correcting the spatial frequency component of the image, a conversion coefficient determination means for determining a conversion coefficient for converting a luminance signal or a color difference signal of an image, and the sampling correction means using the conversion coefficient determined by the conversion coefficient determination means. A spatial frequency component converting means for converting the spatial frequency component of the image output from
A quantization means for quantizing the spatial frequency component output from the spatial frequency component conversion means and outputting the quantized data; and a variable length coding of the quantized data output from the quantization means to obtain an I picture A variable-length encoding means for outputting as a picture, an I picture output from the variable-length encoding means, and a picture combining means for combining pictures other than the I-picture output from the picture separating means. Characterized compressed image conversion device. 6. The apparatus according to claim 1, wherein said inverse quantization means inversely quantizes the quantized data and outputs a result obtained by adding an intermediate value of the quantization width. 3. The compressed image conversion device according to 1. 7. The apparatus according to claim 1, wherein the spatial frequency conversion means performs at least one of an offset adjustment of one or more attribute values and a gain adjustment of the attribute values.
The compressed image conversion device according to any one of claims 1 to 6. 8. The compressed image conversion apparatus according to claim 1, wherein said spatial frequency conversion means performs n × n matrix conversion. 9. The compressed image according to claim 1, wherein the conversion coefficient determination unit outputs a conversion coefficient from the first color system to the second color system. Conversion device. 10. The compressed image conversion apparatus according to claim 1, wherein the conversion coefficient determination unit outputs a coefficient for converting a white balance. 11. An inverse quantization step of inversely quantizing the quantized data and outputting a spatial frequency component of the image, a transform coefficient determining step of determining a transform coefficient for converting a luminance signal or a color difference signal of the image, A spatial frequency component transforming step of transforming a spatial frequency component of the image output from the inverse quantization step using the transform coefficient of the transform coefficient determining step and outputting the transformed spatial frequency component; A quantization step of quantizing a spatial frequency component output from the step and outputting quantized data. 12. A dequantization procedure for causing a computer to dequantize the quantized data and output a spatial frequency component of the image, and a procedure for determining a transform coefficient for converting a luminance signal or a color difference signal of the image. Using the transform coefficients determined in the transform coefficient determination step, transforming the spatial frequency components of the image output in the inverse quantization step, and outputting the converted spatial frequency components; A quantization procedure for quantizing the spatial frequency component output in the component conversion procedure and outputting the quantized data;
A computer-readable storage medium storing a program for executing the program.