JP5175796B2 - Encoding / preprocessing device, decoding / postprocessing device, encoding device, decoding device, and program - Google Patents

Description

  The present invention relates to an encoding / preprocessing device, a decoding / postprocessing device, an encoding device, a decoding device, and a program for irreversibly encoding a sub-sampled image signal. In particular, the encoding efficiency is deteriorated. The present invention relates to an encoding / preprocessing device, a decoding / postprocessing device, an encoding device, a decoding device, and a program that improve image quality related to a frequency component in the vicinity of ½ of the highest frequency with high visual sensitivity. is there.

  In an encoding system for encoding video such as Super Hi-Vision, the amount of code to be handled is enormous. Therefore, after the amount of code is reduced by the sub-sampling (half-pixel shift) method, Coding using a codec such as H.264 is a common practice.

  The sub-sampling method is a technique as one of encoding / preprocessing in which pixels are thinned out by shifting a half pixel for each pixel in the horizontal or vertical direction, and thus the code amount of the original image can be easily halved. . However, according to the sub-sampling method, for example, a straight line in the horizontal or vertical direction of the original image before thinning may be deformed into a zigzag. When the data transformed into the zigzag is irreversibly encoded, the data is encoded as a straight line in which a component in an oblique direction is suppressed with respect to the original straight line. On the other hand, when this data is decoded, it is decoded as a simple straight line. Therefore, when the original image is restored based on the sub-sampling method as decoding / post-processing, encoding / pre-processing is performed for each pixel in the horizontal or vertical direction. The original position is restored by shifting half a pixel in the opposite direction to the time. Therefore, since the data that was a straight line in the original image is restored in a zigzag manner, the image quality deteriorates. In order to solve this problem, a method has been proposed in which each half-pixel is moved and rearranged by 1/4 pixel in a direction opposite to the direction in which the half-pixel shift is performed to improve image quality (for example, patents). Reference 1).

Japanese Patent No. 3958493

  However, although the technique of Patent Document 1 can solve the problem of image quality degradation due to zigzag deformation, the image quality degradation due to suppression of frequency components in the vicinity of ½ of the maximum frequency determined by the number of pixels of the original image is solved. Not. Since the frequency component in the vicinity of ½ of the maximum frequency is high in visual sensitivity when viewing an image, how to improve the deterioration of this component becomes a problem.

  More specifically, as shown in FIG. 7, in the two-dimensional frequency space (horizontal frequency component, vertical frequency component) of the image signal of the original image, the maximum frequency determined by the number of pixels of the original image is normalized as 1. , (0,0), (0,1), and (1,0), a frequency region (hereinafter referred to as “low frequency region”) is outside the low frequency region band (1 , 0), (0, 1), and (1, 1) can be divided into frequency regions (hereinafter referred to as “out-of-band regions”). As shown in FIG. 7, the encoder folds the high frequency component region (1) in the direction in which the number of pixels is halved by encoding / preprocessing (subsampling method) into the region (2) in the out-of-band region. Encode the data. This means that among the components in the region (1), the component in the vicinity of the C point that is ½ of the maximum frequency with high visual sensitivity moves to the vicinity of the D point. However, normally, the bits assigned to the component near the point D in the encoder are set smaller than the component near the point C having a lower frequency. Therefore, when the original image is restored for the data obtained by decoding the encoded data, the original image is restored to the original area (1) by decoding and post-processing. Therefore, the component near the point C in the area (1) with high visual sensitivity. Is suppressed and restored, resulting in degradation of image quality.

  In addition, most digital video cameras and digital still cameras for consumer use employ a single-plate image sensor, and the green signal that contributes most to the luminance or luminance is converted into an image signal shifted by half a pixel by the sub-sampling method. Output from the imaging system. In this case, the output image signal is de-mosaiced as one of encoding and pre-processing, and then H.264. It is encoded using a codec such as H.264, MPEG2 or JPEG.

  However, in this technique using de-mosaicing, the image signal by the sub-sampling method (that is, the component of the “low-frequency side frequency region”) is not included in the original sub-sampling band of the two-dimensional frequency. An image signal (that is, a component of “out-of-band region”) is forcibly generated. Therefore, compression encoding as a signal in the entire frequency domain increases the amount of code for encoding an image signal by the subsampling method, leading to a decrease in encoding efficiency.

  Accordingly, an object of the present invention is to provide an encoding / preprocessing device, a decoding / postprocessing, which improve image quality related to a frequency component in the vicinity of ½ of the highest frequency with high visual sensitivity without degrading the encoding efficiency. An apparatus, an encoding device, a decoding device, and a program are provided.

In order to solve the above problems, a pretreatment apparatus of the present invention is provided. The pre-processing device of the present invention is a pre-processing device that performs pre-processing on the frequency components of the image signal to be encoded. From the frequency components in the low-frequency region in the two-dimensional frequency space, the horizontal and vertical frequencies A high-frequency component extracting means for extracting a high-frequency component of 3/4 or more of the maximum frequency of each of the extracted high-frequency components, out of the region in the out-of-band region that is outside the low-frequency side frequency region, Placement conversion means for changing the placement to a region that is 1/2 or more of the horizontal and vertical maximum frequencies and 3/4 or less, and each of the high-frequency components extracted from the frequency components of the low-frequency region Sampling rate conversion means for thinning out the number of pixels by 3/4 times with respect to an image signal obtained by adding the difference frequency component obtained by the difference and each of the frequency components subjected to the arrangement conversion;
It is characterized by providing. The encoding apparatus of the present invention includes the preprocessing apparatus of the present invention and encoding means for encoding the image signal thinned out by the sampling rate conversion means.

  The post-processing device of the present invention is a post-processing device that performs post-processing on an image signal decoded with respect to the image signal encoded by the encoding device of the present invention, and inputs the image signal. A sampling rate converting means for expanding the number of pixels by 4/3 times, and a band for extracting a frequency component in the out-of-band region that is outside the low-frequency region from the frequency component of the expanded image signal An external component extracting means, a high frequency component extracting means for extracting a high frequency component of ½ or more of the maximum horizontal and vertical frequencies from the extracted frequency components in the out-of-band region, and the extracted horizontal and vertical high frequencies Arrangement conversion means for changing the arrangement of each of the components to a region of 3/4 or more of the horizontal and vertical maximum frequencies among the regions in the low-frequency region, and in the out-of-band region And an image signal of wavenumber components, characterized in that by adding the image signals of the respective frequency components the placement transform and a signal adding means for restoring the original image signal. The decoding device of the present invention includes a decoding unit that performs an image signal encoded by the encoding device of the present invention, and a post-processing device of the present invention that performs post-processing on the decoded image signal. It is characterized by that.

  Furthermore, the present invention relates to a computer configured as a preprocessing device that performs preprocessing on frequency components of an image signal to be encoded, from a frequency component in a low frequency region in a two-dimensional frequency space, to horizontal and vertical frequencies. Extracting a high-frequency component of ¾ or more of the maximum frequency of each, and extracting each of the extracted high-frequency components horizontally and vertically in an out-of-band region outside the low-frequency region And a difference frequency component obtained by subtracting each of the extracted high-frequency components from the frequency components of the low-frequency region, respectively, And sampling rate conversion by thinning out the number of pixels by 3/4 with respect to an image signal obtained by adding each of the arrangement-converted frequency components. Also characterized as a program to be run.

  Furthermore, the present invention provides a computer configured as a post-processing device that performs post-processing on an image signal decoded with respect to an image signal encoded by the encoding device of the present invention, and inputs the image signal to the number of pixels. A sampling rate conversion by extending the frequency to 4/3 times, and a step of extracting a frequency component in the out-of-band region outside the low-frequency region from the frequency component of the expanded image signal Extracting each of the extracted horizontal and vertical high frequency components from the extracted frequency components in the out-of-band region, respectively, and extracting each of the extracted horizontal and vertical high frequency components The step of re-arranging each of the regions in the frequency region on the band side to regions of 3/4 or more of the maximum horizontal and vertical frequencies, and the frequency generation in the out-of-band region And an image signal of, also characterized as a postprocessor for by adding the image signals of the respective frequency components the placement transform and a step of restoring the original image signal.

  According to the present invention, it is possible to improve the image quality related to the frequency component in the vicinity of ½ of the maximum frequency with high visual sensitivity without degrading the encoding efficiency.

  In particular, a sub-sampling image signal obtained from a single-plate image pickup device or the like is H.264 without de-mosaicing. It is possible to perform lossy encoding using a codec such as H.264, MPEG2 or JPEG.

  Furthermore, the amount of information transmitted or stored can be reduced even for signals that are not subsampled.

1 is a block diagram of an encoding apparatus according to an embodiment of the present invention. It is a block diagram of the decoding apparatus by one Example of this invention. It is a figure which shows arrangement | positioning conversion of the two-dimensional high frequency component of the subsampling signal by one Example of this invention. The arrangement | positioning conversion process of the horizontal high frequency component by the horizontal frequency two-dimensional frequency arrangement | positioning converter of a pre-processing apparatus is shown. The signal processing of the horizontal frequency two-dimensional frequency arrangement converter of the pre-processing device is shown. 5 is an operation flowchart of a preprocessing apparatus according to an embodiment of the present invention. It is an operation | movement flowchart of the post-processing apparatus by one Example of this invention. Fig. 3 shows the folding of a two-dimensional high-frequency component in the prior art.

  Hereinafter, an encoding apparatus (including a preprocessing apparatus for encoding / preprocessing) and a decoding apparatus (including a postprocessing apparatus for decoding / postprocessing) according to an embodiment of the present invention will be described with reference to FIGS. This will be specifically described with reference to FIG.

  FIG. 1 shows a block diagram of an encoding apparatus 100 according to an embodiment of the present invention.

[Encoder]
The encoding apparatus 100 according to the present embodiment includes a preprocessing apparatus 1 for encoding / preprocessing and an encoder 3. Further, the preprocessing device 1 includes a two-dimensional filter 11, a two-dimensional frequency arrangement converter 12, and a horizontal / vertical sampling rate 3/4 converter 13. Further, the two-dimensional frequency arrangement converter 12 includes a horizontal high-frequency component remover 12a, a horizontal high-frequency component extractor 12b, a horizontal high-frequency component two-dimensional frequency arrangement converter 12c, a vertical high-frequency component remover 12d, and a vertical high-frequency component. An extractor 12e, a vertical high-frequency component two-dimensional frequency arrangement converter 12f, and a signal adder 12g are provided.

  Next, each component of the encoding apparatus 100 will be described.

  The two-dimensional filter 11 has a function of inputting a frequency component of an image signal of an arbitrary encoding target image (original image), converting the frequency component into a frequency component in a low frequency region, and sending the frequency component to a two-dimensional frequency arrangement converter 12. Have For example, when the frequency component of the encoding target image is not subsampled and the two-dimensional filter 11 includes a component in an out-of-band region of the two-dimensional frequency space (hereinafter referred to as “out-of-band component”). Removes the out-of-band component to convert it to the frequency component of the “low frequency side frequency region”. Alternatively, the two-dimensional filter 11 interpolates the thinned pixels when the frequency component of the encoding target image is subsampled (decimated for half-pixel shift). , It converts to the frequency component of the “low frequency side frequency region”. Alternatively, when the frequency component of the image signal of an arbitrary encoding target image (original image) has already been converted to the frequency component of the “low frequency region”, the two-dimensional filter 11 is not necessary. . Preferably, the best reproduction image quality is obtained when the band limitation is a frequency characteristic of a cosine roll-off filter. In addition, when the frequency component of the “low frequency side frequency region” has already been converted, the coefficient of the two-dimensional filter 11 can be set to pass through. Hereinafter, the frequency component of the “low frequency region” is referred to as a low frequency signal.

  The two-dimensional frequency arrangement converter 12 subsamples components in the band of 3/4 or more of the highest horizontal and vertical frequencies within the subsampling band from the low frequency side frequency domain signal supplied from the two-dimensional filter 11. An image signal of a differential frequency component obtained by converting each of the extracted high-frequency components from the frequency components in the low-frequency region, respectively, by changing the arrangement to a band of 1/2 or more and 3/4 or less of the highest frequency outside the band. And an image signal obtained by adding the image signals of the frequency components subjected to the arrangement conversion to the horizontal / vertical sampling rate 3/4 converter 13.

  The horizontal / vertical sampling rate 3/4 converter 13 performs sampling rate conversion from the image signal supplied from the two-dimensional frequency arrangement converter 12 by thinning the number of horizontal or vertical pixels to 3/4 times, respectively. To the generator 3.

  The encoder 3 is H.264. H.264, MPEG2 or JPEG is used to perform lossy encoding. The signal output from the encoder 3 can be transmitted by a transmission device (not shown) or stored in a storage device (not shown).

  In more detail, each component of the two-dimensional frequency arrangement converter 12 constituting a part of the preprocessing device 1 will be described.

  The horizontal high-frequency component remover 12a removes signal components in a band whose horizontal frequency is 3/4 or more of the maximum frequency from the input low frequency side frequency domain signal, and the vertical high-frequency component remover 12d and the vertical high-frequency component extractor 12e. Output to.

  The horizontal high-frequency component extractor 12b uses a one-dimensional digital filter that removes signal components in a band whose horizontal frequency is 3/4 or less of the maximum frequency from the input low-frequency side frequency domain signal, and outputs 3 / Four or more high frequency components are extracted and output to the horizontal high frequency component two-dimensional frequency arrangement converter 12c. Since the horizontal high-frequency component remover 12a and the horizontal high-frequency component extractor 12b perform complementary processing, the horizontal high-frequency component extractor 12b is band-limited by the original frequency domain signal and the two-dimensional out-of-band component remover 22a. It is only necessary to calculate the difference from the frequency domain signal.

  The horizontal high-frequency component two-dimensional frequency arrangement converter 12c outputs a high-frequency component whose horizontal frequency extracted by the horizontal high-frequency component extractor 12b is 3/4 or more of the maximum frequency, and the horizontal frequency outside the sub-sampling band is 1 / of the maximum frequency. The arrangement is converted to an area of 2 or more and 3/4 or less and output to the signal adder 12g.

  The vertical high frequency component remover 12d removes a signal component in a band whose vertical frequency is 3/4 or more of the maximum frequency from the input signal, and outputs the signal component to the signal adder 12g.

  The vertical high-frequency component extractor 12e uses a one-dimensional digital filter that removes a signal component in a band whose vertical frequency is 3/4 or less of the maximum frequency from the input signal, and extracts a high-frequency component of 3/4 or more of the maximum frequency. Extracted and output to the vertical high-frequency component two-dimensional frequency arrangement converter 12f.

  The vertical high frequency component two-dimensional frequency arrangement converter 12f receives a high frequency component of 3/4 or more of the maximum vertical frequency from the input signal, and the vertical frequency outside the sub-sampling band is 1/2 or more and 3/4 or less of the maximum frequency. The arrangement is converted to the area and output to the signal adder 12g.

  The signal adder 12g removes the input frequency component image signal from the horizontal high-frequency component two-dimensional frequency arrangement converter 12c, the frequency component image signal from the vertical high-frequency component two-dimensional frequency arrangement converter 12f, and the vertical high-frequency component removal. The frequency component image signals from the converter 12d are added in an arbitrary manner separable on the decoding side, and output to the horizontal / vertical sampling rate 3/4 converter 13.

  As shown in FIG. 3, the preprocessing device 1 of the present embodiment rearranges and converts the high frequency region of 3/4 or more of the horizontal and vertical maximum frequencies to the out-of-band regions (2) and (4), respectively. Furthermore, by thinning out the number of pixels by 3/4, image quality deterioration caused by components around 1/2 of the maximum frequency that occurred in the prior art is improved, and the code amount when sub-sampling and de-mosaicing is further reduced It is to halve.

  Here, signal processing by the horizontal high-frequency component two-dimensional frequency arrangement converter 12c that performs arrangement return of high-frequency components in the horizontal direction will be described.

  FIG. 4a shows a process in which the signal region (1) of 3/4 or more of the horizontal maximum frequency is rearranged to the out-of-band region (2) by the horizontal high-frequency component two-dimensional frequency arrangement converter 12c. Signal components a_ij (i and j are integers of 1 or more). First, the signal is thinned by half by sub-sampling, the signal in the region (1) is folded back into the region (6), and then is rearranged into the region (7). The signal at the stage where it has been converted to the area (7) is in a thinned state, but the signal is interpolated both horizontally and vertically by a digital filter that limits the bandwidth to the original half. To do. Here, b_ij (i and j are integers of 1 or more) indicates the interpolated signal component. Thereafter, it is folded and rearranged into the out-of-band region (2). The vertical high frequency component two-dimensional frequency arrangement converter 12f is processed in the same manner. In the present embodiment, when the high frequency region of 3/4 or more of the horizontal or vertical maximum frequency is rearranged, as shown in FIG. 2, the horizontal high frequency region (1) is changed to the region (2), and the vertical high frequency region ( 3) is converted into the region (4), but the horizontal high frequency region (1) may be converted into the region (4) and the vertical high frequency region (3) may be converted into the region (2).

  Next, the operation of the preprocessing device 1 will be described.

(Operation of pre-processing equipment)
FIG. 5 is a flowchart showing the operation of the preprocessing device 1 of the present embodiment.

  First, in step S501, the two-dimensional filter 11 removes out-of-band components outside the sub-sampling band of the two-dimensional frequency space from the image signal to be encoded when sub-sampling is not performed. In addition, for the subsampled signal, the thinned pixels are interpolated.

  Next, in step S502, the horizontal high frequency component remover 12a removes a high frequency component whose horizontal frequency is 3/4 or more of the maximum frequency from the signal from which the out-of-band component has been removed, and the horizontal high frequency component extractor 12b. Extract high frequency components that are 3/4 or more of the maximum frequency.

  Subsequently, in step S503, the vertical high frequency component remover 12d removes a high frequency component whose vertical frequency is 3/4 or more of the maximum frequency from the signal from which the horizontal high frequency component is removed by the horizontal high frequency component remover 12a. A high-frequency component of 3/4 or more of the maximum frequency is extracted by the vertical high-frequency component extractor 12e.

  Next, in step S504, the horizontal high-frequency component two-dimensional frequency arrangement converter 12c converts the high-frequency component of 3/4 or more of the maximum frequency horizontally from the signal from which the horizontal high-frequency component is extracted by the horizontal high-frequency component extractor 12b. Placement conversion is performed in an out-of-band region that exists outside the sub-sampling band whose frequency is 1/2 or more and 4/3 or less. Similarly, with respect to the vertical component, the vertical high frequency component is extracted from the signal from which the vertical high frequency component is extracted by the vertical high frequency component extractor 12e. Placement conversion is performed in an out-of-band region that exists outside the sub-sampling band whose frequency is 1/2 or more and 4/3 or less.

  Thereafter, in step S505, the signal adder 12g outputs the output of the horizontal high-frequency component two-dimensional frequency arrangement converter 12c, the output of the vertical high-frequency component two-dimensional frequency arrangement converter 12f, and the output of the vertical high-frequency component remover 12d. to add.

  Finally, in step S506, the horizontal / vertical sampling rate 3/4 converter 13 thins out the number of horizontal and vertical pixels from the output signal of the signal adder 12g by 3/4 times.

  The output signal of the preprocessing device 1 is then encoded by the encoder 3 and then transmitted to the outside or stored. On the decoding side, as will be described later, the decoder 5 receives the transmitted signal or reads the accumulated signal and performs decoding processing, and the post-processing device 2 performs post-processing.

  In this way, high-frequency components of 3/4 or more of the horizontal and vertical maximum frequencies are rearranged to out-of-band components outside the sub-sampling band, and only the components in the band of 3/4 or less of the maximum frequency are encoded. Thus, it is possible to improve image quality degradation due to insufficient bit allocation in the vicinity of ½ of the maximum frequency with high visual sensitivity.

  The total number of pixels to be encoded is 9/16 times that of the original image because the band is limited to 3/4 of the maximum frequency in both the horizontal and vertical directions as shown in FIG. This is 9/8 times that of the case where the sub-sampling signals are rearranged horizontally or vertically. However, the total code amount to be encoded is 0 because the signal component in the region (5) does not exist, and is the same as when the sub-sampling signals are rearranged horizontally or vertically.

  Furthermore, the sub-sampling signal obtained from a single-plate image sensor such as a digital camera can be de-mosaiced by using the pre-processing device and post-processing device according to the present embodiment and de-mosaicing after decoding. Thus, the code amount is halved compared to the case of encoding an image signal having the same number of pixels as the original image, and it is possible to achieve an improvement in image quality with a small code amount. In other words, the image quality is improved when the code amount is the same as when the pre-processing apparatus and post-processing apparatus of the present embodiment are not used.

  Next, a decoding apparatus 200 shown in FIG. 2 according to an embodiment of the present invention will be described.

[Decryption device]
The decoding device 200 according to the present embodiment includes a post-processing device 2 for decoding / post-processing and a decoder 5. Further, the post-processing device 2 includes a horizontal / vertical sampling rate 4/3 converter 21 and a two-dimensional frequency arrangement converter 22. Further, the two-dimensional frequency arrangement converter 22 includes a two-dimensional out-of-band component remover 22a, a two-dimensional out-of-band component extractor 22b, a vertical high-frequency component extractor 22c, and a vertical high-frequency component two-dimensional frequency arrangement converter 22d. , A horizontal high-frequency component extractor 22e, a horizontal high-frequency component two-dimensional frequency arrangement converter 22f, and a signal adder 22g.

  Next, each component of the decoding device 200 will be described.

  The decoder 5 inputs the received image signal or the image signal read from the storage device with respect to the image signal encoded by the encoder 3 and performs a decoding process corresponding to the encoding side codec. The horizontal / vertical sampling rate is output to the 4/3 converter 21.

  The horizontal / vertical sampling rate 4/3 converter 21 expands the number of pixels in the horizontal and vertical directions by 4/3 times with respect to the input decoded image signal, and supplies it to the two-dimensional frequency arrangement converter 22. Output.

  The two-dimensional frequency arrangement converter 22 converts a high-frequency component that has been arranged and converted into a band of 1/2 or more and 3/4 or less of the maximum horizontal or vertical frequency with respect to the input image signal whose sampling rate has been converted. Then, the arrangement is converted to a high frequency region of 3/4 or more of the horizontal or vertical maximum frequency, and the image signal of each frequency region signal subjected to the arrangement conversion is added and output.

  In more detail, each component of the two-dimensional frequency arrangement converter 22 constituting a part of the post-processing device 2 will be described.

  The two-dimensional out-of-band component remover 22a receives the output signal of the horizontal / vertical sampling rate 4/3 converter 21, removes a two-dimensional out-of-band component from the input signal by two-dimensional digital filtering, and adds a signal. Output to 22g.

  The two-dimensional out-of-band component extractor 22b receives the output signal of the horizontal / vertical sampling rate 4/3 converter 21, extracts the two-dimensional out-of-band component, and extracts the vertical high-frequency component extractor 22c and the horizontal high-frequency component extractor. To the device 22e. Since the two-dimensional out-of-band component remover 22a and the extractor perform complementary processing, the output signal of the horizontal / vertical sampling rate 4/3 converter 21 and the output signal of the two-dimensional out-of-band component remover 22a It is only necessary to calculate the difference between and.

  The vertical high frequency component extractor 22c removes a signal component in a band whose vertical frequency is 1/2 or less of the maximum frequency from the input signal, and outputs the signal component to the vertical high frequency component two-dimensional frequency arrangement converter 22d.

  The horizontal high frequency component extractor 22e removes a signal component in a band whose horizontal frequency is ½ or less of the maximum frequency from the input signal, and outputs the signal component to the horizontal high frequency component two-dimensional frequency arrangement converter 22f.

  The vertical high frequency component two-dimensional frequency arrangement converter 22d converts a component having a vertical frequency of 1/2 or more of the maximum frequency from the input signal into a region of 3/4 or more of the maximum frequency in the sub-sampling band where it was originally arranged. The arrangement is converted and output to the signal adder 22g.

  The horizontal high-frequency component two-dimensional frequency arrangement converter 22f converts a component having a horizontal frequency of 1/2 or more of the maximum frequency from the input signal into a region of 3/4 or more of the maximum frequency in the sub-sampling band where it was originally arranged. The arrangement is converted and output to the signal adder 22g.

  The signal adder 22g receives the input frequency component image signal from the two-dimensional out-of-band component remover 22a, the frequency component image signal from the vertical high-frequency component two-dimensional frequency arrangement converter 22d, and the horizontal high-frequency component two-dimensional frequency. An image signal (frequency signal or pixel signal) for the original image is restored by adding the frequency component image signal from the placement converter 22f.

  Here, the operation of the post-processing device 2 will be described.

(Operation of post-processing equipment)
FIG. 6 is a flowchart showing the operation of the post-processing apparatus 2 according to the present embodiment.

  First, in step S601, the horizontal / vertical sampling rate 4/3 converter 21 expands the number of horizontal or vertical pixels from the output signal of the decoder 5 to 4/3 times, respectively.

  Next, in step S602, out-of-band components outside the sub-sampling band in the two-dimensional frequency space are removed from the output signal of the horizontal / vertical sampling rate 4/3 converter 21 by the two-dimensional out-of-band component remover 22a. A two-dimensional out-of-band component extractor 22b extracts out-of-band components outside the subsampling band in the two-dimensional frequency space.

  Subsequently, in step S603, the vertical high-frequency component extractor 22c extracts a high-frequency component whose vertical frequency is ½ or more of the maximum frequency, and the horizontal high-frequency component extractor 22e extracts a high-frequency component that is ½ or more of the maximum frequency. To extract.

  Next, in step S604, the vertical high-frequency component two-dimensional frequency arrangement converter 22d obtains the component in which the vertical frequency extracted by the vertical high-frequency component extractor 22c is 1/2 or more of the maximum frequency. The arrangement is converted to a region of 3/4 or more of the frequency. Similarly, with respect to the horizontal component, the horizontal high-frequency component two-dimensional frequency arrangement converter 22f uses the output of the horizontal high-frequency component extractor 22e to convert a component of 1/2 or more of the maximum frequency to the maximum frequency originally arranged. The layout is converted to an area of 3/4 or more.

Finally, in step S605, the signal adder 22g causes the output of the two-dimensional out-of-band component remover 22a, the output of the vertical high-frequency component two-dimensional frequency arrangement converter 22d, and the horizontal high-frequency component two-dimensional frequency arrangement converter 22f. Add the outputs of.

  In this way, the image signal decoded by the decoder 5 is processed by the post-processing device 2 to obtain an image signal including components of the entire sub-sampling band. The output image signal can be used as a signal that does not include an out-of-band component of a two-dimensional frequency, or can be used after two-dimensional sub-sampling.

  As described above, by using the pre-processing device and post-processing device according to an embodiment of the present invention, image quality deterioration related to a component near ½ of the highest frequency with high visual sensitivity is improved and encoding efficiency is improved. It becomes possible to make it.

  Furthermore, as an aspect of the present invention, the encoding device 1 or the decoding device 2 can be configured as a computer that functions as each device. A program for causing a computer to realize each of the above-described components is stored in a storage unit provided inside or outside each computer. Such a storage unit can be realized by an external storage device such as an external hard disk or an internal storage device such as ROM or RAM. The control unit provided in each computer can be realized by control of a central processing unit (CPU) or the like. In other words, the CPU can appropriately read from the storage unit a program in which the processing content for realizing the function of each component is described, and realize the function of each component on the computer. Here, the function of each component may be realized by all or part of the hardware.

  In addition, the program describing the processing contents can be distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM, and such a program can be distributed on a network such as an IP. The program can be distributed by storing the program in the storage unit of the server and transferring the program from the server to another computer via the network.

In addition, a computer that executes such a program can temporarily store, for example, a program recorded on a portable recording medium or a program transferred from a server in its own storage unit. As another embodiment of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and each time the program is transferred from the server to the computer. In turn,
Processing according to the received program may be executed. Note that the program in this aspect includes information provided for processing of an electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer). .

  INDUSTRIAL APPLICABILITY The present invention is useful for image quality improvement and code amount suppression in an encoding system such as Super Hi-Vision that has a large amount of information, a single-panel digital video camera, and a digital still camera. In addition, it is useful for improving coding efficiency without causing deterioration of image quality even for general signals that are not subsampled.

  Although the present invention has been described in detail with specific examples, it will be apparent to those skilled in the art that various modifications and changes can be made without departing from the scope of the claims of the present invention. Therefore, the present invention is not limited to the above embodiments.

DESCRIPTION OF SYMBOLS 1 Pre-processing apparatus 2 Post-processing apparatus 3 Encoder 5 Decoder 11 Two-dimensional filter 12 Two-dimensional frequency arrangement converter 12a Horizontal high frequency component remover 12b Horizontal high frequency component extractor 12c Horizontal high frequency component two-dimensional frequency arrangement converter 12d Vertical high frequency component remover 12e Vertical high frequency component extractor 12f Vertical high frequency component two-dimensional frequency arrangement converter 12g Signal adder 13 Horizontal / vertical sampling rate 3/4 converter 21 Horizontal / vertical sampling rate 4/3 converter 22 Two-dimensional Frequency arrangement converter 22a Two-dimensional out-of-band component remover 22b Two-dimensional out-of-band component extractor 22c Vertical high-frequency component extractor 22d Vertical high-frequency component two-dimensional frequency arrangement converter 22e Horizontal high-frequency component extractor 22f Horizontal high-frequency component two-dimensional frequency arrangement Converter 22g Signal adder 100 Encoding device 200 Decoding device

Claims (6)

A preprocessing device that performs preprocessing on frequency components of an image signal to be encoded,
High-frequency component extraction means for extracting high-frequency components of 3/4 or more of the maximum horizontal and vertical frequencies from the frequency components in the low-frequency region in the two-dimensional frequency space;
Each of the extracted high-frequency components is a region that is not less than ½ of the horizontal and vertical maximum frequencies and is not more than 3/4 of the region in the out-of-band region that is outside the low-frequency region. A layout conversion means for converting the layout to
An image signal obtained by adding an image signal of a difference frequency component obtained by subtracting each of the extracted high frequency components from a frequency component of the low frequency region and an image signal of each of the frequency components subjected to the arrangement conversion Sampling rate conversion means for thinning out the number of pixels by 3/4 times,
A pre-processing apparatus comprising: The pretreatment device according to claim 1;
Encoding means for encoding the image signal thinned out by the sampling rate conversion means;
An encoding device comprising: A post-processing device that performs post-processing on an image signal decoded for an image signal encoded by the encoding device according to claim 2,
Sampling rate conversion means for inputting the image signal and expanding the number of pixels to 4/3 times;
Out-of-band component extraction means for extracting a frequency component in the out-of-band region that is outside the band of the low-frequency side frequency region from the frequency component of the expanded image signal;
High-frequency component extraction means for extracting high-frequency components that are 1/2 or more of the horizontal and vertical maximum frequencies from the extracted frequency components in the out-of-band region;
Arrangement conversion means for arranging and converting each of the extracted horizontal and vertical high-frequency components into a region of 3/4 or more of the maximum horizontal and vertical frequencies among the regions in the low frequency region;
Signal adding means for adding the image signal of the frequency component in the out-of-band region and the image signal of each frequency component subjected to the arrangement conversion to restore the original image signal;
A post-processing device comprising: Decoding means for performing an image signal encoded by the encoding device according to claim 2;
The post-processing device according to claim 3, wherein post-processing is performed on the decoded image signal;
A decoding apparatus comprising: In a computer configured as a preprocessing device that performs preprocessing on the frequency components of the image signal to be encoded,
Extracting a high frequency component of 3/4 or more of the maximum horizontal and vertical frequencies from the frequency components in the low frequency region in the two-dimensional frequency space;
Each of the extracted high-frequency components is a region that is not less than ½ of the horizontal and vertical maximum frequencies and is not more than 3/4 of the region in the out-of-band region that is outside the low-frequency region. Each of the step of converting the arrangement,
The number of pixels is determined for an image signal obtained by adding a difference frequency component obtained by subtracting each of the extracted high frequency components from the frequency component of the low frequency region and the frequency components subjected to the arrangement conversion. Converting the sampling rate by thinning out to 3/4 times;
A program for running A computer configured as a post-processing device that performs post-processing on an image signal decoded for an image signal encoded by the encoding device according to claim 2,
Inputting the image signal, expanding the number of pixels by 4/3, and converting the sampling rate;
Extracting a frequency component in the out-of-band region that is outside the band of the low-frequency side frequency region from the frequency component of the expanded image signal;
Extracting from the extracted frequency components in the out-of-band region, high frequency components that are ½ or more of the horizontal and vertical maximum frequencies, respectively;
Rearranging each of the extracted horizontal and vertical high-frequency components into a region of 3/4 or more of the horizontal and vertical maximum frequencies among the regions in the low frequency region,
Restoring the original image signal by adding the image signal of the frequency component in the out-of-band region and the image signal of each frequency component subjected to the arrangement conversion;
A post-processing program for executing

Images