JP2009010763A - Image processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus which actualizes a high compression ratio keeping an image quality of an image signal after a re-encoding process more excellent. <P>SOLUTION: The apparatus includes a decoding processing portion 10, a filter processing portion 121, an encoding processing portion 11 and a filter controlling portion 122. The decoding processing portion 10 extracts first difficulty-level information for showing the difficulty to encode from the image signal encoded by a first encoding system input from an input means and decodes the image signal. The filter processing portion 121 applies a filter processing to the decoded image signal. The encoding processing portion 11 extracts a second difficulty-level information for showing the difficulty to encode from the image signal to which the filter processing portion 121 has applied the filter processing, encodes the image signal based on a second encoding system, and outputs it. The filter controlling portion 122 controls intensity of the filter processing which the filter processing portion 121 applies to the image signal according to the first difficulty-level information extracted by the decoding processing portion 10 and the second difficulty-level information extracted by the encoding processing portion 11. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention decodes an image signal encoded by the first encoding method, and converts the decoded image signal into an image signal of a second encoding method different from the first encoding method. And an image processing method.

  In recent years, moving image data has been handled as digital data. At that time, orthogonal transformation processing such as discrete cosine transformation and motion compensation processing have been performed for the purpose of efficient transmission and storage of information. An apparatus that complies with an encoding scheme such as MPEG (Moving Picture Expert Group) that compresses an image signal by means of an information signal is becoming widespread in both information distribution such as broadcasting stations and information reception in general homes.

  In particular, the MPEG2 system (ISO / IEC13818-2) is defined as a general-purpose moving image encoding system, and covers both interlaced scanning images and progressive scanning images, as well as standard resolution images and high-definition images. And is now widely used in a wide range of professional and consumer applications.

  In the MPEG2 system, for example, a bit rate of 4 to 8 [Mbps] is assigned for a standard resolution interlaced scan image with a picture size of 720 × 480, and 18 for a high resolution interlaced scan image with a picture size of 1920 × 1080. By assigning a bit rate of ˜22 [Mbps], it is possible to encode an image signal while maintaining good image quality.

  In addition, the MPEG2 system was mainly intended for image signals encoded in a high-quality state such as for broadcasting. However, the MPEG2 system is compatible with an encoding system with a lower bit rate, that is, a higher compression rate than the MPEG1 system. It wasn't. As an encoding method corresponding to such a high compression rate encoding method, standardization of the MPEG4 method was performed. With regard to this image coding system, this standard was approved as an international standard system in December 1998 as ISO / IEC 14496-2.

  As an image encoding method with higher compression efficiency than the MPEG method as described above, an encoding method called JVT (Joint Video Team) has been proposed.

  By the way, the moving picture encoding method conventionally used for digital broadcasting is mainly the MPEG method. An image processing apparatus such as a so-called digital video recorder that records and stores such digital broadcasts and can subsequently reproduce the images is sent in the MPEG format so that it can be recorded for a long time with higher image quality. There is one in which a signal is once decoded and re-encoded by an encoding method such as a JVT method capable of realizing a higher compression rate.

  For such an image signal to be re-encoded, that is, a decoded image of an image signal sent by the MPEG method, image quality degradation such as block noise that occurs when the image signal by the MPEG method is encoded. It is included. For this reason, in an image processing apparatus that performs re-encoding processing, if an image signal including such image quality deterioration is directly subjected to re-encoding processing by the JVT method, it is unnecessary to reproduce the deteriorated portion. There is a possibility that not only the amount of information is broken and the compression efficiency is deteriorated, but also deterioration is more emphasized.

  In order to reduce such image quality degradation, an image processing apparatus that performs re-encoding processing of an image signal is included when encoding processing using an MPEG method or the like is performed on an image signal subjected to decoding processing. In order to weaken image quality deterioration, for example, low-pass filter processing may be performed. Here, if the image processing apparatus performs low-pass filter processing with a uniform intensity on the image signal, the encoding process is relatively easy, and an image region in which image quality degradation hardly occurs is unnecessarily blurred. Therefore, it is not always desirable.

  Therefore, in the re-encoding process, controlling the strength of the filter process according to the degree of image quality deterioration is effective in reducing the image quality deterioration described above. Specifically, the image processing apparatus determines a part that is easy to encode and a part that is difficult to encode at the time of re-encoding processing even when there is almost no deterioration in image quality in the decoded image signal, and performs filtering processing on the easy part as much as possible. Without removing only the difficult high-frequency components, the re-encoding can be performed more efficiently. In general, the difficult part to encode is a part with a lot of movement or a complicated pattern, so even if the high-frequency component of the image signal is removed by the low-pass filter as described above. This is because image quality deterioration is difficult to understand due to visual characteristics.

  In the conventional image processing apparatus, as the pre-stage of the re-encoding process, the degree of difficulty in encoding is evaluated based on the encoding information obtained when the decoding process is performed on the image signal, and the filter The intensity of processing is controlled.

  Patent Document 1 describes an information processing apparatus that decodes a decoded image signal based on sub-data obtained when the image signal is decoded.

JP 2006-20147 A

  However, the relationship that an image that is difficult to encode using the MPEG method becomes an image that is difficult to encode using the JVT method does not necessarily hold. For example, an image signal whose screen fades in or out is difficult to encode by the MPEG method, but it is relatively easy to encode by the JVT method having an encoding processing mode specialized for fading. is there. As described above, the information indicating the difficulty of encoding obtained by the decoding process may not coincide with the difficulty of encoding in the re-encoding process.

  The present invention has been proposed in view of such circumstances, and appropriately controls the strength of filter processing applied to a decoded image signal in order to reduce image quality degradation when re-encoding. Accordingly, an object of the present invention is to provide an image processing apparatus and an image processing method that realize a high compression rate while maintaining better image quality of the image signal after the re-encoding process.

  As means for solving the above-described problems, an image processing apparatus according to the present invention includes an input means for inputting an image signal encoded by the first encoding method, and an image signal input by the input means. Decoding means for extracting first difficulty level information indicating difficulty in encoding an image signal by the first encoding method, and decoding the image signal based on the first encoding method; A filter processing unit that performs a filtering process on the image signal decoded by the decoding unit, and an image signal that has a second encoding scheme different from the first encoding scheme, from the image signal that has been subjected to the filtering process by the filter processing unit. The second difficulty level information indicating the difficulty of encoding the image signal, the encoding means for encoding and outputting the image signal based on the second encoding method, and the decoding means extracted by the decoding means 1 difficulty Comprising a degree information, in accordance with the second difficulty information extracted by the above encoding means, said filtering means and control means for controlling the intensity of the filtering process applied to the image signal.

  The image processing method according to the present invention includes an input step of inputting an image signal encoded by the first encoding method, and an image by the first encoding method from the image signal input by the input step. Extracting first difficulty level information indicating difficulty in encoding a signal, decoding the image signal based on the first encoding method, and decoding the image signal into the image signal decoded by the decoding step A filter processing step for performing filter processing, and a second encoding method indicating difficulty in encoding an image signal by a second encoding method different from the first encoding method from the image signal subjected to the filter processing by the filter processing step. 2 difficulty level information is extracted, the image signal is encoded based on the second encoding method and output, the first difficulty level information extracted in the decoding step, and In the encoding process Depending on the second difficulty information issued, the filtering step and a control step of controlling the intensity of the filtering process applied to the image signal.

  In the present invention, the first difficulty level indicating the difficulty of encoding the image signal by the first encoding method extracted when decoding the image signal, and the first difficulty level extracted when encoding the image signal. When the reencoding of the decoded image signal is performed because the strength of the filter processing applied to the image signal is controlled according to the second difficulty level indicating the difficulty of encoding the image signal by the encoding method 2 Therefore, it is possible to appropriately control the strength of the filter processing performed to reduce the image quality degradation, so that the image signal is encoded into the second encoding method so that a high compression rate can be realized while maintaining a good image quality. be able to.

  An image processing apparatus to which the present invention is applied decodes an image signal encoded by the first encoding method, and converts the image signal into an image signal of a second encoding method different from the first encoding method. This is an image processing apparatus that performs so-called re-encoding processing. Hereinafter, the best mode for carrying out the present invention will be described using the image processing apparatus 1 according to the first embodiment as shown in FIG.

  The image processing apparatus 1 decodes, for example, MPEG encoded data obtained by demodulating a digital broadcast wave by a digital tuner into a baseband video signal, and converts the decoded baseband signal into a DVD or a BD (Blu-ray Disc). ) And HDDVD (High-Definition Digital Versatile Disc) recording data on a high-density recording medium such as HDV (High-Definition Digital Versatile Disc).

  Specifically, as shown in FIG. 1, the image processing apparatus 1 decodes an image signal of a first encoding method such as an MPEG2 method that is input from the outside and outputs a decoded image signal such as a baseband video signal. A decoding processing unit 10 for outputting, an encoding processing unit 11 for encoding the decoded image signal into an image signal of a second encoding method such as an AVC method having higher compression efficiency than the MPEG2 method, and a decoding processing unit 10 A filter processing unit 121 that performs processing to reduce block distortion noise included in the image signal decoded in step S1, a filter control unit 122 that controls operation characteristics of the filter processing unit 121, and an image that has been subjected to re-encoding processing. And a user interface 123 for setting a bit rate of a signal bit stream by an operation input by a user.

  As will be described later, the image processing apparatus 1 according to the present embodiment includes information related to the first encoding method extracted when the decoding processing unit 10 decodes an image signal, and the encoding processing unit 11 has decoded the information. The filter control unit 122 controls the operation characteristics of the filter processing unit 121 according to the information related to the second encoding method extracted when the image signal is encoded. The image signal is encoded by the second encoding method so that the compression rate can be realized.

  Prior to description regarding specific processing of the filter control unit 122, configurations and operations of the decoding processing unit 10 and the encoding processing unit 11 will be described.

  The decoding processing unit 10 includes a buffer 101, a lossless decoding unit 102, an inverse quantization unit 103, an inverse orthogonal transform unit 104, an adder 105, a screen rearrangement unit 106, a frame memory 107, and a motion compensation unit. 108.

  The buffer 101 receives an image signal encoded by the first encoding method from the outside, temporarily stores the input image signal, and synchronizes with a processing unit that performs other decoding processing. Then, the sequentially stored image signals are supplied to the lossless decoding unit 102.

  The lossless decoding unit 102 performs processing such as variable length decoding and arithmetic decoding on the image signal supplied from the buffer 101 based on the first encoding method, and quantized orthogonal transform coefficients. Is supplied to the inverse quantization unit 103.

  In addition, the lossless decoding unit 102 decodes a motion vector stored in the header part of the frame image data when the image signal encoded by the first encoding method is inter-encoded frame image data. And the extracted motion vector is supplied to the motion compensation unit 108.

  The inverse quantization unit 103 performs an inverse quantization process on the quantized orthogonal transform coefficient supplied from the lossless decoding unit 102 and supplies the orthogonal transform coefficient to the inverse orthogonal transform unit 104.

  The inverse orthogonal transform unit 104 performs inverse orthogonal transform processing such as inverse discrete cosine transform and inverse Karhunen-Labe transform on the orthogonal transform coefficient generated by the inverse quantization unit 103, and adds the decoded frame image data To the vessel 105.

  When the frame image data is inter-coded, the adder 105 adds the predicted frame image data generated by the motion compensation unit 108 described later to the frame image data supplied from the inverse orthogonal transform unit 104, This is supplied to the screen rearrangement unit 106 and the frame memory 107. When the frame image data is intra-coded, the adder 105 supplies the frame image data supplied from the inverse orthogonal transform unit 104 to the screen rearrangement unit 106 without performing the above-described addition processing.

  The screen rearrangement unit 106 rearranges the frame image data supplied from the inverse orthogonal transform unit 104 in the time axis direction. Then, the screen rearrangement unit 106 supplies the frame image data rearranged in the time axis direction to the filter processing unit 121 as a baseband video signal.

  The frame memory 107 stores the frame image data supplied from the inverse orthogonal transform unit 104 as reference frame image data for decoding the inter-coded frame image data.

  When the frame image data to be processed is inter-coded, the motion compensation unit 108 performs prediction according to the motion vector supplied from the lossless decoding unit 102 and the reference frame image data stored in the frame memory 107. Frame image data is generated and supplied to the adder 105.

  The decoding processing unit 10 having the above configuration decodes encoded data input from the outside into a baseband video signal, and supplies the decoded baseband video signal to the filter processing unit 121. Furthermore, in the decoding processing unit 10 according to the present embodiment, the lossless decoding unit 102 uses the first quantization value and the generated bit amount stored in the header part of the macroblock of the frame image data during the lossless decoding process. The first difficulty level indicating the difficulty of encoding the image signal by the encoding method is extracted and supplied to the filter control unit 122. In addition to the first difficulty level described above, the lossless decoding unit 102 includes scene change information indicating whether or not there is a scene change in frames before and after the image signal, and motion included in the inter-coded frame image data. Vector information is extracted and supplied to the filter control unit 122.

  In addition, the encoding processing unit 11 performs a process of converting the first encoding method in which the image signal is decoded by the decoding processing unit 10 into encoded data of the second encoding method. Replacement unit 109, subtractor 110, orthogonal transform unit 111, quantization unit 112, lossless encoding unit 113, buffer 114, inverse quantization unit 115, inverse orthogonal transform unit 116, and frame memory 117 A motion vector generation unit 118, a motion compensation unit 119, and a rate control unit 120.

  The screen rearrangement unit 109 converts each frame image data constituting the image signal supplied from the filter processing unit 121 described later according to a GOP (Group of Picture) structure of image compression information according to the second encoding method. The frame image data is rearranged and supplied to the subtractor 110. Further, the screen rearrangement unit 109 supplies the frame image data to the motion vector generation unit 118 when performing the inter encoding process on the frame image data supplied to the subtractor 110.

  The subtractor 110 calculates image data indicating a difference between the frame image data and predicted frame image data supplied from a motion compensation unit 119 described later when performing inter coding processing on the frame image data. The processed image data is supplied to the orthogonal transform unit 111. Note that when the intra-coding process is performed on the frame image data, the subtractor 110 does not perform the above-described difference process on the frame image data supplied from the screen rearrangement unit 109.

  The orthogonal transform unit 111 performs orthogonal transform processing such as discrete cosine transform and Karhunen-Loeve transform on the frame image data supplied from the subtractor 110 and supplies orthogonal transform coefficients to the quantization unit 112.

  The quantization unit 112 quantizes the orthogonal transform coefficient supplied from the orthogonal transform unit 111 with a quantized value corresponding to a control command from the rate control unit 120, which will be described later, and converts the quantized orthogonal transform coefficient into a lossless code. To the quantization unit 113 and the inverse quantization unit 115.

  The lossless encoding unit 113 performs lossless encoding processing such as variable length encoding and arithmetic encoding on a motion vector supplied from a motion vector generation unit 118 described later. Then, the lossless encoding unit 113 stores the motion vector that has been subjected to the above-described lossless encoding process in the header portion of the encoded data that includes the quantized orthogonal transform coefficients supplied from the quantization unit 112. The encoded data is supplied to the buffer 114.

  The buffer 114 stores the encoded data supplied from the lossless encoding unit 113, and outputs the stored encoded data to the outside as an image signal that becomes a continuous bit stream. Further, the buffer 114 supplies information related to the bit rate of the bit stream output to the outside to the rate control unit 120.

  The inverse quantization unit 115 performs inverse quantization processing on the quantized orthogonal transform coefficient supplied from the quantization unit 112 and supplies the inversely quantized orthogonal transform coefficient to the inverse orthogonal transform unit 116.

  The inverse orthogonal transform unit 116 performs an inverse orthogonal transform process on the orthogonal transform coefficient supplied from the inverse quantization unit 115 and supplies the decoded frame image data to the frame memory 117.

  The frame memory 117 stores the decoded frame image data supplied from the inverse orthogonal transform unit 116 as reference frame image data.

  The motion vector generation unit 118 generates a motion vector from the frame image data supplied from the screen rearrangement unit 106, and supplies the generated motion vector to the lossless encoding unit 113 and the motion compensation unit 119.

  The motion compensation unit 119 generates predicted frame image data according to the motion vector generated by the motion vector generation unit 118 and the reference frame image data stored in the frame memory 117, and generates the generated predicted frame image data. Is supplied to the subtractor 110.

  The rate controller 120 controls the bit rate of the encoded data according to the bit rate information supplied from the buffer 114 and the set bit rate information supplied from the user interface 123. Set the quantization value to quantize the coefficients.

  The encoding processing unit 11 configured as described above is not limited to a specific method as long as the first encoding method and the second encoding method are different from each other, but particularly than the first encoding method. A high-quality image signal is obtained by effectively using the storage area by performing re-encoding processing on the image signal using the encoding method with high compression efficiency as the second encoding method and storing the image signal in a storage medium such as a DVD. Can be stored in a storage medium.

  Further, the encoding processing unit 11 obtains information about the quantization value when the quantization unit 112 quantizes the orthogonal transform coefficient and the generated bit amount of the encoded data generated by the lossless encoding unit 113, as the second Is supplied to the filter control unit 122 as the second difficulty level indicating the difficulty of encoding the image signal by the encoding method.

  Next, a filter processing unit 121 that performs filter processing on an image signal such as a baseband video signal decoded by the decoding processing unit 10, and a filter control unit 122 that controls operating characteristics of the filter processing unit 121. Such a configuration will be described with reference to FIGS.

  Specifically, the filter processing unit 121 reduces block noise, frame noise, mosquito noise, and the like included in the image signal encoded by the first encoding method. In order to perform the process of removing the high frequency component, as shown in FIG. 3, the high pass filter 121a for extracting the high frequency component of the image signal and the gain of the image signal of the high frequency component extracted by the high pass filter 121a are set. A gain adjustment unit 121b for adjustment and an adder 121c for adding the image signal whose gain is adjusted by the gain adjustment unit 121b from the input image signal are provided.

  The high pass filter 121a performs a process of extracting a high frequency component from the image signal supplied from the decoding processing unit 10. Specifically, the high-pass filter 121a performs a weighted average operation from the pixel value of the pixel of interest of the frame image data constituting the image signal and the pixel value of the adjacent pixel of the pixel of interest, thereby obtaining a high frequency filter for each frame image data. Extract frequency components. Then, the high pass filter 121a supplies the image signal obtained by extracting the high frequency component to the gain adjusting unit 121b.

  For example, as shown in FIG. 4, the gain adjustment unit 121b adjusts the gain of the image signal supplied from the high-pass filter 121a so as to monotonously decrease to a negative value as the pixel value of each pixel increases. To do. That is, the gain adjustment unit 121b generates an image signal obtained by inverting the pixel value of the high frequency component. Then, the gain adjustment unit 121b supplies the image signal with the gain adjusted as described above to the adder 121c.

  The adder 121c adds the image signal supplied from the decoding processing unit 10 and the image signal supplied from the gain adjustment unit 121b. Here, the image signal supplied from the decoding processing unit 10 is added with the image signal obtained by inverting the pixel value of the high frequency component. Therefore, the adder 121c outputs the image signal from which the high frequency component has been removed. Then, the adder 121c supplies the image signal subjected to the addition process to the encoding processing unit 11.

  The filter processing unit 121 configured as described above divides one frame image data into a total of 16 image areas of 4 × 4 as shown in FIG. 5, for example, in accordance with a control command from the filter control unit 122 described later. Then, for each image area, the filter processing intensity is controlled by determining the filter coefficient based on the characteristics of the high-pass filter 121a and the gain characteristic of the gain adjusting unit 121b, so that the high frequency component from the image signal as described above is controlled. The process which removes is performed.

  Dividing into a plurality of image areas in this way is because the degree of difficulty in encoding and the complexity of the pattern are not uniform within one screen, and it is necessary to perform filter processing with an intensity suitable for each image area. It is.

  Here, if the difference in the intensity of the filter processing between adjacent image regions is large, there is a possibility that a boundary line is seen in the image signal subjected to the filter processing. Therefore, the filter processing unit 121 provides a smoothing region for performing the smoothing process indicated by the dotted line and the hatched line in FIG. 5 at the boundary portion of each image region, and the filter processing strength of this smoothing region is set to the adjacent image region. Using the strength of the filter process, a smoothing process is performed to make the boundary line difficult to see visually.

  The filter processing unit 121 is not limited to the case where one level of smoothing region is provided in each of the vertical direction and the horizontal direction of the boundary portion of each image region as illustrated in FIG. 5. For example, as illustrated in FIG. Filtering processing may be performed by providing smoothing regions having different intensities in two stages in the vertical direction and the horizontal direction of the boundary portion of the image region.

  When a smoothing region as shown in FIG. 6 is provided, the filter processing unit 121 smoothes the filter strength for each portion obtained by dividing the target image region A into a total of nine. Here, the attention image area A is adjacent to the image area B in the right direction, adjacent to the image area C in the lower direction, and adjacent to the image area D in the lower right direction as shown in FIG. And Each image region has a first smoothing region X1 with a vertical boundary line as a central axis, and a horizontal width greater than that of the first smoothing region X1 with the vertical boundary line as a central axis. It is assumed that a second smoothing region X2 excluding the first smoothing region X1 is provided. Furthermore, each image region has a third smoothing region Y1 with the horizontal boundary line as the central axis, and a width in the vertical direction with respect to the horizontal smoothing line Y1 with the horizontal boundary line as the central axis. It is assumed that a fourth smoothing region Y2 excluding the third smoothing region Y1 is provided.

  The target image area A is divided into a part A0 that is not subjected to the smoothing process and a total of eight parts A1 to A8 that are subjected to the smoothing process. The portion A1 is a portion corresponding only to the second smoothing region X2. The portion A2 is a portion corresponding only to the first smoothing region X1. The portion A3 is a portion corresponding only to the fourth smoothing region Y2. The portion A4 is an overlapping portion between the second smoothing region X2 and the fourth smoothing region Y2. The portion A5 is an overlapping portion between the first smoothing region X1 and the fourth smoothing region Y2. The portion A6 is a portion corresponding only to the third smoothing region Y1. The portion A7 is an overlapping portion between the second smoothing region X2 and the third smoothing region Y1. The portion A8 is an overlapping portion between the first smoothing region X1 and the third smoothing region Y1.

  The filter processing unit 121 has S (A), S (B), S (C), and S (D, respectively, with the intensity of the filter processing of the image areas A, B, C, and D set by the filter control unit 122 described later. ), The intensities S (A0) to A (A8) of the filter processing of the portions A0 to A8 constituting the target image area A are calculated by the following equations (1) to (9).

S (A0) = S (A) (1)
S (A1) = (3 × S (A) + S (B)) / 4 Formula (2)
S (A2) = (S (A) + S (B)) / 2 Formula (3)
S (A3) = (3 × S (A) + S (C)) / 4 Formula (4)
S (A4) = (9 × S (A) + 3 × S (B) + 3 × S (C) + S (D)) / 16 Equation (5)
S (A5) = (3 × S (A) + 3 × S (B) + S (C) + S (D)) / 8 (8)
S (A6) = (S (A) + S (C)) / 2 Formula (7)
S (A7) = (3 × S (A) + S (B) + 3 × S (C) + S (D)) / 8 (8)
S (A8) = (S (A) + S (B) + S (C) + S (D)) / 4 (9)

  In this way, the filter processing unit 121 calculates the above-described weighted average value by using the filter processing strength of the target image region and the filter processing strength of the image region adjacent to the target image region. The intensity of the filtering process can be smoothed to make it difficult to see the boundary line between adjacent image areas.

  In addition, it is not desirable to change the strength of the filtering process of image regions belonging to the same position in the time axis direction suddenly, for example, every frame, because it causes image quality deterioration. Therefore, the filter processing unit 121 sets the filter processing intensity of image regions belonging to the same position as, for example, an average value for a plurality of frames continuous in the time axis direction. By smoothing the strength of the filter processing in the time axis direction in this way, the filter processing unit 121 reduces the image quality deterioration of the image signal subjected to the filter processing so as not to change the strength of the filter processing suddenly. be able to.

  Specifically, the filter control unit 122 that controls the operation characteristics of the filter processing unit 121 configured as described above calculates filter coefficients that determine the characteristics of the high-pass filter 121a and the gain characteristics of the gain adjustment unit 121b. That is, the filter control unit 122 controls the strength for removing the high frequency component of the image signal performed by the filter processing unit 121, so that the memory 122a, the memory 122b, the weighting factor setting unit 122c, the multiplier 122d, A multiplier 122e, an adder 122f, and a filter coefficient calculation unit 122g are provided.

  Information related to the first difficulty level supplied from the decryption processing unit 10 is stored in the memory 122a. Specifically, the filter control unit 122 calculates the sum of the multiplication values of the quantized value and the generated bit amount for each image area, and stores the calculated value in the memory 122a as the first difficulty level of each image area. Remember. The filter control unit 122 sequentially supplies the first difficulty level stored in the memory 122a to the multiplier 122d for each image area.

  Information relating to the second difficulty level supplied from the encoding processing unit 11 is stored in the memory 122b. Specifically, the filter control unit 122 calculates the sum of the multiplication values of the quantization value and the generated bit amount for each image area, and stores the calculated value in the memory 122b as the second difficulty level of each image area. Remember me. The filter control unit 122 sequentially supplies the second difficulty level stored in the memory 122b to the multiplier 122e for each image region.

  The weighting factor setting unit 122c, according to the motion vector supplied from the decoding processing unit 10, a first weighting factor α that is multiplied by a first difficulty level by a multiplier 122d described later, and a multiplier 122e described later. A second weighting coefficient β to multiply the difficulty level of 2 is set.

  Specifically, the weight coefficient setting unit 122c calculates an average value and a variance value of motion vectors for each image area described above from the motion vector information. Then, the weighting factor setting unit 122c sets, as the first weighting factor α, a value that monotonically increases from 0 to 1 according to the calculated magnitude of motion in the image area, and sets the magnitude of motion in the image area. Accordingly, a value that monotonously decreases from 1 to 0 is set as the second weighting coefficient β. Here, the value of the second weighting coefficient β is 1−α.

  In this way, the weighting factor setting unit 122c sets the first weighting factor α to be larger than the second weighting factor β for the image region with large motion, and sets the first weighting factor α for the image region with small motion. The weighting factor α is set smaller than the second weighting factor β. The weighting factor setting unit 122c supplies the first weighting factor α to the multiplier 122d and the second weighting factor β to the multiplier 122e.

  The multiplier 122d multiplies the first difficulty level supplied from the memory 122a by the first weighting factor α supplied from the weighting factor setting unit 122c and supplies the result to the adder 122f.

  The multiplier 122e multiplies the second difficulty level supplied from the memory 122b by the first weighting factor β supplied from the weighting factor setting unit 122c and supplies the result to the adder 122f.

  The adder 122f adds the multiplication value supplied from the multiplier 122d and the multiplication value supplied from the multiplier 122e, and supplies the result to the filter coefficient calculation unit 122g as an encoding difficulty level subjected to weighting processing. .

  The filter coefficient calculation unit 122g controls the characteristics of the filter processing unit 121 as follows, for example, according to the encoding difficulty level supplied from the adder 122f and the set bit rate supplied from the user interface 123. The filter coefficient for calculating is calculated.

  That is, the filter coefficient calculation unit 122g stores in advance an internal memory such as a register such as a table showing the correspondence relationship between the encoding difficulty level, the set bit rate, and the filter strength as shown in FIG. The filter coefficient of the filter processing unit 121 is calculated according to the filter strength obtained with reference to FIG.

  In this table, the encoding difficulty level is assigned to 21 levels from 0 to 20 from the low difficulty level, and the set bit rate is assigned to the re-encoding modes EP, LP, SP, and XP from the low value. Here, the re-encoding mode EP is a mode in which the image quality is lower than that of the other modes, but can be stored for a long time when the re-encoded image signal is stored in the storage medium. In addition, the re-encoding mode XP is a mode that has higher image quality than other modes but has a short recordable time.

  The filter coefficient calculation unit 122g performs 0 to 6 from the weakest intensity for each of 84 conditions corresponding to the four levels of encoding difficulty and the four re-encoding modes for the above-described table. Are set in advance and stored in an internal memory such as a register in advance. Since the filter coefficient calculation unit 122g calculates the filter coefficient of the filter processing unit 121 so as to have an appropriate strength, as shown in FIG. 8, the value of the set bit rate corresponding to each re-encoding mode becomes high. Accordingly, the table is set so that the value of the filter strength is decreased along with the increase of the value of the difficulty level of encoding as the value of the encoding difficulty level is increased. Note that, as described above, the filter coefficient calculation unit 122g decreases the filter strength value as the set bit rate value corresponding to each re-encoding mode increases, and the value of the encoding difficulty level. The table is not limited to that shown in FIG. 8 as long as the table is set so that the value of the filter strength increases as the value increases.

  The filter coefficient calculation unit 122g having such a table associates with this table from the encoding difficulty level supplied from the adder 122f and the re-encoding mode corresponding to the set bit rate supplied from the user interface 123. The filter coefficient is calculated with reference to the obtained filter strength and supplied to the filter processing unit 121.

  The filter control unit 122 configured as described above calculates the filter coefficient of the filter control unit 122 according to the flowchart shown in FIG.

  In step S <b> 1, the filter control unit 122 determines whether there is a scene change in the stream of the image signal supplied from the decoding processing unit 10 based on the scene change information D <b> 1 supplied from the decoding processing unit 10. Here, the filter control unit 122 proceeds to step S2 when there is no scene change, and proceeds to step S4 when there is a scene change.

  In step S2, the filter control unit 122 sets the first weighting factor α and the second weighting factor β by the weighting factor setting unit 122c according to the motion vector information supplied from the decoding processing unit 10.

  In step S3, the filter control unit 122 multiplies the first weighting factor α set in step S2 by the multiplier 122d and the first difficulty D3 stored in the memory 122a, and the multiplier 122e performs the step. The second weighting factor β set in S2 is multiplied by the second difficulty level D4 stored in the memory 122b, and a value obtained by adding these two multiplied values by the adder 122f is calculated as the encoding difficulty level. Then, the process proceeds to step S6.

  In step S4, the filter control unit 122 sets the value of the first weighting factor α to 1 and the value of the second weighting factor β to 0 by the weighting factor setting unit 122c. And the filter control part 122 progresses to the process process of step S5.

  In step S5, the filter control unit 122 multiplies the first weighting coefficient α set in step S4 by the multiplier 122d and the first difficulty D3 stored in the memory 122a, and the multiplier 122e performs the step. The second weight coefficient β set in S4 is multiplied by the second difficulty level D4 stored in the memory 122b, and a value obtained by adding these two multiplied values by the adder 122f is calculated as the encoding difficulty level. . Here, since the filter control unit 122 sets the value of the first weighting factor α to 1 and the value of the second weighting factor β to 0, the first difficulty level stored in the memory 122a. D1 is calculated as the encoding difficulty level that has been subjected to the weighting process, and the process proceeds to step S6.

  In step S6, the filter control unit 122 sets the encoding difficulty level quantized in increments of 0 to 21 as described above from the encoding difficulty level calculated in step 3 or step S5 by the filter coefficient calculation unit 122g. Then, the process proceeds to step S7.

  In step S7, the filter control unit 122 uses the encoding difficulty level set in step S6 by the filter coefficient calculation unit 122g and the encoding mode corresponding to the set bit rate D5 supplied from the user interface 123, Filter strength is selected by referring to the table. Then, the filter control unit 122 calculates the filter coefficient D6 corresponding to the selected filter strength by the filter coefficient calculation unit 122g, supplies the filter coefficient D6 to the filter processing unit 121, and ends this processing step.

  Through the processing steps as described above, the filter control unit 122 is the first difficulty indicating the difficulty of encoding the image signal by the first encoding method extracted when the decoding processing unit 10 decodes the image signal. And a second difficulty level indicating the difficulty of encoding the image signal by the second encoding method extracted when the encoding processing unit 11 encodes the image signal. Since 121 controls the strength of the filter processing applied to the image signal, the filter coefficient of the filter processing unit 121 can be appropriately determined for the following reason.

  The first reason is that, generally, the degree of difficulty of encoding with the second encoding method does not change for an image with little motion in the time axis direction, that is, an image with a small motion vector value. Therefore, it is desirable to control the filter strength using the second difficulty level that has a correspondence relationship with the second encoding method rather than the first difficulty level. The second reason is that when the second difficulty level has been subjected to encoding processing in the past for an image having a large motion in the time axis direction such as an image region having a large scene change or motion vector value. This is because it is a value indicating the difficulty of encoding, and it is desirable to control the filter strength using the first difficulty level in which the correspondence relationship with the motion information of the image region currently being processed is accurately taken.

  Paying attention to such characteristics, the filter control unit 122 controls the filter strength using only the first difficulty level when there is a scene change in the determination process of step S1. Further, when there is no scene change, the filter control unit 122 sets the first difficulty level and the second difficulty level according to the magnitude of the motion of the image area based on the motion vector information in step S2 and step S3. The filter strength is controlled using the weighted average value as the encoding difficulty level.

  The filter control unit 122 is not limited to the case where the filter strength is controlled using the encoding difficulty level obtained by the above-described weighted average calculation. For example, the filter strength may be controlled as follows. Good.

  That is, the filter control unit 122 detects a change in the first difficulty level of each image region over the past several frames, and when the change in the first difficulty level is smaller than a predetermined threshold based on the detection result, The filter strength may be controlled using only the first difficulty level. Further, the filter control unit 122 detects a change in the second difficulty level of each image region over the past several frames, and when the change in the second difficulty level is larger than a predetermined threshold based on the detection result, The filter strength may be controlled using only the second difficulty level.

  As described above, the filter control unit 122 is configured according to the motion vector information extracted when the decoding processing unit 10 decodes the image signal, the first difficulty level, and the second difficulty level. As described above, since the intensity of the filter processing applied to the image signal can be appropriately controlled, the image signal is encoded into the second encoding method so that a high compression rate can be realized while maintaining good image quality. be able to.

  In step S7, the filter control unit 122 controls the filter strength according to the set bit rate supplied from the user interface 123 by the filter coefficient calculation unit 122g. Specifically, the filter coefficient calculation unit 122g can reduce the compression rate when the bit rate is relatively high, for example, in the re-encoding mode XP, so refer to a table stored in the internal memory in advance. Select a filter strength that is relatively low. Further, the filter coefficient calculation unit 122g refers to a table stored in advance in the internal memory in order to increase the compression rate compared to when the bit rate is relatively low as in the re-encoding mode EP, for example. Select a strong filter strength. By performing such processing by the filter coefficient calculation unit 122g, the filter control unit 122 outputs the image signal so that a high compression rate can be realized while maintaining good image quality according to the re-encoding mode set by the user. The second encoding method can be encoded.

  In the image processing apparatus 1 described above, the filter strength is controlled using a motion vector obtained when the decoding processing unit 10 decodes an image signal. However, as a second embodiment as shown in FIG. The motion vector generated by the motion vector generation unit 118 of the encoding processing unit 11 is supplied to the filter control unit 122, and the filter strength is obtained using the motion vector obtained when the encoding processing unit 11 encodes the image signal. May be controlled.

  Specifically, the filter control unit 122 according to the second embodiment is a weighted average of the first difficulty level and the second difficulty level according to the motion vector supplied from the encoding processing unit 11. Is used as the encoding difficulty level to control the filter strength.

  In the image processing apparatus 1 according to the first and second embodiments described above, the filter control unit 122 performs processing for setting the filter strength in accordance with the motion vector information. In the embodiment, the first difficulty level and the encoding that the filter control unit 122 supplies from the decoding processing unit 10 without supplying the motion vector from the decoding processing unit 10 or the encoding processing unit 11 to the filter control unit 122. The filter strength may be controlled using only the second difficulty level supplied from the processing unit 11.

  The filter control unit 122 according to the third embodiment controls the filter strength from the first difficulty level and the second difficulty level, for example, as follows.

  That is, the filter control unit 122 detects a change in the first difficulty level of each image region over the past several frames, and when the change in the first difficulty level is smaller than a predetermined threshold based on the detection result, The filter strength is controlled using only the first difficulty level. Further, the filter control unit 122 detects a change in the second difficulty level of each image region over the past several frames, and when the change in the second difficulty level is larger than a predetermined threshold based on the detection result, The filter strength is controlled using only the second difficulty level.

  Note that, as described above, the filter control unit 122 is not limited to the case where the filter strength is controlled using only the first difficulty level or the second difficulty level according to the determination result according to the threshold. The encoding difficulty level is a value obtained by performing a weighted average operation on the first difficulty level and the second difficulty level according to changes in the first difficulty level and the second difficulty level for the past several frames. The filter strength may be set.

  Through the processing steps as described above, the filter control unit 122 is the first difficulty indicating the difficulty of encoding the image signal by the first encoding method extracted when the decoding processing unit 10 decodes the image signal. And a second difficulty level indicating the difficulty of encoding the image signal by the second encoding method extracted when the encoding processing unit 11 encodes the image signal. Since 121 controls the strength of the filter processing applied to the image signal, the image signal having a high compression rate is encoded into the second encoding method while suppressing deterioration in image quality caused by the re-encoding processing for the following reason. Can do.

  The first reason is that, for an image that generally has little movement in the time axis direction, the degree of difficulty of encoding with the second encoding method does not change, so the second difficulty level is higher than the first difficulty level. This is because it is desirable to control the filter strength using the second difficulty level corresponding to the encoding method. The second reason is that when the second difficulty level has been subjected to encoding processing in the past for an image having a large motion in the time axis direction such as an image region having a large scene change or motion vector value. This is because it is a value indicating the difficulty of encoding, and it is desirable to control the filter strength using the first difficulty level in which the correspondence relationship with the motion information of the image region currently being processed is accurately taken.

  The image processing apparatus 1 according to the first to third embodiments described above controls the filter strength using at least the first difficulty level and the second difficulty level, and is shown in FIG. The image processing apparatus 1 according to the fourth embodiment as described above is an apparatus that temporarily stores an image signal encoded by the first encoding method input from the outside in a storage device, and then performs re-encoding processing. It is.

  Specifically, the image processing apparatus 1 according to the fourth embodiment includes an input image storage unit 131, a difficulty level information extraction unit 132, in addition to the configuration of the image processing apparatus 1 according to the first embodiment described above. The difficulty level information storage unit 133 and the bit rate calculation unit 134 are provided.

  The input image storage unit 131 is formed of a storage medium such as a hard disk, for example, stores an image signal encoded by the first encoding method input from the outside, and supplies the stored image signal to the buffer 101.

  The difficulty level information extraction unit 132 analyzes the header information of the image signal encoded by the first encoding method input from the outside, and determines the quantization value, the generated bit amount for each macroblock, and the first difficulty level. Extracted as information and supplied to the difficulty level information storage unit 133.

  The difficulty level information storage unit 133 stores information related to the first difficulty level supplied from the difficulty level information extraction unit 132 in association with an image signal stored in the input image storage unit 131 for each image area. To do. The difficulty level information storage unit 133 supplies the first difficulty level corresponding to the image signal supplied from the input image storage unit 131 to the buffer 101 to the filter control unit 122 and the bit rate calculation unit 134. As described above, in the image processing device 1 according to the fourth embodiment, at the time of the re-encoding process, the decoding processing unit 10 does not supply the information related to the first difficulty level to the filter control unit 122 in advance. The information related to the first difficulty level can be supplied from the difficulty level information storage unit 133 storing the information related to the difficulty level to the filter control unit 122.

  The bit rate calculation unit 134 calculates a bit rate suitable for re-encoding from the quantization value supplied as information related to the first difficulty level from the difficulty level information storage unit 133 and the generated bit amount in units of macroblocks. To the user interface 123. Specifically, the bit rate calculation unit 134 multiplies the quantization value and the generated bit amount in units of macroblocks, calculates the sum of the multiplication values for each frame, and calculates optimum bit rate information for each frame. .

  The bit rate information supplied to the user interface 123 is presented to the user as an optimum bit rate for re-encoding by a display device or the like. In the user interface 123, the set bit rate is set by the user according to the bit rate information presented to the user, and the set bit rate is supplied to the rate control unit 120.

  As described above, the image processing apparatus 1 according to the fourth embodiment can present the optimum bit rate information to the user and set a more optimum bit rate according to the user's preference. For example, in the image processing apparatus 1, for a user who desires to record on a storage medium for a longer time than image quality degradation, the user interface 123 sets a bit rate lower than the optimum bit rate as the set bit rate. Can be controlled. In addition, the image processing apparatus 1 encodes a bit rate higher than the optimum bit rate as a set bit rate by the user interface 123 for a user who wants to suppress degradation in image quality as much as possible rather than the recording time to the storage medium. The processing unit 11 can be controlled.

  Note that the optimum bit rate information calculated by the bit rate calculation unit 134 may be supplied to the rate control unit 120 of the encoding processing unit 11 without using the user interface 123.

  In the image processing apparatus 1 according to the fourth embodiment having the above-described configuration, re-encoding is performed using information on the first difficulty level that the filter control unit 122 uses to control the filter strength. A suitable bit rate can be calculated. By using a bit rate suitable for such re-encoding, the image processing apparatus 1 encodes a high-compression image signal into the second encoding method while suppressing image quality degradation caused by the re-encoding process. can do. Furthermore, when the image signal encoded in this way is recorded on the recording medium, the image signal can be stored for a longer time in the unit storage area, and the storage area can be used efficiently.

  Next, an embodiment in which the image processing apparatus according to the present invention is applied to a communication system 2 that transmits an image signal via a plurality of transmission media will be described.

  As illustrated in FIG. 13, the communication system 2 includes an encoding device 201, a transmission medium 202, an encoding scheme conversion device 203 to which the present invention is applied, a transmission medium 204, and a decoding device 205. .

  The encoding device 201 supplies an image signal obtained by encoding the baseband video signal by the first encoding method to the encoding method conversion device 203 via the transmission medium 202.

  Specifically, as illustrated in FIG. 14, the encoding device 201 includes an A / D conversion unit 301, a screen rearrangement unit 302, a subtractor 303, an orthogonal transformation unit 304, a quantization unit 305, and a lossless unit. The encoding unit 306 includes a buffer 307, an inverse quantization unit 308, an inverse orthogonal transform unit 309, a frame memory 310, a motion vector generation unit 311, a motion compensation unit 312, and a rate control unit 313.

  The A / D converter 301 converts the image signal of the analog source to be transmitted into an image signal of a digital signal and supplies it to the screen rearrangement unit 302.

  The screen rearrangement unit 302 rearranges the frame image data of the image signal supplied from the A / D conversion unit 301 according to the GOP structure of the image compression information related to the first encoding method, and supplies the frame image data to the subtracter 303. .

  The subtractor 303 calculates image data indicating a difference between the frame image data and predicted frame image data supplied from a motion compensation unit 312 described later when performing inter coding processing on the frame image data. The processed image data is supplied to the orthogonal transform unit 304. Note that when the intra-coding process is performed on the frame image data, the subtractor 303 does not perform the above-described difference process on the frame image data supplied from the screen rearrangement unit 302.

  The orthogonal transform unit 304 performs orthogonal transform processing such as discrete cosine transform and Karhunen-Loeve transform on the frame image data supplied from the subtractor 303 and supplies orthogonal transform coefficients to the quantization unit 305.

  The quantization unit 305 quantizes the orthogonal transform coefficient supplied from the orthogonal transform unit 304 with a quantization value corresponding to a control command supplied from the rate control unit 313 described later, and converts the quantized orthogonal transform coefficient into a quantized orthogonal transform coefficient. This is supplied to the lossless encoding unit 306 and the inverse quantization unit 308.

  The lossless encoding unit 306 performs lossless encoding processing such as variable length encoding and arithmetic encoding on a motion vector supplied from a motion vector generation unit 311 described later. Then, the lossless encoding unit 306 stores the motion vector subjected to the above-described lossless encoding process in the header portion of the encoded data including the quantized orthogonal transform coefficients supplied from the quantization unit 305. The encoded data is supplied to the buffer 307.

  The buffer 307 stores the encoded data supplied from the lossless encoding unit 306 and outputs the stored encoded data to the outside as an image signal that becomes a continuous bit stream. Further, the buffer 307 supplies information related to the bit rate of the bit stream output to the outside to the rate control unit 313.

  The inverse quantization unit 308 performs an inverse quantization process on the quantized orthogonal transform coefficient supplied from the quantization unit 305 and supplies the inversely quantized orthogonal transform coefficient to the inverse orthogonal transform unit 309. .

  The inverse orthogonal transform unit 309 performs an inverse orthogonal transform process on the orthogonal transform coefficient supplied from the inverse quantization unit 308 and supplies the decoded frame image data to the frame memory 310.

  The frame memory 310 stores the decoded frame image data supplied from the inverse orthogonal transform unit 309 as reference frame image data.

  The motion vector generation unit 311 generates a motion vector from the frame image data supplied from the screen rearrangement unit 302, and supplies the generated motion vector to the lossless encoding unit 306 and the motion compensation unit 312.

  The motion compensation unit 312 generates predicted frame image data according to the motion vector generated by the motion vector generation unit 311 and the reference frame image data stored in the frame memory 310, and generates the generated predicted frame image data. Is supplied to the subtractor 303.

  The rate control unit 313 controls the quantization value for quantizing the orthogonal transform coefficient in the quantization unit 305 in order to control the bit rate of the encoded data according to the bit rate information supplied from the buffer 307.

  Further, the transmission medium 202 may be a storage medium such as an optical disk, a magnetic disk, and a semiconductor memory, in addition to a broadcast medium and a communication medium such as a satellite broadcast wave, a cable TV network, a telephone line network, and a mobile phone line network. Good.

  The encoding method conversion device 203 decodes an image signal encoded by the first encoding method transmitted from the transmission medium 202 into a baseband video signal, and an image decoded by the decoding device 203a. A filtering device 203b that performs filtering on the signal, and an encoding device 203c that encodes the baseband video signal that has been filtered by the filtering device 203b using the second encoding method and outputs the encoded signal to the transmission medium 204 are provided.

  The description related to the specific configuration of the encoding method conversion apparatus 203 is the same as that of any of the configurations of the image processing apparatus 1 according to the first to fourth embodiments described above, and will be omitted.

  The transmission medium 204 may be a storage medium such as an optical disk, a magnetic disk, and a semiconductor memory in addition to a broadcast medium and a communication medium such as a satellite broadcast wave, a cable TV network, a telephone line network, and a mobile phone line network. .

  The decoding device 205 decodes the image signal encoded by the second encoding method transmitted from the transmission medium 204, and displays the decoded baseband video signal on, for example, a display device.

  Specifically, as illustrated in FIG. 15, the decoding device 205 includes a buffer 401, a lossless decoding unit 402, an inverse quantization unit 403, an inverse orthogonal transform unit 404, an adder 405, and a screen rearrangement unit 406. A D / A conversion unit 407, a frame memory 408, and a motion compensation unit 409.

  The buffer 401 temporarily stores an image signal encoded by the second encoding method, synchronizes with a processing unit that performs other decoding processing, and the lossless decoding unit 402 sequentially stores the image signal. To supply.

  The lossless decoding unit 402 performs processing such as variable length decoding and arithmetic decoding on the image signal supplied from the buffer 401 based on the second encoding method, and quantized orthogonal transform coefficients Is supplied to the inverse quantization unit 403.

  In addition, when the image signal encoded by the second encoding method is inter-encoded frame image data, the lossless decoding unit 402 decodes the motion vector stored in the header portion of the frame image data. The decoded motion vector is supplied to the motion compensation unit 409.

  The inverse quantization unit 403 performs an inverse quantization process on the quantized orthogonal transform coefficient supplied from the lossless decoding unit 402 and supplies the orthogonal transform coefficient to the inverse orthogonal transform unit 404.

  The inverse orthogonal transform unit 404 performs inverse orthogonal transform processing such as inverse discrete cosine transform and inverse Karhunen-Labe transform on the orthogonal transform coefficient generated by the inverse quantization unit 403, and adds the decoded frame image data To the container 405.

  When the frame image data is inter-coded, the adder 405 adds the predicted frame image data generated by the motion compensation unit 409 described later to the frame image data supplied from the inverse orthogonal transform unit 404, and This is supplied to the screen rearrangement unit 406 and the frame memory 408. When the frame image data is intra-coded, the adder 405 supplies the frame image data supplied from the inverse orthogonal transform unit 404 to the screen rearrangement unit 406 without performing the above-described addition processing.

  The screen rearranging unit 406 rearranges the frame image data supplied from the inverse orthogonal transform unit 404 in the time axis direction. Then, the screen rearrangement unit 406 supplies the frame image data rearranged in the time axis direction to the D / A conversion unit 407 as a baseband video signal.

  The D / A conversion unit 407 converts the baseband video signal from a digital signal to an analog signal and outputs the analog signal to a display device or the like.

  The frame memory 408 stores the frame image data supplied from the inverse orthogonal transform unit 404 as reference frame image data for decoding the inter-coded frame image data.

  When the frame image data to be processed is inter-coded, the motion compensation unit 409 performs prediction according to the motion vector supplied from the lossless decoding unit 402 and the reference frame image data stored in the frame memory 408. Frame image data is generated and supplied to the adder 405.

  In the communication system 2 configured as described above, in particular, an image signal encoded by the first encoding method transmitted from the transmission medium 202 is maintained by the encoding method converter 203 while maintaining good image quality. Since the image signal obtained by converting the image signal into the second encoding method so as to achieve a high compression rate can be output to the transmission medium 204, the image signal can be transmitted with higher image quality and higher transmission efficiency. it can.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment. It is a block diagram which shows the whole structure of the image processing apparatus which paid its attention to the structure of a filter process part and a filter control part. It is a block diagram which shows the structure of a filter process part. It is a graph which shows the input-output characteristic of a gain adjustment part. It is a figure which shows the frame image which divided | segmented into a total of 16 image area | regions of 4x4, and provided the smoothing area | region of 1 step | paragraph in the vertical direction and the horizontal direction at each boundary part of an image area, respectively. It is a figure which shows the frame image which divided | segmented into a total of 16 image area | regions of 4x4, and provided the smoothing area | region of 2 steps | paragraphs to each boundary part of an image area in the vertical direction and a horizontal direction, respectively. It is a figure where it uses for description of the smoothing process with respect to an attention image area | region. It is a figure which shows the correspondence of encoding difficulty, re-encoding mode, and filter strength. It is a flowchart with which it uses for description of the operation | movement which concerns on a filter control part. It is a block diagram which shows the structure of the image processing apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the image processing apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the structure of the image processing apparatus which concerns on 4th Embodiment. It is a figure which shows typically the whole structure of a communication system. It is a block diagram which shows the whole structure of a decoding apparatus. It is a block diagram which shows the whole structure of an encoding apparatus.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 2 Communication system, 10 Decoding processing part, 11 Encoding processing part, 101, 114, 307, 401 Buffer, 102, 402 Lossless decoding part, 103, 115, 308, 403 Inverse quantization part, 104, 116, 309, 404 Inverse orthogonal transform unit, 105, 121c, 122f, 405 Adder, 106, 109, 302, 406 Screen rearrangement unit, 107, 117, 310, 408 Frame memory, 108, 119, 312, 409 Motion Compensation unit, 110, 303 Subtractor, 111, 304 Orthogonal transformation unit, 112, 305 Quantization unit, 113, 306 Lossless encoding unit, 118, 311 Motion vector generation unit, 120, 313 Rate control unit, 121 Filter processing unit 121a high-pass filter, 121b gain adjustment unit, 122f Control unit, 122a, 122b memory, 122c weight coefficient setting unit, 122d, 122e multiplier, 122g filter coefficient calculation unit, 123 user interface, 131 input image storage unit, 132 difficulty level information extraction unit, 133 difficulty level information storage unit , 134 Bit rate calculation unit, 201, 203c Encoding device, 202, 204 Transmission medium, 203 Coding method conversion device, 203a, 205 Decoding device, 203b Filter device, 301 A / D conversion unit, 407 D / A conversion unit ,

Claims (9)

Input means for inputting an image signal encoded by the first encoding method;
First difficulty level information indicating the difficulty of encoding the image signal by the first encoding method is extracted from the image signal input by the input means, and the image signal is converted to the first encoding method. Decoding means for decoding based on
Filter processing means for performing filter processing on the image signal decoded by the decoding means;
The second difficulty level information indicating the difficulty of encoding the image signal by the second encoding method different from the first encoding method is extracted from the image signal subjected to the filter processing by the filter processing means. Encoding means for encoding and outputting the image signal based on the second encoding method;
Control means for controlling the strength of the filter processing applied to the image signal by the filter processing means in accordance with the first difficulty level information extracted by the decoding means and the second difficulty level information extracted by the encoding means. An image processing apparatus comprising: The decoding unit divides the image of the image signal input by the input unit into predetermined image areas, extracts the first difficulty level information for each divided image area, and extracts the image signal from the first image area. Decoding based on the encoding scheme of 1,
The filter processing means performs a filter process of intensity set for each image area on each image area of the image signal decoded by the decoding means,
The encoding means divides the image of the image signal filtered by the filter processing means into the image areas, extracts the second difficulty level information for each divided image area, and Encoding a signal based on the second encoding scheme;
The control means includes the first difficulty level information corresponding to each image area extracted by the decoding means and the second difficulty level information corresponding to each image area extracted by the encoding means. The image processing apparatus according to claim 1, wherein the intensity of the filter processing performed by the filter processing unit is controlled for each image region. The decoding means extracts the first difficulty level information and the motion vector from the image signal input by the input means, and decodes the image signal based on the first encoding method;
The control means performs the filtering process according to the first difficulty level information extracted by the decoding means, the second difficulty level information extracted by the encoding means, and the motion vector extracted by the decoding means. 2. The image processing apparatus according to claim 1, wherein the intensity of the filtering process applied to the image signal by the means is controlled. The encoding means extracts the second difficulty level information and a motion vector from the image signal filtered by the filter processing means, and extracts the image signal based on the second encoding method. Encoding,
The control means includes the filter according to the first difficulty level information extracted by the decoding means, the second difficulty level information extracted by the encoding means, and the motion vector extracted by the encoding means. 2. The image processing apparatus according to claim 1, wherein the processing means controls the strength of the filter processing applied to the image signal.   The control means, according to the change in the time axis direction of the first difficulty information extracted by the decoding means and the change in the time axis direction of the second difficulty information extracted by the decoding means, 2. The image processing apparatus according to claim 1, wherein the filter processing means controls the strength of the filter processing applied to the image signal. Bit rate setting means for setting the bit rate of the image signal encoded by the encoding means,
In accordance with the first difficulty level information extracted by the decoding means, the second difficulty level information extracted by the encoding means, and the bit rate set by the bit rate setting means, 2. The image processing apparatus according to claim 1, wherein the filter processing means controls the strength of the filter processing applied to the image signal. A first storage means for storing the image signal input by the input means and reading the stored image signal to the decoding means;
Extraction means for extracting the quantized value of the image signal and the generated bit amount per unit data as the first difficulty level information from the image signal input by the input means;
Second storage means for storing the first difficulty level information extracted by the extraction means in association with the image signal stored in the first storage means;
A bit rate calculating means for calculating a bit rate according to the quantization value stored as the first difficulty level information in the second storage means and the generated bit amount;
7. The image processing apparatus according to claim 6, wherein the bit rate setting means sets the bit rate calculated by the bit rate calculating means to a bit rate of an image signal encoded by the encoding means. A first storage means for storing the image signal input by the input means and reading the stored image signal to the decoding means;
Extraction means for extracting the quantized value of the image signal and the generated bit amount per unit data as the first difficulty level information from the image signal input by the input means;
Second storage means for storing the first difficulty level information extracted by the extraction means in association with the image signal stored in the first storage means;
Bit rate calculating means for calculating a bit rate suitable for encoding by the encoding means in accordance with the quantization value stored as the first difficulty level information and the generated bit amount in the second storage means And further comprising
The bit rate setting means presents the bit rate calculated by the bit rate calculation means, and sets the bit rate of the image signal encoded by the encoding means according to the presented bit rate. The image processing apparatus according to claim 6. An input step of inputting an image signal encoded by the first encoding method;
First difficulty level information indicating the difficulty of encoding the image signal by the first encoding method is extracted from the image signal input by the input step, and the image signal is extracted from the first encoding method. A decoding step for decoding based on:
A filter processing step of performing filter processing on the image signal decoded by the decoding step;
While extracting the second difficulty level information indicating the difficulty of encoding the image signal by the second encoding method different from the first encoding method from the image signal subjected to the filtering process by the filtering process step An encoding step of encoding and outputting the image signal based on the second encoding method;
The intensity of the filter processing applied to the image signal in the filter processing step is controlled according to the first difficulty level information extracted in the decoding step and the second difficulty level information extracted in the encoding step. An image processing method including a control step.

Images