JPH07222064A - Image processing circuit - Google Patents

Abstract

PURPOSE:To allow the circuit to convert a format and to improve picture quality with simple configuration. CONSTITUTION:Plural image format conversion means 2, 3 convert respectively an input image D1 into image D2, D3 whose picture element number is equal to each other but whose spatial resolution differs from each other. A high frequency component detection means 22 detects the magnitude of a high frequency component of the input image D1. An image adaptive synthesis means 23 synthesizes output images from the image format conversion means whose spatial resolution differs depending on the magnitude of the detected high frequency components to provide an output of a composited image D6.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing circuit, and can be applied to an image data transmission device such as a video telephone device or a video conference device.

[0002]

2. Description of the Related Art Conventionally, in image data transmission devices for still images and moving images such as video telephone devices and video conference devices, CCITT (International Telegraph and Telephone Consultative Committee) recommendation H.264. By transmitting image data (hereinafter referred to as CIF image data) according to the format defined by H.261, efficient transmission is possible. That is, even in the transmission between the NTSC area and the PAL / SECAM area, the CIF image data according to the intermediate common format can be efficiently transmitted.

Here, the CIF image data is 28 in the vertical direction.
While one frame image is formed with 8 pixels and 360 pixels in the horizontal direction, for example, image data in the NTSC format forms one frame image with 480 pixels in the vertical direction and 720 pixels in the horizontal direction. Therefore, NTSC on the sending side
It is necessary to convert the number of pixels of format image data (hereinafter referred to as NTSC image data) to match the number of pixels of CIF image data, and the receiving side needs to perform the inverse conversion.

In addition to the format conversion as described above, in the image data transmission apparatus for still images and moving images, the image data is compressed and encoded for transmission in order to be transmitted efficiently. And a decoding configuration is provided as a reception configuration.

[0005]

In such an image data transmitting apparatus, generally, the receiving side performs format reverse conversion after decoding the compressed data. Here, even if the CIF image data compressed by the compression decoding is decoded into the normal CIF image data, the distortion (block distortion) introduced in the CIF image data through the compression encoding / decoding processing. And mosquito noise), and a post-processing filter is required to remove these distortions and improve the image quality. A post-processing filter is generally provided before the format inverse transform configuration because the effect of distortion is magnified if the format inverse transform is performed.

By the way, the format inverse conversion structure is a kind of digital filter because it is an interpolation processing structure using a plurality of pixel data. Therefore, if it is possible to reduce the image distortion by entrusting the function of the post-processing filter to the filter function of the format inverse conversion structure, it is desirable that the structure of this kind of image data transmission device can be simplified accordingly.

The problem has been described above by taking the case of decoding the compressed CIF image data and then converting the format into the NTSC image data. However, not only the image data transmission apparatus as described above, but also other image processing apparatuses that handle image data are required to be able to remove a predetermined distortion at the time of format conversion.

[0008]

According to the present invention, a plurality of format-converted images of an input image are created, and the plurality of images have different spatial resolutions. Then, the magnitude of the high frequency component of the input image is detected,
Corresponding to the magnitude of the high frequency component, a plurality of format-converted converted images are combined on a pixel-by-pixel basis to form a combined image whose spatial resolution changes.

[0009]

When the format of the input image is converted into an image having different numbers of vertical and horizontal pixels, a plurality of pixel data are weighted and added to create the converted pixel data. In this case, the spatial resolution differs depending on the selection of weighting, and in the present invention, the images are converted into a plurality of images having different spatial resolutions. The optimum converted image differs depending on the data of the input image. For example, a converted image having a high spatial resolution is preferable in a portion of the input image where the size of the high frequency component is large. Therefore, if a plurality of format-converted converted images are combined on a pixel-by-pixel basis in accordance with the size of the high-frequency component of the input image, it is possible to obtain a preferable image according to the characteristics of the input image.

That is, in the image processing circuit of the present invention, the format conversion process and the image quality improvement process of the input image are simultaneously executed with a simple configuration.

[0011]

【Example】

(1) First Embodiment Hereinafter, a first embodiment in which the image processing circuit of the present invention is applied to a receiving configuration of an image data transmission device (for example, a videophone device) will be described in detail.

In FIG. 1, the CIF image data D1 which has undergone the following processing is input to the image processing circuit 1 of the first embodiment.

That is, on the transmitting side, an image signal of a predetermined format (hereinafter referred to as NTSC format) obtained by picking up an image of an object to be picked up by a predetermined image pickup means is converted into a digital signal at a predetermined sampling period, and NTS is converted.
C image data is generated and this NTSC image data is converted into CI
After being converted into F image data, compression encoding is performed in a predetermined block unit to generate transmission image data, and this transmission image data (along with a voice signal) is sent to a communication partner (call target). On the receiving side, the received transmission image data is decoded and converted into CIF image data D1, and this CIF image data D1 is input to the image processing circuit 1 shown in FIG.

The image processing circuit 1 of the first embodiment receives the CIF image data D1 and supplies the CIF image data D1 to the two image format conversion circuits 2 and 3 through a predetermined buffer memory.

The image format conversion circuit 2 is formed by a two-dimensional digital filter in which a vertical direction conversion circuit 4 and a horizontal direction conversion circuit 5 are connected in series, and the image format conversion circuit 3 also has a vertical direction conversion circuit 6 and a horizontal direction. The conversion circuit 7 is formed by a two-dimensional digital filter connected in series, whereby the CIF image data D1 is converted into two NTSC image data D2 and D3.

As shown in FIG. 2, each of the vertical direction conversion circuits 4 and 6 interpolates the CIF image data D1 between a plurality of vertically continuous pixels to obtain 288 vertical pixels,
A CIF format input image G1 that forms one frame image with 360 horizontal pixels is converted into an intermediate image G2 that forms one frame image with 480 vertical pixels and 360 horizontal pixels, while each horizontal direction conversion circuit 5, 7 performs interpolation calculation processing on the image data of the intermediate image G2 between a plurality of pixels continuous in the horizontal direction to form an intermediate image G2 that forms one frame image of 480 pixels in the vertical direction and 360 pixels in the horizontal direction.
N to form one frame image with 0 pixels and 720 pixels horizontally
The output image G3 in the TSC format is converted.

Here, the image format conversion circuit 2 is
Compared to the image format conversion circuit 3, the conversion coefficient of the interpolation process is selected so as to generate the output image G3 having a high spatial resolution, and therefore the CIF image data D1 is the NTSC image data of two systems having different spatial resolutions. D2 and D
Converted to 3.

FIG. 3 shows a detailed configuration example of the vertical direction conversion circuit 4 or 6. The vertical direction conversion circuits 4 and 6 have the same configuration except that the vertical direction conversion coefficients used are different. Therefore, the detailed configuration will be described in a unified manner with reference to FIG.

The CIF image data D1 input at a predetermined timing is taken into the frame memory 10. The frame memory 10 operates under the control of the memory controller 11, stores the CIF image data D1 for one frame, and outputs the stored image data D1 in the order of raster scanning.

Each of the four line memories 12A to 12D is capable of accumulating image data D1 for one horizontal line (360 pixels), and is controlled by the memory controller 11 to sequentially store the image data D1. , Is output at a predetermined timing. In the vertical direction conversion circuits 4 and 6, the four line memories 12A to 12D are connected in series, and the image data D1 output from the frame memory 10 is input to the first line memory 12A, whereby continuous operation is performed. The image data D1 at the same position in the horizontal direction of the four lines are output from the line memories 12A to 12D in parallel.

The four multiplication circuits 13A to 13D respectively output the image data output from the corresponding line memories 12A to 12D and the corresponding transform coefficient h (x, 3) output from the vertical transform coefficient memory 15, respectively. h (x, 2), h (x, 1), h (x, 0)
Are multiplied and output to an addition circuit (summation circuit) 14, and the addition circuit 14 adds the image data output from these multiplication circuits 13A to 13D to output image data D4 and D5.
(G2) is formed.

Under the control of the memory control unit 11, the vertical conversion coefficient memory 15 outputs the vertical conversion coefficient h (x, x, which is output from the memory 15 in accordance with the line position output by the intermediate image G2. 3), h (x, 2), h (x, 1), h (x, 0) are switched. In addition, the line memory 12A-
The 12D writing operation and the output operation of the frame memory 10 are controlled to be stopped at a predetermined timing. Vertical transform coefficients h (x, 3), h (x, 2), h (x,
1), h (x, 0) switching, memory control, 2 vertical
Vertical direction 4 from the input CIF image D1 (G1) of 88 pixels
Good conversion to intermediate images D4, D5 (G2) of 80 pixels is performed.

Vertical transformation coefficients h (x, 3), h (x, 2), h (x, 1),
The switching of h (x, 0) and the like will be described in more detail. The CIF image D1 (G which is an input image of the vertical direction conversion circuits 4 and 6
In 1), one frame image is formed with 288 vertical pixels and 360 horizontal pixels, and the intermediate images D4 and D5 that are output images are formed.
Since (G2) forms one frame image with 480 pixels vertically and 360 pixels horizontally, the number of pixels in the vertical direction is 3: 5.

FIG. 4 shows the CIF image D1 (G1) and the intermediate images D4 and D5 as to the positional relationship between pixels which are continuous in the vertical direction.
(G2) is indicated by a circle. From the vertical positional relationship shown in FIG. 4, the intermediate images D4, D5 (G
2) The pixel data at a certain vertical position is the CIF image D1.
It can be understood that a plurality of pixel data in the vicinity of the vertical position of (G1) may be formed by interpolation, and in the first embodiment, from the pixel data of four CIF images D1 (G1), the intermediate data is obtained. It is decided to form some pixel data of the images D4 and D5 (G2).

Here, the pixel data of the CIF image D1 and the pixel data of the intermediate images D4 and D5 are respectively s (m, n) and t (m, n) (m and n are in the vertical and horizontal directions, respectively). Direction position), image data D4 and D5 of an intermediate image in which the number of pixels in the vertical direction is equal to the NTSC image data are generated by the interpolation calculation formula shown in the formula (1).

[0026]

[Equation 1] Here, k = 0, 1, ..., 95, n = 0, 1, ..., 3
59, and if the value of m of the pixel data s (m, n) is negative or larger than the value 287, s (m, n) = 0.

That is, in the vertical direction conversion circuits 4 and 6, the CIF image data D1 stored in the frame memory 10 is sequentially transferred to the line memories 12A to 12D, and the pixel data of four consecutive lines is written into the line memories 12A to 1D.
After being held in 2D, the output pixel data from the line memories 12A to 12D are sequentially given to the multiplication circuits 13A to 13D to be multiplied by the vertical conversion coefficient read synchronously from the vertical conversion coefficient memory 15, Thus, the intermediate image data D4 and D5 are generated.

Here, when the pixel data t (5k + 0, n) represented by the first equation in the above equation (1) is generated, the memory control section 11 reads the frame memory 10. The operation mode is set, and the line memories 12A to 12D are set to the readable and writable operation mode. This allows
From the line memories 12A to 12D, pixel data s (3k + 1, n), s (3k +) of four consecutive lines at the same position n in the horizontal direction are stored.
0, n), s (3k-1, n), and s (3k-2, n) are output. At this time, the corresponding conversion coefficient h (0,3) output from the vertical conversion coefficient memory 15 is output. , H (0,2), h (0,1), h (0,0) are multiplied to obtain pixel data t (5k + 0, n). Line memory 1
From 2A to 12D, pixel data s (3k + 1, n), s (3k + 0, n), s (3k-1, n), s (3k-2, n) differ only in the horizontal position n.
Are sequentially output, and the converted pixel data t (5k + 0, n) having different horizontal positions n are sequentially formed and output. As far as the same line 5k + 0 is concerned, the output conversion coefficient h (0, 0
3), h (0,2), h (0,1), h (0,0) are fixed.

Eventually, the maximum value 35 of the lateral parameter n will be 35.
The conversion for 9 is completed, and the pixel data t (5k + of the next line 5k + 1 represented by the second equation in the above equation (1) is expressed.
Proceed to the generation of 1, n). At the time when this new line 5k + 1 is reached, the line memories 12A, 12B, 12
C and 12D have pixel data s (3k + 2, n), s (3k + 1, n),
s (3k + 0, n) and s (3k-1, n) are stored. Pixel data
When generating t (5k + 1, n), the memory control unit 11 sets the line memories 12A to 12D to the readable and write prohibited operation mode. As a result, the line memory 12A
From ~ 12D, pixel data s (3k + 2, n), s (3k + 1, n), s (3k + 0, n), at four consecutive lines at the same position n in the horizontal direction,
s (3k-1, n) is output. At this time, the corresponding conversion coefficient h (1,3), h output from the vertical direction conversion coefficient memory 15 is output.
(1,2), h (1,1), h (1,0) are multiplied and pixel data t (5k + 1,
n) is obtained. From the line memories 12A to 12D,
Pixel data s (3k + 2, n), s (3k
+ 1, n), s (3k + 0, n), s (3k-1, n) are sequentially output, and the horizontal position n is different as the converted pixel data t (5k + 1, n) Are sequentially formed and output. As far as the same line 5k + 1 is concerned, output conversion coefficients h (1,3), h (1,2), h (1,1), h from the vertical direction conversion coefficient memory 15
(1,0) is fixed.

By the way, during the processing of the line 5k + 1, since the line memories 12A to 12D are set to the write-inhibited state, all the pixel data t (5k + 1, n) after conversion can be obtained for the line 5k + 1. Also, the line memories 12A to 12D are respectively set to the pixel data s (3k + 2,
n), s (3k + 1, n), s (3k + 0, n) and s (3k-1, n) are held, and the process proceeds to the conversion of the next line 5k + 2.

Pixel data t (5k for line 5k + 2
+ 2, n) conversion and pixel data for line 5k + 3
The conversion to t (5k + 3, n) is the same as the conversion to the pixel data t (5k + 2, n) for the line 5k + 0 described above.
The description of the operation is omitted. Further, the conversion to the pixel data t (5k + 4, n) for the line 5k + 4 is the same as the conversion to the pixel data t (5k + 1, n) for the line 5k + 1 described above, and therefore the operation description will be given. Omit it.

As described above, each vertical direction conversion circuit 4,
6 is the CIF image data D1 which is the image data D of the intermediate image.
4 and D5.

Here, the vertical conversion coefficient memory 15 of the vertical conversion circuit 4 has vertical conversion coefficients h (x, 0) to h (x) shown in equation (2).
(x, 3) (x is 0 to 3) is output, and the vertical direction conversion coefficient memory 15 of the vertical direction conversion circuit 6 outputs the vertical direction conversion coefficient shown in the equation (3).
h (x, 0) to h (x, 3) (x is 0 to 3) are output, and thereby two types of intermediate image data D4 and D5 having different spatial resolutions in the vertical direction are generated.

[0034]

[Equation 2]

[Equation 3] FIG. 5 shows a detailed configuration example of the above-described horizontal direction conversion circuit 5 or 7 provided at the stage subsequent to such a vertical direction conversion circuit 4 or 6. The horizontal direction conversion circuits 5 and 7 have the same configuration except that the horizontal direction conversion coefficients used are different. Therefore, the detailed configuration will be described in a unified manner with reference to FIG.

In FIG. 5, each of the horizontal direction conversion circuits 5 and 7 is composed of four 1-pixel memories 1 connected in series.
Image data D of an intermediate image output from the corresponding vertical direction conversion circuit 4 or 6 described above, which includes 6A to 16D.
4 and D5 are captured in the first 1-pixel memory 16A. Each of the 1-pixel memories 16A, ..., 16D has 1
The pixel data D4 or D5 of the pixel is formed so as to be accumulated, and operates under the control of the memory control unit 17. That is, image data D4 and D5 of four pixels continuous in the horizontal direction on the same line (same vertical position) are output in parallel from the four 1-pixel memories 16A to 16D.

, 18D are provided corresponding to the respective one-pixel memories 16A, ..., 16D, and the respective multiplication circuits 18A, ..., 18D are respectively associated with the corresponding one-pixel memories 16A ,. The pixel data output from the horizontal conversion coefficient memory 19 and the horizontal conversion coefficients v (x, 0) to v (x, 3) output from the horizontal conversion coefficient memory 19 are multiplied and output to the adder circuit (sum circuit) 20. To do. The adding circuit 20 adds the output data of the multiplying circuits 18A to 18D and outputs the added data.

Here, the horizontal direction conversion coefficient memory 19 is controlled by the memory control unit 17 and outputs the horizontal direction conversion coefficient v (x, 3) from the memory 19 according to the horizontal position of the output image G3. ), V (x, 2), v (x, 1), v (x, 0). The write operation of the 1-pixel memories 16A to 16D is controlled to be stopped at a predetermined timing. Such a lateral transformation coefficient v (x, 3), v
Image data D4, D5 (G2) of 360 pixels in the horizontal direction is obtained by switching (x, 2), v (x, 1), v (x, 0) and memory control.
From the image data D2, D3 (G
A good conversion to 3) is performed.

Lateral transform coefficients v (x, 3), v (x, 2), v (x, 1),
The switching of v (x, 0) and the like will be described in more detail. Intermediate images D4 and D5 which are input images of the horizontal direction conversion circuits 5 and 7.
Since (G2) forms one frame image with 480 vertical pixels and 360 horizontal pixels, the output images D2 and D3 (G3) form one frame image with 480 vertical pixels and 720 horizontal pixels. The number of pixels in the horizontal direction is 1: 2.

FIG. 6 shows the positional relationship between the pixels which are continuous in the horizontal direction, showing the intermediate images D4 and D5 (G2) and the output image D2.
The circles indicate D3 (G3). From the positional relationship in the horizontal direction shown in FIG. 6, the output images D2 and D3 are
The pixel data at the horizontal position with (G3) is the intermediate image D.
4 and D5 (G2), it is understood that it may be formed by interpolating a plurality of pixel data in the vicinity of the lateral position thereof. In the second embodiment, 4 of the intermediate images D4 and D5 (G2) are formed.
Pixel data of the output images D2 and D3 (G3) is formed from the individual pixel data.

Here, the pixel data of the intermediate images D4 and D5 and the pixel data of the output images D2 and D3 are respectively t (m,
n) and u (m, n) (where m and n are the vertical and horizontal positions, respectively), the interpolation calculation formula shown in Eq. NTSC image data D2 and D3 having the number of pixels in accordance with the NTSC format are generated.

[0041]

[Equation 4] Here, k = 0,1, ..., 359, p = 0,1, ...
, 479, and when the value of n of the pixel data t (m, n) is negative or larger than the value 359, t (m, n) = 0.

That is, in the horizontal direction conversion circuits 5 and 7, the pixel data of the intermediate images D4 and D5 are sequentially transferred to the 1-pixel memory groups 16A to 16D, and the pixel data of continuous 4 pixels is thereby transferred to the 1-pixel memory 16A. .. to 16D, the pixel data of these 1-pixel memories 16A to 16D are output in parallel to the multiplication circuits 18A to 18D and the horizontal direction conversion coefficients v (x, 3), v (x, 2), v ( x, 1), v (x, 0) and N
The TSC image data D2 and D3 are generated.

Here, when generating the pixel data u (p, 2k + 0) of the NTSC image represented by the first equation in the above equation (4), the memory control section 17 does not 1 pixel memory 16
A to 16D are set to the write-inhibited operation mode, whereby the pixel data t (p, k + 1), t of four consecutive pixels stored in the 1-pixel memories 16A, ...
(p, k + 0), t (p, k-1), t (p, k-2) are held as they are, and these pixel data t (p, k + 1), t (p, k + 0) , T (p, k-1), t (p, k-
2) and the corresponding lateral transformation coefficients v (0,3), v (0,2), v (0,
1), the product sum of v (0,0) is obtained by the multiplication circuits 18A to 18D to generate the pixel data u (p, 2k + 0) of the NTSC image.

Pixel data u (p, at the next horizontal position 2k + 1)
2k + 1), the memory control unit 17 sets all the 1-pixel memories 16A to 16D in the write operation mode, and in this state, the 1-pixel memories 16A, ...
Pixel data t (p, k + 1), t (p, k + 0), t (p, k
k-1), t (p, k-2) and the corresponding lateral transformation coefficients v (1,3), v (1,2), v (1,1), v (1,0) at this timing. ) Is calculated by the multiplication circuits 18A to 18D to generate the pixel data u (p, 2k + 1) of the NTSC image.

As described above, all 1-pixel memories 16A
During the cycle in which the stored data of ~ 16D is updated, two sets of horizontal direction conversion coefficients (v (0,3), v (0,2), v (0,1), v (0,0)) are set. v
(1,3), v (1,2), v (1,1), v (1,0)), the image data D2 having twice the number of pixels in the horizontal direction. , D3.

Here, the horizontal conversion coefficient memory 19 of the horizontal conversion circuit 5 has horizontal conversion coefficients v (x, 0) to v shown in the equation (5).
(x, 3) (x is 0 to 3) is output, and the horizontal direction conversion coefficient memory 19 of the horizontal direction conversion circuit 7 outputs the horizontal direction conversion coefficient shown in the equation (6).
v (x, 0) to v (x, 3) (x is 0 to 3) is output, and thus output image data (NTSC image data) D2 having different spatial resolutions not only in the vertical direction but also in the horizontal direction. And D3
Is generated.

[0047]

[Equation 5]

[Equation 6] The NTSC image data D2 and D3 output from the two image format conversion circuits 2 and 3 including the vertical direction conversion circuit and the horizontal direction conversion circuit as described above are difference absolute value calculation circuit (high frequency component detection means) 22 and It is given to the adaptive arithmetic circuit (image adaptive synthesizing means) 23.

[0048]

[Equation 7] The difference absolute value calculation circuit 22 executes the arithmetic processing of the equation (7) between the NTSC image data D2 and D3,
The difference absolute value d of the pixel data at the corresponding position between the NTSC image data D2 and D3 is detected and output. When the difference absolute value d is detected between the image data D2 and D3 having different spatial resolutions, the magnitude of the absolute value d changes in accordance with the magnitude of the high frequency component of the CIF image data D1. As a result, the difference absolute value calculation circuit 22 easily detects the magnitude of the high frequency component included in the original image data D1 based on the two systems of image data D2 and D3 having different spatial resolutions. The absolute difference value d is given to the adaptive calculation circuit 23 as control information for the adaptive calculation.

The adaptive calculation circuit 23 inputs the image data D2 and D3 of two systems having different spatial resolutions to the selection circuit 24, and switches the contact point of the selection circuit 24 according to the difference absolute value d. Thereby, the adaptive arithmetic circuit 23 selectively outputs the image data D2 or D3 in pixel units according to the magnitude of the high frequency component included in the original image data D1,
The NTSC image data D6 in which the image distortion due to the encoding such as the block distortion is reduced and which conforms to the NTSC format is obtained and output.

Here, in the transmission configuration for the image data transmission apparatus having the image processing circuit of the first embodiment, C is used.
The transmission signal is generated while compressing the IF image data in block units. Therefore, block distortion or the like occurs in the CIF image data D1 decoded on the receiving side. Such distortion is a high frequency component, and
It occurs not on the entire area of the F image data D1, but on the block boundary or the like.

In the case of a portion of the CIF image data D1 that originally changes gently (a portion that does not include a high-frequency component; a distortion component if there is a high-frequency component), the NTSC image data D2 with the higher spatial resolution is used. Even if there is, since there is no high frequency component (because the value difference from the frequency pixel is small and it is not related to the weight), the value hardly changes, and the difference between the two NTSC image data D2 and D3. The absolute value d takes a small value. When such a difference absolute value d becomes small, it is well reflected that the image data D3 having a low spatial resolution changes gently,
That is, unnecessary high frequency components are removed, which is preferable.

On the other hand, in the case of the originally fine image portion (the portion including the high frequency component) of the CIF image data D1, since the high frequency component is weakened in the NTSC image data D3 having the lower spatial resolution, two The absolute difference value d between the NTSC image data D2 and D3 is a large value. In such a portion having a large number of high frequency components, it is difficult to perceive block distortion, which is a high frequency component itself, and if the NTSC image data D3 in which a component having a high spatial resolution is suppressed is selected, the image quality is rather deteriorated. Therefore,
In this case, the NTSC image data D2 is preferable.

That is, in the adaptive arithmetic circuit 23, when the difference absolute value d is equal to or larger than the predetermined threshold value V, the image data D2 having a high spatial resolution is selected and output, and the difference absolute value d
Is smaller than a predetermined threshold value V, the image data D3 having a low spatial resolution is selectively output so that a portion having a large change in the original image is not reduced in the spatial resolution, and a portion having a small change in the original image is a space. Block distortion and the like are removed by lowering the resolution.

When the characteristics of the image format conversion circuits 2 and 3 are set as shown in the above equations (1) to (6) and the image data is represented by 256 gradations, the threshold value V is set to 5. When set, the image distortion associated with encoding can be effectively reduced.

The configuration and operation of each unit of the first embodiment have been described above. The operation of the entire image processing circuit 1 of the first embodiment will be described below.

In the above configuration, the decoded CIF
The image data D1 is input to the image format conversion circuits 2 and 3, and is converted into NTSC image data D2 and D3 having different spatial resolutions. These NTSC image data D2 and D3 are stored in the difference absolute value calculation circuit 22 as follows.
The difference absolute value d is converted in pixel units, and the original C
The magnitude of the high frequency component included in the IF image data is detected. The NTSC image data D2 and D3 are selectively output from the adaptive operation circuit 23 based on the absolute difference value d. That is, when the magnitude of the high frequency component is equal to or larger than the predetermined value, the image data D2 having high spatial resolution is selectively output,
When the magnitude of the high-frequency component is smaller than the predetermined value, the image data D3 having a low spatial resolution is selectively output, whereby the image data D6 effectively avoiding the image distortion due to the encoding can be obtained.

According to the above configuration, the CIF image data D
1 for two systems of NTSC image data D with different spatial resolutions
After converting to 2 and D3, by selectively outputting according to the magnitude of the high frequency component included in the original image data, the post-processing filter can be omitted and the image distortion due to encoding can be reduced. Therefore, the image distortion associated with the encoding can be reduced with a simple configuration as a whole. Further, since the post-processing filter can be omitted, the time required for converting the image data can be shortened accordingly.

The detection of the high-frequency component, which is the premise for obtaining such an effect, also applies to the two systems of NTSC image data D2 and D3.
Since this is a method of detecting the absolute difference value of, the overall configuration can be simplified also from this point.

In particular, when the apparatus to which the first embodiment is applied is a video telephone apparatus or the like, an encoding method is recommended, and if it is followed, it is inevitable that image distortion will occur. One embodiment works well.

(2) Second Embodiment Next, a second embodiment of the present invention will be described. here,
FIG. 7 shows the configuration of the image processing circuit 31 of the second embodiment, and the parts corresponding to those in FIG. 1 are designated by the same reference numerals.

In FIG. 7, the second embodiment is different from the first embodiment in that the vertical conversion circuit 32 is used to generate two systems of NTSC image data D2 and D3 having different spatial resolutions. In the image processing circuit 31 of the second embodiment, the CIF is used.
Image data D1 is input to the vertical direction conversion circuit 32, and image data (image data (image data) in which image data D4 having high spatial resolution and image data D5 having low spatial resolution alternately appear in pixel units by the vertical direction conversion circuit 32. After generating image data D7 in which D4 and D5 are time-division multiplexed for each pixel, this image data D7 is converted into image data D4 having a high spatial resolution and image data D5 having a low spatial resolution through the selection circuit 33. And outputs the separated image data D4 and D5 to the corresponding horizontal direction conversion circuits 5 and 7, respectively.

FIG. 8 shows the detailed structure of the vertical direction conversion circuit 32 of the second embodiment for performing the above-mentioned processing. The same or corresponding portions as those in FIG. 3 described above are designated by the same reference numerals. Is shown.

In FIG. 8, the vertical direction conversion circuit 32 transfers the CIF image data D1 fetched in the frame memory 10 to the line memories 12A to 12B, whereby pixel data at the same position in the horizontal direction of four consecutive lines is transferred. The line memories 12A to 12B can output in parallel. The memory control unit 34 uses the line memory 1 at a speed twice that of the first embodiment.
2A to 12B are controlled, and one horizontal position of the intended line position of the intermediate image (D4 and D5) can be converted twice in time division so that the same horizontal position of the same line can be executed. Basically, the reading is controlled so that the image data D1 at the directional position is continuously output twice each.

The vertical direction conversion coefficient memory 35 uses the equation (2) and
The vertical conversion coefficients ha (x, 0) ~ ha (x, 3) and hb (x, 0) ~ hb (x, 3) of the first and second sets respectively corresponding to the equation (3) are stored. Under the control of the memory control unit 34, the vertical conversion coefficients ha (x, ha of the first set and the second set are synchronized with the image data output from the line memories 12A to 12D.
0) ~ ha (x, 3) and hb (x, 0) ~ hb (x, 3) are output alternately.

The vertical conversion circuit 32 includes vertical conversion coefficients ha (x, 0) to ha (x, 3) or hb (x, 0) to hb of the first set or the second set.
(x, 3) and the four pixel data s (A + 0, n), s (A +) output in parallel from the line memories 12A to 12D at that time.
1, n), s (A + 2, n), and s (A + 3, n), the calculation shown in the equations (8) and (9) is executed, and the pixel of the intermediate image D4 having a high spatial resolution The image data D7 that alternately includes the data ta (B, n) and the pixel data tb (B, n) of the intermediate image D5 having a low spatial resolution is generated and output (note that the relationship between the values of the parameters A and B). For formula, see Eq. (1).

[0066]

[Equation 8]

[Equation 9] As described above, the image data D7 is separated by the selection circuit 33 into the image data D4 of the intermediate image having a high spatial resolution and the image data D5 of the intermediate image having a low spatial resolution, and then the corresponding horizontal direction conversion circuit 5 and Given to 7.

The processing configuration and operation after the horizontal direction conversion circuits 5 and 7 are the same as those in the first embodiment, and the description thereof will be omitted.

Therefore, according to the second embodiment as well, it is possible to achieve the same effect as that of the first embodiment in that the image quality can be improved simultaneously with the format conversion, and the vertical conversion circuit of the two image format conversion circuits can be obtained. Since it is shared, the entire configuration can be simplified accordingly.

(3) Third Embodiment Next, a third embodiment of the present invention will be described. here,
FIG. 9 shows the configuration of the image processing circuit 41 of the third embodiment, and the parts corresponding to those of FIGS. 1 and 7 are designated by the same reference numerals.

As shown in FIG. 9, in the third embodiment, not only the vertical direction conversion circuit 32 but also the horizontal direction conversion circuit 42 is
The intermediate image D4 having a high spatial resolution and the intermediate image D5 having a low spatial resolution are commonly used to further simplify the entire configuration.

That is, in the image processing circuit 41, the output data (data in which the image data D4 and D5 are time-division multiplexed in pixel units) D7 from the vertical direction conversion circuit 32 is given to the horizontal direction conversion circuit 42. After the horizontal direction conversion circuit 42 generates the image data D8 in which the image data D2 having a high spatial resolution and the image data D3 having a low spatial resolution alternately appear every two pixels in the horizontal direction,
The image data D8 is separated into the image data D2 having a high spatial resolution and the image data D3 having a low spatial resolution through the selection circuit 33 and output to the absolute difference value calculation circuit 22 and the adaptive calculation circuit 23.

FIG. 10 shows a detailed structure of the horizontal direction conversion circuit 42 in the third embodiment which performs the above operation. The same or corresponding portions as those in FIG. 5 described above are designated by the same reference numerals. It is attached.

In FIG. 10, this horizontal direction conversion circuit 42
In FIG. 2, two-pixel memories 43A to 43D are arranged in place of the one-pixel memories 16A to 16D, and the memory control unit 44 controls writing and reading operations of these two-pixel memories 43A to 43D. . The memory control unit 44 outputs the image data D7 output from the vertical direction conversion circuit 32, which includes the image data D4 having a high spatial resolution and the image data D5 having a low spatial resolution, for each pixel to 2 pixels.
The pixel data is stored in the pixel memories 43A to 43D one by one,
Therefore, from the pixel memories 43A to 43D, the image data D
For 4 or D5, four pixel data continuous in the horizontal direction can be output in parallel.

The horizontal direction conversion coefficient memory 45 uses the equation (5) and
It has horizontal conversion coefficients va (x, 0) ~ va (x, 3) and va (x, 0) ~ vb (x, 3) of the first and second sets respectively corresponding to the equation (6). , The conversion coefficients va of the first set and the second set are synchronized with the image data output from the two-pixel memories 43A to 43D.
(x, 0) ~ va (x, 3) and vb (x, 0) ~ vb (x, 3) are output alternately.

The horizontal direction conversion circuit 42 includes the first or second set of horizontal direction conversion coefficients va (x, 0) to va (x, 3) or vb (x, 0) to vb.
(x, 3) and four pixel data ta (p, C + 0), ta output from the two pixel memories 43A to 43D in parallel at that time.
(p, C + 1), ta (p, C + 2) and ta (p, C + 3), or tb (p, C +
0), tb (p, C + 1), tb (p, C + 2) and tb (p, C + 3),
By performing the calculations shown in the equations (10) and (11), the pixel data ua (p, D) of the NTSC image D2 having a high spatial resolution and the pixel data ub (p, D) of the NTSC image D3 having a low spatial resolution. Image data D8 that alternately includes and is generated and output (for the relationship between the values of the parameters C and D, see equation (4)).

[0076]

[Equation 10]

[Equation 11] As described above, after the image data D8 is separated into the NTSC image data D2 having a high spatial resolution and the NTSC image data D3 having a low spatial resolution by the selection circuit 33,
It is given to the absolute difference calculation circuit 22 and the adaptive calculation circuit 23. Difference absolute value calculation circuit 22 and adaptive calculation circuit 23
Is the same as that of the first embodiment, and the description thereof is omitted.

Therefore, according to the third embodiment as well, it is possible to achieve the same effect as the first embodiment in that the image quality can be improved at the same time as the format conversion, and the vertical direction conversion circuit of the two image format conversion circuits can be obtained. Also, since the horizontal direction conversion circuit is shared, the entire structure can be simplified accordingly.

(4) Fourth Embodiment Next, a fourth embodiment of the present invention will be described. This 4th
The embodiment is different from the above-described image processing circuits of the first to third embodiments in the configuration of the adaptive arithmetic circuit (image adaptive synthesizing means). Hereinafter, the fourth embodiment will be described focusing on this difference. To do.

In the fourth embodiment, NTSC image data D2 and D3 of two systems having different spatial resolutions are weighted and added so that a portion having a large change in the original image does not lower the spatial resolution. NT where the spatial resolution is reduced in the part where the change is small in the original image
The SC image data D9 is obtained and output.

FIG. 11 shows a fourth embodiment for realizing such a function.
3 illustrates a detailed configuration of an adaptive calculation circuit 53 according to an embodiment.

In FIG. 11, in the adaptive calculation circuit 53, the difference absolute value d obtained by the difference absolute value calculation circuit 22 is given to the coefficient calculation circuit 54, and the coefficient calculation circuit 54
Is a weighting coefficient w (where w is 1 to
0) is calculated and given to the coefficient conversion circuit (1-x) 55 and the multiplication circuit 56. The coefficient conversion circuit 55 finds another weighting coefficient 1-w and gives it to the multiplication circuit 57. Each multiplication circuit 5
Image data D2 or D3 corresponding to 6 and 57 respectively
Is input, and each of the multiplication circuits 56 and 57 receives the coefficient w or the coefficient 1 given to the corresponding image data D2 or D3.
Multiply by -w for weighting. These values are added by the adder circuit 5
8 is added and added, and is output as final NTSC image data D9.

Therefore, the image synthesizing process in the adaptive operation circuit 53 is expressed by the equation (12).

[0083]

[Equation 12] Here, the conversion curve of the absolute difference value d and the weighting coefficient w (w is 1 to 0) by the coefficient calculation circuit 54 may be arbitrarily selected as long as the image quality improving effect can be exhibited.
The conversion curve shown in 2 can be mentioned. In this conversion curve, a coefficient w that rises from a value 0 to a value 1 is generated according to the value of the absolute difference value d on the basis of a predetermined threshold value V (therefore, it is equivalent to the adaptive operation circuit of the first embodiment). .

Accordingly, in the fourth embodiment,
The same NTSC image data as in the first embodiment is output.

According to the configuration shown in FIG. 11, the absolute difference value d
Even if the two image data D2 and D3 having different spatial resolutions are weighted and added according to the above, the same effect as that of the first embodiment can be obtained.

Further, in this case, the characteristics of the coefficient calculation circuit 54 can be switched to freely switch the characteristics of the output image as desired, and the visual characteristics of the received image can be further improved.

For example, as shown in FIG. 13, the characteristic of the coefficient calculation circuit 54 may be set so that the weighting coefficient w changes in proportion to the absolute difference d up to a predetermined threshold and then becomes a constant value. Also, as shown in FIG. 14, after the value 0 is held up to a predetermined threshold value, the weighting coefficient w changes in proportion to the absolute difference value d, and then the characteristic of the coefficient calculation circuit 54 is set so as to become a constant value. Alternatively, as shown in FIG. 15, the characteristic of the coefficient calculation circuit 54 may be set so that the weighting coefficient w changes smoothly until the value changes from 0 to 1.

(5) Other Embodiments In the above embodiments, the case where the horizontal direction conversion circuit generates the NTSC image data after the vertical direction conversion circuit generates the intermediate image has been described. Not limited to this, it can be widely applied to the case where the horizontal direction conversion circuit generates the intermediate image and then the vertical direction conversion circuit generates the NTSC image data.

Further, in the above embodiment, the case where the interpolation processing is performed so that the number of pixels is increased with respect to the CIF image data is described in consideration of the application of the present invention to the video telephone apparatus, the video conference apparatus and the like. However, the present invention is not limited to this, and can be widely applied to the case where the format of the image data is converted so that the number of pixels is decreased, and the invention is not limited to the case where the NTSC image data is generated. It can be widely applied to the case of generating image data of the PAL system or the like.

That is, the present invention can be widely applied to an image processing apparatus having an image format conversion processing structure, and can be applied not only for transmission but also for recording and reproduction.

Further, in the above embodiment, NTSC
The case where the magnitude of the high-frequency component is detected in the original image by detecting the difference absolute value of the image data has been described, but the present invention is not limited to this and, for example, a method such as discrete cosine transform is applied to the image. When transmitting data, the magnitude of the high frequency component may be detected with reference to the frequency information obtained from the transmitted image data.

Furthermore, in the above embodiment, the case where two image format conversion circuits are provided and two types of image data having different spatial resolutions are obtained and combined is shown.
It is also possible to obtain three or more types of image data having different spatial resolutions by three or more image format conversion circuits and combine them according to the magnitude of the high frequency component (including simple selection).

[0093]

As described above, according to the present invention, the format of the input image is converted into the output images of two or more systems having different spatial resolutions, and the output image of the plurality of systems is converted into the magnitude of the high frequency component of the input image. Since the composite image whose spatial resolution changes correspondingly is formed, the image quality can be improved at the same time as the format conversion.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a first embodiment.

FIG. 2 is an explanatory diagram of image format conversion according to the first embodiment.

FIG. 3 is a block diagram showing a configuration of a vertical direction conversion circuit of the first embodiment.

FIG. 4 is a diagram used for explaining the operation.

FIG. 5 is a block diagram showing a configuration of a horizontal direction conversion circuit of the first embodiment.

FIG. 6 is a diagram used for explaining the operation.

FIG. 7 is a block diagram showing an overall configuration of a second embodiment.

FIG. 8 is a block diagram showing a configuration of a vertical direction conversion circuit according to a second embodiment.

FIG. 9 is a block diagram showing an overall configuration of a third embodiment.

FIG. 10 is a block diagram showing a configuration of a horizontal direction conversion circuit according to a third embodiment.

FIG. 11 is a block diagram showing a configuration of an adaptive arithmetic circuit according to a fourth embodiment.

FIG. 12 is a characteristic curve diagram (No. 1) for explaining the operation.

FIG. 13 is a characteristic curve diagram (No. 2) used for explaining the operation.

FIG. 14 is a characteristic curve diagram (No. 3) for explaining the operation.

FIG. 15 is a characteristic curve diagram (No. 4) used for explaining the operation.

[Explanation of symbols]

1, 31, 41 ... Image processing circuit, 2, 3 ... Image format conversion circuit, 4, 6, 32 ... Vertical direction conversion circuit, 5, 7, 42 ... Horizontal direction conversion circuit, 22 ... Difference absolute value calculation circuit (high frequency Component detection means), 23, 53 ... Adaptive arithmetic circuit (image adaptive synthesis means), 24, 33 ... Selection circuit.

Claims (8)

[Claims] 1. A plurality of image format conversion means for converting an input image into an image having a number of pixels different from that of the input image and having the same number of pixels but different spatial resolutions, and a high frequency of the input image. The high-frequency component detecting means for detecting the magnitude of the component and the output image from each of the image format converting means having different spatial resolutions are combined according to the detected high-frequency component size, and the combined image is synthesized. An image processing circuit comprising: an image adaptive synthesizing means for outputting. 2. The first and second image format converting means are provided as the plurality of image format converting means, and the image adaptive synthesizing means outputs the first and second image format converting means from the first and second image format converting means. The image processing circuit according to claim 1, wherein the second output image is combined in correspondence with the magnitude of the detected high frequency component. 3. The high frequency component detecting means detects the magnitude of the high frequency component of the input image by detecting the absolute difference value of the pixel data at the same position in the first and second output images. The image processing circuit according to claim 2. 4. The image adaptive synthesizing means selectively outputs the first or second output image for each pixel according to whether or not the magnitude of the detected high frequency component is equal to or larger than a predetermined threshold value. The image processing circuit according to claim 2 or 3, wherein a composite image whose spatial resolution changes in accordance with the magnitude of the high frequency component is formed. 5. The image adaptive synthesizing means weights and adds the first and second output images pixel by pixel based on the detected magnitude of the high frequency component to obtain the magnitude of the high frequency component. The image processing circuit according to claim 2 or 3, wherein a composite image corresponding to which the spatial resolution changes is formed. 6. An intermediate image forming unit for converting the number of pixels of the input image in the horizontal direction or the vertical direction of the input image to form an intermediate image by the first and second image format converting means, respectively. An output image forming unit for converting the number of pixels of the intermediate image in the vertical direction or the horizontal direction of the input image and converting the intermediate image into the first or second output image. The image processing circuit according to any one of 2 to 5. 7. An intermediate image forming unit is shared as an intermediate image forming unit of each of the first and second image format converting means, and the first or second image format converting unit is time-shared. The image processing circuit according to claim 6, wherein the image processing circuit is selectively used as an intermediate image forming unit of the means. 8. One output image forming unit is shared as the output image forming unit of each of the first and second image format converting means, and the first or second image format converting unit is time-shared. The image processing circuit according to claim 6 or 7, wherein the image processing circuit is selectively used as an output image forming unit of the means.