JP3046379B2 - Image coding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus for encoding a moving image in a video conference or a video phone.

[0002]

2. Description of the Related Art In recent years, moving picture coding systems have been actively studied with the standardization approaching. Also, intelligent coding has been proposed as a future coding method, and as one direction of research, a coding method that uses knowledge about an image in some form has been studied. As one of such encoding methods, a method has been proposed in which a face area is detected from a moving image signal of a person image and a large number of quantization bits are allocated to the image signal of the face area.

[0003] As an example, the 1989 image coding symposium PCST89, 7-15 includes an area other than the face area when quantizing DCT coefficients obtained by DCT (Discrete Cosine Transform) coding of a moving image signal. A technology for relatively improving the image quality of a decoded image of a face region by making the quantization step size larger than the quantization step size determined by the buffer amount of the buffer for storing the encoded data is disclosed. ing. According to this technique, in a region other than the face, there is a lot of quantization noise of the DCT coefficient, and thus there is a problem that a visually conspicuous distortion peculiar to DCT coding such as block distortion occurs in a decoded image.

[0004]

As described above, in the conventional moving picture coding technique of allocating a large number of quantization bits to the image signal of the face area in the moving picture signal of the person image, the area other than the face is used. Since a large amount of quantization noise of DCT coefficients is generated, there is a problem that a visually distinctive distortion peculiar to DCT coding such as block distortion occurs in a decoded image.

An object of the invention, provides significant without the occurrence of distortion, picture coding apparatus that can be enhanced quality of the decoded image of a specific region in the region other than the specific area, such as the face area Is to do.

[0006]

SUMMARY OF THE INVENTION The present invention provides an encoding method.
Of the input image signal other than the specified area
Selectively in at least one of the spatial direction and the time axis direction
Low-pass filter means for filtering in
The output image signal from the low-pass filter means.
It is a basic feature to perform coding. That is, in the present invention, for example, a specific area in an image signal sequentially input in frame units is detected, and an area specifying signal for specifying and specifying the specific area and other areas is output.
In accordance with the area designation signal, the image signal of an area other than the specific area in the input image signal is selectively filtered by the low-pass filter means. Then, the output image signal from the low-pass filter means is encoded by an encoding circuit.

According to another aspect of the present invention, an arbitrary specific area is specified by the area specifying device, and the same area specifying signal as described above is output. According to the area designation signal, the image signal of an area other than the specific area is selectively filtered by the low-pass filter means ,
Further, the output image signal from the low-pass filter means is encoded by an encoding circuit.

[0008]

When a high-frequency component is reduced through a low-pass filter before encoding an image signal of a moving image and the resolution is reduced, D
When a moving picture is coded using spatial correlation such as CT coding, the amount of information generated by quantization is reduced.

In the present invention, for example, when an image signal of a human image is targeted, low-pass filter processing is selectively performed only in a region other than the face in the spatial direction, the time axis direction, or in both directions, or The filter characteristics are changed according to the detection result of the area. As a result, the amount of information generated in areas other than the face during quantization is reduced, and the image quality of the face area is equivalently improved.

When the low-pass filter processing is performed in the time axis direction, the resolution is reduced only in the moving part of the area other than the face, but the information generated in the encoding is only from the moving part. , The amount of information generated from areas other than the face is effectively suppressed.

In these cases, image quality degradation in areas other than the face is a decrease in resolution, and unsightly distortion such as block distortion that occurs when the quantization step size is increased without applying a low-pass filter is generated. And subjective image quality is improved. In addition, when low-pass filtering is performed in the time axis direction, the resolution does not decrease in a stationary background part, so that the subjective image quality is not impaired, and the image quality is improved by removing noise instead. I do.

Further, since the present invention is a method of performing a filtering process before encoding, it is only necessary to add a face detection circuit and a low-pass filter to the conventional moving picture encoding apparatus, and a new buffer control method and the like are required. No ingenuity is required.

[0013]

Embodiments of the present invention will be described below with reference to the drawings. For the sake of simplicity, in the following embodiments, an example will be described in which a P × 64 kbps video standard coding method (see CCITT recommendation H.261) is used as a coding method. Of course, the present invention can use other encoding schemes.

In the embodiment shown in FIG. 1, a terminal 100 to which an image signal of a moving image is input is connected to a write terminal of a frame memory 101 for storing one frame of image signal (referred to as frame data). A read terminal of the frame memory 101 includes an input terminal of a face area detection circuit 102 for detecting a face area of a human image in frame data, and filtering of low-pass characteristics in a spatial direction and / or a time axis direction. To the input terminal of the low-pass filter 103 that performs An output terminal of the face area detection circuit 102 is connected to a control terminal of the low-pass filter 103.

The configuration after the low-pass filter 103 is the same as that of a normal image encoding device using the standard encoding method. That is, the output of the low-pass filter 103 is coupled to the subtractor 104, the inter-frame / intra-frame determination circuit 105, the switch 106, and the motion compensation inter-frame prediction circuit 113. A loop filter 114 is connected to an output terminal of the motion compensation inter-frame prediction circuit 113.

The motion-compensated inter-frame prediction circuit 113 includes a frame memory, a motion vector detection circuit, and a variable delay circuit, and creates an inter-frame prediction signal motion-compensated for each macroblock. The inter-frame prediction signal is sent to the subtractor 104 through the loop filter 114, and the subtracter 104 calculates a difference from the macroblock data from the low-pass filter 103.

The inter-frame / intra-frame decision circuit 105 compares the signal power of the difference signal output from the subtractor 104 with the signal power of the macroblock data output from the low-pass filter 103, and performs prediction processing on the frame. It is determined whether to perform inter-frame prediction or intra-frame prediction. Switch 10
6 selects the output signal of the subtractor 104 which is an inter-frame prediction signal or the output signal of the low-pass filter 103 which is an intra-frame prediction signal in response to the judgment result of the inter / intra judgment circuit 105 I do.

The output terminal of the switch 106 is DCT (D
The output terminal is connected to an input terminal of an island cosine transform circuit 107. The output terminal of the DCT circuit 107 is connected to the quantization circuit 108 and the valid / invalid determination circuit 109. The DCT circuit 107 generates DCT coefficient data as an output. Quantization circuit 108
Quantizes each DCT coefficient data and outputs, for example, a quantization table number and a quantization index as a quantization result. The valid / invalid determination circuit 109 calculates signal power for each block from the DCT coefficient data, and determines whether the block is a valid block or an invalid block.

An output terminal of the quantization circuit 108 is connected to an input terminal of an inverse quantization circuit 110 which performs a process reverse to that of the quantization circuit 108. The output terminal of the inverse quantization circuit 110 is D
It is connected to the input terminal of the inverse DCT circuit 111 that performs the reverse process of the CT circuit 107. An output terminal of the inverse DCT circuit 111 is connected to one input terminal of the adder 112. The output terminal of the switch 115 is connected to the other input terminal of the adder 112. The output terminal of the loop filter 114 is connected to the input terminal of the switch 115. Adder 1
12 adds the inter-frame prediction signal from the loop filter 114 input via the switch 115 and the output signal from the inverse DCT circuit 111 to generate a local decoded signal. An output terminal of the adder 112 is connected to an input terminal of the motion compensation inter-frame prediction circuit 113.

The multiplexing circuit 116 includes a decision signal from the inter-frame / intra-frame decision circuit 105, an output signal from the quantization circuit 108, a valid / invalid decision signal from the valid / invalid decision circuit 109, and a motion compensation inter-frame prediction circuit. The motion vector information from 113 and the filter ON / OFF signal from loop filter 114 are multiplexed. An output terminal of the multiplexing circuit 111 is connected to an input terminal of the variable length coding circuit 117. An output terminal of the variable length coding circuit 117 is connected to a writing terminal of a buffer memory 118 for matching the amount of information generated by the coding device with the transmission rate of the line. The reading terminal of the buffer memory 118 is connected to a transmission line. The buffer memory 118 has a function of generating information indicating the buffer amount, and the buffer amount information is supplied to the encoding control circuit 119. Encoding control circuit 1
19 controls the quantization circuit 108 and the valid / invalid determination circuit 109 according to the buffer amount.

Next, the operation of the image coding apparatus according to this embodiment will be described with reference to FIGS.

An image signal of a moving image is sequentially input to the image input terminal 100 in frame units. This image signal is obtained, for example, as follows. An analog image signal obtained by imaging a person with a TV camera (not shown) is converted into a luminance signal Y and two types of color signals (or color difference signals) C.
_B and C _R, and converted into digital data called CIF or QCF by an A / D converter and a format converter.

This image signal is sequentially written as frame data in the frame memory 101 one frame at a time. The frame data written in the frame memory 101 is read out in units of a plurality of blocks or frames, and supplied to the face area detection circuit 103 and the low-pass filter 104. The reading unit of this frame data is D
This is a unit having a size larger than the size of a block (8 pixels × 8 pixels), which is a conversion unit (encoding unit) in the CT circuit 107. The face area detection circuit 102 is provided in the frame memory 1
01, the face area in the current frame data is detected by calculating the difference between the current frame data newly input from the first frame data and the previous frame data stored in the internal frame memory in units of macroblocks. And outputs an area specifying signal for specifying whether each macroblock in the area is a face area or another area. The face area detection circuit 102 updates the contents of the frame memory with the current frame data for each macroblock for which the face area has been detected.

As will be described in detail later, the low-pass filter 103 performs filtering of low-pass characteristics in a spatial direction and / or a time axis direction. Here, first, a case where the low-pass filter 103 performs filtering in the spatial direction will be described. Since filtering in the spatial direction requires a plurality of blocks to be filtered and several pixels around the blocks, the frame memory 10
The contents of 1 are read out in units of the plurality of blocks and several pixels around them.

For example, as shown in FIG. 2A, the unit read from the frame memory 101 is H.264. The luminance signal Y having the same size as that of the macroblock defined in H.261
The color difference signals C _B, when the 24 × 16 pixel consisting C _R, as shown in the 2 (b) ~ (d) drawing, a luminance signal Y and color difference signals C _B, with C _R total, 32 pixels Frame data is input from the frame memory 101 to the low-pass filter 103 in units of an area having a size of × 24 pixels.

When the currently input macroblock is a face area, the low-pass filter 103 performs filtering on the macroblock without performing filtering according to the area designation signal input from the face area detection circuit 102. ,
The data is converted into data of a predetermined macroblock format (macroblock data) and output. If the data is in an area other than the face, the data is filtered and then converted to macroblock data and output.

The output of the low-pass filter 103 is supplied to one input terminal of the subtractor 104 and the inter-frame / intra-frame decision circuit 10.
5 is supplied. The subtractor 104 outputs a difference signal between the output signal from the low-pass filter 103 and the inter-frame prediction signal from the loop filter 114. The inter / intra-frame determination circuit 105 compares the signal power of the difference signal output from the subtractor 104 with the signal power of the output signal from the low-pass filter 103 on a macroblock basis, and the former is smaller. For example, it is determined that inter-frame prediction should be selected, and if the latter is smaller, it is determined that intra-frame prediction should be performed. The switch 106 selects the difference signal from the subtractor 104 when the determination result of the inter-frame / intra-frame determination circuit 105 is inter-frame prediction, and outputs the output of the low-pass filter 103 when the intra-frame prediction is performed. Select a signal.

The signal selected by the switch 106 is
DCT coding is performed by the DCT circuit 107 to generate DCT coefficient data. The DCT coefficient data is input to the quantization circuit 10
Then, for example, a quantization table number and a quantization index are output as a quantization result. The output of the quantization circuit 108 is inversely quantized by the inverse quantization circuit 110 and further inverse DCT by the inverse DCT circuit 111.

The local decoded data obtained by the inverse DCT is stored in a frame memory built in the motion compensation inter-frame prediction circuit 113, and is referred to for encoding the next frame data. That is, the motion compensation inter-frame prediction circuit 113 compares the local decoded data of the previous frame stored in the built-in frame memory with the current frame data from the low-pass filter 103 to generate motion vector information. . Motion compensation inter-frame prediction circuit 1
13 further uses a variable delay circuit to select macroblock data corresponding to the motion vector information from the local decoded data, and supplies this macroblock data to the loop filter 114 as an inter-frame prediction signal. The loop filter 114 is configured by a spatial filter, removes noise included in the inter-frame prediction signal from the motion compensation inter-frame prediction circuit 113, and supplies the noise to the subtractor 104.

The DCT coefficient data is also input to the valid / invalid determination circuit 109. The valid / invalid determination circuit 109
Calculate signal power for each block from DCT coefficient data,
The signal power is compared with a threshold to determine whether the block is a valid block or an invalid block.

The decision signal from the inter-frame / intra-frame decision circuit 105, the quantization table number and quantization index from the quantization circuit 108, the valid / invalid decision signal from the valid / invalid decision circuit 109, and the motion compensation inter-frame prediction. Motion vector information from circuit 113 and loop filter 1
14 is supplied to the multiplexing circuit 1
16 according to a predetermined format. The output of the multiplexing circuit 116 is converted into a variable length code such as a Huffman code by the variable length coding circuit 117. The output of the variable length coding circuit 117 is written to the buffer memory 118 and read at a speed that matches the transmission rate. The read signal is sent to the transmission path.

On the other hand, the buffer amount information generated from the buffer memory 118 is supplied to the encoding control circuit 119. The encoding control circuit 119 estimates the current amount of generated information of the encoding device from the buffer amount information, and controls the quantization circuit 108 and the valid / invalid determination circuit 109 based on the estimation result. Specifically, when the amount of generated information is large, the amount of generated information is reduced by increasing the quantization step size in the quantization circuit 108 or by increasing the threshold for determination in the validity / invalidity determination circuit 109. When the amount of generated information is small, control is performed to increase the amount of generated information by performing the reverse operation. Instead of switching the quantization step size, adaptive dropout may be used.

In the image coding apparatus of this embodiment described above, of the image signals input to the terminal 100,
Macroblocks in areas other than the face are subjected to the low-pass filter 10
By filtering at 3, high frequency components are removed or suppressed. Therefore, the quantization performed by the quantization circuit 108 reduces the amount of information generated from the high-frequency component, and obtains a result equivalent to reducing the number of quantization bits allocated to regions other than the face. Moreover, in this manner, the image quality of the face area can be improved and the subjective image quality can be improved without causing noticeable distortion such as block distortion in an area other than the face.

In the face area detection circuit 102, detection is performed for each macroblock as described above. The filtering of the low-pass filter 103 is performed by the face area detecting circuit 10.
2 is performed on the macro block in the face region and the signals of the pixels around the macro block in accordance with the region designating signal from step 2. Therefore, the detection accuracy of the face area detection circuit 102 may be an accuracy capable of identifying whether the entire target macroblock is a face area or another area. However, when higher detection accuracy is obtained, for example, information indicating the position of the boundary between the face area and the area other than the face is included in the area designation signal sent from the face area detection circuit 102 to the low-pass filter 103. be able to. Thereby, in the low-pass filter 103,
When this boundary crosses the inside of a macroblock, filtering can be performed only on a region other than the face in the macroblock. According to this method, encoding efficiency is further improved, and a boundary between regions having different resolutions does not occur in regions other than the face, so that a more natural image can be obtained.

The image coding apparatus of the present invention basically includes a face area detection circuit 102 and a low-pass filter 103 for selectively filtering an image signal of a face area according to an area designation signal from the face area detection circuit 102. It is only added to a normal image encoding device, and since the circuit configuration and encoding control after encoding may be the same as those of the ordinary image encoding device, the implementation is extremely easy.

The present invention provides a method other than DCT coding as an image coding method, for example, DPCM (Different
ial PCM). In short, the present invention is all effective when using an encoding method utilizing correlation in the spatial direction, represented by these methods.

Next, a specific example of the face area detection method in the face area detection circuit 102 in FIG. 1 will be described with reference to FIG. FIG. 3 shows an inter-frame difference image of the frame data input to the face area detection circuit 102. The information of the inter-frame difference image is binarized by a preset first threshold level, and is converted into “0” or “1” data. Next, in the vertical direction and the horizontal direction of the screen, the number of pixels for which the binarized data is “1”, that is, the number of pixels having a value equal to or greater than the first threshold value is counted.
A histogram of the number of pixels (x-axis and y-axis histograms) is created. The face area is detected based on the histogram.

The detection of the face area is performed from the detection of the crown 30. The crown 30 searches the y-axis histogram from above,
It is detected by selecting a point Ys that first exceeds the second threshold value set in advance. After the crown 30 is detected, the left end 31 and the right end 32 of the head are detected. The detection of the left end 31 and the right end 32 of the head 30 is performed by searching the x-axis histogram from the left and right, and selecting points Xs and Xe exceeding the second threshold for the first time. In this case, in order to prevent an area having the width of the shoulder 34 from being detected, a y-axis histogram is created only for an area having a width か ら from the crown 30. As a result, a y-axis histogram having a head width is obtained. The width △ is determined by the following equation, for example, where the size of the image is X × Y.

Δ = (Y−Ys) × ββ = 1/4 (or 1/5) Next, the lower end of the face area is detected. For this detection,
The head length is determined by multiplying the head width by a certain ratio r. As a value of r, about 1.3 to 1.6 is appropriate. By adding the head length to the value of the head detected previously, the lower end of the face area is obtained. By the above processing, the face area is specified by the rectangle 50 defined by the coordinates Xs, Xe, Ys, Ye as shown in FIG.

FIG. 4 shows another face area detecting method. In the method of FIG. 3, the coordinates Xs, Xe, Ys, and Ye are obtained at a resolution of one pixel, but in the method of FIG. 4, the inter-frame difference image is divided into a plurality of blocks 40 having a certain size. The size of the block 40 may be smaller than or the same as the size of the coding block. In block 40, the number of pixels exceeding a first preset threshold is counted.

Next, the count value of the number of pixels in the block 40 from the upper left end of the screen is sequentially compared with a second threshold value. At the time of this comparison, first, the y coordinate Ys of the block whose count value exceeds the second threshold is detected as the top 30.

Next, within the range of the width Δ (Δ is determined, for example, in the same manner as in the method of FIG. 4), the count value is set to the second value.
Among the blocks exceeding the threshold value, the coordinates of the block 41 having the left end coordinates are obtained as the left end coordinates Xs of the face area, and the coordinates of the block 42 having the right end coordinates are obtained as the right end coordinates Xe of the face area. The lower end coordinates Ye of the face area are determined by, for example, the method of FIG. Also according to this method, as shown in FIG. 5, the face area has coordinates Xs, Xe,
It is specified by a rectangle 50 defined by Ys and Ye.

FIG. 6 shows a face area detecting circuit using the principle of FIG. In this face area detection circuit, FIG.
From the inter-frame difference image shown in FIG.
The histogram shown in FIG. 7 (c) and the x-axis histogram shown in FIG. By extracting the coordinates of the change points of these histograms, the face area is designated by a rectangle as shown in FIG.

It is conceivable that a large amount of signal components of the inter-frame difference image are generated from a portion where movement is likely to occur, such as a shoulder. For this reason, an x-axis histogram including a shoulder portion is created as shown above BB ′ in FIG. 7B. Therefore, a vertical region of the inter-block difference image is restricted using the y-axis histogram, and an x-axis histogram is created within the region. As a result, an x-axis histogram of only the face region not including the shoulder portion as shown above the line AA 'in FIG. 7B is obtained.

In FIG. 6, an image signal from the frame memory 101 in FIG. The frame memory 201 delays the input image signal by one frame. The subtracter 202 outputs an inter-frame difference image signal by subtracting the image signal output from the frame memory 201 from the input image signal. Histogram creation circuits 203 and 204 respectively create y-axis and x-axis histograms from the inter-frame difference image signal. It should be noted that the x-axis and y-axis histograms can be created using the inter-field difference image instead of the inter-frame difference image.

The average value circuits 205 and 206 calculate an average value from the created y-axis and x-axis histograms. The comparators 207 and 208 use the average value as a threshold value and compare the average value with each histogram created by the histogram creation circuits 203 and 204. Change point detection circuits 209 and 2
10 is based on the x-axis and y values from the outputs of the comparators 207 and 208.
The coordinates of the change point of the axis histogram are detected, and the coordinate data is output. The output of the change point detection circuit 209 corresponding to the y-axis is used to determine the range for creating the y-axis histogram.
It is supplied to the x-axis histogram creation circuit 204.

Each of the histogram creating circuits 203 and 204 includes, for example, a binarizing circuit 301, a counter 302, a memory 303, and a coordinate address control circuit 304 as shown in FIG.

The coordinate data from the change point detection circuits 209 and 210 is supplied to the area calculation circuit 211. As shown in FIG. 5, the area calculation circuit 211 sets the rectangular area 50 of FIG. 5 including the coordinates Xs, Xe, Ys, and Ye of the change points of the x-axis and y-axis histograms as the face area, Region designation signal 212 for distinguishing and designating the region
Is output. This area designation signal 212 is, for example, the terminal 2
1-bit data that is set to “0” when the image signal input to 00 is a face area and “1” when the image signal is outside the face area is used.

FIG. 9 shows another example of the face area detection circuit of FIG. The face area detection circuit includes a histogram memory 401 which is an external circuit for creating a histogram,
Frame memory 402, counter 403, address conversion circuit 404, comparator 405, subtractor 406, adder 40
7 and selectors 408 to 411 and the CPU 412. This face area detection circuit detects a face area in the following three stages (a) to (c). FIG.
In FIG. 5, the signal flows in the stages (a), (b) and (c) are indicated by solid lines, broken lines and dashed lines, respectively.

(A) Frame data is stored in the frame memory 4
02 (b) Obtain an inter-frame difference image signal between the frame data in the frame memory 402 and the next frame data, and create an x-axis and y-axis histogram using this signal. (C) Calculation by the CPU 411 The calculation result is output as a region designation signal composed of map data in response to a request from the image encoding device.

The stage (c) is further divided into four stages: (1) detection of a face area, (2) determination of whether or not a detection result can be obtained, (3) creation of a map, and (4) output of map data. .

Next, the low-pass filter 10 shown in FIG.
3 will be described. As described above, filtering in the low-pass filter 103 may be performed in a spatial direction, in a time axis direction, or in a combination thereof.

FIG. 10 shows a low-pass filter for performing spatial filtering. In FIG. 10, a pixel memory 501 and a line memory 502 delay the output signal of this filter by one pixel and one line, respectively. Output terminals of these memories 501 and 502 are coupled to an adder 503. The output signal of the adder 503 is halved by a bit shift and then coupled to a subtractor 504. The bit shift is realized only by the operation of connection between the output of the adder 503 and the input of the subtractor 504, and does not require any special hardware.

The output of the subtractor 504 is a coefficient α (0 <α <
1) is input to a coefficient multiplier 505 for multiplication. The coefficient multiplier 505 is composed of, for example, a ROM (Read Only Memory), and the output signal of the subtractor 504 and the area designation signal from the face area detection circuit 102 are input to the address input. The adder 506 adds the output signal of the coefficient multiplier 505 and the input signal of the filter to generate an output signal of the filter.

In this low-pass filter, first, a pixel value one pixel before the current pixel of the input image signal is obtained by a pixel memory 501, a line memory 502, an adder 503, and a bit shift (1/2). The average value with the pixel value one line before the current line is calculated. The difference between the average value and the pixel value of the current pixel is obtained by the subtractor 504. This difference is multiplied by α by the coefficient multiplier 505 and added to the pixel value of the current pixel by the adder 506, thereby obtaining an output signal that has been subjected to low-pass characteristic filtering in the spatial direction.

The ROM constituting the coefficient multiplier 505 includes:
An output signal from the subtractor 504 is input as a part of the address data, and an area designation signal 212 from the face area detection circuit 102 is input as another part of the address data. At the address corresponding to the area designating signal (“0”) of the face area of the ROM, zero-value data is stored. At the address corresponding to the area designating signal (“1”) of the area other than the face, the corresponding value is stored. Input data corresponding to the address (subtractor 50
4) is multiplied by a coefficient α. Therefore, when the image signal of the face area is input to the low-pass filter, the value of the output signal of the coefficient multiplier 505 becomes 0, and the output signal of the adder 506 becomes the same as the input image signal. That is, this filter outputs an input image signal as it is in the face area, and outputs an image signal which has been subjected to low-pass characteristic filtering in an area other than the face.

FIG. 11 shows a low-pass filter for performing filtering in the time axis direction. A frame memory 60 for delaying the output signal of this filter by one frame
The output of 1 is coupled to a subtractor 602. Subtractor 602
Subtracts the input signal from the output signal of the line memory 601. The output of the subtractor 602 is coupled to a coefficient multiplier 603 for multiplying by a coefficient β (0 <β <1). The coefficient multiplier 603 is constituted by a ROM, and the output signal of the subtractor 602 and the area designation signal 212 from the face area detection circuit 102 are input to the address input thereof. The adder 604 adds the output signal of the coefficient multiplier 603 and the input signal of the filter to generate an output signal of the filter.

In this low-pass filter, a subtractor 602 obtains an inter-frame difference image signal between the filter output signal of the previous frame and the image signal of the current frame stored in the frame memory 601. This inter-frame difference image is multiplied by β in the coefficient multiplier 603 and added to the input signal in the adder 604, thereby obtaining an output signal subjected to low-pass filtering in the time axis direction. This low-pass filter is in the form of a recursive linear filter, and therefore has a stronger filter action as β approaches 1. The ROM constituting the coefficient multiplier 603 stores, as in the coefficient multiplier 505 in FIG. 10, 0-value data at an address corresponding to the area designation signal of the face area, and stores the area designation signal of an area other than the face. At the address corresponding to the data, data of a value obtained by multiplying the input data (output signal of the subtractor 602) corresponding to the address by the coefficient α is stored.
Therefore, the filter outputs the input image signal as it is in the face area, and outputs an image signal subjected to low-pass filtering in the area other than the face.

FIG. 12 shows another example of a low-pass filter for performing filtering in the time axis direction. FIG.
In FIG. 2, the coefficient multiplier 603 in FIG. This nonlinear circuit 6
Reference numeral 05 denotes a ROM, and 0-value data is stored in an address corresponding to the area specifying signal of the face area, and input data corresponding to the address is stored in an address corresponding to the area specifying signal of the area other than the face. (Output signal of the subtracter 602) stores data of a value obtained by multiplying by a coefficient for giving a predetermined nonlinear input / output characteristic.

The input / output characteristics of the nonlinear circuit 605 are shown in FIG.
(A) or FIG. 13 (b). According to the input / output characteristics of FIG. 13A, a non-linear filter in which the function of the low-pass filter is applied only to the small amplitude region is realized. This characteristic is effective for removing small amplitude noise. According to the input / output characteristics shown in FIG. 13B, small-amplitude noise is removed, and a moving portion of the image signal is also subjected to a low-pass filter in the time axis direction. The nonlinear circuit 605 may be configured so that the input / output characteristics of FIGS. 13A and 13B can be switched. In that case, FIG.
The characteristic shown in FIG. 13A is set, and the characteristic shown in FIG. 13B is set for an image signal in an area other than the face.

According to the method of filtering the low-pass characteristic in the time axis direction, the resolution does not decrease because the difference between frames does not increase in an image signal of a still portion such as a background portion.

FIG. 14 shows a low-pass filter obtained by combining a low-pass filter in the spatial direction and a low-pass filter in the time axis direction. 14, parts corresponding to those in FIGS. 10 and 11 are denoted by the same reference numerals. This low-pass filter includes X,
It is configured by a recursive filter using the values of pixels having a positional relationship of A, B, and C. When the value of the pixel X is input to this filter, the pixel value of the pixel B is stored in the pixel memory 501, and the pixel value of the pixel C is stored in the line memory 502. Thereby, the adder 503 and the bit shift (×
）), Passes through a subtractor 504, a coefficient multiplier 505, and an adder 506, and is subjected to a low-pass filter in the spatial direction similarly to the low-pass filter of FIG.

On the other hand, when the pixel value of the pixel A is given from the frame memory 601, the subtractor 602, the coefficient multiplier 603 and the adder 6, as in the low-pass filter of FIG.
A low-pass filter in the time axis direction is applied through the step 04.

The multiplication coefficients α and β of the coefficient multipliers 505 and 603 are preferably variable by external control for the following reasons. The nature of the image degradation differs between the low-pass filter in the spatial direction and the low-pass filter in the time axis direction. That is, according to the low-pass filter in the spatial direction,
While the resolution of the background portion and the motion region both decrease uniformly, the resolution of the background portion does not decrease according to the low-pass filter in the time axis direction. On the other hand, when a low-pass filter is applied in the time axis direction, a "blur" that leaves a tail in a moving area occurs, and the degree of the blur changes depending on the magnitude of the movement. The appearance of these image degradations greatly changes depending on the bit rate of the encoded data, the magnitude of the motion of the image, and the like.

If the multiplication coefficients α and β, that is, the filter coefficients are made variable, the distribution of the filtering strength of the low-pass filter in the spatial direction and the time axis direction can be changed to obtain the information which is the object of the present invention. It is possible to control the appearance of image degradation while maintaining the effect of reducing the amount. If this control can be performed by the user, the user can change the image quality of the decoded image according to his / her preference. The variation of the multiplication coefficients α and β is achieved, for example, by giving data for designating the values of α and β to the ROMs constituting the coefficient multipliers 505 and 603 as a part of the address data.

Although the low-pass filter 103 described above is constituted by a recursive filter, it may be constituted by a non-recursive filter. Further, the entire region including the face region and the other region of the input image signal may be filtered.
In that case, it is necessary to switch the filter characteristics so that the filtering is performed more strongly in the area other than the face area than in the face area. Hardware change for this purpose is basically unnecessary, only the coefficient multipliers 505, 603, 6
It is only necessary to change the contents of the ROM used for the 05.

Another embodiment will be described with reference to FIG.
In FIG. 16, the same parts as those in FIG.
Description is omitted. In this embodiment, the face area detection circuit 102 in FIG. The area specifying device 120 includes a coordinate input device 801 and an area calculating circuit 802 as shown in FIG. As the coordinate input device 801, a device such as a keyboard capable of directly inputting coordinates by numerical values, a pointing device such as a mouse, or a touch panel is used. In particular, according to the mouse and touch panel,
The user can input coordinates exactly corresponding to the position on the image while viewing the image on the display screen, which is convenient.

The area calculation circuit 802 has the same configuration as the area calculation circuit 211 in FIG.
An area designation signal 803 for distinguishing and designating the area defined by the coordinates and the other area is created and output from the coordinate data of the plurality of coordinates. When specifying an area, for example, as shown in FIG.
Is specified by a mouse or a touch panel. As a result, a rectangular area 91 having points A and B as corner points is defined. Alternatively, as shown in FIG. 19, by specifying a large number of points, a polygonal region 92 composed of line segments connecting those points is defined. Instead of a polygon, a spline curve can be used.

According to this embodiment, filtering is performed by the low-pass filter 103 on an area excluding an arbitrary specific area which is not limited to a face area determined to be important by the user.

FIG. 20 shows a videophone system constructed by developing the embodiment of FIG. The source coder 701 is a portion except for the face area detection circuit 102 in FIG. 1, and receives an image signal from the TV camera 700. The region specifying device 120 is connected to the low-pass filter 103 shown in FIG. 1 in the source coder 701. The coded image signal output from the source encoder 701 is subjected to error correction coding by the FEC encoder 702, and the interface (H.221 framing, IS
(Including a DN interface). On the other hand, the coded image signal transmitted from the partner terminal via the transmission path is
The signal is input to the decoder 705 and error correction decoded. The error-corrected decoded image signal is supplied to a source decoder 706 configured to perform a process reverse to that of the source coder 701.
And a decoded image signal is output.

A monitor mode selection switch 707 operated by the user selects an output image signal from the low-pass filter 103 in the source coder 701 in the self monitor mode, and a source decoder 706 in the other party monitor mode.
Is selected. The selected image signal is converted by the video encoder 708 into a video signal compatible with a TV system such as NTSC, PAL or SECAM, and supplied to the monitor display 710 through the superimpose circuit 709. The superimpose circuit 709 is supplied with a region designation signal from the region designation device 120. Superimpose circuit 709
Generates an image signal representing the frame of the region based on the region designation signal, and adds the image signal to the image signal from the video encoder 708. As a result, on the screen of the monitor display 710, the frame is superimposed and displayed on the image. From the display of the line segment, the user can confirm the specified area.

In this embodiment, the region designation by the region designation device 120 is performed by the user on the transmission side with respect to the low-pass filter in the source coder 701. In the superimpose circuit 709, an image signal indicating a frame of the region 91 or 92 specified as shown in FIG. 18 or 19 is added to the image signal from the video encoder 708.

On the other hand, the area specifying signal from the area specifying device 120 is applied to the low-pass filter 103 in the source coder 701.
Is also supplied. In the low-pass filter 103, a specific area is obtained similarly to the superimpose circuit 709.
Filtering is performed only on the image signal of the region other than the region.

In the self-monitoring mode, the image after low-pass filtering is performed on the monitor display 710.
Is displayed with. After an area is specified by the area specifying device 120, the area specifying signal is continuously output. Thereby, the frame line indicating the designated area is superimposed on the image on the monitor display 710 and displayed. From this display, the user can confirm the effect of filtering in an area other than the specified specific area. Therefore,
While confirming this effect, the area designation can be performed again.

FIG. 21 shows a videophone system according to still another embodiment. The same parts as those in FIG. 17 are denoted by the same reference numerals, and description thereof will be omitted. In this example,
The region specification for the low-pass filter in the source coder 701 on the transmission side is performed by the region specification device 120 on the reception side. That is, on the receiving side of the encoded image signal, the coordinate data output from the area specifying device 120 is
The coded image signal is transmitted to the transmission side via the interface 703 and the transmission path.

For example, if the interfaces 703 and 704 are When the framing and communication procedures according to H.221 and H242 are used, a channel for data transmission can be opened at an arbitrary timing even during the communication of the encoded image signal. Coordinate data can be transmitted using this data transfer channel.

On the transmitting side of the encoded image signal, the received coordinate data is supplied to a low-pass filter in the source coder 701 via the interface 704. In the low-pass filter, the image signal is filtered only to a specific region specified from the input region specifying signal in the same manner as described above.

In the second, third and fourth embodiments,
The image signal in the designated area may be filtered by a low-pass filter. This is effective, for example, when it is desired to display only a specific portion in an image with particular blur.

According to the present invention, the image quality is reduced by selectively or intense low-pass filter processing on the area other than the face before encoding, so that the area other than the face is easily noticeable such as block distortion. It is possible to improve the subjective image quality by improving the image quality of the face area without causing distortion.

Further, according to the present invention, a face area detecting circuit and a filter are selectively applied only to a specific area in accordance with the result of face area detection or a filter characteristic is changed in accordance with the result of detection. This can be realized only by providing a low-pass filter capable of performing the above-described operations, and the circuit configuration and encoding control after encoding may be the same as those in the related art, and the implementation is extremely easy.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of an image encoding device according to the present invention.

FIG. 2 is a diagram for explaining a format of a macroblock and a format of a signal input to a low-pass filter.

FIG. 3 is a diagram for explaining the principle of face area detection.

FIG. 4 is a diagram for explaining another principle of face area detection.

FIG. 5 is a diagram showing that a detected face area is defined by a rectangle.

FIG. 6 is a block diagram showing a configuration example of a face detection circuit in FIG. 1;

FIG. 7 is a view for explaining a face area detection operation of the face detection circuit in FIG. 6;

8 is a block diagram showing a configuration example of a histogram creation circuit used in the face detection circuit of FIG.

FIG. 9 is a block diagram showing another configuration example of the face detection circuit in FIG. 1;

FIG. 10 is a block diagram of an example of a low-pass filter in FIG. 1;

11 is a block diagram of another example of the low-pass filter in FIG.

FIG. 12 is a block diagram of another example of the low-pass filter in FIG. 1;

13 is a diagram showing input / output characteristics of the nonlinear circuit in FIG.

FIG. 14 is a block diagram of still another example of the low-pass filter in FIG. 1;

FIG. 15 is a diagram for explaining the operation of the low-pass filter of FIG. 14;

FIG. 16 is a block diagram showing another embodiment of the image encoding device according to the present invention.

FIG. 17 is a block diagram showing the configuration of an area designation device used in FIG.

FIG. 18 is a view for explaining a method of specifying an area of an input image in the apparatus of FIG. 16;

FIG. 19 is a view for explaining another method for specifying an area of an input image in the apparatus of FIG. 16;

FIG. 20 is a block diagram of an embodiment of a videophone device using the image encoding device according to the present invention.

FIG. 21 is a block diagram of another embodiment of the videophone device using the image encoding device according to the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 101 ... Frame memory 102 ... Face area detection circuit 103 ... Low-pass filter 105 ... Inter-frame / inside judgment circuit 107 ... DCT circuit 108 ... Quantization circuit 109 ... Valid / invalid judgment circuit 110 ... Inverse quantization circuit 111 ... Inverse DCT Circuit 113: motion compensation inter-frame prediction circuit 114: loop filter 116: multiplexing circuit 117: variable length coding circuit 118: buffer 119: coding control circuit 120: area designation device

Continuation of the front page (56) References JP-A-63-299680 (JP, A) JP-A-2-17777 (JP, A) JP-A-2-22987 (JP, A) JP-A-2-44884 (JP JP-A-2-57067 (JP, A) JP-A-62-178071 (JP, A) JP-A-3-6185 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB Name) H04N 7/10 H04N 7/14-7/173 H04N 7/20-7/22 H04N 7/30 H04N 1/41 G06T 9/00

Claims (6)

(57) [Claims] 1. An image encoding apparatus for encoding an input image signal of a moving image, wherein an image signal of an area other than a specific area in the input image signal is selectively selected in at least one of a spatial direction and a time axis direction. An image encoding apparatus, comprising: a low-pass filter unit for filtering the image signal; and an encoding unit for encoding an output image signal from the low-pass filter unit. 2. An image coding apparatus for coding an input image signal of a moving image, comprising: a region specifying signal for detecting a specific region in the input image signal and specifying the specific region and the other region in a distinguishable manner. And a low-pass filter for selectively filtering an image signal of an area other than the specific area in the input image signal in at least one of a spatial direction and a time axis direction according to the area specifying signal. An image encoding apparatus comprising: a filter unit; and an encoding unit that encodes an output image signal from the low-pass filter unit. 3. An image coding apparatus for coding an input image signal of a moving image, comprising: a region specifying signal for detecting a specific region in the input image signal, and specifying the specific region and the other region. And low-pass filter means for filtering the input image signal according to the area designation signal so that an image signal of an area other than the specific area is more strongly filtered than an image signal of the specific area. And an encoding means for encoding an output image signal from the low-pass filter means. 4. An area designating means for outputting an area designating signal for distinguishing and designating an arbitrary specific area and other areas in an input image signal, and according to the area designating signal, An image comprising: low-pass filter means for selectively filtering an image signal of an area other than the specific area; and encoding means for encoding an output image signal from the low-pass filter means. Encoding device. 5. The low-pass filter means includes: a first low-pass filter for performing filtering in a spatial direction; a second low-pass filter for performing filtering in a time-axis direction; Means for coupling the output of one of the first and second low-pass filters to the input of the other filter, and individually determining the filtering strength of each of the first and second low-pass filters. The image coding apparatus according to any one of claims 1 to 4, further comprising means for controlling. 6. An image encoding / decoding device for encoding and transmitting an input image signal, and decoding the encoded image signal on a receiving side, comprising: an arbitrary specific region in the input image signal; An area specifying unit that outputs an area specifying signal for specifying the area by distinguishing the area from the other area; a unit that transmits an area specifying signal output from the area specifying unit from the receiving side to the transmitting side; Low-pass filter means for selectively filtering an image signal other than a specific area designated by the area designation signal from the receiving side in a signal; and encoding an output image signal from the low-pass filter means An image encoding / decoding device comprising: an encoding unit.