JPH06233292A - Picture signal transmitting device - Google Patents

Abstract

PURPOSE:To obtain picture quality which sufficiently fulfills a function as a video telephone even at the same transmission bit rate as the conventional one by preferentially assigning transmission information quantity to the face area of a person, specially, an eye part. CONSTITUTION:In addition to the standard information source coder of H.261, a picture signal transmitting device is provided with a quantization equipment control part 4 which extracts the face area through the use of a macroblock from which a movement vector is detected, detects the eye part from within the face area through the use of a DCT output and lets a quantization step size for the eye area be small at the time of quantization by a quantization equipment 5. Then, transmission information quantity is preferentially assigned to the face part, specifically, eyes and a mouth, being more important information for the other reception party so as to make picture quality superior. Transmission information quantity is preferentially assigned to the face area, specifically, the eye part, of the person and transmission to let the picture quality of the face area, specifically, the eye area, be high quality can be enabled. Therefore, even in low bit rate transmission, the subjectively preferable picture for which an expression is easy to be understood is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal transmission apparatus suitable for one-to-one communication with images such as a videophone.

[0002]

2. Description of the Related Art Video coding systems for videophones and videoconferences include, for example, H.264. 261 is CCITT (International Telegraph and Tele
phoneConsultative).

The above Recommendation H.264. In the H.261 moving image encoding system, the video format signal is a CIF (Common Inter
mediate format) or QCIF format, and the converted signal is INTRA for each macro block (MB) which is a set of small blocks composed of a plurality of pixels.
Either (intra) mode or INTER (inter) mode is designated. In the INTRA mode, DCT (Discrete Cosine Transform) is applied to the intra-frame data in small block units, and in the INTER mode, the DCT is applied to the inter-frame difference data in small block units. Further, in the moving image coding method, a motion vector is obtained by motion detection for each macro block (MB), and motion compensation is performed. Further, the DCT coefficient after the above DCT is quantized, and the macroblock (MB) address (macroblock address) and type (macroblock type) are quantized.
And side information such as a motion vector and the like are output as transmission information after variable length coding.

[0004]

By the way, the above transmission information is once stored in the transmission data buffer. Therefore, the quantization step when quantizing the DCT coefficient is determined by the amount of data in the transmission data buffer.
No consideration is given to the content of the image in the quantization.

In general, when transmitting moving image information, the more the transmission information is assigned (the more image information can be transmitted), the better the image quality of the decoded image will be. That is, if the quantization step at the time of the above quantization is made fine and the transmission information obtained by the quantization at the fine quantization step is transmitted, the image quality of the image decoded later becomes good. However, since the number of bits that can be propagated is usually limited by the capacity of the line (telephone line, etc.), it is not possible to secure a sufficient amount of transmission information to obtain a good image quality after decoding, and the conventional TV cannot be used. Low quality is unavoidable in telephones.

Therefore, the present invention has been proposed in view of the above-mentioned circumstances, and even if the transmission bit rate is the same as the conventional one (even if the number of bits that can be transmitted is limited), it is a videophone. It is an object of the present invention to provide an image signal transmission device capable of obtaining an image quality capable of sufficiently performing the above function.

[0007]

The image signal transmission apparatus of the present invention has been proposed to achieve the above-mentioned object, for example, in the case of a so-called videophone, the input image of the input image whose upper body is the object. An image signal transmission device for compressing and encoding and transmitting an image signal, comprising extracting a face area and an eye area from an image of an upper half of a person, and transmitting the amount of transmission information of the image of the eye area of the face area to another. It is provided with a transmission information amount control means for allocating more than the transmission information amount of the image of the area.

Here, the transmission information amount control means sets the order of allocating the transmission information amount in the face region in the order of the eye region, the mouth region, the nose region and the chin region. Further, the transmission information amount control means detects the eye area for all consecutive frames, and the position of the eye area detected in the current frame is the position of the eye area detected in the immediately preceding frame of the current frame. On the other hand, if it changes abruptly, the detection result of the eye area of the current frame is invalidated and the detection result of the eye area of the previous frame is used.

[0009]

According to the present invention, the transmission information amount is preferentially assigned to the face area of a person, particularly the eye portion, so that a subjectively preferable image whose facial expression is easy to understand can be obtained even in low bit rate transmission. .

[0010]

Embodiments of the present invention will be described below with reference to the drawings.

The image signal transmitting apparatus of the embodiment of the present invention is shown in FIG.
As shown in FIG. 2, a device for compressing and encoding and transmitting an image signal of an input image intended for the upper half of the person, extracting the face area and the eye area from the image of the upper half of the person, The quantizer control unit 4 is provided as a transmission information amount control means for allocating the transmission information amount of the image in one of the eye regions to be larger than the transmission information amount of the image in the other regions.

Here, the quantizer control section 4 assigns the order of allocating a large amount of transmission information in the face area to the eye area,
The mouth area, nose area, and jaw area are in this order. Further, the quantizer control unit 4 detects the eye area in all consecutive frames, and the position of the eye area detected in the current frame is the position of the eye area detected in the immediately preceding frame of the current frame. If the position changes abruptly, the detection result of the eye area of the current frame is invalidated, and the detection result of the eye area of the previous frame is used.

In this embodiment, as the compression encoding method of the image signal, the above-mentioned recommendation H.264 of the moving image encoding method for the videophone and videoconference is used. An example in which H.261 is applied is given.

Hereinafter, this Recommendation H. An image signal transmission device (image signal encoding device) of this embodiment to which H.261 is applied will be specifically described.

In FIG. 1, a terminal 10 is an input terminal to which input video data is supplied. This input video data is sent to the motion compensation image memory 8. The motion compensation image memory 8 has a function as an image memory (frame memory) as well as a variable delay function for motion compensation. The input video data is compared with the data of the previous frame in the motion compensation image memory 8 to perform motion compensation interframe prediction. Further, the motion vector detected by the motion compensation image memory 8 is
It is sent to the subsequent stage via the terminal 18.

The input video data is also sent to the converter 1. The converter 1 is a small block that constitutes a macroblock (MB) of the input video data.
DCT is applied to the unit image data. The converter 2 is
The difference data (prediction error data) obtained by subtracting the prediction image data supplied from the motion compensation image memory 8 via the loop filter 9 and the input video data from the terminal 10 by the subtracter 11 DC as converter 1
T. The output of the converter 1 becomes the DCT coefficient data in the INTRA mode, and the output of the converter 2 becomes the I
It becomes the DCT coefficient data in the NTER mode. These coefficient data are sent to the quantizer control unit 4, which will be described in detail later.

The quantizer 5 next to the quantizer controller 4 quantizes the output data from the quantizer controller 4. The output data of the quantizer 5 is output from the quantization coefficient output terminal 17 and is sent to the configuration of the local decoder.

The quantized DCT coefficient data sent to the local decoding structure is passed through an inverse quantizer 6 which performs the inverse quantization of the quantizer 5, and the inverse transform of the DCT is performed. The data is sent to one data input terminal of the adder 13 via the IDCT converter 7.

The other data input terminal of the adder 13 is connected to the common terminal of the changeover switch 12 in which the grounded switched terminal b is selected in the INTRA mode and the switched terminal a side is selected in the INTER mode. There is. The output terminal from the motion compensation image memory 8 is supplied to the switched terminal a via the in-loop filter. The output data of the adder 13 is sent to the motion compensation image memory 8. The ON / OFF information of the in-loop filter 9 is output via the terminal 19.

Here, the coding control unit 3 selects INTER / I by selecting a mode in which the amount of transmission information is smaller.
NTRA discrimination is performed. It should be noted that the INTER / INTRA discrimination in the small block unit is performed according to the recommendation H.
When the information source encoder 261 is used, it is performed on the image plane before performing DCT, but in low bit rate transmission, there is a margin in the computing capacity of the computing unit that performs DCT, and transmission buffer control Is stable,
It has been proposed to perform INTER / INTRA discrimination after DCT. In the present embodiment, the latter is used because it is necessary to output the INTRA DCT for all the small blocks when extracting the eye portion from the human face area of the image. The coding mode selection information in the coding control unit 3 is sent to the subsequent stage configuration via the terminal 14, and the transmission / non-transmission information is output from the terminal 15. Also, the INTER / INTRA discrimination data of the encoding control unit 3 is sent to the quantizer control unit 4.

The quantizer control unit 4 is also supplied with the data of the motion vector detected in the motion compensation image memory 8. The quantizer control unit 4 stores the addresses of the macroblocks (MB) in which the motion vectors have been detected, and the number of macroblocks (MB) in one frame exceeds the threshold value. If so, it is assumed that the person's head is at that position. On the contrary, when the number of macroblocks (MB) that exceeds the certain threshold value, that is, the head of the person is not detected, the result of the previous frame is succeeded.

Further, the quantizer control section 4 uses the DCT coefficient data (DCT coefficient data of INTRA) of the converter 1 as a small block corresponding to (face width × vertical width of eyes) of the person in the image. The sum of the DCT coefficients of the frequency components including a lot of eye parts is obtained in the unit of the lump, and the sum is scanned in the head region, and as shown in FIG. Among the candidates for the large eye region (a group of small blocks), the one in which the sum of the DCT coefficients of the macro blocks (MB) below both ends of the face width is the smallest is determined as the eye region.

The eye area is obtained for all consecutive frames, but in order to reduce the effect of erroneous detection, the eye area in the next frame (current frame) is suddenly separated from the eye position in the previous frame. When the eye position is detected, the detection result of the frame (current frame) is invalidated, and the result (eye position) of the previous frame is used again.

In the face area, below the eyes are the nose, mouth, and chin, and their respective positions are obtained from the eye position and the head area. Then, to these regions, a small quantization step size is assigned according to the priority order such that the first is the eye, the second is the mouth, the third is the nose, and the chin. That is, the quantization step size of the eye region is set to be the smallest, and the mouth, the nose, and the jaw are sequentially increased in size. The other parts are changed depending on the amount of data in the transmission data buffer.

According to the thus obtained quantization characteristic (quantization step size for each area), the DCT coefficient data in the INTER or INTRA mode is converted into the quantizer 5 described above.
In the above, the quantized data is variable-length coded by a variable-length coder as a later-stage configuration (not shown) to be transmission information. The above quantization characteristics are also
It is sent from the terminal 16 to the next-stage configuration.

Next, referring to the flowcharts of FIGS. 2 and 3, the above-mentioned processing for obtaining the head area from the macroblock group (macroblock group) in which the motion vector is detected in the quantizer control section 4 is described above. More detailed than the explanation.

2 and 3, first, in step S10, the recommendation H. GOB (group of picture) address gn in the H.261
Macroblock address mn, identification label of the macroblock group in which the motion vector is detected, label, line
The line number ln and the macroblock number lmn of the macroblock plane are each initialized to 0, and the threshold value of the head size (the number of macroblocks) is set to MBT. Here, the total number of GOBs is GN, the total number of macroblocks per GOB is MN, the total number of lines is LN, and the total number of macroblocks per line is LMN. In this embodiment, CN = 12, MN = 33, LN = 18PLM
Let N = 22.

Next, step S11 and step S12
Then, in order to know the adjacency state of the macroblock in which the motion vector is detected, as shown in A and B of FIG. 4, from the GOB address and the macroblock address (GOB.MB address), the line number and the macroblock address (line. The operation for converting to the MB address) is performed for the LMN line.

Then, in step S13, the line ln
By detecting the end points (start point and end point position) of the block of the detected macroblock of the motion vector and comparing which macroblock group the block belongs to with the label of the macroblock block of the above line. Judge and label. If the block of lines being processed is the top of a new macroblock group, label number 1
The line number is increased to lnt (label).
In this step S13, the macroblock sandwiched between the macroblocks having the same label and having the detected motion vector is also interpolated as the head region.

In step S14, ln = ln + 1 is set,
If the line ln is not larger than the final line LN in step S15, lmn is set to 0 in step S16 and the process returns to step S11. If it is larger, the processing for that frame is completed, and therefore the head area detection in that frame is performed. Go to processing.

Upon proceeding to the head area detection processing, in step S17, a label variable lab used for checking the size of the detected macroblock group of each label and a flag face indicating whether or not a face area is detected. And are set to 0, and lab is incremented by 1 in step 18 shown in FIG.

In the next step S19, if lab is smaller than the total label number label, step S20.
Proceed to. In this step S20, the size of the macroblock group labeled "lab" (total number of macroblocks)
And MBT are determined, and if the size of the macroblock group is smaller than MBT, the process returns to step S18, and if it is larger, the process proceeds to step S21.

In this step S21, face is set to 1, the identification label of the macroblock group regarded as the head region is set to lab, and the label of the macroblock group regarded as the head is left, and then the process returns to step S18.

In step S19, lab is la
If it is determined that it is larger than bel, the process proceeds to step S22. In step S22, it is determined whether face is 1, and if it is 1, the process proceeds to step S24. In this step S24, the end point flm of each line l of the label
(L) and frm (l) are stored in the memory as a result of the frame currently being processed. On the other hand, the above step S2
If it is determined in 2 that it is not 1, the process proceeds to step S23. In this step S23, the end points and vertices of the current frame are compared with reference to the end points and lump vertices of the previous frame, and the end points and the vertices are combined so that the sum of the two detection results becomes the final result. Correct the data and store it in memory.

Next, a flow chart of the processing for obtaining the eye area in the head area will be described with reference to FIG.

In FIG. 5, first, step S25.
, The face width fw and the eye vertical width e from the horizontal width of the block of macroblocks in which the motion vector regarded as the head is detected.
h, a small block (block) for the distance fh between the crown and nose
Calculate in units.

In step S26, from the processing in macro block units in which four small blocks are arranged in two rows and two columns, for example,
In order to move to the processing in units of the small blocks, in order to obtain the top vertex and the end point converted from the macro block address to the block address, the top line number is lntb = lnt.
(Flabel) × 2, line l at block address
Fl (l) = flm (l / 2) × 2 at the left end and f at the right end
r (l) = frm (l / 2) × 2 is set, and the process proceeds to the eye area extraction process from the head.

In step S27, the line l is lntb.
If it is smaller than + fh, the process proceeds to step S28. In step S28, when scanning the fw × eh block groups in the head region while shifting them by one block, the variable le (l) indicating the position of the upper left end of the block group is changed to
Let le (l) = fl (l). In the next step S29, the sum sm of the DCT coefficients of the frequency components including many fw × eh small blocks with this le (l) as the upper left corner sm.
Find f.

Here, FIGS. 6A and 6B show examples of which frequency component DCT coefficient is selected. This is because the upper and lower eyelids and eyebrows have horizontal stripes and a large difference in brightness from the skin, so that a good recognition rate can be obtained by searching for a portion having a large frequency in the y direction. That is, in the case of the example of FIG. 6, the 8 × 8 D shown in FIG. 9B obtained by DCT of the 8 × 8 pixel block shown in FIG.
With respect to the block made up of CT coefficients, the sum of the DCT coefficients in the hatched portion of FIG. The DCT in the example of FIG. 6 can be obtained by the mathematical expression (1) shown in the equation 1.

[0040]

[Equation 1]

Next, in step S30, the size of smf of the small block group being processed is compared with the upper M pieces of smf calculated so far, and if it is in the Mth rank, it is registered in that rank. In step S31, le (l) = le (l)
The value is incremented by 1, and the process proceeds to step S32.

In step S32, le (l) is changed to f
r (l) -fw + 1 is compared, and if le (l) is larger, step S35 sets l = l + 1, and then step S
27, and if smaller, returns to step S29.

On the other hand, in step S27, l is lntb.
If it is determined that it is larger than + fh, step S
Proceed to 33. In this step S33, as shown in FIG. 7, among several groups of small blocks b of M eye candidates, several blocks (small blocks b indicated by diagonal lines in the figure) located directly below the group of small blocks b are selected. The smallest sum of DCT coefficients is determined as the eye area. This is because there is a tendency that a small block group with a lot of high frequency components such as ears and nose is easily selected among the eye candidates.
Although there is a mouth containing high-frequency components under the nose, since the cheeks are under the eyes and most of them are DC components, the probability that the small block group finally selected is the eye part by doing this Because it will be higher. Since the quantization step is determined for each macroblock, all the macroblocks including the small block group are finely quantized as eye regions.

In the next step S34, in order to reduce the influence of the erroneous detection of the eye region, the result of the current frame is vertically and horizontally 2 compared with the position of the eye detected in the previous frame.
If the result is more than a macroblock away, the result is reserved and the result of the previous frame is used as the eye area. Also,
If the detection result of the next frame is close to the pending result of the frame just processed, then that result is adopted.

Since the eye area is obtained at the end of the processing for one frame, this detection result is used for the quantization step control of the next frame.

As described above, according to the image signal transmission device of the present embodiment, the H. In addition to the standard information source encoder of H.261, a face area is extracted using a macro block in which a motion vector is detected, and an eye portion is detected from the face area using a DCT output, and a quantizer is used. By including the quantizer control unit 4 that reduces the quantization step size for the eye region at the time of quantization in 5, the amount of transmission information is preferentially applied to the human face region, particularly the eye region. Will be assigned (the face area, especially the eye area can be transmitted with high image quality). Therefore, even in low bit rate transmission, it is possible to obtain a subjectively preferable image with an easy-to-understand expression. .

That is, in the apparatus of this embodiment, the recommendation H.264 is used. 261 and a simple method by using the result calculated in the encoding process, the face area, which is more important information to the receiving party, especially the transmission information amount is preferentially applied to the eyes and mouth. By allocating to improve the image quality, the overall subjective image quality can be improved compared to the case where the same encoding method and transmission bit rate are used even if the image quality is slightly degraded in areas other than the face. Will be able to.

Further, by reducing the allocation amount of transmission information in the background and the like, which is not important for the videophone, the predetermined transmission bit rate will not be exceeded.

[0049]

According to the image signal transmitting apparatus of the present invention,
By preferentially allocating the amount of transmission information to the face area of a person, particularly the eye part, even if the transmission bit rate is the same as before (even if the number of bits that can be transmitted is limited), for example, as a so-called videophone. It is possible to obtain an image quality that can sufficiently fulfill the function.

[Brief description of drawings]

FIG. 1 is a block circuit diagram showing a schematic configuration of an embodiment.

FIG. 2 is a flowchart showing a first half of a process of detecting a head region.

FIG. 3 is a flowchart showing the latter half of the process of detecting the head region.

FIG. 4 is a diagram for explaining the conversion from a GOB address / macroblock address to a line address / macroblock address in step S11 of FIG. 2;

FIG. 5 is a flowchart showing a process of detecting an eye area.

FIG. 6 is a diagram showing frequency components used when detecting eye positions.

FIG. 7 is a diagram showing a process for determining one from a block group that is a candidate for M eyes.

[Explanation of symbols]

1 --- INTRA DCT converter 2 ...- INTER DCT converter 3--Encoding control unit 4--Quantizer control unit 5--・ ・ ・ Quantizer 6 ・ ・ ・ Dequantizer 7 ・ ・ ・ ・ IDCT converter 8 ・ ・ ・ ・ Image memory with variable delay function for motion compensation 9 ・ ・ ・..In-loop filter 11 ... Subtractor 12 ... Changeover switch 13 ... Adder

Claims (3)

[Claims] 1. An image signal transmitting apparatus for transmitting an image signal of an input image intended for the upper half of a person, wherein a face area and an eye area of an image of the upper half of a person are extracted, and eyes of the face area are extracted. An image signal transmission apparatus comprising: a transmission information amount control means for allocating a larger amount of transmission information amount of an image of an area than that of an image of another region. 2. The transmission information amount control means sets the order of allocating a large amount of transmission information in the face region in the order of eye region, mouth region, nose region and chin region. The image signal transmission device described. 3. The transmission information amount control means detects the eye area for all consecutive frames, and the position of the eye area detected in the current frame is detected in the previous frame immediately before the current frame. If it changes abruptly with respect to the position of the area, invalidate the detection result of the eye area of the current frame,
The image signal transmission device according to claim 1, wherein the detection result of the eye area of the previous frame is used.