JPH0993578A - Image transmitter, image receiver, image tranmsission method and image reception method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain image communication with small block distortion without decreasing a frame rate even when a large motion takes place in an image. SOLUTION: An A/D converter 19 of the image transmitter converts a video signal into a digital signal and the digital signal is stored in image frame memories 22, 23 by selecting a switch 20 for each frame. A read and write control means 21 decides a picture element sampling method so as to be contained in a frame rate commanded by an object frame rate setting means 26, based on a signal processing time of a QCIF image obtained by dividing an image of a CCIR601 level. The read address is given to the image frame memories 22, 23. A QCIF coding means 25 encodes the image of the QCIF level by DC transformation and quantization to send the coded signal to a receiver via a communication network 28 together with the QCIF number.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to image communication.
An image transmitting device and an image which can transmit and receive a higher resolution image in the case of a still image by reducing the number of encoded frames even when the movement is intense by changing the resolution according to the movement of the image The present invention relates to a receiving device, an image transmitting method, and an image receiving method.

[0002]

2. Description of the Related Art In recent years, audio and video has been used in ITU-TT.
The communication video encoding method, multiplexing method, and communication procedure for visual services have been officially recommended, and accordingly, each company has released a video conference terminal device such as a video conference system and a video TV phone that conforms to CCITT recommendations. There is. However Recommendation H. 261 Full Common Interm
FCIF (352 dots, 288)
A line, hereinafter referred to as 352 * 288. When encoding a moving image with), the compression capacity was not sufficient for a certain line rate depending on the motion, and functional deterioration such as block distortion and reduction of the encoding frame rate was observed.

Here, a conventional image communication method and image communication device will be described with reference to FIGS. FIG. 25 is a block diagram showing the configuration of a conventional image transmitting apparatus. FIG. 26 is a block diagram showing the configuration of a conventional image receiving device. 27 is a block diagram showing the internal structure of the encoding means 7, and FIG. 28 is a block diagram showing the internal structure of the decoding means 12.

As shown in FIG. 25, the image transmitting apparatus is an A / D
The converter 1, the switches 2 and 6, the read / write control means 3, the image frame memories 4,5, the encoding means 7 and the multiplexing means 8 are included, and the output signals thereof are received by the receiving device via the communication network 9. Is output to. Further, as shown in FIG. 26, since the image receiving apparatus receives the signal from the communication network 10, the separating means 11, the decoding means 12, the switches 13 and 17, the read / write control means 14, the image frame memory 15, 1
6, a D / A converter 18 is included.

On the other hand, the encoding means 7 includes a subtracter 57, switches 58 and 59, an adder 60, an encoding control means 61, a converter 62, a quantizer 63, an inverse quantizer 64, as shown in FIG. An inverse converter 65, an in-loop filter 66, and a motion compensation image memory 67 are included.

The decoding means 12 is, as shown in FIG. 28, an adder 68, switches 69 and 70, a decoding control means 71, an inverse quantizer 72, an inverse converter 73, an image memory 7 for motion compensation.
4 and an in-loop filter 75.

Now, the image communication method will be described with reference to FIGS. 29 and 30.
This will be described with reference to FIG. FIG. 29 is an explanatory diagram of conversion from a conventional CCIR601 level image (704 * 576) to an FCIF level image (352 * 288). See also FIG.
From the conventional CCIR601 level image from Quarter Comm
It is explanatory drawing of the conversion to the image (176 * 144) of QCIF level which is on Intermediate Format. As shown in FIG. 25, the video signal is input to the A / D converter 1, digitized, and written in the image frame memory 4 or 5.
The reading / writing control means 3 selects which one to write.
This is to prevent the read and write operations from accessing the same memory at the same time, and the switches 2 and 6 are alternately switched in reverse phase.

The digital video signal written in the image frame memory 4 or 5 is read from the image frame memory 4 or 5 and converted into a format of QCIF or FCIF. This is done as follows, for example. That is, when the original image is at the CCIR601 level as shown in FIG. 29, in order to create an FCIF (352 * 288) image,
One dot is read every two dot clocks, and one line is read every two line cycles. As shown in Figure 30, QC
To create an IF image, one dot is read from a CCIR601 level image at a four-dot clock and one line is read every four line cycles.

The encoding means 7 in FIG. 25 encodes an FCIF or QCIF image, and the multiplexing means 8 multiplexes video data, audio data and other data. Then, this multiplexed data is sent to the communication network 9.

Here, the internal operation of the encoding means 7 will be described with reference to FIG. At the time of encoding the first frame, the switch 58 is switched to the 58a side to perform intra-frame encoding, the video signal is input to the converter 62, and DCT (discrete cosine transform) is performed. The transform coefficient DCTed here is quantized by the quantizer 63, and the quantization index q and the quantization characteristic qz of the transform coefficient are transmitted.

Further, the quantization index q of this transform coefficient
Is input to the inverse quantizer 64 to be inversely quantized, and the inverse transformer 65
Inverse DCT is performed. The video data subjected to the inverse DCT is passed through the adder 60 as it is at the time of encoding the first frame, and it is used as video data for one FCIF or QC.
One IF is stored in the motion compensation image memory 67. At this time, as the identification flag p of INTER / INTRA, “I
NTRA "is transmitted, and" transmission "is transmitted as the transmission / non-transmission identification flag t. The motion vector v and the on / off signal f of the in-loop filter 66 are not transmitted.

The video signal of the next frame is encoded by the encoding means 7.
When inter-frame coding is performed,
A video signal is input to the motion compensation image memory 67, a motion vector search is performed, and a motion vector v is transmitted. The image based on the motion vector is filtered by the in-loop filter 66, and the subtractor 57 performs an operation with the previously input video signal to output the on / off signal f of the in-loop filter 66. In this case, the switch 58 is switched to the 58b side, and the differential video signal obtained by the subtractor 57 is input to the converter 62 to perform DCT processing. Then, after quantizing this signal by the quantizer 63, the quantization index q of the transform coefficient and the quantization characteristic qz are transmitted.

At this time, "INTER" is transmitted as the identification flag p of INTER / INTRA, and "transmission" or "non-transmission" is transmitted as the transmission / non-transmission identification flag t.
On the other hand, the quantization index q of the transform coefficient is converted to the inverse quantizer 6
Inverse quantization is performed in 4, and inverse DCT is performed in the inverse converter 65. Inverse DC
The subtracted difference video data is given to the adder 60, is added to the previous image based on the motion vector, and one FCIF or one QCIF is stored in the motion compensation image memory 67 as video data. Thereafter, this operation is repeated while interposing the intra-frame coding so that the error of the inverse DCT is not accumulated.

Next, the image receiving method will be described with reference to FIGS.
This will be described with reference to FIG. FIG. 31 is an explanatory diagram of conversion from a conventional FCIF level image to a CCIR 601 level image. In addition, FIG. 32 shows a conventional QCIF level image as a CC
It is explanatory drawing of conversion into the image of IR601 level.

As shown in FIG. 26, the multiplexed data from the communication network 10 is supplied to the demultiplexing means 11 and separated into video data, audio data and other data. And FCI
The F or QCIF video data is input to the decoding means 12 and decoded into a digital video signal. The decoded video signal is written in the image frame memory 15 or 16. The reading / writing control means 14 selects which one to write. This is for preventing reading and writing from accessing the same memory at the same time, and switches 13 and 17 are alternately switched in opposite phase.

Next, FCIF (352 * 288) or QCIF written in the image frame memory 15 or 16
Read (176 * 144) video signal, CCIR
Reconstruct a 601 level (704 * 576) image.

This is done, for example, as follows. FCI
When reconstructing an image of one frame of CCIR601 from an F (352 * 288) image, the same pixel is read at a clock cycle of 2 dots and the same line is read at a cycle of 2 lines. FIG. 31 schematically shows the positional relationship of pixels in the conversion at this time.

From QCIF (176 * 144) image to CC
When reconstructing an image of one frame of IR601, the same pixel is read at a 4-dot clock cycle and the same pixel is read at a 4-line cycle. FIG. 32 schematically shows the positional relationship of pixels in the conversion at this time. The D / A converter 18 in FIG. 26 analogizes these reconstructed images and outputs an analog video signal.

Here, the internal operation of the decoding means 12 will be described with reference to FIG. The quantization index q of the transform coefficient is input to the inverse quantizer 72, and the quantization characteristic qz
Inverse quantization based on. The inversely quantized transform coefficient is input to the inverse transformer 73 and inverse DCT is performed to produce a video signal. When the first frame is received, that is, INTER / INT
When the identification flag p of RA indicates “INTRA”, the switch 69 is switched to 69a for intra-frame encoding.
Switch to the side. In this case, the video signal is output as it is. This video signal is equivalent to one FCIF or QCIF
One image is stored in the motion compensation image memory 74.

When the next frame is received, that is, INTER /
When the INTRA identification flag p indicates "INTER", interframe coding is performed. In this case, the quantization index q of the transform coefficient is input to the inverse quantizer 72 and inversely quantized based on the inverse quantization characteristic qz. Then, the inversely quantized transform coefficient is input to the inverse transformer 73 and inverse DCT is performed to create a differential video signal.

On the other hand, the received motion vector v is given to the motion compensation image memory 74 to calculate the motion compensation image data. The motion compensation image data is filtered based on the on / off signal f of the in-loop filter 75. Then, the motion compensation image data is given to the adder 68 via the switch 70 and added to the differential video signal. The video signal thus reproduced is equivalent to one FCIF or QCIF.
One image is stored in the motion compensation image memory 74.

Thereafter, this operation is repeated based on the identification flag of INTER / INTRA. Depending on the image transmitting method and the image receiving method, FCIF or QCIF
The image is transmitted and received after being converted into.

[0023]

In such a conventional image communication method, the resolution is not changed with respect to the movement of the image. Therefore, when a large motion occurs in the image, the processing time for processing a large number of blocks increases. Therefore, it is necessary to coarsen the quantization step in the DCT transform of the encoding process. In this case, as a result, the frame rate must be reduced, and block distortion is noticeable.

The present invention has been made in view of such conventional problems, and does not reduce the number of coded frames even in the case of an image having a lot of motion, and in the case of a still image, an image of higher resolution is obtained. It is an object of the present invention to realize an image communication device and an image receiving device, and an image transmitting method and an image receiving method capable of transmitting and receiving the.

[0025]

According to a first aspect of the present invention, there is provided an image transmitting apparatus for compressing and transmitting an image of horizontal n dots and vertical m lines, and transmitting the analog video signal from a digital video signal. An A / D converter for converting to
A first switch for alternately switching the digital video signal of the A / D converter in frame units; first and second image frame memories for alternately storing the digital video signal switched by the first switch; A second switch for alternately switching the digital video signals read from the first and second image frame memories in opposite phase to the first switch for output and setting a target number of encoded frames per unit time 16 pieces of QCIF images (QCI) in which pixels are thinned out at intervals of 4 dots in the horizontal direction and 4 lines in the vertical direction with respect to the target frame rate setting means for switching and the digital video signal read by switching with the second switch.
QCI for compressing and encoding each of F0 to QCIF15)
A switching control signal is applied to the F-encoding means and the first and second switches, and an image of one frame is subjected to QCI.
A read control signal is output to the first and second image frame memories so as to be divided into F0 to QCIF15, and the target number of coded frames per unit time output from the target frame rate setting means, and the QCIF coding means. From the encoding time of one QCIF image processed in step 1, whether to output all 16 sheets of QCIF0 to QCIF15 per frame, output four sheets or one sheet, to the QCIF encoding means. It is characterized by further comprising: first read / write control means for instructing, and multiplexing means for multiplexing and outputting data including audio data to the video data output from the QCIF encoding means. It is a thing. And are provided.

The invention of claim 3 of the present application is an image transmission method for transmitting an image by compressing and encoding an image of horizontal n dots and vertical m lines, and converting an analog video signal into a digital video signal, The digital video signal is alternately switched on a frame-by-frame basis, and the switched digital video signal is stored alternately in the first and second image frame memories and held in the first and second image frame memories. The digital video signal is alternately switched and output, the target number of encoded frames per unit time is set, and the interval of horizontal 4 dots and vertical 4 lines for the switched digital video signal of horizontal n dots and vertical m lines. The QCIF image (QCIF0 to QCIF15) in which pixels are thinned out is compressed and encoded, and the image of one frame is QCIF0 to QCIF.
A read control signal is generated so as to be divided into fifteen, and QCIF0 to QCI per frame are calculated from the target number of coded frames per unit time and the coding time of one QCIF image.
Whether to output all 16 sheets of F15 or 4 sheets,
It is characterized by instructing whether to output one sheet, and multiplexing and outputting data including QCIF-coded video data and audio data.

According to the image transmitting apparatus and the image transmitting method of the first or third aspect, normally, in order to transmit an image having the resolution of FCIF, QCIF is transmitted at a rate of 4 sheets / frame. When the motion of the image is large, the image is dropped to an image having the resolution of QCIF, and the QCIF is transmitted at the rate of 1 sheet / frame. On the contrary, when sending a still image, CC
Increased the image to the resolution of IR601 and 16 QCIF /
Send at a frame rate. Therefore, when a large amount of motion occurs, it is not necessary to process a large number of blocks or roughen the DCT quantization step, and it is possible to transmit an image with less block distortion without lowering the frame rate.

According to the image receiving apparatus and the receiving method of the second or fourth aspect, when QCIF is received as an image having an FCIF resolution at a rate of 4 sheets / frame,
The pixels of one dot interval which are thinned out in the horizontal direction and the vertical direction are respectively interpolated to compose one frame image. When the movement of the image is large, the QCIF is received at a rate of 1 sheet / frame. In this case, each pixel at 2-dot intervals is interpolated to compose one frame image. When reproducing an almost still image, QCIF is received at a rate of 16 sheets / frame, and images having a resolution of CCIR 601 are combined.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION An image communication method and an image communication apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the image transmitting apparatus of this embodiment, and FIG. 2 is a block diagram showing the configuration of the image receiving apparatus of this embodiment.

As shown in FIG. 1, the image transmitting apparatus comprises an A / D converter 19, switches 20, 24, read / write control means 21, image frame memories 22, 23, QCIF encoding means 25, target frame rate setting means. 26, and a multiplexing means 27, the output signal of which is a communication network 28.
Is output to the receiving device via.

The A / D converter 19 is a circuit which receives an analog video signal, converts it into a digital video signal, and outputs it to the first switch 20. The switch 20 is a switch that switches an input signal in units of frames and outputs it to the first image frame memory 22 or the second image frame memory 23. The image frame memories 22 and 23 are frame memories for temporarily storing one frame of digital video signal, and the write / read clock thereof is given by the first read / write control means 21. Switch 2
Reference numeral 4 denotes a switch which is switched in a phase opposite to that of the switch 20 by a switching control signal of the read / write control means 21 and alternately takes out the video signals of the image frame memories 22 and 23 in frame units.

The read / write control means 21 controls the read clocks of the image frame memories 22 and 23 to convert the CCIR 601 images held in these memories into QCIF level images, and the QCIF encoding means. 25. The QCIF encoding means 25 is means for compression-encoding a QCIF level image by intra-frame encoding or inter-frame encoding. QCIF
The encoding means 25 compresses and encodes a necessary QCIF block among QCIF0 to QCIF15, and particularly QCI.
The processing time of the compression encoding required for F0 is output to the read / write control means 21.

The target frame rate setting means 26 is a means for setting the frame rate of an image to be transmitted, and the value is given to the read / write control means 21. The multiplexing unit 27 is a unit that multiplexes the video data output from the QCIF encoding unit 25, the audio data and other data, and sends the multiplexed data to the communication network 28.

As shown in FIG. 2, the image receiving device is a communication network 2.
It receives the signal from 9 and includes a demultiplexing unit 30, a QCIF decoding unit 31, switches 32 and 36, a read / write control unit 33, image frame memories 34 and 35, and a D / A converter 37. .

The separating means 30 is means for separating the data received via the communication network 29 into video data, audio data and other data, and supplying the video data to the QCIF decoding means 31. QCIF decoding means 31
Is the encoding means 2 of FIG.
This is a means for performing decompression decoding in the reverse procedure of 5. In addition, QCI
The F decoding means 31 outputs a QCIF position signal described later to the second read / write control means 33.

The third switch 32 is switched by the switching control signal of the read / write control means 33, and the digital video signal of the QCIF unit decoded by the QCIF decoding means 31 is switched frame by frame, and the third image frame is displayed. It is a switch provided to the memory 34 or the fourth image frame memory 34. The read / write control means 33 is a means for converting a video signal of QCIF level into a video signal of one frame by controlling a write clock to the image frame memories 34 and 35.

The image frame memories 34 and 35 are frame memories for temporarily storing one frame of digital video signal, and the read clock thereof is the read / write control means 3.
Given by 3. The fourth switch 36 is a switch that is switched in a phase opposite to that of the third switch 32, and alternately takes out the video signals of the image frame memories 34 and 35 in frame units. The D / A converter 37 converts the digital video signal output from the switch 36 into an analog video signal.

FIG. 3 is a block diagram showing the configuration of the QCIF encoding means 25, which includes a subtractor 38 and switches 39, 4
0, an adder 41, an encoding control means 42, a converter 43, a quantizer 44, an inverse quantizer 45, an inverse converter 46, an in-loop filter 47, and a motion compensation image memory 48.

As shown in FIG. 3, the video signal switched by the switch 24 of FIG.
9 is input to the motion compensation image memory 48. The subtractor 38 is a circuit for subtracting the video signal output from the in-loop filter 47 from the video signal input from the switch 24, and the difference data is given to the other input end 39b of the switch 39. The switch 40 has two inputs 40a and 40b and is switched in synchronization with the switch 39 by a switching control signal of the encoding control means 42.

The converter 43 is a circuit for performing DCT on either the video signal in the frame switched by the switch 39 or the video signal between the frames. The quantizer 44 is a circuit for quantizing the data of the converter 43, and outputs a quantization index q. The inverse quantizer 45 is a circuit that inversely quantizes the generated quantization index q. Inverter 46
Is a circuit for inversely transforming the data inversely quantized by the inverse quantizer 45, and its output is given to the adder 41.

The adder 41 is a circuit for adding difference data to the video signal of the previous frame obtained via the switch 40, and the output thereof is given to the motion compensation image memory 48. The motion compensation image memory 48 is QCIF0 to QCI.
The video signal up to F15 is held, and the motion of the image of each block with respect to the image of the previous frame is output as a motion vector v. In-loop filter 47
Is a filter for removing distortion in the image held in the motion compensation image memory 48 by smoothing,
An on / off signal f indicating whether or not the operation is performed is output.

The encoding control means 42 includes switches 39, 4
The switching control signal is output to 0, the quantization characteristic qz is instructed to the quantizer 44, and INTER / INTRA is output.
The identification flag p and the transmission / non-transmission identification flag t are generated.

On the other hand, FIG. 4 is a block diagram showing the configuration of the QCIF decoding means 31, which is an adder 49 and a switch 5.
0, 51, decoding control means 52, inverse quantizer 53, inverse converter 54, motion compensation image memory 55, and in-loop filter 55.

As shown in FIG. 4, the decoding control means 52 is
Identification flag p of INTER / INTRA, QCIF position signal qn, transmission / non-transmission identification flag t, quantization characteristic qz
Is a means for outputting a switching control signal to the switches 50 and 51 and performing control for decoding into a video signal in QCIF units. The inverse quantizer 53 is a circuit that performs inverse quantization based on the quantization characteristic qz when the quantization index q is given. The inverse transformer 54 is the inverse quantizer 53.
Is a circuit that performs inverse DCT when the data of (1) is input.

The adder 49 is a circuit for adding the difference data to the video signal of the previous frame input via the switch 51 when the difference data is output from the inverse converter 54. The switch 50 is a switch for switching between a video signal for inter-frame coding and a video signal for intra-frame coding and outputting the video signal to the motion compensation image memory 55. The motion compensation image memory 55 is a memory that stores the video signals of QCIF0 to QCIF15 based on the motion vector v. In-loop filter 5
Reference numeral 6 is a filter for smoothing distortion in the image held in the motion compensation image memory 55 based on the on / off signal f, and its output is the input terminal 5 of the switch 51.
Given to 1b.

Next, the image transmission method of this embodiment will be described with reference to FIG.
This will be described with reference to FIG. 5 and 6 show CCIR60
It is explanatory drawing which shows conversion from 1 image to FCIF image (FCIF0-FCIF3). Figure 7 shows the FCIF image (FC
IF0) to QCIF image (QCIF0 to QCIF3)
It is explanatory drawing which shows the conversion to. Figure 8 shows the FCIF image (F
CIF1) to QCIF image (QCIF4 to QCIF
It is explanatory drawing which shows the conversion to 7). FIG. 9 shows an FCIF image (FCIF2) to a QCIF image (QCIF8 to QCI).
It is explanatory drawing which shows the conversion to F11). And FIG. 10 shows the FCIF image (FCIF3) to the QCIF image (QCI
It is an explanatory view showing conversion to F12-QCIF15).

As shown in FIG. 1, the video signal is input to the A / D converter 19 and digitized. Then, the digital image signal is switched by the switch 20 and written in the frame memory 22 or 23. The reading / writing control means 21 selects which one to write. This is to prevent reading and writing from accessing the same memory at the same time, and the switches 20 and 24 are alternately switched in reverse phase.

Next, the digital video signal written in the image frame memory 22 or 23 is read out and converted into the QCIF format. At this time, the read / write control means 2
1 frame of the digital video signal (C
Per CIR601), as shown in FIGS.
16 IF images (CIF0 to QCIF15), or FIG.
4 QCIF images (QCIF0 to QCIF
3) or convert it into one QCIF image (QCIF0) and output it.

The conversion at this time will be described. The image of CCIR601 shown in FIG. 5 is 16 dots * 1 as an example.
Although only two lines are shown, the actual number of pixels in one frame is much larger than this, and it is assumed that the same conversion as in this figure is performed for those pixels. First, as shown in FIG. 5, one CCIR601 image is converted into four FCIF images (FCIF0 to FCIF3) shown in FIG.
Then, as shown in FIG. 7, the FCIF image (FCIF0) is converted into four QCIF images (QCIF0 to QCIF3). Further, as shown in FIG. 8, the FCIF image (FCIF
1) 4 QCIF images (QCIF4 to QCIF7)
Convert to As shown in FIG. 9, the FCIF image (FCI
F2) includes four QCIFs (QCIF8 to QCIF11)
Convert to Further, as shown in FIG. 10, the FCIF image (F
CIF3) with four QCIF images (QCIF12-QC
IF15).

When 16 QCIFs are output per one frame (CCIR 601) as digital video data, QCIF0 to QCIF15 are output and QCI is output.
When outputting four F images, QCIF0 to QCIF3 are output, and when outputting one QCIF, QCIF0 is output.

The converted QCIF video signal is passed through the switch 24 of FIG. 1 and input to the QCIF encoding means 25 for encoding. At this time, the QCIF encoding means 25 encodes one frame in order from QCIF0. When QCIF0 is completely encoded, QC
Read / write control means 2 is used to determine the time taken for encoding IF0.
Notify 1 1 is set in the target frame rate setting means 26.
The target number of encoded frames per second is set, and this value is output to the read / write control means 21.

The read / write control means 21 calculates the permissible time for the encoding process per frame from the target number of encoded frames per second input from the target frame rate setting means 26. And QCIF encoding means 25
16 QCIF images per frame (QCIF0 to QCIF0)
15) Decide whether to output, output four (QCIF0 to QCIF3), or output one (QCIF0).
Here, since QCIF0 to QCIF15 are images that are almost the same in content, it is assumed that the coding processing time of each QCIF is the same. The number of output QCIF images at this time is determined based on the following three inequalities.

QCIF0 coding time ≦ (1 / target frame rate) ÷
16 ... Outputs 16 QCFI images. (1 / target frame rate) ÷ 16 <QCIF0 coding time ≦ (1 / target frame rate) ÷ 4 ... Four QCFI images are output. (1 / target frame rate) ÷ 4 ≦ QCIF0 encoding time ... One QCFI image is output.

In this way, the QCIF encoding means 25
Has 16 QCIF images per frame (QCIF
0 to QCIF15) or 4 sheets (QCIF0 to QCIF
3), or one image (QCIF0) is encoded.

Here, the internal operation of the QCIF encoding means 25 will be described with reference to FIG. At the time of encoding the first frame, the switch 39 is switched to the side of 39a for performing the intra-frame encoding, and the video signal is converted into the converter 43.
To perform DCT. The transform coefficient DCTed here is input to the quantizer 44 and quantized, and the quantization index q and the quantization characteristic qz of the transform coefficient are transmitted.

Further, the quantization index q is the inverse quantizer 4
Inverse quantization is performed in 5, and inverse conversion is performed in the inverse converter 46. Here, the video signal subjected to the inverse DCT is passed through the adder 41 as it is at the time of encoding the first frame, and is sent to the address of the corresponding QCIF number in the motion compensation image memory 48 as QCIF.
Store one sheet. At this time, the identification flag p of INTER / INTRA is set to "INTRA", and the transmission / non-transmission identification flag t is set to "transmission". The motion vector v
And the in-loop filter f does not transmit.

This operation is performed by the read / write control means 21.
16 frames of QCIF images (QCIF0 to QCIF1) per frame (CCIR601)
5), or 4 QCIF images (QCIF0 to QCIF
3), or for one QCIF image (QCIF0), which indicates this QCIF position number qn (1 to 15)
Also send.

When the video signal of the next frame is input to the QCIF encoding means 25, when performing inter-frame encoding, the video signal is input to the motion compensation image memory 48 to search for a motion vector. , Its motion vector v
Send At this time, the motion vector search is performed using the QC
This is performed between the QCIF images with the IF numbers, but if the QCIF image with the QCIF number is not coded in the previous frame, the motion vector search is performed with the QCIF image with the most approximate QCIF number. .

For example, in FIGS. 5 to 10, QCIF
If 4, QCIF8 and QCIF12 are not encoded in the previous frame, a search is performed between QCIF0 and QCIF0. QCIF5, QCIF9, QCIF
If 13 has not been coded in the previous frame, a search is performed with QCIF1. Incidentally, Q
If CIF1 is also not encoded, QCIF
Search with 0. QCIF6, QCIF10, QCIF
If these are not coded in the previous frame for 14, the search with QCIF2 is performed. Incidentally, Q
If CIF2 is not encoded, QCIF
Search with 0. QCIF7, QCIF11, QCIF
If these are not coded in the previous frame for 15, search is performed with QCIF3. Incidentally, Q
If CIF3 is not encoded, QCIF
Search with 0.

Next, the image based on the motion vector is filtered by the in-loop filter 47. Then, the subtracter 38 subtracts from the video signal, and the in-loop filter 47
The on / off signal f of is output. Then switch 3 to 3
Switching to the 9b side, the differential video data is input to the converter 43. The data subjected to the DCT processing here is the quantizer 44.
Quantize with. And the quantization index q of the transform coefficient
And the quantization characteristic qz are transmitted. At this time, INTER /
The identification flag p of INTRA is set to "INTER", and the transmission / non-transmission identification flag t transmits "transmission" in the case of transmission and "non-transmission" in the case of non-transmission.

On the other hand, the quantization index q of the transform coefficient is inversely quantized by the inverse quantizer 45, and the inverse DCT is performed by the inverse transformer 46.
I do. The differential video data subjected to the inverse DCT is given to the adder 41, and is added to the image based on the motion vector. Then, one QCIF image is stored in the address of the corresponding QCIF number in the motion compensation image memory 48.

This operation is performed by the read / write control means 2
16 frames of QCIF images (QCIF0 to QCIF1) per frame (CCIR601) based on control of 1
5), or 4 QCIF images (QCIF0 to QCIF
3), or for one QCIF image (QCIF0), which indicates this QCIF position number qn (1 to 15)
Send

Thereafter, this operation is repeated while interposing intra-frame coding so that the error of the inverse DCT is not accumulated. Then, the QCIF encoding means 25 encodes QCIF and outputs video data. Next, the multiplexing means 2
Reference numeral 7 multiplexes video data, audio data, and other data. Then, this multiplexed data is output to the communication network 28.

Next, the image receiving method of this embodiment will be described. As shown in FIG. 2, the multiplexed data from the communication network 29 is supplied to the demultiplexing means 30 and separated into video data, audio data and other data. The QCIF video data is input to the QCIF decoding means 31 to be decoded.

Here, the internal operation of the QCIF decoding means 31 will be described with reference to FIG. The quantization index q of the transform coefficient is input to the inverse quantizer 53 and inversely quantized based on the quantization characteristic qz. Here, when the inversely quantized transform coefficient is given to the inverse transformer 54 and inverse DCT is performed, a video signal is obtained.

When the first frame is received, that is, INTER
When the identification flag p of / INTRA indicates "INTRA", the switch 50 is switched to the side of 50a for intra-frame coding, and the video signal is output as it is.
This video signal corresponds to the QC of the motion compensation memory 55.
One QCIF is stored in the IF number address.

Such operation is performed by the QCIF position number qn.
Based on (1 to 15), one frame (CCIR60
1) per 16 QCIF images (QCIF0 to QCI
F15), or 4 QCIF images (QCIF0 to QCI
F3) or one QCIF image (QCIF0).

When the next frame is received, that is, INTER /
When the INTRA identification flag p indicates "INTER", interframe decoding is performed. In this case, the quantization index q of the transform coefficient is input to the inverse quantizer 53 and inversely quantized based on the quantization characteristic qz. The inversely quantized transform coefficient is given to the inverse transformer 54, and inverse DCT is performed to create differential video data.

On the other hand, the received motion vector v is given to the motion compensation image memory 55 to calculate the motion compensation image data. The motion compensation at this time is QC of the QCIF number.
Although it is performed between the IF images, if the QCIF image with the QCIF number is not decoded in the previous frame, motion compensation is performed with the QCIF image with the most approximate QCIF number.

That is, in FIGS. 5 to 10, QCIF
When 4, QCIF8 and QCIF12 have not been decoded in the previous frame, motion compensation is performed with QCIF0. QCIF5, QCIF9, QC
If these are not decoded in the previous frame for IF13, motion compensation is performed with QCIF1. If QCIF1 has not been decoded, QCIF0 is used. QCIF6, QCIF10, Q
If the CIF 14 has not been decoded in the previous frame, motion compensation is performed with the QCIF 2. If QCIF2 has not been decoded, motion compensation is performed with QCIF0. QCIF7,
If QCIF11 and QCIF15 have not been decoded in the previous frame, motion compensation is performed with QCIF3. If QCIF3 has not been decoded, motion compensation is performed with QCIF0.

Then, the motion compensation image data is filtered based on the on / off signal f of the in-loop filter 56. The created image data for motion compensation is switch 5
1 is passed and input to the adder 49 to be added to the difference video data. In this way, the video signal is reproduced.

This video signal also stores one QCIF at the address of the corresponding QCIF number in the motion compensation image memory 55. Such operation is performed by the QCIF position number qn.
1 frame based on (1-15) (CCIR601)
Therefore, 16 QCIF images (QCIF0 to QCIF1
5), or 4 QCIF images (QCIF0 to QCIF
3) or for one QCIF image (QCIF0). Thereafter, this operation is repeated based on the INTER / INTRA identification flag.

The decoded video signal is written in the image frame memory 34 or 35. The read / write control means 33 selects which one to write. This is to prevent reading and writing from accessing the same memory at the same time, and switches 32 and 36 are alternately switched in reverse phase.

The QCIF decoding means 31 is the last numbered QCIF (QCIF0 or QCI which constitutes one frame).
It is detected that the decoding of F3 or QCIF15) is completed, and the read / write control means 33 is notified. When the QCIF decoding means 31 informs the read / write control means 33 that the decoding of all the QCIFs forming one frame is completed, the read / write control means 33 starts reading the video signal from the image frame memory 34 or 35. , CCIR601 image is reconstructed.

From the QCIF image at this time, CCIR60
A method of reconstructing one image will be described with reference to FIGS. 11 to 24. 11 and 12 show C in this embodiment.
When combining CIR601 level resolution, FCIF
It is explanatory drawing to the conversion from an image (FCIF0-FCIF3) to a CCIR601 image. Figure 13 shows CCIR601
When combining level resolutions, QCIF images (QCI
It is explanatory drawing which shows conversion from F0-QCIF3) to an FCIF image (FCIF0). FIG. 14 shows a QCIF image (QCIF image) when the CCIR601 level resolution is combined.
It is an explanatory view showing conversion from 4-QCIF7) to an FCIF image (FCIF1). FIG. 15 shows a QCIF image (QCIF8 when combining CCIR601 level resolution).
-QCIF11) to FCIF image (FCIF2) conversion. FIG. 16 shows a QCIF image (QCIF1) when a CCIR601 level resolution is combined.
It is an explanatory view showing conversion from 2-QCIF15) to an FCIF image (FCIF3).

FIGS. 17 and 18 show FC in this embodiment.
When combining IF level resolutions, FCIF images (F
It is explanatory drawing which shows the conversion from CIF0-FCIF3) to a CCIR601 image.

FIG. 19 is an explanatory view showing conversion from a QCIF image (QCIF0) to an FCIF image (FCIF0) when FCIF level resolutions are combined. FIG. 20 is an explanatory diagram showing conversion from a QCIF image (QCIF1) to an FCIF image (FCIF1) when FCFC level resolutions are combined. FIG. 21 shows a case in which the QCIF image (QCIF2) to the F
It is explanatory drawing which shows the conversion to a CIF image (FCIF2). FIG. 22 shows the case of combining FCIF level resolutions.
From QCIF image (QCIF3) to FCIF image (FCI
It is an explanatory view showing conversion to F3).

FIG. 23 shows an FCIF image (FCIF image) when the QCIF level resolution is combined in this embodiment.
It is an explanatory view showing conversion from 0) to a CCIR601 image. FIG. 24 shows that, in the present embodiment, when the QCIF level resolutions are combined, QCIF images (QCIF0) to F
It is explanatory drawing which shows conversion to a CIF image (FCIF0). In these figures, the CCIR601 image shows the first 16
Although only the dot * 12 line is shown, the same conversion is performed for other pixels not shown.

16 QCIF images (QCIF0 to QCI
When reconstructing one CCIR601 image from F15), conversion is performed as follows. That is, as shown in FIG.
The QCIF images (QCIF0 to QCIF3) are converted into one FCIF image (FCIF0). FIG.
4 QCIF images (QCIF4 to QCI
It is converted into one FCIF image (FCIF1) using F7). As shown in FIG. 15, four QCIF images (QC
It is converted into one FCIF image (FCIF2) using IF8 to QCIF11). As shown in FIG. 16, four QCIF images (QCIF12 to QCIF15) are used to convert into one FCIF image (FCIF3). And finally, the four FCIF images (FCIF0
~ FCIF3) and one CCIR6 shown in FIG.
01 Convert to image.

Four QCIF images (QCIF0 to QCIF
When reconstructing one CCIR601 image from 3), conversion is performed as follows. That is, as shown in FIG. 19, one QCIF image (QCIF0) is used to convert into one FCIF image (FCIF0). As shown in FIG. 20, one FCIF image (QCIF1) is used to generate one FCIF image.
Convert to image (FCIF1). 21 as shown in FIG.
One FCI using one QCIF image (QCIF2)
Convert to F image (FCIF2). As shown in FIG. 22, one FC using one QCIF image (QCIF3)
Convert to IF image (FCIF3). Finally, as shown in FIG. 12, four FCIF images (FCIF0-FIF
CIF3) is used to convert to the CCIR601 image of FIG.

When reconstructing one CCIR601 image from one QCIF image (QCIF0), conversion is performed as follows. As shown in FIG. 24, one QCIF image (QC
IF0) is used to convert into a single FCIF image (FCIF0). Then, as shown in FIG. 23, one FCIF image (FCIF0) is used for conversion into a CCIR601 image. Then, the D / A converter 37 in FIG. 2 analogizes these reconstructed images and outputs a video signal.

By the image communication method as described above, it is possible to transmit and receive an image whose image quality is not visually deteriorated at the designated frame rate. According to the present invention as described above, the resolution of an image can be controlled according to the movement of the image without changing the frame rate.

[0083]

As described in detail above, according to the present invention, by controlling the resolution according to the movement of the image,
It is not necessary to process a large number of blocks or to roughen the quantization step of DCT when a large motion occurs in the image. Therefore, it is possible to transmit and receive an image with less block distortion without lowering the frame rate of the image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image transmitting apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an image receiving apparatus in this embodiment.

FIG. 3 is a QCIF provided in the image transmitting apparatus according to the present embodiment.
It is a block diagram which shows the internal structure of an encoding means.

FIG. 4 is a QCIF provided in the image receiving apparatus according to the present embodiment.
It is a block diagram which shows the internal structure of a decoding means.

FIG. 5 is a CC in the image transmitting method according to the embodiment of the present invention.
From the IR601 image to the FCIF image (FCIF0 to FCI
It is explanatory drawing (the 1) of conversion to F3).

FIG. 6 is a CC in an image transmitting method according to an embodiment of the present invention.
From the IR601 image to the FCIF image (FCIF0 to FCI
It is explanatory drawing (the 2) of conversion to F3).

FIG. 7 illustrates an FCIF image (FCIF0) to a QCIF image (QCIF0 to QCI) in the image transmitting method according to the present exemplary embodiment.
It is explanatory drawing of the conversion to F3).

FIG. 8 illustrates an FCIF image (FCIF1) to a QCIF image (QCIF4 to QCI) in the image transmitting method according to the present exemplary embodiment.
It is explanatory drawing of the conversion to F7).

FIG. 9 illustrates an FCIF image (FCIF2) to a QCIF image (QCIF8 to QCI) in the image transmission method of the present embodiment.
It is explanatory drawing of the conversion to F11).

FIG. 10 is a diagram showing FCIF images (FCIF3) to QCIF images (QCIF12 to QIF) in the image transmitting method according to the present embodiment.
It is explanatory drawing of the conversion to CIF15).

FIG. 11 shows an image receiving method according to an embodiment of the present invention,
FCI for decoding CCIR601 level resolution
CC image 601 from F image (FCIF0 to FCIF3)
It is explanatory drawing (the 1) of conversion to an image.

FIG. 12 shows an image receiving method according to an embodiment of the present invention,
FCI for decoding CCIR601 level resolution
CC image 601 from F image (FCIF0 to FCIF3)
It is explanatory drawing (the 2) of conversion to an image.

FIG. 13 shows a CCIR in the image receiving method of the present embodiment.
QCIF images (QCIF0 to QCIF3) to FCIF images (FCI when decoding 601 level resolution)
It is explanatory drawing of the conversion to F0).

FIG. 14 shows a CCIR in the image receiving method of the present embodiment.
QCIF images (QCIF4 to QCIF7) to FCIF images (FCI when decoding 601 level resolution)
It is explanatory drawing of the conversion to F1).

FIG. 15 shows a CCIR in the image receiving method of the present embodiment.
From the QCIF image (QCIF8 to QCIF11) to the FCIF image (FC
It is explanatory drawing of the conversion to IF2).

FIG. 16 shows a CCIR in the image receiving method of the present embodiment.
QCIF images (QCIF12 to QCIF15) to FCIF images (F
It is explanatory drawing of the conversion to CIF3).

FIG. 17 is a diagram showing the FCIF in the image receiving method of the present embodiment.
FCIF image (FCI when decoding level resolution
It is explanatory drawing (the 1) of conversion from F0-FCIF3) to a CCIR601 image.

FIG. 18 is a diagram showing an FCIF in the image receiving method of the present embodiment.
FCIF image (FCI when decoding level resolution
It is explanatory drawing (the 2) of the conversion from F0-FCIF3) into a CCIR601 image.

FIG. 19 shows an image receiving method according to an embodiment of the present invention,
It is explanatory drawing of the conversion from a QCIF image (QCIF0) to an FCIF image (FCIF0) at the time of decoding the resolution of an FCIF level.

FIG. 20 is a diagram showing an FCIF in the image receiving method of the present embodiment.
QCIF image (QCI
It is explanatory drawing of conversion from F1) to an FCIF image (FCIF1).

FIG. 21 shows the FCIF in the image receiving method of the present embodiment.
QCIF image (QCI
It is explanatory drawing of conversion from F2) to an FCIF image (FCIF2).

FIG. 22 is a diagram showing the FCIF in the image receiving method of the present embodiment.
QCIF image (QCI
It is explanatory drawing of conversion from F3) to an FCIF image (FCIF3).

FIG. 23 is an image receiving method according to an embodiment of the present invention,
It is explanatory drawing of the conversion from the FCIF image (FCIF0) to the CCIR601 image when decoding the resolution of a QCIF level.

FIG. 24 is a diagram showing an image receiving method according to an embodiment of the present invention,
It is explanatory drawing of the conversion from a QCIF image (QCIF0) to an FCIF image (FCIF0) when decoding the resolution of a QCIF level.

FIG. 25 is a block diagram showing a configuration of a conventional image transmission device.

FIG. 26 is a block diagram showing a configuration of a conventional image receiving device.

FIG. 27 is a block diagram showing an internal configuration of an encoding means provided in a conventional image transmitting apparatus.

FIG. 28 is a block diagram showing an internal configuration of a decoding means provided in a conventional image receiving apparatus.

FIG. 29 is a CCIR 601 in a conventional image transmission method.
It is explanatory drawing of the conversion from a level image to an FCIF level image.

FIG. 30: CCIR 601 in a conventional image transmission method
It is explanatory drawing of the conversion from a level image to a QCIF level image.

FIG. 31 is an explanatory diagram of conversion from an FCIF level image to a CCIR 601 level image in the conventional image receiving method.

FIG. 32 is an explanatory diagram of conversion from a QCIF level image to a CCIR601 level image in the conventional image receiving method.

[Explanation of symbols]

19 A / D converter 20, 24, 32, 36, 39, 40, 50, 51 Switch 21, 33 Read / write control means 22, 23, 34, 35 Image frame memory 25 QCIF encoding means 26 Target frame rate setting Means 27 Multiplexing means 28,29 Communication network 30 Separation means 31 QCIF decoding means 37 D / A converter 38 Subtractor 41,49 Adder 42 Encoding control means 43 Converter 44 Quantizer 45,53 Dequantization Device 46, 54 inverse converter 47, 57 in-loop filter 48, 55 motion compensation image memory 52 decoding control means

Claims (4)

[Claims] 1. An image transmitting device for compressing and transmitting an image of horizontal n dots and vertical m lines, which is A for converting an analog video signal into a digital video signal.
/ D converter, a first switch that alternately switches the digital video signal of the A / D converter in frame units, and a first and second switch that alternately stores the digital video signal switched by the first switch. An image frame memory, a second switch that alternately outputs the digital video signals read from the first and second image frame memories in an opposite phase to the first switch, and a target code of unit time Target frame rate setting means for setting the number of converted frames, and 16 pixels in which pixels are thinned out at intervals of 4 dots in the horizontal direction and 4 lines in the vertical direction for the digital video signal read by switching with the second switch. QCIF image (QCIF0-
QCIF coding means for compressing and coding QCIF15) respectively, and a switching control signal is given to the first and second switches, and one frame image is QCIF0 to QCIF1.
The read control signal is divided into 5 parts, and the read control signal is divided into
Image frame memory, the target number of coded frames per unit time output from the target frame rate setting means, and one Q processed by the QCIF coding means.
From the coding time of the CIF image, QCI per frame
First read / write control means for instructing the QCIF encoding means whether to output all 16 sheets of F0 to QCIF15, four sheets or one sheet, and the QCIF code An image transmitting apparatus comprising: a multiplexing unit configured to multiplex data including audio data with video data output from the multiplexing unit and output the multiplexed data. 2. An image receiving apparatus for reproducing data transmitted from the image transmitting apparatus according to claim 1 into an image, wherein the data received via the communication network is separated into data including video data and audio data. And a video data obtained by the separation means, which is 4 dots wide,
QCI, which is an image in which pixels are thinned out at intervals of four vertical lines
QCIF decoding means for decoding to F digital video signal and detecting whether all 16 QCIF0 to QCIF15 are decoded per frame, four are decoded, or one is decoded, and a digital video signal And a third image frame memory for alternately storing each of the frames, and a QCIF digital video signal output from the QCIF decoding means is switched for each frame, and stored in the third and fourth image frame memories. A third switch for outputting, a fourth switch for alternately outputting the digital video signal read from the third and fourth image frame memories in an opposite phase to the third switch, and outputting the third switch, And a switching control signal to the fourth switch, and the QCI obtained from the QCIF decoding means.
Second read / write control means for generating a read control signal in the third and fourth image frame memories for combining an image of one frame using the F digital video signal; and the fourth switch. And a D / A converter for converting a digital video signal obtained from the above into an analog video signal. 3. An image transmitting method for transmitting an image by compressing and encoding an image of horizontal n dots and vertical m lines, wherein an analog video signal is converted into a digital video signal, and the digital video signal is converted into frame units. Alternately, the switched digital video signals are stored alternately in the first and second image frame memories, and the digital video signals held in the first and second image frame memories are alternately stored. By switching and outputting, the target number of encoded frames per unit time is set, and pixels are thinned out at intervals of 4 dots in the horizontal direction and 4 lines in the vertical direction for the switched digital video signal of the horizontal n dots and vertical m lines. QCIF image (QCIF0 to QCIF15)
Is compressed and encoded, and a read control signal is generated so as to divide an image of one frame into QCIF0 to QCIF15. Of 16
Characterized by instructing whether to output all sheets, four sheets, or one sheet, and to multiplex and output data including QCIF encoded video data and audio data Image transmission method. 4. An image receiving method for reproducing an image from data transmitted by the image transmitting method according to claim 3, wherein the received data is separated into data including video data and audio data, and the video data is , A pixel is thinned out at an interval of 4 dots in the horizontal direction and 4 lines in the vertical direction, and is decoded into a digital video signal of QCIF which is an image, and QCIF0 to 0 per frame
It is detected whether 16 pictures of QCIF15 are all decoded, four pictures are decoded, or one picture is decoded, the QCIF digital video signal is switched for each frame, and the switched digital video signals are switched to the third and fourth pictures. Alternately stored in the image frame memory, read out the digital video signals held in the third and fourth image frame memories, and combine one frame image by using a plurality of QCIF digital video signals. An image receiving method characterized in that a read control signal is generated in the third and fourth image frame memories, and the digital video signals obtained from the third and fourth image frame memories are converted into analog video signals.