JPH05236464A - Mehtod for transmitting video signal - Google Patents

Abstract

PURPOSE:To prevent mismatch error from being dispersed or propagated even when executing motion compensation. CONSTITUTION:Concerning this method for transmitting video signals, the image of one frame is divided into plural blocks MBLK in a prescribed size, the image data of the divided blocks MBLK are transmitted by in-frame encoding and inter-frame encoding, and motion compensation is used as well. Two columns or two rows adjacent each other in the block MBLK are transmitted by in-frame encoding. Two columns or two rows to be transmitted by this in-frame encoding are successively changed in the row or column direction in respect to the image of one frame.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for encoding and transmitting a video signal, and more particularly to a forced refresh method therefor.

[0002]

2. Description of the Related Art In a video telephone system or the like, when a video signal is transmitted, if the video signal is transmitted as it is, the amount of data that must be transmitted becomes extremely large. Therefore, the video signal is subjected to data compression by predictive coding. A circuit or method for transmitting after that is considered.

As the predictive coding, there are intra-frame coding and inter-frame coding. In both codings, for example, a block of 16 pixels (horizontal) × 16 pixels (vertical) is processed as its processing unit. I am trying. In this case, the intra-frame encoding is to compress the video signal by utilizing the autocorrelation between the pixel data in the same frame and to transmit the compressed image data. In addition, the inter-frame coding is to obtain the difference between the pixel data of a certain frame and the pixel data of the next frame, compress the difference, and transmit the data.

Therefore, since the amount of image data obtained by inter-frame coding is smaller than that of image data obtained by intra-frame encoding, the image data obtained by intra-frame encoding is first transmitted when transmitting image data. Is sent to the transmission line as an initial value, and thereafter, image data obtained by interframe coding may be sent to the transmission line. FIG. 10 shows an example of such an encoding circuit. In this example, H.264 standardized by CCITT is used. 261 is used. That is, the analog video signal S11 to be transmitted is
The digital video signal S is supplied to the A / D converter 12
A / D converted to 12, and this signal S12 is CIF conversion circuit 1
3 and is converted into a CIF video signal S13. This signal S13 is supplied to the preprocessing filter 14 to be a band-limited video signal S14, and this signal S14 is written in the frame memory 15.

Then, the circuit 20 in the subsequent stage from the frame memory 15 receives the H.264 signal. 261 is a coder circuit. That is, at the time of intra-frame coding, the switch circuits 21 and 65 are
Is connected in the state shown in the figure, and the signal S14 written in the frame memory 15 is read out as a signal S15 (= S14),
This signal S15 is supplied to the DCT circuit (discrete cosine transform circuit) 22 through the switch circuit 21 to generate the DCT signal S22.
The signal S22 is supplied to the quantizing circuit 23 and requantized into a signal S23 having a smaller number of bits. This signal S24 is converted to S24,
The buffer memory 25 sends it to the transmission line 1 at a constant data rate.

At this time, data such as various parameters necessary for decoding the original video signal S11 from the signal S24 is transferred from the encoding control circuit 29, which will be described later, to VL.
It is supplied to the C circuit 24 and sent to the line 1 together with the signal S24.

Further, 26 is a local decoder, which supplies the signal S23 from the quantizing circuit 23 to the quantizing circuit 61, and performs a complementary process to the quantizing circuit 23 to perform the signal S61.
Is quantized into a signal S61, and the signal S61 is supplied to the inverse DCT circuit 62.
Converted to 62. In this case, the circuits 61 and 62 are the same as the circuit 2
Since the processing complementary to that of signals 3 and 22 is performed, the signal S61 is the signal S22.
And the signal S62 is a signal obtained by restoring the signal S15.

Then, the signal S62 is supplied to the adder circuit 63, and the signal S65 from the switch circuit 65 is supplied to the adder circuit 63 so that both signals S62 are supplied from the adder circuit 63.
Since the addition signal S63 of 62 and S65, in this case S65 = 0, S63 = S62 is taken out and this signal S63 is written in the frame memory 64. The signal S63 becomes a signal corresponding to the signal S15 one frame before depending on the processing time of the circuits 22, 23, 61 and 62. Also, in this case, S
Since 63 = S62, the signal S63 is a video signal obtained by restoring the signal S15.

Therefore, the intra-frame coded video signal S24 is sent to the line 1, and the video signal S63 of the memory 64 is equal to the signal S24.
The video signal S15 restored from

On the other hand, at the time of interframe coding, the switch circuits 21 and 65 are connected in a state opposite to that shown in the figure, and the signal S15 from the frame memory 15 is supplied to the subtraction circuit 27 and is written in the frame memory 64. Signal S63
Is read out as a signal S64 (= S63), and this signal S64
Is supplied to the subtraction circuit 27, and the signal S is output from the subtraction circuit 27.
A signal S27 which is the difference between 15 and the signal S64 is taken out. This signal S27 is sent to the DCT through the switch circuit 21.
It is supplied to the circuit 22.

Therefore, the signal S27 is the video signal S11 of a certain frame and the video signal S11 of the previous frame.
And the difference signal S27 is DCT processed and requantized to be taken out as a signal S23. The signal S23 is converted into a signal S24 and sent to the line 1.

At this time, the signal S64 read from the memory 64 is added through the switch circuit 65 to the adder circuit 63.
Is supplied to the addition circuit 63,
The signal S23 is converted into the signal S62 by the quantization circuit 61 and the inverse DCT circuit 62 and supplied. Therefore, the signal S63 supplied to the memory 64 is the signal S one frame before.
The video signal S15 is restored from 64 and the signal S62.

Further, motion compensation is performed by the motion vector detection circuit 28. That is, for example, in a scene in which a ball flies in a stationary background, the image data of the ball needs to be transmitted in the first frame, but from the second frame, the ball (moving object) is transmitted. ), The data indicating the movement amount (vector) of
You can express the movement of the ball. Therefore, a motion vector detecting circuit 28 is provided, and the signal S15 from the memory 15 and the signal S64 from the memory 64 are supplied to the detecting circuit 28 to detect a moving object between two consecutive frames. The output corrects the signal S63 of the memory 64.

Reference numeral 29 is an encoding control circuit, which is
According to the signal S22 from the DCT circuit 22, the energy of the signal S22 at the time of intra-frame coding and the energy of the signal S22 at the time of the inter-frame coding are calculated, and the signal S22 having the smaller energy is the quantization circuit 23. Intra-frame coding or inter-frame coding is selected.

Further, the encoding control circuit 29 determines the number of requantization bits in the quantization circuit 23, that is, the number of requantization bits in the quantization circuit 23 according to the remaining amount of the signal S24 in the buffer memory 25 so that the buffer memory 25 does not overflow or underflow. Change the quantization step size.

The above is an example of predictive coding of video signals, particularly intraframe coding and interframe coding.

[0017]

By the way, in the case where the inter-frame coding and the motion compensation are performed as described above, when the communication is started or a line error occurs, a mismatch block is generated on the receiving side. However, this will affect subsequent reproduced images. For this reason, it is necessary to occasionally transmit the image data by intra-frame encoding to update the reproduced image (hereinafter, such processing is referred to as “forced refresh”).

CCITT Recommendation H.264 Forced refresh in 261 specifies sending blocks of image data by intraframe coding at least once every 132 transmission frames. And
In the case of the CIF video signal S13, this is equivalent to generating and transmitting three blocks by intraframe coding per frame. Therefore, the encoding control circuit 2
9 also selects intraframe coding so as to achieve such a ratio.

FIG. 11 shows the state of the forced refresh. That is, in this figure, the video signal is CI
Since it is an F signal, the screen has a size of 352 pixels (horizontal) x 288 pixels (vertical), and this screen has a block of 16 pixels (horizontal) x 16 pixels (vertical) (a "macro block"). It is divided into 22 blocks (horizontal) x 18 blocks (vertical) in units of MBLK.

The forced refresh starts from the rightmost blocks MBLK to MBLK (shown by diagonal lines) on the 0th row, the 6th row, and the 12th row. Move in the direction. When the forced refresh reaches the leftmost block, 1
From the rightmost block of the line below, it is performed to the left again,
After that, it is similarly refreshed.

Therefore, in the entire screen, the forced refresh cycle is completed in 22 blocks × 18 blocks / 3 blocks = 132 frames = 4.4 seconds. Therefore, when starting transmission of image data or when an error occurs on the playback screen, wait about 4.4 seconds to obtain a normal playback screen.

However, since the motion compensation has not been considered in the forcible refresh so far, as described below, when the motion compensation is performed, a mismatch error remains, and in the worst case, the influence of the error is exerted. It may remain forever.

That is, in FIG. 12A, it is assumed that each area surrounded by a thin line indicates a block MBLK, and of these blocks MBLK, the block E does not include a mismatch error due to the forced refresh. Further, it is assumed that blocks A, B, D, G, and H on the left side, which is the direction of the forced refresh, include mismatch errors (shown by hatching).

Then, after one frame, the forced refresh advances one block to the left as indicated by the arrow AW, so that the block D is forcibly refreshed as shown in FIG. 12B and no mismatch error is included.

However, in this frame, block E
When motion compensation is applied to the block A, and there is a motion vector from the block ABDE to the block E including parts of blocks A, B, D, and E as shown in FIG. The block ABDE is added to E. Then, at this time, the block ABDE contains a mismatch error. Therefore, in block E,
The mismatch error is included again.

In the system for performing motion compensation as described above, a mismatch error may be diffused or propagated to an adjacent block MBLK due to the effect of motion compensation. And
In such a case, it takes time to completely remove the mismatch error, and in the worst case, the mismatch error may not be removed.

Further, as described above, when the image data transmission is started, in the forced refresh, it takes about 4.4 seconds until a normal reproduction screen is obtained, so special processing is required.

The present invention is intended to provide a forced refresh method which solves the above problems.

[0029]

Therefore, in the present invention, when the reference numerals of the respective parts correspond to the embodiments described later, an image of one frame is divided into a plurality of blocks of a predetermined size.
In the video signal transmission method in which the image data of the divided block MBLK is transmitted by intra-frame encoding and inter-frame encoding, and the motion compensation is also used, the blocks MBLK are adjacent to each other.
The columns or 2 rows are transmitted by intra-frame encoding, and the 2 columns or 2 rows transmitted by the intra-frame encoding are sequentially changed in the row direction or the column direction with respect to the image of one frame. It was done.

[0030]

When a block is motion compensated, it is updated by intraframe coding. Therefore, even if there is a mismatch error, it will not be diffused or propagated.

[0031]

EXAMPLE In the example shown in FIG. 1, the motion compensation recommended by CCITT is ± 15 pixels. Then, in this case, the forced refresh is performed for each block of two columns adjacent to each other, and as shown by an arrow AW, the column of the blocks MBLK to be forcibly refreshed is
Each frame is moved to the left in the figure in units of one column.

According to this structure, the entire screen is scanned every 22 frames by the intra-frame coding block MBLK, so that even if there is a mismatch error, it is removed.

That is, in FIG. 2, each area surrounded by a thin line shows a block MBLK of 16 pixels × 16 pixels, and among these blocks MBLK, blocks B, E, H,
It is assumed that the blocks of C, F, I and the column including them do not include a mismatch error due to the forced refresh.
The direction of the forced refresh is in the left direction in the figure, as indicated by the arrow AW.
It is assumed that blocks A, D and G on the left side of E and H and blocks in a column including them include a mismatch error (shown by hatching).

If the motion compensation is applied to the block E after one frame, in FIG. 3, the area (cross hatch) EX of 15 pixels in the right direction from the left side of the block E is shown in FIG. The area ATOI within 15 pixels from each side of the block E can be the compensation source of the motion compensation. At this time, in the area ATOI, the area OKMB having 16 pixels to the right from the right side of the block E does not include a mismatch error.

However, of the area ATOI, the area XERR of 15 pixels to the left from the left side of the block E contains a mismatch error. That is, the pixels in this area XERR are
If transferred to block E by motion compensation, block E
The mismatch error will be diffused or propagated.

The same applies to the block E as to the blocks B and H above and below it, and further to all the blocks in the column including the blocks B, E and H.

Therefore, when the mismatch error is diffused or propagated after one frame due to the motion compensation, the diffusion range of the mismatch error is the shaded area in FIG. 4, that is, the column including the blocks B, E, and H. There is a band-shaped area of 15 pixels from the left side to the right.

However, according to the present invention, the forced refresh is performed for every two columns of the block MBLK. Therefore, after one frame, as shown in FIG.
And the column containing B, E, H is forced refreshed, so even if the mismatch error of the column containing blocks A, D, G attempts to spread to the column containing blocks B, E, H, it will be forced refreshed. Will result in correct image data.

Since the above is true for all columns of blocks, the mismatch error does not spread or propagate even if motion compensation is performed.

Thus, according to the present invention, the diffusion or propagation of the mismatch error can be prevented even if the motion compensation is performed. Moreover, even if an error occurs in the reproduced image due to a line error or the like, the error is corrected 22 frames later, that is, within 1 second. Therefore, when the transmission of the image data is started, or when an error occurs in the reproduction screen, the normal reproduction screen can be obtained within 1 second, even when the transmission of the image data is started. , Does not require special processing.

In the example shown in FIGS. 6 and 7, the motion compensation is ± 7 pixels. Also in this case, the forced refresh is performed for blocks of two columns adjacent to each other. However, in this case, this forced refresh is
The scanning is performed every other frame in the horizontal direction in units of one column.

In this way, the entire screen is scanned every 44 frames by the intra-frame coding block MBLK, and any mismatch error is removed.

That is, also in FIG. 6, similar to the above,
Of the blocks MBLK, blocks B, E, H, C, F, I
And a column including these does not include a mismatch error due to forced refresh. Further, the progress direction of the forced refresh is the left direction in the figure as indicated by the arrow AW, and the blocks A, D and G on the left side of the blocks B, E and H and the column including them are mismatch error (shaded lines). (Shown) is included.

If the motion compensation is applied to the block E after one frame, in FIG. 6, the area (cross hatching) EX of 7 pixels in the right direction from the left side of the block E is shown in FIG. The area ATOI, which is an application destination area (rewritable area) and is within 7 pixels from each side of the block E, can be a compensation source for the motion compensation.
At this time, in the area ATOI, the area OKMB having 16 pixels to the right from the right side of the block E does not include a mismatch error.

However, in the area ATOI, the area XERR of 7 pixels to the left from the left side of the block E contains a mismatch error. That is, the pixels in this area XERR are
If transferred to block E by motion compensation, block E
The mismatch error will be diffused or propagated.

The same applies to the block E as to the blocks B and H above and below it, and further to all the blocks in the column including the blocks B, E and H.

Therefore, when the mismatch error diffuses or propagates one frame after the motion compensation, the diffusion range of the mismatch error is the column including the blocks B, E, and H in FIG. It becomes a strip-shaped area of 7 pixels in the right direction (a strip-shaped area in the vertical direction including the area EX).

Further, if motion compensation is applied to the block E after the next frame, that is, two frames later, in FIG. 7, an area of 14 pixels in the right direction from the left side of the block E (cross hatch). EX shown is an area to which it is applied (rewritten area), and an area ATOI within 7 pixels from each side of the block E can be a compensation source for the motion compensation. And at this time, the area
Of the ATOI, the area OKMB of 9 pixels to the right from the right side of the block E does not include a mismatch error. But,
Of the area ATOI, the area XERR of 7 pixels to the left from the left side of the block E includes a mismatch error. That is, if the pixels in the area XERR are transferred to the block E by motion compensation, the mismatch error is diffused or propagated in the block E.

The same applies to the block E as to the blocks B and H above and below it, and further to all the blocks in the column including the blocks B, E and H.

Therefore, when the mismatch error diffuses or propagates one frame after the motion compensation, the diffusion range of the mismatch error is the column including the blocks B, E, and H in FIG. To the right
It becomes a strip-shaped area of 14 pixels.

That is, in the case of this example, FIGS.
As shown in, even if the same motion compensation is repeated over two frames, the horizontal diffusion of the mismatch error is less than one block.

Further, according to the present invention, the forced refresh is performed every two columns of the block MBLK and every other frame. Therefore, after two frames, the block A,
The column including D, G and B, E, H is forcibly refreshed, so that even if the mismatch error of the column including blocks A, D, G is attempted to diffuse to the column including blocks B, E, H, Will be removed by forced refresh, and the image data will be correct.

Since the above is true for all the columns of blocks, the mismatch error does not spread or propagate even if motion compensation is performed.

Thus, also in this example, even if motion compensation is performed, it is possible to prevent the diffusion or propagation of the mismatch error. Moreover, even if an error occurs in the reproduced image due to a line error or the like, the error is corrected within 44 frames, that is, within 1.5 seconds. Therefore, when the transmission of the image data is started or when an error occurs in the reproduction screen, a normal reproduction screen can be obtained in a little less than 1 second.

FIG. 8 shows an example of the forced refresh algorithm according to the present invention. The motion compensation is ± 15. That is, in this example, block MB
Counters RCNT are prepared for each row of LK, and the initial values of these counters RCNT are, for example, as shown in FIG.
It is set equal to the number of the corresponding column of that counter RCNT. Then, the processing of FIG. 8 is executed for each frame and for each counter RCNT.

That is, in step 81, RCNT =
It is checked whether or not 21, and when RCNT = 21, the processing of the coding control circuit 29 proceeds from step 81 to step 82, and in this step 82, all the blocks MBLK of the column corresponding to the counter RCNT are forced. After being refreshed, the process then proceeds to step 83, where RCNT = 0 is set, and then this routine ends.

When RCNT ≠ 21 in step 81, the process proceeds from step 81 to step 84. In this step 84, it is checked whether RCNT = 20. If RCNT = 20, the process proceeds to step The routine proceeds from step 84 to step 85, in which all the blocks MBLK in the corresponding column of the counter RCNT are forcibly refreshed, and then the processing is step 8
In step 86, the counter RCNT is incremented by "1", and then this routine ends.

Further, in step 84, RCNT ≠ 20
If so, the process proceeds from step 84 to step 86.

Therefore, the block MBLK in the column of RCNT = 21 or 20 is forcibly refreshed in step 82 or 85. Then, at this time, since the initial values of the adjacent counters RCNT differ by "1",
The columns of the block MBLK that are forcibly refreshed are always two columns adjacent to each other.

The counter RCNT for the column forcibly refreshed is also the counter RC for the column not forcibly refreshed.
NT is also incremented by "1" for each frame in step 86, and when RCNT = 21, RC
It is incremented again from NT = 0.

Therefore, two columns are forcedly refreshed for each frame, and the columns to be forcibly refreshed are scanned one column to the left for each frame.

In the above description, the blocks MBLK to be forcibly refreshed are arranged in two columns and the two columns are scanned leftward, but they may be scanned rightward. In addition, the block MBLK to be forcibly refreshed has two horizontal rows,
These two rows may be scanned downward or upward. Furthermore, of the block MBLK of 2 columns or 2 rows,
For the block MBLK in the subsequent column or row, motion compensation may be prohibited instead of performing forced refresh.

[0063]

According to the present invention, even if motion compensation is performed, it is possible to prevent the diffusion or propagation of mismatch error. Moreover, even if an error occurs in the reproduced image due to a line error, etc., after 22 frames, that is, within 1 second,
The error is corrected. Therefore, when the transmission of the image data is started, or when an error occurs in the reproduction screen, the normal reproduction screen can be obtained within 1 second, even when the transmission of the image data is started. , Does not require special processing.

[Brief description of drawings]

FIG. 1 is a diagram for explaining forced refresh in the present invention.

FIG. 2 is a diagram for explaining a processing state according to the present invention.

FIG. 3 is a view showing a sequel to FIG. 2;

FIG. 4 is a view showing a sequel to FIG. 3;

FIG. 5 is a view showing a sequel to FIG. 4;

FIG. 6 is a diagram for explaining another processing state according to the present invention.

FIG. 7 is a view showing a sequel to FIG. 6;

FIG. 8 is a flowchart showing an example of the algorithm of the present invention.

9 is a diagram for explaining the flowchart of FIG. 8. FIG.

FIG. 10 is a system diagram for explaining an encoding circuit.

FIG. 11 is a diagram for explaining a conventional example.

FIG. 12 is a diagram for explaining the problems of the conventional example.

[Explanation of symbols]

 1 Transmission line 12 A / D converter 13 CIF conversion circuit 14 Pre-processing filter 15 Frame memory 20 Coder circuit 22 DCT circuit 23 Quantization circuit 24 VLC circuit 25 Buffer memory 26 Local decoder 28 Motion vector detection circuit 29 Coding control circuit 61 Quantum Circuit 62 Inverse DCT circuit 64 Frame memory MBLK macro block RCNT counter

Claims (1)

[Claims] 1. An image of one frame is divided into a plurality of blocks having a predetermined size, image data of the divided blocks is transmitted by intra-frame coding and inter-frame coding, and motion compensation is also used. In the method of transmitting a video signal, the two columns or two rows adjacent to each other of the block are transmitted by the intraframe coding, and the two columns or two rows transmitted by the intraframe coding are transmitted by the intraframe coding. A method of transmitting a video signal, which is sequentially changed in a row direction or a column direction with respect to the image of one frame.