JP2688294B2 - Video storage device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture storage device used in a videophone, a video conference, etc., and more particularly to an image quality control system thereof.

[0002]

2. Description of the Related Art Conventionally, techniques in such a field include:
For example, there is one described in the following literature.

Reference 1: I II TRANSACTIONS ON COMMUNICATIONS (IEEE TRANSAC)
TIONS ON COMMUNICATIONS) COM-32 [3] (19
84-3) (US) H. CHEN et al. "Scene Adaptive Coder" P.P. 225
232 Document 2; CCITT SGXV Recommendation H.264. 261 Conventionally, as a standard moving image storage device for compressing and encoding moving image data, there is one described in Document 2. Hereinafter, the configuration will be described with reference to the drawings.

FIG. 2 is a functional block diagram showing an example of the configuration of a conventional moving image storage device.

This moving picture storage device is, for example, an individual circuit,
Alternatively, it is configured by a program control of a computer and has an image input means 1 for inputting a video signal (moving image signal) Si from a television camera or the like. On the output side of the image input means 1, an intra-frame / inter-frame determination means 2, a current frame image storage means 3, and an image selection means 5 are connected. The previous frame image storage means 4 is connected to the input side of the intra-frame / inter-frame determination means 2 and the image selection means 5.

The output side of the image selecting means 5 is connected to an orthogonal transforming means 6, a quantizing means 7, and a variable length coding means 8 having a buffer. A code amount control means 9 is connected to the output side of the variable length coding means 8, and a decoded image calculation means 10 is connected to the output side of the quantization means 7. Further, the output side of the variable length coding means 8 is connected to a coded data storage means 12 via a data bus 11.

Next, the operation of FIG. 1 will be described with reference to FIG.

FIG. 3 is a diagram showing an example of block division of FIG. 2, and an example of the number of divisions of 10 is shown in this figure.
The number of divisions may be any number.

In FIG. 2, when the moving image signal Si is input to the image input means 1, the image input means 1 performs analog / digital conversion (hereinafter referred to as A / D conversion) of the moving image signal Si, and then The current frame image data S1 having a spread in the horizontal direction and the vertical direction is output to the in-frame / inter-frame determining means 2, the current frame image storing means 3, and the image selecting means 5, and as shown in FIG. The image data S1 is spatially divided into a plurality of processing blocks.

The intra-frame / inter-frame determining means 2 determines the current frame image data S1 in the order of divided processing blocks.
And the previous frame image data S4, which is a decoded image of the image of the frame immediately before the current frame image data S1 (predetermined unit time), are input, and between the current frame image data S1 and the previous frame image data S4 in the processing block. And the density variation amount in the processing block of the current frame image data S1 are compared. Then, when the correlation is large, the subsequent processing is performed using the interframe difference data of the current frame image data S1 and the previous frame image data S4 (interframe mode), and when the correlation is small, the subsequent processing is performed. It is determined which of the current frame image data S1 (intra-frame mode) is appropriate, and the intra-frame / inter-frame determination result S2 is output to the image selecting means 5. An example of this intra-frame / inter-frame determination method will be described below.

The current frame image data S1 is converted into fc (i,
j) (i: horizontal pixel address, j: vertical pixel address), and fp (i,
j), the horizontal start address of the processing block is xs, the horizontal end address is xe, and the vertical start address is y.
When s is the vertical end address and ye is ye, the intra-frame / inter-frame determining means 2 calculates A and B in the following equations (1) and (2). Then, when either of the conditions (i) and (ii) of the following expression (3) is satisfied, it is determined to be the inter-frame mode, and when neither is satisfied, the intra-frame mode is determined, and the intra-frame / inter-frame determination result S2 Is output. (I) A ≧ B (ii) A ≦ THL1 (predetermined threshold value) (3) The image selection means 5 determines the intra-frame / inter-frame determination result S.
According to 2, when the intra-frame mode is determined, the current frame image data S1 is selected, and when the inter-frame mode is determined, the inter-frame difference data between the current frame image data S1 and the previous frame image data S4 is calculated, The calculated interframe difference data is selected, and the selection result is output to the orthogonal transform means 6 as the encoded input image data S5.

The orthogonal transform means 6 performs, for example, a Descrete Cosine Transform (hereinafter referred to as DCT) operation based on the encoded input image data S5, and calculates the size of the frequency spectrum of the image data based on the following equation (4). , And outputs the orthogonal transformation data S6 to the quantizing means 7.

[0013]

(Equation 1)

The orthogonal transformation data F (u, v) output as a result of the transformation processing by the equation (4) has a relatively large value in a region where u and v are small, and a small value in a region where u and v are large. It is a general tendency to take. This is because the density distribution of the image has a large spatial redundancy, and a portion having extremely large fluctuation has a low occurrence probability.

The quantizing means 7 has the orthogonal transformation data S6 and the quantizing step width S outputted from the code amount controlling means 9.
9 is input, and the quantized data S7 is output to the variable-length coding means 8 and the decoded image calculation means 10, for example, by the method according to the following equation (5).

[0016]

(Equation 2)

However, QP; Quantization step width QF (u, v); Quantized data By the processing in the quantization means 7, the orthogonal transformation data F
The quantized output is 0 for a small component of (u, v). The greater the number of 0s, the easier the data compression.

In the variable length coding means 8, the quantized data S
7, a predetermined code is assigned so that the code length becomes relatively small with respect to the magnitude of the occurrence probability of the value, and the assigned code is used as encoded data S8a via the data bus 11 of the system. And outputs the code amount S8b assigned to one processing block to the code amount control unit 9 as well as outputting the code amount to the encoded data storage unit 12.

The code amount control means 9 inputs the code amount S8b for each processing block, determines a quantization step width S9 considered to be appropriate, and outputs it to the quantization means 7. An example of the control direction of the quantization step width S9 is described in Document 1 above. The basic idea is as follows.

The standard code amount of one processing block is BS, and the code amount S8b when actually encoded is BR.
(M) (m; block number), and the quantization step width S9 is qp (m), this quantization step width qp (m)
Is obtained from the following equation. However, α is a positive number and is proportional to the reciprocal of the buffer capacity inside the device that buffers the encoded data. As shown in equation (6), the sum of the code amount S8b is larger than the sum of the standard code amount BS. , Qp (m) is increased and the code amount of the next processing block is decreased. On the contrary, when the total sum of the code amount S8b is smaller than the total sum of the standard code amount BS, qp (m) is reduced to increase the code amount of the next processing block so that the final code amount S9 becomes appropriate. Value.

The decoded image calculation means 10 inputs the quantized data S7 of each processing block and performs the following steps (i) and (i).
The decoded image is calculated by i) and (iii), and the content of the previous frame image storage means 4 is updated as the previous frame image data S4.

(I) Inverse quantization: The processing block number is m, the quantization step width S9 used at the time of quantization is qp (m), and the quantized data S7 is QF.
^(m) (u, v) and the inverse quantized data IQF (u, v)
Is calculated from the following equation (7). IQF ^(m) (u, v) = QF ^(m) (u, v) x qp (m) (7) (ii) Inverse DCT Inverse DCT output data fi
(I, j) is obtained from the following equation (8).

[0023]

(Equation 3)

However, N, C (u), C (v); same as the definition of the equation (4) (iii) Decoded image calculation Decoded image (reproduced image) fr (i, j), previous frame image storage means Previous frame image data S4 read from
Be fm (i, j). The intra-frame / inter-frame determination result S2 is (a) in the intra-frame mode fr (i, j) = fi (i, j) (9) (b) in the inter-frame mode fr (i, j) = fi (i, j) + fm (i, j) (10) In the case of (a) and (b), both of the fr (i, j) after the calculation of the decoded image fr (i, j) for one block are completed. , J)
Is written in the previous frame image storage means 4 as the previous frame image data S4.

As described above, moving image data is encoded and stored.

[0026]

However, the conventional moving image storage device has the following problems.

The upper limit of the code amount S8b in the coded data S8a output from the variable length coding means 8 depends on the data transfer rate of the medium used in the coded data storage means 12. Therefore, in a scene with a lot of movement, it is necessary to increase the quantization step width S9 output from the code amount control means 9 in order to prevent the code amount S8b from increasing. However, when this kind of processing is applied,
There is a problem that the image quality is deteriorated. Therefore, even if the other performance (for example, frame period, spatial resolution, etc.) is reduced, the flexibility of being able to immediately deal with the demand for converting the moving image data into a high-quality database is provided. Was lacking in.

The present invention provides a moving image storage apparatus which solves the problem of the above-mentioned conventional technique that the image quality is deteriorated in a scene in which the motion is intense.

[0029]

In order to solve the above problems, the present invention compresses moving image data input for each frame by image input means using a quantization step size to obtain encoded data and its code. Means for generating a quantity and outputting the coded data to a data bus for storage, code quantity control means for inputting the code quantity and determining the quantization step width, and decoding the coded data A moving image storage apparatus including a decoded image calculation unit that calculates a decoded image includes the following unit.

That is, the present invention is provided with image quality calculation means for inputting the decoded image, calculating the image quality of the decoded image, and outputting it to the outside of the apparatus, and image quality control means. The image quality control means compares the decoded image quality with an image quality command signal for the decoded image requested from outside the apparatus, and if the decoded image quality is lower than the image quality requested by the image quality command signal, It has a function of controlling the code amount control means to change the quantization step width to a small value, and controlling the image input means to thin out the frame of the image data when the code amount exceeds the upper limit value. ing.

[0031]

According to the present invention, since the moving image storage device is configured as described above, when an image command signal is input to the image control means from outside the device, the image control means calculates the image quality by the image quality calculation means. The decoded image quality is compared with the image quality command signal, and if the decoded image quality is lower than the image quality required by the image quality command signal, the code amount control means is controlled to update the quantization step width to a small value and the decoding is performed. Improve the quality of the image.

Further, in order to appropriately select a frame image in the input moving image data for increasing the code amount from the standard value and improving the image quality of the decoded image, the image input means is controlled so that the code amount is When the value exceeds the upper limit value, the image input means thins out frames to suppress the generation of the code amount. This makes it possible to flexibly encode the image quality requested from the outside. Therefore, the above problem can be solved.

[0033]

FIG. 1 is a functional block diagram of a moving image storage apparatus showing an embodiment of the present invention, in which elements common to those in FIG. 2 of the prior art are designated by common reference numerals.

In this moving image storage device, instead of the image input means 1 and the code amount control means 9 of FIG. 2, an image input means 1A and a code amount control means 9A having different configurations are provided. Furthermore, image quality calculation means 20 and image quality control means 21 are newly added.

The output of the image inputting means 1A is controlled based on the frame thinning command signal S21b from the image quality controlling means 21, and when the frame thinning command signal S21b is not supplied, the moving image signal Si is inputted as in the conventional case. It has a function of outputting the frame image data S1.
It has a function of stopping the transfer of the current frame image data S1 to the current frame image storage means 3 when the frame thinning command signal S21b is supplied.

The image quality calculation means 20 has its input side connected to the image input means 1A and the output side of the decoded image calculation means 10, and its output side connected to the input side of the image control means 21 and the data bus 11. The image quality calculation means 20 inputs the decoded image from the decoded image calculation means 10 and the current frame image data S1, calculates an image quality evaluation parameter SNR representing the image quality of the decoded image, and the decoded image quality calculation result S
It has a function of outputting 20 to the image control means 21 and the data bus 11.

The image quality control unit 21 has an input-side image quality calculating means 20, the variable length coding unit 8, the code amount control unit 9A
And the data bus 11, and its output side is connected to the input sides of the code amount control means 9A and the image input means 1A. The image control means 21 has a code amount S8b for each processing block, a decoded image quality calculation result S20, a quantization step width S9, and an image quality command signal S from the data bus 11.
It has a function of inputting 30 and outputting the quantization step width control signal S21a and the frame thinning command signal S21b for improving the image quality to the code amount control means 9A and the image input means 1A. The code amount control means 9A inputs the code amount S8b and the quantization step width control signal S21a, determines the optimum quantization step width S9, and determines it.
1a and the function of outputting to the image control means 21.

Next, the operation of FIG. 1 will be described with reference to FIG.

FIG. 4 is an operation waveform diagram of the moving image storage processing with respect to the image quality requested by the image quality command signal S30 from the outside, and shows the transition of the operation of the moving image storage processing due to the fluctuation of the image quality requested from the outside. There is.

In FIG. 1, when the frame thinning command signal S21b from the image quality control means 21 is not input to the image input means 1A, the image input means 1A inputs the moving image signal Si as in the conventional case, The current frame image data S1 is output to the in-frame / inter-frame determining means 2, the current frame image storing means 3 and the image selecting means 5, and the image data is spatially divided into a plurality of blocks.

Then, as in the conventional case, the intra-frame / inter-frame determining means 2, the current frame image storing means 3, the previous frame image storing means 4, the image selecting means 5, the orthogonal transforming means 6, the quantizing means 7, and the variable. The long coding means 8 performs coding processing, and the variable length coding means 8 outputs the coded data S8a and the code amount S8b.

In the image quality calculating means 20, the decoded image fr (i, j) and the current frame image data S1 are synchronized with the calculation of the decoded image fr (i, j) in the decoded image calculating means 10.
(Fc (i, j)) is input and, for example, the following equations (11) and (1
The image quality evaluation parameter SNR is calculated based on 2).

[0043]

(Equation 4)

Where xω is the number of pixels in the horizontal direction of the image data yω is the number of pixels in the vertical direction of the image data RMS; error between the original image and the decoded image The image quality evaluation parameter SN calculated by the image quality calculation means 20.
R is output to the image quality control means 21 and the data bus 11 of the system as the decoded image quality calculation result S20. The decoded image quality calculation result S20 output to the data bus 11 is
For example, it is displayed on a console CRT (CRT) connected to the outside of this apparatus.

For example, an image quality command signal S30 corresponding to the image quality evaluation parameter SNR is input to the image quality control means 21 via the data bus 11 from a keyboard or the like on the console. The image quality control means 21 inputs the code amount S8b for each processing block, the decoded image image quality calculation result S20, the quantization step width S9, and the image quality command signal S30, and is necessary to obtain the image quality requested from the outside. When the quantization step width control signal S21a is calculated and it is determined that it is necessary to thin out the coding in order to prevent the increase in the code amount S8b output from the variable length coding means 8,
The frame thinning command signal S21b is output to the image input means 1A.

Next, the calculation method (1) of the quantization step width control signal S21a and the calculation method (2) of the frame thinning command signal S21b in the image control means 21 will be described.

(1) Quantization step width control signal S21a
Of the decoded image quality calculation result S20 is SNROUT, the value of the image quality command signal S30 is SNRIN (having the same dimension as SNROUT), and the quantization step width control signal S21a is qpd.
Then, the qpd is obtained from the following equation (13).

[0048]

(Equation 5)

However, a and b are constants which are preset as positive numbers, and the value of a is set to around 1 and the value of b is set to 1 or more. As the quantization step width control signal qpd, a control value that lowers the quantization step width S9 is calculated when the image quality of the encoded image is lower than the image quality requested from the outside. However, when the required image quality is satisfied as in SNRIN ≦ SNROUT (14), a value (for example, 0) that does not affect the operation of the code amount control means 9A is output.

(2) Frame thinning command signal S21b
The value of the decoded image image quality calculation result S20 is lower than the image quality externally requested by the image quality command signal S30 by performing an operation to reduce the quantization step width S9 with the quantization step width control signal S21a. When the quantization step width S9 is further reduced to improve the image quality, the code amount S8b increases and the code amount data accumulating means 12
The data transfer capacity of the above may be exceeded. Image quality control means 2
In No. 1, when it is determined that the data transfer capability is exceeded, the frame thinning command signal S21b is output to the image input unit 1A.

The image input means 1A receives the frame thinning-out command signal S21b and receives the current frame image storage means 3
The transfer of the current frame image data S1 to is stopped. As a result, the code amount S8b generated by creating a frame that is not encoded in moving image data that is continuously input satisfies the data transfer rate of the encoded data accumulating means 12 as a whole. It is suppressed within the range. The conditions for generating the frame thinning command signal S21b are as follows.
Assume that the conditions of the following expressions (15), (16), and (17) are satisfied. SNRIN> SNROUT (15) _t {Σ BR (m)-(Z + 1) BS} / (z + 1)> βBMAX ^{m = tz} However, β = 0.7 to 0.8 ... (16) qp (M) <QL (17) In the equations (16) and (17), z is an appropriately determined constant and is set to be an average value for several seconds. BMAX
Is set to the maximum value of the data transfer rate of the encoded data accumulating means 12, and QL is set to a value which is 5 to 6 times the quantization step width S9 considered as the threshold value of the quantization step width S9.

When the quantization step width control signal S21a output from the image control means 21 as described above is input to the code amount control means 9A, the code amount control means 9A calculates the following equation (18). The quantization step width S9 is updated and given to the quantization means S21a and the image quality control means 21. The transition of the encoding process by the image quality command signal S30 requested from the outside is shown in FIG.

In FIG. 4, both the NMD when a normal image quality level is required (normal mode) and the HMD when a high image quality is required (high image quality mode) are compared.
In the high image quality mode HMD, the image quality can be improved by reducing the quantization step width S9 for the image data of the frame 1 and the frame 2, and the generated code amount S8.
There is no possibility that b also exceeds the upper limit.

However, in frame 3 and frame 5,
There is a possibility that the generated code amount S8b increases and exceeds the upper limit of the code amount S8b. Therefore, the frame thinning command signal S21b is used to suppress the code generation amount S8b by thinning the frames 4 and 6 by coding, and the coded frame is determined to have the image quality that meets the required level. The processing suitable for the application that requires moving image data of high image quality is performed.

In this processing, the moving image data of the frames 4 and 6 which have been thinned by encoding are interpolated by another means, for example, by referring to decoded images of preceding and following frames which are high quality encoded. , By approximately restoring the moving image data of the thinned frame,
It is possible to obtain high-quality image data.

In this embodiment, the image control means 21
The image command signal S30 is compared with the decoded image quality calculation result S20, and the quantization step width control signal S21a relating to the quantization step width determination is output, and the encoding process is performed on each frame of the moving image data that is sequentially input. By outputting the frame thinning-out command signal S21b indicating whether or not to perform the setting, it is possible to flexibly set the coding parameter suitable for the required image quality. Therefore, it is possible to realize a highly versatile and high-performance moving image storage device that can flexibly respond to external requests.

The present invention is not limited to the above embodiment,
Various modifications are possible. For example, the calculation method of the quantization step width control signal S21a is not limited to the method of the equation (13), and if the method of outputting the optimum quantization step width control signal S21a for improving the image quality is the above A method other than the example may be used.

[0058]

As described in detail above, according to the present invention, the image quality calculation means and the image quality control means are provided so that the image quality of the decoded image can be requested from the outside, and the encoding suitable for the requested image quality is performed. Since the parameters can be flexibly set, it is possible to realize a highly versatile and high-performance moving image storage device that can flexibly respond to external requests.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a moving image storage device showing an embodiment of the present invention.

FIG. 2 is a functional block diagram of a conventional moving image storage device.

FIG. 3 is a diagram showing an example of block division in FIG. 2;

FIG. 4 is an operation waveform diagram of the moving image storage processing in FIG.

[Explanation of symbols]

 1A image input means 2 intra-frame / inter-frame determination means 3 current frame image storage means 4 previous frame image storage means 5 image selection means 6 orthogonal transformation means 7 quantization means 8 variable length coding means 9A code amount control means 10 decoded image Calculation means 11 Data bus 12 Coded data storage means 20 Image quality calculation means 21 Image quality control means Si image signal S1 Current frame image data S2 In-frame / inter-frame determination result S4 Previous frame image data S5 Coded input image data S6 Orthogonal conversion data S7 Quantized data S8a Encoded data S8b Code amount S9 Quantization step width S20 Decoded image quality calculation result S21a Quantization step width control signal S21b Frame thinning command signal S30 Image quality command signal

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shigeo Kure 1-7-12 Toranomon, Minato-ku, Tokyo Oki Electric Industry Co., Ltd. (56) Reference JP-A-4-290088 (JP, A) JP Sho 62-108686 (JP, A) JP-A-3-76488 (JP, A)

Claims (1)

(57) [Claims] 1. A moving image data input for each frame by an image input means is compressed using a quantization step size,
Means for generating coded data and its code amount, outputting the coded data to a data bus for storage, and code amount control means for inputting the code amount and determining the quantization step width; And a decoded image calculation unit that decodes the encrypted data to calculate a decoded image, and an image quality calculation unit that inputs the decoded image, calculates the decoded image quality of the decoded image, and outputs the decoded image to the outside of the device. , Comparing the decoded image quality with an image quality command signal for the decoded image requested from outside the apparatus, and if the decoded image quality is at a lower level than the image quality requested by the image quality command signal, the code amount control means To reduce the quantization step width to a small value, and when the code amount exceeds the upper limit value, the image input means is controlled to thin out the frames of the image data. Moving image storage device, characterized in that a control means is provided.