JPH07303252A - Picture encoding/decoding method - Google Patents

Abstract

PURPOSE:To realize appropriate assignment by distributing information amounts assigned to a whole picture based on the respective feature amounts of divided small pictures and controlling the encoding output information amounts of the respective small pictures to be the divided information amounts. CONSTITUTION:An HDTV signal 402 inputted to an input terminal 401 is divided into the small pictures 404-407 of standard TV sizes by a picture division part 403. Feature amount calculation parts 408-411 find the feature amounts 412-415 of the divided small pictures. An information assignment part 416 distributes the information amounts assigned to the whole pictures to the small pictures based on the found feature amounts. The small pictures are encoded in respective information compression parts 421-424, and the outputs are inputted to buffer memory parts 425-428. Feed-back control is conducted so that the outputs become the information amounts 417-420 distributed in the information assignment part 416. Even if the characters of the divided small pictures largely differ from each other, the information amounts can appropriately be assigned and the decoding picture of satisfactory picture quality can be obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention is applicable to high-definition (HDT
V: high density television) In a high-efficiency coding method in which a signal with a high sampling rate such as an image is divided into a plurality of standard TV size small images and processed in parallel, the characteristics of the divided small images are significantly different. However, the present invention relates to an image encoding method that enables appropriate allocation of information amount and as a result can obtain a decoded image with high image quality.

[0002]

2. Description of the Related Art When a signal sampled at a high frequency such as a high-definition image (HDTV image) is encoded with high efficiency, a parallel processing method is used to operate the hardware in real time.

As shown in FIG. 4, consider a case in which an HDTV signal is spatially divided into four parts to form a standard TV size small image and the encoding process is performed in parallel. In FIG. 4, 101 is H
DTV image, 102A, 102B, 102C, 102D
Is a standard TV size small image.

FIG. 5 is a block diagram showing the functional arrangement of an apparatus which spatially divides the HDTV signal into four parts to form a standard TV size small image and performs parallel encoding processing.
1 is an input terminal, 202 is an HDTV signal, 203 is an image division unit, 204 to 207 are standard TV size small images, 20
8 to 211 are information compression units, 212 to 215 are buffer memory units, 216 to 219 are encoded outputs, 220 is a multiplexing unit, 221 is encoded output data, and an output terminal 222.

In FIG. 5, when the HDTV signal is spatially divided into four to form a standard TV size small image and the encoding processing is performed in parallel, first, on the transmitting side, the input terminal 20 is used.
The HDTV signal 202 input from the 1
3 small images of four standard TV sizes 204 to 20
Divide into 7. The divided small images 204 to 207 of the standard TV size are respectively information compression units 208 to 21.
It is input to 1, and the encoding process is performed. The encoded output is
The data is input to the buffer memory units 212 to 215 and feedback-controlled so that its output has a constant amount of information. Encoded outputs 216 to 21 each having a fixed amount of information
9 is input to the multiplexing unit 220, put together into one encoded output data 221, and then sent to the transmission path via the output terminal 222.

Now, considering the characteristics of the input HDTV signal, which one of the four small images is divided into four small images, although it naturally varies in the screen portion in each frame. It is difficult to specify whether is easy or difficult to encode. Therefore, when the information amount of R (bit / s) is assigned to the entire HDTV signal, the amount of information assigned to each small image is equal to 1/4 R (bit / s) to any small image. It is usually done.

[0007]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
Considering the case where the V image is encoded by R (bit / s) without being divided (it may be considered that ideal information allocation is made in the screen), depending on the image, a small image to be divided The number of generated bits in the four areas corresponding to is not necessarily equal.

For example, as shown in FIGS. 6 and 7, at first, there is a very fine and vigorous region in one part (a) of the screen, and the other parts (b), (c), (d). ) Is relatively flat, and a case where this fine region gradually moves to the portion (b) with time is considered. Initially, the parts (b), (c), and (d) are 1 / 4R (bit).
/ S) can be coded with a much smaller amount of information than
The remaining amount of information is transferred to the portion (a), which is difficult to encode with a small amount of information, so that the portion (a) is ¼.
Encoding is performed by applying a larger amount of information than R (bit / s). After that, the fine area in which much information should be used gradually shifts to (b), so that the amount of information assigned to (a) is gradually assigned to (b).

By the way, in the division type encoding, if the encoding process is performed assuming that the characteristics of the four small images are equal to each other, as in the conventional method, 1 / 4R is originally obtained.
The area to be encoded is forcibly suppressed to 1/4 R (bit / s) by spending more information than (bit / s), and as a result, the image quality of the decoded image is deteriorated. I will end up. Further, conversely, a region in which a decoded image with a sufficiently high image quality can be obtained with an information amount smaller than 1 / 4R (bit / s) may be encoded with 1 / 4R (bit / s),
In this case, wasteful information is being sent.

The present invention has been made to solve the above problems, and an object of the present invention is to divide a signal having a high sampling rate such as an HDTV image into a plurality of standard TV size small images. It is an object of the present invention to provide a technique capable of allocating an appropriate amount of information even when the properties of divided small images are significantly different in a high-efficiency encoding method for parallel processing.

Another object of the present invention is to provide an image coding method capable of obtaining a decoded image with good image quality.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[0013]

The outline of the representative one of the inventions disclosed in the present application will be briefly described as follows.

On the transmitting side, the image to be encoded is divided into a plurality of small images, and then each divided small image is subjected to high efficiency encoding in parallel and transmitted, and on the receiving side, it is transmitted. An image coding / decoding method that decodes coded data for a plurality of divided small images and then synthesizes the original image to obtain the original image, and obtains the feature amount of each divided small image. Based on the feature amount, the information amount allocated to the entire image is distributed to each divided small image, and when encoding each small image, each encoded output information amount is divided into the distributed information amount. It is characterized in that it is controlled so that

Further, as a feature amount of each of the divided small images, a variance within a frame which is a spatial feature amount and a variance of inter-frame difference values which is a motion feature amount are used.

[0016]

According to the above-mentioned means, by using the feature quantity that reacts to the difficulty of encoding, whether each divided small image is easy or difficult to encode, that is, high even if there is little information. Since it is possible to predict in advance whether the image quality will be obtained, or whether high image quality will not be obtained without allocating a lot of information, it is possible to encode at R (bit / s) as a whole, even if there is little information. R '(bit / s) for small images that are judged to have high image quality
The control is performed so that the information of (R ′ <R) is assigned, and R ″ (bit / s) (R ″) is applied to a small image that is determined to have high image quality unless much information is assigned. >
It becomes possible to control so as to allocate the information of R).

As a result, information can be assigned to the entire image according to the characteristic amount of the input image.
The image quality of the decoded image can be improved.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image coding / decoding method according to the present invention will be described in detail below with reference to the drawings.

(Embodiment 1) In the image coding / decoding method of Embodiment 1 according to the present invention, an image to be coded is divided into a plurality of small images on the transmitting side, and then each of the divided small images is divided. On the other hand, high efficiency coding is applied in parallel and transmitted,
At the receiving side, it is an image coding / decoding method for decoding the transmitted coded data for a plurality of small images and then synthesizing them to obtain the original image. Then, based on the obtained feature amount, the information amount assigned to the entire image is distributed to each divided small image, and when encoding each small image, the encoded output information amount is The control is performed so that the information amount is distributed.

FIG. 1 is a block diagram showing the functional arrangement of an apparatus (apparatus including a computer) for implementing the image coding / decoding method of the first embodiment, in which 401 is an input terminal and 40 is an input terminal.
2 is an HDTV signal, 403 is an image dividing unit, and 404 to 40
7 is a standard TV size small image, 408 to 411 are feature amount calculation units, 412 to 415 are feature amounts in each small image, 4
16 is an information allocation unit, 417 to 420 are information amounts allocated to each small image, 421 to 424 are information compression units, 4
25 to 428 are buffer memory units, 429 to 432 are encoded outputs, 433 is a multiplexing unit, 434 is encoded output data, and 435 is an output terminal.

The implementation device of the first embodiment appropriately determines the amount of information to be assigned to each small image based on the feature amount of each small image calculated prior to encoding, and its configuration is as follows. The structure is as shown in FIG.

In the operation of this embodiment, as shown in FIG. 1, first, the HDTV signal 402 is input to the input terminal 401. The input HDTV signal 402 includes four standard TV size small images 404-
It is divided into 407. Each of the divided small images of the standard TV size has a feature amount calculation unit 408 to 411.
And the feature quantities 412 to 415 in each small image are obtained. The information allocation unit 41 using the obtained feature amount
6, the information amount 417 to 42 assigned to each small image
0 is calculated. Here, the total amount of information 417 to 420 assigned to each small image is controlled to be equal to the amount of information assigned to the whole.

On the other hand, each small image has an information compression section 421.
.. to 424 to be encoded. The encoded output is input to the buffer memory units 425 to 428,
Feedback control is performed so that the output becomes the information amount 417 to 420 determined by the information allocation unit 416. Encoded output 42 each having a fixed amount of information
9 to 432 are input to the multiplexing unit 433, combined into one encoded output data 434, and then output to the transmission path via the output terminal 435.

As can be seen from the above description, the first embodiment
According to the above, whether or not each divided small image is easy to encode, that is, high image quality can be obtained even with a small amount of information, by using a feature amount that responds to the difficulty of encoding. , Or it is possible to predict in advance whether high image quality will not be obtained unless a large amount of information is allocated. Therefore, when encoding with R (bit / s) as a whole, high image quality can be obtained even if there is little information. It is determined that high quality cannot be obtained unless a large amount of information is assigned to the small image determined to be R '(bit / s) (R'<R). It is possible to perform control so that the information of R ″ (bit / s) (R ″> R) is assigned to the small image.

With this, it becomes possible to allocate information to the entire image according to the characteristic amount of the input image, so that the image quality of the decoded image can be improved.

(Embodiment 2) The image encoding / decoding method according to Embodiment 2 of the present invention considers a case where an HDTV signal is divided into four standard TV size small images and processed. It is assumed that a variance within a frame, which is a quantity, and a variance between inter-frame difference values, which is a motion feature quantity, are used. Also, discrete cosine transform is used as the information compression method. In this case, when the information of R (bit / s) is assigned as a whole, the amount of information assigned to each small image is calculated as follows.

First, the input HDTV signal is converted into four standard TVs.
The image is divided into small images each having a size, and each of the four divided small images is divided into 16 × 16 small blocks.
The pixel value in the small block is B (k, l) [(k, l) = 0,
1, 2, ..., 15], and the difference between the small block and the small block one frame before is D (k, l) [(k, l) =
0, 1, 2, ..., 15], the _intra- frame variance σ ² _intra and the _inter- frame variance σ ² _inter of each small block are calculated according to the following equation 1.

[0028]

[Equation 1]

However, here, B bar and D bar are B
It is the average value of (k, l) and D (k, l).

Next, for each of the four small images, the obtained σ ² _intra and σ ² _inter are averaged within the small image, and the average _intra- frame variance σ ² _intra bar and the average inter-frame variance for the small image are averaged. Find the σ ² _inter bar. By using these two values, the spatiotemporal feature amount is obtained as the feature amount F of the small image as shown in the following Expression 2.

[0031]

[Equation 2]

Generally, the average _intra- frame variance σ ² _intra bar becomes large when the screen includes many high-definition regions, and when a large amount of information is not used, high image quality cannot be obtained. . Further, the average _inter- frame variance σ ² _inter bar becomes large when there is a large amount of motion, and similarly, when a large amount of information is not used, high image quality cannot be obtained. Therefore, the value of F shows a large value when it is difficult to code, that is, high image quality cannot be obtained without using a lot of information, and shows a small value otherwise. It can be used as an index of difficulty in conversion.

Therefore, using the spatiotemporal feature amount Fm, (m = 1, 2, 3, 4) obtained for each of the four small images, the information amount Rm, (m = 1 to be assigned to each small image. , 2,
3, 4) is calculated by the following equation (3).

[0034]

[Equation 3]

In this way, the value of Rm is set to a large value for the small image that is relatively difficult to code among the four small images, and is assigned when the coding of each small image is executed. By performing the encoding control so that the information amount is obtained, it becomes possible to perform the bit allocation according to the difficulty of encoding each small image. Further, the amount of information of the entire HDTV image can be controlled to be the following Expression 4.

[0036]

[Equation 4]

FIG. 2 shows an example of changes in the spatio-temporal feature amount Fm and accompanying changes in the assigned information amount Rm. In this example,
Up to time t ', the spatiotemporal features of each small image are almost equal,
After that, the spatiotemporal feature of the small image 1 gradually increases,
The feature amounts of the small images 3 and 4 become smaller, and the spatiotemporal feature amount ratio at time t ″ is F ₁ : F ₂ : F ₃ : F ₄ = 7: 4: 3:
It becomes 2. On the other hand, when information is assigned to each small area by the method described above, R ₁ : R ₂ :
R ₃ : R ₄ = 1: 1: 1: 1, same for any small image 1 /
Information of 4R (bit / s) is assigned. On the other hand, after time t ′, the allocation state gradually changes according to the change of the spatiotemporal feature amount, and at time t ″, R ₁ : R ₂ :
R ₃ : R ₄ = 7: 4: 3: 2. The information allocation at the time t ″ is 7 / 16R (bit / s) for the small image 1, 4 / 16R (bit / s) for the small image 2, and 3 / 16R for the small image 3.
R (bit / s), 2/16 R (bit /
s).

FIG. 3 is a block diagram showing the functional arrangement of an apparatus (apparatus including a computer) for implementing the image coding / decoding method according to the second embodiment. 601 is an input terminal and 60 is an input terminal.
2 is an HDTV signal, 603 is an image division unit, and 604 to 60
7 is a standard TV size small image, 608 to 611 are intra-frame variance calculation units, 612 to 615 are inter-frame variance calculation units, 616 to 619 are spatiotemporal feature amount calculation units, and 620 to 6
Reference numeral 23 denotes the spatiotemporal feature quantities F ₁ , F ₂ , F ₃ , F ₄ , 624 denotes an information allocation unit, 625 to 628 denote information quantities R ₁ , R ₂ , R ₃ , R ₄ and 629 to be allocated to each small image. 632 is a discrete cosine transform unit, 633 to 636 are quantization units, and 637 to 6
Reference numeral 40 is a code allocation unit, 641 to 644 are buffer memory units, 645 to 648 are encoded outputs, 649 is a multiplexing unit, 650 is encoded output data, and 651 is an output terminal.

As shown in FIG. 3, the operation of the apparatus according to the second embodiment is as follows. First, the HDTV signal 602 is input to the input terminal 601.
Is entered. The input HDTV signal 602 has four standard TV size images 604-
It is divided into 607. The divided small images 604 to 607 of the standard TV size are respectively included in the intra-frame variance calculation units 608 to 611 and the inter-frame variance calculation unit 61.
Is input to 2-615, average frame variance sigma ² _intra bars and the average inter-frame variance sigma ² _{i NTER} bars in each small image is determined, the space-time feature quantity calculator 61 using these
6 to 619, the spatiotemporal feature F ₁ for each small image,
_{F 2, F 3, F 4} (620~623) is obtained.

The obtained spatiotemporal feature amount Fm (m = 1,
2, 3, 4), the information allocation unit 624 uses the information amounts R ₁ , R ₂ , R ₃ , R ₄ (625) allocated to each small image.
~ 628) is calculated.

On the other hand, the small images 604 to 607 are input to the discrete cosine transform units 629 to 632, respectively, subjected to the discrete cosine transform, and then input to the quantization units 633 to 636. The quantizing unit quantizes the discrete cosine transform coefficient, the codes are assigned by the code assigning units 637 to 640, and then sent to the buffer memory units 641 to 644. In each of the buffer memory units 641 to 644, the quantizing unit 63 is arranged so that its output becomes the information amount 625 to 628 determined by the information allocating unit 624.
Feedback control is applied to 3 to 636. The coded outputs 645 to 648, each of which has a certain amount of information, are multiplexed by the multiplexing unit 6.
The data is input to 49, put together into one encoded output data 650, and then sent out to the transmission line via the output terminal 651.

In the first and second embodiments, the case of dividing into four has been described, but the case of dividing into a plurality can be considered in exactly the same manner. Further, as the feature amount, another feature amount that can predict the difficulty of encoding can be used.

As can be seen from the above description, the second embodiment
According to FIG. 2, as shown in FIG. 2, the spatiotemporal feature amount of each small image is substantially equal until time t ′, and thereafter, the small image 1 gradually increases.
Of the small images 3 and 4 become small, and the spatio-temporal feature ratio at time t ″ becomes
F ₁ : F ₂ : F ₃ : F ₄ = 7: 4: 3: 2. On the other hand, when information is assigned to each small area by the method described above, R ₁ : R ₂ : R ₃ : R ₄ = 1: 1: 1 until time t ′.
1: 1 with the same 1 / 4R (bit / s) for any small image
Information is assigned. On the other hand, after time t ′, the allocation state gradually changes according to the change in the spatiotemporal feature amount, and at time t ″, R ₁ : R ₂ : R ₃ : R ₄ = 7: 4: 3:
It becomes 2. The information allocation at time t ″ is 7 in the small image 1.
/ 16R (bit / s), 4 / 16R (bi
t / s), the small image 3 has 3 / 16R (bit / s), and the small image 4 has 2 / 16R (bit / s).

As a result, information can be assigned to the entire image according to the feature amount of the input image.
The image quality of the decoded image can be improved.

Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above embodiments, and it goes without saying that various modifications can be made without departing from the scope of the invention.

[0046]

The effects obtained by the representative ones disclosed in the present application will be briefly described as follows.

Before encoding, it is possible to predict in advance how much information should be assigned to each of the divided small images. Therefore, if the information generation amount of the divided small image is controlled based on the predicted value, the information amount can be allocated more efficiently than fixed allocation, and as a result, the image quality of the decoded image can be improved. Can be improved.

[Brief description of drawings]

FIG. 1 is a block configuration diagram showing a functional configuration of an implementation device (device including a computer) of an image encoding / decoding method according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of a change in a spatiotemporal feature amount Fm and a change in an associated information amount Rm associated therewith according to a second embodiment of the present invention.

FIG. 3 is a block configuration diagram showing a functional configuration of an implementation device (device including a computer) of the image encoding / decoding method according to the second embodiment.

FIG. 4 is a diagram for explaining a case where a conventional HDTV signal is spatially divided into four to form a standard TV size small image, and encoding processing is performed in parallel.

FIG. 5 is a block configuration diagram showing a functional configuration of an apparatus that spatially divides a conventional HDTV signal into four to form a standard TV size small image and performs encoding processing in parallel.

[Fig. 6] Fig. 6 is a diagram for describing a problem of a conventional image encoding / decoding method.

[Fig. 7] Fig. 7 is a diagram for describing a problem of a conventional image encoding / decoding method.

[Explanation of symbols]

201, 401, 601 ... Input terminals, 202, 402,
602 ... Input HDTV signal, 203, 403, 603 ...
Image division unit, 204 to 207, 404 to 407, 604
~ 607 ... Divided standard TV size small image, 208 ~
211, 421-424 ... Information compression section, 212-21
5, 425-428, 641-644 ... Buffer memory section, 216-219 ... Encoded output, 220, 433, 6
49 ... Multiplexing unit, 221, 434, 650 ... Multiplexed encoded data, 222, 435, 651 ... Output terminal,
408 to 411 ... Feature amount calculation unit, 412 to 415 ... Obtained feature amount, 416, 624 ... Information allocation unit, 41
7-420, 625-628 ... Allocated information amount,
429 to 432 ... Encoded output data, 608 to 611 ...
Intra-frame variance calculator, 612 to 615 ... Inter-frame variance calculator, 616 to 619 ... Spatio-temporal feature amount calculator, 62
0-623 ... Spatio-temporal feature amount, 629-632 ... Discrete cosine transform unit, 633-636 ... Quantization unit, 637-64
0 ... Code assigning unit.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display area H03M 7/36 8842-5J H04N 7/015 G06F 15/66 330 C H04N 7/00 A

Claims (2)

[Claims] 1. A transmission side divides an image to be encoded into a plurality of small images, then performs high-efficiency encoding on each of the divided small images in parallel and transmits the small images. An image coding / decoding method for decoding the transmitted coded data for a plurality of small images and then synthesizing them to obtain the original image, wherein the feature amount of each of the divided small images is obtained and obtained. The information amount allocated to the entire image is distributed to each of the divided small images based on the obtained feature amount, and the encoded output information amount is distributed to each small image when encoding. An image coding / decoding method characterized in that control is performed so as to obtain a quantity. 2. The image coding / decoding method according to claim 1, wherein the feature amount of each of the divided small images is a spatial feature amount within a frame and a motion feature amount between frames. An image coding / decoding method characterized by using variance of difference values.