JPH11122610A - Image coding method, image decoding method and system for them - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a coder side to apply adaptive image division in response to a characteristic of an input image, to transmit coded and multiplexed data for each division image and to allow a decoder side to decode an original image from data subject to adaptively division coded. SOLUTION: A characteristic extract section 5 of an image coder 1 extracts a characteristic from an input image signal 3 and a division control section 7 outputs a division control signal in response to the characteristic to an image division section 11. Division image signals 13a-13f divided by the image division section 11 are coded respectively by coding sections 15a-15f and multiplexed by a multiplexer section 21. Multiplexed data 33 received by an image decoder 31 via a transmission line 25 are separated into coded data 37a-37f by a demultiplexer section 35, decoded by decoding sections 41a-41f respectively and synthesized by an image synthesis section 47 according to screen position information sets 39a-39f as an original image as an output image signal 49.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding method, an image decoding method, and an apparatus for performing an encoding process for each of a plurality of small images obtained by dividing one image.

[0002]

2. Description of the Related Art High-definition television (hereinafter, HDTV)
Technology for improving the encoding and decoding speed for images having a high sampling rate such as
Japanese Patent Application Laid-Open No. 7-298258 discloses an "image encoding / decoding method".

According to this method, an image to be encoded on a transmitting side is divided into a plurality of small images, each small image is subjected to high-efficiency encoding in parallel and transmitted, and on the receiving side, the transmitted small image is transmitted. The gist is to decode encoded data for a plurality of small images, respectively, and then combine them to obtain a restored image of the original image.

[0004]

However, in the above-described image encoding / decoding method, data is separated at the time of decoding a coded / multiplexed signal by dividing the image into a plurality of divided images, and the data is individually decoded. Although it is necessary to combine a plurality of divided images, there is a problem that which divided image is displayed in which region after the combination must be determined in advance on the encoding side and the decoding side. .

[0005] Further, when one image is divided and coded, it is not possible to adaptively change the structure of the divided image, for example, the number of divided images, the divided position, the size of the divided image, and the like. There has been a problem that high-quality images cannot be transmitted using the capacity sufficiently.

[0006] In view of the above problems, an object of the present invention is to provide an image encoding apparatus capable of adaptively dividing an image in accordance with the characteristics of an input image and encoding each divided image. A method and an encoding device are provided. It is another object of the present invention to provide an image decoding method and an image decoding apparatus capable of restoring an original image from adaptively divided and encoded data.

[0007]

According to the present invention, a plurality of image data are input, and each of the input image data is encoded to obtain a plurality of encoded data. In an image encoding method for multiplexing a plurality of coded data and outputting multiplexed data, position information of each image when decoding and combining the plurality of coded data and outputting the coded data is referred to as the coded data. This is an image encoding method that is intended to be inserted into the multiplexed data or the multiplexed data.

According to the present invention, there is further provided an input means for inputting a plurality of image data, and the image data input to the input means are respectively encoded together with position information on which each image is decoded and then synthesized. Image encoding, comprising: a plurality of encoders for outputting; and multiplexing means for multiplexing a plurality of encoded data output from the plurality of encoders and outputting multiplexed data. Device.

The present invention also provides input means for inputting a plurality of image data, a plurality of encoders for respectively encoding the image data input to the input means and outputting coded data, Multiplexing means for multiplexing a plurality of pieces of encoded data output from the encoder and position information of each image when decoding, combining and outputting these encoded data, and outputting multiplexed data. An image encoding apparatus according to the first aspect.

Further, according to the present invention, multiplexed data coded and multiplexed for each of a plurality of images is input, the multiplexed data is separated into respective coded data, and the separated coded data is decoded. , An image decoding method for synthesizing a decoded image and outputting it as one image, extracting position information of each image inserted in input multiplexed data, and extracting decoded image data This is an image decoding method that combines data according to the obtained position information.

The present invention is also an image decoding apparatus for decoding each image data from multiplexed data encoded and multiplexed for each of a plurality of images and combining them as one image,
Separating means for separating the input multiplexed data into respective coded data, and a plurality of decoding means for decoding the separated coded data respectively and extracting the position information of each image inserted in the coded data An image decoding apparatus, comprising: an image synthesizer for synthesizing decoded image data according to extracted position information and outputting the synthesized image data as one image.

The present invention is also an image decoding apparatus for decoding each image data from multiplexed data coded and multiplexed for each of a plurality of images and combining them as one image,
Separating means for separating input multiplexed data into respective coded data and extracting position information of each image inserted in the multiplexed data, and a plurality of decoding means for respectively decoding the separated coded data An image decoding apparatus, comprising: an image synthesizer for synthesizing decoded image data according to extracted position information and outputting the synthesized image data as one image.

According to the present invention, a plurality of images are respectively encoded, and the position information for forming one image by arranging the decoded images during decoding is inserted into the encoded data or the multiplexed data. This allows the encoding side to freely change the number of images to be encoded and the size of each screen.

[0014]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is a block diagram of an image encoding device and an image decoding device according to the present invention. Referring to FIG. 1, an image encoding apparatus 1 includes a feature extraction unit 5 that extracts features of an image from an input image signal 3, controls a division size, the number of divisions, a division position, and the like of an input image and externally executes division control information. 9; an image dividing unit 11 for dividing an image into a maximum of six small images; and outputting the divided small images. Each of the divided small images is encoded in parallel to generate encoded data 17a to 17f. Encoding unit 1 to output
The multiplexing unit 21 is configured to multiplex 5a to 15f and coded data 17a to 17f and output the multiplexed data 23 to the transmission line 25.

The image decoding device 31 is connected to the transmission line 25.
A separating unit 35 that separates the multiplexed data 33 input from the multiplexed data 33 into a plurality of coded data, decoding units 41a to 41f that decode the plurality of coded data 37a to 37f separated by the separating unit 35 in parallel, Decoding units 41a to 41
f, the decrypted data 43a to 43f
An image synthesizing section 47 for synthesizing the two images and outputting as an output image signal 49 is provided.

Next, the operation of the image encoding device 1 will be described. An input image signal 3 which is high-definition image information such as HDTV (High Definition Television) is input to a feature extraction unit 5.
And input to the image dividing unit 11. The feature extracting unit 5 extracts features such as where in the input image the image is fine. The extracted features are sent to the division control unit 7.
The division control unit 7 determines the division method of the input image, for example, the number of divisions, the division size, the division position, etc., according to the extracted features. In the present embodiment, since the degree of parallelism of the encoding unit is 6, image division can be performed in up to six regions.

The image dividing section 11 divides an image into a plurality of areas according to the dividing method determined by the division control section 7, and outputs divided image signals 13a to 13f, respectively. The divided image signals 13a to 13f are input to the encoding units 15a to 15f, respectively, and are encoded by an encoding method such as MPEG.

Information such as the size and division position of each divided image in accordance with the division method determined by the division control unit 7 is also input to the encoding units 15a to 15f. It is inserted into the encoded data. The coded data of each area is multiplexed by the multiplexing unit 21, output as multiplexed data 23 of one stream, and sent out to the transmission path 25.

Next, division of an image will be described with reference to FIG. Here, a case where the input image is made up of 1920 horizontal pixels and 960 vertical lines as shown in FIG. 2A will be described. For example, when each of the encoding units 15a to 15f performs an encoding process with a maximum of 720 horizontal pixels and 480 vertical lines, an input image of 1920 pixels and 960 lines is divided into three or more parts horizontally and two or more parts vertically. Need to be divided into small areas.

As an example, FIG. 2B shows a case in which each area is divided into six parts each having the same size, that is, 640 pixels in the horizontal direction and 480 lines in the vertical direction. However, the input image may have different image details for each region. For this reason, if all regions are divided into the same size and the same encoding process is performed, the degradation in image quality due to the encoding process (including the compression process) differs between the regions.

Therefore, in order for the degree of image quality degradation of the decoded and synthesized restored image to be substantially uniform within the screen and to be a natural image, the region with high definition is relatively narrowed by narrowing the divided region. On the other hand, it is necessary to lower the compression ratio and conversely increase the compression ratio in areas where the definition is low, thereby increasing the compression ratio to homogenize the image quality of each area.

The feature extraction unit 5 shown in FIG. 1 detects which region of the image has the definition, and the division control unit 7 narrows the division region of the high definition part accordingly. FIG. 2C shows an example of six divisions when the definition of the image at the center of the screen is high. The left and right areas are each horizontal 7
There are 20 pixels and the center area is 480 horizontal pixels, and the compression rate at the center is 倍 times that of the left and right areas. In other words, the central area is assigned an information amount 1.5 times that of the left and right areas to reproduce high definition.

Further, in order to reflect the intention of the creator of the encoded data, the division method can be controlled by externally providing division control information 9 to the image encoding device 1. For example, when it is desired to improve the image quality of the gazing point of the image, it is possible to improve the image quality of the gazing point by giving the position information of the gazing point through the division control information 9 and reducing the divided area of the gazing point. Also, in order to always improve the image quality of the central portion of the image that is easily noticeable, it is also possible to always reduce the division size of the central portion of the image.

Next, the operation of the image decoding apparatus 31 will be described. The multiplexed data 3 encoded and multiplexed by the image encoding device 1 via the transmission path 25 is provided to the image decoding device 31.
3 is input. The separation unit 35 outputs the multiplexed data 33
Are separated, and the image encoded data 37a to 37a to 3
7f is output. The identification information for separation is inserted into the multiplexed data 33, and it is possible to relatively easily separate based on this.

The coded data 37a to 37f separated by the separation unit 35 are respectively decoded by the decoding units 41a to 41f.
At 1f, decoding is performed for each divided area. The decoded divided image is synthesized by the image synthesizing unit 47 based on the division information, and is output from 49 as one high-definition image.

When the division method such as the number of divisions, the division size, and the division position is made variable as described above, if the decoding side cannot know the division information indicating how the division is performed on the encoding side, However, it is impossible to combine the decoded divided images to form one original image.

Further, without changing the dividing method, FIG.
Even in the case of the fixed division as in (b), if the division information indicating which divided image belongs to which position area is not transmitted, the image decoding apparatus side should correctly combine. I can't do that either.

Next, a method of inserting the division information transmitted from the image encoding device to the image decoding device will be described.

MPE as an international standard for video coding
Although there is an encoding system such as G, a user data area is provided in MPEG video encoded data, and the user can freely insert data in this area. For this reason, the division control unit 7 in FIG. 1 gives the encoding units 15a to 15f information on the position of the image to be encoded by each encoding unit in the image in FIG. The conversion units 15a to 15f insert the position information as user data.

Conversely, on the decoding side, the decoding unit 41a of FIG.
To 41f extract location information from the user data,
The screen position information 39a to 39f are output to the image synthesizing section 47, and the divided images are synthesized according to the output.

As the position information of the image division, for example, the coordinates of the reference position in each divided image with respect to the coordinate axes on the original image to be divided, or the coordinates of both ends of the diagonal line of each divided image are indicated. Can be.

In the example of division shown in FIG. 2B, if the origin of the coordinate axes of the original image is set at the upper left, the position information of each divided image is (0,0), (640, 0),
(1280,0), and from bottom left to right (0,48)
0), (640, 480) and (1280, 480).

In the example of division shown in FIG. 2B, when the position information of each divided image is indicated by both end points of a diagonal line (for example, a diagonal line from upper left to lower right), the position with respect to the upper left divided image The information is (0,0) and (639,
479). Similarly, the position information for the upper center divided image is (640, 0) and (1279, 479). There is an advantage that the position information indicated by the diagonal line includes the size information of the divided image.

In addition, the MPEG multiplexing standard allows private data to be inserted in addition to video and audio. It is also possible to insert information on the position of each image in this area. In this case, the screen position information (or division control information) 19 from the division control unit 7 in FIG. The multiplexing unit 21 inserts, as private data, a set of identification information to be added when multiplexing the encoded data 17a to 17f of each divided region and position information of each divided region.

In the image decoding apparatus 31, the separating unit 3 shown in FIG.
5 is screen position information 3 indicating the position of the image of each divided area from the private data when separating various data.
9a to 39f can be extracted and output to the image synthesizing unit 47, and the image synthesizing unit 47 can perform the synthesizing process accordingly. By doing so, it is possible to make the division size and the number of divisions free without keeping the number of divisions on the encoding side and the size of the image to be divided constant.

Up to now, 1920 horizontal pixels and 960 vertical pixels
The case where the image of the line is divided horizontally into three and vertically into two has been described. Another example of division is shown in FIG. FIG.
The image of 1920 pixels in the horizontal direction as shown in FIG. 3A is sampled and converted by a factor of 3/4 in the horizontal direction to obtain an image as shown in FIG.
There is also a case where an image is divided into 40 pixels and encoded. In this case, two or more horizontal divisions and two or more vertical divisions are required. FIG. 3C shows 720 horizontal pixels and 48 vertical pixels.
This is an example of division in a case where 0 pixels are divided into four.

FIG. 3B shows the position information in this case.
Is used based on the horizontal 1440 pixel image. Although not shown in FIG. 1 on the decoding side, the output of the image synthesizing unit 47 is sampled and converted to 4/3 times in the horizontal direction, and then output as an original 1920 pixel image.

By inserting data indicating the position of each divided image in the original image into the user data area or the private data area in the multiplexed image encoded data, Whether the original input image size is changed, how many images are divided, or the size of the image to be divided is changed, the position on the decoding side is different on the decoding side. This makes it possible to combine images without any problem.

Although the case where the image is divided without overlapping the areas as shown in FIGS. 2 and 3 has been described above, the horizontal division into three and the vertical division into two as shown in FIG. It is also conceivable that the divided areas are overlapped, for example, by 32 pixels in the horizontal direction, and each is encoded.

This is because, in the case of an encoding method using motion compensation between frames such as MPEG, when an object moves on the boundary of the area, the object appears in the reference image in the area where the object newly appears in the area. It does not exist and cannot be coded using interframe coding with motion compensation. Therefore, in order to widen the area of the reference image, the areas of the divided images may be overlapped and coded. MPEG
Encoding method, in addition to the image size in the encoded data,
There is also an area to which information on which area of the size is to be displayed is added. Therefore, as shown in FIG. 4, even when the boundaries of the divided regions are overlapped and divided and each is encoded,
It is also possible to decode by the image decoding device of FIG. 1 and synthesize an image.

Next, the insertion position of the image division information in the MPEG encoding method will be described. FIG.
The extension part or the user data area in each layer of the image data is shown. The extension data or user data 51 following the extension data start code or the user data start code in the sequence layer, the extension after the group of picture header in the GOP layer. Group extension data (GED) or user data start code (UDS) following the start code (ESC)
C), the user data area 53, and picture extension data (P
The image division information can be inserted into the user data area 55 following ED) or UDSC.

FIG. 6 is a diagram showing a data structure of a packet layer which is a lower layer of the MPEG1 system stream. The stream ID of the packet start code is “1”
011 1101 ", private 1 packet,
When the stream ID is “1011 1111” and the private 2 packet is defined, the image division information can be inserted using these private packets.

FIG. 7 is a diagram showing a packet structure of a packetized elementary stream (PES) of an MPEG2 program stream. In this case as well, the type of the packet is similarly designated by the stream ID, and the above-described image division information can be inserted using the private 1 packet and the private 2 packet.

When the MPEG private packet is used as described above, the position information of each image divided from the original image into a plurality of areas is collectively converted into one data when the encoding device multiplexes the data other than the video and audio data. It can be inserted as data and separated and used by the separation unit of the decoding device.

[0045]

As described above, according to the present invention, 1
When one high-definition image is divided into a plurality of areas, each area is encoded, and multi-channel transmitted image data is decoded to reproduce high-definition image data, the degree of freedom of image division is increased.

For example, horizontal 1920 pixels (pix)
HDTV is not only divided horizontally into 640 pix, but also into 720, 480, and 720 pix to reduce the compression ratio at the center of the screen that is easy to notice.
Higher quality images can be transmitted even with the same transmission capacity as before.

Further, according to the present invention, since the method of dividing an image is free, it is also possible to detect a fine area of the image and relatively increase the code amount allocated to the area. Further, even if the number of horizontal and vertical pixels to be processed is changed, the decoding side can cope with it, and the degree of freedom on the division encoding side is greatly increased.

Further, according to the present invention, the conventional NTN can be used without using an encoder and a decoder dedicated to high definition images.
Encoders and decoders with SC definition can encode and decode high definition images.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image encoding device and an image decoding device according to the present invention.

FIG. 2 shows an input image size (a) and an example of image division (b);
It is explanatory drawing of (c).

FIG. 3 is an explanatory diagram of a case where an image is subjected to sampling conversion and divided.

FIG. 4 is an explanatory diagram in a case where an image is divided by overlapping areas.

FIG. 5 is an explanatory diagram of a user area in each layer of MPEG2 video encoded data in which image division information can be arranged.

FIG. 6 is a view for explaining a user-usable private packet in an MPEG1 system stream.

FIG. 7 is a diagram for explaining a user-usable private packet in an MPEG2 PES packet.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image coding apparatus, 3 ... Input image signal, 5 ... Feature extraction part, 7 ... Division control part, 9 ... External division control information, 1
1: image division unit, 13a to 13f: divided image signal, 15
a to 15f: coding unit, 17a to 17f: coded data, 19: screen position information, 21: multiplexing unit, 23: multiplexed data, 25: transmission path, 31: image decoding device, 33
... multiplexed data, 35 ... separation unit, 37a-37f ... coded data, 39a-39f ... screen position information, 41a-4
1f: decoding unit, 43a to 43f: decoded data, 45
a to 45f: screen position information, 47: image synthesizing unit, 49 ...
Output image signal.

Continued on the front page (72) Inventor Hideo Tsurufusa 8 Shinsugita-cho, Isogo-ku, Yokohama-shi, Kanagawa Pref.

Claims (8)

[Claims] 1. A plurality of image data are input, each of the input image data is encoded to obtain a plurality of encoded data, and the plurality of encoded data are multiplexed to output multiplexed data. In the image coding method, the position information of each image when decoding and combining and outputting the plurality of coded data is inserted into each of the coded data or the multiplexed data. Image coding method. 2. An image encoding method comprising dividing an input image into a plurality of images and encoding the divided images to obtain a plurality of encoded data, and multiplexing the plurality of encoded data to output multiplexed data. Extracting a feature of the input image, determining a method of dividing the image according to the extracted feature or control information input from the outside, and dividing the image into at least two images according to the determination result. Each of the divided images is encoded to obtain the plurality of encoded data, and the position information of each image at the time of decoding and combining and outputting these encoded data is included in the encoded data or An image encoding method characterized by being inserted into multiplexed data. 3. An image coding method which divides an input image into a plurality of images and encodes each of the plurality of images to obtain a plurality of encoded data, and multiplexes the plurality of encoded data to output multiplexed data. In the method, an input image is divided into a plurality of images such that boundaries are overlapped, and each of the divided images is encoded to obtain a plurality of encoded data, and the encoded data is decoded. An image encoding method, wherein position information of each image at the time of multiplexing, combining and outputting is inserted into the coded data or the multiplexed data. 4. An input means for inputting a plurality of image data, and a plurality of image data input to the input means, each of which is encoded together with position information on which each image is decoded and then synthesized, to output encoded data. And a multiplexing unit that multiplexes a plurality of pieces of encoded data output from the plurality of encoders and outputs multiplexed data. 5. An input unit for inputting a plurality of image data, a plurality of encoders for respectively encoding the image data input to the input unit and outputting encoded data, and the plurality of encoders Multiplexing means for multiplexing a plurality of pieces of encoded data output from and the position information of each image when decoding and combining and outputting these encoded data, and outputting multiplexed data. An image encoding device characterized by the following. 6. A multiplexed data coded and multiplexed for each of a plurality of images is input, the multiplexed data is separated into respective coded data, and the separated coded data is decoded and decoded. An image decoding method of combining the obtained images and outputting as one image, extracting position information of each image inserted in the input multiplexed data,
An image decoding method comprising combining decoded image data according to extracted position information. 7. An image decoding apparatus for decoding each image data from multiplexed data coded and multiplexed for each of a plurality of images and combining them as one image, wherein each of the input multiplexed data is A plurality of decoders for respectively decoding the separated coded data and extracting the position information of each image inserted in the coded data; and the decoded image data. An image decoding device, comprising: image synthesizing means for synthesizing according to extracted position information and outputting as one image. 8. An image decoding apparatus for decoding each image data from multiplexed data coded and multiplexed for each of a plurality of images and combining them as one image, wherein each of the input multiplexed data is Separating means for separating the position information of each image inserted in the multiplexed data while separating the coded data into a plurality of coded data, a plurality of decoders for respectively decoding the separated coded data, and the decoded image data An image decoding device, comprising: image synthesizing means for synthesizing according to extracted position information and outputting as one image.