JP2001045494A - Image encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the generated quantity of encoded data in an image encoding device without deteriorating the image in an image area that is visually important. SOLUTION: In this image encoding device, an area dividing part 1 divides an image shown by an input digital image signal 101 into a plurality of areas. An image signal subjected to area division is inputted to a hierarchizing part 2 and an encoding part 4. The part 2 classifies each area according to a prescribed standard and outputs a class identification signal 104 corresponding to each area. The signal 103 is synchronized with the image signal 102 of a corresponding area and inputted to the part 4. The part 4 encodes the signal 102 while changing an encoding control parameter such as an encoding time interval and quantization accuracy in accordance with the signal 103 for each area.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding technique used for an image communication device such as a videophone or a video conference and an image storage / recording device such as a digital VTR.

[0002]

2. Description of the Related Art Various image coding techniques have been developed for improving the efficiency of digital image transmission and recording.

FIG. 9 is a block diagram of the International Telegraph and Telephone Consultative Committee ITU-
Recommendation H. T. FIG. 261 is a block diagram illustrating the conventional image encoding device illustrated in H.261. In the apparatus shown in FIG. 9, first, an input digital image signal 101 to be encoded is input to a differentiator 21. The differentiator 21 calculates a difference between the input digital image signal 101 and a prediction signal 136 described later, and outputs the difference signal as a prediction error signal 131. The encoding unit 22 encodes the prediction error signal 131 and outputs encoded data 132. As a method of encoding in the encoding unit 22, a prediction error signal is transformed from a spatial domain to a frequency domain using DCT (discrete cosine transform), which is a kind of orthogonal transform, and a transform coefficient obtained by the DCT is converted into a linear quantized signal. A technique for optimizing is adopted in the above recommendation.

[0004] The encoded data 132 obtained by the encoding unit 22 is branched into two, one of which is sent to a receiving-side decoding device (not shown), and the other is input to a decoding unit 23 in the present device. . The decoding unit 23 performs an operation opposite to that of the encoding unit 22, and performs a decoding prediction error signal 133 based on the encoded data 132.
Is output. The adder 24 outputs the decoded prediction error signal 1
By adding 33 to the prediction signal 136, a decoded image signal 134 is obtained and output. The decoded image signal 134 is collectively stored for one frame in a memory 25 such as a frame memory. The memory 25 delays the stored decoded image signal by one frame and outputs it as a decoded image signal 135 of the previous frame. The prediction unit 26 performs motion compensation prediction using the input digital image signal 101 to be encoded and the decoded image signal 135 one frame before, and outputs a prediction signal 136 and a motion vector 137. Then, the motion vector 137
Are sent to a receiving-side decoding device (not shown), and the prediction signal 136 is sent to the differentiator 21 and the adder 24.

[0005]

The conventional image encoding apparatus is configured as described above, and the encoding section encodes one frame of image almost uniformly regardless of the content of the image to be encoded. In addition, since the memory holds only one frame of an image and can be rewritten for each frame, the encoding efficiency cannot be improved.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and performs encoding in accordance with the contents of an image. It is an object of the present invention to provide an image encoding device that improves efficiency.

[0007]

In order to achieve the above-mentioned object, a first image encoding apparatus according to the present invention divides an input image into a plurality of regions and outputs an image signal provided with region division information. An area dividing unit, and a layering unit that classifies each area divided according to a predetermined criterion, and outputs a class identification signal indicating a class of each area,
A coding unit for coding the region-divided image signal based on the set coding control parameter and outputting coded data including the region division information; A coding control parameter is set according to a class identification signal given to each region from the unit.

According to the present invention, there is provided an image decoding apparatus which receives encoded data which is divided into a plurality of areas, and is encoded for each area, and which decodes the encoded data to obtain a decoded image. For the area from which the encoded data is obtained, the area division information is separated from the encoded data, and for each area based on the separated area division information, according to the area identification signal given at the time of encoding. Decoding and outputting an image signal.

Further, a second image encoding apparatus according to the present invention comprises:
An area dividing unit that divides an input image into a plurality of areas and outputs an image signal to which area dividing information is added, and classifies each area divided according to a predetermined criterion, and classifies each area. A plurality of image memories that store past image signals, and read out image signals similar to the region-divided input image signals from the plurality of image memories as prediction source image signals. A prediction unit that generates a prediction signal from the prediction source image signal using a prediction function, and outputs a prediction signal and a number of a memory from which the prediction source image signal is read and an address of the prediction source image signal in the memory; For each region, a prediction error extraction unit that extracts and outputs a prediction error signal from the input image signal and the prediction signal that have been region-divided, and encodes the prediction error signal and includes region division information An encoding unit that outputs encoded data, a decoding unit that decodes encoded data output from the encoding unit and outputs a decoded prediction error signal, and, for each region, a decoding prediction error signal and a prediction signal. A signal combining unit that combines the decoded image signals, and using the decoded image signal for each region output from the signal combining unit, distributing to the plurality of image memories based on the class identification signal from the hierarchical unit. A memory control unit that updates the storage contents of the plurality of memories by storing the information.

According to the present invention, there is provided an image decoding apparatus for inputting encoded data divided into a plurality of areas and encoding each area, and for obtaining a decoded image by decoding the encoded data. A plurality of image memories storing past image signals based on the class identification signal for each area, and the number of the image memory output from the image encoding device and the address of the prediction source image signal in the memory, A prediction unit that outputs an image signal specified from the input image memory number and address as a prediction signal, and separates region division information from encoded data, and for each region based on the separated region division information, A decoding unit that decodes encoded data according to a class identification signal used in encoding and outputs a decoded prediction error signal; and a decoding prediction error signal and the prediction signal for each region. A signal synthesizing unit for synthesizing a decoded image signal from the plurality of image memories based on a class identification signal for each of the regions output from the signal synthesizing unit. And a memory control unit for updating the storage contents of the plurality of image memories by storing the image data.

According to the first configuration, the image signal of each divided area is encoded using an encoding control parameter corresponding to the class of each area. By controlling the coding control parameters of the coding unit so that the amount of coded data generated in an area determined to be low in class, ie, low in importance, can be improved in coding efficiency.

Here, the area division in the area dividing section and the class determination in the layering section can be performed by a method based on an edge of an image, a method based on an activity of an image, a method based on motion data of an image, and the like.

As the encoding control parameter, a time interval at which encoding is performed, a quantization accuracy at the time of encoding, a subsampling rate, and a threshold value of an amount of encoded data can be used. By controlling these parameters based on the class identification signal obtained by the layering unit,
It is possible to reduce the amount of encoded data generated in a region of low importance.

Further, according to the second configuration, the past image signal used for the prediction is sorted and stored for each area in the memory corresponding to the class of the area, and the prediction source image used for the prediction in the prediction unit is stored. As a signal, an image signal similar to the image signal of each region to be encoded is read from the memory and used.

By reading the prediction source image signal from the memory corresponding to the class identification signal from the hierarchical unit, the time required for searching for the prediction source image signal can be reduced.

Further, the encoding time interval is changed for each area according to the class of the area, and the updating interval of the memory is changed accordingly, so that the amount of generated encoded data can be reduced and the memory can be rewritten. Work load can be reduced, and coding efficiency can be improved.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a first embodiment of an image coding apparatus according to the present invention. As shown in the figure, an input digital image signal 101 to be encoded is first input to the area dividing unit 1. The area dividing unit 1 has a memory for storing image signals such as a frame memory, and uses an input digital image signal 101 for one frame to form an input image on the memory. Attention is paid to a specific shape or the like included therein, and the image is divided into a plurality of regions.

As a method of area division, for example, an edge of the input image is detected by performing a differentiation operation or the like on the input image formed on the image storage memory, and a portion of the input image surrounded by the edge is detected. Can be used as one region. For example, when the present apparatus is applied to a video conference, an image composed of a person and a background as shown in FIG. 2 is generally given as an input image. Region 201 of a person with a shadow, region 202 of a stationary person, and region 20 of a desk constituting a background or foreground
3. It can be divided into a painting area 204, a clock area 205, and a wall 206 area.

When the area is divided by edges, for example, taking a face image of a person as an example, a region surrounded by edges representing faces is surrounded by edges such as eyes, nose and mouth. These areas are detected, but the areas inside these face areas can be integrated into the face area without being divided into separate areas, and the entire face can be treated as one area. In this case, since it is not necessary to perform processing for each fine area at the time of encoding, the encoding efficiency is increased. The integration of the regions at this time is performed by a method such as pattern recognition.
That is, when a certain area is determined to be a face area by pattern recognition or the like, the area surrounding the edge in the area is not divided into independent areas, but the entire face is combined into one area.

When this region division is completed, the region division unit 1
Outputs an image signal 102 obtained by adding area division information to an input digital image signal 101. This image signal 102
Indicates that the image data in the memory of the area dividing unit 1 is sequentially output for each divided area. That is, in FIG. 2 as an example, first, image data of the area of the person 201 is output as an image signal, and then image data of the area of the person 202 is output as an image signal. Are sequentially output. In this case, the image signal of each area is added with address information indicating which portion on the screen the image signal falls on. In addition, as the image signal 102, for example, similarly to the input digital image signal 101, an image on the memory of the area dividing unit 1 may be output from the upper left to the lower right along the scanning line. Adds, as overhead information, identification information indicating in which region each image signal is included.

The image signal 102 containing the area division information obtained in this way is branched into two and input to the layering unit 2 and the encoding unit 4.

The layering section 2 has an image storage memory such as a frame memory, like the area dividing section 1.
The image of each area is configured on the memory using the image signal 102 output from the CPU, the importance of each area in the entire image is determined based on the image of each configured area, and the importance is determined according to the importance. To classify each area.

At this time, in the case where the image signal 102 is output in order for each area, the image of each area is formed in order on the memory of the hierarchical unit 2 and the image is formed on the memory. The importance of each area is determined in order and classified. In the case of a method in which overhead information indicating an area is added to an input image signal and output is performed according to a normal scanning order, images of all areas are formed on the memory of the layering unit 2 and then each area is formed. The importance is determined.

The importance of an image area is higher in an area where the viewer of the image is more noticeable in the image, but the criterion naturally depends on the target to which the apparatus is applied. For example, in a video conference, the importance of a person is high, and the importance of a still life such as a desk or wall serving as a background or foreground is low. In addition, even in the same person, it can be said that the person who is speaking (that is, has movement) has a higher importance. Therefore, FIG.
In the example shown in FIG. 7, when the classes are divided into three classes in each region, the person region 201 is classified into the class 1 having the highest importance, the person region 202 is classified into the next class 2 and And the area of the wall can be classified into the class 3 having the lowest importance. In addition, as the index indicating the importance, the magnitude of the movement of the region in the image, the frequency of the movement, and the like can be considered. In this case, the importance is higher in an area where the movement is large and an area where the movement is frequent. Such criteria for determining the degree of importance are appropriately set according to the target to which the apparatus is applied.

For example, in the case of performing the region division by the edge detection described above, the layering unit 2 performs pattern recognition or the like on the shape of the edge of each region, thereby obtaining a person region and other background walls. And the like, and further taking into account movement information of each area between the current frame and the previous frame, the area of the person who is speaking and the area of a person who is not speaking are distinguished. The region of the person who is speaking is classified into class 1 having the highest importance, and the regions other than the person are classified into class 3 having the lowest importance.

The layering section 2 classifies each area in this way and outputs a class identification signal 103 indicating the class of each area to the encoding section 4.

The encoding unit 4 encodes the image signal 102 including the area division information from the area division unit 1 by PCM, DPCM, or the like, and outputs encoded data 105. At the time of this encoding, the class identification signal 103 from the hierarchical unit 2 is input to the encoding unit 4 in synchronization with the image signal 102 for each region. In accordance with the class identification signal 103, the encoding unit 4 performs encoding while changing an encoding control parameter when encoding the image signal 102 of each area.

The encoding control parameter includes, for example, a time interval at which encoding is performed. In accordance with the class identification signal 103 from the hierarchical unit 2, the class of lower importance increases the encoding time interval. That is, Class 1 with high importance
The image signal of the area included in is encoded every frame,
In the following, as the degree of importance decreases, the encoding frequency is increased by extending the encoding time interval such that the class 2 region is encoded once every several frames and the class 3 region is encoded once every tens of frames. Thinning out. This makes it possible to reduce the amount of encoded data generated as compared to a case where the entire screen is uniformly encoded, and to improve encoding efficiency. The encoding time interval of each class is appropriately determined in consideration of a trade-off between image quality and encoding efficiency.

Here, for example, in a video conference or the like,
It is conceivable that the information necessary for the image is only the state of the person and the state of the background or the like is not particularly required. In such a case, only the person areas of classes 1 and 2 are coded and the background of the class 3 is encoded. For example, a method of stopping the encoding of the image signal in the area such as the above may be considered. As a result, the amount of encoded data can be further reduced. In this case, in the decoding device, encoded data of an area such as a background cannot be obtained, and thus, for example, a blue-back signal is output to an area such as a background, or a background image stored in advance is output. To obtain a decoded image.

As the encoding control parameter, in addition to the encoding time interval, the quantization accuracy when encoding is performed by the encoding unit 4 can be used. That is, in accordance with the class identification signal 103 from the layering unit 2, the region of the class of high importance is coded so that the quantization accuracy is reduced so as to be able to reproduce the details of the image, and conversely, the region of the class of low importance is The generation of coded data is suppressed by coarsening the quantization accuracy in the region, whereby the amount of generated coded data can be reduced as a whole while maintaining the quality of the image in the region of high importance. .

In addition, a method of changing a sub-sampling rate when encoding an image signal can be considered as an encoding control parameter. This will be described with reference to FIG. In FIG. 3, it is assumed that circles A1 to D4 respectively represent pixels in the image. When coding a region of a class having a high importance, the image signals corresponding to all pixels are coded. That is, all the pixels A1 to D4 in FIG. 3 are encoded. Conversely, when encoding a region of low importance, for example, pixels are sub-sampled at a rate of one every two in the horizontal and vertical directions,
Pixels B1 to B4, C1 to C4, D1 to D4 are thinned out,
Only the image signals corresponding to the pixels A1 to A4 are encoded.
In this way, by increasing the degree of pixel thinning (sub-sampling rate) in a region of lower importance, the amount of encoded data generated as an entire image can be reduced. Note that the degree of thinning is appropriately set according to the importance of the class.

It is also possible to change the threshold value of the encoded data generation amount as an encoding control parameter. That is, the encoding unit 4 sets a threshold value (upper limit value) of the encoded data generation amount for each class, and performs control to stop encoding when the threshold value is exceeded. For example, the coded data generation amount threshold value is set to a large value for a class with high importance so that all image signals can be coded. Conversely, the threshold value is set small for a class with low importance, and when the amount of encoded data reaches the threshold value, the encoding of the signals thereafter is stopped. This threshold value is set so as to be smaller as the class has lower importance. This makes it possible to reduce the amount of encoded data generated for the entire image while maintaining the quality of the image in the region of high importance.

In this case, the decoding device cannot obtain data on the image after the coding is discontinued, but this portion forms the entire image by using the image of the previous frame as it is. .

As described above, according to the present embodiment, the image to be encoded is divided into regions, the divided regions are divided into classes according to importance, and when the image signal of each region is encoded, the By changing the control parameters at the time of encoding in accordance with the class of the region and reducing the amount of encoded data in the region of low importance, the encoding efficiency of the entire image can be improved.

In this embodiment, the region is divided by the edges in the image, and the classification (hierarchization) is performed based on the shape of the edge. However, other various regions may be divided and classified. A method is conceivable.

For example, for area division and class division,
Texture analysis, a technique similar to edge detection, can be used. The texture analysis is a method of dividing an image into regions having a certain structure (texture), for example, dividing a vertical stripe pattern region and a horizontal stripe pattern region. Walls and desks have a certain structure (pattern)
In many cases, by performing texture analysis, it is possible to divide an area into an area such as a background wall and other areas. Classification is performed by performing pattern recognition on the shape and texture of each of the obtained regions.

The methods of edge detection, texture analysis, and the like are division methods in which the image is divided into variable-shaped regions surrounded by a contour line composed of one pixel (or at most several pixels) according to the content of the image. The present invention is effective not only when dividing and processing not only an image formed of such a variable-shaped area but also an image formed of block units of a fixed size as in the related art. That is, in this case, a blocking unit is provided before the region dividing unit, and the image signal is divided by the blocking unit into blocks of a fixed size (for example, 8 × 8 pixels). Then, the area dividing unit 1 collectively sets blocks having image signals similar to each other among the divided blocks into one area. The layering unit performs pattern recognition on the shape, pattern, etc. of each area to determine whether each area represents a person or a background, and classifies the area. do. When the data is divided into blocks as described above, a block-based coding method such as DCT can be used.

It is also possible to perform area division and class division based on the activity of the image signal. The activity is a degree of change of the image signal, and uses, for example, a square error from an average value of the image signal, that is, variance. More specifically, the image is divided into blocks of a fixed size (for example, 8 × 8 pixels), and the average value of the image signal and the variance from the average value are obtained for each block, and those values are calculated. Area division and class division are performed on the basis. For example, since an image such as a background wall often has a fixed pattern, blocks having the same average value and variance value are grouped into one region as a wall or the like, and classified into classes of low importance. I do. In addition, since the variance value, that is, the activity is large in a block including an edge such as a contour of a person, the area surrounded by such a block having a large activity is regarded as one area and area division is performed, and the shape of the area is determined. It is also possible to determine whether or not the area is a person area by performing pattern recognition on, and classify according to the determination.

It is also possible to perform area division and class division using time information using image data of a past frame. That is, the image data of the past frame is stored in the memory, the image of the past frame is compared with the input image of the current frame, and the motion of each part in the image is detected. Then, the area is divided into a part where the movement has occurred and a part where the movement has been stopped. Then, the area where there is movement is classified into a class having a high importance, and the area where there is no movement is classified into a class having a low importance.
Here, in addition to simply dividing the area based on the presence / absence of motion, it is also possible to combine portions having substantially the same magnitude of motion as one area. In this case, according to the magnitude of the motion of each area, for example, the area where the motion is larger is classified into the class having higher importance.

When the area is divided by the motion of the image,
In addition to classifying based on the magnitude of the movement, the classification can also be performed using the frequency of the movement. That is,
Movement information for a plurality of frames is stored in the hierarchy unit, and the frequency of movement over a plurality of frames in each area is obtained. As described above, by using the movement history over a plurality of frames, it is possible to classify the temporarily stationary area without lowering the importance.

Next, a second embodiment of the present invention will be described. FIG. 4 is a block diagram showing the second embodiment.

In the first embodiment, the coding control parameter of one coding unit is changed in accordance with the class of each area when each area is coded. , A plurality of encoding units of different encoding systems are used, and the encoding unit is selected according to the class of each area.

In FIG. 4, the operations of the area dividing section 1 and the hierarchical section 2 are the same as in the first embodiment.

The encoding units 4a, 4b and 4c employ different encoding systems. For example,
The encoding unit 4a performs encoding such as PCM or DPCM, which generates a large amount of encoded data but can obtain a decoded signal without distortion. The encoding unit 4b performs encoding such as quantization so as to reduce the amount of encoded data generated to some extent.
The encoding unit 4c performs very coarse encoding such as outputting only the average value of the pixel levels in the area.

At the time of encoding of each area, a class identification signal 103 synchronized with the image signal 102 of the area is supplied to the switch unit 6. The switch unit 6 switches according to the class identification signal 103 so that the image signal 102 of each area is input to the encoding unit corresponding to the class of the area. For example, if the region of the image signal 102 is the class with the highest importance, such as a moving person,
Encoding is performed by the encoding unit 4a that can obtain a high-quality image on the decoding side, and encoded data 105a is obtained. In the case of a region included in a class of intermediate importance, such as a stationary person, the encoding is performed by the encoding unit 4b, and the encoded data 105
Obtain b. Further, in the case of a region of a low importance class, such as a background, in which a signal on the decoding side may be significantly different from an input signal, the encoding unit 4c encodes the encoded data 1
05c is obtained.

As described above, in the present embodiment, the encoding efficiency can be improved by reducing the amount of encoded data generated in the region of low importance.

In this embodiment, the case where the region is divided into three stages is described. However, when the region is divided into other stages, the same number of different encoding systems as the number of stages are used. Parts may be provided.

The number of encoding units does not necessarily have to match the number of classes, and the number of encoding units can be reduced by changing the encoding control parameters as in the first embodiment. It is also possible. That is, two encoding units are provided for three classes, and one of the encoding units is assigned to 2 classes.
In accordance with one class, the encoding unit changes the encoding control parameter according to the class. This allows
Similar effects can be obtained.

Further, the encoding method in each of the encoding units 4a to 4c is not limited to the above-mentioned examples, but the quality of the decoded image required for each class and the allowable amount of encoded data generation amount , An appropriate method may be used for each class.

Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram showing the configuration of the third embodiment.

In the present embodiment, control according to the class of each area is applied to encoding using inter-frame prediction. Coding by inter-frame prediction, as is well known,
A prediction signal is generated by predicting an image of the current frame from an image of a past frame stored in the memory, and a prediction error signal is obtained from the prediction signal and the input actual image signal by, for example, taking a difference. Is obtained, and the prediction error signal is encoded and output. In this case, a device for obtaining a similar prediction signal is provided on the decoding side, and the decoding side forms a decoded signal by adding the prediction error signal sent from the encoding side to the prediction signal.

In FIG. 5, the operations of the area dividing section 1 and the hierarchical section 2 are the same as those in the first embodiment.

In this embodiment, a plurality of memories (frame memories, etc.) 10a, 10b, and 10c are provided, and each memory stores an image of a specific class. That is, the configuration in FIG. 5 shows an example in which the areas are classified into three classes. Hereinafter, for convenience, the memory 10
a as the memory for the area of the most important class, the memory 10b as the memory for the intermediate class,
The memory 10c is used as a memory for the least important area. In this case, for example, when an image as shown in FIG. 2 is encoded, as shown in FIG. 6, data of a moving person area 201 is stored in the memory 10a, and The data of the person's area 202 is stored, and the data of the foreground and background desk 203, the painting 204, the clock 205, the wall 206, and the like are stored in the memory 10c.

The contents of the memories 10a to 10c are images obtained by encoding and decoding the input image signal, and represent the images of the past frames, and each frame or each frame at a certain interval. This process will be described later.

When encoding the image signal 102 of each area divided by the area dividing section 1, the prediction section 12 stores an image signal similar to the image signal 102 of each area in the memories 10a to 10c.
Search from. That is, in the present embodiment, the area dividing unit 1
, The image signal 102 of each divided area is sequentially output collectively for each area, and the prediction unit 12 outputs a set of image signals of each area and image data (image data) of each area stored in the memories 10a to 10c. An image similar to each region to be encoded is found from the image in the memory by comparing the group of signals. For this comparison, for example, a method is conceivable in which a difference between the image signal 102 of the region to be encoded and each image data in the memory is obtained, and the image data in the memory having the smallest difference signal is regarded as a similar image. Can be

The similar image signal obtained by searching the memory in this way is used by the prediction section 12 as a prediction source image signal. That is, the prediction unit 12
Performs prediction using a predetermined prediction function based on the prediction source image signal, and generates a prediction signal 112. The generation of the prediction signal is performed for each of the regions divided by the region dividing unit 1. The prediction signal 112 is output to the prediction error extraction unit 14 and the signal synthesis unit 17. Also, the prediction unit 1
2 outputs a signal 114 indicating the number of the memory in which the similar image used for the prediction is stored and the address of the similar image on the memory at the same time as the prediction signal 112 toward the decoding side. On the decoding side, a prediction unit and three memories similar to those of the image encoding apparatus are provided, and the signal 114 indicating the memory number and the address is used when the decoding side generates a prediction signal.

The prediction error extraction unit 14 calculates a prediction error signal 11 from the image signal 102 and the prediction signal 112 for each region.
6 is extracted. As a method of extracting the prediction error signal 116, a difference between the image signal 102 and the prediction signal 112 may be simply obtained, or a difference may be obtained after performing a predetermined weighting on both.

The prediction error signal 114 is encoded by the encoding unit 4 according to a predetermined encoding method. And the encoding unit 4
Outputs encoded data 105 obtained by encoding the prediction error signal.

The decoding unit 16 decodes the coded data 105 of the prediction error signal output from the coding unit 4 by performing the operation reverse to that of the coding unit 4 to obtain the decoded prediction error signal 12.
Outputs 0. The decoded prediction error signal 120 is combined with the above-described prediction signal 112 by the signal combining unit 17. As a result, a decoded image signal 122 is generated. As a combining method, there is a method of simply adding, or a method of adding weights after adding both, and these depend on a method of extracting a prediction error in the prediction error extracting unit 14.

The decoded image signal 122 is supplied to the memory controller 18
Is input to The memory control unit 18 switches according to the class identification signal 103 output from the hierarchical unit 2 so that the decoded image signal 122 is input to the memory corresponding to the class of each area for each signal of each area. Switch. The data stored in the memories 10a to 10c is updated every time the decoded image signal 122 is input.

At the time of this update, a memory (memory 10c in this embodiment) for storing an area such as a background having no image motion is of a type in which stored data is simply rewritten to a newly inputted decoded image signal. Instead, it is also possible to use a memory that stores (overwrites) a newly input decoded image signal on past stored data.

In this case, for example, assuming that the person 201 has moved in the image shown in FIG.
The movement of 01 causes each part of the background in the image to appear and disappear, and when the contents of the memory 10c that stores the image data of each area of the background are updated, the background area stored in the memory 10c is sequentially large. Will cover the part. Therefore, for example, in a frame, when the person 201 moves on the screen and the background of the part behind the person originally appears, according to this configuration, the decoding side has the same memory content as the present apparatus. In this case, only the address information of the newly appearing background area is sent from the encoding apparatus, and the decoding side can read the image data of the wall or the picture from the memory and form an image. Therefore, once all the background images have been formed in the memories on both the encoding side and the decoding side, it is no longer necessary to transmit the encoded data from the encoding side, and only the address information is transmitted. It will be enough. Therefore, the generation amount of the encoded data is sequentially reduced.

Here, the encoding unit 4 encodes all the prediction error signals 116 for each area with constant encoding control parameters. However, as in the first embodiment, the encoding unit 4 sends the prediction error signal 116 from the hierarchizing unit 2. It is also possible to adopt a configuration in which the encoding control parameter of the encoding unit 4 is changed and controlled by the class identification signal 103 signal. As a result, the coding efficiency is further improved.

In this embodiment, only one encoding unit is provided. However, as in the second embodiment, a plurality of encoding units having different encoding systems are provided and A configuration in which the encoding unit that performs encoding for each area is changed according to the class identification signal. Here, if the encoding unit that encodes an area of a certain class is an encoding unit that does not perform the prediction error calculation, the prediction is not performed, and the corresponding memory is unnecessary.

In the present embodiment, when generating the prediction signal 112, the prediction unit 12 searches all the memories for image data similar to the image signal 102 obtained by dividing the input current frame into regions. However, the configuration may be such that the prediction unit 12 is controlled by the class identification signal 103 from the hierarchy unit 2 to search only the memory corresponding to the class of each area. In this case, the search for similar image data is facilitated, so that the coding efficiency can be improved.

Further, in this embodiment, the contents of the memory are:
Although rewriting is performed every time a signal is encoded and decoded by the encoding unit 4 and the decoding unit 4, a configuration in which rewriting of the memory is performed at a preset time interval may be adopted. In other words, the memory corresponding to the class of higher importance is frequently rewritten, and the time interval of memory rewriting is increased for the class of lower importance. This speeds up the process of updating the memory.

Here, when background information is not particularly necessary in a video conference or the like, the rewriting interval of the memory for storing the background information may be made infinite, that is, the rewriting may not be performed. in this case,
It is necessary to store the background information in the background information memory in advance. For example, in a video conference, it can be realized by a method of recording background information in a memory in advance before the conference starts. FIG. 7 shows a block diagram of an apparatus having such a configuration. This apparatus differs from the apparatus of FIG. 5 in that the decoded image signal 122 is not input to the memory 10c.

In the present embodiment, at the time of prediction, only one of the most similar image signals is searched from the memory and used for prediction. However, the present embodiment is based on ISO MPEG (Moving Pilot).
In the case of applying to the MPEG1 standard recommended by the “Cture Experts Group”, it is also possible to read a plurality of similar image signals from the memory and combine them. In MPEG1, continuous frames are not sequentially encoded in time series. For example, after encoding the first frame, the fourth frame is encoded, and then the second and third frames are encoded. , The coding order is changed. According to this method, image information close to the images of the second and third frames can be obtained by using the information of the first and fourth frames, for example, by averaging them, and the like. By performing prediction based on this, it is possible to reduce the power of the prediction error signal, that is, to reduce the amount of encoded data generated. In this embodiment, if each of the memories 10a to 10c can store image data for a plurality of frames for each frame, the apparatus of this embodiment can be applied to such a standard.

In this embodiment, the shape and size of the image area indicated by the prediction source image signal read from the memories 10a to 10c in the prediction unit 12 are determined by the shape and size of the area of the image signal 102 to be encoded. If the size differs from the size, it can be said that interpolation and extrapolation are performed, and the prediction is performed after the shape and size of the region of the prediction source image signal are adjusted to the shape and size of the region of the image signal 102. .
Since the prediction source image signal and the image signal 102 are shifted in time by one frame (or several frames), shapes and sizes of images showing the same object may be different. Here, when the prediction unit 12 performs prediction using prediction source image signals having different shapes and sizes, the power of the prediction error signal increases, which leads to an increase in the amount of encoded data generated. Therefore, here, an image signal having the same shape and size as the area of the image signal 102 is formed by interpolating by interpolation and extrapolation into the area of the image indicated by the prediction original image signal, and the prediction is performed based on this. To FIG.
FIG. 8 is a diagram for explaining the state of interpolation, in which the outline of the region of the image signal 102 to be encoded is indicated by a thick line in FIG. The outline of the area of the original image signal) is shown by the thick line in FIG. In the drawing, white circles and black circles represent pixels, respectively, and particularly, black circles represent pixels included in the area of the similar image signal in the memory. In such a case, when the prediction is performed, extrapolation prediction is performed because information on the shaded pixel portion in (B) cannot be directly obtained from (A). If a simple method is adopted, a method of extending the same value as the pixel at the end point of the area (A) to the outside can be applied. Also, a method of performing interpolation by setting the value of each pixel in the hatched portion to a value proportional to the distance from the center of the region to the pixel can be used. Here, the case where the area of the similar image signal is expanded has been described, but the same applies to the case where the area is reduced. In any case, the interpolation is performed in this manner, and the shape and size of the region indicated by the prediction source image signal are made to match the shape and size of the region to be encoded, and then the prediction is performed. Therefore, the amount of encoded data can be reduced.

[0071]

As described above, according to the present invention,
The input image is divided into regions according to the content of the image, and the divided regions are classified according to their visual importance.
By changing the coding control parameters and coding method according to the class of the region when coding each region, by suppressing the occurrence of coded data of visually insignificant regions,
The amount of encoded data generated can be reduced while maintaining the quality of an image in a visually important area.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a first embodiment of an image encoding device according to the present invention.

FIG. 2 is a diagram illustrating an example of an input image to an image encoding device.

FIG. 3 is a diagram for explaining sub-sampling.

FIG. 4 is a block diagram showing a second embodiment of the image encoding device according to the present invention.

FIG. 5 is a block diagram showing a third embodiment of the image encoding device according to the present invention.

FIG. 6 is a diagram for explaining memory contents of memories 10a to 10c in a third embodiment.

FIG. 7 is a block diagram in a case where one memory is not updated in the third embodiment.

FIG. 8 is a diagram for explaining interpolation of a similar image signal read from a memory in the third embodiment.

FIG. 9 is a block diagram of a conventional image encoding device.

[Explanation of symbols]

1 area dividing section, 2 layering section, 4 coding section, 101
Input digital image signal, 102 area-divided image signal, 103 class identification signal, 105 encoded data.

Claims (21)

[Claims] 1. An area dividing unit that divides an input image into a plurality of areas and outputs an image signal to which area dividing information is added, and classifies each area divided according to a predetermined criterion. A layering unit that outputs a class identification signal indicating a class of each area; and coded data including area division information, which encodes the area-divided image signal based on a set encoding control parameter. And an encoding unit that outputs an encoding control parameter according to a class identification signal given to each region from the hierarchical unit in the encoding unit. Encoding device. 2. The image coding apparatus according to claim 1, wherein the area dividing unit detects an edge of the input image, performs area division of the input image based on the edge, and An image coding apparatus characterized in that a class is determined in accordance with the shape of an edge of an image. 3. The image encoding device according to claim 1, wherein the region dividing unit analyzes a texture of the input image, performs region division of the input image based on the texture, and the layering unit includes: An image coding apparatus characterized in that a class is determined according to the type of texture of the image. 4. The image encoding apparatus according to claim 1, wherein the region dividing unit measures the size of the activity of the input image, performs region division of the input image based on the size of the activity, and Is an image coding apparatus for determining a class according to the size of the activity of a divided image. 5. The image encoding apparatus according to claim 1, wherein the region dividing unit has an image memory for storing a past image, and compares the past image stored in the image memory with the input image. Detecting the motion of the input image, dividing the area based on the detected motion magnitude, and determining the class according to the motion magnitude of the divided image. Image coding device. 6. The image coding apparatus according to claim 5, wherein the layering unit has means for storing the magnitude of the motion detected by the region dividing unit for a predetermined number of frames. An image coding apparatus characterized by measuring the frequency of motion and determining a class according to the measured frequency of motion. 7. The image encoding apparatus according to claim 1, wherein a time interval at which encoding is performed is controlled according to a class identification signal assigned to each area. Image encoding device. 8. The image encoding apparatus according to claim 1, wherein a quantization parameter at the time of encoding is controlled according to a class identification signal assigned to each area. Image encoding device. 9. The image encoding device according to claim 1, wherein the encoding unit has a function of performing subsampling on the image signal, and class identification assigned to each region. An image coding apparatus, wherein the size of subsampling performed at the time of coding is controlled according to a signal. 10. The image encoding apparatus according to claim 1, wherein a threshold value of an amount of encoded data is set according to a class identification signal assigned to each area, An image coding apparatus, wherein the control of the coding unit is performed according to the threshold value. 11. An image decoding apparatus which receives coded data divided into a plurality of regions and coded for each region, and obtains a decoded image by decoding the coded data. For the region where is obtained, the region division information is separated from the encoded data, and decoding is performed for each region based on the separated region division information in accordance with the region identification signal given at the time of encoding. An image decoding device for outputting an image signal. 12. The image decoding apparatus according to claim 11, wherein a previously stored image signal is output to an area where encoded data is not obtained. 13. The image decoding apparatus according to claim 11, wherein a decoded image of a previous frame is output as it is for an area where encoded data is not obtained. 14. An area dividing unit that divides an input image into a plurality of areas and outputs an image signal to which area dividing information is added, and classifies each area divided according to a predetermined criterion. A layering unit that outputs a class identification signal indicating a class of each region; a plurality of image memories that store past image signals; and a plurality of image signals similar to the region-divided input image signal as prediction source image signals. From the image memory, generates a prediction signal from the prediction source image signal using a prediction function, and outputs the prediction signal and the number of the memory from which the prediction source image signal was read, and the address of the prediction source image signal in the memory A prediction unit, a prediction error extraction unit for extracting and outputting a prediction error signal from the input image signal and the prediction signal obtained by dividing the region, for each region; An encoding unit that outputs encoded data including the division information; a decoding unit that decodes the encoded data output from the encoding unit and outputs a decoded prediction error signal; and a decoding prediction error signal for each region. And a signal combining unit that combines the decoded image signal from the prediction signal, and using the decoded image signal for each region output from the signal combining unit, based on the class identification signal from the hierarchical unit, A memory control unit for updating the storage contents of the plurality of memories by distributing and storing the image data in an image memory. 15. The image encoding apparatus according to claim 14, wherein the encoding unit is controlled according to a class identification signal assigned to each area. 16. The image coding apparatus according to claim 14, wherein a plurality of coding units for performing coding by different coding methods and outputting coded data including region division information are provided. And a switch unit for selecting any one of the plurality of encoding units according to the class identification signal given to the image encoding device. 17. The image encoding apparatus according to claim 14, wherein a prediction source image signal is read out from only a specific image memory for a plurality of image memories in accordance with a class identification signal assigned to each area. An image encoding device, characterized in that: 18. The image coding apparatus according to claim 14, wherein a time interval for updating the storage contents of the image memory using the decoded image signal is controlled according to the class identification signal. apparatus. 19. The image encoding apparatus according to claim 14, wherein the image memory stores image signals for a plurality of frames, and the prediction unit reads the image signals for a plurality of frames from the image memory and encodes the image signals. An image coding apparatus for generating a prediction signal by combining. 20. An image decoding apparatus which receives coded data divided into a plurality of regions and coded for each region, and obtains a decoded image by decoding the coded data. A plurality of image memories in which signals are stored based on a class identification signal for each region, and the number of the image memory output from the image encoding device and the address of the prediction source image signal in the memory are input;
A prediction unit that outputs an image signal specified from the input image memory number and address as a prediction signal, and separates region division information from encoded data, and for each region based on the separated region division information, A decoding unit that decodes the encoded data according to the class identification signal used at the time of encoding and outputs a decoded prediction error signal, and decodes a decoded image signal from the decoded prediction error signal and the prediction signal for each region. A signal synthesizing unit to be synthesized, and the decoded image signals synthesized for each region output from the signal synthesizing unit are sorted and stored in the plurality of image memories based on a class identification signal for each of the regions. A memory control unit for updating storage contents of the plurality of image memories. 21. The image decoding device according to claim 20, wherein an image signal is stored in advance for a part of the plurality of image memories, and the decoded image signal is used for the part of the image memory. An image decoding apparatus characterized in that the stored content is not updated by the user.