JP4404556B2 - Video encoding method and system, and video decoding method and system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing system and method, and more particularly to an image signal encoding system and decoding system having an image object having an arbitrary shape and constituting an input / output video component.
[0002]
[Prior art]
In recent years, moving image coding means in units of objects has been actively used in moving image coding, and coding in units of objects is also adopted in MPEG-4, which is being standardized by ISO / IEC. In this method, an image object (hereinafter, also simply referred to as “object”) that is a component constituting an image is encoded together with shape information representing its outline, and these objects are combined in a decoder to generate an encoding target image. Reconfigure.
[0003]
In order to encode an object having an arbitrary shape, a rectangular area having this inside is set as an encoding target area, and a conventionally used encoding method for a moving image having a rectangular shape is applied.
[0004]
However, when the pixel value outside the object is encoded as 0 in the encoding target region, the value of the prediction residual component of the pixel becomes large at the boundary of the object due to the inter-frame prediction process. As a result, the prediction efficiency is greatly impaired, causing the encoding efficiency to deteriorate.
[0005]
In order to solve this problem, the pixel value near the boundary is substituted around the object and encoded as a rectangular area. When the object is synthesized in the decoder, the shape information is used to obscure the pixels outside the object. A method called “padding” is used.
[0006]
FIG. 14 is a diagram illustrating an example of a configuration of an object-based moving image encoding system including padding processing, which is adopted in ISO / IEC 14496-2MPEG-4.
[0007]
Referring to FIG. 14, the input image encoding means 103 encodes an input image 1000 having a rectangular shape. The first storage unit 101 stores the image signal 1001 transferred from the input image encoding unit 103 and subjected to padding processing. The second storage unit 102 stores shape information 1002.
[0008]
The padding processing unit 105 transfers the address 1005 representing the pixel position in the shape information stored in the second storage unit 102 to the second storage unit 102, and the shape information stored in the second storage unit 102 1008 is read, and a pixel outside the object in the image signal stored in the first storage unit 101 is identified.
[0009]
Then, the padding processing unit 105 transfers the address 1007 representing the pixel position outside the object in the image signal to the first storage unit 101, and refers to the image signal 1008 stored in the first storage unit 101. Then, the pixel value to be substituted into the address 1007 is calculated and substituted into the address 1007.
[0010]
The padding process in the padding processing unit 105 is realized by performing the padding process for the one-dimensional pixel column twice.
[0011]
FIG. 15 is a diagram illustrating an area to be padded within the encoding target area. In the encoding target area 40, padding processing is first performed on a set 42 of pixel columns including the object in the horizontal direction outside the object 41. Thereafter, padding processing in the vertical direction is performed on the region 43 not included in the pixel column set 42. Based on the image signal 1009 in which an appropriate pixel value is embedded outside the object by the above processing, the input image encoding unit 103 encodes the input image 1000 and inputs the shape information 1002 to the shape information encoding unit 104. The data is transferred or stored as encoded data together with the shape information encoded by.
[0012]
The image signal 1001 to be subjected to the padding process is an input image signal transferred from the input image encoding unit 103 or a reference image generated in the input image encoding unit 103 and used for inter-frame prediction. It may be. In addition, when the encoder performs padding processing on a reference image used for inter-frame prediction, the decoder also needs to perform padding processing on the reference image.
[0013]
[Problems to be solved by the invention]
In the conventional system described above, when the object boundary is identified in the padding process, all the pixels in the encoding target area are searched twice regardless of the shape of the object. have.
[0014]
Therefore, the present invention has been made in view of the above-described problems, and an object of the present invention is to provide a padding processing method that reduces the amount of computation during search by using processing during encoding and decoding of shape information. It is an object to provide a moving image encoding method, a moving image decoding method, and a system using the above.
[0015]
[Means for Solving the Problems]
The moving image encoding system of the present invention that achieves the above object is provided for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape, and is an external area of the image object in the rectangular area. A video encoding system having a padding process for substituting an appropriate pixel value into an input image encoding means for encoding an input image signal having an image object therein, and padding in the input image encoding means A first storage unit that stores an image signal having an image object to be processed and, if necessary, transferred to the input image encoding unit; and a shape that represents the shape of the image object in the rectangular region A format for generating encoded shape data by encoding information and generating boundary information indicating the position of the boundary of the image object Information encoding means, second storage means for storing the boundary information, and boundary information stored in the second storage means are read out as necessary and stored in the first storage means. A padding for calculating an address representing a pixel position outside the image object, calculating a pixel value corresponding to the address from the image signal stored in the first storage means, and substituting And processing means.
[0016]
The moving image decoding system according to the present invention applies an appropriate pixel value to an external region of the image object in the rectangular shape region for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape. An image decoding system having a padding process to be substituted, wherein the image decoding means generates an image signal by decoding encoded image data having an image object having an arbitrary shape inside with reference to a reference image And a first storage means for storing the decoded image signal as an image signal to be subjected to padding processing and supplying as a reference image in the decoding processing of the next frame in the image decoding means, and shape information of the image object The encoded shape data, which is encoded data, is decoded to generate shape information, and the position of the boundary of the image object is determined. Shape information decoding means for generating boundary information, second storage means for storing the boundary information, and boundary information stored in the second storage means are read out as necessary, and An address representing a pixel position outside the image object is calculated from the image signals stored in the storage means, and a pixel value corresponding to the address is calculated from the image signal stored in the first storage means And padding processing means for substitution.
[0017]
The moving image encoding method of the present invention is a pixel value suitable for an external area of the image object in the rectangular shape area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape. Is a moving image encoding method for performing padding processing for substituting, and (a) an input image signal having an image object therein is encoded, and an image signal having an image object to be padded is first storage means And (b) encoding shape information representing the shape of the image object in the rectangular shape region, generating encoded shape data, and generating boundary information representing the position of the boundary of the image object Storing in the second storage means; and (c) reading the boundary information stored in the second storage means as necessary. The address representing the pixel position outside the image object is calculated from the image signals stored in the first storage means, and the address is determined from the image signal stored in the first storage means. And a padding process for calculating and substituting the pixel value to be substituted.
[0018]
The moving image decoding method of the present invention provides an appropriate pixel value in an external area of the image object in the rectangular shape area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape. A moving picture decoding method for performing padding processing to be substituted, comprising: (a) decoding a coded image data having an image object having an arbitrary shape therein with reference to a reference image, and generating an image signal (B) storing the decoded image signal in the first storage means as an image signal to be padded, and (c) storing the image signal stored in the first storage means in the next frame. A step of reading as a reference image in the decoding process; and (d) generating shape information by decoding encoded shape data, which is data obtained by encoding the shape information of the image object. And generating boundary information representing the position of the boundary of the image object and storing the boundary information in a second storage means; and (e) boundary information stored in the second storage means as required. An address representing a pixel position outside the image object is calculated from the image signals stored in the first storage unit, and the address is calculated from the image signal stored in the first storage unit. And a padding process for calculating and substituting the pixel value corresponding to.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
A preferred embodiment of the present invention will be described. In the preferred embodiment of the moving image coding system of the present invention, referring to FIG. 1, an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape is included in a rectangular region. Of these, a moving picture coding system having padding processing for substituting appropriate pixel values into an external area of an image object, and an input picture coding means (103) for coding an input picture signal having the picture object inside A first storage unit (101) for storing an image signal having an image object to be subjected to padding processing in the input image encoding unit (103), and transferring it to the input image encoding unit as necessary; and a rectangular shape Encodes shape information that represents the shape of the image object in the area to generate encoded shape data, and The shape information encoding means (104) for generating boundary information representing the position, the second storage means (102) for storing the boundary information, and the boundary information stored in the second storage means (102) are required. Is read out, and an address representing a pixel position outside the image object is calculated from the image signals stored in the first storage means (101) and stored in the first storage means (101). Padding processing means (105) for calculating and substituting the pixel value corresponding to the address from the existing image signal.
[0020]
An image signal having an image object stored in the first storage means (101) and subjected to padding processing is an input image.
[0021]
An image signal stored in the first storage means (101) and having an image object to be subjected to padding processing is used as a reference image in inter-frame prediction.
[0022]
Referring to FIG. 2, the shape information encoding means (104) refers to the block dividing unit (106) that divides the shape information into a plurality of blocks, and the shape information in the block. A block determination unit (107) that generates block classification information indicating whether the block is on, inside, or on the boundary, and the block classification information is referred to. An intra-block shape information encoding unit (108) that encodes the internal shape information, and a multiplexing unit (109) that multiplexes the block classification information and the intra-block encoded shape information to generate encoded shape data. The block classification information and the in-block shape information are transferred to the second storage means (102) as boundary information.
[0023]
The padding processing means (105) refers to the block classification information stored in the second storage means (102), and the block stored in the second storage means only when the block is on the boundary of the image object. Reference is made to the internal shape information, and an address calculation unit (110 in FIG. 3) for calculating an address representing a pixel position outside the image object from the block classification information and the internal shape information is stored in the first storage means. A padding value substitution unit (111 in FIG. 3) is provided that refers to the image signal, calculates a padding value that is a pixel value corresponding to the address, and performs a substitution process of the padding value to the address.
[0024]
In the preferred embodiment of the video decoding system of the present invention, referring to FIG. 9, an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape is included in the rectangular region. A moving image decoding system having padding processing for assigning appropriate pixel values to an external area of an image object, which refers to a reference image and decodes encoded image data having an image object having an arbitrary shape inside The image decoding means (112) for generating an image signal, the decoded image signal is stored as an image signal to be subjected to padding processing, and is supplied as a reference image in the decoding processing of the next frame in the image decoding means. 1 storage means (101) and the encoded shape data, which is data obtained by encoding the shape information of the image object, are decoded to generate the shape information. In addition, shape information decoding means (113) for generating boundary information representing the position of the boundary of the image object, second storage means (102) for storing boundary information, and second storage means are stored. The boundary information is read as necessary, and an address representing a pixel position outside the image object is calculated from the image signal stored in the first storage unit, and stored in the first storage unit (101). Padding processing means (105) for calculating and substituting the pixel value corresponding to the address from the image signal.
[0025]
In a preferred embodiment of the moving picture decoding system of the present invention, the shape information decoding means (113) is outside the image object, and the block obtained by demultiplexing the encoded shape data and dividing the rectangular shape area is outside the image object. A demultiplexing unit (114 in FIG. 10) for generating block classification information indicating whether the block is inside or on the boundary, and intra-block coding shape information of the block on the boundary of the image object, Shape information of the entire image object is generated with reference to the intra-block shape information decoding unit (115 in FIG. 10) that decodes the intra-block shape information from the inner encoded shape information, the block classification information, and the intra-block shape information. A shape information integration unit (116 in FIG. 10), and transfers the block classification information and the in-block shape information as boundary information to the second storage means. Configured.
[0026]
Further, the padding processing means refers to the block classification information stored in the second storage means (102), and the block stored in the second storage means only when the block is on the boundary of the image object. An address calculation unit (110 in FIG. 3) that refers to shape information and calculates an address representing a pixel position outside the image object from block classification information and in-block shape information, and an image stored in the first storage means A padding value substitution unit (111 of 111) that calculates a padding value that is a pixel value corresponding to the address by referring to the signal and performs substitution processing of the address padding value.
[0027]
In the moving image encoding system of the present invention, the input moving image encoding means (103) decomposes the input image into encoding units, and detects the amount of motion between the input image and the reference image for each encoding unit. Then, a motion compensation is performed using the motion amount to create a predicted image, and a means for encoding a difference between the predicted image and the input image is provided, and a padding processing means converts the padding area into the motion amount. Limited to the inside of the detection range.
[0028]
In the moving image encoding system of the present invention, the image decoding unit generates a predicted image by decoding a motion amount and a difference image for each encoding unit and performing motion compensation on the reference image using the motion amount. And a means for reconstructing an image encoded by adding the difference image, and the padding processing means limits the padding area within the detection range of the motion vector.
[0029]
In an embodiment of the moving image encoding system of the present invention and the moving image decoding system of the present invention, the padding processing means first performs the padding region identification processing using a horizontal pixel row as a processing unit, After performing a padding value substitution process for a pixel row in the horizontal direction having pixels included in the image object, a padding value is substituted by the padding value substitution process in the horizontal direction among the pixels outside the image object. A padding value substitution process is performed on the pixels that have not been detected using the vertical pixel column as a processing unit.
[0030]
In the video encoding system of the present invention and the video decoding system of the present invention, the padding processing means performs a padding value substitution process for a plurality of adjacent pixel columns in a direction orthogonal to the pixel columns. This is achieved by copying the padding values consecutive in
[0031]
The moving image encoding method of the present invention is suitable for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape, and is suitable for an external region of the image object in the rectangular shape region. A moving image encoding method for performing padding processing for substituting values, wherein (a) an input image signal having an image object therein is encoded, and an image signal having an image object to be padded is first stored. (B) encoding the shape information representing the shape of the image object in the rectangular shape region to generate encoded shape data and generating boundary information representing the position of the boundary of the image object (C) the boundary information stored in the second storage means is read out as needed, and the first storage means stores the boundary information stored in the second storage means; An address representing a pixel position outside the image object is calculated from the image signals stored in the storage means, and a pixel value corresponding to the address is calculated from the image signal stored in the first storage means. Each of the above-described processes (a) to (c) including the padding process to calculate and substitute is included.
[0032]
In the embodiment of the present invention, each of the processes (a) to (c) may be realized by program control executed on a computer constituting the moving image encoding apparatus. The present invention is implemented by loading a program from a storage medium storing the program into a memory of a computer (which may be a digital signal processor for image processing), and the present invention includes this recording medium. .
[0033]
The moving image decoding method of the present invention provides an appropriate pixel value in an external area of the image object in the rectangular shape area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape. A moving picture decoding method for performing padding processing to be substituted, wherein (a) a process of generating an image signal by decoding encoded image data having an image object having an arbitrary shape inside with reference to a reference image (B) a process of storing the decoded image signal in the first storage means as an image signal to be padded, and (c) a decoding process of the next frame of the image signal stored in the first storage means (D) processing for reading out as a reference image in (d) generating shape information by decoding encoded shape data, which is data obtained by encoding shape information of the image object, Processing for generating boundary information representing the position of the boundary of the image object and storing it in the second storage means; (e) reading the boundary information stored in the second storage means as necessary; Of the image signals stored in the first storage unit, an address representing a pixel position outside the image object is calculated, and a pixel value corresponding to the address is calculated from the image signal stored in the first storage unit Including the above-described processes (a) to (e).
[0034]
In the embodiment of the present invention, each of the processes (a) to (e) may be realized by program control executed on a computer constituting the moving image decoding apparatus. Is implemented by loading a program from a storage medium storing the program into a memory of a computer (which may be a digital signal processor for image processing as a computer), and the present invention also includes this recording medium.
[0035]
【Example】
Next, embodiments of the present invention will be described in detail with reference to the drawings.
[0036]
FIG. 1 is a diagram showing a configuration of a moving image encoding system according to an embodiment of the present invention. In the following, an example in which the present invention is applied when padding processing is performed on a reference image used for inter-frame prediction will be described. The same padding process as in the embodiment can be applied.
[0037]
Referring to FIG. 1, input image encoding means 103 encodes input image signal 1000 to generate encoded image data 1010. Further, an image signal 1001 representing a decoded image of the current frame is generated and stored in the first storage unit 101. When encoding the next frame, the input image encoding unit 103 refers to this image signal as the reference image 1009 and performs inter-frame prediction.
[0038]
The shape information encoding unit 104 encodes the object shape information 1002 inside the input image signal 1000 to generate encoded shape information 1003. At the same time, boundary information 1004 representing the position of the boundary of the object is transferred to the second storage means 102 within the encoding target area.
[0039]
The padding processing unit 105 designates an address 1005 for switching boundary information to be referred to in the second storage unit 102 at each stage of the processing process, and receives boundary information 1006 corresponding thereto. Then, referring to the boundary information 1006, a region corresponding to the outside of the object is identified from among the image signals stored in the first storage unit 101, and a padding value that is a pixel value to be substituted into the corresponding region is calculated. Then, the value is assigned to the corresponding area in the first storage unit 101.
[0040]
The padding processing unit 105 sends the address 1007 indicating the position of the pixel to be referred to in the padding value calculation process or the position of the pixel to be substituted to the first storage unit 101, and reads / writes the pixel value 1008.
[0041]
FIG. 2 is a diagram showing an example of a configuration for outputting encoded shape information defined by ISO / IEC 14496-2 MPEG-4 in the shape information encoding means 104 of FIG.
[0042]
Referring to FIG. 2, the block division unit 106 divides the shape information 1002 into the shape information 1011 inside each block for a plurality of blocks obtained by dividing the encoding target region, and sends the shape information 1011 to the block determination unit 107.
[0043]
The block determination unit 107 generates block classification information 1012 indicating whether the block is inside the object, outside the object, or on the object boundary from the shape information 1011 inside the block.
[0044]
The intra-block shape information encoding unit 108 refers to the block classification information 1012, encodes the intra-block shape information 1011 corresponding to the case where the block is on the boundary, and generates encoded intra-block shape information 1013.
[0045]
The multiplexing unit 109 multiplexes the block classification information 1012 and the intra-block encoded shape information 1013 into the encoded shape information 1003.
[0046]
If the encoding target area is a rectangle having a width W and a height H, and a block to be divided is a square having a side B, the block classification information is a matrix having a width W / B and a height H / B as shown in FIG. Represented as:
[0047]
In FIG. 11, black (represented by the hatched portion from the upper right end of the block to the lower left end), white, and gray (represented by the hatched portion from the upper left end of the block to the lower right end). Each block represents outside the object, inside the object, and on the boundary of the object.
[0048]
The shape information encoding unit 103 transfers the block classification information 1012 and the intra-block shape information 1011 as boundary information 1004 to the storage unit 102.
[0049]
FIG. 3 is a diagram showing an example of the configuration of the padding processing means 105 in one embodiment of the present invention. Referring to FIG. 3, the padding processing unit 105 includes an address calculation unit 110 and a padding substitution unit 111.
[0050]
FIG. 4 is a flowchart showing the flow of processing in the padding processing means 105 in one embodiment of the present invention. The operation of the padding processing means 105 will be described with reference to FIGS.
[0051]
The address calculation unit 110 searches for block classification information using the block classification information 1006 stored in the second storage unit 102 as the start position (step 201).
[0052]
If the block is on the boundary of the object (step 202), the corresponding in-block shape information 1006 is read out, and the padding area to be padded is identified (step 203). Then, padding processing of the pixel row in the horizontal direction is performed (step 204).
[0053]
The processes in steps 202 to 204 are continued until the search for block classification information is completed (step 205).
[0054]
Thereafter, the left end of the encoding target region is set as a start position (step 206), and padding value substitution processing is performed on the vertical pixel columns (step 207).
[0055]
The padding value substitution position is updated (step 208), and the padding process in the vertical direction repeats the processes of steps 207 and 208, in which the target pixel column reaches the right end of the encoding target area (step 209).
[0056]
Hereinafter, the padding process will be described in the order.
[0057]
FIG. 5 is a flowchart showing details of the padding area identification process (step 202) in FIG. In FIG. 5, Y represents the y coordinate of the upper left end point of the block on the boundary, and Y ′ represents the y coordinate of the upper left end point of the block referenced immediately before. FIG. 12 is a diagram showing coordinates in the encoding target area. A horizontal direction is an x-axis and a vertical direction is a y-axis with the upper left end point of the encoding target region as an origin.
[0058]
In order to store the left end point of the identified padding area, the address calculation unit 110 has B registers, register 0 to register B-1, and stores pixel coordinates in each of them. Here, (xi, yi) represents the coordinates stored in the register i. In addition, a memory for storing a y-coordinate representing a padding area to be subjected to padding processing in the vertical direction is incorporated.
[0059]
First, the address calculation unit 110 determines Y by referring to the block classification information 1006 and compares it with Y ′ (step 210).
[0060]
When Y and Y ′ are equal, the in-block shape information 1006 corresponding to the corresponding block is searched for each pixel, and the coordinates of the right end of the padding area and the left end of the padding area are identified. The pixel that is itself outside the object and the right adjacent pixel is inside the object is the right edge of the padding area, and the pixel that is itself outside the object and the left adjacent pixel is inside the object is the left edge of the padding area.
[0061]
If the coordinate (x, y) is the left end point (YES in step 212), the coordinate (x, y) is stored in the corresponding register y-Y (step 212).
[0062]
If the coordinate (x, y) is the right end point, it is determined whether or not the left end point corresponding to the register y-Y is stored (step 213).
[0063]
If coordinates are stored in the register y-Y, the corresponding address 1014 in the image signal stored in the storage means 101 is calculated by using (xi, yi) to (x, y) as padding areas. The data is transferred to the padding value substitution unit 111 (step 214).
[0064]
At this time, yi = y from step 243. Thereafter, the register y-Y is reset (step 215).
[0065]
If no coordinates are stored in the register y-Y, the address 1014 is calculated using the leftmost coordinates (0, y) to (x, y) of the encoding target area as the padding area, and the padding value substitution unit 111 is calculated. (Step 216).
[0066]
On the other hand, if Y is larger than Y ′ in step 210, it is determined whether or not coordinates are stored in each register before step 211 (step 218).
[0067]
If the coordinates are stored in the register i (NO in step 218), the padding value is calculated by calculating the address 1014 using (xi, yi) to the rightmost coordinates (W, yi) of the encoding target area as the padding area. After transferring to the assigning unit 111 (step 219), the register i is reset (step 220).
[0068]
On the other hand, when no coordinates are stored in the register i, it indicates that there is no pixel corresponding to the inside of the object in the pixel column whose y coordinate is Y ′ + i.
[0069]
The address calculation unit 110 stores the y coordinate Y ′ + i in the internal memory as a padding area in the vertical direction (step 221).
[0070]
The above processing is performed from the register 0 (step 217) to the register B-1 (steps 222 and 223).
[0071]
Thereafter, the address calculation unit 110 determines whether Y is larger than Y ′ + B (step 224). If Y is larger than Y ′ + B, a pixel string that does not include an object between the lower end of the block that has just undergone padding area identification processing and the upper end of the block that is currently performing padding area identification processing. Exist continuously. The address calculation unit 110 stores the y coordinate from the Y coordinate Y ′ + B + 1 to Y−1 as a padding area in the vertical direction in the internal memory (step 225).
[0072]
FIG. 6 is a flowchart showing details of the padding value substitution process (step 203) in the horizontal direction in FIG. The padding value substitution process in FIG. 6 is a means described in ISO / IEC 14496-2 MPEG-4, but the present invention can also be applied to other padding value substitution processes.
[0073]
When the left end point of the padding area is the left end point of the encoding target area, the padding value assigning unit 112 stores the pixel value 1008 on the right of the right end of the padding area as the padding value and is stored in the first storage unit 101. Substitute into the corresponding position of the pixel signal (step 233).
[0074]
When the right end point of the padding area is the right end point of the encoding target area, the pixel value 1008 adjacent to the left of the left end of the padding area is substituted as the padding value into the corresponding position of the pixel signal stored in the storage unit 101. (Step 235).
[0075]
On the other hand, if both end points of the padding area are not end points of the encoding target area, the first value of the pixel value 1008 on the right side of the right end of the padding area and the pixel value 1008 on the left side of the left end of the padding area Substituting into the corresponding position of the pixel signal stored in the storage means 101 (step 234).
[0076]
A search section, a padding area, and a padding value in a pixel row in the horizontal direction will be described with reference to the example shown in FIG.
[0077]
A broken line 41 in FIG. 13 represents the boundary of the object. In FIG. 13, blocks arranged in the horizontal direction represent block classification information, and colored blocks (hatched in the drawing) are blocks on the object boundary. Points 400 and 409 represent the left and right ends in the encoding target region in the pixel column. Here, the padding process of the pixel column from the pixel 400 to the pixel 409 will be described. The search target in this pixel column is an area inside the boundary block, that is, a section from pixel 401 to pixel 404 and a section from pixel 405 to pixel 408. The pixel 402, the pixel 403, the pixel 406, and the pixel 407 that are the end points of the padding area are identified by the search process described above. The padding values assigned to the padding area from pixel 400 to pixel 402, the padding area from pixel 403 to pixel 406, and the padding area from pixel 407 to pixel 409 are the pixel value of pixel 402, pixel 403, and pixel 406, respectively. Are the average of the pixel values and the pixel value of the pixel 407.
[0078]
Referring to FIG. 4 again, after the padding area identification process and the horizontal padding value substitution process are completed, the address calculation unit 110 determines the left end of the encoding target area as the start position of the padding process in the vertical direction ( Step 206).
[0079]
Then, padding value substitution processing for sequentially substituting the values of adjacent pixels that have been padded is performed using the vertical pixel row as a processing unit (step 207).
[0080]
The padding process for each pixel column in the vertical direction is the same as the padding value substitution process in the horizontal direction shown in FIG.
[0081]
In this embodiment, the shape information encoding means 104 has been described as an example in which the encoded shape information is composed of block classification information and intra-block encoded shape information. However, the outline of the object is expressed by a simple geometric figure. Of course, the present invention can also be applied to other shape information encoding methods using the outline of the object.
[0082]
In addition, when shape information encoding does not use approximate information, the shape information encoding means 104 has a new means for identifying the position of the block on the boundary of the object at the same time as the shape encoding. The present invention can also be implemented by transferring the boundary information 1004 to the storage unit 102.
[0083]
In the above-described embodiment, the case where the padding process is performed on all of the encoding target areas outside the object has been described. However, in the inter-frame prediction encoding with motion compensation, the padding area is within the motion vector detection range. It is also applicable to the case where is limited. In this case, if the identified padding area exceeds the detection range, the padding value substitution process may be performed only for blocks within the detection range from the boundary of the object.
[0084]
Next, a second embodiment of the present invention will be described. The configuration of the second embodiment of the present invention is the same as that of the embodiment described with reference to FIG. 1, and the processing in the padding processing means 105 is different. FIG. 7 is a flowchart showing the processing flow of the padding processing means 105 in the second embodiment of the present invention.
[0085]
In the flowchart of the embodiment shown in FIG. 4, the same configuration as that in the horizontal direction is applied to the padding value substitution processing in the vertical direction, whereas in the present embodiment, the horizontal direction is Different padding processing is performed (step 238).
[0086]
FIG. 8 is a flowchart showing the process flow of the padding value substitution process in the vertical direction.
[0087]
The padding value substitution unit 111 refers to the address 1014 transferred from the address calculation unit, and performs a padding value substitution process by copying the pixel area in the vertical padding area.
[0088]
If the upper end of the padding area is the upper end of the encoding target area, the pixel line immediately below the lower end of the padding area is copied to each pixel line in the padding area (step 233).
[0089]
If the lower end of the padding area is the lower end of the encoding target area, the pixel line immediately above the upper end of the padding area is copied to each pixel line in the padding area (step 235).
[0090]
If both ends of the padding area are different from the boundary of the encoding target area, the average of pixels having the same x coordinate is taken from the pixel line one above the upper end of the padding and the pixel line immediately below the lower end of the padding. The newly generated pixel column is copied to the pixel column in the padding area (step 234).
[0091]
According to the second embodiment of the present invention, pixel values continuously stored in the memory can be collectively copied to another memory space. Thereby, further speeding up of processing can be realized.
[0092]
FIG. 9 is a diagram showing the configuration of an embodiment of the moving picture decoding system of the present invention. Referring to FIG. 9, this moving image decoding system does not have the input image encoding unit 103 and the shape information encoding unit 104 as compared with the moving image encoding system shown in FIG. Shape information decoding means 113 is provided.
[0093]
Referring to FIG. 9, the image decoding unit 112 decodes the encoded image signal 1100 to generate an output image signal 1102. Here, the output image signal has an encoding target area represented by a rectangular outer frame, and has an object of an arbitrary shape therein.
[0094]
Further, the image decoding unit 112 stores the output image signal of the current frame in the storage unit 101. When this image signal is encoded in the next frame, inter-frame prediction is performed as a reference image 1009.
[0095]
The shape information decoding unit 113 decodes and outputs the object shape information 1103 based on the encoded shape data 1101 and stores the boundary information 1004 in the storage unit 102.
[0096]
The padding processing unit 105 sends a read request 1005 to the second storage unit 102 and receives boundary information 1006 for each processing unit for performing padding processing.
[0097]
An address 1007 representing an area corresponding to the outside of the object is calculated from the image signal stored in the first storage unit 101 with reference to the boundary information 1006, and the image stored in the first storage unit 101 is calculated. A pixel value 1008 to be substituted into the corresponding area is calculated with reference to the signal 1008 and substituted into the corresponding area in the first storage unit 101.
[0098]
Details of the processing in the padding processing unit 105 are the same as the padding processing unit 105 in the above-described embodiment, and a description thereof will be omitted.
[0099]
FIG. 10 is a diagram showing a configuration of the shape information decoding unit 113 in the embodiment of the moving image decoding system of the present invention.
[0100]
Referring to FIG. 10, the demultiplexer 114 divides the encoded shape data 1101 into block classification information 1106 and intra-block encoded shape information 1104.
[0101]
The intra-block shape information decoding unit 115 generates intra-block shape information 1105 by decoding the intra-block encoded shape information 1104.
[0102]
The shape information integration unit 116 refers to the block classification information 1106 and the in-block shape information 1105 to generate shape information 1103 for the entire object.
[0103]
Then, the shape information decoding unit 113 transfers the block classification information 1106 and the in-block shape information 1105 to the storage unit 102.
[0104]
【The invention's effect】
As described above, according to the present invention, the pixel search area in the padding process can be limited, the amount of calculation can be reduced, and high-speed processing can be realized.
[0105]
This is because, in the present invention, the search for the block having no object boundary is omitted by using the search section information obtained by encoding the shape information.
[0106]
As described above, in the conventional padding process, it is necessary to perform a search twice for all the pixels in the encoding target area, whereas in the present invention, the pixels that need to be searched are: It is only an area formed by blocks that are batched from the upper left end to the lower right end in FIG.
[0107]
Of the processes necessary to reduce the search for pixels, the object outline information is obtained by shape information encoding, so a new amount of computation is not required. Since the amount of calculation required is much smaller than that of the search, a significant reduction in the amount of calculation can be obtained by the present invention.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a first embodiment of a moving image encoding system of the present invention.
FIG. 2 is a diagram showing a configuration of shape information encoding means in the first embodiment of the present invention.
FIG. 3 is a diagram showing a configuration of padding processing means in the first exemplary embodiment of the present invention.
FIG. 4 is a flowchart showing a processing flow of padding processing means in the first exemplary embodiment of the present invention.
FIG. 5 is a flowchart showing details of a processing flow in step 202 (padding area identification processing) in FIG. 4;
6 is a flowchart showing details of a processing flow of step 203 (padding value substitution processing) in FIG. 4;
FIG. 7 is a flowchart showing a processing flow of padding processing means in the second exemplary embodiment of the present invention.
8 is a flowchart showing details of a processing flow in step 238 in FIG. 7;
FIG. 9 is a diagram showing a configuration of an embodiment of a moving image decoding system of the present invention.
FIG. 10 is a diagram showing a configuration of shape information encoding means in an embodiment of the video decoding system of the present invention.
FIG. 11 is a diagram illustrating an example of block classification information.
FIG. 12 is a diagram showing a coordinate system in an encoding target area.
FIG. 13 is a diagram illustrating an example of a search section and a padding area in a horizontal pixel row.
FIG. 14 is a diagram illustrating a configuration of a conventional video encoding system including padding processing.
FIG. 15 is a diagram illustrating an example of an image object and a corresponding padding area.
[Explanation of symbols]
40 Encoding target area
41 Image objects
42 Horizontal padding target area
43 Vertical padding process target area
101, 102 Storage means
103 Input is image encoding means
104 Shape information encoding means
105 Padding processing means
106 Block division
107 Block judgment unit
108 In-block shape information encoding unit
109 Multiplexer
110 Address calculator
111 Padding value substitution part
112 Image decoding means
113 Shape information decoding means
114 Demultiplexer
115 In-block shape information decoding unit
116 Shape information integration unit
1000 input images
1001, 1008, 1009 Image signal
1002, 1103 Shape information
1003, 1101 Encoded shape data
1004, 1006 Boundary information
1005, 1007 address
1010, 1100 Encoded image data
1011, 1105 Shape information in block
1012, 1106 Block classification information
1013, 1104 Intra-block coding shape information
1102 Output image
400, 401, 402, 403, 404, 405, 406, 407, 408, 409 pixels

Claims (13)

A moving image provided with a padding process for substituting an appropriate pixel value into an external area of the image object in the rectangular area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape An encoding system comprising:
Input image encoding means for encoding an input image signal having an image object therein;
First storage means for storing an image signal having an image object to be subjected to padding processing in the input image encoding means, and transferring it to the input image encoding means, if necessary;
In the rectangular shape region, shape information representing the shape of the image object is divided into a plurality of blocks and encoded in block units to generate encoded shape data, and used in the process of encoding the shape information. Block classification information indicating whether the block is outside, inside, or on the boundary of the image object, and shape information in the block when the block is on the boundary of the image object Shape information encoding means for generating boundary information;
Second storage means for storing the boundary information;
An address representing a pixel position outside the image object is calculated with reference to boundary information stored in the second storage means, and the address is determined from an image signal stored in the first storage means and calculates the pixel values, wherein the padding processing means for processing to assign,
The padding processing means refers to the block classification information, and when the block is outside the image object, specifies all pixels in the block as pixels outside the image object, and the block is the image object. A moving picture encoding system characterized by specifying a pixel outside the image object with reference to shape information in the block of the block when it is on the boundary . The moving image encoding system according to claim 1, wherein an image signal stored in the first storage means and having an image object to be subjected to the padding process is an input image. The moving image encoding system according to claim 1, wherein the shape information encoding means generates encoded shape data by multiplexing the block classification information and the encoded in-block shape information . A moving image provided with a padding process for substituting an appropriate pixel value into an external area of the image object in the rectangular area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape A decryption system,
Image decoding means for generating an image signal by decoding encoded image data having an image object having an arbitrary shape with reference to a reference image;
First storage means for storing the decoded image signal as an image signal to be subjected to padding processing, and supplying the decoded image signal as a reference image in decoding processing of the next frame in the image decoding means;
The coded shape data, which is data obtained by coding the shape information of the image object, is decoded for each block as a coding unit to generate shape information, and the block is outside or inside the image object. Shape information decoding means for generating boundary information consisting of block classification information indicating whether the block is on the boundary, and shape information in the block indicating shape information when the block is on the boundary of the image object;
Second storage means for storing the boundary information;
An address representing a pixel position outside the image object is calculated with reference to boundary information stored in the second storage means, and an address corresponding to the address is calculated from an image signal stored in the first storage means Padding processing means for performing processing for calculating and substituting pixel values ,
The padding processing means refers to the block classification information, and when the block is outside the image object, specifies all pixels in the block as pixels outside the image object, and the block is the image object. A moving picture encoding system characterized by specifying a pixel outside the image object with reference to shape information in the block of the block when it is on the boundary . The shape information decoding means inputs encoded shape data obtained by multiplexing block classification information and encoded intra-block shape information, decodes intra-block shape information after demultiplexing, and generates block classification information and a block. 5. The moving picture decoding system according to claim 4 , wherein the shape information of the image object in the image signal is generated by integrating the inner shape information . The input video encoding means is
Decomposing the image signal into coding units, detecting a motion amount between the image signal and the reference image for each coding unit, creating a predicted image by performing motion compensation using the motion amount, Means for encoding the difference between the predicted image and the image signal;
2. The moving image encoding system according to claim 1, wherein the padding processing means limits an object external pixel to which a padding value is substituted within the motion amount detection range. The image decoding means is
Decode the motion amount and difference image for each coding unit, create a predicted image by performing motion compensation on the reference image using the motion amount, and reconstruct the image by adding it to the difference image Including means to
5. The moving picture decoding system according to claim 4, wherein the padding processing unit limits an object external pixel to which a padding value is substituted within a detection range of the moving vector . A moving image code that performs padding processing for substituting an appropriate pixel value in an external area of the image object in the rectangular area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape A method of
(A) encoding an input image signal having an image object therein, and storing the image signal having an image object to be subjected to padding processing in a first storage unit;
(B) In the rectangular shape region, shape information representing the shape of the image object is divided into a plurality of blocks and encoded in units of blocks to generate encoded shape data, and in the process of encoding the shape information Used to represent block classification information indicating whether the block is outside, inside, or on the boundary of the image object, and shape information when the block is on the boundary of the image object Generating boundary information composed of in-block shape information and storing it in the second storage means;
(C) calculating an address representing a pixel position outside the image object with reference to boundary information stored in the second storage means, and from the image signal stored in the first storage means, Calculating a pixel value corresponding to an address and performing a padding process for performing a substitution process , and
The step of performing the padding process refers to the block classification information, and when the block is outside the image object, specifies all pixels in the block as pixels outside the image object, and the block is the image A moving picture coding method characterized by specifying a pixel outside the image object with reference to shape information in the block of the block when the object is on the boundary of the object . 9. The moving picture encoding method according to claim 8 , wherein said step (b) generates encoded shape data by multiplexing block classification information and encoded intra-block shape information . Moving picture decoding that performs padding processing for substituting appropriate pixel values in an external area of the image object in the rectangular area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape A method,
(A) referring to a reference image, decoding encoded image data having an image object having an arbitrary shape therein, and generating an image signal;
(B) storing the decoded image signal in the first storage means as an image signal to be padded;
(C) reading the image signal stored in the first storage means as a reference image in the decoding process of the next frame;
(D) Decoding encoded shape data, which is data obtained by encoding the shape information of the image object, for each block serving as an encoding unit to generate shape information, and whether the block is outside the image object, Generating boundary information including block classification information indicating whether the block is inside or on the boundary, and shape information in the block indicating shape information when the block is on the boundary of the image object; Storing in the storage means;
(E) An address representing a pixel position outside the image object is calculated with reference to boundary information stored in the second storage unit, and the address is calculated from the image signal stored in the first storage unit. Performing a padding process to calculate and substitute a pixel value corresponding to
The step of performing the padding process refers to the block classification information, and when the block is outside the image object, specifies all pixels in the block as pixels outside the image object, and the block is the image A moving picture coding method characterized by specifying a pixel outside the image object with reference to shape information in the block of the block when the object is on the boundary of the object . Step (d)
Encoded shape data in which block classification information and encoded in-block shape information are multiplexed are input, and after demultiplexing, the in-block shape information is decoded, and the block classification information and in-block shape information are integrated. The moving picture decoding method according to claim 10 , wherein shape information of an image object in the image signal is generated . A moving image code that performs padding processing for substituting an appropriate pixel value in an external area of the image object in the rectangular area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape Device.
(A) encoding an input image signal having an image object therein, and storing the image signal having an image object to be subjected to padding processing in a first storage unit;
(B) In the rectangular shape region, shape information representing the shape of the image object is divided into a plurality of blocks and encoded in units of blocks to generate encoded shape data, and in the process of encoding the shape information Used to represent block classification information indicating whether the block is outside, inside, or on the boundary of the image object, and shape information when the block is on the boundary of the image object Generating boundary information composed of in-block shape information and storing it in the second storage means;
(C) calculating an address representing a pixel position outside the image object with reference to boundary information stored in the second storage means, and from the image signal stored in the first storage means, Calculating a pixel value corresponding to an address and performing a padding process for performing a substitution process , and
The step of performing the padding process refers to the block classification information, and when the block is outside the image object, specifies all pixels in the block as pixels outside the image object, and the block is the image If it is on the boundary of the object, refer to the shape information in the block of the block to identify the pixel outside the image object,
A recording medium on which is recorded a program for causing each of the processes (a) to (c) described above to be executed on a computer constituting the moving image encoding apparatus. Moving picture decoding that performs padding processing for substituting appropriate pixel values in an external area of the image object in the rectangular area for an image object that is a part of an image signal having a rectangular shape and has an arbitrary shape A device,
(A) referring to a reference image, decoding encoded image data having an image object having an arbitrary shape therein, and generating an image signal;
(B) storing the decoded image signal in the first storage means as an image signal to be padded;
(C) reading the image signal stored in the first storage means as a reference image in the decoding process of the next frame;
(D) Decoding encoded shape data, which is data obtained by encoding the shape information of the image object, for each block serving as an encoding unit to generate shape information, and whether the block is outside the image object, Generating boundary information including block classification information indicating whether the block is inside or on the boundary, and shape information in the block indicating shape information when the block is on the boundary of the image object; Storing in the storage means;
(E) An address representing a pixel position outside the image object is calculated with reference to boundary information stored in the second storage unit, and the address is calculated from the image signal stored in the first storage unit. Performing a padding process to calculate and substitute a pixel value corresponding to
The step of performing the padding process refers to the block classification information, and when the block is outside the image object, specifies all pixels in the block as pixels outside the image object, and the block is the image If it is on the boundary of the object, refer to the shape information in the block of the block to identify the pixel outside the image object,
A recording medium on which is recorded a program for causing each of the processes (a) to (e) described above to be executed on a computer constituting the moving image encoding apparatus.

Images