JP2002204453A - Image object content generator and padding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image object content generator that conducts padding processing with high quality. SOLUTION: The image object content generator 10 is provided with a key generation section 11 that generates a key that is a reference for distinguishing an object pixel from a background pixel for each pixel from a received video signal, a frame size changing section 12 that changes a frame sizes of the video signal, a low frequency component passing section 13 that eliminates a high frequency component of the video signal processed by the frame size changing section 12, a pixel compensation section 14 that uses a Low Pass Extrapolation padding algorithm to apply padding processing to the video signal processed by the low frequency component passing section 13 and a pixel selection section 15 that selects the video signal processed by the low frequency component passing section 13 or the video signal processed by the pixel compensation section 14.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image object material generating apparatus, and more particularly to a padding process for filling the outside of an image object.

[0002]

2. Description of the Related Art In addition to being suitable for conventional storage media and digital broadcasting, an image compression method suitable for mobile terminals and the Internet, etc., which has been remarkably developed in recent years,
PEG (Moving Picture Coding Experts Group) -4
Is attracting attention. MPEG-4 is an object-based image encoding system that has one important feature of decomposing a video into a background and constituent elements (objects) such as people and buildings, and compressing and expanding each object as a unit. is there. As this application, there is a need to enlarge and reduce the screen size. For example, there is a case where an image captured by a home video camera is displayed on a small-sized liquid crystal display panel such as a mobile phone. There is also a need to require image material data such as a person image in an arbitrary size. For example, on a background taken in advance,
This is a case in which a video image is synthesized by separately capturing a person image or the like shot with a blue background or the like.

FIG. 18A shows a block of N × N pixels (N is a positive integer) including a boundary between an object pixel and a background pixel when an image including an object and a background is scanned in the direction of an arrow. It is a figure showing one. FIG.
6A is a graph illustrating a change in a pixel value (for example, a luminance value) of a video signal when an image in the block illustrated in FIG. Here, the horizontal axis indicates the position in the x direction, the vertical axis indicates the pixel value at that position, and the pixel values of the object pixel and the background pixel are digital values representing a gray scale in the range of 0 ≦ Y ≦ 255. It is shown as a value. CCD (Charge
The boundary portion of the video actually captured by using a coupled device or the like becomes an image as shown on the right side of FIG. 18A when enlarged, and the background video signal is mixed with the object portion or the background is mixed. Is mixed with the video signal of the object. As a result, as shown in FIG. 18A, when a still image composed of an object and a background is scanned in the direction of the arrow, the video signal has a substantially constant value inside the object (0 ≦ x <x0). However, near the boundary x0 between the object pixel and the background pixel, a high frequency change as shown in FIG. 18B is shown, and the change gradually approaches a constant value as the distance from the boundary part increases.

When a video block cut out from a frame includes object pixels and background pixels as in the block shown on the right side of FIG. The video signal is shown in FIG.
Since it contains high-frequency components as shown in (b), it is not suitable for shape adaptive discrete cosine transform (SA-DCT).
For this reason, in MPEG-4, it is necessary to perform a padding process for removing a high frequency component of a video signal before performing a shape adaptive discrete cosine transform. The padding process is a process of removing a high-frequency portion by supplying a substitute pixel value to a background pixel adjacent to the object pixel (filling the background pixel with the substitute pixel value).

An example of a conventional padding processing apparatus is disclosed in, for example, Japanese Patent Application Laid-Open No. Hei 10-210470, and the processing will be described with reference to FIG.

FIG. 19 is a block diagram showing a configuration of a conventional padding processing apparatus disclosed in Japanese Patent Application Laid-Open No. 10-210470. The padding processing device includes a boundary block detection unit 100, a block scanning unit 210, a block compensation unit 220, and an SA-DCT unit 300. The boundary block detection unit 100 calculates N including the object pixel and the background pixel.
A plurality of video blocks of the same size of × N pixels (N is a positive integer) are scanned, and a background pixel (padding) located between object pixels on the same row or column in the video block. Pixel). The block scanning unit 210 calculates a substitute pixel value using the pixel value of the object pixel. The block compensator 220 supplies a substitute pixel value to a padding pixel to generate a padded video block. As a result, in the compensated object region, padding pixels, which are background pixels sandwiched between the object pixels, are filled with the substitute pixel values, and as a result, high-frequency components at the boundary portions are removed. By performing the padding process in the block compensating unit 220 in this manner, the SA-DCT unit 300 appropriately converts the input video signals of the object region and the background region including the object pixel and the background pixel into a shape adaptive discrete cosine signal. Can be converted.

However, in this conventional padding processing apparatus, since the pixel value of the padding pixel is simply replaced with the substitute pixel value, there is a great possibility that the quality of the generated image is deteriorated. For this reason, for example, when it is attempted to change the size of the material data cut out from the input image to a predetermined size, it is difficult to generate the high-quality pixel data. Also, if padding processing is to be realized without degrading the quality of the clipped object image, a large memory capacity is required, and the circuit scale increases, power consumption increases, and the processing speed decreases. There is also the problem of getting lost.

[0008]

SUMMARY OF THE INVENTION In view of the above problems, the present invention provides a padding apparatus capable of performing a padding process without deteriorating the quality of an object image, and a padding device using such a padding process. A first object is to provide an image object material generation device capable of generating a high quality image object material.

A second object of the present invention is to provide a padding device and an image object material generating device which can be realized with a small circuit scale and can perform high-speed processing.

In order to achieve the first object, an image object material generating apparatus according to the present invention comprises a key for generating a key serving as a reference for distinguishing an object pixel from a background pixel for each pixel from an input video signal. Generating means, frame size changing means for changing the frame size of the video signal,
A low-frequency component passing unit that removes a high-frequency component of the video signal processed by the frame size changing unit; and a video signal processed by the low-frequency component passing unit, based on a key generated by the key generation unit. , Low Pass
Pixel compensation means for performing padding processing by an Extrapolation padding algorithm, and pixel selection means for selecting any of a video signal processed by the low-frequency component passing means and a video signal processed by the pixel compensation means It is characterized by.

As described above, in the image object material generating apparatus, the key generating means generates a key as a reference for distinguishing an object pixel and a background pixel for each pixel from an input video signal. The frame size changing means changes a frame size of the video signal. The low frequency component passing means removes high frequency components of the video signal processed by the frame size changing means. The pixel compensating means includes a video signal processed by the low-frequency component passing means;
Performs padding using the Low Pass Extrapolation padding algorithm. The pixel selection unit selects one of a video signal processed by the low-frequency component passing unit and a video signal processed by the pixel compensation unit.

In order to achieve the second object,
In the image object material generating apparatus according to the present invention, in the image object material generating apparatus, the pixel compensating means may further include a buffer memory for holding a video signal value and a key value of each pixel sequentially input, and An alternative pixel value generation unit that generates an alternative pixel value from the video signal of the object pixel based on the key value of each pixel, and information indicating the position of a padding pixel based on the sequentially input key value of each pixel. A padding pixel position holding unit for holding the padding pixel, specifying a padding pixel with reference to the padding pixel position holding unit, and using the substitute pixel value and the value of the video signal in the buffer memory for the specified padding pixel. And a filtering unit for performing a filtering process.

As a result, the information indicating the position of the padding pixel is held in the padding pixel position holding unit, and a filtering process using the substitute pixel value is performed according to the information.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to FIGS. FIG. 1 is a block diagram showing a configuration of an image object material generation device 10 according to the present invention. The image object material generation device 10 changes the frame size of the image represented by the input video signal,
It has a function of smoothing and outputting a boundary between an object image and a background image in a frame, and outputs a key generation unit 11, a frame size conversion unit 12, a low frequency passing unit 13, a pixel compensation unit 14, a pixel selection unit 15, and an image. A data holding unit 16 is provided.

The key generation unit 11 receives a video signal representing an input image to the image object material generation apparatus 10, receives a parameter input related to a key setting from a user, and generates a key signal based on the parameter. I do. Here, the user inputs, for example, in the case of using a chroma key, to specify a color. The key is
It is used to distinguish between an object pixel and a background pixel in the input image, and here indicates how far the attribute of each pixel is from a specific color designated by the user. Specifically, the attribute of each pixel is composed of a luminance value Y and two color difference values Cb and Cr, and the key is a hue obtained from the color difference of each pixel and a color of the color designated by the user. It is expressed as the difference from the hue.

The frame size converter 12 enlarges or reduces an image represented by a video signal input to the image object material generating device 10 to convert the frame size. Input of the frame size change input from the user and input of the frame size of the input image by giving new pixel values and key values by linear interpolation for pixels newly generated for the original frame At the given magnification.

The low frequency passing section 13 removes high frequency components of the input video signal. When the frame size is expanded in the previous frame size conversion unit 12, the input video signal is output as it is, and when the frame size is reduced, high-frequency components generated by pixel thinning or the like are removed.

The pixel compensating section 14 is a processing section implemented by software such as a program, hardware such as a logic circuit, or a combination thereof. The pixel compensating section 14 includes a background pixel in each video block including an object pixel and a background pixel. Then, an alternative pixel value is given according to the Low Pass Extrapolation padding algorithm, and the video block is padded.

For example, the pixel selection unit 15 outputs the output of the low-frequency passing unit 13 and the pixel compensation unit 1 for each frame.
4 is switched. That is, based on an instruction from the user, when the image of the background area is requested, the output of the low-frequency passing unit 13 is selected, and when the image of the object area is requested, the output of the pixel compensation unit 14 is selected. Select and output.

The image data holding unit 16 includes the pixel selection unit 1
5 is stored in the buffer memory, and the image data is output according to an external output request. The use of a FIFO as the buffer memory facilitates communication with an external interface.

FIG. 2A is a diagram showing the contents of a video signal processed by the image object material generating apparatus 10 shown in FIG. FIG. 2B is a diagram showing an example of a configuration of a video signal and a key signal used in the image object material generation device 10 shown in FIG. FIG. 2 (c)
Is a diagram showing keys when hue is represented by a circle and saturation is represented by a radius.

As shown in FIG. 2A, each pixel of one still image (frame) is scanned from left to right on the uppermost horizontal scanning line of the frame, and then on the next lower horizontal scanning line. Scanning is performed in the same order from left to right. The video signal for one frame representing the input image of the image object material generation device 10 is a video signal in which video signals representing the attributes of each pixel on a scanning line are serially arranged in the above-described scanning order. A pixel is a minimum unit of a rectangle when a frame is divided into a horizontal direction and a vertical direction, and each pixel is uniquely represented by, for example, a luminance signal Y, a blue color difference signal Cb, and a red color difference signal Cr. It has one attribute.

More specifically, as shown in FIG. 2A, the horizontal scanning line direction is defined as the X axis, and the vertical scanning line direction is defined as the Y axis.
Assuming that the coordinates of the upper left corner of one frame are (0, 0), each pixel is represented by a coordinate value (x, y). Further, one frame is divided into M (M is a positive integer of 2 or more) pixels in the horizontal scanning line direction, and is divided into N (N is a positive integer of 2 or more) pixels in the vertical scanning line direction. In this case, each pixel is scanned on the uppermost horizontal scanning line in the order of coordinates (0, 0), coordinates (1, 0), coordinates (2, 0),..., Coordinates (M, 0). ,
Next, coordinates (0, 1), coordinates (1, 1), coordinates (2,
The scanning is performed in the order of 1),..., Coordinates (M, 1). Pixels on the following horizontal scanning lines are sequentially scanned in the same manner, and finally, the pixel on the bottom horizontal scanning line has coordinates (0, N), coordinates (1, N), coordinates (2, N),.・, Coordinates (M, N)
, And the scanning is terminated.

Thus, the video signal representing one frame is
As shown in the upper part of FIG. 2B, a video signal representing each pixel (Y, Cb, Cr) on each coordinate is serially arranged in the above scanning order. Hereinafter, the value of the video signal representing the attribute of each pixel is referred to as a pixel value.

Further, as shown in the lower part of FIG.
In the image object material generating apparatus 10, a key indicating how far the hue of each pixel is away from the hue designated by the user is calculated for each pixel based on the above pixel value, and The information is stored in association with each other and used in each process. For example, in the case of a chroma key with a blue background, the key is a difference in hue from blue and is represented by an integer value between 0 and 255.

As shown in FIG. 2C, when the hue is represented by a circle and the saturation is represented by a radius, the key changes the blue color specified by the parameter regardless of the deviation in the radius (saturation) direction. Key value = 255 × θ / π with respect to an angle θ (0 ≦ θ ≦ π) formed by a radius passing through the point indicating the pixel and a radius passing through the point indicating the hue of the pixel.
(Rounded down).

Although the chroma key is used here, other keys or other methods may be used, and the background need not be limited to blue. Further, the pixel value of each pixel is represented by (Y, Cb, Cr), but may be represented by another method. Further, the key value may be calculated by another method.

FIG. 3 is a block diagram showing an example of the internal configuration of the key generator 11 shown in FIG. Key generator 1
1 includes a key calculation unit 31 and a parameter setting unit 32.

The key calculator 31 generates a key signal representing an object pixel and a background pixel contained in the input video signal by using a specific condition (for example, color) from the input video signal. The condition for generating the key signal is determined by the parameter setting unit 32
Is determined by the value set in the above, but this value can be changed according to the image. Video signal and key calculation unit 3
1 is input to the frame size conversion unit 12.

The parameter setting section 32 receives an input of a condition by a user from an operation section (not shown) such as a keyboard, and sets the input condition as a parameter of a key. Here, the condition is the hue value of the background pixel.

FIG. 4 is a block diagram showing an example of the internal configuration of the frame size converter 12 shown in FIG. The frame size conversion unit 12 includes a buffer memory 41, a buffer memory 42, a multiplier setting unit 43, a multiplier 44, a multiplier 45, and an adder 46.

Buffer memory 41 and buffer memory 42
Means that one line of pixel data (pixel value and key value) in the same row is held. The multiplier setting unit 43 sets a multiplier for the multiplier 44 and the multiplier 45. The multiplier 44 multiplies the output of the buffer memory 41 by the multiplier set by the multiplier setting unit 43. The multiplier 45 multiplies the output of the buffer memory 42 by the multiplier set by the multiplier setting unit 43. The adder 46 adds the outputs of the multiplier 44 and the multiplier 45. When scaling up or down the frame size, the frame size conversion unit 12 configured as described above needs to generate a pixel value and a key value for a new pixel, and thus uses the data of two pixels in the same row. New pixel data is generated by performing the interpolation process and the thinning process. Hereinafter, the interpolation processing and the thinning processing will be described.

The multiplier setting unit 43 sets a multiplier in the multiplier 44 and the multiplier 45. The multiplier is a value determined according to the frame size enlargement / reduction ratio. For example, in the case of Q / P (where P, (Q is a positive integer) performs processing on successive P pixels, and outputs a video signal of the Q pixel. If Q is greater than “2”, j / (P is a multiplier for the i-th data of the pixel data of the output Q pixel.
-1) <i / (Q-1) <(j + 1) / (P-1) is found (where i and j are positive integers), and the multiplier 4
The multiplier of 4 is 1− (P−1) × ((i / (Q−1)) − (j / (P−1))), and the multiplier of the multiplier 45 is (P−
1) × ((i / (Q−1)) − (j / (P−1))) When Q is “1” or “2”, it corresponds to the thinning-out processing, and when Q is “1”, the multiplier is used only for the first time.
4 is set to “1”; otherwise, all are set to “0”. When Q is “2”, “1” is set to the multiplier 44 at the first time, “1” is set to the multiplier 45 at the last time, and “0” is set at other times.

The multipliers 44 and 45 multiply the multiplier set in this way by the multiplier setting unit 43 with the data held in the buffer memories 41 and 42 and output the result to the adder 46. The adder 46 adds the multiplication results of the multiplier 44 and the multiplier 45 and outputs the result. As described above, the frame size conversion unit 12 converts the data of a new pixel generated by the thinning-out or interpolation processing into
Linear interpolation is performed using data of adjacent pixels. The pixel data (pixel value and key value) whose frame size has been converted in this way becomes input data to the low-frequency passing unit 13.

FIG. 5 shows the low-frequency passage section 13 shown in FIG.
FIG. 2 is a block diagram showing an example of the internal configuration of the device. The low-frequency passing unit 13 includes buffer memories 51 to 53, dividers 54 to 5
6, an adder 57 and a multiplexer 58.

For the input data from the frame size converter 12, the filtering process is performed on three consecutive pixels, for example, the ratio of the weighting factors is 1: 2: 1.
The filtering process described above will be described. The buffer memories 51 to 53 hold data of adjacent pixels in the same row. The dividers 54 to 56 use the data held in the buffer memories 51 to 53 as dividends and divide the divisors as “4”, “2”, and “4”, respectively. The adder 57 adds the division results of the dividers 54 to 56. The ratio of the weighting coefficients for the filtering process can be set arbitrarily, and the divisor may be determined in the same manner. Further, the number of continuous pixels on which the filtering process is performed may be arbitrarily set. However, in that case, it is necessary to set a ratio according to the number of pixels.

The multiplexer 58 is connected to the low-frequency passing section 13.
Either the input data to the input or the signal subjected to the filtering process is selected. In particular, when the frame size is reduced, it is necessary to remove high-frequency components, so a signal subjected to a filtering process is selected. The data selected by the multiplexer 58 is supplied to the pixel compensation unit 14 and the pixel selection unit 15.

FIG. 6 is a block diagram showing an example of the internal configuration of the pixel compensator 14 shown in FIG. Pixel compensation unit 1
4 is a background pixel (padding pixel) adjacent to the object pixel
Is subjected to padding processing for compensating for the substitute pixel value. Here, low-pass extrapo
Adopt lation padding algorithm.

The Low Pass Extrapolation padding algorithm is performed in the following three stages. In the first step, for a pixel having a key value equal to or larger than a preset threshold value, that is, an object pixel, the value of the pixel (for example, a luminance value,
An average value of the color differences Cr and Cb) is obtained for each N × N pixel block (N is a positive integer). In the second step, the pixels having the key value less than the threshold value, that is, the pixel values of the background pixels are replaced with the average value obtained in the first step. The third step is a two-dimensional filtering process of averaging the pixel values of four pixels in the upper, lower, left and right directions for a pixel having a key value less than the threshold value, that is, a background pixel, and replacing the pixel value of the pixel with the average value. Is applied.

The pixel compensator 14 includes a buffer memory 61,
An average calculator 62, a padding processor 63, a line buffer 64, and a filter processor 65 are provided. The buffer memory 61 is a memory having a storage area for holding pixel values and key values for a video block of N × N pixels (N is a positive integer). As a first step of the Low Pass Extrapolation padding algorithm, the average value calculation unit 6
Reference numeral 2 refers to pixel data that is sequentially written one pixel at a time in the buffer memory 61, compares a key among them with a threshold value separately input by the user, and has a key equal to or greater than the threshold value. The pixel values of the pixels are summed, and the number of pixels is counted. The average value of the luminance is obtained when the data of the pixels of one video block, that is, N lines is held in the buffer memory 61, and the process proceeds to the next stage.

As a second stage, the padding processing section 63
Retrieves data held in the buffer memory 61, detects a pixel having a key less than the above-described threshold,
After the pixel value of the pixel is replaced with the average value calculated by the average value calculation unit 62, the pixel value is output to the line buffer 64. The line buffer 64 is a memory having a storage area for holding pixel values and key values for three lines.

As a third stage, the filter processing unit 65
The pixel value and the key value of each pixel stored in the line buffer 64 are sequentially extracted, and if the key value of the pixel is less than the threshold value, the pixel values of four pixels adjacent to the pixel above, below, left, and right are line The data is read from the buffer 64 and the average of the pixel values of four pixels adjacent to the pixel is calculated. Further, the pixel value of the pixel is replaced with the average value, and the video signal of the pixel is output to the pixel selection unit 15.

In the above processing, the pixel compensating section 14
This can be realized by three-stage pipeline processing. Further, in the above configuration, in order to realize the first-stage processing, the (luminance) of the pixel having the key value equal to or larger than the threshold value is set.
A circuit for calculating the average of the values is required. Since the second and subsequent steps cannot be executed until the average value is obtained, a buffer memory for storing the data of the pixels for N lines is necessary. Since the upper and lower pixels are referred to in the third stage processing, a buffer memory for at least two lines and one pixel is required. Therefore, the low pass extrapolation padding process requires a processing unit that executes three steps and a buffer memory that stores pixel data for (N + 2) lines.

FIG. 7 is a block diagram showing another example of the internal configuration of the pixel compensator 14 shown in FIG. The pixel compensating unit 14 illustrated in FIG. 7 includes a buffer memory 71, a pixel identifying unit 72, an object pixel average value calculating unit 73, a padding map holding unit 74, and a substitute pixel supplying unit 75. The Low Pass Extrapolation padding algorithm in the pixel compensator 14 is performed in the following two stages.

The buffer memory 71 has a storage area for storing pixel data (pixel value and key value) of a video block of N × N pixels (N is a positive integer). The input pixel data is sequentially stored in the buffer memory for each pixel.

The pixel identifying section 72 has a low pass extrapolati
In the first stage of the padding algorithm, the key value of each pixel input from the previous low-frequency passing unit 13 is compared with the threshold value to determine whether the pixel is a padding pixel or an object pixel. Identify. If the value of the key is equal to or greater than the threshold value, the pixel identification unit 72 outputs “0” indicating that the pixel is an object pixel to the padding map holding unit 74, and outputs the data of the pixel to the object pixel average. Output to the value calculator 73. If the key value is less than the threshold value, "1" indicating that the pixel is a padding pixel
Is output to the padding map holding unit 74.

The object pixel average value calculation unit 73 calculates the low pass E
In the first stage of the xtrapolation padding algorithm, the average value of the pixel values of the object pixels identified by the pixel identifying unit 72 is calculated. Specifically, each time the pixel value of the object pixel identified by the pixel identifying unit 72 is input, the pixel value is sequentially added, and the number of pixels is counted.
When the data for the line is held, the average of the pixel values of the object pixels is obtained by dividing the sum of the pixel values of the object pixels by the number of counted pixels.

The padding map holding unit 74 stores a map indicating whether the pixel is a padding pixel or an object pixel by one bit of “1” or “0” for each pixel in the video block of N × N pixels. This is a memory having a storage area for writing, and a map is written for each pixel by the pixel identification unit 72.

As described above, N is stored in the buffer memory 71.
In parallel with the image data for the line being held, the average value of the pixel values is calculated by the object pixel average value calculation unit 73, the padding map is completed by the padding map holding unit 74, and the pixel compensation unit 14 Starts processing of the next stage of the Low Pass Extrapolation padding algorithm. In addition,
Here, the padding map holding unit 74 is “1” or “0”.
Although the padding map indicating whether the pixel is a padding pixel or an object pixel is stored in the one bit, the present invention is not limited to this example. For example, the padding map may be stored in the image block of the padding pixel identified by the pixel identifying unit 72. May be stored in the memory.

The alternative pixel supply unit 75 is provided with a Low Pass Extrapo
In the second stage of the lation padding algorithm, the map stored in the padding map storage unit 74 or the position of the padding pixel on the video block is referred to, and two-dimensional filtering is performed on the padding pixel. If the adjacent pixels are padding pixels, it is not necessary to refer to the pixel values held in the buffer memory 71 as the pixel values of the four pixels in the upper, lower, left, and right directions required in the two-dimensional filtering processing. The average value calculated by the calculation unit 73 is used. Whether or not the four adjacent pixels on the upper, lower, left, and right sides are padding pixels can be determined by referring to the map held in the padding map holding unit 74.

In the above description, the padding map holding section 74 holds the padding map of N × N pixels. However, the present invention is not limited to this example, and the padding map holding section 74 stores (N + 2) × A map storage area for (N + 2) pixels is provided, and the above-described padding map of N × N pixels is stored in the inner N × N pixels, and the outer peripheral pixels are initialized as padding pixels. You may leave. As a result, the exception processing of the outer peripheral portion adjacent to another video block is eliminated, and the unnaturalness of the image after the two-dimensional filtering at the boundary portion between the video blocks can be eliminated. Padding map holding unit 74
Is stored in the padding map holding unit 74 in advance in the case where the position of the padding pixel on the video block is stored instead of the padding map as described above.
2) A map storage area for (N + 2) pixels is provided, and the peripheral pixels of the video block are initially set as padding pixels, so that exceptional processing of the peripheral portion is eliminated. It is possible to eliminate the unnaturalness of the image at the boundary between them.

As described above, the pixel compensator 1 shown in FIG.
According to FIG. 4, compared to the pixel compensator 14 shown in FIG.
Since the number of image memories and storage areas for holding pixel values and key values having a large data amount can be reduced, there is an effect that the circuit scale of the pixel compensator 14 can be reduced. When the image object material generating apparatus 10 having the same circuit scale as that of the related art is configured using such a pixel compensator 14, the storage capacity of the image memory is reduced by the reduced circuit scale by the pixel compensator 14 as shown in FIG. If the number of bits of the pixel value is increased by increasing the number of pixels, the resolution of the video signal can be improved, and there is an effect that a higher quality image object material can be provided.

Further, according to the pixel compensating section 14 shown in FIG. 7, by creating a padding map in the padding map holding section 74, the padding processing section 63 shown in FIG. And the padding map is completed when the pixel value and the key value for one video block are written into the buffer memory 71.
Can initiate two-dimensional filtering. On the other hand, in the pixel compensating unit 14 shown in FIG. 6, when the pixel value and the key value for one video block are written in the buffer memory 61, the average value of the pixel values of the object pixels is obtained. Only then, the filter processing section 65 cannot start the two-dimensional filtering processing until the padding processing section 63 completes the padding processing for at least two lines of pixels. Therefore, the pixel compensating unit 14 shown in FIG.
4 has an effect that the padding process and the two-dimensional filtering process can be completed faster.

The operation of the pixel compensator 14 configured as described above will be described with reference to the flowcharts shown in FIGS. In the following, a description will be given by taking a luminance value as an example of the data of each pixel. However, actually, the color difference Cr value and the Cb value are also determined in the same procedure as the luminance value.
Padding processing and two-dimensional filtering are performed. For the padding and two-dimensional filtering of each chrominance Cr value and Cb value, three counters for obtaining an average value may be prepared and the processing may be performed in parallel with the processing for the luminance value, or the luminance value → the chrominance Processing may be performed sequentially such as Cr value → color difference Cb value.

FIG. 8 shows the average value calculation unit 62 shown in FIG.
9 is a flowchart showing an example of the processing procedure of FIG. The average value calculation unit 62 calculates, for each pixel data sequentially input,
The following average value calculation processing (S1 to S3) is repeated for all the pixels in the video block of N × N pixels (loop A).

That is, the average value calculation unit 62 compares the key value of each pixel sequentially input with the threshold value related to the user's input (S1), and determines the pixel having a key equal to or larger than the threshold value, that is, The luminance value of the object pixel is added to the counter (S2),
The number of pixels is counted (S3). The key value of the pixel is compared with the threshold value related to the input by the user (S1). If the key value of the pixel is smaller than the threshold value (background pixel), the key value of the next input pixel is The threshold value is compared with a threshold value related to a user input (S1).

When the processing of summing the luminance values of the object pixels for all the pixels in the video block is completed, the average value calculating section 62 sets the value of the counter to which the luminance value has been added as the dividend and the number of counted pixels as the divisor. The average of the luminance values is calculated, and the calculation result is output to the padding processing unit 63 (S4).

After that, the average value calculation section 62 clears the counter to which the luminance value has been added, and then repeats the processing shown in the above flow chart sequentially for a new video block of N × N pixels. As described above, the average value calculation unit 62
When the average value of the luminance values of the object pixels in the video block is obtained, the following processing by the padding processing unit 63 is started.

FIG. 9 is a flowchart showing a processing procedure of the padding processing section 63 shown in FIG. The padding processing unit 63 performs, for every pixel in the N × N pixel video block held in the buffer memory 61,
The following padding processing (S10 to S10)
S13) is repeated (loop B).

That is, the padding processing section 63 reads out the pixel value and the key value of one pixel stored in the buffer memory 61 (S10). Next, the key value of the read pixel is compared with the threshold value related to the user's input (S11). If the key value of the pixel is less than the threshold value (background pixel), the luminance of the pixel is determined. The value is replaced with the average value obtained by the average value calculation unit 62 (S12), and the pixel value and the key value of the pixel are output to the line buffer 64 (S12).
13). The key value of the read pixel is compared with the threshold value according to the user's input (S11). If the key value of the pixel is equal to or greater than the threshold value (object pixel), the pixel value is read from the buffer memory 61. The pixel value and the key value are output to the line buffer 64 (S13).

Here, when the padding processing section 63 completes the padding processing for the pixels of three lines (that is, 3N pixels) of the video block, the subsequent filtering processing section 65 starts the next processing. Further, when the padding processing section 63 finishes the padding processing for all the pixels in the N × N pixel video block held in the buffer memory 61, the padding processing section 63 sets a new video block held in the buffer memory 61. The process shown in the above flowchart is repeated for each pixel.

FIG. 10 is a flowchart showing an example of the processing procedure of the filter processing section 65 shown in FIG. When the pixel values and the key values for three lines are written into the line buffer 64, the filter processing unit 65 focuses on the pixels in the video block in the order of the scanning described above (hereinafter, the focused pixel is referred to as the focused pixel). ), And the average value calculation processing (S20 to S26) of the following adjacent pixels is sequentially repeated (loop C).

Note that the pixels located at the end of each video block, such as the pixels on the first line, have the following 2
Exceptional processing needs to be performed in the dimensional filtering process. In this case, if there is no pixel adjacent to the pixel of interest, the adjacent pixel is regarded as a background pixel, and the luminance value is determined in step S4 in FIG. Calculate as the average value obtained in. Further, the line buffer 64 includes
After the filter processing unit 65 finishes processing for two lines,
It is assumed that old data is sequentially overwritten with new data for each pixel.

First, the filter processing unit 65 reads the pixel value and the key value of the pixel of interest held in the line buffer 64 (S20), and the key value of the read pixel of interest is less than the threshold value, that is, the pixel of interest. Is determined as a background pixel (S21). If the result of the determination is that the key value of the pixel of interest is greater than or equal to the threshold value (object pixel), the video signal of the pixel value read from the line buffer
5 (S26).

It is determined whether or not the read key value of the pixel of interest is less than the threshold value (S21). If the result of determination is that the key value of the pixel of interest is less than the threshold value (background pixel), Total processing for calculating the average value of the luminance values of the four pixels for the four pixels (upper, lower, left, and right) adjacent to the pixel (S22 to S22)
23) is repeated (loop D).

That is, the filter processing unit 65 reads the luminance value of the pixel values of the adjacent pixels from the line buffer 64 (S22), and adds the read luminance value to the counter (S23).

When the filter processing section 65 completes the processing for the four pixels in the upper, lower, left and right directions adjacent to the pixel of interest, the value of the counter to which the luminance values of the four pixels have been added is divided by “4” to obtain an average value. (S24) After replacing the luminance value of the target pixel with the average value (S25), the counter is cleared, and the video signal whose luminance value has been replaced with the average value is output to the pixel selection unit 15 (S26).

When the above processing is completed for all the pixels in the video block, the filter processing section 65 waits for the data of the pixels for three lines in the new video block to be written in the line buffer 64, and then starts the new processing. The process shown in the above flowchart is repeated for pixel data in a proper video block.

As described above, according to the pixel compensating unit 14, in a video block including an object pixel and a background pixel, the pixel value of the background pixel is replaced with the average pixel value of the object pixel. This has the effect that the values are made uniform and the high-frequency component of the video signal at the boundary between the object pixel and the background pixel is reduced to the frequency due to the pixel value change in video block units.

Further, according to the pixel compensating unit 14, the background pixel in the video block is replaced by the average value of the luminance values of the four pixels adjacent to the background pixel. Is reflected, the background pixel can be interpolated with a substitute pixel value with a small deviation of the pixel value from the adjacent object pixel, and a higher quality image compression with reduced image quality deterioration can be performed. There is an effect that can be.

FIG. 11 is a block diagram of the pixel identifying section 72 shown in FIG.
9 is a flowchart illustrating an example of a processing procedure between an object pixel average value calculation unit 73. Hereinafter, the processing procedure of the pixel discriminating unit 72 and the object pixel average value calculating unit 73 using the luminance value and the key value of each pixel shown in FIGS. 13A and 14A, respectively. Is specifically described.

FIG. 13A is a diagram showing an example of the luminance value of each pixel in the input data of the pixel compensating section 14. FIG.
FIG. 3B is a diagram showing the luminance of an input image displayed based on the input data shown in FIG. FIG.
In (a), the luminance value Y takes a value of 0 ≦ Y ≦ 255. In FIG. 13B, in the video block of 8 × 8 pixels, a shaded portion represents a pixel having a luminance value of Y = 255, a hatched portion represents a pixel having a luminance value of Y = 150, and the other portions represent a luminance value of Y = 255. 0 represents a pixel. FIG. 13 (b)
As shown visually in FIG. 5, in the block, a pixel having a luminance value of "255" is included adjacent to a pixel having a luminance value of "0", and the image has a very high frequency.

FIG. 14A is a diagram showing an example of a key value calculated from the color difference of each pixel in the input data of the pixel compensating section 14. FIG. 14B is a diagram showing a padding map of the padding map holding unit 74 obtained by setting the threshold value to “100” from the key values shown in FIG. The key value shown in FIG. 14A indicates how far the hue of each pixel is from the blue hue from “0” to “255”, for example, when the parameter indicating the background in the chroma key is set to blue. It is shown by the value up to. In this case, as the key value is closer to "255", the hue of the pixel is closer to the skin color which is the opposite color of blue, and as the key value is closer to "0", the hue of the pixel is closer to blue.

In the padding map shown in FIG. 14B, a map value “0” in each pixel indicates that the pixel is an object pixel, and a map value “1” indicates that the pixel is a background pixel. (Padding pixel). In the padding map shown in FIG. 14B, when the size of the video block is set to N × N pixels, the map value of one pixel further around the video block is set to “1”. It is created with a size of (N + 2) × (N + 2) pixels. Here, the size of the video block is 8
Since the size is × 8 pixels, the size of the padding map is 10 × 10 pixels. Although the padding map holding unit 74 holds a padding map of this size here, it is not necessary to actually create a padding map of this size. May be subjected to two-dimensional filtering assuming that they are padding pixels.

In the pixel compensating section 14 shown in FIG. 7, for each of N × N (here, N = 8) pixels in the video block, the luminance values shown in FIG. 2
The two color differences and the key values shown in FIG. 14A are input to the buffer memory 71 and the pixel identification unit 72 in parallel. The pixel identification unit 72 and the object pixel average value calculation unit 73
The following average value calculation processing and padding map generation processing (S31 to S35) are sequentially repeated in parallel in the above-described scanning order for all the pixels in the video block of N × N pixels (loop A). .

That is, the pixel value and the key value of each pixel input to the pixel compensating unit 14 are input to the pixel identifying unit 72 in parallel with the sequential writing to the buffer memory 71. First, the key value of the input pixel is compared with the threshold value related to the input by the user (S31),
As a result of the comparison, if the key value of the pixel is smaller than the threshold value (background pixel), the padding is performed by associating the map value “1” indicating that the pixel is a padding pixel with the coordinate position of the pixel. Write to the map holding unit 74 (S3
2).

For example, for the pixel having the coordinate value (0, 0) shown in FIG. 14A, the key value is “46” and the threshold value is “100”, so that the key value <threshold. (S31), which is a padding pixel. Accordingly, the pixel identifying unit 72 writes “1” into the padding map holding unit 74 as the map value of the pixel having the coordinate value (0, 0) (S3).
2). On the other hand, in parallel with this, the pixel value and the key value of the pixel having the coordinate value (0, 0) are written into the buffer memory 71.

Similarly, for example, the pixel having the coordinate value (1, 0) shown in FIG.
Since the threshold value is “100”, the key value <the threshold value (S31), indicating that the pixel is a padding pixel.
On the other hand, the pixel identifying unit 72 sets “1” as the map value of the pixel having the coordinate value (1, 0) as the padding map holding unit 7.
4 (S32). On the other hand, in parallel with the writing to the padding map holding unit 74, the pixel value and the key value of the pixel having the coordinate value (1, 0) are written to the buffer memory 71. Hereinafter, the same processing as described above is performed on the pixel whose key value is less than “100”.

The pixel identifying section 72 compares the key value of the input pixel with a threshold value related to the input by the user (S3).
1) As a result, if the key value of the pixel is equal to or larger than the threshold value (object pixel), a map value “0” indicating that the pixel is an object pixel is associated with the coordinate position of the pixel. The data is written into the padding map holding unit 74 (S33), and the input pixel value is output to the object pixel average value calculating unit 73. Next, the object pixel average value calculation unit 73
After adding the luminance value of the pixel input from Step 2 to the counter (S34), the number of pixels is counted (S3).
5).

For example, for the pixel having the coordinate values (1, 4) shown in FIG. 14A, the key value is “148” and the threshold value is “100”. It becomes a key value (S31), and it can be seen that this pixel is an object pixel. Therefore, the pixel identifying unit 72 calculates the coordinate value (1, 4)
"0" is written to the padding map holding unit 74 as the map value of the pixel of (S33). Here, since this pixel is an object pixel, the following processing is further performed. That is, the object pixel average value calculation unit 73 converts the luminance value “255” shown in FIG. 13A from the pixel values of the pixels having the coordinate values (1, 4) input through the pixel identification unit 72. The acquired and acquired luminance value “255” is added to a counter for summing the luminance values (S34), and the number of pixels is counted up by “1” (S35). On the other hand, in parallel with this, the pixel value of the pixel having the coordinate value (1, 4) and the key value are written in the buffer memory 71.

Hereinafter, the coordinate values (2, 4) to (5,
The same processing as described above is performed on the pixel having the key value of “100” or more in 4).

In this way, a padding map of each pixel is sequentially created in the scanning direction by one pixel for a video block of N × N (N = 8 in FIG. 14A) pixels,
The luminance values of the pixels whose key value is “100” or more are added, the number of pixels is counted, and the pixel value and the key value of each pixel are written into the buffer memory 71. When the processing is completed for all the pixels in the video block of N × N pixels, specifically, when the above processing is completed for the pixel having the coordinate value (7, 7) in FIG. The unit 73 calculates the average value of the brightness values of the object pixels by dividing the value of the counter for summing the brightness values by the number of counted pixels, and outputs the calculation result to the alternative pixel supply unit 75 (S36).

As a result, in the padding map holding unit 74, a map in which “0” or “1” indicating whether the pixel is a padding pixel (background pixel) or an object pixel is written for all pixels in the video block. Is completed, and the average value of the luminance values of the object pixels is obtained in the object pixel average value calculation unit 73. Here, based on the luminance value shown in FIG. 13A and the padding map shown in FIG. 14B, when the average value of the luminance values of the object pixels in this video block is calculated, The value is 228 (rounded down). The object pixel average value calculation unit 73 outputs the obtained average value of the luminance values to the substitute pixel supply unit 75, and uses the counter used in the luminance value addition processing (S34) and the pixel number counting processing (S35). clear. At the same time that the average of the luminance values of the object pixels is obtained, the writing of the pixel values and the key values to the buffer memory 71 is completed for all the pixels in the video block.

Next, the processing shown in the following flowchart by the substitute pixel supply unit 75 at the subsequent stage is started for the video block. The pixel identification unit 72 and the object pixel average value calculation unit 73 include the substitute pixel supply unit 75 After the processing for two lines is completed, the processing shown in the above flowchart is repeated for a new video block of N × N pixels.

FIG. 12 is a flowchart showing an example of the processing procedure of the alternative pixel supply unit 75 shown in FIG. Hereinafter, using the luminance value and the map value of each pixel shown in FIGS. 13A and 14B and referring to the processing result of FIG. The procedure will be specifically described. FIG. 15A shows that the alternative pixel supply unit 75
FIG. 15 is a diagram illustrating luminance values after two-dimensional filtering processing is performed on the video block illustrated in FIG. 14A using the padding map illustrated in FIG. FIG. 15 (b)
FIG. 14 is a diagram visually illustrating a luminance value after the two-dimensional filtering process performed by the alternative pixel supply unit 75. FIG. 15 (c)
FIG. 16 is a diagram showing a correspondence between a luminance value of each pixel in the video block shown in FIG. 15B and hatching.

The substitute pixel supply unit 75 holds the average value of the luminance values obtained by the object pixel average value calculation unit 73,
For all the pixels in the video block of N × N pixels written in the buffer memory 71, in the order of scanning described above,
The following two-dimensional filtering processing (S41 to S4
8) is repeated (loop B).

That is, the substitute pixel supply unit 75 first pays attention to each pixel in the order of the above-described scanning (S41), and refers to the map value of the padding map holding unit 74 to determine whether the pixel of interest is a padding pixel. Is determined (S42). As a result, if the target pixel is a padding pixel, the following process of calculating the average value of the luminance values of the four adjacent pixels (S43 to S46) is repeated for the four pixels on the upper, lower, left, and right adjacent to the target pixel (loop C). .

That is, the substitute pixel supply unit 75 determines whether or not the adjacent pixel is a padding pixel with reference to the map value of the padding map holding unit 74 (S43). The average value of the luminance values is added to the counter (S44). If the result of determination in step S43 is that the adjacent pixel is not a padding pixel, the luminance value of the adjacent pixel is read from the buffer memory 71 (S43).
45), The read luminance value is added to the counter (S46).

Specifically, first, the substitute pixel supply unit 75
Attention is paid to the pixel having the coordinate value (0, 0), and the padding map shown in FIG. Since the map value of the pixel having the coordinate value (0, 0) is "1", it is a padding pixel (S42). Therefore, the substitute pixel supply unit 75 calculates the average value of the luminance values for the adjacent pixels on the upper, lower, left, and right sides of the coordinate value (0, 0) as follows.

First, the padding map is referred to for the pixel having the coordinate value (0, -1) located above the pixel having the coordinate value (0, 0), which is the target pixel, and whether or not the pixel is a padding pixel is determined. Is determined (S43). In FIG. 14B, the pixel having the coordinate value (0, -1) is a pixel that is not included in the video block, and is a padding pixel having a map value of “1”. The luminance value of the padding pixel is replaced with the average luminance value of the object pixel calculated in step S36 in FIG. Therefore, the substitute pixel supply unit 75 adds the average value “228” to the counter for summing the luminance values (S
44).

Next, the coordinate value (0, 0) which is the pixel of interest
The pixel with the coordinate value (1, 0) located to the right of is referred to the padding map, and it is determined whether or not the pixel is a padding pixel (S43). In FIG. 14B, the pixel having the coordinate value (1, 0) is a padding pixel because the map value is “1”. Therefore, the alternative pixel supply unit 75
Adds the average value "228" to the counter for summing the luminance values (S44).

Next, the coordinate value (0, 0) which is the pixel of interest
It is determined whether the pixel is a padding pixel by referring to the padding map for the pixel having the coordinate value (0, 1) located below (S43). In FIG. 14B, the pixel having the coordinate value (0, 1) is a padding pixel because the map value is “1”. Therefore, the alternative pixel supply unit 75
Adds the average value "228" to the counter for summing the luminance values (S44).

Finally, the coordinate value (0, 0) which is the pixel of interest
The pixel with the coordinate value (-1, 0) located to the left of the pixel is referred to the padding map, and it is determined whether or not the pixel is a padding pixel (S43). In FIG. 14B, the pixel having the coordinate value (−1, 0) is a pixel that is not included in the video block, and is a padding pixel having a map value of “1”. Therefore, the substitute pixel supply unit 75 adds the average value “228” to the counter for summing the luminance values (S44).

When the processing is completed for the four pixels in the upper, lower, left and right directions adjacent to the pixel of interest, the substitute pixel supply unit 75 divides the value of the counter to which the luminance values of the four pixels are added by “4” and averages the luminance values. After obtaining the value and replacing the luminance value of the pixel of interest with the average value (S47), the counter is cleared and the luminance value replaced with the average value is output to the pixel selection unit 15 (S48).

Specifically, as described above, the coordinate values (0,
When the pixel of 0) is set as the target pixel, the four pixels adjacent to the top, bottom, left, and right are all padding pixels, so the value of the counter for summing the luminance values is “912”, and the luminance value of the four adjacent pixels of the target pixel is The average value is “228”. FIG.
As illustrated in (a), the substitute pixel supply unit 75 sets the luminance value “0” of the target pixel to the average value “228” of the luminance values of the adjacent four pixels.
(S47), and outputs the luminance value “228” to the pixel selection unit 15 (S48).

Hereinafter, the average value "228" of the luminance values of the four adjacent pixels is obtained in the same manner as described above up to the target pixel of the coordinate values (0, 3), and the average value "228" is substituted. The luminance value is output to the pixel selection unit 15.

Next, attention is paid to the pixel having the coordinate value (1, 3). The difference between the pixel having the coordinate values (1, 3) and the pixel described above can be understood by referring to the padding map.
Some of the adjacent pixels are object pixels. Referring to the padding map while focusing on the pixel having the coordinate value (1, 3), the pixel of interest is a padding pixel because the map value is “1” (S42). Therefore, the substitute pixel supply unit 75 calculates the average value of the luminance values for the adjacent pixels on the upper, lower, left, and right sides of the coordinate value (1, 3).

First, referring to the padding map for the pixel having the coordinate value (1, 2) located above the pixel of interest having the coordinate value (1, 3), it is determined whether or not the pixel is a padding pixel ( S43). In FIG. 14B, the pixel having the coordinate value (1, 2) is a padding pixel because the map value is “1”. Therefore, the substitute pixel supply unit 75 adds the average value “228” to the counter for summing the luminance values (S44).

Next, it is determined whether or not the pixel having the coordinate value (2, 3) located to the right of the target pixel is a padding pixel (S43). In FIG. 14B, the pixel having the coordinate value (2, 3) is a padding pixel with a map value of “1”, so the alternative pixel supply unit 75 sets the average value “228” in the counter that sums the luminance values. It is added (S44).

Further, it is determined whether or not the pixel having the coordinate value (1, 4) located below the pixel of interest is a padding pixel (S43). In FIG. 14B, the pixel having the coordinate value (1, 4) is an object pixel because the map value is “0”. Therefore, the alternative pixel supply unit 75
Reads the luminance value “255” of the pixel having the coordinate value (1, 4) shown in FIG. 13A from the buffer memory 71 (S4).
5) Add the read luminance value “255” to the counter that sums the luminance values (S46).

Finally, it is determined whether or not the pixel having the coordinate value (0, 3) located to the left of the target pixel is a padding pixel (S43). In FIG. 14B, the pixel having the coordinate value (0, 3) is a padding pixel because the map value is “1”. Therefore, the substitute pixel supply unit 75 supplies the average value “228” to the counter that sums the luminance values.
Is added (S44).

When the pixel having the coordinate value (1, 3) is set as the target pixel as described above, the value of the counter for summing the luminance values is “939”, and the average value of the luminance values of the four pixels adjacent to the target pixel is It becomes "234". As shown in FIG. 15A, the substitute pixel supply unit 75 replaces the luminance value “255” of the target pixel with the average luminance value “234” of the four adjacent pixels (S4).
7), and outputs the video signal to the pixel selection unit 15 (S4)
8).

Further attention is paid to the pixel having the coordinate value (2, 3). The difference between the pixel having the coordinate value (2, 3) and the pixel described above is that the luminance value of the adjacent pixel is replaced by the alternative pixel supply unit 75.

When attention is paid to the pixel having the coordinate values (2, 3),
Since the map value is "1", this pixel is a padding pixel (S43). In this case, as in the case of focusing on the pixel having the coordinate value (1, 3), the average value of the luminance values of the four pixels adjacent to the target pixel in the vertical and horizontal directions is obtained. The luminance value of the pixel having the value (1, 3) has been replaced with “234” immediately before. However, here, when the adjacent pixel is a padding pixel, the average value calculated by the object pixel average value calculation unit 73 should be used uniformly without using the luminance value replaced by the two-dimensional filtering as the luminance value. And In this case, since only the pixel having the coordinate value (1, 4) adjacent to the target pixel having the coordinate value (2, 3) is the object pixel, the luminance value “255” is read out from the buffer memory 71 and read. Upon calculation, the average value of the four adjacent pixels is “234”. FIG.
As shown in FIG. 5A, the substitute pixel supply unit 75 sets the luminance value “255” of the target pixel to the average value “2” of the luminance values of the four adjacent pixels.
34 ”(S47), and the luminance value is replaced with the pixel selection unit 1
5 (S48).

Hereinafter, the average value of the luminance values of the four adjacent pixels in the upper, lower, left and right directions is obtained for each pixel up to the pixel having the coordinate value (0, 4) in the same manner as described above, and the luminance value replaced by the average value is obtained. The data is output to the pixel selection unit 15 (S47, S48).

Attention is paid to each pixel in the video block in the order of scanning (S41), and it is determined whether or not the pixel of interest is a padding pixel by referring to the map value of the padding map holding unit 74 (S42). If the target pixel is not a padding pixel, the substitute pixel supply unit 75 reads the luminance value of the pixel from the buffer memory 71 (S49) and outputs the luminance value to the pixel selection unit 15 (S48).

Specifically, referring to the padding map shown in FIG. 14B, the pixel having the next coordinate value (1, 4) is an object pixel because the map value is "0". (S42). The substitute pixel supply unit 75 does not perform the two-dimensional filtering on the object pixel, reads out the luminance value “255” of the pixel having the coordinate value (1, 4) from the buffer memory 71 (S41), and shows it in FIG. Thus, the luminance value is output to the pixel selection unit 15 (S48). Hereinafter, as shown in FIG. 15A, up to the pixel having the coordinate value (5, 4), the luminance values “255” and “25” read from the buffer memory 71 by the substitute pixel supply unit 75 in the same manner as described above.
5 ”,“ 150 ”and“ 150 ”are sequentially in the pixel selection unit 1
5 is output. Also, in the order of scanning, the next coordinate value (6,
The method of two-dimensional filtering of each pixel of 4), coordinate values (7, 4) and coordinate values (0, 5) is as described above.

Next, attention is paid to the pixel having the coordinate value (1, 5). The pixel having the coordinate value (1, 5) is different from the above-described pixel in that two of the adjacent pixels are object pixels. The map value of the target pixel is “1” and is a padding pixel. When two-dimensional filtering is performed on the target pixel by calculating the average value of the four adjacent pixels on the upper, lower, left, and right sides, the two pixels adjacent to the lower and left sides of the target pixel are padding pixels. It is not necessary to refer to the buffer memory 71 because the average value of the luminance values of the object pixels is used. On the other hand, two pixels adjacent to the target pixel above and to the right, that is, the pixel having the coordinate values (1, 4) and the pixel having the coordinate values (2, 5) have map values of “0”, respectively.
It is necessary to read the luminance value of each pixel of the coordinate value (1, 4) and the coordinate value (2, 5) by referring to the buffer memory 71.
That is, it is necessary to refer to the buffer memory 71 twice. In this case, the luminance value of the two pixels adjacent to the bottom and left is “228”, and the luminance value of the two pixels adjacent to the top and right is
Since the luminance value of each pixel is “255” as shown in FIG. 13A, the value of the counter for summing the luminance values is “966”, and the average value of the luminance values of the four adjacent pixels is “241”. ". The alternative pixel supply unit 75 is configured as shown in FIG.
As shown in (a), the luminance value of the pixel having the coordinate value (1, 5) is replaced with the average value “241”, and the luminance value is output to the pixel selection unit 15. Hereinafter, two-dimensional filtering is performed on all the pixels of the video block by the above-described method, and the luminance value is replaced.

When the processing is completed for all the pixels in the video block of N × N pixels, the substitute pixel supply section 75 repeats the processing shown in the above flowchart for the video block of new N × N pixels.

As shown in FIG. 15B, the alternative pixel supply unit 7
It can be seen that, in the video of the video block on which padding and two-dimensional filtering have been performed according to No. 5, the luminance value is interpolated over the entire background pixel as compared with the video at the time of input shown in FIG. Moreover, by two-dimensional filtering, padding pixels adjacent to the object pixel
Since the pixel value of the object pixel is reflected, it can be said that the image quality is stored with higher accuracy.

As described above, when the filter processing unit 65 shown in FIG. 6 performs two-dimensional filtering on padding pixels, the line buffer 64 must always be referred to for data of four pixels surrounding each padding pixel. According to the alternative pixel supply unit 75 shown in FIG. 7, the number of references to the buffer memory 71 can be greatly increased by referring to the states (map values) of the four pixels surrounding the padding pixels held in the padding map holding unit 74. Can be reduced.
The data stored in the line buffer 64 and the buffer memory 71 is generally multi-valued, and the data stored in the padding map storage 74 is binary.
By referring to a small amount of data, two-dimensional filtering can be performed, and a bus band for accessing the image memory can be effectively used. Thereby, there is an effect that the power consumption of the image object material generation device 10 can be significantly reduced.

In FIG. 7, by generating a padding map, it is possible to obtain the same result as the alternative pixel value compensation processing for padding pixels. In addition, the generation of the padding map and the average value calculation processing are performed in parallel. Therefore, the result can be obtained faster than in the case where the pixel compensator 14 shown in FIG. 6 performs the substitute pixel value compensation process for padding pixels and the average value calculation process serially. it can. In addition, the padding map holding unit 74 does not require a large storage capacity as compared with the line buffer 64 that holds the pixel value and the key value for each pixel, so that the circuit scale of the image object material generation device 10 can be reduced. is there.

FIG. 16 is a diagram showing the number of times the buffer memory 71 is referred to in the two-dimensional filtering of the substitute pixel supply unit 75 for each pixel of the video block. As described above, when the alternative pixel supply unit 75 performs the two-dimensional filtering, it is necessary to read the luminance value of a pixel adjacent to the target pixel from the buffer memory 71 because the target pixel is a padding pixel. And only when it is adjacent to the object pixel. Further, since it is sufficient to refer to the padding map as to whether or not the target pixel is a padding pixel, it is not necessary to refer to the buffer memory 71. As shown in the figure, the number of times the buffer memory 71 is referred to for the luminance value of an adjacent pixel in the video block depends on each pixel from coordinate values (1, 3) to (5, 3) and the coordinate value (0, 4), (6, 4), (0, 7) and (6, 7)
Once for each pixel. Also, coordinate values (1, 5), (5, 5), (1, 6) and (5,
In each of the pixels in 6), two adjacent pixels are object pixels, and thus each pixel is referred to twice. In the figure, since the two-dimensional filtering is not performed on the portion of the object pixel indicated by oblique lines, the number of references is zero.

In comparison with this, the pixel compensator 14 shown in FIG. 6 does not perform two-dimensional filtering on the object pixel, so that the number of times of reference to the luminance value of the adjacent pixel is 0 as described above. In order to determine whether or not the pixel is an object pixel, it is necessary to read out the key value of the pixel of interest from the line buffer 64. For background pixels,
In order to determine whether or not each pixel of interest is an object pixel, it is necessary to read out the key value of the pixel of interest, and it is necessary to read out the luminance values of four adjacent pixels from the line buffer 64.

Therefore, according to the pixel compensating unit 14 shown in FIG. 7 as described above, the number of times of referring to the image memory can be greatly reduced as compared with the pixel compensating unit 14 shown in FIG. This has the effect.

On the other hand, the pixel compensator 1 shown in FIG.
In No. 4, while the number of times of reading pixel values from the buffer memory 71 for the four pixels adjacent to the pixel of interest has decreased, new processing of reading map values from the padding map holding unit 74 has increased. The luminance value and the color difference are each 8-bit data, while the data is 1-bit data at most. Therefore, when comparing the two pixel compensating units 14, the buffer band between the buffer memory 71 and the substitute pixel supplying unit 75 for accessing the image memory is smaller than that between the line buffer 64 and the filtering unit 65. It can be seen that significant savings have been made. Further, here, as the number of times that the substitute pixel supply unit 75 reads the pixel value from the buffer memory 71 is reduced, the average value is read from the object pixel average value calculation unit 73 by the same number of times. Are also 8 bits each. However, the readout of the average value from the object pixel average value calculation unit 73 does not require an address unlike the case of reading out a pixel value from an image memory, so that the alternative pixel supply unit 75 does not need to generate an address signal. Since the address value is transmitted to the image memory, the bus bandwidth can be reduced. As a result, the reduced bus bandwidth can be used for other processing. Otherwise, the power consumption of the image object material generation device 10 can be sufficiently reduced.

FIG. 17 is a diagram showing an example of application of the image object material generating apparatus 10 to a product. PC card 19
Reference numeral 0 denotes a card board on which the image object material generation device 10 according to the present embodiment shown in FIG. 1 is mounted as a logic circuit. The personal computer 191 includes a processing unit including a CPU, a storage unit including a hard disk, an external connection unit such as a card slot 192, an operation unit including a touch pad 193, and the like.
And a display unit made up of 94 or the like. The hard disk of the personal computer 191 stores an image editing program for editing an image such as a photograph captured by a digital camera or a scanner, or the PC card 1.
Driver software and the like for using the image processing engine 90 as an image editing engine are installed. The card slot 192 is an insertion slot for inserting the PC card 190 into the personal computer 191. The touch pad 193 is an operation unit for operating a pointer on an image displayed on the liquid crystal display panel 194. The liquid crystal display panel 194 is a display unit that displays an image or text being edited, an operation instruction to a user by various programs, an operation state, and the like.

The user can use the personal computer 1 by using the image editing program and the PC card 190.
On the 91, editing such as enlarging / reducing the frame size of an image fetched from the outside or enlarging / reducing only the object in the image and pasting it to another image can be performed at high speed.

As described above, according to the image object material generating apparatus of the present embodiment, an input video signal can be selected and output as an image object material and a background material. Outputs a video signal that has been padded. This padding process is performed in fewer steps than the conventional method, the number of times of referencing the memory is small, and the memory bandwidth can be used effectively. Discontinuity between padded video blocks can also be improved, and hardware real-time processing including frame size conversion and output video selection can be performed.

The image object material generating apparatus according to the present invention has been described based on the embodiments. However, the present invention is not limited to the embodiments.

For example, in the present embodiment, the background pixels in each video block are used as padding pixels, but the object pixels in each video block may be used as padding pixels.
In the above embodiment, the pixel value of the padding pixel is replaced with the average value of the pixel values of the object pixels. However, the pixel value of the padding pixel may be replaced with the average value of the pixel values of the background pixels. With either method, it is possible to smooth the change in the video signal on a block-by-block basis and to perform padding processing with high quality.

Further, the image object material generating apparatus according to the present invention may include means for cutting out an object from the input image data. That is, key generation, size change, high frequency removal, pixel compensation (padding), and the like may be performed on the clipped object.

In the above embodiment, the image data holding unit 16 always stores the image data from the pixel selection unit 15 in the buffer memory. Instead, the image data holding unit 16 stores the image data in accordance with an external instruction. The operation may be performed by selecting whether to hold the data or pass the data as it is.

Further, the Low Pass Ext in the present embodiment
In the rapolation padding algorithm, two-dimensional filtering processing for averaging the pixel values of four pixels in the upper, lower, left, and right directions is performed. However, the filtering processing is not limited to this, and other filtering processing such as a Gaussian filter may be used. Is also good.

[0125]

According to the present invention, there is provided an image object material generating apparatus, comprising: a key generating means for generating a key serving as a reference for distinguishing an object pixel and a background pixel for each pixel from an input video signal; Frame size changing means for changing the size; low-frequency component passing means for removing high-frequency components of the video signal processed by the frame size changing means; and a video signal processed by the low-frequency component passing means. Based on the key generated by the generating unit, a pixel compensating unit that performs padding processing by a Low Pass Extrapolation padding algorithm, a video signal processed by the low frequency component passing unit, and a video signal processed by the pixel compensating unit. Pixel selection means for selecting any one of the above.

That is, according to this image object material generating apparatus, the pixel compensating means performs padding processing using the Low Pass Extrapolation padding algorithm. Is filtered.
Finer padding processing in which the pixel values of adjacent pixels are reflected is performed. As a result, there is an effect that the image object material generation device can generate a high quality image object material.

In the image object material generating apparatus according to the present invention, in the image object material generating apparatus, the pixel compensating means may further comprise a buffer memory for holding a sequentially input video signal value and a key value of each pixel; An alternative pixel value generation unit that generates an alternative pixel value from a video signal of the object pixel based on a key value of each pixel sequentially input;
A padding pixel position holding unit that holds information indicating a position of a padding pixel based on a key value of each pixel sequentially input, and a padding pixel specified by referring to the padding pixel position holding unit, and the specified padding is specified. The pixel includes a filtering unit that performs a filtering process using the substitute pixel value and the value of the video signal in the buffer memory.

As a result, a useless step of simply replacing the pixel value of the padding pixel with the substitute pixel value is bypassed, and the padding process in which the replacement and the filtering process are integrated is executed. As a result, an intermediate buffer memory for storing a state in which the pixel values of the padding pixels are merely replaced with alternative pixel values is not required, and a process of repeatedly reading alternative pixel values from the buffer memory is not required. This has the effect of reducing the scale and improving the processing speed.

The present invention is a general-purpose padding device having only the function of a pixel compensating means which is a characteristic component of the image object material generating device (performing a padding process for filling the outside of the image object). It is realized as an image object material generation method and a padding method using the characteristic components of the image object material generation device and the padding device as processing steps, or as a program that causes a computer to execute those steps. You can also.
It goes without saying that the program can be distributed via a computer-readable recording medium such as a CD-ROM or a transmission medium such as a communication path.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image object material generation device according to the present invention.

FIG. 2A is a diagram illustrating the content of a video signal processed by the image object material generation device illustrated in FIG. 1; FIG. 2B is a diagram illustrating an example of a configuration of a video signal and a key signal used in the image object material generation device illustrated in FIG. 1. (C) is a diagram showing keys when the hue is represented by the circumference and the saturation is represented by the radius.

FIG. 3 is a block diagram illustrating an example of an internal configuration of a key generation unit illustrated in FIG. 1;

FIG. 4 is a block diagram illustrating an example of an internal configuration of a frame size conversion unit illustrated in FIG. 1;

FIG. 5 is a block diagram showing an example of an internal configuration of a low-frequency passing unit shown in FIG.

FIG. 6 is a block diagram illustrating an example of an internal configuration of a pixel compensator illustrated in FIG. 1;

FIG. 7 is a block diagram illustrating another example of the internal configuration of the pixel compensation unit illustrated in FIG. 1;

FIG. 8 is a flowchart illustrating an example of a processing procedure of an average value calculation unit illustrated in FIG. 6;

FIG. 9 is a flowchart illustrating a processing procedure of a padding processing unit illustrated in FIG. 6;

FIG. 10 is a flowchart illustrating an example of a processing procedure of a filter processing unit illustrated in FIG. 6;

11 is a flowchart illustrating an example of a processing procedure of a pixel identification unit and an object pixel average value calculation unit illustrated in FIG. 7;

FIG. 12 is a flowchart illustrating an example of a processing procedure of a substitute pixel supply unit illustrated in FIG. 7;

FIG. 13A is a diagram illustrating an example of a luminance value of each pixel in input data of a pixel compensation unit. FIG.
FIG. 4 is a diagram illustrating luminance of an input image displayed based on the input data illustrated in FIG.

FIG. 14A is a diagram illustrating an example of a key value calculated from a color difference of each pixel in input data of a pixel compensation unit. FIG. 15B is a diagram showing a padding map of the padding map holding unit obtained by setting the threshold value to “100” from the key values shown in FIG.

15A is a diagram illustrating luminance after the alternative pixel supply unit performs a two-dimensional filtering process on the video block illustrated in FIG. 13A using the padding map illustrated in FIG. 14B. It is a figure showing a value. (B) is a figure which shows visually the brightness value after two-dimensional filtering processing by a substitute pixel supply part. FIG. 17C is a diagram showing the correspondence between the brightness value of each pixel in the video block shown in FIG. 15B and hatching.

FIG. 16 is a diagram showing, for each pixel of the video block, the number of times a buffer memory is referred to in the two-dimensional filtering of the alternative pixel supply unit.

FIG. 17 is a diagram illustrating an application example of an image object material generation device to an actual product.

FIG. 18A shows one of N × N pixel (N is a positive integer) block including a boundary between an object pixel and a background pixel when an image including an object and a background is scanned in the direction of an arrow. FIG. 18B is a graph illustrating a change in a pixel value (for example, a luminance value) of a video signal when an image in the block illustrated in FIG. 18A is scanned in the x direction.

FIG. 19 is a block diagram showing a configuration of a conventional padding processing device disclosed in Japanese Patent Application Laid-Open No. 10-210470.

[Explanation of symbols]

 Reference Signs List 10 image object material generation device 11 key generation unit 12 frame size conversion unit 13 low frequency passage unit 14 pixel compensation unit 15 pixel selection unit 16 image data holding unit

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Tadashi Kobayashi 1006 Kazuma Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. F term (reference) 5B057 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CD05 CF04 CH18 5C059 KK11 LB05 LB15 MA00 MA23 MB06 MB27 PP16 PP28 PP29 SS08 SS10 UA02 UA12 UA38 5C076 AA21 AA22 BB04 BB06 CB01

Claims (28)

[Claims] A key generation unit configured to generate a key serving as a reference for distinguishing an object pixel and a background pixel for each pixel from an input video signal; a frame size changing unit configured to change a frame size of the video signal; A low-frequency component passing unit that removes a high-frequency component of the video signal processed by the frame size changing unit; and a video signal processed by the low-frequency component passing unit, based on the key generated by the key generation unit. , Lo
w Pass Extrapolation a pixel compensating unit that performs padding processing by a padding algorithm, and a pixel selecting unit that selects one of a video signal processed by the low-frequency component passing unit and a video signal processed by the pixel compensating unit. An image object material generation device, comprising: 2. The image processing apparatus according to claim 1, wherein the key generation unit further includes a reference value input unit configured to receive an input of a reference value for generating a key from the video signal. 2. The image object material generating apparatus according to claim 1, wherein a key is generated from the image object material. 3. The frame size changing unit further includes a magnification input unit that receives an input of a magnification such as enlargement, reduction, or equal magnification of a frame size, wherein the frame size changing unit has a frame size according to the input magnification. 2. The method according to claim 1, wherein
Or the image object material generation device according to 2. 4. The low-frequency passing unit further performs a low-frequency passing process on the input video signal when the frame size changing process by the frame size changing unit is a reduction process, The image processing apparatus according to claim 1, further comprising a selection unit configured to directly output the video signal.
3. The image object material generation device according to any one of 3. 5. The image object material generating device according to claim 1,
5. The image object material generating apparatus according to claim 1, further comprising a video data holding unit having a storage area for holding output data of the image compensating unit. 6. The image object material generating apparatus according to claim 5, wherein said video data holding means operates by selecting whether or not to hold a video signal in accordance with an external instruction. 7. A buffer memory for holding a video signal value and a key value for one video block consisting of N × N (N is a natural number) pixels, An average value calculation unit that adds the value of the video signal of a pixel having a key value equal to or greater than a set threshold value, counts the number of pixels, and obtains an average value of the video signal value of the pixel; A padding processing unit that extracts a key value held in a memory, detects a pixel having a key value less than the threshold value, and replaces a value of a video signal of the pixel with an average value obtained by the average value calculation unit. The value of the video signal output from the padding processing unit is 2
A buffer memory for holding N pixels; extracting a key value held in the buffer memory; detecting a pixel whose extracted key value is less than the threshold value; A filter processing unit that refers to a buffer memory for a value and replaces a value of a video signal of a detected pixel with an average of the four adjacent pixels.
7. The image object material generation device according to any one of 6. 8. The pixel compensating means further comprises: a buffer memory for holding a video signal value and a key value of each pixel sequentially input; and a memory for storing the object based on the key value of each pixel sequentially input. An alternative pixel value generation unit that generates an alternative pixel value from a video signal of a pixel; a padding pixel position holding unit that holds information indicating a position of a padding pixel based on a key value of each pixel sequentially input; A padding pixel is specified by referring to the pixel position holding unit, and a filtering unit that performs a filtering process on the specified padding pixel using the substitute pixel value and the value of the video signal in the buffer memory is provided. The image object material generation device according to claim 1. 9. The padding pixel average value calculation unit that detects the object pixel based on the key value and calculates an average value of video signal values of the detected object pixel. The image object material generating apparatus according to claim 8, wherein the filtering unit performs the filtering process using an average value calculated by the padding pixel average value calculation unit as the substitute pixel value. 10. The filtering unit further includes: a replacement pixel value holding unit that holds the replacement pixel value; and a pixel sequentially holding a video signal value and a key value in the buffer memory. A padding pixel detection unit for detecting whether or not a pixel of interest is a padding pixel; If an adjacent pixel surrounding the pixel is an object pixel, an adjacent pixel value reading unit that reads the value of the video signal from the buffer memory; a substitute pixel value of the adjacent pixel read by the adjacent pixel value reading unit and a value of the video signal And an adjacent pixel average value calculation unit that calculates an average value of the padding pixels. Serial and replaces at the calculated average value in the adjacent pixel value calculating unit according to claim 9 image object material generating apparatus according. 11. The image according to claim 10, wherein the adjacent pixel value reading unit determines whether an adjacent pixel surrounding the padding pixel is a padding pixel with reference to the padding pixel position holding unit. Object material generation device. 12. The padding pixel position holding section, wherein:
By arranging padding pixels in the outer one pixel portion for a video block of × N (N is a natural number) pixels,
The image object material generating apparatus according to claim 8, wherein information indicating a position of a padding pixel of a video block of (N + 2) × (N + 2) pixels is held. 13. The padding pixel position holding unit holds, for each pixel, 1-bit data indicating whether or not the pixel is a background pixel, corresponding to the position of each pixel in the video block. The image object material generation device according to any one of claims 8 to 12. 14. In a video block including both a background pixel and an object pixel, a pixel value of a padding pixel defined as one of the background pixel and the object pixel is replaced with an alternative pixel value to thereby obtain a correspondence between the background and the object. A padding device for smoothing a boundary, comprising: image data holding means for holding a pixel value of a pixel in the video block; alternative pixel value holding means for holding the alternative pixel value; and a padding pixel in the video block. Map holding means for holding a map indicating the position of the image data, and filtering means for filtering the padding pixels in the video block by referring to the map using the alternative pixel values as the pixel values of the padding pixels. A padding device comprising: 15. The map according to claim 1, wherein one bit of data indicating whether or not each pixel is a padding pixel is held in correspondence with the position of each pixel in the video block. 15. The padding device according to 14. 16. The map, when the size of the video block is N × N pixels, is generated with a size of (N + 2) × (N + 2) pixels, with an outer peripheral portion having a width of 1 pixel as padding pixels. The padding device according to claim 14 or 15, wherein: 17. The filtering unit focuses on a pixel in the video block, and determines whether or not the focused pixel is a padding pixel with reference to the map,
If it is determined that the pixel is a padding pixel, the substitute pixel value held by the alternative pixel value holding means is read out for a pixel adjacent to the pixel according to the map if the pixel is a padding pixel. By reading the pixel values of the pixel data from the image data holding means, calculating the average value of the pixel values of the adjacent pixels, and replacing the pixel value of the pixel of interest with the obtained average value. The padding device according to any one of claims 14 to 16. 18. The padding apparatus further comprises: identifying whether the pixel is a padding pixel based on a key value associated with the pixel, and storing a result as the map in the map holding unit. 18. The padding device according to claim 14, further comprising means. 19. The padding device further calculates an average value of pixel values of pixels identified as not padding pixels as a result of the identification by the identification unit, and sends the average value to the alternative pixel value holding unit as the alternative pixel value. 19. The padding device according to claim 18, further comprising a substitute pixel value generating unit for storing. 20. The padding device further comprises: a pixel value writing unit for storing a pixel value of each pixel sequentially inputted in the image data holding unit; and the identification unit comprises: 20. The padding device according to claim 19, wherein the identification is performed in parallel with writing by the pixel value writing unit. 21. A key generation step of generating a key serving as a reference for distinguishing an object pixel and a background pixel for each pixel from an input video signal, a frame size changing step of changing a frame size of the video signal, A low frequency component passing step of removing a high frequency component of the video signal processed in the frame size changing step; and a video signal processed in the low frequency component passing step, based on the key generated in the key generation step. A pixel compensation step of performing padding processing by a Low Pass Extrapolation padding algorithm, and a pixel selection step of selecting any of the video signal processed in the low frequency component passing step and the video signal processed in the pixel compensation step. Image object raw material characterized by including Method. 22. The pixel compensating step further includes a buffer memory writing step of writing a video signal value and a key value of each pixel sequentially input to a buffer memory, and a key value of each sequentially input pixel. An alternative pixel value generating step of generating an alternative pixel value from the video signal of the object pixel; and writing information indicating the position of the padding pixel to the padding pixel position holding unit based on the sequentially input key value of each pixel. A padding pixel position writing step, a padding pixel is specified by referring to the padding pixel position holding unit, and a filtering process is performed on the specified padding pixel by using the substitute pixel value and the value of the video signal in the buffer memory. 22. The image object material generation method according to claim 21, further comprising: Method. 23. In a video block including both a background pixel and an object pixel, a pixel value of a padding pixel defined as one of the background pixel and the object pixel is replaced with an alternative pixel value to thereby obtain a background pixel and an object pixel. An image memory writing step of writing pixel values of pixels in the video block to an image memory; and a method of generating the substitute pixel value and storing the substitute pixel value in a substitute pixel value holding unit. A pixel value generation step, a map generation step of generating a map indicating the positions of padding pixels in the video block and storing the map in a map holding unit, and using the alternative pixel value as a pixel value of the padding pixel, Filtering the padding pixels in the block with reference to the map. Padding wherein the Mukoto. 24. The map according to claim 24, wherein the map stores 1-bit data indicating whether each pixel is a padding pixel in correspondence with the position of each pixel in the video block. 23. The padding method according to 23. 25. A program for causing a computer to execute the steps according to claim 21. 26. A program for causing a computer to execute the steps according to claim 22. 27. A program for causing a computer to execute the steps according to claim 23. 28. A program for causing a computer to execute the steps according to claim 24.