JP2002271799A - Moving image processing unit, moving image processing method, and moving image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a data quantity to be encoding-processed in encoding a moving image. SOLUTION: When encoding data to reproduce a moving image including a 1st image and a 2nd image after the 1st image and outputting the encoded data, a difference calculation section 11 obtains a difference between the 1st image and the 2nd image. A motion block extract section 12 extracts a motion block whose difference obtained by the difference calculation section 11 exceeds a prescribed threshold value. Then a data output section 13 encodes data as to the motion block in succession to data as to the 1st image and provides an output. Thus, synthesizing the data as to the 1st image with the data as to the motion block can reproduce a moving image. Thus, since a background part or the like without a motion is omitted and only a part with a motion is encoded and outputted, the data quantity to be encoded can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, a moving image processing method, and a moving image processing program.
PEG (Moving Picture Expert)
The present invention relates to an image processing apparatus, a moving image processing method, and a moving image processing program for performing shape coding of the s Group) 4 standard.

[0002]

2. Description of the Related Art In core profile encoding processing in MPEG4, which is one of the MPEG standards, the shape of an object included in a moving image is extracted, and encoding processing is performed on the data of the extracted shape. This c
The orefile encoding is performed by DCT (Discrete).
Instead of performing processing in block units of 8 pixels × 8 pixels as in te Cosine Transform processing, coding is performed by adding shape signals of binary data.

[0003]

SUMMARY OF THE INVENTION
In file encoding, shape information is complicated,
There is a problem that the amount of data to be encoded is large. Therefore, it is desired to reduce the amount of data to be encoded and to simplify the processing. The present invention has been made in order to solve the above-described drawbacks of the related art, and has as its object the purpose of an image processing device, a moving image processing method, and a data amount to be encoded that are smaller.
And a moving image processing program.

[0004]

An image processing apparatus according to a first aspect of the present invention encodes and outputs data for reproducing a moving image including a first image and a second image subsequent to the first image. What is claimed is: 1. A moving image processing apparatus, comprising: a difference calculating unit that calculates a difference value between the first image and the second image; and a motion that extracts a motion block whose difference value obtained by the difference calculating unit exceeds a predetermined threshold value. Block extracting means;
Data output means for encoding and outputting the data about the motion block following the data about the image, and synthesizing the data about the first image and the data about the motion block. Can be reproduced.

[0005] An image processing apparatus according to a second aspect of the present invention comprises:
2. The difference calculation unit according to claim 1, wherein the difference calculation unit obtains a difference value between the first image and the second image for each pixel, and the motion block extraction unit calculates the difference value obtained by the difference calculation unit. Extract a motion block including a pixel exceeding a predetermined threshold value. An image processing apparatus according to a third aspect of the present invention is the image processing apparatus according to the first or second aspect, further comprising: a unit that removes an isolated point from the difference value obtained for each pixel by the difference calculating unit. As for the value, the motion block extracting means extracts a motion block including a pixel exceeding a predetermined threshold value.

An image processing apparatus according to a fourth aspect of the present invention comprises:
2. The motion block extraction device according to claim 1, wherein the difference calculation means obtains a difference value between a motion vector in the first image and a motion vector in the second image for each block which is a processing unit of the image. The means is characterized by extracting a motion block in which the difference value of the motion vector obtained by the difference calculation means exceeds a predetermined threshold.

According to a fifth aspect of the present invention, there is provided an image processing apparatus comprising:
5. The data output unit according to claim 1, wherein the number of motion blocks extracted by the motion block extraction unit occupies half or more of the number of blocks constituting the entire second image. And encoding and outputting data on the first image and data on the motion block.

[0008] An image processing apparatus according to claim 6 of the present invention comprises:
In any one of claims 1 to 5, a predetermined spatial smoothing filter process is performed on the first image and the second image before a difference value is obtained by the difference calculation unit. An image processing method according to claim 7 of the present invention is a moving image processing method for encoding and outputting data for reproducing a moving image including a first image and a second image subsequent to the first image, A difference calculating step for calculating a difference value between a first image and the second image, a motion block extracting step for extracting a motion block in which the difference value obtained in the difference calculating step exceeds a predetermined threshold;
A data output step of encoding and outputting data on the motion block, following the data on the motion image, and synthesizing the data on the first image and the data on the motion block. Can be reproduced.

[0009] An image processing method according to claim 8 of the present invention comprises:
In claim 7, in the difference calculation step,
A difference value between the first image and the second image is obtained in pixel units. In the motion block extraction step, a motion block including a pixel whose difference value obtained in the difference calculation step exceeds a predetermined threshold is extracted. It is characterized by doing.

[0010] The image processing method according to claim 9 of the present invention comprises:
According to claim 7 or 8, the difference value obtained for each pixel in the difference calculation step, further comprising the step of removing an isolated point, the difference value after the isolated point removal, the predetermined threshold value in the motion block extracting step It is characterized in that a motion block including a pixel exceeding the pixel is extracted.

According to a tenth aspect of the present invention, in the seventh aspect, in the difference calculating step, a difference value between a motion vector in the first image and a motion vector in the second image is calculated by: It is obtained for each block which is a processing unit of an image, and in the motion block extracting step, a motion block in which a difference value of a motion vector obtained in the difference calculating step exceeds a predetermined threshold value is extracted.

[0012] In the image processing method according to claim 11 of the present invention, in the data output step, in the data output step, the number of motion blocks extracted in the motion block extraction step is equal to the second motion block number. The data of the first image and the data of the motion block are encoded and output only when they occupy half or more of the number of blocks constituting the entire image.

According to a twelfth aspect of the present invention, there is provided the image processing method according to any one of the seventh to eleventh aspects, wherein the first image and the second image are determined before the difference value is obtained by the difference calculation step. Is characterized by performing the spatial smoothing filter processing of An image processing program according to claim 13 of the present invention is a moving image processing program that encodes and outputs data for reproducing a moving image including a first image and a second image subsequent to the first image. A difference calculating step of calculating a difference value between a first image and the second image, a motion block extracting step of extracting a motion block whose difference value obtained in the difference calculating step exceeds a predetermined threshold value, A data output step of encoding and outputting data on the motion block, following the data on the image, and synthesizing the data on the first image and the data on the motion block to generate a moving image. It is characterized in that it can be reproduced.

According to a fourteenth aspect of the present invention, in the image processing program according to the thirteenth aspect, in the difference calculating step, a difference value between the first image and the second image is obtained for each pixel. In the block extracting step, a motion block including a pixel whose difference value obtained in the difference calculating step exceeds a predetermined threshold value is extracted.

The image processing program according to claim 15 of the present invention according to claim 13 or 14, further comprising the step of removing an isolated point from the difference value obtained for each pixel in the difference calculating step. With respect to the subsequent difference value, a motion block including a pixel exceeding a predetermined threshold value is extracted in the motion block extraction step.

According to a sixteenth aspect of the present invention, in the image processing program according to the thirteenth aspect, in the difference calculating step, a difference value between a motion vector in the first image and a motion vector in the second image is calculated by: It is obtained for each block which is a processing unit of an image, and in the motion block extracting step, a motion block in which a difference value of a motion vector obtained in the difference calculating step exceeds a predetermined threshold value is extracted.

According to a seventeenth aspect of the present invention, in the image processing program according to any one of the thirteenth to sixteenth aspects, in the data output step, the number of motion blocks extracted in the motion block extraction step is the second motion block number. The data of the first image and the data of the motion block are encoded and output only when they occupy half or more of the number of blocks constituting the entire image.

An image processing program according to an eighteenth aspect of the present invention is the image processing program according to any one of the thirteenth to seventeenth aspects, wherein the first value is obtained before the difference value is calculated by the difference calculation step.
A predetermined spatial smoothing filter process is performed on the image and the second image. In short, since a background portion with no motion is omitted and only a portion with motion is encoded and output, the amount of data to be encoded can be further reduced. Thereby, the processing can be simplified. Since the processing is performed in units of blocks, there is no problem of unnatural boundaries in the synthesized image as in the case of encoding an arbitrary shape.

[0019]

Next, an embodiment of the present invention will be described with reference to the drawings. In the drawings referred to in the following description, the same parts as those in the other drawings are denoted by the same reference numerals. FIG. 1 is a block diagram showing an embodiment of a moving image processing apparatus according to the present invention. As shown in the figure, the moving image processing apparatus according to the present embodiment includes a difference calculation unit that obtains a difference value between a first image that is the first image and a second image that is an image subsequent to the first image. 11, a motion block extraction unit 12 for extracting a motion block in which the difference value obtained by the difference calculation unit 11 exceeds a predetermined threshold value, and encoding and outputting the data for the motion block following the data for the first image. And a data output unit 13.

In such a configuration, the first frame (first image) is encoded and output as a reference image over the entire image. This image is, for example, JPEG (Joint Ph)
autographic Experts Group)
It is an image. Next, with respect to the next frame (second image) following the frame, only blocks having motion are encoded based on the frame (first image).
In this case, it is determined whether or not there is a motion in the block in units of a block which is a unit of the DCT processing. As a result, it is divided into those that have movement in the block and those that do not move,
Encode only blocks with motion. More specifically, for the first image and the second image following it,
A difference value between pixels is calculated for each block. If the calculated difference value is larger than a predetermined threshold, it is determined that the pixel has a motion. This threshold is provided by
This is to avoid the influence of noise. For example, a certain range is set for each value of YUV or RGB for each pixel (for example, plus or minus 5 for an 8-bit image),
If it is within the range, it is determined that there is no movement.

In the above encoding process, a plurality of frame units such as 30 frames are subjected to image processing. The plurality of frames to be subjected to image processing may be stored in a buffer or the like in advance, if necessary, and read and processed. The data encoded and output as described above is decoded, and sequentially combined with the first frame, thereby restoring the original image.

It should be noted that if the proportion of the block occupied by the motion in the image is more than a certain value (for example, more than 50%, that is, more than half), the code amount is rather increased. is not. In addition, processing such as diffusion / degeneration and removal of isolated points may be performed on data after detecting a difference value between two consecutive images.
By this processing, it is possible to remove noise and a very small area, and after this removal, it is determined whether or not the block includes a motion area in block units.

The above processing will be further described with reference to FIG. FIG. 2 is a flowchart showing an image processing method in the moving image processing apparatus shown in FIG.
In the figure, first, it is determined whether the image is the first image (step S101). If it is the first image, the entire image is encoded (step S101 → S105). The encoded data is output (step S106).

If it is not the first image in step S101, a difference value from the first image is calculated (step S101 → S102). After detecting the difference value, processing such as removal of an isolated point is performed (step S103).
The above processing is performed for each block constituting the image in pixel units. If the difference value after the removal of the isolated points does not exceed the predetermined threshold, it is determined that there is no motion, and the process is performed on the next pixel (step S104 → S102). If the difference value after the removal of the isolated point exceeds a predetermined threshold value, it is determined that there is a motion, and the block including the pixel is encoded (step S104 → S105). The encoded data is output (step S106).

Since the encoding process is performed as described above, the following moving image processing method is realized in the present apparatus. That is, this is a moving image processing method for encoding and outputting data for reproducing a moving image including a first image and a second image subsequent to the first image, wherein the first image, the second image, Calculating a difference value of the motion block; a motion block extracting step of extracting a motion block in which the difference value obtained in the difference calculating step exceeds a predetermined threshold value; and following the data of the first image, And a data output step of encoding and outputting the data of the first image and the data of the first image and the data of the motion block. It will be.

In the difference calculating step, a difference value between the first image and the second image is obtained for each pixel, and in the motion block extracting step, the difference value obtained in the difference calculating step is calculated. A motion block including a pixel exceeding a predetermined threshold is extracted. In addition, the method further includes the step of removing an isolated point from the difference value obtained for each pixel in the difference calculation step. Extract blocks.

In the data output step, the number of motion blocks extracted in the motion block extraction step occupies at least half of the number of blocks constituting the entire second image. The data on one image and the data on the motion block are encoded and output. By performing the encoding process as described above, since the encoding only needs to be performed for a block having motion, the amount of code can be reduced. In addition, image synthesis in the decoding process is very simple because it is only necessary to superimpose two image data. In the moving image processing according to the present invention, since only the moving part is encoded, the commercial part (C
M) and the like, and are particularly suitable for images in which the background does not move much.

Here, if a program for realizing the operation shown in FIG. 2 and the like is prepared and the computer is controlled by the program, it is obvious that the moving picture coding processing can be performed in the same manner as described above. is there. As a recording medium for recording and holding this program, various recording media can be used in addition to a semiconductor memory, a magnetic disk, an optical disk, etc., not shown in FIG.

The above processing is performed on a plurality of frame units, so that the applicable range is wide. Hereinafter, a development example of the present invention will be described. (Development Example 1) In the moving image processing, the data amount is large in the first frame in both rectangular shape encoding and shape encoding. Therefore, if not all the data of the first frame is output but only the range where no data exists in the subsequent frame, the data amount can be further reduced.

(Development Example 2) The motion vector of the entire image is
It is obtained in block units, and a certain percentage of the value (for example, 50
%) Are the same, and if the motion vectors are the same in a plurality of consecutive frames (for example, 30 frames), only blocks having values different from the motion vectors that occupy the majority are coded. .

An image is created by integrating blocks having the same motion vector throughout the 30 frames, and the image is encoded. In the case of decoding, the integrated image is combined with the coded block in frame units to restore the original image. In this way, by performing processing by paying attention to the motion vector, it is possible to apply even when the camera is moving in parallel, etc., so that the applicable range becomes wider. When processing is performed by focusing on a motion vector, in step S102 in FIG. 2, a difference value between the motion vector in the first image and the motion vector in the second image is calculated for each block which is a processing unit of the image. In step S104 in the figure, a motion block in which the difference value of the motion vector obtained in the difference calculation step exceeds a predetermined threshold value is extracted.

(Development Example 3) When an image contains a large amount of high-frequency noise, 3 ×
3 is applied to the spatial smoothing filter processing. in this case,
Before performing the processing shown in FIG. 2 described above, a predetermined spatial smoothing filter processing is performed on the first image and the second image.

(Expansion Example 4) If the data of the first frame is displayed without being superimposed, an image of only a moving block is obtained, but important information is often included in a moving portion. By giving priority to data, there is less risk of important information leaking. For example, in the error processing of MPEG4, processing is performed from the backward direction in addition to the forward direction.
At this time, such processing is performed only on a moving block. By doing so, a block with movement is preferentially treated, and there is less concern that important information will be leaked.

(Development Example 5) A motion sensor may be provided in a device on which the present device is mounted, and the function of the present device may be enabled when the motion sensor detects that the device is not moving. When a motion picture made the (development Example 6) processed is composed of an image in motion, such as a background image and the person, leave create a separate whole background image, and outputs it to the first, and then the motion Only a certain portion of the image data may be output. In this case, the processing is simpler than outputting one sheet at a time.

[0035]

As described above, according to the present invention, the amount of data to be encoded can be further reduced by omitting the background portion without motion and encoding and outputting only the portion having motion. There is an effect that the processing can be simplified.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a moving image processing device according to the present invention.

FIG. 2 is a flowchart showing a processing procedure of a moving image processing method according to the present invention.

[Explanation of symbols]

 11 difference calculation unit 12 motion block extraction unit 13 data output unit

Claims (18)

[Claims] 1. A moving image processing apparatus for encoding and outputting data for reproducing a moving image including a first image and a second image subsequent to the first image, wherein the first image and the second image A difference calculating means for calculating a difference value between the second image and the second image; a motion block extracting means for extracting a motion block in which the difference value obtained by the difference calculating means exceeds a predetermined threshold value; And data output means for encoding and outputting data on the motion block, wherein the data on the first image and the data on the motion block are combined to reproduce a moving image. Characteristic moving image processing device. 2. The method according to claim 1, wherein the difference calculating means includes:
Calculating a difference value between the image of the second image and the second image in pixel units, wherein the motion block extracting means extracts a motion block including a pixel whose difference value obtained by the difference calculating means exceeds a predetermined threshold value. The moving image processing device according to claim 1, wherein 3. The image processing apparatus according to claim 1, further comprising: a unit that removes an isolated point from the difference value obtained for each pixel by the difference calculation unit. The moving image processing device according to claim 1, wherein a motion block including the following is extracted. 4. The method according to claim 1, wherein the difference calculating means includes a step of:
The difference value between the motion vector in the image and the motion vector in the second image is obtained for each block which is a processing unit of the image. In the motion block extraction means, the difference between the motion vectors obtained in the difference calculation means is obtained. 2. A motion block whose value exceeds a predetermined threshold is extracted.
The moving image processing apparatus according to the above. 5. The data output means according to claim 1, wherein said number of motion blocks extracted by said motion block extraction means occupies at least half of the number of blocks constituting the entire second image. The moving image processing apparatus according to claim 1, wherein data on one image and data on the motion block are encoded and output. 6. The method according to claim 1, wherein a predetermined spatial smoothing filter process is performed on the first image and the second image before a difference value is obtained by the difference calculation unit. 3. The moving image processing device according to claim 1. 7. A moving image processing method for encoding and outputting data for reproducing a moving image including a first image and a second image subsequent to the first image, wherein the first image and the second image a difference calculation step of calculating a difference value between two images, a motion block extracting step the difference value obtained in the difference calculation step for extracting a motion block exceeds a predetermined threshold, following the data for the first image And a data output step of encoding and outputting data on the motion block, wherein the data on the first image and the data on the motion block are combined to reproduce a moving image. Characteristic moving image processing method. 8. In the difference calculating step, a difference value between the first image and the second image is obtained in pixel units. In the motion block extracting step, the difference value obtained in the difference calculating step is obtained. 8. The moving image processing method according to claim 7, wherein a motion block including a pixel exceeding a predetermined threshold value is extracted. 9. The method according to claim 9, further comprising the step of removing an isolated point from the difference value obtained for each pixel in the difference calculation step. 8. A motion block including a motion block is extracted.
Or the moving image processing method according to 8. 10. In the difference calculation step, a difference value between a motion vector in the first image and a motion vector in the second image is obtained for each block which is a processing unit of the image. 8. The moving image processing method according to claim 7, wherein in the step, a motion block whose difference value of the motion vector obtained in the difference calculating step exceeds a predetermined threshold value is extracted. 11. In the data output step,
Only when the number of motion blocks extracted in the motion block extraction step occupies half or more of the number of blocks constituting the entire second image, the data of the first image and the The moving image processing method according to any one of claims 7 to 10, wherein the data is encoded and output. 12. The method according to claim 7, wherein a predetermined spatial smoothing filter process is performed on the first image and the second image before obtaining a difference value in the difference calculation step. 5. The moving image processing method according to item 1. 13. A moving image processing program for encoding and outputting data for reproducing a moving image including a first image and a second image subsequent to the first image, wherein the first image and the second image A difference calculation step for calculating a difference value between the second image and the second image; a motion block extraction step for extracting a motion block in which the difference value obtained in the difference calculation step exceeds a predetermined threshold value; And a data output step of encoding and outputting data on the motion block, wherein the data on the first image and the data on the motion block are combined to reproduce a moving image. Characteristic moving image processing program. 14. In the difference calculation step, a difference value between the first image and the second image is obtained in pixel units. In the motion block extraction step, the difference value obtained in the difference calculation step is obtained. 14. A moving image processing program according to claim 13, wherein a motion block including a pixel exceeding a predetermined threshold value is extracted. 15. The method according to claim 15, further comprising the step of removing an isolated point from the difference value obtained for each pixel in the difference calculating step, wherein the difference value after the removal of the isolated point exceeds a predetermined threshold value in the motion block extracting step. The moving image processing program according to claim 13, wherein a motion block including the following is extracted. 16. In the difference calculating step, a difference value between a motion vector in the first image and a motion vector in the second image is obtained for each block which is a processing unit of the image. 14. The moving image processing program according to claim 13, wherein in the step, a motion block in which the difference value of the motion vector obtained in the difference calculating step exceeds a predetermined threshold value is extracted. 17. In the data output step,
Only when the number of motion blocks extracted in the motion block extraction step occupies half or more of the number of blocks constituting the entire second image, the data of the first image and the 17. The method according to claim 13, wherein the data is encoded and output.
The moving image processing program according to any one of the above. 18. The method according to claim 13, wherein a predetermined spatial smoothing filter process is performed on the first image and the second image before obtaining a difference value in the difference calculation step. 4. A moving image processing program according to claim 1.