JP5147487B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing technique applied to, for example, a gaming machine such as a 3D game device or an action puzzle game, and more particularly to an improvement of a technique for rewriting a part of a moving image.

  Conventionally, the method of composing a screen with 2D / 3D real-time CG has been common, but in recent years, with the advancement of video compression technology, the method of composing the screen with movie playback has spread. The method of configuring the screen by movie playback may be applied to the entire screen, or may be applied to a part of the screen and combined with the real-time CG to display both.

The method of using movie playback has the following advantages.
(1) Pre-rendering CG provides higher image quality and higher image quality than real-time CG.
(2) Live video can be displayed.
(3) Movie playback control is easier to program than real-time CG control.
(4) The load on the programmer can be reduced.
(5) The screen can be created by the designer alone without any programmers, and the screen adjustment can be completed.
(6) Development man-hours can be reduced.

  Disadvantage of the method of configuring the screen in movie playback is that even if a small difference in part of the screen, to express what is different, you must prepare a separate video file .

  For example, for a video of a scene where one colored pencil is purchased, in order to support eight colored pencils, eight moving image files must be prepared. These moving image files differ only in the color information of the pencil part, and the other parts are completely the same. As in this example, when there are a plurality of movies in which a part of the screen is different but the other parts are common, these movies are called different color movies.

  A difference is limited to a specific part in the screen and the other parts are common, and can be regarded as a different color movie. In the above example, even if there is a scene where a ballpoint pen is purchased in addition to a color pencil, if the ballpoint pen is in a position almost overlapping with the color pencil and other parts are common, it can be handled as the same color-different movie. If different color movies are handled individually, even if they are compressed, a data size proportional to the number of movies is required.

  The problem to be solved by the present invention is to eliminate the redundancy of different color movies and increase the compression efficiency.

An image processing apparatus and an image processing method according to the present invention convert a plurality of moving images that differ only in the colors of some areas into a base file and a difference file, and compress the converted base file and the difference file. It is characterized by.

With such a configuration, by converting a plurality of video images into a base file and differencing file, information of the difference file becomes small, compressing the reduced-differencing file base file and the amount of information Thus, the overall compression efficiency can be increased.

Preferably, the image processing means, to create the base file by extracting the minimum value of each color component for each pixel of the plurality of moving images, of all pixels for each color component of the plurality of moving image It can create the base file to calculate the average value. Further, the video processing means creates the base file by extracting the maximum value of each color component for each pixel from the plurality of moving images , or selects one color difference to be a target from the plurality of moving images. movie select the image you or with the base file. Further, the image processing means, truncate data out of the effective area of the converted differencing file, along with the data before Symbol differencing file data in the effective area, of those said difference file that records the relative position of the data of the data before and Kibe over scan file.

Further, the present invention is an image reproducing apparatus and an image reproducing method for reproducing a moving image processed by the image processing apparatus and image processing method, and eject the base file and the difference file, data of the base file characterized by additively synthesizing the data of the difference file and.

  Further, the present invention configures a machine-readable recording medium storing a program for causing a computer to execute any one of the image processing methods.

  Here, the recording medium is a medium in which information, mainly digital data, and a program are recorded by some physical means, and can cause a processing device such as a computer or a dedicated processor to perform a predetermined function. It is.

  In short, what is necessary is just to download a program to a computer and execute a predetermined function by some means. For example, flexible disk, fixed disk, magnetic tape, magneto-optical disk CD, CD-ROM, CD-R, DVD-RAM, DVD-ROM, DVD-R, PD, MD, DCC, ROM cartridge, RM with battery backup Includes memory cartridges, flash memory cartridges, non-volatile RAM cartridges and the like.

According to the present invention, it is possible to increase the compression efficiency when there are a plurality of moving images that differ only in the color of a partial region.

  Preferred embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram of an image processing apparatus showing an embodiment of the present invention. In FIG. 1, the image processing apparatus includes a 3D processing unit 301 and a movie video processing unit 303. The movie video processing unit 303 converts the CD-ROM data into movie video MV by the decoder 310. The 3D processing unit 301 decodes the CD-ROM data by the decoder 311 to obtain a 3D model (3DM), and the modeling unit 312 performs modeling such as polygon generation. Note that the drawing processor 305 can temporarily store each image of the movie video MV in the texture buffer 313, read it appropriately, and draw it together with the character.

  Here, when rewriting a part of a moving image (moving image), for example, as shown in FIG. 2, moving images 30, 32, When encoding 34, 36, and 38, if each moving image 30 to 38 is stored on a disk, the amount of information becomes large as it is, and cannot be stored on a disk of a specific capacity, and compression efficiency is improved. It cannot be raised.

  Therefore, as shown in FIG. 3, the moving image 30 including the arrow A is used as a base file (parent), and the difference (child difference) between the moving image 30 and each moving image 32, 34, 36, 38 is extracted. If the extracted movies (difference files) 42, 44, 46, and 48 are used, the amount of information of the difference extracting movies 42, 44, 46, and 48 is smaller than that of the moving images 32, 34, 36, and 38. Has a very high compression ratio, which is 1/5 to 1/7 when the moving image 30 is compressed. When reproducing the difference extraction movies 42 to 48, the original moving images 32, 34, and 36 are obtained by adding and synthesizing the moving image 30 serving as the base (parent) and the difference extracting movies 42 to 48 serving as child differences, respectively. , 38 can be reproduced.

  When encoding the moving images 30 to 38, as shown in FIG. 4, the file F1 for storing the RGB format material (A) of the moving image 30 and the RGB format material (B) of the moving image 32 are stored. The moving images 30 and 32 are fetched from the file F2 to be processed into the differentiator 50, and the difference 50 extracts the difference (B−A) between the moving image 30 (A) and the moving image 32 (B). The extracted movie (child difference) (BA) is stored in the RGB file F3.

  Next, the moving image 30 (A) from the file F1 and the difference extraction movie (child difference) (BA) from the file F3 are taken into the converter 52, and the moving image file of the moving image 30 (A) is read by the converter. RGB / YUV conversion 52 is performed on the difference extraction movie (child difference) (BA), and the converted moving image 30 (A) is output as YUV format data D1 to an encoder 54 such as MPEG and converted. The difference extraction movie (child difference) (BA) is output to the encoder 54 as YUV data D2. Thereafter, the encoder 54 encodes the moving image 30 (A) and the difference extraction movie (child difference) (BA), stores the moving image 30 (A) in a moving image file F4 such as MPEG, and extracts the difference. The movie (child difference) (BA) is stored in a moving image file F5 such as MPEG. Differences between the moving image 30 and the moving images 34, 36, and 38 can be similarly extracted.

  In general, when the color space of the material data is RGB and the color space immediately before the encoding process is YUV, color conversion from RGB to YUV is performed before encoding.

  On the other hand, the process of converting the different color movie into the base movie and the difference movie in this embodiment can be performed before or after the RGB-YUV conversion.

  However, on an actual machine, it is necessary to perform the reverse conversion process (decoding process) for reproducing the different color movie from the base movie and the difference movie in the same color space as that used in the conversion process in the encoding. This is because real-time processing requires real-time processing with limited hardware specifications, so the color space to which this method is applied and the base movie creation method, taking into account the ease of processing on the actual device. Select.

  Next, when performing the decoding process, as shown in FIG. 5, the moving image 30 stored in the moving image file F4 such as MPEG and the difference extraction movie such as MPEG stored in the moving image file F5 such as MPEG ( B-A, C-A, DA, and EA) are taken into the decoder 56 and decoded. For example, the data (A) and the difference extraction movie (BA) relating to the moving image 30 are decoded by the decoder 56, and the data (A) and the data (BA) obtained by the decoding are respectively YUV-format data D3. , D4 to the converter 58. Thereafter, in the converter 58, YUV-RGB conversion processing is performed on the YUV format data D3 and D4, and the converted data (A) and data (BA) are converted into RGB format data D5 and D6, respectively. 60. The adder / synthesizer 60 adds and synthesizes the RGB format data (A) D5 and the data (BA) D6, and outputs the resultant data as RGB format data (B) D7.

  Similarly, the data (A) and the difference extraction movie (CA, DA, EA) relating to the moving image 30 are decoded, and the YUV format data (A) and the data (C-) obtained by decoding are decoded. A, DA, EA) are converted from YUV to RGB format, and the converted RGB format data (A) and data (CA, DA, EA) are added and synthesized. The added and combined data is output as data (C, D, E) of moving images 34, 36, 38 in RGB format.

  When converting a moving image (movie) 30 among a plurality of moving images 30 to 38 of different colors as a base file (parent) and moving images 42, 44, 46, and 48 into sub-files (difference files) as child differences Since the amount of information is reduced by performing processing before and after encoding, the compression efficiency can be increased. In particular, the compression efficiency can be further improved for a moving image including objects having the same shape but different colors.

  There are four ways to create a base file from a different color movie.

(A) Minimum Value Method A base file is created by extracting the minimum value of each color component for each pixel from a plurality of different-color movie images.

  Specifically, the minimum value is calculated independently for each frame, each pixel, and each color component. At this time, if there are a plurality (n) of different color movies, the minimum value of n samples is calculated.

(B) Average Value Method A base file is created by calculating an average value of all pixels for each color component among a plurality of different color movies.

  Specifically, the average value is calculated independently for each frame, for each pixel, and for each color component. At this time, if there are a plurality (n) of different color movies, an arithmetic average of n samples is calculated.

(C) Maximum Value Method A base file is created by extracting the maximum value of each color component for each pixel from a plurality of different color movies.

  Specifically, the maximum value is calculated independently for each frame, each pixel, and each color component. At this time, when there are a plurality (n) of different color movies, the maximum value of n samples is calculated.

(D) Selection Method One target different color movie image is selected from a plurality of different color movies and regarded as a base file.

  When creating a base file using any one of the above methods, either a method of calculating with RGB or two methods of calculating with YUV can be used.

  Next, the encoding process will be described with reference to the flowchart of FIG. First, when creating a base file, a base file is created based on a plurality of different-color movies MV0 to MVn-1 and used as a parent beam (S1).

  Next, a difference movie (child difference file) between the base file and each color difference movie is generated (S2), encoding is performed by MPEG or the like based on the generated difference movie and the base file (S3), and processing in this routine Exit.

  Next, processing during reproduction will be described with reference to the flowchart of FIG. First, based on the compressed parent compression movie and the compressed child difference movie, a frame decoding process such as MPEG is performed (S11), and the frame of the different color movie is restored from this processing result (S12), and all the frames are finished. Steps S11 and S12 are continued until all the frames have been processed (S13), and the processing in this routine is ended.

  Next, a processing method for creating a base file (parent) by the minimum value method will be described with reference to the flowchart of FIG. First, as color components for each pixel among a plurality of different-color movie images, for example, the minimum values for R, G, and B are selected (S21), and it is determined whether or not this process is completed for all pixels (S22). When the processing is finished for all the pixels, the processing in this routine is finished.

  According to this minimum value method, the calculation formula of the difference is simple, and if the addition synthesis function of the graphic chip renderer is used, the CPU is not burdened.

  Next, a processing method for creating a base file based on the average value method will be described with reference to the flowchart of FIG. After obtaining the sum of all pixels for each color component, for example, R, G, B, for a plurality of different color movie images, dividing by n indicating the number of all pixels, and averaging all pixels for each color component After the value is calculated (S31), it is determined whether or not the processing for all the pixels and the respective color components is finished (S32). When the processing is finished for all the pixels, the processing in this routine is finished.

  According to this average value method, the color value of the difference movie becomes smaller, so that the compression efficiency can be further increased.

  Next, a processing method for creating a base file by the maximum value method will be described with reference to the flowchart of FIG. First, the maximum value is selected for each of the color components R, G, and B for each pixel among a plurality of different-color movie images (S41), and it is determined whether or not this process has been completed for all pixels (S42). When the process is finished for this pixel, the process in this routine is finished.

  According to this maximum value method, the calculation formula of the difference is simple, and if the subtracting and synthesizing function of the graphic chip renderer is used, no load is applied to the CPU.

  When creating a base file using the selection method, it is possible to create a base file by using any one of the different color movies that are the materials. At this time, it is possible to arbitrarily select one movie of different colors, but if a value close to the average value movie is selected, the value of the difference movie becomes small, so that compression becomes easier. For example, a different color movie in which the square error of all frames, all pixels, and all color components is a minimum value is adopted.

  Further, according to the selection method, the total number of movies can be saved, and the compression efficiency can be increased accordingly.

  Next, a method of generating a difference movie when the average value method or the selection method is applied will be described with reference to the flowchart of FIG. A base file obtained by the average value method or the selection method is used, and for each color component, for example, R, G, B, 256 is added to the difference between the different color movie and the parent movie, and the sum is divided by 2 (S51). ). It is determined whether or not this process has been completed for all pixels (S52). When all the pixels have been completed, a child difference movie effective area cutout process is executed (S53), and the process in this routine is terminated.

  Next, a method of generating a difference movie when the minimum value method is applied will be described with reference to the flowchart of FIG. A base file obtained by the minimum value method is used, and the difference between the color difference movie and the minimum value movie is calculated for each color component, for example, R, G, B (S61). It is determined whether or not this process has been completed for all pixels (S62). When all the pixels have been completed, a child difference movie effective area cutout process is executed (S63), and the process in this routine is terminated.

  Next, a method of generating a difference movie when the maximum value method is applied will be described with reference to the flowchart of FIG. The base file obtained by the maximum value method is used, and the difference between the maximum value movie and the color difference movie is calculated for each color component, for example, R, G, B (S71). It is determined whether or not this process has been completed for all the pixels (S72). When all the pixels have been completed, the child difference movie effective area cutout process is executed (S73), and the process in this routine is terminated.

  Next, the effective area cutout process of the difference movie will be described with reference to the flowchart of FIG. After the difference movie is generated based on the difference movie generation process of FIGS. 11 to 13, the area where the difference data has a value other than 0 is examined (S81). Difference data belonging to an area where the difference area is out of the effective area is extracted, and it is determined whether or not this process has been completed for all frames (S82).

  When processing is completed for all frames, if there is an invalid area around the entire frame as the difference data, the difference data belonging to the invalid area is truncated to reduce the screen size of the movie and The relative positions x and y of the difference movie are stored (S83). The recorded information is used at the time of reproduction, and the processing in this routine is completed by reducing the movie screen size.

  Note that the boundary of the reduced screen size data may be adjusted for the convenience of the compression codec and playback program. In this case, in the case of MPEG, it can be set to 16 integer multiples.

  Also, since zero data can be compressed small, the effect of improving the compression efficiency is small, but the decoding and composition processing during reproduction has the effect of reducing the load.

Next, a method for restoring a frame of a different color movie when the average value method or the selection method is applied will be described with reference to FIG. First, when a frame of a different color movie is restored by applying an average value method or a selection method, a clip operation is performed on the parent movie + 2 × child difference movie-256 for each color component, for example, R, G, B. (S91).

  In clip (x), for example, as a saturation calculation of [0, 255], an operation of 0 is performed if 0 or less, and 255 is performed if 255 or more. This is because, depending on the compression, the calculation result may be out of range, so a saturation calculation is necessary. It is determined whether or not this process has been completed for all pixels (S92), and when this process has been completed for all pixels, the process in this routine is terminated.

  Next, a method for restoring a frame of a different color movie when the minimum value method is applied will be described with reference to FIG. First, when restoring a frame of a different color movie by applying the minimum value method, a clip operation is executed for (parent movie + child difference movie) for each color component, for example, R, G, B (S101). This process is executed for all pixels, and it is determined whether or not the process has been completed for all pixels (S102). When the process is completed for all pixels, the process in this routine is ended.

  Next, a method for restoring a frame of a different color movie when the maximum value method is applied will be described with reference to FIG. First, when restoring a frame of a different color movie by applying the maximum value method, a clip operation is executed for each of the color components, for example, R, G, and B (parent movie-child difference movie) (S111). This process is executed for all the pixels, and it is determined whether or not the process has been completed for all the pixels (S112). When all the pixels are completed, the process in this routine is ended.

  In this embodiment, a movie of 1.5 seconds is formed into 512 × 160, and as the different color movie, for example, the color of the arrow of each moving image is changed to create a total of eight different color movies. When eight files were encoded without creating a difference file, all of them were parent files (base files), so the total capacity was 2880 KB.

  On the other hand, when one is a base file (parent) and seven are difference movies (difference files), the parent file + 7 difference files = 748 KB.

  In this case, the data reduction rate is 0.78 / 2.8 = 26%, and it has been found that the data can be reduced to about 1/4 compared to the original size.

  According to the present embodiment, by converting a plurality of different color video data into a base file and a plurality of difference files, the information amount of the difference file is reduced, and each difference file in which the information amount of the base file and the information amount is reduced is reduced. The overall compression efficiency can be increased by compressing.

  4 and 5, the embodiment using MPEG as the video codec has been described. However, MPEG is an example, and it can be changed to another video codec or combined with any video codec to realize the present invention. Of course.

  Note that the image processing apparatus of the present invention is not limited to the game apparatus described in the above embodiment, but also an apparatus that displays an image based on a plurality of different color movie images, for example, a pachinko machine (first type pachinko machine, second type). It can also be applied to a game machine such as a rotary pachinko machine (slot machine), a game machine such as a ball ball, a simulation device using an image, or the like.

1 is a block diagram of an image processing apparatus showing an embodiment of the present invention. It is a figure for demonstrating the state of a moving image when encoding with a base file. It is a figure for demonstrating the state of a moving image when encoding based on one base file and four difference files. It is a functional block diagram of the encoding process which concerns on this embodiment. It is a block diagram of a decoding processing function according to the present embodiment. It is a flowchart for demonstrating the process at the time of the encoding which concerns on this embodiment. It is a flowchart for demonstrating the process at the time of the reproduction | regeneration which concerns on this embodiment. It is a flowchart for demonstrating the method of producing a base file by the minimum value method which concerns on this embodiment. It is a flowchart for demonstrating the method of producing a base file by the average value method which concerns on this embodiment. It is a flowchart for demonstrating the method of producing a base file by the maximum value method which concerns on this embodiment. It is a flowchart for demonstrating the difference movie production | generation method by the average value method or selection method which concerns on this embodiment. It is a flowchart for demonstrating the difference movie production | generation method by the minimum value method which concerns on this embodiment. It is a flowchart for demonstrating the difference movie production | generation method by the maximum value method which concerns on this embodiment. It is a flowchart for demonstrating the effective area cutout method of the difference movie which concerns on this embodiment. 5 is a flowchart for explaining a method of restoring a frame of a different color movie by an average value method or a selection method according to the present embodiment. It is a flowchart for demonstrating the restoration method of the frame of a different color movie by the minimum value method which concerns on this embodiment. It is a flowchart for demonstrating the restoration method of the frame of a different color movie by the maximum value method which concerns on this embodiment.

Explanation of symbols

  301 3D processing unit, 303 movie video processing unit, 304 audio processing unit, 305 drawing processor, 306 synchronization control unit, 400 to 430 object

Claims (16)

An image processing apparatus for processing an image,
Storage means for storing a plurality of moving images in which only colors of some areas are different;
Video processing means for extracting and processing the plurality of moving images stored in the storage means,
The video processing means includes
Based on the plurality of moving images, a base file and a difference file are generated, and the generated base file and the difference file are compressed and processed.
An image processing apparatus.   2. The video processing unit according to claim 1, wherein the base file is generated by calculating a minimum value independently for each frame, each pixel, and each color component from the plurality of moving images. The image processing apparatus described.   The video processing means generates the base file by calculating an average value independently for each frame, each pixel, and each color component from the plurality of moving images. The image processing apparatus described.   The video processing means generates the base file by calculating a maximum value independently for each frame, each pixel, and each color component from the plurality of moving images. The image processing apparatus described.   The image processing apparatus according to claim 1, wherein the video processing unit selects one target moving image from the plurality of moving images as the base file.   The video processing means uses an area where no difference information exists between the base file and the difference file as an invalid area and an area other than the effective area as an effective area from the effective area of the converted difference file. 2. The data of the effective area is used as data of the difference file, and the relative position between the data of the difference file and the data of the base file is recorded. The image processing apparatus described. An image reproduction device for reproducing a moving image processed by the image processing device according to any one of claims 1 to 6,
Storage means for storing a moving image including the base file and the difference file;
Video playback means for taking out and playing back the moving image stored in the storage means,
The image reproduction device characterized in that the video reproduction means adds and synthesizes the data of the base file and the data of the difference file. An image processing method for extracting and processing a moving image stored in a storage means,
A first step of converting a plurality of moving images having different colors only in some areas into a base file and a difference file;
A second step of compressing the converted base file and the difference file;
An image processing method characterized by comprising:   9. The base file is generated by calculating the minimum value independently for each frame, each pixel, and each color component from the plurality of moving images in the first step. The image processing method as described.   The base file is generated by calculating an average value independently for each frame, each pixel, and each color component from the plurality of moving images in the first step. The image processing method as described.   The base file is generated by calculating the maximum value independently for each frame, each pixel, and each color component from the plurality of moving images in the first step. The image processing method as described.   9. The image processing method according to claim 8, wherein, in the first step, one target moving image is selected from the plurality of moving images as the base file.   When the area where the difference information between the base file and the difference file does not exist is set as an invalid area, and the other area is set as an effective area, data outside the effective area of the converted difference file is truncated, A third step of setting the data in the effective area as the data of the difference file, and a fourth step of recording a relative position of the data of the difference file and the converted base file. The image processing method according to claim 8. An image reproduction method for extracting and reproducing a moving image processed by the image processing method according to any one of claims 8 to 13,
An image reproduction method for extracting the base file and the difference file stored in a storage unit and adding and synthesizing the base file data and the difference file data.   A program for causing a computer to execute the image processing method according to any one of claims 8 to 14.   16. A machine-readable recording medium in which the program according to claim 15 is stored.

Images