JP2003078880A - Image processor, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor, an image processing method and a program that can relax unnaturalness of a moving picture at fast forward reproduction. SOLUTION: A data extract section 13 extracts image data of a prescribed number of frames from a series of frame image data configuring a moving picture at a prescribed frame interval, a data composite section 14 composes the extracted image data comprising a prescribed number of the frames to generate composite frame image data. Then a moving picture signal generating section 15 generates a moving picture signal Sout on the basis of a series of the composite frame image data. Thus, the unnaturalness of the moving picture can be relaxed more in comparison with a conventional image processing method where frame images are skipped for fast forward reproduction so as to obtain images with a smooth motion.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device, an image processing method and a program, and more particularly to an image processing device, an image processing method and a program for generating moving image data for fast forward reproduction. .

[0002]

2. Description of the Related Art The principle of recording and reproducing moving images is based on a perceptual phenomenon called "apparent movement". "Physical movement" is a perceptual phenomenon in which still images are displayed one by one, and it looks as if moving images are moving. "Intermittent movement" Also called.

That is, a moving image is recorded by sampling an instantaneous image of a target space as a still image (hereinafter, referred to as a frame image) at regular time intervals. Then, the reproduction of the moving image is performed by temporally continuously displaying the sampled series of frame images. In the following description, the number of times a frame image is sampled per unit time will be referred to as an imaging rate [frame / sec]. In addition, a moving image as a set of frame images obtained by sampling will be referred to as a video stream, and a device that samples the frame images at an imaging rate to generate a video stream will be referred to as an imaging device.

An imaging device for continuously sampling the image of a target space as a frame image and recording "movement" is, for example, a camera used for shooting a movie, and a photographic film is exposed to continuously photograph. There is a device that shoots, and a device that electrically records a frame image, such as a video camera. The computer graphics device is also an imaging device that samples a frame image generated by a computer operation in a virtual two-dimensional or three-dimensional space.

In sampling by the imaging device,
In general, a frame image is generated by temporally dividing and acquiring “motion” information using a mechanical or electronic shutter. In this case, in the period required for sampling one frame image (one frame period), the shutter can be opened and the information of the imaging target can be acquired (hereinafter, referred to as shutter open period To).
And at least one period during which the shutter is closed and information of the imaging target cannot be acquired (hereinafter, shutter closing period Tc).

In one frame period F, the shutter open period T
When there is one o and one shutter closing period Tc, the relationship between the shutter opening period To, the shutter closing period Tc, and the imaging rate R is as follows.

[0007]

[Equation 1] To [sec] + Tc [sec] = F [sec / frame] = 1 / (R [frame / sec]) (1)

FIG. 9 is a diagram for explaining an example of the shutter open period To and the shutter close period Tc in one frame period F of the video stream. As shown in FIG. 9A, at least one shutter opening period To and at least one shutter closing period Tc are present in one frame period F. Further, as can be seen by comparing FIGS. 9B and 9C, one frame period F in FIG. 9B corresponds to 1 in FIG. 9C.
It is four times longer than the frame period F, and therefore the imaging rate is ¼ due to the relationship of the expression (1). Further, as shown in FIGS. 9D and 9E, the lengths of one frame period F are equal, but the lengths of the shutter opening period To and the shutter closing period Tc are different. The minimum lengths of the shutter open period To and the shutter close period Tc differ depending on the imaging device.

The moving image to be picked up differs depending on whether the length of the shutter opening period To is slower or faster than the moving speed of the moving image. This specific example will be described with reference to FIGS.

FIG. 10 is a diagram for explaining an object that moves with respect to the background. Reference numeral 1 is a moving object, reference numeral 2 is an image pickup device, reference numeral 3 is an area of an imaged space, and reference numeral 3 is a reference numeral. 4 indicates the background, respectively. As shown in FIGS. 10A and 10B, it is assumed that the object 1 crosses the imaging region 3. FIG. 11 is a diagram showing an example of a moving image in which the object 1 moves. In the example of FIG. 11, since the relative moving speed of the object 1 with respect to the imaging region 3 is sufficiently slow with respect to the shutter opening period To, the image of the object 1 during the shutter opening period To does not change much. Therefore, as shown in FIG. 11, the contour of the object 1 is relatively sharp.

On the other hand, if the object 1 whose moving speed is higher than that at the time of shooting of FIG. 11 is shot under the same condition as that of shooting of the video stream of FIG. 11, for example, as shown in the moving image of FIG. The outline of is blurred. This is because the image of the object 1 changes significantly during the shutter open period To.

Now, based on the above-described principle of capturing a moving image, fast-forward reproduction for reproducing n or n (where n is a natural number) forward or backward with respect to the time axis will be described.

FIG. 13 is a diagram for explaining the relationship between one frame period F of a video stream reproduced at n times fast-forward and one frame period F of a video stream displayed on a display device. FIG. 13A shows an original video stream, FIG. 13B shows a video stream to be played in n-fold fast forward playback, and FIG.
C indicates one frame period F of the frame image displayed on the display device.

The video stream (FIG. 13B) to be reproduced in the n-fold speed fast-forward reproduction is the original video in a space where the time changes at a speed n times as fast as when the original video stream (FIG. 13A) was shot. N when shooting the stream
It is equivalent to a video stream obtained when shooting is performed at a double imaging rate. That is, the number of frames reproduced per unit time (hereinafter referred to as frame rate) is ideally n times the frame rate of the original video stream.

However, since the display rate, which is the number of frames displayed per unit time on the display device, cannot be changed in general, the frame rate of the original video stream is usually reduced by some method,
Processing is being performed to match the display rate of the display device.

FIG. 14 is a diagram for explaining a conventional image data processing method in the case of performing n-fold fast-forward reproduction. In FIG. 14, the symbols m and n represent arbitrary natural numbers. In the processing method illustrated in FIG. 14, one frame image at the beginning is extracted from n frame images in the original video stream and displayed, and n-1 frame images that follow are skipped, and then n frames are again displayed. The process of extracting and displaying the frame image of the eye is repeated.
As a result, the frame image of the original video stream is n
Displayed every other sheet.

FIG. 15 is a diagram showing a shutter open period To and a shutter close period Tc in one frame of image data processed by the method shown in FIG. FIG. 15A shows the original video stream and FIG. 15B shows
FIG. 15C shows a video stream to be played in n-fold fast-forward playback, and FIG. 15C shows a video stream processed by the method shown in FIG.

The video stream (FIG. 15C) generated by the processing method shown in FIG. 14 has a space in which time changes at a speed n times as fast as when the original video stream (FIG. 15A) was shot. It is equivalent to a video stream obtained when the image capturing rate is the same as when the stream was captured and the shutter release period To is 1 / n. FIG. 16 is a diagram showing an example of a moving image processed by the method shown in FIG.

As another conventional image processing method for performing fast-forward reproduction, for example, there is a method of extracting a part of a frame image following the frame image being displayed and overwriting it. This is DV (Digital Vide
This is the method used in the VTR device of the o) method.

[0020]

By the way, in the conventional image processing method shown in FIG. 14, the information of the skipped n-1 frame images is discarded and is not reflected in the image at the time of fast-forward reproduction. Further, even in the DV method in which a part of the frame image is extracted and overwritten, the information of the image that has not been extracted is discarded, and similarly, it is not reflected in the image during fast-forward reproduction. As described above, since the moving image reproduced by the conventional image processing method for performing the fast-forward reproduction lacks much information with respect to the original moving image, the motion lacks smoothness and is awkward. There is a problem with unnatural images.

The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus, an image processing method, and a program capable of alleviating the unnaturalness of moving images during fast-forward reproduction, as compared with the prior art. Especially.

[0022]

In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention uses image data of a series of frames forming a moving image at predetermined frame intervals. Data extracting means for extracting image data of a predetermined number of frames, data synthesizing means for synthesizing the extracted image data of a predetermined number of frames, and generating a moving image signal based on a series of the synthetic image data. And a moving image signal generating means.

According to the image processing apparatus of the first aspect of the present invention, in the data extraction means, the image of a predetermined number of frames at predetermined frame intervals is extracted from the image data of a series of frames forming a moving image. Data is extracted.
The image data of the extracted predetermined number of frames are combined by the data combining means. In the moving image signal generation means, a moving image signal is generated based on the series of the combined image data.

The data synthesizing means may average the image data of the same pixel in the image data of the extracted predetermined number of frames to generate the synthetic image data.

Further, the difference between the image data of a predetermined frame and the image data of another frame in the predetermined number of frames is detected for each predetermined pixel block, and the image data changes according to the detection result. Pixel block specifying means for specifying a pixel block is provided, and the data synthesizing means includes the pixel data of the pixel block specified by the pixel block element specifying means among the image data of the other frame, and the predetermined frame. It may be combined with the image data of.

The image processing method according to the second aspect of the present invention comprises a data extraction step of extracting a predetermined number of frame image data at predetermined frame intervals from a series of frame image data forming a moving image. A data synthesizing step of synthesizing the image data of the extracted predetermined number of frames, and a moving image signal generating step of generating a moving image signal based on a series of the synthetic image data.

In the data synthesizing step,
Image data of the same pixel in the extracted image data of a predetermined number of frames may be averaged to generate the composite image data.

Further, the difference between the image data of a predetermined frame and the image data of another frame in the predetermined number of frames is detected for each predetermined pixel block, and the image data changes according to the detection result. A pixel block specifying step of specifying a pixel block; and the data combining step, in the image data of the other frame, the pixel data of the pixel block specified in the pixel block element specifying step, and the predetermined frame. It may be combined with the image data of.

A program according to the third aspect of the present invention is
A program of a processing device for processing image data,
A data extraction step of extracting image data of a predetermined number of frames at a predetermined frame interval from a series of image data of a moving image, and a data combination for combining the image data of the extracted predetermined number of frames. And a moving image signal generation step of generating a moving image signal based on a series of the above-mentioned combined image data.

In the data synthesizing step,
Image data of the same pixel in the extracted image data of a predetermined number of frames may be averaged to generate the composite image data. In addition, the difference between the image data of a predetermined frame and the image data of another frame in the predetermined number of frames is detected for each predetermined pixel block, and a pixel block whose image data changes according to the detection result is detected. A pixel block specifying step for specifying, and the data combining step, in the image data of the other frame, the pixel data of the pixel block specified in the pixel block element specifying step and the image data of the predetermined frame. And may be combined.

[0031]

BEST MODE FOR CARRYING OUT THE INVENTION <First Embodiment> First, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a schematic block diagram showing a configuration example of an image processing apparatus according to the first embodiment of the present invention. In FIG. 1, reference numeral 11 is a frame extraction unit, reference numeral 12 is a frame image holding unit, reference numeral 13 is a data extraction unit, reference numeral 1
Reference numeral 4 denotes a data synthesizing unit, and reference numeral 15 denotes a moving image signal generating unit.

The frame extraction unit 11 extracts image data of a series of frame images (hereinafter, referred to as frame image data) from the input moving image data Sin, and holds the frame image holding unit 12 in the frame image holding unit 12. Let As the input moving image data Sin, for example, AVI (Audio
Video Interleaving) format and MPEG2 (Motion Pictu)
The frame extraction unit 11 can be configured for moving image data of various formats such as the re image cording Experts Group 2) format. For example, in the case of moving image data compressed in the MPEG2 format, a device such as an MPEG2 video decoder can be applied. Further, the frames extracted from the moving image data Sin may be extracted from all the frames included in the moving image data Sin, or some of them may be thinned out. The frame extraction unit 11
Can be replaced with a computer graphics device or the like that directly generates the frame image data.

The frame image holding unit 12 holds the frame image data extracted by the frame extracting unit 11 and the combined frame image data combined by the data combining unit 14. Also, the frame image data read in response to the read request is output to the data extraction unit 13 or the moving image signal generation unit 15. Frame image holding unit 1
As the device 2, any device capable of holding image data, such as a hard disk device, a semiconductor memory, or a magneto-optical disk, can be applied.

The data extracting unit 13 is a frame extracting unit 11
A predetermined number of frame image data are extracted at a predetermined frame interval from the series of frame image data extracted in. For example, when performing fast-forward reproduction at n times speed, frame image data for p frames (p is a natural number satisfying p <n) is extracted at intervals of n frames.

The data synthesizing unit 14 synthesizes a predetermined number of frame image data extracted by the data extracting unit 13 and causes the frame image holding unit 12 to hold the combined frame image data. As a method of synthesizing the image data, for example, a method of averaging the image data of the same pixel in the image data of the predetermined number of frames extracted by the data extracting unit 13 can be used. For example, for a color image,
Arbitrary position (x, y) in the combined frame image
The R component of the pixel data of is set to the average value of the R components of the pixel data of the same position (x, y) in a predetermined number of frame images that are the composition source. The G component of the pixel data at the arbitrary position (x, y) in the combined frame image is set to the average value of the G components of the pixel data at the same position (x, y) in the predetermined number of frame images to be combined. To be done. Arbitrary position (x,
The B component of the pixel data of y) is set to the average value of the B component of the pixel data of the same position (x, y) in a predetermined number of frame images that are the composition source.

The moving image signal generating section 15 generates a moving image signal Sout based on the series of combined frame image data held in the frame image holding section 12. The moving image signal to be generated may be an analog signal such as a composite video signal, or the moving image data S
Similar to in, moving image data of various formats such as AVI format and MPEG2 format may be used.

The operation of the image processing apparatus of FIG. 1 having the above configuration will be described. FIG. 2 is a flow chart for explaining the operation of the image processing apparatus shown in FIG. Step ST1: The frame extraction unit 11 extracts frame image data from the moving image data Sin. The frame image data may be extracted from all the frames included in the moving image data Sin or may be extracted by thinning out some frames.

Step ST2: The data extraction unit 13 extracts a predetermined number of frame image data at predetermined frame intervals from the series of frame image data from the frame extraction unit 11 held in the frame image holding unit 12. . The frame from which the image data is extracted may be a predetermined number of continuous frames within a predetermined frame interval, or may be a frame appropriately dispersed within this frame interval. Further, the data may be extracted from all the pixels of the frame image data at once,
A part thereof may be sequentially extracted.

Step ST3: The predetermined number of frame image data extracted in step 2 are combined. For example, when the pixel data of the same pixel in a predetermined number of frame image data is sequentially extracted by the data extracting unit 13, these pixel data are combined and averaged, and the pixel data of the same pixel of newly generated combined frame image data is generated. As a result, the frame image holding unit 12 sequentially holds the frame image. By performing this processing for all the pixels in the frame image, one composite frame image data is generated,
It is held in the frame image holding unit 12.

FIG. 3 is a diagram for explaining extraction of image data in step ST2 and composition of image data in step ST3. In FIG. 3, reference sign m
Represents an arbitrary natural number, and the symbols n and p represent natural numbers satisfying 1 ≦ p ≦ n. In the case of n-fold fast-forward reproduction, as shown in FIG. 3, for example, from frame m to frame (m + n
Of the n frame image data up to -1), p frame image data from frame m to frame (m + p-1) is extracted by the data extraction unit 13, and the remaining frame (m + p) to frame (m + n-). N up to 1)
-P frames are skipped. Then, the extracted frame image data is combined by the data combining unit 13 to generate new combined frame image data.

FIG. 4 is a diagram for explaining a shutter open period and a shutter closed period in one frame period of the combined frame image data. FIG. 4A shows a video stream of an original moving image, FIG. 4B shows a video stream to be reproduced in n-fold speed fast-forward reproduction, and FIG. 4C shows the image data extraction processing and the synthesis processing shown in FIG. The generated video stream is shown.

The video stream (FIG. 4C) generated by the processing shown in FIG. 3 is the original video stream (FIG. 4).
Shooting in a space where time changes at a speed n times faster than when shooting A), with the same shooting rate as when shooting the original video stream, and opening the shutter with a shutter release period To of 1 / n p times p times It is equivalent to the video stream obtained when Therefore, the image information discarded in the conventional processing shown in FIG. 15 is reflected in the composite frame image data in the processing of the present invention shown in FIG.

Step ST4: The moving image signal generation section 15 generates a moving image signal from the series of combined frame image data held in the frame image holding section 12.

The data extraction process of step 2 and the data combination process of step ST3 may be performed after the frame extraction of the moving image as the combination source is completed in the frame extraction process of step 1, or at least the step It may be performed after the extraction of the frames necessary for the synthesis in the data synthesis process of 3 is completed, or may be performed sequentially with the frame extraction process of step 1. In addition, the moving image signal generation processing in step ST4
It may be performed after all the composite frame image data is generated and held in the frame image holding unit 12, or may be performed after a part thereof is held, or new composite frame image data is generated. You can go there every time.

FIG. 5 is a diagram showing an example of a moving image combined in the image processing apparatus shown in FIG. As can be seen by comparing with the moving image generated by the conventional image processing for fast-forward reproduction shown in FIG. 16, the moving image shown in FIG. 5 has a blurred outline of an image of a moving object. 12
As shown in, the image is close to a moving image when an object moving at high speed is imaged. That is, an image similar to that when the shutter open period To is lengthened is obtained by combining the image data. This image is a result of the fact that the information about the motion of the missing object is given in the moving image shown in FIG. 16, and the unnaturalness of the movement is alleviated as compared with the moving image shown in FIG.

As described above, according to the image processing apparatus shown in FIG. 1, in the data extraction unit 13, a series of frame image data forming a moving image is used to extract images of a predetermined number of frames at predetermined frame intervals. Data is extracted,
The data combining unit 14 combines the extracted predetermined number of frame image data to generate combined frame image data, and the moving image signal generation unit 15 generates the moving image signal Sout based on the series of combined frame image data. Is generated. Therefore, compared to the conventional image processing for fast-forward reproduction, the unnaturalness of a moving image can be alleviated and an image with smooth motion can be obtained.

Further, by causing the processing of generating the moving image signal Sout in the image processing apparatus shown in FIG. 1 to be performed in real time with respect to the input of the moving image data Sin, for example, a DVD (digital versatile disc) or VTR (video ta).
It is possible to obtain a smooth-moving image with less loss of image information than before when performing fast-forward playback or rewind playback in a moving image playback device such as a pe recorder).

Further, in the computer graphics device, for example, in a state in which the image pickup rate is intentionally increased, FIG.
By generating a video stream via the image processing device shown in FIG. 1, a more natural image can be generated that expresses the “flow of images” of an object that moves at high speed. In addition, even in a moving image editing apparatus that does not require real-time processing, it is possible to create a high-speed reproduced image with smoother movement than in the past.

<Second Embodiment> Next, a second embodiment of the present invention will be described with reference to FIGS. 6 and 7. The second embodiment relates to an image processing apparatus capable of reducing the amount of calculation required for image processing as compared with the first embodiment. FIG. 6 is a schematic block diagram showing a configuration example of an image processing apparatus according to the second embodiment of the present invention. In FIG. 6, reference numeral 16 indicates a pixel block specifying unit, and the same reference numerals in FIG. 1 and FIG. 6 indicate the same constituent elements.

The pixel block specifying unit 16 determines the difference between the predetermined frame image data and the other frame image data among the predetermined number of frame image data extracted by the data extracting unit 13 at predetermined frame intervals. The detection is performed for each predetermined pixel block, and the pixel block in which the image data changes is specified according to the detection result.
However, the pixel block includes one or more pixels.

For example, the difference between frame m and frame (m + 1) to frame (m + n-1) in FIG.
The detection is performed for each k pixel block (j and k represent natural numbers). In this case, for example, the difference between specific pixel data in the j × k pixel block may be detected, or the data subjected to orthogonal transformation such as discrete cosine transform in the j × k pixel block may be identified. You may detect the difference regarding the component of. Whether or not the detected difference value exceeds a predetermined threshold value is determined in all pixel blocks, and the pixel block in which the image data has changed is specified according to the determination result.

The data synthesizing means 14 'includes the pixel block specifying unit 16 of the above-mentioned other frame image data.
The pixel data of the pixel block specified in 3 and the predetermined frame image data described above are combined.

FIG. 7 is a flow chart for explaining the operation of the image data processing apparatus shown in FIG. 6, and the same reference numerals as those in FIG. 2 indicate the steps for performing the same processing. Below, FIG.
Steps for performing processing different from that will be described.

Step ST5: Among the predetermined number of frame image data extracted in step ST2, the difference between the predetermined frame image data and other frame image data is detected for each predetermined pixel block. A pixel block in which the image data has changed is specified according to the detection result.

Step ST3 ': The pixel data in the pixel block specified in step ST5 and the pixel data in the same pixel block of the predetermined frame image data are combined and averaged. That is, only the pixel blocks that have changed with respect to the predetermined frame image data are combined and averaged, and the pixel blocks that have not changed are not combined and averaged.

As described above, according to the image processing apparatus shown in FIG. 6, the unnaturalness of the moving image can be alleviated as compared with the conventional one as in the image processing apparatus shown in FIG. Further, since only the pixel blocks in which the change in the image data is detected are combined and averaged, the calculation amount required for the processing can be reduced as compared with the image processing apparatus shown in FIG.

<Third Embodiment> Next, a third embodiment of the present invention will be described with reference to FIG. In the image processing apparatus shown in FIG. 1 and FIG. 6 described above, all the constituent elements can be configured by hardware, but as will be described below, a part or all of the configuration is changed according to the program. It can also be realized by a processor that executes processing.

FIG. 8 is a schematic block diagram showing a configuration example of an image processing apparatus according to the third embodiment of the present invention. In FIG. 8, reference numeral 17 is a CPU (central processing unit).
nit), reference numeral 18 is a RAM (random access memory)
Reference numerals 19 and 21 denote I / O units, and reference numeral 2
Reference numeral 0 represents a storage device, reference numeral 22 represents a display device, and reference numeral 23 represents a bus.

The CPU 17 processes the moving image data Sin stored in the storage device 20 in the steps shown in the flowchart of FIG. 2 or FIG. 7 according to the program stored in the storage device 20. The moving image data Sout equivalent to that generated in the image processing apparatus shown.
Is generated and stored again in the storage device. The generated moving image data Sout is supplied to the display device 22 via the I / O unit 21 to display a moving image. RAM18 is C
It provides a storage area for the program of the PU 17 and a storage area temporarily used in the process of the CPU 17.

The I / O unit 19 reads the data stored at the designated address of the storage device 20 under the control of the CPU 17 and outputs the data to the bus 23.
The data input from is stored in the designated address of the storage device 20. Under the control of the I / O unit 19, the storage device 20 reads out the data stored at the designated address and outputs the data to the I / O unit 19, and also the I / O unit 19.
The data input from the unit 19 is stored in the designated address. The program of the CPU 17 including the steps shown in the flowchart of FIG. 2 or FIG. 7, the original moving image data Sin, the moving image data Sout, and other data are stored.

The I / O unit 21 supplies the image data input from the bus 23 to the display device 22 under the control of the CPU 17. The display device 22 displays an image according to the image data input from the I / O unit 21.

According to the image processing apparatus of FIG. 8 having the above-mentioned configuration, the program stored at the predetermined address of the storage device 20 is read by the I / O unit 19 and the bus 2
3 is stored in the program storage area of the RAM 18. By executing the processing shown in the flowchart of FIG. 2 or FIG. 7 by the program stored in the RAM 18, the moving image data Sin stored at a predetermined address of the storage device 20 is processed and the moving image data So is stored.
ut is generated and stored in the storage device 20. The moving image data Sout is read from the storage device 20 by another image reproducing program and supplied to the display device 22 to display the fast-forward reproduced image of the original moving image data Sin.

As described above, the image processing apparatus shown in FIG. 8 can also realize the same function as that of the image processing apparatus shown in FIG. 1 or 6. Therefore, compared to the conventional image processing for fast-forward reproduction, the unnaturalness of a moving image can be alleviated and an image with smooth motion can be obtained.

Note that the above-described first to third embodiments are merely examples for explaining the present invention, and the present invention is not limited even if various modifications obvious to those skilled in the art are made to these embodiments. It is feasible. For example, the frame image holding unit 12 shown in FIGS. 1 and 6 holds the extracted frame image data and the composite frame image data, but these may be held in different frame image holding units. Also,
The processing shown in the flowchart of FIG. 2 or FIG. 7 can also be realized by the server device and the terminal device connected via the network.

[0065]

As described above, according to the present invention, it is possible to reduce the unnaturalness of a moving image during fast-forward reproduction and obtain a smooth moving image as compared with the conventional technique.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing a configuration example of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of the image processing apparatus shown in FIG.

FIG. 3 is a diagram for explaining extraction and combination of image data.

FIG. 4 is a diagram for explaining a shutter opening period and a shutter closing period in one frame period of combined frame image data.

5 is a diagram showing an example of a moving image combined in the image processing apparatus shown in FIG.

FIG. 6 is a schematic block diagram showing a configuration example of an image processing apparatus according to a second embodiment of the present invention.

FIG. 7 is a flowchart for explaining the operation of the image data processing device shown in FIG.

FIG. 8 is a schematic block diagram showing a configuration example of an image processing apparatus according to a third embodiment of the present invention.

[Fig. 9] Fig. 9 is a diagram for describing an example of a shutter opening period and a shutter closing period in one frame period of a video stream.

FIG. 10 is a diagram for explaining an object that moves with respect to the background.

11 is a diagram showing an example of a moving image in which the object shown in FIG. 10 moves.

12 is a diagram showing an example of a moving image when the object shown in FIG. 10 moves at high speed.

[Fig. 13] Fig. 13 is a diagram for describing the relationship between one frame period of a video stream reproduced at n times fast-forwarding and one frame period of a video stream displayed on a display device.

FIG. 14 is a diagram for explaining a conventional image data processing method in the case of performing n-fold fast-forward reproduction.

15 is a diagram showing a shutter opening period and a shutter closing period in one frame of image data processed by the method shown in FIG.

16 is a diagram showing an example of a moving image processed by the method shown in FIG.

[Explanation of symbols]

11 ... frame extracting unit, 12 ... frame image holding unit, 1
3 ... Data extracting unit, 14 ... Data synthesizing unit, 15 ... Moving image signal generating unit, 16 ... Pixel block specifying unit, 17 ... CP
U, 18 ... RAM, 19, 21 ... I / O section, 20 ... Storage device, 22 ... Display device.

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C018 MA01                 5C052 AA01 AB02 AC05 CC06 CC11                       DD04                 5C053 FA14 FA23 GA11 GB04 GB06                       GB38 KA05 LA01 LA06

Claims (12)

[Claims] 1. A data extracting means for extracting a predetermined number of frames of image data at predetermined frame intervals from a series of frames of image data forming a moving image, and the extracted predetermined number of frames of image data. An image processing apparatus comprising: a data synthesizing unit for synthesizing a moving image signal; and a moving image signal generating unit for generating a moving image signal based on a series of the above-mentioned synthesized image data. 2. The image processing apparatus according to claim 1, wherein the data synthesizing unit averages image data of the same pixel in the image data of the extracted predetermined number of frames to generate the synthetic image data. . 3. A difference between image data of a predetermined frame and image data of another frame in the predetermined number of frames is detected for each predetermined pixel block, and the image data changes according to the detection result. A pixel block specifying unit for specifying a pixel block; and the data combining unit, in the image data of the other frame, the pixel data of the pixel block specified by the pixel block element specifying unit, and the predetermined frame. The image processing apparatus according to claim 1, wherein the image processing apparatus and the image data of (1) are combined. 4. The image processing apparatus according to claim 1, further comprising frame extraction means for extracting image data of the series of frames from input moving image data. 5. A data extracting step of extracting image data of a predetermined number of frames at predetermined frame intervals from image data of a series of frames forming a moving image, and image data of the extracted predetermined number of frames. An image processing method comprising: a data synthesizing step for synthesizing a moving image signal; and a moving image signal generating step for generating a moving image signal based on a series of the synthesized image data. 6. The image processing method according to claim 5, wherein in the data synthesizing step, the image data of the same pixel in the image data of the extracted predetermined number of frames is averaged to generate the synthetic image data. . 7. A difference between image data of a predetermined frame and image data of another frame in the predetermined number of frames is detected for each predetermined pixel block, and the image data changes according to the detection result. A pixel block specifying step of specifying a pixel block; and the data combining step, in the image data of the other frame, the pixel data of the pixel block specified in the pixel block element specifying step, and the predetermined frame. The image processing method according to claim 5, wherein the image data is combined with the image data. 8. The image processing method according to claim 5, further comprising a frame extraction step of extracting image data of the series of frames from moving image data. 9. A program of a processing device for processing image data, comprising a data extracting step of extracting image data of a predetermined number of frames at predetermined frame intervals from image data of a series of frames forming a moving image. A program having a data synthesizing step of synthesizing the image data of the extracted predetermined number of frames, and a moving image signal generating step of generating a moving image signal based on a series of the synthetic image data. 10. The program according to claim 9, wherein in the data synthesizing step, the image data of the same pixel in the image data of the extracted predetermined number of frames is averaged to generate the synthetic image data. 11. A difference between image data of a predetermined frame and image data of another frame in the predetermined number of frames is detected for each predetermined pixel block, and the image data changes according to the detection result. A pixel block specifying step of specifying a pixel block; and the data combining step, in the image data of the other frame, the pixel data of the pixel block specified in the pixel block element specifying step, and the predetermined frame. The program according to claim 9, which is combined with the image data of. 12. The program according to claim 9, further comprising a frame extraction step of extracting image data of the series of frames from the input moving image data.