JP2006235689A - Image processor, image display device, image pickup device, image processing method, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that in the case that the size of a correction image cut out from a frame image is always fixed, even a frame image does not need to be corrected is also cut out. <P>SOLUTION: Images are acquired one by one by an image acquisition part 110 per a certain volume of images in a time series order. A motion vector of the certain volume of focused images included in the images which are sequentially acquired is calculated by using the certain volume of images which were acquired prior to the certain volume of focused images by one in the time series order by a motion vector calculation part 120. For the certain volume of focused images, a correction position vector for representing a position to cut out the predetermined image is calculated by a correction position calculation part 130 which is equipped with a correction limit value calculation part for calculating a correction limit value equivalent to a cutting size of the predetermined image. The correction limit value changes corresponding to the size of the correction position vector. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing technique for acquiring a frame image from a moving image or a time-series continuous still image, cutting out a predetermined image from the acquired frame image, and generating a corrected frame image.

  In recent years, a desired image such as an athletic meet has been taken as a moving image by using a video camera, a digital camera, and a photographing device such as a mobile phone having an image photographing function. At that time, when the photographing apparatus is photographed by hand, image blurring occurs in the photographed image due to camera shake. For this reason, conventionally, an image processing technique for correcting image blur at the time of shooting has been proposed (for example, Patent Document 1).

  Patent Document 1 discloses an image processing technique that corrects camera shake by calculating a motion vector of a captured field image and shifting the field image by an integral vector calculated from the calculated motion vector.

Japanese Patent No. 2957851

In the image processing technique described in Patent Document 1, when each frame image is shifted in a direction to cancel the motion of the video due to camera shake, the frame image is trimmed and output according to the maximum shift amount so that the video is not lost. . However, the image processing technique described in Patent Document 1 has a problem that the size of the corrected image cut out from the frame image is always constant, and the frame image is cut out even for an image that does not need to be corrected.

The present invention has been made to solve such a problem, and an image processing device, an image display device, an image capturing device, an image processing method, a program, and a recording in which the size of a corrected image cut out from a frame image is variable. The purpose is to provide a medium.

  An image processing apparatus according to an aspect of the present invention that achieves the above object acquires a fixed amount of an image from a moving image or a time-series continuous still image, and cuts out a predetermined image from the acquired fixed amount of image. An image processing apparatus that generates a corrected image, and for each fixed amount of images in time-series order, an image acquisition unit that sequentially acquires the fixed amount of images, and of the fixed amount of images acquired sequentially, A motion vector calculation unit that calculates a motion vector of a certain amount of image using a certain amount of image immediately before the fixed amount of image in time series order, and the predetermined amount of the image A correction position calculation unit that calculates a correction position vector indicating a position to cut out the image of the image, and the correction position calculation unit calculates a correction limit value corresponding to a size for cutting out the predetermined image. Part further comprising a configured such that the correction limit value is changed according to the magnitude of the corrected position vector.

  According to the image processing apparatus configured as described above, the correction limit value corresponding to the size for cutting out a predetermined image changes according to the size of the correction position vector corresponding to a certain amount of image shift. Therefore, by continuously changing the cutout size of the frame image according to the size of the correction position vector, normal display (state of Smax (n) = 0) and correction display (0 <Smax (n) ≦ Smax_max) Can be switched gradually.

  Here, the “fixed amount of image” corresponds to the “frame image” in the embodiment, but is not limited to the frame image.

  The “corrected image” corresponds to the “corrected frame image” in the embodiment, and the “image acquisition unit” corresponds to the “frame image acquisition unit 110” in the embodiment.

  According to the image processing device of the present invention, the correction limit value calculation unit reduces the size of the predetermined image when the size of the predetermined image is larger than the size of the correction position vector, When the size of the predetermined image cut out is smaller than the correction position vector, it is preferable to increase the size of the predetermined image cut out.

According to the image processing apparatus of the present invention, the correction limit value calculation unit calculates the correction limit value Smax (n).
Smax (n) = Smax (n−1) + α × {Smax (n−1) −Smax (n−2)} − β × {Smax (n−1) −S (n−1)}
Smax (n-1): correction limit value for a fixed amount image in the time series (n-1) th order Smax (n-2): correction for a fixed amount image in the time series order (n-2) Limit value
S (n-1): correction position vector α for a fixed amount of images in time-series order (n−1): α: positive correction coefficient β: 1 or less positive calculation: positive correction coefficient 0.1 or less It is preferable to use the formula.

  According to the image processing apparatus of the present invention, α is preferably a constant of 0.9 ≦ α <1.

According to the image processing apparatus of the present invention,
β is a constant of 0 <β ≦ 0.1, or the larger the value of {Smax (n−1) −S (n−1)}, the smaller the value of β, or {Smax (n−1) −Smax (n -2)} decreases the value of β as the value of
It is preferable to do so.

  The image processing apparatus according to the present invention further includes a correction image generation unit that generates a correction image for the acquired fixed amount of image based on the calculated correction position vector, and the correction position calculation unit Calculates the size of the correction image and the enlargement ratio of the correction image based on the correction limit value Smax (n), and the correction image generator generates the correction image based on the calculated size of the correction image and the enlargement ratio of the correction image. It is preferable that the predetermined image is cut out to generate a corrected image.

  Here, the “correction image generation unit” corresponds to the “correction frame image generation unit 140” in the embodiment.

  An image display device according to another aspect of the present invention acquires a fixed amount of an image from a moving image or a time-series continuous still image, and generates a corrected image by cutting out a predetermined image from the acquired fixed amount of image. An image acquisition unit that sequentially acquires the fixed amount of images for each fixed amount of images in time series, and a fixed amount of interest among the sequentially acquired fixed amount of images A motion vector calculation unit that calculates a motion vector of the image using a certain amount of image immediately before the fixed amount of image in time series order, and the predetermined amount of image A correction position calculation unit that calculates a correction position vector indicating a position to cut out an image, and the correction position calculation unit calculates a correction limit value corresponding to a size at which the predetermined image is cut out. The The provided configured such that the correction limit value is changed according to the magnitude of the corrected position vector.

  An image pickup apparatus according to still another aspect of the present invention acquires a fixed amount of an image from a moving image or a still image that is continuous in time series, and cuts out a predetermined image from the acquired fixed amount of image to obtain a corrected image. An image capturing device to be generated, for each fixed amount of images in time series, an image acquisition unit that sequentially acquires the fixed amount of images, and a fixed amount of image of interest among the sequentially acquired fixed amount of images A motion vector calculation unit that calculates a certain amount of motion vector using a certain amount of the image immediately before the fixed amount of time image in chronological order, and the predetermined image for the fixed amount of image of interest A correction position calculation unit that calculates a correction position vector indicating a position to cut out, and the correction position calculation unit further includes a correction limit value calculation unit that calculates a correction limit value corresponding to a size at which the predetermined image is cut out. Provided, configured such that the correction limit value is changed according to the magnitude of the corrected position vector.

  An image processing method according to still another aspect of the present invention acquires a fixed amount of an image from a moving image or a still image that is continuous in time series, and cuts out a predetermined image from the acquired fixed amount of image to obtain a corrected image. An image processing method to be generated, an image acquisition step of sequentially acquiring the fixed amount of images for each fixed amount of images in time series, and a fixed amount of images of interest among the sequentially acquired fixed amount of images A motion vector calculation step of calculating a motion vector of a predetermined amount using a certain amount of image immediately before the fixed amount of image in time series order, and the predetermined image for the fixed amount of image of interest A correction position calculation step of calculating a correction position vector indicating a position to cut out, wherein the correction position calculation step calculates a correction limit value corresponding to a size at which the predetermined image is cut out. Extent further comprising a configured the correction limit value is changed according to the magnitude of the corrected position vector.

  A program according to still another aspect of the present invention acquires a fixed amount of an image from a moving image or a time-series continuous still image, and generates a corrected image by cutting out a predetermined image from the acquired fixed amount of image. A program for causing a computer to execute a process, and for each of a fixed amount of images in time series, an image acquisition process for sequentially acquiring the fixed amount of images, and attention is paid to the fixed amount of images acquired sequentially. A motion vector calculation process for calculating a motion vector of a fixed amount of image using a fixed amount of image immediately before the fixed amount of image in time series order, and for the fixed amount of image, A correction position calculation process for calculating a correction position vector indicating a position to cut out the predetermined image, wherein the correction position calculation process corresponds to a size corresponding to the size at which the predetermined image is cut out. Further comprising a correction limit value calculation processing for calculating a limit value, configured such that the correction limit value is changed according to the magnitude of the corrected position vector.

A recording medium according to still another aspect of the present invention acquires a fixed amount of an image from a moving image or a time-series continuous still image, and generates a corrected image by cutting out a predetermined image from the acquired fixed amount of image. A computer-readable recording medium storing a program for causing a computer to execute processing, and for each fixed amount of images in time series order, an image acquisition process for sequentially acquiring the fixed amount of images, and the sequential A motion vector calculation process for calculating a motion vector of a certain amount of image of interest in a certain amount of acquired images, using a certain amount of image immediately before the certain amount of image in time series order; A correction position calculation process for calculating a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest, and the correction position calculation process A correction limit value calculation process for calculating a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. It is a readable recording medium.
Configured as follows.

  Next, an embodiment of the present invention will be described. FIG. 1 is an explanatory diagram showing a schematic configuration of an image processing apparatus as an embodiment of the present invention. This image processing apparatus is a general-purpose computer, and includes an input interface (I / F) 101, a CPU 103, a RAM 102, a ROM 104, a hard disk 105, and an output interface (I / F) 106, which are connected to each other via a bus 107. Has been. The input I / F 101 is connected to a digital video camera 200 (image capturing device) and a DVD player 210 as devices that input moving images, and the output I / F 106 is a device (image display device) that outputs moving images. Video projector 300 and display 310 are connected. In addition, if necessary, a drive device that can read data from a storage medium that stores a moving image and an image display device that can output a moving image can be connected. Note that moving images include time-sequential still images.

  In addition, the image capturing apparatus and the image display apparatus may each include an image processing apparatus according to the present invention.

  The input I / F 101 acquires a frame image from a moving image having a predetermined number of pixels and a predetermined pixel value such as an RGB format, and converts it into frame image data that can be processed by the computer 100. In the present embodiment, it is assumed that frame image data is acquired from a predetermined number of field images constituting the frame image. The frame image is usually composed of a plurality of predetermined number of field images according to the recording method. For example, in the case of a moving image recorded by the NTSC system, it is an interlace system, and 60 frame images per second constitute 30 frame images per second. Therefore, the input I / F 101 acquires one frame image using the pixel values of the two field images. Of course, in the case of a progressive method (non-interlace method) in which one field image constitutes one frame image, one frame image is acquired using one field image.

  The converted frame image data is stored in the RAM 102 or the hard disk 105. The CPU 103 executes predetermined image processing on the stored frame image data, and stores it again in the RAM 102 or the hard disk 105 as corrected frame image data. In the case of predetermined image processing performed until the corrected frame image data is generated, the RAM 102 and the hard disk 105 are used as a working memory for the image processing data as necessary. Then, the stored corrected frame image data is converted into predetermined image data via the output I / F 106 and sent to the video projector 300 or the like.

  The application program in which the predetermined image processing is recorded may be stored in advance in the hard disk 105 or the ROM 104. For example, the application program may be supplied from the outside by a computer-readable recording medium such as a CD-ROM, and the DVD- It may be stored by being stored in the hard disk 105 via the R / RW drive. Of course, the data may be stored in the hard disk 105 by accessing a server or the like that supplies an application program via network means such as the Internet and downloading data.

  The CPU 103 functions as an image processing apparatus by reading an application program stored in the hard disk 105 or the ROM 104 via the bus 107 and executing the read application program under a predetermined operating system. In particular, as shown in FIG. 1, by executing this application program, the CPU 103 causes the frame image acquisition unit 110, the motion vector calculation unit 120, the correction position calculation unit 130, the correction frame image generation unit 140, the correction motion vector to be corrected. The calculation unit 150 and the image output unit 160 function.

  Each part manages the following processes. The frame image acquisition unit 110 acquires frame images from a moving image input to the computer 100 in chronological order. The motion vector calculation unit 120 calculates a movement vector of the acquired frame image and passes it to the correction position calculation unit 130. The correction position calculation unit 130 calculates an image position for cutting out a predetermined image from the frame image as a correction position vector using a motion vector or the like. The correction frame image generation unit 140 generates a correction frame image based on the calculated correction position vector and the like. The corrected motion vector calculation unit 150 calculates a motion vector for the generated corrected frame image as a corrected motion vector, and passes it to the correction position calculation unit. The image output unit 160 converts the generated corrected frame image into predetermined image data and outputs it.

  Next, processing performed by each unit in the image processing apparatus according to the present embodiment will be described with reference to the processing flowchart of FIG. When the process shown in FIG. 2 is started, first, in step S10, the frame image acquisition unit 110 performs a process of taking the frame image F (n) into the computer 100 in order of time series from the input moving image. In the following description of the image processing apparatus in the present embodiment, (n) means nth in time series order.

  In this embodiment, it is assumed that frame images are automatically captured sequentially in time-series order with the input of a moving image as a trigger. Of course, the frame image to be captured may be designated. As a method for specifying a frame image, an identification number of a frame image that is normally added may be specified when a frame image is acquired from a moving image and imported into the computer 100. Alternatively, the user may directly designate an image while viewing a frame image displayed on a display (not shown) such as a monitor provided in the video camera.

  Next, in step S <b> 20, the frame image acquisition unit 110 determines whether or not there is a frame image F (n) acquired by being captured by the computer 100. If the frame image F (n) acquired from the moving image exists (YES), the process proceeds to the next step. If not (NO), the process in the image processing apparatus is terminated.

  Next, in step S30, the motion vector calculation unit 120 performs processing for calculating the motion vector V0 (n) of the acquired frame image F (n). Here, calculation processing of the motion vector V0 (n) of the frame image F (n) will be described. The relative position of the frame image F (n) is calculated with respect to the n−1th frame image F (n−1) in time series order, and the frame image F (n) is arranged at the relative position. Then, the shift amount of the screen position of the frame image F (n) with respect to the screen position of the frame image F (n−1) is calculated as the number of pixels in the X and Y directions. A motion vector V0 (n) of the frame image F (n) is represented by a vector having the number of pixels in the X direction as an X component and the number of pixels in the Y direction as a Y component. Data V0 (n) of the calculated motion vector V0 (n) is stored in the RAM 102 or the like by the CPU 103.

  The relative position between the frame images can be obtained by a method disclosed in Patent Document 1 or a known image processing method such as image pattern matching or feature point tracking. Since the specific relative position calculation processing method is not the essence of the present invention, the description thereof will be omitted.

  Here, in order to facilitate understanding of the description of the processing contents of the calculation processing of the corrected position vector S (n) in the next step S40 and the calculation processing of the corrected motion vector V1 (n) in the following step S60, The relationship between the motion vector V0 (n) of the frame image F (n), the corrected position vector S (n), and the corrected motion vector V1 (n) will be described in advance with reference to FIGS.

  FIG. 3 shows the first frame image F (1), the second frame image F (2), and F (3) acquired in the time series order first. The present invention provides an image processing technique for generating a corrected frame image by a correction process for cutting out a predetermined image from a frame image based on a correction amount, and each frame image cut out when the correction amount is 0 Frame images FK (1), FK (2), and FK (3) indicated by broken lines in FIG.

  Here, as shown in FIG. 3, the frame image FK (n) has a screen size smaller than the frame image F (n) by a variable correction limit value Smax (n). Specifically, the image has a small screen size by a correction limit value Sxmax (n) that is variable in the left and right directions in the X direction and by a correction limit value Symax (n) that is variable in the Y direction. That is, the cut image has a screen size that is 2 × Sxmax (n) smaller in the X direction and 2 × Symax (n) smaller in the Y direction.

  The calculation of the variable correction limit value Smax (n) (X component is Sxmax (n) and Y component is Symax (n)) will be described in detail in the process of step S42 below.

  Similarly, FIG. 3 (lower side) shows frame images FK (1), FK (2), and FK (3) arranged according to the obtained relative positions. As apparent from FIG. 3, for example, the motion vector V0 (2) of the frame image F (2) is equal to the motion vector of the frame image FK (2). That is, the motion vector V0 (2) of the frame image F (2) is the center of the frame image FK (1) (in this embodiment, the “center” is the diagonal line of the rectangle constituting the frame image or the correction frame image). It is obtained as the position of the center of the frame image FK (2) with respect to the intersection. Similarly, the motion vector V0 (3) of the frame image FK (3) is obtained as the position of the center of the frame image FK (3) with respect to the center of the frame image FK (2). The X components of the motion vectors V0 (2) and V0 (3) are represented by the number of pixels in the X direction, and the Y component is represented by the number of pixels in the Y direction.

  In FIG. 3, since the frame image F (1) has no previous frame image in chronological order, the frame image F (1) is treated as not moving in this embodiment, and the frame image FK (1 The motion vector V0 (1) in FIG.

  Next, the corrected position vector S (n) and the motion vector V0 (n) of the frame image F (n) and the corrected motion vector V1 (n) calculated in step S60 (FIG. 2) are used with reference to FIG. Explain the relationship. FIG. 4 is an explanatory diagram showing an enlargement of the vicinity of the center of the frame images FK (1) to FK (3) shown in FIG. 3 (lower part). Here, the corrected frame image HFG (2) indicates a state in which the frame image FK (2) is corrected by the correction position vector S (2) of the frame image F (2) calculated in step S40 (FIG. 2). ing.

  As apparent from FIG. 4, the frame image FK (2) is obtained by moving the position of the center of the frame image FK (1) according to the number of pixels of the X and Y components of the motion vector V0 (2). The frame image HFG (2) is obtained by moving the center position of the frame image FK (2) according to the number of pixels of the X component and the Y component of the correction position vector S (2). Similarly, the frame image FK (3) is obtained by moving the center position of the frame image FK (2) according to the number of pixels of the X and Y components of the motion vector V0 (3), and the corrected frame image HFG. (3) shows the position of the center of the frame image FK (3) moved according to the number of pixels of the X component and Y component of the correction position vector S (3).

  Also, as shown in FIG. 4, the corrected motion vector V1 (2) indicates the relative position of the center of the corrected frame image HFG (2) with respect to the center of the frame image FK (1). V1 (3) indicates a relative position of the center of the correction frame image HFG (3) with respect to the center of the correction frame image HFG (2). Since the motion vector V0 (1) is 0, the correction position vector S (1) is also 0 (described later). Therefore, the frame image FK (1) is the same as the corrected frame image HFG (1).

  Here, in the motion vector V0 (n) and the corrected motion vector V1 (n), the X component is positive in the drawing right direction and the Y component is positive in the drawing upward direction, but the corrected position vector S (n) is moving. Contrary to the motion vector, the X component is positive in the left direction of the drawing, and the Y component is positive in the downward direction of the drawing. Accordingly, the correction position vectors S (2) and S (3) of the frame images FK (2) and FK (3) are opposite to the motion vectors V0 (2) and V0 (3) as shown in FIG. It goes in the positive direction.

  According to the present embodiment, as described with reference to FIG. 4, the corrected position vector S (n), the corrected position vector S (n−1), the motion vector V 0 (n), and the corrected motion vector V 1 (n -1) and an image processing technique obtained from the three vectors to suppress the correction delay, and suppress the irregular screen movement by suppressing the correction amount within the variable correction limit value Smax (n). it can.

  Next, in step S40 (FIG. 2), the correction position calculation unit 130 performs a calculation process of the correction position vector S (n). This process will be described in detail with reference to the flowchart of FIG.

  When the process of step S40 is started, first, in step S410, a process of inputting the motion vector V0 (n) of the focused frame image F (n) is performed. Here, the CPU 103 reads out the motion vector V (n) obtained in the previous processing step S30 and stored in the memory such as the RAM 102, thereby performing input processing.

  Next, in step S420, a process of inputting the corrected motion vector V1 (n-1) of the previous frame image in time series order with respect to the frame image F (n) of interest is performed. The CPU 103 reads out the corrected motion vector V1 (n−1) calculated in step S60 (FIG. 2) and stored in a memory such as the RAM 102, thereby performing input processing.

  In step S430, a process of inputting the correction position vector S (n-1) of the previous frame image in time series order with respect to the frame image F (n) of interest is performed. Specifically, the CPU 103 reads the corrected position vector S (n−1) calculated in step S40 (FIG. 2) and stored in a memory such as the RAM 102, thereby performing input processing.

  In step S420 and step S430, in the case of “n = 1”, that is, the first frame image in time series order, the frame image immediately preceding the target frame image in time series order (hereinafter simply referred to as “previous frame image”). ) Does not exist. In this case, as described above, the frame image F (n) is not moved in the present embodiment, and as a result, the corrected motion vector V1 (1) is 0 and the corrected position vector S (1) is also 0. .

  Next, in step S440, attenuation coefficient K and D values are determined for each of the X and Y components. The attenuation coefficients K and D are coefficients used in a predetermined calculation formula used in the next processing step S490, and are coefficients that determine the size of the correction position vector, that is, the correction amount. The attenuation coefficients K and D are both decimal numbers of 1 or less.

In this embodiment, the values of the attenuation coefficients K and D are determined using at least one of the following three examples.
“First Embodiment”: The amount of movement of an image calculated for a corrected frame image obtained by correcting the previous frame image (hereinafter simply referred to as “previous correction frame image”) and the amount of movement of the frame image of interest The values of the damping coefficients K and D are determined. Further, when the movement amount of the frame image of interest is equal to or greater than the movement amount of the previous correction frame image, the attenuation coefficient K is based on the direction of the motion vector of the frame image of interest and the direction of the correction motion vector of the previous correction frame image. , A method for determining the value of D.
“Second Embodiment”: Attenuation coefficient K based on the ratio of the correction amount of the previous frame image to the correction limit value indicating the correctable range and the determination of whether the correction amount of the previous frame image is “increase” or “decrease”. , A method for determining the value of D.
“Third embodiment”: The motion vector of the frame image of interest and the corrected motion vector of the previous frame image with respect to the difference amount between the correction limit value that is the correctable range and the correction amount of the previous frame image A method of determining the values of the attenuation coefficients K and D depending on how much the difference amount is.
“First Example of Decay Factor Determination Method”
First, regarding the processing performed in step S440 (FIG. 5), details of the determination processing of the attenuation coefficients K and D in the first embodiment will be described using the processing flowchart shown in FIG. The method of determining the values of the attenuation coefficients K and D in the first embodiment corrects from the magnitude and direction of the motion vector of the frame image of interest and the size and orientation of the corrected frame image one time earlier in the chronological order. The amount is calculated, and the correction amount of the focused frame image is calculated in accordance with the direction change and the magnitude of the motion vector. Here, the processing for the X component of the frame image of interest will be described. However, in processing step S440, the same processing as the processing for the X component shown in FIG. Determine the values of the damping coefficients K, D for. Then, in the next processing step S490 (FIG. 5), which will be described later, when the X component and Y component of the correction position vector are calculated by a predetermined arithmetic expression, the values of the determined attenuation coefficients K and D are used.

  When the process is started, an extraction process of the X component V0x (n) of the motion vector V0 (n) of the frame image of interest is first performed in step S441. The X component V0x (n) is indicated by the number of pixels in the X direction. As shown in FIG. 4, the X component V0x (n) is positive if the vector direction is the right direction of the drawing and negative if the vector direction is the left direction. Note that the Y component V0y (n) of the motion vector V0 (n) of the frame image of interest is positive if the vector direction is upward in the drawing as shown in FIG. 4, and negative if downward.

  Next, in step S442, an X component V1x (n-1) of the corrected motion vector V1 (n-1) of the previous frame image in time series order is extracted from the frame image of interest. Of course, as described above, when the frame image of interest is the first frame image in time series order, V1x (n−1) is zero.

  Next, in step S443, the absolute value of V0x (n) and the absolute value of V1x (n-1) are calculated. Then, in the next processing step S444, it is determined whether or not the absolute value of V0x (n) is greater than or equal to the absolute value of V1x (n-1). If NO in step S444, that is, if the moving amount of the focused frame image is smaller than the moving amount of the previous correction frame image, the process proceeds to step S445, where the values of the attenuation coefficients K and D are determined as K1 and D1, and step S490 is performed. Go to (Fig. 5).

  On the other hand, if “YES” in the step S444, that is, if the moving amount of the focused frame image is equal to or larger than the moving amount of the previous correction frame image, the process proceeds to a step S446, and the product BSx of V0x (n) and V1x (n−1) is set. calculate. Then, in the next processing step S447, it is determined whether BSx is 0 or more. That is, here, it is checked whether the vector V0x (n) and the vector V1x (n-1) are in the same direction (0 or more) or in the opposite direction (less than 0).

  If NO in step S447 (that is, in the reverse direction), the process proceeds to step S448, where the values of the attenuation coefficients K and D are determined as K2 and D2, and the process proceeds to step S490 (FIG. 5). Here, the value of K2 is larger than the value of K1, and the value of D2 is larger than the value of D1.

  On the other hand, if YES in step S447 (that is, in the same direction), the process proceeds to step S449, where the values of the attenuation coefficients K and D are determined as K3 and D3, and the process proceeds to step S490 (FIG. 5). Here, the value of K3 is larger than the value of K2, and the value of D3 is larger than the value of D2.

  An example of the values of the attenuation coefficients K and D in the first embodiment is shown in the attenuation coefficient table GKT1 in FIG. Here, the value of the attenuation coefficient K is desirably a value satisfying K1 <K2 <K3 in a range of 0.9 ≦ K ≦ 1. Further, the value of the attenuation coefficient D is desirably a value satisfying D1 <D2 <D3 in a range of 0.5 ≦ D ≦ 1. When the movement amount of the frame image to be corrected is smaller than the movement amount of the previous correction frame image, the movement amount of the frame image to be corrected is reduced with respect to the movement amount of the previous frame image. Therefore, if the correction amount of the frame image to be corrected is the same correction amount as that of the previous frame image, the probability that the correction amount reaches the correction limit value increases. Therefore, a small amount of attenuation coefficient is used to reduce the correction amount and make it difficult to reach the correction limit value.

Further, when the movement amount of the frame image to be corrected is equal to or larger than the movement amount of the previous correction frame image, when the movement amounts of these two frame images are in different directions, the screen position of the frame image to be corrected is This means that the image is reversed and moved in the direction approaching the screen position of the previous correction frame image. Therefore, the amount of correction of the frame image to be corrected is reduced by reducing the value of the attenuation coefficient compared to the case of the same direction. The values of the attenuation coefficients K1 to K3 and D1 to D3 are not limited to the present embodiment, and any values may be used as long as the above-described conditions are satisfied.
“Second Embodiment of Damping Coefficient Determination Method”
Next, details of the process of determining the attenuation coefficients K and D in the second embodiment will be described with reference to the process flowchart shown in FIG. In the method of determining the values of the attenuation coefficients K and D in the second embodiment, the correction amount is calculated based on a predetermined ratio RS as to whether or not it is close to the correction limit value, and further, the correction amount is increased by a predetermined arithmetic expression. It is intended to calculate the correction amount of the frame image to be noticed by determining whether it is reduced or not. Here, the processing for the X component of the frame image of interest is also described. However, in processing step S440, the same processing as the processing for the X component shown in FIG. Determine the values of the damping coefficients K, D for. Then, in the next processing step S490 (FIG. 5) to be described later, when the X component and the Y component of the correction position vector are calculated by a predetermined arithmetic expression, the values of the determined attenuation coefficients K and D are used, respectively. is there.

  When the processing here is started, first, input processing of the correction limit value Sxmax (n−1) of the previous frame image is performed in step S451. Specifically, the CPU 103 reads the value of the X component of Smax (n−1) calculated in step S42 and stored in a memory such as the RAM 102.

  In step S451 and step S453, in the case of “n = 1”, that is, the first frame image in time series order, the previous frame image does not exist. In this case, the correction limit value Sxmax (n−1) is set to 0.

  Next, in step S452, the X component Sx (n−1) of the corrected position vector S (n−1) of the previous frame image in time series order with respect to the frame image F (n) of interest is extracted. Process. Here, as described above, the X component Sx (n−1) is represented by the number of pixels with the left direction on the screen as the positive direction. Of course, the Y component Sy (n−1) is represented by the number of pixels with the screen lower direction being positive.

  Next, in step S453, the value of the ratio RS obtained by dividing the absolute value of Sx (n-1) by Sxmax (n-1) is calculated. The larger the RS value, the closer the correction amount is to the correction limit value.

  In step S454, the X component V0x (n) of the motion vector V0 (n) of the frame image F (n) of interest is extracted. The X component V0x (n) is indicated by the number of pixels in the X direction. As shown in FIG. 4, the X component V0x (n) is positive if the vector direction is the right direction of the drawing and negative if the vector direction is the left direction. As for the Y component V0y (n) of the motion vector V0 (n) of the frame image F (n) of interest, as shown in FIG. Negative.

  Next, in step S455, extraction of the X component V1x (n-1) of the corrected motion vector V1 (n-1) of the previous frame image in time series order with respect to the frame image F (n) of interest. Process. As described above, V1x (n−1) is 0 when the frame image F (n) of interest is the first frame image in time series order.

  Using each vector extracted in steps S452, S454, and S455, in the next step S456, a process of calculating a trend value KHx from “Expression 1” is performed.

KHx = Sx (n-1) * (V0x (n) -V1x (n-1)) ... "Expression 1"
Next, in step S457, it is determined whether or not the tendency value KHx is 0 or more. If YES, the correction is determined to be “increase” in step S459, and if NO, the process of determining “decrease” is performed in step S458. The difference between Sx (n−1), that is, the correction direction based on the correction position vector of the previous frame image, and V0x (n) −V1x (n−1), that is, the motion vector of the target frame image and the correction vector of the previous frame image. Compared with the moving direction of the image based on it, it is determined that the direction is increased when the directions are opposite, and the direction is decreased when the directions are the same.

  Next, processing for determining the values of the attenuation coefficients K and D using the attenuation coefficient table GKT2 is performed in step S460 from the calculated ratio RS and the determination information of “increase” or “decrease”. An example of the attenuation coefficient table GKT2 is shown in FIG. The attenuation coefficient table is stored in the hard disk 105 or the like. From the data of the ratio RS and the information of “increase” or “decrease”, the CPU 103 reads the value of the corresponding address from the attenuation coefficient table GKT2, and stores the attenuation coefficients K and D. Determine the value.

  As shown in FIG. 9, in the second embodiment, in the attenuation coefficient table GKT2, the value of the ratio RS is rounded off to the second decimal place to a value of 0.1. Then, the values of the attenuation coefficients K and D were set so as to become smaller values than the previous values as the ratio RS became larger. Furthermore, the value corresponding to the “increase” information is set to a value smaller than the value corresponding to the “decrease” information. Even when the ratio RS is the same, if the increase is determined, the correction direction and the moving direction of the image are opposite to each other. , D is reduced to reduce the correction amount. Of course, the value of the ratio RS may be a finer step size, or conversely, a coarser step size. Furthermore, the step width is not the same and can be changed.

The attenuation coefficient table GKT2 shown in FIG. 9 shows an example of specific numerical values for the attenuation coefficients K and D. The value of the attenuation coefficient K is in the range of 0.9 ≦ K ≦ 1, and the attenuation coefficient The value of D may be changed in accordance with the value of RS in the range of 0.5 ≦ D ≦ 1. At this time, it is preferable that the values of the attenuation coefficients K and D are set to be smaller than the previous values as the value of RS becomes larger. When the RS value is large, the correction amount is closer to the correction limit value than when the RS value is small. Therefore, the attenuation coefficient is reduced to reduce the correction amount.
“Third embodiment of damping coefficient determination method”
Next, details of the determination processing of the attenuation coefficients K and D in the third embodiment will be described using the processing flowchart shown in FIG. The method of determining the values of the attenuation coefficients K and D in the third embodiment is that when calculating the correction amount of the frame image of interest, when correcting with the correction amount of the previous correction frame image in time series order, The number of corrections up to the correction limit value is examined to calculate the correction amount of the frame image to be noted. Here, the processing for the X component of the frame image of interest is also described. However, in processing step S440, the same processing as the processing for the X component shown in FIG. Determine the values of the damping coefficients K, D for. Then, in the next processing step S490 (FIG. 5) to be described later, when the X component and the Y component of the correction position vector are calculated by a predetermined arithmetic expression, the values of the determined attenuation coefficients K and D are used, respectively. is there.

  When the process is started, the correction limit value Sxmax (n−1) is first input in step S461. Specifically, the CPU 103 reads the value of the X component of Smax (n−1) calculated in step S42 and stored in a memory such as the RAM 102.

  In step S461 and step S466, in the case of “n = 1”, that is, the first frame image in time series order, the previous frame image does not exist. In this case, the correction limit value Sxmax (n−1) is set to 0.

  In step S462, the X component V0x (n) of the motion vector V0 (n) of the frame image F (n) of interest is extracted. The X component V0x (n) is indicated by the number of pixels in the X direction. As shown in FIG. 4, the X component V0x (n) is positive if the vector direction is the right direction of the drawing and negative if the vector direction is the left direction. As for the Y component V0y (n) of the motion vector V0 (n) of the frame image F (n) of interest, as shown in FIG. Negative.

  Next, in step S463, extraction processing of the X component V1x (n-1) of the corrected motion vector V1 (n-1) of the previous frame image in time series order with respect to the frame image of interest is performed. Of course, as described above, when the frame image of interest is the first frame image in time series order, V1x (n−1) is zero.

  Next, in step S464, the X component Sx (n-1) of the corrected position vector S (n-1) of the previous frame image in time series order with respect to the frame image F (n) of interest is extracted. Process. Here, as described above, the X component Sx (n−1) is represented by the number of pixels with the left direction on the screen as the positive direction. Of course, the Y component Sy (n−1) is represented by the number of pixels with the screen lower direction being positive.

Next, in step S465, the vector difference value BST is calculated from “Expression 2”.
BST = | V0x (n) −V1x (n−1) |
Next, in step S466, the correction remaining amount value SZT is calculated from “Expression 3”.
SZT = Sxmax (n−1) − | Sx (n−1) |
Next, in step S467, the correction limit frame number T is calculated from “Expression 4”.
T = SZT ÷ BST “Equation 4”
From the value of the correction limit frame number T calculated in step S467, the values of the attenuation coefficients K and D are determined using the attenuation coefficient table GKT3 in step S468. An example of the attenuation coefficient table GKT3 is shown in FIG. The attenuation coefficient table GKT3 is stored in the hard disk 105 or the like, and from the data of the correction limit frame number T, the CPU 103 reads the value of the corresponding address from the attenuation coefficient table GKT3 and determines the values of the attenuation coefficients K and D.

  In the attenuation coefficient table GKT3 shown in FIG. 11, the correction limit frame number T is a value obtained by rounding the first decimal place to an integer value. The attenuation coefficient table GKT3 shows an example of specific numerical values. The value of the attenuation coefficient K is in the range of 0.9 ≦ K ≦ 1, and the value of the attenuation coefficient D is 0.5 ≦ D ≦. In the range of 1, it may be changed according to the value of the correction limit frame number T. At this time, it is preferable to set the attenuation coefficients K and D to be larger as the value of T becomes larger. When the value of T is large, there is a margin in the amount of correction until the correction limit value is reached compared to when the value of T is small, so the attenuation coefficient is increased and the amount of correction of the frame image is increased. .

  As described above, the processing for the X component described in the first to third embodiments is similarly performed for the Y component, the values of the attenuation coefficients K and D are determined for each of the X and Y components, and the process proceeds to processing step S490. move on.

  When the values of the attenuation coefficients K and D are determined for the X and Y components according to the first to third embodiments, the process returns to step S490 (FIG. 5), and the frame image F ( The process of calculating the correction position vector S (n) of n) is performed. In the present embodiment, the following “Expression 5” is used as the predetermined arithmetic expression.

S (n) = K * S (n-1) + D * (V0 (n) -V1 (n-1)) ... "Formula 5"
Specifically, the correction position vector S (n) of the focused frame image F (n) is obtained by using the “Expression 5” to calculate the correction position vectors Sx (n) and Sy (n) for the X component and the Y component. It is obtained by calculating each.

  As apparent from “Expression 5”, the motion vector V0 (n) of the frame image F (n) of interest and the corrected motion of the corrected frame image HFG (n−1) obtained by correcting the frame image F (n−1). Since a difference from the vector V1 (n−1) is obtained and a value obtained by multiplying the difference by the attenuation coefficient D is used for calculation of the corrected position vector S (n), the corrected frame image F (n) and the previous frame image The probability that the movement of the image is smooth between the correction frame images HFG (n−1) increases.

  Also, according to the calculation method of the correction position vector according to “Expression 5”, when the frame image to be corrected is an image shot at a constant screen movement speed such as panning or tilting, V0 (n) is not a variable. Since it is a constant, the calculated S (n) tends to be a constant value. In other words, the correction of a frame image having a continuous movement in a certain direction such as panning and tilting also has an effect that the followability to the movement of the original image taken is good.

In “Expression 5”, when the frame image of interest is the first in time-series order (n = 1), the frame image F (1) is not moved in the present embodiment as described above. 1) = 0. By the way,
When n = 2, S (2) = K × 0 + D × (V0 (2) −0) = D × V0 (2)
In the case of n = 3, S (3) = K × S (2) + D × (V0 (3) −V1 (2))
Thus, a corrected position vector is calculated.

  Thus, the correction position vector calculation processing routine (FIG. 2, step S40) is completed, and then the process proceeds to step S42 (FIG. 2). In step S42, a correction limit value Smax (n) is calculated. The correction limit value Smax (n) is a value corresponding to the cut width (size) of the nth frame image.

At this time, the correction position calculation unit 130 corrects the correction position vector S (n−1) of the (n−1) th frame in time series and the correction limit value Smax of the (n−1) th frame in time series. (N-1) From the correction limit value Smax (n-2) of the (n-2) th frame in time series order, using the following "Expression 6", the correction limit value of the nth frame in time series order Smax (n) is calculated and stored in the RAM 102.
Smax (n) = Smax (n−1) + α × {Smax (n−1) −Smax (n−2)} − β × {Smax (n−1) −S (n−1)}
... "Formula 6"
Here, in the case of “n = 1” (first frame image in time series order), Smax (n−1) and Smax (n−2) are set to 0, and “n = 2” (second in time series order). In the case of a frame image), Smax (n−2) is set to 0.
"Correction coefficient α"
The correction coefficient α is a positive constant of 1 or less, and has an effect of smoothly changing Smax (n) without suddenly changing it. Empirically, the value of α is preferably 0.9 ≦ α <1.
"Correction coefficient β"
When β is a constant Correction coefficient β is a positive constant of 0.1 or less. If the cropping width is larger than the shift amount of the correction frame, the cropping width is reduced. If the cropping width is smaller than the shift amount, the cropping width is increased. There is an effect to.

In this manner, Smax (n) corresponding to the cutout width (size) of the frame image can be configured to change according to the magnitude of the correction position vector S (n) corresponding to the shift amount of the frame image.
When β is a variable, the value of the correction coefficient β is {Smax (n−1) −S (n−1)} or {Smax (n−1) −Smax within the range of 0 <β ≦ 0.1. You may change according to the value of (n-2)}. Specifically, within the range of 0 <β ≦ 0.1, the cut width is corrected by decreasing the value of the correction coefficient β as the value of {Smax (n−1) −S (n−1)} increases. The probability of becoming smaller than the frame shift amount can be suppressed. Alternatively, within the range of 0 <β ≦ 0.1, Smax (n) is likely to increase by decreasing the value of the correction coefficient β as the value of {Smax (n−1) −Smax (n−2)} increases. It is difficult to decrease, and the probability that the cut width becomes smaller than the shift amount of the correction frame can be suppressed.

  Even in this case, Smax (n) corresponding to the cut-out width (size) of the frame image can be configured to change according to the magnitude of the correction position vector S (n) corresponding to the shift amount of the frame image. .

  Further, a maximum value Smax_max of Smax (n) is determined so that the correction limit value Smax (n) does not become infinitely large, and when the calculation result Smax (n) of “Expression 6” is larger than Smax_max, It is corrected to Smax (n) = Smax_max.

  FIG. 13 shows the relationship between the trajectory of the frame position before and after correction and the correctable range determined by the correction limit value Smax (n). In FIG. 13, the vertical axis indicates the screen position of each frame image in the X direction or Y direction, and the horizontal axis indicates the number of frame images representing the time-series order of the acquired frame images. Incidentally, the left end of the horizontal axis indicates the frame image F (1). The thin solid line indicates the trajectory of the frame position of the frame image before correction, and the thick solid line indicates the trajectory of the frame position of the corrected frame image HFG (n) after correction. A thin dotted line indicates a range that can be corrected (correction limit value Smax (n)).

  As shown in FIG. 13, Smax (n) corresponding to the cut-out width (size) of the nth frame image is normally a constant value regardless of time, but according to the image processing of the present invention, the frame image Changes adaptively according to the magnitude of the correction position vector S (n) corresponding to the shift amount.

  As shown in FIG. 13, when the time point of the arrow A where the frame position before the correction is rapidly changed is compared with the time point of the arrow B where the amount of change in the frame position before the correction is relatively small, the time point of the arrow A From the time point to the point of arrow B, the correction position vector S (n) corresponding to the shift amount of the frame image decreases, and Smax (n) corresponding to the cut-out width (size) of the frame image gradually increases accordingly. You can see that it gets smaller. That is, in FIG. 13, Smax (n) corresponding to the cutout width (size) of the frame image is adaptively changed according to the magnitude of the correction position vector S (n) corresponding to the shift amount of the frame image. I understand.

  This completes the correction limit value Smax (n) calculation process (FIG. 2, step S42), and then proceeds to step S44 (FIG. 2). In step S <b> 44, the correction position calculation unit 130 performs calculation processing for the size and enlargement ratio of the correction frame image.

  In step 44, the correction position calculation unit 130 calculates the width (size) of the correction frame image HFG (n) from the correction limit value Smax (n) obtained in S42 using the following “Expression 7” and “Expression 8”. And calculate the magnification.

(Width of HFG (n)) = (Width of frame image) −2 × Smax (n) ... “Expression 7”
(Magnification ratio (%) with respect to HFG (n)) = (width of frame image) / (width of HFG (n)) × 100 (Equation 8)
This completes the process of calculating the size (width) and magnification of the corrected frame image (FIG. 2, step S44), and then proceeds to step S50 (FIG. 2). In step S50, the correction frame image generation unit 140 performs a generation process of the correction frame image HFG (n).

  In step S50, the correction frame image generation unit 140 uses the correction frame image generation unit to calculate the center of the frame image by the correction position vector S (n) obtained in step S40 with the size obtained in “expression 7” in step S44. The image shifted from is cut out and output as a corrected frame image HFG (n). However, when the correction position vector S (n) is larger than the correction limit value Smax (n) obtained in step S42, the correction position vector S (n) is shifted by Smax (n).

  FIG. 12 shows an example of a corrected frame image HFG (3) generated by cutting out a predetermined image from the frame image F (3) when n = 3 and having the size calculated by “Expression 7” in step S44. It was. On the upper side of FIG. 12, a frame image FK (3) that is a predetermined image when correction is not performed, that is, when the correction amount is 0, is indicated by a broken line. Then, on the lower side of FIG. 12, in accordance with the correction position vector S (3) calculated in step S40, correction is performed by correcting the screen position by Sx (3) in the X direction and Sy (3) in the Y direction. A frame image HFG (3) is shown. Thus, by correcting the screen position, the amount of movement of the image with respect to the frame image can be suppressed.

  Next, in step S60, the corrected motion vector calculation unit 150 performs a process of calculating a corrected motion vector V1 (n) for the generated corrected frame image HFG (n). As described above, the corrected motion vector calculated here is used in the next step S40 in order to calculate the corrected motion vector of the next frame image in time series order. Of course, the corrected motion vector is calculated as a vector of the X component V1x (n) and the Y component V1y (n).

  Next, in step 65, the correction frame image generation unit 140 enlarges the correction frame image HFG (n) according to the enlargement ratio obtained in “Expression 8” of S44. This process does not necessarily have to be performed within the image processing apparatus. In that case, in step S70, the corrected frame image HFG (n) and its enlargement ratio are output to the image display device, and the correction frame image HFG (n) is enlarged by the image display device.

  Next, in step S70, the corrected frame image HFG (n) is output. In the present embodiment, the generated corrected frame image HFG (n) is converted into an image signal required by an image display device such as the video projector 300 by the output I / F 106 (FIG. 1) to generate a corrected frame image. Every time it is output immediately. By doing this, the time from acquisition of the frame image to be corrected to the output of the corrected frame image is shortened, and there is a possibility that correction can be performed in real time. Of course, the output destination of the correction frame image HFG (n) may be a predetermined image file provided in the RAM 102 or the hard disk 105, and a series of a plurality of generated correction frame images may be stored.

  Thus, the process for the frame image F (n) is completed, and n is updated in step S80. Then, the process returns to step S10, and the subsequent processing is repeated with the next frame image as a new frame image F (n) in chronological order, and the corrected frame images are output for all the acquired frame images.

  According to the image processing apparatus of the embodiment of the present invention, Smax (n) corresponding to the cut-out width (size) of the frame image is the magnitude of the correction position vector S (n) corresponding to the shift amount of the frame image. It will change adaptively according to. In this way, by continuously changing the cropping size of the frame image, the normal display (state of Smax (n) = 0) and the correction display (state of 0 <Smax (n) ≦ Smax_max) are gradually switched. be able to.

  As mentioned above, although one embodiment of the present invention has been described, the present invention is not limited to such an embodiment, and can of course be implemented in various forms without departing from the spirit of the present invention. is there.

  (Modification 1) For example, in the above-described embodiment, the relative positional relationship between the frame image F (n) and the frame image F (n−1) is determined based on the movement vector of the frame image F (n) acquired from the moving image. If the screen position data exists as metadata of the frame image by means of detecting the amount of movement, such as a speed sensor or acceleration sensor, when moving images are taken, the screen position data is used to move A vector may be calculated. In this way, it is not necessary to perform image processing such as image pattern matching or feature point tracking, and the processing load relating to motion vector calculation can be reduced.

  (Modification 2) In the embodiment, the screen size of the generated correction frame image is 2 × Sxmax (n) in the horizontal direction and 2 × Symax (in the vertical direction with respect to the screen size of the acquired frame image. Although the image is smaller by n), in this modification, an image that has been enlarged to the same screen size as the acquired frame image may be used in the generation process of the corrected frame image. In this way, the moving image based on the corrected frame image has the same screen size as the original moving image before correction, and the viewer can enjoy the moving image having the same screen size.

  The screen is enlarged by making the number of pixels of the corrected frame image the same as the number of pixels of the frame image. An increase in the number of pixels constituting an image can be realized by pixel interpolation processing. In addition to the bi-linear method, the interpolation process may use a known method such as the bi-cubic method or the nearest neighbor method.

  (Modification 3) In addition, the image processing apparatus in the above embodiment is configured by a general-purpose computer. However, the present invention is not limited to this, and is configured by a mobile computer or a workstation. May be. Or, for digital cameras, video cameras, DVD recorders / players, video projectors, TVs, mobile phones, and other devices that can record or play moving images, and various devices that can record or play still images in chronological order, The image processing apparatus of the present invention may be incorporated.

  (Modification 4) Further, the image processing apparatus according to the above embodiment acquires a frame image from all the predetermined number of field images constituting the frame image, and corrects a position where the predetermined image is cut out from the acquired frame image. However, the present invention is not limited to this, and when the frame image is composed of N (N is an integer of 2 or more) field images, 1 to N−1 of these N field images. Of these, an image obtained from an arbitrary number of field images may be used as a frame image, and a position where a predetermined image is cut out may be corrected.

1 is an explanatory diagram showing a schematic configuration of an image processing apparatus as an embodiment of the present invention. It is a flowchart explaining the process of an image processing apparatus. It is explanatory drawing for demonstrating the motion vector of a frame image. It is explanatory drawing for demonstrating the relationship between the motion vector of a frame image, a correction | amendment motion vector, and a correction | amendment position vector. It is a flowchart explaining the calculation process of a correction position vector. It is a flowchart explaining the determination process of the attenuation coefficient in a 1st Example. It is an attenuation coefficient table for demonstrating the value of the attenuation coefficient in a 1st Example. It is a flowchart explaining the determination process of the attenuation coefficient in a 2nd Example. It is an attenuation coefficient table for demonstrating the value of the attenuation coefficient in a 2nd Example. It is a flowchart explaining the determination process of the attenuation coefficient in a 3rd Example. It is an attenuation coefficient table for demonstrating the value of the attenuation coefficient in a 3rd Example. It is explanatory drawing explaining the method to produce | generate a correction | amendment frame image from a frame image. It is a figure which shows the relationship between the locus | trajectory of the frame position before and behind correction | amendment, and the correctable range determined by correction | amendment limit value Smax (n).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Computer, 101 ... Input I / F, 102 ... RAM, 103 ... CPU, 104 ... ROM, 105 ... Hard disk, 106 ... Output I / F, 107 ... Bus line, 110 ... Frame image acquisition part, 120 ... Motion vector Calculation unit 130 ... correction position calculation unit 140 ... correction frame image generation unit 150 ... correction motion vector calculation unit 160 ... image output unit 200 ... video camera 210 ... DVD player 300 ... projector 310 ... display F (n) ... frame image, FK (n) ... frame image, HFG (1) ... corrected frame image, Smax (n) ... correction limit value, Sxmax (n) ... X component of correction limit value, Symax (n) ... Y component of correction limit value, V0 (n) ... motion vector, V1 (n) ... correction motion vector, S (n) ... correction position base Torr, Sx (n) ... X component of the corrected position vector, Sy (n) ... Y component of the corrected position vector, K ... damping coefficient, D ... attenuation coefficient, GKT1~3 ... damping coefficient table.

Claims (13)

An image processing apparatus that acquires a fixed amount of an image from a moving image or a time-series continuous still image, and generates a corrected image by cutting out a predetermined image for the acquired fixed amount of image,
For each fixed amount of images in time series, an image acquisition unit that sequentially acquires the fixed amount of images,
A motion vector calculation that calculates a motion vector of a certain amount of image of interest among the certain amount of images acquired sequentially using a certain amount of image immediately before the certain amount of image in time series. And
A correction position calculation unit that calculates a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest;
With
The correction position calculation unit further includes a correction limit value calculation unit that calculates a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. Image processing device. The image processing apparatus according to claim 1,
The correction limit value calculating unit reduces the size of cutting out the predetermined image when the size of cutting out the predetermined image is larger than the size of the correction position vector, and the size of cutting out the predetermined image is the correction position. An image processing apparatus for increasing a size for cutting out the predetermined image when the size is smaller than a vector. The image processing apparatus according to claim 1,
The correction limit value calculation unit calculates the correction limit value Smax (n).
Smax (n) = Smax (n−1) + α × {Smax (n−1) −Smax (n−2)} − β × {Smax (n−1) −S (n−1)}
Smax (n-1): correction limit value for a fixed amount image in the time series (n-1) th order Smax (n-2): correction for a fixed amount image in the time series order (n-2) Limit value
S (n-1): correction position vector α for a fixed amount of images in time-series order (n−1): α: positive correction coefficient β: 1 or less positive calculation: positive correction coefficient 0.1 or less An image processing apparatus using a formula. The image processing apparatus according to claim 3,
α is an image processing apparatus in which 0.9 ≦ α <1. The image processing apparatus according to claim 3, wherein:
β is an image processing apparatus in which 0 <β ≦ 0.1. The image processing apparatus according to claim 3, wherein:
An image processing apparatus that decreases the value of β as the value of {Smax (n−1) −S (n−1)} increases. The image processing apparatus according to claim 3, wherein:
An image processing apparatus that decreases the value of β as the value of {Smax (n−1) −Smax (n−2)} increases. An image processing apparatus according to any one of claims 3 to 7,
A correction image generating unit that generates a correction image for the acquired fixed amount of image based on the calculated correction position vector;
The correction position calculation unit calculates the size of the correction image and the enlargement ratio of the correction image based on the correction limit value Smax (n),
The image processing device, wherein the correction image generation unit generates a correction image by cutting out the predetermined image based on the calculated size of the correction image and the magnification of the correction image. An image display device that acquires a fixed amount of an image from a moving image or a time-series continuous still image, and generates a corrected image by cutting out a predetermined image for the acquired fixed amount of image, and displays the corrected image.
For each fixed amount of images in time series, an image acquisition unit that sequentially acquires the fixed amount of images,
A motion vector calculation that calculates a motion vector of a certain amount of image of interest among the certain amount of images acquired sequentially using a certain amount of image immediately before the certain amount of image in time series. And
A correction position calculation unit that calculates a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest;
With
The correction position calculation unit further includes a correction limit value calculation unit that calculates a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. Image display device. An image capturing apparatus that acquires a fixed amount of an image from a moving image or a time-series continuous still image, and generates a corrected image by cutting out a predetermined image for the acquired fixed amount of image,
For each fixed amount of images in time series, an image acquisition unit that sequentially acquires the fixed amount of images,
A motion vector calculation that calculates a motion vector of a certain amount of image of interest among the certain amount of images acquired sequentially using a certain amount of image immediately before the certain amount of image in time series. And
A correction position calculation unit that calculates a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest;
With
The correction position calculation unit further includes a correction limit value calculation unit that calculates a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. Imaging device. An image processing method for acquiring a fixed amount of an image from a moving image or a still image continuous in time series, and cutting out a predetermined image and generating a corrected image for the acquired fixed amount of image,
For each fixed amount of images in chronological order, an image acquisition step of sequentially acquiring the fixed amount of images,
A motion vector calculation that calculates a motion vector of a certain amount of image of interest among the certain amount of images acquired sequentially using a certain amount of image immediately before the certain amount of image in time series. Process,
A correction position calculation step of calculating a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest;
With
The correction position calculating step further includes a correction limit value calculating step for calculating a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. Image processing method. A program for causing a computer to execute a process of acquiring a fixed amount of an image from a moving image or a still image continuous in time series, and cutting out a predetermined image and generating a corrected image for the acquired fixed amount of image. And
For each fixed amount of images in chronological order, an image acquisition process for sequentially acquiring the fixed amount of images,
A motion vector calculation that calculates a motion vector of a certain amount of image of interest among the certain amount of images acquired sequentially using a certain amount of image immediately before the certain amount of image in time series. Processing,
A correction position calculation process for calculating a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest;
With
The correction position calculation process further includes a correction limit value calculation process for calculating a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. program. A program for causing a computer to execute a process of acquiring a fixed amount of an image from a moving image or a time-series continuous still image and cutting out the predetermined image and generating a corrected image for the acquired fixed amount of image is recorded. A computer-readable recording medium,
For each fixed amount of images in chronological order, an image acquisition process for sequentially acquiring the fixed amount of images,
A motion vector calculation that calculates a motion vector of a certain amount of image of interest among the certain amount of images acquired sequentially using a certain amount of image immediately before the certain amount of image in time series. Processing,
A correction position calculation process for calculating a correction position vector indicating a position to cut out the predetermined image for the fixed amount of image of interest;
With
The correction position calculation process further includes a correction limit value calculation process for calculating a correction limit value corresponding to a size for cutting out the predetermined image, and the correction limit value changes according to the magnitude of the correction position vector. A recording medium readable by a computer on which a program is recorded.

Images