JP2005217721A - Apparatus and method for generating still picture - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem with a conventional apparatus and method that had to create a plurality of images for forming a rectangular region, frame by frame, in the case of generating a rectangular still picture such as a panorama picture from a plurality of the frame images. <P>SOLUTION: A plurality of acquired frame images are arranged according to their relative positions, the image regions of the frame images in the laid out state is distinguished as an image existing region by using pixels identified by a prescribed method, and a rectangular still picture which has the largest image size and includes full frame images is automatically calculated from the image existing region. As a result, the rectangular still picture desired by a user is prepared from the plurality of acquired frame images. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

    The present invention relates to a technique for joining a plurality of target images acquired from a moving image or the like and generating a rectangular still image from the joined target images.

  2. Description of the Related Art Conventionally, a technique for connecting captured images and generating a panoramic image that is longer in one direction than the original image is known. Recently, in addition to superimposing a plurality of images taken with a digital still camera, a predetermined number of frame images are taken out from a moving image continuously taken with a digital video camera and connected together. (See Patent Document 1 below).

JP2003-198902A

  In such a panoramic image generating apparatus, a moving image captured by a digital video camera is temporarily recorded on a recording medium, and when reproducing the moving image, timing information for capturing a still image and calculated movement amounts in the horizontal and vertical directions And are read. Thereafter, the panoramic image is generated by superimposing the still images at positions corresponding to the movement amounts of the plurality of captured still images.

  However, such an image generation technique has a problem in that the operation of generating a rectangular panoramic image by superimposing a plurality of frame images must be performed manually and requires complicated operations. Extracting information on the overlap (movement amount) of images captured at a predetermined timing and connecting the images together (hereinafter also referred to as stitching) can be automated, but from a plurality of overlapping images, When cutting out a rectangular panoramic image, the user has been forced to move the cursor to specify an area and cut out unnecessary images.

  The present invention has been made to solve at least a part of such problems, and an object thereof is to easily generate a rectangular still image such as a panoramic image.

The still image generating apparatus of the present invention that achieves the above-described object provides:
A still image generating device that connects a plurality of target images acquired from a moving image or the like and generates a rectangular still image from the connected images,
An image acquisition unit for acquiring the plurality of target images;
A positional relationship data generating unit that generates positional relationship data representing a relative positional relationship of connecting the plurality of acquired target images;
An image area recognition unit that arranges the plurality of target images according to the positional relationship data and recognizes an image existence area defined by a logical sum of the plurality of arranged target images;
A rectangular image area determining unit that determines a rectangular image area that can be generated from the recognized image existence area;
The gist of the invention is that the image processing apparatus includes a still image extraction unit that extracts the rectangular still image using the plurality of target images existing in the determined rectangular image region.

  The still image generation method corresponding to this still image generation apparatus is a still image generation method for connecting a plurality of target images acquired from a moving image or the like and generating a rectangular still image using the connected images. Acquiring the plurality of target images; generating a positional relationship data representing a relative positional relationship between the acquired plurality of target images; and arranging the plurality of target images according to the positional relationship data. A step of recognizing a logical sum image region defined by a logical sum of the plurality of target images arranged, and a step of calculating a rectangular still image region that can be generated from the recognized logical sum image region. This is the gist.

  According to such a still image generation technique, all image regions created by a plurality of target images are examined, and a rectangular still image region that can be generated without missing an image is determined from the image regions. Thereafter, a rectangular still image of this size is extracted using a plurality of target images existing in the determined rectangular image region.

Here, the recognition of the image area may be performed by the pixel that identifies the image existence area, and a rectangular image may be calculated using the identified pixel. In this way, the image area where the target image exists can be easily recognized by examining the presence or absence of identification attached to the pixel.
The image acquisition unit includes, from a moving image composed of a plurality of frame images, a reference target image specified as a reference for acquisition in the moving image, and at least a plurality of frame images existing before or after the reference target image A still image generating apparatus as means for acquiring a target image.

  Further, from a moving image composed of a plurality of frame images, a reference target image designated as a reference for acquisition in the moving image is included, and at least a plurality of frame images existing before or after the reference target image are acquired as target images. It can be configured. In this case, by specifying a reference target image as a reference for a moving image, a rectangular still image defined as a logical sum can be extracted from a plurality of images including the target image. In this case, the plurality of target images may include reference target images, and may be frame images acquired at predetermined intervals from the moving image. If the moving image is a person who has photographed a predetermined range of area over time, a wide rectangular area can be obtained in this way, and so-called panoramic image generation is facilitated.

  When acquiring a plurality of target images, an image obtained by cutting out a peripheral image of the original image to be acquired by a predetermined ratio may be acquired as the target image. By manually cutting off surrounding images by a predetermined ratio, a still image can be generated using an image portion with less distortion.

  Alternatively, a configuration in which designation of at least one target image among a plurality of target images is received, and a maximum area among rectangular image areas including the designated target image is determined from the recognized image existence area. Conceivable. This is because some rectangular regions may occur depending on the combination of target images.

  Furthermore, as the rectangular still image to be generated, the maximum rectangular still image that is an image having an aspect ratio specified from the outside may be calculated. By doing this, it is possible to automatically check the largest rectangular still image that includes a part of the designated target image and has the designated aspect ratio.

  The present invention can also be understood as a computer program and a recording medium on which the program is recorded. As the recording medium, various computer-readable media such as a flexible disk, a CD-ROM, an IC card, and a punch card can be used. In this way, it is possible to generate a still image using an image portion with less distortion. In this way, it is possible to generate a still image using an image portion with less distortion. Examples of the computer program and the recording medium include a case where a storage device such as a hard disk placed on a network is stored so as to be read and executed by another computer via the network. Of course.

  Next, embodiments of the present invention will be described based on examples. FIG. 1 is an explanatory diagram showing a schematic configuration of a still image generating apparatus as an embodiment of the present invention. This image processing apparatus is configured using a general-purpose computer, and includes a keyboard 110 and a mouse 120 as apparatuses for inputting information to the computer 100, and a display 130 and a printer 180 as apparatuses for outputting information. As a device for inputting a moving image to the computer 100, a digital video camera 160, a CD-R / RW drive 140, and a DVD-R / RW drive 150 are provided. In addition, if necessary, a drive device that can read data from a storage medium that stores a moving image can be provided.

  The computer 100 functions as a still image generation device by executing an application program for generating a rectangular still image using a frame image that is a target image acquired from a moving image under a predetermined operating system. . In particular, as shown in FIG. 1, the application program is executed to function as an image acquisition unit 20, a positional relationship data generation unit 30, an image area recognition unit 50, and a rectangular image calculation unit 70.

  Each part manages the following processes. The image acquisition unit 20 captures a moving image input to the computer 100, acquires a plurality of frame images, and passes them to the positional relationship data generation unit 30. The positional relationship data generation unit 30 calculates a relative position between the plurality of frame images and passes it to the image region recognition unit 50 as data. The image area recognition unit 50 checks the image area where the frame image exists from the relative position data of the plurality of frame images that have been passed, and passes the result to the rectangular image calculation unit 70 as data. The rectangular image calculator 70 calculates the largest predetermined rectangular still image that can be generated from the image area data in which the frame image exists.

Next, regarding the processing performed by the still image generating apparatus of the present embodiment,
(A) Image acquisition unit 20 and positional relationship data generation unit 30:
(B) Image region recognition unit 50:
(C) Rectangular image calculation unit 70:
(D) Modification:
This will be described in the following order.

(A) Image acquisition unit and positional relationship data generation unit:
The image acquisition unit 20 and the positional relationship data generation unit 30 perform processing for capturing a plurality of frame images from a moving image and generating the relative positions as coordinate data. This process will be described with reference to the flowchart of FIG. When the processing is started, first, a reference frame image serving as a reference image is designated for a rectangular still image formed by connecting a plurality of frame images, and processing for taking in the computer 100 is performed (step S31). The reference frame image is designated by an operator of the still image generation apparatus (hereinafter simply referred to as “operator”) by inputting data using the keyboard 110, the mouse 120, or the like. Specifically, a moving image is displayed on the display 130, the moving image is stopped at a predetermined position (elapsed time), and the displayed frame image is designated as a reference image. In addition to this, the reference image may be displayed by sequentially displaying frame images, for example, by frame-by-frame, and an image that is considered appropriate may be specified by the operator, or may be extracted at each frame of a moving image or at a predetermined interval. The frame may be displayed in a film strip shape and selected by the operator.

  For the reference frame image designated in this way, the first frame image and the last frame image used for creating a rectangular still image are designated from the successive frame images before and after the reference frame image, and the processing is taken into the computer 100. This is performed (step S32). The operator inputs and designates data with the keyboard 110, the mouse 120, and the like in the same manner as taking in the reference frame image. For example, it may be specified as five frames before and after the reference frame, or images before and after displayed in a film strip shape may be directly specified.

  Other methods for specifying each frame image include a method of specifying an identification number of a frame image that is normally added when capturing a frame image from a moving image and capturing it into the computer 100, or a predetermined time interval. For example, a method of automatically capturing a number of still images from consecutive frame images before and after a designated reference frame image can be considered. The above processing corresponds to the processing of the image acquisition unit 20.

  Next, for a plurality of frame images including a reference frame image designated by the operator, a process of calculating a relative position between the frame images is performed (step S35). The relative positions of all the frame images are obtained by calculating the positional relationship between two frame images described later for a plurality of captured frame images. In this embodiment, the relative positions of the designated reference frame image and the plurality of frame images are calculated one by one, and the relative positions of all the frame images are calculated.

  The calculation process of the relative position between two frame images will be described with reference to FIG. 3A and 3B show two frame images G1 and Gn, respectively. The two images shown in the figure are taken from a moving image obtained by panning the camera from the upper left to the lower right in the mountain landscape against the sunset. The relative positional relationship between the two frame images is usually an image that can be grasped as a translational displacement that can be grasped as a parallel movement, a rotational displacement that can be grasped as a rotation around a predetermined rotation center, and a relative size change caused by zooming. In most cases, this can be grasped by assuming three deviations in magnification. In the case of two images obtained by the camera pan illustrated in FIGS. 3A and 3B, there is no rotation shift or magnification shift between the two images. The deviation can be obtained as a translation vector.

  More specifically, the translation vector can be expressed as a set of the number of pixels representing the displacement of the pixels in the horizontal direction and the vertical direction between the two images. Such a shift in the number of pixels can be generated by a known processing technique such as image pattern matching or feature point tracking. If the relative position data calculated from the translation vector thus generated is the number of pixels Xn in the horizontal (X) direction of the screen and the number of pixels Yn in the vertical (Y) direction of the screen, the upper left corner C1 of the frame image G1 is set as the reference point. When (0, 0) is set, the upper left corner Cn of the frame image Gn has coordinates (Xn, Yn). This state is shown in FIG. In this way, the relative position of the frame image Gn with respect to the frame image G1 is calculated.

  Further, when the rotation and the image magnification are different between the frame image G1 and the frame image Gn, the relative position can be calculated by, for example, a combination of optical flow estimation and pattern matching. There are various optical flow estimation methods. For example, the gradient-based method is based on the assumption that the luminance of an object is invariable between observed frame images, and the density distribution space of the object in the image. Using the relationship between the gradient and the time gradient, the movement of the subject relative to the photographing apparatus is estimated. Based on the result of this optical flow estimation, roughly the translational movement, rotation, and change in zoom magnification of the image are estimated, and on the basis of this estimation result, the frame image G1 and the frame image Gn become frame images having the same magnification. The image processing is performed as described above, and the frame position after the image processing is newly set as the frame image G1 and the frame image Gn, and the relative position is calculated by performing pattern matching between the images.

  As described above, the relative position calculation process between the two frame images is performed between the reference frame image designated by the operator and all other frame images by correcting the rotation of the frame image and the image magnification. Then, the X and Y coordinate data of the upper left corner of another frame image when the upper left corner of the reference frame image designated by the operator is set as the origin (0, 0) is generated as positional relationship data representing the amount of deviation ( FIG. 2, step S36). These processes correspond to the process of the positional relationship data generation unit 30. The positional relationship data obtained in this way is used for processing of the image region recognition unit 50 together with the frame image.

(B) Image area recognition unit:
Next, processing of the image area recognition unit 50 will be described. The image area recognition unit 50 arranges all the frame images designated by the operator in one image area based on the X and Y coordinate data, and is an image area that is the logical sum of all the frame images, that is, all the frame images. A process for recognizing an area where an image formed by connecting the images exists is performed. This process will be described with reference to the flowchart of FIG. When the process is started, first, all the frame images are arranged based on the X and Y coordinate data of the upper left corner, which is the positional relationship data generated in the process step S36 (FIG. 2) (step S51). ). For example, when all the frame images designated by the operator are four consecutive frame images (F1 to F4) having the same image magnification, and the reference frame image is assumed to be F2, as shown in FIG. Thus, based on the X and Y coordinate data with the origin (0, 0) at the upper left corner of the reference frame image F2, the other three frame images F1, F3, and F4 are respectively (X1 , Y1), (X3, Y3), and (X4, Y4).

  Next, as shown by a broken line in FIG. 5B, a process of setting a rectangular identification image R including an image area that is a logical sum of all the arranged frame images is performed (FIG. 4, step S52). . Here, the identification image R is preferably set so as to be as small as possible. This is to reduce the load of the process of adding “marks” to the pixels constituting the identification image R, which will be described later (step S54). In this embodiment, a minimum rectangular image in which all frame images are stored inside is set as the identification image R. Of course, as long as the image has a size that includes all frame images, any shape or size may be used.

  In this embodiment, the size of the identification image R is obtained from the X and Y coordinate data of all the frame images and the screen size of the frame image. Assuming that each frame image is taken by panning up and down, left and right as in the example of FIG. 5, there is no difference in rotation and image magnification, and the difference in screen tilt and screen size for all frame images You can think of it as not. Therefore, as shown in FIG. 5B, the vertical length of the identification image R is the difference between the maximum value and the minimum value (Y3−Y1) of the Y coordinate values at the upper left corner of the screen of all frame images. The length in the Y direction of the image, that is, the vertical screen length H is added. The horizontal length of the identification image R is the difference between the maximum and minimum X coordinate values (X4−X1) of the upper left corner of the screen of all the frame images. The screen length W is added. In this embodiment, the size of the identification image R is obtained in this way. When a frame image with corrected rotation and image magnification is included, the relative position of the frame image and the inclination and size of the screen are checked one by one to determine the vertical length and horizontal length of the identification image R. You can ask for.

  Next, in order to store the region where the frame image exists and the region where the frame image does not exist in the set identification image R, an image region which is a logical sum of all the frame images (hereinafter referred to as “image existence region”) A process of resetting the coordinate system based on the identification image R is performed (step S53). That is, as shown in FIG. 5 (c), the process of resetting the image presence area to the X, Y coordinate system with the upper left corner of the screen of the identification image R as the origin (0, 0) is performed. By this processing, the image existence area has common X and Y coordinate data in the identification image R. In this way, in the identification image R, the image area where the frame image exists and the image area where the frame image does not exist can be distinguished according to whether they are inside or outside the image existence area.

  In the identification image R including the image existence area in which the coordinates are reset in this way, among the pixels constituting the identification image R, the pixels that are entirely included in the image existence area are identified by adding “marks”, and the frame image is identified. A process of distinguishing from the non-existing area is performed (step S54). By this processing, the presence / absence of the frame image can be identified by the presence / absence of the “mark” attached to the pixels constituting the identification image R. This is a so-called labeling process in image recognition.

  The size of the pixels constituting the identification image R is preferably set small in order to reduce the identification error of the image existence area. In this embodiment, the identification image R is set as an image having the same resolution as that of the frame image in order to discriminate between the inside and outside of the image existence area with a small error. Then, as described above, in order to reliably check the area where the frame image exists, all the pixels constituting the identification image R are marked with “marks” in the pixels included in the image existing area. Naturally, the size of the pixels constituting the identification image R may be determined from the processing load performed by the rectangular image calculation unit 70 described later and the identification error of the image existence area. Also, the pixel to be “marked” may be a pixel including the pixel center or a pixel including a predetermined area ratio.

  The “mark” can be assigned by various methods. For example, by performing binarization processing in which all the pixels of the identification image R existing in the frame image screen are set to pixel data “1” and the other pixels are set to pixel data “0”, the pixel data “1” is changed. A method of “marking” can be considered. Alternatively, a method of adding RGB data to a pixel and setting at least one of the RGB data as a “mark” is also conceivable. FIG. 5C illustrates an identification image R divided into a pixel area “marked” in this way, that is, an image presence area RS (shaded portion) in which a frame image exists and an image non-area RN in which no frame image exists. . The above processing corresponds to the processing in the image area recognition unit 50.

(C) Rectangular image calculation unit:
Next, a process of calculating the largest screen size that can be generated from the image area RS with a rectangular still image specified by the operator is performed. The processing performed by the rectangular image calculation unit 70 will be described with reference to the flowchart of FIG. First, a process for designating a reference frame image as a reference for a rectangular still image is performed (step S71). As the reference frame image, the one specified by the operator in the processing step S31 (FIG. 2) is normally specified by default. However, when the reference frame image is changed, data is input using the keyboard 110 or the like here. specify.

  Next, processing for designating the aspect ratio of the finally generated rectangular still image is performed (step S72). The designation method is performed by inputting data with the keyboard 110 or the like for the aspect ratio E desired by the operator. In this embodiment, the aspect ratio E is the horizontal length when the vertical length is 1.

  In order to calculate the maximum rectangular still image of aspect ratio E including the reference frame image according to the reference frame image and aspect ratio E specified by the above processing, as shown in FIG. I, J) is performed to calculate the maximum rectangular area (step S74). The coordinates (I, J) are X and Y coordinates with the upper left corner of the screen of the identification image R as the origin (0, 0).

  In the process of calculating the maximum rectangular area (step S74), in the identification image R, each pixel is alternately expanded in the vertical and negative (± Y) directions from the reference pixel from which calculation starts, and the horizontal and horizontal (± X) of the screen is determined. A rectangular area formed by the upper left corner LT and the lower right corner RB of the screen while enlarging the area by a length obtained by multiplying the vertical length of one pixel by the specified aspect ratio E in the direction, and performing the enlargement process of the area. This is a process of calculating a rectangular region having a maximum size that satisfies the condition by checking whether all the pixels existing in the pixel are “marked” pixels.

  This processing will be described with reference to the flowchart of FIG. When this process is started, first, variables m and n are set to an initial value 0 (step S741). Variables m and n represent the number of pixels that increase in the upward direction of the screen and the number of pixels that increase in the downward direction of the screen, respectively, from the reference pixel that is a reference for starting the processing. The reason why the variables m and n are set to the value 0 is to make the rectangular area calculation process start, that is, to set the state of only one reference pixel.

Next, in order to obtain the maximum rectangular area to be calculated by an operation for searching from the screen center (I, J) of the reference frame image, a process of setting the X and Y coordinates of the reference pixel to (I, J) is performed (step S742). ). As shown in FIG. 10, assuming that the center coordinates of the reference pixel are (I, J), the coordinates of the upper left corner point LT and the lower right corner point RB of the rectangular area defined by the variables m and n are
LT (I- (m + 0.5) ELy, J- (m + 0.5) Ly)
RB (I + (n + 0.5) ELy, J + (n + 0.5) Ly)
It becomes. Here, E is the aspect ratio, and Ly is the vertical length of one pixel. Therefore, the coordinate values of LT and RB are set in this way, and the calculation of the maximum rectangular area is started from the screen center of the reference frame image.

  In the identification image R, the screen center (I, J) of the reference frame image may not match the center of the reference pixel depending on the arrangement position and the screen size of the reference frame image. In this case, the subsequent processing steps may be performed by moving the screen center of the reference frame image to the pixel center of the reference pixel. Although the center of the screen is shifted by a maximum of 0.5 pixels, it may be corrected as necessary after the process of calculating the maximum rectangular area.

  After the above-described processing is completed, the variables m and n are alternately increased one by one, and the rectangular area whose diagonals are the upper right corner point LT and the lower left corner point RB is sequentially enlarged, and the rectangular area is increased. The presence or absence of “marks” attached to the pixels inside is checked. If all the pixels in the rectangular area have “marks” attached, the rectangular area defined by the upper right corner point LT and the lower left corner point RB is enlarged, and “marks” are attached to the pixels in the rectangular area. If a pixel that has not been processed is found, the processing is stopped there. Hereinafter, this process will be described in detail.

  First, a process for enlarging the screen upward by one pixel from the reference pixel is performed (FIG. 8, step S743). Specifically, the variable m is increased by one. At this time, the presence of the frame image is examined in a rectangular area formed by the upper left corner point LT and the lower right corner point RB (step S744). Therefore, step S744 is executed many times while increasing the variable m. FIG. 11 illustrates the third process of increasing the variable m (step S743), that is, a state in which the variable m is increased from the value 2 to the value 3. When the variable m is the value 2, the increment area that can be formed on the upper side and the left side of the screen indicated by the shaded portion is an image area that is newly increased by enlarging the screen with the variable m as the value 3. . Therefore, it is only necessary to check the presence or absence of “marks” only for the pixels included in this increment area that is increased each time the variable m increases by one.

  The pixels included in the increment area increase by one pixel in the vertical direction of the screen, that is, by Ly, so that the boundary of the pixel always coincides with the boundary of the increment area, but the horizontal direction of the screen is a value E · Ly multiplied by the aspect ratio E. Since it increases step by step, the horizontal boundary of one pixel may not match the boundary of the increment region. In the present embodiment, if the center of the pixel is a pixel included in the increment region, the presence of the “mark” is examined. Of course, only the pixels in which the entire pixel is included in the increment region may be examined. Further, the area ratio of the pixels included in the increment region may be determined and examined in advance.

  Thus, the presence or absence of the frame image in the increment area is checked based on the presence or absence of the “mark” of the pixel, and the frame image exists in all the rectangular areas created by the upper left corner point LT and the lower right corner point RB (there is a “mark”). The process which determines whether or not is performed (step S745). As a result of the determination, if there is a pixel without a “mark” (step S745: NO), the variable m is decreased by a value 1 assuming that the rectangular area has been enlarged (step S746). That is, the position of the upper left corner point LT is returned to the previous position. Thus, the process proceeds to the next process. On the other hand, if it is determined that all the pixels have “marks” as a result of the determination (step S745: YES), the position of the upper left corner point LT is left as it is, and the process proceeds to the next process.

  Next, a process for enlarging the screen downward by one pixel is performed (step S747). Specifically, the variable n is increased by the value 1. At this time, the presence / absence of a frame image in the rectangular area defined by the upper left corner point LT and the lower right corner point RB is checked based on the presence / absence of a pixel “mark” (step S748). Similarly to the state shown in FIG. 11, when the variable n is increased, the increment area becomes a new area that is newly increased by enlarging the screen. The increment area when the variable n is increased by the value 1 is not particularly shown, but it can be easily understood that the increment areas are formed at the bottom and right sides of the screen. The presence or absence of “marks” is examined for the pixels in this incremental region.

  Even when the variable n is increased, as in process step S745, the presence or absence of “mark” for the pixel in the increment area is checked, and whether or not there is a frame image in all the rectangular areas created by the upper left corner point LT and the lower right corner point RB. A process of determining whether or not is performed (step S750). As a result of the determination, if there is a pixel having no “mark” (step S750: NO), it is determined that the increment region has been enlarged too much, and a process of reducing the variable n by a value 1 is performed (step S751). The position of the lower right corner point RB is returned to the previous position. On the other hand, if the result of determination is that all the pixels have “mark” (step S750: YES), the position of the lower right corner point RB is not returned and the process proceeds to the next process.

  Next, in order to check whether or not the rectangular area formed by the upper left corner point LT and the lower right corner point RB is enlarged, whether or not the value m + n, which is the sum of the variables m and n, is larger than the value in the previous process. Is determined (step S755). If the value m + n has not increased (step S755: NO), the rectangular area is not enlarged, and therefore the rectangular area created by the upper left corner point LT and the lower right corner point RB whose coordinates are determined from the values of the variables m and n at this time. Is the maximum rectangular area (step S756), the process shown in FIG. 8 is terminated, and the process returns to step S78 shown in FIG.

  When the value m + n is increased (step S755: YES), the subsequent processing is performed in three cases. First, when both m and n are increased, the process of checking the presence or absence of a “mark” of the pixel in the increment region by further moving the positions of the upper left corner point LT and the lower right corner point RB is repeated. Therefore, returning to the processing step S743, the above processing (steps S743 to S755) is repeated while incrementing the variables m and n by one.

  When only the variable m is increased, there is no frame image in the image area outside the position of the lower right corner point RB. Therefore, the position of the lower right corner point RB is fixed and only the upper left corner point LT is moved further. Thus, only the process of examining the “mark” of the pixel in the increment area is performed. Therefore, in this case, as shown in the flowchart of FIG. 9A, the screen is enlarged in the upper left direction, that is, the above-described determination process is continued while only the variable m is increased one by one among the variables m and n. .

  The processing performed in steps S743a to S746a in the flowchart of FIG. 9A is basically the same as the processing from steps S743 to S746 described in FIG. Therefore, description of the processing performed by each step is omitted. In step S755a, it is determined whether or not the variable m has increased. If the variable m has increased (step S755a: YES), the process returns to step S743a and the processing is continued. If it has not increased (step S755a: NO), it is determined that the rectangular area has become the maximum. That is, assuming that the rectangular area formed by the upper left corner point LT and the lower right corner point RB whose coordinates are determined from the values of the variables m and n at this time is the maximum rectangular area (step S756a), this processing routine is terminated, and FIG. The process returns to step S78 shown in FIG.

  When only the variable n is increased, there is no frame image in the image area outside the position of the upper left corner point LT. Therefore, the position of the upper left corner point LT is fixed, and only the lower right corner point RB is moved further. Then, the process of examining the “mark” of the pixel in the increment region is continued. Therefore, in this case, as shown in the flowchart of FIG. 9B, the screen is enlarged in the lower right direction, that is, the above-described determination process is continued while only the variable n is increased one by one among the variables m and n. To do.

  The processing performed in steps S747b to S751b in the flowchart of FIG. 9B is basically the same as the processing from steps S747 to S756 described in FIG. Therefore, description of the processing performed by each step is omitted. In step S755b, it is determined whether or not the variable n has increased. If the variable n has increased (step S755b: YES), the process returns to step S747b to continue the processing. If it has not increased (step S755b: NO), it is determined that the rectangular area has become the maximum. That is, the rectangular area created by the upper left corner point LT and the lower right corner point RB whose coordinates are determined from the values of the variables m and n at this time is the maximum rectangular area (step S756b), and this processing routine is terminated. The process returns to step S78 shown in FIG.

  As described above, while determining the increase of the variables m and n, the set processing is repeatedly performed to calculate the maximum rectangular area formed by the upper left corner point LT and the lower right corner point RB (FIG. 6, step S74). ) And the process proceeds to the next processing step S78.

  In step S78, the screen portion included in the maximum rectangular area determined in the identification image R is cut out from the plurality of frame images specified by the operator, and processing is performed to generate a maximum rectangular still image having the specified aspect ratio. FIG. 12 shows a maximum rectangular still image having an aspect ratio E that can be generated from the plurality of frame images illustrated in FIG. The diagonal imaginary line in the figure is a line indicating the aspect ratio 1: E of the screen of the rectangular still image. As described above, when the center of the screen of the reference frame image is shifted from the center of the pixel constituting the identification image R, the image is generated with correction as necessary. Above, the process in the rectangular image calculation part 70 is complete | finished, and all the processes which the still image generation device of a present Example performs are complete | finished.

  As described above, according to the present embodiment, since the maximum rectangular still image that can be generated can be automatically calculated from a plurality of designated frame images and the aspect ratio, the operator can obtain a rectangular still image having a desired shape and size. Can be easily obtained.

(D) Modification:
The embodiment of the present invention has been described above by way of an example. However, the present invention is not limited to such an embodiment, and can be implemented in various forms without departing from the spirit of the present invention. Of course. For example, as a modification, in the process of the image acquisition unit 20, when capturing a plurality of frame images from a moving image, surrounding images may be captured by being cut off at a predetermined rate.

  Such processing as this modification may be performed when capturing a frame image in steps S31 and S32 shown in FIG. FIG. 13 shows a state in which a frame image F1s obtained by cutting off the periphery of the frame image F1 is taken as an example. In the subsequent processes, the frame image F1s may be handled as the frame image F1 in the above embodiment. When calculating the relative position of the frame image, the same processing as in this embodiment is performed if the upper left corner position C1s of the screen is handled as C1.

  In an optical system such as a lens, there are distortions such as various aberrations and variations in light quantity. Such distortion is usually greater at the periphery of the image. According to the present modification, although the screen to be captured is small, a still image is generated using an image with a small peripheral distortion portion of the image, so that a still image with less distortion can be generated. The ratio of the image to be cut off may be cut off by approximately 20% in the screen area ratio, or may be set according to the optical performance of the photographing apparatus that captured the moving image, the resolution of the image sensor, or the like.

  In this embodiment, when calculating the maximum rectangular area, the calculation of the rectangular area where the frame image exists is started from the center of the reference frame image. However, as a modified example, the calculation may be started from other than the center of the screen. The operator may set the cursor by moving the cursor to a position where processing is to start (not shown) and clicking the mouse 120 while viewing the reference frame image displayed on the display 130. Such a technique is useful for designating an image portion to be included in a rectangular still image to be generated, among the reference frame images.

  Furthermore, in the present embodiment, when calculating the maximum rectangular area, the variable m is first increased by one pixel and the upper left corner point LT is moved, and the subsequent calculation process is proceeded. A configuration in which n is increased by one pixel is also possible. In this way, the order of increasing pixels is only reversed, and there is no particular change. In the above embodiment, a horizontally long rectangular area is assumed, but a vertically long rectangular area may be obtained.

  In this embodiment, the maximum rectangular still image having the aspect ratio E specified by the operator is calculated. However, as a modified example, the maximum rectangular still image not specifying the aspect ratio may be calculated. A method in this case will be described with reference to FIG. In this embodiment, the pixel is increased by one pixel in the vertical direction of the screen and is increased by the length automatically determined by the aspect ratio E in the horizontal direction of the screen. However, in this modification, not only in the vertical direction of the screen but also in the horizontal direction of the screen. The maximum rectangular still image is calculated by incrementing by one pixel.

  As shown in FIG. 14, in this modification, the vertical direction of the screen is increased from the reference pixel to m in the upper direction of the screen and n in the lower direction of the screen. The process of setting the number of increased pixels to p in the left direction and q in the right direction of the screen is performed. Then, like the present embodiment, after increasing by one pixel in the order of the variables m and n, and subsequently increasing by one pixel in the order of the variables p and q, respectively, the increment regions M, N, P, and Q shown in FIG. It is checked whether or not a pixel is marked with “mark”. For example, if there is a pixel without “mark” in the increment region P, p is fixed to a value reduced by one, and the remaining m, n, and q are incremented and the pixels for those increment regions M, N, and Q are increased. Check for the presence of the “mark”. When this process is repeated and m, n, p, and q all become fixed values, the maximum rectangular area is obtained. Further, the order of increasing one pixel at a time is not limited to this, and any order of m, n, p, and q may be used.

It is a schematic block diagram of one Example of this invention. It is a flowchart which shows an example of the process of the still image generation device in an Example. It is explanatory drawing shown about calculation of the relative position of the frame image in an Example. It is a flowchart which shows an example of the process in the positional relationship data generation part in an Example. It is explanatory drawing shown about the arrangement | positioning of a frame image, and the setting of the image area | region R. FIG. It is a flowchart which shows an example of the process in the image area memory | storage part in an Example. It is explanatory drawing shown about the screen center of the reference | standard frame image in an Example. It is a flowchart which shows an example of the process which calculates the largest rectangular still image of the designated aspect ratio. It is a flowchart which shows an example of the continuation process which similarly calculates the largest rectangular still image. It is explanatory drawing which shows the position of the upper left corner point LT and the lower right corner point RB for the largest rectangular area calculation. It is explanatory drawing which illustrates about the increment area | region which investigates the presence or absence of a frame image. It is explanatory drawing which illustrates about the largest rectangular still image produced | generated in an Example. In a modification, it is explanatory drawing shown about the cutting off of the periphery of a frame image. It is explanatory drawing which shows the calculation method of the largest rectangular still image which does not set an aspect ratio in the modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 20 ... Image acquisition part 30 ... Position relation data generation part 50 ... Image area memory | storage part 70 ... Rectangular image calculation part 100 ... Computer 110 ... Keyboard 120 ... Mouse 130 ... Display 140 ... CD-R / RW drive 150 ... DVD-R / RW drive 160 ... Digital video camera 180 ... Printer

Claims (10)

A still image generating device that connects a plurality of target images acquired from a moving image or the like and generates a rectangular still image from the connected images,
An image acquisition unit for acquiring the plurality of target images;
A positional relationship data generating unit that generates positional relationship data representing a relative positional relationship of connecting the plurality of acquired target images;
An image area recognition unit that arranges the plurality of target images according to the positional relationship data and recognizes an image existence area defined by a logical sum of the plurality of arranged target images;
A rectangular image area determining unit that determines a rectangular image area that can be generated from the recognized image existence area;
A still image generating apparatus comprising: a still image extracting unit that extracts the rectangular still image using the plurality of target images existing in the determined rectangular image region. The still image generating device according to claim 1,
The image area recognition unit is means for recognizing the image existence area based on the presence of pixels constituting each of the plurality of target images.
The rectangular image region determination unit is a unit that determines the rectangular image region using information on the position of the pixel. The still image generating device according to claim 1 or 2,
The image acquisition unit includes, from a moving image composed of a plurality of frame images, a reference target image specified as a reference for acquisition in the moving image, and at least a plurality of frame images existing before or after the reference target image A still image generating apparatus as means for acquiring a target image.   The still image generating apparatus according to claim 3, wherein the plurality of target images include frame images acquired at predetermined intervals from the moving image including the reference target image. The still image generating device according to any one of claims 1 to 4,
The image acquisition unit is a still image generation device that is a unit that acquires, as the target image, an image obtained by cutting off a predetermined percentage of an image around the original image to be acquired when acquiring the plurality of target images. The still image generating device according to any one of claims 1 to 5,
The rectangular image region determination unit
Means for accepting at least one designation among the plurality of target images;
A still image generating apparatus comprising: means for determining a maximum area among rectangular image areas including the designated target image from the recognized image existence area. The still image generating device according to any one of claims 1 to 5,
The rectangular image region determination unit
Means for accepting designation of an aspect ratio of the rectangular image region to be determined;
And a means for determining a maximum area of the rectangular image areas having the aspect ratio from the recognized image existence area. A method of connecting a plurality of target images acquired from a moving image or the like and generating a rectangular still image from the connected images,
Obtaining the plurality of target images;
Generating positional relationship data representing a relative positional relationship in which the plurality of acquired target images are connected;
Arranging the plurality of target images according to the positional relationship data, recognizing an image existence region defined by a logical sum of the plurality of arranged target images;
Determining a rectangular image area that can be generated from the recognized image presence area;
A still image generating method for extracting the rectangular still image using the plurality of target images existing in the determined rectangular image region. A program for causing a computer to execute a process of connecting a plurality of target images acquired from a moving image or the like and generating a rectangular still image from the connected images,
A function of acquiring the plurality of target images;
A function of generating positional relationship data representing a relative positional relationship of connecting the acquired plurality of target images;
A function of arranging the plurality of target images according to the positional relationship data and recognizing an image existence region defined by a logical sum of the plurality of arranged target images;
A function of determining a rectangular image area that can be generated from the recognized image existence area;
A still image generation program for causing a computer to realize a function of extracting the rectangular still image using the plurality of target images existing in the determined rectangular image region.   A recording medium on which the program according to claim 9 is recorded so as to be readable by a computer.

Images