JP5035614B2 - Imaging apparatus and program - Google Patents

Description

The present invention relates to an imaging apparatus and a program, and more particularly, to an imaging apparatus and a program that insert a specified area image in a sub-image into a specified area in a main image.

  In the field of image processing such as a personal computer, a workstation, or a digital camera, image composition on a display or a monitor screen is widely performed. For example, in Patent Document 1 below, a technique for embedding a reduced or enlarged image of a designated area image in a sub-image stored in an image memory in a designated area in a main image held in a frame memory. (Hereinafter referred to as the prior art).

  By the way, the “designated region” in this prior art is a “rectangle” such as a square or a rectangle, but depending on the image composition, a shape other than a rectangle, for example, a “horizontal trapezoid” region having different left and right side lengths, Although the filling of the sub-image that is “rectangular” may be required, the above-described conventional technology cannot meet this requirement.

Japanese Patent Application Laid-Open No. 5-114026

  That is, in the above technique, even if the main image and the sub-image are combined, it is difficult to obtain a desired image because of its low degree of freedom. Obtaining was an issue.

Accordingly, an object of the present invention is to provide an imaging apparatus and program capable of effectively synthesizing a main image and a sub image.

The present invention provides an imaging unit that captures a subject and acquires a captured image, a first image acquisition unit that acquires the captured image as a first image, and a second image acquisition unit that acquires a second image. First area acquisition means for acquiring a first square area which is a rectangle on the first image plane; and second area acquisition for acquiring a second square area on the second image plane Means for converting the first image in the first square area, which is a rectangular area, into a shape based on the shape of the second square area, and obtaining a converted image; and An image in a second square area in the second image and a combining unit that combines the converted image, and the converting unit performs a trapezoidal correction on the converted image generated by the converting unit. Of the images obtained by cutting out a part of the first image An image pickup apparatus or a program which is characterized in that the transformation of the image so that the relationship between one and similar.
According to another aspect of the present invention, an imaging unit that captures an image of a subject to acquire a captured image, a first image acquisition unit that acquires the captured image as a first image, and a second image are acquired. Second image acquisition means, first area acquisition means for acquiring a first square area on the first image plane, and second square area that is a rectangle on the second image plane Second area acquisition means for converting the second image in the second square area, which is a rectangular area, to obtain a converted image by applying a shape conversion based on the shape of the first square area. A conversion unit; and an image in the first square area in the first image; and a combination unit that combines the conversion image. The conversion unit generates the conversion image generated by the conversion unit. The image obtained when keystone is corrected is a part of the second image. An image pickup apparatus or a program which is characterized in that the conversion of the image to be in one and similarity relationship in the image.

  According to the present invention, a main image and a sub image can be effectively combined.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking application to an imaging apparatus such as a digital camera as an example.

  FIG. 1 is a conceptual configuration diagram of an imaging apparatus according to an embodiment of the present invention. In this figure, an imaging device 1 captures an image of a subject (not shown) via an optical system 2, converts the image into an electrical signal, and outputs it as an image signal, such as a CCD or CMOS, and an imaging unit. An image processing unit 4 for performing necessary image processing such as gamma correction on the image pickup signal from 3, and displaying the image signal after image processing as a through image for composition confirmation, and reproducing and displaying a photographed image A display unit 5 including a liquid crystal display panel for displaying various setting menus as necessary, a storage unit 6 for storing captured images, and various operation buttons such as a shutter button 7 and a mode button 8. An operation unit 9 having a battery, a power supply unit 10 including a battery, an input / output unit 11 that exchanges image data and the like with an external device (such as a personal computer), and a control unit that performs overall control of the operation of the imaging apparatus 1. And a 2. In addition to this, a strobe light emitting unit or the like is provided, but is omitted here in order to avoid complication of the drawing.

  The control unit 12 includes, for example, a CPU 13, a RAM 14, a ROM 15, and the like. The CPU 13 reads the control program and control data stored in the ROM 15 into the RAM 14 and executes them, so that it is necessary for the imaging apparatus 1. Various functions (photographing function, playback function, etc.) are realized.

  FIG. 2 is a diagram showing a simplified flow of a control program executed by the CPU 13 of the control unit 12. This program is executed when the power of the imaging apparatus 1 is turned on. When this program is started, first, the operation mode (photographing mode / playback mode) of the image pickup apparatus 1 is determined (step S1). This operation mode can be arbitrarily changed by the mode button 8 of the operation unit 9 as desired by the user (photographer).

  If the shooting mode is determined in step S1, the shooting process is executed (step S2). If the playback mode is determined, the playback process is executed (step S3).

  FIG. 3 is a diagram showing a simplified flow of the photographing process. In this figure, when shooting processing is started, first, a through image is displayed (step S11), and it is determined whether or not it is in an image composition mode (step S12). If it is not the image composition mode, it is determined that the camera is in the normal shooting mode, and a pressing operation of the shutter button 7 of the operation unit 9 (hereinafter referred to as shutter release) is determined (step S13), and the shutter release is determined. Then, the captured image output from the image processing unit 4 is captured (step S14), and the captured image is stored in the storage unit 6 (step S15).

On the other hand, when the image compositing mode is determined in step S12, the shutter release is determined (step S16). When the shutter release is determined, the captured image of the image capturing unit 3 output from the image processing unit 4 is “first image”. In addition to capturing as an “image” (step S17), a predetermined image (for example, a template image) stored in the storage unit 6 is captured as a “second image” (step S18), and the first image and the second image are stored. A synthesis process described later is executed on the image (step S19), and the synthesized image is stored and saved in the storage unit 6 (step S15).
Here, the “second image” that is a predetermined image is captured after capturing the “first image” that is a captured image, but the order may be reversed depending on the situation. Absent.

  FIG. 4 is a diagram showing a simplified flow of the reproduction process. In this figure, when the reproduction process is started, it is first determined whether or not it is an image composition mode (step S21). If the mode is not the image composition mode, it is determined that the playback mode is normal, and a thumbnail list of captured images stored in the storage unit 6 is displayed on the display unit 5 (step S22). In response to the selection operation (step S23), the selected image is displayed on the display unit 5 (step S24).

  On the other hand, when the image composition mode is determined in step S21, a thumbnail list of captured images stored in the storage unit 6 is displayed on the display unit 5 (step S25), and the "first image" by the user is displayed. In response to the selection operation (step S26) and the “second image” selection operation (step S27), a composition process described later is executed on the selected images (first image and second image). After the composite image is displayed on the display unit 5 (step S29) or in parallel with the display, the composite image is stored and saved in the storage unit 6 (step S30).

  FIG. 5 is a diagram showing a simplified flow of the synthesizing process (step S19 in FIG. 3 and step S29 in FIG. 4) common to the photographing process and the reproducing process, and FIG. 6 is a conceptual diagram of the synthesizing process. In these figures, when the synthesis process is started, first, an image included in the rectangular designated area 22 in the first image 21 (hereinafter referred to as a synthesized image 23) is extracted (step S31 and FIG. 6). (A)). Here, it is assumed that the synthesized image 23 includes the letter “A” of the alphabet for convenience of explanation.

  Next, a synthesis area 25 is set in the second image 24 (step S32 and FIG. 6B). The shape of the synthesis region 25 is arbitrary, and here, for example, it is assumed that it is a horizontal trapezoidal shape having different left and right side lengths.

  Next, the shape of the synthesized image 23 is deformed in accordance with the shape of the synthesis region 25 (step S33 and (c) of FIG. 6). Here, as an example corresponding to the shape of the synthesis region 25, the shape of the synthesized image 23 is transformed into a horizontal trapezoidal image 26 in accordance with the shape of the synthesis region 25. For this modification, it is preferable to use a known affine transformation.

  Next, the converted composite image 23 (that is, the horizontal trapezoidal image 26 in this example) is embedded in the composite region 25 in the second image 24 and composited to obtain a final composite image 27. (Step S34 and FIG. 6D).

  As described above, according to the present embodiment, when the image (the synthesized image 23) included in the rectangular designated area 22 of the first image 21 is fitted into the synthesized area 25 of the second image 24, the first In consideration of the shape of the composite region 25 of the second image 24, the above-described filling can be easily performed after appropriately changing the shape of the composite image 23, which is not a special technique described in the beginning. An effect can be obtained.

  In addition, this invention is not limited to the above embodiment, Of course, various development examples and modifications are included within the scope of the idea, and for example, the following may be adopted.

  In the above-described embodiment, when performing the image composition in the shooting mode, the image included in the rectangular designated area 22 of the first image 21 (the image to be combined 23) in the composition area 25 of the second image 24. However, this may be reversed. That is, although not shown, the image (the synthesized image 23) included in the rectangular designated area of the second image 24 may be fitted into the synthesized area of the first image 21.

  In the above-described embodiment, the first image 21 at the time of image composition in the shooting mode is a shot image shot by the imaging unit 3, and the second image 24 is stored in the storage unit 6. This is a predetermined image (for example, a template image that is an image for synthesis), but this may be reversed. That is, the first image 21 may be a predetermined image (for example, a template image) stored in the storage unit 6, and the second image 24 may be a captured image captured by the imaging unit 3. Alternatively, instead of a predetermined image (for example, a template image) stored in the storage unit 6, another image, for example, a through image may be used.

  For example, a first image that is an image for synthesis may be selected, and a second image that is a through image may be displayed in a desired region of the first image.

  At this time, it is desirable that the through image is displayed after the shape of the image is deformed in accordance with the synthesis region by the above-described synthesis processing. That is, in this state where the second image, which is a through image, is deformed and displayed in a desired area in the first image, which is the image for composition, by pressing the shutter key. Thus, it is desirable to generate a composite image by replacing the through image described above with a captured image.

  Accordingly, there is an advantage that a desired composite image can be photographed and combined while confirming the state after the image combination as a through image. Conventionally, it has not been possible to take a picture for image synthesis while taking into account the quality of the synthesized image, so that this configuration is particularly advantageous.

  This will be described with a specific example. Now, for example, an image in which something like a frame (hereinafter referred to as a frame for convenience of description) is assumed as the first image, and a through image (second image) is included in the frame of the first image. Consider displaying an image. In this case, the photographer adjusts the composition while looking at the monitor image displayed on the display unit 5 (through image in the frame of the first image), and takes a picture when the desired composition is obtained. (Shutter release), however, the photographed image is as if the subject was fitted in the frame, and eventually, a composite image overflowing with playfulness such as a so-called “printer” can be obtained.

  Here, when the through image is fitted into the frame, the size of the original through image is reduced in accordance with the size of the frame. Further, when the shape of the frame is a shape other than a rectangle, along with the above reduction, the shape of the original through image is deformed according to the shape of the frame.

  Further, the shape of the synthesized image 23 is deformed corresponding to the shape of the synthesis area 25 (step S33 and (c) of FIG. 6), as illustrated above, the shape of the synthesis area 25 is a horizontal trapezoidal shape. In this case, not only the shape of the synthesized image 23 is transformed into the horizontal trapezoidal image 26 in accordance with it, but also other modes are included. That is, not only the shape of the synthesized image 23 is deformed so as to fit exactly in the synthesized area 25 but also includes a mode in which the synthesized image 23 is deformed to a certain extent beyond the synthesized area 25. As a result, a composite image in a more natural state may be obtained.

Further, the synthesized image 23 may be the entire first image 21.
FIG. 7A is a conceptual diagram in the case where the entire first image 21 is the combined image 23. In this figure, the composite image 23 is the entire first image 21, and is too large to fit in the composite area 25 of the second image 24. Therefore, in this case, the entire image to be synthesized 23 (= first image 21) is reduced, and the reduced image 28 is deformed in correspondence with the shape of the synthesis region 25.

In addition, in the case of the composite image 23 that is too large for the main subject to enter the composite region 25, the main part of the main subject of the composite image 23 may be cut out.
FIG. 7B is a conceptual diagram showing a countermeasure for the combined image 23 in which the main subject is too large. In this figure, the synthesized image 23 extracted from the rectangular designated area 22 in the first image 21 is considerably larger than the synthesized area 25 of the second image 24. Even if the shape of the synthesized image 23 is deformed in correspondence with the shape, it does not fit in the synthesis region 25. Therefore, in such a case, an appropriate range is cut out with reference to the area center point Pa of the combined image 23, the cut-out image 29 is deformed corresponding to the shape of the combined region 25, and then the second image. What is necessary is just to insert and synthesize | combine in the 24 synthetic | combination area | regions 25.

Alternatively, the following may be performed.
FIG. 7C is a conceptual diagram showing another countermeasure against the synthesized image 23 in which the main subject is too large. Also in this figure, the synthesized image 23 extracted from the rectangular designated area 22 in the first image 21 is considerably larger than the synthesized area 25 of the second image 24. Even if the shape of the synthesized image 23 is deformed in correspondence with the shape of Therefore, in such a case, the vertical or horizontal dimension (in this case, the vertical dimension) of the image to be synthesized 23 is reduced, the reduced image 30 is deformed in accordance with the shape of the synthesis region 25, and then the second image. What is necessary is just to insert and synthesize | combine in the 24 synthetic | combination area | regions 25.

  Or in the said embodiment, the image (henceforth the synthesized image 23) contained in the rectangular-shaped designation | designated area | region 22 in the 1st image 21 is extracted (step S31 and (a) of FIG. 6). For this extraction, for example, although an image contour extraction technique can be applied, depending on the content of the image, a plurality of rectangular regions may be extracted. Inconvenience to extraction.

  FIG. 8 is a conceptual diagram showing a countermeasure when a plurality of rectangular areas have been extracted. In this figure, a plurality of rectangular areas 31 to 34 extracted from the first image 21 are “framed” and displayed on the monitor screen of the display unit 5. The user selects a desired area from the rectangular areas 31 to 34. For example, if the lower left rectangular area 33 indicated by a thick frame is selected, the rectangular area 33 becomes the above-described rectangular designated area 22, and thereafter, an image included in the rectangular designated area 22 is a synthesized image. 23 is extracted.

Or it may be as follows.
FIG. 9 is a conceptual diagram of adjustment of the aspect ratio. First, the illustrated example assumes that the entire image 103 is fitted into the trapezoidal region 101 of the image 110. Here, it is assumed that the image 103 is obtained by cutting out a predetermined region in the image 120 into a rectangle.

An example of processing in that case is shown below.
First, trapezoidal correction of the area 101 is performed to create a rectangular area 102.
At this time, the affine parameter A when the keystone correction is performed is stored in the RAM 14.

  Then, the aspect ratio of the area 102 and the image 103 is compared, and the image 103 is processed so that the aspect ratio of the area 102 is obtained. Specifically, when the horizontal dimension of the region 102 is a and the vertical dimension is b, the horizontal dimension of the image 103 is a ′ and the vertical dimension is b ′, a / a ′ <= b / b. In the case of ′, the aspect ratio of the image 103 is wider than the aspect ratio of the region 102. Therefore, the portion that protrudes from the aspect ratio of the region 102 is cut out equally to the left and right, and the cut image is multiplied by b / b ′. Thus, an image 105 that exactly matches the region 102 is generated, and this image 105 is set as an image 104.

  In addition, when a / a ′> b / b ′, the aspect ratio of the image 103 is vertically wide with respect to the aspect ratio of the region 102. By multiplying the clipped image by a / a ′, an image 106 that exactly matches the region 102 is generated, and this image 106 is set as an image 104.

  In other words, the image 104 is equivalent to an image 104 ′ similar to the image 103 that is cut out and applied to the region 102.

  Here, zooming is performed after the image is cut out, but the image may be cut out after zooming.

  In the above description, the left and right or upper and lower areas are cut out equally. However, this is not necessary. The main subject is detected, and the image is cut out based on the position of the main subject. You may let them. For example, the center position of the main subject may be calculated, and an aspect ratio portion equivalent to that of the region 102 may be formed from the calculated center position while leaving an image left and right or vertically evenly, and the remaining portions may be deleted. .

  This time, the image 104 created as described above is subjected to inverse keystone correction using the affine parameter A, which is an affine parameter when the region 101 is converted to the region 102, and the shape of the image 104 is changed to the image 104. It is deformed to have the shape of 107. That is, the image 107 has a shape just equal to the area 101.

  In this way, an image 110 ′ obtained by combining the image 107 with the area 101 portion of the image 110 can be combined.

  Thereby, since the image 103 is rectangular, the following merits can be obtained. When the region 101 selected from the image 110 is, for example, a region that becomes a television screen frame when a television having a rectangular parallelepiped display region is photographed from an oblique direction, the above conversion method is used. The image 107 is expressed as if it looks like the image 104 in the image 110 if the television is viewed from the front.

  That is, in the image 110, the image 107 is expressed as if it is an image that is naturally displayed on the television.

  As described above, in the above image composition method, the region 101 is a region that is not represented as a rectangle in the image 110 because the region that was originally a rectangle in the image 110 was captured obliquely. In addition, a desired image having a rectangular shape (for example, the image 103) is appropriately processed so as to be displayed in a correct aspect ratio when the region 101 is photographed from the front, and is combined with the image 110. Make it possible.

  Therefore, since the image 104 and one of the partial images of the image 103 have a similar relationship, when the region 101 is rectangular when taken from a certain direction, the image 103 is included in the image 110. Can be embedded in the region 101 more naturally.

  In the embodiment according to FIG. 9 described above, the region 101 in the image 110 is trapezoidal. However, the present invention is not limited to this and may be as follows.

  FIG. 10 is another conceptual diagram of image deformation. In this figure, a trapezoid obtained by setting a region 101 ′ that includes a region 101 and has a shape similar to the region 101 in the image 110, cuts out the image 110 inside the region 101 ′, and performs trapezoid correction. The correction image 130 is acquired, and the region 131 corresponding to the region 101 in the trapezoid correction image 130 (the region 101 itself is also trapezoidally corrected along with the trapezoid correction of the region 101 ′. 9 is acquired, the trapezoid corrected image 130 obtained here is set as the image 110, the area 131 is reset as the area 101, and the image conversion shown in FIG. 9 is performed. I do not care.

  Similarly, this postcard can be applied to the embodiment according to FIG.

  FIG. 11 is still another image deformation conceptual diagram. In this figure, in the embodiment according to FIG. 6, the region 25 in the image 24 has a trapezoidal shape. However, the region in the image 24 includes the region 25 and is similar in shape to the region 25. 25 ′ is set, and the image 24 inside the region 25 ′ is cut out to obtain the trapezoid correction image 140 in which the trapezoid correction is performed, and the region 141 corresponding to the region 25 in the trapezoid correction image 140 is obtained. (According to the trapezoidal correction of the region 25 ′, the region 25 itself is also trapezoidally corrected. In other words, the region 141 is a rectangle.) The image conversion according to FIG. 6 may be performed after the area 141 is reset as the area 25.

  This makes it possible to obtain images with different textures.

  This eliminates the need for, for example, the trapezoidal correction from the region 101 to the region 102 and the accompanying reverse trapezoidal correction from the image 104 to the image 107, thereby simplifying the processing.

1 is a conceptual configuration diagram of an imaging apparatus according to an embodiment of the present invention. It is a figure which shows the simplified flow of the control program performed with CPU13 of the control part 12. FIG. It is a figure which shows the simple flow of an imaging process. It is a figure which shows the simplification flow of a reproduction | regeneration process. It is a figure which shows the simplification flow of the compositing process common to an imaging | photography process and a reproduction | regeneration process. It is a conceptual diagram of a synthetic | combination process. It is a figure which shows the modification of embodiment. It is a conceptual diagram which shows the countermeasure when a several rectangular area has been extracted. It is a conceptual diagram of adjustment of an aspect ratio. It is another image deformation | transformation conceptual diagram. It is another image deformation | transformation conceptual diagram.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 3 Imaging part (imaging means)
6 Storage unit (storage means)
12 control unit (first image acquisition means, second image acquisition means, third image acquisition means, first area acquisition means, second area acquisition means, third area acquisition means, fourth area (Acquisition means, conversion means, synthesis means)
21 First image 22 Rectangle-shaped designated area (first square area)
24 Second image 25 Composite area (second square area)
26 Horizontal trapezoidal image (converted image)
29 Cutout image 30 Reduced image

Claims (6)

Imaging means for capturing an image of a subject and acquiring a captured image;
First image acquisition means for acquiring the captured image as a first image;
Second image acquisition means for acquiring a second image;
First area acquisition means for acquiring a first square area that is a rectangle on the first image plane;
Second area acquisition means for acquiring a second square area on the second image plane;
Conversion means for converting a shape of the first image in the first square region, which is a rectangular region, so as to be applied based on the shape of the second square region, and obtaining a converted image;
Combining means for combining the image in the second square area in the second image with the converted image;
With
The conversion means is configured so that an image obtained when the converted image generated by the conversion means is trapezoidally corrected is similar to one of the images cut out of a part of the first image. An image pickup apparatus that performs image conversion. Imaging means for capturing an image of a subject and acquiring a captured image;
First image acquisition means for acquiring the captured image as a first image;
Second image acquisition means for acquiring a second image;
A first region acquisition means for acquiring a first square region on the first image plane,
Second area acquisition means for acquiring a second square area that is a rectangle on the second image plane;
Conversion means for converting a second image in the second square area, which is a rectangular area, to obtain a converted image by applying shape conversion so as to be applied based on the shape of the first square area;
Combining means for combining the image in the first square region in the first image and the converted image;
With
The converting means is configured so that an image obtained when the converted image generated by the converting means is trapezoidally corrected is similar to one of the images cut out from a part of the second image. An image pickup apparatus that performs image conversion. Display means for sequentially displaying the captured image obtained by sequentially captured as a through image by the image pickup means,
A first image in which the third image is sequentially generated by the synthesizing unit, with the photographed image obtained by sequential imaging by the imaging unit as the first image, the image stored in advance as the second image, Synthesis control means;
Further comprising
The display means sequentially displays the third image sequentially generated by the first composition control means as a through image instead of the captured image when the image composition mode is set. The imaging apparatus according to claim 1 or 2 . A recording unit that records a captured image obtained by imaging by the imaging unit according to a shooting instruction;
The synthesized image is generated by the synthesizing unit using the captured image obtained by imaging by the imaging unit in response to the imaging instruction as the first image and the image stored in advance as the second image. Second synthesis control means;
Further comprising
The recording means is configured to cause the recording means to record the third image generated by the second composition control means in place of the photographed image when the image composition mode is set. The imaging device according to claim 3 . A computer of an imaging apparatus including an imaging unit that captures an image of a subject and acquires a captured image,
First image acquisition means for acquiring the captured image as a first image;
Second image acquisition means for acquiring a second image;
First area acquisition means for acquiring a first square area that is a rectangle on the first image plane;
Second area acquisition means for acquiring a second square area on the second image plane;
Conversion means for converting a shape of the first image in the first square region, which is a rectangular region, so as to be applied based on the shape of the second square region, and obtaining a converted image;
Combining means for combining the image in the second square area in the second image with the converted image;
A program that functions as
The conversion means is configured so that an image obtained when the converted image generated by the conversion means is trapezoidally corrected is similar to one of the images cut out of a part of the first image. A program characterized by converting an image. A computer of an imaging apparatus including an imaging unit that captures an image of a subject and acquires a captured image,
First image acquisition means for acquiring the captured image as a first image;
Second image acquisition means for acquiring a second image;
A first region acquisition means for acquiring a first square region on the first image plane,
Second area acquisition means for acquiring a second square area that is a rectangle on the second image plane;
Conversion means for converting a second image in the second square area, which is a rectangular area, to obtain a converted image by applying shape conversion so as to be applied based on the shape of the first square area;
Combining means for combining the image in the first square region in the first image and the converted image;
A program that functions as
The converting means is configured so that an image obtained when the converted image generated by the converting means is trapezoidally corrected is similar to one of the images cut out from a part of the second image. A program characterized by converting an image.

Images