JP4379363B2 - Image tilt correction with angle of view maintained - Google Patents

Description

  The present invention relates to image inclination correction, and more particularly to a technique for performing inclination correction while suppressing a reduction in the angle of view of an image.

  When an image is captured by a digital still camera (hereinafter referred to as “DSC”), if the DSC itself is tilted at the time of imaging, an image expressed by tilting the subject is generated. In recent years, the DSC tends to be inclined at the time of imaging with the downsizing and weight reduction of the DSC.

  A technique is disclosed in which the tilt of a DSC at the time of imaging is detected and the tilt of the captured image is corrected using the detected tilt of the DSC (for example, Patent Document 1).

JP 2004-343476 A JP-A-5-150318

  In the above-described conventional technology, a corrected image is generated by cutting out an image of a rectangular area tilted so as to correct the tilt of the DSC from the captured image. For this reason, the image tilt correction in the conventional technique has a problem that the angle of view of the image after correction is narrower than the angle of view of the image before correction. If the angle of view of the image after correction is narrowed, the composition of the image after correction may be significantly different from the composition desired by the user at the time of imaging.

  Such a problem is not limited to the case of correcting the tilt of the image captured by the DSC, but is a problem common to the case of correcting the tilt of the image captured by the image generation apparatus.

  The present invention has been made to solve the above-described conventional problems, and provides a technique capable of correcting the inclination of an image while suppressing a narrowing of the angle of view of the image assumed by the user. The purpose is to do.

In order to solve at least a part of the above problems, a first image generation device of the present invention includes:
A display for displaying the subject to show the composition of the image to the user;
An imaging unit that generates an image using an imaging element;
An inclination amount detection unit that detects an inclination amount during imaging that represents an amount of inclination along the vertical plane of the image generation device at the time of imaging by the imaging unit;
A correction processing unit that performs image inclination correction,
The image pickup unit is an area on the image pickup element, and is configured by a display corresponding region corresponding to the range of the subject displayed on the display unit and an additional region added to the outer periphery of the display corresponding region. Generate an image corresponding to the generation area as the original image,
The correction processing unit uses the image corresponding to the display corresponding area and the image corresponding to the additional area in the original image based on the amount of inclination at the time of imaging to affect the influence of the inclination of the image generation apparatus. A corrected image that is an image corrected to be removed is generated.

  In this image generation device, an image generation area that is an area on the image sensor and includes a display corresponding area corresponding to the range of the subject displayed on the display unit and an additional area added to the outer periphery of the display corresponding area Is generated as an original image. Therefore, when generating a corrected image corrected so as to remove the influence of the tilt of the image generation apparatus, an image corresponding to the additional area can be used in addition to the image corresponding to the display corresponding area in the original image. . Therefore, in this image generation apparatus, it is possible to correct the inclination of the image while suppressing the narrowing of the angle of view of the image assumed by the user.

  In the image generation apparatus, the correction processing unit rotates the display corresponding area on the image sensor about the predetermined rotation center point in the direction opposite to the inclination direction of the image generation apparatus by the amount of inclination during imaging. An image corresponding to the corrected image region that is a region may be generated as the corrected image.

  In this way, the corrected image can be an image representing a subject in a range having the same size as the range of the subject displayed on the display unit. Therefore, the inclination of the image can be corrected while maintaining the angle of view of the image assumed by the user.

In the above image generation device,
An image generation area setting unit that variably sets the image generation area on the image sensor so as to include the correction image area according to the amount of tilt at the time of imaging may be provided.

  In this way, the image generation area of a necessary size can be set based on the amount of tilt at the time of imaging, and resources such as an imaging element and a storage area can be effectively used.

In the above image generation device,
A zoom lens,
A lens driving unit for driving the zoom lens,
When the set image generation area does not fit on the image sensor, the lens driving unit can be set so that the image generation area setting section can fit the image generation area on the image sensor. In addition, the zoom lens may be driven.

  In this way, when the set image generation area does not fit on the image sensor, the zoom lens is driven so that the image generation area setting unit can set the image generation area to fit on the image sensor. Is done. Therefore, after the zoom lens is driven, the image generation area can be set so as to fit on the image sensor. Therefore, it is possible to correct the inclination of the image while suppressing the narrowing of the angle of view of the image assumed by the user.

In the above image generation device,
A focus position acquisition unit that acquires a focus position at the time of imaging in the original image;
The correction processing unit may set the predetermined rotation center point to a point on the image sensor corresponding to the focus position.

  In this way, the corrected image can be made closer to the composition intended by the user.

In the above image generation device,
A subject position detector that detects the position of a predetermined type of subject in the original image;
The correction processing unit may set the predetermined rotation center point to a point on the image sensor corresponding to the position of the predetermined type of subject.

  Even in this case, the corrected image can be made closer to the composition intended by the user.

Further, the image generation apparatus includes a first image generation mode for performing image inclination correction, and a second image generation mode for performing no image inclination correction,
When the first image generation mode is set, the display corresponding area on the image sensor is set as a smaller area than when the second image generation mode is set. It is good.

  In this way, it is possible to use different imaging modes depending on whether the inclination of the image is suppressed while suppressing the narrowing of the angle of view of the image assumed by the user, or when imaging is performed using the imaging element effectively.

In order to solve at least a part of the above problems, the second image generation apparatus of the present invention provides:
A display for displaying the subject to show the composition of the image to the user;
An imaging unit that generates an image using an imaging element,
The image pickup unit is an area on the image pickup element, and is configured by a display corresponding region corresponding to the range of the subject displayed on the display unit and an additional region added to the outer periphery of the display corresponding region. An image corresponding to the generation area is generated as an original image, and an image file including data representing the original image and data representing the position of the area on the original image corresponding to the display corresponding area is generated.

  This image generation apparatus can generate an image file including data representing the original image and data representing the position of the area on the original image corresponding to the display corresponding area. Using this image file, in addition to the image corresponding to the display-corresponding area in the original image, the image corresponding to the additional area is also used to generate a corrected image corrected so as to remove the influence of the tilt of the image generating device. can do. Therefore, in this image generation device, it is possible to generate an image file for correcting the inclination of the image while suppressing the angle of view of the image assumed by the user from being narrowed.

  Note that the present invention can be realized in various modes. For example, an image generation method and apparatus, an image processing method and apparatus, an image correction method and apparatus, and a function of these methods or apparatuses are realized. The present invention can be realized in the form of a computer program, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
B. Second embodiment:
C. Third embodiment:
D. Fourth embodiment:
E. Variations:

A. First embodiment:
FIG. 1 is an explanatory diagram schematically showing the configuration of a digital still camera 100 as a first embodiment of the present invention. The digital still camera (hereinafter referred to as “DSC”) 100 according to the first embodiment includes a lens 102, a lens driving unit 104 that drives the lens 102 to adjust the focal length, and a lens driving control that controls the lens driving unit 104. A circuit 106, an image sensor 108 that converts light received on the light receiving surface through the lens 102 into an electrical signal, an A / D converter 110 that converts the electrical signal output from the image sensor 108 into digital data, an external device, An interface unit (I / F unit) 112 for exchanging information, a display unit 114 for displaying images and various setting information, an operation unit 116 for user operation, a CPU 118 for controlling each unit of the DSC 100, An internal memory 200 and an angle sensor 120 are provided. Each component of the DSC 100 is connected to each other via a bus 122.

  In the present embodiment, a CCD (Chaged Coupled Device) is used as the image sensor 108, and a biaxial acceleration sensor is used as the angle sensor 120.

  In this embodiment, a liquid crystal monitor is used as the display unit 114. At the time of imaging by the DSC 100, an image of the subject is displayed on a liquid crystal monitor as the display unit 114. The user determines the composition of the image with reference to the subject displayed on the display unit 114. Note that the display unit 114 only needs to display a subject in order to show the composition of the image to the user, and other devices such as a viewfinder may be used as the display unit 114. Further, in this specification, “subject” means an imaging target such as a person, an object, or a landscape that is captured over the entire image.

  The internal memory 200 stores a computer program that functions as the image generation processing unit 210. The image generation processing unit 210 includes functions as an inclination amount detection unit 212, a correction processing unit 214, and a mode selection reception unit 216. The CPU 118 executes an image generation process for generating an image by reading out and executing a computer program as the image generation processing unit 210 from the internal memory 200.

  Here, the DSC 100 according to the present embodiment has, as an imaging mode, an inclination correction mode in which image inclination correction is performed and a non-inclination correction mode in which image inclination correction is not performed. The imaging mode is set when the mode selection receiving unit 216 receives a mode selection instruction from the user via the operation unit 116. The CPU 118 performs image generation processing according to the set imaging mode.

  When the non-tilt correction mode is set as the imaging mode, normal image generation processing that does not perform image tilt correction is executed. Specifically, light projected on the image sensor 108 through the lens 102 is converted into an electrical signal, further converted into digital data by the A / D converter 110, and then image data generated through development processing and encoding is obtained. It is stored in a predetermined area of the internal memory 200. At this time, the generated image is an image representing the subject displayed on the display unit 114 at the time of imaging, that is, an image having a composition intended by the user. The generated image data may be stored in an external memory via the I / F unit 112.

  On the other hand, when the tilt correction mode is set as the imaging mode, an image generation process for correcting the tilt of the image is executed as described below.

  FIG. 2 is a flowchart showing the flow of image generation processing when the tilt correction mode is set as the imaging mode. The image generation processing shown in FIG. 2 is started when the mode selection receiving unit 216 (FIG. 1) receives a selection instruction for selecting a tilt correction mode by the user. In step S110, the image generation processing unit 210 (FIG. 1) performs area setting. FIG. 3 is an explanatory diagram showing an example of area setting. FIG. 3 shows the light receiving surface of the image sensor 108 (FIG. 1). FIG. 3 shows a point O that is the center point of the light receiving surface of the image sensor 108, and an X axis and a Y axis that are orthogonal coordinate axes of the light receiving surface of the image sensor 108 that passes through the point O. The X axis and the Y axis are set to be parallel to the upper and lower sides and the left and right sides of the light receiving surface of the image sensor 108, respectively. FIG. 3 also shows the subject projected on the light receiving surface of the image sensor 108 for reference.

  As shown in FIG. 3, the image generation processing unit 210 (FIG. 1) sets a display corresponding area DA and an original image generation area SA on the light receiving surface of the image sensor 108. Here, the display corresponding area DA means an area on the light receiving surface of the image sensor 108 corresponding to the range of the subject displayed on the display unit 114 (FIG. 1). That is, an image captured in the display corresponding area DA on the image sensor 108 is displayed on the display unit 114. On the other hand, the original image generation area SA means an area on the light receiving surface of the image sensor 108 used for generation of an original image SI described later. That is, the image captured in the original image generation area SA on the image sensor 108 becomes the original image SI.

  As shown in FIG. 3, the display corresponding area DA is set to an area smaller than the light receiving surface of the image sensor 108. On the other hand, the original image generation area SA is set as an area composed of a display corresponding area DA and an additional area AA (shown with hatching in FIG. 3) added to the outer periphery of the display corresponding area DA. That is, the original image generation area SA is set to an area larger than the display corresponding area DA and including the display corresponding area DA. Therefore, in the DSC 100 of the present embodiment, an image of a subject in a wider range than the range of the subject displayed on the display unit 114 is generated as the original image SI. In this embodiment, the display corresponding area DA and the original image generation area SA are set as rectangular areas whose sides are parallel to the sides of the light receiving surface of the image sensor 108. Further, the original image generation area SA is set to an almost entire area of the light receiving surface of the image sensor 108. In the following description, the width (the length in the X direction) of the display corresponding area DA is represented as Wd, and the height (the length in the Y direction) is represented as Hd. Similarly, the width of the original image generation area SA is expressed as Ws, and the height is expressed as Hs.

  In step S120 (FIG. 2), the image generation processing unit 210 (FIG. 1) receives an imaging instruction from the user. The imaging instruction is performed, for example, by pressing a release button included in the operation unit 116 (FIG. 1).

  In step S130 (FIG. 2), the tilt amount detection unit 212 (FIG. 1) detects an image capturing tilt amount α representing the tilt of the DSC 100 during image capturing. The inclination amount detection unit 212 receives the output from the biaxial acceleration sensor as the angle sensor 120 immediately after receiving the imaging instruction in step S120, and calculates the inclination of the DSC 100 along the vertical plane as the imaging inclination amount α. To do. When the DSC 100 is tilted, the image sensor 108 is similarly tilted. Therefore, it can be said that the imaging tilt amount α represents the tilt of the image sensor 108. In general, the DSC 100 is used in two basic postures, a horizontal orientation and a vertical orientation. Therefore, the imaging tilt amount α is detected as a value in a range from −45 degrees to +45 degrees with the clockwise direction being positive with respect to the basic posture. In FIG. 3, the amount of tilt α during imaging is displayed. Note that the inclination amount detection unit 212 may calculate the imaging inclination amount α using the output of the angle sensor 120 at the moment or immediately before the reception of the imaging instruction in step S120.

  In step S140 (FIG. 2), the image generation processing unit 210 (FIG. 1) generates the original image SI. FIG. 4 is an explanatory diagram showing an example of the original image SI. As described above, the original image SI is generated as an image captured in the original image generation area SA (FIG. 3). Therefore, as shown in FIG. 4, the original image SI is an image representing a subject in a wider range than the range of the subject displayed on the display unit 114 (FIG. 1). In the example of FIG. 4, the original image SI is an image in which the horizontal line is inclined under the influence of the inclination of the DSC 100 at the time of imaging. The generated original image SI is temporarily stored in a predetermined area of the internal memory 200.

  In step S150 (FIG. 2), the correction processing unit 214 (FIG. 1) generates a corrected image CI based on the original image SI and the imaging tilt amount α. The corrected image CI is an image representing a subject in the same size range as the range of the subject displayed on the display unit 114, and is an image that has been corrected so as to remove the influence of tilt when the DSC 100 is imaged. FIG. 5 is an explanatory diagram showing the concept of generation of the corrected image CI. FIG. 5 shows a light receiving surface of the image sensor 108. The corrected image CI is generated by cutting out an image corresponding to the corrected image area CA (FIG. 5) on the image sensor 108 from the original image SI (FIG. 4). Here, as shown in FIG. 5, the corrected image area CA is an area obtained by rotating the display corresponding area DA on the image sensor 108 about the point O in the direction opposite to the tilt direction of the DSC 100 by the imaging tilt amount α. It is.

The four vertices of the corrected image area CA are represented as Z1, Z2, Z3, and Z4 as shown in FIG. 5, and the coordinates on the image sensor 108 are Z1 (Xz1, Yz1), Z2 (Xz2, Yz2), and Z3. (Xz3, Yz3), Z4 (Xz4, Yz4). At this time, the coordinates of the four vertices of the corrected image area CA are expressed by the following equations (1) to (4).

  The correction processing unit 214 (FIG. 1) calculates the position of the corrected image area CA on the image sensor 108 by the above formulas (1) to (4), and calculates the position of the corrected image area CA and the original image generation area. From the relationship with the position of the SA, the position of the corrected image CI in the original image SI is calculated, and the corrected image CI is generated. FIG. 6 is an explanatory diagram illustrating an example of the generated corrected image CI. As shown in FIG. 6, the generated corrected image CI is an image in which the influence of the tilt of the DSC 100 at the time of imaging is corrected. In addition, the angle of view represented in the corrected image CI is the same as the angle of view displayed on the display unit 114. In the present embodiment, the correspondence relationship between the imaging tilt amount α and the position of the corrected image CI in the original image SI is uniquely determined. Therefore, the correspondence relationship may be calculated in advance and stored in the internal memory 200, and the correction processing unit 214 may generate the corrected image CI with reference to the stored correspondence relationship.

  In step S160 (FIG. 2), after the image generation processing unit 210 (FIG. 1) performs predetermined development processing and encoding on the corrected image CI, the data of the corrected image CI is stored in the internal memory 200 (FIG. 1). Store in a predetermined area.

  As described above, when the tilt correction mode is set, the DSC 100 according to the present embodiment generates a corrected image CI that is corrected so as to remove the influence of tilt when the DSC 100 is imaged. At this time, the original image generation area SA is set as an area larger than the display corresponding area DA and including the display corresponding area DA (see FIG. 5). Therefore, the original image SI is an image representing a subject in a wider range than the range of the subject displayed on the display unit 114 (FIG. 1). Therefore, the corrected image CI can be an image representing a subject in a range having the same size as the range of the subject displayed on the display unit 114. Therefore, the DSC 100 according to the present embodiment can correct the inclination of the image while suppressing the angle of view of the image assumed by the user from being narrowed.

  In addition, the DSC 100 of the present embodiment has two imaging modes, a tilt correction mode and a non-tilt correction mode. Therefore, in the tilt correction mode, the tilt of the image can be corrected while suppressing the narrowing of the angle of view of the image assumed by the user. In the non-tilt correction mode, the image generation using the image sensor 108 is effectively performed. It can be performed.

B. Second embodiment:
FIG. 7 is an explanatory diagram schematically showing a configuration of a digital still camera 100a as the second embodiment. The difference from the DSC 100 of the first embodiment shown in FIG. 1 is that in the DSC 100a of the second embodiment, the image generation processing section 210 includes an original image generation area setting section 217 for setting the original image generation area SA. It is a point.

  FIG. 8 is a flowchart showing the flow of image generation processing when the tilt correction mode is set as the imaging mode in the second embodiment. The difference from the image generation process of the first embodiment shown in FIG. 2 is that the setting of the original image generation area SA (step S134 of FIG. 8) in the area setting (step S110 of FIG. 2) of the first embodiment is different. This is executed after receiving the imaging instruction (step S132 in FIG. 8). In the image generation process of the second embodiment, as will be described later, the original image generation area SA is variably set according to the imaging tilt amount α.

  In step S112, the image generation processing unit 210 (FIG. 7) sets the display corresponding area DA. The method for setting the display corresponding area DA is the same as in the first embodiment.

  In step S130 (FIG. 8), the tilt amount detection unit 212 (FIG. 7) detects an image capturing tilt amount α representing the tilt of the DSC 100 during image capturing. In step S132, the image generation processing unit 210 (FIG. 7) determines whether an imaging instruction has been received. When it is determined that the imaging instruction has not yet been received, the process returns to step S130 again to detect the imaging tilt amount α. On the other hand, if it is determined that an imaging instruction has been received, the process proceeds to step S134. That is, in steps S130 and S132, the imaging tilt amount α is continuously detected until the imaging instruction is received.

  In step S134 (FIG. 8), the original image generation area setting unit 217 (FIG. 7) sets the original image generation area SA. The original image generation area SA is set based on the imaging tilt amount α.

FIG. 9 is an explanatory diagram showing an example of a method for setting the original image generation area SA. As shown in FIG. 9, in the second embodiment, the original image generation area SA is set as a minimum rectangular area including the corrected image area CA. Since the corrected image area CA is uniquely determined based on the display corresponding area DA set in step S112 and the imaging tilt amount α detected in step S130, the original image generation area SA is also uniquely determined. Specifically, the width Ws and the height Hs of the original image generation area SA are calculated by the following equations (5) to (8).

  It should be noted that the correspondence relationship between the imaging inclination amount α and the original image generation area SA is calculated in advance and stored in the internal memory 200, and the original image generation area setting unit 217 refers to the stored correspondence relation to The image generation area SA may be set.

  The processes in steps S140 to S160 (FIG. 8) are the same as those in the first embodiment shown in FIG. That is, an original image SI corresponding to the set original image generation area SA is generated, a corrected image CI is generated based on the original image SI and the imaging tilt amount α, predetermined development processing and encoding are performed, and the image Data is stored.

  As described above, when the tilt correction mode is set, the DSC 100a according to the second embodiment generates the corrected image CI that is corrected so as to remove the influence of the tilt at the time of imaging of the DSC 100a. At this time, as in the first embodiment, the original image generation area SA is set as an area larger than the display corresponding area DA and including the display corresponding area DA, so that the original image SI is displayed on the display unit 114. The image represents a subject in a wider range than the subject range. Therefore, the corrected image CI can be an image representing a subject in a range having the same size as the range of the subject displayed on the display unit 114. Therefore, the DSC 100a according to the second embodiment can correct the inclination of the image while suppressing the narrowing of the angle of view of the image assumed by the user.

  In the DSC 100a of the second embodiment, the original image generation area SA is set as a minimum rectangular area including the corrected image area CA based on the imaging tilt amount α (see FIG. 9). Therefore, it is possible to generate the original image SI in which the range of the subject necessary and sufficient for correcting the image inclination is captured. Therefore, it is possible to reduce the power consumption associated with the processing and to effectively use resources such as an image sensor and a storage area.

C. Third embodiment:
FIG. 10 is an explanatory diagram schematically showing the configuration of a digital still camera 100b as a third embodiment. The difference from the DSC 100 of the first embodiment shown in FIG. 1 is that in the DSC 100b of the third embodiment, the image generation processing unit 210 includes a focus position detection unit 218 that detects the focus position of the original image SI. Is a point.

  FIG. 11 is a flowchart illustrating the flow of image generation processing when the tilt correction mode is set as the imaging mode in the third embodiment. The difference from the image generation process of the first embodiment shown in FIG. 2 is that the focus position is detected in step S142 in the third embodiment. Further, in the generation of the corrected image CI in step S150, the center point of inclination correction is set to the focus position detected in step S142, which is different from the first embodiment.

  The processes in steps S110 to S140 (FIG. 11) are the same as those in the first embodiment shown in FIG.

  In step S142, the focus position acquisition unit 218 (FIG. 10) acquires the focus position at the time of capturing the original image SI. In general, the focus position of the DSC 100b is set to one of one or more preset points. Alternatively, the user may be able to set an arbitrary point as the focus position. The focus position acquisition unit 218 acquires the focus position set at the time of imaging as the position on the image sensor 108.

  In step S150 (FIG. 11), the correction processing unit 214 (FIG. 10) generates a corrected image CI based on the original image SI and the imaging tilt amount α. FIG. 12 is an explanatory diagram showing the concept of generation of the corrected image CI. The corrected image CI is generated by cutting out an image corresponding to the corrected image area CA on the image sensor 108 from the original image SI, as in the first embodiment. However, in the third embodiment, the corrected image area CA is set as an area obtained by rotating the display corresponding area DA on the image sensor 108 by the imaging tilt amount α around the focus position point P instead of the point O. The

When the coordinates of the focus position point P are expressed as P (Xp, Yp), the coordinates Z1 (Xz1, Yz1), Z2 (Xz2,) of the four vertices Z1, Z2, Z3, Z4 of the corrected image area CA in the third embodiment. Yz2), Z3 (Xz3, Yz3), and Z4 (Xz4, Yz4) are represented by the following formulas (9) to (12).

  Note that a plurality of focus positions may be acquired in step S142. In this case, the rotation center for obtaining the corrected image area CA is set at the position of the point O as in the first embodiment. The rotation center may be any one of the acquired plurality of focus positions, the center position of the plurality of focus positions, or the center of gravity of the plurality of focus positions.

  The correction processing unit 214 (FIG. 10) calculates the position of the corrected image area CA on the image sensor 108 by the above formulas (9) to (12), and generates the corrected image CI as in the first embodiment. To do. Thereafter, in step S160 (FIG. 11), predetermined development processing and encoding are performed on the corrected image CI, and the data of the corrected image CI is stored in a predetermined area of the internal memory 200.

  As described above, when the tilt correction mode is set, the DSC 100b according to the third embodiment generates the corrected image CI that is corrected so as to remove the influence of the tilt at the time of imaging of the DSC 100b. At this time, as in the first embodiment, the original image generation area SA is set as an area larger than the display corresponding area DA and including the display corresponding area DA, so that the original image SI is displayed on the display unit 114. The image represents a subject in a wider range than the subject range. Therefore, the corrected image CI can be an image representing a subject in a range having the same size as the range of the subject displayed on the display unit 114. Therefore, the DSC 100b according to the third embodiment can correct the inclination of the image while suppressing the narrowing of the angle of view of the image assumed by the user.

  In the DSC 100b of the third embodiment, the corrected image area CA is set as an area obtained by rotating the display corresponding area DA by the imaging tilt amount α around the focus position point P. Therefore, it is possible to generate a corrected image CI that is closer to the composition intended by the user.

D. Fourth embodiment:
FIG. 13 is a flowchart illustrating the flow of image generation processing when the tilt correction mode is set as the imaging mode in the fourth embodiment. The difference from the image generation process of the second embodiment shown in FIG. 8 is that, in the fourth embodiment, the processes of steps S136 and S138 are executed after the original image generation area setting process of step S134. . The configuration of the DSC 100 in the fourth embodiment is the same as that of the second embodiment shown in FIG.

  In step S136, the image generation processing unit 210 (FIG. 7) determines whether or not the original image generation area SA set in step S134 is within the light receiving surface of the image sensor 108 (FIG. 7). FIG. 14 is an explanatory diagram showing an example of the relationship between the original image generation area SA and the light receiving surface of the image sensor 108. When the imaging tilt amount α is large, as shown in FIG. 14, the original image generation area SA set based on the imaging tilt amount α may be a wider area than the light receiving surface of the image sensor 108. In this case, the original image SI having a size necessary for generating the corrected image CI cannot be generated. Therefore, when it is determined in step S136 that the original image generation area SA is not within the light receiving surface of the image sensor 108, the process proceeds to step S138. On the other hand, when it is determined that the original image generation area SA is within the light receiving surface of the image sensor 108, an original image SI having a size necessary for generating the corrected image CI can be generated. It progresses to the process after S140.

In step S138 (FIG. 13), the image generation processing unit 210 (FIG. 7) controls the lens drive control circuit 106 and the lens drive unit 104 to drive the lens 102, and the original image generation area SA and the display corresponding area DA. Reset the settings. Specifically, the lens 102 is wide so that the range of the subject displayed on the display unit 114 (FIG. 7) can be fixed and the original image generation area SA can be set within the light receiving surface of the image sensor 108. Driven to the side. FIG. 15 is an explanatory diagram showing the relationship between the original image generation area SA reset after driving the lens 102 and the image sensor 108.
If the range of the subject displayed on the display unit 114 is fixed, when the lens 102 is driven to the wide side, the size of the display corresponding area DA on the light receiving surface of the image sensor 108 becomes small. Therefore, when the lens 102 is driven to the wide side, the size of the corrected image area CA is also reduced, and the original image generation area SA that includes the corrected image area CA can be reset as a smaller area. The image generation processing unit 210 drives the lens 102 to the wide side until the original image generation area SA can be reset as shown in FIG. 15, and then resets the original image generation area SA and the display corresponding area DA. Set.

  If the original image generation area SA cannot be reset so as to be within the light receiving surface of the image sensor 108 even if the lens 102 is driven to the wide end, the process becomes an error, and image tilt correction is not executed. . Alternatively, the lens 102 is driven to the wide end, the original image generation area SA is reset as large as possible, and the corrected image area CA is reduced by reducing the inclination amount of the corrected image area CA with respect to the display corresponding area DA. It may be set so as to be included in the area SA, and a corrected image CI corresponding to the set corrected image area CA may be generated.

  The processing contents of steps S140 to S160 are the same as those in the second embodiment shown in FIG.

  As described above, when the tilt correction mode is set, the DSC 100 of the fourth embodiment generates the corrected image CI that is corrected so as to remove the influence of the tilt when the DSC 100 is imaged. At this time, as in the first embodiment, the original image generation area SA is set as an area larger than the display corresponding area DA and including the display corresponding area DA, so that the original image SI is displayed on the display unit 114. The image represents a subject in a wider range than the subject range. Therefore, the corrected image CI can be an image representing a subject in a range having the same size as the range of the subject displayed on the display unit 114. Therefore, the DSC 100 according to the fourth embodiment can correct the inclination of the image while suppressing the angle of view of the image assumed by the user from being narrowed.

  Further, in the DSC 100 of the fourth embodiment, if the set original image generation area SA does not fit within the light receiving surface of the image sensor 108, the original image generation area SA fits within the light receiving surface of the image sensor 108. The lens 102 is driven so as to be able to be reset to. Therefore, the degree of freedom for setting the display corresponding area DA can be improved. That is, for example, even if the display corresponding area DA is initially set to substantially the same area as the light receiving surface of the image sensor 108, the image corresponding to the initially set display corresponding area DA is driven by driving the lens 102 thereafter. The inclination of the image can be corrected while suppressing the narrowing of the angle of view. Therefore, for example, such an inclination correction can be realized by adding an angle sensor to a conventional DSC.

E. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

E1. Modification 1:
In each of the above-described embodiments, the imaging tilt amount α is detected by the angle sensor 120 (FIG. 1). However, the imaging tilt amount α may be detected from the original image SI. The detection of the amount of tilt α during imaging from the original image SI is performed using an edge detection operator that reacts in the horizontal or vertical direction to a line that is assumed to be horizontal or vertical in the original image SI. Then, the amount of inclination of the detected line in the original image SI is calculated.

  In addition, when detecting the imaging inclination amount α from the original image SI, it may be performed using the original image SI after executing a predetermined development process.

E2. Modification 2:
In the third embodiment, the rotation center for obtaining the corrected image area CA is set to the focus position point P. However, the rotation center is set to a position corresponding to the position of a predetermined type of subject in the original image SI. May be. For example, a human face or a human whole body is set as the predetermined type of subject. When a human face is set as a predetermined type of subject, a face area in the original image SI is detected using a face detection technique, and a predetermined position in the face area is rotated to obtain a corrected image area CA. It may be set at the center. As the face detection technique, known techniques such as face detection by pattern matching, face detection by a skin color region, face detection by a classifier such as SVM (Support Vector Machine), etc. can be adopted. In addition, when the whole body of a human is set as a predetermined type of subject, the whole body area is detected after the face area is detected, and a predetermined position in the whole body area is set as the rotation center for obtaining the corrected image area CA. May be.

E3. Modification 3:
In each of the embodiments described above, the tilt correction is performed during the image generation process. However, the tilt correction may be performed using the generated image after the image generation. Specifically, when the tilt correction mode is set, the DSC 100 creates an image file including data of the original image SI and information indicating the position of the area on the original image SI corresponding to the display corresponding area DA. Generated. At the time of image inclination correction, first, an imaging inclination amount α is detected from the original image SI. Thereafter, an image of a region obtained by correcting the region on the original image SI corresponding to the display corresponding region DA with the tilt amount α during imaging is cut out from the original image SI, and the corrected image CI is generated. Even in this case, it is possible to correct the inclination of the image while suppressing the narrowing of the angle of view of the image assumed by the user.

  Such an image file can be realized as a well-known Exif file. In addition, it is also possible to record an image capturing tilt amount α detected by the angle sensor 120 of the DSC 100 in the image file, and perform tilt correction using the recorded image capturing tilt amount α.

E4. Modification 4:
In each of the above embodiments, the original image generation area SA is set as a rectangular area, but it is not always necessary to set it as a rectangular area. When the original image generation area SA is set as a rectangular area, the aspect ratio of the original image generation area SA does not have to be the same value as the aspect ratio of the display corresponding area DA.

E5. Modification 5:
In each of the above embodiments, the imaging tilt amount α is detected as a value in a range from −45 degrees to +45 degrees. However, the imaging tilt amount α is limited to a smaller predetermined range, and the predetermined range. If an image-taking tilt amount α exceeding is detected, it may be assumed that the tilt is intended by the user, and the image tilt correction may not be performed.

E6. Modification 6:
In each of the above-described embodiments, the inclination correction when generating an image having a horizontally long composition has been described. However, even when an image having a vertically long composition (a so-called portrait image) is generated, the inclination correction can be similarly performed. . For example, it is determined whether the composition intended by the user is horizontally long or vertically long from the value of the detected inclination amount α during imaging. When it is determined that the composition is vertically long, the imaging tilt amount α is detected with reference to when the DSC 100 is installed vertically, and image tilt correction is performed based on the imaging tilt amount α.

E7. Modification 7:
In each of the above embodiments, the user instructs the selection of the imaging mode, but the imaging mode may be automatically performed based on the imaging tilt amount α. For example, the non-tilt correction mode is set when the amount of tilt α during imaging is small, and the tilt correction mode is set when the value of the amount of tilt α during imaging exceeds a predetermined first threshold value. The non-tilt correction mode may be set when the value is equal to or greater than a predetermined second threshold value that is greater than the first threshold value.

E8. Modification 8:
In each of the above embodiments, the corrected image area CA is set as an area obtained by rotating the display corresponding area DA. That is, although the corrected image area CA is set as an area having the same size as the display corresponding area DA, the corrected image area CA is not necessarily required to be an area having the same size as the display corresponding area DA.

E9. Modification 9:
The configuration of the DSC 100 in each of the above embodiments is merely an example, and the configuration of the DSC 100 can be other configurations. In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. Also good.

BRIEF DESCRIPTION OF THE DRAWINGS Explanatory drawing which shows schematically the structure of the digital still camera as 1st Example of this invention. The flowchart which shows the flow of an image generation process when the inclination correction mode is set as an imaging mode. Explanatory drawing which shows an example of area | region setting. Explanatory drawing which shows an example of the original image SI. Explanatory drawing which shows the concept of the production | generation of the correction | amendment image CI. Explanatory drawing which shows an example of the produced | generated correction image CI. Explanatory drawing which shows schematically the structure of the digital still camera as 2nd Example. 12 is a flowchart illustrating a flow of image generation processing when an inclination correction mode is set as an imaging mode in the second embodiment. Explanatory drawing which shows an example of the setting method of original image generation area SA. Explanatory drawing which shows schematically the structure of the digital still camera as 3rd Example. 10 is a flowchart illustrating a flow of image generation processing when an inclination correction mode is set as an imaging mode in the third embodiment. Explanatory drawing which shows the concept of the production | generation of the correction | amendment image CI. 10 is a flowchart illustrating a flow of image generation processing when an inclination correction mode is set as an imaging mode in the fourth embodiment. Explanatory drawing which shows an example of the relationship between original image generation area SA and the light-receiving surface of an image pick-up element. Explanatory drawing which shows the relationship between the original image production | generation area | region SA reset after lens drive, and an image pick-up element.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Digital still camera 102 ... Lens 104 ... Lens drive part 106 ... Lens drive control circuit 108 ... Imaging element 110 ... A / D converter 112 ... I / F part 114 ... Display unit 116 ... Operation unit 118 ... CPU
120: Angle sensor 122 ... Bus 200 ... Internal memory 210 ... Image generation processing unit 212 ... Inclination amount detection unit 214 ... Correction processing unit 216 ... Mode selection receiving unit 217. .. Original image generation area setting unit 218 ... Focus position detection unit

Claims (5)

An image generation device,
A display for displaying the subject to show the composition of the image to the user;
An imaging unit that generates an image using an imaging element;
An inclination amount detection unit that detects an inclination amount during imaging that represents an amount of inclination along the vertical plane of the image generation device at the time of imaging by the imaging unit;
A correction processing unit that performs image inclination correction,
The imaging unit is an area on the image sensor, the image composed of the outer periphery appended additional area of display corresponding region corresponding to a range of subject said that is displayed on the display unit and the display corresponding region Generate an image corresponding to the generation area as the original image,
The correction processing unit uses the image corresponding to the display corresponding area and the image corresponding to the additional area in the original image based on the amount of inclination at the time of imaging to affect the influence of the inclination of the image generation apparatus. Generate a corrected image that is corrected to be removed ,
The correction processing unit is a correction image region that is a region obtained by rotating the display corresponding region on the image sensor about the predetermined rotation center point in the direction opposite to the tilt direction of the image generation device by the amount of tilt during imaging. Is generated as the corrected image,
The image generation device further includes:
An image generation area setting unit configured to variably set the image generation area on the image sensor so as to include the correction image area in accordance with the amount of tilt during imaging;
The image generation apparatus has a first image generation mode for correcting image inclination, and a second image generation mode for not correcting image inclination,
When the first image generation mode is set, the display corresponding area on the image sensor is set as a smaller area than when the second image generation mode is set. , Image generation device. The image generation apparatus according to claim 1 , further comprising:
A zoom lens,
A lens driving unit for driving the zoom lens,
When the set image generation area does not fit on the image sensor, the lens driving unit can be set so that the image generation area setting section can fit the image generation area on the image sensor. And an image generating device for driving the zoom lens. The image generation apparatus according to claim 1 , further comprising:
A focus position acquisition unit that acquires a focus position at the time of imaging in the original image;
The correction processing unit sets the predetermined rotation center point to a point on the image sensor corresponding to the focus position. The image generation apparatus according to claim 1 , further comprising:
A subject position detector that detects the position of a predetermined type of subject in the original image;
The correction processing unit sets the predetermined rotation center point to a point on the image sensor corresponding to the position of the predetermined type of subject. An image generation method comprising:
(A) displaying a subject to show the composition of the image to the user;
(B) generating an image using an image sensor;
(C) detecting a tilt amount during imaging that represents an amount of tilt along the vertical plane of the image sensor at the time of imaging in the step (b);
(D) a step of correcting the inclination of the image,
Wherein step (b) is a region on the image sensor, between the step (a) outer periphery appended additional area of display corresponding region corresponding to the range of the object that is displayed and the display corresponding region in A step of generating an image corresponding to the configured image generation region as an original image,
In the step (d), based on the amount of inclination at the time of imaging, the influence of the inclination at the time of imaging is obtained by using both the image corresponding to the display corresponding area and the image corresponding to the additional area in the original image. Ri step der to generate a corrected image is corrected image to remove,
In the step (d), the display corresponding region on the image sensor is rotated around the predetermined rotation center point in the direction opposite to the tilt direction at the time of image capture in the step (b) by the tilt amount at the time of image capture. An image corresponding to the corrected image area is generated as the corrected image,
The image generation method further includes:
(E) comprising a step of variably setting the image generation area on the image sensor so as to include the corrected image area in accordance with the amount of tilt during imaging,
The image generation method , wherein the display corresponding region on the image sensor is set as a small region as compared to a case where the image inclination correction in the step (d) is not performed .

Images