JP2016134686A - Imaging apparatus and imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging method capable of imaging an image having been subjected to trapezoidal correction processing as intended by a user.SOLUTION: An imaging apparatus comprises: an input device; an image processing unit generating secondary image data having been subjected to deformation processing of projecting primary image data obtained from an imaging element on a plane having predetermined angles to a light reception surface of the imaging element; an image display device displaying an image based upon the secondary image data in synchronism with a frame period; and an AF control unit controlling a focusing operation of an imaging lens. The input device receives an input identifying a position of an AF region in the secondary image data displayed on the image display device. The image processing unit performs conversion processing of projecting the position of the AF region in the secondary image data on the light reception surface. The AF control unit puts the imaging lens in a focusing operation in the AF region projected on the light reception surface.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus and an imaging method for performing deformation processing on primary image data obtained from an imaging element.

  For example, when an image is taken with an imaging device such as a digital camera looking up at a building that is a subject, the building is distorted into a trapezoidal shape in the upper part of the image. For example, Japanese Patent Application Laid-Open No. 2007-43545 discloses a technique for detecting a posture of an imaging device when an object is imaged by an acceleration sensor and performing a trapezoid correction process for automatically correcting a trapezoidal distortion of the object. ing.

  In addition, in an imaging apparatus capable of performing an autofocus operation, a technique is known in which the position of an AF area that is an autofocus area is moved based on a user instruction.

JP 2007-43545 A

  As in the technique disclosed in Japanese Patent Application Laid-Open No. 2007-43545, when a trapezoidal correction process is performed on a captured image, the position of the AF area designated by the user before shooting is changed by the execution of the keystone correction process. It may move to an unintended position.

  The present invention has been made in view of the above-described points, and an object thereof is to provide an imaging apparatus and an imaging method capable of imaging an image that has been subjected to trapezoidal correction processing as intended by the user.

  An imaging device according to one embodiment of the present invention is arranged so that a light receiving surface is orthogonal to an optical axis of an imaging lens, and an imaging element that acquires primary image data every predetermined frame period, in synchronization with the frame period From a focal length detection unit that acquires a focal length of the imaging lens, a first angle that is an angle in the pitch direction with respect to the light receiving surface, and a second angle that is an angle in the yaw direction with respect to the light receiving surface An input device that accepts an input of a correction angle including two values, and using the focal length value and the correction angle value in synchronization with the frame period, the primary image data is transmitted to the light receiving surface. An image processing unit that generates secondary image data subjected to a deformation process for projecting onto a plane that forms the first angle in the pitch direction and forms the second angle in the yaw direction with respect to the light receiving surface; In sync with the cycle An image display device that displays an image based on the secondary image data; and an AF control unit that performs control to change a focusing distance of the imaging lens, and the input device displays the image on the image display device. Accepting an input designating the position of the AF area in the secondary image data, the image processing unit converts the position of the AF area in the secondary image data onto a plane including the primary image data The AF control unit executes processing so that the imaging lens is in a focused state at a position corresponding to the AF area projected on the plane including the primary image data on the light receiving surface. The focusing distance of the imaging lens is changed.

  In addition, an imaging method according to one embodiment of the present invention is arranged so that a light receiving surface is orthogonal to an optical axis of an imaging lens, and an imaging element that acquires primary image data every predetermined frame period, and is synchronized with the frame period. A focal length detection unit that acquires a focal length of the imaging lens, a first angle that is an angle formed in a pitch direction with respect to the light receiving surface, and a second angle that is formed in a yaw direction with respect to the light receiving surface. An input device that accepts an input of a correction angle including two values consisting of an angle; and the focal length value and the correction angle value in synchronization with the frame period; An image processing unit that generates secondary image data subjected to a deformation process for projecting onto a plane that forms the first angle in the pitch direction and projects the second angle in the yaw direction with respect to the light receiving surface; In the frame period An imaging method for an imaging apparatus comprising: an image display device that displays an image based on the secondary image data; and an AF control unit that performs control to change a focusing distance of the imaging lens. Receiving an input for designating the position of the AF area in the secondary image data displayed on the image display device by the apparatus; and the position of the AF area in the secondary image data by the image processing unit, A step of executing a conversion process for projecting onto a plane including primary image data, and the AF control unit corresponding to the AF area projected onto the plane including the primary image data on the light receiving surface. Changing the focusing distance of the imaging lens so that the imaging lens is in a focused state at a position to be focused.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device and imaging method which can image the image which performed the trapezoid correction process as a user intends are realizable.

It is a block diagram explaining the structure of the imaging device of 1st Embodiment. It is a perspective view of the back side of the imaging device of a 1st embodiment. 3 is a flowchart of an imaging operation of the imaging apparatus according to the first embodiment. It is a flowchart of a trapezoid correction process. It is a figure for demonstrating a trapezoid correction process. It is a figure for demonstrating a trapezoid correction process. It is a figure for demonstrating a trapezoid correction process. It is a figure for demonstrating a trapezoid correction process. It is a figure for demonstrating a trapezoid correction process. 6 is a flowchart of AF area movement processing according to the first embodiment. It is a figure which shows the example of a display of the AF area | region display icon of 1st Embodiment. It is a figure which shows the example of a display of the AF area | region display icon of 1st Embodiment. It is a flowchart of AF processing of a 1st embodiment. It is a figure which shows the example of the shape of AF area | region in the primary image data in 1st Embodiment. It is a flowchart of AF area movement processing of the second embodiment. It is a figure which shows the example of the shape of AF area | region in the primary image data in 2nd Embodiment. It is a figure which shows the example of the shape of AF area | region in secondary image data in 2nd Embodiment. It is a figure which shows the example of a display of the AF area | region display icon of 2nd Embodiment. It is a flowchart of AF processing of a 2nd embodiment.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In the drawings used for the following description, the scale of each component is made different in order to make each component recognizable on the drawing. It is not limited only to the quantity of the component described in (1), the shape of the component, the ratio of the size of the component, and the relative positional relationship of each component.

(First embodiment)
As shown in FIGS. 1 and 2, the imaging device 1 includes an imaging device 3 such as a CCD or a CMOS image sensor, an image display device 6, an input device 7, and a control unit 10. In the present embodiment, as an example, the imaging apparatus 1 includes a main body 2 that houses the imaging element 3 and the control unit 10, and an imaging lens 4 that is fixed to the main body 2, and is imaged by the imaging lens 4. It has the form of a so-called digital camera that stores the optical image as electronic data. The imaging device 1 may be a so-called interchangeable lens digital camera in which the imaging lens 4 can be removed from the main body 2, or a lens-integrated digital camera in which the imaging lens 4 cannot be removed from the main body 2. It may be a camera.

  Hereinafter, as the definition of the orientation direction of the imaging apparatus 1, the optical axis direction of the imaging lens 4 is defined as the Z axis. When the imaging apparatus 1 is in an upright state, the direction orthogonal to the optical axis and horizontal is the X axis, and the direction orthogonal to and perpendicular to the optical axis is the Y axis. Further, in order to represent the direction of change in the posture of the imaging device 1, the rotation direction around the X axis of the imaging device 1 is referred to as the pitch direction, the rotation direction around the Y axis is referred to as the yaw direction, and the rotation direction around the Z axis is referred to as the rotation direction. Called the roll direction.

  The image sensor 3 has a rectangular light receiving surface. The light receiving surface of the image sensor 3 is disposed so as to be orthogonal to the optical axis of the image pickup lens 4. The imaging lens 4 can perform an autofocus operation, and includes an AF mechanism unit 4a that moves some or all of the lenses to change the focusing distance. In the present embodiment, as an example, the imaging lens 4 is a so-called zoom lens that can change the focal length. The zoom operation of the imaging lens 4 may be in the form of manual zoom performed by the force input from the user, or the form of electric zoom performed by the force generated by the electric motor incorporated in the imaging lens 4. It may be.

  The imaging lens 4 of the present embodiment includes a focal length detection unit 4 b that outputs information on the current focal length of the imaging lens 4 to the control unit 10. The configuration of the focal length detection unit 4b is a known technique such as a configuration in which a focal length is detected using a rotary encoder, or a number of pulses for operating a stepping motor for zoom operation is counted. For example, when the imaging device 1 is a lens-integrated type, the focal length detection unit 4b may be included in the control unit 10.

  The imaging device 1 may include a lens shutter mechanism or a focal plane shutter mechanism. The imaging device 1 may include an aperture mechanism in the imaging lens 4.

  The image display device 6 includes, for example, a liquid crystal display device or an organic EL display device, and displays an image. The outer shape of the display surface of the image display device 6 of this embodiment is a rectangle. The image display device 6 displays a graphical user interface (GUI) of the imaging device 1, displays a live view that functions as a finder during an imaging operation, displays recorded image data, and the like.

  In the present embodiment, as an example, the image display device 6 is disposed in the main body 2. The image display device 6 may be separated from the main body 2 and may be disposed in another electronic device that is connected to the main body 2 by wired communication or wireless communication.

  The input device 7 includes one or a plurality of operation members such as a lever switch, a dial switch, a button switch, and a touch sensor, for the user to input an operation instruction of the imaging device 1.

  In the present embodiment, as an example, the input device 7 includes a power operation switch 7a, a release switch 7b, a four-way switch 7c, a dial switch 7d, a touch panel 7e, and a two-way switch 7f. The touch panel 7 e is disposed on the display surface of the image display device 15. Note that a part or all of the input device 7 may be separated from the main body 2 and disposed in another electronic device connected to the main body 2 by wired communication or wireless communication. .

  The release switch 7b is a so-called two-stage push button switch including a first release switch and a second release switch that operate with different push amounts or different push forces. In the present embodiment, when the pushing force applied to the release switch 7b is increased, the first release switch is turned on first, and then the second release switch is turned on. The state where only the first release switch is in the ON state is a so-called half-pressed state, and the state where the second release switch is in the ON state is a so-called fully-pressed state.

  The control unit 10 includes a CPU (arithmetic device) 11, a RAM (storage device) 12, a flash memory (auxiliary storage device) 13, an image processing unit 16, an input / output device, a power control device, and the like. It has the structure which controls the operation | movement mentioned later of the imaging device 1 based on a predetermined program. In the present embodiment, the control program for the imaging apparatus 1 is stored in the flash memory 13 or the like that is a nonvolatile storage medium.

  The control unit 10 is electrically connected to the image sensor driving unit 8. The image sensor driving unit 8 drives the image sensor 3 under the control of the control unit 10. The image sensor driving unit 8 converts the two-dimensional image signal output from the image sensor 3 into primary image data that is a digital signal. The outer shape of the image generated based on the primary image data is rectangular. Hereinafter, an image generated based on the primary image is simply referred to as primary image data. Regarding the coordinates in the primary image data, when the imaging device 1 is held in an upright state, the axis along the vertical direction in the visual field is the y axis, and the axis along the horizontal direction in the visual field is the x axis. In the present embodiment, as an example, the image sensor 3 and the image sensor drive unit 8 are shown as separate bodies, but the image sensor drive unit 8 may be integrally included in the image sensor 3.

  The RAM 12 has a plurality of storage areas for a control program 12a, an image data buffer 12b, and a log information buffer 12c. The control program 12a is a storage area for storing a control program read from the flash memory 13 when the imaging apparatus 1 is powered on.

  The image data buffer 12b is a storage area for storing primary image data output from the image sensor driving unit 8 and secondary image data generated by the image processing unit 16 described later. Hereinafter, the data stored in the image data buffer 12b is simply referred to as image data when it is not necessary to distinguish between primary image data and secondary image data.

  The image data buffer 12b may be a storage area provided in a video memory dedicated to image data handled by the image processing unit 16. The log information buffer 12c is a storage area for storing variables used for execution of trapezoidal distortion correction processing to be described later, AF position designation coordinates, and the like.

  The AF control unit 14 detects the imaging state of the imaging lens 4 within a predetermined region of the light receiving surface of the imaging device 3 and controls the autofocus operation by the imaging device 1. More specifically, the AF control unit 14 controls the AF mechanism unit 4a to change the in-focus distance while detecting the contrast in the AF region which is a predetermined region in the primary image data. Set to the in-focus state. That is, the imaging apparatus 1 of the present embodiment can perform an autofocus operation generally called a contrast detection method. Note that the form of the autofocus operation of the imaging apparatus 1 is not limited to the above-described contrast detection method. For example, the imaging apparatus 1 includes an imaging element 3 in which a distance measuring sensor that detects a phase difference of a subject image is disposed on an imaging surface, and performs imaging based on phase difference information (defocus amount) detected by the distance measuring sensor. An autofocus operation generally called a phase difference detection method that drives the lens 4 to bring it into focus may be possible.

  In this embodiment, the coordinates of the AF area in the primary image data can be changed according to the operation of the input device 7 by the user. In this embodiment, as an example, the change of the coordinates of the AF area in the primary image data is performed by the user operating the input device 7 such as the touch panel 7e or the four-way switch 7c.

  The coordinates and size of the AF area in the primary image data may be changed automatically by the control unit 10 based on a predetermined program.

  The AE control unit 15 controls the exposure amount for the primary image data acquired from the image sensor 3. The AE control unit 15 calculates an exposure value based on the subject brightness obtained from the primary image data.

  The image processing unit 16 performs predetermined image processing on the image data stored in the image data buffer 12b. Hereinafter, the image data that has been subjected to image processing by the image processing unit 16 is referred to as secondary image data. The image processing unit 16 of the present embodiment includes a trapezoidal distortion correction coefficient calculation unit 16a and a correction processing unit 16b for performing trapezoidal correction described later on the image data.

  The trapezoidal correction coefficient calculation unit 16a uses the value of the focal length of the imaging lens 4 obtained from the focal length detection unit 4b and the value of the correction angle (described later) input via the input device 7 to calculate the keystone correction. A correction coefficient necessary for executing the process is calculated. The correction processing unit 16b performs image interpolation processing with coordinate conversion on the primary image data based on the correction coefficient calculated by the trapezoidal correction coefficient calculation unit 16a. The image interpolation processing performs image interpolation processing such as bicubic interpolation and bilinear interpolation.

  In the present embodiment, as an example, the functional configurations of the AF control unit 14, the AE control unit 15, and the image processing unit 16 described above are included in the control unit 10. Note that the functional configurations of the AF control unit 14, the AE control unit 15, and the image processing unit 16 may be, for example, a form using dedicated hardware such as a dedicated processor circuit that executes each function.

  In addition, the imaging apparatus 1 of the present embodiment includes a power connection unit 21 connected to a commercial power source, a power source 20 such as an AC adapter or a battery, and a storage medium connection unit 23 connected to a storage medium 22 such as a flash memory card. Prepare. In the present embodiment, as an example, the power source 20 is a battery and is detachably accommodated in the main body 2. In the present embodiment, as an example, the storage medium 22 is a flash memory card and is detachably accommodated in the main body 2. Note that the battery and the storage medium 22 serving as the power source 20 may be fixed in the main body 2. The storage medium 22 may be arranged in another electronic device that is separated from the main body 2 and connected to the main body 2 by wired communication or wireless communication.

  Next, the trapezoid correction process executed in the image processing unit 16 will be described with reference to the flowchart shown in FIG. The trapezoidal correction process is generally a virtual that directly faces a subject by performing geometric deformation processing for correcting trapezoidal distortion on primary image data captured at a position not facing the subject. This is a process of generating secondary image data as if it were captured from a specific viewpoint.

  In the following, in order to simplify the description of the geometric deformation process for correcting the trapezoidal distortion, the orientation of the imaging device 1 is in the pitch direction (in the vertical direction and the inclination angle is relative to the position facing the subject. The case where it arrange | positions in the position which makes | forms a predetermined angle in the direction which arises is demonstrated.

  For example, as shown in FIG. 5 and FIG. 6, when the front surface 40 a that is a rectangle of the building 40 is imaged by the imaging device 1 so as to look up from the actual viewpoint P close to the ground, as shown in FIG. On the image data 50, the front surface 40a has a trapezoidally distorted shape. At this time, in imaging from the actual viewpoint P, the imaging apparatus 1 is in a posture in which the optical axis of the imaging lens 4 is directed upward by a first angle αy from the horizontal to the pitch direction. The reason why the front surface 40a is distorted on the primary image data 50 is that the photographing magnification changes in the vertical direction as the distance from the imaging device 1 to the front surface 40a increases as the distance increases from bottom to top.

  On the other hand, when the imaging apparatus 1 is arranged and imaged at a virtual viewpoint P ′ facing the front surface 40a of the building 40, the front surface 40a has a rectangular shape on the primary image data 50 as shown in FIG. .

  In the trapezoidal correction process, the subject image in the primary image data 50 taken from the real viewpoint P has a shape that approximates the subject image taken from the virtual viewpoint P ′, which is a virtual viewpoint, as shown in FIG. The primary image data 50 is transformed into secondary image data 50 ′. In other words, the primary image data 50 on the plane A parallel to the light receiving surface of the image sensor 3 is projected onto the plane B inclined by the first angle αy in the pitch direction with respect to the plane A. The applied secondary image data 50 '.

  Specifically, in the trapezoidal correction process, first, in step S31, the value of the focal length f of the imaging lens 4 is acquired from the focal length detection unit 4b and stored in the log information buffer 12c of the RAM 12.

  Next, in step S32, the correction angle is acquired and stored in the log information buffer 12c of the RAM 12. Here, as described above, the correction angle is a first angle αy that is an angle formed in the pitch direction with respect to the light receiving surface of the image sensor 3 and an angle that is formed in the yaw direction with respect to the light receiving surface of the image sensor 3. It includes two angle values with a certain second angle αx.

  The value of the correction angle is determined by the user operating the input device 7. For example, the value of the first angle αy increases or decreases when the user operates the four-way switch 7c in the vertical direction, and the value of the second angle αx increases or decreases when the user operates the four-way switch 7c in the horizontal direction. To do. Further, for example, the values of the first angle αy and the second angle αx may be increased or decreased according to the direction of the drag operation that the user traces on the touch panel 7e. The value of the correction angle may be continuously changed or may be changed stepwise.

  Next, in step S33, the primary image data 50 is subjected to trapezoidal correction processing using the focal length f value and the correction angle value stored in the log information buffer 12c to generate secondary image data 50 ′. To do.

  The following mathematical formula (1) is expressed by the coordinates (x, y) of the primary image data 50 of the plane A and the coordinates (x of the secondary image data 50 ′ of the plane B inclined by the angle α with respect to the plane A in the pitch direction ', Y').

[Equation 1]
y = {H · k1 · y ′} / {H−k2 · y ′}
x = {H · x ′} / {H−k2 · y ′}
Here, k1 and k2 are conversion coefficients,
k1 = cos α−sin α · tan β
k2 = H-2sin α · tan β
It is.

  H is the number of pixels on a predetermined side of the effective pixel region of the image sensor 3 corresponding to the correction angle input by the user via the input device 7. For example, as shown in FIG. 7, when photographing with the long side of the effective pixel region of the image sensor 3 being horizontal and correcting the trapezoidal distortion in the vertical direction of the visual field, H is the effective pixel region of the image sensor 3. The number of pixels in the short side direction. Specifically, the number of pixels corresponding to H is obtained by dividing the length (pixel pitch) per pixel in the image sensor 3 with respect to the length in the short side direction of the effective pixel region of the image sensor 3. Result. Although not shown, when the image is taken with the long side of the effective pixel area of the image sensor 3 horizontal, and when correcting the trapezoidal distortion in the horizontal direction of the visual field, H is the long side of the effective pixel area of the image sensor 3 The number of pixels in the direction. In this case, the number of pixels corresponding to H is obtained by dividing the length (pixel pitch) per pixel in the image sensor 3 with respect to the length in the long side direction of the effective pixel region of the image sensor 3. Become.

  Further, the angle β is calculated by the following mathematical formula (2).

[Equation 2]
β = arctan {L / (2 · f)}
L is the length of a predetermined side of the effective pixel area of the image sensor 3 corresponding to the correction angle input by the user via the input device 7 (unit is indicated in mm). For example, as shown in FIG. 7, when photographing with the long side of the effective pixel area of the image sensor 3 being horizontal and correcting the trapezoidal distortion in the vertical direction of the field of view, L is the effective pixel area of the image sensor 3. Is the length in the short side direction. Although not shown, when taking a picture with the long side of the effective pixel area of the image sensor 3 being horizontal and correcting the trapezoidal distortion in the horizontal direction of the visual field, L is the long side of the effective pixel area of the image sensor 3 The length of the direction. f is a focal length of the imaging lens 4 (unit is shown in mm). That is, the angle β is a half value of the angle of view in the direction in which the trapezoidal distortion of the imaging apparatus 1 is performed.

  The value of the focal length f may be a value input via the input device 7 by the user. For example, when the imaging device 1 is of the interchangeable lens type and the imaging lens 4 that cannot acquire focal length information is attached, the user inputs the value of the focal length f, so that the trapezoid desired by the user is obtained. Correction can be performed. In addition, by enabling the user to input the value of the focal length f, it is possible to perform keystone correction that produces a sense of perspective according to the user's intention. When the user can input the value of the focal length f, the user selects whether to perform the keystone correction using the value acquired from the imaging lens 4 or the value input by the user. Means may be provided.

  In step S33, trapezoidal correction processing in the short side direction is performed on the primary image data 50 by using the above formulas (1) and (2) to generate secondary image data 50 '. The coordinates (x ′, y ′) of the secondary image data after the keystone correction are calculated by back calculating the relational expression with the coordinates (x, y) of the primary image data 50 indicated by the above formula (1). The More specifically, in step S33, using the above equations (1) and (2), the coordinates (x ', y') of the primary image data 50 with respect to the coordinates (x ', y') after the keystone correction are performed. , Y) is defined. Next, the relational expression based on the defined mathematical formula (1) is reversely calculated, and the coordinates (x ′, y ′) of the secondary image data 50 ′ are calculated from the coordinates (x, y) of the input primary image data 50. By obtaining the trapezoidal correction process in the short side direction (y direction in the example of FIG. 7) of the primary image data 50 according to the correction angle input in the pitch direction, the primary image data 50 shown as an example in FIG. To generate secondary image data 50 'as shown in FIG.

  In the above illustration, the trapezoidal correction process has been described for the action performed in the vertical direction (pitch direction) in the field of view. Note that the trapezoidal correction process is similarly performed in the horizontal direction (yaw direction) in the field of view. In this case, the trapezoidal correction processing is performed, for example, by using the primary image data captured from the actual viewpoint P where the angle between the front surface 40a of the building 40 and the optical axis of the imaging lens 4 in the horizontal direction is not a right angle as the surface of the building 40. A trapezoidal correction is performed to approximate an image taken from a virtual viewpoint P ′ directly facing 40a. Under such shooting conditions, the rectangular front surface 40a of the building 40 is asymmetric in the left-right direction in the primary image data. In this case, the trapezoidal correction process is performed by converting the primary image data 50 or the secondary image data 50 ′ subjected to the trapezoidal correction process in the pitch direction to the plane B inclined by the second angle αx with respect to the plane A in the yaw direction. The image subjected to the deformation process to be projected is defined as secondary image data 50 ′. Specifically, the coordinates (x ′, y ′) of the secondary image data 50 ′ of the plane B inclined by the second angle αx in the yaw direction with respect to the plane A are 1 Relationship in which the variable (x, y) of the coordinate related to the next image is replaced with (y, x), and the variable (x ′, y ′) of the coordinate related to the secondary image is replaced with (y ′, x ′). It becomes an expression. In addition, with respect to Equation (2), L [mm] related to the value of β is the length of the effective pixel region of the image sensor 3 in the long side direction.

  As described above, the image processing unit 16 according to the present embodiment forms the primary image data 50 in the pitch direction with respect to the plane A parallel to the light receiving surface of the image sensor 3, with respect to the light receiving surface. Then, the secondary image data subjected to the deformation process for projecting onto the plane B forming the second angle αx in the yaw direction is generated.

  Note that the details of the image processing necessary for generating the trapezoid correction processing described above are well-known techniques, and thus detailed description thereof will be omitted.

  Next, the operation of the image pickup apparatus 1 having the configuration described above will be described. The flowchart shown in FIG. 3 shows an operation when the power supply of the imaging apparatus 1 is on and in the imaging operation mode. In the imaging operation mode, as an initial state, primary image data is acquired from the imaging device 3 every predetermined frame period, and secondary image data after image processing by the image processing unit 16 is sequentially performed on the image display device 6. Live view display operation for display output. In the imaging operation mode, the imaging device 1 executes an operation in which the imaging device 1 stores the subject image as electronic data in the storage medium 22 in response to an operation input to the release switch 7b. Note that the imaging apparatus 1 may be capable of executing a reproduction mode in which electronic image data stored in the storage medium 22 is reproduced and displayed on the image display apparatus 6.

  The imaging operation mode is, for example, when the user operates the power operation switch 7a to input the imaging apparatus 1 to be in a dormant state, when an operation to switch to another operation mode is input, or when the control unit 10 If it is determined to enter the hibernation state, the process ends. The determination process for ending the imaging operation mode is omitted in the flowchart of FIG.

  In the imaging operation mode, power is supplied to the imaging device 3 and the imaging device driving unit 8. Further, in order to perform the live view display operation, the acquisition of the primary image data 50 from the image sensor 3 is performed every predetermined frame period. In the imaging operation mode, first, in step S11, the primary image data 50 output from the imaging element driving unit 8 is stored in the image data buffer 12b of the RAM 12.

  Next, in step S12, it is determined whether or not the keystone correction mode is selected by the user. The trapezoidal correction mode is an operation mode in which the imaging apparatus 1 performs the above-described trapezoidal correction process on the primary image data 50 using the image processing unit 16. Whether or not the trapezoid correction mode is selected can be switched by the user operating the four-way switch 7c or the touch panel 7e on the GUI displayed on the image display device 6, for example.

  If it is determined in step S12 that the keystone correction mode has been selected by the user, the process proceeds to step S14 after the keystone correction process shown in FIG. 4 is executed in step S13. By executing step S13, secondary image data 50 'obtained by performing keystone correction processing on the primary image data 50 is stored in the image data buffer 12b.

  On the other hand, if it is determined in step S12 that the keystone correction mode is not selected by the user, step S13 is skipped and the process proceeds to step S14.

  In step S <b> 14, the image data stored in the image data buffer 12 b of the RAM 12 is displayed on the image display device 6. If step S13 is skipped, the primary image data 50 that has not been subjected to the trapezoidal correction process is displayed on the image display device 6 in step S14. On the other hand, when step S13 is executed, the secondary image data 50 'subjected to the trapezoidal correction process is displayed on the image display device 6.

  In the secondary image data 50 ′ subjected to the trapezoidal correction process, the field of view of the imaging device 1 is not rectangular as shown in FIG. 9. In step S14, the entire secondary image data 50 'may be displayed on the image display device 6, or a part of the secondary image data 50' trimmed into a rectangular area as shown by a two-dot chain line in FIG. These areas may be displayed on the image display device 6.

  Next, in step S15, it is determined whether or not an input for instructing movement of the AF area is made via the input device 7. As described above, the input for instructing the movement of the AF area in the present embodiment is performed by the user operating the input device 7 such as the touch panel 7e or the four-way switch 7c. In the present embodiment, as an example, the control unit 10 determines that an input for instructing movement of the AF area is made when the user taps the area of the image data displayed on the image display device 6. To do. In this embodiment, the control unit 10 recognizes the coordinates in the image data corresponding to the position tapped by the user as the AF position designation coordinates.

  In the present embodiment, as an example, as shown in FIG. 11, the shape of the AF area icon 60 in the image displayed on the image display device 6 is the image data displayed on the image display device 6 in step S <b> 14. Regardless of whether or not the trapezoidal correction process has been performed, the image display device 6 is a rectangle or rectangle surrounded by four sides parallel to the four sides of the outer shape of the display surface.

  If it is determined in step S15 that an input for instructing movement of the AF area has been made, the process proceeds to step S40. In step 40, the AF area movement process shown in the flowchart of FIG. 10 is executed.

  In the AF area movement process, first, in step S41, it is determined whether or not the trapezoid correction mode has been selected by the user. If it is determined in step S41 that the keystone correction mode has not been selected by the user, the process proceeds to step S42.

  In step S42, AF position designation coordinates input by the user via the input device 7 are acquired. Here, since the keystone correction mode is not executed, the primary image data 50 that has not been subjected to the keystone correction process is displayed on the image display device 6 on the image display device 6. Therefore, the AF position designation coordinates in step S42 are represented by the coordinates (x, y) of the primary image data 50 on the plane A.

  Next, in step S43, the AF position designation coordinates are stored in the log information buffer 12c of the RAM 12. In step S44, as shown in FIG. 11, an AF area icon 60 indicating the position of the AF area is superimposed and displayed on the AF position designation coordinates in the primary image data 50 displayed on the image display device 6. To do. After execution of step S44, the process returns to step S11 of FIG.

  On the other hand, if it is determined in step S41 of the AF area movement process that the keystone correction mode is selected by the user, the process proceeds to step S45.

  In step S45, the AF position designation coordinates input by the user via the input device 7 are acquired. Here, since the keystone correction mode is executed, the secondary image data 50 ′ subjected to the keystone correction processing is displayed on the image display device 6 on the image display device 6. Therefore, the AF position designation coordinates in step S45 are represented by the coordinates (x ′, y ′) of the secondary image data 50 ′ of the plane B inclined by the angle α in the pitch direction with respect to the plane A.

  Next, in step S46, the AF position designation coordinates are stored in the log information buffer 12c of the RAM 12. Then, in step S47, as shown in FIG. 12, an AF area icon 60 indicating the position of the AF area is superimposed on the AF position designation coordinates in the secondary image data 50 ′ displayed on the image display device 6. indicate. After execution of step S47, the process returns to step S11 in FIG.

  On the other hand, if it is determined in step S15 that there is no input instructing movement of the AF area, the process proceeds to step S16.

  In step S16, it is determined whether or not the first release switch of the release switch 7b is in an ON state. That is, it is determined whether or not the release switch 7b is half pressed.

  If it is determined in step S16 that the first release switch of the release switch 7b is in the OFF state, the process returns to step S11. That is, until the release switch 7b is operated, the operations from step S11 to step S15 are repeatedly executed in synchronization with a predetermined frame period. The live view in which the primary image data 50 or the secondary image data 50 ′ displayed on the image display device 6 is constantly updated by repeating the operations from step S11 to step S15 in synchronization with a predetermined frame period. Operation is performed. Further, the operations from step S11 to step S15 are repeated in synchronization with a predetermined frame period, whereby the AF position designation coordinates are changed according to the operation input to the input device 7 by the user, and the AF position designation coordinates are changed. The position of the AF area icon 60 in the live view display changes according to the change.

  On the other hand, if it is determined in step S16 that the first release switch of the release switch 7b is in the ON state, the process proceeds to step S17. In step S17, the AF process of the flowchart shown in FIG. 13 is executed.

  In the AF process, first, as shown in step S61, it is determined whether or not the keystone correction mode is selected by the user.

  If it is determined in step S61 that the keystone correction mode has not been selected by the user, the process proceeds to step S62.

  In step S62, the AF position designation coordinates in the primary image data 50 stored in the log information buffer 12c of the RAM are read. In step S63, the AF control unit 16 detects the contrast value in the AF area with the AF position designation coordinate in the primary image data 50 as the center, and controls the AF mechanism unit 4a to set the in-focus distance. The imaging lens 4 is brought into a focused state by changing. As described above, when the imaging apparatus 1 is capable of the phase difference detection type autofocus operation, the autofocus operation is detected by the distance measuring sensor in the AF area centered on the AF position designation coordinates. This is done using the phase difference information. In step S64, for example, by changing the color of the AF area icon 60 displayed on the image display device 6, it is displayed that the imaging lens 4 is in focus.

  On the other hand, if it is determined in step S61 that the trapezoidal correction mode is selected by the user, the process proceeds to step S65.

  In step S65, the AF position designation coordinates in the secondary image data 50 'stored in the log information buffer 12c of the RAM are read. Next, in step S66, conversion processing for projecting AF position designation coordinates in the secondary image data 50 ′ and AF areas that are square or rectangular in the secondary image data 50 ′ onto the primary image data 50 is executed. . This conversion process is performed by using the above-described equation (1) of the trapezoid correction process.

  For example, the AF area displayed by the square AF area icon 60 ′ as shown in FIG. 12 by the image display device 6 becomes a square in the primary image data 50 as shown in FIG. Have different shapes.

  In step S67, the AF control unit 16 controls the AF mechanism unit 4a while detecting the contrast value or phase difference information in the AF area in the primary image data 50 calculated in step S66, and the in-focus distance. Is changed to bring the imaging lens 4 into a focused state. In step S68, for example, by changing the color of the AF area icon 60 'displayed on the image display device 6, it is displayed that the imaging lens 4 is in focus.

  After the AF process is executed, the process proceeds to step S18. In step S18, it is determined whether or not the first release switch of the release switch 7b is in an OFF state. If it is determined in step S18 that the first release switch of the release switch 7b is in the OFF state, it is determined that the user has stopped the framing operation, and the process returns to step S11 again.

  On the other hand, when it is determined in step S18 that the first release switch of the release switch 7b is in the ON state, the process proceeds to step S19. In step S19, it is determined whether or not the second release switch of the release switch 7b is in the ON state.

  If it is determined in step S19 that the second release switch of the release switch 7b is in the OFF state, the process returns to step S11.

  On the other hand, if it is determined in step S19 that the second release switch of the release switch 7b is in the ON state, the process proceeds to step S20 to execute the imaging operation. In the imaging operation, the imaging device 3 is driven so that imaging using the exposure value calculated by the AE control unit 15 is performed, and the obtained image data is stored in the image data buffer 12b as imaging image data.

  Next, in step S21, it is determined whether or not the keystone correction mode is selected by the user. If it is determined in step S21 that the trapezoid correction mode is selected by the user, the process proceeds to step S22, and the above-described trapezoid correction process is performed on the image data for imaging. In step S23, the image data for imaging subjected to the trapezoidal correction process is stored in the storage medium 22 as an electronic file in a predetermined format.

  On the other hand, if it is determined in step S21 that the keystone correction mode has not been selected by the user, step S22 is skipped and the process proceeds to step S23 to store the image data for imaging as an electronic file in a predetermined format. Store in the medium 22.

  As described above, the imaging apparatus 1 according to the present embodiment includes the imaging element 3 that acquires primary image data every predetermined frame period, and the focal point that acquires the focal length of the imaging lens 4 in synchronization with the frame period. The distance detection unit 4b includes two values including a first angle αy that is an angle formed in the pitch direction with respect to the light receiving surface of the image sensor 3 and a second angle αx that is an angle formed in the yaw direction with respect to the light receiving surface. Using the input device 7 for inputting the correction angle and the focal length value and the correction angle value in synchronization with the frame period, the primary image data is formed in the pitch direction with respect to the light receiving surface. An image processing unit 16 for generating secondary image data subjected to a deformation process for projecting onto a plane having a second angle αx in the yaw direction with respect to the light receiving surface, and based on the secondary image data in synchronization with the frame period. Image display device for displaying images And, equipped with a.

  According to the imaging apparatus 1 of the present embodiment having such a configuration, trapezoidal correction processing for correcting the trapezoidal distortion of the subject can be executed in two directions of the pitch direction and the yaw direction of the imaging apparatus 1. The degree of the trapezoid correction process (correction angle) can be freely changed by operating the input device 7. In the present embodiment, the secondary image data 50 ′, which is the result of the keystone correction process, is displayed on the image display device 6 as a live view, and the result of changing the correction angle is immediately displayed in the live view. Reflected. For this reason, if the imaging device 1 of this embodiment is used, the user can perform the trapezoid correction process on the image as intended.

  Further, the imaging apparatus 1 of the present embodiment further includes an AF control unit 14 that performs control to change the focal distance of the imaging lens 4, and in the secondary image data displayed on the image display apparatus 6 by the input device 7. An input designating the position of the AF area is received, and the image processing unit 16 executes a conversion process for projecting the position of the AF area in the secondary image data onto the plane including the primary image data, and the AF control unit 14 The focusing distance of the imaging lens 4 is changed so that the imaging lens 4 is in a focused state at a position corresponding to the AF area projected onto the plane including the primary image data on the light receiving surface of the imaging device 3. Let

  According to such an imaging apparatus 1 of the present embodiment, the user confirms the secondary image data 50 ′ subjected to the trapezoidal correction process displayed on the image display apparatus 6 as a live view while confirming the input apparatus 7. By operating the, the position of the AF area, which is the area for autofocusing, can be moved. The position of the AF area input by the user is converted into coordinates in the primary image data 50 before the keystone correction process is performed, and an autofocus operation is executed based on these coordinates. For this reason, according to the present embodiment, the AF area in which the user designates the position in the subject is not shifted in position with respect to the subject due to the trapezoidal correction process, and the keystone correction process is performed as intended by the user. Images can be taken.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. Hereinafter, only differences from the first embodiment will be described, and the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In the first embodiment described above, the AF area is surrounded by four sides parallel to the four sides of the outer shape of the rectangular display surface of the image display device 6 during live view display displayed on the image display device 6. Square or rectangular.

  On the other hand, in the present embodiment, the AF area is a square or rectangle surrounded by four sides parallel to the four sides of the outer shape of the primary image data which is a rectangle in the primary image data. That is, the AF area corresponds to an area surrounded by four sides parallel to the four sides of the outer shape of the light receiving surface on the light receiving surface of the image sensor 3.

  Therefore, the present embodiment differs from the first embodiment in the contents of the AF area movement process in step S40 and the AF process in step S17.

  FIG. 15 is a flowchart of AF area movement processing by the imaging apparatus 1 of the present embodiment. In the AF area movement processing of the present embodiment, first, in step S41, it is determined whether or not the trapezoid correction mode has been selected by the user. If it is determined in step S41 that the keystone correction mode has not been selected by the user, the process proceeds to step S42.

  The operations from step S42 to step S44 performed when the keystone correction mode is not selected by the user are the same as those in the first embodiment. After execution of step S44, the process returns to step S11 of FIG.

  On the other hand, if it is determined in step S42 that the keystone correction mode has been selected by the user, the process proceeds to step S45.

  In step S45, the AF position designation coordinates input by the user via the input device 7 are acquired. Here, since the keystone correction mode is executed, the secondary image data 50 ′ subjected to the keystone correction processing is displayed on the image display device 6 on the image display device 6. Therefore, the AF position designation coordinates in step S45 are represented by the coordinates (x ′, y ′) of the secondary image data 50 ′ of the plane B inclined by the angle α in the pitch direction with respect to the plane A.

  Next, in step S50, a conversion process for projecting the AF position designation coordinates in the secondary image data 50 'onto the primary image data 50 is executed. This conversion process is performed by using the above-described equation (1) of the trapezoid correction process. By executing step S50, the AF position designation coordinates are expressed by the coordinates (x, y) of the primary image data 50 of the plane A. In step S51, the AF position designation coordinates in the primary image data 50 are stored in the log information buffer 12c of the RAM.

Next, in step S52, as shown in FIG. 16, in the primary image 50, a square or rectangular primary AF area 61 having a predetermined size centered on the AF position designation coordinates calculated in step S50 is generated. To do. Next, in step S53, a conversion process for generating a secondary AF area 61 ′ obtained by projecting the primary AF area 61 in the primary image 60 onto the secondary image data 50 ′ is executed. In this conversion process, the formula (1) of the trapezoid correction process described above is transformed to obtain the coordinates (x ′, y ′) of the secondary image data from the coordinates (x, y) of the primary image data. It is done by using. As shown in FIG. 17, the secondary AF area 61 ′ in the secondary image data 50 ′ has a different shape from the primary AF area 61 according to the result of the trapezoidal correction process. As shown in FIG. 18, an AF area icon 62 ′ indicating the secondary AF area 61 ′ is superimposed and displayed on the AF position designation coordinates in the secondary image data 50 ′ displayed on the image display device 6. After execution of step S54, the process returns to step S11 of FIG.

  FIG. 19 is a flowchart of AF processing by the imaging apparatus 1 of the present embodiment. In the AF processing of this embodiment, first, in step S71, the AF position designation coordinates in the primary image data 50 stored in the log information buffer 12c of the RAM are read.

  In step S72, the AF control unit 16 controls the AF mechanism unit 4a to detect the focus distance while detecting the contrast value in the AF area centered on the AF position designation coordinate in the primary image data 50. The imaging lens 4 is brought into a focused state by changing. In step S73, for example, by changing the color of the AF area icon displayed on the image display device 6, it is displayed that the imaging lens 4 is in focus. After the AF process is executed, the process proceeds to step S18 shown in FIG.

  As described above, in this embodiment as well, in the same way as in the first embodiment, the user performs live view on the secondary image data 50 ′ subjected to the trapezoid correction process displayed on the image display device 6. By operating the input device 7 while confirming, it is possible to move the position of the AF area, which is the area for autofocusing. The position of the AF area input by the user is converted into coordinates in the primary image data 50 before the keystone correction process is performed, and an autofocus operation is executed based on these coordinates. For this reason, according to the present embodiment, the AF area in which the user designates the position in the subject is not shifted in position with respect to the subject due to the trapezoidal correction process, and the keystone correction process is performed as intended by the user. Images can be taken.

  Note that the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the gist or concept of the invention that can be read from the claims and the entire specification. An apparatus and an imaging method are also included in the technical scope of the present invention.

  In addition, the imaging apparatus according to the present invention is not limited to the form of a digital camera as described in the above-described embodiment, and may be in the form of an electronic device having an imaging function, for example. Examples of electronic devices having an imaging function include portable communication terminals, game machines, digital media players, watches, voice recorders, binoculars, and the like.

1 imaging device,
2 body part,
3 image sensor,
4 Imaging lens,
4a AF mechanism,
4b focal length detector,
6 image display device,
7 Input device,
7a Power operation switch,
7b Release switch,
7c 4-way switch,
7d dial switch,
7e touch panel,
7f 2-way switch,
8 image sensor drive unit,
10 control unit,
11 CPU,
12 RAM,
12a control program,
12b image data buffer,
12c log information buffer,
13 Flash memory,
14 AF control unit,
15 AE control unit,
16 image processing unit,
16a Keystone correction coefficient calculation unit,
16b correction processing unit,
20 power supply,
21 power connection,
22 storage media,
23 Storage medium connection unit.

Claims (4)

An image sensor that is arranged so that the light receiving surface is orthogonal to the optical axis of the imaging lens, and acquires primary image data for each predetermined frame period;
A focal length detection unit that acquires a focal length of the imaging lens in synchronization with the frame period;
An input device for receiving an input of a correction angle including two values including a first angle that is an angle made in the pitch direction with respect to the light receiving surface and a second angle that is an angle made in the yaw direction with respect to the light receiving surface; ,
Using the focal length value and the correction angle value in synchronization with the frame period, the primary image data is formed at the first angle in the pitch direction with respect to the light receiving surface, and with respect to the light receiving surface. An image processing unit for generating secondary image data subjected to a deformation process for projecting onto a plane forming the second angle in the yaw direction;
An image display device that displays an image based on the secondary image data in synchronization with the frame period;
An AF control unit that performs control to change the focusing distance of the imaging lens;
With
The input device accepts an input for designating a position of an AF area in the secondary image data displayed on the image display device;
The image processing unit executes a conversion process of projecting the position of the AF area in the secondary image data onto a plane including the primary image data;
The AF control unit focuses the imaging lens so that the imaging lens is in a focused state at a position corresponding to the AF area projected on the plane including the primary image data on the light receiving surface. An imaging apparatus characterized by changing a distance. The outer shape of the AF area in the secondary image data is rectangular,
The image processing unit executes a conversion process of projecting the position and shape of the AF area in the secondary image data onto a plane including the primary image data;
The AF control unit adjusts the imaging lens based on a contrast value or phase difference information of an image corresponding to the AF area projected on a plane including the primary image data on the light receiving surface. The imaging apparatus according to claim 1, wherein a focusing distance of the imaging lens is changed so as to be in a focused state. The outer shape of the AF area on the light receiving surface is rectangular,
The image processing unit performs a deformation process of projecting the outer shape of the AF area on the light receiving surface onto a plane including the secondary image data,
2. The imaging according to claim 1, wherein the image display device superimposes and displays the shape of the AF area projected on a plane including the secondary image data on an image based on the secondary image data. apparatus. An image sensor that is arranged so that the light receiving surface is orthogonal to the optical axis of the imaging lens, and acquires primary image data for each predetermined frame period;
A focal length detection unit that acquires a focal length of the imaging lens in synchronization with the frame period;
An input device for receiving an input of a correction angle including two values including a first angle that is an angle made in the pitch direction with respect to the light receiving surface and a second angle that is an angle made in the yaw direction with respect to the light receiving surface; ,
Using the focal length value and the correction angle value in synchronization with the frame period, the primary image data is formed at the first angle in the pitch direction with respect to the light receiving surface, and with respect to the light receiving surface. An image processing unit for generating secondary image data subjected to a deformation process for projecting onto a plane forming the second angle in the yaw direction;
An image display device that displays an image based on the secondary image data in synchronization with the frame period;
An AF control unit that performs control to change the focusing distance of the imaging lens;
An imaging method for an imaging apparatus comprising:
Receiving an input for designating a position of an AF area in the secondary image data displayed on the image display device by the input device;
Executing a conversion process of projecting the position of the AF area in the secondary image data onto a plane including the primary image data by the image processing unit;
The AF control unit adjusts the imaging lens so that the imaging lens is in focus at a position corresponding to the AF area projected on the plane including the primary image data on the light receiving surface. Changing the focal distance;
An imaging method comprising:

Images