JP2016111451A - Imaging control device, imaging device and imaging control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To image a subject at a position that is previously selected by a user in automatic imaging of a moving subject more surely than in the conventional practice.SOLUTION: An imaging control device 103, 106 includes first image acquisition means for acquiring a first image generated by first imaging, second image acquisition means for acquiring information about motion of an imaging device 100, shifting a shift element 101a so as to make an imaging area by the imaging device precede the motion in the direction of the motion to make the imaging device perform second imaging, and acquiring a second image generated by the second imaging, subject image detection means for detecting a second subject image which matches at least a portion of the first subject image included in the first image in the second image, and imaging instruction means for making the imaging device performing imaging for recording in accordance with the detection of the second subject image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus that performs user assistance for so-called panning.

  There is panning as an imaging technique that expresses the sense of speed of a moving subject. In the panning, the photographer, that is, the user pans the imaging device in accordance with the movement of the subject, so that the moving subject stops within the imaging range (imaging screen) and the background flows (shakes). Do.

  However, in panning, if the panning speed of the imaging device by the user is different from the moving speed of the subject, a captured image that is blurred to the subject is obtained. In order to solve this problem, Patent Document 1 discloses a method of absorbing the difference between the moving speed of the subject and the panning speed by shifting the shift lens with respect to the optical axis in order to assist the user in performing panning. Has been. Specifically, for positioning the subject at the center of the image based on the panning speed of the imaging device detected by the gyro sensor and the motion vector of the subject detected between periodically generated images (frame images). The shift amount of the shift lens is calculated. Then, a good panning can be performed by shifting the shift lens by the shift amount.

  However, even if user assistance for such panning is performed, the panning speed and orientation of the imaging device may change when the user presses the shutter button to capture a still image. A blurred captured image (still image) may be obtained.

  With respect to this problem, in Patent Document 2, if a motion vector different from the motion vector detected in the peripheral region is detected in the central region of the imaging range, and the amount of the different motion vector is smaller than a predetermined amount, A method for determining that the shot is a panning shot and the subject is not shaken is disclosed. In this method, when the above determination is made, a still image is automatically captured in panning. Patent Document 3 discloses a method of automatically performing still image capturing on a subject when the subject reaches an automatic imaging position selected by a user during moving image capturing.

JP 2006-317848 A JP 2009-130628 A JP 2014-107750 A

  However, in the method disclosed in Patent Document 2, since automatic imaging is performed in response to determining that the motion vector amount of the subject is smaller than a predetermined amount, still image imaging may not be performed at a position desired by the user. There is sex. In general, in panning, the exposure time is adjusted to be longer than usual. For example, it is only necessary to automatically capture a still image when the subject moves to the front of the user, but automatic imaging is performed before that. The still image in front of the user cannot be acquired.

  Further, in the method disclosed in Patent Document 3, theoretically, a still image is automatically imaged in response to a subject reaching an automatic imaging position desired (selected) by a user. However, when the panning speed is high, there is a possibility that a still image at the automatic imaging position cannot be obtained due to a time lag from when the subject reaches the automatic imaging position until actual imaging processing is performed. For example, FIG. 13A shows that an imaging point image is acquired by capturing a still image of a fixed subject (tree) at an automatic imaging position desired by the user in advance. FIG. 13B shows a pattern matching between the frame image periodically generated in the panning of the moving main subject (car) and the imaging point image shown in FIG. It shows that automatic imaging of a still image of the main subject is performed in accordance with matching. However, as shown in FIG. 14 (a), when the moving speed of the main subject, that is, the panning speed is high, even if the imaging process is started at the automatic imaging position, it is actually shifted from the automatic imaging position. There is a possibility that a captured still image is acquired. Also, as shown in FIG. 14B, the main subject that has reached the automatic imaging position overlaps the fixed subject shown in FIG. 13A, so that the pattern matching itself between the frame image and the imaging point image is good. There is also a risk that it will not be done.

  The present invention provides an imaging control apparatus and an imaging apparatus that can more reliably perform imaging of a main subject at a position selected in advance by a user in automatic imaging of a moving main subject.

  An imaging control apparatus according to one aspect of the present invention controls an imaging apparatus that performs imaging using a shift element that is shiftable with respect to an optical axis. The imaging control device acquires first image acquisition means for acquiring the first image generated by the first imaging, and information relating to the movement of the imaging device, and the imaging region by the imaging device is more Second image acquisition means for shifting the shift element so as to precede the direction of movement, causing the imaging device to perform second imaging, and acquiring a second image generated by the second imaging; In the second image, subject image detection means for detecting a second subject image that matches at least a part of the first subject image included in the first image, and the second subject image are detected. Accordingly, the image pickup apparatus includes an image pickup instruction unit that causes the image pickup apparatus to perform image pickup for recording.

  An imaging control program according to another aspect of the present invention is a computer program that causes a computer that controls an imaging apparatus that performs imaging using a shift element that is shiftable with respect to an optical axis to execute an imaging control process. The imaging control process acquires the first image acquisition process for acquiring the first image generated by the first imaging and the information related to the movement of the imaging apparatus, and the imaging area of the imaging apparatus is more responsive than the above movement. A second image acquisition process in which the shift element is shifted so as to precede the direction of the image, causing the imaging apparatus to perform the second imaging, and a second image generated by the second imaging is acquired; And subject image detection processing for detecting a second subject image that matches at least a part of the first subject image included in the first image, and detection of the second subject image. And imaging instruction processing for causing the imaging apparatus to perform recording imaging.

  According to the present invention, imaging for acquiring the second image is performed by shifting the shift element so that the imaging region precedes the movement direction of the imaging device, and the second image is preliminarily obtained. A subject image that matches the subject image in the first image prepared by the user is detected. Thus, automatic imaging such as panning for a moving subject can be reliably performed at a position selected in advance by the user as compared with the related art.

1 is a block diagram illustrating a configuration of an imaging apparatus that is Embodiment 1 of the present invention. 5 is a flowchart illustrating imaging point image acquisition processing according to the first embodiment. 5 is a flowchart illustrating a panning automatic imaging process according to the first embodiment. 3 is a flowchart illustrating subject detection / imaging processing in Embodiment 1. 5 is a flowchart showing a panning assist process in Embodiment 1. 9 is a flowchart showing a panning automatic imaging process in Embodiment 2 of the present invention. 10 is a flowchart showing a panning automatic imaging process in Embodiment 3 of the present invention. 10 is a flowchart showing still image capturing processing in Embodiment 4 of the present invention. FIG. 5 is a diagram illustrating an image acquired when the shift lens is positioned at the center and an image acquired when the shift lens is shifted in the panning direction in each embodiment. The figure explaining the process when a user reduces panning speed in an Example. FIG. 10 is a diagram illustrating automatic panning imaging in the second embodiment. FIG. 6 is a diagram illustrating the movement timing of the shift lens in the second embodiment. The figure explaining the conventional method of performing automatic imaging in the position which a user desires. The figure explaining the phenomenon which arises when panning speed is too high.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows the configuration of an imaging apparatus 100 that is Embodiment 1 of the present invention. The imaging apparatus 100 may be not only a typical camera such as a digital still camera and a digital video camera, but also an electronic device having a camera function such as a mobile phone with a camera function or a computer with a camera. Further, the imaging apparatus 100 as a typical camera may be a lens-integrated camera having an imaging optical system 101 to be described later (not removable), and the imaging optical system 101 can be attached and detached. An interchangeable lens camera may be used.

  The imaging optical system 101 includes a plurality of lenses, a shutter, and a diaphragm, and forms a subject image on light from a subject (not shown). AX is the optical axis of the imaging optical system 101. The movement of the focus lens and zoom lens (both not shown) in the imaging optical system 101 in the optical axis direction, the shift of the shift lens 101a with respect to the optical axis AX, and the operation of the shutter and the diaphragm (none shown) are performed by the CPU 103. Controlled by. A shift lens (optical element) 101a as a shift element is shifted (moved) in a direction orthogonal to the optical axis AX by a shift actuator (not shown), so that an object-side region captured by the imaging optical system 101, that is, imaging. The imaging area by the apparatus 100 is shifted.

  The imaging element 102 is configured by a photoelectric conversion element such as a CCD sensor or a CMOS sensor, and photoelectrically converts the subject image formed by the imaging optical system 101 and outputs an electrical signal (hereinafter referred to as an imaging signal).

  The image processing unit 106 performs various image processing on the imaging signal received from the imaging element 102 through the CPU 103 to generate image data, and outputs the generated image data to the CPU 103 and the primary storage unit 104. The image processing unit 106 also performs subject detection processing (which will be described later) using pattern matching in two images (data) as subject image detection means.

  The angular velocity sensor 105 is constituted by a gyro sensor, detects an angular velocity of movement of the imaging device 100, and outputs an electrical signal corresponding to the angular velocity (hereinafter referred to as an angular velocity signal) to the CPU 103. In this embodiment, the angular velocity sensor 105 detects a panning movement of the imaging apparatus 100 by the user, and the CPU 103 receives an angular velocity signal as information regarding the movement. Panning means horizontal movement (rotation) in a narrow sense, but tilting that is vertical movement (rotation) may be included as panning in a broad sense.

  The CPU 103 as a computer controls each part of the imaging apparatus 100 based on various input signals and realizes each function of the imaging apparatus 100 according to a computer program described later. Note that the CPU 103 may perform at least a part of the processing performed by the image processing unit 106 described above. The image processing unit 106 and the CPU 103 constitute an imaging control device. The image processing unit 106 and the CPU 103 perform a panning automatic imaging process as user assistance when panning the main subject (moving subject) that the user moves while panning the imaging apparatus 100. The panning automatic imaging process is an imaging control process for automatically imaging the main subject when the main subject reaches a position (imaging point) selected in advance by the user during the panning. The CPU 103 functions as a first image acquisition unit, a second image acquisition unit, and an imaging instruction unit.

  The primary storage unit 104 is configured by a volatile memory such as a RAM, and temporarily stores (holds) information and image data generated by imaging, or is used as a work area of the CPU 103. Information and data held in the primary storage unit 104 are used for image processing in the image processing unit 106 or recorded on the recording medium 107. The secondary storage unit 108 includes a nonvolatile memory such as an EEPROM, and stores the above-described computer program (firmware, imaging control program, etc.) for controlling the imaging apparatus 100 and various setting information used by the CPU 103. ing.

  The recording medium 107 records image data and the like generated by imaging and temporarily stored in the primary storage unit 104. The recording medium 107 is a medium that can be attached to and detached from the imaging apparatus 100, such as a semiconductor memory or an optical disk. By attaching the recording medium 107 removed from the imaging apparatus 100 to an external apparatus such as a personal computer or a printer, the image data recorded on the recording medium 107 is displayed or output (printed) on the external apparatus. can do.

The display unit 109 displays a live view image (moving image) before the image capturing for recording is performed in the image capturing apparatus 100, displays a recording image (still image or moving image) generated by the image capturing for recording, or displays a GUI. Or display an image. The display unit 109 is a touch panel that can be touched. The operation unit 110 includes various input devices that accept user operations and output operation information to the CPU 103. The input devices include contact type input devices such as buttons, levers, and touch panels, and non-contact type input devices using sound and line of sight.
(Relationship between subject detection processing and shift lens)
Next, an outline of the relationship between the subject detection process performed by the image processing unit 106 and the shift lens 101a in the panning automatic imaging process performed by the imaging apparatus 100 of the present embodiment will be described with reference to FIG. In the subject detection process, an imaging point image (first image) acquired in advance by imaging a specific subject (first imaging) is prepared. Pattern matching is performed between this imaging point image and a subject detection image (second image) generated by imaging (second imaging) during panning of the main subject (lens shift state described later). . Then, subject image data (second subject image) that matches at least part of the image data (first subject image) of the specific subject included in the imaging point image is detected in the subject detection image by the pattern matching. .

  Note that the imaging (first imaging) for generating the imaging point image may be performed by the imaging apparatus 100 of the present embodiment as described later, but may be performed by another imaging apparatus. In the latter case, an imaging point image generated by imaging by the other imaging device may be taken into the imaging device 100 of the present embodiment through the recording medium 107 or communication.

  In FIG. 9 (a), the panning automatic imaging process is performed in a non-lens shift state where the shift lens 101a is in a non-shift position where the shift lens 101a is in the center of the movable range (a position where the center of the shift lens 101a coincides with the optical axis AX). It shows how it starts. On the other hand, FIG. 9B shows a state where the shift lens 101a performs the panning automatic imaging process in a lens shift state at a shift position shifted in the same direction as the panning direction of the imaging apparatus 100. Note that the circle shown as the shift lens 101a in these drawings only schematically shows the position of the shift lens 101a, and the circle does not show the actual size of the shift lens. This is the same in other drawings in which the shift lens 101a is indicated by a similar circle.

  The imaging region by the imaging device 100 in the lens shift state is shifted in the shift direction of the shift lens 101a from the imaging region corresponding to the orientation of the imaging device 100 (imaging optical system 101) in the non-lens shift state. For this reason, when the imaging apparatus 100 is panned, the imaging region precedes the panning movement by moving from the non-lens shift state to the lens shift state. That is, an image obtained by imaging in the lens shift state includes an image of a subject existing ahead in the panning direction as compared with an image obtained by imaging in the non-lens shift state. Therefore, when the image obtained by imaging in the lens shift state is used as the subject detection image, the distance between the imaging point images is faster than when the image obtained by imaging in the non-lens shift state is used. Pattern matching can be started.

When the shift lens 101a is also used as an anti-vibration lens for correcting image blur due to camera shake, the non-shift position is not limited to the center of the movable range of the shift lens 101a. It may be a shifted position. When the shift lens 101a is used as an anti-vibration lens, the shift lens 101a is shifted in a direction to suppress image blur based on an angular velocity signal corresponding to camera shake from the angular velocity sensor 105. Smaller than the shift amount in processing.
(Imaging point image acquisition processing)
Next, imaging point image acquisition processing in the present embodiment will be described with reference to the flowchart of FIG. Here, it is assumed that the CPU 103 has received a signal instructing acquisition of an imaging point image generated by a touch operation on the display unit 109 displaying a GUI image. The CPU 103 and the image processing unit 106 execute the panning automatic image capturing process including the image capturing point image acquisition process and the subject detection process and other processes according to the image capturing control program. This is the same not only in this embodiment but also in other embodiments.

  In step S201, the CPU 103 starts imaging for generating a live view image that is a moving image. Specifically, the CPU 103 controls the imaging optical system 101 to cause the imaging element 102 to perform a photoelectric conversion operation, and causes the image processing unit 106 to perform predetermined image processing on the imaging signal from the imaging element 102. Frame images constituting the live view image are sequentially generated at a predetermined cycle. These frame images are sequentially held in the primary storage unit 104.

  Next, in step S <b> 202, the CPU 103 sequentially acquires frame images held in the primary storage unit 104.

  In step S203, the CPU 103 sequentially displays the frame images sequentially acquired in step S202 on the display unit 109. As a result, the live view image is displayed on the display unit 109.

  Next, in step S204, the CPU 103 determines whether or not the shutter button provided in the operation unit 110 has been fully pressed (SW2 ON). The shutter button can be pressed halfway (SW1 ON) and fully pressed, imaging preparation processing such as photometry and autofocus is started in response to the half-pressing operation, and recording imaging is performed in response to the fully-pressing operation. . If the shutter button has been fully pressed, the process proceeds to step S205, and if not, the process returns to step S202.

  In step S205, the CPU 103 stores, in the secondary storage unit 108, a frame image generated at the timing of the full pressing operation of the shutter button among the frame images acquired in step S202 as an imaging point image.

In step S206, the CPU 103 ends imaging for generating a live view image.
(Panning automatic imaging process 1)
Next, the panning automatic imaging process 1 performed by the imaging apparatus 100 of the present embodiment will be described with reference to the flowchart of FIG. Here, a start instruction signal for a panning automatic imaging process generated by a user's touch operation on the display unit 109 displaying the GUI image is input to the CPU 103, and the live-view image imaging and the live-view image display unit 109 The process after the display of is started will be described. In addition, it is assumed that the user has started panning immediately after the touch operation.

  In step S <b> 301, the CPU 103 acquires an angular velocity signal (information regarding motion) output from the angular velocity sensor 105.

  Next, in step S302, the CPU 103 obtains the panning direction of the imaging apparatus 100 from the angular velocity signal acquired in step S301. Then, the shift actuator 101 is controlled to shift the shift lens 101a from the non-shift position to the shift position in the panning direction.

  In step S <b> 303, the CPU 103 obtains a frame image generated by imaging and held in the primary storage unit 104.

  In step S304, the CPU 103 displays the frame image acquired in step S303 on the display unit 109. As described above, in this embodiment, the frame image generated by the imaging in the lens shift state is displayed as the live view image. Further, as will be described later, the same frame image is used as the subject detection image.

  Next, in step S305, the CPU 103 reads out the imaging point image stored in the secondary storage unit 108 and outputs it to the image output unit 106, and also uses the frame image acquired in step S303 as the subject detection image as an image processing unit. It outputs to 106. The image processing unit 106 performs a subject detection process for performing pattern matching between the imaging point image and the subject detection image, and the CPU 103 performs a panning assist process as an imaging process according to the result of the subject detection process. Do. Details of subject detection processing (pattern matching) and imaging processing (panning assist processing) will be described later.

When the subject detection process and the imaging process are completed, in step S306, the CPU 103 determines whether the subject detection process (pattern matching) and the imaging process (the panning assist process) are successful and the panning automatic imaging is completed. . If completed, the present process is terminated, and if not, the process returns to step S301.
(Subject detection process)
Next, the subject detection process performed in step S304 will be described in more detail using the flowchart of FIG.

  In step S401, the image processing unit 106 performs pattern matching between the imaging point image acquired in step S303 and the subject detection image. Pattern matching methods include a method of confirming similarity using a cross-correlation function, and similarity based on the difference in luminance values of pixels at corresponding positions such as SSD (Sum of Squared Difference) and SAD (Sum of Absolute Difference). Various methods such as a method for confirming sex may be used.

  Next, in step S402, whether or not the CPU 103 has detected subject image data that matches all of the image data of the specific subject included in the imaging point image in the subject detection image by the pattern matching performed in step S401. Determine. If so, the process proceeds to step S403, and if not, this process ends (skip to step S306). In this step, it may be determined whether or not subject image data matching a part of the image data of the specific subject included in the imaging point image is detected in the subject detection image.

  In step S403, the CPU 103 controls the shift actuator to return the shift lens 101a to the non-shift position (center).

In step S404, the CPU 103 performs a panning assist process described below as the imaging process.
(Panning assist process)
Next, the panning assist process as the imaging process performed in step S404 will be described in detail with reference to the flowchart of FIG. In the panning assist process, the position of the shift lens 101a in the shift direction so as to reduce the relative displacement between the imaging area that moves together with the imaging apparatus 100 and the main subject (moving subject) that is the target of recording imaging, that is, Recording is performed by controlling the position of the imaging region.

  In step S501, the CPU 103 extracts a main subject (main subject image) that the user is following by panning the imaging device 100 from a frame image generated for a live view image in a non-lens shift state. As a method for extracting a main subject, there is a method using a motion vector obtained between sequentially generated frame images as disclosed in Patent Document 1. Specifically, when the imaging device 100 is panned following the main subject that the user moves, a motion vector corresponding to a shift between the movement of the main subject and the panning of the imaging device 100 is detected in an area where the main subject exists. . This motion vector is a small vector with respect to the background motion vector flowing around the main subject. From this, the moving main subject can be detected (extracted). The main subject may be extracted using other methods.

  Next, in step S <b> 502, the CPU 103 acquires an angular velocity signal indicating the panning speed of the imaging apparatus 100 from the angular velocity sensor 105.

  In step S503, the CPU 103 calculates the moving speed (angular speed) of the main subject using the angular velocity signal acquired in step S502 and the motion vector obtained in the area where the main subject used in step S501 exists. . Here, other methods may be used as long as the moving speed of the main subject can be obtained.

  In step S504, the CPU 103 converts the moving speed (angular speed) of the main subject acquired in step S503 into a target shift amount (or target shift position) of the shift lens 101a.

  Next, in step S505, the CPU 103 opens the shutter while shifting the shift lens 101a toward the target shift amount (or target shift position) obtained in step S504, and the shutter is opened in accordance with the elapse of a predetermined exposure time. Close. Between the opening and closing of the shutter, the CPU 103 causes the imaging device 102 to perform photoelectric conversion for recording imaging, and holds the imaging signal generated by the imaging device 102 by the photoelectric conversion in the primary storage unit 104. Let

  In step S506, the CPU 103 causes the image processing apparatus 106 to generate still image data (recording image) from the imaging signal held in the primary storage unit 104, and the still image data is stored in the recording medium 107 as a still image file. Record. Then, this process ends.

  As described above, in this embodiment, in the panning automatic imaging process for the moving main subject, the shift lens 101a is shifted so that the imaging area of the imaging apparatus 100 precedes the panning direction of the imaging apparatus 100. To obtain a subject detection image. Thereby, the subject detection process (pattern matching) between the imaging point image and the subject detection image can be performed at an earlier timing than in the case where such shift lens 101a is not shifted (conventional). Therefore, it is possible to perform automatic imaging (imaging for recording) at an imaging point more reliably than in the past.

  In the first embodiment, the case where the frame image generated by the imaging in the lens shift state is also displayed on the display unit 109 as the live view image has been described. However, the user who views this live view image may reduce the panning speed so that the main subject moves near the center of the screen. As a result, there is a possibility that the main subject cannot be smoothly tracked through the live view image, or the main subject may be lost when the lens shift state returns to the non-lens shift state. This will be described with reference to FIG.

  FIG. 10A shows a live view image 1001 displayed on the display unit 109 in a non-lens shift state before the start of panning (panning). In the live view image 1001, a main subject (car) exists near the center.

  FIG. 10B shows a live view image 1003 in a lens shift state for quickly performing pattern matching during panning. In the live view image 1003, a part of the specific subject (tree) located at the imaging point 1002 is reflected, while the main subject exists at a position shifted to the right side (opposite to the panning direction) from the image center. To do. A user who views such a live view image 1003 intentionally moves the main subject to the vicinity of the center of the live view image 1004 as shown in FIG. Reduce the panning speed. As a result, the inconvenience as described above occurs.

  For this reason, in this embodiment, the frame image (second image) acquired by imaging in the lens shift state (second imaging) is not displayed as a live view image on the display unit 109, but for pattern matching. Used only as a subject detection image. On the other hand, the frame image (third image) acquired by imaging in the non-lens shift state (third imaging) can be displayed as a live view image even during pattern matching. . In this embodiment, the CPU 103 functions as an output control unit. Further, the display on the display unit 109 may be replaced with an output to a display device such as an external monitor.

Note that the configuration of the imaging device 100 and the acquisition process of the imaging point image in the present embodiment are the same as those in the first embodiment (FIGS. 1 and 2), and thus description thereof is omitted.
(Overview of live view image display)
First, an outline of the display of the live view image during the panning automatic imaging process in the present embodiment will be described with reference to FIGS. 11 and 12.

  FIGS. 11A and 11C show how a frame image acquired by imaging in a non-lens shift state is displayed on the display unit 109 as a live view image. FIGS. 11B and 11D show a state in which a frame image (subject detection image) acquired by imaging in the lens shift state is not displayed as a live view image on the display unit 109. FIG. By alternately repeating the states shown in FIGS. 11A to 11D during the panning automatic imaging process, the pattern can be obtained by imaging in the non-lens shift state while pattern matching using the subject detection image is performed. A live view image can be displayed on the display unit 109. Thereby, the inconvenience due to the frame image acquired by the imaging in the lens shift state being displayed as the live view image can be solved.

  FIG. 12A shows the timing of shifting the shift lens 101a to the shift position and returning to the non-shift position. In this figure, an exposure interval (t1) is a time interval for performing exposure (photoelectric conversion) of the image sensor 102 in order to periodically generate a frame image, and is determined by the frame rate of the live view image. The exposure period (t2) is the time during which the image sensor 102 is exposed and is determined by the shutter speed and the like. The readout period (t3) is a time for reading out the imaging signal from the imaging device 102 during and after exposure. The lens shift possible period (t4) is a time during which the shift lens 101a can be moved between the non-shift position and the shift position. When the shift lens 101a is moved during the exposure period (t2), the generated frame image is blurred. Therefore, the shift lens 101a needs to be moved outside the exposure period (t2). Therefore, the lens shift possible period (t4) is “exposure interval (t1) −exposure period (t2)”.

The maximum lens shift time (t5) shown in FIG. 12B indicates the time required to move the shift lens 101a from the non-shift position to the end position (shift end) in the shift direction. If the maximum lens shift time (t5) is shorter than the lens shift possible period (t4), the shift lens 101a can be shifted from the non-shift position to the shift end.
(Panning automatic imaging process 2)
Next, the panning automatic imaging process 2 performed by the imaging apparatus 100 of the present embodiment will be described with reference to the flowchart of FIG. Also here, a start instruction signal for a panning automatic imaging process generated by a user's touch operation on the display unit 109 displaying the GUI image is input to the CPU 103, and the live view image imaging and live view image display unit 109 is input. The process after the display of is started will be described. In addition, it is assumed that the user has started panning immediately after the touch operation.

  First, in step S601, the CPU 103 initializes a counter to zero. The counter value is an integer value and is used to determine whether or not to display a live view image.

  Next, in step S602, the CPU 103 adds 1 to the counter.

  In step S603, the CPU 103 determines whether the counter value is a multiple of two. If the counter value is a multiple of 2, the process proceeds to step S607; otherwise, the process proceeds to step S604.

  In step S <b> 607, the CPU 103 acquires an angular velocity signal (information regarding motion) output from the angular velocity sensor 105.

  Next, in step S608, the CPU 103 obtains the panning direction of the imaging apparatus 100 from the angular velocity signal acquired in step S607. Then, the shift actuator 101 is controlled to shift the shift lens 101a from the non-shift position to the shift position in the panning direction. Specifically, the CPU 103 shifts the shift lens 101a from the non-shift position to the shift position within the lens shift possible period (t4) shown in FIG.

  Next, in step S <b> 609, the CPU 103 acquires a frame image generated by imaging and held in the primary storage unit 104 as a subject detection image. As described above, in this embodiment, the frame image generated by the imaging in the lens shift state is not displayed as the live view image but is used as the subject detection image. Then, the CPU 103 proceeds to step S610.

  On the other hand, in step S604, the CPU 103 controls the shift actuator to return the shift lens 101a from the shift position to the non-shift position (center). Specifically, the CPU 103 returns the shift lens 101a to the non-shift position within the lens shift possible period (t4) shown in FIG.

  Next, in step S605, the CPU 103 acquires a frame image generated by imaging and held in the primary storage unit 104 as a display image for a live view image.

  Next, in step S606, the CPU 103 displays the display image acquired in step S605 on the display unit 109 as a live view image. As described above, in this embodiment, the frame image generated by the imaging in the non-lens shift state is displayed as the live view image. Further, this frame image may be used as a subject detection image. Then, the CPU 103 proceeds to step S610.

  In step S <b> 610, the CPU 103 reads the imaging point image stored in the secondary storage unit 108 in the imaging point image acquisition process described in the first embodiment (FIG. 2), and outputs it to the image output unit 106. The frame image acquired in step S609 (and step S606) is output to the image processing unit 106 as a subject detection image. The image processing unit 106 performs a subject detection process for performing pattern matching between the imaging point image and the subject detection image, and the CPU 103 performs a panning assist process as an imaging process according to the result of the subject detection process. Do. The subject detection process (pattern matching) and the imaging process (panning assist process) are the same as those described in the first embodiment with reference to the flowcharts of FIGS.

  In step S611, the CPU 103 determines whether the subject detection process (pattern matching) and the imaging process (panning assist process) are successful and the panning automatic imaging is completed. If it has been completed, this process ends. Otherwise, the process returns to step S602.

  When the imaging apparatus 100 is a lens interchangeable type and the interchangeable lens apparatus including the imaging optical system 101 is detachable, the maximum lens shift time (t5) may be longer than the lens shift possible period (t4). is there. In this case, exposure of the image sensor 102 is started during the shift from the non-shift position of the shift lens 101a to the shift end, and a frame image including blur is generated. For this reason, the CPU 103 acquires the maximum lens shift time (t5) through communication with the interchangeable lens device. If it is determined that the maximum lens shift time (t5) is longer than the lens shift possible period (t4), the shift lens 101a is shifted in step S608 within the lens shift possible period (t4). Up to that.

  As described above, also in this embodiment, in the panning automatic imaging process for the moving main subject, the shift lens 101a is shifted so that the imaging area of the imaging device 100 precedes the panning direction of the imaging device 100. To obtain a subject detection image. Thereby, the subject detection process (pattern matching) between the imaging point image and the subject detection image can be performed at an earlier timing than in the case where such shift lens 101a is not shifted (conventional). Therefore, it is possible to perform automatic imaging (imaging for recording) at an imaging point more reliably than in the past.

  In addition, in the present embodiment, the shift and return of the shift lens 101a are alternately repeated in the panning automatic imaging process. The frame image generated by imaging in the lens shift state is not displayed on the display unit 109 as a live view image, and only the frame image generated by imaging in the non-lens shift state is displayed on the display unit 109 as a live view image. indicate. As a result, when a frame image generated by imaging in the lens shift state is displayed as a live view image, it is possible to avoid inconveniences such as the user reducing the panning speed so that the main subject is located near the center of the screen. Can do.

  In the first and second embodiments, it has been described that subject detection processing (pattern matching) is quickly performed by shifting the shift lens 101a so that the imaging region precedes the panning direction in the panning automatic imaging processing.

  However, when the panning speed (that is, the moving speed of the main subject) is low, automatic imaging at the position selected by the user can be easily performed without shifting the shift lens 101a as such.

For this reason, in this embodiment, whether to shift the shift lens 101a so as to precede the imaging region in the panning direction is selected according to the panning speed. Note that the configuration of the imaging device 100 and the acquisition process of the imaging point image in the present embodiment are the same as those in the first embodiment (FIGS. 1 and 2), and thus description thereof is omitted.
(Panning automatic imaging process 3)
Next, the panning automatic imaging process 3 performed in the imaging apparatus 100 of the present embodiment will be described with reference to the flowchart of FIG. Also here, a start instruction signal for a panning automatic imaging process generated by a user's touch operation on the display unit 109 displaying the GUI image is input to the CPU 103, and the live view image imaging and live view image display unit 109 is input. The process after the display of is started will be described. In addition, it is assumed that the user has started panning immediately after the touch operation.

  In step S <b> 701, the CPU 103 acquires an angular velocity signal (information regarding motion) output from the angular velocity sensor 105.

  In step S702, the CPU 103 obtains the panning speed (angular speed) of the imaging apparatus 100 from the angular velocity signal acquired in step S701, and determines whether the panning speed is greater (faster) than a predetermined threshold. The threshold value may be variable according to the shutter speed, the maximum lens shift time (t5) shown in FIG. If the panning speed is greater than the threshold, the process proceeds to step S703 (to S706). If the panning speed is equal to or less than the threshold, the process proceeds to step S707.

  Steps S703 and S704 are the same as steps S303 and S304 of FIG.

  In step S705, the CPU 10 holds the shift lens 101a at the non-shift position without shifting to the shift position. The image processing unit 106 performs pattern matching in the same manner as in step S401 in FIG. When the subject image data that matches the image data of the specific subject included in the imaging point image is detected by pattern matching, the CPU 10 performs the panning assist process in the same manner as in step S404 in FIG.

  Next, in step S706, it is determined whether the pattern matching and the panning assist process are successful and the panning automatic imaging is completed. If it has been completed, this process ends. Otherwise, the process returns to step S703.

  On the other hand, in step S707, the panning automatic imaging process 1 shown in FIG. 3 or the panning automatic imaging process 2 shown in FIG. 6 is performed. Then, this process ends.

  Thus, in the present embodiment, when performing the panning automatic imaging process, it is selected whether or not to shift the shift lens 101a so that the imaging area precedes the panning direction according to the panning speed. Thereby, such a shift is performed even when it is not necessary to shift the shift lens 101a so that the imaging region precedes the panning direction, thereby avoiding wasteful consumption of the power of the imaging device 100. Can do.

  In this embodiment, whether or not to shift the shift lens 101a according to the panning speed is selected immediately after the start of the panning automatic imaging process. However, the shift lens according to the panning speed during the panning automatic imaging process is selected. It may be selected whether or not to shift 101a.

  In the first to third embodiments, the case where the panning assist process is performed in the panning automatic imaging process has been described. However, as will be described below, the shift lens 101a may be shifted so that the imaging region precedes the panning direction even when the panning automatic imaging processing is performed without performing the panning assist processing.

Note that the configuration of the imaging device 100 and the acquisition process of the imaging point image in the present embodiment are the same as those in the first embodiment (FIGS. 1 and 2), and thus description thereof is omitted. Also, the panning automatic imaging process performed in the present embodiment is the same as described in any one of the first to third embodiments (FIGS. 3, 6, and 7). However, in place of the panning assist process in the first to third embodiments, a still image imaging process described below is performed as an imaging process in step S404 in FIG.
(Still image capture processing)
The still image capturing process will be described with reference to the flowchart of FIG.

  In step S801, the CPU 103 opens the shutter and closes the shutter when a predetermined exposure time has elapsed. Between the opening and closing of the shutter, the CPU 103 causes the imaging device 102 to perform photoelectric conversion for recording imaging, and holds the imaging signal generated by the imaging device 102 by the photoelectric conversion in the primary storage unit 104. Let

  Next, in step S <b> 802, the CPU 103 causes the image processing apparatus 106 to generate still image data (recording image) from the imaging signal held in the primary storage unit 104, and stores the still image data in the recording medium 107. Record as. Then, this process ends.

  As described above, even when the panning automatic imaging process is performed without performing the panning assist process, the shift lens 101a is shifted so that the imaging area precedes the panning direction, thereby automatically capturing the image at the imaging point selected by the user. Can be performed more reliably than in the past.

In each of the above embodiments, the case where the shift element is the shift lens 101a included in the imaging optical system 101 has been described. However, the imaging element 102 may be shifted with respect to the optical axis AX as a shift element.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

100 Imaging device 101a Shift lens 103 CPU
105 Angular velocity sensor 106 Image processing unit

Claims (8)

An imaging control device that controls an imaging device that performs imaging using a shift element that is shiftable with respect to an optical axis,
First image acquisition means for acquiring a first image generated by the first imaging;
Obtaining information related to the movement of the imaging device, shifting the shift element so that an imaging region by the imaging device precedes the direction of the movement, and causing the imaging device to perform second imaging; Second image acquisition means for acquiring a second image generated by the second imaging;
Subject image detection means for detecting a second subject image that matches at least a part of the first subject image included in the first image in the second image;
An imaging control apparatus comprising: an imaging instruction unit that causes the imaging apparatus to perform recording imaging in response to detection of the second subject image. The second image acquisition means repeatedly shifts and returns the shift element, performs the second imaging for each shift and performs the third imaging for each return,
The imaging control device does not cause the imaging device to display or output the second image generated for each second imaging, and displays or outputs the third image generated for each third imaging. The imaging control apparatus according to claim 1, further comprising output control means for outputting.   3. The imaging control according to claim 1, wherein the second image acquisition unit selects whether to perform the shift of the shift element and the second imaging according to the speed of the movement. 4. apparatus.   The imaging instruction means controls the position of the shift element in the shift direction so as to reduce the relative displacement between the imaging area that moves together with the imaging device and the moving subject, and performs the recording imaging for the moving subject. The imaging control device according to any one of claims 1 to 3, wherein The imaging control device according to any one of claims 1 to 4,
An imaging apparatus comprising: an imaging element that performs photoelectric conversion for imaging.   6. The image pickup apparatus according to claim 5, wherein the shift element is an optical element included in an image pickup optical system provided in the image pickup apparatus so as to be integrated or detachable, or the image pickup element. The imaging device is detachable from an interchangeable lens device having an imaging optical system,
The imaging apparatus according to claim 5, wherein the second image acquisition unit acquires information related to the movement of the imaging apparatus from a motion detection unit provided in the imaging apparatus or the interchangeable lens apparatus. . An imaging control program as a computer program that causes a computer that controls an imaging apparatus that performs imaging using a shift element that is shiftable with respect to an optical axis to execute imaging control processing,
The imaging control process includes
A process of acquiring a first image generated by the first imaging;
Obtaining information related to the movement of the imaging device, shifting the shift element so that an imaging region by the imaging device precedes the direction of the movement, and causing the imaging device to perform second imaging; Processing for obtaining a second image generated by the second imaging;
In the second image, a process of detecting a second subject image that matches at least part of the first subject image included in the first image;
An imaging control program comprising: processing for causing the imaging apparatus to perform recording imaging in response to detection of the second subject image.