JP2016218106A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device that sets an AF area tracking a movement of a subject of a tracking target with high accuracy.SOLUTION: An imaging device comprises: an imaging unit that continuously shoots a subject in a frame unit to generate a shot image; a display unit that displays in real time the shot image generated by the imaging unit; and a control unit that displays an AF frame indicative of an area to be automatically focused with the AF frame overlapped on the shot image to be in real time displayed on the display unit. The control unit is configured to detect a motion m of the subject from the shot images N-2 and N-1 of the previous frame; predict a position Pn of a specific subject in the shot image N of a current frame on the basis of the detected motion frame; and display the AF frame in the shot image to be displayed on the display unit on the basis of the predicted position.SELECTED DRAWING: Figure 5

Description

  The present disclosure relates to an imaging apparatus that sets an autofocus area following the movement of a subject to be tracked.

  2. Description of the Related Art Conventionally, there is an imaging apparatus that tracks a desired subject and sets an autofocus (AF) region that performs a focusing operation at a position corresponding to the position of the subject that changes every moment (for example, Patent Document 1).

  The camera of Patent Literature 1 detects a candidate area including a tracking target in the shooting screen using at least one of color information and luminance information in the shooting screen, and includes the candidate area based on the size of the candidate area. One focus area is selected from a plurality of focus areas (AF areas) to be subjected to focus adjustment, and an AF frame is displayed on the photographing screen for the selected focus area. By displaying the AF frame in this way, it is possible to reduce frequent switching of the display position of the AF frame (that is, the position of the focus area) for a large subject.

JP 2014-016630 A

  In the camera of Patent Document 1, the AF frame currently displayed in the shooting screen is displayed at a position determined based on the image of the previous frame. For this reason, the actual position of the current subject may deviate from the display position of the AF frame (that is, the position of the AF area).

  The present disclosure provides an imaging apparatus that sets the display position of an AF frame by accurately following the movement of a subject to be tracked.

  In an aspect of the present disclosure, an imaging apparatus capable of performing an automatic focusing operation following a movement of a specific subject is provided. An imaging device includes an imaging unit that continuously captures a subject in units of frames to generate a captured image, a display unit that displays a captured image generated by the imaging unit in real time, and an imaging that is displayed in real time on the display unit And a control unit that displays an AF frame that is superimposed on the image and indicates a region to be automatically focused. The control unit detects the movement of the specific subject from the captured image of the previous frame, predicts the position of the specific subject in the captured image of the current frame based on the detected motion, and displays based on the predicted position. An AF frame is displayed in the captured image displayed on the screen.

  The imaging apparatus according to the present disclosure predicts the position of a subject to be tracked and displays an AF frame. For this reason, it is possible to display the AF frame by accurately following the movement of the subject to be tracked.

The figure which shows the structure of the digital camera which concerns on this indication Rear view of digital camera Diagram explaining AF frame in digital camera Flowchart showing the operation of tracking AF in the first embodiment The figure for demonstrating the setting of the tracking area | region based on the predicted position of a tracking object Flowchart showing the operation of tracking AF in the second embodiment

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The inventor (s) provides the accompanying drawings and the following description in order for those skilled in the art to fully understand the present disclosure, and is intended to limit the subject matter described in the claims. Not what you want.

  Hereinafter, an embodiment of an imaging device according to the present disclosure will be described with reference to the drawings.

(Embodiment 1)
1. Configuration of Digital Camera An example of the electrical configuration of the digital camera according to Embodiment 1 will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of the digital camera 100. The digital camera 100 is an imaging device that captures an object image formed by an optical system 110 including one or a plurality of lenses with a CCD 140.

  The image data generated by the CCD 140 is subjected to various processes by the image processing unit 160 and stored in the memory card 200. Hereinafter, the configuration of the digital camera 100 will be described in detail.

  The optical system 110 includes a zoom lens and a focus lens 115. The subject image can be enlarged or reduced by moving the zoom lens along the optical axis. The focus of the subject image can be adjusted by moving the focus lens 115 along the optical axis.

  The lens driving unit 120 drives various lenses included in the optical system 110. The lens driving unit 120 includes, for example, a zoom motor that drives the zoom lens and a focus motor that drives the focus lens 115.

  The diaphragm 300 is a mechanism that adjusts the amount of transmitted light by adjusting the size of the light opening in accordance with a user setting or automatically.

  The shutter 130 is a means for shielding the light transmitted through the CCD 140. The shutter 130, together with the optical system 110 and the diaphragm 300, constitutes an optical system unit that controls optical information indicating a subject image.

  The CCD 140 captures the subject image formed by the optical system 110 and generates image data at a predetermined frame rate. The CCD 140 includes a color filter, a light receiving element, and an AGC (Auto Gain Controller). The light receiving element converts the optical signal collected by the optical system 110 into an electrical signal, and generates image information. The AGC amplifies the electric signal output from the light receiving element. The CCD 140 further includes a drive circuit for performing various operations such as exposure, transfer, and electronic shutter. Details will be described later.

  An ADC (A / D converter) 150 converts analog image data generated by the CCD 140 into digital image data.

  The image processing unit 160 performs various processes on the digital image data generated by the CCD 140 and converted by the ADC (A / D converter) 150. The image processing unit 160 operates under the control of the controller 180. The image processing unit 160 generates image data to be displayed on the display monitor 220 or generates image data to be stored in the memory card 200. For example, the image processing unit 160 performs various processes such as gamma correction, white balance correction, and flaw correction on the image data generated by the CCD 140. Further, the image processing unit 160 converts the image data generated by the CCD 140 into H.264. It compresses by the compression format etc. based on H.264 standard or MPEG2 standard. The image processing unit 160 can be realized by a DSP (Digital Signal Processor), a microcomputer, or the like.

  The controller 180 is a control unit that controls the entire digital camera 100. The controller 180 can be realized by a semiconductor element or the like. The controller 180 may be configured only by hardware, or may be realized by combining hardware and software. The controller 180 can be realized by a microcomputer, CPU, MPU, ASIC, FPGA, or the like.

  The buffer 170 functions as a work memory for the image processing unit 160 and the controller 180. The buffer 170 can be realized by, for example, a DRAM (Dynamic Random Access Memory), a ferroelectric memory, or the like.

  The card slot 190 is detachable from the memory card 200. The card slot 190 can be mechanically and electrically connected to the memory card 200.

  The memory card 200 includes a flash memory, a ferroelectric memory, and the like, and can store data such as an image file generated by the image processing unit 160.

  The built-in memory 240 is composed of a flash memory or a ferroelectric memory. The built-in memory 240 stores a control program for controlling the entire digital camera 100 and the like.

  The operation member 210 is a generic name for a user interface that receives an operation from a user. The operation member 210 includes, for example, a selection button, a determination button, a lever, a dial, a touch panel and the like that receive an operation from the user.

  The display monitor 220 can display an image (live view image) indicated by image data generated by the CCD 140 and an image (reproduced image) indicated by image data read from the memory card 200. The display monitor 220 can also display various menu screens for performing various settings of the digital camera 100. The display monitor 220 is composed of a liquid crystal display device or an organic EL display device.

  FIG. 2 is a view showing the back surface of the digital camera 100. As shown in FIG. 2, the operation member 210 includes a release button 211, a selection button 213, a determination button 214, a moving image recording button 217, and the like. When the operation member 210 receives an operation by the user, the operation member 210 transmits various instruction signals to the controller 180.

  The release button 211 is a two-stage push button. When the release button 211 is pressed halfway by the user, the controller 180 executes autofocus control (AF control), auto exposure control (AE control), and the like. When the release button 211 is fully pressed by the user, the controller 180 records image data captured at the timing of the pressing operation on the memory card 200 or the like as a recorded image.

  The selection button 213 is a push button provided in the up / down / left / right directions. The user can move the cursor or select various condition items displayed on the display monitor 220 by pressing any direction of the selection button 213.

  The decision button 214 is a push button. When the determination button 214 is pressed by the user while the digital camera 100 is in the shooting mode or the playback mode, the controller 180 displays a menu screen on the display monitor 220. The menu screen is a screen for setting various conditions for shooting / playback. When the determination button 214 is pressed when a setting item for various conditions is selected, the controller 180 finalizes the setting of the selected item.

2. Operation of Digital Camera The digital camera 100 of the present embodiment has an autofocus mode in which a focusing operation is automatically performed and a manual mode in which a focusing operation is performed manually. Furthermore, the digital camera 100 has a tracking AF mode in which autofocus is performed following a specific subject as the autofocus mode.

  FIG. 3 is a diagram illustrating an example of a captured image (live view image) displayed on the display monitor 220. The digital camera 100 according to the present embodiment displays an AF frame 20 indicating an AF area following the movement of a specific subject 10 as a tracking target. Here, the specific subject to be tracked is determined, for example, when the user designates the specific subject in the image displayed on the display monitor 220 by touch or key operation. The digital camera 100 according to the present embodiment predicts the position of a specific subject in the captured image from the motion vector of the specific subject to be tracked, and determines an AF frame based on the predicted position. As a result, the deviation between the position of the AF frame and the actual position of the tracking target can be reduced.

  Hereinafter, the tracking AF operation in the digital camera 100 will be described with reference to FIGS. 4 and 5. FIG. 4 is a flowchart showing the tracking AF operation in the digital camera 100. FIG. 5 is a diagram for explaining the setting of the tracking area (the position of the AF frame) based on the predicted position of the specific subject that is the tracking target. Processing described below is mainly executed by the controller 180 of the digital camera 100. In the following, the tracking AF operation for time t (current frame n) will be described. Further, it is assumed that a specific subject to be tracked has already been designated by the user.

  The controller 180 detects the position of the specific subject in the captured image N-1 of the previous frame (frame n-1) of the current frame (frame n) and the captured image of the second previous frame (frame n-2). A motion vector (movement amount) is calculated from the position of the specific subject at N-2, and the calculation result is stored in the built-in memory 240 (S11).

  Next, the controller 180 calculates a motion vector (from the captured image N-1 of the previous frame (frame n-1) and the captured image N-2 of the previous frame (frame n-2) ( The amount of movement) information is read from the built-in memory 240 (S12).

  The controller 180 predicts the position of the specific subject in the captured image N of the current frame based on the acquired motion vector (movement amount) (S13).

Specifically, a predicted position Pn of a specific subject in the current frame captured image N is obtained by the following equation (1).
Pn (xn, yn) = Pn-1 (xn-1, yn-1) + m (Δx, Δy) (1)
Pn-1: position of a specific subject in a captured image of one frame before m: position of a specific subject in a captured image one frame before (Pn-2) from a position (Pn-2) of a specific subject in a captured image of two frames before -1) Motion vector to

  In the image (live view image) displayed on the display monitor 220, the controller 180 displays the AF frame 20 at a position corresponding to the predicted position Pn in the captured image (S14).

  In this way, in the image displayed on the display monitor 220, the AF frame is displayed at a position corresponding to the position Pn predicted from the captured image of the past frame, so that the position of the specific subject displayed on the display monitor 220 and the AF are displayed. The shift of the frame position can be reduced. Therefore, the user can visually recognize the AF frame without feeling uncomfortable.

  Thereafter, the controller 180 performs a tracking process based on the color information with the predicted position Pn as the center (S15).

  Specifically, the controller 180 detects a specific subject 10 based on the color information in a predetermined region 30 centered on the predicted position Pn. At this time, the controller 180 displays the AF frame 20 at a position corresponding to the region 30 in the captured image of the current frame n on the display monitor 220. Then, the controller 180 detects the specific subject 10 based on the color information in the region 20, sets a predetermined region including the detected specific subject 10 as the AF region, and contrast based on the value of the pixel included in the AF region. Perform AF.

  In this way, by performing the tracking process (detection of a specific subject) around the predicted position Pn, the tracking process is performed in an area where the tracking target is expected to move, and the actual tracking target is detected. The tracking target can be detected at a position closer to the position. In addition, the possibility of losing sight of the tracking target can be reduced. Therefore, tracking accuracy can be improved.

3. Effects As described above, the digital camera 100 according to the present embodiment can perform an automatic focusing operation following the movement of a specific subject. The digital camera 100 captures a subject continuously in units of frames and generates a captured image, a display monitor 220 that displays a captured image generated by the CCD 140 in real time, and a display monitor 220 that displays the captured image in real time. And a controller 180 that displays an AF frame 20 indicating a region to be automatically focused by being superimposed on the captured image. The controller 180 detects the motion of the specific subject 10 from the captured images N-1 and N-2 of the previous frame, and determines the position Pn of the specific subject 10 in the captured image N of the current frame based on the detected motion. Based on the predicted position Pn, the AF frame 20 is displayed in the captured image displayed on the display monitor 220.

  As described above, the digital camera 100 predicts the position of the specific subject in the current frame image based on the motion vector calculated from the past frame image, and displays the AF frame 20 based on the predicted position Pn. Thereby, it is possible to reduce the deviation between the position of the specific subject displayed on the display monitor 220 and the position of the AF frame.

  Further, the controller 180 may detect the specific subject 10 for the following operation in the region 30 based on the predicted position Pn. Thereby, since the tracking operation is performed in the region where the tracking target is expected to move, the tracking target can be detected at a position closer to the actual position of the tracking target. In addition, the possibility of losing sight of the tracking target can be reduced. Therefore, tracking accuracy can be improved.

(Embodiment 2)
In the first embodiment, the position of a specific subject is always predicted based on the motion vector. However, when there is no regularity in the movement of a specific subject, the method according to the first embodiment does not always realize accurate tracking. In this embodiment, the position of a specific subject is predicted only when the same movement amount is detected a predetermined number of times in succession.

  Hereinafter, the operation of the digital camera 100 of the present embodiment will be described with reference to the flowchart of FIG. The configuration of the digital camera 100 is as described in the first embodiment.

  The controller 180 specifies the captured images of two consecutive frames (for example, the frame n-1 and the frame n-2, the frame n-2 and the frame n-3,..., The frame n-5 and the frame n-6). A motion vector (amount of movement) is calculated from the position of the subject, and the calculation result is stored in the built-in memory 240 (S20).

  Next, the controller 180 acquires information on a predetermined number (for example, 5) of motion vectors obtained previously from the built-in memory 240 (S21). Here, information on motion vectors calculated from the respective sets of frame n-1 and frame n-2, frame n-2 and frame n-3,..., Frame n-5 and frame n-6 is acquired.

  Based on the acquired motion vector information, the controller determines whether or not the same amount of movement has been detected for a predetermined number of times (for example, five times) (S22). Here, the movement amount may be determined to be the same not only when the movement amounts are exactly the same, but also when the variation in the movement amount is within a predetermined range. For example, when the difference between each movement amount and the average value of the movement amounts is within a predetermined value, it may be determined that the movement amounts are the same.

  The fact that the same movement amount has been detected for a predetermined number of times is considered to mean that the specific subject is moving at a constant speed (movement amount). Therefore, when the same movement amount is detected for a predetermined number of times (YES in S22), the controller 180 determines the specific subject in the captured image of the current frame based on the motion vector (movement amount) obtained in the previous frame. The position is predicted (S23). The predicted position is obtained by the above-described equation (1).

  Then, the controller 180 displays the AF frame 20 at a position corresponding to the predicted position Pn in the image (live view image) displayed on the display monitor 220 (S24), and based on the color information centered on the predicted position Pn. A tracking process is performed (S25).

  On the other hand, the fact that the same movement amount has not been detected for a predetermined number of times is considered that the specific subject is not moving at a constant speed (movement amount), and it is considered that the reliability of the motion vector is low. Therefore, when the same movement amount is not continuously detected a predetermined number of times (NO in S22), position prediction based on the motion vector is not performed. That is, the controller 180 displays the AF frame at a position corresponding to the position of the specific subject detected by color tracking one frame before in the image displayed on the display monitor 220 (S26). Further, the controller 180 detects a specific subject based on the color information from the entire captured image (S27).

  According to the present embodiment, the position of the specific subject is predicted using the motion vector only when it is highly likely that the specific subject is moving at a constant speed (movement amount). The tracking accuracy can be improved, and more accurate tracking operation can be realized.

(Other embodiments)
As described above, the first embodiment has been described as an example of the technique disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to an embodiment in which changes, replacements, additions, omissions, and the like are appropriately performed. Moreover, it is also possible to combine each component demonstrated in the said Embodiment 1, and it can also be set as a new embodiment. Therefore, other embodiments will be exemplified below.

  (1) When a specific subject is detected based on the color information, a change in the size of the specific subject can also be recognized. When the size of the specific subject is small, it is considered that the specific subject is moving away from the digital camera 100. Therefore, in the above embodiment, the value of m in Equation (1) may be corrected based on a change in the size of a specific subject. Specifically, when the size of a specific subject is small, the value of m may be corrected to be smaller.

(2) In the above embodiment, the subject tracking process based on the color information is performed in steps S15, S25, and S27 of the flowchart of FIG. 4, but instead of the color information,
Alternatively, the subject may be tracked based on the luminance information in addition to the color information. Alternatively, tracking of the subject may be performed based on the motion vector instead of or in addition to the color information. Alternatively, subject tracking may be performed by appropriately combining color information, luminance information, and motion vectors.

  (3) In the above embodiment, an example in which the controller 180 calculates the motion vector has been described. However, a dedicated processor that calculates the motion vector may be provided separately from the controller 180. Thereby, the load of the controller 180 can be reduced, and higher speed processing can be expected.

  (4) Although the above embodiment has been described using a digital camera as an example of an imaging apparatus, the imaging apparatus is not limited to this. The idea of the present disclosure can be applied to various imaging devices capable of shooting moving images, such as a digital video camera, a smartphone, and a wearable camera.

  (5) In the above embodiment, the image sensor is constituted by a CCD, but the image sensor is not limited to this. The image sensor may be an NMOS image sensor or a CMOS image sensor.

  (6) The idea of the above embodiment can be applied to both an interchangeable lens camera and a lens-integrated camera.

  As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided. Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above technique. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings and detailed description. Moreover, since the above-mentioned embodiment is for demonstrating the technique in this indication, a various change, replacement, addition, abbreviation, etc. can be performed in a claim or its equivalent range.

  The present disclosure is applicable to an imaging apparatus that can capture a moving image. Specifically, the present invention can be applied to various imaging devices that can shoot moving images, such as digital video cameras, smartphones, and wearable cameras.

10 Subject (tracking target)
20 AF frame 100 Digital camera 110 Optical system 115 Focus lens 120 Lens drive unit 130 Shutter 140 CCD
150 A / D converter 160 Image processor 170 Buffer 180 Controller 190 Card slot 200 Memory card 210 Operation member 220 Display monitor 240 Built-in memory 300 Aperture

Claims (5)

An imaging device capable of automatic focusing operation following the movement of a specific subject,
An imaging unit that continuously captures a subject in units of frames and generates a captured image;
A display unit for displaying a captured image generated by the imaging unit in real time;
A control unit that displays an AF frame indicating an area to be automatically focused, superimposed on a captured image displayed in real time on the display unit,
The control unit detects a motion of a specific subject from the captured image of the previous frame, predicts a position of the specific subject in the captured image of the current frame based on the detected motion, and based on the predicted position And displaying the AF frame in the captured image displayed on the display unit,
Imaging device.   The imaging apparatus according to claim 1, wherein the control unit performs a detection operation of the specific subject in an area based on the predicted position for an automatic focusing operation that follows the movement of the specific subject.   The imaging apparatus according to claim 1, wherein the control unit predicts a position of the specific subject only when a difference in movement amount indicated by a motion vector continuously calculated a predetermined number of times is within a predetermined range. .   The control unit detects a motion vector from a captured image one frame before the current frame and a captured image two frames before, and predicts the position of the specific subject in the captured image of the current frame based on the motion vector. The imaging apparatus according to claim 1.   The control unit performs an automatic focusing operation following the movement of the specific subject by detecting the specific subject based on color information and / or a motion vector of the captured image. 5. The imaging device according to any one of 4.

Images