JP5789937B2 - Image tracking device and imaging device - Google Patents

Description

  The present invention relates to an image tracking device and an imaging device.

  In the photographed image, an area surrounding the subject to be tracked is determined, color information of the area is stored as a template image, and one tracking area is selected from a plurality of areas determined to be similar to the template image. There is an image tracking device to be determined (see, for example, Patent Document 1).

JP 2008-187331 A

  However, when there are a plurality of subjects with similar color information on the screen, the above-described determination method tends to cause erroneous tracking.

  An object of the present invention is to provide an image tracking device and an imaging device with less false tracking.

The present invention solves the above problems by the following means .

The invention according to claim 1 is similar to the template in a storage unit that stores a subject area including a current focus area in which autofocus is performed as a template, and in a first search range including the current focus area. A first search means for searching for a first region having high characteristics, and a second that has a high similarity to the template in a second search range including the first search range and a region outside the first search range. A second search means for searching for an area; and comparing the first area and the second area with the template to obtain a degree of similarity to each of the templates; and one of the first area and the second area If the similarity degree is high, the area with the higher similarity degree is given priority. If the similarity degree is the same, the first area is given priority, and then auto And area determination means for determining a next focus area for the scan, an image tracking apparatus comprising the.
According to a second aspect of the present invention, in the image tracking device according to the first aspect, in setting the first search range, a next focus area is determined from an autofocus area of the first area or the second area. In this case, the image tracking device is characterized in that the first search range is set around the autofocus area.
According to a third aspect of the present invention, in the image tracking device according to the first or second aspect, the search range is set when a next focus area is not determined from the autofocus area of the first area or the second area. Is an image tracking device characterized by not changing the first search range.
A fourth aspect of the present invention is an imaging apparatus including the image tracking device according to any one of the first to third aspects.
The invention according to claim 5 determines at least one AF area among a plurality of AF areas and performs focus control using the determined focus state of the AF area, and the determined AF to set a wider first search range than the region of the AF area determined comprises a region of the area, a second search range including the first search range and the first search region outside the range By searching the first search range, the setting unit detects an image of the first area having high similarity to the reference image including the area information of the AF area, and searches for the second search range. A search unit that detects an image of a second region having high similarity to the reference image, and the control unit has a degree of similarity between the image of the first region and the reference image of the second region. Similarity between the image and the reference image The AF area is determined using information on the first area, and the degree of similarity between the image in the second area and the reference image is the degree of similarity between the image in the first area and the reference image The AF area is determined using information on the second area, and the degree of similarity between the image in the first area and the reference image and the degree of similarity between the image in the second area and the reference image Are equal, the AF area is determined using information of the first area .
A sixth aspect of the present invention is the imaging apparatus according to the fifth aspect, further comprising a focus detection unit that detects a focus state of at least one of the AF areas, and the control unit is configured to detect the first region. When the degree of similarity between the image and the reference image is higher than the degree of similarity between the image in the second area and the reference image, the AF area is determined using the focus state of the AF area included in the first area If the similarity between the image in the second area and the reference image is higher than the similarity between the image in the first area and the reference image, the focus state of the AF area included in the second area is determined. The AF apparatus is used to determine the AF area.
The invention according to claim 7 is the imaging device according to claim 5 or 6, wherein the storage unit stores the reference image including an image of the area of the AF area determined by the control unit. It is an imaging device characterized by including.

  According to the present invention, it is possible to provide an image tracking device and an imaging device with less false tracking.

It is a conceptual block diagram of the camera to which one Embodiment of the focus detection apparatus which concerns on this invention is applied. It is a figure which shows AF area set in the imaging | photography screen of an imaging optical system. It is a flowchart which shows the AF process of this embodiment. It is a flowchart which shows the initial tracking control of the tracking control part in S103 of FIG. It is a figure explaining the method in which a tracking control part tracks a to-be-photographed object. It is a flowchart which shows the tracking calculation process of S104 of FIG. It is a flowchart which shows the area determination process of S109 of FIG.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a conceptual configuration diagram of a camera 1 to which an embodiment of a focus detection apparatus according to the present invention is applied.
The camera 1 according to the present embodiment is a so-called digital single-lens reflex camera that includes a camera body 10 and an imaging lens 20 that can be attached to and detached from the camera body 10.
The imaging lens 20 includes a plurality of lens groups L1, L2, and L3 that form an imaging optical system, and a lens driving motor 22 inside the lens barrel 21. The lens unit L2 is a focus adjustment lens that can move in the direction of the optical axis OA and adjust the image formation position, and is driven to move the focus by the lens driving motor 22. The lens driving motor 22 is controlled by a control device 30 provided in the camera body 10 described later.

  The camera body 10 includes an image sensor 11, a quick return mirror 12A, a sub mirror 12B, a finder optical system 13, a photometry unit 14, a phase difference AF detection unit 15, an operation member 16, and a control device 30. I have.

  The imaging element 11 is a photoelectric conversion element such as a CCD or CMOS that converts subject light into an electrical signal, and each pixel of red (R), green (G), and blue (B) is arranged in a predetermined arrangement pattern. ing. The imaging device 11 converts the image information of the imaging surface by the imaging optical system of the imaging lens 20 into an electrical signal and images it, and sends an image signal corresponding to color information and luminance information corresponding to each pixel to the control device 30. Output.

  The quick return mirror 12A is provided so as to be movable between an action position interposed in the optical path from the imaging optical system of the imaging lens 20 to the image sensor 11 and a retracted position not interposed in the optical path. The quick return mirror 12A reflects the incident light beam to the finder optical system 13 (finder screen 13A) at the operating position. In addition, a semi-transmission region that transmits part of the incident light beam is formed in part of the quick return mirror 12A.

  The sub mirror 12B is provided on the back side of the quick return mirror 12A. The sub mirror 12 </ b> B reflects the incident light beam transmitted through the semi-transmissive region of the quick return mirror 12 </ b> A toward the phase difference AF detection unit 15.

The finder optical system 13 includes a finder screen 13A, a pentaprism 13B, and an eyepiece lens 13C.
The finder screen 13A is provided at a position optically equivalent to the image sensor 11, and an incident light beam guided by the quick return mirror 12A forms an image. The pentaprism 13B and the eyepiece 13C are configured so that the photographer can visually recognize the subject image formed on the finder screen 13A as an erect image.

The photometric unit 14 includes a photometric lens 14A and a photometric sensor 14B.
The photometric lens 14A guides the subject image formed on the finder screen 13A via the pentaprism 13B to the photometric sensor 14B.
Similar to the image sensor 11, the photometric sensor 14B has red (R), green (G), and blue (B) pixels arranged in a predetermined arrangement pattern, and is imaged by a lens optical system. The image information of the surface is captured.
Then, the photometric sensor 14B outputs a photometric signal corresponding to the color information and luminance information corresponding to each pixel to the control device 30. The control device 30 detects the brightness of the imaging surface based on the photometric signal from the photometric sensor 14B and determines the exposure. Moreover, the control apparatus 30 performs tracking control by the tracking control part 36 mentioned later based on the photometry signal from the photometry sensor 14B.

  The operation member 16 is configured by various switches (not shown) for operating the camera 1. For example, a mode selection switch for selecting the operation mode of the camera 1, an area selection switch for selecting an AF (autofocus) area, a release button for instructing start of focus adjustment (AF) and photographing, etc. are included. It is.

  FIG. 2 is a diagram showing the AF area a set in the photographing screen 17 of the photographing optical system. In the present embodiment, the phase difference AF detection unit 15 is provided with 51 AF areas (focus detection areas, ranging points) at predetermined positions in the imaging region 17.

  Further, the phase difference AF detection unit 15 includes a line sensor arranged corresponding to the AF area column. The line sensor is a pair of charge storage type photoelectric conversion element arrays (CCD line sensor or the like). A pair of light beams that have been divided into two by passing through different regions of the imaging lens 20 are collected by a condenser lens, and a pair of subject images are re-formed on the line sensor. The line sensor S accumulates charges for a set time for the pair of subject images, and sends an AF image detection signal corresponding to the luminance distribution of the pair of subject images to the control device 30 described later. Output. The charge accumulation time of the line sensor is controlled by an AF-CCD control unit 31 in the control device 30 described later.

The control device 30 is configured by a CPU, a memory, and the like, and performs operation control of the camera 1. Further, information necessary for these processes and controls is temporarily stored.
As described above, the control device 30 detects the brightness of the imaging surface based on the photometric signal from the photometric sensor 14B and determines the exposure.

  Further, as shown in FIG. 1, the control device 30 includes an AF-CCD control unit 31, a defocus calculation unit 32, a focus area position determination unit 33, and a lens as functional units related to focus adjustment (AF). A drive amount calculation unit 34, a lens drive control unit 35, and a tracking control unit 36 are provided.

  The AF-CCD control unit 31 reads an AF image detection signal from the line sensor S provided in the phase difference AF detection unit 15 and outputs the AF image detection signal to the defocus calculation unit 32.

  The tracking control unit 36 uses a template image (reference) as an image corresponding to the tracking target position manually designated by the photographer or the tracking target position automatically set by the camera 1 in the object scene image captured by the photometric sensor 14B. (Image) is stored in a memory (not shown). The position of the in-focus target object is recognized by searching an image area that matches or is similar to the template image from images that are repeatedly photographed thereafter.

  The defocus calculation unit 32 is based on the AF image detection signal of the line sensor S read out by the AF-CCD control unit 31 with respect to the image area where the target target is recognized by the tracking control unit 36. Then, a defocus amount representing a focus adjustment state (a focus shift amount) of the imaging lens 20 is calculated.

  The focus area position determination unit 33 determines an AF area where the imaging lens 20 is finally focused based on the defocus amount calculated by the defocus calculation unit 32.

The lens driving amount calculation unit 34 is based on the defocus amount calculated by the defocus calculation unit 32 corresponding to the AF area determined by the user or the focus area position determination unit 33. The driving amount of the focus adjustment lens (lens group L2) of the lens is calculated.
Here, the lens drive amount is calculated by calculating the lens target position that is the target of the drive position of the focus adjustment lens (lens group L2). The lens target position corresponds to the position of the focus adjustment lens (lens group L2) where the defocus amount is almost zero.

  The lens drive control unit 35 controls the driving of the lens drive motor 22 of the imaging lens 20 based on the lens drive amount calculated by the lens drive amount calculation unit 34, that is, the lens target position with respect to the focus adjustment lens (lens group L2). Output a signal. In response to this drive control signal, the lens drive motor 22 drives the focus adjustment lens (lens group L2) and moves it to the lens target position, thereby performing focus adjustment.

The camera 1 configured as described above is controlled by the control device 30 and performs AF and photographing according to the following flowchart.
FIG. 3 is a flowchart showing the AF processing of the present embodiment.
First, the control device 30 determines whether or not the switch (SW) is turned on by operating the release button (operation member) 16 in S101. SW corresponds to the half-press start of the camera 1, and the AF operation is started when the camera 1 is switched from OFF to ON.

When S101 is YES, in S102, the control device 30 determines whether or not the AF operation is a first AF operation. That is, immediately after the SW is switched from OFF to ON, the stage where no AF calculation result is output is the first AF operation.
In the case of the first AF operation (S102, YES), the process proceeds to S103, and the control device 30 performs the first tracking control by the tracking control unit 36 in S103. Details of the initial tracking control will be described with reference to a flowchart shown in FIG.
S104, which proceeds in S102 and NO, is a case where tracking control is already performed after the AF operation is started. The control device 30 performs a tracking calculation subroutine by the tracking control unit 36. The tracking calculation subroutine will be described with reference to the flowchart shown in FIG.

In S105 following S103 and S104, the control device 30 instructs the AF-CCD control unit 31 to cause the phase difference AF detection unit 15 (line sensor) to perform accumulation control. Note that this processing may be performed in parallel with the operations of S102 to S104.
In S <b> 106, the control device 30 performs A / D conversion of the charge accumulated in the phase difference AF detection unit 15 (line sensor) by the AF-CCD control unit 31 and reads a signal.

In step S <b> 107, the control device 30 obtains an image shift amount by correlation calculation from the read charge signal, and the defocus calculation unit 32 calculates the defocus amount.
In S108, the control device 30 determines whether or not the calculation of the defocus amount for the predetermined number of areas has been completed. The predetermined number here is the number of AF areas corresponding to the tracking area. If all the areas included in the phase difference AF detection unit 15 can be calculated at high speed, all the AF areas may be calculated.

In the case of S108, YES, in S109, the control device 30 performs a subroutine of area determination processing by the focus area position determination unit 33. Details will be described with reference to a flowchart shown in FIG.
In S <b> 110, the control device 30 performs lens drive amount calculation by the lens drive amount calculation unit 34 based on the determined defocus amount of the adopted area.
In S111, the control unit drives the lens drive motor 22 by the lens drive control unit 35 according to the result of the lens drive amount calculation in S110, and moves the focus adjustment lens (lens group L2) to the lens target position, thereby focusing. Make adjustments.

  FIG. 4 is a flowchart showing the initial tracking control of the tracking control unit 36 in S102 of FIG. FIG. 5 is a diagram illustrating a method in which the tracking control unit 36 tracks a subject.

In S201, the subject color information of the image in the area corresponding to the AF area (AF area a1 shown in FIG. 5) in the tracking initial image is stored.
In S202, an area indicating color information similar to the color information of the AF area a1 is detected in the periphery of the AF area a1 in the initial tracking image.
In S203, the detected area is set as an initial subject area A.
In S204, the color information of the subject area A is stored in the memory as a template image used for the subsequent tracking process.
In S205, an area obtained by enlarging the subject area A by predetermined pixels is set as a search range. Here, it is assumed that two types of search ranges are set, a narrow search range n and a search range w wider than the narrow search range n.

FIG. 6 is a flowchart showing the tracking calculation process in S104 of FIG.
In step S301, a region having the same size as the subject region A of the template image is sequentially cut out from the search range n and the search range w, the cut-out image and the template image are compared, and a difference in color information is calculated for each pixel.
In S302, the area B having the smallest color information difference is searched, and that area is determined as a new tracking area. Here, the image information of the template image may be updated using the image information of the new tracking subject area.
In the present embodiment, as shown in FIG. 4, the tracking region Bn for the region B having the smallest color information difference, that is, the region having high similarity to the template image, from the narrow search range n and the wide search range w, respectively. The tracking area Bw is determined.

  In S303, an area enlarged by predetermined pixels around the new tracking area is set as a new search range. When a plurality of new tracking areas are output in S302, a search range is set for each.

FIG. 7 is a flowchart showing the area determination process in S109 of FIG.
In S401, it is determined whether there is similarity between the tracking area Bn and the tracking area Bw determined in the narrow search range n and the wide search range w, respectively. If there is similarity in one of the tracking areas, it is determined that there is similarity (S401, YES). For this similarity, the real number may be divided by a threshold value, or the threshold value may be divided for each range to indicate the degree of similarity.

When there is similarity (S401, YES), the similarities bn and bw of the tracking areas Bn and Bw are compared in S402, and the similarity bn of the tracking area Bn is equal to or greater than the similarity bw of the tracking area Bw. It is determined whether or not. When the similarity bn of the tracking area Bn is greater than or equal to the similarity bw of the tracking area Bw (S402, YES), the process proceeds to S403.
In S403, a determination is made to determine an area to be used for focus adjustment from the AF area corresponding to the tracking area Bn. That is, the AF area corresponding to the tracking area Bn is searched for an area that satisfies a predetermined threshold from the previous focus state and has a minimum defocus amount.
As described above, in this embodiment, when the similarity bn of the tracking area Bn and the similarity bw of the tracking area Bw are approximately the same, the tracking area Bn in the narrow search range n is preferentially searched for the AF area.

In S404, it is determined whether or not an employment area has been determined.
When the adopted area is not determined in S404 (S404, NO), in S405, a determination is made to determine an area used for focus adjustment from the AF areas corresponding to the tracking area Bw. From the defocus amount of the AF area corresponding to the tracking area Bw, an area that satisfies a predetermined threshold from the previous focus state and has the minimum defocus amount is searched. In step S406, it is determined whether or not an employment area has been determined.

  If the adopted area is not determined in S406 (S406, NO), or if it is determined in S401 that there is no similarity to the template of the tracking area Bn and the tracking area Bw, an AF area other than the tracking areas Bn and Bw is determined in S407. Make a hiring decision. If the process proceeds from S406, that is, if the tracking area is determined, this process may be performed in a preliminary manner. For example, it may be omitted if the calculation time is insufficient during continuous shooting.

  In S408, it is determined whether or not an employment area has been determined. If the adopted AF area is not determined in S408, the adopted AF area is set as the previous adopted AF area in S409. The adopted area does not necessarily have to be the previous adopted AF area and may be undecided, or a process of determining the adopted AF area again from the remaining AF areas may be performed.

  After S409 is completed and if YES in S404, S406, and S408, in S410, information on the AF area that is the adopted AF area among the plurality of tracking areas is output to the tracking function. The information includes the position of the AF area, which determination has passed and the adopted AF area has been determined. This is used for the purpose of referring to the tracking function to select from a plurality of tracking areas to be used as the next reference, or to determine which area the search range is set as a reference.

  For example, in the search range, when the adopted AF area is determined from the AF area of the tracking area in S403 or S405, the search range is set around the adopted AF area, and the adopted AF is selected from the AF area other than the tracking area in S407. When the area is determined, it is determined that the search range is not changed from the previous time.

When the similarity bn of the tracking area Bn is smaller than the similarity bw of the tracking area Bw in S402 (S402, NO), the process proceeds to S411.
In S411, a determination is made to determine an area used for focus adjustment from the AF area corresponding to the tracking area Bw. That is, the AF area corresponding to the tracking area Bw is searched for an area that satisfies a predetermined threshold from the previous focus state and has a minimum defocus amount.
In S412, it is determined whether or not an employment area has been determined.
If the adopted area is not determined in S412 (S412, NO), in S413, a determination is made to determine an area used for focus adjustment from the AF areas corresponding to the tracking area Bn. From the defocus amount of the AF area corresponding to the tracking region Bn, an area that satisfies a predetermined threshold from the previous focus state and has the smallest defocus amount is searched. In step S414, it is determined whether or not an employment area has been determined.

  When the adoption area is not determined in S414 (S414, NO), adoption determination of AF areas other than the tracking areas Bn and Bw is performed in S415.

  In S416, it is determined whether or not an employment area has been determined. If the adopted AF area is not determined in S416, the adopted AF area is set as the previous adopted AF area in S417. The adopted area does not necessarily have to be the previous adopted AF area and may be undecided, or a process of determining the adopted AF area again from the remaining AF areas may be performed. After S417 ends, the process proceeds to S410.

As described above, this embodiment has the following effects.
In the present embodiment, in the tracking calculation process, an area having high similarity to the template image is searched for in a narrow search range and a wide search range that are spread around the template image. Then, the AF area to be tracked next time is searched by giving priority to the area searched in the narrow area. As a result, when similar subjects are spread over a range where the distance from the camera is substantially equal, it becomes difficult for false tracking to occur, and it is possible to continue focusing while maintaining the continuity and capturing ability of the subject. .

  1: camera, 30: image tracking device, 15: phase difference AF detection device, 31: AF-CCD control unit, 32: defocus calculation unit, 33: focus area position determination unit, 34: lens drive amount calculation unit, a1 : Focus area, n: narrow search range, w: wide search range, Bn: tracking area, Bw: tracking area

Claims (7)

A storage unit for storing, as a template, a subject area including a current focus area where autofocus is performed;
First search means for searching for a first region having high similarity to the template within a first search range including the current focus area;
A second search means for searching for a second region having high similarity to the template within a second search range including the first search range and a region outside the first search range;
The first region and the second region are compared with the template to determine the degree of similarity with each template, and when the degree of similarity of one of the first region and the second region is high, the degree of similarity is If the higher area is prioritized and the degree of similarity is the same, area determining means that prioritizes the first area and then determines the next focus area for auto-focusing;
An image tracking apparatus comprising: The image tracking device according to claim 1,
The first search range is set by setting the first search range around the auto focus area when the next focus area is determined from the auto focus area of the first area or the second area. An image tracking device characterized by that. The image tracking device according to claim 1 or 2,
The image tracking apparatus according to claim 1, wherein the search range is set such that the first search range is not changed when a next focus area is not determined from the autofocus area of the first area or the second area.   An imaging device comprising the image tracking device according to any one of claims 1 to 3. A control unit that determines at least one of the AF areas among a plurality of AF areas, and performs focus control using the determined focus state of the AF area;
A first search range including the determined AF area and wider than the determined AF area, and a second search including the first search range and an area outside the first search range. A setting section for setting the range,
By searching the first search range, an image of a first area having high similarity to a reference image including information on the area of the AF area is detected, and the reference is obtained by searching the second search range. A search unit that detects an image of the second region having high similarity to the image,
When the degree of similarity between the image of the first area and the reference image is higher than the degree of similarity between the image of the second area and the reference image, the control unit uses the information of the first area to perform the AF An area is determined, and when the degree of similarity between the image of the second area and the reference image is higher than the degree of similarity between the image of the first area and the reference image, the information of the second area is used to perform the AF An area is determined , and when the similarity between the image of the first area and the reference image and the similarity between the image of the second area and the reference image are equal, the information of the first area is used to perform the AF. An imaging apparatus characterized by determining an area . An imaging apparatus according to claim 5, wherein
A focus detection unit that detects a focus state of at least one of the AF areas;
When the similarity between the image in the first area and the reference image is higher than the similarity between the image in the second area and the reference image, the control unit determines whether the AF area included in the first area Determine the AF area using the focus state,
When the degree of similarity between the image in the second area and the reference image is higher than the degree of similarity between the image in the first area and the reference image, the focus state of the AF area included in the second area is used. An image pickup apparatus that determines the AF area. The imaging apparatus according to claim 5 or 6, wherein
An image pickup apparatus comprising: a storage unit that stores the reference image including an image of the AF area determined by the control unit.

Images