JP5541396B2 - Image tracking device - Google Patents

Description

  The present invention relates to an image tracking device and an image tracking method.

  Conventionally, when shooting a moving subject, the subject image of the indicated portion in the screen is acquired as a template image, and the position of the subject image similar to the template image is searched (template matching) among images repeatedly shot, An image tracking device that tracks a subject moving within a screen is known (for example, Patent Document 1).

JP 2001-243478 A

  However, it is difficult to accurately set the size and position of the search range according to the correlation calculation result and the motion vector of the moving subject, and there is a problem that a subject different from the moving subject in the image is erroneously detected as a desired moving subject. is there.

An image tracking device according to a first aspect of the present invention includes a setting unit that sets a first search region and a second search region smaller than the first search region in the object scene image generated by the imaging unit, and a tracking First information corresponding to a correlation between reference information corresponding to an object and image information in the first search region in the object scene image, and the second search in the reference information and the object scene image. Based on a comparison result between the calculation unit that calculates second information corresponding to the correlation with the image information in the region, and the first information and the second information, the first information and the second information And a detection unit that detects the position of the tracking target using at least one of the tracking targets .

  According to the present invention, the first calculation result obtained by calculating the correlation with the reference information in the first search area and the second calculation result obtained by calculating the correlation between the reference information in the second search area different from the first search area; Since the position of the target image in the object scene image is detected based on one of the above, erroneous detection of a subject different from the moving subject can be prevented.

1 is a cross-sectional view illustrating a configuration of an imaging device including an image tracking device according to an embodiment; 1 is a block diagram illustrating a configuration of an imaging device including an image tracking device according to an embodiment; Front view of the second image sensor The figure which shows the filter of the pixel of a 2nd image pick-up element. The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The flowchart which shows the image tracking processing of one embodiment Flow chart showing tracking control initial processing Flow chart showing new tracking subject position determination processing The figure for demonstrating the fixed area | region in a modification The figure for demonstrating the 1st search area | region and 2nd search area | region in a modification.

  The focus adjustment state of the photographic lens (in this embodiment, the defocus amount) is detected in a plurality of focus detection areas set in the photographic screen, and the photographic lens is focused based on the defocus amount in one of the areas. The automatic focus adjustment (AF) function to be driven and the image of the subject to be tracked in the photographed image are stored as a template image (reference image). An image pickup apparatus having an image tracking function for tracking a subject to be tracked while searching for a position (template matching), and an image tracking device for tracking the tracking target while driving a photographing lens with the AF function and the image tracking function ( An embodiment of a single lens reflex digital still camera) will be described.

  FIG. 1 shows a configuration of an imaging apparatus (single-lens reflex digital still camera) 1 including an image tracking apparatus according to an embodiment. In FIG. 1, illustration and description of camera devices and circuits not directly related to the present invention are omitted. In the camera 1 according to the embodiment, an interchangeable lens 3 having a photographing lens 8 and a diaphragm 21 is detachably attached to the camera body 2. The camera body 2 is provided with a first image sensor 4 for capturing an object scene image and recording the image. The first image sensor 4 can be constituted by a CCD, a CMOS, or the like. At the time of photographing, the quick return mirror 5 and the sub mirror 6 are retracted to a position outside the photographing optical path indicated by a solid line, the shutter 7 is opened, and a subject image is formed on the light receiving surface of the first image sensor 4 by the photographing lens 8.

  At the bottom of the camera body 2, a focus detection optical system 9 and a distance measuring element 10 for detecting the focus adjustment state of the photographing lens 8 are provided. In this embodiment, an example is shown in which a focus detection method based on a pupil division type phase difference detection method is employed. The focus detection optical system 9 guides the pair of focus detection light fluxes that have passed through the photographing lens 8 to the light receiving surface of the distance measuring element 10 and forms a pair of optical images. The distance measuring element 10 includes, for example, a pair of CCD line sensors, and outputs a focus detection signal corresponding to the pair of optical images. Before shooting, the quick return mirror 5 and the sub mirror 6 are set at positions in the shooting optical path as indicated by broken lines, and the pair of focus detection light beams from the shooting lens 8 are transmitted through the half mirror portion of the quick return mirror 5. Then, the light is reflected by the sub mirror 6 and guided to the focus detection optical system 9 and the distance measuring element 10.

  A finder optical system is provided on the upper part of the camera body 2. Before shooting, the quick return mirror 5 and the sub mirror 6 are in the positions indicated by broken lines, and part of the subject light from the shooting lens 8 is reflected by the quick return mirror 5 and guided to the focusing screen 11, and on the focusing screen 11. A subject image is formed. The liquid crystal display element 12 superimposes and displays information such as a focus detection area mark on the object scene image formed on the focusing screen 11 and displays various shooting information such as an exposure value at a position outside the object scene image. indicate. The object scene image on the focusing screen 11 is guided to the eyepiece window 15 via the penta roof prism 13 and the eyepiece lens 14 so that the photographer can visually recognize the object scene image.

The finder optical system at the top of the camera body 2 is provided with a second image sensor 16 that captures an object scene image for subject tracking and photometry. The object scene image formed on the focusing screen 11 is re-imaged on the light receiving surface of the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. FIG. 3 is a front view showing a detailed configuration of the second image sensor 16. The second imaging element 16 includes a plurality of pixels (photoelectric conversion elements) 40 (in this embodiment, 16 horizontal pixels × 12 vertical pixels) arranged in a matrix. Further, each pixel 40 is divided into three portions 40a, 40b, and 40c as shown in FIG. 4, and primary color filters of red R, green G, and blue B are provided in these portions, respectively. Thereby, the RGB signal of the object scene image can be output for each pixel 40. Although details will be described later, tracking control and exposure calculation are performed based on the object scene image signal imaged by the second image sensor 16. The tracking control and the exposure calculation may be performed based on the scene image signal captured by the first image sensor 4.

  The camera body 2 is also provided with a body drive control device 19 and an operation member 20. The body drive control device 19 is constituted by peripheral components such as a microcomputer, a memory, and an A / D converter, the details of which will be described later, and performs various controls and calculations of the camera 1. The operation member 20 includes a switch and a selector for operating the camera 1 such as a shutter button, a focus detection area selection switch, and a shooting mode selection switch.

  The interchangeable lens 3 is provided with a zooming lens 8a, a focusing lens 8b, a diaphragm 21, a lens drive control device 22, and the like. In this embodiment, the photographing lens 8 is representatively represented by a zooming lens 8a, a focusing lens 8b, and a diaphragm 21, but the configuration of the photographing lens 8 is not limited to the configuration shown in FIG. The lens drive control device 22 includes a microcomputer (not shown) and peripheral components such as a memory, a drive circuit, and an actuator, and performs drive control of the lenses 8a and 8b and the diaphragm 21 and detection of their positions. Lens information such as a focal length and an open aperture value of the interchangeable lens 3 is stored in a memory built in the lens drive control device 22.

  The body drive control device 19 and the lens drive control device 22 communicate via the contact 23 of the lens mount, and transmit information such as a lens drive amount and an aperture value from the body drive control device 19 to the lens drive control device 22. Lens information and aperture information are transmitted from the lens drive control device 22 to the body drive control device 19.

  FIG. 2 shows a detailed configuration of the body drive control device 19. Note that illustration and description of control functions not directly related to the present invention are omitted. The body drive control device 19 includes an element control circuit 19a, an A / D converter 19b, a microcomputer 19c, a memory 19d, and the like. The element control circuit 19 a controls charge accumulation and reading of the second image sensor 16. The A / D converter 19b converts the analog image signal output from the second image sensor 16 into a digital image signal. When the shutter button of the operation member 20 is half-pressed by the photographer, a half-push switch (not shown) is turned on, and as shown in FIG. 1, part of the subject light that has passed through the photographing lens 8 is a quick return indicated by a broken line. The light is reflected by the mirror 5 and guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. While the shutter button is half-pressed, the second imaging element 16 can repeatedly and periodically capture the object scene image.

  The microcomputer 19c constitutes a tracking control unit 19e, an exposure control unit 19f, a focus detection calculation unit 19g, and a lens drive amount calculation unit 19h according to a software form. The memory 19d stores information such as a template image for image tracking and defocus amount, lens information such as focal length, open F value, aperture value, and image shift amount to defocus amount conversion coefficient of the taking lens 8. To do.

The tracking control unit 19e uses a template image as an image corresponding to the tracking target position manually specified by the photographer or the tracking target position automatically set by the camera 1 in the object scene image captured by the second image sensor 16. It is stored in the memory 19d as a (reference image), and the position of the object is recognized by searching an image area that matches or is similar to the template image from images that are repeatedly photographed thereafter. The exposure calculation unit 19f calculates an exposure value based on the image signal captured by the second image sensor 16.

The focus detection calculation unit 19g detects the focus adjustment state of the photographing lens 8, here the defocus amount, based on the focus detection signal corresponding to the pair of optical images output from the distance measuring element 10. Although details will be described later, a plurality of focus detection areas are set in the shooting screen of the shooting lens 8, and the distance measuring element 10 outputs a focus detection signal corresponding to a pair of optical images for each focus detection area. Then, the focus detection calculation unit 19g detects the defocus amount based on the focus detection signal corresponding to the pair of optical images for each focus detection area. When the shutter button of the operation member 20 is half-pressed by the photographer, a half-push switch (not shown) is turned on, and as shown in FIG. 1, part of the subject light that has passed through the photographing lens 8 is a quick return indicated by a broken line. The light is guided to the distance measuring element 10 through the sub-mirror 6 and the focus detection optical system 9 through the half mirror part of the mirror 5, and the focus detection calculation part 19g performs the focus detection calculation. The lens driving amount calculation unit 19h converts the detected defocus amount into a lens driving amount. The focus adjustment state is repeatedly and periodically detected while the shutter button is half-pressed.

  Next, an image tracking operation according to an embodiment will be described. 5 to 7 are diagrams for explaining the image tracking operation according to the embodiment, and FIGS. 8 to 10 are flowcharts illustrating the image tracking process according to the embodiment. The microcomputer 19c manually designates the tracking target subject among the subject images picked up by the second image sensor 16, or the camera automatically sets the tracking target subject, and then presses the shutter button of the operation member 20. When half-pressed, subject tracking processing starts. In the following description, a coordinate system in which the horizontal direction is the x axis and the vertical direction is the y axis in FIGS. 6 and 7 is used.

  The quick return mirror 5 is set in the photographing optical path shown by the broken line in FIG. 1 except when the shutter button is fully pressed to shoot, and part of the subject light incident from the photographing lens 8 is on the focusing screen 11. Is imaged. The object scene image on the focusing screen 11 is guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18, and the object scene image signal is repeatedly output from the second image sensor 16. The

  A plurality of focus detection areas are set on the shooting screen of the shooting lens 8, and an area mark is superimposed on the object scene image on the focusing screen 11 by the liquid crystal display element 12 to display the position of each focus detection area. In this embodiment, as shown in FIG. 5, an example is shown in which focus detection areas 45a to 45g (denoted as a to g in the figure) are set at seven locations in the photographing screen. When an arbitrary area is selected by the focus detection area selection switch of the operation member 20, the mark of the area is lit up.

  As shown in FIG. 5, when the focus detection area 45c is selected by the focus detection area selection switch of the operation member 20, and the shutter button of the operation member 20 is half-pressed in this state, the focus detection area 45c is stored as the first AF area. 19d. Thereby, the subject in the first AF area is designated as the tracking target. In this example, the photographer selects the first AF area and manually designates the subject to be tracked. For example, in a camera having a function of automatically recognizing the subject, the first AF is performed based on the subject recognition result. An area and a tracking target subject may be set.

In step S <b> 1 of FIG. 8, an initial tracking image (an image acquired first after starting the image tracking process) is acquired by the second image sensor 16. Image information captured by the second image sensor 16 is represented by RGB values for each pixel.
R [x, y], G [x, y], B [x, y],
x = 1 to 16, y = 1 to 12 (1)

Furthermore, based on the RGB value for each pixel, color information and luminance information for each pixel are calculated. Here, the color information is represented by RG and BG, which are values indicating the degree of color deviation, as shown in the following equation (2). Further, it is represented by L calculated from the following equation (2) based on the exposure time T, gain G, and color synthesis coefficients Kr, Kg, Kb when an image is acquired as luminance information.
RG [x, y] = Log ₂ (R [x, y]) − Log ₂ (G [x, y])
BG [x, y] = Log ₂ (B [x, y]) − Log ₂ (G [x, y])
L [x, y] = Log ₂ (Kr × R [x, y] + Kg × G [x, y] + Kb × B [x, y]) − Log ₂ (T) −Log ₂ (G)
... (2)

  Next, the tracking control initial process shown in FIG. 9 is executed in step S2. In step S101 of FIG. 9, image information (see equation (2)) of the position designated by the photographer in the initial tracking image (here, the focus detection area 45c which is the initial AF area) is stored as subject color information. In step S102, as shown in FIG. 6A, the same color information area indicating the same color information as the subject color information is detected in the position peripheral portion of the focus detection area 45c (see FIG. 5) in the tracking initial image. In step S103, a rectangular area including the same color information area is set as an initial tracking subject area 47. Here, an example in which the tracking subject area 47 is determined based on the subject color information is shown, but in order to simplify the processing, the size of the tracking subject area is uniformly set to 4 × 4 pixels, or The size of the tracking subject area may be determined according to the distance information of the photographing lens 8 and the image magnification of the subject.

In step S104, the tracking subject area 47 of the tracking initial image determined in step S103 is stored in the memory 19d as a template image 48. As shown in FIG. 6A, when the size of the tracking subject region 47 is 4 × 4 pixels and the start point position (pixel position corresponding to the upper left corner of the square) is (x, y) = (7, 5) The color information and luminance information of the template image 48 shown in FIG. 6B are expressed as the following formula (3).
RGref [rx, ry] = RG [x, y],
BGref [rx, ry] = BG [x, y]),
Lref [rx, ry] = L [x, y],
rx = 1 to 4, ry = 1 to 4, x = 7 to 10, y = 5 to 8 (3)

  In step S105, an area for searching for an image that matches or is similar to the template image 48 in the next tracking image to be captured is determined. Here, as shown in FIG. 6A, an area included in a pixel range (12 × 12 pixels) expanded by 4 pixels in the vertical and horizontal directions of the tracking subject area 47 is set as the first search area 49. Further, an area included in the pixel range (8 × 8 pixels) enlarged by 2 pixels in the vertical and horizontal directions of the tracking subject area 47 is set as the second search area 50. In FIG. 6A, the first search area 49 is an area included in the ranges of x = 3 to 14 and y = 1 to 12. The second search area 50 is an area included in the ranges of x = 5 to 12 and y = 3 to 10.

  Preferably, the first search area 49 and the second search area 50 are set around the tracking subject area 47. Since the tracking target subject moves with the detected position, that is, the tracking subject region 47 as a base point, by setting the first search region 49 and the second search region 50 around the tracking subject region 47, the tracking target subject is set. Can be detected quickly, and the responsiveness of image tracking can be improved. It should be noted that the first search area 49 and the second search area 50 may be areas of a predetermined size enlarged by predetermined pixels in the vertical and horizontal directions, or may be changed according to the tracking result and the size of the tracking subject. Good.

The tracking control initial process is thus completed, and the process returns to step S3 in FIG. In step S3 of FIG. 8 after the return, it is determined whether or not the shutter button of the operation member 20 has been fully pressed, and the processes after step S4 are repeated until the shutter button is fully pressed. In step S4, a tracking next image is acquired from the second image sensor 16, and the image information R [x, y], G [x, y], B [x, y] for the tracking next image is acquired in the same manner as in the process of step S1. (X = 1 to 16, y = 1 to 12)
And color information RG [x, y], BG [x, y] and luminance information L [x, y] are calculated.

  In subsequent step S5, a tracking calculation process is performed. A region having the same size as the tracking subject region 47 shown in FIG. 6A is sequentially cut out from the first search region 49 in the tracking next image. Then, for each pixel corresponding to the cut-out image and the template image 48 shown in FIG. 6B, the correlation is calculated, that is, the difference Diff of the image information is calculated.

The difference Diff is calculated by the following equation. As shown in FIG. 7A, the first search area 49
Is set to (scx, scy) = (3,1).
Diff [dx, dy] =
{ABS (RG [scx + dx-1 + rx, scy + dy-1 + ry] -RGref [rx, ry])
+ ABS (BG [scx + dx-1 + rx, scy + dy-1 + ry] -BGref [rx, ry]) + ABS (L [scx + dx-1 + rx, scy + dy-1 + ry] -Lref [rx, ry])}
dx = 1 to 9, dy = 1 to 9, rx = 1 to 4, ry = 1 to 4, scx = 3, scy = 1
... (4)

Note that dx and dy in the equation (4) indicate ranges (amounts) in which the start point position of the clipped image is shifted. That is, in FIG. 7A, dx = 1 corresponds to x = 3, and dy = 1 corresponds to y = 1. Equation (4) is obtained by acquiring a cutout image while shifting an area of the same size as the tracking subject area 47 in the first search area 49 by one pixel, and an image for each pixel between the cutout image and the template image 48. This indicates that the difference Diff of information is to be obtained. For example, as shown in FIG. 7A, images sequentially extracted from the first search area 49 (49a, 49b, and 49c in FIG. 7A) and the template image 48 are sequentially compared, and the respective images are compared. The difference Diff is calculated for the clipped image. The cut-out image indicating the minimum value among these differences Diff is most similar to the template image 48 (see FIG. 6B).

In step 6, a new tracking subject position determination process for determining a new tracking subject position shown in FIG. 10 is performed. In step S201, the smallest value (first minimum difference value MinDiff) is detected among the differences Diff of the image information between the template image 48 and the clipped image calculated using the above equation (4). Then, the coordinate value (Mindx, Mindy) of the difference Diff when it becomes the first minimum difference value MinDiff is detected.

In step S202, the difference Diff indicating the correlation between the template image 48 and the image information included in the above-described second search region 50 (x = 5 to 12, y = 3 to 10 in FIG. 7A). The smallest value (second minimum difference value MinDiff_S) is detected. And the coordinate value (Mindx_S, Mindy_S) of the difference Diff when it becomes the 2nd minimum difference value MinDiff_S is detected. In addition,
In FIG. 7A, the range in which the start point position of the cutout image corresponding to the second search area 50 is shifted is dx = 3 to 7, and dy = 3 to 7.

In step S203, it is determined whether or not the first minimum difference value MinDiff and the second minimum difference value MinDiff_S are different values. When the first minimum difference value MinDiff and the second minimum difference value MinDiff_S are the same value, a negative determination is made in step S203, and the process proceeds to step S207 described later. When the first minimum difference value MinDiff and the second minimum difference value MinDiff_S are different values, an affirmative determination is made in step S203 and the process proceeds to step S204. Here, the first minimum difference value MinDiff and the second
As a case where the minimum difference value MinDiff_S becomes a different value, the following cases can be considered.
(A1) A case where the tracking target subject is detected in the wide first search region 49 because the tracking target subject has a fast moving speed within the screen.
(A2) The tracking target subject exists in the second search area 50, which is a narrow area because the moving speed of the tracking target subject in the screen is slow or stationary, but similar in the background. When a subject that has the color information that is different from the subject to be tracked is detected (pseudo matching).

In step S204, the first minimum difference value MinDiff and the second minimum difference value MinDiff_S are compared, that is, the difference is calculated. Then, it is determined whether the calculated difference is less than a predetermined threshold value Wth1. When this difference is larger than the threshold value Wth1, a negative determination is made in step S204, and the process proceeds to step S207 described later. In this case, the possibility of the above (A1) is considered high. The threshold value Wth1 is a fixed value determined, for example, by a prior simulation, and is recorded in a memory (not shown). In this case, the threshold value Wth1 follows even when the subject to be tracked moves largely within the shooting screen, and is similar even when a subject having a high similarity with the template image 48 exists in the background. It is set to a value that is less likely to cause pseudo-matching for a subject with a high degree. That is, when the average value of the minimum difference value MinDiff is 6.0 to 7.0, the threshold value Wth1 is set to around 1.0.

If the difference value is less than the threshold value Wth1, an affirmative determination is made in step S204 and the process proceeds to step S205. Here, as a case where the value of the difference is less than the threshold value Wth1, the following cases can be considered.
(B1) The position of the subject to be tracked is at the position of the first minimum difference value MinDiff, but pseudo matching is performed within the second search subregion.
(B2) The movement speed of the subject to be tracked is within the second search area, but a subject having similar color information existing in the background (first search area 49) is detected that is different from the subject to be tracked. Case (pseudo-matching).

In step S205, it is determined whether or not a value related to the moving distance of the tracking target subject is equal to or smaller than a predetermined threshold value Wth2. In the present embodiment, for example, based on the result of the new tracking subject position detection process in the past five image tracking operations (5 cycles) (steps S4 to S9 in FIG. 8), the past tracking target in the shooting screen is displayed. It is determined whether or not the moving distance per cycle of the subject is equal to or less than a threshold value Wth2 (for example, two pixels). If the movement distance in the shooting screen in one cycle of the tracking target is 2 pixels or less, an affirmative determination is made in step S205 and the process proceeds to step S206. At this time, the possibility of the above (B2) is high because it is unlikely that a tracking target subject having a relatively small moving distance will suddenly move greatly within the shooting screen based on the history of the subject position of the tracking target in the past. That is, it is assumed that the subject to be tracked exists in the second search area 50 in the shooting screen. Instead of comparing the moving distance per cycle of the subject to be tracked in the past on the shooting screen with the threshold value Wth2, the moving speed of the tracking subject may be compared with the threshold value Wth2. In step S206, the position of the difference Diff that becomes the second minimum difference value MinDiff_S detected in the second search area 50 is determined as the new tracking subject area 47. That is, an area of 4 × 4 pixels starting from the coordinate value (Mindx_S, Mindy_S) is set as a new tracking subject area 47.

If the movement distance in the shooting screen in one cycle of the tracking target exceeds 2 pixels, a negative determination is made in step S205 and the process proceeds to step S207. In this case, since the moving distance of the subject to be tracked in the shooting screen is large, it is considered that the possibility of (B1) is high. In step S207, the position of the difference Diff that becomes the first minimum difference value MinDiff detected in the first search area 49 is determined as the new tracking subject area 47. That is, an area of 4 × 4 pixels starting from the coordinate value (Mindx, Mindy) is set as a new tracking subject area 47. Note that, when a negative determination is made in step S203, that is, when the first minimum difference value MinDiff and the second minimum difference value MinDiff_S are equal, the first minimum difference value MinDiff detected in the first search region 49 is also determined. Is determined as a new tracking subject region 47 based on the position corresponding to. FIG. 7B is a diagram exemplifying the position of the new tracking subject area 47 determined by the above-described new tracking subject position determination processing. In the example shown in FIG. 7B, the start point position is (x, y)
A region of 4 × 4 pixels where = (8,5) is determined as a new tracking subject region 47. That is, FIG. 7B shows a case where the subject to be tracked is detected in the second search area 50. The new tracking subject position determination process is thus completed, and the process returns to step S7 in FIG.

  In step S7 of FIG. 8, a new tracking subject region 47 determined by the new tracking subject position determination process by comparing the first minimum difference value MinDiff or the second minimum difference value MinDiff_S with a preset similarity threshold Dth. Is determined to be similar to the template image 48. The similarity threshold Dth is set as a value that allows appropriate similarity determination in a normal shooting scene.

In step S8, when it is determined in step S7 that the new tracking subject area 47 and the template image 48 are similar, the template image 48 is updated. In the present embodiment, a new template image 48 is generated by adding 20% of the image information of the new tracking subject area 47 to 80% of the image information such as color information and luminance information of the original template image 48, for example. . The calculation of the update process can be expressed as the following equation (5).
RGref [rx, ry] = 0.8 × RGref [rx, ry] + 0.2 × RG [x, y],
BGref [rx, ry] = 0.8 × RGref [rx, ry] + 0.2 × BG [x, y]),
Lref [rx, ry] = 0.8 × Lref [rx, ry] + 0.2 × L [x, y],
rx = 1 to 4, ry = 1 to 4, x = 8 to 11, y = 5 to 8 (5)

As a result of the update process of the template image 48, the latest image information is updated little by little with respect to the image information of the original template image 48. The reliability of subject tracking can be improved by suppressing the influence of the change in the face direction of the subject. Note that the ratio of the image information of the original template image 48 and the ratio of the image information of the new tracking subject area 47 may be varied according to the value of the minimum difference value MinDiff.

  In step S9, a new first search area 49 and a new second search area 50 are determined for the new tracking subject area 47. Here, as shown in FIG. 7 (b), the area of x = 4-15 and y = 1-12 surrounding the tracking subject area 47 (x = 8-11, y = 5-8) is the first search area. 49, an area of x = 6 to 13 and y = 3 to 10 is set as the second search area 50. Further, after the new tracking subject area 47 is determined, focusing control is performed on the tracking target included in the new tracking subject area 47. In the focus control, when at least one of the focus detection areas 45a to 45g is included in the new tracking subject area 47, the defocus amount detected in the focus detection areas 45a to 45g included in the new tracking subject area 47 The smallest of these is adopted, and the focus of the photographic lens 8 is adjusted based on the defocus amount. In the focus control, when the focus detection areas 45a to 45g are not included in the new tracking subject region 47, the distance to the new tracking subject region 47 or the focus detection areas 45a to 45g employed in the previous tracking result are used. Focus detection areas 45a to 45g to be employed may be determined based on at least one of the distances to When the above process ends, the process returns to step S3.

  When a full pressing operation of the shutter button of the operation member 20 is detected in step S3, the process proceeds to step S10 and photographing control is executed. In the imaging control, an exposure calculation is performed based on the luminance information of the new tracking subject area 47 determined in step S9, and an aperture value and a shutter speed are calculated. The shutter 7 and the aperture 21 are driven and controlled in accordance with these exposure values, and the first image sensor 4 captures an image.

According to the embodiment described above, the following operational effects can be obtained.
(1) The second imaging device repeatedly captures the object scene image formed by the photographing lens 8 to generate the object scene image. The tracking control unit 19e calculates a first minimum difference value MinDiff indicating a correlation between the image information of the first search region 49 in the object scene image and the template image 48 based on the target image, and the object scene First search area 49 in the image
The second minimum difference value MinDiff_S indicating the correlation between the image information in the second search region 50 different from the template image 48 is calculated. The tracking control unit 19e detects the position of the target image in the object scene image based on one of the first minimum difference value MinDiff and the second minimum difference value MinDiff_S. Therefore, it is possible to prevent erroneous detection of a subject different from the subject to be tracked in the shooting screen and maintain tracking accuracy.

(2) The first search area 49 is configured by an area larger than the second search area 50, and the tracking control unit 19e determines that the difference between the first minimum difference value MinDiff and the second minimum difference value MinDiff_S is a threshold value Wth1. It is determined whether or not. Then, the tracking control unit 19e, when the difference between the first minimum difference value MinDiff and the second minimum difference value MinDiff_S exceeds the threshold Wth1, based on the first minimum difference value MinDiff, the target image in the scene image The position of was detected.
The tracking control unit 19e detects the position of the target image based on the second minimum difference value MinDiff_S when the difference between the first minimum difference value MinDiff and the second minimum difference value MinDiff_S is less than the threshold value Wth1. I did it. Therefore, even when the movement of the subject to be tracked in the shooting screen changes suddenly, the tracking accuracy can be maintained because the movement of the subject can be followed.

(3) The tracking control unit 19e determines whether the moving distance of the target image in the object scene image is less than the threshold value Wth2 based on the position of the target image in a plurality of times (for example, five cycles). . When the difference between the first minimum difference value MinDiff and the second minimum difference value MinDiff_S is determined to be less than the threshold value Wth1 and the movement distance is determined to be less than the threshold value Wth2, the tracking control unit 19e The position of the target image is detected based on the difference value MinDiff_S. Further, when the difference between the first minimum difference value MinDiff and the second minimum difference value MinDiff_S is determined to be less than the threshold value Wth1, and the movement distance is determined to be equal to or greater than the threshold value Wth2, based on the first minimum difference value MinDiff The position of the target image is detected. Therefore, it is possible to prevent the occurrence of pseudo matching with respect to the background or the like in the object scene image and follow the movement of the subject to be tracked.

The imaging device 1 according to the embodiment described above can be modified as follows.
(1) The position of the first search area 49 and the position of the second search area 50 are not limited to those set around the position of the tracking subject area 47 determined in the new tracking subject position determination process one cycle before. For example, a predetermined area (previous area) centered on the position of the tracking subject area 47 determined in the new tracking subject position determination process one cycle before and a predetermined area (fixed area) fixed in the object scene image are The first search area 49 and the second search area 50 are set. The fixed area can be set as shown in an area R surrounded by a thick line in FIG. 11, for example. FIG. 11A shows the case where the region R is the entire shooting screen, and FIG. 11B shows the region R in the vicinity of the center of the shooting screen (start position (x, y) = (6, 4), 6 × 6 pixels). ), And FIG. 11C shows a region where the region R includes the focus detection area 45 (the start point position is (x, y) =
The case of (4,4), 10 × 6 pixels) is shown. Hereinafter, referring to FIG. 11 and FIG.
An example regarding the position where the search area 49 and the second search area 50 are set will be described. 11 and 12, description will be made with the coordinate system set as in the case of FIGS.

(1-1) FIG. 12A shows a case where the first search area 49 is set as the fixed area on the entire photographing screen shown in FIG. 11A and the second search area 50 is set as the previous area. . In FIG. 12A, the fixed area may be a predetermined area including the vicinity of the center of the photographing screen shown in FIG. 11B or the focus detection area shown in FIG. FIG. 12A shows an example in which a 6 × 6 pixel area having a start point position (x, y) = (4, 4) is set as the second search area 50.

(1-2) In FIG. 12B, the first search area 49 is set as the previous area. Further, the second search area 50 is set as a fixed area. FIG. 12B shows a case where a predetermined area including the focus detection area shown in FIG. 11C is set as the fixed area, but the entire photographing screen shown in FIG. 11A or FIG. It may be near the center of the shooting screen shown in (b). FIG. 12B shows a case where an area of 10 × 10 pixels having a starting point position (x, y) = (2, 2) is set as the first search area 49. Note that, as described above, the second search area 50 is set to a fixed area of 10 × 6 pixels with the starting point position (x, y) = (4, 4). In addition, a region overlapping with the first search region 49 and the second search region 50, that is, a 6 × 8 pixel region having a start point position (x, y) = (4, 4) in FIG. It may be set as a search area.

(1-3) Further, based on the position of the subject to be tracked in the object scene image acquired a plurality of times in the past, the position of the subject to be tracked in the object scene image acquired next time is predicted. In this case, a predetermined area (predicted area) centered on the predicted position can be set as the first search area 49 or the second search area 50. In this case, the prediction region is predicted based on a calculation result obtained by calculating a motion vector or the like using a known technique, for example. When the prediction area can be set, the first search area 49 and the second search area 50 are set by any two areas of the fixed area, the previous area, and the prediction area. Since the case where the fixed area and the previous area are used is the same as (1-1) and (1-2) above, the case where the previous area and the prediction area are used and the case where the fixed area and the prediction area are used will be described.

  In FIG. 12C, the first search area 49 is set as the previous area, and the second search area 50 is set as the prediction area. In FIG. 12C, an area of 10 × 10 pixels with the start point position (x, y) = (2, 2) is defined as the first search area 49, and the start point position is (x, y) = (2, 3). In the example, an area of 6 × 6 pixels is set as the second search area 50. In FIG. 12C, the first search area 49 may be set as the prediction area, and the second search area 50 may be set as the previous area.

(1-4) FIG. 12D shows a case where the first search area 49 is set as a fixed area on the entire photographing screen shown in FIG. 11A and the second search area 50 is set as a prediction area. . The entire photographing screen, that is, the entire area of the object scene image may be set as a fixed area. Note that the fixed region may be a predetermined region including the vicinity of the center of the photographing screen shown in FIG. 11B or the focus detection area shown in FIG. FIG. 12D shows an example in which an area of 6 × 6 pixels having a starting point position (x, y) = (2, 3) is set as the second search area 50. In FIG. 12D, the first search area 49 may be set as a prediction area, and the second search area 50 may be set as a fixed area.

(2) When the region including the focus detection area described above is set as the second search region 50, is the process in step S205 of FIG. 10, that is, whether the movement distance of the subject to be tracked in the past one cycle is equal to or less than the threshold value Wth2 It is not necessary to execute the process of determining whether or not. For example, when the region R in FIG. 11C is set as the second search region 50, the user uses the focus detection area 45 to try to catch the subject to be tracked. Therefore, the calculation result based on the first search region 49 that does not include the focus detection area 45 is highly likely to be pseudo-matched. Accordingly, the process proceeds to step S206 without performing the process in step S205, and the calculation result based on the second search region can be selected, so that the processing load can be reduced.

(3) Instead of the threshold value Wth1 being a fixed value, it may be a variable value based on the first minimum difference value MinDiff and the second minimum difference value MinDiff_S. In this case, for example, the threshold value Wth1 is automatically calculated based on the history of the minimum difference value obtained as a result of a plurality of processes. That is, in the process of the past 5 cycles, the maximum value and the minimum value of the first minimum difference value MinDiff or the second minimum difference value MinDiff_S when determined to be similar by the similarity determination process in step S7. The difference may be set as the threshold value Wth1.

(4) The tracking control unit 19e sets a priority for the first minimum difference value MinDiff and the second minimum difference value MinDiff_S, and the difference between the first minimum difference value MinDiff and the second minimum difference value MinDiff_S is less than the threshold value Wth1. In this case, the position of the image to be tracked may be detected based on the minimum difference value for which a high priority is set. In this case, the tracking control unit 19e gives high priority to the minimum difference value corresponding to the search area including the focus detection area 45c provided near the center of the shooting screen, for example, in the first search area 49 and the second search area 50. What is necessary is just to set a ranking.

  In addition, the present invention is not limited to the above-described embodiment as long as the characteristics of the present invention are not impaired, and other forms conceivable within the scope of the technical idea of the present invention are also within the scope of the present invention. included. The embodiments and modifications used in the description may be configured by appropriately combining them.

1 imaging device, 16 second imaging element,
19 body drive control device, 19e tracking control unit

Claims (4)

A setting unit that sets a first search region and a second search region smaller than the first search region in the scene image generated by the imaging unit;
First information corresponding to a correlation between reference information corresponding to a tracking target and image information in the first search region in the object scene image, and the second information in the reference information and the object scene image A calculation unit that calculates second information corresponding to the correlation with the image information in the search area;
A detection unit that detects a position of the tracking target using at least one of the first information and the second information based on a comparison result between the first information and the second information. Image tracking device. An image tracking device according to claim 1,
The setting unit sets the first search area and the second search area in the first scene image generated by the imaging unit, and after the first scene image is generated, the imaging unit A third search region and a fourth search region smaller than the third search region are set in the generated second scene image;
The calculation unit includes first information corresponding to a correlation between the reference information and image information in the first search region in the first scene image, and the reference information and the first scene. After calculating the second information corresponding to the correlation with the image information in the second search area in the image,
Third information corresponding to a correlation between the reference information and image information in the third search region in the second scene image, and the fourth information in the reference information and the second scene image An image tracking device that calculates fourth information corresponding to a correlation with image information in a search area . An image tracking device according to claim 1 ,
The detection unit includes a comparison unit that compares the first information and the second information;
An image tracking apparatus comprising: a detection unit that detects a position of the tracking target using at least one of the first information and the second information based on a comparison result of the comparison unit. An image tracking device according to claim 3 ,
When the difference between the first information and the second information is a predetermined value or more, the detection unit detects the position of the tracking target using the first information, and the first information and the second information An image tracking device , wherein the position of the tracking target is detected using the second information when the difference between and is not equal to or greater than a predetermined value .