JP2010093320A - Image tracking device and image tracking method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the amount of operation required for template matching. <P>SOLUTION: A reference information about an image to be tracked is memorized (Fig. 5 (b)). A subject field image of a subject field is image-formed by an image forming optical system, and the image-formed subject field image is repeatedly captured. A partial image is part of the subject field image (Fig. 5 (a)), and the partial image is segmented (Fig. 5 (c)-(e)). The partial image is divided so that a region corresponding to a plurality of pixels may become one block in the partial image (Fig. 5 (c)-(e)). An image information on at least one of color information and brightness information is computed for every block. Based on the image information and the reference information for every block, the position for tracking is detected in the subject field image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image tracking device and an image tracking method.

When shooting a moving subject, the subject image of the specified part in the screen is acquired as a template image, and the position of the subject image similar to the template image is searched (template matching) among the images to be repeatedly shot. 2. Description of the Related Art An image tracking device that tracks a moving subject is known (see, for example, Patent Document 1).
JP 2006-058431 A

  In the conventional image tracking device described above, if the number of pixels of an image that is repeatedly shot is small, it is difficult to detect a small subject in the screen and it is difficult to follow the fine movement of the subject. It is better that the number of pixels of the image used for matching is large. However, there is a problem that the amount of calculation required for template matching increases as the number of pixels increases.

(1) A first aspect of the present invention is a storage means for storing reference information relating to an image to be tracked, an imaging means for repeatedly capturing an object scene image of an object image formed by an image forming optical system, A partial image of the scene image is cut out, the partial image is divided so that a region corresponding to a plurality of pixels in the partial image becomes one block, and image information regarding at least one of color information and luminance information is An image tracking device includes a calculation unit that calculates each block, and a detection unit that detects a position of a tracking target in an object scene image based on image information and reference information for each block.
(2) The invention according to claim 2 is the image tracking device according to claim 1, further comprising a detecting means for detecting a focus adjustment state of the imaging optical system at a focus detection position set corresponding to the object scene image. And the calculating means sets the shift amount of the cutout position when cutting out a plurality of partial images from the object scene image to be smaller as the focus position is closer to the focus detection position.
(3) The invention according to claim 3 is the image tracking device according to claim 1 or 2, wherein the focus adjustment state of the imaging optical system at a plurality of focus detection positions set corresponding to the object scene image. The calculating means sets a shift amount of the cutout position when cutting out a plurality of partial images from the object scene image according to the focus adjustment state for each focus detection position.
(4) The invention according to claim 4 is the image tracking device according to any one of claims 1 to 3, wherein the calculation means determines the cut-out position when cutting out a plurality of partial images from the object scene image. The shift amount is set according to the number of pixels of the tracking target image.
(5) The invention according to claim 5 is the image tracking device according to any one of claims 1 to 4, wherein the calculation means sets a range for extracting the partial image from the object scene image, and The shift amount of the cutout position when cutting out a plurality of partial images is set to be smaller as it is closer to the center of the range.
(6) The invention according to claim 6 is the image tracking device according to any one of claims 1 to 5, wherein the calculation means determines the cut-out position when cutting out a plurality of partial images from the object scene image. The shift amount is set smaller as it is closer to the center of the object scene image.
(7) The invention according to claim 7 is the image tracking device according to any one of claims 1 to 6, wherein the calculation means calculates the cut-out position when cutting out a plurality of partial images from the object scene image. The shift amount is set to be smaller as the difference sum is smaller.
(8) The invention according to claim 8 is the image tracking device according to any one of claims 1 to 7, wherein the calculating means determines the cut-out position when cutting out a plurality of partial images from the object scene image. The shift amount is set as a predetermined pixel.
(9) The invention according to claim 9 is the image tracking device according to any one of claims 1 to 8, wherein the calculation unit is configured such that the number of pixels of the partial image is equal to the number of pixels of the image to be tracked. Cut out a partial image.
(10) The invention according to claim 10 is the image tracking device according to any one of claims 1 to 9, wherein the calculation means has a block corresponding to a plurality of pixels in the tracking target image. In this way, the reference information is also divided so that the number of divisions of the reference information is equal to the number of divisions of the partial image.
(11) The invention according to claim 11 is the image tracking device according to any one of claims 1 to 10, wherein the calculating means extracts the distance information of the tracking target when extracting the partial image from the object scene image. The number of pixels of the partial image is set according to
(12) In the image tracking device according to any one of claims 1 to 11, the calculation unit calculates an average value of at least one of the color information and the luminance information for each block. The image information is calculated based on the average value.
(13) The invention of claim 13 stores reference information relating to an image to be tracked, repeatedly captures a scene image of the scene formed by the imaging optical system, and a part of the scene image. The partial image is cut out, the partial image is divided so that an area corresponding to a plurality of pixels in the partial image becomes one block, and image information regarding at least one of color information and luminance information is calculated for each block. This is an image tracking method for detecting the position of the tracking target in the object scene image based on the image information and the reference information.

  According to the present invention, the amount of calculation required for template matching can be reduced.

  The focus adjustment state of the photographic lens (in this embodiment, the defocus amount) is detected in a plurality of focus detection areas set in the photographic screen, and the photographic lens is focused based on the defocus amount of one of the areas. The automatic focus adjustment (AF) function to be driven and the image of the subject to be tracked in the photographed image are stored as a template image (reference image), and an image that is the same as or similar to the template image among the repeatedly captured images An image pickup device having an image tracking function that tracks a subject to be tracked while searching for a position (template matching), and an image tracking device that tracks the tracking target while driving a photographic lens by the AF function and the image tracking function ( An embodiment of a single lens reflex digital still camera) will be described.

  FIG. 1 shows a configuration of an imaging apparatus (single-lens reflex digital still camera) 1 including an image tracking apparatus according to an embodiment. In FIG. 1, illustration and description of camera devices and circuits not directly related to the present invention are omitted. In a camera 1 according to an embodiment, an interchangeable lens 3 is detachably attached to a camera body 2. The camera body 2 is provided with a first image sensor 4 for capturing an object scene image and recording the image. The first image sensor 4 can be constituted by a CCD, a CMOS, or the like. At the time of photographing, the quick return mirror 5 and the sub mirror 6 are retracted to a position outside the photographing optical path indicated by a solid line, the shutter 7 is opened, and a subject image is formed on the light receiving surface of the first image sensor 4 by the photographing lens 8.

  At the bottom of the camera body 2, a focus detection optical system 9 and a distance measuring element 10 for detecting the focus adjustment state of the photographing lens 8 are provided. In this embodiment, an example is shown in which a focus detection method based on a pupil division type phase difference detection method is employed. The focus detection optical system 9 guides the pair of focus detection light fluxes that have passed through the photographing lens 8 to the light receiving surface of the distance measuring element 10 and forms a pair of optical images. The distance measuring element 10 includes, for example, a pair of CCD line sensors, and outputs a focus detection signal corresponding to the pair of optical images. Before shooting, the quick return mirror 5 and the sub mirror 6 are set at positions in the shooting optical path as indicated by broken lines, and the pair of focus detection light beams from the shooting lens 8 are transmitted through the half mirror portion of the quick return mirror 5. Then, the light is reflected by the sub mirror 6 and guided to the focus detection optical system 9 and the distance measuring element 10.

  A finder optical system is provided on the upper part of the camera body 2. Before shooting, the quick return mirror 5 and the sub mirror 6 are in the positions indicated by broken lines, and part of the subject light from the shooting lens 8 is reflected by the quick return mirror 5 and guided to the focusing screen 11, and on the focusing screen 11. A subject image is formed. The liquid crystal display element 12 superimposes and displays information such as a focus detection area mark on the object scene image formed on the focusing screen 11 and displays various shooting information such as an exposure value at a position outside the object scene image. indicate. The object scene image on the focusing screen 11 is guided to the eyepiece window 15 via the penta roof prism 13 and the eyepiece lens 14 so that the photographer can visually recognize the object scene image.

  The finder optical system at the top of the camera body 2 is provided with a second image sensor 16 that captures a field image for subject tracking and photometry. The object scene image formed on the focusing screen 11 is re-imaged on the light receiving surface of the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. The second imaging element 16 includes a plurality of pixels (photoelectric conversion elements) arranged in a matrix (in this embodiment, an example in which 640 pixels in the horizontal direction and 480 pixels in the vertical direction are arranged in a square) is shown. Yes. Further, each pixel is divided into three parts, and red R, green G, and blue B primary color filters are provided in these parts, respectively. Thereby, the RGB signal of the object scene image can be output for each pixel. Although details will be described later, tracking control and exposure calculation are performed based on the object scene image signal imaged by the second image sensor 16. The tracking control and the exposure calculation may be performed based on the scene image signal captured by the first image sensor 4.

  The camera body 2 is also provided with a body drive control device 19 and an operation member 20. The body drive control device 19 is constituted by peripheral components such as a microcomputer, a memory, and an A / D converter, the details of which will be described later, and performs various controls and calculations of the camera 1. The operation member 20 includes a switch and a selector for operating the camera 1 such as a shutter button, a focus detection area selection switch, and a shooting mode selection switch.

  The interchangeable lens 3 is provided with a zooming lens 8a, a focusing lens 8b, a diaphragm 21, a lens drive control device 22, and the like. In this embodiment, the photographing lens 8 is representatively represented by a zooming lens 8a, a focusing lens 8b, and a diaphragm 21, but the configuration of the photographing lens 8 is not limited to the configuration shown in FIG. The lens drive control device 22 includes a microcomputer (not shown) and peripheral components such as a memory, a drive circuit, and an actuator, and performs drive control of the lenses 8a and 8b and the diaphragm 21 and detection of their positions. Lens information such as a focal length and an open aperture value of the interchangeable lens 3 is stored in a memory built in the lens drive control device 22.

  The body drive control device 19 and the lens drive control device 22 communicate via the contact 23 of the lens mount, and transmit information such as a lens drive amount and an aperture value from the body drive control device 19 to the lens drive control device 22. Lens information and aperture information are transmitted from the lens drive control device 22 to the body drive control device 19.

  FIG. 2 shows a detailed configuration of the body drive control device 19. Note that illustration and description of control functions not directly related to the present invention are omitted. The body drive control device 19 includes an element control circuit 19a, an A / D converter 19b, a microcomputer 19c, a memory 19d, and the like. The element control circuit 19 a controls charge accumulation and reading of the second image sensor 16. The A / D converter 19b converts the analog image signal output from the second image sensor 16 into a digital image signal. When the shutter button of the operation member 20 is half-pressed by the photographer, a half-press switch (not shown) is turned on, and as shown in FIG. The light is reflected by the mirror 5 and guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. While the shutter button is half-pressed, the second imaging element 16 can repeatedly and periodically capture the object scene image.

  The microcomputer 19c constitutes a tracking control unit 19e, an exposure control unit 19f, a focus detection calculation unit 19g, and a lens drive amount calculation unit 19h according to a software form. The memory 19d stores information such as a template image for image tracking and defocus amount, lens information such as focal length, open F value, aperture value, and image shift amount to defocus amount conversion coefficient of the taking lens 8. To do.

  The tracking control unit 19e uses a template image as an image corresponding to the tracking target position manually specified by the photographer or the tracking target position automatically set by the camera 1 in the object scene image captured by the second image sensor 16. It is stored in the memory 19d as a (reference image), and the position of the object is recognized by searching an image area that matches or is similar to the template image from images that are repeatedly photographed thereafter. The exposure calculation unit 19f calculates an exposure value based on the image signal captured by the second image sensor 16.

  The focus detection calculation unit 19g detects the focus adjustment state of the photographing lens 8, here the defocus amount, based on the focus detection signal corresponding to the pair of optical images output from the distance measuring element 10. Although details will be described later, a plurality of focus detection areas are set in the shooting screen of the shooting lens 8, and the distance measuring element 10 outputs a focus detection signal corresponding to a pair of optical images for each focus detection area. Then, the focus detection calculation unit 19g detects the defocus amount based on the focus detection signal corresponding to the pair of optical images for each focus detection area. When the shutter button of the operation member 20 is half-pressed by the photographer, a half-push switch (not shown) is turned on, and as shown in FIG. 1, part of the subject light that has passed through the photographing lens 8 is a quick return indicated by a broken line. The light is guided to the distance measuring element 10 through the sub-mirror 6 and the focus detection optical system 9 through the half mirror part of the mirror 5, and the focus detection calculation part 19g performs the focus detection calculation. The lens driving amount calculation unit 19h converts the detected defocus amount into a lens driving amount. The focus adjustment state is repeatedly and periodically detected while the shutter button is half-pressed.

  Next, an image tracking operation according to an embodiment will be described. 3 to 5 are diagrams for explaining the image tracking operation according to the embodiment, and FIGS. 6 to 7 are flowcharts illustrating the image tracking process according to the embodiment. The quick return mirror 5 is set in the photographing optical path shown by the broken line in FIG. 1 except when the shutter button is fully pressed to shoot, and part of the subject light incident from the photographing lens 8 is on the focusing screen 11. Is imaged. The object scene image on the focusing screen 11 is guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18, and the object scene image signal is repeatedly output from the second image sensor 16. The

  A plurality of focus detection areas are set on the shooting screen of the shooting lens 8, and an area mark is superimposed on the object scene image on the focusing screen 11 by the liquid crystal display element 12 to display the position of each focus detection area. In this embodiment, as shown in FIG. 3, an example is shown in which focus detection areas 45a to 45g (denoted as a to g in the figure) are set at seven locations in the photographing screen. When an arbitrary area is selected by the focus detection area selection switch of the operation member 20, the mark of the area is lit up.

  As shown in FIG. 3, when the focus detection area 45b is selected by the focus detection area selection switch of the operation member 20, and the shutter button of the operation member 20 is half-pressed in this state, the focus detection area 45b is stored as the first AF area. 19d. Thereby, the subject in the first AF area is designated as the tracking target. In this example, the photographer selects the first AF area and manually designates the subject to be tracked. For example, in a camera having a function of automatically recognizing the subject, the first AF is performed based on the subject recognition result. An area and a tracking target subject may be set.

In step 1 of FIG. 6, an initial tracking image (an image acquired first after starting the image tracking process) is acquired by the second image sensor 16. Image information captured by the second image sensor 16 is represented by RGB values for each pixel.
R [x, y], G [x, y], B [x, y],
x = 1 to 640, y = 1 to 480 (1)

  Next, the tracking control initial process shown in FIG. In step 101 of FIG. 7, the image information (refer to equation (1)) of the position designated by the photographer in the initial tracking image (here, the focus detection area 45b which is the first AF area) is stored as subject color information. In step 102, as shown in FIG. 4A, the same color information area 46 indicating the same color information as the subject color information is formed in the periphery of the position of the focus detection area 45b (see FIG. 3) in the tracking initial image. In step 103, a rectangular area including the same color information area 46 is set as an initial tracking subject area 47.

  Here, an example in which the tracking subject area 47 is determined based on the subject color information is shown, but in order to simplify the processing, the size of the tracking subject area is uniformly set to 3 × 3 pixels, or The size of the tracking subject area may be determined according to the distance information of the photographing lens 8 and the image magnification of the subject.

In step 104, in the tracking subject area 47 of the initial tracking image, an area corresponding to a plurality of pixels is divided into three blocks and three vertical blocks as one block. If the base point of the coordinates of the tracking subject area 47 is (x, y) = (i, j) and the number of pixels of the tracking subject area 47 is refh and refv, the RGB image for each pixel in the tracking subject area 47 of the tracking initial image The information is expressed as follows:
R [x, y], G [x, y], B [x, y],
x = i to i + refh, y = j to j + refv (2)
Based on the RGB image information for each pixel, RGB values bR, bG, bB for each block are calculated. The RGB values bR, bG, and bB for each block are, for example, average values of R, G, and B values of all the pixels included in the block.

Next, in step 105, color information and luminance information for each block are calculated based on the RGB values bR, bG, and bB for each block. Here, RG and BG, which are values indicating the degree of color deviation, are used as color information, and each block is represented as bRGref [rx, ry], bBGref [rx, ry] (rx, ry = 1 to 3), L calculated from the exposure time T, gain Gain, and color synthesis coefficients Kr, Kg, and Kb when an image is acquired as luminance information is represented as bLref [rx, ry] for each block.
bRGref [rx, ry] = Log ₂ (bR [rx, ry]) − Log ₂ (bG [rx, ry]),
bBGref [rx, ry] = Log ₂ (bB [rx, ry]) − Log ₂ (bG [rx, ry]),
bLref [rx, ry] = Log ₂ (Kr × bR [rx, ry] + Kg × bG [rx, ry] + Kb × bB [rx, ry]) − Log ₂ (T) −Log ₂ (Gain). (3)

  The color information bRGref [rx, ry], bBGref [rx, ry] and luminance information bLref [rx, ry] of each block are stored in the memory 19d as a blocked template image 48 as shown in FIG. In this embodiment, an example in which the tracking subject region 47 is divided into horizontal 3 × vertical 3 blocks is shown. However, the number of blocks is not limited to the number of divisions in this embodiment.

  In step 106, an area for searching for an image that matches or is similar to the template image 48 in the next tracking image to be captured is determined. Here, as shown in FIG. 4A, a horizontal srch and vertical srcv pixel range surrounding the tracking subject area 47 is set in the search area 49. Preferably, the search area 49 is set around the tracking subject area 47. Since the tracking target subject moves from the currently detected position, that is, the tracking subject region 47 as a base point, by setting the search region 49 around the tracking subject region 47, the tracking target subject can be detected quickly. And the responsiveness of image tracking can be improved. The search area 49 may be an area having a predetermined size enlarged by predetermined pixels in the vertical and horizontal directions, or may be changed according to the tracking result and the size of the tracking subject.

  The tracking control initial process is thus completed, and the process returns to step 3 in FIG. In step 3 of FIG. 6 after the return, it is determined whether or not the shutter button of the operation member 20 has been fully pressed, and the processes in and after step 4 are repeated until the shutter button is fully pressed. In step 4, a tracking next image is acquired from the second image sensor 16, and image information R [x, y], G [x, y], B [x, y] (x = 1) as in the processing in step 1. ˜640, y = 1˜480).

  In the following step 5, tracking calculation is performed. An area having the same size as the tracking subject area 47 having the number of pixels refh and refv shown in FIG. 4A is sequentially cut out from the search area 49 in the following tracking image, and the cut-out image is shown in FIG. 4B as a blocked template. The image 48 is divided so as to have the same number of blocks as the image 48. Further, as described above, color information bRG [rx, ry], bBG [rx, ry] (rx, ry = 1 to 3) and luminance information bL for each block based on the RGB values bR, bG, and bB for each block. [rx, ry] is calculated. Then, the difference Diff of the image information is calculated for each block corresponding to the blocked template image 48 and the blocked clipped image.

The difference Diff is calculated by the following equation.
Diff = ΣΣ {ABS (bRG [rx, ry] −bRGref [rx, ry]) + ABS (bBG [rx, ry] −bBGref [rx, ry]) + ABS (bL [rx, ry] −bLref [rx, ry ])} (4)
In the equation (4), ΣΣ represents a sum calculation of the color differences of all the blocks of rx = 1 to 3 and ry = 1 to 3. A clipped image is acquired while shifting an area of the same size as the tracking subject area 47 in the search area 49 by a predetermined pixel, and a difference Diff of image information for each block is obtained between the clipped image and the template image 48. Go.

  For example, as shown in FIG. 5A, the images sequentially cut out from the search area 49 are blocked, and the blocked template image (b) and the blocked cut images (c), (d), and (e) are sequentially compared. Then, the difference Diff is calculated for each of them, and the cut-out image (here, the image shown in FIG. 5E) showing the minimum difference among these differences Diff is the largest in the template image 48 (see FIG. 4B). It will be similar. When determining the region most similar to the template image 48, the defocus amount detected in the focus detection areas 45a to 45g corresponding to each region may be taken into consideration.

It should be noted that the shift amount of the cutout region may be a predetermined value, and is made smaller as it is closer to the center of the search region 49 (the position where the subject to be tracked at the time of the previous tracking computation), and increased as the distance from the center increases. It may be. Since the tracking target subject moves based on the currently detected position, that is, the tracking subject region 47, the search region 49 is set around the tracking subject region 47, and the closer to the center of the search region 49, the closer to the center of the search region 49, By reducing the shift amount, the subject to be tracked can be detected quickly and accurately, and the responsiveness and accuracy of image tracking can be improved.
Alternatively, the cutout region shift amount may be set according to the number of pixels of the template image 48. The smaller the number of pixels of the template image 48, the smaller the area occupied by the tracking target in the object scene image. Therefore, for example, the responsiveness and accuracy of image tracking can be improved by reducing the shift amount of the cutout region as the number of pixels of the template image 48 is smaller.

Alternatively, the closer to the center position of the object scene (the center of the photographing screen of the photographing lens 8), the smaller the shift amount of the cutout region may be, and the larger the distance from the center of the object scene. In general, a moving subject to be tracked is often captured near the center of the shooting screen, so that the closer to the center of the shooting screen, the smaller the shift amount of the cutout area, so that the subject to be tracked can be detected quickly and accurately. It is possible to improve the response and accuracy of image tracking.
Alternatively, the shift amount of the cutout region may be reduced as it is closer to at least one of the positions corresponding to the focus detection areas 45a to 45g set in the object scene. Since the subject in the first AF area is the tracking target in the tracking control initial process in step 2, the moving subject to be tracked is often captured near the position corresponding to the focus detection areas 45a to 45g. The accuracy of image tracking and focus adjustment can be improved by reducing the shift amount of the cutout region as it is closer to the position corresponding to 45a to 45g.
Or according to the focus adjustment state detected about the focus detection areas 45a-45g (as the defocus amount is small), you may make small the shift amount of the cut-out area | region in the focus detection areas 45a-45g vicinity. In general, the focus detection areas 45a to 45g in which large defocus amounts are detected are more likely to be focus detection areas 45a to 45g located at positions corresponding to the background that is not the tracking target intended by the photographer. Therefore, the accuracy of image tracking and focus adjustment can be improved by setting the shift amount of the cutout region according to the focus adjustment state.

  Furthermore, the difference Diff of the image information is taken, and when the difference value is larger than the threshold value, the shift amount when searching for the neighbor is increased, and when the difference value is smaller than the threshold value, the shift amount is decreased. You may do it. Alternatively, the shift amount may be made smaller as the image information difference Diff is smaller. The smaller the difference in image information for a certain clipping region, the higher the probability that the tracking target subject will be near the clipping region. Therefore, the tracking target subject can be detected quickly and accurately by reducing the shift amount of the clipping region. It is possible to improve the responsiveness and accuracy of image tracking.

  In the above-described embodiment, the example in which the number of pixels in the region to be cut out from the search region 49 and compared with the template image is the same as the number of pixels refh and refv of the template image is shown. Depending on the distance accuracy and the like, the number of pixels in the cutout region may be larger or smaller than the number of pixels refh and refv of the template image. Even if the tracking target subject moves in the optical axis direction of the photographic lens relative to the imaging device and the size of the tracking target subject changes, the size of the cutout area can be changed according to the change. It is possible to reliably detect the target subject. At this time, by making the number of blocks in the cutout area equal to the number of blocks in the template image, the same effect as when the template image is enlarged or reduced can be obtained, and the accuracy of image tracking can be improved.

  In step 6, a cutout area of the cutout image having the smallest image information difference Diff from the template image 48 is detected, and the cutout area is determined as a new tracking subject area 47. After the new tracking subject area 47 is determined, focusing control is performed on the tracking target included in the new tracking subject area 47. In the focus control, when at least one of the focus detection areas 45a to 45g is included in the new tracking subject area 47, the defocus amount detected in the focus detection areas 45a to 45g included in the new tracking subject area 47 The smallest of these is adopted, and the focus of the photographic lens 8 is adjusted based on the defocus amount. In the focus control, when the focus detection areas 45a to 45g are not included in the new tracking subject region 47, the distance to the new tracking subject region 47 or the focus detection areas 45a to 45g employed in the previous tracking result are used. Focus detection areas 45a to 45g to be employed may be determined based on at least one of the distances to Here, the above-described blocking process may be performed based on the image information of the new tracking subject region 47 and updated as a new template image 48.

  When updating the template image 48, the latest image information is added to the original image information of the template image 48 by adding 20% of the image information of the new subject area 47 to the image information 80% of the original template image 48. The reliability of subject tracking may be improved by adding information little by little to suppress the influence of a sudden change in the tracking subject due to some cause.

  The template image 48 may be updated only when the size of the position difference Diff determined as the tracking subject region 47 is smaller than the threshold value, instead of every time the tracking calculation is performed.

  In step 7, a new search area 49 is determined for the new tracking subject area 47. Here, as shown in FIG. 4A, a horizontal srch and vertical srcv pixel range surrounding the tracking subject area 47 is set in the search area 49. The search area 49 may be an area having a predetermined size enlarged by a predetermined number of pixels in the vertical and horizontal directions, or may be changed according to the tracking result and the size of the tracking subject. After determining the search area 49, the process returns to step 3.

  When the full pressing operation of the shutter button of the operation member 20 is detected in step 3, the process proceeds to step 8 to execute photographing control. In the imaging control, an exposure calculation is performed based on the luminance information of the new tracking subject area 47 determined in step 8, and an aperture value and a shutter speed are calculated. The shutter 7 and the aperture 21 are driven and controlled in accordance with these exposure values, and the first image sensor 4 captures an image.

  In the above-described embodiments and their modifications, all combinations of the embodiments and the modifications are possible.

  According to the above-described embodiment and its modifications, the following operational effects can be achieved. (1) The amount of calculation required for template matching can be reduced while increasing the number of pixels of an image to be template matched to improve image tracking accuracy. (2) The position of the tracking target can be accurately detected. (3) The position of the tracking target can be accurately detected from the entire range of the captured image with a small amount of calculation. (4) An optimum shift amount corresponding to the number of pixels of the template image is set, and high responsiveness of image tracking can always be maintained regardless of the number of pixels of the template image. (5) The subject to be tracked can be detected quickly and accurately, and the responsiveness and accuracy of image tracking can be improved. (6) Template matching can be easily calculated, and the tracking target subject can be detected quickly and accurately. (7) Even when the tracking target subject moves in the optical axis direction of the photographing lens with respect to the imaging apparatus and the size of the tracking target subject changes, the size of the cutout region can be changed according to the change. Thus, the subject to be tracked can be reliably detected.

1 is a cross-sectional view illustrating a configuration of an imaging device including an image tracking device according to an embodiment; 1 is a block diagram illustrating a configuration of an imaging device including an image tracking device according to an embodiment; The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The flowchart which shows the image tracking processing of one embodiment Flow chart showing tracking control initial processing

Explanation of symbols

DESCRIPTION OF SYMBOLS 1; Imaging device, 8; Shooting lens, 16; 2nd image sensor, 19; Body drive control apparatus, 19a; Tracking control part, 19d;

Claims (13)

Storage means for storing reference information about the image to be tracked;
Imaging means for repeatedly capturing a field image of the field imaged by the imaging optical system;
A partial image of the scene image is cut out, the partial image is divided so that a region corresponding to a plurality of pixels in the partial image becomes one block, and at least one of color information and luminance information Calculating means for calculating image information for each block;
An image tracking apparatus comprising: a detecting unit configured to detect a position of the tracking target in the object scene image based on the image information and the reference information for each block. The image tracking device according to claim 1,
Detecting means for detecting a focus adjustment state of the imaging optical system at a focus detection position set corresponding to the object scene image;
The image tracking device according to claim 1, wherein the calculation unit sets a shift amount of a cutout position when cutting out the plurality of partial images from the object scene image as a position closer to the focus detection position. In the image tracking device according to claim 1 or 2,
Detecting means for detecting a focus adjustment state of the imaging optical system at a plurality of focus detection positions set corresponding to the object scene image;
The calculation means sets an amount of shift of a cutout position when cutting out a plurality of the partial images from the object scene image according to the focus adjustment state for each focus detection position. Tracking device. In the image tracking device according to any one of claims 1 to 3,
The calculation means sets an amount of shift of a cutout position when cutting out a plurality of the partial images from the object scene image according to the number of pixels of the tracking target image. . In the image tracking device according to any one of claims 1 to 4,
The calculation means sets a range of the partial image to be cut out from the object scene image, and a shift amount of the cut-out position when cutting out the plurality of partial images from the range is smaller as it is closer to the center of the range. An image tracking device characterized by setting. In the image tracking device according to any one of claims 1 to 5,
The calculation means sets the shift amount of the cutout position when cutting out a plurality of the partial images from the object scene image so as to be closer to the center of the object scene image. . In the image tracking device according to any one of claims 1 to 6,
The image tracking device according to claim 1, wherein the calculation unit sets a shift amount of a cutout position when cutting out the plurality of partial images from the scene image as the sum of the differences is smaller. In the image tracking device according to any one of claims 1 to 7,
The image tracking device according to claim 1, wherein the calculating means sets a shift amount of a cutout position when cutting out the plurality of partial images from the scene image as a predetermined pixel. In the image tracking device according to any one of claims 1 to 8,
The image tracking device, wherein the calculating means cuts out the partial image so that the number of pixels of the partial image is equal to the number of pixels of the tracking target image. In the image tracking device according to any one of claims 1 to 9,
The calculation unit also divides the reference information so that a region corresponding to a plurality of pixels in the tracking target image becomes one block, and equalizes the division number of the reference information and the division number of the partial image. An image tracking device characterized by that. In the image tracking device according to any one of claims 1 to 10,
The image tracking device according to claim 1, wherein when calculating the partial image from the object scene image, the calculation unit sets the number of pixels of the partial image according to distance information of the tracking target. In the image tracking device according to any one of claims 1 to 11,
The image tracking device, wherein the calculating means calculates an average value of at least one of color information and luminance information for each block, and calculates the image information based on the average value. Stores reference information about the image to be tracked,
Repetitively capture the field image of the field imaged by the imaging optical system,
Cut out a partial image of the scene image,
The partial image is divided so that a region corresponding to a plurality of pixels in the partial image becomes one block,
Calculating image information related to at least one of color information and luminance information for each block;
An image tracking method, comprising: detecting a position of the tracking target in the object scene image based on the image information and the reference information for each block.

Images