JP5966249B2 - Image tracking device - Google Patents

Description

  The present invention relates to an image tracking device.

  Conventionally, when shooting a moving subject, the subject image of the indicated portion in the screen is acquired as a template image, and the position of the subject image similar to the template image is searched (template matching) among images repeatedly shot, 2. Description of the Related Art An image tracking device that tracks a subject that moves in a horizontal direction within a screen is known. For example, Patent Document 1 discloses a technique for improving tracking accuracy by performing template matching using a plurality of template images.

JP 2008-187331 A

  However, since the size of the relatively close moving subject on the image increases, there is a problem that the detection accuracy of the subject image similar to the template image is lowered and the subject cannot be tracked.

An image tracking device according to a first aspect of the present invention includes an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a scene image, and a tracking target in the scene image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images and a calculation result for each of the first and second template images by the calculation unit are compared, and based on the comparison result, A selection unit that selects a calculation result, a detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit, and a predetermined value continuously by the selection unit An update unit that replaces image information of the second template image for the calculation result selected by the selection unit as the first template image when the calculation result for the second template image is selected a number of times. And comprising.
An image tracking device according to a second aspect of the present invention includes an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a scene image, and a tracking target in the scene image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images and a calculation result for each of the first and second template images by the calculation unit are compared, and based on the comparison result, A selection unit that selects a calculation result; and a detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit. Continuously Predetermined number of times, if you select the calculation result for the second template image, inhibits selection of the operation result for the first template image.
According to a third aspect of the present invention, there is provided an image tracking device that repeatedly captures an image formed by an imaging optical system to generate a scene image, and a tracking target in the scene image. A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images and a calculation result for each of the first and second template images by the calculation unit are compared, and based on the comparison result, A selection unit that selects a calculation result; and a detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit. Communicating When a calculation result for the second template image is selected a predetermined number of times, the calculation unit calculates the first template image for the calculation result of the first template image calculated after the next time. The correction is made to make it difficult for the selection result to be selected by the selection unit.
According to a fourth aspect of the present invention, there is provided an image tracking device that repeatedly captures an image formed by an imaging optical system to generate a scene image, and a tracking target in the scene image. A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region; an image in a search region provided in the object scene image; The calculation unit that calculates the correlation between each of the first and second template images, the determination unit that determines that the tracking target is relatively large in the object scene image, and the determination unit that If it is determined that the tracking target is relatively large, the calculation result for the second template image is given priority among the calculation results for the first and second template images by the calculation unit. If the determination unit does not determine that the tracking target is relatively large, the calculation unit compares the calculation results for the first and second template images, and based on the comparison result, A selection unit that selects a calculation result; and a detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit .
An image tracking device according to a seventh aspect of the present invention includes an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a field image, and a tracking target in the field image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images, a focus adjustment unit that calculates a focus state of the imaging optical system and performs focus adjustment, and a calculation performed by the focus adjustment unit Based on the result, a determination unit that determines whether or not the tracking target is approaching the optical axis direction of the imaging optical system, and the determination unit that determines that the tracking target is approaching The first and second by the arithmetic unit The calculation result for the second template image is preferentially selected from the calculation results for each of the two template images, and when the approach of the tracking target is not determined by the determination unit, the calculation unit by the calculation unit A selection unit that compares the calculation results for each of the first and second template images and selects the calculation result based on the comparison result, and the scene image based on the calculation result selected by the selection unit And a detection unit for detecting the position of the image to be tracked.
An image tracking device according to an eighth aspect of the present invention includes an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a field image, and a tracking target in the field image A generating unit that generates a first template image and a second template image obtained by enlarging a part of the first template image based on a region; and an image in a search region provided in the object scene image And a calculation unit that calculates a correlation between each of the first and second template images, and a calculation result for each of the first and second template images by the calculation unit, A selection unit that selects the calculation result based on the detection result, a detection unit that detects a position of an image to be tracked in the scene image based on the calculation result selected by the selection unit, and a detection unit that detects the detection result. The Based on the position of the tracking target image, a setting unit that sets a new tracking target area, image information in the new tracking target area, and image information of the second template image at a predetermined ratio. An update unit that combines and updates the second template image, and the generation unit enlarges the entire first template image when the first template image is small, and When the generation unit generates the second template image by enlarging the entire first template image, the update unit generates image information in the new tracking target area, The ratio of the image information in the new tracking target area is increased with respect to the composition ratio with the image information of the second template image.
An image tracking device according to a tenth aspect of the present invention includes an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a scene image, and a tracking target in the scene image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region ; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images, a selection unit that selects a calculation result of one of the first and second template images by the calculation unit, and the selection A detection unit for detecting a position of a tracking target in the scene image based on the calculation result selected by the unit, and when a new scene image is generated, based on the new scene image The first template A update unit for updating a preparative image the second template image, the update unit, a predetermined number of consecutive by the selection unit, when the calculation result is selected for said second template image, The second template image for the calculation result selected by the selection unit is set as the first template image.
An image tracking device according to an eleventh aspect of the present invention is directed to an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a scene image, and a tracking target in the scene image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region ; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images, a selection unit that selects a calculation result of one of the first and second template images by the calculation unit, and the selection A detection unit for detecting a position of a tracking target in the scene image based on the calculation result selected by the unit, and when a new scene image is generated, based on the new scene image The first template A update unit for updating a preparative image the second template image, the selecting unit, a predetermined number of times in succession, if you select the calculation result for the second template image is the first Selection of calculation results for template images is prohibited.
An image tracking device according to a twelfth aspect of the present invention includes an imaging unit that repeatedly captures an image formed by an imaging optical system to generate a field image, and a tracking target in the field image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region ; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images, a selection unit that selects a calculation result of one of the first and second template images by the calculation unit, and the selection A detection unit for detecting a position of a tracking target in the scene image based on the calculation result selected by the unit, and when a new scene image is generated, based on the new scene image The first template A update unit for updating a preparative image the second template image, by the selection unit, when a predetermined number of times in succession, the operation result to the second template image is selected, the arithmetic unit, The calculation result of the first template image calculated after the next time is corrected to make it difficult for the selection unit to select the calculation result of the first template image.
An image tracking device according to a thirteenth aspect of the present invention is an image tracking device that repeatedly captures an image formed by an imaging optical system to generate a field image, and a tracking target in the field image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region ; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images, a determination unit that determines that the tracking target is relatively large in the scene image, and a new scene When an image is generated, an update unit for updating the second template image and the first template image based on the new scene image, and the tracking target by the determination unit is relatively large judging If The calculation result for the second template image is preferentially selected from the calculation results for the first and second template images by the calculation unit, and the tracking target becomes relatively large by the determination unit. If not determined, a selection unit that selects one of the calculation results of the first and second template images by the calculation unit, and the object scene based on the calculation result selected by the selection unit A detection unit that detects a position of the image to be tracked in the image .
An image tracking device according to a fourteenth aspect of the present invention is an image tracking device that repeatedly captures an image formed by an imaging optical system to generate a field image, and a tracking target in the field image A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a region ; an image in a search region provided in the object scene image; A calculation unit that calculates a correlation between each of the first and second template images, and when a new scene image is generated, the first template image based on the new scene image An update unit that updates the second template image , a focus adjustment unit that calculates a focus state of the imaging optical system and performs focus adjustment, and a tracking target is based on a calculation result by the focus adjustment unit. In the optical axis direction of the imaging optical system A determination unit that determines whether or not the tracking target is approaching, and each of the first and second template images by the calculation unit when the determination unit determines that the tracking target is approaching If the calculation result for the second template image is preferentially selected from the calculation results for and the approach of the tracking target is not determined by the determination unit, the first and second templates by the calculation unit A selection unit that selects any calculation result of the image; and a detection unit that detects a position of the tracking target in the object scene image based on the calculation result selected by the selection unit.

  According to the present invention, the position of a tracking target image can be detected based on the calculation result of a plurality of template images including at least one first template image obtained by enlarging an image region included in the tracking target region.

1 is a cross-sectional view illustrating a configuration of an imaging device including an image tracking device according to an embodiment; 1 is a block diagram illustrating a configuration of an imaging device including an image tracking device according to an embodiment; The block diagram which shows the function with which the tracking control part of embodiment is equipped The top view which illustrates the composition and field image of the 2nd image sensor The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The flowchart which shows the image tracking processing of one embodiment Flow chart showing tracking control initial processing Flow chart showing enlarged template update process The figure for demonstrating the production | generation process of the expansion template image in a modification The figure for demonstrating the production | generation process of the expansion template image in a modification

  The focus adjustment state of the photographic lens (in this embodiment, the defocus amount) is detected in a plurality of focus detection areas set in the photographic screen, and the photographic lens is focused based on the defocus amount in one of the areas. The automatic focus adjustment (AF) function to be driven and the image of the subject to be tracked in the photographed image are stored as a template image (reference image). An image pickup apparatus having an image tracking function for tracking a subject to be tracked while searching for a position (template matching), and an image tracking device for tracking the tracking target while driving a photographing lens with the AF function and the image tracking function ( An embodiment of a single lens reflex digital still camera) will be described.

  FIG. 1 shows a configuration of an imaging apparatus (single-lens reflex digital still camera) 1 including an image tracking apparatus according to an embodiment. In FIG. 1, illustration and description of camera devices and circuits that are not directly related to the image tracking function are omitted. In the camera 1 according to the embodiment, an interchangeable lens 3 having a photographing lens 8 and a diaphragm 21 is detachably attached to the camera body 2. The camera body 2 is provided with a first image sensor 4 for capturing an object scene image and recording the image. The first image sensor 4 can be constituted by a CCD, a CMOS, or the like. At the time of photographing, the quick return mirror 5 and the sub mirror 6 are retracted to a position outside the photographing optical path indicated by a solid line, the shutter 7 is opened, and a subject image is formed on the light receiving surface of the first image sensor 4 by the photographing lens 8.

  At the bottom of the camera body 2, a focus detection optical system 9 and a distance measuring element 10 for detecting the focus adjustment state of the photographing lens 8 are provided. In this embodiment, an example is shown in which a focus detection method based on a pupil division type phase difference detection method is employed. The focus detection optical system 9 guides the pair of focus detection light fluxes that have passed through the photographing lens 8 to the light receiving surface of the distance measuring element 10 and forms a pair of optical images. The distance measuring element 10 includes, for example, a pair of CCD line sensors, and outputs a focus detection signal corresponding to the pair of optical images. Before shooting, the quick return mirror 5 and the sub mirror 6 are set at positions in the shooting optical path as indicated by broken lines, and the pair of focus detection light beams from the shooting lens 8 are transmitted through the half mirror portion of the quick return mirror 5. Then, the light is reflected by the sub mirror 6 and guided to the focus detection optical system 9 and the distance measuring element 10.

  A finder optical system is provided on the upper part of the camera body 2. Before shooting, the quick return mirror 5 and the sub mirror 6 are in the positions indicated by broken lines, and part of the subject light from the shooting lens 8 is reflected by the quick return mirror 5 and guided to the focusing screen 11, and on the focusing screen 11. A subject image is formed. The liquid crystal display element 12 superimposes and displays information such as a focus detection area mark on the object scene image formed on the focusing screen 11 and displays various shooting information such as an exposure value at a position outside the object scene image. indicate. The object scene image on the focusing screen 11 is guided to the eyepiece window 15 via the penta roof prism 13 and the eyepiece lens 14 so that the photographer can visually recognize the object scene image.

  The finder optical system at the top of the camera body 2 is provided with a second image sensor 16 that captures an object scene image for subject tracking and photometry. The object scene image formed on the focusing screen 11 is re-imaged on the light receiving surface of the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18.

  FIG. 4A is a front view showing a detailed configuration of the second image sensor 16. In FIG. 4A, a coordinate system is used in which the horizontal direction is the x-axis and the vertical direction is the y-axis. The second image sensor 16 includes a plurality of pixels (photoelectric conversion elements) 40 (for example, 640 pixels wide × 480 pixels vertically) arranged in a matrix. In FIG. 4A, for the sake of illustration, the number of pixels 40 is smaller than the above-mentioned horizontal 640 × vertical 480. Further, each pixel 40 is provided with a primary color filter of red R, green G, and blue B, respectively. Thereby, the RGB signal (field image signal) of the object scene image can be output for each pixel 40 to the body drive control device 19 (FIG. 2) described later. Although details will be described later, tracking control and exposure calculation are performed by the body drive control device 19 based on the object scene image signal imaged by the second image sensor 16. The body drive control device 19 may perform the tracking control and the exposure calculation based on the object scene image signal output by the imaging by the first image sensor 4.

  The camera body 2 is also provided with a body drive control device 19 and an operation member 20. The body drive control device 19 is constituted by peripheral components such as a microcomputer, a memory, and an A / D converter, the details of which will be described later, and performs various controls and calculations of the camera 1. The operation member 20 includes a switch and a selector for operating the camera 1, such as a release button, a focus detection area selection switch, and a shooting mode selection switch.

  The interchangeable lens 3 is provided with a zooming lens 8a, a focusing lens 8b, a diaphragm 21, a lens drive control device 22, and the like. In this embodiment, the photographing lens 8 is representatively represented by a zooming lens 8a, a focusing lens 8b, and a diaphragm 21, but the configuration of the photographing lens 8 is not limited to the configuration shown in FIG. The lens drive control device 22 includes a microcomputer (not shown) and peripheral components such as a memory, a drive circuit, and an actuator, and performs drive control of the lenses 8a and 8b and the diaphragm 21 and detection of their positions. Lens information such as a focal length and an open aperture value of the interchangeable lens 3 is stored in a memory built in the lens drive control device 22.

  The body drive control device 19 and the lens drive control device 22 communicate via the contact 23 of the lens mount unit. Through this communication, the body drive control device 19 transmits information such as a lens drive amount and an aperture value to the lens drive control device 22, and the lens drive control device 22 transmits lens information and aperture information to the body drive control device 19.

  FIG. 2 shows a detailed configuration of the body drive control device 19. Note that illustration and description of control functions not directly related to the image tracking function are omitted. The body drive control device 19 includes an element control circuit 19a, an A / D converter 19b, a microcomputer 19c, a memory 19d, and the like. The element control circuit 19 a controls charge accumulation and reading of the second image sensor 16. The A / D converter 19b converts the analog image signal output from the second image sensor 16 into a digital image signal. When the release button of the operation member 20 is half-pressed by the photographer, a half-press switch (not shown) is turned on, and as shown in FIG. The light is reflected by the mirror 5 and guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. While the release button is half-pressed, the second imaging element 16 can repeatedly and periodically capture the field image.

  The microcomputer 19c constitutes an exposure control unit 19e, a focus detection calculation unit 19f, a lens drive amount calculation unit 19g, and a tracking control unit 19h according to a software form. The memory 19d stores information such as a template image for image tracking and defocus amount, lens information such as focal length, open F value, aperture value, and image shift amount to defocus amount conversion coefficient of the taking lens 8. To do.

  The exposure control unit 19e calculates an exposure value based on the image signal captured by the second image sensor 16. The focus detection calculation unit 19f detects the focus adjustment state of the photographing lens 8, here the defocus amount, based on the focus detection signal corresponding to the pair of optical images output from the distance measuring element 10. Although details will be described later, a plurality of focus detection areas are set in the shooting screen of the shooting lens 8, and the distance measuring element 10 outputs a focus detection signal corresponding to a pair of optical images for each focus detection area. The focus detection calculation unit 19f detects the defocus amount based on the focus detection signal corresponding to the pair of optical images for each focus detection area. When the release button of the operation member 20 is half-pressed by the photographer, a half-press switch (not shown) is turned on, and as shown in FIG. The light is guided to the distance measuring element 10 through the sub mirror 6 and the focus detection optical system 9 through the half mirror part of the mirror 5, and the focus detection calculation part 19f performs the focus detection calculation. The lens driving amount calculation unit 19g converts the detected defocus amount into a lens driving amount. The focus adjustment state is repeatedly and periodically detected while the release button is pressed halfway.

  The tracking control unit 19h performs an image tracking operation (processing). That is, the tracking control unit 19h selects an image corresponding to the tracking target position manually specified by the photographer or the tracking target position automatically set by the camera 1 from the object scene image captured by the second image sensor 16. A template image (reference image) is stored in the memory 19d. Then, the tracking control unit 19h recognizes the position of the target by searching for an image area that matches or is similar to the template image from the object scene images repeatedly photographed by the second image sensor 16 for each photographing period.

  As shown in FIG. 3, the tracking control unit 19h functionally includes a template generation unit 191, a calculation unit 192, a selection unit 193, a detection unit 194, an update unit 195, and a setting unit 196 in order to perform image tracking processing. . The template generation unit 191 generates a template image using the object scene image signal output from the second image sensor 16, and stores the template image in the memory 19d. The computing unit 192 computes a correlation between the scene image and the template image that are repeatedly photographed at each photographing cycle. The selection unit 193 selects a calculation result indicating that the correlation with the template image is the highest among the calculation results obtained by the calculation unit 192. The detection unit 194 detects (searches) an image region that matches or is similar to the template image from the scene image based on the calculation result selected by the selection unit 193. The update unit 195 performs an update process on the template image generated by the template generation unit 191. The setting unit 196 sets a search area for detecting an image area matching or similar to the template image in the scene image, and sets the detected matching or similar image area as a tracking target area described later. The details of processing by the template generation unit 191, the calculation unit 192, the selection unit 193, the detection unit 194, the update unit 195, and the setting unit 196 will be described later.

  Next, an image tracking operation according to an embodiment will be described. 5 to 9 are diagrams for explaining the image tracking operation according to the embodiment, and FIGS. 10 to 12 are flowcharts illustrating the image tracking process according to the embodiment. In the body drive control device 19, the user manually specifies a subject to be tracked (hereinafter referred to as a tracking target subject) from among the subject images captured by the second image sensor 16, or the camera automatically selects a tracking target subject. After the setting, when the release button of the operation member 20 is pressed halfway, the subject tracking process is started. In the following description, a coordinate system in which the horizontal direction is the x-axis and the vertical direction is the y-axis in FIGS.

  The quick return mirror 5 is set in the photographing optical path shown by a broken line in FIG. 1 except when the release button is fully pressed to shoot, and part of the subject light incident from the photographing lens 8 is on the focusing screen 11. Is imaged. The object scene image on the focusing screen 11 is guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18, and the object scene image signal is repeatedly output from the second image sensor 16. The

  A plurality of focus detection areas are set on the shooting screen of the shooting lens 8, and the liquid crystal display element 12 superimposes area marks on the object scene image on the focusing screen 11 to display the position of each focus detection area. . In this embodiment, as shown in FIG. 5, an example is shown in which focus detection areas 45a to 45g are set at seven locations in the shooting screen. When an arbitrary area is selected by the focus detection area selection switch of the operation member 20, the mark of the area is lit up.

  As shown in FIG. 5, when the focus detection area 45c is selected by the focus detection area selection switch of the operation member 20, and the release button of the operation member 20 is pressed halfway in this state, the focus detection area 45b is stored as the first AF area. 19d. As a result, the subject in the first AF area is designated as the tracking target subject. In this example, the photographer selects the first AF area and manually designates the tracking target subject. For example, in a camera having a function of automatically recognizing the subject, the first AF area is based on the subject recognition result. In addition, a tracking target subject may be set.

  In step S1 of FIG. 10, the element control circuit 19a causes the second image sensor 16 to acquire a tracking initial image (an image acquired first after starting the image tracking process). The image information of the initial tracking image captured by the second image sensor 16 is represented by RGB values for each pixel as a field image signal. The microcomputer 19c calculates color information and luminance information of the object scene image based on the RGB values of the object scene image signal input from the second image sensor 16. At this time, the microcomputer 19c combines the image information of each of the 640 × 480 pixels 40 included in the second image sensor 16 into, for example, the image information of the adjacent 8 × 8 pixels 40 as one block. Perform blocking. In this case, the microcomputer 19c performs blocking by averaging or summing adjacent 8 × 8 pieces of image information (that is, RGB values). As a result, as shown in FIG. 4B, the image information of each of the 640 × 480 pixels 40 of the object scene image is surrounded by 80 × 60 blocks 41 (thick lines in FIG. 4B). (In range) image information. In the following description, the above block-formed block is referred to as a block pixel 41. In FIG. 4B, a part of the entire scene image is shown enlarged.

When blocking is performed as described above, the image information captured by the second image sensor 16 is represented by RGB values for each block pixel 41 as in the following Expression (1).
R [x, y], G [x, y], B [x, y],
x = 1 to 80, y = 1 to 60 (1)

Then, the microcomputer 19c calculates the color information and luminance information of each block pixel 41 based on the RGB value for each block pixel 41 described above. Here, the color information is represented by RG and BG, which are values indicating the degree of color deviation, as shown in the following equation (2). Further, it is represented by L calculated from the following equation (2) based on the exposure time T, gain G, and color synthesis coefficients Kr, Kg, Kb when an image is acquired as luminance information.
RG [x, y] = Log ₂ (R [x, y]) − Log ₂ (G [x, y])
BG [x, y] = Log ₂ (B [x, y]) − Log ₂ (G [x, y])
L [x, y] = Log ₂ (Kr × R [x, y] + Kg × G [x, y] + Kb × B [x, y]) − Log ₂ (T) −Log ₂ (G)
... (2)

  Next, in step S2, the tracking control unit 19h executes tracking control initial processing shown in FIG. In step S101 of FIG. 11, the template generation unit 191 of the tracking control unit 19h includes image information ((2) of the position (here, the focus detection area 45b which is the initial AF area) specified by the photographer in the initial tracking image. Is stored in the memory 19d as subject color information. In step S102, as shown in FIG. 6A, the template generation unit 191 displays color information similar to the subject color information in the position peripheral portion of the focus detection area 45b (see FIG. 5) in the tracking initial image. The same color information area shown is detected.

  In step S <b> 103, the template generation unit 191 sets the rectangular area including the same color information area as the initial tracking subject area 47. Although an example in which the tracking subject area 47 is determined based on the subject color information is shown here, the size of the tracking subject area 47 is uniformly set to 4 × 4 block pixels 41 in order to simplify the processing. Alternatively, the size of the tracking subject region 47 may be determined according to the distance information of the photographing lens 8 or the subject image magnification.

In step S104, the template generation unit 191 stores the tracking subject area 47 of the tracking initial image determined in step S103 as the template image 48 in the memory 19d. As shown in FIG. 6A, when the size (size) of the tracking subject region 47 is TmpSize, the color information and luminance information of the template image 48 shown in FIG. expressed. In the following description, the template image 48 generated based on the tracking initial image is referred to as an initial template image. In the following equations, the variable t is a variable indicating the type of template image, and t = 0 corresponds to the initial template image 48.
Ref [t, RG, i, j] = RG [x + i, y + j]
Ref [t, BG, i, j] = BG [x + i, y + j]
Ref [t, L, i, j] = L [x + i, y + j]
t = 0, i = 1 to TmpSize, j = 1 to TmpSize (3)

  In step S <b> 105, the template generation unit 191 creates the enlarged template image 50 based on the initial template image 48. As shown in FIG. 6C, the enlarged template image 50 has an internal image area 50a included in the initial template image 48 (an area surrounded by a broken line in FIG. 6C) and the initial template image 48. It is the image expanded to the same size. In other words, the enlarged template image 50 corresponds to an image obtained when the initial template image 48 is zoomed up. Therefore, as will be described later, when the subject approaches the camera 1, the tracking target subject can be detected by the enlarged template image 50.

  When creating the enlarged template 50, the template generation unit 191 first determines the inside of the initial template image 48 as the internal image region 50a. In FIG. 7A, as an example of the internal image region 50a, the inner region is determined by two block pixels for each of the upper and lower sides (y direction) of the initial template image 48 and two block pixels for each of the left and right sides (x direction). Shows the case. Then, the template generation unit 191 stores the determined internal image area 50a (Scrop [cl, i, j]) in the memory 19d. A plurality of internal image areas 50a may be determined. FIG. 7B shows a case where three types of internal image areas 50a1 to 50a3 are determined as an example. In this case, the template generation unit 191 determines the internal image areas 50a1 to 50a3 so that the sizes of the internal image areas 50a1 to 50a3 become smaller in steps.

When the internal image area 50a is determined as described above, the template generation unit 191 enlarges the internal image area 50a to the same size as the initial template image 48 by the interpolation process shown in the following equation (4), and the enlarged template image 50 (Ref [t, cl, i, j], t = 1) is generated. When the variable t is 1, it indicates that the enlarged template image 50 is used.
Scrup [RG, s, u] = RG [x + s + n, y + u + n]
Scrup [BG, s, u] = BG [x + s + n, y + u + n]
Scrup [L, s, u] = L [x + s + n, y + u + n]
s = 1 to the size of the enlarged template image, u = 1 to the size of the enlarged template image,
n = number of difference pixels between the initial template image and the internal image area (4)

  When a plurality of internal image regions 50a are determined (for example, the internal image regions 50a1 to 50a3 in FIG. 7B), the template generation unit 191 performs the initial template image by the interpolation processing described above for each internal image region 50a. Enlarge to the same size as 48. As a result, as shown in FIG. 6D, a plurality of types of enlarged template images 50_1, 50_2, and 50_3 having different zoom-up magnifications are generated. In addition, since the subject to be tracked exists far away and only a small area can be occupied in the object scene image, that is, the size of the same color information area, the subject tracking area 47 and the initial template image 48 is reduced. In such a case, the template generation unit 191 cannot determine the internal image area 50a inside the initial template image 48 by the above method. Therefore, for example, when the size of the initial template image 48 is 4 × 4 block pixels 41, the template generation unit 191 sets the same image information as the initial template image 48 as the enlarged template image 50. When the template generation unit 191 generates the enlarged template image 50, it stores it in the memory 19d.

  The tracking control initial process is thus completed, and the process returns to step S3 in FIG. In step S3 of FIG. 10 after the return, the microcomputer 19c determines whether or not the release button of the operation member 20 has been fully pressed. Until the release button is fully pressed, the microcomputer 19c repeats the processes of steps S4 to S11. In step S4, the microcomputer 19c uses the image information R [x, y], the image information R [x, y], in the same manner as in step S1, for the object scene image (tracking next image) captured by the second image sensor 16 in the next imaging cycle. G [x, y] and B [x, y] (x = 1 to 80, y = 1 to 60) are acquired, and color information RG [x, y], BG [x, y] and luminance information L [ x, y] is calculated.

  In step S5, the setting unit 196 sets an area (search area) 49 (FIG. 6A) for searching for an image that matches or is similar to the template image 48. Here, the setting unit 196 sets, for example, an area included in the block pixel range expanded by 4 block pixels in the vertical and horizontal directions of the tracking subject area 47 as the search area 49.

  The setting unit 196 preferably sets the search area 49 with the tracking subject area 47 as the center. The subject to be tracked moves with the position detected this time, that is, the tracking subject region 47 as a base point. For this reason, by setting the search area 49 around the tracking subject area 47 by the setting unit 196, it is possible to quickly detect the subject to be tracked and to improve the responsiveness of image tracking. The search area 49 may be a fixed-size area including a predetermined number of block pixels in the vertical and horizontal directions, or may be changed according to the tracking result and the size of the tracking target subject.

  In subsequent step S6, the calculation unit 192 of the tracking control unit 19h performs a tracking calculation process. The computing unit 192 sequentially cuts out an area having the same size as the tracking subject area 47 shown in FIG. 6A from the search area 49 in the tracking next image. Then, the computing unit 192 computes the correlation between the clipped image and the template image 48 shown in FIG. 6B for each corresponding pixel, that is, the difference Diff [t (= 0), sx, sy of the image information. ] Is calculated. Further, the calculation unit 192 calculates a correlation between the cut-out image and the enlarged template image 50 shown in FIG. 6C for each corresponding block pixel 41, that is, the difference Diff [t (= 1), sx, sy] is calculated.

Specifically, the calculation unit 192 calculates the difference Diff [t, sx, sy] by the following equation (5), where (x, y) is the starting point position of the search area 49.

In the equation (5), sx and sy indicate ranges (amounts) in which the start point position of the cut-out image is shifted. Expression (5) indicates that the following processes (1) and (2) are performed.
(1) A clipped image is acquired while shifting an area of the same size as the tracking subject area 47 in the search area 49 by one block pixel.
(2) A difference Diff of image information for each block pixel is obtained between the cut-out image and the template image 48 and between the cut-out image and the enlarged template image 50.

  For example, as shown in FIG. 8A, images sequentially cut out from the search area 49 (47a, 47b, 47c in FIG. 8A) and the template image 48 are sequentially compared, and each cut-out image is obtained. A difference Diff is calculated. Similarly, images sequentially extracted from the search area 49 (47a, 47b, 47c in FIG. 8A) and the enlarged template image 50 are sequentially compared, and a difference Diff is calculated for each of the clipped images. Then, the cut-out image that indicates the minimum value among these differences Diff is an image that is most similar to the template image 48 or the enlarged template image 50 and that has a high degree of similarity.

  As illustrated in FIG. 6D, when a plurality of types of enlarged template images 50_1, 50_2, and 50_3 having different zoom-up magnifications are generated, the calculation unit 192 compares the variable t with the variable t. The above equation (5) is calculated by substituting the number of types of template images. In this case, the calculation unit 192 substitutes 4 as a variable t, which is the total number of one type of initial template image 48 and three types of enlarged template images 50_1, 50_2, and 50_3. By the calculation using Expression (5), the calculation unit 192 calculates a difference Diff for each of the generated template image 48 and the plurality of enlarged template images 50_1, 50_2, and 50_3.

  In step 7, new tracking subject position determination processing for determining a new tracking subject position is performed. When the difference Diff is calculated by the calculation unit 192 using the above equation (5), the selection unit 193 selects the smallest value (minimum difference value MinDiff) among the values of the plurality of differences Diff. When the minimum difference value MinDiff is selected, the detection unit 194 detects the coordinate value (Mindx, Mindy) of the difference Diff when the minimum difference value MinDiff is obtained. Then, as illustrated in FIG. 8B, the setting unit 196 sets a new tracking subject region 47 around the coordinate values (Mindx, Mindy) detected by the detection unit 194. In other words, the setting unit 196 sets a cutout region having the highest similarity with the template image 48 or the enlarged template image 50 as a new tracking subject region 47.

  The detecting unit 194 further determines whether or not the variable t that gives the minimum difference value MinDiff is 1. When the variable t is 1, the detection unit 194 increases the count number of the counter by 1, and when the variable t is 0, the detection unit 194 resets the count number of the counter to 0. This counter is referred to by the updating unit 195 when a template image 48 described later is replaced with an enlarged template image 50.

  In step S8, the tracking control unit 19h compares the minimum difference value MinDiff with the preset similarity threshold Dth, so that the new tracking subject region 47 determined in step S7 is the template image 48 or the enlarged template image 50. It is determined whether or not it is similar. The similarity threshold Dth is set as a value that allows appropriate similarity determination in a normal shooting scene. The determination result in step S8 is used for focus adjustment in step S12 described later.

In step S <b> 9, the update unit 195 determines whether or not the template image 48 needs to be updated using a preset update threshold Upth. The update threshold Upth is set as a value that allows appropriate similarity determination in a normal shooting scene. In step S9, when it is determined that the similarity between the new tracking subject area 47 and the template image 48 is high, the template image 48 is updated. The update unit 195 determines that the template image 48 needs to be updated when the value of the minimum difference value MinDiff is equal to or less than the update threshold value Upth. In the present embodiment, the updating unit 195 adds, for example, 80% of the image information such as color information and luminance information of the original template image 48 to 20% of the image information of the new tracking subject area 47, and adds a new template. An image 48 is generated. That is, when it is determined for the first time that update processing is necessary, the update unit 195 performs update processing on the initial template image 48 to generate a new template image 48. In subsequent processing, the update unit 195 sequentially creates new template images 48. An update process is performed on it. The calculation of the update process can be expressed as the following equation (6).
Ref [0, RG, i, j] = 0.8 × Ref [0, RG, i, j] + 0.2 × RG [x + i−1, y + j−1],
Ref [0, BG, i, j] = 0.8 × Ref [0, BG, i, j] + 0.2 × BG [x + i−1, y + j−1],
Ref [0, L, i, j] = 0.8 × Ref [0, L, i, j] + 0.2 × L [x + i−1, y + j−1],
i = 1 to TmpSize, j = 1 to TmpSize, x, y are positions of the new tracking subject region 47 (6)

  As a result of the update process of the template image 48, the latest image information is updated little by little with respect to the image information of the original template image 48. Therefore, it is possible to improve the reliability of subject tracking by suppressing the influence of a change in the tracking subject due to some cause (for example, a change in the orientation of the face of the subject). Note that the ratio of the image information of the original template image 48 and the ratio of the image information of the new tracked subject area 47 may be varied according to the value of the minimum difference value MinDiff.

  In step S10, the update unit 195 performs an update process of the enlarged template image 50 shown in FIG. In step S <b> 201, it is determined whether or not an update process for the enlarged template image 50 is necessary, using a preset enlargement update threshold value Scrupth. The enlargement update threshold value Scrupth is set as a value that allows appropriate similarity determination in a normal shooting scene. The update unit 195 determines that the update processing of the enlarged template image 50 is necessary when the value of the minimum difference value MinDiff is equal to or smaller than the enlargement update threshold value Scrupth.

  In subsequent step S202, the updating unit 195 updates the enlarged template image 50 in the same manner as in the template image 48 update process described above. That is, the update unit 195 adds a new enlarged template image by adding 20% of the image information of the new tracking subject region 47 to, for example, 80% of the image information such as color information and luminance information of the original enlarged template image 50. 50 is generated. In step S2, if the initial template image 48 is small in size and the same image as the initial template image 48 is set as the enlarged template image 50, the update unit 195 adds image information of the new tracking subject region 47. The ratio is increased from the above 20%, and the image information of the new tracking subject region 47 is added by, for example, 40% to generate a new enlarged template image 50.

  In step S203, the updating unit 195 continuously sets m as the number of times that the position of the tracking subject detected in the enlarged template image 50 is set as the new tracking subject region 47 while repeating the above processing for each imaging cycle. It is determined whether or not the number of times has been reached. That is, the update unit 195 determines whether or not the tracking subject in the object scene is large because the tracking subject is approaching the camera 1. In this case, the update unit 195 determines whether the count number by the counter is m. If the count is not m, the enlarged template image update process is terminated without performing the following process.

When the count number is m, the updating unit 195 replaces the template image 48 with the enlarged template image 50 using the following equation (7) in step S204.
Ref [0, RG, i, j] = Ref [1, RG, i, j]
Ref [0, BG, i, j] = Ref [1, BG, i, j]
Ref [0, L, i, j] = Ref [1, L, i, j]
Note that i = 1 to TmpSize, j = 1 to TmpSize (7)

  When the count number is less than m, the tracking control unit 19h performs the tracking subject detection process using the template image 48 and the enlarged template image 50 shown in FIG. When the count number reaches m and the updating unit 195 performs the above replacement process, the tracking control unit 19h displays the enlarged template image 50 shown in FIG. 9A in FIG. As a template image 48 shown in FIG. The template image 48 and the enlarged template image 50 shown in FIG. 9A are the same as the template image 48 shown in FIG. 6B and the enlarged template image 50 shown in FIG. 6C, respectively.

  If the template image 48 is replaced in step S204, the update unit 195 generates a new enlarged template image 50 in step S205. In other words, the update unit 195 generates the enlarged template image 50 by determining the internal image region 50a as the new tracking target region 47 and enlarging it to the same size as the template image 48 by interpolation processing. As a result, an enlarged template image 50 (FIG. 9B) corresponding to the internal image area 50a surrounded by the broken line inside the new tracking target area 47 shown in FIG. 8B is generated. In other words, the update unit 195 generates the enlarged template image 50 based on the position of the new tracking subject region 47 when the count number of the counter is m.

  The update process of the enlarged template image 50 is thus completed, and the process returns to step S11 in FIG. In step S11 after the return, the focus detection calculation unit 19f performs focus control using the new tracking subject region 47 determined in the subject position determination process in step S7. In this case, when the tracking control unit 19h determines that the newly determined tracking subject region 47 is similar to the template image 48 or the enlarged template image 50 (step S8), the focus detection calculation unit 19f Focus control is performed on the tracking target included in the subject area 47. When at least one of the focus detection areas 45a to 45g is included in the new tracking subject area 47, the focus detection calculation unit 19f detects the data detected in the focus detection areas 45a to 45g included in the new tracking subject area 47. The smallest focus amount is adopted, and the focus of the photographic lens 8 is adjusted based on the defocus amount. When the focus detection areas 45a to 45g are not included in the new tracking subject area 47, the focus detection calculation unit 19f is set to the distance from the new tracking subject area 47 or the focus detection areas 45a to 45g employed in the previous tracking result. Focus detection areas 45a to 45g to be employed may be determined based on at least one of the distances to When the tracking control unit 19h determines that the newly determined tracking subject area 47 is not similar to the template image 48 or the enlarged template image 50 (step S8), the focus detection calculation unit 19f detects the previous processing. The focus adjustment is performed using the defocus amount. When the above process ends, the process returns to step S3.

  If the full pressing operation of the release button of the operation member 20 is detected in step S3, the process proceeds to step S12, and the microcomputer 19c executes photographing control. In the imaging control, the exposure calculation unit 19e performs an exposure calculation based on the luminance information of the new tracking subject area 47 determined in step S7, and calculates an aperture value and a shutter speed. In accordance with these exposure values, the microcomputer 19c drives and controls the shutter 7 and the diaphragm 21, and causes the first image sensor 4 to perform imaging.

According to the embodiment described above, the following operational effects can be obtained.
(1) The second image sensor 16 repeatedly captures an image formed by the photographing lens 8 to generate an object scene image. The template generation unit 191 generates a plurality of template images 48 and 50 including at least one enlarged template image 50 based on the tracking subject area 47 that is a tracking target area included in the object scene image. The calculation unit 192 calculates a correlation between the search area 49 inside the object scene image and each of the plurality of template images 48 and 50 generated by the template generation unit 191. The selection unit 193 compares the calculation results for each of the plurality of template images 48 and 50 by the calculation unit 192, and selects the calculation result based on the comparison results. The detection unit 194 detects the position of the tracking target subject in the object scene image based on the calculation result selected by the selection unit 193. When generating the enlarged template image 50, the template generation unit 191 enlarges the image area included in the tracking target area 47 by interpolation processing, and generates the enlarged template image 50 based on the enlarged image area. I tried to do it. Specifically, the template generation unit 191 generates an initial template image 48 different from the enlarged template image 50 based on the image information of the tracking subject area 47. Then, the template generation unit 191 sets the internal image region 50a inside the initial template image 48, and enlarges the set internal image region 50a by interpolation processing so as to be the size of the initial template image 48, thereby expanding the enlarged template. An image 50 is generated. Therefore, since the enlarged template image 50 corresponds to an image obtained when the initial template image 48 is zoomed up, even if the tracking target subject approaches the camera 1, the enlarged template image 50 uses the tracking target object. Can be detected.

(2) The setting unit 196 sets a new tracking target area 47 based on the position of the tracking target subject detected by the detection unit 194. When it is determined that the plurality of template images 48 and 50 are to be updated based on the calculation result by the calculation unit 192, the update unit 195 uses the image information of the new tracking target area 47 set by the setting unit 196. The plurality of template images 48 and 50 are updated. Accordingly, as a result of the update process, the latest image information is updated little by little with respect to the image information of the original template image 48 or the enlarged template image 50. Therefore, it is possible to improve the reliability of subject tracking by suppressing the influence of a change in the tracking subject due to some cause (for example, a change in the orientation of the face of the subject).

(3) The calculation unit 192 calculates a difference value between the image information of the enlarged template image 50 and the image information in the search area 49, and the update unit 195 enlarges according to the difference value calculated by the calculation unit 192. The necessity of updating the template image 50 is determined. In this case, the updating unit 195 updates the enlarged template image 50 by combining the image information in the new tracking target area 47 and the image information of the enlarged template image 50 at a predetermined ratio. Therefore, the latest image information is added to the enlarged template image 50 when the value of the minimum difference value MinDiff is equal to or smaller than the enlargement update threshold value Scrupth, that is, when the similarity is high. As a result, it is possible to improve the detection accuracy of the tracking target subject by the enlarged template image 50 in the subsequent processing. Further, as described in (2) above, since the latest image information is updated little by little with respect to the image information of the enlarged template image 50, the influence of changes in the tracking subject due to some cause is suppressed. Thus, the reliability of subject tracking can be improved.

(4) When the internal image region 50 a cannot be set in the initial template image 48, the template generation unit 191 generates the enlarged template image 50 using the same image information as the image information of the initial template image 48. When the enlarged template image 50 is generated by the template generation unit 191 using the same image information as the image information of the initial template image 48, the update unit 195 adds new information to the image information of the enlarged template image 50. The ratio of combining image information in the tracking target area 47 is increased. Therefore, when the same enlarged template image 50 as that of the initial template image 48 is used, the enlarged template image 50 can be updated to an image corresponding to a case where the initial template image 48 is zoomed up by an early update process. As a result, even when the tracking target subject approaches the camera 1, the tracking target can be detected by the enlarged template image 50.

(5) When the calculation result for the enlarged template image 50 is selected a predetermined number of times (m times) successively by the selection unit 193, the update unit 195 is based on the image information in the new tracking target region 47. Thus, a new enlarged template image 50 is generated. Further, the updating unit 195 replaces the image information of the enlarged template image 50 before generating the new enlarged template image 50 as a new template image 48. Therefore, since the enlarged template image 50 is regenerated using the image information of the latest tracking target area 47, when the calculation result of the enlarged template image 50 is continuously selected, that is, the tracking target subject is the camera 1. Thus, even if the subject is relatively close, the tracking target subject can be continuously detected.

(6) The selection unit 193 indicates that the similarity between the template images 48 and 50 and the search area 49 is the highest among the plurality of calculation results corresponding to the plurality of template images 48 and 50, respectively. Was selected. Specifically, the calculation unit 192 calculates a difference value between each of the image information in the plurality of template images 48 and 50 and the image information of the search area 49, and the selection unit 193 is calculated by the calculation unit 192. When the difference value is the smallest, it is determined that the similarity is the highest. Therefore, even when template matching using a plurality of template images 48 and 50 is performed, occurrence of erroneous detection and the like can be suppressed and tracking accuracy can be maintained.

The imaging device 1 according to the embodiment described above can be modified as follows.
(1) The updating unit 195 may not update the initial template image 48. In this case, when the update unit 195 determines that the update process is performed in step S9, the update unit 195 copies the initial template image 48 to generate a duplicate template image. Then, the update unit 195 performs the above-described update process on the duplicate template image to generate a new template image (update template image). When it is determined that the update process is performed for the second time and thereafter, the update unit 195 performs the update process on the update template image. That is, the update unit 195 performs an update process on the update template image and / or the enlarged template image 50 in a series of processes. Then, the tracking control unit 19h may perform the tracking calculation process described above in the search area 49 using the initial template image 48, the updated template image, and the enlarged template image 50. As a result, the initial template image 48 is kept without being updated. Therefore, when the tracking target subject moves away from the camera 1 again, the tracking target subject is again used by using the initial template image 48. It can be detected.

  In addition, when the number of times the position detected in the enlarged template image 50 is set as the new tracking subject area 47 reaches a predetermined number of times continuously, the tracking control unit 19h detects the initial template image 48. Processing is added so that the determined position is less likely to be adopted as the new tracking subject area 47. In this case, when the count number by the counter is n, the calculation unit 192 adds a predetermined coefficient k to the value of the difference Diff [0, sx, sy] between the initial template image 48 and the search area 49, Correction is performed so that the calculation result of the initial template image 45 is not easily selected by the selection unit 193. That is, by increasing the value of the difference Diff [0, sx, sy], the calculation result by the initial template image 48 becomes difficult to be adopted as the minimum difference value MinDiff. As a result, when the subject to be tracked is close to the camera 1, that is, when the subject to be tracked in the object scene image is large, a subject having a background different from the tracked subject (that is, a small subject) is set as the tracking target. It is possible to prevent erroneous detection.

  Furthermore, when the number of times that the position detected in the enlarged template image 50 is set as the new tracking subject region 47 becomes a predetermined number of times continuously, the tracking control unit 19h is detected in the initial template image 48. The selection of the position by the selection unit 193 may be prohibited. In this case, if tracking processing is performed using the initial template image 48 and the enlarged template image 50, the selection unit 193 selects the enlarged template image 50 and the search area when the count number by the counter is n. Only the value of the difference Diff [1, sx, sy] from 49 is selected.

(2) The template generation unit 191 expands and enlarges the initial template image 48 by interpolation processing instead of generating the enlarged template image 50 by expanding the internal image region 50a in the initial template image 48 by interpolation processing. A template image may be generated. That is, as illustrated in FIG. 13, the template generation unit 191 generates an enlarged template image 50 ′ in which the image information of the initial template image 48 is zoomed up.

(3) The template generation unit 191 may generate the enlarged template image 50 from the image information of the subject region 47 instead of generating the enlarged template image 50 from the internal image region 50a of the initial template image 48. In this case, as illustrated in FIG. 14, the template generation unit 191 sets an internal image region 50 a ′ inside the subject tracking region 47 set in the object scene image. Then, the template generation unit 191 may generate the enlarged template image 50 by enlarging the internal image region 50a ′ by interpolation processing.

(4) Based on the lens information input from the lens drive control device 22 of the interchangeable lens 3 via the contact 23 of the lens mount unit, the selection unit 193 performs calculation with priority on the calculation result using the enlarged template image 50. May be. When it is determined that the interchangeable lens 3 is zoomed up or the distance of the subject is approaching based on the lens information, that is, when the size of the tracking subject increases in the shooting screen, the selection unit 193 Selects a calculation result using the enlarged template image 50 with priority. As a priority method, if the difference value of the calculation result of the enlarged template image 50 is within the similarity threshold, the result of the enlarged template image 50 may be selected regardless of the result of the other template images. Alternatively, the calculation result other than the enlarged template image 50 may be weighted and the result having the smallest difference value may be selected. As a result, when the tracking target in the object scene image becomes large, it is possible to prevent erroneous detection of a background subject different from the tracking target (that is, a small subject) as the tracking target.

(5) The detection unit 194 replaces the counter t incremented by 1 when the variable t giving the minimum difference value MinDiff is 1, and based on the focus control result by the focus detection calculation unit 19f, The count number may be increased by one. In this case, the focus detection calculation unit 19f determines that the tracking target subject is approaching the camera 1 along the optical axis direction of the photographing lens 8 based on the previous defocus amount and the current defocus amount. In this case, the detection unit 194 increases the count number of the counter by one. When the count number is m, the selection unit 193 may select a calculation result using the enlarged template image 50. As a result, when the subject to be tracked is close to the camera 1, that is, when the subject to be tracked in the object scene image is large, a subject having a background different from the tracked subject (that is, a small subject) is set as the tracking target. It is possible to prevent erroneous detection.

(6) The calculation unit 192 is not limited to calculating the difference Diff. The calculation unit 192 may perform four arithmetic operations such as addition and division between the image information of the template image 48 and the image information of the enlarged template image 50 and the image information of the cut-out image, or the least square method. Etc. may be used. Also in this case, the selection unit 193 selects a calculation result that minimizes the value, that is, a calculation result indicating that the similarity is the maximum.

  In addition, the present invention is not limited to the above-described embodiment as long as the characteristics of the present invention are not impaired, and other forms conceivable within the scope of the technical idea of the present invention are also within the scope of the present invention. included. The embodiments and modifications used in the description may be configured by appropriately combining them.

1 imaging device, 16 second imaging element,
19 body drive control device, 19c microcomputer,
19f focus detection calculation unit, 19h tracking control unit,
23 contact points, 191 template generation unit,
192 calculation unit, 193 selection unit,
194 detector, 195 updater,
196 Setting section

Claims (14)

An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit that compares the calculation results for each of the first and second template images by the calculation unit, and selects the calculation results based on the comparison results;
A detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit;
When the calculation result for the second template image is selected a predetermined number of times continuously by the selection unit, the image information of the second template image for the calculation result selected by the selection unit is An image tracking device comprising: an updating unit that replaces the first template image. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit that compares the calculation results for each of the first and second template images by the calculation unit, and selects the calculation results based on the comparison results;
A detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit;
The image tracking device that prohibits selection of a calculation result for the first template image when the selection unit continuously selects a calculation result for the second template image a predetermined number of times. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit that compares the calculation results for each of the first and second template images by the calculation unit, and selects the calculation results based on the comparison results;
A detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit;
When the selection unit selects a calculation result for the second template image continuously a predetermined number of times, the calculation unit calculates the calculation result of the first template image calculated after the next time, An image tracking device that performs correction for making the calculation result of the first template image difficult to be selected by the selection unit. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A determination unit that determines that the tracking target is relatively large in the object scene image;
When the determination unit determines that the tracking target is relatively large, out of the calculation results for the first and second template images by the calculation unit, the calculation results for the second template image If the determination unit does not determine that the tracking target is relatively large, the calculation unit compares the calculation result for each of the first and second template images, and compares the result. A selection unit for selecting the calculation result based on :
An image tracking device comprising: a detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit. The image tracking device according to claim 4,
The imaging optical system includes a zooming lens,
The determination unit is an image tracking device that determines that the tracking target is relatively large when the zooming lens is zoomed up. The image tracking device according to claim 4,
The imaging optical system includes a focusing lens,
An image tracking device that determines that the tracking target is relatively large when lens information of the focusing lens indicates that the subject distance is approaching. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A focus adjustment unit that calculates a focus state of the imaging optical system and performs focus adjustment;
A determination unit that determines whether or not the tracking target is approaching the optical axis direction of the imaging optical system, based on a calculation result by the focus adjustment unit;
When the determination unit determines that the tracking target is approaching, the calculation result for the second template image among the calculation results for the first and second template images by the calculation unit If the approach of the tracking target is not determined by the determination unit, the calculation unit compares the calculation results for each of the first and second template images, and based on the comparison result A selection unit for selecting the calculation result;
An image tracking apparatus provided with the detection part which detects the position of the image of the said tracking object in the said scene image based on the said calculation result selected by the said selection part. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging a part of the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit that compares the calculation results for each of the first and second template images by the calculation unit, and selects the calculation results based on the comparison results;
A detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit;
A setting unit for setting a new tracking target area based on the position of the tracking target image detected by the detection unit;
An update unit that combines the image information in the new tracking target area and the image information of the second template image at a predetermined ratio to update the second template image;
The generating unit generates the second template image by enlarging the entire first template image when the first template image is small;
When the generation unit generates the second template image by enlarging the entire first template image, the update unit generates image information in the new tracking target region and the second template. An image tracking device that increases a ratio of image information in the new tracking target area with respect to a composition ratio of the image with the image information. The image tracking device according to claim 1, 2, 3, or 8.
The calculation unit calculates a difference between the image in the search area and the first template image and outputs a first difference value, and calculates the difference between the image in the search area and the second template image. Calculate the difference and output the second difference value,
The selection unit is an image tracking device that selects a smaller difference value between the first difference value and the second difference value. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit for selecting any one of the calculation results of the first and second template images by the calculation unit;
A detection unit that detects a position of a tracking target in the object scene image based on the calculation result selected by the selection unit;
When a new scene image is generated, and a update unit for updating the second template image and the first template image based on the new scene image,
The update unit, when the calculation result for the second template image is selected a predetermined number of times continuously by the selection unit, the second template image for the calculation result selected by the selection unit, An image tracking device as the first template image. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit for selecting any one of the calculation results of the first and second template images by the calculation unit;
A detection unit that detects a position of a tracking target in the object scene image based on the calculation result selected by the selection unit;
When a new scene image is generated, and a update unit for updating the second template image and the first template image based on the new scene image,
The image tracking device that prohibits selection of a calculation result for the first template image when the selection unit continuously selects a calculation result for the second template image a predetermined number of times. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A selection unit for selecting any one of the calculation results of the first and second template images by the calculation unit;
A detection unit that detects a position of a tracking target in the object scene image based on the calculation result selected by the selection unit;
When a new scene image is generated, and a update unit for updating the second template image and the first template image based on the new scene image,
When the selection unit selects a calculation result for the second template image continuously a predetermined number of times, the calculation unit calculates the calculation result of the first template image calculated after the next time, An image tracking device that performs correction for making the calculation result of the first template image difficult to be selected by the selection unit. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
A determination unit that determines that the tracking target is relatively large in the object scene image;
When a new scene image is generated, an update unit for updating the second template image and the first template image based on the new scene image,
When the determination unit determines that the tracking target is relatively large, out of the calculation results for the first and second template images by the calculation unit, the calculation results for the second template image A selection unit that selects one of the calculation results of the first and second template images by the calculation unit when the determination unit does not determine that the tracking target is relatively large. and,
An image tracking device comprising: a detection unit that detects a position of an image to be tracked in the object scene image based on the calculation result selected by the selection unit. An imaging unit that repeatedly captures an image formed by the imaging optical system and generates an object scene image;
A generating unit that generates a first template image and a second template image obtained by enlarging the first template image based on a tracking target region in the object scene image;
A computing unit for computing a correlation between an image in a search area provided in the scene image and each of the first and second template images;
When a new scene image is generated, an update unit for updating the second template image and the first template image based on the new scene image,
A focus adjustment unit that calculates a focus state of the imaging optical system and performs focus adjustment;
A determination unit that determines whether or not the tracking target is approaching the optical axis direction of the imaging optical system, based on a calculation result by the focus adjustment unit;
When the determination unit determines that the tracking target is approaching, the calculation result for the second template image among the calculation results for the first and second template images by the calculation unit When the determination unit does not determine the approach of the tracking target, a selection unit that selects one of the calculation results of the first and second template images by the calculation unit;
An image tracking device provided with the detection part which detects the position of the said tracking object in the said scene image based on the said calculation result selected by the said selection part.

Images