JP5233646B2 - Image tracking device, imaging device, and image tracking method - Google Patents

Description

  The present invention relates to an image tracking device, an imaging device, and an image tracking method.

When shooting a moving subject, the subject image of the specified part in the screen is acquired as a template image, and the position of the subject image similar to the template image is searched (template matching) among the images to be repeatedly shot. 2. Description of the Related Art An image tracking device that tracks a moving subject is known (see, for example, Patent Document 1).
JP 2006-058431 A

  In the conventional image tracking device described above, if the number of pixels of an image that is repeatedly shot is small, it is difficult to detect a small subject in the screen and it is difficult to follow the fine movement of the subject. It is better that the number of pixels of the image used for matching is large. However, there is a problem that the amount of calculation required for template matching increases as the number of pixels increases.

An image tracking device according to the present invention includes an imaging unit that repeatedly captures an object scene image formed by an imaging optical system, and a first image having a predetermined first resolution based on the object field image. Based on the creation means for creating the evaluation information and the second evaluation information at a predetermined second resolution higher than the first resolution, the reference information about the image to be tracked and the first evaluation information, First detection for detecting a candidate position of the tracking target in the image and detecting a position of the tracking target in the object scene image based on the reference information and the second evaluation information as necessary And detecting means for performing.
An imaging apparatus according to the present invention includes an image tracking device, a focus detection unit that detects a focus adjustment state of the imaging optical system for the position of the tracking target detected by the detection unit, and a focus adjustment state detected by the focus detection unit. And a focus adjusting means for adjusting the focus of the imaging optical system.
An image tracking method according to the present invention repeatedly captures a scene image of a scene formed by an imaging optical system, and uses first evaluation information at a predetermined first resolution based on the scene image. Generating second evaluation information at a predetermined second resolution higher than the first resolution, setting a first search range in the object field, and corresponding to the first search range in the first evaluation information A second search range narrower than the first search range is set in the object scene based on the evaluation information of the portion to be tracked and the reference information about the image to be tracked, and the second search range of the second evaluation information The position of the tracking target in the object scene image is detected based on the evaluation information and the reference information of the part corresponding to.

An image tracking device according to the present invention includes an imaging unit that repeatedly captures a scene image of a scene formed by an imaging optical system, and a first resolution at a predetermined first resolution based on the scene image. Based on the creation means for creating the 1 evaluation information and the second evaluation information at a predetermined second resolution higher than the first resolution, the reference information on the image to be tracked, and the first evaluation information, The tracking target in the scene image is detected based on the reference information and the second evaluation information, if necessary, by performing a first detection for detecting a candidate position of the tracking target in the scene image. Detecting means for detecting the position of the second detection means, and determination means for determining whether or not to perform the second detection. The detection means performs the second detection by the determination means. If it is determined not to do so, The candidate position detected in the detection of the position is set as the position of the tracking target, and the determination unit is configured according to the degree of similarity between the evaluation information of the portion corresponding to the candidate position in the first evaluation information and the reference information. And determining whether or not to perform the second detection.
An imaging apparatus according to the present invention detects the focus adjustment state of the imaging optical system for the position of the tracking target detected by the image tracking apparatus according to any one of claims 1 to 5 and the detection means. Focus detection means, and focus adjustment means for adjusting the focus of the imaging optical system based on the focus adjustment state detected by the focus detection means.
The image tracking method according to the present invention repeatedly captures a scene image of a scene imaged by an imaging optical system, and first evaluation information at a predetermined first resolution based on the scene image. And second evaluation information at a predetermined second resolution higher than the first resolution, a first search range is set in the object scene, and the first evaluation information includes the first evaluation range. A second search range narrower than the first search range is set in the object scene based on the evaluation information of the portion corresponding to one search range and the reference information on the tracking target image, and the tracking target First detection for detecting a candidate position of the tracking target in the object scene image is performed based on the reference information regarding the image of the image and the first evaluation information, and if necessary, the reference information and the second Based on the evaluation information Second detection is performed to detect the position of the tracking target, and the reference information and the second information are determined according to the similarity between the evaluation information of the portion corresponding to the candidate position in the first evaluation information and the reference information. Based on the evaluation information, it is determined whether to perform the second detection for detecting the position of the tracking target in the object scene image, and when it is determined not to perform the second detection, The candidate position detected by the first detection is set as the tracking target position.

  The focus adjustment state of the photographic lens (in this embodiment, the defocus amount) is detected in a plurality of focus detection areas set in the photographic screen, and the photographic lens is focused based on the defocus amount in one of the areas. The automatic focus adjustment (AF) function to be driven and the image of the subject to be tracked in the photographed image are stored as a template image (reference image). An image pickup apparatus having an image tracking function for tracking a subject to be tracked while searching for a position (template matching), and an image tracking device for tracking the tracking target while driving a photographing lens with the AF function and the image tracking function ( An embodiment of a single lens reflex digital still camera) will be described.

  FIG. 1 shows a configuration of an imaging apparatus (single-lens reflex digital still camera) 1 including an image tracking apparatus according to an embodiment. In FIG. 1, illustration and description of camera devices and circuits not directly related to the present invention are omitted. In a camera 1 according to an embodiment, an interchangeable lens 3 is detachably attached to a camera body 2. The camera body 2 is provided with a first image sensor 4 for capturing an object scene image and recording the image. The first image sensor 4 can be constituted by a CCD, a CMOS, or the like. At the time of photographing, the quick return mirror 5 and the sub mirror 6 are retracted to a position outside the photographing optical path indicated by a solid line, the shutter 7 is opened, and a subject image is formed on the light receiving surface of the first image sensor 4 by the photographing lens 8.

  At the bottom of the camera body 2, a focus detection optical system 9 and a distance measuring element 10 for detecting the focus adjustment state of the photographing lens 8 are provided. In this embodiment, an example is shown in which a focus detection method based on a pupil division type phase difference detection method is employed. The focus detection optical system 9 guides the pair of focus detection light fluxes that have passed through the photographing lens 8 to the light receiving surface of the distance measuring element 10 and forms a pair of optical images. The distance measuring element 10 includes, for example, a pair of CCD line sensors, and outputs a focus detection signal corresponding to the pair of optical images. Before shooting, the quick return mirror 5 and the sub mirror 6 are set at positions in the shooting optical path as indicated by broken lines, and the pair of focus detection light beams from the shooting lens 8 are transmitted through the half mirror portion of the quick return mirror 5. Then, the light is reflected by the sub mirror 6 and guided to the focus detection optical system 9 and the distance measuring element 10.

  A finder optical system is provided on the upper part of the camera body 2. Before shooting, the quick return mirror 5 and the sub mirror 6 are in the positions indicated by broken lines, and part of the subject light from the shooting lens 8 is reflected by the quick return mirror 5 and guided to the focusing screen 11, and on the focusing screen 11. A subject image is formed. The liquid crystal display element 12 superimposes and displays information such as a focus detection area mark on the object scene image formed on the focusing screen 11 and displays various shooting information such as an exposure value at a position outside the object scene image. indicate. The object scene image on the focusing screen 11 is guided to the eyepiece window 15 via the penta roof prism 13 and the eyepiece lens 14 so that the photographer can visually recognize the object scene image.

  The finder optical system at the top of the camera body 2 is provided with a second image sensor 16 that captures an object scene image for subject tracking and photometry. The object scene image formed on the focusing screen 11 is re-imaged on the light receiving surface of the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. The second imaging element 16 includes a plurality of pixels (photoelectric conversion elements) arranged in a matrix (in this embodiment, an example in which 640 pixels in the horizontal direction and 480 pixels in the vertical direction are arranged in a square) is shown. Yes. Further, each pixel is divided into three parts, and red R, green G, and blue B primary color filters are provided in these parts, respectively. Thereby, the RGB signal of the object scene image can be output for each pixel. Although details will be described later, tracking control and exposure calculation are performed based on the object scene image captured by the second image sensor 16. The tracking control and the exposure calculation may be performed based on the scene image captured by the first image sensor 4.

  The camera body 2 is also provided with a body drive control device 19 and an operation member 20. The body drive control device 19 is constituted by peripheral components such as a microcomputer, a memory, and an A / D converter, the details of which will be described later, and performs various controls and calculations of the camera 1. The operation member 20 includes a switch and a selector for operating the camera 1, such as a release button, a focus detection area selection switch, and a shooting mode selection switch.

  The interchangeable lens 3 is provided with a zooming lens 8a, a focusing lens 8b, a diaphragm 21, a lens drive control device 22, and the like. In this embodiment, the photographing lens 8 is representatively represented by a zooming lens 8a, a focusing lens 8b, and a diaphragm 21, but the configuration of the photographing lens 8 is not limited to the configuration shown in FIG. The lens drive control device 22 includes a microcomputer (not shown) and peripheral components such as a memory, a drive circuit, and an actuator, and performs drive control of the lenses 8a and 8b and the diaphragm 21 and detection of their positions. Lens information such as a focal length and an open aperture value of the interchangeable lens 3 is stored in a memory built in the lens drive control device 22.

  The body drive control device 19 and the lens drive control device 22 communicate via the contact 23 of the lens mount, and transmit information such as a lens drive amount and an aperture value from the body drive control device 19 to the lens drive control device 22. Lens information and aperture information are transmitted from the lens drive control device 22 to the body drive control device 19.

  FIG. 2 shows a detailed configuration of the body drive control device 19. Note that illustration and description of control functions not directly related to the present invention are omitted. The body drive control device 19 includes an element control circuit 19a, an A / D converter 19b, a microcomputer 19c, a memory 19d, and the like. The element control circuit 19 a controls charge accumulation and reading of the second image sensor 16. The A / D converter 19b converts the analog image signal output from the second image sensor 16 into a digital image signal. When the release button of the operation member 20 is half-pressed by the photographer, a half-press switch (not shown) is turned on, and as shown in FIG. 1, part of the subject light that has passed through the photographing lens 8 is a quick return indicated by a broken line. The light is reflected by the mirror 5 and guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. While the release button is pressed halfway, the second image sensor 16 can repeatedly and periodically capture the scene image.

  The microcomputer 19c constitutes a tracking control unit 19e, an exposure control unit 19f, a focus detection calculation unit 19g, and a lens drive amount calculation unit 19h according to a software form. The memory 19d stores information such as a template image for image tracking and defocus amount, lens information such as focal length, open F value, aperture value, and image shift amount to defocus amount conversion coefficient of the taking lens 8. To do.

  The tracking control unit 19e uses a template image as an image corresponding to the tracking target position manually specified by the photographer or the tracking target position automatically set by the camera 1 in the object scene image captured by the second image sensor 16. It is stored in the memory 19d as a (reference image), and the position of the object is recognized by searching an image area that matches or is similar to the template image from images that are repeatedly photographed thereafter. The exposure calculation unit 19f calculates an exposure value based on the image signal captured by the second image sensor 16.

  The focus detection calculation unit 19g detects the focus adjustment state of the photographing lens 8, here the defocus amount, based on the focus detection signal corresponding to the pair of optical images output from the distance measuring element 10. Although details will be described later, a plurality of focus detection areas are set in advance in the object field formed by the photographing lens 8. The distance measuring element 10 outputs a focus detection signal corresponding to the pair of light images for each focus detection area, and the focus detection calculation unit 19g defocuses based on the focus detection signal corresponding to the pair of light images for each focus detection area. Detect the amount. When the release button of the operation member 20 is half-pressed by the photographer, a half-press switch (not shown) is turned on, and as shown in FIG. 1, part of the subject light that has passed through the photographing lens 8 is a quick return indicated by a broken line. The light is guided to the distance measuring element 10 through the sub-mirror 6 and the focus detection optical system 9 through the half mirror part of the mirror 5, and the focus detection calculation part 19g performs the focus detection calculation. The lens driving amount calculation unit 19h converts the detected defocus amount into a lens driving amount. The focus adjustment state is repeatedly and periodically detected while the release button is pressed halfway.

  Next, an image tracking operation according to an embodiment will be described. 3 to 8 are diagrams for explaining the image tracking operation according to the embodiment, and FIGS. 9 and 10 are flowcharts illustrating the image tracking processing according to the embodiment. The quick return mirror 5 is set in the photographing optical path shown by a broken line in FIG. 1 except when the release button is fully pressed to shoot, and part of the subject light incident from the photographing lens 8 is on the focusing screen 11. Is imaged. The object scene image on the focusing screen 11 is guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18, and is repeatedly imaged by the second image sensor 16. Thereby, the object scene image signal is repeatedly output from the second image sensor 16.

  A plurality of focus detection areas are set in the range of the shooting screen of the shooting lens 8, and an area mark is superimposed on the object scene image on the focusing screen 11 by the liquid crystal display element 12, and the position of each focus detection area is determined. indicate. In this embodiment, as shown in FIG. 3, an example is shown in which focus detection areas 45a to 45g (denoted as a to g in the figure) are set at seven locations in the photographing screen. When an arbitrary area is selected by the focus detection area selection switch of the operation member 20, the mark of the area is lit up.

  As shown in FIG. 3, when the focus detection area 45b is selected by the focus detection area selection switch of the operation member 20, and the release button of the operation member 20 is half-pressed in this state, the focus detection area 45b is stored as the first AF area. 19d. Thereby, the subject in the first AF area is designated as the tracking target. In this example, the photographer selects the first AF area and manually designates the subject to be tracked. For example, in a camera having a function of automatically recognizing the subject, the first AF is performed based on the subject recognition result. An area and a tracking target subject may be set.

In step S <b> 1 of FIG. 9, an initial tracking image (an image acquired first after starting the image tracking process) is acquired by the second image sensor 16. Image information captured by the second image sensor 16 is represented by RGB values for each pixel.
R [x, y], G [x, y], B [x, y],
x = 1 to 640, y = 1 to 480 (1)

  Next, in step S2, the tracking control initial process shown in FIG. 10 is executed by the tracking control unit 19e. In step S100 of FIG. 10, the tracking control unit 19e creates evaluation information for obtaining a template image used for the image tracking process based on the initial tracking image acquired in step S1. Here, two types of images (evaluation images) having different resolutions are created from the initial tracking image, and these are used as evaluation information. For example, the evaluation image 40a shown in FIG. 4A and the evaluation image 40b shown in FIG. 4B are evaluated by reducing the resolution of the tracking initial image obtained from the shooting screen shown in FIG. Create as information. Here, the resolution of the evaluation image 40a is higher than that of the evaluation image 40b. For example, a 64 × 48 pixel evaluation image 40a and a 32 × 24 pixel evaluation image 40b are created from a tracking initial image of 640 × 480 pixels.

  Note that any method may be used as a method of reducing the resolution of the tracking initial image when creating the evaluation information in step S100. For example, in a tracking initial image, a predetermined number of adjacent pixels are grouped into one block, and the RGB values of all the pixels included in the block are averaged or summed for each block. The resolution of the initial image can be lowered. At this time, some pixels may be thinned out. Alternatively, one pixel in each block may be selected as the representative pixel of the block, and the evaluation image may be represented by the RGB value of the representative pixel.

  In step S101, the tracking control unit 19e displays image information ((1) of the position (here, the focus detection area 45b which is the initial AF area) designated by the photographer in the initial tracking image acquired in step S1 of FIG. Is stored as subject color information. As the color information used at this time, image information obtained from the second image sensor 16 may be used, or information of an evaluation image may be used. In step S102, as shown in FIG. 5A, the tracking control unit 19e displays color information similar to the subject color information in the position peripheral portion of the focus detection area 45b (see FIG. 3) in the tracking initial image. The same color information area 46 shown is detected. In subsequent step S103, the tracking control unit 19e sets the rectangular area including the same color information area 46 detected in step S102 as the initial tracking subject area 47.

  Here, an example in which the tracking subject area 47 is determined based on the subject color information is shown, but in order to simplify the processing, the size of the tracking subject area is uniformly set to 3 × 3 pixels, or The size of the tracking subject area may be determined according to the distance information of the photographing lens 8 and the image magnification of the subject.

  In step S104, the tracking control unit 19e includes the evaluation information created in step S100, that is, the images corresponding to the tracking subject region 47 in the tracking initial image determined in step S103, among the evaluation images 40a and 40b having different resolutions. Information is stored in the memory 19d as a template image. Thus, the template image 48a shown in FIG. 5B is stored in the memory 19d for the evaluation image 40a, and the template image 48b shown in FIG. 5C is stored in the memory 19d for the evaluation image 40b. The Each of these template images represents reference information regarding the image to be tracked.

  In step S105, the tracking control unit 19e sets a tracking target search range (first search range) in the first tracking calculation executed in step S6 of FIG. Here, as shown in FIG. 5A, a predetermined range surrounding the tracking subject region 47 is set as the first search range 49. Preferably, the first search range 49 is set around the tracking subject area 47. Since the tracking target subject moves with the currently detected position, that is, the tracking subject region 47 as a base point, by setting the first search range 49 around the tracking subject region 47, the tracking target subject is detected quickly. Image tracking responsiveness can be improved. The first search range 49 may be a region having a predetermined size enlarged by a predetermined number of pixels vertically and horizontally, or may be changed according to the tracking result or the size of the tracking subject.

  The tracking control initial process is thus completed, and the process returns to step S3 in FIG. In step S3 after the return, the tracking control unit 19e determines whether or not the release button of the operation member 20 has been fully pressed. If the release button is not fully pressed, the process proceeds to step S4. In step S4, a tracking next image is acquired by the second image sensor 16, and image information R [x, y], G [x, y], B [x, y] (x = 1) is obtained in the same manner as in step S1. ˜640, y = 1˜480). Thereby, for example, the following tracking image 41 shown in FIG. 6A is acquired.

  In subsequent step S5, the tracking control unit 19e creates two types of evaluation information having different resolutions based on the tracking next image acquired in step S4, as in step S100 of FIG. For example, from the tracking next image 41 of 640 × 480 pixels shown in FIG. 6A, an evaluation image 41a of 64 × 48 pixels shown in FIG. 6B and an evaluation of 32 × 24 pixels shown in FIG. 6C. The image 41b is created as evaluation information.

  In step S6, the tracking control unit 19e determines that the template image stored in the memory 19d in step S104 in FIG. 10 has a lower resolution and the evaluation information created in step S5 has a lower resolution. Based on the above, the first tracking calculation (first tracking calculation) is executed. That is, the first tracking calculation is executed based on the template image 48b in FIG. 5C and the evaluation image 41b in FIG. The state of the first tracking calculation is shown in FIG.

  In the first tracking calculation, as shown in FIG. 7, for each position in the portion corresponding to the first search range 49 (see FIG. 5A) set in step S <b> 105 of FIG. 10 in the evaluation image 41 b, A region having the same size as the template image 48b is cut out sequentially. Next, the degree of similarity between the image information corresponding to each area of the cut out evaluation image 41b and the template image 48b is calculated, and the degree of coincidence with the position where the degree of similarity is a predetermined value or more, that is, the template image 48b is predetermined. The position of the region exceeding the degree is detected as a tracking target candidate position. For example, if the similarity between the image information corresponding to the areas 42a, 42b, and 42c and the template image 48b in FIG. 7 is equal to or greater than a predetermined value, the positions of these areas 42a to 42c are set as candidate positions to be tracked. Detect each. In addition, said similarity is calculated | required by totaling the difference of RGB value about each pixel of the image information for every cut-out area | region, and the template image 48b. That is, the smaller the total value of RGB values in each pixel, the higher the degree of similarity, and the larger the total value, the lower the degree of similarity.

  Further, when the similarity is greater than or equal to a predetermined value for a plurality of areas, the area having the highest similarity, that is, the area most similar to the template image 48b is specified, and only the position of the area is determined. It may be detected as a tracking target candidate position. For example, if the similarity between the image information corresponding to the region 42b and the template image 48b is the highest among the regions 42a to 42c, the position of the region 42b is detected as a tracking target candidate position.

  In step S7, the tracking control unit 19e determines whether or not to execute the second tracking calculation (second tracking calculation). This determination is made based on the result of the first tracking calculation in step S6. For example, as described above, when a plurality of tracking target candidate positions are detected, that is, when the number of regions whose similarity is equal to or greater than a predetermined value is two or more, or no tracking target candidate positions are detected. In this case, that is, when the number of regions whose similarity is equal to or greater than a predetermined value is 0, it is necessary to detect the position of the correct tracking target by the second tracking calculation, and an affirmative determination is made in step S7 and the process proceeds to step S8. . On the other hand, if only one candidate position to be tracked is detected, that is, if the number of regions whose similarity is equal to or greater than the predetermined value is 1, it is determined that the second tracking calculation is not necessary, and step S7 is denied. Determination is made and the process proceeds to step S10 without executing steps S8 and S9.

  However, even when there is one tracking target candidate position detected in the first tracking calculation in step S6, there are a plurality of focus detection areas 45a to 45g corresponding to the candidate position in FIG. There is. In such a case, an affirmative determination is made in step S7 in order to narrow down the focus detection area to be subjected to focusing control to one by detecting the exact position of the tracking target by the second tracking calculation, and step S8. Proceed to That is, in step S7, when only one tracking target candidate position is detected and the number of focus detection areas corresponding to the candidate position is 1, it is determined that the second tracking calculation is not performed. In this way, the second tracking calculation is executed as necessary. Note that the determination in step S7 may be omitted and the second tracking calculation may always be executed.

  In step S8, the tracking control unit 19e determines a predetermined range corresponding to the candidate position based on the candidate position of the tracking target detected by the first tracking calculation in step S6 as a tracking target search range in the second tracking calculation. Set to the object field as (second search range). Here, for example, a range that is enlarged vertically and horizontally by the predetermined number of pixels of the evaluation image 41a around the tracking target candidate position is set as the second search range. Thus, the second search range is set to be narrower than the first search range. Further, when the portion of the evaluation image 41a corresponding to the candidate position of the tracking target corresponds to the range of the corresponding scene in the evaluation image 41b, specifically, the resolution of the evaluation image 41a is reduced to be equivalent to the evaluation image 41b, the tracking target The range in which the image information (RGB values) corresponding to the candidate position is reflected in the evaluation image 41b is preferably set as the second search range.

  In step S9, the tracking control unit 19e determines that the template image stored in the memory 19d in step S104 in FIG. 10 has a higher resolution and the evaluation information created in step S5 has a higher resolution. Based on the above, the second tracking calculation is executed. That is, the second tracking calculation is executed based on the template image 48a in FIG. 5B and the evaluation image 41a in FIG. The state of this second tracking calculation is shown in FIG.

  In the second tracking calculation, first, an area having the same size as the template image 48a is sequentially cut out for each position of the portion corresponding to the second search range set in step S8 in the evaluation image 41a. For example, as shown in FIG. 8, when a range that is enlarged by one pixel up, down, left, and right around the tracking target candidate position 42b obtained by the first tracking calculation is set as the second search range 50, the candidate position 42b The corresponding area 43a and the areas indicated by reference numerals 43b to 43i obtained by shifting the area 43a by one pixel vertically and horizontally are cut out. Next, similar to the first tracking calculation in step S6, the similarity between the image information corresponding to each area of the cut out evaluation image 41a and the template image 48a is calculated, and the position of the area with the highest similarity is calculated. That is, the position of the region most similar to the template image 48a is detected as the tracking target position. For example, if the similarity between the image information corresponding to the area 43h and the template image 48a is the highest, the position of the area 43h is detected as the position of the tracking target. Here, when there are a plurality of candidate positions as a result of the first tracking calculation, the second tracking calculation may be performed for each candidate position.

  In step S10, the tracking control unit 19e determines the area corresponding to the position of the tracking target detected by the second tracking calculation in step S9 as the new tracking subject area 47. However, if it is determined in step S7 that the second tracking calculation is not performed, the area corresponding to the tracking target candidate position detected by the first tracking calculation in step S6 is set as a new tracking subject area 47. Thereby, the position of the new tracking target is detected.

  After the new tracking subject area 47 is determined as described above, the focus control in step S12 is performed on the tracking target included in the tracking subject area 47. When at least one of the focus detection areas 45a to 45g is included in the new tracking subject area 47, the focus control is performed in the focus detection areas 45a to 45g included in the new tracking subject area 47 by the focus detection calculation unit 19g. The smallest defocus amount detected is adopted. Based on this adopted defocus amount, the lens drive amount calculation unit 19h and the lens drive control device 22 adjust the focus of the photographing lens 8. In the focus control, when the focus detection areas 45a to 45g are not included in the new tracking subject region 47, the distance to the new tracking subject region 47 or the focus detection areas 45a to 45g employed in the previous tracking result are used. Focus detection areas 45a to 45g to be employed may be determined based on at least one of the distances to Here, based on the image information of the portion corresponding to the new tracking subject region 47 in the evaluation image 41a in FIG. 6B and the evaluation image 41b in FIG. 6C created in step S5, the template images 48a and 48b. May be updated.

  When the template images 48a and 48b are updated, the template images 48a and 48b are added by adding 20% of the image information of the new tracking subject area 47 to 80% of the image information of the original template images 48a and 48b. The latest image information may be added little by little to the original image information to suppress the influence of a sudden change in the tracking subject due to some cause, thereby improving the subject tracking reliability.

  The template images 48a and 48b are not updated every time the tracking calculation is performed, but only when the similarity calculated at the position determined as the new tracking subject region 47 is larger than a predetermined threshold value. You may do it.

  In step S <b> 11, the tracking control unit 19 e sets a new first search range 49 for the new tracking subject area 47 in the scene. Here, as in step S <b> 105, a predetermined range surrounding the tracking subject region 47 is set as the first search range 49. The search area 49 may be an area having a predetermined size enlarged by a predetermined number of pixels in the vertical and horizontal directions, or may be changed according to the tracking result and the size of the tracking subject.

  After the first search area 49 is set in step S11, the focus control as described above is performed in step S12. After performing the focus control in step S12, the process returns to step S3, and the processes in steps S4 to S12 are repeatedly executed until the release button of the operation member 20 is fully pressed. Thereby, a new object scene image is repeatedly acquired as a tracking next image by the second image sensor 16 in step S4, and the position of the tracking target is repeatedly detected in step S10 based on the tracking next image.

  If a full pressing operation of the release button of the operation member 20 is detected in step S3, the process proceeds to step S13, and photographing control is executed in step S13. In the imaging control, an exposure calculation is performed by the exposure calculation unit 19f based on the luminance information of the new tracking subject area 47 determined in step S10, and an aperture value and a shutter speed are calculated. The shutter 7 and the diaphragm 21 are driven and controlled in accordance with these exposure values, and a captured image obtained by imaging with the first imaging element 4 is recorded. When the shooting control of step S13 is executed, the flowchart of FIG.

  According to the embodiment described above, the following operational effects are obtained.

(1) The imaging device 1 repeatedly captures an image of the object scene formed by the photographing lens 8 with the second image sensor 16 as a tracking next image (step S4). Based on this scene image, the tracking controller 19e creates evaluation images 41a and 41b having different resolutions as evaluation information (step S5). Then, the tracking control unit 19e executes a first tracking calculation for detecting the tracking target candidate position in the object scene image based on the template image 48b representing the reference information regarding the tracking target image and the evaluation image 41b ( Step S6). Further, if necessary, a second tracking calculation for detecting the position of the tracking target in the object scene image is executed based on the template image 48a and the evaluation image 41a (step S9). Thus, by performing the tracking calculation using the evaluation image having a resolution lower than that of the original scene image, it is possible to reduce the calculation amount required for template matching in the tracking calculation.

(2) The imaging apparatus 1 sets the first search range 49 in the object scene by the tracking control unit 19e (steps S11 and 105) and sets a second search range narrower than the first search range 49 ( Step S8). In step S6, the first tracking calculation is performed based on the image information of the portion corresponding to the first search range 49 in the evaluation image 41b and the template image 48b. In step S9, the second tracking information in the evaluation image 41a. The second tracking calculation is performed based on the image information of the part corresponding to the search range and the template image 48a. As a result, the amount of calculation can be further reduced by limiting the range in which the second tracking calculation is performed.

(3) In step S8, it is preferable to set the range of the object scene corresponding to the tracking target candidate position detected by the first tracking calculation as the second search range. In this way, the second search range can be set appropriately.

(4) The imaging device 1 determines whether or not to perform the second tracking calculation in step S9 by the tracking control unit 19e (step S7), and when it is determined not to perform the detection, the first tracking calculation in step S6 detects it. The tracked candidate position of the tracking target is set as the tracking target position (step S10). Since it did in this way, when it is not necessary to perform a 2nd tracking calculation, a 2nd tracking calculation can be abbreviate | omitted and the amount of calculations can be reduced further.

(5) Whether or not the tracking control unit 19e performs the second tracking calculation in step S7 according to the similarity between the image information of the part corresponding to the first search range 49 in the evaluation image 41b and the template image 48b. Determine whether. That is, in the first tracking calculation of step S6, the similarity between the image information of each part corresponding to the plurality of candidate positions in the evaluation image 41b and the template image 48b is calculated for a plurality of candidate positions to be tracked. . When the number of candidate positions whose similarity calculated in this way exceeds a predetermined threshold is 0 or 2 or more, it is determined that the second tracking calculation is performed. Further, the number of candidate positions whose similarity exceeds the threshold is 1, and the number of focus detection areas corresponding to candidate positions whose similarity exceeds the threshold among the focus detection areas 45a to 45g is 1. In this case, it is determined that the second tracking calculation is not performed. By making such a determination, it is possible to appropriately determine whether or not to perform the second tracking calculation.

(6) When the imaging apparatus 1 detects the position of the tracking target in step S10, the imaging apparatus 1 performs focusing control for the position (step S12). That is, a defocus amount indicating the focus adjustment state of the photographic lens 8 is detected by the focus detection calculation unit 19g, and based on the detected defocus amount, the lens drive amount calculation unit 19h and the lens drive control device 22 Adjust the focus. Thereby, it is possible to always focus on the tracking target that moves.

  In the embodiment described above, in step S5, two types of evaluation images with different resolutions are created as evaluation information based on the following tracking image acquired in step S4, and using these two types of evaluation images, In steps S6 and S9, two-stage tracking calculation is performed. However, the type of evaluation information created in the present invention and the number of stages of tracking operations to be executed are not limited to this. In other words, three or more types of evaluation images with different resolutions may be created as evaluation information, and among these evaluation images, the ones with the lowest resolution to the higher ones may be used in order to perform the tracking calculation in three or more stages. Further, the resolution image to be used may be changed depending on the image magnification of the subject, other photographing conditions, and the like.

  In the second tracking calculation performed in step S9, the second tracking calculation may be used as it is as an evaluation image without reducing the resolution of the subsequent tracking image acquired in step S4. In this case, in steps S5 and S100, as the evaluation information based on the acquired initial tracking image or subsequent tracking image, the same evaluation image as the initial tracking image or the subsequent tracking image and the resolution of the initial tracking image or the subsequent tracking image are reduced. An evaluation image is created as two types of evaluation information having different resolutions.

1 is a cross-sectional view illustrating a configuration of an imaging device including an image tracking device according to an embodiment; 1 is a block diagram illustrating a configuration of an imaging device including an image tracking device according to an embodiment; The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The figure for demonstrating the image tracking method of one Embodiment The flowchart which shows the image tracking processing of one embodiment Flow chart showing tracking control initial processing

Explanation of symbols

DESCRIPTION OF SYMBOLS 1; Imaging device, 8; Shooting lens, 16; 2nd imaging device, 19; Body drive control device, 19e; Tracking control part, 19d;

Claims (8)

Imaging means for repeatedly capturing a field image of the field imaged by the imaging optical system;
Creating means for creating first evaluation information at a predetermined first resolution and second evaluation information at a predetermined second resolution higher than the first resolution based on the scene image;
Based on the reference information on the tracking target image and the first evaluation information, a first detection for detecting the tracking target candidate position in the object scene image is performed, and if necessary, the reference information and the Detecting means for performing second detection for detecting a position of the tracking target in the object scene image based on second evaluation information ;
Determining means for determining whether or not to perform the second detection,
When the detection means determines that the second detection is not performed by the determination means, the candidate position detected by the first detection is set as the position of the tracking target,
The determination means determines whether or not to perform the second detection according to the similarity between the evaluation information of the portion corresponding to the candidate position in the first evaluation information and the reference information. An image tracking device. The image tracking device according to claim 1,
First setting means for setting a first search range in the object scene;
A second setting means for setting a second search range narrower than the first search range in the object scene,
The detection means performs the first detection based on the evaluation information of the portion corresponding to the first search range in the first evaluation information and the reference information, and the first of the second evaluation information. An image tracking apparatus that performs the second detection based on evaluation information of a portion corresponding to two search ranges and the reference information. The image tracking device according to claim 1 ,
First setting means for setting a first search range in the object scene;
A second setting means for setting, as a second search range, a range corresponding to the tracking target candidate position in the object scene,
The detection means performs the first detection based on the evaluation information of the portion corresponding to the first search range in the first evaluation information and the reference information, and the first of the second evaluation information. An image tracking apparatus that performs the second detection based on evaluation information of a portion corresponding to two search ranges and the reference information. In the image tracking device according to any one of claims 1 to 3 ,
The detecting means detects a plurality of the candidate positions;
The determination unit calculates the similarity between the evaluation information of each part corresponding to the candidate position in the first evaluation information and the reference information, and the calculated similarity exceeds the predetermined threshold. An image tracking device, wherein the second detection is determined to be performed when the number of positions is 0 or 2 or more. The image tracking device according to claim 4 ,
The determination means has a number of candidate positions where the similarity exceeds the threshold, and the similarity is the threshold among a plurality of focus detection positions set in advance in the object scene. When the number of focus detection positions corresponding to the candidate positions exceeding 1 is 1, it is determined that the second detection is not performed. The image tracking device according to any one of claims 1 to 5 ,
A focus detection means for detecting a focus adjustment state of the imaging optical system for the position of the tracking target detected by the detection means;
An imaging apparatus comprising: a focus adjustment unit that performs focus adjustment of the imaging optical system based on the focus adjustment state detected by the focus detection unit. Repetitively capture the field image of the field imaged by the imaging optical system,
Based on the scene image, creating first evaluation information at a predetermined first resolution and second evaluation information at a predetermined second resolution higher than the first resolution,
Setting a first search range in the object scene;
Based on the evaluation information of the portion corresponding to the first search range in the first evaluation information and the reference information regarding the image to be tracked, the second narrower than the first search range in the object scene. Set the search range,
Based on the reference information on the tracking target image and the first evaluation information, first detection for detecting the tracking target candidate position in the object scene image is performed, and if necessary, the reference information and Based on the second evaluation information, performing a second detection for detecting the position of the tracking target in the scene image,
Based on the reference information and the second evaluation information, according to the similarity between the evaluation information of the portion corresponding to the candidate position in the first evaluation information and the reference information, the scene image in the scene image Determining whether to perform the second detection for detecting the position of the tracking target;
An image tracking method , wherein when it is determined not to perform the second detection, the candidate position detected by the first detection is set as the position of the tracking target . Imaging means for repeatedly capturing a field image of the field imaged by the imaging optical system;
Creating means for creating first evaluation information at a predetermined first resolution and second evaluation information at a predetermined second resolution higher than the first resolution based on the scene image;
First detection for detecting a candidate position of the tracking target in the object scene image based on the reference information on the tracking target image and the first evaluation information , and the reference information and the second evaluation information A detection means for performing a second detection for detecting a position of the tracking target in the object scene image,
Determination means for determining whether or not to perform the second detection according to the similarity between the reference information and the information of the portion corresponding to the candidate position in the first evaluation information,
The detection means, when it is determined by the determination means that the second detection is not performed, sets the candidate position detected by the first detection as the position of the tracking target. apparatus.

Images