JP2016201636A - Image tracking device and image tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image tracking device and an image tracking method, where incorrect detection of a tracking target is less likely to occur, even if there are many subjects similar to the original tracking target.SOLUTION: An image tracking device is configured to include selection of a plurality of tracking target candidates among captured images (selection means) S12, and setting of a tracking target based on the shooting state among the plurality of tracking target candidates thus selected (setting means) S17, S19, S20. The setting means sets the tracking target based on at least one of whether or not the imaging apparatus including an image tracking device is in quiescent state, and whether the focus length of a taking lens included in the imaging apparatus is on the short focus end side or long focus end side.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image tracking device and an image tracking method for setting a tracking target in a captured image.

  An image tracking device that can set a tracking target in a captured image is known. For example, Patent Document 1 describes a specific configuration of this type of image tracking apparatus.

  The image tracking device described in Patent Literature 1 captures a plurality of tracking target candidates in descending order of similarity to a reference image (template image) from a captured image, and is similar among the captured plurality of tracking target candidates. The one with the highest degree is set as the tracking target. By capturing a plurality of tracking target candidates, it is possible to easily return to the original tracking target even if the tracking target is set in error. For example, when one of a plurality of soccer players wearing the same uniform is being tracked, even if another player is set as the tracking target depending on the player's orientation, the player who is the original tracking target at a later timing When the direction changes, the tracking target returns to the player at that time.

Japanese Patent No. 4935380

  In the image tracking device described in Patent Document 1, a subject having the highest similarity is set as a tracking target. Therefore, in the scene where the similarity is lowered depending on the direction of the tracking target as in the example of the soccer player, an erroneous setting of the tracking target cannot be avoided. In addition, even in a scene where a difference between the template image and the template image is smaller than the original tracking target in the captured image, an erroneous setting of the tracking target cannot be avoided. That is, in the image tracking device described in Patent Document 1, when there are many subjects similar to the original tracking target, there is a problem that the tracking target is likely to be erroneously set due to the similarity of these subjects changing. Is done.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image tracking device that is unlikely to cause erroneous detection of a tracking target even when there are many subjects similar to the original tracking target. And providing an image tracking method.

  An image tracking apparatus according to an embodiment of the present invention sets a tracking target based on a shooting state from selection means for selecting a plurality of tracking target candidates from a captured image, and a plurality of tracking target candidates selected. Setting means.

  According to an embodiment of the present invention, since the tracking target is set based on the shooting state, even in a scene where the degree of similarity decreases depending on the direction of the tracking target, as in the example of the soccer player. Even in a scene where the difference between the captured image and the template image is smaller than the original tracking target, erroneous setting of the tracking target can be avoided. That is, according to an embodiment of the present invention, even if there are many subjects similar to the original tracking target, erroneous detection of the tracking target can be avoided.

In one embodiment of the present invention, the setting means includes
(1) Whether or not the photographing device including the image tracking device is in a stationary state,
(2) Whether the focal length of the photographing lens provided in the photographing apparatus is the short focal end side or the long focal end side,
The tracking target may be set based on at least one of the above.

The image tracking device according to an embodiment of the present invention may include a motion estimation unit that estimates the motion of a subject in a captured image. In this case, the setting means
(A) When the imaging device is stationary,
(B) When the focal length of the taking lens is on the short focal end side,
(C) When the difference value between the previous captured image and the current captured image is less than the first threshold value,
When at least one of the above is satisfied, the tracking target may be set based on the estimation result of the motion of the subject by the motion estimation means from the plurality of tracking target candidates.

  In one embodiment of the present invention, the setting unit compares the previous captured image with the current captured image, the difference value of the entire captured image is greater than or equal to a second threshold value, and a predetermined area in the captured image When the difference value at is less than the third threshold, a tracking target candidate that appears in the predetermined area may be set as a tracking target.

  The image tracking device according to an embodiment of the present invention includes a storage unit that stores a reference image representing a tracking target, and a similarity that calculates a similarity between the stored reference image and each subject image in the captured image. It is good also as a structure provided with a degree calculation means. In this case, when the difference value of the entire captured image is equal to or greater than the second threshold value and the difference value in the predetermined area in the captured image is equal to or greater than the third threshold value, the setting unit calculates the result of the similarity calculation unit The subject having the highest similarity may be set as the tracking target based on the above.

  The image tracking method according to an embodiment of the present invention includes a step of selecting a plurality of tracking target candidates from a captured image, and tracking based on a predetermined shooting state from the selected plurality of tracking target candidates. And setting a target.

  According to an embodiment of the present invention, an image tracking device and an image tracking method are provided that are unlikely to cause erroneous detection of a tracking target even when there are many subjects similar to the original tracking target.

It is a block diagram which shows the structure of the imaging device which concerns on one Embodiment of this invention. It is a figure which shows the flowchart of the image tracking process performed by the system controller with which an imaging device is provided in one Embodiment of this invention. It is a figure which shows the example of a screen displayed on the display screen of LCD of an imaging device when wide-angle imaging | photography is performed in one Embodiment of this invention. It is a figure which shows the example of a screen displayed on the display screen of LCD of an imaging device when telephoto shooting is performed in one Embodiment of this invention.

  An imaging apparatus according to an embodiment of the present invention will be described below with reference to the drawings. In the following, a digital single lens reflex camera will be described as an embodiment of the present invention. Note that the photographing apparatus is not limited to a digital single lens reflex camera, but includes, for example, a mirrorless single lens camera, a compact digital camera, a video camera, a camcorder, a tablet terminal, a PHS (Personal Handy phone System), a smartphone, a feature phone, a portable game machine, and the like. The apparatus may be replaced with another type of apparatus having a photographing function.

[Configuration of the photographing apparatus 1]
FIG. 1 is a block diagram showing a configuration of the photographing apparatus 1 of the present embodiment. As shown in FIG. 1, the photographing apparatus 1 includes a system controller 100, an operation unit 102, a drive circuit 104, a photographing lens 106, a diaphragm 108, a shutter 110, a solid-state image sensor 112, a signal processing circuit 114, an image processing engine 116, Buffer memory 118, card interface 120, LCD (Liquid Crystal Display) control circuit 122, LCD 124, ROM (Read Only Memory) 126, gyro sensor 128, acceleration sensor 130, geomagnetic sensor 132, and GPS (Global Positioning System) sensor 134 I have. Note that the photographing lens 106 is a zoom lens composed of a plurality of lenses, but is shown as a single lens in FIG. 1 for convenience.

  The operation unit 102 includes various switches necessary for the user to operate the photographing apparatus 1, such as a power switch, a release switch, and a photographing mode switch. When the user operates the power switch, power is supplied from the battery (not shown) to the various circuits of the photographing apparatus 1 through the power line.

  The system controller 100 includes a CPU (Central Processing Unit) and a DSP (Digital Signal Processor). After supplying power, the system controller 100 accesses the ROM 126, reads out a control program, loads it into a work area (not shown), and executes the loaded control program to control the entire photographing apparatus 1.

  When the release switch is operated, the system controller 100 is measured by, for example, a photometric value calculated based on an image captured by the solid-state image sensor 112 or an exposure meter (not shown) built in the photographing apparatus 1. The diaphragm 108 and the shutter 110 are driven and controlled via the drive circuit 104 so that proper exposure can be obtained based on the photometric values. More specifically, drive control of the aperture 108 and the shutter 110 is performed based on an AE function designated by a shooting mode switch such as a program AE (Automatic Exposure), shutter priority AE, aperture priority AE, and the like. Further, the system controller 100 performs AF (Autofocus) control together with AE control. An active method, a phase difference detection method, a contrast detection method, or the like is applied to the AF control. The AF mode includes a central single-point ranging mode using a single central ranging area, a multi-point ranging mode using a plurality of ranging areas, and the like. The system controller 100 controls driving of the photographing lens 106 via the driving circuit 104 based on the AF result, and adjusts the focus of the photographing lens 106. Since the configuration and control of this type of AE and AF are well known, detailed description thereof is omitted here.

  The light flux from the subject passes through the photographing lens 106, the diaphragm 108, and the shutter 110 and is received by the light receiving surface of the solid-state image sensor 112. The solid-state imaging device 112 is a single-plate color CCD (Charge Coupled Device) image sensor having a Bayer pixel arrangement. The solid-state imaging device 112 accumulates an optical image formed by each pixel on the light receiving surface as a charge corresponding to the amount of light, and generates R (Red), G (Green), and B (Blue) image signals. Output. The solid-state image sensor 112 is not limited to a CCD image sensor, and may be replaced with a CMOS (Complementary Metal Oxide Semiconductor) image sensor or other types of imaging devices. The solid-state image sensor 112 may also have a complementary color filter mounted thereon.

  The signal processing circuit 114 performs predetermined signal processing such as clamping and demosaicing on the image signal input from the solid-state imaging device 112 and outputs the processed signal to the image processing engine 116. The image processing engine 116 performs predetermined signal processing such as matrix operation, Y / C separation, and white balance on the image signal input from the signal processing circuit 114 to generate a luminance signal Y and color difference signals Cb and Cr. , And compressed in a predetermined format such as JPEG (Joint Photographic Experts Group). The buffer memory 118 is used as a temporary storage location for processing data when the image processing engine 116 executes processing. Further, the storage format of the captured image is not limited to the JPEG format, and may be a RAW format in which only minimal image processing (for example, black level correction) is performed.

  A memory card 200 is detachably inserted into a card slot of the card interface 120.

  The image processing engine 116 can communicate with the memory card 200 via the card interface 120. The image processing engine 116 stores the generated compressed image signal (captured image data) in the memory card 200 (or a built-in memory (not shown) provided in the image capturing apparatus 1).

  Further, the image processing engine 116 buffers the generated luminance signal Y and color difference signals Cb and Cr in a frame memory (not shown) in units of frames. The image processing engine 116 sweeps the buffered signal from each frame memory at a predetermined timing, converts it into a video signal of a predetermined format, and outputs it to the LCD control circuit 122. The LCD control circuit 122 modulates and controls the liquid crystal based on the image signal input from the image processing engine 116. Thereby, the photographed image of the subject is displayed on the display screen of the LCD 124. The user can view a real-time through image (live view) photographed with appropriate exposure and focus based on the AE control and AF control through the display screen of the LCD 124.

  When the user performs a reproduction operation of the photographed image, the image processing engine 116 reads the photographed image data designated by the operation from the memory card 200 or the built-in memory, converts it into an image signal of a predetermined format, and the LCD control circuit 122. Output to. The LCD control circuit 122 performs modulation control on the liquid crystal based on the image signal input from the image processing engine 116, so that a captured image of the subject is displayed on the display screen of the LCD 124.

[Image tracking processing]
FIG. 2 shows a flowchart of image tracking processing executed by the system controller 100 in an embodiment of the present invention. The image tracking process in FIG. 2 is started for the first time when, for example, an image to be tracked (hereinafter referred to as “tracking target template image” for convenience of description) is set. The second and subsequent executions of the image tracking process in FIG. 2 are started each time a captured image is updated (a new frame image is captured).

  Illustratively, an image of a subject that falls within a designated frame designated in the photographed image by a user operation, a subject image automatically set on the photographing device 1 side, a storage area such as the buffer memory 118 or the ROM 126. An image designated by a user operation from among a plurality of pre-stored prescribed images is set as a tracking target template image. The set tracking target template image is temporarily stored in a storage area such as the buffer memory 118 during the image tracking process shown in FIG. When the image tracking process of FIG. 2 is executed, AF control is performed so that the subject set as the tracking target is in focus.

[S11 in FIG. 2 (type detection of tracking target template image)]
In this embodiment, a large number of type determination images (images of people, animals, plants, vehicles, buildings, etc.) are stored in advance in a storage area such as the ROM 126. In some cases, an image of a subject extracted from a captured image in response to a user operation is additionally stored in a storage area such as the buffer memory 118 as a type determination image.

  In this processing step S11, pattern matching processing is performed between the tracking target template image and the type determination image stored in the storage area such as the buffer memory 118 or the ROM 126, and the type determination with the smallest difference value from the tracking target template image is performed. The type of image for use (person, animal, plant, vehicle, building, etc.) is detected as the type of tracking target template image. The technique for detecting the type of image is a well-known technique, and can be referred to, for example, in Japanese Unexamined Patent Application Publication Nos. 2010-81246 and 2011-212695. Further, the type of the tracking target template image does not change unless the tracking target template image is changed. Therefore, the execution of this processing step S11 is limited to the first execution of the image tracking process of FIG. 2 (when the tracking target template image is set or executed immediately after the setting change), and the second and subsequent times of the image tracking process are performed. Omitted at run time.

[S12 in FIG. 2 (Selection of Tracking Target Candidate)]
In this processing step S12, a plurality of tracking target candidates are selected from the captured image. Illustratively, a plurality of image areas having a high similarity to the tracking target template image are searched from the captured images by pattern matching processing, and subjects appearing in the searched image areas are selected as tracking target candidates. In the present embodiment, an image area having a small difference value with the tracking target template image (less than a predetermined threshold) is searched as an image area having a high similarity. Hereinafter, for convenience of explanation, the threshold used in this processing step S12 is referred to as “candidate selection threshold”.

  The threshold for candidate selection includes a threshold related to the similarity of shape to the tracking target template image and a threshold related to the similarity of color to the tracking target template image (color threshold). Hereinafter, the threshold relating to the similarity of shape is referred to as “shape threshold”, and the threshold relating to the similarity of color is referred to as “color threshold”. The shape threshold and the color threshold are set according to the type of the tracking target template image detected in processing step S11 (detection of the type of tracking target template image). Examples of setting the shape threshold and the color threshold are given below.

Setting example 1 (type of tracking target template image is person)
A person is likely to change its shape when viewed from the photographing apparatus 1 side when the direction of the body changes. On the other hand, although the color of skin changes with illumination, it is not easy to change compared to the shape. Therefore, in this setting example 1, the shape threshold is set to a moderate value, and the color threshold is set to a strict value.

・ Setting example 2 (Tracking target template image type is animal)
Animals are more diverse than people. Therefore, in this setting example 2, the shape threshold value is set to a normal value (intermediate value between a loose value and a strict value), and the color threshold value is also set to a normal value.

・ Setting example 3 (Tracking target template image type is car)
Artifacts such as cars fall into a category with less change in shape and color depending on the shooting situation compared to other types. Therefore, in setting example 3, the shape threshold is set to a normal value, and the color threshold is set to a strict value.

・ Setting example 4 (Tracking target template image type is flower)
This setting example 4 is a setting example when a flower is macro-photographed. In this setting example 4, the shape threshold value is set to a strict value and the color threshold value is also set to a strict value in order to make a strict determination on a subject that slightly shakes during macro shooting.

[S13 in FIG. 2 (Acquisition of Shooting State Information)]
In this processing step S13, information on the current shooting state is acquired. In the present embodiment, as the current shooting state information, for example,
(1) State of the photographing apparatus 1 (stationary state or non-stationary state)
(2) Focal length of the taking lens 106 (focal length on the short focal end side or focal length on the long focal end side)
Is acquired.

Acquisition example of shooting state information (1) In the present embodiment, separately from the image tracking process of FIG. 2, a determination process for determining whether or not the housing of the shooting apparatus 1 is in a stationary state is periodically executed. Has been. In this determination process, for example, when the amplitude of a signal component having a frequency equal to or higher than a certain frequency among the vibration detection signals input from the gyro sensor 128 is continuously within a certain threshold for a certain period of time, it is determined to be in a stationary state. If it does not fall within the threshold value, it is determined that it is in a non-stationary state. In this processing step S13, information (whether it is a stationary state or a non-stationary state) according to the determination result of this determination processing is acquired. Typically, the stationary state of the housing of the photographing apparatus 1 includes a state where the photographing apparatus 1 is fixed to a tripod.

  The stationary state of the imaging device 1 may be determined based on information output from other sensors such as the acceleration sensor 130, the geomagnetic sensor 132, and the GPS sensor 134 instead of the gyro sensor 128. In order to improve the determination accuracy, for example, sensor fusion technology may be applied so that information output from these sensors may be used in combination.

-Acquisition example of shooting state information (2) By communicating with the shooting lens 106, information on the focal length (angle of view) of the shooting lens 106 is acquired. In this processing step S13, when an intermediate focal length between the short focal end (wide angle end) and the long focal end (telephoto end) is defined as the intermediate focal length, a focal length shorter than the intermediate focal length is expressed as “short focal end. Is obtained as information on the focal length of the “side”, and a focal length greater than or equal to the middle focal length is obtained as information on the focal length of the “long focal end”.

[S14 in FIG. 2 (Determination of the state of the photographing apparatus 1)]
In this processing step S14, it is determined whether or not the photographing apparatus 1 is in a stationary state based on the information acquired in processing step S13 (acquisition of photographing state information).

[S15 in FIG. 2 (Focal Length Determination of Shooting Lens 106)]
This process step S15 is executed when it is determined in process step S14 (state determination of the image capturing apparatus 1) that the image capturing apparatus 1 is in a non-stationary state (S14: NO). In this processing step S15, it is determined whether the focal length of the photographing lens 106 is the short focal end side or the long focal end side based on the information acquired in the processing step S13 (acquisition of shooting state information).

[S16 in FIG. 2 (Determination of Change in Captured Image)]
This processing step S16 is performed when it is determined in processing step S15 (focal length determination of the photographing lens 106) that the focal length of the photographing lens 106 is on the long focal end side (telephoto end side) (S15: long focal end side). ) Is executed. In this processing step S16, the difference value between the captured image of the previous frame (the frame immediately before the current frame) and the captured image of the current frame is less than a predetermined value (for example, a predetermined level between corresponding pixels (pixels having the same address)). It is determined whether or not the number of objects that are greater than or equal to the difference is less than a predetermined number), in other words, whether or not the overall change in the captured image is small between the previous frame and the current frame.

[S17 in FIG. 2 (Tracking Target Setting)]
This processing step S17
(A) When it is determined in the processing step S14 (state determination of the image capturing device 1) that the image capturing device 1 is in a stationary state (S14: YES),
(B) When it is determined in processing step S15 (focal length determination of the photographing lens 106) that the focal length of the photographing lens 106 is on the short focal end side (wide angle end side) (S15: short focal end side).
(C) When it is determined that the difference value between the captured image of the previous frame and the captured image of the current frame is less than a predetermined value in the processing step S16 (determination of captured image change) (S16: YES)
It is executed in either case.

  When the photographing apparatus 1 is in a stationary state (above (A)), the photographing range is fixed. Therefore, basically, the change in the background is small, and the change in the captured image is generally small between the previous frame and the current frame. Further, when the focal length of the photographing lens 106 is on the short focal end side (wide-angle end side) (hereinafter, for convenience of explanation, it is described as “at the time of wide-angle photographing”. (B) above), the photographing apparatus 1 is hardly moved. Or it is often moved slowly. In this case, considering the extremely short time (one frame period in the present embodiment), it can be said that the photographing range is substantially fixed. Therefore, in this case as well, there is basically little change in the background, and there is generally little change in the captured image between the previous frame and the current frame. Similarly, when the difference value between the previous frame and the current frame is less than a predetermined value ((C) above), the shooting range is substantially fixed when considered in a very short time. It can be said that there is little change in the range and little change in the background.

  That is, this processing step S17 is executed when the photographing range is substantially fixed. In this way, when the shooting range is substantially fixed, it is not necessary to consider the movement of the shooting device 1, and therefore tracking is performed from the history (past shot images) stored in the storage area such as the buffer memory 118. It is easy to estimate the movement of the object. Therefore, in this processing step S17, the motion (motion vector) of the tracking target tracked in the previous frame is estimated. The tracking target motion vector is estimated by using a known estimation technique such as optical flow (gradient method, block matching method, etc.) or frequency analysis of a captured image.

  In this processing step S17, the position of the tracking target in the current frame is estimated based on the estimated motion vector, and the tracking target candidate closest to the estimated position is selected in processing step S12 (selection of tracking target candidate). The selected tracking target candidates are selected from the plurality of tracking target candidates, and the selected tracking target candidates are set as tracking targets. Note that there is no history when the image tracking process in FIG. 2 is executed for the first time. Therefore, for example, a tracking target candidate having the highest similarity to the tracking target template image is set as a tracking target from among a plurality of tracking target candidates selected in processing step S12 (selection of tracking target candidate), or shooting is performed. A tracking target candidate that appears in the center area of the range is set as a tracking target. When the tracking target is set in this processing step S17, the image tracking process in FIG.

[S18 in FIG. 2 (determination of change in predetermined area)]
This process step S18 is performed when it is determined in process step S16 (determination of captured image change) that the difference value between the captured image of the previous frame and the captured image of the current frame is greater than or equal to a predetermined value (S16: NO). In other words, it is executed when there is a large change in the captured image as a whole between the previous frame and the current frame. In this processing step S18, whether or not the difference value between the previous frame and the current frame is less than a predetermined value (a value different from the above) for an image in a predetermined area (the central area in the present embodiment) within the shooting range. Is determined.

[S19 in FIG. 2 (Tracking Target Setting)]
This processing step S19 is performed when it is determined in processing step S18 (predetermined change in the predetermined area) that the difference value between the previous frame and the current frame is less than the predetermined value for the image in the central area within the shooting range (S18). : YES). When the focal length of the photographic lens 106 is on the long focal end side (telephoto end side) (hereinafter, for convenience of explanation, it is referred to as “at the time of telephoto shooting”), a specific subject may be chased and panned or tilted. Many. When panning and tilting are performed well, typically, the tracking target continues to be positioned in the center area of the shooting range. Therefore, in this processing step S19, a tracking target candidate located in the central region of the shooting range is set as a tracking target. When the tracking target is set in this processing step S19, the image tracking process in FIG.

  In some cases, panning and tilting are performed while the tracking target is intentionally removed from the central region of the shooting range. Therefore, the predetermined area applied in this processing step S19 is not limited to the central area, but may be an area designated by the user. Also, during pan photography or tilt photography, images (backgrounds, etc.) other than the tracking target move relative to the photographing range. Therefore, the subject with the least movement in the entire captured image may be set as the tracking target.

[S20 in FIG. 2 (Setting of Tracking Object)]
This processing step S20 is performed when it is determined in processing step S18 (predetermined change in predetermined area) that the difference value between the previous frame and the current frame is greater than or equal to a predetermined value for the image in the central area within the shooting range (S18). : NO). For example, if a large amount of camera shake occurs during panning or tilting shooting while chasing a specific subject during telephoto, the position of the tracking target may change greatly. In this case, since the entire captured image including the tracking target changes greatly, it is difficult to estimate where the tracking target is located in the captured image. Therefore, in this processing step S20, the similarity between the tracking target template image and each subject image in the captured image is calculated, and the subject with the highest calculated similarity is set as the tracking target. When the tracking target is set in this processing step S20, the image tracking process in FIG.

  Next, four specific photographing examples will be described.

・ Shooting example 1 (Wide-angle shooting example)
FIG. 3 shows a display screen example of the LCD 124 during wide-angle shooting. In Figure 3, reference numeral _O A ~ O _C to each subject in the captured image. A subject surrounded by a frame is a tracking target candidate. More specifically, a subject surrounded by a thick frame is a tracking target. 3A shows a captured image of the previous frame, and FIG. 3B shows a captured image of the current frame.

In the first photographing example, wide-angle photographing is performed, and the photographing range is substantially fixed. Therefore, the tracking target does not appear at a position suddenly separated between frames. Thus, in the process step S17 (setting of the tracking target), the motion vector of the tracking target candidate O _A ~ O _C is estimated based on the history, the subject from the tracking target candidate O _A ~ O _C based on the estimated result O _A is set as a tracking target (see Figure 3 (b)).

・ Shooting example 2 (Telephoto shooting example)
FIG. 4 shows a display screen example of the LCD 124 during telephoto shooting. In Figure 4, reference numeral _O D ~ O _G to each subject in the captured image. A subject surrounded by a frame is a tracking target candidate. More specifically, a subject surrounded by a thick frame is a tracking target. 4A shows a captured image of the previous frame, and FIGS. 4B to 4D show captured images of the current frame in the shooting examples 2 to 4, respectively.

In this shooting example 2, telephoto shooting is performed, but the shooting device 1 is in a stationary state and the shooting range is fixed. Therefore, the tracking target does not appear at a position suddenly separated between frames. Thus, in the process step S17 (setting of the tracking target), based on the history is estimated motion vector of the tracking target candidate O _D ~ O _F is subject from the tracking target candidate O _D ~ O _F based on the estimation result O _D is set as a tracking target (see Figure 4 (b)).

・ Shooting example 3 (Telephoto shooting example)
In the present photographing example 3, the tracking target is pan-photographed at the time of telephoto. Therefore, in the process step S19 (setting of the tracking target) from the tracking target candidate O _D ~ O _F, subject O _D caught on (central region in this case) a predetermined area in the imaging range is set as a tracking target ( (Refer FIG.4 (c)).

・ Shooting example 4 (Telephoto shooting example)
In the fourth photographing example, the tracking target is pan-photographed at the time of telephoto, but the position is greatly changed due to the occurrence of camera shake or the like. Therefore, in the process step S20 (setting of the tracking target), the highest object O _D similarity to the tracking target template image is set as a tracking target (see FIG. 4 (d)).

  Thus, according to one embodiment of the present invention, the tracking target is set by a method suitable for the shooting state at that time, regardless of the similarity between the tracking target template image and the image of each subject. Therefore, even when there are many subjects similar to the original tracking target, erroneous detection of the tracking target can be avoided.

  The above is the description of the exemplary embodiments of the present invention. Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention. For example, the embodiment of the present invention also includes contents appropriately combined with embodiments or the like clearly shown in the specification or obvious embodiments.

DESCRIPTION OF SYMBOLS 1 Image pick-up apparatus 100 System controller 102 Operation part 104 Drive circuit 106 Shooting lens 108 Aperture 110 Shutter 112 Solid-state image sensor 114 Signal processing circuit 116 Image processing engine 118 Buffer memory 120 Card interface 122 LCD control circuit 124 LCD
126 ROM
128 Gyro sensor 130 Acceleration sensor 132 Geomagnetic sensor 134 GPS sensor 200 Memory card

Claims (6)

A selection means for selecting a plurality of tracking target candidates from the captured image;
A setting means for setting a tracking target based on a shooting state among a plurality of selected tracking target candidates;
Comprising
Image tracking device. The setting means includes
(1) Whether or not a photographing device including the image tracking device is in a stationary state,
(2) The focal length of the photographing lens provided in the photographing apparatus is the short focal end side or the long focal end side,
Setting the tracking target based on at least one of
The image tracking device according to claim 1. Motion estimation means for estimating the motion of the subject in the captured image,
The setting means includes
(A) When the imaging device is stationary,
(B) When the focal length of the taking lens is on the short focal end side,
(C) When the difference value between the previous captured image and the current captured image is less than the first threshold value,
If at least one of the following is satisfied, the tracking target is set based on the estimation result of the motion of the subject by the motion estimation means from the plurality of tracking target candidates.
The image tracking device according to claim 1. The setting means includes
When comparing the previous captured image with the current captured image, if the difference value of the entire captured image is greater than or equal to the second threshold and the difference value in a predetermined area within the captured image is less than the third threshold, A tracking target candidate that appears in the predetermined area is set as the tracking target.
The image tracking device according to any one of claims 1 to 3. Storage means for storing a reference image representing the tracking target;
Similarity calculation means for calculating the similarity between the stored reference image and the image of each subject in the captured image;
With
The setting means includes
When the difference value of the entire photographed image is equal to or greater than a second threshold value and the difference value in a predetermined area in the photographed image is equal to or greater than a third threshold value, the similarity is calculated based on the calculation result by the similarity calculation means. The subject with the highest is set as the tracking target.
The image tracking device according to claim 4. Selecting a plurality of tracking target candidates from the captured image;
Setting a tracking target based on a predetermined shooting state from among a plurality of selected tracking target candidates;
including,
Image tracking method.

Images