JP2013157725A - Tracking device and tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a tracking device that can track a tracking object even when there is a similar color at the periphery of the tracking object.SOLUTION: An image acquisition unit 13061 acquires an image including a tracking object which is a tracking target from, for example, a RAM 118 described later. A tracking region setting unit 13062 sets a tracking region to the tracking object at an image of (N-1)th frame. A color acquisition position determination unit 13063 determines a plurality of color acquisition positions on the basis of the position of the tracking region. A similar color detection unit 13064 compares a tracking color which is a color of the tracking region at the (N-1)th frame with each color of the color acquisition positions at the Nth frame, and selects a color acquisition position about which the color is the most similar as the most similar color position. A tracking region update unit 13065 has the tracking region setting unit 13062 set the position based on the most similar color position as the tracking region at the Nth frame. This is repeated and tracking of the tracking object is performed.

Description

  The present invention relates to a tracking device and a tracking method for tracking a subject.

  In general, a technique for performing automatic focusing control (AF) or automatic exposure control (AE) so as to follow a specific subject when shooting a moving body or shooting a moving image is known. Tracking processing is used to follow such a specific subject. There are various types of tracking processes, such as those using luminance information, those using color information, and those using face detection. For example, Patent Document 1 discloses a technique for tracking a subject based on the color information of the subject. For example, when tracking a subject based on color information, a color similar to the color of an area set as a tracking target is searched for in the image.

Japanese Patent No. 2605004

  When tracking a tracking subject that is a tracking target based on color information, if a non-tracking subject having a color similar to the tracking subject appears around the tracking subject, the tracking target shifts from the tracking subject to a non-tracking subject, Tracking of a non-tracking subject is started, and the tracking subject may be lost.

  Accordingly, an object of the present invention is to provide a tracking device and a tracking method capable of continuing tracking of a tracking subject even when a subject having a color similar to the color of the tracking subject exists around the tracking subject.

  In order to achieve the object, according to one aspect of the present invention, a tracking device sets an image acquisition unit that acquires an image, a partial area of the image as a tracking area, and tracks the color of the tracking area. A tracking region setting unit for setting a color, a comparison region setting unit for setting at least three regions having a predetermined positional relationship in the image as comparison regions, and a color most similar to the tracking color among the comparison regions A similar color detection unit that selects the comparison region as an update region, and a tracking region update unit that causes the tracking region setting unit to perform tracking region update that resets the tracking region based on the update region. It is characterized by.

  To achieve the above object, according to another aspect of the present invention, the tracking device includes an image acquisition unit that acquires an image, and a template storage unit that stores a predetermined positional relationship of at least three comparison regions as a point template. A tracking area setting unit that sets a part of the image as a tracking area and sets a color of the tracking area as a tracking color, and the position of the point template based on the position of the tracking area in the image. Based on the update region, a similar color detection unit that selects, as an update region, the comparison region having the color most similar to the tracking color among the comparison regions of the set point template And a tracking area updating unit that causes the tracking area setting unit to execute tracking area updating for resetting the tracking area.

  In order to achieve the above object, according to one aspect of the present invention, a tracking method includes acquiring an image, setting a partial area of the image as a tracking area, and setting the color of the tracking area to a tracking color. Setting, setting at least three regions having a predetermined positional relationship in the image as comparison regions, and selecting the comparison region having the color most similar to the tracking color as the update region among the comparison regions And resetting the tracking area based on the update area.

  To achieve the above object, according to another aspect of the present invention, a tracking method includes acquiring an image, setting a part of the image as a tracking region, and tracking the color of the tracking region. Setting a color, setting a position of a point template having a predetermined positional relationship among at least three comparison areas based on the position of the tracking area in the image, and setting the position of the comparison area of the point template. The comparison area having the color most similar to the tracking color is selected as an update area, and the tracking area is reset based on the update area.

  According to the present invention, it is possible to provide a tracking device and a tracking method capable of continuing tracking of a tracking subject even if a subject having a color similar to the color of the tracking subject exists around the tracking subject.

The figure which shows an example of a structure of the imaging device provided with the tracking apparatus which concerns on one Embodiment of this invention. The figure for demonstrating a face detection process. 6 is a flowchart illustrating an example of a shooting operation of the imaging apparatus according to the embodiment. The flowchart which shows an example of the tracking process as a tracking method which concerns on 1st Embodiment. 5 is a flowchart illustrating an example of color information acquisition position determination processing according to the first embodiment. FIG. 5 is a diagram for explaining an example of a color information acquisition position obtained by a color information acquisition position determination process according to the first embodiment. 6 is a flowchart illustrating an example of the most similar color determination process according to the first embodiment. The figure for demonstrating acquisition of the most similar color by the most similar color judgment process which concerns on 1st Embodiment. The figure for demonstrating an example of the conventional color tracking technique. The figure for demonstrating the inconvenience which may arise in the conventional color tracking technique. FIG. 5 is a diagram for explaining an example of a color information acquisition position obtained by a color information acquisition position determination process according to the first embodiment. The flowchart which shows an example of the tracking process which concerns on 2nd Embodiment. The flowchart which shows an example of the tracking process which concerns on 3rd Embodiment. 10 is a flowchart illustrating an example of color information acquisition position determination processing according to the fourth embodiment. The figure for demonstrating an example of the color information acquisition position obtained by the color information acquisition position judgment process which concerns on 4th Embodiment. 10 is a flowchart illustrating an example of the most similar color determination process according to the fifth embodiment.

[First Embodiment]
A first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a configuration of an imaging apparatus including a tracking device according to the present embodiment. An imaging apparatus 100 illustrated in FIG. 1 includes a photographing optical system 102, a focus adjustment mechanism 104, a diaphragm 106, a diaphragm driving mechanism 108, a shutter 110, a shutter driving mechanism 112, an imaging element 114, and an imaging element interface ( IF) circuit 116, RAM 118, display element 120, display element drive circuit 122, touch panel 124, touch panel drive circuit 126, recording medium 128, system controller 130, operation unit 132, and ROM 134. doing.

  The photographing optical system 102 is an optical system for condensing a light flux F from a subject (not shown) on the light receiving surface of the image sensor 114. The photographing optical system 102 has a plurality of lenses such as a focus lens. Further, the photographing optical system 102 may have a zoom mechanism. The focus adjustment mechanism 104 includes a motor and its drive circuit. The focus adjustment mechanism 104 drives the focus lens in the photographing optical system 102 in the optical axis direction (the one-dot chain line direction in the drawing) under the control of the CPU 1301 in the system controller 130.

  The diaphragm 106 is configured to be openable and closable, and adjusts the amount of the light beam F incident on the image sensor 114 via the photographing optical system 102. The aperture drive mechanism 108 has a drive mechanism for driving the aperture 106. The diaphragm driving mechanism 108 drives the diaphragm 106 according to the control of the CPU 1301 in the system controller 130.

  The shutter 110 is configured to place the light receiving surface of the image sensor 114 in a light shielding state or an exposure state. The exposure time of the image sensor 114 is adjusted by the shutter 110. The shutter drive mechanism 112 has a drive mechanism for driving the shutter 110, and drives the shutter 110 under the control of the CPU 1301 in the system controller 130.

  The imaging element 114 has a light receiving surface on which a light flux F from a subject condensed through the photographing optical system 102 is imaged. The light receiving surface of the image sensor 114 is configured by two-dimensionally arranging a plurality of pixels, and a color filter is provided on the light incident side of the light receiving surface. Such an image sensor 114 converts an image (subject image) corresponding to the light beam F formed on the light receiving surface into an electrical signal (hereinafter referred to as an image signal) corresponding to the light amount. Here, as the imaging element 114, imaging elements having various configurations such as a CCD system and a CMOS system are known. Various arrangements such as a Bayer arrangement are known as the color arrangement of the color filter. In the present embodiment, the configuration of the image sensor 114 is not limited to a specific configuration. Therefore, image pickup devices with various configurations can be used.

  The image sensor IF circuit 116 drives the image sensor 114 under the control of the CPU 1301 in the system controller 130. The image sensor IF circuit 116 reads the image signal obtained by the image sensor 114 according to the control of the CPU 1301 in the system controller 130, and performs CDS (correlated double sampling) processing or AGC (automatic operation) on the read image signal. Analog processing such as gain control processing is performed. Further, the image sensor IF circuit 116 converts the analog processed image signal into a digital signal (hereinafter referred to as image data).

  The display element 120 is a liquid crystal display (LCD), for example, and displays various images such as an image for live view and an image recorded on the recording medium 128. The display element driving circuit 122 drives the display element 120 based on the image data input from the CPU 1301 of the system controller 130 and causes the display element 120 to display an image.

  The touch panel 124 is integrally formed on the display screen of the display element 120, and detects a contact position of a user's finger or the like on the display screen. The touch panel drive circuit 126 drives the touch panel 124 and outputs a contact detection signal from the touch panel 124 to the CPU 1301 of the system controller 130. The CPU 1301 detects a contact operation on the display screen of the user from the contact detection signal, and executes processing according to the contact operation.

  The recording medium 128 is a memory card, for example, and records an image file obtained by a shooting operation. The image file is a file configured by adding a predetermined header to image data. In the header, data indicating shooting conditions, data indicating a tracking position, and the like are recorded as tag data.

  The system controller 130 is a control circuit for controlling the operation of the imaging apparatus 100, and includes a CPU 1301, an AF control circuit 1302, an AE control circuit 1303, an image processing circuit 1304, a focal length acquisition circuit 1305, and a tracking processing circuit. 1306, a face detection circuit 1307, and a memory control circuit 1308.

  The CPU 1301 includes blocks outside the system controller 130 such as the focus adjustment mechanism 104, the aperture driving mechanism 108, the shutter driving mechanism 112, the display element driving circuit 122, and the touch panel driving circuit 126, and each control circuit inside the system controller 130. Control the behavior.

  The AF control circuit 1302 controls contrast AF processing. Specifically, the AF control circuit 1302 extracts the high frequency component of the image data obtained by the image sensor IF circuit 116, and acquires the focus evaluation value for AF by integrating the extracted high frequency component. . The CPU 1301 controls the focus adjustment mechanism 104 to bring the focus lens into focus while evaluating the contrast of the image data according to the focus evaluation value.

  The AE control circuit 1303 controls the AE operation. Specifically, the AE control circuit 1303 calculates subject luminance using the image data obtained by the image sensor IF circuit 116. The CPU 1301 calculates the aperture amount (aperture value) of the aperture 106 during exposure, the opening time of the shutter 110 (shutter speed value), the image sensor sensitivity, the ISO sensitivity, and the like according to the subject brightness.

  The image processing circuit 1304 performs various image processing on the image data. This image processing includes color correction processing, gamma (γ) correction processing, compression processing, and the like. The image processing circuit 1304 also performs decompression processing on the compressed image data.

  The focal length acquisition circuit 1305 is provided when the photographing optical system 102 has a zoom mechanism. A focal length acquisition circuit 1305 acquires the focal length from the photographing optical system 102. The tracking processing circuit 1306 executes tracking processing according to the present embodiment described in detail later. The tracking processing circuit 1306 includes an image acquisition unit 13061, a tracking region setting unit 13062, a color acquisition position determination unit 13063, a similar color detection unit 13064, a tracking region update unit 13065, a reliability determination unit 13066, and a tracking color. And a center-of-gravity search unit 13067. The image acquisition unit 13061 captures, for example, an image including a tracking subject that is a tracking target from the RAM 118 described later into the tracking processing circuit 1306. The tracking area setting unit 13062 sets a tracking area for the tracking subject in the image of the (N−1) th frame. The color acquisition position determination unit 13063 determines a plurality of color acquisition positions based on the position of the tracking area. The similar color detection unit 13064 compares the tracking color, which is the color of the tracking area in the (N-1) -th frame image, with each color at the color acquisition position in the N-th frame image, and has the most similar color. The color acquisition position is selected as the most similar color position. The tracking area update unit 13065 causes the tracking area setting unit 13062 to reset the position based on the most similar color position to the tracking area in the Nth frame image. The reliability determination unit 13066 determines the reliability of the tracking region update unit 13065 for updating the tracking region. This is repeated to track the tracking subject. The tracking color centroid search unit 13067 performs a tracking color centroid search, which is a conventional color tracking technique. That is, the tracking color centroid search unit 13067, for example, performs a raster scan in the search area determined based on the position of the tracking area of the previous frame, searches for an area having the same color information as the tracking color, and determines the centroid position. decide.

  The face detection circuit 1307 detects the face of the subject (person) in the image data. Here, the face detection process will be briefly described. In the face detection process, the correlation amount between the image data obtained in each frame and the face parts 402, 404, and 406 as shown in FIG. The face part 402 is image data corresponding to a shadow pattern around a person's nose, the face part 404 is image data corresponding to a shadow pattern around a person's eye, and the face part 406 is a person This is image data corresponding to the shadow pattern around the mouth. The amount of correlation between the image data and the face parts 402, 404, and 406 is maximized when a predetermined arrangement indicating a person's face is obtained as shown in FIG. At this time, it is assumed that a face exists in a region 408 including the face parts 402, 404, and 406. Note that the size of the face parts 402, 404, and 406 may be changed according to a preset search face size. Here, in FIG. 2B, the face area is a rectangular area, but it may be a circular area. The memory control circuit 1308 is an interface that performs control for the CPU 1301 and the like to access the RAM 118, the recording medium 128, and the ROM 134.

  The operation unit 132 is various operation members operated by the user. The operation unit 132 includes, for example, a release button, a moving image button, a mode dial, a selection key, a power button, and the like. The release button has a 1st release switch and a 2nd release switch. The 1st release switch is a switch that is turned on when the user presses the release button halfway. When the 1st release switch is turned on, a shooting preparation operation such as AF processing is performed. The 2nd release switch is a switch that is turned on when the user fully presses the release button. When the 2nd release switch is turned on, an exposure operation for still image shooting is performed. The moving image button is an operation member for instructing the start or end of moving image shooting. When the user presses the moving image button, the moving image shooting process is started. When the moving image button is pressed during the moving image shooting process, the moving image shooting process is terminated.

  The mode dial is an operation member for selecting shooting settings of the imaging apparatus. In the present embodiment, for example, a still image shooting mode and a moving image shooting mode can be selected as shooting settings of the imaging apparatus. The still image shooting mode is a shooting setting for shooting a still image. The moving image shooting mode is a shooting setting for shooting a moving image. The selection key is an operation member for selecting or determining an item on the menu screen, for example. When the selection key is operated by the user, an item is selected or determined on the menu screen. The power button is an operation member for turning on or off the power of the imaging apparatus. When the power button is operated by the user, the imaging apparatus 100 is activated and becomes operable. If the power button is operated while the imaging apparatus is activated, the imaging apparatus 100 enters a power saving standby state.

  The ROM 134 stores program codes for the CPU 1301 to execute various processes. In addition, the ROM 134 stores various control parameters such as control parameters necessary for the operation of the photographing optical system 102, the aperture 106, the image sensor 114, and the like, and control parameters necessary for image processing in the image processing circuit 1304. ing. Further, the ROM 134 also stores face part data used for face detection in the face detection circuit 1307, data for displaying a tracking frame, and the like.

  The RAM 118 is, for example, an SDRAM, and has a work area, a tracking image area, a tracking color log area, and a tracking data log area as storage areas. The work area is a storage area provided in the RAM 118 for temporarily storing data generated in each part of the imaging apparatus 100 such as image data obtained by the imaging element IF circuit 116. The tracking image area is a storage area provided in the RAM 118 for temporarily storing tracking image data. The tracking image data is data of an image including a tracking subject in the tracking process. This tracking image data is used in the tracking process. The tracking color log area is a storage area provided in the RAM 118 for temporarily storing the tracking color that is the color information of the tracking area in the tracking process. In the tracking process, the most similar color position is selected by comparing the tracking color stored in the tracking color log area with the color at the color acquisition position. The tracking data log area is a storage area provided in the RAM 118 for temporarily storing the tracking data log. The tracking data log is a log in which the tracking position obtained as a result of the tracking process is recorded.

  Next, the operation of the imaging apparatus according to the present embodiment will be described. A flowchart showing the photographing operation of the imaging apparatus 100 is shown in FIG. The CPU 1301 reads the necessary program code from the ROM 134 and controls the operation of FIG.

  In step S100, the CPU 1301 determines whether or not the current shooting setting of the imaging device 100 is the still image shooting mode. As described above, the shooting setting is set by the mode dial. When it is determined in step S100 that the shooting setting is the still image shooting mode, the CPU 1301 starts the live view operation in step S102. In this live view operation, the CPU 1301 controls the shutter driving mechanism 112 to open the shutter 110 and then controls the image sensor IF circuit 116 to cause the image sensor 114 to start imaging. Thereafter, the CPU 1301 inputs image data stored in the work area of the RAM 118 as a result of imaging by the imaging device 114 to the image processing circuit 1304. The image processing circuit 1304 performs image processing for live view display on the input image data. Subsequently, the CPU 1301 inputs image data that has been subjected to image processing for live view display to the display element driving circuit 122 and causes the display element 120 to display an image. By repeatedly executing such a display operation, an image of the subject is displayed on the display element 120 as a moving image. The user can observe the subject by this moving image display.

  The CPU 1301 determines whether or not the 1st release switch is turned on in step S104. The CPU 1301 continues the live view operation until it is determined in step S104 that the 1st release switch is turned on.

  If it is determined in step S104 that the first release switch has been turned on, the CPU 1301 performs a release AF process in step S106. In the release AF process, the CPU 1301 drives the focus lens to the in-focus position by scan driving. In scan drive, the CPU 1301 controls the focus adjustment mechanism 104 to drive the focus lens in one direction within a predetermined scan range, and causes the AF control circuit 1302 to sequentially calculate focus evaluation values. The CPU 1301 evaluates the focus based on the focus evaluation value. The CPU 1301 stops driving the focus lens at the lens position where the focus evaluation value is maximized. Such scan driving is performed when the difference between the position of the focus lens before AF and the in-focus position is large.

  In step S108, the CPU 1301 sets a tracking area and controls the display element driving circuit 122 to display a tracking frame on the display element 120. Here, for example, a subject focused by the release AF can be set as the tracking subject. Further, when a face is detected by the face detection circuit 1307, the face can be set as a tracking subject. Furthermore, when a subject displayed on the screen of the display element 120 is designated by the touch panel 124, the subject can be set as a tracking subject. The set tracking area in the tracking subject is referred to as a tracking area. This tracking area may be one pixel or a plurality of pixels. This embodiment is particularly effective when the tracking area is a relatively narrow area such as one pixel. In addition, the CPU 1301 searches for a rectangular area that includes color information whose difference from the color information of the tracking area is equal to or less than a predetermined threshold, and sets this area as the same color area.

  The CPU 1301 controls the display element driving circuit 122 to display a tracking frame representing the tracking area on the display element 120. The tracking frame may be a rectangular frame with the tracking area as the center of gravity or a frame representing the same color area. That is, the tracking frame is displayed at the tracking target position on the screen of the display element 120. The tracking area and the same color area are recorded in the tracking data log area of the RAM 118. The color information of the tracking area is recorded in the tracking color log area.

  In step S110, a tracking process is executed. This tracking process is mainly executed by the tracking processing circuit 1306. Details of this tracking process will be described later. In step S112, the CPU 1301 performs AF processing so as to focus on the subject at the tracking position, and performs AE processing so that exposure of the subject at the tracking position is appropriate. In the AF process after the tracking process, the CPU 1301 drives the focus lens to the in-focus position by scan driving or wobbling driving. In wobbling driving, the CPU 1301 determines whether or not the focus evaluation value calculated by the AF control circuit 1302 when the focus lens is driven has increased with respect to the focus evaluation value at the previous lens position. The CPU 1301 finely drives the focus lens in the same direction as the previous time when it is determined that the focus evaluation value has increased, and finely drives the focus lens in the opposite direction to the previous time when the focus evaluation value has decreased. . Such an operation is repeated at high speed to gradually drive the focus lens to the in-focus position. In the AE process, the CPU 1301 sets an aperture amount (aperture value) of the aperture 106 during the main exposure, in which the luminance of the subject at the tracking position calculated by the AE control circuit 1303 is set to a predetermined appropriate amount (appropriate exposure amount). The opening time (shutter speed value) of the shutter 110 is calculated.

  In step S114, the CPU 1301 determines whether the 2nd release switch is turned on. If it is determined that the 2nd release switch is not turned on, the process returns to step S110. Thus, the tracking process and the AF / AE process are continued until the 2nd release switch is turned on.

  If it is determined in step S114 that the 2nd release switch is turned on, the CPU 1301 controls the display element driving circuit 122 in step S116 to hide the tracking frame. In step S118, the CPU 1301 performs processing for recording still image data on the recording medium 128. That is, the CPU 1301 controls the shutter drive mechanism 112 to close the shutter 110 and controls the aperture drive mechanism 108 to narrow the aperture 106 to the previously calculated aperture value. Subsequently, the CPU 1301 controls the shutter driving mechanism 112 to cause the image sensor 114 to perform imaging (exposure) while opening the shutter 110 for the previously calculated opening time. Thereafter, the CPU 1301 causes the image processing circuit 1304 to process still image data obtained via the image sensor 114. The CPU 1301 adds a header to the still image data processed by the image processing circuit 1304 to generate a still image file, and records the generated still image file on the recording medium 128. In step S120, the CPU 1301 adds data indicating the tracking position obtained as a result of the tracking process in step S110 to the still image file previously recorded on the recording medium 128. Thereafter, the CPU 1301 ends the operation shown in FIG.

  When it is determined in step S100 that the shooting setting is the moving image shooting mode, the CPU 1301 starts the live view operation in step S122. The CPU 1301 determines whether or not the moving image button is turned on in step S124. In step S124, the CPU 1301 continues the live view operation until it is determined that the moving image button is turned on.

  If it is determined in step S124 that the moving image button has been turned on, the CPU 1301 sets a tracking area in step S126 and controls the display element driving circuit 122 to display a tracking frame on the display element 120. In step S128, the tracking process described in detail later is executed. In step S130, the CPU 1301 performs AF processing so as to focus on the subject at the tracking position, and performs AE processing so that exposure of the subject at the tracking position is appropriate. In the AF process in step S130, the focus lens is driven to the in-focus position by wobbling driving.

  In step S132, the CPU 1301 performs processing for recording moving image data on the recording medium 128. That is, the CPU 1301 controls the aperture driving mechanism 108 to reduce the aperture 106 to the aperture value calculated in the AE process, and performs imaging (exposure) on the image sensor 114 for a time corresponding to the shutter speed value calculated in the AE process. Let After the exposure is completed, the CPU 1301 generates a moving image file and records it on the recording medium 128. Further, the CPU 1301 processes the moving image data obtained via the image sensor 114 in the image processing circuit 1304, and records the moving image data processed in the image processing circuit 1304 in a moving image file.

  In step S134, the CPU 1301 simultaneously records data indicating the tracking position obtained as a result of the tracking process in step S128 in the moving image file previously recorded on the recording medium 128. In step S136, the CPU 1301 determines whether the moving image button has been turned off. If it is determined that the movie button is not turned off, the process returns to step S128. Thus, in the moving image shooting mode, the tracking process and the recording of moving image data are continued until the moving image button is turned off.

  If it is determined in step S136 that the moving image button has been turned off, the CPU 1301 controls the display element driving circuit 122 in step S138 to hide the tracking frame. Thereafter, the CPU 1301 ends the operation shown in FIG.

  The tracking process according to the present embodiment will be described with reference to FIG. In step S200, the CPU 1301 controls the imaging element IF circuit 116 to cause the imaging element 114 to perform imaging. In step S <b> 202, the CPU 1301 captures the data of the image captured by the image sensor 114 into the RAM 118 via the image sensor IF circuit 116. Image data captured in the RAM 118 is captured by the image acquisition unit 13061 in the tracking processing circuit 1306 into the tracking processing circuit 1306 as necessary.

  In step S204, the color acquisition position determination unit 13063 in the tracking processing circuit 1306 executes color information acquisition position determination processing. The color information acquisition position determination process will be described with reference to the flowchart shown in FIG. In the present embodiment, five color acquisition positions are set in the color information acquisition position determination process.

The tracking area setting unit 13062 determines a correction magnification in the X direction (correction magnification (X direction)) in step S300. The correction magnification (X direction) is calculated by the following equation (1).
Correction magnification (X direction) = same color area width (X direction) / first constant (1)
Here, the same color region width (X direction) represents the length of the same color region in the X direction. The same color area is a rectangular area obtained as a result of searching for an area having the same color information as the tracking area, as described in the description of step S108. The first constant is a predetermined constant.

In step S302, the color acquisition position determination unit 13063 determines a correction magnification in the Y direction (correction magnification (Y direction)). The correction magnification (Y direction) is calculated by the following equation (2).
Correction magnification (Y direction) = same color area width (Y direction) / second constant (2)
Here, the same color region width (Y direction) represents the length of the same color region in the Y direction. The second constant is a predetermined constant. The second constant may be the same value as the first constant or a different value.

In step S304, the color acquisition position determination unit 13063 determines an interval in the X direction (interval (X direction)) of a plurality of tracking positions to be set. Here, the interval (X direction) is calculated by the following equation (3).
Interval (X direction) = first reference interval × correction magnification (X direction) (3)
Here, the first reference interval is a predetermined interval.

In step S306, the color acquisition position determination unit 13063 determines an interval in the Y direction (interval (Y direction)) of a plurality of tracking positions to be set. Here, the interval (Y direction) is calculated by the following equation (4).
Interval (Y direction) = second reference interval × correction magnification (Y direction) (4)
Here, the second reference interval is a predetermined interval, and may be the same value as the first reference interval or a different value.

In step S308, the color acquisition position determination unit 13063 determines a color acquisition position (center). Here, the X coordinate x and the Y coordinate y of the color acquisition position (center) are obtained by the following equations (5) and (6).
x = tracking position (X direction) (5)
y = tracking position (Y direction) (6)
Here, the tracking position (X direction) is the X coordinate of the tracking position, which is the position of the center of gravity of the current tracking area. The tracking position (Y direction) is the Y coordinate of the current tracking position.

In step S310, the color acquisition position determination unit 13063 determines the color acquisition position (right). Here, the X coordinate x and the Y coordinate y of the color acquisition position (right) are obtained by the following equations (7) and (8).
x = tracking position (X direction) + interval (X direction) (7)
y = tracking position (Y direction) (8)
In step S312, the color acquisition position determination unit 13063 determines the color acquisition position (left). Here, the X coordinate x and Y coordinate y of the color acquisition position (left) are obtained by the following equations (9) and (10).
x = tracking position (X direction) −interval (X direction) (9)
y = tracking position (Y direction) (10)
In step S314, the color acquisition position determination unit 13063 determines the color acquisition position (upper). Here, the X coordinate x and Y coordinate y of the color acquisition position (upper) are obtained by the following equations (11) and (12).
x = tracking position (X direction) (11)
y = tracking position (Y direction) −interval (Y direction) (12)
In step S316, the color acquisition position determination unit 13063 determines the color acquisition position (bottom). Here, the X coordinate x and the Y coordinate y of the color acquisition position (bottom) are obtained by the following equations (13) and (14).
x = tracking position (X direction) (13)
y = tracking position (Y direction) + interval (Y direction) (14)
After step S316, the process returns the color acquisition position (center), the color acquisition position (right), the color acquisition position (left), the color acquisition position (upper), and the color acquisition position (lower) as return values, as shown in FIG. Return to the tracking process described with reference.

  According to the color information acquisition position determination process described above, a color acquisition position as shown in FIG. 6 is obtained. That is, in FIG. 6, the car is the tracking subject 310, and the tracking area 320 is set therein. A rectangular area having the same color as that of the tracking area 320 is set as the same color area 330. Here, the center of gravity position of the tracking area 320 is set to the color acquisition position (center) 350C by the color information acquisition position determination process. The color acquisition position (right) 350R is on the right side of the tracking area 320, the color acquisition position (left) 350L is on the left side of the tracking area 320, and the color acquisition position (up) 350T is on the upper side of the tracking area 320. A color acquisition position (lower) 350B is set on the lower side.

  Returning to FIG. 4, the description of the tracking process will be continued. In step S206, the tracking processing circuit 1306 determines the color acquisition position (center) 350C, the color acquisition position (right) 350R, the color acquisition position (left) 350L, and the color acquisition position (upper) determined in the color information acquisition position determination process. The color information at each position of 350T and the color acquisition position (bottom) 350B is acquired. In step S208, the similar color detection unit 13064 in the tracking processing circuit 1306 executes the most similar color determination process.

  The most similar color determination process will be described with reference to the flowchart shown in FIG. In step S400, the similar color detection unit 13064 selects a color acquisition position (center) 350C, a color acquisition position (right) 350R, a color acquisition position (left) 350L, a color acquisition position (upper) 350T, and a color acquisition position (lower) 350B. Among them, the position where the color information is most similar to the color information of the tracking color is selected. This position is called the most similar color position. As color information of each color acquisition position, color information of one pixel at each color acquisition position may be used, or color information of a plurality of pixels centering on each color acquisition position may be used.

  In step S402, the similar color detection unit 13064 determines whether the difference between the color information of the tracking color and the color information of the most similar color position is less than a predetermined threshold value. If it is less than the predetermined threshold value, the similar color detection unit 13064 sets information indicating that the most similar color position exists and information on the selected most similar color position in the return value in step S404, and the processing refers to FIG. Return to the processing explained. On the other hand, if it is determined in step S402 that the value is equal to or greater than the predetermined threshold value, the similar color detection unit 13064 sets information indicating that the most similar color position is not present in the return value in step S406, and the processing is illustrated in FIG. Return to the processing explained. The accuracy of tracking is improved by evaluating the difference between the color information of the tracking color and the color information of the most similar color position.

  For example, as shown in FIG. 8, consider a case where an automobile that is the tracking subject 310 is moving to the left. In this case, the color acquisition position (center) 350C, color acquisition position (right) 350R, color acquisition position (left) 350L, color acquisition position (upper) 350T, and color acquisition position (lower) 350B are similar to the tracking color. Is selected to be the color acquisition position (left) 350L. Here, if the difference between the color information of the tracking color and the color information of the color acquisition position (left) 350L is less than a predetermined threshold value, the information indicating that the most similar color position exists in the return value and the most similar color position are obtained. Information that the position (left) is 350L is set.

  Returning to FIG. 4, the description of the tracking process will be continued. The tracking area update unit 13065 in the tracking processing circuit 1306 determines whether or not there is a most similar color position in step S210. When it is determined that there is no most similar color position, the tracking area update unit 13065 does not update the tracking area in step S212. Thereafter, the process proceeds to step S216. On the other hand, if it is determined in step S210 that there is the most similar color position, the tracking area update unit 13065 has selected the tracking area by the tracking area setting unit 13062 in step S214 in the most similar color determination process. Update to the closest color position. The tracking area may be a point that is one pixel indicating the most similar color position, or may be an area having a width with the point being the center of gravity. Thereafter, the process proceeds to step S216.

  In step S216, the reliability determination unit 13066 in the tracking processing circuit 1306 determines the reliability of the tracking area. For example, the reliability determination unit 13066 determines the reliability from the saturation of the reference image data. Specifically, when the saturation and luminance of the tracking area are equal to or higher than a predetermined value, it is determined that the tracking area is reliable. These threshold values for determining reliability can be set as appropriate.

  The tracking processing circuit 1306 records the final tracking area in the tracking data log area of the RAM 118 in step S218. The tracking processing circuit 1306 also acquires the same color area to be used in the next tracking process and records it in the tracking data log area of the RAM 118. The tracking processing circuit 1306 records the color information of the tracking area in the tracking color log area. However, if it is determined in step S216 that the reliability is not reliable, the tracking position or the like may not be recorded. By using the reliability determination result, tracking accuracy is improved.

  In step S220, the CPU 1301 controls the display element driving circuit 122 to update the display position of the tracking frame to a position corresponding to the tracking position stored in step S218. That is, the tracking frame displayed on the display element 120 is updated, and the user can operate the imaging apparatus 100 while confirming the tracking position on the display element 120. Thereafter, the CPU 1301 ends the tracking process of FIG. 4, and the process returns to the process described with reference to FIG. 3.

  According to this embodiment, the tracking area in the next frame is updated to a position having the color most similar to the tracking area among the five color acquisition positions. In this way, the tracking area is updated based on the color similarity, and the tracking subject is tracked.

  Thus, for example, the image acquisition unit 13061 functions as an image acquisition unit that acquires an image. For example, the tracking area setting unit 13062 functions as a tracking area setting unit that sets a part of an image as a tracking area and sets the color of the tracking area as a tracking color. For example, the color acquisition position determination unit 13063 functions as a comparison region setting unit that sets at least three regions having a predetermined positional relationship in the image as comparison regions. For example, the similar color detection unit 13064 functions as a similar color detection unit that selects, as an update region, a comparison region having a color most similar to the tracking color among the comparison regions. For example, the tracking area update unit 13065 functions as a tracking area update unit that causes the tracking area setting unit to execute tracking area update for resetting the tracking area based on the update area. For example, the reliability determination unit 13066 functions as a reliability determination unit that determines the reliability of the update region determined by the similar color detection unit. For example, the focal length acquisition circuit 1305 functions as a focal length acquisition unit that acquires the focal length of the optical system used when capturing an image. For example, the display element 120 functions as a tracking area notification unit that displays the tracking area.

  A conventionally known color tracking technique and the color tracking technique according to the present embodiment will be compared. A conventional color tracking technique will be described with reference to FIG. As shown in FIG. 9A, it is assumed that a tracking area 420 is set on the tracking subject 410. Here, the color information in the tracking area 420 has been acquired. In the next frame, that is, the frame in which the tracking subject 410 has moved as shown in FIG. 9B, the search area 450 determined based on the position of the tracking area 420 of the previous frame is raster-scanned, for example. An area having the same color information as the tracking area 420 of the frame is searched. Then, as shown in FIG. 9C, for example, the center of gravity area of the area having the same color information is updated as a new tracking area 420.

  If the conventional color tracking technique as described with reference to FIG. 9 is used, the following problems may occur. For example, it is assumed that the tracking area 420 in the tracking subject 410 has been tracked as desired from FIGS. 10 (a) to 10 (b). Here, as shown in FIG. 10C, when an object 480 having a color similar to that of the tracking area 420 approaches the tracking subject and enters the search area 450, the tracking area 420 is shown in FIG. The object 480 may be updated. In this case, there is a possibility that the tracking subject 410 will be lost and the object 480 will be tracked thereafter.

  On the other hand, in the present embodiment, a color acquisition position having a color most similar to the tracking color is selected. Therefore, it can be said that color resolution is very high in the color tracking according to the present embodiment. For this reason, the possibility that the tracking subject is lost is extremely low as compared with the prior art. In this embodiment, the tracking area is displaced within the tracking subject, but since the color acquisition position is appropriately determined according to the size of the same color area in the color information acquisition position determination process, the tracking area is within the tracking subject. It is controlled to fit. For example, as shown in FIG. 11, when the car that is the tracking subject 310 occupies a larger range of the image than in the case shown in FIG. At this time, the interval between the color acquisition positions 350 is wider than that shown in FIG. As a result, the tracking subject 410 is appropriately tracked.

  According to the present embodiment, the calculation performed by the CPU 1301 is a calculation for selecting the most similar color position by comparing the tracking color with the colors at several color acquisition positions, and has been described with reference to FIG. Unlike the technology, it does not require a search operation by scanning. Therefore, in the present embodiment, the amount of calculation performed is significantly less than in the prior art. Therefore, according to the present embodiment, high-speed processing can be realized. Furthermore, this speeding up reduces the amount of movement of the tracking subject between frames. This improves the tracking accuracy in the color tracking principle according to the present embodiment.

  In this embodiment, five color acquisition positions are used, but any number of color acquisition positions may be used as long as there are three or more points forming a surface. In addition, one point in the color acquisition position does not necessarily coincide with the original tracking area. However, if five points as in the present embodiment are set, the tracking subject may be tracked in any manner and may be tracked.

In the color information acquisition position determination process, the focal length of the photographing optical system 102 may be taken into consideration. For example, the CPU 1301 calculates a focal length coefficient based on the focal length acquired by the focal length acquisition circuit 1305. The CPU 1301 considers this focal length coefficient as in the following formulas (15) and (16), for example, in the calculation of the interval in step S304 and step S306 of the color information acquisition position determination process described with reference to FIG. Can do.
Interval (X direction) = first reference interval × correction magnification (X direction) × focal length coefficient (15)
Interval (Y direction) = second reference interval × correction magnification (Y direction) × focal length coefficient (16)
In this case, the correction magnification may not be used. However, the accuracy is higher when the correction magnification and the focal length coefficient are taken into consideration as in the above equations (15) and (16).

  By considering the focal length coefficient in this way, the interval is increased when the focal length is long, and the interval is decreased when the focal length is short. As a result, similar to the explanation given with reference to FIG. 11, for example, when the zoomed-up subject is occupied by a tracking subject occupying a large range in the image, the tracking subject is zoomed out and the tracking subject occupies a small range in the image. Compared with the case where the color acquisition position is, the interval between the color acquisition positions becomes large. As a result, the accuracy of color tracking according to the present embodiment is improved. Note that the number, area, arrangement, and the like of the color acquisition additional positions may be changed according to the focal length.

  In the present embodiment, the color acquisition position is calculated in the color information acquisition position determination process described with reference to FIG. 5, but information indicating the positional relationship of the color acquisition positions is stored in advance in the ROM 134 as a point template. The color acquisition position determination unit 13063 may be configured to determine the arrangement of the template in the color information acquisition position determination process. Also, a plurality of templates having different positional relationships of color acquisition positions are prepared, and the template is selected based on conditions such as the same color area in the color information acquisition position determination process, and the arrangement of the templates is determined. May be. Further, the template may be selected according to the focal length of the photographing optical system 102. By using a template, the amount of processing is reduced and high-speed processing is realized. When a template is used, a similarity ratio, a color distribution pattern, and the like can be used in addition to the most similar color.

  Thus, for example, the ROM 134 functions as a template storage unit that stores a predetermined positional relationship of at least three comparison areas as a point template. For example, the color acquisition position determination unit 13063 is based on the position of the tracking area in the image. It functions as a template setting unit for setting the position of the point template.

[Second Embodiment]
A second embodiment will be described. Here, differences from the first embodiment will be described, and the same portions will be denoted by the same reference numerals and description thereof will be omitted. In the present embodiment, when it is determined that there is no most similar color position in the first embodiment, that is, for example, when the tracking area is out of the tracking subject in the color tracking according to the first embodiment, tracking is performed by scanning. Conventional techniques for searching for regions are used.

  FIG. 12 is a flowchart showing the tracking process according to the present embodiment. As shown in this figure, steps S500 to S508 are the same as steps S200 to S208 of the first embodiment described with reference to FIG. That is, the CPU 1301 controls the imaging element IF circuit 116 in step S500 to cause the imaging element 114 to perform imaging, and the image data captured by the imaging element 114 in step S502 is stored in the RAM 118 via the imaging element IF circuit 116. take in. The tracking processing circuit 1306 executes color information acquisition position determination processing in step S504, acquires color information at each position of the color acquisition positions determined in the color information acquisition position determination processing in step S506, and in step S508. The most similar color determination process is executed.

  Subsequently, the tracking area update unit 13065 in the tracking processing circuit 1306 determines whether or not there is a most similar color position in step S510. When it is determined that there is the most similar color position, the tracking area update unit 13065 determines the most similar color position as the tracking area in step S512. Thereafter, the process proceeds to step S522. On the other hand, if it is determined in step S510 that there is no most similar color position, the tracking color centroid search unit 13067 in the tracking processing circuit 1306 performs a tracking color centroid search in step S514. Here, the tracking color centroid search is color tracking using the prior art described with reference to FIG. 9, for example. Thus, for example, the tracking color centroid search unit 13067 functions as a similar region search unit that scans an image to search a similar region similar to the tracking color and determines the centroid position of the similar region.

  The tracking area update unit 13065 determines whether or not there is a tracking color centroid in step S516. When it is determined that there is a tracking color centroid, the tracking area update unit 13065 determines a tracking area based on the color centroid in step S518. Thereafter, the process proceeds to step S522. When it is determined in step S516 that there is no tracking color centroid, the tracking area update unit 13065 does not update the tracking area in step S520, and the process proceeds to step S522.

  Steps S522 to S5226 are the same as steps 216 to 220 of the first embodiment described with reference to FIG. That is, the reliability determination unit 13066 determines the reliability of the tracking area in step S522. In step S524, the tracking processing circuit 1306 records the final tracking area in the tracking data log area of the RAM 118. In the next tracking process, the same color area to be used is also acquired and recorded in the tracking data log area of the RAM 118. In step S526, the CPU 1301 controls the display element driving circuit 122 to update the display position of the tracking frame to a position corresponding to the tracking position stored in step S524. Thereafter, the CPU 1301 ends the tracking process of FIG. 12, and the process returns to the process described with reference to FIG. Other processes are the same as those in the first embodiment.

  According to the present embodiment, even when the tracking subject is lost in the first embodiment, the tracking subject can be found by searching a wide area using the conventional technique. According to the present embodiment, the tracking subject can be tracked with higher accuracy than in the first embodiment.

[Third Embodiment]
A third embodiment will be described. Here, differences from the second embodiment will be described, and the same portions will be denoted by the same reference numerals and description thereof will be omitted. In this embodiment, the frame rate is increased when it is determined that the most similar color position is present in the second embodiment, the frame rate is decreased when it is determined that there is no most similar color, and the tracking color centroid is used. .

  FIG. 13 is a flowchart illustrating an example of the tracking process according to the present embodiment. The processing from step S600 to step S612 is the same as the processing from step S500 to step S512 according to the second embodiment described with reference to FIG. That is, the CPU 1301 controls the imaging element IF circuit 116 in step S600 to cause the imaging element 114 to perform imaging, and the image data captured by the imaging element 114 in step S602 is stored in the RAM 118 via the imaging element IF circuit 116. take in. The tracking processing circuit 1306 executes a color information acquisition position determination process in step S604, acquires color information at each of the color acquisition positions determined in the color information acquisition position determination process in step S606, and in step S608, acquires the color information. A similar color determination process is executed. The tracking processing circuit 1306 determines whether or not there is a most similar color position in step S610. If it is determined that there is the most similar color position, the tracking processing circuit 1306 determines the most similar color position as the tracking area in step S612.

  Thereafter, the tracking processing circuit 1306 outputs an instruction to the CPU 1301 to change the frame rate to a high frame rate in step S614. Here, the high frame rate is 60 to 120 fps, for example. Thereafter, the process proceeds to step S626.

  Further, the processing from step S616 to step S620 is the same as the processing from step S514 to step S518 according to the second embodiment described with reference to FIG. That is, if it is determined in step S610 that there is no most similar color position, the tracking processing circuit 1306 performs a tracking color centroid search in step S616, and determines in step S618 whether there is a tracking color centroid. If it is determined that there is a tracking color centroid, the tracking processing circuit 1306 determines the color centroid as the tracking area in step S620.

  Thereafter, the tracking processing circuit 1306 outputs an instruction to the CPU 1301 to change the frame rate to a low frame rate in step S622. Here, the low frame rate is, for example, 15 to 30 fps. Thereafter, the process proceeds to step S626.

  Further, the processing from step S624 to step S630 is the same as the processing from step S520 to step S526 according to the second embodiment described with reference to FIG. That is, if it is determined in step S618 that there is no tracking color centroid, the tracking processing circuit 1306 does not update the tracking area in step S5624, and the process proceeds to step S626. The tracking processing circuit 1306 determines the reliability of the tracking area in step S626. In step S628, the tracking processing circuit 1306 records the final tracking area in the tracking log area of the RAM 118. In the next tracking process, the same color area to be used is also acquired and recorded in the tracking log area of the RAM 118. In step S630, the CPU 1301 controls the display element driving circuit 122 to update the display position of the tracking frame to a position corresponding to the tracking position stored in step S628. Thereafter, the CPU 1301 ends the tracking process of FIG. 13, and the process returns to the process described with reference to FIG. 3. Other processes are the same as those in the second embodiment.

  As described above, the processing speed of the technique according to the present invention using the most similar color position is very fast compared to the conventional technique that performs the tracking color gravity center search. Therefore, in this embodiment, the frame rate is changed in accordance with this processing speed. According to the present embodiment, a necessary and sufficient time for processing is ensured, and the processing speed can be increased as a whole. By executing the determination of the tracking area based on the most similar color position at a high frame rate, an image difference between frames is reduced, so that the accuracy of color tracking based on the most similar color position is improved.

  Of course, the above 60 to 120 fps, 15 to 30 fps, and the like are examples. These frame rates are the frame rate at which the display element 120 displays an image, the frame rate at which an image is recorded on the recording medium 128, or the auto focus, auto exposure, or auto white balance frame used when shooting the image. It may be changed according to the rate. By changing according to the frame rate such as autofocus, a necessary tracking frame rate is ensured.

[Fourth Embodiment]
A fourth embodiment will be described. Here, differences from the first embodiment will be described, and the same portions will be denoted by the same reference numerals and description thereof will be omitted. In the present embodiment, the color information acquisition position determination process in the first embodiment is different.

  FIG. 14 is a flowchart illustrating an example of the color information acquisition position determination process according to the present embodiment. As shown in this figure, steps S701 to S716 are the same as steps S300 to S316 in the first embodiment described with reference to FIG. That is, the color acquisition position determination unit 13063 calculates a correction magnification (X direction) in step S700, calculates a correction magnification (Y direction) in step S702, calculates an interval (X direction) in step S704, and performs step S706. The interval (Y direction) is calculated at. Next, the color acquisition position determination unit 13063 calculates a color acquisition position (center) in step S708, calculates a color acquisition position (right) in step S710, calculates a color acquisition position (left) in step S712, In S714, the color acquisition position (upper) is calculated, and in step S716, the color acquisition position (lower) is calculated.

  In step S718, the color acquisition position determination unit 13063 calculates an integrated value (X direction). Here, the integrated value (X direction) is an integrated value in the X direction of the motion vector of the tracking area in a plurality of past frames. Here, the number of frames is a predetermined value. In step S720, the color acquisition position determination unit 13063 determines whether the integrated value (X direction) is greater than a predetermined threshold value. If it is determined that the integrated value (X direction) is less than or equal to the predetermined threshold, the process proceeds to step S726.

If it is determined in step S720 that the color acquisition position determination unit 13063 is greater than the predetermined threshold, the color acquisition position determination unit 13063 determines the X coordinate x and Y coordinate y of the color acquisition additional position (right) in steps S722 and (17) and ( 18).
x = tracking position (X direction) + interval (X direction) × 2 (17)
y = tracking position (Y direction) (18)
Subsequently, the color acquisition position determination unit 13063 determines the X coordinate x and the Y coordinate y of the color acquisition additional position (left) based on the following equations (19) and (20) in step S724.
x = tracking position (X direction) −interval (X direction) × 2 (19)
y = tracking position (Y direction) (20)

  In step S726, the color acquisition position determination unit 13063 calculates an integrated value (Y direction). Here, the integrated value (Y direction) is an integrated value in the Y direction of the motion vector of the tracking area in a plurality of past frames. Here, the number of frames is a predetermined value. In step S720, the CPU 1301 determines whether the integrated value (Y direction) is larger than a predetermined threshold value. If it is determined that the integrated value (Y direction) is equal to or less than the predetermined threshold, the process returns to the tracking process described with reference to FIG.

If it is determined in step S728 that it is greater than the predetermined threshold value, the color acquisition position determination unit 13063 sets the X coordinate x and Y coordinate y of the color acquisition additional position (upper) in steps S730 and (21) and ( 22).
x = tracking position (X direction) (21)
y = tracking position (Y direction) −interval (Y direction) × 2 (22)
Subsequently, the color acquisition position determination unit 13063 determines the X coordinate x and the Y coordinate y of the color acquisition additional position (lower) based on the following formulas (23) and (24) in step S732.
x = tracking position (X direction) (23)
y = tracking position (Y direction) + interval (X direction) × 2 (24)
Thereafter, the processing returns to the tracking processing described with reference to FIG.

  Return values when returning to the tracking process include color acquisition position (center), color acquisition position (right), color acquisition position (left), color acquisition position (upper), color acquisition position (lower), and added Color acquisition additional position (right) and color acquisition additional position (left), and / or color acquisition additional position (upper) and color acquisition additional position (lower). Other processes are the same as those in the first embodiment.

  In the present embodiment, when it is determined in step S720 that it is larger than the predetermined threshold, that is, when the speed in the X direction of the tracking area is larger than a predetermined value, the color acquisition additional position is set to the left and right. Similarly, when it is determined in step S728 that the speed is larger than the predetermined threshold, that is, when the speed of the tracking area in the Y direction is higher than a predetermined value, the color acquisition additional position is set up and down. That is, for example, as shown in FIG. 15, when the tracking subject 310 is moving to the left faster than a predetermined speed, the color acquisition position (center) 350C, the color acquisition position (right) 350R, the color acquisition position (left) 350L, In addition to the color acquisition position (upper) 350T and the color acquisition position (lower) 350B, a color acquisition additional position (left) 360L and a color acquisition additional position (right) not shown because they are out of the image area are set. The Therefore, in the present embodiment, in the most similar color determination process described with reference to FIG. 7, the position having the color most similar to the tracking color among these seven points is selected as the most similar position.

  According to the present embodiment, when the speed of the tracking subject is high, it is possible to prevent the tracking subject from being out of the color acquisition position and consequently not being tracked.

  In the present embodiment, one color acquisition additional position is added vertically and / or left and right, but a plurality of color acquisition additional positions may be added according to the speed of the tracking subject. Further, the color acquisition additional position may be added only in the moving direction according to the moving direction of the tracking subject. Further, the color acquisition position interval may be changed according to the speed of the tracking subject. Further, the area of the region used for color comparison may be changed according to the speed of the tracking subject.

Further, the arrangement of the color acquisition positions based on the motion vector of the tracking area may be arranged diagonally instead of vertically and horizontally. For example, the rotation angle θ is determined by the following equation (25).
θ = tan ⁻¹ (integrated value (Y direction) / integrated value (X direction)) (25)
Then, using the rotation angle θ, the X-coordinate x ′ and the Y-coordinate y ′ of the color acquisition position can be obtained by the following equations (26) and (27), respectively.
x ′ = x × cos θ−y × sin θ (26)
y ′ = x × sin θ + y × cos θ (27)
Here, x and y are values obtained by the above formulas (7) to (14), respectively.

  As described above, by arranging the color acquisition position obliquely in accordance with the moving direction of the tracking subject, it is possible to prevent the tracking subject from being out of the color acquisition position and as a result, the tracking subject is not tracked. Needless to say, the fourth embodiment and the modifications described above may be used in combination with the second embodiment or the third embodiment.

  When a point template is used for determining the color acquisition position, various point templates may be prepared in advance, and an appropriate point template may be selected from them according to the moving speed of the tracking subject. .

[Fifth Embodiment]
A fifth embodiment will be described. Here, differences from the first embodiment will be described, and the same portions will be denoted by the same reference numerals and description thereof will be omitted. This embodiment is different from the first embodiment in the most similar determination process described with reference to FIG. The most similar color determination process according to the present embodiment will be described with reference to the flowchart shown in FIG.

  In step S800, the similar color detection unit 13064 selects the color from among the color acquisition position (center), the color acquisition position (right), the color acquisition position (left), the color acquisition position (upper), and the color acquisition position (lower). Selects the most similar color position that is most similar to the tracking color. In step S802, the similar color detection unit 13064 determines whether the difference between the color information of the tracking color and the color information of the most similar color position is less than a predetermined threshold.

  If it is determined in step S802 that the color is less than the predetermined threshold, the similar color detection unit 13064 determines that the difference between the color at the color acquisition position (center) and the color at the most similar position is greater than the predetermined threshold in step S804. Determine whether it is larger. If it is determined in this determination that it is greater than the predetermined threshold value, the similar color detection unit 13064 sets information indicating that the most similar color position is present and information on the most similar color position in the return value in step S806, and the processing is as shown in FIG. Returning to the tracking process described with reference to FIG.

  On the other hand, if it is determined in step S804 that the color value is equal to or less than the predetermined threshold value, the similar color detection unit 13064 has a difference between the color information of the tracking color and the color information of the most similar color position is less than the predetermined threshold value in step S808. It is determined whether or not. If it is determined in this determination that it is less than the predetermined threshold value, the similar color detection unit 13064 returns the information indicating that the most similar color position is present and the color acquisition position (center) as information on the most similar color position in the return value in step S810. ) And the process returns to the tracking process described with reference to FIG. If it is determined in step S808 that the threshold is equal to or greater than the predetermined threshold, the process proceeds to step S806.

  If it is determined in step S802 that the threshold value is equal to or greater than the predetermined threshold, the similar color detection unit 13064 sets information indicating that the most similar color position is not present in the return value in step S812, and the processing is described with reference to FIG. Return to the described tracking process. Other processes are the same as those in the first embodiment.

  According to this embodiment, when the color acquisition position having the color most similar to the tracking color is close to the color acquisition position (center), the color acquisition position having the color most similar to the tracking color is used as the tracking area. Also, the color acquisition position (center) is preferentially adopted. That is, in this embodiment, the weight of the central color acquisition position that matches the previous tracking area among the plurality of color acquisition positions is high.

  As described above, the color tracking technique according to the present invention has high color resolution. For this reason, the position of the tracking area tends to be updated more finely than necessary. On the other hand, according to the present embodiment, unnecessary updating of the tracking area is suppressed to some extent, and stable tracking is realized. Needless to say, the fifth embodiment may be used in combination with the second to fourth embodiments.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, the problem described in the column of problems to be solved by the invention can be solved and the effect of the invention can be obtained. The configuration in which this component is deleted can also be extracted as an invention. Furthermore, constituent elements over different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 100 ... Imaging device, 102 ... Imaging optical system, 104 ... Focus adjustment mechanism, 106 ... Aperture, 108 ... Aperture drive mechanism, 110 ... Shutter, 112 ... Shutter drive mechanism, 114 ... Imaging element, 116 ... Imaging element interface (IF) Circuit: 118 ... RAM, 120 ... Display element, 122 ... Display element drive circuit, 124 ... Touch panel, 126 ... Touch panel drive circuit, 128 ... Recording medium, 130 ... System controller, 132 ... Operation unit, 134 ... ROM, 1301 ... CPU DESCRIPTION OF SYMBOLS 1302 ... AF control circuit, 1303 ... AE control circuit, 1304 ... Image processing circuit, 1305 ... Focal length acquisition circuit, 1306 ... Tracking processing circuit, 13061 ... Image acquisition part, 13062 ... Tracking area setting part, 13063 ... Color acquisition position Determination unit, 13064 ... Similar color detection unit, 13065 ... Tracking Frequency updating unit, 13066 ... reliability determination unit, 13,067 ... tracking color centroid searching unit, 1307 ... face detection circuit, 1308 ... memory controller.

Claims (18)

An image acquisition unit for acquiring images;
A tracking area setting unit that sets a part of the image as a tracking area and sets a color of the tracking area as a tracking color;
A comparison area setting unit that sets at least three areas having a predetermined positional relationship in the image as comparison areas;
A similar color detection unit that selects the comparison region having the color most similar to the tracking color among the comparison regions as an update region;
A tracking area update unit that causes the tracking area setting unit to execute a tracking area update that resets the tracking area based on the update area;
A tracking device comprising: An image acquisition unit for acquiring images;
A template storage unit that stores a predetermined positional relationship of at least three comparison regions as a point template;
A tracking area setting unit that sets a part of the image as a tracking area and sets a color of the tracking area as a tracking color;
A template setting unit for setting the position of the point template based on the position of the tracking area in the image;
A similar color detection unit that selects, as an update region, the comparison region that has the color most similar to the tracking color among the comparison regions of the set point template;
A tracking area update unit that causes the tracking area setting unit to execute a tracking area update that resets the tracking area based on the update area;
A tracking device comprising:   The comparison area setting unit is configured to select a number of the comparison areas, a distance between the tracking area and the comparison area, a line connecting the tracking area and the comparison area, and any image of the image according to a moving speed of the tracking area. The tracking device according to claim 1, wherein at least one of an angle formed with an axis and an area of the tracking region is changed. Further comprising a focal length acquisition unit for acquiring a focal length of an optical system used when capturing the image;
The comparison area setting unit includes the number of comparison areas, the distance between the tracking area and the comparison area, a line connecting the tracking area and the comparison area, and the horizontal axis of the image according to the focal length. Changing at least one of the corners and the area of the tracking region;
The tracking device according to claim 1. One of the comparison areas and the tracking area coincide;
The similar color detection unit changes the weighting of the comparison area that matches the tracking area and the comparison areas other than the comparison area in the comparison area according to the similarity between the color of the comparison area and the tracking color. And determining the update area,
The tracking device according to any one of claims 1 to 4, wherein the tracking device is characterized in that:   The frame rate at which the tracking region update unit performs the tracking region update includes a frame rate for displaying the image, a frame rate for recording the image, and auto-focus, automatic exposure used when capturing the image, 6. The tracking device according to claim 1, wherein the tracking device is changed according to at least one of a frame rate of auto white balance.   The frame rate at which the tracking area update unit performs the tracking area update when the difference between the color of the comparison area and the tracking color is equal to or less than a predetermined value is increased. The tracking device according to any one of claims.   The tracking device according to claim 1, further comprising a tracking region notification unit that displays the tracking region.   The tracking device according to claim 1, further comprising a reliability determination unit that determines the reliability of the update region determined by the similar color detection unit. When the difference between the color of the comparison area and the tracking color is equal to or greater than a predetermined value, the similar color detection unit does not determine the update area,
When the similar color detection unit does not determine the update region, the tracking region update unit does not cause the tracking region setting unit to execute the tracking region update,
The tracking device according to claim 1, wherein the tracking device is one of the following. The image processing apparatus further includes a similar area search unit that scans the image and searches for a similar area similar to the tracking color, and determines a centroid position of the similar area.
When the difference between the color of the comparison area and the tracking color is a predetermined value or more,
The similar region search unit determines the position of the center of gravity,
The tracking area update unit causes the tracking area setting unit to reset the tracking area based on the gravity center position.
The tracking device according to claim 1, wherein the tracking device is one of the following.   The said comparison area setting part determines the said positional relationship of the said comparison area according to the magnitude | size of the area | region which has the color similar to the said tracking color including the said tracking area. The tracking device according to any one of claims.   The comparison area setting unit changes an angle formed by a line connecting the tracking area and the comparison area and an arbitrary axis of the image according to a moving direction of the tracking area. The tracking device according to any one of 12.   13. The comparison area according to claim 1, wherein the comparison area is at least five areas including a position that coincides with the tracking area and four positions on the upper, lower, left, and right sides of the tracking area. The tracking device according to item.   The template setting unit, according to the moving speed of the tracking area, the number of the comparison areas, the interval between the comparison areas, the angle formed by the line connecting the comparison areas and an arbitrary axis of the image, and the tracking area The tracking device according to claim 2, wherein one point template is selected from a plurality of the point templates having at least one of different areas. Further comprising a focal length acquisition unit for acquiring a focal length of an optical system used when capturing the image;
The template setting unit may include a number of the comparison regions, an interval between the comparison regions, an angle formed by a line connecting the comparison regions and an arbitrary axis of the image, and an area of the tracking region according to the focal length. Selecting one of the point templates from a plurality of the different point templates.
The tracking device according to claim 2. Acquiring images,
Setting a partial area of the image as a tracking area, and setting a color of the tracking area as a tracking color;
Setting at least three regions having a predetermined positional relationship in the image as comparison regions;
Selecting the comparison area having the color most similar to the tracking color among the comparison areas as an update area;
Resetting the tracking area based on the update area;
The tracking method characterized by comprising. Acquiring images,
Setting a partial area of the image as a tracking area, and setting a color of the tracking area as a tracking color;
Setting a position of a point template having a predetermined positional relationship of at least three comparison areas based on the position of the tracking area in the image;
Selecting the comparison area having the color most similar to the tracking color as the update area among the comparison areas of the point template;
Resetting the tracking area based on the update area;
The tracking method characterized by comprising.

Images