JP2012203613A - Image processing device, image processing method, recording medium, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To track an image with a light load.SOLUTION: An image processing device calculates an evaluation value, which is expressed as the sum of confidence degrees obtained by compounding, at variable compound ratios, a matching degree of a first feature quantity and a matching degree of a second feature quantity between a target image including an object to be tracked and a comparison image which is an image of a comparison area to be compared with the target image of a predetermined frame, to obtain compound ratios which make the maximum evaluation value. Based on the confidence degrees in which the compound ratios making the maximum evaluation value are set, an image corresponding to the target image is detected.

Description

  The present technology relates to an image processing apparatus and method, a recording medium, and a program, and particularly to an image processing apparatus and method, a recording medium, and a program that can track an image with a light load.

  Digital cameras often have an autofocus function that automatically focuses a subject. As a result, the user can surely shoot the subject in focus with a simple operation of simply pointing the camera at the subject and operating the release switch.

  Further, in the case where a tracking function is provided, even if the subject moves, the subject is automatically tracked, so that it is possible to perform shooting while the subject is in focus.

  As a technique for automatically tracking a subject, there is a technique described in Non-Patent Document 1, for example.

`` Ensemble Tracking '' Shai Avidan, Mitsubishi Electric Research Labs, 201 Broadway Cambridge, MA02139, avidan@merl.com

  However, since the technique described in Non-Patent Document 1 uses a boosting technique, the calculation amount is enormous, and it is difficult to apply it to a digital camera which is a consumer image processing apparatus.

  The present technology has been made in view of such a situation, and makes it possible to track an image with a light load.

  One aspect of the present technology is a first feature amount matching degree between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image in a first frame. And calculating the evaluation value represented by the sum of the reliability obtained by mixing the matching degree of the second feature quantity with a predetermined mixing ratio when the mixing ratio is changed, and maximizing the evaluation value An image corresponding to the target image in the second frame is detected based on the calculation unit for obtaining the mixing rate when the evaluation value becomes and the reliability with which the mixing rate when the evaluation value is maximized is set. An image processing apparatus including a detection unit.

  The first frame and the second frame may be one of the odd frame and the even frame and the other.

  The image processing apparatus may further include a calculation unit that calculates the reliability of the target image and the scan image using the scan image of the scan area of the second frame as the comparison image.

  The detection unit can detect the scan image having the maximum reliability of the target image and the scan image as an image corresponding to the target image.

  The calculation unit sets the image of the reference area of the first frame as the target image, sets a plurality of areas including at least a part of the target image of the reference area as a positive area, and sets the target image of the reference area as the target image. A plurality of regions not included are defined as negative regions, a first reliability that is the reliability of the target image in the reference region and a plurality of images in the positive region is calculated, and a plurality of the target images in the reference region A second reliability that is the reliability with the image of the negative region is calculated, and a first product sum that is a product sum of the first reliability and the first weighting factor of the positive region is calculated. Calculating a second product sum that is a product sum of the second reliability and the second weighting factor of the negative region, and calculating the sum of the first product sum and the second product sum. It can be calculated as the evaluation value.

  The first weighting factor may be a value obtained by dividing a constant by the number of positive regions, and the second weighting factor may be a value obtained by dividing the constant by the number of negative regions.

  The calculation unit is a region corresponding to the coordinates of an image corresponding to the target image of the second frame, and an image in a region of a third frame further after the second frame is newly added to the target As the image, the evaluation value is calculated in the third frame to obtain the mixing ratio when the evaluation value is maximized, and the detection unit determines the evaluation value based on the image of the third frame. An image corresponding to the new target image of the third frame in a fourth frame further after the third frame, based on the reliability with which the mixing ratio when the maximum value is set is Can be detected.

  The image processing apparatus may further include a display unit that displays a marker in an area corresponding to the coordinates of the image corresponding to the target image.

  The image forming apparatus may further include a drive unit that drives the position of the camera so that an image corresponding to the target image is arranged at a predetermined position on the screen.

  The first feature amount may be luminance information, and the second feature amount may be color information.

  An image processing method, a recording medium, and a program according to an aspect of the present technology are an image processing method, a recording medium, and a program corresponding to the above-described image processing apparatus according to the present technology.

  In the aspect of the present technology, the first feature amount matching degree between the target image including the object to be tracked and the comparison image that is the image of the comparison region compared with the target image of the predetermined frame An evaluation value expressed by the sum of the reliability obtained by mixing the degree of matching of the two feature quantities at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing is performed when the evaluation value is maximized. A rate is required. An image corresponding to the target image is detected based on the reliability with which the mixing ratio when the evaluation value is maximized is set.

  As described above, according to one aspect of the present technology, an image can be tracked with a light load.

It is a block diagram which shows the structure of the digital camera of this technique. It is a flowchart explaining a tracking process. It is a figure explaining extraction of a field. It is a figure explaining an evaluation value It is a figure explaining a scan. It is a figure explaining the display of a marker. It is a figure explaining extraction of a field.

  FIG. 1 is a block diagram illustrating a configuration of a digital camera 1 of the present technology. The digital camera 1 includes a CPU (Central Processing Unit) 11, a lens 12, an output unit 13, an input unit 14, and a storage unit 15.

  The CPU 11 executes various processes. The lens 12 images a subject and supplies the image data to the CPU 11. The output unit 13 is configured by, for example, an LCD (Liquid Crystal Display) and displays an image captured by the lens 12. The output unit 13 includes a speaker and outputs necessary warning sounds and the like. The input unit 14 includes a release switch operated by the user, a member that adjusts the shutter speed and the exposure time, and the like. The storage unit 15 stores captured image data, a program for operating the CPU 11, and the like.

  For example, the drive unit 41 pans and tilts the camera 1 in a predetermined direction in a state where the camera 1 is mounted on a predetermined table (not shown).

  The CPU 11 has functional blocks of a capture unit 21, a cutout unit 22, an initialization unit 23, a calculation unit 24, a setting unit 25, a calculation unit 26, a detection unit 27, a display unit 28, a focus unit 29, and a determination unit 30. ing. Each unit can send and receive signals as necessary.

  The capturing unit 21 captures an image. The cutout unit 22 cuts out a predetermined portion from the captured image. The initialization unit 23 initializes the coefficients. The calculation unit 24 performs calculation at each place. The setting unit 25 sets a coefficient. The computing unit 26 performs computations at various places. The detection unit 27 detects a position. The display unit 28 displays a marker. The focus unit 29 performs focus adjustment. The determination unit 30 performs various determination processes.

  In this embodiment, each unit is functionally configured when a program is executed, but may of course be configured as hardware.

  FIG. 2 is a flowchart for explaining the tracking process. Hereinafter, the tracking process of the digital camera 1 will be described with reference to FIG.

  In step S1, the capturing unit 21 captures an image. That is, an image of a predetermined frame F1 of an image captured by the lens 12 is captured. This capturing is performed from an image captured by the lens 12 and stored in the storage unit 15.

  In step S2, the cutout unit 22 cuts out a region including the object and a region not including the object from the frame image captured in the process of step S1. An object is an image of a target that the user desires to track, for example, the face of a subject. This face clipping will be described with reference to FIG.

  FIG. 3 is a diagram for explaining segmentation. As shown in FIG. 3, an object 102 is displayed in a frame 101 (corresponding to the frame F1) that is an image captured in the process of step S1. For example, a rectangular area including the object 102 is set as a reference area 111-0. The image in the reference area 111-0 is the target image 114. When the marker 231 is displayed on the subsequent frame 201 (corresponding to the frame F2) in the process of step S10 described later (see FIG. 6 described later), it corresponds to the coordinates of the marker 231 on the frame 201. A region on the frame 101 to be used is a reference region 111-0. In the first frame before the process of step S10 is executed, a rectangular area centered on a point designated by the user operating the input unit 14 is set as the reference area 111-0.

  In step S2, regions 111-1, 111-2,..., 111-Np including at least a part of the target image 114 of the reference region 111-0 are cut out. That is, Np areas are cut out as positive areas including the target image 114 of the reference area 111-0. Similarly, regions 112-1, 112-2,..., 112-Nn that do not include the target image 114 of the reference region 111-0 are cut out. That is, Nn areas are cut out as negative areas not including the target image 114 of the reference area 111-0.

Next, in step S3, the initialization unit 23 initializes the weighting factors w _P and w _N of each region. The weighting factors w _P and w _N are expressed by the following equation (1). Weight coefficient _{w P} is positive area 111-J (J = 1,2, ···, Np) is a weighting factor, the weighting factor _{w N} is negative region 112-J (J = 1,2, ·· ., Nn).

As represented by the formula (1), the weight coefficient w _P of the positive region, the constant G _P is a value obtained by dividing the number Np of positive areas, weighting coefficient w _N negative region, the negative constant G _N This is a value divided by the number of areas Nn. The value of the weighting factor w _P positive region is the same in each region 111-J. Similarly, the value of the weight coefficient w _N negative region is the same in each region 112-J. The value of the constant G _P, G _N is predetermined at the time of factory digital camera 1 is set.

The value of the constant _G P, _{G N,} for example to set either to 0.5, the value of the constant _{G P} is 0.8, the value of the constant _{G N} can also be 0.2. Of the weight coefficients w _P and w _N , the weight with the corresponding constant set to a larger value becomes stronger. Constant G _P, by setting the value of G _N to a predetermined value, the balance of the weight coefficient w _P and the weighting coefficient w _N, can be appropriately adjusted.

  In step S4, the calculation unit 24 calculates an evaluation value Eval (K). The evaluation value Eval (K) is expressed by Expression (2). The reliability Confidence (K) in Expression (2) is expressed by Expression (3). K is an integer value that is changed, for example, from 0 to 256.

That is, in Expression (2), the reliability Confidence (K) between the target image 114 of the reference region 111-0 and the images of the plurality of positive regions 111-1, 111-2,. It is said. This is the reliability Confidence (K) of the first term on the right side of Equation (2). The reliability Confidence (K) between the target image 114 in the reference area 111-0 and the images in the plurality of negative areas 112-1, 112-2,... Is the second reliability. This is the reliability Confidence (K) of the second term on the right side of Equation (2). First reliability and positive regions 111-1 and 111-2, the product-sum of the first weighting factor _{w P} of ... is the first product-sum, the second reliability and negative regions 112- 1,112-2, sum of products between the second weighting factor _{w N} of ... are the second sum-of-products. The sum of the first product sum and the second product sum is used as the evaluation value Eval (K).

  In Expression (3), feat_A is the degree of matching between the target image 114 including the object to be tracked and the first feature amount (eg, luminance information) of the comparison image, and feat_B is the second feature amount (eg, color) Information) matching degree. K means a mixing ratio of the matching degree feat_A of the first feature quantity and the matching degree feat_B of the second feature quantity. As can be seen from Equation (3), the reliability Confidence (K) represents the probability that the comparison image matches the target image 114, and the larger the value, the more likely the comparison image matches the target image 114. High nature. Of course, feature quantities other than luminance information and color information can be used.

  The true value of Σ in the first term on the right side in Equation (2) means that only the confidence (K) in the positive region is summed. The comparison image to be compared with the target image 114 when calculating Confidence (K) of the positive region of the first term on the right side in Expression (2) is an image of the positive region 111-J. Similarly, the true value of Σ in the second term means that only the Confidence (K) in the negative region is summed. When calculating Confidence (K) of the negative region in the second term on the right side in Expression (2), the comparison image compared with the target image 114 is an image of the negative region 112-J.

  In step S5, the calculation unit 24 obtains a mixing rate Km that maximizes the evaluation value Eval (K). That is, the value of the evaluation value Eval (K) is calculated by sequentially changing the value of the mixing ratio K from 0 to 256. Then, the maximum value is selected from the 257 evaluation value Eval (K) values, and the mixture ratio Km that maximizes the evaluation value Eval (K) value is determined.

  FIG. 4 is a diagram for explaining the evaluation value. When the value of the mixing rate K is sequentially changed from 0 to 256, the evaluation value Eval (K) changes as shown in FIG. In the example of FIG. 4, the mixing rate K that maximizes the value of the evaluation value Eval (K) is Km. The mixing rate Km that maximizes the evaluation value Eval (K) is the optimal mixing rate for detecting the target image 114 including the object 102 of the frame. As will be described later in steps S8 and S9, in the next frame, the reliability Confidence (K) is calculated using this mixing ratio Km. That is, the evaluation value Eval (K) is a function for determining the optimum mixing ratio Km.

  Therefore, in step S6, the setting unit 25 sets the mixing ratio Km obtained in step S5 to the reliability Confidence (K) of the equation (3).

  As described above, the learning process of the mixing ratio K is performed in the first frame by the processes in steps S1 to S6, and then the tracking process is performed in the second frame in subsequent steps S7 to S11.

  In step S7, the capturing unit 21 captures an image. That is, the image of the frame F2 next to the frame F1 captured in step S1 is read from the storage unit 15 and captured.

  In step S8, the calculation unit 26 scans the scanned image on the captured image, and calculates the reliability Confidence (K) of each scanned image. That is, the image of the reference area 111-0 of the frame F1 is determined as the target image 114 in the process of step S2. A scan image of a scan area having a size corresponding to the target image 114 at a predetermined position on the current frame (that is, the frame F <b> 2 captured in step S <b> 7) is extracted as a comparison image and compared with the target image 114. Then, a first feature amount matching degree feat_A and a second feature amount matching degree feat_B between the target image 114 and the scanned image are calculated. The reliability degree Confidence (K) is calculated by applying the calculated matching degree feat_A of the first feature quantity and the matching degree feat_B of the second feature quantity to Equation (3). As the mixing ratio K at this time, the value Km set in step S6 is used.

  FIG. 5 is a diagram for explaining scanning. As shown in FIG. 5, a scan image 222-1 of the scan region 221-1 at a predetermined position on the frame 201 (that is, the frame F2) captured in step S7 is extracted as a comparison image and designated in step S2. It is compared with the target image 114 of the previous frame F1. The size of the scan area 221-1 is the same as that of the reference area 111-0. That is, the scanned image 222-1 has the same size as the target image 114. The reliability Confidence (K) between the target image 211 and the scan image 222-1 is calculated in a state where the value of the mixing ratio K in Expression (3) is set to the maximum value Km.

  The comparison area on the frame 201 is sequentially moved to comparison areas 211-1, 211-2, 211-3,..., And the same processing is repeated. The range to be scanned on the frame 201 may be the entire frame 201, but the coordinates of the reference area 111-0 specified in step S2 (that is, the coordinates at which the marker 231 is displayed in the previous processing in step S10). Can be within a predetermined distance. The amount of calculation can be reduced by limiting the scanning range.

  In step S9, the detection unit 27 detects a region where the reliability Confidence (K) is maximized. That is, the reliability Confidence (K) having the largest value is selected from the reliability Confidence (K) of each scan region 221-J (J = 1, 2,...) Calculated in the process of Step S8. Then, the scan area 221 -M corresponding to the reliability Confidence (K) is selected. The image of the scan area 221 -M on the frame 201 (frame F2) is set as an image 232 corresponding to the target image 114 on the frame 101 (frame F1). That is, it is determined that the target image 114 in the reference area 111-0 on the frame 101 is moved to the scan area 221-M in the frame F2 and displayed as the image 232 (see FIG. 6 described later).

  In step S10, the display unit 28 displays the marker 231 at the detected position. FIG. 6 is a diagram for explaining the display of the marker 231. In FIG. 6, an image 232 including the object 102 is displayed in the scan area 221 -M. A marker 231 is displayed at the position of the scan area 221-M. That is, the marker 232 is displayed with respect to the image 232. The focus unit 29 drives and adjusts the lens 12 so that the image is focused on the image 232 displayed in the marker 231. The user can check the marker 231 to confirm where the focus is now.

  In step S11, the determination unit 30 determines whether to end tracking. When the user operates the input unit 14 and instructs to stop tracking, the tracking process is terminated.

  If tracking is not instructed, the process returns to step S1, and an image of the next frame F3 is further captured. In step S2, a process for cutting out an area including the object is performed. In the case of the first frame F1, since the process of step S10 has not yet been performed, the reference area 111-0 is set based on the position designated by the user. However, in this case, since the coordinates of the image 232 corresponding to the previous target image 114 are known in the process of step S10, the next frame of the coordinates corresponding to the area 221-M where the marker 231 of the frame 201 is displayed. An area 301 is set as a new reference area 111-0, and the cutout process is performed based on the new reference area 111-0.

  FIG. 7 is a diagram for explaining the second region extraction. As shown in FIG. 7, the region 311-0 of the frame 301 (that is, the frame F3) newly acquired in the process of the second step S1 is on the frame 201 (that is, the frame F2) of FIG. This is an area corresponding to the scan area 221-M. This region 311-0 is set as a reference region of a new frame 301, and an image displayed there is set as a new target image 314. The cut-out unit 22 uses the new reference area 311-0 as a reference to create new positive areas 311-1, 311-2,... And new negative areas 312-1, 312-2,. cut.

  Thereafter, the same processing is performed. That is, an image corresponding to the coordinates of the image 232 corresponding to the target image 114 of the frame F2 and in the region of the frame F3 after the frame F2 is set as a new target image 314. (K) is calculated. That is, the evaluation value Eval (K between the new target image 314 and the new positive areas 311-1, 311-2,... And the new negative areas 312-1, 312-2,. ) Is calculated.

  Further, the mixing ratio Km when the calculated evaluation value Eval (K) becomes maximum is obtained. Then, based on the reliability Confidence (K) in which the mixing ratio Km when the evaluation value Eval (K) is maximized based on the image of the frame F3 is set, the frame F4 (not shown) further after the frame F3 ), An image corresponding to the new target image 314 of the frame F3 is detected.

  Such processing is repeated for each frame, and when the object 102 moves, the marker 231 tracks and displays the movement destination. The processes of steps S1 to S6 are executed on one of the consecutive odd frames and even frames, and the processes of steps S7 to S11 are executed on the other.

  Note that the second term on the right side of Equation (2) can be omitted. However, in this case, the quality of the tracking function is deteriorated as compared with the case where it is not omitted.

  Further, instead of the normalization process in Expression (3), that is, without dividing by the value 256, (1-K) may be used instead of (256-K).

  In step S10, the marker 231 is displayed. However, the driving unit 41 is driven, and the position of the camera 1 is panned and tilted so that the object 102 is always located at a predetermined position (for example, the center) in the frame. It can also be controlled.

  This technology uses only the information obtained for each frame. For example, it does not use information obtained from images between multiple frames, such as motion vectors, or uses a distance measuring device. Will be quick and easy. Further, since the present technology has a small amount of calculation, it can be applied to a video camera, a surveillance camera, and other small and inexpensive image processing apparatuses in addition to a digital camera to track an object in real time.

  The series of processes described above can be executed by hardware or can be executed by software.

  When a series of processing is executed by software, a program constituting the software is stored in the storage unit 15.

  In this specification, the program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or when a call is made. It may be a program that performs processing at a necessary timing.

  The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

The present technology can be configured as follows.
(1)
The degree of matching of the first feature amount and the second feature amount between the target image including the object to be tracked and the comparison image that is an image of the comparison region compared with the target image of the first frame. An evaluation value represented by the sum of the reliability obtained by mixing the matching degree at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is calculated. A calculation unit to be obtained;
An image processing apparatus comprising: a detection unit configured to detect an image corresponding to the target image in a second frame based on the reliability set with the mixing ratio when the evaluation value is maximized.
(2)
The image processing apparatus according to (1), wherein the first frame and the second frame are one of the odd frame and the even frame and the other.
(3)
A scan unit that scans the scan area of the second frame as the comparison image, and further includes a calculation unit that calculates the reliability of the target image and the scan image.
The image processing apparatus according to (1) or (2).
(4)
The detection unit detects the scan image having the maximum reliability of the target image and the scan image as an image corresponding to the target image. The image according to (1), (2), or (3) Processing equipment.
(5)
The calculation unit sets the image of the reference area of the first frame as the target image, sets a plurality of areas including at least a part of the target image of the reference area as a positive area, and sets the target image of the reference area as the target image. A plurality of regions not included are defined as negative regions, a first reliability that is the reliability of the target image in the reference region and a plurality of images in the positive region is calculated, and a plurality of the target images in the reference region Calculating a second reliability that is the reliability with the image of the negative region, and calculating a first product sum that is a product sum of the first reliability and the first weighting factor of the positive region. Calculating a second product sum that is a product sum of the second reliability and the second weighting factor of the negative region, and calculating the sum of the first product sum and the second product sum. Calculated as the evaluation value (1) to (4 ).
(6)
The first weighting factor is a value obtained by dividing a constant by the number of positive regions, and the second weighting factor is a value obtained by dividing the constant by the number of negative regions. The image processing apparatus according to any one of 5).
(7)
The calculation unit is a region corresponding to the coordinates of an image corresponding to the target image of the second frame, and an image in a region of a third frame further after the second frame is newly added to the target As the image, the evaluation value is calculated in the third frame, and the mixing ratio when the evaluation value is maximized is obtained,
The detection unit, based on the reliability at which the mixing ratio when the evaluation value is maximized based on the image of the third frame is set, is set to a fourth level further after the third frame. The image processing device according to any one of (1) to (6), wherein an image corresponding to the new target image of the third frame is detected in a frame.
(8)
The image processing apparatus according to any one of (1) to (7), further including a display unit that displays a marker in a region corresponding to the coordinates of the image corresponding to the target image.
(9)
The image processing apparatus according to any one of (1) to (8), further including a drive unit that drives a position of the camera so that an image corresponding to the target image is arranged at a predetermined position on the screen.
(10)
The first feature amount is luminance information;
The image processing apparatus according to any one of (1) to (9), wherein the second feature amount is color information.
(11)
Matching between the first feature amount and the second feature amount between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image of a predetermined frame An evaluation value represented by the sum of the reliability obtained by mixing the degrees at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is obtained. A calculation step;
And a detecting step of detecting an image corresponding to the target image based on the reliability with which the mixing ratio when the evaluation value is maximized is set.
(12)
Matching between the first feature amount and the second feature amount between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image of a predetermined frame An evaluation value represented by the sum of the reliability obtained by mixing the degrees at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is obtained. A calculation step;
A recording in which a program for causing a computer to execute processing including: a detection step of detecting an image corresponding to the target image based on the reliability with which the mixing ratio when the evaluation value is maximized is set is recorded Medium.
(13)
Matching between the first feature amount and the second feature amount between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image of a predetermined frame An evaluation value represented by the sum of the reliability obtained by mixing the degrees at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is obtained. A calculation step;
And a detection step of detecting an image corresponding to the target image based on the reliability set with the mixing ratio when the evaluation value is maximized.

DESCRIPTION OF SYMBOLS 1 Digital camera, 12 Lens, 13 Output part, 14 Input part, 15 Storage part, 21 Capture part, 22 Extraction part, 23 Initialization part, 24 Calculation part, 25 Setting part, 26 Calculation part, 27 Detection part, 28 Display Part, 29 focus part, 30 judgment part

Claims (13)

The degree of matching of the first feature amount and the second feature amount between the target image including the object to be tracked and the comparison image that is an image of the comparison region compared with the target image of the first frame. An evaluation value represented by the sum of the reliability obtained by mixing the matching degree at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is calculated. A calculation unit to be obtained;
An image processing apparatus comprising: a detection unit configured to detect an image corresponding to the target image in a second frame based on the reliability set with the mixing ratio when the evaluation value is maximized. The image processing apparatus according to claim 1, wherein the first frame and the second frame are one of an odd frame and an even frame. A scan unit that scans the scan area of the second frame as the comparison image, and further includes a calculation unit that calculates the reliability of the target image and the scan image.
The image processing apparatus according to claim 2. The image processing apparatus according to claim 3, wherein the detection unit detects the scan image having the maximum reliability of the target image and the scan image as an image corresponding to the target image. The calculation unit sets the image of the reference area of the first frame as the target image, sets a plurality of areas including at least a part of the target image of the reference area as a positive area, and sets the target image of the reference area as the target image. A plurality of regions not included are defined as negative regions, a first reliability that is the reliability of the target image in the reference region and a plurality of images in the positive region is calculated, and a plurality of the target images in the reference region Calculating a second reliability that is the reliability with the image of the negative region, and calculating a first product sum that is a product sum of the first reliability and the first weighting factor of the positive region. Calculating a second product sum that is a product sum of the second reliability and the second weighting factor of the negative region, and calculating the sum of the first product sum and the second product sum. The image according to claim 4, wherein the image is calculated as the evaluation value. Image processing device. The first weighting factor is a value obtained by dividing a constant by the number of positive regions, and the second weighting factor is a value obtained by dividing the constant by the number of negative regions. Image processing device. The calculation unit is a region corresponding to the coordinates of an image corresponding to the target image of the second frame, and an image in a region of a third frame further after the second frame is newly added to the target As the image, the evaluation value is calculated in the third frame, and the mixing ratio when the evaluation value is maximized is obtained,
The detection unit, based on the reliability at which the mixing ratio when the evaluation value is maximized based on the image of the third frame is set, is set to a fourth level further after the third frame. The image processing apparatus according to claim 5, wherein an image corresponding to the new target image of the third frame is detected in a frame. The image processing apparatus according to claim 5, further comprising: a display unit that displays a marker in an area corresponding to the coordinates of the image corresponding to the target image. The image processing apparatus according to claim 5, further comprising a drive unit that drives the position of the camera so that an image corresponding to the target image is arranged at a predetermined position on the screen. The first feature amount is luminance information;
The image processing apparatus according to claim 5, wherein the second feature amount is color information. Matching between the first feature amount and the second feature amount between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image of a predetermined frame An evaluation value represented by the sum of the reliability obtained by mixing the degrees at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is obtained. A calculation step;
And a detecting step of detecting an image corresponding to the target image based on the reliability with which the mixing ratio when the evaluation value is maximized is set. Matching between the first feature amount and the second feature amount between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image of a predetermined frame An evaluation value represented by the sum of the reliability obtained by mixing the degrees at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is obtained. A calculation step;
A recording in which a program for causing a computer to execute processing including: a detection step of detecting an image corresponding to the target image based on the reliability with which the mixing ratio when the evaluation value is maximized is set is recorded Medium. Matching between the first feature amount and the second feature amount between a target image including an object that is a tracking target and a comparison image that is an image of a comparison region compared with the target image of a predetermined frame An evaluation value represented by the sum of the reliability obtained by mixing the degrees at a predetermined mixing ratio when the mixing ratio is changed is calculated, and the mixing ratio when the evaluation value is maximized is obtained. A calculation step;
And a detection step of detecting an image corresponding to the target image based on the reliability set with the mixing ratio when the evaluation value is maximized.

Images