JP5024033B2 - Object tracking device, object tracking program, and object tracking method - Google Patents

Description

  The present invention relates to a subject tracking device, a subject tracking program, and a subject tracking method for tracking a subject within a frame that is sequentially imaged.

  2. Description of the Related Art In recent years, in an imaging apparatus, a technique for performing focus control while tracking a subject by extracting a subject to be tracked (pixel block) from sequentially captured frames and detecting it in the next frame. Exists.

Also proposed is a subject tracking device that updates the subject to be extracted every time the frame is updated so that the subject to be tracked can move quickly and reliably even if it is small. (For example, refer to Patent Document 1).
JP-A-9-134418

  However, in the case of the object target tracking device described above, it is effective and can be tracked if the tracking target can be clearly identified. In this case, it becomes difficult to specify the tracking target. For this reason, there arises a problem that the tracking target is lost or the erroneously detected erroneous target is tracked.

  The present invention has been made in view of such a conventional problem, and is a subject tracking device, a subject tracking program, and a subject that can accurately track a subject in frames that are sequentially captured. It aims to provide a tracking method.

In order to solve the above-mentioned problem, the invention according to claim 1 is based on a pixel block to be tracked in a current frame that is sequentially imaged and determines a pixel block to be tracked in the next frame to track the subject to be imaged. A tracking device that selects a candidate pixel block to be tracked in the next frame based on a tracked pixel block in the current frame; and a plurality of pixel blocks are selected as candidate pixel blocks by the selection unit. When selected, the distance between the selected candidate pixel blocks is the first distance that is the distance between the candidate pixel blocks that are the farthest and the distance between the candidate pixel blocks that are the closest to the position. The calculation means for calculating the second distance, and the inter-block distance coefficient prepared as a coefficient inversely proportional to the distance, A first multiplying unit that substitutes a distance of 1 and multiplies the evaluation difference absolute value between the farthest candidate pixel blocks, and the second distance is substituted into the interblock distance coefficient, and the closest A second multiplying unit that multiplies the evaluation difference absolute value between candidate pixel blocks having a positional relationship, and a range that is equal to or smaller than a predetermined value among a value obtained by the first multiplying unit and a value obtained by the second multiplying unit A decision is made to determine the tracking target pixel block in the next frame based on the exclusion means that leaves the candidate pixel blocks that have obtained a certain value and excludes the other, and the candidate pixel block that remains after the exclusion means Means.

  According to a second aspect of the present invention, in the first aspect of the present invention, an evaluation value indicating a degree of approximation is provided for each pixel block having the same size as the tracking target pixel block within a predetermined search range in the next frame. Evaluation value calculating means for calculating is further provided, and the selecting means selects candidate pixel blocks to be tracked based on the evaluation value for each pixel block calculated by the evaluation value calculating means. .

According to a third aspect of the present invention, in the second aspect of the present invention, the determination means is a candidate remaining by the exclusion of the exclusion means based on the evaluation value of the pixel block calculated by the evaluation value calculation means. The tracking target pixel block in the next frame is determined from the pixel block.

According to a fourth aspect of the present invention, there is provided an imaging system according to any one of the first to third aspects, wherein an imaging unit that captures an image and sequentially outputs the frame, and an optical system that forms a subject image on the imaging unit. And a driving means for driving the optical system, and a control means for controlling the driving means so that the image in the tracking target block determined by the determining means is in focus.

According to a fifth aspect of the present invention, there is provided a computer having an apparatus for determining a tracking target pixel block in a next frame and tracking an object to be captured based on a tracking target pixel block in a current frame that is sequentially imaged. Selection means for selecting a candidate pixel block to be tracked in the next frame based on the tracking target pixel block in the current frame, and when a plurality of pixel blocks are selected as candidate pixel blocks by the selection means, the selection is performed. In the distance between the candidate pixel blocks, the first distance that is the distance between the candidate pixel blocks having the farthest positional relationship and the second distance that is the distance between the candidate pixel blocks having the closest positional relationship are calculated. The calculation means substitutes the first distance for the inter-block distance coefficient prepared as a coefficient inversely proportional to the distance. First multiplication means for multiplying the evaluation difference absolute value between the farthest candidate pixel blocks having a positional relationship, substituting the second distance for the inter-block distance coefficient, and The second multiplication means for multiplying the evaluation difference absolute value, and the candidate that has obtained a value within a predetermined value range among the value obtained by the first multiplication means and the value obtained by the second multiplication means Excluding means for leaving pixel blocks and excluding the other , and functioning as determining means for determining a pixel block to be tracked in the next frame based on candidate pixel blocks remaining after exclusion of the excluding means .

The invention according to claim 6 is a subject tracking method for tracking a subject by determining a subject pixel block in a next frame based on a subject pixel block in a current frame that is sequentially imaged. A selection step of selecting a candidate pixel block to be tracked in the next frame based on the tracking target pixel block in the current frame, and a plurality of pixel blocks selected as candidate pixel blocks by the selection step In the distance between the selected candidate pixel blocks, a first distance that is a distance between candidate pixel blocks having the farthest positional relationship and a second distance that is a distance between candidate pixel blocks having the closest positional relationship And the inter-block distance coefficient prepared as a coefficient inversely proportional to the distance. A first multiplication step of multiplying the farthest relative candidate pixel blocks by the evaluation difference absolute value, and substituting the second distance for the inter-block distance coefficient, and the closest position A second multiplication step for multiplying the evaluation difference absolute value between the related candidate pixel blocks, a value obtained in the first multiplication step, and a value obtained in the second multiplication step, which are less than or equal to a predetermined value A pixel block to be tracked in the next frame is determined based on an exclusion step in which candidate pixel blocks having values in a range remain and exclude the other, and a candidate pixel block remaining in the exclusion step And a determining step.

  According to the present invention, it is possible to accurately track a subject to be captured in frames that are sequentially imaged.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a digital camera according to an embodiment of the present invention. This digital camera has a function of performing AE (automatic exposure) and AF (autofocus) by tracking an object to be imaged, and mainly includes the following parts.

  That is, the digital camera has a CCD 2 as an imaging unit that photoelectrically converts an optical image of a subject imaged by the optical system 1 and outputs it as an imaging signal. The CCD 2 is driven by a drive signal generated by a CCD drive circuit 4 based on a timing signal generated by a timing generator (TG) 3. The output signal of the CCD 2 is converted into a digital signal by the A / D converter 5 and then sent to the image processing unit 6.

  The optical system 1 includes a focus lens and a lens motor for moving the focus lens on the optical axis, and the lens motor is driven by a drive signal generated by the lens drive circuit 8 in accordance with a command from the control unit 7. Thus, the position of the focus lens on the optical axis is controlled.

  The image processing unit 6 performs RGB interpolation processing for generating R, G, and B color component data (RGB data) for each pixel with respect to the imaging signal after A / D conversion, and luminance signal (Y) from the RGB data. And YUV conversion processing for generating YUV data consisting of color difference signals (U, V) for each pixel, and digital signal processing for improving image quality such as auto white balance and edge enhancement. The YUV data converted by the image processing unit 6 is sequentially stored in the SDRAM 9.

  In addition, the digital camera has a display unit 10 including a liquid crystal monitor and a driving circuit thereof. When the recording mode for shooting is set, the YUV data stored in the SDRAM 9 is converted into a video signal in the image processing unit 6 every time one frame is accumulated in the SDRAM 9, and then the display unit 10 (liquid crystal monitor). ) Is displayed on the screen as a through image. When shooting in the recording mode, the image data temporarily stored in the SDRAM 9 is compressed and encoded by the JPEG conversion unit 11 using the JPEG method, and then configured by, for example, various memory cards via the external memory I / F 12. It is recorded in the external memory 13 as a still image file.

  The still image file recorded in the external memory 13 is appropriately read in accordance with the user's selection operation in the reproduction mode, is decompressed by the JPEG conversion unit 11 and developed as SDRAM data on the SDRAM 9, and then is statically displayed on the display unit 10. Displayed as an image.

  The key input block 14 includes a plurality of keys such as a shutter key, a mode selection key, a power key, and a MENU key, and the operation state of each key is scanned by the control unit 7 as needed. The shutter key has a so-called half shutter function that allows a user to perform a two-stage operation of a half-pressed position for performing a shooting advance notice and a full-pressed position for instructing an actual shooting operation.

  The control unit 7 is composed of a CPU and its peripheral circuits, and has a built-in counter T. The control unit 7 controls each part of the digital camera, uses AE control based on luminance information included in the imaging signal, and a contrast detection method. AF control, subject tracking control described later, and the like are performed. Various programs for causing the control unit 7 to perform these controls and various data necessary for each control are stored in the flash memory 16 which is a nonvolatile memory capable of rewriting stored data.

  Next, the operation of the digital camera having the above configuration will be described. When the power switch is turned on and the photographing mode is set, the CCD driving circuit 4 drives the CCD 2 based on the timing signal generated by the TG 3 so that the frame F (YUV data) converted by the image processing unit 6 is As shown in FIG. 2, the data are sequentially stored in the SDRAM 9. Further, every time one frame is accumulated in the SDRAM 9, it is converted into a video signal in the image processing unit 6, and then displayed on the screen as a through image on the display unit 10 (liquid crystal monitor).

  In frame F... Shown in FIG. 2, one of two consecutive frames is a current frame F1, and the other is a reference frame F2. The reference frame F2 is the next frame (next frame) of the current frame F1. As shown in FIG. 3 (1a), in the present embodiment, the range of 8 × 8 pixels in the current frame F1 is the reference pixel block B1, that is, the pixel block being tracked. In the present embodiment, in one frame, an area of 16 pixels in the horizontal direction and the vertical direction is set as a search range centering on an area corresponding to the reference pixel block B1. Accordingly, within the search range, the processing described later is performed for each evaluation pixel block of 8 × 8 pixels, and the next reference pixel block is traced.

  On the other hand, the control unit 7 executes processing as shown in the flowchart of FIG. 4 based on the control program stored in the flash memory 16. First, “1” is set to the value of the counter T for counting the number of pixel blocks (8 × 8 pixels) that have been evaluated within the search range (step S1). Next, in the search range, the evaluation value of the first (C-th) evaluation pixel block, which is a pixel block shifted by one pixel around the region corresponding to the reference pixel block B1, is calculated (step S2).

  That is, an evaluation pixel block B2 indicated by a broken line in FIG. 3 (2b) is selected from a plurality of evaluation pixel blocks existing in the search range, and pixel data that is a value of each pixel constituting the selected evaluation pixel block B2 Based on the pixel data of each pixel constituting the reference pixel block B1 shown in FIG. 1 (b), the degree of approximation between the pair of pixels corresponding to each other when the two pixel blocks B1 and B2 are overlapped. (A difference absolute value or a square difference value of pixel data of both pixels) is obtained, and the sum of these pixel evaluation values is used as a block evaluation value of the selected evaluation pixel block B2. Therefore, the higher the degree of approximation between the two pixel data, the smaller the pixel evaluation value, that is, the higher the degree of approximation (correlation) between the evaluation pixel block B2 and the reference pixel block B1, the smaller the block evaluation value. .

More specifically, as described above, the 8 × 8 pixel range is the reference pixel block B1, the same position in the reference pixel block B1 and the evaluation pixel block B2 is the origin (0, 0), and the search range is When 16 pixels are set in the horizontal and vertical directions, the block evaluation value is calculated by the following formula.
B (x, y) = Σ (Pref (x, y) [i] −Pcur (x, y) [i]) ^ 2
(However, i = 0 to 63, x = −16 to 16, y = −16 to 16 (refer to FIG. 3 for Pref and Pcur).

  Next, it is determined whether or not the calculated evaluation value is equal to or less than a predetermined threshold value (step S3). As a result of the determination in step S3, when the calculated evaluation value is equal to or less than a predetermined threshold, that is, the degree of approximation (correlation degree) between the evaluation pixel block B2 and the reference pixel block B1 whose evaluation value is calculated in step S2. ) Is high, the evaluation pixel block B2 is selected as a selection target pixel block (step S4).

  Subsequently, it is determined whether or not the value of the counter T counting the number of evaluated pixel blocks has reached the number of all pixel blocks in the search range (step S5). If the counter T is not equal to the total number of pixel blocks and the calculation of the evaluation values for all the pixel blocks within the search range has not been completed, the counter T is incremented (step S6), and from step S2 Repeat the process.

  Therefore, the processes in steps S2 to S5 are repeated a number of times corresponding to the number of pixel blocks in the search range until the determination in step S5 is YES. In addition, by the process in step S4, a pixel block whose evaluation value is equal to or less than the threshold value, that is, one or a plurality of pixel blocks having a high degree of approximation (correlation) with the reference pixel block B1, is selected as the selection target pixel block. It will be.

  When the calculation of the evaluation values for all the pixel blocks in the search range is completed and the determination in step S5 is YES, it is determined whether or not there are a plurality of selection target pixel blocks selected in step S4. (Step S7). At this time, if the tracking target can be clearly identified, the pixel block whose evaluation value is equal to or less than the threshold value is single, and the pixel block selected as the selection target pixel block in step S3 is single. The determination in step S7 is NO. Accordingly, the process proceeds from step S7 to step S8, and this single selection target pixel block is determined as a tracking pixel block.

  However, under a situation where it is desired to capture a frame that is generally uniform and has little change, a plurality of pixel blocks may be selected as the selection target pixel block in step S4, and in this case, the determination in step S7 is performed. Yes. Therefore, in this case, the process proceeds from step S7 to step S9, and the inter-pixel block distance is calculated.

  That is, as illustrated in FIG. 5, when four pixel blocks Ba, Bb, Bc, and Bd are selected as selection target pixel blocks, the distance L1 between Ba and Bb that is the farthest positional relationship is the closest. The distance L2 between Bc to Bd, which is the positional relationship, is calculated. Next, using the inter-block distance coefficient C (L) prepared as a coefficient (parameter) inversely proportional to the distance L and the calculated inter-pixel block distance, “inter-block distance coefficient × evaluation difference absolute value” Is calculated (step S10). Here, in the above example, the evaluation difference value is Ba−Bb of the difference between the evaluation values of the pixel block Ba and the pixel block Bb, and Bc−Bd of the difference between the evaluation values of the pixel block Bc and the pixel block Bd. Therefore, in step S9, C (L1) × | Ba−Bb | and C (L2) × | Bc−Bd | are calculated.

  Next, it is determined whether or not the value calculated in step S10 is within a selection range that is a predetermined value or less (step S11). If the result of determination in step S11 is outside the selection range (step S11; NO), the pair of pixel blocks is excluded from the selection target pixel block (step S12), and if it is within the selection range (step S12). In S11; YES), the pair of pixel blocks is left as a selection target pixel block (step S13).

  Accordingly, for example, if C (L1) × | Ba−Bb | is outside the selection range and C (L2) × | Bc−Bd | is within the selection range, the pixel block Ba is processed by the processing in step S12. , Bb are excluded from the selection target pixel block, and the pixel block Bc, Bd remains as the selection target pixel block by the processing in step S13.

  Therefore, in step S14, the minimum evaluation value pixel block which is the pixel block having the smallest block evaluation value among the plurality of pixel blocks remaining as the selection target pixel block is finally determined as the tracking pixel block. Therefore, in the pixel blocks Bc and Bd remaining as the selection target pixel block, for example, when the block evaluation value of the pixel block Bd is lower and the degree of approximation (correlation) with the reference pixel block B1 is higher, Bd is determined as the tracking pixel block.

  Therefore, even when capturing a frame that is generally uniform and has little change, the subject to be tracked in the sequentially captured frames is not lost without losing sight of the tracking target or tracking an erroneously detected erroneous target. The object can be accurately tracked.

  When the tracking pixel block is determined in step S14 or step S8 as described above, the tracking pixel block becomes the reference pixel block B1, and the above-described processing is repeated. Thereby, it is possible to perform focus control while tracking the object to be imaged by extracting the object to be tracked (pixel block) from the frames that are sequentially imaged and detecting this in the next frame.

  That is, as described above, the optical system 1 includes the focus lens and a lens motor for moving the focus lens on the optical axis, and the lens motor generates a drive signal generated by the lens drive circuit 8 in accordance with a command from the control unit 7. , The position of the focus lens on the optical axis is controlled. At this time, by performing the focus control so that the region set as the reference pixel block B1 is in focus, it is possible to perform shooting while continuing the focused state even if the subject is moved. Become.

  In the present embodiment, whether to exclude from the selection target pixel block or remain as the selection target pixel block is determined based on “inter-block distance coefficient × evaluation value difference absolute value”. This determination may be made based solely on the “distance”. Further, based on the distance from the center point of the reference pixel block B1 instead of the inter-block distance, a pixel block whose distance is a predetermined value or more may be excluded from the selection target pixel block. Furthermore, in the embodiment, the case where the present invention is applied to a digital camera has been described in detail. It is applicable without being done.

1 is a block diagram of a digital camera according to an embodiment of the present invention. It is explanatory drawing which shows the flame | frame sequentially imaged, the present frame, and a reference frame. It is explanatory drawing which shows a reference | standard pixel block and an evaluation pixel block. It is a flowchart which shows the process sequence in one embodiment of this invention. It is explanatory drawing which shows an example of a selection object pixel block and the distance between blocks.

Explanation of symbols

1 Optical system 2 CCD
3 TG
4 CCD drive circuit 5 A / D converter 6 Image processing unit 7 Control unit 8 Lens drive circuit 9 SDRAM
DESCRIPTION OF SYMBOLS 10 Display part 11 JPEG conversion part 13 External memory 14 Key input block 16 Flash memory

Claims (6)

A target tracking device for tracking a target by determining a target pixel block in a next frame based on a target pixel block in a current frame that is sequentially imaged,
Selection means for selecting candidate pixel blocks to be tracked in the next frame based on the tracked pixel blocks in the current frame;
When a plurality of pixel blocks are selected as candidate pixel blocks by this selection means, the distance between the selected candidate pixel blocks is the first distance that is the distance between the farthest candidate pixel blocks, A calculating means for calculating a second distance that is a distance between candidate pixel blocks in a close positional relationship;
First multiplying means for substituting the first distance for an interblock distance coefficient prepared as a coefficient inversely proportional to the distance, and multiplying the evaluation difference absolute value between the farthest candidate pixel blocks;
Second multiplying means for substituting the second distance for the inter-block distance coefficient and multiplying the evaluation difference absolute value between the closest candidate pixel blocks having the positional relationship;
Of the values obtained by the first multiplying means and the values obtained by the second multiplying means, the candidate pixel blocks that have obtained values that are in a range equal to or less than a predetermined value remain, and the other is excluded Means,
A subject tracking apparatus, comprising: a determination unit that determines a pixel block to be tracked in the next frame based on a candidate pixel block remaining after the exclusion by the exclusion unit. An evaluation value calculating means for calculating an evaluation value indicating a degree of approximation for each pixel block of the same size as the tracking target pixel block within a predetermined search range in the next frame;
2. The object target tracking device according to claim 1, wherein the selecting means selects a candidate pixel block to be the tracking target based on the evaluation value for each pixel block calculated by the evaluation value calculating means. . The determining means determines a tracking target pixel block in the next frame from candidate pixel blocks remaining after exclusion by the excluding means based on the evaluation value of the pixel block calculated by the evaluation value calculating means. The subject tracking apparatus according to claim 2 . Imaging means for imaging and sequentially outputting the frame;
An optical system for forming a subject image on the imaging means;
Driving means for driving the optical system;
Control means for controlling the driving means so that the image in the tracking target block determined by the determining means is focused;
The photographic subject tracking device according to any one of claims 1 to 3, further comprising a. A computer having an apparatus for determining a tracking target pixel block in a next frame and tracking a subject to be imaged based on the tracking target pixel block in a current frame that is sequentially imaged,
Selection means for selecting candidate pixel blocks to be tracked in the next frame based on the tracked pixel blocks in the current frame;
When a plurality of pixel blocks are selected as candidate pixel blocks by this selection means, the distance between the selected candidate pixel blocks is the first distance that is the distance between the farthest candidate pixel blocks, A calculating means for calculating a second distance that is a distance between candidate pixel blocks in a close positional relationship;
A first multiplying unit for substituting the first distance for an interblock distance coefficient prepared as a coefficient inversely proportional to the distance and multiplying the evaluation difference absolute value between the farthest candidate pixel blocks;
A second multiplication means for substituting the second distance into the inter-block distance coefficient and multiplying the evaluation difference absolute value between the closest candidate pixel blocks having the positional relationship;
Of the values obtained by the first multiplying means and the values obtained by the second multiplying means, the candidate pixel blocks that have obtained values that are in a range equal to or less than a predetermined value remain, and the other is excluded means,
Determining means for determining a pixel block to be tracked in the next frame based on the candidate pixel block remaining by the exclusion of the exclusion means;
Subject tracking program characterized by functioning as an object. A target tracking method for tracking a target by determining a target pixel block in a next frame based on a target pixel block in a current frame that is sequentially imaged,
A selection step of selecting candidate pixel blocks to be tracked in the next frame based on the tracked pixel blocks in the current frame;
When a plurality of pixel blocks are selected as candidate pixel blocks by this selection step, the distance between the selected candidate pixel blocks is the first distance that is the distance between the farthest candidate pixel blocks, and the most A calculation step of calculating a second distance that is a distance between candidate pixel blocks in a close positional relationship;
A first multiplication step of substituting the first distance into an inter-block distance coefficient prepared as a coefficient inversely proportional to the distance, and multiplying the evaluation difference absolute value between the farthest candidate pixel blocks,
A second multiplication step of substituting the second distance into the inter-block distance coefficient and multiplying the evaluation difference absolute value between the nearest candidate pixel blocks having a positional relationship;
Of the values obtained in the first multiplication step and the values obtained in the second multiplication step, the candidate pixel blocks that have obtained values in a range equal to or smaller than a predetermined value remain, and the other is excluded. An exclusion step to
A determination step of determining a tracking target pixel block in the next frame based on the candidate pixel block remaining after the exclusion in the exclusion step;
The object tracking method characterized by including.

Images