JP2009152725A - Automatic tracing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a specific object and to stably trace the object. <P>SOLUTION: A memory part 25 stores a reference pattern and a matching processing part 21 detects an area image that best matches the reference pattern from a photographed image. An extraction part 22 extracts the area image from the photographed image. When an integrated value of the differential absolute values of luminance values of corresponding pixels between the area image and the reference pattern is a fixed value and larger, a pattern obtained by correcting luminance so that the luminance values of respective pixels of the area image having the upper limit value or larger become the upper limit value of the differential absolute value is stored in the memory part 25 as a new reference pattern and update the reference pattern. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an automatic tracking device and method for an object used for autofocus, a monitoring device, and the like.

  In such a digital camera, many of which are equipped with an autofocus function in a photographing apparatus, for example, a digital camera, an autofocus area (hereinafter referred to as an AF area) is set on the photographing screen. When the subject is overlapped, the focus of the photographic lens is adjusted so that the subject automatically comes into focus.

  Also, there is a digital camera having a function of tracking the target subject so that the subject is always in focus by moving the target subject (hereinafter referred to as the target subject) by following the AF area. Are known. In this way, in order for the AF area to follow the target subject, for example, pattern matching processing is performed using the target subject image as a standard pattern, and an image portion highly correlated with the target subject image is detected from the captured image. .

  By the way, when the default standard pattern is continuously used in the pattern matching process, if a change in the size, orientation, brightness, orientation, etc. of the target subject on the shooting screen occurs, the standard pattern is retained. The difference between the target subject image and the actual target subject image becomes large, and the target subject cannot be detected. Therefore, there is a technique in which the standard pattern is sequentially updated so that the image of the detected area becomes a new standard pattern (see, for example, Patent Document 1). In Patent Document 1, an average value of a standard pattern and a region image detected using the standard pattern or an image obtained by performing a linear combination operation is used as a new standard pattern.

  On the other hand, when the standard pattern is sequentially updated as described above, when another object is reflected in the target subject, or when a shield passes through the front of the target subject, the standard pattern is thereby changed. In some cases, the standard pattern may be replaced with an image of the shielding object. If the standard pattern is replaced with an image of the shielding object, the target subject cannot be tracked. Therefore, in order to solve such a problem, Patent Document 2 proposes a pattern matching process using a plurality of standard patterns obtained at different times.

In Patent Document 2, a standard pattern that performs pattern matching processing using a plurality of standard patterns including a standard pattern used in the past, and obtains the largest matching degree among the maximum matching degrees of the plurality of standard patterns. The position of the area image detected based on the above is set as the position of the target subject, and the area image at that position is added to the new standard pattern.
JP-A-6-30417 JP 2001-60269 A

  As described in Patent Document 2, the method using a plurality of standard patterns has an effect of preventing the target pattern from being difficult to track due to the standard pattern becoming an image of the shielding object due to the passage of the shielding object. However, since a standard pattern composed of an image of a target object that is older in time is used together with the latest standard pattern, the target object cannot be detected with sufficient accuracy, resulting in a problem that tracking accuracy is lowered.

  The present invention has been made to solve the above-described problems, and can accurately detect a target subject even if the target subject to be tracked is blocked or reflected due to passage of a shield or the like. It is an object of the present invention to provide an automatic tracking apparatus and method capable of tracking.

  In order to solve the above problem, in the tracking device according to claim 1 of the present invention, a holding unit that holds an image of a specific subject as a standard pattern, a standard pattern stored in the holding unit from the input image, A matching processing means for detecting a matching image area by pattern matching processing, an extraction means for extracting an image of the area detected by the matching processing means as an area image from the input image, and a standard held in the holding means The determination means for determining the level of correlation between the pattern and the extracted area image, and the difference in luminance value between corresponding pixels of the standard pattern and the area image when the determination means determines that the correlation is low For each pixel of the region image whose value is equal to or greater than a predetermined upper limit value, a correction unit that performs a correction process for reducing the absolute difference value, and a determination unit When the correlation is determined to be low, the area image corrected by the correction means is set as a new standard pattern. When the correlation is determined to be high, the area image extracted by the extraction means is set as a new standard pattern. Update control means for updating the standard pattern held in the holding means.

  In the automatic tracking device according to claim 2, the determination unit obtains a difference absolute value of luminance values between corresponding pixels of the standard pattern and the region image, and a difference integrated value obtained by integrating the difference absolute value of each pixel is a predetermined value. When it is smaller than the threshold value, it is determined that the correlation is high.

  In the automatic tracking device according to claim 3, when the luminance value of the pixel of the region image to be corrected is larger than the luminance value of the pixel of the corresponding standard pattern, Replace with the luminance value obtained by adding the upper limit value to the luminance value of the pixel, and if the luminance value is smaller than the luminance value of the pixel of the corresponding standard pattern, replace it with the luminance value obtained by subtracting the upper limit value from the luminance value of the pixel of the standard pattern. It is a thing.

  In the automatic tracking device according to claim 4, the correction means replaces the luminance of the pixel of the region image to be corrected with an average value of the luminance value of the pixel of the corresponding standard pattern. .

  6. The automatic tracking method according to claim 5, wherein each time an input image is inputted, a matching step for detecting a region of an image that most closely matches a held standard pattern by pattern matching processing, and a region detected with the standard pattern Luminance value of a pixel corresponding to the standard pattern among the pixels of the image in the detected area when the correlation is determined to be low by the determination step A correction step for reducing the difference absolute value for a pixel whose difference absolute value is greater than or equal to a predetermined upper limit value, and a specified region corrected by the correction step when the determination step determines that the correlation is low Update image with a new standard pattern, and when it is determined that the correlation is high, the image of the specified area is the new standard pattern. Tsu is one having a flop.

  According to the present invention, when the standard pattern is sequentially updated with the region image of the region specified by the pattern matching process, if the standard pattern and the detected region image have low correlation, the standard pattern corresponds to the region image. Since each pixel of the region image in which the absolute value of the luminance value between the pixels at the position is greater than or equal to a predetermined value is corrected, the correction is performed to reduce the absolute value of the difference. The luminance change that occurs in the image is limited, and the correlation between the new standard pattern and the target subject is not significantly reduced, so the target subject can be detected accurately even if an obstacle passes or is reflected. Can be tracked.

  FIG. 1 shows a configuration of a digital camera 2 embodying the present invention. The system controller 3 controls the photographing unit 5 and the automatic tracking unit 6 of the digital camera 2 based on various operation signals from the operation unit 4. The operation unit 4 includes a release button for instructing photographing, operation buttons for performing various settings, keys, and the like. By operating the operation unit 4, an AF area to be described later can be initialized.

  The photographing unit 5 is provided with a photographing lens 10, and an image sensor 11 is disposed behind the photographing lens 10. The subject light passes through the photographing lens 10 and enters the light receiving surface of the image sensor 11. The photographing lens 10 includes a focus mechanism 12 that adjusts the focus. The focus mechanism 12 includes a focus lens 12a, a motor that moves the focus lens 12a in the optical axis direction, a motor driver, and the like, and adjusts the focus by moving the focus lens 12a.

  The image sensor 11 captures a subject and outputs a captured image. The image sensor 11 includes a large number of light receiving elements on a light receiving surface thereof, and converts a subject image formed by the photographing lens 10 into an electrical image signal. Output. As the image sensor 11, various types such as a CCD type and a MOS type can be used.

  The signal processing unit 13 performs various signal processes on the image signal from the image sensor 11. The signal processing unit 13 outputs image data obtained by performing signal processing such as signal amplification processing, digital conversion processing, γ correction processing, and white balance correction processing on the image signal from the image sensor 11.

  The image recording unit 14 stores a photographed image of a still image photographed in response to an operation when a photographing instruction is given by operating the release button. For example, the memory card is detachable from the digital camera 2. And a circuit for reading and writing image data.

  The display unit 15 includes an LCD that displays an image, a driver that drives the LCD, and the like. The image data from the signal processing unit 13 is sent to the display unit 15 one after another, and a through image is displayed based on the image data. The photographer can perform framing by observing the through image. The display unit 15 displays an AF frame indicating the AF area so as to overlap the through image. The photographer can perform the initial setting of the AF area by referring to the AF frame displayed on the display unit 15, and can check the AF area that is automatically updated.

  The AF control unit 16 constitutes an AF unit together with the focus mechanism 12. The AF control unit 16 operates the focus mechanism 12 to detect a position where the focus of the photographing lens 10 matches the subject, and sets the focus lens 12a at the matching lens position. The AF control unit 16 controls the focus mechanism 12 so that the subject in the AF area set on the shooting screen is in focus.

  As for the AF area, a range designated by the operation of the operation unit 4 is initially set in the AF control unit 16 via the automatic tracking unit 6, and after the initial setting, the target subject (hereinafter, target) is set by the automatic tracking unit 6. The AF area is automatically updated and set so as to move according to the movement of the subject). That is, the AF area automatically tracks the target subject.

  The AF control unit 16 matches the focus of the photographing lens 10 by a so-called contrast detection method. That is, based on the image data from the signal processing unit 13, an AF evaluation value obtained by integrating the high-frequency components of the images in the AF area is obtained for each captured image, and the focus lens 12a is placed at the lens position where the AF evaluation value is maximized. Move. Note that the autofocus method is not limited to the above-described contrast of the subject image.

  The automatic tracking unit 6 is an automatic tracking device that tracks a target subject, which is a specific subject, using an AF area as a processing region. The input unit 20 acquires image data (captured image) from the signal processing unit 13 as an input image. The acquired image data has a luminance value corresponding to the luminance signal level for each pixel, for example.

  The input unit 20 performs a normalization process that makes the luminance distribution of the captured image uniform to a certain degree. This normalization process may be performed on the entire captured image, but the normalization process may be performed for each area where pattern matching described later is performed. Note that normalization processing may not be performed.

  The matching processing unit 21 performs a known pattern matching (template matching) process using a standard pattern (template image) on the input captured image based on the image data. The matching processing unit 21 detects a region of an image portion (hereinafter referred to as a region image) of p × q pixels that most closely matches, for example, a standard pattern composed of p × q pixels, that is, a correlation value that is highest with respect to the standard pattern -Specify and output area information indicating the area.

  In the pattern matching processing by the matching processing unit 21, it is possible to specify an area that most closely matches the standard pattern on the captured image, and the area information includes information such as the center position, size, and shape of the area. It is. For example, in the case of using normalized correlation pattern matching processing, the correlation value is M (standard pattern luminance value), the luminance value of the region image in the captured image is I, and the number of each pixel is N (= p × q ), The correlation value (correlation coefficient) r is obtained by the following equation (1).

  The extraction unit 22 extracts an image in the region indicated by the region information, that is, a region image from the captured image, and outputs the image to the buffer memory 24 under the control of the update control unit 23 described later. The memory unit 25 holds an image of a tracking target object as a standard pattern, acquires a region image from the buffer memory 24, and stores this as a standard pattern.

  In the initial setting of automatic tracking, the area surrounded by the AF frame designated by the operation of the operation unit 4 is set as the AF area. The extraction unit 22 extracts an image in the AF area as a region image, and this is stored as a standard pattern in the memory unit 25 via the buffer memory 24. After the initial setting, the extraction unit 22 extracts an image of the region specified by the update control unit 23 as a region image, and this is written in the buffer memory 24 as a new standard pattern. In response to the input of the update signal from the update control unit 23, the memory unit 25 updates the standard pattern by acquiring the area image as the standard pattern from the buffer memory 24.

  The update control unit 23 controls the update of the standard pattern in the memory unit 25, and determines whether pattern matching is possible based on the level of the correlation value, thereby controlling the update of the AF area. The update control unit 23 receives the region information and the correlation value from the matching processing unit 21. Further, the update control unit 23 is set with a first threshold value for determining whether or not the AF area should be updated in comparison with the correlation value.

  The update control unit 23 controls the AF control unit 16 to set the region indicated by the region information from the matching processing unit 21 as a new AF area when the correlation value is equal to or greater than the first threshold value, and An AF frame indicating the area is displayed on the display unit 15. On the other hand, if the correlation value is smaller than the first threshold and an area where sufficient correlation is obtained cannot be detected, the operator is requested to reset the AF area.

  The correction processing unit 26 determines whether the correlation between the standard pattern and the region image is high or low, and performs correction processing for correcting the region image when the correlation is low. The determination of the level of correlation is performed by integrating the absolute value of the difference between the luminance values of corresponding pixels of the standard pattern in the memory unit 25 and the corresponding region image in the buffer memory 24 extracted as a match with the standard pattern in the memory unit 25. It is determined whether or not the integrated difference value is equal to or greater than a preset second threshold value. When it is smaller than the second threshold, the correlation is high, and when it is equal to or higher than the second threshold, the correlation is low. Note that, in comparison with the determination of high or low correlation using the first threshold value, the determination of high or low correlation using the second threshold value has a higher criterion for determining that the correlation is high. .

  In the correction processing unit 26, an upper limit value for the absolute difference value is set in advance, and each of the region images in which the absolute value difference between the luminance values of corresponding pixels of the standard pattern and the extracted region image is equal to or greater than the upper limit value. A correction process is performed on the luminance of the pixel so that the absolute value of the difference becomes small. In this example, the absolute value of the difference between the luminance of each pixel of the area image that is equal to or higher than the upper limit value and the pixel of the standard pattern corresponding to the luminance is set as the upper limit value.

Therefore, when the luminance value of a pixel at an arbitrary position in the standard pattern is A _i and the luminance value of a pixel at a position in the area image corresponding to the pixel A _i is B _iold , the difference integrated value G is It calculates | requires by Formula (2).
G = Σ | A _i −B _iold | (2)

Further, when the upper limit value of the absolute difference between each pixel of the standard pattern and the region image is L, the luminance value _Binew of the pixel of the region image after the correction process is _obtained by the following equations (3) to (5). It is done.
When | A _i −B _iold | ≦ L, B _new = B _i (3)
| A _i −B _iold |> L and A _i <B _iold
B _new = A _i + L (4)
| A _i -B _iold |> L and A _i > B _iold
B _new = A _i −L (5)

  When correction processing is performed by the correction processing unit 26, the update control unit 23 sends an update signal to the memory unit 25 after completion of the correction processing, and when correction processing is not performed, an area image is extracted. An update signal is sent immediately after writing to the buffer memory 24. Thus, when it is determined that the correlation between the area image and the standard pattern is low, the corrected area image is used as a new standard pattern, and when it is determined that the correlation is high, the extracted area image is used as it is. Update as a new standard pattern.

  In this example, when determining whether the correlation between the standard pattern and the region image is high or low, it is based on the magnitude of the difference integrated value obtained by integrating the difference absolute value, but the determination method is not limited to this. Absent. For example, a correlation value calculated when the matching processing unit 21 performs pattern matching may be used. In this case, for example, the level of the correlation may be determined based on the magnitude of the reciprocal of the correlation value r obtained by the above equation (1).

  As shown in FIG. 2, in the photographed image 30, an AF area 31a is designated, for example, in a rectangular shape by the operation of the operation unit 4, or an area specified by pattern matching processing is set as the AF area 31a. Then, the area image in the AF area 31a is stored and held in the memory unit 25 as a standard pattern. The AF frame 32 is designated as the outer periphery of the AF area 31a.

  When a new captured image is input, an image region having the highest correlation value is detected by pattern matching processing using a standard pattern stored and held in the memory unit 25, and the correlation value is lower than the first threshold value. When the area is large, the area is set as a new AF area 31b. By repeating such processing, when the target subject moves on the captured image, the target subject is tracked so that the AF area 31b moves to the destination.

  Next, the operation of the above configuration will be described. When the digital camera 2 is set to the shooting mode, shooting by the shooting unit 5 is started and a through image is displayed on the display unit 15. For example, in this initial state, the AF area is set at the center of the shooting screen, and the focus mechanism 12 is driven under the control of the AF control unit 16 so that the subject in the AF area is in focus. Further, every time one screen is shot for displaying a through image, image data is sent from the shooting unit 5 to the automatic tracking unit 6, and the image data is sent to the matching processing unit 21 via the input unit 20. Input to the extraction unit 22.

  When making the target subject follow the AF area, as shown in FIG. 3, the photographer operates the operation unit 4 while referring to the AF frame displayed on the display unit 15, to The size and position of the AF frame are set so as to surround the face, and the tracking of the target subject is instructed.

  When the tracking of the target subject is instructed, the system controller 3 gives region information corresponding to the set size and position of the AF frame to the extraction unit 22. As a result, an image in the region indicated by the region information is extracted as a region image, and the region image is written to the memory unit 25 via the buffer memory 24 as a standard pattern.

  After the standard pattern is held in the memory unit 25 as described above, when image data for one new screen is input to the automatic tracking unit 6, the image data is input to the matching processing unit 21 via the input unit 20. And is extracted by the extraction unit 22. Then, the matching processing unit 21 reads a standard pattern from the memory unit 25, and pattern matching processing is performed on a new captured image using the standard pattern. In this pattern matching process, a region of the image having the highest correlation value is detected, and region information indicating the region and the correlation value are output to the update control unit 23.

  The update control unit 23 compares the correlation value with the first threshold value. In this comparison, for example, when the correlation value is greater than or equal to the first threshold value, the AF control unit 16 changes the area indicated by the input area information to be a new AF area in order to update the AF area. The setting of the display unit 15 is changed so that an AF frame indicating the area is displayed.

  If the correlation value is smaller than the first threshold value, it is assumed that there is no area image that matches the set target subject, and the AF area is not updated. For example, the operator is prompted to reset the AF area. The guidance is displayed on the display unit 15.

  When the correlation value is equal to or greater than the first threshold as described above, the extraction unit 22 is operated to extract the image of the region indicated by the region information from the update control unit 23, and the extracted image That is, the area image is written in the buffer memory 24.

  When the area image is written in the buffer memory 24, the correction processing unit 26 reads the luminance value of the pixel of the standard pattern written in the memory unit 25 and reads the luminance value of the pixel of the area image in the buffer memory 24. A difference absolute value of luminance values between the corresponding pixels is obtained. Then, by integrating the absolute difference values of these pixels, an integrated difference value is calculated and compared with the second threshold value.

  In the above comparison, if the difference integrated value is smaller than the second threshold value, an update instruction is immediately issued to the memory unit 25 via the update control unit 23, and in response to this instruction, the memory unit 25 Acquires and stores the area image in the buffer memory 24 as a new standard pattern.

  On the other hand, when the difference integrated value is greater than or equal to the second threshold value, correction processing by the correction processing unit 26 is performed. As a result of this correction processing, for each pixel of the area image whose absolute difference value exceeds the upper limit value, for the pixel whose luminance value is greater than the luminance value of the corresponding standard pattern pixel, the upper limit is set to the luminance value of the standard pattern pixel. It is replaced with the luminance value obtained by adding the values. In addition, a pixel whose luminance value is smaller than the luminance value of the pixel of the corresponding standard pattern is replaced with a luminance value obtained by subtracting the upper limit value from the luminance value of the pixel of the standard pattern.

  After the area image is corrected as described above, an update instruction is issued to the memory unit 25. In response to this instruction, the memory unit 25 sets the corrected area image as a new standard pattern. Obtained from the buffer memory 24 and stored.

  After the standard pattern is updated, when image data of the next photographed image is input to the automatic tracking unit 6, pattern matching processing is performed using the updated standard pattern as described above. Thereafter, if the correlation value between the area image and the standard pattern is equal to or greater than the first threshold value, the AF area is updated and the standard pattern is updated by the same procedure. When the standard pattern is updated, if the difference integrated value of the absolute difference value between corresponding pixels of the standard pattern and the newly extracted region image is smaller than the second threshold value, the extracted region image Is a new standard pattern, and if the difference integrated value is equal to or greater than the second threshold value, the area image that has been subjected to the correction process is a new standard pattern.

  By updating the standard pattern as described above, for example, even when the shielding object passes through the front surface of the target subject, the luminance change that occurs in the area image due to the passage of the shielding object reaches the upper limit value. Limited in scope. For this reason, the correlation between the new standard pattern to be updated and the target subject, even when the brightness of the target subject changes or the direction changes, as well as when the shielding object passes, etc. Can be detected and tracked with high accuracy.

  In the above embodiment, as a method of reducing the difference absolute value when the difference absolute value exceeds the upper limit value, the luminance value of the region image is corrected so that the difference absolute value becomes the upper limit value. Absent. For example, pixels of a region image whose difference absolute value exceeds the upper limit value may be replaced with the average value of the luminance values of the pixels of the region image and the corresponding standard pattern pixels.

  In the above embodiment, the target subject to be tracked is specified by the operation of the operation unit. However, a specific target, for example, a human face is automatically recognized by the digital camera, and the face is set as the target subject. Also good. Further, although an example of a digital camera has been described, the present invention is not limited to this, and the present invention is not limited to this. Applicable.

  Further, not only the AF area but also an AE area for measuring subject brightness for performing exposure control, an AWB area for performing color measurement for adjusting white balance, and the like can be used to move following the target subject. . Furthermore, the present invention can also be used when an image shot and recorded by a video camera is used as an input image. The present invention may be applied to a surveillance camera, and the shooting direction may be changed to follow the movement of the subject.

It is a block diagram which shows the structure of the digital camera which implemented this invention. It is explanatory drawing explaining the movement of AF area. It is a flowchart which shows the process sequence for tracking an object subject.

Explanation of symbols

2 Digital Camera 4 Operation Unit 5 Imaging Unit 6 Automatic Tracking Unit 21 Matching Processing Unit 22 Extraction Unit 23 Update Control Unit 25 Memory Unit 26 Correction Processing Unit

Claims (5)

In an automatic tracking device that detects and tracks a specific subject on input images that are sequentially input,
Holding means for holding the image of the specific subject as a standard pattern;
Matching processing means for detecting, by pattern matching processing, an image area that most closely matches the standard pattern stored in the holding means from the input image;
Extraction means for extracting an image of the area detected by the matching processing means as an area image from the input image;
Determination means for determining the level of correlation between the standard pattern held in the holding means and the extracted region image;
When it is determined that the correlation is low by the determination unit, the difference between the pixels of the region image in which the absolute value difference between the corresponding pixels of the standard pattern and the region image is equal to or greater than a predetermined upper limit value Correction means for performing correction processing to reduce the absolute value;
When the determination unit determines that the correlation is low, the region image corrected by the correction unit is used as a new standard pattern. When the correlation unit determines that the correlation is high, the region image extracted by the extraction unit is used. An automatic tracking apparatus comprising: an update control means for updating a standard pattern held in the holding means as a new standard pattern.   The determination means obtains a difference absolute value of luminance values between corresponding pixels of the standard pattern and the region image, respectively, and when a difference integrated value obtained by integrating the difference absolute value of each pixel is smaller than a predetermined threshold value, The automatic tracking device according to claim 1, wherein it is determined that the correlation is high.   When the luminance value of the pixel of the region image to be corrected is larger than the luminance value of the corresponding standard pattern pixel, the correction means adds the upper limit value to the luminance value of the pixel of the standard pattern. When the luminance value is smaller than the luminance value of the pixel of the corresponding standard pattern, the luminance value is obtained by subtracting the upper limit value from the luminance value of the pixel of the standard pattern. The automatic tracking device according to 1 or 2.   3. The automatic tracking according to claim 1, wherein the correction unit replaces the luminance of the pixel of the region image to be corrected with an average value of the luminance value of the pixel of the corresponding standard pattern. apparatus. In an automatic tracking method for detecting and tracking a specific subject on input images that are sequentially input,
Each time an input image is input, a matching step for detecting a region of an image that most closely matches a held standard pattern by pattern matching processing,
A determination step of determining the level of correlation between the standard pattern and the image of the detected area;
When the correlation is determined to be low by the determination step, the absolute value of the difference in luminance value from the corresponding pixel of the standard pattern among the pixels of the image of the detected region is equal to or greater than a predetermined upper limit value A correction step for reducing the absolute value of the difference with respect to a pixel;
When it is determined that the correlation is low by the determination step, the image of the specified region corrected by the correction step is used as a new standard pattern, and the image is specified when the correlation is determined to be high. And an update step of using the image of the region as a new standard pattern.

Images